You are on page 1of 35

Gauss-Siedel Method

Major: All Engineering Majors Authors: Autar Kaw (Modified by P. Goel for IDC103)

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM Undergraduates
http://numericalmethods.eng.usf.edu

3/25/2010

Gauss-Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
An iterative method. Basic Procedure :
-Algebraically solve each linear equation for xi -Assume an initial guess solution array -Solve for each xi and repeat -Use absolute relative approximate error after each iteration to check if error is within a pre-specified tolerance.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Why?
Obviously it isn t possible to obtain better than machine precision for a solution of SLEs (using Gaussian Elimination and LU Decomposition). Actually, the situation is worse for large systems: it isn t possible to get close to machine precision in direct methods.

The Gauss-Seidel Method allows the user to control round-off error. In fact, iterative methods can be used to improve the solution obtained by direct methods.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:

a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1


a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2
. . . . . .

If: the diagonal elements are


non-zero

Rewrite each equation solving


for the corresponding unknown

an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn

ex:
First equation, solve for x1 Second equation, solve for x2

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
Rewriting each equation

x1 =

c1 - a12 x 2 - a13 x3 KK - a1n x n a11


c2 - a21 x1 - a23 x3 KK - a2 n xn a22 M M M cn -1 - an -1,1 x1 - an -1, 2 x2 KK - an -1,n -2 xn -2 - an -1, n xn an -1,n -1 ann

From Equation 1

x2 =

From equation 2

xn -1 = xn =

From equation n-1

cn - an1 x1 - an 2 x2 - KK - an , n -1 xn -1

From equation n

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form of each equation

c1 - a1 j x j x1 =
j =1 j 1

cn -1 xn -1 =

j =1 j n -1

a
n

n -1, j

xj

a11

an -1,n -1

c2 - a2 j x j x2 =
j =1 j2

c n - a nj x j xn =
j =1 jn

a 22

a nn
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form for any row i

ci - aij x j xi =
j =1 j i

aii

, i = 1,2,K , n.

How or where can this equation be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equati ons to solve for each value of x i. Important: Remember to use the most recent value of xi. Which means to apply values calculated to the calculations remaining in the current iteration.

x1 x 2 M xn -1 xn

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error

a i =

new i

-x
new i

old i

100

So when has the answer been f ound?


The iterations are stopped when the absolute relative approximate error is less than a prespecified tolerance for all unknowns.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


The system of equati ons

25 5 1 a1 106 .8 64 8 1 a = 177 .2 2 144 12 1 a3 279 .2

Initial Guess: Assume an initial guess of

a1 1 a = 2 2 a 3 5

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Rewriting each equation

106.8 - 5a 2 - a 3 a1 = 25
177.2 - 64a1 - a 3 a2 = 8
279.2 - 144a1 - 12a 2 a3 = 1

25 5 1 a1 106 .8 64 8 1 a = 177 .2 2 144 12 1 a3 279 .2

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Applying the initial guess and solving for ai
a1 1 a = 2 2 a 3 5
Initial Guess

a1 =

106.8 - 5(2) - (5) = 3.6720 25

177.2 - 64(3.6720 ) - (5) a2 = = -7.8510 8


a3 = 279.2 - 144(3.6720 ) - 12(- 7.8510 ) = -155.36 1

When solving for a2, how many of the initial guess values were used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Finding the absolute relative approximate error

a i

xinew - xiold = 100 new xi


3.6720 - 1.0000 x100 = 72.76% 3.6720

At the end of the first iteration

a 1 =

a1 3.6720 a = - 7.8510 2 a 3 - 155 .36


The maximum absolute relative approximate error is 125.47%

- 7.8510 - 2.0000 = x100 = 125.47% - 7.8510

a 3 =

- 155.36 - 5.0000 x100 = 103.22% - 155.36


http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Using

Iteration #2
a1 = 106.8 - 5(- 7.8510 ) - 155.36 = 12.056 25
177.2 - 64(12.056 ) - 155.36 = -54.882 8
the values of ai are found:

a1 3.6720 a = - 7.8510 2 a 3 - 155.36


from iteration #1

a2 =

a3 =

279.2 - 144(12.056 ) - 12(- 54.882 ) = -798.34 1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Finding the absolute relative approximate error 12.056 - 3.6720 At the end of the second iteration a 1 = x100 = 69.543% 12.056 a 12.056
a 2 = - 54.882 - (- 7.8510) x100 = 85.695% - 54.882

a = - 54.882 2 a3 - 798.54

a 3 =

- 798.34 - (- 155.36 ) x100 = 80.540% - 798.34

The maximum absolute relative approximate error is 85.695%

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1


Repeating more iterations, the following values are obtained
Iteration

a1 3.6720 12.056 47.182 193.33 800.53 3322.6

a 1 %
72.767 69.543 74.447 75.595 75.850 75.906

a2 7.8510 54.882 255.51 1093.4 4577.2 19049

a 2 %
125.47 85.695 78.521 76.632 76.112 75.972

a3 155.36 798.34 3448.9 14440 60072 249580

a 3 %
103.22 80.540 76.852 76.116 75.963 75.931

1 2 3 4 5 6 Notice

The relative errors are not dec reasing at any significant rate

a 1 0.29048 Also, the solution is not converging to the true s olution of a 2 = 19.690 a 3 1.0857
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall


What went wrong?
Even though done correctly, the answer is not converging to the correct answer This example illustrates a pitfall of the Gaus s-Siedel method: not all systems of equations will converge.

Is there a fix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall


GAUSS-SEIDEL CONVERGENCE THEOREM: If A is diagonally dominant, then the Gauss-Seidel method converges for any starting vector x. A sufficient, but not necessary condition.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

aii aij
j =1 j i

for all i

and

aii > aij


j =1 j i

for at least one i

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall


Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other c oefficients in that row and at least one row with a diagonal coefficient greater than the sum of the other coefficients in that row. Which coefficient matrix is diagonally dominant?
2 5.81 34 [A ] = 45 43 1 123 16 1
124 34 56 [ B] = 23 53 5 96 34 129

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


Given the system of equations The coefficient matrix is:

12 x1 + 3 x2- 5 x3 = 1

x1 + 5 x2 + 3x3 = 28
3x1 + 7 x2 + 13x3 = 76
With an initial guess of

12 3 - 5 [A] = 1 5 3 3 7 13
Will the solution converge using the Gauss-Siedel method?

x1 1 x = 0 2 x3 1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


Checking if the coefficient matrix is diagonally dominant
12 3 - 5 [A] = 1 5 3 3 7 13

a11 = 12 = 12 a12 + a13 = 3 + - 5 = 8

a 22 = 5 = 5 a 21 + a 23 = 1 + 3 = 4
a33 = 13 = 13 a31 + a32 = 3 + 7 = 10

The inequalities are all true and at least one row is strictly greater than: Therefore: The solution should converge using the Gauss-Siedel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


Rewriting each equation
12 3 - 5 a1 1 1 5 3 a = 28 2 3 7 13 a3 76

With an initial guess of


x1 1 x = 0 2 x 3 1
x1 = 1 - 3(0) + 5(1) = 0.50000 12

1 - 3x 2 + 5 x3 x1 = 12 28 - x1 - 3x3 x2 = 5

28 - (0.5) - 3(1) x2 = = 4.9000 5


76 - 3(0.50000 ) - 7(4.9000 ) x3 = = 3.0923 13
http://numericalmethods.eng.usf.edu

76 - 3x1 - 7 x 2 x3 = 13

Gauss-Seidel Method: Example 2


The absolute relative approximate error

0.50000 - 1.0000 a 1 = 100 = 100.00% 0.50000

a
a

4.9000 - 0 = 100 = 100.00% 4.9000


3.0923 - 1.0000 = 100 = 67.662% 3.0923

The maximum absolute relative error after the first iteration is 100%
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


After Iteration #1
x1 0.5000 x = 4.9000 2 x3 3.0923

Substituting the x values into the equations


x1 =

After Iteration #2
x1 0.14679 x = 3.7153 2 x3 3.8118

1 - 3(4.9000 ) + 5(3.0923) = 0.14679 12

28 - (0.14679 ) - 3(3.0923) x2 = = 3.7153 5

x3 =

76 - 3(0.14679 ) - 7(4.900 ) = 3.8118 13


http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


Iteration #2 absolute relative approximate error
a 1 = 0.14679 - 0.50000 100 = 240.61% 0.14679

3.7153 - 4.9000 = 100 = 31.889% 3.7153

3.8118 - 3.0923 = 100 = 18.874% 3.8118

The maximum absolute relative error after the fi rst iteration is 240.61%

This is much larger than the maxi mum absolute relative error obtai ned in iteration #1. Is this a problem?
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2


Repeating more iterations, the following values are obtained
Iteration 1 2 3 4 5 6 a1 0.50000 0.14679 0.74275 0.94675 0.99177 0.99919

a 1 %
100.00 240.61 80.236 21.546 4.5391 0.74307

a2 4.9000 3.7153 3.1644 3.0281 3.0034 3.0001

a 2 %
100.00 31.889 17.408 4.4996 0.82499 0.10856

a3 3.0923 3.8118 3.9708 3.9971 4.0001 4.0001

a 3 %
67.662 18.876 4.0042 0.65772 0.074383 0.00101

x1 0.99919 The solution obtained x = 3.0001 is close to the exact solution of 2 x3 4.0001

x1 1 x = 3 . 2 x 3 4

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 3


Given the system of equations

3 x1 + 7 x2 + 13 x3 = 76

Rewriting the equations

x1 + 5 x2 + 3x3 = 28
12 x1 + 3x2 - 5 x3 = 1
With an initial guess of

76 - 7 x2 - 13 x3 x1 = 3

x1 1 x = 0 2 x3 1

28 - x1 - 3x3 x2 = 5

1 - 12 x1 - 3 x 2 x3 = -5
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 3


Conducting six iterations, the following values are obtained
Iteration a1

a 1 %

A2

a 2 %
100.00 94.453 112.43 109.63 109.92 109.89

a3

a 3 %
98.027 110.96 109.80 109.90 109.89 109.89

1 2 3 4 5 6

21.000 196.15 1995.0 20149 2.0364105 2.0579105

95.238 0.80000 110.71 14.421 109.83 116.02 109.90 1204.6 109.89 12140 109.89 1.2272105

50.680 462.30 4718.1 47636 4.8144105 4.8653106

The values are not converging. Does this mean that the Gauss-Seidel method cannot be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not diagonally dominant But this is the same set of equations used in example #2, which did converge.

3 7 13 [A] = 1 5 3 12 3 - 5
12 3 - 5 [ A] = 1 5 3 3 7 13

If a system of linear equations is not diagonally dominant, check to see if rearranging the equati ons can form a di agonally dominant matrix.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Not every system of equations can be rearranged to have a diagonally dominant coefficient matrix.
Observe the set of equati ons

x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9

x1 + 7 x 2 + x3 = 9
Which equation(s) prevents this set of equation from having a diagonally dominant coefficient matrix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method -Algorithm for the Gauss-Seidel Method -Pitfalls of the Gauss -Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method

Questions?

http://numericalmethods.eng.usf.edu

Additional Resources
For all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit http://numericalmethods.eng.usf.edu/topics/gauss_seid el.html

THE END
http://numericalmethods.eng.usf.edu