CME302

COMPUTATIONAL METHODS IN
ENGINEERING
Part 3 – Linear Algebraic Equation

Dr.Namık KILIÇ e-mail: kilicna@mef.edu.tr 18th March2024

Part3 – Linear Algebraic Eqns.
Chap.11 – Special Matrices and
Gauss Siedel
Chap.12 – Case Studies
 Special Matrices and Gauss-Seidel
Certain special matrices have a particular structure that can be
exploited to develop efficient solution schemes.
The first part of this chapter is devoted to two such systems:
▪ banded and
▪ symmetric matrices.

The second part of the chapter turns to an alternative to elimination

methods, that is, approximate iterative methods.
The focus is on the Gauss-Seidel method, which employs initial
guesses and then iterates to obtain refined estimates of the solution.
 Banded Matrices
A banded matrix is a square matrix that has all elements equal to zero,
with the exception of a band centered on the main diagonal. These
matrices typically occur in solution of differential equations.
The dimensions of a banded system can be quantified by two
parameters: the band width BW and half-bandwidth HBW. These two
values are related by BW = 2HBW + 1.
Methods like Gauss elimination are often inefficient in solving
banded matrices for cases where pivoting is unnecessary because
none of the elements outside the band would change from their
original values of zero.
Thus, unnecessary space and time would be expended on the storage
and manipulation of these useless zeros.
 Banded Matrices
Parameters used to quantify the
dimensions of a banded system.
BW and HBW designate the
bandwidth and the half-bandwidth,
respectively.
 Tridiagonal Systems
A tridiagonal
system—that is,
one with a
bandwidth of 3

Notice that we have changed our notation for the coefficients from a’s and b’s
to e’s, f’s, g’s, and r’s. This was done to avoid storing large numbers of useless
zeros in the square matrix of a’s.
This space-saving modification is advantageous because the resulting
algorithm requires less computer memory
Where do Tridiagonal Systems Come From in Engineering? (Beams)

d 2 y wLx wx 2
EI 2 = −
dx 2 2
Where do Tridiagonal Systems Come From in Engineering? (Beams)

d 2 y wLx wx 2
EI 2 = −
dx 2 2

d 2 y yi −1 − 2 yi + yi +1
=
dx 2
x 2
yi −1 − 2 yi + yi +1 wLxi wxi2
EI = −
x 2
2 2

x 2  wLxi wxi2 
yi −1 − 2 yi + yi +1 =  − 
EI  2 2 
Where do Tridiagonal Systems Come From? (Heat Transfer)

d 2T
2
+ h (Ta − T ) = 0
dx
 The Thomas LU Algorithm
(LU Decomposition Version)

The Thomas Algorithm, which can be expressed as an L

U decomposition method, can be used to efficiently
solve tridiagonal systems.
The algorithm consists of three steps:
▪ decomposition,
▪ forward and
▪ back substitution,
and has all the advantages of L U decomposition.
 The Thomas LU Algorithm - example

Solve the following tridiagonal

system with the Thomas algorithm.

First, the decomposition is implemented as

Matrix has been

transformed to
 The Thomas LU Algorithm - example
and the LU decomposition is

The forward substitution is implemented as

r2 = 0.8 − (−0.49)40.8 = 20.8
r3 = 0.8 − (−0.645)20.8 = 14.221
r4 = 200.8 − (−0.717)14.221 = 210.996 Thus, the right-
hand-side vector has been modified to

back substitution and obtain the solution

T4 = 210.996∕1.323 = 159.480
T3 = [14.221 − (−1)159.48]∕1.395 = 124.538
T2 = [20.800 − (−1)124.538]∕1.550 = 93.778
T1 = [40.800 − (−1)93.778]∕2.040 = 65.970
Flops
▪ GAUSS ELIMINATION: ▪ THOMAS ALGORITHM:

+ O ( n2 )
2n 3
8n – 7
3
 Cholesky Decomposition

Recall that a symmetric matrix is one where aij = aji for all i and j.
In other words, [A] = [A]T

This symmetry offer computational advantages because only half the

storage is needed and, in most cases, only half the computation time
is required for their solution
a symmetric matrix can be decomposed [A] = [L][L]T

A positive definite matrix is one for

which the product {X}T [A]{X} is greater
than zero for all nonzero vectors {X}
 Cholesky Decomposition - example
Apply Cholesky decomposition to the
symmetric matrix

l11 = √a11 = √6 = 2.4495

For the second row (k = 2), l21 = a21 / l11 = 15 / 2.4495 = 6.1237
l22 = √a22 − l 2 21 = √55 − (6.1237) 2 = 4.1833
For the third row (k = 3), (i = 1)
l31 = a31 / l11 = 55 / 2.4495 = 22.454
and (i = 2) l32 = (a32 − l21l31 ) / l22 = 225 − 6.1237(22.454) / 4.1833 = 20.917

the Cholesky decomposition yields

 Gauss-Seidel
Iterative or approximate methods provide an alternative to the
elimination methods.
The Gauss-Seidel method is the most commonly used iterative
method.
The system [A]{x}={b} is reshaped by solving the first equation for
x1, the second equation for x2, and the third for x3, …and nth equation
for xn.
For conciseness, we will limit ourselves to a 3×3 set of equations.
 Gauss-Seidel
b1 − a12 x2 − a13 x3
x1 =
a11
b2 − a21 x1 − a23 x3
x2 =
a22
b3 − a31 x1 − a32 x2
x3 =
a33
Now we can start the solution process by choosing guesses for the x’s.
A simple way to obtain initial guesses is to assume that they are zero.
Gauss-Seidel

a11 x1 + a12 x2 + a13 x3 = b1

0 0
b1 − a12 x2 + a13 x3
x1 =
a11

b1
x1 =
a11
Gauss-Seidel
 Gauss-Seidel
New x1 is substituted to calculate x2 and x3. The procedure is repeated until
the convergence criterion is satisfied:

xij − xij −1
 a ,i = j
100%   s
xi

For all i, where j and j − 1 are the present and previous

iterations.
 Gauss-Seidel - example
Use the Gauss-Seidel method to obtain the
solution

First, solve each of the equations for its

unknown on the diagonal
 Gauss-Seidel - Jacobi
As each new x value is computed Gauss-Seidel Jacobi
in the Gauss-Seidel method, it is
immediately used in the next
equation to determine another x
value.
Thus, if the solution is
converging, the best available
estimates will be employed.
An alternative approach, called
Jacobi iteration, utilizes a
somewhat different tactic. Rather
than using the latest available
x’s, this technique uses previous
estimates to compute a set of
new x’s on the basis of a set of
old x’s.
 Convergence Criterion for Gauss-Seidel Method
The Gauss-Seidel method has two fundamental problems as with any
iterative method:
• It is sometimes nonconvergent, and
• If it converges, can converge very slowly.
Recall Fixed-point Iteration
xi +1 = g ( xi )

Ei +1 = g’( ) Ei
g
1
x
Gauss-seidel as a Fixed Point Iteration
b1 − a12 x2
x1 =
a11
b2 − a21 x1
x2 =
a22

x1 = g1 ( x1 , x2 )

x2 = g 2 ( x1 , x2 )
Convergence Criteria

g1 g 2 g1 g 2
+ 1 + 1
x1 x1 x2 x2

g1 g1 a12

=0 =
x1 x2 a11

g 2 a21 g 2
= =0
x1 a22 x2

a21 a12
1 1
a22 a11
Diagonal Dominance

a21 a12
1 1
a22 a11

a22  a21
a11  a12

The absolute value of the diagonal coefficient in each of the

equations must be larger than the sum of the absolute values of
the other coefficients in the equation.
Diagonal Dominance
g 2 ( x1 , x2 ) : 11x1 + 13x2 = 286 g1 ( x1 , x2 ) : 11x1 − 9 x2 = 99

g1 ( x1 , x2 ) : 11x1 − 9 x2 = 99 g 2 ( x1 , x2 ) : 11x1 − 13x2 = 286

How to Deal with it
Many physically-based models are naturally diagonally dominant.
If not, rearrange equations.

But some cannot be rearranged!

Relaxation
new
x
i = x new
i + (1 −  ) x old
i

Under relaxation (  1).

 = 0.5 :

x new
i = 0.5 x new
i + 0.5 x old
i
Over relaxation (  1).
 = 1.5 :
x new
i = 1.5 x new
i + 0.5 x
old
i
 LAPACK MATLAB COMMANDS

Linear algebra
functions in
MATLAB® provide
fast, numerically
robust matrix
calculations.
Capabilities
include a variety of
matrix
factorizations,
linear equation
solving,
computation of
eigenvalues or
singular values,
and more.
LAPACK MATLAB COMMANDS
LAPACK MATLAB COMMANDS
LAPACK MATLAB COMMANDS
LAPACK MATLAB COMMANDS
 MATRIX ALGEBRA COMMANDS
MATLAB Functions
x=A\b returns the solution x of Ax = b, obtained by Gauss elimination. If the
equations are overdetermined (A has more rows than columns), the leastsquares
solution is computed.
[L,U] = lu(A) Doolittle’s decomposition A = LU. On return, U is an upper triangular
matrix and L contains a row-wise permutation of the lower triangular matrix.
[M,U,P] = lu(A) returns the same U as above, but now M is a lower triangular matrix
and P is the permutation matrix so that M = P*L. Note that here P*A = M*U.
L = chol(A) Choleski’s decomposition A = LLT .
B = inv(A) returns B as the inverse of A (the method used is not specified).
n = norm(A)returns the largest singular value of matrix A (singular value
decomposition of matrices is not covered in this text).
n = norm(A,inf) returns the infinity norm of matrix A (largest sum of elements in a row
of A).
c = cond(A) returns the condition number of the matrix A (probably based on the
largest singular value of A)
 GAUSS ELIMINATION

The function mygauss

combines the
elimination and the
back substitution
phases.

During back
substitution, b is
overwritten by the
solution vector x, so
that b contains the
solution upon exit
 LU DECOMPOSITION CODE

In this version of
LU decomposition
the original A is
destroyed and
replaced by its
decomposed form
[L\U].
 LU DECOMPOSITION CODE
This function carries
out the solution
phase (forward and
back substitutions).
It is assumed that
the original
coefficient matrix
has been
decomposed, so
that the input is A =
[L\U].
The contents of b
are replaced by y
during forward
substitution.
Similarly, back
substitution
overwrites y with the
solution x.
 CHOLESKY DECOMPOSITION
At the conclusion of
decomposition, L is
extracted with the
MATLAB command
tril(A).
If a negative L2 j j is
encountered during
decomposition, an error
message is printed and
the program is
terminated.
 CHOLESKY DECOMPOSITION

After the coefficient matrix A has been decomposed, the solution of Ax = b can be obtained by
the usual forward and back substitution operations. The following function (given without
derivation) carries out the solution phase.
 GAUSS SIEDEL
The equations Ax = b are in
scalar notation
 GAUSS SIEDEL

where the weight ω is

called the relaxation
factor.
 GAUSS SIEDEL
The function
GaussSeidel is an
implementation of
the Gauss–Seidel
method with
relaxation.

It automatically
computes ωopt from
former equation
using k = 10 and p =
1.
The user must
provide the function
func that computes
the improved x from
the iterative formulas
 Case Studies: Linear Algebraic Equations
Civil/Environmental Engineering: The matrix inverse is used to
determine the complex cause-effect interactions between forces in
the members of a truss.
Mechanical/Aerospace Engineering: Linear equations are employed
to determine the steady-state configuration of a mass-spring system.
ANALYSIS OF A STATICALLY DETERMINATE TRUSS
Analysis of a Statically Determinate Truss
Analysis of a Statically Determinate Truss
Force Balance in matrix form:

 0.866 0 −0.5 0 0 0   F1   0 
 0.5 0 0.866 0 0 0   F2  −1000 
    
 −0.866 −1 0 −1 0 0   F3   0 
  H  =  
 −0.5 0 0 0 −1 0  2  0 
 0 1 0.5 0 0 0   V2   0 
    
 0 0 −0.866 0 0 −1  V3   0 

Solution:

F1 = −500 F2 = 433 F3 = −866

H2 = 0 V2 = 250 V3 = 750
 SPRING-MASS SYSTEMS

A system composed of three

masses suspended vertically
by a series of springs. The
positions of the masses are
referenced to local
coordinates with origins at
their position before release.

(a) Before release. (b) After release.

FREE-BODY DIAGRAMS
FORCE BALANCES
Force balances based on Newton’s Second Law:
ma = F
d 2x
m 2 = FD − FU
dt
For the three blocks:
d 2 x1
m1 2 = 2k ( x2 − x1 ) − kx1 + m1 g
dt
d 2 x2
m2 2 = k ( x3 − x2 ) − 2k ( x2 − x1 ) + m2 g
dt
d 3 x3
m3 2 = −k ( x3 − x2 ) + m3 g
dt
STEADY-STATE
Set second derivatives to zero and collect terms:

3kx1 − 2kx2 = m1 g
−2kx1 + 3kx2 − kx3 = m2 g
−kx2 + kx2 = m3 g

or in matrix form:
 K  X  = W 
where {X} = vector of unknowns, {W} = vector of weights, and [K] is called the
stiffness matrix

 3k −2k 
 K  = −2k 3k −k 
 −k −k 
SOLUTION
Substitute parameter values:

 30 −20   x1   19.62 
 −20 30 −10   x  =  29.43 
  2  
 −10 −10   x3  24.525

and solve for the displacements:

▪ x1 = 7.36
▪ x2 = 10.06
▪ x3 = 12.51.
 QUIZ #1

a. PLOT y=f(x), and visually investigate potential roots

b. Find the roots with MATLAB commands ‘roots’. Send copy of your
command line.
c. Find the roots with MATLAB command ‘fzero’. Send copy of your
command line.
d. Find the smallest positive zero of the function using your own MATLAB
code with tolerance 1x10-5 . Use any of the methods ‘Bisection, Newton-
Raphson or Secant’
e. Please indicate why you have chosen the method in section d.

Send your results and your code to kilicna@mef.edu.tr with e-mail.