F

INITE ELEMENT M ETHOD

(MEMED03/MPMED01)

1
 LECTURE 4
MATRIX AND VECTORS: PRELIMINARY

2
 MATHEMATICAL PRELIMINARY
• Vector
– a collection of scalars, defined using a bold typeface with braces
y
 a1   a1 
a     a1  a2
  a  a2  or a 
{a }   2  a  a2 
   3 j
i x
aN  a1
k
a  a1i  a2 j  a3k a3
• Matrix z

– a collection of vectors, defined using a bold type face with brackets

– dimension = N×K. When N = K, it is called a square matrix

M11 M12 M1K 

M M22 M2K 
[M]   21 
 
 
MN1 MN2 MNK 
3
 MATRIX
– Transpose of a matrix: Change of row and column

 M11 M21 MN1 

M MN2 
[M] 
T  12 M22
 {a} T  a1 a2 aN 
 
 
M1K M2K MNK 

– Symmetric and Skew-symmetric matrices

M11 M12 M1N   0 M12 M1N 
M M2N   M

M22
 0 M2N 
[ S ]  [ S]  [ A]  [ A]T   12 
T 12

   
   
M1N M2N MNN   M1N M2N 0 

– Identity matrix
1 0 0
[I]  0 1 0 
 
0 0 1
4
 VECTOR-MATRIX CALCULUS
• Addition
{c}  {a}  {b},  c i  ai  bi , i  1, ,N
{d}  {a}  {b},  di  ai  bi , i  1, ,N
k{a}  {ka1 ka2 kaN } T
[C]  [ A]  [B],  Cij  A ij  Bij , i  1, ,N, j  1, ,K
[D]  [ A]  [B],  Dij  A ij  Bij , i  1, ,N, j  1, ,K
• Scalar product between two vectors (must be the same dim)
a  b  a1b1  a2b2  a3b3  a1   b1 
   
a  a2  b  b2 
a  b  a b cos  a  b 
 3  3
• Norm (Magnitude of a vector)
a  aa

5
 DETERMINANT
• Similar to the norm of a vector
• Only defined for a square matrix
• If a determinant is zero, the matrix is not invertible
• A matrix is singular when its determinant is zero
• For a 2x2 matrix:
a11 a12
A   a11a22  a12a21
a21 a22
• For a 3x3 matrix
a11 a12 a13
A  a21 a22 a23
a31 a32 a33
a22 a23 a21 a23 a21 a22
 a11  a12  a13
a32 a33 a31 a33 a31 a32
 a11(a22a33  a23a32 )  a12 (a21a33  a23a31 )  a13 (a21a32  a22a31 )
6
 VECTOR-MATRIX CALCULUS cont.
• Vector product
– Scalar product  result = scalar
– Vector product  result = vector

i j k
a  b  a1 a2 a3
b1 b2 b3
 (a2b3  a3b2 )i  (a3b1  a1b3 ) j  (a1b2  a2b1 )k
a2b3  a3b2 
  a×b
  a3b1  a1b3 
a b  a b  b
 1 2 2 1

n
a  b  a b sin  n
θ
aa  0
b  a  a  b a

7
 MATRIX-VECTOR MULTIPLICATION
• Matrix  Vector = Vector
m11 m12 m13   a1   m11a1  m12a2  m13a3 
   
c  [M]  a  m21 m22 m23   a2   m21a1  m22a2  m23a3 
 
m31 m32 m33  a3  m31a1  m32a2  m33a3 

3 {c}N1  [M]NK {a}K1

c i   mija j , i  1,2,3 K
c i   mija j , i  1, N
j1

j1

• Vector  Matrix  Vector = Scalar

bT  [M]  a  b1(m11a1  m12a2  m13a3 )
b2 (m21a1  m22a2  m23a3 )
b3 (m31a1  m32a2  m33a3 )

8
 MATRIX-MATRIX MULTIPLICATION
• Matrix  Matrix = Matrix
3
[C]  [ A][B] Cij   A ikBkj , i, j  1,2,3
k 1

• Inverse of a matrix:
– A square matrix [A] is invertible, then
[ A][B]  [I] [B][ A]  [I]

1
1  a11 a12  1  a22 a12 
[B]  [ A]      a 
 21
a a 22  A  21 a11 

– If a matrix is singular (|A| = 0), then the inverse does not exist

9
 RULES OF MATRIX MULTIPLICATION

• Associative rule:( AB)C  A(BC)

• Distributive rule:A(B  C)  AB  AC

• Non-commutative:AB  BA

• Transpose of product:( AB)T  BT A T , ( ABC)T  CTBT A T

• Inverse of product:( AB)1  B1A 1, ( ABC)1  C1B1A 1

10
 MATRIX EQUATION
– N unknowns (x1, x2, …, xN) and N equations
– unique solution if all equations are independent

a11x1  a12 x 2   a1N xN  b1  a11 a12 a1N 

a a2N 
a21x1  a22 x 2   a2N xN  b2 
a22

[ A]  21

 
 
aN1x1  aN2 x 2   aNN xN  bN aN1 aN2 aNN 

– Matrix form:  x1   b1 
x  b 
 2  
[ A]  {x }  {b} {x }    , {b}   2 
   
– Solution: [A]–1 exists or [A] is not singular  xN  bN 
[ A]1[ A]  {x }  [ A]1  {b}

[I]  {x }  [ A]1  {b}

{x }  [ A]1  {b}
11
 EIGEN VALUE AND EIGEN VECTOR
• Eigen value problem
[ A]  {x }   {x}

 : Eigen value
{x } : Eigen vector

• How to solve?
[ A]  {x }   {x }  {0}
[ A  I]  {x }  {0}

– {x} = {0} is a solution (trivial solution)

– In order to have non-trivial solution, the determinant must be zero.
A  I  0
– Calculate  from this equation and calculate {x } from the eigen value
problem

12
 EIGEN VALUE AND EIGEN VECTOR
• Characteristic equation
a11   a12 a1n
a21 a22   a2n
0 n  C1n1  .....Cn1  Cn  0

an1 ann  

– A solution for [A]3x3 case

• Eigen vectors
– After solving for eigen values, substitute each of them to eigen problem

[ A]  {x i }  i {x i }

– Since A  iI  0 is singular, no unique solution exists

– Practice example in the textbook

13
 QUADRATIC FORM
• Quadratic form: quadratic function of all components

F  a11x12  a22 x 22   ann xn2  a12 x1x 2  a13 x1x 3   an,n1x n x n1

• Matrix notation  a11 a12 a1n   x1 

a a22 a2n   x 2  n n
F  {x } [ A]{ x }  {x1,x 2 , ,x n }
T  21      aij x i x j
    i1 j1
 
 an1 an2 ann   xn 

• Symmetric part is enough ([B] is not sym)

F  {x } T [B]{ x } 1 1
[B]  [B  BT ]  [B  BT ]  [BS ]  [B A ]
2 2
FS  {x } T [BS ]{ x } F  {x } T [B]{ x }  { x } T [BS ]{ x }  FS

14
 POSITIVE DEFINITE MATRIX
• Positive definite

{x } T [ A]{x }  0, for all { x } in Rn

{x } T [ A]{x }  0, only if { x }  {0}

• Positive semi-definite

{x } T [ A]{x }  0, for all { x} in Rn

{x } T [ A]{x }  0, for some { x }  {0}

• Positive definiteness = each column of the matrix is linearly

independent = the matrix is invertible = the matrix is not
singular = the matrix equation has a unique solution.

15
 MAXIMA & MINIMA OF FUNCTIONS
• Single Variable f(x)
– Taylor series expansion
f 1 2f
f(x  x)  f(x)  x  ( x)2  HOT
x x  x 2 x 2 xx

– In order for f to be extremum,

f
0
x x  x
2f
– Condition for minima: 0
x 2 xx

2f
– Condition for maxima: 2 0
x xx

16
 MAXIMA & MINIMA OF FUNCTIONS cont.
• Multi-Variable f(x)
– Taylor series expansion
fn
1 n n 2f
f( x  x )  f( x )   x i   x i x j  HOT
i1 x i x x
2 i1 j1 xix j
x x

– In order for f to be extremum, Hessian

f f f matrix Hij
   0
x1 x x x 2 x x
xn x x

– Condition for minima: [H] is positive definite

– Condition for maxima: [H] is negative definite

17
 MINIMUM PRINCIPLE
• Function in quadratic form

F( x )  21 {x } T [ A]{x }  {x } T {b}

– [A]: stiffness of the structure, {x}: displacement, {b}: applied force

– Potential energy
– structure is in equilibrium when F has a minimum value

• Matrix equation
[ A]{x }  {b}

– Solution of the matrix equation minimizes the quadratic form F.

18
 Tutorial #1
1. For the two matrices [A] and [B], answer the following
questions.
(a) Evaluate the matrix–matrix multiplication [C] = [A][B].
(b) Evaluate the matrix–matrix multiplication [D] = [B][A].
 1 7 2  3 7 2 
 A   B  3 4 3   2 1 8 
6 5 7  7 4 5 

2. Calculate the inverse of the matrix

 4 2
 A   3 7 
 

3. Solve the following simultaneous system of equations using

the matrix method: 4x  3x  3
1 2

x1  3x 2  3
19
 Tutorial #1
4. Find the eigen values and eigen vectors
5 2 0 
[ A]  2 2 0 
 
0 0 1
5. A function f(x1, x2) of two variables x1 and x2 is given by
1  1 1  x1  0 
f(x1,x 2 )  x1 x 2       1  x x 2    1
2  1 2   x 2  2 
(a) Multiply the matrices and express f as a polynomial in x1 and x2.
(b) Determine the extreme (maximum or minimum) value of the function
and corresponding x1 and x2.
(c) Is this a maxima or minima?

20
LECTURE 5-7
Simultaneous Linear Equations

21
Topics to be covered in these lectures
1. Gaussian Elimination
2. Gauss-Seidel Method
3. LU Decomposition
4. Cholesky and LDL Decomposition

22
Gaussian Elimination

23
 Naïve Gaussian Elimination
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution

24
 Forward Elimination

The goal of forward elimination is to transform the

coefficient matrix into an upper triangular matrix

 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 
25
 Forward Elimination

A set of n equations and n unknowns

a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
. .
. .
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn

(n-1) steps of forward elimination

26
 Forward Elimination

Step 1
For Equation 2, divide Equation 1 aby
11 and
a21 multiply by .

 a21 
 a (a11x1  a12 x2  a13 x3  ...  a1n xn  b1 )
 11 
a21 a21 a21
a21x1  a12 x2  ...  a1n xn  b1
a11 a11 a11

27
 Forward Elimination

Subtract the result from Equation 2.

a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
a21 a21 a21
a21x1  a12 x2  ...
−  a1n xn  b1
a11 a11 a11
_________________________________________________
 a21   a21  a21
 a22  a12  x2  ...   a2 n  a1n  xn  b2  b1
 a11   a11  a11

or a x  ...  a x  b
'
22 2
'
2n n
'
2

28
 Forward Elimination

Repeat this procedure for the remaining

equations to reduce the set of equations as
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
'
a32 x2  a33
'
x3  ...  a3' n xn  b3'
. . .
. . .
. . .

an' 2 x2  an' 3 x3  ...  ann

'
xn  bn'

End of Step 1
29
 Forward Elimination

Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3" n xn  b3"
. .
. .
. .

an" 3 x3  ...  ann

"
xn  bn"
End of Step 2
30
 Forward Elimination

At the end of (n-1) Forward Elimination steps, the

system of equations will look like
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3" n xn  b3"
. .
. .
. .

 n 1 n 1 
ann xn  bn

End of Step (n-1)

31
Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' '
a23 '
 a2 n x2     b2' 
 22    
0 0 "
a33  a3n   x3    b3 
" "

    
           
 0 0 0 (n1 )
   
0 ann   xn  bn 
(n-1 )

32
 Back Substitution

Solve each equation starting from the last equation

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 

Example of a system of 3 equations

33
Back Substitution Starting Eqns

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

 n 1 n 1 
ann xn  bn

34
 Back Substitution

Start with the last equation because it has only one unknown
( n 1)
b
xn  n
( n 1)
a nn

35
 Back Substitution

( n 1)
b
xn  n
( n 1)
a nn

bii 1  ai,ii11 xi 1  ai,ii12 xi  2  ...  ai,in1 xn

xi  i 1 for i  n  1,...,1
aii

i 1
  aiji 1 x j
n
bi
j i 1
xi  i 1 for i  n  1,...,1
a ii

36
 Example 1

The upward velocity of a rocket is given at three

different times

Table 1 Velocity vs. time data.

Time, t s  Velocity, v m/s 

5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

vt   a1t 2  a 2 t  a3 , 5  t  12.

Find the velocity at t=6 seconds .
37
 Example 1 Cont.

Assume
vt   a1t 2  a2t  a3 , 5  t  12.

Results in a matrix template of the form:

t12 t1 1  a1   v1 
 2   a   v 
t 2 t2 1  2  2
t32 t3 1 
 a3 
  v3 

 

Using data from Table 1, the matrix becomes:

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

38
 Example 1 Cont.

 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a2   177.2    64 8 1  177.2 
       
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution

39
Forward Elimination

40
 Number of Steps of Forward Elimination

Number of steps of forward elimination is

(n1)(31)2

41
 Forward Elimination: Step 1
 25 5 1  106.8  Divide Equation 1 by 25 and
 64 8 1  177.2 
  64
multiply it by 64,  2.56 .
144 12 1  279.2 25
25 5 1  106.8 2.56  64 12.8 2.56  273.408

. 64 8 1  177.2
Subtract the result from  64 12.8 2.56  273.408
Equation 2
0  4.8  1.56   96.208

 25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208
Equation 2  
144 12 1  279.2 

42
 Forward Elimination: Step 1 (cont.)
 25 5 1  106.8 
Divide Equation 1 by 25 and
 0  4.8  1.56   96.208
  144
144 12 1  279.2  multiply it by 144,  5.76 .
25

25 5 1  106.8 5.76  144 28.8 5.76  615.168

.
Subtract the result from144 12 1  279.2
Equation 3  144 28.8 5.76  615.168
0  16.8  4.76   335.968
25 5 1  106.8 
Substitute new equation for
 0  4.8  1.56   96.208 
Equation 3  
 0  16.8  4.76   335.968
43
 Forward Elimination: Step 2
25 5 1  106.8  Divide Equation 2 by −4.8
 0  4.8  1.56   96.208 
  and multiply it by −16.8,
 0  16.8  4.76   335.968  16.8
 3 .5 .
 4 .8
0  4.8 1.56   96.208 3.5  0 16.8  5.46   336.728
0  16.8  4.76  335.968
Subtract the result from
Equation 3  0  16.8  5.46   336.728
0 0 0.7  0.76

25 5 1  106.8 
Substitute new equation for  
 0  4.8  1.56   96.208
Equation 3
 0 0 0.7  0.76 
44
Back Substitution

45
 Back Substitution

25 5 1  106.8  25 5 1   a1   106.8 

 0  4.8  1.56   96.2   0  4.8  1.56 a    96.208
     2  
 0 0 0.7  0.7   0 0 0.7   a3   0.76 

Solving for a3
0.7a3  0.76
0.76
a3 
0.7
a3  1.08571

46
 Back Substitution (cont.)

25 5 1   a1   106.8 
 0  4.8  1.56 a    96.208
   2  
 0 0 0.7   a3   0.76 

Solving for a2
 4.8a2  1.56a3  96.208
 96.208  1.56a3
a2 
 4.8
 96.208  1.56 1.08571
a2 
 4.8
a2  19.6905
47
 Back Substitution (cont.)

25 5 1   a1   106.8 
 0  4.8  1.56 a    96.2
   2  
 0 0 0.7   a3   0.76 

Solving for a1
25a1  5a2  a3  106.8
106.8  5a2  a3
a1 
25
106.8  5 19.6905  1.08571

25
 0.290472
48
Naïve Gaussian Elimination
Solution
 25 5 1  a1  106.8 
 64 8 1 a2   177.2 
     
144 12 1  a3  279.2

 a1  0.290472
a    19.6905 
 2  
 a3   1.08571 

49
 Example 1 Cont.

Solution  a1  0.290472
a    19.6905 
The solution vector is
 2  
 a3   1.08571 

The polynomial that passes through the three data points is then:

vt   a1t 2  a2t  a3

 0.290472t 2  19.6905t  1.08571, 5  t  12

v6  0.2904726  19.69056  1.08571

2

 129.686 m/s .
50
 Naïve Gauss
Elimination Pitfalls

51
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9

0 10  7  x1   3 
6 2  
3 x2  11   
    
5  1 5   x3   9 

52
Is division by zero an issue here?

12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9

12 10  7  x1  15
6 5 3   x2   14
    
 5  1 5   x3   9 

53
 Is division by zero an issue here?
YES

12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28

12 10  7  x1  15 12 10  7  x1   15 

6 5 3   x2   14  0
     0 6.5  x2   6.5
    
24  1 5   x3  28 12  21 19   x3   2

Division by zero is a possibility at any step

of forward elimination
54
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Exact Solution

 x1  1
 x   1
 2  
 x3  1
55
 Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 6 significant digits with chopping

 x1   0.9625 
 x    1.05 
 2  
 x3  0.999995
56
 Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 5 significant digits with chopping

 x1   0.625 
 x    1.5 
 2  
 x3  0.99995
Is there a way to reduce the round off error?
57
 Avoiding Pitfalls

Increase the number of significant digits

• Decreases round-off error
• Does not avoid division by zero

58
 Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting

• Avoids division by zero
• Reduces round off error

59
Gauss Elimination with
Partial Pivoting

60
 Pitfalls of Naïve Gauss Elimination
• Possible division by zero
• Large round-off errors

61
 Avoiding Pitfalls

Increase the number of significant digits

• Decreases round-off error
• Does not avoid division by zero

62
 Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting

• Avoids division by zero
• Reduces round off error

63
 What is Different About Partial
Pivoting?

At the beginning of the kth step of forward elimination,

find the maximum of

akk , ak 1,k ,................, ank

If the maximum
a
of the values is
pk

in the p th row,k  p  n, then switch rows p and k.

64
Matrix Form at Beginning of 2nd Step
of Forward Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' '
a23  ' 
a2 n x2   ' 
b2
 22    
 0 a32' '
a33  a3n   x3   b3 
' '

    
           
 0 a'n 2 ' '
a n 3 an 4 '
   
ann   xn  bn 
'

65
Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0  7 6 1 2   x2   6
    
0 4 12 1 11  x3    8 
    
 0 9 23 6 8 x
   
4 9
0  17 12 11 43  x5   3 

Which two rows would you switch?

66
Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 
  2   
0 4 12 1 11  x3    8 
    
 0 9 23 6 8 x
   
4 9
0  7 6 1 2   x5   6

Switched Rows

67
 Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution

68
 Forward Elimination
Same as naïve Gauss elimination method except that we switch
rows before each of the (n-1) steps of forward elimination.

69
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination

a11 a12 a13  a1n   x1   b1 

 0 a' '
a23  ' 
a2 n x2   ' 
b2
 22    
 0 a32' '
a33  a3n   x3   b3 
' '

    
          
 0 a'n 2 ' '
a n 3 an 4 '
   
ann   xn  bn 
'

70
 Matrix Form at End of Forward
Elimination

a11 a12 a13  a1n   x1   b1 

 0 a' '
a23 '
 a2 n x2     b2' 
 22    
0 0 "
a33  a3n   x3    b3 
" "

    
          
 0 0 0 (n1 )
   
0 ann   xn  bn 
(n-1 )

71
Back Substitution Starting Eqns

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

 n 1 n 1 
ann xn  bn

72
 Back Substitution

( n 1)
b
xn  n
( n 1)
a nn

i 1 n
i 1
bi   aij x j
j i 1
xi  i 1 for i  n  1,...,1
a ii

73
 Example 2

Solve the following set of equations

by Gaussian elimination with partial
pivoting

 25 5 1  a1  106.8 
 64   
8 1 a2  177.2  
     
144 12 1  a3  279.2

74
 Example 2 Cont.

 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a   177.2    
    2   64 8 1  177.2 
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution

75
Forward Elimination

76
 Number of Steps of Forward Elimination

Number of steps of forward elimination is (n1)=(31)=2

77
 Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.

 25 5 1  106.8  144 12 1  279.2

 64 8 1  177.2    64 8 1  177.2 
   
144 12 1  279.2  25 5 1  106.8 

78
 Forward Elimination: Step 1 (cont.)

144 12 1  279.2 Divide Equation 1 by 144 and

 64 8 1  177.2  64
  multiply it by 64,  0.4444 .
 25 5 1  106.8  144

144 12 1  279.2 0.4444  63.99 5.333 0.4444  124.1

Subtract the result from 64 1  177.2

.
8
Equation 2  63.99 5.333 0.4444  124.1
0 2.667 0.5556  53.10

Substitute new equation for144 12 1  279.2

Equation 2  0 2.667 0.5556  53.10
 
 25 5 1  106.8 
79
 Forward Elimination: Step 1 (cont.)

144 12 1  279.2 Divide Equation 1 by 144 and

 0 2.667 0.5556  53.10 25
  multiply it by 25,  0.1736 .
 25 5 1  106.8  144

144 12 1  279.2 0.1736  25.00 2.083 0.1736  48.47

.
25 5 1  106.8
Subtract the result from
Equation 3  25 2.083 0.1736  48.47
0 2.917 0.8264  58.33

Substitute new equation for 144 12 1  279.2

Equation 3  0 2.667 0.5556  53.10
 
 0 2.917 0.8264  58.33
80
 Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.

144 12 1  279.2 144 12 1  279.2

 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33
   
 0 2.917 0.8264  58.33  0 2.667 0.5556  53.10

81
 Forward Elimination: Step 2
(cont.)
Divide Equation 2 by 2.917 and
144 12 1  279.2
 0 2.917 0.8264  58.33 multiply it by 2.667,
  2.667
 0 2.667 0.5556  53.10  0.9143.
2.917
0 2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33

.
0 2.667 0.5556  53.10
Subtract the result from  0 2.667 0.7556  53.33
Equation 3
0 0  0.2   0.23

144 12 1  279.2 
Substitute new equation for  0 2.917 0.8264  58.33 
Equation 3  
 0 0  0.2   0.23

82
Back Substitution

83
 Back Substitution

144 12 1  279.2  144 12 1   a1   279.2 

 0 2.917 0.8264  58.33    0 2.917 0.8264 a    58.33 
     2  
 0 0  0.2   0.23  0 0  0.2   a3   0.23

Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15

84
 Back Substitution (cont.)

144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2  a3   0.23

Solving for a2
2.917a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264 1.15

2.917
 19.67
85
 Back Substitution (cont.)

144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3   0.23

Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917
86
Gaussian Elimination with Partial
Pivoting Solution

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

 a1  0.2917
a    19.67 
 2  
 a3   1.15 
87
Gauss Elimination with
Partial Pivoting
Another Example

88
 Partial Pivoting: Example

Consider the system of equations

10 x1  7 x2  7
 3x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6
In matrix form
 10  7 0  x1   7 
 3 2.099 6  x  3.901
   2 =  
 5  1 5  x 3   6 

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping

89
 Partial Pivoting: Example

Forward Elimination: Step 1

Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.

Performing Forward Elimination

 10  7 0  x1   7  10 7 0  x1   7 
 3 2.099 6  x   3.901

 5
 2   
 1 5  x3   6 
  0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 

90
 Partial Pivoting: Example

Forward Elimination: Step 2

Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3
Performing the row swap
7 7

10 0  x1   7  10 0  x1   7 
 0  0.001 6  x   6.001 0   x    2.5 
  2     2.5 5  2   
 0 2.5 5  x3   2.5   0  0.001 6  x3  6.001

91
 Partial Pivoting: Example

Forward Elimination: Step 2

Performing the Forward Elimination results in:

10  7 0   x1   7 
 0 2.5 5   x    2.5 
  2   
 0 0 6.002  x3  6.002

92
 Partial Pivoting: Example

Back Substitution
Solving the equations through back substitution

6.002
10  7 0   x1   7  x3  1
 0 2.5 5   x    2.5  6.002
  2   
 0 0 6.002  x3  6.002 2.5  5 x3
x2   1
2.5

7  7 x 2  0 x3
x1  0
10

93
 Partial Pivoting: Example

Compare the calculated and exact solution

The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

 x1   0   x1   0 
X  calculated   x2    1 X  exact   x2    1
 x3   1   x3   1 

94
Determinant of a Square
Matrix
Using Naïve Gauss
Elimination
Example

95
 Theorem of Determinants

If a multiple of one row of [A]nxn is added or

subtracted to another row of [A]nxn to result in
[B]nxn then det(A)=det(B)

96
 Theorem of Determinants

The determinant of an upper triangular matrix [A]nxn is given by

det A   a11  a22  ...  aii  ...  ann

n
  a ii
i 1

97
 Forward Elimination of a
Square Matrix

Using forward elimination to transform [A]nxn to an upper

triangular matrix, [U]nxn.

Ann  U  nn

det  A  det U 

98
 Example
Using naïve Gaussian elimination find the
determinant of the following square
matrix.
 25 5 1
 64 8 1
 
144 12 1

99
Forward Elimination

100
 Forward Elimination: Step 1
 25 5 1
 64 8 1 Divide Equation 1 by 25 and
  multiply it by 64,
64
 2.56 .
144 12 1 25

25 5 1 2.56  64 12.8 2.56

. 64 8 1
Subtract the result from  64 12.8 2.56
Equation 2
0  4.8  1.56
 25 5 1 
Substitute new equation for  0  4.8  1.56
Equation 2  
144 12 1 
101
 Forward Elimination: Step 1
 25 5 1  (cont.)
 0  4.8  1.56 Divide Equation 1 by 25 and
  multiply it by 144,
144
 5.76 .
144 12 1  25
25 5 1 5.76  144 28.8 5.76
144 12 1
 144 28.8 5.76
.
Subtract the result from
Equation 3
0  16.8  4.76

25 5 1 
Substitute new equation for  0  4.8  1.56 
Equation 3  
 0  16.8  4.76
102
 Forward Elimination: Step 2
Divide Equation 2 by −4.8
25 5 1 
 0  4.8  1.56  and multiply it by −16.8,
   16.8
 0  16.8  4.76  3 .5 .
 4 .8
0  4.8  1.56  3.5  0  16.8  5.46
.

Subtract the result from

0  16.8  4.76
Equation 3  0  16.8  5.46
0 0 0.7

Substitute new equation for 25 5 1 

 0  4.8  1.56
Equation 3  
 0 0 0.7 
103
Finding the Determinant

After forward elimination

 25 5 1 25 5 1 
 64 8 1   0  4.8  1.56
   
144 12 1  0 0 0.7 
.

det A   u11  u22  u33

 25   4.8 0.7
 84.00

104
LU Decomposition

105
 LU Decomposition

LU Decomposition is another method to solve a set of

simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is

needed.

106
 LU Decomposition
Method
For most non-singular matrix [A] that one could conduct Naïve Gauss
Elimination forward elimination steps, one can always write it as

[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix

107
 How does LU Decomposition work?

If solving a set of linear equations [A][X] = [C]

If [A] = [L][U] then [L][U][X] = [C]
Multiply by [L]-1
Which gives [L]-1[L][U][X] = [L]-1[C]
Remember [L]-1[L] = [I] which leads to [I][U][X] = [L]-1[C]
Now, if [I][U] = [U] then [U][X] = [L]-1[C]
Now, let [L]-1[C]=[Z]
Which ends with [L][Z] = [C] (1)
and [U][X] = [Z] (2)

108
 LU Decomposition

How can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]

109
Is LU Decomposition better than Gaussian
Elimination?

Solve [A][X] =
[B]nxn = size of the matrix
T = clock cycle time and
Forward Elimination Decomposition to LU
 8n 3 32n   8n 3 20n 
CT | FE  T   8n 
2
 CT | DE  T   4n 2  
 3 3   3 3 

Back Substitution Forward Substitution


CT | BS  T 4n 2  12n  
CT | FS  T 4n 2  4n 
Back Substitution


CT | BS  T 4n 2  12n 
110
 Is LU Decomposition better than Gaussian
Elimination?

To solve [A][X] = [B]

Time taken by methods

Gaussian Elimination LU Decomposition

 8n3 4n   8n3 4n 
T   12n 2   T   12n 2  
 3 3   3 3 

T = clock cycle time and nxn = size of the matrix

So both methods are equally efficient.

111
 To find inverse of [A]

Time taken by Gaussian Elimination Time taken by LU Decomposition

 nCT |FE CT |BS   CT |DE  n  CT |FS  n  CT |BS
 8n 4 4n 2   32n3 20n 
 T   12n 
3
  T   12n 2  
 3 3   3 3 

112
 To find inverse of [A]

Time taken by Gaussian Elimination Time taken by LU Decomposition

 8n 4 4n 2   32n3 20n 
T   12n 
3
 T   12n 2  
 3 3   3 3 

Table 1 Comparing computational times of finding inverse of a matrix using

LU decomposition and Gaussian elimination.
n 10 100 1000 10000
CT|inverse GE / CT|inverse LU 3.288 25.84 250.8 2501

For large n, CT|inverse GE / CT|inverse LU ≈ n/4

113
Method: [A] Decomposes to [L] and
[U]

1 0 0 u11 u12 u13 

A  LU    21 1 0  0 u 22 u 23 
 31  32 1  0 0 u 33 

[U] is the same as the coefficient matrix at the end of the forward elimination step.
[L] is obtained using the multipliers that were used in the forward elimination process

114
 Finding the [U] matrix

Using the Forward Elimination Procedure of Gauss Elimination

 25 5 1
 64 8 1
 
144 12 1
 25 5 1 
 2.56; Row 2  Row12.56   0  4.8  1.56
64
Step 1:
25
144 12 1 
25 5 1 
 5.76; Row3  Row15.76   0  4.8  1.56 
144
25
 0  16.8  4.76
115
 Finding the [U] Matrix

25 5 1 
Matrix after Step 1:  0  4.8  1.56 
 
 0  16.8  4.76

25 5 1 
 16.8
Step 2:  3.5; Row3  Row 23.5   0  4.8  1.56
 4.8
 0 0 0.7 

25 5 1 
U    0  4.8  1.56
 0 0 0.7 

116
 Finding the [L] matrix

1 0 0
 0
 21 1
 31  32 1

Using the multipliers used during the Forward Elimination Procedure

a 64
From the first step  25 5 1  21  21   2.56
of forward  64 8 1 a11 25
elimination    31 
a31 144
  5.76
144 12 1 a11 25

117
 Finding the [L] Matrix

From the second 25 5 1 

step of forward  0  4.8  1.56   32  a32   16.8  3.5
elimination   a22  4.8
 0  16.8  4.76

 1 0 0

L  2.56 1 0 
5.76 3.5 1

118
 Does [L][U] = [A]?

 1 0 0 25 5 1 
LU   2.56 1 0  0  4.8  1.56 
5.76 3.5 1  0 0 0.7 
?

119
Using LU Decomposition to
solve SLEs
Solve the following set of
linear equations using LU
Decomposition
 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2

Using the procedure for finding the [L] and [U] matrices

 1 0 0 25 5 1 
A  LU   2.56 1 0  0  4.8  1.56
5.76 3.5 1  0 0 0.7 

120
 Example

Set [L][Z] = [C]  1 0 0  z1  106.8 

2.56 1 0  z   177.2 
  2   
5.76 3.5 1  z 3  279.2

Solve for [Z] z1  10

2.56 z1  z 2  177.2
5.76 z1  3.5 z 2  z 3  279.2

121
 Example

Complete the forward substitution to solve for [Z]

z1  106.8
z 2  177.2  2.56 z1  z1   106.8 
 177.2  2.56106.8
 96.2
Z    z2    96.21
z3  279.2  5.76 z1  3.5 z 2  z3   0.735 
 279.2  5.76106.8  3.5 96.21
 0.735

122
 Example

25 5 1   x1   106.8 
Set [U][X] = [Z]
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 

Solve for [X] The 3 equations become

25a1  5a2  a3  106.8

 4.8a2  1.56a3  96.21
0.7a3  0.735

123
 Example

Substituting in a3 and using the

From the 3rd equation second equation

0.7a3  0.735  4.8a2  1.56a3  96.21

a3 
0.735  96.21  1.56a3
a2 
0.7  4.8
a3  1.050  96.21  1.561.050
a2 
 4.8
a2  19.70

124
 Example

Substituting in a3 and a2 using Hence the Solution Vector is:

the first equation

25a1  5a2  a3  106.8  a1  0.2900

106.8  5a2  a3 a    19.70 
a1 
25
 2  
106.8  519.70  1.050
 a3   1.050 

25
 0.2900

125
Finding the inverse of a square
matrix

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]

126
Finding the inverse of a square
matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

b11  1  b12  0
b  0 b  1
A  21     A  22    
         
       
bn1  0 bn 2  0

The remaining columns in [B] can be found in the same manner

127
 Example: Inverse of a Matrix

Find the inverse of a square matrix [A]

 25 5 1
A   64 8 1
144 12 1

Using the decomposition procedure, the [L] and [U] matrices are found to be

 1 0 0 25 5 1 
A  LU   2.56 1 0  0  4.8  1.56
5.76 3.5 1  0 0 0.7 

128
 Example: Inverse of a Matrix

Solving for the each column of [B] requires two steps

1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]

 1 0 0  z1  1
Step 1: LZ   C   2.56 1 0  z2   0
5.76 3.5 1  z3  0
This generates the equations:
z1  1
2.56 z1  z 2  0
5.76 z1  3.5 z 2  z3  0
129
 Example: Inverse of a Matrix

Solving for [Z]

z1  1
 z1   1 
z 2  0  2.56 z1
 0  2.561
Z    z2    2.56
 2.56  z3   3.2 
z3  0  5.76 z1  3.5 z 2
 0  5.761  3.5 2.56
 3.2

130
 Example: Inverse of a Matrix

25 5 1  b11   1 
Solving [U][X] = [Z] for [X]  0  4.8  1.56 b    2.56
   21   
 0 0 0.7  b31   3.2 

25b11  5b21  b31  1

 4.8b21  1.56b31  2.56
0.7b31  3.2

131
 Example: Inverse of a Matrix

Using Backward Substitution

3.2
b31   4.571 So the first column of
0.7
the inverse of [A] is:
2.56  1.560b31
b21 
4.8 b11   0.04762 
2.56  1.5604.571 b    0.9524
  0.9524
4.8  21   
b11 
1  5b21  b31 b31   4.571 
25
1  5 0.9524   4.571
  0.04762
25

132
 Example: Inverse of a Matrix

Repeating for the second and third columns of the inverse

Second Column Third Column

 25 5 1 b12  0  25 5 1 b13  0
 64 8 1 b   1  64 8 1 b   0
   22       23   
144 12 1 b32  0 144 12 1 b33  1
b12   0.08333 b13   0.03571 
b    1.417  b    0.4643
 22     23   
b32    5.000  b33   1.429 

133
Example: Inverse of a Matrix

The inverse of [A] is

 0.04762  0.08333 0.03571 

A1   0.9524 1.417  0.4643
 4.571  5.000 1.429 

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

134