P. 1
MIT2_29F11_lect_8

# MIT2_29F11_lect_8

|Views: 2|Likes:
MIT2_29F11_lect_8
MIT2_29F11_lect_8

See more
See less

04/24/2013

pdf

text

original

# 2.

29 Numerical Fluid Mechanics
Fall 2011 – Lecture 8
REVIEW Lecture 7:
• Direct Methods for solving Linear Equation Systems
– Determinants and Cramer’s Rule (and other methods for a small
number of equations)
– Gauss Elimination
• Algorithm
– Forward Elimination/Reduction to Upper Triangular System
– Back-Substitution
– Number of Operations: O(n
3
)
• Numerical implementation and stability
– Partial Pivoting
– Equilibration
– Full Pivoting
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
1
1

Gauss Elimination: Review
Reduction/Elimination: Step k
Back-Substitution
Reduction: Step (n-1)
Result after step (n-1) is an
Upper triangular system!
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
2
2

Gauss Elimination: Pivoting Review
Reduction
Partial Pivoting by Columns
Step k
New Row k
Pivot Elements
New Row i
Required at
each step!
A. Partial Pivoting
i. Search for largest available coefficient in column below pivot element
Two Solutions:
ii. Switch rows k and i
B. Complete Pivoting
i. As for Partial, but search both rows and columns
ii. Rarely done since column re-ordering changes order of x’s
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
3
3

Gauss Elimination: Review
Numerical Stability
• Partial Pivoting
– Equilibrate system of equations (Normalize or scale variables)
– Pivoting by Columns
– Simple book-keeping
• Solution vector in original order
• Full Pivoting
– Does not necessarily require equilibration
– Pivoting by both row and columns
– More complex book-keeping
• Solution vector re-ordered
Partial Pivoting is simplest and most common
Neither method guarantees stability due to large number
of recursive computations (round-off error)
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
4
4

TODAY’s Lecture: Systems of Linear Equations II
• Direct Methods
– Gauss Elimination
• Algorithm
– Forward Elimination/Reduction to Upper Triangular System
– Back-Substitution
– Number of Operations: O(n
3
)
• Numerical implementation and stability
– Well suited for dense matrices,
– Issues: round-off, cost, does not vectorize/parallelize well
• Special cases, Multiple right hand sides, Operation count
– LU decomposition/factorization
– Error Analysis for Linear Systems
– Condition Number
– Special Matrices: Tri-diagonal systems
• Iterative Methods
– Jacobi’s method, Gauss-Seidel iteration, etc
– Convergence
2.29
Numerical Fluid Mechanics PFJL Lecture 8, 5
5

• Chapter 10 of “Chapra and Canale, Numerical Methods for
Engineers, 2006/2009.”
– Any chapter on “Solving linear systems of equations” in CFD
references provided. For example: chapter 5 of “J. H. Ferziger and M.
Peric, Computational Methods for Fluid Dynamics. Springer, NY, 3
rd
edition, 2002”
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
6
6

Special Applications of Gauss Elimination
•Complex Systems
–Replace all numbers by complex ones, or,
–Re-write system of n complex equations into 2n real equations
•Nonlinear Systems of equations
–Newton-Raphson: 1
st
order term kept, use 1
st
order derivatives
–Secant Method: Replace 1
st
order derivatives with finite-difference
–In both cases, at each iteration, this leads to a linear system, which can be solved
by Gauss Elimination
•Gauss-Jordan: variation of Gauss Elimination
–Elimination
• Eliminates each unknown completely (both below and above the pivot row) at each step
• Normalizes all rows by their pivot
–Elimination leads to diagonal unitary matrix (identity): no back-substitution needed
–Number of Ops: about 50% more expensive than Gauss Elimination (n
3
/2 vs. n
3
/3
multiplications/divisions)
2.29
Numerical Fluid Mechanics PFJL Lecture 8, 7
7

Gauss Elimination: Multiple Right-hand Sides
A X = B
Reduction
Step k
Total Computation Count = ?
k
Reduction: Nr
Back Substitution: Nb
If n >> p, we expect Nr >> Nb
But, if n ~ p ? (next slide)
n-k p
n
k
X is a [n x p] matrix
. X=
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
8
8

Gauss Elimination: Multiple Right-hand Sides
Number of Ops
Reduction/Elimination: Step k
: n-k divisions
p equations as this one
n÷1
For reduction, the number of ops is: ¿
(2 p +1)( n ÷ k) + 2( n ÷ k)*( n ÷ k +1) =
k =1
3
| n
2
n |
3 2
2 + n ÷ ÷ ( ÷1) + pn n ( ÷1) = ( + pn
|
n n O n )
3 3
Back-Substitution
\ .
: p * ( (n-k-1)+(n-k)+2 )= p * ( 2(n-k) +1 ) add./mul./div.
n÷1
+ 2(n ÷ k) + 1 = + ( ÷1)( n +1) pn
Number of ops for back-substitution:
p p
¿
p p n =
2
k =1
(the first p before the sum is for the
evaluations of the p x
n’s
)
Grand total number of ops is (
3
+ pn
2
)
:
can be inefficient for large p
O n
Numerical Fluid Mechanics PFJL Lecture 8,
2.29
9
9

Gauss Elimination: Multiple Right-hand Sides
Number of Ops, Cont’d
Reduction
at end of step k
k
n-k p k
i. Reduction for each right-hand side can be inefficient if p >>>
ii. In addition, one may need to redo the Reduction each time if RHS is result
of iterations and unknown a priori
A x
1
= b
1
, A x
2
= b
2
, etc, where vector b
2
is a function of x
1
, etc
=> LU Factorization / Decomposition
n
Numerical Fluid Mechanics PFJL Lecture 8, 10
10
2.29

A
where
is an upper triangular matrix
1.
2.
LU Decomposition/Factorization:
LU Decomposition: Separates time-consuming elimination for A from that for b / B
The coefficient Matrix
is decomposed as
is a lower triangular matrix
and
Then the solution is performed in two simple steps
Forward substitution
Back substitution
How to determine
and
?
Numerical Fluid Mechanics PFJL Lecture 8, 11
11
2.29

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
j
Gauss Elimination (GE): iteration eqns.
for the reduction at step k are
This gives the final changes occurring
in reduction steps k = 1 to k = i-1
Unchanged after step i-1:
Below diagonal:
Become and remain 0 in step j:
i
After reduction step i-1:
Above and on diagonal:
Numerical Fluid Mechanics PFJL Lecture 8, 12
12
2.29

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
j
PFJL Lecture 8, 13 Numerical Fluid Mechanics
2.29
After reduction step i-1:
Above and on diagonal:
Below diagonal:
Unchanged after step i-1:
Become and remain 0 in step j:
i
Now, to evaluate the changes that
accumulated from when one starts the
elimination, let’s try to sum this iteration
equation, from:
• 1 to i-1 for above and on diagonal
• 1 to j for below diagonal
Gauss Elimination (GE): iteration eqns.
for the reduction at step k are
This gives the final changes occurring
in reduction steps k = 1 to k = i-1
As done in class, you can also sum up to
an arbitrary r and see which terms remain.
13

Below diagonal:
Unchanged after step i-1:
∑ these step-k eqns. from (k=1 to i-1) =>
Gives the total change above diagonal:
∑ this step-k eqns. from (k=1 to j) =>
Gives the total change below diagonal:
Gauss Elimination (GE): iteration eqns.
for the reduction at step k are
This gives the final changes occurring
in reduction steps k = 1 to k = i-1
i
After reduction step i-1:
Above and on diagonal:
LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
j
Become and remain 0 in step j:
Numerical Fluid Mechanics PFJL Lecture 8, 14
14
2.29

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
j
Change in reduction steps k = 1 to k = i-1 :
(1)
(2)
After reduction step i-1:
PFJL Lecture 8, 15 Numerical Fluid Mechanics
2.29
Above and on diagonal:
Below diagonal:
Unchanged after step i-1:
Become and remain 0 in step j:
We obtained: Total change above diagonal
We obtained: Total change below diagonal
Now, if we define:
and use them in equations (1) and (2) =>
i
15

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
Sum stops at diagonal
Result seems to be a ‘Matrix product’:
=
x
i
Below diagonal
k
Lower triangular Upper triangular
0
0
j
k
Above diagonal
=
x
i
k
0
0
k
j
Numerical Fluid Mechanics PFJL Lecture 8, 16
16
2.29

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
GE Reduction directly yields LU factorization
Compact storage:
no need for additional memory (the unitary
diagonal of L does not need to be stored)
Lower triangular
=
Upper triangular
=
Number of Operations for LU?
Lower diagonal implied
(referred to as the Doolittle decomposition)
Numerical Fluid Mechanics PFJL Lecture 8, 17
17
2.29

LU Decomposition / Factorization
via Gauss Elimination, assuming no pivoting needed
GE Reduction directly yields LU factorization
=
=
Lower triangular
Upper triangular
Compact storage
Lower diagonal implied
Number of Operations for LU?
Same as Gauss Elimination:
less in Elimination phase (no RHS operations), but more in double back-substitution phase
Numerical Fluid Mechanics PFJL Lecture 8, 18
18
2.29

Pivoting in LU Decomposition / Factorization
Before reduction, step k
Pivoting if
or else
To do this interchange of rows i and k,
Pivot element vector
Forward substitution, step k
Interchange rows i and k
use a pivot vector:
Dummy var.
( )
,
k
nn
a
If
Numerical Fluid Mechanics PFJL Lecture 8, 19
19
2.29

LU Decomposition / Factorization:
Variations
• Doolittle decomposition:
– m
ii
=1, stored in L
• Crout decomposition:
– Directly impose diagonal of U equal to 1’s.
– Sweeps both by columns and rows
– Reduce storage needs
– Each element of A only employed once
• Matrix inverse: AX=I => (LU)X=I
| |
3 3 3
2n
2 2
| 2n
3
8n
– Numbers of ops: O + pn + pn for p = n, => + 2n =
  |
3 3 3

Forward Backward
|
\
LU Decomp. Substitution Substitution
.
Numerical Fluid Mechanics PFJL Lecture 8, 20
20
2.29

Recall Lecture 2: The Condition Number
• The condition of a mathematical problem relates to its
sensitivity to changes in its input values
• A computation is numerically unstable if the
uncertainty of the input values are magnified by the
numerical method
• Considering x and f(x), the condition number is the
ratio of the relative error in f(x) to that in x.
( ) x = x ' x x ÷ ) • Using first-order Taylor series f f ( ) + f ( )( x
f ( ) x ÷ f ( ) f ( )( x ' x x ÷ x)
• Relative error in f(x):
~
f x f x ( ) ( )
x '( ) f x
• Condition Nb = Ratio of relative errors:
K =
p
f ( ) x
Numerical Fluid Mechanics PFJL Lecture 8, 21
21
2.29

x
Linear Systems of Equations
Error Analysis
Linear systems
Function of one variable
The condition number K is a
measure of the amplification of the
relative error by the function f(x)
Condition number
How is the relative error of
dependent on errors in b?
Example
Using MATLAB with different ’s (see tbt8.m): b
Small changes in b give large changes in x
The system is ill-Conditioned
Numerical Fluid Mechanics PFJL Lecture 8, 22
22
2.29

Linear Systems of Equations: Norms
Vector and
Matrix Norms:
Evaluation of
Condition Numbers
Properties:
requires use of Norms
Sub-multiplicative /
Associative Norms
(Banach Algebra/space)
Numerical Fluid Mechanics PFJL Lecture 8, 23
23
2.29

Examples of Matrix Norms
“Maximum Column Sum”
“Maximum Row Sum”
“The Frobenius norm” (also called Euclidean
norm)”, which for matrices differs from:
“The l-2 norm” (also called spectral norm)
Numerical Fluid Mechanics PFJL Lecture 8, 24
24
2.29

Linear Systems of Equations
Error Analysis: Perturbed Right-hand Side
Vector and Matrix Norms
Perturbed Right-hand Side implies
Properties
Subtract original equation
Relative Error Magnification
Condition Number
Numerical Fluid Mechanics PFJL Lecture 8, 25
25
2.29

Linear Systems of Equations
Error Analysis: Perturbed Coefficient Matrix
Vector and Matrix Norm
Perturbed Coefficient Matrix implies
PFJL Lecture 8, 26 Numerical Fluid Mechanics
2.29
Properties
Subtract unperturbed equation
Relative Error Magnification
Condition Number
(Neglect 2
nd
order)
26

Using
Cramer’s
rule:
Example: Ill-Conditioned System
n=4
a = [ [ 1. 0 1. 0] ' [ 1. 0 1. 0001] ' ]
b= [ 1 2] '
ai =i nv( a) ;
a_nr m=max( abs( a( 1, 1) ) + abs( a( 1, 2) ) ,
abs( a( 2, 1) ) + abs( a( 2, 2) ) )
ai _nr m=max( abs( ai ( 1, 1) ) + abs( ai ( 1, 2) ) ,
abs( ai ( 2, 1) ) + abs( ai ( 2, 2) ) )
k=a_nr m*ai _nr m
r =ai * b
x=[ 0 0] ;
m21=a( 2, 1) / a( 1, 1) ;
a( 2, 1) =0;
a( 2, 2) = r add( a( 2, 2) , - m21*a( 1, 2) , n) ;
b( 2) = r add( b( 2) , - m21*b( 1) , n) ;
x( 2) = b( 2) / a( 2, 2) ;
x( 1) = ( r add( b( 1) , - a( 1, 2) *x( 2) , n) ) / a( 1, 1) ;
x'
tbt6.m
Ill-conditioned system
4-digit Arithmetic
Numerical Fluid Mechanics PFJL Lecture 8, 27
27
2.29

Using
Cramer’s
rule:
Relatively Well-conditioned system
Example: Better-Conditioned System
n=4
a = [ [ 0. 0001 1. 0] ' [ 1. 0 1. 0] ' ]
b= [ 1 2] '
ai =i nv( a) ;
a_nr m=max( abs( a( 1, 1) ) + abs( a( 1, 2) ) ,
abs( a( 2, 1) ) + abs( a( 2, 2) ) )
ai _nr m=max( abs( ai ( 1, 1) ) + abs( ai ( 1, 2) ) ,
abs( ai ( 2, 1) ) + abs( ai ( 2, 2) ) )
k=a_nr m*ai _nr m
r =ai * b
x=[ 0 0] ;
m21=a( 2, 1) / a( 1, 1) ;
a( 2, 1) =0;
a( 2, 2) = r add( a( 2, 2) , - m21*a( 1, 2) , n) ;
b( 2) = r add( b( 2) , - m21*b( 1) , n) ;
x( 2) = b( 2) / a( 2, 2) ;
x( 1) = ( r add( b( 1) , - a( 1, 2) *x( 2) , n) ) / a( 1, 1) ;
x'
tbt7.m
4-digit Arithmetic
Numerical Fluid Mechanics PFJL Lecture 8, 28
28
2.29

MIT OpenCourseWare
http://ocw.mit.edu
2.29 Numerical Fluid Mechanics
Fall 2011