Professional Documents
Culture Documents
1
Introduction
Nonlinear Equations
a11 x1 a12 x1 x2 a1n ( xn ) 5 b1
a21 ( x1 ) 3 a 22e x2 a2 n ( x2 ) 3 / xn b2
a x a x a x b
n1 1 n2 2 nn n n 3
Review of Matrices
a 11 a 12 a 1m
a a 2m
a 22 2nd row Elements are indicated by a
[A] ij
21
a n1 a n2 a nm n m
row column
mth column
Square matrix:
c m m 1
- [A]nxm is a square matrix if n=m.
- A system of n equations with n unknonws has a square
coefficient matrix.
4
Review of Matrices
• Symmetric matrix:
If aij = aji [A]nxn is a symmetric matrix
• Diagonal matrix:
[A]nxn is diagonal if aij = 0 for all i=1,...,n ; j=1,...,n
and ij
• Identity matrix:
[A]nxn is an identity matrix if it is diagonal with aii=1
i=1,...,n . Shown as [I]
5
Review of Matrices
• Upper triangular matrix:
[A]nxn is upper triangular if aij=0 i=1,...,n ; j=1,...,n and i>j
• Lower triangular matrix:
[A]nxn is lower triangular if aij=0 i=1,...,n ; j=1,...,n and i<j
• Inverse of a matrix:
[A]-1 is the inverse of [A]nxn if [A]-1[A] = [I]
• Transpose of a matrix:
[B] is the transpose of [A]nxn if bij=aji Shown as [A] or [A]T
6
Special Types of Square Matrices
5 1 2 16 a11 1
1 3 7 39 1
[ A] [ D]
a 22 [I ]
2 7 9 6 1
16 39 6 88 a nn 1
Symmetric Diagonal Identity
a11 a12 a1n a11
a a
[ A]
a 22 a 2n [ A] 21 22
a nn n1
a a nn
Upper Triangular Lower Triangular
7
Review of Matrices
• Matrix multiplication:
r
Note: [A][B] [B][A] cij aik bkj
k 1
8
Review of Matrices
a11 a12
For example A augmented with the
21
a a 22
• Determinant of a matrix:
A single number. Determinant of [A] is shown as |A|.
9
Chapter 9
Gauss Elimination
10
Solving Small Numbers of Equations
There are many ways to solve a system of linear
equations:
• Graphical method
• Cramer’s rule For n ≤ 3
• Method of elimination
• Numerical methods for solving larger number of
linear equations:
- Gauss elimination (Chap.9)
- LU decompositions and matrix inversion(Chap.10)
11
1. Graphical method
• For two equations (n = 2):
a11 x1 a12 x2 b1
a21 x1 a22 x2 b2
• Solve both equations for x2: the intersection of the lines
presents the solution.
a11 b1
x2
x1 x2 (slope)x1 intercept
a12 a12
a b
x2 21 x1 2
a22 a22
• For n = 3, each equation will be a plane on a 3D coordinate
system. Solution is the point where these planes intersect.
• For n > 3, graphical solution is not practical.
12
Graphical Method -Example
• Solve:
3x1 2 x2 18
x1 2 x2 2
• Plot x2 vs. x1, the
intersection of the
lines presents the
solution.
13
Graphical Method
Mathematically
• Coefficient matrices of (a) & (b) are singular.
There is no unique solution for these systems.
Determinants of the coefficient matrices are
zero and these matrices can not be inverted.
A.x B
Where [A] is the coefficient matrix:
18
3. Method of Elimination
• The basic strategy is to successively solve one of
the equations of the set for one of the
unknowns and to eliminate that variable from
the remaining equations by substitution.
19
Elimination of Unknowns Method
2.5x1 + 6.2x2 = 3.0
Given a 2x2 set of equations:
4.8x1 - 8.6x2 = 5.5
22
Naive Gauss Elimination Method
Step 1 : Forward Elimination: Reduce the system to an
upper triangular system.
23
Naive Gauss Elimination Method (cont’d)
Note:
- In these steps the 1st eqn is the pivot equation and a11 is
the pivot element.
- Note that a division by zero may occur if the pivot
element is zero. Naive-Gauss Elimination does not check
for this.
a11 a12 a13 a1n x 1 b1
0 a a23 a2n x 2 b2
22
The modified system is 0 a32 a33 a3n x 3 b3
indicates that the 0 an2 an3 ann x n bn
system is modified once.
24
Naive Gauss Elimination Method (cont’d)
0 0 a33
a3n x 3 b3
0 0 an3 ann
xn bn
Repeat steps (1.1) and (1.2) upto (1.n-1).
j i 1
xi ( i 1)
for i n-1, n-2, ..., 1
a ii
26
Summary of Naive Gauss Elimination Method
27
Pseudo-code of Naive Gauss Elimination Method
k,j
28
Naive Gauss Elimination Method
Example 1
Solve the following system using Naive Gauss Elimination.
6 –2 2 4 | 16
12 –8 6 10 | 26 R2-2R1
3 –13 9 3 | -19 R3-0.5R1
-6 4 1 -18 | -34 R4-(-R1)
29
Naive Gauss Elimination Method
Example 1 (cont’d)
Step 1: Forward elimination
30
Naive Gauss Elimination Method
Example 1 (cont’d)
Step 2: Back substitution
Find x4 x4 =(-3)/(-3) = 1
Find x3 x3 =(-9+5*1)/2 = -2
Find x2 x2 =(-6-2*(-2)-2*1)/(-4) = 1
Find x1 x1 =(16+2*1-2*(-2)-4*1)/6= 3
31
Naive Gauss Elimination Method
Example 2
(Using 6 Significant Figures)
3.0 x1 - 0.1 x2 - 0.2 x3 = 7.85
0.1 x1 + 7.0 x2 - 0.3 x3 = -19.3 R2-(0.1/3)R1
0.3 x1 - 0.2 x2 + 10.0 x3 = 71.4 R3-(0.3/3)R1
32
Naive Gauss Elimination Method
Example 2 (cont’d)
x3 = 7.00003
x2 = -2.50000
x1 = 3.00000
Exact solution:
x3 = 7.0
x2 = -2.5
x1 = 3.0
33
Gauss elimination (operation counting)
Forward Elimination
n
Inner loop:
1 = n - (k +1) +1 = n - k
j=k+1
n
Second loop: 2 + (n - k) (2 n) k (n k)
i=k+1
34
Gauss elimination (operation counting)
n 1 n 1 n 1
(n 2 2n)
k 1
1 2(n 1)
k 1
k
k 1
k2
(n 1))(n)
(n 2n)(n 1) 2(n 1)
2
2
(n 1)(n)(2n 1) n3
= + O(n2 )
6 3
35
Gauss elimination (operation counting)
Back Substitution
n
Inner Loop: 1 n (i 1) 1 n - i
ji 1
n 1 n 1 n 1
Outer Loop:
1 (n i) (1 n)1 i
i 1 i 1 i 1
(n 1)n
(1 n)(n 1)
2
n2
= + O (n)
2
36
Gauss elimination (operation counting)
Grand Total: the entire effort requires n3/3 + O(n2) flops altogether.
37
Pitfalls of Gauss Elimination Methods
1. Division by zero
2 x2 + 3 x3 = 8
a11 = 0
4 x1 + 6 x2 + 7 x3 = -3
(the pivot element)
2 x1 + x2 + 6 x3 = 5
38
Pitfalls of Gauss Elimination (cont’d)
3. Ill-conditioned systems
x1 + 2x2 = 10
1.1x1 + 2x2 = 10.4 x1 = 4.0 & x2 = 3.0
x1 + 2x2 = 10
1.05x1 + 2x2 = 10.4
x1 = 8.0 & x2 = 1.0
39
Pitfalls of Gauss Elimination (cont’d)
4. Singular systems.
• When two equations are identical, we would loose one
degree of freedom and be dealing with case of n-1
equations for n unknowns.
40
Techniques for Improving Solutions
x2
2 2.0001 3(2 / 3)
x1
3 0.0003
Significant
Figures x2 x1
3 0.667 -3.33
4 0.6667 0.000
5 0.66667 0.3000
6 0.666667 0.33000
7 0.6666667 0.333000
43
Partial Pivoting
44
Pivoting: Example
45
Scaling: Example
48
Example: Gauss Elimination
2x 2 x 4 0
2x 1 2x 2 3x 3 2x 4 2
4x 1 3x 2 x 4 7
6x 1 x 2 6x 3 5x 4 6
a) Forward Elimination
0 2 0 1 0 6 1 6 5 6
2 2 3 2 2 2 2 3 2 2
R 4
R 1
4 3 0 1 7 4 3 0 1 7
6 1 6 5 6 0 2 0 1 0
50
Example: Gauss Elimination (cont’d)
6 1 6 5 6
2 2 3 2 2 R 2 0.33333 R 1
4 3 0 1 7 R 3 0.66667 R 1
0 2 0 1 0
6 1 6 5 6
0 1.6667 5 3.6667 4
R3
R 2
0 3.6667 4 4.3333 11
0 2 0 1 0
6 1 6 5 6
0 3.6667 4 4.3333 11
0 1.6667 5 3.6667 4
0 2 0 1 0 51
Example: Gauss Elimination (cont’d)
6 1 6 5 6
0 3.6667 4 4.3333 11
0 1.6667 5 3.6667 4 R 3 0.45455 R 2
0 2 0 1 0 R 4 0.54545 R 2
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 2.1818 3.3636 5.9999 R 4 0.32000 R 3
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 0 1.5600 3.1199
52
Example: Gauss Elimination (cont’d)
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 0 1.5600 3.1199
b) Back Substitution
3.1199
x4 1.9999
1.5600
9.0001 5.6364 1.9999
x3 0.33325
6.8182
11 4.3333 1.9999 4 0.33325
x2 1.0000
3.6667
6 5 1.9999 6 0.33325 11.0000
x1 0.50000 53
6
Gauss-Jordan Elimination
54
Gauss-Jordan Elimination
Diagonalization by both forward and backward elimination
in each column. a11 a12 a13 ... a1n x1 b1
a a 23 ... a 2n x 2 b 2
21 a 22
a 31 a 32 a 33 ... a 3n x 3 = b3
...
a n1 a n2 a n3 ... a nn x nn b n
Perform elimination both backwards and forwards until:
1 0 0 0 x1 x1
0 1 0 0 x x
2 2
0 0 1 0 x3 = x3
0 0 0 1 x n x n
3
Operation count for Gauss-Jordan is: n
O(n 2 )
(slower than Gauss elimination) 2
55
Gauss-Jordan Elimination- Example
0 2 0 1 0 1 0.16667 1 0.83335 1
R 4
2 2 3 2 2 R 1
2 2 3 2 2
4 3 0 1 7 R 4 / 6.0 4 3 0 1 7
6 1 6 5 6 0 2 0 1 0
1 0.16667 1 0.83335 1
2 2 3 2 2 R 2 2 R 1
4 3 0 1 7 R 3 4 R 1
0 2 0 1 0
56
1 0.16667 1 0.83335 1
0 1.6667 5 3.6667 2
0 3.6667 4 4.3334 7
0 2 0 1 0
Dividing the 2nd row by 1.6667 and reducing the second
column. (operating above the diagonal as well as below)
gives:
1 0 0 0 0.49999
0 1 0 0 1.0001
0 0 1 0 0.33326
0 0 0 1 1.9999
60
Gauss-Jordan Matrix Inversion
Gauss-Jordan Matrix Inversion (with 2 digit arithmetic):
CHECK:
[ A ] [ A ]-1 = [ I ]
[ A ]-1 { b } = { x }
62