EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

1 Number of Operations (25 points)

A. Hybrid Method

Solution
After forward elimination with Gauss elimination method, the matrix becomes
 
a11 a12 · · · a1,n−1 a1n b1
 0 a22 · · · a2,n−1 a2n b2 
 
 .. .. .. .. .. ..  ,
 . . . . . .  (1)
 
 0 0 · · · an−1,n−1 an−1,n b3 
0 0 ··· 0 ann bn

To eliminate all elements above the diagonal,

(1) The first step is to calculate the coefficient an−1,n /an,n . This is 1 multiplication.
(2) Multiply this coefficient onto bn . This is 1 multiplication.
(3) Subtract the resultant value from the (n − 1)th row. This requires 1 addition. The total is now 2
miltiplications and 1 addition.
(4) Repeat (1) for the (n − 2)th row, and so on to the 1st row. Once all an−1,n to a1n are eliminated,
the above 3 steps are repeated (n − 1) times. The total operations are then 2(n − 1) multiplications
and n − 1 additions.
(5) Repeat (1) to (4) for columns n − 1 to 2. From column n to column 2, the number of multiplications
decreases from 2(n − 1) to 2, and the number of additions decreases from n − 1 to 1, because of the
decrease of the number of elements to eliminate in each column.
The total operations are then
n
X
2(m − 1) = n2 − n multiplications, (2)
m=2

and
n
X 1
(m − 1) = n(n − 1) additions. (3)
m=2
2

The final solution is obtained from (??) by dividing b0i with the coefficient in the same row, a0ii . The
total operations needed are n multiplications. Since the forward elimination requires 61 n(2n2 + 3n − 5)
multiplications and 31 (n − 1)n(n + 1) additions (see (1) and (2) in Lecture07), the total operations are

1 1 3 5
n(2n2 + 3n − 5) + n2 − n + n = n3 + n2 − n multiplications, (4)
6 3 2 6
and
1 1 n3 n2 5
n(n − 1) + (n − 1)n(n + 1) = + − n additions. (5)
2 3 3 2 6
The following tables lists the number of operations for the three methods. Gauss elimination has
1 3 1
n + n2 − n multiplications, (6)
3 3
and
1 3 1 2 5
n + n − n additions. (7)
3 2 6

1
EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

and Gauss-Jordan elimination has

1 2
n (n + 1) multiplications, (8)
2
and
1
(n − 1)n(n + 1) additions. (9)
2

Gauss Gauss-Jordan Hybrid

n
× + × + × +
3 17 11 18 12 20 11
10 430 375 550 495 475 375
50 44,150 42,875 63,750 62,475 45,375 42,875
100 343,300 338,250 505,000 499,950 348,250 338,250

B. Gauss and Gauss-Jordan Eliminations

a. Gauss Elimination
Follow the same procedure in the Lecture07,
• The first step is to calculate the coefficient a21 /a11 . This is 1 multiplication.
• Multiply this coefficient onto all coefficients in the 1st row except a11 , and onto the right-hand-side
term b11 to b1m . There are n − 1 + m such multiplications.
• Subtract the resultant coefficients and the RHS from the 2nd row coefficients. There is no need
to calculate the first term since it is zero by definition. This requires n − 1 + m additions.
• Repeat this for the 3rd, 4th row and so on, until the nth row. Therefore the above procedure is
repeated (n − 1) times.
• After the first column is eliminated, total operations required are (n − 1)(n + m) multiplications
and (n − 1)(n − 1 + m) additions.
• To eliminate the coefficients in the 2nd column below the 2nd row, the procedure is exactly
the same as above but the size of the coefficient matrix is reduced by one. Therefore the total
operations are (n − 2)(n − 1 + m) multiplications and (n − 2)(n − 2 + m) additions.
The total forward elimination process then requires
n
X 1
(i + m)(i − 1) = (n − 1)n(3m + 2n + 2) multiplications, (10)
i=2
6

and
n
X 1
(i − 1 + m)(i − 1) = (n − 1)n(3m + 2n − 1) additions. (11)
i=2
6
For the backward substitution,
• Calculating xn for each bn,i , where i = 1, · · · , m, requires m multiplication.
• Substituting xn into the (n − 1)th equation, and solving for xn−1 require two multiplications
and one addition for each bn−1,i , where i = 1, · · · , m. The total is 2m multiplications and m
additions.
• Solving for xn−2 requires 3m multiplications and 2m additions.
The total operations for backward substitution are
n
X m
m i= n(n + 1) multiplications, (12)
i=1
2

2
EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

and
n
X m
m (i − 1) = n(n − 1) additions. (13)
i=1
2
The total multiplications/divisions are then
1 m 1 1
(n − 1)n(3m + 2n + 2) + n(n + 1) = n3 + mn2 − n. (14)
6 2 3 3
The total addition/subtractions are
1 m 1 1 1
(n − 1)n(3m + 2n − 1) + n(n − 1) = n3 + mn2 − n2 − mn + n. (15)
6 2 3 2 6
b. Gauss-Jordan Elimination
The numbers of operations include
• Normalize a11 by dividing the first row by a11 from a12 to a1n and b11 to b1m . There are n−1+m
multiplications.
• Multiply a21 onto the 1st row elements except a11 . There are n − 1 + m multiplications
• Subtract the resultant 1st row from the 2nd row, n − 1 + m additions
• Repeat for the subsequent rows, each has n − 1 + m multiplications and n − 1 + m additions
The total operations for eliminating the 1st column include n(n − 1 + m) multiplications and (n −
1)(n − 1 + m) additions. To normalize the 2nd row, there are (n − 2 + m) multiplications since the
first element is already zero. Eliminating 2nd column then takes n(n − 2 + m) multiplications and
(n − 1)(n − 2 + m) additions. The total operations are
n
X 1 2 1 1
n (i − 1 + m) = n (n + 2m − 1) = n3 + mn2 − n2 multiplications, (16)
i=1
2 2 2

and
n  
X 1 1 1
(n − 1) (i − 1 + m) = n(n − 1)(n + 2m − 1) = n3 + (m − 1)n2 + − m n additions. (17)
i=1
2 2 2

2 Direct Elimination (25 points)

a. Gauss elimination without pivoting  
−2 3 1 9
 3 4 −5 0 
1 −2 1 −4
R2 − 3/(−2)R1 → R2
R3 − 1/(−2)R1 → R3  
−2 3 1 9
 0 17/2 −7/2 27/2 
0 −1/2 3/2 1/2
R3 − (−1/17)R2 → R3  
−2 3 1 9
 0 17/2 −7/2 27/2 
0 0 22/17 22/17
Backward substitution gives
x3 = (22/7)/(22/7) = 1

3
EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

,
x2 = [27/2 − (−7/2)(1)]/(17/2) = 2
x1 = [9 − (1)(1) − (3)(2)]/(−2) = −1

b. Gauss-Jordan elimination  
−2 3 1 9
 3 4 −5 0 
1 −2 1 −4
R1 /(−2) → R1  
1 −3/2 −1/2 −9/2
 3 4 −5 0 
1 −2 1 −4
R2 − 3R1 → R2
R3 − R1 → R3  
1 −3/2 −1/2 −9/2
 0 17/2 −7/2 27/2 
0 −1/2 3/2 1/2
R2 /(17/2) → R2  
1 −3/2 −1/2 −9/2
 0 1 −7/17 27/17 
0 −1/2 3/2 1/2
R1 − (−3/2)R2 → R1
R3 − (−1/2)R2 → R3  
1 0 −2 −3
 0 1 −7/17 27/17 
0 0 22/17 22/17
R3 /(22/17) → R3  
1 0 −2 −3
 0 1 −7/17 27/17 
0 0 1 1
R1 − (−2)R3 → R1
R2 − (−7/17)R3 → R2  
1 0 0 −1
 0 1 0 2 
0 0 1 1

c. Doolittle LU factorization  
−2 3 1
U= 0 17/2 −7/2 
0 0 22/17
 
1 0 0
L =  −3/2 1 0 
−1/2 −1/17 1
Solve
Lb0 = b (18)
 
1 0 0 9
 −3/2 1 0 0 
−1/2 −1/17 1 −4

4
EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

b01 = 9
b02 = 0 − (−3/2)b01 = 27/2
b03 = −4 − (−1/2)b01 − (−1/17)b02 = 4 + 9/2 + 27/34 = 22/17
Solve
Ux = b0 (19)
 
−2 3 1 9
 0 17/2 −7/2 27/2 
0 0 22/17 22/17
22/17
x3 = =1
22/17
27/2 + (7/2)(1)
x2 = =2
17/2
9 − (3)(2) − (1)(1)
x1 = = −1
−2

3 Frank Matrix (20 points)

The following table lists values of max |x − x0 | obtained with two different methods of solving Ax0 = b.

n LU Fac Gauss-Jordan
5 2.22 × 10−14 6.66 × 10−15
10 1.27 × 10−10 3.79 × 10−10
15 1.58 × 10−5 1.26 × 10−4
25 2.82 × 103 3.54 × 102
The following table lists values of the condition numbers obtained with different definitions of norm
qP P
n 1-norm (max column sum) ∞-norm (max row sum) Frobenius norm a2ij
5 8.91 × 102 1.01 × 103 6.90 × 102
10 3.84 × 107 4.50 × 107 3.18 × 107
15 1.80 × 1013 2.25 × 1013 1.55 × 1013
25 1.34 × 1018 1.77 × 1018 1.18 × 1018

4 Nonlinear System of Equations (30 points)

The equations are
f (x1 , x2 , x3 ) = x31 − 2x1 x2 + x3 = 0,
g(x1 , x2 , x3 ) = x1 x22 − x1 x3 − 5 = 0, (20)
h(x1 , x2 , x3 ) = −x1 + x2 x3 + x33 = 0.

The linear equations for ∆x = (∆x1 , ∆x2 , ∆x3 ) are

∂f ∂f ∂f
∆x1 + ∆x2 + ∆x3 = −f,
∂x1 ∂x2 ∂x3
∂g ∂g ∂g
∆x1 + ∆x2 + ∆x3 = −g, (21)
∂x1 ∂x2 ∂x3
∂h ∂h ∂h
∆x1 + ∆x2 + ∆x3 = −h,
∂x1 ∂x2 ∂x3

5
EP501 Numerical Methods Homework 3 Due Oct 11, 2013 11:59 pm

or
(3x21 − 2x2 )∆x1 −2x1 ∆x2 +∆x3 = −f,
(x22 − x3 )∆x1 +2x1 x2 ∆x2 −x1 ∆x3 = −g, (22)
−∆x1 +x3 ∆x2 +(x2 + 3x23 )∆x3 = −h.
With the initial approximation x = (x1 , x2 , x3 ) = (1, 1, −1), it becomes

∆x1 −2∆x2 +∆x3 = 2,

2∆x1 +2∆x2 −∆x3 = 3, (23)
−∆x1 −∆x2 +4∆x3 = 3.

The solution is ∆x1 = 5/3, ∆x2 = 10/21, ∆x3 = 9/7. Substituting this into (20) and (22) will yield a new
set of (23) with different coefficients. The iteration continues until the error threshold is reached.
The final solution is
x1 = 1.82371704897354,
x2 = 1.86862473395972,
x3 = 0.75010502266191.
A plot of kxk2 versus iteration steps shows the iteration converges after 8 steps. The decrease of the error
is faster than a straight line so the rate of convergence is faster than linear.