Professional Documents
Culture Documents
Numerical A. Note
Numerical A. Note
CHAPTER ONE
Introduction
Numerical technique is widely used by scientists and engineers to solve their problems. A major
advantage for numerical technique is that a numerical answer can be obtained even when a problem
has no analytical solution. However, result from numerical analysis is an approximation, in general,
which can be made as accurate as desired. The reliability of the numerical result will depend on an
error estimate or bound, therefore the analysis of error and the sources of error in numerical
methods is also a critically important part of the study of numerical technique.
1.1. Errors and Approximations in Computation
A computer has a finite word length and so only a fixed number of digits are stored and used
during computation. This would mean that even in storing an exact decimal number in its
converted form in the computer memory, an error is introduced. This error is machine
dependent and is called machine epsilon. In general, we can say that
= − .
By Natnael W. Page 1
Numerical Methods
1.2.4. Absolute error: is the numerical difference between the true value of a quantity and its
′ ′
approximate value. Thus if is the approximate value of quantity then − is called
′
the absolute error and denoted by . Therefore = − . The unit of exact or unit of
approximate values expresses the absolute error.
′
′
1.2.5. Relative Error: The relative error defined by = = . Where is the
By Natnael W. Page 2
Numerical Methods
′
1.2.6. Percentage error: The percentage error in which is the approximate value of is given
′
by = 100 ∗ = 100 ∗ . The percentage error is also independent of units.
Example: Find the absolute, relative and percentage errors if is rounded-off to three
decimal digits. Given = 0.005998
Solution: If is rounded-off to three decimal places we get = 0.006. Therefore,
= −
= 0.005998 − 0.006 = −0.000002.
= =| | = 0.000002
.
= = = = 0.0033344 and
.
= = ∗ 100 = 0.33344.
Example: If = 0.51 and correct to two decimal places then ∆ = 0.005 and the relative
.
accuracy is given by × 100 ≅ 0.98%
.
Exercise:
′
a. An approximate value of is given by = = 3.1428571 and its true
i. − +
ii. −
By Natnael W. Page 3
Numerical Methods
1.3. General Formula for errors
In this section, we derive a general formula for the error committed in using a certain formula for a
functional relation. Let = ( , ,…, ) -------------------------------------------------------------------- (1)
be the function of several variables ( , ,…, ) , and let the error in each be ∆ . Then the error ∆ in
is given by +∆ = ( +∆ , + ∆ ,…, +∆ ) ---------------------------------------------- (2)
Using Taylor’s series for more than two variables, to expand the R.H.S of above, we get
+∆ = ( , ,…, )+ ∆ +∆ +⋯+ ∆ +
(∆ ) + (∆ ) + ⋯ + (∆ ) --------------------------------------------------- (3)
∆
Assuming the errors ∆ , ∆ , … . , ∆ in all are small and that ≪ 1 so that the terms containing
Equation (5) represents the general formula for Errors. If equation (5) divided by we get relative error
∆ ∆ 1 ∆ 2 ∆
= = + +⋯+ ---------------------------------------------------------------------- (6)
1 2
Also from equation (5), by taking modulus we get maximum absolute error.
By Natnael W. Page 4
Numerical Methods
that when the given numbers are added then the magnitude of absolute error is the sum of
the magnitudes of the +∆ =( +∆ )−( +∆ )
or +∆ =( − ) + (∆ −∆ )
∆ ∆ 1 ∆ 2
∴ Absolute error is given by ∆ = ∆ −∆ and Relative error is = − .
∆ ∆ 1 ∆ 2
But we know that |∆ | ≤ |∆ 1 | + |∆ 2 | and ≤ + therefore on taking
modulus of relative errors and absolute errors to get its maximum value, we have
∆ ∆ 1 ∆ 2
|∆ | ≤ |∆ 1 | + |∆ 2 | which is the maximum absolute error and ≤ + which
1 … 1 1 … 1 ∆ … 1
Now = = , = = ,…, = = therefor
1 … 1 2 … 2 …
∆ ∆ 1 ∆ 2 ∆
= + + ⋯+
1 2
∆ ∆ ∆ ∆
⇒ = ≤ + +⋯+ and
∆ ∆
= = ×( … )
∆ =∆ 1 +∆ 2 +⋯+∆
1 2
∆ ∆ 1 ∆ 2 ∆ 1 1 ∆ 2 1 ∆ 1 ∆ 2
We have = + = 1 × + 1 × − = −
1 2 2 2 1 2
2 2
∆ ∆ ∆
Therefore, ( )= ≤ + and
∆
( ) = |∆ | ≤
By Natnael W. Page 5
Numerical Methods
5 2 10 −15 2
Solution: = 3 , = 3 , = 4 and
∆ =∆ +∆ +∆
In general, the errors ∆ , ∆ , ∆ may be negative or positive and hence we take the absolute
values of the terms of the right side.
This gives, (∆ ) = ∆ + ∆ + ∆
Now let ∆ = ∆ = ∆ = 0.001 and = = = 1 then the relative maximum error is given by
(∆ ) .
( ) . = = = 0.006
Example: Evaluate the sum = √3 + √5 + √7 to four significant digits and find its absolute and
relative errors.
Solution: We have √3 = 1.732, √5 = 2.236, √7 = 2.646 hence = 6.614
= 0.0005 + 0.0005 + 0.0005 = 0.0015
The total absolute error shows that the sum is correct to three significant figures only.
.
Hence we take = 6.61 and thus = = 0.0002.
.
Example: The percentage error in which is given by = + , is not allowed to exceed 0.2%, find
By Natnael W. Page 6
Numerical Methods
∆
Solution: The percentage error in = × 100 = 0.2
. . . . . × .
Therefore ∆ = × = × + =
× .
. × .
∆ ∆ ∆ (∆ ) ( . )
Percentage in = × 100 = × = × = = = 0.12
( . )
Exercise:
a. If =3 − 6 find the in at = 1, if the error in is 0.05.
b. Define the term absolute error in = 10.00 ± 0.05, = 0.0356 ± 0.0002
= 15300 ± 100 = 62000 ± 500 Find the Maximum value of in
i. + + +
ii. +5 −
iii.
c. Find the product of the numbers 56.54 and 12.4 which both correct to the significant digits
given.
d. Find the quotient = , where = 4.536 and = 1.32 both and being correct to the
By Natnael W. Page 7
Numerical Methods
CHAPTER TWO
Numerical Solutions of Non-linear Equations
Method for finding the root of an equation can be classified as direct and Iterative methods.
i. Direct method: in some cases, roots can be found by using direct/analytical methods.
For a quadratic equation + + = 0 the roots of the equation obtained by
√ √
= and = . These are called closed form solution. Similar
formula also available for cubic and bi-quadratic polynomial equations but we rarely
remember them. For higher order polynomial equations and non-polynomials it is
difficult and in many cases impossible to get closed form solutions.
ii. Iterative method: those methods also known as trial and error methods are based on
the idea of successive approximation starting with one or more initial approximation to
the value of the root to obtain the accurate solution. The above quadratic equation can
be found by iterative method in the following manner.
a. = , = 0,1,2, …
b. = , = 0,1,2, …
c. = , = 0,1,2, …
Step3: If ( ). ( ) < 0 then set = else = then apply the formula of step2.
By Natnael W. Page 8
Numerical Methods
Step4: stop evaluation when the difference of two successive values of obtained from step2, is numerically less
than the prescribed accuracy .
= ( )
( )
( )
If an error tolerance has been prescribed in advance, it is possible to determine the number of steps
required in the bisection method. Let be a root in ( , ) and | − |< implies < by
( ) ( )
taking logarithms (with any convenient base), we obtain > .
= ( ); = ( )
+
= ; = ( )
2
No
Is
. <0
Yes No
Is
− Yes Stop
= ; = = ; = <
2
By Natnael W. Page 9
Numerical Methods
Example: Using Bisection method find the real root of the equation ( ) = 3 − √1 + .
Solution: ( ) = 3 − √1 + = 0 is transdental equation then,
( ) = −1 and ( ) = 1.643
Therefore a root lies between 0 and 1.
1st approximation = = 0.5, (0.5) = 0.2837 > 0 so the root lies between 0 and 0.5.
.
2nd approximation = = 0.25, (0.25) = −0.3669 < 0 so the root lies between 0.25
and 0.5.
. .
3rdapproximation = = 0.375, (0.375) = −0.0439 < 0 so the root lies between
( = , ( ))
( = , ( ))
By Natnael W. Page 10
Numerical Methods
The secant line over the interval [ , ] is the chord between ( , ( )) and ( , ( )). The two right
angles in the figure are similar, which mean that = .
( ) ( )
( ) ( ) ( )( )
This implies that = ( )
= − ( )
; then we can compute ( ) and repeat the
( ) ( )
process with the interval [ , ], if ( ). ( ) < 0 or to the interval [ , ], if ( ). ( ) < 0.
Step1: Choose two initial guess values (approximations) and (where < ) such that
( ). ( ) < 0.
( ) ( )
Step2: Find the next approximation = ( )
; and also evaluate ( ).
( )
Step3: If ( ). ( ) < 0 set = then go to the next step if not, rename = and then go to the
next step.
( ) ( )
Step4: Evaluate the successive approximation using the formula = ( )
, = 2,3, …
( )
But before applying the formula for , ensure that whether ( ). ( ) < 0, if not rename as
and proceed.
Step5: stop the evaluation when | − | < , where is the prescribed accuracy.
Example: Find a real root of the equation ( ) = − 3 using Regula-falsi method correct to
three decimal places.
( ) = (1.0352) = −0.0852 thus, the root lies between 1.0352 and 1.5.
( ) ( )
2ndapproximation: = ( )
= 1.0456 and ( ) = (1.0456) = −0.0252
( )
By Natnael W. Page 11
Numerical Methods
In general = ( ), = 1,2,3, …
= ( )
= ( )
= ( )
= ( )
= ( ), = 1,2,3, …
Step4: Stop evaluation where relative error less than the prescribed accuracy .
′
Note: The iteration method = ( ) is convergent if ( ) < 1,
′ ′( ) > 1 or ′( ) < −1
When ( ) >1⟹
By Natnael W. Page 12
Numerical Methods
Example1: Find the root of the equation = 3 − 1 correct to three decimal places
Now the given equation can be rewrite as = ( + 1) = ( )(say) then we can check
= ( )= ( 0 + 1) = 0.667
...
= 0.6072 = 0.6071
Now, & being almost the same. Hence the required root is given by 0.6071.
method.
We get = ( )= = 0.81649
√ .
= ( )= = 0.7419, … , = ( ) = 0.75487
√ .
By Natnael W. Page 13
Numerical Methods
( )= Tangent line
= ( )
′( ( ) ( ) ( )
Form the figure = )= , ∴ − = ′( , ⇒ = − ′(
) )
( ) ( )
Still better approximation value = − ′( )
in general, = − ′( )
, = 0,1,2, ….
The other way of interpreting Newton’s method is the Taylor series at . Thus, we could write
( )
⇒ℎ=− ′( )
( )
⇒ − =− ′()
( )
⇒ = − ′( )
By Natnael W. Page 14
Numerical Methods
( )
Then, generally = − ′( )
, = 0,1,2, … . & lim ⟶∞ = where is the desired
root.
( )
step2: Find next (first) approximation by using the formula = − ′( )
.
step3: Follow the above procedure to find the successive approximation using the formula
( )
= − ′(
, = 1,2,3, ….
)
step4: Stop the process when | − |< where is the prescribed accuracy.
( )
= − ( )
= 0.53134337 , ( ) = −0.4180 × 10
( )
= − ( )
= 0.51790991 , ( ) = −0.4641 × 10
( )
= − ( )
= 0.51775738 , ( ) = −0.5926 × 10
( )
= − ( )
= 0.51775736 , ( ) = −0.2910 × 10
⇒ |−0.2910 × 10 | < 10
By Natnael W. Page 15
Numerical Methods
Secant line
= ( )
By Natnael W. Page 16
Numerical Methods
, , =
( − ) ( )
= −
( )− ( )
= +
By Natnael W. Page 17
Numerical Methods
Example: Find a real root of the equation ( ) = − 3 using Secant method correct to
three decimal places.
Solution: We have ( ) = − 3 = 0 assume = 1 and = 1.5 as initial guess values
then (1) = −0.2817 and (1.5) = 3.7225 therefore the root lies between 1 and 1.5.
( ) ( )
= − = 1.0352 , ( ) = (1.0352) = −0.0852
( ) ( )
( ) ( )
= − ( ) ( )
= 1.0456 , ( ) = (1.0456) = −0.0252
( ) ( )
= − ( ) ( )
= 1.0500 , ( ) = (1.0500) = 0.0005
( ) ( )
= − ( ) ( )
= 1.0499 , ( ) = (1.0499) = 0.0000
Exercise
1. Find the root of = correct to two decimal places using Bisection method.
2. Use the Bisection method to find a root of the equation − 4 − 9 in the interval (2, 3),
accurate to four decimal places.
3. How many number of iteration is needed is to find the solution of the equation
cos( ) − + 2 = 0 on the interval [0,1] having with an accuracy less than = 0.0001
4. Given these three equations
i. − − 10 = 0
ii. − =0
iii. +4 =0
Determine the initial approximation and use those to find the roots correct to three
decimal places with the following methods
a. Regula-Falsi method
b. Secant method
c. Newton Raphson method
5. Find the approximate value of (17) Using Newton Raphson method, correct to six decimal
places starting with = 2.
6. Find the root of the equations
i. − =1
ii. −3 − +2 =0
By Natnael W. Page 18
Numerical Methods
CHAPTER THREE
SOLVING SYSTEM OF EQUATIONS
3.1. Direct Methods
Consider a system of linear equations in unknowns.
+ + ⋯+ =
+ + ⋯+ =
------------------------------------------------------------------- (3.1)
⋮
+ +⋯+ =
Where ( , , = 1,2,3, … , ), are the known coefficients.
( , = 1,2,3, … , ) , are the known values
( , = 1,2,3, … , ) , are unknown to be determined
In the matrix notation, the system (3.1) can be written as
= ------------------------------------------------------------------------------------------------------ (3.2)
The matrix [ ∕ ] is called the augmented matrix. Where is the column matrix and is the
× matrix. Therefore, the system of equation = can be directly solved in the following
cases.
i. = , the equation (3.2) become
=
=
----------------------------------------------------------------- (3.3)
⋱ ⋮
=
∑
= where ≠ 0, = 1,2,3, … ,
This method of solving equation is called forward substitution method.
iii. = , the equation (3.2) become
By Natnael W. Page 19
Numerical Methods
+ + … + =
+ … + =
+ … + =
----------------------------------------- (3.5)
⋱ ⋮
, + , =
=
Solving for the unknowns in the in the order , , … , , we get
− ,
−∑
= , = ,
, … , =
By Natnael W. Page 20
Numerical Methods
1 −1 0
= −2 3 −4
−2 3 −3
1
0 0 −1
−3
Therefore, = = −2
−4 3 = 5 3
−2
−3 1 63
−3
Hence, the solution is = = 5
6
3.1.3. Gaussian Elimination Method
This is a method reduction of equation (3.1) in to an equivalent upper triangular system
which is then solved by back substitution method.
Consider the 3 × 3 system
+ + =
+ + = ---------------------------------------------------------------- (3.8)
+ + =
In this first stage of elimination: multiply the first row by & respectively
and subtract from the second and third rows. We get
( ) ( ) ( )
+ =
( ) ( ) ( )
----------------------------------------------------------------------------- (3.9)
+ =
( ) ( )
⎧ = − , = − ⎫
⎪ ( ) ( ) ⎪
Where, = − , = − -------------------------------------- (3.10)
⎨ ⎬
⎪ ( )
= − ,
( )
= − ⎪
⎩ ⎭
( )
Second stage of elimination: multiply the first row in (3.4) by ( ) and subtract from
By Natnael W. Page 21
Numerical Methods
Partial pivoting
In the 1st stage elimination: Search an element from the first column which is largest element
in magnitude and brought as the first pivot by interchanging the first equation with the
equation having largest element in magnitude, and the same for the 2nd stage elimination
among the − 1 elements leaving the first equation and the same for the 3rd stage
elimination among the − 2 elements leaving the first and second equations … and so on.
Additionally, to reduce round-off error; it is often necessarily to perform row interchanges
even when the pivot elements are not zero.
Example: Using the four decimal places computer, solve
0.729 + 0.81 + 0.9 = 0.6867
+ + = 0.8338
1.331 + 1.21 + 1.1 = 1.0000
Its exact solution rounded to four decimal places, is
= 0.2245 , = 0.2814 & = 0.3279
Solution without pivoting
0.729 0.81 0.9 0.6867
1 1 1 0.8338
1.331 1.21 1.1 1.0000
= = 1.372 and ← −
= = 1.826 & ← −
( )
= ( ) = 2.423 and ← −
By Natnael W. Page 22
Numerical Methods
↔ , = = 0.7513 and ← −
= = 0.5477 ← −
( )
↔ , = ( ) = 0.6171 and ← −
3 0.333 4
4 0.3333 1
5 0.33333 0.7
6 0.333333 0.67
7 0.3333333 0.667
8 0.33333333 0.6667
The exact solution of this problem is = and = .
Round-off errors can never be completely eliminated. However, they can be minimized
by using high precision arithmetic and pivoting.
By Natnael W. Page 23
Numerical Methods
b. System Condition
A system of equations = is said to be ill-conditioned or unstable if it is highly sensitive to
small changes in and i.e., small change in or causes a large change in the solution of
the system. On the other hand if small changes in and give small changes in the solution,
the system is said to be stable, or, well conditioned. Thus in an ill-conditioned system, even the
small round off errors effect the solutions very badly. Unfortunately it is quite difficult to
recognize an ill-conditioned system.
Example: Consider the following two almost identical systems
− =1 − =1
and
− 1.00001 = 0 − 0.99999 = 0
The respective solutions are: (100001, 100000) and (– 99999, – 100000)
Obviously the two solutions differ very widely as the result of small change in the value of .
Therefore, the system is ill conditioned.
+ + =1
4 +3 − =6
3 +5 +3 =4
1 1 1 0 0 1
4 3 −1 = 0 0 1
3 5 3 0 0 1
= + +
+ + +
By Natnael W. Page 24
Numerical Methods
=1, =1, =5
By Natnael W. Page 25
Numerical Methods
× + × = −5 ⇒ 4 = −12 ⇒ = −3
× + × + × = 83 ⇒ ( ) = 25 ⇒ =5
Now we have to solve = , that is,
2 0 0 14
1 4 0 = −101
7 −3 5 155
7
Solving the system, then the solution is = −27
5
At the second step, we have to solve = = , that is,
2 1 7 7
0 4 −3 = −27
0 0 5 5
3
Solving the system, then the solution is = −6
1
By Natnael W. Page 26
Numerical Methods
4 + + =2
+5 +2 = −6 Using Jacobi iteration method
+2 +3 = −4
( )
Take initial approximation as = [0.5, −0.5, −0.5] and perform three iterations in each case.
( ) ( ) ( )
⎧ =− + − 2 = 0.75
⎪ ( ) ( ) ( )
Solution: =− +2 + 6 = −1.1
⎨
⎪ ( )
=−
( )
+2
( )
+ 4 = −1.1667
⎩
( ) 1 ( ) ( )
⎧ =− + − 2 = 1.0667
⎪ 4
( ) 1 ( ) ( )
=− +2 + 6 = −0.8833
⎨ 5
⎪ ( ) 1 ( ) ( )
⎩ =− +2 + 4 = −0.8500
3
( ) 1 ( ) ( )
⎧ =− + − 2 = 0.9333
⎪ 4
( ) 1 ( ) ( )
=− +2 + 6 = −1.0733
⎨ 5
⎪ ( ) 1 ( ) ( )
⎩ =− +2 + 4 = −1.1000
3
By Natnael W. Page 27
Numerical Methods
3.2.2. Gauss-Seidel Method
We write the Gauss-Seidel method for the equation (3.1)
( ) ( ) ( ) ( )
=− + +⋯+ −
( ) ( ) ( ) ( )
=− + + ⋯+ −
------------------------------ (3.17)
⋮
( ) ( ) ( ) ( )
=− ( + +⋯+ , − )
Which may be rearrange in the form of
( ) ( )
= −∑ +
( ) ( ) ( )
+ = −∑ + ------------------------------------------------------- (3.18)
⋮
( ) ( ) ( )
+ +⋯+ , =
This process is continued till the values of , , … , are to the desire degree of accuracy.
The error in Gauss-Seidel method can be evaluated by ( ) = ( ) − ( ) .
Note: Gauss–Seidel method is a modification of Jacobi method. The convergence is Gauss–Seidel
method is more rapid than in Jacobi Method.
Note: In the absence of any better estimates, the initial approximations are taken as
( ) ( ) ( )
= 0, = 0,… , =0
By Natnael W. Page 28
Numerical Methods
This iteration process is continued. The results are tabulated as follows correcting to four decimal
places.
( ) ( ) ( )
By Natnael W. Page 29
Numerical Methods
Solution:
Step 1: Compute the characteristic polynomial
1− 2 3
det( − ) = 0 5− 6 = (1 − )[(5 − ) − 36]
0 6 5−
= (1 − )(−1 − )(11 − )
Step 2: The Eigen-values are = 1, = −1, = 11.
Step 3: Compute corresponding eigenvectors
For = 1, we have
1− 2 3 0 2 3
( − ) = 0 5− 6 = 0 4 6 =0
0 6 5− 0 6 4
1
⇒ = = 0
0
For = −1, we have
1− 2 3 2 2 3
( − ) = 0 5− 6 = 0 6 6 =0
0 6 5− 0 6 6
1
⇒ = = 2
−2
For = 11, we have
1− 2 3 −10 2 3
( − ) = 0 5− 6 = 0 −6 6 =0
0 6 5− 0 6 −6
1
⇒ = = 2
2
By Natnael W. Page 30
Numerical Methods
( )
3. Scale so that the unity component remains unity:
( )
= ( ) ( )
( )
4. Repeat steps 2 and 3 with = . Iterate to convergence. At convergence,
the value is the largest (in absolute value) eigenvalue of , and the vector
is the corresponding eigenvector (scaled to unity on the unity component).
The general algorithm for the power method is as follows:
( )
= ( ) = ( ) ( ) ------------------------------------------------------------- (3.23)
When the iterations indicate that the unity component could be zero, a different unity
component must be chosen. The method is slow to converge when the magnitudes (in
absolute value) of the largest eigenvalues are nearly the same. When the largest
eigenvalues are of equal magnitude, the power method, as described, fails.
Example: Find the largest (in absolute value) eigenvalue and the corresponding
eigenvector of the given matrix
8 −2 −2
= −2 4 −2
−2 −2 13
( )
Solution: Assume = [1.0 1.0 1.0]. Scale the third component to unity.
8 −2 −2 1.0 4.00 0.444444
( ) ( ) ( )
= −2 4 −2 1.0 = 0.00 , = 9.00, = 0.000000
−2 −2 13 1.0 9.00 1.000000
8 −2 −2 0.444444 1.555555
( )
= −2 4 −2 0.000000 = −2.888888 ,
−2 −2 13 1.000000 12.111111
0.128440
( ) ( )
= 12.111111, = −0.238532
1.000000
To generalize in table
The results of the first two iterations presented above and subsequent iterations are
presented in Table. These results were obtained on a 13-digit precision computer. The
iterations were continued until changed by less than 0.000001 between iterations. The
final solution for the largest eigenvalue, denoted as , and the corresponding
eigenvector is
= 13.870584 and = [−0.291794 −0.143499 1.000000]
By Natnael W. Page 31
Numerical Methods
This problem converged very slowly (30 iterations), which is a large number of iterations
for a 3 × 3 matrix.
By Natnael W. Page 32
Numerical Methods
1 0 0 8 −2 −2
= 1 0 and = 0
1 0 0
( )
Solve for by forward substitution using =
1 0 0
⎡−1 ⎤
1.0
⎢ 1 0⎥
⎢4 ⎥ = 1.0
⎢−1 −5 ⎥ 1.0
⎣4 1⎦
7
= 1.0
−1 5
= 1.0 − (1.0) =
4 4
−1 −5 5 15
= 1.0 − (1.0) − =
4 7 4 7
( ) ( )
Solve for by back substitution using =
8 −2 −2 1.0
⎡ 7 −5⎤ ⎡5⎤
⎢0 ⎥ ⎢ ⎥
⎢ 2 2 ⎥ =⎢ 4 ⎥
⎢ 75 ⎥ ⎢ 15 ⎥
⎣ 0 0
7 ⎦ ⎣7 ⎦
= [1.0 − (−2.0)(0.5) − (−2)(0.2)]⁄8 = 0.30
5 5
= − − (0.2) (7⁄2) = 0.50
4 2
15 75
= ( ) ( ) = 0.20
7 7
Scale ( ) so that the unity component is unity.
0.30 1.00000
( ) ( ) ( )
= 0.50 = 0.300000 = 1.66667
0.20 0.66667
The results of the first iteration presented above and subsequent iterations are presented
in the Table below. These results were obtained on a 13-digit precision computer. The
iterations were continued until changed by less than 0.000001 between
iterations. The final solution for the smallest eigenvalue and the corresponding
eigenvector is = = . = 2.508981 and
= [1.000000 2.145797 0.599712] Therefore, to generalize in table
By Natnael W. Page 33
Numerical Methods
By Natnael W. Page 34
Numerical Methods
( , ,…, ) = 0
( , ,…, ) = 0
--------------------------------------------------------------------------------------- (3.33)
……………
( , ,…, ) = 0
Or ( ) = 0 where = [ , , … , ] and = [ , , … , ]
( ) ( ) ( ) ( )
If = , ,…, is an initial approximation to the solution vector then we can write
( ) ( ) ( )
the method as = − , = 0,1,2, …------------------------------------------ (3.34)
⁄ ⁄ ,…, ⁄
⁄ ⁄ ,…, ⁄
where = is the Jacobian matrix of the functions
⋮ ,⋯, ⋮
⁄ ⁄ ,…, ⁄ ( ( ))
, , … , evaluated at ( ).If the method converges, then its rate of convergence is two.
The iterations are stopped when ( ) − ( ) < , where is the error tolerance.
Example1: Perform three iterations of the Newton-Raphson method to solve the system of
+ + =7
equations Take initial approximation as = 1.5 & = 0.5
+ =9
Solution:
( )= + + −7=0
( )= + −9=0
( , ) ( , ) 2 + +2
= =
( , ) ( , ) 3 3
3 −( + 2 )
= where = | | = 3 (2 + ) − 3 ( + 2 )
−3 2 +
We can write now the method as
1 3 −( + 2 ) + + −7
⇒ = − , = 0,1,2, …
−3 2 + + −9
Using ( , ) = (1.5,0.5) , we get
1.5 1 0.75 −2.5 −3.75 2.2675
= − =
0.5 −14.25 −6.75 3.5 −5.50 0.9254
2.2675 1 2.5691 −4.1183 1.0963 2.0373
= − =
0.9254 −49.4951 −15.4247 5.4604 3.4510 0.9645
2.0013
=
0.9987
Example2:
a. Solve the system by Newton’s method
10 + sin( + ) = 1
8 − ( − )=1
12 + =1
Take one step form a suitable starting point. To obtain a suitable starting point, we use the
approximation sin( + ) ≈ 0 , cos( − ) ≈ 1 & ≈ 0 for the system of equations
( , , ) = 10 + sin( + ) − 1 = 0
( , , )=8 − ( − )−1= 0
( , , ) = 12 + −1= 0
By Natnael W. Page 35
Numerical Methods
By Natnael W. Page 36
Numerical Methods
Let ( , ) be a suitable approximation to ( , ). Then we write the general iteration method for
= ( , )
the solution of (3.34) as -------------------------------------------- (3.38)
= ( , ), = 0,1,2, …
If the method is converges, then lim → = and lim → =
The functions and are called the iteration functions but not all lead to converge.
− = ( , )− ( , )
Subtracting (3.38) from (3.37) we get, ------------------------ (3.39)
− = ( , )− ( , )
Let = − and = − be errors in the iteration. If ( , ) is a closed
approximation to the root ( , ) then we use usually test the condition, maximum absolute row sum
| ( , )| + ( , ) < 1
norm, ----------------------------------------------------------------- (3.40)
| ( , )| + ( , ) <1
at the initial approximation ( , ).
The method can be easily generalized to a system of equations in unknouns.
By Natnael W. Page 37
Numerical Methods
Exercise
1. Solve the following system of equations using Gauss-Elimination method
− + =1 +3 +6 = 2
(a) −3 + 2 − 3 = −6 (b) −4 +2 = 7
2 −5 +4 = 5 3 − +4 = 9
5 + + + =1
+ 7 + + = 12
c)
+ + 6 + = −5
+ + + = −6
2. Solve the following set of simultaneous linear equations by the matrix inverse method
2 +3 − =1
a. − +2 + = 8
− 3 − 2 = −13
− +3 − = 1
−3 +5 = 2
b.
− + =0
+ 2 − = −5
3. Solve the following set of simultaneous linear equations using the LU decomposition method
+ + =9
i. 2 − 3 + 4 = 13
3 + + 5 = 40
2 + − =6
ii. − 3 + 5 = 11
− + 5 + 4 = 13
4. Solve the following set of simultaneous linear equations using Jacobi and Gauss-Seidal iteration
method
2 − + 5 = 15
a. 2 + + =7
+ 3 + = 10
5 + 2 + = 12
b. + 4 + 2 = 15
+ 2 + 5 = 20
4 +2 =4
2 +8 +2 = 0
c.
2 +8 +2 =0
3 + 4 = 14
5. Solve the following systems of non-linear equations by any suitable method
+ = 11
a.
+ =7
= 3 −7
b.
= 2( + 1)
+ + =
c. + =1
+ =
By Natnael W. Page 38
Numerical Methods
CHAPTER FOUR
Interpolation
4.1. Finite Difference Operators
Assume that we have a table of values ( ), = 1,2,3, … , -------------------------------------- (4.1)
of any function = ( ) where the value of being equally spaced i.e. = + ℎ.
To determine the values of ( ) or ( ) for some intermediate values of x, the following
different types of difference are found useful.
4.1.1. Forward Difference
Let = ( ) be any function given by the values , , , … , which is takes for the
equidistant values , , , … , of the independent variable , then
− , − ,…, − are called the first differences of the function .
They are denoted by ∆ , ∆ , … , .
∆ = −
∆ = −
Therefore, we have ------------------------------------------------------------- (4.2)
…
∆ = −
The symbol ∆ is called the forward difference operator. The differences of the first
differences denoted by ∆ , ∆ , … , ∆ are called the second differences, where
∆ = ∆[ ∆ ] = − 2 +
… ----------------------------------------------------------------- (4.3)
∆ = ∆[ ∆ ] = − 2 +
…
∆ is called second forward difference operator.
∆ =∆ −∆ = −3 +3 −
Similarly, … ----------------------------------- (4.4)
∆ =∆ −∆
∆
∆
∆ ∆
∆ ∆
∆ ∆ ∆
∆ ∆
∆ ∆
∆
∆
By Natnael W. Page 39
Numerical Methods
The above table is called a diagonal difference table. The first term in the table is y . It is
called the leading term. The differences ∆y , ∆ y , ∆ y , …. are called the leading
differences.
Alternative Notation of forward difference operator is ∆ ( ) = ( + ℎ) − ( )
This follows that ∆ ( + ℎ) = ( + ℎ + ℎ) − ( + ℎ) = ( + 2ℎ) − ( + ℎ)
Similarly,∆ ( ) = ∆[∆ ( )]
= ∆[ ( + ℎ) − ( )]
= ∆ ( + ℎ) − ∆ ( )
= ( + 2ℎ) − ( + ℎ) − [ ( + ℎ) − ( )]
= ( + 2ℎ) − 2 ( + ℎ) + ( )
∆ is called the second difference of ( ) and ℎ is the step size.
∇
∇
∇ ∇
∇ ∇
∇ ∇ ∇
∇ ∇
∇ ∇
∇
∇
By Natnael W. Page 40
Numerical Methods
⁄ = − ----------------------------------------------------------- (4.7)
…
= ⁄ − ⁄
The central difference table (Table 3)
⁄ ⁄
⁄ ⁄ ⁄
⁄ ⁄
By Natnael W. Page 41
Numerical Methods
= ( )+ ( )+ ( )+⋯
! !
(Expanding by Taylor’s series method)
= 1+ + +⋯ ( )
! !
∴ ( )= ( )
Hence, the identity is = .
We have already proved that = ∆ + 1 and = . Apply the algorithm,
By Natnael W. Page 42
Numerical Methods
We get
∆ ∆ ∆
⇒ℎ = = [ 1 + ∆] = ∆ − + − +⋯
∆ ∆ ∆
⇒ = ∆− + − +⋯ .
Relation between &∇: ∇ ( ) = ( ) − ( − ℎ) = ( ) − ( )
⇒ ∇= 1 −
∇=
Relation between the operators ∆ , , & : From the definition we know that
( )= + ℎ − − ℎ
i. ( )= + ℎ − − ℎ
⁄ ( )− ⁄ ( )
=
⁄ ⁄ ( )
= −
⁄ ⁄
∴ = −
( )= ⁄ ( − 1) ( ) =
Further
⁄
∆ ( )
⁄
∴ = ∆
And from the mean operator result we get
⁄
=∆
ii. ( )= + ℎ + − ℎ
⁄ ⁄ ( )
= +
1 ⁄ ⁄
∴ = +
2
iii. ∇ ( ) = [ ( ) − ( − ℎ)]
= ( )− ( − ℎ)
= ( + ℎ) − ( ) = ∆ ( )
∴ ∇= ∆
and ∇ ( ) = ∇ ( + ℎ)
= ( + ℎ) − ( ) = ∆ ( )
⇒ ∇ =∆
∴ ∇= ∇
By Natnael W. Page 43
Numerical Methods
By Natnael W. Page 44
Numerical Methods
Solution:
⁄ ⁄ ⁄
a. since 1 + ∆= therefore + = +1 =1+∆+1=∆+2
∆ ∇ ( )
b. −∆ = − = − = − =( − )
∇ ( )
= {(1 + ∆) − (1 − ∇)} = (∆ + ∇)
∆ ∇
Hence, − ∆ = ∆ + ∇.
∇
The polynomial through a specific set of points may take many different forms, but all forms are
equivalent. Any form can be manipulated into any other form by simple algebraic rearrangement
The Taylor series is a polynomial of infinite order. Thus,
( ) = ( ) + ( )( − ) + ( )( − ) + ( )( − ) + ⋯----------- (4.8)
! !
It is of course, impossible to evaluate an infinite number of terms. The Taylor polynomial of degree
n is defined by
( ) = ( )+ ( )------------------------------------------------------------------------------------- (4.9)
where the Taylor polynomial ( ), and the remainder term ( ) are given by
( )= ( )( )( − ) ≤ ≤ ---------------------------------------------- (4.11)
( )!
By Natnael W. Page 45
Numerical Methods
By Natnael W. Page 46
Numerical Methods
− 2( − )− −2 + ∆
⇒ = = =
2ℎ 2ℎ 2! ℎ
⋮
∆
⇒ = , = 3,4,5, …
!
Substitute the values in (4.14) we get
∆ ∆
( )= + ( − )+ ( − )( − ) + ⋯.
!
∆
+ ( − )( − )…( − )------------------------------- (4.15)
!
Setting = then − = ℎ
− = − + − = − −( − ) = ℎ − ℎ = ( − 1)ℎ
Similarly, − = ( − 2)ℎ
− = ( − 3)ℎ
⋮
− = ( − ( − 1))ℎ
Equation (4.15) reduces to
( )
( )= + !∆ + ∆ + ⋯.
!
( )…( ( ))
+ ∆ ------------------------------------ (4.16)
!
The above formula is called Newton’s forward interpolation formula.
It is useful for interpolating near the beginning of the tabular values.
The error term for the Newton forward-difference polynomial can be obtained from the
general error term equation (4.13)
( )= ( − )( − )…( − ) ( )( )
( )!
We know that − = ℎ, − = ( − 1)ℎ,…, − = ( − ( − 1))ℎ
Therefore the error term can be written as
( )= ( − 1)( − 2) … ( − )ℎ ( )( )---------------------- (4.17)
( )!
( )= ( )( )
And also can be written as, ℎ
+1
Example1: Evaluate = = 0.05 using the following table
0.00 0.10 0.20 0.30 0.40
1.0000 1.2214 1.4918 1.8221 2.2255
By Natnael W. Page 47
Numerical Methods
By Natnael W. Page 48
Numerical Methods
( )= = + ( − )= + (−ℎ)
− −∇ ∇
= = =
−ℎ −ℎ ℎ
When =
( )= = + ( − )+ ( − )( − )
−
= + (−2ℎ) + (−2ℎ)(−ℎ) = + 2( − )+ (2ℎ )
−ℎ
− 2( − )− −2 + ∇
⇒ = = =
2ℎ 2ℎ 2! ℎ
⋮
∇
⇒ =
!ℎ
Substitute the values in (4.18) we get
∇ ∇
( )= + ( − )+ ( − )( − ) + ⋯.
!
∇
+ ( − )( − )…( − )-------------------------------------- (4.19)
!
Setting = then − = ℎ
= − − + − = ℎ + ℎ = ( + 1)ℎ
Similarly, − = ( + 2)ℎ
− = ( + 3)ℎ
⋮
− = ( + ( − 1))ℎ
Equation (4.19) reduces to
( )
( )= + !∇ + ∇ + ⋯.
!
( )…( ( ))
+ ∇ --------------------------------------------------------- (4.20)
!
The above formula is called Newton’s backward interpolation formula.
It is useful for interpolating near the end of the tabular values.
The error term of backward interpolation is also can be written as
( )= ( + 1)( + 2) … ( + )ℎ ( )( )---------------------- (4.21)
( )!
Example: The area of a circle of diameter is given for the following values.
80 85 90 95 100
5026 5674 6362 7088 7854
By Natnael W. Page 49
Numerical Methods
By Natnael W. Page 50
Numerical Methods
10 11 12 13 14
10 23967 28060 31788 35209 38368
= 10 23967
4093
= 11 28060 −365
3728 58
= 12 31788 −307 −13
3421 45
= 13 35209 −262
3159
= 14 38368
= 12.2 − 12 = 0.2
( )
0.2 1 ( 1 1.2 0.2
= 31788 + 3421 + 3728) + + (−307)
1 2 2 2 2
1
1.2 ( 1 2.2 1.2 (
+ 45 + 58) + + −13)
3 2 2 4 4
( )
= 31788 + (0.1)(7149) + (0.02)(−307) + (−0.016)(103)
+ (−0.0016)(−13)
By Natnael W. Page 51
Numerical Methods
By Natnael W. Page 52
Numerical Methods
( )= ( )
( − )( − ) ( − )( − )
= ( ) + ( )
( − )( − ) ( − )( − )
( − )( − )
+ ( )
( − )( − )
Similarly, the Cubic Lagrange’s interpolation polynomial is
( )= ( )
( − )( − )( − )
= ( )
( − )( − )( − )
( − )( − )( − )
+ ( )
( − )( − )( − )
( − )( − )( − )
+ ( )
( − )( − )( − )
( − )( − )( − )
+ ( )
( − )( − )( − )
The Lagrange polynomial pass through each of the points used in its construction and data
are not required to be specified with in ascending or descending order. The computations
of ( ) is simple but the method is still not particularly efficient for large value of .
Example1: Construct the linear Lagrange interpolation polynomial that pass through the
points (2,4) and (5,1).
Solution: ( )= = ( − 5) and ( )= = ( − 2)
So that ( )= ( ) ( )+ ( ) ( )
= ( − 5)(4) + ( − 2)(1)
=− +6
(2,4)
(5,1)
Example2:
a. Use the number (called nodes) =2, = 2.75 & = 4 to find the second
Lagrange interpolating polynomial for ( ) = 1⁄ .
b. Use this polynomial to approximate (3) = 1⁄3.
By Natnael W. Page 53
Numerical Methods
Solution:
a. We first determine the coefficient polynomials ( ), ( )& ( ). In nested form they
( . )( )
are ( ) = = ( − 2.75)( − 4)
( . )( )
( − 2)( − 4) −16
( )= = ( − 2)( − 4)
(2.75 − 2)(2.75 − 4) 15
( )( . )
and ( ) = = ( − 2)( − 2.75)
( )( . )
Also, ( ) = (2) = 1⁄2 , ( ) = (2.75) = 4⁄11 ( ) = (4) = 1⁄4, so
( )= ( ) ( )
1 64 1
=( − 2.75)( − 4) − ( − 2)( − 4) + ( − 2)( − 2.75)
3 165 10
1 35 49
= − +
22 88 44
( ) ⁄
b. An approximate to 3 = 1 3 is
9 105 49
(3) ≈ (3) = − + ≈ 0.32955
22 88 44
By Natnael W. Page 54
Numerical Methods
The advantage of the above method is that there is no need to start all over again if their
additional pairs of data are added. We simply need to compute additional divided
differences.
Since order polynomial interpolation of a given ( + 1) pairs of datais unique, thus the
above polynomial and Lagrangian polynomial are exactly the same.
Example: Use Newton’s Divided Difference formula and evaluate (3.0), given
3.2 2.7 1.0 4.8 5.6
22.0 17.8 14.2 38.3 51.7
Solution:
( ) [ , ] [ , , ] [ , , , ] [ , , , , ]
3.2 .
.
2.7 17.8 .
2.118 − .
1.0 14.2 2.012 .
6.342 0.0865
4.8 38.3 2. .63
16.750
5.6 51.7
The third degree polynomial fitting all points from = 3.2 to = 4.8 is given by
( )= + ( − ) + ( − )( − ) + ( − )( − )( − )
( ) = 22 + 8.4( − 3.2) + 2.856( − 3.2)( − 2.7)
− 0.528( − 3.2)( − 2.7)( − 1)
The fourth degree polynomial fitting all points from = 3.2 to = 5.6 is also given by
By Natnael W. Page 55
Numerical Methods
( ) = ( ) + ( − )( − )( − )( − )
( ) = ( ) + 0.256( − 3.2)( − 2.7)( − 1)( − 4.8)
(3) ≈ (3) = 22 + 8.4(3 − 3.2) + 2.856(3 − 3.2)(3 − 2.7)
−0.528(3 − 3.2)(3 − 2.7)(3 − 1)
= 20.2120
(3) ≈ (3) = 20.2120 + 0.256(3 − 3.2)(3 − 2.7)(3 − 1)(3 − 4.8)
= 20.2120 + 0.055296
= 20.267296
( , )
( , )
( , )
( , )
( , )
By Natnael W. Page 56
Numerical Methods
Example: The upward velocity of a rocket is given as a function of time in the table
below.
Find the velocity at = 16 seconds.
, , / , , /
0 0 0 0
10 227.4 rewrite in to 10 227.4
20 517.35 15 362.78
15 362.78 ascending 20 517.35
22.5 602.97 22.5 602.97
30 901.67 30 901.67
Solution:
−
( )= + ( − ) , ≤ ≤
−
517.35 − 362.78
= 362.78 + ( − 15) , 15 ≤ ≤ 20
20 − 15
= 362.78 + 30.913( − 15) , 15 ≤ ≤ 20
(16) ≈ 362.78 + 30.913(16 − 15)
= 393.7 ⁄
The most common piecewise polynomial approximation uses cubic polynomials between
each successive pair of nodes is called cubic spline interpolation.
( )
…
Now we have
+ 1 grid points , ( = 0,1, … , )
intervals ≤ ≤ , ( = 0,1, … , − 1)
cubic spline ( ), ( = 0,1, … , − 1)
− 1 interior grid points , ( = 0,1, … , − 1)
4 coefficient to be determined
By Natnael W. Page 57
Numerical Methods
By Natnael W. Page 58
Numerical Methods
⇒ ( )= --------------------------------------------------------------------------------------- (4.35)
( ) ( )
= ( )= − ( +2 ) ----------------------------------------------- (4.36)
But ( ) the slope for the last subinterval is
( )=3 ℎ +2 ℎ + ---------------------------------------------------- (4.37)
After using , and
( )=3 ( − )ℎ +2 ℎ .
( ) ( )
+ − ( +2 )
( ) ( )
⇒ ( )= + ℎ -------------------------------------------- (4.38)
From equation (4.35) and (4.36) =3 ℎ +2 ℎ +
By substituting the values of , , and
( ) ( ) ( ) ( )
− ( +2 )= + (2 + )
for = 1,2, … ,
( ) ( ) ( ) ( )
⇒ + + = − ---------------------- (4.39)
Now for equally spaced arguments ℎ = ℎ equation (4.39) becomes
+4 + = [ ( )−2 ( )+ ( )]---------------------------------- (4.40)
While, ( ) for the equally spaced becomes,
( )= [( − ) +( − ) ]
+ ( − ) ( )−
+ ( − ) ( )− ------------------------------------------------------------ (4.41)
Equation (4.41) gives the cubic spline interpolation while equation (4.40) gives the condition
for .
Remark: = = 0 (Natural spline) and this one is the most common boundary
condition.
Example: Find the natural spline interpolating for the given data
0 1 2 3
( ) 1 2 33 244
By Natnael W. Page 59
Numerical Methods
4 + = 180
⇒
+4 = 1056
Solving this system and then = −24 and = 276
1
( ) = [( − ) +( − ) ]
6
+( − ) ( ) −
+( − ) ( )−
1
( ) = [(1 − ) (0) + ( − 0) (−24)]
6
+(1 − ) 1 − (0)
+( − 0) 2 − (−24)
= 4 + 5 + 1, 0≤ ≤1
1
( ) = [( − ) +( − ) ]
6
+( − ) ( ) −
+( − ) ( )−
1
( ) = [(2 − ) (−24) + ( − 1) (276)]
6
+(2 − ) 2 − (−24)
+( − 1) 33 − (276)
= 50 − 162 + 167 − 53, 1≤ ≤2
1
( ) = [( − ) +( − ) ]
6
+( − ) ( ) −
+( − ) ( )−
1
( ) = [(3 − ) (276) + ( − 2) (0)]
6
+(3 − ) 33 − (276)
+( − 2) 244 − (0)
= −46 + 414 − 985 + 715, 2≤ ≤3
By Natnael W. Page 60
Numerical Methods
Exercise
∆
1. Evaluate
2. Show that = +∆ +∆ +∆
3. If ( ) = 2 − + 3 + 1 then show that ∆ ( ) = 12 + 10
4. Find the missing figures in the following table
0 5 10 15 20 25
7 11 18 32
9. Use the following table to evaluate 16° and compare with the exact value.
° 0° 5° 10° 15° 20° 25° 30°
° 0 0.0875 0.1763 0.2679 0.3640 0.4663 0.5774
10. Construct the natural cubic spline for the following data.
( )
0.1 −0.62049958
0.2 −0.28398668
0.3 0.00660095
0.4 0.24842440
11. Use the divided-difference method to obtain a polynomial of least degree that fits the
values shown.
a. 0 1 2 −1 3 b. 1 3 −2 4 5
( ) −1 −1 −1 −7 5 ( ) 2 6 −1 −4 2
By Natnael W. Page 61
Numerical Methods
12. The following table gives the normal weights of babies during the first 12 months of life:
ℎ 0 2 5 8 10 12
ℎ 7.5 10.25 16 18 18 21
Find the weight of babies during 6.1 month of life.
13. A natural cubic spline is defined by
( ) = 1 + ( − 1) − ( − 1) 1≤ ≤2
( )= 3
( ) = 1 + ( − 2) + ( − 2) + ( − 2) 2≤ ≤3
4
If is interpolates the data(1,1), (2,1), and (3,0), find , , , and .
By Natnael W. Page 62
Numerical Methods
Chapter Five
Least Square Method
5.1. Linear Least Square Method
In experimental, social, and behavioral sciences, an experiment or survey often produces
a mass of data. To interpret the data, the investigator may resort to graphical methods.
For instance, an experiment in physics might produce a numerical table of the form
⋯
------------------ (5.1)
⋯
And from it, + 1 points on a graph could be plotted.
A reasonable tentative conclusion is that the underlying function is linear and that the
failure of the points to fall precisely on a straight line is due to experimental error.
Assuming that = +
What are the coefficients and ? Thinking geometrically, we ask: What line most nearly
passes through the nine points plotted?
= +
To answer this question, suppose that a guess is made about the correct values of
and . This is equivalent to deciding on a specific line to represent the data. In general,
the data points will not fall on the line = + . If by chance the datum falls on
the line, then
+ − =0
If it does not, then there is a discrepancy or error of magnitude
| + − |
The total absolute error for all + 1 points is therefore
( , )=∑ | + − |
In practice, it is common to minimize a different error function of and :
By Natnael W. Page 63
Numerical Methods
Plot the original data points and the line using a finer set of grid points.
Solution: The system will be
644 52 227
+1 = = =
52 5 17
Then we get = 0.4864 and = −1.6589.
( , ) = ∑ (0.4864 − 1.6589 − )
= 2.9354 + 3.0482 + 2.8612 + 1.7841 + 0.1522 = 10.7813
Example2: Find the best fit values of and so that = + fits the data given in
the table
0 1 2 3 4
1 1.8 3.3 4.5 6.3
By Natnael W. Page 64
Numerical Methods
By Natnael W. Page 65
Numerical Methods
=∑ −2∑ ∑ +∑ ∑ ∑
As in the linear case, for to minimized is necessarily that ⁄ = 0, for each
= 0,1, … , . Thus, for each , we must have
0= = −2 ∑ + 2∑ ∑
This gives + 1 normal equations in the + 1 unknowns . These are
∑ ∑ =∑ for each = 0,1, … , ----------------------------- (5.4)
It is helpful to write the equations as follows:
∑ + ∑ + ∑ +⋯+ ∑ =∑
⎧
∑ + ∑ + ∑ +⋯+ ∑ =∑
⎨ ⋮
⎩ ∑ + ∑ + ∑ + ⋯+ ∑ =∑
These normal equations have a unique solution provided that the ’s are distinct.
Example: Fit the data below with the discrete least squares polynomial of degree at
most 2.
1 0 1.0000
2 0.25 1.2840
3 0.50 1.6487
4 0.75 2.1170
5 1.00 2.7183
Solution: For this problem, = 2, = 5, and the three normal equations are
∑ + ∑ + ∑ =∑
∑ + ∑ + ∑ =∑
∑ + ∑ + ∑ =∑
5 + 2.5 + 1.875 = 8.7680
2.5 + 1.875 + 1.5625 = 5.4514
1.875 + 1.5625 + 1.3828 = 4.4015
Solving the system of equations gives
{ = 1.0051, = 0.8647, = 0.8432}
Thus the least squares polynomial of degree 2 fitting the data in the above table is
( ) = 1.0051 + 0.8647 + 0.8432
The total error use this table
1 2 3 4 5
0 0.25 0.50 0.75 1.00
1.0000 1.2840 1.6487 2.1170 2.7183
( ) 1.0051 1.2740 1.6483 2.1279 2.7130
− ( ) −0.0051 0.0100 0.0004 −0.0109 0.0053
By Natnael W. Page 66
Numerical Methods
b. Exponential
Suppose an exponential curve of the form =
Taking logarithm on both the sides,
We get = + -------------------------------------------------- (5.5)
That is, = + where = , = , =
The normal equation for (5.5) is,
∑ = + ∑
---------------------------------------------------- (5.6)
∑ = ∑ + ∑
On solving the above two equations, we get and .
Then = , =
Example: Fit a curve = to the following data:
x 2 3 4 5 6
y 144 172.8 207.4 248.8 298.5
By Natnael W. Page 67
Numerical Methods
⇒ ∑ = ∑ + ∑ ---------------------------------------------------- (5.7)
And =0 ⟹∑ 2 − − (− )=0
⇒ ∑ = ∑ + ∑ ----------------------------------------------- (5.8)
Dropping the suffix in (5.7) and (5.8) then the normal equations are,
∑ = ∑ + ∑
√
---------------------------------------------------------------- (5.9)
∑ √ = ∑ + ∑
√
Example: Use the method of least squares to fit the curve = + √ to the
following table of values:
0.1 0.2 0.4 0.5 1 2
21 11 7 6 5 6
Solution:
√ 1 1 √
√
0.1 21 0.3162 3.1623 100 210 6.6402
0.2 11 0.4472 2.2361 25 55 4.9192
0.4 7 0.6324 1.5811 6.25 17.5 4.4268
0.5 6 0.7071 1.4142 4 12 4.2426
1 5 1 1 1 5 5
2 6 1.4142 0.7071 0.25 3 8.4852
1 1
√ √
√
= 4.2 = 4.5171 = 10.1008 = 136.5 = 302.5 = 33.714
∑ = ∑ + ∑
√ 302.5 = 136.5 + 10.1008
⇒
∑ √ = ∑ + ∑ 33.714 = 10.1008 + 4.2
√
Solving the system of equations we get = 1.9733 and = 3.2815
.
Hence the required equation is = + 3.2815√ .
By Natnael W. Page 68
Numerical Methods
⎡1 ⎤ ⎡ ⎤
⎢ ⎥ ⎢ ⎥
⎢ ⎥ =⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣ ⎦ ⎣ ⎦
We get = ≈ −0.050465 =− = ≈ 4.12251
Consequently, the least square approximating polynomial of degree 2 for the function
( )= on the interval [0,1] is
( ) = −0.050465 + 4.12251 − 4.12251
By Natnael W. Page 69
Numerical Methods
Now however, assume that we have five times the confidence in the accuracy of the
final two data points, relative to the other points.
Define a square weighting matrix :
1 0 0 0 0
⎛0 1 0 0 0⎞
= ⎜0 0 1 0 0⎟
0 0 0 5 0
⎝0 0 0 0 5⎠
Now we perform the following operations:
∗ =
∗ ∗ = ∗
( ∗ ) ∗( ∗ )∗ =( ∗ ) ∗ ∗
= [( ∗ ) ∗ ( ∗ )] ∗ ( . ) ∗ ( ∗ )
=[ ∗ ] ∗ ∗ ----------------------------------- (5.12)
0.1365
=
1.5105
Then the best fit estimate is = 0.1365 + 1.5105
b. Weighted Least Square for continuous functions
Definition: An integrable function ( ) is called a weight function on the interval if
( ) ≥ 0 for all in , but ( ) ≠ 0 on any subinterval of .
The purpose of a weighted function is to assign varying degrees of important to
approximations on a certain portions of the interval.
The function which are continuous on [ , ] and are given explicitly, we have
=∫ ( )[ ( ) − ( )]
Where ( )= + +⋯+ + =∑
⇒ =∫ ( ) ( )−∑ =minimum
The necessarily condition is to have a minimum value is that = 0, = 0,1, . . ,
This gives a system of ( + 1) unknown , , … , . These equations are called
normal equations. The normal equation become
∫ ( ) ( )−∑ = 0, = 0,1, … , ------------------------------ (5.13)
By Natnael W. Page 70
Numerical Methods
= −2 ∫ [(1 + 2 )−( + + )] =0
= −2 ∫ [(1 + 2 )−( + + )] =0
This gives
⎧ + + =
⎪ 2 3
⎪
−
+ + =
⎨ 2 3 4 2
⎪
⎪ 2 −3
⎩3 + + =
4 5 6
Solving the system of equations
= 1.9879, = −0.6611, = 0.0156
( ) = 0.0156 − 0.6611 + 1.9879
For = 2 and ( ) =
We have = ∫ [(1 + 2 )−( + + )]
The normal equations are
= −2 ∫ [(1 + 2 )−( + + )] =0
= −2 ∫ [(1 + 2 )−( + + )] =0
= −2 ∫ [(1 + 2 )−( + + )] =0
This gives
⎧ + + =
⎪
+ + =
⎨
⎪
⎩ + + =
Solving the system of equations
= 3.3540, = −2.4082, = 0.4800
( ) = 0.4800 − 2.4082 + 3.3540
By Natnael W. Page 71
Numerical Methods
Exercise:
1. Fit a straight line = + to the following data by method of least square
0 1 3 6 8
1 3 2 5 4
2. Fit the second degree parabola to the following data by the least square method
1 2 3 4 5
1090 1220 1390 1625 1915
4. A pressure and volume of gas are related by the equation = ( and are
constants). Fit this equation to for the data given below
0.5 1.0 1.5 2.0 2.5 3.0
1.62 1.0 0.75 0.62 0.52 0.46
5. Find the normal equations that arise from filling by the least squares method, an
equation of the form = , to set of points (0,0), , 1 , ( , 3) and
( , 2) solve for and .
6. Find the linear least squares polynomial approximation to ( ) on the indicated
interval if
a. ( ) = + 3 + 2, [0,1];
b. ( ) = , [0,2];
c. ( ) = , [1,3];
7. Find the least squares polynomial approximation of degree two to the functions and
intervals in exercise 6.
8. Compute the error for the approximations in exercise 6 and 7.
By Natnael W. Page 72
Numerical Methods
Chapter 6
Numerical Differentiation and Integration
6.1. Numerical Differentiation
The method of obtaining the derivatives of a function using a numerical technique is
known as Numerical Differentiation. Numerical Differentiation is required for these two
situations
1. The function values are known but the function is unknown, such functions are
called tabular functions.
2. The function to be differentiating is complicated and therefore it is difficult to
differentiate. The choice of formula
a. Newton forward/backward formula – if the derivative at a point near the
beginning/end of a set of values respectively.
b. Newton divided difference/Lagrange Interpolation formula – if the values of
are not equally spaced.
6.1.1. Derivative using Newton’s forward interpolation formula:
Consider Newton’s forward interpolation formula
( ) ( )( )
= + ∆ + ∆ + ∆ + ⋯------------------- (6.1)
! !
= ----------------------------------------------------------------------------- (6.2)
Differentiation (6.1) w.r.t we get
=∆ + ∆ + ∆ + ⋯------------------------------- (6.3)
Differentiation (6.2) w.r.t we get
= -------------------------------------------------------------------------------- (6.4)
Now from equation (6.3) and (6.4) =
⇒ = ∆ + ∆ + ∆ + ⋯ --------------------- (6.5)
Expression (6.5) becomes the value of at any is not tabulated.
The formula (6.5) becomes simple for tabulated values of , in particular
when = and = 0.
Putting = 0 in (6.5) we get
= ∆ − ∆ + ∆ − ∆ + ⋯ -------------------- (6.6)
By Natnael W. Page 73
Numerical Methods
⇒ = ∇ + ∇ + ∇ + ⋯ -------------------- (6.13)
Expression (6.13) us the value of at any is not tabulated.
The formula (6.13) becomes simple for tabulated values of , in particular
when = and = 0.
Putting = 0 in (6.13) we get
= =∇ + ∇ + ∇ + ∇ + ⋯ --------------- (6.14)
By Natnael W. Page 74
Numerical Methods
= 7 − (12) + (6) = 3
= ∆ −∆ + ∆ −⋯
= [12 − 6] = 6
We have = 1, ℎ = 1, and = 1.5 ⇒ = 0.5
= ∆ + ∆ + ∆ +⋯
( . ) ( . ) ( . )
= 7+ (12) + (6)
= [7 − 0.25] = 6.75
= ∆ − ∆ + ∆ − ∆ +⋯
= 19 − (18) + (6) = 12
= ∆ −∆ + ∆ −⋯
= [18 − 6] = 12
Example 2: A rod is rotating in a plane about one of its ends. If the
following table gives the angle radians through which the rod has turned
for different values of time seconds, find its angular velocity at = 0.7 .
, 0 0.2 0.4 0.6 0.8 1
By Natnael W. Page 75
Numerical Methods
.
= = 1, ℎ = 0.2 and = = 0.7 where = = −1.5
.
From Newton’s interpolation formula, we have
= ∇ + ∇ + ∇ +⋯
.
( . ) ( . ) ( . )
= 1.2 + (0.3) + (0.02) + ⋯
.
= [1.2 − 0.3 − 0.0008]
.
= 4.496 / This represents the angular velocity.
= [∇ + ( + 1)∇ ]
.
=( [0.3 + (−1.5 + 1)(0.02)]
. )
= 7.25 / This represents the angular acceleration.
6.2. Numerical Integration
Numerical integration is used to obtain approximate answers for definite integrals
that cannot be solved analytically (has no explicit anti-derivative). Numerical
integration is a process of finding the numerical value of a definite integral
=∫ ( ) ------------------------------------------------------------------------------ (6.17)
Where, a function = ( ) is not known explicitly. But we give only a set of values
of the function = ( ) corresponding to the same values of . To evaluate the
integral, we fit up a suitable interpolation polynomial to the given set of values of
( ) and then integrate it within the desired limits. Here we integrate an
approximate interpolation formula instead of = ( ). When this technique is
applied on a function of single variable, the process is called Quadrature.
By Natnael W. Page 76
Numerical Methods
By Natnael W. Page 77
Numerical Methods
=∫ ( ) = [ + ]
⋮
=∫ ( )
( ) = [ + ]
Now adding , , … , we get
+ +⋯+ =
∫ ( ) +∫ ( ) +⋯+ ∫ ( )
( )
= [ + ]+ [ + ]+⋯+ [ + ]
⇒ ∫ ( ) = [( + )+( + ) + ⋯+( + )]
= [( + ) + 2( + + ⋯+ )]-------------------------- (6.20)
The formula (6.20) is called Trapezoidal rule for numerical integration.
The error committed in this formula is given by
( )
≈− ( )=− ( ) where = < < =
b. Simpson’s ⁄ rule
Substituting = 2 in the General quadrature formula given by (6.19) and
neglecting the third and other higher order differences
We get
∆
=∫ ( ) =ℎ 2 + 2∆ + −2
=ℎ 2 + 2( − )+ ( −2 + )
= [ +4 + ]
Similarly,
=∫ ( ) = [ +4 + ]
=∫ ( ) = [ +4 + ]
⋮
⁄ =∫ ( )
( ) = [ +4 + ]
Adding , , … , ⁄ we get,
= + + ⋯+ ⁄ =
∫ ( ) +∫ ( ) + ⋯+∫ ( )
( )
= [ +4 + ]+ [ +4 + ]+⋯
By Natnael W. Page 78
Numerical Methods
+ [ +4 + ]
( + )
= +4( + + ⋯ + ) -------------------------------------- (6.21)
+2( + + ⋯ + )
The above rule is known as Simpson’s one-third rule. The error
committed in Simpson’s one-third rule is given by
( )( ( ) ( )(
≈− )=− ) where = < < =
( for sub intervals of lengths ℎ)
c. Simpson’s ⁄ rule
We assume that within any three consecutive subintervals of width h, the
interpolating polynomial ( ) approximating ( ) is of degree 3.
Hence substituting = 3, i.e., the General quadrature formula and
neglecting all the differences above ∆ , we get
=∫ ( )
∆
=ℎ 3 + ∆ + 9− ∆ + − 27 + 9
3 + 9( − )+ ( −2 + )
=ℎ
+ ( −3 +3 + )
= [ +3 +3 + ]
Similarly
=∫ ( ) = [ +3 +3 + ]
⋮
⁄ =∫ ( )
( ) = [ +3 +3 + ]
Adding , , … , ⁄ we get,
= + + ⋯+ ⁄ =
( + )+
= 3( + + + + ⋯ + + ) -------------- (6.22)
+2( + + ⋯ + )
Simpson’s three-eighths rule can be applied when the range [ , ]
is divided into a number of subintervals, which must be a
multiple of 3.
The error in Simpson’s three-eighths rule is
( )( ) where
≈− = < < =
( for sub intervals of lengths ℎ)
By Natnael W. Page 79
Numerical Methods
∫ ( + ) = + = ( − )+ -- (6.26)
( )= [ + ]= + ( )------------------------- (6.27)
From (6.26) and (6.27) = − and =
( )( )
⇒ =( − ) = = ⇒ =
⇒ ∫ ( ) ≈ ( )=( − ) ( )
If ∫ ( ) ≈ (1 − (−1)) ∫ = 2 (0)--------------- (6.28)
.
Example: Find the integral ∫ . 5
. . .
Solution: ∫ . 5 = (1.3 − 0.1)
= 1.2 (0.7)
( . )
= 1.2 5(0.7) = 1.0357
By Natnael W. Page 80
Numerical Methods
( + + + )
( )+ ( )=
+ ( + + + )
( + )+ ( + )
= ----------- (6.30)
+ ( + )+ ( + )
We have the four equations
+ =2
⎫
+ = 0⎪
+ = ⎬ and can be determine the four
⎪
+ = 0⎭
unknowns , , , .
Solving simultaneously, we get = = 1 and
=± and =∓ .
√ √
Therefore, the two point Gauss quadrature is given by
∫ ( ) ≈ − + ---------------------------------------- (6.31)
√ √
Therefore, ∫ =∫ =∫
≈ − + = +
√ √
√ √
= 0.41277 + 0.27954 = 0.69231
c. Three point Gaussian quadrature formula
For = 3, 2(3) − 1 = 5 (degree five polynomial)
∫ ( ) ≈ ( )+ ( )+ ( ), ≤ ≤ ,
≤ ≤ and ≤ ≤
By Natnael W. Page 81
Numerical Methods
= , = , = and =± , = 0, =∓ .
Therefore, the two point Gauss quadrature is given by
To evaluate = ∫ ≈ 5 ( ) +8 (0) + 5 (− )
By Natnael W. Page 82
Numerical Methods
−0.77460 0.55556
4 0.86114 0.34785
0.33998 0.65214
−0.33998 0.65214
−0.86114 0.34785
By Natnael W. Page 83
Numerical Methods
3
= (0.2)[0.0962 + 3(0.0916 + 0.0874) + 0.0836] = 0.0538
8
Then,
2 2.3 2.6
( ) 0.0698 0.0608 0.0538
Exercise
1. find and at = 1, =2
1 2 3 4 5 6
1 8 27 64 125 216
2. Find the first and second derivative for = 15, = 23
15 17 19 21 23 25
√ 3.873 4.123 4.359 4.583 4.796 5.000
3. The table given below reveals the velocity ‘ ’ of a body during the time ‘ ’ specified. Find its
acceleration at = 1.1.
1.0 1.1 1.2 1.3 1.4
43.1 47.7 52.1 56.4 60.8
4. A rod is rotating a place the following table given the angle (radian) through with the rod has
turned for various values of the time seconds find
a. the angular velocity of the rod
By Natnael W. Page 84
Numerical Methods
By Natnael W. Page 85
Numerical Methods
Chapter 7
Numerical Solutions of Differential Equations
1. Taylor’s Series Method
( )
Differentiating successively we obtain , ,…
( − )
( ) = ( )+( − ) ( )+ ( )+⋯
2!
( )
= +( − ) + + ⋯ -------------------------------------------------------------------- (7.3)
!
Substituting the values of , , , … we obtain ( ) for all values of for which (7.3)
converges.
( + ℎ) = ( ) = = + + + ⋯ --------------------------------------------------------- (7.4)
! !
Where 0(ℎ ) represent all the terms involving third and higher power of ℎ.
By Natnael W. Page 86
Numerical Methods
Solution:
2. Euler’s Method
The Taylor’s series method that we have discussed in the previous section yields solution of
differential equations in the form of a power series.
We now proceed to describe methods which give the solution in the form of table values at
equally spaced points.
By Natnael W. Page 87
Numerical Methods
∫ =∫ ( , )
= +( − ) ( , )
∴ = +ℎ ( , ) ------------------------------------------------------------------------------------------ (7.7)
Similarly if ≤ ≤ , we have
= + ( , )
= +ℎ ( , ) ---------------------------------------------------------------------------------------------- (7.8)
Which is the mean of the slopes of the tangents at the points corresponding to = and
= .
( )
Thus, we obtain = + [ ( , )+ ( , )]
( )
Where is given by (2). is the modified value of .
By Natnael W. Page 88
Numerical Methods
( ) ( )
Let = + ( , )+ , is the second modified value of .
Let be the final value obtained to the desired accuracy. Using this value of we compute
= +ℎ ( + ℎ, )
( ) ( )
Now let = + [ ( , )+ ( , )] and is the modified value of .
( ) ( )
Let = + ( , )+ , is the second modified value of .
Then we proceed to calculate as above and we continue the process till we calculate .
Now = + ℎ = 0.1
Now = + ℎ = 0.2
( )
= + [ ( , )+ ( , )]
By Natnael W. Page 89
Numerical Methods
.
= 2+ [1 + 1 + (0.1)(2.1)] = 2.1105
( ) ( )
= + ( , )+ ,
.
=2+ [1 + 1 + (0.1)(2.1105)] = 2.1105
( )
= + [ ( , )+ ( , )]
.
= 2.1105 + [1 + (0.1)(2.1105) + 1 + (0.2)(2.2316)] = 2.2434
( ) ( )
= + ( , )+ ,
.
= 2.1105 + [1 + (0.1)(2.1105) + 1 + (0.2)(2.2434)] = 2.2435
( )
Similarly = 2.2435
.
= 2.2435 + [1 + (0.2)(2.2435) + 1 + (0.3)(2.3884)] = 2.4018
( ) ( )
= + ( , )+ ,
.
= 2.2435 + [1 + (0.2)(2.2435) + 1 + (0.3)(2.4018)] = 2.4020
( ) ( )
= + ( , )+ ,
.
= 2.2435 + [1 + (0.2)(2.2435) + 1 + (0.3)(2.4020)] = 2.4020
By Natnael W. Page 90
Numerical Methods
4. Runge-Kutta Method
The Taylor’s series method of solving differential equation numerical is handicapped by the
problem of finding the higher order derivatives. Euler’s method is less efficient in practical
problems since it requires ℎ to be small for obtaining reasonable accuracy.
The Runge-Kutta Methods do not require the calculations of higher order derivatives and they
are designed to give greater accuracy with the advantage of requiring only the function values
at some selected points on the sub-interval.
Hence the Modified Euler’s method is the R-K method of second order.
Calculate successively: =ℎ ( , )
=ℎ ( + ℎ⁄2 , + ⁄2 )
=ℎ ( + ℎ⁄2 , + ⁄2 )
=ℎ ( + ℎ, + )
and ∆ = ( +2 +2 + )
By Natnael W. Page 91
Numerical Methods
Example: Compute (0.1) and (0.2) by R-K method 4 order for differential equation.
= + , (0) = 1
=ℎ ( , )
=ℎ ( + ℎ⁄2 , + ⁄2)
=ℎ ( + ℎ⁄2 , + ⁄2)
=ℎ ( + ℎ, + )
and ∆ = ( +2 +2 + )
∴ = + ∆ = 1 + 0.1169 = 1.1169
∴ (0.1) = 1.1169
By Natnael W. Page 92
Numerical Methods
∴ (0.2) = 1.2774
Exercise:
1. Use the Euler’s method to solve the following differential equation over the interval [0,1]
With the step size ℎ = 0.2
=− + + , (0) = 1
2. Solve =1− , (0) = 0 using Euler’s method. Find at = 0.1 and = 0.2
Compare the result with result of exact solution.
3. Solve the equation in #1 using Modified Euler method for the values of at = 0.2,
= 0.4 and = 0.6
4. Use the R-K 4 order method to find numerical solution at = 0.8
For = + , (0.4) = 0.41and take ℎ = 0.2
5. Using R-K method compute the initial value problem = (2 − ) with (2) = 1 for
= 2.2 = 2.4 (take ℎ = 0.1). Compare the result with result of exact solution.
By Natnael W. Page 93
Numerical Methods
By Natnael W. Page 94