You are on page 1of 58

ME 6185 : Advanced Numerical Analysis

Course Teacher: Dr. Md. Mamun

Roots of polynomials and transcendental equations; Error Analysis;


Determinants and matrices; Eigen values and Eigen vectors; Solution of linear
and non-linear algebraic equations ; Interpolation methods; Curve fitting;
Numerical Differentiation and Integration.

Solution of first-order ordinary differential equations, Solution of first-order


system of differential equations, Solution of higher-order differential equations,
Solving differential equations by finite differences with engineering applications.
Introduction
Introduction
What is Numerical Analysis and Why?
 All physical systems in the universe, if mathematically
modeled, produce differential equations.
 Engineers and Scientists have to solve the differential
equations to design, analyze and predict the behavior of the
physical systems.

 A vast majority of the differential equations are non-


linear, and difficult to solve analytically (in many cases no
analytical solution exists till date).

 Numerical analysis provides the methodology to solve


those differential equations numerically and thus facilitate
the design, analysis and description of the physical systems.
What will we learn in this course?
 The step by step processes to solve smaller (humanly
possible) system of differential equations by hand
calculator.
 Relevant basic numerical techniques such as root finding of
algebraic/transcendental equations, simultaneous equations,
differentiation, integration, curve fitting, etc. which are necessary
for the solution and development of the differential equations..
 The development of the generic algorithms (step by
step processes) to solve larger (humanly impossible) system of
differential equations and other relevant techniques as described
above by using digital computers.
 Matlab and/or C computer programming language
to implement those algorithms to solve for larger system of
equations.
Numerical Computing Process
Different Forms of Mathematical Equations
Alternative View of the
Computing Process
Simplified View of the Computing Process
Flowchart of the Computing Process
Example of Applied Numerical Analysis

Slender Fins
Example of Applied Numerical Analysis
Steady One-dimensional Conduction through Fins
qconv or qradiation

qx qx+dx

Conservation of Energy:

qcond , x  qcond , x  x  qconvection


OR

qcond , x  qcond , x  x  qradiation


For a constant cross section area:

d 2T
kAc 2  hP(T  T )  0
dx
d 2T hP
2
 (T  T )  0
dx kA
hP
m 
2

kA
d 2T
2
 m 2
(T  T )  0
dx

Define:   T  T

d 2
2
m  0
2

dx
At the base of the fin:

base  Tbase  T
Tip of A Fin
Fin Performance Analysis
 Temperature Distribution along the longitudinal direction

 Heat Dissipated

 Fin Efficiency
Some Engineering Problems
Example-2: The general equation relating to 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑖,
𝑣𝑜𝑙𝑡𝑎𝑔𝑒 𝑉, 𝑟𝑒𝑠𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑅 and 𝑖𝑛𝑑𝑢𝑐𝑡𝑎𝑛𝑐𝑒 𝐿 of a serial
circuit is given by
𝑑𝑖
𝐿 + 𝑖𝑅 = 𝑉
𝑑𝑡
Find the value of current after 𝟐 𝒔𝒆𝒄𝒐𝒏𝒅𝒔, if
𝑟𝑒𝑠𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑅 = 20 𝑜ℎ𝑚𝑠, 𝑖𝑛𝑑𝑢𝑐𝑡𝑎𝑛𝑐𝑒 𝐿 = 50 𝐻 and
𝑣𝑜𝑙𝑡𝑎𝑔𝑒 𝑉 = 240 𝑣𝑜𝑙𝑡𝑠. 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑖 = 0 𝑤ℎ𝑒𝑛 𝑡 = 0.

Example-3: An object with a mass of 10 kg is falling under


the influence of earth gravity. Find its velocity after 5
seconds if it starts from the rest. The object experiences a
retarding force equal to 0.25 of its velocity.
𝑑𝑣
𝑚 = 𝑚𝑔 − 0.25𝑣 20
𝑑𝑡
Roots of Polynomials and
Transcendental Equations
(Single Root)
Root Finding Methods
 Graphical Method

 Iterative Method
 Bisection (Halving) Method
 False Position (Regula Falsi) Method
 Newton-Raphson Method
 Secant Method
 Fixed Point Method
Bracketing Methods
(Or, two point methods for finding roots)
Chapter 5

• Two initial guesses for the


root are required. These
guesses must “bracket” or
be on either side of the
root.

== > Fig. 5.1

• If one root of a real and


continuous function,
f(x)=0, is bounded by
values x=xl, x =xu then
f(xl) . f(xu) <0. (The function
changes sign on opposite sides of 23
the root)
The Bisection Method
For the arbitrary equation of one variable, f(x)=0
1. Pick xl and xu such that they bound the root of
interest, check if f(xl).f(xu) <0.

2. Estimate the root by evaluating f[(xl+xu)/2].

3. Find the pair


• If f(xl). f[(xl+xu)/2]<0, root lies in the lower interval,
then xu=(xl+xu)/2 and go to step 2.
24
• If f(xl). f[(xl+xu)/2]>0, root
lies in the upper interval, then
xl= [(xl+xu)/2, go to step 2.
xl  xu
xl 
• If f(xl). f[(xl+xu)/2]=0, then 2
 100%
root is (xl+xu)/2 and xl  xu
terminate. 2
or
4. Compare es with ea xl  xu
xu 
2
 100%
xl  xu
5. If ea< es, stop. Otherwise 2
repeat the process.

25
Figure 5.6

26
Bisection Method – An Example
 The given equation, x3 – 20 = 0 = f(x)

Let, xl = 1, f(xl) = -19 {-ve}


so, xl = 1

Now let, xu = 3, f(xu) = 7 {+ve}


so, xu = 3

Therefore, the root in [1, 3], this is called bracketing.

Iteration, i = 1:
x1 = (xl + xu)/2 = (1 + 3)/2 = 2 and f(x1) = -12 {-ve}
so, xl = 2 and xu = 3
Bisection Method – An Example
Iteration, i = 2:
x2 = (xl + xu)/2 = (2 + 3)/2 = 2.5 and f(x2) = -4.375 {-ve}
so, xl = 2.5 and xu = 3
Iteration, i = 3:
x3 = (xl + xu)/2 = (2.5 + 3)/2 = 2.75 and f(x3) = 797 {+ve}
so, xl = 2.5 and xu = 2.75

Iteration, i = 4:
x4 = (xl + xu)/2 = (2.5 + 2.75)/2 = 2.625 and f(x4) = -1.91 {-ve}
so, xl = 2.625 and xu = 2.75
Iteration, i = 5:
x5 = (xl + xu)/2 = (2.625 + 2.75)/2 = 2.6875 and f(x5) = -0.589 {-ve}
so, xl = 2.6875 and xu = 2.75
Bisection Method – An Example
Iteration, i = 6:
x6 = (xl + xu)/2 = (2.6875 + 2.75)/2 = 2.71875 and f(x2) = 0.0959 {-ve}
so, xl = 2.6875 and xu = 2.71875
Similarly, the iterations are continued until the desired accuracy is
achieved.

Desired Accuracy:
Roots accurate up to n decimal places
Roots accurate up to n digits after decimal
[ n could be 2, 3, 4, 5 …….]

Always take atleast n+1 digits after decimal during


iterations (n+2 digits are recommended)
Bisection Method – An Example
For example, if 3 digits accuracy is desired, it is
recommended to keep 5 digits during each iteration.

Desired accuracy is achieved if during 2 to 3 successive


iterations, the required number of digits remain
unchanged ( 3 is recommended).

For example, if 3 digits accuracy is desired, and in 3


successive iterations, the root approximations are
7.18532,
7.18514 and
7.18509,
then the iterations are stopped.
Bisection Iteration xl xu xi f(xi)

Method – An 1 1 3 2 -12
2 2 3 2.5 -4.375
Example 3 2.5 3 2.75 0.797
In the previous 4 2.5 2.75 2.625 -1.912
example, if the 5 2.625 2.75 2.6875 -0.589
desired accuracy is 6 2.6875 2.75 2.71785 0.0959
4 digits after 7 2.6875 2.71785 2.703125 -0.249
decimal, then we 8 2.703125 2.71785 2.710938 -0.077
can do iterations
9 2.710938 2.71875 2.714844 0.009
and summarize the
10 2.710938 2.714844 2.712891 -0.034
result in a table as
follows: 11 2.712891 2.714844 2.713867 -0.012
12 2.713867 2.714844 2.714355 -0.001
13 2.714355 2.714844 2.7146 0.004
[Note that the
table is generated 14 2.714355 2.7146 2.714478 0.001
as one row for one 15 2.714355 2.714478 2.714417 0.000
complete iteration] 16 2.714417 2.714478 2.714447 0.000
Evaluation of Method

Pros Cons
• Easy • Slow
• Always find root • Know a and b that
• Number of iterations bound root
required to attain an • Multiple roots
absolute error can be • No account is taken of
computed a priori. f(xl) and f(xu), if f(xl)
is closer to zero, it is
likely that root is
closer to xl . 32
How Many Iterations will It Take?
• Length of the first Interval Lo=b-a
• After 1 iteration L1=Lo/2
• After 2 iterations L2=Lo/4

• After k iterations Lk=Lo/2k

Lk
e a  100% ea  es
x
33
• If the absolute magnitude of the error is
es  x 4
 10
100%
and Lo=2, how many iterations will you have
to do to get the required accuracy in the
solution?
4 2
10  k  2 k  2 104  k  14.3  15
2

34
The False-Position Method
(Regula-Falsi)
• If a real root is
bounded by xl and xu of
f(x)=0, then we can
approximate the
solution by doing a
linear interpolation
between the points [xl,
f(xl)] and [xu, f(xu)] to
find the xr value such
that l(xr)=0, l(x) is the
linear approximation
of f(x).

== > Fig. 5.12

35
Procedure
1. Find a pair of values of x, xl and xu such that
fl=f(xl) <0 and fu=f(xu) >0.
2. Estimate the value of the root from the
following formula:

xl f u  xu f l
xi 
fu  fl
and evaluate f(xi).

36
3. Use the new point to replace one of the original
points, keeping the two points on opposite sides of
the x axis.

If f(xi)<0 then xl=xi == > fl=f(xi)

If f(xi)>0 then xu=xi == > fu=f(xi)

If f(xi)=0 then you have found the root and need go


no further!

4. Repeat iterations (steps 2 and 3) until the desired


accuracy is achieved. 37
Regula Falsi Method – An Example
 The given equation, x3 – 20 = 0 = f(x)

Let, xl = 1, f(xl) = -19 {-ve}


so, xl = 1

Now let, xu = 3, f(xu) = 7 {+ve}


so, xu = 3

Therefore, the root in [1, 3], this is called bracketing.

Iteration, i = 1:
fl = f(xl) = -19, fu = f(xu) = 7
x1 = (xl * fu – xu * fl) / (fu – fl) = 2.461538 and f(x1) = - 5.085 {-ve}
so, xl = 2.461538 and xu = 3
Regula Falsi Method – An Example
Iteration, i = 2:
fl = f(xl) = -5.08512, fu = f(xu) = 7
x1 = (xl * fu – xu * fl) / (fu – fl) = 2.68811 and f(x1) = - 0.576 {-ve}
so, xl = 2.68811 and xu = 3
Iteration, i = 3:
fl = f(xl) = - 0.575898, fu = f(xu) = 7
x1 = (xl * fu – xu * fl) / (fu – fl) = 2.711819 and f(x1) = - 0.0574 {-ve}
so, xl = 2.711819 and xu = 3
Iteration, i = 4:
fl = f(xl) = - 0.0573924, fu = f(xu) = 7
x1 = (xl * fu – xu * fl) / (fu – fl) = 2.714162 and f(x1) = - 0.0056 {-ve}
so, xl = 2.714162 and xu = 3
Regula Falsi Method – An Example
Again, if the desired accuracy is 4 digits after decimal, then
we can do iterations and summarize the result in a table as
follows:
Iteration xl f(xl) xu f(xu) xi f(xi)

1 1 -19.000000 3 7 2.461538 -5.085116


2 2.461538 -5.085116 3 7 2.68811 -0.575898
3 2.68811 -0.575898 3 7 2.711819 -0.057392
4 2.711819 -0.057392 3 7 2.714162 -0.005644
5 2.714162 -0.005644 3 7 2.714393 -0.000554
6 2.714393 -0.000554 3 7 2.714415 -0.000054
7 2.714415 -0.000054 3 7 2.714417 -0.000005
8 2.714417 -0.000005 3 7 2.714418 -0.000001

[Note that the table is generated as one row for one


complete iteration]
Regula Falsi Method
• Why this method?
– Faster
– Always converges for a single root.

Always check by substituting estimated root in the original


equation to determine whether f(xi) ≈ 0.

 To check the roots of a polynomial equation using Matlab:


let, f(x) = ax3 + bx2 +cx +d = 0
Matlab command: roots ( [ a b c d ] )

41
Newton-Raphson Method
• Most widely used method (one point method)
• Based on Taylor series expansion:
x 2
f ( xi )  f ( xi 1 )  f ( xi 1 )x  f ( xi 1 )  Ox 3
2!
The root is the value of x i when f(xi )  0
Rearrangin g,
Solve for
 i 1 )( xi  xi 1 )
0  f(xi 1 )  f (x
f ( xi 1 )
xi  xi 1 
f ( xi 1 ) Newton-Raphson formula
42
Newton-Raphson Method
• A convenient method for
functions whose
derivatives can be
evaluated analytically.
• It may not be convenient
for functions whose
derivatives cannot be
evaluated analytically.
Fig. 6.5
43
Newton-Raphson Method – An Example
 The given equation, x3 – 20 = 0

Let, x0 = 3, f(x0) = 7, f’(x0) = 27

Iteration, i = 1:
x1 = x0 – f(x0)/f’(x0) = 3 – 7/27 = 2.74074074

Iteration, i = 2:
x2 = x1 – f(x1)/f’(x1) = 2.74074074 – 0.5875127/22.53498
= 2.7146696
Newton-Raphson Method – An Example
So, if the desired accuracy is 4 digits after decimal, then we
can do iterations and summarize the result in a table as
follows:
Iteration xi-1 f(xi-1) f’(xi-1) xi

1 3 7 27 2.740740741

2 2.740741 0.587512 22.53498 2.714669625


3 2.71467 0.005571 22.10829 2.71441764

4 2.714418 5.17E-07 22.10419 2.714417617

5 2.714418 0 22.10419 2.714417617

[Note that the table is generated as one row for one


complete iteration]
Newton-Raphson Method – Transcendental Equation Example
 The given equation,

Let, x0 = 3, f(x0) = 20.22666, f’(x0) = 19.09554


**Must remember that sinx, cosx, etc. should be calculated in radians.

Iteration, i = 1:

x1 = x0 – f(x0)/f’(x0) = 3 – 20.22666/19.09554
= 1.9407656
Iteration, i = 2:
x2 = x1 – f(x1)/f’(x1) = 1.9407656 – 7.896419/6.602494
= 0.744790406
Newton-Raphson Method – Transcendental Equation
Tabulating the iterations for an accuracy of 4 decimal places:
Iteration xi-1 f(xi-1) f’(xi-1) xi
1 3 20.22666 19.09554 1.940765632
2 1.940766 7.896419 6.602494 0.744790406
3 0.74479 2.783818 2.84123 -0.23500276
4 -0.235 0.557723 1.763082 -0.55133682
5 -0.55134 0.052353 1.428004 -0.58799821
6 -0.588 0.000742 1.387491 -0.58853263
7 -0.58853 1.59E-07 1.386897 -0.58853274
8 -0.58853 7.22E-15 1.386897 -0.58853274

 To check the roots of a transcendental equation using Matlab:


let, ex + sinx = 0
Matlab command: fzero( ‘ exp(x) + sin(x)’ , 3)
The Secant Method
• A slight variation of Newton’s method for
functions whose derivatives are difficult to
evaluate. For these cases the derivative can be
approximated by a backward finite divided
difference.

1 xi  xi 1

f ( xi ) f ( xi )  f ( xi 1 )
xi  xi 1
xi 1  xi  f ( xi ) i  1,2,3, 
f ( xi )  f ( xi 1 )
48
The Secant Method
• Requires two initial
estimates of x , e.g, xo, x1.
However, because f(x) is
NOT required to change
signs between estimates, it is
not classified as a
“bracketing” method.
• In each successive iteration,
• Old xi+1 becomes xi,
• Old xi becomes xi-1
Fig. 6.7
• New xi+1 is then estimated • However, like Newton’s Method,
Convergence is NOT guaranteed
49
for all xo, f(x)
Secant Method – An Example
 The given equation, x3 – 20 = 0

Let, x0 = 3 and x1 = 5, so , f(x0) = 7 and f(x1) = 105

Iteration, i = 1: (xi-1 = 3, fi-1= 7, xi = 5, fi = 105)

x2 = x1 – [f1(x1 – x0)/(f1 - f0)]


= 5 – [105(5 – 3) / (105 – 7)]
= 2.857143
Iteration, i = 2: (xi-1 = 5, fi-1= 105, xi = 2.857143, fi = 3.323615)

x3 = x2 – [f2(x2 – x1)/(f2 - f1)]


= 2.857143 – [3.323615(2.857143 – 5) / (3.323615 – 105)]
= 2.787097
Secant Method – An Example
Tabulating the iterations for an accuracy of 4 decimal places:
i xi-1 f(xi-1) xi f(xi) xi+1 f(xi+1)

1 3 7 5 105 2.857143 3.323615

2 5 105 2.857143 3.32361516 2.787097 1.649913

3 2.857143 3.323615 2.787097 1.649912524 2.718046 0.080317

4 2.787097 1.649913 2.718046 0.080317054 2.714513 0.002108

5 2.718046 0.080317 2.714513 0.002108045 2.714418 2.82E-06

6 2.714513 0.002108 2.714418 2.81542E-06 2.714418 9.89E-11

7 2.714418 2.82E-06 2.714418 9.89111E-11 2.714418 0


Simple Fixed-point Iteration
•Rearrange the function so that x is on the
left side of the equation:
f ( x)  0  g ( x)  x
xi  g ( xi 1 ) xo given , i  1, 2, ...
•Bracketing methods are “convergent”.
•Fixed-point methods may sometime
“diverge”, depending on the stating point
(initial guess) and how the function behaves.
52
Simple Fixed-point Iteration
Example:
f ( x)  x  x  2
2
x0
g ( x)  x  2
2

or
g ( x)  x  2
or
2
g ( x)  1 
x

53
Simple Fixed-point Iteration
Convergence:
• x=g(x) can be
expressed as a pair of
equations:
y1=x
y2=g(x) (component
equations)
• Plot them separately.

Figure 6.2 54
Simple Fixed-point Iteration
Convergence:
• Fixed-point iteration converges if
g ( x)  1 (slope of the line f(x)  x)
•When the method converges, the error is
roughly proportional to or less than the error of
the previous step, therefore it is called “linearly
convergent.”

55
Fixed Point Method – An Example
 The given equation, x3 – 20 = 0
Assuming x ≠ 0, x = √(20/x) , g(x) = √(20/x) , g’(x) = -√(5/x3)
Let, x0 = 3, | g’(x0) | < 1

Iteration, i = 1:

x1 = g(x0) = √(20/3) = 2.581989


Iteration, i = 2:
x2 = g(x1) = √(20/2.581989) = 2.783158
Iteration, i = 3:

x3 = g(x3) = √(20/2.783158) = 2.680687


Fixed Point Method – An Example
Tabulating the iterations for an accuracy of 4 decimal places:
Iteration xi-1 g(xi-1) xi
1 3 2.581989 2.581989
2 2.581989 2.783158 2.783158
3 2.783158 2.680687 2.680687
4 2.680687 2.731442 2.731442
5 2.731442 2.705945 2.705945
6 2.705945 2.718664 2.718664
7 2.718664 2.712297 2.712297
8 2.712297 2.715479 2.715479
9 2.715479 2.713887 2.713887
10 2.713887 2.714683 2.714683
11 2.714683 2.714285 2.714285
12 2.714285 2.714484 2.714484
13 2.714484 2.714384 2.714384
14 2.714384 2.714434 2.714434
15 2.714434 2.714409 2.714409
16 2.714409 2.714422 2.714422
Some practice problems
•Solve the following equations to find a single root accurate up to
4 decimal places:

58