Numerical Methods
Lecturer : M H M R S Dilhani
Module : IS5301 Numerical Methods
Type : Core
No. Credits : 3
Prerequisites : None
Objective:
Provide an introduction to the students with numerical
approximation tools for solving engineering and science
problems.
Intended Learning Outcomes:
After following this module, students will be able to do the following.
• Understand of numerical approximation techniques.
• Use numerical techniques to solve different engineering and science
problems.
• Understand why numerical methods work and their limitation.
• Use of Matlab for solving numerical problems.
IS 5301 - M H M R S Dilhani 3
Mode of Assessment:
1. Continuous assessment 30%
• Assignments 20%
• Quizzes 10%
• 5 marks penalty for less than 80% class participation
2. End Semester Examination 70%
• Necessary condition to pass the module:
• Earn at least 50% of the marks of Assignments and Quizzes
IS 5301 - M H M R S Dilhani 4
Introduction
• In Linear Algebra, you learned that solving systems of linear
equations can be implemented by using row reduction as an algorithm.
• However, when these methods are not successful, we use the concept
of numerical methods.
• Numerical methods are used to approximate solutions of equations
when exact solutions can not be determined via algebraic methods.
• They construct successive approximations that converge to the exact
solution of an equation or system of equations.
IS 5301 - M H M R S Dilhani 5
Introduction
• Numerical method provide a way for the engineer to translate the language
of mathematics and physics into information that may be used to make
engineer decisions.
• This translation is implemented so that the calculation may be done by
machines (Computers).
• The types of problems that you encounter as an engineer may involve a
wide variety of mathematical phenomena, and hence it will benefit you to
have an equally wide range of numerical methods with which to approach
some of these problems.
• This course will provide you with an introduction to several of those
numerical methods which you may then find opportunity to practice later in
the curriculum.
IS 5301 - M H M R S Dilhani 6
Write down a few examples in which numerical
methods offer a net benefit to the engineer
IS 5301 - M H M R S Dilhani 7
This course consists of six major units
1) Numerical Solution of nonlinear equations
2) Interpolating techniques
3) Numerical solution of systems of linear equations
4) Numerical differentiation and integration
5) Numerical solution of ordinary differential equations
6) Numerical solutions to partial differential equations
IS 5301 - M H M R S Dilhani 8
1. Numerical solution of nonlinear equations
• The goal of this topic is to examine three different numerical methods
that are used to solve systems of nonlinear equations in several
variables.
Consider equation of the form = , is a nonlinear function.
Example:
= + + =
�
= + cos − =
In general we can write
= + + + =
This is called as polynomial function.
IS 5301 - M H M R S Dilhani 9
+ + = Quadratic equation
− ± −
= There is
a
+ + + = Cubic equation formula
+ + + + = Biquadratic equation
+ + + + + = There is no formula
IS 5301 - M H M R S Dilhani 10
• One of the first numerical methods developed to find the root of a
nonlinear equation = was the bisection method (also called
binary-searched method).
Theorem: Intermediate Value Theorem (IVT)
An equation � = , where f(x) is a real continuous function has
at least one root between and if < .
IS 5301 - M H M R S Dilhani 11
• If > , there may or may not be any root between
and .
If the function does not change sign between the two points, roots
of the equation = may still exist between the two points.
IS 5301 - M H M R S Dilhani 12
If the function does not change sign between the two points, there
may not be any roots of the equation = between the two points.
IS 5301 - M H M R S Dilhani 13
• If < , then there may be more than one root between
and . So the theorem only guarantees one root between
and .
If the function changes sign between the two points, more than one
root for the equation = may exist between the two points.
IS 5301 - M H M R S Dilhani 14
1.1 Bisection method
• The bisection method in a mathematics is a root-finding method which
repeatedly bisects an interval and then selects a subinterval in which a
root must lie for further processing.
• It is a very simple and robust method, but the main disadvantage is
that convergence is slow.
• In order to use the bisection method, we have to have the condition,
< for some and .
IS 5301 - M H M R S Dilhani 15
• If we have < , then we have at least one root in , .
Proof:
Taking = as the approximate root
, = , i� <
, = , i� >
IS 5301 - M H M R S Dilhani 16
• If = ,� is the root and we stop the process.
• Again define the new approximate root ,
=
, = , i� <
, = , i� >
So, if we continue this process, then we have the root � in
, = , i� <
, = , i� >
� �
Where =
IS 5301 - M H M R S Dilhani 17
The root will be , if = . This is the algorithm to
find the root � of = .
Now,
− = | − |
− = | − |
− = | − |
�
�
IS 5301 - M H M R S Dilhani 18
If � is the error at this stage, � < | − |
If we need an accuracy of �, then we get it by maintaining
� <�
i.e. � − <�
>
�
log log �
or >
log
IS 5301 - M H M R S Dilhani 19
Example:
Find the root of = − by using the bisection method. Let the
starting interval is [ , ] and error is � = . .
= −
∴ =− , =
∴� � < , has opposite signs.
So, we can use bisection method to find the root, as has at least
one root in between 2 and 3.
IS 5301 - M H M R S Dilhani 20
Then, = = = .
= . − =6.25-5=1.25
Update with . So the new interval = [ , . ]
Error at this stage = − = . − = .
IS 5301 - M H M R S Dilhani 21
Lets do it in a tabular format
Update New −
2 3 -1 4 2.5 1.25 =m 0.5
= .
2 2.5 -1 1.25 2.25 0.0625 = . 0.25
2 2.25 -1 0.0625 2.125 -0.4843 = . 0.125
2.125 2.25 -0.4843 0.0625 2.1875 -0.2148 = . 0.0625
2.1875 2.25 -0.2148 0.0625 2.21875 -0.07715 = . 0.03125
2.21875 2.25 -0.07715 0.0625 2.2344 -0.007568 = . 0.0156
2.2344 2.25 -0.007568 0.0625 2.2422 0.02746 = . 0.0078
2.2344 2.2422 -0.007568 0.02746 2.2383 0.009986 = . 0.0039
IS 5301 - M H M R S Dilhani 22
Thus, with the seventh iteration, we note that the final interval, [2.2344,
2.25] has a width less than 0.01 and . < . , and therefore,
we choose = 2.2422 to be our approximation of the root.
Example 2:
Suppose you have to solve the function, − = , in the interval
− . , . using the bisection method. Then find the number of times
you should bisect the interval to have an accuracy of 0.0001.
IS 5301 - M H M R S Dilhani 23
Advantages of Bisection method:
a) The bisection method is always convergent since the method
brackets the root, the method is guaranteed to converge
b) As iterations are conducted, the interval gets halved. So one can
guarantee the error in the solution of the equation.
Drawbacks of Bisection method:
a) The convergence of the bisection method is slow as it is simply
based on halving the interval.
b) If one of the initial guesses is closer to the root, it will take larger
number of iterations to reach the root.
IS 5301 - M H M R S Dilhani 24
c) If a function is such that it just touches the x-axis, such as
= = it will be unable to find the lower guess, , and
upper guess, , such that < .
The equation = = has a single root at = that can not be
bracketed.
IS 5301 - M H M R S Dilhani 25
d) For functions where there is a singularity (a point where the
function becomes infinite) and it reverses sign at the singularity, the
bisection method may converge on the singularity.
The equation = = has no root, but changes sign.
�
IS 5301 - M H M R S Dilhani 26
An example includes, = , where =− , = are valid
�
initial guesses which satisfy < . However, the function is
not continuous and the theorem that a root exists is also not applicable.
IS 5301 - M H M R S Dilhani 27
1.2 Newton-Raphson method
We have = . If is a value close to the root �, then the point p
at which the tangent drawn to the curve of y = cuts the axis is
closer to � than .
=
=
T������
�
�
=
,
IS 5301 - M H M R S Dilhani 28
So the coordinate , of that point could be taken to be a better
approximation to the root �.
=
� � ′ � �
Now, = = −
� � �′ �
Now, taking we can do the same and get
= − ′
IS 5301 - M H M R S Dilhani 29
In general we get a scheme of approximation to the root � by
= − ′
T������
IS 5301 - M H M R S Dilhani 30
Another way of obtaining Newton-Rapson method
Taylor’s Series
− ′
− −
= + + " + +
! ! !
If � is the root of = , th�n �xpanding about a point which is
near �, using Taylor’s series,
�− ′
�− �−
� = + + " + +
! ! !
� =0, so we have
�− ′
�− �−
= + + " + +
! ! !
IS 5301 - M H M R S Dilhani 31
�− is the error initial stage.
Define � = � −
So,
′
� �
= +� + " + +
! !
Since � is small, we can neglect � , � , � , …
∴ ≈ +� ′
� �
� ≈ −
�′ �
� �
∴�≈ − ′ =
� �
IS 5301 - M H M R S Dilhani 32
As is a better approximation to � than , we can obtain a closer
value to � using .
� �
= − and we continue.
�′ �
So we get,
= − ′
=
IS 5301 - M H M R S Dilhani 33
Example 1:
Solve the nonlinear equation, − . + . using the Newton-
Raphson method with a good initial value (starting value) .
Put = , = . > &
= , =− . <
So, we shall take = = = .
Now, = − . + .
′ = − .
IS 5301 - M H M R S Dilhani 34
− . + .
= − ′
= −
− .
n � � ′ � �
0 0.5 -0.684375 -2.6125 0.238038
1 0.238038 0.033675 -2.79541 0.250085
2 0.250085 7.48E-06 -2.79413 0.250087
3 0.250087 4.2E-13 -2.79413 0.250087
∴ � = 0.250087
IS 5301 - M H M R S Dilhani 35
Example 2:
Solve + + = using N-R method with the initial value
= . .
= + + , ′ = +
+ +
= − ′
= −
-0.88125 +
-0.94063
-0.97031
-0.98516
-0.99258
-0.99629
-0.99814
-0.99907
-0.99954
-0.99977
-0.99994
-0.99988
-0.7625
-0.525
-0.05
�� ��
= − =−
��
IS 5301 - M H M R S Dilhani 36
Note:
+ + = has a multiple root at = − . Any equation =
which has a multiple roots at = � could be written as;
� = �−� � =
where is a function such that � ≠ . m is the order of the
multiple root or the multiplicity. SO, if we know the multiplicity of the
root, then we can modify N-R method as :
� ��
= −
�′ ��
IS 5301 - M H M R S Dilhani 37
Suppose you have function, = and � is a root of this function.
If, ′ � ≠ → � is a simple root.
′ � =
If → � is a double root.
" � ≠
IS 5301 - M H M R S Dilhani 38
Example:
= + =
We can re-write the function as,
= + =
= is a multiple root.
′ = + → ′ =
" = + → ′ = ≠
Therefore, Multiplicity =2 (double root)
IS 5301 - M H M R S Dilhani 39
Advantages of N-R method
1) Convergence is faster (quadratic convergence)
2) Required only one guess
Disadvantages of N-R method
1) Division by zero
Example: = − . + . ∗ =
− . + . ∗
= −
− .
If = or = . the denominator will equal to zero.
IS 5301 - M H M R S Dilhani 40
2) Oscillations near local maximum and minimum
Consider a function = + = which has no real roots. So we
can see the tangents are oscillating near local minimum or maximum
points.
− .75 − . .
− .
41
IS 5301 - M H M R S Dilhani
4) Root jumping
In some cases where the function is oscillating and has a number
of roots, one may choose an initial guess close to a root. However, the
guess may jump and converge to some other root.
� = ��� � =
.
. .
IS 5301 - M H M R S Dilhani 42
1.3 Fixed Point Iteration Method
If the equation = is rearranged in the form,
�= �
Then an iterative method may be written as,
� = � , = , , ,…
where is the number of iterative steps and � is the initial guess. This
method is called the ‘Fixed point iteration’ or ‘Successive
Substitution method’
IS 5301 - M H M R S Dilhani 43
Definition: Fixed Point Iteration Method
If � = � , then we say � is a fixed point for the function � .
Condition for convergence
If and ′ are continuous on an interval [ , ] about their root �
of the equation � = � , and if ′ < for all � in the interval
, , then the fixed point iterative process, � = � , =
, , , … will converge to the root X = � for any initial approximation
X within the interval , .
IS 5301 - M H M R S Dilhani 44
Example 1:
Use the fixed point iterative method to determine a solution accurate
within for − − = , on , .
= − − =
We can rewrite the function as ;
− − =
= +
= + / ∈[ , ]
IS 5301 - M H M R S Dilhani 45
Then = + / ∈ [ , ] for all ∈ [ , ].
′
= + . =
+
′ = .
′
. = .
′ = .
| ′ |< for all ∈ [ , ].
∴ has an unique fixed point in [ , ].
IS 5301 - M H M R S Dilhani 46
Let =
= �
∴ = .
= .
= .
= .
= .
= .
− = . <
= .
= .
∴ = . (correct to 2 decimal places)
IS 5301 - M H M R S Dilhani 47
Example 2:
Find a root of − − = by using fixed point iteration method.
Lets consider
= and = .
−
IS 5301 - M H M R S Dilhani 48
∴ = = ; = , , ,…
−
0 1 2 3 4 5 6 7 8 9
0.833333
-3.51792
-2.49835
-0.28419
-2.47164
-0.29241
-2.46913
-0.1098
-0.2932
2
10 11 12 13 14 15 16 17
-2.46889
-0.29327
-2.46886
-0.29328
-2.46886
-0.29328
-2.46886
-0.29328
So the iterative process with gone into an infinite loop without
convergence.
IS 5301 - M H M R S Dilhani 49
Why it diverges?
=
�
−
′ =
−
| ′ . |= . >
′ ∴ diverges
| |= . >
IS 5301 - M H M R S Dilhani 50
Now lets consider another function for .
= + /
/
∴ = = + , = , , ,…
Let the initial guesses be 0, 1 and 2.
0 1 2 3 4 5
� 0 1.495349 1.681304 1.700534 1.702486 1.702684
� 1 1.626577 1.694943 1.701919 1.702627 1.702698
� 2 1.732051 1.705671 1.703007 1.702737 1.702709
IS 5301 - M H M R S Dilhani 51
So, the initial process with is converging to 1.702 with any initial
guess.
The iterative process converges very quickly to the root, which is the
intersection of = and =
Why it converges?
= + /
′ = /
+
| ′ |= . <
| ′ |= . < ∴ converges
IS 5301 - M H M R S Dilhani 52
Lets consider another function for
+
=
+
∴ = = ; = , , ,…
Let the initial guess be 1.7
0 1 2 3 4 5 … 44
1.7 1.704868 1.700985 1.70408 1.701611 1.70358 … 1.702706
So, the iterative process of with any initial guess will converge,
but very slowly.
IS 5301 - M H M R S Dilhani 53
Graphical interpretation of Fixed point iteration procedure
Monotone Convergence (when < ′ <
Oscillating Convergence (when − < ′ <
Monotone Divergence (when ′ >
Oscillating Divergence (when ′ >−
IS 5301 - M H M R S Dilhani 54
IS 5301 - M H M R S Dilhani 55
