WEEK 1

Introduction to Numerical Methods

 Mathematical modeling
 Approximation and round off errors
 Truncation errors and Taylor Series 2
LESSON OUTCOMES
At the end of this topic, the students will be able:

• To describe numerical techniques as

compared to analytical methods
• To use Taylor series expansion to approximate
a function
• To perform error analysis associated with
numerical methods
 Why Numerical Method?

• Could handle large systems of equations,

nonlinearity and complex geometries that is not
common
• It provide approximate solutions to many of the
engineering problems.

4
 What is Numerical Method
• Powerful analysis tool in problem solving and
understanding problem in mathematical language
• Techniques by which mathematical problems are
formulated, so that they can be solved with
arithmetic operations
• The role of numerical method in solving engineering
problem:
PROBLEM
INTERPRETATION
FORMULATION SOLUTION
The results obtained can
Fundamental laws are used to Suitable numerical methods
then be used to
develop mathematical are then selected to solve
predict/analyze/understand
equations that can represent the mathematical equations
the specific problem better
the specific problem
5
 Mathematical modeling
Is the use of mathematics to
• Describe real world phenomena
• Investigate important questions about the observed world
• Explain real world phenomena
• Test ideas
• Make predictions about the real world

• The real world refers to

• Engineering Physics
• Physiology Ecology
• Wildlife management Chemistry
• Economics Sports
• Etc

6
• A mathematical model is represented as a functional
relationship of the form

7
• Dependent variable
Observed behaviour/state/phenomenon of a system
Characteristic that reflects behaviour or state of the system
i.e. y, f(x), f(t)
• Independent variable
Dimension that determine a system i.e. time, t , x
• Parameter
Quantity that serves to relate to functions and variables
Reflective of the system’s properties or composition
• Forcing functions
External influence that acts on system i.e. acceleration gravity, g

8
 Example

Apply Newton’s second law, F = ma

And also can write as dv

m F
dt
or Eq. relates a linear position x
d 2x (dependent variable) to the applied
m 2 F force, F (forcing function) and the
dt time, t (independent variable). The
mass, m is the only parameter in
the above model.
9
Example of mathematical modeling

10
 Example
Assume that interested to predict the velocity of the falling parachutist with time

 Use fundamental knowledge to find a

mathematical equation correlates the velocity
to the various forces acting on the parachutist

Newton’s 2nd law of Motion

“the time rate change of momentum of a
body is equal to the resulting force acting on
it.”
The model is formulated as

F = ma

F=net force acting on the body (N)

m=mass of the object (kg)
a=its acceleration (m/s2)

11
 dv
acceleration a 
dt
F  FD  FU
dv F
 FD  downwardforceof gravity
dt m
FU  upward forceof air resistance

FD  mg

FU   cv

Model relates acceleration of falling

object to the forces acting on it,

dv mg  cv c
 g v
dt m m
(differential equation)

12
 Exact or analytical solution: it exactly satisfies the original equation
forcing functions t,s v,(m/s)
dependent variable 0 0.00

2 16.40

4 27.77

6 35.64

8 41.10

10 44.87

12 47.87

 53.39

parameter
independent variable

Analytical solution of the parachutist

13
 Unfortunately, there are many mathematical
models that cannot be solved exactly.

Numerical solution that approximates the exact solution

14
 dv mg  cv c
 g v
dt m m

dv v v( t  v( t )
i 1 ) i
 
dt t ti 1  ti

v( t  v( t )
i 1 ) i c
g v(t )
ti 1  ti m i

 c 
v(ti1 )  v(ti )   g  v(ti ) (ti 1  ti )
 m  The use of finite difference to approximate the
first derivative of v with respect to t

Approximate or
numerical solution
15
 Numerical solution

t,s v,(m/s)

0 0.00

2 19.60

4 32.00

6 39.85

8 44.82

10 47.97

12 49.96

 53.39

 c  Comparison between the exact and

v(ti1 )  v(ti )   g  v(ti ) (ti 1  ti ) numerical solution
 m 

16
Approximation and Roundoff Errors
• Significant figures
Numbers to be used in confidence

98 2 significant figures

98.09 4 significant figures

0.0098 2 significant figures

17
Important of significance figures in numerical methods:

• Numerical methods yield approximate results, therefore,

need to develop criteria to specify the confident in
approximate result.

• Although quantities such as , e, or √7 represent specific

quantities, they cannot be expressed exactly by a limited
number of digits. Computers retain only a finite number of
significant figures.
 = 3.141592653589793238462643…

18
 • Accuracy and Precision How closely a computed
or measured value agree
Increasing accuracy with the true value

Increasing precision

Inaccurate & imprecise accurate & imprecise

How closely individual

computed or measured
Inaccurate & precise accurate & precise
values agree with each
other

19
 • Error definitions
Truncation errors ~ result
when approximations are
used to represent exact
Error mathematical procedures

Round-off errors ~ result

True value when numbers having a
limited significant figures
are used to represent
exact numbers

Approximation
value
True value = Error + Approximation value

20
 True value = error + approximation value (1)

True error (Et)= true value – approximation (2)

(3)

• εt designates true percent relative error

21
 Calculation of errors
True value of length of a bridge is 10,000 cm.
When you measure, the length recorded is
9,999 cm. Compute the true error and true
percentage relative error of the bridge.

22
Answer: 1 cm and 0.01%
However, in actual situation, true value is rarely available.
Therefore, need to estimate the true value  approximation

(4)

In numerical method, iterative approach is used to

compute answer, in which error is estimated as the
difference between previous and current approximations.
(5)

 The signs of error can be negative or positive,

 Absolute value of error, |εa| need to be lower than
prespecified percent tolerance, εs

n is significant figures.
23
Error estimates for iterative method
Suppose that we have exponential function as,

Starting with the simplest version, ex=1, add

terms to estimate e0.5. Compute true (εt) and
approximate error (εa) after each term is added
until |εa| falls below εs , conforming to 3
significant figures. Note that true value of e0.5
is 1.648721.
24
• Answer

25
• Round-off errors Results when numbers having limited significant
figures are used to represent exact numbers

ln 2 = 0.693 147 180 559 945 309 41...

A device only shows 8
significant numbers, so
round-off error is discrepancy
introduced by omission of
significant figures.
Round-off error for this case is
0.00000000055994530941
26
• Other example of roundoff error

27
 Truncation errors and Taylor series
• Truncation errors
• Truncation error is the discrepancy introduced by the fact that
numerical methods may employ approximations to represent exact
mathematical operations and quantities.

• Truncation error are errors resulted from using an approximation

in place of an exact mathematical procedure.

• The difference between the calculated value using exact

mathematical equation and approximation mathematical equation.

28
• Taylor series
Provides a means to predict a function value at one point in terms of the
function value and its derivative at another point

Zero order

First order

Second
order

nth order

29
Taylor series by defining a step size h = xi+1 - xi

(6)

f ( n 1) ( ) n 1
Rn  h (7)
(n  1)!

30
31
• Remainder for the Taylor series Expansion
 Remainder term, Rn, accounts for all terms from (n+1) to infinity.
f ( n 1) ( ) n 1
Rn  h
(n  1)!
 It also usually expressed as:
Rn  O(h n1 )
 Let say we truncated the Taylor series expansion after zero-order term to
yield f ( xi1 )  f ( xi )
 Remainder term, Rn for zero order version
f 3( xi ) 3
f ' '( xi )
R0  f '( xi ) h  h  2
h  ...
2! 3!
 Let truncate the remainder itself,
R0  f '( xi ) h
 This result is still inexact because neglected second and higher order terms.
32
Remainder for the Taylor series Expansion

33
 Alternative simplification that tranforms the approximation into an
equivalence based on graphical insight
 derivative mean-value theorem states that if a function f(x) and its
derivative are continous over interval from xi to xi+1,
 there exist at least one point on the function that has a slope, designated by
f’(ξ), parallel to line joining f (xi) and f(xi+1)

Thus,

So,
Zero order version

First order version

34
• Numerical
differentiation

Forward finite divided difference

Backward finite divided difference

Centered finite divided difference

35
Forward finite divided difference approximation of first derivative

(14) Where,

(15)

Backward finite divided difference approximation of first derivative

f(xi )  f(xi 1 ) R1
f'(xi )   Where, h  xi  xi 1
h h (16)
f
f'(xi )  i  O(h)
h
Centered finite divided difference approximation of first derivative

(17)

Where, h  xi 1  xi  xi  xi 1

36
 Example

Use forward and backward difference approximations of O(h) and a centered

difference approximation of O(h2) to estimate the first derivative of

f(xi )  0.1x 4  0.15x3  0.5x 2  0.25x  1.2

at x = 0.5 using a step size h = 0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.

37
Exercise

Use forward and backward and a centered difference to estimate the first
derivative of the function

f(xi )  0.5x3  0.1x 2  0.8x  0.7

at x = 0.5 using a step size h=0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.
Ans:
h=0.5
FDM:1.525 41.9%
BDM:0.875 18.60%
CDM:1.200 11.63%
h=0.25
FDM:1.26875 18.02%
BDM:0.94375 12.21%
CDM:1.10625 2.91%

38
• Higher derivatives

• Second forward finite difference approximation of higher derivatives

• Second backward finite difference approximation of higher derivatives

• Second centred finite difference approximation of higher derivatives

39
• Error propagation
This section is to study how errors in numbers can
propagate through mathematical functions. If we multiply
two numbers that have errors, we would like to estimate
the error in the product.

If a function f is dependent on
(a) a single independent variable x : f(x)
(b) two independent variables x and y : f(x, y)
(c) several independent variables x1, x2, x3, ... ,xn : f(x1 , x2 ,..., xn ).

40
Function of a single variable.
Let x be the true value and
x* be an approximate value of x

Then, TSE for f(x) computed near f(x*) is given by

f ' ' ( x*)
f(x)  f(x*)  f'(x * )( x  x*)  ( x  x*)2  ...
2
Truncating after the first derivative term and rearranging the remaining terms
to give
f(x)  f(x*)  f ' (x*)( x  x*)
(21)
f(x*)  f ' (x*) x *
where
f ' (x*)  f ( x)  f ( x*) is an estimate of the error of the function
x*  x  x * is an estimate of the error of independent variable x

Eq.(21) provides 2 capabilities:

1. to approximate the error in f(x) knowing its derivative.
2. to approximate the error in the independent variable x.
41
 Example

Given a value of x* = 2.5 with an error of Δx* = 0.01, estimate the

resulting error in the function, f(x)=x3.

Solution
f(x*)  f ' (x*) x *
So,
f(x*)  3(2.5) 2 (0.01)  0.1875
Because f (2.5)  15.625, it can predict that
f( 2.5 )  15.625  0.1875

Or the true value lies between 15.4375 and 15.8125. In fact, if x ~2.49,
f(x) could be 15.4382 and if x ~ 2.51, it would be 15.8132.
The first order error analysis provides a fairly close estimate of the true error.

42
Exercise

Knowing a value of x* = 2.0 with an error of Δx* = 0.01, estimate the resulting
error in the function

f(x) = 0.5x3-0.1x2+0.8x-0.7

Ans: f(2.0)=4.5 ± 0.064

43
Function of a more than One variable.

(22)

(23)

Refer section 4.2.2

for examples

44
• Relative error

Refer section 4.2.3

for example

• Condition number

𝑥 = 𝑎𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑖𝑜𝑛 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑥
𝑓 𝑥 = 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑤𝑖𝑡ℎ 𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝑜𝑓 𝑥

45
• Condition no equals 1 indicates that function’s
relative error is identical to the relative error in
x
• Condition no greater than 1 indicates relative
error is amplified.
• Condition no less than 1 indicates relative
error is attenuated.
• Function with very large values are said to be
ill-conditioned.

46
• Total Numerical Error
Total numerical errors = truncation error + round off error
Roundoff error ↓ by increase no. of significant figures or
reduce no. of computation in analysis
Truncation error ↓ by decreasing step size (h) or increase
no. of computation in analysis

47
Control numerical error
 avoid subtract 2 nearly equal numbers to
avoid loss of significance
 Use Taylor series for truncation and roundoff
error analysis
 Perform numerical experiments
- repeat computation with different step size or method
and compare results

48