LEC #3: Numerical Differentiation

Numerical Differentiation is a method used to approximate the value of a derivative

over a continuous region [a,b].
Let f(x) is a continuous function with step size h. There are forward, backward and
centered difference methods to approximate the derivatives of f(x) at a point x i.

3.1 Forward Difference Approximation of the First Derivative

We know
f x x f x
f x lim
x 0 x
For a finite ' x' .
f x x f x
f x
x

f (x)

x x x x

Figure 3.1: Graphical representation of forward

difference approximation of first derivative
So if you want to find the value of f x at x xi , we may choose another point

' x ' ahead as x xi 1 . This gives

f xi 1 f xi
f xi
x

f xi 1 f xi
Where x xi 1 xi
xi 1 xi

Numerical Analysis /Lec. 5 - 24 -

Fourth Stage
Example 3.1
14 104
The velocity of a rocket is given by t 2000 ln 9.8t , 0 t 30
14 104 2100t

Where ' ' is given in m/s and 't ' is given in seconds.
Use forward difference approximation of the first derivative of t to calculate the

acceleration at t 16 s . Use a step size of t 2s .

Solution
ti 1 ti
a ti
t
ti 16
t 2

ti 1 ti t 16 2 18

18 16
a 16
2

14 10 4
18 2000 ln 9.8 18 453.02 m / s
14 10 4 2100 18

14 104
16 2000 ln 9.8 16 392.07 m / s
14 10 4 2100 16

Hence
18 16 453.02 392.07
a 16 30.475m / s 2
2 2

The exact value of a 16 can be calculated by differentiating

d 14 10 4 14 10 4 2100t d 14 10 4
at 2000 ln 9.8t 2000 9.8
dt 14 10 4 2100t 14 10 4 dt 14 10 4 2100t

14 10 4 2100t 14 10 4
2000 1 2
2100 9.8 29.674m / s 2
14 10 4 14 10 4
2100t

The absolute relative true error is

True Value - Approximate Value 29.674 30.475
t 100 100 2.6993%
True Value 29.674

Numerical Analysis /Lec. 3 - 25 -

Fourth Stage
3.2 Backward Difference Approximation of the First
Derivative We know
f x x f x
f x lim
x 0 x
For a finite ' x' ,
f x x f x
f x
x
If ' x' is chosen as a negative number,
f x x f x
f x
x
f x f x x
f x
x
This is a backward difference approximation as you are taking a point backward
from x . To find the value of f x at x xi , we may choose another point ' x'

behind as x xi 1 . This gives

f xi f xi 1
f xi
x

f xi f xi 1
where x xi xi 1
xi xi 1

f (x)

x
x x x

Figure 3.2 Graphical representation of backward

difference approximation of first derivative

Numerical Analysis /Lec. 3 - 26 -

Fourth Stage
Example 3.2
The velocity of a rocket is given by
14 104
t 2000 ln 9.8t , 0 t 30
14 10 4 2100 t

Use backward difference approximation of the first derivative of t to calculate the

acceleration at t 16 s . Use a step size of t 2s .

Solution
ti ti 1
at
t
ti 16
t 2
ti 1 ti t 16 2 14
16 14
a 16
2

14 10 4
16 2000 ln 9.8 16 392.07 m / s
14 10 4 2100 16

14 10 4
14 2000 ln 9.8 14 334.24m / s
14 10 4 2100 14

16 14 392.07 334.24
a 16 28.915m / s 2
2 2
The absolute relative true error is
29.674 28.915
t 100 2.557%
29.674

Numerical Analysis /Lec. 3 - 27 -

Fourth Stage
3.3 Central Difference Approximation of the First Derivative
As shown above, both forward and backward divided difference approximation of the
first derivative are accurate on the order of 0 x . Can we get better approximations?

Yes, another method to approximate the first derivative is called the Central
difference approximation of the first derivative.
From Taylor series
f xi f xi
f xi 1 f xi f xi x x
2
x
3
(1)
2! 3!
f xi 2 f xi 3
f xi 1 f xi f xi x x x (2)
2! 3!
Subtracting equation (2) from equation (1)
2f xi 3
f xi 1 f xi 1 f xi 2 x x
3!
f xi 1 f xi 1 f xi 2
f xi x
2 x 3!
f xi 1 f xi 1 2
f xi 0 x
2 x

f xi 1 f xi 1
f xi
2 x

Hence showing that we have obtained a more accurate formula as the error is of the
order of 0 x 2 .

f (x)

x x x x x

Figure 3.3 Graphical Representation of central

difference approximation of first derivative.

Numerical Analysis /Lec. 3 - 28 -

Fourth Stage
Example 3.3
The velocity of a rocket is given by

14 10 4
t 2000 ln 9.8t ,0 t 30 .
14 10 4 2100t

Use central divided difference approximation of the first derivative of t to

calculate the acceleration at t 16s . Use a step size of t 2s .

Solution
ti 1 ti 1
a ti
2 t
ti 16
ti 1 ti t 16 2 18
ti 1 ti t 16 2 14
18 14 18 14
a 16
22 4

14 10 4
18 2000 ln 9.8 18 453.02 m / s
14 10 4 2100 18

14 10 4
14 2000 ln 9.8 14 334.24 m / s
14 10 4 2100 14

18 14 453.02 334.24
a 16 29.695m / s 2
4 4
The absolute relative true error is
29.674 29.695
t 100 0.070769%
29.674
The results from the three difference approximations are given in Table 1.
Table 1 Summary of a 16 using different divided difference approximations.

Type of Difference a 16
t %
Approximation m / s2
Forward 30.475 2.6993
Backward 28.915 2.557
Central 29.695 0.070769
Clearly, the central difference scheme is giving more accurate results because the
order of accuracy is proportional to the square of the step size.

Numerical Analysis /Lec. 3 - 29 -

Fourth Stage
3.4 Higher Order Derivatives

Example: Second order derivative:

Note that for the centered formulation, it is a derivation of a derivative:
f xi 1 f xi f xi f xi 1
x x f xi 1 2 f xi f xi 1
f '' x
x ( x) 2
Forward f '' x
f xi 2 2 f xi 1 f xi
( x) 2
Backward f '' x
f xi 2 f xi 1 f xi 2
( x) 2
Centered f '' x
f xi 1 2 f xi f xi 1
( x) 2

I) Forward Difference Methods

First Derivative
f xi 1 f xi
f ' ( xi )
x
Second Derivative
f xi 2 2 f xi 1 f xi
f ' ' ( xi )
( x) 2

Third Derivative

3 f xi 3 3 f xi 2 3 f xi 1 f xi
f ( xi ) 3
( x)

Fourth Derivative

4 f xi 4 4 f xi 3 6 f xi 2 4 f xi 1 f xi
f ( xi )
( x) 4

II) Backward Difference Methods

First Derivative
f xi f xi 1
f ' ( xi )
x
Second Derivative
f xi 2 f xi 1 f xi 2
f ' ' ( xi )
( x)2
Numerical Analysis /Lec. 3 - 30 -
Fourth Stage
Third Derivative

3 f xi 3 f xi 1 3 f xi 2 f xi 3
f ( xi )
( x)3

Fourth Derivative

4 f xi 4 f xi 1 6 f xi 2 4 f xi 3 f xi 4
f ( xi )
( x) 4

III) Central Difference Methods

First Derivative
f xi 1 f xi 1
f ' ( xi )
2 x
Second Derivative
f xi 1 2 f xi f xi 1
f ' ' ( xi )
( x)2

Third Derivative

3 f xi 2 2 f xi 1 2 f xi 1 f xi 2
f ( xi )
2( x ) 3

Fourth Derivative

4 f xi 2 4 f xi 1 6 f xi 4 f xi 1 f xi 2
f ( xi )
( x)4

Numerical Analysis /Lec. 3 - 31 -

Fourth Stage