Numerical Methods

Numerical Differentiation
Lecture 23

1
 Lecture 23
Numerical
Differentiation

 First order derivatives

 High order derivatives
 Richardson Extrapolation
 Examples

2
Motivation
 How do you evaluate the Time Displacement

derivative of a tabulated (second) (meters)

function. 0 30.1

 How do we determine the

5 48.2
velocity and acceleration
from tabulated 10 50.0
measurements.
15 40.2

3
Recall
df f ( x  h)  f ( x )
 lim
dx h0 h

Taylor Theorem :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3
f ( x  h)  f ( x )  f ' ( x ) h    O(h 4 )
2! 3!
n
E  O(h )   real , finite C , such that : E  C h
n

E is of order h n  E is approaching zero at rate similar to h n

4
Three Formula
df ( x) f ( x  h)  f ( x)
Forward Difference : 
dx h

df ( x) f ( x)  f ( x  h)
Backward Difference : 
dx h

df ( x) f ( x  h)  f ( x  h)
Central Difference : 
dx 2h

Which method is better? How do we judge them?

5
The Three Formulas

6
Forward/Backward Difference
Formula
Forward Difference : f ( x  h)  f ( x )  f ' ( x ) h  O ( h 2 )
 f ' ( x) h  f ( x  h)  f ( x)  O ( h 2 )
f ( x  h)  f ( x )
 f ' ( x)   O ( h)
h
_______________________________________________________
Backward Difference : f ( x  h)  f ( x )  f ' ( x ) h  O ( h 2 )
 f ' ( x ) h  f ( x )  f ( x  h)  O ( h 2 )
f ( x )  f ( x  h)
 f ' ( x)   O ( h)
h

7
Central Difference Formula
Central Difference :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 3) ( x ) h 3
f ( x  h)  f ( x  h )  2 f ' ( x ) h  2  ...
3!
f ( x  h)  f ( x  h)
 f ' ( x)   O(h 2 )
2h

8
The Three Formula
(Revisited)
df ( x) f ( x  h)  f ( x)
Forward Difference :   O ( h)
dx h

df ( x) f ( x)  f ( x  h)
Backward Difference:   O ( h)
dx h

df ( x) f ( x  h)  f ( x  h)
Central Difference :   O (h 2 )
dx 2h

Forward and backward difference formulas are comparable in accuracy.

Central difference formula is expected to give a better answer.

9
Higher Order Formulas
f ( 2) ( x)h 2 f (3) ( x) h3 f ( 4) ( x)h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2) ( x) h 2 f (3) ( x)h3 f ( 4) ( x)h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2) ( x ) h 2 f ( 4) ( x ) h 4
f ( x  h)  f ( x  h)  2 f ( x )  2 2  ...
2! 4!
f ( x  h )  2 f ( x )  f ( x  h)
 f ( 2) ( x )  2
 O ( h 2
)
h

f ( 4) ( )h 2
Error  
12
10
Other Higher Order
Formulas
f ( x  h)  2 f ( x)  f ( x  h)
f ( 2) ( x ) 
h2
f ( x  2h )  2 f ( x  h )  2 f ( x  h )  f ( x  2h )
f (3) ( x ) 
2h 3
f ( x  2h )  4 f ( x  h )  6 f ( x )  4 f ( x  h )  f ( x  2 h )
f (4) ( x ) 
h4

Central Formulas with Error  O (h 2 )

Other formulas for f ( 2) ( x ), f (3) ( x )... are also possible.
You can use Taylor Theorem to prove them and obtain the error order.

11
Example
 Use forward, backward and centered difference
approximations to estimate the first derivate of:
f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2
at x = 0.5 using step size h = 0.5 and h = 0.25
 Note that the derivate can be obtained directly:
f’(x) = –0.4x3 – 0.45x2 – 1.0x – 0.25
The true value of f’(0.5) = -0.9125
 In this example, the function and its derivate are
known. However, in general, only tabulated data
might be given.
12
Solution with Step Size =
0.5
 f(0.5) = 0.925, f(0) = 1.2, f(1.0) = 0.2
 Forward Divided Difference:
f’(0.5)  (0.2 – 0.925)/0.5 = -1.45
|t| = |(-0.9125+1.45)/-0.9125| = 58.9%
 Backward Divided Difference:
f’(0.5)  (0.925 – 1.2)/0.5 = -0.55
|t| = |(-0.9125+0.55)/-0.9125| = 39.7%
 Centered Divided Difference:
f’(0.5)  (0.2 – 1.2)/1.0 = -1.0
|t| = |(-0.9125+1.0)/-0.9125| = 9.6%
13
Solution with Step Size =
0.25
 f(0.5)=0.925, f(0.25)=1.1035, f(0.75)=0.6363
 Forward Divided Difference:
f’(0.5)  (0.6363 – 0.925)/0.25 = -1.155
|t| = |(-0.9125+1.155)/-0.9125| = 26.5%
 Backward Divided Difference:
f’(0.5)  (0.925 – 1.1035)/0.25 = -0.714
|t| = |(-0.9125+0.714)/-0.9125| = 21.7%
 Centered Divided Difference:
f’(0.5)  (0.6363 – 1.1035)/0.5 = -0.934
|t| = |(-0.9125+0.934)/-0.9125| = 2.4%
14
Discussion
 For both the Forward and Backward difference,
the error is O(h)
 Halving the step size h approximately halves the
error of the Forward and Backward differences
 The Centered difference approximation is more
accurate than the Forward and Backward
differences because the error is O(h2)
 Halving the step size h approximately quarters
the error of the Centered difference.
15
Richardson Extrapolation
f ( x  h)  f ( x  h)
Central Difference : f ' ( x)   O(h 2 )
2h

Can we get a better formula?

Hold f ( x) and x fixed :

f ( x  h)  f ( x  h)
 ( h) 
2h
 (h)  f ' ( x)  a2 h 2  a4 h 4  a6 h 6  ...

16
Richardson Extrapolation
Hold f ( x ) and x fixed :
f ( x  h)  f ( x  h)
 (h) 
2h
 ( h )  f ' ( x )  a2h 2  a4h 4  a6h 6  ...
2 4 6
h h h h
 ( )  f ' ( x )  a2    a4    a6    ...
2 2 2 2
Use two derivate
h 3 4 15
 ( h )  4 ( )  3 f ' ( x )  a4h  a6h6  ... estimates to
2 4 16 compute a third,
4 1
 f ' ( x )   ( h / 2)   ( h )  O ( h 4 ) more accurate
3 3 approximation

17
Richardson Extrapolation
Example
 Use the function:

f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2

Starting with h1 = 0.5 and h2 = 0.25, compute

an improved estimate of f’(0.5) using Richardson
Extrapolation

 Recall the true value of f’(0.5) = -0.9125

18
Solution
 The first-derivative estimates can be computed
with centered differences as:
f (0.5  h )  f (0.5  h )
 (h)  at x  0.5
2h
f (1)  f (0) 0.2  1.2
 (0.5)    1.0, |  t |  9.6%
1 1
f (0.75)  f (0.25)
 (0.25)   0.934375, |  t |  2.4%
0.5
The improved estimate can be obtained by applying :
4 1 4 1
f ' (0.5)   ( h / 2)   ( h )  ( 0.934375)  ( 1)  0.9125
3 3 3 3
Which produces the exact result for this example
19
Richardson Extrapolation
Table
D(0,0)=Φ(h)

D(1,0)=Φ(h/2) D(1,1)

D(2,0)=Φ(h/4) D(2,1) D(2,2)

D(3,0)=Φ(h/8) D(3,1) D(3,2) D(3,3)

20
Richardson Extrapolation
Table

 h 
First Column : D ( n,0)    n 
2 

Others :
4m 1
D ( n, m )  m D ( n, m  1)  m D (n  1, m  1)
4 1 4 1

CISE301_Topic6 21
Example
Evaluate numerically the derivative of :
f(x)  x cos(x) at x  0.6

Use Richardson Extrapolation with h  0.1

Obtain D(2,2) as the estimate of the derivative .

22
Example
First Column
f(x  h) - f(x - h)
 (h) 
2h
f(0.7) - f(0.5)
 (0.1)   1.08483
0.2
f(0.65) - f(0.55)
 (0.05)   1.08988
0.1
f(0.625) - f(0.575)
 (0.025)   1.09115
0.05
23
Example
Richardson Table
D (0,0)  1.08483 , D(1,0)  1.08988, D(2,0)  1.09115

4m 1
D ( n, m )  m D ( n, m  1)  m D (n  1, m  1)
4 1 4 1
4 1
D (1,1)  D (1,0)  D (0,0)  1.09156
3 3
4 1
D ( 2 ,1)  D ( 2 ,0)  D (1,0)  1.09157
3 3
16 1
D ( 2 ,2)  D ( 2 ,1)  D (1,1)  1.09157
15 15
24
Example
Richardson Table

1.08483 This is the best

1.08988 1.09156 estimate of the
derivative of the
1.09115 1.09157 1.09157 function.

All entries of the Richardson table are estimates of the

derivative of the function.
The first column are estimates using the central
difference formula with different step size h.

25