4 - Numerical Differentiation

Numerical Methods
Numerical Differentiation
 Finite Difference Method

 Lagrangian Methods
1
Motivation
 How do you evaluate the Time Displacement
(second) (meters)
derivative of a tabulated
function. 0 30.1
 How do we determine the 5 48.2

velocity and acceleration
from tabulated 10 50.0
measurements.
15 40.2
2
 Finite Difference Methods:
o Forward Differences:
y ( x  h)  y ( x)  y ( x)
Ey( x)  y ( x)  y ( x)
E 1  
e hD  E  1  
hD  ln(1  )
3
 Upon expanding the logarithm as a
Taylor series, we have,
2 1 3 1 4
hD         ............
2 3 4
Finite difference formula,
2 y ( x) 1 3 1 4
hDy ( x)  hy( x)  y ( x)    y ( x)   y ( x)  ............
2 3 4
2 2 2 2 3 11 4
h D y ( x)  h y( x)   y ( x)   y ( x)   y ( x)  ..................
12
4
 Find y’(3.0)
x y Δy (10-5) Δ2y Δ3y Δ4 y Δ5y Δ6y
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 5
 Using the table, we find
1 1

0.1 y (3.0)  0.01075  (0.00067)  (0.00006)  -0.0110655
2 3
y(3.0)  -0.110655
 The exact value is
1
y (3.0)   2  0.111111......
3
6
 For a heat transfer problem;
0
1
T 1 1 2 1 3 
 T0   T0   T0  ..................
x x0 h 2 3 
T 1 1 1 
 T 
1 0 T  T2  2T1  T0   T3  3T2  3T1  T0   .........
x x0 h 2 3  7
o Backward Differences:
y ( x  h )  y ( x )  y ( x  h )
Ey ( x)  y ( x)  Ey ( x) e hD  E  (1  ) 1
E  1  E hD   ln(1  )
  1  E 1
1 2 1 3
hD   ln(1  )        ...........
2 3
8
 We have
1 2 1 3
hDy ( x)  hy  y ( x)   y ( x)   y ( x)  ...........
2 3
which is a backward difference formula for the

derivative in terms of differences
For second derivative;
1 11
h 2 D 2 y ( x)  h 2 y   2 y ( x)   3 y ( x)   4 y ( x)  ...........
2 12
9
 Find y’(3.6)
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 10
1 1

0.1 y (3.6)  0.00793  (0.00048)  ( 0.00002)  -0.007696
2 3
y(3.6)  -0.07696
1
y (3.6)   2  0.0771604
3. 6
11
o Central Differences:
  E1 / 2  E 1 / 2  e hD / 2  e  hD / 2
hD -1 
  2 Sinh( ) , hD  2Sinh ( )
2 2
Expanding the inverse hyperbolic sine in a

Taylor series
  1   3 1 1  3   5 1 
hD  2         ............ 

 2 2  2  3 2  4  2  5  12
 
 To compute derivatives at the tabular points,
the equation must be altered by expanding;
1

2  2 2
2 1  1   1
4  4 
 1 3 
 1 2 ( ) 2 2
  
 1   2 2    ............  1
 
 2 4 2!  4  
 
multiply the hD equation; we get;
2
  1   3 3   5    2 3   2  
hD  2        .....   1      .....
 2 6  2  40  2   8 8 4 
   
13
 Then, the equation is simplified to
1 3 1
hD       5  .........
6 30
We get a central difference formula;

1 3 1
hDy ( x )  hy ( x )  y ( x)   y ( x)   5 y ( x )  .........

6 30
2 2 2 1 4 1 6
h D y ( x)  hy( x)   y ( x)   y ( x)   y ( x)  .........
12 90
14
 Recall that
1
  ( E 1 / 2  E 1 / 2 )
2
1 1
y0  ( E1 / 2  E 1 / 2 )( E 1 / 2  E 1 / 2 ) y0  ( E  E 1 )
2 2
1
y0  ( y ( x  h)  y ( x  h)
2
1

 3 y0  ( 2 y ( x  h)   2 y ( x  h)
2

15
 Find y’(3.3)
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 16
1
0.1 y(3.3)  0.294120  0.31250  1 (0.00050  0.00062)  -0.00918
2 12
y(3.3)  -0.0918
1
y (3.0)   2  0.0918273
3.3
17
 Find y’’(3.3)
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 18
0.12 y(3.3)  0.00056

y(3.3)  0.056
2
y(3.3)  3
 0.05565
3.3
The higher the order of the derivative, the less
accurate the actual evaluation !!!
19
 Derivatives at Non-Tabular Points
o Let
dp 1
x1  x  ph 
dx1 h
dy dy dp
y( x  ph)  
dx1 dp dx1
dy
hy( x  ph) 
dp
20
 Forward Difference Formula:
p ( p  1) 2
y ( x  ph)  y ( x)  py ( x)   y ( x)
2!
p ( p  1)( p  2) 3
  y ( x)  ........
3!
dy (2 p  1) 2 (3 p 2  6 p  2) 3
hy( x  ph)   y ( x)   y ( x)   y ( x)  .......
dp 2 6
21
 Find y’(3.05) for x=3.0 h=0.1 p=1/2
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 22
1 1 1
( 2.  1) (3  6  2)
0.1 y(3.05)  0.01075  2 (0.00067)  4 2 (0.00006)
2 3
y(3.05)  -0.10745
1
y (3.05)   2
 -0.10749798
3.05
23
 Backward Difference Formula:
p ( p  1) 2
y ( x  ph)  y ( x)  py ( x)   y ( x)
2!
p( p  1)( p  2) 3
  y ( x)  ........
3!
dy (2 p  1) 2 (3 p 2  6 p  2) 3
hy( x  ph)   y ( x )   y ( x)   y ( x)  ......
dp 2 6
24
 Find y’(3.52) for x=3.5 h=0.1 p=0.2
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 25
 y’(3.52) for x=3.5 h=0.1 p=0.2
2 4 2
( 2.  1) (3  6  2)
0.1 y(3.52)  0.00841  10 (0.00050)  100 10 ( 0.00006)
2 6
y(3.52)  -0.080932
1
y (3.54)   2
 -0.07979826
3.54
26
 Central Difference Formula:
The Stirling formula;

p2 2
y ( x  ph)  y ( x)  py ( x)   y ( x)
2!
p( p 2  1) 3
  y ( x)  ........
3!
dy 2 (3 p 2  1) 3
hy ( x  ph)   y ( x)  p y ( x)   y ( x)  ........
dp 6
27
 Find y’(3.44) for x=3.4 h=0.1 p=0.4
3.0 0.33333
-1075
3.1 0.32258 67
-1008 -6
3.2 0.31250 61 +1 1
-947 -5 -2 y ( x) 
3.3 0.30303 56 -1 7 x
-891 -6 -5
3.4 0.29412 50 +4 -13
-841 -2 -8
3.5 0.28571 48 -4
-793 -6
3.6 0.27778 42 28
 y’(3.44) for x=3.4 h=0.1 p=0.4
1 2
1
y0  ( y ( x  h)  y ( x  h)  3 y0 
2
 
( y ( x  h)   2 y ( x  h)
2
1  3(0.4) 2  1 
0.1 y(3.44)  (0.28571  0.30303)  0.4(0.00050)   (0.00048  0.00056)
2  12 
y(3.44)  -0.08456
1
y (3.54)   2
 0.0845051
3.44 29
 Langrange Formulae
df f ( x  h)  f ( x )
 lim
dx h0 h
Taylor Theorem :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3
f ( x  h)  f ( x )  f ' ( x ) h    O(h 4 )
2! 3!
n n
E  O(h )   real , finite C , such that : E  C h
E is of order h n  E is approaching zero at rate similar to h n
30
df ( x) f ( x  h)  f ( x)
Forward Difference : 
dx h
df ( x) f ( x)  f ( x  h)
Backward Difference : 
dx h
df ( x) f ( x  h)  f ( x  h)
Central Difference : 
dx 2h
Which method is better? How do we judge them?

31
f ( x  h)  f ( x )
f ( x ) 
h
f ( x )  f ( x  h)
f ( x) 
h
f ( x  h)  f ( x  h)
f ( x) 
2h
32
Forward Difference : f ( x  h)  f ( x)  f ' ( x) h  O ( h 2 )
 f ' ( x ) h  f ( x  h)  f ( x )  O ( h 2 )
f ( x  h)  f ( x )
 f ' ( x)   O ( h)
h
_______________________________________________________
Backward Difference : f ( x  h)  f ( x )  f ' ( x ) h  O ( h 2 )
 f ' ( x ) h  f ( x )  f ( x  h)  O ( h 2 )
f ( x )  f ( x  h)
 f ' ( x)   O ( h)
h
33
Central Difference :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 3) ( x ) h 3
f ( x  h)  f ( x  h)  2 f ' ( x ) h  2  ...
3!
f ( x  h)  f ( x  h)
 f ' ( x)   O(h 2 )
2h
34
df ( x) f ( x  h)  f ( x)
Forward Difference :   O ( h)
dx h
df ( x) f ( x)  f ( x  h)
Backward Difference:   O ( h)
dx h
df ( x) f ( x  h)  f ( x  h)
Central Difference :   O(h 2 )
dx 2h
Forward and backward difference formulas are comparable in accuracy.

Central difference formula is expected to give a better answer.
35
f ( 2) ( x)h 2 f (3) ( x)h3 f ( 4) ( x)h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2) ( x)h 2 f (3) ( x)h3 f ( 4) ( x)h 4
f ( x  h)  f ( x )  f ' ( x ) h     ...
2! 3! 4!
f ( 2) ( x ) h 2 f ( 4) ( x ) h 4
f ( x  h)  f ( x  h )  2 f ( x )  2 2  ...
2! 4!
f ( x  h)  2 f ( x )  f ( x  h)
 f ( 2) ( x )  2
 O ( h 2
)
h
f ( 4) ( )h 2
Error  
12
36
f ( x  h)  2 f ( x)  f ( x  h)
f ( 2) ( x ) 
h2
f ( x  2h )  2 f ( x  h )  2 f ( x  h )  f ( x  2h )
f (3) ( x ) 
2h 3
f ( x  2 h )  4 f ( x  h )  6 f ( x )  4 f ( x  h )  f ( x  2h )
f ( 4) ( x ) 
h4
Central Formulas with Error  O ( h 2 )

Other formulas for f ( 2) ( x ), f (3) ( x )... are also possible.
You can use Taylor Theorem to prove them and obtain the error order.
37
Numerical Differentiation (Example)
 Use forward, backward and centered difference
approximations to estimate the first derivate of:
f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2
at x = 0.5 using step size h = 0.5 and h = 0.25
 Note that the derivate can be obtained directly:
f’(x) = –0.4x3 – 0.45x2 – 1.0x – 0.25
The true value of f’(0.5) = -0.9125
 In this example, the function and its derivate
are known. However, in general, only tabulated
data might be given.
38
Solution with Step Size = 0.5
 f(0.5) = 0.925, f(0) = 1.2, f(1.0) = 0.2
 Forward Divided Difference:
f’(0.5)  (0.2 – 0.925)/0.5 = -1.45
|t| = |(-0.9125+1.45)/-0.9125| = 58.9%
 Backward Divided Difference:
f’(0.5)  (0.925 – 1.2)/0.5 = -0.55
|t| = |(-0.9125+0.55)/-0.9125| = 39.7%
 Centered Divided Difference:
f’(0.5)  (0.2 – 1.2)/1.0 = -1.0
|t| = |(-0.9125+1.0)/-0.9125| = 9.6%
39
Solution with Step Size = 0.25
 f(0.5)=0.925, f(0.25)=1.1035, f(0.75)=0.6363
 Forward Divided Difference:
f’(0.5)  (0.6363 – 0.925)/0.25 = -1.155
|t| = |(-0.9125+1.155)/-0.9125| = 26.5%
 Backward Divided Difference:
f’(0.5)  (0.925 – 1.1035)/0.25 = -0.714
|t| = |(-0.9125+0.714)/-0.9125| = 21.7%
 Centered Divided Difference:
f’(0.5)  (0.6363 – 1.1035)/0.5 = -0.934
|t| = |(-0.9125+0.934)/-0.9125| = 2.4%
40
Discussion
 For both the Forward and Backward difference,
the error is O(h)
 Halving the step size h approximately halves the
error of the Forward and Backward differences
 The Centered difference approximation is more
accurate than the Forward and Backward
differences because the error is O(h2)
 Halving the step size h approximately quarters
the error of the Centered difference.
41
 Richardson Extrapolation
f ( x  h)  f ( x  h )
Central Difference : f ' ( x)   O(h 2 )
2h
Can we get a better formula?
Hold f ( x) and x fixed :

f ( x  h )  f ( x  h)
 (h) 
2h
 (h)  f ' ( x)  a2 h 2  a4 h 4  a6 h 6  ...
42
Hold f ( x ) and x fixed :
f ( x  h)  f ( x  h)
 (h ) 
2h
 (h )  f ' ( x )  a2h 2  a4h 4  a6h6  ...
2 4 6
h h h h
 ( )  f ' ( x )  a2    a4    a6    ...
2 2 2 2
Use two derivate
h 3 4 15
 (h )  4 ( )  3 f ' ( x )  a4h  a6h 6  ... estimates to
2 4 16 compute a third,
4 1
 f ' ( x )   ( h / 2)   ( h )  O ( h 4 ) more accurate
3 3 approximation
43
 Use the function:
f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2
Starting with h1 = 0.5 and h2 = 0.25, compute

an improved estimate of f’(0.5) using
Richardson Extrapolation
 Recall the true value of f’(0.5) = -0.9125
44
Numerical Differentiation (Solution)
 The first-derivative estimates can be computed
with centered differences as:
f (0.5  h )  f (0.5  h )
 (h)  at x  0.5
2h
f (1)  f (0) 0.2  1.2
 (0.5)    1.0, |  t |  9.6%
1 1
f (0.75)  f (0.25)
 (0.25)   0.934375, |  t |  2.4%
0. 5
The improved estimate can be obtained by applying :
4 1 4 1
f ' (0.5)   ( h / 2)   (h )  ( 0.934375)  ( 1)  0.9125
3 3 3 3
Which produces the exact result for this example
45
 Richardson Extrapolation Table
D(0,0)=Φ(h)
D(1,0)=Φ(h/2) D(1,1)
D(2,0)=Φ(h/4) D(2,1) D(2,2)
D(3,0)=Φ(h/8) D(3,1) D(3,2) D(3,3)

46
 h 
First Column : D ( n,0)    n 
2 
Others :
4m 1
D ( n, m )  m D ( n, m  1)  m D ( n  1, m  1)
4 1 4 1
47
Evaluate numerically the derivative of :
f(x)  x cos(x) at x  0.6
Use Richardson Extrapolation with h  0.1
Obtain D(2,2) as the estimate of the derivative .
48
f(x  h) - f(x - h)
 (h) 
2h
f(0.7) - f(0.5)
 (0.1)   1.08483
0.2
f(0.65) - f(0.55)
 (0.05)   1.08988
0.1
f(0.625) - f(0.575)
 (0.025)   1.09115
0.05
49
D (0,0)  1.08483 , D(1,0)  1.08988, D(2,0)  1.09115
4m 1
D ( n, m )  m D ( n, m  1)  m D ( n  1, m  1)
4 1 4 1
4 1
D (1,1)  D (1,0)  D (0 ,0)  1.09156
3 3
4 1
D ( 2 ,1)  D ( 2 ,0)  D (1,0)  1.09157
3 3
16 1
D ( 2 ,2)  D ( 2 ,1)  D (1,1)  1.09157
15 15
50
1.08483 This is the best
1.08988 1.09156 estimate of the
derivative of the
1.09115 1.09157 1.09157 function.
All entries of the Richardson table are estimates of the

derivative of the function.
The first column are estimates using the central
difference formula with different step size h.
51

4 - Numerical Differentiation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

4 - Numerical Differentiation

Uploaded by

Copyright:

Available Formats

Numerical Methods

 Finite Difference Method

 How do we determine the 5 48.2

 The exact value is

which is a backward difference formula for the

For second derivative;

 The exact value is

Expanding the inverse hyperbolic sine in a

We get a central difference formula;

 The exact value is

0.12 y(3.3)  0.00056

 The exact value is

 The exact value is

The Stirling formula;

 The exact value is

Which method is better? How do we judge them?

Forward and backward difference formulas are comparable in accuracy.

Central Formulas with Error  O ( h 2 )

Can we get a better formula?

Hold f ( x) and x fixed :

f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2

Starting with h1 = 0.5 and h2 = 0.25, compute

 Recall the true value of f’(0.5) = -0.9125

D(2,0)=Φ(h/4) D(2,1) D(2,2)

D(3,0)=Φ(h/8) D(3,1) D(3,2) D(3,3)

Use Richardson Extrapolation with h  0.1

Obtain D(2,2) as the estimate of the derivative .

All entries of the Richardson table are estimates of the

You might also like