NUMERICAL DIFFERENTIATION

The derivative of f (x) at x0 is:

f  x0  h  f ( x0 )
f  x0   Limit
h 0
h
An approximation to this is:

f  x0  h  f ( x0 )
f  x0   for small values of h.
h
Forward Difference
Formula
Let f ( x )  ln x and x0  1.8
Find an approximate value for f 1.8

f (1.8  h)  f (1.8)
h f (1.8) f (1.8  h)
h
0.1 0.5877867 0.6418539 0.5406720
0.01 0.5877867 0.5933268 0.5540100
0.001 0.5877867 0.5883421 0.5554000

The exact value of f 1.8  0.55 5

Assume that a function goes through three points:

 x0 , f  x0 ,  x1 , f  x1  and  x2 , f  x2 .

f ( x)  P( x)

P  x   L0  x  f  x0   L1  x  f  x1   L2  x  f  x2 

Lagrange Interpolating Polynomial

P  x   L0  x  f  x0   L1  x  f  x1   L2  x  f  x2 
( x  x1 )( x  x2 )
P x  f  x0 
( x0  x1 )( x0  x2 )
( x  x0 )( x  x2 )
 f  x1 
( x1  x0 )( x1  x2 )
( x  x0 )( x  x1 )
 f  x2 
( x2  x0 )( x2  x1 )
f ( x )  P ( x )
2 x  x1  x2
P  x   f  x0 
( x0  x1 )( x0  x2 )
2 x  x0  x2
 f  x1 
( x1  x0 )( x1  x2 )
2 x  x0  x1
 f  x2 
( x2  x0 )( x2  x1 )
If the points are equally spaced, i.e.,

x1  x0  h and x2  x0  2h
2 x0  ( x0  h)  ( x0  2h)
P  x0   f  x0 
x0  ( x0  h)x0  ( x0  2h)
2 x0  x0  ( x0  2h)
 f  x1 
( x0  h)  x0 ( x0  h)  ( x0  2h)
2 x0  x0  ( x0  h)
 f  x2 
( x0  2h)  x0 ( x0  2h)  ( x0  h)
  3h  2h h
P  x0   f  x0   f  x1   2 f  x2 
2h 2
h 2
2h

P  x0    3 f  x0   4 f  x1   f  x2 
1
2h
Three-point formula:

f  x0    3 f  x0   4 f  x0  h  f  x0  2h
1
2h
If the points are equally spaced with x0 in the middle:
x1  x0  h and x2  x0  h
2 x0  ( x0  h)  ( x0  h)
P  x0   f  x0 
x0  ( x0  h)( x0  ( x0  h)
2 x0  x0  ( x0  h)
 f  x1 
( x0  h)  x0 ( x0  h)  ( x0  h)
2 x0  x0  ( x0  h)
 f  x2 
( x0  h)  x0 ( x0  h)  ( x0  h)
 h
P  x0   f  x0   2 f  x1   2 f  x2 
0 h
h 2
2h 2h

Another Three-point formula:

f  x0    f  x0  h  f  x0  h
1
2h
Alternate approach (Error estimate)
Take Taylor series expansion of f(x+h) about x:
2 3
f  x  h  f  x   hf  x   f x  f x  
h 2  h 3 
2 3!
h2 2  h3 3 
f  x  h  f  x   hf  x   f x  f x  
2 3!

f  x  h  f  x  h2 3 
 f  x   f  x   f x  
h 2 
h 2 3!
.............. (1)
f  x  h  f  x 
 f  x   O h
h
f  x  h  f  x 
f  x    O h
h

f  x  h  f  x 
f  x   Forward Difference
h Formula

h2 3 
Oh  f  x   f x  
h 2 
2 3!
 4h 2  2  8h 3  3 
f  x  2h  f  x   2hf  x   f x  f x  
2 3!
4h 2  2  8h 3  3 
f  x  2h  f  x   2hf  x   f x  f x  
2 3!

f  x  2h   f  x  4h 2  3 
 f  x   f x  f x  
2h  2 
2h 2 3!
................. (2)
f  x  h  f  x  2
 f  x   f  x   f x  
h 2  h 3 
h 2 3!
.............. (1)

f  x  2h   f  x  4h 2  3 
 f  x   f x  f x  
2h  2 
2h 2 3!
................. (2)

2 X Eqn. (1) – Eqn. (2)

 f  x  h  f  x  f  x  2 h   f  x 
2 
h 2h
2h2 3  6h3 4 
 f  x   f x   f x   
3! 4!
 f  x  2 h  4 f  x  h  3 f  x 
2h
2 3
 f x   f x   f x   
2h  3  6 h 4 

3! 4!
 
 f  x   O h2
 f  x  2h   4 f  x  h   3 f  x 
2h
 
 f  x   O h2

 f  x  2h   4 f  x  h   3 f  x 
f  x  
2h
 
 O h2

 f  x  2h   4 f  x  h   3 f  x 
f  x  
2h
Three-point Formula

 
2 3
f 3   x   f 4   x   
2 h 6 h
O h2  
3! 4!
The Second Three-point Formula
Take Taylor series expansion of f(x+h) about x:
h2 2  h3 3 
f  x  h  f  x   hf  x   f x  f x  
2 3!
Take Taylor series expansion of f(x-h) about x:
2 3
f  x  h  f  x   hf  x   f x   f x  
h 2  h 3 
2 3!
Subtract one expression from another
2h 3  3  2h 6  6 
f  x  h  f  x  h  2hf  x   f x  f x   
3! 6!
 2h 3  3  2h 6  6 
f  x  h  f  x  h  2hf  x   f x  f x   
3! 6!

f  x  h  f  x  h h2 3  h5 6 
 f  x   f x   f x  
2h 3! 6!

f  x  h  f  x  h h2 3  h5 6 
f  x    f x   f x  
2h 3! 6!
 f  x  h  f  x  h
f  x  
2h
 O h2 
 
2 5
f 3   x   f 6   x   
h h
O h2  
3! 6!

f  x  h  f  x  h 
f  x  
2h
Second Three-point Formula
Summary of Errors

f  x  h  f  x 
f  x   Forward Difference
h Formula

h2 3 
Oh  f  x   f x  
h 2 
Error term
2 3!
Summary of Errors continued

First Three-point Formula

 f  x  2h   4 f  x  h   3 f  x 
f  x  
2h

 
2 3
f x  f x   
2 h 3  6 h 4 
Error term O h2  
3! 4!
Summary of Errors continued

Second Three-point Formula

f  x  h  f  x  h 
f  x  
2h

 
2 5
f 3   x   f 6   x   
h h
Error term O h2  
3! 6!
Example:
f  x   xe x

Find the approximate value of f 2 with h  0.1

x f x
1.9 12.703199
2.0 14.778112
2.1 17.148957
2.2 19.855030
Using the Forward Difference formula:

f   x0    f  x0  h   f  x0 
1
h

f   2 
1
0.1
 f  2.1  f  2 

1
 17.148957  14.778112
0.1
 23.708450
Using the 1st Three-point formula:

f  x0    3 f  x0   4 f  x0  h  f  x0  2h
1
2h

f 2 
1
 3 f ( 2)  4 f ( 2.1)  f ( 2.2)
2  0.1

1
  3  14.778112  4  17.148957
0.2
 19.855030 
 22.032310
Using the 2nd Three-point formula:

f  x0    f  x0  h  f  x0  h
1
2h

f 2 
1
 f ( 2.1)  f (1.9)
2  0.1

1
 17.148957  12.703199 
0.2
 22.228790
The exact value of f 2 is : 22.167168
Comparison of the results with h = 0.1
The exact value of f   2 is 22.167168

Formula f   2 Error
Forward Difference 23.708450 1.541282
1st Three-point 22.032310 0.134858
2nd Three-point 22.228790 0.061622
Second-order Derivative
h2 2  h3 3 
f  x  h  f  x   hf  x   f x  f x  
2 3!
2 3
f  x  h  f  x   hf  x   f x  f x  
h 2  h 3 
2 3!
Add these two equations.
2h 2  2  2h 4  4 
f  x  h  f  x  h  2 f  x   f x  f x  
2 4!
2h2  2  2h4  4
f  x  h  2 f  x   f  x  h  f  x  f  x 
2 4!
f  x  h  2 f  x   f  x  h 2
 f x  f x  
2  2h  4 
2
h 4!

f 2   x  
f  x  h   2 f  x   f  x  h  
2 h 2
f 4 
x  
2
h 4!

f  x  h  2 f  x   f  x  h
f 2 
x 
h2
 NUMERICAL INTEGRATION
b

 f ( x )dx  area under the curve f(x) between

a x  a to x  b.

In many cases a mathematical expression for f(x) is

unknown and in some cases even if f(x) is known its
complex form makes it difficult to perform the integration.
 y
f(b)
f(x)

f(a)

x0 = a x0 = b x
y
f(b)
f(x)

f(a)

x0 = a x0 = b x
Area of the trapezoid

The length of the two parallel sides of the trapezoid

are: f(a) and f(b)
The height is b-a

ba
b

 f ( x )dx   f ( a )  f ( b )
a
2

  f ( a )  f ( b )
h
2
Simpson’s Rule:

x2 x2

x0
f ( x )dx   P ( x )dx
x0

x1  x0  h and x2  x0  2h
 ( x  x1 )( x  x2 )
 
x2 x2
P  x  dx  f  x0 dx
x0 x0 ( x0  x1 )( x0  x2 )
( x  x0 )( x  x2 )

x2
 f  x1 dx
x0 ( x1  x0 )( x1  x2 )
( x  x )( x  x )
 ( x  x )( x  x )
x2
 0
f  x  dx
1
2
x0 2 0 2 1
 x2 x2

x0
f ( x )dx   P ( x )dx
x0

  f  x0   4 f ( x1 )  f ( x 2 )
h
3
y
f(b)
f(x)

f(a)

x0 = a x0 = b x
y
f(b)
f(x)

f(a)

x0 = a x0 = b x
y
f(b)
f(x)

f(a)

x0 = a x0 = b x
y
f(b)
f(x)

f(a)

x0 = a x0 = b x
Composite Numerical Integration
Riemann Sum

The area under the curve is subdivided into n

subintervals. Each subinterval is treated as a
rectangle. The area of all subintervals are added
to determine the area under the curve.

There are several variations of Riemann sum as

applied to composite integration.
y
Left Riemann Sum
 b Riemann
 xIn Left a / n
x1 sum,
a the left-
side sample of
x2    x is
theafunction
used as the
x3 height
a  2ofthe
x
individual
rectangle.
a x xi  a   i  1  x
b
x

 f  x  dx   f  x   x
b

a i
i 1
y
Right Riemann Sum
In 
 xRight 
 bRiemann
a / n
x1  the
sum, a right-side
x
sample of the
x2  a is2used
function  x as
the height of the
x3  a  3rectangle.
individual x

a x xi  a  i x
b
x

 f  x  dx   f  x  x
b

a i
i 1
 Midpoint Rule
y
 x In bthe / n
 aMidpoint
Rule, the sample at
a  middle
x1  the  the
2 1  1of x / 2
a   2  2  1is
x2 subinterval used
 x / 2
as the height of the
a   2  3  1  x / 2 
x3  individual
rectangle.

a x xi  a   2  i  1  x / 2 
b
x

 f  x  dx   f  x  x
b

a i
i 1
Composite Trapezoidal Rule:
Divide the interval into n subintervals and apply
Trapezoidal Rule in each subinterval.

h n 1

b

 f ( x )dx   f a   2 f ( xk )  f (b)
a
2 k 1 
where

ba
h and x k  a  kh for k  0, 1, 2, ... , n
n
 π

Find  sin (x)dx

0

by dividing the interval into 20 subintervals.

n  20
ba π
h 
n 20
kπ
x k  a  kh  , k  0, 1, 2, ....., 20
20
π
h n 1

0 sin( x )dx  2  f a   2
k 1
f ( x k )  f ( b )

π  19
 kπ  
  sin0   2 sin   sinπ 
40  k 1  20  
 1.995886
Composite Simpson’s Rule:
Divide the interval into n subintervals and apply
Simpson’s Rule on each consecutive pair of subinterval.
Note that n must be even.

h ( n / 2 ) 1
b

 f ( x )dx   f a   2  f (x 2k )
a
3 k 1

n/ 2

 4 f ( x2 k 1 )  f (b)
k 1 
where
ba
h and x k  a  kh for k  0, 1, 2, ... , n
n
π

Find  sin (x)dx

0

by dividing the interval into 20 subintervals.

ba π
n  20 h 
n 20
kπ
x k  a  kh  , k  0, 1, 2, ....., 20
20
π
π  9
 2kπ 
0 sin( x )dx  60 sin0  2
k 1
sin
 20 


10
 ( 2k  1)π  
 4 sin   sin( π )
k 1  20  
 2.000006