You are on page 1of 16

# ENGRD 241 Lecture Notes

## NUMERICAL INTEGRATION (C&C 4th ed. PART 6)

Basic Issues in Integration
What does an integral represent?
b

d b

f (x) dx = AREA

g(x, y)

c a

dx dy = VOLUME

b

f (x) dx = nlim

k 1

f (x k )

Objective:
b

a

## When would we want to do this?

1. Integrand is too complicated to integrate analytically, e.g.,
2
2 cos(1 x ) 0.5x
e dx
0
1 0.5x
2. Integrand is not precisely defined by an equation,
i.e., we are given a set of data (xi,(xi)), i=1,...,n.

## All methods are applicable to integrands that are functions.

Some are applicable to tabulated values.

Key concepts:
1. Integration is a summing process. Thus virtually all numerical approximations
can be represented by
n
b
I = f(x)dx = w if(xi ) E t
i0
a
where: wi = weights
xi = sampling points
Et = truncation error
2. Closed & Open forms:
Closed forms include the end points a & b in xi.
Open forms do not.

xn

To evaluate:

x f(x)dx
0

## 1. Divide [xo, xn] into n segments (n 1)

2. Within each segment approximate (x) by an mth order polynomial,
pm(x) = a0 +a1x + a2x2 + + amxm
The polynomial order need not be the same for all segments. Then:

xn
x2
x1
xn
f(x)dx = pm (x)dx + pm (x)dx + + p m (x)dx
x0 1
x0 2
x0 n
x0
the mi's may be the same or different. Integrate each polynomial exactly.
Order m of polynomial pm(x) determines Newton-Cotes formulas:
m
1
2
3
4
5

Polynomial
linear
quadratic
cubic
quartic
quintic

Formula
Trapezoid
Simpson's 1/3
Simpson's 3/8
Boole's Rule
Boole's Rule

Error
O(h2)
O(h4)
O(h4)
O(h6)
O(h6)

## Trapezoid Rule (C&C 4th ed., Sec. 21.1, p. 586):

For each segment (or one segment), let (x) p1(x) = a0 +a1x
a. Determine a0 + a1x from Newton DD Polynomial:
f(x ) f(x i 1 )
p1 (x) = f(x i-1 ) i
(x x i-1 )
x i x i 1
b. Integrating [use trapezoid area formula, C&C 4th ed., Box 21.1]:
xi

p1 (x)dx (x i x i 1 )

x i 1

f (x i ) f (x i1 )
2

## c. Truncation Error [C&C 4th ed., Box 21.2]:

xi

x i 1

(x)dx (x i x i 1 )

f (x i ) f (x i 1 ) 1
(x i x i 1 )3 f "()
2
12

## Integrates a linear function correctly: f "() = 0.

Easily derived by considering Taylor Series.

## Composite Trapezoid Rule (C&C 4th ed., Sec. 21.1.2, p. 588):

For multiple segments, sum segment areas to approximate integral:
xn

I=

f (x)dx =

(x x
i

i 1 )

i 1

x0

f(x i ) f(x i 1 )

E ti
2

## where Eti is truncation error in ith interval.

If data are evenly spaced, i.e., xi xi-1 =

xn x0
= h for i=1,...,n
n

n 1

f(x 0 ) 2
f(x i ) f(x n )

i1

I (xn x0)
2n

1
or: I h f(x 0 )
2

n 1

i 1

1
f(x i ) f(x n )
2

f ( )
i

nf

so the error is

i=1

(x x0 )h 2
nh3
Et =
f = n
f
12
12

Note: If n is
doubled, h h/2
and Et Ea/4

## NOTE: C&C notation is different

h = (xn-x0)/n ==> (b-a)/n and E t

nh3
(b a)3
"
"
12
12n 2

## Simpson 1/3 Rule (C&C 4th ed., Sec. 21.2 p. 596):

Improving the estimate of the integral
fit Lagrange polynomials to three points (a pair of segments)
integrate those polynomials to obtain a general formula for integrals.
the resulting function must correctly integrate quadratic polynomials.
xn

Evaluate

xn

f (x)dx p (x)dx
2

x0

x0

## For each pair of segments use Lagrange Interpolating Polynomial:

(x x i )(x x i1 )
(x x i1 )(x x i1 )
(x x i1 )(x x i )
f (x i1 )
f (x i )
f (x i 1 )
(x i1 x i )(x i1 x i1 )
(x i x i 1 )(x i x i 1 )
(x i 1 x i 1 )(x i 1 x i )
If points are evenly spaced, integrating yields:
p2 (x)
x i 1

x i 1

f (x)dx (x i1 x i 1 )

f (x i1 ) 4f (x i ) f (x i 1 )
f (x i 1 ) 4f (x i ) f (x i 1)
h
6
3

## Simpson's 1/3 Rule Alternative derivation

Choose weights in f (-1) + f (0) + f (1) so that over [ -1, +1]:
1

a.

0

## x dx 0 (1) (0) (1)

0

1
1

x 2 dx

2
0 (1) 2 1 (0) 2 (1) 2
3

## from 2nd equation: 0 = 2; from 3rd: 0 + 2 = 2/3;

hence 0 = 2 = 1/3 and from 1st equation 1 = 4/3
b. General formula for evenly spaced points (note interval above had width of 2):
x i 1

p2 (x)dx (x i1 x i 1 )

f x i1 4f x i f x i1

x i 1
xi+1

c. Error:

f (x) dx

i-1

xi1

p 2 (x) dx

i-1

1
(x 2 x 0 ) h 4 f (4) ( )
180

## Integrates a cubic exactly: (4)() = 0.

Because (x2 x0) = 2h, the error term becomes (C&C 4th ed., Table 21.2, p. 604):
1
(x x i-1 )5 (4)
h 5f (4) ( ) = i+1
f ( )
90
2880
Derivation of Simpson 1/3 Rule error term
Let:

x = -h, 0, +h
h

I = f(x)dx = (f o + xf o +
-h

x2
f o+
2!

) dx

2h3
2h5 
f o+
f + . . . (only odd powers of h survive)
6
5! o

h
h
2h2
2h 4 4
S =
f -h + 4f(0) + f(+ h)
6f

o
o
o

3
3
2
4!

= 2hf o +

2hf0

2h3
2h5 
f0
f
6
3(4!)

## Both have only odd terms.

I - S = Truncation Error:
4
(2h)h 
1
=
f + 2h 0(h6 ) = h 5f (4) ( )
90
180

## Page 6-5 of 6-16

Composite Simpson 1/3 Rule (C&C 4th ed., Sec. 21.2.2, p. 599)
Sum pairs of segment areas to approximate the given integral:
xn
n 1
f(x ) 4f(xi ) f(xi1 )

I = f(x)dx = (x i1 x i1 ) i 1
E ti

6
i1,3,5
x
0

## If all data are evenly spaced, i.e., xi xi-1 = h for i=1,...,n:

n 1
n 2

h
I f (x 0 ) 4
f (x i ) 2
f (x j ) f (x n )

3
i 1,3,5
j2,4,6

f ( ) nf (4)
n

## It can also be shown that

so:

i =1

(x n x 0 )h 4 (4)
(x n x 0 )5 (4)
nh 5 (4)
Et
f
f
f
180
180
180n 4

## Note: If n is doubled, h--> h/2 and Et --> Et /16

Simpson 3/8 Rule (C&C 4th ed., Sec. 21.2.3, p. 601):
For each triple of segments, let (x) p3(x) = a0 + a1x + a2x2 + a3x3
a. Replace a + a x + a x2+ a x3 w/ Lagrange Polynomials and integrate
0

x3

p (x)dx (x
x
3

x0 )

x3

c. Error:

x3

## f(x 0 ) 3f(x1 ) 3f(x 2 ) f(x 3 )

f(x ) 3f(x1 ) 3f(x 2 ) f(x 3 )
3h 0
8
8
1

f(x)dx = x p (x)dx 80
x
3

Et

(x3 x0)h4f(4)()

( x x )5
1
3
( x3 x0 )h 4 f (4) ( ) h5 f (4) ( ) 3 0 f (4) ( )
80
80
6480

## Page 6-6 of 6-16

Notes: 1. Composite Simpson 1/3 Rule and Simpson 3/8 Rule errors are
(x n x 0 ) 4 (4)
h f
180

vs.

(x n x 0 ) 4 (4)
h f
80

Simpson 1/3 rule is slightly more accurate over the interval [x0,xn] when
the two methods use the same value of h.
But both methods have the same order of accuracy.
2. Simpson 3/8 Rule requires multiples of three segments.
Simpson 1/3 Rule requires even number of segments.
3. Can use a combination of T, S 1/3, and/or S 3/8 when number of
segments is neither even nor divisible by 3 or when data in unevenly
spaced.
Higher order does not always yield higher accuracy.
1.5

Example: I =

c e

x2

dx = 1.00000

0.2

n
1
2
3
4
6
8
9
12
15
16
32
64

Trap.
1.05191
1.00499*
1.00187*
1.00099
1.00042
1.00023
1.00019
1.00011
1.00008
1.00006
1.00001
1.00000

Simp. 1/3

Simp. 3/8

0.98935
0.99561
0.99965
0.99994*
0.99998*

0.99985

1.00000*

0.99998*
1.00000*
1.00000*

1.00000*
Asterisk (*) denotes more accurate answer for each n.

## Newton-Cotes Integration -- Other Cases:

1. NC formulas can be derived for any number of segments and spacing.
Just integrate Lagrange Polynomial to get weight on each value of function:
f(xi), i = 0,...,n;
Formula can be open or closed.
2. Can always use multiple-application Trapezoidal Rule.
Can use a Simpson Rule when adjacent segments have equal hs.
3. Integrate interpolating spline.
See C&C 4th ed.,Table 21.2 Closed formulas, p. 604, and Table 21.4 Open formulas, p. 608.

## Richardson Extrapolation (C&C 4th ed., Sec. 22.2.1, p. 615)

Given two numerical estimates obtained using different h's, compute a higher-order
estimate.
Scheme:
Starting with a step size h1, the exact value of some operation A is:
A = A(h1) + O (h1n)
Suppose we reduce step size to h2
A = A(h2) + O (h2n)
Multiplying the second equation by r = (h1/h2)n and then subtracting the
first equation yields [where the bracketed error terms tend to cancel]:
(r-1) A = r A(h2) A(h1) + [r O(h2n) O(h1n)]
n

h1
A(h 2 ) A(h1 )
h
A= 2
+ O(h1m)
n
h1
1
h2

m n+1

Richardson Extrapolation Example -- Numerical Differentiation (C&C 4th ed., Sec. 23.2, p. 635):
n

h1
D(h 2 ) D(h1 )
h
D = 2
n
h1
1
h2

Start with O(h2) centered first difference approx. from Table 23.3:
f(x i h) f(x i h)
f f
1 1
2h
2h
Substituting h2 = h and h1 = 2h, Richardson Extrapolation yields the next
highest order centered first-difference formula in Table 23.3:
4
f '(xi) = (1/3) [ 4 D(h) D(2h) ] + O (h ) =
f (x i ) D(h)

1 1
1

4 (f1 f 1) (f 2 f 2 ) + O (h4) =

3 2h
4h

(-f 2 8f1 8f 1 f 2 )
+ O (h4)
12h

## Richardson Extrapolation Example -- Numerical Integration

(C&C 4th ed., Example 22.1, p. 613):
Suppose we use the Trapezoidal Rule to integrate. If we use step size h
I = T(2h) + O(4h2).
To improve estimate, reduce step size:
I = T(h) + O(h2).
Combining to eliminate error of order h2 yields
1
22 T(h) T(2h)
I
O(h 4 ) =
[ 4 T(h) T(2h) ] + O(h4)
2
3
2 1
1. Greater weight is placed on the more accurate estimate
2. Weighting coefficients sum to unity, i. e. (4 1)/3 = 1
3. Actually Simpsons 1/3 rule: (h/3)[(xi-1) + 4(xi) + (xi+1)]
Romberg Integration (C&C 4th ed., Sec. 22.2, p. 615)
Systematic application of Richardson extrapolation with: hi+1 = hi/2
Benefit:

## Get a high-order Newton-Cotes formula easily.

Can watch convergence.

## General formula, C&C Eq. (22.8):

4k 1 I j1,k 1 I j,k 1
I j1,k 1 I j,k 1
I j,k

1,k

1
4k 1 1
4k 1 1
where:

j = level of subdivision
k = level of integration O(h2k)

## Tabular Organization of Romberg Integration

Ij,k = Ij+1,k-1 +
j

1
2
3
4
5

I j1,k 1 I j,k 1
4k 1 1

# Trap.
segs.
n
1
2
4
8
16
1
4k 1 1

k=1
O(h2)
Trap
I1,1
I2,1

k=2
O(h4)

k=3
O(h6)

k=4
O(h8)

k=5
O(h10)

1/63

1/255

I1,2
I1,3
I2,2

I3,1

1/3

1/15

## Romberg Integration: Continuation of earlier integration example.

1.5

Example 1:

I =

ce x dx = 1.0000000

1.2

Trap
n
1
2
4
8
16
32
64

Trap
O(h2)
1.0519096
1.0049891
1.0009873
1.0002322
1.0000572
1.0000142
1.0000036

Simpson
O(h4)
0.9893489
0.9996533
0.9999806
0.9999988
0.9999999
1.0000000

Boole
O(h6)
1.0003403
1.0000024
1.0000000
1.0000000
1.0000000

O(h8)
0.9999970
1.0000000
1.0000000
1.0000000

Example 2:

I (a+1)x a dx

a=6

Trap
n

Trapezoid
O(h2)

Simp 1/3
O(h4)

Boole 5-pt
O(h6)

O(h8)

3.500000

1.239583

1.002604

1.000000

2
4
8
16

1.804688
1.214233
1.054403
1.013654

1.017415
1.001127
1.000071

1.000041
1.000001

1.000000

O(h10)
1.000000

Example 3:

I (a+1)x a dx

a = 1.5

Trap

Trapezoid

Simp 1/3

Boole 5-pt

O(h2)

O(h4)

O(h6)

O(h8)

1
2
4
8
16

1.250000
1.066942
1.017545
1.004531
1.001159

1.005922
1.001080
1.000193
1.000034

1.000757
1.000134
1.000024

1.000124
1.000022

O(h10)
1.000022

## Gauss Quadrature (C&C 4th ed., Sec. 22.3, p. 620)

Requires: (x) to be explicitly known so we can pick any xi
Approach:

n 1

i 0

f(x)dx f(x ) R
i

i = weighting factors
xi = sampling points selected optimally

where:

## Rn= truncation error

Pick points & weights cleverly to integrate a polynomial of order (2n - 1) exactly.
For n = 2 Gauss Quadrature will be accurate for cubics.
Trapezoidal Rule is accurate for linear functions.
For n=3 exact result for polynomials of order up to and including 5
With 3 points we want exact results for polynomials of order 5 over the interval [-1,
+1].
1

1dx = 2 = 01 + 11 + 21

## xdx = 0 = 0x0 + 1x1 + 2x2

1
1

x 2 dx =

2
= 0x02 + 1x12 + 2x22
3
1

## x dx = 0 = 0x0 + 1x1 + 2x2

3

1
1

x 4 dx =

2
= 0 x04 + 1 x14 + 2 x24
5
1

x dx = 0 = 0 x0 + 1 x1 +
5

## With six equations there are six unknowns,

i.e., 3 unknown weights (0, 1, 2) and
3 unknown sampling points (x0, x1, x2).

2 x25

## Section 6: Numerical Integration

Solving these six equations for the six unknowns results in:
0 = 0.555555556

1= 0.888888889

2= 0.555555556

x0 = 0.774596669

x1 = 0.000000000

x2 = +0.774596669

## I 0 f(x0) + 1 f(x1) + 2 f(x2)

See C&C Table 22.1, p. 626, for Gauss-Legendre formulas, n = 2, 3, 4, 5, and 6.
Gauss Quadrature Truncation Error (C&C 4th ed., Sec. 22.3.4, p. 627)
In general, n sampling points will provide an exact solution for a 2n1 order
polynomial. With n = # of sampling points
2n 1
2(b a)
n!4

Rn =
3
(2n 1) (2n)!

f(2n)()

## (similar to C&C except their n = # pts.1)

Because (b-a) = h, the error with the composite Gauss rule is O (h2n) globally with
n = # of pts..
This shows a superiority of order over the Newton-Cotes formulas.
Higher order need not always mean higher accuracy.
Arbitrary intervals: Gauss Quadrature: When the limits of integration are [a,b]
instead of [1,+1] (C&C, p. 621)
1

f(x)dx .

## Assume that there is a variable x which is linearly related to xd such that:

y = a0 + a1x
If y = a, corresponds to x = 1 and y = b, corresponds to x = 1, then
a = a0 + a1(1)

and

b = a0 + a1(+1)

a0 = (b+a)/2

and

a1 = (ba)/2

y=

Thus,

ba
ba
+ 2 x
2

-1

f(y)dy =

and

dy =

ba
dx
2

ba
(b a) (b-a)x b a
f
dx

2
2

n 1

i f(y[xi ])

i 0

## Section 6: Numerical Integration

1.5

Example 1: Evaluate

0.2

## (1.5 0.2) (15 0.2)x

= 0.65 x + 0.85
2
1 .5 0 .2
dy =
dx = 0.65 dx
2

1. Change of variables:

y=

1.5

2. With

y2

dy = 0.65

e[0.65x 0.85] dx

0.2

2
2
2
I 0.65{0 e[0.65x0+0.85] + 1 e[0.65x1+0.85] + 2 e[0.65x2+0.85] }

where:

0 = 0.555555556
1 = 0.888888889

x0 = 0.774596669

2 = 0.555555556

x2 = +0.774596669

x1 = 0.000000000

Itrue
Igauss
Note: with 3 points, i.e., 2 segments: Itrap
ISimp.1/3
1

Example 2:

= 0.65882
0.65860, et = 0.03%
= 0.66211, et = 0.50%
= 0.65181, et = 1.06%

c dy
2 cos( y/2)

n
1
2
4
8

pts
2
3
5
9

Romberg
1.08253170
1.00066008
0.99999114
0.99999987

Gauss Quad
0.99955157
1.00000812
1.00000000
1.00000000

n
1
2
4
8

pts
2
3
5
9

Romberg
O(h2)
O(h4)
O(h6)
O(h8)

Gauss Quad
O(h4)
O(h6)
O(h10)
O(h18)

## Gauss Quadrature Formulas for Special Integrals

In each of these schemes, f(x) is a well behaved function, that is,
continuously differentiable. Also each of these schemes has its own
tabulated sampling points and weighting factors (not in C&C).
Gauss-Legendre

f x dx

e x f x dx

Gauss-Laguerre

Gauss-Hermite

log-weighted
1

f x dx

ln x f x dx
0

Gauss-Chebyshev
1

f x
1 x2

dx

## Advantages (A) and Disadvantages (D) of Gauss Quadrature

A1) With n points obtain a formula that integrates polynomials 1, x, x2, ..., x2n-1
because of 2n free parameters: very high order.
Excellent for well-behaved functions!
A2) Special formulas are available and can be derived for special weighting
functions, and infinite and semi-infinite intervals.
D1) Errors can depend upon f (2n)() so the method is only dependable if high
order derivatives are well behaved.
D2) When composite Gauss formulas are applied (Gauss Quadrature applied to
separate panels), endpoints cannot be reused as they are with Trapezoid,
Simpson and Romberg.
D3) With an adaptive strategy that selectively subdivides intervals where
truncation error may be large, cannot reuse points as one can with Trapezoid
and Simpson. Gauss also fails to provide an estimate of local error as do
Trapezoid, Simpson, and Romberg by clever use of the same function values
with different weights.
D4) Inconvenient to remember special Gauss points and Gauss weights.

## Improper Integrals (C&C 4th ed., Sec. 22.4, p. 627)

1. Use Gauss Quadrature for that special interval.
2. Transform variables on the outer portion of the interval. (Beware of singularities at what
was infinity.) For any function that approaches zero at least as fast as 1/x2 as x
approaches infinity, perform change of variables:
1
x=
and dx = dt/t2
t
t 1/a

f(x)dx =

x a

t 1/b

1
f ( ) dt
t
t

(22.27)

which is valid for ab > 0. For integration limits that pass through zero, implement the
integration in two (or three) steps:
b

## f (x) dx f (x) dx f (x) dx

where a >0The first integral may be evaluated using equation (22.27) and the second
may be evaluated using a Newton-Cotes formula or Gauss Quadrature.
For a detailed example (on the cumulative normal distribution), see C&C Example
22.6, page 629-630.
Multiple Integrals (C&C 4th ed., Sec. 21.5, page 608)
bd

I =

f(x,y)dydx
a c

d

= f(x,y) dy dx

a c

j1

(x) f(x,y ) dx
j

i1

j1

ij f(xi ,y j ) =

i ij

f(xi ,y j )

i1 j1

## If d = d(x) or c = c (x), then values of ij will depend upon xi.

Another Idea: Monte Carlo methods for multivariate integration. Randomly selected
points from within the range of integration and average function values used. Sampling
error is n where n random points are generated.

## Integration when the Integrand contains a singularity (not covered in C&C)

In the following, the function f(x) is well behaved, i.e., it is continuously differentiable in the
neighborhood of the singularity of the integrand or its derivative. Here is an integral with a singularity at
the lower limit:
1

f (x)
dx problem at the singular point x = 0
x

In addition, if the derivative of the integrand goes to infinity, it is possible that the error goes to
infinity as well. An example of this situation is:
u

x f (x)dx

## To integrate, we need to control error near the singularity.

Divide the integration region to isolate singularity, then------Approach 1a: Try to subtract out singularity and numerically integrate only
the remainder.
u

x f (x)dx

## where f(x) is continuously differentiable at x = 0.

a) Subtract out the singularity at x = 0, and integrate numerically:
u

[f(x) - f(0)]

x dx

u

f (0)

x dx f (0)

1.5

1.5

0

1.5

1.5

## Page 6-16 of 6-16

Approach 1b: Even better, include the first derivative when isolating the singularity, i.e.,:
a) Subtract out the singularity at x = 0, and integrate numerically:
u

x dx

u

f (0) xf (0)

x dx f (0)

1.5

1.5

f (0)

2.5

2.5

u

1.5

1.5

f (0)

2.5

2.5

u

f (x) dx

## Let x = y2, then dx = 2ydy ==> xa f(x) dx = 2 y2a+1 f(y2) dy

For0<a<1, 1<2a+1 so the singularity is eliminated.
Does not entirely solve the problem, but makes it less severe.
Use Trapezoid or Simpson rule near the problem. For a = :
u

f (x)

x dx f (y ) y(2y)dy 2 f (y 2 ) y 2 dy

## Approach 3: Integrate by parts:

x f(x)dx x
a

a 1

f(x) x a 1f '(x)dx

## Approach 4: Use the appropriate Gauss Quadrature approaches specialized for

the particular singular function (see above).