# Chapter

---
Numerical integration

Aman W
Department of Applied Physics
University of Gondar
Contents

• Introduction
• Numerical integration
– Simple rules
– Romberg Integration
– MONTE CARLO INTEGRATION (BY STONE
THROWING)
• References: Numerical Recipes has a pretty good discussion
Introduction

• The function to be integrated
will typically be in one of the
following three forms:
– A simple continuous function
such as polynomial, an
exponential, or a
trigonometric function.
– A complicated continuous
function that is difficult or
impossible to differentiate or
integrate directly.
– A tabulated function where
values of x and f(x) are given
at a number of discrete points,
as is often the case with
experimental or field data.
Numerical integration

• Integration:
?
? = න ? ? ??
?

is the total value, or
summation, of f(x) dx over
the range from a to b:
• The integration of a function
may require some
mathematical concept to do
analytically but
• When a function cannot be integrated analytically, or is very
difficult to integrate analytically, one generally turns to
numerical integration methods.
• A traditional way to perform numerical integration by hand is
to take a piece of graph paper and count the number of boxes
or quadrilaterals lying below a curve of the integrand.
• For this reason numerical integration is also called numerical
quadrature even when it becomes more sophisticated than
simple box counting.
• Quadrature has become a synonym for numerical integration
– Primarily in 1d,
– higher dimensional evaluation is sometimes dubbed cubature
Numerical integration

• Riemann definition of an
intgeral is: sum over boxes
width h
• The Riemann definition of
an integral is the limit of the
sum over boxes as the width
h of the box approaches zero
(Fig):

? ? is the area under the graph of f(x)
from a to b. Here we break up the
න ? ? ?? = lim ෍ ? ? ℎ
ℎ−0 area into four regions of equal widths
? ?=1 h.
• The numerical integral of a function f(x) is approximated as
the equivalent of a finite sum over boxes of height, f(x) and
width, wi: The integral can be written as

?
‫? ?׬‬ ? ?? ≈ σ?
?=1 ? ?? ?? 1

• which is similar to the Riemann definition equ() except that
there is no limit to an infinitesimal box size.
– The wi are weights, and
– the xi are abscissas
• This Equation () is the standard form for all integration
algorithms;
• The goal of any numerical integration method is to choose
abscissas and weights such that errors are minimized for the
smallest n possible for a given function
• To get a unique solution using numerical integration
– Assuming that f(x) is finite and continuous on the interval [a,b]
– The function f(x) is evaluated at N points in the interval [a; b], and the
function values fi = f(xi) are summed with each term in the sum
weighted by wi.
• While in general the sum in eq(1) gives the exact integral only
when N →∞, it may be exact for finite N if the integrand is a
polynomial.
• Generally, the precision increases as N gets larger, with round-
off error eventually limiting the increase.
• The different integration algorithms amount to different ways
of choosing the points and weights.
• There are many integration algorithms, Quadrature. Some of
– Newton Cotes Formula
• Trapezoid Rule
• Simpson's Rule
– Monte Carlo
• The trapezoid & Simpson’s rule are examples of Newton-
• All of them integrand f (x) ≈ polynomial of order O = 1, 2, N
Newton-Cotes Integration Formulas

• The Newton-Cotes formulas are the most common numerical
integration schemes.
• They are based on the strategy of replacing a complicated
function or tabulated data with an approximating function that
is easy to integrate:

b b
I   f ( x)dx   f n ( x)dx
a a

f n ( x)  a0  a1 x    a n1 x n1  a n x n
Newton-Cotes Examples
• The integrating function can be
polynomials for any order - for
example,
– (a) straight lines or
– (b) parabolas.
• The integral can be
approximated in one step or in a
series of steps to improve
accuracy.
The Trapezoidal Rule
• The Trapezoidal rule is the first of
Newton-Cotes closed integration
formulas, corresponding to the
case where the polynomial is first
order, it uses a straight-line
approximation for the function:
b b
I   f ( x)dx   f1 ( x)dx
a a

• The area under this first order
polynomial is an estimate of the f (a)  f (b)
integral of f(x) between the limits I  (b  a)
2
of a and b: Trapezoidal rule
• Error of the Trapezoidal Rule
• When we employ the integral under a straight line segment to
approximate the integral under a curve, error may be
substantial:
1
Et   f ( )(b  a ) 3
12
• where ? lies somewhere in the interval from a to b.
Composite Trapezoidal Rule

• One way to improve the accuracy of
the trapezoidal rule is to divide the
integration interval from a to b into a
number of segments and apply the
method to each segment
• We use N points xi(i = 1; N) evenly
spaced at a distance h apart
throughout the integration region [a;
b] and include the endpoints.
• This means that there are (N -1)
intervals of length h:

?−?
ℎ=
?−1

?? = ? + ? − 1 ℎ
Composite Trapezoidal Rule
• The areas of individual segments can then be added to yield
the integral for the entire interval.
x1 x2 xn

I  f ( x)dx   f ( x)dx     f ( x)dx
x0 x1 xn 1

• Substituting the trapezoidal rule for each integral yields:
f ( x0 )  f ( x1 ) f ( x1 )  f ( x2 ) f ( xn 1 )  f ( xn )
I h h  h
2 2 2

• The trapezoid rule takes each integration interval i and
constructs a trapezoid of width h in it (Fig).
Composite Trapezoidal Rule

• This approximates f(x) by a
straight line in each interval
i and uses the average height
(fi+fi+1)/2 as the value for f.
• The area of each such
trapezoid is

xi  h h f i  f i 1  1

xi
f ( x)dx 
2

2
hf i  1 hf i 1
2
• In order to apply the trapezoid rule to the entire region [a; b],
we add the contributions from each subinterval:
b h h
a
f ( x)dx 
2
f i  hf 2  ...hf N 1  hf N
2

• You will notice that because the internal points are counted
twice (as the end of one interval and as the beginning of the
next), they have weights of h/2 + h/2 = h,
• whereas the endpoints are counted just once and on that
account have weights of only h/2.
• In terms of our standard integration rule equ(), we have
?? = ℎൗ2 , ℎ, … , ℎ, ℎൗ2
• Integrate the function f ? = 2? 2 − ? − 4 from a=0 to b=5 using
the Trapezoidal rule,
• finding the integral by hand is straight forward and we can calculate
the value of the integral by hand, which turns out to be 50.833
• Trapezoidal rule is defined as
b h h
a
f ( x)dx 
2
f i  hf 2  ...hf N 1  hf N
2
• The Trapezoidal rule is implemented by taking a general
mathematical function f(x) as argument, together with the input data
for the problem
• The integration limits a and b and the numerical resolution
parameter n.
– Function g(t), a=0, b=5, n=100.
import numpy as np a = 0.0 # Setup input values
def f(x): b = 5.0
return 2.0*x**2 - x - 4.0 n=4
s = trapezoidal(f, a, b, n, True) # call
trapezoidal function
def trapezoidal(f, a, b, n):
print s
x = np.linspace(a, b, n+1)
y = f(x)

for i in range(n+1):
print "%5d %10.4f %10.4f" % (i,
x[i], y[i])
s = y[0] + 2.0 * sum(y[1:n]) +
y[n]
h = float(b - a) / n
return s * h / 2.0
def main(): • def Trapezoidal(f, a, b, n):
– h = (b-a)/float(n)
a = 0.0 – s = 0.5*(f(a) + f(b))
b = 5.0 – for i in range(1,n,1):
• s = s + f(a + i*h)
s = trapezoidal(f, a, b, 4) return h*s
print "%5d %12.4f" % (4, s)
• from math import exp # or from math import *
s = trapezoidal(f, a, b, 10) • def g(t):

print "%5d %12.4f" % (10, s) •
return exp(-t**4)
a = -2; b = 2 n = 1000
s = trapezoidal(f, a, b, 20) •

result = Trapezoidal(g, a, b, n)
print result
print "%5d %12.4f" % (20, s)
This will produce the output:
4 53.4375
10 51.2500
20 50.9375
Simpson’s Rules

• More accurate estimate of an integral is obtained if a high-
order polynomial is used to connect the points. The formulas
that result from taking the integrals under such polynomials
are called Simpson’s rules. Simpson’s 1/3 Rule
• Results when a second-order interpolating polynomial is used.
Simpson’s rule

• For each interval, Simpson’s
rule approximates the
integrand f(x) by a parabola
(Fig ):
• Using that Lagrange form for a
For each interval, Simpson’s
rule approximates the
integrand f(x) by a parabola
? ? = ?? 2 + ?? + ?
• with all intervals equally
spaced.
Simpson’s 1/3 Rule
• Simpson’s 1/3 rule corresponds to using second-order
polynomials. Using the Lagrange form for a quadratic fit of
three points: Integration over the three points simplifies to:
b b
I   f ( x ) dx  f 2 ( x ) dx
a a

a  x0 b  x2
 ( x  x1 )( x  x2 ) ( x  x0 )( x  x2 ) ( x  x0 )( x  x1 ) 
x2

I x  ( x0  x1 )( x0  x2 ) 0 ( x1  x0 )( x1  x2 ) 1 ( x2  x0 )( x2  x1 ) 2 dx
f ( x )  f ( x )  f ( x )
0

ba
I
h
 f ( x0 )  4 f ( x1 )  f ( x2 ) h
3 2

Simpson’s 1/3 Rule
Error of Simpson’s 1/3 Rule

• An estimate for the local truncation error of a single
application of Simpson’s 1/3 rule is:
1
f  b  a
4  5
Et  
2880
where again  is somewhere between a and b.
• This formula indicates that the error is dependent upon the

fourth-derivative of the actual function as well as the distance
between the points.
• Note that the error is dependent on the fifth power of the
step size (rather than the third for the trapezoidal rule).
• Error can thus be reduced by breaking the curve into parts.
Composite Simpson’s 1/3 Rule
• Simpson’s 1/3 rule can be used on a
set of subintervals in much the same
way the trapezoidal rule was, except
there must be an odd number of
points.
• Because of the heavy weighting of the
internal points, the formula is a little
more complicated than for the
trapezoidal rule:

   
xn x2 x4 xn
I x0
fn x dx  x0
fn x  dx  x2
fn x  dx   x n2
fn x dx
h h h
I
3
 f  0
x  4 f  1   2   f x2  4 f x3  f x4 
x  f x 
3

3
 f xn2   4 f xn1   f xn 

 n1 n2

h  
I  f x0   4  f xi   2  f xi   f xn 
3


i1
i, odd
j2
j, even 

• Simpson’s rule requires the elementary integration to be over
pairs of intervals, which in turn requires that the total number
of intervals be even or that the number of points N be odd.
• In order to apply Simpson’s rule to the entire interval, we add
up the contributions from each pair of subintervals, counting
all but the first and last endpoints twice:
?
ℎ 4ℎ 2ℎ 4ℎ 4ℎ ℎ
න ? ? ?? ≈ ?1 + ?2 + ?3 + ?4 + ⋯ + ??−1 + ??
? 3 3 3 3 3 3

• In terms of our standard integration rule, we have
ℎ 4ℎ 2ℎ 4ℎ 4ℎ ℎ
?? = ൗ3 , , , … , , ൗ3
3 3 3 3

• The integral can be approximated by weighted sum

n
I    i f ( xi )
i 1

• The wi are weights, and the xi are abscissas
• Gaussian quadrature achieves high accuracy and efficiency by
optimally selecting the abscissas
• It is usual to apply a change of variables to make the integral
map to [-1,1]
• There are also a number of different families of Gaussian

• So far we have considered regular spaced abscissas, although
we have considered the possibility of adapting spacing
• We’ve also looked solely at closed interval formulas
• Gaussian quadrature achieves high accuracy and efficiency by
optimally selecting the abscissas
• It is usual to apply a change of variables to make the integral
map to [-1,1]
• There are also a number of different families of Gaussian
• Let’s look at a related example first
Midpoint Rule: better error properties than
expected
b Despite fitting a
 f ( x)dx  (b  a) f ( x
a
mid ) single value error
occurs in second
derivative
 a  b  (b  a )
3
 (b  a ) f   f ' ' ( x )
 2  24
f(x)

a xm b x
Coordinate transformation on to [-1,1]

• The transformation is a simple linear mapping

ba ba
t x
2 2
 x  1  t  a

x  1  t  b

a t1 t2 b
b 1 ba ba ba 1
a
f ( t )dt  
1
f(
2
x
2
)(
2
)dx   g( x )dx
1
Recall the original series approximation
n
I   f ( x)dx   ci f ( xi )  c1 f ( x1 )  c2 f ( x2 )    cn f ( xn )
1

1
i 1

Consider, n=2, then we have
1
 1
f(x)dx  c1f(x 1 )  c2f(x 2 )

-1 x1 x2 1
• We have 4 unknowns, c1, c2, x1, x2, so we can choose these
values to yield exact integrals for f(x)=x0,x,x2,x3
1

1
f(x)dx  c1f(x 1 )  c2f(x 2 )
• Evaluate the integrals for f = x0, x1, x2, x3
• Yields four equations for the four unknowns

f 1
 1   1dx  2  c 1  c 2 c 1  1
 1
c  1
 1
 2
f  x   xdx  0  c 1 x 1  c 2 x 2 1
1 
 2
1
  x1 
f  x   x dx   c 1 x 12  c 2 x 22
2 2
 3
1
 3  1
f 1
 x   x 3 dx  0  c 1 x 13  c 2 x 23
3  x2  3
 1

1 1 1
I   f ( x)dx  f ( )  f ( )
1
3 3
Summary

• Low order Newton-Cotes formulae have comparatively slow
convergence properties
– Higher order methods have better convergence properties
but can suffer from numerical instabilities
– High order ≠ high accuracy
• Applying Richardson Extrapolation to the compound trapezoid
rule gives Romberg Integration
– Very good convergence properties for a simple method
• Gaussian quadrature, and related methods, show good stability
and are computationally efficient
– Implemented in many numerical libraries
Python Libraries

• Finding roots is a common-enough problem that
people have written some good libraries to take care
of it for you.
• Scipy .optimize, a subset of the scientifc
Python(scipy) library. Scipy includes
implementations of the bisection method (bisect ()),
Newton's method (newton()), and several others
routine.
• Scipy provides higher-level tools such as numeric
integration, differential equation solvers, and so on.
Packages in Scipy

• Special Functions • Input/Output (scipy.io)
(scipy.special) • Genetic Algorithms
• Signal Processing (scipy.ga)
(scipy.signal) • Statistics (scipy.stats)
• Fourier Transforms • Distributed Computing
(scipy.fftpack) (scipy.cow)
• Optimization • Fast Execution (weave)
(scipy.optimize) • Clustering Algorithms
• General plotting (scipy.[plt, (scipy.cluster)
xplt, gplt]) • Sparse Matrices*
• Numerical Integration (scipy.sparse)
(scipy.integrate)
• Linear Algebra (scipy.linalg)
• Numpy provides:
• extension package to Python for multi-dimensional arrays
• closer to hardware (efficiency)
• designed for scientific computation (convenience)
• Also known as array oriented computing
• Numpy: a package to provide real matrices to Python.
• Numpy uses the LINPACK linear algebra package, which is
written in FORTRAN and is extremely fast.
• Numpy, short for “Numerical Python",
• Numpy is at the base of Python’s scientific stack of tools. Its
purpose is simple: implementing efficient operations on
many items in a block of memory. Understanding how it
works in detail helps in making efficient use of its flexibility
Overview of NumPy

• N-D array • Universal
• N-dimensional array of Function(UFunc)
rectangular data • functions that operate element-by-
element and return result
• Element of the array can • fast-loops registered for each
be C-structure or simple fundamental data-type
• sin(x) = [sin(xi) i=0..N]
data-type.
• x+y = [xi + yi i=0..N]
• Fast algorithms on
machine data-types (int,
float, etc.)
Congra!

• Wishing U all the best in your Future Carrier