Lecture 3 1
Interpolation, Smoothing, Extrapolation
A typical numerical application is to find a smooth parametrization
of available data so that results at intermediate (or extended)
positions can be evaluated.
What is a good estimate for
y for x=4.5, or x=15 ?
Options: if have a model, y=f(x),
then fit the data and extract
model parameters. Model then
used to give values at other
points.
If no model available, then use a
smooth function to interpolate
Winter Semester 2006/7 Computational Physics I
Lecture 3 2
Interpolation
Start with interpolation. Simplest  linear interpolation. Imagine
we have two values of x, x
a
and x
b,
and values of y at these points,
y
a
, y
b
. Then we interpolate (estimate the value of y at an
intermediate point) as follows:
y = y
a
+
(y
b
y
a
)
(x
b
x
a
)
(x x
a
)
a
b
Winter Semester 2006/7 Computational Physics I
Lecture 3 3
Interpolation
Back to the initial plot:
y = y
a
+
(y
b
y
a
)
(x
b
x
a
)
(x x
a
)
Not very satisfying. Our intuition
is that functions should be
smooth. Try reproducing with a
higher order polynomial. If we
have n+1 points, then we can
represent the data with a
polynomial of order n.
Winter Semester 2006/7 Computational Physics I
Lecture 3 4
Interpolation
Fit with a 10th order polynomial.
We go through every data point
(11 free parameters, 11 data
points). This gives a smooth
representation of the data and
indicates that we are dealing with
an oscillating function. However,
extrapolation is dangerous !
Winter Semester 2006/7 Computational Physics I
Lecture 3 5
Lagrange Polynomials
For n+1 points (x
i
,y
i
), with
there is a unique interpolating polynomial of degree n with
i = 0,1,, n x
i
x
j i
p(x
i
) = y
i
i = 0,1,, n
Can construct this polynomial using the Lagrange polynomials,
defined as:
L
i
(x) =
(x x
0
)(x x
i 1
)(x x
i +1
)(x x
n
)
(x
i
x
0
)(x
i
x
i 1
)(x
i
x
i +1
)(x
i
x
n
)
Degree n (denominator is constant), and
L
i
(x
k
) =
i, k
Winter Semester 2006/7 Computational Physics I
Lecture 3 6
Lagrange Polynomials
The Lagrange Polynomials can be used to form the interpolating
polynomial:
p(x) = y
i
L
i
(x)
i =0
n
= y
i
i =0
n
x x
k
x
i
x
k
k =0, k i
n
*
* Example: 10th order polynomial
Lagrange=0.
*
Do I=0,10
term=1.
Do k=0,10
If (k.ne.I) then
term=term*(xx0(k))/(x0(I)x0(k))
Endif
Enddo
Lagrange=Lagrange+Y0(I)*term
Enddo
Winter Semester 2006/7 Computational Physics I
Lecture 3 7
Lagrange Polynomials
Error estimation of the interpolation/extrapolation:
Define err(x) = f (x) p(x) where f (x) is original function,
p(x) is interpolating function
Choose x x
i
for any i = 0,1,, n
Now define
F(x) = f (x) p(x) f (x) p(x) ( )
(x x
i
)
i =0
n
(x x
i
)
i =0
n
Now look at properties of F(x)
Winter Semester 2006/7 Computational Physics I
Lecture 3 8
Inter and Extrapolation Error
F(x
i
) = 0 for all i = 0,1,, n and F(x) = 0.
I.e., F(x) has n + 2 zeroes
Rolle's theorem: There exists a between x, x
0
, x
1
,, x
n
such that
F
(n+1)
() = 0
so,
0 = f
(n+1)
() f (x) p(x) ( )
(n +1)!
(x x
i
)
i =0
n
p
(n+1)
= 0
( )
but x is arbitrary, so
e(x) =
f
(n+1)
()
(n +1)!
(x x
i
)
i =0
n
for some between x, x
0
, x
1
,, x
n
Winter Semester 2006/7 Computational Physics I
Lecture 3 9
Inter and Extrapolation Error
Suppose we are trying to interpolate a sine function (our example).
We have 11 data points (n=10). Then
f
(n+1)
(x) =
d
11
sin x
dx
11
= cos x so f
(n+1)
() 1
e(x)
(x x
i
)
i =0
n
(n +1)!
<
(b a)
11
11!
for interpolation
For extrapolation, the error grows as the power (n+1)
Winter Semester 2006/7 Computational Physics I
Lecture 3 10
Splines
A spline is polynomial interpolation between the data points which
satisfies the following conditions:
Assume have data x
k
, y
k
{ }
with k = 0, n i.e., n +1 points
(also known as knots)
Define a = x
0
, b = x
n
and arrange so that x
0
< x
1
<< x
n1
< x
n
1. S(x) = S
k
(x) for x
k
x x
k +1
k = 0,1,, n 1
2. S(x
k
) = y
k
k = 0,1,, n
3. S
k
(x
k +1
) = S
k +1
(x
k +1
) k = 0,1,, n 2 i.e., S(x) is continuous
Winter Semester 2006/7 Computational Physics I
Lecture 3 11
Splines
S
k
(x) = y
k
+
y
k +1
y
k
x
k +1
x
k
(x x
k
)
Linear Spline:
Quadratic Spline:
S
k
(x) = y
k
+ z
k
(x x
k
) +
z
k +1
z
k
2(x
k +1
x
k
)
(x x
k
)
2
z
0
has to be fixed, for example from requiring S
k
(a) = z
0
= 0.
then,
z
k +1
= z
k
+ 2
y
k +1
y
k
x
k +1
x
k
Winter Semester 2006/7 Computational Physics I
Lecture 3 12
Splines
The cubic spline satisfies the following conditions:
1. S(x) = S
k
(x) for x
k
x x
k +1
k = 0,1,, n 1
S
k
(x) = S
k, 0
+ S
k,1
(x x
k
) + S
k, 2
(x x
k
)
2
+ S
k, 3
(x x
k
)
3
2. S(x
k
) = y
k
k = 0,1,, n
3. S
k
(x
k +1
) = S
k +1
(x
k +1
) k = 0,1,, n 2 i.e., S(x) is continuous
4. S
k
(x
k +1
) = S
k +1
(x
k +1
) k = 0,1,, n 2 i.e., S (x) is continuous
5. S
k
(x
k +1
) = S
k +1
(x
k +1
) k = 0,1,, n 2 i.e., S (x) is continuous
Winter Semester 2006/7 Computational Physics I
Lecture 3 13
Splines
Need at least 3rd order polynomial to satisfy the conditions.
Number of parameters is 4n. Fixing S
k
(x
k
)=y
k
gives n+1
conditions. Fixing S
k
(x
k+1
)=S
k+1
(x
k+1
) gives an additional n1
conditions. Matching the first and second derivative gives another
2n2 conditions, for a total of 4n2 conditions. Two more
conditions are needed to specify a unique cubic spline which
satisfies the conditions on the previous page:
S (a) = 0 S (b) = 0
Natural cubic spline
Can take other options for the boundary conditions
Winter Semester 2006/7 Computational Physics I
Lecture 3 14
Splines
Linear spline
Quadratic spline
Cubic spline
Winter Semester 2006/7 Computational Physics I
Lecture 3 15
Cubic Splines
The cubic spline is optimal in the following sense:
1. It is accurate to fourth order, and
2. It is the minimum curvature function linking the set of data
points.
f(x)S(x)
5
384
max
axb
f
4 ( )
(x) h
4
where h = max
k
x
k +1
x
k
Cubic spline satisfies S (x)
[ ]
2
dx
a
b
f (x)
[ ]
2
a
b
dx
Curvature is defined as
f (x)
1+ f (x)
2
( )
3/ 2
f (x)
Any smooth interpolating
function must have
curvature at least as
large as a cubic spline
Winter Semester 2006/7 Computational Physics I
Lecture 3 16
Cubic Splines
Proof of 2.
Start with algebraic identity F
2
S
2
= (F S)
2
2S(S F)
Let F = f (x), S = S (x)
then
f (x)
[ ]
2
a
b
dx S (x)
[ ]
2
a
b
dx = f (x) S (x)
[ ]
2
a
b
dx 2 S (x) S (x) f (x)
[ ]
a
b
dx
f (x) S (x)
[ ]
2
a
b
dx 0
S (x) S (x) f (x)
[ ]
a
b
dx = S (x) S (x) f (x)
[ ]
x
k
x
k+1
dx
k =0
n1
Now we use integration by parts to solve the integrals
Winter Semester 2006/7 Computational Physics I
Lecture 3 17
Cubic Splines
Recall:
u(x) v (x)dx
a
b
= u(x)v(x)
a
b
u (x)v(x)dx
a
b
´´ S (x) ´´ S (x) ÷ ´´ f (x) ( )
a
b
l
dx = ´´ S (x) ´ S (x) ÷ ´ f (x) ( )
x
k
x
k+1
÷ ´´´ S (x) ´ S (x) ÷ ´ f (x) ( )
x
k
x
k+1
l
dx

.
`
·
k =0
n÷1
¯
The first term is
´´ S (x) ´ S (x) ÷ ´ f (x) ( )
x
k
x
k+1
k =0
n÷1
¯
= ´´ S (b) ´ S (b) ÷ ´ f (b) ( ) ÷ ´´ S (a) ´ S (a) ÷ ´ f (a) ( )
= 0 From the boundary conditions for natural cubic spline
´´´ S (x) is a constant (since we have a cubic) and can be taken out of the integral, so
´´ S (x) ´´ S (x) ÷ ´´ f (x) ( )
a
b
l
dx = ÷ ´´´ S
k
´ S (x) ÷ ´ f (x) ( )
x
k
x
k+1
l
k =0
n÷1
¯
dx
= ´´´ S
k
k =0
n÷1
¯
S(x) ÷ f (x) ( )
x
k
x
k+1
but since S(x
k
) = f (x
k
), = 0
Winter Semester 2006/7 Computational Physics I
Lecture 3 18
Cubic Splines
f (x)
[ ]
2
a
b
dx S (x)
[ ]
2
a
b
dx 0
We have proven that a cubic spline has a smaller or equal
curvature than any function which fulfills the interpolation
requirements. This also includes the function we started with.
Physical interpretation: a clamped flexible rod picks the minimum
curvature to minimize energy  spline
Winter Semester 2006/7 Computational Physics I
Lecture 3 19
Data Smoothing
If we have a large number of data points, interpolation with
polynomials, splines, etc is very costly in time and multiplies the
number of data. Smoothing (or data fitting) is a way of reducing.
In smoothing, we just want a parametrization which has no model
associated to it. In fitting, we have a model in mind and try to
extract the parameters.
Data fitting is a full semester topic of its own.
A few brief words on smoothing of a data set. The simplest
approach is to find a general function with free parameters which
can be adjusted to give the best representation of the data. The
parameters are optimized by minimizing chi squared:
2
=
(y
i
f (x
i
;
))
2
w
i
2
i =0
n
Winter Semester 2006/7 Computational Physics I
Lecture 3 20
Data Smoothing
2
=
(y
i
f (x
i
;
))
2
w
i
2
i =0
n
are the parameters of the function to be fit
y
i
are the measured points at values x
i
w
i
is the weight given to point i
In our example, let’s take
f (x; A,) = Acos(x +)
And set
w
i
= 1 i
Now we minimize
2
as a function of A and
Winter Semester 2006/7 Computational Physics I
Lecture 3 21
Data Smoothing
Best fit for A=1, =3/2
Acos(x +) = 1 cos(x + 3 / 2) = cos x cos 3 / 2 sin xsin3 / 2 = sin x
Winter Semester 2006/7 Computational Physics I
Lecture 3 22
Exercises
1. Calculate the Lagrange Polynomial for the following data:
0.018 2.
0.135 1.
0.368 0.5
1 0
y x
2. For the same data, find the natural cubic spline coefficients.
Plot the data, the lagrange polynomial and the cubic spline
interpolations.
3. Smooth the data in the table with the function f(x)=Aexp(bx).
What did you get for A,b ?
Interpolation Start with interpolation. Simplest  linear interpolation. Imagine we have two values of x, xa and xb, and values of y at these points, ya, yb. Then we interpolate (estimate the value of y at an intermediate point) as follows:
(yb y = ya + (xb
ya ) (x xa )
xa )
b
a
Winter Semester 2006/7
Computational Physics I
Lecture 3 2
then we can represent the data with a polynomial of order n.Interpolation Back to the initial plot: (yb y = ya + (xb ya ) (x xa ) xa ) Not very satisfying. If we have n+1 points. Our intuition is that functions should be smooth. Winter Semester 2006/7 Computational Physics I Lecture 3 3 . Try reproducing with a higher order polynomial.
However. We go through every data point (11 free parameters.Interpolation Fit with a 10th order polynomial. extrapolation is dangerous ! Winter Semester 2006/7 Computational Physics I Lecture 3 4 . This gives a smooth representation of the data and indicates that we are dealing with an oscillating function. 11 data points).
defined as: (x x0 ) (x xi 1 )(x xi +1 ) (x xn ) Li (x) = (xi x0 ) (xi xi 1 )(xi xi +1 ) (xi xn ) Degree n (denominator is constant).1. . and Li (xk ) = Winter Semester 2006/7 i.Lagrange Polynomials For n+1 points (xi.1.yi). with i = 0. k Lecture 3 5 Computational Physics I .n xi xj i there is a unique interpolating polynomial of degree n with p(xi ) = yi i = 0. .n Can construct this polynomial using the Lagrange polynomials.
10 If (k. Do k=0. k x xk xk i xi * * Example: 10th order polynomial Lagrange=0. * Do I=0.10 term=1.I) then term=term*(xx0(k))/(x0(I)x0(k)) Endif Enddo Lagrange=Lagrange+Y0(I)*term Enddo Winter Semester 2006/7 Computational Physics I Lecture 3 6 .Lagrange Polynomials The Lagrange Polynomials can be used to form the interpolating polynomial: p(x) = n i=0 yi Li (x) = n i=0 n yi k = 0.ne.
p(x) is interpolating function Choose x xi for any i = 0. .1. n Now define n F(x) = f (x) p(x) ( f (x ) p(x )) (x i=0 n xi ) xi ) (x i=0 Now look at properties of F(x) Winter Semester 2006/7 Computational Physics I Lecture 3 7 .Lagrange Polynomials Error estimation of the interpolation/extrapolation: Define err(x) = f (x) p(x) where f (x) is original function.
x0 . xn Winter Semester 2006/7 Computational Physics I Lecture 3 8 .1. .and Extrapolation Error F(xi ) = 0 for all i = 0. . . F(x) has n + 2 zeroes Rolle's theorem: There exists a F (n +1) ( ) = 0 so. so f (n +1) ( ) e(x ) = (n + 1)! n (x i=0 xi ) for some between x.. between x. n I.Inter. x0 .e. xn such that ( ) ( f (x ) p(x )) (n + 1)! n (p (n +1) =0 ) (x i=0 xi ) but x is arbitrary. 0= f (n +1) and F(x ) = 0. x1 . x1 .
Inter. the error grows as the power (n+1) Winter Semester 2006/7 Computational Physics I Lecture 3 9 . Then d 11 sin x f (n +1) (x) = = cos x 11 dx n so f (n +1) ( ) 1 (x e(x ) i=0 xi ) (n + 1)! (b a)11 < for interpolation 11! For extrapolation. We have 11 data points (n=10).and Extrapolation Error Suppose we are trying to interpolate a sine function (our example).
S(xk ) = yk x xk +1 k = 0. n + 1 points (also known as knots) Define a = x0 . yk } with k = 0.1. . n i. b = xn and arrange so that x0 < x1 < < xn 1 < xn A spline is polynomial interpolation between the data points which satisfies the following conditions: 1..e.1. S(x) is continuous Computational Physics I Winter Semester 2006/7 Lecture 3 10 . n 1 k = 0. Sk (xk +1 ) = Sk +1 (xk +1 ) k = 0.Splines Assume have data { x k . n 2 i. .1.e.. . n 3. S(x) = Sk (x) for xk 2.
y yk zk +1 = zk + 2 k +1 xk +1 xk xk ) + Sk (x) = yk + zk (x Winter Semester 2006/7 Computational Physics I Lecture 3 11 . for example from requiring Sk (a) = z0 = 0. then.Splines Linear Spline: yk +1 Sk (x) = yk + xk +1 yk (x xk xk ) Quadratic Spline: zk +1 zk (x xk )2 2(xk +1 xk ) z0 has to be fixed.
S(xk ) = yk x xk +1 k = 0.e.1. S (x) is continuous Winter Semester 2006/7 Computational Physics I Lecture 3 12 .. Sk (xk +1 ) = Sk +1 (xk +1 ) k = 0.2 (x k = 0. n 2 i.e. n 1 xk )2 + Sk. 3 (x x k )3 xk ) + Sk. n 2 i. S (x) is continuous 5. .1.1. .e. Sk (xk +1 ) = Sk +1 (xk +1 ) k = 0. . S(x) = Sk (x) for xk Sk (x) = Sk..1 (x 2.0 + Sk..1. S(x) is continuous 4. . . Sk (xk +1 ) = Sk +1 (xk +1 ) k = 0. n 2 i.Splines The cubic spline satisfies the following conditions: 1.1. n 3.
Number of parameters is 4n. for a total of 4n2 conditions.Splines Need at least 3rd order polynomial to satisfy the conditions. Matching the first and second derivative gives another 2n2 conditions. Fixing Sk(xk+1)=Sk+1(xk+1) gives an additional n1 conditions. Two more conditions are needed to specify a unique cubic spline which satisfies the conditions on the previous page: S (a) = 0 S (b) = 0 Natural cubic spline Can take other options for the boundary conditions Winter Semester 2006/7 Computational Physics I Lecture 3 13 . Fixing Sk(xk)=yk gives n+1 conditions.
Splines Linear spline Cubic spline Quadratic spline Winter Semester 2006/7 Computational Physics I Lecture 3 14 .
Cubic Splines The cubic spline is optimal in the following sense: 1. Curvature is defined as (1 + f (x) ) (x)] dx 2 f (x) 2 3/2 f (x) b Cubic spline satisfies a Winter Semester 2006/7 [S (x)] dx 2 b [f a Computational Physics I Any smooth interpolating function must have curvature at least as large as a cubic spline Lecture 3 15 . It is the minimum curvature function linking the set of data points. It is accurate to fourth order. and f(x)S(x) 5 max f ( 4 ) (x) h 4 384 a x b where h = max k xk +1 xk 2.
S = S (x) then b S 2 = (F S)2 2S(S F) [f (x)] dx 2 b [S 2 (x)] dx = 2 b [f (x) S (x)] dx 2 S (x)[ S (x) 2 a b f (x)]dx a b a a [ f (x) S (x)] dx S (x)[ S (x) 0 n 1 xk+1 k = 0 xk a b f (x)]dx = S (x)[ S (x) f (x)]dx a Now we use integration by parts to solve the integrals Winter Semester 2006/7 Computational Physics I Lecture 3 16 .Cubic Splines Proof of 2. Start with algebraic identity F 2 Let F = f (x).
Cubic Splines Recall: b b b u(x)v (x)dx = u(x)v(x) a b a a u (x)v(x)dx xk+1 S (x) ( S (x) f (x))dx = n 1 k=0 S (x) ( S (x) f (x)) x xk+1 k S (x) ( S (x) f (x))dx a xk The first term is n 1 k=0 S (x) ( S (x) f (x)) x xk+1 k = S (b) ( S (b) f (b)) S (a) ( S (a) f (a)) = 0 From the boundary conditions for natural cubic spline S (x) is a constant (since we have a cubic) and can be taken out of the integral. so b S (x) ( S (x) f (x))dx = = n 1 k=0 n 1 k=0 xk+1 Sk xk ( S (x) f (x))dx xk+1 k a Sk ( S(x) f (x)) x but since S(xk ) = f (xk ). = 0 Winter Semester 2006/7 Computational Physics I Lecture 3 17 .
This also includes the function we started with. Physical interpretation: a clamped flexible rod picks the minimum curvature to minimize energy .spline Winter Semester 2006/7 Computational Physics I Lecture 3 18 .Cubic Splines b [ f (x)] dx 2 b [ S (x)] dx 2 0 a a We have proven that a cubic spline has a smaller or equal curvature than any function which fulfills the interpolation requirements.
In fitting. Smoothing (or data fitting) is a way of reducing. ))2 wi2 Lecture 3 19 Winter Semester 2006/7 Computational Physics I . we have a model in mind and try to extract the parameters. we just want a parametrization which has no model associated to it. A few brief words on smoothing of a data set. In smoothing. etc is very costly in time and multiplies the number of data. The parameters are optimized by minimizing chi squared: 2 = n i=0 (yi f (xi . Data fitting is a full semester topic of its own. splines. interpolation with polynomials. The simplest approach is to find a general function with free parameters which can be adjusted to give the best representation of the data.Data Smoothing If we have a large number of data points.
let’s take And set f (x. A. ))2 wi2 are the parameters of the function to be fit yi are the measured points at values xi wi is the weight given to point i In our example.Data Smoothing 2 = n i=0 (yi f (xi . ) = A cos(x + ) i wi = 1 2 Now we minimize as a function of A and Winter Semester 2006/7 Computational Physics I Lecture 3 20 .
=3 /2 A cos(x + ) = 1 cos(x + 3 / 2) = cos x cos 3 / 2 sin x sin 3 / 2 = sin x Winter Semester 2006/7 Computational Physics I Lecture 3 21 .Data Smoothing Best fit for A=1.
For the same data. Calculate the Lagrange Polynomial for the following data: x 0 0.018 2. y 1 0. find the natural cubic spline coefficients.5 1. Plot the data.Exercises 1. 3.135 0.b ? Winter Semester 2006/7 Computational Physics I Lecture 3 22 . What did you get for A.368 0. the lagrange polynomial and the cubic spline interpolations. Smooth the data in the table with the function f(x)=Aexp(bx). 2.