Function Approximation

David Florian Hoyle

UCSC

[11/14] November 2014

Objective

Obtain an approximation for f (x ) by another function b

f (x )
Two cases:
f (x ) is known in all its domain, but it is very expensive to calculate
it.
f (x ) is known only in a finite set of points: Interpolation.
Outline

1 Approximation theory
1 Weierstrass approximation theorem
2 Minimax approximation
3 Orthogonal polynomials and least squares
4 Near minimax apoproximation
2 Interpolation
1 The interpolation problem
2 Different representations for the interpolating polynomial
3 The error term
4 Minimizing the error term with Chebyshev nodes
5 Discrete least squares again
6 Piececwise polynomial interpolation: splines
Approximation methods

We want to approximate f (x ) : [a, b ] → R by a linear combination

of polynomials
n
f (x ) ≈ ∑ cj ϕ j ( x )
j =1

where
x ∈ [a, b ]
n : Degree of interpolation
cj : Basis coefficients
ϕj (x ) : Basis functions which are polynomials of degree ≤ n.
Approximation methods: Introduction

When trying to approximate f (x ) : C ⊂ R → R by

f (x ) ≈ ∑nj=1 cj ϕj (x )
We need to rely on a concept of distance between f (x ) and
∑nj=1 cj ϕj (x ) at the points we choose to make the approximation.
We are dealing with normed vector spaces of functions which can be
finite or infinite dimensional.
The space of continuos functions in Rn , the space of infinite
sequences l p , the space of measurable functions Lp and so on
Approximation methods: Introduction

If we define an inner product in this spaces then we have the

induced norm L2 :
  21
Zb
kf (x )kL2 =  f (x )2 w (x ) dx 
a

it is usefull for least squares approximation.

Approximation theory, is based on uniform convergence of f (x ) to
∑nj=1 cj ϕj (x ) . In this case, we use the supreme norm

kf (x )k∞ = max |f (x )|
x ∈[a,b ]
Approximation methods: Local vs Global

Local approximations are based on the Taylor approximation

theorem.
Global approximations are based on the Weierstrass approximation
theorem.
Approximation methods: Weierstrass approximation
theorem

Theorem
Let f (x ) be a continuous function on [a, b ] (i.e., f ∈ C [a, b ]) , then for
all e > 0,there exists a sequence of polynomials pn (x ) of degree ≤ n
that converges uniformly to f (x ) on [a, b ]. That is, ∀e > 0, ∃N ∈ N
and polynomials pn (x ) such that

∀n ≥ N, ∀x ∈ [a, b ] then kf (x ) − pn (x )k∞ ≤ ε

Where k·k∞ is the sup norm or L∞ norm:

kf (x ) − pn (x )k∞ = max |f (x ) − pn (x )|
x ∈[a,b ]

In other words,

lim pn (x ) = f (x ) for all x ∈ [a, b ]

n−→∞
Approximation methods: Weierstrass approximation
theorem

Another version

Theorem
if f ∈ C [a, b ],then for all e > 0 there exists a polynomial p (x ) such that

kf (x ) − pn (x )k∞ ≤ ε ∀x ∈ [a, b ]

This theorem tell us that any continuous function on a compact

interval can be approximated arbitrarily well by polynomials of any
degree.
Approximation methods: Weierstrass approximation
theorem

A constructive proof of the theorem is based on the Bernstein

polynomials defined on [0, 1]
n     
k n k
pn (x ) = ∑ f x (1 − x ) n −k
k =0
n k

such that
lim pn (x ) = f (x ) for all x ∈ [0, 1]
n−→∞
uniformly.
Weierstrass theorem is conceptually valuable but it is not
practical. From a computational point perspective it is not efficient
to work with Bernstein polynomials.
Bernstein polynomials converge very slowly!
Approximation methods: Minimax approximation

Also called the Best polynomial approximation

We want to find the polynomial pn (x ) of degree ≤ n that best
approximates a function f (x ) uniformly with f (x ) ∈ C [a, b ] :
For this, we are going to look for the infimun of the distance
between f and all possible degree ≤ n polynomial approximations
Recall that the uniform error term (i.e., using the L∞ norm) is

kf (x ) − pn (x )k∞ = max |f (x ) − pn (x )|
x ∈[a,b ]

Define dn (x ) as the infimun of the distance between f and all

possible pn (x ) approximations of f

dn (f ) = inf kf − pn k∞
pn
 
= inf max |f (x ) − pn (x )|
pn x ∈[a,b ]
Approximation methods: Minimax approximation

Let pn∗ (x ) be the polynomial for which the infimum is obtained

dn (f ) = kf − pn∗ k∞

It can be shown that pn∗ exists, it is unique and it is characterized

by a property called the equioscillation property.
Algorithms to compute pn∗ are difficult/complex/not efficients.
Example: Remez algorithm.
Standard solution: Chebyshev least squares polynomial
approximation closely approximate the minmax or best
polynomial.
This strategy is called: Near minmax approximation.
Approximation methods: Minimax approximation

Existence and uniqueness of the minimax polynomial:

Theorem
Let f ∈ C [a, b ]. Then for any n ∈ N there exists a unique pn∗ (x ) that
minimizes kf − pn k∞ among all polynomials of degree ≤ n.

Sadly, the proof is not constructive so we have to rely on an

algorithm that help us compute pn∗ (x ) .
Approximation methods: Minimax approximation

But we can characterize the error generated by the minmax

polynomial, pn∗ (x ) in terms of its oscillation property:

Theorem
Let f ∈ C [a, b ]. The polynomial pn∗ (x ) is the minimax polynomial of
degree n that approximates f (x ) in [a, b ], if and only if, the error
f (x ) − pn∗ (x ) , assumes the values + ∗
− kf − pn k∞ with an alternating
change of sign in at least n + 2 points in [a, b ]. That is, in
a ≤ x0 < x1 < ... < xn+1 ≤ b we have the following

f (x ) − pn∗ (x ) = (−1)j kf − pn∗ k∞ for j = 0, 1, 2, ...n + 1

Approximation methods: Minimax approximation

The theorem give us the desired shape of the error when we want
L∞ approximation (the best approximation).
For example it says that the maximum error of a cubic
approximation should be achieved at least five times and that the
sign of the error should alternate between these points.
Approximation methods: Minimax approximation

Example 1: Consider the function f (x ) = x 2 in the interval [0, 2].

Find the minimax linear approximation (n = 1) of f (x ) .
Consider the approximating polynomial p1 (x ) = ax + b and the
errors
e (x ) = x 2 − ax − b

Notice that the function e (x ) = x 2 − ax − b has a maximun when

2x − a = 0, that is when x = 2a .
a
x= 2 belongs to the interval [0, 2] when 0 ≤ a ≤ 4.

2 2 2 2
Evaluating e 2a = 2a − a 2a − b = a4 − a2 − b = − a4 − b

According to the oscillation property we need two more extremun

points:
Let’s take x = 0 and x = 2. Thus e (2) = 4 − 2a − b and
e (0) = −b.
Approximation methods: Minimax approximation

Example 1 (cont.):
According to the oscillation property we must have
h (0) = −h ( 2a ) = h (2) which implies that

h (0) = h (2)
−b = 4 − 2a − b
a=2

and
a
h (0) = −h
2
a2
−b = +b
4
but we already know that a = 2 thus
1
b=−
2
Approximation methods: Minimax approximation

Example 1 (cont.):
Then , the approximating polynomial is
1
p1∗ (x ) = 2x −
2
and the maximun error
1 1
max x 2 − 2x − =
[0,2] 2 2

If you try to find the quadratic minmax approximation to

f (x ) = x 2 , you will notice that things start to get complicated.
There is no general characterization-based algorithm to
compute the minimax polynomial approximation.
The Remez algorithm is based on known optimal
approximation for certain f (x ) .
Approximation methods: Minimax approximation

Example 1 (cont.):Oscillating property of the error for a minmax

approximation e (x ) = x 2 − 2x − 21 for x ∈ [0, 2]
v
Approximation methods: Least squares

Some basic concepts:

Weighting function: w (x ) on [a, b ] is any function that is positive
Zb
and has a finite integral over [a, b ],that is w (x ) dx < ∞
a
Inner product relative to w (x ): Let f , g ∈ C[a, b ] then the inner
product is
Zb
hf , g i = f (x ) g (x ) w (x ) dx
a
The inner product induces a norm given by:
  21
Zb
kf (x )kL2 =  f (x )2 w (x ) dx 
a
Approximation methods: Least squares

 n
Assume a general family of polynomials ϕj (x ) j =0 that
consititute a basis for C (X ) .
Assume that f (x ) is defined on [a, b ]. We want to approximate f by
a linear combination of polynomials: pn (x ) = ∑nj=0 cj ϕj (x )
Define the error as

E (x ) = f (x ) − pn (x )

The least squares approximation problem is to find the

polynomial of degree ≤ n that is closest to f (x ) in the L2 −norm
among all the polynomials of degree ≤ n.
  21
Zb
min kE (x )kL2 = min  E (x )2 w (x ) dx 
fb fb
a
Approximation methods: Least squares

Notice that minimizing the L2 −distance between f and pn (x )

(i.e., kE (x )kL2 ) is equivalent to minimizing the square of the
L2 −distance, given by kE (x )k2L2 :

Zb
min kE (x )k2L2 = min E (x )2 w (x ) dx
fb fb
a
Zb
= min (f (x ) − pn (x ))2 w (x ) dx
fb
a

The above problem is the continuos least square problem

Approximation methods: Least squares

Since pn (x ) = ∑nj=0 cj ϕj (x ) then solve the following problem

Zb
!2
n
minn w ( x ) f ( x ) − ∑ cj ϕ j ( x ) dx
{cj }j =0 a j =0

FOC’s w.r.t ck :

Zb
!
n
2 w (x ) f (x ) − ∑ cj ϕ j ( x ) ϕk (x ) dx = 0 ∀k = 0, 1, 2, ...n
a j =0
Approximation methods: Least squares

The normal equations are given by

n Zb Zb
∑ cj ϕk (x ) ϕj (x ) w (x ) dx = f (x ) ϕk (x ) w (x ) dx ∀k = 0, 1, 2, ...n
j =0 a a

Or in terms of inner product

n
∑ cj ϕk , ϕj = hf , ϕk i ∀k = 0, 1, 2, ...n
j =0
Approximation methods: Least squares

More explicitely, the normal equations are a system of n linear

equations in n unknowns:

c0 h ϕ0 , ϕ0 i + c1 h ϕ0 , ϕ1 i + c2 h ϕ0 , ϕ1 i + ... + cn h ϕ0 , ϕn i = hf , ϕ0 i
c0 h ϕ1 , ϕ0 i + c1 h ϕ1 , ϕ1 i + c2 h ϕ1 , ϕ2 i + ... + cn h ϕ1 , ϕn i = hf , ϕ1 i
...
c0 h ϕn , ϕ0 i + c1 h ϕn , ϕ1 i + c2 h ϕn , ϕ1 i + ... + cn h ϕn , ϕn i = hf , ϕn i
Approximation methods: Least squares

In matrix form
    
h ϕ0 , ϕ0 i h ϕ0 , ϕ1 i ... h ϕ0 , ϕn i c0 hf , ϕ0 i
 h ϕ1 , ϕ0 i h ϕ1 , ϕ1 i ... h ϕ1 , ϕn i   c1   hf , ϕ1 i 
  = 
 ... ... ... ...   ...   ... 
h ϕn , ϕ0 i h ϕn , ϕ1 i ... h ϕn , ϕn i cn hf , ϕn i

In more compact form

Hc = b
Approximation methods: Least squares

Candidates for ϕj (x ) :
Monomials constitute a basis for C (X ) :

1, x, x 2 , x 3 , ..., x n

Orthogonal polynomials relative to w (x ):

Zb
ϕk (x ) ϕj (x ) w (x ) dx = 0 for k 6= j
a
Approximation methods: Least squares

If we use Monomials the approximation for f (x ) can be written as

n
pn (x ) = ∑ cj x j
j =0

The least squares problem is

Zb
!2
n
minn f (x ) − ∑ cj x j
dx
{ cj } j = 0 a j =0

The FOC w.r.t ck :

Zb Zb
!
n
−2 f (x ) x k dx + 2 ∑ cj x j x k dx = 0 ∀k = 0, 1, 2, ...n
a a j =0
Approximation methods: Least squares

FOC’s are expressed as

Zb Zb
!
n
∑ cj x j +k
dx = f (x ) x k dx ∀k = 0, 1, 2, ...n
a j =0 a

Zb   Zb
Where ∑nj=0 cj x j +k dx = ∑nj=0 cj x j +k dx and
a a
Zb b
j +k +1 j +k +1 j +k +1
x j +k dx = jx+k +1 = b j +k−+a 1 thus,
a
a

Zb
!
n
b j +k +1 − aj +k +1
∑ j +k +1
cj = f (x ) x k dx ∀k = 0, 1, 2, ...n
j =0 a
Approximation methods: Least squares

The
 above is a linear system of equations with n + 1 unknowks
n
( cj j =0 ) and n + 1 equations.
If we assume that [a, b] = [0, 1] then the system of equations to
n
determine each of the cj j =0 becomes

n   Z1
1
∑ j +k +1
cj = f (x ) x k dx ∀k = 0, 1, 2, ...n
j =0 0
Approximation methods: Least squares

Explicitely the system is given by

       
1 1 1 1
c0 + c1 + c3 + ... + cn
k +1 k +2 k +3 k +n+1
Z1
= f (x ) x k dx ∀k = 0, 1, 2, ...n
0
Approximation methods: Least squares

for k = 0
      Z1
1 1 1
c0 + c1 + c3 + ... + cn = f (x ) dx
2 3 n+1
0
for k = 1
        Z1
1 1 1 1
c0 + c1 + c3 + ... + cn = f (x ) xdx
2 3 4 n+2
0
for k = 2
        Z1
1 1 1 1
c0 + c1 + c3 + ... + cn = f (x ) x 2 dx
3 4 5 n+3
0
for k = n
        Z1
1 1 1 1 f (x ) x n dx
n+1 c0 + n+2 c1 + n+3 c3 + ... + n+n+1 cn =
0
Approximation methods: Least squares

In matrix form, the system canbe written as

 
Z1

 f (x ) dx


0
 
 
   Z1 
1 1 1 ... 1   
2 3 n +1 c0  f (x ) xdx 
 1 1 1 ... 1  
c1 

0
 2  
 1 3 4 n +2    
 3 1 1 ... 1 
c2  =  Z1 
4 5 n +3 
f (x ) x 2 dx 
  
...  
 
 ... ... ... ... ...   
1 1 1 ... 1 cn  
n +1 n +2 n +3 n +n +1  0 

 ... 


 Z1 

 f (x ) x n dx 
0
or
Hc = b
H is a HILBERT MATRIX
Approximation methods: Least squares

Problems with linear systems of equations with Hilbert Matrices:

Hc = b
H is an ill-conditioned matrix: Increasing rounding errors as we
increase n.
Condition number increases as n increases.
Approximation methods: Least squares

 n
If the polynomial family ϕj (x ) j =0 is orthogonal, that is
h ϕn , ϕm i = 0 for n 6= m then the system becomes
    
h ϕ0 , ϕ0 i 0 0 ... 0 c0 hf , ϕ0 i

 0 h ϕ1 , ϕ1 i 0 ... 0   c1   h f , ϕ 1 i
 =


 ... ... ... ... ...   ...   ... 
0 0 0 ... h ϕn , ϕn i cn hf , ϕn i

In this case, the coefficients are given by

hf , ϕk i
ck = ∀k = 0, 1, 2, ...n
h ϕk , ϕk i
Thus
n
hf , ϕk i
pn (x ) = ∑ h
ϕ (x )
ϕk , ϕk i j
j =0
Approximation methods: Least squares

Computations for finding the coefficients {ck }nk =0 are easy to

perform when using a family of orthogonal polynomials to
approximate a function.
 n
A family of orthogonal polynomials ϕj (x ) j =0 have always a
recursive representation which make computations even faster.
Orthogonal polynomials: Most common families

There are many families of orthogonal polynomials that are basis for
function spaces:
Chebyshev: For x ∈ [−1, 1]
Tn (x ) = cos (n arccos x )
Legendre: For x ∈ [−1, 1]
(−1)n d n  2
n
Pn ( x ) = [ 1 − x ]
2n n! dx n
Laguerre: For x ∈ [0, ∞]
ex d n
x n e −x


Ln ( x ) =
n! dx n
Hermite: For x ∈ [−∞, ∞]
2 d n  −x 2 
Hn (x ) = (−1)n e x e
dx n
Most of this polynomials come from the soultion of ”important”
difference equations.
Orthogonal polynomials: Most common families

go to inneficient matlab script!

orthogonal families.m
Orthogonal polynomials: Recursive representation

Theorem
Let { ϕn (x )}n∞=0 be an orthogonal family of monic polynomials on [a, b ]
relative to an inner product h·, ·i ,
ϕ0 (x ) = 1, ϕ1 (x ) = x − h ϕ 0(x ), ϕ 0(x )i . Then { ϕn (x )}n∞=0 satisfies the
hx ϕ (x ), ϕ (x )i
0 0
recursive scheme

ϕn+1 (x ) = (x − δn ) ϕn (x ) − γn ϕn−1 (x )

where
hx ϕn , ϕn i h ϕn , ϕn i
δn = and γn =
h ϕn , ϕn i h ϕ n −1 , ϕ n −1 i

The above theorem is the so colled Gram-Schimdt process to find an

orthogonal family of monic polynomials.
Orthogonal polynomials: Recursive representation

A Chebyshev polynomial is given by

Ti (x ) = cos (i arccos x ) for i : 0, 1, 2, ..., n

and it is defined for x ∈ [−1, 1].

Chebyshev polynomials have the follwoing recursive
representation:
T0 (x ) = 1
T1 (x ) = x
Ti +1 (x ) = 2xTi (x ) − Ti −1 (x ) for i = 2, 3, ..., n
Orthogonal polynomials: Recursive representation

Recursive representation of Chebyshev polinomials: Ti (x ) can

be wrriten as
Ti (x ) = cos (i θ ) for i : 1, 2, ..., n
where θ = arccos x.
Using trigonometric identities we have
Ti +1 (x ) = cos ((i + 1) θ ) = cos (i θ ) cos (θ ) − sin (i θ ) sin (θ )
Ti −1 (x ) = cos ((i − 1) θ ) = cos (i θ ) cos (θ ) + sin (i θ ) sin (θ )
thus
Ti +1 (x ) = cos (i θ ) cos (θ ) − Ti −1 (x ) + cos (i θ ) cos (θ )
= 2 (cos (i θ ) cos (θ )) − Ti −1 (x )
Notice that cos (θ ) = cos (arccos x ) =⇒ cos (θ ) = x and that
cos (i θ ) = Ti (x ) . Therefore
Ti +1 (x ) = 2xTi (x ) − Ti −1 (x ) for i = 2, 3, ..., n
Orthogonal polynomials: Recursive representation

Recursive representation of Chebyshev polinomials (cont.):

The first five Chebyshev polynomials are
T0 (x ) = 1
T1 (x ) = x
T2 (x ) = 2xT1 (x ) − T0 (x ) = 2x 2 − 1
 
T3 (x ) = 2xT2 (x ) − T1 (x ) = 2x 2x 2 − 1 − x
= 4x 3 − 3x
   
T4 (x ) = 2xT3 (x ) − T2 (x ) = 2x 4x 3 − 3x − 2x 2 − 1
= 8x 4 − 8x 2 + 1
Chebyshev polynomials are polynomials with leading coefficient of
2n−1
Orthogonal polynomials: Recursive representation

Recursive representation of Chebyshev monic polinomials

(cont.):
The monic Chebyshev polynomial is defined as

en (x ) = cos (n arccos x )
T for n > 1
2n−1
Using the general recursive formula for orthogonal monic polynomials
we obtain the recursion shceme for the monic Chebyshev polynomial
e0 (x ) = 0
T
e1 (x ) = x
T

T e1 (x ) − 1 T
e2 (x ) = x T e0 (x )
2
and
en (x ) − 1 T
en+1 (x ) = x T
T en−1 (x ) for n ≥ 2
4
Orthogonal polynomials: Recursive representation

Going from the monic recursive representation to the original

Chebyshev representation:

2n e 2n−1 e 1 2n−2 e
Tn + 1 ( x ) = x T n ( x ) − Tn−1 (x ) for n ≥ 2
2n 2n − 1 4 2n−2
1 1 1 1
Tn+1 (x ) = x n−1 Tn (x ) − Tn−1 (x )
2n 2 4 2n−2

2n 1 2n
Tn+1 (x ) = x n−1 Tn (x ) − Tn−1 (x )
2 4 2n−2
1
= 2xTn (x ) − 22 Tn−1 (x )
4
= 2xTn (x ) − Tn−1 (x )
Orthogonal polynomials

The weigthing function for a Chebyshev polynomial must satisfy

the orthogonality condition

Z1 
0 for i 6= j
Ti (x ) Tj (x ) w (x ) dx =
Cj for i = j
−1

where Cj is a constant
Integrating by substitution and solving for w (x ) yields

1
w (x ) = √
1 − x2
Orthogonal polynomials

Orthogonality property of Chebyshev polynomials:


Z1  0 for i 6= j
Ti (x ) Tj (x ) w (x ) dx = π
for i = j 6= 0
 2
−1 π for i =j =0
Orthogonal polynomials: Chebyshev least squares

Approximate f ∈ C([−1, 1]) with Chebyshev polinomials.

Coefficients of the continuous least squares are given by
h f , Tk i
ck = ∀k = 0, 1, 2, ...n
hTk , Tk i
Z1
x ) Tk ( x )
f (√
dx
1−x 2
−1
=
Z1
Tk ( x ) 2
√ dx
1−x 2
−1

where Tk (x ) = cos (k arccos x ) or it is given by its recursive version.

Notice that the denominator has two cases: For k = 0 →
Z1 Z1
T0 ( x ) 2 Tk ( x ) 2
√ dx = π and for k = 1, 2, ..., n → √ dx = π
2.
1−x 2 1−x 2
−1 −1
Orthogonal polynomials: Chebyshev least squares

Therefore
Z1
1 f (x )
c0 = √ dx
π 1 − x2
−1
and
Z1
2 f (x ) Tk (x )
ck = √ dx ∀k = 1, 2, ...n
π 1 − x2
−1
The Chebyshev polynomial approximation is
n
Tn (x ) = c0 + ∑ ck Tk (x )
k =1
Orthogonal polynomials: Chebyshev least squares

Near minimax property of Chebyshev least squares:

Theorem
Let pn∗ (x ) be the minmax polynomial approximation to f ∈ C [−1, 1]. If
Tn∗ (x ) is the nth degree Chebyshev least square approximation to
f ∈ C [−1, 1] then
 
4
kf − pn k∞ ≤ kf − Tn k∞ ≤ 4 + 2 ln n kf − pn∗ k∞
∗ ∗
π

and
lim kf − Tn∗ k∞ = 0 uniformly
n→∞

Chebyshev least squares approximation is nearly the same as the

minmax polynomial approximation and that as n → ∞, then Tn∗
converges to f uniformly.
Orthogonal polynomials: Chebyshev least squares

Chebyshev least squares (cont.):EXAMPLE

Approximate f (x ) = exp (x ) for x ∈ [−1, 1]
First order polynomial approximation of exp (x ) is

exp (x ) ≈ c0 T0 (x ) + c1 T1 (x )

where
T0 (x ) = cos ((0) (arccos x )) = 1
T1 (x ) = cos (arccos x ) = x
and
Z1
1 exp (x )
c0 = √ dx = 1.2661
π 1 − x2
−1

Z1
2 exp (x ) x
c1 = √ dx = 1.1303
π 1 − x2
−1
Orthogonal polynomials: Chebyshev least squares

Chebyshev least squares (cont.):EXAMPLE

Second order polynomial approximation of exp (x ) is

exp (x ) ≈ c0 T0 (x ) + c1 T1 (x ) + c2 T2 (x )

where c0 , c1 , T0 and T1 are the same as before and

T2 (x ) = cos (2 arccos x )

andc
Z1
2 exp (x ) T2 (x )
c2 = √ dx = 0.2715
π 1 − x2
−1
Orthogonal polynomials: Chebyshev least squares

Chebyshev least squares (cont.):EXAMPLE

go to inneficient matlab script!

cheby least squares.m

Type CC for each marix of coefficients....
Interpolation: Basics

Usually we don’t have the value of f (x ) for all its domain.

We only have the value of f (x ) at some finite set of points:

(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))

Interplation nodes or points: x0 , x1 , ..., xn

Interpolation problem: Find the degree ≤ n polynomial pn (x )
that passes through these points:

f (xi ) = pn (xi ) ∀i : 0, ...n

Interpolation: Basics

Existence and uniqueness of the interpolating polynomial

Theorem
If x0 , ..., xn are distinct, then for any f (x0 ) , ..., f (xn ) there existis a
unique polynomial pn (xi ) of degree ≤ n such that the interpolation
conditions
f (xi ) = pn (xi ) ∀i : 0s, ...n
are satisfied.
Linear interpolation

The simplest case is linear interpolation (i.e., n = 1) with two

data points
(x0 , f (x0 )) , (x1 , f (x1 ))
The interpolation conditions are:

f (x0 ) = p1 (x0 )
= a0 + a1 x0

f (x1 ) = p1 (x1 )
= a0 + a1 x1
Linear interpolation

Solving the above system yields

f ( x1 ) − f ( x0 )
 
a0 = f (x0 ) − x0
x1 − x0

f ( x1 ) − f ( x0 )
a1 =
x1 − x0
Thus, the interpolating polynomial is

f (x1 ) − f (x0 ) f ( x1 ) − f ( x0 )
     
p1 (x ) = f (x0 ) − x0 + x
x1 − x0 x1 − x0
Linear interpolation

Notice that the interpolating polynomial can be written as

Power form
f (x1 ) − f (x0 ) f (x1 ) − f (x0 )
     
p1 (x ) = f (x0 ) − x0 + x
x1 − x0 x1 − x0

Newton form
f (x1 ) − f (x0 )
 
p1 (x ) = f (x0 ) + (x − x0 )
x1 − x0

Lagrange form

x − x1 x − x0
   
p1 (x ) = f (x0 ) + f (x1 )
x0 − x1 x1 − x0

We have the same interpolating polynomial p1 (x ) written in two

different forms.
Quadratic interpolation

If we assume n = 2 and three data points

(x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))

Quadratic interpolation

The interpolation conditions are

f (x0 ) = p2 (x0 ) = a0 + a1 x0 + a2 x02

f (x1 ) = p2 (x1 ) = a0 + a1 x1 + a2 x12

f (x2 ) = p2 (x2 ) = a0 + a1 x2 + a2 x22
Quadratic interpolation

In matrix form the interpolation conditions are

1 x0 x02
    
a0 f ( x0 )
 1 x1 x 2   a1  =  f (x1 ) 
1
1 x2 x22 a2 f ( x2 )

or im more compact form

Va = b
Notice that V is a Vandermonde matrix which is ill-conditioned.
The condition number of V is large so it is better to compute the a0 s
by using another form of writing the interpolating polynomial.
Quadratic interpolation

But we can still do it by hand since this is a 3by3 matrix!

Solving the above system yields
x1 x2 −x0 x2 x0 x1
 
   
a0 −x1 )(x0 −x2 )
 (x0 −( x1 +x2 )
(x0 −x1 )(x1 −x2 )
x0 +x2
(x0 −x2 )(x1 −x2 )
−(x0 +x1 )  f ( x0 )
 a1  =    f ( x1 ) 
 (x0 −x1 )(x0 −x2 ) (x0 −x1 )(x1 −x2 ) (x0 −x2 )(x1 −x2 ) 
a2 1 −1 1 f ( x2 )
(x0 −x1 )(x0 −x2 ) (x0 −x1 )(x1 −x2 ) (x0 −x2 )(x1 −x2 )
Quadratic interpolation

Or by using the Matlab symbolic toolbox

>> syms a b c
>> A = [1 a a^2; 1 b b^2; 1 c c^2];
>> inv(A)
ans =
[ (b*c)/((a - b)*(a - c)), -(a*c)/((a - b)*(b - c)),
(a*b)/((a - c)*(b - c))]
[ -(b + c)/((a - b)*(a - c)), (a + c)/((a - b)*(b - c)),
-(a + b)/((a - c)*(b - c))]
[ 1/((a - b)*(a - c)), -1/((a - b)*(b - c)), 1/((a - c)*(b
- c))]
Quadratic interpolation

Solving the above system yields the coefficients:

− x0 x2
   
x1 x2
a0 = f ( x0 ) + f ( x1 )
(x0 − x1 ) (x0 − x2 ) ( x0 − x1 ) ( x1 − x2 )
 
x0 x1
+ f (x2 )
( x0 − x2 ) ( x1 − x2 )

− ( x1 + x2 )
   
x0 + x2
a1 = f ( x0 ) + f ( x1 )
(x0 − x1 ) (x0 − x2 ) ( x0 − x1 ) ( x1 − x2 )
− ( x0 + x1 )
 
+ f (x2 )
( x0 − x2 ) ( x1 − x2 )

−1
   
1
a2 = f ( x0 ) + f ( x1 )
( x0 − x1 ) ( x0 − x2 ) (x0 − x1 ) (x1 − x2 )
 
1
+ f ( x2 )
(x0 − x2 ) (x1 − x2 )
Quadratic interpolation

The approximating second order polynomial in ”power” form is

p2 (x ) = a0 + a1 x + a2 x 2

where a0 , a1 and a2 are defined above.

Notice that p2 (x ) is a linear combination of n + 1 = 3 monomials
each of degree 0, 1, 2.
Quadratic interpolation

After ”some” algebra we can write p2 (x ) in different forms:

Lagrange form

( x − x1 ) ( x − x2 ) ( x − x0 ) ( x − x2 )
   
p2 (x ) = f (x0 ) + f ( x1 )
( x0 − x1 ) ( x0 − x2 ) ( x1 − x0 ) ( x1 − x2 )
(x − x0 ) (x − x1 )
 
+ f ( x2 )
( x2 − x0 ) ( x2 − x1 )

The above is a linear combination of n + 1 = 3 polynomials of

degree n = 2. The coefficients are the interpolated values
f (x0 ) , f (x1 ) and f (x2 ) .
Quadratic interpolation

By doing ”some” algebra we can write p2 (x ) in different forms:

Newton form

f ( x1 ) − f ( x0 )
 
p 2 ( x ) = f ( x0 ) + ( x − x0 )
x1 − x0

f (x2 )−f (x1 )
 
f (x1 )−f (x0 )

(x2 −x1 )
− (x1 −x0 )
+  ( x − x0 ) ( x − x1 )
( x2 − x0 )

The above is a linear combination of n + 1 = 3 polynomials each of

degree 0, 1, 2. The coefficients are the called divided differences.
Interpolation: The general case

The interpolation conditions when we have n + 1 data points:

{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))}

f (xi ) = pn (xi ) ∀i : 0, ...n

pn (xi ) written in ”power” form is

n
pn (xi ) = ∑ aj x j
j =0
Interpolation: The general case

The interpolation conditions can be written as

n
∑ aj xi
j
f ( xi ) = ∀i : 0, ...n
j =0

or
f (x0 ) = a0 + a1 x0 + ... + an x0n

f (x1 ) = a0 + a1 x1 + ... + an x1n

...

f (xn ) = a0 + a1 xn + ... + an xnn

Interpolation: The general case

In matrix form
... x0n
    
1 x0 a0 f ( x0 )
 1 x1
 ... x1n 

 a1   f ( x1 ) 
= 
 ... ... ... ...   ...   ... 
1 xn ... xnn an f (xn )

The matrix to be inverted is a Vandermonde matrix: Also an

ill-conditioned matrix.
Interpolation: The general case

We can also generalize the lagrange form of the interpolating

polynomial:

pn (x ) = f (x0 ) ln,0 (x ) + f (x1 ) ln,1 (x ) + ... + f (xn ) ln,n (x )

 n
where ln,j (x ) j =0 are a family of n + 1 polynomials of degree n
given by



(x − x0 ) ... x − xj −1 x − xj +1 ... (x − xn )
ln,j (x ) =



∀0 ≤ j ≤ n
xj − x0 ... xj − xj −1 xj − xj +1 ... xj − xn

More compactly
n
∑f


pn ( x ) = xj ln,j (x )
j =0
Interpolation: The general case

For j = 0
n x − xj
(x − x1 ) ... (x − xn )
ln,0 (x ) =
(x0 − x1 ) ... (x0 − xn )
= ∏ x0 − xj
j =0
j 6 =0

For j = 1
n x − xj
(x − x0 ) (x − x2 ) ... (x − xn )
ln,1 (x ) =
(x1 − x0 ) (x1 − x2 ) ... (x1 − xn )
= ∏ x1 − xj
j =0
j 6 =1

For j = n
n x − xj
(x − x0 ) (x − x2 ) ... (x − xn−1 )
ln,n (x ) =
(xn − x0 ) (xn − x2 ) ... (xn − xn−1 )
= ∏ x2 − xj
j =0
j 6 =2
Interpolation: The general case

For any ∀0 ≤ j ≤ n
n x − xj
ln,j (x ) = ∏ xi − xj
j =0
j 6 =i

The lagrange form of the interpolating polynomial is

n
∑f


pn ( x ) = xj ln,j (x )
j =0

with ln,j (x ) defined above.

It turns out that computing the lagrange polynomial is more efficient
than solving the vandermonde matrix!
Interpolation: The general case

We can also generalize the newton form of the interpolating

polynomial

pn (x ) = c0 + c1 (x − x0 ) + c2 (x − x0 ) (x − x1 ) ... + cn (x − x0 ) (x − x1 ) ...

where the coefficients c0 , c1 , ..., cn are called the divided difference

and are denoted by
c 0 = d ( x0 )
c 1 = d ( x1 , x0 )
c 2 = d ( x2 , x1 , x0 )
...
cn = d (xn , ..., x1 , x0 )
Interpolation: The general case

The divided differences are defined as

d ( x0 ) = f ( x0 )

f ( x1 ) − f ( x0 )
d ( x1 , x0 ) =
x1 − x0

d (x2 , x1 ) − d (x1 , x0 )
d ( x2 , x1 , x0 ) =
x2 − x0
   
f (x2 )−f (x1 ) f (x1 )−f (x0 )
x2 −x1 − x1 −x2
=
x2 − x0
Interpolation: The general case

The divided differences are defined as (Cont.)

d (x3 , x2 , x1 ) − d (x2 , x1 , x0 )
d ( x3 , x2 , x1 , x0 ) =
x3 − x0
   !    !
f (x )−f (x )
3 2 f (x )−f (x )
2 1 f (x2 )−f (x1 ) f (x1 )−f (x0 )
x3 −x2 − x2 −x1 x2 −x1 − x −x
x3 −x2 − x2 −x0
1 2

=
x3 − x0

d (xn , ..., x1 , x0 ) − d (xn−1 , ..., x1 , x0 )

d (xn , ..., x1 , x0 ) =
xn − x0
Interpolation: The general case

The generalization of the Newton form of the interpolating

polynomial is

pn (x ) = d (x0 ) + d (x1 , x0 ) (x − x0 ) + d (x2 , x1 , x0 ) (x − x0 ) (x − x1 ) + ...

+ d (xn , ..., x1 , x0 ) (x − x0 ) (x − x1 ) ... (x − xn−1 )

or
n
j −1
p n ( x ) = d ( x0 ) + ∑d xj , ..., x1 , x0 ∏ ( x − xk )
j =1 k =0
Interpolation: The interpolation error

Theorem
Let f (x ) ∈ Cn+1 [a, b ]. Let pn (x ) a polynomial of degree ≤ n such that
it interpolates f (x ) at the n + 1 distinct nodes {x0 , x1 , ..., xn } . Then
∀x ∈ [a, b ], there exists a ξ n ∈ (a, b ) such that
n
1
f (x ) − pn (x ) = f ( n + 1 ) ( ξ n ) ∏ ( x − xk )
(n + 1) ! k =0

Fact
The error term for the nth Taylor approximation around the point x0 is

f (n +1) ( ξ n )
( x − x0 ) n + 1
(n + 1) !
Interpolation: The interpolation error

Notice that applying the supremum norm to the interpolation error

yields
n
1
kf (x ) − pn (x )k∞ ≤
(n + 1) !
f (n +1) ( ξ n )
∞
∏ ( x − xk )
k =0 ∞

or

max |f (x ) − pn (x )|
x ∈[a,b ]
!
  n
1
≤
(n + 1) !
max
ξ n ∈[a,b ]
f (n +1)
(ξ n ) max ∏ ( x − xk )
x ∈[a,b ] k =0

The R.H.S is an upper bound for the interpolation error.

Interpolation: The interpolation error

We would like to have

( !)
  n
1
lim max f (n +1)
(ξ n ) max ∏ (x − xk ) =0
n→∞ (n + 1)! ξ n ∈[a,b ] x ∈[a,b ] k =0

thus
lim (f (x ) − pn (x )) = 0
n→∞
But nothing guarantees convergence (neither point or uniform
convergence).
Interpolation: The interpolation error

The maximun error depends on the interpolation nodes !

n
{x0 , x1 , ..., xn } through the term maxx ∈[a,b ] ∏ (x − xk ) .
k =0
Notice that no other term depends on the interpolating nodes once
we look for the maximum error.
We can choose the nodes in order to minimize the interpolation
error: ( )
n
min
{x0 ,...,xn }
max ∏ (x − xk )
x ∈[a,b ] k =0
Interpolation: Choosing the nodes

What happens if we work with uniformly spaced nodes?, that is with

nodes such that
i −1
 
xi = a + (b − a) for i : 1, .., n
n−1

Recall that:
We want to interpolate a function f (x ) : [a, b ] → R
n is the degree of the interpolating polynomial: pn (x )
The interpolation conditions are

f (xi ) = pn (xi ) ...for i : 0, .., n

so, if n = 10, we need n + 1 = 11 data points.

Interpolation: Choosing the nodes

Runge’s example: Let f (x ) = 1+1x 2 defined on the interval [−5, 5].

Find the lagrange polnomial approximation for n = 11.

go to inneficient matlab script!

runge example lagrange interpolation UN.m

Play increasing the degree of interpolation and see that there is no
convergence.
The graph replicates figure 6.6 of Judd’s book.
Interpolation: Choosing the nodes

But we can choose the nodes in order to obtain the smallest value for
n n
∏ ( x − xk ) = max ∏ (x − xk )
x ∈[a,b ] k =0
k =0 ∞

Chebyshev polynomials one more time: Recall the monic

Chebyshev polynomial is

ej (x ) = cos (j arccos x )
T
2j −1
with x ∈ [−1, 1] and j = 1, 2, ..., n
en (x ) are given by the soltution to
Then, the zeros of T
en (x ) = 0
T
cos (n arccos x ) = 0
cos (nθ ) = 0
where θ = arccos x. Thus θ ∈ [0, π ].
Interpolation: Choosing the nodes

Zeros occur when cos (nθ ) = 0, that is

2k − 1
 
nθ = π for k = 1, 2, ..n
2

Notice that
2k − 1
 
cos π
2
 π 
= cos kπ −
2 
π π
= cos (kπ ) cos − sin (kπ ) sin
| {z2 } | {z } 2
=0
=0
=0
Interpolation: Choosing the nodes

en (x ) = 0 has n different roots given by

The equation T
 
1
nθ = k − π for k = 1, 2, ..n
2

This means that

For k = 1
π
θ1 =
2n
For k = 2  
3 π
θ2 =
2 n
For k = n
2n − 1
 
π
θn =
2 n
Interpolation: Choosing the nodes

Roots for cos (nθ ) where θ ∈ [0, π ] :

n=1 n=2 n=3 ... n

π π π π
k=1 2 4 6 2n
k=2 3π 3π 3
4 6 2n π
k=3 5π 5
6 2n π
...  
k=n 2n−1 π
2n
Interpolation: Choosing the nodes

Plooting cos (j θ ) for θ ∈ [0, π ] and j = 1, 2, ..., n

go to inneficient matlab script!

cheby nodes.m
Interpolation: Choosing the nodes

We want the roots of the monic chebyshev and we have the roots of
the cosine function:
 
1
nθ = k − π for k = 1, 2, ..n
2

but θ = arccos x, thus

2k − 1
 
arccos x = π for k = 1, 2, ..n
2n

then the roots of the chebyshev polynomials are

2k − 1
  
xk = cos π for k = 1, 2, ..n
2n

en (x ) are the same as the roots of Tn (x ) .

Notice that the roots of T
Interpolation: Choosing the nodes

Ploting chebyshev nodes

go to inneficient matlab script!

cheby nodes.m
Interpolation: Choosing the nodes

The following theorem summarizes some characteristics of chebyshev

polynomials

Theorem
The Chebyshev polynomial Tn (x ) of degree n ≥ 1 has n zeros in [−1, 1]
at
2k − 1
  
xk = cos π for k = 1, 2, ..n
2n
Moreover, Tn (x ) assumes its extremum at
 
kπ
xk∗ = cos for k = 0, 1, .., n
n

with
Tn (xk∗ ) = (−1)k for k = 0, 1, .., n
Interpolation: Choosing the nodes

Corollary
The monic Chebyshev polynomial T en (x ) has the same zeros and
extremum points as Tn (x ) but with extremum values given by
k
en (x ∗ ) = (−1) for k = 0, 1, ..., n
T k
2n − 1
Interpolation: Choosing the nodes

Extrema of Chebyshev polynomials:

Tn (x ) = cos (n arccos x )

then
dTn (x )
= Tn0 (x )
dx  
n
= − sin (n arccos x ) − √
1 − x2
n sin (n arccos x )
= √
1 − x2

Notice that Tn0 (x ) is a polynomial of degree n − 1 with zeros given

by
Tn0 (x ) = 0
Interpolation: Choosing the nodes

Excluding the endpoints of the domain, x = −1 or x = 1, then,

extremum points occurs when
sin (n arccos x ) = 0
or when
sin (nθ ) = 0
for θ ∈ (0, π ). Thus
nθk = kπ for all k = 1, 2, ..., n − 1
Solving for x yields
kπ
θ = arccos x =
n 
kπ
=⇒ xk∗ = cos for k = 1, 2, ..., n − 1
n
Obviusly, extrema also occur at the endpoints of the domain (i.e,
x = −1 or x = 1), that is when k = 0 or when k = n.
Interpolation: Choosing the nodes

The extremum values of Tn (x ) occurs when

Tn (x ∗ ) = cos (n arccos x ∗ )
   
kπ
= cos n arccos cos
n
 
kπ
= cos n
n
= cos (kπ )
= (−1)k for k = 0, 1, ..., n

Notice that we are including the endpoints of the domain.

The above result implies

max |Tn (x )| = 1
x ∈[−1,1]
Interpolation: Choosing the nodes

Extrema for monic Chebyshev polynomials are characterized by the

same points since
Ten (x ) = Tn (x )
2n−1
thus
Ten0 (x ∗ ) = Tn0 (x ∗ ) = 0 for k = 0, 1, ..., n
k k
en (x ) are given by
But the extremum values of T

Tn xk∗


∗
en (x ) =
T k for k = 0, 1, ..., n
2n−1
(−1)k
= n−1 for k = 0, 1, ..., n
2
Therefore
max en (x ) = 1
T
x ∈[−1,1] 2n−1
Interpolation: Choosing the nodes

An important property of monic Chebyshev polynomials is given by

the following theorem

Theorem
en (x ) is a monic polynomial of degree n defined on [−1, 1], then
If p

max en (x ) = 1 ≤ max |p
T en (x )|
x ∈[−1,1] 2n − 1 x ∈[−1,1]

for all monic polynomials of degree n.

Interpolation: Choosing the nodes

Recall that we want to choose the interpolation nodes {x0 , ..., xn } in

order to solve
( )
n
min
{x0 ,...,xn }
max ∏ (x − xk )
x ∈[a,b ] k =0

Choosing the interpolation nodes is the same as choosing the zeros

n
of ∏ ( x − xk ) .
k =0
n
Notice that ∏ (x − xk ) is a monic polynomial of degree n + 1.
k =0
Therefore it must be the case that
n
max en+1 (x ) = 1 ≤ max ∏ (x − xk )
T
x ∈[−1,1] 2n x ∈[−1,1] k =0
Interpolation: Choosing the nodes

n
The smallest value that maxx ∈[−1,1] ∏ ( x − xk ) can take is 21n .
k =0
Therefore
n
1
max ∏ (x − xk )
x ∈[−1,1] k =0
=
2n

= max en+1 (x )
T
x ∈[−1,1]
which implies that
n
∏ (x − xk ) = Ten+1 (x )
k =0
n
Therefore the zeros of ∏ (x − xk ) must be the zeros of Ten+1 (x )
k =0
which are given by
  
2k + 1
xk = cos π for k = 1, 2, ..n + 1
2 (n + 1)
Interpolation: Choosing the nodes

n
Since maxx ∈[−1,1] ∏ ( x − xk ) = 1
2n then the maximum
k =0
interpolation error becomes
  n
1
max |f (x ) − pn (x )| ≤
x ∈[a,b ] (n + 1) !
max
ξ n ∈[a,b ]
f (n +1) ( ξ n ) max ∏ (x
x ∈[a,b ] k =0

  
1 1
max |f (x ) − pn (x )| ≤ max f (n +1) ( ξ n )
x ∈[a,b ] (n + 1) ! ξ n ∈[a,b ] 2n
Chebyshev nodes eliminate violent oscillations for the error term
compared to uniform spaced nodes.
Interpolation with Chebyshev nodes has better convergence
properties.
It is possible to show that pn (x ) → f (x ) as n → ∞ uniformly. This
is not guaranteed under uniform spaced nodes.
Interpolation: Choosing the nodes

Runge’s example with chebyshev nodes

runge example cheby nodes.m

Interpolation: Choosing the nodes

Figure 6.2 of Miranda and Flecker book:

Comparing the interpolation errors of f (x ) = exp(−x ) defined in
x ∈ [−5, 5] with 10-node polynomial approximation

example miranda cheby nodes

Interpolation through regresssion

The interpolation conditions require to have the same number of

data points (interpolation data) and unknown coefficients in order to
proceed.
But we can also have the case, where the data points exceed the
number of unknow coefficients.
For this case, we can use discrete least squares: Use m
interplation points to find n < m coefficients.
The omitted terms are high degree polynomials that may produce
undesirable oscillations.
The result is a smoother function that approximates the data.
Interpolation through regresssion

Objective: Construct a degree n polynomial , b f (x ) , that

approximates the function f for x ∈ [a, b ] using m > n interpolation
nodes.
n
f (x ) = ∑ cj Tj (xk )
b
j =0
Interpolation through regresssion

Algorithm:
Step 1: Compute the m Chebyshev interpolation nodes on
[−1, 1]:

2k − 1
  
zk = cos π for k = 1, ..., m
2m

As if we want an m −degree Chebyshev interpolation.

Step 2: Adjust the nodes to the interval [a, b ] :

b−a
 
xk = ( zk + 1 ) + a for k = 1, ..., m
2

Step 3: Evaluate f at the nodes:

yk = f (xk ) ...for k = 1, ..., m

Interpolation through regresssion

Algorithm (Cont.):
Step 4: Compute the Chebyshev least squares coefficients
The coefficients that solve the discrete LS problem
" #2
m n
min ∑ yk − ∑ cj Tj (zk )
k =1 j =0

are given by
m
∑ yk Tj (zk )
k =1
cj = m for j = 0, 1, ..., n
∑

2
Tj (zk )
k =1

where zk is the inverse transformation of xk :

2xk − (a + b )
zk =
b−a
Interpolation through regresssion

Finally, the LS Chebyshev approximating polynomial is given by

n
f (x ) =
b ∑ cj Tj (z )
j =0

where z ∈ [−1, 1] and it is given by

2x − (a + b )
z=
b−a
and the cj are estimated using the LS coefficients
m
∑ yk Tj (zk )
cj = km=1 for j = 0, 1, ..., n
∑

2
Tj (zk )
k =1
Piecewise linear approximation

If we have interpolation data given by

{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))}
We can divide the interpolation nodes in subintervals of the form

[xi , xi +1 ] for i = 0, 1, ..., n − 1

Then we can perform linear interpolation in each subinterval:

Interpolation conditions for each subinterval:

f (xi ) = a0 + a1 xi
f (xi +1 ) = a0 + a1 xi +1
Piecewise linear approximation

Linear interpolation in each subinterval yields [xi , xi +1 ]:

The interpolating coefficients:

f (xi +1 ) − f (xi )
 
a0 = f (xi ) − xi
xi +1 − xi

f (xi +1 ) − f (xi )
a1 =
xi +1 − xi
Piecewise linear interpoland:

x − xi
 
pi (x ) = f (xi ) + (f (xi +1 ) − f (xi ))
xi +1 − xi
Piecewise linear approximationPiecewise polynomial
approximation: Splines

Example: (x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))

Then we have two subintervals

[x0 , x1 ] and .[x1 , x2 ]

The interpolating function is given by:

 
f (x ) + x −x0 (f (x ) − f (x )) for x ∈ [x0 , x1 ]
0 x −x 1 0
f (x ) =
b  1 0
f (x1 ) + xx2−−xx11 (f (x2 ) − f (x1 )) for x ∈ [x1 , x2 ]
Piecewise polynomial approximation: Splines

A spline is any smooth function that is piecewise polynomial but also

smooth where the polynomial pieces connect.
Assume that the interpolation data is given by
{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xm , f (xm ))} .
A function s (x ) defined on [a, b ] is a spline of order n if:
s is C n−2 on [a, b ]
s (x ) is a polynomial of degree n − 1 on each subinterval [xi , xi +1 ]
for i = 0, 1, .., m − 1
Notice that an order 2-spline is the piecewise linear interpolant
equation.
Piecewise polynomial approximation: Splines

A cubic spline is a spline of order 4:

s is C 2 on [a, b ]
s (x ) is a polynomial of degree n − 1 = 3 on each subinterval
[xi , xi +1 ] for i = 0, 1, .., m − 1

s (x ) = ai + bi x + ci x 2 + di x 3 for x ∈ [xi , xi +1 ], i = 0, 1, .., m − 1

Piecewise polynomial approximation: Splines

Example of cubic spline: Assume that we have the following 3 data

points: (x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))
There are two subintervals: [x0 , x1 ] and .[x1 , x2 ].
A cubic spline is a function s such that
s is C 2 on [a, b ]
s (x ) is a polynomial of degree 3 on each subinterval:

s0 (x ) = a0 + b0 x + c0 x 2 + d0 x 3 for x ∈ [x0 , x1 ]

s (x ) =
s1 (x ) = a1 + b1 x + c1 x 2 + d1 x 3 for x ∈ [x1 , x2 ]

Notice that in this case we have 8 unknowns:

a0 , a1 , b0 , b1 , c0 , c1 , d0 , d1
Piecewise polynomial approximation: Splines

Example (Cont.): We need 8 conditions

Interpolation and continuity at interior nodes conditions

y0 = s0 (x0 ) = a0 + b0 x0 + c0 x02 + d0 x03

y1 = s0 (x1 ) = a0 + b0 x1 + c0 x12 + d0 x13
y1 = s1 (x1 ) = a1 + b1 x1 + c1 x12 + d1 x13
y2 = s1 (x2 ) = a1 + b1 x2 + c1 x22 + d1 x23
Piecewise polynomial approximation: Splines

Example (Cont.): We need 8 conditions

First and second derivatives must agree at the interior nodes

s00 (x1 ) = s10 (x1 )

b0 + 2c0 x1 + 3d0 x12 = b1 + 2c1 x1 + 3d1 x12

00 00
s0 (x1 ) = s1 (x1 )
2c0 + 6d0 x1 = 2c1 + 6d1 x1
Piecewise polynomial approximation: Splines

Up to know we have 6 conditions, we need two more conditions

3 ways to obtain the additional conditions:
Natural spline: s 0 (x0 ) = s 0 (x2 ) = 0
Hermite spline: If we have information on the slope of the original
function at the end points:

f 0 (x0 ) = s 0 (x0 )

f 0 (x2 ) = s 0 (x2 )
Secant Hermite spline: use the secant to estimate the slope at the
end points
s (x1 ) − s (x0 )
s 0 (x0 ) =
x1 − x0
0 s (x2 ) − s (x1 )
s (x2 ) =
x2 − x1
Piecewise polynomial approximation: Splines

Generalization of cubic splines:

See Judd’s book!!!