Numerical Analysis: MTL851: Dr. Mani Mehra

Numerical Analysis : MTL851
Dr. Mani Mehra
Department of Mathematics
Indian Institute of Technology Delhi
March 20, 2023
Dr. Mani Mehra (IIT Delhi) Numerical Analysis : MTL851 March 20, 2023 1 / 53
Interpolation
Interpolation methods are the basis for many other procedures that you will
study in this course, such as numerical integration and differentiation.
They are behind the ways that we use to solve ordinary and partial
differential equations.
They demonstrate important theory about polynomials and the accuracy of
numerical methods.
They are one of the more important ways that curves are drawn on your
computer screen.
There is a rich history behind interpolation. It really began with the early
studies of astronomy when the motion of heavenly bodies was determined
from periodic observations.
An application of interpolation that you see every day is in weather
forecasting. When you watch the weather forecasts on television, you may
wonder where these (usually) correct projections come from.
Interpolating Polynomials
Interpolating Polynomials
We will be most interested in techniques adapted to situations where the

data are far from linear. The basic principle is to fit a polynomial curve to
the data. The reason for using polynomials has already been stated-they are
nice functions and their evaluation requires only those arithmetic operations
that computers can do.
Suppose that we have the following data pairs-x-values and f (x)-values
where f (x) is some unknown function:
x 3.2 2.7 1.0 4.8 5.6

f (x) 22.0 17.8 14.2 38.3 51.7
First, we need to select the points that determine our polynomial. The
maximum degree of the polynomial is always one less than the number of
points. Suppose we choose the first four points. If the cubic is
ax 3 + bx 2 + cx + d, we can write four equations involving the unknown
coefficients a, b, c, and d.
if x = 3.2 : a(3.2)3 + b(3.2)2 + c(3.2) + d = 22.0
if x = 2.7 : a(2.7)3 + b(2.7)2 + c(2.7) + d = 17.8
if x = 1.0 : a(1.0)3 + b(1.0)2 + c(1.0) + d = 14.2
if x = 4.8 : a(4.8)3 + b(4.8)2 + c(4.8) + d = 38.3
After solving the above system we get

a = −0.5275, b = 6.4952, c = −16.1177, d = 24.3499 and our polynomial is
−0.5275x 3 + 6.4952x 2 − 16.1177x + 24.3499.
We seek a better and simpler way of finding such interpolating polynomials.
This procedure is awkward, especially if we want a new polynomial that is
also made to fit at the point (5.6, 51.7), or if we want to see what difference
it would make to use a quadratic instead of a cubic. Furthermore, this
technique leads to an ill-conditioned system of equations. For this example,
the condition number is about 2700.
Lagrangian Polynomials
We will first look at one very straightforward approach-the Lagrangian
polynomial. The Lagrangian polynomial is perhaps the simplest way to
exhibit the existence of a polynomial for interpolation with unevenly spaced
data. Data where the x-values are non equispaced often occur as the result
of experimental observations or when historical data are examined.
We want to construct a polynomial Pn (x) of degree n that passes through
(n + 1) points. Consider the polynomial
(x − x0 )(x − x1 ) · · · (x − xk−1 )(x − xk+1 ) · · · (x − xn )
Pn,k (x) = .
(xk − x0 )(xk − x1 ) · · · (xk − xk−1 )(xk − xk+1 ) · · · (xk − xn )
We observe from above, Pn,k (x) is a polynomial of degree n,
Pn,k (xi ) = 0, i 6= k and Pn,k (xk ) = 1. So based on these properties:
n
X
Pn (x) = yk Pn,k (x),
k=0
from the above we observe that Pn (xi ) = yi . The function Pn,k (x) is called
the Lagrange fundamental polynomials and Pn (x) is the Lagrange
Dr. polynomial.
Mani Mehra (IIT Delhi) Numerical Analysis : MTL851 March 20, 2023 6 / 53
Example
Consider the following data:
x 3.2 2.7 1.0 4.8

f (x) 22.0 17.8 14.2 38.3
Using Lagrange interpolation polynomial to find the interpolated value for

x = 3.0.
Solution :
Any polynomial of degree n passes through (n + 1) points is unique.
To prove above, we assume that there is another polynomial Qn (x) and
consider the polynomial
T (x) = Pn (x) − Qn (x).
Since Pn (x) and Qn (x) are both polynomials of degree ≤ n, then T (x) is
also a polynomial of degree ≤ n satisfying the conditions
T (xi ) = Pn (xi ) − Qn (xi ) = 0, i = 0, 1, · · · , n.
Therefore T (x) is a polynomial of degree ≤ n which has (n + 1) distinct

roots x0 , · · · , x1 . This implies that T (x) ≡ 0, because a polynomial T (x) of
degree n has exactly n roots. Therefore, Pn (x) ≡ Qn (x).
Error of Interpolation
When we fit a polynomial Pn (x) to some data points, it will pass exactly
through those points, but between those points Pn (x) will not be precisely
the same as the function f (x) that generated the points (unless the function
is that polynomial). How much is Pn (x) different from f (x)? How large is
the error of Pn (x)?
We can write error function as
En (x) = f (x) − Pn (x) = (x − x0 )(x − x1 ) · · · (x − xn )g (x).
Obviously, f (x) − Pn (x) − E (x) = 0, So

f (x) − Pn (x) − (x − x0 )(x − x1 ) · · · (x − xn )g (x) = 0.
To determine g (x), we now use the interesting mathematical device of

constructing an auxiliary function. We call this auxiliary function W (t), and
define it as
W (t) = f (t) − Pn (t) − (t − x0 )(t − x1 ) · · · (t − xn )g (x).
Note in particular that x has not been replace by t in the g (x) portion. W
is really a function of t and x but we are only interested in variations of t.
We now examine the zeros of W (t). Certainly at t = x0 , x1 , · · · , xn the W
function is zero, but it also zero if t = x there are then a total of n + 2
values of t that makes W (t) = 0.
We now impose the necessary requirements on W(t) for the law of mean
value to hold. W(t) must be continuous and differentiable. If this is so,
there is a zero to its derivative W 0 (t) between each of the n + 2 zeros of
W (t), a total number of n + 1 zeros. Similarly W n+1 (t), which must have
at least one zero in the interval that has x0 , xn or x as endpoints. Call this
value of t = ξ. We then have,
d n+1
W n+1 (ξ) = 0 = [f (t) − Pn (t) − (t − x0 )(t − x1 ) · · · (t − xn )g (x)]|t=ξ
dt n+1
= f n+1 (ξ) − (n + 1)!g (x),
So,
f n+1 (ξ)
g (x) = .
(n + 1)!
We now have our error term
f n+1 (ξ)
En (x) = (x − x0 )(x − x1 ) · · · (x − xn ) ,
(n + 1)!
with ξ on the smallest interval that contains {x, x0 , x1 , · · · , xn }.
Example
Using sin(0.1) = 0.09983 and sin(0.2) = 0.19867, find an approximate value of
sin(0.15) by Lagrange interpolation. Obtain abound on the truncation error.
Divided Differences
There are two disadvantages to using the Lagrangian polynomial for
interpolation. First, it involves more arithmetic operations than does the
divided-difference method we now discuss. Second, and more importantly, if
we desire to add or subtract a point from the set used to construct the
polynomial, we essentially have to start over in the computations.
Consider the nth-degree polynomial written in a special way:
Pn (x) = a0 +(x −x0 )a1 +(x −x0 )(x −x1 )a2 +· · ·+(x −x0 )(x −x1 ) · · · (x −xn )an .
If we chose the ai so that Pn (x) = f (x) at the (n + 1) known points,
(xi , fi ), i = 0, · · · , n, then Pn (x) is an interpolating polynomial. Where ai is
readily determined by using what are called the divided differences of the
tabulated values.
A special standard notation for divided differences is
f1 − f0
f [x0 , x1 ] = ,
x1 − x0
called the first divided difference between x0 and x1 .
In general,
ft − fs
f [xs , xt ] = ,
xt − xs
is the first divided difference between xs and xt . Observe that the order of
the points is immaterial:
ft − fs fs − ft
f [xs , xt ] = = = f [xt , xs ].
xt − xs xs − xt
Higher-order differences are defined in terms of lower-order differences.
f [x1 , · · · , xn ] − f [x0 , · · · , xn−1 ]

f [x0 , x1 , · · · , xn ] = .
xn − x0
For finding ai , set Pn (xi ) = fi , then we get ai = f [x0 , · · · , xi ]. We then can

write:
Pn (x) = f [x0 ] + (x − x0 )f [x0 , x1 ] + (x − x0 )(x − x1 )f [x0 , x1 , x2 ] + · · ·
+ (x − x0 ) · · · (x − xn−1 )f [x0 , · · · , xn ].
Example
Consider the following data:
x 3.2 2.7 1.0 4.8 5.6

f (x) 22.0 17.8 14.2 38.3 51.7
Write the interpolating polynomial of degree-3 that fits the data of above Table
at all points from x0 = 3.2 to x3 = 4.8.
Solution:
Spline
To obtain reasonably accurate results using interpolation, we may have to

use polynomials of high degrees. With polynomials of high degrees, not only
the computation becomes costly, but also computed results become
unreliable because of round-off errors. In order to keep the degree of
interpolating polynomial small and also to achieve accurate results, we use
piece-wise interpolation. We subdivide the given interval into a number of
sub-intervals [xi−1 , xi ], i = 1, 2, · · · , n and approximate the function by some
lower degree polynomial in each sub-interval.
A spline fits a set of nth-degree polynomials, gi (x), between each pair of
points, from xi−1 to xi . The points at which the splines join are called knots.
If the polynomials are all of degree-1, we have a linear spline and the curve
would appear as in the accompanying figure.
As you can observed from figure, the slopes are discontinuous where the
segments join. Splines of degree greater than 1 do not have this problem.
Most often cubic splines are used.
The Equation for a Cubic Spline :We will create a succession of cubic
splines over successive intervals of the data. (There is no requirement that
the points be evenly spaced). Each spline must join with its neighboring
cubic polynomials at the knots where they join with the same slope and
curvature. (The end splines have only one neighbor, so their slope and
curvature is not so constrained.)
We write the equation for a cubic polynomial, gi (x), in the ith interval,
between points (xi , yi ), (xi+l , yi+1 ). It looks like the solid curve shown here.
The dashed curves are other cubic spline polynomials. It has this equation:
gi (x) = ai (x − xi )3 + bi (x − xi )2 + ci (x − xi ) + di
Thus, the cubic spline function we want is of the from g (x) = gi (x) on the
interval [xi , xi+1 ] for i = 0, 1, · · · , n − 1. and meet these conditions:
a) gi (xi ) = yi , i = 0, · · · , n − 1 and gn−1 (xn ) = yn .
b) gi (xi+1 ) = gi+1 (xi+1 ), i = 0, 1, · · · , n − 2
c) gi0 (xi+1 ) = gi+1
0
(xi+1 ), i = 0, 1, · · · , n − 2
00 00
d) gi (xi+1 ) = gi+1 (xi+1 ), i = 0, 1, · · · , n − 2
If there n + 1 points, the number of intervals and the number of gi (x) are n.
Thus there are four times n unknowns, which are the [ai , bi , ci , di ] for
i = 0, 1, · · · , n − 1. Condition (a) immediately gives
di = yi , i = 0, 1, · · · , n − 1.
Condition (b) then gives
yi+1 = gi+1 (xi+1 ) = gi (xi+1 )
= ai (xi+1 − xi )3 + bi (xi+1 − xi )2 + ci (xi+1 − xi ) + yi
= ai hi3 + bi hi2 + ci hi + yi ,
where hi = xi+1 − xi . To relate the slopes and curvatures of the joining,
gi0 (x) = 3ai (x − xi )2 + 2bi (x − xi ) + ci ,
gi00 (x) = 6ai (x − xi ) + 2bi , i = 0, 1, · · · , n − 1.
Let Si = gi00 (xi ) and Sn = gn−1
00
(xn ). We have
Si = 6ai (xi − xi ) + 2bi = 2bi , Si+1 = 6ai (xi+1 − xi ) + 2bi = 6ai hi + 2bi ,
Hence we can write
Si Si+1 − Si
bi = , ai =
2 6hi
We using the value of ai , bi , di and solve for ci we get
yi+1 − yi 2hi Si + hi Si+1
ci = − .
hi 6
We now invoke the condition that the slopes of the two cubic that join at
(xi , yi ) are the same. For the equation in the ith interval, condition (c)
becomes, with x = xi ,
yi0 = 3ai (xi − xi )2 + 2bi (xi − xi ) + ci = ci
In the previous interval, from xi−1 to xi , the slope at its right end will be
yi0 = 3ai−1 (xi −xi−1 )2 +2bi−1 (xi −xi−1 )+ci−1 = 3ai−1 hi−1
2
+2bi−1 hi−1 +ci−1 .
Equating these, and substituting for ai , bi , ci , di their relationship in terms of
S and y we get
yi+1 − yi 2hi Si + hi Si+1
yi0 = −
hi 6
S − S S yi − yi−1 2hi−1 Si−1 + hi−1 Si
i i−1 2 i−1
=3 hi−1 +2 hi−1 + −
6hi−1 2 hi−1 6
Simplifying this equation, we get
y − y yi − yi−1
i+1 i
hi−1 Si−1 + (2hi−1 + 2hi )Si + hi Si+1 = 6 −
hi hi−1
= 6(f [xi , xi+1 ] − f [xi−1 , xi+1 ]).
Take S0 = 0 and Sn = 0. This makes the end cubics approach linearity at

their extremities. This condition, called a natural spline, matches precisely
to the drafting device. This technique is used very frequently.
Another often used condition is to force the slopes at each end to assume
specified values. When that information is not known, the slope might be
estimated from the points. If f 0 (x0 ) = A and f 0 (x1 ) = B, we use these
relations (note that divided differences are employed):
At left end: 2h0 S0 + h0 S1 = 6(f [x0 , x1 ] − A)

At right end: hn−1 Sn−1 + 2hn−1 Sn = 6(B − f [xn−1 , xn ]).
Take S0 = S1 ,Sn = Sn−1 . This is equivalent to assuming that the end
cubics approach parabolas at their extremities.
Take S0 as a linear extrapolation from S1 and S2 , and Sn , as a linear
extrapolation from Sn−2 and Sn−2 . Only this condition gives cubic
spline curves that match exactly to f (x) when f (x) is itself a cubic.
For condition (d), we use these relations:
(h0 + h1 )S1 − h0 S2
At left end : S0 =
h1
(hn−2 + hn−1 )Sn−1 − hn−1 Sn−2
At right end: Sn = .
hn−2
This is called not a knot condition.
Example
Fit the given data with a natural cubic spline curve, and evaluate the spline values
g (0.66) and g (1.75).
x 0.0 1.0 1.5 2.25

f (x) 2.0000 4.4366 6.7134 13.9130
Solution: For a natural cubic spline, we use S0 = 0, S3 = 0 and solve

3.0 0.5 S1 12.7020
= ,
0.5 2.5 S2 30.2754
giving S1 = 2.2920 and S2 = 11.6518. Using these S 0 s, we compute the

coefficients of the individual cubic splines to arrive at
We use g0 to find g (0.66) : It is 3.4659 and use g2 to find g (1.75) : It is 8.7087.

Least-Squares Approximations
Least square approximations are most commonly used approximations for
approximating a function f (x) which may be given in tabular form or known
explicitly over a given interval.
The best approximation in the least square sense is defined as that for which
the constants ci , i = 0, · · · , n are determined so that the error function will
be minimum.
For functions whose values are given at N + 1 points x0 , x1 , · · · , xN , we have
N
X h n
X i2
I (c0 , c1 , · · · , cn ) = W (xk ) f (xk ) − ci φi (xk ) ,
k=0 i=0
where W (x) > 0 is the weight function.

For functions which are continuous on [a, b] and are given explicitly, we have
Z b h n
X i2
I (c0 , c1 , · · · , cn ) = W (x) f (x) − ci φi (x) dx.
a i=0
The coordinate functions φi (x) are usually chosen as
φi (x) = x i , i = 0, 1, · · · , n
and W (x) = 1.
The necessary conditions for error function I will be minimum is that
∂I
= 0, i = 0, 1, · · · , n.
∂ci
This gives a system of n + 1 linear equations in n + 1 unknowns

c0 , c1 , · · · , cn . These equations are called normal equations. The normal
equations are
N
X h n
X i
W (xk ) f (xk ) − ci φi (xk ) φj (xk ) = 0, j = 0, 1, · · · , n,
k=0 i=0
Z b h n
X i
W (x) f (x) − ci φi (x) φj (x)dx = 0, j = 0, 1, · · · , n.
a i=0
Example
Obtain a linear polynomial approximation to the function f (x) = x 3 on the
interval [0, 1] using the least square approximation with W (x) = 1.
Example
Find the least squares approximation of second degree for the discrete data
x -2 -1 0 1 2
f (x) 15 1 1 3 19
Non-Linear Data
In many cases, of course, data from experimental tests are not linear, so we
need to fit to them some function other than a first-degree polynomial.
Popular forms that are tried are the exponential form
y = ax b or y = ae bx
We can develop normal equations for these analogously to the preceding

development for a least-squares line by setting the partial derivatives equal
to zero. Such nonlinear simultaneous equations are much more difficult to
solve than linear equations. Thus, the exponential forms are usually
linearized by taking logarithms before determining the parameters:
ln y = ln a + b ln x or ln y = ln a + bx.
We now fit the new variable z = ln y as a linear function of ln x or x as

described earlier.
Use of Orthogonal Polynomials
System of normal equations for a polynomial fit is ill conditioned when the
degree is high. Even for a cubic least-squares polynomial, the condition
number of the coefficient matrix can be large.
In one experiment, [Atkinson (1985)] a cubic polynomial was fitted to 21
data points. When the data were put into the coefficient matrix of normal
equations, its condition number (using 2-norms) was found to be 22,000!
This means that small differences in the y -values will make a large difference
in the solution.
However, if we fit the data with orthogonal polynomials such as the
Chebyshev polynomials, the condition number of the coefficient matrix is
reduced to about 5 and the solution is not much affected by the
perturbations.
Next, we will discuss about well known orthogonal polynomials, Legendre
polynomials and Chebyshev polynomials.
Legendre Polynomials
The Legendre polynomials Pn (x) defined on [−1, 1] are given by
M
X (2n − 2m)!x n−2m
Pn (x) = (−1)m ,
m=0
2n m!(n − m)!(n − 2m)!
where M = n/2 or (n − 1)/2 whichever is an integer. In particular

1 1
(3x 2 − 1), P3 (x) = (5x 3 − 3x)
P0 (x) = 1, P1 (x) = x, P2 (x) =
2 2
1 1
P4 (x) = (35x − 30x + 3), P5 (x) = (63x − 70x 3 + 5x).
4 2 5
8 8
The Legendre polynomials satisfy the differential equation
(1 − x 2 )y 00 − 2xy 0 + n(n + 1)y = 0.
The Legendre polynomials satisfy the recurrence relation

(n + 1)Pn+1 (x) = (2n + 1)xPn (x) − nPn−1 (x).
Legendre polynomials possess the following properties:
1) Pn (x) is an even polynomial if n is even and an odd polynomial if n is
odd.
2) Pn (x) are orthogonal polynomials and satisfy
Z 1 (
0 m 6= n
Pm (x)Pn (x)dx = 2
−1 2n+1 m = n.
3) Pn (−x) = (−1)n Pn (x).
Chebyshev Polynomials : The Chebyshev polynomials of the first
kind Tn (x) defined on [−1, 1] are given by
Tn (x) = cos(n cos−1 x) = cos nθ,
where θ = cos−1 x or x = cos θ.
These polynomials satisfy the differential equation (DE)
(1 − x 2 )y 00 − xy 0 + n2 y = 0.
One independent solution of the above DE gives Tn (x) and second one gives
Chebyshev polynomials of the second kind Un (x)
Un (x) = sin[(n + 1) cos−1 x]
The Chebyshev polynomials Tn (x) satisfy the recurrence relation
Tn+1 (x) = 2xTn (x) − Tn−1 (x)

T0 (x) = 1, T1 (x) = x.
The Chebyshev polynomials Tn (x) possess the following properties:

1) Tn (x) is a polynomial of degree n. If n is even Tn (x) is an even
polynomial and if n is odd, Tn (x) isan oddpolynomial.
2) Tn (x) has n simple zeros, xk = cos 2k−12n π , k = 1, 2, · · · , n on the
interval [−1, 1].
3) Tn (x) assumes extreme values at n + 1 points
xk = cos(kπ/n), k = 0, 1, · · · , n and the extreme value at xk is (−1)k .
4) Tn (x) are orthogonal
√ with respect to the weight function
W (x) = 1/ 1 − x 2

Z 1
Tm (x)Tn (x) 0, m 6= n

√ dx = π2 , m = n 6= 0
−1 1 − x2 
π, m = n = 0.

Example
Using the Chebyshev polynomials, obtain the least squares approximation of
second degree for f (x) = x 4 on [−1, 1].

Numerical Analysis: MTL851: Dr. Mani Mehra

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Numerical Analysis: MTL851: Dr. Mani Mehra

Uploaded by

Copyright:

Available Formats

Numerical Analysis : MTL851

Dr. Mani Mehra

March 20, 2023

We will be most interested in techniques adapted to situations where the

x 3.2 2.7 1.0 4.8 5.6

After solving the above system we get

x 3.2 2.7 1.0 4.8

Using Lagrange interpolation polynomial to find the interpolated value for

T (x) = Pn (x) − Qn (x).

T (xi ) = Pn (xi ) − Qn (xi ) = 0, i = 0, 1, · · · , n.

Therefore T (x) is a polynomial of degree ≤ n which has (n + 1) distinct

Obviously, f (x) − Pn (x) − E (x) = 0, So

To determine g (x), we now use the interesting mathematical device of

with ξ on the smallest interval that contains {x, x0 , x1 , · · · , xn }.

Higher-order differences are defined in terms of lower-order differences.

f [x1 , · · · , xn ] − f [x0 , · · · , xn−1 ]

For finding ai , set Pn (xi ) = fi , then we get ai = f [x0 , · · · , xi ]. We then can

x 3.2 2.7 1.0 4.8 5.6

To obtain reasonably accurate results using interpolation, we may have to

Take S0 = 0 and Sn = 0. This makes the end cubics approach linearity at

At left end: 2h0 S0 + h0 S1 = 6(f [x0 , x1 ] − A)

This is called not a knot condition.

x 0.0 1.0 1.5 2.25

Solution: For a natural cubic spline, we use S0 = 0, S3 = 0 and solve

giving S1 = 2.2920 and S2 = 11.6518. Using these S 0 s, we compute the

We use g0 to find g (0.66) : It is 3.4659 and use g2 to find g (1.75) : It is 8.7087.

where W (x) > 0 is the weight function.

This gives a system of n + 1 linear equations in n + 1 unknowns

We can develop normal equations for these analogously to the preceding

We now fit the new variable z = ln y as a linear function of ln x or x as

where M = n/2 or (n − 1)/2 whichever is an integer. In particular

The Legendre polynomials satisfy the recurrence relation

Tn+1 (x) = 2xTn (x) − Tn−1 (x)

The Chebyshev polynomials Tn (x) possess the following properties:

You might also like