Curve Fitting

Polynomial Interpolation (section 6.5)

Given n distinct input values {x1 , x2 , ... , xn } and associated output values {y1 , y2 , ... , yn }
which are not necessarily distinct , we can always find a unique polynomial of degree at most
n 1 (one less than the number of nodes {(x1 , y1 ) , , (xn , yn )})
p(x) = an1 xn1 + + a1 x + a0
such that

p(x1 ) = y1

p(x2 ) = y2
..

p(xn ) = yn
For example , we can find a polynomial p(x) = a2 x2 + a1 x + a0 that sends the input values
{1 , 2 , 1} to the output values {0 , 0 , 3}. In

p(1)
=
0
a2 a1 + a0

p(2) = 0
4a2 + 2a1 + a0

p(1) = 3
a +a +a
2
1
0
and in the matrix form:

1 1 1

2
1

fact, we must have:

= 0
1 1 1
a2
0

4 2 1 a1 = 0
= 0

= 3
1 1 1
a0
3

a2

1 a1 = 0

1
a0
3

In Mathematics it is proved that if the input values {x1 , x2 , ... , xn } are distinct the
coefficient matrix is invertible and therefore this system has a unique solution, which results
in a unique polynomial that sends the input values xi to the output values yi .

To find a0 and a1 and a2 , we must solve this particular system in Matlab by entering the
coefficient matrix and the vector of the entris on the right-hand side:
>> A = [1 1 1 ; 4 2 1 ; 1 1 1];
>> b = [0 ; 0 ; 3]
>> A\b
The command A\b is equivalent to the operation A1 b. This command results in the values

1.5000

1.5000

3.0000
and therefore:
p(x) = 1.5000 x2 + 1.5000 x + 3.0000
However for large number of noes this method is not recommended as it is not an efficient
method for finding the polynomial and that the matrix of coefficients might be ill-conditioned
(we study this concept later on). Instead we use the Lagrange and the Newton forms for the
interpolating polynomial.

Lagrange form of Interpolating Polynomial (section 6.5.1)



 x1 = 1


the Lagrange polynomials are:
For the input values  x2 = 2


 x3 = 1
L1 (x) =

(x x2 )(x x3 )
(x 2)(x 1)
1
=
= (x 2)(x 1)
(x1 x2 )(x1 x3 )
(1 2)(1 1)
6

L2 (x) =

(x x1 )(x x3 )
(x + 1)(x 1)
1
=
= (x + 1)(x 1)
(x2 x1 )(x2 x3 )
(2 + 1)(2 1)
3

L3 (x) =

(x + 1)(x 2)
1
(x x1 )(x x2 )
=
= (x + 1)(x 2)
(x3 x1 )(x3 x2 )
(1 + 1)(1 2)
2

These polynomials are of degree 2 (one less than the number of nodes). The Lagrange
polynomials enjoy this property:


 L1 (x1 ) = 1


 L1 (x2 ) = 0


 L1 (x3 ) = 0



 L2 (x1 ) = 0


 L2 (x2 ) = 1


 L2 (x3 ) = 0



 L3 (x1 ) = 0


 L3 (x2 ) = 0


 L3 (x3 ) = 1

For the output values



 y1 = 0


 y2 = 0


 y3 = 3
the combination
p(x) = y1 L1 (x) + y2 L2 (x) + y3 L3 (x)
which is of degree at most 2 sends the input values {x1 , x2 , x3 } to the output values
{y1 , y2 , y3 }. For example:
p(x2 ) = y1 L1 (x2 ) + y2 L2 (x2 ) + y3 L3 (x2 ) = (y1 )(0) + (y2 )(1) + (y3 )(0) = y2
Note that for this particular example we have:

3
3
3
p(x) = y1 L1 (x) + y2 L2 (x) + y3 L3 (x) = (x + 1)(x 2) = x2 + x + 3
2
2
2
Note. In you homework assignments once you find this polynomial to be
p(x) = 32 (x + 1)(x 2) then you can leave it like that and you dont have to expand it.
Example. Suppose that the values


 f (1) = 0


 f (2) = 0


 f (1) = 3
of a function f are given. Use the interpolating polynomial to estimate f (1.7) .
Example. We have already calculated the interpolating polynomial p(x) = 23 x2 + 32 x + 3
that satisfies



 p(1) = 0


 p(2) = 0


 p(1) = 3

Now we use it to approximate:

3
3
f (1.7) p(1.7) = (1.7)2 + (1.7) + 3 = 1.2150
2
2


 x1 = 1


Example. Let us take the same input values  x2 = 2
but change the output values to


 x3 = 1


 y1 = 32


 y2 = 43


 y3 = 56
Then the interpolating polynomial is ( 23 )L1 (x) + ( 43 )L2 (x) + ( 56 )L3 (x)
o
o
o
3 n1
4 n1
5 n 1
= ( ) (x 2)(x 1) + ( ) (x + 1)(x 1) + ( ) (x + 1)(x 2)
2 6
3 3
6
2
4

1
4
5
= (x 2)(x 1) + (x + 1)(x 1) + (x + 1)(x 2)
4
9
12
An interesting note. If {x1 , x2 } are two distinct points in the real line, and if {y1 , y2 } are
two output values assigned to those input values, then the interpolating polynomial for these
two data points is of degree at most 1 and satisfies

p(x1 ) = y1
p(x ) = y
2
2
and the Lagrange form of it is
p(x) =

x x2
x x1
y1 +
y2
x1 x2
x2 x1

p(x) =

x x2
x1 x
y1 +
y2
x1 x2
x1 x2

equivalently

If x is located between x1 and x2 , then the weights are positive and they sum to one.
Example. Assume that the point x is in a location between x1 and x2 one-third from x1 and
two-thirds from x2 :
2
1
x = x1 + x2
3
3
Then we have
1
2
p(x) = y1 + y2
3
3
So

p

2
1
x1 + x2
3
3

2
1
= p(x1 ) + p(x2 )
3
3

This nice property only hols for linear interpolation, i.e. when p(x) is of degree at most one.

Here I explained the code on page 216 and its output

Newtons form of Interpolating Polynomial (section 6.5.2)

Definition. For any set of input values {x1 , x2 , x3 , ...} and output values {y1 , y2 , y3 , ...}
the divided differences are defined recursively by:
second order divided differences:
[yi , yj ] =

yj yi
xj xi

higher order divided differences:

[yi , yi+1 , ... , yj1 , yj ] =

[yi+1 , ... , yj1 , yj ] [yi , yi+1 , ... , yj1 ]

xj xi

Note. A nice property of the divided differences is that changing the order of its argument
does not affect its value; for example we have [y3 , y5 , y12 , y17 ] = [y12 , y5 , y17 , y3 ]
Note. If the output values yi are values of a function f , then instead of
[yi , yi+1 , ... , yj1 , yj ] the notation f[xi , xi+1 , ... , xj1 , xj ] is used.
Example. For the function whose values are given by
x

4.6

5.5

f (x)

-5

-21.3

-40

-7.3

10

Calculate f [5 , 7 , 2 , 4]
Solution. Since the reordering does not matter, we calculate f [2 , 4 , 5 , 7]

xk

f [ ] = f ()

f[ , ]

55
42

35(5)

-5

= 10
20(10)
72

= 35
25(35)
74

-40
10(40)
75

f[ , , , ]

= 5

40(5)
54

f[ , , ]

= 6 = f [2 , 4 , 5 , 7]

= 20

= 25

10

So the answer is f [2 , 4 , 5 , 7] = 6
Exercise. Calculate f [5 , 7 , 2 , 4] without reordering to see that it gives the same answer.
xk

f [ ] = f ()

-40

f[ , ]

f[ , , ]

f[ , , , ]

?
7

10

?
?

?
?

?
4

-5

Theorem. If {x1 , x2 , ... , xn } are some distinct input values, then the Newtons form of the
interpolating polynomial p(x) that agrees with f at these input points

p(x1 ) = f (x1 )

p(x2 ) = f (x2 )
..

p(xn ) = f (xn )
is

p(x) = f (x1 ) + f [x1 , x2 ](x x1 ) + f [x1 , x2 , x3 ](x x1 )(x x2 ) + + f [x1 , x2 , ... , xn ](x
x1 ) (x xn1 ).
Example. The following information has been received about the values of a function f . Find
the polynomial with the least degree interpolating the data:
x

14

16

Then approximate f (2.3).

Example.
xk

f [ ] = f ()

f[ , ]

84
42

32
62

1
4
12 14
28

=3
13
84

14
1614
86

f[ , , , ]

= 2

148
64

f[ , , ]

= 18

= 21

=1

16

The interpolating polynomial is:

1
1
p(x) = 4 + 2(x 2) + (x 2)(x 4) (x 2)(x 4)(x 6)
4
8
1
7
= x3 + x2 5x + 8
8
4

writing this expanded form is optional

1
1
f (2.3) p(2.3) = 4 + 2(2.3 2) + (2.3 2)(x 4) (2.3 2)(2.3 4)(2.3 6) = 4.2366
4
8
Note. Because of irrelevance of reordering, the formula that is written on page 220 of the
textbook is the same as the one we wrote above.
8

Here I explained the code on page 222 and its output

Curve fitting with a linear combination of arbitrary

functions (section 6.8)

Suppose that n functions f1 (x) , ... , fm (x) and n data points (x1 , yn ) , ... , (xn , yn ) are given
, where as usual we assume that the input values {x1 , ..., xn } are distinct. We want to find a
Linear Combination c1 f1 (x) + + cm fm (x) of the functions that would best fit the data set
(x1 , y1 ) , ... , (xn , yn )
One way of formalizing this is to calculate the combination c1 f1 (x) + + cm fm (x) at the
input values xi :

c f (x ) + + cm fm (x1 )

1 1 1
..
.

c f (x ) + + c f (x )
1 1

m m

and then take the differences between these values and the associated output values:

y (c1 f1 (x1 ) + + cm fm (x1 ))

1
..
.

y (c f (x ) + + c f (x ))
n

1 1

m m

and then search for those c1 , ... , cm that minimizes the sum of squares:
min

c1 , ... , cm

n
o2
n
o2
E = y1 (c1 f1 (x1 ) + + cm fm (x1 )) + + yn (c1 f1 (xn ) + + cm fm (xn ))

so we treat c1 , ... , cm as some unknowns in this problem. In this problem, the necessary and
sufficient condition for having the minimum is to have the partial derivatives equal to zero (we
do not prove this fact in this course).
E
=0
c1

n
o
2 y1 (c1 f1 (x1 ) + + cm fm (x1 )) (f1 (x1 )) +
10

n
o
+2 yn (c1 f1 (xn ) + + cm fm (xn )) (f1 (xn )) = 0
After dropping 2 , we get:
n
o
y1 (c1 f1 (x1 ) + + cm fm (x1 )) (f1 (x1 )) +
n
o
+ yn (c1 f1 (xn ) + + cm fm (xn )) (f1 (xn )) = 0
This is equivalent to having:
n
o
n
o
f1 (x1 ) (c1 f1 (x1 ) + + cm fm (x1 ) + + f1 (xn ) (c1 f1 (xn ) + + cm fm (xn ) =
= f1 (x1 )y1 + + f1 (xn )yn
and this is a linear equation on the unknowns c1 , ... , cm . This linear
 equation is found by
E
E E
E
setting
= 0. By setting all the derivatives
,
, ... ,
equal to zero we will
c1
c1 c2
cm
find m linear equations involving the unknowns c1 , ... , cm . The m equations are the ones that
appear on page 239 of the textbook in the display (6.97).
Now the question here is this: Is there a simple way of solving this system of linear equations
to find the values of the unknowns c1 , ... , cm ?. The answer is yes. Here is the way:
First: Form the n m matrix

f1 (x1 )

f1 (x2 )
X=

..

f1 (xn )

fm (x1 )

fm (x2 )

..

fm (xn )

Note that the first column of this matrix is the set of values of the function f1 at the input
points {x1 , x2 , ... , xn }, the second row is the set of values of the function f2 , and so on.

11

Second: Form the column matrix

y1

.
Y = ..

yn
the solution to the system is the matrix

c
1
..
.

cm

= (X 0 X)1 X 0 Y

where here X 0 represents the transpose of the matrix X ; the same command in Matlab
generates the transpose. Note that the matrix X 0 X
X 0 |{z}
X
|{z}

mn nm

is m m, therefore its inverse , (X 0 X)1 , is also m m. On the other hand, the matrix X 0 Y
X 0 |{z}
Y
|{z}

mn n1

is m 1. Then the multiplication

0
(X 0 X)1 X
Y
| {z } |{z}
mm

m1

is m 1.
Note. There does not seem to be a direct mentioning in the textbook of this method of
finding the unknowns c1 , ... , cm .
Example. The following m-file , which uses the technique described above, gives the same
result that the books m-file generates on page 240 of it.
x = [0.6 ; 0.8 ; 0.85 ; 0.95 ; 1.0 ; 1.1 ; 1.2 ; 1.3 ; 1.45 ; 1.6 ; 1.8] ;
Y = [0.08 ; 0.06 ; 0.07 ; 0.07 ; 0.07 ; 0.06 ; 0.06 ; 0.06 ; 0.05 ; 0.05 ; 0.04] ;
X = [1./x , exp(2 x . 2)./x] ;
12

inv(X0 X) (X0 Y)
By calling this m-file at the command line ; we get the values
c1 = 0.0743
c2 = 0.0597
these are the same values that the textbook has found on page 240 , but the textbook uses a
somewhat different implementation.

Here I worked on the example on pages 239 and 240, and I got the same
result by running our own code

13

Curve fitting with polynomials (polynomial regression)

(section 6.4)

Suppose again that some distinct input values {x1 , ..., xn } and some output values {y1 , ..., yn }
are given. As we saw in the section on interpolation it is possible to find a polynomial of
degree less than or equal to n 1 that satisfies

p(x1 ) = y1

p(x2 ) = y2
..

p(xn ) = yn
However if we restrict ourselves to specific classes of polynomials , for example linear
polynomials or quadratic polynomials and so on, then we may no longer be able to find a
polynomial in that specific class that would satisfy the equalities at all xi s. For example
consider the class of all polynomials of degree less than or equal to m. An arbitrary
polynomial in that class is of the form
p(x) = am xm + am1 xm1 + + a1 x + a0
Now that we know we are no longer to have equalities at the xi s then at least we can search
for a polynomial in that class that minimizes the distance:
(y1 p(x1 ))2 + + (yn p(xn ))2
So we want to find the coefficients am , am1 , ... , a1 , and a0 so as to minimize the distance:
min

am , ... , a1 , a0

n
o2
n
o2
m1
m1
y1 (am xm
+ +a1 x1 +a0 ) + + yn (am xm
+ +a1 xn +a0 )
1 +am1 x1
n +am1 xn

But we have already seen how to solve such distance minimization problems !. In fact this is a
minimization problem for a linear combination of the functions
xm , xm1 , ... , x , 1

14

For this we need to form the matrix X whose column are the values of these functions at the
input values {x1 , ..., xn }

xm
xm1

1
1

x1

X=

xm
xm1

2
2
..
..
.
.

x2
..
.

1
..
.

xm
xm1

n
n

xn 1

and then we have the coefficients of the polynomial to be the solution of

am

am1

..
. = (X 0 X)1 X 0 Y

a1

a0
Be careful about the order of the coefficients you find through this equation. This fives the
coefficients of the polynomial p(x) = am xm + am1 xm1 + + a1 x + a0 the best fits the data
n
o
points (x1 , y1 ) , ... , (xn , yn ) .

Now let us solve the Example 6-3 of the textbook (on pages 208 to 210) using our own way.
The input values are
{0 , 0.4 , 0.8 , ... , 6.0}
We generate it using this piece of code:
x = [0 : 0.4 : 6] 0 ;
This vector has 16 entries; he transpose operator creates this vector as a column vector. Then
we enter the column vector Y :
Y = [0 3.0 4.5 5.8 5.9 5.8 6.2 7.4 9.6 15.6 20.7 26.7 31.1 35.6 39.3 41.5]0 ;
The problem is to find a fourth-order polynomial p(x) = a4 x4 + a3 x3 + a2 x2 + a1 x + a0 that
best fits the data points
(x1 , y1 ) , (x2 , y2 ) , ... , (x16 , y16 )
15

To solve this problem, we create the columns of the matrix X :

X = [x. 4 , x. 3 , x. 2 , x , ones(16, 1)];
Then we put the line of code:
inv(X0 X) X0 Y
Here the m-file code that generates the coefficients of this polynomial for us:

x = [0 : 0.4 : 6] 0 ;

Y = [0 3.0 4.5 5.8 5.9 5.8 6.2 7.4 9.6 15.6 20.7 26.7 31.1 35.6 39.3 41.5]0 ;

X = [x. 4 , x. 3 , x. 2 , x , ones(16, 1)];

inv(X0 X) X0 Y
When we run this program, we get the following column as the answer:
ans =
0.2644
3.1185
10.1927
12.8780
0.2746
So the required polynomial is
p(x) = 0.2644 x4 + 3.1185 x3 10.1927 x2 + 12.8780 x 0.2746
Now here we add two more lines to the code in order to plot the graph of the polynomial and
the data points (x1 , y1 ) , ... , (xn , yn ) to see the closeness of the polynomial to those data
points:

16

x = [0 : 0.4 : 6] 0 ;

Y = [0 3.0 4.5 5.8 5.9 5.8 6.2 7.4 9.6 15.6 20.7 26.7 31.1 35.6 39.3 41.5]0 ;

X = [x. 4 , x. 3 , x. 2 , x , ones(16, 1)];

inv(X0 X) X0 Y

plot(x , Y , 0 or0 )

hold on

plot(x , X p)

Here is the plot of the data set and the polynomial:

45

40

35

30

25

20

15

10

5
0

Here the program on pages 209 and 210 was shown on the screen to
compare it with our code which is much shorter

17

Curve Fitting With a Linear Equation (linear regression)

(section 6.2)

This is special case of section 6.4 in which we want to find a polynomial of degree 1
P (x) = ax + b
that best fits a given set of data points
(x1 , y1 ) , ... , (xn , yn )
In this case, the matrix X has two columns:

X=

x1
x2
..
.
xn

the first column consists of the input values {x1 , ... , xn }.

For example let us take the input values
0 , 10 , 20 , ... , 100
and the output values

0.94 , 0.96 , 1.0 , 1.05 , 1.07 , 1.09 , 1.14 , 1.17 , 1.21 , 1.24 , 1.28

And we want to find a polynomial p(x) = ax + b of degree one that best fits the data set
(x1 , y1 ) , ... , (x11 , y11 )

Then the following m-file will give us the values for a and b and plots the data points together
with the graph of the polynomial that is a straight line.
18

x = [0 : 10 : 100] 0 ;
Y = [0.94 0.96 1.0 1.05 1.07 1.09 1.14 1.17 1.21 1.24 1.28]0 ;
X = [x , ones(length(x), 1)] ;
p = inv(X0 X) X0 Y
plot(x , Y , 0 or0 )
hold on
plot(x , X p)

Here the program on pages 199 and 200 was shown on the screen to
compare it with our code which is much shorter

19

Curve fitting with nonlinear equations (section 6.3)

If you are interested in modeling a date set by some nonlinear equation, then you may be able
to transform the model into a linear form and then use a linear regression to find the best
fitting model. For example, if you want to fit the model y = axb to your data set, then you
may take the log function of both sides and write the model as ln(y) = ln(a) + b ln(x) which is
now a linear model whose coefficients can be determined using a linear regression philosophy.
Once you find the best coefficients of this transformed model , which are ln(a) and b, then you
need to transform it back to the original format. Some other transformations are displayed on
page 202 of the textbook; I explained them from the textbook, in class. Let us work on
exercise 6.28 of the textbook for this new subject.
Example (Exercise 6.28 of the textbook). Bacteria growth rate can be modeled with the
equation Nt = N0 e(tt0 ) where t0 is some fixed time, and, and N0 are some parameters to
be determined so that the model best fits the following data set. The time t is measure in
hours and the number N is measured in

cells
ml .

35

1990

70,800

2,810,000

141,250,000

Solution. According to the table, the initial time is zero, therefore t0 = 0 and the model to
be fitted is Nt = N0 e t . Here N0 and are the parameters of the model. We transform the
model into a linear model:
ln(Nt ) = ln(N0 ) + t
Now we form the matrix

1 0

1 2

X= 1 4

1 6

1 8

ln(35)

ln(1990)

Y =
ln(70, 800)

ln(2, 810, 000)

ln(141, 250, 000)

20

3.5553

7.5959

= 11.1676

14.8486

18.7660

And the coefficients of the linear model are:

3.6519
a

0 =
1.8837
a1
These values are ln(N0 ) and . To get N0 , just exponentiate:
N0 = exp(3.6519) = 38.5470
So the best model to describe that data set is

N = 38.5470 e1.8837 t
Now let us plot the data set together with the graph of the function N = 38.5470 e1.8837 t in
one figure:
4

x 10
5

4.5

3.5

2.5

1.5

0.5

0
0

0.5

1.5

2.5

This is the m-file I have used for this job: t = [0 2 4 6 8] 0 ;

N = [35 1990 70800 2810000 141250000] 0 ;
Y = log(N) ;
X = [ones(5, 1) t] ;
21

3.5

A = inv(X0 X) X0 Y ;
a0 = A(1, 1) ;
a1 = A(2, 1) ;
N0 = exp(a0) ;
mu = a1 ;
t1 = [0 : 0.01 : 8.5] ;
N fitted = N0 exp(mu t1) ;
plot(t1, N fitted) ;
hold on
plot(t, N,0 or0 ) ;
The Matlab function polyfit can do the linear regression and more efficiently than
(X 0 X)1 X 0 Y . But be careful that this function return the coefficient of the interpolating
polynomial from the biggest degree to the lowest degree. So, here is the equivalent code while
using the function polyfit :
t = [0 2 4 6 8] 0 ;
N = [35 1990 70800 2810000 141250000] 0 ;
Y = log(N) ;
X = [ones(5, 1) t] ;
p = polyfit(t, Y, 1) ;
a1 = p(1) ;
a0 = p(2) ;
N0 = exp(a0) ;
mu = a1 ;
t1 = [0 : 0.01 : 8.5] ;
N fitted = N0 exp(mu t1) ;
plot(t1, N fitted) ;
hold on
plot(t, N,0 or0 ) ;

22