Curve Fitting

Curve Fitting

Dr. Sukanta Deb

Department of Physics, Cotton University

Panbazar, Guwahati (Assam)

Lecture - I

Dr. Sukanta Deb Curve Fitting 1 / 37

 Curve Fitting

Outline of the Talk

1 Curve Fitting

Dr. Sukanta Deb Curve Fitting 2 / 37

 Curve Fitting

Table of Contents

1 Curve Fitting

Dr. Sukanta Deb Curve Fitting 3 / 37

 Curve Fitting

Topics in the Syllabus

In this Chapter
Fitting a Straight Line
Non-Linear Curve Fitting
(a) Power Function
(b) Polynomial of nth Degree
(c) Exponential Function

Dr. Sukanta Deb Curve Fitting 4 / 37

 Curve Fitting

Method of Least Squares

How do I fit the data points, say, (x1 , y1 ), (x2 , y2 ), (x3 , y3 ) and
(x4 , y4 ) with a curve y = f (x), where f (x) = a0 + a1 x (say)?

Dr. Sukanta Deb Curve Fitting 5 / 37

 Curve Fitting

Method of Least Squares

The method of least square states that “A curve is a best fit if the
sum of the square of the deviations of the individual points from
the curve is minimum."
Let us consider a set of tabulated data points (xi , yi ), where
i = 1, 2, . . . , m. Here xi ’s are the values of independent variables
and yi ’s are the corresponding dependent variables. Let y = f (x)
be a function which approximates the given data yi , corresponding
to the values of xi ’s. The deviations between the actual tabulated
values of yi ’s and the values of yi ’s obtained from the
approximation y = f (x) are given by
d1 = y1 − f (x1 )
d2 = y2 − f (x2 )
..
.
dm = ym − f (xm )
Dr. Sukanta Deb Curve Fitting 6 / 37
 Curve Fitting

Method of Least Squares

The sum of the squares of the deviations is given by

m
X
S = d21 + d22 + · · · + d2m = d2i .
i=1

Minimization of S is obtained from the condition

∂S
=0 (1)
∂aj

where aj ’s are the coefficients of the approximating function. The

values of aj ’s are obtained by solving the set of equations resulting
from Equation (1).

Dr. Sukanta Deb Curve Fitting 7 / 37

 Curve Fitting

Fitting a Straight Line

Given a set of data points (xi , yi ), i = 1, 2, . . . , m, the task is to
find a polynomial P1 (x) of degree 1 (straight line):

P1 (x) = a0 + a1 x

which minimizes the “the sum of the squared deviations":

m m
[yi − P1 (xi )]2 = [yi − a0 − a1 xi ]2 ,
X X
S=
i=1 i=1

where a0 and a1 represent the intercept and the slope of the

straight line, respectively. To determine the values of a0 , a1 , we
have to use the minimization condition:

∂S ∂S
= 0, =0 (2)
∂a0 ∂a1

Dr. Sukanta Deb Curve Fitting 8 / 37

 Curve Fitting

Fitting a Straight Line

m
∂S X
∴ = −2 (yi − a0 − a1 xi )
∂a0 i=1
m
∂S X
= −2 (yi − a0 − a1 xi )(xi )
∂a1 i=1

Imposition of the conditions of Equation (2) yields:

m
X m
X
ma0 + a1 xi = yi
i=1 i=1
m
X Xm Xm
a0 xi + a1 x2i = x i yi
i=1 i=1 i=1

These equations are called Normal Equations. These equations

can be solved using the methods of solving linear system of
equations. They can also be solved using the Cramer’s rule, if the
determinant of the coefficient matrix is non-zero (non-trivial).
Dr. Sukanta Deb Curve Fitting 9 / 37
 Curve Fitting

Fitting a Straight Line

Let us apply Cramer’s rule to solve the above two equations:
 P P 

 yi xi 
 
 
P P 2 P P 2 P P
 xi yi xi  yi xi − xi xi yi
a0 =  P  =

m x2i − ( xi )2
P P
 m xi 
 
 
P P 2
 xi xi 

 P 
 m yi 

 
 
P P 
 xi xi yi  m xi yi − xi yi
P P P
a1 =   P  = 
m x2i − ( xi )2
P P
 m xi 
 
 
P P 2
 xi xi 
Dr. Sukanta Deb Curve Fitting 10 / 37
 Curve Fitting

Example
Fit a straight line to the x and y values in the first two columns of
the following Table:
xi yi x2i xi yi P1 (xi )
1 1.3 1 1.3 1.18
2 3.5 4 7.0 2.72
3 4.2 9 12.6 4.25
4 5.0 16 20.0 5.79
5 7.0 25 35.0 7.33
6 8.8 36 52.8 8.87
7 10.1 49 70.7 10.41
8 12.5 64 100.0 11.94
9 13.0 81 117.0 13.48
10 15.6 100 156.0 15.02
P 2
S ∼ 2.34
P P P
xi = 55 yi = 81.0 xi = 385 xi yi = 572.4
[P1 (xi ) − yi ]2
P
S=

Dr. Sukanta Deb Curve Fitting 11 / 37

 Curve Fitting

Example
Let the straight line be given by
P1 (x) = a0 + a1 x,
where a0 , a1 are calculated to be
yi x2i − xi xi yi 385(81) − 55(572.4)
P P P P
a0 = = = −0.360
m x2i − ( xi )2 10(385) − (55)2
P P

m x i yi − x i yi 10(572.4) − 55(81)
P P P
a1 = = = 1.538
m x2i − ( xi )2 10(385) − (55)2
P P

Therefore, the least square (or the best) straight line fit is

P1 (x) = −0.360 + 1.538x

Note: Find the values of a0 and a1 in the above problem by

constructing the normal equations from the given data points and
then using Gauss elimination method for the solutions.
Dr. Sukanta Deb Curve Fitting 12 / 37
 Curve Fitting

Example

Figure: Filled circles denote the tabulated data points, whereas, the solid
line shows the straight line fit to the given data.

Dr. Sukanta Deb Curve Fitting 13 / 37

 Curve Fitting

Least Squares Polynomial Fitting

Given a set of data points (xi , yi ), i = 1, 2, . . . , m, the task is to

find a polynomial Pn (x) of degree n < m:

Pn (x) = a0 + a1 x + · · · + an xn

which minimizes the “the sum of the squared deviations":

m m h i2
2
X X
S= [yi − Pn (xi )] = yi − (a0 + a1 xi + a2 x2i + · · · + an xni ) ,
i=1 i=1

For S to be minimum, we must have

∂S
= 0, j = 0, 1, 2, . . . , n
∂aj

Dr. Sukanta Deb Curve Fitting 14 / 37

 Curve Fitting

Least Squares Polynomial Fitting

m
∂S X
= −2 (yi − a0 − a1 xi − · · · − an xni )
∂a0 i=1
m
∂S X
= −2 (xi )(yi − a0 − a1 xi − · · · − an xni )
∂a1 i=1
..
.
m
∂S X
= −2 (xni )(yi − a0 − a1 xi − · · · − an xni )
∂an i=1

Setting these equations to zero, i.e.,

∂S ∂S ∂S
= 0, =0 , = 0,
∂a0 ∂a1 ∂an

Dr. Sukanta Deb Curve Fitting 15 / 37

 Curve Fitting

Least Squares Polynomial Fitting

we get:
m
X m
X m
X m
X
ma0 + a1 xi + a2 x2i + · · · + an xni = yi
i=1 i=1 i=1 i=1
m m m m
xn+1
X X X X
a0 xi + a1 x2i + · · · + an i = xi yi
i=1 i=1 i=1 i=1
..
.
m m m m
xn+1
X X X X
a0 xni + a1 i + · · · + an x2n
i = xni yi
i=1 i=1 i=1 i=1

These equation are called normal equations and can be solved

using the methods of solving linear systems of equations.

Dr. Sukanta Deb Curve Fitting 16 / 37

 Curve Fitting

Rules for Constructing Normal Equations

Let the curve to be fitted be given by
y ≈ f (x) =a0 + a1 x. (3)
Note that there are two unknown coefficients a0 and a1 . So we
will require two normal equations. They can obtained in the
following four steps:
1 Discretize the Equation (3) with

yi =a0 + a1 xi , i = 1, 2, 3, . . . , m (4)
2 To get the first normal equation, take a sum on both sides
over the total number of data points, say m. That is
m
X m
X m
X
yi = a0 + a1 xi
i=1 i=1 i=1
m
X m
X m
X m
X
⇒ yi =ma0 + a1 xi ⇒ ma0 + a1 xi = yi
i=1 i=1 i=1 i=1
Dr. Sukanta Deb Curve Fitting 17 / 37
 Curve Fitting

Rules for Constructing Normal Equations

3 To get the second normal equation, first multiply equation (4)
by xi

xi yi =a0 xi + a1 x2i

4 And then take the sum over the total number of data points:

m
X m
X m
X m
X m
X m
X
xi yi = a0 xi + a1 x2i ⇒ a0 xi + a1 x2i = xi yi
i=1 i=1 i=1 i=1 i=1 i=1

General Rule: If there are (n + 1) number of unknowns, then the

second, third,....,(n + 1)th normal equations can be obtained by
multiplying the discretized equation by xi , x2i , . . . , xni , respectively
and then summing the resulting equation over the total number of
data points. The first normal equation is simply obtained by taking
the sum on both sides of the original discretized equation.
Dr. Sukanta Deb Curve Fitting 18 / 37
 Curve Fitting

Methods to Solve the Normal Equations

You may use any one of the methods for solving linear system of
equations to solve the normal equations to get the values of the
parameters of the fitted curve:
1 Gauss elimation method
2 Gauss-Jordan method
3 Gauss-Seidel Iterative method
4 Jacobi Iterative method
Other methods also do exist! But the above methods are numerical
and solutions are easily guaranteed! Besides this, you have learnt
these methods in the previous lecture of this course and applying
them in other problems will help you for their better understanding!

Dr. Sukanta Deb Curve Fitting 19 / 37

 Curve Fitting

Least Squares Polynomial Fitting

Let us now set
m
X
xki = sk , k = 0, 1, . . . , 2n
i=1

and denoting the right hand side entries as b0 , b1 , . . . , bn , the

above equations can be written as:

s0 a0 + s1 a1 + · · · + sn an = b0
s1 a0 + s2 a1 + · · · + sn+1 an = b1
..
.
sn a0 + sn+1 a1 + · · · + s2n an = bn

This is a system of (n + 1) equations in (n + 1) unknowns

a0 , a1 , . . . , an . These equations are called Normal Equations.
Dr. Sukanta Deb Curve Fitting 20 / 37
 Curve Fitting

Least Squares Polynomial Fitting

This system now can be solved to obtain these (n + 1) unknowns,
provided a solution to the system exists. The system has a
unique solution if xi ’s are distinct. The system can be written
in the following matrix form:
    
s0 s1 . . . sn a0 b0
 s1 s2 . . . sn+1   a1   b1 
    
 .  =  . 
. .. ..     
.
. . .   ..   .. 
sn sn+1 . . . s2n an bn
or sa = b
     
s0 s1 . . . sn a0 b0
 s1 s2 . . . sn+1   a1   b1 
    

where s = 
 .. ..  , a =  ..  , b =  .. 
..     
. . .   .  .
sn sn+1 . . . s2n an bn

Dr. Sukanta Deb Curve Fitting 21 / 37

 Curve Fitting

Least Squares Polynomial Fitting

Let us define
xn1
 
1 x1 x21 ...
1

x2 x22 ... xn2 

V = .. .. .. .. 
. . . . 
 

1 xm x2m . . . xnm

It follows that the normal equations can be written as

   
a0 y1
a1 
 
 y2 
 
V TV  T

 ..  = V
  . 
 . 
 .   . 
an ym

The matrix V is known as the Vandermonde matrix. This is the

easiest way if you have lesser number of data points!
Dr. Sukanta Deb Curve Fitting 22 / 37
 Curve Fitting

Example 1: Linear Polynomial Fit

xi 0.635 1.835 3.035 4.235 5.435
yi 7.50 3.55 2.20 1.46 1.000
The linear polynomial is: P1 (x) = a0 + a1 x. The Vandermonde
matrix is :
   
1 x1 1 0.635
1 x  1 1.835
 2  
V = 1 x3  = 1 3.035
   
   
1 x4  1 4.235
1 x5 1 5.435
 
7.50
" # 3.55 " #
T 5.0000 15.1750 
T 
 15.7100
∴V V = , V 2.20 =

15.1750 60.4561   29.5719
1.46
1.00
Dr. Sukanta Deb Curve Fitting 23 / 37
 Curve Fitting

Example 1: Linear Polynomial Fit

Therefore, the normal equations are

" #" # " #
5.0000 15.1750 a0 15.7100
=
15.1750 60.4561 a1 29.5719
" # " #
a 6.9585
⇒ 0 =
a1 −1.2575

Therefore, the linear polynomial fit is

P1 (x) = 6.9585 − 1.2575x

Dr. Sukanta Deb Curve Fitting 24 / 37

 Curve Fitting

Example 2: Quadratic Polynomial Fit

xi 0.635 1.835 3.035 4.235 5.435
yi 7.50 3.55 2.20 1.46 1.000
x2i 0.4032 3.3672 9.2112 17.9352 29.5392
The quadratic polynomial is: P2 (x) = a0 + a1 x + a2 x2 . The
Vandermonde matrix is :
1 x1 x21
   
1 0.635 0.4032
1 x x2  1 1.835 3.3672 
 2 2  
V = 1 x3 x23  = 1 3.035 9.2112 
   
 2  
1 x4 x4  1 4.235 17.9352


1 x5 x25 1 5.435 29.5392

 
 7.50   
5.000 15.1750 60.456 3.55
  15.7100
∴ V T V = 15.175 60.456 270.892  , V T 2.20 = 29.5719
     
 
60.456 270.892 1290.586 1.46 90.9672
1.00
Dr. Sukanta Deb Curve Fitting 25 / 37
 Curve Fitting

Quadratic Polynomial Fit

Therefore, the normal equations are:

    
5.0000 15.1750 60.456 a0 15.7100
15.1750 60.456 270.892  a1  = 29.5719
    
60.456 270.892 1290.586 a2 90.9672
   
a0 9.3421
⇒ a1  = −3.5428
   
a2 0.3765

Therefore, the quadratic polynomial fit is:

P2 (x) = 9.3421 − 3.5428x + 0.3765x2

Dr. Sukanta Deb Curve Fitting 26 / 37

 Curve Fitting

Linearization (For Non-Polynomial Fits)

Dr. Sukanta Deb Curve Fitting 27 / 37

 Curve Fitting

Non-Polynomial Fits
Exponential Fit: Let
y = aebx
Taking the natural logarithm on both sides

log y = log[aebx ] = log(a) + |{z}

b x
| {z } | {z }
Y a0 a1

The problem therefore reduces to finding a least squares

straight line through the given data points:

(x1 , log y1 ), (x2 , log y2 ), . . . , (xm , log ym ).

Then,
a = ea0 and b = a1
Power Law Fit: Let
y = axb
Dr. Sukanta Deb Curve Fitting 28 / 37
 Curve Fitting

Non-Polynomial Fits

Taking the natural logarithm on both sides

log y = log[axb ] = log(a) + |{z}

b log x
| {z } | {z } | {z }
Y a0 a1 X

The problem therefore reduces to finding a least squares straight

line through the given data points:

(log x1 , log y1 ), (log x2 , log y2 ), . . . , (log xm , log ym ).

Then,
a = ea0 and b = a1

Dr. Sukanta Deb Curve Fitting 29 / 37

 Curve Fitting

Example 1: Exponential Fit [y = aebx ]

Fit a function of the form y = aebx to the following data:
xi 2 4 6 8 10
yi 4.077 11.084 30.128 81.897 222.62

The given relation is y = aebx . Taking natural logarithm on both

sides,
log y = log a + |{z}
b |{z}
x
| {z } | {z }
Y a0 a1 X

Thus, if we set log y = Y, log a = a0 , b = a1 , x = X, the above

relation takes the form Y = a0 + a1 X, which is a straight line. We
know that a0 and a1 are given by

Yi Xi2 − Xi Xi Yi m Xi Yi − Xi Yi
P P P P P P P
a0 = 2 , a1 =
m Xi2 − ( Xi )2
P 2
m Xi − ( Xi )
P P P

Now, we make the following Table:

Dr. Sukanta Deb Curve Fitting 30 / 37
 Curve Fitting

Example 1: Exponential Fit [y = aebx ]

X=x Y = log y X2 XY
2 1.405 4 2.810
4 2.405 16 9.620
6 3.405 36 20.430
8 4.405 64 35.240
10 5.505 100 54.050
P P P 2 P
X = 30 Y = 17.025 X = 220 XY = 122.150

(17.025)(220) − (30)(122.150)
∴ a0 = = 0.405
5(220) − (302 )
5 ∗ (122.150) − (30) ∗ (17.025)
a1 = = 0.500
5(220) − (302 )
Hence a = ea0 = e0.405 = 1.499, and b = a1 = 0.500.

⇒ y = aebx = 1.499e0.5x

Dr. Sukanta Deb Curve Fitting 31 / 37

 Curve Fitting

Example 2: Exponential Fit [y = aebx ]

xi 0.635 1.835 3.035 4.235 5.435

yi 7.50 3.55 2.20 1.46 0.100
log(yi ) 2.0149 1.2669 0.7885 0.3784 0.0000

The linear fit is:

log y = log a + |{z}
b x
| {z } | {z }
Y a0 a1
   
1 x1 1 0.635
1
 x2 
 1
 1.835
The Vandermonde matrix is: V = 1 x3  = 1 3.035
   
   
1 x4  1 4.235
1 x5 1 5.435

Dr. Sukanta Deb Curve Fitting 32 / 37

 Curve Fitting

Example 2: Exponential Fit [y = aebx ]

 
2.0149
" # 1.2669 " #
T 5.000 15.1750 
T 
 4.4487
∴V V = , V 0.7885 =

15.175 60.4561   7.6000
0.3784
0.0000
Therefore, the normal equations are:
" #" # " #
5.0000 15.1750 a0 4.4487
=
15.1750 60.4561 a1 7.6000
" # " #
a 2.1337
⇒ 0 =
a1 −0.4099
Therefore,
y = 8.4461e−0.4099x
Dr. Sukanta Deb Curve Fitting 33 / 37
 Curve Fitting

Example 1: Power Law Fit [y = axb ]

xi 0.635 1.835 3.035 4.235 5.435

yi 7.50 3.55 2.20 1.46 0.100
log(xi ) -0.4541 0.6070 1.1102 1.4434 1.6929
log(yi ) 2.0149 1.2669 0.7885 0.3784 0.0000

The linear fit is:

log y = log a + |{z}
b log x
| {z } | {z } | {z }
Y a0 a1 X

The Vandermonde
  matrix
 is: 
1 log(x1 ) 1 −0.4541
1 log(x ) 1 0.6070 
 2   
V = 1 log(x3 ) = 1 1.1102 
   
   
1 log(x4 ) 1 1.4434 
1 log(x5 ) 1 1.6929

Dr. Sukanta Deb Curve Fitting 34 / 37

 Curve Fitting

Example 1: Power Law Fit [y = axb ]

 
2.0149
" # 1.2669 " #
T 5.000 4.3994 
T 
 4.4487
∴V V = , V 0.7885 =

4.3994 6.7564   1.2756
0.3784
0.0000
Therefore, the normal equations are:
" #" # " #
5.0000 4.3994 a0 4.4487
=
4.3994 6.7564 a1 1.2756
" # " #
a 1.6943
⇒ 0 =
a1 −0.9144
Therefore,
y = P (x) = 5.44281x−0.9144

Dr. Sukanta Deb Curve Fitting 35 / 37

 Curve Fitting

Assignment
1 Fit the values of a0 and a1 so that y = a0 + a1 x fits the
following data:
xi 0 1 2 3 4
yi 1.0 2.9 4.8 6.7 8.6
2 Fit a polynomial of the second degree to the data points
(x, y) given by (0, 1), (1, 6) and (2, 17).
3 Determine the constants a and b, by the method of least
squares, such that the curve y = aebx fits the data:
xi 2 4 6 8 10
yi 4.077 11.084 30.128 81.897 222.62
4 Fit a function of the form y = axb for the following data:
xi 61 26 7 2.6
yi 350 400 500 600

Dr. Sukanta Deb Curve Fitting 36 / 37

 Curve Fitting

Contact Me?

QPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPR

THANK YOU!

For any queries, suggestions:

k sukantodeb@gmail.com

T 9810362087

Í https://sites.google.com/site/sukantodeb

QPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPR
Dr. Sukanta Deb Curve Fitting 37 / 37