12/9/2020

Curve Fitting: Least Squares

Method

September, 2020

Least Squares Method

• The curve of best fit is that for which e’s are as
small as possible.
• Principle: The sum of the squares of the errors is
a minimum.
• Let the set of data points be (xi,yi), i = 1,2,3, … …,
n and let the curve given y = f(x) be fitted to this
data. At x = xi, the experimental (or observed)
value of the ordinate is yi and the corresponding
value on the fitting curve is f(xi). If ei is the error
of approximation at x = xi, then we have:
ei = yi – f(xi) ---- (1)

1
 12/9/2020

Cont’d ….
If we write:

S= [y1 – f(x1)]2 + [y2 – f(x2)]2 + [y3 – f(x3)]2 + … + [yn – f(xn)]2

S = e12 + e22 + e32 + … + en2 ----(2)

Cont’d…
1. Linear Curve Fitting
2. Non - Linear Curve Fitting
a) Exponential Function
b) Power Function
c) Polynomial of the nth degree

2
 12/9/2020

1. Linear Curve Fitting/Fitting a

Straight Line
Let y = f(x) = a+bx be the straight line to be fitted
to the given data (xi,yi), where i = 1,2,3,…,n. Then
corresponding to the equation (2), we have

S= [y1 – (a+bx1)]2 + [y2 – (a+bx2)]2 + [y3 –

(a+bx3)]2 + … + [yn – (a+bxn)]2 ---- (3)

Cont’d…

Differentiate eqn(3) w.r.t. a. For S to be minimum. We have,

s
 0  2 y1  (a  bx1 )  2 y2  (a  bx2 )  2 y3  (a  bx3 )  ...  2 yn  (a  bxn )
a
y1  y2  y3  ...  yn  a  a  a  ...  a  b x1  x2  x3  ...  xn 
n n

y
i 1
i  n  a  b xi
i 1
 (4)

Again, Differentiate eqn(3) w.r.t. b. For S to be minimum. We

have,

3
 12/9/2020

Cont’d…
n n

y
i1
i  n  a  bxi
i1
(4)
n n n

i i i i
x
i1
y  a x  b x 2

i1 i1
(5)

These equations are known as the Normal Equations.

Since xi and yi are known quantities, on solving eqn (4 & 5)
we can determine the constants a and b.

Example:

Fit the curve of the form y = a + bx to the following data set.

x 1 2 3 4 5
y 3 4 5 6 8

Solution:
Given:
No. of data set (n) =5
The given curve is: y = a + bx - (1)
Normal Equations of eqn(1) are
 y  n  a  b  x 2 
 xy  a  x  b  x 3 
2

4
 12/9/2020

Example:
The values of  x,  x ,  xy ,  y are calculate shown in
2

the following table

x y xy x2
1 3
2 4
3 5
4 6
5 8
∑x =15 26 90 55

Substituting these values in equation 2 and 3

Example:
On solving, we get
a = 1.6
b = 1.2
Hence, the required linear best fit
relation is y = 1.6 + 1.2x

5
 12/9/2020

Non – Linear Curve Fitting

a) Exponential Function
Let the curve be y  ae bx  (1)
Taking log on both sides. We get,
log y = log a + bx*loge
Y = A + bx - (2) [∵loge = 1]
Where,
Y = log y
A = log a

Exponential Function
Write Normal Equation
𝑌 =𝑛∗𝐴+𝑏 𝑥

∑ 𝑥𝑌 =A∑ 𝑥 + 𝑏 ∑ 𝑥

6
 12/9/2020

Exponential Function:: Example

Determine the constants a and b by the method
of least squares such that y  aebx fits the
following data.
x 2 4 6 8 10
y 4.077 11.084 30.128 81.897 222.62

Non – Linear Curve Fitting

b) Power Function
Let the curve be y  ax b
Taking log on both sides. We get,
log y = log a +b*log x

Y = A + bX -(2)

Where,
Y = log y
A = log a
X = log x

7
 12/9/2020

Power Function
Write Normal Equation

Power Function:: Example

Using least squares method, fit the curve of
the form y = axb to the following data set.
x 1 2 3 4 5
y 0.50 2.00 4.50 8.00 12.50

8
 12/9/2020

Non – Linear Curve Fitting

c) Polynomial of the nth degree
Let the second order polynomial be
y = a + bx +cx2 - (1)
The Normal Equations are
𝑦 =𝑎∗𝑛+𝑏 𝑥+𝑐 𝑥

𝑥𝑦 = 𝑎 ∗ 𝑥+𝑏 𝑥 +𝑐 𝑥

𝑥 𝑦=𝑎 𝑥 +𝑏 𝑥 +𝑐 𝑥

Example:
Fit a second order polynomial to the data in the
following table.
x 1 2 3 4
y 6 11 18 27

For the following set of data, fit a parabolic

curve using Least Squares Method and find y(2)
x 0.5 1 1.5 4.5 6.5 7.5
y 2.5 2.7 3.5 6.5 8.4 9.5

9
 12/9/2020

Example
a) y=a+bx2 ------(1)
y=a+bX ---- (2)
Where,
X = x2
b) y=a+b*1/x ------(1)

c) y=alogex +b (1)
y=aX+b
X = logex

d) y=ax+b+c/x (1)
xy =ax2+bx+c
Y= ax2+bx+c
Where,
Y = xy

Example
e) xya = b
logx + alogy = logb
logy =(logb-logx)/a
log y = 1/a*(logb) – (1/a)*logx

Y = A +BX

Where,
A= 1/a*logb
B= -1/a
Logx=X

10