Applied Numerical Methods

Dr. Khaled M. Ahmida

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE-FITTING: LEAST SQUARE

Curve fitting vs Interpolation
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE-FITTING: LEAST SQUARE

Given a set of data:

Least square curve fitting falls into 2 methods; linear & non-linear
Assuming a linear type:
y=bx+a where a,b are to be determined
At any point (xj, yj) we have yj=bxj+a
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE-FITTING: LEAST SQUARE

If (xj, yj) is on a straight line, then yj -(bxj+a) =0

If (xj, yj) is NOT on a straight line (as in most cases, i.e. Least-Square) then:

rj  y j  (bx j  a)  0 for j  1,2,..., n

|rj| is the curve-fitting error.

If n > 2, we have more equations than variables  over-determined system.

There is no exact solution but is solved by minimizing the error  minimum-norm
solution.
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

The total error is given by: l1 approximation (1-norm approximation )
n n

r  y
j 1
j
j 1
j  (bx j  a)  1st order approximation

Least Square is: (continuously differentiable, l2 approximation)

n
q(a, b)   ( y j  (bx j  a)) 2 This part is Least-Square
j 1

The error should be minimized, thus this function should be minimized to find a&b
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

The LINEAR form:
This part is LINEAR

q depends on a and b. A necessary condition for q to be minimum is,


 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

These are the equations to use for Linear Least-Square method
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

Example #1: consider the points:

Find an approximation of these data using Linear Least square curve-fitting.

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

The solution by linear approximation:

Solving  a=0.9601 b=0.6670

And the linear Eqn. is y=0.9601+0.667x
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: LINEAR LEAST SQUARE

They correspond to the exact polynomial:

Approximated  y=0.9601+0.667x

This representation is poor, so we need a higher-order polynomial

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: M-DEGREE LEAST SQUARE

Curve-fitting by POLYNOMIAL of degree m (and number of Eqns=n):

The (m+1) conditions are now:

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: QUADRATIC LEAST SQUARE

In case of a QUADRATIC polynomial:

After minimization (using first derivatives=0), the normal Equations are:

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: QUADRATIC LEAST SQUARE

These are the equations to use for Quadratic Least-Square method
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING: QUADRATIC LEAST SQUARE

Example 2: fit the following points to a second degree polynomial, using LS
method
(0,5), (2,4), (4,1), (6,6), (8,7)
Solution: after subs, we get:
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING VS. INTERPOLATION

Consider the data:
 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING VS. INTERPOLATION

Using Lagrange interpolation, the Polynomial
is forced to intercept every point.
Interpolated function is overfit to the data.

Alternative: Least square curve fitting

 Numerical Analysis Course/Academia
Dr. Khaled M. Ahmida

CURVE FITTING VS. INTERPOLATION

Quadratic polynomial is used.

p ( x)  b0  b1 x  b2 x 2

Objective: is to minimize the following

polynomial and find the constants.
10

 k k
p ( x )  y 2

k 1

p ( x)  0.856 x 2  4.277 x  6.367