2/15/16

REGRESSION AND INTERPOLATION

Lec. 6.1: Introduction
Dr. Niket Kaisare
Department of Chemical Engineering
IIT–Madras

NPTEL Course: MATLAB Programming for Numerical Computations — Week-6

Example: Regression
• Given the following data:

x 0.8 1.4 2.7 3.8 4.8 4.9

y 0.69 1.00 2.02 2.39 2.34 2.83

Regression:
Obtain a straight line that
best fits the data

1
 2/15/16

Example: Interpolation
• Given the following data:

x 0.8 1.4 2.7 3.8 4.8 4.9

y 0.69 1.00 2.02 2.39 2.34 2.83

Interpolation:
“Join the dots” and
find a curve passing
through the data.

Regression vs. Interpolation

• Given the following data:

x 0.8 1.4 2.7 3.8 4.8 4.9

y 0.69 1.00 2.02 2.39 2.34 2.83

• In regression, we are interested in fitting a chosen function to data

y = 0.45 + 0.47x

• In interpolation, given finite amount of data, we are interested in

obtaining new data-points within this range.
At x = 2.0, y = 1.87

2
 2/15/16

What Comes Next

• This lecture (for demo):

• Linear Regression: Fit a straight line to the given data

• Newton’s Interpolation: For values at intermediate points

• L–6.2: “Curve fitting” in multiple parameters & lsqcurvefit

• L–6.3: “Parameter Estimation” (using these concepts)

• L–6.4: Interpolation (using spline and pchip)

Linear Least Squares

• Fit a straight line ! = #$ + #& ' to the data:

x 0.8 1.4 2.7 3.8 4.8 4.9

y 0.69 1.00 2.02 2.39 2.34 2.83

• Parameters #$ and #& satisfy the following equations:

(Computational Techniques, Module 5: http://nptel.ac.in/courses/103106074/15)

#$( + #& * '+ = * !+ ( * '+ * !+

+ #$ +
#& =
+ +

#$ * '+ + #& * '+, = * '+ !+ * '+ * '+, * '+ !+

+ + + + + +

3
 2/15/16

Newton’s Divided Difference Formula

x y D D2
0.8 0.69 !, − !& -, − -&
', − '& '/ − '&
1.4 1.00
!/ − !, -/ − -,
2.7 2.02 '/ − ', '0 − ',
3.8 2.39 ⋮
⋮ -12& − -12,
4.8 2.34
!1 − !12& '1 − '12,
4.9 2.83 '1 − '12&

End of Lecture 6.1

4
 2/15/16

REGRESSION AND INTERPOLATION

Lec. 6.2: Linear Least Squares Regression
Dr. Niket Kaisare
Department of Chemical Engineering
IIT–Madras

NPTEL Course: MATLAB Programming for Numerical Computations — Week-6

Linear Regression for Multiple Parameters

• Data: '& ,5 &, 6& ; !& , ' , ,5 ,, 6, ; !, , ⋯ , ' 9 ,5 9 ,69 ; !9
• Model to fit: ! = #$ + #& ' + #, 5 + #/ 6

⎡1 x1 u1 w1 ⎤ ⎡a0 ⎤ ⎡ y1 ⎤
⎢1 x
⎢
⎢% %
2 u2 w2 ⎥ ⎢ a1 ⎥ ⎢ y2 ⎥
%
⎥⎢ ⎥ = ⎢ ⎥
% ⎥ ⎢ a2 ⎥ ⎢ % ⎥
(
Φ = XT X )−1 X T Y
Least Squares Solution
⎢1 x u N wN ⎥⎦ ⎢⎣ a3 ⎥⎦ ⎢⎣ y N ⎥⎦
⎣$&& N&# &&& " ! $#"
X Φ Y

(Computational Techniques: Module-5 Part-2: http://nptel.ac.in/courses/103106074/16)

5
 2/15/16

Example

x 0.8 1.4 2.7 3.8 4.8 4.9

y 0.69 1.00 2.02 2.39 2.34 2.83

Using MATLAB lsqcurvefit

• Standard syntax:
phi=lsqcurvefit(@(p,xData) fName(p,xData),p0,xData,yData);
• phi parameter vector
• p0 vector of initial guesses
• xData, yData data arrays with ( rows
• fName provides !:;<=> = ? '; Φ
• lsqcurvefit minimizes the error between !<ABA and !:;<=>

6
 2/15/16

End of Lecture 6.2

REGRESSION AND INTERPOLATION

Lec. 6.3: Functional and Nonlinear Regression
Dr. Niket Kaisare
Department of Chemical Engineering
IIT–Madras

NPTEL Course: MATLAB Programming for Numerical Computations — Week-6

7
 2/15/16

Example: Reaction Rate

• Arrhenius model for reaction rate:
C = D$ E 2F/HI J 1
• We will solve it in two ways:

1. Linear least squares regression taking logarithm

M 1
ln C = ln D$ + − + S ln J
⏟
N P T
R

2. Using MATLAB function lsqnonlin

Parameter Estimation using Matrix Method (OLS)

Reaction rate (in mol/(l.s)) for various C and T values
400 K 450 K 500 K 550 K 600 K
1 mol/l 1.48 1.67 1.86 1.96 2.16
2 mol/l 2.35 2.79 3.07 3.37 3.62
3 mol/l 3.28 3.78 4.24 4.48 5.00
4 mol/l 4.12 4.64 5.15 5.76 6.08

C = D $E 2F/HI J 1 log ! = #$ + #& ' + #, 5

ln C ln D $ 1/P ln J

8
 2/15/16

Using MATLAB lsqnonlin

• Standard syntax:
phi=lsqnonlin(@(p) fName(p),p0);
• phi parameter vector
• p0 vector of initial guesses

• fName provides vector of errors, E+ = !+ − ? ' + ;Φ

• lsqcurvefit minimizes sum of square errors

End of Lecture 6.3

9
 2/15/16

REGRESSION AND INTERPOLATION

Lec. 6.4: Interpolation Options in MATLAB
Dr. Niket Kaisare
Department of Chemical Engineering
IIT–Madras

NPTEL Course: MATLAB Programming for Numerical Computations — Week-6

Interpolation in MATLAB

• Most popular interpolation techniques:

• spline Cubic spline interpolation
• pchip Piecewise Cubic Hermite Polynomial

• Syntax:
yInterpolated = spline(xData,yData,xval);
• xData,yData S×1 data vectors
• yInterpolated values interpolated at xVal (can be vectors)

10
 2/15/16

Example: Temperature variation in a day

time 00 01 02 03 04 05 06 07 08 09 10 11 12
T 25.6 25.4 25.1 24.9 24.9 25.2 25.9 26.3 27.1 29.3 30.8 31.2 32.1
time 13 14 15 16 17 18 19 20 21 22 23 24
T 31.0 30.3 31.4 30.6 31.8 29.6 28.4 28.1 28.2 27.4 26.8 26.1

We are interested in finding temperature at various times during the day,

in addition to the ones where data is available.

We interpolate or “fill in” the missing data

Example: Vehicle speed in front of Govt. Hospital

time (s) 0 10 20 30 40 50 60 70 80 90
speed 45 32 0 0 7 12 20 15 29 55

11
 2/15/16

End of Lecture 6.4

12