Polynomial Regression
● Some engineering data is poorly represented by a
straight line.
● For these cases a curve is better suited to fit the data.
● The least squares method can readily be extended to fit
the data to higher order polynomials.
Polynomial Regression (cont’d)
A parabola is preferable
Polynomial Regression (cont’d)
• A 2nd order polynomial (quadratic) is defined by:
yo
a a
1xa
2x
e 2
● The residuals between the model and the data:
ii i
2
e y a
o a
1x ax 2i
● The sum of squares of the residual:
S
r
e
yi
a
oa
2
x
i
1ia
x 2
2
i
2
Polynomial Regression (cont’d)
Sr
2(yi aoa1xi a2xi2)0
ao
Sr
2(yi aoa1xi a2xi2)xi 0
a1
Sr
2(yi aoa1xi a2xi )xi 0
2 2
a2
yi
naoa
1x
ia
22
x
i 3 linear equations
with 3 unknowns
x y
i i
ao x
ia
1
2
x
i
a2
3
x
i (ao,a1,a2), can be
solved
xy
a2
x
a
i ix
a
x
o
2
i 1
3
i 2
4
i
Polynomial Regression (cont’d)
● A system of 3x3 equations needs to be solved to
determine the coefficients of the polynomial.
n i
x a
i
2
x
0 yi
i
x 2
x
i
3
x
i
a
1
xi
iy
2
2
x
i
i
3
x 4
x
i
a
2 x
iyi
● The standard error & the coefficient of determination
Sr St Sr
sy/x r
2
n3 St
Polynomial Regression (cont’d)
General:
The mth-order polynomial:
yaax
o 1a
x
2
2
m
m
.....
ax e
● A system of (m+1)x(m+1) linear equations must be solved for
determining the coefficients of the mth-order polynomial.
● The standard error:
S
sy/x r
nm
1
St Sr
● The coefficient of determination: r
2
St
Polynomial Regression- Example
Fit a second order polynomial to data:
xi yi xi2 xi3 xi4 xiyi xi2yi x 15 i
0 2.1 0 0 0 0 0
1 7.7 1 1 1 7.7 7.7
y 152
xi3 225
.6
i
2 13.6 4 8 16 27.2 54.4 i 55
x2
3 27.2 9 27 81 81.6 244.8
4 40.9 16 64 256 163.6 654.4
5 61.1 25 125 625 305.5 1527.5
15 152.6 55 225 979 585.6 2489 i 979
x4
xy 585
.6
i i
x y2488
2
.8
i i
Polynomial Regression- Example
(cont’d)
● The system of simultaneous linear equations:
6 1555
a0152
.6
1555
225
a1585
.6
55225
2
979
a 2488
.
8
a
02. 1
47857
,
a 2.
35929
,
a21.
86071
y2
.
2
.
47857 2
35929
x1
.
86071
x
2
ti r e
39S i
3.74657
2
S y y 2513
.
Polynomial Regression- Example
(cont’d)
xi yi ymodel ei2 (yi-y`)2
0 2.1 2.4786 0.14332 544.42889
1 7.7 6.6986 1.00286 314.45929
2 13.6 14.64 1.08158 140.01989
3 27.2 26.303 0.80491 3.12229
4 40.9 41.687 0.61951 239.22809
5 61.1 60.793 0.09439 1272.13489
15 152.6 3.74657 2513.39333
•The standard error of estimate:
3.74657
s/x
1.12
y
63
•The coefficient of determination:
2513
.
393
.
74657
2
r
0
.
99851
,rr
2
0
.999
2513
.
39