0 views

Uploaded by duyvu

lecture

- Econometrics
- Generalized Weibull Distributions
- Instructors Manual (Principles of Econometrics).pdf
- chap19
- Statistics for Business and Economics: bab 14
- qt presantation.pptx
- Time Series Analysis Auto Saved)
- action figure bungee jump 1213
- A Project Paper on Relationship Between Hourly Compensation In
- Sprexpnntl Trnds Info Tech
- Excel Forecasting With Excel
- Loaiza Et Al, 2011. Which Statistics Should Tropical Biologists Learn
- Lec 11
- portfolio
- 20 MASTER Exercise answers MBA.docx
- SYSID09
- Factors Affecting Project Performance in the Agriculture and Social Sectors: A Review of Postevaluation Reports Between 1991 and 1997
- Critical Reviews 32
- SYDE214 OUTLINE F07
- Lily Standards

You are on page 1of 8

Slide 1 of 5

Lecture 11b

Polynomial Regression

Brian G. Higgins

Department of Chemical Engineering & Materials Science

University of California, Davis

April 2014, Hanoi, Vietnam

ECM6Lecture11bVietnam_2014.nb

Background

We will now extend the ideas from Lecture 11a to find the best fit in a least squares sense for an

arbitrary polynomial:

P HxL = a0 + a1 x + a2 x2 + a3 x3 + + an xn

As before we will be given a set of M data points in the form

and our objective is to find the parameters ak that minimize the least square error. Thus

M

k=1

We know from calculus that to minimize the least square error, the desired values of ai must satisfy

E2 HLL

ai

= 0 , i = 0, 1, , n

Note that since the xk ' s and yk ' s are constant. Thus we have n+1 linear equations for the n+1 parameters ak

ECM6Lecture11bVietnam_2014.nb

Example 1

Suppose we have the following data points

881, 5.12<, 83, 3<, 86, 2.48<, 89, 2.34<, 815, 2.18<<

and we want to find the parameters ak that give the least square error for a guess function

PHxL = a0 + a1 x + a2 x 2 + a3 x 3

We proceed as before and define the following quantities

data3 = 881, 5.12<, 83, 3<, 86, 2.48<, 89, 2.34<, 815, 2.18<<;

x@i_D := data3@@i, 1DD; y@i_D := data3@@i, 2DD; M = Length@data3D;

P@x_D := a0 + a1 x + a2 x2 + a3 x3

M

i=1

Evaluating the least squares function gives

E2 @P, MD

H- 5.12 + a0 + a1 + a2 + a3 L2 + H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L2 + H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L2 +

H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L2 + H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L2

To take the derivative of the sum of squares with respect to a parameter we proceed as follows

D@E2 @P, MD, a0 D

2 H- 5.12 + a0 + a1 + a2 + a3 L + 2 H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L + 2 H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L +

2 H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L + 2 H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L

Thus it is a simple matter to generate the various equations required to determine the parameters ai :

ECM6Lecture11bVietnam_2014.nb

eqns =

8D@E2 @P, MD, a0 D 0, D@E2 @P, MD, a1 D 0, D@E2 @P, MD, a2 D 0, D@E2 @P, MD, a3 D 0<

82 H- 5.12 + a0 + a1 + a2 + a3 L +

2 H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L + 2 H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L +

2 H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L + 2 H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L 0,

2 H- 5.12 + a0 + a1 + a2 + a3 L + 6 H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L +

12 H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L + 18 H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L +

30 H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L 0,

2 H- 5.12 + a0 + a1 + a2 + a3 L + 18 H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L +

72 H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L + 162 H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L +

450 H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L 0,

2 H- 5.12 + a0 + a1 + a2 + a3 L + 54 H- 3 + a0 + 3 a1 + 9 a2 + 27 a3 L +

432 H- 2.48 + a0 + 6 a1 + 36 a2 + 216 a3 L + 1458 H- 2.34 + a0 + 9 a1 + 81 a2 + 729 a3 L +

6750 H- 2.18 + a0 + 15 a1 + 225 a2 + 3375 a3 L 0<

As before we use Solve to find the values of the parameters

sol3 = Flatten@Solve@eqnsDD

8a0 6.41245, a1 - 1.55201, a2 0.181872, a3 - 0.00648415<

The least square error is

E2 @P, MD . sol3

0.120889

Let us compare our previous results with Mathematica's Fit function

cubicFit = FitAdata3, 91, x, x2 , x3 =, xE

6.41245- 1.55201 x + 0.181872 x2 - 0.00648415 x3

Now let us visualize the fit graphically

plt1 = Plot@P@xD .sol3, 8x, 0, 15<, PlotStyle 8Blue, Thick<, Frame True,

FrameLabel 8Style@"x", 16D, Style@"PHxL", 16D<, PlotRange AllD;

plt2 = ListPlot@data3, PlotStyle 8PointSize@LargeD, Red<D;

Show@plt1, plt2, PlotRange AllD

6

PHxL

ECM6Lecture11bVietnam_2014.nb

10

12

14

x

It is apparent from the plots that a cubic polynomial is not an exceptionally good fit visually. How can we

evaluate the goodness of the fit quantitatively. We leave this question for later.

Note as the order of the polynomial increases, so does its ability to oscillate.

ECM6Lecture11bVietnam_2014.nb

Example 2

Problem Statement

Consider the following set of {x,y} data

882, 0.75<, 84, 0.1875<, 85, 0.1200<, 86, 0.0833<, 88, 0.0469<<

Determine the parameter values c and n in the model

y = c xn

that gives the best fit of the data in a least squares sense.

By taking the natural log of the model we get

Y=m+nX

where the following definitions are used

To proceed we need to transform the data

data5 = 882., 0.75<, 84., 0.1875<, 85., 0.1200<, 86., 0.0833<, 88., 0.0469<<;

data5b = data5 . 8x_, y_< 8Log@xD, Log@yD<

880.693147, - 0.287682<, 81.38629, - 1.67398<,

81.60944, - 2.12026<, 81.79176, - 2.48531<, 82.07944, - 3.05974<<

As before we define functions that extract out the X and Y data , and define our model and the least

squares function

X@i_D := data5b@@i, 1DD; Y@i_D := data5b@@i, 2DD; M = Length@data5bD;

Pow@X_D := m + n X

M

i=1

It is a simple matter to determine the equations for the parameters m and n:

ECM6Lecture11bVietnam_2014.nb

82 H0.287682 + m + 0.693147 nL + 2 H1.67398 + m + 1.38629 nL +

2 H2.12026 + m + 1.60944 nL + 2 H2.48531 + m + 1.79176 nL + 2 H3.05974 + m + 2.07944 nL 0,

1.38629 H0.287682 + m + 0.693147 nL + 2.77259 H1.67398 + m + 1.38629 nL +

3.21888 H2.12026 + m + 1.60944 nL + 3.58352 H2.48531 + m + 1.79176 nL +

4.15888 H3.05974 + m + 2.07944 nL 0<

sol = Flatten@Solve@eqnsDD

8m 1.09838, n - 1.99983<

Finally we need to transform this parameters back to the original definitions

regressParam = c m . sol

c 2.99929

y = 2.2

Note instead of using the loge to transform our power function into a linear model, we could have used

log10 or for that matter log to any base.

Here is the plot in terms of the transformed variables (X, Y)

plt1 = Plot@Hm + n XL . sol, 8X, 0.5, 2.6<, PlotStyle 8Blue, Thick<, Frame True,

FrameLabel 8Style@"X", 16D, Style@"Y=PHXL", 16D<, PlotRange AllD;

plt2 = ListPlot@data5b, PlotStyle 8PointSize@LargeD, Red<D;

Show@plt1, plt2, PlotRange AllD

0

Y=PHXL

-1

-2

-3

-4

0.5

1.0

1.5

2.0

X

Here is the plot in terms of the original variables (x, y)

2.5

ECM6Lecture11bVietnam_2014.nb

plt1 = PlotA3 x-2 , 8x, 2, 10<, PlotStyle 8Blue, Thick<, Frame True,

FrameLabel 8Style@"x", 16D, Style@"PHxL", 16D<, PlotRange AllE;

plt2 = ListPlot@data5, PlotStyle 8PointSize@LargeD, Red<D;

Show@plt1, plt2, PlotRange AllD

0.7

0.6

0.5

PHxL

0.4

0.3

0.2

0.1

0.0

2

10

References

These notes and the examples were adapted from the follow texts:

M. J. Maron, Numerical Analysis. A Practical Approach, 2nd Edition, Macmillan Publishing Company,

1987

A. J. Pettoprezzo, Introductory Numerical Analysis, Dover Publications, 1984

- EconometricsUploaded byRjnbk
- Generalized Weibull DistributionsUploaded byChaparrón Bonaparte
- Instructors Manual (Principles of Econometrics).pdfUploaded byManuel Alberto Pérez Pérez
- chap19Uploaded bykishoreparasa
- Statistics for Business and Economics: bab 14Uploaded bybalo
- qt presantation.pptxUploaded byAkash Patel
- Time Series Analysis Auto Saved)Uploaded bymussaiyib
- action figure bungee jump 1213Uploaded byapi-211386895
- A Project Paper on Relationship Between Hourly Compensation InUploaded bySk.Ashiquer Rahman
- Sprexpnntl Trnds Info TechUploaded byiweinel
- Excel Forecasting With ExcelUploaded bydeltahawk61
- Loaiza Et Al, 2011. Which Statistics Should Tropical Biologists LearnUploaded byLuis Francisco Solis
- Lec 11Uploaded byJoy AJ
- portfolioUploaded byapi-294742482
- 20 MASTER Exercise answers MBA.docxUploaded bySimbarashe Stewart Kamwendo
- SYSID09Uploaded byAndrea Colaço
- Factors Affecting Project Performance in the Agriculture and Social Sectors: A Review of Postevaluation Reports Between 1991 and 1997Uploaded byIndependent Evaluation at Asian Development Bank
- Critical Reviews 32Uploaded byNadia Jimena C A
- SYDE214 OUTLINE F07Uploaded bycartoon_nate
- Lily StandardsUploaded bylilyvraa1
- Arg02_Quantitative Tools and ExcelUploaded byAndrea Franzini
- AISUploaded bybabehoney
- OR_LeastSquare_Group7.docxUploaded bySupragya Saurabh
- Analysis.rtfUploaded byArslan Azhar
- MRAUploaded bymathworld_0204
- Matlab ScriptsUploaded bymahmoud
- Basic Elements for Understanding Regression AnalysisUploaded byJhabindra Pokharel
- 36110Uploaded byr cx
- NeighborhoodsUploaded byAdminAli
- 35853-130063-1-PBUploaded byGolamRabbane

- ECM6Lecture14cVietnam_2014.npdfUploaded byduyvu
- ECM6Lecture14bVietnam_2014Uploaded byduyvu
- ECM6Lecture14aVietnam_2014Uploaded byduyvu
- ECM6Lecture11dVietnam_2014Uploaded byduyvu
- ECM6Lecture11cVietnam_2014Uploaded byduyvu
- ECM6Lecture11aVietnam_2014Uploaded byduyvu
- ECM6Lecture9Vietnam_2014Uploaded byduyvu
- ECM6Lecture8Vietnam_2014Uploaded byduyvu
- ECM6Lecture6Vietnam_2014Uploaded byduyvu
- ECM6Lecture5Vietnam_2014Uploaded byduyvu
- ECM6Lecture4Vitenam_2014Uploaded byduyvu
- ECM6Lecture2Vietnam_2014Uploaded byduyvu
- ECM6Lecture3Vietnam_2014Uploaded byduyvu

- skenario 1,2,3,4Uploaded bysepta afifin
- interactional evidentialsUploaded byIrene Vázquez Espinoza
- Project Paper Proposal - FinalUploaded byJessie Radaza Tutor
- Using the Template for a Course Report-EnglishUploaded byBashar Saleh
- VP Director HR In Greater New York City Resume Debbie RyanUploaded byDebbie Ryan
- “A STUDY ON FUNCTIONING OF RUBBER BOARD IN KERALA WITH REFERENCE TO KANNUR DISTRICT”Uploaded byJeffin Thomas
- NEGOTIATION, COMMUNICATION, AND DECISION STRATEGIES.pdfUploaded byruiznelson
- Logistic SPSSUploaded bytskrishna7
- 06013134Uploaded byamit_saxena_10
- SturgesUploaded byAris Viveros
- Arrays Tables and SetUploaded byRitwik Kumar
- Pribluda_Improving the Quality of UterotonicsUploaded byDhaka2012
- A Comprehensive Approach to Reformer Tube InspectionUploaded byHéctor S. Rodríguez
- Pawelski and TeutschUploaded byVanganuru Sunil
- g7fcat2SEUploaded byAnonymous kNyVDtnx
- Media Planning Class PptUploaded byTimothy Foreman
- Mp Ta Discom ShgUploaded byAmit Kumar Singh
- Role of PowerUploaded byTarun Joshi
- The Pullout Performance of Square and Dsr Screw ThreadsUploaded byInternational Journal of Research in Engineering and Technology
- When to Use Nonlinear Finite Element AnalysisUploaded bymailtokasaravanan4100
- Positivist vs InterpretivistUploaded byNiki K Shilek
- Total Station Setup and OperationUploaded byNugroho Purwono
- Information Commissioner's 2008-09 report cardsUploaded byThe Globe and Mail
- MSc Thesis - Harshal v Patil - RepositoryUploaded bygeorgadam1983
- Employee Cynicism and Resistance to Organisational ChangeUploaded byWessam Hashem
- Knowledge of Malaria Among Albanian TravellersUploaded byDr Saad Masood Butt
- lkjhUploaded bySergiu Gorgos
- Summary Diffusion of Innovation TheoryUploaded byAena Cho
- Stat 1 - Elementary StatisticsUploaded byCristyflor Escordial
- Incisal Coverage or Not in Ceramic Laminate Veneers_ a Systematic Review and Meta-AnalysisUploaded byhanan omar