1 views

Uploaded by Cesily

Stat2

- Surveying by Dr. Ramachandra
- Tutorial Lab Fit
- linear correlation and regression
- 线性回归分析 linear regression.ppt
- Introduction to Coordinate Systems and Projections - Instructional Guide.pdf
- Disse 2
- Aboveground and Belowground Biomass Allocation in Native Prosopis Caldenia Burkart Secondaries Woodlands
- Statistik-06
- R09 Correlation and Regression
- ch11
- 00-Perhitungan Regresi Sederhana Utk Uji Linearitas
- Introduction to Survey Weights Pri Version
- 9_Ashwani_Power_System_State_Estimation.pdf
- Final Exam Review Session Solutions - Stat 101 - Fall 2013
- Chapter 2
- Trends, RB-k Assign 8,9
- regression-converted.pdf
- JESD
- Mukhyi, Nurul & Dwi Asih
- 2566-5090-1-SM

You are on page 1of 26

University of Canterbury, Christchurch, New Zealand

Semester 2, 2016

Statistical Learning, with applications in R (Springer, 2013) with permission

from the authors: G. James, D. Witten, T. Hastie and R. Tibshirani.

,1 / 26

Linear regression

supervised learning that assumes there is an approximately

linear relationship between the predictors X1 , X2 , . . . , Xp and

the response Y .

regression is an extremely useful and widely used method.

,2 / 26

50

100

200

300

25

5

10

15

Sales

20

25

20

15

Sales

10

15

10

5

Sales

20

25

10

TV

20

30

40

50

Radio

20

40

60

80

100

Newspaper

,3 / 26

Y = 0 + 1 X + ,

where 0 and 1 are two unknown parameters and is an

error term with E () = 0.

Given some parameter estimates 0 and 1 , the prediction of

Y at X = x is given by

y = 0 + 1 x.

,4 / 26

Let yi = 0 + 1 xi be the prediction of Y at X = xi , the

predictor value at the ith training observation. Then, the ith

residual is defined as

ei = yi yi ,

where yi is the response value at the ith training observation.

The least squares approach chooses 0 and 1 to minimize

the residual sum of squares (RSS)

RSS =

n

X

i=1

ei2

n

n

X

X

2

=

(yi yi ) =

(yi 0 1 xi )2 .

i=1

i=1

,5 / 26

15

10

5

Sales

20

25

Advertising example

50

100

150

200

250

300

TV

y = 7.03 + 0.0475x

,6 / 26

Advertising example

3

0.05

2.15

0.04

0.06

2.5

2.2

0.03

2.3

3

5

3

6

the predictor.

B. Robertson, University of Canterbury

,7 / 26

Pn

(x x)(yi y)

Pn i

1 = i=1

)2

i=1 (xi x

and

0 = y 1 x,

where x and y are the sample means of x and y , respectively.

,8 / 26

10

5

10

Y

10

10

B. Robertson, University of Canterbury

,9 / 26

The standard errors for the parameter estimates are

s

2

1

x

SE(0 ) =

+ Pn

n

)2

i=1 (xi x

and

,

)2

i=1 (xi x

SE(1 ) = pPn

where =

p

V ().

using the residual standard error (RSE)

sP

n

i )2

i=1 (yi y

RSE =

.

np1

B. Robertson, University of Canterbury

,10 / 26

Hypothesis testing

If 1 = 0, then the simple linear model reduces to Y = 0 + ,

and X is not associated with Y .

To test whether X is associated with Y , we perform a

hypothesis test:

H0 : 1 = 0 (there is no relationship between X and Y )

HA : 1 6= 0 (there is some relationship between X and Y )

If the null hypothesis is true (1 = 0), then

t=

1 0

SE(1 )

B. Robertson, University of Canterbury

,11 / 26

Intercept

TV

Coefficient

7.0325

0.0475

Std. Error

0.4578

0.0027

t-statistic

15.36

17.67

p-value

<0.0001

<0.0001

,12 / 26

Once we have established that there is some relationship

between X and Y , we want to quantify the extent to which

the linear model fits the data.

The residual standard error (RSE) provides an absolute

measure of lack of fit for the linear model, but it is not always

clear what a good RSE is.

An alternative measure of fit is R-squared (R 2 ),

Pn

(yi yi )2

RSS

2

R =1

= 1 Pi=1

,

n

TSS

)2

i=1 (yi y

where TSS is the total sum of squares.

,13 / 26

Quantity

Residual standard error (RSE)

R2

Value

3.26

0.612

because it always lies between 0 and 1.

A good R 2 value usually depends on the application.

,14 / 26

Y = 0 + 1 X1 + . . . + p Xp + ,

where 0 , 1 , . . . , p are p + 1 unknown parameters and is

an error term with E () = 0.

Given some parameter estimates 0 , 1 , . . . , p , the prediction

of Y at X = x is given by

y = 0 + 1 x1 + . . . + p xp .

,15 / 26

Y

X2

y = 0 + 1 x1 + 2 x2 .

B. Robertson, University of Canterbury

X1

,16 / 26

The parameters 0 , 1 , . . . , p are estimated using the least

squares approach.

We choose 0 , 1 , . . . , p to minimize the sum of squared

residuals

RRS =

n

X

(yi yi )2

i=1

n

X

i=1

,17 / 26

Intercept

TV

Radio

Newspaper

Coefficient

2.939

0.046

0.189

-0.001

Std. Error

0.3119

0.0014

0.0086

0.0059

t-statistic

9.42

32.81

21.89

-0.18

p-value

<0.0001

<0.0001

<0.0001

0.8599

,18 / 26

hypothesis test:

H0 : 1 = 2 = . . . = p = 0 (there is no relationship)

HA : at least one j is non-zero (there is some relationship)

If the null hypothesis is true (no relationship), then

F =

(TSS - RSS)/p

RSS/(n p 1)

,19 / 26

Once we have established that there is some relationship

between the reponse and the predictors, we want to quantify

the extent to which the multiple linear model fits the data.

The residual standard error (RSE) and R 2 are commonly used.

For the advertising data we have:

Quantity

Residual standard error (RSE)

R2

F-statistic

Value

1.69

0.897

570

,20 / 26

interaction effects.

Consider the standard linear model with two predictors

Y = 0 + 1 X1 + 2 X2 + .

the standard model

Y = 0 + 1 X1 + 2 X2 + 3 X1 X2 + .

,21 / 26

Sales = 0 + 1 Tv + 2 Radio + 3 (Tv Radio) + .

The results are:

Intercept

TV

Radio

TvRadio

Coefficient

6.7502

0.0191

0.0289

0.0011

Std. Error

0.248

0.002

0.009

0.000

t-statistic

27.23

12.70

3.24

20.73

p-value

<0.0001

<0.0001

0.0014

<0.0001

,22 / 26

polynomial regression.

Consider the simple linear model

Y = 0 + 1 X + .

of X in the model. For example, a quadratic model is

Y = 0 + 1 X + 2 X 2 + .

,23 / 26

50

30

20

10

40

Linear

Degree 2

Degree 5

50

100

150

200

Horsepower

,24 / 26

The figure suggests that

mpg = 0 + 1 Horsepower + 2 Horsepower2 + ,

may fit the data better than a simple linear model.

The results are:

Intercept

Horsepower

Horsepower2

Coefficient

56.9001

-0.4662

0.0012

Std. Error

1.8004

0.0311

0.0001

t-statistic

31.6

-15.0

10.1

p-value

<0.0001

<0.0001

<0.0001

,25 / 26

for linear regression (R does this automatically for us).

Deciding on important variables.

Outliers and high leverage points.

Non-constant variance and correlation of error terms.

Collinearity.

,26 / 26

- Surveying by Dr. RamachandraUploaded byvrsec-rao
- Tutorial Lab FitUploaded byOmer Altimimi Omer
- linear correlation and regressionUploaded byYilin Zhu
- 线性回归分析 linear regression.pptUploaded byvictor_chung_23
- Introduction to Coordinate Systems and Projections - Instructional Guide.pdfUploaded byAnonymous jvaG8m7
- Disse 2Uploaded byjairampatna
- Aboveground and Belowground Biomass Allocation in Native Prosopis Caldenia Burkart Secondaries WoodlandsUploaded bybriologo2
- Statistik-06Uploaded byNurh4y
- R09 Correlation and RegressionUploaded byTorakiSato
- ch11Uploaded byAídaMariana
- 00-Perhitungan Regresi Sederhana Utk Uji LinearitasUploaded byAgus Purwanto
- Introduction to Survey Weights Pri VersionUploaded byAnonymous PKVCsG
- 9_Ashwani_Power_System_State_Estimation.pdfUploaded bysf111
- Final Exam Review Session Solutions - Stat 101 - Fall 2013Uploaded byPi
- Chapter 2Uploaded byTeo Liang Wei
- Trends, RB-k Assign 8,9Uploaded bychristian
- regression-converted.pdfUploaded byTnt 1111
- JESDUploaded byIriNa En
- Mukhyi, Nurul & Dwi AsihUploaded bypatamxitin
- 2566-5090-1-SMUploaded byNadzar Canggih Hendro
- 1891654Uploaded byJawad Gujjar
- Corrosion&MaterialsUploaded bysaeid59
- Comparative study of ROC regression techniques—Applications for the computer-aided diagnostic system in breast cancer detectionUploaded byssignn
- 401-slrUploaded byPoonam Naidu
- LM11 Fixed (1)Uploaded byNikolay Griyseinko Kholavsky
- NmrUploaded byHardySyaSaputra
- 2012.Version1Uploaded byPi
- Econometrics Jan 2018Uploaded bybhaskkar
- 56_dorUploaded byAmy Gray
- A statistical method for assessing network stability using the Chow testUploaded byGiuseppe Marsico

- SQL Server Forensic Analysis Chapter SQL Server ForensicUploaded byAshar Fraz
- Hemorrhoids Pathophysiology to TreatmentUploaded byKen Cheung
- BAEN414-614-Syllabus-Fall-2012.pdfUploaded byHotib Perwira
- 07-12-30+Workshop+AttendanceUploaded byricky_soni32
- Business Ethics Ppt FinalUploaded bySreena Binu
- DA3976 CMT Quick Ref GuideUploaded bywood1024
- 07 July 1991Uploaded byMonitoring Times
- MHI - PRESENT STATUS AND FUTURE DEVELOPMENT OF DOUBLE HULL TANKERS.pdfUploaded byTermiteEnkay
- Fluidized BedUploaded byZahrotul Hayati
- c 630 – c 630m – 01 ;Qzyzmc9dnjmwts1sruqUploaded byHumberto Gutierrez
- PhD_Thesis_on_Big_Data_in_Official_Stati.pdfUploaded byJoel Yury Vargas Soto
- SMV_Validator-Manual_P1_1.pdfUploaded bynizam
- steve madden brand auditUploaded byapi-316514346
- lesson plan 1 (1)Uploaded byapi-287920249
- Forensic Audit 08012011Uploaded byPreeti Hooda
- Agreement for SaleUploaded bySougata Basu
- Solar Street Lighting 1Uploaded byDaniel Okere
- DxDiagUploaded byHambaTuhanYme
- Microstructural Characterization and Phase Transf.pdfUploaded byeng_diegorossetto
- 1971 Bennet,The Talofloc Decolorization ProcessUploaded bynghi
- Sledgehammer TrainingUploaded byxceelent
- BYU FSAE Race Car StudyUploaded bygosculptor
- Experiment No 1 MinhajUploaded byMinhaj Akbar
- Group a Tata CorusUploaded byChÄndra Sharma
- Maths FormulasUploaded bymukeshnt
- Arch Engle 1982Uploaded byFernando Manuel Castillo Mego
- SLLPUploaded byTarun
- Pomegranate de SeederUploaded byKamal Bharakhda
- BRVN_Cannes Lions 2014 Press Lions WinnersUploaded byBrands Vietnam
- IVF clinic ProductsUploaded byivfworld