Linear Regression - Topics

     

Basics of Linear Regression Variation in Linear Regression Linear Regression Analysis Goodness of Fit Standard Error terms for Linear Regression Hypothesis testing

Regression - Types

Linear Regression


A statistical technique that uses a single, independent variable (X) to estimate a single dependent variable (Y). Based on the equation for a line:
Y = b + mX

Linear Regression - Model

Y
? (the actual value of Yi)

Yi

 Y ! FF X  Ii

Xi

Linear Regression - Model

Yi

F

F

Ii

Population

Regression Coefficients for a . . .

Y = b0 + b1Xi + e Y = b0 + b1Xi

Sample

ANOVA
3-hour 70 77 76 80 84 78 1-day 61 75 67 63 66 68 10-week 82 88 90 96 92 98

D - Variati
SST is a measure f the t tal variati f bservati s. A measure f the differe ces i bservati s.

SST
Due t treatme ts.

SST SSE SST = SST + SSE

and m/unexplained.

Linear Regression - Variation

Ice Cream Example
X Temperature 63 70 73 75 80 82 85 88 90 91 92 75 98 100 92 87 84 88 80 82 76 Y Sales 1.52 1.68 1.8 2.05 2.36 2.25 2.68 2.9 3.14 3.06 3.24 1.92 3.4 3.28 3.17 2.83 2.58 2.86 2.26 2.14 1.98

Ice Cream Sales

4 3.5 3 2.5 2 1.5 1 0.5 0 0 20 40 60 80 100 120

= 2.53

Linear egressi n - Variati n

Ice Cream Example
X Temperature 63 70 73 75 80 82 85 88 90 91 92 75 98 100 92 87 84 88 80 82 76 Y Sales 1 52 1 68 18 2 05 2 36 2 25 2 68 29 3 14 3 06 3 24 1 92 34 3 28 3 17 2 83 2 58 2 86 2 26 2 14 1 98

Ice
4 35 3 25 2 15 1 05 0 0 20 40

eam Sa es

60

80

100

120

Y ! b0  b1 X !

Sample Regression Line

Linear egressi n - Variati n

SS
ue t regressi n.

SST SSE
and m/unexplained.

SST = SS + SSE

Linear Regression - Variation

Y _
SST = (Yi - Y)2 _ SSR = (Yi - Y)2 SSE =(Yi - Yi )2

_ Y X

Xi

Determining the Regression Line/Model


T m r S

   

Select Tools, Data Analysis, Regression Provide the X range Provide the Y range Output the analysis to a new sheet

Manual Calculations

                         

                       

Use Excel (or any other popular statistical software)

X r

Determining the Regression Line/Model using Excel

SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations ANOVA df Regression Residual Total 1 19 20 SS MS F 6.35405596 6.354056 297.6496823 0.405601183 0.021347 6.759657143 P-value 5.7673E-08 4.58812E-13 0.969534312 0.939996782 0.936838718 0.1461076 21

Intercept X Variable 1

Coefficients Standard Error t Stat -2.534985905 0.295223266 -8.58667 0.060727986 0.003519947 17.25253

Determining the Regression Line/Model Manual Calculations

2 SSE =(Yi - Yi ) _2 SSR = (Yi - Y) SST = (Yi - Y)2

_ SSx =(Xi - X )2 _ 2 SSy =(Yi - Y)_

MSE = SSE / df MSR = SSR / df

b1=SSxy/SSx

SSxy =(Xi - X )(Xi - Y )

b0 = Y

b1X

R2 = SSR/SST
SYX ! SSE n-2

t-test = b1 / Sb1

Measures of Model Goodness

1. 2. 3. 4.

R2 Coefficient of Determination F-test > F-crit or p-value less than alpha Standard Error t-test

Hypothesis testing for F



 

Testing to see if the linear relationship between X and Y is significant at the population level. t-test Follow the 5-step process


H 0: H A: t-crit, alpha or alpha/2, n-2 df

Standard Error Terms in Linear Regression



Se (standard error of the estimate)

A measure of variation around the regression line
If the Se is small Standard deviation Of the Errors

Sb1 (standard error of the the sampling distribution of b )

1

Standard deviation of the slopes A measure of the variation of the slopes from different samples If the Sb1 is small our b1 estimate is probably very accurate Estimates of

b1 b1 b1

Linear Regression Example

  

Petfood, Estimate Sales based on Shelf Space Two sets of samples, 12 observations each Perform a Regression Analysis on both sets of data
Space 5 5 5 10 10 10 15 15 15 20 20 20 Sales 1. 2.2 1. 1. 2. 2. 2. 2. 2. 2. 2. 3.1 Space 5 5 5 10 10 10 15 15 15 20 20 20 Sales 1. 1. 1. 2 2. 2. 3.5 3.2 3.3 4.2 4. 4.5

Sample1

Sample2