23 views

Uploaded by kishorenayark

Inference In Regression

- AP Statistics Chapter 1-3 Outlines
- QNT 561, Week 5 Exercises, Chapter 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17,
- Supervision as a Determinant of Public Secondary School Teachers’ Effectiveness in Central Senatorial District of Delta State
- IJAIEM-2018-04-01-25.pdf
- Additive Solution
- 6 Urbanization of Karachi Shaheen
- 1259-3800-1-PB
- My Stata Command Details.pdf
- Output
- A PREDICTIVE MODEL FOR OZONE UPLIFTING IN OBSTRUCTION PRONE ENVIRONMENT
- SSRN-id2463999
- ACADEMIC ACHIEVEMENT AMONG UPPER PRIMARY SCHOOL STUDENTS IN RELATION TO THEIR MENTAL PRESSURE AFTER KEDARNATH DISASTER
- Regresi Neww
- advertisement.pdf
- Ujiolah Data
- Case of Study Effects of Organisational Culture on Employees Per
- Effects of Organisational Culture on Employees Per
- 1968_J.pdf
- ASSIM 5 new
- Notebook File Chapter 8

You are on page 1of 27

Yi = 0 + 1 Xi +

i

Yi value of the response variable in the i th trial 0 and 1 are parameters Xi is a known constant, the value of the predictor variable in the i th trial

i

iid N (0, 2 )

i = 1, . . . , n

Inference concerning 1

Tests concerning 1 (the slope) are often of interest, particularly H0 : 1 = 0 Ha : 1 = 0 the null hypothesis model Yi = 0 + (0)Xi +

i

implies that there is no relationship between Y and X. Note the means of all the Yi s are equal at all levels of Xi .

Elements of a statistical test

Null hypothesis, H0 Alternative hypothesis, Ha Test statistic Rejection region

Errors

A type I error is made if H0 is rejected when H0 is true. The probability of a type I error is denoted by . The value of is called the level of the test. A type II error is made if H0 is accepted when Ha is true. The probability of a type II error is denoted by .

P-value

The p-value, or attained signicance level, is the smallest level of signicance for which the observed data indicate that the null hypothesis should be rejected.

Null Hypothesis

If the null hypothesis is that 1 = 0 then b1 should fall in the range around zero. The further it is from 0 the less likely the null hypothesis is to hold.

If we nd that our estimated value of b1 deviates from 0 then we have to determine whether or not that deviation would be surprising given the model and the sampling distribution of the estimator. If it is suciently (where we dene what sucient is by a condence level) dierent then we reject the null hypothesis.

Only have a nite sample Dierent nite set of samples (from the same population / source) will (almost always) produce dierent point estimates of 0 and 1 (b0 , b1 ) given the same estimation procedure Key point: b0 and b1 are random variables whose sampling distributions can be statistically characterized Hypothesis tests about 0 and 1 can be constructed using these distributions. The same techniques for deriving the sampling distribution of b = [b0 , b1 ] are used in multiple regression.

Sampling Dist. Of b1

The point estimator for b1 is b1 = )(Yi Y ) (Xi X )2 (Xi X

The sampling distribution for b1 is the distribution of b1 that arises from the variability of b1 when the predictor variables Xi are held xed and the errors are repeatedly sampled Note that the sampling distribution we derive for b1 will be highly dependent on our modeling assumptions.

For a normal error regression model the sampling distribution of b1 is normal, with mean and variance given by E {b1 } = 1 2 {b1 } = 2 )2 (Xi X

First step

To show )(Yi Y ) = (Xi X we observe )(Yi Y ) = (Xi X = = = )Yi (Xi X )Yi Y (Xi X )Yi Y (Xi X )Yi (Xi X )Y (Xi X ) (Xi X n (Xi ) + Y Xi n )Yi (Xi X

This will be useful because the sampling distribution of the estimators will be expressed in terms of the distribution of the Yi s which are assumed to be equal to the regression function plus a random error term.

b1 as convex combination of Yi s

b1 can be expressed as a linear combination of the Yi s b1 = = = where ki = )(Yi Y ) (Xi X )2 (Xi X )Yi (Xi X from previous slide )2 (Xi X ki Yi ) (Xi X )2 (Xi X

Now the estimator is simply a convex combination of the Yi s which makes computing its analytic sampling distribution simple.

Properties of the ki s

It can be shown (using simple algebraic operations) that ki ki Xi = 0 = 1 1 )2 (Xi X

ki2 =

(possible homework). We will use these properties to prove various properties of the sampling distributions of b1 and b0 .

Reminder: useful fact:

A linear combination of independent normal random variables is normally distributed More formally: when Y1 , . . . , Yn are independent normal random variables, the linear combination a1 Y1 + a2 Y2 + . . . + an Yn is normally distributed, with mean ai E {Yi } and variance ai2 2 {Yi }

Since b1 is a linear combination of the Yi s and each Yi = 1 Xi + 0 + ei is (conditioned on Xi , 1 , and 0 ) an independent normal random variable, then then distribution of b1 under sampling of the errors is normal as well b1 = From previous slide E {b1 } = ki E {Yi }, 2 {b1 } = ki2 2 {Yi } ki Yi , ki = ) (Xi X )2 (Xi X

This means b1 N (E {b1 }, 2 {b1 }). To use this we must know E {b1 } and 2 {b1 }.

b1 is an unbiased estimator

This can be seen using two of the properties E {b1 } = E { = = = 0 = 1 So now we know the mean of the sampling distribution of b1 and conveniently (importantly) its centered on the true value of the unknown quantity 1 (the slope of the linear relationship). ki Yi } ki E {Yi } ki (0 + 1 Xi ) ki + 1 ki Xi

= 0 (0) + 1 (1)

Variance of b1

Since the Yi are independent random variables with variance 2 and the ki s are constants we get 2 {b1 } = 2 { = = 2 ki Yi } = ki2 2 = 2 1 )2 (Xi X ki2 2 {Yi } ki2

and now we know the variance of the sampling distribution of b1 . This means that we can write b1 N (1 , 2 )2 ) (Xi X

How does this behave as a function of 2 and the spread of the Xi s? Is this intuitive? Note: this assumes that we know 2 . Can we?

Estimated variance of b1

When we dont know 2 then one thing that we can do is to replace it with the MSE estimate of the same Let s 2 = MSE = where SSE = and i ei = Yi Y plugging in we get 2 {b1 } = s2 {b1 } = 2 )2 (Xi X s2 )2 (Xi X ei2 SSE n2

Recap

We now have an expression for the sampling distribution of b1 when 2 is known b1 N (1 , 2 )2 ) (Xi X (1)

When 2 is unknown we have an unbiased point estimator of 2 s2 s2 {b1 } = )2 (Xi X As n (i.e. the number of observations grows large) s2 {b1 } 2 {b1 } and we can use Eqn. 1. Questions

When is n big enough? What if n isnt big enough?

b1 is normally distributed so (b1 1 )/( standard normal variable. Why? 2 {b1 }) is a

We dont know 2 {b1 } because we dont know 2 so it must be estimated from data. We have already denoted its estimate s2 {b1 } If using the estimate s2 {b1 } we will show that b1 1 t (n 2) s {b1 } where s {b1 } = s2 {b1 }

For now we need to rely upon the following theorem:

Cochrans Theorem

For the normal error regression model SSE = 2 i )2 (Yi Y 2 (n 2) 2

and is independent of b0 and b1 Intuitively this follows the standard result for the sum of squared normal random variables Here there are two linear constraints imposed by the regression parameter estimation that each reduce the number of degrees of freedom by one. We will revisit this subject soon.

A denition: Let z and 2 ( ) be independent random variables (standard normal and 2 respectively). The following random variable is dened to be a t-distributed random variable: t ( ) = z

2 ( )

This version of the t distribution has one parameter, the degrees of freedom

b1 1 t (n 2) s {b1 } Is a so-called studentized statistic. To derive the distribution of this statistic, rst we do the following rewrite b1 1 b1 1 {b } = s {b 1} 1 s {b1 }

{b1 }

And note the following s2 {b1 } = 2 {b1 }

P MSE 2 (Xi X ) 2 P 2 (Xi X )

MSE SSE = 2 2 (n 2)

where we know (by the given simple version of Cochrans theorem) that the distribution of the last term is 2 and indep. of b1 and b0 2 (n 2) SSE 2 (n 2) n2

But by the given denition of the t distribution we have our result b1 1 t (n 2) s {b1 } because putting everything together we can see that b1 1 s {b1 } z

2 (n2) n2

Now that we know the sampling distribution of b1 (t with n-2 degrees of freedom) we can construct condence intervals and hypothesis tests easily Things to think about What does the t-distribution look like? Why is the estimator distributed according to a t-distribution rather than a normal distribution? When performing tests why does this matter? When is it safe to cheat and use a normal approximation?

- AP Statistics Chapter 1-3 OutlinesUploaded byaml97
- QNT 561, Week 5 Exercises, Chapter 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17,Uploaded byDanielle King
- Supervision as a Determinant of Public Secondary School Teachers’ Effectiveness in Central Senatorial District of Delta StateUploaded byIOSRjournal
- IJAIEM-2018-04-01-25.pdfUploaded byAnonymous vQrJlEN
- Additive SolutionUploaded bySwastik Mohapatra
- 6 Urbanization of Karachi ShaheenUploaded byMaira Khan
- 1259-3800-1-PBUploaded byAbdul Sattar Ansari
- My Stata Command Details.pdfUploaded bytaposh
- OutputUploaded byDewi Shintia HuLumudi
- A PREDICTIVE MODEL FOR OZONE UPLIFTING IN OBSTRUCTION PRONE ENVIRONMENTUploaded byIAEME Publication
- SSRN-id2463999Uploaded byrahman
- ACADEMIC ACHIEVEMENT AMONG UPPER PRIMARY SCHOOL STUDENTS IN RELATION TO THEIR MENTAL PRESSURE AFTER KEDARNATH DISASTERUploaded byAnonymous CwJeBCAXp
- Regresi NewwUploaded byYora Saki
- advertisement.pdfUploaded byArun Mandal
- Ujiolah DataUploaded byRecca Damayanti
- Case of Study Effects of Organisational Culture on Employees PerUploaded byMario Amza
- Effects of Organisational Culture on Employees PerUploaded bymindworkz pro
- 1968_J.pdfUploaded bySusan Li HB
- ASSIM 5 newUploaded bysaleh
- Notebook File Chapter 8Uploaded byamy12young
- Hasil Regresi Nisa2 100 MeanUploaded byKhoirunisa Arifah
- PRAKTIKUM KIMIA MEDISINAL.docxUploaded byPanser Biru
- Datascience InterviewUploaded bySenthil Kumar
- US Federal Trade Commission: wp290Uploaded byftc
- SPE 077533 (Archer) Effects Well Test Anal Pressure and Flowrate NoiseUploaded byBeshoy Faiz
- 2008 stat exam.pdfUploaded byElle Smart
- Chap10 ExcelUploaded byzahidshafiquebhatti
- Are Some Mutual Fund Managers Better than Others?Uploaded bySenchroma
- Stat a IntroUploaded bydontcare16355
- Various UnexpectedUploaded bywirodihardjo

- Understanding Machine Learning Theory AlgorithmsUploaded byHigor Eduardo
- JoeArmstrong_ErlangALanguageForProgrammingReliableSystemsUploaded bykishorenayark
- Goroutine Concurrency Scheduling Model Deep Analysis Handcuffs a Coroutine Pool - Pan Jianfeng's Programming NotesUploaded bykishorenayark
- Comparing Caffe, Torch, Theano and TensorFlowUploaded bykishorenayark
- Programming AkkaUploaded bykishorenayark
- Learning Bayesian Networks - Neapolitan R E - Allbookfree.tkUploaded bykool420
- CS231n Convolutional Neural Networks for Visual Recognition.pdfUploaded bykishorenayark
- Array Fire GPU programming in C++Uploaded bykishorenayark
- DataMining-ch3Uploaded bykishorenayark
- Ml AlgorithmsUploaded bykishorenayark
- Adaboost MatasUploaded bySơn Nguyễn Quý

- ProbabilityUploaded byjj012586
- Coefficient of VariationUploaded byCamilo Lillo
- AnovaUploaded byAriep Ilham
- Lecture 8Uploaded bypurething
- Statistical Analysis in JASP - A Students Guide v1.0Uploaded byTinoRepaso
- Gologit ModelsUploaded bySidra Nazir
- Properties of the Normal DistributionUploaded bylegendre.julie1840
- Survival AnalysisUploaded byFajar Jum'at
- Psy 335 Week 3 Quiz (All Correct)Uploaded byteacher.theacestud
- Contrast Sensitivity Measures by Two Diferent TestUploaded byCEB
- 531notesv2Uploaded bycoxshuler
- Stata ExcelUploaded bychompoonoot
- Credit ScoringUploaded bySomesh Pal
- UserManual WarpPLS V2 2Uploaded byrina andriani
- Running a T-test in ExcelUploaded byKim Soliman
- 106 學年第一學期 應用生物統計學 for DownloadUploaded by謝書豪
- Practical Bayesian Model Evaluation Using Leave-One-out Cross-Validation and WAIC - Vehtari Gelman Gabry 2016Uploaded byjohseb71
- Effect Size J. Pediatr. Psychol. 2009 Durlak 917 28Uploaded byAlina Ciabuca
- t -test.pptUploaded byDrRuchi Garg
- Quantitative Methods (Tutorials 1)Uploaded byBenneth Yankey
- Assignment 1Uploaded byAnindya Chakraborty
- How to forecast using ARIMAUploaded byNaba Kr Medhi
- biostatsUploaded byAamir Bugti
- Seminar 5 OutlineUploaded byjeremydb77
- vision dummy.pdfUploaded byDango Daikazoku
- tarea regresionUploaded byJosué Acosta
- What Statistical Analysis Should I UseUploaded byAbhishek2009GWU
- IBM SPSS AMOS.pdfUploaded byJulian Mantong
- Sampling Distributions 1Uploaded byHazel Papagayo
- Andrew Rosenberg- Lecture 5: Linear Regression with Regularization CSC 84020 - Machine LearningUploaded byTuytm2