15 views

Original Title: Lectures of Weighted Least Squares

Uploaded by JongHun Park

- Week2_handout
- Assumptions Regression
- Experimental Design Optimization And
- Homework 3 Answers Updated (1)
- Homework_3_Answers_updated.docx
- 13
- Regression
- Direct democracy Education
- 43_1_rgrssn
- Analytical Chemistry.- Curvas de Calibracion, Bondad de Ajuste, Ponderación
- Cheat Sheet
- 2010_methods
- ForecastForecasting with term structure
- Chapter 01
- business statistics exercises answers
- 10
- kimed 1 parameter b.doc
- Impact of Sex Appeals In TV Advertisement
- PSQ Reviewer 1
- Economic Consequences Related Ch. 11

You are on page 1of 19

i = 1, . . . , n.

2

Var(Y | X = xi) = Var(i) =

.

wi

Weighted least squares is an estimation technique which

weights the observations proportional to the reciprocal of

the error variance for that observation and so overcomes the

issue of non-constant variance.

7-1

Yi = 0 + 1Xi + i

i = 1, . . . , n

the quantity

Sw (0, 1) =

n

X

wi(yi 0 1xi)2

i=1

inversely proportional to the corresponding variances; points

with low variance will be given higher weights and points with

higher variance are given lower weights.

7-2

0 = y w 1xw

P

wi(xi xw )(yi y w )

1 =

P

wi(xi xw )2

where xw and y w are the weighted means

P

xw =

wixi

P

wi

yw =

wiyi

.

P

wi

Some algebra shows that the weighted least squares estimates are still unbiased.

7-3

Var(1) = X

2

wi(xi xw )2

1

Var(0) = X

wi

+X

x2

w

wi(xi xw )2

variables, the sampling distributions are still normal.

us an unbiased estimator of 2 so we can derive t tests for

the parameters etc.

7-4

w1, . . . , wn.

Sw ( ) =

n

X

i=1

the weighted sum of squares.

= (X tW X )1X tW Y .

7-5

yi = 0 + 1xi + i

2.

where Var(i) = x2

i

by taking

yi0 =

yi

xi

x0i =

1

xi

0i =

i

.

xi

yi0 = 1 + 0x0i + 0i

where Var(0i) = 2.

7-6

S1(0, 1) =

n

X

(yi0 1 0x0i)2

i=1

n

X

!2

1

yi

1 0

xi

i=1 xi

!

n

X

1

2

=

(y

x

)

0

1

i

i

2

x

i

i=1

=

Hence the weighted least squares solution is the same as the

regular least squares solution of the transformed model.

7-7

Y = X +

where Var() = W 1 2.

to

wi.

7-8

Y 0 = W 1/2Y

X 0 = W 1/2X

0 = W 1/2.

Y 0 = X 0 + 0

Using the results from regular least squares we then get the

solution

=

X0

t

X0

1

X 0tY 0 =

1

t

X WX

X tW Y .

7-9

The Weights

w1, . . . , wn.

We may have a probabilistic model for Var(Y | X = xi) in

which case we would use this model to find the wi.

omit some observations from the fitted model.

7-10

The Weights

single measure but an average of ni actual measures and the

original measures each have variance 2.

2

=

ni

This situation often occurs in cluster surveys.

7-11

Unknown Weights

use weighted least squares.

One way to do this is possible when there are multiple repeated observations at each value of the covariate vector.

number of replicates will be observed for each set value of

the covariate vector.

We can then estimate the variance of Y for each fixed covariate vector and use this to estimate the weights.

7-12

Pure Error

Then a fitted model could be

yij = 0 + 1xj + ij

i = 1, . . . , nj ; j = 1, . . . , k.

eij = yij yij = (yij y j ) + (y j yij ).

7-13

Pure Error

Note that the pure error term does not depend on the mean

model at all.

variance at each value of x.

s2

j =

nj

X

1

(yij y j )2

nj 1 i=1

j = 1, . . . , k.

the weights in a weighted least squares regression model.

7-14

Unknown Weights

For observational studies, however, this is generally not possible since we will not have repeated measures at each value

of the covariate(s).

Sometimes, however, we may be willing to assume that the

variance of the observations is the same within each level

of some categorical variable but possibly different between

levels.

In that case we can estimate the weights assigned to observations with a given level by an estimate of the variance for

that level of the categorical variable.

7-15

Two-Stage Estimation

least squares regression to the data.

we examine plots of the residuals against a categorical variable which we suspect is the culprit for this problem.

a categorical variable likely to affect variance.

mean model.

7-16

Two-Stage Estimation

nj observations with Z = j (j = 1, . . . , k).

If the error variability does vary with the levels of this categorical variable then we can use

j2 =

X

1

ri2

nj 1 i:z =j

i

studentized reiduals ri but that does not work well.

7-17

Two-Stage Estimation

cj =

j2

X

1

ri2

nj 1 i:z =j

i

n

1 X

ri2

n i=1

weights in a weighted least squares regression in the second

stage.

made using the results of the weighted least squares model.

7-18

a number of criticisms have been raised.

would result in different estimated weights and this variability

is not properly taken into account in the inference.

Indeed the authors acknowledge that the true sampling distributions are unlikely to be Student-t distributions and are

unknown so inference may be suspect.

categorical variable explaining the variance heterogeneity can

be found.

7-19

- Week2_handoutUploaded byhsdfighsi
- Assumptions RegressionUploaded bySwapnil Patil
- Experimental Design Optimization AndUploaded byW00W
- Homework 3 Answers Updated (1)Uploaded byRoanBasilla
- Homework_3_Answers_updated.docxUploaded byMariana Oliveira
- 13Uploaded byShubham Tewari
- RegressionUploaded byblw2002
- Direct democracy EducationUploaded byan3zil
- 43_1_rgrssnUploaded byEbookcraze
- Analytical Chemistry.- Curvas de Calibracion, Bondad de Ajuste, PonderaciónUploaded byhugoboss_dark
- Cheat SheetUploaded byshaziadurrani
- 2010_methodsUploaded bySaurabh Pandey
- ForecastForecasting with term structureUploaded byCésar Chávez La Rosa
- Chapter 01Uploaded byekonomistik
- business statistics exercises answersUploaded byPaul Davis
- 10Uploaded byHassen Hedfi
- kimed 1 parameter b.docUploaded bySabda Kartika
- Impact of Sex Appeals In TV AdvertisementUploaded byRizka Sarastri Sumardiono
- PSQ Reviewer 1Uploaded byJan Michael Abarquez
- Economic Consequences Related Ch. 11Uploaded bynita selvia
- rpne721Uploaded byJoko Sukowo
- SBCO Forecasting HB StudentUploaded byVictoria Bailey
- Lecture 06Uploaded byNaeem Ahmed Hattar
- VARUploaded byOladipupo Mayowa Paul
- GDP INF EXPO PG 1Uploaded byUmair Ashraf
- Time Series ForecastingUploaded byraymar2k
- EconomicsUploaded byakshay patri
- Firm ChairUploaded bypmcsic
- Air Pollution and Environmental Injustice, Are the Socially Deprived Exposed to More PM2.5 Pollution in Hong KongUploaded byanon_926179562
- ClassicalLinearModel.pdfUploaded byYoussef Alami

- statistics chapter 4 projectUploaded byapi-306798358
- Syllabus_MTech_AdvMnfgUploaded byDhinakar Annadurai
- LR_Updated.pdfUploaded byNipun Goyal
- cu 3008 Assignment 2 - Due Feb 25thUploaded byJim Hack
- Dummy RegressionUploaded byshravan_iitm
- stataUploaded bydhirensahuatrisk
- _AbstractsUploaded byGerard Crotty
- Ww2 Coastal Edu Kingw Statistics R Tutorials Simplelinear HtUploaded byNandkishore Srinivasan
- SGS MIN TP2005 03 Flotation Plant Design and Production Planning.pdfUploaded byPedro Pablo
- 04_Jackknife.pdfUploaded byuniversedrill
- 215 Final Exam Formula SheetUploaded byH.C. Z.
- Basic Stata CommandsUploaded bytamzid11
- DAFN_HW1Uploaded byAadarsh Saxena
- Structural Equation ModellingUploaded byMuhammad Imran Alam
- WEB LINKSUploaded byAaron Marquez
- Convergence of MLE to MVUE of Reliability for Exponential Class Software Reliability ModelsUploaded byEditor IJRITCC
- Paneldata(Dummy)Uploaded byAli Awais Cheema
- Gretl ManualUploaded byVictor Villano
- Ken Black QA 5th chapter14 SolutionUploaded byRushabh Vora
- 15 Seasonal ArimaUploaded byGerardo Aguilar Rosel
- HW1Uploaded byRoberto Valdez Ponce
- A9R911BUploaded byLouis-Pierre Lepage
- 13Uploaded byShubham Tewari
- Adaptive LassoUploaded byJoanne Wong
- Elements of Statistics - Fergus Daly, Et AlUploaded bymach20_aardvark8064
- eiPackRnewsUploaded byvujo
- CSE Final Upto 3rd Year Syllabus 17.01.13Uploaded byAbhishek Das
- Applied Linear Statistical ModelUploaded byJordan Brian
- Residuals Statisticsa.docxUploaded bymuklis
- AP Statistics Test ReviewUploaded byPongman92