Search on Cross Validated… 1

SPONSORED BY

Home
In simple linear regression, how does the derivation of the variance of the Ask Question

Questions
residues support its 'Constant Variance' Assumption?
Tags
Asked 3 years, 6 months ago Active 3 years, 6 months ago Viewed 298 times
Users

Unanswered In simple linear regression:

Featured on Meta

3 𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙𝑠 = 𝑌 ̂ − 𝑌
Opt-in alpha test for a new Stacks editor

We can derive that: Visual design changes to the review

queues
𝑉 𝑎𝑟(𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙𝑠) = 𝑉 𝑎𝑟(𝑌 ̂ − 𝑌 ) = (𝐼 − 𝐻) ⋅ 𝜎 2
2
Hot Meta Posts
(𝜎 2 is the variance of 𝑌 ) (See derivation of Var(residuals))
23 Would Cross Validated want Machine
So my question is: Learning Theory questions that are no…

13 Some tags need better names

2
Since ℎ𝑖𝑖 (diagonal elements in H) is different for each 𝑖 , how come the (𝐼 − 𝐻) ⋅ 𝜎 ,
5 What are some “best practices” for
which is the variance of the residuals, is a constant (one of the assumptions of the LR)?
structuring/formatting a post?

Another thing that bothers me is:

Linked

We know that the MSE (which is calculated by the sum of squared residuals divided by n- 24 In simple linear regression, where does the
formula for the variance of the residuals
p) is an unbiased estimate of the variance of the errors, so what are the relationships come from?
between MSE, the variance of the errors and the variance of the residuals? Which one is
1 Is the variance of the residuals in linear
the LR assumption targeting at? regression constant (assuming constant-
variance noise)?

regression variance residuals error linear Related

10 Expected Value and Variance of Estimation

edited Jul 16 '17 at 9:24 asked Jul 16 '17 at 6:38 of Slope Parameter 𝛽1 in Simple Linear
Share Cite Edit Follow Mooncrater user2027016 Regression
557 1 8 18 31 2
7 Homoscedasticity Assumption in Linear
Regression vs. Concept of Studentized
Residuals
1 There's no assumption of constant variance of residuals in regression. There is a constant variance
assumption however. (BTW can you please be consistent with residuals vs "residues" in your Q? They 6 Meaning of variance term in confidence
should all be residuals) – Glen_b Jul 16 '17 at 6:50 interval for Multiple Linear Regression

1 Thanks a lot for the comment! My confusion is that: one of the assumptions of LR is homoscedasticity - 0 MSE Bias Variance tradeoff in estimating
(constant variance) of the ERRORS, and in order to check that, we usually generate residual plots, which the variance of noise for MLE linear
are used to check whether the variance of the RESIDUALS is a constant; and given the derivation in the regression
post, the variance of the RESIDUALS doesn't have to be a constant, so how can we use the residual plots
to test the homoscedasticity property of the ERRORS? Thanks! And I will make the terms consistent in 2 Checking the constant variance assumption
the post. – user2027016 Jul 16 '17 at 7:21 for residuals vs fitted plots: What about for
the same fitted values?
2 you're getting to the nub of an important question. Have you ever wondered why many stats packages by
default offer plots not of raw residuals but residuals divided by 𝑠√‾
1‾‾‾‾
− ℎ‾𝑖𝑖 ? e.g. take a look at what R's 5 How to quantify bias and variance in simple
linear regression?
plot.lm does – Glen_b Jul 16 '17 at 7:29

@Glen_b, this is very insightful! It definitely helps a lot to clear my confusions, thank you sooo much! – Hot Network Questions
user2027016 Jul 16 '17 at 7:54
Should I log users in if they enter valid login info in
add a comment registration form?

Why do we still teach the determinant formula for

1 Answer Active Oldest Votes cross product? And is it as bad as I think it is?

How to best clean a large historical corpus ridden

with OCR errors
The assumption is that the errors have constant variance. The corresponding residuals don't have How does everyone not become poor over time?
constant variance because points with larger values on the diagonal of the hat matrix (𝐻 ) pull the
Pattern matching for repeated elements
2 fit more toward themselves than points with smaller values do, that is, they have greater influence
∂𝑦^𝑖 Who predicted the existence of the muon
on their own fitted value: = ℎ𝑖𝑖 , the 𝑖 th diagonal element of 𝐻 . neutrino?
∂𝑦𝑖
Arrays with non-inline style formulas in rows in
KaTeX
As you say, the variance-covariance matrix of the residuals is given by 𝜎 2 (𝐼− 𝐻), which is
normally estimated by 𝑠2 (𝐼 − 𝐻) , and so the standard error of the 𝑖 th residual is 𝑠√‾
1‾‾‾‾‾
− ℎ𝑖𝑖‾. Why do banks have capital requirements on
deposits?

𝑦𝑖 −𝑦𝑖̂ ESTA denied because overstay - how to appeal?

As a result it's common to standardize residuals, scaling them to have constant variance: .
𝑠√1−ℎ𝑖𝑖 Can there be C# without F# in signature?

more hot questions

edited Jul 16 '17 at 23:01 answered Jul 16 '17 at 13:40
Share Cite Edit Follow Glen_b Question feed
244k 27 506 876

add a comment

Your Answer

Links Images Styling/Headers Lists Blockquotes Code HTML Tables Advanced help

Post Your Answer

Not the answer you're looking for? Browse other questions tagged regression variance residuals

error linear or ask your own question.

CROSS VALIDATED COMPANY STACK EXCHANGE Blog Facebook Twitter LinkedIn Instagram
NETWORK
Tour Stack Overflow
Technology
Help For Teams
Life / Arts
Chat Advertise With Us
Culture / Recreation
Contact Hire a Developer
Science
Feedback Developer Jobs
Other
Mobile About
Disable Responsiveness Press
Legal
Privacy Policy
Terms of Service
site design / logo © 2021 Stack Exchange Inc; user contributions licensed
Cookie Settings under cc by-sa. rev 2021.2.5.38499