91 views

Uploaded by Arun Prasath

Variance Covariance Matrix and properties

- twp02-26
- Sage - Girden, 1992 ANOVA Repeated Measures
- Competition Between Tax Havens Does Proximity
- Measurement Error in Time-Series Studies of Air Pollution Concepts and Consequences
- Modified Consumer Purchase Intension of Foreign Brands Available in Pakistan
- Statistics Exercises
- Regression.pdf
- PERIODOGRAM ANALYSIS WITH MISSED OBSERVATION BETWEEN TWO VECTOR VALUED STOCHASTIC PROCESS.
- Pca
- comprehensive-data-exploration-with-python.pdf
- tmpB44
- [D. N. Elhance] Fundamentals of Statistics(B-ok.xyz)
- How to Prove Causality
- Stats - Repeated Measures
- T02
- pauline.pdf
- MB_lecture_11.pdf
- diboyidaikj
- Linear Regression
- TWJ_Volume 2(1)_(7) Dienny R Rahmani (2016).pdf

You are on page 1of 4

June 1, 2009

Covariance

Covariance measures the degree to which two variables change or vary together (i.e. co-vary).

On the one hand, the covariance of two variables is positive if they vary together in the same

direction relative to their expected values (i.e. if one variable moves above its expected

value, then the other variable also moves above its expected value). On the other hand, if

one variable tends to be above its expected value when the other is below its expected value,

then the covariance between the two variables is negative. If there is no linear dependency

between the two variables, then the covariance is 0.

The covariance of two random variables, Xi and Xj , can be mathematically represented

as

cov(Xi , Xj ) = E[(Xi i )(Xj j )]

(1)

where i = E(Xi ) and j = E(Xj ).1 This relationship can be further generalized to a

using matrix notation

multivariate situation and for estimated coefficients (i.e. s)

h

i

= ( )( )T = E(T ) T

cov()

(2)

where is a vector of estimated coefficients and =

more explicitly represented as

cov(1 , 1 ) cov(1 , 2 )

cov(2 , 1 ) cov(2 , 2 )

cov() =

...

...

cov( , ) cov( , )

k

1

k

2

E()

... cov(1 , k )

... cov(2 , k )

,

...

...

... cov(k , k )

(3)

where the diagonal of this matrix represents the variance of the vector with k elements.

Therefore, the matrix can be rewritten as

1 , 2 ) ... cov(1 , k )

var(1 )

cov(

cov(2 , 1 )

var(2 ) ... cov(2 , k )

cov() =

(4)

.

...

...

...

...

cov( , ) cov( , ) ... var( )

k

1

k

2

k

This essentially represents the covariance or variance-covariance matrix.

1

= (X T X)1 X T y

(5)

= (X T X)1 X T (X + u)

Then distribute the first set of terms

= (X T X)1 X T X + (X T X)1 X T u

And simplify, recalling that (X T X)1 (X T X) = 1

= + (X T X)1 X T u

Finally, subtract from both sides to get

= (X T X)1 X T u

h

i

= E ( )( )T

cov()

(6)

(7)

Insert Eq. 6 into Eq. 7 and then reorder the terms (remember (AB)T = B T AT )

= E ((X T X)1 X T u)((X T X)1 X T u)T

cov()

= E (X T X)1 X T uuT X(X T X)1

cov()

Take the expectation (the Xs are non-stochastic, and it is assumed that E(uuT ) = 2 I)

= (X T X)1 X T E(uuT )X(X T X)1

cov()

= (X T X)1 X T 2 IX(X T X)1

cov()

And simplify, producing the final equation for the variance-covariance matrix

= 2 (X T X)1

cov()

(8)

In this last equation, 2 is the homoskedastic variance of ui and (X T X)1 is the inverse

matrix appearing in the OLS estimator.

= 2 (X T X)1

cov()

To estimate the variance-covariance matrix, the variance is needed. Since the true variance

is unknown, it must be estimated. An unbiased estimator of 2 for the k-th variable case is

P 2

ui

2

(9)

=

nk

which is equivalent to

2 =

uT u

nk

(10)

where u is the estimated residuals, n is the number of observations, and k the number of

parameters being estimated.

Although

2 can be computed directly from the estimated residuals, there is a second

approach to calculating it. First, recall that

T SS = ESS + RSS

(11)

where T SS is the total sum of squares, ESS is the explained sum of squares, and RSS is the

residual sum of squares. This equation can be rewritten as

RSS = T SS ESS

(12)

RSS = uT u

(13)

T SS = y T y nY 2

ESS = T X T y T nY 2

(14)

(15)

where the term nY 2 is known as the correction for mean. Therefore, substituting Eqs. 13-15

into Eq. 12 produces

uT u = y T y T X T y T

(16)

Covariance and correlation are related but not equivalent statistical measures. In particular,

the correlation of two variables, Xi and Xj , is their normalized covariance, which is defined

as

E [(Xi i )(Xj j )]

,

(17)

i,j =

i j

where i,j is the correlation coefficient, i = E(Xi ), j = E(Xj ), i is the standard deviation

of Xi , and j is the standard deviation of Xj . Because the correlation is normalized, it is

dimensionless (i.e. it is a pure number without a unit of measure). By contrast, covariance

does have a unit of measurethe product of the units of two variables.

Excercise

n = 15

Y = 68.7333

= 5.5027 4.1535 3.3787 8.2903

y T y = 259651

1031

11370

T

X y =

35981

10176

1. Compute the variance-covariance matrix. You can also compute R2 .

2. Use the variance-covariance matrix to make inferences about the estimated coefficients.

3. Is there any evidence of violations of the OLS assumptions given the above information

and your calculations?

4. The covariance matrix of the three variables of interest (X1 , X2 , X3 ) is

X1

X2

X3

X1 53.5523 167.2238 4.1667

X2 167.2238 522.9238 12.5238

X3 4.16667 12.5238 12.3810

Given that the standard deviations between X1 and X2 are 7.3179 and 22.8675, respectively, what is the correlation between these two variables?

- twp02-26Uploaded bysalkhaniae
- Sage - Girden, 1992 ANOVA Repeated MeasuresUploaded byEntre
- Competition Between Tax Havens Does ProximityUploaded byvishal
- Measurement Error in Time-Series Studies of Air Pollution Concepts and ConsequencesUploaded bycchien
- Modified Consumer Purchase Intension of Foreign Brands Available in PakistanUploaded byAlexander Decker
- Statistics ExercisesUploaded byman420
- Regression.pdfUploaded bykamalkant05
- PERIODOGRAM ANALYSIS WITH MISSED OBSERVATION BETWEEN TWO VECTOR VALUED STOCHASTIC PROCESS.Uploaded byIJAR Journal
- PcaUploaded byAngga Dwi
- comprehensive-data-exploration-with-python.pdfUploaded byNew Mahoutsukai
- tmpB44Uploaded byFrontiers
- [D. N. Elhance] Fundamentals of Statistics(B-ok.xyz)Uploaded byhareshtank
- How to Prove CausalityUploaded byteebone747
- Stats - Repeated MeasuresUploaded bylamiahhh
- T02Uploaded bykeven319hk4304
- pauline.pdfUploaded byjoneskmw
- MB_lecture_11.pdfUploaded byrui
- diboyidaikjUploaded bySirnagalihDua Cisurupan
- Linear RegressionUploaded bysofianina05
- TWJ_Volume 2(1)_(7) Dienny R Rahmani (2016).pdfUploaded byDienny Redha Rahmani
- Chapter 9bjUploaded bySyrone John David
- www.myengg.com / JEE Main Maths Model Paper 4Uploaded byPremKumarKalikiri
- The Influence of Political Power in Predicting the Outcome of RevolutionsUploaded bymariaglenna
- Random Variables_Random ProcessUploaded byPrasanth
- Stochastic ApproachUploaded byAnta Cristina Ionescu
- Effect of Late sitting on employee performance-1.pdfUploaded bymeesum110
- Himanshu a 21Uploaded byiceman7264
- Chap 010Uploaded bymallick5051rajat
- Properties of Random VariablesUploaded byJulia Levinson
- List of FormulaUploaded byChristina Hea

- GSR part 4.pdfUploaded byArun Prasath
- Sheath Size vs Element SizeUploaded byArun Prasath
- 34-ST-03-101Uploaded byArun Prasath
- Karvy Investment Strategy Report 2018Uploaded byRV Reddy
- Tc Temp Limits RangeUploaded bybhaktamelt
- The Great Book of Best Quotes of All Time. | Original |Uploaded byAbhi Sharma
- 8 - Tom Voss - Tritium Detection Methods and LimitationsUploaded byArun Prasath
- Safety Standards Wheel Poster IAEAUploaded byLucas Kahwage
- The Three Questions.pdfUploaded byArun Prasath
- A Tutorial on Bayesian Estimation and Tracking Applicable to Nonlinear and Non-Gaussian ProcessesUploaded bygiunghi
- A Haunted HouseUploaded byAnahí Tessa
- Bayesian InferenceUploaded byArun Prasath
- An Introduction to the Kalman Filter (FULL VERSION)Uploaded bykasra_tm
- the rose and the nightingaleUploaded byIndah Maya Rosanty
- Sysid4Uploaded byPavan Kumar
- Sysid4Uploaded byPavan Kumar

- gravity10_thus3Uploaded byMohd Azruan Abdullah
- polyfit.pdfUploaded bydarafeh
- Application of Ordinary Least Square Method in NonlinearUploaded byEny Enasty
- Decision ScienceUploaded byNikhil Kasat
- STATA CommandsUploaded byDanish Us Salam
- Durbin Watson TestUploaded byTrung Vũ
- Assignment-06.xlsxUploaded bysameer imdad
- Herramientas y Trucos Para StataUploaded byxxxshebyxxx
- Endogeneity and Instrumental VariablesUploaded byjustin bal
- AMOS_CFA.pdfUploaded byTar TwoGo
- The Reliability of the Manchester Triage System (MTS) a Meta-AnalysisUploaded byDaniel Alejandro Viteri Alvarez
- SEM-IfM_Read-Only.pdfUploaded byJoaquim Silva
- Subset Selection in Regression AnalysisUploaded byarifulmondal
- app2Uploaded byTiara Marselia
- Principal Components AnalysisUploaded bymorphyjack
- 3slsUploaded byAldina Safira
- CorrRegr-SPSS.docUploaded byJuan Carlos Elias Yovera
- Psych R PackageUploaded byAman Machra
- summary ekmet.docxUploaded bykhalisha adelia
- Linear RegressionHandoutUploaded byHorváth Dániel
- Economatrics 1Uploaded byShoaibTahir
- SPSSbookUploaded byapi-3752301
- Multiple Linear Regression in Data MiningUploaded byakirank1
- Reis - Streamlined LCA Thru Materials ClassificationUploaded byISSSTNetwork
- Hoyle CFA Chapter - FinalUploaded bysanzit
- 7. Heckman - Correction for Nonrandom Samples Nobel Prize Winning ConceptUploaded byRACHVIK07
- PublicUploaded bydatasenter
- Reading Statistics and ResearchUploaded bydurantier
- D_Tutorial 6 (Solutions)Uploaded byAlfie
- Lecture 11 AnovaUploaded byJane Winner