91 views

Uploaded by Arun Prasath

Variance Covariance Matrix and properties

- Chapter 6(Economic Selection Indexes)
- Risk Models
- ACCA P3 Course Notes
- Random Matrix Theory, Part II
- Using ACF and PACF to select MA(q) or AR(p) models
- 5.pdf
- Peter Young_Recursive Estimation and Time Series Analysis
- A Comparison of Performances of Conventional Tillage Implements Versus Namibia Specific Conservation Tillage Implements Under Ogongo, Namibia Conditions
- 1
- Pre Processing
- 03 Index Model (1)
- Estimating Factor Models and Managing Risk in R
- Elec214_L16
- 04-multivariate-vectors-and-samples-2008-1203418099646432-3
- ds.docx
- 2012-004
- Corelation Coeficient
- cvarter
- QZ220_1_JUN09_FORM01_AS
- Uncertainty analysis for filter radiometry based on the uncertainty associated with integrated quantities

You are on page 1of 4

June 1, 2009

Covariance

Covariance measures the degree to which two variables change or vary together (i.e. co-vary).

On the one hand, the covariance of two variables is positive if they vary together in the same

direction relative to their expected values (i.e. if one variable moves above its expected

value, then the other variable also moves above its expected value). On the other hand, if

one variable tends to be above its expected value when the other is below its expected value,

then the covariance between the two variables is negative. If there is no linear dependency

between the two variables, then the covariance is 0.

The covariance of two random variables, Xi and Xj , can be mathematically represented

as

cov(Xi , Xj ) = E[(Xi i )(Xj j )]

(1)

where i = E(Xi ) and j = E(Xj ).1 This relationship can be further generalized to a

using matrix notation

multivariate situation and for estimated coefficients (i.e. s)

h

i

= ( )( )T = E(T ) T

cov()

(2)

where is a vector of estimated coefficients and =

more explicitly represented as

cov(1 , 1 ) cov(1 , 2 )

cov(2 , 1 ) cov(2 , 2 )

cov() =

...

...

cov( , ) cov( , )

k

1

k

2

E()

... cov(1 , k )

... cov(2 , k )

,

...

...

... cov(k , k )

(3)

where the diagonal of this matrix represents the variance of the vector with k elements.

Therefore, the matrix can be rewritten as

1 , 2 ) ... cov(1 , k )

var(1 )

cov(

cov(2 , 1 )

var(2 ) ... cov(2 , k )

cov() =

(4)

.

...

...

...

...

cov( , ) cov( , ) ... var( )

k

1

k

2

k

This essentially represents the covariance or variance-covariance matrix.

1

= (X T X)1 X T y

(5)

= (X T X)1 X T (X + u)

Then distribute the first set of terms

= (X T X)1 X T X + (X T X)1 X T u

And simplify, recalling that (X T X)1 (X T X) = 1

= + (X T X)1 X T u

Finally, subtract from both sides to get

= (X T X)1 X T u

h

i

= E ( )( )T

cov()

(6)

(7)

Insert Eq. 6 into Eq. 7 and then reorder the terms (remember (AB)T = B T AT )

= E ((X T X)1 X T u)((X T X)1 X T u)T

cov()

= E (X T X)1 X T uuT X(X T X)1

cov()

Take the expectation (the Xs are non-stochastic, and it is assumed that E(uuT ) = 2 I)

= (X T X)1 X T E(uuT )X(X T X)1

cov()

= (X T X)1 X T 2 IX(X T X)1

cov()

And simplify, producing the final equation for the variance-covariance matrix

= 2 (X T X)1

cov()

(8)

In this last equation, 2 is the homoskedastic variance of ui and (X T X)1 is the inverse

matrix appearing in the OLS estimator.

= 2 (X T X)1

cov()

To estimate the variance-covariance matrix, the variance is needed. Since the true variance

is unknown, it must be estimated. An unbiased estimator of 2 for the k-th variable case is

P 2

ui

2

(9)

=

nk

which is equivalent to

2 =

uT u

nk

(10)

where u is the estimated residuals, n is the number of observations, and k the number of

parameters being estimated.

Although

2 can be computed directly from the estimated residuals, there is a second

approach to calculating it. First, recall that

T SS = ESS + RSS

(11)

where T SS is the total sum of squares, ESS is the explained sum of squares, and RSS is the

residual sum of squares. This equation can be rewritten as

RSS = T SS ESS

(12)

RSS = uT u

(13)

T SS = y T y nY 2

ESS = T X T y T nY 2

(14)

(15)

where the term nY 2 is known as the correction for mean. Therefore, substituting Eqs. 13-15

into Eq. 12 produces

uT u = y T y T X T y T

(16)

Covariance and correlation are related but not equivalent statistical measures. In particular,

the correlation of two variables, Xi and Xj , is their normalized covariance, which is defined

as

E [(Xi i )(Xj j )]

,

(17)

i,j =

i j

where i,j is the correlation coefficient, i = E(Xi ), j = E(Xj ), i is the standard deviation

of Xi , and j is the standard deviation of Xj . Because the correlation is normalized, it is

dimensionless (i.e. it is a pure number without a unit of measure). By contrast, covariance

does have a unit of measurethe product of the units of two variables.

Excercise

n = 15

Y = 68.7333

= 5.5027 4.1535 3.3787 8.2903

y T y = 259651

1031

11370

T

X y =

35981

10176

1. Compute the variance-covariance matrix. You can also compute R2 .

2. Use the variance-covariance matrix to make inferences about the estimated coefficients.

3. Is there any evidence of violations of the OLS assumptions given the above information

and your calculations?

4. The covariance matrix of the three variables of interest (X1 , X2 , X3 ) is

X1

X2

X3

X1 53.5523 167.2238 4.1667

X2 167.2238 522.9238 12.5238

X3 4.16667 12.5238 12.3810

Given that the standard deviations between X1 and X2 are 7.3179 and 22.8675, respectively, what is the correlation between these two variables?

- Chapter 6(Economic Selection Indexes)Uploaded byJha Zel
- Risk ModelsUploaded byShiv Narayan
- ACCA P3 Course NotesUploaded bySyed Wajahat Ali
- Random Matrix Theory, Part IIUploaded byKelvin Guu
- Using ACF and PACF to select MA(q) or AR(p) modelsUploaded bybboyvn
- 5.pdfUploaded byEugenio Martinez
- Peter Young_Recursive Estimation and Time Series AnalysisUploaded bymahinur_alam6555
- A Comparison of Performances of Conventional Tillage Implements Versus Namibia Specific Conservation Tillage Implements Under Ogongo, Namibia ConditionsUploaded byaogu1
- 1Uploaded bySTATPERSON PUBLISHING CORPORATION
- Pre ProcessingUploaded byPragati Singh
- 03 Index Model (1)Uploaded byStacc
- Estimating Factor Models and Managing Risk in RUploaded byMark Sinsheimer
- Elec214_L16Uploaded bysaravkiru
- 04-multivariate-vectors-and-samples-2008-1203418099646432-3Uploaded byRoberto García
- ds.docxUploaded byjames me
- 2012-004Uploaded byKim-Bich Long
- Corelation CoeficientUploaded bynitin30
- cvarterUploaded bypasaitow
- QZ220_1_JUN09_FORM01_ASUploaded byanik_kurp
- Uncertainty analysis for filter radiometry based on the uncertainty associated with integrated quantitiesUploaded byNational Physical Laboratory
- DKE53_ch2.pdfUploaded byVinoth Nagaraj
- 11ACFPAUploaded bysom
- Acemoglu review of PT the effects of constitutions.pdfUploaded byhyper
- Honey I Shrunk the Sample Covariance MatrixUploaded byNemanja Jeremic
- Chap 008Uploaded byminibod
- Tvsvar Final July 04Uploaded byVu Bui Quoc
- 604 ARMA QuestnsUploaded byOmotola Awojobi
- US Federal Reserve: 200734papUploaded byThe Fed
- Spectrum-Sensing Algorithms for Cognitive Radio Based on Statistical CovariancesUploaded bykukulkan9
- c11037Uploaded bybensbloom159

- Bayesian InferenceUploaded byArun Prasath
- The Great Book of Best Quotes of All Time. | Original |Uploaded byAbhi Sharma
- GSR part 4.pdfUploaded byArun Prasath
- Safety Standards Wheel Poster IAEAUploaded byLucas Kahwage
- the rose and the nightingaleUploaded byIndah Maya Rosanty
- 8 - Tom Voss - Tritium Detection Methods and LimitationsUploaded byArun Prasath
- Tc Temp Limits RangeUploaded bybhaktamelt
- Sheath Size vs Element SizeUploaded byArun Prasath
- 34-ST-03-101Uploaded byArun Prasath
- Karvy Investment Strategy Report 2018Uploaded byRV Reddy
- The Three Questions.pdfUploaded byArun Prasath
- A Haunted HouseUploaded byAnahí Tessa
- An Introduction to the Kalman Filter (FULL VERSION)Uploaded bykasra_tm
- A Tutorial on Bayesian Estimation and Tracking Applicable to Nonlinear and Non-Gaussian ProcessesUploaded bygiunghi
- Sysid4Uploaded byPavan Kumar

- notes.420.02[1]Uploaded byDino V Escalona
- QMS102 204 Binomial Poisson Calculator TricksUploaded byy2k1000
- Exponential Disttribution propertiesUploaded byClaudio Amaury C. Jiménez
- Normal Distribution - Wikipedia, The Free EncyclopediaUploaded byÁlvaro Jiménez
- Abou-Faycal Trott ShamaiUploaded byNguyễn Minh Dinh
- MA6451-Probability and Random ProcessesUploaded bymohan
- Exam_4_Solutions.pdfUploaded bySurendra Singh Gaud
- PertUploaded bythissr
- Calculating VARUploaded bySubramanian Rajagopalan
- normality skewness kurtosisUploaded byapi-285777244
- Monte Carlo ProbabilityUploaded byanalese1
- 2 mark questionsUploaded bysaransuriyan
- SM-57Probability and Statistics.pdfUploaded bypavan kumar kvs
- e25m1Uploaded byali_isa_3
- Excess Loss VariableUploaded byvbatta2
- Quiz on DispersiionUploaded byHassan Saleem
- Aggregation of Correlated Risk Portfolios - Models and Algorithms.pdfUploaded byhc87
- Livro_distribuicoesUploaded byVanêzä Souza
- Chapter 1 ProbabilityUploaded byRenuga Subramaniam
- Lecture 8 Random VariablesUploaded byAzam-Savaşçı Anderson Mohammad
- Machine LearningUploaded byIonuț-Mihai Burlacu
- Overdispersion in Modelling Accidents and in Empirical Bayes EstimationUploaded byjodomofo
- Mth514NotesCh4,5,6,10Uploaded byt_man222
- Relationship of Mean Median ModeUploaded byKit Tamayo
- ProblemSheet06MT3073 CRVUploaded byusama
- Pertemuan-3-PPC BU IKE UB LALALA YEYEUploaded bydearsdea
- Lab12 MonteCarlo StudentUploaded byTai Zhou
- Stat 330Uploaded bySamantha Yu
- Newbold Chapter 7Uploaded byKazi93_thefat
- Call Letter for Reliable Techno DesignsUploaded byanon-430485