Dougherty

Type author name/s here

Introduction to Econometrics,
5th edition
Chapter heading
Review: Random Variables,
Sampling, Estimation, and
Inference

© Christopher Dougherty, 2016. All rights reserved.

 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance

cov X , Y    XY  E   X   X  Y  Y  

The covariance of two random variables X and Y, often written sXY, is defined to be the
expected value of the product of their deviations from their population means.

1
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance

cov X , Y    XY  E   X   X  Y  Y  

If X and Y are independent,

cov X , Y   E   X   X  Y  Y  
  E  X   X    E  Y  Y  
  E  X   E   X    E  Y   E  Y  
   X   X  Y  Y   0  0  0

If two variables are independent, their covariance is zero. To show this, start by rewriting
the covariance as the product of the expected values of its factors.

2
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance

cov X , Y    XY  E   X   X  Y  Y  

If X and Y are independent,

cov X , Y   E   X   X  Y  Y  
  E  X   X    E  Y  Y  
  E  X   E   X    E  Y   E  Y  
   X   X  Y  Y   0  0  0

We are allowed to do this because (and only because) X and Y are independent (see the
earlier sequence on independence.

3
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance

cov X , Y    XY  E   X   X  Y  Y  

If X and Y are independent,

cov X , Y   E   X   X  Y  Y  
  E  X   X    E  Y  Y  
  E  X   E   X    E  Y   E  Y  
   X   X  Y  Y   0  0  0

The expected values of both factors are zero because E(X) = mX and E(Y) = mY. E(mX) = mX
and E(mY) = mY because mX and mY are constants. Thus the covariance is zero.

4
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. Y = V + W,
If
cov X , Y   cov X , V   cov X , W 

There are some rules that follow in a perfectly straightforward way from the definition of
covariance, and since they are going to be used frequently in later chapters it is worthwhile
establishing them immediately. First, the addition rule.
5
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. IfY = V + W,
cov X , Y   cov X , V   cov X , W 

2. IfY = bZ, where b is a constant,

cov X , Y   b cov X , Z 

Next, the multiplication rule, for cases where a variable is multiplied by a constant.

6
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. Y = V + W,
If
cov X , Y   cov X , V   cov X , W 

2. Y = bZ, where b is a constant,

If
cov X , Y   b cov X , Z 

3. Y = b, where b is a constant,
If
cov X , Y   0

Finally, a primitive rule that is often useful.

7
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. IfY = V + W,
cov X , Y   cov X , V   cov X , W 

Proof

Since Y = V + W, m Y = mV + mW

cov X , Y   E  X   X  Y  Y  
 E  X   X   [V  W ]  [ V  W ] 
 E  X   X   V  V    X   X   W  W  
 cov X , V   cov X , W 

The proofs of the rules are straightforward. In each case the proof starts with the definition
of cov(X, Y).

8
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. IfY = V + W,
cov X , Y   cov X , V   cov X , W 

Proof

Since Y = V + W, mY = mV + mW

cov X , Y   E  X   X  Y  Y  
 E  X   X   [V  W ]  [ V  W ] 
 E  X   X   V  V    X   X   W  W  
 cov X , V   cov X , W 

We now substitute for Y and re-arrange.

9
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

1. IfY = V + W,
cov X , Y   cov X , V   cov X , W 

Proof

Since Y = V + W, mY = mV + mW

cov X , Y   E  X   X  Y  Y  
 E  X   X   [V  W ]  [ V  W ] 
 E  X   X   V  V    X   X   W  W  
 cov X , V   cov X , W 

This gives us the result.

10
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

2. IfY = bZ, where b is a constant,

cov X , Y   b cov X , Z 

Proof

Since Y = bZ, mY = bmZ

cov X , Y   E   X   X  Y  Y  
 E   X   X  bZ  b Z  
 bE  X   X  Z   Z  
 b cov X , Z 

Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y
terms have been replaced by the corresponding bZ terms.

11
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

2. IfY = bZ, where b is a constant,

cov X , Y   b cov X , Z 

Proof

Since Y = bZ, mY = bmZ

cov X , Y   E   X   X  Y  Y  
 E   X   X  bZ  b Z  
 bE  X   X  Z   Z  
 b cov X , Z 

b is a common factor and can be taken out of the expression, giving us the result that we
want.

12
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Covariance rules

3. IfY = b, where b is a constant,

cov X , Y   0

Proof

Since Y = b, mY = b

cov X , Y   E  X   X  Y  Y  
 E  X   X  b  b  
 E 0
0

The proof of the third rule is trivial.

13
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Example use of covariance rules

Suppose Y = b1 + b2Z

cov X , Y   cov X , [b1  b2 Z ]

 cov X , b1   cov X , b2 Z 
 b2 cov X , Z 

Here is an example of the use of the covariance rules. Suppose that Y is a linear function of
Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and
then use covariance rule 1 (second line).
14
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Example use of covariance rules

Suppose Y = b1 + b2Z

cov X , Y   cov X , [b1  b2 Z ]

 cov X , b1   cov X , b2 Z 
 b2 cov X , Z 

cov(X, b1) = 0, using covariance rule 3. The multiplicative factor b2 may be taken out of the
second term, using covariance rule 2.

15
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Corresponding to the covariance rules, there are parallel rules for variances. First the
addition rule.

16
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

2. Y = bZ, where b is a constant,

If
var  Y   b 2 var  Z 

Next, the multiplication rule, for cases where a variable is multiplied by a constant.

17
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

2. Y = bZ, where b is a constant,

If
var  Y   b 2 var  Z 

3. Y = b, where b is a constant,
If
var  Y   0

A third rule to cover the special case where Y is a constant.

18
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

2. Y = bZ, where b is a constant,

If
var  Y   b 2 var  Z 

3. Y = b, where b is a constant,
If
var  Y   0

4. Y = V + b, where b is a constant,
If
var  Y   var  V 

Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of
practical value that it is worth keeping it in mind separately.

19
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Proof

var Y   cov Y , Y   cov [V  W ], Y 

 cov V , Y   cov W ,Y 
var( X )  E ( X   X )2 
 cov V , [V  W ]  cov W ,[V  W ]
 E ( X   X )( X   X )
 cov V , V   cov V , W   cov W ,V   cov W ,W 
 cov( X , X )
 var V   var W   2 cov V ,W 

The proofs of these rules can be derived from the results for covariances, noting that the
variance of Y is equivalent to the covariance of Y with itself.

20
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Proof

var Y   cov Y , Y   cov [V  W ], Y 

 cov V , Y   cov W ,Y 
 cov V , [V  W ]  cov W ,[V  W ]
 cov V , V   cov V , W   cov W ,V   cov W ,W 
 var V   var W   2 cov V ,W 

We start by replacing one of the Y arguments by V + W.

21
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Proof

var Y   cov Y , Y   cov [V  W ], Y 

 cov V , Y   cov W ,Y 
 cov V , [V  W ]  cov W ,[V  W ]
 cov V , V   cov V , W   cov W ,V   cov W ,W 
 var V   var W   2 cov V ,W 

We then use covariance rule 1.

22
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Proof

var Y   cov Y , Y   cov [V  W ], Y 

 cov V , Y   cov W ,Y 
 cov V , [V  W ]  cov W ,[V  W ]
 cov V , V   cov V , W   cov W ,V   cov W ,W 
 var V   var W   2 cov V ,W 

We now substitute for the other Y argument in both terms and use covariance rule 1 a
second time.

23
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

1. Y = V + W,
If
var Y   var V   var W   2 cov V , W 

Proof

var Y   cov Y , Y   cov [V  W ], Y 

 cov V , Y   cov W ,Y 
 cov V , [V  W ]  cov W ,[V  W ]
 cov V , V   cov V , W   cov W ,V   cov W ,W 
 var V   var W   2 cov V ,W 

This gives us the result. Note that the order of the arguments does not affect a covariance
expression and hence cov(W, V) is the same as cov(V, W).

24
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

2. Y = bZ, where b is a constant,

If
var  Y   b 2 var  Z 

Proof

var Y   cov Y , Y   cov bZ , bZ 

 b 2 cov Z , Z 
 b 2 var Z 

The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as
cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as
common factors.
25
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

3. Y = b, where b is a constant,
If
var  Y   0

Proof

var Y   cov b, b   0

The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is
constant, it has zero variance.

26
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

4. Y = V + b, where b is a constant,
If
var  Y   var  V 

Proof

var Y   var V  b   var V   var b   2 cov V , b 

 var V 

The fourth variance rule starts by using the first. The second term on the right side is zero
by variance rule 3. The third is also zero by covariance rule 3.

27
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Variance rules

4. Y = V + b, where b is a constant,
If
var  Y   var  V 

Proof

var  Y   var  V 

0 mV
V
0 V+b
mV + b

The intuitive reason for this result is easy to understand. If you add a constant to a
variable, you shift its entire distribution by that constant. The expected value of the
squared deviation from the mean is unaffected.
28
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y

because it depends on the units of measurement of X and Y.

29
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

A better measure of association is the population correlation coefficient because it is

dimensionless. The numerator possesses the units of measurement of both X and Y.

30
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

The variances of X and Y in the denominator possess the squared units of measurement of
those variables.

31
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

However, once the square root has been taken into account, the units of measurement are
the same as those of the numerator, and the expression as a whole is unit free.

32
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

If X and Y are independent, rXY will be equal to zero because sXY will be zero.

33
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

If there is a positive association between them, sXY, and hence rXY, will be positive. If there
is an exact positive linear relationship, rXY will assume its maximum value of 1. Similarly, if
there is a negative relationship, rXY will be negative, with minimum value of –1.
34
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

If X and Y are independent, rXY will be equal to zero because sXY will be zero.

35
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

If there is a positive association between them, sXY, and hence rXY, will be positive. If there
is an exact positive linear relationship, rXY will assume its maximum value of 1.

36
 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

Correlation

 XY
population correlation  XY 
coefficient  X2  Y2

Similarly, if there is a negative relationship, rXY will be negative, with minimum value of –1.

37
Copyright Christopher Dougherty 2016.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.

The content of this slideshow comes from Section R.4 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/

Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.

2015.12.17