Introduction To Econometrics, 5 Edition: Chapter 3: Multiple Regression Analysis

Type author name/s here
Dougherty
Introduction to Econometrics,
5th edition
Chapter heading
Chapter 3: Multiple Regression
Analysis
© Christopher Dougherty, 2016. All rights reserved.

PRECISION OF THE MULTIPLE REGRESSION COEFFICIENTS
True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
This sequence investigates the variances and standard errors of the slope coefficients in a
model with two explanatory variables.
1

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
The expression for the variance of ̂ 2 is shown above. The expression for the variance of̂ 3
is the same, with the subscripts 2 and 3 interchanged.
2

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
The first factor in the expression is identical to that for the variance of the slope coefficient
in a simple regression model.
3

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
The variance of ̂ 2 depends on the variance of the disturbance term, the number of
observations, and the mean square deviation of X2 for exactly the same reasons as in a
simple regression model.
4

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
The difference is that in multiple regression analysis the expression is multiplied by a

factor which depends on the correlation between X2 and X3.
5

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
The higher is the correlation between the explanatory variables, positive or negative, the
greater will be the variance.
6

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
This is easy to understand intuitively. The greater the correlation, the harder it is to
discriminate between the effects of the explanatory variables on Y, and the less accurate
will be the regression estimates.
7

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
Note that the variance expression above is valid only for a model with two explanatory
variables. When there are more than two, the expression becomes much more complex
and it is sensible to switch to matrix algebra.
8

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
 u2 1
standard deviation of ˆ 2  
 X 2 i  X 2  1  rX 2 , X 3
2 2
The standard deviation of the distribution of ̂ 2 is of course given by the square root of its
variance.
9

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
 u2 1
standard deviation of ˆ 2  
 X 2 i  X 2  1  rX 2 , X 3
2 2
With the exception of the variance of u, we can calculate the components of the standard
deviation from the sample data.
10

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
1 2 nk 2
E   uˆ i   u
n  n
The variance of u has to be estimated. The mean square of the residuals provides a
consistent estimator, but it is biased downwards by a factor (n – k) / n , where k is the
number of parameters, in a finite sample.
11

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
1 2 nk 2 1
E   uˆ i   u ˆ 
2
 i
ˆ
u 2
n  n u
nk
Obviously we can obtain an unbiased estimator by dividing the sum of the squares of the
residuals by n – k instead of n. We denote this unbiased estimator ˆ u .
2
12

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3
 u2 1  u2 1
 2ˆ2    
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
1 2 nk 2 1
E   uˆ i   u ˆ 
2
 i
ˆ
u 2
n  n u
nk
ˆ u2 1 ˆ u2 1
s.e.  ˆ2     
 X 2i  X 2 
2
1  r 2
X2 ,X3 n MSD  X 2  1  r 2
X2 ,X3
Thus the estimate of the standard deviation of the probability distribution of ̂ 2 , known as
the standard error of ̂ 2 for short, is given by the expression above.
13
. reg EARNINGS S EXP if COLLBARG==1

----------------------------------------------------------------------------
Source | SS df MS Number of obs = 75
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
Residual | 7841.35558 72 108.907716 R-squared = 0.1159
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
We will use this expression to analyze why the standard error of S is larger for the union
subsample than for the non-union subsample in wage equation regressions using Data Set
21.
14

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
To select a subsample in Stata, you add an ‘if’ statement to a command. The COLLBARG
variable is equal to 1 for respondents whose rates of pay are determined by collective
bargaining, and it is 0 for the others.
15

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
Note that in tests for equality, Stata requires the = sign to be duplicated.
16

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
In the case of the union subsample, the standard error of S is 0.5365.
17

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 422) = 29.48
Model | 7270.82789 2 3635.41394 Prob > F = 0.0000
-----------+------------------------------ Adj R-squared = 0.1184
Total | 59314.065 424 139.891663 Root MSE = 11.105
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.866279 .2438803 7.65 0.000 1.386907 2.34565
EXP | 1.100186 .2223238 4.95 0.000 .6631858 1.537186
_cons | -15.9847 4.623791 -3.46 0.001 -25.07323 -6.896172
----------------------------------------------------------------------------
In the case of the non-union subsample, the standard error of S is 0.2439, less than half as
large.
18
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
ˆ u2 1 ˆ u2 1
 
s.e. ˆ2   
 

 X 2i  X 2   
2 2 2
1 rX2 ,X3 n MSD X 2 1 rX2 ,X3
We will explain the difference by looking at the components of the standard error. It is
convenient to start by rearranging the expression for the standard error as the product of
the four factors.
19
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Decomposition of the standard error of S
Component ˆ u n MSD(S) rS, EXP s.e.
Union 0.5365
Non-union 0.2439
Factor product
Union
Non-union
We will arrange the components of the standard error as a table.
20

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
1
ˆ u2  RSS
nk
We will start with ˆ u . Here is RSS for the union subsample.
21

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
1
ˆ u2  RSS
nk
There are 75 observations in the non-union subsample. k is equal to 3. Thus n – k is equal

to 72.
22

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 72) = 4.72
Model | 1027.91667 2 513.958336 Prob > F = 0.0119
-----------+------------------------------ Adj R-squared = 0.0913
Total | 8869.27225 74 119.85503 Root MSE = 10.436
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.42955 .536452 2.66 0.010 .3601522 2.498947
EXP | .1918676 .5901747 0.33 0.746 -.9846242 1.368359
_cons | 1.01708 10.84695 0.09 0.926 -20.60593 22.64009
----------------------------------------------------------------------------
1
ˆ u2  RSS
nk
RSS / (n – k) is equal to 108.908. To obtain ˆ u , we take the square root. This is 10.436.
23
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 0.5365
Non-union 0.2439
Factor product
Union
Non-union
We place this in the table, along with the number of observations.
24

----------------------------------------------------------------------------
-----------+------------------------------ F( 2, 422) = 29.48
Model | 7270.82789 2 3635.41394 Prob > F = 0.0000
-----------+------------------------------ Adj R-squared = 0.1184
Total | 59314.065 424 139.891663 Root MSE = 11.105
----------------------------------------------------------------------------
-----------+----------------------------------------------------------------
S | 1.866279 .2438803 7.65 0.000 1.386907 2.34565
EXP | 1.100186 .2223238 4.95 0.000 .6631858 1.537186
_cons | -15.9847 4.623791 -3.46 0.001 -25.07323 -6.896172
----------------------------------------------------------------------------
Similarly, in the case of the non-union subsample, ˆ u is the square root of 123.325, which is
11.105. We also note that the number of observations in that subsample is 425.
25
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 0.5365
Non-union 11.105 425 0.2439
Factor product
Union
Non-union
We place these in the table.
26
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 0.5365
Non-union 11.105 425 7.3467 0.2439
Factor product
Union
Non-union
We calculate the mean square deviation of S for the two subsamples from the sample data.
27
. cor S EXP if COLLBARG==1

(obs=75)
| S EXP
--------+------------------
S | 1.0000
EXP | -0.5866 1.0000
. cor S EXP if COLLBARG==0

(obs=425)
| S EXP
--------+------------------
S | 1.0000
EXP | -0.5796 1.0000
The correlation coefficients for S and EXP are –0.5866 and –0.5796 for the union and non-
union subsamples, respectively. (Note that "cor" is the Stata command for computing
correlations.)
28
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union
Non-union
These entries complete the top half of the table. We will now look at the impact of each
item on the standard error, using the mathematical expression at the top.
29
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436
Non-union 11.105
The ˆ u components need no modification. It is a little larger for the non-union subsample,
and so has an adverse effect on the standard error.
30
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155
Non-union 11.105 0.0485
The number of observations is much larger for the non-union subsample, so the second
factor is much smaller than that for the union subsample.
31
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155 0.3605
Non-union 11.105 0.0485 0.3689
Perhaps surprisingly, the variance in schooling is similar for the two subsamples.
32
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155 0.3605 1.2348
Non-union 11.105 0.0485 0.3689 1.2271
The correlation between schooling and work experience is also similar for the two
subsamples. Note that the sign of the correlation makes no difference since it is squared.
33
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155 0.3605 1.2348 0.5366
Non-union 11.105 0.0485 0.3689 1.2271 0.2439
Multiplying the four factors together, we obtain the standard errors. (The discrepancy in
the last digit of the union standard error has been caused by rounding error.)
34
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155 0.3605 1.2348 0.5366
Non-union 11.105 0.0485 0.3689 1.2271 0.2439
We see that the reason that the standard error is smaller for the non-union subsample is
that there are far more observations than in the non-union subsample. Otherwise the
standard errors would have been about the same.
35
1 1 1
s.e.  ˆ2   ˆ u   
n MSD( X 2 ) 1  rX22 , X 3
Union 10.436 75 7.6932 –0.5866 0.5365
Non-union 11.105 425 7.3467 –0.5796 0.2439
Factor product
Union 10.436 0.1155 0.3605 1.2348 0.5366
Non-union 11.105 0.0485 0.3689 1.2271 0.2439
The goodness of fit, as measured by ˆ u , is slightly inferior for the non-union sample and
this has a marginal offsetting effect. The other two factors are very similar for the two
subsamples.
36
Copyright Christopher Dougherty 2016.
These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section 3.3 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
www.oxfordtextbooks.co.uk/orc/dougherty5e/.
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2016.04.29

Introduction To Econometrics, 5 Edition: Chapter 3: Multiple Regression Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Econometrics, 5 Edition: Chapter 3: Multiple Regression Analysis

Uploaded by

Copyright:

Available Formats

Type author name/s here

© Christopher Dougherty, 2016. All rights reserved.

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

The difference is that in multiple regression analysis the expression is multiplied by a

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

True model Fitted model

. reg EARNINGS S EXP if COLLBARG==1

. reg EARNINGS S EXP if COLLBARG==1

. reg EARNINGS S EXP if COLLBARG==1

. reg EARNINGS S EXP if COLLBARG==1

In the case of the union subsample, the standard error of S is 0.5365.

. reg EARNINGS S EXP if COLLBARG==0

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

We will arrange the components of the standard error as a table.

. reg EARNINGS S EXP if COLLBARG==1

We will start with ˆ u . Here is RSS for the union subsample.

. reg EARNINGS S EXP if COLLBARG==1

There are 75 observations in the non-union subsample. k is equal to 3. Thus n – k is equal

. reg EARNINGS S EXP if COLLBARG==1

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 0.5365

We place this in the table, along with the number of observations.

. reg EARNINGS S EXP if COLLBARG==0

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 0.5365

Non-union 11.105 425 0.2439

We place these in the table.

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 7.6932 0.5365

Non-union 11.105 425 7.3467 0.2439

. cor S EXP if COLLBARG==1

. cor S EXP if COLLBARG==0

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 7.6932 –0.5866 0.5365

Non-union 11.105 425 7.3467 –0.5796 0.2439

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 7.6932 –0.5866 0.5365

Non-union 11.105 425 7.3467 –0.5796 0.2439

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 7.6932 –0.5866 0.5365

Non-union 11.105 425 7.3467 –0.5796 0.2439

Union 10.436 0.1155

Non-union 11.105 0.0485

Decomposition of the standard error of S

Component ˆ u n MSD(S) rS, EXP s.e.

Union 10.436 75 7.6932 –0.5866 0.5365