Chapter 4: Multiple Linear

Regression (Inference)

1. OLS: Sampling Distribution

2. Hypothesis Testing: Single Parameter
3. Hypothesis Testing: Single Linear Combination of
Parameters
4. Hypothesis Testing: Multiple Linear Restrictions

Ch04 Page 1
OLS: Sampling
Distribution
Assumption MLR.6 (Normality)
The population error is independent of the explanatory variables and is normally distributed with zero mean and
variance Normal .

We have that for the population model

,

Then, under assumption MLR.4, MLR.5, and MLR.6:

,
,

Assumptions MLR.1 - MLR.5 are known as the

Gauss-Markov Assumptions.
Assumptions MLR.1 - MLR.6 are known as the
Classical Linear Model Assumptions.

Theorem (Sampling Distribution of the OLS)

Under the CLM assumptions MLR.1 through MLR.6, conditional on the Recall that under MLR.1-MLR.5 one has
sample values of the independent variables,
.
Therefore, ⛔ The error variance, , is unknown
and therefore needs to be estimated. ⛔
.

Theorem ( Distribution for the Standardized Estimators)

Under the CLM assumptions MLR.1 through MLR.6, The Distribution
Let have a standard normal distribution and
let have a chi-square distribution with
. degrees of freedom. Further assume that and
are independent. Define, then,
has a distribution with degrees of
where is the number of unknown parameters in the population freedom.

Ch04 Page 2
 . degrees of freedom. Further assume that and
are independent. Define, then,
has a distribution with degrees of
where is the number of unknown parameters in the population freedom.
model , ( slope parameters and the .
intercept ) and is the degrees of freedom ( ).

Ch04 Page 3
Hypothesis Testing: Single
Parameter
1. H0: β_l=0
2. H0: β_l=a_l
3. p-Values (t Test)
4. Confidence Intervals

Ch04 Page 4
 :
Recall the population model can be written as
.

We are interested in testing the null hypothesis

The population parameter is equal to zero, i.e. after controlling for the other independent variables, there is no effect of on .

Against three potential alternative hypothesis, namely

These are not true
,
One-sided Hypothesis
,
,
Two-sided Hypothesis .

Test Statistic ( statistic or ratio):

The statistic will be used to test the above null hypothesis. The farther the estimated coefficient is away from zero, the less likely it is that the null
hypothesis holds true. Therefore, we will reject the null hypothesis if the ratio is 'sufficiently large', i.e., far away from zero. But what does “far” away
from zero mean? Critical value

Under the null hypothesis one can define a significance level (or 'level') which is .

One-Sided Alternative:
Suppose we have decided on a significance level, as this is the most popular
choice. Thus, we are willing to mistakenly reject when it is true of the time.
➡ While has a distribution under so that it has zero mean - under the
alternative , the expected value of is positive.
Thus, we are looking for a "sufficiently large" positive value of in order to
reject in favor of Negative values of provide no evidence
in favor of .

The definition of "sufficiently large," with a

significance level, is the 95th percentile in a
distribution with degrees of freedom;
denote this by .

The rejection rule is that is rejected in favor

of at the significance level if
.

Example: Wage equation

Test whether, after controlling for education and tenure, higher work experience leads
to higher hourly wages, i.e.,

against .

: 522 = 526-(3+1)
5% Critical Value: 1.645

Ch04 Page 5
 5% Critical Value: 1.645
1% Critical Value: 2.326
statistic:

, or , is statistically significant even at the level.

One-Sided Alternative:
Suppose we have decided on a significance level, as this is the most popular choice.
Thus, we are willing to mistakenly reject when it is true of the time.
➡ While has a distribution under so that it has zero mean - under the alternative
, the expected value of is negative.
Thus, we are looking for a "sufficiently large" negative value of in order to reject
in favor of Positive values of provide no evidence in favor of
.

The definition of "sufficiently large," with a

significance level, is the 5th percentile in a
distribution with degrees of freedom;
denote this by .

The rejection rule is that is rejected in favor

of at the significance level if
.

Example: Student Performance and Class Size

Test whether, after controlling for annual teacher compensation ( ) and the number
of staff per one thousand students ( ), lower student enrollment ( ) leads to higher
standardized tenth-grade math test ( )., i.e.,

against .

: 404 = 408-(3+1)
5% Critical Value: -1.645
1% Critical Value: -2.326
statistic:

Ch04 Page 6
 , or , is statistically insignificant even at the level.

We fail to reject the at the level of significance.

Two-Sided Alternative:
Suppose we have decided on a significance level, as this is the most popular choice.
Thus, we are willing to mistakenly reject when it is true of the time.
➡ While has a distribution under so that it has zero mean - under the alternative
, the expected value of can be either negative or positive.
Thus, we are looking for a "sufficiently large" non-zero value of in order to reject
in favor of Close-to-zero values of provide no evidence in favor
of .

The rejection rule is that is rejected in favor

of at the significance level if
.

For a two-tailed test, is chosen to make the area in each tail of the distribution equal
2.5%. In other words, is the 97.5th percentile in the distribution with degrees
of freedom.

Example: Determinant of College GPA

: 137 = 141-(3+1)
5% Critical Value: 1.960
1% Critical Value: 2.576

statistic:
against

against

Ch04 Page 7
 against

against

, (or ), , (or ), are statistically significant at the 1 level.

, (or ) is statistically insignificant at the 5 level.

Ch04 Page 8
 :
Generally, if the null hypothesis is
Usually
, or in
economics.

where is our hypothesized value (unknown) of , then the appropriate statistic is

This statistic is distributed as a . The usual ratio is obtained by setting .

This measures how many standard deviations is away from the hypothesized values .

We can use the general statistic to test against one-sided or two-sided alternatives.

We find the critical value for a one-sided or two-sided alternatives exactly as before!
➡ The difference is in how we compute the statistic, not in how we obtain the appropriate .⬅

Example: Campus Crime and Enrollment

against .

: 95 = 97-(1+1)
5% Critical Value: 1.662
1% Critical Value: 2.368
statistic:

, or , is statistically different from one at

the level.

Ch04 Page 9
 -Values ( Test)
Definition
Given the observed value of the statistic, what is the smallest significance
level at which the null hypothesis would be rejected? This level is known as
the -value for the test.

In the example with and , the -value is

computed as

❗ It is clear that a null hypothesis is rejected if and only if the

corresponding -value is smaller than the significance level.

The -value commonly reported by default in most

statistical packages refer to the -value associated with the
standard ratio that test .

Ch04 Page 10
Guidelines for discussing economic and statistical significance
If a variable is statistically significant, discuss the magnitude of the coefficient to get an idea of its economic or practical importance
The fact that a coefficient is statistically significant does not necessarily mean it is economically or practically significant!
If a variable is statistically and economically important but has the "wrong" sign, the regression model might be misspecified
If a variable is statistically insignificant at the usual levels 10%, 5%, or 1% one may think of dropping it from the regression
If the sample size is small, effects might be imprecisely estimated so that the case for dropping insignificant variables is less strong

Ch04 Page 11
Confidence Intervals
Definition
Confidence intervals are also called interval estimates because they provide a
range of likely values for the population parameter, , and not just a point
estimate .

Let be the critical value associated with at two-sided test at the 95% level of significance, then by construction

,
,
,
.

Lower Bound:
Upper Bound:

For very large sample sizes one can work with the following:

99% level of significance:

95% level of significance:
90% level of significance:

Performing Two-Tailed Hypothesis Tests with Confidence Intervals

Once a confidence interval is constructed, it is easy to carry out two-tailed hypotheses
tests. If the null hypothesis is , then is rejected in favour of if
does not belong to the confidence interval at the pre-set level of significance.

Example: Model of R&D Expenditures

Unity is included in the 90% confidence interval for

which means that we cannot reject the
against at the 10% level of significance.

Ch04 Page 12
Hypothesis Testing:
Single Linear
Combination of
Parameters
Consider the multiple linear regression model:

We are now interested in testing the hypothesis

, ➡ ,
. .

Let us define , then

, ➡ , ➡ ,
. . .

Now notice that , and therefore we can write our original model as

.
.
.

Therefore one can test the , by simply running a modified regression of on , , , , and then do a simple
test on the coefficient multiplying on this augmented regression.

Example: Return to a College Education

The unrestricted estimated model is:

,
.

It is equivalent to testing

,
.

in the modified regression

The reported -value is 0.142 but this is based on a two-tailed

alternative, so for a one-tailed alternative the corresponding -
value is 0.071=0.142/2. Therefore we reject the in favor of
the alternative at the 10% level of significance (but not at the
5%).

Ch04 Page 13
Although this is correct
,

These are not true

,

Ch04 Page 14
Hypothesis Testing:
Multiple Linear
Restrictions
1. Testing Exclusion Restrictions
2. p-Values (F Test)
3. General Linear Restrictions

Ch04 Page 15
Testing Exclusion
Restrictions
Recall the population model can be written as
.
Since setting these
We are interested in testing the multiple null hypothesis parameters equal to zero
effectively excludes regressors ,
, , . , and then we call these
"exclusion restrictions."
Against the alternative hypothesis is not true.

is true (restricted model) Regress on , , , , , In this example,

is true (unrestricted model) Regress on , , , , , , , , and .

Test Statistic ( statistic or ratio):

Define the number of restrictions in

as

The statistic will be used to test the above null hypothesis. The farther the estimated coefficients are away from zero, the less
likely it is that the null hypothesis holds true. Therefore, we will reject the null hypothesis if the ratio is 'sufficiently large', i.e.,
substantially larger than zero. But what does “substantially larger” than zero mean? Critical value

Under the null hypothesis one can define a significance level (or 'level') which is .

It can be shown that under and assuming that MLR.1-MLR.6 (CLM) assumptions The Distribution
hold, then the statistic is distributed as an random variable with degrees of Let and and assume that and
freedom in the numerator and degrees of freedom in the denominator, are independent. Then has
i.e., an distribution with degrees of freedom.
. .
The integer is called the numerator degrees of
freedom, and is called the denominator degrees of
freedom.

Suppose we have decided on a significance level, as this is the

most popular choice. Thus, we are willing to mistakenly reject
when it is true of the time.

The definition of "substantially larger," with a

significance level, is the 95th percentile in a
distribution with degrees of freedom in the
numerator and degrees of freedom in
the denominator; denote this by .

The rejection rule is that is rejected in favor

of at the significance level if
.

Example: Major League Baseball Players' Salaries

, , ,
is not true.

Ch04 Page 16
 : 347 = 353-(5+1)
: 350 = 353-(5+1-3)
1% Critical Value: 3.78
statistic:

We reject the hypothesis that , , and have

no effect on salary at the 1% level of significance.

The -Squared Form of the Statistic

Recall that and therefore

Restricted: ,
Unrestricted: .

✔ In our example:
Therefore one can re-write the statistics in terms of the restricted ( ) and unrestricted ( ) :

The F Statistic for Overall Significance of a Regression

A special set of exclusion restrictions is routinely tested by most regression packages. These restrictions have the same interpretation,
regardless of the model. In the model with independent variables:

, , , do not help to explain , ➡ ,

is not true. is not true.

It turns out that in this case one has and therefore

✔ In our example:

Ch04 Page 17
-Values ( Test)

Definition
It is the probability of observing a value of at least as large as we did,
given that the null hypothesis is true, i.e.,

where denotes the actual statistic, and denote the distribution

with degrees of freedom in the numerator, and degrees of
freedom in the denominator.

Ch04 Page 18
General Linear
Restrictions
Consider the following model

We are now interested in testing the hypothesis

, There are
is not true. restrictions.

is true (restricted model) Regress on a constant

is true (unrestricted model) Regress on , , , .

We cannot use the

version of the statistics
for this type of general
linear restrictions

Ch04 Page 19