Multiple Regression Analysis:

Estimation
Prepared by Rizka Isnaini Husna, S.E.

Econometrics Tutorial for Regular 65 Class

Outline
1. Review: Definition of Multiple Regression Model
2. Review: Interpretation of Multiple Regression
3. Review: Key Assumptions in Multiple Linear Regression
4. Review: Omitted Variable Bias
5. STATA exercise
Definition of the Multiple Regression
Simple Regression Model Multiple Regression Model
One-to-one Many-to-one

x1
x y
x2 y

x3
…or more
Definition of the Multiple Regression
A model linear where the dependent variable is a function of
independent variables plus an error term.

𝑦 = 𝛽0 + 𝛽1 𝑥1 + 𝛽2 𝑥2 + 𝛽3 𝑥3 + ⋯ + 𝛽𝑘 𝑥𝑘 + 𝑢

Example: wage equation

log 𝑤𝑎𝑔𝑒 = 𝛽0 + 𝛽1 𝑒𝑑𝑢𝑐 + 𝛽2 𝑒𝑥𝑝𝑒𝑟 + 𝛽3 𝑒𝑥𝑝𝑒𝑟 2 + 𝑢

Interpretation of the Multiple Regression

Consider an estimated equation

෢0 + 𝛽
𝑦ො = 𝛽 ෢1 𝑥1 + 𝛽
෢2 𝑥2
Interpretation of the Multiple Regression

Consider an estimated equation

෢0 + 𝛽
𝑦ො = 𝛽 ෢1 𝑥1 + 𝛽
෢2 𝑥2

The estimate effect of 𝑥1 on 𝑦 is

෢1 ∆𝑥1 , holding 𝑥2 fixed

∆𝑦ො = 𝛽
Interpretation of the Multiple Regression

Consider an estimated equation

෢0 + 𝛽
𝑦ො = 𝛽 ෢1 𝑥1 + 𝛽
෢2 𝑥2

The estimate effect of 𝑥1 on 𝑦 is

෢1 ∆𝑥1 , holding 𝑥2 fixed

∆𝑦ො = 𝛽

Similarly, the estimate effect of 𝑥2 on 𝑦 is

෢2 ∆𝑥2 , holding 𝑥1 fixed

∆𝑦ො = 𝛽
Interpretation of the Multiple Regression

Consider an estimated equation

෢0 + 𝛽
𝑦ො = 𝛽 ෢1 𝑥1 + 𝛽
෢2 𝑥2

The estimate effect of 𝑥1 on 𝑦 is

෢1 ∆𝑥1 , holding 𝑥2 fixed

∆𝑦ො = 𝛽

Similarly, the estimate effect of 𝑥2 on 𝑦 is Partial effect

෢2 ∆𝑥2 , holding 𝑥1 fixed

∆𝑦ො = 𝛽
Interpretation of the Multiple Regression
෣
log(𝑤𝑎𝑔𝑒) = 0.128 + 0.09𝑒𝑑𝑢𝑐 + 0.041𝑒𝑥𝑝𝑒𝑟 − 0.0007𝑒𝑥𝑝𝑒𝑟 2

Questions
1. What is the estimated effect of education on wage?
2. What is the estimated effect of experience on wage?
 Key Assumptions in Multiple Linear Regression (MLR)
1. Linear in Parameters
2. Random Sampling
3. No Perfect Collinearity
4. Zero Conditional Mean
5. Homoskedasticity
 Key Assumptions in Multiple Linear Regression (MLR)
3. No Perfect Collinearity
In the sample (and therefore in the population), there are no exact linear
relationships among the independent variables.

• The independent variables are allowed to be correlated; they just cannot

be perfectly correlated.
• Violation: perfect collinearity. Example: put the same variable measured in
different units into equation.
 Key Assumptions in Multiple Linear Regression (MLR)
Quick test.
Which one of these equations violates no perfect collinearity assumption?

1. 𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 = 𝛽0 + 𝛽1 𝑖𝑛𝑐𝑜𝑚𝑒 + 𝛽2 𝑖𝑛𝑐𝑜𝑚𝑒 2 + 𝑢

2. 𝑣𝑜𝑡𝑒𝐴 = 𝛽0 + 𝛽1 𝑒𝑥𝑝𝑒𝑛𝑑𝐴 + 𝛽2 𝑒𝑥𝑝𝑒𝑛𝑑𝐵 + 𝛽3 𝑡𝑜𝑡𝑒𝑥𝑝𝑒𝑛𝑑 + 𝑢
Where totexpend = expendA + expendB
 Key Assumptions in Multiple Linear Regression (MLR)
4. Zero Conditional Mean
The error u has an expected value of zero given any value of
independent variables.
What does this mean?
• All factors in the error term must be uncorrelated with the independent
variables.
• Violation: omitted variable bias (discussed later)
 Key Assumptions in Multiple Linear Regression (MLR)
5. Homoskedasticity
The error u has the same variance given any value of the
independent variables.
What does this mean?
• Variance of error u does not depend on values of independent variables.
• Violation: heteroskedasticity
 Omitted Variable Bias
• Occurs when we omit a relevant variable that actually belongs in the true
model and this variable correlated with independent variable
• Example: wage equation
log 𝑤𝑎𝑔𝑒 = 𝛽0 + 𝛽1 𝑒𝑑𝑢𝑐 + 𝛽2 𝑎𝑏𝑖𝑙 + 𝑢
Since ability is not observed, we instead estimate
log 𝑤𝑎𝑔𝑒 = 𝛽0 + 𝛽1 𝑒𝑑𝑢𝑐 + 𝑣
where 𝑣 = 𝛽2 𝑎𝑏𝑖𝑙 + 𝑢
 Omitted Variable Bias
• More ability leads to higher productivity and therefore higher wages
• Also, there are reasons to believe that educ and abil are positively
correlated: on average, individuals with more innate ability choose higher
levels of education.
• Thus, the estimates of educ on wage is biased.
• We can predict the direction of the bias.
 Omitted Variable Bias
Summary of bias in β1 when x2 is omitted

Corr(x1,x2) > 0 Corr(x1,x2) < 0

β2> 0 Positive bias Negative bias
β2< 0 Negative bias Positive bias

When the two correlations go in the same direction, the bias is positive.
When opposite, the bias is negative.
 STATA exercise:
Use file CEOSAL2
1. Estimate the effect of firm sales and market value on CEO salary. Make
the model of the constant elasticity variety for both independent
variables. Write the results out in equation form.
2. Add profits to the model. Would you say that these firm performance
variables explain most of the variation in CEO salaries?
3. Add the variable ceoten. What is the estimated percentage return for
another year of CEO tenure, holding other factors fixed?
4. Find the correlation coefficient between the variables log(mktval) and
profits. Are these variables highly correlated?
 STATA exercise:
Use life_expec.xls
1. Import the file into STATA
2. What type of this data?
3. Change the name of variable “LEX” to “life_expec”
4. Label variable “GDP” to “GDP per capita in US$”
5. Regress PuHEX, PrHEX, GDP, POP, CO2 on life_expec
6. Save the data
Source
Wooldridge, J. M. (2016). Introductory Econometrics, 6e. Boston:
Cengage Learning.