You are on page 1of 54

Economics A Morden Approach

-
Prof. Wang Li
1
The Simple Regression
Model (1)
y = |
0
+ |
1
x + u
Economics A Morden Approach -
Prof. Wang Li
2
Chapter Outline
 Definition of the Simple Regression Model
 Deriving the Ordinary Least Squares
Estimates
 Mechanics of OLS
 Units of Measurement and Functional Form
 Expected Values and Variances of the OLS
estimators
 Regression through the Origin
Economics A Morden Approach -
Prof. Wang Li
3
Lecture Outline
 Simple linear regression model
 Some terminology
 A simple assumption
 Zero Conditional Mean Assumption
 Ordinary Least Squares
 Deriving OLS Estimates
 A brief introduction to EViews
Economics A Morden Approach -
Prof. Wang Li
4
 Equation y = |
0
+ |
1
x + u
has only one nonconstant regressor x,
it is called a simple linear regression
model,
or two-variables regression model,
or bivariate linear regression model.
∆y = |
1
∆x if ∆u = 0
 Thus, the change in y is simply |
1
multiplied
by the change in x.
Simple linear regression model
Economics A Morden Approach -
Prof. Wang Li
5
Some Terminology
 In the linear regression model,
where y = |
0
+ |
1
x + u, we typically refer to
left-hand side variable y as the
– Dependent Variable, or
– Explained Variable, or
– Response Variable, or
– Predicted Variable, or
– Regressand
Economics A Morden Approach -
Prof. Wang Li
6
 In the simple linear regression of y on x,
we typically refer to right-hand side
variable x as the
– Independent Variable, or
– Explanatory Variable, or
– Control Variable , or
– Predictor Variable , or
– Regressor
Some Terminology, cont.
Economics A Morden Approach -
Prof. Wang Li
7
 The coefficients |
0
, |
1
are called the
regression coefficients.
|
0
is also called the constant term or the
intercept term, or intercept parameter.
|
1
represents the marginal effects of the
regressor, x. It is also called the slope
parameter.
Some Terminology, cont.
Economics A Morden Approach -
Prof. Wang Li
8
 The variable u is called the error term
or disturbance in the relationship.
 It represents factors other than x that
can affect y.


Some Terminology, cont.
u education Earnings + + =
1 0
| |
Economics A Morden Approach -
Prof. Wang Li
9
 Meaning of linear:
linear means linear in parameters, not
necessarily mean that y and x must have a
linear relationship.
 There are many cases that y and x have
nonlinear relationship, but after some
transformation, they are linear in
parameters.
For example, y=e
|
0
+|
1
x+u

natural log: ln(y)=|
0
+|
1
x+u
Meaning of linear
Economics A Morden Approach -
Prof. Wang Li
10
Example of
Returns to Education
u education wage + + =
1 0
| |
 A model of human capital investment
implies getting more education should
lead to higher earnings

 education , wage and u are random
variables in the equation.

Economics A Morden Approach -
Prof. Wang Li
11
A Simple Assumption
 The average value of u, the error term, in the
population is 0. That is,
expected value of u E(u) = 0
 It it restrictive?
This is not a restrictive assumption, since we
can always use |
0
to normalize E(u) to 0.
If for example, E(u)=5. Then
y = (|
0
+5)+ |
1
x + (u-5),
therefore, E(u’)=E(u-5)=0.
• As long as the intercept |
0
is included in the
equation, the assumption of E(u) = 0 holds.
Economics A Morden Approach -
Prof. Wang Li
12
Zero Conditional Mean Assumption
 We need to make a crucial assumption about
how u and x are related
 This assumption involves the expected
conditional distribution value of u given any
value of x.
E(u|x) = E(u)
– It means that, for any given value of x, the
average of the unobservables is the same,
– that u and x are completely unrelated.
therefore E(u|x) must equal the average
value of u in the entire population.
Economics A Morden Approach -
Prof. Wang Li
13
 What does it mean?
u and x are completely unrelated
In the example of education, suppose u
represents innate ability, conditional mean
assumption E(u|x) = E(u) means
E(ability|edu=6)=E(ability|edu=18)
The average level of ability is the same
regardless of years of education.
 Since we have assumed E(u) = 0, therefore,
E(u|x) = E(u) = 0
Zero Conditional Mean Assumption
Economics A Morden Approach -
Prof. Wang Li
14
 Use x to predict the average value of y
– take expectations conditional on x
E[y|x] = E[|
0
+ |
1
x+u|x]
= |
0
+ |
1
x+ E[u|x]
 Zero Conditional mean assumption:
E(u|x) = E(u) = 0
We got, E[y|x] = |
0
+ |
1
x
Its important implication:
The population regression function is a linear
function of x, where for any x the
distribution of y is centered about E(y|x).
Zero Conditional Mean Assumption
Economics A Morden Approach -
Prof. Wang Li
15
.
.
x
1
=5 x
2
=10
E(y|x) = |
0
+|
1
x
y
f(y)
Conditional
distribution of y
given x
Zero Conditional Mean Assumption
The population
regression function
(PRF)
Economics A Morden Approach -
Prof. Wang Li
16
Deriving the Ordinary Least
Squares Estimates
 Basic idea of regression is to estimate the
population parameters from a sample set.
 Let {(x
i
,y
i
): i=1, …,n} denote a random
sample of size n from the population.
 Since these data come from simple
regression model, for each observation in
this sample, it will be the case that
y
i
= |
0
+ |
1
x
i
+ u
i
Economics A Morden Approach -
Prof. Wang Li
17
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
u
1
u
2
u
3
u
4
x
y
Population regression line, sample data points
and the associated error terms
PRF
E(y|x) = |
0
+ |
1
x
Economics A Morden Approach -
Prof. Wang Li
18
 We must decide how to use these data to
obtain estimates of the intercept and slope
in the population regression
 The method of Ordinary Least Squares (OLS)
plays a major role to obtain estimators.
 OLS estimators are the best linear unbiased
estimators.
 The name “ordinary least squares” comes
from the fact that these estimates minimize
the sum of squared residuals.
Deriving the Ordinary Least
Squares Estimates
Economics A Morden Approach -
Prof. Wang Li
19
 To derive the OLS estimator we need to
realize that our main assumption of
E(u|x) = E(u) = 0 also implies that
u and x are uncorrelated, independent
Covariance Cov(x,u) = E(xu) = 0
 Why? Remember from basic probability that
Cov(X,Y) = E(XY) – E(X)E(Y) and
E(XY)= E(X)E(Y) when x and y are independent.
∵E(u|x) = E(u)=0 ∴Cov(x,u) = E(xu) = 0
Deriving the Ordinary Least
Squares Estimates
Economics A Morden Approach -
Prof. Wang Li
20
 Equations E(u) = 0 and E(xu) = 0 can be
written just in terms of x, y, |
0
and |
1
:
since u = y – |
0
– |
1
x
Zero mean: E(y – |
0
– |
1
x) = 0
Zero conditional mean: E[x(y – |
0
– |
1
x)] = 0
 This two Equations imply 2 restrictions on
the joint probability distribution of (x,y) in
the population.
 Technically, values of the independent
variable is the same as treating the x
i
as
fixed in repeated samples.
Deriving OLS continued
Economics A Morden Approach -
Prof. Wang Li
21
 The method of moments approach to
estimation implies imposing the population
moment restrictions on the sample moments
 What does this mean?
Recall that for E(X), the mean of a population
distribution, a sample estimator of E(X) is
simply the arithmetic mean of the sample
 These are called moment restrictions.
Deriving OLS using M.O.M
Economics A Morden Approach -
Prof. Wang Li
22
 We want to choose values of the parameters that
will ensure that the sample versions of our
moment restrictions are true
E(y – |
0
– |
1
x) = 0
E[x(y – |
0
– |
1
x)] = 0
 The sample versions of our 2 restrictions above
are as follows:
( )
( )
¦
¦
¹
¦
¦
´
¦
= ÷ ÷
= ÷ ÷
¿
¿
=
÷
=
÷
0
ˆ ˆ
0
ˆ ˆ
1
1 0
1
1
1 0
1
n
i
i i i
n
i
i i
x y x n
x y n
| |
| |
Deriving OLS using M.O.M
 Then we are going to solve the equation set.
Economics A Morden Approach -
Prof. Wang Li
23
More Derivation of OLS
 Given the definition of a sample mean, and
properties of summation, we can rewrite
the 2 restrictions as follows:
x y
x y
1 0
1 0
ˆ ˆ
or
,
ˆ ˆ
| |
| |
÷ =
+ =
where y¯ is the sample average
of the y
i
24
More Derivation of OLS
( ) ( )
( ) ( )
( )( ) ( )
¿ ¿
¿ ¿
¿
= =
= =
=
÷ = ÷ ÷
÷ = ÷
= ÷ ÷ ÷
n
i
i i
n
i
i
n
i
i i
n
i
i i
n
i
i i i
x x y y x x
x x x y y x
x x y y x
1
2
1
1
1
1
1
1
1 1
ˆ
ˆ
0
ˆ ˆ
|
|
| |
 The sample versions of the second restriction
( ) 0
ˆ ˆ
1
1 0
1
= ÷ ÷
¿
=
÷
n
i
i i i
x y x n | |
From basic properties of the summation operator


( ) ( )( ) y y x x y y x
i
n
i
i
n
i
i i
÷ ÷ = ÷
¿ ¿
= = 1 1


x y
1 0
ˆ ˆ
| | ÷ =
Economics A Morden Approach -
Prof. Wang Li
25
the OLS estimated slope
( )( )
( )
( ) 0 that provided
ˆ
1
2
1
2
1
1
> ÷
÷
÷ ÷
=
¿
¿
¿
=
=
=
n
i
i
n
i
i
n
i
i i
x x
x x
y y x x
|
So the OLS estimated slope is
the numerator is
sample covariance
between x and y
The denominator is the
sample variance of x
• Dividing both the numerator and the
denominator by n-1 will make sense.
Economics A Morden Approach -
Prof. Wang Li
26
Summary of OLS slope estimate
 The slope estimate is the sample
covariance between x and y divided by the
sample variance of x.
 If x and y are positively correlated, the
slope will be positive.
 If x and y are negatively correlated, the
slope will be negative.
 Only need x to vary in our sample.
Economics A Morden Approach -
Prof. Wang Li
27
the OLS estimated intercept
( )( )
( )
¿
¿
=
=
÷
÷ ÷
=
n
i
i
n
i
i i
x x
y y x x
1
2
1
1
ˆ
|
 Once we have the slope estimate:
x y
1 0
ˆ ˆ
| | ÷ =
 It is straightforward to obtain the
intercept estimate, given y¯ and x¯.
Economics A Morden Approach -
Prof. Wang Li
28
More OLS
 Intuitively, OLS is fitting a line through
the sample points.
 The residual, û, is an estimate of the
error term, u, and is the difference
between the fitted line (sample regression
function) and the sample point.

i i i
u x y
ˆ
ˆ ˆ
1 0
+ + = | |
i i
x y
1 0
ˆ ˆ
ˆ
| | + =
Sample regression function
Economics A Morden Approach -
Prof. Wang Li
29
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
û
1
û
2
û
3
û
4
x
y
Sample regression line, sample data points
and the associated estimated error terms
(residuals)
x y
1 0
ˆ ˆ
ˆ
| | + =
OSL regression line
The notation yˆ emphasizes
that the predicted values
from this equation
are estimates
Economics A Morden Approach -
Prof. Wang Li
30
Alternate approach
 The residual for observation i is the
difference between the actual y
i
and its
fitted value:
( ) ( )
¿ ¿
= =
÷ ÷ = =
n
i
i i
n
i
i
x y u L
1
2
1 0
1
2
ˆ ˆ
ˆ
| |
 Given the intuitive idea of fitting a line, we
can set up a formal minimization problem.
 That is, we want to choose our parameters
minimizing the following:
i i i i i
x y y y u
1 0
ˆ ˆ
ˆ ˆ
| | ÷ ÷ = ÷ =
Economics A Morden Approach -
Prof. Wang Li
31
Alternate approach, continued
 We use calculus to solve the minimization
problem for the two parameters , The
first order conditions:
( )
( ) 0
ˆ ˆ
2
ˆ
0
ˆ ˆ
2
ˆ
1
1 0
1
1
1 0
0
= ÷ ÷ ÷ =
c
c
= ÷ ÷ ÷ =
c
c
¿
¿
=
=
n
i
i i i
n
i
i i
x y x
L
x y
L
| |
|
| |
|
 That is the following:
( ) ( )
¿ ¿
= =
÷ ÷ =
n
1 i
2
i 1 0 i
n
1 i
2
i
β β
x β β y u
1 0
ˆ ˆ
ˆ min
)
ˆ
,
ˆ
(
Economics A Morden Approach -
Prof. Wang Li
32
Alternate approach, continued
 These are the same as we have before
(sample versions of 2 restrictions, moment
restrictions ), multiplied by -2n.
( )
( )
¦
¦
¹
¦
¦
´
¦
= ÷ ÷
= ÷ ÷
¿
¿
=
=
0
ˆ ˆ
0
ˆ ˆ
1
1 0
1
1 0
n
i
i i i
n
i
i i
x y x
x y
| |
| |
 So we call these are The first order
conditions restrictions .
Economics A Morden Approach -
Prof. Wang Li
33
 {(x
i
,y
i
): i=1, …,n} are actual values in a
random sample set from the population.
 Given a random sample set, the OLS method
is used to estimate the slope and intercept
parameters in the population model.
 In population regression function
E(y|x) = |
0
+ |
1
x , the |
0
and |
1
are true values.
Summary of OLS estimate
 In the OSL regression line
are estimated values by given random
sample set. are predicted values forming
the fitted line.
1 0
ˆ
,
ˆ
| |
x y
1 0
ˆ ˆ
ˆ
| | + =
y
ˆ
Which of the followings are right?
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
t t
X Y | o + =
t t t
X Y µ | o + + =
√(2)



√(6)
√(7)
t t t
X Y µ | o + + =
ˆ
ˆ
t t t
X Y µ | o + + =
ˆ
ˆ
ˆ
t t
X Y | o
ˆ
ˆ
+ =
t t
X Y | o
ˆ
ˆ
ˆ
+ =
t t t
X Y µ | o
ˆ
ˆ
ˆ
+ + =
t t t
X Y µ | o
ˆ
ˆ
ˆ
ˆ
+ + =
n t , , 2 , 1 =
Checking Understanding
Economics A Morden Approach -
Prof. Wang Li
35
Lecture Summary
 Introduce the simple linear regression
model.
 Introduce the method of Ordinary Least
Squares to estimate the slope and
intercept parameters using data from a
random sample.
 The name “ordinary least squares” comes
from the fact that these estimates
minimize the sum of squared residuals.
Economics A Morden Approach -
Prof. Wang Li
36
A brief introduction
to EViews
Using Eviews for OLS regression
Economics A Morden Approach -
Prof. Wang Li
37
EViews Workfile
 EViews provides sophisticated data analysis,
regression, and forecasting tools on
Windows-based computers.
 A workfile is simply a container for EViews
objects
• Most of your work in EViews will involve
objects such as data series, groups,
equations, and vectors, that are contained
in a workfile
• Your first step in any project will be to
create a new workfile.
Economics A Morden Approach -
Prof. Wang Li
38
EViews Workfile Window
Economics A Morden Approach -
Prof. Wang Li
39
EViews Workfile
 Creating a Workfile by Describing its
Structure
• Select File/New Workfile... from the main
menu to open the Workfile Create dialog.
• You will select (depending on your Dataset):
 Unstructured for Cross-Section
 Dated regular frequency for time series
 Balanced Panel for a simple panel dataset
Economics A Morden Approach -
Prof. Wang Li
40
EViews Workfile
The default coefficient vector
EViews stores the estimated
coefficients in it.
A default series to hold the
estimated residuals
Economics A Morden Approach -
Prof. Wang Li
41
Getting Data from Excel
• Do your own project
• You create a workfile by describing its
structure
• you may put data into Excel file before
• You need import data from .xls
• EViews objects also have procedures, or
procs
• Click on Procs/Import/Read Text-Lotus-Excel
Economics A Morden Approach -
Prof. Wang Li
42
Getting Data from Excel
Economics A Morden Approach -
Prof. Wang Li
43
Creating a Workfile from a
Foreign Data Source
 You may directly open a foreign data
source as an EViews workfile.
• Foreign Data are the non-EViews format
data
• In the second approach, you simply open
and read data from a foreign data source.
• EViews will analyze the data source, create
a workfile, and then automatically import
your data.
Economics A Morden Approach -
Prof. Wang Li
44
Creating a Workfile from a
Foreign Data Source
 Import your data by “Copy and Paste”
EViews will open the foreign data source
and to read the data into an new EViews
workfile.
• Copy the foreign data source to the Windows
clipboard
• Right click on the gray area in your EViews
window.
• Select Paste as new Workfile
 Such an approach, while convenient, is only
practical for small amounts of data.
Economics A Morden Approach -
Prof. Wang Li
45
Creating a Workfile from a
Foreign Data Source
 You may directly open a foreign data
source as an EViews workfile.
– First select File/Open/Foreign Data as
Workfile..., to bring up the file Open
dialog.
– Clicking on the Files of type combo box
brings up a list of the file types that
EViews currently supports for opening a
workfile.
Economics A Morden Approach -
Prof. Wang Li
46
Creating a Workfile from a
Stata file
 We need to import Stata files to
illustrate examples in text book and do
computer excercises.
 File/Open/Foreign Data
as Workfile…
 the Files of type
combo box
Economics A Morden Approach -
Prof. Wang Li
47
Creating a Workfile from
a Stata file
 EViews will open
the selected file,
validate its type,
and display a
tabbed dialog.
 Select variables
tab should be
used to choose
the series data
to be included.
Economics A Morden Approach -
Prof. Wang Li
48
Creating a Workfile from
a Stata file
 EViews reads
the specific
data into a
new workfile.
 It opens the
imported data
as a group
window in a
spreadsheet
view.
Economics A Morden Approach -
Prof. Wang Li
49
Make Equation
 Create an equation object
 select Object/New Object.../Equation
– Opens Equation Specification dialog box
 Specifying an equation by List
wage c educ
• Here c is a built-in EViews series that is
used to specify a constant in a regression.
u educ wage + + =
1 0
| |
Economics A Morden Approach -
Prof. Wang Li
50
Make Equation
 You may include auto-series in the list of
variables. If the auto-series expressions
contain spaces, they should be enclosed in
parentheses.
 For example: modify the equation
log(wage) c educ
That is
u educ wage + + =
1 0
) log( | |
 Press the Name button and name the equation
as edu1
– The edu1 equation object will be placed in
the workfile
Economics A Morden Approach -
Prof. Wang Li
51
Estimating a Regression Model
• Perform your estimation by the sample
• Press the Estimate button
• Choose Estimation method --LS least
squares
• and choose sample range
• Then click the OK (确定) button
• Output view containing estimation results
appears
• For any object you can view it in many ways.
• An equation also has a representation view
and so on.
Economics A Morden Approach -
Prof. Wang Li
52
Forecasting
• Forecasting from an Estimated Equation
• You may perform forecasts based upon this
model with the actual data.
• Click on the Forecast button in the equation
to open the forecast dialog
• Static Forecast perform a series of one-
step ahead forecasts
• Click the OK button
• EViews will present a graph of the forecast
of the model
• – the fitted line along with the 95%
confidence intervals for this forecast.
Economics A Morden Approach -
Prof. Wang Li
53
Forecasting
• Sample range
• By default, EViews will fill the forecast
series with the values of the actual
dependent variable (the workfile sample)
• If you want to do out-of-sample forecast
– Procs/Resize Current page…
– Insert data for out-of-sample
independent variables before performing
forecast
Economics A Morden Approach -
Prof. Wang Li
54
Forecasting
 Note that you are responsible for supplying
the values for the independent variables in
the out-of-sample forecasting period
 After putting data for out-of-sample
independent variables, You must specify the
sample range to be used for the forecast.
– You can find forecast results of
dependent variables in forecast series,
for example wagef