
Prof. Wang Li
1
The Simple Regression
Model (1)
y = 
0
+ 
1
x + u
Economics A Morden Approach 
Prof. Wang Li
2
Chapter Outline
Definition of the Simple Regression Model
Deriving the Ordinary Least Squares
Estimates
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of the OLS
estimators
Regression through the Origin
Economics A Morden Approach 
Prof. Wang Li
3
Lecture Outline
Simple linear regression model
Some terminology
A simple assumption
Zero Conditional Mean Assumption
Ordinary Least Squares
Deriving OLS Estimates
A brief introduction to EViews
Economics A Morden Approach 
Prof. Wang Li
4
Equation y = 
0
+ 
1
x + u
has only one nonconstant regressor x,
it is called a simple linear regression
model,
or twovariables regression model,
or bivariate linear regression model.
∆y = 
1
∆x if ∆u = 0
Thus, the change in y is simply 
1
multiplied
by the change in x.
Simple linear regression model
Economics A Morden Approach 
Prof. Wang Li
5
Some Terminology
In the linear regression model,
where y = 
0
+ 
1
x + u, we typically refer to
lefthand side variable y as the
– Dependent Variable, or
– Explained Variable, or
– Response Variable, or
– Predicted Variable, or
– Regressand
Economics A Morden Approach 
Prof. Wang Li
6
In the simple linear regression of y on x,
we typically refer to righthand side
variable x as the
– Independent Variable, or
– Explanatory Variable, or
– Control Variable , or
– Predictor Variable , or
– Regressor
Some Terminology, cont.
Economics A Morden Approach 
Prof. Wang Li
7
The coefficients 
0
, 
1
are called the
regression coefficients.

0
is also called the constant term or the
intercept term, or intercept parameter.

1
represents the marginal effects of the
regressor, x. It is also called the slope
parameter.
Some Terminology, cont.
Economics A Morden Approach 
Prof. Wang Li
8
The variable u is called the error term
or disturbance in the relationship.
It represents factors other than x that
can affect y.
Some Terminology, cont.
u education Earnings + + =
1 0
 
Economics A Morden Approach 
Prof. Wang Li
9
Meaning of linear:
linear means linear in parameters, not
necessarily mean that y and x must have a
linear relationship.
There are many cases that y and x have
nonlinear relationship, but after some
transformation, they are linear in
parameters.
For example, y=e

0
+
1
x+u
natural log: ln(y)=
0
+
1
x+u
Meaning of linear
Economics A Morden Approach 
Prof. Wang Li
10
Example of
Returns to Education
u education wage + + =
1 0
 
A model of human capital investment
implies getting more education should
lead to higher earnings
education , wage and u are random
variables in the equation.
Economics A Morden Approach 
Prof. Wang Li
11
A Simple Assumption
The average value of u, the error term, in the
population is 0. That is,
expected value of u E(u) = 0
It it restrictive?
This is not a restrictive assumption, since we
can always use 
0
to normalize E(u) to 0.
If for example, E(u)=5. Then
y = (
0
+5)+ 
1
x + (u5),
therefore, E(u’)=E(u5)=0.
• As long as the intercept 
0
is included in the
equation, the assumption of E(u) = 0 holds.
Economics A Morden Approach 
Prof. Wang Li
12
Zero Conditional Mean Assumption
We need to make a crucial assumption about
how u and x are related
This assumption involves the expected
conditional distribution value of u given any
value of x.
E(ux) = E(u)
– It means that, for any given value of x, the
average of the unobservables is the same,
– that u and x are completely unrelated.
therefore E(ux) must equal the average
value of u in the entire population.
Economics A Morden Approach 
Prof. Wang Li
13
What does it mean?
u and x are completely unrelated
In the example of education, suppose u
represents innate ability, conditional mean
assumption E(ux) = E(u) means
E(abilityedu=6)=E(abilityedu=18)
The average level of ability is the same
regardless of years of education.
Since we have assumed E(u) = 0, therefore,
E(ux) = E(u) = 0
Zero Conditional Mean Assumption
Economics A Morden Approach 
Prof. Wang Li
14
Use x to predict the average value of y
– take expectations conditional on x
E[yx] = E[
0
+ 
1
x+ux]
= 
0
+ 
1
x+ E[ux]
Zero Conditional mean assumption:
E(ux) = E(u) = 0
We got, E[yx] = 
0
+ 
1
x
Its important implication:
The population regression function is a linear
function of x, where for any x the
distribution of y is centered about E(yx).
Zero Conditional Mean Assumption
Economics A Morden Approach 
Prof. Wang Li
15
.
.
x
1
=5 x
2
=10
E(yx) = 
0
+
1
x
y
f(y)
Conditional
distribution of y
given x
Zero Conditional Mean Assumption
The population
regression function
(PRF)
Economics A Morden Approach 
Prof. Wang Li
16
Deriving the Ordinary Least
Squares Estimates
Basic idea of regression is to estimate the
population parameters from a sample set.
Let {(x
i
,y
i
): i=1, …,n} denote a random
sample of size n from the population.
Since these data come from simple
regression model, for each observation in
this sample, it will be the case that
y
i
= 
0
+ 
1
x
i
+ u
i
Economics A Morden Approach 
Prof. Wang Li
17
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
u
1
u
2
u
3
u
4
x
y
Population regression line, sample data points
and the associated error terms
PRF
E(yx) = 
0
+ 
1
x
Economics A Morden Approach 
Prof. Wang Li
18
We must decide how to use these data to
obtain estimates of the intercept and slope
in the population regression
The method of Ordinary Least Squares (OLS)
plays a major role to obtain estimators.
OLS estimators are the best linear unbiased
estimators.
The name “ordinary least squares” comes
from the fact that these estimates minimize
the sum of squared residuals.
Deriving the Ordinary Least
Squares Estimates
Economics A Morden Approach 
Prof. Wang Li
19
To derive the OLS estimator we need to
realize that our main assumption of
E(ux) = E(u) = 0 also implies that
u and x are uncorrelated, independent
Covariance Cov(x,u) = E(xu) = 0
Why? Remember from basic probability that
Cov(X,Y) = E(XY) – E(X)E(Y) and
E(XY)= E(X)E(Y) when x and y are independent.
∵E(ux) = E(u)=0 ∴Cov(x,u) = E(xu) = 0
Deriving the Ordinary Least
Squares Estimates
Economics A Morden Approach 
Prof. Wang Li
20
Equations E(u) = 0 and E(xu) = 0 can be
written just in terms of x, y, 
0
and 
1
:
since u = y – 
0
– 
1
x
Zero mean: E(y – 
0
– 
1
x) = 0
Zero conditional mean: E[x(y – 
0
– 
1
x)] = 0
This two Equations imply 2 restrictions on
the joint probability distribution of (x,y) in
the population.
Technically, values of the independent
variable is the same as treating the x
i
as
fixed in repeated samples.
Deriving OLS continued
Economics A Morden Approach 
Prof. Wang Li
21
The method of moments approach to
estimation implies imposing the population
moment restrictions on the sample moments
What does this mean?
Recall that for E(X), the mean of a population
distribution, a sample estimator of E(X) is
simply the arithmetic mean of the sample
These are called moment restrictions.
Deriving OLS using M.O.M
Economics A Morden Approach 
Prof. Wang Li
22
We want to choose values of the parameters that
will ensure that the sample versions of our
moment restrictions are true
E(y – 
0
– 
1
x) = 0
E[x(y – 
0
– 
1
x)] = 0
The sample versions of our 2 restrictions above
are as follows:
( )
( )
¦
¦
¹
¦
¦
´
¦
= ÷ ÷
= ÷ ÷
¿
¿
=
÷
=
÷
0
ˆ ˆ
0
ˆ ˆ
1
1 0
1
1
1 0
1
n
i
i i i
n
i
i i
x y x n
x y n
 
 
Deriving OLS using M.O.M
Then we are going to solve the equation set.
Economics A Morden Approach 
Prof. Wang Li
23
More Derivation of OLS
Given the definition of a sample mean, and
properties of summation, we can rewrite
the 2 restrictions as follows:
x y
x y
1 0
1 0
ˆ ˆ
or
,
ˆ ˆ
 
 
÷ =
+ =
where y¯ is the sample average
of the y
i
24
More Derivation of OLS
( ) ( )
( ) ( )
( )( ) ( )
¿ ¿
¿ ¿
¿
= =
= =
=
÷ = ÷ ÷
÷ = ÷
= ÷ ÷ ÷
n
i
i i
n
i
i
n
i
i i
n
i
i i
n
i
i i i
x x y y x x
x x x y y x
x x y y x
1
2
1
1
1
1
1
1
1 1
ˆ
ˆ
0
ˆ ˆ


 
The sample versions of the second restriction
( ) 0
ˆ ˆ
1
1 0
1
= ÷ ÷
¿
=
÷
n
i
i i i
x y x n  
From basic properties of the summation operator
( ) ( )( ) y y x x y y x
i
n
i
i
n
i
i i
÷ ÷ = ÷
¿ ¿
= = 1 1
x y
1 0
ˆ ˆ
  ÷ =
Economics A Morden Approach 
Prof. Wang Li
25
the OLS estimated slope
( )( )
( )
( ) 0 that provided
ˆ
1
2
1
2
1
1
> ÷
÷
÷ ÷
=
¿
¿
¿
=
=
=
n
i
i
n
i
i
n
i
i i
x x
x x
y y x x

So the OLS estimated slope is
the numerator is
sample covariance
between x and y
The denominator is the
sample variance of x
• Dividing both the numerator and the
denominator by n1 will make sense.
Economics A Morden Approach 
Prof. Wang Li
26
Summary of OLS slope estimate
The slope estimate is the sample
covariance between x and y divided by the
sample variance of x.
If x and y are positively correlated, the
slope will be positive.
If x and y are negatively correlated, the
slope will be negative.
Only need x to vary in our sample.
Economics A Morden Approach 
Prof. Wang Li
27
the OLS estimated intercept
( )( )
( )
¿
¿
=
=
÷
÷ ÷
=
n
i
i
n
i
i i
x x
y y x x
1
2
1
1
ˆ

Once we have the slope estimate:
x y
1 0
ˆ ˆ
  ÷ =
It is straightforward to obtain the
intercept estimate, given y¯ and x¯.
Economics A Morden Approach 
Prof. Wang Li
28
More OLS
Intuitively, OLS is fitting a line through
the sample points.
The residual, û, is an estimate of the
error term, u, and is the difference
between the fitted line (sample regression
function) and the sample point.
i i i
u x y
ˆ
ˆ ˆ
1 0
+ + =  
i i
x y
1 0
ˆ ˆ
ˆ
  + =
Sample regression function
Economics A Morden Approach 
Prof. Wang Li
29
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
û
1
û
2
û
3
û
4
x
y
Sample regression line, sample data points
and the associated estimated error terms
(residuals)
x y
1 0
ˆ ˆ
ˆ
  + =
OSL regression line
The notation yˆ emphasizes
that the predicted values
from this equation
are estimates
Economics A Morden Approach 
Prof. Wang Li
30
Alternate approach
The residual for observation i is the
difference between the actual y
i
and its
fitted value:
( ) ( )
¿ ¿
= =
÷ ÷ = =
n
i
i i
n
i
i
x y u L
1
2
1 0
1
2
ˆ ˆ
ˆ
 
Given the intuitive idea of fitting a line, we
can set up a formal minimization problem.
That is, we want to choose our parameters
minimizing the following:
i i i i i
x y y y u
1 0
ˆ ˆ
ˆ ˆ
  ÷ ÷ = ÷ =
Economics A Morden Approach 
Prof. Wang Li
31
Alternate approach, continued
We use calculus to solve the minimization
problem for the two parameters , The
first order conditions:
( )
( ) 0
ˆ ˆ
2
ˆ
0
ˆ ˆ
2
ˆ
1
1 0
1
1
1 0
0
= ÷ ÷ ÷ =
c
c
= ÷ ÷ ÷ =
c
c
¿
¿
=
=
n
i
i i i
n
i
i i
x y x
L
x y
L
 

 

That is the following:
( ) ( )
¿ ¿
= =
÷ ÷ =
n
1 i
2
i 1 0 i
n
1 i
2
i
β β
x β β y u
1 0
ˆ ˆ
ˆ min
)
ˆ
,
ˆ
(
Economics A Morden Approach 
Prof. Wang Li
32
Alternate approach, continued
These are the same as we have before
(sample versions of 2 restrictions, moment
restrictions ), multiplied by 2n.
( )
( )
¦
¦
¹
¦
¦
´
¦
= ÷ ÷
= ÷ ÷
¿
¿
=
=
0
ˆ ˆ
0
ˆ ˆ
1
1 0
1
1 0
n
i
i i i
n
i
i i
x y x
x y
 
 
So we call these are The first order
conditions restrictions .
Economics A Morden Approach 
Prof. Wang Li
33
{(x
i
,y
i
): i=1, …,n} are actual values in a
random sample set from the population.
Given a random sample set, the OLS method
is used to estimate the slope and intercept
parameters in the population model.
In population regression function
E(yx) = 
0
+ 
1
x , the 
0
and 
1
are true values.
Summary of OLS estimate
In the OSL regression line
are estimated values by given random
sample set. are predicted values forming
the fitted line.
1 0
ˆ
,
ˆ
 
x y
1 0
ˆ ˆ
ˆ
  + =
y
ˆ
Which of the followings are right?
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
t t
X Y  o + =
t t t
X Y µ  o + + =
√(2)
√(6)
√(7)
t t t
X Y µ  o + + =
ˆ
ˆ
t t t
X Y µ  o + + =
ˆ
ˆ
ˆ
t t
X Y  o
ˆ
ˆ
+ =
t t
X Y  o
ˆ
ˆ
ˆ
+ =
t t t
X Y µ  o
ˆ
ˆ
ˆ
+ + =
t t t
X Y µ  o
ˆ
ˆ
ˆ
ˆ
+ + =
n t ， ， 2 , 1 =
Checking Understanding
Economics A Morden Approach 
Prof. Wang Li
35
Lecture Summary
Introduce the simple linear regression
model.
Introduce the method of Ordinary Least
Squares to estimate the slope and
intercept parameters using data from a
random sample.
The name “ordinary least squares” comes
from the fact that these estimates
minimize the sum of squared residuals.
Economics A Morden Approach 
Prof. Wang Li
36
A brief introduction
to EViews
Using Eviews for OLS regression
Economics A Morden Approach 
Prof. Wang Li
37
EViews Workfile
EViews provides sophisticated data analysis,
regression, and forecasting tools on
Windowsbased computers.
A workfile is simply a container for EViews
objects
• Most of your work in EViews will involve
objects such as data series, groups,
equations, and vectors, that are contained
in a workfile
• Your first step in any project will be to
create a new workfile.
Economics A Morden Approach 
Prof. Wang Li
38
EViews Workfile Window
Economics A Morden Approach 
Prof. Wang Li
39
EViews Workfile
Creating a Workfile by Describing its
Structure
• Select File/New Workfile... from the main
menu to open the Workfile Create dialog.
• You will select (depending on your Dataset):
Unstructured for CrossSection
Dated regular frequency for time series
Balanced Panel for a simple panel dataset
Economics A Morden Approach 
Prof. Wang Li
40
EViews Workfile
The default coefficient vector
EViews stores the estimated
coefficients in it.
A default series to hold the
estimated residuals
Economics A Morden Approach 
Prof. Wang Li
41
Getting Data from Excel
• Do your own project
• You create a workfile by describing its
structure
• you may put data into Excel file before
• You need import data from .xls
• EViews objects also have procedures, or
procs
• Click on Procs/Import/Read TextLotusExcel
Economics A Morden Approach 
Prof. Wang Li
42
Getting Data from Excel
Economics A Morden Approach 
Prof. Wang Li
43
Creating a Workfile from a
Foreign Data Source
You may directly open a foreign data
source as an EViews workfile.
• Foreign Data are the nonEViews format
data
• In the second approach, you simply open
and read data from a foreign data source.
• EViews will analyze the data source, create
a workfile, and then automatically import
your data.
Economics A Morden Approach 
Prof. Wang Li
44
Creating a Workfile from a
Foreign Data Source
Import your data by “Copy and Paste”
EViews will open the foreign data source
and to read the data into an new EViews
workfile.
• Copy the foreign data source to the Windows
clipboard
• Right click on the gray area in your EViews
window.
• Select Paste as new Workfile
Such an approach, while convenient, is only
practical for small amounts of data.
Economics A Morden Approach 
Prof. Wang Li
45
Creating a Workfile from a
Foreign Data Source
You may directly open a foreign data
source as an EViews workfile.
– First select File/Open/Foreign Data as
Workfile..., to bring up the file Open
dialog.
– Clicking on the Files of type combo box
brings up a list of the file types that
EViews currently supports for opening a
workfile.
Economics A Morden Approach 
Prof. Wang Li
46
Creating a Workfile from a
Stata file
We need to import Stata files to
illustrate examples in text book and do
computer excercises.
File/Open/Foreign Data
as Workfile…
the Files of type
combo box
Economics A Morden Approach 
Prof. Wang Li
47
Creating a Workfile from
a Stata file
EViews will open
the selected file,
validate its type,
and display a
tabbed dialog.
Select variables
tab should be
used to choose
the series data
to be included.
Economics A Morden Approach 
Prof. Wang Li
48
Creating a Workfile from
a Stata file
EViews reads
the specific
data into a
new workfile.
It opens the
imported data
as a group
window in a
spreadsheet
view.
Economics A Morden Approach 
Prof. Wang Li
49
Make Equation
Create an equation object
select Object/New Object.../Equation
– Opens Equation Specification dialog box
Specifying an equation by List
wage c educ
• Here c is a builtin EViews series that is
used to specify a constant in a regression.
u educ wage + + =
1 0
 
Economics A Morden Approach 
Prof. Wang Li
50
Make Equation
You may include autoseries in the list of
variables. If the autoseries expressions
contain spaces, they should be enclosed in
parentheses.
For example: modify the equation
log(wage) c educ
That is
u educ wage + + =
1 0
) log(  
Press the Name button and name the equation
as edu1
– The edu1 equation object will be placed in
the workfile
Economics A Morden Approach 
Prof. Wang Li
51
Estimating a Regression Model
• Perform your estimation by the sample
• Press the Estimate button
• Choose Estimation method LS least
squares
• and choose sample range
• Then click the OK (确定) button
• Output view containing estimation results
appears
• For any object you can view it in many ways.
• An equation also has a representation view
and so on.
Economics A Morden Approach 
Prof. Wang Li
52
Forecasting
• Forecasting from an Estimated Equation
• You may perform forecasts based upon this
model with the actual data.
• Click on the Forecast button in the equation
to open the forecast dialog
• Static Forecast perform a series of one
step ahead forecasts
• Click the OK button
• EViews will present a graph of the forecast
of the model
• – the fitted line along with the 95%
confidence intervals for this forecast.
Economics A Morden Approach 
Prof. Wang Li
53
Forecasting
• Sample range
• By default, EViews will fill the forecast
series with the values of the actual
dependent variable (the workfile sample)
• If you want to do outofsample forecast
– Procs/Resize Current page…
– Insert data for outofsample
independent variables before performing
forecast
Economics A Morden Approach 
Prof. Wang Li
54
Forecasting
Note that you are responsible for supplying
the values for the independent variables in
the outofsample forecasting period
After putting data for outofsample
independent variables, You must specify the
sample range to be used for the forecast.
– You can find forecast results of
dependent variables in forecast series,
for example wagef