Chapter 2

Simple linear regression mode OR the Classical linear regression model

Introduction: - In economics the relationship between variables are mainly explained in the form
of dependent & independent variables. The dependent variable is that variable which its average
value is computed using the already known values of the explanatory variable(s). But the values
of the explanatory variables are obtained from fixed or in repeated sampling of the population.
Example Suppose the amount of commodity demanded by an individual is depend on the price of
the commodity, income of individual, price of other goods & etc. Then from this statement
quantity demanded is the dependent variable which its value is determined by the price of the
commodity and income of the individual, Price of other goods etc. And price of the commodity,
income of individuals & price of other goods are independent (explanatory) variables whose
value is obtained from the population using repeated sampling. The relationship between these
dependent and independent variable is a concern of regression analysis. i.e.

Qd = f (P, P0, Y etc) -------------------- (2.1)

If we study the relationship between dependent variable & one independent variable i.e. Qd= f (P)
this is known as simple two variable regression model because there are one dependent Qd & one
independent P regression model. However if the dependent variable is depending upon more than
one independent variables such as Qd: f (P, P0, Y) it is known as multiple regression analysis.
The functional relation ship between the dependent and independent variable may be linear or
non-linear.

2.1 The simple linear regression Analysis

The relationship between the dependent & independent variable suggested by economic theory is
usually specified as exact or deterministic relationships. But in reality the relationship between
economic variables are inexact or stochastic or in deterministic in nature.
Ex. Suppose consumption expenditure for a commodity is depending up on current income of the
individual citrus-paribus & assumes that the functional relationship is linear then we can write it.
Ct =  +Yt ------------------------------------------------------2.2
Then for each specific value of current income (Yt) there will be only one corresponding value of
consumption expenditure. This shows that consumption expenditure is depend upon current
income. But consumption expenditure is not only determined by income alone but different
variables such as wealth, previous income, tradition etc affect consumption expenditure. Then
there is inexact relation ship between these two variables and to capture those factors which
affects consumption expenditure in equation 2.2 we in corporate a variable „U‟. Then we can
write the equation as follows
Ct =  +Yt + Ut----------------------------------------- (2.3)
Where Ct is the dependent variable, Yt is independent variable,  &  are regression parameters,
Ui is the stochastic disturbance term or error term.
We introduce „U‟ – random term due to the following reasons.
i. Omission of variables from the function. In economic reality each variable is
influenced by very large number of factors and each variable may not be included in
the function because of
a) Some of the factors may not be known.
b) Even if we know them the factors may not be measured statistically example
psychological factors (test, preferences, expectations etc) are not measurable
c) Some factors are random appearing in an unpredictable way & time. Example
epidemic earth quacks e.t.c.

1 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 d) Some factors may be omitted due to their small influence on the dependent variables
e) Even if all factors are known, the available data may not be adequate for the measure
of all factors influencing a relationship
ii. The erratic nature of human beings:- The human behavior may deviate from the
normal situation to a certain extent in unpredictable way.
iii. Misspecification of the mathematical model:- we may wrongly specified the
relationship between variables. We may form linear function to non- linearly related
relationships or we may use a single equation models for simultaneously determined
relationships.
iv. Error of aggregation: - Aggregation of data introduces error in relationship. In
many of Economics data are available in aggregate form ex. Consumption, income
etc is found in aggregate form which we are added magnitudes referring to
individuals where behavior is dissimilar.
v. Errors of measurement:- when we are collecting data we may commit errors of
measurement
In order to take in to account the above source of error we introduce in econometric functions a
random term variable which is usually denoted by the latter U & is called error term, random
disturbance term or stochastic term of the function. By introducing this random term variable in
the function the model will be just like equation number (2.3). The relationship between variables
will be split in to two parts.
Example From equation (2.3)
 +Yt represents the exact relationships explained by the line
 Apart represented by the random term Ui is the unexplained part by the line. This
can be explained using the following graph.
Y

Yn Ct= =  +yt

Un
Ct Consumption

Un
Yn

U2

Y1 X
0 Current income (Yt)
Figure 1
The line Ct: =  +Yt shows /explain/ the exact relation ship between consumption & income but
other variables that affect consumption expenditure are scattered around the straight line. Then
the true relationship is explained by the scatter of observations between Ct &Yt.
2 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Ct =  +Yt + Ut ------------------------ (2. 4)
Variation in Explained
+= ++ Unexplained
consumption + variation+ variation
+ +
To estimate this equation we need data on Ct, Yt+&Ut, since Ut is never observed like other
variables (Ct & Yt) we should guess the value of „U‟, that is we should make some assumptions
+
about the shape of each Ui (mean, S.E, Covariance etc)

2.2 Assumptions of the linear stochastic regression model.

To guess the value of „U‟ we make some assumptions about Ui & divided these assumptions in to
three
a) Some refer to the distribution of random variable Ui.
b) Some to the relationship between Ui & the explanatory variables
c) Some refer to the relationship between the explanatory variables themselves

2.2.1 Assumption about Ui

i) Ui – is a random real variable:- The value which Ui-may assume in any one period depends
on chance, it may be positive, negative or zero.
ii) The mean value of Ui in any particular period is zero i.e.

E(Ui) = 0 or  ui  0
i 0
iii) Homoscedasticity: (Constant Variance). The variation of each Ui around all values of the
explanatory value is the same i.e. the deviation of Ui around the straight line (in figure 1) is
remain the same var (Ui)=  u
2

iv) The variable Ui has a normal distribution with mean zero & variance of Ui.
Ui is N(0,  u )
2

v) Ui is serially independent:- the value of U in one period is not depend up on the value of
Ui in other period of time means the co-variance between Ui & Uj is equal to zero
Cov (UiUj) = 0

Cov (UiUj) = E [ Ui – E (Uj)] [Uj –E(U)]

By assumption ii – the E(Ui) = 0 then

= E [Ui-0] [Uj-0]

= E(Ui) E(Uj)

Again by assumption E(Ui) = 0

Cov (UiUj ) = 0

2.2.1 Assumption about Ui & Xi

i- The disturbance term Ui is not correlated with explanatory variables. It means
Ui‟s & Xi‟s are not moving together or the covariance between Ui & Xi‟s are zero
Cov (UiXi) =0
Cov (UiXi)
E [Ui  E (Ui)][ Xi  E ( Xi )]
3 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 By assumption we have E(Ui)=0 then
= E{[Ui-0][Xi-E(Xi)]}
= E{UiXi-UiE(Xi)]
= E(UiXi)-E(Ui)E(Xi)]
Again by assumption E(Ui)=0
= E(UiXi)-0E(Xi)]
= E(UiXi) since the value of Xi's are fixed then
=XiE(Ui)=0
Cov (UiUj)=0
ii- The explanatory variables Xi's are measured with out error i.e no problem of
aggregation, round off etc. If there is such problem in the measurement it will be
absorbed by the random term Ui.

2.2.2 Relation ship about explanatory variables

If there are more than one explanatory variable the relationships is assumed that they are not
perfectly correlated with each other.
Ex.
Yt=   1 X 1   2 X 2   3 X 3  Ui
X1&X2, X2&X3, X1&X3 are not correlated with each others. i.e. no multicollinearity.

The distribution of the dependant variable Y

Given the following relationship between variables

Yi =   Xi  Ui
Mean of Yi (Expected value of Yi) can be found as follow

E(Yi)= E[   Xi  Ui ]
E(Yi) =   Xi  E (Ui)
Where E (Ui) = 0 by assumption
E(Y) =   Xi ----- is the mean value of the dependent variable Yi
Variance of Yi =
Var (Yi) = E [Yi-E (Yi)]2
Substitute in place of E(Y) =   Xi
Var (Yi) =E [Yi-E (   X ) ]2
Again in place of Yi substitute Yi =   Xi  Ui
Var (Yi) = E    Xi  Ui    X i 2
Var (Yi)= E(Ui)2
From our previous assumption the variance of Ui is equal to E (Ui)2 =u2 then

Var (Yi)=E(Ui)2 = u2 which is constant.

The distribution of Y with mean & variance will be
Yi_N (   Xi ,  u )
2

4 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
2.3 Estimation of the model
The relationship

Yi =   Xi  Ui                2.5
Holds for population of the values X&Y. Since these values of the population are unknown we do
not know the exact numerical values of  & β' s. To calculate or obtain the numerical values of 
& β we took sample observations for Y & X. By substituting these values in the population
regression we obtain sample regression which gives an estimated value of  & β given by
ˆ & ˆ respectively then the sample regression line is given by

Yˆi  ˆ  ˆXi                2.6

The true relationships between variables (that explain the population) is given by

Yˆi    Xi  Ui                  2.7

If you estimated this relationship using sample observation we get the estimated relationship
which has the following

Yˆi  ˆ  ˆXi  Ui                2.8

We can estimate the value of &β using least square method (OLS) or classical least squares
(CLS).The reasons to start or use OLS or CLS methods are many
i. The parameters obtained by this methods have some optimal properties i.e. BLUE (Best,
Linear, Unbiased Estimators).
ii. The computational procedure of OLS is fairly simple as compared to other econometric
methods.
iii. OLS is one of the most commonly employed methods in estimating econometric models.
iv. The mechanics of OLS is simple to understand.
v. OLS is an essential component of most other econometric techniques
From the sample observations we will have

Yˆi  ˆ  ˆXi  ei                2.9

ei  Yˆi  ˆ  ˆXi                2.10
Finding values for the estimates ˆ & ˆ which will minimize the square of residuals  ei 2

^ ^

 ei 2  [Yi  ˆ  ˆxi ]2              2.11

i 1 i 1
To find the values of  & β that minimize this sum, we have to differentiate with respect to
ˆ & ˆ & set the partial derivatives equal to zero

  ei 2
 2 (Yi  ˆ  ˆXi )  0            2.12
ˆ
5 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
   ei 2
 2 (Yi  ˆ  ˆXi ) Xi  0              2.13
ˆ

First take equation number 2.12to find the value of ̂

 2 (Yi  ˆ  ˆXi )  0 Run the sum over the equation

 2 (Yi   (ˆ  ˆXi )  0

ˆ  nˆ
 2 Yi  2nˆ  2ˆ  Xi  0

 2 Yi  2nˆ  2ˆ  Xi =0

2 Yi  2ˆ  Xi  2nˆ Divided by 2n to get ̂

ˆ 
 Yi  ˆ  Xi
n n
 Yi  Y
n
&
 Xi  x
n
ˆ
ˆ  Y  x                2.14
Take equation number (2.13) to find the value of ˆ

 2 (Yi  ˆ )  (2ˆ  Xi ) X  0 Multiply by X

 2 (YiXi  ˆXi ˆXi 2 )  0 Sum it over

 2( YiXi  ˆ  Xi ˆ  Xi 2 )  0

YiXi  ˆ  Xi  ˆ  Xi 2
               2.15

Substitute equation (2.14 ) in to equation (2.15 )

YiXi  (Y  ˆx ) Xi  ˆ  Xi 2

YiXi  Y   Xi  ˆx  Xi  ˆ  Xi 2

6 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 YXi  Y  Xi  ˆx  Xi  ˆ  Xi 2

We know that Y 
 Y & x   X substituted
n n
 
Yi Xi ˆ Xi
YiXi  n   n  Xi  ˆ  Xi 2

Yi  Xi ( Xi ) 2

YiXi  n  ˆ n  ˆ  Xi Multiplied both sides by n 2

nYiXi  Yi  Xi  ˆ ( Xi )  ˆ  Xi n 2 2

nYiXi  Yi  Xi  nˆ  Xi  ˆ  Xi  2 2

nYiXi  Yi  Xi  ˆ n Xi  ( Xi )  2 2

n YiXi   Yi  Xi
ˆ                 2.16
n Xi  ( X )
2 2

The numerical value of ˆ & ˆ can be found in deviation forms. To write the above equation
number 2.16 in deviation form
Take the numerator which is

n Xi  Yi   Yi  Xi
Yi  Xi
Added & subtracted
n Xi Yi  Yi  Xi  n Xi Yi  Yi  X   Xi Yi  Xi Yi 
= n Xi Yi  Yi  Xi   Xi Yi   Xi Yi
= n Xi Yi  Yi  Xi   Xi Yi   Xi Yi

= n Xi  Yi 
 Yi Xi n  Xi Yi  n  X n  Y
n
 n
 n n
Take n in common

n  XiYi  Y  X  x Y  nxy 
n  Xi  x Yi  y  -------------------------2.17

This equation is equal to the numerator of equation number 2.16.

Again from equation 2.16 take the denominator
n Xi 2   Xi   n Xi 2  2  Xi   Xi 
2
 2
 2

 n Xi 2  2 Xi  Xi   Xi 
2

 n Xi 2  2nx  X  n 2 x 2
 n  Xi 2
 2 x  X  nx 2
 n ( xi 2  x ) 2                  2.18

By taking equation 2.17 as numerator & 2.18 as denominator

7 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 n ( X 1  X )(Y1  Y )
ˆ 
n X 1  X ) 2
X1  X  xi & Yi  Y  yi Substitute in the above equation
n xyi
ˆ 
n xi 2

ˆ 
 xiyi                  2.19
 xi 2

2.4 Statistical tests of Estimates

The two most commonly used tests in econometrics are r2 i.e. square of correlation coefficient &
the standard error of tests ( s.e.).

 The Square of Correlation Coefficient = r2 /R2/

When we estimate a model of two variable case (one independent (X) & one dependent variable
Y) we find r2. But if we have more than two variable case (one dependent variable & more than
one independent variables (X1,X2...Xn) we will have the coefficient of determination R2.
Definition of r2/R2/After estimation of ˆ & ˆ from the sample data observations of Y & X
using OLS method, we need to know how 'good' is this fit of the line to the sample observations
of Y&X. Means measure the dispersion of the sample observation around the regression line. The
closer the observations to the line the better is the explanation of the variation of Y by the change
in the explanatory variables (X's)
r2 shows the percentage of the total variation of the dependent variable that can be explained by
the independent variable X.
Y Unexplained variation X
Yi = ˆ  ˆxi

Total variation

Y
Explained variation

8 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
Suppose a researcher may have Yi=+βXi+ Ui model. To estimate this model he took some
sample observation to estimate the value of  & β. In his estimation all the data may fall below,
above or on the line. Then using R2 he can observe that whether the regressions line will give the
best fit for these data or not.
 Yi is the observed sample value
 Y is the mean value of the sample
 Yˆ is the estimated regression line using sample data
 Yi - Y  shows by how much the actual sample value is deviating from the sample mean
value. This is called total variation represent by small y.
 Yˆi  Y Explain by how much the estimated values are deviating from the sample mean
value. This is called explained variation & represent by ŷi
 Y  Yˆi This also shows that the difference between the actual value of Yi & the
estimated value of Yi ( Yˆi ). This is called unexplained variation represent by ei.
Therefore;
Total variation: yi  Yˆi  Y                2.20
Explained variation yˆ  Yˆ  Y                2.21
Unexplained variation ei  Yi  Yˆi                2.22
Sum it over each equation & squared it. We will have

 yi 2
  (Yi  Y ) 2                2.23
We square it because the sum of deviation of any variable around its mean value is zero then to
avoid this we make it squared.

 yˆi   (Yˆi  Y )
2 2
                 2.24
 ei   (Yi  Yˆ )
2 2
                 2.25
We can write equation no (2.20) as follows

yi  Yˆi  Y  Yi  yi  Y                2.26
From equation (2.21)
yˆi  Yˆ  Y  Yˆi  yˆi  Y            2.27
Substitute these equations in equation number 2.21
ei = Yi- Yˆ from the above equation (2.26 &2.27 ).
Yi  yi  Y
&
Yˆ  yˆ  Y
ei  yi  Y   yˆi  Y 
ei  yi  Y  yˆ  Y
ei  yi  yˆi
yi  ei  yˆi                2.28
This shows that each deviation of the sample observed values of Y from its mean Yi  Yˆ  yi
consists of two components

9 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 yˆi  Yˆi  Y which shows the explained amount by the regression line
i.
ii. ei  Yi  Yˆ = the unexplained variation by the regression line
By Taking equation number 2.28 yi  yˆi  ei Sum it over

 yi   ( yˆi  ei) Squared it

^ ^

 yi   ( yˆi  ei )
i 1
2

i 1
2

 yi   yˆi  2 yˆei   ei
2 2 2
                 2.29

From this equation 2  yˆi ei is equal to zero. We can prove it.

yˆ  Yˆ  Y We know that from equation 2.14

Y    ˆxi again
Y  ˆ  ˆx Then if we substitute these in yi  y  Y
yi  ˆ  ˆXi  ˆ  ˆx
yi  ˆ ( Xi  x )
( Xi  x )  xi is in deviation from
yi  ˆxi            2.30
We also know that ei  yi  yˆi since yi  ̂xi
Substitute ei  yi  ˆxyi                2.31
 yˆiei  from  equation  2.29 ˆxi( yi  ˆxi)
=2
= ˆ  xi ( yi  ˆxi )
= ˆ  xyi  ˆ  xi 2 2

From equation 2.19 we know that ˆ 

 xiyi then substitute in the above equation in place of
 xi 2

ˆ
 yˆei  ˆ  xiyi  ˆ  xi
2 2 2

= ˆ  xyi  ˆ  xi  2

=
 xiyi  xiyi   xiyi  xi  2

 xi  2
 xi 
2

 xiyi
 ŷiei  xi  xiyi   yixi   0
 2

 ŷiei  0 Then equation number 2.29 can be written as follow

 yi 2
  yi   ei 2 2
                 2.32
10 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
Total variation = Explained variation + Unexplained variation
Divided both sides of equation number (2.32) by 2
yi & you will get 
 yi 2


 yˆ 2


 ei 2

 yi 2
 yi 2
 yi 2

1
 yˆ 2


 ei 2

 yi 2
 yi 2

 yˆ 2

Is the % of explained sum of squares which is r2

 yi 2

1 r2 
 ei                2.33
2

 yi 2

Then r 2 
 yˆi                2.34
2

 yi 2

Again from equation 2.32

r 2
 1
 ei 2

               2.35
 yi 2

ˆ
2.5 The relationship between r2 and the slope coefficient  .
We know that

r 2

 yˆi 2

.
 yi 2

From equation 2.21

yˆ i  Yˆ  Y
yˆi  ˆ  ˆXi  ˆ  ˆx
yˆi  ˆ ( xi  x )

yˆi  ̂xi Substitute in equation 2.34

From equation
 yˆ  (ˆxi ˆ xi
2
2 2 2
)
r 2
  
 yi 2
 yi 2
 yi 2

From equation number 2.19 we know that

ˆ 
 xiyi
 xi 2

Then substitute in place of ˆ

ˆ   xi 2
xiyi
.
 xi 2
 yi 2

11 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 r 2  ˆ
 xyi .
 yi 2
Or if you further substitute ˆ again =
 xiyi
 xi 2

r 2  ˆ
 xiyi .  xiyi = ( xiyi ) 2

           2.36
 xi  yi  xi . yi
2 2 2 2

Then the value of r2 can be found in different ways

r 2  yˆi 2

. or 1
e 2

or 
 xiyi or ̂  xi 2

or
( xiyi ) 2
 yi 2
 yi 2
 yi  yi
2 2
 xi . yi
2 2

Limiting values of r2

r 2
 1
 ei 2

 yi 2

We have three options

1) If all the sample observations lie on the regression line there will be no scatter point‟s i.e
the difference between the actual value & observed value is zero i.e.
ei = Yi- Y =zero. Then r2 will be 1
If the regression line explains a part of the actual values i.e. some are not explained by the

regression line then ei will have some values

 ei 2

will be greater than zero but less than one

 yi 2

then the r2 will lie between zero & one

0  r2  1

3) If the regression line does not explain any part of the variation then
 ei 2

will be one then

 yi 2

r 2 will be zero.

2.6 Significance test of the Parameter estimates

The value of ˆ & ˆ which are obtained from the sample observation must be tested whether
they are statistically reliable to explain the population parameters or not. To test the statistical
reliability of the sample estimates we should know their mean & variance. To do this we follow
the following steps.
a) develop the formula for the computation of the mean & variance of the least square
estimates
b) Explain the statistical significance of the estimates using standard error & t-test
c) Construct confidence intervals for the estimates ˆ & ˆ .

12 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
Mean & Variance of the least square estimates
 Mean of ˆ
We can obtain mean value of ˆ from equation number 2.20

ˆ   xiyi .
 xi 2

In place of yi substitute equation number 2.21 then

ˆ  xi (Yi  Y ) .
 xi 2


 xiYi  Y  xi .
 xi 2

By definition it is known that the sum of any variable deviations from its mean is equal to zero.
Then  xi  0 then Y xi  0 
 xiYi . =  xiYi                     2.37
ˆ    xi 
 xi 2 2

The value of the independent variables is a set of fixed values, which do not change from sample
xi
to sample. Then will be a constant number & lets‟ represent it by K & equation 2.37 can
 xi 2
be written as
ˆ   kiYi                        2.38
We know that from equation number 2.5

Yi    i  ui Substitute in equation 2.38

ˆ   ki(  Xi  ui)
ˆ    ki    kiXi   kiui              2.39
 ki  0 We can prove it as follows
ki 
xi
  ki 
 xi By definition
 xi 2
 xi 2

 x  0 Because the sum of any variables deviations from its mean is zero
Then
 xi   ki  0
 xi 2

Again  kiXi  1

To prove it we know that Ki 

 xi
 xi 2

 xiX
 KiXi  xi                  2.40
 2

13 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
xi   Xi  x  Substitute in the above equation 2.40
 ( X  x)x
 kiXi 
 xi 2

 X  x  x                      2.41
2

 xi 2

We can further simplify this as follows. Take the denominator

 xi 2
  ( X  x ) 2   X 2  2 xX  x 2  
= X 2
 2 x  Xi  nx 2
Since x 2 is constant number summation of xi means multiply by n. Again here x means
X then substitute in place of x .
n
 X . X  n  X 
2

X 2
2
n
  n 
 
( X ) 2
 X  2

X 2
2
n
 n.
n2
( X ) 2  X  2

X 2

n
2
n2


 X  X
 X 2  2 n  X   n X
 
 X 2  2x  X  x  X
x =X2 2
 x  X substitute in the denominator of equation number 2.41
X 2
 x X
 KiXi  X 1
 2
 x X
Then  kiXi  1
xi  Xi  x
Xi  xi  x
Substitute in the equation  kiXi in place of Xi
xi
 ki( xi  x ) We know that ki 
 xi 2
 xi ( xi  x )   xi  x  xi
=
2

 xi  xi 2 2

 xi   xi again by definition  xi  0
2

=
 xi  xi 2 2

14 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
  kiXi   xi
2
xi
1
 2

Now we turn in to equation number 2.39 & write it as follows (Remember

that  
xi  0 & kiXi  1)
ˆ =  +  kiUi

 xi substitute it
Again  ki 
 xi 2

ˆ   
 xiui
 xi 2

Take the expected (average or mean value of ˆ )

 xiui )
E ( ˆ )  E (  )  E ( Since xi‟s are constant
 xi
 E ( ) 
 xiE  ui
 xi 2

Again from the assumption number 2 of OLS E (ui)=0

E ( ˆ )   - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2 . 42
The mean of OLS estimate ˆ is equal to the true population parameter mean value i.e 
 Variance of ˆ
Var ( ˆ ) + E[ ˆ -  ( ˆ )]2
From equation 2.42 we know that  ( ˆ )   substitute it
Var ( ˆ ) = E[ ˆ -  ] 2
From equation 2-38 we know that

ˆ  xiyi  xi (Yi  y )
 = =  kiYi
 xi 2  xi 2
Then Var ( ˆ ) = var (  kiYi ) =  ki 2 var ( Yi ) - - - - - - - - - - - - - - -- - - 2.43
xi
ki 
 xi 2
First let‟s find var (Yi )  E[Yi  E (Yi )]2
We know that Yi    Xi  Ui
E (Yi )  E ( )  E ( Xi )  E (ui)
E (Yi )    xi  0
By assumption number 2 E (ui)  0
Substitute   Xi + Ui in place of Y &   Xi in place of E (Yi ). then
2
Var (Yi )  E[  Xi  Ui  (  Xi )]
= E[  Xi  Ui    Xi ]2

15 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Var Yi  E[Ui]2  u 2 - - - - - - - - - - - - - - - - - - -2.44
From equation no 2.43 we have the following
Var ( ˆ ) =  ki 2 Var ( Yi )
From equation no 2.44 Var (Yi )  u 2
Var ( ˆ )   ki 2u 2
2
 xi 
 u  ki - - - - - We know that  ki  E 
2 2
2 
2

  xi 
2
 xi 
= u  2

  xi 
2

 xi 2
= u
( xi 2 ) 2
1
Var ( ( ˆ )   2 - - - - - - - - - - - - - - -- - - - - - -2.45
 xi 2
 2
Now we can say that ˆ has a mean of  & Variance of
 xi 2
u 
 , 
Then ˆ ~N  xi 
 Mean of ̂
 xiyi
From Equation no 2.14 we have ˆ  yˆ  ˆxˆ a gain from equation 2.19 ˆ  & from
2
 xi
equation number 2.38 we have ̂   kiyi substituted
ˆ  y  ( kiYi) x
 y  x   kiYi
 Yi
  x  kiYi
n
Take Yi &  as common b/c x & ki are constant
1 
̂    x kiYi - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - 2.45
n 
Take the expected Value of ̂ since n, x & ki are constant the expected value of a
constant is a constant it self.
1 
E (ˆ )     x ki (Yi ) W know that
n 
E (Yi )  E (  xi  Ui)
   xi   (i)
By definition E (ui)  0 then it will be   xi

16 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 1 
E (ˆ )     x ki(  Xi )
n 
 1 
    x ki  X  Xix ki
n n 
n  xi
  x  ki    x i  kixi]
n n
   x  ki  x  kiXi
We know that  ki  0 &  kixi  1
   x  x
E (ˆ )   - - - - - - - - - - - - - - - - - - - -- - - - - - -- -- - 2.46
 Variance of ̂
1 
Var (ˆ )  E[  E (ˆ )]2 we know that from equation no 2.45 that ˆ     x ki yi
n 
then substitute it
 1 
Var (ˆ )  Var  E[  x ki]Yi 
 n 
Since n, x & ki are constant number their variance is constant.
2
1 
Var (ˆ     x ku var Yii
n 
We know that Var Yi= u from equation no 2.43 substitute in place of Var Yi. Then we will
2

have.
2
1
Var (ˆ )  u 2    x ki
n
1 1 
= u 2   2  x 2 ki 2  2 x ki
n n 
We know that the summation over a constant number is equal to multiplying the constant number
by n
n 2 x  ki 
 u  2  x 2  ki 2 
n n 
We proved that  ki  0
  xi   xi 2
2
1
Again  ki   2
2 
 
  xi  ( xi  xi 2
2

Substitute these values

n 1  2 x (0) 
 u 2  2  x 2 
n  xi 2
n 
2 1 x2 
 u   2 
If you take the common denominator n  xi 2
 n  xi 
  xi 2 x 2 n 
Var (ˆ )  u 2  2 

 n  xi 
17 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
We proved that  xi 2  ( Xi  x ) 2   xi 2  nx 2 (you can refer the devation from equation
no 2.40)
We can summarize the mean & variance of the variables as follows

Yi ~ N (  xi ), u 2 
 2   xi  x n 
2
 2
 ~ N  , u 
ˆ  

 n  xi
2
 
 u  2
ˆ ~ N  , 2 
  xi 
 Standard Error (S.E) values of ˆ & ˆ
S.E is the square root of variance

S.E ( ˆ )  var( ˆ )  ˆ
2
             (2.47)
 xi 2

S.E ( ˆ )  var(ˆ )  ˆ 2 
X 2

             (2.48)
n xi 2

Sine ˆ 2 can not be easily computed it will be substituted by

 ei 2

Equation 2.47 &

n2
2.48 can be written as follows

ˆ  ei 1
2

S.E (  )  .
n  2  xi 2

S.E ( ˆ ) 
 ei . X 2 2

(n xi )(n  2)2

 Standard error test of the least square estimates

From the sample observation of Y&X we can estimate or obtain the value of ˆ & ˆ . Since the
observations are sample taken from the population then due to sampling errors the estimates may
not truly explain the population. Then it is necessary to apply test of significance in order to
a) Measure the size of the error committed
b) Determine the degree of confidence that the estimates will lie.
Though there are other methods used for the purpose, we use standard error test which is very
popular in many econometric research. The standard error test (S.E) will help us to determine
whether the estimates ˆ & ˆ are coming from a population whose parameters are zero called the
null hypothesis.
Ho =  i=0
This means there is no relation ship between the dependent & independent variables.
Or the samples from which ˆ & ˆ are coming from a population whose parameters are different
from zero are known as alternative hypothesis
H1 =  i  0
This means there is a relationship between the dependent & independent variables.

18 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
  Interpretation of S.E
Once we obtain the standard deviations of ˆ & ˆ using equation number 2.47 & 2.48
respectively& by comparing these estimated values with their s.e we can interpret the results as
follows.
Economic interpretation
If we have Yi  ˆ  ˆxi
Case A) Acceptance of the null hypothesis means
Accept Ho =  i=0
ˆ
Reject the alternative that H1 =  i  0. This will occur if S.E ( ˆ ) 
2
It means the estimation are statistically insignificant or
- We accept the null hypothesis that the true parameter  is zero or
- The independent variable is insignificant.
- The sample parameter do not explain the population parameter or
- The independent variable do not influence the dependent variables (no relationship
between the dependent & independent variables)
Then the above equation will be written as
Yi  ̂
ˆ
Because the value of Ho =  is not different zero or the slope of the line is zero
i. Again in case of the intercept ̂
If we accept the null hypothesis HO=  =0 & reject the alternative H1    0 .
ˆ
This will occur if S.E ( ˆ ) 
2
And the meaning of ̂ is statistically insignificant or the equation will not have intercept or the
intercept is not differed from zero. Then
Since ˆ  0
Yi  ̂xi
Case B) Reject the null hypothesis & accept the alternative
i. Reject Ho = ˆ 0 =0 & accept the alternative that H1  1  0 .
ˆ
This will occur if S.E ( ˆ ) 
2
It means
- The estimates are statistically significant
- The estimates are significantly different from zero
- The independent variables will influence the dependent variable or there is a relation ship
between the dependent & independent variables.
ii. Reject HO= ̂ =0 & accept the alternative that H1  ˆ  0 . This will be observed if
ˆ
S.E( ˆ ) 
2
Again it means the equation will have intercept or ̂ is statistically significant

19 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
  Geometric interpretation of the S.E. test
ˆ
1) If we find that if S.E ( ˆ )  we accept he null hypothesis that HO= ̂ =0 & reject the
2
alternative H1  ˆ  0 . In this case if we have the equation Yi  ˆ  ˆxi since ˆ  0 we
will write this equation as Yi  ̂xi and we do not have the intercept. Then the equation will
pass through the origin (fig. a)

Yi  ̂xi Yi  ˆ  ˆxi

0
X
0 0
Fig (a) fig. (b)
ˆ
2) If we find S.E. ( ˆ )  we will reject the null hypothesis that HO= ̂ =0 & accept the
2
alternative H1  ˆ  0 . Then in this case we will have intercept term & the equation
will be Yi  ˆ  ˆxi (fig. b)
ˆ
3) If we find that S.E ( ˆ )  we accept the null hypothesis that Ho = ˆ0 =0 & reject
2
the alternative that H1  1  0 . Then the equation will be Yi  ̂ because ˆ  0 set

20 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Y

Yˆ  ˆ  ˆxi
Y  ̂i

0 0 X
X Fig. d
Fig. c

ˆ
4) If we find that S.E ( ˆ )  we reject the null hypothesis Ho = ˆ =0 & accept the
2
alternative that H  ˆ  0 . Then the equation will be Yˆ  ˆ  ˆxi
1 1

 The student t- test

The second statistical test, next to S.E. will be under taken using t-test. This t-test will be
applicable if the sample size is less than 30 & the population parameters distribution is normal.
To apply t-test we follow the following steps.
i. Define the null hypothesis & alternative hypothesis
The null hypothesis Ho = ˆi =0 Alternative hypothesis = H1  ˆ1  0 .
ii. Choose the level of significance (5% or 1%)
Level of significance shows that rejecting a null hypothesis when it is true [committing type I
error). Or choosing a certain level of probability with which we would be willing to risk error
Type I is called significance level.
Ex. If the level of significance is 5%, then there are 5 chances out of 100 that we would reject the
null hypothesis (i.e. you commit error & reject Ho = ˆi =0 when it is correct & accepted)
OR we are 95% confident that we have made the right decision; & only with 5% of probability
that we might have done wrong.
iii. Define the number of degrees of freedom (d.f.). i.e. N-K
N = total number of sample size
K = number of estimated variables (ˆ & ˆ )
The d.f = N-K
ˆ
t
S .E ( ˆ )
ˆ
a) If t  we reject the null hypothesis Ho = ˆ0 =0 & accept the
S .E ( ˆ )
alternatives, i.e. H  ˆ  0 .
1 1
OR t= falls in the critical region

21 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 ˆ
b) If t  we accept the null hypothesis Ho = ˆ0 =0 & reject the
S .E ( ˆ )
alternatives H1  ˆ1  0 .

OR t= falls in the acceptance region

Ho = ˆ0 =0
H1  ˆ1  0 . Acceptance region
H1  ˆ1  0 .
Rejection region
Rejection region

-2.228 0 + 2.228
Figure (e)
 Interpretation of t test
ˆ
If t  it means reject the null hypothesis Ho = ˆ0 =0
S .E ( ˆ )
Accept t the alternative H1  ˆ1  0
 The estimated values are statistically significant or
 The sample observation can explain the population parameters.
 There is a relation between the dependent & independent variables.
Then the t-value will lie in the critical region (shaded) Suppose if we chose 5% level of
significance and the total number of sample size is 12 & the number of estimated variables are
~
( ˆ &  ) only two. Then the d.f will be 12.-2=10. Again if we are undertaking two tall test then
 5% 
we will have 0.025   in each side (see figure e)
 2 
Then to find the t-value at 5% (0.25 each side) & 10 d.f. We can read from the t-table as follows
a) In the first raw find 0.025
b) In the first column find 10 then at the intersection point of the raw & the column
you will get t-table value 2.228.
ˆ
Again if t  it means accept Ho = ˆ0 =0
S .E ( ˆ )

22 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Reject the alternative hypothesis H 1  ˆ1  0
 The estimated values are statistically insignificant
 No relationship between the dependent & independent variables
 The sample observations do not explain the population parameters.
Then the t-table value will lie in the acceptance region
 Simple inspection to calculate t-test
ˆ
At 5% level of significance we can follow the following rules. Calculate t-values by .
S .E ( ˆ )
Then if this value is greater than +2 or less than -2 we reject the null hypothesis Ho = ˆ =0 & 0

accept the alternative H 1  ˆ1  0 . And the t-value will lie in the critical region. If calculated
ˆ
t smaller than +2 & greater than -2 or (if -2<t<2) we accept the null hypothesis &
S .E ( ˆ )
reject the alternative.

 Confidence intervals for ˆ & ˆ

When we accept the alternative hypothesis that H 1  ˆ1  0 & reject the null hypothesis that Ho
= ˆ 0 =0. It doesn‟t mean that our estimates ˆ is the correct estimate of the population
parameter ˆ . It simply means that our estimates come from a population whose parameters are
different from zero (there is a relation ship between the dependent & independent variables).
Then by constructing confidence intervals we can define how our estimates are closer to the true
population parameters. Interpretation of confidence intervals. If we have 95% confidence level.
i. In the long run 95 out of 100 cases will contain the true parameter  i in the limit or
intervals. OR
ii. We are 95% confident that the unknown population parameters (  i ) will lie with in
the limit /interval. OR
iii. In 5% of the cases the population parameter will lie outside the confidence limits.
But it doesn‟t mean that or we can not say that the confidence interval contains the true
population parameter (  i ). Because the probability contains specific fixed interval is either 1 or
0. Confidence interval from the student t- distribution

ˆi  
t                            2.48
S ( ˆ )
With in( n-k) df.
ˆ = estimated value from the sample
 = is the population parameter
If we choose any confidence level say 95%. We find from the t-table i.e the value
+ t0.025 with d.f of (n-k).
P {-t0.025<t<t+0.025} = 0.95-------------------------------------------2.49
Substitute equation number 2.48 in equation number 2.49
ˆi  
P{-t0.025 <t0.025} = 0.95
S .E ( ˆ )
23 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
Multiply both sides by S.E. ( ˆ )
P{-t0.025 (S.E. ( ˆ ) ) ˆ   <t0.025.S.E. ( ˆ ) } = 0.95
i  ˆ  t 0.025S.E.(ˆ )                 2.50
This confidence interval shows that the unknown population parameter  i will lie with in the
defined limit 95 times out of 100. OR we are 95% confident that the unknown population
parameter  i will lie with in this limit.
Ex.1.
In Table 2.1 the investment expenditure and the long run interest rate over the ten year period is
given.Test the hypothesis that investment is interest elastic by fitting regression line to the data
given & conducting the relevant test of significance. To answer the question we follow the steps
of econometric methodology.
i. The economic theory states that “investment is interest sensitive.”
ii. Mathematical model of the theory
a) Selecting variables:- Investment is the dependent variable (endogenous) and interest rate
is exogenously determined (independent variable)
b) A priori expectation of the sign of the parameters:- There is a negative (inverse
relationship) between Investment & rate of interest
c) Magnitude of the parameter:- The theory says that investment is interest elastic then the
coefficient is greater than one.
d) Specification of the model:- This will be a single equation model which investment will
be affected by rate of interest but investment will not affect rate of interest. The
relationship between investment and rate of interest is assumed to be linear (This will be
determined by the researcher).
Then
Yt    Xt
Where Yt= is the level of interest
Xt = is rate of interest,  >1 &  is the intercept
iii. Specification of the Econometric model.
There are other variables which will affect investment these are marginal efficiency of capital
(MEC), saving, consumption, political stability etc. Since these & other variables are not
incorporated in the mathematical model of investment function we can capture these
variables by incorporating a random term Ui in our model.

Yt    Xt +Ui
Then by adding the random (error, or stochastic or disturbance) term Ui we convert the
mathematical (exact) relationship between investment & rate of interesting in to in exact
relationship of Econometric models.
iv. Obtaining data:- A sample of 10 years observation data are given to estimate the
model & the type of data are time series data.
v. Estimation of the econometric model
The economic relationship is explained using a single equation model then the most
appropriate method of estimating this equation is OLS method. To estimate this model we use
table 2.1 & obtaining the following results in deviation forms.
 yixi  1.42  xi 2
 0.00064
 yi  25826.4
2
 ei 2
 25,654

24 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
  yi  Y  753.4
 xi  x  0.056
n n

From equation number 2.18 we have ˆ 

 xiyi then
 xi 2

 1.424
ˆ 
 2225
0.00064
ˆ  Yˆ  ˆxˆ From equation number 2.14
ˆ  753.4  (1.424 x0.056)  878
Then the estimated regression line will be
Yˆ  878  2225 Xi
vi. Evaluation of the estimates:-
After estimation & obtaining values of the coefficient ( ˆ & ˆ ) of the variable we should have to
evaluate the results obtained using Econometrics method
a) Economic interpretation of the results:- at this stage check whether the obtained results
are economically meaningful or not.
Yˆ  878  2225xi
i. If interest rate is zero i.e Xt =0 then Yˆ  878 means if the interest rate is zero investment
will be equal to 878 birr. This is the interpretation of the constant term ̂ in case of
linear equation
ii. ˆi Indicates that if rate of interest increase/decrease by 1 birr then investment will
decrease/increase on the average by 2225 birr. Therefore it passes the economic criterion
because it explains the inverse relationship between investment & interest rate.
b) Since the model passes a prior economic criterion, the next step is to test the reliability of
the estimated parameters using statistical tests using r2, & S.E tests.
i. the correlation coefficient test r2
To estimate r2 we can use the formula of 2.31-2.35 let‟s us

r 2  ˆ
 xiyi
 yi 2
We know that ˆ = -2.225  xiyi  5.424.  yi 2
 25,826.4
 2.225(1.424)
r2   0.123
25.8264.4
This means 12.3% of the change investment is accounted (explained) by interest rate &
the remaining 87.71, is not explained by rate of interest but by some other factors
represented by Ui in our model.
ii. Standard error test:-

S.E ( ˆ )  ˆu 2 x 2

since ˆu 2 can not be obtained we can

n xi 2

approximately measure using

 ei 2

n2

25 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 S.E( ˆ ) 
 ei  xi 2 2

n  2 n xi 2

From our estimation we obtain the following values  ei 2

 22657
 Xi 2
 .032 n-2=10-2=8  xi 2
 0.00064
(22658) (0.32) 90.632
S.E( ˆ )    17701.563  133.046
(8) 8(0.00064) 0.00512
Again S.E ( ( ˆ ) can be obtained using equation number 2.47

S.E ( ( ˆ ) 
 ei 2

=
22658
=
22658
=2103.661
n2  xi 2
8(0.00064) 0.00512
Having obtained the value of S.E (ˆ ) and S.E ( ( ˆ ) we can undertake the S.E Test using
hypothesis testing.
ˆ
Test of S.E (ˆ ) : If  S .E (ˆ ) then we can reject the null hypothesis Ho= ˆ  0 &
2
accept the alternative that H1= ˆ  0 .
ˆ  878 & S.E (ˆ ) = 133.046
ˆ 876
 = 439> 133.046
2 2
ˆ
Then  S .E (ˆ ) we can conclude that rejecting the null hypothesis & accept the
2
alternative.

Test of S.E ( ˆ ) : If S .E ( ˆ )  we can accept the null hypothesis and reject alternative.
2
 2225
S .E ( ˆ ) - 2103.661 & = = 1112.5
2 2

Since S .E ( ˆ )  we accept the null hypothesis & reject the alternative
2

The student t- distributions

Given the 5% level of significance level we can have the following approximate measurements of
t-test
887
t= = 6.66 since t-value is greater than 2 we can reject the null hypothesis & accept
133.046
the alternative
ˆ
t=
S .E ( ˆ )
Again if t> + 2 we reject the null hypothesis & accept the alternative
 2225
t=  1.058 since the t value is greater than -2 then we accept the null hypothesis &
2,103.661
reject the alternative. To summarize we can writ the results as follows
26 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Ŷt  887 - 2225xt
S.E. (133.046) (2103.66)
t (6.66) (-1.058)
The equation passed the economic or a priori criterion but we found different results in statistical
test.
a) ̂ Satisfied the statistical test because
̂
S.E. (ˆ ) < & also t>2 we can say that we reject the null hypothesis H 0  ˆ  0 & accept
2
the alternative H1  ˆ  0 . It means that the equation should have an intercept term.
b) ˆ Does not satisfies the statistical test because
ˆ
S.E. ˆ > & which means acceptance of the null hypothesis that H 0  ˆ  0
2
 ˆ is statistically insignificant
 Investment (Y) is not affected by the change in the interest rate or investment is not
interest sensitive.
 No relation ship between investment and interest rate because ˆ =0
And & reject the alternative H 1  ˆ  0 means ˆ is not different from zero.
Again this is supported by the t-test. Since t= -1.058 & which is greater than -2 or t is found
between + 2 then we can conclude that we accept the null hypothesis H 0  ˆ  0 and
- The estimates ˆ is statistically insignificant
- ˆ is not different from zero
- No relationship between investment & interest rate.
Example 2
Given the data on table 2.2. fit the data and estimate the income elasticity.

27 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Table 2.1

Invest- Rates of
ment interest (Yi  Y ) ( Xi  x ) ei =

Year Yi Xi Yi Xi yi2 xi2 yixi Estimated Yˆi yi - Yˆi ei2

1958 656 0.05 -97.4 -0.006 9476.76 0.000036 0.5844 766.75 -110.75 1265.56

59 804 0.045 50.6 -0.011 2560.36 0.000121 -0.5566 777.875 26.125 682.5156
60 836 0.045 82.6 -0.011 6822.76 0.000121 -0.9086 777.875 58.125 3378.516
61 765 0.055 11.6 -0.001 134.56 0.000001 -0.0116 755.625 9.375 87.89
62 777 0.06 23.6 0.004 556.96 0.000016 0.0944 744.5 32.5 1056.25
63 711 0.06 -42.4 0.004 1797.76 0.000016 -0.1696 744.5 -33.5 1122.25
64 755 0.06 1.6 0.004 2.56 0.000016 0.0064 744.5 10.5 110.25
65 745 0.05 -6.4 -0.006 40.96 0.000036 0.0384 766.75 -10.75 390.0625
66 696 0.07 -57.4 0.14 3294.76 0.000196 -0.8036 722.25 -26.25 689.0625
67 787 0.065 33.6 0.009 1128.96 0.000081 0.3024 733.375 53.625 2875.641

 Yi  Xi
 yi  xi  yi  xi  yixi  yˆi  ei  ei
2 2 2 2
=7,534 =0.56

Yˆ =753.4 X =0.056 =0 =0 =25826.4 =0.00064 =-1.424 =7534 =0 =22658

28 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 Table 2.2

Log Y-

Consum Income (Logy) log Yˆ log Yˆ =

Year ption Xt Log Yt LogXt Log yt Log xi Log y2 Log xi2 (logx) estimated (e) ei2
Yt

0
58
8 17 0.9 1.23 -0.317 -0.399 0.1004 0.1596 0.1286 0.9163 -0.0132 . 000176

59 12 27 1.079 1.431 -141 -0.1986 0.0198 0.0394 0.02797 1.069 0.0101 0.000103

60 15 36 1.176 1.556 -0.044 -0.0737 0.00193 0.00543 0.00323 1.164 0.0121 0.000146

61 18 46 1.255 1.663 0.0352 0.0327 0.001244 0.00107 0.00115 1.245 0.01037 0.000108

62 22 57 1.342 1.756 0.1224 0.1258 0.0149 0.0158 0.0154 1.316 0.0267 0.000716

63 23 67 1.361 1.826 0.1417 0.196 0.02 0.0384 0.0278 1.369 -0.00729 0.0000531
64 26 81 1.415 1.908 0.1949 0.278 0.038 0.0776 0.0543 1.431 -0.01668 0.000278

 log X  log xi 2

 log yi 2
 log Yi  log Ŷ  ei 2

 log Yi =11.37
=0.1965 =0.3374 logx =8.51 =0.00158
8.53 Y X =0.2564
=1.22 =1.624

29 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 2.7. Properties of the least square Estimates

2.7.1 The Gauss-Markov Theorem

Given the assumptions of CLS model, the least square estimates obtained from a sample possess some optimal properties.
These are the least square estimators are linear & unbiased estimators. These are also the best of linear unbiased
estimators; it means the estimators have minimum variance with in the classes of linear unbiased estimators. This theorem
is sometime referred to as the BLUE theorem. An estimator of OLS said to be BLUE (best linear unbiased estimator) if
the following holds true.
a) It is linear, if both the sample parameters are a linear function of the dependent variable.
ˆ & ˆ are a linear function of Yi
b) It is unbiased if the expected value of the sample parameters is equal to the true value of the
population parameter.
E (ˆ )   E ( ˆ )   i
Where ˆ & ˆ are sample parameters, and ˆ & ˆ are the true population parameter & E means expected or
average values.
It has minimum variance i.e. if the variance of ˆ & ˆ has smallest variable as compared to
c)
any other variance obtained from other econometric methods.
Gauss Markov Theorem: - Given the assumption of CLS model, the least square estimators satisfies the properties of
BLUE

1) The property of linearity

From equation 2.19 we have value of ˆ

ˆ  xiyi And from equation number 2.38 we proved that

 xi 2

xi
̂   kiYi Where ki 
 xi 2

From the above of X is constant in the sample & ki is constant then

̂   kiYi i.e. an estimate ̂ 2 is a linear function of Y's or ˆ is a linear function of the values of the dependent
variable.
By the same analogy equation 2.14 we have
ˆ  Y  ˆx
Substitute equation no 2.46 in place of ˆ
̂ = Y  x  kiYi
 Yi  x
̂ =
n
 kiYi
1
̂ =
n
Yi  x  kiYi
1 
̂ =    x kiYi
n 

30 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 1
We know that , x & ki are constant then ̂ is a linear function of the dependent variable Yi
n
Thus both ˆ & ˆ are expressed as linear function of the Y's.
2) Unbiased ness:- The bias of the estimation is defined as the difference between its expected value & the true
parameter.
Bias =E ( ˆ )  
If the estimation is unbiased its bias is zero
i.e. E ( ˆ )   we proved this in the previous page equation number 2.42
Again bias =E (ˆ )   if the estimation is unbiased its bias is zero E (ˆ )   Also we proved this & you can refer
equation number 2.46
3) The minimum variance property (best estimator)
The property of minimum variance is the main reason for the popularity of the OLS method. Best in this sense means
definitely superior. One should know that when we say OLS estimator is best estimator it will have a minimum variance
as compared to any estimators obtained using other econometric methods such as 2SLS, 3SLS.
Maximum Likely hood estimators etc.

2.7.2. Importance of the BLUE Properties:-

One may ask that why do we attach so much importance to the BLUE properties of the OLS estimates. But the reasons are
a) The property of linearity is desirable because it facilitates the computation of the estimates.
b) The property of unbiased ness by itself is not important.If we have a very large number of samples, the estimators
of the parameters obtained from these samples will on the average give the true value of  's
c) Best (minimum variance as compared to others). The least square variance property will be desirable if it is
combined with small bias because an estimate may have zero variance and yet have an enormous bias. The
importance of this property is apparent when we want to apply the standard test of significance for ˆ & ˆ and to
construct the confidence intervals for estimates.

2.7.3. Maximum Likelihood Estimation

A concept frequently utilized in econometrics is that of maximum-likelihood. The basic important concept of maximum
likelihood is the fact that different statistical populations generate different samples. i.e. any one sample being scrutinized
is more likely to have come from some populations rather than from others.
Ex.1
If one where sampling coins tosses & a sample mean of 0.5 were obtained (representing half heads or half tails), it
would be clear that the most likely population from which the sample were drawn would be a population with a true mean
of 0.5
Ex.1.
Assuming that [X1,X2,X3,X4,X5,X6,X7,X8] are drawn from a normal population with a given variance but unknown mean.
Again assume that these sample observations are drawn from distribution A or distribution B.

31 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
Probability

Distribution A Distribution B

x6 x2 x3 x4 x5 x8 x7 x1

Given the distribution A & B. then if the true population were B, then the probability that we would have obtained the
sample shown would be quite small. But if the true population were A then the probability that we would have drawn the
sample would be substantially large. Select the observation from population A as the most likely to have yielded the
observed data.We define the maximum likelihood estimator of  as the value of ˆ which would most likely generate the
observed sample observations Y1, Y2, Y3 --- Yn. Then if Yi is normally distributed & each of Y's is drawn independently
then the maximum-likelihood estimation maximizes. P(Y1) P(Y2) . . . P(Yn)Where each P represents a probability
associated with the normal distribution.P(Y1) P(Y2)--- P(Yn) is often referred to as the likelihood function. The likelihood
function depends up on not only on the sample values but also in the unknown parameters of the problems (ˆ & ˆ ) . In
describing the likelihood function we often think of the unknown parameters as varying while the Y's (dependent
variables) are fixed.This seems reasonable because finding the maximum likelihood estimation involves a search over
alternative parameter estimators which would be most likely to generate the given sample. For this reason the likelihood
function must be interpreted differently from the joint probability distribution. In the latter case the Y's are allowed to vary
& the underlying parameters are fixed & the reverse is true in case of maximum likelihood. Now we are in a position to
search for the maximum likelihood estimators of the parameters of the two variable regression models.

Yi    xi  ui                        2.1
We know that Yi N (  xi ,  2 )
I.e Y is normally distributed with mean (  xi ) & variance  2
Assume that all the assumptions of least squares & further assume that the disturbance term has normal distribution. Will
the estimators (   ) different from the least square estimators? Will such estimators possess the desirable properties?
In our model Yi    Xi  Ui sample consists 'n' observations in Y&X. Then we will have a mean value
of (1 , 1 , x1 ) , ( 2 ,  2 , x2 ) , ( 3 , 3 , x3 ) ... ( n ,  n , xn ) but a common variance of  2 . Why we will have a different
mean but a constant variance? The reason is simple that a random variable X assumes different value with a
probability of density function of f(x) but fixed values of Yi. The joint probability density function can be written as a
product of n individual density function

32 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
 
F[Y1 , Y2 , Y3 ...Yn / 1  1 x1 ,  2 ] = f Y1 / 1  1 x1 ,  2 f Y2 /  2   2 x2 ,  2  
f [Y3 /  3   3 x3 ,  ] f Y4 /  41   4 x4 ,  ... f Yn /  n   n xn , 
2 2 2
    2.2
This probability distribution may be written as follows

F Y   2 2  1
 
Exp  Yi    xi  ... f y n /  n   n xn ,  2     ---------------2.3
1
2 2

2 
Where exp. denotes exponential function. Then the likelihood function
L[Y1 , Y2 ...Yn ,  ,  ,  2 ] = PY1 PY2 )    P(Yn  1) P(Yn 
Let the likelihood function is represented by L
 1  Y1   1  1 xi  
2
 1  Y2   2   2 x2  2 
L  2   2  2
1
ex  

  2
2  2
1
exp  

   
 2    2   

 1  Y   3   31 x3  
2
 1  Y   4   4 x4  2 
*  2 2 
1
2 exp   3

  2 2   1
2
exp   4

 
 2    2   
 1  Yn   n   n x n  n 
1

 
2
. . . 2 exp  
2
                 2.4
 2    
Sum it over i.e. from 1 up to n.
1
 1  Y   i1  ixi 
2
 n
 
2
L  2 )  exp   i 2
 n
          2.5
  2   
If you represent the N product factor by 
 1  Yi    iXi  
1 2

L  N 2  exp  
^ 2

        -----------------------------2.6
2

i 1  2   
The value of Y1,Y2 ...Yn are given but the value of  &  ,  2 are not known then this function can be
called likelihood function and denoted by Lf ( ,  &  2 )
 1  Yi    ixi  
2
1
Lf ( ,  , )  N (2 ) exp  
2
        ------------------2.7
2 2

 2   
And the equation can be written as
 1  Yi    ixi  
2
1
Lf ( ,  , )  N 2
exp           -------------------2.8
2 2  2   
Take log of L
n n
ln Lf  log 2  log  2  1 2
2  Yi  i  xi                 2.9
2
2 2
Differentiate w.r.t.  ,  & 
2
setting these with equal to zero.
 ln Lf 1


2 2
 (Yi    Xi )(1)  0            2.10
 ln Lf 1


2 2
 (Yi    Xi )2 Xi  0              2.11

33 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)
  ln Lf n 1
 2

2 2

2 4
 (Yi    Xi )2 Xi  0             2.12
Equation number 3.10 will be equal to
 (Yi    Xi )2 Xi  0              2.13
Since  1  2  0
2
Equation no 3.11 will be again equal to
 (Yi    Xi )( Xi )  0                  2.14
Since 1  2  0
2
Substitute equation 2.13 in to equation 3.14 and we will have
~
Yi  n~    xi       -------------------------------------2.15
~ ~
Yi Xi  N  X   Xi 2                2.16
The two equations again will give us the same normal equation OLS. From equation number 3.12 we can to
obtain the value of  2
n 1
   Yi    Xi ) 2 =0
2 2
2 4

1
Since 4  0 we left with
2
n
  (Yi    Xi ) 2  0
2 2
n
 (Yi    Xi ) 2  2 2
 2  (Yi    Xi ) 2  n
1
 2   (Yi    Xi ) 2
n
1
2 
n
 (Yi  ˆ  ˆXi ) 2

We know that Yˆ  ˆ  ˆXi

Then Yi- Yˆ  Ui or Yi    Xi  Ui
Yi  ˆ  ˆXi Then Yi-  Ŷ  Yi    Xi  Ui
Therefore
1
2 
n
 u 2                  -2.17

The ML estimation is different from OLS estimator of OLS. The variance was
 ei 2

in OLS but it is
1
 ui 2
n2 n
in case of ML.Thus the variance of ML is biased estimator of  2 but it is unbiased in the case of OLS. But as the
sample size increases the ML variance converges to the true population variance.

34 | P a g e Jimma University; Department of Economics BY Alemu A.and Getache K.( Phd candidates)