Section 1 Notes GSI: Kyle Emerick

EEP/IAS 118 September 1st, 2011

Derivation of OLS Estimator

In class we set up the minimization problem that is the starting point for deriving the formulas for the OLS intercept and slope coecient. That problem was,
N 0 , 1 i=1

min

0 1 xi )2 . (yi

(1)

As we learned in calculus, a univariate optimization involves taking the derivative and setting equal to 0. Similarly, this minimization problem above is solved by setting the partial derivatives equal to 0. That is, 0 and set it equal to 0. We then do the same thing for 1 . This take the derivative of (1) with respect to gives us, N W 0 1 xi ) = 0 = 2(yi (2) 0
i=1

and, W = 1
N

0 1 xi ) = 0 2xi (yi
i=1

(3)

N 0 1 xi )2 . Now our task is to solve (2) and (3) using some Note that I have used W to denote i=1 (yi 0 (this is algebra tricks and some properties of summations. Lets start with the rst order condition for N Equation (2)). We can immediately get rid of the 2 and write i=1 yi 0 1 xi = 0. Now lets rearrange N this expression and make use of the algebraic fact that i=1 yi = N y . This leaves us with,

0 = N y 1 x N N .

(4)

We simply divide everything by N and amazing, we have the formula that Professor Sadoulet gave in lecture! That is, 0 = y 1 x . (5) Now lets consider solving for 1 . This one is a bit more tricky. We can rst get rid of the 2 and rearrange N 0 xi 1 x2 = 0. Now lets substitute our result for 0 into this expression Equation (3) to get i=1 xi yi i and this gives us,
N

1 x 1 x2 = 0 xi yi ( y ) xi i
i=1

(6)

Note that the summation is applying to everything in the above equation. We can distribute the sum to each term to get,
N N N N

xi yi y
i=1 i=1

1 x xi +
i=1

1 xi
i=1

x2 i = 0.

(7)

We have of course used the property that you can always pull a constant term out in front of a summation. N N ). We Lets again use the property that i=1 yi = N y (and of course this also means that i=1 xi = N x 1 . This gives, apply these facts to Equation (7) and solve for 1 =
N y i=1 xi yi N x . N 2 2 i=1 xi N x

(8)

Doesnt quite look like the formula from class, right? Well, let us just use a couple more tricks. You can N N )(yi y ) = i=1 xi yi N x y . You can also easily either look up or derive for yourself that i=1 (xi x N N 2 2 2 . These two can be derived very easily using algebra. Now we derive that i=1 (xi x ) = i=1 xi N x substitute these two properties into (8) and we have something that looks very, very familiar: 1 = All done!
Page 1
N )(yi i=1 (xi x N )2 i=1 (xi x

y )

(9)