Notes

Section 3.1 provides an overview of the Method of Ordinary Least Squares. The text should be easy to
follow from page 55 to the top of page 58. Equations 3.1.4 and 3.1.5 are derived in Appendix 3A.1. The
partial derivative of

2
can be obtained using the chain rule:
(

2
)

1
= (

1
)
We move the summation operation to the outside because the derivative of a sum is equal to the sum
of the derivatives (see http://en.wikipedia.org/wiki/Sum_rule_in_differentiation). The rest should be
fairly straightforward
1
.
It is also straightforward to rearrange equation 3.1.4:

= [

]
Now substitute the expression above into 3.1.5:

= [

+
2

2

Multiplying both sides by n and rearranging yields the first the first part of equation 3.1.6. Going to the
second step requires that you use:

+( ) =

+
Where you have to use the trick:
+ ( )
You can use a similar trick for the denominator in the first part of equation 3.1.6
+ (
2

2
)


1
This is an easy to understand refresher - http://www.scienceforums.net/topic/29473-introduction-to-calculus-
differentiation/. The refresher briefly explains how we find the maximum or minimum of a function. Also see e.g.
http://users.etown.edu/p/pauls/ec309/lectures/lec04_unconst.html.


Please see if you can work out the rest on your own. If you have any trouble feel free to consult with me.
There are fewer gaps in the rest of the chapter. The only other part that I found less straightforward
when I went through the chapter is property 4 of ki in 3A.2 See if you can figure this out though.
Again, please make sure you study all the sections of Appendix 3A. In particular, 3A.5 and 3A.6 are good
candidates for exam questions.
Well cover Chapters 4, 5, and 6 in the next lecture.