Professional Documents
Culture Documents
Marnie McLean
Room 344 Mathematics and Statistics Building,
marnie.mclean@glasgow.ac.uk
October 2018
Chapter 2 part 1- Smoothing Methods
Smoothing methods:
Splines
The more knots used, the more flexible the regression function
can become.
One approach - use “too many” knots (one knot per observation
in the most extreme case) and use a penalty term to control for
the smoothness (section 2.5.1).
Yi = f (xi ) + i , i = 1, . . . , n
Pn
the solution f̂ (xi ) = yi minimises i=1 (yi − f (xi ))2 .
λ is a fixed constant,
a ≤ x1 ≤ · · · ≤ xn ≤ b,
Rb
The term λ a
[f 00 (x)]2 dx is referred to as a roughness penalty .
Pn 2
i=1 (yi − f (xi )) - measures closeness to the data
Rb
λ a [f 00 (x)]2 dx - penalizes curvature in the function.
Drawbacks
Yi = f (xi ) + i , i = 1, . . . , n
a curve estimate can be produced by fitting the regression
Therefore,
p
X
f (x) = βj bj (x)
j=0
Yi = β0 + β1 xi + εi ,
1 x1
1 x2
. .
design matrix = X = B(x) = B =
. .
. .
1 xn
basis functions are b0 (x) = 1, b1 (x) = x
x1p
1 x1 . .
. x2 . .
x2p
. . . . .
X = B(x) = B = . . . .
.
. . . . .
1 xn . . xnp
basis functions are b0 (x) = 1, b1 (x) = x, . . . , bp (x) = xp
A simple case would be for a linear spline basis with one knot
at x = 0.5 e.g.
1, x, (x − 0.5)+