Time Series Analysis

Lasse Engbo Christiansen

Department of Applied Mathematics and Computer Science

Technical University of Denmark

September 22, 2017

1 / 18
Outline of the lecture

I Regression based methods, 2nd part:

I Global trend models (Sec. 3.4)

I R examples

2 / 18
Trend models

I Linear regression model

I Functions of time are taken as the independent variables
YN +j = f T (j )θ + εN +j

3 / 18
Linear trend - A motivation

I Observations for t = 1, . . . , N . Naive formulation of the model: Yt = φ0 + φ1 t + εt

I If we want to forecast YN +j given information up to N we use Y bN +j |N = φ̂0 + φ̂1 (N + j )
I However, for on-line applications N + j can be arbitrary large
I The problem arise because φ0 and φ1 are defined w.r.t. the origin 0

I Defining the parameters w.r.t. the origin N we obtain the model: Yt = θ0 + θ1 (t − N ) + εt

I Using this formulation we get: YbN +j |N = θ̂0 + θ̂1 j
I Same model, different parameterisation.

4 / 18
Linear trend in a general setting

I The general trend model:

YN +j = f T (j )θ + εN +j
 
1
I The linear trend model is obtained when: f (j ) =
j
I It follows that for N + 1 + j :
 T
 T   
1  1 0 1 
YN +1+j = θ + εN +1+j =   θ + εN +1+j
j +1  1 1 j 
| {z }
L

I The 2 × 2 matrix L defines the transition from f (j ) to f (j + 1)

5 / 18
Trend models in general

I Model: YN +j = f T (j )θ + εN +j
I Requirement: f (j + 1) = Lf (j )
I Initial value: f (0)
I In Section 3.4 some trend models which fulfill the requirement above are listed.
I Constant mean: YN +j = θ0 + εN +j
I Linear trend: YN +j = θ0 + θ1 j + εN +j
2
I Quadratic trend: YN +j = θ0 + θ1 j + θ2 j2 + εn+j
2 k
I k ’th order polynomial trend: Yn+j = θ0 + θ1 j + θ2 j2 + · · · + θk jk ! + εN +j
I Harmonic model with the period p: YN +j = θ0 + θ1 sin 2π 2π
p j + θ2 cos p j + εN +j

6 / 18
Estimation
I Model equations written for all observations Y1 , . . . , YN

YN = x N θN + ε
   T   
Y1 f (−N + 1) ε1
 Y2  f T (−N + 2)  ε2 
 ..  =  θN +
     
 ..  .. 
 .   .   . 
YN f T (0) εN

I OLS-estimates:
−1 T −1
θbN = (x T
N x N ) x N Y N = F N hN

N
X −1
FN = xT
NxN = f (−j )f T (−j )
j =0

N
X −1
hN = x T
NY = f (−j )YN −j
j =0

7 / 18
`-step prediction

I Prediction:
bN +`|N = f T (`)θbN
Y

I Variance of the prediction error:

bN +`|N ] = σ 2 1 + f T (`)F −1 f (`)

 
V [YN +` − Y N

I 100(1 − α)% prediction interval:

p
bN +`|N ± tα/2 (N − p)
Y V [eN (`)]
q
=Y
bN +`|N σ 1 + f T (`)F −1
± tα/2 (N − p)b N f (`)

where σ̂ 2 = εT ε/(N − p) (p is the number of estimated parameters)

8 / 18
Updating the estimates when YN +1 is available

I Task:
I Going from estimates based on t = 1, . . . , N , i.e. θbN to
I estimates based on t = 1, . . . , N , N + 1, i.e. θbN +1
I without redoing everything. . .
I Solution:

F N +1 = F N + f (−N )f T (−N )
h N +1 = L−1 h N + f (0)YN +1
θbN +1 = F −1
N +1 h N +1

9 / 18
 Local Trend Models - an Example
6 observations (N = 6):
4.0
3.5
3.0
y

2.5
2.0

1 2 3 4 5 6

10 / 18
 Global Linear Trend:

T
YN +j = θ0 + θ1 j + εN +j ⇒ f (j ) = 1 j
Linear Model form:

2.0 1 −5
     
ε1
2.5 1 −4 ε2 
      
3.5 1 −3 θ0 + ε3  ⇔ y = x 6 θ + ε
 
 =
3.0 1 −2 θ1
 ε4 
    
4.0 1 −1  ε5 
3.5 1 0 ε6

11 / 18
Global linear trend: Estimation

 
6 −15
F6 = xT
6 x6 =
−15 55
 
18.5
h6 = xT
6 y =
−40.5
    
0.5238 0.1429 18.5 3.905
θb6 = F −1
6 h6 = =
0.1429 0.0571 −40.5 0.329

12 / 18
 Global linear trend: Estimation with R

F6 <- t(x)%*%x
h6 <- t(x)%*%y
(th.hat6 <- solve(F6, h6))

## [,1]
## [1,] 3.9047619
## [2,] 0.3285714

## check
(lm(y~0+x))

##
## Call:
## lm(formula = y ~ 0 + x)
##
## Coefficients:
## x1 x2
## 3.9048 0.3286

13 / 18
Global linear trend: Estimation - global linear trend
5.0
4.5
4.0
3.5
y
3.0
2.5
2.0

1 2 3 4 5 6 7
t
14 / 18
 Global linear trend: Prediction

Linear predictor:

b6+`|6 = f (`)T θb6 = 3.905 + 0.328`

Y
LS-estimate for σ 2 :

b2 = (y − x 6 θb6 )T (y − x 6 θb6 )/(6 − 2)

σ
(−0.262)2 + 0.0902 + 0.5812 + (−0.248)2 + 0.4242 + (−0.405)2
=
4
= 0.4532

15 / 18
 Global linear trend: Prediction Error

ε6 (`) = Y6+` − Y
b6+`|6

b2 1 + f T (`)F −1


Var(ε
d 6 (`)) = σ 6 f (`)
    

0.5238 0.1429 1
= 0.4532 1 + 1 `
0.1429 0.0571 `
= 0.4532 (1.5238 + 0.2858` + 0.0571`2 )

For example,
d 6 (1)) = 0.6192 .
b7|6 = 4.234 with Var(ε
Y
90% prediction interval:
q
b7|6 ± t0.05 (6 − 2) Var(ε
Y d 6 (1)) = 4.234 ± 1.320

16 / 18
Global linear trend: Estimation - global linear trend
5.0
4.5
4.0
3.5
y
3.0
2.5
2.0

1 2 3 4 5 6 7
t
17 / 18
Global linear trend: Updating the parameters

I New observation: y7 = 3.5.

F7 = F 6 + f (−6)f T (−6)
     
6 −15 1
7 −21
= + 1 −6 = ,
−15 55 −6 −21 91
h7 = L−1 h 6 + f (0)y 7
 −1      
1 0 18.5 1 22
= + 3.5 = ,
1 1 −40.5 0 −59
    
0.4643 0.1071 22 3.896
θb7 = = .
0.1071 0.0357 −59 0.250

18 / 18