Autocorrelation in Time Series

KNNL – Chapter 12
 Issues in Autocorrelated Data
• When error terms are correlated (not independent),
problems occur when using ordinary least squares
(OLS) estimates
– Regression Coefficients are Unbiased, but not Minimum
Variance
– MSE underestimates s2
– Standard errors of regression coefficients based on OLS
underestimate the true standard error
–  Inflated t and F statistics and artificially narrow
confidence intervals

Autocorrelated Errors (1st Order)   t   t 1  ut

where ut  uncorrelated disturbances (typically assumed to be normal)
 First-Order Model - I
First-Order Autoregressive Model ( AR(1)) :
Simple Regression: Yt   0  1 X t   t t  1,..., n  t   t 1  ut
  autoregression parameter with   1
ut ~ N  0, s 2  and independent

Generalizes to Multiple Regression:

Yt   0  1 X t1  ...   p 1 X t , p 1   t t  1,..., n  t   t 1  ut

Properties of Errors (assumption regarding 1 for model consistency):

 s2 
1 ~ N  0, 2 
 1  
 s2  s2
 2  1  u2  E  2    E 1  E u2   0 s  2    s 1  s u2    
2 2 2 2 2
2 
s 
2

 1    1  2
s 2
Covariance: s  2 , 1  s  1  u2 , 1  s  2   s u2 , 1  s  2   0 
2 2

1  2
s 2
s  2 , 1 1  2
Corrrelation:   2 , 1   
s  2 s 1 s 2
s 2

1  2 1  2
 First-Order Model - II
In General:

 t   t 1  ut     t  2  ut 1   ut    t  2   ut 1  ut  ...    s ut  s
2

s 0

 s   2s 2 
s2
E  t   0 s  t   s   ut  s     s ut  s   s   
2 2 2 2s

 s 0  s 0 s 0 1  2
 ss 2
Covariance: s  t ,  t  s   s0
1  2
Correlation:   t ,  t  s    s s0

1  2  n 1 
 
  1   n2 
s 2
σ 2 ε   2
2   1  n 3 
1   
 
  n 1  n  2  n 3 
 1 

AR(2) :  t  1 t 1   2 t  2  ut Even Higher or models can be fit as well.

 Test For Independence - Durbin-Watson Test
Yt   0  1 X t1  ...   p 1 X t , p 1   t  t   t 1  ut ut ~ NID  0, s 2   1
H 0 :   0  Errors are uncorrelated over time
H A :   0  Positively correlated
1) Obtain Residuals from Regression
2) Compute Durbin-Watson Statistic (given below)
3) Obtain Critical Values from Table B.7, pp. 1330-1331 (R will provide a p-value)
If DW  d L  p  1, n   Reject H 0 If DW  dU  p  1, n   Conclude H 0 Otherwise Inconclusive
n

  et  et 1 
2

Test Statistic: DW  t 2
n

e
t 1
2
t

s 2
E  t   0 E  t  t 1 
1  2
n n n n
s 2 n
  e  e    e   e  2 et et 1  2 e  2n
2 2 2 2
t 1 t 1
1  2
t t t
t 2 t 2 t 2 t 2 t 1

 Under H 0 , expect DW  2
 Autocorrelation - Remedial Measures
• Determine whether a missing predictor variable can
explain the autocorrelation in the errors
• Include a linear (trend) term if the residuals show a
consistent increasing or decreasing pattern
• Include seasonal dummy variables if data are quarterly
or monthly and residuals show cyclic behavior
• Use transformed Variables that remove the (estimated)
autocorrelation parameter (Cochrane-Orcutt and
Hildreth-Lu Procedures)
• Use First Differences
• Estimated Generalized Least Squares
 Transformed Variables
Suppose  is known: Yt   0  1 X t   t  t   t 1  ut
Let Yt '  Yt  Yt 1    0  1 X t   t      0  1 X t 1   t 1  
 0 1     1  X t   X t 1     t   t 1    0 1     1  X t   X t 1   ut

 Yt '   0'  1' X t'  ut (Standard Simple linear regression with independent errors)
where:
Yt '  Yt  Yt 1 X t'  X t   X t 1  0'   0 1    1'  1

In Practice, we need to estimate  with a sample based value r

Yt '  Yt  rYt 1 X t'  X t  rX t 1
^
Fit: Y '  b0'  b1' X ' and if errors are uncorrelated, back transform to:
b0' s b0' 
s b1  s b1' 
^
Y  b0  b1 X where: b0  s b0   b1  b1'
1 r 1 r
 Cochrane-Orcutt Method
• Start by estimating  in Model: t = t-1 + ut by
regression through the origin for residuals (see below)
• Fit transformed regression model (previous slide)
• Check to see if new residuals are uncorrelated (Durbin-
Watson test), based on the transformed model
• If uncorrelated, stop and keep current model
• If correlated, repeat process with new estimate r based
on current regression residuals from the original (back
transformed) model
n

e e
t 1 t
r t 2
n

e
t 2
2
t 1
 Hildreth-Lu and First Difference Methods
• Hildreth-Lu Method
– Find value of r (between 0 and 1) that minimizes the SSE for
the transformed model by grid search
– Apply the transformed analysis based on the estimated r

• First Differences Method

– Uses  = 1 in transformed model (Yt’ = Yt – Yt-1 Xt’ = Xt – Xt-1 )
– Set b0’ = 0 and fits regression through origin of Y’ on X’
– When back-transforming:

b0  Y  b X '
1 b1  b 1
'
Forecasting with Autocorrelated Errors
Makes use of any of the 3 estimation techniques (C-O, H-L, First Differences):
Yt   0  1 X t   t  t   t 1  ut
 Yt   0  1 X t   t 1  ut   Yn 1   0  1 X n 1   n  un 1

3 Elements:
^
1. Expected Value:  0  1 X n 1 Estimated as Y n 1  b0  b1 X n 1
2. Multiple of period n Error Term:  n Estimated as ren
3. Current disturbance un 1 ~ N  0, s 2 

Forecast for period n  1 (note the notation is "Forecast", not F -distribution :

^
Fn 1  Y n 1  ren
Standard Error of the Prediction (based on transformed model):
 

s pred  MSE ' 1 
2 1
 n

X n 1  X ' 
'



 n  1
 
X i'  X '  
2

 i 2

Approximate 95% PI: Fn 1  t 1   2  ; n  3 s pred (First Differences has n - 2 df)