Forecasting

JY Le Boudec

1
 Contents

1. What is
forecasting ?
2. Linear Regression
3. Avoiding Overfitting
4. Differencing
5. ARMA models
6. Sparse ARMA
models
7. Case Studies

2
1. What is forecasting ?
Assume you have been able to define the nature of
the load
It remains to have an idea about its intensity

It is impossible to forecast without error

The good engineer should

Forecast what can be forecast

Give uncertainty intervals

The rest is outside our control

3
4
 2. Linear Regression
Simple, for simple cases
Based on extrapolating the explanatory variables

5
6
7
8
 Estimation and Forecasting

In practice we estimate from y, , yt

When computing the forecast, we pretend is known, and thus
make an estimation error
It is hoped that the estimation error is much less than the
confidence interval for forecast

In the case of linear regression, the theorem gives the global error
exactly
In general, we wont have this luxury

9
10
 We saw this already
A case where estimation error versus prediction uncertainty can
be quantified

Prediction interval if model is known

Prediction interval accounting for estimation (t = 100 observed
points)
11
 3. The Overfitting Problem
The best model is not necessarily the one that fits best

12
Prediction for the better model

This is the overfitting problem

13
 How to avoid overfitting
Method 1: use of test data
Method 2: information criterion

14
15
16
Best Model for Internet Data,
polynomial of degree up to 2

17
d=1

18
Best Model for Internet Data,
polynomial of degree up to 10

19
4. Differencing the Data

20
21
22
Point Predictions from
Differenced Data

23
Background On Filters (Appendix
B)
We need to understand how to use discrete filters.

Example: write the Matlab command for

24
25
 A simple filter

Q: compute X back from Y

26
27
28
29
Impulse Response

30
31
32
A filter with stable inverse

33
 How is this prediction done ?

This is all very intuitive

34
35
Prediction assuming differenced
data is iid

36
 Prediction Intervals
A prediction without prediction intervals is only a small part of the story
The financial crisis might have been avoided if investors had been aware of
prediction intervals

37
38
39
 Compare the Two

Linear Regression with 3 parameters + variance Assuming differenced data is iid

40
41
 5. Using ARMA Models
When the differenced data appears stationary but not iid

42
Test of iid-ness

43
44
ARMA Process

45
46
ARMA Processes are Gaussian
(non iid)

47
48
49
ARIMA Process

50
 Fitting an ARMA Process
Called the Box-Jenkins method
Difference the data until stationary
Examine ACF to get a feeling of order (p,q)
Fit an ARMA model using maximum likelihood

51
 Fitting an ARIMA Model
Apply Scientific Method
1. make stationary and normal (how ?)
2. bound orders p,q
3. fit an ARMA model to Yt -
i.e. Yt - ARMA
4. compute residuals and verify white noise and normal

Fitting an ARMA model

Pb is :
given orders p,q
given (x1, xn) (transformed data)
compute the parameters of an ARMA (p,q) model that
maximizes the likelihood
Q:What are the parameters ?

A: the mean , the polynomial coefficients k and k , the noise

variance 2
52
This is a non-linear optimization
problem
Maximizing the likelihood is a non-linear optimization problems

Usually solved by iterative, heuristic algorithms,

may converge to a local maximum
may not converge

Some simple, non MLE, heuristics exist for AR or MA models

Ex: fit the AR model that has the same theoretical ACF as the
sample ACF

Common practice is to bootstrap the optimization procedure by

starting with a best guess
AR or MA fit, using heuristic above

53
Fitting ARMA Model is Same as
Minimizing One-Step ahead
prediction error

54
Best Model Order

55
Check the Residuals

56
Example

57
58
 Forecasting with ARMA
Assume Yt is fitted to an ARMA process

The prediction problem is:

given Y1=y1,,Yt=yt find the conditional distribution of Yt+h

We know it is normal, with a mean that depends on (y1,,yt) and

a variance that depends only on the fitted parameters of the
ARMA process

There are many ways to compute this; it is readily done by

Matlab

59
60
61
Forecasting Formulae for ARIMA
Y = original data
X = differenced data, fitted to an ARMA model
1. Obtain point prediction for X using what we just saw
2. Apply Proposition 6.4.1 to obtain point prediction for Y
3. Apply formula for prediction interval

There are several other methods, but they may have numerical
problems. See comments in the lecture notes after prop 6.5.2

62
63
 Improve Confidence Interval If
Residuals are not Gaussian (but
appear to be iid)
Assume residuals are
not gaussian but are
iid
How can we get
confidence
intervals ?

Bootstrap by
sampling from
residuals

64
65
With gaussian With bootstrap from
assumption residuals

66
 6. Sparse ARMA Models
Problem: avoid many parameters when the degree of the A and
C polynomials is high, as in the previous example

Based on heuristics
Multiplicative ARIMA, constrained ARIMA
Holt Winters

67
68
Holt Winters Model 1: EWMA

69
70
EWMA is OK when there is no
trend and no periodicity

71
72
73
74
75
76
77
 Constrained ARIMA

Sparse models give less

accurate predictions but
have much fewer
parameters and are
simple to fit.

(corrected or not)
78
7. Case Studies

79
80
81
82
83
84
85
86
87
hh == 11

88
89
hh == 22

90
91
 log
log
hh == 11

92
 Conclusion
Forecasting is useful when savings matter; for example
Save money on server space rental
Save energy

Capturing determinism is perhaps most important and easiest

Prediction intervals are useful to avoid gross mistakes

Re-scaling the data may help

vous de jouer.

93