TOPIC FIVE

V. Forecasting
V.1 The Box-Jenkins approach

Having fitted an ARMA model to {x1, x2, ... xn} we have the equation:

Xn+k = µ + ,1 (xn+k-1 – µ) + ... + ,p (xn+k-p – µ) + en+k + .1en+k-1 + ...+ .qen+k-q

x1 x2 ... ... xn .... xn+k

S

?
• • •

1 2 n n+k time

x̂ n (k) = Forecast value of xn+k, given all observations up until time n.

= k-step ahead forecast at time n.

In the Box-Jenkins approach, x̂ n (k) is taken as E(Xn+k | X1 , ... , Xn), i.e. x̂ n (k) is the conditional
expectation of the future value of the process, given the information currently available.

From result 2 in ST3053 (section A), we know that E(Xn+k | X1 , ... , Xn) minimises the mean square error
E(Xn+k – h( X1 , ... , Xn))2 of all functions h(X1 , ... , Xn).

x̂ n (k) is calculated as follows from the equation for Xn+k:

• Replace all unknown parameters by their estimated values

• Replace random variables X1, ..., Xn by their observed values x1 , ... , xn.

• Replace random variables Xn+1 , ... , Xn+k-1 by their forecast values, x̂ n (1) , ... , x̂ n (k-1)

• Replace variables e1 , ... , en by the residuals eˆ1 , ... , eˆ n

• Replace variables en+1 , ... , en+k-1 by their expectations 0.

Example: AR(2) model xn = µ + !1 ( xn-1 - µ ) + ! 2 ( xn-2 - µ ) + en . Since

X n+1 = µ + !1 ( X n – µ ) + ! 2 ( X n"1 – µ ) + en+1

X n+2 = µ + !1 ( X n+1 – µ ) + ! 2 ( X n – µ ) + en+2
we have
xˆn (1) = µˆ + !ˆ1 ( xn " µˆ ) + !ˆ 2 ( xn-1 " µˆ )
xˆn (2) = µˆ + !ˆ1 ( xˆn (1) " µˆ ) + !ˆ 2 ( xn " µˆ )
Example: 2-step ahead forecast of an ARMA(2,2) model

xn = µ + !1 ( xn-1 - µ ) + ! 2 ( xn-2 - µ ) + en + "2en#2 .

Since xn+2 = µ + !1 ( xn+1 - µ ) + ! 2 ( xn - µ ) + en+ 2 + "2en , we have

xˆn (2) = µˆ + !ˆ1 ( xˆn (1) - µˆ ) + !ˆ 2 ( xn - µˆ ) + "ˆ2eˆn

The (forecast) error of the forecast x̂ n (k) is

x n+k - xˆ n (k )

The expected value of this error is

E(xn+k - xˆ n (k) | x1,...,x n ) = xˆ n (k) - xˆ n (k) = 0

Hence the variance of the forecast error is

E((x n+k ! xˆ n (k ))2 | x1 ,..., x n )

This is needed for confidence interval forecasts as it is more useful than a point estimate.

For stationary processes, it may be shown that x̂ n (k) ! µ as k ! " . Hence, the variance of the
forecast error tends to E(xn+k-µ)2 = )2 as k % &, where )2 is the variance of the process.

V.2 Forecasting ARIMA processes

If X is ARIMA(p,d,q) then Z = ! d X is ARMA(p,q).

• Use methods reviewed to produce forecasts for Z

• Reverse the differencing procedure to produce forecasts for X

Example: if X is ARIMA(0,1,1) then Z = !X is ARMA(0,1), leading to the forecast ẑn (1) .

But Xn+1 = Xn + Zn+1, so xˆ n (1) = x n + zˆ n (1)

Question: Find x̂ n (2) for an ARIMA(1,2,1) process.

Let Z n = ! 2 X n and assume Zn = µ + , (Zn-1 – µ) + en + .en-1, but

Z n+ 2 = !2 X n+ 2 = ( X n+ 2 " X n+1 ) " ( X n+1 " X n )

= X n+ 2 " 2 X n+1 + X n
so Xn+2 = 2Xn+1 – Xn + Zn+2. Hence,

ˆ ˆ n (1) ! µˆ
xˆ n (2) = 2xˆ n (1) ! x n + zˆ n (2) = 2xˆ n (1) ! x n + µˆ +!(z

V.3 Exponential smoothing and Holt-Winters

• The Box-Jenkins method requires a skilled operator in order to obtain reliable results.

• For cases where only a simple forecast is needed, exponential smoothing is much simpler (Holt,
1958).

A weighted combination of past values is used to predict future observations. For example, the first
forecast for an AR model is obtained by

( 2
xˆn (1) = ! xn + (1 " ! ) xn "1 + (1 – ! ) xn "2 + ... )
or

"
!
xˆn (1) = ! # (1- ! )i xn-i = xn
i =0 1- (1- ! ) B

!
!
• The sum of the weights is ! " (1-!)i = =1
i=0 1-(1-!)

• Generally we use a value of , such that 0 < , < 1, so that there is less emphasis on historic values
further back in time (usually, 0.2 6 , 6 0.3).

• There is only one parameter to control, usually estimated via least squares.

• The weights decrease geometrically – hence the name exponential smoothing.

Updating forecasts is easy with exponential smoothing:

Xn-1 Xn Xn+1

5
?
5
| | |
n-1 n n+1

It is easy to see that

xˆn (1) = (1- ! ) xˆn-1 (1) + ! xn = xˆn-1 (1) + ! ( xn - xˆn-1 (1))

Current forecast = previous forecast + , 7 (error in previous forecast).

 • Simple exponential smoothing can’t cope with trend or seasonal variation.

• Holt-Winters smoothing can cope with trend and seasonal variation

• Holt –Winters can sometimes outperform Box-Jenkins forecasts.

V.4 Linear filtering

input process linear filter output process

xt yt

time Series filter weights time series

A linear filter is a transformation of a time series {xt} (the input series) to create an output series {yt}
which satisfies:

!
yt = #a k x t-k .
k= "!

The collection of weights {ak : k % Z} forms a complete description of the filter.

The objective of the filtering is to modify the input series to meet particular objectives, or to display
specific features of the data. For example, an important problem in analysis of economic time series is
detection, isolation and removal of deterministic trends.

In practice, a filter {ak : k % Z} normally contains only a relatively small number of non-zero
components.

Example: regular differencing. This is used to remove a linear trend. Here a0 = 1, a1 = -1, ak = 0
otherwise. Hence yt = xt – xt-1.

Example: seasonal differencing. Here a0 = 1, a12 = -1, ak = 0 otherwise, and yt = xt – xt-12.

Example: if the input series is a white noise and the filter takes the form {.0 = 1, .1, ... , .q}, then the
output series is MA(q), since
q
yt = " ! k et -k
k =0

If the input series, x, is AR(p), and the filter takes the form {,0 = 1, -,1, ... , -,p}, then the output series is
white noise
p
yt = xt " # ! k xt -k = et
k =1