Session 2

Forecasting Mean of a Stationary Time Series

(ARIMA modelling)
PGPEX-term iv
 NONSTATIONARY PROCESSES

Yt  Yt 1  at Yt  Y0  a1  ...  at 1  at

E(Yt )  Y0  E(u1 )  ...  E(un )  Y0

 Y2  population variance of (Y0  a1  ...  at 1  at )
t

 population variance of (a1  ...  at 1  at )

  u2  ...   u2   u2
 t u2 The effect of shock is
permanent. This is a random
walk model

10/20/2017 Forecasting based on ARIMA model 2

 NONSTATIONARY PROCESSES

Random walk with

drift

Yt    Yt 1  at

Yt  t  Y0  a1  ...  at 1  at
E(Yt )  Y0  t

 Y2t  t u2

10/20/2017 Forecasting based on ARIMA model 3

1
 NONSTATIONARY PROCESSES
Deterministic trend (Trend stationary process)

Yt    t  at

EYt     t

It is nonstationary because the expected value of Yt is not independent of t.

10/20/2017 Forecasting based on ARIMA model 4

2
Different forms for the DF Test Regressions
Dickey Fuller tests are also known as  tests: , , .
The null (H0) and alternative (H1) models in each case are
i) H0: yt = yt-1+at
H1: yt = yt-1+at, <1
This is a test for nonstationary against a stationary
autoregressive process of order one (AR(1))

ii) H0: yt = yt-1+  +at

H1: yt = yt-1++at, <1
This is a test for a non- stationary series with drift against a
stationary AR(1) with drift.

iii) H0: yt = yt-1+ + at

H1: yt = yt-1++t+at, <1
This is a test for a non-stationary series with drift against a
stationary AR(1) with drift and a time trend.

10/20/2017 Forecasting based on ARIMA model 5

 Can we make any transformation to make a
series Stationary
r.w. : yt = yt-1 + at
(1)
If we take (1) and subtract yt-1 from
both sides:
yt - yt-1 = at
yt = at (white noise)
We say that we have induced stationarity
by “differencing once”.
=> Yt is I(1) why:

Yt  Y0  a1  ...  at 1  at
10/20/2017 Forecasting based on ARIMA model 6
 Modeling The Stationary Series

Basic models for stationary time series

• Autoregressive (AR) model
• Moving average (MA) model
• Autoregressive Moving Average (ARMA)
model

10/20/2017 Forecasting based on ARIMA model 7

Yt = φ0 + φ1Yt – 1 + at , at white noise
Autoregressive Processes AR(1)
3.5 5.0

3.0 1  0.4 4.5

1  0.7
2.5 4.0

2.0 3.5

1.5 3.0

1.0 2.5

0.5 2.0

0.0 1.5
200 225 250 275 300 325 350 375 400 200 225 250 275 300 325 350 375 400

Y Y

23 400

22 1  0.95 360
1  1
21
320
20

19 280

18
240
17
200
16

15 160
200 225 250 275 300 325 350 375 400 200 225 250 275 300 325 350 375 400

Y Y

10/20/2017 Forecasting based on ARIMA model 8

 Autocorrelation
• Correlation between a variable and its lagged
version (one time-step or more)

= Observation in time period t

= Observation in time period t – k
= Mean of the values of the series
= Autocorrelation coefficient for k-step lag

10/20/2017 Forecasting based on ARIMA model 9

 Correlogram or ACF plot
• Plots the ACF or Autocorrelation function (rk)
against the lag (k).
• Plus-and-minus two-standard errors are
displayed as limits to be exceeded for
statistical significance.
• Reveals lagged variables that can be
potentially useful for forecasting.

10/20/2017 Forecasting based on ARIMA model 10

 Additional tools for Exploring the data
• Carry out Portmanteau test (Ljung-Box):

• Q has to be compared with chi-square cut-off with m degrees

of freedom.

10/20/2017 Forecasting based on ARIMA model 11

 Graphical Presentation
Autoregressive Model
• Yt = φ0 + φ1Yt – 1 + at , at white noise
(uncorrelated, mean zero and constant
variance random process)

ACF plot
Errors should be white noise, that can also be tested by Lung-Box Q statistic
With degree of freedom would be m-g, where g is the order of AR process.

10/20/2017 Forecasting based on ARIMA model 12

 PACF

A stationary autoregressive process has a geometrically

decaying acf number of spikes of pacf = AR order

PACF (partial ACF) plot

10/20/2017 Forecasting based on ARIMA model 13
 ACF plot of Non stationary series

10/20/2017 Forecasting based on ARIMA model 14

 MA(1) model
• Yt = θ0 + at + θ1 at – 1 , at white noise

θ1 = 0.8

θ1 = – 0.8

10/20/2017 ACF plot Forecasting based on ARIMA model 15

 ARMA(p,q) model
• Yt = φ0 + φ1Yt – 1 + φ2Yt – 2 + … + φpYt – p
+ at + θ1 at – 1 + θ2 at – 2 + … + θq at – q
εt white noise
Such a model has non-zero ACF and non-zero PACF at all lags

• If an ARMA(p,q) model is to be fitted, the parameters φ0,

φ1, φ2,…, φp, θ1, θ2,…, θq have to be estimated from the
data, under the restriction that the estimated values
produce a stationary process

• AR(p) is ARMA(p,0)
• MA(q) is ARMA(0,q)

10/20/2017 Forecasting based on ARIMA model 16

 Forecast

10/20/2017 Forecasting based on ARIMA model 17

 Forecast

10/20/2017 Forecasting based on ARIMA model 18

 Forecast

10/20/2017 Forecasting based on ARIMA model 19

 MA model

10/20/2017 Forecasting based on ARIMA model 20

 Forecast

10/20/2017 Forecasting based on ARIMA model 21

 ARIMA(p,d,q) model
• If d-times differenced series is ARMA(p,q), then
original series is said to be ARIMA(p,d,q).
• ARIMA stands for ‘Autoregressive Integrated
Moving average’.
• If Wt is the differenced version of Yt, i.e.,
Wt = Yt – Yt – 1,
then Yt can be written as
Yt = Wt + Wt – 1 + Wt – 2 + Wt – 3 + … .
Thus, the series Yt is an ‘integrated’ (opposite of
‘differenced’) version of the series Wt.
• If Yt is ARIMA(p,d,q), it is non-stationary.
• However, its d-times differenced version, an
ARMA(p,q) process, can be stationary.

10/20/2017 Forecasting based on ARIMA model 22

 Forecast from Differenced Series to
Non-stationary Series

10/20/2017 Forecasting based on ARIMA model 23

Box-Jenkins ARIMA model-building
• Model identification
– If the time plot ‘looks’ non-stationary, difference it until
the plot looks stationary
– Look at ACF and PACF plots for possible clue on model
order (p, q)
– When in doubt (regarding choice of p and q), use the
principle of parsimony: A simple model is better than a
complex model
• Estimate model parameters
• Check residuals for health of model
• Iterate if necessary
• Forecast using the fitted model

10/20/2017 Forecasting based on ARIMA model 24