You are on page 1of 59

Time Series Analysis and

Forecasting

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Introduction
A time

series is a set of observations generated

sequentially in time
Continuous
The

vs. discrete time series

observations from a discrete time series, made

at some fixed interval h, at times 1, 2,, N may


be denoted by x(1), x(2),, x(N)
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Introduction (cont.)
Discrete

time series may arise in two ways:

1- By sampling a continuous time series


2- By accumulating a variable over a period of time

Characteristics

of time series

Time periods are of equal length


No missing values

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Components of a time series


A t im e s e r ie s
p a tte rn c o m p o n e n t

r a n d o m (e r ro r ) c o m p o n e n t

tre n d p a tte rn
s e a s o n a l p a tte rn
c y c lic p a t t e r n
s ta tis tic a l p a tte r n

x =F +x

Time Series Analysis Lecture Notes


t
t
t
MA(4030)Prepared By TMJA
Cooray

Areas of application
Forecasting
Determination
Design

of a transfer function of a system

of simple feed-forward and feedback

control schemes

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Forecasting
Applications

Lead

Economic and business planning


Inventory and production control
Control and optimization of industrial processes

time of the forecasts

is the period over which forecasts are needed


Degree

of sophistication

Simple ideas

Moving averages
Simple regression techniques

Complex statistical concepts

Box-Jenkins methodology

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Approaches to forecasting
Self-projecting

approach

Cause-and-effect

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

approach

Approaches to forecasting (cont.)


Self-projecting

approach

Advantages

Quickly and easily applied


A minimum of data is required
Reasonably short-to medium-term
forecasts
They provide a basis by which
forecasts developed through other
models can be measured against

Cause-and-effect
Advantages

Bring more information


More accurate medium-to
long-term forecasts

Disadvantages

Disadvantages

approach

Not useful for forecasting into the


far future
Do not take into account external
factors

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Forecasts of the explanatory


time series are required

Some traditional self-projecting models


Overall

trend models

The trend could be linear, exponential, parabolic, etc.


A linear Trend has the form

Trendt = A + Bt

Short-term changes are difficult to track

Smoothing

models

Respond to the most recent behavior of the series


Employ the idea of weighted averages
They range in the degree of sophistication
The simple exponential smoothing method:

Ft

Notes
z t Az t 1Time
MA(4030)Prepared
(Series
1 Analysis
A)FLecture

at
tBy
1 TMJA
Cooray

Some traditional self-projecting


models (cont.)
Seasonal

models

Very common
Most seasonal time series also contain long- and short-

term trend patterns


Decomposition

models

The series is decomposed into its separate patterns


Each pattern is modeled separately
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Drawbacks of the use of traditional


models
There

is no systematic approach for the identification


and selection of an appropriate model, and therefore,
the identification process is mainly trial-and-error
There is difficulty in verifying the validity of the
model
Most traditional methods were developed from intuitive

and practical considerations rather than from a statistical


foundation
Too narrow to

deal efficiently with all time series


Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

ARIMA models
Autoregressive

Integrated Moving-average
Can represent a wide range of time series
A stochastic
stochastic modeling approach that can be used
to calculate the probability of a future value lying
between two specified limits

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

ARIMA models (Cont.)


In

the 1960s Box and Jenkins recognized the


importance of these models in the area of
economic forecasting
Time series analysis - forecasting and control
control
George E. P. Box

Gwilym M. Jenkins

1st edition was in 1976

Often

called The Box-Jenkins approach


B o x - J e n k in s m o d e ls
U n iv a r ia te

M u ltiv a r ia te (tr a n s fe r fu n c tio n )


Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Transfer function modeling

Yt

= (B)Xt where

(B) = 0 + 1B + 2B2 + ..
B

is the backshift operator

BmXt = Xt - m
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Transfer function modeling (cont.)


The

study of process dynamics can achieve:

Better control
Improved design

Methods

for estimating transfer function models

Classical methods

Based on deterministic perturbations


Uncontrollable disturbances (noise)
noise are not accounted for,
and hence, these methods have not always been successful

Statistical methods

Make allowance for noise


noise
The Box-Jenkins
Timemethodology
Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Process control
Feed-forward

control

Feedback

control

Deviation from target output


P

Deviation from
target output

Nt

Nt

( B)(B)B

1
1

L (B) L 2 (B)B

f 1

Compensating
variable Xt+
zt

Control equation

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

L 11 (B) L 2 ( B)B f

Compensating
variable Xt+
Control equation

Process control (cont.)


D is tu r b a n c e s
m easu ed

u n m e a su r ed

fe e d -fo r w a r d c o n to l

fe e d b a c k c o n tr o l

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Process control (cont.)


The

Box-Jenkins approach to control is to typify the


disturbance by a suitable time series or stochastic model
and the inertial characteristics of the system by a
suitable transfer function model
The Control equation, allows the action which should
be taken at any given time to be calculated given the
present and previous states of the system
Various ways corresponding to various levels of
technological sophistication can be used to execute a
control action called for by the control equation

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

The Box-Jenkins model building


process
Model identification

Model estimation

No
Modify
model

Is model
adequate ?

Yes
Forecasts

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

The Box-Jenkins model building


process (cont.)

Model identification

Model estimation

The objective is to minimize the sum of squares of


errors

Model validation

Autocorrelations
Partial-autocorrelations

Certain diagnostics are used to check the validity of the


model

Model forecasting

The estimated model is used to generate forecasts and


confidence limits of the forecasts
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Important Fundamentals
A Normal

process
Stationarity
Regular differencing
Autocorrelations (ACs)
The white noise process
The linear filter model
Invertibility
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

A Normal process (A Gaussian


process)
The

Box-Jenkins methodology analyze a time series as a


realization of a stochastic process.
The observation zt at a given time t can be regarded as a

realization of a random variable zt with probability density


function p(zt)
The observations at any two times t 1 and t2 may be regarded as
realizations of two random variables zt1, zt2 and with joint
probability density function p(z t1, zt2)
If the probability distribution associated with any set of times is

multivariate Normal distribution, the process is called a normal


or Gaussian process
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Stationary stochastic processes


In

order to model a time series with the Box-Jenkins

approach, the series has to be stationary


In practical terms,
terms

the series is stationary if tends to

wonder more or less uniformly about some fixed level


In statistical terms,
terms

a stationary process is assumed to

be in a particular state of statistical equilibrium, i.e.,


p(xt) is the same for all t

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Stationary stochastic processes (cont.)


the

process is called strictly stationary


stationary

if the joint probability distribution of any m

observations made at times t1, t2, , tm is the same as


that associated with m observations made at times t1 + k,
t2 + k, , tm + k
When

m = 1, the stationarity assumption implies

that the probability distribution p(zt) is the same


for all times t
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Stationary stochastic processes (cont.)


In

particular, if zt is a stationary process, then the

first difference zt = zt - zt-1and higher differences


dzt are stationary

Most

time series are nonstationary

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Achieving stationarity
Regular

differencing (RD)

(1st order)
(2nd order)

xt = (1 B)xt = xt xt-1

2xt = (1 B)2xt = xt 2xt-1 + xt-2

B is the backward shift operator


It

is unlikely that more than two regular differencing

would ever be needed


Sometimes

regular differencing by itself is not

sufficient and prior transformation is also needed


Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Some nonstationary series

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Some nonstationary series (cont.)

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Some nonstationary series (cont.)


How
Howcan
canwe
wedetermine
determinethe
the
number
numberof
ofregular
regular
differencing
differencing??

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Autocorrelations (ACs)
Autocorrelations are

statistical measures that indicate


how a time series is related to itself over time

The

autocorrelation at lag 1 is the correlation between


the original series zt and the same series moved
forward one period (represented as zt-1)

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Autocorrelations (cont.)
The

theoretical autocorrelation function


E (z t )(z t k )
k
2z

The

sample autocorrelation
N k

rk

(z
t 1

z )(z t k z )

2
(
z

z
)
t
t 1

k 0,1,2,...k

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Autocorrelations (cont.)
A graph

of the correlation values is called a


correlogram
correlogram
In practice, to obtain a useful estimate of the
autocorrelation function, at least 50 observations
are needed
The estimated autocorrelations rk would be
calculated up to lag no larger than N/4

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

A correlogram of a nonstationary
time seies

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

After one RD

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

After two RD

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

The white noise process


The

Box-Jenkins models are based on the idea that


a time series can be usefully regarded as generated
from (driven by) a series of uncorrelated
independent shocks
shocks et
E e t 0 var e t e2
1
k
0

k0
k0

Such

a sequence et, et-1, et-2, is called a white


noise process
process
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

The linear filter model


( B)
White noise

Linear filter

xt

et
A linear filter
filter

is a model that transform the


white noise process et to the process that generated
the time series xt
x t e t 1e t 1 2 e t 2 ... (B)e t
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

The linear filter model (cont.)


is the transfer function


function of the filter

(B) 1 1 B 2 B ... j B with 0 1


2

j 0

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

The linear filter model (cont.)


The

linear filter can be put in another form

x t 1x t 1 2 x t 2 ... e t

x
j

j1

This

t j

et

form can be written

( B) x t e t
( B) 1

jB j

z t (B)at
(B)z t at

jTime
1 Series Analysis Lecture
( BNotes
)
MA(4030)Prepared By TMJA
Cooray

1 (B)

Stationarity and invertibility


conditions for a linear filter
For

a linear process
to be stationary,
stationary

(B) must converge


for B 1

If

the current observation xt


depends on past observations
with weights which decrease as
we go back in time, the series is
called invertible

For

a linear process to be
invertible,
invertible (B) must converge

for B 1

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Model building blocks


Autoregressive

(AR) models
Moving-average (MA) models
Mixed ARMA models
Non stationary models (ARIMA models)
The mean parameter
The trend parameter

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Autoregressive (AR) models


An

autoregressive model of order p

x t 1x t 1 2 x t 2 ... p x t p e t

(B) x t e t

(B) 1 1B 2 B 2 ... p B p
The

autoregressive process can be thought of as


the output from a linear filter with a transfer
function -1(B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
Time Series Analysis Lecture Notes
equation
equation
MA(4030)Prepared By TMJA

The

Cooray

Moving-average (MA) models


A moving-average

model of order q

x t e t 1e t 1 2 e t 2 ... q e t q
x t (B)e t
2

(B) 1 1B 2 B ... q B

The

moving-average process can be thought of as


the output from a linear filter with a transfer
function (B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
Time Series Analysis Lecture Notes
equation
equation
MA(4030)Prepared By TMJA

The

Cooray

Mixed AR and MA (ARMA) models


A moving-average

written as

process of 1st order can be

1
zt a t
(1 1B)
(1 1B 12 B 2 13 B3 ...) x t e t
x t 1x t 1 12 x t 2 13 x t 3 ... e t

Hence,
Hence

if the process were really MA(1), we


would obtain a non parsimonious representation in
terms of an autoregressive model
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Mixed AR and MA (ARMA) models


(cont.)
In

order to obtain a parsimonious model,


sometimes it will be necessary to include both AR
and MA terms in the model
(B)
An ARMA(p, q) model
xt
et

(B)

The ARMA process

can be thought of as the


output from a linear filter with a transfer function
(B)/(B),
(B) when the input is white noise at
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

The Box-Jenkins model building


process
Model identification
Autocorrelations
Partial-autocorrelations

Model

estimation
Model validation
Certain diagnostics are used to check the validity of the

model
Model

forecasting
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Partial-autocorrelations (PACs)
Partial-autocorrelations are

another set of statistical


measures are used to identify time series models

PAC

is Similar to AC, except that when calculating


it, the ACs with all the elements within the lag are
partialled out (Box & Jenkins, 1976)

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Partial-autocorrelations (cont.)
PACs

can be calculated from the values of the ACs


where each PAC is obtained from a different set of
linear equations that describe a pure
autoregressive model of an order that is equal to
the value of the lag of the partial-autocorrelation
computed

PAC

at lag k is denoted by kk

The double notation kk is to emphasize that kk is the

autoregressive parameter k of the autoregressive model


of order k
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Model identification
The

sample ACs and PACs are computed for the


series and compared to theoretical autocorrelation
and partial-autocorrelation functions for candidate
models investigated
Stationarity and
invertibility
conditions

Theoretical ACs and


PACs

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Stationarity and invertibility


conditions
For

a linear process
to be stationary,
stationary

For

a linear process to be
invertible,
invertible

(B) must converge

(B) must converge

for B 1

for B 1

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Stationarity requirements for AR(1)


model

For an AR(1) to be stationary:

-1 < 1 < 1
i.e., the roots of the characteristic
equation 1 - 1B = 0 lie outside
the unit circle

For an AR(1) it can be shown that:


k = 1 k 1 which with 0 = 1 has

the solution
k = 1k
k>0
i.e., for a stationary AR(1) model,
the theoretical autocorrelation
function decays exponentially to
zero, however, the theoretical
partial-autocorrelation function has
a cut off after the 1st lag
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Invertibility requirements for a MA(1)


model
For

a MA(1) to be invertible:

-1 < 1 < 1

i.e., the roots of the characteristic equation 1 - 1B = 0


lie outside the unit circle
1

For

a MA(1) it can be shown that:

k 1
0

12

k 1
k 1

i.e., for an invertible MA(1) model, the theoretical


autocorrelation function has a cut off after the 1st lag,
however, the theoretical partial-autocorrelation function
Time Series Analysis Lecture Notes
decays exponentially
to zero
MA(4030)Prepared By TMJA
Cooray

Higher order models


For

an AR model of order p > 1:

The autocorrelation function consists of a mixture of

damped exponentials and damped sine waves


The partial-autocorrelation function has a cut off after
the p lag
For

a MA models of order q > 1:

The autocorrelation function has a cut off after the q lag


The partial-autocorrelation function consists of a

mixture of damped exponentials and damped sine


waves
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Permissible regions for the AR and


MA parameters

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Theoretical ACs and PACs (cont.)

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Theoretical ACs and PACs (cont.)

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Model identification

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Model estimation

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray

Model verification

Time Series Analysis Lecture Notes


MA(4030)Prepared By TMJA
Cooray