You are on page 1of 59

# Time Series Analysis and

Forecasting

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Introduction
A time

## series is a set of observations generated

sequentially in time
Continuous
The

## at some fixed interval h, at times 1, 2,, N may

be denoted by x(1), x(2),, x(N)
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Introduction (cont.)
Discrete

## 1- By sampling a continuous time series

2- By accumulating a variable over a period of time

Characteristics

of time series

## Time periods are of equal length

No missing values

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Components of a time series

A t im e s e r ie s
p a tte rn c o m p o n e n t

r a n d o m (e r ro r ) c o m p o n e n t

tre n d p a tte rn
s e a s o n a l p a tte rn
c y c lic p a t t e r n
s ta tis tic a l p a tte r n

x =F +x

## Time Series Analysis Lecture Notes

t
t
t
MA(4030)Prepared By TMJA
Cooray

Areas of application
Forecasting
Determination
Design

control schemes

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Forecasting
Applications

Lead

## Economic and business planning

Inventory and production control
Control and optimization of industrial processes

## is the period over which forecasts are needed

Degree

of sophistication

Simple ideas

Moving averages
Simple regression techniques

## Complex statistical concepts

Box-Jenkins methodology

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Approaches to forecasting
Self-projecting

approach

Cause-and-effect

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

approach

Self-projecting

approach

Advantages

## Quickly and easily applied

A minimum of data is required
Reasonably short-to medium-term
forecasts
They provide a basis by which
forecasts developed through other
models can be measured against

Cause-and-effect
Advantages

## Bring more information

More accurate medium-to
long-term forecasts

Disadvantages

Disadvantages

approach

## Not useful for forecasting into the

far future
Do not take into account external
factors

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Forecasts of the explanatory

time series are required

Overall

trend models

## The trend could be linear, exponential, parabolic, etc.

A linear Trend has the form

Trendt = A + Bt

Smoothing

models

## Respond to the most recent behavior of the series

Employ the idea of weighted averages
They range in the degree of sophistication
The simple exponential smoothing method:

Ft

Notes
z t Az t 1Time
MA(4030)Prepared
(Series
1 Analysis
A)FLecture

at
tBy
1 TMJA
Cooray

## Some traditional self-projecting

models (cont.)
Seasonal

models

Very common
Most seasonal time series also contain long- and short-

Decomposition

models

## The series is decomposed into its separate patterns

Each pattern is modeled separately
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

models
There

## is no systematic approach for the identification

and selection of an appropriate model, and therefore,
the identification process is mainly trial-and-error
There is difficulty in verifying the validity of the
model
Most traditional methods were developed from intuitive

foundation
Too narrow to

## deal efficiently with all time series

Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

ARIMA models
Autoregressive

Integrated Moving-average
Can represent a wide range of time series
A stochastic
stochastic modeling approach that can be used
to calculate the probability of a future value lying
between two specified limits

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

In

## the 1960s Box and Jenkins recognized the

importance of these models in the area of
economic forecasting
Time series analysis - forecasting and control
control
George E. P. Box

Gwilym M. Jenkins

Often

## called The Box-Jenkins approach

B o x - J e n k in s m o d e ls
U n iv a r ia te

## M u ltiv a r ia te (tr a n s fe r fu n c tio n )

Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

## Transfer function modeling

Yt

= (B)Xt where

(B) = 0 + 1B + 2B2 + ..
B

## is the backshift operator

BmXt = Xt - m
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

The

Better control
Improved design

Methods

## for estimating transfer function models

Classical methods

## Based on deterministic perturbations

Uncontrollable disturbances (noise)
noise are not accounted for,
and hence, these methods have not always been successful

Statistical methods

## Make allowance for noise

noise
The Box-Jenkins
Timemethodology
Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Process control
Feed-forward

control

Feedback

control

P

Deviation from
target output

Nt

Nt

( B)(B)B

1
1

L (B) L 2 (B)B

f 1

Compensating
variable Xt+
zt

Control equation

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

L 11 (B) L 2 ( B)B f

Compensating
variable Xt+
Control equation

## Process control (cont.)

D is tu r b a n c e s
m easu ed

u n m e a su r ed

fe e d -fo r w a r d c o n to l

fe e d b a c k c o n tr o l

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

The

## Box-Jenkins approach to control is to typify the

disturbance by a suitable time series or stochastic model
and the inertial characteristics of the system by a
suitable transfer function model
The Control equation, allows the action which should
be taken at any given time to be calculated given the
present and previous states of the system
Various ways corresponding to various levels of
technological sophistication can be used to execute a
control action called for by the control equation

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## The Box-Jenkins model building

process
Model identification

Model estimation

No
Modify
model

Is model
adequate ?

Yes
Forecasts

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## The Box-Jenkins model building

process (cont.)

Model identification

Model estimation

## The objective is to minimize the sum of squares of

errors

Model validation

Autocorrelations
Partial-autocorrelations

## Certain diagnostics are used to check the validity of the

model

Model forecasting

## The estimated model is used to generate forecasts and

confidence limits of the forecasts
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Important Fundamentals
A Normal

process
Stationarity
Regular differencing
Autocorrelations (ACs)
The white noise process
The linear filter model
Invertibility
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

process)
The

## Box-Jenkins methodology analyze a time series as a

realization of a stochastic process.
The observation zt at a given time t can be regarded as a

## realization of a random variable zt with probability density

function p(zt)
The observations at any two times t 1 and t2 may be regarded as
realizations of two random variables zt1, zt2 and with joint
probability density function p(z t1, zt2)
If the probability distribution associated with any set of times is

## multivariate Normal distribution, the process is called a normal

or Gaussian process
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

In

## approach, the series has to be stationary

In practical terms,
terms

## wonder more or less uniformly about some fixed level

In statistical terms,
terms

## be in a particular state of statistical equilibrium, i.e.,

p(xt) is the same for all t

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

the

stationary

## observations made at times t1, t2, , tm is the same as

that associated with m observations made at times t1 + k,
t2 + k, , tm + k
When

## that the probability distribution p(zt) is the same

for all times t
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

In

## first difference zt = zt - zt-1and higher differences

dzt are stationary

Most

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Achieving stationarity
Regular

differencing (RD)

(1st order)
(2nd order)

xt = (1 B)xt = xt xt-1

It

Sometimes

## sufficient and prior transformation is also needed

Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

How
Howcan
canwe
wedetermine
determinethe
the
number
numberof
ofregular
regular
differencing
differencing??

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Autocorrelations (ACs)
Autocorrelations are

## statistical measures that indicate

how a time series is related to itself over time

The

## autocorrelation at lag 1 is the correlation between

the original series zt and the same series moved
forward one period (represented as zt-1)

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Autocorrelations (cont.)
The

## theoretical autocorrelation function

E (z t )(z t k )
k
2z

The

sample autocorrelation
N k

rk

(z
t 1

z )(z t k z )

2
(
z

z
)
t
t 1

k 0,1,2,...k

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Autocorrelations (cont.)
A graph

## of the correlation values is called a

correlogram
correlogram
In practice, to obtain a useful estimate of the
autocorrelation function, at least 50 observations
are needed
The estimated autocorrelations rk would be
calculated up to lag no larger than N/4

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

A correlogram of a nonstationary
time seies

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

After one RD

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

After two RD

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

The

## Box-Jenkins models are based on the idea that

a time series can be usefully regarded as generated
from (driven by) a series of uncorrelated
independent shocks
shocks et
E e t 0 var e t e2
1
k
0

k0
k0

Such

## a sequence et, et-1, et-2, is called a white

noise process
process
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

( B)
White noise

Linear filter

xt

et
A linear filter
filter

## is a model that transform the

white noise process et to the process that generated
the time series xt
x t e t 1e t 1 2 e t 2 ... (B)e t
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

## is the transfer function

function of the filter

2

j 0

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

The

## linear filter can be put in another form

x t 1x t 1 2 x t 2 ... e t

x
j

j1

This

t j

et

## form can be written

( B) x t e t
( B) 1

jB j

z t (B)at
(B)z t at

jTime
1 Series Analysis Lecture
( BNotes
)
MA(4030)Prepared By TMJA
Cooray

1 (B)

## Stationarity and invertibility

conditions for a linear filter
For

a linear process
to be stationary,
stationary

for B 1

If

## the current observation xt

depends on past observations
with weights which decrease as
we go back in time, the series is
called invertible

For

a linear process to be
invertible,
invertible (B) must converge

for B 1

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Model building blocks

Autoregressive

(AR) models
Moving-average (MA) models
Mixed ARMA models
Non stationary models (ARIMA models)
The mean parameter
The trend parameter

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

An

## autoregressive model of order p

x t 1x t 1 2 x t 2 ... p x t p e t

(B) x t e t

(B) 1 1B 2 B 2 ... p B p
The

## autoregressive process can be thought of as

the output from a linear filter with a transfer
function -1(B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
Time Series Analysis Lecture Notes
equation
equation
MA(4030)Prepared By TMJA

The

Cooray

## Moving-average (MA) models

A moving-average

model of order q

x t e t 1e t 1 2 e t 2 ... q e t q
x t (B)e t
2

(B) 1 1B 2 B ... q B

The

## moving-average process can be thought of as

the output from a linear filter with a transfer
function (B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
Time Series Analysis Lecture Notes
equation
equation
MA(4030)Prepared By TMJA

The

Cooray

A moving-average

written as

## process of 1st order can be

1
zt a t
(1 1B)
(1 1B 12 B 2 13 B3 ...) x t e t
x t 1x t 1 12 x t 2 13 x t 3 ... e t

Hence,
Hence

## if the process were really MA(1), we

would obtain a non parsimonious representation in
terms of an autoregressive model
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

(cont.)
In

## order to obtain a parsimonious model,

sometimes it will be necessary to include both AR
and MA terms in the model
(B)
An ARMA(p, q) model
xt
et

(B)

## can be thought of as the

output from a linear filter with a transfer function
(B)/(B),
(B) when the input is white noise at
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

## The Box-Jenkins model building

process
Model identification
Autocorrelations
Partial-autocorrelations

Model

estimation
Model validation
Certain diagnostics are used to check the validity of the

model
Model

forecasting
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Partial-autocorrelations (PACs)
Partial-autocorrelations are

## another set of statistical

measures are used to identify time series models

PAC

## is Similar to AC, except that when calculating

it, the ACs with all the elements within the lag are
partialled out (Box & Jenkins, 1976)

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Partial-autocorrelations (cont.)
PACs

## can be calculated from the values of the ACs

where each PAC is obtained from a different set of
linear equations that describe a pure
autoregressive model of an order that is equal to
the value of the lag of the partial-autocorrelation
computed

PAC

at lag k is denoted by kk

## autoregressive parameter k of the autoregressive model

of order k
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

Model identification
The

## sample ACs and PACs are computed for the

series and compared to theoretical autocorrelation
and partial-autocorrelation functions for candidate
models investigated
Stationarity and
invertibility
conditions

PACs

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Stationarity and invertibility

conditions
For

a linear process
to be stationary,
stationary

For

a linear process to be
invertible,
invertible

for B 1

for B 1

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

model

## For an AR(1) to be stationary:

-1 < 1 < 1
i.e., the roots of the characteristic
equation 1 - 1B = 0 lie outside
the unit circle

## For an AR(1) it can be shown that:

k = 1 k 1 which with 0 = 1 has

the solution
k = 1k
k>0
i.e., for a stationary AR(1) model,
the theoretical autocorrelation
function decays exponentially to
zero, however, the theoretical
partial-autocorrelation function has
a cut off after the 1st lag
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

## Invertibility requirements for a MA(1)

model
For

a MA(1) to be invertible:

-1 < 1 < 1

## i.e., the roots of the characteristic equation 1 - 1B = 0

lie outside the unit circle
1

For

k 1
0

12

k 1
k 1

## i.e., for an invertible MA(1) model, the theoretical

autocorrelation function has a cut off after the 1st lag,
however, the theoretical partial-autocorrelation function
Time Series Analysis Lecture Notes
decays exponentially
to zero
MA(4030)Prepared By TMJA
Cooray

For

## damped exponentials and damped sine waves

The partial-autocorrelation function has a cut off after
the p lag
For

## The autocorrelation function has a cut off after the q lag

The partial-autocorrelation function consists of a

## mixture of damped exponentials and damped sine

waves
Time Series Analysis Lecture Notes
MA(4030)Prepared By TMJA
Cooray

MA parameters

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Model identification

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Model estimation

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray

Model verification

## Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray