BSC HONOURS IN STATISTICS

HSTS 203 : Time Series Analysis

University Of Zimbabwe

L Dhliwayo
Department of Statistics
University of Zimbabwe
Contents
1 The Model Building Strategy 1
1.1 What this Unit is all About . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The Model Building Strategy . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Time Series and Stochastic Processes . . . . . . . . . . . . . . . . . . . . 2

i
 Unit 1
The Model Building Strategy

1.1 What this Unit is all About

Unit Objectives

At the end of this unit students are expected to be able to

1.2 The Model Building Strategy

Finding an appropiate model is not an easy task. We will develop a model building
strategy which was developed by Box and Jenkins in 1976.
The Box-Jenkins methodology consist of a four step iterative procedure.

Step 1. Tentative Identification: Historical data is used to tentatively identify an

appropriate Box-Jenkins model.

Step 2. Estimation: historical data is used to estimate the parameters of the tentatively
identified model.

Step 3. Diagnostic Checking: various diagnostics are used to check the adequacy of
the tentatively identified model, and ifneed be, to suggest an improved model,
which is then regarded as a new tentatively identified model.

Step 4. Forecasting: once a final model is obtained, it is used to forecast future time
series values.

The following diagram depicts the steps in the Box-Jenkins iterative approach to models
building.
*** diagram
In the process of model selection, we shall try to adhere to the principle of parsimony:

Definition 1.1 (The Principle of Parsimony) The model used should require the
smallest possible number of parameters that will adequately represent the data.

1
2 The Model Building Strategy

1.3 Time Series and Stochastic Processes

Let us consider the probabilistic structure that underlies time series observations.
Let us denote the tth observation in a time series by Zt . This means the observation
observed at time t. Lets assume that the observations are equally spaced in time.
Inorder to model the uncertainty in our observations, we assume that for each time
point, t, Zt is a random variable. Thus the behaviour of Zt will be determined by a
probability distribution. However, the most important feature of the time series models
is that we assume that the observations made at different time points, t, are statistically
independent. It is precisely this independence that we wish to investigate.
So for two points ti and tj , the joint behaviour of Zi and Zj will be determined from
joint bivariate distribution. In general the probabilistic nature of the collection of
random observations Z1 , Z2 , . . . , Zn reflected in their multivariate joint distribution.
We will consider the set of observations Z1 , Z2 , . . . , Zn as a part of much larger time
series going indefinitely into the future and possibly into the past.
The sequence of random variables . . . Z−1 , Z0 , Z1 , Z2 , . . . , Zn is called a stochastic pro-
cess.
The complete probabilistic structure of such a process is determined by the set of
distribution of all infinite collection of Zi0 s
Let us consider the mean, variance and covariance of these joint distributions.
Let us consider the stochastic process {Zt : t = 0, ±1, ±2, . . .}

Definition 1.2 (Mean Function) Let {Zt } be a stochatic process, then the mean
function is defined by

µt = E(Zt ) for t = 0, ±1, ± 2, . . . (1.1)

Definition 1.3 (Autocovariance Function) Let {Zt } be a stochatic process with

mean µt and k be the lag between Zt and Zt−k , then the autocovariance function between
Zt and Zt−k is defined by

γk = E(Zt − µt )(Zt−k − µt−k )

(1.2)
= E(Zt Zt−k ) − µt µt−k

Definition 1.4 (Autocorrelation Function) Let {Zt } be a stochatic process with

mean µt and variance γ0 and autocovariance γk , then the autocorrelation function (acf )
denoted ρk is defined as:
γk
ρk = (1.3)
γ0

Note:

1. The autocovariance function and autocorrelation function are both measures of

linear dependence between variables.

2. The autocorraletion function (acf) is the preferred measure of linear dependence

because it is a unitless measure. This makes it easier to interpret.
The Model Building Strategy 3

The acf has two interesting properties:

1. The acf is an even function of the lag in that ρk = ρ−k

2. |ρk | ≤ 1

The acf is used to determine the moving average part for an ARMA model [to be
discussed latter].

Definition 1.5 (Partial Autocorrelation Function (PACF)) This is the correla-

tion between Zt and Zt−k after removing the effect of the intervening variables Zt−1 , Zt−2 , . . . , Zt−k+1
for any stationary series at lag k and is denoted by φkk :

φkk = corr(Zt , Zt−k |Zt−1 , Zt−2 , . . . , Zt−k+1 ) (1.4)

The PACF is used to determine the autoregressive part for an ARMA model [to be
discussed latter].

Stationarity

Whenever we make statistical inference, it is important that the laws governing the
process do not change, that is, the process is in statistical equilibrium. This brings us
to the definition of stationarity.

Definition 1.6 (Strictly Stationary) A stochastic process is said to be strictly sta-

tionary if the joint distribution of Zt1 , Zt2 , . . . , Ztn is the same as the joint distribution
of Zt1 −k , Zt2 −k , . . . , Ztn −k for all choices of t1 , t2 , . . . , tn and choices of time lag k.

This implies that

E(Zt ) = E(Zt−k ) = µt for all t and k

(1.5)
and V ar(Zt−k ) = V ar(Zs−k )

γt,s = Cov(Zt , Zs )
= Cov(Zt−s , Z0 ) = Cov(Z0 , Zs−t )
(1.6)
= Cov(Z0 , Z|s−t| )
= γ0,|s−t|

Thus, the covariance depends on time only through the time difference k = |s − t| and
not on the actual times t and s. This means for a stationary process we can rewrite
above as

γk = Cov(Zt , Zt−k )
(1.7)
ρk = Corr(Zt , Zt−k )
4 The Model Building Strategy

Properties of γ

1. γ0 = V ar(Zt ), ρ0 = 1

2. γk = γ−k , ρk = ρ−k

3. |γk | ≤ γ0 , |ρk | ≤ 1

This means that if strictly stationary process has a finite variance, then the covariance
function must depend only on the time lag.

Definition 1.7 (Second Order Stationary) A time series {Zt } having finite sec-
ond moment is said to be second order stationary or weakly stationary if

1. E(Zt ) = µ

2. Cov(Zt , Zt−k ) = γk for all t

Note: From the above definition, we see that both the mean and the autocovariance
function do not depend on time.

Definition 1.8 (Ergodic) A time series {Zt } is ergodic in some parameter θ if an ap-
propriate parameter t̂heta constructed from a single realisation converges almost surely
to theta, that is to say, ensample averaging is equivalent to sample averaging in the
above sense.

Note: In this course the term stationary (without word strictly) means weakly station-
ary.