Unit 1 – Stationary processes

MSc. Marı́a Belen Segovia

Escuela Superior Politécnica del Litoral

2022
Objectives

I Introduce different stationary processes.

Moving Average Process

Let {wt } ∼ (0, σ 2 ) for t ∈ Z. Then, the time series {Xt } is said to
be a moving average process of order q (MA(q)) if

Xt = wt + θ1 wt−1 + θ2 wt−2 + ... + θq wt−q

where θ1 , ..., θq are real constant values.

The mean of Xt is given by:

E [Xt ] = E[wt + θ1 wt−1 + .. + θq wt−q ]

= E[wt ] + θ1 E[wt−1 ] + ... + θq E[wt−q ] (1)
=0
Moving Average Process
Let θ0 = 1, the autocovariance function is given by:

γ(k) = cov(Xt , Xt+k ) = E[Xt Xt+k ] − 02

= E[(θ0 wt + . . . + θq wt−q )(θ0 wt+k + . . . + θq wt+k−q )]
q X
q (1)
X
= θr θs E[wt−r wt+k−s ]
r=0 s=0

Where
(
σ2, if t − r = t + k − s
E[wt−r wt+k−s ] =
0, Otherwise

As r, s ≤ q then t − r 6= t + k − s for any r, s if |k| > q, thus,

(
0, if |k| > q
γ(k) = 2
Pq−|k|
σ r=0 θr θr+|k| , if |k| ≤ q
Moving Average Process

I The mean is constant and γ(k) does not depend of t, so the

MA(q) process is weakly stationary.
I The variance of a MA(q) process can be obtained using
var(Xt ) = γ(0) = σ 2 qr=0 θr2
P

I The autocorrelation function for a MA(q) process is given by


0,
 if |k| > q
Pq−|k|
ρ(k) = θr θr+|k|
 r=0
 Pq 2
, if |k| ≤ q
r=0 θr
An important property of a MA(q) process is that the
autocorrelation function takes a value of zero when |k| > q.
Moving Average Process

We can simulate a MA(1) process Xt = wt − 0.8wt−1 . The

following figure to the left shows the 500 observations and at the
right is ACF autocorrelation function). The significative ρ(k)
happen when k = 0 y k = 1.
Autoregressive processes

Let {wt } ∼ (0, σ 2 ) for t ∈ Z

The time series {Xt } is said to be an autoregressive process of p
order (AR(p)) if

Xt = α1 Xt−1 + α2 Xt−2 + .. + αp Xt−p + wt

where α1 , α2 , .., αp are constants.

Autoregressive processes
AR(1)

The AR(1) process or also called autoregressive process of first

order is given by

Xt = α1 Xt−1 + wt

Where wt is a purely random process with wt ∼ (0, σ 2 ).

The equation represents a regression of prediction of the current

value xt as a function of the past value xt−1 of the series. Thus,
the term autoregression is suggested.

This process can be rewritten like an infinite sum of a purely

random process, through consecutive substitutions.
Autoregressive processes
AR(1)

Xt = α1 Xt−1 + wt
= α1 (α1 Xt−2 + wt−1 ) + wt
= α12 Xt−2 + α1 wt−1 + wt
= α12 (α1 Xt−3 + wt−2 ) + α1 wt−1 + wt
= α13 Xt−3 + α12 wt−2 + α1 wt−1 + wt
..
.
∞
α1j wt−j
X
Xt =
j=0
Autoregressive processes
AR(1)

The mean of an AR(1) process is given by:

X∞
E[Xt ] = E[ αj wt−j ]
j=0
∞
X
j
= α E[wt−j ]
j=0

=0
Autoregressive processes
AR(1)

The autocovariance function is given by:

γ(k) = cov[Xt , Xt+k ] = E[Xt Xt+k ] − E[Xt ]E[Xt+k ]

= E[Xt Xt+k ]
 
∞ ∞
α1j wt+k−j 
X X
= E α1i wt−i
i=0 j=0
∞ X
∞
α1i α1j E[wt−i wt+k−j ]
X
=
i=0 j=0

Where:
(
σ2, if j = i + k
E[wt−i wt+k−j ] =
0, Otherwise
Autoregressive processes
AR(1)

Thus, the sum is simplified to:

∞
X
γ(k) = σ 2 α1i α1i+k
i=0
∞
X
= σ 2 α1k α12i
i=0

Based on this expression, we can see that the covariance is infinite

when |α1 | ≥ 1. On the other hand, when |α1 | < 1, the covariance
is given by:

σ 2 α1k
γ(k) =
1 − α12
Autoregressive processes
AR(1)

Remember that:
When |α1 | < 1, then α12 < 1,and for what we know of the
geometric series, then:
∞
X 1
α12i =
i=0
1 − α12
The variance and the autocorrelation function are given by:
σ2
V ar(Xt ) = γ(0) = 1−α21

To get an even function defined for all integer k we can use the
modulus operator to write:
γ(k) |k|
ρ(k) = γ(0) = α1
Autoregressive processes
The general AR(p) process

Using the backshift operator, the equation for a general AR(p)

process can be written like:

Xt − α1 Xt−1 − α2 Xt−2 − . . . − αp Xt−p = wt

(1 − α1 B − α2 B 2 − . . . − αp B p )Xt = wt
φ(B)Xt = wt

Definition The process {Xt } AR(p) general is causal if there is a

I ψ(B) = 1 + ψ(B) + ψ2 (B 2 ) + ...
such that ∞
P
i=0 |ψ| < ∞ y Xt = ψ(B)wt

The AR(p) process, φ(B)Xt = wt is causal if and only if all the

roots of φ(B) falls out of the unitary circle.
Autoregressive processes
ACF Comparison

500 observations from the AR(1)process , Xt = 0.9Xt−1 + wt with

its autocorrelation function.
Autoregressive processes
ACF Comparison

500 observations from the MA(1) process Xt = wt − 0.8wt−1 with

its corresponding autocorrelation function.
References

I Shumway, R. H., Stoffer D.S. (2016). Time Series Analysis

and Its Applications With R Examples.(4dn). Springer.

I Chatfield, C.,Xing H.(2019). The analysis of time series: an

introduction with R.(7edn) Boca Raton: Taylor Francis,
2019. ISBN 9781138066137
I Ioannis Papastathopoulos Time series course UoE 2020.