3/1/24, 10:22 AM 1 Characteristics of Time Series 1.

4 Stationary Time Series

1 Characteristics of Time Series 1.4 Stationary Time

Series
Aaron Smith
2022-11-12
This code is modified from Time Series Analysis and Its Applications, by Robert H. Shumway, David S. Stoffer https://github.com/nickpoison/tsa4
(https://github.com/nickpoison/tsa4)

The most recent version of the package can be found at https://github.com/nickpoison/astsa/ (https://github.com/nickpoison/astsa/)

You can find demonstrations of astsa capabilities at https://github.com/nickpoison/astsa/blob/master/fun_with_astsa/fun_with_astsa.md

(https://github.com/nickpoison/astsa/blob/master/fun_with_astsa/fun_with_astsa.md)

In addition, the News and ChangeLog files are at https://github.com/nickpoison/astsa/blob/master/NEWS.md

(https://github.com/nickpoison/astsa/blob/master/NEWS.md).

The webpages for the texts and some help on using R for time series analysis can be found at https://nickpoison.github.io/
(https://nickpoison.github.io/).

UCF students can download it for free through the library.

#install.packages(
# pkgs = "remotes"
#)
#remotes::install_github(
# repo = "nickpoison/astsa/astsa_build"
#)

options(
digits = 3,
scipen = 99
)
rm(
list = ls()
)

Stationary time series are series with regularity in the behavior.

Definition 1.6 strictly stationary time series

A strictly stationary time series is one for which the probabilistic behavior of every collection of values is identical to that of the time shifted set.

P (x t ≤ c1 , x t ≤ c2 , … , x t ≤ ck ) = P (x t ≤ c1 , x t ≤ c2 , … , x t ≤ ck ) ∀k = 1, 2, … , k ∀ time points tj , ∀cj ∈ R ∀ time shifts h

1 2 k 1 +h 2 +h k +h

If a time series is strictly stationary, then all of the multivariate distribution functions for subsets of variables must agree with their counterparts in
the shifted set for all values of the shift parameter h.

Example:
k=1
P (x s ≤ c) = P (x t ≤ c) ∀s, t

The probability function is constant with respect to time. Furthermore if the expected value exists, it is constant with respect to time and μs = μt .

Note a random walk is not strictly stationary because the mean changes with time.

k=2
P (x s ≤ c1 , x t ≤ c2 ) = P (x s+h ≤ c1 , x t+h ≤ c2 ) ∀ time points s, t and time shift h

the autocovariance satisfies

γ(s, t) = γ(s + h, t + h) ∀ time points s, t and time shift h

The autocovariance function depends only on the time difference between s and t , and not on the actual times.

Definition 1.7 weakly stationary time series

A weakly stationary time series, x t , is a finite variance process such that

the mean value function, μt , is constant and does not depend on time t , and
the autocovariance function, γ(s, t) , depends on s and t only on the difference |s − t|.

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 1/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series
When we use the term stationary, we mean weakly stationary.

Stationarity requires regularity in the mean and autocorrelation functions so that these quantities may be estimated by averaging.

A strictly stationary time series with finite variance is also weakly stationary. The converse is not correct unless other conditions hold.

Proposition:
If a time series is Gaussian and weakly stationary, then it is also strictly stationary.

Proposition:
When a time series is weakly stationary, then

E(x t ) = μt = μ (constant expected value)

γ(t + h, t) = cov(x t+h , x t ) = cov(x h , x 0 ) = γ(h, 0) (autocovariance depends on shift, not time)

Definition 1.8
The autocovariance function of a stationary time series will be written as

γ(h) = cov(x t+h , x t ) = E[(x t+h − μ)(x t − μ)]

Definition
The autocorrelation function (ACF) of a stationary time series will be written as

γ(t + h, t) γ(h)
ρ(h) = =
− −−−−−−−−−−−−−−−
√ γ(t + h, t + h)γ(t, t) γ(0)

The Cauchy-Schwarz inequality shows that −1 ≤ ρ(h) ≤ 1 ∀h

Example 1.19 Stationarity of White Noise

The mean and autocovariance functions of the white noise series are easily evaluated as μwt = 0 and

0 if h ≠ 0
γ w (h) = cov(wt+h , wt ) = {
2
σw if h = 0

0 if h ≠ 0
ρ w (h) = {
1 if h = 0

White noise satisfies the conditions of weakly stationary. If the white noise variates are also normally distributed or Gaussian, the series is also
strictly stationary, as can be seen by the fact that the noise would also be iid.

set.seed(
seed = 823
)
v_rnorm <- rnorm(
n = 10000
)
astsa::acf1(
series = v_rnorm
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 2/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

## [1] -0.01 0.00 -0.01 -0.01 0.00 -0.02 -0.01 -0.02 0.00 0.01 0.00 0.00
## [13] 0.01 0.00 0.01 -0.02 -0.01 0.00 0.01 0.00 0.01 -0.01 -0.01 0.01
## [25] 0.01 0.01 0.00 0.02 0.01 0.00 0.01 0.00 -0.02 -0.01 -0.02 0.00
## [37] 0.01 0.00 0.01 0.00 0.00 -0.01 0.01 0.00 0.00 0.00 -0.01 -0.01
## [49] 0.00 0.01 0.00 0.00 0.01 0.00 0.00 0.01 -0.01 -0.01 0.00 0.00
## [61] -0.01 0.01 -0.01 -0.01 -0.01 0.01 0.00 0.02 0.00 0.01 0.01 0.00
## [73] -0.01 0.00 0.02 -0.01 0.00 0.01 0.01 0.01 -0.02 0.00 0.01 -0.01
## [85] -0.01 0.02 -0.01 0.01 0.00 0.00 -0.01 0.00 0.01 -0.01 0.00 -0.01
## [97] -0.01 -0.01 0.02 0.01 0.00 0.00 -0.01 0.00 0.01 0.01 0.00 0.00
## [109] 0.00 -0.01

astsa::acf2(
series = v_rnorm
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 3/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]
## ACF -0.01 0 -0.01 -0.01 0 -0.02 -0.01 -0.02 0 0.01 0 0 0.01
## PACF -0.01 0 -0.01 -0.01 0 -0.02 -0.01 -0.02 0 0.01 0 0 0.01
## [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## ACF 0 0.01 -0.02 -0.01 0 0.01 0 0.01 -0.01 -0.01 0.01 0.01
## PACF 0 0.01 -0.02 -0.01 0 0.01 0 0.01 -0.01 -0.01 0.01 0.01
## [,26] [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,37]
## ACF 0.01 0 0.02 0.01 0 0.01 0 -0.02 -0.01 -0.02 0 0.01
## PACF 0.01 0 0.02 0.01 0 0.02 0 -0.02 -0.01 -0.02 0 0.01
## [,38] [,39] [,40] [,41] [,42] [,43] [,44] [,45] [,46] [,47] [,48] [,49]
## ACF 0 0.01 0 0 -0.01 0.01 0 0 0 -0.01 -0.01 0
## PACF 0 0.01 0 0 -0.01 0.01 0 0 0 -0.01 -0.01 0
## [,50] [,51] [,52] [,53] [,54] [,55] [,56] [,57] [,58] [,59] [,60] [,61]
## ACF 0.01 0 0 0.01 0 0 0.01 -0.01 -0.01 0.00 0 -0.01
## PACF 0.01 0 0 0.01 0 0 0.01 -0.01 -0.01 0.01 0 -0.01
## [,62] [,63] [,64] [,65] [,66] [,67] [,68] [,69] [,70] [,71] [,72] [,73]
## ACF 0.01 -0.01 -0.01 -0.01 0.01 0 0.02 0.00 0.01 0.01 0 -0.01
## PACF 0.01 -0.01 -0.01 -0.01 0.01 0 0.02 0.01 0.01 0.01 0 0.00
## [,74] [,75] [,76] [,77] [,78] [,79] [,80] [,81] [,82] [,83] [,84] [,85]
## ACF 0 0.02 -0.01 0 0.01 0.01 0.01 -0.02 0 0.01 -0.01 -0.01
## PACF 0 0.02 -0.01 0 0.01 0.01 0.01 -0.02 0 0.01 -0.01 -0.01
## [,86] [,87] [,88] [,89] [,90] [,91] [,92] [,93] [,94] [,95] [,96] [,97]
## ACF 0.02 -0.01 0.01 0.00 0 -0.01 0 0.01 -0.01 0 -0.01 -0.01
## PACF 0.02 -0.01 0.01 0.01 0 -0.01 0 0.01 -0.01 0 -0.01 -0.01
## [,98] [,99] [,100] [,101] [,102] [,103] [,104] [,105] [,106] [,107] [,108]
## ACF -0.01 0.02 0.01 0 0 -0.01 0 0.01 0.01 0 0.00
## PACF -0.02 0.01 0.01 0 0 -0.01 0 0.01 0.01 0 0.01
## [,109] [,110]
## ACF 0 -0.01
## PACF 0 -0.01

Example 1.20 Stationarity of a Moving Average

The three-point moving average process from Gaussian white noise is stationary.

2
wt ∼ iidN (0, σ w )

wt−1 + wt + wt−1
vt =
3

μvt = 0

⎧
⎪
⎪
⎪ 1
⎪
⎪ 2
⎪
⎪ σw when h = −2
⎪
⎪
⎪ 9
⎪
⎪
⎪ 2
⎪
⎪
2
σw when h = −1
⎪
⎪
⎪ 9
⎪
3
2
γ v (h) =⎨ σw when h = 0
⎪ 9
⎪
⎪
⎪ 2
⎪
⎪ 2
⎪ σw when h = 1
⎪
⎪
⎪ 9
⎪
⎪
⎪ 1
⎪ 2
σw when h = 2
⎪
⎪
⎪
⎪ 9
⎩
⎪
0 when |h| > 2

⎧
⎪
⎪
⎪ 1
⎪
⎪
⎪
⎪ when h = −2
⎪
⎪
⎪ 9
⎪
⎪
⎪ 2
⎪
⎪ when h = −1
⎪
⎪
⎪ 9
⎪
3
ρ v (h) =⎨ when h = 0
⎪ 9
⎪
⎪
⎪ 2
⎪
⎪
⎪ when h = 1
⎪
⎪
⎪ 9
⎪
⎪
⎪ 1
⎪ when h = 2
⎪
⎪
⎪
⎪ 9
⎩
⎪
0 when |h| > 2

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 4/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

v_rnorm = rnorm(
n = 10000,
mean = 0,
sd = 1
)
filter_rnorm = filter(
x = v_rnorm,
sides = 2,
filter = rep(
x = 1/3,
times = 3
)
)
astsa::acf1(
series = filter_rnorm
)

## [1] 0.67 0.34 0.01 0.02 0.03 0.03 0.02 0.01 0.01 0.01 0.02 0.02
## [13] 0.01 0.00 -0.01 0.00 0.01 0.01 0.00 -0.01 0.00 0.00 0.01 0.02
## [25] 0.01 0.01 0.00 0.00 0.00 0.01 0.01 0.00 0.00 0.00 0.00 -0.01
## [37] -0.01 -0.01 0.00 0.01 0.02 0.03 0.02 0.01 0.00 0.00 0.00 0.01
## [49] 0.00 0.01 0.01 0.02 0.02 0.01 0.00 -0.01 0.00 0.00 0.01 0.00
## [61] 0.00 -0.01 -0.02 -0.01 0.00 0.01 0.01 0.01 0.00 0.00 0.00 0.01
## [73] 0.01 0.01 0.01 0.00 -0.01 0.00 0.01 0.02 0.01 0.00 -0.01 0.00
## [85] 0.00 0.01 0.01 0.02 0.01 0.01 0.01 0.00 0.00 -0.01 0.00 0.02
## [97] 0.03 0.03 0.02 0.01 0.00 0.00 0.00 -0.02 -0.02 -0.04 -0.03 -0.03
## [109] -0.01 -0.01

astsa::acf2(
series = filter_rnorm
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 5/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]
## ACF 0.67 0.34 0.01 0.02 0.03 0.03 0.02 0.01 0.01 0.01 0.02 0.02 0.01
## PACF 0.67 -0.20 -0.24 0.35 -0.12 -0.14 0.22 -0.10 -0.08 0.16 -0.06 -0.08 0.13
## [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## ACF 0.00 -0.01 0.00 0.01 0.01 0.0 -0.01 0.00 0.00 0.01 0.02 0.01
## PACF -0.06 -0.06 0.12 -0.05 -0.06 0.1 -0.05 -0.03 0.08 -0.03 -0.03 0.06
## [,26] [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,37]
## ACF 0.01 0.00 0.00 0.00 0.01 0.01 0.00 0.00 0.00 0.00 -0.01 -0.01
## PACF -0.03 -0.04 0.08 -0.03 -0.04 0.06 -0.03 -0.03 0.05 -0.04 -0.02 0.05
## [,38] [,39] [,40] [,41] [,42] [,43] [,44] [,45] [,46] [,47] [,48] [,49]
## ACF -0.01 0.00 0.01 0.02 0.03 0.02 0.01 0.00 0.00 0.00 0.01 0.00
## PACF -0.02 -0.01 0.06 -0.02 -0.01 0.03 -0.02 -0.02 0.05 -0.03 -0.01 0.04
## [,50] [,51] [,52] [,53] [,54] [,55] [,56] [,57] [,58] [,59] [,60] [,61]
## ACF 0.01 0.01 0.02 0.02 0.01 0.00 -0.01 0.00 0.00 0.01 0.00 0.00
## PACF -0.01 -0.01 0.05 -0.03 -0.03 0.04 -0.01 -0.01 0.03 -0.01 -0.02 0.01
## [,62] [,63] [,64] [,65] [,66] [,67] [,68] [,69] [,70] [,71] [,72] [,73]
## ACF -0.01 -0.02 -0.01 0.00 0.01 0.01 0.01 0.00 0.00 0 0.01 0.01
## PACF -0.02 -0.01 0.04 -0.01 -0.02 0.03 -0.01 -0.03 0.04 0 -0.02 0.03
## [,74] [,75] [,76] [,77] [,78] [,79] [,80] [,81] [,82] [,83] [,84] [,85]
## ACF 0.01 0.01 0.00 -0.01 0.00 0.01 0.02 0.01 0.00 -0.01 0.00 0.00
## PACF 0.00 -0.02 0.02 -0.01 0.01 0.01 0.00 -0.02 0.03 -0.03 0.01 0.01
## [,86] [,87] [,88] [,89] [,90] [,91] [,92] [,93] [,94] [,95] [,96] [,97]
## ACF 0.01 0.01 0.02 0.01 0.01 0.01 0.00 0 -0.01 0 0.02 0.03
## PACF 0.00 0.00 0.02 -0.02 0.00 0.02 -0.03 0 0.02 0 0.00 0.03
## [,98] [,99] [,100] [,101] [,102] [,103] [,104] [,105] [,106] [,107] [,108]
## ACF 0.03 0.02 0.01 0.00 0.00 0.00 -0.02 -0.02 -0.04 -0.03 -0.03
## PACF -0.01 -0.01 0.02 -0.01 -0.02 0.03 -0.04 -0.01 0.01 -0.01 -0.02
## [,109] [,110]
## ACF -0.01 -0.01
## PACF 0.02 -0.01

Example 1.21 A Random Walk is Not Stationary

A random walk is not stationary because its autocovariance function depends on time.

2
γ(s, t) = min(s, t)σ w

A random walk with drift is not stationary because the mean is a function on t

μxt = δt

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 6/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

set.seed(
seed = 823
)
v_rnorm = rnorm(
n = 100000
)
cumsum_rnorm = cumsum(
x = v_rnorm
) # two commands in one line
wd = v_rnorm + 0.2;
v_randomwalkdrift = cumsum(
x = wd
)
astsa::acf1(
series = cumsum_rnorm
)

## [1] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [16] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [31] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [46] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [61] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [76] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [91] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [106] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [121] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [136] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [151] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [166] 1.00 1.00 1.00 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [181] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [196] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [211] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [226] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [241] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [256] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [271] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [286] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [301] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [316] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99

astsa::acf1(
series = v_randomwalkdrift
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 7/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

## [1] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [16] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [31] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [46] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [61] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [76] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [91] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [106] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [121] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [136] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
## [151] 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.99
## [166] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [181] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [196] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [211] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [226] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [241] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [256] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [271] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [286] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [301] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
## [316] 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99

Example 1.22 Trend Stationarity

For example, if

x t = α + βt + yt

yt is stationary

then the mean function is

μx,t = E(x t ) = α + βt + μy

which is not independent of time, the process is not stationary.

The autocovariance function, however, is independent of time, because

γ x (h) =cov(x t+h , x t )

=E[(α + β(t + h) + yt+h − α − β(t + h) − μt+h )(α + βt + yt − α − βt − μt )]

=E[(yt+h − μt+h )(yt − μt )]

=γ y (h)

The model has trend stationarity.

Example of trend stationarity

An example of such a process is the price of chicken series.

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 8/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

data(
list = "chicken",
package = "astsa"
)
astsa::trend(
series = chicken,
lwd = 2
)

lm_chicken <- lm(

formula = chicken~time(chicken)
)
summary(
object = lm_chicken
)

##
## Call:
## lm(formula = chicken ~ time(chicken))
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.741 -3.473 0.825 2.774 11.580
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -7131.0225 162.4134 -43.9 <0.0000000000000002 ***
## time(chicken) 3.5921 0.0808 44.4 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.7 on 178 degrees of freedom
## Multiple R-squared: 0.917, Adjusted R-squared: 0.917
## F-statistic: 1.97e+03 on 1 and 178 DF, p-value: <0.0000000000000002

astsa::tsplot(
x = chicken,
ylab = "cents per pound",
col = 4,
lwd = 2
)
abline(
reg = lm_chicken
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 9/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

Properties of autocovariance
The autocovariance function of a stationary process has several special properties.

non-negative definite
bounded by the variance
symmetric about the origin

autocovariance is non-negative definite

γ(h) is non-negative definite ensuring that variances of linear combinations of the variates x t will never be negative.

n n

0 ≤ var(a1 x 1 + … + an x n ) = ∑ ∑ aj ak γ(j − k) ∀n ∈ Z

j=1 k=1

autocovariance is bounded by the variance

2
γ(0) = E[(x t − μx ) ]

|γ(h)| ≤ γ(0) (use Cauchy-Schwarz)

autocovariance is symmetric about the origin

γ((t + h) − t) = cov(x t+h , x t ) = cov(x t , x t+h ) = γ(t − (t + h))

γ(h) = γ(−h)

# Definition 1.10 jointly stationary

Two time series, say, x t and yt , are said to be jointly stationary if they are each stationary, and the cross-covariance function is a function only on
the lag h.

cov(x t+h , yt ) = E[(x t+h − μx )(yt − μy )]

When jointly stationary applies, we denote the covariance as γxy (h) .

Definition 1.11
The cross-correlation function (CCF) of jointly stationary time series x t and yt is defined as

γ xy (h)
ρ xy (h) = and
−−−−−−−−
√ γ x (0)γ y (0)

− 1 ≤ ρ xy (h) ≤ 1

In general, cov(x 2 , y1 ) ≠ cov(x 1 , y2 ) and ρxy (h) ≠ ρ xy (−h) , but when we switch the subscript order

ρ xy (h) = ρ yx (−h)

Example 1.23 Joint Stationarity

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 10/15
3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series
Consider the two series, x t and yt , formed from the sum and difference of two successive values of a white noise process,

x t = wt + wt−1

yt = wt − wt−1

wt are independent

E(wt ) = 0

2
var(wt ) = σ w

then the following holds:

2
γ x (0) =γ y (0) = 2σ w

2
γ x (1) =γ x (−1) = σ x

2
γ y (1) =γ y (−1) = −σ x

2
γ xy (1) =cov(x t+1 , yt ) = cov(wt+1 + wt , wt = wt−1 ) = σ w (only one term in the expansion is non-zero)

⎧
⎪
⎪
⎪
⎪
⎪ −1/2 when h = −1

ρ v (h) =⎨ 0 when h = 0
⎪
⎪
⎪ 1/2 when h = 1
⎪
⎩
⎪
0 when |h| > 1

set.seed(
seed = 823
)
n <- 10000
v_rnorm <- rnorm(
n = n
)
x <- v_rnorm[-n] + v_rnorm[-1]
y <- v_rnorm[-n] - v_rnorm[-1]
var(
x = data.frame(
x = x,
y = y
)
)

## x y
## x 1.9784034 0.0000889
## y 0.0000889 2.0187481

# the acf1() plots show correlation, the variance of 2 give correlation = 1/2
astsa::acf1(
series = x
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 11/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

## [1] 0.49 -0.02 -0.02 -0.01 -0.01 -0.03 -0.03 -0.02 0.00 0.01 0.01 0.01
## [13] 0.01 0.00 0.00 -0.02 -0.02 0.00 0.01 0.00 0.00 -0.01 -0.01 0.01
## [25] 0.02 0.01 0.01 0.02 0.02 0.01 0.01 0.00 -0.02 -0.03 -0.03 -0.01
## [37] 0.00 0.00 0.01 0.00 -0.01 -0.01 0.00 0.01 0.00 0.00 -0.01 -0.02
## [49] 0.00 0.01 0.00 0.00 0.01 0.00 0.00 0.00 -0.01 -0.02 0.00 0.00
## [61] 0.00 0.00 -0.01 -0.02 -0.01 0.00 0.01 0.02 0.02 0.02 0.02 0.00
## [73] -0.01 0.00 0.01 0.00 0.00 0.01 0.02 0.01 -0.01 0.00 0.01 -0.01
## [85] -0.01 0.01 0.00 0.00 0.01 0.00 -0.01 -0.01 0.00 -0.01 -0.01 -0.01
## [97] -0.02 -0.01 0.02 0.02 0.00 -0.01 -0.02 0.00 0.01 0.01 0.01 0.01
## [109] 0.00 -0.01

astsa::acf1(
series = y
)

## [1] -0.50 0.01 -0.01 0.00 0.01 -0.01 0.00 -0.01 0.01 0.01 0.00 0.00
## [13] 0.01 -0.01 0.03 -0.02 0.00 0.00 0.01 -0.01 0.02 -0.01 -0.01 0.02
## [25] 0.00 0.00 -0.01 0.01 0.00 -0.01 0.02 0.00 -0.01 0.01 -0.01 0.00
## [37] 0.01 -0.01 0.01 -0.01 0.01 -0.02 0.02 -0.01 0.00 0.00 0.00 -0.01
## [49] 0.00 0.01 -0.01 -0.01 0.01 -0.01 -0.01 0.02 -0.01 -0.01 0.01 0.00
## [61] -0.01 0.01 -0.01 0.00 -0.01 0.02 -0.02 0.02 -0.01 0.01 0.00 0.00
## [73] -0.01 -0.01 0.02 -0.02 0.00 0.00 0.00 0.01 -0.02 0.01 0.02 -0.01
## [85] -0.01 0.02 -0.02 0.01 0.00 0.01 -0.02 0.00 0.01 -0.01 0.01 0.00
## [97] 0.00 -0.02 0.02 0.00 -0.01 0.00 -0.01 0.00 0.01 0.00 0.00 0.00
## [109] 0.00 0.00

astsa::ccf2(
x = x,
y = y,
type = "covariance"
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 12/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

Example 1.24 Prediction Using Cross-Correlation

Consider the problem of determining possible leading or lagging relations between two series x t and yt . If the model

yt = Ax t−l + wt

holds, then

xt leads yt for l > 0

xt lags yt for l < 0

The analysis of leading and lagging relations might be important in predicting the value of yt from x t .

Assuming that the noise wt is uncorrelated with x t , the cross-covariance function can be computed as

γ yx (h) =cov(yt+h , x t )

=cov(Ax t+h−l + wt+h , x t )

=cov(Ax t+h−l , x t )

=Aγ x (h − l)

By (Cauchy–Schwarz) the largest absolute value of autocovariance/autocorrelation is with lag of zero (h = l ). The cross-covariance function will
look like the autocovariance of the input series x t , and it will have

a peak on the positive side if x t leads yt and

a peak on the negative side if x t lags yt .

set.seed(
seed = 823
)
x = rnorm(
n = 100
)
# l = 5
y = lag(x = x,k = -5) + rnorm(n = 100)
astsa::ccf2(
x = y,
y = x,
ylab = 'Cross correlation',
type = "correlation"
)
text(
x = 9,
y = 1.1,
labels = 'x leads'
)
text(
x = -8,
y = 1.1,
labels = 'y leads'
)

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 13/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series

astsa::ccf2(
x = y,
y = x,
ylab = 'Cross covariance',
type = 'covariance'
)
text(
x = 9,
y = 1.1,
labels = 'x leads'
)
text(
x = -8,
y = 1.1,
labels = 'y leads'
)

Weak stationarity forms the basis for much of time series analysis. The fundamental properties of the mean and autocovariance functions are
satisfied by many models that appear to generate plausible sample realizations.

The three points moving average example is an example of weakly stationary linear process.

Definition 1.12 linear process

A linear process, x t , is defined to be a linear combination of white noise variates wt , and is given by

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 14/15

3/1/24, 10:22 AM 1 Characteristics of Time Series 1.4 Stationary Time Series
∞

xt = μ + ∑ ψj wt−j

j=−∞

∑ |ψj | < ∞ (absolutely convergent)

j=−∞

Autocovariance of linear process

The absolute convergence requirement makes the autocovariance defined. In the multiplication only the terms with the same white noise value
give non-zero covariance.

2
γ x (h) = σ w ∑ ψj+h ψj

j=−∞

Recall that γx (h) = γ x (−h) .

∞
If ∑j=−∞ ψj 2
< ∞, then the variance of the linear process will be finite.

1
In the moving average example ψ−1 = ψ0 = ψ1 = , ψj = 0 j ≠ −1, 0, 1 .
3

When a linear process is dependent on future values, it is useless for prediction. A causal linear process has coefficient zero for future values.

An important case in which a weakly stationary series is also strictly stationary is the normal or Gaussian series.

Definition 1.13 Gaussian process

A process, x t , is said to be a Gaussian process if the n-dimensional vectors

T
x = (x t , x t , … , x t ) ,
1 2 n

for every collection of distinct time points t1 , t2 , … , tn , and every positive integer n, have a multivariate normal distribution.

Defining the n × 1 mean vector

T
E(x) = μ = (μt , μt , … , μt )
1 2 n

and the n × n covariance matrix as

var(x) = Γ = {γ(ti , tj )|i, j = 1, 2, … , n},

which is assumed to be positive definite

1 1 −1 T −1 n
f (x) = exp ( (x − μ) Γ (x − μ)) ∀x ∈ R
−n/2
− −
−
(2π) √ |Γ| 2

Important aspects of linear and Gaussian processes:

If a Gaussian time series, x t , is weakly stationary, then

μt is constant and
γ(ti , tj ) = γ(|ti − tj |)

μ and Γ are independent of time

All the finite distributions of the series x t depend only on lag and not on the actual times, and hence the series must be strictly stationary.
∞
Wold Decomposition gives us that a stationary non-deterministic time series with ∑j=0 ψ2j < ∞ is a causal linear process.

A linear process need not be Gaussian, but if a time series is Gaussian, then it is a causal linear process with wt 2
∼ iidnormal(0, σ w ) . Hence
stationary Gaussian processes form a basis for modeling many time series.

It is not enough for the marginal distributions to be Gaussian for the process to be Gaussian.

Counter-example:
Let X and Z be independent normal and let

Z if XZ > 0
Y = {
−Z if XZ ≤ 0

X and Y are normal, but (X, Y ) is not bivariate normal.

file:///G:/My Drive/Time Series/NEU/Tài liệu/1-Characteristics-of-Time-Series-1.4-Stationary-Time-Series.html 15/15