STAT 411 - Time Series

Dr Mabikwa

BIUST

February 20, 2023

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 1 / 21

 CHAPTER 3: Trends

1 Introduction Regression Estimation of

Difference between stochastic
and deterministic trends
2 Deterministic trend
3 Estimation of constant mean
White Noise Error
Moving Average Error
Decaying ACF
Random Walk Error
4 Regression Methods
Regression Estimation of the
Linear Trend
cyclical or seasonal trend
5 Efficiency of Regression
Estimates
Seasonal trend
Linear trend
6 Interpreting Regression Outputs
7 Evaluating the Model
Residual Analysis
Assessing Normality
Assessing Independence

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 2 / 21

 Introduction

Trends in time series can be classified as stochastic or deterministic.

Stochastic vs Deterministic trends

Stochastic trend: shows inexplicable changes in direction, and transient

trends are attributed to high serial correlation with random error. Such
trends are common in financial series and can be simulated in R using
models such as the random walk or autoregressive process.

In contrast, when we have some plausible physical explanation for a trend

we will usually wish to model it in some deterministic manner. For
example, a deterministic increasing trend in the data may be related to an
increasing population, or a regular cycle may be related to a known
seasonal frequency. Deterministic trends and seasonal variation can be
modelled using regression.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 3 / 21

 Difference between stochastic and deterministic trends

We extrapolate deterministic models to make forecasts. Short term

extrapolation are justified by claiming underlying trends will usually change
slowly in comparison with the forecast lead time. Thus, this should be
based on fitted line of recent data (only) rather than a high-order
polynomial model.

This chapter looks at various regression models suitable for analyses of

deterministic and regular seasonal changes in time series data. The
logarithmic transformations, often used to stabilise the variance is also
considered.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 4 / 21

 Deterministic Trend

A possible model might be

Yt = µt + et ,

where µt is a deterministic function and et is the unobserved variation

around µt .
µt is periodic with period 12; that is, it should satisfy

µt = µt−12

for all t.
We may assume that E (et ) = 0 for all t so that µt = E (Yt ).
So, µt is a deterministic trend function of the time series.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 5 / 21

In other cases, we might hypothesize a deterministic trend that is
linear in time
µt = β 0 + β 1 t
or perhaps a quadratic time trend

µt = β0 + β1 t + β2 t 2

The model
Yt = µt + et
with E (et ) = 0 for all t means that the trend µt applies for all time.
Thus, if µt = β0 + β1 t, we are assuming that the same linear time
trend applies forever.
We should have good reasons for assuming such a model, not just
based on observed series.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 6 / 21

 Estimation of a Constant Mean
A series with constant mean
Yt = µ + et
with E (et ) = 0 for all t.
We want to estimate µ on the basis of an observed series Y1 , . . . , Yn .
A common estimate of µ is the series mean
n
1X
Ȳ = Yt
n
t=1

Properties of Ȳ : Since {et } has mean 0, we have

E (Ȳ ) = µ
Assume that {et } is a stationary time series with ACF ρk . Then
" n−1 # " n−1
#
γ0 X |k| γ0 X k
V (Ȳ ) = (1 − )ρk = 1+2 (1 − )ρk
n n n n
k=−n+1 k=1

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 7 / 21

 i. White Noise Error
Take Yt = µ + et , where et is white noise. That is, Yt = µ + et . Then

γ0
E (Ȳ ) = µ & V (Ȳ ) =
n
since ρk = 0 for k ̸= 0.

ii. Moving Average (MA) Error

Take Yt = µ + Xt , where Xt is the MA et + θet−1 . Then

E (Ȳ ) = µ

and
n−1
" #
γ0 X k
V (Ȳ ) = 1+2 (1 − )ρk
n n
k=1
σ2
 
n−1
= (1 + θ2 ) + ( )θ
n n

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 8 / 21

iii. Decaying ACF
If ρk decays fast enough with increasing k such that
∞
X
|ρk | < ∞
k=0

For large n, we can show that

∞
γ0 X
V (Ȳ ) ≈ ρk
n
k=−∞

So, variance is inversely proportional to n. If ρk = ϕ|k| , where |ϕ| < 1,

then  
γ0 1 + ϕ
V (Ȳ ) ≈
n 1−ϕ

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 9 / 21

iv. Random Walk Error
Take Yt = µ + Xt , where Xt is the random walk Xt = Xt−1 + et .

Then E (Ȳ ) = µ but

 
n n X
t
!
1 X 1 X
V (Ȳ ) = V Yt = 2V  ej 
n2 n
t=1 t=1 j=1
n
1 σ2 X 2
= V (e1 + 2e2 + · · · + ne n ) = i
n2 n2
i=1
(n + 1)(2n + 1)
= σ2
6n
In this special case, V (Ȳ ) increases with n (sample size). Clearly this
is unacceptable, and we need to consider other estimation techniques
for non-stationary series.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 10 / 21

 Regression Methods
1 Regression Estimation of the Linear Trend
Consider Yt = µt + Xt , where µt = β0 + β1 t.
To determine µt , we need to estimate β0 and β1 .
We can use the least squares method which chooses β0 and β1 by
minimizing
n
X n
X
2
Q(β0 , β1 ) = [Yt − µt ] = [Yt − (β0 + β1 t)]2
t=1 t=1

This gives
Pn
(Y − Ȳ )(t − t̄)
β̂1 = Pn t
t=1
2
;
t=1 (t − t̄)

β̂0 = Ȳ − β̂1 t̄

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 11 / 21

Example:
Consider the random walk process simulated in Chapter 2. Suppose we
want to fit a linear time trend and estimate the slope and intercept
through the least-squares method. The R code is as follows:

set.seed(9)
x <- w <- rnorm(1000)
for (t in 2:1000) x[t] <- x[t - 1] + w[t]
plot(x, type = "l", xlab="Time")
model <-lm(x~time(x))
summary(model)
abline(model)

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 12 / 21

Estimated slope and intercept are = 0.026 and = 15.727 respectively.
display of random walk with the least squares regression trend line
superimposed is provided below.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 13 / 21

 2 Regression Estimation of cyclical or seasonal trend
Now suppose we wish to model data with cyclical or seasonal trends, such
as monthly average temperature, monthly airline ticket sales, and monthly
electricity production quantities. The observed series can be represented as

Yt = µt + Xt

where E (Xt ) = 0 for all t such that the 12 months parameters(constants),

β1 , β2 , ..., β12 are estimated through µt as follows:

β for t = 1, 13, 25, . . .
 1


β2

for t = 2, 14, 26, . . .
µt = .

 ..


β12 for t = 12, 24, 36, . . .


Sometimes this is also called a seasonal means model.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 14 / 21

 Efficiency of Regression Estimates
Efficiency of estimators is measured using the variance of the estimator.

1 Seasonal trend
For the BW temperature case, the jth month coefficient is
N−1
1 X
β̂j = Yj+12i
N
i=0

where N is the number of years.

Since β̂j is an average (of every 12th observation), we have

N−1
" #
γ0 X k
V (β̂j ) = 1+2 (1 − )ρ12k
N N
k=1

γ0
NB: If Xt is white noise, then Var (B̂j ) reduces to N (as expected).

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 15 / 21

 2 Linear trend
Consider Yt = µt + Xt , where µt = β0 + β1 t.

We can show that the least squares estimate of β1 is

Pn
(t − t̄)Yt
β̂1 = Pt=1
n 2
t=1 (t − t̄)

From this, we see that

s−1
n X
" #
12γ0 24 X
V (β̂1 ) = 1+ (t − t̄)(s − t̄)ρs−t
n(n2 − 1) n(n2 − 1)
s=2 t=1

If {Xt } is a white noise process, then

12σ 2
V (β̂1 ) =
n(n2 − 1)

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 16 / 21

 Interpreting Regression Outputs

Consider Yt = µt + Xt , where µt = β0 + β1 t, say.

Once we find β̂0 and β̂1 , we can then find the estimate of µt as

µ̂t = β̂0 + β̂1 t

This is typically referred to the fitted of predicted value of Yt .

The model errors are Xt = Yt − µt . These are estimated by the
model residuals
X̂t = Yt − µ̂t

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 17 / 21

 Evaluating the Model

If the {Xt } process has constant variance γ0 , then we can be estimate it by

n
2 1 X 2
s = X̂t
n−p
i=1

where p is the number of parameters estimated to determine µ̂t . - The

quantity n − p is the degrees of freedom of s 2 .
The quantity s 2 is not unit free; so, not a good way to evaluate
model goodness-of-fit.
The coefficient of determination R 2 is a unit free measure of
goodness-of-fit
R 2 = Corr (Yt , µ̂t )

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 18 / 21

 1 Residual Analysis
Recall that the model errors Xt = Yt − µt are estimated by the model
residuals X̂t = Yt − µ̂t .

If the trend model is reasonably correct, then X̂t should behave roughly
like {Xt }.

If the {Xt } is white noise, then the X̂t should behave roughly like
independent (normal) random variables with zero mean and standard
deviation s.

We may standardize the residuals as X̂t /s but there are better ways that
take the regression model into account.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 19 / 21

2 Assessing Normality
The QQ-plot of the residuals may be used to assess normality

However, an excellent test of normality is known as the Shapiro-Wilk

test. It rejects normality if the correlation between the residuals and
the corresponding normal quantiles is small.
3 Assessing Independence
The runs test can be used to find evidence against independence.
Runs above or below their median are counted.
A small number of runs would indicate that neighboring residuals are
positively dependent and tend to hang together over time.
Too many runs would indicate that the residuals oscillate back and
forth across their median. Then neighboring residuals are negatively
dependent.
So either too few or too many runs lead us to reject independence.

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 20 / 21

 R session

Establish the seasonal mean model using the Botswana temperature

data ”BWtemp” shared and interpret your results.
(Hint: Exclude the constant in your analysis).
Assess the normality of the data through the following:
(a) qq plot
(b) shapiro-wilk’s test
Perform the run test to assess independence on the data

Dr Mabikwa (BIUST) STAT 411 - Time Series February 20, 2023 21 / 21