Nonstationary time series

Nonstationarity means presence of trend either in

i. Mean, ⟹ only deterministic trend

ii. Variance, ⟹ only stochastic trend
iii. Mean and variance⟹ both deterministic and stochastic trend

Nonstationary with unit root

In AR(1) process, for example,

y t =ϕ0 + ϕ1 y t −1+ ε t

Under the restrictions

|ϕ 1|<1, andt → ∞

We have
∞
ϕ
y t = 0 + ∑ ϕi1 ε t −1
1−ϕ1 i=0

ϕ0
E ( y t )=
1−ϕ 1
σ2
and V ( y t ) =
1−ϕ21

⟹ No trend in mean, no trend in variance

⟹ y t is stationary

But, suppose, in AR(1), ϕ 1=1 :

y t =ϕ0 + y t −1+ ε t

This model is known as Random walk with drift. Drift means intercept (ϕ 0)

In this case,
 t−1
y t = y o +ϕ 0 t +∑ ε t −i
i=0

E ( y t )= y 0 +ϕ 0 t

V ( y t ) =σ 2 t

⟹ Trend in mean and variance

⟹ y t is nonstationary

So, in AR(1) nonstationarity appears when |ϕ 1|=1

If you drop the intercept term in AR(1),

y t = y t−1 +ε t

This model is known as Random walk without drift.

In this case

t−1
y t = y o + ∑ ε t−i
i=0

E ( y t )= y 0 ⟹ no trend in mean

V ( y t ) =σ 2 t ⟹ trend in variance

⟹ y t is non stationary

So, in both cases, nonstationarity appears when|ϕ 1|=1

But, the type of nonstationarity is different:

In Random walk with drift, the series, y t , exhibits both deterministic and stochastic trend

In Random walk without drift, the series, y t , exhibits only stochastic trend

Common character of nonstationarity in both cases is the presence of stochastic trend

Now, let we try to understand the meaning of |ϕ 1|=1 which is responsible for nonstationarity.

Start with AR(1) process again,

y t =ϕ0 + ϕ1 y t −1+ ε t

By using lag operator,

y t =ϕ0 + ϕ1 Ly t +ε t

y t ( 1−ϕ1 L ) =ϕ 0+ ε t

Now, set

1−ϕ1 L=0

This equation is called inverse characteristic equation of AR(1) process

Set
1
z=
L

Therefore, we have

z−ϕ 1=0

This is called AR characteristic equation of order 1

The solution of this equation,

z=ϕ 1

is called characteristic root

If the value of the characteristic root is equal to 1, it is called unit root.

So, in AR(1),

unit root means

z=1
or,

ϕ 1=1

Therefore, the presence of unit root in AR(1) implies that the series is nonstationary
exhibiting stochastic trend with or without deterministic trend.

If ϕ 1<1 , the series is stationary

Nonstationarity with unit root means nonstationarity with stochastic trend.

What is stochastic trend?

Trend means change of a variable with time:

Δ y t = y t − y t −1

In random walk model without drift,

ϕ 0=0, and ϕ 1=1

and the AR(1) process becomes

y t = y t−1 +ε t

Therefore, the trend,

Δ y t = y t − y t −1 =ε t , which is stochastic

Stochastic trend is unobserved but it is revealed in trend in variance of the series.

How it is generated?

To understand it, express AR(1) with unit root in the following form:

t−1
y t = y o + ∑ ε t−i
i=0
This expression states that the value of y in period t is obtained by accumulating the shocks
t−1
over time , ∑ ε t−i , with its initial value, y o.
i=0

Stochastic trend means accumulation of shocks over time. For this reason, presence of
stochastic trend in a series creates a lot of problem.

The stochastic behaviour of yt described by

y t = y t−1 +ε t

is nonstationary, but its first difference, Δ y t = y t − y t −1 =ε t , is stationary.

For this reason, the random walk model follows difference stationary process (DSP):

the series is originally non stationary, but it becomes stationary after taking its difference.

Consider now AR(2) series:

y t =ϕ0 + ϕ1 y t −1+ ϕ2 y t −2+ ε t

By introducing lag operator,

y t =ϕ0 + ϕ1 L y t +ϕ 2 L2 yt +ε t

or,

y t ( 1−ϕ1 L−ϕ 2 L2)=ϕ0 + ε t

Set
1−ϕ1 L−ϕ2 L2=0

This quadratic equation is the inverse characteristic equation of AR(2) process

Set
1
z= to get
L

z 2−ϕ 1 z−ϕ 2=0

This is AR characteristic equation of order 2, the solution of it

ϕ1 ± √ϕ 21+ 4 ϕ 2
z=
2

There are 2 characteristics in AR(2): z1 and z2

Now,

if i) z1 = 1, z2 =1, there are 2 unit roots ⟹ y t is non stationary exhibiting stochastic trend

ii) z1 = 1, z2 < 1, or z1 <1, z2 = 1, there is 1 unit root⟹ y t is non stationary exhibiting

stochastic trend

iii) z1 <1, z2 <1, there is no unit root⟹ y t is stationary

While yt both in (i) and (ii) are nonstationary, the nature of nonstationarity is different
because the nature of stochastic trend is different.

The sum of the roots,

z 1+ z2 =ϕ1

and product of the roots,

z 1 × z2=−ϕ2

Therefore, AR(2) series will contain 2 unit roots only when

ϕ 1=2

and

ϕ 2=−1

In the case of AR (2), maximum number of unit root = 2, and in this case the series to be
differenced twice to have stationary series and the series will be integrated of order 2.

If the AR(2) series has one unit root, then we have to take 1 st difference to make it stationary,
and in this case it will be integrated of order 1.
For AR (p) series:

y t =ϕ0 + ϕ1 y t −1+ ϕ2 y t −2+ …+ϕ p y t − p+ ε t

the characteristic equation will be of order p:

ϕ ( z )=z p−ϕ1 z p−1−ϕ2 z p−2−…−ϕ p=0

¿
If ϕ ( z )=( z−1 ) ϕ ( z )=0

the AR(p) series contains 1 unit root, and the series is called integrated of order 1, I(1)

¿
Here, ϕ ( z ) is AR characteristic polynomial of order p-1

If
2
ϕ ( z )=( z−1 ) ϕ¿∗¿ z =0¿
( )

the AR(p) series contains 2 unit root, the series is called integrated of order 2.

ϕ ¿∗¿ (z ) ¿ is AR characteristic polynomial of order p-2,

In this way,

if ϕ ( z )=( z−1 )d ϕd ( z )=0

the AR(p) series contains d unit root, and the series is called integrated of order d.

ϕ d ( z ) is AR characteristic polynomial of order p-d

In AR(p) series, the maximum number of unit roots is p, in that case the series is called
integrated of order p.

Non-stationarity without unit root

A nonstationary series without unit root exhibits only deterministic trend. Suppose that a
series, yt, always changes by the same fixed amount from one period to the next:

Δ y t =β
or,
y t − y t−1=β

The general solution to this linear difference equation is

y t = y 0 + βt

Here, yt is exhibiting deterministic linear time trend.

The actual time behaviour of yt is described by the following way:

y t =α + βt + ε t

ε t N ( 0 ,σ 2 ), cov ( ε t , ε t −k ) =0

The population regression function is

E ( y t )=α + βt

This is the popular trend model used to estimate growth rate. The trend in mean value is
completely predictable and is known as the deterministic trend.

The variance of yt,

2
V ( y t ) =E ( y t −E ( y t ) ) =σ 2

The variance of yt is time invariant and dose not exhibit any trend.

cov ( y t , y t− k ) =E ( y t −E ( y t ) )( y t −k −E ( y t−k ) )=0

While variance and covariance are time invariant, the mean of yt is time variant. Thus, yt is
nonstationary time series exhibiting deterministic trend. Deterministic trend is a systematic
change of the mean level of a series over time.
After detrending the series it becomes stationary:

zt = yt – E(yt) = εt

Therefore, yt is originally non-stationary, but after de trending it becomes stationary. The data
generating process (DGP) of yt follows in this case is known as the trend stationary process
(TSP).

The issue whether a time series is of DSP or TSP is important because the dynamic properties
of the two processes are different. While the TSP is predictable, the DSP is not completely
predictable. For TSP, the effects of external shocks are temporary, while for DSP any random
shock to the series has a permanent effect. A TSP has a trend in the mean but no trend in the
variance, but a DSP has a trend in the variance with or without trend in the mean1.

TSP DSP
1. Non stationary 1. Non stationary
2. Stationary by detrending 2. Stationary by taking difference
3. Trend in mean – deterministic trend 3. Trend in mean – deterministic
4. No trend in variance or covariance trend, or no trend in mean (without
– no stochastic trend deterministic trend)
5. Effect of shock is transitory, it dies 4. Trend in variance or covariance –
out shortly stochastic trend. The stochastic
trend incorporates all the random
shocks (ε1 to εt) that have permanent
effects on the level of yt.
5. Effect of shock is permanent, it
persists for long period

Unit Root Test

Unit root ⟹ characteristic roots equal to 1

Presence of unit root⟹ the series is nonstationary exhibiting stochastic trend

How to check it?

1
A random walk without drift has no trend in the mean values of the variable.
Plot the data

Plot ACF

If ACF decays slowly, indication of unit root

Many statistical tests for unit roots are available now for detecting unit root.

Dickey-Fuller unit root test

The Dickey–Fuller (1979) test considers only AR(1) model:

y t =ϕ1 y t−1 +ε t (1)

y t =ϕ0 +ϕ1 y t−1 + ε t (2)

y t =ϕ0 +ϕ1 y t−1 + βt + ε t ()

Before carrying out test we need to estimate the model by applying OLS.

For AR(1), presence of unit root means |ϕ 1|=1

Therefore, perform the following test after estimation:

H 0 :|ϕ1|=1

H 1 :|ϕ1|<1

Here, H0 ⟹ presence of unit root, or random walk

H1⟹ stationary

In (3), the null hypothesis should be

H 0 :|ϕ1|=1 , β=0

H 1 :|ϕ1|<1 , β ≠ 0

Here, H0 ⟹ DSP
H1⟹ TSP

How to select the form of AR(1)?

Model selection is very much crucial for testing unit root. If the observed series does not

exhibit an increasing or decreasing trend, then the appropriate model will be either (1) or, (2).

Figure 1 Time path of a series without trend

Case I: a series without unit root Case II: A series with unit root

Figure 3.2 Time path of a series with trend

Case I: a series without unit root Case II: A series with unit root

To control for serial correlation, the Dickey–Fuller test estimates a model of the following

form:

∆ y t =ρ y t−1 + ε t (4)

∆ y t =ϕ 0+ ρ y t −1+ ε t (5)

∆ y t =ϕ 0+ ρ y t −1+ βt + ε t (6)

The hypothesis for (4) and (5) is

H 0 :|ρ|=0

H 1 :| ρ|<0

The hypothesis for (6) is

H 0 :|ρ|=0 , β=0

H 1 :| ρ|<0 , β ≠ 0
Test statistics for Dickey–Fuller test

The unit root test is a one-sided left tail test.

The test statistic under H0 is

ϕ^ 1−1 ^ρ
t= = (7)
SE ( ϕ^ 1 ) SE ( ^ρ )

The OLS estimate of ϕ1

T

∑ y t−1 y t
ϕ^ 1= t =1T (8)
2
∑y t −1
t=1

or,
T T

∑ y t−1 y t ∑ εt y t −1
t =1
ϕ^ 1= T
=ϕ1 + t =1T
∑ y 2t −1 ∑ y 2t−1
t=1 t=1

Therefore,
E ( ϕ^ 1) =ϕ1 (9)

and
2
σ2 σ2 σ2 1−ϕ1
V ( ϕ^ 1 ) = T
= T
= 2
=
σ T
∑y 2
t−1 T ∑ y 2t −1 T 2
(10)
t =1 t=1 1−ϕ1
T

We know that, for very large T, the t distribution will be standard normal in asymptotic sense,

t A N (0 , 1)
→
If ε t N ( 0 ,σ 2 ), then it can be shown that

1−ϕ21
(
ϕ^ 1 N ϕ 1 ,
T )

But, under H0, ϕ^ 1 →

A N (1 , 0 ) which clearly does not make any sense.

Under the null hypothesis of unit root, the t statistic is not asymptotically normally

distributed, and special critical values are required.

For higher order AR process

Augmented Dickey-Fuller (ADF) unit root test

Said and Dickey (1984) extended the basic autoregressive unit root test.

This test is referred to as the augmented Dickey-Fuller (ADF) test.

The ADF test also the null hypothesis that a time series yt is I(1) is tested against the

alternative that it is I(0).

If a time series follows autoregressive process of higher order we need to incorporate some

augmented terms to convert it into AR(1).

Consider that yt follows AR(2) process:

y t =ϕ0 + ϕ1 y t −1+ ϕ2 y t −2+ ε t (11)

We can modify the model as

y t =ϕ0 +¿
or,
Δ y t =ϕ 0+ ¿
or,
Δ y t =ϕ 0+ ρ y t−1+ β1 Δ y t −1+ ε t (12)

Here, β 1=−ϕ 2 , ρ=ϕ1 +ϕ2 −1

Equation (12) is AR(1) with augmented term β 1 Δ y t−1

and the model is augmented Dickey Fuller (ADF) model.

If yt follows AR(2), we need 1 augmented term to convert it into AR(1).

Therefore, (12) is ADF of order 1. The AR(1) series is ADF of order 0.

For AR(3)

y t =ϕ0 + ϕ1 y t −1+ ϕ2 y t −2+ ϕ3 y t−3 +ε t (13)

y t =ϕ0 +(ϕ1 +ϕ 2+ ϕ3) y t −1 −(ϕ 2+ ϕ3 )( y t−1− y t−2)−ϕ3 ( y t −2− y t −3)+ ε t

Δ y t =ϕ 0+ ¿

Δ y t =ϕ 0+ ρ y t−1+ β1 Δ y t −1+ β 2 Δ y t −2 + ε t (14)

It is AR(1) with 2 augmented terms

If yt follows AR(p) process, we have to incorporate (p-1) augmented terms into the model to

make it AR(1):
p−1
Δ y t =ϕ 0+ ρ y t−1+ ∑ β j Δ y t− j+ ε t (15)
j=1

If we include trend component into the AR(p) model, the ADF formulation will be

p−1
Δ y t =ϕ 0+ ρ y t−1+ ∑ β j Δ y t− j+ βt + ε t (16)
j=1
AR(p) process will be ADF (p-1)

The value of p is set in such a way that the error εt is not serially correlated.

The ARMA (p, q) process is ADF of order ∞

Let the following ARMA(p, q) process

ϕ ( L ) y t=θ ( L ) ε t

If the roots of θ(L) are outside the unit circle, we can write the following sequence:

ϕ ( L ) θ ( L )−1 y t =ε t

It is AR of order infinity. Therefore, the ADF form of ARMA(p, q) model is

∞
Δ y t =ρ y t −1 + ∑ β j Δ y t− j + ε t (17)
j=1

The infinite order of AR or ADF cannot be estimated with finite data set.

Selection of lag length

The inference of ADF test is very sensitive to the selection of lag length (Schwert 1989,

Agiakloglou and Newbold 1992, Harris 1992).

It is shown by Ng and Perron (1995) that if p is chosen on the basis of some information
criteria (IC).

The method of selecting lag length on the basis of information criteria considers a trade-off

between the size distortions because of the inclusion of too few lags and the power losses

because of the inclusion of too many lags.

In this approach the optimum lag length (p*) is obtained as:

p¿ =arg min IC ( p), IC ( p )=ln σ^ 2 + p CT (18)

p
p ≤ p≤ p
min max
T

Here,σ^ 2p is the OLS residuals from the pth order ADF regression; CT is a penalty function

defined differently in different information criterion to be used.

The most popularly used information criteria are

Akaike information criterion (AIC) proposed by Akaike (1974)

Bayesian information criterion (BIC) proposed by Schwartz (1978), and

Hannan and Quinn (1979) information criterion (HQIC).

In AIC, CT = 2, for BIC, CT = ln T, for HQIC, CT = ln(ln T).

AIC ( p )=Tln σ^ 2p +2 p (19)

BIC ( p )=Tln σ^ 2p + pln(T ) (20)

HQIC ( p )=Tln σ^ 2p + pln(ln ( T )) (21)

In this procedure, we have to set an upper bound pmax for p and estimate the ADF regression

with p = pmax. If the coefficients for last lagged differences are not statistically significant, we

need to reduce the lag length one by one and repeat the process. A useful rule of thumb for

determining pmax, suggested by Schwert (1989), is

 .25
T
pmx =12 × ( )
100
(21)