lOMoARcPSD|7481553

Time Series Summary

Time Series (Technische Universiteit Delft)

StuDocu is not sponsored or endorsed by any college or university

Downloaded by Iconic F. (iconicfount1@gmail.com)
 lOMoARcPSD|7481553

T IME S ERIES
S UMMARY

by

ALEX AN DER VAN DEN BERGHE (4274075)

October 30, 2017

[1]

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

0. C ONTENTS

1 Handout I 1
1.1 Examples of time series and objectives of time series analysis . . . . . . . . . . . . . . . . . . 1
1.1.1 Objectives of Time Series Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 General approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Financial Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Recap Covariance and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3.1 Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3.2 Correlation-Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Stationary Models and measuring dependence using correlations . . . . . . . . . . . . . . . . 2
1.4.1 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4.2 Autocovariance and Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4.3 Estimating moments using sample moments . . . . . . . . . . . . . . . . . . . . . . . 2
1.4.4 Time series models used for this course: . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4.5 IID noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4.6 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4.7 Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4.8 First order moving average process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4.9 First-order autoregressive process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Removing Trend and Seasonal Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5.1 Removing trend components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Testing the estimated noise terms: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6.1 Testing for independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6.2 Testing for Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6.3 Testing for White-Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6.4 Testing for normality: Gaussian QQ-Plot . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Handout II 8
2.1 Linear Processes: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 MA(q)-models:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 AR(p)-models: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.3 Estimation of AR(p)-models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 ARMA(p, q)-models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 ARIMA(p, d, q)-models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Conditional probability and expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 Forecasting Stationary Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.1 Introduction to prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.2 One step ahead prediction for AR(p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.3 Two step ahead prediction for AR(p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.4 One step ahead prediction for MA(1). . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.5 Two step ahead prediction for MA(1). . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.6 l step ahead prediction for MA(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.7 Introduction to best linear predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.8 Best linear predictor: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6 Partial Autocorrelation Function (PACF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6.1 Formal Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.6.2 Equivalent Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.6.3 Distribution of sample PACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Time Series i WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

C ONTENTS ii

3 Handout III 16
3.1 Modeling Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.1 Financial Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.2 Heavy tails: Kurtosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 ARCH-models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.1 ARCH(1)-model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.2 Properties of ARCH(1) model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.3 Squared ARCH(1) process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.4 Existence of a stationary ARCH(1)-process . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.5 Summary ARCH(1). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.6 Estimation of ARCH(1)-processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.7 Standardized residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.8 Forecasting with ARCH(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.9 Recursion relation for predicting ARCH(1) . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.10 Kurtosis of stationary ARCH(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.11 Choice for IID-sequence { Z t } . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.12 Weaknesses of ARCH models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.13 Extension to ARCH(m) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 GARCH-models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.1 Properties of GARCH(1,1) model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.2 ARMA(1,1) representation of GARCH(1,1) . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.3 Further properties and extensions of GARCH(1,1) . . . . . . . . . . . . . . . . . . . . . 21

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1. H ANDOUT I

1.1. E XAMPLES OF TIME SERIES AND OBJECTIVES OF TIME SERIES ANALYSIS

A time series is a collection of observations x t , recorded at separate individual times t.

1.1.1. O BJECTIVES OF T IME S ERIES A NALYSIS

1. Provide a compact description of the data
2. Identify global trends and or seasonal components
3. Predict future values of the series
4. Separation (or filtering) of noise from a signal
5. Simulate systems in which time series serve as input

This is done by:

• Finding a suitable probability model that models the uncertainty an a time series
• Specifying the joint distribution of C t , 0 ≤ t ≤ n )

1.1.2. G ENERAL APPROACH

This is done by plotting the time series and looking for:

• trend / seasonal component

• large changes in behavior
• extreme outlying observations

Afterwards, the trend and seasonal components are removed so the remaining residuals are stationary. Then
a probability model is fitted on the residuals, which can be used to predict future values of the time series.

1.2. F INANCIAL T IME S ERIES

if {P t } is the price of an asset at time t, then simple gross return is defined as:
Pt
(1.1)
P t −1

The simple return is defined as:

P t − P t −1
Rt = (1.2)
P t −1
And the log-return is defined as:

l t = l og (
Pt
P t −1
P t − P t −1
= l og (1 + ) (1.3)
P t −1
= l og (1 + R t )
≈ Rt

Time Series 1 WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.3. R ECAP C OVARIANCE AND C ORRELATION 2

1.3. R ECAP C OVARIANCE AND C ORRELATION

1.3.1. C OVARIANCE
if E[X 12 ] < ∞ and E[X 22 ] < ∞.

C ov ( X 1 , X 2 ) = E [( X 1 − E [ X 1 ])( X 2 − E [ X 2 ])]
= E [ X 1 ⋅ X 2 ] − E [ X 1 ]E [ X 2 ]
(1.4)

For X 1 and X 2 independent, Cov(X 1 , X 2 ) = 0. This is not a reversible property. Except for the bivariate normal
distribution.

C ov (a X + bY + c, Z ) = a ⋅ C ov ( X , Z ) + b ⋅ C ov (Y , Z ) (1.5)

1.3.2. C ORRELATION -C OEFFICIENT

ρ(X 1 , X 2 ) = √
C ov ( X 1 , X 2 )
(1.6)
V ar ( X 1 )V ar ( X 2 )

−1 ≤ ρ ( X 1 , X 2 ) ≤ 1
The correlation measures linear dependence of X 1 and X 2 .

1.4. S TATIONARY M ODELS AND MEASURING DEPENDENCE USING CORRELA -

TIONS
1.4.1. S TATIONARITY
Let {X t } be a time series with E[X t2 ] < ∞. Then X t is stationary if:

1. E[X t ] = µ, is independent of t.

2. Cov(X t +h , X t ) is independent of t, and only depends on the value for h

3. From property 2 it also follows that Var(X t ) = Cov(X t , X t ) is constant.

1.4.2. AUTOCOVARIANCE AND AUTOCORRELATION

The autocovariance of X at lag h is defined as:

γ X (h ) = C ov ( X t +h , X t ) (1.7)

and the autocorrelation of X at lag h is:

ρ X (h ) = = ρ ( X t +h , X t )
γ X (h )
(1.8)
γ X (0)

Note that these functions are even functions so that γ X (h ) = γ X (−h ) and ρ X (h ) = ρ X (−h ).

1.4.3. E STIMATING MOMENTS USING SAMPLE MOMENTS

Sample mean:
1 n
x̄ ∶= ∑ x t (1.9)
n t =1

Sample Autocovariance:
n −∣h ∣
γ̂(h ) ∶= ∑ (x t +h − x̄ )(x t − x̄ )
1
(1.10)
n t =1

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.4. S TATIONARY M ODELS AND MEASURING DEPENDENCE USING CORRELATIONS 3

Sample Autocorrelation:

n −∣h ∣
∑ (x t +h − x̄ )(x t − x̄ )
γ̂(h )
ρ̂ (h ) = =
t =1
γ̂(0)
(1.11)
n −∣h ∣
∑ (x t − x̄ )2
t =1

1.4.4. T IME SERIES MODELS USED FOR THIS COURSE :

1. Independent and Identically Distributed (IID) noise

2. White noise

3. Random walk

4. Moving average processes (MA)

5. Autoregressive processes (AR)

6. ARMA processes

7. ARCH processes

8. GARCH processes

1.4.5. IID NOISE

{X t } is a sequence of Independent Identically Distributed random variables.
E [ X t ] = 0 and V ar ( X t ) = σ2 .

σ2 , if s = t
C ov ( X s , X t ) = { (1.12)
0, if otherwise

hence, {X t } is stationary and:

σ2 , if h = 0
γ X (h ) = {
if h ≠ 0
(1.13)
0,

1, if h = 0
ρ X (h ) = {
0, if h ≠ 0
(1.14)

1.4.6. W HITE N OISE

A sequence {X t } of uncorrelated random variables, with E[X t ] = 0 and Var(X T ) = σ2 is called white noise:

σ2 , if h = 0
γ X (h ) = {
if h ≠ 0
(1.15)
0,
and
1, if h = 0
ρ X (h ) = {
0, if h ≠ 0
(1.16)

This looks similar to the IID noise but it is not equivalent. A sequence {X t } can be WN but not IID when it is
for example not independently distributed, such as:

⎧
⎪
⎪Zt , if t is even
X t = ⎨ Zt2−1
⎪
⎪
⎩ , if t is odd
√
2
With {Z t } ~IID N(0, 1).
This is clearly not an indepently distributed sequence and is therefore not IID. It is however WN(0, 1)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.4. S TATIONARY M ODELS AND MEASURING DEPENDENCE USING CORRELATIONS 4

Distribution of the sample ACF for white noise: Identifying white noise from data can be done by looking
at the autocorrelation plot.
if {X t } ~WN(0, σ2 ) then for all h ≥ 1:
√
n ρˆn (h ) is normally distributed ( N(0, 1) ) as n goes to infinity.
Thus: ρˆn (h ) ≈ N (0, n1 ) (for sufficiently large n) and with a probability of 0.95 (2σ):

−√ ≤ ρˆn (h ) ≤ √
1.96 1.96
(1.17)
n n

Note: these are the two lines that appear in ACF plots in R. If the autocorrelations for all h ≥ 1 fall between
these two lines, you can say with 95% confidence that the sequence is white noise.

1.4.7. R ANDOM WALK

A sequence {X t }, where:

X t = Z1 + Z2 + ... + Z t , with Z t ~I I D (0, σ2 ) (1.18)

E [X t ] = 0
V ar ( X t ) = V ar ( Z1 ) + V ar ( Z2 ) + ... + V ar ( Z t ) = t σ2
(1.19)

Hence:

C ov ( X t +h , X t ) = C ov ( X t + Z t +1 + ... + Z t +h , X t )
= C ov ( X t , X t ) + C ov ( Z t +1 , X t ) + ... + C ov ( Z t +h , X t )
= C ov ( X t , X t )
(1.20)

= V ar ( X t ) = t σ2

This series is therefore not stationary.

1.4.8. F IRST ORDER MOVING AVERAGE PROCESS

{X t } is an MA(1) process if it is stationary and satisfies:

X t = Z t + θZ t −1 , with { Z t } ~W N (0, σ2 ) and θ ∈ R (1.21)

⎧
⎪σ2 (1 + θ 2 ) if h = 0
⎪
⎪
⎪ 2
C ov ( X t , X t +h ) = ⎨σ θ if h = ±1
⎪
(1.22)
⎪
⎪
⎪
⎩0 if ∣h ∣ ≥ 2

⎧
Therefore:
⎪
⎪ if h = 0
⎪
⎪
1
ρ X (h ) = ⎨ 1+θθ2 if h = ±1
⎪
(1.23)
⎪
⎪
⎪
⎩0 if ∣h ∣ ≥ 2

1.4.9. F IRST- ORDER AUTOREGRESSIVE PROCESS

{X t } is an AR(1) process if it is stationary and satisfies:

X t = φX t −1 + Z t , with Z t ~W N (0, σ2 ) (1.24)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.5. R EMOVING T REND AND S EASONAL C OMPONENTS 5

X t = φX t −1 + Z t
= φ(φX t −2 + Z t −1 ) + Z t
= φ2 X t −2 + φZ t −1 + Z t
k −1
= φk X t −k + ∑ φ j Z t − j (1.25)
j =0
∞
Ð→ ∑ φ j Z t − j
j =0

Provided∣φ∣ < 1

Clearly E[X t ] = 0. For h ≤ 0:

γ X (h ) = C ov ( X t , X t +h )
φh
= σ2
1 − φ2
(1.26)

= φh γ X (0)

For the process to be stationary,

E [ X 0 ] = 0 and V ar ( X 0 ) = γ X (0) = 1−σφ2
2

1.5. R EMOVING T REND AND S EASONAL C OMPONENTS

The classical decomposition of a time series is as follows:

X t = mt + st + Yt (1.27)
with:

• m t is a trend component

• s t is a seasonal component (periodic function)

• Y t is a random noise term (stationary)

The analysis is then done by estimating and extracting the trend and seasonal components and then finding
a suitable probabilistic model for Y t .
The removal of trend components can be done in 3 ways:

• Linear filters (not part of this course)

• Polynomial fitting

• Differencing

1.5.1. R EMOVING TREND COMPONENTS

P OLYNOMIAL FITTING
fit a polynomial of degree k ≤ 1 of the form:

m t = a 0 + a 1 t + ... + a k t k (1.28)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.6. T ESTING THE ESTIMATED NOISE TERMS : 6

Estimate parameter values using LSQ:

n
mi n ∑ (x t − a 0 − a 1 t − ... − a k t k )2 (1.29)
t =1

This results in a trend estimate m̂ t and a de-trended series:

Y t = X t − m̂ t = X t − aˆ0 − aˆ1 t − ... − aˆk t k (1.30)

D IFFERENCING
The backward shift operator B is defined by:

B X t = X t −1 (1.31)

and the differencing operator ∇ by:

∇( X t ) = X t − X t −1 = (1 − B ) X t (1.32)

Differentiating once will remove linear trends, while differentiating twice will remove quadratic drift.

1.6. T ESTING THE ESTIMATED NOISE TERMS :

1.6.1. T ESTING FOR INDEPENDENCE
The turning point test: When looking at a series and looking at three successive values then the three values
can occur in 6 possible orders (fig 1.1):

Figure 1.1: Possible turning points for three successive values

If the time series is independent, each of these possibilities will have an equal probibility. 2/3 of the possibil-
ities are turning points so in a series of n points, the amount of turning points Tn is expected to be:

E [Tn ] = (n − 2)
2
(1.33)
3

The statistic is then defined as Un :

Tn − 2(n − 2)/3
Un ∶= √ ~N (0, 1) (1.34)
(16n − 29)/90

H0 or the null hypothesis that the series is stationary is rejected for large or small values of Un (typically
outside of the 2σ or 3σ bounds)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

1.6. T ESTING THE ESTIMATED NOISE TERMS : 7

1.6.2. T ESTING FOR S TATIONARITY

Kwiatowsky, Phillips, Schmidt and Shin (KPSS-test).
Consider the model:

X t = a 0 + ξU t + Z t , with { Z t } ~W N (0, σ2z )

(1.35)
U t = U t −1 + ǫt , with {ǫt } ~I I D (0, σ2ǫ

The KPSS-test has: H0 ∶ ξ = 0 and H1 ∶ ξ ≠ 0.

When the p-value is greater than 0.05, the null hypothesis is accepted.

1.6.3. T ESTING FOR W HITE -N OISE

U SING THE ACF

If { X t } ~W N (0, σ2 ):

• for any h ≤ 0: ρˆn (h ) ≈ N (0, 1/n ) (for n sufficiently large)

• With probability 0.95:

−√
1.96
n
≤ ρˆn (h ) ≤ 1.96
√
n

L JUNG -B OX -T EST
H
ρˆn (h )2
Q LB = n (n + 2) ∑
n −h
(1.36)
h =1

If { X t }~W N (0, σ2 ), then Q LB ≈ χ2 ( H ) (for n sufficiently large). Determine critical values for test from signif-
icance and χ2 -distribution with H degrees of freedom.

1.6.4. T ESTING FOR NORMALITY: G AUSSIAN QQ-P LOT

x −µ
Let:
F ( x ) = Φ( ) (1.37)
σ
Note that:

F ( X (i ) ≈ F n ( X (i ) , with X (i ) =
i
n (1.38)
X (i ) ≈ F −1 (i /n ) = µ + σΦ−1 (1/n )

So, if the hypothesis that { X t } is distributed normally holds, then the points:

(Φ−1 ( ), X (i ) )
i
n +1
(1.39)

Are approximately on a straight line.

Other tests for normality:

• Shapiro-Wilk test

• Jarque-Bera test

• etc.

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2. H ANDOUT II

2.1. L INEAR P ROCESSES :

A time series { X t } is a linear process if it has the representation:

∞
X t = ∑ ψ j Zt − j (2.1)
j =−∞

Where { Z t }~W N (0, σ2 ),

and {ψ j } is a sequence of constants with:
∑ ∣ψ j ∣ < ∞
∞

j =−∞
A process is called causal if ψ j = 0 for j < 0, and can be written as follows:

X t = Ψ(B ) Z t
Ψ(B ) = ψ0 + ψ1 B + ψ2 B 2 + ...
(2.2)

2.1.1. MA( Q )- MODELS :

A moving average process of order q is defined as:

X t = Z t + θ1 Z t −1 + ... + θq Z t −q (2.3)
Where { Z t }~W N (0, σ ), and θq ≠ 0.
2

This is a causal linear process with:

if 0 ≤ j ≤ q
ψj = {
θj
if j > q
(2.4)
0

Note that θ0 is by definition always equal to 1.

We write:

X t = Θ(B ) Z t
Θ(B ) = 1 + θ1 B + θ2 B 2 + ... + θq B q
(2.5)

⎧
⎪
⎪
⎪
q −h
⎪σ2 ∑ θ j θ j +h if ∣h ∣ ≤ q
γ X (h ) = ⎨ j =0
⎪
(2.6)
⎪
⎪
⎪
⎩0 if ∣h ∣ > q

If { X t } is a stationary time series with mean zero, for which γ(h ) = 0 for |h| > q, it can be represented as a
MA(q) process.

Time Series 8 WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.1. L INEAR P ROCESSES : 9

2.1.2. AR( P )- MODELS :

A time series follows an AR(p) model of it is a stationary solution of:

X t = ϕ1 X t −1 + ϕ2 X t −2 + ... + ϕp X t −p + Z t (2.7)
Where Z t ~W N (0, σ2 ).

We write: Φ(B ) X t = Z t
with ϕ(B ) the autoregressive operator: Φ(B ) = 1 − ϕ1 B − ϕ2 B 2 − ... − ϕp B p

AR(1)- MODELS

X t = ϕX t −1 + Z t
⎧
⎪
⎪ if ∣ϕ∣ < 1
∞
⎪
⎪ ∑ ϕ j Zt − j
⎪ j =0
=⎨ ∞
(2.8)
⎪
⎪
⎪− ∑ ϕ− j Z t + j if ∣ϕ∣ > 1
⎪
⎪
⎩ j =1

It can be seen that the solution for ∣ϕ∣ > 1 is not useful for prediction, since it is non-causal and depends on
the future.
The solution for ∣ϕ∣ < 1 is the unique causal solution to the AR defining equation.

C AUSAL AR( P )- MODELS

An AR(p)-process is causal if:
∞ ∞
X t = ∑ ψ j Zt − j , With ∑ ∣ψ j ∣ < ∞ (2.9)
j =0 j =0

And the roots of the equation

Φ(z ) = 1 − ϕ1 z − ... − ϕp z p = 0 (2.10)

Are outside of the unit circle (the roots always satisfy |z| > 1)

Definition AR(p): Φ(B ) X t = Z t

X t = Ψ(B ) Z t
(2.11)
Causal linear process:

Therefore: Φ(B )Ψ(B ) = 1

And the coefficients {ψ j } satisfy the equation:

(1 − ϕ1 z − ϕ2 z 2 − ... − ϕp z p )(1 + ψ1 z + ψ2 z 2 + ...) = 1 (2.12)

The solution for the coefficients {ψ j } can be found by reorganizing the coefficients.

2.1.3. E STIMATION OF AR( P )- MODELS

The estimation of autoregressive models is done by maximizing the conditional likelihood L (ϕ1 , ..., ϕp , σ).

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.2. ARMA( P, Q )- MODELS 10

Example Estimation of an AR(1) model:

AR(1), X t = ϕX t −1 + Z t , { Z t }~I I D (0, σ2 ). Assume Z t ~N (0, σ2 )

L (ϕ, σ2 ) = f (x 1 , ..., x n ∣ϕ, σ2 )

n
= f (x 1 ∣ϕ, σ2 ) ∏ f (x t ∣x t −1 , ϕ, σ2 )
t =2 (2.13)
n
(x t − ϕx t −1 )
2
= f (x 1 ∣ϕ, σ2 ) ∏(2πσ2 )−1/2 exp(− )
t =2 2σ2

After fitting, the residuals (or standardized residuals) should be tested for WN and N.

If the residuals are not white noise or normal, the model is inappropriate for the data. The question then
becomes: how do we select the right AR model? The goal is to find the most parsimonious model (the model
that uses the smallest amount of parameters) that fits the data well.
This is done using the Akaike Information Criterion or the Corrected version thereof:

AIC = −2l og (L (ϕ1 , ..., ϕp , σ)) + 2(p + 1)

2(p + 1)n
AICC = −2l og (L (ϕ1 , ..., ϕp , σ)) +
(2.14)
n −p −2

2.2. ARMA( P, Q )- MODELS

A time series { X t } is an ARMA(p, q) process if { X t } is a stationary solution of:

X t − ϕ1 X t −1 − ... − ϕp X t −p = Z t + θ1 Z t −1 + ... + θq Z t −q (2.15)

Where { Z t }~W N (0, σ2 )

If θ1 = ... = θq = 0, then { X t }~AR (p )
If ϕ1 = ... = ϕp = 0, then { X t }~M A (q )

An MA(q)-process { X t } is invertible, i.e.,

∞ ∞
Zt = ∑ π j X t − j with ∑ ∣π j ∣ < ∞ (2.16)
j =0 j =0

if and only if the roots of the equation:

Θ(z ) = 1 + θ1 z + ... + θq z q = 0 (2.17)

Are outside the unit circle (so the roots satisfy |z| > 1).

A causal, stationary solution { X t } of the equations:

X t − ϕ1 X t −1 − ... − ϕp X t −p = Z t + θ1 Z t −1 + ... + θq Z t −q (2.18)

exists if and only if the roots of:

Φ(z ) = 1 − ϕ1 z − ... − ϕp z p = 0 (2.19)
Are outside the unit circle.
Then we can write:

Θ(B )
X t = Ψ(B ) Z t Ψ(B ) =
Φ(B )
with (2.20)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.3. ARIMA( P, D, Q )- MODELS 11

An ARMA(p, q) process { X t } is invertible, i.e.,

∞ ∞
Zt = ∑ π j X t − j with ∑ ∣π j ∣ < ∞ (2.21)
j =0 j =0

if and only if all roots of:

Θ(z ) = 1 + θ1 z + ... + θq z q = 0 (2.22)

Are outside of the unit circle.

2.2.1. C ONCLUSION
Type of process ACF PACF
AR(p) Exponential decay Zero after lag p
MA(q) Zero after lag q Exponential decay
ARMA(p,q) Exponential decay Exponential decay

Table 2.1: Caption

The steps to identifying an appropriate ARMA(p,q) model are listed below:

1. Make a time-plot of the data

2. Apply transformations to transform the data into a stationary time-series (differencing, polynomial
fitting,...)

3. Identify values for p and q using the ACF and PACF

4. Estimate the parameters of the model (Maximize the likelihood)

5. Perform residual analysis

If multiple ARMA models seem reasonable, compare them using either the smallest ψ-weights or the smallest
AIC(C)-value.

2.3. ARIMA( P, D, Q )- MODELS

If the d-times differenced time series is modeled by an ARMA(p, q)-model, then we speak of an ARIMA(p, d,
q)-model.

A zero-mean ARIMA(2,1,1)-model for { X t } means that

Y t = (1 − B ) X t = X t − X t −1 (2.23)

satisfies:
Y t − ϕ1 Y t −1 − ϕ2 Y t −2 = Z t + θ1 Z t −1 (2.24)

2.4. C ONDITIONAL PROBABILITY AND EXPECTATION

The conditional probability mass function of Y given X=x is defined as:

P Y ∣ X ( y ∣x ) = P (Y = y ∣ X = x ) (2.25)
Whenever P(X=x)>0.
It follows from the definition that:

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.5. F ORECASTING S TATIONARY T IME S ERIES 12

P ( X = x, Y = y )
P Y ∣ X ( y ∣x ) =
P (X = x )
(2.26)

If X and Y are independent, then:

P ( X = x, Y = y ) = P ( X = x )P (Y = y )
P Y ∣ X ( y ∣x ) = P Y ( y )
(2.27)

The conditional expectation of Y given X is denoted by E[Y | X=x] and defined as:

E [Y ∣ X = x ] = ∑ p Y ∣ X ( y ∣x ) (2.28)
y

Properties of conditional expectation

1. If X and Y are independent, then:

E [Y ∣ X = x ] = E [Y ] (2.29)

2. Expectation of conditional expectation

E [Y ] = ∑ E [Y ∣ X = x ]P ( X = x )
x (2.30)
= E [E [Y ∣ X ]]
3. Taking out what is known:
E [ f ( X )g ( Z )∣ X ] = f ( X )E [g ( Z )∣ X ] (2.31)

2.5. F ORECASTING S TATIONARY T IME S ERIES

2.5.1. I NTRODUCTION TO PREDICTION
We call Fn the information set of series { X t } at time n. The problem of (one step ahead) prediction consists
of finding a function f (Fn ) that is closest to X n +1 .

Suppose that X 1 , X 2 , ... is a sequence of random variables with E [ X i2 ] < ∞ and E [ X i ] = µ. The random variable
f ( X 1 , X 2 , ..., X n ) that minimizes:
E [( X n +1 − f ( X 1 , ..., X n ))2 ] (2.32)
is given by:
f ( X 1 , ..., X n ) = E [ X n +1 ∣Fn ] (2.33)

The best predictor for X n +1 given Fn is given by:

P n X n +1 = E [ X n +1 ∣Fn ] (2.34)

2.5.2. O NE STEP AHEAD PREDICTION FOR AR( P )

If { X t } ~AR(p), then:

X n +1 = ϕ1 X n + ϕ2 X n −1 + ... + ϕp X n −p + Zn +1
p
= ∑ ϕi X n +1−i + Zn +1
(2.35)
i =1

The best predictor is then:

p
P n X n +1 = E [ X n +1 ∣Fn ] = ∑ ϕi X n +1−i (2.36)
i =1

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.5. F ORECASTING S TATIONARY T IME S ERIES 13

Where we assume that { Z t } ~I I D (0, σ2 ). The one step ahead forecast error is then:

e n (1) = X n +1 − P n X n +1 = Zn +1 (2.37)

So we have:
X n +1 − P n X n +1 = Zn +1 (2.38)

If { Z t } ~N (0, σ2 ), a 95% prediction interval is given by:

P n X n +1 − 1.96 ∗ σ ≤ X n +1 ≤ P n X n +1 + 1.96σ (2.39)

2.5.3. T WO STEP AHEAD PREDICTION FOR AR( P )

If { X t } ~AR(p), then:

X n +2 = ϕ1 X n +1 + ϕ2 X n + ... = ϕp X n + 2 − p + Zn +2 (2.40)
The best predictor then becomes:
p
P n X n +2 = E [ X n +2 ∣Fn ] = ϕ1 P n X n +1 + ∑ ϕi X n +2−i (2.41)
i =2

The two-step ahead forecast error is then:

e n (2) = X n +2 − P n X n +2
= ϕ1 ( X n +1 − P n X n +1 ) + Zn +2 (2.42)
= ϕ1 Zn +1 + Zn +2

Note that Var(e n (2)) ≥ Var(e n (1)). This method can be extended to an l-step ahead forecast.

2.5.4. O NE STEP AHEAD PREDICTION FOR MA(1)

If { X t } ~MA(1), then:
X n +2 = Zn +2 + θ1 Zn +1 (2.43)
The best predictor:
P n X n +2 = E [ X n +2 ∣Fn ] = 0 (2.44)
The two step ahead forecast error is:

e n (2) = X n +2 − P n X n +2 = Zn +2 + θ1 Zn +1 (2.45)

2.5.5. T WO STEP AHEAD PREDICTION FOR MA(1)

If { X t } ~MA(1), then:
X n +1 = Zn +1 + θ1 Zn (2.46)
The best predictor:
P n X n +1 = E [ X n +1 ∣Fn ] = θ1 Zn (2.47)
The one step ahead forecast error is:

e n (1) = X n +1 − P n X n +1 = Zn +1 (2.48)

Since we only consider invertible MA processes:

∞
Zn = ∑ π j X n − j (2.49)
j =0

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.6. PARTIAL AUTOCORRELATION F UNCTION (PACF) 14

2.5.6. L STEP AHEAD PREDICTION FOR MA(1)

For l ≤ 2, best predictor:
P n X n +l = E [ X n +l ∣Fn ]
= E [ Zn +l + θ1 Zn +l −1 ∣Fn ] = 0
(2.50)

l-step ahead forecast error:

e n (l ) = X n +l − P n X n +l = Zn +l + θ1 Zn +l −1 (2.51)

2.5.7. I NTRODUCTION TO BEST LINEAR PREDICTOR

Suppose we have a zero-mean stationary time-series { X t } and we wish to predict X n +1 using Fn .
We compute the best linear predictor:
Find the random variable
f ( X 1 , X 2 , ..., X n ) = a 0 + a 1 X n + a 2 X n −1 + ... + a n X 1 (2.52)
That minimizes:
E [( X n +1 − f ( X 1 , X 2 , ..., X n ))2 ] (2.53)

2.5.8. B EST LINEAR PREDICTOR :

We want to predict the value X n +h of a stationary time series with mean 0 and autocovariance function γ(⋅)
in terms of the values {X n , X n −1 , ..., X 1 .
So the goal is to find a 0 , a 1 , ..., a n , such that the linear combination:
P n X n +h = a 0 + a 1 X n + ... + a n X 1 (2.54)
Has the minimum mean squared prediction error:
E [ X n +h − a 0 − a 1 X n − ... − a n X 1 ]2 (2.55)

The minimizing is then done using:

E [ X n +h − a 0 − a 1 X n − ... − a n X 1 ]2 = 0
δ
δa j
⎧
⎪
⎪
⎪E [ X n +h − a 0 − ∑ a i X n +1−i ] if j = 0
n
⎪
⎪
(2.56)
0=⎨ i =1
⎪
⎪
⎪E [( X n +h − ∑ a i X n +1−i ) X n +i − j ] if j = 1, 2, ..., n
n
⎪
⎪
⎩ i =1

Using that E [ X t ] = 0 and that E [ X s X t ] = C ov ( X s , X t ) = γ(s − t ),

one finds that a 0 = 0 and an = (a 1 , ..., a n ) uniquely satisfies:

⎡ γ(0) γ(1) ⋯ γ(n − 1)⎤ ⎡ ⎤ ⎡ ⎤

⎢ ⎥ ⎢ a1 ⎥ ⎢ γ(h ) ⎥
⎢ γ(1) ( ) ⋯ ( − )⎥ ⎢ a ⎥ ⎢ γ(h + 1) ⎥
⎢ ⎥⎢ 2⎥ ⎢ ⎥
⎢ ⎥⎢ ⎥ = ⎢ ⎥
γ 0 γ n 2
⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢γ(n − 1) γ(n − 2) ⋯ ⎥ ⎢an ⎥ ⎢γ(h + n − 1)⎥
(2.57)
⎣ γ( 0 ) ⎦⎣ ⎦ ⎣ ⎦
Γn ⋅ an = γn (h )

If Γn is non-singular, then:
an = Γ−
n ⋅ γn (h )
1
(2.58)

2.6. PARTIAL AUTOCORRELATION F UNCTION (PACF)

For identifying an MA(q)-process, one can look at the ACF, which is zero for lags > q. For an AR(p)-process
however, this approach does not work. The ACF decreases exponentially but it does not help in identifying
the order p of the AR(p)-process. This is where the Partial AutoCorrelation Function (PACF) comes into play.
It can be used to identify the order of the AR(p)-processes.

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

2.6. PARTIAL AUTOCORRELATION F UNCTION (PACF) 15

2.6.1. F ORMAL D EFINITION

The PACF of a stationary process { X t } is the function h ↦ α(h ) for h ≤ 1 defined by α(1) = ρ (1) and:

α(h ) = ρ ( X t +h − PI X t +h , X t − PI X t ) (2.59)

Where
I = Ih,t = {t + 1, ..., t + h − 1} (2.60)

2.6.2. E QUIVALENT D EFINITION

Let { X t } denote a stationary time-series.
Consider the following Best Linear Predictors (BLP):

B LP ( X t ∣ X t −1 ) = ϕ1,1 X t −1
B LP ( X t ∣ X t −1 , X t −2 ) = ϕ2,1 X t −1 + ϕ2,2 X t −2
B LP ( X t ∣ X t −1 , X t −2 , X t −3 ) = ϕ3,1 X t −1 + ϕ3,1 X t −2 + ϕ3,3 X t −1
(2.61)

⋮=⋮

Then:
α(h ) = ϕh,h (2.62)

taking t = 2, 3, ... gives:

B LP ( X 2 ∣ X 1 ) = ϕ1,1 X 1
B LP ( X 3 ∣ X 2 , X 1 ) = ϕ2,1 X 2 + ϕ2,2 X 1
B LP ( X 4 ∣ X 3 , X 2 , X 1 ) = ϕ3,1 X 3 + ϕ3,1 X 2 + ϕ3,3 X 1
(2.63)

⋮=⋮

This implies α(h ) = ϕhh is the coefficient of X 1 in the best linear predictor P 1∶h X h +1 , i.e. ϕhh is uniquely
determined by:
⎡ γ(0) γ(1) ⋯ γ(h − 1)⎤ ⎡ ⎤ ⎡ ⎤
⎢ ⎥ ⎢ ϕh1 ⎥ ⎢ γ(1) ⎥
⎢ γ(1) γ(0) ⋯ γ(h − 2)⎥ ⎢ ϕh2 ⎥ ⎢ γ(2) ⎥
⎥ ⎢ ⎥ ⎢
⎢ ⎥
⎢ ⎥⎢ ⎥=⎢ ⎥
⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
(2.64)
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢γ(h − 1) γ(h − 2) ⋯ γ(0) ⎥ ⎢ ⎥ ⎢ ⎥
⎣ ⎦ ⎣ϕhh ⎦ ⎣γ(h )⎦

2.6.3. D ISTRIBUTION OF SAMPLE PACF

Let { X t } be a causal AR(p)-process with { Z t } ~I I D (0, σ2 ). Then for h>p:
√
n α̂(h ) Ð→ N (0, 1)
D
(2.65)

This result can be used for order selection of AR processes using the following relations:
√
∣α̂(h )∣ > 1.96/ n for0 ≤ h ≤ p
{ √
∣α̂(h )∣ ≤ 1.96/ n forh > p
(2.66)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3. H ANDOUT III

3.1. M ODELING V OLATILITY

3.1.1. F INANCIAL T IME S ERIES
if {P t } is the price of a stock at time t, then the growth rate (simple return) is defined by:
P t − P t −1
Rt = (3.1)
P t −1
If the returns represent a small percentage change, then:

R t ≈ l t = ∇[l og (P t )] (3.2)

Let Ft = σ( X 1 , ..., X t )
denote the information up to and including time t.

1. ARMA modeling is about E [ X t ∣Ft −1 ]

2. GARCH modeling is about V ar ( X t ∣Ft −1 )

Usually, volatility is defined as the Conditional standard-deviation of log-returns.

3.1.2. H EAVY TAILS : K URTOSIS

For a random variable Y with expectation µ, its kurtosis is defined by:

E [(Y − µ)4 ]
κY =
(E [(Y − µ)2 ])2
(3.3)

For Y ~N(0, 1), κY = 3. The excess-kurtosis is defined by κY − 3.

The sample kurtosis of a sample Y1 , Y2 , ..., Yn is defined as:

∑ (Yi − Ȳn )4
n
1

κ̂n =
n
i =1
(3.4)
( n1 ∑ (Yi − Ȳn )2 )2
n

i =1

3.2. ARCH- MODELS

3.2.1. ARCH(1)- MODEL
Let { Z t } ~IID(0, 1). The ARCH(1) model is defined by:

X t = σt Z t
σ2t = ω + αX t2−1
(3.5)

We assume ω, α ≥ 0.
Suppose { Z t } ~N(0, 1), then
X t ∣ X t −1 ~N (0, ω + αX t2−1 ) (3.6)
This conditional distribution has non-constant variance, so it is called Conditionally Heteroscedastic (CH).

Time Series 16 WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3.2. ARCH- MODELS 17

3.2.2. P ROPERTIES OF ARCH(1) MODEL

Suppose { X t } ~ARCH(1), then

X t ∈ Ft

σ2t ∈ Ft −1

Conditional Properties:

1. E [ X t ∣Ft −1 ] = 0

2. V ar ( X t ∣Ft −1 ) = σ2t

Unconditional Properties:

1. E [ X t ] = 0

2. for h > 0, C ov ( X t , X t +h ) = 0

3. V ar ( X t ) = ω + αV ar ( X t −1 )

So with proper initialization, { X t } is a white-noise series.

3.2.3. S QUARED ARCH(1) PROCESS

The squared ARCH(1) process is an AR(1) process.

X t2 − (ω + αX t2−1 ) = σ2t Z t2 − σ2t = Vt

(3.7)
X t2 = ω + αX t2−1 + Vt

It turns out that Vt ~W N (0, σ2 ).

This means that if { X t } ~ARCH(1), then the PACF of { X t2 } is zero after lag 1.

3.2.4. E XISTENCE OF A STATIONARY ARCH(1)- PROCESS

Since { X t2 } ~AR(1), there exists a causal stationary process

X t2 = ω + αX t2−1 + Vt if α ∈ [0, 1) (3.8)

For a stationary ARCH(1)-process:

V ar ( X t ) =
ω
1−α
(3.9)

3.2.5. S UMMARY ARCH(1)

If { X t } ~ARCH(1), then:

• A causal stationary process exists if ω ≥ 0 and α ∈ [0, 1). In that case { X t } ~W N (0, ω/(1 − α))

• The conditional mean is zero, whereas:

V ar ( X t ∣Ft −1 ) = σ2t = ω + αX t2−1 (3.10)

• The squared process { X t2 } is AR(1) with nonzero mean and non-Gaussian noise terms.

X t2 = ω + αX t2−1 + Vt (3.11)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3.2. ARCH- MODELS 18

3.2.6. E STIMATION OF ARCH(1)- PROCESSES

The conditional likelihood:
n
L (ω, α∣x 1 ) = ∏ f ω,α (x t ∣x t −1 ) (3.12)
i =2

If { Z t } ~N(0,1), then:
x t2
f ω,α (x t ∣x t −1 ) = exp(− )
1
2π(ω + αx t2−1 ) 2(ω + αx t2−1 )
(3.13)

Then find (ω̂, α̂) that maximize L. (Note that unless a large sample size is used, this likelihood function tends
to be flat.)

3.2.7. S TANDARDIZED RESIDUALS

Consider the ARCH(1)-model with intercept:

X t = µ + σt Z t σ2t = ω + αX t2−1 (3.14)

With { Z t } ~IID(0,1).
After estimation using maximum likelihood this gives (µ̂, ω̂, α̂).

The residuals are defined by: X t − µ̂.

√
The predicted volatility is defined by: σ̂t = ω̂ + α̂X t2−1

The standardized residuals are defined by: Zˆt =

X t −µ̂
σ̂t

3.2.8. F ORECASTING WITH ARCH(1)

Suppoze { Z t } ~N(0,1), then:
X t +1 ∣Ft ~N (0, σ2t +1 ) = ω + αX t2 ) (3.15)

This means that the forecast is always zero, however, the prediction bands change over time.

h-step ahead forecasting error:

E [ X t +h ∣Ft ] = 0 V ar ( X t +h ∣Ft ) = E [ X t2+h ∣Ft ] (3.16)

E [ X t2+h ∣Ft ] = ω(1 + α + ... + αh −2 ) + αh −1 σ2t +1

for (h → ∞)
ω (3.17)
1−α
→

3.2.9. R ECURSION RELATION FOR PREDICTING ARCH(1)

In general:
E [ X t2+h ∣Ft ] = ω + αE [ X t2+h −1 ∣Ft ] (3.18)
If we then define:
v (h ) ∶= V ar ( X t +h ∣Ft ) (3.19)
then:
v (h ) = ω + αv (h − 1) for h ≥ 2
v (1) = σ2t +1
(3.20)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3.3. GARCH- MODELS 19

3.2.10. K URTOSIS OF STATIONARY ARCH(1)

The kurtosis of a stationary ARCH(1)-process { X t } depends on the kurtosis of the IID-sequence { Z t }:

1 − α2
κX = κZ > κZ
1 − να2
(3.21)

Provided that α2 > 1/ν and ν = E [ Z t4 ]√

If { Z t } ~N(0,1), then ν = 3 and α < 1/ 3

3.2.11. C HOICE FOR IID- SEQUENCE { Z t }

• Standard Normal Distribution

• Standardized student-t, such that V ar ( Z t = 1

√
ν−2
Zt ~ tν for ν > 2
ν

Where t ν denotes a student-t with ν degrees of freedom and density:

Γ((ν + 1)/2) x2
f (x ) = √ (1 + )−(ν+1)/2 for 2 < ν ≤ ∞
Γ(ν/2) ν − 2π ν−2

• Generalized Error Distribution with density:

¿
ν exp(− 21 ∣x /λ∣ν ) Á Γ(1/ν)
for 0 < ν ≤ ∞ and λ = Á
À
λ21+1/λ Γ(1/ν) 22ν Γ(3/ν)

If ν = 2, then this corresponds to N(0,1) and for 0 < ν < 2 this yields heavier tails.

3.2.12. W EAKNESSES OF ARCH MODELS

• The ARCH model assumes that positive and negative shocks have the same effect on the volatility

• The ARCH model provides a way to describe conditional variance. It does not give indications on what
causes such behavior.

• The ARCH model is restrictive, i.e., constraints, such as 3α2 < 1, limit the ability of ARCH models with
conditional Gaussian innovations to capture excess kurtosis.

3.2.13. E XTENSION TO ARCH( M )

Let { Z t } ~IID(0,1). The ARCH(m) model is then defined by:

X t = σt Z t
σ2t = ω + α1 X t2−1 + ... + αm X t2−m
(3.22)

If { X t } ~ARCH(m), then { X t2 } ~AR(m). So the PACF of { X t2 } can be used to determine the order m.

3.3. GARCH- MODELS

Let { Z t } ~IID(0,1). The GARCH(1,1) model is defined by:

X t = σt Z t
σ2t = ω + αX t2−1 + βσ2t −1
(3.23)

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3.3. GARCH- MODELS 20

3.3.1. P ROPERTIES OF GARCH(1,1) MODEL

Suppose { X t } ~GARCH(1,1), then:
X t ∈ Ft

σ2t ∈ Ft −1

Conditional properties:

E [ X t ∣Ft −1 ] = 0 V ar ( X t ∣Ft −1 ) = σ2t (3.24)

Unconditional properties:

1.
E [X t ] = 0

2. for h > 0

C ov ( X t , X t +h ) = 0

3. Suppose α + β < 1. With proper initialization, { X t } is a white noise series with:

V ar ( X t ) =
ω
1−α−β

3.3.2. ARMA(1,1) REPRESENTATION OF GARCH(1,1)

If { X t } ~GARCH(1,1), then { X t2 } admits a non-Gaussian ARMA(1,1) representation:

X t2 = ω + (α + β) X t2−1 + Vt − βVt −1 (3.25)

Where {Vt } is non-Gaussian noise.

proof: From
X t = σt Z t

we get:

X t2 − σ2t = σ2t Z t2 − σ2t = σ2t ( Z t2 − 1) =∶ Vt

So:
β( X t2−1 − σ2t −1 ) = βVt2−1

Subtracting the two previous equations from eachother gives:

X t2 − βX t2−1 − (σ2t − βσ2t −1 ) = Vt − βVt −1

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)

 lOMoARcPSD|7481553

3.3. GARCH- MODELS 21

3.3.3. F URTHER PROPERTIES AND EXTENSIONS OF GARCH(1,1)

• GARCH models also lead to heavy tails

• Many variations of GARCH exist: APARCH, GARCH in mean, I-GARCH,...

• ARMA models can be combined with GARCH models.

Example:

Y t = µ + ϕY t −1 + X t

With:
X t = σt Z t σ2t = ω + αX t2−1 + βσ2t −1

• Modeling multiple time-series is very interesting from a practical point of view as well.

Time Series WI3411TU

Downloaded by Iconic F. (iconicfount1@gmail.com)