ACF and PACF of an AR(p)

We will only present the general ideas on how to obtain

the ACF and PACF of an AR(p) model since the details
follow closely the AR(1) and AR(2) cases presented
before.
Recall that AR(p) model is given by the equation
X
t
=
1
X
t1
+
2
X
t2
+. . . +
p
X
tp
+
t
For the ACF, rst we multiply by X
tk
both side of the
autoregressive model equation to obtain,
X
tk
X
t
=
1
X
tk
X
t1
+. . . +
p
X
tk
X
tp
+
t
X
tk
90
By taking expectation at both sides this equation, we get

k
=
1

k1
+
2

k2
+. . . +
p

kp
or written in lag-operator
(B)
k
= 0
with (B) = 1
1
B
2
B
2
. . .
p
B
p
The implied set of equations for dierent values of
k = 1, 2, . . . , are known as Yule-Walker equations.
We try again a solution of the form
k
=
k
. which leads
to the equation

p1

p2
. . .
p
= 0
The solutions to this equation are the reciprocal roots
91

1
,
2
, . . . ,
p
of the characteristic polynomial (B)
The general solution for the dierence equation is

k
=
p

i=1
A
i
(
i
)
k
= A
1
(
1
)
k
+A
2
(
2
)
k
+. . . A
p
(
p
)
k

k
has an exponential behavior and cyclical patterns
(damped sine wave) may appear if some of the

j
s are
complex numbers
Theorem. Given a general dierence equation of the
form C(B)Z
t
= 0 where
C(B) = 1 +C
1
B +C
2
B
2
+. . . C
n
B
n
and
C(B) =

n
i=1
(1 R
i
B) so the R

i
s are the reciprocal
roots of the equation C(B) = 0, we have that the solution
is Z
t
=

n
i=1
A
i
R
t
i
(without proof).
92
For the PACF we can apply Cramers rule for
k = 1, . . . , p which can gives us an expression for P
kk
.
If k > p, then P
kk
= 0 so the PACF of an AR(p) must cut
down to zero after lag k = p, where p is the order of the
AR model.
ACF and PACF for Moving Average models
Lets start with the MA(1) given the equation
X
t
=
t
+
t1
with the model parameter and
t
N(0,
2
)
Lets nd an expression for the ACF,
k
For lag k = 0

0
= V ar(X
t
) = V ar(
t
) +
2
V ar(
t1
) =
2
(1 +
2
)
93
For lag 1, consider the product X
t
X
t1
. Using the MA(1)
equation, we obtain
X
t
X
t1
= (
t
+
t1
)(
t1
+
t2
)
=
t

t1
+
t1
2
+
t2

t
+
2

t1

t2
If we take expected value at both sides of the equation,
E(X
t
X
t1
) =
1
=
2
Additionally for any value of k
X
t
X
tk
=
t

tk
+
t

tk1
+
t1

tk
+
2

t1

tk1
Then, if k > 1, E(X
t
X
tk
) =
k
= 0
94
The ACF of an MA(1) is given by

k
=

/(1 +
2
); k = 1
0 k > 1
Using
k
= 0 for k > 1,we can show that the PACF
P
kk
=
kk
=

k
(1
2
)
1
2(k+1)
k 1
Contrary to its ACF, which cuts o after lag 1, the
PACF of an MA(1) model decays exponentially.
For a general MA(q) process, the ACF cuts down to
zero after lag q and the PACF will have exponential
behavior depending on the characteristic roots of
(B) = (1 +
1
B +
2
B
2
. . . +
q
B
q
) = 0.
95
Instead of trying to nd equations in the general case, we
will look at somes examples using simulation.
Firstly, we consider an MA(1) process with parameters
= 0.9 and = 0.9.
Then, we consider an MA(2) process with parameters
(
1
= 0.85,
2
= 0.5) and with parameters
(
1
= 0.85,
2
= 0.5)
Finally, we consider an MA(4) process with parameters
(
1
= 0.9,
2
= 0.8,
3
= 0.75,
4
= 0.4)
Again, we are using this function arima.sim
xt=arima.sim(1000,model=list(ma=0.9))
96
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

2
0
2
4
0 10 20 30 40 50
0
.
0
0
.
4
0
.
8
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
2
0
.
0
0
.
2
0
.
4
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
MA(1) process with alpha =.9, sigma^2=1
97
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

2
0
2
4
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
1
.
0
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
5

0
.
3

0
.
1
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
MA(1) process with alpha=.9, sigma^2=1
98
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

2
0
2
4
0 10 20 30 40 50
0
.
0
0
.
4
0
.
8
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
2
0
.
2
0
.
6
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
MA(2) process with Theta1= .85 and Theta2=.5
99
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

2
0
2
4
0 10 20 30 40 50

0
.
2
0
.
2
0
.
6
1
.
0
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
3

0
.
1
0
.
1
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
MA(2) process with Theta1=.85 and Theta2=.5
100
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

4
0
2
4
6
0 10 20 30 40 50

0
.
2
0
.
2
0
.
6
1
.
0
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
2
0
.
0
0
.
1
0
.
2
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
MA(4) process with Theta1=.9, Theta2=.8, Theta3=.75,Theta4=.4
101
To obtain the ACF and PACF for an ARMA(p,q)
process, we need to follow the same strategy used to
obtain the ACFs and PACFs for AR and MA models.
Recall that an ARMA(p,q) process is dened by the
equation,
X
t
=
1
X
t1
+. . . +
p
X
tp
+
t
+
1

t1
+. . . +
q

tq
As before, if we multiply by X
tk
both sides of the
equation and take expected value, we obtain

k
=
1

k1
+ . . . +
p

kp
+ E(X
tk

t
)
1
E(X
tk

t1
)
. . .
q
E(X
tk

tq
)
Since
E(X
tk

ti
) = 0; k > i
102
we obtain that

k
=
1

k1
+ . . . +
p

kp
; k q + 1
We know (divide by
0
) that this equivalent to

k
=
1

k1
+ . . . +
p

kp
; k q + 1
which gives the Yule-Walker equations but with the restriction
k q + 1.
For a lag k q + 1, the autocorrelation function of an
ARMA(p,q) process has a similar behavior to the ACF of a
pure AR(p) process.
However, the rst q autocorrelations
1
,
2
,...
q
depend on both
autoregressive and moving average parameters.
The PACF for an ARMA (p,q) is complicated and usually not
needed.
103
This PACF will have a similar behavior as the PACF of a
MA(q) process.
Lets look at some examples for simulated data of an
ARMA(1,1) processes.
The examples consider 1000 simulations. The AR coecient is
0.95 (0.6) and MA coecient is 0.5.
We will also consider an ARMA(2,1) process where the AR
part is built with r = 0.95 = 0.42 and the MA parameter is
= 0.7.
Finally, we will show an ARMA(10,2) process where AR part
is deened with 10 complex reciprocal roots.
104
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

1
5

5
0
5
1
0
0 10 20 30 40 50
0
.
0
0
.
4
0
.
8
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
4
0
.
0
0
.
4
0
.
8
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
ARMA(1,1) process with Phi=.95, Theta=.5
105
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

2
0
2
4
0 10 20 30 40 50
0
.
0
0
.
4
0
.
8
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
4
0
.
0
0
.
4
0
.
8
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
ARMA(1,1) process with Phi=.6, Theta=.5
106
Time
a
s
.
t
s
(
x
t
)
0 200 400 600 800 1000

3
0

1
0
1
0
3
0
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
1
.
0
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
ARMA(2,1) process
107
Time
a
s
.
t
s
(
x
t
)
0 100 200 300 400

4
0
2
4
0 10 20 30 40 50

0
.
5
0
.
0
0
.
5
1
.
0
Lag
A
C
F
Series xt
0 10 20 30 40 50

0
.
4
0
.
0
0
.
4
Lag
P
a
r
t
i
a
l

A
C
F
Series xt
ARMA(10,2) process
108