Professional Documents
Culture Documents
Hamilton, J. (1994) - Time Series Analysis PDF
Hamilton, J. (1994) - Time Series Analysis PDF
0 in the
solutions (1.2.14) and [1.2.15] for the second-order system, If, furthermore, all of
the eigenvalues are less than 1 in absolute valve, then the system is stable, and its
dynamics are represented as a weighted average of decaying exponentials or de-
caying exponentials oscillating in sign. For example, consider the following second-
order difference equation:
Ye = O6j-1 + 0202 +
From equations (1.2.14) and [1.2.15], the eigeavalues of this system are given by
0.6 + VOT HOD,
Af = 086
2
a = SMO FMOY 4
oe : aa
From (1.2.25), we have
1 = AA, ~ Az) = 0.778
2 = Ally ~ Ay) = 0.222,
‘The dynamic multiplier for this system,
Bey
Fat = ell + eas,
2, pii-Order Di erence Equations 13is plotted as a function of j in panel (a) of Figure 1.4. Note that as j becomes
larger, the patter is dominated by the larger eigenvalue (A,), approximating a
simple geometric decay at rate Ay
If the eigenvalues (the solutions to [J.2.16) are real but atleast one i greater
than unity in absolute value, the system is explosive. If Ay denotes the eigenvalue
that is largest in absolute value, the dynamic multiplier is eventually dominated by
an exponential function of that
Other interesting possiblities arise if some of the eigenvalues are complex.
‘Whenever this is the case, they appear as complex conjugates. For example, if
p = Land 63 + 4G, <0, then the solutions A; and A, in (1.2.14] and (1.2.15] are
complex conjugates. Suppose that A, and A, are complex conjugates, written as
Asati 11.2.3]
Anam bi 1.233]
For the p = 2 case of {1.2.14] and [1.2.15], we would have
a= o2 1234
b= (2)\V== (1.235)
Our goal is to characterize the contribution to the dynamic multiplier
‘when A; is & complex number asin [1.2.32]. Recall tha to raise a complex number
to a power, we rewrite (1.2.32] in polar coordinate form:
Ay = Reoos(o) + ésin(), 1.236)
where @ and R are defined in terms of @ and b by the following equations:
R=VveTe
cos(®) = aR
sin(®) = BIR
Note that R is equal to the modulus of the complex number A,
‘The eigenvalue a; in [1.2.36] ean be written as!
A= Rie“,
and s0
A = Rife] = Rifcos(@) + isin(@p). (1.237)
‘Analogously, if 4 isthe complex conjugate of 4, thea
Az = Ricos(6) ~ ésin(o),
which can be written®
as = Rie“)
‘Thus
AL = Rife] = Recos(@) ~ ésin(@p]- (1.2.38)
a tan parr pn a nuns pci ge ee
‘See eqution(A.3.25] in the Mathematical Review (Appendix A) athe end ofthe Book.
See equnton (A'3.26,
14 Chapter 1 | Difference Equations
10 ne
() 6 = 06,6, = 02
124
ma
toe
©) $= 05,4: = 08
FIGURE 1.4 Dynamic multiplier for second-order difference equation for differ-
ent values of gy and dy (plot of 2y.,/2w, as a function of the lag ).
‘Substituting (1.2.37) and (1.2.38] into [1.2.2] gives the contribution ofthe complex:
conjugates to the dynamic multiplier 3,7:
GM + GAL = e RI feos(a) + ésin(@)] + e4Ri[cos(@) ~ ésin( 4]
es + e,] Rhcos(@p) + Fler ~ es R'sin(@),
‘The appearance of the imaginary number é in [1.2.39] may seem a little
troubling. After all, this calculation was intended to give the effect of a change in
the real-valued variable w, on the real-valued variable y,., as predicted by the real-
valued system [1.2.1], and it would be odd indeed if the correct answer involved
the imaginary number /! Fortunately, it tums out from [1.2.25] that f A, and A;
‘are complex conjugates, then c, and cy are complex conjugates; that is, they can
01.239]
1.2. pth-Order Difference Equations 15be writen at
anatpi
ana-Bi
for some real numbers a and A. Substituting these expressions into [1.2.39] yields
6A, + cok = [Ca # Bi) + (a ~ fi] Rican) + F(a + i) ~ (a ~ A] Risin)
[2a] Ricos(@) + (2B Risin(@)
rR’ cos(@f) - 28R'sin(@j),
Which i striely rel
‘Thus, when some of the eigenvalues are complex, they contribute terms
proportional 1 R’cos() and R’sn(@) tothe dynamic multiplier dy,,.dw, Note
that if R = 1—that is, if the complex eigenvalues have unit modulas—the mul-
tipliers are periodic sine and cosine functions of j. A given increase in w increases
Yen, for some ranges of j and decreases y,,, over other ranges, with the impulse
‘ever dying out at j=. Ifthe complex eigenvalues are less than in modulus
(<1) the impulse again follows a sinusoidal pattern though its amplitude decays
2 the rate R- tthe complex eigenvalues are greater than lin modulus (R > 1),
the amplitude ofthe sinusoids explodes at the rate!
For an example of dynamic behavior characterized by decaying sinusoids,
consider the seconthorder system
= 059-1 — O8).-2 + me
‘The eigenvalues for this system are given from [1.2.14] and (1.2.15}
03 + VOSF= aOR
0.25 + 0.861
with modulus
R= UBF = ORF = 0:
Since R 2- 6,
ing that A, i real, the left side of this expression isa positive number and
the inequality would be satisfied for any value of ¢ > 2. If, on the other hand,
4, <2, we can square both sides to conclude that A, will exceed unity whenever
OF > 4 48, + OF
>i 4
‘Thus, in the real region, A, will be greater than unity either if > 2 oF if (by, ds)
lies northeast ofthe line 6, = 1 ~ 6, ia Figure 1.5. Similarly, with real elgenvaives,
the arithmetically smaller eigenvalue (A,) will be less than 1 whenever
Rove
2
VETS < -2- &
VETTE, > 2 + by
Again, if < —2, this must be satisfied, and in the case when > —2, we can
square both sides:
BEF Mb > 44 4a, + 8
belt by
‘Thus, inthe real region, A, will be less than ~1 if ether $4 < ~2 or (by, 6) les
to the northwest of the line ¢s = 1+ 4, in Figure 1.5.
‘The system is thus stable whenever (¢), ds) lies within the triangular region
of Figure 15.
General Solution of a pth-Order Difference Equation
with Repeated Eigenvalues
In the more general case of a difference equation for which F has repeated
eigenvalues and 5 < p linearly independent eigenvectors, result [1.2.17] i gener-
alized by using the Jorden decomposition,
F = MJM" (1.2.49)
where Misa (p x p) matrix and J takes the form
hoo
je bore
oo 4,
18 Chapeer 1 | Difference Equations
with
410-00
Ox 1 00
00 00
* 2a
O00
Geer 0c
for A, an eigenvalue of F. If [1.2.17] is replaced by (1.2.40), then equation [1.2.19]
generalizes to
Pe MEM! (1.2.9)
where
Hone
oho 8
Estee! : :
ok
“Moreover, fom [1.2.4], if Ji of dimension (n, % n), thea
MAY Qa? Glare
Oa APY Qian
He (1.2.43)
a
where
‘ na =D) 3-B1
o otherwise
Equation (1.2.43] may be verified by induction by multiplying [1.2.41] by [1.2.43]
and noticing that )) + (44) = (33).
For example, consider again the second-order diference equation, this time
‘with repeated roots. Then
ye MI eG eee
O-{
0 that the dynamic multiplier takes the form
Bel a ff = bh + bain
Tong-Run and Present-Value Calculations
If the eigenvalues are all less than 1 in modulus, then F’ in (1.2.9) goes to
‘zero as j becomes large. If all values of w and y are taken to be bounded, we can
"Tis expres it taken fom Chiang (980, . 44),
12. pth-Order Difference Equations 19think of a “solution” of y, in terms of the infinite history of w,
Yee ME Wie + UaWins + amas Hy 1.2.44]
where ¥ i given by the (1, 1) element ofF/ and takes the particular form of [1.2.29]
in the case of distinct eigenvalues.
tis also straightforward to calculate the effect on the present value of y of
1 transitory increase in w. This is simplest to find if we first consider the slightly
more general problem ofthe hypothetical consequences ofa change in any element
of the vector ¥, on any element of €,., in a general system of the form of [1.2.5]
‘The answer to this more general problem can be inferred immediately from [1.2.9}
=F (1.245)
‘The tue dynamic multiplier of interest, 2y,. is just the (1, 1) element of the
(p x p) matrix in [1.245]. The effect on the present value of § of a change ia v
is given by
aD Peay S
a3 a%
ee
is thus the (1, 1) element of the (p x p) matrix in [1.2.46]. This value is given by
the following proposition,
Proposition 13: Ifthe eigenvalues ofthe (p Xp) matrix F defined in 1.2.3] are
all less than in ioduls, then the mats (Ly ~ BE)? exists and the effet of
Ww on the preset value of yi given by ts (I 1) element:
MQ ~ 68 ~ 68 ~ +++ ~ by-sB™" ~ 6,80)
[Note that Proposition 1.3 includes the earlier result for a first-order system
(equation (1.1.14) as a special case.
‘The cumulative effect of a one-time change in w, on Jp, Yreus ~~. can be
considered a special case of Proposition 1.3 with no discounting, Setting = 1 in
Proposition 1.3 shows that, provided the eigenvalues of F are all less than 1 in
‘modulus, the cumulative effcet of « one-time change in w on y is given by
& Sot = ud = b=
om,
7) 12.47)
Notice again that [1.2.47] can alternatively be interpreted as giving the even-
tual long-run effect on y of a permanent change in w:
Mey fos, Weay
tim By Boos g Yoon
eta eee eee
tin Si Sek Sak + Bete Ml - bo b 4).
20° Chapter t'| Difference Equations
APPENDIX 1.A. Proofs of Chapter I Propositions
1 Proof of Proposition 1.1, "The eigenvalues of satisty
IF - al = 0. pad
For the matrix F defined in equation (1.2.3), this determinant would be
en ee
100 0 o| joao--- 00
o 10 0 of-looa---- 00
ooo 1 of looo- oa
G-) & 6 %
1 gh Was}
exert site pair vales fo in equation (L221 rc hat Ti cae
Tr! =1, Ag]
22 Chapter 1 | Difference Equations
where Tis given by [J.A.4] and [1.4.8]. Wetng ou the fst column of the matrix system
Sr equations [1.8.9] explicitly, we have
art art ey pe
apt ae? el fo
ara eo} fo
aoa Meee ae
ae vacate acl ieee
“This gives a system ofp linear equations inthe p unknowns (#1. Brovided
thatthe Aare all tine, the sltion canbe show tobe”
-—___ 1 ____
"GWA AY
1
Bae
@
1
= Ay)
‘Substituting these values into [1,221] gives equation [1.225].
1 Proof Proposition 1.3, Te sca i this proposition shat if he eigenvalues of
ev ess than Brin modulus then te invene of (0p ~ A) es. Suppose the inverse
th, BE) eid oot exis. Thea the determinant, ~ AP would have tobe zee. But
Ul, ~ BF © |=B- OF ~ 8", = (AMF ~ 8
so that ~ “1 would have tbe zero whenever he inverse of (1, ~ BF lf exist
Butts would man tat pv" is.an eigenvalue of F, which ruled Gu by The assumption
fatal eigenvalues of Fare stiles than Bia models. Thus, the atin Ty ~ BP
‘ba be nonsngalar
Since [ly BF]! exis it satisfies the equation
0, - AFI“, - 6] = 1, 1.A10)
Let xy denote the row f, column j element of [ly ~ BE], and write [1.4.20] 2s
iy Ka 22 ae] [= Be Bde oo Bp By
Bata ct te|| “BD Oia
Sa pel et prea) id paw
To.
O10
904
“The tm is then to find the (1) element of ly ~ AF", that sc ind the value
of. To 60 this we need only consider theft row Of equations in (1.A.11)
1 ~ Bi, Bb + Boyer ~ BH)
“Bt outro
i
=f 0+ 0g. (LAs
‘ee Lemma 2 of Chiang (1980.14)
“Appendix 1.A. Proofs of Chapter 1 Propositions 23Consider postmultiplying this system of equations by 2 mate with 1s along the principal
siagonal, inthe fow p, column p'~ I positon, snd Os elsewhere:
10-00
o1--00
00+ pt.
‘The effet ofthis operation is to multiply the pth column of x matrix by and add the result
tothe (p ~ 1h column:
1 By ~B, ~~ Bly Be, ~ 86,
“pt ° o
fis tar ony] & Jan 0-0 9
o 6 ° i
Next mutiny the (p= 1)th colin by and ad the cea othe (p ~ 2th clu,
Prceating hs tion we crave a” cane
Bu xe 0 ay) x
eG 0+ Oo. [LAs
‘The first equation in (1.A.13] states that
2a (= B= BR
= Br) = 3
2a = MC ~ By ~ Bo =~ Bd),
5 claimed in Proposition 13.
Chapter T References
Chiang, Cin Lone, 1980. An Inroduction wo Stchate Proce and Thr Applications.
untnglon, NY Kreger.
Goat, Stephen M_I9TR “The Demand for Money Revised,” Brookings Paper on
Economic Activity 3:577-638. i Papen
Sargent, Thomas 3.1987. Macroeconomic Theory, 248. Beson:Asadeic Pres
24 Chapter I | Difference Equations
Lag Operators
2.1. Introduction
‘The previous chapter analyzed the dynamics of linear difference equations using.
tmatrix algebra. This chapter develops some of the same results using time series
‘operators. We begin with some introductory remarks on some useful time series
operators
‘A time series is a collection of observations indexed by the date of each
‘observation, Usually we have collected data beginning at some particular date (say,
1 = 1) and ending at another (say, t = 7):
Our 9d
We often imagine that we could have obtained earlier observations (yo, ¥-1,
Yan. 4) oF later observations (Yr4s; Yran - --) had the process been observed
{or more time. The observed sample ().. i+.» « Yr) could then be viewed as a
finite segment of a doubly infinite sequence, denoted {y)
Cyne Lee Yeadon Ye Yao Pe Pret dreae ooh
obierved sample
‘Typically, a time series (yt... is identified by describing the th element.
For example, a time wend isa series whose value at date 1s simply the date of the
observation:
yet
We could also consider a time series in which each element is equal to a constant
«regardless of the date of the observation f:
yne.
Another important time series is a Gaussian white noise process, denoted
y
where (eJf.» isa sequence of independent random variables each of which has
8N(, 0°) distribution.
‘We are used to thinking ofa function such as y = f(x) ory = gC, w) a8 an
‘operation that accepts as input a number (x) or group of aumbers (t, ») and
‘roduces the output (3). A time series operator transforms one time series or group
25of time series into a new time series. It accepts as input a sequence such as
{)F--» OF a group of sequences such a8 (Lx) —m w)f=-») and has as output 2
new sequence {y)7.-». Again, the operator is summarized by describing the value
‘of a typical clement of {y)7.—» in terms of the corresponding elements of
tri»
‘An example ofa time series operator is the multiplication operator, repre-
sented as
2 Bir pay
Although ts wetten exactly the same way as simple scalar multiplication, equation
{2.11} i acolly shorthand for an infinite sequence of multiplications, one for
cach date The operator multiplies the value takes on at any date by some
constant to generate the val of y for that dat.
‘Another example of a time series operator is the addition operator:
yam tm
Here the value of y st any date #is the sum ofthe values that x and w tke on for
that date,
Since the mukiplication or addition operators amount oelementby-element
sltipliction or addition, they obey all he standard rales of algebra For example,
if we multiply each observation of {xJ7.-. by 6 and each observation of
{w)f—= by B and add the results,
Bx, + Bey
the outcome is the same as if we had first added {x}Z_. to {w,?
‘multiplied each element of the resulting series by 8:
Als, + w)
‘A highly useful operator is the lg operator. Suppose that we start with a
sequence {x)/_-., and generate a new sequence {y,)7_ - ., Where the value of y for
date ris equal the value x took on at date — 1
Ye = Kray: 1.2)
“This i described applying the lag operator to (rf. -» The operation isrepe-
sented by the symbol Ls
and then
Lars on 13)
Consider the result of applying the lag operator twice to a series:
LL) = Lis) = Fx
Such a double application of the lag operator is indicated by “L?™
Be, = a
In general, for any integer k,
Di, = oe pas]
Notice that if we first apply the multiplication operator and then the lag
operator, asin
21 BS, Bins
the result will be exactly the seme as if we had applied the lag operator first and
then the multiplication operator:
Hien Bs,
26 Chapter2 | Lag Operators
‘Thus the lag operator and multiplication operator are commutative:
L(Bx) = BL,
Similarly, if we frst add two series and then apply the lag operator tothe result,
Gn We + ms tae
the result the same as if we had applied the lag operator before adding:
GW) > reas Mad > Heer + Mie
“Thus, the lag operator is distributive over the addition operator:
Le, + wi) = Lee + Lie
‘We thus see thatthe lag operator follows exactly the same algebraic rules as
the multiplication operator. For this reason, itis tempting to use the expression
“multiply y, by L” rather than “operate on {y}7~—» by L." Although the latter
expression is technically more correct, this text will often se the former shorthand
expression to facilitate the exposition.
‘Faced with atime series defined in terms of compound operators, we are free
to use the standard commutative, associative, and distributive algebraic laws for
multiplication and addition to express the compound operator in an alternative
form. For example, the process defined by
Ym (@ + BLL,
is exactly the same as
= (AL + BLA, = a, + Br
To take another example,
(1 = ADO = AD = = AL aL + AL,
(By AL + Ava), LS}
Ont Ader + Odden
‘An expression such as (al. + BL) is referred to as a polynomial inthe lag
operator. Iti algebraically similar to a simple polynomial (az + bz?) where 2 is,
4 scalar. The difference is that the simple polynomial (az + 62%) refers to a
particular number, whereas a polynomial inthe lag operator (al. + 612) refers 10
fan operator that would be applied to one time series (xJ%.-» to produce a new
time series {y)F= =.
Notice that if 7.» is just a series of constants,
for allt,
then the lag operator applied to x, produces the same series of constants
Lr nase.
Thus, for example,
(aL + BL? + ye (a + B+ y)6. e146
2, First-Order Difference Equations
Let ws now return to the first-order difference equation analyzed in Section 1.1
Ye = Oye + Me [2.2.1]
2.2, First-Order Difference Equations 27Equation {2.2.1} canbe rewrten using the lag operator [2.1.3] a8
Ym OLy, + we
‘This equation, in turn, can be rearranged using standard algebra,
Yim OLy, = my
(= 6Dy, = 222)
Next consider “multiplying” both sides of [2.2.2] by the following operator:
(FOL FPR 4 OL ++ + 61), R23)
“The result would be
(L4 OL + PLE + OL ++ + GLY ~ Ly,
QO + PEs PL +--+ grqw, 2241
Expanding out the compound operator on the lft side of [2.2.4] results in
(14 6h + GL + PD +++ + LY - OL)
HO 4OL 4 PEF OL + gL)
= (FOL FPL EOD ++ SLNOL 25}
HUFOLE GE LOD ++ gL)
(OLE SLE PD +o FOL OLY)
=a gL,
Substituting [22.5] ito [2.2.4] yields
(= PLY, = OLE PLE PD 4 + LIM, (226)
Writing [2.2.6] out explicitly using (2.1.4) produces
A Og Mt Ohar + Phen + Pha Hee Pine
EO Amt Oia + Pig + Mas ++ Ge. (227)
Novice that equation (2.2.7 sential to eqution [1.1.7]. Applying the
ee eee ee ee rere
were employed in the previous chapter to arive a [1.1 7)
Tei imeresting to reflect onthe naar of he opertor (2.2.3) as becomes
large: We saw in 22.5} hat
(1+ OL + GLP + BL H+ + GLA - By, =», - Vy
That is, (1+ OL + GL? + GL) + +++ + GLI — @L)y, differs from y,
tern jeg <1 andy isa tnt sun ted 99°,
will become negligible ax ¢ becomes large:
(LOL + FL + PLP ++ + PLY Ly, =y, for large,
A sequence {y)7.—. is said to be bounded if there exists a finite number F such
that
ld <¥ for alle,
‘Thus, when || <1 and when we are considering applying an operator toa bounded.
sequence, we can think of
(4 ol + OL + OL +--+ OL)
28 Chapter 2 | Lag Operators
ss approximating the inverse of the operator (I~ 6L), with this approximation
made arbitrarily accurate by choosing j suficiently large:
(= GL)! = fim + OL + PLE BL H+ HL). (2.28)
This operator (1 — #L)~ has the property
(1 - 61)" ~ 41)
where “I denotes the identity operator:
y= ye
‘The following chapter discusses stochastic sequences rather than the deter-
ministic sequences studied here. There we will speak of mean square convergence
and stationary stochastic processes in place of limits of bounded deterministic
sequences, though the practical meaning of {2.2.8} will be little changed.
Provided that [6 < 1 and we restrict ourselves to bounded sequences or
stationary stochastic processes, both sides of [2.2.2] can be “divided” by (I~ #L)
to obtain
y= aL
aH Os + Par + Pts + 229)
Itshould be emphasized that if we were not restricted to considering bounded
sequences or stationary stochastic processes (wT. = and {7 -» then expression
[2.2.9] would not bea necessary implication of (22 1]. Equation [2.29] isconsstent
with (2.2.1), but adding term agg’,
= OB 1m, + das + Pan + Phas tees (22.10)
produces another series consistent wit [2.2.1] for any constant a, To verity that
{22.10} is consistent with [22.1], multiply [2.2.10] by (L ~ 62):
(1 = 6byy, = 1 = ob + (1 ~ @L)(1 ~ 80),
all ~ 08" +,
so that [2.2.10] is consistent with [2.2.1] for any constant a,
“Although any process ofthe form of [2.2.10] is consistent withthe difference
equation [2.2.1], notice that since [a <1,
i
‘Thus, even f{wjf. isa bounded sequence, the solution {y)7—~ given by [2.2.10]
js unbounded uniess a, = 0 in (2.210. Thus, there was a particular reason for
defining the operator [2.2.8] tobe the iaverse of (1 ~ 4)—namely, (1 ~ #L)~*
defined in (2.2.8) i the unique operator satisfying
(1 ety ~ 61) 1
that maps a bounded sequence {w Jf» into a bounded sequence (y f=»
“The aature of (I~ 9L)~! whes [gl = 1 wil be discussed in Section 2.5.
2.3. Second-Order Difference Equations
Consider next a second-order difference equation:
Yee bier + bayea + Me p34]
2.3. Second-Order Difference Equations 29Rewriting this in lag operator form produces
(= bb ~ bob? \y, = 23.2)
‘The left side of [2.3.2] contains a second-order polynomial in the lag operator
L. Suppose we factor this polynomial, that is, find numbers A, and A such that
(L= $0 = dL) = (= AL) AL) = (= Bay + AJL + AVAL). (23.3)
‘This i just the operation in [2.1.5] in reverse. Given values for , and dy, we seek
numbers A, and A; with the properties that
Ath
and
Aa = bs
For example, if; = 0.6 and ¢, = —0.08, then we should choose A, = 0.4 and
Ay = 02:
(1 = 062 + 0.0824) = (1 ~ 0.42)(1 ~ 0.21), R34)
It is easy enough to see that these values of A and A, work for this numerical
example, but how are A, and A; found in general? The tabk is to choose Ay and Az
0 as to make sure that the operator on the right side of [2.3.3] is identical to that
on the left side. This will be true whenever the following represent the identical
functions of 2:
(1 ~ $z ~ 6229) = (1 ~ Ave) ~ Anz): P35)
This equation simply replaces the lag operator L in [2.3.3] with a scalar 2, What
is the point of doing so? With [2.3.5], we can now ask, For what values of 2 i the
right side of [2.3.5] equal to zero? The answer is, if either 2 = Ay! or z = Az},
then the right side of (23.5] would be zero. It would not have made sease to ask
an analogous question of [2.3.3]—L denotes a particular operator, not a number,
and L = Aj" is not a sensible statement
‘Why should we care thatthe right side of [2.3.5] is zero if 2 = Ay? or if.
Az? Recall that the goal was to choose A, and A; so that the two sides of [2.3.5]
represented the identical polynomial in z. This means that for any particular value
2 the two functions must produce the same number. If we find a value of z that
Sets the right side to zero, that same value of z must set the left side to zero as
well. But the values of z that set the left side to zero,
(= a2 ~ by) 0, 236
ae given by the quadratic formula:
4S Es pam
b+ VTE
cere Dag eee 23.8)
Setting z = z, or z makes the left side of [2.3.5] zer0, while
Ag" sets the right side of [2.3.5] t0 zero. Thus
agteg 2.39)
Apher 2310]
30° Chapter2 | Lag Operators
|
|
|
Returning to the numerical example (2.3.4] in which # = 0.6 and gy = ~0.08,
‘we would calculate
0.6 = VOSF = HOR)
Bu 2(0.08) ns
06 + YOKE = HOB
8
and so
ues) = 04
5.0) = 02,
as was found in (23.4)
‘When 1 + 4¢; <0, the values 2, and 2, are complex conjugates, and their
reciprocals Ay and A can be found by first writing the complex number in polar
coordinate form, Specifically, write
sath
2, = Ri[cos(6) + ésin(@)] = Re
Rober = R-fc09(0) ~ ésin()}
‘Actually, there is a more direct method for calculating the values of A, and.
A; from 4, and ds. Divide both sides of [2.3.5] by 2*:
(7 ~ bet ~ 4) = TAYE ~ AD) (23.11)
and define A to be the variable =!
a (23.12)
‘Substituting [2.3.12] into [2.3.11] produces
Q? = dA ~ 6) = @— ANA ~ A). (2.3.13)
‘Again, 2.3.13] must hold for all values of A in order for the two sides of [2.3.5]
to represent the same polynomial. The values of A that set the right side t0 zero
areA = Ayand A ™ A, These same values must set the left side of [2.3.13] 0 zero
8 well
i (2 = ba - 6) #0. (23.14)
‘Thus, to calculate the values of Ay and A; that factor the polynomial in (2.3.3), we
can find the roots of [2.3.14] dectiy from the quadratic formuls
open Be ve nt (23.15)
1 AVE esas
For the example of [2.3.4], we would thus calculate
6 + UBF = HOB
05+ VOF= TOD). 4,
VOR = MOR)
= 02.
2.3. Second-Order Difference Equations 31.cis instructive to compare these results with those in Chapter 1. There the
dynamics of the second-order difference equation (2.3.1] were summarized by
calculating the eigenvalues of the matrix F given by
[* ‘4 (23.17)
The eigenvalues of F were seen to be the two values of A that satisfy equation
(1213)
OF ~ GA ~ by) = 0.
‘But this i the same calculation as in (2.3.14). This finding is summarized in the
following proposition,
Proposition 2.1: Factoring the polynomial (1 ~ @L ~ dsL3) as
(1 @L ~ bb) = (1 ~ AD) - AL) (23.18)
4s the same calculation as finding the eigenvalues of the mairix F in (2.3.17). The
eigenvalues h, and dz of ¥ are the same as the parameters hand Az in [2.3.18], and
are given by equations (2.3.15) and [2.3.16]
The correspondence between calculating the eigenvalues of a matrix and
factoring a polynomial inthe lg operators very instructive, However, introduces
‘one minor source of posible semantic confusion about which we have tobe care
Recall from Chapter I thatthe system 2.31] i stable if both A, and Ay ae less
than 1 in modulus and explosive if either A, or Ay is greater than 1 in modulus,
Sometimes ths is described as the equirement tha the roots of
OF 6A 3.19]
lic inside the unit circle. The posible confusion i that it soften convenient to
work dtetl withthe polynomial inthe form in which i appears in (2.3.2),
(1 62 - 62 =0, 2329
whose rots, we have seen, are the reciprocals of those of 2.3.19]. Ths, we could
say with equal accuracy that “the difference equation {2.3.1} is stable whenever
the roots of [2.3.19] lie inside the unit cirle” or that “he difference equation
(23:1) isstable whenever the roots of [2.3.20] lie ouside the uit crle.” The two
‘atements mean exactly the same thing. Some scholars refer simply tothe “roots
of the difference equation [2.3.1]," though this raises the possiblity of confusion
between [23.19] and (2.3.20), This book will fallow the convention of using the
term “eigenvalues” to refer to the roots of [2.3.19], Wherever the term “roots is
used, we will indicate explcily the equation whose roots are being described
‘From here on inthis section, it assumed thatthe second-order dtterence
equation is stable, with te elgenvalues A, and Ay distinct and both inside the unit
titel. Where this is the case, the inverses
(LAD AIL EARL? AL +
(1= Aghyt = 14 AL +L + aD +
‘ce wel defined for bounded sequences. Write [2.3.2 a factored form:
(1 = ALI ~ Asbiy, =
and operate on both ses by (L'~ AL)=#(1 ~ AgL)7>
= (= ALINE ~ AL), p32
32 Chapter2 | Lag Operators
Following Sargent (1987, p. 184), when A, + As, we can use the following operator:
fr (eae eerie
ana fa 4g} 23.22)
[Notice that this is simply another way of writing the operator in [2.3.21}
{rx
(= a”
ie {as = AL) = A(t = pe}
“OAD = ey
‘Thus, [2.3.21] can be writen as
naa ayf
- { REAL FALE ALE]
Be kb ate 4802 + dpe
Y= les + cam, + (6s + elias + AE + call
+ [ead + elma toe (23.23)
where
= AMA, = a) (2.3.24)
2 = —Aal(Ay — da). [2.3.25]
From [2.3.23] the dynamic multiplier can be read off credy as
Bet mod + adie
the same result arrived atin equations [1.2.24] and [1.2.25].
Ta, pih-Order Difference Equations
‘These techniques generalize in a straightforward way to a pth-order difference
‘equation of the form
Y= Dior + Palin H+ Oy Inap + Me 4a]
Write (2.4.1 in terms of lag operators as
(1 ~ iL ~ dL? — +++ ~ 4L¢, 42]
Factor the operator on the lft side of [2.4.2] a8
(= ib = dal? ~~ 6,17) = (L- ALYL~ AL) = AgL). (2.4.3)
‘This is the same as finding the values of (Aas « «Ap such that the following
polynomials ate the same for all z:
(1 = ye — ye? =o = yah) = (1 = Aus = Aas) = 242)
24. oth-Order Difference Equations 33nd
‘As in the second-order system, we multiply both sides ofthis equation by z
define A= 2:
(ar = gat ~ dar — A - @)
SO =AIA =a) = AD
Clearly, setting A = A, for i= 1,2, ..., orp causes the right side of [2.4.4] to
equal zero. Thus the values (Ay, As,» 4) must be the numbers that set the lft
side of expression [2.4.4] to zero as wel
Ay = bE = SAP EA = bpd = 4 0. 245)
‘This expression again is identical to that given in Proposition 1.1, which charac-
terized the eigenvalues (41,2, - - , A) ofthe matrix F defined in equation [1.2.3
‘Thus, Proposition 2.1 readily generalizes.
(244)
Proposition 2.2: Factoring a pth-order polynomial in the log operator,
(1 = OL ~ Gab? — «++ — pL?) = (1 - ALY Aa) «+ (1 = AL),
{5 the same calculation as finding the eigenvalues ofthe matris F defined in [1.23].
The eigenvalues (As, Az, - Ag) of Fare the same as the parameters (hay - +
Ay) in [2.4.3] and are given By the solutions to equation (2.4.5)
The difference equation (2.4.1]isstableif the eigenvalues (the roots of [2.4.5])
le inside the unit citele, or equivalently if the roots of
1m br = 2? = = 6
le outside the unit circle
Assuming that the eigenvalues are inside the unit citcle and that we are
restricting ourselves to considering bounded sequences, the inverses (1 ~ AL)",
(1= AD),..., (1 = ApL)-* all exist, permitting the difference equation
(ADO = Abs AD,
24.6)
to be written as
Yee (= ALA = ALE (= ALI, 47)
Provided further that the eigenvalues (A,, Aa, 2,) are all distinct, the poly-
nomial associated with the operator on the tight side of [2.4.7] can again be ex-
panded with partial fractions
1
OA = ha) T=)
2.48)
(Cea acer haa ere;
Following Sargent (1987, pp. 192-93), the values of (6.x. that make (2.48)
true can be found by multiplying both sides by (L~ Az) — 238) + (1 ~ Aya):
1m c(h ~ Ass)(1 aye) = aya)
$+ ex = Avz)(L = Age) ++ (= Aya) + 249]
4 Gl — Az) = Az) + (= Apa.
Equation [2.4.9] has to hold forall values of z Since itis a (p ~ 1)th-order
polynomial, if (Ci, a, «+ 5G) are chosen so thet [2.49] holds for p particular
34° Chapter 2 | Lag Operators
distinct values of 2, then [2.6.9] must hold forall z. To ensure that [2.4.9] holds
requires that
T= 6(l = ATI = Aa) = ar)
Age!
= AY A ae
For [24.9] to hold for = A531, requires
o" Oe Pant]
ae 4.2)
OA, = AI Oy = Bed
Note again that these are identical to expression 1.2.25] in Chapter 1. Recall from
the discussion there that cy +c # <9 +G = 1
To conclude, [2.4.7] can be written
“Tia “TaD”
ex(l + AL + AL? + ALP + Jw beg(L + AGL + ARLP + ARL? ++,
Heeb GL EASL + ABLE + AL? + Dw,
Yale tet oes + lw + [oh + Gah Ht OADM
FLAT + GAR 8 + GARD [2.4.13]
FUGA + CAL + + GAs +
where (Ci, cos -- - 6) are given by equations [2.4.10] through [2.4.12]. Again,
the dynamic multiplet canbe read dee off [2.4.13
Seat = [aM + aah + + Gah, (24.14)
reproducing the ceslt from Chapter
‘There is a very convenient way to calculate the effect of w on the present
value of y using the lag operator representation. Write [2.4.13] as
Yom VO bien, Ma Uahig te RAS)
where
= TeM + dt + oa 24.16)
[Next rewrite [2.415] in lag operator notation as
n= Wm eas7)
‘here o(L) denotes an infinite order polynomial inthe lag operator:
WL) = Wo FAL + Ul? + WAL? +
Pra ery eee re |Notice that ¥, is the dynamic multiplier (2.4.14). The effect of w, on the present
value of y is given by
24.18}
Thinking of Y(2) a «polynomial ina real number 2,
OE) = Wet et aaah + Hae Eo
it appears that he mutpr (2.4.18 i simply this polynomial evluted at z = 6
a3 Bias
ime
‘But comparing [2.4.17] with [2.47] it
= UB) = Wot B+ B+ WB + =>. [24.19]
apparent that
WL) = [(l ~ ALY ~ AL) (La),
som 243 enn ha
HL) = [= OL = dyLt LA
We conde hat
2) 2 Uh = 2 = bet = be
for ny ve oan pur,
WB) = (1 = by ~ dB? — +++ - 06° [2.4.20]
suming 24) 48] ve hat
ad fed
te aap Raa
reproducing the claim in Proposition 1.3. Again, the long-run multiplier obtains
as the special case of [2.4.21] with B = 1
Beay 4 oop Bens
Pees,
20, * Bes
2.5. Initial Conditions and Unbounded Sequences
Section 1.2 analyzed the fllowing problem. Given a pth-order difference equation
Y= bye + Oar H+ May + Mee 5.)
initial values of y,
Yous Yor soYors psa
and @ sequence of values for the input variable w,
{oy ie ge 253)
36 Chapter 2 | Lag Operators
we sought to calculate the sequence of values for the output variable y
{900 Ys «od
Certainly there are systems where the question is posed in precisely this form. We
may know the equation of motion forthe system [2.51] and its eurrent state [2.5.2]
and wish to characterize the values that {y, 1... - 31} might take on for different
specifications off, Wy» Wi
However, there ate maay examples in economics and finance in which
theory specifies just the equation of motion [2.5.1] and a sequence of driving
variables [2.5.3]. Clearly, these two pieces of information slone are insufficient to
determine the sequence (yo, y.,-.. .yJ,and some additional theory beyond that
contained in the difference equation [25.1] is needed to describe fully the de-
endence of y on w. These additional restrictions can be of interest in their own
ight and also help give some insight into some of the technical details of mani
lating diference equations. For these reasons, this section discusses in some depth
an example of the role of initial conditions and their implications for solving dit-
ference equations
Let P, denote the price ofa stock and D, its dividend payment. If an investor
buys the stock at date «and sells it at ¢ + 1, the investor will ean a yield of
D{P, om the dividend and a yield of (P,, ~ P)/P, in capital gains. The investor's
total return (7) thus
Foes = (Pras ~ PYP, + DAP,
‘A very simple model ofthe stock market posts that the return investors earn on
stocks is constant across time periods
1 (Poa ~ PIP, + DIP, 7 >0. 25.4)
Equation [2.5.4] may seem too simplistic to be of much practical interest; it
assumes among other things that investors have perfect foresight about future stock
prices and dividends. However, a slightly more realistic model in which expected
stock returns are constant involves a very similar set of technical issues. The ad>
‘vantage ofthe perfectforesight model (2.54) is that it can be discussed using the
tools already in hand to gain some futher insight into using lag operators to solve
difference equations.
Mulipy (2.5.4 by P, to arsive at
TP, = Pras B+ Dy
Pay = (1 + OP, ~ De 25.5]
Equation 2.55] willbe recognized asa first-order difference equation ofthe form
of [lel] With yy = Pays @ = (1+ Phy and w, = —D, From (1.1.7), we know
that (2°55) implies that
Pras = (14 FP) (14 HYDy (1+ 'D,—(L*Ds .5,6)
Hee (1+ Dia = Dy
I the sequence {Dp, Dy, ..., D}-and the value of Pp were given, then [2.5.6]
‘could determine the vales of (Py, Pa... Pas)- Butt only the values (Dy, Ds,
1D} ate given, then equation [25.6] would not be enough to pin down {P,,
Py... Bish There are an infinite number of possible sequences (P,P, .-.
Piss) consistent with [2.5.5] and with a given (Dy, Dy,» D). This infinite
umber of possibilities is indexed by the inital value Py
ple epee ee, eee ey EEE‘A further simpli
paths for (Py, Pay +P,
‘assumption helps clarify the nature of these different
ni}: Suppose that dividends are constant over time:
D,=D foralle
Then [2.5.6] becomes
Para (Lt nmr [dete
toed t ye D
is erp, 12 Gt et Bom
= nn - OD
= (1+ ORs = Din} + Din.
Consider first the solution in which Py = Dir. Ifthe initial stock price should
happen to take this value, then 2.5.7] implies that
P= Dkr psa]
forall, In this solution, dividends are constant at D and the stock price is constant
at Dir. With no change in stock prices, investors never have any capital gains or
losses, and their return is solely the dividend yield D/P = r, In a world with no
changes in dividends this seems to be a sensible expression ofthe theory represent
by (2.5.4). Equation [2.5.8] is sometimes described as the “market fundamentals
solution to (2.5.4) for the case of constant dividends
“However, even with constant dividends, equation [2.5.8] snot the only result
consistent with [2.5.4]. Suppose that the inital price exceeded Dir
Py > Dir
Investors seem to be valuing the stock beyond the potential ofits constant dividend
stream. From (2.5.7] this could be consistent with the asset pricing theory (2.5.4]
provided that P, exceeds Dir by an even larger amount. AS long as investors all
believe that prices will continue to rise over time, each will earn the required return
from the realized capital gain and [2.5.4] will be satisfied. This scenario has
reminded many economists of a speculative bubble in stock prices.
If.such bubbles are to be ruled out, additional knowledge about the process
for {P)t._..isrequired beyond that contained inthe theory of (2.5.4). For example,
we might argue that finite world resources put an upper limit on feasible ttock
prices, as in
\Pl O and
this operator is not defined. In this ease, a lag operator representation can be
sought forthe recursive substitution forward that led from (2.5.5] to (2.5.13}. This
is accomplished using the inverse of the lag operato
Lot, = Mets
which extends result [2.14] to negative values of k. Note that L:
inverse of the operator L:
LL) = Las =
In general,
LAL = L-+,
with L° defined as the identity operator:
Lon We
40 Cheaper? | Lag Operators
‘Now consider multiplying [2.5.15] by
hain ae Ce pa
to obtain
(ee be Le QPL
XM 4NLP
SMF tmnt se ete
$F OME xD
-a4nE BP. = [EP + [-U.:
+ fap ee FLT
which is identical to [2.5.13] with cin [2.5.13] replaced with ¢ + 1.
‘When r > Oand (Pr. -. ia bounded sequence, the left side ofthe preceding
‘equation will approach P,. as T becomes large. Thus, when r > 0 and (P)7
and (DJF. -« are bounded sequences, the limit of the operator in [2.5.17] ¢
and could be viewed as the inverse ofthe operator on the left side of [2.5.15]:
= (4+ nip = -G 4 ne
KEE EDL + Lt LEH
Applying ths limiting operator to [2.5.15] amounts to solving the difference equa-
tion forward asin [2.5.14] and selecting the market fundamentals solution among,
the set of possible time paths for {P);_ given a particular time path for dividends
(Dit.
“Thus, given a first-order difference equation of the form
(= Oly, = Hy (2.5.18)
Sargent’s (1987) advice was to solve the equation “backward” when |] <1 by
sulipying by
Wei aM sors ett eE sy (25.19)
and to solve the equation “forward” when |4| > 1 by multiplying by
ot!
ee (25.20)
es
Defining the inverse of {1 - L] in this way amounts to selecting an operator
[1 6b} with the properties that
{1 = 61)" x [1 = 62] = 1 (the identity operator)
and that, when its applied to a bounded sequence {w)f=-»,
et ec
the result is another bounded sequence.
‘The conclusion from this discussion is that in applying an operator such as
(1 ~ 62)", we are implicitly imposing a boundedness assumption that rules out
Sc samnbdd:EGAAOLAAL-UiAde SAAUUULMAGIEE MIADLGAMULAE = ETphenomena such as the speculative bubbles of equation (2.5.7] a priori. Where
that is our intention, so much the better, though we should not apply the rules
{2.5.19} of [25.20] without some reflection on their economic content.
Chapter 2 References
Sargent, Thomas J, 1987. Macroeconomic Theory, 24 ed. Boston: Academic Press
Whiteman, Charl H, 1988. Linear Rational Expectations Models: A User's Guide. Mi
seapalis: University of Minnesota Pres,
42° Chapter 2 | Lag Operators
Stationary
ARMA Processes
‘This chapter introduces univariate ARMA processes, which provide a very useful
class of models for describing the dynamics ofan individual time series, The chapter
begins with definitions of some of the key concepts used in time series analysis.
Sections 3.2 through 3.5 then investigate the properties of various 4 RMA processes.
Section 3.6 introduces the autocovariance-generating function, which is useful for
analyzing the consequences of combining different time series and for an under-
standing of the population spectrum. The chapter concludes with a discussion of
invertblity (Section 3.7), which can be important for selecting the ARMA rep-
resentation of an observed time series that is appropriate given the uses to be made
of the model,
31. Expectations, Stationarity, and Ergodicity
Expectations and Stochastic Processes
Suppose we have observed a sample of size T of some random variable ¥,
Whe Yar eo Ieh Bat]
For example, consider a collection of T independent and identically distributed
(iid) variables,
Ce Baz
with
e.~ NO, 2%)
This is referred to as a sample of size T from a Gaussian white noise process.
‘The observed sample [3.1.1] represents T particular numbers, but this set of
TT numbers is only one possible outcome of the underlying stochastic process that
generated the data. Indeed, even f we were to imagine having observed the process
for an infinite period of time, arriving atthe sequence
0)
the infinite sequence {yf would stil be viewed as a single realization from a
time series process. For example, we might set one computer to work generating
an infinite sequence of i..d. N(0, 0%) variates, {e/9)7.-., and a second computer
generating @ separate sequence, {e{}... We would then view these as two
independent realizations of a Gaussian White noise process.
Te Pet Yor Mae Yas ee Yan Irae Peano
43Imagine a battery of J such computers generating sequences {{"I7. =,
OPH Ot.) and consider selecting the observation associated with
date 1 from each sequence:
LPP
‘This would be described as a sample of J realizations of the random variable ¥,
‘This random variable has some density, denoted fy(y,), which is called the un-
conditional density of ¥,. For example, for the Gaussian white noise process, this
density is given by
cue 3
00 = ee en 32]
‘The expectation of the tth observation of atime series refers to the mean of
this probability distribution, provided it exists:
20%) = [trod &, B13]
‘We might view this asthe probability int of the ensemble average:
£(%) = plim cu & v9 Bad
For example, if (¥.)2
poise process {¢,)
represents the sum of a constants plus « Gaussian white
Y= wt 6 B15}
chen its mean is
EY) = 1 + Ble) =m Bag
ICY, isa time tend plus Gaussian white noise,
¥, = pt +e, (3.4.7)
then its mean is
E(Y) = pe B18)
Sometimes for emphasis the expectation E(¥,) is ealled the unconditional
‘mean of ¥,, The unconditional mean is denoted yy:
BY) = my
Note that this notation allows the general possibilty thatthe mean can be a function
ofthe date of the observation «. For the process [3.1.7] involving the time trend,
the mean (3.1.8) is a function of time, whereas for the constant plus Gaussian white
noise, the mean [3.1.6] is not a function of time.
‘The variance of the random variable ¥, (denoted 7, is similarly defined as
aoe EW, =m)? = [0 = md Fuld by, 1g)
Sb: iiaaaiead aad REAL Pada
For example, for the process (3.1.7), the variance is
Ye = EY, — BoP = ECC)
‘Autocovariance
Given a particular realization such as {y}!}7... on a time series process,
‘consider constructing a vector x!" associated with date ¢. This vector consists of
the [j + 1] most recent observations on y as of date ¢ for that realization:
- a8 generating one paricular value of the
vector x, and want to calculate the probability distribution ofthis vector xf? across
realizations i. This distibuton is called the joint disozbution of (Yn Views +»
Y,..). From this distribution we can calculate the jth autocovariance of Y, (denoted
»)
we [fe fo Ge= m0 = wed
% fi terontiYo Yonrs HD a ear dy,» [3.1.10]
EC BN med)
Note that [2.1.10] has the form ofa covariance between two vaiables X and Y:
Cov(X, ¥) = E(X - wx)(¥ ~ py).
Thus [3.1.10] could be desrbed asthe covariance of ¥ with its own lagged value:
tence, the tem “autocovarance." Notice farther toms (3110] that the Olt at
tocovadane is atthe variance of Y, a8 atipated By the notation min [3199,
"The autocovaraace yycan be viewed ast (1,j +1) element ofthe varianoe-
covaance matt of the Vectors, For lib reason the sulocovarances ee 6e
seribed asthe second moments of ie proves for Y
“Again it ay be helpful to think of the ih autocovasince asthe pobebilty
Unit fen cael overage:
Ye = pli ane (YP = wd 42) ~ bah Bin]
As an example of calculating autocovariances, note that for the process in
[2.1.5] the autocovariances are all zero for j + 0:
mh EM, = Ws) = Eee) = 0 for] #0.
Stationarity
If neither the mean ja nor the autocovariances 7, depend on the date 1, then
the process for ¥, is said to be covariancesstationary o weakly stationary:
proce
E(Y) = for all:
EY, w(K = w) = for all cand any
3.1, Expectations, Stationarity, and Ergodicity 45For example, the process in [3.1.5] is covariance-stationary
E(Y) = w
BUY, ~ why ~ 1) = ¢ es
By contrast, the process of [3.1.7] is not covariance-stationary, because its mean,
Bit, isa function of time.
Notice that if a process is covariance-stationary, the covariance between Y,
and ¥,-, depends only on j, the length of time separating the observations, and
not on é, the date of the observation. It follows that for a covariance-stationary
process, 7, and _, would represent the same magnitude. To see this, reall the
efinition
EY, ~ WN ~ 0) (3.4.12)
If the process is covariance-stationary, them this magnitude is the same for any
value of ¢ we might have chosen; for example, we can replace # with ¢ + j
= Eres — MMesiiy =H) = BM = WY =) = EOF, 1% —
But refercing again tothe definition [3.1.12], this last expression is just the definition
of y.;. Thus, for any covariance-stationary process,
y= yr) forall integers. 1.3]
A different concept i that of strict stationarity. A process is said to be strictly
stationary if, for any Values of jy, ja.» » jy the joint distribution of (¥,, Yon.
Youn «=» Yig,) depends only on the intervals separating the dates (j., js,»
4j)'and not on the date itself (0). Notice that if a process is strictly stationary with
finite second moments, then it must be covariance-stationary—if the densities over
which we are integrating in [3.1.3] and [3.1.10] do aot depend on time, then the
‘moments 4, and y, will not depend on time. However, itis possible to imagine a
process that is covariance-stationary but not strictly staonary; the mean and au-
‘ocovariances could not be functions of time, but perhaps higher moments such as
EY) are,
In this text the term “stationary” by itself is taken to mean “covariance-
stationary.
‘A process {¥;} is said to be Gaussian if the joint density
Pr deonon stron Ser Yens © Yesid)
fs Gaussian for any ji, ja, .-- » jy Since the mean and variance are all that are
needed to parameterize a multivariate Gausian dstibution completely, a covariance-
stationary Gaussian process is strictly stationary.
Ergodicity
We have viewed expectations ofa time series in terms of ensemble averages
such as [3.1.4] and [3.1.11]. These definitions may seem a bit contrived, since
usually all one has available i a single realization of size T from the process, which
we earlier denoted {y{, yf, . .. , y9}- From these observations we would cal
culate the sample mean J. This, of course, is not an ensemble average but rather
a time average:
yearn Sop. Basa
46 Chapter3 | Stationary ARMA Processes
‘Whether time averages suchas [3.1.14] eventually converge to the ensemble concept
(Y) for a stationary process has to do with ergodicity. A covariance stationary
process is said to be ergodic for the mean if [3.1.14] converges in probability t0
BLY) as T— =." A process will be ergodic for the mean provided thatthe auto-
covariance 7, goes to zero sufficiently quickly as j becomes large. In Chapter 7 we
will se that if the autocovariances for a covariance-stationary process satisfy
Siie=, 115]
then {¥) is ergodic for the mean.
Similarly, ieestationary process is said to be ergodic for second
moments if
(ur -a 3,0 = ws wy
forall. Sufficient conditions for second-moment ergodicity will be presented in
Chapter 7. Inthe special case where (Y} is a stationary Gaussian process, condition
[B.1.15] is sufficient to ensure ergodicity for all moments.
For many applications, stationarity and ergodicity turn out to amount to the
same requirements. For purposes of clarifying the concepts of stationarity and
ergodicity, however, it may be helpful to consider an example of a process that is
stationary but not ergodic. Suppose the mean i for the ith realization
{y(n is generated from a N(0, M) distribution, say
Y= +e, (1.16)
Here (e) is 8 Gaussian white noise process with mean zero and variance o* that
is independent of ji. Notice that
by = E(u) + Ble) = 0.
Also,
Ye = Bul + 6h = at oF
and
a = EO + 6M + 6) =a for) #0.
‘Thus the process of [3.1.16] is eovariance-stationary. It does not satisy the sufficient
condition [3.1.15] for ergodicity for the mean, however, and indeed, the time average
an 3 v9 = am Fw +e) = 0 +0 Be
‘converges to n rather than to 2er0, the mean of ¥,
3.2. White Noise
‘Thebasic building block forall the processes considered in this chapter isasequence
{eJfs-x whose elements have mean zero and variance o?,
G24)
8.23)
and for which the e
‘Omen “ergodicity” ic ued ia a more general ease see Anderson and Moore (197%, p. 319) oF
Henan (1970 pp 201-2.
3.2. White Noise 47Ele) = 0 fore ts, B23)
A process satisfying (3.2.1] through [3.2.3] is described as a whive noise proces.
‘We shall on occasion wish to replace [3.2.3] with the slightly stronger condition
that the e's are independent across time:
é €, independent for ¢ # +. B24}
Notice that 3.2.4] implies [3.2.3] but [3.2.3] does not imply [3.2.4]. A process
satisfying (3.2.1) Unrough [3.2.4] is called an independent white noise process.
Finally, if [3.2.1] through {3.2.4} hold along with
4 ~ NO. 2°), B23)
then we have the Gaussian white noise process.
333. Moving Average Processes
The First-Order Moving Average Process
Let {¢} be white noise asin [3.2.1] through [3.2.3], and consider the process
Yaw t et benny Ba.
Where wand @ could be any constants. This time series is called a first-order moving
‘average process, denoted MA(1). The term “moving average” comes from the fact
that ¥, is constructed from a weighted sum, akin to an average, of the two most
recent values of &.
"The expectation of ¥, is given by
E(Y) = Eu + 6, + 66,1) = w+ Ele) + OE (6,1) = [332]
‘We used the symbol ufo the constant term in [3.3.1] in anticipation of the result
that this constant term turas out to be the mean of the process.
The variance of ¥, is
EY, -
Ble, + 86.3)
Ele + 20Q,-1 + Pet») B33]
40+ 0o*
(1+ ee
“The fist autocovaiance is
CY, — wXYi-1 — w) = Ele, + 8€,-s)(e.-1 + 6€,-2)
Eletins + 062 + 86a + Pest) (334)
04 bot 4040.
Higher autocovatiaces ae all zero
ECL, = Whey 1) = Ele + Be Meiey# eras) =O for />1. B35}
Since the mean and autocovariances are not functions of time, an MA(1) process
is covariance-stationary regardless of the value of 6, Furthermore, [3.1.15]is clearly
satisfied
Spa
white noise, then the MA(1) process [3.3.1] is ergodic for
1 + Bo? + [or
‘Thus, if (6) is Gaussi
all moments.
48 Chapter 3 | Stationary ARMA Processes
‘The jth autocorrelation of a covariance-stationary process (denoted p) is de-
fined as its th autocovariance divided by the variance:
py ah B36
‘Again the terminology arises from the fact that pis the correlation between Y,
and ¥,
Cour, i
Gort Yd = rae) Want)” Vie Vin ~
Since p isa correlation, jp = 1 forall, by the Cauchy-Schwarz inequality. Notice
also thatthe Oth autocorrelation p, is equal to unity for any covariance-stationary
process by definition.
From (3.3.3) and [3.3.4], the frst autocorrelation for an MA(1) process is
tiven by
o_o
+ Ae T+
Higher autocorrelations are all ero.
“The autocorrelation ,can be plotted as a function of /as in Figure 3.1. Panel
(@) shows the autocorrelation function for white noise, while panel (b) gives the
autocorrelation function for the MA(1) process:
Y= 64 086-1.
For different specifications of # we would obtain different values for the first
autocorrelation pin (3.3.7). Positive values of @ induce positive autocorrelation
in the series. In this ease, an unusually large value of Y, i likely to be followed
by a larger-than-average value fr ¥,, jut a a smaller-than-average ¥, may well
be followed by a smaller-than-average ¥,... By contrast, negative values of @ imply
negative autocorrelation—a large ¥, might be expected to be followed by a small
value for ¥,.
‘The values for p, implied by different specifications of @ are plotted in Figure
3.2, Notice that the largest possible value for pis 0.5; this occurs if @ = 1. The
‘smallest value for pis ~0.5, which occurs if @ = ~1. For any value of p, between
=0.5 and 0:5, there are two different values of 6 that could produce that auto-
cortelation, This is because the value of i(1 + 62) is unchanged if 8 is replaced
by 1/8:
a 3.7)
(uy 62-00)
PTs OF” FILS WF]
For example, the processes
Ym e+ 0.56,-1
and
e+ ear
would have the same autocorrelation function:
2 0.
MTs +055
o..
We will have more to say about the relation between two MA(1) processes that
share the same autocorrelation function in Sestion 3.7
5.3, Moving Average Processes 49(2) White noise: ¥, = 2,
(b) MAQ): ¥, = 6, + 0.8e,-5
. oa : ae oe 2
(©) MAG): ¥,= 6 = 2661+ 036-2 (8) ARM): Y= 08%... + 6
= olse2) + 086,
o
(©) ARG): ¥, = -0.8¥.4 + 6
FIGURE 3.1 Autocorrelation functions for assorted ARMA processes.
The qth-Order Moving Average Process
A qih-order moving average process, denoted MA(g), is characterized by
Yom at e+ Biter + Batica t+ Bing B38]
‘where (6) satisfies [3.2.1] through [3.2.3] and (8,
numbers. The mean of [3.3.8] is again given by p:
Gy + 48) could be any real
E(Y,) = w+ Ele.) + 6°E (6-1) + BE (e,-2) + + GEC,
‘The variance of an MA(q) process is
We EU, = WP = Ele Brea t Bina t+ Opin? BBS]
50 Chapter3 | Stationary ARMA Processes
a
FIGURE 3.2. The first autocorrelation (p,) for an MA(1) process possible for
different values of 6.
Since the e's are uncorrelated, the variance [3.3.9] is
4 Olot + Oho? Hoe + Oho (LF RHA + OBo® (8.3.10)
Beet
ABU + Bera + Baa Ong)
% (61a Banja t Bafnpen t+ Beicpeal Ba.)
= BLO + Ga Oya + OaaBaehajaa t+ lef)
Terms involving e's at diferent dates have been dropped because their product
has expectation zeto, and Oy is defined to be unity. For j > q, there are no e's with
common dates in the definition of 7, and so the expectation is zero. Thus,
1, + Gash + Ooh +--+ 6408,-Jre? forfe 12....g
"Lo forj>
ai Boil
For example, for an MA(2) process,
welt + apo
n= lh + elo?
n= (elo?
wanes 0.
For any values of (8, 6s -- » 9), the MA(@) process is thus covariance-
stationary. Condition [3.1.15] is satisfied, so for Gaussian ¢, the MA(q) process is
also ergodic forall moments, The autocorrelation function is zero after q lags, as
in pane! (c) of Figure 3.1.
The Infinite-Order Moving Average Process
"The MA(q) process can be written
x,
wt 3 Ge.)
“ser equine [AS in Aen A heed oft os
3.3. Moving Average Processes. 51ih 6 1 Comider the proses hat result as ¢—+
Yen Stes we tees en tates te. BAS)
This could be described as an MA(s) process. To preserve notational flexibility
later, we will use ¥’s forthe coefficients ofan infinite-order moving average process
1nd @'s for the coefficients of a finte-order moving average process
‘Appendix 3.A to this chapter shows that the infinite sequence in [3.3.13]
generates a well defined covariance-stationary process provided that
Bw<* 33.14)
Ie is often convenient to work with a slightly stronger condition than [3.3.14
Sic 1315)
A sequence of numbers {y}7=0 satisfying [3.3.14] is suid to be square summable,
whereas a sequence satisfying [3.3.15] is said to be absolutely summable. Absolute
summability implies squaresummability, but the converse does not hold—there
are examples of square-summablé sequences that are not absolutely summable
(again, see Appendix 3.A).
‘The mean and autocovariances of an MA(e) process with absolutely sum
rable coetficents can be calculated from a simple extrapolation of the results for
an MA(g) process:?
BCX) = fim Ela + dons + dieu Yate to + Ursin) 13316)
=
EY, ~ a
= fim Et, + Vibe + Matra to ee 3a
slim + Wee eae?
y= EY, = wy —
% ( ws) ~ 3.18)
= Oy) + Ypte + Yost + Hats +9)
Noreoer, an MA) poet wth absolutely sunmablcoetcienshas soli
Spice pay]
Hence, an MA(e) proces satsying [33.15 is ergoic forthe mean (ee Appendix
eee
2Alaale oma jn ad exinene of he eond mame (ate fen condos
wo peri inecuepng he ero meson ad vaunonSpaaiele WO eee
‘of random variables such that ie os
Me
Bind