Introduction to Time Series Analysis. Lecture 1.

Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series modelling: Chasing stationarity.

Organizational Issues
Peter Bartlett. bartlett@stat. Ofce hours: Thu 1:30-2:30 (Evans 399). Fri 3-4 (Soda 527). Brad Luen. bradluen@stat. Ofce hours: Tue/Wed 2-3pm (Room TBA). http://www.stat.berkeley.edu/bartlett/courses/153-fall2005/ Check it for announcements, assignments, slides, ... Text: Time Series Analysis and its Applications, Shumway and Stoffer.

Organizational Issues
Computer Labs: Wed 121 and Wed 23, in 342 Evans. You need to choose one of these times. Please email bradluen@stat with your preference. First computer lab sections are on September 7. Classroom Lab Section: section is on September 2. Fri 121, in 330 Evans. First classroom lab

Assessment: Lab/Homework Assignments (40%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab). The last assignment will involve analysis of a data set that you choose. Midterm Exam (25%): scheduled for October 20, at the lecture. Final Exam (35%): scheduled for Thursday, December 15.
3

A Time Series
SP500: 19601990 400

350

300

250

200

150

100

50

0 1960

1965

1970

1975 year

1980

1985

1990

A Time Series
SP500: JanJun 1987 340

320

300

280

260

240

220 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

A Time Series
SP500 JanJun 1987. Histogram 30

25

20

15

10

0 240

250

260

270 $

280

290

300

310

A Time Series
SP500: JanJun 1987. Permuted. 340

320

300

280

260

240

220

20

40

60

80

100

120

Objectives of Time Series Analysis

1. Compact description of data. 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation.

Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.
(Makridakis, Wheelwright and Hyndman, 1998)
x 10
4

12

10

10

20

30

40

50

60

70

80

90

Transformed data
12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

10

Trend
12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

11

Residuals
1.5

0.5

0.5

10

20

30

40

50

60

70

80

90

12

Trend and seasonal variation

12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

13

Objectives of Time Series Analysis

1. Compact description of data. Example: Classical decomposition: 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation.

X t = Tt + St + Yt . Example: Seasonal adjustment. Example: Predict sales.

14

Unemployment data
Monthly number of unemployed people in Australia.
8 x 10
5

(Hipel and McLeod, 1994)

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

15

Trend
8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

16

Trend plus seasonal variation

8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

17

Residuals
8 x 10
4

6 1983

1984

1985

1986

1987

1988

1989

1990

18

Predictions based on a (simulated) variable

8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

19

Objectives of Time Series Analysis

1. Compact description of data: Xt = Tt + St + f (Yt ) + Wt . 2. Interpretation. 3. Forecasting. 4. Control. Example: Seasonal adjustment. Example: Predict unemployment. Example: Impact of monetary policy on unemployment. Example: Global warming.

5. Hypothesis testing. 6. Simulation.

Example: Estimate probability of catastrophic events.

20

Overview of the Course

1. Time series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods 3. Spectral analysis 4. State space models(?)

21

Overview of the Course

1. Time series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models 3. Spectral analysis 4. State space models(?)

22

Overview of the Course

1. Time series models 2. Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?)

23

Overview of the Course

1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) (a) ARMAX models. (b) Forecasting, Kalman lter. (c) Parameter estimation.

24

Time Series Models

A time series model species the joint distribution of the sequence {Xt } of random variables. For example: P [X1 x1 , . . . , Xt xt ] for all t and x1 , . . . , xt . Notation: X1 , X2 , . . . is a stochastic process. x1 , x2 , . . . is a single realization. Well mostly restrict our attention to second-order properties only: EXt , E(Xt1 Xt2 ).

25

Time Series Models

Example: White noise: Xt W N (0, 2 ). i.e., {Xt } uncorrelated, EXt = 0, VarXt = 2 . Example: i.i.d. noise: {Xt } independent and identically distributed. P [X1 x1 , . . . , Xt xt ] = P [X1 x1 ] P [Xt xt ]. Not interesting for forecasting: P [Xt xt |X1 , . . . , Xt1 ] = P [Xt xt ].

26

Gaussian white noise

1 P [Xt xt ] = (xt ) = 2
2.5 2

xt

x2 /2

dx.

1.5

0.5

0.5

1.5

2.5

10

15

20

25

30

35

40

45

50

27

Gaussian white noise

2.5

1.5

0.5

0.5

1.5

2.5

10

15

20

25

30

35

40

45

50

28

Time Series Models

Example: Binary i.i.d.
1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 35 40 45 50

P [Xt = 1] = P [Xt = 1] = 1/2.

29

Random walk
St =
t i=1

Xi .
8

Differences:

St = St St1 = Xt .

10

15

20

25

30

35

40

45

50

30

Random walk
ESt ? VarSt ?
10

10

15

10

15

20

25

30

35

40

45

50

31

Random Walk
Recall S&P500 data.
SP500: JanJun 1987 340

(Notice that its smooth)

320

300

280

260

240

220 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

32

Random Walk
Differences:
SP500, JanJun 1987. first differences 10

St = St St1 = Xt .
8

10 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

33

Trend and Seasonal Models

Xt = T t + S t + E t = 0 + 1 t +
6

(i cos(i t) + i sin(i t)) + Et

5.5

4.5

3.5

2.5

50

100

150

200

250

34

Trend and Seasonal Models

Xt = T t + E t = 0 + 1 t + E t
6

5.5

4.5

3.5

2.5

50

100

150

200

250

35

Trend and Seasonal Models

Xt = T t + S t + E t = 0 + 1 t +
6

(i cos(i t) + i sin(i t)) + Et

5.5

4.5

3.5

2.5

50

100

150

200

250

36

Trend and Seasonal Models: Residuals

0.5

0.4

0.3

0.2

0.1

0.1

0.2

0.3

0.4

0.5

50

100

150

200

250

37

Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt , St . (b) Differencing. (c) Nonlinear transformations (log, 3. Fit model to residuals. ).

38

Nonlinear transformations
Recall: Monthly sales.
12 x 10
4

(Makridakis, Wheelwright and Hyndman, 1998)

12

11.5
10

11

10.5
8

10

9.5

9
4

8.5

8
2

7.5

10

20

30

40

50

60

70

80

90

10

20

30

40

50

60

70

80

90

39

Differencing
Recall: S&P 500 data.
SP500: JanJun 1987 340 10 SP500, JanJun 1987. first differences 8 320 6

4 300 2

280

$ 1987.05 1987.1 1987.15 1987.2 1987.25 year 1987.3 1987.35 1987.4 1987.45 1987.5

2 260 4

6 240 8

220 1987

10 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

40

Differencing and Trend

Dene the lag-1 difference operator,
(think rst derivative)

Xt = Xt Xt1 = (1 B)Xt , where B is the backshift operator, BXt = Xt1 . If Xt = 0 + 1 t + Yt , then X t = 1 + If Xt =

k i=0

Yt .

i ti + Yt , then
k

Xt = k!k +
1

Yt , Xt .

where

Xt =

k1

Xt ) and
41

Xt =

Differencing and Seasonal Variation

Dene the lag-s difference operator,
s Xt

= Xt Xts = (1 B s )Xt ,

where B s is the backshift operator applied s times, B s Xt = B(B s1 Xt ) and B 1 Xt = BXt . If Xt = Tt + St + Yt , and St has period s (that is, St = Sts for all t), then
s Xt

= Tt Tts +

s Yt .

42

Least Squares Regression

Model: X t = 0 + 1 t + W t 0 + Wt , = 1 t 1 1 1 W1 W2 1 2 0 + = . . . . . 1 . . . . 1 T WT
Z w

X 1 X2 . . . XT
x

43

Least Squares Regression

x = Z + w. Least squares: choose to minimize w Solution satises the normal equations:

2

= x Z

= 2Z (x Z ) = 0.

If Z Z is nonsingular, the solution is unique: = (Z Z)1 Z x.

44

Least Squares Regression

Properties of the least squares solution ( = (Z Z)1 Z x): Linear. Unbiased. For {Wt } i.i.d., it is the linear unbiased estimator with smallest variance. Other regressors Z: polynomial, trigonometric functions, piecewise polynomial (splines), etc.

45

Outline
1. Objectives of time series analysis. Examples. 2. Overview of the course. 3. Time series models. 4. Time series modelling: Chasing stationarity.

46