Introduction To Time Series Analysis. Lecture 1

Introduction to Time Series Analysis. Lecture 1.
Peter Bartlett
1. Organizational issues.
2. Objectives of time series analysis. Examples.
3. Overview of the course.
4. Time series models.
5. Time series modelling: Chasing stationarity.
Organizational Issues
Peter Bartlett. bartlett@stat. Office hours: Thu 1:30-2:30 (Evans 399).
Fri 3-4 (Soda 527).
Brad Luen. bradluen@stat. Office hours: Tue/Wed 2-3pm (Room
TBA).
http://www.stat.berkeley.edu/bartlett/courses/153-fall2005/
Check it for announcements, assignments, slides, ...
Text: Time Series Analysis and its Applications, Shumway and Stoffer.
Organizational Issues
Computer Labs:
Wed 121 and Wed 23, in 342 Evans.
You need to choose one of these times. Please email bradluen@stat with
your preference. First computer lab sections are on September 7.
Classroom Lab Section:
section is on September 2.
Fri 121, in 330 Evans. First classroom lab
Assessment:
Lab/Homework Assignments (40%): posted on the website.
These involve a mix of pen-and-paper and computer exercises. You may use
any programming language you choose (R, Splus, Matlab). The last
assignment will involve analysis of a data set that you choose.
Midterm Exam (25%): scheduled for October 20, at the lecture.
Final Exam (35%): scheduled for Thursday, December 15.
3
A Time Series
SP500: 19601990
400
350
300
250
200
150
100
50
0
1960
1965
1970
1975
year
1980
1985
1990
A Time Series
SP500: JanJun 1987
340
320
300
280
260
240
220
1987
1987.05
1987.1
1987.15
1987.2
1987.25
year
1987.3
1987.35
1987.4
1987.45
1987.5
A Time Series
SP500 JanJun 1987. Histogram
30
25
20
15
10
0
240
250
260
270
280
$
290
300
310
A Time Series
SP500: JanJun 1987. Permuted.
340
320
300
280
260
240
220
20
40
60
80
100
120
Objectives of Time Series Analysis

1. Compact description of data.
2. Interpretation.
3. Forecasting.
4. Control.
5. Hypothesis testing.
6. Simulation.
Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.
(Makridakis, Wheelwright and Hyndman, 1998)
4
12
x 10
10
10
20
30
40
50
60
70
80
90
Transformed data
12
11.5
11
10.5
10
9.5
8.5
7.5
10
20
30
40
50
10
60
70
80
90
Trend
12
11.5
11
10.5
10
9.5
8.5
7.5
10
20
30
40
50
11
60
70
80
90
Residuals
1.5
0.5
0.5
10
20
30
40
50
12
60
70
80
90
Trend and seasonal variation

12
11.5
11
10.5
10
9.5
8.5
7.5
10
20
30
40
50
13
60
70
80
90

1. Compact description of data.
Example: Classical decomposition:
2. Interpretation.
X t = Tt + St + Yt .
Example: Seasonal adjustment.
3. Forecasting.
Example: Predict sales.
4. Control.
6. Simulation.
14
Unemployment data
Monthly number of unemployed people in Australia.
(Hipel and McLeod, 1994)
x 10
7.5
6.5
5.5
4.5
4
1983
1984
1985
1986
1987
15
1988
1989
1990
Trend
5
x 10
7.5
6.5
5.5
4.5
4
1983
1984
1985
1986
1987
16
1988
1989
1990
Trend plus seasonal variation

5
x 10
7.5
6.5
5.5
4.5
4
1983
1984
1985
1986
1987
17
1988
1989
1990
Residuals
4
x 10
6
1983
1984
1985
1986
1987
18
1988
1989
1990
Predictions based on a (simulated) variable

5
x 10
7.5
6.5
5.5
4.5
4
1983
1984
1985
1986
1987
19
1988
1989
1990

1. Compact description of data:
Xt = Tt + St + f (Yt ) + Wt .
2. Interpretation.
Example: Seasonal adjustment.
3. Forecasting.
4. Control.
Example: Predict unemployment.

Example: Impact of monetary policy on unemployment.
6. Simulation.
Example: Global warming.
Example: Estimate probability of catastrophic events.
20
Overview of the Course

1. Time series models
(a) Stationarity.
(b) Autocorrelation function.
(c) Transforming to stationarity.
2. Time domain methods
3. Spectral analysis
4. State space models(?)
21

(a) AR/MA/ARMA models.
(b) ACF and partial autocorrelation function.
(c) Forecasting
(d) Parameter estimation
(e) ARIMA models/seasonal ARIMA models
22

(a) Spectral density
(b) Periodogram
(c) Spectral estimation
23

(a) ARMAX models.
(b) Forecasting, Kalman filter.
(c) Parameter estimation.
24
Time Series Models

A time series model specifies the joint distribution of the sequence {Xt } of random variables.
For example:
P [X1 x1 , . . . , Xt xt ] for all t and x1 , . . . , xt .
Notation:
X1 , X2 , . . . is a stochastic process.
x1 , x2 , . . . is a single realization.
Well mostly restrict our attention to second-order properties only:
EXt , E(Xt1 Xt2 ).
25
Time Series Models

Example: White noise: Xt W N (0, 2 ).
i.e., {Xt } uncorrelated, EXt = 0, VarXt = 2 .
Example: i.i.d. noise: {Xt } independent and identically distributed.
P [X1 x1 , . . . , Xt xt ] = P [X1 x1 ] P [Xt xt ].
Not interesting for forecasting:
P [Xt xt |X1 , . . . , Xt1 ] = P [Xt xt ].
26
Gaussian white noise

1
P [Xt xt ] = (xt ) =
2
xt
x2 /2
dx.
2.5
1.5
0.5
0.5
1.5
2.5
10
15
20
25
27
30
35
40
45
50
Gaussian white noise

2.5
1.5
0.5
0.5
1.5
2.5
10
15
20
25
28
30
35
40
45
50
Time Series Models

P [Xt = 1] = P [Xt = 1] = 1/2.
Example: Binary i.i.d.

1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
0
10
15
20
25
29
30
35
40
45
50
Random walk
St =
Pt
i=1
Xi .
Differences: St = St St1 = Xt .
8
10
15
20
25
30
30
35
40
45
50
Random walk
ESt ? VarSt ?
10
10
15
10
15
20
25
31
30
35
40
45
50
Random Walk
Recall S&P500 data.
(Notice that its smooth)

SP500: JanJun 1987
340
320
300
280
260
240
220
1987
1987.05
1987.1
1987.15
1987.2
1987.25
year
32
1987.3
1987.35
1987.4
1987.45
1987.5
Random Walk
St = St St1 = Xt .
Differences:
SP500, JanJun 1987. first differences
10
10
1987
1987.05
1987.1
1987.15
1987.2
1987.25
year
33
1987.3
1987.35
1987.4
1987.45
1987.5
Trend and Seasonal Models

Xt = T t + S t + E t = 0 + 1 t +
6
(i cos(i t) + i sin(i t)) + Et
5.5
4.5
3.5
2.5
50
100
150
34
200
250

Xt = T t + E t = 0 + 1 t + E t
6
5.5
4.5
3.5
2.5
50
100
150
35
200
250

Xt = T t + S t + E t = 0 + 1 t +
6
(i cos(i t) + i sin(i t)) + Et
5.5
4.5
3.5
2.5
50
100
150
36
200
250
Trend and Seasonal Models: Residuals

0.5
0.4
0.3
0.2
0.1
0.1
0.2
0.3
0.4
0.5
50
100
150
37
200
250
Time Series Modelling

1. Plot the time series.
Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
(c) Nonlinear transformations (log,
3. Fit model to residuals.
38
).
Nonlinear transformations
Recall: Monthly sales.
4
12
(Makridakis, Wheelwright and Hyndman, 1998)

12
x 10
11.5
10
11
10.5
8
10
9.5
9
4
8.5
8
2
7.5
10
20
30
40
50
60
70
80
90
39
10
20
30
40
50
60
70
80
90
Differencing
Recall: S&P 500 data.
SP500: JanJun 1987
SP500, JanJun 1987. first differences
340
10
8
320
6
4
300
280
2
260
4
6
240
8
220
1987
1987.05
1987.1
1987.15
1987.2
1987.25
year
1987.3
1987.35
1987.4
1987.45
1987.5
40
10
1987
1987.05
1987.1
1987.15
1987.2
1987.25
year
1987.3
1987.35
1987.4
1987.45
1987.5
Differencing and Trend

Define the lag-1 difference operator,
(think first derivative)
Xt = Xt Xt1 = (1 B)Xt ,
where B is the backshift operator, BXt = Xt1 .
If Xt = 0 + 1 t + Yt , then
Xt = 1 + Yt .
If Xt =
Pk
i=0
i ti + Yt , then
k Xt = k!k + k Yt ,
where k Xt = (k1 Xt ) and 1 Xt = Xt .

41
Differencing and Seasonal Variation

Define the lag-s difference operator,
s Xt = Xt Xts = (1 B s )Xt ,
where B s is the backshift operator applied s times, B s Xt = B(B s1 Xt )
and B 1 Xt = BXt .
If Xt = Tt + St + Yt , and St has period s (that is, St = Sts for all t), then
s Xt = Tt Tts + s Yt .
42
Least Squares Regression

Model:
X
1
X2
..
.
XT
| {z
x
X t = 0 + 1 t + W t

0
+ Wt ,
= 1 t
1

1 1
1 2
0 +
=
.. ..
1
. .
| {z }

1 T
} |
{z
}
|
Z
43
W1
W2
..
.
WT
{z
w
x = Z + w.
Least squares: choose to minimize kwk2 = kx Zk2 .
Solution satisfies the normal equations:
= 0.
kwk2 = 2Z 0 (x Z )
If Z 0 Z is nonsingular, the solution is unique:
= (Z 0 Z)1 Z 0 x.
44

Properties of the least squares solution ( = (Z 0 Z)1 Z 0 x):
Linear.
Unbiased.
For {Wt } i.i.d., it is the linear unbiased estimator with smallest
variance.
Other regressors Z: polynomial, trigonometric functions, piecewise
polynomial (splines), etc.
45
Outline
1. Objectives of time series analysis. Examples.
2. Overview of the course.
3. Time series models.
4. Time series modelling: Chasing stationarity.
46

Introduction To Time Series Analysis. Lecture 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Time Series Analysis. Lecture 1

Uploaded by

Copyright:

Available Formats

Introduction to Time Series Analysis. Lecture 1.

Fri 121, in 330 Evans. First classroom lab

Objectives of Time Series Analysis

Classical decomposition: An example

Trend and seasonal variation

Objectives of Time Series Analysis

Example: Predict sales.

(Hipel and McLeod, 1994)

Trend plus seasonal variation

Predictions based on a (simulated) variable

Objectives of Time Series Analysis

Example: Seasonal adjustment.

Example: Predict unemployment.

Example: Global warming.

Example: Estimate probability of catastrophic events.

Overview of the Course

Overview of the Course

Overview of the Course

Overview of the Course

Time Series Models

Time Series Models

Gaussian white noise

Gaussian white noise

Time Series Models

Example: Binary i.i.d.

(Notice that its smooth)

Trend and Seasonal Models

(i cos(i t) + i sin(i t)) + Et

Trend and Seasonal Models

Trend and Seasonal Models

(i cos(i t) + i sin(i t)) + Et

Trend and Seasonal Models: Residuals

Time Series Modelling

(Makridakis, Wheelwright and Hyndman, 1998)

SP500, JanJun 1987. first differences

Differencing and Trend

(think first derivative)

where k Xt = (k1 Xt ) and 1 Xt = Xt .

Differencing and Seasonal Variation

Least Squares Regression

Least Squares Regression

Solution satisfies the normal equations:

Least Squares Regression

You might also like