Chapter _3

Introduction to Basic Regression Analysis with

Time Series Data

1
W H AT IS A TIME S E R I E S ?

Essentially, Time Series is a sequence of

numerical data obtained at regular time
intervals.
Occurs in many areas: economics,
finance, environment, medicine

The aims of time series analysis are

■ to describe and summarize time series

data,
■ fit models, and make forecasts

2
W H Y ARE TIME SERIES DATA
different from other data?
Data are not independent

■ Time series analysis focuses on modeling

the dependency of a variable on its own
past, and on the present and past values
of other variables.

3
WHAT ARE U SE RS LOOKING FOR
Economic Time Series?
Important features of economic
indicator series include
■ Direction
■ Turning points
■ In addition, we want to see if the
series is increasing/decreasing
more slowly/faster than it was
before

4
 FORECASTING HORIZONS

Long Term
■ 5 + years into the future

■ R&D, plant location, product planning

■Medium Term

■ 1 season to 2 years

■ Aggregate planning,

capacity planning, sales

forecasts
■Short Term

■ 1 day to 1 year, less than 1 season

■ Demand forecasting, staffing levels,

purchasing, inventory levels

5
EXAMPLES OF TIME SERIES DATA

Number of babies born in each

hour
Daily closing price of a stock.
The monthly trade balance of
Japan for each year.
GD P of the country, measured
each year.

6
TIME SERIES EXAMPLE

Exports,1989-
How the data t Year x=Valu
1998
(x) and time 1 1989 e
(t) is 2
44,320 1990 52,86
recorded and 3 1991 5
presented 453,092 1992
5
39,424 1993
6 34,444
7 1995
1994
8 1996
49,805
47,870
9 1997
59,404
10 1998
70,214
74,626
7
TIME SERIES
Export
s
Coordinates 80,000

(t,x) is 75,000
70,000
establis hed 65,000

in the 2 axis 60,000

55,000
(1, 44,320) 50,000

(2, 52,865) 45,000

40,000

(3, 53,092) 35,000

30,000
etc. 1988 1990 1992 199
4
199
6
199
8
200
0
.
8
TIME SERIES

A graphical Exports
repres entation
80,000
of time 75,000
series.
We us e x a s a 70,000

function of t: 65,000
60,000
x = f(t) 55,000
Data points 50,000

connected by 45,000
40,000
a curve 35,000
30,000
1988 1990 1992 1994 1996 1998 2000

9
 IMPORTANCE

■ Understand the past.

What happened over the last years,
months?

■ Forecast the future.

 Government wants to know future of
unemployment rate, percentage increase in
cost of living etc.
 For companies to predict the demand for their
product etc.

10
TIME-SERIES COMPONENTS

Tren Cyclical
d
Time-Series

Seasonal Random

11
 Trend (Tt )
Trend: the long-term patterns or
movements in the data.
long-term upward or downward
pattern of movement.

The trend of a time series is not

always linear.

12
Seasonal variation (St )
Regular periodic fluctuations that
occur within year.
Examples:
Consumption of heating oil, which
is high in winter, and low in
other seasons of year.
Gasoline consumption, which is
high in summer when most people
g o on vacation.

13
 Seasonal variation (St )

Summer Summer
3
02
5
2
0
1
5
1
0
5 Winte Winter
0 r
-
5
-
10 14
C A U S E S OF S E A SO NA L EFFECTS

Possible causes are

■ Natural factors
■ Administrative or legal measures
■ Social/cultural/religious traditions
(e.g., fixed holidays, timing of
vacations)

15
 CYCLICAL VARIATION ( CT )

• Cyclical variations are similar to seasonal

variations. Cycles are often irregular both in
height of peak and duration.
• Examples:
• Long-term product demand cycles.
• Cycles in the monetary and financial
sectors.

16
CYCLICAL COMPONENT

Long-term wave-like patterns

Often measured peak to peak or
trough to trough
1 Cycle
Sales

Year 23
IRREGULAR C OMPONENT

Unpredictable, random, “residual”

fluctuations
D u e to random variations of
■ Nature
■ Accidents or unusual events
“Noise” in the time series

18
C A U S E S OF IRREGULAR EFFECTS

Possible causes
■ Unseasonable weather/natural
disasters
■ Sampling error
■ Nonsampling error

19
 OUR AIM

 is to understand and identify different

variations so that we can easily predict the
future variations separately and combine
together.

20
STOCHASTIC PROCESSES
 A random or stochastic process is a collection of
random variables ordered in time.
 All time series may be divided into two big
classes - stationary and non-stationary.
1. STATIONARY STOCHASTIC PROCESSES
 Stationary process - a random process with a
constant mean, variance and covariance.
 In other words, a time series Yt is stationary if its
mean, variance and covariance do not depend on
t.
 If at least one of the three requirements is not
met, then the process is not-stationary
 To explain stationarity, let Yt be a stochastic time
series with these properties:
 Mean:
 Variance: = =

 Covariance: =
 a time series with these characteristics is know as
weakly or covariance stationary or second-order
stationary, or wide sense, stochastic process
 Now if Yt is to be stationary, its mean, variance,
and autocovariance (at various lags) remain the
same no matter at what point we measure them;
that is, they are time invariant.
 Such a time series will tend to return to its mean
(called mean reversion) and fluctuations around
this mean (measured by its variance) will remain
constant.
WHY ARE STATIONARY TIME SERIES SO IMPORTANT?

 Because if a time series is non-stationary, we can

study its behavior only for the time period under
consideration.
 Each set of time series data will therefore be for
a particular time period. As a consequence, it is
not possible to generalize it to other time periods.
2. NON STATIONARY STOCHASTIC PROCESSES

 A non-stationary time series will have a time

varying mean or a time-varying variance or both.
 It is often said that asset prices, such as stock
prices or exchange rates, follow a random walk;
that is, they are non-stationary.
 We distinguish two types of random walks: (1)
random walk without drift (i.e., no constant or
intercept term) and (2) random walk with drift (i.e.,
a constant term is present).
A. RANDOM WALK WITHOUT DRIFT
 Suppose is a white noise error term with mean 0
and variance σ2 and serially uncorrelated . Then the
series Yt is said to be a random walk if
= +
 In the random walk model shows the value of Y at
time t is equal to its value at time (t − 1) plus a
random shock; thus it is an AR(1) model.
 E (Yt) = Yo
 Var (Yt) = tσ2
B. RANDOM WALK WITH DRIFT
Let us modify as follows:
= +
where δ is known as the drift parameter.
= + t.δ
=
Hence, for RWM with drift the mean as well as the
variance increases over time, again violating the
conditions of stationary.
In short, RWM, with or without drift, is a non-stationary
stochastic process.
3. UNIT ROOT STOCHASTIC PROCESS
 Let us write the RWM as

= +

If ρ = 1, becomes a RWM (without drift). If ρ is in fact 1, we face

what is known as the unit root problem, that is, a situation of
non-stationarity.
The name unit root is due to the fact that ρ = 1.
If, however, |ρ| ≤ 1, that is if the absolute value of ρ is less than one,
then it can be shown that the time series is stationary.
Thus the terms non-stationarity, random walk, and unit root can be
treated as synonymous.
 
2.6. SPURIOUS REGRESSION

 regressing a non stationary time series on one or more non stationary time series may
often lead to spurious [meaningless] regression
 Statistically significant regression coefficients, but not reliable

Variable Coefficient Std. error t statistic

------------------------------------------------------------------------------------------------------
C -13.2556 0.6203 -21.36856
X 0.3376 0.0443 7.61223
R2 = 0.1044 d = 0.0121

--------------------------------------------------------------------------------------------------------
-
 
 R2 > d is a good rule of thumb to suspect that the estimated regression is spurious
4. TREND STATIONARY (TS) AND DIFFERENCE STATIONARY (DS)
STOCHASTIC PROCESSES

 The distinction between stationary and non-

stationary stochastic processes (or time series)
has a crucial bearing on whether the trend is
deterministic or stochastic.
 Broadly speaking, if the trend in a time series is
completely predictable and not variable, we call it
a deterministic trend, whereas if it is not
predictable, we call it a stochastic trend.
TESTS OF STATIONARITY
1. The Unit Root Test

= +
= + where δ = (ρ − 1) and , as usual, is
the first-difference operator.
Estimate and test the (null) hypothesis that δ = 0.
If δ = 0, then ρ = 1, that is we have a unit root,
meaning the time series under consideration is
non-stationary.
 Dickey–Fuller (DF) test
 null hypothesis is that δ = 0; that is, there is a unit root-the time
series is non-stationary.
 The alternative hypothesis is that δ is less than zero; that is, the
time series is stationary.
 If the computed absolute value of the tau-statistic (|τ |) exceeds
the DF or MacKinnon critical tau values, we reject the
hypothesis that δ = 0, in which case the time series is stationary.
 On the other hand, if the computed |τ | does not exceed the
critical tau value, we do not reject the null hypothesis, in which
case the time series is non-stationary
 Example: The results of the regressions is as follows: The
dependent variable in is
 = 28.2054 − 0.00136
t = (1.1576) (−0.2191) R2 = 0.00056 d = 1.35
The critical 1, 5, and 10 percent τ values are −3.5064,
−2.8947, and −2.5842.

The estimated δ coefficient is negative, implying that the

estimated ρ is less than 1. The estimated τ value is
−0.2191, which in absolute value is below even the 10
percent critical value of −2.5842. Since, in absolute
terms, the former is smaller than the latter, our conclusion
is that the GDP time series is not stationary.
TRANSFORMING NONSTATIONARY TIME SERIES

 To avoid the spurious regression problem that

may arise from regressing a non-stationary time
series on one or more non-stationary time series,
we have to transform non-stationary time series
to make them stationary.
 The transformation method depends on whether
the time series are difference stationary (DSP) or
trend stationary (TSP).
A. DIFFERENCE-STATIONARY PROCESSES

 If a time series has a unit root, the first differences of

such time series are stationary. Therefore, the solution
here is to take the first differences of the time series.
 Let For convenience, let Dt =GDPt
 Now consider the following regression:
 = 16.0049 − 0.06827Dt-1
 t = (3.6402) (−6.6303) R2 = 0.3435 d = 2.0344
 The 1 percent critical DF τ value is −3.5073. Since
the computed τ (= t) is more negative than the critical
value, we conclude that the first-differenced GDP is
stationary; that is, it is I(0).
B. TREND-STATIONARY PROCESS

 The simplest way to make Trend-Stationary

Process time series stationary is to regress it on
time and the residuals from this regression will
then be stationary.
 In other words, run the following regression:

Yt = B1 + B2t + Ut
Now,
 Ut = Yt – B1-B2t will be stationary. U is known
as a (linearly) detrended time series.
5. INTEGRATED STOCHASTIC PROCESSES
 a time series becomes stationary after differencing
it once called integrated of order one I(1)
 if it has to be differenced twice to become
stationary; integrated of order two I(2)
 if differenced d times to make it stationary, then it
is integrated of order d; I(d)
 a stationary time series is integrated of order zero;
I(0)