Basic Regression Analysis With TS Data Nature of TS Data

Type author
Samuel usingname/s hereetal.

Carter
Chapter Two:
Introduction to basic regression
Chapter
analysis heading
with time series data
Time series Data Analysis: Objectives of the chapter
After successfully completing this chapter you are able to:
• Describe general behavior of time sires data

• Explain the difference between stationary and non-
stationary time series data
• Understand the concept of spurious regression
• Perform unit root tests
• Understand the concept of integrated processes
• Explain the concept of co-integration
• Test for existence of co-integration
• Run VCM
1
Time series Data Analysis: Nature of TS data
1. Temporal ordering
• A TS data set is provided in order of time (chronological)
For example: 2020 immediately precede the data for 2021 and
so forth.
• The past can affect the future but not the future.
2. Stochastic process
 A stochastic process is just a sequence of random variables
indexed by time.
In TS stochastic is a “synonym for random”
 when we collect a TS data set, we obtain one possible
outcome, or realization of the stochastic process.
1
 Therefore, in a TS data set we can only see a single realization,

why?
• Because we cannot go back in time and start the process again
What if time travel is possible?
We would generally obtain a different realization for the
stochastic process and this is why we think of TS data as an
outcome of random variable.
Comparing it with cross sectional data
If you take different samples from your population you get
different outcomes for the interest variable, and this is what
makes OLS estimators a random estimate.
1
MON T H T H H T T T .
TUS H T T T T H T H .
WED T H H T T H T H .
THR H T T T T H T T .
FRI T T T H H T H T .
SAT H H H H H T H T .
SUN T H H T H H T T .
Example: Assume a person flipping a coin from Monday to Sunday, what

would be the outcome of his action? If ordered without replacing,
1
MON T H T H H T T T .
A
TUS H T T T T H T H .
WED T H H T T H T H .
THR H T T T T H T T .
FRI T T T H H T H T .
SAT H H H H H T H T .
SUN T H H T H H T T .
 This is a stochastic process a (set of all possible realizations)

and it plays the role of population.
A is the realization of a stochastic process 1
 The following two natures are not true for all time TS data
3. Trend time series
• Many economic variables have a tendency of growing up
overtime.
• Considering existence of time trend in TS is important while
making a casual inference among a TS data,
• if not we will face a problem of Spurious regression
The trend in a TS can be:
o Deterministic
Or
o Stochastic
1
4. Seasonality
• Often TS data exhibits some periodicity.
Example: in monthly data retail sales will tend to jump in
months of holidays.
• Seasonality can be dealt with adding a set of seasonal
dummies in our regression model.
1
Time series Data Analysis: Some examples of TS Models
1. Static model
• If y and are a TS variables and the data are dated
contemporaneously, we have
• We call it a static model because we are modeling a

contemporaneous relationship between the two.
• We use it when we believe that the change in our explanatory
variable has an immediate effect on our dependent variable.
• Naturally, we can have several explanatory variables in a static

regression model.
1
• We can use it to model trade offs (static phillip curve)
1
2. Finite Distributed lag Model (FDL)

• It is a modification of the static model, where the explanatory
variables are allowed to influence the dependent variable with
a time lag.:
is an FDL of order q, where q is finite.

Example: The fertility rate may depend on the tax value of a
child, but for biological and behavioral reasons, the effect may
have a lag,.
1
Where: is general fertility rate (children born per 1, 000 women in

year ) and is tax exemption in year .
• If there is a one time shock in a past period, the dependent
variable will change temporarily by the amount indicated by
the coefficient of the corresponding lag.
• Consider FDL of order 2
• at time , increases by one unit from to and then reverts to its

previous level at time (temporary change):
1
Then by setting the errors to be zero,
• So
i.e., is the immediate change in due to to the one unit increase in

At time , and usually called impact propensity or impact multiplier.
1
• Similarly
Is change in one period after the temporary change, and
Is change in two periods after the temporary change.

 At time , has reverted back to its initial level: because only two lags
of appears in the FDL model.
 In summary,
Lag distribution: is a function of , which summarizes the dynamic effects

that a temporary increase in has on
1
• When can also graph the lag distribution, to summarize the dynamic effect
that a temporary increase inhas on .
the largest effect is at the first lag. After that, the effect starts to vanishes (if
the initial shock was transitory).
1
• If there is a permanent shock in a past period, i.e., the explanatory

variable permanently increases by one unit, the effect on the
dependent variable will be the cumulated effect of all relevant lags.
This is a long-run effect on the dependent variable
• At time, permanently increases by a unit form to :
Then:
1
• With a permanent increase in , after one period, will increase

by , and after two periods, will increase by and then stay
there.
 The sum of the coefficients on current and lagged , is the long
run change in given a permanent increase in , and is called
long run propensity (LRP) or long run multiplier.
In summary, in a FDL model of order ,
1
• the lag distribution can be plotted by graphing the (estimated) as a

function of .
o For any horizon , we can define the cumulative effect as , which is
interpreted as the change in the expected outcome periods after a
permanent, one-unit increase in .
o Once the have been estimated, one may plot the estimated cumulative
effects as a function of .
o The LRP is the cumulative effect after all changes have taken
place; it is simply the sum of all of the coefficients.
1
Example: suppose that , where is an interest rate and is the

inflation rate. What are the impact and long run propensities?
• We can have more than one explanatory variable appearing with
lags, or we can add contemporaneous variables to a FDL model.
1
3. An Autoregressive Model (AR)

 An AR model, An autoregressive process, is one where a variable
depends on past values of itself.
o The general representation with lagged values, an autoregressive
model (process) of order , is given by:
 AR models can be used to describe the time paths of variables and

capture their correlations between current and past values; they are
generally used for forecasting.
o Past values are used to forecast future values.
1
3. An Autoregressive Distributed Lag Model (ARDL)

 A more general model that includes both finite distributed lag
models and autoregressive models as special cases is the
autoregressive distributed lag model
This model, with lags of , the current value of , is abbreviated as an

model.
 These models are used for both forecasting and policy analysis.
Read about
Infinitely distribute lag model
Moving average model
1
END of slide
Time series Data Analysis: Stationary and Non-stationary TS
• On the Graph the GDP variable displays up ward trending behavior.

• Inflation rate appears wondering up and down with no discernable pattern or trend.
 The figures on the RHS are change of each variable on the LHS
1
o The TS of the changes on the RHS display a behavior that can be

described as irregular ups and downs.
o While change in inflation rate appear to fluctuate around a
constant value, the change in GDP variable appears to fluctuate
around up ward trend.
Which data series represents stationary variables and which are observations
on non stationary variables?
1
Time series Data Analysis: Stationary stochastic process
o Broadly speaking stationarity implies that the distribution of TS

variables does not change overtime.
Formally,
A time series is stationary if its mean and variance are constant overtime,
and if the covariance between two values of from the series depends only
on the length of time separating the two values and not on the actual time
the variables are observed.
1
o We observe that the two moments are time invariant.

o This kind of stationarity is known as weakly stationarity process or
Covariance stationarity.
o Such a TS has a property of mean reversion i.e., it will tend to
return to its mean and fluctuate around it.
Note: A TS is strictly stationary if all the moments of its
probability distribution are time invariant.
1
o Let be a particular realization modeled using a univariate TS

model.
o i.e. is stochastic process were a it is related to the its past values
and current and past error terms.
o Such models does not contain ay explanatory variables.
o AR (1) model, is a useful univariate TS model for explaining the
difference between stationary and non stationary series.
o Hence, before embarking on exploring a new concept let explore
what a stationary TS looks using AR(1) model:
1
Our AR(1) is :
Where the error term is a white noise (purely random)
Note: in the context of TS models the error terms are known as shocks or
Innovations.
o The assumption implies that is stationary.
o In general, the model implies each realization of random variable
contains the portion of last period value plus an error drown from the
a distribution with mean zero and variance.
o To show is stationary let follow a recursive substitution
1
o Conceder the value of at a time then its value at time and so on…
o These values are
The mean of
1
o This is because the error tem has a mean zero and is negligible for
large .
o For an infinite series if ,
o Hence the variance of is:
Using variance definition:
1
o the covariance between two values of is:
o is a classic example of stationary process with mean zero and covariance

which depends on the length of time difference between the two values.
o A real world data seldom has zero mean value, to solve this problem
o We can introduce a none zero mean and this process is known as
demeaning.
o To do so, we first replace with
1
If which is the mean value of
o we can accommodate a nonzero mean in by

 either working with the “demeaned” variable or
 by introducing the intercept term in the autoregressive process
o Corresponding to these two ways, we describe the “de-meaned”
variable ( ) as being stationary around zero, or the variable as
stationary around its mean value ()
1
o Example suppose , and

o The figure shows the added influence of the constant term. The series now
fluctuates around a nonzero value
1
o Another extension to our AR (1) model is to consider it fluctuating around

a linear trend ()
o In this case we let the demeaned series behave like an autoregressive
model:
Which can be rearranged as
Now and
o This is example of trend stationary process, the detrended series is
stationary; is stationary around the deterministic trend line
1
Example suppose , and

o The detrended series has a constant variance, and covariance that depend only on
the time separating observations, not the time at which they are observed.
1
Time series Data Analysis: NonStationarity
o The precise problem created by nonstationarity and the solution

to the problem depends on the nature of the nonstationarity .
o The most important types of nonstationarity in economic TS data
are trend and break nonstationarity.
o In this class we focus on trend nonstationarity.
o A trend is a persistent long-term movement of a variable
overtime. A TS variable fluctuating around its trend may have:
o Deterministic trend (well behaving non-random ) or
o Stochastic trend (random and varies overtime)
In starting Example GDP has a definite trend (deterministic) and INF has a
stochastic trend.
1
Time series Data Analysis: Deterministic trend
o The simplest model for a deterministic trend for a variable is the

linear trend model
o Another popular trend is one where, on average, a variable is

growing at a constant percentage rate.
o In this case the deterministic trend for is , and will be trend

stationary if is stationary.
o we could have also quadratic or Quebec trends
1
Time series Data Analysis: Stochastic trend
o In economics it is more appropriate to model economic TS as

having stochastic rather than deterministic trend. Why?
o because it is hard to reconcile the predictability implied by a
deterministic trend.
o In stochastic trend it would be easy to predict why the TS inc or dec.
Some examples of stochastic trend (Random walk model):

o If we rewrite our model as
And is white noise.

o The value of the series tomorrow is its value today plus unpredictable
change, because the path followed by consists of a random steps
1
o Lets make some recursive substitution:
The stochastic
trend (because it
changes to
unpredictable
directions)
Thus
o A random walk has a mean equal to its initial value and variance that
increases overtime, eventually becoming infinity.
1
Random Walk with Drift

o Some series have an obvious upward or downward tendency.
o In such cases the best model for the TS must include an
adjustment for the tendency of the series.
o This adjustment leads to an extension of the random walk model
to include a drift term:
o Now each realization of the variable contains, an intercept plus the last
period value of and the error .
1
o Notice how the TS data appear to be “wandering” as well as “trending” upward (e).
o In general, random walk with drift models show definite trends either upward (when
the drift is positive) or downward (when the drift is negative).
1
o Again, we can get a better understanding of this behavior by

applying recursive substitution:
o The value of at timeis made up of an initial value, the stochastic trend

component and a deterministic trend component .
Two
o
The variable wanders up and cond f the
it
down as well as increases by a statio ions for
n
fixed amount at each time . viola arity are
ted
1
o We can extend the random walk model even further by adding a

time trend:
o the addition of a time-trend variable t strengthens the trend behavior.
The
additional
term has the
effect of
o Where we have used the formula for a sum of an arithmetic
strengthening
the trend
progression, behavior
1
Time series Data Analysis: Consequences of Stochastic Trends
o Estimating regressions involving variables with stochastic trends

introduce special problems:
o OLS estimates no longer have approximate normal distributions in
large samples. Hence, interval estimates and hypothesis tests will
no longer be valid.
o Precision of estimation would be lost and conclusions about relationships
between variables could be wrong.
o The t-statistics has a non-normal distribution even in large sample
size (very high significant t-statistic)
o One particular hazard is that two totally independent random walks can
appear to have a strong linear relationship when none exists.
o Outcomes of this nature have been given the name spurious
regressions.
1
Example suppose and

o Suppose we regress on . Since and are uncorrelated nonstationary processes,
the from the regression of on should tend to zero;
o that is, there should not be any relationship between the two variables.
1
o As you can see, the coefficient of is highly statistically significant, and, although
the value is low, it is statistically significantly different from zero.
o You may be tempted to conclude that there is a significant statistical relationship
between and
1
o This is in a nutshell the phenomenon of spurious or nonsense regression.

o The extremely low Durbin–Watson value suggests very strong first-order
autocorrelation.
o furthermore, an is a good rule of thumb to suspect that the estimated regression
is spurious
1
Time series Data Analysis: policy Importance
o There are also important policy considerations for distinguishing

between stationary and nonstationary variables.
o With nonstationary variables each error or shock has a lasting
effect, and these shocks accumulate.
o With stationary variables the effect of a shock eventually dies out
and the variable returns to its mean.
o Whether a change in a macroeconomic variable has a permanent or
transitory effect is essential information for policy makers.
o Thus our next steps would be answering the questions of:
 How can we test whether a series is stationary or nonstationary,
 and how do we conduct regression analysis with nonstationary data?
1
Type author
Samuel usingname/s hereet al.
Carter
Chapter Two:
Introduction to basic regression
Chapter
analysis heading
with time series data
Unit Root Test

Time series Data Analysis: Unit Root Test
oThere are many tests for assessing whether a series is stationary

or nonstationary. These includes:
 Graphical analysis, The correlogram test and
 Unit root tests (Dickey-Fuller test and Philips Person test …)
oThe most popular one is the Dickey–Fuller (DF) test for a unit root.
oBased up on the behavior of a given type of TS process, there are
three variations of the DF test, each one designed for a di fferent
alternative hypothesis.
1
Time series Data Analysis: Unit Root Test
o The alternative hypothesis are
 that is stationary around a zero mean.
 that is stationary around a nonzero mean.
 that is stationary around a linear deterministic trend.

o The choice between these tests can be guided by the nature of
the data, revealed by plotting the series against time.
o If it is not obvious from a plot which test is the most relevant,
more than one test equation can be used to check the
robustness of a test conclusion.
1
Time series Data Analysis: DF Test (No constant, No trend)
o Consider the following AR (1) process
o Here one way of testing for stationarity is to examine the value of

i.e. test whether is equal to one or significantly less than one.
o Such kinds of tests are known as unit root test for stationarity.
o To make it formal consider
Where is a white noise
1
o We can test for stationarity by using
Why is it a one till test?

o We can make it more convenient by subtracting from both sides
Where and , the hypothesis can be written as
1
The hypothesis can be written as
o The null hypothesis is that the series is nonstationary, in other words if

we don’t reject the null, then we conclude that it is a nonstationary
process.
o If we reject the null that then we conclude that the series is stationary.
1
Time series Data Analysis: DF Test (with constant, No trend)
o Consider a time series that has no definite continuous trend, and that
is not obviously centered around zero.
o Suppose we wish to test whether this series is better represented by a
stationary or a nonstationary random walk.
o The nonstationary random walk is set up as the null hypothesis
o the stationary AR(1) process becomes the alternative hypothesis
 Under , the series fluctuates around a constant mean.

 Under , it wanders upward and downward but does not exhibit a clear
trend in either direction and does not tend to return to a constant mean.
1
Time series Data Analysis: DF Test (with constant, No trend)
o An obvious way to specify the null hypothesis in terms of the

parameters in the unrestricted:
o But it has become more common to simply specify the null as
o One way to justify omission of from is recall if then , therefore, testing

for is sufficient.
o To make it more convenient, the test equation can be specified as,
o Our and will not change (are the same as before).

 If we reject the the TS is stationary around a none zero mean.
1
Time series Data Analysis: DF Test (with constant and trend)
o For a series that have a clear time trend, we need to modify test for
unit root.
o If we carry out a DF test on a trending but stationary series, we will
probability have a little power rejecting a unit root.
o Hence, it is necessary to control for time while making this test, not to
mistaken trend stationary process to difference stationary process.
o If is stationary around a linear trend and described by the process
o If is random walk with drift, then it is described as
1
o The DF test with intercept and trend is designed to discriminate

between these two models.
o The latter became the null hypothesis and the former the alternative
hypothesis.
o If the null hypothesis is rejected we conclude the series is trend
stationary.
o failure to reject the null hypotheses suggests is nonstationary, or at
least there is insufficient evidence to prove otherwise.
o The relevant equation for the test would be obtained by rewriting our
basic equation as:
1
o The relevant null hypothesis would be
o in the null hypothesis is ignored, and it can be justified using the same
line of reasoning presented earlier.
Where is the deterministic trend.

o With these definitions of and λ, setting implies and , giving the
random walk with drift.
1
Time series Data Analysis: DF Testing procedure
• Given the TS the appropriate DF model selected first by plotting the TS

of the original observations on the variable
o If the series is fluctuating around a sample average of zero, we use
the test equation one.
o If the series appears wondering around a sample average that is non
zero use test equation two.
o if the series appears wondering or fluctuating around a linear trend
use test equation three.
o To test the hypothesis in all the three cases, estimate the test equation
by least square and examine for .
1
Time series Data Analysis: DF Testing procedure
• Why ?
• Unfortunately the has no longer t-distribution (Asymptotic SND even in
large sample size] that we have used in may cases to test a zero null
for a regression coefficient. Why?
o Because when the is true, is not stationary, the variance increases
as the sample size increases. And this increase will alter the
distribution of the usual .
o Recognizing this fact Dickey and Fuller developed an appropriate
critical values often called
o The critical values in depends on the type of equation we are using.
1
• Recall that to carry out this one tail test of significance, we reject the null
hypothesis of stationarity if:
Don't reject if
1
Example: to test for a unit root in three-month T-bill rate we estimated and
obtained the following result
A, can we carry out the usual t-test? B, what does the coefficient of
imply?
1
A, we cannot use the usual t-test to check for the hypothesis because
existence of nonstationarity may result a non STD which affects the
outcome of our test.
1
B, and , Thus the . At a 10% critical level,

thus we fail to reject our
1
 Which model is inappropriate?

Model one is inappropriate because is positive and above one
 is LGDP stationary?
NO, show why?
1
Time series Data Analysis: Augmented DF Test
• It is an important extension of the DF test which allows for the

possibility that the error term is autocorrelation.
• Such autocorrelation is likely to occur if our earlier model does not
have the sufficient lag terms to capture full dynamic nature of the
process.
• The representative equation is given as:
o The inclusion of the lagged changes is intended to clean up any serial

correlation
1
• While augmented Dickey–Fuller tests remain the most popular tests for
unit roots, the power of the tests is low in the sense that
• they often cannot distinguish between a highly persistent stationary
process (where is very close but not equal to 1) and a nonstationary
process (where ).
• The power of the test also diminishes as deterministic terms constant
and trend are included in the test equation.
• tests that have been developed with a view to improving the power of
the test: the Elliot, Rothenberg, and Stock (ERS), Phillips and Perron
(PP), Kwiatkowski, Phillips, Schmidt, and Shin (KPSS), and Ng and
Perron (NP) tests.
1
• The ERS test proposes removing the constant/trend effects from the
data and performing the unit root test on the residuals.
• The distribution of the t-statistic is now devoid of deterministic terms
(i.e., the constant and/or trend).
• The PP test adopts a nonparametric approach that assumes a general
autoregressive moving-average structure and uses spectral methods to
estimate the standard error of the test correlation.
• Instead of specifying a null hypothesis of nonstationary, theKPSS test
specifies a null hypothesis that the series is stationary or trend
stationary. NP tests suggest various modifications of the PP and ERS
tests..
1
END of slide

Basic Regression Analysis With TS Data Nature of TS Data

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Basic Regression Analysis With TS Data Nature of TS Data

Uploaded by

Copyright:

Available Formats

Type author

Samuel usingname/s hereetal.

After successfully completing this chapter you are able to:

• Describe general behavior of time sires data

 Therefore, in a TS data set we can only see a single realization,

Example: Assume a person flipping a coin from Monday to Sunday, what

 This is a stochastic process a (set of all possible realizations)

• We call it a static model because we are modeling a

• Naturally, we can have several explanatory variables in a static

• We can use it to model trade offs (static phillip curve)

2. Finite Distributed lag Model (FDL)

is an FDL of order q, where q is ﬁnite.

Where: is general fertility rate (children born per 1, 000 women in

• at time , increases by one unit from to and then reverts to its

Then by setting the errors to be zero,

i.e., is the immediate change in due to to the one unit increase in

Is change in one period after the temporary change, and

Is change in two periods after the temporary change.

Lag distribution: is a function of , which summarizes the dynamic effects

• If there is a permanent shock in a past period, i.e., the explanatory

• With a permanent increase in , after one period, will increase

In summary, in a FDL model of order ,

• the lag distribution can be plotted by graphing the (estimated) as a

Example: suppose that , where is an interest rate and is the

3. An Autoregressive Model (AR)

 AR models can be used to describe the time paths of variables and

3. An Autoregressive Distributed Lag Model (ARDL)

This model, with lags of , the current value of , is abbreviated as an

• On the Graph the GDP variable displays up ward trending behavior.

o The TS of the changes on the RHS display a behavior that can be

o Broadly speaking stationarity implies that the distribution of TS

o We observe that the two moments are time invariant.

o Let be a particular realization modeled using a univariate TS

Where the error term is a white noise (purely random)

Using variance definition:

o the covariance between two values of is:

o is a classic example of stationary process with mean zero and covariance

If which is the mean value of

o we can accommodate a nonzero mean in by

o Example suppose , and

o Another extension to our AR (1) model is to consider it ﬂuctuating around

Which can be rearranged as

Example suppose , and

o The precise problem created by nonstationarity and the solution

o The simplest model for a deterministic trend for a variable is the

o Another popular trend is one where, on average, a variable is

o In this case the deterministic trend for is , and will be trend

o In economics it is more appropriate to model economic TS as

Some examples of stochastic trend (Random walk model):

And is white noise.

o Lets make some recursive substitution:

Random Walk with Drift

o Again, we can get a better understanding of this behavior by

o The value of at timeis made up of an initial value, the stochastic trend

o We can extend the random walk model even further by adding a

o the addition of a time-trend variable t strengthens the trend behavior.

o Estimating regressions involving variables with stochastic trends

Example suppose and

o This is in a nutshell the phenomenon of spurious or nonsense regression.

o There are also important policy considerations for distinguishing

Unit Root Test

oThere are many tests for assessing whether a series is stationary

o The alternative hypothesis are

 that is stationary around a zero mean.