Time Series Forecasting

How to model a time series for prediction?

ARIMA
ARIMA model believes that information is
hidden in

Past of the Series; Yt-1, Yt-2, and so on.

In the residuals of the series; et-1, et-2, and

so on
ARIMA Model:

 The general form is given as ARIMA (p, d, q):

 p is the number of autoregressive terms.

 d is the number of differences.

 q is the number of moving average terms.

 Forecasting a univariate time series
 One can use the average method where future forecasts are calculated as a simple
average of the observed data

for h= 1,2…..

The problem with this method is that it gives equal weights to recent past and
distant past while generating forecasts.
 To take care of the same, Exponential Smoothing Forecast is developed that assigns
more weight to the recent past and less weight to the distant past.
Exponential Smoothing

Simple Exponential Smoothing (Brown)

Holt’s linear trend method

Holt-Winter’s seasonal method

Simple Exponential Smoothing

 This method is suitable for forecasting data with no clear trend or seasonal pattern.
 Forecasts are calculated using weighted averages, where the weights decrease
exponentially as observations come from further in the past i.e., the smallest weights are
associated with the oldest observations:

where 0 ≤ α ≤ 1 is the smoothing parameter

 The one-step-ahead forecast for time T+1 is a weighted average of all of the observations
in the series Y1, …, YT.
 The rate at which the weights decrease is controlled by the smoothing parameter α.
 For any α between 0 and 1, the weights attached to the observations decrease exponentially as we go back in
time, hence the name “exponential smoothing”.
 If α is small (i.e., close to 0), more weight is given to observations from the more distant past. If α is large
(i.e., close to 1), more weight is given to the more recent observations.

 For the extreme case where α=1,  est( yT+1|T = yT)

Weighted Average form

where 0≤α≤1 is the smoothing parameter

 Component form

An alternative representation is the component form. For simple exponential smoothing, the only component
included is the level, ℓt. Component form representations of exponential smoothing methods comprise a forecast
equation and a smoothing equation for each of the components included in the method. The component form of
simple exponential smoothing is given by:
Choosing the smoothing parameters

 The application of every exponential smoothing method requires the smoothing parameters and the initial
values to be chosen. In particular, for simple exponential smoothing, we need to select the values
of α and ℓ0. All forecasts can be computed from the data once we know those values.
 In some cases, the smoothing parameters may be chosen in a subjective manner — the forecaster specifies
the value of the smoothing parameters based on previous experience.
 However, a more reliable and objective way to obtain values for the unknown parameters is to estimate them
from the observed data.
 The unknown parameters and the initial values for any exponential smoothing method can be estimated by
minimising the SSE. The residuals are specified as 

 Hence, we find the values of the unknown parameters and the initial values that minimise
 Linear Trend Model
 Holt (1957) extended simple exponential smoothing to allow the forecasting of data with a trend. This
method involves a forecast equation and two smoothing equations (one for the level and one for the trend):

 Where, ℓt denotes an estimate of the level of the series at time t, bt denotes an estimate of the trend (slope) of
the series at time t, α is the smoothing parameter for the level, 0≤α≤1, and β∗ is the smoothing parameter for
the trend, 0≤β∗≤1.
 As with simple exponential smoothing, the level equation here shows that ℓt is a weighted
average of observation yt and the one-step-ahead training forecast for time t, here given
by ℓt−1+bt−1. The trend equation shows that bt is a weighted average of the estimated trend
at time t based on ℓt−ℓt−1 and bt−1, the previous estimate of the trend.
 The h-step-ahead forecast is equal to the last estimated level plus h times the last estimated
trend value. Hence the forecasts are a linear function of h.
Now, we move to the NEXT
challenge of …
…Estimation and modeling of
VARIANCE!!!
 Estimated volatility should be …

 AUTOREGRESSIVE-depends on its own past.

 CONDITIONAL-variance depends upon past information.

 HETEROSKEDASTICITY- it is for non-constant variance

Kindly note this…

Above all, it should
be deterministic!!!!!!!!
 ARCH MODEL
AutoRegressive Conditional
Heteroskedasticity Model  
 Mean Generating Function -Regression
Model with Constant Variance

Yt     X t   t
Error Term is
INDEPENDENT
 t ~ N (0,  ) 2

Note: The Error term is

STOCHASTIC!!!
Kindly note an interesting fact –
et is not having auto-correlation
but e2t is having auto-correlation!!!
It means that for e2t we may have AR process
but we cannot have AR process for et

 Therefore, ARCH models e2t and not et.

 Kindly understand that the
model we are going to estimate
is based on the following
assumptions –

For MEAN
EQUATION … et is not having auto-
correlation.

For VARIANCE
EQUATION …

e2t is having auto-correlation.

 How to ensure that
variance of Error is
not stochastic!!!

But, we need a model for

deterministic volatility!!!!

What to do?
Decompose into two parts
– one deterministic and
another stochastic!!!!!!
ARCH (1) Model

Yt     X t   t
 t  z t . ht , where z t ~ IIDN (0,1)
where ht = Variance
Deterministic Part
Stochastic Part

ht      2
t 1
Now, we are ready with ARCH (1) Model

 Given the mean equation –

With
Yt     X t   t
 t  zt . ht , where zt ~ IIDN (0,1)
We get …

ht      2
t 1
Generalization ….
ARCH (p) Model
 Given the mean equation –

Yt     X t   t
With

 t  zt . ht , where zt ~ IIDN (0,1)

We get …

p
ht     
i 1
i
2
t i
How to Estimate the ARCH
Model?
Pre-conditions…

 The series should show the sign of Volatility Clustering;

and it can be tested using Ljung-Box Statistics on squared
series.

 Second, it should have ARCH effect i.e. it should be Auto-

Regressive and should be having conditional
heteroscedasticity.
 Do you think that the following series may be
considered for ARCH modeling?

Tata
TataSteel
SteelBBSSLLLtd.
Ltd.
.2
.2

.1
.1

.0
.0

-.1
-.1

-.2
-.2

-.3
-.3
 Make an Assumption!

Let nt = e2 – h and substitute in the Variance Equation of ARCH

Model

We get…
 t    1
2 2
t 1   2 2
t 2 ...   p 2
tp  t
Kindly note that …

 The following regression equation:

 2
t    1 2
t 1   2 2
t 2 ...   p 2
t p  t

is nothing but AR(p) process in e2t.

Estimating ARCH …

 Obtain residuals using mean equation.

 And, then Fit the following auxiliary regression:

 2 t    1 2 t 1   2 2 t  2 ...   p 2 t  p  t
Kindly note that the estimates of
VOLATILITY can never be
NEGATIVE!!!!!!!!

It means that the Coefficients of

ARCH should always be POSITIVE
and their sum less than one to
ensure stationarity of the process.
Formal definition of ARCH…
  
 In a ARCH(p) for conditional variance, forecasts depend on a (non-
negatively) weighted sum of past squared residuals from some conditional
mean function model:
p
ht 1 = 2t 1|t      i  t2i 1
i 1

where w > 0, a1; a2; . . .; ap > 0, and is necessary and sufficient for weak
stationarity.
Estimation of Unconditional Variance…

 Using the ARCH model, we can estimate the long-run UNCONDITIONAL VARIANCE as
–


 2

1  (1   2  ...   p )