Welcome back. You are signed in as rakshitsingh1582@gmail.com. Not you?

Understanding Auto Regressive Moving

Average Model — ARIMA
Farhad Malik Follow
Sep 19, 2018 · 6 min read

In my article “How Do I Predict Time Series?”, I provided an overview of time series

analysis. Core of the article focused on the concept that future values in a time series are
dependent on its past values.

This article serves as an overview of a powerful yet simple model known as ARIMA.
Additionally it provides a comparison of two models: GARCH and EWMA. Both models
assume that the recent events have higher precedence than former events. This model is
important in data science and neural networks.

Please read FinTechExplained disclaimer.

What Is ARIMA?
ARIMA stands for Auto Regressive Integrated Moving Average. ARIMA is a simple
stochastic time series model that we can use to train and then forecast future time
points.

ARIMA can capture complex relationships as it takes error terms and observations of
lagged terms. These models rely on regressing a variable on past values.

ARIMA is Auto Regressive — (AR)

Auto Regressive (AR) property of ARIMA is referred to as P.

Past time points of time series data can impact current and future time points. ARIMA
models take this concept into account when forecasting current and future values.
ARIMA uses a number of lagged observations of time series to forecast observations. A
weight is applied to each of the past term and the weights can vary based on how recent
they are.

AR(x) means x lagged error terms are going to be used in the ARIMA model.

ARIMA relies on AutoRegression. Autoregression is a process of regressing a variable on

past values of itself. Autocorrelations gradually decay and estimate the degree to which
white noise characterizes a series of data.

ARIMA is Integrated — (I)

If a trend exists then time series is considered non stationary and shows seasonality.
Integrated is a property that reduces seasonality from a time series. ARIMA models have
a degree of differencing which eliminates seasonality. More information on differencing
is outlined in my article “How Do I Predict Time Series?”.

D property of ARIMA represents degree of differencing.

ARIMA is Moving Average — (MA)

Error terms of previous time points are used to predict current and future point’s
observation. Moving average (MA) removes non-determinism or random movements
from a time series. The property Q represents Moving Average in ARIMA. It is expressed
as MA(x) where x represents previous observations that are used to calculate current
observation.

Moving average models have a fixed window and weights are relative to the time. This
implies that the MA models are more responsive to current event and are more volatile.

P (AutoRegressive), D (Integrated) and Q (Moving

Average) are the three properties of ARIMA model
Coefficients are calculated recursively. Model is chosen such that the estimated results
calculated from the model are closer to the actual observed values. This process is
iterative in nature.

EWMA Vs GARCH
In this section, I wanted to highlight two main models: EWMA and GARCH. EWMA and
GARCH models revolve around the concept of model persistence.

Model Persistence describes rate at which the observation will revert to its long term
value following a large movement. If we are observing volatility then high persistence
means that if there is a shock movement in the market then the volatility will take longer
to revert to the mean.

Persistence of reliable models should be lower than 1.

Persistence greater than 1 means that the model is not stable and there is no reversion to
the mean.

EWMA Is Exponentially Weighted Moving Average

EWMA is a moving average (MA) model. It forecasts variance in time series data by
taking the weighted average of the previous day’s estimated variance and previous day’s
return. Hence EWMA utilises a linear regression model of the current values of time
series against both the current and previous unobserved random shocks. These random
shocks are known white noise error terms.

The model assumes that future forecasts are a function of historic events. However it
also assumes that recent events have higher precedence than former events.

EWMA applies weights to the previous and the latest returns (known as innovation)
such that the recent observations are weighted heavily. The weights decrease at an
exponential rate as the time moves backwards. The weights sum to 1. The weights are
assigned to consider the latest observations to have more importance than the older
observations.

This is the reason why the model is known as Exponentially Weighted Moving Average
(EWMA) forecasting model.

At a high level, the recursive EWMA model can be written as:

Variable Under Observation = Current Shock + Last

Shock x Weight of last shock for EWMA(1) model
Autocorrelation is computed as: Coefficient for lagged random shock / (1 + Coefficient for
lagged random shock)
This formula ensures that the weight is smaller for distant observations when compared
to recent observations to indicate that recent observations have more importance.

GARCH -Generalised Autoregressive Conditional Hetroskedastic

model
GARCH is an alternative method for calculating conditional variance (co variance). The
model assumes that the return at time (t) has a particular distribution and the standard
deviation of the distribution is indexed by time.

GARCH assumes that the latest return of the stock is dependent on the previous return.
Hence the variance is conditional on the previous returns.

GARCH is a function of three terms:

1. Long Term Variance

2. Square of Last Return

3. Previous Variance

A weight is applied to each of the term. These terms are known as GAMMA, ALPHA and
BETA. The formula to calculate conditional variance under GARCH is:

(Gamma x Long Term Variance) + (Alpha x Square

of Last Return) + (Beta x Previous Variance)
The fundamental rule of GARCH is that Gamma + Alpha + Beta should add up to 1.
GAMMA, ALPHA and BETA are calculated such that the estimated and actual observed
values are as close as possible.

Long Term Average Covariance Rate is then

calculated as = (Gamma)/(1-Alpha-Beta)
Long Term Variance tells us about ‘data’s stickiness’ to a value e.g. 1% means data will
move towards 1 as time moves ahead. It is essentially assigning a weight to the average
long run variance estimate which implies that the model recognises mean reverting
characteristic of volatility which is also explained above as model persistence. Beta is the
decay rate and it is exponential.

EWMA Similarities With GARCH

We can see that EWMA is a special case of GARCH because Gamma x Long Term
Variance is set to 0 in EWMA. This enforces ALPHA and BETA to add up to 1 because we
only take previous return squared and previous variance into account in EWMA.

It has permanent persistence and longer term forecasts are therefore different between
GARCH and EWMA models.

Finally, I wanted to explain why one would choose GARCH over EWMA.

When Should We Use GARCH Over EWMA?

 GAMMA and Long Term Variance rate of GARCH models help in explaining historic
volatility better.

GARCH models are good at modelling volatility clustering. Volatility clustering is the
concept that indicates that high period volatility tend to be followed by periods of high
volatility.

When Should We Use EWMA Over GARCH?

If we notice that in GARCH model, alpha + beta > 1 then we must use EWMA model
because it is then more stable than GARCH.

Remember alpha + beta is known as persistence and persistence is always lower than 1
for a stable model.

ARIMA Assumptions
ARIMA model is based on a number of assumptions including:

Data does not contain anomalies

Model parameters and error term is constant

Historic timepoints dictate behaviour of present timepoints which might not hold in
stressed market data conditions

Time series is stationary

Summary
This article provided an overview of ARIMA and highlighted main differences between
EWMA and GARCH. These models can be used to forecast time series.

Hope it helps. Please let me know if you have any feedback.

Data Science Arima Timeseries Machine Learning Fintech

About Help Legal

Get the Medium app