Stages in Model Building for Time Series Data

I. Preliminary Stage

Identify the stationary input series through transformation.

II. Identification Stage

Identify preliminary model using the characteristics of the sample acf and sample
pacf.

III. Estimation of the Parameters

After identifying a tentative model, the next step is to estimate the parameters of
the model. With full generality, we consider the general ARMA (p,q) model. The
parameters to be estimated are,

’ = (1, 2, …, p)

’ = (1, 2, …, q)
2 = Var(t2) and  = E(t)

The parameters in the ARMA process are estimated using non-linear least
squares. Consider the ARMA (1,1) process,

y t  1y t 1  ε t  θ1ε t 1 (1)

ε t  y t  1y t 1  θ1ε t 1 (re  arranging terms) (2)
ε t  y t  1y t 1  θ1 (y t 1  1y t  2  θ1ε t  2 ) (substitut ing ε t -1 as(2))
ε t  y t  (1  θ1 )y t 1  1θ1y t  2  θ12ε t  2
Clearly, the ARMA (1,1) process is non-linear in the parameters. Hence, the non-
linear least-squares estimation procedure is used to obtain the estimates.

1
Non-Linear Estimation

Consider the model,

Yt  f( X t , α)  ν t , t  1,2,..., T
X t  (X t1 , X t2 ,, X tp )
'
is a set of independen t variables
α  (α1 , α 2 , α p ) is a vector of parameters
'

ν t is a white noise process with mean 0 and variance  2 .

The least-squares estimators can be calculated iteratively as follows:

Step 1. Given any vector of initial estimates (normally using OLS), say *,
compute for,

~
e  y  ŷ and SSE  ~
e~
'
e  ( y  ŷ)' ( y  ŷ)
Approximate the model f(Xt , ) with the first-order Taylor series expansion
about the initial value *,

f( α)  f( α )  X α* δ
*
where,
δ  (α  α ) and X α*  X tj is the (T x p) matrix of the
*

partial derivative s at α in the linear approximat ion,

*

f(X t , α)
X tj  t  1,2, , T and j  1, 2, , p
α j
and   (X α* X α* ) 1 (X α* e~ ).
'

Note that for a linear model, X α* is fixed at X , while for a non-linear model,
this changes from iteration to iteration.

Step 2. Obtain the updated least square estimates

α̂   *  δ and the SSE( α̂)

2
 Important Notes in Estimation using EVIEWS

a. The starting values for  are based on OLS/TSLS (default). You can assign
your own estimates, say using the method of moments.

b. The algorithm method used is the Marquardt algorithm (default) or the Berndt,
Hall, Hall, Hausman (BHHH) algorithm.
c. The convergence value is at 0.0001 (default).
d. The inverted (or reciprocal) AR and MA roots are being reported. If the
absolute value of the inverted real AR root exceeds 1 or if a pair of complex
inverted roots are outside the unit circle (that is, with modulus greater than 1),
it means that the AR process is explosive (non-stationary).
e. If the inverted roots of the MA process are outside the unit circle, we say that
the process is non-invertible.
f. If the estimated MA process has roots with modulus close to one, it is a sign
that you may have over-differenced the data. The process will be difficult to
estimate and even more difficult to forecast.
g. MA models are notoriously difficult to estimate, in particular, try to avoid
higher order MA terms unless absolutely required by the model.
h. Eviews provides automatic backcasting of the MA terms, if you encounter
convergence problems, you may wish to turn off this option (just click the
option icon).

IV. Diagnostic Checking – Verification of the Assumptions

The basic assumption is that {t} is a white noise process with mean 0 and
constant variance. We analyze the residuals, et, and verify whether the following
assumptions are satisfied:

a. Portmanteau lack of fit (test for white noise)

H o :  e1   e2     eK  0
K
QLB  T (T  2) (n  K ) ˆ k2
k 1

View/Residual Tests/Correlogram-Q-Statistics

b. Non-constancy of Variance

ARCH -Auto-Regressive Conditional Heteroskedasticity Test

The ARCH test statistic is computed from an auxiliary regression. To test the null
hypothesis that there is no ARCH up to order q in the residuals, run the
regression,

3
 e 2t  β 0  β1e 2t 1  β 2 e 2t 2    β q e 2t q   t

The test statistic is T*R2 (Obs*R-squared in EViews) and is known as the Engle’s
LM test statistic (Robert Engle, 1982). This statistic is asymptotically distributed
as chi-square with q degrees of freedom.

View/Residual Tests/ARCH-LM and plug in the number of lags (q).

One can also look at the Correlogram Squared Residuals (View/Residual

Tests/Correlogram Squared Residuals) where a portmanteau type of test (Q
statistic) using the squared of the residuals.

c. Normality of the White Noise Process

Use the Jarque-Bera Statistic

View/Residual Tests/Histogram-Normality Test

V. Model Selection Criteria

There may be several adequate models that can be used to represent the given set
of data. The selection criterion is normally based on summary statistics from
residuals computed from a fitted model or from forecast errors calculated from
out-of-sample (or in-sample) forecasts.

a. Akaike Information Criterion

AIC(m)  T * log( ζ̂ ε2 )  2m
where ζ̂ ε2 is the Maximum Likelihood Estimator of  2 .
The optimal order of the model is chosen by the value m (the number of
parameters), which is a function of p and q, so that AIC(m) is minimum.

b. Schwartz’s Bayesian Criterion (SBC)

SBC(m)  T * log( ζ̂ ε2 )  m * log(T)

c. Forecast Errors – MAPE and/or RMSE

If the main purpose of the model is to forecast future values, alternative

criteria for the model selection can be based on the errors of out-of-sample
forecasts. One can use the mean absolute percentage error (MAPE) or the
root mean square error (RMSE).