Pompea College of Business

Business Forecasting
ECON 6635/ECON 3343
Dr. Esin Cakan

*Slides are adopted from OTexts.org/fpp3/ with courtesy of Rob Hyndman

Chp11. Advanced Forecasting Models

OTexts.org/fpp3/
Outline

1 Complex seasonality

2 Vector autoregression

3 Neural network models

4 Bootstrapping and bagging

2
Examples

Weekly US finished motor gasoline products

Thousands of barrels per day

1990 1995 2000 2005 2010 2015

Year

3
Examples

5 minute call volume at North American bank

Call volume

400
300
200
100
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33
Weeks
400
Call volume

300

200

100

1 2 3 4
Weeks

4
Examples

Turkish daily electricity demand

25
Electricity Demand (GW)

20

15

10

2000 2002 2004 2006 2008

Year

5
TBATS model

TBATS
Trigonometric terms for seasonality
Box-Cox transformations for heterogeneity
ARMA errors for short-term dynamics
Trend (possibly damped)
Seasonal (including multiple and
non-integer periods)

6
TBATS model
yt = observation at time t

(y ω− 1)/ω if ω =6 0;
t
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
X
st−mi + dt
i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
X
st−mi + dt
i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X

i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X

i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X

i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p q
X X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X

i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p q
X X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 λj(i) + s∗(i)
cosFourier-like sin λ(i)
j,t−1 seasonal
(i)
j + γterms
1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t

(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) = 
TBATS
log yt if ω = 0.
Trigonometric M
M seasonal periods
yt(ω) = `t−1 + φbt−1 + (i)
X
st−m + dt
Box-Cox i=1 i

`t = `t−1 + φbt−1 + αdt global and local trend

ARMA
bt = (1 − φ)b + φbt−1 + βdt
p Trend X q
X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1Seasonal j=1
ki s(i)
j,t =
(i)
sj,t−1 λj(i) + s∗(i)
cosFourier-like sin λ(i)
j,t−1 seasonal
(i)
j + γterms
1 dt
s(i) (i)
X
t = sj,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
Complex seasonality

gasoline %>% tbats() %>% forecast() %>% autoplot()

Forecasts from TBATS(1, {0,0}, −, {<52.18,12>})
10

level
80
.

8
95

1990 2000 2010 2020

Time
8
Complex seasonality

calls %>% tbats() %>% forecast() %>% autoplot()

Forecasts from TBATS(0.555, {0,0}, −, {<169,6>, <845,4>})
800

600

level
400 80
.

95

200

0 10 20 30
Time
9
Complex seasonality

telec %>% tbats() %>% forecast() %>% autoplot()

Forecasts from TBATS(0.005, {4,2}, −, {<7,3>, <354.37,7>, <365.25,3>})

25

level
20
80
.

95

15

10

2000 2004 2008

Time
10
TBATS model

TBATS
Trigonometric terms for seasonality
Box-Cox transformations for heterogeneity
ARMA errors for short-term dynamics
Trend (possibly damped)
Seasonal (including multiple and non-integer periods)
Handles non-integer seasonality, multiple seasonal
periods.
Entirely automated
Prediction intervals often too wide
11
Very slow on long series
Outline

1 Complex seasonality

2 Vector autoregression

3 Neural network models

4 Bootstrapping and bagging

12
Outline

1 Complex seasonality

2 Vector autoregression

3 Neural network models

4 Bootstrapping and bagging

13
Neural network models

Simplest version: linear regression

Input Output
layer layer
Input #1
Input #2
Output
Input #3
Input #4

14
Neural network models

Simplest version: linear regression

Input Output
layer layer
Input #1
Input #2
Output
Input #3
Input #4
Coefficients attached to predictors are called “weights”.
Forecasts are obtained by a linear combination of inputs.
Weights selected using a “learning algorithm” that
14
minimises a “cost function”.
Neural network models

Nonlinear model with one hidden layer

Input Hidden Output
layer layer layer
Input #1
Input #2
Output
Input #3
Input #4

15
Neural network models

Nonlinear model with one hidden layer

Input Hidden Output
layer layer layer
Input #1
Input #2
Output
Input #3
Input #4

A multilayer feed-forward network where each layer

of nodes receives inputs from the previous layers.
15
Inputs to each node combined using linear
Neural network models

Inputs to hidden neuron j linearly combined:

4
X
z j = bj + wi,j xi .
i=1
Modified using nonlinear function such as a sigmoid:
1
s(z) = ,
1 + e−z
This tends to reduce the effect of extreme input
values, thus making the network somewhat robust to
outliers.

16
Neural network models

Weights take random values to begin with, which

are then updated using the observed data.
There is an element of randomness in the
predictions. So the network is usually trained
several times using different random starting
points, and the results are averaged.
Number of hidden layers, and the number of
nodes in each hidden layer, must be specified in
advance.

17
NNAR models

Lagged values of the time series can be used as

inputs to a neural network.
NNAR(p, k): p lagged inputs and k nodes in the
single hidden layer.
NNAR(p, 0) model is equivalent to an
ARIMA(p, 0, 0) model but without stationarity
restrictions.
Seasonal NNAR(p, P, k): inputs
(yt−1 , yt−2 , . . . , yt−p , yt−m , yt−2m , yt−Pm ) and k
neurons in the hidden layer.
NNAR(p, P, 0)m model is equivalent to an
ARIMA(p, 0, 0)(P,0,0)m model but without
18
stationarity restrictions.
NNAR models in R

The nnetar() function fits an NNAR(p, P, k)m

model.
If p and P are not specified, they are
automatically selected.
For non-seasonal time series, default p = optimal
number of lags (according to the AIC) for a linear
AR(p) model.
For seasonal time series, defaults are P = 1 and p
is chosen from the optimal linear model fitted to
the seasonally adjusted data.
Default k = (p + P + 1)/2 (rounded to the nearest
integer). 19
Sunspots

Surface of the sun contains magnetic regions

that appear as dark spots.
These affect the propagation of radio waves and
so telecommunication companies like to predict
sunspot activity in order to plan for any future
difficulties.
Sunspots follow a cycle of length between 9 and
14 years.

20
NNAR(9,5) model for sunspots
fit <- nnetar(sunspotarea)
fit %>% forecast(h=20) %>% autoplot()
Forecasts from NNAR(9,5)
3000

2000
sunspotarea

1000

1900 1950 2000

Time 21
Prediction intervals by simulation
fit %>% forecast(h=20, PI=TRUE) %>% autoplot()
Forecasts from NNAR(9,5)
3000

2000
sunspotarea

level
80
95
1000

1900 1950 2000

Time
22
Outline

1 Complex seasonality

2 Vector autoregression

3 Neural network models

4 Bootstrapping and bagging

23