International Journal of Forecasting 37 (2021) 949–970

Contents lists available at ScienceDirect

International Journal of Forecasting

journal homepage: www.elsevier.com/locate/ijforecast

U-Convolutional model for spatio-temporal wind speed

forecasting
∗
Bruno Quaresma Bastos a , Fernando Luiz Cyrino Oliveira a , , Ruy Luiz Milidiú b
a
Department of Industrial Engineering, Pontifical Catholic University of Rio de Janeiro, RJ, Brazil
b
Department of Informatics, Pontifical Catholic University of Rio de Janeiro, RJ, Brazil

article info a b s t r a c t

Keywords: The increasing penetration of intermittent renewable energy in power systems brings
Convolutional neural networks operational challenges. One way of supporting them is by enhancing the predictability
Spatio-temporal forecasting of renewables through accurate forecasting. Convolutional Neural Networks (Convnets)
Renewable energy
provide a successful technique for processing space-structured multi-dimensional data.
Deep learning
In our work, we propose the U-Convolutional model to predict hourly wind speeds for
Time series
a single location using spatio-temporal data with multiple explanatory variables as an
input. The U-Convolutional model is composed of a U-Net part, which synthesizes input
information, and a Convnet part, which maps the synthesized data into a single-site
wind prediction. We compare our approach with advanced Convnets, a fully connected
neural network, and univariate models. We use time series from the Climate Forecast
System Reanalysis as datasets and select temperature and u- and v-components of
wind as explanatory variables. The proposed models are evaluated at multiple locations
(totaling 181 target series) and multiple forecasting horizons. The results indicate that
our proposal is promising for spatio-temporal wind speed prediction, with results that
show competitive performance on both time horizons for all datasets.
© 2020 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

1. Introduction such as wind and solar energy, has been continuously

Society is moving towards adopting cleaner energy
solutions for power generation, prioritizing the use of
resources which are sustainable on a long-term basis,
and which do not contribute to climate change. Policies
and regulations worldwide have created a basis for the
growth of renewables in the form of financial incentives
(e.g., feed-in tariff policies), penetration targets, and other
mechanisms (e.g., renewables quotas in energy portfo-
lios). As a consequence of the execution of such poli-
cies, the penetration of intermittent renewable energy,
∗ Corresponding author.
E-mail addresses: brunoq.b@aluno.puc-rio.br (B.Q. Bastos),
cyrino@puc-rio.br (F.L. Cyrino Oliveira), milidiu@inf.puc-rio.br
(R.L. Milidiú).
increasing (IEA, 2017).
The increased penetration of intermittent renewables
brings new challenges to modern power systems (Le Xie
et al., 2011). Higher variability in power generation may
increase the need for operating reserves and further re-
quire better controls for balancing very-short-term gen-
eration and demand (Le Xie et al., 2011), hence incurring
higher operational costs. One way to support operational
planning and reduce operational costs is to enhance the
predictability of renewable power generation via accurate
and informative forecasting.
Many studies have addressed the issue of wind speed
and wind power forecasting (for a review, refer to Jung
and Broadwater (2014), Lei, Shiyan, Chuanwen, Hongling,
and Yan (2009), Vargas et al. (2019), Zhu and Genton
(2012), and the references therein), and, more recently, of
solar irradiance and solar power forecasting (Antonanzas

https://doi.org/10.1016/j.ijforecast.2020.10.007
0169-2070/© 2020 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
et al., 2016; David, Luis, & Lauret, 2018). Technological
advances in metering and communication have led to
the increasing availability of data measured at different
locations over time; as a consequence, spatio-temporal
approaches have begun to gain attention in forecasting
practices (Tascikaraoglu et al., 2016). The use of spatially
distributed information in modeling has improved the ac-
curacy of temporal forecasts. For example, in Dowell and
Pinson (2016), the spatio-temporal approach of sparse
vector auto-regression (sVAR) outperformed single-site
model benchmarks such as auto-regressive (AR) and vec-
tor auto-regressive (VAR) models with regard to the task
of one-step-ahead wind power forecasting. In Persson,
Bacher, Shiga, and Madsen (2017), the spatio-temporal
approach of gradient boosted regression trees (GBRTs)
provided competitive results from solar power forecasts
compared to the single-site benchmark models of AR and
GBRT, among others.
The notion that the use of spatio-temporal information
may improve wind and solar power generation forecasts
is quite reasonable, since wind and solar power gener-
ation are affected by meteorological conditions, which
may span across large regions. In this sense, for exam-
ple, the use of information from nearby meteorological
stations could be useful to predict information at a tar-
get wind farm. Additionally, the use of spatio-temporal
information on meteorological variables such as humidity,
temperature, precipitation, and so forth, may also be
useful in the process of wind and solar modeling. Statis-
tical methods and machine-learning methods have been
adopted to forecast renewable power generation with
spatio-temporal approaches, some of which consider in-
formation on meteorological variables (e.g., Persson et al.
(2017), Tascikaraoglu, Sanandaji, and Chicco et al. (2016)).
Examples of statistical methods used in spatio-temporal
forecasting include the compressive spatio-temporal fore-
casting (CSTF) model (Tascikaraoglu, Sanandaji, & Chicco
et al., 2016), the spatio-temporal vector auto-regressive
model (André, Dabo-Niang, Soubdhan, & Ould-Baba, 2016),
the sparse vector auto-regressive model (sVAR) (Dow-
ell & Pinson, 2016), the trigonometric direction diurnal
model (TDD) (Xie, Gu, Zhu, & Genton, 2014), and oth-
ers (Aryaputera, Yang, Zhao, & Walsh, 2015; He, Yang,
Zhang, & Vittal, 2014; Koivisto et al., 2016). Examples of
machine learning methods include ensembles of decision
trees (DTs) and support vector regression (SVR) (Heiner-
mann & Kramer, 2016), gradient boosted regression trees
(GBRTs) (Persson et al., 2017), and a fuzzy model (Da-
mousis, Alexiadis, Theocharis, & Dokopoulos, 2004).
Recently, deep learning frameworks and techniques
have been gaining ever more attention in different areas
(e.g., see Li, Shang, and Wang (2018), Liu, Luo, Lian, and
Gao (2018), Shi et al. (2015)) due to breakthroughs in
computer vision, machine translation, speech recognition,
and other complex tasks. Convolutional neural networks
(Convnets) have been one of the techniques responsible
for breakthroughs in computer vision (see Krizhevsky,
Sutskever, and Hinton (2012)). Convnets are neural net-
works specially designed to process multi-dimensional
data, within which the ordering of the elements matters
– such as images (2D arrays). Advances in Convnets’ el-
ements (e.g., factorized convolutions (Chollet, 2016) and
950
International Journal of Forecasting 37 (2021) 949–970
residual mappings (He, Zhang, Ren, & Sun, 2016a)) have
allowed further improvements to be made to the tech-
nique.
Spatio-temporal prediction using models that are based
on Convnets is an ongoing research topic in many areas
such as meteorology (Shi et al., 2015) and computer
vision (Denton & Birodkar, 2017; Mathieu, Couprie, &
LeCun, 2015; Villegas, Yang, Hong, Lin, & Lee, 2017).
Shi et al. (2015) proposed a model, known as ConvL-
STM, which extends recurrent neural networks (RNNs)
with Long Short-Term Memory (LSTM) cells (Hochreiter &
Schmidhuber, 1997) to perform convolutional operations.
The model was used to solve precipitation nowcasting and
video frame prediction. Mathieu et al. (2015) proposed a
multi-scale framework based on Convnets and adversarial
training (Goodfellow et al., 2014) to predict future video
frames. More recent work on video frame prediction sug-
gests decomposing video into content and motion (Den-
ton & Birodkar, 2017; Villegas et al., 2017); moreover
this work combines a multitude of techniques such as
Convnets, RNNs, and adversarial training. U-Net (Ron-
neberger, Fischer, & Brox, 2015), a technique originally
developed for image segmentation, has recently been
used for video frame prediction (Liu, Luo et al., 2018).
Convnets have already been adopted in the realm of
renewable energy prediction. Zhu, Li, Mo, and Wu (2017)
propose a Convnet that provides univariate wind power
predictions by rearranging the univariate data into a 2-D
vector. Liu, Mi, and Li (2018) decompose wind speed data
into low- and high-frequency series. The low-frequency
series is predicted via Convnet with an LSTM layer on
top of it, whereas the high-frequency series are predicted
with standard Convnets. Chen, Zhang, Zhang, Peng, and
Cai (2019) uses a combination of multi-factor correlation,
Convnets, and LSTM to predict wind speeds at a target
site. In Chen et al. (2019), the authors build a 3D matrix
containing meteorological factors for all sites at histori-
cal times and use Convnets to extract spatial features of
meteorological factors at various sites and times. LSTM is
then used to extract temporal features.
This work proposes an architecture which combines
U-Net and standard Convnet: the U-Convolutional model.
In this architecture, the U-Net – recently used to predict
video frames (see Liu, Luo et al. (2018)) – is used to
synthesize and produce high-level features from spatio-
temporal data; the high-level features acquired with
U-Net are fed into a Convnet, which produces the wind
speed prediction. The proposed architecture is trained
jointly; this means that the high-level features obtained
via U-Net are produced by optimizing a single value
(the wind speed error for a single coordinate). In the
context of this work, the addition of calendar variables
and identity mappings (He et al., 2016a) to the pro-
posed U-Convolutional model are also investigated. The
proposed U-Convolutional model is compared to other
architectures which use convolutional layers as a basis
(such as standard Convnets and Convnets built with sim-
plified Inception modules (Chollet, 2016)), fully connected
neural networks with spatio-temporal information, and
benchmark univariate models (such as Box & Jenkins (Box
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
& Jenkins, 1976) and BATS (Livera, Hyndman, & Sny-
der, 2011)). In the convolutional-based models, each in-
put channel receives one explanatory variable (Maçaira,
Thomé, Oliveira, & Ferrer, 2018) that is a spatio-temporal
process.
As well as proposing an architecture which differs
from the one in Chen et al. (2019), this work also differs
from Chen et al. (2019) in another aspect: meteorologi-
cal variables (temperature and u- and v-components of
wind) are adopted to directly map wind speed at a sin-
gle site. Specifically, the proposed architecture learns the
relations between the exploratory variables and output
data without being given any specific information on the
correlation between the variables. The contributions of
this work are the following:
• The proposition of a deep learning architecture that
combines U-Net architecture and Convnet to process
spatio-temporal information from multiple variables
and forecast wind speeds at a single location;
• The investigation of Convnet architectures which
include simplified inception modules, identity map-
pings, and calendar variables in the task of spatio-
temporal wind speed predictions;
• A comparison of the proposed and Convnet architec-
tures with fully connected neural network modeled
in a spatio-temporal setting and against traditional
univariate models such as BATS (Livera et al., 2011),
Box and Jenkins (1976), and Naïve models;
• Outlining a way forward in modeling wind speeds
with multiple explanatory data.
This paper is structured as follows: after the in-
troductory remarks, the fundamentals of the proposed
U-Convolutional model are presented in Section 2. Next, in
Section 3, the proposed approach to spatio-temporal wind
prediction with Convnets is outlined, and the other Con-
vnet architectures are described. Section 4 presents the
dataset and the experimental set-up adopted in this study.
In Section 5, the results are presented and discussed; the
proposed U-Convolutional models are compared against
other Convnet architectures, fully connected neural net-
work, the BATS model, the ARIMA model, and the Naïve
model. The final Section provides the conclusions of this
work and perspectives for future work.
2. Fundamentals
This work proposes an architecture which combines
U-Net and Convnet to process spatio-temporal data re-
lated to multiple explanatory variables and to predict
wind speeds at a single site. Before detailing the approach,
the elements that comprise the proposed architecture are
introduced.
2.1. Feature extraction in convolutional layers
Let X be a 2-D input array; i.e., X = (xi,j ). The array
X may be the values of an input plane preceding a given
convolutional layer, where i and j are axes which retain
the spatial disposition of values in the 2-D input array
(e.g., height and width axes of an image). Furthermore,
let W be a 2-D filter with M elements in the i-axis and
951
International Journal of Forecasting 37 (2021) 949–970
N elements in the j-axis; i.e., W = (wi,j ) ∈ RM ×N . Then,
the convolution operation of the filter W over the 2-D
input array X may be described as in Eq. (1) (Goodfellow,
Bengio, & Courville, 2017).1
M −1 N −1
∑∑
Si,j = (X ∗ W )i,j = Xi+m,j+n Wm,n (1)
m=0 n=0
The output Si,j in Eq. (1) is related to one single unit in
the feature map of a convolutional layer. After performing
the convolution operation for all units in the given feature
map, we obtain the number of units in the resulting
feature map, S = (Si,j ).
In deep learning, inputs to Convnets are usually 3-D
arrays with height, width, and depth axes. In images, the
height and width axes are related to the spatial disposi-
tion of pixels. The depth axis, on the other hand, is related
to the number of color channels of an image. To account
for a 3-D input array, the kernel of the convolutional
layer is made volumetric, with each channel of the kernel
performing a convolution operation on a specific channel
of the input array (Dumoulin & Visin, 2016). To form a
feature map, the output of each convolution operation on
the channels is summed, and then a bias term is added
(element-wise) to each resulting unit in the spatial grid.
After adding the bias term, a non-linearity is applied
element-wise to the resulting units in the feature map.
By doing this, the convolutional layer is able to map
features which are non-linear transformations of the in-
put. In deep learning, a usual non-linearity is the ReLU
activation (Glorot, Bordes, & Bengio, 2011), defined as
g(z) = max(0, z). This activation function has a number
of properties which are desirable for training deep neural
networks and extracting features such as the production
of sparse representations. As discussed in LeCun, Bengio,
and Hinton (2015), the ReLU activation excels when many
layers are used in neural networks.
2.2. Pooling layers
The pooling layer, also known as the subsampling
layer (LeCun, Bottou, Bengio, & Haffner, 1998), reduces
the dimension of the feature map by compressing all the
information on several units from a local neighborhood
of a feature map into one single value. This procedure
makes the pooling layer invariant to small translations
and distortions in the data (Goodfellow et al., 2017; LeCun
et al., 2015, 1998). A pooling layer may take, for instance,
the maximum value or the average value of units in the
local neighborhood, which are respectively known as max
pooling and average pooling
Let there be a pooling layer of size P × P, which is
applied to a single output feature map of a convolution
layer of size Q × Q . Then, the output of the pooling
operation, for a stride of size Ns , is a map of size (((Q −
P)/Ns ) + 1) × (((Q − P)/Ns ) + 1).
1 As explained in Goodfellow et al. (2017), Eq. (1) shows the cross-
correlation function. For further details, please refer to Goodfellow et al.
(2017).
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
2.3. Fully connected layers
Fully connected (FC) layers are usually the final layers
in a Convnet, and are stacked on top of modules of con-
volutional layers and pooling layers. Since the modules of
convolutional layers and pooling layers extract features
from data, FC layers act directly upon the extracted fea-
tures. By doing this, FC layers map the extracted features
of the data to the final target of the Convnet.
Each one of the neurons in an FC layer is connected to
all the neurons in the preceding layer, and each connec-
tion is associated with a trainable parameter (weight) plus
a bias term. Therefore, the neuron in the FC layer may be
computed by the dot product of inputs and its respective
weights, followed by the addition of a bias term. Let a
neuron in an FC layer have n inputs, let f be an activation
function, b be the bias term, and let xi and wi be the input
and weight values respective to that neuron’s
∑n ith input.
Then, the neuron may be computed as f (( i=1 wi xi ) + b).
The configuration of the Convnet’s output layer will
depend on the task being solved by the network. For
example, if we intend to solve a multi-class classification
with N output classes, the output layer may be designed
to have N neurons with a softmax activation function. In
the case of wind speed prediction for a single location site,
the output layer will have only one neuron.
2.4. Identity mappings
In He et al. (2016a), one of the most prominent deep
learning architectures (Gu et al., 2018) was proposed:
the ResNet. The architecture, which aimed to address
the degradation problem of very deep neural networks,
created residual blocks so that the layers of the neural
network would perform residual mapping. The hypothesis
was that the neural network would be easier to optimize
with layers that performed residual mappings. Without
residual mapping, the neural network would have to learn
unconstrained mappings, and learning the unconstrained
mappings would be harder.
Formally, the residual mapping may be formulated as
follows:
y = x + F (x, Wi ) (2)
where y and x are the output and input vectors of the
layer(s) performing residual mapping. Wi is the parameter
vector related to the mapping performed by the layer(s),
and F (x, Wi ) is the function that will be learned by the
residual mapping.
2.5. U-Net
Convnets were originally developed to solve image
classification tasks where the output was a class label
(e.g., CIFAR, ImageNet). However, many other visual tasks
require pixel-by-pixel classification with very limited data
(e.g., fewer than thousands of samples). The U-Net (Ron-
neberger et al., 2015) was proposed in the context of
biomedical image segmentation, and addresses these is-
sues of the traditional Convnets, producing more precise
segmentation with very few training examples.
952
International Journal of Forecasting 37 (2021) 949–970
Introduced in 2015, the U-Net is a special kind of Con-
vnet, which takes its name from the shape of the neural
network architecture. The U-Net is based upon the fully
convolutional network presented in Long, Shelhamer, and
Darrell (2014) and takes only convolutional, subsampling,
and upsampling layers. The architecture presents a con-
tracting path (convolution followed by downsampling)
which is used to extract high-level features from the
input image, followed by an expansive path (upsampling
followed by convolution), which creates a high-resolution
segmentation map (for more details, refer to Ronneberger
et al. (2015)).
3. Methodology
The use of spatial information has been shown to en-
hance the predictability of wind (Dowell & Pinson, 2016;
Persson et al., 2017; Tascikaraoglu, Sanandaji, & Chicco
et al., 2016). Thus, adopting more advanced techniques,
which process spatio-temporal information and predict
wind speed, may lead to better tools for power systems
operations. Convnets are a powerful technique for pro-
cessing spatial data. This work develops architectures
based on convolutional and fully convolutional neural
networks to leverage spatio-temporal information about
multiple explanatory variables with the goal of predicting
wind speeds at a single location.
Let there be explanatory variables {X1 , . . . , XM } which
are spatio-temporal processes. Then, each explanatory
variable is a random variable which may take values in
location s ∈ S and time t ∈ T , where S is the spatial
domain (in our case, S ⊂ R2 ) and T ⊂ R is the time
domain. In the context of wind prediction, potential ex-
planatory variables are temperature, pressure, presence of
rain, the components u and v of wind, and others.
To process spatio-temporal variables with Convnets,
this work proposes assigning each explanatory variable to
one of the input channels of the Convnet. Considering this
set-up, for a given input channel, the height and width
axis of the Convnets are going to represent latitude and
longitude, respectively, and each value at a given position
in the input channel will be the value of the associated
explanatory variable at the respective latitude-longitude
pair. Moreover, all values in a given input channel will
be related to the same time step. For clarity, let the
input variable X1 be written as X1,t . One could add in-
formation about a given variable at different time steps
by including lags of the spatio-temporal variables in the
input channels. Additionally, one could consider multi-
ple explanatory variables in the input channels. Let l ∈
{0, . . . , L} denote lagged steps and let m ∈ {1, . . . , M } de-
note explanatory variables; the multivariate input to the
Convnets, Xt , may be written as Xt = {Xm,t −l }.
Considering this setting, the objective is to use in-
formation about the multivariate 3D input tensor, Xt ,
to predict a single value yt +h , which denotes the wind
speed h-steps ahead of time for a single site (i.e., for a
single latitude-longitude coordinate). This work proposes
the use of neural networks to perform such a mapping.
Specifically, deep learning techniques are adopted to de-
sign neural network architectures that extract high-level
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
features from the multi-dimensional tensors and map
the extracted features into the output variable in R. The
backbones of the proposed architecture are the U-Net and
Convnet architectures.
3.1. U-Convolutional models
The U-Net maps an image to a segmentation mask,
solving pixel-by-pixel classification. It is a powerful tech-
nique for image segmentation. More recently, the U-Net
was adopted to predict future frames in videos. Specifi-
cally, in Liu, Luo et al. (2018), an adversarial framework is
adopted, and the generator, which produces the predic-
tion for the next frame, is a U-Net. As pointed out in Liu,
Luo et al. (2018), the U-Net is selected as the predictor due
to its good performance in image-to-image translation.
The inputs to the U-Net in Liu, Luo et al. (2018) are frames
in present and past times; the output is the frame at one
step ahead of the present time.
In the proposed architecture, which will be referred to
as U-Conv model, the U-Net is used differently, since its
final layer is not associated with a target tensor (e.g., seg-
mentation mask, image, or video frame). Instead, the final
layer of the U-Net is connected to a Convnet, which out-
puts a single value (i.e., wind speed at h-step ahead of
time). Consequently, the U-Net’s final layer is a feature
map that is obtained as a result of the optimization pro-
cess that minimizes weights considering only a single
output variable. With this setting, we design the U-Net
to map the 3D input tensor, Xt , to a 3D output tensor
that has the same dimensions as the input tensor. This
output tensor is connected to a Convnet-based architec-
ture, which produces the final single-value output for our
problem (the wind speed prediction).
3.1.1. Input design
In the U-Conv model design for wind speed prediction,
the u-component of wind, v -component of wind, and the
temperature at times t, t − 1, and t − 2 are considered as
input variables. Let these three variables be, respectively,
U, V , and K . One may write the input tensor as Xt =
{Ut −l , Vt −l , Kt −l }, l ∈ {0, 1, 2}. Hence, Xt totals nine input
variables. Each variable is assigned to one input channel.
Thus, the input array has the shape nlatitude × nlongitude × 9,
which accounts, respectively, for the number of latitude
points (height axis), number of longitude points (width
axis), and the three explanatory variables at times t, t − 1,
and t − 2 (channels axis).
The output variable is the wind speed at the time
(i,j)
t + h for a given (i, j) coordinate; i.e., yt +h . The wind
speed is composed of the u- and v -components of wind,
which is why these variables are selected as exploratory
(i,j) (i,j)
variables for the proposed task. Let ut and vt be the
u- and v -components of wind for a given (i, j) coordinate
at time t. Then, the wind √ speed for the same coordinate
(i,j) (i,j) (i,j)
is calculated as yt = (ut )2 + (vt )2 .
Fig. 1 illustrates the solution that is proposed for wind
speed spatio-temporal prediction with the U-Conv model.
In (A), the variables that comprise the input tensor are
illustrated: variables U, V , and K at times t, t − 1, and t − 2.
In (B), the structure of the input tensor is detailed: each
953
International Journal of Forecasting 37 (2021) 949–970
2D tensor in (A) is concatenated in the channel axis to
form the input tensor of dimensions nlatitude × nlongitude × 9.
Illustration (C) shows the U-Conv model overall structure:
a model that is composed of two parts – a U-Net model
and a Convnet-based model. The U-Conv model outputs a
single value (h-step-ahead wind speed for a given (i, j) co-
ordinate). In (D), the target variable of a single coordinate
is detailed.
3.1.2. U-Net design
The design of the U-Net part of the U-Conv model fol-
lows the structure proposed in Ronneberger et al. (2015).
However, in the present work, the spatial resolution of
the input and the output is smaller than the one in Ron-
neberger et al. (2015): the datasets used in this work have
a spatial dimension of 10 × 10 and 9 × 9 (accounting for
specific regions in Brazil), whereas in Ronneberger et al.
(2015), the spatial dimension is 512 × 512 (accounting
for medical images with a 512 × 512 pixel resolution).
In Ronneberger et al. (2015), the contracting path is com-
posed of 3 × 3 convolutions and 2 × 2 maxpooling with
a stride of 2. Considering the reduced spatial information
in our input setting, the proposed architecture is designed
with 2 × 2 convolutions and 2 × 2 pooling with a stride
of 1 in the contracting path. Moreover, despite following
the same idea of doubling the number of filters at each
2 × 2 subsampling, the proposed design also differs in
terms of the number of filters at the convolutional layers
(instead of the 64 filters in the first convolutional layer,
the proposed model uses 24 or 48 filters, which are se-
lected in a hyperparameter grid search). The expansive
path follows the idea of the original work: upsampling
of feature maps followed by convolution that halves the
number of features, which are concatenated with output
maps from the contracting path, and then followed by two
convolutional layers. The expansion path is symmetric to
the contracting path in terms of the number of filters in
the convolutional layers.
3.1.3. Convnet design
To build the Convnet part of the model, convolu-
tion and subsampling layers are combined, increasing the
number of feature maps (representational power) as the
size of the feature maps (resolution) decreases (LeCun
et al., 1998). Considering this setting, a module com-
prising two convolutional layers followed by one pooling
layer is created. Two such modules are stacked to act as
the feature extractor of the Convnet part of the model -
this design is inspired by the first layers of the VGG-19
model (He et al., 2016a; Simonyan & Zisserman, 2014).
The Convnet part of the proposed model ended up
with 8 to 9 layers. The first two layers were convolutional
layers with kernels sized 2 × 2; the third layer was a
pooling layer sized 2 × 2. The fourth and fifth layers
were convolutional layers with kernels sized 2 × 2. The
sixth layer was a pooling layer sized 2 × 2. The first six
layers are the ones that perform the feature extraction.
After the sixth layer, the output map is reshaped into
a one-dimensional vector. The one-dimensional vector is
then fed to a fully connected layer (the seventh layer of
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. 1. Illustration of input-output mapping with U-Conv model.
the Convnet design). The last layer is the output layer
– comprising a single neuron with linear activation.2
A dropout layer between the fully connected layer
and the output layer may be included in the Convnet
design. The adoption of the dropout layer is treated as
a hyperparameter choice; since other techniques that
prevent overfitting are adopted (e.g., early stop in train-
ing), a dropout might not be needed. The hyperparam-
eter choices are made through experiments, which are
described in Section 4.
3.1.4. Model variants
A second version of the U-Conv model is proposed,
named residual U-Conv model (or U-ConvRes), where
residual connections are added to the Convnet part of
the model. The residual connections are added in every
convolutional module of the Convnet. Specifically, the
U-ConvRes model adopts a variant of the identity map-
ping where a 1 ×1 convolution is applied in the residual
connection. This variant is adopted in order to match the
input and output sizes of feature maps (for details, see He
et al. (2016a), He, Zhang, Ren, and Sun (2016b)).
Another model variant investigated in this study is
the one where calendar variables are adopted to cap-
ture seasonality in the proposed problem. The goal in
developing such a variant is to understand through exper-
iments whether adding seasonal indicator variables im-
proves the forecasting accuracy of the proposed models.
Specifically, the calendar variables are dummy variables
for (a) the hour of the day, (b) the month of the year,
and (c) the day of the week, which totals 43 exogenous
variables. These variables are incorporated (via concate-
nation) into the input of the fully connected part of the
U-Conv and U-ConvRes models.
2 The number of filters in the convolutional layers and the number
of neurons in the fully connected layer may vary in the experiments.
Nevertheless, all U-Conv models follow the idea that the number of
filters in the first convolutional module is lower than that in the second
convolutional module.
954
International Journal of Forecasting 37 (2021) 949–970
The full architectures of the U-Conv model and the
variants are illustrated in Figs. 2. In the Figures, the filters
in the first convolutional module of the U-Net part of
the model are referred to as f0; the filters in the first
and second convolutional modules of the Convnet part
of the model are referred to as f1 and f2, respectively.
Fully connected layers are referred to as fc, and the output
layer is named as out. The number of filters and the
regularization are hyperparameter choices. The selected
hyperparameters are specified in Appendix A.
3.2. Convnet and inception models
Following the idea of using explanatory variables in
the channel axis of the 3D input tensor, it was also pos-
sible to develop other Convnet models to solve spatio-
temporal wind speed forecasting. Specifically, this work
develops a standard Convnet, Convnet with identity map-
pings, Convnet with simplified inception modules (for
details, see Chollet (2016)), and Convnet with simplified
inception modules and identity mappings. These archi-
tectures – which we refer to as Convnet, residual Convnet
(or ConvRes), Inception, and residual Inception (or Incep-
tionRes), respectively) – are illustrated in Figs. 3(a), 3(b),
3(c), and 3(d), respectively. By investigating other Con-
vnet architectures, one may be able to identify which
other types of architectures would be suitable for solving
spatio-temporal wind speed forecasting.
In He et al. (2016a), the residual mapping is adopted to
every few layers; for example, the 34-layer residual net-
work has a residual mapping that encompasses two con-
secutive layers. In the architectures tested in this work,
identity mapping is adopted for every convolutional (or
inception) module. This is illustrated in Fig. 3(b) (and
Fig. 3(d)). Similar to the U-ConvRes, the 1 ×1 convolu-
tion variant of the identity mappings are implemented to
create the ConvRes and InceptionRes models.
Concerning the inception-based models, the architec-
ture design substitutes convolutional layers in Convnets
and Residual Convnets in inception modules (see Figs. 3(a)
and (b) versus Figs. 3(c) and (d)). Standard convolution
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. 2. (a) U-Conv; (b) U-ConvRes; (c) U-Conv w/calendar info; (d) U-ConvRes w/calendar info.
attempts to learn cross-correlation and spatial correlation
with its filters (Chollet, 2016). Conversely, the Inception
idea aims to split the processing of cross correlation and
spatial correlation by factoring the convolution into mul-
tiple kernels. In this context, the idea in this work is to
understand whether adopting this decoupling technique
(inception idea) improves spatio-temporal wind forecast-
ing.
955
International Journal of Forecasting 37 (2021) 949–970
As well as experimenting with identity mappings in
Convnet and Inception architectures, this work also tests
seasonal variables in the four models depicted in Fig. 3.
The addition of seasonal variables follows the same prin-
ciples as in the U-Conv model: they total 43 dummy
variables, which are concatenated to the flattened output
of the feature extractor, serving as the input to the fully
connected part of the models.
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. 3. (a) Convnet; (b) ConvRes; (c) Inception; (d) InceptionRes.
3.3. Training
To build the proposed models, a loss function, which
provides a measure of how bad the model is, needs to be
defined. The loss function selected for the model training
was the mean squared error, a popular loss function for
regression tasks (see Eq. (3)). To estimate the models,
we also need to define the learning algorithm that min-
imizes the selected loss function. To search the weights
in the parametric space in the Convnets such that the
loss function of our task is minimized, the Adam al-
gorithm (Kingma & Ba, 2014) was selected. Adam was
selected because it combines properties from Adagrad
(Duchi, Hazan, & Singer, 2011) (dealing with sparse gra-
dients) and RMSprop (dealing with non-stationary objec-
tives), and because Adam is usually considered robust to
hyperparameter choices (Goodfellow et al., 2017).
N
1 ∑
MSE = (yi − ŷi )2
N
i=1
where yi is the true value for the ith example and ŷi is the
predicted value for the ith example.
4. Empirical evaluation
4.1. Datasets
In this work, two samples from the Climate Forecast
System Reanalysis (CFSR) dataset (Saha et al., 2014) are
used to evaluate the proposed architectures. The dataset
was obtained from the Research Data Archive (RDA) web-
site. Both samples from the CFSR dataset contain data
on the u-component of wind, v -component of wind, and
temperature for latitude-longitude pairs. The u- and
v -components of wind were obtained for a specified
height of 10 m above ground, whereas the temperature
was acquired with ground surface level in mind.3 The first
3 For wind generation prospecting, due to differences in wind
behavior for different height levels, special attention should be paid
to the heights of wind measurements when considering the proposed
approach for wind speed modeling.
International Journal of Forecasting 37 (2021) 949–970
sample data for latitude-longitude pairs, where latitudes
range from −9.5◦ to −14.0◦ at a 0.5◦ spatial resolution
and longitudes range from −44.5◦ to −40.0◦ at a 0.5◦ spa-
tial resolution. These coordinates are related to an area
in Brazil comprising, approximately, the entire state of
Bahia. For this purpose, this first sample of the CFSR
dataset is referred to as Bahia Wind.
For the other sample, the data is acquired for latitude-
longitude pairs where latitudes range from −4.191◦ to
−5.826◦ at, approximately, a 0.204◦ spatial resolution,
and longitudes range from −36.000◦ to −37.841◦ at, ap-
proximately, a 0.204◦ spatial resolution. These coordi-
nates are related to an area in Brazil comprising a region
in the State of Rio Grande do Norte, one of the regions
with the highest incidence of constant wind in Brazil. This
second sample of the CFSR dataset is referred to as Rio
Grande do Norte Wind or RN Wind.
The data acquired for each variable at each coordinate
(3) consists of an hourly time series which dates from 2011-
04-01 00:00 until 2017-01-01 0:00 for Bahia Wind, and
from 2011-04-01 00:00 until 2018-03-01 00:00 for Rio
Grande do Norte Wind.
In summary, there are 50,448 hourly data for each one
of the 10 ×10 locations (latitude-longitude pair) for each
of the 3 variables considered for Bahia Wind, and 60,648
hourly data for each one of the 9 ×9 locations for each
one of the 3 variables considered for Rio Grande do Norte
Wind.
In Figs. 4 and 5, the explanatory variables (at the time
step of t = 0 hours) are illustrated in a heatmap for Bahia
Wind and Rio Grande do Norte Wind, respectively.
4.2. Experimental setup
The proposed models were adopted to predict wind
speed one hour and six hours ahead of time for all co-
ordinates of the Bahia Wind and Rio Grande do Norte
Wind datasets. Thus, models were developed for each
(i, j) latitude-longitude pair of each dataset. All models
map a 3D input tensor with spatio-temporal information,
(i,j)
Xt , into a single target value, yt +h , which is the wind
956
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. 4. Heatmaps of meteorological variables for time step t = 0 hours of Bahia Wind.
Fig. 5. Heatmaps of meteorological variables for time Step t = 0 hours of Rio Grande do Norte Wind.
speed h-steps ahead for a given (i, j) coordinate. In the
experiments, the same input tensor, Xt , was adopted for
all target coordinates. Convergence in neural networks
is faster when input data is normalized (LeCun, Bottou,
Orr, & Müller, 2012). Thus, to develop the neural net-
work models, a preprocessing step was adopted where
each input variable in Xt was normalized to achieve an
approximate average of zero and a standard deviation of
one.
Considering the spatial size of Bahia Wind and Rio
Grande do Norte Wind (10 ×10 and 9 ×9, respectively)
and that four modeling variants have been selected for
experiment (design schemes with or without calendar
variables and residual connections) for each coordinate
with four different neural network architectures (U-Conv,
Inception, Convnet, and a benchmark fully connected neu-
ral network detailed below), a great number of models
need to be developed (1600 models for Bahia Wind and
1296 models for Rio Grande do Norte Wind) for each
forecasting horizon.
In such a setting, optimizing the hyperparameters (such
as the learning rate, batch size, number of neurons, and
so forth) using a grid search for each location for all
variants may be highly time consuming. To avoid this, the
following strategy was adopted: a single location is se-
lected for a hyperparameter search on the most simple
model variant (no calendar variables and no residual
connection) for each algorithm. The best hyperparame-
ter configuration would then be adopted for all other
variants of that algorithm in order to train models for
all other coordinates. The best hyperparameter configu-
ration is selected considering only the one-hour-ahead
problem; the same hyperparameter configuration is used
to model the six-hour-ahead problem. Additionally, to
avoid an extensive grid search, the best Convnet con-
figuration (obtained through a hyperparameter search)
was adopted as the Convnet part of the U-Convolutional
957
International Journal of Forecasting 37 (2021) 949–970
models. Consequently, the hyperparameter search of the
U-Convolutional models was restricted to the U-Net part
of the model (as well as learning and regularization hy-
perparemeters). It is possible that these strategies pro-
duce sub-optimal models for the variants (and locations)
we are investigating (since the variants are not subjected
to a hyperparameter search). However, it allows us to
evaluate whether including (a) calendar variables and (b)
residual connections improve models with fixed hyperpa-
rameter designs, and allows us to compare the different
architectures since they are all the subject of the same
strategy.
Before performing the hyperparameter search, Bahia
and RN datasets are split into training and test sets. The
test set contains the last 20% of samples in the datasets
(accounting for 10,089 samples in the case of Bahia Wind
and 12,129 samples in the case of Rio Grande do Norte
Wind). The test set is used to evaluate how well the mod-
els perform with unseen data. To search the hyperparam-
eters of the proposed architectures, 5-fold cross validation
(CV) is applied to the training set (thus, the validation
sets in the CV setting include 20% of the samples of the
training set). The best hyperparameter is the one whose
model returns the lowest average MAE (on the 5-folds
of the CV setting). After deciding on the best hyperpa-
rameters, the training set is divided into fixed training
and validation sets, with the validation set containing
the last 20% of the data from the training set. Following
this, the training of all the variants and architectures is
performed for all coordinates and horizons, optimizing
the parameters (weights) of the architectures based on
the loss on the validation training. After estimating all
models, the predictions are made on the test data, and
the accuracy measures are evaluated.
In the hyperparameter search, regularization tech-
niques (such as dropout (Srivastava, Hinton, Krizhevsky,
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Sutskever, & Salakhutdinov, 2014) and weight regular-
ization) are tested; the goal is to have hyperparameter
configurations which are robust to overfitting. The train-
ing protocol adopted was the same for all architectures
(both in a hyperparameter search and in standard train-
ing). The training was set to have 200 epochs with early
stopping when loss did not decrease (on validation) for
5 epochs. (The early stopping strategy is also a regular-
ization technique.) The batch size was set as 256 for all
algorithms.
4.3. Accuracy measures
The accuracy measure adopted to select the best hy-
perparameter configuration was the Mean Absolute Error
(MAE) (evaluated on the validation set). MAE is a scale-
dependent metric that is very common in forecasting
literature (Hyndman & Koehler, 2006), in general, and in
wind speed prediction (e.g., Hu, Zhang, Yu, and Xie (2016),
Li, Shi, and Zhou (2011), Tascikaraoglu, Sanandaji, Poolla,
and Varaiya (2016)).
To evaluate how well the proposed models perform
with unseen data, other metrics are also adopted. The
idea is to provide different references of error measures
other than only MAE. In this context, the Root Mean
Squared Error (RMSE), the Symmetric Mean Absolute Per-
centage Error (sMAPE), and the Mean Absolute Scaled
Error (MASE) are also provided. The RMSE is selected as a
second option for the scale-dependent measure, whereas
sMAPE and MASE are selected with the aim of provid-
ing percentage-base-error and scaled-error references, re-
spectively. For a more detailed discussion on accuracy
measures for forecasting, see Hyndman and Koehler
(2006).
Let yi and ŷi be, respectively, the true and predicted
values of the ith example. Then, the aforementioned ac-
curacy measures are described as follows:
√ regard to the test set. In this section, the Convnet models
RMSE = MSE
N
1 ∑
MAE = |yi − ŷi |
N
i=1
N ( ) persistence model). The ANN benchmark also uses spatio-
1 ∑ |yi − ŷi |
sMAPE = 200 ×
N |yi + ŷi |
i=1
N
( )
1 ∑ yi − ŷi
MASE =
N 1
∑N
|yi − yi−1 |
i=1 N −1 i=2
5. Results and discussion
The work was developed in Python using Keras (Chollet
et al., 2015) to create and train neural network architec-
tures. The proposed U-Convolutional architectures were
trained on an NVIDIA GTX 1660Ti GPU (the training for a
single coordinate could take up to 15 min, depending on
early stopping and the dataset); the other models were
executed on a CPU with 8 cores (the training of the in-
ception architecture – which presented more parameters
International Journal of Forecasting 37 (2021) 949–970
than Convnet and ANN – can take two or more hours for
a single coordinate).
The first step of the experiments was to select the best
hyperparameters for the simplest model variant of each
algorithm at a single location (center coordinate for both
datasets), as discussed in Section 4.2. For each architec-
ture, a different set of hyperparameter configurations was
tested. In Appendix A, the hyperparameter search is de-
tailed, and the best configurations and respective results
(average MAE on validation folds) are presented (results
for a fully connected neural network benchmark with
spatio-temporal information, detailed below, are also pre-
sented). It is relevant to note that the best hyperparame-
ter configurations of the same algorithms were different
for Bahia Wind and RN Wind. This could either be due
to weight initialization or the intrinsic differences in the
datasets.
The models created for each algorithm differ in terms
of capacity (i.e., total number of parameters). For ex-
ample, regarding the Bahia Wind dataset, the Convnet
model with no calendar variables and no residual con-
nections has 456,353 parameters (weights), while the
U-Convolutional model with no calendar variables and
no residual connections has 993,194 parameters. A model
with a higher capacity may have more room to overfit
the dataset, in which case regularization is especially im-
portant. The best hyperparameter configuration for all the
algorithms considered a weight regularization technique.
Only the Inception architecture had both dropout and
weight regularization as regularization techniques (for
the Bahia Wind dataset). This indicates that, in order to
perform well on the validation folds (and, possibly, to
generalize unseen data effectively), it is crucial that the
architectures are designed with regularization, especially
weight regularization.
The second step of the experiments was to train the
models for all coordinates, considering the best hyper-
parameter configuration found for a single coordinate.
The final step was then to evaluate the models with
(4)
are compared with each other and against more tradi-
tional benchmark models: fully connected neural net-
works (ANN) with 4 hidden layers, the Box & Jenkins
(5) (B&J) model (Box & Jenkins, 1976), the BATS model (Livera
et al., 2011), and the Naive model (also known as the
(6) temporal inputs (the same input variables as the Convnet
models). Additionally, variants with calendar variables
and residual connections are also tested for the ANN
algorithm (which was subject to the same training process
(7)
as the Convnet models - with a hyperparameter search
followed by full training and model evaluation). The last
three methods (i.e., B&J, BATS, and Naïve) were developed
in a univariate forecasting setting (each coordinate has
a wind speed time series, which is used to train the
models). BATS was originally developed to model time
series which exhibit complex seasonal patterns (Livera
et al., 2011), and the B&J method is a traditional and
robust method for univariate time series forecasting. BATS
and B&J were implemented with the forecast package in
the R (Hyndman & Khandakar, 2008) (the B&J model was
fitted with auto.arima function, and the BATS model with
bats function).
958
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
5.1. Global accuracy measures
In order to provide a global view of the performance of
the models, the average value of the accuracy measures
at all coordinates is provided for Bahia Wind and RN
Wind in Tables 1 and 2, respectively. With the results
in Tables 1 and 2, it is possible to answer some of the
questions that inspired the experiments. For example, it
is possible to analyze whether adding calendar variables
improves the forecast accuracy of the proposed models.
Considering the global measure, it is possible to note that
adding calendar variables improves the accuracy of all
architectures for both datasets and forecasting horizons.
For example, in the case of Bahia Wind, the average MAE
of the U-Conv model is 0.2084 with calendar variables
against 0.2255 with no calendar variables for the one-
hour ahead forecast, and 0.5359 against 0.5538 for the
six-hour ahead forecast. The same happens for all other
architectures, indicating that incorporating calendar vari-
ables after the feature extraction improves the forecast
accuracy of the Convnet-based models (and that incorpo-
rating them as input data to ANN models improves their
performance as well). The variants with calendar vari-
ables outperform those without calendar variables despite
not being fully optimized (i.e., the best hyperparameter
configuration was selected based on a grid search of the
variant without calendar variables).
Conversely, the addition of identity mappings is not
unanimous in terms of how it impacts the performance
of the models. For example, U-Conv models (with and
without calendar variables) are slightly improved by the
addition of identity mappings for almost all cases except
for the six-hour-ahead forecast horizon on the Bahia Wind
dataset; this is also the case for the Convnet model with-
out calendar variables. On the other hand, ANN models
are worsened by the addition of residual connections in
almost all cases; Inception with no calendar variables
and Convnet with calendar variables also present similar
behavior to ANN models. It could be hypothesized that
adding identity mappings would improve models with
higher capacities (such as the U-Conv models, which is
more than ten times the size of the ANN on the RN Wind
dataset, and more than 2.5 times the size of the ANN on
the Bahia Wind dataset, as detailed in Appendix A). How-
ever, this might not be case since Inception models also
have a higher capacity (than ANN and Convnet models, for
instance), and present mixed results when comparing the
variant with identity mappings against the one without
it. In this case, by looking only at the global accuracy
measures, conclusive remarks may not be drawn on how
(or if) identity mappings improve forecast accuracy.
Looking at the different algorithms, it is possible to
note that, despite not directly considering the spatial
disposition of the input data (3D input tensors are trans-
formed into a 1D vector), fully connected neural networks
(ANN architectures) are competitive. ANN architectures
have the best global accuracy measures in one-hour-
ahead forecasts for Bahia Wind (ANN and ANNRes,4 both
4 ANNRes is the ANN architecture with the addition of identity
mappings connecting the output of the first hidden layer to the output
of the fourth hidden layer.
959
International Journal of Forecasting 37 (2021) 949–970
with calendar variables and present-best and second-best
accuracies) and for RN Wind (ANN and ANNRes, both with
calendar variables, present the second-best and third-best
accuracies). However, when six-hour-ahead forecasts are
considered, ANN architectures do not present the best ac-
curacy measures, and Convnet-based architectures, which
specialize in processing spatial data, obtain the best re-
sults. Considering this, it may be hypothesized that when
the forecasting horizon grows higher, spatial information
gains more relevance in the modeling set-up (and, since
ANN architectures do not directly take spatial information
into account, ANN is outperformed by Convnet-based
models). Such a hypothesis seems plausible because the
weather spans and moves over large regions.
The proposed U-Convolutional models are competitive
in terms of both forecasting horizons for both datasets.
For example, the proposed U-ConvRes model (with cal-
endar variables) presents the best accuracy measures for
one-hour-ahead forecasting in the RN Wind dataset, and
the third-best global accuracy measures in all other cases.
Additionally, the proposed U-Conv model (with calendar
variables) is ranked in the top 5 in global accuracy mea-
sures in all cases. Specifically, the U-Conv model presents
the second-best results for six-hour-ahead forecasting in
the Bahia Wind dataset.
The standard Convnet model with calendar variables
(and no residual connections) presents the best global
accuracy measures for six-hour-ahead forecasts in both
datasets. This result might indicate that traditional con-
volution is able to leverage different information from
explanatory variables of input channels to build a model
with good performance in the task. Perhaps one could
have expected a more complex type of processing of
the spatial information that outperforms the standard
convolution for six-hour-ahead forecasting in the global
measures. However, it seems clear that the Inception ar-
chitectures (i.e., Inception and InceptionRes) are outper-
formed by the Convnet model (with calendar variables)
on a six-hour-ahead forecasting task. Conversely, looking
at the proposed U-Conv architectures (also with calendar
variables), the global accuracy measures in the six-hour-
ahead problem are quite similar to those of the aforemen-
tioned Convnet model (e.g., the U-Conv model presents
an MAE of 0.5359 and an sMAPE of 0.6994 against the
MAE of 0.5346 and sMAPE 0.6992 of the Convnet model
– for Bahia Wind) – which might indicate that adopting
U-Net may be equally as beneficial as only using standard
convolutional layers. It is important to put into perspec-
tive, however, that the U-ConvRes model with calendar
variables outperforms the Convnet model on one-hour-
ahead forecasting in terms of global accuracy measures.
It may be hypothesized that by adopting the U-Net to
synthesize 3D input data, the features extracted by the
U-ConvRes model are more refined – which might make
the U-ConvRes model more ‘‘stable’’ than the Convnet
model when it comes to modeling multiple coordinates,
also in the short-term case.
5.2. Best models per location
After analyzing global accuracy measures, a second
perspective which might be evaluated is that of model
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú International Journal of Forecasting 37 (2021) 949–970

Table 1
Average accuracy measures (over all coordinates) - Bahia Wind.
Horizon One-hour horizon Six-hours horizon
Model Calendar MAE RMSE SMAPE MASE MAE RMSE SMAPE MASE
ANN – 0.2091 0.2838 8.8800 0.6906 0.5786 0.7555 21.9273 0.5785
ANN Yes 0.1987 0.2700 8.5624 0.6575 0.5674 0.7411 21.6148 0.5677
ANNRES – 0.2126 0.2896 9.0763 0.7021 0.5911 0.7694 22.6331 0.5920
ANNRES Yes 0.2014 0.2755 8.7036 0.6660 0.5791 0.7543 22.2715 0.5807
Convnet – 0.2239 0.3051 9.6481 0.7381 0.5537 0.7206 21.1251 0.5549
Convnet Yes 0.2101 0.2882 9.2198 0.6948 0.5346 0.6992 20.5221 0.5370
ConvRes – 0.2220 0.3026 9.5769 0.7337 0.5542 0.7217 21.1677 0.5553
ConvRes Yes 0.2106 0.2892 9.2777 0.6975 0.5422 0.7083 20.8874 0.5448
Inception – 0.2445 0.3298 10.3200 0.8059 0.5887 0.7626 22.2959 0.5900
Inception Yes 0.2342 0.3171 9.9698 0.7735 0.5768 0.7494 21.9465 0.5794
InceptionRes – 0.2455 0.3314 10.3634 0.8100 0.6006 0.7779 22.7818 0.6022
InceptionRes Yes 0.2330 0.3152 9.9230 0.7700 0.5858 0.7605 22.2649 0.5882
U-Conv – 0.2255 0.3081 9.5343 0.7397 0.5538 0.7196 21.0210 0.5552
U-Conv Yes 0.2084 0.2871 8.9977 0.6858 0.5359 0.6994 20.4807 0.5380
U-ConvRes – 0.2133 0.2921 9.1534 0.7039 0.5590 0.7276 21.2563 0.5605
U-ConRes Yes 0.2017 0.2772 8.8035 0.6676 0.5389 0.7029 20.6847 0.5421
ARIMA – 0.2513 0.3519 10.7021 0.8245 0.8754 1.0943 30.8176 0.8486
BATS – 0.2507 0.3495 10.5057 0.8225 0.8579 1.0707 30.1451 0.8354
Nave – 0.3145 0.4218 12.6965 – 1.0691 1.3309 38.0337 –

Table 2
Average accuracy measures (over all coordinates) - RN Wind.
Horizon One-hour horizon Six-hours horizon
Model Calendar MAE RMSE SMAPE MASE MAE RMSE SMAPE MASE
ANN – 0.1984 0.2844 4.1143 0.5680 0.6203 0.8146 12.0766 0.5021
ANN Yes 0.1845 0.2657 3.8090 0.5290 0.5998 0.7883 11.5990 0.4864
ANNRES – 0.2006 0.2883 4.1848 0.5748 0.6213 0.8152 12.1215 0.5031
ANNRES Yes 0.1851 0.2675 3.8491 0.5306 0.5992 0.7879 11.6114 0.4857
Convnet – 0.2066 0.2948 4.2750 0.5911 0.6064 0.7986 11.8557 0.4905
Convnet Yes 0.1932 0.2779 4.0106 0.5542 0.5752 0.7597 11.1402 0.4663
ConvRes – 0.2047 0.2930 4.2641 0.5866 0.6030 0.7949 11.8076 0.4879
ConvRes Yes 0.1880 0.2733 3.9356 0.5392 0.5754 0.7603 11.1730 0.4664
Inception – 0.2046 0.2928 4.2418 0.5854 0.6160 0.8104 12.0223 0.4984
Inception Yes 0.1926 0.2763 3.9636 0.5519 0.5883 0.7765 11.3996 0.4768
InceptionRes – 0.2041 0.2926 4.2323 0.5841 0.6182 0.8153 12.0437 0.5003
InceptionRes Yes 0.1880 0.2716 3.8875 0.5386 0.5932 0.7835 11.4876 0.4807
U-Conv – 0.2030 0.2883 4.1883 0.5811 0.6097 0.8030 11.8969 0.4932
U-Conv Yes 0.1905 0.2719 3.9170 0.5459 0.5813 0.7670 11.2479 0.4711
U-ConvRes – 0.2001 0.2850 4.1372 0.5723 0.6065 0.7977 11.8275 0.4910
U-ConRes Yes 0.1816 0.2622 3.7530 0.5202 0.5777 0.7616 11.1952 0.4683
ARIMA – 0.2844 0.3933 5.9123 0.8048 1.0384 1.2914 20.1717 0.8291
BATS – 0.2834 0.3890 5.8790 0.8024 0.9819 1.2265 18.8075 0.7882
Nave – 0.3523 0.4756 7.1160 – 1.2798 1.5804 24.7799 –

performance per location. Since there are 100 coordinates
for Bahia Wind and 81 coordinates for RN Wind and
17 algorithms, in order to save space, only the number
of coordinates where each architecture performed best
(i.e., lowest MAE) in the test set is presented in this
section. In Appendix B, the heatmaps of errors (MAE) in
the test set are illustrated for each algorithm, dataset, and
forecast horizon.
Tables 3 and 4 present the number of coordinates
where each algorithm had the lowest MAE per test set
for the Bahia Wind and RN Wind datasets, respectively.
The Diebold Mariano (DM) test was performed for the
null hypothesis whereby the given best model is less
accurate than the other models. In parenthesis in Tables 3
and 4, the same metric is provided, considering a sig-
nificance level of α = 5%. It might be noted that there
is no single model which outperforms in all coordinates.
Overall, the analyzes undertaken that consider global ac-
curacy measures are corroborated by the results shown in
960
Tables 3 and 4: most ‘‘winning’’ models (i.e., those which
are selected as the best models for a greater number of co-
ordinates) have calendar variables, ANN models appear to
be more prominent in the one-hour forecasting problem,
and U-Conv models appear to be in the top tier for one-
hour- and six-hour-ahead forecasting in both datasets.
Specifically, considering the one-hour forecast for the RN
Wind dataset, the U-ConvRes model with calendar vari-
ables is the go-to model for 31 coordinates (almost 40%
of the coordinates) considering the significance level of
α = 5% - presenting the lowest MAE in the test set for
42 coordinates (more than 50% of the coordinates).
One point that is interesting to note is that the Incep-
tionRes model with calendar variables was the best model
for 17 coordinates of the Bahia Wind in one-hour-ahead
forecasting (and had the lowest MAE in the test set for
21 coordinates). However, looking at the global accuracy
measures, it did not appear to be a top-tier model. This
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú International Journal of Forecasting 37 (2021) 949–970

Table 3
Number of coordinates with the lowest MAE in the test set (significant at α = 5%) - Bahia Wind.
One-hour horizon Six-hour horizon
Architecture Calendar Qty Architecture Calendar Qty
InceptionRes Yes 21 (17) U-Conv Yes 33 (10)
U-Conv Yes 19 (7) Convnet Yes 27 (6)
ANN Yes 17 (15) U-ConvRes Yes 18 (7)
U-ConvRes Yes 16 (8) ConvRes Yes 17 (5)
ANNRes Yes 11 (9) ANN Yes 3 (-)
Convnet Yes 8 (2) Convnet – 1 (-)
Inception Yes 3 (-) ConvRes – 1 (-)
ConvRes Yes 3 (1) ANN – - (-)
ANNRes – 1 (1) ANNRes Yes - (-)
U-Conv – 1 (-) ANNRes – - (-)
ANN – - (-) Inception Yes - (-)
Convnet – - (-) Inception – - (-)
ConvRes – - (-) InceptionRes Yes - (-)
Inception – - (-) InceptionRes – - (-)
InceptionRes – - (-) U-Conv – - (-)
U-ConvRes – - (-) U-ConvRes – - (-)
Box & Jenkins – - (-) Box & Jenkins – - (-)
BATS – - (-) BATS – - (-)
Nave – - (-) Nave – - (-)

Table 4
Number of coordinates with the lowest MAE in the test set (significant at α = 5%) - RN Wind.
One-hour horizon Six-hour horizon
Architecture Calendar Qty Architecture Calendar Qty
U-ConvRes Yes 42 (31) ConvRes Yes 25 (7)
ANN Yes 20 (12) Convnet Yes 23 (3)
ANNRes Yes 9 (2) U-ConvRes Yes 18 (2)
InceptionRes Yes 6 (1) U-Conv Yes 8 (1)
U-Conv Yes 2 (-) Convnet – 3 (-)
ConvRes Yes 2 (-) ConvRes – 1 (-)
ANN – - (-) Inception Yes 1 (-)
ANNRes – - (-) InceptionRes Yes 1 (-)
Convnet Yes - (-) U-Conv – 1 (-)
Convnet – - (-) ANN Yes - (-)
ConvRes – - (-) ANN – - (-)
Inception Yes - (-) ANNRes Yes - (-)
Inception – - (-) ANNRes – - (-)
InceptionRes – - (-) Inception – - (-)
U-Conv – - (-) InceptionRes – - (-)
U-ConvRes – - (-) U-ConvRes – - (-)
Box & Jenkins – - (-) Box & Jenkins – - (-)
BATS – - (-) BATS – - (-)
Nave – - (-) Nave – - (-)

means that the model performed well at a few coordi-
nates and performed poorly at other coordinates (this is
illustrated at the Appendix B, where MAE at corner co-
ordinates are especially high; i.e., over 0.40, for Inception
architectures one hour ahead for Bahia Wind). Conversely,
the ANN model is not ranked first for one-hour-ahead
forecasting of Bahia Wind (in Table 3), and it appears to
obtain the best global accuracy measures in this task. This
means that, overall, the model performs well for most
coordinates, despite not being the best model for most of
them. This is also the case for the Convnet model with
calendar variables in a six-hour-ahead forecast task for
both datasets.
The ranking illustrated in Tables 3 and 4 might be
related to different locations in the region grid. At this
point, with an aggregated evaluation of models, it is not
possible to say whether specific algorithms are better at
specific coordinates (or regions). However, it is possible to
see, in Appendix B, that the coordinates at the top-right
961
corner of the grid are usually related to the highest MAEs.
This is probably related to how the wind blows in the
region (from the top right to the bottom left). Convnet
architectures (Convnet and Convres) and U-Conv archi-
tectures (U-Conv and U-ConvRes) with calendar variables
have a better time predicting the top-right corner for
both datasets in a six-hour forecasting horizon. This might
be the reason why (1) these models have better global
accuracy measures, and (2) are selected as best models
more times than the ANN and Inception architectures in
a six-hour-ahead forecasting task.
5.3. Additional remarks
The proposed approach could be applied to a real-
world setting where information (on the meteorological
variables) is updated hourly and the models are executed
hourly to predict h-hours ahead of time. However, real
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
power system operation may require the models to pro-
vide hourly predictions 24 h ahead of time. This is the case
for the Brazilian Power System, where the centralized
operation considers hourly forecasts 24 h ahead of time
to plan daily operations. To apply the proposed approach
in such circumstances, one could create 24 models, where
each model would use any information available at the
time t to predict wind speeds for a given step ahead
(i.e., one model would be developed to predict the wind
speed at a time t + 1, a second model would be developed
to predict the wind speed at a time t + 2, and so on). De-
spite the implementation drawbacks 24 h ahead of time,
the proposition and study of models for h-hours-ahead-
of-time predictions are frequent in the literature, which
indicates the relevance of studying such models (Dowell
& Pinson, 2016; Feng, Cui, Hodge, & Zhang, 2017; Hu
& Wang, 2015). Other designs based on the proposed
U-Conv model may be developed to perform 24-hour-
ahead forecasting (e.g., by making the last layer have 24
output neurons instead of one single output neuron).
6. Conclusions
The increasing penetration of intermittent renewable
energy in modern power systems is a challenge in the
operation of power systems. One way of providing sup-
port to the operation of a system is to develop advanced
models, which provide accurate forecasts for the intermit-
tent resources. In this work, the U-Convolutional model is
proposed. It combines the U-Net architecture and Convnet
to predict wind speed for a single location using multiple
spatio-temporal explanatory variables as the input to the
model. This work also investigates other Convnet archi-
tectures for the same task; these include Convnets with
inception modules and standard Convnets. Additionally,
this work investigates whether the addition of seasonal
indicator variables and identity mappings improve the
models that are part of the experiments.
The results indicate that the proposed U-Conv archi-
tecture is promising, and that adding calendar variables
after feature extraction considerably improves the per-
formance of all neural network architectures tested in
this work. The proposed U-Convolutional architectures
provided competitive global accuracy measures (high best
accuracies for one-hour-ahead forecasting in a dataset,
and third-best accuracies in all other cases), and were
selected as best models for different coordinates at both
time horizons (in a specific case, the U-ConvRes model
was selected as the best model for almost 40% of the coor-
dinates on one-hour ahead-forecasting in a dataset). The
results also indicate that the proposed U-Convolutional
architectures are more stable than the other architectures
when it comes to forecasting different time horizons in
different datasets. This may be due to the higher capabil-
ity of synthesizing input data with a U-Net architecture in
the model design.
In this work, to account for temporality, a modeling
setup where features at present and past times are in-
cluded in the channel axis was adopted. This type of
setup is present in video frame prediction tasks. For fu-
ture work, we propose to account for the temporality in
962
International Journal of Forecasting 37 (2021) 949–970
our task by hybridizing Convnet with Recurrent Neural
Networks (RNNs). RNNs are a type of neural network
that present feedback loops in their architecture, allowing
them to keep, in the network, information about past
inputs. Due to this property, RNNs are well suited to the
model data that are sequentially generated such as time
series, speech signals, and many others.
One topic of interest for future work is related to
designing the proposed spatio-temporal forecasting task
with fully convolutional models, which would map 3D
input tensors to 2D or 3D input tensors, directly providing
a forecast for all coordinates in the region. Considering a
fully convolutional setting, transfer learning studies may
also be beneficial – for example, adopting a segmentation
task (e.g., classifying extreme wind speeds in the region)
to train the fully convolutional neural network, and then
fine-tuning the same neural network in the regression
(prediction) task. Additionally, it would be of special in-
terest to compare the fully convolutional model with the
models that generate reanalysis data.
With these observations, we outline a way forward
in modeling wind speeds with multiple explanatory data
and Convnet modeling.
Acknowledgments
We would like to thank the editor and the reviewers
for their insightful comments, which allowed us to sub-
stantially improve our paper. This work was supported by
the Brazilian Coordination for the Improvement of Higher
Level Personnel (CAPES) under Grant 001, by the Carlos
Chagas Filho Research Support Foundation of the State of
Rio de Janeiro (FAPERJ) under Grants 202.673/2018 and
211.086/2019, the Brazilian National Council for Scien-
tific and Technological Development (CNPq) under Grant
307403/2019-0, and by the International Institute of Fore-
casters IIF/SAS 2017-2018 Grant Award. This material is
also based upon work supported by the Air Force Office
of Scientific Research (AFOSR), USA under award number
FA9550-19-1-0020.
Appendix A. Hyperparameter configurations
In this section, the hyperparameter configuration de-
signs tested in the experiments in this work are listed.
The hyperparameter configurations include (1) architec-
tural information such as the number of filters in first
and second convolutional modules in Convnet designs,
(2) learning protocol information, which includes infor-
mation regarding learning rate, weight decay, batch size,
and early stopping, and (3) regularization techniques such
as dropout and weight regularization. Additionally, this
section provides the best hyperparameter configuration
for each dataset, detailing the total number of parameters
and the average Mean Absolute Error (MAE) on validation
folds. As detailed in Section 4.2, the selection of the best
hyperparameters was made based on experiments at a
single coordinate, located in the center of each dataset’s
grid. Furthermore, the grid search considered only the
one-hour-ahead task.
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
A.1. Bahia wind dataset
A.1.1. Standard convolutional neural networks (Convnet)
For the standard Convolutional Neural Network (Con-
vnet) architecture, a total of 140 hyperparameter configu-
rations were tested. The grid search was performed in two
steps: a full grid search (on 128 configurations), followed
by a refinement phase (grid search on 12 configurations),
where new values were tested for a few hyperparameters.
Below, the values tested for each hyperparameter in the
full grid search are listed.
• activation - convolutional layers: ReLU;
• activation - fully connected layers: ReLU;
• activation - output layer: linear;
• filters - 1st convolutional module: 24, 32;
• filters - 2nd convolutional module: 168, 224;
• pooling function: maximum, average;
• neurons - fully connected layer: 168, 224;
• dropout rate - convolutional layers: None;
• dropout rate - fully connected layer: None, 0.25;
• weight regularization - convolutional layers: None,
0.00005;
• weight regularization - fully connected layers: None,
0.00005;
• early stopping (epochs without loss decrease): 5;
• learning rate: 0.001;
• weight decay: 0.0001;
• batch size: 256;
• epochs: 200.
After performing the grid search on 128 configurations,
an additional round of hyperparameter searches was im-
plemented in order to test a different value of the number
of filters on the first convolutional module (i.e., 48 filters)
and different values for weight regularization. Below, the
list of hyperparameters tested on a new grid search are
detailed.
The best configuration of the Convnet architecture was
the one with 48 filters in the first convolutional mod-
ule, 224 filters in the second convolutional module, 224
neurons on fully connected layers, the average pooling
function, and a weight regularization of 0.00005 on con-
volutional layers and of 0.000025 on fully connected lay-
ers; the other hyperparameters are the ones described
above. The average MAE of validation folds for this hyper-
parameter configuration was 0.2232, and the total num-
ber of parameters for the architecture was 456,353.
• activation - convolutional layers: ReLU;
• activation - fully connected layers: ReLU;
• activation - output layer: linear;
• filters - 1st convolutional module: 32, 48;
• filters - 2nd convolutional module: 224;
• pooling function: average;
• neurons - fully connected layer: 224;
• dropout rate - convolutional layers: None;
• dropout rate - fully connected layer: None;
• weight regularization - convolutional layers:
0.00005, 0.0001;
• weight regularization - fully connected layers: None,
0.000025, 0.00005;
963
International Journal of Forecasting 37 (2021) 949–970
• early stopping (epochs without loss decrease): 5;
• learning rate: 0.001;
• weight decay: 0.0001;
• batch size: 256;
• epochs: 200.
A.1.2. U-Convolutional architecture
The best architecture found for the Convnet (in
Appendix A.1.1) was adopted as a reference for the Con-
vnet part of the U-Convolutional architecture. By doing
this, only hyperparameters related to the U-Net, the learn-
ing protocol, and regularization were subject to a grid
search. In total, 64 hyperparameter configurations were
tested. Below, the values of the hyperparameters tested
on the grid search are listed.
The best configuration of the U-Convolutional model
was the one with 24 filters in the first module of the
U-Net part of the architecture (the other modules follow
the logic of the U-Net design; e.g., doubling values on
the contracting path), no dropout, no weight regulariza-
tion of convolutional layers, and weight regularization
of 0.000025 on fully connected layers, with a learning
rate of 0.001. The average MAE on validation folds for
the best configuration was 0.2082. The best configuration
produced a model with 993,194 parameters (weights).
• activation - U-Net: eLU;
• activation - convolutional layers: ReLU;
• activation - fully connected layers: ReLU;
• activation - output layer: linear;
• filters - 1st convolutional module (U-Net part): 24,
48;
• filters - 1st convolutional module (Convnet part): 48;
• filters - 2nd convolutional module (Convnet part):
224;
• pooling function: average;
• neurons - fully connected layers: 224;
• dropout rate - convolutional layers: None, 0.1;
• dropout rate - fully connected layer: None, 0.25;
• weight regularization - convolutional layers: None,
0.00005;
• weight regularization - fully connected layers: None,
0.000025;
• early stopping (epochs without loss decrease): 5;
• learning rate: 0.001, 0.002;
• weight decay: 0.0001;
• batch size: 256;
• epochs: 200.
A.1.3. Inception architecture
For the Inception architecture, a total of 128 hyper-
parameter configurations were tested. Below, the values
tested for each hyperparameter in the full grid search are
listed. The best configuration for the Inception model was
the one with 72 filters on the first inception module, 504
filters on the second inception module, 168 neurons on
the fully connected layer, max pooling, a dropout rate
of 0.25, and a weight regularization of 0.00005 for con-
volutional and fully connected layers. The average MAE
in validation folds was 0.2250. The best configuration
produced a model with 900,697 parameters.
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú International Journal of Forecasting 37 (2021) 949–970

Fig. B.6. MAE on test set for Bahia Wind (horizon = 1-hour ahead. (For interpretation of the references to colour in this figure legend, the reader
is referred to the web version of this article.)

• activation - convolutional layers: ReLU; • weight regularization - convolutional layers: None,

• activation - fully connected layers: ReLU; 0.00005;
• activation - output layer: linear; • weight regularization - fully connected layers: None,
• filters - 1st inception module: 24, 72; 0.00005;
• filters - 2nd inception module: 168, 504; • early stopping (epochs without loss decrease): 5;
• pooling function: maximum, average; • learning rate: 0.001;
• neurons - fully connected layers: 168, 504; • weight decay: 0.0001;
• dropout - fully connected layers: None, 0.25; • batch size: 256;
• epochs: 200.
964
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú International Journal of Forecasting 37 (2021) 949–970

Fig. B.7. MAE on test set for Bahia Wind (horizon = 6-hours ahead. (For interpretation of the references to colour in this figure legend, the reader
is referred to the web version of this article.)

A.1.4. Fully Connected Neural Networks (ANN) • activation - fully connected layers: ReLU;
For the ANN architecture, a total of 108 hyperparam- • activation - output layer: linear;
eter configurations were tested. Below, the values tested • neurons - 1st and 4th layers: 100, 200;
for each hyperparameter on the full grid search are listed. • neurons - 2nd and 3rd layers: 200, 300;
The best configuration for the ANN model was the one • dropout rate: None, 0.1, 0.25;
with 200 neurons on 1st and 4th layers, 300 neurons on • weight regularization - all layers: None, 0.0001,
2nd and 3rd layers, no dropout, a weight regularization 0.00005;
of 0.0001, and a learning rate of 0.001. The best model • early stopping (epochs without loss decrease): 5;
had an average MAE in validation folds of 0.2081. The best • learning rate: 0.005, 0.001, 0.0005;
configuration produced a model with 391,201 parameters. • weight decay: 0.0001;
965
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. B.8. MAE on test set for RN Wind (horizon = 1 h ahead. (For interpretation of the references to colour in this figure legend, the reader is
referred to the web version of this article.)
• training epochs: 200;
• batch size: 256.
A.2. Rio Grande do Norte wind dataset
A.2.1. Standard convolutional neural networks (Convnet)
For the Rio Grande do Norte (RN) Wind dataset, 128
hyperparameter configurations were tested for the Con-
vnet architecture (different from the Bahia Wind, where
966
International Journal of Forecasting 37 (2021) 949–970
there was no refinement step). The 128 configurations
tested are the same list as in the full grid search for Bahia
Wind (see Appendix A.1.1). The best configuration of the
Convnet architecture was the one with 32 filters in the
first convolutional module, 168 filters in the second con-
volutional module, 224 neurons in fully connected layers,
an average pooling function, a weight regularization of
0.00005 in convolutional layers, and no regularization in
fully connected layers. The average MAE on validation
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Fig. B.9. MAE on test set for RN Wind (horizon = 6 h ahead. (For interpretation of the references to colour in this figure legend, the reader is
referred to the web version of this article.)
folds for this hyperparameter configuration was 0.1910,
and the total number of parameters for the architecture
was 291,025.
A.2.2. U-Convolutional architecture
The best architecture found for the Convnet (see
Appendix A.2.1) was adopted as a reference for the Con-
vnet part of the U-Convolutional architecture. By doing
this, only hyperparameters related to the U-Net, the learn-
ing protocol, and regularization were subject to a grid
967
International Journal of Forecasting 37 (2021) 949–970
search. In total, 64 hyperparameter configuration were
tested. Below, the values of the hyperparameters tested
on the grid search are listed.
The same hyperparameter configurations listed in
Appendix A.1.2 for Bahia Wind were tested for RN Wind.
The best configuration of the U-Convolutional model was
the one with 48 filters in the first module of the
U-Net part of the architecture, no dropout, a weight reg-
ularization of 0.00005 on convolutional layers, a weight
regularization of 0.000025 on fully connected layers, and
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú International Journal of Forecasting 37 (2021) 949–970

Table B.5 B.2. Rio Grande do Norte wind dataset

Model reference on heatmap figures.
ANN ANN ANNRes ANNRes
no calendar calendar no calendar calendar
Below, in Figs. B.8 and B.9, the MAEs for the one-
Convnet Convnet ConvRes ConvRes hour- and six-hour-ahead forecasts of the Rio Grande
no calendar calendar no calendar calendar do Norte Wind are provided. The heatmap ranges from
Inception Inception InceptionRes InceptionRes 0.15 MAE (darkest blue) to 0.30 or higher MAE (darkest
no calendar calendar no calendar calendar
U-Conv U-Conv U-ConvRes U-ConvRes
red) for the heatmaps of the one-hour-ahead horizon, and
no calendar calendar no calendar calendar ranges from 0.45 MAE (darkest blue) to 0.90 or higher
B&J BATS Nave – MAE (darkest red) for the heatmaps of the six-hour-ahead
horizon.

a learning rate of 0.001. The average MAE on validation References

folds for the best configuration was 0.2000. The best con-
figuration produced a model with 2,433,754 parameters.
A.2.3. Inception architecture
For the Inception architecture, the same 128 hyperpa-
rameter configurations listed for Bahia Wind (see
Appendix A.1.3) were tested for RN Wind. The best con-
figuration for the Inception model was the one with 72
filters in the first inception module, 504 filters in the sec-
ond inception module, 504 neurons in the fully connected
layer, max pooling, no dropout, and a weight regular-
ization of 0.00005 for convolutional and fully connected
layers. The average MAE on validation folds was 0.1947.
The best configuration produced a model with 1,070,713
parameters.
A.2.4. Fully Connected Neural Networks (ANN)
For the ANN architecture, the same 108 hyperparame-
ter configurations listed for Bahia Wind (see
Appendix A.1.4) were tested for RN Wind. The best config-
uration for the ANN model was the one with 100 neurons
on the 1st and 4th layers, 300 neurons on the 2nd and 3rd
layers, no dropout, a weight regularization of 0.00005, and
a learning rate of 0.0005. The best model had an average
MAE on validation folds of 0.1879. The best configuration
produced a model with 223,801 parameters.
Appendix B. Mean absolute errors on all coordinates
In this section, the heatmap of Mean Absolute Errors
(MAEs) are provided for each dataset and forecasting hori-
zon. With the heatmaps, it is possible to compare errors in
different architectures for all coordinates of each dataset.
For reference, Table B.5 illustrates the positioning of the
heatmap for each models.
B.1. Bahia wind dataset
Below, in Figs. B.6 and B.7, the MAEs for the one-hour-
and six-hour-ahead forecasts of the Bahia Wind are pro-
vided. The heatmap ranges from 0.14 MAE (darkest blue)
to 0.44 or higher MAE (darkest red) for the heatmaps of
the one-hour-ahead horizon, and ranges from 0.40 MAE
(darkest blue) to 0.78 or higher MAE (darkest red) for the
heatmaps of the six-hour-ahead horizon.
André, M., Dabo-Niang, S., Soubdhan, T., & Ould-Baba, H. (2016). Pre-
dictive spatio-temporal model for spatially sparse global solar radi-
ation data. Energy, 111, 599–608. http://dx.doi.org/10.1016/j.energy.
2016.06.004, URL http://www.sciencedirect.com/science/article/pii/
S0360544216307769.
Antonanzas, J., Osorio, N., Escobar, R., Urraca, R., Martinez-de Pison, F.,
& Antonanzas-Torres, F. (2016). Review of photovoltaic power
forecasting. Solar Energy, 136, 78–111. http://dx.doi.org/10.1016/J.
SOLENER.2016.06.069, URL https://www.sciencedirect.com/science/
article/pii/S0038092X1630250X.
Aryaputera, A. W., Yang, D., Zhao, L., & Walsh, W. M. (2015). Very
short-term irradiance forecasting at unobserved locations using
spatio-temporal kriging. Solar Energy, 122, 1266–1278. http://dx.doi.
org/10.1016/j.solener.2015.10.023, URL http://www.sciencedirect.
com/science/article/pii/S0038092X15005745.
Box, G., & Jenkins, G. (1976). Time series analysis: Forecasting and control.
San Francisco, California: Holden-Day.
Chen, Y., Zhang, S., Zhang, W., Peng, J., & Cai, Y. (2019). Multifac-
tor spatio-temporal correlation model based on a combination
of convolutional neural network and long short-term memory
neural network for wind speed forecasting. Energy Conversion and
Management, 185, 783–799. http://dx.doi.org/10.1016/j.enconman.
2019.02.018, URL http://www.sciencedirect.com/science/article/pii/
S0196890419302006.
Chollet, F. (2016). Xception: Deep learning with separable convolutions.
(pp. 1–14). http://dx.doi.org/10.1109/CVPR.2017.195, arXiv preprint
arXiv:1610.02357.
Chollet, F., et al. (2015). Keras. URL https://keras.io.
Damousis, I. G., Alexiadis, M. C., Theocharis, J. B., & Dokopoulos, P.
S. (2004). A fuzzy model for wind speed prediction and power
generation in wind parks using spatial correlation. IEEE Transactions
on Energy Conversion, 19(2), 352–361. http://dx.doi.org/10.1109/TEC.
2003.821865, URL http://ieeexplore.ieee.org/document/1300701/.
David, M., Luis, M., & Lauret, P. (2018). Comparison of intraday
probabilistic forecasting of solar irradiance using only endogenous
data. International Journal of Forecasting, 34(3), 529–547.
http://dx.doi.org/10.1016/j.ijforecast.2018.02.003, URL https://w
ww.scopus.com/inward/record.uri?eid=2-s2.0-85046850329&doi=1
0.1016%2fj.ijforecast.2018.02.003&partnerID=40&md5=e741d71b53
ed081adbd954a8c81253bc.
Denton, E., & Birodkar, V. (2017). Unsupervised learning of disen-
tangled representations from video. CoRR abs/1705.10915, arXiv:
1705.10915.
Dowell, J., & Pinson, P. (2016). Very-short-term probabilistic wind
power forecasts by sparse vector autoregression. IEEE Transactions
on Smart Grid, 7(2), 763–770. http://dx.doi.org/10.1109/TSG.2015.
2424078.
Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient
methods for online learning and stochastic optimization.
Journal of Machine Learning Research, 12(Jul), 2121–2159.
http://dx.doi.org/10.1109/CDC.2012.6426698, URL http://jmlr.or
g/papers/v12/duchi11a.htmlhttp://dl.acm.org/citation.cfm?id=20210
68%5Cnhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.2
32.1303&rep=rep1&type=pdf#page=265.
Dumoulin, V., & Visin, F. (2016). A guide to convolution arithmetic for
deep learning: Tech. rep., URL https://arxiv.org/abs/1603.07285.

968
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Feng, C., Cui, M., Hodge, B.-M., & Zhang, J. (2017). A data-driven multi-
model methodology with deep feature selection for short-term
wind forecasting. Applied Energy, 190, 1245–1257. http://dx.doi.
org/10.1016/j.apenergy.2017.01.043, URL http://www.sciencedirect.
com/science/article/pii/S0306261917300508.
Glorot, X., Bordes, A., & Bengio, Y. (2011). Deep sparse rectifier neural
networks. 15, In Proceedings of the 14th international conference on
artificial intelligence and statistics (Vol. 15) (pp. 315–323).
Goodfellow, I., Bengio, Y., & Courville, A. (2017). Deep learning (1st ed.).
Cambridge, MA: MIT Press.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D.,
Ozair, S., et al. (2014). Generative adversarial nets. In Z. Ghahra-
mani, M. Welling, C. Cortes, N. D. Lawrence, K. Q. Weinberger
(Eds.), Advances in neural information processing systems (Vol. 27)
(pp. 2672–2680). Curran Associates, Inc., URL http://papers.nips.cc/
paper/5423-generative-adversarial-nets.pdf.
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., et al.
(2018). Recent advances in convolutional neural networks. Pat-
tern Recognition, 77, 354–377. http://dx.doi.org/10.1016/j.patcog.
2017.10.013, URL http://www.sciencedirect.com/science/article/pii/
S0031320317304120.
He, M., Yang, L., Zhang, J., & Vittal, V. (2014). A spatio-temporal
analysis approach for short-term forecast of wind farm generation.
IEEE Transactions on Power Systems, 29(4), 1611–1622. http://dx.doi.
org/10.1109/TPWRS.2014.2299767, URL http://ieeexplore.ieee.org/
document/6727513/.
He, K., Zhang, X., Ren, S., & Sun, J. (2016a). Deep residual learning for
image recognition. In 2016 IEEe conference on computer vision and
pattern recognition (pp. 770–778).
He, K., Zhang, X., Ren, S., & Sun, J. (2016b). Identity mappings in deep
residual networks. In B. Leibe, J. Matas, N. Sebe, & M. Welling
(Eds.), Computer vision – ECCV 2016 (pp. 630–645). Cham: Springer
International Publishing.
Heinermann, J., & Kramer, O. (2016). Machine learning ensem-
bles for wind power prediction. Renewable Energy, 89, 671–
679. http://dx.doi.org/10.1016/j.renene.2015.11.073, URL http://
www.sciencedirect.com/science/article/pii/S0960148115304894.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory.
Neural Computing, 9(8), 1735–1780. http://dx.doi.org/10.1162/neco.
1997.9.8.1735.
Hu, J., & Wang, J. (2015). Short-term wind speed prediction us-
ing empirical wavelet transform and Gaussian process regres-
sion. Energy, 93, 1456–1466. http://dx.doi.org/10.1016/j.energy.
2015.10.041, URL http://www.sciencedirect.com/science/article/pii/
S0360544215014097.
Hu, Q., Zhang, S., Yu, M., & Xie, Z. (2016). Short-term wind speed or
power forecasting with heteroscedastic support vector regression.
IEEE Transactions on Sustainable Energy, 7(1), 241–249. http://dx.
doi.org/10.1109/TSTE.2015.2480245, URL http://ieeexplore.ieee.org/
document/7335638/.
Hyndman, R. J., & Khandakar, Y. (2008). Automatic time series forecast-
ing: the forecast package for R. Journal of Statistical Software, 26(3),
1–22, URL http://www.jstatsoft.org/article/view/v027i03.
Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of
forecast accuracy. International Journal of Forecasting, 22(4), 679–
688. http://dx.doi.org/10.1016/j.ijforecast.2006.03.001, URL https:
//www.sciencedirect.com/science/article/pii/S0169207006000239?
via%3Dihub.
IEA (2017). World energy outlook 2017 (pp. 1–15). IEA, http://dx.
doi.org/10.1016/0301-4215(73)90024-4. URL https://www.iea.org/
weo2017/.
Jung, J., & Broadwater, R. P. (2014). Current status and future
advances for wind speed and power forecasting. Renewable
& Sustainable Energy Reviews, 31, 762–777, http://dx.doi.org/
10.1016/J.RSER.2013.12.054. URL https://www.sciencedirect.com/
science/article/pii/S1364032114000094.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimiza-
tion. (pp. 1–15). [ISSN: 09252312] ISBN: 9781450300728, http://doi.
acm.org.ezproxy.lib.ucf.edu/10.1145/1830483.1830503. URL http://
arxiv.org/abs/1412.6980.
969
International Journal of Forecasting 37 (2021) 949–970
Koivisto, M., Seppänen, J., Mellin, I., Ekström, J., Millar, J., Mam-
marella, I., et al. (2016). Wind speed modeling using a vector
autoregressive process with a time-dependent intercept term.
International Journal of Electrical Power & Energy Systems, 77,
91–99. http://dx.doi.org/10.1016/j.ijepes.2015.11.027, URL https://
www.sciencedirect.com/science/article/pii/S0142061515004470.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet clas-
sification with deep convolutional neural networks. Advances In
Neural Information Processing Systems, 1–9. http://dx.doi.org/10.
1016/j.protcy.2014.09.007.
Le Xie, Carvalho, P. M. S., Ferreira, L. A. F. M., Juhua Liu, Krogh, B. H.,
Popli, N., et al. (2011). Wind integration in power systems: Oper-
ational challenges and possible solutions. Proceedings of the IEEE,
99(1), 214–232. http://dx.doi.org/10.1109/JPROC.2010.2070051.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature,
521(7553), 436–444. http://dx.doi.org/10.1038/nature14539, URL
http://www.nature.com/doifinder/10.1038/nature14539.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based
learning applied to document recognition. Proceedings of the IEEE,
86(11), 2278–2323. http://dx.doi.org/10.1109/5.726791.
LeCun, Y. A., Bottou, L., Orr, G. B., & Müller, K.-R. (2012). Efficient
backprop. In G. Montavon, G. B. Orr, & K.-R. Müller (Eds.), Neural
networks: Tricks of the trade: Second edition (pp. 9–48). Berlin,
Heidelberg: Springer Berlin Heidelberg, http://dx.doi.org/10.1007/
978-3-642-35289-8_3.
Lei, M., Shiyan, L., Chuanwen, J., Hongling, L., & Yan, Z. (2009). A
review on the forecasting of wind speed and generated power.
Renewable & Sustainable Energy Reviews, 13(4), 915–920. http://dx.
doi.org/10.1016/J.RSER.2008.02.002, URL https://www.sciencedirect.
com/science/article/pii/S1364032108000282.
Li, X., Shang, W., & Wang, S. (2018). Text-based crude oil price
forecasting: A deep learning approach. International Journal
of Forecasting, http://dx.doi.org/10.1016/j.ijforecast.2018.07.006,
URLhttps://www.scopus.com/inward/record.uri?eid=2-s2.0-
85055654907&doi=10.1016%2fj.ijforecast.2018.07.006&partnerID=
40&md5=7ed2dec017f48dada0e1fa9664828e4c.
Li, G., Shi, J., & Zhou, J. (2011). Bayesian Adaptive combination
of short-term wind speed forecasts from neural network mod-
els. Renewable Energy, 36(1), 352–359. http://dx.doi.org/10.1016/
j.renene.2010.06.049, URL https://www.sciencedirect.com/science/
article/pii/S0960148110003228.
Liu, W., Luo, W., Lian, D., & Gao, S. (2018). Future frame prediction
for anomaly detection – A new baseline. In The IEEE conference on
computer vision and pattern recognition.
Liu, H., Mi, X., & Li, Y. (2018). Smart deep learning based wind
speed prediction model using wavelet packet decomposition, con-
volutional neural network and convolutional long short term
memory network. Energy Conversion and Management, 166, 120–
131. http://dx.doi.org/10.1016/j.enconman.2018.04.021, URL http:
//www.sciencedirect.com/science/article/pii/S019689041830356X.
Livera, A. M. D., Hyndman, R. J., & Snyder, R. D. (2011). Forecasting
time series with complex seasonal patterns using exponential
smoothing. Journal of the American Statistical Association, 106(496),
1513–1527. http://dx.doi.org/10.1198/jasa.2011.tm09771.
Long, J., Shelhamer, E., & Darrell, T. (2014). Fully convolutional net-
works for semantic segmentation. CoRR abs/1411.4038, arXiv:1411.
4038.
Maçaira, P. M., Thomé, A. M. T., Oliveira, F. L. C., & Ferrer, A. L. C.
(2018). Time series analysis with explanatory variables: A sys-
tematic literature review. Environmental Modelling & Software, 107,
199–209. http://dx.doi.org/10.1016/j.envsoft.2018.06.004, URL http:
//www.sciencedirect.com/science/article/pii/S136481521730542X.
Mathieu, M., Couprie, C., & LeCun, Y. (2015). Deep multi-scale video
prediction beyond mean square error. CoRR abs/1511.05440, arXiv:
1511.05440.
Persson, C., Bacher, P., Shiga, T., & Madsen, H. (2017). Multi-site
solar power forecasting using gradient boosted regression trees.
Solar Energy, 150, 423–436. http://dx.doi.org/10.1016/j.solener.
2017.04.066, URL http://www.sciencedirect.com/science/article/pii/
S0038092X17303717.
B.Q. Bastos, F.L. Cyrino Oliveira and R.L. Milidiú
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional
networks for biomedical image segmentation. In Medical im-
age computing and computer-assisted intervention – MICCAI 2015,
(pp. 234–241). Springer International Publishing, http://dx.doi.org/
10.1007/978-3-319-24574-4_28.
Saha, S., Moorthi, S., Wu, X., Wang, J., Nadiga, S., Tripp, P., et
al. (2014). The NCEP climate forecast system version 2. Jour-
nal of Climate, 27(6), 2185–2208. http://dx.doi.org/10.1175/JCLI-D-
12-00823.1, URL http://journals.ametsoc.org/doi/abs/10.1175/JCLI-
D-12-00823.1.
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-k., & Woo, W.-c.
(2015). Convolutional LSTM network: A machine learning approach
for precipitation nowcasting. In C. Cortes, N. D. Lawrence, D. D. Lee,
M. Sugiyama, & R. Garnett (Eds.), Advances in neural information
processing systems (Vol. 28) (pp. 802–810). Curran Associates, Inc.
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional net-
works for large-scale image recognition. URL https://arxiv.org/abs/
1409.1556.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhut-
dinov, R. (2014). Dropout: A simple way to prevent neural
networks from overfitting. Journal of Machine Learning Research, 15,
1929–1958. http://dx.doi.org/10.1214/12-AOS1000.
Tascikaraoglu, A., Sanandaji, B. M., Chicco, G., Cocina, V., Spertino, F.,
Erdinc, O., et al. (2016). Compressive spatio-temporal forecasting of
meteorological quantities and photovoltaic power. IEEE Transactions
on Sustainable Energy, 7(3), 1295–1305. http://dx.doi.org/10.1109/
TSTE.2016.2544929.
970
International Journal of Forecasting 37 (2021) 949–970
Tascikaraoglu, A., Sanandaji, B. M., Poolla, K., & Varaiya, P. (2016).
Exploiting sparsity of interconnections in spatio-temporal wind
speed forecasting using wavelet transform. Applied Energy, 165,
735–747. http://dx.doi.org/10.1016/j.apenergy.2015.12.082.
Vargas, S. A., Esteves, G. R. T., Maçaira, P. M., Bastos, B. Q., Oliveira, F. L.
C., & Souza, R. C. (2019). Wind power generation: A review and a
research agenda. Journal of Cleaner Production, 218, 850–870. http://
dx.doi.org/10.1016/j.jclepro.2019.02.015, http://www.sciencedirect.
com/science/article/pii/S0959652619303944.
Villegas, R., Yang, J., Hong, S., Lin, X., & Lee, H. (2017). Decomposing
motion and content for natural video sequence prediction. CoRR
abs/1706.08033, arXiv:1706.08033.
Xie, L., Gu, Y., Zhu, X., & Genton, M. G. (2014). Short-term spatio-
temporal wind power forecast in robust look-ahead power system
dispatch. IEEE Transactions on Smart Grid, 5(1), 511–520. http://dx.
doi.org/10.1109/TSG.2013.2282300.
Zhu, X., & Genton, M. G. (2012). Short-term wind speed forecasting for
power system operations. International Statistical Review, 80(1), 2–
23, http://dx.doi.org/10.1111/j.1751-5823.2011.00168.x. URL http:
//doi.wiley.com/10.1111/j.1751-5823.2011.00168.x.
Zhu, A., Li, X., Mo, Z., & Wu, R. (2017). Wind power prediction based
on a convolutional neural network. In 2017 international confer-
ence on circuits, devices and systems (pp. 131–135). IEEE, http://
dx.doi.org/10.1109/ICCDS.2017.8120465, URL http://ieeexplore.ieee.
org/document/8120465/.