Discussions and Closures
(e.g., Asefa 2009; Sharkey 1999; Shu and Burn 2004) that
Closure to “One-Day-Ahead Streamflow
Forecasting Using Artificial Neural
Networks and a Meteorological Mesoscale
Model” by Alvaro Linares-Rodriguez,
Vicente Lara-Fanego, David Pozo-Vazquez,
and Joaquin Tovar-Pescador
DOI: 10.1061/(ASCE)HE.1943-5584.0001163
Downloaded from ascelibrary.org by URI LIBRARIES on 10/20/15. Copyright ASCE. For personal use only; all rights reserved.
Alvaro Linares-Rodriguez 1; Vicente Lara-Fanego 2;
David Pozo-Vazquez 3; and Joaquin Tovar-Pescador 4
1
Ph.D. Candidate, Polytechnic Univ. College, Univ. of Jaén, Building
A3, Campus Lagunillas, 23071 Jaén, Spain. E-mail: alinares@
ujaen.es
2 Hence, it would be better to allocate the portion of the testing sub-
Researcher, Synermet Weather Solutions S.L., C/ Fuente de Don Diego,
30. 23001 Jaén, Spain. E-mail: vlara@synermet.com
3
Associate Professor, Polytechnic Univ. College, Univ. of Jaén, Building
A3, Campus Lagunillas, 23071 Jaén, Spain. E-mail: dpozo@ujaen.es
4
Professor, Polytechnic Univ. College, Univ. of Jaén, Building A3, Campus
Lagunillas, 23071 Jaén, Spain (corresponding author). E-mail:
jtovar@ujaen.es
The authors would like to thank the discussers for providing some
comments regarding the paper (Bozorg-Haddad et al. 2015). The
following are responses/clarifications to the points raised by the
discussers.
The aim of the original paper was to develop a one-day-ahead
streamflow forecasting model using artificial neural networks
(ANNs). The case study was conducted on the northeast Guadal-
quivir catchment (southeast Spain) over a two-year period. Seven
hundred and twenty-six days were used to train and validate the
model. As the authors stated in the original paper, “a larger input
datasets are preferable for training” an ANN model (Linares-
Rodriguez et al. 2015). In our case study, daily streamflow values
of previous years were not used because there was no flood event
since 2001. Training the ANN with a large dataset of low flows
could produce unreliable estimates for flood events. Therefore,
only streamflow values from 2009 to 2010 were used, including
two flood events in the calibration period and another in the inde-
pendent period. Although this period is sufficient to train the ANN
(Kişi 2007), a more detailed optimization should be conducted
to avoid problems from the limited number of training patterns
(Linares-Rodriguez et al. 2013).” For that reason, a new optimiza-
tion method was proposed. With that optimization procedure, the
model was able to guarantee its generalization capability and avoid
overfitting problems as the authors stated. The original manuscript
described in details such a new procedure:
1. The calibration dataset (626 inputs) was split into three subsets
(training, validation, and testing), and randomly sampled ac-
cording to the proportions 75%, 15%, and 10%, respectively.
The ANN was trained with the training subset, the validation
subset was used within the learning procedure to avoid over-
training via early stopping technique, and the test subset was
used to evaluate and rank the ANN model.
2. Step (1) was repeated 5,000 times, and the 5,000 ANNs were
evaluated and ranked according to the value of a fitness func-
tion (RMSE) over the test subsets.
3. The 10 best ANN models achieving best scores over
the test subsets were selected and used to build an ANN
ensemble model. The authors stated: “It has been shown
© ASCE 07015022-1
J. Hydrol. Eng., 07015022
the robustness and reliability of an ANN can be often signifi-
cantly improved by appropriately combining several ANN
models into an ANN ensemble model” (Linares-Rodriguez
et al. 2015).
Only with that optimization procedure was the proposed model
able to generate reliable and accurate one-day-ahead streamflow
forecasting. The independent dataset (last 100 days) was used to
evaluate the ensemble model.
The discussers mentioned that, “In general, owing to the fact
that the testing subset should not have any influence on training
the ANNs, the calibration phase should only benefit from the
training and validation subsets : : : It is obvious that using
the higher percentages of datasets in the training subset within
the calibration phase will increase the accuracy of forecasting.
set to the training subset to provide a robustly trained network”
(Linares-Rodriguez et al. 2015). The authors agree with the dis-
cussers that, in general, the common practice is to split the cal-
ibration set into training and validation subsets, but in this case the
optimization procedure for building an optimal ensemble model is
based explicitly on the evaluation of the ANN models over the test
subset. The 10 best ANN models selected were optimal for an
ensemble model due to their diversity; all ANN models were built
with random initial weights and random (75%) training sets. As
the authors stated in the original paper: “There are two major as-
sumptions behind this procedure: (1) by varying the initial param-
eters, the training algorithm is enabled to search other parts of the
parameter space, increasing overall chances of finding a global
error minimum; (2) since measured data are affected by noise
and an ANN is too sensitive to calibration data quality, a cleaner
choice of training dataset should better reflect the real hydrologic
state and thereby attain more reliable results.”
The discussers mentioned that “the applied multilayer percep-
tron (MLP) is not capable of providing a proper extrapolation
to predicate the out-of-range samples : : : ” For these authors, the
results of the proposed ensemble model described in the original
paper lead to a more favorable assessment of the model. The dis-
cussers support that statement in another ad hoc case study; how-
ever, (1) they do not build an ensemble model with the proposed
optimization model; (2) they do build a ANN model with the
common training, validation, and testing subsets; and (3) the num-
ber of input patterns is quite limited (408). For the reasons ex-
plained above, a simple ANN model is not able to ensure the
generalization capability of the model. In fact, the model proposed
by the discussers achieves improvable results. For example: (1) The
r2 score over the independent set is 0.872 ¼ 0.76, what may be
considered unsatisfactory; and (2) the discussers do not calculate
the Nash-Sutcliffe coefficient of efficiency (R2 ) nor the persistence
index (PI), especially defined for streamflow forecasting (Dawson
and Wilby 2001; Legates and McCabe 1999; Tayfur 2012). In par-
ticular, the PI score plays an important role to evaluate the well-
known problem of the timing errors often occurring when previous
flow values are used.
The discussers mentioned that “the forecasting accuracy would
deteriorate owing to two reasons: (1) using the testing subset
through the calibration phase, and (2) ignoring the efficient use
of the available datasets.” As explained above, the testing subset
plays a key role in the optimization procedure to build an optimal
ensemble model. Furthermore, by repeating the procedure 5,000
J. Hydrol. Eng.
 times leaving random the initial weights and the training subset,
the proposed method makes an efficient use of the available
datasets.
References
Asefa, T. (2009). “Ensemble streamflow forecast: A GLUE-based neural
network approach.” J. Am. Water Resour. Assoc., 45(5), 1155–1163.
Bozorg-Haddad, O., Hosseini-Moghari, S. M., and Azarnivand, A. (2015).
“Discussion of “one-day-ahead streamflow forecasting using artificial
neural networks and a meteorological mesoscale model”.” J. Hydrol.
Eng., in press.
Dawson, C. W., and Wilby, R. L. (2001). “Hydrological modelling using
artificial neural networks.” Progr. Phys. Geogr., 25(1), 80–108.
Downloaded from ascelibrary.org by URI LIBRARIES on 10/20/15. Copyright ASCE. For personal use only; all rights reserved.
Kişi, Ö. (2007). “Streamflow forecasting using different artificial neural
network algorithms.” J. Hydrol. Eng, 10.1061/(ASCE)1084-0699
(2007)12:5(532), 532–539.
© ASCE 07015022-2
J. Hydrol. Eng., 07015022
Legates, D. R., and McCabe, G. J., Jr. (1999). “Evaluating the use of
‘goodness-of-fit’ measures in hydrologic and hydroclimatic model
validation.” Water Resour. Res., 35(1), 233–241.
Linares-Rodriguez, A., Lara-Fanego, V., Pozo-Vazquez, D., and Tovar-
Pescador, J. (2015). “One-day-ahead streamflow forecasting using
artificial neural networks and a meteorological mesoscale model.”
J. Hydrol. Eng., 10.1061/(ASCE)HE.1943-5584.0001163, 05015001.
Linares-Rodriguez, A., Ruiz-Arias, J. A., Pozo-Vazquez, D., and Tovar-
Pescador, J. (2013). “An artificial neural network ensemble model
for estimating global solar radiation from Meteosat satellite images.”
Energy, 61, 636–645.
Sharkey, A. J. C. (1999). Combining artificial neural nets: Ensemble and
modular multi-net systems, Springer, New York.
Shu, C., and Burn, D. H. (2004). “Artificial neural network ensembles and
their application in pooled flood frequency analysis.” Water Resour.
Res., 40(9), W09301.
Tayfur, G. (2012). Soft computing in water resources engineering: Artifi-
cial neural networks, fuzzy logic and genetic algorithms, WIT Press/
Computational Mechanics.
J. Hydrol. Eng.