Stochastic Environmental Research and Risk Assessment

https://doi.org/10.1007/s00477-020-01910-0
(0123456789().,-volV)(0123456789().
,- volV)

ORIGINAL PAPER

Short term rainfall-runoff modelling using several machine learning

methods and a conceptual event-based model
Rana Muhammad Adnan1 • Andrea Petroselli2 • Salim Heddam3 • Celso Augusto Guimarães Santos4 •

Ozgur Kisi5,6

Accepted: 16 October 2020

 Springer-Verlag GmbH Germany, part of Springer Nature 2020

Abstract
The applicability of four machine learning (ML) methods, ANFIS-PSO, ANFIS-FCM, MARS and M5Tree, together with
multi model simple averaging (MM-SA) ensemble method, is investigated in rainfall-runoff modeling at hourly timescale.
The results are compared with the conceptual EBA4SUB model using rainfall and runoff data from Samoggia River basin,
Italy. The capability of the methods is measured using five statistics, Nash–Sutcliffe efficiency, root mean squared error,
mean absolute error, scatter index, and adjusted index of agreement. Comparison of single ML reveals that the ANFIS-
PSO, ANFIS-FCM and MARS produce similar accuracy which is better than the M5Tree model. MM-SA ensemble model
improves the accuracy of ANFIS-PSO, ANFIS-FCM, MARS and M5Tree models with respect to RMSE by 8.5%, 5%,
7.4% and 28.8%, respectively. Comparison with the conceptual event-based method indicates that the ML methods
generally performs superior to the EBA4SUB; however, latter method provides better accuracy than the M5Tree and
MARS in some cases.

& Andrea Petroselli 3

Agronomy Department, Faculty of Science, Hydraulics
petro@unitus.it Division University, 20 Août 1955, Route El Hadaik, BP 26,
Skikda, Algeria
& Ozgur Kisi
4
ozgur.kisi@iliauni.edu.ge; ozgurkisi@duytan.edu.vn Department of Civil and Environmental Engineering, Federal
University of Paraı́ba, Paraı́ba, Brazil
Rana Muhammad Adnan
5
rana@hhu.edu.cn Civil Engineering Department, Ilia State University, Tbilisi,
Georgia
Salim Heddam
6
s.heddam@univ-skikda.dz Institute of Research and Development, Duy Tan University,
Da Nang 550000, Vietnam
Celso Augusto Guimarães Santos
celso@ct.ufpb.br
1
State Key Laboratory of Hydrology-Water Resources and
Hydraulic Engineering, Hohai University, Nanjing 210098,
China
2
Department of Economy, Engineering, Society and Business
(DEIM), University of Tuscia, Viterbo, Italy

123

Graphic abstract
Keywords EBA4SUB  Conceptual event-based method  Machine learning  Rainfall-runoff modeling
1 Introduction
Accurate rainfall-runoff modelling has always been one of
the hottest topics for researchers in hydrology, because it
has a critical role in water resources management, hydro-
power development, urban planning, irrigation and other
agro-hydrological/meteorological activities’ planning (Al-
izadeh et al. 2017). With the increasing depletion of
available water resources throughout the world, accurate
estimation of runoff becomes more vital and demands
always more accurate methods for modelling the rainfall-
runoff process. However, accurate estimation of runoff is a
challenging task due to the complicated relationship
between rainfall and runoff. River runoff changes not only
are dependent on rainfall but also on many other meteo-
rological parameters such as evapotranspiration, solar
radiation, wind speed, air temperature, etc., and also
catchment specific characteristics such as topography,
shape, slope, altitude, soil type, land cover, moisture
holding capacity of soil, and so on. For successful running
of rainfall-runoff models, the data about all these param-
eters are usually required and this is a difficult and chal-
lenging task especially in developing countries and in
scarcely gauged catchments (Wu and Chau 2011).
If reliable data of the mentioned climatic and site
specific factors are not available, black box models are
preferred over conceptual and physically based models for
modelling rainfall-runoff process due to their low input
data requirements (Nourani et al. 2009). In black box
models category, autoregressive models were used for
runoff modelling in the past (Unal et al. 2004; Niedzielski
2007; Moeeni and Bonakdari 2017; Fathian et al. 2019;
Sun et al. 2019). However, the effects of the above-men-
tioned parameters on runoff make it a nonlinear and non-
123
Stochastic Environmental Research and Risk Assessment
stationary process that is difficult to be captured by these
conventional autoregressive black box models (Nayak et al.
2013). For modelling such complex nonlinear and non-
stationary process, machine learning models, such as,
neural network (ANN) based, fuzzy based and regression
based machine learning models have been applied suc-
cessfully in recent years.
In the last two decades, ANN based machine learning
models have been applied for modeling rainfall-runoff
process (Kasiviswanathan and Sudheer 2013,2017; Partal
et al. 2015; Papacharalampous et al. 2019; Aghelpour and
Varshavian 2020). Birikundavyi et al. (2002), de Vos and
Rientjes (2005), Antar et al. (2006) and Chen et al. (2014)
used the multilayer perceptron (MLP) neural network
model for predicting runoff of Mistassibi River, Geer
catchment, Blue Nile Basin, and Jinsha River, respectively,
and Senthil Kumar et al. (2004) applied radial basis func-
tion (RBF) neural network model in comparison of MLP
neural network model for rainfall-runoff modeling. They
found RBF performed better than MLP neural network
model, and Jain et al. (2004) utilized the back propagation
(BP) neural network model for daily runoff modeling of
Kentucky River basin. Kisi (2007) applied Levenberg–
Marquardt (LM) neural network in comparison of BP
neural network model and he found that the LM performed
better than the BP neural network model. Kisi (2008) used
generalized regression neural network (GRNN) model in
comparison of RBF and feed forward neural network
(FFNN) models. He found that the GRNN provided better
results in comparison of other neural network models.
Recently, long short-term memory neural network
(LSTMNN) model has received a lot of attention in runoff
modeling due to its great non-linear modeling ability (Yuan
et al. 2018; Zhu et al. 2020). In addition to modeling
Stochastic Environmental Research and Risk Assessment
precisely rainfall-runoff relationship, ANN models esti-
mated rainfall time series alone very accurately (Singh and
Borah 2013). Aksoy and Dahamsheh (2009) utilized the
two artificial neural networks (FFN and RBF) models in
comparison of multiple linear regression (MLR) model to
predict the monthly rainfall data of three climatic stations
in Jordan.
Although the ANN models were successfully applied in
literature for rainfall-runoff modelling, however, these
models could not capture the uncertainties in the rainfall-
runoff data very well. To capture these uncertainties, fuzzy
models can be used due to their ability to handle the
uncertain data (Chen et al. 2019). Therefore, a hybrid
combination of neural network and fuzzy system namely
ANFIS (Adaptive Neuro-Fuzzy Inference System) would
be more useful to precisely model rainfall-runoff process
(Nourani and Komasi 2013). Nayak et al. (2004) utilized
the ANFIS model in comparison to the ANN for modeling
the runoff of Baitarani River in India. They found that the
ANFIS performed better than the ANN models. Remesan
et al. (2008) applied ANFIS model for rainfall-runoff
modeling and compared the results with neural network
auto regressive with exogenous input (NNARX) model.
Jothiprakash et al. (2009) used ANFIS for modelling runoff
of an intermittent river. They trained ANFIS models with
different number of membership functions and in results
they found that the ANFIS model with three membership
functions provided the best results. Talei et al. (2010)
applied ANFIS for event-based rainfall-runoff modeling in
comparison of storm water management model (SWMM).
In the results, they found that the ANFIS estimated peak
flow more precisely than the SWMM model. Shiri and Kisi
(2010) developed a new wavelet ANFIS model for esti-
mation of short term and long term runoff. Further devel-
opment in the research field of the ANFIS model
application for runoff prediction were ANFIS-grid parti-
tioning (ANFIS-GP) and ANFIS-subtractive clustering
(ANFIS-SC) which has been applied by Sanikhani and Kisi
(2012). They used the monthly flow data of the Besiri and
the Baykan Station on the Bitlis Stream in Turkey. Their
study results demonstrated that the ANFIS-SC model fits
better than the ANFIS-GP in runoff prediction. The results
of the research done by Adnan et al. (2017) also validates
the outcome of Sanikhani and Kisi (2012) study. They
found ANFIS-SC was better in predicting river runoff of
Gilgit Basin in comparison of ANFIS-GP, FFNN, RBNN
and GRNN models. Kisi et al. (2018) developed the
ANFIS-fuzzy C-Means clustering (ANFIS-FCM) for long
term runoff prediction of two hydrometric stations in USA.
They found that the ANFIS-FCM provided better results
than the other two ANFIS models. To reduce the ANFIS
model’s uncertainty and improve the model prediction
performance, optimal selection of parameters is very
essential. For this purpose, particle swarm optimization
(PSO) heuristic algorithm was adopted to tuning the
parameters of ANFIS models (Zhang et al. 2018).
Among the regression based machine learning models,
used for decades, multivariate adaptive regression splines
model (MARS) and M5 model tree (M5Tree) have been
utilized in many hydrological applications (Yaseen et al.
2016; Adnan et al. 2018, 2019; 2020; Yin et al. 2018; Al-
Sudani et al. 2019; Rezaie-Balf et al. 2019; Mehdizadeh
et al. 2019; Kisi et al. 2019; Nourani et al. 2019a, b). Al-
Sudani et al. (2019) applied MARS model for modeling
runoff of Tigris River in comparison to the least square
support vector regression (LSSVR) model. Adnan et al.
(2019) utilized the MARS models in predicting monthly
river flows of Chakdara and Kalam stations. They also used
the LSSVR and the optimally pruned extreme learning
machine (OP-ELM) models. In the results, they found that
both MARS and LSSVR provided better results for the
both stations than the OP-ELM model. Rezaie-Balf et al.
(2019) applied the MARS models coupling with complete
ensemble empirical mode decomposition with adaptive
noise (CEEMDAN) denoising technique, for streamflow
prediction of High Dam. They evaluated CEEMDAN-
MARS with NN-CEEMDAN and M5-CEEDMAN models
and found that the MARS model combined with CEEM-
DAN gave superior performance compared to the other
models. Mehdizadeh et al. (2019) used the MARS models
to model monthly runoff of two hydro-meteorological
stations in Iran and two stations in Canada. They evaluated
MARS models with other autoregressive models using
local and external data and found that the MARS models
outperformed the autoregressive models. The M5Tree was
used to predict long term runoff of four different stations,
namely, Besiri and Malabadi located in Turkey, Hit and
Baghdad located in Iraq using the periodicity as input by
Yaseen et al. (2016); to forecast monthly and daily river
flow of Hunza River basin using nearby climatic data by
Adnan et al. (2018); to model runoff over the Mediter-
ranean region incorporating climate signal information in
comparison to the LSSVR and MARS models by Kisi et al.
(2019); to model the rainfall-runoff using three different
data division strategies using wavelet M5tree model by
Nourani et al. (2019a, b).
In this paper, the recently developed conceptual hydro-
logical model named Event-Based Approach for Small and
Ungauged Basins (EBA4SUB) is also evaluated for rain-
fall-runoff modeling. Few applications related to the
EBA4SUB model can be found in the literature. Grimaldi
and Petroselli (2015) firstly utilized this model in an Italian
basin for peak discharge estimation in comparison of
rational formula method. Then, Piscopia et al. (2015)
applied this method successfully and found satisfactory
results compared to rational formula. Then, Petroselli and
123

Grimaldi (2018) evaluated the model results with observed
discharges of five catchments and found satisfactory
results. To test the model capability in areas which have
less information and data, recently Młyński et al. (2018),
Petroselli et al. (2019a, b; 2020) and Vojtek et al. (2019)
applied the model for determining design peak flow in
ungauged basins of Bosnia Herzegovina, Iran, Poland and
Slovakia in comparison to other methods and found
promising results using EBA4SUB for these catchments.
In all of the aforementioned studies related to EBA4-
SUB, the model has been utilized only for design flood
estimation comparing its results with other rainfall-runoff
hydrological models. However, EBA4SUB has not been
compared with the machine learning methods in rainfall-
runoff modeling yet. Hence, the main contribution of the
present study will be the first time application of EBA4-
SUB hydrological model in comparison of ANFIS-FCM,
ANFIS-PSO, MARS and M5Tree machine learning models
and improving the results by applying multi model simple
averaging ensemble method (Nourani et al. 2019a, b).
2 Materials and methods
2.1 Case study
The selected case study, shown in Fig. 1, is the Samoggia
River basin, tributary of the Reno River and located in
central Italy. The Digital Elevation Model (DEM) at 20-m
Fig. 1 Study area. Localization (left), DEM (center) and land cover (right)
123
Stochastic Environmental Research and Risk Assessment
resolution was provided by the Italian Geographic Military
Institute, land cover data were retrieved from the Coordi-
nation of Information on Environment (CORINE) Database
website, and soil data has been derived from the soil map
provided by the local administration. The elevations of the
investigated area range within the interval 51–883 m, the
total contributing area is 178.5 km2, and the basin average
slope is approximately 19%. Regarding the land use, the
selected case study is characterized by mountain areas
covered mostly by broadleaved woods, and by bottom
valleys that are mainly floodplains covered by farmlands
and urbanized. Regarding the soil data, the case study can
be classified as a mix between loamy sand and sandy loam.
For the selected case study, three years (2014, 2015 and
2016) of synchronized hourly rainfall and runoff data have
been selected and used in the present work. Regarding the
rainfall data, the rainfall information has been averaged
using the Thiessen polygon method and considering three
raingauges present in the basin area.
2.2 EBA4SUB
Event-Based Approach for Small and Ungauged Basins
(EBA4SUB) estimates the design hydrograph and its peak
discharge related to a given return period and is based on
an event-based approach that can be used in ‘‘small’’ and
‘‘ungauged’’ watersheds. For ‘‘small’’ we mean having a
total contributing area smaller than few hundreds of square
kilometers, so that the assumption of spatially homogenous
Stochastic Environmental Research and Risk Assessment
rainfall is still acceptable, while for ‘‘ungauged’’ we mean
without discharge measurements. The EBA4SUB frame-
work consists of three steps: (a) gross rainfall identification
(design hyetograph based on Depth-Duration-Frequency—
DDF—curves or observed rainfall data can be used),
(b) excess rainfall estimation, and (c) design hydrograph
calculation. Other than rainfall information, the only input
data needed by EBA4SUB are the DEM, the land use and
the soil type information.
2.2.1 Gross rainfall
If DDF curves based on assumed return periods are available,
EBA4SUB can estimate the gross rainfall hyetograph
selecting the desired rainfall duration (usually equal to the
basin concentration time, that is estimated from DEM using
empirical formulas) and the desired rainfall pattern (rectan-
gular, triangular, or Chicago). In this work, anyway, since we
have available three years of continuous rainfall at hourly
timescale, we followed a different approach. First, we
identified single independent rainfall events that are defined
as rainfall sequences preceded and followed by at least 24
consecutive hours without rain thus assuming the physical
hypothesis that the initial abstraction phenomena return to
the pre-event condition in one day (Grimaldi et al. 2012).
Then, we modeled the rainfall-runoff relationship according
what expressed in the following.
2.2.2 Excess rainfall
Excess rainfall is determined based on the observed gross
rainfall hyetograph using the procedure suggested by Gri-
maldi et al. (2013), named Curve Number for Green-Ampt
(CN4GA). CN4GA is a mixed approach combining the two
methods. First, the SCS-CN procedure (NRCS 2008) is
used to determine the ponding time and the cumulative
excess rainfall, as follows:
8 cell, which represent parameters affecting the shape of the
< ðP  IaÞ2
Pn ¼ P þ S  Ia for P  Ia ð1Þ
:
0 for P\Ia
where Pn is the cumulative excess rainfall (mm), P is the
cumulative gross precipitation (mm), S is the maximum
potential basin retention (mm), and Ia is the initial
abstraction due to the interception, infiltration and surface
storage (mm). In the present analysis, Ia is assumed as the
cumulative gross rainfall between the event start and the
first moment when observed runoff is [ 0, while S is
derived from Eq. (1) computing cumulative rainfall and
runoff from observed data. The next step is to determine
the temporal distribution of the excess rainfall based on the
Green-Ampt equation (Santos et al. 2003; Green and Ampt
1911):
8
<  iðtÞ for  t\tpond
q0 ð t Þ ¼ N s ð2Þ
: KS 1 þ for t [ tpond
F ðt Þ
where q0(t) is the infiltration rate, tpond is the ponding time,
F(t) is the cumulative infiltration, Ks is the saturated
hydraulic conductivity, and Ns is the moisture-tension
parameter. This equation assumes that tpond is reached
when P(t) equals to Ia, while Ks is automatically calibrated
so that the total cumulative net runoff obtained by the SCS-
CN procedure equals to the corresponding one computed
by the Green-Ampt method. The other Green-Ampt equa-
tion parameters are determined starting from the selected
case study soil type based on look up tables.
2.2.3 Runoff hydrograph
The design hydrograph is determined using the DEM and
the width function based instantaneous unit hydrograph
(WFIUH). DEM at standard resolution is appropriate for
lumped modeling, but higher resolution is preferred if
available (Petroselli 2012).
The following steps are performed to pre-process the
DEM: (1) a Physical Erosion Model for PIT removal
(PEM4PIT) is used to remove the pits and flat areas
(Santini et al. 2009), (2) the drop analysis is used to extract
the river network (Tarboton et al. 1991), and (3) an opti-
mized single flow direction algorithm is used to define the
flow path based on Nardi et al. (2008). The WFIUH is
based on the following formula (Grimaldi et al. 2012):
Lc ð xÞ Lh ð xÞ
WFIUH ðtÞ ¼ þ ð3Þ
vc ð xÞ vh ð xÞ
where Lc and Lh are respectively the length of the path for
channel and hillslope cell x of the DEM, which are deter-
mined based on optimized flow direction, and vc and vh are
respectively surface flow velocity for channel and hillslope
WFIUH. Then, after the WFIUH is defined, the hydrograph
Q(t) is obtained according to:
Zt
QðtÞ ¼ A WFIUH ðt  sÞPn ðsÞds ð4Þ
0
where A is the basin area (km2), t is the rainfall duration
(h), s is the time step, and Pn(s) is the CN4GA excess
rainfall (mm/h). Regarding the estimation of surface flow
velocities, vh can be linked according empirical formulas to
local slope and land cover data (Grimaldi et al. 2012),
while vc represents the only parameter to be determined,
and its value can be calibrated for each event so that the
modeled peak discharge can be similar to the observed one.
123

2.3 ANFIS
ANFIS is a type of ANN based on fuzzy inference system
(Jang 1993), in which the ANN and fuzzy logic comple-
ment each other. ANFIS integrates both ANN and fuzzy
logic features; thus, it is able to take advantage of both in a
single system. It is able to model the uncertain scenarios
using fuzzy logic and ANN learns such a model (Jang et al.
1997; Jalalkamali 2015). Two main types of fuzzy infer-
ence systems are well known, i.e. Mamdani and Sugeno
systems (Kennedy and Eberhart 1948; Mamdani and
Assilian, 1975). In this study, the Sugeno fuzzy inference
system was chosen because it (1) is computationally effi-
cient; (2) works well with linear techniques; (3) works well
with optimization and adaptive techniques; (4) guarantees
output surface continuity; and (5) is well-suited to mathe-
matical analysis. Further details can be found in Jang
(1993); and Jang et al. (1997). The dataset was divided into
three equal parts, and a cross validation was performed, in
which two parts were used for training and one for testing;
however, each method was tested using all data. In the
presented study, two ANFIS methods were implemented,
one is ANFIS optimized with particle swarm optimization
(PSO) and other is ANFIS with fuzzy C-means clustering
(FCM). PSO and FCM are briefly explained in the fol-
lowing sections.
2.3.1 PSO
The concept for the optimization of nonlinear functions
using particle swarm methodology was firstly introduced
by Kenndy and Eberhart (1948), when they called such a
methodology as particle swarm optimization (PSO). Since
then, PSO has been widely applied in non-linear problems.
It is an evolutionary algorithm firstly inspired by the
behavior of birds, but now it is also based on biological and
sociological behavior of other organisms such as ants, bees
and fishes (Bashir and El-Hawary 2009; Santos et al.
2010, 2011). One can note that a school of fish or a swarm
of birds or insets searches for protection or food in a very
typical manner, which could be applied to a manner to
search for a global minimum of an objective function in a
feasible space. Every member of the swarm searches for
the best in its locality, learns from its own experience and
shares with other members. The PSO mimics such a
behavior, in which each individual of the swarm is a par-
ticle that has a position and a velocity. Thus, the member of
a swarm will communicate the good seen positions to each
other and adjust their own position and velocity. Further
details can be found in Kennedy and Eberhart (1948), and
Santos et al. (2010, 2011). Based on several tests, the
123
Stochastic Environmental Research and Risk Assessment
following values were set: population size = 25, max-it-
eration = 500 and 1000.
2.3.2 FCM
The fuzzy C-means clustering (FCM), a method developed
by Dunn (1973) and improved by Bezdek (1981), is used
here in the ANFIS for clustering. It allows one piece of
data to belong to two or more clusters. Further details about
the method can be obtained in the aforementioned refer-
ences. However, it is based on minimization of an objective
function, the fuzzy partitioning is carried out by an iterative
optimization of such function and the iteration stops when
a termination criterion is reached. Thus, data are bound to
each cluster by means of a membership function, which
represents the fuzzy behavior of such procedure. Therefore,
a matrix is built, whose factors are numbers between 0 and
1 and represent the degree of membership between data
and centers of clusters.
2.4 MARS
Multivariate adaptive regression splines (MARS) is a non-
linear and non-parametric method proposed by Friedman
(1991), which is able to predict a continuous dependent
parameter in high-dimensional data through a forward–
backward procedure. Several basis functions are added to
the model during the forward stepwise stage to initially
overfit the data. It is worth noting that the basis functions
are dependent on a spline function which split the dataset
into piecewise linear segments. The MARS-based models
are described as follows:
X
M
f ð x Þ ¼ d0 þ dm hm ð X Þ ð5Þ
m¼1
where hm(x) are spline functions, d0 and dm are coefficients
that can be computed by the minimum sum of squared
errors from splines functions, and M is the number of
functions. The unnecessary basis functions are eliminated
during the backward stage, which is called as pruning
stage. The functions are eliminated by the generalized
cross-validation criterion. Thus, this procedure improves
the prediction accuracy and M is determined (Kisi and
Parmar 2016). More detailed about the MARS algorithm is
available in Friedman (1994).
2.5 M5 model tree
M5 model tree (M5Tree) is a regression tree, which is
based on two steps: (1) creating a decision tree using a
splitting criterion, and (2) designing the model tree by
Stochastic Environmental Research and Risk Assessment

pruning the overgrown tree and replace the sub-trees with PN

½ðQ0 Þi  ðQc Þi 2
linear regression functions (Singh et al. 2010). NSE ¼ 1  Pi¼1
N
; 1\NES  1 ð9Þ
i¼1 ½ðQ0 Þi  Q 0 2
The M5Tree uses the same concept of a binary decision
tree to split the data, but instead of the class labels, it uses SI ¼ RMSE=Q0 ð10Þ
linear regression functions at the nodes, which predict PN
ðQ0 Þi  ðQc Þi
continuous numerical attributes. The splitting criterion is d ¼ 1  PN  i¼1   ð11Þ
ðQc Þ  Q0 ðQo Þ  Q0 
based on treating the standard deviation (sd) of the class i¼1 i i
values that reach a leaf as a measure of error at that leaf. in which, N is the number of data, Qo, Qc, Q0 are the
The expected reduction in this error is calculated as a result measured, calculated, mean measured, hourly streamflow,
of testing each attribute at that leaf. The standard deviation respectively.
reduction (SDR) is computed as: The statistical parameters of the applied data are sum-
X jN i j marized in Table 1. As seen from the table, whole data
SDR ¼ sd ðN Þ  sd ðNi Þ ð6Þ
jN j have three years, so they first were split into three equal
parts. Then, each model was trained using two parts and
where sd is the standard deviation, N is the examples that
tested with remaining part. At each step, training and
reach the leaf and Ni is the subset of examples with the ith
testing parts were changed. Thus, all the models were
outcome of the potential set.
tested with the whole data. It is apparent from Table 1 that
One may note that data in child nodes present less sd
the precipitation and streamflow data have considerably
than those at the parent node, which makes it purer. Thus,
high skewed distributions (Skewness ranges 9.76–10.4 for
after all split possibilities, M5 selects the one that maxi-
precipitation and 11.9–19.7 for streamflow).
mizes the expected error reduction, which may lead to a
Results for hourly streamflow prediction using the
large tree-like structure inducing to an over-fitting. Then,
ANFIS-PSO with different combinations of streamflow
the aforementioned step (2) takes place when, for example,
(Q) and precipitation are summarized in Table 2. For the
a sub-tree is replaced by a leaf (Pal and Deswal 2009).
employment of ANFIS-PSO, population was set to 25 and
two iterations were applied as 500 and 1000. As Table 2
shows, for the combinations of Q and precipitation used to
3 Results and discussion predict the Q, this study yielded diverse results. Using only
previous precipitation as input variable (combination 1–4),
In the presented study, we applied five models using different
the ANFIS-PSO performed relatively worse, with mean
input combination for forecasting hourly streamflows using
RMSE between 4.39 m3/s and 6.56 m3/s, and mean MAE
the cross-validation technique. The proposed models were:
in the range of 1.33–1.66 m3/s, and the high NSE values
(1) ANFIS-PSO, (2) ANFIS-FCM, (3) MARS, (4) M5Tree
and (5) the multi model with simple averaging called MM-
SA. MM-SA is commonly used in the literature (Nourani
Table 1 The statistical parameters of the applied data
et al. 2019a, b) and its results are calculated by simply aver-
aging the results of the single implemented methods (e.g., Whole data M1 M2 M3
ANFIS-PSO, ANFIS-FCM, MARS, M5Tree and EBA4- Precipitation, mm
SUB). Results of the different models and modelling strate- Mean 0.113 0.110 0.107 0.123
gies with different input combinations were assessed and Min 0 0 0 0
compared, and then further validated by comparison with the Max 16.5 12.2 14.1 16.5
EBA4SUB model. As a basis for comparison, models were Median 0 0 0 0
validated using five statistics indices: Nash–Sutcliffe effi-
Skewness 10.3 9.76 10.4 10.1
ciency (NSE), root mean squared error (RMSE), mean
Std. dev 0.582 0.501 0.578 0.658
absolute error (MAE), scatter index (SI) and adjusted index of
Streamflow, m3/s
agreement (d) calculated as follows:
Mean 1.406 1.732 1.078 1.410
1X
N
Min 0 0 0 0
MAE ¼ jðQ0 Þi  ðQc Þi j ð7Þ
N i¼1 Max 298 167 104 298
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Median 0.310 0.590 0.197 0
u N
u1 X Skewness 19.7 14.2 11.9 19.4
RMSE ¼ t ½ðQ0 Þi  ðQc Þi 2 ð8Þ Std. dev 6.63 6.36 3.90 8.71
N i¼1
M1 = 2016, M2 = 2015, M3 = 2014

123
 Stochastic Environmental Research and Risk Assessment

Table 2 Test statistics of the ANFIS-PSO using different combinations of streamflow and precipitation for hourly streamflow prediction
Statistics Cross validation Test data set Input combination
(i) (ii) (iii) (iv) (v) (vi) (vii) (viii)

RMSE M1 2016 4.58 4.51 3.28 7.55 1.40 1.83 1.37 1.57
M2 2015 4.17 5.19 3.81 4.67 1.01 0.96 0.90 0.88
M3 2014 9.43 5.27 6.07 7.45 2.97 3.02 2.69 2.69
Mean 6.06 4.99 4.39 6.56 1.79 1.94 1.65 1.71
MAE M1 2016 1.48 1.41 1.23 1.66 0.26 0.29 0.27 0.28
M2 2015 1.32 1.43 1.40 1.71 0.25 0.26 0.23 0.23
M3 2014 1.46 1.16 1.40 1.60 0.29 0.29 0.28 0.27
Mean 1.42 1.33 1.34 1.66 0.27 0.28 0.26 0.26
NSE M1 2016 0.483 0.499 0.735 - 0.408 0.951 0.917 0.954 0.939
M2 2015 - 0.140 - 0.767 0.049 - 0.427 0.934 0.940 0.947 0.949
M3 2014 0.457 0.635 0.516 0.271 0.884 0.880 0.905 0.905
Mean 0.360 0.600 0.433 0.369 0.923 0.912 0.935 0.930
In the table, input combinations are (i) Pt-1, Pt-2, …, Pt-10, (ii) Pt-1, Pt-2, …, Pt-20, (iii) Pt-1, Pt-2, …, Pt-30, (iv) Pt-1, Pt-2, …, Pt-40, (v) Pt-1,
Pt-2, …, Pt-30, Qt-1, (vi) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, (vii) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, Qt-3 and (viii) (v) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-
2, Qt-3, Qt-4. Among the P inputs, input combination (iii) provided the best accuracy for the M1 and M2 data sets and therefore, after the 4th
combination, streamflows were added to this combination. For the M3 set, however, input combination (ii) provided the best accuracy

obtained by the model was 0.735 using the combination 3
for the M1 dataset. Differences between the three datasets
(M1–M3) were slight, though overall performances
declined slightly as the models included more precipitation
data (combination 1–4). However, in regard to NSE values,
it is clear from Table 2 that the lowest accuracy was
obtained using the M2 dataset. Inclusion of previous
Q contributed to increasing the accuracy of the three
datasets (M1–M3), where the improvement was most sig-
nificant in case of M1 and M2 datasets. In general, the
reduction of RMSE and MAE occurred most effectively for
those subsets (M1 and M2), significantly more than the M3
subset. All in all, the most accurate models were those
having the previous Q as input variables (combination
5–8), yet the most significant improvement was achieved
using the model with the combination 7, for which the best
performance was obtained, with mean NSE value of 0.935
and mean RMSE and MAE of 1.65 m3/s and 0.26 m3/s,
respectively. In addition, the least accurate models were
those featuring only the previous precipitations as predic-
tors without the addition of any Q lag, which were not
improved in any case as more previous precipitation were
included. Hence, it was demonstrated that the inclusion of
previous Q for the input variables significantly helped in
reducing the RMSE and MAE value, yet it did introduce a
remarkable improvement of the models’ performances and
yielded the best accuracies.
Results using the ANFIS-FCM models are summarized
in Table 3. For this method, iteration was set to 100 and
cluster number from 2 to 8 was tried and 2 clusters
provided the best accuracy. As can be seen by referring to
the statistical indices reported in Table 3, all ANFIS-FCM
models using the M1–M3 subset are not improved in terms
of RMSE, MAE and NSE values. Comparing the results
using only the precipitation as input variables (combination
1–4), shows the deficiency of the ANFIS-FCM to capture
the high variation across the entire dataset. In one hand, the
performances of the ANFIS-FCM appear to have slightly
improved for the input combination 1, 2 and 4, using the
M1 subset, presenting NSE values that are slightly higher
than those using the ANFIS-PSO: 0.519 versus 0.483,
0.509 versus 0.499, and 0.560 versus - 0.408, respec-
tively. In the other hand, for the combination 4, it can be
clearly observed that the ANFIS-PSO was more accurate
than the ANFIS-FCM with 17.8% improvement in the NSE
value. No differences in model performance (as indicated
by the NSE) are apparent between the input combinations
1–4 for the M2 subset. The lowest NSE value (0.445) was
obtained using the combination 2. In addition, for the M3
subset, the NSE values in both input combinations (1–4) is
very small, reflecting the limitation of the models to cor-
rectly predict the Q using only the precipitation as input
variables, leading to the necessity to use previous stream-
flow record as input. Comparisons between the results in
Table 3 are informative. They suggest that models based on
previous Q values (combination 5–8) perform as well as
those with only the precipitation for all the three subsets
(M1–M3), but markedly high using the M1 subset. Models
for subset M1 all perform well for all combinations (5–8),
with NSE values in excess of 0.950, small RMSE and MAE

123
Stochastic Environmental Research and Risk Assessment

Table 3 Test statistics of the ANFIS-FCM using different combinations of streamflow and precipitation for hourly streamflow prediction
Statistics Cross validation Test data set Input combination
(i) (ii) (iii) (iv) (v) (vi) (vii) (viii)

RMSE M1 2016 4.42 4.46 4.24 4.23 1.41 1.32 1.32 1.33
M2 2015 11.4 39.2 4.34 4.33 1.01 1.32 0.89 0.89
M3 2014 6.45 6.50 6.35 6.34 2.97 2.68 2.55 2.56
Mean 7.42 16.72 4.98 4.97 1.80 1.77 1.59 1.59
MAE M1 2016 1.41 1.37 1.41 1.40 0.30 0.31 0.29 0.29
M2 2015 1.67 2.18 1.71 1.71 0.31 0.31 0.28 0.29
M3 2014 1.48 1.34 1.31 1.30 0.31 0.28 0.28 0.28
Mean 1.52 1.63 1.48 1.47 0.31 0.30 0.28 0.29
NSE M1 2016 0.519 0.509 0.557 0.560 0.951 0.957 0.957 0.957
M2 2015 - 7.55 - 99.6 - 0.232 - 0.230 0.933 0.957 0.948 0.948
M3 2014 0.454 0.445 0.470 0.471 0.884 0.905 0.914 0.914
Mean 2.841 33.5 0.420 0.420 0.923 0.940 0.940 0.940
In the table, input combinations are (i) Pt-1, Pt-2, …, Pt-10, (ii) Pt-1, Pt-2, …, Pt-20, (iii) Pt-1, Pt-2, …, Pt-30, (iv) Pt-1, Pt-2, …, Pt-40, (v) Pt-1,
Pt-2, …, Pt-30, Qt-1, (vi) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, (vii) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, Qt-3 and (viii) (v) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-
2, Qt-3, Qt-4. Among the P inputs, input combination (iii) provided the best accuracy and therefore, after the 4th combination, streamflows were
added to this combination

compared to the mean value (Table 4). For M3 subset, the
models are relatively lower than for M1 subset, with NSE
in the range of 0.884–0.914 and RMSE substantially in
excess of the average values. Regarding the M2 subset, the
developed ANFIS-FCM was able to successfully predict
the streamflow in one case for the four-input combination
(5–8). The mapping accuracy of the M2 dataset was good
with NSE values ranging from 0.933 for combination 5 to
0.957 for combination 6. The model performed slightly less
well compared to the M1 dataset and significantly higher
than the M3, and similar accuracies were found for the
combination 5 with NSE of 0.957. Not surprisingly,
accuracy as measured by mean NSE, RMSE and MAE was
higher for the combination 7 with values of 0.940, 1.59 m3/
s and 0.28 m3/s, respectively.

Table 4 Test statistics of the MARS models using different combinations of streamflow and precipitation for hourly streamflow prediction
Statistics Cross validation Test data set Input combination
(i) (ii) (iii) (iv) (v) (vi) (vii) (viii)

RMSE M1 2016 4.61 4.33 4.34 4.34 1.43 1.76 1.64 1.80
M2 2015 5.08 4.85 4.82 4.82 1.19 0.93 0.87 0.90
M3 2014 6.47 6.24 6.16 6.16 2.87 2.50 2.39 2.39
Mean 5.39 5.14 5.11 5.11 1.83 1.73 1.63 1.70
MAE M1 2016 1.51 1.42 1.46 1.46 0.26 0.25 0.24 0.25
M2 2015 1.61 1.65 1.72 1.72 0.22 0.20 0.18 0.18
M3 2014 1.45 1.32 1.30 1.30 0.28 0.25 0.24 0.24
Mean 1.52 1.52 1.52 1.49 0.25 0.23 0.22 0.22
NSE M1 2016 0.477 0.539 0.536 0.536 0.950 0.923 0.934 0.920
M2 2015 - 0.688 - 0.540 - 0.522 - 0.522 0.907 0.943 0.950 0.947
M3 2014 0.450 0.489 0.501 0.501 0.892 0.918 0.925 0.925
Mean 0.538 0.538 0.538 0.520 0.916 0.928 0.936 0.931
In the table, input combinations are (i) Pt-1, Pt-2, …, Pt-10, (ii) Pt-1, Pt-2, …, Pt-20, (iii) Pt-1, Pt-2, …, Pt-30, (iv) Pt-1, Pt-2, …, Pt-40, (v) Pt-1,
Pt-2, …, Pt-20, Qt-1, (vi) Pt-1, Pt-2, …, Pt-20, Qt-1, Qt-2, (vii) Pt-1, Pt-2, …, Pt-20, Qt-1, Qt-2, Qt-3 and (viii) (v) Pt-1, Pt-2, …, Pt-20, Qt-1, Qt-
2, Qt-3, Qt-4. Among the P inputs, input combination (ii) provided the best accuracy and therefore, after the 4th combination, streamflows were
added to this combination

123
 Stochastic Environmental Research and Risk Assessment

Table 5 Test statistics of the M5Tree using different combinations of streamflow and precipitation for hourly streamflow prediction
Statistics Cross validation Test data set Input combination
(i) (ii) (iii) (iv) (v) (vi) (vii) (viii)

RMSE M1 2016 5.02 4.63 4.73 4.54 2.61 2.60 2.60 2.60
M2 2015 4.42 4.34 4.88 4.76 1.17 2.58 2.58 2.58
M3 2014 6.69 4.89 5.63 5.91 2.59 2.55 2.48 2.52
Mean 5.38 4.62 5.08 5.07 2.12 2.58 2.55 2.57
MAE M1 2016 1.51 1.44 1.46 1.40 0.25 0.24 0.24 0.24
M2 2015 1.46 1.38 1.42 1.40 0.18 0.24 0.24 0.24
M3 2014 1.55 1.30 1.31 1.34 0.265 0.24 0.23 0.24
Mean 1.51 1.37 1.40 1.38 0.23 0.24 0.24 0.24
NSE M1 2016 0.379 0.472 0.449 0.491 0.832 0.833 0.833 0.833
M2 2015 - 0.279 - 0.233 - 0.560 - 0.484 0.910 0.836 0.836 0.836
M3 2014 0.411 0.686 0.584 0.541 0.912 0.915 0.919 0.917
Mean 0.356 0.500 0.531 0.505 0.885 0.861 0.863 0.862
In the table, input combinations are (i) Pt-1, Pt-2, …, Pt-10, (ii) Pt-1, Pt-2, …, Pt-20, (iii) Pt-1, Pt-2, …, Pt-30, (iv) Pt-1, Pt-2, …, Pt-40, (v) Pt-1,
Pt-2, …, Pt-30, Qt-1, (vi) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, (vii) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-2, Qt-3 and (viii) (v) Pt-1, Pt-2, …, Pt-30, Qt-1, Qt-
2, Qt-3, Qt-4. Among the P inputs, input combination (iv) provided the best accuracy for the M2 and M3 data sets. For the M1 set, however, input
combination (iv) provided the best accuracy and therefore, after the 4th combination, streamflows were added to this combination

Table 6 Test statistics of the

Statistics Cross validation Test data set ANFIS-PSO ANFIS-FCM MARS M5Tree MM-SA
best ANFIS-PSO, ANFIS-FCM,
MARS, M5Tree and MM-SA in RMSE M1 2016 1.37 1.32 1.64 2.61 1.37
hourly streamflow prediction
M2 2015 0.90 0.89 0.87 1.17 0.79
M3 2014 2.69 2.55 2.39 2.59 2.38
Mean 1.65 1.59 1.63 2.12 1.51
MAE M1 2016 0.27 0.29 0.24 0.25 0.23
M2 2015 0.23 0.28 0.18 0.18 0.19
M3 2014 0.28 0.28 0.24 0.265 0.24
Mean 0.26 0.28 0.22 0.23 0.22
NSE M1 2016 0.954 0.957 0.934 0.832 0.954
M2 2015 0.947 0.948 0.950 0.910 0.959
M3 2014 0.905 0.914 0.925 0.912 0.926
Mean 0.935 0.940 0.936 0.885 0.946
SI M1 2016 0.802 0.770 0.959 1.527 0.800
M2 2015 0.837 0.825 0.812 1.090 0.736
M3 2014 1.928 1.831 1.712 1.856 1.705
Mean 1.189 1.142 1.161 1.491 1.080
d M1 2016 0.932 0.928 0.942 0.937 0.943
M2 2015 0.918 0.901 0.937 0.937 0.931
M3 2014 0.829 0.930 0.939 0.932 0.939
Mean 0.893 0.920 0.939 0.935 0.938
MM-SA is multi model with simple averaging. The estimates of this model were simply calculated by
averaging the estimates of the optimal ANFIS-PSO, ANFIS-FCM, MARS and M5Tree models

The generated overall accuracies using the MARS 5.11 to 5.39 m3/s, mean MAE ranging from 1.49 to
model are displayed in Table 4. The application of MARS 1.52 m3/s, and mean NSE value ranging from 0.520 to
model for the four first input combinations (i.e. 1–4) pro- 0.530, respectively. The M1 subset produced a better result
duced a lower accuracy with mean RMSE ranging from with NSE ranging from 0.477 to 0.539, compared to the

123
Stochastic Environmental Research and Risk Assessment

Fig. 2 The scatterplot ANFIS-PSO ANFIS-FCM

comparison of observed and 200 200
predicted runoff by machine y = 0.9409x + 0.0659
learning methods—M1 data set y = 0.9748x + 0.0611 R² = 0.9575
150 R² = 0.954 150

Model, m3/s

Model, m3/s
100 100

50 50

0 0
0 50 100 150 200 0 50 100 150 200
Observed, m3/s Observed, m3/s

MARS
M5Tree
250 y = 1.0748x - 0.1195 300
R² = 0.95
200 250 y = 1.1164x - 0.1648
Model, m3/s

Model, m3/s
200 R² = 0.8895
150
150
100
100
50 50

0 0
0 50 100 150 200 250 0 50 100 150 200 250 300
Observed, m3/s Observed, m3/s

MM-SA
250
y = 1.0267x - 0.0394
200 R² = 0.9586
Model, m3/s

150

100

50

0
0 50 100 150 200 250
Observed, m3/s

M2 and M3 subsets. The statistical indices illustrate that
the results from these input combinations having only the
precipitation as input variables are the poorest. Comparison
of the results from MARS method with the result listed in
Tables 3 and 4 using ANFIS-FCM and ANFIS-PSO
models, illustrated that none of applied models was able to
achieve the best accuracy using only precipitation as input
variable. When the Q was included as input (combination
5–8), statistical indices for the three subsets (M1–M3)
indicate that the MARS model using the combination 7
generated significantly better result than the other as sup-
ported by the higher mean NSE value of 0.936 and mean
RMSE and MAE of 1.63 m3/s, 0.220 m3/s, respectively. In
addition, the NSE values, however, indicate that the MARS
performances for the M1 subset produced a better result
only for combination 5. For numerical comparison, the
NSE value of M1 subset improves the M2 and M3 subsets
by 4.3% and 5.8%, respectively. Overall accuracies
regarding the RMSE, MAE, and NSE values were pro-
duced using the M2 subset for the combination 7, with
values equal to 0.87 m3/s, 0.18 m3/s and 0.950,
respectively.
The M5 model tree (M5Tree) was also used to explore
the relation between hourly streamflow and precipitation

123
 Stochastic Environmental Research and Risk Assessment

Fig. 3 The scatterplot ANFIS-PSO ANFIS-FCM

comparison of observed and 125 125
predicted runoff by machine y = 0.8889x + 0.1178 y = 0.9538x + 0.0998
learning methods—M2 data set 100 R² = 0.9508 100 R² = 0.9485

Model, m3/s

Model, m3/s
75 75

50 50

25 25

0 0
0 25 50 75 100 125 0 25 50 75 100 125
Observed, m3/s Observed, m3/s

MARS M5Tree
125 125
y = 1.0015x + 0.0279 y = 0.9081x + 0.0764
100 R² = 0.9525 100 R² = 0.9098
Model, m3/s

Model, m3/s
75 75

50 50

25 25

0 0
0 25 50 75 100 125 0 25 50 75 100 125
Observed, m3/s Observed, m3/s

MM-SA
125
y = 0.9381x + 0.0805
100 R² = 0.9593
Model, m3/s

75

50

25

0
0 25 50 75 100 125
Observed, m3/s

using several input combinations. Summaries of the accu-
racy of M5Tree model using the M1–M3 subsets are shown
in Table 5, where the RMSE, MAE, and the NSE of the
eight input combinations are reported, together with the
mean value calculated for the M1–M3 subset. These show
that similar to the ANFIS-PSO, ANFIS-FCM, and MARS
models, the M5Tree was not able to correctly predict the
Q using only the precipitation as input variable, and the
models worked poorly with high RMSE and MAE values,
and the lowest NSE values. Table 5 shows that there are
greater variations between the accuracies of the three
subsets (M1–M3), and the best accuracy with the highest
NSE value of 0.686 was obtained using the combination 2
for the M3 subset, while they had dropped drastically for
the M1 and M2 subsets and become less than 0.50, rep-
resenting very low accuracies. Accordingly, with only
precipitation as input, the M5Tree did not show significant
improvement despite increased of the number of the pre-
vious precipitation for combination 1–4. Modifying the
structure of the input variables by adding the previous
streamflow (combination 5–8) may result in a significant
improvement of the models’ performances for the three
subsets. According to Table 5, for the M1 subset, the
performances of the M5Tree for the four input

123
Stochastic Environmental Research and Risk Assessment

Fig. 4 The scatterplot ANFIS-PSO ANFIS-FCM

comparison of observed and
350 350
predicted runoff by machine y = 0.9672x + 0.0719 y = 0.9502x + 0.0843
learning methods—M3 data set 300 R² = 0.9087 300 R² = 0.9156
250 250

Model, m3/s

Model, m3/s
200 200
150 150
100 100
50 50
0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
Observed, m3/s Observed, m3/s

M5Tree
MARS 350
350 y = 0.8881x + 0.1515
y = 0.9172x + 0.1157 300
300 R² = 0.9126
R² = 0.9252 250

Model, m3/s
250
Model, m3/s

200
200
150 150

100 100
50 50
0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
Observed, m3/s Observed, m3/s

MM-SA
350
y = 0.9307x + 0.1059
300 R² = 0.9257
250
Model, m3/s

200
150
100
50
0
0 50 100 150 200 250 300 350
Observed, m3/s

combinations (5–8) remained very close, with nearly equal
RMSE (& 2.60 m3/s), MAE (& 0.23 m3/s) and NSE
(& 0.83). The same conclusion was made with respect to
the M3 subset for which we observed that the M5Tree
models possess approximately the same RMSE
(& 2.50 m3/s), MAE (& 0.24 m3/s) and NSE (& 0.915)
values. In addition, for the M2 subset, the M5Tree using
the combination 5 appeared to be the most accurate model
with high NSE value above 0.91, RMSE and MAE below
1.20 m3/s and 0.20 m3/s, respectively. Moreover, it is clear
from the Table 5 that the accuracy of the model becomes
constant with equal NSE values of 0.836.
In Table 6, we compared the results obtained using the
best models according to the lowest mean values of the
RMSE, MAE, SI and the highest NSE and d with those
obtained using the multi model with simple averaging
(MM-SA). Replacing the individual models with the MM-
SA model for forecasting streamflow slightly increased the
overall accuracy of the ANFIS-FCM for which the NSE
and d values slightly increased (NSE from 0.940 to 0.946
and d from 0.920 to 0.938), the mean RMSE and SI

123
 Stochastic Environmental Research and Risk Assessment

Table 7 Comparison of
Method RMSE MAE NSE SI d RMSE MAE NSE SI D
machine learning and
EBA4SUB methods in Event 1 Event 2
modeling rainfall-runoff of
various events ANFIS-PSO 16.2 4.89 0.892 0.665 0.921 7.23 2.20 0.948 0.485 0.945
ANFIS-FCM 16.4 4.89 1.000 0.674 0.921 6.86 2.34 1.000 0.460 0.941
MARS 15.1 4.45 0.905 0.623 0.927 9.21 2.79 0.916 0.618 0.933
M5Tree 15.7 5.00 0.898 0.646 0.917 15.8 4.11 0.753 1.057 0.903
MM-SA 15.1 4.38 0.906 0.621 0.928 7.49 2.37 0.944 0.502 0.942
EBA4SUB 20.3 7.23 0.830 0.834 0.886 7.20 2.70 0.949 0.842 0.934
Event 3 Event 4
ANFIS-PSO 4.54 1.77 0.946 0.452 0.927 3.55 0.75 0.952 0.501 0.959
ANFIS-FCM 3.20 1.41 1.000 0.319 0.945 3.62 0.83 1.000 0.511 0.955
MARS 3.80 1.44 0.962 0.378 0.944 3.46 0.81 0.954 0.489 0.956
M5Tree 6.48 2.30 0.891 0.645 0.908 6.60 1.82 0.833 0.931 0.901
MM-SA 3.60 1.34 0.966 0.358 0.947 3.88 0.94 0.942 0.547 0.949
EBA4SUB 3.90 1.61 0.960 0.388 0.938 7.06 2.09 0.809 0.996 0.889
Event 5 Event 6
ANFIS-PSO 2.44 0.73 0.773 0.683 0.900 0.84 0.44 0.985 0.144 0.955
ANFIS-FCM 2.45 0.75 1.000 0.685 0.897 1.19 0.53 1.000 0.203 0.944
MARS 2.53 0.59 0.757 0.708 0.920 1.10 0.44 0.974 0.188 0.955
M5Tree 2.23 0.50 0.812 0.623 0.932 1.24 0.50 0.967 0.211 0.949
MM-SA 2.30 0.59 0.799 0.644 0.920 0.99 0.41 0.979 0.169 0.957
EBA4SUB 5.54 2.59 -0.163 1.548 0.668 3.26 1.89 0.774 0.556 0.810

Fig. 5 Comparison of machine learning and EBA4SUB methods in modeling rainfall-runoff of event 1

123
Stochastic Environmental Research and Risk Assessment
Fig. 6 Comparison of machine learning and EBA4SUB methods in modeling rainfall-runoff of event 2
marginally decreased (RMSE from 1.59 to 1.51 m3/s and
SI from 1.142 to 1.080), and with negligible improvement
of the MAE value, which dropped from 0.28 to 0.22 m3/s.
While it can be clearly observed that ANFIS-PSO, ANFIS-
FCM, and MARS models perform significantly better than
the M5Tree, and an improvement is observed when
including the previously measured Q as input, it is also
clear that these models cannot surpass the MM-SA model
in terms of overall accuracies. Comparisons of the mea-
sured and calculated streamflow by all developed models
revealed that the MM-SA model (NSE = 0.946) had a
superior performance in comparison to the ANFIS-PSO
(NSE = 0.935), ANFIS-FCM (NSE = 0.940), MARS
(NSE = 0.936) and M5Tree (NSE = 0.885) models. The
MM-SA model decreased the RMSE value of ANFIS-PSO
by 8.48%, 5.03% for ANFIS-FCM, 7.36% for MARS, and
28.77% for M5Tree. To sum up significant improvement in
Q forecasting by the MM-SA in comparison with M5Tree
model is observed in NSE value which increased from
& 0.885 to & 0.946 as well as the RMSE that decreased
from 2.12 to 1.51 m3/s. The scatterplot of observed and
predicted Q using the machines learning methods for M1–
M3 subset are showed in Figs. 2, 3 and 4.
Finally, we compared in Table 7 the results obtained
using the developed models with those obtained using the
EBA4SUB model for simulating six extreme streamflow
events during the period under study. The six events are
designated as Event 1 to Event 6. For Event 1 as seen in
Table 7 and Fig. 5, performance metrics show that MM-SA
is better than the other models according to the RMSE,
MAE, SI and d values, while the NSE value (& 0.906) of
the aforementioned model is below that of the ANFIS-
FCM (NSE = 1.00) model, slightly higher than that of the
MARS (NSE = 0.905) and significantly higher than those
obtained using ANFIS-PSO (NSE = 0.892), M5Tree
(NSE = 0.898) and EBA4SUB (NSE = 0.898). In overall,
it can be concluded that the EBA4SUB has the poorest
accuracies among all proposed models with high RMSE
and MAE of 20.3 m3/s and 5 m3/s, respectively. This is not
surprising since EBA4SUB has been mainly developed to
be a parsimonious model, in terms of input parameters, to
be used for design hydrograph estimation using the same
information of the rational formula. The MM-SA has a
marginally lower MAE, RMSE and SI (referring to
Table 7) compared to MARS model, but overall the dif-
ference between the two is negligible. In conclusion, our
results confirm that the EBA4SUB has limitations in cor-
rectly simulating the streamflow when several extreme
values (peaks) are presented, whereas the machines learn-
ing models were notably able to accurately simulate the
123

Fig. 7 Comparison of machine learning and EBA4SUB methods in modeling rainfall-runoff of event 3
extreme events. For Event 2 as seen in Table 7 and Fig. 6,
the ANFIS-FCM performed better than all other models in
terms of RMSE (& 6.86 m3/s), SI (0.460) and NSE
(& 1.00) values; in general ANFIS-FCM tended to cor-
rectly predict the peaks values more accurate than all the
other models. The superiority of the ANFIS-FCM was
more evident when compared to the M5Tree, for which an
improvement of 56.58% and 43.06% in terms of RMSE
and MAE was achieved. The results of the M5Tree model
were worse than those obtained using all the other models;
in addition, the aforementioned model tends to poorly
estimate the streamflow peaks values, with large RMSE
(& 15.80 m3/s), MAE (& 4.11 m3/s), SI (& 1.057), and
low NSE (& 0.753) and d (0.903) values. If the accuracy
prediction of the EBA4SUB is analyzed, in addition to
relatively high NSE (& 0.949) the RMSE value was
slightly smaller ((& 7.20 m3/s), when compared to the
ANFIS-PSO (RMSE = 7.23 m3/s; NSE = 0.948) and MM-
SA (RMSE = 7.49 m3/s; NSE = 0.944). Results for Event
3 (Table 7 and Fig. 7) showed that the best accuracy was
obtained using the ANFIS-FCM with the lowest RMSE of
3.20 m3/s and SI of 0.319 and the highest NSE of 1.00. In
addition, MARS, MM-SA and EBA4SUB showed higher
NSE values as compared to ANFIS-PSO. The differences
were the largest for M5Tree, and the three aforementioned
123
Stochastic Environmental Research and Risk Assessment
models were very close together. However, the MM-SA
provided slightly better performances, according to the
RMSE, MAE, SI, NSE and d. For Event 4 (Table 7 and
Fig. 8), it is clear that, the comparative results between the
proposed models reveal that the best accuracy was obtained
by the MARS model in accord to the RMSE (3.46 m3/s),
MAE (0.81 m3/s) and SI (& 0.489) values, slightly lower
than the ANFIS-PSO model (RMSE = 3.55 m3/s;
MAE = 0.75 m3/s; SI = 0.501). In addition, the lowest
accuracy was obtained using the EBA4SUB with higher
RMSE, MAE and SI values of 7.06 m3/s, 2.09 m3/s and
0.996 and lower NSE and d values of 0.809 and 0.889,
respectively. For Event 5 (not shown for brevity), The
M5Tree provided the best ability with the lowest RSME
(2.23 m3/s), MAE (0.50 m3/s), SI (0.623) and the highest d
(0.932). According to the NSE, however, the ANFIS-FCM
(NSE = 1.000) has better efficiency than the M5Tree
(NSE = 0.812). The accuracy of M5Tree model is closely
followed by the MM-SA model with respect to all five
comparison criteria. Finally, for Event 6 (not shown for
brevity), the ANFIS-PSO got the 1st rank in modeling of
this rainfall runoff event with the lowest RMSE (0.84 m3/
s), MAE (0.44 m3/s) and SI (0.144). This model is closely
followed by the MM-SA model. In addition, the ANFIS-
Stochastic Environmental Research and Risk Assessment
Fig. 8 Comparison of machine learning and EBA4SUB methods in modeling rainfall-runoff of event 4
FCM model has the higher NSE compared to ANFIS-PSO
and other models.
4 Conclusions
The study investigated the applicability of four machine
learning methods, ANFIS-PSO, ANFIS-FCM, MARS and
M5Tree, in short term rainfall-runoff modeling and results
were compared with those of the physically event-based
method, EBA4SUB. First, four methods were implemented
considering several input scenarios including previous
hourly precipitation and streamflow data obtained from
Samoggia River basin, Italy and cross validation was
applied to get more robust predictions. Then, four rainfall-
runoff events were selected from the data and the EBA4-
SUB model was applied to predict runoff from rainfall and
its results were compared with the machine learning
methods. Finally, multi model simple averaging ensemble
method was applied to the results and compared with the
single methods. Comparison of the results produced the
following conclusions:
• It was observed that the least accurate models were
found from the models having the previous precipitations
as inputs only without the addition of any streamflow.
• Inclusion of streamflow information considerably
improved the models accurcies by remarkably reducing
root means square error and mean absolute error and
increasing Nash–Sutcliffe efficiency.
• Among the single machine learning methods, ANFIS-
PSO, ANFIS-FCM and MARS were found to have
similar accuracy and they performed superior to the
M5Tree in rainfall-runoff modeling.
• MM-SA ensemble model was found to be useful
method and it improved the single models’ accuracy;
increments in RMSE of ANFIS-PSO, ANFIS-FCM,
MARS and M5Tree are 8.5%, 5%, 7.4% and 28.8%,
respectively.
• Comparison of the methods in event based prediction
indicated that the machine learning methods generally
outperformed the physically event-based method,
EBA4SUB, whereas, in some cases it provided better
accuracy than the M5Tree (Event 2 and Event 3) and
MARS (only Event 2).
The study revealed that the developed ML methods
(e.g., ANFIS-PSO, ANFIS-FCM, MARS) which use lim-
ited number of parameters can be useful in ungauged
basins where runoff information, needed for calibrating
advanced physically based models, are not available. In
this study, the developed ML methods were compared and
123

assessed using rainfall-runoff data from of one case study
in Italy. For future works, more data from different basins
can be utilized for evaluating the proposed methods.
Acknowledgements The authors are deeply grateful to Attilio
Castellarin, University di Bologna, DICAM Department, Italy, for
having provided the watershed data used in the present manuscript.
References
Adnan RM et al (2017) Application of soft computing models in
streamflow forecasting. In: Proceedings of the institution of civil
engineers-water management, Vol 172, No 3. Thomas Telford
Ltd. 123–134 October 2017
Adnan RM, Yuan X, Kisi O, Adnan M, Mehmood A (2018) Stream
flow forecasting of poorly gauged mountainous watershed by
least square support vector machine, fuzzy genetic algorithm and
M5 model tree using climatic data from nearby station. Water
Resour Manage 32(14):4469–4486
Adnan RM, Liang Z, Trajkovic S, Zounemat-Kermani M, Li B, Kisi
O (2019) Daily streamflow prediction using optimally pruned
extreme learning machine. Journal of Hydrology 577:123981
Adnan, R. M., Liang, Z., Heddam, S., Zounemat-Kermani, M., Kisi,
O., & Li, B. (2020). Least square support vector machine and
multivariate adaptive regression splines for streamflow predic-
tion in mountainous basin using hydro-meteorological data as
inputs. Journal of Hydrology, 586, 124371.
Aghelpour P, Varshavian V (2020) Evaluation of stochastic and
artificial intelligence models in modeling and predicting of river
daily flow time series. Stoch Env Res Risk Assess 34(1):33–50
Aksoy H, Dahamsheh A (2009) Artificial neural network models for
forecasting monthly precipitation in Jordan. Stoch Environ Res
Risk Assess 23(7):917–931
Alizadeh MJ, Kavianpour MR, Kisi O, Nourani V (2017) A new
approach for simulating and forecasting the rainfall-runoff
process within the next two months. J Hydrol 548:588–597
Al-Sudani ZA, Salih SQ, Yaseen ZM (2019) Development of
multivariate adaptive regression spline integrated with differen-
tial evoltion model for streamflow simulation. J Hydrol 573:1–12
Antar MA, Elassiouti I, Alam MN (2006) Rainfall–runoff modeling
using artificial neural networks technique: a Blue Nile catchment
case study. Hydrol Process 20(5):1201–1216
Bashir ZA, El-Hawary ME (2009) Applying wavelets to short-term
load forecasting using PSO-based neural networks. IEEE Trans
Power Syst 24:20–27
Bezdek JC (1981) Pattern recognition with fuzzy objective function
algoritms. Plenum Press, New York
Birikundavyi S, Labib R, Trung HT, Rousselle J (2002) Performance
of neural networks in daily streamflow forecasting. J Hydrol Eng
7(5):392–398
Chen L, Singh VP, Guo S, Zhou J, Ye L (2014) Copula entropy
coupled with artificial neural network for rainfall–runoff simu-
lation. Stoch Env Res Risk Assess 28(7):1755–1767
Chen CS, Jhong YD, Wu WZ, Chen ST (2019) Fuzzy time series for
real-time flood forecasting. Stoch Env Res Risk Assess
33(3):645–656
de Vos NJ, Rientjes THM (2005) Constraints of artificial neural
networks for rainfall–runoff modeling: trade-offs in hydrological
state representation and model evaluation. Hydrol Earth Syst Sci
9:111–126
Dunn JC (1973) A fuzzy Relative of the ISODATA process and its
use in detecting compact well-separated clusters. J Cybern
3:32–57
123
Stochastic Environmental Research and Risk Assessment
Fathian F, Fakheri-Fard A, Ouarda TB, Dinpashoh Y, Nadoushani
SSM (2019) Multiple streamflow time series modeling using
VAR–MGARCH approach. Stoch Env Res Risk Assess
33(2):407–425
Friedman JH (1991) Multivariate adaptive regression splines. Ann
Stat 19:1–67
Green WH, Ampt GA (1911) Studies on soil physics. J Agric Sci
4(1):1–24
Grimaldi S, Petroselli A (2015) Do we still need the rational formula?
An alternative empirical procedure for peak discharge estimation
in small and ungauged basins. Hydrol Sci J 60:66–67
Grimaldi S, Petroselli A, Nardi F (2012) A parsimonious geomor-
phological unit hydrograph for rainfall–runoff modeling in small
ungauged basins. Hydrol Sci J 57(1):73–83
Grimaldi S, Petroselli A, Romano N (2013) Curve-number/green-
ampt mixed procedure for streamflow predictions in ungauged
basins: parameter sensitivity analysis. Hydrol Process
27(8):1265–1275
Jain A, Sudheer KP, Srinivasulu S (2004) Identification of physical
processes inherent in artificial neural network rainfall–runoff
models. Hydrol Process 18:571–581
Jalalkamali A (2015) Using of hybrid fuzzy models to predict
spatiotemporal groundwater quality parameters. Earth Sci Inf
8:885–894
Jang JSR (1993) ANFIS: adaptive-network-based fuzzy inference
system. IEEE Trans Syst Man Cybern 23:665–685
Jang JSR, Sun CT, Mizutani E (1997) Neuro-fuzzy and soft
computing: a computational approach to learning and machine
intelligence. IEEE Trans Autom Control 42(10):1482–1484
Jothiprakash V, Magar RB, Kalkutki S (2009) Rainfall–runoff models
using adaptive neuro–fuzzy inference system (ANFIS) for an
intermittent river. Int J Artif Intell 3:1–23
Kasiviswanathan KS, Sudheer KP (2013) Quantification of the
predictive uncertainty of artificial neural network based river
flow forecast models. Stoch Env Res Risk Assess 27(1):137–146
Kasiviswanathan KS, Sudheer KP (2017) Methods used for quanti-
fying the prediction uncertainty of artificial neural network based
hydrologic models. Stoch Env Res Risk Assess 31(7):1659–1670
Kennedy J, Eberhart R, (1948) Particle swarm optimization. In:
Proceedings of the IEEE international conference on neural
networks Vol 4, 1942–1948
Kisi O (2007) Streamflow forecasting using different artificial neural
network algorithms. J Hydrol Eng 12(5):532–539
Kisi O (2008) River flow forecasting and estimation using different
artificial neural network techniques. Hydrol Res 39(1):27–40
Kisi O, Parmar KS (2016) Application of least square support vector
machine and multivariate adaptive regression spline models in
long term prediction of river water pollution. J Hydrol
534:104–112
Kisi O, Shiri J, Karimi S, Adnan RM (2018) Three different adaptive
neuro fuzzy computing techniques for forecasting long-period
daily streamflows. In: Roy S, Samui P, Deo R, Ntalampiras S
(eds) Big data in engineering applications. Studies in Big Data,
vol 44. Springer, Singapore, pp 303–321. https://doi.org/10.
1007/978-981-10-8476-8_15
Kisi O, Choubin B, Deo RC, Yaseen ZM (2019) Incorporating
synoptic-scale climate signals for streamflow modelling over the
Mediterranean region using machine learning models. Hydrol
Sci J 64(10):1240–1252
Mamdani EH, Assilian S (1975) An experiment in linguistic synthesis
with a fuzzy logic controller. Int J Man Mach Stud 7:1–13
Mehdizadeh S, Fathian F, Safari MJS, Adamowski JF (2019)
Comparative assessment of time series and artificial intelligence
models to estimate monthly streamflow: a local and external data
analysis approach. J Hydrol 579:124225
Stochastic Environmental Research and Risk Assessment
Młyński D, Petroselli A, Wałe˛ga A (2018) Flood frequency analysis
by an event-based rainfall-runoff model in selected catchments
of southern Poland. Soil Water Res 13:170–176
Moeeni H, Bonakdari H (2017) Forecasting monthly inflow with
extreme seasonal variation using the hybrid SARIMA-ANN
model. Stoch Env Res Risk Assess 31(8):1997–2010
Nardi F, Grimaldi S, Santini M, Petroselli A, Ubertini L (2008)
Hydrogeomorphic properties of simulated drainage patterns
using DEMs: the flat area issue. Hydrol Sci J 53(6):1176–1193
Natural Resources Conservation Service (NRCS) (2008) Hydrology,
national engineering handbook. Washington DC: US Department
of Agriculture, part 630
Nayak PC, Sudheer KP, Rangan DM, Ramasastri KS (2004) A neuro-
fuzzy computing technique for modeling hydrological time
series. J Hydrol 291(1–2):52–66
Nayak PC, Venkatesh B, Krishna B, Jain SK (2013) Rainfall-runoff
modeling using conceptual, data driven, and wavelet based
computing approach. J Hydrol 493:57–67
Niedzielski T (2007) A data-based regional scale autoregressive
rainfall-runoff model: a study from the Odra River. Stoch Env
Res Risk Assess 21(6):649–664
Nourani V, Komasi M (2013) A geomorphology-based ANFIS model
for multi-station modeling of rainfall–runoff process. J Hydrol
490:41–55
Nourani V, Komasi M, Mano A (2009) A multivariate ANN-wavelet
approach for rainfall–runoff modeling. Water Resour Manage
23(14):2877
Nourani V, Elkiran G, Abdullahi Tahsin A (2019) Multi-region
modeling of daily global solar radiation with artificial intelli-
gence ensemble. Nat Resour Res 28(4):1217–1238
Nourani V, Davanlou Tajbakhsh A, Molajou A, Gokcekus H (2019)
Hybrid Wavelet-M5 Model tree for rainfall-runoff modeling.
J Hydrol Eng 24(5):04019012
Pal M, Deswal S (2009) M5 model tree based modeling of reference
evapotranspiration. Hydrol Process Int J 23(10):1437–1443
Papacharalampous G, Tyralis H, Koutsoyiannis D (2019) Comparison
of stochastic and machine learning methods for multi-step ahead
forecasting of hydrological processes. Stoch Env Res Risk
Assess 33(2):481–514
Partal T, Cigizoglu HK, Kahya E (2015) Daily precipitation
predictions using three different wavelet neural network algo-
rithms by meteorological data. Stoch Env Res Risk Assess
29(5):1317–1329
Petroselli A (2012) LIDAR data and hydrological applications at the
basin scale. GISci Remote Sens 49(1):139–162
Petroselli A, Grimaldi S (2018) Design hydrograph estimation in
small and fully ungauged basin: a preliminary assessment of the
EBA4SUB framework. J Flood Risk Manage 11:197–210
Petroselli A, Mulaomerović-Šeta A, Lozančić Ž (2019a) Comparison
of methodologies for design peak discharge estimation in
selected catchments of Bosnia and Herzegovina.
GRAÐEVINAR 71(9):729–739
Petroselli A, Vojtek M, Vojteková J (2019b) Flood mapping in small
ungauged basins: a comparison of different approaches for two
case studies in Slovakia. Hydrol Res 50(1):379–392
Petroselli A, Asgharinia S, Sabzevari T, Saghafian B (2020)
Comparison of design peak flow estimation methods for
ungauged basins in Iran. Hydrol Sci J 65(1):127–137
Piscopia R, Petroselli A, Grimaldi S (2015) A software package for
the prediction of design flood hydrograph in small and ungauged
basins. J Agric Eng XLV I(432):74–84
Remesan R, Shamim MA, Han D, Mathew J (2008) ANFIS and
NNARX based rainfall-runoff modeling. In 2008 IEEE Interna-
tional Conference on Systems, Man and Cybernetics (pp.
1454–1459). IEEE
Rezaie-Balf M, Nagann SR, Kisi O, El-Shafie A (2019) Enhancing
streamflow forecasting using the augmenting ensemble proce-
dure coupled machine learning models: case study of Aswan
High Dam. Hydrol Sci J 64(13):1629–1646
Sanikhani H, Kisi O (2012) River flow estimation and forecasting by
using two different adaptive neuro-fuzzy approaches. Water
Resour Manage 26(6):1715–1729
Santini M, Grimaldi S, Nardi F, Petroselli A, Rulli MC (2009)
Preprocessing algorithms and landslide modelling on remotely
sensed DEMs. Geomorphology 113(1–2):110–125
Santos CAG, Srinivasan VS, Suzuki K, Watanabe M (2003)
Application of an optimization technique to a physically based
erosion model. Hydrol Process 17(5):989–1003. https://doi.org/
10.1002/hyp.1176
Santos CAG, Pinto LEM, Freire PKMM, Mishra SK (2010)
Application of a particle swarm optimization to a physically-
based erosion model. Wars Univ Life Sci—SGGW Ann Land
Reclam 42(1):39–49
Santos CAG, Freire PKMM, Mishra SK, Soares Júnior A (2011)
Application of a particle swarm optimization to the tank model.
IAHS Publ 347:114–120
Senthil Kumar AR, Sudheer KP, Jain SK, Agarwal PK (2004)
Rainfall–runoff modeling using artificial neural network: com-
parison of networks types. Hydrol Process 19(6):1277–1291
Shiri J, Kisi O (2010) Short-term and long-term streamflow forecast-
ing using a wavelet and neuro-fuzzy conjunction model. J Hydrol
394(3–4):486–493
Singh P, Borah B (2013) Indian summer monsoon rainfall prediction
using artificial neural network. Stoch Env Res Risk Assess
27(7):1585–1599
Singh KK, Pal M, Singh VP (2010) Estimation of mean annual flood
in Indian catchments using backpropagation neural network and
M5 model tree. Water Resour Manage 24(10):2007–2019
Sun Y, Niu J, Sivakumar B (2019) A comparative study of models for
short-term streamflow forecasting with emphasis on wavelet-
based approach. Stoch Env Res Risk Assess 33(10):1875–1891
Talei A, Chua LHC, Quek C (2010) A novel application of a neuro-
fuzzy computational technique in event-based rainfall–runoff
modeling. Expert Syst Appl 37(12):7456–7468
Tarboton DG, Bras RL, Rodriguez-Iturbe I (1991) On the extraction
of channel networks from digital elevation data. Hydrol Process
5(1):81–100
Ünal NE, Aksoy H, Akar T (2004) Annual and monthly rainfall data
generation schemes. Stoch Env Res Risk Assess 18(4):245–257
Vojtek M, Petroselli A, Vojteková J, Ashgarynia S (2019) Flood
inundation mapping in small and ungauged basins: sensitivity
analysis using the EBA4SUB and HEC-RAS modeling
approach. Hydrol Res 50(4):1002–1019
Wu CL, Chau KW (2011) Rainfall–runoff modeling using artificial
neural network coupled with singular spectrum analysis. J Hydrol
399(3–4):394–409
Yaseen ZM, Kisi O, Demir V (2016) Enhancing long-term streamflow
forecasting and predicting using periodicity data component:
application of artificial intelligence. Water Resour Manage
30(12):4125–4151
Yin Z, Feng Q, Wen X, Deo RC, Yang L, Si J, He Z (2018) Design
and evaluation of SVR, MARS and M5Tree models for 1, 2 and
3-day lead time forecasting of river flow data in a semiarid
mountainous catchment. Stoch Env Res Risk Assess
32(9):2457–2476
Yuan X, Chen C, Lei X, Yuan Y, Adnan RM (2018) Monthly runoff
forecasting based on LSTM–ALO model. Stoch Env Res Risk
Assess 32(8):2199–2212
Zhang Z, Zhang Q, Singh VP, Shi P (2018) River flow modelling:
comparison of performance and evaluation of uncertainty using
123
 Stochastic Environmental Research and Risk Assessment

data-driven models and conceptual hydrological model. Stoch Publisher’s Note Springer Nature remains neutral with regard to
Env Res Risk Assess 32(9):2667–2682 jurisdictional claims in published maps and institutional affiliations.
Zhu S, Luo X, Yuan X, Xu Z (2020) An improved long short-term
memory network for streamflow forecasting in the upper
Yangtze River. Stoch Environ Res Assess 34:1–17

123