Journal of Hydrology 403 (2011) 201–212

Contents lists available at ScienceDirect

Journal of Hydrology
journal homepage: www.elsevier.com/locate/jhydrol

Comparison of three artificial intelligence techniques for discharge routing

Rahman Khatibi a,⇑, Mohammad Ali Ghorbani b, Mahsa Hasanpour Kashani b, Ozgur Kisi c
a
Consultant Mathematical Modeller, Swindon, UK
b
Department of Water Engineering, Tabriz University, Tabriz, Iran
c
Department of Civil Engineering, Erciyes University, Kayseri, Turkey

a r t i c l e i n f o s u m m a r y

Article history: The inter-comparison of three artificial intelligence (AI) techniques are presented using the results of
Received 1 October 2010 river flow/stage timeseries, that are otherwise handled by traditional discharge routing techniques. These
Received in revised form 20 February 2011 models comprise Artificial Neural Network (ANN), Adaptive Nero-Fuzzy Inference System (ANFIS) and
Accepted 6 March 2011
Genetic Programming (GP), which are for discharge routing of Kizilirmak River, Turkey. The daily mean
Available online 25 March 2011
This manuscript was handled by A.
river discharge data with a period between 1999 and 2003 were used for training and testing the models.
Bardossy, Editor-in-Chief, with the The comparison includes both visual and parametric approaches using such statistic as Coefficient of Cor-
assistance of Fi-John Chang, Associate Editor relation (CC), Mean Absolute Error (MAE) and Mean Square Relative Error (MSRE), as well as a basic scor-
ing system. Overall, the results indicate that ANN and ANFIS have mixed fortunes in discharge routing,
Keywords: and both have different abilities in capturing and reproducing some of the observed information. How-
Inter-comparison ever, the performance of GP displays a better edge over the other two modelling approaches in most
Model pluralism of the respects. Attention is given to the information contents of recorded timeseries in terms of their
Discharge routing peak values and timings, where one performance measure may capture some of the information contents
Artificial intelligence modelling but be ineffective in others. Thus, this makes a case for compiling knowledge base for various modelling
GP, ANFIS, ANN techniques.
Kizilirmak
Ó 2011 Elsevier B.V. All rights reserved.

1. Introduction (iv) advocating the selection of particular models for particular

Applying artificial intelligence (AI) techniques to modelling
timeseries of river flows emerged in the last two decades reaching
the proof-of-concept stage of their developments to supplement
traditional discharge routing techniques. These techniques serve
significant practical problems in flood risk management. Tradi-
tional routing techniques are top-down approaches and are limited
by their inherent assumptions; whereas AI techniques are ‘‘bot-
tom-up’’ data-driven approaches exploiting the information con-
tained in the data. Both top-down and bottom-up modelling
practices should be used inclusively in a modelling culture, where
there is an increasing consensus that each technique is more suit-
able to particular circumstances than the others. Thus, just as it is
important to test new modelling approaches, it is equally impor-
tant to learn their strength and weaknesses in a pluralistic model-
ling culture. The focus of this paper is on (i) inter-comparison of
three AI techniques, (ii) gaining an insight into their performance,
(iii) avoiding the mindset of searching for a superior model, and
⇑ Corresponding author.
E-mail addresses: rahman_khatibi@yahoo.co.uk (R. Khatibi), cusp2004@yahoo.
com (M.A. Ghorbani), mahsakashani2003@yahoo.com (M.H. Kashani), kisi@erciyes.
edu.tr (O. Kisi).
problems.
Flood routing equations have been transformed into sophisti-
cated modelling software tools and at an international arena there
are both freeware (e.g. HEC-RAS) and commercial applications
including DELFT-FEWS, ISIS and MIKE series. The modelling en-
gines of these applications are based on the full or approximate
solutions of the Saint–Venant equations and since 2000, even 2D
modelling has been feasible at professional organisations. How-
ever, they require extensive survey and hydrometric data but these
are costly and sometimes there are various reasons not to be able
to meet data requirements. Traditionally, simplified modelling
techniques have been used to solve the problems posed by lack
of data but now AI techniques can also be employed. The focus
of this paper is to use such techniques to emulate the role of sim-
plified discharge routing techniques.
The importance of flood studies throughout the world need not
be overemphasised and simplified techniques, such as the Muskin-
gum (discharge routing), Muskingum–Cunge methods or the vari-
able parameter diffusion techniques have restrictive assumptions
and therefore their applications may not give sensible results most
of the times. Timeseries analysis is also capable of simulations,
similar to discharge routing but without any significant restric-
tions. One of the problems is that a diversity of timeseries analysis
models can be applied for discharge routing but there is little

0022-1694/$ - see front matter Ó 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.jhydrol.2011.03.007
202 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
knowledge on their comparative performance. This paper applies
three such techniques for predicting the outflow of a river reach
from its given inflows using the Artificial Neural Network (ANN),
Adaptive Nero-Fuzzy Inference System (ANFIS) and Genetic Pro-
gramming (GP). The focus in using these techniques is their in-
ter-comparisons. The study uses daily river flows data for the
Kizilirmak catchment, north Turkey, which is one of the most
important water resources in the country.
2. Literature review
AI techniques are data-driven modelling approaches and as
such, they are bottom-up approaches for not making any prior
assumptions about the model structure. These models are intelli-
gent in the sense that they use a proportion of the datapoints from
the timeseries to identify their inherent structure and use the
remaining datapoints to test their predictions. The strength of
these approaches comes forth when they are compared with a host
of routing models, which are top-down approaches for making
assumptions about the model structure. Top-down models include
those from the Muskingum method of discharge routing tech-
niques to the full hydrodynamic waves using the Saint–Venant
technique, as well as blackbox unit hydrograph and conceptual
models. All these top-down models are common in having a set
of equations derived on the basis of some assumptions, where
the more broad brush (coarse resolution) the assumptions are,
the less data-intensive they are but their predictions will suffer
from a greater degree of uncertainty and less able to predict spatial
distributions.
All these models are just prediction techniques and each has its
own strength and weaknesses. This review focuses on three
artificial techniques of ANN, ANFIS and GP and differentiates
between the works on the proof-of-concept and on consolidating
experience.
The proof-of-concept for the application of ANN is supported by
a number of research works to implement discharge routing mod-
els performing comparatively well with respect to conventional
discharge routing models and accounting for non-linearity inher-
ent in the data, e.g. see Thirumalaiah and Deo (1995), Tawfik
et al. (1997), Campolo et al. (1999), and Peters et al. (2006). Tawfik
et al. (1997) modelled the hysteresis nature of the stage–discharge
curve at a given stream section using ANN. Peters et al. (2006)
approximated the HEC–RAS routing module with an ANN model
for a 60 km stream length, and emphasised the advantage of using
ANNs for forecasting.
Campolo et al. (1999) applied ANNs to flood forecasting in an
upland catchment, but noted that almost all forecasted flood peaks
were underestimated. Various research works have been reported
to explain the symptomatic problem of underestimation in pre-
dicting extreme values in ANN models and these include: (i) inad-
equate representation of high-flow datapoints in the training
dataset, as suggested by Karunanithi et al. (1994); (ii) the need
for gap reduction in the model datapoint, e.g. Hsu et al. (1995) pro-
posed log transformations of flow values to reduce the gap be-
tween the high and low flow conditions; and (iii) a lack of
information in the network as suggested by Dawson and Wilby
(1998).
Although ANN is quite powerful for modelling various real
world problems, these models also have shortcomings. A major
one is that poor performance is likely when their performances
are extrapolated outside the range of the data used in their train-
ing. Also, if the input data suffer from undue uncertainty, the per-
formance of ANN is expected to be poor, in which case a fuzzy
system such as ANFIS may perform better.
ANFIS networks require a twofold process: (i) both firing
strengths and output coefficients have to be optimised; and (ii)
the overall number of rules must carefully be chosen (Panella
and Gallo, 2005). Lee and Han (2005) compared the potentials of
different neurofuzzy models for simulating real-time flood fore-
casting applications with (i) the traditional linear transfer function
model (TF-ARX type), (ii) naïve models, which predict future values
set equal to current ones, and (iii) trend model, which predict fu-
ture values from the linear extrapolation of the previous two flow
values. One of the neurofuzzy models is K-Nearest-Neighbour
(KNN) validation scheme, which is one of the most fundamental
and simple machine learning procedure for cases with little or no
prior knowledge about the distribution of the data. They conclude
that the KNN validation scheme, which has a great potential and
shows that the performance of a model improves by using the back
propagation neural network model; and achieves a better perfor-
mance than, naïve, trend models, linear transfer function and
feed-forward neural networks models. However, they devised a
special testing data and showed that under this test condition a lin-
ear transfer function model was more stable than all the neuro-
fuzzy models for extreme flood events needing extrapolation
beyond their training dataset. They conclude that AI models are
generally incapable of performing extrapolations if the test event
is very different to their training data.
The proof-of-concept for the application of GP to discharge
routing problems is supported by a number of research works,
which show that GP emulates flood forecasting problems and is a
viable alternative to ANNs and traditional models. GP applications
include: Savic et al. (1999), Babovic and Keijzer (2002), Khu et al.
(2001), Liong et al. (2002), and Aytek et al. (2008). Savic et al.
(1999) highlight the advantages of GP over ANNs and conceptual
models in flow prediction for the Kirkton catchment, UK. Khu
et al. (2001) use GP under real-time conditions to forecast runoff
of Orgeval catchment in France; measure its performance using
observed and calculated values; compare these with the perfor-
mance of other methods such as the Kalman filter; and these
results indicate an acceptable accuracy for their GP model. Sheta
and Mahmoud (2001) use GP to forecast the River Nile flow in
the Northern Sudan. Aytek and Kisi (2008) apply GP for modelling
suspended sediment in streams showing improved performance
over conventional rating curves and multi-linear regression
techniques.
Sivapragasam et al. (2007) investigate the performance of non-
linear Muskingum models and report the possibility of producing
erroneous results by using discharge routing techniques over
events with complex storms profiles resulting in multi-peaked
hydrographs. They use GP successfully to route single-peaked
and multi-peaked flood hydrographs quite accurately. The peaks
are very accurately predicted and there is no time lag in the occur-
rence of the peak values, unlike with the non-linear Muskingum
model. The GP model is also found to be time invariant.
An overview of the reported works is that these academic re-
search works are focused on testing new techniques to provide evi-
dence for the proof-of-concept to apply ANN, ANFIS and GP, but the
consolidation of knowledge is left to practitioners. However, expe-
rience is not often transformed into a body of knowledge for every-
one to tap on. The review shows that critical views have been
presented showing that ANN are liable to underestimations and
poor on extrapolation; advanced training scheme can be more
important than the model structure among neurofuzzy modelling
approaches; observed events contain information on the shape of
the whole hydrograph, the magnitude of the individual peaks
and the timing to their peaks and the relevant information on these
features should be understood systematically. Also, there is no
consensus on the choice and use of performance measures, for
which there is a large number of alternatives available. This work
 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
therefore compares these three techniques and focuses on the full
information content of the predicted flows to qualify the perfor-
mance of these three models.
3. Outline of the methods used
Ghorbani et al. (2010) argue that timeseries analysis is carried
out through the following phases:
Phase 1: Review the data to be used in timeseries and if applica-
ble, identify possible discontinuities in both independent and
dependent variables through chaos and catastrophe theory;
select the appropriate software application for the timeseries
analysis; divide the data into blocks of training data, validation
data and application data; and prepare datasets according to
the software applications.
Phase 2: Implement the timeseries analysis as per selected mod-
elling application; set the parameters as per selected method of
timeseries analysis and software application; and produce
results. This includes both training the model and using it in
the prediction mode.
Phase 3: Post-process the results in relation to training, valida-
tion and application; and if applicable, carry out some appropri-
ate sensitivity tests.
3.1. Artificial Neural Networks (ANNs)
ANNs are parallel information processing systems consisting of
a set of neurons or nodes arranged in layers and when weighted in-
puts are used, these nodes provide suitable conversion functions.
Any layer consists of pre-designated neurons and each neural net-
work includes one or more of these interconnected layers. Fig. 1
represents a three layered structure that consists of (i) input layer,
I, (ii) hidden layer, H, and (iii) output layer, O. Further information
on ANNs can be found in e.g. Haykin (1999).
The type of ANN used in this study is a multi-layer feed-forward
perceptron (MLP) trained with the use of back propagation learn-
ing algorithm. The operation process of these networks is so that
the input layer accepts the data and the intermediate layer pro-
cesses them and finally the output layer displays the resultant out-
puts of the model. During the modelling stage, coefficients related
Fig. 1. Multi layer perceptron neural network.
203
to present errors in nodes are corrected through comparing the
model outputs with recorded input data.
Connection weights are first initialised randomly by assigning a
small positive or negative random value through the following
procedure:
1. Input–output patterns are selected randomly using the training
data presented to ANN.
2. Actual network outputs are calculated for the current input
after the application to the activation function.
3. Performance measure is selected, e.g. Mean Square Error (MSE)
and the values are calculated.
4. Connection weights are adjusted to minimise the MSE.
5. Steps (2)–(5) are repeated for each pair of input–output vector
in the training datasets, until no significant change in the MSE is
detected for the system.
The final connection weights are kept fixed at the completion of
training and new input patterns are presented to the network to
produce the corresponding output consistent with the internal rep-
resentation of the input/output mapping.
3.2. Adaptive NeuroFuzzy Inference System (ANFIS)
An adaptive network is utilised to embed the Sugeno fuzzy
model into its framework to facilitate the learning of the Sugeno
fuzzy model. This can also compute gradient vectors systemati-
cally. This network architecture is called Adaptive-Network-based
Fuzzy Inference System or Adaptive NeuroFuzzy Inference System
(ANFIS). This is a universal approximator first introduced by Jang
(1993) capable of approximating any real continuous function on
a compact set to any degree of accuracy. It identifies a set of
parameters through a hybrid learning rule combining the back
propagation gradient descent error and a least-squares method.
It can be used as a basis for constructing a set of fuzzy ‘‘If–Then’’
rules with appropriate membership functions in order to generate
the preliminary stipulated input–output pairs. Fig. 2 represents a
typical ANFIS architecture, and outline as follows:
Layer 1: Every node in this layer is an adaptive node with a node
function that may be a generalised bell membership function or
a Gaussian membership function.
Layer 2: Every node in this layer is a fixed node labelled, P, rep-
resenting the firing strength of each rule, and is calculated by
the fuzzy AND connective of the ‘product’ of the incoming
signals.
Layer 3: Every node in this layer is a fixed node labelled N, rep-
resenting the normalised firing strength of each rule. The ith
node calculates the ratio of the ith rule’s firing strength to the
sum of two rules’ firing strengths.
Layer 4: Every node in this layer is an adaptive node with a node
function indicating the contribution of ith rule toward the over-
all output.
Layer 5: The single node in this layer is a fixed node labelled, R,
indicating the overall output as the summation of all incoming
signals.
The above comprise three different types of components, as fol-
lows (see Lughole, 2003):
1. Premise parameters as non-linear parameters that appear in the
input membership functions.
2. Consequent parameters as linear parameters that appear in the
rules consequents (output weights).
3. Rule structure that needs to be optimised to achieve a better lin-
guistic interpretability.
204 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
Fig. 2. A typical ANFIS architecture, see Moghaddamnia et al. (2009).
3.3. Genetic Programming (GP)
GP is truly a ‘‘bottom-up’’ process, for not making any assump-
tion on the structure of the relationship between independent and
dependent variables but it identifies an appropriate relationship
for any given timeseries. The construction of the relationship is
made possible by two components and efficient emulation of evo-
lutionary processes become possible only when these components
work hand-in-hand. These components are: (i) a set of components
of functions and their parameters (referred to as the terminal set),
which emulates the role of proteins or chromosomes in biological
systems; and (ii) a parse tree, which is a functional set of basic
operators and those selected in this study are:
fþ; ; ; xg
fþ; ; ; x2 g
fþ; ; ; x3 g
The relationship between independent and dependent variables
are often referred to as the ‘‘model,’’ the ‘‘program,’’ or the ‘‘solu-
tion.’’ Whatever, the terminology, the identified relationship in a
particular GP modelling is continually evolving and never fixed.
The evolution starts from an initially selected random population
of models, where the fitness value of each model is evaluated using
the values of the independent and dependent variables. As the pop-
ulation evolves from one generation to another, new models re-
place the old ones by having demonstrably better performance.
There are various selection methods and the method used in
this study is referred to as the Gene Expression Programming
(GEP) based on evolving computer programs of different sizes
and shapes encoded in linear chromosomes of fixed lengths,
Ferreira (2001a,b). The chromosomes are composed of multiple
genes, each gene encoding a smaller subprogram. Furthermore,
the structural and functional organisation of the linear chromo-
somes allows the unconstrained operation of important genetic
operators, such as mutation, transposition and recombination. It
has been reported that GEP is 100–10,000 times more efficient
than GP systems (Ferreira, 2001a,b) for a number of reasons,
including: (i) the chromosomes are simple entities: linear, com-
pact, relatively small, easy to manipulate genetically (replicate,
mutate, recombine, etc.); (ii) the parse trees or expression trees
are exclusively the expression of their respective chromosomes;
they are entities upon which selection acts; and according to fit-
ness, they are selected to reproduce with modification.
Applying operators like crossover and mutation to the winners,
‘‘children’’ or ‘‘offspring’’ are produced, in which crossovers are
responsible for maintaining identical features from one generation
to another but mutation causes a random change in the parse tree,
although data mutation is also possible. This completes the opera-
tions at the initial generation and the process is repeated until ter-
mination. There are now various software applications for
implementing GP models and the GenXpro (Ferreira, 2001a,b)
was used in this study. The parametric choices in this study are gi-
ven in Table A1, Appendix A.
4. Case study and data used description
This study uses daily mean river discharge data of two stations
(Stations: 1501 and 1543) on the Kizilirmak River, the Kizilirmak
Basin, Turkey. The locations of the stations are shown in Fig. 3. There
are 4 years of observed data (1461 days) available (1999–2003) for
both stations. The daily timeseries at the upstream and downstream
ð1aÞ stations are given in Fig. 4. The daily discharge data for the period
1999 and 2001 is used for training the models but the records for
ð1bÞ 2002 (Fig. 5) and 2003 (Fig. 6) are used for their prediction and
assessment of the performance of each AI modelling techniques.
It is noted that for the year 2001 records, there is a significant
ð1cÞ
inflow from a possible tributary between the gauging stations.
The models will be trained without filtering out this period and
therefore this will serve as a way to assess the flexibility of the
predictions by the AI techniques employed in this paper. The
performance of the models is evaluated by three goodness-of-fit
measures: Coefficient of Correlation (CC) as expressed by (2), Mean
Absolute Error (MAE), as expressed by (3), and Mean Squares
Relative Error (MSRE), as expressed by (4):
X
n
½ðQ o Þi  ðQ o Þ½ðQ e Þi  ðQ e Þ
CC ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð2Þ
Pn 2 Pn 2
i¼1 ½ðQ o Þi  ðQ o Þ i¼1 ½ðQ e Þi  ðQ e Þ
i¼1
1X n
MAE ¼ jQ  Q o j ð3Þ
n i¼1 e
n  2
1X Qo  Qe
MSRE ¼ ð4Þ
n i¼1 Qo
where Q is discharge; the subscripts ‘o’ and ‘e’ represent the ob-
served and estimated values; the average value of the associated
variable is represented with a bar above it and n is the total number
of records.
CC reflects on the degree of cause-and-effect relationship be-
tween the independent and dependent variables and has the abil-
ity to capture the information content both in terms of shape and
the actual agreement between the values. The MAE and the MSRE
can be used together to diagnose the variation in the errors in the
results. MAE would reflect if the results suffer from a bias and
MSRE records in real units the level of overall agreement between
the observed and modelled datasets. It is a non-negative number
 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212 205

Kizilirmak Basin and Gaging Stations

Fig. 3. Location of the stations (Stations: 1501 and 1543) on Kizilirmak River.

Fig. 4. Daily river timeseries at upstream and downstream stations (1999–2003). Fig. 5. Multi-peaked hydrograph in Kizilirmak River (2002).

that has no upper bound and for a perfect model, the result would tude events; instead, it evaluates all deviations from the observed
be zero. It provides no information about underestimation or over- values, both in an equal manner and regardless of their sign. MAE
estimation and is not weighted towards high(er) or low(er) magni- is comparable to the total sum of absolute residuals.
206 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212

5.3. Preliminary implementation of ANN model

The ANN model was implemented in the MATLAB software and

the data were processed by (i) scaling as per (6) and (ii) normalis-
ing the data (e.g. mapstd function) to ensure that their mean is zero
and standard deviation, r = 1.

Fig. 6. Single-peaked hydrograph in Kizilirmak River (2003).
5. Model development and analysis
5.1. Setting up the model
The models were implemented by reviewing the recorded time-
series, which recognised the importance of the individual peaks
and multi-peaks parts of the hydrographs for assessing the perfor-
mance of each AI technique. The recorded data for 2002 in Fig. 5
shows that there are four events with distinctive peaks reflecting
significant hydrological activities possibly superimposed on
groundwater-driven baseflow. Similarly, the record for 2003 in
Fig. 4 shows a number of peaks reflecting hydrological activities
but only one of these has been selected to study the model perfor-
mances for the simulation of single-peak events. This exposes the
models to severe tests.
5.2. Preliminary investigations
Qi  Q
Qn ¼ ð6Þ
r
where the subscripts ‘i’ and ‘n’ represent the ith discharge data and
normalisation, respectively. The best fit model structure was deter-
mined by the performance criteria and using trial and error method,
which led to the identification of 3-9-1 structure (input layer with
three neurons, hidden layer with nine neurons using the tan-sigmoid
function and output with one neuron and using a linear transfer func-
tion in the output layer). Default training parameters are summarised
in Table A1, Appendix A.
5.4. ANFIS model
The characteristic of the optimal ANFIS structure was obtained
in this preliminary study. The preliminary modelling led to the
identification of: input membership function type, output mem-
bership function; and number of input membership functions, 3,
3, 3 for the inputs It, It1 and Qt1. The default parameters are pre-
sented in Table A1, Appendix A.
5.5. GP model
Data were reviewed and the preliminary model structure given
in (5) was constructed to select the GP functional operators
expressed by (1a)–(1c). A preliminary investigation was carried

Table 2
A linear model was selected to set the values of the various de- Input variables used for ANN (MLP) model.
fault parameters and set the model structure, as follows:
Model Input variables Output variable
Q t ¼ f ðIt ; It1 ; Q t1 Þ ð5Þ Model 1 I(t) Q(t)
Model 2 I(t), I(t  1) Q(t)
where It = inflow at current time t, It1 = inflow at time t  1 and Model 3 I(t), I(t  1), Q(t  1) Q(t)
Qt1 = outflow at time t  1. Model 4 I(t), I(t  1), I(t  2), Q(t  1) Q(t)

Table 1
Comparing performance of basic operators of GP using CC, MAE and MSRE.
 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
Table 3
CC values for preliminary investigations.
out to select the most appropriate expression and the results are
shown in Table 1.
The results show that the model performance using the multi-
peak data is significantly better than the single-peak ones for all
three basic functions of (1a)–(1c) but the sensitivity to the choice
between these functions is not high. Overall, (1c) performs better
than the others and this is selected for the rest of the study. The de-
fault parameters are presented in Table A1, Appendix A.
5.6. Model structure
The model structure was first investigated by different combi-
nations of inflows and outflows of the river reach. Table 2 presents
model structures investigated.
Fig. 7. Model performances for single-peaked outflow hydrograph (2003) – GP:
closest to the observed values; ANFIS: closer to GP; ANN: poor.
207
A preliminary set of results are shown in Table 3, in which GP
uses (1c). According to these results, Models 1 and 2 perform
poorly and they are rejected as some of the results produce nega-
tive flow values in the predicted timeseries. Model 3 performs bet-
ter than Model 4, although this is not consistent due to one or two
exceptions and likewise the performance of Model 4 is satisfactory
but not as good as that of Model 3. Generally, GP performs better
than the ANN and ANFIS models but minor inconsistencies are
noted. Model 3 is therefore selected for its overall better perfor-
mance and is used in the rest of the study.
6. Results
Figs. 7 and 8 provide a visual comparison for the modelled out-
puts using ANN, ANFIS and GP with the observed flows for the sin-
Fig. 8. Model performances for multi-peaked outflow hydrograph (2002) – GP:
closest to observed values; ANN: some problems; ANFIS: poor.
208 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212

gle- and multi-peaked hydrographs, respectively. It is informative
to capture the information content by each model compared with
the observed flows and compare the findings with the outcome of
the literature review, if possible. These are summarised as follows:
(i) Overall, the GP model in these test runs performs better than
both the ANFIS and ANN models, where the ANFIS and ANN models
have a mixed fortune. (ii) The performance of ANN has a mixed for-
tune, as displayed in Fig. 7, it has a very poor performance for the
single-peaked record by severely overestimating the hydrograph
(as explained later) whereas its performance as in Fig. 8 for the
multi-peaked records is reasonably well and comparable to that
of GP but with slightly poor low values related to Peaks 2 and 3.
(iii) The performance of the ANFIS model has also a mixed fortune,
as it has a comparable performance to GP for the single-peaked
event (Fig. 7); but a poor performance for the multi-peaked event
(Fig. 8).
The above comparison based on visual observation of the
hydrographic features is now studied further using performance
measures presented in Table 4 in terms of CC, MAE and MSRE val-
ues. These results show the overall performance of the models and
provide a focus to the model with better performances. Evidently,
none of the three models outperforms the others using whichever
of the three performance measures in all of the evaluations (one
single-peak event, four individual peaks of the multi-peak event
and their overall performance); and none of the models necessarily
perform consistently for all the results. As discussed later in detail,

Table 4
Comparison of ANN, ANFIS and GP models using CC, MAE and MSRE.

a
Note that the ANN model scores the highest CC value for the single-peaked records, even though Fig. 7 shows visually that ANN performs poorly.
 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
Fig. 9. Scatter between modelled and observed flows for single-peaked timeseries.
Fig. 10. Scatter between modelled and observed flows for multi-peaked timeseries.
this is the truth of modelling, which fits common sense better than
the still received view of seeking for a super model outperforming
rival ones all the time.
The performances of the individual models are counted with a
score of 3 for good (best), 2 for satisfactory and 1 for poor perfor-
Fig. 11a. Impacts of coarser time interval resolution on accuracy – the ANN model.
209
mances and these are given in Table 4, although for some of the re-
sults there may be not much to choose between them. The results
show that the performance of the GP model is overall better by
scoring 51 points and the ANN model is ranked the second by scor-
ing 44 and the ANFIS model the last by scoring 27. The CC score of
ANN is 18 and is better than that even GP scoring 16 and that bet-
ter than ANFIS (8) reflecting the earlier conclusion that ANN cap-
tures the shape but introduces bias. Thus, the fortunes of ANN
are overturned by MAE (7 for ANN, 12 for ANFIS and 17 for GP)
but not by MSRE (14 for ANN, 9 for ANFIS and 18). Notably, MAE
and MSRE of GP score more than those of ANFIS and ANN. It should
be highlighted that these performance measures are rather blind to
the anomalous result of the ANN model as reflected in Fig. 7 but
this will be discussed in detail later.
Scatter diagrams are also a measure of performance and Figs.
9 and 10 confirm the best performance of GP for both single-
peaked and multi-peaked data. They show that ANFIS closely fol-
lows GP for the single-peaked data and comparatively better
than ANN but for the multi-peaked record, the situation changes.
This lack of pattern is highlighted for both sets of data in both
figures.
Sensitivity tests were carried out to investigate the effects of
data density on model performances. So both training and predic-
tion data were made sparser by removing: (i) every alternate value
(interval: 2 days), (ii) every 4 day data (interval: 5 days), and (iii)
every 9 day data (interval: 10 days). The investigation covered all
three AI techniques and the comparison of these results with ob-
served and modelled ones at a daily interval are given in Figs.
11a–11c. They show that: (i) all the models give the most accurate
results when the time interval is 1.0 day and their quality of pre-
dictions deteriorate by increasing the time interval; (ii) GP main-
tains some degree of robustness; (iii) ANN at the intervals of 5
and 10 days perform very poorly; and (iv) ANFIS produces unac-
ceptable results at the interval of 10 days. The general behaviour
of these results confirm the received view that the problem of res-
olution is the problem of ‘‘diminishing returns’’ in the sense that:
(i) the gain in short model running times with coarser time interval
resolutions is at the expense of the accuracy of modelling results;
(ii) significant improvements in accuracy are obtained by higher
resolutions up to a delineating point beyond which there is little
gain in accuracy but the model run times can become excessively
long. This is shown in Fig. 12.
210 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212

Fig. 11b. Impacts of coarser time interval resolution on accuracy – the ANFIS model.

Fig. 11c. Impacts of coarser time interval resolution on accuracy – the GP model.

Table 5
Summarising results.

Events Visual comparison Performance Scatter

measures diagram
Single-  ANN: Phantom  ANN: Medium  ANN: Poor
peak feature
 ANFIS: Phase lag  ANFIS: Low  ANFIS: Good
 GP: Phase lag  GP: High  GP: Good
Multi-  ANN: Moderate  ANN: Medium  ANN: Poor
peak
 ANFIS: Poor  ANFIS: Poor  ANFIS: Good
 GP: Good  GP: High  GP: Good
Overall  ANN: Mixed  ANN: Medium  ANN: Good
 ANFIS: Poor  ANFIS: Poor  ANFIS: Poor
 GP: Good  GP: High  GP: Good
Fig. 12. Behaviour of time interval resolution as a diminishing value problem.
 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212
7. Discussion
This study investigates three widely used AI models and em-
ploys three inter-comparison approaches: visual comparison,
devising a scoring system based on performance measures and
the scatter diagrams. The study focuses on a host of observed infor-
mation in terms of the actual values of the peaks and troughs, the
timing of these stationary points, the shapes of the hydrographs,
inflows contributing to the hydrographs. The results presented in
this paper and summarised further in Table 5 show that: (i) each
modelling technique is only good in emulating some of these infor-
mation contents of the data but they do not generally reproduce
them in full; (ii) the effectiveness of each performance measure
is limited, as each captures certain features.
The paper promotes a pluralistic modelling culture to commu-
nicate the collective strengths and weaknesses of the models of dif-
ferent modelling techniques and performance measures. Arguably,
a framework is yet to emerge for integrating knowledge for such a
culture. For instance, this study identified a rather poor perfor-
mance for ANFIS but one remains minded of the reported strength
of ANFIS in coping with data of poor quality and therefore it is very
important not to overwrite that gained knowledge through the re-
ported works by other researchers.
The performance of the ANN model for the single-peaked record
seems anomalous for two reasons: (i) The numerical CC value of
ANN scores the highest (0.88) compared visually with ANFIS and
GP despite the better performance of the latter two models. The
high score in terms of CC is surprising and a further investigation
confirmed that the value to be correct. (ii) As shown in Fig. 7, the
overestimation in the predicted hydrograph by ANN seems to be
a phantom feature attributable to a significant amount of lateral
inflow in the 2001 training data. Therefore, even though the pre-
dicted results are erroneous, the feature seems to be for a reason.
As tributary inflows may or may not happen during all flow events,
this produces a degree of haphazardness and different modelling
techniques would respond in a different manner and each AI tech-
nique have a different capability in capturing such features. It may
be reported without producing the result that the phantom feature
was indeed removed when the record of Year 2000 was used to
train the ANN model, in which case the lateral flow was physically
eliminated from modelling.
The GP generally performs well in all cases except in the case
of single-peak prediction event, for which there seems to be a
significant phase difference between the time to predicted and
observed peak. The phase difference is as much as 2 days but
this is relative better compared with the performance of the AN-
FIS model as its phase difference is 3 days. The phase difference
for the predicted multi-peak event is less significant and is order
of 1 day.
Top-down and bottom-up models differ in their approach to
their structure, so that: top-down models have prescribed
structures but bottom-up models make decisions and reconstruct
it; and in doing so, they are faced with the parsimony of their
structure. One implication of the principle of parsimony is to select
an approach with a minimum number of assumptions and there-
fore this principle immediately turns the focus on the structure
of the model. The thrust of this principle is that modellers should
be minded with the selection of the structure of their models
and ensure that elaborate model structures are avoided. There is
not a recommended approach for this and therefore this study de-
vised a set of preliminary investigation (see the results presented
in Tables 1–3) for this purpose. Practical experience shows if the
performances of particular models can be improved by elaborate
structures at the expense of parsimony, in the long run they be-
come susceptible to inadvertent failures.
211
It is noted that this study does not employ a diversity of perfor-
mance measures and this is normally the case as there is no such
formal recommendation on inter-comparison of model perfor-
mances. The studies normally employ deterministic frameworks
and use absolute values with a focus on certain features of pre-
dicted values, overlooking inherent uncertainties. Inherent uncer-
tainties are normally dealt with by statistical approaches, when
there is a significant numbers of datapoints, according to which
they are processed by ranking and banding; some sort of distribu-
tion is fitted to the processed datapoints; and the goodness-of-fit is
tested by higher order statistical tests. With an increasing empha-
sis on risk-based approaches, the focus nowadays is on the conse-
quences in terms of the frequency of adverse outcomes and their
impacts but not on deterministic values. Todini et al. (2006) argue
that it is necessary to reduce uncertainty for effective decision-
making and suggest a procedure for considering damage con-
sequences caused by floods, as the probability of overtopping
increases. Minded with consequences, it becomes feasible to
develop robust band of values or risk zones, rather than using
deterministic values. Khatibi (2006) discusses such a procedure
for flood warning and more details are given in Sayers et al.
(2003) for flood risk management. Arguably, similar procedures
are applicable in other water timeseries and risk-based approaches
are expected to make comparisons more realistic.
Although no knowledge base has been compiled for the strength
and weaknesses of individual models under a diversity of physical
and land use conditions, arguably the future of diversity of
modelling techniques is the need for some sort of a pluralistic
framework, creating suitable conditions for combining them on
their strengths. For instance, Lee and Han (2005) argue that neural
networks are good at recognising patterns but they are not good at
explaining how they reach their decisions. This opaque nature has
hampered its wider applications in real time operations. Fuzzy
logic systems can reason with imprecise information and are good
at explaining their decisions but they cannot automatically acquire
the rules they use to make those decisions.
8. Conclusions
In this study, applicability and capability of three AI techniques,
ANN, ANFIS and GP, are investigated for routing river discharge
timeseries. Kizilirmak River, north Turkey, was used as a case for
the inter-comparison of their performances. Initial modelling deci-
sions were made for all three modelling techniques including the
preliminary review and selection of data for training and predic-
tions and the assessment of the performance using single-peaked
and multi-peaked series.
The results show that the GP model performs better in predict-
ing single-peaked series and multi-peaked series but some phase
lag is observed between the predicted and observed values. GP in
this study seemed to be insensitive to lateral inflows. The ANN
model captures the peaks of the multi-peaked event better than
the ANFIS but suffers from a gross sensitivity to lateral inflows in
modelling a single-peaked event, although this may be attributed
to the existence of lateral flow in the training data but the lack
of lateral inflows in the prediction data may be resurrected as
anomalous features.
The paper argues that deterministic frameworks for inter-com-
parisons are in themselves poor to focus on significant features of
the timeseries and to assess significant differences. When the
difference in performances of two models is slight, the determinis-
tic framework of inter-comparison amplifies the differences lead-
ing to anomalous results. This study focused on such anomalies
and avoided the pitfalls by considering different features and
performance measures, as well as complementing quantitative
212 R. Khatibi et al. / Journal of Hydrology 403 (2011) 201–212

Table A1
Default parameter values used by the model.

ANN ANFIS GP
Training Values Training parameters Values Training Values
parameters parameters
Goal 0 Input membership function Triangular Crossover rate 0.3
(trimf)
Epochs 150 Number of input membership 333 Mutation rate 0.044
functions
max_fail 6 Output membership function Constant Population size 28
mem_reduc 1 Epochs 70 Number of 5000
generations
min_grad 1e010 Optimal method Hybrid Arithmetic Plus, minus,
functions multiply
Mu 0.001 Number of fuzzy rules 27
mu_dec 0.1
mu_inc 10
mu_max 10,000,000,000
Training Levenberg–Marquardt
algorithm (trainlm)

assessments with qualitative evaluations. The paper argues that
risk-based approaches should overcome such anomalies.
The paper is minded to avoid superlative expressions such as
superior performance, where such a mindset defies common sense
by seeking for a superior performance of a supposed perfect model
all the times for all problems. Instead, inter-comparison of differ-
ent modelling approaches should be directed towards compiling
a knowledge base to facilitate the emergence of pluralistic model-
ling practices, where strengths and weaknesses of the individual
modelling approaches are captured with respect to the diversity
of the context. If AI models can be employed to identify the various
hydrographic features, arguably these techniques will truly
become tools of artificial intelligence.
Acknowledgement
The authors are grateful to the Turkish Electrical Power
Resources Survey and Development Administration for making
available the daily river discharge data.
Appendix A
See Table A1.
References
Aytek, A., Asce, M., Alp, M., 2008. An application of artificial intelligence for rainfall–
runoff modelling. J. Earth Syst. Sci. 117 (2), 145–155.
Aytek, A., Kisi, O., 2008. A genetic programming approach to suspended sediment
modeling. J. Hydrol. 351, 288–298.
Babovic, V., Keijzer, M., 2002. Rainfall runoff modelling based on genetic
programming. Nord. Hydrol. 33 (5), 331–346.
Campolo, M., Andreussi, P., Soldati, A., 1999. River flood forecasting with a neural
network model. Water Resour. Res. 35, 1191–1197.
Dawson, C.W., Wilby, R., 1998. An artificial neural network approach to rainfall–
runoff modelling. Hydrol. Sci. J. 43, 14–66.
Ferreira, C., 2001a. Gene expression programming in problem solving. In: 6th Online
World Conference on Soft computing in Industrial Applications (Invited Tutorial).
Ferreira, C., 2001b. Gene expression programming: a new adaptive algorithm for
solving problems. Complex Syst. 13 (2), 87–129.
Ghorbani, M., Khatibi, R., Aytek, A., Makarynskyy, O., 2010. Sea water level
forecasting using genetic programming and artificial neural networks. J.
Comput. Geosci. 36 (5), 620–627.
Haykin, S., 1999. Neural Networks: A Comprehensive Foundation, second ed.
Prentice Hall, Upper Saddle River, New Jersey.
Hsu, K.L., Gupta, H.V., Sorooshian, S., 1995. Artificial neural network modeling of the
rainfall–runoff process. Water Resour. Res. 31 (10), 2517–2530.
Jang, J.R., 1993. Anfis: adaptive-network-based fuzzy inference system. IEEE Trans.
Syst. Man Cybernet. 23, 665–685.
Karunanithi, N., Grenney, W.J., Whitley, D., Bovee, K., 1994. Neural networks for
river flow prediction. J. Comput. Civil Eng. 8 (2), 201–220.
Khatibi, R., 2006. Barriers inherent in flood forecasting and their treatments. In:
Knight, D.W., Shamseldin, A.Y. (Eds.), River Basin Management for Flood Risk
Mitigation. Taylor and Francis Group Plc., London, pp. 569–585.
Khu, S.T., Liong, S.Y., Babovic, V., Madsen, H., Muttil, N., 2001. Genetic programming
and its application in real-time runoff forming. J. Am. Water Resour. Assoc. 37
(2), 439–451.
Lee, H.X., Han, D., 2005. Exploration of neuro-fuzzy models in real time flood
forecasting. In: Proceedings of the 2008 International Conference on Artificial
Intelligence and Pattern Recognition, ISRST, Orlando, FL, USA, 7–10 July 2008,
pp. 264–268. ISBN: 978-1-60651-000-1. <http://www.promoteresearch.org/
2008/aipr/index.html>.
Liong, S.Y., Gautam, T.R., Khu, S.T., Babovic, V., Keijzer, M., Muttil, N., 2002. Genetic
programming: a new paradigm in rainfall runoff modeling. J. Am. Water Resour.
Assoc. 38 (3), 705–718.
Lughole, E., 2003. Online Adaptation of Takagi-Sugeno Fuzzy Inference Systems.
Technical Report, Fuzzy Logic Laboratorium, Linz-Hagenberg.
Moghaddamnia, A., Remesan, R., Hassanpour Kashani, M., Mohammadi, M., Han,
D., Piri, J., 2009. Comparison of LLR, MLP, Elman, NNARX and ANFIS models-
with a case study in solar radiation estimation. J. Atmos. Sol.-Terr. Phys. 71,
975–982.
Panella, M., Gallo, A.S., 2005. An input/output clustering approach to the synthesis
of ANFIS networks. IEEE Trans. Fuzzy Syst. 13 (1).
Peters, R., Schmitz, G., Cullmann, J., 2006. Flood routing modelling with artificial
neural networks. Adv. Geosci. 9, 131–136.
Savic, D.A., Walters, G.A., Davidson, G.W., 1999. A genetic programming approach to
rainfall–runoff modelling. Water Resour. Manage. 13, 219–231.
Sayers, P.B., Gouldby, B.P., Simm, J.D., Meadowcroft, I., Hall, J., 2003. Risk,
Performance and Uncertainty in Flood and Coastal Defence. Technical Report
FD2302/TR1. Department of Environment, Food and Rural Affairs (DEFRA)
Research and Development (R&D), UK, p. 98. <http://www.safecoast.org/editor/
databank/File/risk,%20performance%20and%20uncertainty%20in%20flood%20
defence.pdf> (accessed 16.02.11).
Sheta, A.F., Mahmoud, A., 2001. Forecasting using genetic programming. In:
Proceedings the 33rd Southeastern Symposium on System Theory, Athens,
OH, USA, pp. 343–347.
Sivapragasam, C., Maheswaran, R., Venkatesh, V., 2007. Genetic programming
approach for flood routing in natural channels. Hydrol. Process. doi:10.1002/
hyp.6628.
Tawfik, M., Ibrahim, A., Fahmy, H., 1997. Hysteresis sensitive neural network for
modeling rating curves. J. Comput. Civil Eng. 11, 206–211.
Thirumalaiah, K., Deo, M.C., 1995. Flood estimation using neural networks. In:
International Conference on Disaster and Mitigation, Anna University, Madras,
pp. B2-34–B2-39.
Todini, E., Alberoni, P., Butts, M., Collier, C., Khatibi, R., Samuels, P.G., Weerts, A.,
2006. ACTIF best practice – understanding and reducing uncertainty in flood
forecasting. In: Proceedings International Conference on Innovation Advances
and Implementation of Flood Forecasting Technology, Tromso, Norway; <http://
www.actif-ec.net/conference2005/proceedings/PDF%20docs/Session_09_ACTIF_
best_practice/ACTIF_best_practice_paper_1_Uncertainty_in_flood_forecasting_
V2.pdf> (accessed 16.02.11).