Direct Search Approaches Using Genetic Algorithms

for Optimization of Water Reservoir Operating Policies

Sh. Momtahen1 and A. B. Dariane2

Abstract: The direct search approach to determine optimal reservoir operating policies is proposed with a real coded genetic algorithm
共GA兲 as the optimization method. The parameters of the policies are optimized using the objective values obtained from system simula-
Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.

tions. Different reservoir release rules or forms, such as linear, piecewise linear, fuzzy rule base, and neural network, are applied to a
single reservoir system and compared with conventional models such as stochastic dynamic programming and dynamic programming and
regression. The results of historical and artificial time series simulations show that the GA models are generally superior in identifying
better expected system performance. Parsimony of policy parameters is inferred as a principle for selecting the structure of the policy, and
Fourier series can be helpful for reducing the number of parameters by defining the time variations of coefficients. The proposed method
has shown to be flexible and robust in optimizing various types of policies, even in models that include nonlinear, nonseparable objective
functions and constraints.
DOI: 10.1061/共ASCE兲0733-9496共2007兲133:3共202兲
CE Database subject headings: Water resources; Optimization models; Evolutionary computation; Simulation models; Algorithms.

Introduction Rt = St + Qt − bt 共2兲

Finding optimal operating policies for a reservoir system has been
a major area of study in water resources systems. An operating
policy is a set of rules for determining the quantities of water to
be stored and to be released under various conditions 共Wurbs
1993兲. Optimization models are useful tools to identify reservoir
operating policies, but there are some computational presump-
tions in these models that restrict their efficiency and flexibility.
Most of these models are not able to represent the complex physi-
cal, hydrological, and operational characteristics of the system
adequately. Moreover, the objective functions or the form of the
operating rules must often be defined under assumptions which
further constrain the utility of the models. Linear policies and
separable objective functions are examples of this kind.
In chance-constrained linear programming models for ex-
ample, the model is restricted to some very simple rules called
linear decision rules 共LDR兲. These rules relate releases to storage,
inflow, and decision parameters, and represent crucial assump-
tions in the modeling effort 共Yeh 1985兲. Two typical LDR forms
as described by ReVelle et al. 共1969兲 and Loucks 共1970兲 are
Rt = St − bt 共1兲
1
Water Resources Management Ph.D. Student, Civil Engineering
Dept., K. N. Toosi Univ. of Technology, Tehran, Iran. E-mail:
momtahen@alborz.kntu.ac.ir
2
Assistant Professor, Civil Engineering Dept., K. N. Toosi Univ. of
Technology, Tehran, Iran. E-mail: borhani@kntu.ac.ir
Note. Discussion open until October 1, 2007. Separate discussions
must be submitted for individual papers. To extend the closing date by
one month, a written request must be filed with the ASCE Managing
Editor. The manuscript for this paper was submitted for review and pos-
sible publication on January 11, 2005; approved on April 10, 2006. This
paper is part of the Journal of Water Resources Planning and Manage-
ment, Vol. 133, No. 3, May 1, 2007. ©ASCE, ISSN 0733-9496/2007/3-
202–209/$25.00.
where Rt, Qt, and St⫽release, inflow, and beginning storage val-
ues, respectively, for period t and bt⫽operations parameter for
time t. The optimal value of bt is to be found through the appli-
cation of an appropriate optimization method.
Stochastic dynamic programming 共SDP兲 as another optimiza-
tion model is extensively used to determine optimal reservoir
operating policies. SDP is perhaps the most popular representa-
tive of the explicit stochastic optimization 共ESO兲 approach, in
which the stochastic nature of inflows is explicitly considered
in the model. The model is frequently used in optimization of
reservoir systems 共Butcher 1971; Torabi and Mobasheri 1974;
Stedinger et al. 1984兲.
In these models, the decision and state spaces are discretized
and the operating rules are defined as an optimal index of a de-
cision variable for each index of state variables and time periods.
In general, storage levels, inflows, and inflow forecasts are de-
fined as the states in SDP. These models are very flexible in
defining the form of the operating policy. In fact, there are no
presumptions in the form of the policies except the discretization
of state space. An example of such policies is demonstrated in
Fig. 1. In multireservoir systems, however, the model is restricted
by the “curse of dimensionality” which does not allow for a fine
discretization of the state space, or consideration of many reser-
voirs. Moreover, SDP models are restricted to separable objective
functions and constraints.
Implicit stochastic optimization 共ISO兲 is another approach for
defining operating policies. The dynamic programming and re-
gression 共DPR兲 model proposed by Karamouz and Houck 共1987兲
is a representative of it. As a first step in applying ISO, a deter-
ministic optimization model 共usually in the form of dynamic pro-
gramming兲 optimizes the reservoir releases for some sequences of
streamflow. The “state-decision” vectors result from this process.
The operating policy, having a predefined form 共e.g., a linear
polynomial兲, is fit to these vectors in the next step to derive a
“mean” optimal release policy. ISO models have the capability to

202 / JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007

J. Water Resour. Plann. Manage. 2007.133:202-209.


Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.
Fig. 1. A typical SDP rule for a particular inflow interval and
operating period
consider various predefined forms of policies, such as linear
and nonlinear polynomials, artificial neural networks 共ANN兲,
fuzzy rule bases, etc. 共Bhaskar and Whitlatch 1980; Karamouz
and Houck 1987; Raman and Chandramouli 1996; Russell and
Campbell 1996兲.
According to the main idea of the ISO models, any function
that provides a better fit to the state-decision vectors can better
operate the system. However, this assumption is actually not valid
for various reasons. For example, all of the state decision points
are considered to be worth the same in the fitting process. How-
ever, operations during some exceptional conditions, such as
droughts and floods, can be more important than operations dur-
ing normal conditions. On the other hand, the frequent occurrence
of normal conditions may dominate the fitting process and reduce
the weight of exceptional conditions to influence the optimal val-
ues of operating policy parameters.
In the direct search approach, optimization of operating poli-
cies is accomplished directly by finding optimal parameters of the
policy using system simulation results 共e.g., Oliveira and Loucks
1994兲. A direct search model can theoretically optimize any kind
of objective function and operating policy which can be used in
the simulation models. All the behavioral patterns of inflow se-
quences can take part in the optimized policy since the objective
function is directly represented by simulated operating results.
The idea seems to be simple; however, a proper optimiza-
tion method for this approach must possess some special capa-
bilities. The method should be able to optimize any objective
function without presumptions of separability, differentiability,
continuity, linearity, etc. Therefore, traditional optimization meth-
ods, such as linear and dynamic programming, are not proper
for this approach.
The purpose of this paper is to evaluate the direct search ap-
proach in optimization of reservoir systems operation. For this
purpose, an optimization model is proposed using genetic algo-
rithms 共GAs兲 and is compared with other methods.
Optimization Methods in the Direct
Search Approach
Among the various optimization methods available for finding a
solution to reservoir operation problems, nonlinear programming
共NLP兲 and evolutionary algorithms are the methods that can sat-
isfy the requirements of direct search models. These methods can
JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007 / 203
J. Water Resour. Plann. Manage. 2007.133:202-209.
be considered as representatives of the deterministic point-by-
point—or iterative—approach and the random search approach,
respectively 共Deb 2001兲.
Some NLP methods can be used in direct search models, but
they have major weaknesses. They need a good initial solution for
converging to the global optimum; otherwise they may tend to
converge to suboptimal 共local兲 solutions. This is an important
problem, especially in the optimization of complex operating
policies with numerous parameters.
In evolutionary algorithms, and particularly GAs, all regions
of the search space are potentially under exploration by a popu-
lation of solutions. Many of the points are selected and evaluated
in the search space. The best point of the whole process is con-
sidered as the optimum solution. This method does not require a
good initial solution, it avoids the problem of becoming trapped at
local optima, and it does not require continuity of the search
space. As GA is a random search method, its results may vary in
different runs of the process. In fact, the final results represent
some very good solutions, but not the exact global optimum. Be-
cause of system uncertainties in the real world, however, it is
enough to achieve these good solutions as opposed to seeking a
mathematically and theoretically perfect optimum. Hence, the use
of GAs as an optimization method for reservoir system optimiza-
tion seems justified as a direct search approach.
Genetic Algorithm
In a GA, the parameters of the problem should be coded in a
chromosome string that represents a candidate solution to the op-
timization problem. In the first step of the search process, an
initial population of chromosomes is generated, often randomly.
These are considered as “parents,” and their offspring are gener-
ated by reproduction 共or selection兲, crossover, and mutation op-
erators to make the next generation. The offspring would become
the parents of the next generation, and the process continues in a
balance between exploration of search space and convergence to a
neighborhood of the optimal solution. GAs are motivated by the
principles of natural selection and sexual reproduction. Based on
these principles, more fit individuals 共or better solutions兲 are more
likely to survive 共or be selected兲 and pass their superior charac-
teristics to their offspring. The solutions in the new generation are
therefore expected to be generally more fit than their parents.
After several generations, when the solutions are evolved enough,
the best solution is taken as the optimal solution of the problem
共Deb 2001兲.
In spite of the potential capabilities of GAs, there are only a
few studies that make use of them in optimization of water
resource systems operation. In nearly all of these, GAs have
been shown to have quite remarkable capabilities. Oliveira and
Loucks 共1994兲 estimated the optimal operating policies of a
multireservoir system by GAs. They reported their application of
GAs to be a robust method for operating various kinds of multi-
reservoir systems with complicated objective functions. They
also reported that, contrary to SDP models, computational time
does not increase severely for GAs by increasing the state dimen-
sions in multireservoir systems. This work is extended by Oliveira
and Loucks 共1997兲. Wardlaw and Sharif 共1999兲 and Sharif and
Wardlaw 共2000兲 used GAs in a deterministic optimization model
for a multireservoir system. They concluded that GAs are robust
and acceptable methods and are potentially proper replacements
for conventional methods. Cai et al. 共2001兲 combined GAs
with linear programming to solve nonlinear water management
 models. In this method, some complicated variables are consid- Table 1. System Monthly Demands and Inflows
ered fixed so that a linear programming model can solve the rest Inflow 共106 m3兲
of the problem. The complicated variables are thereafter im-
proved by a GA. This method shows superior performance if the Demand Standard
complicated variables are defined properly. Chen 共2003兲 proposed Month 共106 m3兲 Average deviation
a GA to optimize the operating rule curves of a reservoir system January 760 571.7 335.6
efficiently. Tung et al. 共2003兲 used a GA to optimize operational February 770 825.7 463.6
rules and concluded that it is a powerful tool in finding optimal March 710 1,166.5 684.3
water resources management strategies. April 660 1,755.4 830.7
The application of GAs in optimization of water resources May 640 1,538.4 736.2
operation can be divided into two optimization types: “period-of- June 640 848.9 360.5
record release” and “policy parameters” 共Labadie 2004兲. Sharif July 640 501.5 194.8
and Wardlaw 共2000兲 and Oliveira and Loucks 共1997兲 are the rep-
August 830 331.2 130.1
Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.

resentatives of these types, respectively. The framework of the

September 770 242.4 82.2
first type is inferred of rules from DP models. The second type is
October 760 200.4 55.3
a direct search approach using GAs as one of the only proper
optimization methods. The decision variables of this type are the November 760 301.6 213.1
policy parameters, which are generally fewer in number than that December 720 516.1 450.1
of the first type. The resulting shorter chromosome makes GA
computations of the second type more effective. Therefore this
type is used in this paper. Operating Models

Three types of optimization models are used for identifying op-

erating policies: SDP, ISO, and GA direct search models. In ad-
Application in a Reservoir System
dition, operating results of the standard operating policy 共SOP兲
are also considered as a comparison base. This policy is
A single reservoir system is used to evaluate the direct search GA

冦
method and compare it with conventional methods such as DPR St + Qt − Smax if St ⬎ Dt − Qt + Smax
and SDP. A segment of historical data is used to calibrate models
Rt = St + Qt − Smin if St ⬍ Dt − Qt + Smin 共5兲
and generate the artificial time series. The policies are evaluated
using system simulation results for the calibration and verification Dt otherwise
segments of the historical time series and also the artificial series. In fact, SOP policy is simply Rt = Dt, which is compared with
All of the simulations begin in April, and a full reservoir is as- Eq. 共4a兲 and modified, if necessary, to satisfy that constraint. This
sumed as an initial condition. This can be a realistic assumption procedure is followed for all of the other policies too, as shown as
because of the relatively high average inflow and low demand in 1. Rt = R共t , St , Qt , Dt兲;
this month and its previous months. 2. St+1 = St + Qt − Rt, if St+1 ⬎ Smax then St+1 = Smax, if St+1 ⬍ Smin
A single reservoir system on the Dez River in southwest Iran is then St+1 = Smin; and
selected as a case study. It is a multipurpose system with a capac- 3. Rt = St + Qt − St+1
ity of 4,400⫻ 106 m3. For simplicity, water supply is considered
as the only operating aim. Evaporation and direct rainfall on the
reservoir also are ignored and monthly demands are fixed. Forty- ISO Models
one years of monthly inflows, from April 1957 to March 1998, are
used as the historical time series. The monthly demands and the A deterministic dynamic programming model is used to create the
average and standard deviation of the inflows are presented in state-decision vectors from the calibration data. State and decision
Table 1. The first 31 years of the historical time series are con- variables in this DP are considered as the initial and final reser-
sidered as the calibration data, and the remaining 10 years are voir volumes for each month, respectively. The recursive relations
used for verification. are
The objective function of this problem is defined as minimi-
zation of the cumulated operating loss function. The monthly loss f nt 共St兲 = min关Zt + f t+1
n−1
共St+1兲兴 共6兲
St+1
function is considered as a nonsymmetric quadratic function

Zt = 再
0.4共Dt − Rt兲2 if Rt ⬍ Dt
0.1共Rt − Dt兲2 otherwise
共3兲
where St, Qt, and Rt⫽initial reservoir volume, system inflow, and
release, respectively, during period t, and Zt⫽system loss associ-
ated with period t, which is the month number from 1 to 372.
In this backward DP, the reservoir capacity is discretized into
where Zt⫽monthly loss.
30 equal intervals 共31 points兲 and 11,532 共=31 states⫻ 31 years
The problem is subject to these constraints:
⫻ 12 months兲 state-decision vectors are used for defining various
Smin ⱕ St ⱕ Smax 共4a兲 policy functions of ISO models to be fitted.
DPR: The first and simplest rule that is considered for fitting
Rt ⱖ Rmin 共4b兲 on the state-decision vectors is a linear function. The function is
given as
St+1 = St + Qt − Rt 共4c兲 R t = a tS t + b tQ t + c t 共7兲
where S, Q, and R⫽reservoir storage, inflow, and, release, where at, bt, and ct are defined by linear regression for each
respectively. month.

204 / JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007

J. Water Resour. Plann. Manage. 2007.133:202-209.


Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.
Fig. 2. ANN structure
DPN: As the next ISO model, an ANN is used for fitting the
state-decision vectors. This model is called “DPN” by Raman and
Chandramouli 共1996兲 which means a DPR model using neural
network policy structure. For the DPN model, a simple three-
layer ANN is considered 共Fig. 2兲, with five neurons in its hidden
layer.
The ANN architecture is decided by the mean square error
共MSE兲 index. Fewer neurons in the hidden layer lead to larger
MSE, and more neurons make the ANN more complicated,
often without decreasing the MSE appreciably. More compli-
cated ANNs are more capable to be overfitted to calibration data,
which can lead to good calibration results but poor performance
in verification.
For this DPN model, 12 monthly ANNs are trained using DP
state-decision vectors.
DPF: A fuzzy rule base 共FRB兲 is used to generalize the results
of the deterministic DP model. This model is called “DPF” like
the DPN model where a FRB policy is replaced with ANN. The
rule base consists of some fuzzy rules, such as
If x is A1 and y is B1 then z is C1
where x and y⫽input 共state兲 variables; z⫽output 共decision兲 vari-
able; and A1, B1, and C1⫽fuzzy values.
In the DPF model, the parameters of FRB are estimated using
a fuzzy clustering method 共Zimmermann 1996兲. The FRB is
obtained by 5, 3, and 5 Gaussian membership functions for res-
ervoir storage, inflow, and release, respectively. The number of
membership functions is determined in a trial-and-error procedure
to minimize the loss function.
ESO „SDP… Models
The recursive relations of the SDP model are
St+1
冋
f nt 共St,Qt兲 = min Zt + 兺 P共Qt+1兩Qt兲 · f t+1
Qt+1
n−1
共St+1,Qt+1兲
A first-order Markov chain is used for inflow patterns, and a
backward optimization process is iterated in the annual cycle to
achieve the steady-state solution. In this process, reservoir storage
in the beginning of each month and system inflow are considered
as the state variables, and reservoir storage in the beginning of the
next month is considered as the decision variable.
JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007 / 205
J. Water Resour. Plann. Manage. 2007.133:202-209.
Fig. 3. Representation of GAR policy parameters as a chromosome
Two SDP models with 30 and 60 discrete intervals for reser-
voir storage and 5 and 10 intervals for inflow, respectively, are
considered as ESO models.
Direct Search GA Models
A real-coded GA is used in these models in which any decision
variable is encoded as a single gene with a real number and arbi-
trary precision. The following forms of operating policies are
used for optimization by this GA:
GAR: The policy of this model is the same as for DPR policies
关Eq. 共7兲兴, where at, bt, and ct are the policy parameters or decision
variables for each time period, t. The problem can be represented
in a chromosome as Fig. 3.
The parent population size, crossover rate 共pc兲, mutation rate
共pm兲, and tournament size are assumed equal to 200, 0.6,
0.02, and 2, respectively. Fig. 4 shows evolution of the objective
function in GA generations using different runs. It indicates how
these runs converge to the same objective function.
GAR-F: In the direct search models, continuous functions,
such as Fourier series, can be used for defining the time variations
of policy parameters. The Fourier series for a policy parameter x
in period t can be represented as
x共t兲 = A0 + A1 cos共2␲t/T兲 + B1 sin共2␲t/T兲 + A2 cos共4␲t/T兲
+ B2 sin共4␲t/T兲 + ¯ 共10兲
where Ai and Bi⫽coefficients and T⫽total number of time peri-
ods. For example, the Fourier series of the bt coefficient in Eq. 共7兲
is obtained as
bt = 0.1295 − 0.0055 cos共2␲t/12兲 − 0.0282 sin共2␲t/12兲
共8兲
+ 0.0006 cos共4␲t/12兲 + 0.0119 sin共4␲t/12兲 共11兲
This type of operating policy can be optimized using the direct
search approach, in contrast to ESO and ISO models. A few terms
of a Fourier series can properly estimate all monthly parameters,
or parameters of even shorter periods. Therefore, the number
册 共9兲
Fig. 4. GAR optimization results for some different runs
Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.

Fig. 5. Value of the first coefficient of Eq. 共7兲 obtained from DPR,
Fig. 6. Form of operating rule for GAPL model
GAR, and GAR-F models

of policy parameters and chromosome length of a GA can be

reduced which improves the performance of the optimization
process.
This model with the policy of Eq. 共7兲 and Fourier series coef-
ficients is defined as GAR-F. The first five terms of the Fourier
series are used for each of the three policy coefficients. This
model has only 15 parameters 共less then half of the number of
parameters in the GAR model兲.
Fig. 5 shows the optimized monthly values for the first
coefficient of the linear policy obtained from DPR, GAR, and
GAR-F models. The value of all coefficients is shown in Table 2.
GAPL: A simple piecewise linear rule represents the policy of
this GA model. The rule can be assumed as an improved version
of the SOP as shown in Fig. 6. It has seven parameters 共XL, XR,
Y L, Y R, ␤L, ␤R, and a兲 for each month.
In addition to the conventional constraints, there is a simple
hidden constraint in this model
XL ⬍ XR
A simple repairing strategy is used to handle this constraint:
when the constraint is violated, XL and XR are replaced with each
other.
GAN: This model is the direct search counterpart of DPN.
This 252-parameter model is more complicated than the previous
GA models.
GAF: This model uses a FRB as the operating policy like DPF.
The fuzzy rule in this model is defined as
If x = A1 and y = B1 then z = ax + by + c 共13兲
where a, b, and c⫽rule parameters; x and y⫽input 共state兲
variables; z⫽output 共decision兲 variable; and A1 and B1⫽fuzzy
variables.
Three and five fuzzy variables are considered for storage and
inflow variables, respectively, in each period. This model has 540
parameters and can be seen as an extended version of GAR.
Comparison of the Results
In comparing performance of the models, it should be noted that
all comparisons are from results of system simulations of the
operating policies. The policies can generally be used in any op-
erating condition and not specifically for the determined inflows.
The resulting objective values for the calibration data are then
expected to be weaker than the deterministic optimization results.
共12兲
Hence, the results of the deterministic DP are assumed as a base-
line for the best possible operations with perfect foresight.
To obtain a thorough evaluation of the properties of the poli-
cies, an ARMA model is used to generate artificial time series and
the system is simulated using them. An ARMA共1,1兲 is fitted to the
normalized standardized calibration inflows. The AR and MA
parameters are equal to 0.825 and −0.164, respectively. Twenty
artificial 44 monthly series are generated and used to simulate the
obtained optimal policies.

Table 2. Value of Coefficients of Eq. 共7兲 Obtained from Different Models

DPR GAR GAR-F
t a b c a b c a b c
1 0.123 0.787 192.5 0.028 0.817 312.8 0.097 0.579 327.0
2 0.139 0.637 132.2 0.137 0.548 213.6 0.110 0.506 283.3
3 0.179 0.586 −77.5 0.082 0.402 237.1 0.136 0.391 186.9
4 0.234 0.517 −451.5 0.112 0.313 65.3 0.159 0.289 67.3
5 0.179 0.377 −251.4 0.154 0.280 −33.2 0.167 0.216 −2.3
6 0.101 0.272 74.3 0.164 0.082 87.4 0.157 0.155 34.3
7 0.102 0.298 103.1 0.172 −0.261 194.3 0.141 0.099 163.4
8 0.115 0.277 312.1 0.146 −0.102 375.7 0.129 0.074 309.4
9 0.120 0.480 252.3 0.099 0.456 345.5 0.125 0.122 395.6
10 0.125 0.661 268.2 0.120 0.239 405.9 0.121 0.255 402.8
11 0.121 0.621 299.3 0.086 0.331 478.4 0.112 0.427 370.1
12 0.115 0.395 350.5 0.078 0.376 415.0 0.101 0.554 343.7

206 / JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007

J. Water Resour. Plann. Manage. 2007.133:202-209.

Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.

Fig. 8. Comparison of loss function differences between calibration

and verification, and calibration and deterministic DP
Fig. 7. Comparison of annual losses of different models

The values of annual operation losses are shown in Table 3 and
Fig. 7 for calibration, verification, and average of artificial time
series. Differences between calibration and verification annual
losses and between losses of deterministic DP and other models
are also mentioned in Table 3 and in Fig. 8. The verification
losses are generally more than the calibration due to possible
overfitting during calibration and a flooding period in the verifi-
cation data set.
The form of the operating rule and the optimization method
influence the simulation results. The calibration results of differ-
ent policies of an optimization approach show the potential ability
of the policy to hedge against inflow uncertainty. On the other
hand, the consistency of the policy can be evaluated by compar-
ing calibration and verification results. A policy with good results
both in calibration and verification is suitable for future unknown
conditions, but the policy may be overfitted to the calibration data
if calibration results are not repeated in verification.
In the ISO models, ANN and FRB structures produce the best
and the worst calibration objective functions, respectively. The
simple linear form is a bit weaker than ANN in calibration, but it
is the best in verification. Therefore the ANN seems to be over-
fitted. These policy structures have the same properties in GA
models, too. However, GA models are generally better than ISO
models. The piecewise linear structure which can only be opti-
mized by direct search is the best policy in the calibration runs.
However, it is severely limited in verification. This means that
this policy is also overfitted. The GAR-F with the fewest param-
eters shows the opposite performance. This model is a bit weaker
than GAR in calibration, but it is the best model in verification.
A principle similar to the principle of parsimony of parameters
in time series modeling can therefore be inferred here, too. A
model with fewer parameters and robust performance is more
appropriate for a policy optimization, and the resulting policy is
expected to be more consistent in model verification and in res-
ervoir operations under future—but unknown—conditions.
Fewer policy parameters not only help to prevent overfitting,
but also reduce the optimization computations, especially in the
GA method. This can be inferred from Fig. 9 which illustrates
evolution trend of the best chromosomes of different GA genera-
tions. GA chromosomes are assumed to be converged to optimal
solution when the differences among their fitnesses approach to
zero. The converging times for policies with large numbers of
parameters, like GAN, and GAF, are greater than for approaches
with fewer parameters, such as GAR. This may be due to two

Table 3. Simulated Loss Functions of the Models

Annual penalty Difference 共%兲
Number Calibration
of policy Artificial and Calibration
Model variable Calibration Verification time series verification and DP
ISO
DPR 36 214,451 371,000 224,788 42.2 70.3
DPN 252 191,641 376,127 693,097 49.0 52.2
DPF 180 283,288 777,521 330,700 63.6 125.0
ESO
SDP共30,4兲 1,488 258,321 393,607 295,811 34.4 105.2
SDP共60,10兲 7,320 197,250 384,942 227,404 48.8 56.7
Direct search
GAR 36 187,530 362,188 219,666 48.2 49.0
GAR-F 15 192,144 353,444 220,472 45.6 52.6
GAPL 84 171,574 474,082 257,828 63.8 36.3
GAN 252 176,858 380,135 1,006,982 53.5 40.5
GAF 540 274,363 729,661 564,650 62.4 117.9
SOP — 293,489 450,475 324,090 34.8 133.1
Deterministic DP — 125,889 — 132,498 — 0.0

JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007 / 207

J. Water Resour. Plann. Manage. 2007.133:202-209.

Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.
Fig. 9. Evolution of GA models within the processing time
reasons: 共1兲 GA needs more computational effort to optimize with
longer chromosomes; and 共2兲 the complex rules with more param-
eters take more effort to apply in each period of simulation.
Summary and Conclusion
In this paper, the direct search approach is proposed as an opti-
mization model of reservoir operating policies. The policies are
obtained directly by optimization of their parameters using the
simulation results. GA is selected as a proper optimization
method in this approach, since any parametric type of policies can
be optimized by this method. Using this approach, the policies are
properly optimized contrary to ISO methods in which frequent
normal operating conditions dominate the optimization results
and prevent deriving true optimal policies. On the other hand, the
curse of dimensionality does not exist in the direct search models
because there is no need to discretize the state space of the prob-
lem. However, the optimization method of direct search models
must possess some special capabilities to be proper for use by this
approach. GAs possess these capabilities sufficiently.
The direct search models using GA optimization method are
applied to a reservoir system and compared with some conven-
tional models involving ESO and ISO approaches. Policies with
linear, ANNs, and FRBs are obtained using a dynamic program-
ming and regression method as ISO models. The policy of the
SDP model is evaluated as the representative of the ESO ap-
proach. In addition to the policies of the ISO models, other policy
structures, such as piecewise linear form, are optimized using the
direct search. In this approach, it is also possible to define the
time variations of policy parameters as a mathematical function in
spite of defining different rules for each period of the year. A
Fourier series is used as this function.
A parsimony principle can be inferred from the results of this
study about the policy structures. A smaller number of policy
parameters is an advantage due to this principle, as a large
number of parameters may lead the model to overfitting and in-
crease the required computational effort. Hence, according to
our results, a simple linear policy may operate the system better
208 / JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007
J. Water Resour. Plann. Manage. 2007.133:202-209.
than a complex ANN. Using a Fourier series to define time varia-
tions, the number of policy parameters can be decreased and the
linear policy with a Fourier series has been the best model in
verification.
Artificial time series are also used to confirm the simulation
results of the historical series. The inferences from average results
of artificial time series were generally the same as those of the
historical series. It seems that using long artificial time series is
unnecessary in optimization of policies and historical series are
adequate either in the ISO approach or in direct search.
The main weakness of GA models is the dissimilarity of the
values of the optimized parameters obtained from different runs
of the model. The parameters may vary in spite of relatively
stable results of the objective function. This can be considered as
an advantage, too, because it lets the decision maker choose a
policy among some good policies obtained from different runs.
The direct search approach for optimization of reservoir op-
eration policy parameters using GAs is a very attractive model. In
this model, various forms of policies can be examined in spite of
the limitations imposed by the restrictive assumptions of conven-
tional optimization methods. Optimization of inseparable objec-
tive functions, such as reliability, and even multiobjective optimi-
zations are easily possible with this approach.
Acknowledgments
The writers would like to thank Professor Mac McKee, Director
of the Utah Water Research Laboratory, for reviewing an early
version of the paper.
Notation
The following symbols are used in this paper:
A1 , B1 , C1 ⫽ fuzzy variables;
a , b , c ⫽ policy coefficients;
D ⫽ system demand;
 Q ⫽ system inflow;
R ⫽ system release;
S ⫽ system storage;
T ⫽ number of periods in a year;
t ⫽ operating period;
X,Y ⫽ coordinates;
Z ⫽ objective function; and
␤ ⫽ line slope.
References
Bhaskar, N. R., and Whitlatch, E. E. 共1987兲. “Comparison of reservoir
Downloaded from ascelibrary.org by University of Hong Kong on 03/10/13. Copyright ASCE. For personal use only; all rights reserved.
linear operating rules using linear and dynamic programming.” Water
Resour. Bull., 23共6兲, 1027–1036.
Butcher, W. S. 共1971兲. “Stochastic dynamic programming for optimum
reservoir operation.” Water Resour. Bull., 7共1兲, 115–123.
Cai, X., McKinney, D. C., and Lasdon, L. S. 共2001兲. “Solving nonlinear
water management models using a combined genetic algorithm and
linear programming approach.” Adv. Water Resour., 24共1兲, 667–676.
Chen, L. 共2003兲. “Real coded genetic algorithm optimization of long term
reservoir operation.” J. Am. Water Resour. Assoc., 39共5兲, 1157–1165.
Deb, K. 共2001兲. Multiobjective optimization using evolutionary algo-
rithms, Wiley, Chichester, U.K., 81–165.
Karamouz, M., and Houck, M. H. 共1987兲. “Comparison of stochastic and
deterministic dynamic programming for reservoir operation rule gen-
eration.” Water Resour. Bull., 23共1兲, 1–9.
Labadie, J. W. 共2004兲. “Optimal operation of multireservoir systems:
State-of-the-art review.” J. Water Resour. Plann. Manage., 130共2兲,
93–111.
Loucks, D. P. 共1970兲. “Some comments on linear decision rules and
chance constraints.” Water Resour. Res., 6共2兲, 668–671.
Oliveira, R., and Loucks, D. P. 共1994兲. “Derivation of operating policies
JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT © ASCE / MAY/JUNE 2007 / 209
J. Water Resour. Plann. Manage. 2007.133:202-209.
for multireservoir systems by genetic algorithms.” Advances in water
resources technology and management, Balkema, Rotterdam, The
Netherlands, 221–228.
Oliveira, R., and Loucks, D. P. 共1997兲. “Operating rules for multireservoir
systems.” Water Resour. Res., 33共4兲, 839–852.
Raman, H., and Chandramouli, V. 共1996兲. “Deriving a general operating
policy for reservoirs using neural network.” J. Water Resour. Plann.
Manage., 122共5兲, 342–347.
ReVelle, C., Joeres, E., and Kirby, W. 共1969兲. “The linear decision rule in
reservoir management and design. I: Development of the stochastic
model.” Water Resour. Res., 5共4兲, 767–777.
Russell, S. O., and Campbell, P. F. 共1996兲. “Reservoir operating rules
with fuzzy programming.” J. Water Resour. Plann. Manage., 122共3兲,
165–170.
Sharif, M., and Wardlaw, R. 共2000兲. “Multireservoir systems optimization
using genetic algorithms: Case study.” J. Comput. Civ. Eng., 14共4兲,
255–263.
Stedinger, J., Sule, B., and Loucks, D. 共1984兲. “Stochastic dynamic pro-
gramming models for reservoir operation optimization.” Water Re-
sour. Res., 20共11兲, 1499–1505.
Torabi, M., and Mobasheri, F. 共1974兲. “A stochastic dynamic program-
ming model for the optimum operation of a multipurpose reservoir.”
Water Resour. Bull., 9共6兲, 1089–1099.
Tung, C., Hsu, S., Liu, C. M., and Li, S., Jr. 共2003兲. “Application of the
genetic algorithm for optimizing operation rules of the LiYuTan res-
ervoir in Taiwan.” J. Am. Water Resour. Assoc., 39共3兲, 649–657.
Wardlaw, R., and Sharif, M. 共1999兲. “Evaluation of genetic algorithms
for optimal reservoir system operation.” J. Water Resour. Plann. Man-
age., 125共1兲, 25–33.
Wurbs, R. A. 共1993兲. “Reservoir-system simulation and optimization
models.” J. Water Resour. Plann. Manage., 119共4兲, 455–472.
Yeh, W. W.-G. 共1985兲. “Reservoir management and operation models:
A state-of-the-art review.” Water Resour. Res., 21共12兲, 1797–1818.
Zimmermann, H. J. 共1996兲. Fuzzy set theory and its applications, Kluwer
Academic, Boston, 245–266.