Environmental Modelling & Software 85 (2016) 293e298

Contents lists available at ScienceDirect

Environmental Modelling & Software

journal homepage: www.elsevier.com/locate/envsoft

Short communication

A Matlab toolbox for designing Multi-Objective Optimal Operations of

water reservoir systems
M. Giuliani*, Y. Li, A. Cominola, S. Denaro, E. Mason, A. Castelletti
Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza L. da Vinci, 32, I-20133, Milano, Italy

a r t i c l e i n f o a b s t r a c t

Article history: Water reservoir operations have great potential for contributing positively to the development of
Received 25 May 2016 different socio-economic sectors as well as for reducing the vulnerabilities of water systems caused by
Received in revised form changing hydroclimatic and anthropogenic forcing. This motivates the search for advanced, flexible, and
25 July 2016
open tools supporting the design of operating policies capable of meeting multiple competing objectives.
Accepted 23 August 2016
Available online 15 September 2016
This work contributes the Multi-Objective Optimal Operations (M3O) Matlab toolbox, which allows users
to design Pareto optimal (or approximate) operating policies for managing water reservoir systems
through several alternative state-of-the-art methods. Version 1.0 of M3O includes Deterministic and
Keywords:
Optimal reservoir operations
Stochastic Dynamic Programming, Implicit Stochastic Optimization, Sampling Stochastic Dynamic Pro-
Multi-objective optimization gramming, fitted Q-iteration, Evolutionary Multi-Objective Direct Policy Search, and Model Predictive
Water resources management Control. The toolbox is designed to be accessible to practitioners, researchers, and students, and to
Matlab toolbox provide a fully commented and customizable code for more experienced users.
© 2016 Elsevier Ltd. All rights reserved.

Software availability into commercial software packages (e.g., Environment, 2003;

Name of software: M3O: Multi-Objective Optimal Operations
Version: 1.0 - tested on Matlab R2015a
Developers: Matteo Giuliani, Yu Li, Andrea Cominola, Simona
Denaro, Emanuele Mason, and Andrea Castelletti
Contact email: matteo.giuliani@polimi.it
Year first available: 2016
Available from: GitHub repository (http://mxgiuliani00.github.io/
M3O-Multi-Objective-Optimal-Operations/)
1. Introduction
The problem of optimally designing operating policies for water
storages has been extensively studied in the literature (for a review,
see Yeh, 1985; Labadie, 2004; Castelletti et al., 2008a, and references
therein) and is still existent due to the expected increasing fre-
quency and magnitude of water crises over the next decades as a
consequence of growing population demands and climate change
(Lehner et al., 2006). Many state-of-the-art methods for water sys-
tems analysis, simulation, and optimization have been implemented
* Corresponding author.
E-mail address: matteo.giuliani@polimi.it (M. Giuliani).
Hameed and O'Neill, 2005; Yates et al., 2005), strongly limiting the
possibility of exploring and modifying them for customized appli-
cations. Recently, some open source, flexible tools were released for
specific analysis, such as supporting water systems planning via
water system simulation models (Matrosov et al., 2011) or simu-
lating climate change impacts accounting for the presence of water
reservoirs (Turner and Galelli, 2016). However, despite the growing
interest in multi-objective water management (e.g., Kasprzyk et al.,
2013; Giuliani et al., 2014a; Ward et al., 2015), a comprehensive tool
implementing multiple control policy design methods for the daily
operations of multipurpose water reservoirs is still needed.
In this paper, we contribute a Matlab toolbox for designing the
optimal daily operations of multipurpose water reservoir systems
and exploring the tradeoffs between competing objectives. The
toolbox, called Multi-Objective Optimal Operations (M3O), allows
users to design Pareto optimal (or approximate) operating policies
through several alternative state-of-the-art methods. A solution is
defined as Pareto optimal (or nondominated) if no other solution
gives a better value for one objective without degrading the per-
formance in at least one other objective. The application of different
techniques on the same case study contributes a step-forward with
respect to traditional literature review papers as the availability of
the source code, along with the possibility of cross-comparing the
results on the same problem, allows a better understanding of pros

http://dx.doi.org/10.1016/j.envsoft.2016.08.015
1364-8152/© 2016 Elsevier Ltd. All rights reserved.
294 M. Giuliani et al. / Environmental Modelling & Software 85 (2016) 293e298

and cons of each approach. At the same time, we implemented m ð,Þ for t¼0,1,…,H1 is the m-th immediate cost func-
infinite, gtþ1
M3O in a modular structure, which allows practitioners, re- tion (with m¼1,…,M) associated with the time transition from t to
searchers, and students to easily customize, and possibly further tþ1, g is a discount factor, and J(,) is a statistic used to filter the
develop, the implemented code according to their specific noise generated by the disturbances. In version 1.0 of M3O, this
requirements. latter is the expected value (i.e., J(,)¼E(,)) while other statistics
(e.g., minimax) will be included in future updates of the toolbox. It
2. Description of the toolbox is worth noting that in the case of simulation-based methods such
as ISO, EMODPS, or MPC, the time horizon H is finite and equal to
Version 1.0 of M3O toolbox includes seven state-of-the-art the length of the time-series of disturbances used for the simula-
methods for the design of Pareto optimal (or approximate) con- tion. In these cases, the expected value over the probability distri-
trol policies, which are listed in Table 1. A quick overview on these bution of the disturbances can be approximated with the average
methods is provided in the next sections. value over a sufficiently long time-series of disturbances' re-
alizations (Pianosi et al., 2011). Finally, all methods but EMODPS are
2.1. Problem statement originally single-objective and the generation of the Pareto front is
obtained adopting the weighting method (Gass and Saaty, 1955),
The general multi-objective water reservoir operations problem with the single-objective optimization repeated for every Pareto
can be formulated as a discrete-time, periodic, non-linear, sto- optimal point generated by adapting the weighting of the objec-
chastic Markov Decision Process (Castelletti et al., 2008a) as tives. This method allows exploring only convex tradeoff curves,
follows: with gaps in correspondence to concave regions.






p
¼ min J ¼

J 1 ; J 2 ; …; J M

(1a) 2.2. Dynamic programming
p
Deterministic Dynamic Programming (DDP) and Stochastic Dy-
subject to
namic Programming (SDP) are probably the most widely used
xtþ1 ¼ ft ðxt ; ut ; εtþ1 Þ t ¼ 0; …; H  1 (1b) methods for designing the optimal operations of water reservoir
systems. They formulate the operating policy design problem as a
sequential decision-making process, where a decision taken at a
mt ðxt Þbut 2U t ðxt Þ t ¼ 0; …; H  1 (1c)
given time step produces not only an immediate cost, but also affects
the next system state and, through that, all the subsequent costs
εtþ1  ft t ¼ 0; …; H  1 (1d) (Loucks et al., 2005; Soncini-Sessa et al., 2007). Specifically, the
expected long-term cost of an operating policy is computed for each
x0 given (1e) state variable (e.g., reservoir storage) by means of the value function.
In principle, DP can be applied under relatively mild modeling
pbfmt ð,Þ; t ¼ 0; …; H  1g (1f) assumptions but, in practice, its use in real-world problems is
challenged by three curses, namely the curse of dimensionality
where xt 2ℝnx , ut 2U t ðxt Þ4ℝnu , and εt 2ℝnε are the state (i.e., (Bellman, 1957), the curse of modeling (Tsitsiklis and Van Roy, 1996),
storage), control (i.e., release decision), and stochastic disturbance and the curse of multiple objectives (Powell, 2007). A variety of
(i.e., inflow) vectors of dimension nx, nu, and nε, respectively. In the approximate dynamic programming methods (see Powell, 2007;
adopted notation, the time subscript of a variable indicates the time Busoniu et al., 2010, and references therein) have been developed
instant when its value is deterministically known. The reservoir for overcoming these curses, including the methods described in the
storage is measured at time t and thus is denoted as xt, while inflow next sections which are implemented in the M3O toolbox.
in the interval [t,tþ1) is denoted as qtþ1 because it can be known
only at the end of the time interval. Each objective function Jm 2.3. Implicit Stochastic Optimization
(assumed to be a cost) is formulated as
Implicit Stochastic Optimization (ISO) designs the operating
" #
X
H1 policy relying on the results of deterministic optimization to derive
J m
¼ lim Jε1 ;…;εH gt gtþ1
m
ðxt ; ut ; εtþ1 Þ (2) “optimal” reservoir releases under different system conditions
H/∞
t¼0 (Celeste and Billib, 2009). The ISO procedure is structured in three
main steps: (1) find the optimal sequence of release decision for a
where H is the problem time horizon generally assumed to be
given trajectory of inflow, for example via DDP; (2) select a set of
variables for conditioning the operating policy; (3) run a regression
Table 1 analysis to define a function mapping the variables selected in step
Methods included in the M3O toolbox. 2 into release decisions obtained in step 1. Different forms of policy
Method Acronym Reference can be employed in the ISO procedure, such as linear and nonlinear
polynomials, artificial neural networks, randomized trees, or fuzzy
Deterministic Dynamic DDP Bellman (1957)
Programming
rules (e.g., Momtahen and Dariane, 2007). In the M3O toolbox, we
Stochastic Dynamic SDP implement the regression by relying on a piecewise linear
Programming approximation of the operating policy, also known as Standard
Implicit Stochastic Optimization ISO Celeste and Billib Operating Policy (Draper and Lund, 2004), which maps the reser-
(2009)
voir storage into the release decision.
Sampling Stochastic Dynamic SSDP Faber and Stedinger
Programming (2001)
Fitted Q-Iteration FQI Castelletti et al. (2010) 2.4. Sampling Stochastic Dynamic Programming
Evolutionary Multi-Objective EMODPS Giuliani et al. (2016)
Direct Policy Search Sampling Stochastic Dynamic Programming (SSDP) replaces the
Model Predictive Control MPC Bertsekas (2005)
explicit probabilistic description of the system disturbances of SDP
 M. Giuliani et al. / Environmental Modelling & Software 85 (2016) 293e298
with the use of multiple scenarios of reservoir inflow as an
empirical distribution of streamflow variability (Faber and
Stedinger, 2001). These scenarios can be either multiple historical
time-series (Kelman et al., 1990) or ensemble streamflow pro-
jections (Faber and Stedinger, 2001), with these latter providing
frequently updated information about the current system condi-
tions. The use of streamflow scenarios overcomes the DP curse of
modeling and allows a better characterization of both the spatial
and temporal correlations of the streamflow, which are often
oversimplified in DP applications. In fact, while the difficulty of
including space variability in the identification of the disturbance's
probability density function can reduce the accuracy of distur-
bances representation, temporal correlation can be properly
accounted for only by using a dynamic stochastic model, which
further contributes to the curse of dimensionality. In SSDP, the
policy design assesses release decisions via simulation over the
considered scenarios of inflows, which maintain the streamflow
hydrograph intact, thus accurately representing both flow persis-
tence and spatial correlation.
2.5. Fitted Q-iteration
Fitted Q-iteration (FQI) is a batch-mode reinforcement learning
(RL) algorithm that combines RL concepts of off-line learning and
functional approximation of the value function (Ernst et al., 2005;
Castelletti et al., 2010). This method relies on the experience con-
tained in a sample dataset previously collected either from the
system observations or via model simulations (see run_doe.m
function), which is used for estimating an approximation of the
value function. This makes it possible to mitigate the DP curse of
dimensionality by adopting a very coarse discretization of the state-
control space. In addition, it allows overcoming the curse of
modeling by learning a policy which, beside the state variables, can
be conditioned on some exogenous variables (e.g., observations of
inflows, precipitation, snow water equivalent) in a model-free
setting. In the M3O toolbox, we implement the approximation of
the value function via tree-based regression (Geurts et al., 2006).
We plan to include the multi-objective extension of this method
(Castelletti et al., 2013) in future updates of the toolbox.
2.6. Evolutionary multi-objective direct policy search
Evolutionary multi-objective direct policy search (EMODPS) is a
simulation-based optimization approach that combines direct
policy search, nonlinear approximating networks, and multi-
objective evolutionary algorithms (Giuliani et al., 2016). EMODPS
exploits the parameterization of the operating policies and explores
the associated parameter space to find a parameterized policy that
optimizes the expected system performance. The advantage of
EMODPS with respect to DP is threefold: it overcomes the curse of
dimensionality, as it avoids the computation of the value function;
it overcomes the curse of modeling, as it can be combined with any
simulation model and allows the direct use of exogenous infor-
mation to condition the decisions (Giuliani et al., 2015); it over-
comes the curse of multiple objectives, as the use of multi-objective
evolutionary algorithms prompts the generation or Pareto
approximate set of solutions for up to ten objectives (Giuliani et al.,
2014b). In the M3O toolbox, EMODPS relies on a piecewise linear
approximation of the operating policy, also known as Standard
Operating Policy (Draper and Lund, 2004) (see std_oper-
ating_policy.m function) combined with the NSGAII algorithm (Deb
et al., 2002). More complex parameterization of the operating
policies (Giuliani et al., 2014c) and alternative multi-objective
evolutionary algorithms (Zatarain-Salazar et al., 2016) can be
used for handling more complex problems and will be included in
295
future updates of the toolbox.
2.7. Model predictive control
Model Predictive Control is a real-time control approach based
on the sequential resolution of multiple open-loop control prob-
lems defined over a finite, receding time horizon (Bertsekas, 2005;
Scattolini, 2009). At each time t, a forecast of the external distur-
bances (e.g., the inflow), called nominal value, is provided over the
finite future horizon [t,tþh]. The corresponding sequence of
optimal decisions is then obtained by solving a mathematical
programming problem assuming that the realization of the dis-
turbances will be equal to the predicted nominal value. However,
only the first control is actually applied and, at time tþ1, a new
problem is formulated over the horizon [tþ1,tþ1þh] on the basis of
the updated information available (Mayne et al., 2000; Castelletti
et al., 2008b). MPC provides two main advantages with respect to
DP: it overcomes the curse of dimensionality as searching the
sequence of optimal decisions over a finite horizon does not require
the computation of the value function; it overcomes also the curse
of modeling as it allows exploiting additional and updated infor-
mation at each time instant to better inform operational decisions.
The MPC implementation included in M3O relies on deterministic
forecasts of disturbances, while we will include the stochastic ex-
tensions of this method (e.g., Pianosi and Soncini-Sessa, 2009; Raso
et al., 2014) in future updates of the toolbox.
2.8. Toolbox architecture
The M3O toolbox is organized into multiple tailored directories
associated to the different methodologies implemented (see Fig. 1).
A shared directory (i.e., sim) contains the functions representing
features specific to the water system used for testing the methods. In
particular, it includes the crucial simLake.m function, which imple-
ments the dynamic simulation of the system (over a pre-specified
time horizon associated to a time-series of inflow data) and repre-
sents the main interface between the model of the water system and
the optimization routines provided in the tailored directories.
This modular structure makes the development of additional
code easy, either in terms of adding models of new water systems
by replicating and adapting the structure of the sim directory, or in
terms of contributing new policy design methods by implementing
the corresponding optimization routine in specific tailored di-
rectories and interfacing them with the simLake.m function.
Moreover, the availability of the run_[methodname].m functions
facilitates using only portions of the code. For example, if a user is
interested in a particular optimization method, he/she can down-
load only one run_[methodname].m along with the corresponding
tailored directory. Finally, separate readme files in each directory
provide detailed information about the setting of each method and
how to start new case studies.
3. Example application
3.1. Case study
We showcase the application of M3O on a case study application
implemented in the simLake.m function, which represents a typical
water system composed by a natural catchment, which produces
the inflow qtþ1 that feeds the water reservoir. This latter is operated
on a daily basis according to two conflicting objectives, namely
flood control along the lake shores and water supply to down-
stream users (see Fig. 2). The reservoir dynamics is described by a
mass-balance equation (see massBalance.m function) of volume of
water stored stþ1¼stþqtþ1rtþ1, where the release rtþ1 depends on
296 M. Giuliani et al. / Environmental Modelling & Software 85 (2016) 293e298
Fig. 1. Organization of the M3O toolbox.
the daily release decision ut provided by the operating policy,
which is constrained within a certain zone of operation discreption
by the maximum and minimum feasible release functions (see
max_release.m and min_release.m, respectively) due to the presence
of some physical (e.g., activation of the spillways) and normative
(e.g., minimum environmental flow) constraints depending on the
current water level of the reservoir ht.
The current version of M3O includes the implementation of the
model of Lake Como, a regulated lake in Northern Italy (Fig. 3, left
Fig. 2. Schematic representation of the generic water system targeted for the appli-
cation of the toolbox, composed by a natural catchment, which produces the inflow to
a controlled water reservoir, whose regulation impact on a number of stakeholders'
interest, namely flood control and downstream water supply.
panel). The lake is fed by a 4552 km2 catchment, where the lake
inflow and effluent is the Adda River, which supports several
agricultural districts with a total surface of 1400 km2. Major crops
are cereals, especially maize, along with temporary grasslands for
livestock. Beside water supply, the regulation of the lake, which has
an active storage capacity of 254 Mm3, aims also to prevent flooding
along the lake shores, especially in Como city, which is the lowest
point of the shoreline. The zone of operation discreption for this
system is illustrated in the right panel of Fig. 3, which forces the
lake operator to completely close the dam if the level is
below 0.5 m and, conversely, to completely open the dam if the
level is above 1.25 m.
The competing interests of flood control and irrigation supply
can
be modeled
as a 2-dimensional objective function vector



J ¼
J flood ; J irr
defined according to eq. (2) by using the following
formulations for the immediate costs (see immediate_costs.m
function):
 Flooding: the daily lake level excess with respect to the flooding
threshold (hFLO ¼ 0.8 m), i.e.:
  
flood
gt ¼ max htþ1  hFLO ; 0
 Irrigation: the daily water deficit with respect to the down-
stream demand w ¼ 370 m3/s, i.e.
gtirr ¼ maxððw  rtþ1 Þ; 0Þ
Note that to simplify the problem, we assume the system as
stationary (i.e., we remove the seasonality) and adapted the values
of inflows, water demand, and flooding threshold to maintain the
real conflicts between water supply and flood protection. However,
the underlying idea of the M3O toolbox is to provide a suite of
methods customizable for a variety of users' applications on the
basis of the characteristics of the problem to solve (e.g., number of
reservoirs, use of information, number of objectives).
 M. Giuliani et al. / Environmental Modelling & Software 85 (2016) 293e298 297

Fig. 3. Map of the Lake Como system (left panel) and associated zone of operation discreption for the operations of the lake, bounded by the maximum and minimum feasible
release functions (right panel).

3.2. Numerical results

The design of the Pareto optimal (approximate) set of policies

can be performed through the following code, which is related to
SDP (see the main_script.m for the syntax of the other methods):

where the code above solves Nalt policy design problems, each time
using a different convex combination of the two objectives on the
basis of the value of the weights lambda according to the weighting
method.
The performance of the resulting sets of Pareto optimal oper-
ating policies is reported in Fig. 4, where the flooding objective Jflood
is plotted on the x-axis, while the irrigation supply objective Jirr is
plotted on the y-axis. The black arrows indicate the directions of
increasing preference, with the ideal solution that would be set in
the bottom-left corner of the figure, while each solution represents Fig. 4. Performance of the different sets of Pareto optimal operating policies designed
through the methods implemented in the M3O toolbox.
a different tradeoff between the two objectives. Results show that,
not surprisingly, the solutions designed via Deterministic Dynamic
Programming (black squares) attain the best performance as they number of objectives). In addition, differently from other com-
are obtained under the assumption of deterministic knowledge of mercial software, M3O is a Matlab toolbox providing a fully com-
future inflows. The uncertainty associated to these latter degrades mented code, which can be easily customized as well as further
the performance of the other sets of solutions. It is worth noting developed for including new case study applications or new
that all the methods attain similar performance when optimizing a methodologies. This feature also facilitates the conversion of the
single objective for the extremes of the Pareto front, while code to other programming languages, such as Octave, Python,
obtaining a good coverage of the entire tradeoff curve is more Cþþ, which may reduce the computational requirements of the
challenging, even though we are considering only two objectives. implemented Matlab code.
However, the differences in terms of performance of the different M3O is freely available for noncommercial research and
Pareto fronts seems limited and, likely, not decision-relevant on educational uses trough a GitHub repository.1 We intend to main-
this simple benchmark, while it could be more significant on more tain and further develop the M3O toolbox for responding to users'
complex applications. needs and suggestions. Beside extending some methodologies
already implemented, imminent updates would include (i) the
4. Outlook application of the tools to a cyclo-stationary system, which thus
requires designing cyclo-stationary operating policies that varies
M3O is a modular and flexible toolbox for designing Pareto depending on the day of the year; (ii) introduction of other
optimal (approximate) operating policies for regulating water
reservoir systems. M3O includes several state-of-the-art methods
that can be selected depending on the specific characteristics of the
1
problem to solve (e.g., number of reservoirs, use of information, http://mxgiuliani00.github.io/M3O-Multi-Objective-Optimal-Operations/.
298 M. Giuliani et al. / Environmental Modelling & Software 85 (2016) 293e298
objective functions, such as hydropower production; (iii) applica-
tion to multi-reservoir systems, both in cascade or in parallel. No-
tifications about future releases of the toolbox will be announced
on our website http://www.nrm.deib.polimi.it.
Acknowledgement
We would like to thank all the current and past members of the
Natural Resources Management group (www.nrm.deib.polimi.it) at
Politecnico di Milano who contributed to the development of the
methods implemented in the M3O toolbox.
References
Bellman, R., 1957. Dynamic Programming. Princeton University Press, Princeton.
Bertsekas, D., 2005. Dynamic programming and suboptimal control: a survey from
ADP to MPC. Eur. J. Control 11, 4e5.
Busoniu, L., Babuska, R., De Schutter, B., Ernst, D., 2010. Reinforcement Learning and
Dynamic Programming Using Function Approximators. CRC Press, New York.
Castelletti, A., Galelli, S., Restelli, M., Soncini-Sessa, R., 2010. Tree-based reinforce-
ment learning for optimal water reservoir operation. Water Resour. Res. 46,
W09507.
Castelletti, A., Pianosi, F., Restelli, M., 2013. A multiobjective reinforcement learning
approach to water resources systems operation: Pareto frontier approximation
in a single run. Water Resour. Res. 49.
Castelletti, A., Pianosi, F., Soncini-Sessa, R., 2008a. Water reservoir control under
economic, social and environmental constraints. Automatica 44 (6), 1595e1607.
Castelletti, A., Pianosi, F., Soncini-Sessa, R., 2008b. Receding horizon control for
water resources management. Appl. Math. Comput. 204 (2), 621e631.
Celeste, A., Billib, M., 2009. Evaluation of stochastic reservoir operation optimiza-
tion models. Adv. Water Resour. 32 (9), 1429e1443.
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T., April 2002. A fast and elitist multi-
objective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6 (2), 182e197.
Draper, A., Lund, J., 2004. Optimal hedging and carryover storage value. J. Water
Resour. Plan. Manag. 130 (1), 83e87.
Environment, D.W., 2003. MIKE BASIN: a Versatile Decision Support Tool for Inte-
grated Water Resources Management and Planning.
Ernst, D., Geurts, P., Wehenkel, L., 2005. Tree-based batch mode reinforcement
learning. J. Mach. Learn. Res. 6 (1), 503e556.
Faber, B., Stedinger, J., 2001. Reservoir optimization using sampling SDP with
ensemble streamflow prediction (ESP) forecasts. J. Hydrol. 249 (1), 113e133.
Gass, S., Saaty, T., 1955. Parametric objective function - Part II. Operations Res. 3,
316e319.
Geurts, P., Ernst, D., Wehenkel, L., 2006. Extremely randomized trees. Mach. Learn.
63 (1), 3e42.
Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., Reed, P., 2016. Curses, tradeoffs, and
scalable management: advancing evolutionary multi-objective direct policy
search to improve water reservoir operations. J. Water Resour. Plan. Manag. 142
(2).
Giuliani, M., Galelli, S., Soncini-Sessa, R., 2014a. A dimensionality reduction
approach for Many-Objective Markov Decision Processes: application to a water
reservoir operation problem. Environ. Model. Softw. 57, 101e114.
Giuliani, M., Herman, J., Castelletti, A., Reed, P., 2014b. Many-objective reservoir
policy identification and refinement to reduce policy inertia and myopia in
water management. Water Resour. Res. 50, 3355e3377.
Giuliani, M., Mason, E., Castelletti, A., Pianosi, F., Soncini-Sessa, R., 2014c. Universal
approximators for direct policy search in multi-purpose water reservoir man-
agement: a comparative analysis. In: Proceedings of the 19th IFAC World
Congress. Cape Town (South Africa).
Giuliani, M., Pianosi, F., Castelletti, A., 2015. Making the most of data: an informa-
tion selection and assessment framework to improve water systems operations.
Water Resour. Res. 51 (11), 9073e9093.
Hameed, T., O'Neill, R., 2005. River management decision modelling in IQQM. In:
Proceedings of the International Congress on Modelling and Simulation.
Modelling and Simulation Society of Australia and New Zealand, pp. 170e176.
Kasprzyk, J., Nataraj, S., Reed, P., Lempert, R., 2013. Many objective robust decision
making for complex environmental systems undergoing change. Environ.
Model. Softw. 42, 55e71.
Kelman, J., Stedinger, J.R., Cooper, L.A., Hsu, E., Yuan, S.-Q., 1990. Sampling stochastic
dynamic programming applied to reservoir operation. Water Resour. Res. 26
(3), 447e454.
Labadie, J., 2004. Optimal operation of multireservoir systems: state-of-the-art
review. J. Water Resour. Plan. Manag. 130 (2), 93e111.
Lehner, B., Do €ll, P., Alcamo, J., Henrichs, T., Kaspar, F., 2006. Estimating the impact of
global change on flood and drought risks in Europe: a continental, integrated
analysis. Clim. Change 75 (3), 273e299.
Loucks, D., van Beek, E., Stedinger, J., Dijkman, J., Villars, M., 2005. Water Resources
Systems Planning and Management: an Introduction to Methods, Models and
Applications. UNESCO, Paris, France.
Matrosov, E., Harou, J., Loucks, D., 2011. A computationally efficient open-source
water resource system simulator - application to London and the Thames Ba-
sin. Environ. Model. Softw. 26 (12), 1599e1610.
Mayne, D.Q., Rawlings, J.B., Rao, C.V., Scokaert, P.O., 2000. Constrained model pre-
dictive control: stability and optimality. Automatica 36 (6), 789e814.
Momtahen, S., Dariane, A., 2007. Direct search approaches using genetic algorithms
for optimization of water reservoir operating policies. J. Water Resour. Plan.
Manag. 133 (3), 202e209.
Pianosi, F., Quach, X., Soncini-Sessa, R., 2011. Artificial neural networks and multi
objective genetic algorithms for water resources management: an application
to the hoabinh reservoir in vietnam. In: Proceedings of the 18th IFAC World
Congress. Milan, Italy.
Pianosi, F., Soncini-Sessa, R., 2009. Real-time management of a multipurpose water
reservoir with a heteroscedastic inflow model. Water Resour. Res. 45 (10),
W10430.
Powell, W., 2007. Approximate Dynamic Programming: Solving the Curses of
Dimensionality. Wiley, NJ.
Raso, L., Schwanenberg, D., van de Giesen, N., van Overloop, P., 2014. Short-term
optimal operation of water systems using ensemble forecasts. Adv. Water
Resour. 71, 200e208.
Scattolini, R., 2009. Architectures for distributed and hierarchical model predictive
control - a review. J. Process Control 19 (5), 723e731.
Soncini-Sessa, R., Castelletti, A., Weber, E., 2007. Integrated and Participatory Water
Resources Management: Theory. Elsevier, Amsterdam, NL.
Tsitsiklis, J., Van Roy, B., 1996. Feature-based methods for large scale dynamic
programming. Mach. Learn. 22, 59e94.
Turner, S., Galelli, S., 2016. Water supply sensitivity to climate change: an R package
for implementing reservoir storage analysis in global and regional impact
studies. Environ. Model. Softw. 76, 13e19.
Ward, V., Singh, R., Reed, P., Keller, K., 2015. Confronting tipping points: can multi-
objective evolutionary algorithms discover pollution control tradeoffs given
environmental thresholds? Environ. Model. Softw. 73, 27e43.
Yates, D., Sieber, J., Purkey, D., Huber-Lee, A., 2005. WEAP21 a demand-, priority-,
and preference-driven water planning model: part 1: model characteristics.
Water Int. 30 (4), 487e500.
Yeh, W., 1985. Reservoir management and operations models: a state of the art
review. Water Resour. Res. 21 (12), 1797e1818.
Zatarain-Salazar, J., Reed, P.M., Herman, J.D., Giuliani, M., Castelletti, A., 2016. A
diagnostic assessment of evolutionary algorithms for multi-objective surface
water reservoir control. Adv. Water Resour. 92, 172e185.