IMPLICIT STOCHASTIC OPTIMIZATION AND SIMULATION

D.P. LOUCKS
School of Civil and Environmental Engineering
Cornell University
Hollister Hall
Ithaca, New York 14853-3501
USA

ABSTRACT. This paper will review how historical and stochastic

hydrological time series data have been incorporated into optimization and
simulation models designed to aid water resources development planning and
management. Optimization modelling, used most often for preliminary
design, allocation, and sequencing or scheduling analyses, can be either
deterministic or stochastic with respect to the incorporation of
hydrological time series. Deterministic models can be solved for numerous
alternative and equally likely hydrological sequences to obtain at least
some idea of the probability distributions of various design or operating
variable values required to achieve specific objectives. Alternatively,
hydrological uncertainty can be incorporated directly into some
formulations of stochastic optimization models. Both approaches using
optimization have their advantages and limitations. Regardless of the
sophistication of the optimization modelling, the output from these
preliminary analyses should be studied further and evaluated using
simulation. Recent work at a number of research institutes in Europe and
North America over the past three years has resulted in an approach to
dynamic river system simulation that can incorporate multiple
hydrologicalal sequences in each time step of the simulation. This permits
some estimation of the reliabil i ty of various system performance indicators
in each time period of the simulation.

1. Introduction

There is ample evidence that optimization of simulation models can be

useful tools for ident.ifying and evaluating the impacts of various
alternative systems that are designed and operat.ed to solve a variety of
water resources problems. Yet, these models are not likely to be very
useful unless they consider the uncertain conditions affecting the future
performance of those systems. This includes the uncertain future demands
imposed on those systems, the uncertain future costs of those syst.ems, and
the uncertain quantities and qualities of the flow within those systems.
Assumptions made regarding each of these sources of uncertainty can have
a major impact on system design, operat.ion and performance.
307
J.B. Marco et al. (eds.),
Stochastic Hydrology and its Use in Water Resources Systems Simulation and Optimization, 307-318.
© 1993 Kluwer Academic Publishers.
308

These facts have served to motivate the development of stochastic models;

models that take into consideration at least some of the important sources
of uncertainty and its impact on system design and operation. All existing
stochastic models can be classified as one of two types: explicit or
implicit. For the purpose of these notes, explicit stochastic models will
be defined as those that incorporate, within the model, at least some of
the uncertainty associated with at least some of the inputs or outputs.
All models that consider, for example, multiple possible flows with their
associated probabilities, or multiple possible demands or costs with their
estimated probabilities, would be examples of explicit stochastic models.
Models whose variables are joint probabilities, or whose constraints
include chance constraints with specified probabilities less than one, are
also explicit stochastic models.
Implicit stochastic models are deterministic models designed and used for
identifying and evaluating system performance in its uncertain or
stochastic environment. Most, if not all, simulation models can be placed
in this category if they are used, for example, with multiple flow
sequences to obtain multiple outputs that can be converted to probability
distributions of future system performance.
Take, for example, the sequent peak algorithm for sizing a reservoir to
produce specified releases in each period t. The algorithm computes
accumulated storage volume deficits in each period t, given known Inflows
and Releases.

Initial Deficitt - Inflowt + Release t = Final Deficit t

which is equal to the Initial Deficit t+l . If the resulting deficits are
less than 0, then

Final Defici tt = Initial Defici t U.l =0

Assumirg an initial deficit of 0 and solving these series of equations
for two cycles of periods t, where in each cycle the Inflow and Release
vectors are identical, will result in a finite Required Capacity if the
mean Release is no more than the mean Inflow

Required Capacity = max (Final Deficitt )

t

Clearly, different inflow sequences will result in different Required

Capacities. Even more clearly, if we consider different Release (or
demand) sequences as well as Inflow sequences, we will likely obtain an
even wider range of Required Capacities. Using the sequent peak algorithm
in multiple simulations using multiple, but equally likely, Inflow and
Release sequences will provide a basis for defining a distribution function
of Required Capacities.
Now, this analysis can be extended to find the capacity that should be
selected based on some economic or other criterion and, very importantly,
based on a more realistic simulation of the reservoir system (see Figure
1). A realistic simulation of this problem would likely indicate that the
reservoir will have to be a little bigger to achieve the same probabilistic
performance as predicted by the sequent peak algorithm simulations. This
 309

is because in the sequent peak simulations discussed above, no

consideration is given to evaporation or to a realistic operating policy.
It is this latter issue, how to define an effective reservoir operating
policy, possibly along with an appropriate active storage capacity, that
has attracted, and still attracts, the attention of numerous stochastic
modellers.

1.0

Prob(Required
Capacity ( K)

o~-c~ ______________ ~

K • Capacity

Figure 1. Outcome of multiple simulations of the sequent-peak algorithm.

Implicit stochastic optimization models are deterministic models used to

identify and evaluate reservoir release (or groundwater pumping) rules and
water use allocation rules, perhaps together with design parameters for a
hydrologicalally stochastic environment. Reservoir release rules typically
define the releases as a function of storage volume and time of year.
Similarly, groundwater pumping rates may be functions of aquifer head and
time of year. Allocation or diversion functions are typically functions
of streamflow at the withdrawal site, or of storage in upstream reservoirs.
All of these implicit models used to identify these release and allocation
functions assume that the probability of all unknown variables is one.
This is clearly not the case. So why use deterministic models? It seems
to me there is only one reason: they are simpler and cheaper to develop and
solve. Hence if they can be used in a way that will provide some good
operating policy and design variable values for more thorough evaluation
using a stochastic simulation model of that system, why not?

2. Optiaization of Reservoir Operation

One of the first applications of implicit stochastic optimization was that

of Young (1967) who used deterministic dynamic programming with streamflow
generation and regression techniques to derive stochastic operating
policies for an existing reservoir of known capacity. Without attempting
to review ali the details of Young's excellent work, which others have
310

extended over the past twenty years. consider the following introductory
example.
If one can assume that the inflows and the objective function OBJt (St.
Qt •.Rt) of initial storage St. inflow Qt and Release Rt in each within-year
period t (t = 1.2 •...• T). are constant from year to year. then
deterministic dynamic programming can be used to find a steady-state
release rule that is a function of reservoir storage St and the within-year
period t (t = 1.2 •.•.• T). For example. a backward-moving approach would
involve solving for the cumulative optimal (maximum or minimum) sum of
objective function values Ftn(St) for an increasing number n of remaining
periods given each initial storage St. The sequence of recursion
equations. beginning with any FtO(St) = 0 for all St.

Ftn(St) = max or min {OBJt(St. Qt Rt) + Ft +l n-1(St+l)}

Rt
(Rt feasible)

must be solved for each value of St and for increasing values of n as is

necessary to reach a steady state. Steady state is achieved when

Ftn(St) - Ftf(St) = constant

and the particular release Rt that defines Ftn(St) are constant from one
year. n. to the next. n-T. for all within-year periods t and St.
Now. repeating this procedure for m different. but equally likely.
hydrologicalal yearly sequences would produce m different reservoir release
policies for each within-year period t. One can then compute the mean
release for each initial storage level St in each within-year period t. or
find a least-squares fit to a piecewise linear. quadratic. or other type
of continuous release rule function Rt = ft(St) if desired. to obtain an
improved steady-state operating policy for a more thorough evaluation using
simulation.
If it is appropriate to define the operating policy as a function of the
inflow Qt as well as initial storage St. the deterministic recursive
equations can be expressed as functions FnD(St.Qt) of storage St and inflow
Qt just as they would be if explicit stochastic dynamic programming were
used. In the explicit case. the sequence of recursion equations needs to
be solved only once to define a policy. In this implicit stochastic
dynamic programming case. numerous policies would have to be derived for
numerous annual hydrologicalal sequences. as before. Then some averaging
or regression technique could be used to define the policy. This policy
would require a forecasting of the flow for the current period. In real
time applications. the period of forecasting could be daily even though the
model time step might be monthly. since the model results can be converted
to a daily release rate.

3. Multi-purpose. Multi-reservoir Design and Operation

The so-called yield model. introduced in Biswas (1976) and Loucks et al.
 311

(1981). and extended by Lall and Miller (1988). is an example of a

deterministic optimization model that can identify both design and
operating policy variable values that influence the reliability of water
allocations, or yields, as well as their amounts. Without going into all
the extensions and variations of this class of models, a simple example can
illustrate the approach.
Consider a three-reservoir system as shown below in Figure 2. Reservoirs
A, Band C are to be operated jointly to provide a reliable water supply
or yield at Site D. Part of this yield is diverted to a water use. The
remainder flows downstream along with any excess flows. The allocation or
diversion to the water use will be a function of the total yield available
at Site D. The combined reservoir release rule, at Site D, will specify
how much water is to be released from the upstream reservoirs as a function
of their total storage. The reliability of the flow to be achieved at Site
D is specified. Assume the quantity desired with that reliability is also
known.

Release D
Rule Site Diversion
Site

Figure 2. A multi-reservoir water resource system.

To be determined are the reservoir capacities and operating policies that
will provide those reliable flows, or "yields", at Site D. To reduce the
number of constraints as well as to identify (for those who are used to
distinguishing between) over-year and wi thin-year storage requirements, two
sets of storage continuity and reservoir capacity constraints are used.
The over-year continuity constraints at Sites s = A and C equate initial
storage SJI, in year y, plus the unregulated inflow, Q/,
less the total
release Ryl to the final storage in Year y, or the initial storage S5 y•1 in
the following year. The total release is equated to a yield component Y5
which is the same in all years the yield is to be achieved, plus an excess
release Eyr. Hence:
Syl + Qyl - y l + Ey' = Sl y•1
for those Years y in which the yield is to be made available, and
312

S I Q5 E 5 - S5
Y + Y - Y - y+l
for those Years y in which the yield is not to be made available. The
reliability of the yield is approximately equal to the number of years the
yield is available divided by one plus the number of years of record,
providing, and this is important, that the excess release Eyl in those
failure years is less than the yield yr.
At Reservoir Site B, the incremental inflow QyB - Qr' and excess release
from Reservoir A must be included in the inflow.
B for all successful y
= S y+l
and
SyB + QyB + Q'
y + E'
y _ -y
R_B = SBy+l for all unsuccessful y
By the selection of the number of critical years in which yield failure
is permitted, the reliability (probability of success) is determined.
Clearly, each sequence of years in which for all Sites s the Qy is greater
than the mean annual flow can be grouped into a single constraint if
desired for model size reduction.
These yield variables Yare annual values having annual reliabilities.
Their distribution within the year may require additional within-year
storage. For each within-year period t, the within-year continuity
constraints are:

St l + 13tT - Yt l = Slt+l for s = A, C

St S + 13ty B + Yt' - Yt B = SBtt!
wh1'e 13 t are the distribution coefficients for within-year yield inflows
(~ 13t = 1.0). Since excess releases are not to be allocated at Site D,
there is no value building capacity to regulate them.
The over-year capacity Ku l is added to the within-year capacity K"I to
provide an estimate of required total capacity KS at each reservoir.
S S < K I for all s
Y - 0

and
for all s, t
or

St l S K"s for all t, Si and KOI + K"I = KI for all s

At Site D the total yield Yt D available in each within-year period t is

Yt D = min {qDyt - qByt - qCyt } + Yt B + Yt C

y
y e success years
 313

where qyt S is the unregula)ed flow at Site s in Year y, Period t. Each

within-year period yeild Yt at Site D can be allocated am~ng the diversion
use and the downstream uses, as required. Now, if all Yt are predefined,
i.e. are known variables, then one could find the least-cost combination
of reservoir capacities required. Other objectives might include the
yields YtD as unknowns. Nevertheless, the resulting model is
deterministic, yet it provides probabilistic information. It recognizes
the stochastic nature of the inflows and hence of the releases too.
To define the operating policy that will produce these series of yields
Yt D and maintain their reliabili ties, a storage-release rule can be
constructed at each reservoir site using the required over-year storage Ko
and within-year storage volumes, St. For this single-capacity yield
problem, there will be only two zones, at most, of storage for each time
period t. Figure 3 shows what this reservoir release rule policy may look
like.

Capacity

Storage
Volume

~------~--------------~~------~ 0
KS

o
T
Within-year Period

Figure 3. A reservoir release rule derived from the yield model.

If the actual storage volume in the reservoir in a period t is within the
lower zone, only the yield Yt should be released. One should be able to
do this successfully with the prespecified reliability. If the storage
volume is in the upper zone, additional water could be released if desired.
This policy can be simulated, and adjustments to capacity and to the
release rule can be made as required to improve the design and operation
of the system. If additional incremental yields at lower reliabili ties are
included in the model, additional zones of storage with their increased
releases would be defined in the release rule shown in Figure 3.
For a more coordinated operation, it would be desirable to define a
single-release rule at Site D as a function of total storage upstream.
This can be done in the same manner as the individual reservoir release
314

rules were constructed, except that Ka$ is replaced with Es Ko$ and each StS
is replaced with Es St s • Next, balancing functions are needed to define
the desired distributlon of upstream storage given all possible values of
total upstream storage. Regretfully, very little information can be
obtained from the model solution to help one derive these functions. Hence
for reservoirs in parallel, a combination of space rules that equalize the
probability of filling during filling conditions, storage rules that
equalize the probability of emptying during drawdowns and common sense need
to be used to derive some balancing functions before a simulation can be
made of the coordinated joint operation of multiple reservoir systems using
a single release rule. In any event, it is unlikely the system as defined
by the optimization model will perform as indicated without some
modification based on a more realistic simulation.

4. Integrated River System Operation

A number of deterministic models combining optimization and simulation have

been developed to analyze the operation of multiple river basin projects
subjected to stochastic flows. Kuczera (1989) describes one of them that
is typical of many. The river system is represented as a network linear
program, and as such specialized linear programming algorithms designed to
solve network problems can be used to reduce the model running time. This
is useful as the number of components and/or periods being simulated
increases, especially if these models are to be used in an interactive
environment. On the network, each link has a cost (say of pumping) or
penalty (say for a deficit or shortfall in meeting a demand). Optimization
is used to direct flows throughout the network in a way that minimizes the
total cost plus total penalty over a specified planning period.
To illustrate the basic network formulation, consider the single
reservoir and demand situation shown in Figure 4. The reservoir capacity
and the demand for the use must be specified. The reservoir supplies the
user through the conduit, where pumping could be required. ReI eases
downstream are also possible. The inflow to the reservoir is also assumed
known.

WATER USER

Figure 4. A portion of a river system showing a reservoir used for water

supply.
 315

Figure 5 shows the network representation of the system in Figure 4. The

network shown is for two contiguous periods, t and t+1, within the total
number of periods of simulation.

from t+ 1
.....
(O.O.k)
..,...,----...
- to t+2

--

reservoir capacity
inflow in period t
demand in period t
link penal ty per mit
flow
Figure 5. Network representation of Figure 4 for periods t and t+1. (Link
triplets are cost or penalty per unit flow and minimum and maximum flows.)

For each period a shortfall network (links to Nodes C and D from the
balance node) is added to permit demand shortages at the demand site. The
balance node requirement (R.) is the difference between the sum of all
known nodal inflows less the sum of all known nodal outflows or demands.
If the sum of the known model outflows or demands is less than the sum of
known nodal inflows, RB is negative and unavoidable spills into the stream
channels (the links from nodes A and B to the balance node) will occur.
If the sum of known demand outflows is greater than the sum of known
inflows, all demands will not be satisfied, and RB will be positive.
Spills from the system for every period t are sent (e.g. for periods t
and t+1 on the links from Nodes A and B) to the balance node. If no demand
shortages occur, the spills arriving at the balance node will satisfy
exactly the balancing node requirement RB• If demand shortages occur, the
need to maintain a water balance over all simulation periods results in an
excess of water at the balance node exactly equal to the demand shortages.
The shortfall links, such as those from the balance node to the demand zone
Nodes C and D, convey this excess water to the demand sites experiencing
shortages, thereby establishing a mass balance. Their shortfall links are
used only when it is not possible to meet the demands. Hence a very large
penalty, Cr , is assigned to all shortfall links.
Carryover storage between successive periods is modelled using carryover
links, for example the link from Node A to Node B, where A and B represent
the same reservoir site, but in two succeeding periods. The flow in the
link represents the storage volume in the reservoir at the end of period
316

t or the initial storage at the beginning of period t+1. This flow is

limited to the storage capacity of the reservoir. Avoidable spills from
a reservoir are implicitly penalized through the need to use the shortfall
network. These unavoidable spills will induce a demand shortage which must
then be satisfied by flow in the high penalty shortfall arcs.
Nonlinear costs and penalties are incorporated into models like these by
adding multiple links between any two nodes.
This deterministic network linear optimization model is used to simulate
the distribution of a set of known inflows through multiple stream reaches
and reservoirs to multiple demand sites over multiple time periods.
Sensitivity analyses permit the identification of tradeoffs between
capacities and system performance, for example between capacities and
demand satisfaction reliabilities. The system operation, however, is not
necessarily how it would happen in practice, unless of course this type of
model, with continuously updated data, is run periodically in real time to
determine all reservoir releases and demand allocations. It would be
difficult, even with multiple solutions of the model for multiple
streamflow sequences, to infer reservoir operating policies for stochastic
inflows, if indeed such policies were needed instead of using the model
itself for real time operation.
Another approach to implicit stochastic modelling for long-term planning
of multiple reservoir operation is presented by Simonovic (1987). This
approach combines deterministic optimization and simulation in an iterative
three-level algorithm. At the first level, a simulation approach is used
for computing non-economic loss functions associated with water demand
deficits. These loss functions are used in an optimization model to derive
rp.servoir operating rules in the second level. The third level uses
optimization to determine the reservoir yield functions that can be used
for long-term planning.
Each of these approaches just discussed for deriving operating policies
for complex integrated river systems has combined deterministic
optimization and simulation modelling techniques. Through their
appropriate use over multiple-flow sequences, they provide information that
can help define multiple reservoir release and demand allocation rules.
In an attempt to develop a method of realistically simulating and
evaluating such policies subject to stochastic flows and a possibly
changing demand environment, a simulation model called IRIS has been
developed (Pegram et al., 1989). This interactive river system simulation
model is deterministic, uses no optimization, and requires as input data
all reservoir capacit ies and release rules for single or groups of
reservoirs operated together, allocation functions at each site where
diversions are made, and reservoir storage balancing functions for all
reservoirs operated as a group using a single combined reservoir release
rule. IRIS is only able to compare alternative operating policies and
decisions, not define them (similar to HEC models).
There are a few features that distinguish IRIS from most other
deterministic simulation models of· river systems, but perhaps the
significant one to be mentioned here is that it is designed to be run over
real time, say from now to 2020 or some other year in the future. This
simulation into the future can include a scenario of changing demands,
operating policies, and even system capacities. Since the model is
indicating what might be happening in specific simulation periods in the
 317

future, say in January of 1997 or in July of 2003, given an operating

policy scenario, and since we clearly do not know what the future
streamflows will be, say from now until 1997 or 2003, the model permits the
simulation in each time step of multiple streamflow sequences. Hence each
simulation flow, storage or energy variable in each period of simulation
takes on as many values as there are flow sequences incorporated within the
simulation. This permits some estimates of the reliability and
vulnerability of system performance associated with a given stochastic
operating policy and design scenario. Of course the end result after a
full simulation of n time steps is the same as if a single but different
flow sequence were used in n separate simulations. The latter approach,
however, does not allow an examination of the variability of any simulated
variable at any intermediate time step until the end of all simulations.

5. Conclusions

To be useful to water resource system managers and operators, any system

design or operating policy must be defined for, and adaptive to, changing
and uncertain future demands. Various modelling approaches have been
proposed for deriving such operating and design policies using both
explicit and implicit stochastic optimization and simulation.
This paper has been a brief introductory review of some implicit, i.e.
deterministic, modelling approaches developed and used to derive stochastic
design and operating policies. These can be compared with models that
include the explicit consideration of multiple flows and storage volumes,
and their probabilities, discussed elsewhere in this Institute's
proceedings.
One basis for the comparison of all explicit and implicit stochastic
models is how well each of them are able to define and/or evaluate policies
of the type actually used by the planners, managers and operators of the
actual system under analysis. Another basis for comparison is how easy
each of the modelling approaches can be implemented and the resul ts
evaluated, possibly using a realistic static or dynamic simulation model.
A third basis for comparison is how each of the modelling approaches fits
the problem needing analyzing, and the time and money available to carry
out the analysis.
Just as multiple system performance criteria can lead to multiple
policies and designs, multiple modelling selection criteria will likely
lead to multiple modelling approaches. In this case, the analysts just
might have to be the decision makers.

6. References

Bhaskar, N.R. and Whitlatch, E.E. (1980) 'Derivation of monthly reservoir

release policies', Water Resources Research 16, No.6, pp 987-993.
Biswas, A.K. (1976) 'Systems approach to water management', McGraw-Hill
Book Co., New York, pp 164-215.
Kuczera, G. (1989) 'Fast multireservoir multiperiod linear programming
models', Water Resources Research 25, No.2, pp 169-176.
Lall, U. and Miller, C.W. (1988) 'An optimization model for screening
318

multipurpose reservoir systems', Water Resources Research 24, No.7, pp

953-968.
Loucks, D.P., Stedinger, J.R. and Haith, D.A. (1981) 'Water resources
systems planning and analysis', Prentice-Hall, Englewood Cliffs, - New
Jersey, pp 339-355.
Pegram, G.C. et al. (1989) 'IRIS - An interactive river system simulation
program: general introduction and description and user's manual',
Cornell University, Ithaca, NY.
Simonovic, S. (1987) 'The implicit stochastic model for reservoir yield
optimization', Water Resources Research 23, No. 12, pp 2159-2165.
Young, G.K. (1967) 'Finding reservoir operating rules', ASCE Journal of the
Hydraulics Division 93, No. HY5, pp 297-321.