Environmental Modelling & Software 22 (2007) 1349e1361

www.elsevier.com/locate/envsoft

Neural-optimal control algorithm for real-time regulation of in-line

storage in combined sewer systems
Suseno Darsono, John W. Labadie*
Department of Civil and Environmental Engineering, Colorado State University, Fort Collins, CO 80523-1372, USA
Received 29 October 2005; received in revised form 15 September 2006; accepted 16 September 2006
Available online 9 November 2006

Abstract

Attempts at implementing real-time control systems as a cost-effective means of minimizing the pollution impacts of untreated combined
sewer overflows have largely been unsustained due to the complexity of the real-time control problem. Optimal real-time regulation of flows
and in-line storage in combined sewer systems is challenging due to the need for complex optimization models integrated with urban stormwater
runoff prediction and fully dynamic routing of sewer flows within 5e15 min computational time increments. A neural-optimal control algorithm
is presented that fully incorporates the complexities of dynamic, unsteady hydraulic modeling of combined sewer system flows and optimal
coordinated, system-wide regulation of in-line storage. The neural-optimal control module is based on a recurrent Jordan neural network archi-
tecture that is trained using optimal policies produced by a dynamic optimal control module. The neural-optimal control algorithm is demon-
strated in a simulated real-time control experiment for the King County combined sewer system, Seattle, Washington, USA. The algorithm
exhibits an effective adaptive learning capability that results in near-optimal performance of the control system while satisfying the time con-
straints of real-time implementation.
Ó 2006 Elsevier Ltd. All rights reserved.

Keywords: Artificial intelligence; Combined sewers; Hydraulic sewer models; Neural networks; Optimal control; Real-time control; Urban stormwater
management

1. Introduction storage capacity, have motivated the search for cost-effective

A combined sewer system (CSS) is a type of urban drainage
network constructed for collecting and transporting both urban
wastewater and stormwater flows; i.e., combined wet and dry
weather flows. These systems create serious pollution problems
in adjacent water bodies when high storm flow loadings result
in discharges exceeding interceptor sewer and treatment plant
capacities. Combined sewers are no longer constructed, but
are still found in many large cities worldwide. The high cost
and disruption to residences and businesses of converting to
separated systems, or adding sufficient off-line detention
* Corresponding author. Tel.: þ1 970491 6898; fax: þ1 970491 7727.
E-mail addresses: sdarsono@tstinc.com (S. Darsono), labadie@engr.
colostate.edu (J.W. Labadie).
methods for controlling combined sewer overflows (CSOs).
In addition, stormwater alone often carries high pollution
loads, and in the U.S., federal and state standards are requiring
assurance that storm runoff does not pollute adjacent rivers and
streams (U.S. EPA, 1999). In some cases, sewer separation can
actually increase pollution to adjacent receiving waters since
stormwater combined with sanitary flows may have previously
been treated, whereas stormwater is often directly discharged
into receiving waters in separated systems (AMSA, 1994).
Real-time regulation of in-line storage in combined sewer
systems through control of gates, pumps, and weirs is an
approach to reducing untreated overflows that has been suc-
cessfully demonstrated in several cities including Milwaukee,
USA (Loucks et al., 2004), Quebec, Canada (Pleau et al.,
2005), Tielt, Belgium (Vanrolleghem et al., 2005), Saverne,

1364-8152/$ - see front matter Ó 2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.envsoft.2006.09.005
1350 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
France (Vazquez et al., 2003), and Ense-Bremen, Germany
(Weyand, 2002). Unfortunately, these examples of successful
ongoing implementations are rare due to past difficulties in
the robustness and reliability of required computer control
equipment, sensor and communication devices, as well as
inadequate software and modeling capabilities. Schutze et al.
(2002) claim, however, that current advances in hardware and
software technologies have overcome many of these past
obstacles, and at reasonable cost, thereby providing a renewed
impetus for implementation of real-time control of CSSs.
The maximum utilization of spatially distributed in-line
storage in a combined sewer system is an inexpensive method
(relative to capital construction) of reducing the polluting
effects of untreated spills to receiving waters (Labadie,
1993). The goal is to provide optimal regulation of control
structures in the sewer network such that CSOs are minimized
or even eliminated. The hypothesis is that aggregate in-line
storage capacity in a combined sewer network is often sizable,
and if properly managed over time and space, can help reduce
pollution from untreated CSOs to adjacent water bodies. Even
if elimination of CSOs is not possible under ideal control strat-
egies, incorporation of real-time control into plans for
constructing additional detention storage and other facilities
may reduce sizing requirements of these facilities, and there-
fore costs, by making optimum use of available in-line and
detention storage.
Real-time control (RTC) is most effective if integrated over
the entire sewer network, resulting in a large-scale, spatially
distributed optimal control problem. According to Pleau
et al. (2005), most current RTC implementations are limited
to local reactive or supervisory control. Aside from the
large-scale nature of integrated, system-wide or global optimal
control, the optimization is highly nonlinear, requiring integra-
tion with models for accurately simulating stormwater runoff
and sewer hydraulics, including fully dynamic unsteady flow
modeling in the sewer network. The optimization is also
dynamic, with further computational stress arising from the
need to repeatedly solve the optimal control problem within
5e15 min time intervals as rainfall forecasts and measured
levels and flows are updated in real-time.
With computational time and complexity the primary limi-
tation in employing the accurate models necessary for effec-
tive real-time control, application of dynamic or recurrent
artificial neural networks (ANN) may provide the analysis
speed, generalization ability and high fault tolerance needed
for effective implementation. Presented herein is an explora-
tion of the potential usefulness of dynamic artificial neural
networks for real-time control of combined sewer systems.
A highly accurate, but computationally time consuming, opti-
mal control model is utilized to provide the training data set
for a recurrent ANN under a wide range of sewer inflow con-
ditions. Performance of the ANN is compared with the optimal
control module using validation data sets not included in the
ANN training. The combined sewer system of the King
County Wastewater Treatment Division, Washington, USA is
chosen as a case study to demonstrate the viability of the neu-
ral-optimal control model.
2. Optimal control module
2.1. Formulation
According to Vanrolleghem et al. (2005), there are three
basic approaches to real-time control of CSSs: volume-based
RTC, pollution-based RTC and emission-based RTC.
Although Rauch and Harremoës (1999) show that minimiza-
tion of pollution impacts is not necessarily achieved with vol-
ume-based RTC, Duchesne et al. (2004) explain that the
computational complexity of receiving water quality models,
and the dearth of available data necessary for calibration of
such models in urban areas, makes volume-based RTC the
most practical approach. Therefore, reducing the occurrence
and magnitude of CSOs, and thereby reducing pollution im-
pacts on receiving waters, is the primary goal of the real-
time control system. The primary objectives are to minimize
untreated overflows while maximizing through-flows to the
wastewater treatment plant for a storm event occurring over
T time intervals:
X
T 2X
mþn X
m
minimize cit u2it þ wi s2i;Tþ1 ð1Þ
u;x
t¼1 i¼mþnþ1 i¼1
subject to:
stþ1 ¼ st þ ð0:5ðBðkÞ
ðkÞ
t ut þ Bt1 ut1 Þ þ rt Þconv ð2Þ
 
0  stþ1  smax qðkÞ ; hðkÞ ð3Þ
 
0  ut  umax qðkÞ ; hðkÞ ð4Þ
where ut ¼ ðu1t ; .; umt ; umþ1;t ; .; umþn;t ; umþnþ1;t ; .; u2mþn ÞT
is the vector of node discharge rates (m3/s) at the end of time
interval t throughout the sewer network (initial release rates
u0 are assumed to be given), with m representing the number
of control structures such as regulators, pump stations, weirs,
orifices, etc., in the system. It is assumed that the portion of the
state vector ðs1t ; .; smt Þ represents temporary storage accumu-
lated behind these structures located in the interceptor and lat-
eral or trunk sewers (m3). It is assumed that check structures
are not directly located within the interceptor sewer, but rather
regulators control discharges from each lateral into the inter-
ceptor, as well as bypassed overflows to the receiving waters
if the interceptor sewer is surcharged at that location.
The interceptor sewer is divided into n sections correspond-
ing to the location of each regulator controlling discharges
from a lateral sewer. The portion of the state vector
ðsmþ1;t ; .; smþn;t Þ represents zero storage nodes within the in-
terceptor sewer that are specified for the convenience of flow
routing calculations. The portion of the control vector
ðu1t ; .; umt Þ represents controlled discharges from each regu-
lator or pump station into the interceptor sewer or downstream
trunk sewer; the portion ðumþ1;t ; .; umþn;t Þ are flows at each
section of the interceptor sewer; and the portion
ðumþnþ1;t ; .; u2mþn;t Þ represents untreated overflows from the
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
system at regulator stations and other control structures. A typical
configuration for a regulator station is shown in Fig. 1.
ðkÞ
The routing matrices Bt in Eq. (2) are current estimates at
iteration k of routing coefficients calculated from a fully dy-
namic sewer hydraulics model, as described subsequently in
more detail. The vectors qðkÞ ; hðkÞ represent sewer discharges
and heads, respectively, over each sewer section i and discrete
time interval t as also simulated by the hydraulic  model. Max-
imum capacities on temporary storage smax qðkÞ ; hðkÞ Þ and
node discharge umax qðkÞ ; hðkÞ Þ are shown to be dynamic func-
tions of qðkÞ ; hðkÞ since these bounds are dependent on hydrau-
lic flows and heads throughout the sewer network. These
iteratively adjusted restrictions on temporary in-line storage
and discharges prevent upstream surcharge conditions from
occurring in the lateral or trunk sewers. Duchesne et al.
(2001) claim that some surcharging should be allowed for
maximum utilization of in-line storage in a combined sewer
network, but this also can increase the danger of upstream
flooding if inaccurate predictions of sewer inflows occur.
The objective function of Eq. (1) minimizes total weighted
overflows (squared) from the CSS, where the weighting coef-
ficients cit can vary both spatially and temporally. That is,
receiving water impacts may be more sensitive to overflows
at certain locations than others. Tidal and other temporal influ-
ences may also necessitate changes in the weighting coeffi-
cients over time. Notice that spills are squared in the
objective function, which has a number of benefits. First, min-
imizing the squared spills has a smoothing effect on the solu-
tion, and tends to produce more stable discharges, avoiding
oscillations and surges in the sewer system. In addition, adja-
cent water bodies can better absorb the polluting effects of
spills if they are smoothed over time, rather than a first-flush
shock of untreated spills that can intensify damage to aquatic
habitat and species. P an iterative approach involving successive solution of the op-
The final term in the objective function m 2
i¼1 wi si;Tþ1 min-
imizes residual storage in the system at the end of the opera-
tional time horizon, which indirectly maximizes through-flow
to the wastewater treatment plant. Weighting factors wi serve
to tradeoff this objective with the primary objective of mini-
mizing untreated spills. If forecasts indicate that a storm event
Interceptor
Regulator uit
Gate
Overflow
um+n+i,t Trunk Sewer
Outfall
Gate Regulator
Station i
Fig. 1. Typical regulator station with both regulator gate and outfall gate.
1351
may be immediately followed by another, it may be necessary
to increase this weighting factor to provide sufficient capacity
for the on-coming event.
The constraints in Eq. (2) essentially maintain mass balance
in each sewer reach, where rt is spatially distributed storm-
water inflows to the CSS predicted from an urban stormwater
runoff model, such as the RUNOFF module of the U.S. EPA
SWMM model (Huber and Dickinson, 1988). These predic-
tions are assumed to be updated in real-time as the storm event
progresses and new rainfall forecasts are generated. Since the
discharges ut are instantaneous flow rates, they are averaged
over the time interval and multiplied by a conversion factor
conv to convert them to storage units per time interval.
2.2. Iterative calculation of hydraulic sewer flows
ðkÞ
The ðm þ nÞ  ð2m þ nÞ routing matrices Bt in Eq. (2)
account for attenuation and lagging of upstream releases to
downstream nodes and system spills. The superscript (k) is
ðkÞ
an iteration index indicating that elements of Bt are updated
through successive solution of a fully dynamic unsteady flow
model that solves the complete Saint Venant equations
throughout the sewer network. Although it is possible to di-
rectly incorporate numerical approximations of the linearized
Saint Venant equations as constraints in the above formulation,
as proposed by Unver and Mays (1990), this results in an
unwieldy, large-scale quadratic programming problem when
applied to an entire sewer network over a dynamic time hori-
zon. Linearizing the Saint Venant equations also removes
important nonlinear terms and prevents accurate simulation
of complex hydraulic conditions in the sewers, such as transi-
tions between supercritical and subcritical flow regimes.
A method originally proposed by Labadie (1993) suggests
timal control model and the hydraulic sewer routing model
(Fig. 2). The optimal control model OPTCON is first executed
with current approximations of the routing coefficients (i.e.,
ðkÞ
elements of matrix Bt ), as well as initial estimates of bounds
on temporary storage and discharge (i.e., Eqs. (3) and (4)). Op-
timal node discharge control solutions ut are then converted
into gate or pump settings, and the sewer network is again sim-
ulated using the complete Saint Venant equations. Based on
these results, the routing coefficients used in the optimal con-
ðkþ1Þ ðkþ1Þ
trol model are updated as: fbij gðkþ1Þ
t ¼ ðujt Þ=ðuit Þ, as
ðkþ1Þ
shown in Fig. 3, where uit represent flows simulated using
the dynamic hydraulic routing model under the current opti-
mal gate or pump settings. Routing matrix elements fbij gðkÞ
t
for any iteration k are positive if node i receives outflow
from node j, or 0 if the two nodes are unconnected. For spills
and downstream releases from node i, fbij gðkÞ t ¼ 1.
Along with routing coefficients, limits on sewer reach storage
smax ðqðkþ1Þ ; hðkþ1Þ Þ and discharge limits umax ðqðkþ1Þ ; hðkþ1Þ Þ
are updated from the hydraulic sewer simulation model in
the next iteration to assure hydraulic feasibility of solutions
from the optimal control model.
Although the control problem defined by Eqs. (1)e(4) re-
flects a simplified problem of volume and discharge control in
1352 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
Fig. 2. Flow chart of successive, iterative solution of optimal control model
OPTCON and hydraulic sewer model UNSTDY.
a combined sewer network, interactive simulation using the
complete Saint Venant equations allows bounds on flows
through gates and regulators in the control model and maximum
sewer levels to be adjusted based on levels calculated by the
hydraulic simulation model that guarantee sufficient head for
downstream releases. Convergence occurs when successive cal-
culations of routing coefficients coincide to a desired error tol-
erance. In effect, inclusion of the routing coefficients in Eq. (2)
is a means of indirectly incorporating the complete Saint Venant
equations into the optimal control algorithm.
2.3. Solution of Saint Venant equations
The fully dynamic unsteady flow module UNSTDY is
applied to one-dimensional hydraulic routing in a combined
sewer network, including flows through junctions and control
structures (Chen and Chai, 1991). The basic Saint Venant
equations of conservation of mass and momentum are solved
in UNSTDY using a fully implicit numerical scheme:
vA vq
þ ¼ q[ ð5Þ
vt vx
vðrqÞ vðrqVÞ vh  
þ þ rgA þ rgA S0  Sf ¼ 0 ð6Þ
vt vx vx
where for each sewer section, A is flow cross-sectional area; q
is discharge; h is depth of flow; V is mean velocity; q[ is lateral
flow per unit length of the sewer section; x is longitudinal dis-
tance; t is time; S0 is bed slope; Sf friction slope; r is fluid
mass density; and g is gravitational acceleration.
Application of the fully implicit finite difference numerical
scheme used in UNSTDY has the advantage of producing sta-
ble solutions that do not require time step restrictions from the
Courant conditions. The model includes routines for calculat-
ing transitions between subcritical and supercritical flow condi-
tions, with supercritical flows solved using the kinematic wave
equations. The Preissmann slot technique allows evaluation of
unsteady flow in a storm sewer network under submerged con-
ditions, where a hypothetical slot at the top of the pipe can at-
tain higher pressurized flow wave celerity, thereby allowing the
same set of unsteady flow equations to be applied to surcharge
flow conditions. The width of the slot in UNSTDY is assumed
to be 0.1% of the maximum width of a conduit under surcharg-
ing conditions. In UNSTDY, the upstream boundary conditions
are based on stormwater inflow predictions from the RUNOFF
module of the U.S. EPA SWMM model (Huber and Dickinson,
1988). Downstream boundary conditions can be defined as dis-
charge hydrographs, stage hydrographs, stage-discharge rating
curves, or storage basins.
The hydraulic routing module can handle a branched sewer
network using a dendritic system of junctions with up to three
inflow branches and one outflow pipe. The hydraulic equations
for sewer flow through a confluence junction are the continuity
and energy equations (Chen and Chai, 1991). Since real-time
operation of urban drainage networks requires in-line or off-
line storage for detaining storm flood hydrographs from up-
stream areas, control devices are required to regulate storage
of stormwater flow. UNSTDY allows inclusion of weirs, sluice
gates, radial gates, siphons, in-line and side orifices, wet-well
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
Fig. 3. Calculation of routing coefficients used in optimal control model from flows simulated by numerical hydraulic model.
and dynamic head pump stations, and storage facilities in the
sewer network routing.
2.4. Optimization algorithm
Several optimization algorithms have been proposed for
real-time, online solution of the problem embodied in Eqs.
(1)e(4), or similar formulations. These include the general-
ized reduced gradient algorithm (GRG) (Duchesne et al.,
2004), projected Lagrangian method (MINOS) (Pleau et al.,
2005), regulator theory (LDQ control) (Marinaki et al.,
1999), and genetic algorithms (Butler and Schütze, 2005).
Since all of these algorithms are assumed to be implemented
in an online, real-time environment, the sewer hydraulics mod-
eling is unduly simplified to accommodate the use of these
algorithms. The optimization problem of Eqs. (1)e(4), includ-
ing iterative evaluation of dynamic, unsteady sewer hydraulics
using UNSTDY, is efficiently solved using discrete time opti-
mal control theory based on Pontryagin’s maximum principle
(Pontryagin et al., 1962), with use of penalty functions for
indirectly accommodating the state-space constraints (Eq.
(3)). Details on the OPTCON algorithm can be found in
Albuquerque and Labadie (1997), where the same algorithm
was also applied to optimal regulation of an irrigation canal
under fully dynamic, unsteady flow conditions.
3. Neural network module
3.1. Jordan recurrent ANN architecture
The OPTCON optimal control module for minimizing
CSOs is easily solved off-line for sewer networks, but integra-
tion with the RUNOFF model for watershed inflow predictions
and successive execution of the sewer hydraulics model
UNSTDY may exceed the clock time limitations required
for real-time implementation, particularly for large-scale com-
bined sewer networks. Neural networks are important methods
of artificial intelligence related to computational intelligence
1353
and learning. Several studies have shown that artificial neural
networks (ANN) are an effective tool for controlling complex,
nonlinear systems (Parisini and Zoppoli, 1994). Adaptive
control of combined sewer systems in real-time requires a re-
current ANN to model dynamic operational trajectories.
The Jordan architecture (Fig. 4) selected for the ANN struc-
ture adapts well with time-varying systems, since the main
purpose of the dynamic neural control module is to compute
optimal real-time gate controls based on current and previous
rainfall data and previous gate control decisions:
X
I X
K
vhj ðtÞ ¼ whj;iþK yi ðtÞ þ whjk xk ðt  1Þ þ qhj ð7Þ
i¼1 k¼1
 
zj ðtÞ ¼ hj vhj ðtÞ ; j ¼ 1; .; J ð8Þ
X
J
vok ðtÞ ¼ wokj zj ðtÞ þ qok ð9Þ
j¼1
 
xk ðtÞ ¼ ok vok ðtÞ ; k ¼ 1; .; K ð10Þ
where I is the total number of external inputs applied to neu-
rons in the input layer; K is the total number of internal inputs
originating from neurons of the output layer and applied to K
neurons in the input layer in the next time step; J is the total
number of neurons in the hidden layer; whji is the synaptic
weight of connection of neuron i in the input layer to neuron
j in the hidden layer; wokj is the synaptic weight of connection
of neuron j in the hidden layer to neuron k in the output layer;
yi ðtÞ is the external input signal entering neuron i in the input
layer at time t; xk ðtÞ is the output from neuron k in the output
layer, serving as internal input to the input layer in the next
time step; qhj is the threshold or bias unit applied to neuron j
in the hidden layer; qok is the threshold or bias unit applied
to neuron k in the output layer; vhj ðtÞ is the net internal activity
level of neuron j in the hidden layer; vok ðtÞ is the net internal
activity level of neuron k in the output layer; zj ðtÞ is the output
1354 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
from neuron j in the hidden layer produced from operation of
activation function hj ðvhj ðtÞÞ on net internal activity vhj ðtÞ for
neuron j; and ok ðvok ðtÞÞ is the activation function operating
on net internal activity vok ðtÞ for neuron k in the output layer.
The logistic function is the commonly used activation func-
tion for multilayer perceptrons that guarantees output in the
range 0  yj ðtÞ  1 (Haykin, 1994):
 
1
zj ðtÞ ¼ hj vhj ðtÞ ¼ h ð11Þ
1 þ evj ðtÞ
  1
xk ðtÞ ¼ ok vok ðtÞ ¼ o ð12Þ
1 þ evk ðtÞ
Internal inputs to the recurrent ANN are current, lag-1, and
lag-2 spatially distributed rainfall data, recognizing the likely
correlation of successive rainfall inputs over time and space.
To facilitate smooth, dynamic operation of gates in the system,
gate-opening controls from the previous time period serve as
internal inputs to the ANN. Since current period gate openings
are the output of the ANN, the number of output neurons is the
same as the number of gate controls in the system.
Fig. 4. Schematic of Jordan architecture recurrent artificial neural network.
3.2. Training and testing of recurrent ANN
Training of the dynamic neural control module is a super-
vised learning process for determining the optimal connection
weights whji , wokj and bias weights qhj , qok from the input/output
training data set. The input data are rain gauge measurements
for various historical storm events and the output data set are
the optimal gate controls calculated off-line by the optimal
control module. Training of the Jordan recurrent network
follows the standard generalized delta rule (Freeman, 1994) in-
volving feed-forward and error back-propagation calculations.
The process begins with standard normal transformation of the
inputeoutput training set data, since the activation function in
the neural network can only produce outputs in the range be-
tween 0 and 1. Re-transformation of neuron output values
from neurons in the output layer results in actual gate control
settings. Initial weights are determined using random number
generation, and a gradient-type optimization procedure adjusts
the connection weights until the sum-of-squares error deviation
between ANN outputs and training set outputs is minimized
(Haykin, 1994).
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
Optimal values of the learning parameter and momentum
constant are based on experimental procedures to minimize
error and maximize convergence efficiency of the training
algorithm. However, training of recurrent neural networks
may suffer from slow convergence to local minima and must
be conducted with great care (Hassoum, 1995). The learning
process is embedded in an experimental procedure to evaluate
the minimum square error (MSE) for determining the opti-
mum number of neurons in the hidden layer. Testing the
dynamic neural control module is a process of evaluating
model parameters from the learning or training step and is
equivalent to a validation of the model. Since testing is an
evaluation of the performance of the previously estimated
model parameters, the input and output data sets for estimating
parameters (for training) should be different from the input
and output data set for validating the parameters (for testing).
4. Case study: West Point combined sewer system,
Seattle, Washington, USA
4.1. Description of case study
The West Point Treatment Plant collection system of the
King County Wastewater Treatment Division, Seattle, Wash-
ington, USA is used for demonstrating the capabilities of the
neural-optimal control algorithm. The portion of the service
area of the West Point Treatment Plant that is included in
this modeling study covers over 26,000 ha, which includes
160 km of gravity sewers with diameters up to 3.66 m, 11
pumping stations and 17 regulator stations (Fig. 5). Although
the average capacity of the West Point Treatment Plant is
5.83 m3/s, the plant has been recently expanded to handle
wet weather peak flow rates up to 19.3 m3/s. Fig. 5 shows
the extent of the West Point combined sewer system as mod-
eled in this study, but it should be noted that this configuration
does not reflect recent expansions, upgrades and improve-
ments to the King County wastewater system designed to sub-
stantially reduce the volume and frequency of combined sewer
overflows (King County, 2004). Although not up-to-date, this
configuration was deemed acceptable for demonstrating the
viability of the neural-optimal control algorithm.
Metro Seattle, which merged with King County as a single
agency in 1994, originally developed the CATAD Computer
Augmented Treatment and Disposal system in the early
1970s, one of the first attempts to implement a supervisory
control and data acquisition (SCADA) system for real-time
regulation of in-line storage in a combined sewer system.
With the main control center located at the West Point Treat-
ment Plant, CATAD is designed to monitor and control pump
and regulator stations, including telemetry of real-time data on
water levels, gate positions, tide levels, and pump speed data
(King County, 2004). Recent attempts have been made to
upgrade CATAD to monitor rainfall and flow conditions in
the major trunk sewers and interceptors for model prediction
of sewer inflows and optimal control of selected regulator sta-
tion gates in real-time. Unfortunately, problems at the Interbay
1355
Pump Station and computer hardware have prevented use of
the predictive control components in CATAD.
4.2. RUNOFF model calibration and validation
Labadie (1993) describes calibration of the RUNOFF
model for predicting stormwater inflows from the Seattle
area urban catchments into the combined sewer system. Model
calibration was conducted by staff of the King County Depart-
ment of Natural Resources, Wastewater Treatment Division
(formerly Metro Seattle). The West Point service area was por-
tioned into 400 drainage subbasins. Subbasin characteristics
including average slope, overland flow lengths, roughness
coefficients, percent imperviousness, and infiltration parame-
ters were evaluated in the calibration, with further validation
studies comparing simulated inlet hydrographs with measured
flows for storm events not included in the calibration. Ten-
minute rainfall data for the stormwater modeling were
collected at 17 sites within the Seattle City limits, along
with measurements at the National Weather Service station
at SeaTac Airport and Sand Point in North Seattle. Flow
data for model calibration were measured at more than 60
points throughout the West Point service area using transport-
able flow meters.
4.3. UNSTDY hydraulic model development
Stormwater hydrographs generated from the RUNOFF
model are routed into 22 inlet locations serving as upstream
boundary conditions and lateral inflows to the combined sewer
network (Fig. 5). The sewer network modeled in UNSTDY is
confined to only those portions with 1.22 m diameter pipe
sizes and higher, with smaller pipe sections modeled using
kinematic wave approximations. For the numerical modeling,
the sewer network was divided into 260 sections, with distance
between sections ranging from 18.3 m to 91.4 m based on
desired numerical accuracy, changes in slope or existence of
a weir or other control structure. Pipe and gate sizes, slopes,
roughness coefficients and junction data were provided to
the hydraulic model, along with rating tables for downstream
boundary conditions based on the normal flow approximation.
The UNSTDY hydraulic module routes inlet stormwater
hydrographs through the sewer network, with dry weather
flows provided as initial flow conditions. To facilitate linkage
of the hydraulic model with the optimal control module, UN-
STDY allows specification of optimal regulator station releases
calculated from OPTCON as interior boundary conditions, and
then calculating the gate openings required to produce those
flows.
4.4. Optimal control module
For this demonstration, Fig. 5 depicts the regulator sta-
tions assumed available for simulated real-time control,
with the remaining uncontrollable regulator stations modeled
in UNSTDY as fixed weirs. Although the optimal control
module is confined to optimization of the regulator stations
1356 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361

Fig. 5. Combined sewer collection system of the West Point Treatment Plant, King County Wastewater Treatment Division, Washington USA.

along the Elliot Bay Interceptor, the UNSTDY hydraulic
model calculates flows from the Duwamish pump station
into the Elliot Bay Interceptor, as well as flows from the
North Interceptor into the West Point Treatment Plant. These
hydraulic calculations allow specification of the treatment
capacity available for flows originating from the Elliot Bay
Interceptor.
Fig. 6 is a schematic of the OPTCON model formulation
for the Elliot Bay Interceptor. Notice that storage variables
in the interceptor sewer are 0-storage dummy variables neces-
sary for the state-space formulation of the optimal control
problem as defined in Eq. (2), even though it is assumed
that there are no gates or other control structures within the
interceptor. Flows in the interceptor are indirectly controlled
by gates in the regulator stations. Weighting factors cit in the
objective function (Eq. (1)) were uniformly set to 125, with
the penalty on final storage wi set to 12.5, although these
weights are easily changed for prioritization of overflows as
to location and time of day (e.g., considering tidal influences
on polluting impacts of combined sewer overflows).
Rain gauge data in the Seattle area were used to define 11
diverse, spatially distributed storm events over the study area.
Rainfall data in 10 min time increments were obtained from
National Weather Service stations in the area: SeaTac Airport
south of the study area and Sand Point located north of the
study area. Although other rain gauges are maintained by
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361 1357

Fig. 6. Schematic of the Elliot Bay portion of the West Point collection system modeled in OPTCON.

the City of Seattle, the NWS gauges provide the most com-
plete data set and at the desired time increment of 10 min.
The first ten events were utilized as training data sets for
the neural-optimal control model, with Storm #11 used for
testing and validation. Fig. 7 shows the inflow hydrographs
for Storm #11 for the study area corresponding to the con-
trollable regulators along the Elliot Bay Interceptor, as shown
in Fig. 6. Inflows r5 entering the Elliot Bay Interceptor from
the Duwamish Pump Station were estimated from application
of the RUNOFF model to Southern Service area and UN-
STDY for routing flows to the Duwamish Pump Station.
Although future work can consider a larger number and va-
riety of storm events, these were considered to be a reason-
ably diverse representation of storm events suitable for this
demonstration.
Each of the 11 storm events produced from the rainfall da-
tabase provides input to OPTCON to compute optimal gate
controls for each event in 10-min time increments. For the
routing links shown in Fig. 6, routing coefficients in routing
ð0Þ
matrix Bt are set to 1.0 as initial approximations in the iter-
ative process depicted in Fig. 2. Convergence of the routing
coefficients typically requires at most eight iterations, as
shown in Fig. 8 for the trunk sewer between Hanford #1 and
Hanford #2 as an example.
Fig. 9 demonstrates the results of application of OPTCON
in reducing combined sewer overflows under fully integrated
dynamic optimal control. These results are contrasted with
the uncontrolled solution where diversions from the trunk
sewers are allowed to enter the interceptor sewer until sur-
charging occurs at that location, resulting in spills or overflows
1358 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361

3.5 8
7
Inflow Hydrographs (m3/s)

3
r1

Overflows (m3/s)
6
r2
2.5 5
r3
r4 4
2
r5
r6 3
1.5
r7 2
r8
1 1
r9
r10 0
0.5 0 60 120 180 240 300 360 420

0 Time (min.)
0 50 100 150 200 250 300 350 400
OPTCON Uncontrolled
Time (min.)
Fig. 9. Comparison of total overflows from OPTCON solution versus uncon-
Fig. 7. Inflow hydrographs to the study area for Storm #11.
trolled operation for Storm #11.

from the previous time step serve as internal inputs. The

at that regulator station. Storm #11 is a large event, which re-
gradient-type back-propagation procedure is applied to opti-
sults in overflows in spite of the application of the optimal
mal estimation of the synaptic weights whji , wokj and bias units
control strategy, but with total overflows and peak discharge
qhj , qok . The gradient algorithm requires specification of learn-
rates substantially reduced under optimal control. These re-
ing parameter h and momentum constant a, which govern
sults clearly demonstrate the viability of optimal use of in-
the convergence rate and stability of the learning process.
line storage in the combined sewer system. However, since
For this application, numerous experiments with these param-
these optimal controls are based on perfect foreknowledge of
eters resulted in specification of the optimal parameters
the storm event, they cannot be implemented for actual real-
h ¼ 0.10 and a ¼ 0.30 as provided the minimum root mean
time control. Therefore, OPTCON solutions for Storms #1
square error (RMSE) in the learning process, and requiring
through #10 are utilized as training data sets for the recurrent
100 epochs or iterations.
neural network. The trained ANN can then be validated using
Masters
p ffiffiffiffiffiffi (1996) recommends the number of hidden neurons
results from Storm #11, which allows demonstration of opti-
¼ mn for networks with a single hidden layer, where n is the
mal real-time control without the presumption of perfect fore-
number of input neurons and m is the number of output neu-
knowledge of the storm event.
rons. This technique is only used to obtain an approximation
of the ideal hidden layer size. An experimental approach
4.5. Training the recurrent artificial neural network was utilized to determine the optimal number of neurons in
the hidden layer of the recurrent neural network, with results
The hydraulic routing and optimal control modules evaluate indicating that 60 neurons gave the minimum value of
the optimal gate controls for the training data set comprising RMSE (Fig. 10).
Storms 1 through 10, which are then used as desired outputs
in the learning step of the recurrent ANN. Ten-minute rainfall 4.6. Testing the neural-optimal control model
hyetographs are external inputs to the module and outputs
Testing of the neural-optimal control model is a process of
1.1 validating the ANN parameters from the learning or training
step using inputs not included in the training data set. The
Routing Coefficients

1.05
6 No. of
Hidden
5 Nodes
1
10
RMSE (x 10-4)

4 20
30
0.95 3
40

2 50
60
0.9
1
0

100

150

200

250

300

350

400

450

500

550

600

650
50

Time (min.) 0
20 30 40 50 60
Iter 0 Iter 2 Iter 4 Iter 6 Iter 8 Epochs

Fig. 8. Convergence of routing coefficients for trunk sewer reach between Fig. 10. Experimental results for determining optimal number of hidden nodes
Hanford #1 and Hanford #2. in recurrent ANN.
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361 1359

input and output data sets for estimating parameters (i.e., train-
ing) should differ from the input and output data sets for
validating the ANN (i.e., testing). Storm #11 serves as the val-
idation data set, since it was not included in the training data
sets for the recurrent ANN. Fig. 11 provides a comparison of
the gate controls produced by the recurrent ANN with those
produced under perfect foreknowledge by OPTCON for the
regulator and outfall gates at the Connecticut Street regulator
station. It should be emphasized that the neural controls are
generated with only current rainfall, past rainfall, and previous
gate controls as inputs. The ANN is provided no forecast
information or foreknowledge of the storm event, and yet
the gate controls are quite close to those produced by OPT-
CON under perfect foreknowledge. Similar results are found
in comparing the neural and OPTCON gate controls for the
other regulator stations in the study area. This indicates that
the neural network displays a learning capability that adapts
the control strategy to the ongoing storm event as rainfall
data are being collected. Although the rainfall event used
for testing differs from all the events used for training of
the ANN, portions of this event are similar to portions of
the events used in the training. The ANN exhibits a pattern-
recognition capability that detects these similarities and ex-
ploits them.
(a)
100
Gate Control (% Open)
The ability of the neural-optimal control model to adapt to
the ongoing storm event is evident in Fig. 12, which provides
a comparison between total direct overflows from the com-
bined sewer system as hydraulically simulated using gate con-
trols from the recurrent ANN versus the controls produced by
OPTCON. It is also noted that the gate controls produced by
the ANN exhibit a smoothness and stability that belies the
high variability of the rainfall inputs. It was originally hypoth-
esized that providing direct rainfall measurements as inputs to
the ANN would not be successful since rainfall data can be
noisy and sporadic, and that it would be necessary to process
the rainfall data through the RUNOFF model and provide the
resulting sewer inflow predictions as the input data sets for the
ANN. These results indicate, however, that rainfall data can
indeed be provided as direct inputs to the ANN, which is
facilitated by the dynamic nature of the recurrent ANN and
the use of time-lagged inputs. For this application, execution
of the recurrent ANN at each time step required only 0.02 s
of CPU time on a 2 GHz Pentium 4 workstation, indicating
that the neural-optimal control model can be easily imple-
mented for adaptive, real-time control of combined sewer
systems.
5. Summary and conclusions
Real-time control of available in-line storage in combined
sewer systems has been demonstrated to be a viable, cost-

80 effective approach to reducing untreated discharges to receiv-

ing waters during wet weather conditions and ameliorating the
60 adverse environmental impacts of sanitary sewage mixed with
stormwater. Furthermore, capital investment in additional stor-
40 age facilities, transport and treatment capacity can likely be
reduced if system planning and design includes real-time reg-
20 ulation of controllable gates, pumps and weirs in the system.
Although several cities in the U.S., Canada, and Europe
0
0 1 2 3 4 5 6 7 have implemented some degree of automation in the regula-
Time (hrs.) tion of storage and conveyance capacity in their combined
NEURAL OPTCON
sewer systems, these have primarily been limited to local
control strategies and use of simplified hydraulic models.
(b) Maximization of the full potential of existing in-line storage,
100 regulator sewer transport, and treatment capacity requires fully
integrated control strategies that effectively model the spatial
Gate Control (% Open)

80

2.5
60
Overflows (m3/s)

2
40
1.5

20 1

0.5
0
0 1 2 3 4 5 6 7 0
Time (hrs.) 0 1 2 3 4 5 6 7

NEURAL OPTCON
Time (hrs.)
NEURAL OPTCON
Fig. 11. Comparison of gate operations under neural control and OPTCON at
Connecticut Street regulator station: (a) regulator gate; (b) outfall gate. Fig. 12. Comparison of total overflows under neural control and OPTCON.
1360 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
and temporal heterogeneities of stormwater inputs to complex
combined sewer systems. Unfortunately, fully integrated sys-
tem-wide control imbedded with realistic, fully dynamic sewer
hydraulic models presents a significant computational chal-
lenge, particularly with the clock time constraints of real-
time implementation.
Presented herein is a neural-optimal control model that
integrates the dynamic hydraulic model UNSTDY with the
optimization model OPTCON based on optimal control the-
ory. Incorporating hydraulic realism in the optimization model
requires a computationally expensive iterative process, which
may violate the clock time constraints of real-time implemen-
tation. In addition, the optimal control strategies would be
difficult to implement since they are based on perfect fore-
knowledge of the ongoing storm event. For these reasons,
a strategy is adopted whereby OPTCON/UNSTDY computa-
tions are conducted off-line for a wide range of spatially dis-
tributed, historical storm events. The resulting optimal
control outputs, along with the rainfall data sets as inputs, pro-
vide training data sets for a recurrent artificial neural network
(ANN). The Jordan architecture adopted for the ANN effec-
tively captures the dynamic, rapid response characteristics of
combined sewer systems. Inputs to the ANN include current
and past spatially distributed rainfall data, as well as previous
controls, with synaptic weights and other parameters in the
ANN optimized to reproduce the optimal gate controls from
OPTCON/UNSTDY as closely as possible. Re-training can
be conducted off-line as new storm events are included in
the database, or if the need arises for altering the weighting
factors and parameters employed in OPTCON. Once trained,
the recurrent ANN can be implemented for real-time control
of combined sewer systems with full consideration of complex
system hydraulics and fully integrated, spatially distributed
system-wide control.
The neural-optimal control algorithm was demonstrated
using the West Point Treatment Plant collection system of the
King County Wastewater Treatment Division, Seattle, Washing-
ton, USA. Although the configuration of the system used in this
case study is not current, and is based on previous model calibra-
tion studies, it is believed that it still provides a convincing dem-
onstration of the viability of the neural-optimal control strategy.
Validation results indicate that the neural-optimal control algo-
rithm closely tracks the optimal gate controls from OPTCON/
UNSTDY for a storm event not included in the training data
sets for the recurrent ANN, while requiring only inputs of cur-
rent and past rainfall measurements and previous gate controls.
The ability of the ANN to adapt the optimal controls to changes
in rainfall intensity and distribution in the ongoing event is
clearly evident in this demonstration.
Further work is focusing on enriching the training data sets
for the ANN to include flow and/or sewer level calculations at
selected measurement locations in the combined sewer sys-
tem. Assuming that the hydraulic model is reliable and well
calibrated, the recurrent ANN can be implemented by aug-
menting the input data set with current real-time flow and level
measurements throughout the combined sewer system. In ad-
dition, the training data sets can be further expanded to include
forecast information from radar-based meteorological fore-
casting systems such as CALAMARÒ (RHEA, Inc.). Future
work will attempt to replace UNSTDY with SWMM 5.0
(Rossman, 2005) for dynamic unsteady flow modeling, and
eventually place the neural control software in a form suitable
for public distribution. Since the optimization model is exe-
cuted off-line in this neural control strategy, it is conceivable
that modeling could be extended to fully integrated control
of the entire urban drainage system, including wastewater
treatment plant operations and prediction of receiving water
quality impacts.
References
Albuquerque, F., Labadie, J., 1997. Optimal nonlinear predictive control for
canal operations. J. Irrig. Drain. Eng. 123 (1), 45e54.
Association of Metropolitan Sewerage Agencies (AMSA), 1994. Approaches to
Combined Sewer Overflow Program Development. AMSA, Washington DC.
Butler, D., Schütze, M., 2005. Integrating simulation models with a view to
optimal control of urban wastewater systems. Environ. Modell. Softw.
20, 415e426.
Chen, Y.-H., Chai, S.-Y., 1991. UNSTDY: Combined Sewer Model User’s
Manual. Chen Engineering Technology, Inc., Ft. Collins, CO.
Duchesne, S., Mailhot, A., Villeneuve, J.-P., 2004. Global predictive real-time
control of sewers allowing surcharged flows. J. Environ. Eng. 130 (5),
526e534.
Duchesne, S., Mailhot, A., Dequidt, E., Villeneuve, J.-P., 2001. Mathematical
modeling of sewers under surcharge for real time control of combined
sewer overflows. Urban Water 3, 241e252.
Freeman, J., 1994. Simulating Neural Networks with Mathematica. Addison
Wesley Publishing Company, Inc., Reading, MA.
Hassoum, M., 1995. Fundamentals of Artificial Neural Networks. The MIT
Press, Cambridge, MA.
Haykin, S., 1994. Neural Networks: A Comprehensive Foundation. IEEE
Press/Macmillan College Publishing Company, NY.
Huber, W., Dickinson, R., 1988. Stormwater Management Model, Version 4:
User’s Manual. Report no. EPA-600-3-88-001a. U.S. Environmental Pro-
tection Agency, Washington DC.
King County, Washington, USA, 2004. 2003e2004 Annual Combined Sewer
Overflow Report. Department of Natural Resources and Parks, Wastewater
Treatment Division, Seattle, WA.
Labadie, J., 1993. Optimal use of in-line storage for real-time urban storm-
water control. In: Cao, C., Yen, B.C., Benedini, M. (Eds.), Urban Storm
Drainage. Water Resources Publications, Inc., Highlands Ranch, CO.
Loucks, E.D., Locke, E.F., Heinz, S.R., Vitasovic, Z.C., 2004. A real-time
control strategy for operating the Milwaukee Metropolitan Sewerage District
(MMSD) conveyance and storage system. In: Sehlke, G., et al. (Eds.),
Proceedings of the 2004 World Water and Environmental Resources
Congress: Critical Transitions in Water and Environmental Resources Man-
agement, June 27eJuly 1, 2004, Salt Lake City, UT. Environmental and Wa-
ter Resources Institute and American Society of Civil Engineers, Reston, VA.
Marinaki, M., Papageorgiou, M., Messmer, A., 1999. Multivariable regulator
approach to sewer network flow control. J. Environ. Eng. 125 (3), 267e276.
Masters, T., 1996. Practical Neural Network Recipes in Cþþ. Elsevier
Science and Technology Books, Burlington, MA.
Parisini, T., Zoppoli, R., 1994. Neural networks for feedback feed-
forward nonlinear control systems. IEEE Trans. Neural Networks 5 (3),
436e449.
Pleau, M., Colas, H., Lavallée, P., Pelletier, G., Bonin, R., 2005. Global
optimal real-time control of the Quebec urban drainage system. Environ.
Modell. Softw. 20, 401e413.
Pontryagin, L., Boltyanskii, V., Gamkrelidze, R., Mishchenko, E., 1962. The
Mathematical Theory of Optimal Processes. Wiley-Interscience Publishing
Co., NY.
 S. Darsono, J.W. Labadie / Environmental Modelling & Software 22 (2007) 1349e1361
Rauch, W., Harremoës, P., 1999. Genetic algorithms in real-time control
applied to minimize transient pollution from urban wastewater systems.
Water Res. 33 (5), 1265e1277.
Rossman, L.A., 2005. Storm Water Management Model: User’s Manual. Ver-
sion 5.0. U.S. Environmental Protection Agency, Cincinnati, OH.
Schutze, M., Campisano, A., Colas, H., Schilling, W., Vanrolleghem, P.A.,
2002. Real-time control of urban wastewater systems e where do we stand
today? In: Proceedings of the Ninth International Conference on Urban
Drainage, Portland, OR, September 8e13.
Unver, O.L., Mays, L.W., 1990. Model for real-time optimal flood control op-
eration of a reservoir system. Water Resour. Manag. 4, 21e46.
1361
U.S. Environmental Protection Agency (U.S. EPA), 1999. Combined Sewer
Overflow Management Fact Sheet. EPA Report No. 832-F-99-041. U.S.
EPA, Washington DC.
Vanrolleghem, P.A., Benedetti, L., Meirlaen, J., 2005. Modelling and real-time con-
trol of the integrated wastewater system. Environ. Modell. Softw. 20, 427e442.
Vazquez, J., François, M., Gilbert, D., 2003. Real-time management of a sew-
age system: verification of the optimality and applicability of graphical
linear programming compared to mixed linear programming. Rev. Sci.
Eau 16 (4), 425e442 (article in French).
Weyand, M., 2002. Real-time control in combined sewer systems in Germany e
some case studies. Urban Water 4, 347e354.