Professional Documents
Culture Documents
5 - Sources and Impacts of Uncertainty in Uncalibrated Bioretention Models Using SWMM 5.1.012
5 - Sources and Impacts of Uncertainty in Uncalibrated Bioretention Models Using SWMM 5.1.012
Abstract: Using the USEPA’s Storm Water Management Model version 5.1.012 (SWMM), a case study of a street right-of-way bioretention
system (ROWB) configured as a storage node is compared against SWMM’s Low Impact Development (LID) Controls for urban runoff
retention, detention, and the timing of discharge. Through 12,000 one-year continuous simulations, single parameter perturbations and
Monte-Carlo simulation of the uncalibrated models result in predicted annual runoff coefficients (representing stormwater retention) of
0.19–0.55 for an exfiltrating ROWB compared to 0.61 and 0.72 for a storage node with low and high assumed exfiltration capacity,
respectively. Stormwater detention was represented by the frequency of event peak discharges exceeding an arbitrary low threshold value.
The storage node simulations predicted peak discharges near or exceeding the upper values for the LID Control simulations. The dynamic
representation of flow through porous media in the LID Control predicts greater retention and detention compared to the storage node over the
range of uncalibrated models investigated. Sensitivity analysis of the LID Control parameterization indicates that the relative difference
between the engineered media’s porosity and field capacity have the most significant influence on predicted performance. Poor runoff
retention results in scenarios where the engineered media exhibits a high field capacity relative to its porosity, whereas high field capacity
is a desirable trait that should lead to superior performance. The model’s calculation procedures and neglect of unsaturated flow or preferential
pathways bias model output toward more frequent runoff bypass. The sensitivity analysis also demonstrates that the timing and duration of
the discharge hydrograph are highly variable depending on parameterization. The wide range of potential performance generated from
uncalibrated model parameterization leads to significant concerns for infrastructure planning and implementation, leading potentially to
underperforming infrastructure, or excessive cost. Allocating resources to collect field performance data that enables robust model develop-
ment, calibration, and verification at the green infrastructure (GI) stormwater control measure (SCM) scale offers the opportunity to reduce
uncertainty in model predictions. DOI: 10.1061/JSWBAY.0000944. © 2021 American Society of Civil Engineers.
Author keywords: Bioretention; Hydrologic modeling; Green infrastructure; Low impact development (LID); Sensitivity analysis;
Uncertainty; Storm water management model (SWMM); Stormwater.
increasing, modeling approaches have been slow to evolve. Ac- media to the storage layer exfiltrates to the surrounding soils. In a
cording to an industry survey, one typical approach is to model continuous simulation, drying of the media through ET provides
a media-filled GI SCM as a storage node or stormwater pond, storage recovery. SWMM restricts the media’s lowest moisture
whose storage capacity is dictated by the porosity of the media, content to its wilting point (θwp ). Various mass balance checks be-
and other relevant elements (such as a surface ponding zone and tween SCM layers impose additional constraints on flow through
possible underlying stone storage layer in the case of bioretention) the system (Rossman and Huber 2016). If a unique runoff event is
(Arcadis 2016). Flow routing calculations have been applied as a bounded from the time of the onset of rainfall to the cessation of
steady-discharge (e.g., constant rate exfiltration to subsoils or free drainage from the SCM (exfiltration or underdrain flow), these
through underdrains), or through an adaptation of reservoir-routing. processes are summarized as
From an operational perspective, the active storage available to θmin ≥ θwp : initial moisture content at the onset of storm
manage a runoff event in a stormwater pond is an empty reservoir θwp ≤ θðtÞ < θfc : runoff infiltrating from ponding zone is re-
(a bucket) that fills from the bottom up, and drains by gravity. Con- tained in the media
versely, water flow through a media-filled GI SCM follows a tor- θfc ≤ θðtÞ < ϕ: additional runoff infiltrates from the ponding
tuous path, the rate of which is controlled largely by characteristics zone according to Green-Ampt, is temporarily detained in the me-
of the media and the media’s moisture conditions (which change dia, and percolates to gravel storage layer according to Darcy’s Law
throughout the duration of a runoff event). In underdrained media- Uncalibrated SWMM models are found in the literature
filled GI SCMs, the characteristics of the underdrain can also in- to evaluate bioretention placement for stormwater mitigation
fluence runoff control. In simplistic terms, flow routing through a (Bahrami et al. 2019; Dietrich et al. 2017; Fry and Maxwell 2017;
stormwater pond versus a media-filled GI SCM can be compared to Yang 2018), to prioritize bioretention implementation within mar-
filling and draining a bucket with a hole in it versus watering a ginalized communities (Garcia-Cuerva et al. 2018), to evaluate
potted plant. SCM mitigation under climate change scenarios (Zahmatkesh
The USEPA Stormwater Management Model (SWMM) version et al. 2014), and for GI cost-benefit and triple bottom line analysis
5.1 (2010) was the first open source model to provide routines to (Huang et al. 2018; Li et al 2017; Mei et al. 2018).
explicitly model the processes thought to govern the hydrology of Calibrated studies of the SWMM LID Controls using experi-
media-filled GI SCMs. These routines have recently been incorpo- mental or empirical data are limited. At the watershed scale, Rosa
rated into commercial software, such as InfoWorks ICM (Inno- et al. (2015) successfully calibrated and verified a SWMM model
vyze). SWMM uses the term LID Control, rather than GI SCM. using watershed parameters as calibration parameters, rather than
LID is an industry term that predates GI, and is often used inter- parameters within the LID Controls. Textbook values were used for
changeably by engineers (Fletcher et al. 2014). In the SWMM LID the GI SCM parameterization. Avellaneda et al. (2017) used outfall
Control, runoff from the catchment temporarily ponds on the sur- flow-duration curves to calibrate the hydrologic simulation of a res-
face of a GI SCM such as a bioretention system (Fig. 1), prior to idential neighborhood with 58 individual SCMs. Bioretention
seepage into the engineered media, or bypass through the overflow. SCMs were parameterized using unspecified field data and text-
Once the engineered media surface is saturated, water in the book values. Liu and Fassman-Beck (2016b, 2017) demonstrated
Fig. 1. Bioretention configuration: (a) exfiltrating system that discharges to surrounding in-situ soils; and (b) underdrained system with discharge of
managed flows to a downstream system such as a storm or combined sewer.
Fig. 2. The modeled ROWB and tributary drainage area on Junius St., Brooklyn, New York. Source of lower panel is Google Earth. (©2020 Google
Earth, Data SIO, NOAA, US Navy, NGA, GEBCO.)
for bioretention media in the SWMM user manual (Table 2). The
values in the SWMM user manual are predominantly sourced from
jurisdictional design manuals (Rossman and Huber 2016), without
further reference. Larger values of parameters such as field capacity
or porosity, and such as the upper bounds of parameter values used
herein, are not surprising for GI SCM engineered media. Engi-
neered media are often comprised of materials that maximize
stormwater storage and retention. For example, engineered media Fig. 3. Observed 2008 precipitation at John F. Kennedy International
derived from pumice yields very high field capacity because pum- Airport.
ice holds water within the aggregate itself, unlike a marine sand or
iron sand (Fassman-Beck et al. 2015; Liu and Fassman-Beck
2016a). Because SWMM’s flux calculations between layers dic-
tates that porosity must exceed field capacity to avoid a numerical
Sensitivity Analysis
error, the upper limit of porosity in the allowable range for the cur-
rent analysis is forced reasonably high (0.7), but is not too far above A sensitivity analysis framework was implemented for parameter
the range measured by Liu and Fassman-Beck (2016a). For surface combinations built by sampling values for each parameter from
and storage layers, parameter defaults were adopted as per the uniform distributions. Limited parameterization of the surface layer
user’s manual (Rossman and Huber 2016). and storage layers was also investigated.
The sensitivity analysis framework was controlled by a set of pressure transducer coupled with a primary device, based on expe-
automated Python and R scripts that generate LID Control param- rience of the first author. It does not indicate conditions that could
eters, pre and postprocess inputs/outputs, and coordinate the ex- cause a CSO or any other critical in-sewer conditions. The assumed
change of files between the generated random parameters of the interevent time is 6 h (Driscoll et al. 1989). The assumptions of the
ROWB and the SWMM input files (Fig. 4). The SWMM project magnitude of the threshold flow rate and interevent time influence
file (named with a .INP extension) contains all the information used the model’s determination of individual events. It is used here for
to model the study area and was generated for each sensitivity comparative purposes only, because the conditions are applied con-
analysis simulation to cover all the combinations of generated sistently across all simulations.
ROWB parameters for this site. To provide example insight into the effects of model parameter-
Initially, the sensitivity analysis was performed by systemati- ization on prediction of the timing of discharges from an ROWB,
cally perturbing a single input parameter at a time. However, the runoff hydrograph predicted by multiple model configurations
SWMM’s calculation procedure depends on interactions between were compared for a single storm of approximately 3.9 cm occur-
parameters, e.g., the available storage capacity and the hydraulic ring February 1–2, 2008. This storm was chosen because it is close
conductivity influence water movement from the ponding (surface) to the theoretical 95th percentile event for total depth, which is in
layer to the soil (engineered media) layer, and vary throughout a the range of common design storms recommended for GI SCM
single storm event. Therefore, a Monte Carlo simulation was also design in many jurisdictions in the US.
performed to investigate the effect of randomly and simultaneously
perturbing input parameters within the allowable ranges.
Results and Discussion
Performance Metrics
LID Control Sensitivity to Parameterization for
Runoff retention and detention for the catchment were selected as Retention
metrics for evaluating model sensitivity. The annual runoff coeffi-
cient was selected to quantify retention. It is the ratio of runoff vol- The sensitivity analysis includes 12,000 simulations to quantify
ume discharging from the catchment to annual rainfall, calculated and analyze model output variability caused by the assumed
according to ROWB input parameter values (Table 3). It must be emphasized
that these results are for comparison of how model configuration
Runoff Coefficient influences results, and highlights the uncertainty inherent in the use
Annual Runoff Volume into Combined Sewer of an uncalibrated model. Accuracy cannot be assessed because
¼ ð1Þ there is no comparison to observed performance.
Annual Rainfall
Single-parameter sensitivity analysis for an exfiltrating ROWB
In the current analysis, the runoff volume discharging from the using the LID Control showed that the runoff coefficient is not sen-
catchment is determined as runoff bypassing the ROWB, which sitive to surface or storage layer parameterization. Likewise, the
subsequently enters the combined sewer. runoff coefficient was not influenced by the engineered media’s
The metric for evaluating runoff detention was arbitrarily se- wilting point, hydraulic conductivity, conductivity slope, or soil
lected as the number of peak flows occurring with magnitude suction head (Table 3). It is hypothesized that the wilting point
greater than 0.6 L=s during the simulated 2008 year. The threshold could emerge as a sensitive parameter depending on ET assump-
of 0.6 L=s was identified as a low flowrate that could be feasibly tions. Studies have shown that ET from GI differs substantially
measured with typical field monitoring equipment, such as a from common ET estimates derived from climate data and readily
provided by climate stations, especially during periods when water
is limited (Poë et al. 2015; Voyde et al. 2010; Berretta et al. 2014;
DiGiovanni et al. 2013; Voyde 2011; Wadzuk et al. 2013; Hess
et al. 2017).
The single-parameter sensitivity analysis approach suggests that
the runoff coefficient predicted by the model is predominantly sen-
sitive to field capacity (Table 3). Deeper investigation indicates that
the relative difference between field capacity and porosity is the
most influential factor in SWMM’s LID Control calculations.
SWMM assumes that all water entering the ROWB media layer
Fig. 4. SWMM sensitivity analysis framework.
is retained until the moisture content reaches the engineered
media’s field capacity (Rossman and Huber 2016). Until this state temporary storage is always low even if percolation is significant,
of moisture content, the model assumes there is no percolation out minimizing infiltration from the surface and thereby causing an in-
of the media into the underlying storage layer. In other words, there crease in system overflow (bypass). In practice, preferential flow
is no unsaturated flow that can create flow out of the media. Once paths are available (Liu and Fassman-Beck 2017), inflow may
field capacity is reached, additional runoff entering the media, up to not be uniformly distributed over the surface (Brown et al. 2015),
the capacity dictated by its porosity, is temporarily stored (de- and the media is not necessarily saturated when infiltration and per-
tained). The amount of water entering the media layer from the colation occur (Carpenter and Hallam 2010; Liu and Fassman-Beck
ponding layer at any time step is limited by the media layer’s avail- 2018). Assumptions that percolation out of the media does not oc-
able storage. Available storage is determined by the balance be- cur until the field capacity is satisfied, and that temporary available
tween the difference between the actual moisture content at any storage is limited by the relationship between field capacity and
time and the porosity, and the rate at which water percolates out porosity, are seen as limitations to the LID Control calculations that
of the media and into the underlying storage layer. This latter step introduce error and uncertainty.
is not usually limiting, because the storage layer in an ROWB is The range of model results in Fig. 6(a) from the single parameter
usually large rock that drains freely. However, water in the ponding perturbation sensitivity analysis reflect the ROWB when there is a
zone cannot infiltrate into the engineered media if there is no avail- relatively large difference between field capacity and porosity
able storage in a given time step, and any new inflow bypasses the (Fig. 5). There is always substantial available storage and the an-
ROWB via the overflow. nual runoff coefficient shows little variation (0.2–0.3), suggesting
The difference between the field capacity and porosity can be an the ROWB retains and exfiltrates substantial runoff. However,
important performance-limiting artefact introduced by the LID when the field capacity approaches the assumed porosity (Fig. 5),
Control’s calculation procedure, rather than an actual physical phe- there is little available storage, so water cannot infiltrate the media
nomenon. The LID Control calculates significant available pore from the surface ponding layer [Fig. 6(b), in this case, above a field
space for water to infiltrate into the media layer during runoff con- capacity of approximately 0.3 when porosity is fixed at 0.41]. In-
ditions with an engineered media characterized by low field capac- stead, it bypasses the system and results in high values of the annual
ity relative to porosity (Fig. 5). Conversely, infiltration from the runoff coefficient. Concurrently, little water is available to percolate
surface is substantially limited at any time after the media’s mois- out of the media into the storage layer, and into the surrounding
ture content equals field capacity, when the media’s field capacity soils, shown as a decrease in annual exfiltration for high values
value is close to the porosity (Fig. 5). In this case, the available of field capacity [Fig. 6(c)]. Under these conditions with imposed
limitations on available storage, the runoff coefficient varies more
substantially from 0.2 to 0.55.
The change in runoff response is not easily predicted when all
parameters are perturbed using the Monte Carlo simulation [com-
paring Fig. 7 with Figs. 6(a and b)]. A soil’s actual porosity, field
capacity, wilting point, and hydraulic conductivity will change with
environmental conditions. For example, field capacity at any time is
partly a function of pore size distribution (affected by media set-
tling/compaction), and organic content, both of which will change
over time, while porosity will be substantially affected by media
settling/compaction. Root growth enhances hydraulic conductivity
Fig. 5. Conceptual representation of the importance of field capacity by providing structure, and counteracting clogging and/or compac-
versus porosity. The bucket represents the total volume of engineered
tion from long-term operation (Hatt et al. 2009; Muerdter et al.
media. The water in the bucket represents the maximum retained runoff
2018). While these variations are not expressly modeled, the high
for medias with high and low field capacity. Temporary available sto-
variability in the runoff coefficient resultant from random pertur-
rage for runoff detention depends on the relative difference between the
bation could be considered more representative of the day-to-
media porosity and field capacity.
day variation in operating conditions.
Fig. 6. Example effects of the interaction between field capacity and porosity on the (a and b) runoff coefficient; and (c) exfiltration to surrounding
soils in the SWMM 5.1.012 LID Control.
A sensitivity analysis was used to investigate model prediction un- Some or all data, models, or code that support the findings of this
certainty derived from ROWB flow process representation in study are available from the corresponding author upon reasonable
SWMM version 5.1.012, comparing scenarios where the ROWB request.
was modeled as a storage node or using the LID Control, and un-
certainty derived from parameterization of the ROWB within the
LID Control. The LID Control simulates flow through the engi- Acknowledgments
neered media of a bioretention system using a modified form of
Green-Ampt infiltration coupled with Darcy’s Law, whereas the This work was funded by Riverkeeper, Inc. through the New York
storage node is an empty reservoir that fills and drains independent City Stormwater Infrastructure Matters (SWIM) Coalition. Stevens
of media. Institute of Technology (Hoboken, NJ) provided in-kind support.
Arcadis. 2016. “Green infrastructure performance metrics report prepared applications manual for the storm water management model (SWMM).”
for the city of New York department of environmental protection.” Environ. Modell. Software 25 (6): 813–814. https://doi.org/10.1016/j
Accessed February 28, 2017. http://www.nyc.gov/html/dep/html .envsoft.2009.11.009.
/stormwater/nyc_green_infrastructure_plan.shtml. Gülbaz, S., and C. M. Kazezyılmaz-Alhan. 2017. “An evaluation of hydro-
ASCE. 2017. “Code of ethics.” Accessed January 6, 2019. https://www logic modeling performance of EPA SWMM for bioretention.” Water
.asce.org/code-of-ethics/. Sci. Technol. 76 (11): 3035–3043. https://doi.org/10.2166/wst.2017
Avellaneda, P. M., A. J. Jefferson, J. M. Grieser, and S. A. Bush. 2017. .464.
“Simulation of the cumulative hydrological response to green infra- Hatt, B. E., T. D. Fletcher, and A. Deletic. 2009. “Pollutant removal per-
structure.” Water Resour. Res. 53 (4): 3087–3101. https://doi.org/10 formance of field-scale stormwater biofiltration systems.” Water Sci.
.1002/2016WR019836. Technol. 59 (8): 1567–1576. https://doi.org/10.2166/wst.2009.173.
Bahrami, M., O. Bozorg-Haddad, and L. Hugo. 2019. “Optimizing storm- Hess, A., B. M. Wadzuk, and A. W. Welker. 2017. “Evapotranspiration in
water low-impact development strategies in an urban watershed consid- rain gardens using weighing lysimeters.” J. Irrig. Drain. Eng. 143 (6):
ering sensitivity and uncertainty.” Environ. Monit. Assess. 191 (6): 04017004. https://doi.org/10.1061/(ASCE)IR.1943-4774.0001157.
1–14. https://doi.org/10.1007/s10661-019-7488-y. Hohaia, N., E. Fassman, W. F. Hunt, and K. A. Collins. 2011. “Hydraulic
Berretta, C. S. Poë, and V. Stovin. 2014. “Moisture content behaviour in and hydrologic modelling of permeable pavement.” In Proc., World
extensive green roofs during dry periods: The influence of vegetation Environmental and Water Resources Congress 2011: Bearing Knowl-
and substrate characteristics.” J. Hydrol. 511 (2014): 374–386. https:// edge for Sustainability. Palm Springs, CA: World Environmental and
doi.org/10.1016/j.jhydrol.2014.01.036.
Water Resources Congress. https://doi.org/10.1061/41173(414)61.
Brown, R. A., T. P. O’Connor, and M. Borst. 2015. “Divergent vegetation
Huang, C. L., N. S. Hsu, H. J. Liu, and Y. H. Huang. 2018. “Optimization
growth patterns relative to bioinfiltration unit size and plant placement.”
of low impact development layout designs for megacity flood mitiga-
J. Sustainable Water Built Environ. 1 (3): 04015001. https://doi.org/10
tion.” J. Hydrol. 564 (Sep): 542–558. https://doi.org/10.1016/j.jhydrol
.1061/JSWBAY.0000796.
.2018.07.044.
Brown, R. A., R. W. Skaggs, and W. F. Hunt. 2013. “Calibration and val-
Kaini, P., K. Artita, and J. W. Nicklow. 2012. “Optimizing structural best
idation of DRAINMOD to model bioretention hydrology.” J. Hydrol.
management practices using SWAT and genetic algorithm to improve
486 (Apr): 430–442. https://doi.org/10.1016/j.jhydrol.2013.02.017.
water quality goals.” Water Resour. Manage. 26 (7): 1827–1845.
Carpenter, D. D., and L. Hallam. 2010. “Influence of planting soil mix
https://doi.org/10.1007/s11269-012-9989-0.
characteristics on bioretention cell design and performance.” J. Hydrol.
Li, J., C. Deng, Y. Li, Y. Li, and J. Song. 2017. “Comprehensive benefit
Eng. 15 (6): 404–416. https://doi.org/10.1061/(ASCE)HE.1943-5584
evaluation system for low-impact development of urban stormwater
.0000131.
management measures.” Water Resour. Manage. 31 (15): 4745–4758.
Davis, A. P., J. M. Olszewski, R. G. Traver, R. A. Brown, W. F. Hunt, and
https://doi.org/10.1007/s11269-017-1776-5.
R. Lee. 2012. “Hydrologic performance of bioretention storm-water
control measures.” J. Hydrol. Eng. 17 (5): 604–614. https://doi.org/10 Liu, R., and E. Fassman-Beck. 2016a. “Effect of composition on
.1061/(ASCE)HE.1943-5584.0000467. basic properties of engineered media for living roofs and bioretention.”
Davis, A. P., R. G. Traver, W. F. Hunt, R. Lee, R. A. Brown, and J. M. J. Hydrol. Eng. 21 (6): 06016002. https://doi.org/10.1061/(ASCE)HE
Olszewski. 2011. “Hydrologic performance of bioretention storm-water .1943-5584.0001373.
control measures.” J. Hydrol. Eng. 17 (5): 604–614. https://doi.org/10 Liu, R., and E. Fassman-Beck. 2016b. “Hydrologic response of engineered
.1061/(ASCE)HE.1943-5584.0000467. media in living roofs and bioretention to large rainfalls: Experiments
Dietrich, A., R. Yarlagadda, and C. Gruden. 2017. “Estimating the potential and modeling.” Hydrol. Processes 31 (3): 556–572. https://doi.org/10
benefits of green stormwater infrastructure on developed sites using .1002/hyp.11044.
hydrologic model simulation.” Environ. Prog. Sustainable Energy Liu, R., and E. Fassman-Beck. 2017. “Hydrologic experiments and mod-
36 (2): 557–564. https://doi.org/10.1002/ep.12428. eling of two laboratory bioretention systems under differenct boundary
DiGiovanni, K., F. Montalto, S. Gaffin, and C. Rosenzweig. 2013. “Appli- conditions.” Front. Environ. Sci. Eng. 11 (4): 10. https://doi.org/10
cability of classical predictive equations for the estimation of evapotran- .1007/s11783-017-0951-5.
spiration from urban green spaces: Green roof results.” J. Hydrol. Liu, R., and E. Fassman-Beck. 2018. “Pore structure and unsaturated hy-
Eng. 18 (1): 99–107. https://doi.org/10.1061/(ASCE)HE.1943-5584 draulic conductivity of engineered media for living roofs and bioreten-
.0000572. tion based on water retention data.” J. Hydrol. Eng. 23 (3): 04017065.
Driscoll, E. D., G. E. Palhegyi, E. Strecker, and P. Shelley. 1989. Analysis of https://doi.org/10.1061/(ASCE)HE.1943-5584.0001621.
storm event characteristics for selected rainfall gages throughout the Mei, C., J. Liu, H. Wang, Z. Yang, X. Ding, and W. Shao. 2018. “Integrated
United States. Rep. for the USEPA. Washington, DC: USEPA. assessments of green infrastructure for flood mitigation to support
Dussaillant, A. R., K. W. Potter, and C. H. Wu. 2005. “Infiltration of storm- robust decision-making for sponge city construction in an urbanized
water in bioretention cells: Numerical model and field experiment.” watershed.” Sci. Total Environ. 639 (Oct): 1394–1407. https://doi.org
Ingenieria Hidraulica Mex. 20 (2): 5–17. /10.1016/j.scitotenv.2018.05.199.
Fassman-Beck, E., S. Wang, R. Simcock, and R. Liu. 2015. “Assessing Muerdter, C. P., C. K. Wong, and G. H. LeFevre. 2018. “Emerging inves-
the effects assessing the effects of bioretention’s engineered media com- tigator series: The role of vegetation in bioretention for stormwater
position and compaction on hydraulic conductivity and water holding treatment in the built environment: Pollutant removal, hydrologic func-
capacity.” J. Sustainable Water Built Environ. 1 (4): 04015003. https:// tion, and ancillary benefits.” Environ. Sci. Water Res. Technol. 4 (5):
doi.org/10.1061/JSWBAY.0000799. 592–612. https://doi.org/10.1039/C7EW00511C.
Fletcher, T., et al. 2014. “SUDS, LID, BMPs, WSUD and more: The National Weather Service. n.d. “Weather forecasts.” Accessed June 15,
evolution and application of terminology surrounding urban drainage.” 2017. http://www.weather.gov/okx/KennedyHistorical.