Workshop 2 Solution

Calibrating and Validating a Single Event Model

Question 1: Given the objective of the study, what attributes of simulated flow should you focus
on during calibration?

Because we will be simulating the PMF and other low-frequency events to spillway adequacy
and inundation extents, reproducing peak flows should be the primary consideration during
calibration. Performance metrics that measure goodness-of-fit, like Nash-Sutcliffe efficiency,
indicate if we are accurately reproducing the hydrologic response. The Percent-Bias metric
indicates over or under-prediction of runoff volume.

 Table 1, Table 2, and Table 3 show results for the calibrated events Feb 1986, Jan 1997,
and Jan 2006.

Table 1. Feb 1986 calibration summary

Simulated Peak Observed Peak

Location NSE Qualification
(cfs) (cfs)
CV Dam Inflow - - 11,364 11,200
Ukiah Gage - - 12,368 12,300
Hopland Gage - - 32,598 32,900
Cloverdale Gage - - 40,987 40,700
Healdsburg Gage - - 71,998 71,100
Guerneville Gage - - 97,700 102,000

Table 2. Jan 1997 calibration summary

Simulated Peak Observed Peak

Location NSE Qualification
(cfs) (cfs)
CV Dam Inflow 0.910 Very Good 7,791 8,270
Ukiah Gage - - 11,363 -
Hopland Gage 0.874 Very Good 20,508 20,000
Cloverdale Gage 0.602 Good 29,081 29,000
Healdsburg Gage 0.683 Very Good 62,438 65,663
Guerneville Gage 0.601 Good 90,088 82,100

1
WS – Calibrating and Validating a Single Event Model/Brauer
 Table 3. Jan 2006 calibration summary

Simulated Peak Observed Peak

Location NSE Qualification
(cfs) (cfs)
CV Dam Inflow 0.925 Very Good 14,638 14,600
Ukiah Gage 0.921 Very Good 22,250 22,600
Hopland Gage 0.895 Very Good 35,902 35,600
Cloverdale Gage 0.832 Very Good 50,517 50,700
Healdsburg Gage 0.901 Very Good 62,062 58,900
Guerneville Gage 0.710 Very Good 84,061 86,000

 Table 4 provides a sample summary of the Constant Rate parameter. In this example, the
validation parameter set was an average of the parameters used in three calibration
events. In the constant rate parameter was adjusted.

Table 4. Constant Rate (in/hr)

Subbasin Event: 1986 Event: 1997 Event: 2006 Validation

EF Russian 20 0.28 0.14 0.28 0.23

EF Russian 10 0.25 0.14 0.25 0.21
WF Russian 0.35 0.19 0.21 0.25
Russian 70 0.20 0.28 0.56 0.35
Russian 60 0.10 0.14 0.26 0.17
Russian 50 0.21 0.17 0.30 0.23
Big Sulphur Cr 0.09 0.05 0.28 0.14
Russian 40 0.29 0.16 0.88 0.44
Russian 30 0.14 0.08 0.44 0.22
Dry Creek 30 1.56 1.30 1.33 1.40
Dry Creek 20 0.17 0.14 0.14 0.15
Dry Creek 10 0.20 0.17 0.18 0.18
Laguna 0.34 0.28 0.29 0.30
Santa Rosa Cr 20 0.24 0.2 0.21 0.22
Santa Rosa Cr 10 0.31 0.26 0.27 0.28
Green Valley 0.35 0.29 0.30 0.31
Russian 20 0.92 0.77 0.78 0.82
Austin Cr 0.08 0.07 0.07 0.07
Russian 10 0.20 0.17 0.18 0.18

2
WS – Calibrating and Validating a Single Event Model/Brauer
  Error! Reference source not found. shows results for the 1995 event, run with the
validation parameter set, at each of the observation locations: CV Dam Inflow, Ukiah
Gage, Hopland Gage, Cloverdale Gage, Healdsburg Gage, and Guerneville Gage.

Table 5. Jan 1995 validation summary

Simulated Peak Observed Peak

Location NSE Qualification
(cfs) (cfs)
CV Dam Inflow 0.815 Very Good 8,646 8,780
Ukiah Gage 0.911 Very Good 12,108 12,290
Hopland Gage 0.878 Very Good 20,239 27,600
Cloverdale Gage 0.680 Very Good 27,817 39,400
Healdsburg Gage 0.800 Very Good 56,319 73,000
Guerneville Gage 0.520 Satisfactory 74,577 93,900

Question 2: Is your model sufficient for the study? If not, what should you do to improve
confidence in the model?

Results may vary. In any case, more calibration and validation events will reduce uncertainty in
the model. The amount of calibration and validation events in your study may vary depending on
the amount of events that you have quality data for, or, time and budget available to execute the
project. In this example, we calibrated to three events and validated to one event. The results are
good but overall we would benefit from more calibration and validation events. In the actual
study, the 1986, 1995, 1997, and 2006 events were used for model calibration. The January 1997
calibration event was not used to form the validation parameter set because precipitation data
was found to be inadequate. The model was validated against the 1% AEP storm.

3
WS – Calibrating and Validating a Single Event Model/Brauer