You are on page 1of 35

ADHydro: High-Resolution Process

Based Hydrological Modeling in a High
Performance Computing Environment
Fred Ogden
Norm Jones
David Tarboton
Steve Burian

and 4th year
CI-WATER project
July 15, 2015

Project Staff & Students
Drs. Kristi Hansen, Craig Douglas, Scott Miller, Ye Zhang co-I's
Dr. Robert Steinke: Lead software developer
Dr. Wencong Lai: Hydrodynamics
Dr. Hernan Moreno: Land-atmosphere interaction
Mr. Yoshiyuki Igarashi: Socio-econ./water mgmt.
Nels Frazier (M.S.): workflows, scaling/parameter est.
Leticia Pureza (M.S.): workflows, NOAA-MP coupling, testing
Yanyan Cheng (Ph.D): formulation additions/testing
Jason Regina (M.S./Ph.D.) formulation additions/testing

Located in Laramie, Pop. 30,000
#1-Ranked University in Terms of:
– Elevation (7200 ft)
– Percentage of students from tiny high schools
– Petaflops per capita thanks to NWSC
Center for Computational Hydrology and
Hydrosciences, founded 2013.
New Engineering Initiative ($18M this biennium) plus
new 60,000 sq. ft building.

Moran Campus Cluster 3480 i7 cores 400 TB storage NCAR-Wyoming Supercomputing Center (NWSC) ~74.000 i7 cores UWyo allocation 20% .Computing Resources: Mt.

Engineers/Researchers • Experimentalists • Modelers/forecasters A Digital Divide HPC Specialists c #!/bin/bash vi chmod #PBS -l nodes=4:ppn=8 grep awk mpiexec .

0M • Focused on acquisition of hardware. Total budget $6. education.CI-WATER Project • NSF RII Track 2 Cyberinfrastructure Cooperative Agreement joint between NSF EPSCoR and Utah and Wyoming EPSCoR jurisdictions.Aug. capacity building. 2015 . and outreach. . 2011. development of software. 1. • Sept. 31.

Advance high-resolution multi-physics watershed modeling 4.and computationally. Enhance cyberinfrastructure facilities at collaborating universities.intensive modeling 3. Promote STEM learning and water science engagement across diverse groups .Project Objectives: 1. Enhance access to data. 2.

NWSC . of Utah Data Storage Utah Outreach Urban Hydrology Climate simulations and downscaling NCAR Research U. of Wyoming Hydrologic Modeling Software Engineering HPC.Team and Partners U. Wyoming Outreach Hydrologic Modeling Hydrologic Information Systems Water Resources Decision Support Applications Lab. US Army Corps of Engineers US National Water Center Hydrologic Modeling Geospatial data models Integrated modeling software BYU Utah State U.

and management. engineering. .Enhance access to high-performance computing for water resources research.

GSSHA User's Manual (2003).gsshawiki.Gridded Surface/Subsurface Hydrologic Analysis (GSSHA) model Downer and Ogden. Hydrologic Engineering (2004) . J.

2D overland flow. . wetland and groundwater flow .Square grid (5 to 90 m typical grid size) . or Green-Ampt Redistribution coupling between overland flow and groundwater . .Richards' Eqn. nutrients GSSHA is an excellent hydrologic engineering tool. sediment transport.1D channel routing with hydraulic structures .Multi-physics .GSSHA .Erosion/deposition.Multi-solver.

Flood inundation mapping/post event analysis .Flood forecasting in civil and military contexts . 2013 .Urban flood hydrology/storm drainage/land use change .FEMA Certified for use in flood insurance studies.Channel improvements and levee design .Soil moisture/trafficability predictions .Hurricane storm surge predictions in coastal areas .GSSHA Applications Supported by the DoD Watershed Modeling System (WMS) .

Army Corps of Engineers.S. ERDC) . Coastal & Hydraulics Laboratory.GSSHA Applications Storm surge GSSHA simulation output (U.

But.Soils hydraulic parameters We can teach junior-level engineering students to run GSSHA using the Watershed Modeling System (WMS) software in less than one week. .Where things are located in the watershed .GSSHA Model Simulations We have published numerous papers showing importance of: .Runoff generation mechanism .GSSHA is not efficient for simulating LARGE watersheds because of its square grid formulation.

000 km2 - 3.High resolution important in mountains .Upper Colorado River Basin: 280.Low resolution in broad and extensive basins .A LARGE watershed problem: .1x108 grids at 30 m square grid size .

5% (9.000 ft) Area above 3050 m: 3.2% (10.000 (USBR) Area above 2700 m: 14.000 km Population: 900.000 ft) produces most snow-melt runoff .Upper Colorado River Basin Basin Area: 288.000 km2 Streams: 467.

What are the effects of trans-basin diversions and increases in water consumptive use on the water storage in Lake Powell in coming decades? .What are the potential impacts of climate change on the long-term yield of water from the upper Colorado River basin? . mountain pine bark beetle outbreak affect water supplies? .How will future land-use changes due to development and natural causes such as fire. large-scale hydrologic model to answer three questions: .CI-WATER Component 3 Objective Develop a high-resolution.


Inset shows 10 m Digital Elevation Model (32 GB) .Can describe Amazon basin with minimal distortion .Lines of latitude are horizontal lines .Interrupted Sinusoidal Projection -Preserves area perfectly .Longitudes converge towards the pole .

Quasi-3D variable resolution large watershed model on an unstructured grid .

High Altitude Complexity .

.Diverse Process-Based Runoff Generation surface water: 2D shallow water equations dynamic wave diffusive wave kinematic wave 1D vadose zone coupling 2D saturated groundwater flow two layers that represent perched and unconfined aquifers.

2015 ODE) 2D groundwater flow (parabolic diffusive) gn 2y v √ u2 +v 2 Sy ( h1/ 3 ) ∂H ∂H ∂H = ∂ K x ( H −z b ) + ∂ K y ( H −z b ) +R ∂t ∂ x ∂x ∂ y ∂y ( ) ( ) .Mathematical model ∂h ∂hu ∂ hv + + =0 ∂t ∂ x ∂ y 2D dynamic wave: (hyperbolic convective) 2 2 2 ∂hu ∂huu ∂ huv ∂ z gn x u √ u +v + + =−gh − ∂t ∂x ∂y ∂x h1/ 3 ∂hv ∂ huv ∂hvv ∂z + + =−gh − ∂t ∂x ∂y ∂y 1D vadose zone flow ∂ K (θ) ∂ ψ (θ ) dz = 1+ dθ θ ∂θ ∂z ( ) (Ogden et al.

Numerical model 2D unstructured finite volume method for overland flow and saturated groundwater flow ∂U + ∇⋅F=S ∂t ∂U ∫ ∂ t d Ω+∫ ∇⋅Fd Ω=∫ Sd Ω ∂U ∫ ∂ t d Ω+∮ F⋅ndΓ =∫ Sd Ω 1 n 3 U n+ −U 1 i i + Ω ∑ Fij⋅nij ΔΓ ij =Si Δt i j=1 Upwind Riemann solver for convective flux in overland flow Central difference for diffusion term in groundwater equation .

(2015) .1-D Finite water-content infiltration Ogden et al.

.1-D Finite water-content infiltration Ogden et al. 8 month simulation with 263 cm of rainfall and 1m depth to water table . WRR (2015).

Finite Water Content solution (T-O) (Ogden et al. 200 times faster than T-O and up to 2000 times faster than Richards' (1931) equation with guaranteed mass conservation. WRR 2015) for vadose zone dynamics in response to changes in groundwater table elevation: . Infiltration: Green & Ampt with Redistribution (GAR) (Ogden & Saghafian 1997) enhanced. Steinke. Ogden.“GARTO” Scheme (Lai. WRR 2015). Talbot..

“GARTO” performance: -two pulses of rainfall -water table set at 4Ψb below ground surface. AND guaranteed to conserve mass. -fully coupled surface to groundwater table .advantage of GARTO scheme is that it is explicit. arithmetic. .

.g.Well defined and documented Application Programming Interface (API) . METIS) . with optional load balancers (e.Parallelized using CHARM++ object-oriented run time system.Written in C with C++ and Fortran wrappers needed to call NOAH-MP.Designed to allow addition of alternative process mathematical descriptions .Model Design Philosophy .Open source .

Reservoirs.Topography: USGS NED.Streams: thalweg elevation. water rights . satellite or modeled. . layers.Soils: texture. diversions. thicknesses .Aquifers: alluvial and tributary extent and transmissivity . cross section. SRTM .Forcing: dynamically downscaled climate simulations using Weather Research Forecasting (WRF) model .Inputs .Land use/land cover: airborne. irrigated areas. roughness distribution (from scaling laws) .

River data set impossible to create manually .Use geomorphological cross-section predictors & scaling laws .TauDEM verified using National Hydrography Data Set (NHD) .Almost 500.Upper Colorado River Stream Network .000 km of streams in NHD .

Lakes & Links Land use/cover soils maps Soil thickness model Surficial aquifer maps Mesh Params.Vertex Thinning Channel network Triangle.D.mesh generation Reservoirs and management Scaling laws Channel inputs I.Mesh/Channel work flow (simplified) TauDEM GIS. .

July 17.  NOAH-MP land surface scheme. The model uses state-of-the-art RELIABLE vadose zone solver that is many many times faster than Richards' equation. NWC. Friday. Key features of the model include:  Water management layer.Conclusions     We have created a quasi-3D variable resolution HPC large watershed sumulator that we call ADHydro. .  Forced by WRF  Load balancing provided by CHARM++ ADHydro Workshop.

Acknowledgements    Our excellent collaborators in Utah! NSF EPSCoR RII-Track 2 Cooperative Agreement EPS-1135483 NSF WSC EAR-1360384 Planning and Land Management in a Tropical Ecosystem: Complexities of Land Use and Hydrology Coupling in the Panama Canal Watershed .

W.. joint 10th Federal Interagency Sedimentation Conference and 5th Federal interagency Hydrologic Modeling Conference. Ogden. Ogden.. USA.A.1002/2014WR016487.References Ogden. J. Ogden.A. R. Lai. Lai. Res. Talbot. April 19-23. 2015. Steinke.C. ADHydro: quasi-3D high performance hydrological model. Talbot and J. Steinke...1002/2014WR016454. doi:10. Pep Reno. Steinke. W. F. An efficient and guaranteed stable numerical method for continuous modeling of infiltration and redistribution with a shallow dynamic water table. and C.L. W. 2015. Water Resour. F. F.1002/2015WR017126. Nevada.L. R. W.C. 2015.L. Zhu. 51. Proc. (online) . Validation of finite watercontent vadose zone dynamics method using column experiment with a moving water table and applied surface flux. Res. Wilson. Water Resour. Zhu. F.L. SEDHYD 2015. and R. R. 2015.C..L.C. A new general 1-D vadose zone flow solution method. Water Resour. 51. C. Steinke. Lai. and J. Res. doi: 10. Lai.. doi:10..