AI for Climate & Weather Modelling

Jeff Adie, Principal Solutions Architect, NVAITC

 AGENDA
• Introduction
• Modelling planet Earth
• Modelling Challenges
• Applying AI to Climate Modelling
• Intermission
• Digital Twins
• The Earth-2 Project
Introduction
NVIDIA AI TECHNOLOGY CENTRE (NVAITC)
Established 2015 in Singapore (as NVIDIA Technology Centre)

4
NVAITC EARTH SYSTEM SCIENCE PROJECTS
Over 35 research projects in climate and weather

5
C L I M AT E C H A N G E

6
DRAMATIC RISE IN EXTREME WEATHER ACROSS THE GLOBE 7
 EXTREME APRIL HEAT DRIVEN BY CLIMATE CHANGE
Worldweatherattribution.org

https://spiral.imperial.ac.uk/handle/10044/1/104092 8
 CLIMATE CHANGE – MORE FREQUENT FLOODS

2050

https://coastal.climatecentral.org 9
 The Future Under Climate Change will be Harsh
We urgently need better tools to prepare for it

WATER MANAGEMENT

WILDFIRE PREVENTION WATER MANAGEMENT CROP FORECASTING

 AI + CLIMATE CHANGE
The four ‘M’s

Monitoring Modelling Mitigation Modification/Adaptation

11
Modelling planet Earth
 Our Climate
Components of the Climate System
Five Components:
• Atmosphere
• Hydrosphere
• Cryosphere
• Lithosphere
• Biosphere

These components are coupled together into

a system with processes and cycles (water
cycle, carbon cycle, etc)

13
 MODELLING THE PLANET
The ESM

Weather: State of the Atmosphere

Cryosphere
Climate: Statistics of Weather over Time (usually
minimum > 30 years) Land
Both Climate and Weather modelling use Earth System
Models (ESM) that represent the planet as a coupled
Ocean
subset of climate systems.

Weather Models typically use the Atmosphere or

Atmosphere+Ocean
Atmosphere
Climate Models typically use Atmosphere+Ocean+Land+Ice

14
 MODEL PREDICTION COMPONENTS
The art of predicting the future using the laws of physics

DISCRETIZE PARAMETERIZE PREDICT

DATA ASSIMILATION DYNAMICS PHYSICS TIME INTEGRATION

Combine observations from Fluid dynamics on a Approximate important Integrate the state forward
multiple sources to create rotating sphere at finite physical processes not on a powerful machine to
an accurate initial state resolution resolved by the dynamics. predict the future Earth
system state.

15
Modelling Challenges
 CHALLENGES
Why is it so difficult to predict the weather?

FINITE RESOURCES
CHAOS INCOMPLETE SHORT TIME-TO-SOLUTION Never enough compute to
Sensitivity to initial OBSERVATIONS Time between observations represent the full system or
conditions. Small Can’t determine the initial and prediction must be to assimilate all possible
difference grow rapidly conditions exactly. Need to small in order to accurately data sources.
over time. minimize the error. represent the system.

17
 Today's Climate Predictions are too Low Resolution
Detailed Plans require Detailed Information and Predictions with Credible Cloud Feedbacks
CLIMATE SCIENCE REQUIRES MILLION-X SPEEDUPS
Computational constraints limit model resolution

20
21
CLIMATE SCIENCE REQUIRES MILLION-X SPEEDUPS
Model Resolution is only one axis of computational growth

22
 ENSEMBLES IMPROVE PREDICTION SKILL

Ensemble Types:
Multi-Model Ensembles
Perturbed Parameter Schemes
Stochastic Parameterisation:

23
 It is Hard to Interact with High Resolution Climate Prediction Data.
"We can compute km-scale predictions, but can't effectively extract information content, let alone interact with it"
-- Bjorn Stevens.
 CLIMATE SHAMING
Yes, it’s a thing!

Carbon emission cost of IT is similar to airline industry

Climate models require long runs on very large computers

GPUs provide energy-efficient computing:

MPAS Global Model

350
Kg CO2 per simulated day

300
250
200
150
100
50
0
10 km 3 km
CPU GPU

25
Earth
System
Models

Artificial Intelligence

Accelerated
Computing

How can AI help us improve upon

existing Earth System models?
 WAYS TO COMBINE AI + EARTH SYSTEM MODELS
Augment, Emulate, Data-Driven Models, and Physics Informed Neural Nets

NWP Augmentation Full-Model Emulation/Surrogate Data-Driven Model Physics-Informed Neural Net

AI + Numerical Model Model Data Observations Observations + PDE
Train using model data

29
 AI can augment all stages of the numerical weather prediction pipeline

NWP AUGMENTATION
30
 ECMWF AND NOAA
Pre-operational Automated Tropical Cyclone Detection

TROPICAL CYCLONE DETECTION

31
 Improved prediction of TC rapid intensification by 200% over operational consensus

RAPID INTENSIFICATION PREDICTION

32
 Neural net maps IFS forecast temperature to ground truth using satellite occultation

MODEL BIAS-CORRECTION
33
 AI IMPROVES ENSEMBLE FORECASTS
(2021) AI predictions with 5 ensemble members surpass forecast skill with 10 members

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2020.0092

REDUCING ENSEMBLE COST

34
 nature geoscience article

FILLING-IN MISSING CLIMATE OBSERVATIONS

35
 EMULATION

SAMPLES FROM
EXPENSIVE FUNCTION

FAST
SURROGATE
MODEL

Emulation can convert expensive calculations into real-time operations

ACCELERATION VIA EMULATION

36
 Emulation of E3SM super-parameterized SW and LW radiation gives 10x speedup

FASTER PHYSICAL PARAMETERIZATIONS

37
 ACCELERATION OF ATMOSPHERIC CHEMISTRY
NVIDIA Collaboration with Newcastle University

Surrogate model for Chemical Transport Mechanism

Modelling hundreds of chemical species & reaction rates

High Quality Training data from GEOS-Chem:

• Global, high-resolution (0.5° x 0.625°)

Currently using ConvLSTM approach

Planning to test with AFNO/SFNO DNNs

FASTER PHYSICAL PARAMETERISATIONS

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 38
 Stable, accurate data-driven physical parameterization built directly from high resolution data

NEW PHYSICAL PARAMETERIATIONS

39
 PHYSICS INFORMED NEURAL NETS
Can solve PDEs, or constrain data driven models

https://maziarraissi.github.io/PINNs

PHYSICS INFORMED NEURAL NETWORKS (PINNS)

40
 CLOUD MODELING USING PHYSICS INFORMED NEURAL NETWORKS
NVIDIA Collaboration with IIT-Delhi

Data driven approach to modelling cloud microphysics

• Entrainment/Detrainment rate

ROM approach with a trained DNN

High Quality Training data from multiple sources:

• OpenFOAM, WRF, CM1, Internal model (Matlab)

Testing Tensor Based Neural Network (TBNN)

Also plan to test NVIDIA Modulus

MODELLING CLOUDS WITH PINNS

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 41
DATA DRIVEN AI MODELS
42
Terabytes of

Millions of
“lines of code”

Millions of lines of GPU-accelerated “code”

Can be generated automatically in a < 1 day!
REAL-TIME TSUNAMI PREDICTION
44
https://en.wikipedia.org/wiki/2011_T%C5%8Dhoku_earthquake_and_tsunami
 Fujitsu Leverages ‘Fugaku’ and AI to Deliver Real-Time Tsunami Prediction

REAL-TIME TSUNAMI PREDICTION

45
https://www.fujitsu.com/global/about/resources/news/press-releases/2021/0216-01.html
 MetNet produces more accurate precipitation nowcasts out to 8 hours.

DATA-DRIVEN NOWCASTING
46
 Data-driven model matches IFS model performance for medium-range forecasts, at 5° resolution

MEDIUM-RANGE FORECASTING
47
 (June 2021) DLWP is comparable ECMWF at 2 week timescales

SUBSEASONAL FORECASTING (S2S)

48
publication
 FourCastNet
Full model emulation of ERA5

Scope Global, Medium-Range Weather

Model Type Full-Atmosphere AI Surrogate
Architecture Fourier Neural Operator + Transformer
Resolution 25km
Training Data ERA5 Reanalysis (1979 – Present)
Initial Condition ERA5 / GFS / UFS
Training Time O(1000) GPU-hours (NVIDIA A100)
Inference Time 0.5 sec (2-week forecast)
Calibration IC + Bayesian model uncertainty
Speedup vs NWP O(10,000 – 100,000)
Power Savings O(10,000)
Max Stable Rollout 250+ days
Project Type Open-source
FCN is fully data-driven.
Atmospheric dynamics were learned from data alone.
 FCNNet
FourCast demonstrated thatweat
: A dat a- driven data-driven models
her predict or ofcould work high
unusually at full resolution
resolut ion

FourCast Net , Pat hak et al. (2022), 0.25°, ~1,000,000 Pixels, ViT+AFNO

GNN, Keisler et al. (2022), 1°, 64,000 Pixels, Graph Neural Net works

DLWP, Weyn et al. (2020). 2°, 16K pixels, Deep CNN on Cubesphere/ (2021) ResNet

Weyn et al. (2019), 2.5° N.H only, 72x36, 2.6k pixels, ConvLSTM

Weat herBench, Rasp et al. (2020). 5.625°, 64x32, 2K pixels, CNN

Deuben & Bauer (2018), 6° , 60x30, 1.8K pixels, MLP

 FourCastNet Details
• Data-driven AI surrogate
• 6-hour auto-regressive step
• AFNO or SFNO architecture
• Era5 reanalysis data at 0.25°
• Transformer based model
• FNO for spatial mixing
• 20+ channels (2d shells)
• Skill increases with channels
• Scales to > 4000 GPUs
• Training set: 1979 to 2015
• Validation set: 2016, 2017
• Held out: 2018 onwards
FCN skill now matches or exceeds that of IFS (as measured by ACC and RMSE)
Full model emulation is lightning fast
and extremely power efficient.
 Extreme
winds

1000+ member ensembles reveal the likelihood of dangerous extremes.

 There are now several highly accurate data-driven forecast models at 25km resolution.

https://github.com/NVlabs/FourCastNet https://github.com/198808xc/Pangu-Weather No open-source code available

Intermission
NVIDIA Earth-2 Digital Twin Initiative
Jeff Adie, Principal Solutions Architect, NVAITC
Digital Twins
PHYSICAL SYSTEM DIGITAL TWIN

A digital twin is a virtual replica of a real-world system

 Digital twins allow us to
safely explore the
impacts of our decisions
 NVIDIA makes digital twins fast, powerful, and intuitive

OMNIVERSE NUCLEUS AI FOUNDATIONS

USER EXPERIENCE INTEROPERABILITY AI MODEL-MAKING SERVICES

GRACE - HOPPER SUPERCHIP OVX - SUPERPOD DGX CLOUD and OMNIVERSE CLOUD
ACCELERATED COMPUITING LARGE AI MODELS Access to all NVIDIA technologies from any browser
Omniverse enables physically
accurate interactive 3d environments
The NVIDIA Earth-2 Initiative
 NVIDIA Earth-2 Announcement – GTC Nov 2021
NVIDIA to build Earth-2 Supercomputer to see our future

November 12, 2021 by JENSON HUANG https://blogs.nvidia.com/blog/2021/11/12/earth-2-supercomputer/

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
 EARTH-2: Begins as a vision
Of a highly interactive climate information system for serving society with next-gen climate predictions.
Imagine you could Select a Region of the Planet...
… Ask Questions about Climate Change's Impacts...
On Food, Health, Infrastructure, Energy systems, and more...
 … and Receive Useful, Visual & Statistical Guidance?
From a Highly Interactive Future Climate Information System, at High Resolution, that Serves Society…

Earth Twin Earth

Earth-2 Mission #1

Interacting with Climate

Predictions at Low Latency.
Imagine the System Evolving in Computational Fidelity and Ambition
Eventually fed by a new library of climate predictions so high-resolution they seem impossible today.

Earth Twin Earth

Earth-2 Mission #2

Achieving Next-Gen
Climate Predictions using
Hybrid Physics, Machine
Learning & HPC.
Earth-2 will predict extreme weather and climate across many possible
futures

https://www.nature.com/articles/d41586-022-01913-
7

Extreme Weather Climate Change Impacts

 The Earth-2 platform will couple digital-twins at every scale

Crop Modelling Twin Water Management Twin City Planning Twin

Detailed statistical predictions will help us to plan and adapt

Renewable Energy Water Desalinization Green Cities

 The World is Already Working Hard on these Problems
Example: Project Destination Earth envisions what Earth-System Modeling could be.

Data-driven Models Storm-Resolving Models Unified Observations Exascale Compute

Earth-2 is to become NVIDIA’s Digital Twin of the Earth
Including all software, hardware, data, and observations needed to make it happen

O P E N P L AT F O R M

D E D I C AT E D
SUPERCOMPUTER
H A R D WA R E - S O F T WA R E
CO-DESIGN

AI FOCUSED

R E G I O N A L D E TA I L

OMNIVERSE UI

NUCLEUS INTEROP

CONTINUOUS LEARNING
Earth-2 Began by Exploring
Data-Driven Weather
Prediction

FourCastNet

• Scope Global, Medium Range

• Model Type Full-Model AI Surrogate
• Architecture AFNO (Adaptive Fourier Neural Op.)
• Resolution: 25km
• Training Data: ERA5 Reanalysis
• Initial Condition GFS / UFS
• Inference Time 0.25 sec (2-week forecast)
• Speedup vs NWP O(104-105)
• Power Savings O(104)
 From Weather to Climate

Sample Weather Ensembles Over Time Sample across many possible climate futures
ECMWF IPCC 2014
 Given Future Data, FourCastNet's Speed Allows Fast Tethering
AI nimbly generates details between "checkpoints" saved only infrequently from physics-based climate simulations
-- Bjorn Stevens, GTC 2021
 To Begin, we can Tether to Existing Climate Predictions.
Using the world's current data library of 100-km resolution intergovernmental climate predictions.

Sampling scenarios
Physical projections of
forced change &
Mitigation natural variability Renewable
Energy
Forecasting
Population

Data-driven Extreme
Digital Weather
Tethering. Twin Prediction
Pollution

Disaster
Mitigation
Land-use

MONITORING | MULTI-MODAL DATA FUSION | CALIBRATION PHYSICS-ML MODEL TRAINING & INFERENCE INTERACTIVITY | VISUALIZATION | ANALYSES

RAPIDS | CuNumeric DALI | Modulus | CUDA-X AI | TRITON | TensorRT Omniverse

 But Eventually we want to Tether to High-Resolution Predictions.
Because credible cloud feedbacks and storm dynamics from km-scale simulators matter to predicting regional risk.
 Physics-based models are still the only source of future climate data

Interactive Digital Twin Full model emulation Physics Based Predictions (AI enhanced)

81
 Earth-2 Climate Initiatives
We are pursuing various strategies to improve our climate digital twin

CMIP-6 Initialization or Surrogate ICON 1km tethering OVX hardware / software co-design
Source: Fuss et al., 2014 source: dyamond proj ovx superpods

1km → 5 km AI sub-grid emulation Regional Fine-Tuning and Downscaling Model Auto-Calibration via RL
source: nature.com source source: nature
 Earth-2 will be in Collaboration with International Climate Science
NVIDIA's visualization, engineering, AI & full-stack expertise complement research capacity in academia & government.
 FCN's speed changes everything
Infinite ensembles in seconds, 45,000x less compute, 12,000x less energy.
 SUMMARY

• Modelling our planet is hard. Climate and weather are exascale

class problems

• Improving our projection accuracy is critical to help fight climate

change

• AI can provide new methods to accelerate this process through

data-driven design

• Combining classical mod/sim with AI (Hybrid modelling) gives the

best of both worlds

• There is no plan B for planet earth!

85
Thank you! Questions?