Introduction

Introduction

UQ Software

PCES

References

Polynomial Chaos based uncertainty propagation

Lecture 1: Uncertainty and Spectral Expansions

Bert Debusschere , Cosmin Safta, Khachik Sargsyan, Habib Najm

bjdebus@sandia.gov

Sandia National Laboratories, Livermore, CA KUL UQ Summer School

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Acknowledgement

Omar Knio Roger Ghanem Olivier Le Matre and many others ...

Duke University, Raleigh, NC University of Southern California, Los Angeles, CA LIMSI-CNRS, Orsay, France

This work was supported by:

US Department of Energy (DOE), Ofce of Advanced Scientic Computing Research (ASCR), Scientic Discovery through Advanced Computing (SciDAC) and Applied Mathematics Research (AMR) programs.

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energys National Nuclear Security Administration under contract DE-AC04-94AL85000.

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Goals of this tutorial

An overview of the key aspects of uncertainty quantication 4 Lectures Lecture 1: Context and Fundamentals
Introduction to uncertainty The various aspects of UQ Software for UQ Spectral representation of random variables

Lecture 2: Forward Propagation Lecture 3: Characterization Lecture 4: Advanced Topics

Please do not hesitate to ask questions!!

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Outline

Introduction The Many Aspects of Uncertainty Quantication Software for Uncertainty Quantication Spectral Representation of Random Variables References

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Aspects of Uncertainty Quantication

Enabling Predictive Simulation Types of uncertainty

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Predictive simulations enable science-based design

Emperical design is inefcient and costly

Trial and error does not work well for complex systems

Experiments not always feasible or permissible

Reliability of nuclear weapons Climate change mitigation approaches

Predictive simulations provide insight into the underlying physics that drive complex systems
Identication of key mechanisms Allows for rigorous optimization strategies

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Model validation requires targeted experiments

Model validation requires targeted experiments

MEAN ! RMS !
Comparisons with 1D Raman/Rayleigh/CO-LIF line images (Barlow et al. ) !

Mixture Fraction!

Temperature!

H2O Mass Fraction! CO Mass Fraction!

x/d = 10!

Experiment!
DLR-A Flame: Red = 15,200 !
Fuel: 22.1% CH4, 33.2% H2, 44.7% N2! Coow: 99.2% Air, 0.8% H2O ! Detailed Chemistry and Transport: 12-Step Mechanism (J.-Y. Chen, UC Berkeley) !

LES!

Large Eddy Simulation (LES) validation on turbulent ame (courtesy J. Oefelein, SNL)
Debusschere SNL UQ

Introduction

Introduction

UQ Software

PCES

References

Predictive simulation requires careful assessment of all sources of error and uncertainty

Numerical errors Grid resolution Time step Time integration order Spatial derivative order Tolerance on iterative solvers Condition number of matrices Uncertainties Initial and boundary conditions Model parameters Model equations Coupling between models Sampling noise in particle models Stochastic forcing terms

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Governing equations for LES simulation of multiphase reacting ow

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Constitutive models provide closure for governing equations

Smagorinsky sub-grid scale model

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Dynamic modeling and reacting ows involve additional complexity

Does additional complexity provide more accuracy and less uncertainty?

Debusschere SNL UQ

Introduction

Introduction

UQ Software

PCES

References

Experimental data is used to calibrate models

Experiments Predictive Simulation

Theory

Debusschere SNL

UQ

Introduction

du to enable UQ methods extract information from all Introduction sources Introduction UQ Software predictive PCES = f (u ; ) P (D|, I d)tP (, I ) P (|D, I ) = Formulation simulation P (D )
Introduction Introduction

Formulation

Introduction

= f (u ; ) UQ Software

du dt

PCES

References
Summary

Formulation
du = f (u ; ) dt

UQ Software PCES Summary Introduction Introduction Introduction Introduction

u = f (u ; ) d t Experiments P (u ) P ()du = P (D|, I )p(, I ) P (D|, I )P (, I ) f (u ; ) Predictive P (|D, I ) = P (|D, I ) = P (D|, I )p(, I ) dt P (D ) P (|D , P I )(D =) Simulation P (D ) P (D|, I )P (, I ) P (|D, I ) = u du P (D) = f (u ; ) dt du = u( ; ) d u u P (u ) f (P dt = f (u ; ) Debusschere SNL UQ Forward P (D|, I )P (, I ) dt P (|D, I ) = P (D|, I )p(, IPropagation ) P (D ) P ( u ) P ( ) P (|D, I ) = P (D|, I )p(, I ) Debusschere SNL UQ Theory P (D ) P (|D, I ) = P u(D)
du P (u ) P () dt Inference = f (u ; )
Debusschere SNL UQ UQ not just about propagating uncertainties
Debusschere SNL UQ

du

Formulation Formulation

P (|D, I ) = u

P (D|, I )P (, I ) References Software PUQ (D )UQ Software

PCES PCES

Sum

P (u ) P ()

The term UQ covers a wide range of methods

Debusschere SNL UQ

Debusschere SNL

UQ

Debusschere SNL Debusschere UQ SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

UQ assesses condence in model predictions and allows resource allocation for delity improvements

Parameter inference Determine parameters from data Characterize uncertainties in inferred parameters Propagate input uncertainties through computational models Account for uncertainty from all sources Resolve coupling between sources Analysis Sensitivity analysis Attributions Model calibration, validation, selection, averaging

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Types of uncertainty

Epistemic uncertainty Variable has one particular value, but we dont know what it is Reducible: by taking more measurements, we can get to know the value of the variable better Examples
The mass of the planet Neptune Temperature of the ocean at a particular point and time

Aleatory uncertainty Intrinsic or inherent uncertainty: variable is random; different value each time it is observed Irreducible: taking more measurements will not reduce uncertainty in the value of the variable Examples:
Collisions interactions in molecular systems Sampling noise

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Viewpoints matter

Epistemic versus aleatory sometimes depends on scale level In fully resolved turbulent simulations, velocity eld near a wall is deterministic In mesoscale channel ow, near wall velocity eld is turbulent forcing term (aleatory) In macroscale channel ow, near wall velocity eld provides friction term (epistemic) These lectures follow the Bayesian view: probability represents the

degree of belief in the value of a variable

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Software for Uncertainty Quantication

UQ in computational sciences is maturing Algorithms and research codes are transitionting into engineering

sciences software

US DOE puts high importance on the role of UQ, especially as scientic

research moves into extreme scale

QUEST: Quantication of Uncertainty in Extreme Scale Simulations DOE SciDAC institute: http://quest-scidac.org/ One of the key goals is to make extreme scale UQ approaches available to the scientic community Various UQ methods offered under 4 software products
DAKOTA QUESO GPMSA UQTk

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

UQ Tools DAKOTA: Design Analysis Kit for Optimization and Terascale Applications

http://dakota.sandia.gov/ Large-Scale Optimization, non-intrusive UQ Model calibration, global sensitivity analysis Design of Experiments solution verication, and parametric studies Over 4000 download registrations

govt, industry, academia

Generic interface to black-box application model Java front end Used in a wide variety of applications GNU LPGL license

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

UQ Tools QUESO

MPI/C++ library Large-scale inverse UQ Statistical algorithms for Bayesian inference model calibration, model validation, decision making under uncertainty Parallel multi-chain MCMC Being integrated with DAKOTA

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

UQ Tools UQTk: Uncertainty Quantication Toolkit

http://www.sandia.gov/UQToolkit/ Lightweight C++ library Intrusive Polynomial Chaos UQ methods Non-intrusive sparse-quadrature UQ Bayesian inference Custom representations of uncertain information Well suited for prototyping, tutorial/training purposes

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

UQ Tools GPMSA: Gaussian Process Models for Simulation Analysis

Serial matlab code C++ conversion plans Integration with DAKOTA Bayesian inference Gaussian process surrogates Global sensitivity analysis, forward UQ Model calibration/parameter estimation Statistical models to characterize model discrepancy or structural model

error

Prototyping tool

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Polynomial Chaos Expansions (PCEs)

Representation of random variables with PCEs Convergence Galerkin projection Basis types

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

One-Dimensional Hermite Polynomials

0 (x ) k (x ) = = 1 (1)k ex
2

/2

dk x 2 /2 e , dx k

k = 1, 2, . . .

1 ( x ) = x ,

The Hermite polynomials form an orthogonal basis over [, ] with respect to the inner product 1 i j 2

2 (x ) = x 2 1,

3 (x ) = x 3 3 x , . . .

i (x )j (x )w (x )dx = ij i2

where w (x ) is the weight function w (x ) = ex Note that

x /2 e 2 2 2

/2

is the density of a standard normal random variable

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

One Dimensional Polynomial Chaos Expansion

Consider:
P

u=
k =0

u k k ( )

u : Random Variable (RV)

represented with 1D PCE

uk : PC coefcients (deterministic) k : 1D Hermite polynomial of

Prob. Dens. [-]

4 3 2 1 0.5 1.0

0 0.0

order k

1.5

2.0

: Gaussian RV

u = 0.5 + 0.21 ( ) + 0.12 ( )

A random quantity is represented with an expansion consisting of functions of random variables multiplied with deterministic coefcients
Set of deterministic PC coefcients fully describes RV Separates randomness from deterministic dimensions

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Multidimensional Hermite Polynomials

The multidimensional Hermite polynomial i (1 , . . . , n ) is a tensor product of i i i the 1D Hermite polynomials, with a suitable multi-index i = (1 , 2 , . . . , n ),
n

i (1 , . . . , n ) For example, 2D Hermite polynomials: i 0 1 2 3 4 5 ... p 0 1 1 2 2 2 ...

=
k =1

i (k )
k

i 1 1 2 2 1 1 1 2 2 2 1 ...

i (0,0) (1,0) (0,1) (2,0) (1,1) (0,2) ...

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Multidimensional Inner Products Orthogonality

i j

...

i ()j ()g (1 )g (2 ) g (n )d1 d2 dn

=
k =1

i (k )j (k ) = ij 2 i
k k

where, such that,

P

g ( )

e /2 2

u=
k =0

uk k

i u =
k =0

uk i k = ui 2 i

ui =

u i 2 i

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Multidimensional Polynomial Chaos Expansion

Consider: u=
k =0 P

uk k (1 , . . . , n )

u : Random Variable (RV) represented with multi-D PCE uk : PC coefcients (deterministic) k : Multi-D Hermite polynomials up to order p i : Gaussian RV n: Dimensionality of stochastic space P + 1: Number of PC terms: P + 1 =
(n+p)! n!p!

The number of stochastic dimensions represents the number of independent inputs, degrees of freedom that affect the random variable u
E.g. one stochastic dimension per uncertain model parameter Contributions from each uncertain input can be identied Compact representation of a random variable and its dependencies
Debusschere SNL UQ

Introduction

Introduction

UQ Software

PCES

References

Generalized Polynomial Chaos

PC Type Gauss-Hermite Legendre-Uniform Gamma-Laguerre Beta-Jacobi Domain (, +) [1, 1] [0, +) [1, 1] Density w ( )
1 e 2 2 1 2 x e (+1) (1+) (1) 2+ +1 B (+1, +1)
2

Polynomial Hermite Legendre Laguerre Jacobi

Free parameters none none > 1

> 1, > 1

Inner product: i j

b a

i ( ) j ( ) w ( ) d

Wiener-Askey scheme provides a hierarchy of possible continuous PC

bases, see Xiu and Karniadakis, SISC, 2002.

domain

Legendre-Uniform PC is a special case of Beta-Jacobi PC Beta-Jacobi PC allows tailored accuracy of the PC representation across the

Input parameter domain often dictates the most convenient choice of PC Polynomials functions can also be tailored to be orthogonal w.r.t.

chosen, arbitrary density

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Gamma Distribution
1.0 0.8 0.6 PDF 0.4 0.2 0.00
=0 =1 =2 =3 =4

8 x

10

12

14

16

Useful to represent quantities that are strictly positive Support on [0, +)

Debusschere SNL UQ

Introduction

Introduction

UQ Software

PCES

References

Beta Distribution
1.4 1.2 1.0 PDF 0.8 0.6 0.4 0.2 0.01.0 0.5 0.0 x 0.5 1.0
= =0.5 =5, =1 =1, =3 =2, =2 =2, =5

Useful to represent quantities that vary between set boundaries Can be tailored to give more importance to specic areas Support on [1, 1]
Debusschere SNL UQ

Introduction

Introduction

UQ Software

PCES

References

More Formally: Probability Spaces and Random Variables

Let (, S, P) be a probability space is an event space S is a -algebra on P is a probability measure on (, S) Random variables are functions X : R with a measure

corresponding to their image:

if X 1 (A) S, then dene (A) = P(X 1 (A)). p (x ) = d /dx : the density of the random variable X (with respect to Expectation: f =

Lebesgue measure on R). f d =

f p(x ) dx

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Convergence of PC expansions

General convergence theorems are subject of ongoing research Depends on the type of underlying random variables Wiener-Hermite Chaos has been well-studied Generalized PC less so Ernst et al. 2011: Let : RN such that for i = 1, . . . , N each i : R, be a set of random variables S( ): -algebra generated by the set of random variables L2 (, S( ), P): Hilbert space of real-valued random variables dened on (, S( ), P) with nite second moments Any random variable in this -algebra can be represented with a Polynomial Chaos expansion with germ Does not say that any RV with nite variance can be represented with a

PCE to arbitrary precision

In practice, one rarely knows how many degrees of freedom a RV has Also, RV specication is rarely complete Often, in an engineering sense, PCE of a given order is seen as a model

choice to represent what is known about a RV

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

How do I know my PCE is converged?

Approximation error in PCE is topic of a lot of research Rules of thumb: Higher order PC coefcients should decay Increase order until results no longer change Not always fail-proof ...

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Postprocessing / Analysis

Moments Plotting PDFs of RVs represented with PCEs

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Moments of RVs described with PCEs

P

u=
k =0

uk k ( )

Expectation: u = u0 Variance 2

= =

(u u )2
P

(
k =1 P

uk k ( ))2
P

=
k =1 j =1 P P

uj uk j ( )k ( ) uj uk j ( )k ( )
k =1 j =1 P 2 uk k ( )2

=
k =1

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Plotting PDFs of RVs corresponding to PCEs

u=
k =0

uk k ( )

Analytical formula formula for PDF(u ) exists Involves polynomial root nding, and is hard to generalize to multi-D PCE is cheap to sample Brute-force sampling and bin samples into histogram Use Kernel Density Estimation (KDE) to get smoother PDF with fewer samples ui PDF(u ) = 1 Ns h
Ns

K
i =1

u ui h
x2 2

K is the kernel, h is the bandwidth.

Gaussian KDE is commonly used with K (x ) = 1 e
2

, leading to

PDF(u ) =

1 2 Ns h

Ns

exp
i =1

(u ui )2 2h2

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

Comparison of histograms and KDE

Source Wikipedia, Drleft; licensed under the Creative Commons Attribution-ShareAlike 3.0 License

Bandwidth h needs to be chosen carefully to avoid oversmoothing

Debusschere SNL

UQ

Introduction

Introduction

UQ Software

PCES

References

KDE requires judicious choice of bandwidth h

0.45 0.40 0.35 0.30

True PDF w=w =0.16 w=2w w=w /2 w=w /4 binning

opt opt opt opt

PDF

0.25

0.20 0.15 0.10 0.05 0.00 5 4 3 2

Random variable

Bandwidth h needs to be chosen carefully to avoid oversmoothing Scotts rule-of-thumb is optimal for Gaussian r. v., h = N d +4 With smaller h, the KDE becomes more accurate, but more noisy Binning is similar to smaller h case, but it requires a good choice of Nbins
Debusschere SNL UQ 1

Introduction

Introduction

UQ Software

PCES

References

KDE requires judicious choice of bandwidth h

0.30 0.25 0.20

True PDF w=w =0.16 w=2w w=w /2 w=w /4 binning

opt opt opt opt

PDF

0.15

0.10 0.05 0.00 6 4 2 0 2 4 6

Random variable

Bandwidth h needs to be chosen carefully to avoid oversmoothing Scotts rule-of-thumb is optimal for Gaussian r. v., h = N d +4 With smaller h, the KDE becomes more accurate, but more noisy Binning is similar to smaller h case, but it requires a good choice of Nbins
Debusschere SNL UQ 1

Introduction

Introduction

UQ Software

PCES

References

KDE requires judicious choice of bandwidth h

0.30 0.25 0.20

True PDF w=w =0.16 w=2w w=w /2 w=w /4 binning

opt opt opt opt

PDF

0.15

0.10 0.05 0.00 10 5 0 5 10

Random variable

Bandwidth h needs to be chosen carefully to avoid oversmoothing Scotts rule-of-thumb is optimal for Gaussian r. v., h = N d +4 With smaller h, the KDE becomes more accurate, but more noisy Binning is similar to smaller h case, but it requires a good choice of Nbins
Debusschere SNL UQ 1

Introduction

Introduction

UQ Software

PCES

References

Further Reading

N. Wiener, Homogeneous Chaos", American Journal of Mathematics, 60:4, pp. 897-936, 1938. M. Rosenblatt, Remarks on a Multivariate Transformation", Ann. Math. Statist., 23:3, pp. 470-472, 1952. R. Ghanem and P. Spanos, Stochastic Finite Elements: a Spectral Approach", Springer, 1991. O. Le Matre and O. Knio, Spectral Methods for Uncertainty Quantication with Applications to Computational Fluid Dynamics", Springer, 2010. D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach", Princeton U. Press, 2010. O.G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann, On the convergence of generalized polynomial chaos expansions, ESAIM: M2AN, 46:2, pp. 317-339, 2011. D. Xiu and G.E. Karniadakis, The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations", SIAM J. Sci. Comput., 24:2, 2002. Le Matre, Ghanem, Knio, and Najm, J. Comp. Phys., 197:28-57 (2004) Le Matre, Najm, Ghanem, and Knio, J. Comp. Phys., 197:502-531 (2004) B. Debusschere, H. Najm, P. Pbay, O. Knio, R. Ghanem and O. Le Matre, Numerical Challenges in the Use of Polynomial Chaos Representations for Stochastic Proecesses", SIAM J. Sci. Comp., 26:2, 2004. S. Ji, Y. Xue and L. Carin, Bayesian Compressive Sensing", IEEE Trans. Signal Proc., 56:6, 2008. K. Sargsyan, B. Debusschere, H. Najm and O. Le Matre, Spectral representation and reduced order modeling of the dynamics of stochastic reaction networks via adaptive data partitioning". SIAM J. Sci. Comp., 31:6, 2010. K. Sargsyan, B. Debusschere, H. Najm and Y. Marzouk, Bayesian inference of spectral expansions for predictability assessment in stochastic reaction networks". J. Comp. Theor. Nanosc., 6:10, 2009.

Debusschere SNL

UQ