This action might not be possible to undo. Are you sure you want to continue?
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
An adaptive hierarchical sparse grid
An adaptive hierarchical sparse grid
collocation algorithm for the solution of
collocation algorithm for the solution of
stochastic differential equations
stochastic differential equations
Nicholas Zabaras and Xiang Ma
Materials Process Design and Control Laboratory
Sibley School of Mechanical and Aerospace Engineering
101 Frank H. T. Rhodes Hall
Cornell University
Ithaca, NY 148533801
Email: zabaras@cornell.edu
URL: http://mpdc.mae.cornell.edu/
Symposium on `Numerical Methods for Stochastic Computation and Uncertainty Quantification'
in the 2008 SIAM Annual Meeting, San Diego CA, July 711, 2008
2 2
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Outline of the presentation
o Stochastic analysis
o Issues with gPC and MEgPc
o Collocation based approach
o
Basics of stochastic collocation
o
Issues with existing stochastic sparse grid collocation method
(Smolyak algorithm)
o
Adaptive sparse grid collocation method
o Some numerical examples
3 3
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Motivation
All physical systems have an inherent associated randomness
SOURCES OF UNCERTAINTIES
•Multiscale material information –
inherently statistical in nature.
•Uncertainties in process conditions
•Input data
•Model formulation – approximations,
assumptions.
Why uncertainty modeling ?
Assess product and process reliability.
Estimate confidence level in model predictions.
Identify relative sources of randomness.
Provide robust design solutions.
Engineering
component
Heterogeneous
random
Microstructural
features
Process
Control?
4 4
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Problem definition
Define a complete probability space . We are interested to find a
stochastic function such that for Palmost everywhere (a.e.) ,
the following equation hold:
( ) , , F P Ω
: u D Ω× →¡ ω∈Ω
( ) ( )
( ) ( )
, ; , ,
, ,
L u f D
B u g D
ω ω · ∀ ∈
· ∀ ∈∂
x x x
x x x
where are the coordinates in , L is (linear/nonlinear) differential
operator, and B is a boundary operators.
( )
1 2
, , ,
d
x x x · x K
d
¡
In the most general case, the operator L and B as well as the driving terms f and
g, can be assumed random.
In general, we require an infinite number of random variables to completely
characterize a stochastic process. This poses a numerical challenge in modeling
uncertainty in physical quantities that have spatiotemporal variations, hence
necessitating the need for a reducedorder representation.
5 5
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Representing Randomness
S
Sample space of
elementary events
Real line
Random
variable ξ
MAP
Collection of all
possible outcomes
Each outcome is
mapped to a
corresponding real
value
Interpreting random variables as functions
A general stochastic process is a random field
with variations along space and time – A
function with domain (Ω, Τ, S)
1. Interpreting random variables
2. Distribution of the random variable
E.g. inlet velocity, inlet temperature
( )
1 0.1
o
θ θ ξ · +
3. Correlated data
Ex. Presence of impurities, porosity
Usually represented with a correlation function
We specifically concentrate on this.
6 6
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Representing Randomness
1. Representation of random process
 KarhunenLoeve, Polynomial Chaos
expansions
2. Infinite dimensions to finite dimensions
 depends on the covariance
KarhunenLoèvè expansion
Based on the spectral decomposition of the
covariance kernel of the stochastic process
Random
process
Mean
Set of random
variables to
be found
Eigenpairs of
covariance
kernel
• Need to know covariance
• Converges uniformly to
any second order process
Set the number of stochastic dimensions, N
Dependence of variables
Pose the (N+d) dimensional problem
7 7
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
KarhunenLoeve expansion
1
( , , ) ( , ) ( , ) ( )
i i
i
X x t X x t X x t Y ω ω
∞
·
· +
∑
Stochastic
process
Mean
function
ON random variables
Deterministic functions
Deterministic functions ~ eigenvalues, eigenvectors of
the covariance function
Orthonormal random variables ~ type of stochastic
process
In practice, we truncate (KL) to first N terms
1
( , , ) fn( , , , , )
N
X x t x t Y Y ω · K
8 8
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
The finitedimensional noise assumption
By using the DoobDynkin lemma, the solution of the problem can be described
by the same set of random variables, i.e.
( ) ( ) ( ) ( )
1
, , , ,
N
u u Y Y ω ω ω · x x L
So the original problem can be restated as: Find the stochastic function
such that
: u D Γ× →¡
( ) ( )
( ) ( )
, ; , ,
, ; , ,
L u f D
B u g D
· ∀ ∈ ×Γ
· ∀ ∈∂ ×Γ
x Y x Y x
x Y x Y x
In this work, we assume that are independent random variables with
probability density function . Let be the image of . Then
( ) ( )
1
,
N
i i
i
Y ρ ρ
·
· ∀ ∈Γ
∏
Y Y
( ) { ¦
1
N
i
i
Y ω
·
i
ρ
i
Γ
i
Y
is the joint probability density of with support ( )
1
, ,
N
Y Y · Y L
1
N
N
i
i ·
Γ ≡ Γ ∈
∏
¡
9 9
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Uncertainty analysis techniques
MonteCarlo : Simple to implement, computationally
expensive
Perturbation, Neumann expansions : Limited to small
fluctuations, tedious for higher order statistics
Sensitivity analysis, method of moments : Probabilistic
information is indirect, small fluctuations
Spectral stochastic uncertainty representation: Basis in
probability and functional analysis, can address second
order stochastic processes, can handle large fluctuations,
derivations are general
Stochastic collocation: Results in decoupled equations
10 10
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Spectral stochastic presentation
: ( , , ) X x t Ω a ¡
A stochastic process = spatially, temporally varying
random function
CHOOSE APPROPRIATE
BASIS FOR THE
PROBABILITY SPACE
HYPERGEOMETRIC ASKEY POLYNOMIALS
PIECEWISE POLYNOMIALS (FE TYPE)
SPECTRAL DECOMPOSITION
COLLOCATION, MC (DELTA FUNCTIONS)
GENERALIZED POLYNOMIAL
CHAOS EXPANSION
SUPPORTSPACE
REPRESENTATION
KARHUNENLOÈVE
EXPANSION
SMOLYAK QUADRATURE,
CUBATURE, LH
11 11
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Generalized polynomial chaos (gPC)
Generalized polynomial chaos expansion is used to
represent the stochastic output in terms of the input
0
( , , ) ( , ) ( ( ))
P
i i
i
Z x t Z x t Y ω ψ ω
·
·
∑
Stochastic
output
Askey polynomials in input
Deterministic functions
Stochastic input
1
( , , ) fn( , , , , )
N
X x t x t Y Y ω · K
Askey polynomials ~ type of input stochastic process
Usually, Hermite, Legendre, Jacobi etc.
12 12
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Generalized polynomial chaos (gPC)
Advantages:
1) Fast convergence
1
.
2) Very successful in solving various problems in solid and fluid
mechanics – well understood
2,3
.
3) Applicable to most random distributions
4,5
.
4) Theoretical basis easily extendable to other phenomena
1. P Spanos and R Ghanem, Stochastic finite elements: A spectral approach, SpringerVerlag (1991)
2. D. Xiu and G. Karniadakis, Modeling uncertainty in flow simulations via generalized polynomial chaos
, Comput. Methods Appl. Mech. Engrg. 187 (2003) 137167.
3. S. Acharjee and N. Zabaras, Uncertainty propagation in finite deformations  A spectral stochastic
Lagrangian approach, Comput. Methods Appl. Mech. Engrg. 195 (2006) 22892312.
4. D. Xiu and G. Karniadakis, The WienerAskey polynomial chaos for stochastic differential equations,
SIAM J. Sci. Comp. 24 (2002) 619644.
5. X. Wan and G.E. Karniadakis, Beyond WienerAskey expansions: Handling arbitrary PDFs, SIAM J
Sci Comp 28(3) (2006) 455464.
13 13
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Generalized polynomial chaos (gPC)
Disadvantages:
1) Coupled nature of the formulation makes implementation nontrivial
1
.
2) Substantial effort into coding the stochastics: Intrusive
3) As the order of expansion (p) is increased, the number of unknowns
(P) increases factorially with the stochastic dimension (N):
combinatorial explosion
2,3
.
4) Furthermore, the curse of dimensionality makes this approach
particularly difficult for large stochastic dimensions
1. B. Ganapathysubramaniana and N. Zabaras,
Sparse grid collocation schemes for stochastic natural convection problems, J. Comp Physics,
2007, doi:10.1016/j.jcp.2006.12.014
2. D Xiu, D Lucor, C.H. Su, and G Karniadakis,
Performance evaluation of generalized polynomial chaos, P.M.A. Sloot et al. (Eds.): ICCS 2003,
LNCS 2660 (2003) 346354
3. B. J. Debusschere, H. N. Najm, P. P. Pebray, O. M. Knio, R. G. Ghanem and O. P. Le Maître,
Numerical Challenges in the Use of Polynomial Chaos Representations for Stochastic Processes,
SIAM Journal of Scientific Computing 26(2) (2004) 698719.
( )
!
1
! !
N p
P
N p
+
+ ·
14 14
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of gPC
Gibbs phenomenon: Due to its global support, it fails to capture the
stochastic discontinuity in the stochastic space.
X
Y
0 0.25 0.5 0.75 1
0
0.25
0.5
0.75
1
9.2E07 5.7E06 1.0E05 1.5E05 2.0E05 2.5E05 3.0E05 3.4E05
X
Y
0 0.25 0.5 0.75 1
0
0.25
0.5
0.75
1
6.4E08 4.3E07 9.3E07 1.4E06 1.9E06 2.4E06 2.9E06 3.4E06
X

v
e
l
o
c
i
t
y
Y

v
e
l
o
c
i
t
y
Capture unsteady
equilibrium in
natural convection
Therefore, we need some method which can resolve the
discontinuity locally. This motivates the following Multi  element
gPC method.
15 15
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Multielement generalized polynomial chaos (MEgPC)
Basic concept:
Decompose the space of random inputs into small elements.
In each element of the support space, we generate a new random
variable and apply gPCE.
The global PDF is also simultaneously discretized. Thus, the global PC
basis is not orthogonal with the local PDF in a random element. We
need to reconstruct the orthogonal basis numerically in each random
element. The only exception is the uniform distribution whose PDF is a
constant.
We can also decompose the random space adaptively to increase its
efficiency.
1. X. Wan and G.E. Karniadakis, An adaptive multielement generalized polynomial chaos method for
stochastic differential equations, SIAM J Sci Comp 209 (2006) 901928.
16 16
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Decomposition of the random space
B
Γ
1 2
,1 ,1 ,2 ,2 , ,
1
[ , ) [ , ) [ , ]
k k k k k k d k d
M
k k
k k
a b a b a b
B
φ
·
Γ · × × ×
¹
¹
· Γ · Γ
'
¹
Γ Γ ·
¹
L
U
I
where
1 2
, 1, 2 , k k M · K
We define a decomposition of as
Based on the decomposition , we define the following indicator
random variables:
1
0
k
I
Γ
¹
·
'
¹
,
k
∈Γ Y
otherwise
1, 2 , k M · K
B
Then a local random vector is defined in each element subject to
a conditional PDF
i
Y
i
Γ
( )
ˆ
(  1)
Pr( 1)
k
k
k
k k
f
f I
I
Γ
Γ
· ·
·
Y
Y
where
Pr( 1) 0
k
k B
J I · · >
17 17
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
An adaptive procedure
Let us assume that the gPC expansion of random field in element is
, ,
0
ˆ ˆ ( ) ( )
p
N
k k k i k i k
i
u u ψ
·
·
∑
ξ ξ
k
From the orthogonality of gPC, the local variance approximated by PC
with order is given by
p
2 2 2
, ,
1
ˆ
p
N
k p k i i
i
u σ ψ
·
·
∑
The approximate global mean and variance can be expressed by
u
2
σ
2 2 2
,0 , ,0
1 1
ˆ ˆ Pr( 1), [ ( ) ]Pr( 1)
k k
N N
k B k p k B
k k
u u I u u I σ σ
· ·
· · · + − ·
∑ ∑
Let denote the error of local variance. We can write the exact global
variance as
k
γ
2 2
1
Pr( 1)
k
N
k B
k
I σ σ γ
·
· + ·
∑
18 18
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
An adaptive procedure
We define the local decay rate of relative error of the gPC approximation
in each element as follows:
1
2 2
2 2
,
1
, , 1
2 2
, ,
ˆ
p
p
N
k i i
i N
k p k p
k
k p k p
u ψ
σ σ
η
σ σ
−
· +
−
−
· ·
∑
Based on and the scaled parameter , a random element will
be selected for potential refinement when the following condition is
satisfied
k
η
Pr( 1)
k
B
I ·
1
Pr( 1) , 0 1
k
k B
I
α
η θ α · ≥ < <
where and are prescribed constants. α
1
θ
Furthermore, in the selected random element , we use another
threshold parameter to choose the most sensitive random dimension.
We define the sensitivity of each random dimension as
2
θ
1
2 2
, ,
2 2
1
ˆ ( )
, 1, 2, , ,
ˆ
p
p
i p i p
i N
j j
j N
u
r i d
u
ψ
ψ
−
· +
· ·
∑
K
where we drop the subscript for clarity and the subscript denotes
the mode consisting only of random dimension with polynomial order
k
, i p
⋅
i
Y
p
k
19 19
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
An adaptive procedure
All random dimensions which satisfy
2 2
1,
max , 0 1, 1, 2, , ,
i j
j d
r r i d θ θ
·
≥ ⋅ < < ·
K
K
will be split into two equal random elements in the next time step
while all other random dimensions will remain unchanged.
Through this adaptive procedure, we can reduce the total element
number while gaining the same accuracy and efficiency.
If corresponds to element in the original random space , the
element and will be generated in the next level if the two
criteria are both satisfied.
k
Y [ , ] a b [ 1,1] −
[ , ]
2
a b
a
+
[ , ]
2
a b
b
+
20 20
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
An Example: KO problem
So it can successfully resolve the discontinuity in stochastic space.
However, due to its gPC nature in each element and the tree like adaptive
refinement, this method still suffers from the curse of dimensionality.
21 21
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
First summary
The traditional gPC method fails when there are steep gradients/finite
discontinuities in stochastic space.
gPC related method suffers from the so called curse of dimensionality
when the input stochastic dimension is large. High CPU cost for large
scale problems.
Although the hadaptive MEgPC can successfully resolve discontinuity in
stochastic space, the number of element grows fast at the order . So
the method is still a dimensiondependent method.
Therefore, an adaptive framework utilizing local basis functions that
scales linearly with increasing dimensionality and has the decoupled
nature of the MC method, offers greater promise in efficiently and
accurately representing highdimensional nonsmooth stochastic
functions.
(2 )
N
O
22 22
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Collocation based framework
The stochastic variable at a point in the domain is an N dimensional
function.
The gPC method approximates this N dimensional function using a
spectral element discretization (which causes the coupled set of
equations)
:
N
f → ¡ ¡
:
N
f → ¡ ¡
Spectral descretization represents the
function as a linear combination of sets of
orthogonal polynomials.
Instead of a spectral discretization, use
other kinds of polynomial approximations
Use simple interpolation!
That is, sample the function at some points
and construct a polynomial approximation
23 23
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Collocation based framework
Function value at any point is
simply
Stochastic function in 2 dimensions
Need to represent this function
Sample the function at a finite
set of points
Use polynomials (Lagrange
polynomials) to get a
approximate representation
:
N
f → ¡ ¡
( )
1
{ }
i M
M i ·
· Θ Y
( ) ( )
1
( ) ( ) ( )
M
i i
i
i
f f a
·
·
∑
Y Y Y I
( ) f ξ I
Spatial domain is approximated using a FE, FD, or FV discretization.
Stochastic domain is approximated using multidimensional
interpolating functions
24 24
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
From onedimension to higher dimensions
Denote the one dimensional interpolation formula as
with the set of support nodes:
In higher dimension, a simple case is the tensor product formula
For instance, if M=10 dimensions and we use k points in each direction
Number of points
in each direction,
k
Total number of
sampling points
2 1024
3 59049
4 1.05x10
6
5 9.76x10
6
10 1x10
10
This quickly becomes impossible to use.
One idea is only to pick the
most important points from the
tensor product grid.
25 25
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Conventional sparse grid collocation (CSGC)
1D sampling
2D sampling
In higher dimensions, not all the points are
equally important.
Some points contribute less to the
accuracy of the solution (e.g. points where
the function is very smooth, regions where
the function is linear, etc.). Discard the
points that contribute less: SPARSE GRID
COLLOCATION
Tensor
product
26 26
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Conventional sparse grid collocation
Both piecewise linear as well as polynomial functions, L, can be used to
construct the full tensor product formula above.
The Smolyak construction starts from this full tensor product
representation and discards some multiindices to construct the minimal
representation of the function
Define the differential interpolant, ∆ as the difference between two
interpolating approximations
with
1 i i i
U U
−
∆ · −
0
0 U ·
where , is the multiindex representation of the support
nodes. N is the dimension of the function f and q is an integer (q>N).
k = qN is called the depth of the interpolation. As q is increased more
and more points are sampled.
1
,
( ) ( )( )
N
i i
q N
q
A f f
≤
· ∆ ⊗ ∆
∑
i
L
The sparse grid interpolation of the function f is given by
1 d
i i · + + i L
27 27
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Conventional sparse grid collocation
The interpolant can be represented in a recursive manner as
1 1
, 1,
( ) ( )( ) ( ) ( )( )
N N
i i i i
q N q N
q q
A f f A f f
−
≤ ·
· ∆ ⊗ ∆ · + ∆ ⊗ ∆
∑ ∑
i i
L L
with
1,
0
N N
A
−
·
Can increase the accuracy of the interpolant based on Smolyak’s
construction WITHOUT having to discard previous results.
Can build on previous interpolants due to the recursive formulation
To compute the interpolant , one only needs the function values at
the sparse grid point given by:
, q N
A
1
,
1  
( ... )
N
i i
q N
q N q
H X X
− + ≤ ≤
· × ×
i
U
One could select the sets , in a nested fashion such that
i
X
1 i i
X X
+
⊂
Similar to the interpolant representation in a recursive form, the
sparse grid points can be represented as:
with
1
,
 
( ... )
M
i i
q M
q
H X X
∆ ∆
·
∆ · × ×
i
U
and
1
\
i i i
X X X
−
∆
·
, 1, , q M q M q M
H H H
−
· ∆ U
28 28
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Choice of collocation points and nodal basis functions
One of the choice is the ClenshawCurtis grid at nonequidistant extrema
of the Chebyshev polynomials. ( D.Xiu etc, 2005, F. Nobile etc, 2006)
By doing so, the resulting sets are nested. The corresponding univariate
nodal basis functions are Lagrangian characteristic polynomials:
There are two problems with this grid:
(1) The basis function is of global support, so it will fail when there is
discontinuity in the stochastic space.
(2) The nodes are predetermined, so it is not suitable for implementation
of adaptivity.
29 29
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Choice of collocation points and nodal basis functions
Therefore, we propose to use the NewtonCotes grid using equidistant
support nodes and use the linear hat function as the univariate nodal
basis.
It is noted that the resulting grid points are also nested and the grid has
the same number of points as that of the ClenshawCurtis grid.
Comparing with the Lagrangian polynomial, the piecewise linear hat
function has a local support, so it is expected to resolve discontinuities in
the stochastic space.
The Ndimensional multilinear basis functions can be constructed using
tensor products:
i
j
Y
1
2
i i
j
Y
−
−
1
2
i i
j
Y
−
+
1
30 30
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Comparison of the two grids
ClenshawCurtis grid
NewtonCotes grid
31 31
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
From nodal basis to multivariate hierarchical basis
We start from the 1D interpolating formula. By definition, we have
with and
we can obtain
since
we have
For simplifying the notation, we consecutively number the elements in ,
and denote the jth point of as
i
X
∆
i
j
Y
i
X
∆
hierarchical
surplus
32 32
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Nodal basis vs. hierarchical basis
Nodal basis
Hierarchical basis
33 33
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Multidimensional hierarchical basis
For the multidimensional case, we define a new multiindex set
which leads to the following definition of hierarchical basis
Now apply the 1D hierarchical interpolation formula to obtain the sparse
grid interpolation formula for the multivariate case in a hierarchical form:
Here we define the hierarchical surplus as:
34 34
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Error estimate
1. V. Barthelmann, E. Novak and K. Ritter, High dimensional polynomial interpolation on sparse grids,
Adv. Comput. Math. 12 (2000), 273–288
2. E. Novak, K. Ritter, R. Schmitt and A. Steinbauer,
On an interpolatory method for high dimensional integration, J. Comp. Appl. Math. 112 (1999),
215–228
Depending on the order of the onedimensional interpolant, one can come
up with error bounds on the approximating quality of the interpolant
1,2
.
If piecewise linear 1D interpolating functions are used to construct the
sparse interpolant, the error estimate is given by:
2 3( 1)
, 2
( ) (  log  )
N
q N
f A f O M M
− −
∞
− ·
Where N is the number of sampling points,
If 1D polynomial interpolation functions are used, the error bound is:
( 2)( 1) 1
, 2
( ) (  log  )
k k N
q N
f A f O M M
− + + +
∞
− ·
assuming that f has k continuous derivatives
35 35
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Hierarchical integration
So now, any function can now be approximated by the following
reduced form
u ∈Γ
It is just a simple weighted sum of the value of the basis for all collocation
points in the current sparse grid. Therefore, we can easily extract the
useful statistics of the solution from it.
For example, we can sample independently N times from the uniform
distribution [0,1] to obtain one random vector Y, then we can place this
vector into the above expression to obtain one realization of the solution.
In this way, it is easy to plot realizations of the solution as well as the PDF.
The mean of the random solution can be evaluated as follows:
where the probability density function is 1 since we have assumed
uniform random variables on a unit hypercube .
( )
ρ Y
[0,1]
N
36 36
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Hierarchical integration
The 1D version of the integral in the equation above can be evaluated
analytically:
Since the random variables are assumed independent of each other, the
value of the multidimensional integral is simply the product of the 1D
integrals.
Denoting we rewrite the mean as
Thus, the mean is just an arithmetic sum of the hierarchical surpluses and
the integral weights at each interpolation point.
37 37
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Hierarchical integration
To obtain the variance of the solution, we need to first obtain an
approximate expression for
2
u
Then the variance of the solution can be computed as
Therefore, it is easy to extract the mean and variance analytically, thus
we only have the interpolation error. This is one of the advantages to use
this interpolation based collocation method rather than quadrature based
collocation method.
38 38
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation (ASGC)
Let us first revisit the 1D hierarchical interpolation
1
1
y
2
1
w
2
2
w
2
1
y
2
2
y
3
1
y
3
2
y
3
1
w
3
2
w
4
1
y
4
2
y
For smooth functions, the hierarchical
surpluses tend to zero as the interpolation
level increases.
On the other hand, for nonsmooth function,
steep gradients/finite discontinuities are indicated
by the magnitude of the hierarchical surplus.
The bigger the magnitude is, the stronger the
underlying discontinuity is.
Therefore, the hierarchical surplus is a natural
candidate for error control and implementation
of adaptivity. If the hierarchical surplus is
larger than a predefined value (threshold), we
simply add the 2N neighbor points of the
current point.
39 39
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
As we mentioned before, by using the equidistant nodes, it is easy to
refine the grid locally around the nonsmooth region.
It is easy to see this if we consider the 1D equidistant points of the sparse
grid as a treelike data structure
Then we can consider the interpolation
level of a grid point Y as the depth of the
tree D(Y). For example, the level of point
0.25 is 3.
Denote the father of a grid point as F(Y),
where the father of the root 0.5 is itself,
i.e. F(0.5) = 0.5.
Thus the conventional sparse grid in N
dimensional random space can be
reconsidered as
40 40
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
Thus, we denote the sons of a grid point by
( )
1
, ,
N
Y Y Y · K
From this definition, it is noted that, in general, for
each grid point there are two sons in each
dimension, therefore, for a grid point in a N
dimensional stochastic space, there are 2N sons.
The sons are also the neighbor points of the father,
which are just the support nodes of the hierarchical
basis functions in the next interpolation level.
By adding the neighbor points, we actually add the
support nodes from the next interpolation level, i.e.
we perform interpolation from level i to level i+1.
Therefore, in this way, we refine the grid locally
while not violating the developments of the
Smolyak algorithm
41 41
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
To construct the sparse grid, we can either work through the multiindex
or the hierarchical basis. However, in order to adaptively refine the grid,
we need to work through the hierarchical basis. Here we first show the
equivalence of these two ways in a two dimensional domain.
We always start from the multiindex (1, 1), which means the interpolation
level in first dimension is 1, in second dimension is also 1.
For level 1, the associated point is only 0.5.
So the tensor product of the coordinate is
0.5 0.5 (0.5, 0.5) ⊗ ·
Thus, for interpolation level 0 of smolyak
construction, we have
(1,1)
( , ) (1,1)
(0.5, 0.5)
N N
A w a ·
42 42
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
Recalling, we define the interpolation level k as .
, N k N
A
+
So, now we come to the next interpolation level, 1, 3 k · · i
For this requirement, the following two multiindices are satisfied:
(1, 2), (2,1)
Let us look at the index (1,2)
1
1 i · The associated set of points is 0.5 (j=1).
2
2 i · The associated set of points is 0 (j=1) and 1 (j=2).
So the tensor product is
(0.5) (0,1) (0.5, 0), (0.5,1) ⊗ ·
Then the associated sum is
(1,2) 1,2
(1,1) 1,2
(0.5, 0) (0.5,1) w a w a +
43 43
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
In the same way, for multiindex (2,1), we have the corresponding
collocation points
(0,1) (0.5) (0, 0.5), (1, 0.5) ⊗ ·
So the associated sum is
(2,1) 2,1
(1,1) 2,1
(0, 0.5) (1, 0.5) w a w a +
Recall
1
,
( ) ( )( )
N
i i
q N
q
A f f
≤
· ∆ ⊗ ∆
∑
i
L
So for Smolyak interpolation level 1, we have
( )
1
(1,1) (1,2)
2 1,2 (1,1) (1,1)
3
1,2 (2,1) 2,1
1,2 (1,1) 2,1
( ) (0.5, 0.5) (0.5, 0)
(0.5,1) (0, 0.5) (1, 0.5)
N
i i
A f w a w a
w a w a w a
+
≤
· ∆ ⊗ ⊗∆ · +
+ + +
∑
i
L
44 44
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
So, from this equation, it is noted that we can store the points with the
surplus sequentially. When calculating interpolating value, we only
need to perform a simple sum of each term. We don’t need to search
for the point. Thus, it is very efficient to generate the realizations.
Here, we give the first three level of multiindex and corresponding nodes.
(1,1) (0.5, 0.5)
(1, 2) (0.5, 0), (0.5,1)
(2,1) (0, 0.5), (1, 0.5)
(2, 2) (0, 0), (1, 0), (0,1), (1,1)
(1, 3) (0.5, 0.25), (0.5, 0.75)
(3,1) (0.25, 0.5), (0.75, 0.5)
→
→
→
→
→
→
45 45
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
In fact, the most important formula is the following
From this equation, in order to calculate the surplus, we need the
interpolation value at just previous level. That is why it requires that we
construct the sparse grid level by level.
Now, let us revisit the algorithm in a different view. we start from the
hierarchical basis. We still use the twodimension grid as an example.
Recalling
46 46
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
As previous given, the first level in a sparse grid is always a point
(0.5,0.5). There is a possibility that the function value at this point is 0 and
thus the refinement terminates immediately. In order to avoid the early
stop for the refinement process, we always refine the first level and keep
a provision on the first few hierarchical surpluses.
It is noted here, the associated multiindex with this point (0.5, 0.5) is
(1,1),i.e. and it corresponds to level of interpolation k = 0. (0.5, 0.5) (1,1) →
So the associated sum is still
(1,1)
( , ) (1,1)
(0.5, 0.5)
N N
A w a ·
Now if the surplus is larger than the threshold, we add sons
(neighbors) of this point to the grid.
47 47
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
In onedimension, the sons of the point 0.5 are 0,1.
According to the definition, the sons of the point (0.5,0.5) are
(0, 0.5), (1, 0.5), (0.5, 0), (0.5,1)
which are just the points in the level 1 of sparse grid. Indeed,
the associated multiindex is
(0, 0.5), (0,1) (2,1), (0.5, 0), (0.5,1) (1, 2) → →
Recalling when we start from multiindex
(1, 2) (0.5, 0), (0.5,1), (2,1) (0, 0.5), (1, 0.5) → →
So actually, we can see they are equivalent.
Now, let us summarize what we have
1: (0, 0.5), (1, 0.5), (0.5, 0), (0.5,1) k ·
So the associated sum is
(1,2) 1,2 (2,1) 2,1
(1,1) 1,2 (1,1) 2,1
(0.5, 0) (0.5,1) (0, 0.5) (1, 0.5) w a w a w a w a + + +
48 48
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
After calculating the hierarchical surplus, now we need to refine.
Here we assume we refine all of the points, i.e. threshold is 0.
Notice here, all of the son points are actually from next
interpolation level, so we still construct the sparse grid level by level.
First, in one dimension, the sons of point 0 is 0.25, 1 is 0.75.
So the sons of each point in this level are:
(0, 0.5) (0.25, 0.5), (0,1), (0, 0)
(1, 0.5) (0.75, 0.5), (1,1), (1, 0)
(0.5, 0) (0.5, 0.25), (1, 0), (0, 0)
(0.5,1) (0.5, 0.75), (1,1), (0,1)
→
→
→
→
Notice here, we have some redundant points. Therefore, it is important
to keep track of the uniqueness of the newly generated (active) points.
49 49
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation
After delete the same points, and arrange ,we can have
(0, 0), (1, 0), (0,1), (1,1) (2, 2)
(0.5, 0.25), (0.5, 0.75) (1, 3)
(0.25, 0.5), (0.75, 0.5) (3,1)
→
→
→
which are just all of the support nodes in k=2.
Therefore, we can see by setting the threshold equal zero, conventional
sparse grid is exactly recovered.
We actually work on point by point instead of the multiindex.
In this way, we can control the local refinement of the point.
Here, we equally treat each dimension locally, i.e., we still add the two
neighbor points in all N dimensions. So there are generally 2N son
(neighbor) points.
If the discontinuity is along one dimension, this method can still detect it.
So in this way, we say this method can also detect the important dimension.
50 50
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation: implementation
As shown before, we need to keep track of the uniqueness of the newly
active points.
To this end, we use the data structure <set> from the standard template
library in C++ to store all the active points and we refer to this as the
active index set.
Due to the sorted nature of the <set>, the searching and inserting is
always very efficient. Another advantage of using this data structure is
that it is easy for a parallel code implementation. Since we store all of
the active points from the next level in the <set>, we can evaluate the
surplus for each point in parallel, which increases the performance
significantly.
In addition, when the discontinuity is very strong, the hierarchical
surpluses may decrease very slowly and the algorithm may not stop
until a sufficiently high interpolation level. Therefore, a maximum
interpolation level is always specified as another stopping criterion.
51 51
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation: Algorithms
Let be the parameter for the adaptive refinement threshold. We propose the
following iterative algorithm beginning with a coarsest possible sparse grid , i.e.
with the Ndimensional multiindex i = (1,…,1), which is just the point (0.5,…,0.5).
0 ε >
1) Set level of Smolyak construction k = 0.
2) Construct the first level adaptive sparse grid .
• Calculate the function value at the point (0.5, …, 0.5).
• Generate the 2N neighbor points and add them to the active index set.
• Set k = k + 1;
3) While and the active index set is not empty
max
k k ≤
• Copy the points in the active index set to an old index set and clear the active index set.
• Calculate in parallel the hierarchical surplus of each point in the old index set according
to
Here, we use all of the existing collocation points in the current adaptive sparse grid
This allows us to evaluate the surplus for each point from the old index set in parallel.
52 52
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid collocation: Algorithms
• For each point in the old index set, if
Generate 2N neighbor points of the current active point according to the above
equation;
Add them to the active index set.
• Add the points in the old index set to the existing adaptive sparse grid Now the
adaptive sparse grid becomes
• Set k = k + 1;
4) Calculate the mean and variance, the PDF and if needed realizations of the solution.
In practice, instead of using surplus of the function value, it is sometimes preferable to
use surplus of the square of the function value as the error indicator. This is because
the hierarchical surplus is related to the calculation of the variance. In principle, we can
consider it like a local variance. Thus, is more sensitive to the local variation of the
stochastic function than .
, , , ,
( ) ( ) ( ) ( )
q N q N q N q N
f A f f A f A f A f
ε ε
∞
− ≤ − + −
Error estimate of the adaptive interpolant
2 3( 1)
2
(  log  )
N
O M M C ε
− −
· +
53 53
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
NUMERICAL EXAMPLES
54 54
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptive sparse grid interpolation
Ability to detect and reconstruct steep gradients
3 2 2 3
1
( , )
 10  10
f x y
x y
− −
·
− − + 2 2 1
1
( , )
 0.3  10
f x y
x y
−
·
− − +
55 55
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
A dynamic system with discontinuity
We consider the following governing equation for a particle moving under the
influence of a potential field and friction force:
2
2
d X dX dh
f
dt dx dt
+ · −
( ) ( )
0 0
0 , / 0 X t x dX dt t v · · · ·
If we set the potential field as , then the differential equation
has two stable fixed points and an unstable fixed point at .
( ) ( ) ( )
4 2
35/ 8 15/ 4 h x x x · −
15/ 35 x · t
The stochastic version of this problem with random initial position in the interval
can be expressed as
with stochastic initial condition
where denotes the response of the stochastic system, ,
and is uniformly distributed random variable over .
2
3
2
35 15
,
2 2
d X dX
f X X
dt dt
+ · − +
( )
0 0
0, ,  0
t
dX
X t Y X X Y
dt
·
· · + ∆ ⋅ ·
( ) , X t Y ( )
0 1 2
/ 2 X x x ≡ +
Y
[ ] 1,1 . −
1 2
  / 2 X x x ∆ · −
0 x ·
[ 1,1] −
56 56
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
A dynamic system with discontinuity
If we choose and a relatively large friction coefficient , a
steady –state solution is achieved in a short time. The analytical steadystate
solution is given by
This results in the following statistical moments
In the following computations, the time integration of the ODE is performed using
a fourthorder RungeKutta scheme and a time step was used.
0
0.05, X · 0.2 X ∆ ·
2 f ·
( )
( )
, 15/ 35, 0.25
, 15/ 35, 0.25
X t Y Y
X t Y Y
¹
→ ∞ · − < −
¹
'
→ ∞ · + > −
¹
¹
( )
1
, 15/ 35
4
X t Y Ε → ∞ · ]
]
( )
45
Var ,
112
X t Y → ∞ · ]
]
0.001 t ∆ ·
The analytic PDF is two dirac mass with unequal strength.
57 57
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of gPC expansion
2 1.5 1 0.5 0 0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Y
P
D
F
1 0.5 0 0.5 1
1.5
1
0.5
0
0.5
1
Y
X
2.5 2 1.5 1 0.5 0 0.5
0
0.5
1
1.5
2
2.5
3
3.5
Y
P
D
F
1 0.5 0 0.5 1
1.5
1
0.5
0
0.5
1
Y
X
Evolution of ( , ) for 0 10 X t Y t ≤ ≤
PDF at 10 t ·
10 p ·
20 p ·
Solution at 10 t ·
58 58
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of gPC expansion
Order
M
e
a
n
(
X
)
5 10 15 20
0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
gPC
Exact
Order
V
a
r
(
X
)
5 10 15 20
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
gPC
Exact
Mean and variances exhibit oscillations with the increase of expansion
order
59 59
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of gPC expansion
Y
X
1 0.5 0 0.5 1
1
0.5
0
0.5
1
p = 5
p = 10
p = 15
p = 20
Exact
Steadystate solution at 25 t ·
Therefore, the gPC expansion fails to converge the exact solutions with
increasing expansion orders. It fails to capture the discontinuity in the
stochastic space due to its global support, which is the well known “Gibbs
phenomenon” when applying global spectral decomposition to problems with
discontinuity in the random space.
60 60
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of Lagrange polynomial interpolation
1 0.5 0 0.5 1
0
1
2
3
4
5
6
7
8
9
10
Y
P
D
F
1 0.5 0 0.5 1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
Y
X
1 0.5 0 0.5 1
0
5
10
15
20
25
30
35
Y
P
D
F
1 0.5 0 0.5 1
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
Y
X
Evolution of ( , ) for 0 10 X t Y t ≤ ≤ Solution at 10 t ·
PDF at 10 t ·
Much better than gPC, but still exhibit oscillation near discontinuity point.
5 k ·
10 k ·
61 61
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of Lagrange polynomial interpolation
Level
M
e
a
n
(
X
)
2 4 6 8 10 12
0.1
0.2
0.3
0.4
0.5
Lagrange Polynomial interpolation
Exact
Level
M
e
a
n
(
X
)
10 11 12 13
0.16
0.162
0.164
0.166
0.168
0.17
Lagrange Polynomial interpolation
Exact
Level
V
a
r
(
X
)
10 11 12 13
0.4
0.401
0.402
0.403
Lagrange Polynomial interpolation
Exact
Level
V
a
r
(
X
)
2 4 6 8 10 12
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Lagrange Polynomial interpolation
Exact
62 62
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Failure of Lagrange polynomial interpolation
Y
X
1 0.5 0 0.5 1
1
0.5
0
0.5
1
k = 3
k = 5
k = 7
k = 9
Exact
Steadystate solution at 25 t ·
Similarly, the CSGC method with Lagrange polynomials also fails to capture
the discontinuity. However, better predictions of the loworder moments are
obtained than the gPC method. Though the results indeed converge, the
converged values are not accurate.
Near the discontinuity point of the solution, there are still several wriggles,
which will not disappear even for higher interpolation levels.
63 63
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Sparse grid collocation with linear basis function
Evolution of ( , ) for 0 10 X t Y t ≤ ≤
5 k · 10 k ·
64 64
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Sparse grid collocation with linear basis function
Y
X
1 0.75 0.5 0.25 0 0.25 0.5 0.75 1
1
0.5
0
0.5
1
k = 3
k = 5
k = 7
k = 9
Exact
Y
P
D
F
0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8
0
10
20
30
40
50
60
Solution at 25 t ·
PDF at 25 t ·
Thus, it is correctly predicting the discontinuity behavior using linear basis
function. And it is also noted that there are no oscillations in the solutions
shown here.
65 65
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Adaptivity
#Points
A
b
s
o
l
u
t
e
e
r
r
o
r
o
f
v
a
r
i
a
n
c
e
10
1
10
2
10
3
10
4
10
5
10
10
10
9
10
8
10
7
10
6
10
5
10
4
10
3
10
2
10
1
CSGC
ASGC: ε = 10
2
Y
L
e
v
e
l
1 0.75 0.5 0.25 0 0.25 0.5 0.75 1
0
5
10
15
20
For CSGC method, a maximum interpolation level of 15 is used and 32769
points are need to reach accuracy of . For the ASGC method, a
maximum interpolation level of 30 is used. Only 115 number of points are
required to achieve accuracy of .
5
1 10
−
×
10
3 10
−
×
66 66
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Highly discontinuous solution
Now we consider a more difficult problem. We set , and .
All other conditions remain as before. The reduction of the friction coefficient,
together with higher initial energy will result in the oscillation of the particle for
several cycles between each of the two stable equilibrium points.
0.05 f ·
0
1 X · 0.1 X ∆ ·
Y
X
1 0.5 0 0.5 1
1
0.5
0
0.5
1
Y
X
1 0.5 0 0.5 1
1.5
1
0.5
0
0.5
1
1.5
Failure of gPC (left) and polynomial interpolation (right) at t = 250s. The
failure of Lagrange polynomial interpolation is much more obvious.
67 67
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Highly discontinuous solution
Y
X
1 0.5 0 0.5 1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
k = 4
Y
X
1 0.5 0 0.5 1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
k = 6
Y
X
1 0.5 0 0.5 1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
k = 8
Y
X
1 0.5 0 0.5 1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
k = 10
Results using
the CSGC
method with
linear basis
functions.
68 68
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
The KraichnanOrszag threemode problem
The governing equations are
(X.Wan, G. Karniadakis, 2005),
1
2 3
2
1 3
3
1 2
2
dx
x x
dt
dx
x x
dt
dx
x x
dt
·
·
· −
subject to stochastic initial conditions:
1 1 2 2 3 3
(0) (0; ), (0) (0; ), (0) (0; ) x x x x x x ω ω ω · · ·
There is a discontinuity when the random initial condition approaches
(x
1
=x
2
) . So let us first look at the discontinuity in the
deterministic solution.
1 2
1 and 1 x x · ·
69 69
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
The KraichnanOrszag threemode problem
Singularity at plane
Time
x
2
0 5 10 15 20 25 30
1.5
1
0.5
0
0.5
1
1.5
x1(0) = 1, x2(0) = 0.95, x3(0) = 1
x1(0) = 1, x2(0) = 0.97, x3(0) = 1
x1(0) = 1, x2(0) = 0.99, x3(0) = 1
x1(0) = 1, x2(0) = 1.00, x3(0) = 1
2
1 x ·
70 70
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
The KraichnanOrszag threemode problem
For computational convenience and clarity we first perform the following
transformation
The discontinuity then occurs at the planes and .
Thus, we first study the stochastic response subject to the following
random 1D initial input:
where . Since the random initial data can cross the plane
we know that gPC will fail for this case.
(X.Wan, G. Karniadakis, 2005)
The resulting governing equations become:
2 2
3 1 2
1 3 2 3 1 2
, ,
dy dy dy
y y y y y y
dt dt dt
· · − · − +
1 2 3
(0; ) 1, (0; ) 0.1 (0; ), (0; ) 0 y y Y y ω ω ω ω · · ·
( 1,1) Y U − :
2
(0; ) y ω
2
0 y ·
1 1
2 2
3 3
2 2
0
2 2
2 2
0
2 2
0 0 1
y x
y x
y x
]
]
]
] ]
]
] ]
· −
]
] ]
]
] ]
] ]
]
]
]
]
1
0 y ·
2
0 y ·
71 71
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
The KraichnanOrszag threemode problem
Time
y
1
0 5 10 15 20 25 30
0.5
0
0.5
1
1.5
2
y1(0) = 0.1, y2(0) = 1.0, y3(0) = 1.0
y1(0) = 0.05, y2(0) = 1.0, y3(0) = 1.0
y1(0) = 0.02, y2(0) = 1.0, y3(0) = 1.0
y1(0) = 0.0, y2(0) = 1.0, y3(0) = 1.0
Singularity at plane
1
0 y ·
72 72
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Onedimensional random input: variance
Time
V
a
r
i
a
n
c
e
o
f
y
3
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 30
Time
V
a
r
i
a
n
c
e
o
f
y
2
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 30
Time
V
a
r
i
a
n
c
e
o
f
y
1
0 5 10 15 20 25 30
0
0.05
0.1
0.15
0.2
0.25
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 30
gPC fails after t = 6s while the ASGC
method converges even with a large
threshold. The solid line is the result
from MCSOBOL sequence with 10
6
iterations.
1
Var( ) y
2
Var( ) y
3
Var( ) y
73 73
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Onedimensional random input: adaptive sparse grid
Y
L
e
v
e
l
1 0.5 0 0.5 1
0
5
10
15
From the adaptive sparse grid, it is seen that most of the refinement
after level 8 occurs around the discontinuity point Y = 0.0, which is
consistent with previous discussion.
The refinement stops at level 16, which corresponds to 425 number of
points, while the conventional sparse grid requires 65537 points.
Adaptive
sparse grid
with
2
10 ε
−
·
74 74
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Onedimensional random input
The ‘exact’
solution is taken
as the results
given by MC
SOBOL
The error level
is defined as
,
30
1,2,3
max Var( ) Var( )
i i MC
t
i
y y
·
·
−
75 75
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Onedimensional random input: long term behavior
Time
M
e
a
n
o
f
y
1
0 20 40 60 80 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
MCSOBOL: 1,000,000
ASGC: ε = 10
2
ASGC: ε = 10
3
Time
V
a
r
i
a
n
c
e
o
f
y
1
0 20 40 60 80 100
0
0.05
0.1
0.15
0.2
MCSOBOL: 1,000,000
ASGC: ε = 10
2
ASGC: ε = 10
3
Time
V
a
r
i
a
n
c
e
o
f
y
2
0 20 40 60 80 100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
MCSOBOL: 1,000,000
ASGC: ε = 10
2
ASGC: ε = 10
3
Time
V
a
r
i
a
n
c
e
o
f
y
3
0 20 40 60 80 100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
MCSOBOL: 1,000,000
ASGC: ε = 10
2
ASGC: ε = 10
3
76 76
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Onedimensional random input: Realizations
These realizations are reconstructed using hierarchical surpluses. It is
seen that at earlier time, the discontinuity has not yet been developed,
which explains the reason that the gPC is accurate at earlier times.
77 77
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Twodimensional random input
Next, we study the KO problem with 2D input:
1 2 1 3 2
(0; ) 1, (0; ) 0.1 (0; ), (0; ) (0; ) y y Y y Y ω ω ω ω ω · · ·
Now, instead of a point, the discontinuity region becomes a line.
Time
V
a
r
i
a
n
c
e
o
f
y
1
0 2 4 6 8 10
0
0.05
0.1
0.15
0.2
0.25
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 10
Time
V
a
r
i
a
n
c
e
o
f
y
2
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 10
78 78
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Twodimensional random input
Time
V
a
r
i
a
n
c
e
o
f
y
3
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
MCSOBOL: 1,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
gPC: p = 10
Y
1
Y
2
1 0.5 0 0.5 1
1
0.5
0
0.5
1
It can be seen that even though the gPC fails at a larger time, the
adaptive collocation method converges to the reference solution given
by MCSOBOL with 10
6
iterations.
79 79
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Twodimensional random input
The error level
is defined as
,
10
1,2,3
max Var( ) Var( )
i i MC
t
i
y y
·
·
−
The number of
points of the
conventional
grid with
interpolation 20
is 12582913.
80 80
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Twodimensional random input: Long term behavior
0 10 20 30 40 50
2
1
0
1
2
3
4
5
x 10
3
Time
E
r
r
o
r
i
n
m
e
a
n
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
0 10 20 30 40 50
2
1
0
1
2
3
x 10
3
Time
E
r
r
o
r
i
n
v
a
r
i
a
n
c
e
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
Error in the mean (left) and the variance (right) of y
1
with different
thresholds for the 2D KO problem in [0, 50].
81 81
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Threedimensional random input
Next, we study the KO problem with 3D input:
1 1 2 2 3 3
(0; ) (0; ), (0; ) (0; ), (0; ) (0; ) y Y y Y y Y ω ω ω ω ω ω · · ·
This problem is much more difficult than any of the other problems
examined previously. This is due to the strong discontinuity ( the
discontinuity region now consists of the planes Y
1
= 0 and Y
2
= 0) and the
moderatelyhigh dimension.
It can be verified from comparison with the result obtained by MCSOBOL
that unlike the previous results, here 2 x 10
6
iterations are needed to
correctly resolve the discontinuity.
Due to the symmetry of y
1
and y
2
and the corresponding random input,
the variances of y
1
and y
2
are the same.
Finally, in order to show the accuracy of the current implementation of the
hadaptive MEgPC, we provide the results for this problem.
82 82
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Threedimensional random input:ASGC
Time
V
a
r
i
a
n
c
e
o
f
y
1
0 1 2 3 4 5 6 7 8 9 10
0.33
0.34
0.35
0.36
0.37
0.38
0.39
MCSOBOL: 2,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
Time
V
a
r
i
a
n
c
e
o
f
y
3
0 1 2 3 4 5 6 7 8 9 10
0.22
0.24
0.26
0.28
0.3
0.32
0.34
0.36
MCSOBOL: 2,000,000
ASGC: ε = 10
1
ASGC: ε = 10
2
ASGC: ε = 10
3
Evolution of the variance of y
1
=y
2
(left) and y
3
(right) for the 3D KO
problem using ASGC
83 83
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Threedimensional random input: MEgPC
Time
V
a
r
i
a
n
c
e
o
f
y
1
0 1 2 3 4 5 6 7 8 9 10
0.33
0.34
0.35
0.36
0.37
0.38
0.39
MCSOBOL: 2,000,000
MEgPC: p = 3
MEgPC: p = 4
MEgPC: p = 5
MEgPC: p = 6
Time
V
a
r
i
a
n
c
e
o
f
y
3
0 1 2 3 4 5 6 7 8 9 10
0.22
0.24
0.26
0.28
0.3
0.32
0.34
0.36
MCSOBOL: 2,000,000
MEgPC: p = 3
MEgPC: p = 4
MEgPC: p = 5
MEgPC: p = 6
Evolution of the variance of y
1
=y
2
(left) and y
3
(right) for the 3D KO
problem using MEgPC with and
4
1
10 θ
−
·
3
2
10 θ
−
·
84 84
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Threedimensional random input
Due to the strong
discontinuity in this
problem, it took
much longer time
for both methods
to arrive at the
same accuracy.
85 85
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem
In this example, we compare the convergence rate of the CSGC and
ASGC methods through a stochastic elliptic problem in two spatial
dimensions.
As shown in previous examples, the ASGC method can accurately
capture the nonsmooth region in the stochastic space. Therefore, when the
nonsmooth region is along a particular dimension (i.e. one dimension is
more important than others), the ASGC method is expected to identify it
and resolve it.
In this example, we demonstrate the ability of the ASGC method to detect
important dimensions when each dimension weigh unequally.
This is similar to the dimensionadaptive method, especially in high
stochastic dimension problems.
86 86
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem
Here, we adopt the model problem
( ) ( ) ( ) ( )
( )
, , , in
, 0 on
N N
a u f D
u D
ω ω ω
ω
−∇⋅ ⋅ ∇ ⋅ · ⋅ ×Γ
⋅ · ∂ ×Γ
with the physical domain . To avoid introducing errors of
any significance from the domain discretization, we take a deterministic
smooth load with homogeneous boundary conditions.
( ) [ ]
{ ¦
2
, 0,1 D x x y · · ∈
( ) ( ) ( ) , , cos sin
N
f x y x y ω ·
The deterministic problem is solved using the finite element method with
900 bilinear quadrilateral elements. Furthermore, in order to eliminate the
errors associated with a numerical KL expansion solver and to keep the
random diffusivity strictly positive, we construct the random diffusion
coefficient with 1D spatial dependence as
( ) ,
N
a x ω
( ) ( )
( )
( ) ( ) ( )
2
2 2 1/ 2
1
8
1
log , 0.5 1 cos 2 1
2
n L
N
N n
n
L
a x e x n Y
π
π
ω π ω
− −
·
 `
− · + −
. ,
∑
where are independent uniformly distributed random variables
in the interval
( ) , 1,
n
Y n N ω · K
3, 3
]
−
]
87 87
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem
This expansion is similar to a KL expansion of a 1D random field with
stationary covariance
( ) ( ) ( )
( )
2
1 2
1 2
2
log , 0.5 , exp
N
x x
a x x x
L
ω
 `
− −
− ·
. ,
Small values of the correlation L correspond to a slow decay, i.e. each
stochastic dimension weighs almost equally. On the other hand, large
values of L results in fast decay rates, i.e., the first several stochastic
dimensions corresponds to large eigenvalues weigh relatively more.
By using this expansion, it is assumed that we are given an analytic
stochastic input. Thus, there is no truncation error. This is different from the
discretization of a random filed using the KL expansion, where for
different correlation lengths we keep different terms accordingly.
In this example, we fix N and change L to adjust the importance of each
stochastic dimension. In this way, we investigate the effects of L on the
ability of the ASGC method to detect the important dimensions.
88 88
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem: N = 5
89 89
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem: N = 5
We estimate the approximation error for the mean and variance.
Specifically, to estimate the computation error in the qth level, we fix N
and compare the results at two consecutive levels, e.g. the error for the
mean is
( )
2
L D
( ) ( )
, 1, q N N q N N
E A u A u
+
] −
]
The previous figures shows results for different correlation lengths at
N=5.
For small L, the convergence rates for the CSGC and ASGC are nearly
the same. On the other hand, for large L, the ASGC method requires
much less number of collocation points than the CSGC for the same
accuracy.
This is because more points are placed only along the important
dimensions which are associated with large eigenvalues.
Next, we study some higherdimensional cases. Due to the rapid increase
in the number of collocation points, we focus on moderate to higher
correlation lengths so that the ASGC is effective.
90 90
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem: higherdimensions
#points:4193
Comparison:
CSGC: 3.5991x10
11
91 91
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Stochastic elliptic problem: higherdimensions
In order to further verify our results, we compare the mean and the
variance when N = 75 using the AGSC method with with the ‘exact’
solution obtained by MCSOBOL simulation with 10
6
iterations.
x
0
0.2
0.4
0.6
0.8
1
y
0
0.2
0.4
0.6
0.8
1
0
0.0001
0.0002
0.0003
0.0004
Relative error of mean
x
0
0.2
0.4
0.6
0.8
1
y
0
0.2
0.4
0.6
0.8
1
0
0.0005
0.001
0.0015
Relative error of variance
8
10 ε
−
·
( ) ( ) ( )
( )
, q N N MC
MC
E A u E u
E u
−
Computational time: ASGC : 0.5 hour MC: 2 hour
92 92
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Application to RayleighBénard instability
Cold wall θ = 0.5
hot wall θ (ω )
X
Y
0 0.25 0.5 0.75 1
0
0.25
0.5
0.75
1
I
n
s
u
l
a
t
e
d
I
n
s
u
l
a
t
e
d
Filled with air (Pr = 0.7). Ra = 2500, which is larger than the critical
Rayleigh number, so that convection can be initialized by varying the hot
wall temperature. Thus, the hot wall temperature is assumed to be a
random variable.
Prior stochastic simulation, several deterministic computations were
performed in order to find out the range where the critical point lies in.
93 93
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: deterministic case
Time
[
K
]
1
/
2
20 40 60 80 100
10
3
10
2
10
1
10
0
θ
h
= 0.30
θ
h
= 0.40
θ
h
= 0.50
θ
h
= 0.55
θ
h
= 0.60
θ
h
= 0.70
Evolution of the
average kinetic
energy for different
hot wall temperatures
As we know, below the critical temperature, the flow vanishes, which
corresponds to a decaying curve. On the other hand, the growing curve
represents the existence of heat convection. Therefore, the critical
temperature lies in the range [0.5,0.55].
94 94
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: deterministic formulation
Steady state Nusselt number which denotes the rate of heat transfer:
1
0
0
1
y
h c
Nu dx
y
θ
θ θ
·
∂
≡
− ∂
∫
1 Conduction
1 Convection
Nu
Nu
· →
> →
This again verifies the critical hot wall temperature lies in the range
[0.5,0.55]. However, the exact critical value is not known to us. So, now we
try to capture this unstable equilibrium using the ASGC method.
95 95
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: stochastic formulation
We assume the following stochastic boundary condition for the hot wall
temperature:
0.4 0.3
h
Y θ · +
where Y is a uniform random variable in the interval [0,1]. Following the
discussion before, both conductive and convective modes occur in this
range.
θ
h
L
e
v
e
l
0.4 0.45 0.5 0.55 0.6 0.65 0.7
2
4
6
8
10
12
θ
h
δ
N
u
0.4 0.45 0.5 0.55 0.6 0.65 0.7
0
0.05
0.1
0.15
1 Nu Nu δ · −
0.541
96 96
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: stochastic formulation
θ
h
θ
0.4 0.45 0.5 0.55 0.6 0.65 0.7
0.05
0
0.05
0.1
0.15
0.2
0.25
θ
h
u
0.4 0.45 0.5 0.55 0.6 0.65 0.7
0
0.02
0.04
0.06
0.08
θ
h
v
0.4 0.45 0.5 0.55 0.6 0.65 0.7
0
1
2
3
Solution of the
variables versus hot
wall temperature at
point (0.1,0.5)
u v
θ
Linear:
conduction
nonlinear:
convection
0.541
97 97
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: stochastic formulation
0.35
0.25
0.15
0.05
0.05
0.15
0.25
0.35
0.45
Max = 0.436984
0.35
0.25
0.15
0.05
0.05
0.15
0.25
0.35
0.45
Max = 0.436984
Prediction of the temperature when using ASGC (left) and the
solution of the deterministic problem using the same (right).
Fluid flow vanishes, and the contour distribution of the temperature is
characterized by parallel horizontal lines which is a typical distribution for
heat conduction.
0.436984
h
θ ·
h
θ
98 98
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: stochastic formulation
0.6
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Max = 0.667891
4
3
2
1
0
1
2
3
4
Max = 4.33161
4
3
2
1
0
1
2
3
4
Max = 4.31858
4
3
2
1
0
1
2
3
4
Max = 4.3208
4
3
2
1
0
1
2
3
4
Max = 4.33384
0.6
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Max = 0.667891
Prediction of the temperature when using ASGC (top) and the
solution of the deterministic problem using the same (bottom).
0.66789
h
θ ·
h
θ
99 99
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: Mean
1.6
1.2
0.8
0.4
0
0.4
0.8
1.2
1.6
Max = 1.73709
1.6
1.2
0.8
0.4
0
0.4
0.8
1.2
1.6
Max = 1.73123
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Max = 0.55
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
Max = 0.549918
1.6
1.2
0.8
0.4
0
0.4
0.8
1.2
1.6
Max = 1.73715
1.6
1.2
0.8
0.4
0
0.4
0.8
1.2
1.6
Max = 1.73128
ASGC (top) MCSOBOL (bottom)
100 100
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
RayleighBénard instability: Variance
0.012
0.011
0.01
0.009
0.008
0.007
0.006
0.005
0.004
0.003
0.002
0.001
Max = 0.0129218
3
2.6
2.2
1.8
1.4
1
0.6
0.2
Max = 3.30617
3
2.6
2.2
1.8
1.4
1
0.6
0.2
Max = 3.28535
3
2.6
2.2
1.8
1.4
1
0.6
0.2
Max = 3.27903
3
2.6
2.2
1.8
1.4
1
0.6
0.2
Max = 3.29984
0.012
0.011
0.01
0.009
0.008
0.007
0.006
0.005
0.004
0.003
0.002
0.001
Max = 0.0128783
ASGC (top) MCSOBOL (bottom)
101 101
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
C
C
O
O
R
R
N
N
E
E
L
L
L
L
U N I V E R S I T Y
Materials Process Design and Control Laboratory Materials Process Design and Control Laboratory
Conclusion
An adaptive hierarchical sparse grid collocation method is developed
based on the error control of local hierarchical surpluses.
Besides the detection of important and unimportant dimensions as
dimensionadaptive methods can do, additional singularity and local
behavior of the solution can also be revealed.
By employing adaptivity, great computational saving is also achieved
than the conventional sparse grid method and traditional gPC related
method.
Solution statistics is easily extracted with a higher accuracy due to the
analyticity of the linear basis function used.
The current computational code is a highly efficient parallel program
and has a wider application in various uncertainty quantification area.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.