Transdimensional inverse problems

in the geosciences

Malcolm Sambridge
Research School of Earth Sciences,
Australian National University
Canberra, ACT, Australia.

Signal processing and Inference for the physical sciences

26th March 2012, The Royal Society.

Collaborators: Thomas Bodin (ANU), Hrvoje Tkalcic (ANU)

Kerry Gallagher (Univ. de Rennes),
Ross Brodie (Geoscience Australia)
1
 Geophysical imaging

A 3-D image of the Earth’s mantle using multi-frequency seismic shear waves

From Zaroli (2010)

2
 Parametrization uncertainty
120˚ 130˚ 140˚ 150˚
110˚
−10˚ −10˚

−20˚ −20˚

−30˚ −30˚

−40˚ −40˚

120˚ 130˚ 140˚ 150˚

110˚

Ambient noise tomography in Australia

Saygin (2007)
140˚ 150˚

−40˚ −40˚

140˚ 150˚

Raypth densities of 400,000 SH wave

WOMBAT and TIGGER arrays
multiple frequency travel times
Arroucau et al (2010)
From Zaroli (2010)

3
 Irregular parametrizations
Static optimized meshes in regional and global studies have become popular

Zaroli et al. (2010) Sambridge & Rawlinson (2005)

Is it possible to use the data to decide how to parametrize the model ?

Chou & Booker (1979); Tarantola & Nercessian (1984); Abers & Rocker (1991); Fukao et al. (1992); Zelt & Smith (1992); Michelini (1995); Vesnaver
(1996); Curtis & Snieder (1997); Widiyantoro & van der Hilst (1998); Bijwaard et al. (1998); Bohm et al. (2000); Sambridge & Faletic (2003).

4
A probabilistic framework
Posterior PDF Likelihood

p(m|d) = C × p(d|m) p(m)

Prior PDF

mT = (m1 , . . . , mk )
k
!
m(x) = mj φj (x)
j=1

Probabilities associated with inferences

model estimation <-> model uncertainty

5
 Sampling and optimization

Optimization Probabilistic sampling

framework framework

Maximum

Markov chain
Parameter Monte Carlo sampling
space
Gradient optimization

Data fit or Log Likelihood function Samples from posterior PDF

φ(m) = ||d − g(m)||22 + α2 ||m||22 p(m|d) = C × p(d|m) p(m)

6
 Transdimensional inversion
'Withfour parameters I can fit an elephant and
with five I can make him wiggle his trunkʼ
J. von Neumann

Posterior PDF p(m|d, I) I , assumptions

Assumption uncertainty

k
!
m(x) = mj ψj (x) Uncertain parametrization
j=1

! N
#
1 1 " |d − g(m)|2
L(d|m) = √ exp − Uncertain Likelihood
(σ 2π)N 2 i σ2
Hierarchical Bayes

Let the data decide

Many applications...
7 Dension, Holmes, Mallick and Smith, 2002
 Markov chain Monte Carlo

Target PDF p(m|d) = C × p(d|m) p(m) M-H algorithm

Proposal PDF q(m2 |m1 )

m1 → m2

M-H ! "
p(m2 |d)q(m1 |m2 )
Acceptance α=1∧ Example

probability p(m1 |d)q(m2 |m1 )

Target

Rejection
Algorithm
Issues:
Efficiency M-H
Dimension Algorithm

8
 Transdimensional inversion

Variable dimension

p(m, k|d) = C × p(d|m, k) p(m|k)p(k)

! "
p(d|m2 , k2 )p(m2 |k2 )p(k2 )q(m1 , k1 |m2 , k2 )
α=1∧ |J|
p(d|m1 , k1 )p(m1 |k1 )p(k1 )q(m2 , k2 |m1 , k1 )

Birth-Death McMC algorithm, RJ-McMC algorithm

Geyer & Moller (1994) Green (1995)

k2 = k1 ± 1 k
!
m(x) = mj φj (x)
j=1

9
 Example regression
' Make everything as simple as possible, but not simplerʼ
Einstein
Birth-Death example
60
1-D Regression Data
40 Partition modelling
20

Recover the discontinuous function d

(gray) from the noisy data (red)
0
mT = (f , x, k)T
!20 True model position
variable
height
!40 variable

!60
0 2 4 6 8 10
Properties of the ensemble x
may be examined

True
Uncertainty in
Ensemble
Average
model dimension
Data

10
 Inversion for 1-D earth models
Using Airborne EM to constrain
subsurface conductivity mT = (c, x, k)T
Conductivity
0.01 0.1
0

10

20

30
Depth (m)

40

50
Uncertainty in
60
discontinuity position
70

80

Marginals in interface
position

Courtesy R. Brodie, M. Hartley

11
 2-D spatial problems
o o
o
0 0 0

o o o
9 S 9 S 9 S

o o o
18 S 18 S 18 S

o o o
27 S 27 S 27 S

o
36 S o
36 S
o
36 S

o
o o
120 E 130 E 140oE 150 oE o o
45 S110o E o o
120 E 130 E 140oE 150 oE o o
120 E 130 E 140oE 150 oE
45 S110o E 45 S110o E

20 uniformly random 2 overlapping Voronoi meshes 4 overlapping Voronoi meshes

Voronoi cells

o
0

o
9 S
Birth-Death McMC with unknowns mT = (f , x, k)T
o
18 S

Point by point averaging of output gives an ensemble

mean, covariance measures.
o
27 S

o
36 S

o o o
130 E 140oE
Effective mesh resolution increases with number of
120 E 150 oE

meshes -> super resolution.

45 S110o E

20 Voronoi meshes
12
 Classic tomography:
single models with a fixed grid
Damping

120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚

110˚ 110˚
!10˚ !10˚ !10˚ !10˚

!20˚ !20˚ !20˚ !20˚ Tikhonov

!30˚
solutions
!30˚ !30˚ !30˚

Smoothing
!40˚ !40˚ !40˚ !40˚

120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚

110˚ 110˚

120˚ 130˚ 140˚ 150˚

120˚ 130˚ 140˚ 150˚ 110˚
110˚ 120˚ 130˚ 140˚ 150˚
110˚
!10˚ !10˚ −10˚ −10˚
!10˚ !10˚

−20˚ −20˚

!20˚ !20˚ −30˚

!20˚ !20˚ −30˚

−40˚ −40˚

120˚ 130˚ 140˚ 150˚

!30˚ !30˚ 110˚
!30˚ !30˚

50 x 35
!40˚ !40˚
B-splines nodes
!40˚ !40˚

120˚ 130˚ 140˚ 150˚

110˚ 120˚ 130˚ 140˚ 150˚
110˚

120˚ 130˚ 140˚ 150˚ 130˚ 140˚ 150˚

110˚ 120˚
110˚
−10˚ −10˚ −10˚ −10˚

−20˚ −20˚ −20˚ −20˚ 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4

−30˚ −30˚
min ||δd − Gδm|| + α||m − mo ||2 + β||∇m||2 2
m
−30˚ −30˚

13
After Bodin et al. (2009)
−40˚ −40˚ −40˚ −40˚

120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚

110˚ 110˚
 2-D tomography
The wisdom of the crowd

Individual models along the Markov chain

mT = (s, x, k)T
120˚ 130˚ 140˚ 150˚ 130˚ 140˚ 150˚ 130˚ 140˚ 150˚
110˚ 120˚ 120˚
110˚ 110˚
!10˚ !10˚ !10˚ !10˚ !10˚ !10˚

!20˚ !20˚ !20˚ !20˚ !20˚ !20˚

!30˚ !30˚ !30˚ !30˚ !30˚ !30˚

!40˚ !40˚ !40˚ !40˚

Best data !40˚ !40˚
fit model
130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚
120˚ 110˚ 110˚
110˚

20 cells 50 cells 30 cells

Optimization solution 64

Ensemble mean
1750 cells + regularization 62
Number of cells
110˚
120˚ 130˚ 140˚ 150˚
110˚
120˚ 130˚ 140˚ 150˚
60
vs

Number of Cells
!10˚ !10˚ !10˚ !10˚ 58
iterations
56

54

!20˚ !20˚ !20˚ !20˚

52

50

48
0 0.5 1 1.5 2 2.5 3 3.5 4
!30˚ !30˚ !30˚ !30˚
Iterations 5
x 10

!40˚ !40˚ !40˚

Estimated PDF
!40˚
for the
110˚
120˚ 130˚ 140˚ 150˚ 110˚
120˚ 130˚ 140˚ 150˚ number of cells

110˚
120˚ 130˚ 140˚ 150˚
110˚
120˚ 130˚ 140˚ 150˚
2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4
−10˚ −10˚ −10˚ −10˚

0 10 20 30 40 50 60 70 80
−20˚ −20˚ −20˚ −20˚
Number of cells

14
−30˚ −30˚ −30˚ −30˚

−40˚ −40˚ −40˚ −40˚

Super resolution & self regularization
120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚
110˚ 110˚

True Data
 Uncertainty estimates
for free

Ensemble average Ensemble variance

120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚
110˚ 110˚
!10˚ !10˚ !10˚ !10˚

!20˚ !20˚ !20˚ !20˚

!30˚ !30˚ !30˚ !30˚

!40˚ !40˚ !40˚ !40˚

120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚

110˚ 110˚

0.0 0.1 0.2 0.3 0.4

2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4

120˚ 130˚ 140˚ 150˚ 130˚ 140˚ 150˚

110˚ 120˚
110˚
−10˚ −10˚ −10˚ −10˚

−20˚ −20˚ −20˚ −20˚

−30˚ −30˚ −30˚ −30˚

−40˚ −40˚ −40˚ −40˚

15
120˚ 130˚ 140˚ 150˚ 120˚ 130˚ 140˚ 150˚
110˚ 110˚

True Data
 Ambient noise imaging
Transdimensional Hierarchical Bayes

mT = (v, λi , x, k)T
2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4

120˚ 130˚ 140˚ 150˚

110˚
1 !10˚ !10˚

p(n|d)
0 !20˚ !20˚
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Number of Cells
1 1 1
120˚ 130˚ 140˚ 150˚
110˚
−10˚ −10˚
p(!|d)

p(a|d)

p(b|d)
!30˚ !30˚

0 0 0
−20˚ −20˚ 0.2 0.3 0.4 0.6 0.8 1 1.2 0 0.5 1 1.5
! a b

PDF Data uncertainty !40˚

!40˚
−30˚ −30˚

120˚ 130˚ 140˚ 150˚

110˚
−40˚ −40˚
0.0 0.1 0.2 0.3 0.4 0.5 0.6
0.0 0.1 0.2 0.3 0.4 0.5 0.6

110˚
120˚ 130˚ 140˚ 150˚
Velocity cell density Output: Error in the ensemble mean
120˚ 130˚ 140˚ 150˚
110˚ 130˚ 140˚
120˚ 150˚
140˚ 150˚ !10˚ !10˚ 110˚
!10˚ !10˚

!20˚ !20˚
!20˚ !20˚

!30˚ !30˚
!30˚ !30˚
−40˚ −40˚

!40˚ !40˚
!40˚ !40˚
140˚ 150˚

120˚ 130˚ 140˚ 150˚

110˚
120˚ 130˚ 140˚ 150˚
110˚

16 From Bodin et al. (2011)

 Hierarchical Bayes
When data uncertainty is uncertain
! N
#
1 1 " |d − g(m)|2
L(d|m) = √ exp − σ → λσ
(σ 2π)N 2 i σ2
! "
1 1 T −1 Cd (λ1 , λ2 )
L(d|m) = exp − (d − g(m)) Cd (d − g(m))
(|Cd |(2π)N )1/2 2
We can solve for
data noise correlation too

Noise is what we don’t fit with the data

p(m, λi , k|d) = C × L(d|m, λi , k) p(m|k)p(λi )p(k)

Markov chain, (m, λi , k) → (m! , λ!i , k! )

! "
p(m! , λ!i , k ! )|d)q(m, λi , k|m! , λ!i , k ! )
α=1∧ |J|
p(m, λi , k)|d)q(m! , λ!i , k ! |m, λi , k)

17
 Hierarchical Bayes
! N
#
1 1 " |d − g(m)|2
L(d|m) = √ exp − 2
(λσ 2π)N 2λ i σ2

Transdimensional with fixed values

for data uncertainty Transdimensional with variable data uncertainty

mT = (f , x, k)T λ and k mT = (f , x, λ, k)T

!estimated = 4 are related
A) 60
40 Hierarchical algorithm
20
Model too complex A) 60

0 Data over fit Recovered model is

!20
d 0 accurate
!40
0 2 4 6 8 10

!estimated = 10 !60
60
0 2 4
x 6 8 10

B)
40
B)
20 Model complexity OK 1
Estimated Model
0 Data fit good
complexity is accurate
!20

!40
0 2 4 6 8 10 0
0 10 20 30 40 50
!estimated = 30 Number of Partitions
60
C) C) 1
40 True
20
Estimated Data Estimated
Model too smooth noise is accurate
0 Data
Data under fit
!20
0
!40 5 10 15 20 25
0 2 4 6 8 10 !

18
 Joint inversion of multiple
classes of data
Receiver function data

mT = (v, λi , x, k)T
38
.(*(+'("&2$0*,+#0&70#+6(8
.(*(+'("&/$0*,+#0

.(*(+'("&2$0*,+#0&70#&0#+6(8

Posterior PDF on the

number of layers
Noise magnitude and correlation length
E< ; < D; D< 9; 9< =;
@+A(&768
F8
>:;
Dispersion data
=:<
!"#$%&'()#*+,-

=:;

9:< 1$"23*(&43'(&5+6%("6+#0&70#+6(8
1$"23*(&43'(&5+6%("6+#0&70#&0#+6(8
9:;
9; >; B; C; D;;
?("+#5&768

Marginal density map on PDF on

a)
subsurface velocity b)
changepoints c) d) e) f)

Receiver function Dispersion Both

data only data only data

19 After Bodin et al. (2012)

 Summary

It is possible to use the data to constrain both the parametrization

and the noise with sampling based approaches.

Trans-dimensional ensemble based approaches look promising for a

range of applications in the geosciences. General methodology.

Ensemble properties are reasonably robust and can be used as an

estimator of the solution. Make use of the wisdom of the crowd.

20
The end

21
 Some papers on Trans-dimensional inversion
Electrical resistivity inversion
Malinverno, A., 2002. Parsimonious Bayesian Markov chain Monte Carlo inversion in a
107 nonlinear geophysical problem, Geophysical Journal International, 151(3), 675–688.

Malinverno, A. & Briggs, V., 2004. Expanded uncertainty quantification in inverse prob1029
lems: Hierarchical Bayes and empirical Bayes, Geophysics, 69, 1005.

Climate histories
Hopcroft, P., Gallagher, K., & Pain, C., 2007. Inference of past climate from borehole temperature data using
Bayesian Reversible Jump Markov chain Monte Carlo, Geophysical Journal International, 171(3), 1430–1439.

Hopcroft, P., Gallagher, K., & Pain, C., 2009. A Bayesian partition modelling approach to resolve spatial variability in
climate records from borehole temperature inversion, Geophysical Journal International, 178(2), 651–666.

Thermochronology
Gallagher, K., Charvin, K., Nielsen, S., Sambridge, M., & Stephenson, J., 2009. Markov chain Monte Carlo (MCMC)
sampling methods to determine optimal models, model resolution and model choice for Earth Science problems,
Marine and Petroleum Geology, 26(4), 525–535.

Stratigraphic modelling
Charvin, K., Gallagher, K., Hampson, G., & Labourdette, R., 2009a. A Bayesian approach to inverse modelling of
stratigraphy, part 1: method, Basin Research, 21(1), 5–25.

22
 Some papers on Trans-dimensional inversion

Geochronlogy
Jasra, A., Stephens, D., Gallagher, K., & Holmes, C., 2006. Bayesian mixture modelling in geochronology via Markov
chain Monte Carlo, Mathematical Geology, 38(3), 269–300.

General
Gallagher, K., Bodin, T., Sambridge, M., Weiss, D., Kylander, M., & Large, D., 2011. Inference of abrupt changes in
noisy geochemical records using Bayesian Transdimensional changepoint models, Earth and Planetary Science
Letters, (October 2011) doi:10.1016/j.epsl.2011.09.015

ANU papers on Seismology, Regression and other geophysical problems

Trans-dimensional inverse problems, Model Comparision and the Evidence.
Sambridge, M., Gallagher, K, Jackson, A and Rickwood, P., Geophy. J. Int., 167, 528-542, doi: 10.1111/j.1365-246X2006.03155.x,
2006.

Seismic tomography with the reversible jump algorithm,

23
Bodin, T. & Sambridge, M., 2009. Geophysical Journal International, 178(3), 1411–1436.

A self-parameterising partition model approach to tomographic inverse problems

Bodin, T., Sambridge, M., and Gallagher, K., Inverse Problems, 25, 055009, doi:10.1088/0266-5611/25/5/055009, 17th March 2009.

Data inference in the 21st Century: Some ideas from outside the box,
Sambridge, M., Bodin, T., Reading, A., and Gallagher, K., 2010. Australian Soc. of Exploration Geophysics 21st International
Geophysics Conference and Exhibition, Extended Abstracts, 22-26 August, Sydney, Australia.

Transdimensional Inversion of Receiver Functions 1and Surface Wave Dispersion

T. Bodin, M. Sambridge, H. Tkalcic, P. Arroucau, K. Gallagher,and N. Rawlinson,
Journal of Geophysical research submitted, 2011. Some papers are at http://rses.anu.edu.au/~malcolm/