Puzzles arising from

climateprediction.net
March, 2006

Myles Allen & many others

Department of Physics, University of Oxford
myles.allen@physics.ox.ac.uk

Oxford University
 Puzzles

„ How was it we found such high climate sensitivities

(equilibrium warming responses to doubling CO2)?
„ Why was anyone surprised?
„ Why do so many IPCC lead authors still regard these
high sensitivities as a statistical curiosity?
„ How do some other studies manage to rule out high
values for the climate sensitivity?
„ What does the ongoing disagreement mean for
“stabilisation targets”?
„ Why are we (climateprediction.net) still alone?

Oxford University
Climateprediction.net: the world’s largest
climate modelling facility

>100,000 volunteers, >130 countries, >11M model-years

Oxford University
 Results from our initial climateprediction.net
experiment (Stainforth et al, 2005)

„ Using simplified model ocean to keep runs short

– 15-year calibration phase to compute ocean heat transport
– 15-year control phase with pre-industrial CO2 (280ppm)
– 15-year 2xCO2 phase with CO2 at 560ppm.
„ Repeat with different initial conditions to average out
noise and quantify sampling uncertainty Double CO2
15 yr, 2 x CO2
Diagnostics from final 8 yrs.

Derived fluxes
Calibration

15 yr spin-up 15 yr, base case CO2 Control

Oxford University
 Parameter perturbations (initial results)

ƒ Critical Relative Humidity [RHcrit]

ƒ Accretion constant [CT]
ƒ Condensation nuclei concentration [CW]
ƒ Ice fall velocity [VF1]
ƒ Entrainment coefficient (EntCoef).
ƒ Empirically adjusted cloud fraction (EACF).

Parameter Low value Standard Value High Value

VF1 0.5 1.0 2.0
CT 5.00E-05 1.00E-04 4.00E-04
RHcrit 0.6 0.7 0.9
CW (sea, land) 1.00E-04 2.00E-04 2.00E-03
2.00E-05 5.00E-05 5.00E-04
EntCoef 0.6 3 9
EACF - 0.5 0.6

Oxford University
 Frequency Distribution of Simulations

From Stainforth et al, 2005

Oxford University
Frequency distribution, eliminating drifting
control simulations

Oxford University
And at about the same time our participants
started reporting models freezing over…

Oxford University
Un-physically strong low-cloud versus surface-
heat-flux feedback in equatorial Pacific

Oxford University
Climate sensitivities from climateprediction.net

Stainforth et al, 2005

Oxford University
And having got excited about the cold ones…

Oxford University
Stainforth et al, 2005, updated courtesy of Ben
Sanderson: raw distribution

Traditional range

Oxford University
Raw histogram plotted against S-1: a Gaussian
with a lump on one side.

Oxford University
 So why was anyone surprised by S>10K?

„ Simple inspection of the distribution of feedback

parameters, λ=F0/S, suggests a Gaussian centred on
S-1=0.35K-1 (S=2.9K) ± 0.2K-1 (2-σ) .
„ This implies similar odds on obtaining a model with
S-1>0.6K-1 (S<1.7K) as S-1<0.1K (S>10K).
„ Would anyone have batted an eyelid if we had
announced a small percentage of models with
sensitivities <1.7K?

Oxford University
 Long and short wave feedbacks
and Global Energy Balance
CS

Heat convergence
Heat convergence

Albedo

Oxford University
Many of these high sensitivity models will prove
significantly less realistic than the original
Climate Sensitivity

Global Top-of-Atmosphere Energy Imbalance

Oxford University
 But not all

Climate Sensitivity

Global Top-of-Atmosphere Energy Imbalance

Oxford University
Why not just weight the distribution of models
by some measure of similarity to observations
and histogram the result?
„ This is the approach used by Tebaldi et al, 2003,
Murphy et al, 2004, and Rougier et al, 2005.
„ The problem is, it’s wrong.
„ Results depend, to first order, on arbitrary and
obscure choices about how models are sampled.

Oxford University
Murphy et al (2004)

Oxford University
Sampling design of Murphy et al only varying
one parameter (entrainment coefficient)

Oxford University
Impact of not including S-2 weighting

Oxford University
And not including parameter-interval weighting

Oxford University
Murphy et al (2004) assuming uniform prior in S-1
(solid) and S (dashed)

Oxford University
 So what is the correct prior to use? Bayes’
theorem and constraining climate sensitivity
P ( data | S ) P ( S )
P ( S | data ) =
P ( data )
„ Definitions:
– P(S|data) = Distribution of S given data (‘what we want’)
– P(data|S) = Likelihood of data given S (‘what we measure’)
– P(S) = Prior distribution for S (‘what we assume’)
– P(data) = Normalisation constant
„ The old-fashioned realist approach:
– Focus on P(data|S), or sample unobservable parameters to
give a uniform P(S) for all S for which P(data|S) is non-zero
– Ask: how much less likely would it be for us to have made
these observations if S were >7K?

Oxford University
Forest et al (2002) assuming uniform prior in S-1
(solid: wrong) and S (dashed: right)

Oxford University
 How can we estimate a distribution for S that
does not depend on arbitrary sampling?

„ Histograms or weighted histograms depend to first

order on sampling design.
„ For example:
– Choosing to weight different regions of parameter space by
the “size” of parameter perturbations makes results depend
on arbitrary definition of model parameters.
– Choosing to sample S-1 uniformly over parameter intervals,
or choosing to sample “forcing” uniformly in an energy
budget analysis (Forster and Gregory), weights values of S
by S-2 before any comparison with observations is made.
„ Instead, we search for “Transfer Functions” between
observable quantities and climate sensitivity.

Oxford University
 A transfer function approach to estimating
climate sensitivity (Piani et al, 2005)

„ We analyze a perturbed physics ensemble to identify

a (vector) observable quantity that scales with the
forecast quantity of interest (sensitivity).
„ We then equate likelihood of observations given
model predictions of this observable with likelihood
of the corresponding value for sensitivity.
„ Although the models do not sample “model space”
in any objective way, we hypothesize that the
transfer function is invariant across models.
„ Most importantly, this hypothesis can be easily
tested by enlarging the ensemble or subdividing it.

Oxford University
 Estimating λ from control climatology

Implied
distribution
allowing for
scatter in
transfer
function Best linear
predictor of
λ/F0 applied
to ERA-40

Oxford University
 Estimating S from control climatology
Distribution from
linear transfer
function (wrong)

Distribution of S
implied by actual
transfer function

Best linear
Likelihood predictor of S
function for applied to
λ/F0 plotted ERA-40
against
sensitivity

Oxford University
Sensitivity of results to EOF truncation

2.5-97.5% range
5-95% range
10-90% range

Oxford University
 Implications of Piani et al

„ Our best estimate of climate sensitivity is 3.3K, with

a 5-95% range of 2.2-6.8K (we can’t pin down more
extreme percentiles).
„ Observable quantities scale with the feedback
parameter and not with climate sensitivity - even if
we search for predictors that are linear in sensitivity.
„ So, if you want to estimate S via λ, you should
estimate a likelihood function for λ and then plot it
against S, without weighting by dλ/dS=-F0/S2.
„ Results predicting S directly and via λ then agree.

Oxford University
Why this is a puzzle, but the alternative is worse

„ Equating likelihood of S with likelihood of λ implies

– observationally constrained PDFs of climate sensitivity
should show a fat tail towards high values and
– P(S>5) ≠ P(λ/F0<0.2).
„ Many people find this deeply troubling, but it makes
sense once you recognise that there is no discrete
set of “possible models” to sample from.
„ The converse, assuming P(S>5)=P(λ/F0<0.2), means
you can appear to rule out high values of S by
choosing to estimate it via a quantity x for which
dx/dS → 0 as S → ∞.

Oxford University
 This is an old problem: Bertrand’s paradox

„ A glass contains water and wine, with a dilution ratio

between 1:1 & 2:1
„ Argument 1:
– We have 1 unit of wine and between 1 & 2 units of water
– “Neutral prior” suggests a best-guess of 1.5 units of water
– “Best guess” dilution ratio of 3:2
„ Argument 2:
– We have 1 unit of water and between 0.5 & 1 units of wine
– “Neutral prior” suggests a best-guess of 0.75 units of wine
– “Best guess” dilution ratio of 4:3
„ Which is correct?

Oxford University
Similar results using a neural network approach:
Knutti et al, 2005

Oxford University
Consequences of restricting TOA flux imbalance
for neural net predictor (Sanderson and Knutti)

Oxford University
Distributions for S estimated from neural net
with models in approximate TOA balance

Oxford University
We were by no means the first to report the
possibility of S>7K

Oxford University
 The fundamental problem

„ All directly observable properties of the climate

system scale with the strength of atmospheric
feedbacks, λ, or S-1:

F2 xCO2
λ=
and S
dλ
→0 as S →∞
dS
„ Note that P(data|S(λ))=P(data|λ)

Oxford University
 Transient observables scale with λ for large S

„ Simplest climate model

(close to a UD-EBM because dT
upwelling traps temperature c = F − λT
anomalies near surface) dt
S ⎛
2
⎛ S0 ⎞ ⎞
„ For a ramp forcing, response T = S − ⎜
⎜ 1 − exp⎜ − ⎟ ⎟
⎟
(e.g. TCR) varies as
S0 ⎝ ⎝ S ⎠⎠
„ For small S (large λ) or slow T ∝ S − ...
forcing (large S0)
2
S0 S0
„ For large S (small λ) or fast T∝ − + ...
forcing (small S0) 2 6S

Oxford University
 So: short-term forcing tells us almost nothing
blue = 0.5K sensitivity, deep red = 20K sensitivity

Frame et al, 2005, also fitting ENSO, background

climate and effective heat capacity

Oxford University
Key uncertainty in LGM is forcing relevant to
modern S: also scales with λ if ΔT is known

ΔF=-6.6±1.5W/m2
ΔT=-5.5±0.5K

Numbers courtesy of Stefan Rahmstorf and Gavin Schmidt, realclimate.org

Oxford University
Symmetric uncertainty in past forcing P(data|F)
→ asymmetric P(data|S) for sensitivity

Oxford University
 Uncertainty in F-Q is also a problem for
estimates based on current energy budget

Oxford University
Lots of studies, same message: weak upper
bound on climate sensitivity

„ If S is “likely” < 4K (P>0.67)

then
„ S is “very likely < ~7K (P>0.9)
and we can only say
„ S “virtually certain” < 10-15K
(P>0.99)

Oxford University
And here’s why:

„ Observable properties of
climate scale with
strength of atmospheric
feedbacks, λ
„ Most constraints give
fairly Gaussian P(data|λ)
(Central Limit Theorem)
„ A Gaussian distribution of
inverse sensitivity gives…
„ no formal upper bound on
climate sensitivity

Oxford University
 Why it should be obvious we cannot rule out
high values of climate sensitivity

„ CO2 levels of 550ppmv are unprecedented with this

configuration of the continents, so any prediction of
the response must rely on computer modelling.
„ We cannot validate out models’ representation of
feedbacks in a world >4K warmer than today.
„ So once the world has warmed by 4K, we cannot
trust a model that tells us the warming will stop
before 5K, and once it has warmed by 5K…

Oxford University
 Implications for stabilisation targets

„ The recent DEFRA report on the Stabilisation 2005

workshop concluded that avoiding a 2K warming
with “relative certainty” would require CO2
concentrations to stay below 400ppmv.
„ This implies “relative certainty” that S<3.8K.
„ We won’t be sure that sensitivity is that low until
decades after greenhouse forcing begins to stabilize.
„ So we won’t know what concentration target will be
required to avoid a 2K warming until well after most
policies have been implemented to achieve it.

Oxford University
 Why is this a problem?

„ Practical measures needed today aiming for

450ppmv stabilisation are identical to those aiming
for 350ppmv: we just need to start phasing out
carbon emissions.
„ Fixating on a concentration target rather than a
temperature target is dangerous, because it implies:
– We’ll be OK until that concentration is reached.
– Once that concentration is passed, it will be too late.

Oxford University
An objective upper bound on climate sensitivity:
The Holy Grail of Climate Research

„ Appears to be required for the implementation of the

Rio Convention’s commitment to “stabilize
atmospheric greenhouse gases…”
„ So an upper bound on climate sensitivity:
– Is very hard to find, requiring a life-long commitment.
– Promises lasting fame and happiness to the finder.
– May not exist.
– And may not do you much good if you find it anyway.

Oxford University
 And the final puzzle: why are we still alone?

„ Climateprediction.net is still the only climate

research project using distributed computing.
„ Reasons:
– “You can’t recover all the data I want.”
» We can recover all the data you will ever look at.
– “You can’t run models in 64-bit precision.”
» Yes we can, and new PCs have 64-bit chips, so we will.
– “You can’t run high-resolution models.”
» We can equal the resolution of HiGEM: is that enough?
– “You can’t run it long enough to spin up the ocean.”
» Maybe you should think about your spin up procedures.

Oxford University
Distributed computing is not just for climate-
resolution models

Oxford University
Just launched: “the climate that might have
been” supported by WWF

Oxford University
 A challenge for the next generation

„ Your research is no longer limited by computing

resources: the distributed computing community is
equal to ~50 Earth Simulators, and most rival
applications are really boring.
„ Distributed computing makes modelling cheaper, but
more importantly, it allows us to do it better.
„ Don’t settle for
– Reduced-physics or “intermediate complexity” models.
– Estimating results from “statistical emulators”.
– Single results from “best guess” parameter choices.
– Having to say “this is what my model does…” in a meeting.
– Small, unrepresentative ensembles.
– Tedious, sequential model tuning exercises.

Oxford University
And you can do this with coupled models: see
bbc.co.uk/climatechange

Oxford University
Over 40,000 active participants running
HadCM3L, all forcings, 1920-2080

Oxford University
First results

Oxford University
 And if you’ve enjoyed this seminar…

„ Watch Tim Palmer explain ensemble climate

forecasting, and show off his golf swing, Tuesday,
March 14th, 8:00p.m. on BBC-2.

Oxford University
Oxford University
Knutti et al, 2005

Oxford University
Proposed Heat Flux Limits for coupled release
OSR

Heat convergence

Oxford University
 Transfer functions in a nutshell

„ Find a pattern of variations between control climates

of the climateprediction.net ensemble that predicts
either S or S-1 in the corresponding models.
– Project members of climateprediction.net ensemble onto
EOFs of HadCM3 control variability.
– Rotate to eliminate cross-correlations across the ensemble.
– Use linear regression against S or S-1 to find the optimal
combination of REOFs that minimizes squared error in
forecast variable.
– Project observed climate onto that optimal combination to
obtain best-guess S or S-1.
– Add variance based on HadCM3 control to obtain PDF.

Oxford University