A Comprehensive

Study of Risk Management

Risk
Analysis
for the
Oil
Industry

A supplement to:
Risk Analysis:
Table of Contents

▲
Biography im Murtha, a registered petroleum
engineer, presents seminars and training
courses and advises clients in building
probabilistic models in risk analysis and
decision making. He was elected to
Distinguished Membership in SPE in
1999, received the 1998 SPE Award in
Economics and Evaluation, and was
1996-97 SPE Distinguished Lecturer in
Risk and Decision Analysis. Since 1992,
more than 2,500 professionals have taken
his classes. He has published Decisions
Involving Uncertainty - An @RISK
Tutorial for the Petroleum Industry. In
25 years of academic experience, he
chaired a math department, taught petro-
leum engineering, served as academic
dean, and co-authored two texts in
mathematics and statistics. Jim has a
Ph.D. in mathematics from the Uni-
versity of Wisconsin, an MS in petroleum
and natural gas engineering from Penn
State and a BS in mathematics from
Marietta College. ◆

W
▲

Acknowledgements hen I was a struggling assistant professor of

mathematics, I yearned for more ideas, for we
were expected to write technical papers and
suggest wonderful projects to graduate students.
Now I have no students and no one is counting
my publications. But, the ideas have been
coming. Indeed, I find myself, like anyone who
teaches classes to professionals, constantly
stumbling on notions worth exploring.
The articles herein were generated during a
few years and written mostly in about 6
months. A couple of related papers found their
way into SPE meetings this year.
I thank the hundreds of people who listened
and challenged and suggested during classes.
I owe a lot to Susan Peterson, John Trahan
and Red White, friends with whom I argue and
bounce ideas around from time to time.
Most of all, these articles benefited by the careful
reading of one person,Wilton Adams, who has often
assisted Susan and me in risk analysis classes.
During the past year, he has been especially helpful
in reviewing every word of the papers I wrote for
SPE and for this publication.Among his talents are
a well tuned ear and high standards for clarity.
I wish to thank him for his generosity.
He also plays a mean keyboard, sings a
good song and is a collaborator in a certain
periodic culinary activity.
You should be so lucky. ◆

A Guide To Risk Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 3

▲

Table of Contents Central Limit Theorem – Polls and Holes . . . . . . . . . . . 5

Estimating Pay Thickness
From Seismic Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Bayes’ Theorem – Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . 12
Decision Trees vs. Monte Carlo Simulation . . . . . . . . 14
When Does Correlation Matter? . . . . . . . . . . . . . . . . . 19
Beware of Risked Reserves . . . . . . . . . . . . . . . . . . . . 23

2 Risk Analysis

A Guide To –
Risk Analysis
R
R isk and decision analysis was born in the middle
of the 20th century, about 50 years after some of
the necessary statistics became formalized.
Pearson defined standard deviation and skewness
in the late 1890s, and Galton introduced
percentiles in 1885.
The term Monte Carlo, as applied to
uncertainty analysis, was mentioned by Metropolis
and Ulam: the Journal of the American Statistical
Association in 1940. D.B. Hertz published his
classic Harvard Business Review article in 1964. A
couple of years later, Paul Newendorp began
teaching classes on petroleum exploration
economics and risk analysis, out of which evolved
the first edition of his text in 1975, the same year
as A.W. McCray and 2 years before R.E. Megill
wrote their books on the subject. Ten years later,
there was commercial software available to do
Monte Carlo simulation.
During this 50-year period, decision analysis,
featuring decision trees, also came of age. Raiffa’s
classic book appeared in 1968. By 1985, there
were several commercial applications of software
on the market.
These developments, in many ways, paralleled
the development of petroleum engineering, with
the basics appearing in the 1930s, mature texts like
The oil and gas industry remained skeptical about
risk analysis throughout most of this development period.
Craft and Hawkins appearing in the late 1950s, and
reservoir simulation and other computer based
methods emerging in the 1970s and 1980s.
Similarly, reservoir characterization and geostatistics
came along in the second half of the 20th century.
Oddly, the oil and gas industry remained
skeptical about risk analysis throughout most of this
development period, usually limiting their accept-
Risk Analysis
Risk Analysis:
An Overview
ance to isolated departments within organizations.
Managers were notoriously unwilling to embrace
results that presented probability distributions for
reserves and net present value (NPV). Consultants
offering services and software vendors know too
well these levels of resistance.
Now, finally, there seems to be broader
acceptance of probabilistic methods, although as
I write, my SPE Technical Interest Group digest
contains strong negativism from traditionalists
about probabilistic prices. Nonetheless, consider
these items:
• the three most recent and five out of the last
seven recipients of the SPE Economics and
Evaluation award have been strong proponents
of risk analysis;
• whereas the index to the last edition of the
Petroleum Engineers’ Handbook only had two
references to “risk,” the forthcoming edition
will feature an entire chapter on the topic;
• my first paper on Monte Carlo simulation was
presented at the Eastern Regional Meeting in
1987 and summarily rejected by the editorial
committees for not being of adequate general
interest. (It was a case study dealing with the
Clinton formation, but the methods were
clearly generic and used popular material
balance notions). Ten years later, I published
Monte Carlo Simulation, Its Status and Future in
the Distinguished Author series;
• the most popular SPE Applied Technology
Workshop focuses on probabilistic methods;
• SPE, SPEE and WPC are working on
definitions that include probabilistic
language; and
3
Risk Analysis:
An Overview
We need to encourage papers and presentations
by people who have applied the methods for several years.
• the largest service companies acquired
smaller companies that market probabilistic
software.
So, we advocates of Monte Carlo and
Decision Trees should be jumping with glee that
the rest of the world has caught up to us, right?
Well, sort of…
The problem with this newly-found technique
is that we are still learning things by our mistakes.
Like anything else that evolves into widespread use,
risk analysis requires more maturation. Only now
do we have the critical mass on proponents to
refine our applications, to discover what works, to
eliminate faulty logic, and to test our ideas by
papers, courses, books and forums.
I have:
• discovered things during the past 5 years that
make some of my papers out of date;
• proposed methods that are too difficult to
explain and therefore impractical in an
organization;
• stumbled on limitations of the software I use
that force us to change procedures;
• witnessed numerous misuses, misquotations,
and misinterpretations, many of them in
technical papers; and
• had ongoing unresolved arguments with
reliable colleagues.
All this spells opportunity for the larger com-
munity interested in risk analysis. There should
be better papers and more of them, and good
discussions and improved software. More people
doing day-to-day modeling should come for-
ward. I know of several companies where the
high profile spokesmen are far less knowledge-
able and less experienced than some of their low
profile colleagues.
I am reminded of an experience while on my
first sabbatical at Stanford in 1972, studying
operations research and management science.
The university granted me full access to all the
libraries, and I read more papers in 12 months
than any 5-year period before or since. I dis-
covered the early years of what were two
journals then, Management Science and Operations
4 Risk Analysis
Research. Many of the authors had later became
f a m o u s . M a ny o f t h e p a p e r s we re l u c i d ,
expositions, rather than arcane dissertations.
Often they raised unanswered questions and
stimulated others. It was clear that these people
had been mulling over these ideas and
discovering them in fields that were still
immature. Once the publications came into
being, they exploded with their knowledge.
We have made some progress toward this
exposition as one can see by the increased
number of papers at several of the SPE and AAPG
meetings. But we need to encourage papers and
presentations by people who have applied the
methods for several years, testing and improving,
discarding ideas that do not travel well from the
cloister to the hearth, comparing alternative solu-
tions, providing case studies, documenting look-
backs, citing success stories within organizations
and touting benefits of company-wide prob-
abilistic methods.
SPE and AAPG have provided ample oppor-
tunities with forums, workshops and conferences
for the general ideas to be promulgated.A natural
next step would be to hold workshops for
advanced users.
About the articles in this publication
This publication is in part a challenge to review
some of our assumptions and check our language. It
is intentionally provocative and open ended.
Themes articulated:
• everyone should use probabilistic not deter-
ministic methods;
• fallacies abound in both deterministic and
probabilistic methods;
• teach people how to ask questions whose
answers require probabilistic methods;
• we often abuse the language of probability
and statistics;
• increased use of risk analysis presents more
opportunity to do things wrong; and
• in order to talk about real options and
optimizations (buzzwords de jour), we need to
do probabilistic cash flow properly. ◆

Central Limit Theorem –
Polls and Holes
W
W hat do exit surveys of voters in presidential elections
have in common with porosities calculated from logs
of several wells penetrating a geological structure? The
answer is that in both cases, the data can be used to
estimate an average value for a larger population.
At the risk of reviving a bitter debate, suppose that
a carefully selected group of 900 voters is surveyed as
they leave their polling booths. If the voters surveyed
a) are representative of the population of a whole and
b) tell the truth, then the ratio
r = (Number of voters for Candidate A in survey)/(number of
voters surveyed)
should be a good estimate of the ratio.
R = (Number of voters for Candidate A in population)/(number
of voters in population)
Moreover, by doing some algebra, the statistician
analyzing the survey data can provide a margin of
error for how close r is to R. In other words, you are
pretty confident that
(r - margin of error) < R < (r + margin of error).
The margin of error formula depends on
three things:
• the level of confidence of the error ( “I am 90%
or 95 % or 99% confident of this”);
• the number of voters who chose candidate A; and
• the number of voters surveyed.
To end this little diversion, here is the approximate
formula for the margin of error in a close race (where
R is roughly 45% to 55%), and we are satisfied with a
95% confidence level (the most common confidence
level used by professional pollsters).
Margin of error = 1/sqrt (N) approx.
Where N = sample size, the number of voters polled.
Thus when N=900, the margin of error would
Here’s the point: the Central Limit Theorem
guarantees these distributions of average properties –
net pay, porosity and saturation – will tend to be normal.
Risk Analysis
Risk Analysis:
Central Limit Theorem
be about 1/30 or 3%.
Thus,if 54% of the voters surveyed said they voted
for candidate A, then there is about a 95% chance that
A will get between 51% and 57% of the full vote.
This method of analysis is a direct consequence
of the Central Limit Theorem, one of the most sig-
nificant results in mathematical statistics.
Suppose you have a distribution X, with mean 
and standard deviation . X can be virtually any
shape. From X, we sample n values and calculate
their mean.Then we take another sample of size n
from X and calculate its mean. We continue this
process, obtaining a large number of samples and
build a histogram of the sample means. This
histogram will gradually take a shape of a smooth
curve.The amazing fact is that that curve, the limit
of the sample means, satisfies these conditions:
• the sample means are approximately normally
distributed;
• has mean equal , but;
• has standard deviation of approximately
 n (the mean standard error).
The key here is that the mean standard error gets
small as n (the sample size) gets large.
We need another fact about normal distribu-
tions, namely that 68% of the values in a normal
distribution lie within one standard deviation of its
mean, and 95% and 99.7% of the values lie within
two and three standard deviations, respectively.
In our case, X is a binomial variable with exactly two
possible values, 0 and 1, where the probability of 1 is the
percentage, p, of people among all voters who voted for
A and the probability of 0 is 1-p.The mean value of this
distribution is p. The standard deviation is p(1-p).
5
Risk Analysis:
Central Limit Theorem

This would be a good time to ask that

important question, “Why bother?”
What does this have to do with porosity?
Structure Area Well Net pay Porosity Sw
1S 5010 1S1 34 0.12 0.31 Application: average net pay, average
1S 5010 1S2 56 0.07 0.20 porosity, average saturations
1S 5010 1S3 74 0.14 0.38 Consider the following typical volumetric
1S 5010 1S4 47 0.09 0.30 formula:
1S 5010 1S5 53 0.18 0.26 G = 43,560Ah(1-Sw)/Bg*E
1S 5010 1S6 29 0.08 0.29 Where A = Area
1S 5010 1S7 84 0.15 0.37 h = Net pay
1S 5010 1S8 40 0.14 0.25  = Porosity
2S 2600 2S1 35 0.12 0.33 Sw = Water saturation
2S 2600 2S2 42 0.07 0.38 Bg = Gas formation volume factor
2S 2600 2S3 37 0.09 0.41 E = Recovery efficiency
2S 2600 2S4 61 0.13 0.26 In this formula, there is one component that
2S 2600 2S5 27 0.10 0.33 identifies the prospect, A, while the other factors
2S 2600 2S6 48 0.08 0.35 essentially modify this component. The variable h,
3S 4300 3S1 68 0.12 0.26 for example, should represent the average net pay
3S 4300 3S2 94 0.16 0.20 over the area A. Similarly,  represents the average
3S 4300 3S3 35 0.07 0.37 porosity for the specified area, and Sw should
3S 4300 3S4 47 0.09 0.30 represent average water saturation.
3S 4300 3S5 68 0.14 0.32 Why is this? Because, even though there may be
3S 4300 3S6 75 0.10 0.28 a large range of net pay values throughout a given
3S 4300 3S7 67 0.12 0.25 area, A, we are interested only in that average net pay
3S 4300 3S8 48 0.13 0.23 which, when multiplied by A, yields the (net sand)
3S 4300 3S9 69 0.09 0.30 volume of the prospect.
3S 4300 3S10 88 0.15 0.23 Consider a database of wells in some play, as
shown in Table 1, grouped into structures. Structure
1 has eight wells, structure 2 has six wells, and so on.
Table 1. Well database
We need to take this a step further. Suppose we
construct a net sand isopach map for structure 1, and
Structure Area h_ave Por_ave Sw_ave then calculate the average net sand for the structure,
1S 5010 52 0.12 0.30 by integrating and dividing by the base area. Then
2S 2600 42 0.10 0.34 we do the same thing for porosity as well as for
3S 4300 66 0.12 0.27 water saturation.This yields a new database,Table 2,
4S … … … … which becomes the basis for a Monte Carlo
simulation. In the simulation, when we sample a
Table 2. Structure database
value for A, we then sample values for average net
pay, average porosity and average saturation for that
Nevertheless, in many applications, as we shall see later, area, which gives one realization of gas in place.
X is a continuous variable, with a shape of a normal, Although the suggested process of obtaining the
lognormal or a triangular distribution, for example. averages involved integration, one could use the
Thus, our sample’s mean (the percentage of the numerical average of the data to approximate the
people in our sample who voted for A) may not be integrated average. Even so, since we often try to
exactly the mean of the normal distribution, but we drill wells in the better locations, it would still be
can be 95% confident that it is within two values of useful to sketch a map with the well locations and
the mean standard error of the true mean. get some idea whether the well average would be

6 Risk Analysis

biased compared with the averages over the entire
structure. If we put a grid over the area and place a
well location in each grid block, then the two
notions of average essentially coincide.
These distributions of averages are normal
Here’s the point: the Central Limit Theorem guar-
antees these distributions of average properties – net
pay, porosity and saturation – will tend to be normal.
Another consequence of the theorem is that these
distributions of averages are relatively narrow, for
example, they are less dispersed than the full
distributions of net pays or porosities or saturations
from the wells, which might have been lognormal or
some other shape. The correct distributions for
Monte Carlo analysis, however, are the narrower,
normal-shaped ones.
One objection to the above argument (that we
should be using narrower, normal distributions) is that
often we do not have ample information to estimate the
average porosity or average saturation.This is true. None-
theless, one could argue that it still might be possible to
imagine what kind of range of porosities might exist
from the best to the worst portions of the structure.
Even the main factor (area or volume)
can be skewed left.
To help your imagination, we do have ample
information from many mature fields where
material balance could provide estimates. We also
have extensive databases with plenty of infor-
mation, from which some range of average values
could be calculated and compared with the
broader ranges of well data.
Always remember that, like all else in Monte
Carlo simulation, you must be prepared to justify
every one of your realizations, for example,
combinations of parameters. Just as we must guard
against unlikely combinations of input parameters by
incorporating correlations in some models, we should
ask ourselves if a given area or volume could
conceivably have such an extreme value for average
porosity or average saturation. If so, then there must
be even more extreme values at certain points within
the structure to produce those averages (unless, of
course, the structure is uniformly endowed with that
property, in which case, I would be skeptical of the
wide disparity from one structure to the next.)
Risk Analysis
Risk Analysis:
Central Limit Theorem
How much difference will this make?
This would be a good time to ask that important
question,“Why bother?”What if this is correct: that
we should be using narrower and more symmetric
shaped distributions for several of the factors in the
volumetric formula? Does it matter in the final
estimate for reserves or hydrocarbon in place? How
much difference could we expect?
Let us consider porosity.When we examine the
database of porosities from all wells (for example, the
average log porosity for the completion interval or
from the layer interval in a multilayer system) as in
Table 1, there are two possible extreme situations.At
one extreme, it is possible that each structure
exhibits a wide range of quality from fairway to
flanks, but the average porosities for the various
structures always fall between 10% and 14%. In this
case, the range of all porosities could easily be several
times as broad as the average porosities.That is, there
are similarities among structures, but not much
homogeneity within a structure.
On the other hand, it is possible that each structure
is relatively homogeneous, but the different structures
are quite dissimilar in quality, with average porosities
ranging from 5 % to 20%.In this case,the two distribu-
tions would be closer together. Note, however, that the
average porosities will always have a narrower
distribution than the complete set of porosities.
Perhaps the contrast is even easier to see with net
pays. Imagine a play where each drainage area tends
to be relatively uniform thick, which might be the
case for a faulted system. Thus, the average h for a
structure is essentially the same as any well thickness
within the structure. Then the two distributions
would be similar. By contrast, imagine a play where
each structure has sharp relief, with wells in the
interior having several times the net sand as wells
near the pinch outs.Although the various structures
could have a fairly wide distribution of average
thicknesses, the full distribution of h for all wells
could easily be several times as broad. The
distribution for A could easily be lognormal if the
drainage areas were natural. In a faulted system,
however, where the drainage areas were defined by
faults, the distribution need not be lognormal. The
7
Risk Analysis:
Central Limit Theorem
Additional complications arise because of
uncertainty about the range of driving mechanisms –
will there be a water drive? Will gas injection or water
injection be effective?
right way to judge whether the type of distribution
matters for an input variable is to compare what
happens to the output of the simulation when one
type is substituted for another.
What about the output
of the simulation, OOIP?
Regardless of the shapes of the inputs to a
volumetric model – be they skewed right, or left, or
symmetric the – output will still be skewed right,
thus approximately lognormal. In fact, as is well
known, the Central Limit Theorem guarantees this.
The argument is straightforward: the log of a
product (of distributions) is a sum of the logs (of
distributions), which tends to be normal. Thus the
product, whose log is normal, satisfies the definition
of a lognormal distribution.
Will the Area distribution
always be lognormal?
The traditional manner of describing area and
treating it as a lognormal distribution is based on
prospects in a play. If we were to select at random
some structure in a play, then the appropriate
distribution likely would be a lognormal. Sometimes,
however, not even the Area parameter should be
modeled by a lognormal distribution.Why? Suppose
a particular prospect is identified from 3-D seismic.
We have seen situations where the base case value of
area or volume is regarded as a mode (most likely).
When asked to reprocess and or reinterpret the data
and provide relatively extreme upside (say P95) and
downside (say P5) areas or volumes, the results are
skewed left: there is more departure from the mode
toward the downside than the upside. Because the
conventional lognormal distribution is only skewed
right, we must select another distribution type, such
as the triangular, beta, or gamma distribution.
Variations of the volumetric formula
Among the numerous variations of the volumetric
formulas, there is usually only one factor that serves
the role of Area in the above argument. For instance,
8 Risk Analysis
H
another common formula is:
OOIP = 7,758Vb(NTG)So/Bo
Vb = Bulk rock volume
NTG = Net to gross ratio
Here,Vb would be the dominant factor, which
could be skewed right and modeled by a lognormal
distribution, while the factors NTG, , So and Bo
would tend to be normally distributed, since they
represent average properties over the bulk volume.
Recovery factors
Recovery factors, which convert hydrocarbon in
place to reserves or recoverable hydrocarbon, are also
average values over the hydrocarbon pore volume.
The recovery efficiency may vary over the structure,
but when we multiply the OOIP by a number to get
recoverable oil, the assumption is that this value is an
average over the OOIP volume. As such, they too
would often be normally distributed. Additional
complications arise, however, because of uncertainty
about the range of driving mechanisms: will there be
a water drive? Will gas injection or water injection be
effective? Some people model these aspects of
uncertainty with discrete variables.
Summary
The Central Limit Theorem suggests that most of the
factors in a volumetric formula for hydrocarbons in
place will tend to have symmetric distributions and can
be modeled as normal random variables.The main factor
(area or volume) can be skewed left or right. Regardless
of the shapes of these input distributions, the outputs of
volumetric formulas, oil and gas in place and reserves,
tend to be skewed right or approximately lognormal.
Because the conventional wisdom is to use lognormal
distributions for all of the inputs,the above argument may
be controversial for the time being.The jury is still out.
We could take a poll and see what users believe. Oh yes,
then we could use the Central Limit Theorem to analyze
the sample and predict the overall opinion.
What goes around comes around.
Stay tuned for other applications of the Central
Limit Theorem. ◆

Estimating Pay Thickness From
Seismic Data
H
H ow do we estimate net pay thickness, and how
much error do we introduce in the process? Usually,
we specify two depths, to the top and bottom of the
target interval and take their difference.The precision
and accuracy of the thickness measurement, therefore,
depends on the precision and accuracy of two
individual measurements. The fact that we are sub-
tracting two measurements allows us to invoke the
Central Limit Theorem to address the questions of
error.This theorem was stated in a previous article. A
suitable version of it is given in the Appendix.
First, let us remind ourselves of some of the issues
about measurements in general. We say a measure-
ment is accurate if it is close to the true value, reliable
or precise if repeated measurements yield similar
results and unbiased if the estimate is as likely to
exceed the true value as it is to fall short. Sometimes
we consider a two-dimensional analogy, bullet holes
in a target. If the holes are in a tight cluster, then they
are reliable and precise. If they are close to the
bullseye, then they are accurate. If there are as many
to the right of the bullseye as the left, and as many
above the bullseye as below, then they are unbiased.
With pay thickness, we are interested in the
precision of measurement in a linear scale, which we
take to mean the range of error. Our estimate for
thickness will be precise if the interval of error is
small. Consider the following situation.
Application: adding errors
A team of geoscientists estimated the gross
thickness of a reef-lagoon complex that resides
between two identifiable seismic events: the lower
anhydrite at the top and a platform at the bottom.
Logs from one well provide estimates of the
But, is that what we really care about: the
theoretical minimum and maximum?
Risk Analysis
Risk Analysis:
Central Limit Theorem
distances from the top marker to the target facies
and from the bottom of the facies to the platform
– namely 100m and 200m respectively, and the
facies thickness is 600m – the overall distance
between the markers is 900m. In one particular
offsetting anomaly, the depth measurement to the
lower anhydrite is 5,000m, plus or minus 54m.
The depth estimate to the platform is 5,900m, plus
or minus 54m.What is the range of measurement
for the thickness of the target facies?
First, we should ask what causes the possible
error in measurement. In the words of the
geoscientists, “If the records are good, the range
for picking the reflection peak should not be
much more than 30 milliseconds (in two-way
time) and at 3,600 m/second that would be about
54m. This, of course, would be some fairly broad
bracketing range, two or three standard deviations,
so the likely error at any given point is much less.
We would also hold that there is no correlation
between an error in picking the lower anhydrite
and the error in picking the platform.”
A further question to the geoscientists revealed
that the true depth would just as likely be greater
than as it would be less than the estimate.That is, the
estimates should be unbiased.
Solution
People who use worst case scenario arguments
would claim that the top of the reefal facies could
be as low as 5,154m and the bottom as high as
5,646m, giving a minimum difference of 492m.
Similarly, they say the maximum thickness would
be (5,754m-5,046m) = 708m. In other words,
they add and subtract 2*54 = 108 from the base
9
Risk Analysis:
Central Limit Theorem

customary to regard the practical range of a normal

Uncorrelated Normal Triangular Uniform distribution to be the mean plus or minus three
99% confidence [535m,665m] [520m ,680m] [503m, 697m] standard deviations, which is 99.7% of the true
95% confidence [551m, 649m] [539m, 661m] [515m, 685m] (infinite) range. Thus the actual depths would be as
90% confidence [558m, 642m] [549m, 651m] [526m, 674m] much as 54m (=3 * 18) off from the estimated
depths. Again, we must be careful to distinguish
Correlated (r=0.7) Normal Triangular Uniform between the theoretical model limits and something
99% confidence [565m; 635m] [555m, 645m] [533m, 665m] of practical importance. Every normal distribution
95% confidence [573m, 627m] [566m, 634m] [548m, 652m] extends from –∞ to +∞.The standard practical range
90% confidence [578m, 623m] [572m, 629m] [557m,643m] extends three standard deviations each side of the
mean. Especially if we are talking about measure-
ments, we know the values must be positive numbers.
Table 1. Effect of Distribution Shape on Ranges (maximum, minimum) for
Net Pay Thickness Next, we run a very simple Monte Carlo
simulation, in which we select 2,000 pairs of values
case of 600m to get estimates of minimum and from these two distributions and calculate Thickness.
maximum thicknesses. The results can be plotted up as a distribution, which
But, is that what we really care about: the the Central Limit Theorem (see below) predicts is
theoretical minimum and maximum? We may be itself another normal distribution, namely:
more interested in a practical range that will cover a Thickness = Normal(600m; 25.5m)
large percentage, say 90%, 95% or 99% of the cases. In other words, the 3-sigma range of values for
A probabilistic approach to the problem says Thickness is [523m; 677m], not [492m; 708m] as the
that the two shale depths are distributions, with worst- and best-case scenarios would have us believe.
means of 5,000m and 5,900m respectively. The In fact, 90% of the time Thickness will be between
assertion that there is no bias in the measurement 558m and 642m, and 99% of the time the value of
suggests symmetric distributions for both depths. Thickness would fall between 535m and 665m.
Among the candidates for the shapes of the Had we taken the distributions to be triangular
distributions are uniform, triangular and normal. (minimum, mode, maximum), the simulation would
One way to think about the problem is to ask show that Thickness has a 99% probability range of
whether the chance of a particular depth becomes [1,706.12ft; 2,231.08ft] or [520m; 680m]. In this
smaller as one moves toward the extreme values in case, we would use:
the range. If the answer is yes, then the normal or Top = Triangular (5,046m; 5,100m; 5,154m) and
triangular distribution would be appropriate, since Bottom = Triangular (5,646m; 5,700m; 5,754m) and
the remaining shape, the uniform distribution, Thickness = Bottom – Top
represents a variable that is equally likely to fall into The result is approximated by Normal(600m;
any portion of the full range.Traditionally, errors in 31.5m).The P5, P95 interval is [549m; 65m].These
measurement have been modeled with normal and other approximations are summarized in Table 1.
distributions. In fact, K.F. Gauss, who is often Finally, we run this model using the uniform
credited with describing the normal distribution - (minimum, maximum) distribution, specifically
but who was preceded by A. de Moivre – touted uniform (4946m,5054m) for the top and uniform
the normal distribution as the correct way to (5,846m, 5,954m) for the bottom. Thanks to the
describe errors in astronomical measurements. Central Limit Theorem, the difference can still be
If the uncertainty by as much as 54m is truly a reasonably approximated by a normal (not uniform)
measurement error, then the normal distribution distribution – although as one can show, the
would be a good candidate. Accordingly, we triangular distribution type actually fits better –
represent the top and bottom depths as: namely Normal (600m; 38.5m).The interval ranges
Bottom = Normal(5,900m; 18m), where its mean is 5,900m shown in the last column of the table were obtained
and its standard deviation is 18m from a simulation with 6,000 iterations.
Top = Normal(5,000m; 18m)
Thick = Bottom – Top –300m (the facies thickness is 300m Summary and refinements
less than the difference between the markers) When we subtract one random variable from
We use 18 for the standard deviation because it is another the resulting distribution tends to be

10 Risk Analysis

We may be more interested in a practical range
that will cover a large percentage, say 90%, 95% or 99%
of the cases.
normal; its coefficient of variation (ratio of standard
deviation to mean) reduces to about 30% from that
of the original variables. When pay thickness is
obtained by subtracting two depth measurements,
the precision of thickness is better than the precision
of the inputs.Thus rather than compounding errors,
the effect is to reduce errors.
One aspect of Monte Carlo simulation that
always needs to be considered is possible correlation
between input variables. In the present context, the
question is whether a large value for Top would
typically be paired with a large or small value for
Bottom.That is, if we overestimate the depth of Top,
would we be more likely to overestimate or
underestimate the value of Bottom or would the
values be uncorrelated (independent)? When they
were asked, the geologists said that if anything, the
two values would be positively correlated: a large
value for Top would tend to be paired with a large
value for Bottom. Essentially, they argued that they
would consistently select the top or the midpoint or
the bottom of a depth curve. How would that
impact the results?
We reran the simulation using a rank correlation
coefficient of 0.7 between the two inputs in each of
the three cases (normal, triangular, uniform shapes).
The results, shown in the lower portion of Table 1,
show a dramatic effect: the error intervals decrease
by nearly 50% from the uncorrelated cases.
Conclusions
The Central Limit Theorem (see below) can be
applied to measurement errors from seismic
interpretations.
When specifying a range of error for an estimate,
we should be interested in a practical range, one that
would guarantee the true value would lie in the
given range 90%, 95% or 99% of the time.
When we estimate thickness by subtracting one
depth from another, the error range of the result is
about 30% smaller than the error range of the depths.
If the depth measurements are positively
correlated, as is sometimes thought to be the case,
this range of the thickness decreases by another 50%.
Risk Analysis
Risk Analysis:
Central Limit Theorem
The Central Limit Theorem
Let Y = X1 + X2 +…+ Xn and Z = Y/n where X1, X2, …, Xn
are independent, identical random variables each
with mean  and standard deviation .Then:
• both Y and Z are approximately normally
distributed;
• the respective means of Y and Z are n* and
; and
• the respective standard deviations are
approximately n  and  n
This approximation improves as n increases.
Note that this says the coefficient of variation, the
ratio of standard deviation to mean, shrinks by a
factor of n for Y.
Even if the Xs are not identical or
independent, the result is still approximately
true: adding distributions results in a distribution
that is approximately normal even if the
summands are not symmetr ic. Moreover,
coefficient of variation diminishes.
When is the approximation not good? Two
conditions will retard this process: a few dominant
distributions, strong correlation among the inputs.
Some illustrations may help.
For instance, we take 10 identical lognormal
distributions each having mean 100 and standard
deviation 40 (thus with coefficient of variation,
CV, of 0.40).
The sum of these distributions has mean 1,000
and standard deviation 131.4 so CV=.131, which is
very close to 0.40/sqrt(10) or 0.127.
On the other hand, if we replace three of the
summands with more dominant distributions, say
each having a mean of 1,000 and varying standard
deviations of 250, 300 and 400, say, then the sum
has a mean of 3,700 and standard deviation, 575,
yielding a CV of .16. The sum of standard
deviations divided by the square root of 10 would
be 389, not very close to the actual standard
deviation. It would make more sense to divide the
sum by the square of root of 3 acknowledging the
dominance of three of the summands. As one can
find by simulation, however, even in this case, the
sum is still reasonably symmetric. ◆
11
Risk Analysis:
Bayes’ Theorem
Bayes’ Theorem –
Pitfalls
D D o you know how to revise the probability of
success for a follow up well?
Consider two prospects, A and B, each having
a chance of success, P(A) and P(B). Sometimes
the prospects are independent in the sense that
the success of one has no bearing on the success
of the other. This would surely be the case if the
prospects were in different basins. Other times,
however, say when they share a common source
rock, the success of A would cause us to revise
the chance of success of B. Classic probability
theory provides us with the notation for the
(conditional) probability of B given A, P(B|A), as
well as the (joint) probability of both being
successful, P(A&B).
Our interest lies in the manner in which we
revise our estimates. In particular, we will ask:
• how much better can we make the chance of
B when A is successful?
That is, how large can P(B|A) be relative to
P(B); and
• if we revise the chance of B upward when
A is a success, how much can or should we
revise the chance of B downward when A is
a failure?
As we shall see, there are limits to these revisions,
stemming from Bayes’Theorem.
Bayes’ Theorem regulates the way two or
more events depend on one another, using
conditional probability, P(A|B), and joint
probability, P(A&B). It addresses independence
and partial dependence between pairs of events.
The formal statement, shown here, has numerous
applications in the oil and gas industry.
Bayes’ Theorem regulates the way two or more
events depend on one another, using conditional
probability, P(A|B), and joint probability, P(A&B).
12
Bayes’ Theorem
1. P(B|A)= P(A|B)*P(B)/P(A)
2. P(A) = P(A&B1) + P(A&B2) + … + P(A&Bn), where
B1, B2, …Bn are mutually exclusive and
exhaustive
We can rewrite part 2 if we use the fact that:
P(A&B) =
2’. P(A) =
P(A|B)*P(B)
P(A|B1)*P(B1) + P(A|B2)*P(B2) +
…+P(A|Bn)*P(Bn)
Part 1 says that we can calculate the conditional
probability in one direction, provided we know the
conditional probability in the other direction along
with the two unconditional probabilities. It can be
derived from the definition of joint probability,
which can be written backward and forward.
P(B|A)P(A) = P(A&B) = P(B&A) = P(A|B)P(B)
Part 2 says that if A can happen in conjunction
with one and only one of the Bi’s, then we can
calculate the probability of A by summing the
various joint probabilities.
There are numerous applications of Bayes’
Theorem. Aside from the two drilling prospects
mentioned above, one well-known situation is the
role Bayes’Theorem plays in estimating the value
of information, usually done with decision trees.
In that context, the revised probabilities
acknowledge the additional information, which
might fall into the mutually exclusive and
exhaustive categories of good news and bad news
(and sometimes no news).
Further, we can define P(~A) to be the
probability that prospect A fails (or in a more general
context that event A does not occur). A rather
obvious fact is that
Risk Analysis
 Risk Analysis:
Bayes’ Theorem

P(A) + P(~A) = 1, which says that

either A happens or it doesn’t. 0
Success B 60%
The events A and ~A are
0
e x h a u s t i ve a n d m u t u a l l y Drill B
40% 0
exclusive. Fail B
Almost as obvious is the 0
30%
similar situation Success A
P(A|B) + P(~A|B) = 1, which divest
says that once B happens,
either A happens or it doesn’t.
Drill A
Armed with these simple 3% 0
Success B
tools, we can point out some 0
common pitfalls in estimating Drill B
97% 1
probabilities for geologically Fail B
0
related prospects. Fail A 70%
divest
Pitfall 1: Upgrading the
prospect too much Divest
Suppose we believe the prospects
are highly dependent on each
other, because they have a
common source and a common potential seal. Figure 1. Decision tree showing conditional probabilities
Suppose P(A)=.2, P(B) = .1, and P(B|A)= .6
This is the type of revised estimate people tend
to make when they believe A and B are highly Pitfall 2: Estimating revised P(S)
correlated.The success of A “proves” the common after bad news
uncertainties and makes B much more likely. Suppose P(A)=.3, P(B) = .2, and P(B|A)= .6
But, consider the direct application of Bayes’ This is precisely the situation depicted in Figure 1.
Theorem: In other words, we believe the two prospects
P(A|B) = P(B|A)*P(A)/P(B) = (.6)*(.2)/.1 = 1.2 are dependent and upgrade the chances for B
Since no event, conditional or otherwise, can once A is a success.
have a probability exceeding 1.0, we have But what if A fails? Then, what is P(B|~A)?
reached a contradiction, which we can blame on Clearly, one should instead downgrade B’s chance of
the assumptions. success when A fails. Some people assign a value to
P(B|~A), but they should exercise great caution
What is the problem? when they do so. Here is why.
When two prospects are highly correlated, they Bayes’Theorem says P(B|-A)=P(-A|B)*P(B)/ P(-A)
must have similar probabilities; one cannot be But P(~A) = 1-P(A) = 0.7
twice as probable as the other. Another way of and P(~A|B)=1-P(A|B) = 1-P(B|A)*P(A)/ P(B) =
looking at this is to resolve the equations: 1-.6*.3/.2 = .1
P(A|B)/P(A) = P(B|A)/P(B), which says that the relative Thus P(B|~A) = (.1)*(.2)/.7 = .03
increase in probability is identical for both A and B. The point is that this value already is completely
determined from our other assumptions; we cannot
Decision tree interpretation just assign a value to it – we are too late. Not only
Conditional probabilities arise naturally in decision that, but .03 is probably much less than most people
trees. For any branch along which there is a would guess.
sequence of chance node, choice node, chance In summary, these two common examples
node, the branches emanating from the second point out both the power of Bayes’ Theorem to
chance node represent events with conditional provide us with informed decisions, and the ease
probabilities. Figure 1 shows a simple example of a with which casual estimates of our chances can
two-prospect tree. lead us. ◆

Risk Analysis 13
Risk Analysis:
Invoking Tools
Decision Trees vs.
Monte Carlo
Simulation
D
D ecision trees and Monte Carlo simulation are the
two principal tools of risk analysis. Sometimes users
apply one tool when the other would be more help-
ful. Sometimes it makes sense to invoke both tools.
After a brief review of their objectives, methods and
outputs, we illustrate both proper and improper
applications of these well-tested procedures.
Decision trees:
Objectives, methodology, results
Decision trees select between competing alternative
choices, finding the choice – and subsequent choices
along a path – which either maximizes value or
minimizes cost.
The output of a tree is a combination of the
optimal path and the expected value of that path.
Thus, decision trees yield one number. The rules of
solving the tree force acceptance of the path with the
highest expected value, regardless of its uncertainty.
In theory, preference functions can be used in
place of monetary values, but in practice, users
seldom go to this level.
Decision trees allow limited sensitivity analysis,
in which the user perturbs either some of the
assigned values or some of the assigned
probabilities, while monitoring the overall tree
value. Typically, the user varies one or two
parameters simultaneously and illustrates the results
with graphs in the plane (how the tree value
changes when one assigned value is varied) or in
space (how the tree value varies when two assigned
Trees, along with their cousins influence diagrams,
are particularly popular for framing problems and
reaching consensus.
14
values are varied together).The traditional tornado
chart also is used to show how each perturbed
variable affects the tree value when all other values
are held fixed. This chart takes its name from the
shape it assumes when the influences of the
perturbed variables are stacked as lines or bars, with
the largest on top.
Trees, along with their cousins influence diagrams,
are particularly popular for framing problems and
reaching consensus. For small to moderate size
problems, the picture of the tree is an effective means
of communication.
One of the most important problem types
solvable by trees is assessing the value of informa-
tion. In this case, one possible choice is to buy
additional information (seismic interpretation, well
test, logs, pilot floods). Solving the tree with and
without this added-information branch and taking
the difference between the two expected values
yields the value added by the information. If the
information can be bought for less than its imputed
value, it is a good deal.
Monte Carlo simulation:
Objectives, methodology, results
Monte Carlo models focus on one or more
objective functions or outputs. Favorite outputs
include reserves, total cost, total time and net
present value (NPV). Their respective inputs
include areal extent, net pay and porosity; line
item costs; activity times; and production
Risk Analysis
 Risk Analysis:
Invoking Tools

25.0%
big Recov
P0 P5 Mode P95 P100 200

Area 976.3 1,300 2,000 2,700 3,023.7 big Pay

25.0%
60
Pay 10.75 20 40 60 69.25 50.0%
med Recov
Recovery 76.9 100 150 200 223.1 150

25.0%
small Recov
Table 1. Defining parameters for inputs (Area, Pay, 100
Recovery) to Monte Carlo model 25.0%
big Area
2700
forecasts, price forecasts, capital forecasts and med Pay
50.0%
+
40
operating expense forecasts.
A Monte Carlo (MC) simulation is the process small Pay
25.0%
+
20
of creating a few thousand realizations of the model
99.0%
Success
by simultaneously sampling values from the input 0
50.0%
distributions.The results of such an MC simulation med Area
2000 +
typically include three items: a distribution for each
25.0%
designated output, a sensitivity chart listing the key small Area
1300 +
tree#1
variables ranked by their correlation with a targeted
1.0%
%
output, and various graphs and statistical summaries Failure
0
featuring the outputs.
Unlike decision trees, MC simulations do not Figure1. A decision tree begging to become a Monte Carlo model.
explicitly recommend a course of action or make a
decision. Sometimes, however, when there are obtained by following some path, multiplying each of
competing alternatives, an overlay chart is used to them by the corresponding probability, which is
display the corresponding cumulative distributions, obtained by taking the product of the branch
in order to compare their levels of uncertainty and probabilities, and summing these weighted values.
their various percentiles. In the tree, each parameter is discretized to only
three representative values, signifying small, medium,
Example: A tree that can be and large. Thus, area can take on the values 1,300,
converted to simulation model 2,000 or 2,700 acres. Of course in reality, area would
Sometimes a decision tree can be reconstructed as a be a continuous variable, taking on any value in
Monte Carlo simulation. This is especially true of between these three numbers. In fact, most people
trees with one root-decision node. The simulation would argue 1,300 is not an absolute minimum, and
would present the result as a distribution, whereas 2,700 is not an absolute maximum.They might be
the tree solution would only give the mean value of more like P10 and P90 or P5 and P95 estimates.We
the distribution.Take, for example, the tree in Figure can think of a small area being is some range, say
1, where the object is to estimate reserves. Note the from 1,000 to 1,500 with 1,300 being a suitable
pattern of a sequence of chance nodes without representative of that class. Similarly, 2,700 might
interspersed choices. This is a giveaway for represent the class from 2,500 to 3,000 acres. Each
conversion. Incidentally, a + sign signifies where a of these subranges of the entire range carries its own
node and its subsequent branches and nodes have probability of occurrence.
been collapsed. In this case, imagine copying the For simplicity, we have made all the triples of
node at “big area” and pasting it at the “med area” values symmetric (for example, 1,300, 2,000 and
and “small area” nodes.We shall convert this tree to 2,700 are equally spaced), but they could be
a simulation and compare the nature and extent of anything. For instance, area could have the values
information that these two models provide us. 1,300, 2,000 and 3,500 for small, medium and large.
In fact, the tree of Figure 1 is not a standard Likewise, we have assigned equal weights to the
decision tree. Instead of the convention of adding the small and large representatives, again for simplicity
values as one moves out along a path, this tree and ease of comparison.
multiplies them together to get barrels.The output is We have made another simplification: we assume
simply the expected value obtained by considering the all possible combinations are realizable. Sometimes,
possible combinations of area, pay and recovery the large value of area would be paired with three

Risk Analysis 15
Risk Analysis:
Invoking Tools

relatively large values of pay, in the between 0 and +1, for example, then during the
Distribution for Reserves/Sample/F6
belief these two parameters are simulation, realizations sampling larger values of area
dependent.This is simple to accom- would tend to be matched with larger values of pay.
0.100 modate in a tree. This technique is routine, simple to implement and
0.090 Meanv=11.96266
0.080 can use historical data when it is available to estimate
0.070
0.060
Building the corresponding the correlation coefficients.The resulting simulation in
0.050 Monte Carlo simulation model this case would feature more extreme values (both
0.040
0.030
Converting to an appropriate large and small) resulting in a larger standard deviation
0.020 Monte Carlo model requires and wider 90% confidence interval than the
0.010
0.000
finding a suitable distribution for uncorrelated case. Somewhat surprisingly, the mean
0 7 14 21 28 35
each of the three inputs – area, pay value will also increase, but not as much as the range.
5% 5%
4.85 20.88 and recovery. In light of the Introducing correlation and examining its effect
discussion above, we took the is a standard exercise in Monte Carlo classes. If
Figure 2. Histogram from simulation small values to be P5 and the big anything, the simulation handles these paired
values to be P95. We also selected relationships more easily than the tree, where the
triangular distributions is each case (which is how dependency is more hard-wired.
Percentage MMSTB we obtained our P0 and P100 values). The
5% 4.9 resulting distributions are shown in Table 1. Example: Comparing
10% 6.0 From the tree analysis,we can calculate the expected alternative mud systems
15% 6.8 value as well as finding the two extreme values (smallest Sometimes a problem can be described with a tree and
20% 7.5 and largest) and their respective probabilities: enhanced with a simulation.A case in point would be
25% 8.2 Expected value 12MMSTB a drilling cost estimate where there is a choice of mud
30% 8.9 Minimum value 2.6MMSTB, P(min) = 1/64 systems. Consider the following problem.
35% 9.5 Maximum value 32.4 MMSTB, P(max) = 1/64 For 4 years, you have been drilling in a field
40% 10.1 What we cannot determine from the tree analysis is where there is a danger of differential sticking. Of
45% 10.8 how likely the reserves would exceed 5 MMSTB, how the 20 wells drilled to date, six of them encountered
50% 11.4 likely t h ey wo u l d b e between 5 MMSTB and 15 stuck pipe. In five of those wells, the drillers fixed the
55% 11.9 MMSTB, how likely they would be less than 12 problem, spending from 4 to 18 days of rig time
60% 12.6 MMSTB, and so on. with an average of 9 days. In one well, they
65% 13.4 The histogram from the Monte Carlo analysis is eventually had to sidetrack at a marginal cost U.S.
70% 14.2 shown in Figure 2, and its corresponding percentiles $700,000 for materials and 12 days of rig time
75% 15.0 in Table 2.Thus, while the mean value coincides with beyond the attempt to free the stuck pipe, which
80% 16.0 the mean from the tree analysis (in part because of the had been 18 days. Average time to drill the wells is
85% 17.3 symmetry of the inputs and lack of correlation), we approaching 45 days.The rig rate is $60,000/day.
90% 19.0 learn much more about the range of possibilities: One option for the next well is to use oil based
95% 21.2 • 90% of the values lie between 4.9 and 20.9 mud, which would greatly reduce the chance of
MMSTB; stuck pipe (to 1/10) and the time to fix it (to 6 days),
Table 2. Percentiles for • there is about a 56% chance of finding less than but cost $70,000/day,
output (reserves, MMSTB)
from simulation 12 MMSTB (it is close to the median); Ordinarily, one would look to the decision tree
• there is only about a 20% chance of exceeding to compare these two alternatives. First, we calculate
16 MMSTB; and the expected value of drilling with the conventional
water based mud system for the three cases: no
Modifying the tree to account for problem, stuck but fixed and sidetrack.
dependency among inputs No problem cost: (45 days) * ($60,000) = $2,700,000
Consider the modified tree branches shown in Stuck and fixed cost: (45 + 9 days)*($60,000) = $3,240,000
Figure 3, where the user believes larger areas Sidetrack cost: (45 + 30)*($60,000)
correspond to larger net pays. Without going into + $700,000 = $5,200,000
detail, the corresponding Monte Carlo simulation The respective probabilities we assign are 14/20,
would handle this relationship by specifying a number 5/20 and 1/20.
between -1 and +1 to indicate the degree of Next, we estimate the costs and probabilities for
correlation between the inputs. If the correlation were the oil based mud system.

16 Risk Analysis
 Risk Analysis:
Invoking Tools

No problem cost: (45 days)*($70,000) = $ 3,150,000 P(StuckWater) 30.0%

Cost when stuck pipe: (45+6)*($70,000) = $3,570,000 P(SideTrack) 5% LowP 5
The respective probabilities are 9/10 and 1/10. P(StuckOil) 5% HighP 95
The resulting tree is shown in Figure 3, indicat- Sample Plow Mode Phigh
ing the correct decision would be to use the water DayRateWater $ 60 60 60 60
based mud with an expected value of $2,960,000 DayRateOil $ 70 70 70 70
rather than the oil-based alternative with an DaysWater_NoStuck 45.0 40.5 45 49.5
expected value of $ 3,171,000. DaysOil_NoStuck 45.0 40.5 45 49.5
DaysSideTrack 12.0 10.8 12 13.2
The Monte Carlo approach StuckWater? 0 1 = stuck
The Monte Carlo model shown in Table 3 captures SideTrack? 0 1=sidetrack
the same information as the tree model, but uses it StuckOil? 0 1 = stuck
differently. Specifically, the simulation model allows DayWater_Stuck 10.4 4 8 18
the user to estimate ranges for the various activities DaysOil_Stuck 7.2 3 6 12
and costs. For comparison purposes, the historical ExtraCost_ST 700.0
minimum and maximum values are treated as P5 and CostWater $ 2,700
P95 to build input distributions, with the option for CostOil $ 3,150
the user to set these percentages. Most people TimeWater 45.0
estimating ranges of this sort, especially cost TimeOil 45.0
estimators, tend toward right-skewed distributions.
Certainly, in this model, we should care about Table 3. Monte Carlo model to compare oil based and water based mud systems
more than just the expected value. For instance,
we may want to know the probability of exceed- analysis, and any user
ing a certain value, such as $4,000,000.We may also would be remiss in Decision Trees Monte Carlo simulation
want to question other inputs (see discussion avoiding this. But the Objectives make decisions quantify uncertainty
below). overall uncertainty is Inputs discrete scenario distributions
Figure 5 shows the comparison with the assump- made more explicit Solution driven by EV run many cases
tions taken from the problem statement. Figure 6 in the Monte Carlo Outputs choice and EV distributions
shows a variation where the oil-based mud takes only simulation, where Dependence limited treatment rank correlation
40 days on average to drill and the chance of sidetrack the very nature of
with water mud is 10%, not 5%. The difference the model begs the Table 4. Comparison between Decision trees and Monte Carlo
between the two cases is clear: as the probability of user to specify the simulation
sidetracking increases, the oil-based mud presents a range of uncertainty.
less risky alternative (the density function of On balance, if I had to pick one model, I would
outcomes is much narrower). Similar analyses can be pick the simulation, in part because I have had far
made easily in the Monte Carlo model. better results analyzing uncertainty and presenting
the details to management when I use simulation.But
Which model is better in this instance? whatever your preference in tools, for this problem
Both the tree and the simulation offer information the combination of both a tree and the simulation
to the user and to the larger audience of the seems to be the most useful.
presentations that can be prepared from them. The
tree emphasizes the two contrasting choices and Which model is better in general?
identifies the extreme outcomes and their prob- When is Monte Carlo more appropriate? In
abilities of occurrence. It is both simple and effec- general, Monte Carlo seems to have more varied
tive.Yet, the tree does not explicitly acknowledge the applications. Any (deterministic) model you can
underlying uncertainty in each of the contributory build in a spreadsheet can be converted to a
estimates (such as days for problem free drilling, cost Monte Carlo model by replacing some of the
of mud system, days for trying to free the stuck pipe, input values with probability distributions. Thus,
days for sidetracking). The user can handle these in addition to the popular resource and reserve
uncertainties – one or two at a time – by using the volumetric product models, people build Monte
tree framework to do a trial-and-error sensitivity Carlo models for costs of drilling and facilities,

Risk Analysis 17
Risk Analysis:
Invoking Tools

25.0%
big Pay
90 +
25.0%
70.0%
big Area No Problem
2700
2700
50.0%
med Pay +
60
Oil Mud TRUE Chance
small Pay
25.0% 0 2960
40 +
25.0%
25.0% Stuck/Fix
big Pay
80 + 3240
tree#1 5.0%
Stuck/Fix
Success
99.0%
0 med Pay
50.0%
+
Decision 5200
40
med Area
50.0% 2960
2000
25.0%
small Pay +
30
25.0%
big Pay
60 +
tree#1 95.0%
No Problem
3150
50.0%
med Pay +
25.0%
30 FALSE Chance
small Area Water Mud
1300
small Pay
25.0%
+
0 3171
15
5.0%
Failure
1.0%
% Stuck/Fix
0 3570

Figure 3. Tree incorporating dependence between area and pay Figure 4. Tree to compare oil based vs. water based mud systems

time for projects (up to a point, when project technical papers appear, many of these issues will
schedule software is more applicable), production be resolved.
forecasts, and all sorts of cashflows, including
those with fiscal terms. Decision trees must come Summary
down to something that compares alternative Table 2 summarizes some of the features of the
choices under uncertainty. two methods, illustrating their principle differ-
Distribution for CostWater/Sample/E22 ences. Don’t be surprised to find someone using a
Combining simulation tree solution to a problem you elect to solve with
1.600
1.400
E22: Meanv=2937.594 and trees simulation or vice versa. Do try to use common
1.200 E23: Meanv=3148.974 It is now possible to build a sense and do as much sensitivity analysis as you
Values in 10 -3

1.000 decision tree where some or can regardless of your choice. ◆

v

0.800
all of the branches from
0.600
0.400 chance nodes contain
0.200 probability distributions
0.000
2 3 4 5
rather than values.Then one
Values in Thousands can run a Monte Carlo
simulation, where each
Figure 5. Comparing oil and water based mud systems iteration is a new solution of
the tree. This technology is
Distribution for CostWater/Sample/E22 relatively new and relatively
untested. Although it holds
1.600
1.400
E22: Meanv=2833.63 promise, this combination
1.200 E23: Meanv=2826.767 requires the user to be
Values in 10 -3

1.000 expert at both decision trees

v

0.800
0.600
and simulation. Some people
0.400 raise questions of logic
0.200 when we solve a tree pop-
0.000
2 3 4 5 ulated with values that are
Values in Thousands
not mean estimates, but
Figure 6. New cost comparison when oil based mud rather extreme possibilities
results in faster drilling. of the inputs. In time, when

18 Risk Analysis
Risk Analysis:
Correlation
When Does Correlation
Matter?
W What is correlation?
Often, the input variables to our Monte Carlo
models are not independent of one another. For
example, consider a model that estimates reserves by
taking the product of area (A), average net pay (h) and
recovery (R).In some environments,the structures with
larger area would tend to have thicker pay.This property
should be acknowledged when samples are selected
from the distributions for A and h. Moreover, a database
of analogues would reveal a pattern among the pairs of
values A and h. Think of a cross-plot with a general
trend of increasing h when A increases, as shown in
Figure 1. Such a relationship between two variables can
best be described by a correlation coefficient.
Examples of correlated variables
Typical pairs of correlated parameters are:
• daily high temperature and air conditioning
cost for a personal residence in New Orleans;
• measured depth to total depth and drilling cost
for wells in the North Sea;
• porosity and water saturation assigned to
completion intervals for wells in a given reservoir;
• porosity and permeability assigned to completion
intervals for wells in a given reservoir;
• operating cost and production rate;
• the permeability-thickness product and the
recovery efficiency for gas wells in a given field;
• height and weight of 10-year old males in the
United States;
• propane price and ethane price on Friday
afternoon NYMEX;
• propane price and Brent crude price; and
• square footage and sales price of used houses
sold in Houston in 1995.
Definition of correlation coefficient
Two variables are said to have correlation coefficient
r if:
cov(x, y)
r
x * y
19
where:
and
Cov(1/n)  (xi x)*(yi y)
x varx
varx(1/n)  (xi x)2
Excel has a function, sumproduct ({x},{y}), that
takes the sum of the products of the corresponding
terms of two sequences {x} and {y}. Thus,
covariance is a sumproduct.
The value of r lies between –1 (perfect negative
correlation) and +1 (perfect positive correlation).
Although there are tests for significance of the
correlation coefficient, one of which we mention
below, statistical significance is not the point of this
discussion. Instead, we focus on the practical side,
asking what difference it makes to the bottom line
of a Monte Carlo model (e.g., estimating reserves or
estimating cost of drilling a well) whether we
include correlation. As we shall see, a correlation
coefficient of 0.5 can make enough of a difference
in some models to worry about it.
Before we illustrate the concept, we need to
point out that there is an alternate definition.
Two types of correlation - ordinary and rank
When correlation was formally defined in the late 19th
century, statisticians recognized one or a few points
with extreme values could unduly influence the
formula for calculating r. Specifically, the contribution
of a pair (xi,yi) to the covariance, namely
(xi x) * (yi y)
could be an order of magnitude larger than the
other terms.
Charles Spearman introduced an alternative for-
mulation, which he labeled “distribution-free” and
called it the rank-order correlation coefficient, in
contrast to the Pearson coefficient defined above.To
Risk Analysis
 Risk Analysis:
Correlation

obtain the Spearman coefficient, one replaces the

original data with their ranks then calculates the Table 1. Area, porosity, and gas saturation (and their ranks) for 13 reefal structures
correlation coefficient using the ranks.Tables 1 and 2 Name Area, km^2 Porosity Sg Rank_Area Rank_por Rank_Sg
provide a simple example illustrating both methods.
S1 10 0.12 0.77 3 1 3
M1 24 0.12 0.85 6 1 10
Impact of correlation A1 37 0.13 0.87 9 3 11
on the output of a model P2 6 0.14 0.81 1 4 6
What does correlation do to the bottom line? Does K3 28.8 0.14 0.91 7 5 13
it alter the distribution of reserves or cost or net D 6 0.15 0.61 1 6 1
U1 34 0.15 0.82 8 7 8
present value (NPV), which is, after all, the objec-
P1 13 0.16 0.81 5 8 6
tive of the model? If so, how? K2 60 0.16 0.78 12 9 4
We can make some generalizations, but remember K4 11 0.17 0.83 4 10 9
Oliver Wendell Holmes’s admonition, “No generali- K1 58 0.18 0.80 11 11 5
zation is worth a damn…including this one.” U2 48 0.22 0.75 10 12 2
Z1 108 0.22 0.89 13 12 12
First, a positive correlation between two inputs
will result in more pairs of two large values and
more pairs of two small values. If those variables are
Table 2. Correlation and Rank Correlation coefficients for the reefal structure data.
multiplied together in the model, for example, a
Ordinary r Area Porosity Sg Rank r Area Porosity Sg
reserves model, then this results in more extreme
Area 1 Area 1
values of the output. Porosity 0.67 1 Porosity 0.57 1
Even in a summation or aggregation model Sg 0.36 -0.02 1 Sg 0.28 -0.12 1
(aggregating production from different wells or
fields, aggregating reserves, estimating total cost by Each student is assigned a correlation coefficient,
summing line items and estimating total time), from 0 to 0.7, and uses it to correlate A with h and h
positive correlation between two summands will with r (the same coefficient for both pairs).The students
cause the output to be more disperse. run the simulations using both these correlations and
In short, in either a product or aggregation model, call out their results, which are tallied in Table Y:
a positive correlation between two pairs of variables
r mean StDev
will increase the standard variation of the output.
0 17.5 7.7
The surprising thing is what happens to the
.1 17.9 8.6
mean value of the output when correlation is .2 18.1 9.1
included in the model. .3 18.3 9.7
For product models, positive correlation between .4 18.6 10.5
factors will increase the mean value of the output. For .5 18.8 10.9
aggregation models, the mean value of the output is .6 19.0 11.3
not affected by correlation among the summands. .7 19.3 11.9
Let us hasten to add that many models are neither
pure products nor pure sums, but rather complex Now of course, even if both correlations were
algebraic combinations of the various inputs. warranted, it would be unlikely they would be
identical. Nevertheless, we must note the standard
Example 1. Correlating parameters in a deviation can increase by as much as 50% and the
volumetric reserves model mean by about 10% under substantial correlation.
A simple example will illustrate the properties of a The message is clear: pay attention to correlations
product model. in volumetric product models if you care about the
A common classroom exercise to illustrate the danger of easily underestimating the mean value by
effect of correlation on reserves uses the model: 5% or more, or if you care about understating the
N = AhR inherent uncertainty in the prospect by 30% or 40%.
With: Almost as important: often correlation makes little
A = Triangular (1000,2000,4000) difference in the results.We just need to check it out.
h = Triangular (20,50,100) While it may not be obvious, nearly all the
R = Triangular (80,120,200) correlations in reserves models cause the dispersion

Risk Analysis 20
Risk Analysis:
Correlation

(and the mean) to increase. It is far less common for the total cost to have a bigger range, but the mean
the correlations to have the opposite effect. cost is unaffected by correlation of any kind.
In more complex models, however, it is possible It should be noted that correlation can have a
for cor relations to serious impact on cost models in terms of contingency.
actually reduce un- Some people define contingency for the total project as
Positive correlation
certainty. One example the difference between some large percentile, say P85
120 we discovered several or P90,and a base value like P50 or the mean total cost.
100 years ago is a capital This contingency could easily grow substantially,
80 project involving con- though, because when we correlate line items, the
Pay

60 struction and operation mean (and to a large degree the P50) remains
40
of a gas-fired electric unchanged but the dispersion (typically) increases.
20
plant. The high positive
0
0 500 1,000 1,500 2,00 2,500 3,000 correlation between Estimating the correlation coefficient
Area price of natural gas (the Once you decide to include correlation in a model,
Figure 1. Correlation between Area and Pay
principal operating cost) you must find a way to estimate the coefficients.This
and price of electricity issue is similar to the more basic issue of what
(the principal factor in distribution to use for a given input variable. In both
Positive correlation revenue) will cause the cases, you can resort to one or more of the following:
0.24 output, NPV, to be far • empirical data;
0.22 more modest in range • experience; and
0.20 (on the order of 50%) • fundamental principles.
0.18
when the correlation is Sometimes we have adequate and appropriate field
0.16
0.14 included, making the data,in which case,we can calculate the (rank order) cor-
0.12 investment less risky. relation coefficient. Likewise, we can use curve-fitting
0.10 With a cost model, procedures (both popular Monte Carlo software pack-
0 20 40 60 80 100 120 adding cor relations ages have them) to generate the probability distribution.
Figure 2. Original data, r = 0.67 between line items is If we have no data or have appropriate data but
one of several refine- only a small sample, we should temper our auto-
ments that can increase matic processes with experience. What does this
R the range of total costs. kind of parameter usually look like: is it symmetric
13 Correlation coefficients or skewed, small or large? And for correlation, how
12
11 in this case are often do these two parameters usually relate to each other:
10
9 thought to be higher positively or negatively?
8
than those in reserve Once in a great while, we can appeal to that
6
models. For instance, higher power: fundamental principles. For example:
4
steel prices can influ- • “this parameter is a sum or average of other
2
ence numerous line parameters; thus it should be approximately
1 2 3 4 5 6 7 8 9 10 11 12 13 items, thereby resulting normal;”
Figure 3. Ranks of data r = 0.57 in a positive correlation • “this parameter is a product of other parameters
among the pairs. One – it should be skewed right;”
of our clients used a • “these two prices represent substitutable
Correlations for NPV
cost model with a matrix commodities, therefore they should be
Production1/D4 0.694 of risk factors vs. line positively correlated;” and
items. When the same • “these two parameters represent competing
Investment/D3 -0.528
risk factor influenced quantities and should be negatively correlated.”
Price1/D6 0.432 two or more line items, Remember, the correlation coefficient is limited
these items became to the range [-1,+1]. Anything in the range [-0.15,
Decline Rate/D5 -0.074
positively correlated. +0.15] could be noise and is unlikely to impact the
-1 -0.5 0 0.5 1
Positive correlation results. It is rare to have correlation coefficients
Correlation Coefficients (the usual type) among above 0.8 or below –0.8. So, make an educated
Figure 4. Sensitivity chart
line items will cause guess, and run the model with several alternate

21 Risk Analysis
 Risk Analysis:
Correlation

values and document the results. You may not Summary

resolve the issue, but you can defuse it. Things to remember:
• correlation is easy to
Another use of correlation: Sensitivity calculate in Excel (the
One argument for studying correlation is to better function Correl);
understand sensitivity analysis. Figure 4 shows one • there are two types
common type of sensitivity chart, sometimes called a of correlation coeffi-
tornado chart. In this case, the numbers represent rank cients: ordinary (Pear-
correlation coefficients between the target output son) and rank-order Figure 5. Negative correlation: investment and NPV
variable (in case a model has two or more outputs) and (Spearman). They
each of the inputs. A negative correlation coefficient tend to differ when one or both the variables is
indicates a model input that tends to increase as the highly skewed;
output decreases and vice versa. Thus, as capital gets • correlation might matter; it depends on the type
large, NPV tends to get small. The relationship is of model and the strength of the correlation;
properly described by correlation, because the other • correlation always will affect the standard
three inputs also affect NPV. Simply knowing that deviation, often but not always increase it;
capital is large does not force NPV to have a certain • correlation will affect the mean value of a
small value. But given a particular, large value of capital product; and
might limit the range of NPV to a smallish part of the • correlation is useful to describe sensitivity of
entire NPV range, a fact made clear by Figure 5. output to inputs. ◆

Risk Analysis 22
Risk Analysis:
Risked Reserves
Beware of
Risked Reserves
“R R isked reserves” is a phrase we hear a lot these days.
It can have at least three meanings:
1. risked reserves might be the product of the
probability of success, P(S), and the mean value
of reserves in case of a discovery. In this case,
risked reserves is a single value;
2. risked reserves might be the probability
distribution obtained by scaling down all the
values by a factor of P(S); or
Figure 1.Lognormal distribution for Prospect A reserves
Figure 2.Comparing the original distributions for A and B
23
3. risked reserves might be a distribution with a
spike at 0 having probability P(S) and a reduced
probability distribution of the success case.
Take as an example Exploration Prospect A. It
has a 30% chance of success. If successful, then its
reserves can be characterized as in Figure 1, a
lognormal distribution with a mean of 200,000
STB (stock tank barrels) and a standard deviation
of 40,000 STB. Then:
• definition 1 yields the single number
0.3*200,000 = 60,000 STB;
• definition 2 yields a lognormal definition with
a mean of 60,000 and a standard deviation of
12,000 (See Figure 2); and
• definition 3 is the hybrid distribution shown in
Figure 3. By contrast, suppose another prospect,
B, has a 15% chance of success and a reserves
distribution with a mean of 400,000 STB and a
standard deviation of 200,000 STB.Then under
definition 1,B would yield the same risk reserves
as A, 0.15*400,000 = 60,000 STB. However, consider
Figure 2, which shows how B would be scaled
compared with A,with the same mean but larger
standard deviation.And Figure 4 shows how the
original distributions compare.
Assigning these two prospects the same number
for the purpose of any sort of ranking could be
misleading. Prospect B is much riskier, both in the
sense that it has only half the probability of success
than does A, and also because even if it is a success, the
range of possible outcomes is much broader. In fact,
the P10, where P=Percentile, of Prospect B equals the
P50 of Prospect A.Thus, if you drilled several Prospect
A types,for fully half of your successes (on average),the
reserves would be less than the 10th percentile of one
prospect B.
The only thing equal about Prospects A and B is
that, in the long run, several prospects similar to
Prospect A would yield the same average reserves as
several other prospects like B. Even this is deceptive,
because the range of possible outcomes for several
prospects like A is much different from the range of
Risk Analysis
 Risk Analysis:
Risked Reserves

possible outcomes of B-types. For instance, if we

consider a program of five wells similar to A, there is a
51% chance of at least one success, and a 9% chance of
success on two or more. However, with five prospects
like B, the corresponding chances are 26% and 2% –
assuming they are geologically independent.

Running economics
What kind of economics these two prospects would
generate is another story. Prospects like A would
provide smaller discoveries more consistent in size.
They would require different development plans and
have different economies of scale than would
prospects like B.
So, does that mean we should run economics?
Well, yes, of course, but the question is with what Figure 3.Hybrid distribution for A showing spike at 0 for failure case
values of reserves do we run economics? Certainly not
with risked reserves according to definition 1,which is
not reality at all.We would never have a discovery with
60,000 STB. Our discoveries for A would range from
about 120,000 STB to 310,000 STB and for B from
about 180,000 STB to 780,000 STB (we are using the
P5 and P95 values of the distributions). So, surely, we
must run economics for very different cases.We could
take a few typical discovery sizes for A (or B), figure a
production schedule, assign some capital for wells and
facilities, sprinkle in some operating expenses and
calculate net present value (NPV) at 10% and IRR
(internal rate of return). My preference is not to run a
few typical economics cases and then average them.
Even if you have the percentiles correct for reserves,
why should you think those carry over to the same Figure 4. Original distributions for A and B
percentiles for NPV or IRR? Rather, I prefer to run
probabilistic economics. That is, build a cashflow ranking prospects. Moreover, the process would
model containing the reserves component as well as indicate the drivers of NPV and of reserves, leading
appropriate development plans. On each iteration, the to questions of management of risks.
field size and perhaps the sampled area might
determine a suitable development plan, which would Summary
generate capital (facilities and drilling schedule), The phrase risked reserves is ambiguous. Clarifying its
operating expense and production schedule – the meaning will help avoid miscommunication.
ingredients, along with prices, for cashflow. The Especially when comparing two prospects, one must
outputs would include distributions for NPV and recognize the range of possibilities inherent in any
IRR. Comparing the outputs for A and B would multiple-prospect program. Development plans must
allow us to answer questions like: be designed for real cases not for field sizes scaled
• what is the chance of making money with A or down by chance of success. Full-scale probabilistic
B? What is the probability that NPV>0? and economics requires the various components of the
• what is the chance of exceeding our hurdle rate model be connected properly to avoid creating
for IRR? inappropriate realizations.The benefits of probabilistic
The answers to these questions together with the cashflow models, however, are significant, allowing us
comparison of the reserves distributions would give to make informed decisions about the likelihood of
us much more information for decision-making or attaining specific goals. ◆

Risk Analysis 24