Managing Risk and Uncertainty.

by George Terhune

Danny asked me to express an outsider’s viewpoint, so I think it might be useful if I first said something
about what my viewpoint is based on. For nearly eleven years I’ve been a member of the Quincy
Library Group, but in real life, before retiring, I was an Air Force pilot for five years, then an airline
pilot for 25 years. In the last half of that airline service I was also a volunteer member of the Air Line
Pilots Association air safety organization, working primarily on the technical problems of landing in
very low visibility. I also participated in a major airline accident field investigation, and helped to
analyze a number of other transport category aircraft accidents.

In my view the job of an airline pilot can most accurately be described as “full time risk manager.” And
because I flew all over the world in a high risk environment, to the best of my knowledge without
scratching an airplane or damaging a passenger, I do claim some expertise in the management of risk
and uncertainty.

I think risk management involves three steps:

1. Define the risks.

2. Compute the risks.
3. Apply the results.

I’ll take those one at a time, using the Sierra Nevada Forest Plan Amendment (the “Framework”) as the
basis for discussion.

People often say “risk” when they mean “the probability of something bad.” I think a more useful
definition for our purpose is “the probability of an event multiplied by the magnitude of the loss if that
event occurs.” Two things about that definition:

First, I would still prefer to keep the two input quantities – probability, and magnitude of loss –
separately in mind as well as considering them together, because the appropriate management response
is often quite different, depending on whether the risk you’re dealing with is a very high probability of
very low loss – for example, playing a nickel slot machine for an hour – or a very low probability of
very high loss – like flying an airplane into a mountain. For today’s purposes, I’ll use the term “risk” in
the sense of combined probability and magnitude of loss.

Second, the Sierra Nevada Forest Plan Amendment gives a very different definition of risk, and
proposes what I think is a very questionable method of managing risk.

You may already be familiar with a statement near the beginning of Chapter 2 in the Supplemental EIS,
in a section titled A Consideration of Uncertainty and Risk in the Sierra Nevada Case. Quote:

“The important short-term risks facing the Forest Service are related to decision processes, not
ecological outcomes.”

How’s that again? The important short-term risks facing the Forest Service are not related to ecological
outcomes? I hope I’m not the only one who has trouble accepting that statement. How could they
possibly have reached that conclusion?
I think they went off track at least twice. The first time was when they defined risk as “...the probability
that any particular adopted management strategy will be seen as undesirable by the majority of
stakeholders…” Oh? Is it really the Forest Service mission to manage a popularity contest, not the
forest? And, if a poll is to be taken, are all “stakeholders” equal? Shouldn’t the people actually at
greatest risk have a larger say in how that risk is defined and managed? As a resident of the Sierra
Nevada, in a town surrounded by national forest, I’m not enthusiastic about having San Francisco
residents defining fuel reduction and fire protection policy for the Sierra Nevada, just as I expect they
would raise objections if I wanted to define San Francisco’s policy on how to prepare for earthquakes.

I believe the Forest Service is responsible for analyzing the risk as objectively as they can, then taking
action on that basis. NEPA requires explanation of the decision in an open process, but that does not
mean turning the decision itself over to the public, much less to a subset of self-proclaimed stakeholders.

Well, maybe this rationale from the same section of the EIS will clear things up. “If it is true both that
stakeholders are willing to consider short-term tradeoffs, and that alternatives under consideration are
indistinguishable in their short-term outcomes, then the focus of short-term risks must shift to concerns
with the decision process.” Since they earlier say that stakeholders usually are willing to accept some
short term disadvantages in exchange for long term gains, the crucial “if” here is the second one, that
alternatives under consideration are indistinguishable in their short-term outcomes. And I think that’s
the second place where the EIS goes off track in its discussion of risk.

When you compare projected outcomes for the various alternatives under consideration, they often don’t
show clear differences in ecological outcome for the first decade or two. It is often only at 30 to 50
years that significant advantages of one alternative over another will become apparent on the ground.
But the inability to distinguish short term differences does not mean there is no difference among
alternatives at the beginning. When you start on the trajectory defined by one of those alternatives,
you’re stuck with it for a while. You can’t just jump from a bad trajectory to a good one at the 20 year
marker, you have to choose your horse at the beginning of the race.

In short, the discussion of risk in the Supplemental EIS is plain wrong. The Forest Service is required
to make decisions based on analysis of the projected ecological outcomes, not by conducting a poll or by
trying to make the decision process run more smoothly.

OK, the definition of risk got off to a bad start. How about the computation of risk?

On the whole, I think the processes used to identify the hazards and compute the probabilities were
reasonably good. The major weakness here is in the lack of adequate, up-to-date information to put into
the computations. I’ll say more about the information problem later.

My main concern is for the third step in the process: how the projected outcomes and the associated risk
analyses were used in the decision itself. I think this is the most crucial link in the chain, and
unfortunately I think it was also the weakest link in the Framework decision process. I’ll illustrate my
concern with one example.

The Forest Service fuel reduction strategy for most of the Sierra Nevada is called Strategically Placed
Area Treatments (SPLATs). The EIS says this concept is based entirely on a paper titled Design of
Regular Landscape Fuel Treatment Patterns for Modifying Fire Growth and Behavior, written by Mark
A. Finney, and reproduced in the Final EIS of January 2001. Finney’s paper is for the most part well
reasoned, includes appropriate qualifiers and cautions, and is clearly written. The problem is that the
patterns of treatment analyzed in Finney’s paper cannot be implemented on the landscape, and even if
they could, the SPLAT strategy is not legitimately based on that paper. The whole thing was corrupted
in the decision process. Here are three examples where Finney’s analysis was significantly
misrepresented in the EIS and badly misused in the decision:

1. The effectiveness claimed for the pattern of treatment described in Finney’s paper depends very
heavily on the regularity of the geometry, but Finney cautions in his own paper that “…the idealized and
artificial treatment patterns would probably never be achievable or even desirable in practice.” Also, the
Forest Service has published its own illustrations said to represent typical SPLATs, but these SPLAT
patterns don’t comply with any of the spacing and overlap requirements stated by Finney as necessary
for an effective pattern.

2. The only actual example analyzed in Finney’s paper, which the Forest Service cites as evidence of
SPLAT effectiveness, is based on treating fuel so as to reduce a fire’s rate of spread by 90 percent in the
treated areas, whereas the Forest Service Standards and Guidelines require only a 50 percent reduction
in the rate of spread in treated areas. This makes a huge difference in the theoretical effectiveness of the
treatment pattern. If 30% of the landscape were treated, Finney’s example shows a 77 percent slowing
of a fire as it moves through his pattern of treatments, whereas that fire would be slowed only 23 percent
if the Framework Standards and Guidelines were implemented on the same pattern. And when you
apply the Forest Service standard of treatment to one of their illustrations of an example SPLAT pattern,
hardly any slowing of a fire is achieved at all.

3. And in fact the actual performance of a Finney pattern wouldn’t ever be as good as Finney’s graph
implies. The only quantifiable lines on that graph of effectiveness refer to continuous strips of
treatment, not to the pattern of disconnected SPLAT treatments that are said to be supported by the
analysis. All patterns of disconnected treatments would perform significantly less well. So, even in
theory, SPLATs would always perform a lot worse than the same amount of treatment done in
connected continuous strips.

I could go on – and in fact I have gone on at length in a paper I would be glad to give you a copy of, if I
can persuade you to take one – but time is too short to keep talking about it, and I think you get the idea.

Considering these distortions, misuses, and misrepresentations of its own information sources and
analyses regarding wildfire behavior, which is one place the Forest Service has a large body of
respectable information and analysis, is there any hope that real progress can be made on this or any
other issue that involves some uncertainty and a high level of risk?

It’s hard to be optimistic, but I think some improvement is possible, and I’d like to make two small
suggestions to get started in that direction.

First, regarding the shortage of information I mentioned earlier, there must be real incentives to collect
and analyze the information that managers need for good decision-making. The problem is that most
forest processes move very slowly, so time and money will be needed to record and preserve a lot of
current information, even when that information doesn’t have any immediate use. I see a lot of lip
service on “adaptive management,” but I don’t see significant efforts to obtain and preserve the
information that is required to make it work. Making adaptive management actually work will require
Forest Service commitment of significant resources to the improvement of its information systems. As
part of this effort it will also be necessary to create attractive job slots and career paths that recognize
the importance of collecting, preserving, and analyzing that information.

Second suggestion. The Forest Service has to make it harder for a manager to off-load responsibility.
“Didn’t happen on my watch” is the oldest and lamest excuse in the world, but it is inevitable that many
Forest Service line officers will operate on that premise in the current system, whether they admit it to
themselves or not. Part of the problem can be seen in the quote I cited earlier: “The short term risks
facing the Forest Service are related to decision processes, not ecological outcomes.” That sounds a
whole lot like “For now, do the process, don’t worry about the forest.” When evaluations and rewards
are seen to depend on short term bureaucratic processes instead of ecological outcomes, there won’t be
very many people willing to risk their careers on gathering expensive information and making tough
decisions today that might not pay off for several years, or maybe even decades.

Furthermore, the “not on my watch” problem is made worse by the practice of shifting line officers from
place to place on a schedule often shorter than the life-span of the projects they are supposed to be
managing. After more than ten years as a very active member of QLG, I can think of only two people
now serving as line officers or deputies, in the Region or in one of the three Forests or seven Ranger
Districts in the QLG area, who have been there more or less from the beginning. Most of those line
officers have been in place only a few years. It’s very hard to have real accountability in the
management of a slow-moving process like the forest, when the Forest Service moves the responsible
officials in and out so quickly.

Finally, I think the attempt to shift responsibility for risk to somebody else is nicely illustrated by
another small example. This example is almost mindlessly simple, but for that very reason I think it’s
all the more disturbing.

At this summer’s County Fair Parade in Quincy, Smokey the Bear was introduced with the same old
slogan, “Remember, ONLY YOU can prevent wildfires.” And Smokey isn’t just saying that in back-
woods towns like Quincy, you can find the same thing right up to date on SmokeyBear.com.

OK, maybe in a very narrow sense you could argue that most of the “preventable” ignitions are
those caused by humans, so Smokey’s slogan may be true, in the same sense that most of the
advertising we see on television is “true.” But if you want Smokey to represent the Forest Service,
not Madison Avenue, then maybe he should tell the real truth about wildfire: the problem isn’t the
number of ignitions, it’s the potential for any particular ignition to become a catastrophic fire.

To make an honest statement on the real issue, Smokey would have to say something like
“Remember, ONLY WE can prevent catastrophic wildfire.” Now wouldn’t that make an interesting
commercial?

George Terhune
939 Bucks Lake Road
Quincy, CA 95971

gterhune@digitalpath.net
530-283-0899