Swinging on the Branch of a Tree: Project

Selection Applications

SIDNEY W . HESS Management Department

College of Business and Administratiou
Drexet University
Philadelphia, Petmsylvania 19104

Selecting research and development projects is always difficult

because data for assessment are seldom available. ICI Americas
used simple decision trees successfully to screen new product
ideas and decide on a major effort in new process research. The
new process research model incorporated only 10 variables,
making estimation feasible; a visual sensitivity analysis gave
management confidence in the model conclusions. The screen-
ing model incorporated qualitative selection criteria (for exam-
ple, quick development, low risk, patentable, low capital, high
sales) into a single quantitative expression of expected net pres-
ent value. ICI Americas used it to select projects from 53 new
product ideas. These examples illustrate the value of a logical,
simple model even when data are sparse.

R esearch and development (R&D) ac-

count for three percent of the devel-
oped world's economy, about $200 billion
in the US alone. Managing R&D activities
in both the public and private sectors is es-
pecially difficult because they entail vast
uncertainties.
Management Science and R&D
Management scientists have been doing
research on research for over 30 years
[Hall and Nauda 1990; Tornatzky and
Fleischer 1990]. Prior to the 1992 competi-
tion, no R&D application (project selection
or otherwise) has ever competed in, let

Copyright *!) 1993, The Institute of Management Sciences RESEARCH AND DEVELOI'MENT—I'ROJbCT SELECTION
0091-2102/93/2306/0005501,25 INDUSTRIES—CHEMICAL
This paper was rtfi/rt-fd.

INTERFACES 23: 6 November-December 1993 (pp. 5-12)

 alone won, the Edelman Award for Man-
agement Science Achievement.
Organizations regularly use project man-
agement techniques, for example, PERT
which was developed by management sci-
entists [Liberatore and Titus 1983]. Risk
analysis and decision analysis are also con-
tributions of our profession. Managers
Join me in swinging on a
branch of a (decision) tree.
tend to use project management, risk anal-
ysis, and decision analysis models near the
end of the development process, when
they know what needs doing and when
market uncertainties become more impor-
tant than technical uncertainties [Booker
and Bryson 1985].
Management science has failed alto-
gether to implement budgeting and project
selection models. We have proposed more
and more sophistication with less and less
practical impact [Hall and Nauda 1990].
The mathematical programming models
we have developed {linear, integer, dy-
namic, or whatever) require data that R&D
cannot practically provide.
Project Selection
One of the more vexing problems in
managing R&D is to decide whether to ini-
tiate a particular project. Information for
such decision making is sparse. Since every
new idea must be consciously rejected, ac-
cepted, or tabled for more information, the
number of these decisions is very large.
Most ideas are rejected intuitively as a re-
sult of informal peer discussion or one or
more levels of supervisory review.
Before it devotes its resources to an idea
INTERFACES 23:6
that has not been rejected intuitively, most
organizations require that it pass formal
screening. The number of proposals that
require screening is still large. Because they
lack data, most organizations use qualita-
tive screening methods: a check list of tar-
get product or process attributes or some-
times a scoring model [Liberatore and Titus
1983],
Management Science and Project
Selection
Management science has had little influ-
ence in R&D project selection; yet in mak-
ing this decision, organizations pick the
targets they will devote their resources to.
Someone said, "Better to do the right thing
poorly than the wrong thing well!" Man-
agement science can help us do the right
thing.
During my years at ICI in line and R&D
management, 1 found decision analysis to
be exceedingly useful in selecting and
screening projects. Sequential decision
making is very characteristic of R&D. R&D
is a poker game: Do we draw another
card? Do we buy information in the labo-
ratory? Do we conduct low-cost research
knowing that expensive decisions can be
made later?
We can use a simple decision tree to dis-
play and clarify this sequential logic. With
some care, we can keep the dimensionality
low, incorporating only the most important
parameters and uncertainties. We can
overcome the estimating problems and
make the model and its conclusions trans-
parent to management. Simplicity is the
virtue [Liberatore and Titus 1983;
Rubenstein 1989].
I will describe some applications in
which I participated, not as an analyst, but
PROJECT SELECTION

as part of management. Whether you are a
manager or management scientist, join me
in swinging on a branch of a (decision)
tree.
Example One: Develop a New Process?
ICl's Canadian subsidiary discovered a
new, but unpatentable, application for an-
thraquinone (AQ), an agent for reducing
pulp mill water pollution. AQ reacts with
certain paper and pulp waste products and
forms solids that can be filtered from the
mill effluent. If proven on a commercial
scale, the world market would be im-
mense.
AQ was distilled from coal tar and used
principally in dyes. Coal tar is a by-prod-
uct of coke production, and coke is only
used to make steel. Therefore, the capacity
for AQ was directly related to the demand
for steel. All the AQ produced in the world
might satisfy only a small fraction of this
potential, but unconfirmed, market. One
chemical competitor, BASF, was known to
have an alternate process for synthesiz-
ing AQ,
A small market would benefit coal tar
producers, not ICI. A large market and our
own proprietary AQ process would benefit
us immensely. Management had no inter-
est in half-way efforts. For example, con-
firming the new market without develop-
ing our own AQ process would give BASF
an uncatchable head start.
We had to decide whether to accelerate
new process research and commercial trials
or stop the modest existing effort.
Conventional analyses would call for
economic evaluations based on cash-flow
projections for construction and operation
of a plant using the new process and sell-
ing to the postulated market. The detailed
numbers were just not available. And un-

Net Present Values

Millions $

V Value if successful $25,0 ± 50%

D Process development expense $3.0 ± 25%
(including presanction engineering)
Cl Commercial development expense $0.5 ± 25%
(before sanction)
C2 Commercial development expense $1.0 ± 25%
(after sanction)
M Market development expense $0.2 ± 25%
R Research expense $0.8 ± 25%

Conditional Probabilities
Pc Probability of commercial success 0.8 ± 0,2
Ps Probability of sanction 0.8 ±0.2
P^ Probability of technical success 0.6 + 0.15
PM Probability of significant market 0.6 ±0.15

Table 1: The AQ project required research for a new manufacturing process and development
of AQ's potential to reduce pulp mill water pollution. We made estimates of the value of the
project if it were successful and the costs and probabilities prior to launch.

November-December 1993
HESS
certainty and risk are not well treated in
such analyses.
The primary risks were the following:
—Would market tests confirm a significant
market?
—Could we develop a new process?
—Even if the answer to both of these
questions is yes, would the ICI board
sanction the new plant capital when
economic conditions were bad and capi-
tal budgets severely curtailed?
—If it were sanctioned and built, would
the venture succeed?
We estimated conditional probabilities and
"confidence limits" (about plus or minus
25 percent) for affirmative outcomes for
each of these risks.
We estimated the following economic
parameters:
—Research expenses to identify a new
process;
—Market development cost to find out if
there was a significant market;
Alternate
process
research 4
market
development
Figure 1: The decision tree for AQ process research and development and AQ market
development considered all failure branches to be zero. Sanctioning by the ICI Board was also
uncertain.
INTERFACES 23:6
—Commercial development expenses be-
fore and after sanction;
—Venture value if successful, specifically a
net present value of $25 million plus or
minus 50 percent; and
—Venture value if unsuccessful, zero . . .
essentially a break-even.
Table 1 shows the parameter and probabil-
ity estimates with their confidence limits.
We cast the above into a decision tree in
which all failure or no-go outcomes were
zero (Figure 1). The expected net present
value (ENPV) of the project was easily
computed by calculating back through the
decision tree. (The equation for ENPV is in
the appendix). Using best estimates from
Table 1 parameters, we obtained an ENPV
of $3.2 million.
But the parameter estimates were clearly
subject to great uncertainty, plus or minus
25 percent in most cases (Table 1). Were
probability distributions assumed for the
nine best estimate parameters in Table 1,
Commercial
development
(continuation)
Commercial
success?
Sanction?
Stop
value ^ SO
PROJECT SELECTION

we could have used a Monte Carlo simula- than do a simulation.

tion to develop the NPV distribution. This
technique would be more useful if better
estimates of parameters and their distribu-
tions were available.
More important, management was un-
comfortable estimating probabilities, let
alone expressing levels of uncertainty
about them. Monte Carlo simulation
would likely raise management anxieties
about probabilities even further. We
elected instead to explore the optimistic
and pessimistic parameter values rather
We easily derived simple algebraic
expressions for ENPV and conditions for
positive ENPV by referring to the decision
tree (Figure 1 and the appendix). The con-
ditions for positive ENPV could then be
displayed as a break-even graph delineat-
ing go and no-go regions. The intent was
to provide visual evidence that uncertain-
ties in estimating the probabilities of tech-
nical success and a significant market
should not render the decision analysis in-
valid.

1.0 r-

GO
Boundaries of Probabilities
I I

.6

Low Payoff

Higher R&D

CO NO GO Best Estimate
O

.2 .4 .6 1.0
Probability of Technicai Success
Figure 2: We felt confident thai the probabilities of technical success and significant market lay
within the dashed rectangle. The curve separated go and no-go regions based on our best esti-
mates of all the parameters and other probabilities. Since the rectangle was well within the go
region, a decision should not depend upon our uncertainty about the probabilities of technical
success and significant market.

November-December 1993
HESS

We plotted the break-even condition us-
ing best estimate parameter values with
probabilities of technical success and a sig-
nificant market as coordinate axes (Figure
2). The region of uncertainty for these two
probabilities, 0.6 plus or minus 0.15 for
each, could then be visually compared to
the break-even curve. Finally, other combi-
nations of parameter estimates defining al-
ternate break-even curves could be viewed
as well (Figure 3),
These visual presentations were a signifi-
cant aid in moving the project forward.
Market development and process research
accelerated. Unfortunately, field tests in
existing pulp mills did not live up to ex-
pectations and the entire project was
cancelled.
Example Two: Screening New Product
Ideas for Development
ICI America s smallest division was
Aerospace, The division needed new prod-
ucts to survive within a fast growing com-
pany.
We organized a brainstorming exercise
with divisi(,)n and corporate representatives

1.0 _

Boundaries of Probabilities
.8

0)

(0
.6

I -2
re
Best Estimate
o
a.

.2 .4 .6 .8 1.0
Probability of Technical Success
Figure 3: The indifference curve moved only slightly when upper limits of R&D cost estimates
were incorporated into its calculation. Using the lowest estimate of payoff just brings the
indifference curve into the rectangle of probability uncertainty.

INTERFACES 23:6 10
PROJECT SELECTION
from marketing, production, and research.
The group suggested 53 new products that
they later discussed and screened using cri-
teria of their own choosing:
Development:
—Low technical risk;
—Quick, preferably less than one year;
—Inexpensive, preferably less than 10 per-
cent of sales;
Sales:
—^High growth;
—Preferably more than $1 million per year
by the fifth year;
Profitability:
—Preferably more than 20 percent of
sales;
Capital requirements:
—^Low (capital was still tight).
Discussion was rapid and unstructured.
We did not record opinions on the individ-
ual criteria, only an overall rating. Position
or personality seemed to sway the review.
Ultimately the group sorted the candidates
into three categories; the top category in-
cluded eight candidates deserving immedi-
ate attention. We were pleased with the 53
suggestions and the criteria, but many of
us were dissatisfied with the screening pro-
cess.
As an alternate screening procedure, we
incorporated the criteria in a decision tree
similar to that shown in Figure 1. Again,
we expressed expected net present value
algebraically (appendix). We estimated the
value from a successful project as the
product of
—Profit margin as a fraction of sales,
—Average annual sales over the first 10
sales years, and
—An integral that summed the discount
factors for these same 10 years of sales.
November-December 1993
Other parameters included the probabili-
ties of technical and commercial success
and costs to prove technical feasibility and
to commercialize.
Estimating specific parameter values was
difficult. Our need, however, was not pre-
cision but gross screening. We used a
three-point scale for each parameter, as-
signing a nominal value if we had no other
basis for choice (for example, our typical
project's time from development to launch
was 2.5 years). We assumed a significantly
higher value if prior knowledge warranted
it (for example, we assigned a project a
higher profit margin if the product should
be patentable, say 35 percent), and a lower
value if prior knowledge warranted it (for
example, if the project was deemed diffi-
cult, we assumed a low probability of tech-
nical success, 0.2).
Hess [1985] gives further details, includ-
ing a comparison of alternate estimating
techniques, a description of specific prod-
uct suggestions, and results.
The important thing is that the simple
decision tree worked again. It provided a
logical consistency among the criteria not
possible by using a check list or arbitrary
weighting.
Conclusions
Management science can make an im-
pact in R&D. We already contribute at the
downstream end of the development pro-
cess where data are available to feed our
elaborate models.
We also can contribute to project selec-
tion where information is scarce and deci-
sions critical. The discipline of a logical
model provides a better basis for selecting
projects than intuition, a check list, or the
power of personality. Decision analysis
11
HESS

captures the sequential decision process ary), pp. 3-9.

characteristic of R&D and the uncertainties Hail, David L and Nauda, Alexander 1990,
"An interactive approach for selecting IR&D
of outcomes. But to be successful, the deci- projects," /£££ Transactions on Engineering
sion tree must be simple, simple, simple. Management, Vol. 37, No. 2 (May), pp. 126-
Management scientists helping R&D 133.
management, or management in any func- Hess, Sidney W. 1985, "Rational ranking of de-
velopment candidates," in Managing High
tional area, must be unafraid to use simple Technology: An Interdisciplinary Perspective,
models when lack of data precludes more eds., B. W. Mar, W, T. Newell, and B. O. Sax-
rigor. Swinging on one branch is more fun berg, Elsevier Science Publishers Besloten
than standing on the ground watching. Vennootschap, Amsterdam, The Netherlands.
Liberatore, Matthew |. and Titus, George |.
Acknowledgments 1983. "The practice of management science
Many of my ICI Americas colleagues in R&D project management," Management
contributed to the work reported here. 1 es- Science, Vol. 29, No. 8 (August), pp. 962-974.
Rubenstein, Albert H. 1989, Managing Technol-
pecially wish to acknowledge Ash Pai from ogif in the Decentralized Firm, John Wiley and
corporate operations research and Don Sons, New York.
Stonestrom who was my director of devel- Tornatzky, Louis G. and Fleischer, Mitchell
opment in the Aerospace Division. 1990, The Processes of Technological Innova-
tion, Lexington Books, Lexington,
APPENDIX Massachusetts.
Anthraquinone Project
Using the parameter definitions in Table
1 the project's expected net present value, Author's Note: Readers interested in the
application of decision analysis to R&D
ENPV = PMPT[P,(P,V - Q) - D - Ci] management should see the recent special
-R- M. issue of Interfaces, Volume 22, Number 6,
Research should continue if ENPV > 0, e.g. November-December, 1992.
if
> + M)/{Pj[P,{P,V -C,)-D- C,]}.

Aerospace Product Screening

ENPV is the same as abt)ve with
Ps = \ since sanctioning is not an issue,
and where
kS e "dt. and
JT--Q
k = the profit margin,
S = the average annual sales for the first
10 years from launch,
T ^ years to launch, and
I = 0.15, the discount rate.
References
Booker, |ane M. and Brysun, Maurice C. 1985,
"Decision analysis in project management:
An overvitw," IEEE Transactions on Engineer-
ing Managemoit, Vol. EM-32, No. 1 (Febru-

INTERFACES 1'3:6 12