dence ki structure, with

feedba On
by
vant making about R&D
rojb
uls
etm s o f r e l e
that
making about
improvement in decision
sible
such
roble T h i suggestsmight best be addressed by simulation approaches
d. Thvement might
solved. improvement

in Section 11.9.
soss describe
proces that
we
thosee

as

VISUALIZATION AND SIMULATION

DLICATIONS OF
PPLICATION
IMPROVEMENT
PLANNING AND PROCESS
TO R E SARCH
11.9 EARC
barriers
information c a n overcome many
The visual display of quantitative
a l dis are
interactive
The most powerful of these pictures
18]. ding and achieve direct
to und

the user can explore their assumptions

ones. nwhich
in w h
numerical
n umerical
feedback.

sections therefore review both pictures and

fecThe following communicate how value can best be created by
help to
aches that can
uncertainties involved in pre-
a p p r o a c t

Because of all the

despite uncertainty. be learned from different kinds
&D,
future, and predicting what might incor
the must explicitly
dicting effective research performance models
all
of research, all believe that
ofresearch, about s o u r c e s and levels
of uncertainty. We
research
rate assumptions area
r esearch
for improvement in
the single most important
this is
main areas:
simulations.

management refine its thinking in four

Simulations can help
that must be pulled" to
1. Identifying
the of value and the "levers
sources

call these "R&D value drivers'").

change performance (we
for a given pattern of attrition,
2. Assessing throughput and bottlenecks, investment
decisions o n resourcing and technology
to help longer-term processes and
decision making
and to study means of accelerating
cycles.
research tasks need to iden-
managers planning
3. Managing risk-project s o u r c e s of uncertainty
and work
tify the most important outstanding in the R&D
at any given point
out which most need resolution
process.
between quality and throughput
Vaking the right strategic trade-offs and how
out how to enrich o r filter the forward pipeline
wnen
working
 SUPPORT
M
S I N M U L A T I O N
CAN
HOW enrichment" (e a
.. library Pu
262 o
"option
n filtering"
( p r e d i c t . a r y ,

filtering" (predictive
to
resource
these

series
processes

extension)
or "option
COmpula.
chase. lead methods).

and assay
tional

Drivers
Value
R&D kind of investment No
11.9.1
be seen as
some

is ot "Ctu
all of
R&D can
at which
the n e w drug
sold
other than
Ultimatelv.
until the point further here.
obtained
is considered

is routinelyusedtoaasses
income which not
out-licensing., v a l u a t i o n s are
through financial
ps less familiar
familiar
Within drugdevelopment, For drug discovery groups
with
entire portfolios. of marketing
of marketinoe
and from the processes
and
projects who are a long way unfamiliar and.
methods, are
financial
measures of process
performance
within R&D imnr
may
sales.
financial
sometimes lead
to a focus,
resistance. This can understood through init
generate what can easily be
initiatives, on improving compression. Such focus acan
ment time
development
or poor performance, suek
or
tives such as cost cutting
root c a u s e s of good
from the
take attention away and the consequent chanceso
of leads and candidates value in the
as the quality profile the desired high the
or of obtaining
failing in development
market. manager that if a druo
to us by a discovery
An example is a
view given the next best thing for
reach a successful market launch,
candidate could not to late stages of
would be to have their compound progress
team
a discovery
because this implies
that any problem was hard
development before failure,
Yet this is exactly the most expensive
a rare event.
to predict, for example,
kind of failure.
visualizations
Value driver development of financial analysis
analysis, a
an intuitive, graphical way
of breaking
first created at DuPont [19], provides stochastic
down the sources of value (Fig. 11.4)
and, in conjunction with
contribution
can be used to quantify the likely
models of the project process,
of different kind of change.
interactions aid understanding can
Graphical representations of complex
For the selection ot
and show the need for integrated decision making.
simulate pipeline
optimum strategies, managers can also visualize and be
volume and quality. The two are inextricably linked, and one cannot
to be
changed independently of considering the other if optimum value is
derived.
Costs in R&D are a mix of short-run cost, dominated by the cost of clinic
trials, and long-run cost, dominated by the fixed assets and specialized exper
tise of discovery and preclinical organizations. Therefore, when seeking
maximize the value of output relative to input, development teams ed o
be concerned with external
nevarch
spending and the length of whereas tasks, resea
teams need to look at their investments in capacity and the way they ude fixed
capacity.
 k n o w i n gt h e
rightquestions
J

competivinteres
enestcos3

8
 with it
needthe
ed to H
of drug discovery brings to
The industrialization to the business objectives
dimensioned according
build facti
ties that are from the
"Big Pharma" has been adopting techniques manufactur
creasi
uring n gy,
decisions on facility sizing, staffing, and scheduling o
wor
to help guide
event simulation (DES)
is a long-established technique within tho Discrele
a version of a proposed or
or
research (OR) field, in which computer
conduct studies to
pOSed existing
system can be built and then used to underst: nd the
impacts of changes to the system. Typical situations that are best su
where randomness, variabili
study with DES models would be those
oritization rules, breakdowns, and interprocess dependences are strongfac
ity, Ppri
[20]. These simulation models typically employ animation as
as aa mean
means
illustrating the process flow and bottlenecks and also provide quantita
analysis. Figure 11.5 shows a typical high-level model for R&D project pr
tive
gression. set up in the business modeling environment Corporate Modele
(CASEwise Corporation). Other toolsets such as Promodel or Arena ofer
finer control over simulation variables and scheduling prioritization rules,
Our colleagues have applied DES in four different pharma companies to
help optimize high-throughput screening (HTS) so as to "debottleneck"a
process. The simulation must represent the type of compound storage and
retrieval equipment, screening platforms, and staff needed to
support screen-
ing a given annual number of targets against a given compound library. The
simulation model helps to answer tactical questions around the
ferent work patterns (24/2 vs. 3 daily shifts,
impact of dif
5-day vs. 7-day working) and also,
at a more
strategic level, identifying which new investments in technology
would best increase total throughput. DES
approaches are also well suited to
simulation of processes such as adverse event
New combinations of simulation and
handling.
optimization allow a much wider set
of R&D decision making
processes to be optimized, at least approximately,
than in the past. This exciting new
approach has been developed to solve
many problems thought to be impossible to solve with
The idea is to trade off the exactness of the
classic options analysis.
look at more complex situations optimization for the ability to
(21].
This approach retains the capability to deal with
uses decisions but
a
approximations to the value of project, rather contingent
Decisions are than exact calculations.
improved iteratively, first improving the to use
by simulating the evolution of the under a
approximation
decision making and then
project given set of for
improving the policies for decision policies
optimizing the (approximate) value at each point in time of the making by
Blau et al. [22] have
needs and success
applied probabilistic network models to simulation.
model
probabilities in pharmaceutical and agrochemical resource
develop
ment, through Monte Carlo
a
analysis. This requires solving the problem oi
scheduling portfolio of projects under uncertainty about
progression. This
approach is tractable for drug development, HoweYer. "the inha
EwProlec
start

Pegulatiory
Submieion
arget
identification Compound library
synthesis and
management Acute toxicity
Initial safety
assessment Approval for
Target validation assessment use
Managenment of
screening collections
and materials
DMPK assessment Proof of concept in
small-scale trials
Assay developmernt
HTS/hit of effectiveness
identication

Chronic toxicity Large-scale clinical

Lead series assessment trials
identification

Lead optimization Scale-up

Formulation
Candidate Drug
recommendation

Analytical methods|
developoment

each stage of work.

Figure 11.5 A high-level attrition model represent the success probabilities and time taken
must at

the return loops where rework is necessary, and the criteria for abandoning a project entirely.
 d
Ity and creativity of
ol
discovery...
discOvery... makes
makes it difhcult
ties" within such models difficult
to capture
applied to capture
research. all he a
Many companies therefore take a pharmaceutical
at different multiproject view of the
he resoure
TesourCes necqe
he aei
stages of drug
another takes over, the discovery.
On the view that
where
drug discovery process can be seen one ne
hit generation and
confirmation, lead identiñcation, lead as a con
candidate selection, where the
project a
resources needed are optimiza
number of
compounds being screened, worked on, made, proportion.
onal to the
point in time. This type of "low'" or or optimi,
intuitive, "five-box" view of the likely"throughput" model can e
demands on resource. give aa Vevery
help to locate a potential bottleneck
iew clea
Such a v
that, as the input rate of
pipeline increases, may limit
downstream flow and cause workwork ina
back up. It appears that for upstr
many large pharma companies
resource has been synthetic this
less from chemistry, because this discipline has benesg
automation than has biological screening. In
of synthetic future, with ofsho
chemistry, limitations further downstream on
biology skills and resources and pharmacology
experience of interdisciplinary
start to limit R&D
productivity. working may
Vhen the most
likely bottleneck stage and
limiting resource
have been
identified, choosing may well then require been
the best
level DES that
management action
lowe
acts behind the scenes to calculate maximum
each relevant step within the bottlenecked research throughput a
process of analysis is much more efficient than a stage. Such a two-sten
bottom-up attempt to
asking critical questions about constraints. man
the R&D universe before
Clearly, however, the "flow" simplification would be invalid on a small
with episodic projects, or in contexts with significant downtime due to
site
over and
setup activities for project-specific assays. Such simulation change
covering multiple projects and multiple assays, with use of a mix ofproblems
and dedicated resources, can shared
rapidly become intractable, and an
monsense selection of the focus area for simulation is essential. early, com-
A "time-and-motion" modeling mind-set can easily blinker
best to improve the value created analysis of how
by research. Truly useful simulations of
drug discovery must go beyond simplistic assumptions about yields at stage
gates and address the causes of failure, starting to
concept of pipeline "quality."
quantify the common
Supporting excerpts from R&D management interviews were:
The danger of a metric-driven approach is to push through quantity insteadof
quality.
Many improvement attempts have knock-on consequences.. more compounds
delivered faster, not necessarily better, and this leads to bottlenecks.
11.9.3 Managing Risk and Uncertainty
Managing risk and uncertainty was increasingly seen as a priority in recent
conferences, reflected in these comments from R&D managemen
interviews:
 not
ptaking, scien
u i l u l
do
tists
highest
The highest priority is to improveou
ability to predict.
from the
Uncertainty from
viewpoint of the discovery rescarcher
trollable and uncontrollable risks [23]. consists both of
nistemic uncertainty-missing knowledge-is due to a lack of infor
mationthat through D you could
"buy" directly or
oroXy methods, it you so chose. These are controllable estimate throughin
risks,
actice they may be unduly expensive to control relative although
pra to the risk
exposure (threat x likelihood).
A leatory uncertainty-the roll of the die-describes risks that cannot
practicably be predicted within the research process, for example, new
failure modes, or modes that can only be detected in late stages of work,
for example, humans. An example of aleatory uncertainty is the with-
drawal in the UK of Bextra on the basis of two serious adverse events
out of 40,000 patients; this could only be discovered after launch |2,.
This, without hindsight, was an "uncontrollable risk."
aleatory
Spending effort on research to attempt to reduce what really
is
is
uncertainty is of time. Accepting some unmanageable risks simply
a waste
limits
of to the pharmaceutical industry. Once the
part of the price entry
concentrate on dis-
are accepted, research people can
of possible knowledge
covering genuinely knowable.
what is
toresearch processmodeling, which encour-
We take a Bayesian approach expressed as a
about the prior degree of uncertainty,
agesexplicit statements outcomes. Simulation
that builds in such
over possible
probability distribution differ-
helping managers to explore future
of a "what-if" nature,
uncertainty will be and to see where the
understand problem structure, where outcomes are
ent scenarios, to choices, or indeed
sensitive to current better informa-
is likely to be
most determines where
choices. This
indifferent to such much to invest in
internal
relatively decisions and how
improve prediction
tion could best help performance,
and in particular,
research (research about process
information. distribu-
such parametric
reliability) that yields in terms of
models could
be couched so discrete
becomes intractable,
Such Bayesian
real problems instead. The cal-
m a t h e m a t i c s for are used
tions, but the with the aid
of computers, (inference)
can be
from assumptions
estimnated
distributions, or
outcomes probabilities,
probability of of
conditional
Culation of multiplication stochastic
methods
exhaustive
obtained through
performed through can be
estimates futures.
problems over possible numbers to
with large that sample the necessary
"rubbish
Carlo techniques) is obtaining
"rubbish in,"
onte challenge to
overcome
distrust of reluctant to
Ihe main
There is a natural
"hard
data" may be business
models. with and
to such working trade-offs,
put teams used to about goals,
Out,and the assumptions
u n d e r l y i n g
r i their
eYnre
 MENT
processes in terms of
success rates numbers: For example, where dodo DECISION M
come from? How far are
they the
the.
.
of the
therapy area? adjustable?,AProjec
proyectindee
Are they
Therefore we increasingly take the ang
the business context, view that De
which may
rather than f
seem like a "black
to have decision
makers interact with box" an dllyysisimul
develop their intuitions and hone more selective simulatio
their
approach, is
focused areas,
decision theory, especially
which is
in the area of
judgment and reasoni
reason he
probability, applied
lied oabi
ns.These
statistics
The visualization of nonintuitive without such practice.
such
trade-offs involving risk and
powerful aid uncertainty
should in silico and insight. Questions frequently
to is
other encountered ayclear
ol
R&D process? What
predictive technologies best be
workflows involving such applied wi
What should be the
approach to selecting "cutoffs"? technologies add mo
An
application of
istic) combined with asimple statistical analysis
decision analytic valuation (receiver operating chas
of false
negatives may be positives
fundamental concept hereand
useful in such cases. The
of the method
but must be "buying information" that can avoid
reliable enough to avoid unnecessary future
cases there
may need even be no need to
wasting valuable opportunities. In m
tions of using the estimate the cost and
of less than proposed in silico method, as it can be proved effort that
im
critical
reliability would destroy value even if meth
a a
available for
predictive
free, owing to the rate of lost it wer
A further
insight is that the best
opportunities (false negatives)
factors that
workfow depends on a combination of
can in many cases be
allowing very rapid graphical feedback expressed in closed
mathematical form
to users of what
then becomes a
alization rather than a stochastic vis
effective for simple binary
simulation tool. This particular approachi
Vs. in silico as
comparisons of methods (e-g., use of in vitro alone
prefilter to in vitro). It can also be extended to evaluation
conditional sequencing for of
groups of compounds, using an extension of the
"sentinel" approach [24].
Such "sentinel" workflow uses a
prediction to select
expensive screen that can confirm predicted hazards compounds for a more
(liabilities, such
ity). It is, provably, the best workfiow in contexts where as toxic
the hazard is
a low prevalence of
anticipated, and where there is a backstop means further down-
stream (e.g., preclinical toxicity testing) for detecting hazards before humans
are exposed. This workflow then allows the compounds predicted as safe to
bypass the expensive hazards screen, without unacceptable risk, and can add
significant value in terms of external screening costs or avoiding use of what
may be a bottleneck resource.
Our simulation work has identified a
value-adding extension of this
approach where if there are two alternative liabilities A and B, a prediction
of the presence of A or B can select compounds for relatively early screening
against either risk factor, leaving the other to be assessed later. For certain
combinations of the ratios of costs of screening and prevalence for A and B.
 w s OF VSUALIZATON AND SIMULATON
P99 A7
269
e l i a b i l i t y on
of rreliability only just above chance can add some value. The
of
rede
rqdetion
ason is that all compounds will eventually be screened for both
ailites oso the
there is no downSde impact of the predictive method due to
vnderi
i a l e cn e g a l i v e s
iaThere
eare 1wo
are t w o key limitations of the use of ROC methods and decision
tic Vauations in such applications., each of which gives rise to an oppor
anal
improving the reach of R&D management simulations
for
tunity
used frequently for assessment of diagnostic methods of
pOC analysis.
11 kinds. requires comparison of a "prediction" with an assumed truth
which
"oold standard." Methods based on Bayesian or belief networks,
can model condithonal probability
can
relationships between many possible
measures of a given state of nature. have much greater
nrony (surrogate)
in comparing a
nwer to model multivariate problems, for example.
single-factor prediction of diffusion rate with a multifactor
in silico
in vitro measurement that includes the additional factor of
ADME
solubility.
1s valid for variable costs but does not
consider the
The decision theory is a
allocation. In many contexts, screening capacity
problem of capacity that broke the
there is a need to consider the "straw
sunk cost. and There is no need
first compound that exceeds capacity.
camel's back," the variable costs rela-
scarce and have trivial
resources that are not
to ration This reasoning leads
value that their use can create.
tive to the potential intuitive flow and capacity
use of easily understood,
naturally back to simulations.
visualizations for the relevant
intuitions
unmet need for helping management
There is therefore
an as-yet value added,
R&D capacity to optimize
to invest and
use
about the best way
conditions of uncertainty.
under
Trade-Offs About
Strategic
11.9.4 Making the Right
Reducing Uncertainty resourc-
investment (or in general,
trade-off between
There is a vital three-way inextricably linked:
and quality a r e
Throughput not mean
ing), throughput,
and quality.
are limited,
more output does
chemical options obvious and may
be counter
when biological and
balance is not always investment an e v e n harder
The right
Detter quality. makes the choice of the rightb o t t l e n e c k in the multistage
intuitive, which expected or to make
a
there is an
problem. For example, if to elevate
this constraint,
lost
to invest with potential
Process, is the right thing compounds
upstream,
selection of
Ore stringent (Architecture
for
"ARBITER"
opportunity? approach
termed uation in Research)
in
developed a n Technology Evaluation
e have and capability, that
combines
Reliable Business Improvement and v i s u a l i z a t i o n
simulation
, w i t h associated
 HOW SIMULATION
CAN SUPPORT MANAGEMENT
MENT DECISIO
270

Bayesian networks Ise an

for multivariate reasoning about cause and eff
R&D with a flow bottleneck model (Fig. 11.6) to llec wiat
help combine
economic aspects of decision making.
This model can, where ni a scientilic
arch eslin
proce
decisions affect potential candidate value, further incorporate sihProrate simple
tion of how the candidate value varies based on the target
Factors such as ease of dosing in this profile can then be causally product profi
the relevant predictors within the research process (e.g., bioavailabil nked
model the value of the predictive methods that might beused
sensitivity analysis of how R&D process choices affect the to
the expected
expected ad perlorm

snab Artate

end nkcos

Ed dendaton

kanye

B A
D
E

Figure 11.6 ARBITER provides a simulation of R&D throughput, stage succe

rates, and resource capacity loading (A) as a function of the methods used, e
sequence of use (including parallel use), which is based on prior estimates or co
pound library quality, prevalence of different compound characteristics (B), pret
tion reliability (C), and user-selectable cutoff levels (D higlighted circle). 1
combination of throughput and candidate expected value (based on variations aroue
the target product profile and the factors influencing development success rates) gi
a direct estimate of the rate at which a particular selection of R&D process can0
expected to contribute value. An average yield of successful projects (which may D
a fraction) can be converted through use of distribution over a measure of pipeline
quality (E), including the chance of having no successes in any given year. See colo

plate.
 and pattern of investment in
R&D process
structure step" gives, throughresources.
by "leaving Var
Varying
the difference
out a
ructu
valuc-add,aa inancial valuation of a research in
stimated dh throughput quality impacts.
and technology use that
hro
Onechallengein applying this approach, which relies on
prior estimates
diction reliability,
is how to deal with
e t

redds to be tested and the "universe" of all differences between

h

turecompound
o dp r e d i c

ence R&D process effectiveness has

of R&D
ience of compounds on which
the
evllected
experie
been based. If new
Jenfoll within "space"
unds fall within the
Oundserties is just another kindpreviously
wtnecomyound sampled, then
of conditioning within knowledge
m

ica
i c a lp r o p e r t i e s

a
Bayesian
all outside this space, then the initial model of both outcomes
fall
outside

: if ifthes
they
andpredictIonshas an unpredictable error. The use of sampling theory and
diversity |16] are therefore promising extensions of the above
modelso f

proach. network tec

technology embedded in the ARBITER tool is also
network
Bavesia

The
The5 for learning both probability relationships (eg., method reliability
el/suitedtorle.
el and the essential structure of cause and efect, from data sets
estimates) a n d

edictions and outcomes can be compared. Colleagues have already

on a large scale for risk management (selection of
applhed this capability
wDere

gplied
ntially suspect claims for further inspection and examination) in the
inSurance industry.

L.10 CONCLUSIONS

ensure that the future for process and per-

R&D managers
pharma long-term trend
How can
is not just like the past, where the
fnrmance improvement increased, economic effectiveness?
be decreased, not
appears to interview quotation here is:
most relevant
Perhaps the
knows.
is at knowing what it
Im not sure how good big pharma
self-aware and self-critical,
become more
pharma by these
So how, in practice can big two opposed views,
evidenced
There are
aDd what can IT do to help?
interview commentsS:
con-
not to have
natural desire
there is a
creative process; considerations.
DISCOvery is a very
business and operational of the bigger
picture
Siraints on this from inform
scientists
need to background.
improvement, we the right
0so Continuous be made
against
can
that individual udgments
"double loop
enables
evidence
that
retrospective

and using the data bases,

aly ively collecting s u c c e s s / f a i l u r e
tracking
can
R&D
management

rning" (including project reliability),

tools now

lailure ana1
prediction support
and tests of simulation

s, of
take full
ake range
full advantag of the wide