Environ Syst Decis (2014) 34:60–87
DOI 10.1007/s10669-014-9491-8
The role of scientific studies in building consensus
in environmental decision making: a coral reef example
Amanda P. Rehr • Mitchell J. Small •
Paul S. Fischbeck • Patricia Bradley •
William S. Fisher
Published online: 16 February 2014
Ó Springer Science+Business Media New York 2014
Abstract We present a new approach for characterizing
the potential of scientific studies to reduce conflict among
stakeholders in an analytic-deliberative environmental
decision-making process. The approach computes a nor-
malized metric, the Expected Consensus Index of New
Research (ECINR), for identifying where additional sci-
entific research will best support improved decisions and
resolve possible conflicts over preferred management
actions. The ECINR reflects the expected change in
agreement among parties over preferred management
actions with the implementation and consideration of new
scientific studies. We demonstrate the ECINR method
A. P. Rehr (&)  M. J. Small  P. S. Fischbeck
Department of Engineering and Public Policy, Carnegie Mellon
University, Baker Hall 129, Pittsburgh, PA 15213, USA
e-mail: pearlgrl@gmail.com
M. J. Small
Department of Civil and Environmental Engineering, Carnegie
Mellon University, Porter Hall 119, Pittsburgh, PA 15213, USA
P. S. Fischbeck
Department of Social and Decision Sciences, Carnegie Mellon
University, BP 208, Pittsburgh, PA 15213, USA
P. Bradley  W. S. Fisher
US Environmental Protection Agency, Office of Research and
Development, Pittsburgh, PA, USA
P. Bradley
National Health and Environmental Effects Research
Laboratory, Atlantic Ecology Division, c/o Florida Keys
National Marine Sanctuary, 33 East Quay Road, Key West,
FL 33040, USA
W. S. Fisher
National Health and Environmental Effects Research
Laboratory, Gulf Ecology Division, 1 Sabine Island Dr,
Gulf Breeze, FL 32561, USA
123
based on a preliminary application to coral reef protection
and restoration in the Guánica Bay Watershed, Puerto
Rico, focusing on assessing and managing anthropogenic
stressors, including sedimentation and pollution from land-
based sources such as sewage, agriculture, and develop-
ment. Structured elicitations of values and beliefs con-
ducted at a coral reef decision support workshop held at La
Parguera, Puerto Rico, are used to develop information for
illustrating the methodology. The ECINR analysis was
focused on a final study group of seven stakeholders,
consisting of resource managers and scientists, who were
not in agreement on the efficacy and respective benefits of
reducing loadings from three sources: sewage, agriculture,
and development. The scenario assumed that loadings
would be reduced incrementally from each source through
a series of management steps, which would be ranked in
order of maximizing anticipated benefits. An examination
of whether beliefs exhibited greater confidence and
coherence between stakeholders when informed by plau-
sible study results followed. The results suggest that new
scientific research would be generally likely to bring peo-
ple who initially disagreed to agree. Seventy-five percent of
the hypothetical research results were projected to result in
more agreement among the stakeholders. However, there
can be situations where prior beliefs may be too different
from the study results to shift perspectives enough to result
in more agreement. Furthermore, in a few cases, hypo-
thetical research results were projected to lead to more
conflict among stakeholders. Priority research, according to
the seven stakeholders, would be to quantify loadings from
agriculture and sewage, and not loadings from develop-
ment, since it is predicted to make little difference in the
outcome. Assuming the stakeholders are conflict-averse,
they would likely opt for research on sewage loadings as
the highest priority. Though preliminary, these results
Environ Syst Decis (2014) 34:60–87
suggest that ECINR can provide useful insights into the
social implications of a research program.
Keywords Environmental decision making  Value of
information analysis  Conflict resolution  Consensus
building  Coral reefs management  Puerto Rico
1 Introduction
Environmental decision making frequently involves issues
over which individuals and groups disagree regarding the
critical scientific underpinnings and the degrees of uncer-
tainty. As an environmental risk is acknowledged and
further studied, associated uncertainties can become
amplified as new data or events expose previously unrec-
ognized aspects of the problem. While people may agree
that management decisions require sufficient information
to justify a response strategy, those who favor a rapid
management response may argue that enough information
is already known, and those who favor a delay may argue
that further study is needed (Cullen and Small 2004a).
When environmental decision-making problems involve
uncertainty, Bayesian statistical inference can be used to
assess probabilities or degrees of belief (e.g., about rela-
tionships between management options and resource con-
ditions) and to update these probabilities when new
evidence or information is obtained (Ellison 1996). Some
with disciplinary roots in Bayesian statistics and those who
focus on the psychological dimensions of human decisions
disagree over whether stakeholders’ degree of belief, which
may be different initially, must rationally tend to move
closer as new evidence is obtained. While new scientific
evidence generally leads to a convergence in posterior
beliefs (among those with different priors), there are cases
where conflicting evidence can lead to a greater divergence
in beliefs (Stiber et al. 2004, 1999). Furthermore, behav-
ioral studies have found that when people hold widely
different worldviews, these are unlikely to change in spite
of repeated additional evidence (Jaynes 2003). In the case
of multi-stakeholder environmental decision making, this
suggests that new studies and information that reduce
uncertainty may not always resolve conflicts.
Currently, there is no clear way to estimate how addi-
tional research affects consensus building among multiple
stakeholders who may differ in their preferences for out-
comes, their beliefs about science and uncertainty, and
their trust in the objectivity and quality of proposed studies.
Such cases would benefit from a method to prioritize
research that moved the decision process forward. In this
study, we propose and demonstrate a new method to
address this deficiency. It combines and builds on existing
tools, such as multi-stakeholder deliberation, multiple
61
criteria analysis (MCA), Bayesian Belief Networks
(BBNs), and value of information (VOI) analysis.
Encouraging multi-stakeholder participation and delib-
eration to build agreement around a preferred management
option is critically important, especially among multiple
decision makers and stakeholders who have differing
objectives and beliefs regarding a problem (Cohen 1997;
DeKay et al. 2002; Reed 2008; Renn 2006). MCA is often
used in decision-making contexts that involve multiple
stakeholders, such as in participatory management of nat-
ural resources. MCA is a tool aimed at helping to evaluate
the relative importance of multiple, possibly conflicting
criteria in a decision scenario (Belton and Stewart 2001;
Makowski et al. 1996; Messner et al. 2006; Cohon 2004;
Kiker et al. 2005). It is important to note that there can be
shortcomings to agreement or consensus as a criterion for
decisions, including the possibility that management
responses may be delayed unnecessarily (Coglianese and
Allen 2004).
Any decision is made more difficult by significant
uncertainty in relevant scientific knowledge. Uncertainty
can include variability in current resource conditions or
incomplete scientific knowledge regarding the causal
relationships between management options and resulting
resource outcomes. Probabilistic techniques and expert
elicitation are existing tools for analyzing decisions where
such uncertainties are present (Cullen and Frey 1999;
Morgan et al. 1990; Cullen and Small 2004b; Hoffman
et al. 2007a, b). A BBN, or graphical network for modeling
probabilistic interrelationships between events, presents an
effective way to represent uncertainty in environmental
decision problems. A BBN can be used to estimate the
probabilities that various decision options will have par-
ticular outcomes and can include corresponding stake-
holder valuations of these outcomes. A BBN is especially
useful when individual nodes of the network can be
updated to see how new evidence might change the pre-
ferred management strategy (Borsuk et al. 2001; Stiber
et al. 1999; Henriksen et al. 2004; Henriksen and Barlebo
2008; Newton et al. 2006; Van Wees 2008).
Reducing uncertainty with new information needs to
take into consideration different preferences as well as
beliefs about uncertainty and trust in the science. VOI
analysis is used in decision analysis to assess the expected
impact of proposed tests, monitoring or research for
reducing uncertainties that matter to a pending decision
(Yokota and Thompson 2004). The VOI can be charac-
terized as an expected increase in value of the optimal
decision informed by the new knowledge, compared with
the choice made before the information was available. It is
important to note that information can only have value if it
has the potential to change the prior (without information)
decision. However, most VOI studies assume (1) a single
123
62
decision maker with known prior beliefs regarding the
probability of different environmental and economic out-
comes associated with each decision option; (2) a known
utility for different outcomes; and (3) a known likelihood
function for scientific studies that inform the probabilities
for reducing the uncertainty associated with some or all of
the decision options. These conditions generally do not
apply in a deliberative-analytical process where multiple
stakeholders often have different views of the scientific
evidence and different values and objectives for the out-
comes (Apostolakis and Pickett 1998; National Research
Council 1996). In these cases, an approach that considers
the potential contribution of scientific information to con-
flict resolution and consensus building is needed (Emerson
et al. 2009; Leach 2006; Lipsky and Ryan 2011).
To characterize the potential VOI for reducing uncer-
tainty and resolving conflicts in a multi-stakeholder
environmental decision, we propose a new metric, the
Expected Consensus Index of New Research (ECINR).
ECINR recognizes that decision support is needed as part
of an iterative analytical-deliberative process involving
scientific studies, assessments, and negotiations among
stakeholders. It assumes that two or more stakeholders
currently prefer different decision options. ECINR is
calculated based on the prior probability (before the new
research is conducted) that a scientific study will lead to a
result that allows stakeholders having conflicting prefer-
ences to reach consensus on the preferred decision option.
ECINR therefore incorporates aspects of MCA, BBNs,
and VOI.
In this study, we demonstrate the ECINR metric in an
assessment of alternative options for coastal ecosystem
management. The need for advanced scientific knowledge
and monitoring to inform these management decisions is
broadly recognized (e.g., Boesch 2006; Borja 2005; Gu-
trich et al. 2005; Matso and Becker 2013). Here, we con-
sider the specific case of research in support of
management of coral reef stressors in Guánica Bay, Puerto
Rico. This study problem was selected because it deals
with an environmental decision-making situation that
involves uncertainty in the relevant science and multiple
stakeholders who likely have different views of the scien-
tific evidence, different values, and objectives for the out-
comes, as well as, an interest in working together to reach
agreement on a plan of action. We use the ECINR to
examine whether beliefs can be expected to exhibit greater
confidence and coherence among stakeholders when
informed by plausible (albeit hypothetical) study results.
We suggest how these results can be used to identify pri-
orities for new research. The information for this study was
obtained from elicitations completed by volunteers partic-
ipating in a coral reef decision-science workshop held at La
Parguera, Puerto Rico, in April 2010.
123
Environ Syst Decis (2014) 34:60–87
Fig. 1 Map of the study site (Ramos-Scharron 2009)
We hypothesized that participants would move to
agreement over preferred management options based on
science that reduces key uncertainties. However, in cases
where prior beliefs regarding the outcomes of decisions are
too disparate, agreement may not be achieved with new
evidence and could even lead to greater divergence.
Therefore, the value of the information would be greater
when the uncertainty being reduced matters in a similar
manner across stakeholders. We also hypothesized that the
value of information would be greater when research
results were more certain.
2 Method
An influence diagram, or BBN, was created based on a
conceptual model of an environmental decision problem
and information elicited from stakeholders, and then used
to compute preferred management actions. The ECINR
method was then applied to identify where additional sci-
entific research would support better informed decisions
and resolve possible conflicts over preferred management
actions. At issue was the control of major coral reef
stressors in Puerto Rico, which include land-based sources
of pollution, overfishing, and global climate change.
A Coral Reefs Decision Support Workshop held in Puerto
Rico in April 2010 [sponsored by the US Environmental
Protection Agency (EPA) and Caribbean Coral Reef
Institute (CCRI)] provided an opportunity to elicit prefer-
ences and beliefs of a group of stakeholders regarding the
efficacy of proposed regulations for coral reef protection.
Below we detail the elements of our approach.
2.1 Descriptions of study problem, data sources,
equations, and calculations
2.1.1 Study problem: control of coral reef stressors
in the Guánica Bay Watershed, Puerto Rico
The Guánica Bay Watershed located in the southwest
region of Puerto Rico (Fig. 1) was the focus of the Coral
Environ Syst Decis (2014) 34:60–87
Reef Decision Workshop. Coral reefs in this area have the
highest abundance of living corals in Puerto Rico and once
were considered an example of ‘‘healthy reefs,’’ since they
had more than 20 % total coral cover (Morelock et al.
2001; Warne et al. 2005). Over the last decade, coral reef
condition near Guánica Bay and La Parguera (to the west)
has declined, presumably affected by a major sea temper-
ature elevation in 2005–2006 (Miller et al. 2006; Whelan
et al. 2007; Woody et al. 2008) and land-based sources of
pollution (Center for Watershed Protection 2008; Warne
et al. 2005).
The coral reef tract in Puerto Rico extends up to 15 km
offshore and covers an area of approximately 2,000 km2
(UNEP 2006). Because of their extent, Puerto Rico’s
coral reefs are estimated to be worth upward of over one
billion dollars per year in ecosystem services (i.e.,
shoreline protection, water quality maintenance, climate
regulation, fish, pharmaceuticals, tourism, recreation,
nutrient cycling, habitat, and nursery areas (Hassan et al.
2005; Principe et al. 2012)—according to a United
Nations Environmental Programme (UNEP) estimate that
coral reefs provide a total of $100,000–$600,000/km2/year
(UNEP 2006). This is likely true in Puerto Rico, which
supports a tourism industry based partially on marine
resource-based activity worth 3.5 billion dollars (Burke
and Maidens 2004).
The abundance and cover of coral reefs across the
Caribbean Sea have declined over the past 30 years due to
local stressors associated with rapid population growth,
which has led to increased development, deforestation for
agriculture, and increased discharge of sewage. Some of
the consequences include high sediment influx, increased
nutrient levels, overfishing, and habitat modification,
which, when combined, threaten 90 % of reefs (Burke and
Maidens 2004). On top of local stressors, there are global
stressors, including ocean warming and acidification. Coral
disease and bleaching (loss of photosynthetic algae) have
been observed. The loss of coral in Puerto Rico has led to a
decline in ecosystem services, including economic benefits
from fisheries, which decreased by 70 % from 1970 to
1990 (Burke and Maidens 2004). Despite the fact that
Puerto Rico’s coral reefs are under government jurisdic-
tion, effective management is limited by poor enforcement
of laws regulating fishing activities and recreation (Garcı́a-
Sais et al. 2005).
A wide range of decision makers and stakeholders now
recognize the priority and urgency for actions to protect
and restore Puerto Rico’s coral reefs. These options may
require economic sacrifices by the Puerto Rico community
and likely tradeoffs with industries, such as agriculture and
municipal and industrial growth. There are conflicting
views among stakeholders on the severity of different
threats, the potential to manage those threats, which actions
63
should be taken, and their likely environmental and
socioeconomic outcomes.
2.1.2 Data sources: discussion-based elicitation exercise
and face-to-face elicitation
Preliminary data were collected from workshop attendees
through an elicitation exercise carried out in a discussion
format and used to structure and populate a template BBN
(See ‘‘Appendix 1’’ for the elicitation questions). Addi-
tional data were then collected through subsequent open
interviews designed to populate prior and conditional
probabilities in BBNs for each of the participants (See
‘‘Appendix 1’’ for the face-to-face elicitation questions;
and Fig. 2 for an example of a BBN for Participant A
showing prior beliefs before implementation of new
research and management options).
We developed the conceptual model used to structure
the discussion-based elicitation exercise using the DPSIR/
DL framework, which covers the relevant components of a
multi-attribute environmental decision-making problem,
and which we demonstrated previously for the case of
managing coral reef stressors in the Florida Keys (Rehr
et al. 2012). The DPSIR/DL framework integrates the
Driving Forces, Pressures, States, Impacts, and Responses
model (which aims to identify the important causal rela-
tionships among anthropogenic environmental stressors,
processes, and outcomes); and the Decision Landscape
model (which aims to ensure that relevant legal, institu-
tional, and social factors affecting a decision, as well as the
knowledge, values, and decision making of participants in
the various elements of the DPSIR process, are recognized
and considered).
Three kinds of data were gathered from participants in the
discussion-based elicitation: (1) preferences, captured by
weighting different resource outcomes or objectives; (2)
current beliefs about scientific relationships (including asso-
ciated uncertainty) among multiple elements in this environ-
mental system, such as between management options,
stressors, and outcomes; and (3) beliefs regarding the efficacy
of possible new research and data collection programs for
reducing key uncertainties and improving future management
decisions. Some beliefs about implementation cost and the
value of affected ecosystem services were also captured.
In addition, three kinds of information were collected
from each of seven final participants in the face-to-face
elicitations: (1) a relative rating of preferences for out-
comes (coral reef health, tourism, fisheries); (2) beliefs
about science including both a breakdown of loadings from
sewage, agriculture, and development and the probability
that restoration of a lagoon upstream from Guánica Bay
would be effective at filtering nutrients and sediments; and
(3) probabilities that different combinations of stressors
123
64 Environ Syst Decis (2014) 34:60–87

Sew Load Inland Water Quality Restore Lagoon? Marine Protect (MPA)
Low 23.0 Total Load Poor 91.4 Yes 0 Yes 0
Medium 24.0 VeryLow 0 Good 8.60 No 100 No 100
High 25.0 Low 0 0.086 ± 0.28 0 0
VeryHigh 28.0 ModLow 2.64
119 ± 89 ModHigh 18.5
High 57.9 Coral Reef Health
VeryHigh 21.0 Ocean Warm/Acid.
Sew Reduction Poor 78.6 Low 50.0
385 ± 160 Lagoon WQ
None 100 Good 21.4 High 50.0
None 100
Forty 0 0.214 ± 0.41
Poor 0
Seventy 0 Good 0
Ninety 0
0
Dev Reduction Tourism Econ. Gr. Comm. & Rec. Fishing
Ag Load None 100 Bay & Ocean Water Quality Low 79.4 Poor 73.6
Low 6.00 Forty 0 Good 10.3 High 20.6 Good 26.4
Medium 11.0 Seventy 0 Poor 89.7 0.206 ± 0.4 0.264 ± 0.44
High 28.0 Ninety 0
VeryHigh 55.0 0
180 ± 83 Coral Links Benefits
Dev Load None 0
Ag Reduction other- 100
Low 40.0 Wq 3.00 Fish Links
None 100 Medium 36.0 Ow 1.0 183 ± 240
None 0
Forty 0 High 23.0 Mp 1.0 Cr 1.0
Seventy 0 VeryHigh 1.0 WqOw 30.0 Ow 1.0
Ninety 0 59.2 ± 43 WqMp 20.0 Coral Eco. Services
Mp 3.00
0 OwMp 5.00 Low 25.0 CrOw 20.0
WqOwMp 40.0 Medium 50.0 CrMp 30.0
High 25.0 OwMp 5.00
Sew Load Research Ag Load Research Dev Load Research 1.13 ± 0.54 CrOwMp 40.0
Low 23.2 Low 7.52 Low 38.8 Coral Effects Research
Medium 24.1 Medium 12.1 Medium 35.1 None 3.00
High 25.0 High 27.8 High 23.2 Wq 5.28
VeryHigh 27.8 VeryHigh 52.6 VeryHigh 2.92 Fisheries Research
Ow 3.76 Coral Eco. Services Research
Mp 3.76 None 3.00
Low 26.2
WqOw 25.8 Cr 3.76
Medium 47.5
WqMp 18.2 Ow 3.76
High 26.2
OwMp 6.80 Mp 5.28
WqOwMp 33.4 CrOw 18.2
CrMp 25.8
OwMp 6.80
CrOwMp 33.4

Fig. 2 Bayesian Belief Network showing prior beliefs for Participant A (gray nodes represent management options)

(marine protection areas (MPAs), water quality and climate
change) would produce outcomes of interest. The final
study group consisted of one natural resource manager,
four scientists, and two resource conservation specialists
who worked for local non-profit organizations.
Note that the exercises and subsequent elicitations are
not intended to provide a representative sample of scientific
or popular opinion on the issue of interest. Rather, they are
intended to reflect the range of values and beliefs held by
particular participants in a decision problem (those asked to
participate in the workshop as a result of their prior and
ongoing managerial, scientific and community experience
and expertise).
2.1.3 Design of BBNs: equations and calculations
The template BBN was designed to represent the current
situation of coral reef stressors and management in the
Guánica Bay Watershed from the viewpoint of stakehold-
ers (See ‘‘Appendix 2’’ for a full explanation of each node
of the BBN). The network depicts that loadings from
agriculture, sewage, and development are polluting inland
water quality and Bay water quality, which in turn, affects
coral health and the benefits that people care about—fish-
eries and tourism. Natural resource managers can imple-
ment management options, which include reducing
loadings from different sources; restoring the Lagoon, and
implementing marine protected areas (MPA). If lagoon
restoration is implemented, an additional node captures
lagoon water quality, which is dependent on the effec-
tiveness of lagoon filtration. If the MPA is implemented,
coral health and fisheries are enhanced. Ocean warming
and acidification are included as a single external variable
that affects coral health and can be set at low, high, or left
to chance. Research can then be carried out by activating
research nodes and their results (for loadings, coral effects,
fisheries, and ecosystem services), and then, their effects on
preferences for management options can be determined.
The total pollution load, Total load, is represented by a
function of the individual sources and their associated
hypothetical reductions (management options) as shown in
Eq. 1:
 
SewRed
TotalLoad ¼ SewLoad  1  þ AgLoad
  100
AgRed
 1 þ DevLoad
 100 
DevRed
 1 ð1Þ
100
Distributions of stressor loadings were computed over the
low, medium, high, and very high categories in a manner
that minimized variance based on stakeholders’ prior
beliefs. Loading values included in the model are relative
(and therefore unitless) though roughly scale to mg/L

123
Environ Syst Decis (2014) 34:60–87
concentrations of suspended solids in unpolluted source
waters (very low = 0–25; low = 25–50), moderately pol-
luted source waters (moderately low = 50–125; moder-
ately high = 125–250), and highly polluted source waters
(high = 250–500; very high = 500–750). The range of
values used for TotalLoad of 0–750 was selected to allow
for a more realistic distribution (with six intervals from low
to very high) than would a smaller range. Since the analysis
is of a comparison of benefits, the actual units used are
irrelevant.
The BBN was designed to be used for computing the
preferred management option based on maximizing bene-
fits and then for performing a comparative benefits
assessment (between without- and with-information con-
ditions). Benefits are computed as shown in Eq. 2:
Benefits ¼ A  Tourism þ B  Fisheries þ C
 Coral Health  Ecosystem Services ð2Þ
where A, B, and C are weightings of the importance of the
outcomes, relative to Ecosystem Services, given by the
volunteers. Netica, a commercially available BBN software
package, is used to build and run the BBN models (Norsys
2010). To examine the effect of plausible new study results
as described earlier, Netica uses probabilistic inference (the
probability of some event given the occurrence of some
other event) to adjust beliefs based on the new evidence,
i.e., Bayes’ theorem. Bayes’ theorem adjusts probabilities
given new evidence by calculating a posterior probability
(H given E) as shown in Eq. 3:
PðHjEÞPðHÞ
PðHjEÞ ¼ ð3Þ
PðHÞ
where H represents a specific hypothesis; P(H) is the prior
probability of H that was inferred before new evidence, E,
became available; P(E|H) is the conditional probability of
seeing the evidence E if the hypothesis H happens to be
true (it is also a likelihood function when it is considered as
a function of H for fixed E); and P(E) is the marginal
probability of E: the a priori probability of witnessing the
new evidence E under all possible hypotheses calculated as
the sum of the product of all probabilities of any complete
set of mutually exclusive hypotheses and corresponding
conditional probabilities as shown in Eq. 4:
X The agreement index is cumulative considering all
PðEÞ ¼ PðEjHi ÞpðHi Þ ð4Þ
2.2 Implementation with the working group
2.2.1 ECINR analysis
A preliminary analysis (before new research was applied)
showed that all of the stakeholders preferred the manage-
ment options of establishing MPAs and restoring the
65
Table 1 Set of nine alternative Reduction Ag Sew Dev
management options for reduc- (%)
ing loadings from sources
40 1 2 3
70 4 5 6
90 7 8 9
lagoon (their first and second choices, respectively). Since
they were already in agreement for their top management
choices, the ECINR analysis was based on the next set of
decision options, reducing nutrient loadings from sewage,
agriculture, and development, for which they were not in
agreement. The scenario assumed that loadings would be
reduced incrementally from each source by 40, 70, and
90 %, which would amount to nine management options in
total (Table 1).
Based on their prior (without new information) beliefs
about the contribution of loadings from different sources,
their stepwise ranking of the nine management options was
determined. Next, three hypothetical new research pro-
grams (one each of agriculture, sewage, and development
loadings) were added to the scenario to clarify the contri-
bution to total loadings from each source. The degree to
which each of the proposed study findings is predicted to
promote agreement (consensus) around options at each step
in their stepwise ranking, CIstep, was computed as shown in
Eq. 5:
CIstep ¼
# of management option selections with concurrence ð5Þ
# of option selections undertaken
where it can be defined further in Eq. 6:
CIstep ¼
ð# management option to which all agreeÞ  ð# shakeholdersÞ
ð# of sequential managements stepsÞ  ð# shakeholdersÞ
ð6Þ
an example of how the agreement index is computed at
each step for two stakeholders is shown in Table 2. When
options are tied, it is assumed that the option selected first
is the one that leads to more agreement at that step and then
the other option is automatically selected in the next step.
management options up to and including the given step.
A limitation of the analysis due to the assumption made for
handling ties is that the option automatically selected
second may not always be the one that would lead to the
most agreement at the given decision step.CINRstep is the
value of each of the study results (after the research is
conducted) at each step. This was computed as the change
in agreement between the with- and without-information
conditions. Next, the individual stakeholder perspectives
123
66 Environ Syst Decis (2014) 34:60–87

Table 2 Example of computing agreement for two stakeholders in a strategies have already been implemented. Next an exam-
hypothetical decision ple of the method used for selecting preferred options in the
Steps Stakeholder 1 Stakeholder 2 Consensus Index stepwise ranking of management options to reduce load-
ings is shown for steps 1 and 2 (an AgRed of 40 % and an
1 1 2 0/2 = 0
AgRed of 70 %, respectively) of each of the possible
2 4 5 0/4 = 0 selections are compared and the option with the highest
3 7 8 0/6 = 0 benefits is selected (and bolded). In Table 4, the complete
4 2 1,3 (1) 4/8 = 0.5 stepwise ranking of all nine management options is shown
5 5 1,3 (3) 6/10 = 0.6 for Participant A based on prior beliefs. This ranking shows
6 3 4,6 (4) 9/12 = 0.75 that Participant A initially favors reductions in agriculture
7 8 4,6 (6) 13/14 = 0.93 followed by sewage and then development. In Table 5, the
8 6 7,9 (7) 16/16 = 1.0 stepwise ranking is repeated after knowledge of a research
9 9 7,9 (9) 18/18 = 1.0 finding that loadings from agriculture are in fact low
The individual rankings of the nine management actions are indicated (AgLow). This ranking shows how changing beliefs from
for each of the two stakeholders, and the consensus index is calculated agricultural loadings being high to agricultural loadings
being low also changes preferences for reductions from
that a particular study will bring stakeholders to agreement agriculture versus other options (now Participant A favors
(before the research is conducted) (Expected CI or ECI) at reductions in sewage followed by development and then
each management step was computed as shown in Eq. 7: agriculture). Table 6 is a comparison of the stepwise
rankings of management options under prior beliefs versus
X
n  
ECIstep ¼ CIstep under new research finding of AgLow. Reductions in
outcome
outcome¼1 ð7Þ sewage and development have risen in the stepwise ranking
 Preposterior Probabilityoutcome (the old and new positions are connected with orange
arrows), while reductions in agriculture have fallen (the old
where an outcome is one of the various ways that the and new positions are connected with blue arrows).
research can turn out, and the Preposterior Probability is
the probability before the research is conducted that a result
will come out a given way, with associated implications for 3 Results
decision preferences. Finally, the value of each of the
research programs (before the research is conducted) at 3.1 Demonstrations of what BBNs produced
each step, ECINRstep, was computed as the expected
change in the agreement between the with- and without- 3.1.1 Prior beliefs about the contribution of loadings
information conditions. from different sources
The study assumed that stakeholders shared the same
beliefs about the accuracy of the research (the likelihood The prior beliefs of the seven volunteers about the contri-
functions or false ?/false- rates) and that the new research bution from agriculture, sewage, and development to total
is nearly perfect, with the probability that the correct loadings are shown in Fig. 3. Three volunteers believed that
inference is made equal to 94 %. sewage had the largest contribution (at 50 %). Two volun-
The following examples illustrate in tabular form how teers believed that agriculture had the largest contribution (at
we identified preferred management options under the 60 and 50 %). All volunteers believed that development did
cases of (1) prior beliefs and (2) research outcomes, using not have the largest contribution. Two believed that all of the
the BBN for Participant A. sources contributed equally. As a group, they believed that
sewage contributed the most to total loadings (average of
40 %), followed by agriculture (average of 37 %), and
2.2.2 Identifying preferred management options finally development (average of 24 %).
under prior beliefs and research outcomes
3.1.2 Change in agreement among stakeholders
In Table 3, the baseline case (for the BBN in Fig. 2) is over preferred options due to study results
shown with initial probabilities of good coral reef health,
high tourism, good fisheries, and their respective associated Depending on the results of the proposed new research
benefits. Next the effects and associated benefits of programs, we predict variable trends in the change in
implementing the preferred management options of MPAs agreement (CINR) among stakeholders regarding preferred
and the Lagoon are shown. Our analysis assumes that these management options across the management steps.

123
Environ Syst Decis (2014) 34:60–87 67

Table 3 Example of the baseline without management options, the effects of implementing MPAs and Lagoon (the baseline for the analysis),
and the strategy for selecting steps 1 and 2 in the stepwise ranking of alternatives for reducing loadings for Participant A
MPAs Lagoon SewRed AgRed DevRed P(GoodCorals) P(HighTourism) P(GoodFisheries) Benefits DBenefits
(%) (%) (%)

Baseline ? No No None None None 21 26 20 183 –

W/mgmt Yes No None None None 40 41 60 320 137
Options ? No Yes None None None 46 31 35 292 109
Baseline Yes Yes None None None 74 57 71 474 291
analysis
!
Yes Yes 40 % None None 74 58 71 476 2
Step 1 ? Yes Yes None 40 % None 74 58 71 477 3
Yes Yes None None 40 % 74 57 71 475 1
Yes Yes 40 % 40 % None 75 59 71 480 6
Step 2 ? Yes Yes None 70 % None 75 59 71 481 7
Yes Yes None 40 % 40 % 75 59 71 479 5

Table 4 Example of a stepwise ranking of management alternatives to reduce loadings based on prior beliefs for Participant A
Mgmt MPAs Lagoon SewRed AgRed DevRed P(GoodCorals) P(HighTourism) P(GoodFisheries) Benefits DBenefits
steps (%) (%) (%)
Baseline Yes Yes None None None 74 57 71 474 –
analysis
!

1 Yes Yes None 40 % None 74 58 71 477 3

2 Yes Yes None 70 % None 75 60 71 481 7
3 Yes Yes None 90 % None 76 61 72 487 13
4 Yes Yes 40 % 40 % None 76 63 72 493 19
5 Yes Yes 70 % 70 % None 77 65 72 500 26
6 Yes Yes 70 % 90 % 40 % 78 68 73 508 34
7 Yes Yes 90 % 90 % 40 % 79 71 73 516 42
8 Yes Yes 90 % 90 % 70 % 80 73 73 522 48
9 Yes Yes 90 % 90 % 90 % 81 74 73 527 53

Table 5 Example of a stepwise ranking of management alternatives to reduce loadings based on a research finding of AgLow for Participant A
Mgmt steps MPAs Lagoon SewRed AgRed DevRed P(GoodCorals) P(HighTourism) P(GoodFisheries) Benefits DBenefits
(%) (%) (%)
Baseline Yes Yes None None None 74 57 71 474 –
analysis
!

AgLow ? Yes Yes None None None 75 60 71 483 9

1 Yes Yes 40 % None None 76 61 72 487 15
2 Yes Yes 70 % None None 76 63 72 492 18
3 Yes Yes 70 % None 40 % 77 65 72 498 24
4 Yes Yes 70 % None 70 % 78 67 72 505 31
5 Yes Yes 90 % None 70 % 78 68 73 509 35
6 Yes Yes 90 % 40 % 70 % 79 71 73 516 42
7 Yes Yes 90 % 70 % 70 % 80 73 73 523 49
8 Yes Yes 90 % 70 % 90 % 81 75 73 528 54
9 Yes Yes 90 % 90 % 90 % 81 76 74 532 58

123
68 Environ Syst Decis (2014) 34:60–87

Table 6 A comparison of the stepwise rankings of management DevMed) was monotonically increasing across the man-
options under prior beliefs versus under new research finding of agement steps (Fig. 5b). These research results were most
AgLow
consistent with the stakeholders’ prior beliefs. The other
Steps Under Prior Beliefs Under New Research =AgLow
five results that led to more agreement among stakeholders,
on average, were found to have incremental changes in
1 Ag 40% Sew 40% agreement that were non-monotonic across the manage-
2 Ag 70% Sew 70% ment steps (Fig. 5c). Three of those results (AgHigh,
SewLow, and DevHigh) produced more agreement at more
3 Ag 90% Dev 40%
of the management steps than they produced less agree-
4 Sew 40% Dev 70% ment, one result (SewMed) produced neither more nor less
5 Sew 70% Sew 90%
agreement, and one result (DevLow) resulted in less
agreement more often than not. These research results
6 Dev 40% Ag 40% tended to be somewhat consistent with the stakeholders’
7 Sew 90% Ag 70% prior beliefs. However, the three study results that appeared
to be the most different from any of the stakeholders’ prior
8 Dev 70% Dev 90%
beliefs (AgLow, Dev V High, and AgMed) had negative
9 Dev 90% Ag 90% value in terms of bringing stakeholders to agreement across
the management steps (Fig. 4). In fact, the incremental
change in agreement in response to these results was
monotonically decreasing (Fig. 5d). In these cases, the
posterior probabilities on the levels of loadings in response
to research results were not large enough to fully shift
perspectives and bring stakeholders to agreement.

3.2 Degree of convergence among stakeholders

3.2.1 Stakeholder expectations that research studies will

lead to agreement (ECINR)

Depending on the different research programs, we predict

Fig. 3 Prior (w/o research) beliefs about the contribution of different
sources to total pollution loadings
Predicted agreement at each management step in response
to each new study result compared with the ‘‘No Research’’
alternative is shown in Fig. 4. The incremental change in
agreement at each management step with each new piece
of information is shown in Fig. 5. Most of the research
results (9 out of 12, or 75 %, which include all results
except for AgLow, AgMed, and Dev V High) were pre-
dicted, on average, to lead to an increase in agreement
among stakeholders across the management steps (Fig. 4).
Therefore, the value of the research results in terms of
bringing stakeholders to agreement was generally positive.
In fact, the change in agreement in response to four of the
research results (Ag V High, SewHigh, Sew V High, and
variable trends among the individual stakeholders in their
expectations about whether the programs will lead to
agreement about preferred management options. Beliefs of
individual stakeholders before the research are conducted
about the degree to which each of the individual research
studies and a combined study will lead to agreement
compared with under the ‘‘No Research’’ alternative
(change in agreement adjusted by preposterior probabili-
ties = ECINR) are shown in Fig. 6. Stakeholders believed
on average before the research was conducted that research
programs to determine sewage loadings and agriculture
loadings would tend to promote the most agreement of the
three research programs on average (ECINR for each of the
two programs is 0.2 averaged across the management
steps) (Fig. 6a, b). Research to determine loadings from
development was expected to produce neither more
agreement nor conflict on average (ECINR is 0.0 averaged
across the management steps) (Fig. 6c). The predicted
ECINR of a combined research study to determine loadings
from all three sources was not conducted here, but is
expected to be higher than that of individual research
programs and probably less than the sum of that of the
individual research programs.

123
Environ Syst Decis (2014) 34:60–87
Fig. 4 Agreement among stakeholders (CIstep) (y-axis) for study results versus no research alternative across the management steps (x-axis) (All
results lead to more agreement, on average, except for Ag Low, Ag Med, and Dev V High)
In terms of prioritizing their research agenda to reduce
uncertainty and resolve conflict, these results suggest that
stakeholders would first conduct research to determine
loadings from sewage and agriculture, and they would
forego research to determine loadings from development
since it is not predicted to change their decisions to a
degree that would allow them to come to agreement. Now
if stakeholders wished to choose only one research program
to implement, and assuming they are conflict-averse, the
trends observed in the individual research outcomes shown
in Fig. 5 would lead them to opt for sewage loadings
research. This is because the incremental changes in
agreement across the management steps for two of the
outcomes of the agricultural loadings research program
(AgLow and AgMed) were zero or negative (with more
conflict produced at each step) (Fig. 5d); whereas the
incremental changes in agreement across the management
steps for all of the outcomes of the sewage loadings
research program were likely to be positive (with more
agreement produced at each step) (b and c).
69
4 Discussion
A new decision support method for identifying where
additional scientific research may be helpful in supporting
better informed decisions, and in resolving stakeholder
conflicts over preferred management actions was presented
and demonstrated. As hypothesized, the participants in our
study most often moved to greater agreement over pre-
ferred management options based on proposed science to
reduce key uncertainties. However, we found that there can
be cases where prior beliefs may be too different from the
study results to shift perspectives and enhance agreement
among participants. Furthermore, there are cases where
new studies are predicted to be more likely to reduce
consensus rather than enhance it.
These results are consistent with arguments in the lit-
erature that well-targeted scientific input is important for
collaborative decision making for coastal ecosystem man-
agement and other environmental issues (Bartlett 2011;
Dreelin and Rose 2008; Gutrich et al. 2005; National
123
70 Environ Syst Decis (2014) 34:60–87

Fig. 5 Incremental change in agreement (CIstep) (y-axis) across the management steps (x-axis) with each new piece of information

Fig. 6 Stakeholder expectations that research studies will lead to agreement over preferred management options (ECINRstep) (y-axis) compared
with no research alternative across the management steps (x-axis)

123
Environ Syst Decis (2014) 34:60–87
Research Council 1996; Ribarova et al. 2011; Sanger et al.
2010), but may not always yield improvements in decision
processes involving multiple stakeholders with conflicting
values and beliefs (Doremus and Tarlock 2005; Nelkin
1975; Peuhkuri 2002; Sarewitz 2004; Gregory et al. 2006).
However, conflict resolution is not the only objective for
evaluating an effective decision-making process (Dietz
2003; Rauschmayer and Wittmer 2006; Beierle 2002). In
particular, coming to agreement on a scientifically misin-
formed and harmful management option should not be
viewed as successful outcome, and scientific studies that
challenge such a consensus can be viewed as providing a
positive contribution to the decision process. In addition,
many other factors besides scientific input can influence the
achievement of conflict resolution among diverse stake-
holders (Bartlett 2011; Connelly and Richardson 2004;
Emerson et al. 2009). The formal framework developed
here for considering the effects of stakeholder values,
beliefs, and perceptions regarding proposed scientific
studies on the likelihood that they will foster consensus
toward a jointly preferred management option provides a
further basis for exploring the potential contributions of
science toward such deliberative policy making.
Considerable effort is currently underway to set priori-
ties for research and data collection by regional, national,
and multinational agencies to better inform management
decisions for coastal and related ecosystems (Boesch 2006;
Borja 2005; Brodie et al. 2012). It is now widely recog-
nized that the provision of scientific information is a social,
as well as a technical process, requiring careful consider-
ation of decision participants, decision context, and the
factors that build credibility and trust in the quality and
utility of the information for building consensus in favor of
a well-designed management plan (Dreelin and Rose 2008;
Gutrich et al. 2005; Matso and Becker 2013; Sanger et al.
2010). Tools to evaluate the potential value of scientific
information for addressing conflict among stakeholders
regarding environmental values, beliefs, outcomes, and
preferred management options can help to further enlighten
this process.
Though preliminary, these results suggest that our new
decision support method can provide useful insights on the
social implications of a research program. The contribution
here is not a test of the hypothesis that the method is
beneficial, but rather the novel invention and pilot study of
the method. Based on our observation, the study partici-
pants were receptive to and generally understood the new
decision method and were satisfied with the manner in
which they were engaged as experts during the problem
scoping and elicitation process. In subsequent research, a
larger and more diverse sample group would help to clarify
71
the results of this study and confirm whether the method is
beneficial.
Different from what we had expected, differences in
preferences for outcomes (values) did not influence the
degree of conflict or agreement in this pilot study. This is
likely because our seven participants placed similar
weightings on the outcomes and also because the outcomes
were generally correlated (coral health * fisheries
health * tourism health). More value-driven conflict over
preferred management actions may have been observed
given the following situations: (1) Had a greater diversity
of stakeholder groups, e.g., farmers and developers, who
were invited but unable to attend (and who would have
likely held value-based conflicts with management options
to reduce loadings from agriculture and development)
participated in the workshop and (2) Had additional out-
comes been included in the model that presented more
complete tradeoffs between the environment and economy.
Additionally, including cost information for management
options and research studies, which was difficult to gather
as it was largely unknown to participants, would have
allowed for more realistic tradeoffs.
Further limitations of the study were reflected in its
inability to show that ECINR would have increased based
on research results that were more certain and when
stakeholder beliefs about the meaning of the study results
(likelihood functions or false ?/false- rates) were not too
different. Varying these factors would likely lead to more
realistic conflicts among stakeholders over research find-
ings. Thus, to address the limitations of the pilot study and
confirm the usefulness of the method going forward, future
work could involve a larger and more diverse sample group
and the inclusion of information reflecting stakeholder trust
in the science, additional outcomes (tradeoffs), and cost
information.
Acknowledgments This is a contribution to the US Environmental
Protection Agency Office of Research and Development’s Ecosystem
Services Research Program. The US Environmental Protection
Agency through its Office of Research and Development collaborated
in the research described here. Any opinions, findings, and conclu-
sions or recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the Agency. We
thank the workshop participants for sharing their expertise and for
their generous input of information, time, and effort. Support for
Mitchell Small was also provided by the Center for Climate and
Energy Decision Making (CEDM), through a cooperative agreement
between the National Science Foundation and Carnegie Mellon
University (SES-0949710).
Appendix 1: Blank elicitation form
123
72 Environ Syst Decis (2014) 34:60–87

Participant Assessment of Objectives, Values, and Beliefs

Coral Reef Decision Support Workshop: Guánica Bay Watershed, Puerto Rico

Guánica Bay Watershed

Importance and Value to you

Resources and Outcomes

(check one box for each outcome) Unimportant Low Medium High Very High

Guánica Bay Water Quality

Coral Reef Health

Fisheries

Drinking Water Quality

Agriculture

Tourism

New Construction and Development

Other

Other

Other

B. (Optional) Please identify your specific objectives for the economic and environmental future
of the Guánica Bay Watershed. You may include items from part A above and, if you wish,
specific goals and measures, such as target water quality standards, percent coral reef recovery,
etc.

123
Environ Syst Decis (2014) 34:60–87
Environmental Threat
(Driver/Pressure):
Example: Smoking
Example: Global Warming
1. Sewage and wastewater
treatment plant loadings
2. Sewage and wastewater
treatment plant loadings
3. Agrochemical discharges
4. Agrochemical discharges
5. Sediment loadings
6. Sediment loadings due to
clear-cutting
7. Sediment loadings due to
building construction
8. Bay water quality (nutrient
level)
9. Bay water quality
(sediment level)
10. Bay water quality (toxics
and pathogens)
11. Ocean acidification
12. Ocean temperature rise
13. Coral reef health
14. Coral reef health
73
Guánica Bay Watershed
Affected Strength of I am ___ confident in my
Relationship is estimate (check one box):
Resource/Outcome (assign % chance
(States/Impacts): with an X): Slightly Somewhat Very
0 X 100 X
Lung Cancer (You might give this answer if you are quite certain
there is a strong relationship.)
0 X 100 X
More Frequent and ( Youmight give this answer if you are quite
Severe Hurricanes uncertain about therelationship.)
Reservoir and drinking 0 100
water quality
0 100
Bay water quality
Reservoir and drinking 0 100
t lit
Bay water quality 0 100
Reservoir and drinking 0 100
water quality
0 100
Bay water quality
0 100
Bay water quality
0 100
Coral reef health
0 100
Coral reef health
0 100
Coral reef health
Coral reef health 0 100
Coral reef health 0 100
Fisheries 0 100
Tourism 0 100
123
74 Environ Syst Decis (2014) 34:60–87

Guánica Bay Watershed

I believe that this By this amount I am ___confident in my

would This
Management Option: (check one box): estimate (check one box):
improve: Outcome:
A Little Moderately A Lot Slightly Somewhat Very

Example: Anti-smoking advertising Respiratory Health X X

Reservoir and
1. Restrictions on agrochemicals Drinking Water
Quality
2. Wastewater treatment wetlands Bay Water Quality

3. Advanced wastewater treatment Bay Water Quality

4. Rio Loco streambank riparian
Bay Water Quality
plantings
5. Hydroseeding of areas with bare
soil in high elevation erodible soil Bay Water Quality
areas
6. Cover crop outreach and cost
Bay Water Quality
share to high elevation coffee farms
7. Restoration of Guánica Lagoon Bay Water Quality
8. Reef education for youth and their
Bay Water Quality
parents
9. Subsidy for shade grown coffee Bay Water Quality

10. Marine protection areas Coral Reef Health

11. Other

12. Other

Identify a portfolio of options that you Bay Water Quality

believe would be best, and predict its
overall effect on Coral Reef Health
1.
2.
3. Other?
4.

123
Environ Syst Decis (2014) 34:60–87 75

Guánica Bay
Watershed

I believe this study will

Key scientific reduce the uncertainty by I am ___confident in my
uncertainties and This uncertainty could be reduced
this amount estimate (check one box):
by the following study or studies
data gaps: (check one box):
A Little Moderately A Lot Slightly Somewhat Very
a. Calibration and use of a X
watershed nutrient model, such as X
SPARROW
X X
Example: Nitrogen
b. Intensive monitoring program for
loss rates in the
sediment & nitrogen transport in Rio (These answers indicate a belief that the
Rio Loco
Loco monitoring study is more likely to be effective
than model development.)

1a.

1.

1b.

1c.

2a.
2.

2b.

2c.
3.

3a

3b.

123
76
Glossary
Drivers Socioeconomic sectors that
drive human activities (Waste
disposal, agriculture,
construction, fisheries, tourism)
Ecosystem Services The products of ecological
functions or processes that
directly or indirectly contribute
to human well-being (clean air
and water, food and fiber,
erosion and flood control,
habitat and biodiversity,
climate stability, and esthetic
enjoyment)
Hydroseeding A planting process which
utilizes a slurry of seed and
mulch, which is transported in
a tank, either truck- or trailer-
mounted and sprayed over
prepared ground in a uniform
layer
Impacts Effects of environmental
degradation on ecosystem
functioning, affecting the
quality and value of ecosystem
services
Management and A number of alternatives that are
policy options under the control of and from
which one or a combination of
several of them (to be
implemented as a strategy) can
be chosen
Pathogen Microorganisms (e.g., bacteria,
viruses, or parasites) that can
cause disease in humans,
animals, and plants
Pressures Human activities that stress the
environment (discharge, boating
activities, climate change, land
use/land cover change, coastal
erosion)
Riparian Of or relating to or located on
the banks of a river or stream
States Reflect condition of the natural
and living phenomena (such as
air, water and soil parameters
and growth, survival, and
reproductive parameters)
123
Environ Syst Decis (2014) 34:60–87
Strength or magnitude The degree to which one variable
of the relationship is associated with or can cause a
(between variables) change in a second variable (i.e.,
between decisions and outcomes)
Toxics Poisonous chemicals
Uncertainty Inability to predict outcomes
due to random variability (for
example, streamflow is
sometimes high and sometimes
low) or incomplete scientific
knowledge regarding causal
relationships (for example, how
does a given concentration of
sediments in the harbor affect
coral reef growth rates)
References
SPARROW: http://water.usgs.gov/nawqa/sparrow/
Questions in the face-to-face elicitation
1. How would you rate the following outcomes in relation to
one another? (e.g., a 1 for tourism and a 2 for fish indicates
that fish health is twice as important as tourism health).
tourism -
fish -
coral -
2. What percentages of the total loadings (nutrient and
sediment) to the Guánica inland water system comes from
development, agriculture, and sewage, respectively? (per-
centages should sum to 100 %)
development -
agriculture -
sewage -
3. How sure are you that the lagoon will work (i.e., be
effective in reducing loadings that enter the Bay)?
I am _% sure that the lagoon will work
4. What are the probabilities that the following sets of
environmental stressors would produce: (a) good/bad coral
reef health; and (b) good/bad fisheries health, respectively?
(percentages should sum to100 %)
stressors for coral reef health:
water quality (WQ),
ocean warming/acidification (OW)
marine protection areas (MPA)
Environ Syst Decis (2014) 34:60–87 77

Stressors for fisheries health: % that it’s all 3 (CR/OW/MPA) -

coral reef health (CR) % that it’s these 2 (CR/MPA) -
ocean warming/acidification (OW) % that it’s these 2 (CR/OW) -
marine protection areas (MPA) % that it’s these 2 (MPA/OW) -
Example 1: If water quality is considered to be most % that it’s only 1 factor (CR) -
responsible, followed by ocean acidification/warming, and % that it’s only 1 factor (MPA) -
then marine protection areas (considered useless in this % that it’s only 1 factor (OW) -
example), and no synergism is assumed, the following
probabilities could apply: Appendix 2: Explanation of BBN
25 % WQ/OW/MPA
20 % WQ/MPA Our BBN was designed to represent the current situation of
25 % WQ/OW coral reefs stressors and management in the Guánica Bay
5 % MPA/OW Watershed, Puerto Rico from the viewpoint of stakeholders.
20 % WQ Based on elicitations and discussions at the workshop in
0 % MPA Puerto Rico, we developed a model in Netica (Norsys 2010)
5 % OW based on the DPSIR/DL framework (Rehr et al. 2012) that
summarizes the essential components involved in coral reefs
Example 2: If water quality combined with ocean warming/ management. We included management options, environ-
acidification and MPAs is thought to be the most important mental processes, and ecosystem services outcomes. Each
set of stressors contributing to coral health, followed by node in Fig. 2 represents a particular variable that is part of
water quality and ocean warming/acidification, and then coral reefs management. Each arrow in Fig. 2 represents a
followed by water quality and MPAs, and assuming causative link between two nodes. In this explanation of the
synergism among the various factors, the following prob- model, we use the BBN and inputs for Participant A.
abilities could apply: At the lower right of the diagram is the endpoint of the
50 % WQ/OW/MPA BBN: Benefits. Benefits is continuous variable distributed
30 % WQ/OW over ten intervals, and a function of Tourism, Fisheries,
10 % WQ/MPA Coral Reef Health and Coral Eco Services (ecosystem ser-
4 % MPA/OW vices), the four inputs or outcomes of interest to stakeholders
5 % WQ that influence the level of benefits. Generally, the greater the
1 % MPA inputs, the greater the resulting benefits. However, the total
2 % OW benefits are weighted according to the elicited values
stakeholders place on each outcome in relation to each other.
a. Probabilities that these sets of stressors lead to good/bad
For example, Participant A believes that coral health is twice
coral reef health:
as important as tourism and fisheries and has the following
% that it’s all 3 (WQ/OW/MPA) - equation for benefits (the particular values applied to the
% that it’s these 2 (WQ/MPA) - weightings are set to correspond to values used throughout
% that it’s these 2 (WQ/OW) - the model and will be discussed later in this document):
% that it’s these 2 (MPA/OW) -
Benefits ¼ 150  Tourism þ 150  Comm: & Rec Fishing
% that it’s only 1 factor (WQ) -
þ 300  Coral Reef Health
% that it’s only 1 factor (MPA) -  CoralEco Services
% that it’s only 1 factor (OW) -
b. Probabilities that these sets of stressors lead to good/bad The following is the data table in Netica for Benefits for
fisheries health: Participant A:

123
78 Environ Syst Decis (2014) 34:60–87

The first input into Benefits, Tourism, is a discrete var- influence the level of tourism. With improvements in these
iable and can be either low or high. Lagoon WQ, Comm. & inputs come improvements in tourism. The following is the
Rec Fishing, and Coral Reef Health are three inputs that data table in Netica for Tourism for Participant A:

123
Environ Syst Decis (2014) 34:60–87
The second input into Benefits, Comm. & Rec Fishing, is
a discrete variable and can be either poor or good. Coral
Reef Health, Marine Protect (MPAs), FishLinks, and
Ocean Warm/Acid are four inputs that influence the level of
Comm. & Rec Fishing. As coral reef health improves and if
MPAs are applied, the probability that fishing will be good
tends to increase. As ocean warming/acidification increa-
ses, the probability that fishing will be good tends to
decrease. FishLinks is a discrete variable that contains
elicited probabilities (that sum to 100 %) that varying sets
of environmental stressors (coral reef health, MPAs, and
ocean warming/acidification) will produce poor or good
fisheries. FishLinks can be influenced by a node Fisheries
Research, a discrete variable that allows for the possibility
79
of testing the effects of different research outcomes for
FishLinks, and updating prior probabilities based on new
evidence. The probability that fishing will be good is
adjusted by the elicited inputs into Fish Links, which can
take into account the belief that there is synergism among
variables. For example, Participant A’s inputs into Fish-
Links are shown in the table below:
The higher the probability placed on a set of stressors
that contains coral reef health and MPAs, the higher the
probability that fishing will be good. The higher the
probability placed on a set of stressors that contains ocean
warming/acidification, the lower the probability that fishing
will be good tends. The following is the data table in Netica
for Comm. & Rec Fishing for Participant A:
123
80 Environ Syst Decis (2014) 34:60–87

123
Environ Syst Decis (2014) 34:60–87
The third input into Benefits, Coral Reef Health, is a
discrete variable and can be either poor or good. Bay &
Water Quality, Marine Protect (MPAs), Coral Links,
and Ocean Warm/Acid are the four inputs that influence
the level of coral health. As bay water quality improves
and if MPAs are applied, coral reef health increases. As
ocean warming/acidification increases, the probability
that coral reef health will be good decreases. Coral
Links is a discrete variable that contains elicited prob-
abilities (that sum to 100 %) that varying sets of
environmental stressors (MPAs, ocean water quality,
and ocean warming/acidification) will produce poor or
good coral health. Coral Links can be influenced by a
node Coral Effects Research, a discrete variable that
allows for the possibility of testing the effects of dif-
81
ferent research outcomes for CoralLinks, and updating
prior probabilities based on new evidence. The proba-
bility that coral reef health will be good is adjusted by
the elicited inputs into Coral Links, which can take into
account the belief that there is synergism among vari-
ables. For example, Participant A’s inputs into Coral-
Links are shown in the table below:
The higher the probability placed on a set of stressors
that contains ocean water quality and MPAs, the higher
the probability that coral reef health will be good. The
higher the probability placed on a set of stressors that
contains ocean warming/acidification, the lower the
probability that fishing will be good tends. The following
is the data table in Netica for Coral Reef Health for
Participant A:
123
82
Coral Reef Health is multiplied by Coral Eco. Ser-
vices, a discrete variable with probabilities set at 25 %
that they are low, 50 % that they are medium, and 25 %
that they are high. Coral Eco. Services can be influenced
by a node Coral Eco. Services Research, a discrete
variable that allows for the possibility of testing the
effects of different research outcomes for Coral Eco.
Services, and updating prior probabilities based on new
evidence.
123
Environ Syst Decis (2014) 34:60–87
Marine Protect (MPAs), an input into both Comm. &
Rec Fishing and Coral Reef Health, is one of the five
management options included in the model. Marine Protect
(MPAs) is a discrete variable and can be either applied
(Yes = 100 %) or not (No = 100 %). Implementation of
MPAs is believed to increase the probabilities that coral
reef health and fishing are good.
Ocean Warm/Acid, an input into both Comm. & Rec
Fishing and Coral Reef Health, is a discrete variable and
Environ Syst Decis (2014) 34:60–87
can be either high or low. Left to chance these probabilities
are set at 50 % that it is low and 50 % that it is high.
Bay & Ocean Water Quality, an input into Coral Reef
Health, is a discrete variable and can be either poor or
good. Inland Water Quality and Lagoon WQ (water qual-
ity) are the inputs that influence the level of bay water
quality. As the probability that inland and lagoon water
quality are good increase, the probability that bay water
quality will be good also increases. Lagoon WQ is a dis-
crete variable and can be either none (if the node is not
activated), poor, or good. This node is only activated when
the management option, Restore Lagoon, is implemented.
Restore Lagoon is a discrete variable and can be either
applied (Yes = 100 %) or not (No = 100 %). Restoring
the lagoon is believed to increase the probability that bay
and ocean water quality will be good if inland water quality
is not too poor. The following is the data table in Netica for
Lagoon WQ for Participant A:
The following is the data table in Netica for Bay &
Ocean Water Quality for Participant A:
83
Inland Water Quality, an input into Lagoon WQ and
Bay & Ocean Water Quality, is a discrete variable and
can be either poor or good. Total Load is the main
input into Inland Water Quality. The following is the
data table in Netica for Inland Water Quality for Par-
ticipant A:
Total Load, the total pollution load, is a continuous
variable distributed over six intervals (very low, low,
moderately low, moderately high, high, and very high) and
a function of individual loading sources (SewLoad, Ag-
Load, and DevLoad) and their associated hypothetical
reductions (SewRed, AgRed, and DevRed) (management
options), as shown in the following equation:
 
SewLoad
Total Load ¼ SewLoad  1  þ AgLoad
  100
AgLoad
 1 þ DevLoad
 100 
DevLoad
 1
100
SewLoad, AgLoad, and DevLoad are discrete variables
and can be low, medium, high, or very high. Loading
distributions for the individual sources were computed over
the low, medium, high, and very high in a manner that
minimized variance based on stakeholders’ prior beliefs.
As the distributions tend toward the very high, the total
load tends toward the very high. Loading values included
in the model are relative (and therefore unitless) though
roughly scale to mg/L concentrations of suspended solids
in unpolluted source waters (very low = 0–25;
low = 25–50), moderately polluted source waters (mod-
erately low = 50–125; moderately high = 125–250), and
highly polluted source waters (high = 250–500; very
123
84 Environ Syst Decis (2014) 34:60–87

high = 500–750). The range of values used for TotalLoad

of 0–750 was thought to allow for a more realistic distri-
bution (with six intervals from low to very high) than
would a smaller range. Since the analysis is of a compar-
ison of benefits, the actual units used are not important.
SewLoad, AgLoad, and DevLoad can be influenced by
the nodes Sew Load Research, Ag Load Research, and Dev
Load Research, which are discrete variables with four
possible outcomes each, and which allow for the possibility
of testing the effects of different research outcomes, and
updating prior probabilities based on new evidence. The
following table in Netica for SewLoad shows the likelihood
functions (false ?/false- rates) for Participant A (they
indicate that the research is nearly perfect, with the prob-
ability that the correct inference is made equal to 94 %): The management options, SewRed, AgRed, and DevRed,
are discrete variables and can be set to a 0 % (None), 40,
70, or 90 % reduction. The following is the first and last
parts of the lengthy data table in Netica for TotalLoad for
Participant A:

123
Environ Syst Decis (2014) 34:60–87
References
Apostolakis GE, Pickett SE (1998) Deliberation: integrating analyt-
ical results into environmental decisions involving multiple
stakeholders. Risk Anal 18(5):621–634
Bartlett G (2011) Joint fact finding and stakeholder consensus building
at the Altamont Wind Resource area in California. In: Burger J
(ed) Stakeholders and scientists. Springer, New York, pp 255–281
Beierle TC (2002) The quality of stakeholder-based decisions. Risk
Anal 22(4):739–749
Belton V, Stewart TJ (2001) Multiple criteria decision analysis.
Springer, New York
Boesch DF (2006) Scientific requirements for ecosystem-based
management in the restoration of Chesapeake Bay and coastal
Lousiana. Ecol Eng 26:6–26
Borja A (2005) The European water framework directive: a challenge
for nearshore, coastal and continental shelf research. Cont Shelf
Res 25(14):1768–1783
Borsuk M, Clemen R, Maguire L, Reckhow K (2001) Stakeholder
values and scientific modeling in the Neuse river watershed.
Group Decis Negot 10:355–373
Brodie JE, Kroon FJ, Schaffelke B, Wolanski EC, Lewis SE, Devlin IC,
Bohnet ZT, Bainbridge JW, Davis AM (2012) Terrestrial pollutant
runoff to the Great Barrier Reef: an update of issues, priorities and
management responses. Mar Pollut Bull 65(4):81–100
Burke L, Maidens J (2004) Reefs at risk in the Caribbean. World
Resources Institute, Washington, DC
Center for Watershed Protection (2008) Guanica bay watershed
management plan. NOAA Coral Reef Program and DRNA,
Puerto Rico
Coglianese C, Allen LK (2004) Does consensus make common sense?
An analysis of EPA’s common sense initiative. Environ Sci
Policy Meet 46(1):11–23
Cohen SJ (1997) Scientist–stakeholder collaboration in integrated
assessment of climate change: lessons from a case study of
Northwest Canada. Environ Model Assess 2(4):281–293
85
Cohon JL (2004) Multiobjective programming and planning. Aca-
demic Press, New York
Connelly S, Richardson T (2004) Exclusion: the necessary difference
between ideal and practical consensus. J Environ Plan Manag
47(1):3–17
Cullen AC, Frey HC (1999) Probabilistic techniques in exposure
assessment: a handbook for dealing with variability and uncer-
tainty in models and inputs. Springer, New York
Cullen AC, Small MJ (2004a) The role and limits of quantitative
assessment. In: McDaniels T, Small MJ (eds) Risk analysis and
society. Cambridge University Press, New York, pp 163–212
Cullen AC, Small MJ (2004b) Uncertain risk. In: McDaniels T, Small
MJ (eds) Risk analysis and society: an interdisciplinary charac-
terization of the field. Cambridge University Press, Cambridge
DeKay ML, Small MJ, Fischbeck PS, Farrow RS, Cullen AC, Kadane
JB, Lave L, Morgan MG, Takemura K (2002) Risk-based
decision analysis in support of precautionary policies. J Risk Res
5(4):391–417
Dietz T (2003) What is a good decision? Criteria for environmental
decision making. Hum Ecol Rev 10(1):33–39
Doremus H, Tarlock AD (2005) Science, judgment and controversy in
natural resource regulation. Public Land Resour Law Rev
26:1–37
Dreelin EA, Rose JB (2008) Creating a dialogue for effective
collaborative decision making: a case study with Michigan
stakeholders. J Great Lakes Res 34:12–22
Ellison AM (1996) An introduction to bayesian inference for
ecological research and environmental decision making. Eco-
logical 6(4):1036–1046
Emerson K, Orr PJ, Keyes DL, McKnight KM (2009) Environmental
conflict resolution: evaluating performance outcomes and con-
tributing factors. Confl Resolut Q 27(1):27–64
Garcı́a-Sais JR, Appeldoorn R, Bruckner A, Caldow C, Christensen
JD, Lilyestrom C, Monaco ME, Sabater J, Williams E, Diaz E
(2005) The state of coral reef ecosystems of the commonwealth
of Puerto Rico. In: Wadell JE (ed) The state of coral reef
123
86
ecosystems of the United States and Pacific Freely Associated
States: 2005
Gregory R, Failing L, Ohlson D, McDaniels TL (2006) Some pitfalls
of an overemphasis on science in environmental risk manage-
ment decisions. J Risk Res 9(7):717–735
Gutrich J, Donovan D, Finucane M, Focht W, Hitzhusen F,
Manopimoke S, McCauley D, Norton B, Sabatieri P, Salzman
J, Sasmitawidjaja V (2005) Science in the public process of
ecosystem management: lessons from Hawaii, Southeast Asia,
Africa and the US Mainland. J Environ Manag 76(3):197–209
Hassan R, Scholes R, Ash N (eds) (2005) Ecosystems and human
wellbeing: current status and trends, Millenium Ecosystem
Assessment. Island Press, Washington, DC
Henriksen HJ, Barlebo HC (2008) Reflections on the use of Bayesian
belief networks for adaptive management. J Environ Manag
88(4):1025–1036
Henriksen HJ, Rasmussen P, Brandt G, von Bülow D, Jensen FV
(2004) Engaging stakeholders in construction and validation of
Bayesian belief networks for groundwater protection. Paper read
at IFAC
Hoffman S, Fischbeck P, Krupnick A, McWilliams M (2007a)
Elicitation from large, heterogeneous expert panels: using
multiple uncertainty measures to characterize information qual-
ity for decision analysis. Decis Anal 4(2):1–19
Hoffman S, Fischbeck P, Krupnick A, McWilliams M (2007b)
Informing risk-mitigation priorities using uncertainty measures
derived from heterogeneous expert panels: a demonstration using
foodborne pathogens. Reliab Eng Syst Saf
Jaynes ET (2003) Probability theory: the logic of science. Cambridge
University Press, Cambridge
Kiker GA, Bridges TS, Varghese A, Seager TP, Linkov I (2005)
Application of multicriteria decision analysis in environmental
decision making. Integr Environ Assess Manag 1(2):95–108
Leach WD (2006) Public involvement in USDA Forest Service policy
making: a literature review. J For 104(1):43–49
Lipsky RS, Ryan CM (2011) Nearshore restoration in Puget sound:
understanding stakeholder values and potential coalitions. Coast
Manag 39(6):577–597
Makowski M, Somlyody L, Watkins D (1996) Multiple criteria
analysis for water quality management in the Nitra basin. J Am
Water Resour Assoc 32(5):937–951
Matso KE, Becker M (2013) Funding science that links to decisions:
case studies involving coastal land use planning projects. Estuar
Coasts. doi:10.1007/s12237-013-9649-5
Messner F, Zwirner O, Karkuschke M (2006) Participation in multi-
criteria decision support for the resolution of a water allocation
problem in the Spree River basin. Land Use Policy 1:63–75
Miller J, Waara R, Muller E, Rogers C (2006) Coral bleaching and
disease combine to cause extensive mortality on reefs in the US
Virgin Islands. Coral Reefs 25(3):418
Morelock J, Ramirez WR, Bruckner A, Carlo M (2001) Status of
coral reefs, southwest Puerto Rico. Caribb J Sci, Online Special
Publication 4:57
Morgan MG, Henrion M, Small MJ (1990) Uncertainty: a guide to
dealing with uncertainty in quantitative risk and policy analysis.
Cambridge University Press, Cambridge
National Research Council (1996) Understanding risk: informing
decisions in a democratic society. Edited by Committee on Risk
Characterization. National Academy Press, Washington, DC
Nelkin D (1975) The political impact of technical expertise. Soc Stud
Sci 5:35–54
Newton AC, Marshall E, Schreckenberg K, Golicher D, te Velde DW,
Edouard F, Arancibia E (2006) Use of a Bayesian belief network
to predict the impacts of commercializing non-timber forest
123
Environ Syst Decis (2014) 34:60–87
products on livelihoods. Ecol Soc 11(2):24. http://www.ecolo
gyandsociety.org/vol11/iss2/art24/
Netica Application for Belief Networks and Influence Diagrams:
User’s Guide; Versions 4.16 for Windows, Vancouver, Canada
Peuhkuri T (2002) Knowledge and interpretation in environmental
conflict: fish farming and eutrophication in the Archipelago Sea,
SW Finland. Landsc Urban Plan 61(2):157–168
Principe P, Bradley P, Yee S, Fisher W, Johnson E, Allen P, Campbell
D (2012) Quantifying coral reef ecosystem services research.
U.S. Environmental Protection Agency, Office of Research and
Development, Triangle Park
Ramos-Scharron CE (2009) Sediment production from natural and
disturbed surfaces in dry tropical areas of La Parguera-Puerto
Rico, 2003–2005. Island Resources Foundation, Dept of Geog-
raphy and the Environment, University of Texas-Austin
Rauschmayer F, Wittmer H (2006) Evaluating deliberative and
analytical methods for the resolution of environmental conflicts.
Land Use Policy 23:108–122
Reed MS (2008) Stakeholder participation for environmental man-
agement: a literature review. Biol Conserv 141(10):2417–2431
Rehr AP, Mitchell J, Small PB, Fisher WS, Vega A, Black K,
Stockton T (2012) A decision support framework for science-
based, multi-stakeholder deliberation: a coral reef example.
Environ Manag 50(6):1204–1218
Renn O (2006) Participatory processes for designing environmental
policies. Land Use Policy 23(1):34–43
Ribarova I, Assimacopoulos D, Jeffrey P, Daniell KA, Inman D,
Vamvakeridou-Lyroudia LS, Melin T, Kalinkov P, Ferrand N,
Tarnaki K (2011) Research-supported participatory planning for
water stress mitigation. J Environ Plan Manag 54(2):283–300
Sanger D, Hernandez D, Libes S, Voulgaris G, Davis B, Smith E,
Shuford R, Porter D, Koepfler E, Bennett J (2010) A case history
of the science and management collaboration in understanding
hypoxia events in Long Bay, South Carolina, USA. Environ
Manag 46(3):340–350
Sarewitz D (2004) How science makes environmental controversies
worse. Environ Sci Policy 7:385–403
Stiber NA, Pantazidou M, Small MJ (1999) Expert system method-
ology for evaluating reductive dechlorination at TCE sites.
Environ Sci Technol 33(17):3012–3020
Stiber NA, Small MJ, Pantazidou M (2004) Site-specific updating and
aggregation of bayesian belief network models for multiple
experts. Risk Anal 24(6):1529–1538
UNEP (2006) In the front line: shoreline protection and other
ecosystem services from mangroves and coral reefs. UNEP-
WCMC, Cambridge
Van Wees JD, Mijnlieff H, Lutgert J, Breunese J, Bos C, Rosenkranz
P, Neele F (2008). A Bayesian belief network approach for
assessing the impact of exploration prospect interdependency: an
application to predict gas discoveries in the Netherlands. AAPG
Bulletin 92(10):1315-1336
Warne AG, Webb RMT, Larsen MC (2005) Water, sediment, and
nutrient discharge characteristics of rivers in Puerto Rico, and
their potential influence on Coral Reefs. Scientific Investigations
Report 2005–5206. Prepared in cooperation with the Puerto Rico
Department of Natural and Environmental Resources. U.S.
Department of the Interior, U.S. Geological Survey, Reston,
Virginia
Whelan KRT, Miller J, Sanchez O, Patterson M (2007) Impact of the
2005 bleaching event on Porites porites and Colpophyllia natans
at Tektite Reef, US Virgin Islands. Coral Reefs 26(3):689–693
Woody K, Atkinson A, Clark R, Jeffrey C, Lundgren I, Miller J,
Monaco M, Muller E, Patterson M, Rogers C, Smith T, Spitzak
T, Waara R, Whelan K, Witcher B, Wright A (2008) Coral
Environ Syst Decis (2014) 34:60–87 87

bleaching in the U.S. Virgin Islands in 2005 and 2006. In: Yokota F, Thompson KM (2004) Value of information analysis in
Wilkinson C, Souter D (eds) Status of Caribbean coral reefs after environmental health risk management decisions: past, present,
bleaching and hurricanes in 2005. Global Coral Reef Monitoring and future. Risk Anal 24(3):635–650
Network, and Reef and Rainforest Research Centre, Townsville

123
Copyright of Environment Systems & Decisions is the property of Springer Science &
Business Media B.V. and its content may not be copied or emailed to multiple sites or posted
to a listserv without the copyright holder's express written permission. However, users may
print, download, or email articles for individual use.