Is Collaboration A Good Investment Modeling

Journal of Public Administration Research And Theory, 2016, 769–786
doi:10.1093/jopart/muw033
Article
Advance Access publication May 21, 2016
Article
Is Collaboration a Good Investment? Modeling

the Link Between Funds Given to Collaborative
Watershed Councils and Water Quality
Tyler A. Scott
Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

University of Georgia
Address correspondence to the author at tyler.scott@uga.edu.
Abstract
Collaborative environmental governance strategies seek to improve management efforts by involv-
ing nongovernmental stakeholders and spanning sectoral and geographic boundaries, but it is
unclear whether government funding for such efforts efforts pays off in terms of improved environ-
mental outcomes. This paper explores a common case of collaborative governance—collaborative
watershed councils (WCs)—and examines whether the actions of collaborative WCs improve water
quality. Coupling longitudinal data concerning 1984 grants given to local WCs in the state of Oregon
with water quality monitoring data sampled at 161 sites, this analysis tests whether there is a prima
facie case that the actions of local collaborative institutions improve environmental outcomes. Along
with presenting some of the first evidence about the impacts of collaborative governance that is
based on an objective outcome metric, this paper also contrasts different council actions and consid-
ers how institutional capacity impacts council efforts. WC actions are shown to be associated with
improved water quality, but the magnitude of improvement is dependent on council attributes and
the type of project for which funds are allocated. This paper also makes a methodological contribu-
tion by demonstrating the use of hierarchical Bayesian modeling to account for spatial and temporal
dependencies present in observational data.
Introduction environmental applications, particularly watershed

Public managers increasingly rely on collaborative and water quality management (Hardy and Koontz
relationships with nonprofit organizations to imple- 2008; Leach et al. 2013). However, while collaborative
ment policies or provide services (Salamon 2002). environmental governance is popular, it is also costly,
Collaborative management with local nonprofit groups time-consuming, and subject to considerable uncer-
gives governments a community-based vehicle through tainty (Margerum 2011). Thus, the lack of evidence
which to implement policies and programs, and pro- concerning the environmental outcomes of collabora-
vide nonprofits with access to funding and other tive governance (Thomas and Koontz 2011) is not just
resources (Nikolic and Koontz 2008). Governance a theoretical gap but a pressing empirical question for
arrangements of this form are very common in public managers who invest in collaboration under the
presumption that the benefits outweigh the costs.
In this paper, I build on the considerable body of
An early version of this article was presented at the 2014 Association for
research discussing the role that governments play in—
Public Policy and Management Research Conference in Albuquerque,
NM. The author thanks Craig Thomas, Ryan Scott, and Grant Blume for and resultant impacts of—supporting collaborative
commenting on drafts, Havard Rue for providing modeling assistance, governance (Ansell and Gash 2008; Berardo, Heikkila,
and three anonymous reviewers for their constructive reviews. and Gerlak 2014; Emerson and Nabatchi 2015a;
© The Author 2016. Published by Oxford University Press on behalf of the Journal of Public Administration Research and Theory, Inc. 769
All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
770 Journal of Public Administration Research and Theory, 2016, Vol. 26, No. 4
Lubell and Fulton 2008; Nikolic and Koontz 2008) literature poses two general rationales as to why invest-
by asking a relatively simple question that has proven ment in collaborative governance can lead to improved
highly elusive in practice (Koontz and Thomas 2006): policy outcomes.
Does collaborative governance improve environmental First, collaborative governance can be more effective
outcomes? There have been numerous studies concern- than efforts carried about a single agent (Lasker, Weiss,
ing the input–output relationship (i.e., outputs such as and Miller 2001). Among the reasons for this expecta-
collaboratively developed plans and policies) (Thomas tion are that collaborative management will facilitate a
and Koontz 2011) but very few that have spoken to the more comprehensive understanding of the policy prob-
environmental outcomes associated with collaborative lem (Leach et al. 2013), be more responsive to local
governance (Margerum 2011; Scott 2015; Ulibarri needs and concerns (Nikolic and Koontz 2008), foster
2015). This analysis addresses the question of whether “buy-in” from stakeholders to support implementa-
collaborative governance improve environmental out- tion (Ansell and Gash 2008; Sabatier 2005), and carry
comes by combining longitudinal water quality moni- greater legitimacy and mitigate public mistrust of gov-
toring data with administrative records of funding ernance (Bryson, Crosby, and Stone 2006; Margerum

given to collaborative watershed management councils 2011). Collaborative governance also addresses nor-
in the state of Oregon. mative considerations related to citizen involvement
To model these data, I use a Bayesian hierarchical (Yang and Callahan 2007), in that stakeholder par-
modeling strategy (Rue, Martino, and Chopin 2009) ticipation can be an effective means to accomplish the
to account for the complex spatiotemporal nature values of good governance (Fung 2015).
of these data (Lindgren, Rue, and Lindström 2011). The second broad rationale is that through prin-
Accordingly, this paper also makes a methodological cipled engagement among participants, collaborative
contribution by demonstrating a method for overcom- governance can foster shared motivation and enable
ing some of the analytical challenges (such as irregular joint actions that could not be accomplished separately
data and spatial and temporal dependencies) that have (Emerson and Nabatchi 2015a). For instance, collabo-
prevented researchers from linking collaborative man- rative governance has been linked to intermediate out-
agement efforts to objective environmental outcome comes such as belief change, social capital, stakeholder
data in the past. learning, and conflict alleviation (Berardo, Heikkila,
The next two sections describe the theoretical and Gerlak 2014; Gerlak and Heikkila 2011; Leach
rationale for this research and provide background et al. 2013). Even if actors do not agree on a given
concerning the case at hand. I then outline my ana- course of action, the development of shared commit-
lytical approach and describe model specifications. The ment and understanding within collaborative institu-
analysis that follows considers whether collaborative tions can enable future collective action by reducing
watershed council (WC) actions (using project-specific bargaining and enforcement costs (Emerson and
funding as a proxy for actions) are associated with Nabatchi 2015a).
changes in water quality, and further how capacity In short, policymakers and public managers choose
building funds enhance the predicted effectiveness of collaborative governance because they believe that it
other council actions. I then discuss the implications will improve the design and implementation of policies
of these results and consider the strengths and weak- and programs and thereby improve policy outcomes.
nesses this approach for measuring the outputs and Within environmental management, collaborative
outcomes of collaborative governance. In conclusion, governance continues to proliferate as a response to
I address how this analysis of the link between col- complex environmental problems (Margerum 2011);
laborative watershed management and water quality however, whether such collaboration leads to better
is relevant to the application and understanding of col- environmental outcomes remains a major—and largely
laborative governance more broadly. open—question (Newig and Fritsch 2009a, 2009b;
Scott 2015; Ulibarri 2015; Young et al. 2013).
This question matters because collaborative gov-
Rationale ernance entails potential costs as well as benefits.
Collaborative governance, “in which a… group of Collaborative governance is deliberative and consen-
autonomous stakeholders deliberates to build con- sus-oriented (Ansell and Gash 2008) and can involve a
sensus and develop networks” (Margerum 2011, 6) in great deal of time and effort (Margerum 2011). Further,
order to “make or implement public policy or man- in contrast with direct government actions where the
age public programs or assets” (Ansell and Gash 2008, implementing agency has stricter control over policy
544), does not just happen, but rather policymakers efforts (Salamon 2002), policy implementation via
choose collaborative governance as a toolbox through collaborative governance has a heightened degree of
which to design and implement policies (Layzer 2008; uncertainty from the perspective of the funding agency
Scott and Thomas 2016). The collaborative governance since the involvement of more actors creates issues of
Journal of Public Administration Research and Theory, 2016, Vol. 26, No. 4 771
accountability and control (Weber 2003). There is also of Oregon Administrative Code item 695-010-0060,1
concern that government support for local collabora- which outlines priorities such as remediating human-
tive governance efforts can detract from the ability of caused alterations, changing land management prac-
these local organizations to operate with responsive- tices to address causal drivers, and focusing on upslope
ness and flexibility, reducing the very qualities that are and upstream improvements. Projects that meet stated
presumed to make such groups effective (Nikolic and priorities are then ranked in terms of expected net ben-
Koontz 2008). Answers as to whether presumed benefits (factoring in project costs) and likelihood of suc-
efits outweigh these potential costs remain elusive in cess. Not only is it thus clear that OWEB aims to fund
large part because the complexity of social-ecological projects and programs with favorable circumstances
systems makes it difficult to trace how government and high success probabilities, but it also is reasonable
support for nonprofit collaborative groups ultimately to assume that the most motivated and capable appli-
impacts environmental outcomes (Thomas and Koontz cants are more likely to win grants. This clearly means
2011). that councils (and targeted watersheds) that do and do
not win grants differ in important ways.

Standard approaches to mitigating this type of selec-
Background tion bias include the use of panel data and controlling
To address this question, I use publicly available water for confounding variables to support an assumption
quality monitoring data to explore the impact of 1984 that assignment is ignorable (and this analysis does in
grants given by a state agency, the Oregon Watershed fact rely upon panel time series data and an extensive
Enhancement Board (OWEB), to local nonprofit stake- set of relevant covariates in order to estimate the rela-
holder councils engaged in ongoing watershed planning tionship between funding and water quality as accu-
and management activities in watersheds across Oregon. rately as possible). However, considering the context in
Water quality data are a good performance indicator in which state or Federal support for local collaborative
this regard because they provide a consistent and compa- management typically occurs, trying to estimate what
rable statewide metric. Moreover, because water quality would have occurred if wasteful projects and/or less
is influenced both by land use and pollutant discharge capable councils had received grants does not reflect
(Tong and Chen 2002), water quality observations pro- realities in implementing grant programs.
vide a more holistic picture of watershed outcomes than Namely, it is difficult to envision a policy for sup-
would land cover or habitat data alone. porting local collaborative management in which
Created by the state legislature in 1995, OWEB funds would not be not be strategically allocated to
gives grants to local governments, WCs, private achieve maximum benefits. In this sense, while projects
firms, nonprofit organizations, and local citizens and and programs that receive funding are those assumed
landowners for restoration and recovery projects to be most likely to show a positive impact, this type
and programs. Grants to local WCs serve myriad of statewide system of support for local collaborative
purposes ranging from environmental assessment management is a good example of such a management
to hiring a full-time coordinator. Recent work raises arrangement more generally. Therefore, the purpose of
questions regarding the scalability of collaborative this analysis is to evaluate whether there is a prima
governance (Ansell and Torfing 2015); examining a facie case that OWEB’s strategically allocated support
statewide effort to support collaborative manage- for nonprofit collaborative WCs engenders water qual-
ment of local watersheds engages this question in ity improvements. The results thus provide a measure
terms of how collaborative governance scales hori- of the impact that a similar state or regional policy for
zontally. The network of local WCs supported by supporting local collaborative management through
OWEB represents an effort to scale collaborative targeted, competitively allocated support can achieve.
management from select localities to a statewide
effort. In this regard, it is important to emphasize Hypotheses
that the case for this analysis is Oregon’s statewide
OWEB’s statewide, grant-based system also provides a
system of local collaborative watershed management
unique way to address one of the challenges typically
councils, not the individual councils themselves. This
faced in estimating the impact of collaborative group
distinction is important, because the competitive
efforts: even if budget data are available, it is difficult
nature of OWEB’s grant process confounds the abil-
to determine the activities and relative effort level of
ity to estimate a traditional average treatment effect
each group. OWEB grant funds provide a consistent
with respect to individual watershed management
councils.
OWEB awards grants competitively using a review 1 See http://www.oregon.gov/OWEB/GRANTS/docs/grant_review_flowchart.
process that assesses prospective projects on the basis pdf.
approximation of collaborative management efforts and more importantly, how these attributes—high or
across the state over a many years. Further, funding low operating budget, coordinator type (staffed, con-
data provide a continuous metric of council activity tracted, or volunteer), years active, and staff full-time
that can be used to estimate how increased support for equivalents (FTEs)—mediate the predicted impact of
collaborative management relates to environmental grant funds given to a WC. These council attribute var-
outcomes. If collaborative watershed management has iables serve two primary roles: First, they provide alter-
an impact on water quality, greater support (as meas- native measures of council strength and activity aside
ured by grant funding in this case) should be associated from OWEB funds received, which helps ensure that
with a larger predicted impact. OWEB funds are not merely a proxy for particularly
strong or weak councils. Second, because it is expected
Hypothesis 1 (H1): Grant funds given to nonprofit that resource-advantaged, politically connected coun-
watershed councils are associ- cils are better equipped to pursue and win competi-
ated with improved water qual- tive grants, grants given to longstanding councils with
ity in the target watershed. larger operating budgets and more staff should have a

This hypothesis is an important starting place for diminished predicted impact on water quality.
this analysis, since H1 addresses the general issue Grant funds given to each council are interacted
of whether collaborative governance is “all talk and with these council attributes to account for the pos-
no action” (Lubell 2004). This research is one of sibility that the measurable association between fund-
the first analyses that is able to leverage objective ing for collaborative management and environmental
measures of both collaborative governance actions outcomes is more reflective of a given council’s ability
and environmental outcomes. Thus, establishing an to steer both grant receipt and policy actions than it is
association between WC actions and water quality about the actions directly facilitated by OWEB funds.
is meaningful in-and-of itself given the dearth of If this explanation is borne out, the model results
evidence concerning the relationship between col- should still identify an association between funds and
laborative governance and environmental outcomes water quality, but this relationship should diminish as
generally (Koontz and Thomas 2006; Newig and a function of council resources.
Fritsch 2009a). Hypothesis 2 (H2):
The predicted impact of grant
H1 assumes that grant funding given to collabo- funds given to nonprofit water-
rative management institutions is made on the basis shed councils is diminished for
of the expected quantity and quality of the proposed resource-advantaged councils.
actions, and thus more funding equates to more and
better management actions. While this assumption is As described above in the rationale section, public
plausible, it is also probable that the council’s politi- support for collaborative governance can also achieve
cal ties and resource advantages influence the amount intermediate outcomes such as belief change, social
of funds a council procures. In other words, due to the capital, and stakeholder learning (Gerlak and Heikkila
competitive nature of the grant process, councils with 2011; Leach et al. 2013). This can enhance problem-
stronger network connections or greater institutional solving capacity and improve the efficacy of poli-
capacity are more likely to win funding for reasons cies and programs by altering existing incentives and
not wholly related to the prospective impact of the reducing transaction costs (Emerson and Nabatchi
project or program itself. For instance, councils with 2015a; Schneider et al. 2003), for instance by garnering
high social capital might have access to key informa- increased support for management actions or decreas-
tion about the grant review process, and full-time ing enforcement costs. OWEB invests in such capac-
staff can allocate more time and effort to proposal ity building by providing grants explicitly for council
writing. If resource-advantaged councils are more support, funding administrative activities, staffing, and
likely to win funding due to factors unrelated to pro- operations. There are significant transaction costs asso-
ject quality, then on the margin the predicted impact ciated with initiating and maintaining interorganiza-
should be lessened for grants given to resource-advan- tional endeavors, and government provision of staffing,
taged councils. Alternatively, it is also plausible that infrastructure, and other resources are shown to facili-
councils with greater institutional capacity are able tate such network relationships (Schneider et al. 2003).
to design better projects and implement them more To put a fine point on a broad and nuanced litera-
effectively; if so, the predicted impact on a dollar-for- ture, the logic for incurring such costs and investing
dollar basis should be heightened for resource-advan- in collaborative institutions is that a well-functioning
taged councils. collaborative group will result in policy and program
To account for either consideration, I also model outputs that draw on more relevant perspectives
how council attributes impact predicted outcomes (Ansell and Gash 2008; O’Leary, Gerard, and Bingham
2006), have increased commitment from participants prior to observing additional data) with the current
(Ansell and Gash 2008; Bryson, Crosby, and Stone data model (Bernardo and Smith 2009). A posterior
2006), and could not be produced separately (Emerson distribution reflects uncertainty about a parameter esti-
and Nabatchi 2015a). Based upon this rationale, one mate after conditioning on the observed data (similar
would expect that projects and programs carried out to how standard errors and confidence intervals are
in watersheds where there is—or has been—greater used to describe parameter uncertainty in a frequen-
investment in capacity building to have a stronger pos- tist model). For this application, there are two primary
itive association with water quality improvements. To advantages of the Bayesian approach. First, since a
test this, I divide funding into four project categories: posterior parameter distribution is estimated by the
outreach, technical support, restoration, and capacity model, it is easy to obtain the posterior probability that
building (each described in greater detail below) and the parameter does or does not exceed a given value
examine how funds provided for capacity building (Blangiardo et al. 2013). Second, Bayesian methods
influence the predicted impact of other project types. ease the use of hierarchical model structures, which
use random effects to model variance at multiple levels

Hypothesis 3 (H3):
Grant funds given to nonprofit of a model. The Bayesian model used in this analysis
watershed councils in conjunc- includes random effects that model spatial and tempo-
tion with capacity building invest- ral dependence among observations. Bayesian methods
ment are associated with stronger are shown to be highly effective for analyzing data with
water quality improvements. this type of spatiotemporal structure (Dunson 2001).
Hierarchical Bayesian models are typically fit
using Markov chain Monte Carlo (MCMC) algo-
Model and Methods rithms (Brooks et al. 2011), which for complex mod-
In keeping with these hypotheses, the goal of this paper els are greatly time- and computationally intensive
is to estimate the association between grants given (Blangiardo et al. 2013). In lieu of MCMC, this anal-
to WCs and subsequent water quality. Water quality ysis uses a more computationally efficient method,
index observations yitj occur at specific spatial loca- integrated nested Laplace approximation (INLA),
tion i in month t within watershed j. Watersheds are developed by (Rue, Martino, and Chopin 2009) and
distinguished using the US Geological Service’s hydro- widely employed for Bayesian hierarchical mod-
logic unit coding (HUC) system; I group observations eling (Cosandey-Godin et al. 2015; Lindgren, Rue,
by HUC8 location, the eight-digit number that distin- and Lindström 2011; Martino, Akerkar, and Rue
guishes a specific sub-basin. 2011). Given that INLA is relatively new and has as-
Observations near to one another in location or of-yet limited penetration into the policy literature,
time, or within the same watershed, are expected to Appendix A provides background on the INLA esti-
exhibit residual dependency (i.e., correlated residuals). mation method more generally.
These data are thus highly similar to many epidemio- The fundamental unit of analysis for this research
logical data contexts, as there is an outcome (water is the average water quality index score (explained
quality) and a “risk factor” or “confounder” (grant in detail below) at a given monitoring site in a given
funded projects) and where the spatial (sample sites month. That is, water quality observations occur at
in the state of Oregon) and temporal (monthly obser- a specific point. Based on the point-reference model
vations) structure of the data must be accounted for specifications of Cameletti et al. (2013) and Cosandey-
in order to make valid inferences. Accordingly, I use Godin et al. (2015), water quality observations are
a suite of Bayesian hierarchical modeling methods modeled within a given watershed at a specific time
found primarily in the epidemiological (Blangiardo and location as:
et al. 2013; Cameletti et al. 2013), spatial econometrics Z
(Bivand, Pebesma, and Gomez-Rubio 2008; Gomez- η = α h[i ] + ∑βz Sitei + f (.) + τ t [i ] (1)
Rubio, Bivand, and Rue 2014), and ecological mod- z =1

eling (Clark and Gelfand 2006; Cosandey-Godin et al.
2015; Wikle 2003) literatures. where α h[i ] is the random intercept estimated for
A Bayesian model derives a statistical result (called observation i in a given HUC8 watershed h; βz is
a posterior distribution) via an inferential process that the coefficient associated with site covariate z for
combines the prior distribution2 (what was assumed observation i (including elevation and distance
from coast); f(.) represents the semi-parametric
function used to model the spatial random effect;
2 This analysis uses noninformative priors, and so in this case posterior and τ t[i ] represents the random temporal effect for
distributions are a function of the observed data and not the prior
distributions.
observation month (which among other purposes
distinguishes between low-flow [June to September] expectation that sample sites that are nearer in loca-
and high-flow [October to May] months [Cude and tion will be more similar than sites that are further
Curtis 2001]). apart. Appendix A provides more extensive detail
The advantage of fitting random group effects is regarding the mechanics of the Stochastic Partial
two-fold. First, random effects account for differing Differential Equations (SPDE) (Lindgren, Rue, and
within-group sample sizes by placing more emphasis Lindström 2011) approach used to model spatial
on the group mean when there are many observations dependency. In brief, this approach divides continu-
in the group and drawing more broadly from the pop- ous space into discrete groups and assumes that
ulation mean when there are very few observations in conditional on neighboring groups, observations are
the group (Gelman et al. 2013). This helps ensure that independent of one another. In a basic sense, this acts
predicted differences between watersheds (or months) similarly to the other random effects fit for water-
are not simply a product of small within-watershed shed and month, whereby the model assumes that
sample sizes by attenuating the group-level estimates observations are independent conditional on the
for watersheds (or months) with fewer water qual- month and watershed in which they occur. Further

ity samples towards the overall mean (i.e., generating details regarding INLA and SPDE methods are pro-
more conservative group-level estimates than would a vided in the context of the analysis presented below
basic fixed-effect approach). Second, and of particular and in Appendix A.
importance for this analysis, is that the random group
effect can itself be modeled as a function of group- Data
level covariates. This includes important water qual-
Dependent Variable
ity control variables, such as the percentage of land in
the watershed that is developed (e.g., paved or con- Water quality data are obtained from the Oregon
tains buildings) and that is used for agricultural pur- Department of Environmental Quality (ODEQ). The
poses. It also includes grant funding, the variable(s) Oregon Water Quality Index (OWQI) integrates eight
of interest. Since grants are given to WCs, it makes water quality variables (temperature, dissolved oxy-
the most sense to aggregate funding by month and gen, biochemical oxygen demand, pH, total nitrogen,
watershed. The HUC8-specific adjustment is thus total phosphorus, total solids, and fecal coliform) into
itself modeled as: one metric (Cude and Curtis 2001). For variable, the
W
analytical measurement is converted into a quality rat-
α h = α 0 + ∑γ wWatershedwh[i ] (2) ing between 10 (worst case) and 100 (ideal). ODEQ
w =1 then computes OWQI scores using a harmonic square
mean formula:
where α 0 is the complete-sample mean and γ w repre-
sents a vector of coefficients corresponding to water- n
1
shed-level variables 1 to W for an observation (i) in a OWQI = n / ∑ (4)
i =1 SIi2
given watershed (h). Likewise, the month-level random
effect (which is cross-classified with the HUC8 ran- where SIi refers to subindex i (e.g., pH level). In this
dom effect since two observations can be in the same method, the most impaired variable imparts the great-
HUC8 region but different months, or vice-versa) is est influence on overall index score (Cude and Curtis
specified as: 2001), which provides a holistic measure of general
M water quality (since a site cannot have a relatively high
τ t = τ 0 + ∑ζ m Monthmt [i ] (3) overall score if it performs poorly on any metric).3 The
m =1 OWQI is designed to facilitate comparisons between
where τ 0 is the complete-sample mean and ζ m rep- watersheds, and thus sub-indices such as pH and total
resents a vector of coefficients corresponding to solids are adjusted to account for geologic variability
month-level variables 1 to M (e.g., precipitation) for (Cude and Curtis 2001).
an observation (i) in a given month (m). While esti- OWQI scores are observed on an intermittent
mated watershed-level ( γ w ) and month-level ( ζ m ) monthly basis from 2012 through August 2015 at 161
coefficients act directly on the random intercept for a monitoring stations.4 Generally, in the data obtained
given watershed or month, these coefficients can still
be interpreted as estimating how a 1-unit change in a 3 Figure 6 in the “Results” section demonstrates the substantive
given group-level variable predicts water quality and categories into which ODEQ classifies OWQI scores and shows the
distribution of scores in across categories for the data used in this
are presented alongside site-level coefficients ( βz ) in
analysis.
table 2. 4 The OWQI database goes back as far as 1992, but I use observations
Finally, f (.) is a shorthand representation of from 2012 to 2015 to match observed watershed council covariates
the spatial random effect used to account for the included in the analysis.
from ODEQ, index scores are tabulated every other Table 1. OWEB Grants to WCs (Projects Completed
month. In a few cases, however, there are only three January 2009–December 2014)
or four total monthly observations in total at a given Project Type N Average Project Cost ($)
site. One major empirical advantage of the hierarchi-
cal random effects approach I use is that it can read- Capacity 457 80,607
ily handle these irregular data. Thus, there is no need Outreach 99 24,441
to drop any data or to impute data for under-sampled Restoration 1,267 65,801
sites or unsampled months. In total, there are 2,196 Technical 161 58,803
unique month-site observations occurring between
January 2012 and August 2015. funding variables: projects completed in the last 1, 2,
and 3 water-years, respectively.5
Independent Variables
The independent variables of interest are obtained from Covariates
the OWEB. OWEB’s grant database contains record of One of the advantages of a spatiotemporal model is

all grant projects funded by OWEB, including location that accounting for temporal and spatial relation-
(watershed), project start and end dates, project type, ships between observations serves to address many of
and funding amount; this analysis specifically consid- the factors that affect water quality. Nonetheless, sev-
ers 1984 grants given to 66 WCs throughout the state eral covariates are included in the model to actively
between 2009 and 2015 (I test funding up to 3 years model key water quality drivers. First, the western
prior to a water quality observation). Table 1 summa- portion of Oregon receives much more rainfall than
rizes the grants given by OWEB. does the eastern portion and has a much different
Measuring the impact of grant projects and pro- climate. Thus, the model includes linear effects for
grams poses several challenges related to how funding latitude and longitude to account for absolute geo-
is coded across space and time. First, while in some graphic location. I also use historical precipitation
cases, a grant is targeted at a specific site (as is often the data from Oregon State University’s PRISM Climate
case for restoration grants), many OWEB grants have a Group6; using raster image data that summarize
more disparate spatial focus. For instance, an outreach precipitation amount by month, I code the monthly
project might be targeted at an entire watershed. To median precipitation value within each HUC8 water-
model the spatial focus of grants, grants are aggregated shed. I use a basin-wide measure instead of coding
by HUC8 for each month. precipitation level at each sample site because precipi-
Second, OWEB’s grant database provides the award tation upstream can also influence water levels and
date, start date, and completion date for each project, thus water quality.
but these dates do not necessarily reflect how funds Further, land use is a well-established driver of
are actually expended. The true pattern of funds use water quality (Meador and Goldstein 2003; Tong
is not apparent from these data (and likely varies on and Chen 2002). Agricultural land use is linked to
a project-by-project basis); for clarity and consistency, increased chemical pollution (Johnes and Heathwaite
I code project funding in terms of the month of pro- 1997; Skaggs, Breve, and Gilliam 1994), and water-
ject completion and then aggregate funds by water- ways near developed land also demonstrate higher
shed and water-year (a water-year is designated from pollution levels (Wang 2001) (e.g., cars that leak oil
October to the following September to represent the onto pavement, which then washes into streams). To
yearly flow cycle). calculate the proportion of each HUC8 watershed
Third, the impact of a project might not necessarily
correspond directly to the actual project period. For
instance, a restoration grant used to restore stream- 5 Other specifications tested include the entire cumulative grant total as
side riparian areas should have an ongoing impact on well as 5- and 4-year cumulative totals, all of which show a very small
association with water quality. These results are generally consistent
stream turbidity by reducing erosion for years after the
with the shorter duration models, but for longer time horizons predicted
project is finished. An OWEB education or outreach impacts appear to be attenuated, as coefficients become smaller and
grant likewise is expected to have an ongoing (or even closer to zero. The other disadvantage of using longer time horizons is
increasing) impact on stakeholder behavior in the that doing so creates less temporal variation across each site-specific
watershed after the project completion date. Of course, funding covariate since any given project remains “on the books” longer.
A curvilinear relationship between funding levels and water quality was
without continuing effort the effects of restoration or
tested by modeling the natural logarithm of funding. These results were
outreach actions likely dissipate or diminish over time. not substantively different, however, and thus for clarity and simplicity
To evaluate the extent to which project impact might the paper presents the linear (untransformed) funding metrics.
fade out (or possibly amplify), I test three different 6 http://prism.oregonstate.edu.
that this used for agricultural purpose or that is devel- Results

oped, I use 30-m by 30-m raster data from the 2011 Control Variables
National Land Cover Database (NLCD) study. Using Table 2 presents the results associated with baseline
ArcGIS, I produce a raster for each land cover type models that include all covariates and total OWEB
(agricultural land [cropland or pasture], wetlands,7 funding given to the WC that oversees a given sample
forest, and developed land), where each 30-m by 30-m site. Table 2 contains three models corresponding to the
pixel is coded as a “1” if that pixel in the comprehen- three different specifications for past funding described
sive NLCD raster corresponds to the designated land above (projects completed in the last 1, 2, and 3 prior
cover type, and a “0” otherwise. I then summarize the water-years). Just like in a typical regression model, the
percentage of land area for each coverage type within coefficients in table 2 (as well as estimates presented in
every HUC8 unit. While yearly land cover data would subsequent tables and figures) represent the predicted
be ideal, the NLCD data satisfy the purposes at hand impact of a 1-unit change in the independent vari-
by providing a consistent way to account for relative able on the dependent variable. Each parameter has
differences in land use between watersheds that might an additive effect on the dependent variable, OWQI

influence water quality. score.10 A coefficient greater than zero indicates that an
Other control variables included in the model are increase in a variable is associated with an increase in
elevation, the historical OWQI average for a sample OWQI score, and a coefficient less than zero indicates
site between 1992 and 2012, and non-OWEB funds a decrease in OWQI score.
spent on restoration projects throughout the state Table 2 presents the mean and quantile values that
as reported in the Oregon Watershed Restoration encompass 95% of the posterior distribution for each
Inventory (OWRI) database.8 As with OWEB funds, parameter.11 For this credible interval (Gelman et al.
this spending is aggregated by watershed. Elevation 2013), bounds that do not span zero indicate a “sig-
data are obtained from the CGIAR Consortium for nificant” parameter, that is, one that is statistically
Spatial Information, using the R raster (Hijmans unlikely to be equal to zero. Each model in table 2 also
2014) to code the elevation of each sample site. includes an intercept, random effects for month and
WC covariates are obtained from a 2012 survey of HUC8, and a spatially correlated error term.12
Oregon WCs conducted by the Network of Oregon While many of the covariates shown in table 2 are
Watershed Councils (NOWC). Similarly to the land not of substantive interest to this paper, the fact that
cover data, more recent data are not available; thus, most perform as expected helps to demonstrate that
council covariates are fit as time-invariant fixed the model is capturing the phenomenon that it intends
effects.9 These survey data are not highly detailed, to capture. First, the Latitude and Longitude variables
but do provide information concerning standard capture the linear effects of location (independent of spa-
operating budgets, staff FTEs, and the number of tially correlated errors). Generally, Oregon has a wetter
years that a council has been active. The operating climate, more forested land, and less agriculture as one
budget question in NOWC’s survey instrument con- moves west and north; thus, in general predicted water
tained an ordinal categorical set of response options
(e.g., “$200,000–$400,000”). I dichotomize this
variable between councils that report an operating 10 Note that I also test alternative specifications, including a log-
transformed OWQI dependent variable. The results do not
budget greater than $200,000 and those that report
substantively change, however, and thus for clarity I present the
an operating budget (not include one-time expendi- untransformed specification.
tures and or project-specific grant funds) smaller than 11 Table 2 also presents the deviance information criterion (DIC) and
$200,000. While the specific operating budget figure Watanabe-Akaike information criterion (WAIC) as for each model. DIC is
for each council likely varies to some extent on a a variation of the traditional Akaike information criterion score adapted
to better suite hierarchical Bayesian models (Spiegelhalter et al. 2002;
yearly basis, the categories provided by NOWC are
Ward 2008), while WAIC is viewed as an improved metric relative to DIC
sufficiently large so as to smooth out much of this better suited for hierarchical Bayesian models (Vehtari and Gelman
variation. 2014). Lower DIC and WAIC scores indicate a better-fitting model.
12 I use the term “random effect” in the more modern sense where the
model takes advantage of the mathematical form of a random effect
7 Wetlands are not included in the model specifications below, as wetlands but does not assume that levels of a random effect are draws from
were not found to be a significant model predictor for any fitted model. a larger population (Hodges 2014). Hodges (2014) describes random
8 http://www.oregon.gov/oweb/monitor/pages/owri.aspx. effects in this way as formal smoothing devices; for instance, in
9 Obviously, staffing levels and budgets of course do change over time. the models in this paper, the random effect modeled for each HUC8
However, just as with land use covariates, it is reasonable to assume watershed smoothes each HUC8 intercept estimate between the
that these data still reflect relative differences between observations completely pooled (i.e., no HUC8 term) and fully nonpooled (i.e., a fixed
even if the true values do not remain static, particularly given that HUC8 effect) alternatives based upon the sample size and variance
I extrapolate only from 2012 to 2014. observed within each HUC8 group (see Gelman et al. 2013).
Table 2. Covariates and Grant Funds in 1, 2, and 3 Water-Years Prior to Observation
Past 1-Year Funds Past 2-Year Funds Past 3-Year Funds
Control variables
Latitude 2.69a (1.13; 4.58) 2.47a (0.99; 4.18) 2.10a (0.70; 4.24)
Longitude −1.88a (−3.26; −0.66) −1.83a (−3.13; −0.65) −1.82a (−3.27; −0.65)
% Agricultural land in HUC8 −0.09 (−0.22; 0.03) −0.08 (−0.20; 0.04) −0.04 (−0.17; 0.10)
% Forested land in HUC8 0.10a (0.004; 0.20) 0.08 (−0.01; 0.18) 0.07 (−0.02; 0.17)
% Developed land in HUC8 −0.35a (−0.64; −0.09) −0.31a (−0.60; −0.04) −0.54a (−1.10; −0.03)
Elevation (10 m) 0.69a (0.16; 1.30) 0.61a (0.11; 1.18) 0.52a (0.04; 1.16)
Monthly median precipitation (inches) −1.75a (−2.68; −0.84) −1.73a (−2.65; −0.83) −1.75a (−2.65; −0.87)
Historical OWQI 0.80a (0.64; 0.92) 0.81a (0.67; 0.93) 0.84a (0.63; 0.95)
OWRI restoration ($100k) −0.05 (−0.21; 0.11) −0.04 (−0.21; 0.13) 0.28 (−0.27; 0.85)
Years active −0.62a (−0.88; −0.36) −0.58a (−0.85; −0.32) −0.55a (−0.83; −0.28)
Operating budget > $200k 1.39 (−1.98; 4.81) 0.75 (−2.94; 4.43) 0.73 (−3.18; 4.54)

Staff FTE −0.51a (−1.03; −0.004) −0.12 (−0.67; 0.40) 0.19 (−0.37; 0.75)
Hypothesis 1
WC grants ($100k) 0.93a (0.22; 1.64) 0.62a (0.23; 1.01) 0.52a (0.24; 0.80)
Hypothesis 2
Years active * WC grants −0.04a (−0.08; −0.001) −0.02 (−0.04; 0.01) −0.01 (−0.03; 0.01)
Operating budget > $200k * WC grants −0.14 (−0.58; 0.30) −0.13 (−0.42; 0.15) −0.15 (−0.37; 0.06)
Staff FTE * WC grants −0.086a (−0.16; −0.01) −0.09a (−0.14; −0.04) −0.09a (−0.13; −0.05)
DIC 17119.26 17113.70 17104.58
WAIC 17135.05 17128.26 17121.92
Note. DIC = deviance information criterion; WAIC = Watanabe-Akaike information criterion.

a
Zero outside the 95% credible interval.
quality should increase as one moves to the north and absolute location; thus, unlike agriculture and forest
to the west. In table 2, moving one meridian to the east cover, the development variable is strong in magnitude.
predicts a −1.8 unit decrease in OWQI score, and mov- As should be the case historical water quality
ing one parallel to the north increases predicted OWQI (Historical OWQI) at a sample site is strongly related
score by more than 2 units. Similarly, increased elevation to predicted water quality. For each 1-unit increase in
predicts improved OWQI scores (as expected since this the historical average, the predicted OWQI at a given
reflects a move upstream above potential pollutants), and site increases by about 0.8 units. The final control vari-
an increase in median precipitation level in a watershed able, OWRI Restoration Spending, reflects the mon-
for the month of observation predicts a decrease in water etary value (in $100k units) of non-OWEB restoration
quality (as rainfall can increase runoff and erosion). projects reported to the OWRI database for a given
The signs of the predicted relationships between watershed (restoration projects that do not relate to
water quality and agricultural land use (% Ag. in water quality, such as salmon recovery efforts without
HUC8) and forest cover (% For. in HUC8) in the a water quality improvement component, are omitted
watershed are in the expected direction (negative for from this calculation).13
agriculture and positive for forest cover). However, the
magnitude of the predicted effects are not strong (less 13 While there is not a consistent, strong relationship between projects
than a 0.1-unit change in OWQI score for a 1% change recording in the OWRI database and water quality, this is most likely
in either cover type); this is because, as just described, due to the user-generated nature of the OWRI database. Of the 2,196
forested land and agriculture have a strong east-west water quality observations included in the model, 2,007 observations
occur in a watershed and time period that has a value of 0 for the OWRI
and north-south distribution, and so are in large part
Restoration Spending variable. In conjunction with the fact that the
captured by absolute spatial location. Development (% majority of projects that are logged in the database are voluntary actions
Dev. in HUC8) is expected to decrease water quality taken by private citizens and landowners from a select few watersheds,
(due to urban runoff and other factor); as anticipated, this likely evidences a lack of participation in the user-generated
each additional percentage of developed land cover in OWRI database by actors in other subregions rather than a meaningful
empirical distribution of restoration actions, since it is unlikely that no
a watershed decreases predicted OWQI by 0.31–0.54
private restoration actions were undertaken in so many other locales.
units (across the three different models). While Oregon’s If private restoration actions in some watersheds are not recorded, this
primary population center is located in the northwest of downwardly biases the estimate for this coefficient since restoration
the state, development in general tracks less well with actions are being taken but are effectively valued at $0 in the model.
Testing H1 and H2: Total Grant Funds impact of funding given to a council formed in the
Turning to the coefficients of substantive interest, same year (yearsActive = 0), with an operating budget
I fit linear terms for OWEB grant funds spent in the less than $200k per year (operatingBudget = 0), and no
previous 1 (column 1), 2 (column 2), and 3 (column full time staff members (staffFTE = 0).
3) water-years. Table 2 shows that there does appear For each time window (corresponding to each of
to be a positive relationship between grant funds in the three columns in table 2), figure 1 clearly shows
the prior time period and water quality, but that this how council resource variables collectively dimin-
relationship generally diminishes as a function of WC ish the predicted impact of funds. Thus, each model
staffing, operating budget, and the number of years a supports H2, finding that the predicted association
council has been active. Because grant funds are inter- between project funds given to a WC and water qual-
acted with years, the council has been active, whether ity diminishes as a function of council resource attrib-
the council’s operating budget is greater than $200k utes. It is also interesting to consider how the posterior
per year, and a council’s staff FTE level, the linear parameter estimates change across the three different
effect of OWEB funding cannot be interpreted in iso- model specifications shown in table 2 and figure 1.

lation. Rather, the predicted impact of grant funds is The coefficients are consistent in sign and magnitude,
dependent on the value of these other variables, and but the predicted impact of funded projects appears to
thus the marginal predicted impact of total OWEB attenuate as the time window increases. This is likely
grant funds is: because the 2 water-year and 3 water-year time win-
dows incorporate actions increasingly distant in time
∂OWQI to water quality observations, leaving more room for
= β1 + β2 yearsActive + β3operatingBudget other intervening factors.
∂ Funds (5)
+ β3 staffFTE
Testing H3: Different Project Type Funding
The easiest way to understand the cumulative effect of H3 predicts that capacity building projects will have a
these interactions is to sample from the posteriors of positive impact on the predicted association between
each parameter and evaluate the density of the sam- other council projects and water quality. To test this,
pled linear combinations (Martins et al. 2013). For instead of modeling all funds to each WC collectively, I fit
each model in table 2, 100,000 random draws are three models (one each for the 1-, 2-, and 3-water-year
taken from the posterior distributions estimated for all funding windows) that distinguish between technical
parameters in the equation above, each time summariz- projects (namely, monitoring and assessment), restora-
ing the linear combination of marginal effects as shown tion projects (e.g., riparian planting or fencing livestock
in equation (1). Figure 1 plots the resultant distribu- away from a stream), outreach projects (e.g., environ-
tion of sampled linear combinations to show how the mental education, information services, and network-
predicted impact of funding is expected to vary given ing with stakeholders), and capacity building projects
the interaction terms. For comparison, I also include a (namely to fund council coordinators and other staff).
distribution of samples from the posterior of the linear Each funding type is also interacted with key institu-
funding coefficient ( β1 in equation (1)), which is equiv- tional characteristics (years active, operating budget, and
alent to a sampled linear combinations of the predicted staff FTE), as well as a full interaction term to account
Figure 1. Sampled linear combinations from posterior marginals, funding to WCs.


Figure 2. Sampled posterior marginals for outreach with and without capacity building funds.
Figure 3. Sampled posterior marginals for tech. (technical) with and without capacity building funds.
for tradeoffs among the four funding types. Figures 2–4 that incorporate outreach multiplied by capacity.
compare linear combinations of posterior samples for Figure 2 shows that in general, the presence of capac-
each project type (accounting for all relevant interac- ity building funds does increase the predicted impact
tions as in figure 1) with and without capacity building of funds received for outreach projects (shown by each
funds.14 The x-axis in each figure represents the predicted with capacity funding posterior density plot shifting
change in the dependent variable given a 1-unit change slightly to the right relative to its comparison posterior
in a given linear combination. density).
First, figure 2 shows the predicted impact of outreach Figure 3, however, shows that the presence of capac-
projects, with and without the presence of capacity ity building funds has very little impact whatsoever on
building funds. The darker lines show the distribu- the predicted impact of technical projects on water
tion of predicted impacts for outreach project funds quality. The posterior distributions with (lighter lines)
received in the past 1, 2, or 3 water-years, respectively, and without (darker lines) capacity building funds are
without factoring in the interaction between outreach very similar for each funding time window. Moreover,
project funds and capacity building funds. The lighter figure 3 shows that regardless of capacity building
lines then show the corresponding posterior marginals funds, technical projects such as monitoring, assess-
ment, or mapping minimally associated with water
14 In Appendix B, Table B1 shows model coefficients. The project-specific quality improvements in any case.
funding models in B1 compare quite favorably in terms of DIC and WAIC
scores to the models presented in table 2. This suggests that the type
While the predicted association between funding
of project does matter, and thus there is value in modeling each type for technical projects and water quality improvements
separately. in a watershed is very small in magnitude and positive

Figure 4. Sampled posterior marginals for restoration with and without capacity building funds.
for the 2- and 3-year time windows, the mean of the qualitative improvement from this simulated shift is
joint posterior is actually negative at the 1-year inter- that of “Poor” observations becoming “Fair,” “Fair”
val. This is likely attributable to the 1-year interval observations becoming “Good,” and “Good” observa-
being subject to greater uncertainty than the 2- and tions becoming “Excellent.”
3-year estimates (as shown by the thick-tailed joint
posterior density for the 1-year estimates in compari-
son to the much narrower 2- and 3-year estimates) Discussion
and not an indicator that technical projects are ini- While it is reasonable to expect that money given
tially harmful. for environmental projects should result in environ-
Finally, figure 4 compares the posterior marginals mental changes, even this basic input–outcome rela-
for restoration funding with and without factoring tionship remains largely unestablished in the case of
in the interaction term for capacity building funds. collaborative governance. This analysis finds that in
As with funding for technical projects, the additional general, public funds given to WCs are associated
interaction term does not appear to make a consider- with subsequent improvements in water quality, indi-
able difference. As with technical projects, at the one cating that in this case collaboration is not “all talk
prior water-year funding interval, the predicted impacts and no action” (Lubell 2004). As evidenced by the
(both with and without factoring in capacity building) considerable funding governments provide to col-
are slightly negative; at the 2- and 3-year intervals, the laborative management bodies (e.g., Sabatier 2005)
predicted impact becomes smaller in magnitude but is many practitioners already assume the viability of
positive and exhibits much less variation (shown by this type of governance approach. However, this
the much narrower joint posterior distribution). work provides some of the first systematic evidence
To understand the relevance of the joint posteriors, quantifying such returns and advances beyond the
it is helpful to consider the observed distribution of limiting “collaborative’’ versus “non-collaborative’’
OWQI scores in terms of the qualitative categories in dichotomy by comparing different collaborative
which the state of Oregon groups water quality obser- actions.
vations. Surface waters are considered to be in “Very One ongoing challenge in this arena is the difficulty
Poor” condition given OWQI scores between 0 and of measuring outcomes in ways that are commen-
59, “Poor” condition given scores between 60 and 79, surate with management data. To best evaluate the
“Fair” condition if between 80 and 84, “Good” condi- impact of WCs, one would need to follow up directly
tion if between 85 and 89, and “Excellent” if at 90 with an evaluation protocol that explicitly monitors
or above. Figure 5 plots the distribution of observed their outputs and subsequent outcomes. However,
OWQI scores included in this analysis, showing that such data collection efforts would obviously be very
most observations are considered to be of “Fair” qual- costly and time-intensive; OWEB is not able to track
ity or above. For comparison, figure 5 then plots a cor- impacts in such detail. By accounting for dependence
responding density with each observed value increased across space and time (in conjunction with key covari-
by 4 units. In this simulation, one can see that the ates such as land cover and precipitation), the spati-
small cluster of “Very Poor” observations shifts right otemporal modeling strategy employed herein enables
but remains substantively unchanged. Much of the the use of environmental monitoring data to examine

Figure 5. Distribution of observed OWQI scores and simulated +4-unit increase.
the link between funding for collaborative groups and within the same category, the full extent to which
subsequent policy outcomes. ODEQ positions moni- inter-project heterogeneity inhibits our ability to
toring sites independently of project sites at locations compare and contrast these different efforts is
that are optimal for capturing the full extent of key not clear.
water quality drivers in a watershed, and the OWQI is Nonetheless, this analysis contributes to the the-
billed explicitly as a tool for evaluating water quality ory and practice of collaborative governance by con-
management effectiveness (Cude and Curtis 2001). In tributing to the limited evidence base (Emerson and
this regard, one can be confident that these monitor- Nabatchi 2015b; Thomas and Koontz 2011) regard-
ing data providing a meaningful gauge of watershed ing the impacts of collaborative efforts. Perhaps more
conditions. importantly, these results provide a basis for further
Nonetheless, it is important to bear in mind poten- evaluation and comparison of different collaborative
tial shortcomings of measuring policy and program governance arrangements. Moreover, this research
impacts using ambient monitoring data. With respect demonstrates the use of hierarchical Bayesian spati-
to measuring the impact of specific management otemporal modeling as a means by which publicly
actions, the fact that different types of projects are available observational data produced by states and
likely to be spatially distributed in different ways the Federal government can be repurposed to test the-
might affect how subsequent impacts are observ- ory and evaluate public programs. One of the major
able through water quality monitoring. For instance, challenges to examining the performance of collabo-
a restoration project takes place at a specific loca- rative governance is the need to measure impacts at
tion and has a concentrated physical impact at the multiple levels and times (Emerson and Nabatchi
restoration site and (assumedly) downstream of this 2015b); this modeling framework provides a means to
location. An outreach project might more widely do so by using data that are systematically collected
broadcast its impact (i.e., one project could impact and comparable a between locations and times. Future
an entire watershed) but also be less concentrated research regarding the functions and actions of collab-
(e.g., since one landowner might change their ferti- orative groups will help build upon the current results
lizer use while a neighbor does not). by speaking in finer detail to where and when impacts
Similarly, the temporal distribution of project should be anticipated.
impacts is also potentially idiosyncratic. This analy-
sis finds that on a 3-year time window, restoration
and outreach projects have a positive association Conclusion
with water quality, but that technical projects do By studying how inputs to collaborative govern-
not. However, it is possible that technical projects— ance relate not just to management outputs but also
for instance data collection and assessment—have a to key policy outcomes, public management scholars
longer lag-time in that they serve to produce data can build an evidence base that speaks to the effi-
that ultimately informs management decisions sev- ciency and effectiveness of collaborative governance
eral years in the future. Particularly given that the for solving public problems. While the results of this
spatial and temporal distribution of project impacts analysis directly concern the link between public sup-
likely differs even among projects that are grouped port for collaborative governance and environmental
outcomes, they also hold implications for the role of potential reason for this is that the impacts of capac-
collaborative governance more generally within public ity building funds are borne out on a longer time
policy and management. horizon in some cases. To address broader public
First, watershed management councils in Oregon— management questions regarding how capacity build-
like many collaborative institutions—are typically set ing improves problem solving in network governance
up as nonprofit organizations. The question of how contexts (Graddy and Chen 2006; Provan and Kenis
nonprofits receive funding to deliver public goods 2008), future work can build upon the current find-
and services has important implications for the effi- ings by factoring in long term investments in staff and
ciency and equity of using collaborative governance administrative resources.
arrangements (Garrow 2014; Marwell and Calabrese
2015; Mosley 2012). After controlling for institu-
Appendix A
tional characteristics that are expected to help coun-
cils compete for and win grants, the predicted impact Integrated Nested Laplace Approximation and the
of funding on water quality becomes stronger. This Stochastic Partial Differential Equations Approach

indicates that councils with additional resources Integrated nested Laplace approximation (INLA)
are better equipped to secure funding for marginal (www.r-inla.org) facilitates Bayesian inference by
projects that might be of lesser impact and supports assuming that model parameters collectively constitute
evidence from other policy sectors that profession- a latent field, θ = {α h , βmτ i , f (.)} (Rue, Martino, and
alization is consequential for gaining public support Chopin 2009). This latent field is defined by a Gaussian
(Suárez 2011). Given that the rationale for collabora- multivariate distribution of mean 0 and precision
matrix Q(Ψ) such that θ ~ N(0, Q (ψ )) (Blangiardo
−1
tive governance is typically based at least in part on
normative goals of participation and inclusion (Fung et al. 2013; Rue and Held 2005; Rue, Martino, and
2015; Nabatchi and Leighninger 2015), future work Chopin 2009). This assumed distribution is known as
should assess the extent to which local collaborative a Gaussian Markov Random Field (GMRF). The core
groups are nominally collaborative—and perhaps idea behind GMRFs is that a vector of model param-
more accurately reflect a small group of powerful, eters has a Gaussian density function for which the
professional actors who leverage network resources precision matrix (the inverse of the covariance matrix)
to pursue their aims. expresses conditional independence relations among
Second, the choice to provide public funds for the elements of the vector (Hodges 2014). In other
nonprofit collaborative organizations to deliver words, two parameters are assumed to be independent
goods and services stands in contrast to other poten- conditional on the rest of the parameter vector when
tial recipients such as local governments and private the precision matrix entry for those two parameters
firms. Thus, OWEB’s support for local collaborative equals zero:
management efforts relates to the vibrant and ongo-
ing discussion of public sector contracting (Bertelli
( )
cov θi , θ j | θ(− ij ) = 0

(A1)
and Smith 2010; Brown, Potoski, and Van Slyke Hodges (2014) provides an intuitive example of how
2016; Hefetz and Warner 2012; Heinrich, Lynn, and this works in the context of a simple one-way ran-
Milward 2010). By examining the impacts associated dom effects model: For the model yij = α j[i ] + ε ij (with
with funds given to local WCs, this analysis contrib- random effect α j = α 0 + ρj ), we assume: (1) that each
utes to a broader understanding of how and when observation yij is independent of other grouping fac-
state or Federal agencies can partner with different tors ( α − j ) conditional on the grouping factor (i.e.,
local organizations in delivering public goods and the random effect) in which observation i occurs
services. Other policy sectors invite similar examina- ( )
( α i ) (that is, cov yij , α − j |α j = 0 ); (2) that the whole
tions of agency support for local collaborative efforts sample mean ( α 0 ) is independent of yij conditional
such as public–private partnerships or collaborative
planning groups.
( )
on the group effect α j (i.e., cov yij , α 0 |α j = 0 ); and
(3) that each random effect is independent of all other
Third, institutional capacity is viewed as key to random effects conditional on the whole sample mean
the success of collaborative governance efforts in any
management sector (Emerson and Nabatchi 2015a).
( )
(i.e., cov α j , α − j | α 0 = 0 ).
Returning to the model at hand, an individual
The results of this analysis indicate that capacity observation y(si ,t ) (at location s and time t ) is mod-
building funds can enhance stakeholder outreach eled by a subset of θ according to its spatiotemporal
efforts. However, the predicted change is relatively characteristics:
small, and there is no corresponding increase in the
predicted impact of technical or restoration projects ( )
y si ,t | θ , ψ ~ p(y(si ,t ) | ∑Aijθ j , ψ ) (A2)
associated with the receipt of capacity funds. One j
In equation (A2), the observation matrix Aij is where and Chopin 2009). This means that posterior estimates
spatial location enters in. To account for spatial are almost wholly generated in light of the data. The
dependence amongst water quality observations, this spatial mesh used for the SPDE approach is shown in
model includes a random spatial effect that also takes figure A1.
the form of a Gaussian random field (GRF) (Simpson, Water quality sampling stations can be used, but
Lindgren, and Rue 2012). To model a continuous spa- do not need to be, as triangle vertices in the mesh
tial process, the stochastic partial differential equa- (Lindgren, Rue, and Lindström 2011). The mesh is
tion (SPDE) method developed by Lindgren, Rue, and more finely grained in areas where there are more
Lindström (2011) indexes the continuous GRF as a water quality sampling stations; larger triangles rep-
GMRF just as INLA models the parameter distribu- resent areas with less information (Cosandey-Godin
tion as a GMRF above (Lindgren and Rue 2015). The et al. 2015). This serves allows the model to estimate
SPDE model divides a continuous spatial domain (the with increased accuracy where there are sufficient
state of Oregon in this case) into a mesh of discrete data, and conversely does not attempt to model with
triangles (Blangiardo et al. 2013). This triangular great detail where there are limited data. This model

grid approximates the continuous field; instead of feature is important since there are far fewer streams—
assuming that every point is correlated with every and thus fewer sampling stations—in the eastern por-
other point, it is now assumed that two points are tion of the state. Conversely, western Oregon has a
independent of one another conditional on the ver- higher density of streams and thus more sample sites.
The mesh is extended beyond the boundaries of the
tices of the triangle in which each point lies (just as
sample area to avoid boundary effects wherein there
above, INLA uses the GMRF approach to assume
( )
that cov θi , θ j |θ(− ij ) = 0 ).
is increased variance at the borders of a spatial field
(as these locations have fewer surrounding values)
The observation matrix Aij contains the values of
(Lindgren and Rue 2015). Spatial correlation is held
the random field at the specific times and locations con-
constant over time, since while conditions at different
tained in the dataset and uses these values for param-
sites might vary, the role that spatial distance plays
eter estimation (Lindgren, Rue, and Lindström 2011).
in these conditions likely remains relatively constant.
Values are only stored for specific points (points within
Finally, including a spatial random effect in the
a triangle can be estimated by extrapolating between model introduces confounding between the spatial
values at neighboring vertices). The latent field is effect and observed covariates that are themselves
linked to model likelihood via A , such that η* = Aη spatially distributed (e.g., elevation) (Hodges 2014),
(Cosandey-Godin et al. 2015): which can bias or inflate the variance of regression
coefficients (Hughes and Haran 2013). Though this is
(( ) )
n
p y si ,t |θ , ψ = ∏p(y(si ,t ) | η* , ψ ) (A3) not an issue for prediction (Hughes and Haran 2013),

i =1 when model parameters are of interest it is necessary
to restrict the spatial effect to the orthogonal comple-
All models are estimated using R and the R-INLA pack- ment of the model covariates (Hodges 2014; Hodges
age (www.r-inla.org). Since there are no prior data in and Reich 2010). Thus, I impose a linear orthogonality
this case, parameter prior estimates noninformative constraint on the spatial field so as to separate the lin-
priors (Gelman et al. 2013) specified vaguely using the ear effects from the spatial random effect (Ingebrigtsen,
default recommended INLA settings (Rue, Martino, Lindgren, and Steinsland 2014). This means that the
Figure A1. Spatial mesh used for SPDE overlaid on monitoring stations.
Table B1. Predicted Impact of Specific Project Types (in $100k Given to WC)
Past 1-Year Funds Past 2-Year Funds Past 3-Year Funds
$ for technical −1.92 (−8.85; 5.00) −0.22 (−4.89; 4.44) 0.91 (−3.61; 5.43)
$ for restoration 1.726a (0.45; 3.00) 0.84a (0.04; 1.64) 0.86a (0.16; 1.57)
$ for outreach 6.181 (−7.26; 19.60) 17.24a (2.28; 32.12) 7.26 (−7.02; 21.56)
$ for capacity −0.36 (−2.39; 1.67) 0.23 (−1.77; 2.23) −0.39 (−2.00; 1.23)
Total $ to WC −2.90a (−5.00; −0.81) −0.14 (−0.29; 0.01) −0.07a (−0.11; −0.02)
Years active * $ for technical 0.15 (−0.33; 0.64) −0.05 (−0.30; 0.20) 0.03 (−0.19; 0.25)
Budget > $200k * $ for technical 0.29 (−3.51; 4.08) 0.19 (−2.83; 3.21) −0.39 (−3.30; 2.52)
Staff FTE * $ for technical −0.31 (−1.01; 0.39) 0.15 (−0.29; 0.60) −0.21 (−0.67; 0.25)
Years active * $ for restoration −0.08a (−0.15; −0.01) −0.04 (−0.08; 0.01) −0.04 (−0.08; 0.001)
Budget > $200k * $ for restoration −0.35 (−0.99; 0.30) 0.15 (−0.34; 0.64) 0.16 (−0.33; 0.65)
Staff FTE * $ for restoration −0.09a (−0.17; −0.02) −0.05 (−0.12; 0.02) −0.06 (−0.12; 0.001)
Years active * $ for outreach −0.18 (−1.48; 1.13) −0.94 (−2.03; 0.15) −0.11 (−1.07; 0.84)

Budget > $200k * $ for outreach 5.36 (−8.08; 18.78) 5.29 (−5.54; 16.09) 4.58 (−5.33; 14.42)
Staff FTE * $ for outreach −1.92 (−5.02; 1.18) −1.30 (−3.60; 1.01) −1.97 (−4.19; 0.26)
Years active * $ for capacity 0.02 (−0.10; 0.14) −0.002 (−0.11; 0.11) −0.01 (−0.11; 0.09)
Budget > $200k * $ for capacity 0.77 (−0.52; 2.05) −0.16 (−1.33; 1.00) −0.46 (−1.62; 0.69)
Staff FTE * $ for capacity −0.01 (−0.26; 0.23) −0.11 (−0.34; 0.13) 0.001 (−0.19; 0.19)
$ for technical * $ for capacity 1.42 (−1.00; 3.83) 1.13 (−0.06; 2.32) 0.004 (−0.61; 0.62)
$ for restoration * $ for capacity −0.03 (−0.18; 0.13) −0.06 (−0.14; 0.02) 0.06 (−0.04; 0.15)
$ for outreach * $ for capacity 6.07 (−4.84; 16.96) 1.60 (−2.676; 5.88) 1.40 (−0.96; 3.77)
DIC 16932.532 16925.037 16897.220
WAIC 16951.351 16932.917 16916.034
Note. DIC = deviance information criterion; WAIC = Watanabe-Akaike information criterion.

a
Zero outside the 95% credible interval.
restricted spatial field represents the spatial effect after Blangiardo, Marta, Michela Cameletti, Gianluca Baio, and Håvard Rue. 2013.
Spatial and spatio-temporal models with R-INLA. Spatial and Spatio-
all covariates have been accounted for.
Temporal Epidemiology 7:39–55.
Brooks, Steve, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011.
Handbook of Markov chain Monte Carlo. Boca Raton, FL: CRC Press.
Appendix B Brown, Trevor L., Matthew Potoski, and David Van Slyke. 2016.
Managing complex contracts: A theoretical approach. Journal of Public
Posterior Credible Intervals for Project-Specific
Administration Research and Theory 26:294–308.
Funding Models Bryson, John M., Barbara C. Crosby, and Melissa Middleton Stone. 2006. The
Table B1 omits covariates (shown as control variables design and implementation of cross-sector collaborations: Propositions
in table 2) that do not relate to Oregon Watershed from the literature. Public Administration Review 66(Suppl 1):44–55.
Enhancement Board funds given to WCs. Cameletti, Michela, Finn Lindgren, Daniel Simpson, and Håvard Rue. 2013.
Spatio-temporal modeling of particulate matter concentration through the
SPDE approach. Advances in Statistical Analysis 97:109–31.
Clark, James S., and Alan E. Gelfand. 2006. Hierarchical modelling for the
References environmental sciences: Statistical methods and applications: Statistical
Ansell, Chris, and Alison Gash. 2008. Collaborative governance in theory methods and applications. New York, NY: Oxford University Press.
and practice. Journal of Public Administration Research and Theory Cosandey-Godin, Aurelie, Elias Teixeira Krainski, Boris Worm, and Joanna
18:543–71. Mills Flemming. 2015. Applying Bayesian spatio-temporal models to fish-
Ansell, Chris, and Jacob Torfing. 2015. How does collaborative governance eries bycatch in the Canadian Arctic. Canadian Journal of Fisheries and
scale? Policy & Politics 43:315–29. Aquatic Sciences 72:186–97.
Berardo, Ramiro, Tanya Heikkila, and Andrea K. Gerlak. 2014. Cude, Curtis G. 2001. Oregon water quality index: A tool for evaluating
Interorganizational engagement in collaborative environmental manage- water quality management effectiveness. Journal of the American Water
ment: Evidence from the South Florida Ecosystem Restoration Task Force. Resources Association 37:125–37.
Journal of Public Administration Research and Theory 24:697–719. Dunson, David B. 2001. Commentary: Practical advantages of Bayesian
Bernardo, José M., and Adrian F. M. Smith. 2009. Bayesian theory, vol. 405. analysis of epidemiologic data. American Journal of Epidemiology
Hoboken, NJ: John Wiley & Sons. 153:1222–26.
Bertelli, Anthony M., and Craig R. Smith. 2010. Relational contracting and Emerson, Kirk, and Tina Nabatchi. 2015a. Collaborative governance regimes.
network management. Journal of Public Administration Research and Washington, DC: Georgetown University Press.
Theory 20(Suppl 1):i21–40. ———. 2015b. Evaluating the productivity of collaborative governance
Bivand, Roger S., Edzer J. Pebesma, and Virgilio Gomez-Rubio. 2008. Applied regimes: A performance matrix. Public Performance & Management
spatial data analysis with R, 2nd ed. New York, NY: Springer. Review 38:717–47.
Fung, Archon. 2015. Putting the public back into governance: The challenges Lindgren, Finn, Håvard Rue, and Johan Lindström. 2011. An explicit
of citizen participation and its future. Public Administration Review link between Gaussian fields and Gaussian Markov random fields:
75:513–22. The stochastic partial differential equation approach. Journal of
Garrow, Eve E. 2014. Does race matter in government funding of nonprofit the Royal Statistical Society, Series B: Statistical Methodology
human service organizations? The interaction of neighborhood pov- 73:423–98.
erty and race. Journal of Public Administration Research and Theory Lubell, Mark. 2004. Collaborative environmental institutions: All talk and no
24:381–405. action? Journal of Policy Analysis and Management 23:549–73.
Gelman, Andrew, Jessica Hwang, and Aki Vehtari. 2014. Understanding pre- Lubell, Mark, and Allan Fulton. 2008. Local policy networks and agricultural
dictive information criteria for Bayesian models. Statistics and Computing watershed management. Journal of Public Administration Research and
24:997–1016. Theory 18:673–96.
Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, Margerum, Richard D. 2011. Beyond consensus: Improving collaborative
and Donald B. Rubin. 2013. Bayesian data analysis. Boca Raton, FL: CRC planning and management. Cambridge, MA: MIT Press.
Press. Martino, Sara, Rupali Akerkar, and Håvard Rue. 2011. Approximate Bayesian
Gerlak, Andrea K., and Tanya Heikkila. 2011. Building a theory of learning inference for survival models. Scandinavian Journal of Statistics, Theory
in collaboratives: Evidence from the Everglades Restoration Program. and Applications 38:514–28.
Journal of Public Administration Research and Theory 21:619–44. Martins, Thiago G., Daniel Simpson, Finn Lindgren, and Håvard Rue. 2013.
Gomez-Rubio, Virgilio, Roger S. Bivand, and Håvard Rue. 2014. Spatial mod- Bayesian computing with INLA: New features. Computational Statistics

els using Laplace approximation methods. In Handbook of regional sci- and Data Analysis 67:68–83.
ence, eds. Manfred M. Fischer and Peter Nijkamp, 1401–17. New York, Marwell, Nicole P., and Thad Calabrese. 2015. A deficit model of collabora-
NY: Springer.. tive governance: Government–nonprofit fiscal relations in the provision
Graddy, Elizabeth A., and Bin Chen. 2006. Influences on the size and scope of child welfare services. Journal of Public Administration Research and
of networks for social service delivery. Journal of Public Administration Theory 25:1031–58.
Research and Theory 16:533–52. Meador, Michael R., and Robert M. Goldstein. 2003. Assessing water quality
Hardy, Scott, and Tomas M. Koontz. 2008. Reducing nonpoint source pollu- at large geographic scales: Relations among land use, water physicochem-
tion through collaboration: Policies and programs across the U.S. States. istry, riparian condition, and fish community structure. Environmental
Environmental Management 41:301–10. Management 31:504–17.
Hefetz, Amir, and Mildred E. Warner. 2012. Contracting or public deliv- Mosley, Jennifer E. 2012. Keeping the lights on: How government fund-
ery? The importance of service, market and management characteristics. ing concerns drive the advocacy agendas of nonprofit homeless ser-
Journal of Public Administration Research and Theory 22:289–317. vice providers. Journal of Public Administration Research and Theory
Heinrich, Carolyn J., Laurence E. Lynn, and H. Brinton Milward. 2010. A state 22:841–66.
of agents? Sharpening the debate and evidence over the extent and impact Nabatchi, Tina, and Matt Leighninger. 2015. Public participation for 21st cen-
of the transformation of governance. Journal of Public Administration tury democracy. San Francisco, CA: Jossey Bass.
Research and Theory 20(Suppl 1):i3–19. Newig, Jens, and Oliver Fritsch. 2009a. Environmental governance:
Hijmans, Robert J., and Jacob van Etten. 2014. raster: Geographic analysis Participatory, multilevel, and effective? Environmental Policy and
and modeling with raster data. R package version 2.310. http://CRAN. Governance 19:197–214.
Rproject.org/package=raster. ———. 2009b. More input-better output: Does citizen involvement improve
Hodges, James S. 2014. Richly parameterized linear models additive, time environmental governance. In In search of legitimacy. Policy making in
series, and spatial models using random effects. Boca Raton, FL: CRC Europe and the challenge of complexity, ed. Ingolfur Bluhdorn, 205–24.
Press. Opladen, Germany: Farmington Hills.
Hodges, James S., and Brian J. Reich. 2010. Adding spatially-correlated Nikolic, Sara J. S., and Tomas M. Koontz. 2008. Nonprofit organizations
errors can mess up the fixed effect you love. The American Statistician in environmental management: A comparative analysis of govern-
64:325–34. ment impacts. Journal of Public Administration Research and Theory
Hughes, John, and Murali Haran. 2013. Dimension reduction and alleviation 18:441–63.
of confounding for spatial generalized linear mixed models. Journal of O’Leary, Rosemary, Catherine Gerard, and Lisa B. Bingham. 2006.
the Royal Statistical Society, Series B: Statistical Methodology 75:139–59. Introduction to the symposium on collaborative public management.
Ingebrigtsen, Rikke, Finn Lindgren, and Ingelin Steinsland. 2014. Spatial Public Administration Review 66(Suppl 1):6–9.
models with explanatory variables in the dependence structure. Spatial Provan, Keith G., and Patrick Kenis. 2008. Modes of network governance:
Statistics 8:20–38. Structure, management, and effectiveness. Journal of Public Administration
Johnes, P. J., and A. L. Heathwaite. 1997. Modelling the impact of land Research and Theory 18:229–52.
use change on water quality in agricultural catchments. Hydrological Rue, Håvard, and Leonhard Held. 2005. Gaussian Markov random fields:
Processes 11:269–86. Theory and applications. Boca Raton, FL: CRC Press.
Koontz, Tomas M., and Craig Thomas. 2006. What do we know and need to Rue, Håvard, Sara Martino, and Nicolas Chopin. 2009. Approximate
know about the environmental outcomes of collaborative management? Bayesian inference for latent Gaussian models by using integrated nested
Public Administration Review 66(Suppl 1):111–21. Laplace approximations. Journal of the Royal Statistical Society, Series B:
Lasker, Roz D., Elisa S. Weiss, and Rebecca Miller. 2001. Partnership synergy: Statistical Methodology 71:319–92.
A practical framework for studying and strengthening the collaborative Sabatier, Paul A. 2005. Swimming upstream: Collaborative approaches to
advantage. The Milbank Quarterly 79:179–205, III–IV. watershed management. Cambridge, MA: MIT Press.
Layzer, Judith A. 2008. Natural experiments: Ecosystem-based management Salamon, L. M. 2002. The tools of government: A guide to the new govern-
and the environment. Cambridge, MA: MIT Press. ance. New York, NY: Oxford University Press.
Leach, William D., Christopher M. Weible, Scott R. Vince, Saba N. Siddiki, Schneider, Mark., John Scholz, Mark Lubell, Denisa Mindruta, and Matthew
and John C. Calanni. 2013. Fostering learning through collabora- Edwardsen. 2003. Building consensual institutions: Networks and
tion: Knowledge acquisition and belief change in marine aquaculture the National Estuary Program. American Journal of Political Science
partnerships. Journal of Public Administration Research and Theory 47:143–58.
24:591–622. Scott, Tyler A. 2015. Does collaboration make any difference? Linking collab-
Lindgren, Finn, and Havard Rue. 2015. Bayesian spatial and spatiotemporal orative governance to environmental outcomes. Journal of Policy Analysis
modelling with R-INLA. Journal of Statistical Software 63:1–25. and Management 34:537–66.
Scott, Tyler A., and Craig W. Thomas. 2016. Unpacking the collaborative tool- Ulibarri, Nicola. 2015. Tracing process to performance of collaborative gov-
box: Why and when do public managers choose collaborative governance ernance: A comparative case study of federal hydropower licensing. Policy
strategies? Policy Studies Journal. doi:10.1111/psj.12162. Studies Journal 43:283–308.
Simpson, Daniel, Finn Lindgren, and Håvard Rue. 2012. Think continuous: Wang, Xinhao. 2001. Integrating water-quality management and land-use
Markovian Gaussian models in spatial statistics. Spatial Statistics 1: planning in a watershed context. Journal of Environmental Management
16–29. 61:25–36.
Skaggs, R. W., M. A. Breve, and J. W. Gilliam. 1994. Hydrologic and water qual- Ward, Eric J. 2008. A review and comparison of four commonly used Bayesian
ity impacts of agricultural drainage. Critical Reviews in Environmental and maximum likelihood model selection tools. Ecological Modelling
Science and Technology 24:1–32. 211:1–10.
Spiegelhalter, David J., Nicola G. Best, Bradley P. Carlin, and Angelika Van Der Weber, Edward P. 2003. Bringing society back in: Grassroots ecosystem man-
Linde. 2002. Bayesian measures of model complexity and fit. Journal of the agement, accountability, and sustainable communities. Cambridge, MA:
Royal Statistical Society, Series B: Statistical Methodology 64:583–639. MIT Press.
Suárez, David F. 2011. Collaboration and professionalization: The contours Wikle, Christopher K. 2003. Hierarchical models in environmental science.
of public sector funding for nonprofit organizations. Journal of Public International Statistical Review 71:181–99.
Administration Research and Theory 21:307–26. Yang, Kaifeng, and Kathe Callahan. 2007. Citizen involvement efforts and
Thomas, C. W., and T. M. Koontz. 2011. Research designs for evaluating the bureaucratic responsiveness: Participatory values, stakeholder pressures,
impact of community-based management on natural resource conserva- and administrative practicality. Public Administration Review 67:249–64.

tion. Journal of Natural Resources Policy Research 3:97–111. Young, Juliette C., Andrew Jordan, Kate R. Searle, Adam Butler, Daniel
Tong, Susanna T. Y., and Wenli Chen. 2002. Modeling the relationship S. Chapman, Peter Simmons, and Allan D. Watt. 2013. Does stake-
between land use and surface water quality. Journal of Environmental holder involvement really benefit biodiversity conservation? Biological
Management 66:377–93. Conservation 158:359–70.

Is Collaboration A Good Investment Modeling

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Is Collaboration A Good Investment Modeling

Uploaded by

Copyright:

Available Formats

Journal of Public Administration Research And Theory, 2016, 769–786

Is Collaboration a Good Investment? Modeling

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Address correspondence to the author at tyler.scott@uga.edu.

Introduction environmental applications, particularly watershed

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

that this used for agricultural purpose or that is devel- Results

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Table 2. Covariates and Grant Funds in 1, 2, and 3 Water-Years Prior to Observation

Past 1-Year Funds Past 2-Year Funds Past 3-Year Funds

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Note. DIC = deviance information criterion; WAIC = Watanabe-Akaike information criterion.

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Figure 1. Sampled linear combinations from posterior marginals, funding to WCs.

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Past 1-Year Funds Past 2-Year Funds Past 3-Year Funds

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Note. DIC = deviance information criterion; WAIC = Watanabe-Akaike information criterion.

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

Downloaded from https://academic.oup.com/jpart/article/26/4/769/2223060 by guest on 10 March 2022

You might also like