See discussions, stats, and author profiles for this publication at: http://www.researchgate.

net/publication/232411770

Valuing local knowledge as a source of expert

data: Farmer engagement and the design of
decision support systems

ARTICLE in ENVIRONMENTAL MODELLING AND SOFTWARE · OCTOBER 2012

Impact Factor: 4.54 · DOI: 10.1016/j.envsoft.2011.09.013

CITATIONS DOWNLOADS VIEWS

15 98 136

6 AUTHORS, INCLUDING:

Michael Winter Chris J. Hodgson

University of Exeter Rothamsted Research
92 PUBLICATIONS 1,461 CITATIONS 16 PUBLICATIONS 222 CITATIONS

SEE PROFILE SEE PROFILE

Ann Louise Heathwaite Dave Robert Chadwick

Lancaster University Bangor University
190 PUBLICATIONS 3,999 CITATIONS 129 PUBLICATIONS 2,004 CITATIONS

SEE PROFILE SEE PROFILE

Available from: David M. Oliver

Retrieved on: 09 September 2015
 Environmental Modelling & Software 36 (2012) 76e85

Contents lists available at SciVerse ScienceDirect

Environmental Modelling & Software

journal homepage: www.elsevier.com/locate/envsoft

Valuing local knowledge as a source of expert data: Farmer engagement and the
design of decision support systems
David M. Oliver a, Rob D. Fish b, *, Michael Winter b, Chris J. Hodgson c, A. Louise Heathwaite d,
Dave R. Chadwick c
a
Biological & Environmental Sciences, School of Natural Sciences, University of Stirling, Stirling FK9 4LA, UK
b
Centre for Rural Policy Research, Department of Politics, University of Exeter, Exeter, Devon EX4 6TL, UK
c
Rothamsted Research, North Wyke, Okehampton, Devon EX20 2SB, UK
d
Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK

a r t i c l e i n f o a b s t r a c t

Article history: Engagement with farmers and landowners is often undertaken by the research community to obtain
Received 9 June 2011 information relating to typical land, livestock and enterprise management and generally centres on
Received in revised form responses to questionnaire surveys. Farmers and land managers are constituted as expert observers of
1 September 2011
ground-level processes and provide diverse information on farming practices, enterprise economics and
Accepted 13 September 2011
underpinning attitudes towards risk. Research projects designed to inform policy and practice may rely
Available online 6 November 2011
on such data to understand better on-the-ground decisions that can impact on environmental quality
and the rural economy. Such approaches to eliciting local-level expert knowledge can generate large
Keywords:
Decision support
quantities of data from which to formulate rules relating to farm enterprise types. In turn, this can help to
Expert inform the structure of Decision Support Systems (DSS) and risk-based tools to determine farming
Farmer practices likely to impact on environmental quality. However, in this paper we advocate the need for
Local knowledge integrated farmer participation throughout the whole research process e from project inception through
Questionnaire survey to community qualitative validation and legitimation - and thus not just for the elicitation of ques-
Stakeholder participation tionnaire responses. With farm questionnaire surveys being adopted widely by the research community,
Uncertainty it is an opportune time to highlight a recent case study of the Taw catchment, Devon, UK. This serves as
Water quality
an example of co-construction of a DSS via a co-ordinated and integrated approach to expert elicitation
with a farmer questionnaire survey as a central methodology. The aim of the paper is to detail the core
aspects of an iterative cycle of participatory environmental management and DSS development for water
quality protection and consider the multiple benefits of co-ordinated programmes of engagement with
the farming community in this process.
Ó 2011 Elsevier Ltd. All rights reserved.

1. Introduction
Farmers and land managers are increasingly targeted by scien-
tists to help inform research and policy tools (e.g. Nettle et al., 2010;
Vignola et al., 2010; Winsten et al., 2010). Knowledge and experi-
ence grounded in everyday land management practices can provide
insight only partly accessible through scientific observation, and
can serve to clarify and extend the evidence base informing deci-
sions. In other words, farmers and land managers are custodians of
salient local knowledge. They bring new reasoning and insight to
bear upon problems that might otherwise be missed by specialist
* Corresponding author.
E-mail address: Fish@exeter.ac.uk (R.D. Fish).
scientific knowledge. Alan Irwin has coined the term “Citizen
Science” to capture the lasting significance of this ‘lay’ expertise
(Irwin, 1995).
A range of research techniques are potentially available to help
scientists elicit farmer and land manager knowledge as the basis for
designing integrated and sustainable approaches to land manage-
ment. In this paper, we examine the key dimensions of an iterative
cycle of participatory environmental management for the devel-
opment of Decision Support Systems (DSS). In general terms, the
purpose of DSS is to translate wider policy concerns for sustainable
agriculture and water resources under climate change into opera-
tional, flexible and adaptive ‘on the ground’ responses. They
therefore remain high on the environmental research agenda even
though few DSS are currently used to inform policy and to drive
policy analysis (van Delden et al., 2011). Opening up such

1364-8152/$ e see front matter Ó 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.envsoft.2011.09.013
 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
frameworks to accommodate the skill and expertise of farmers is
a critical step in developing DSS that have utility to a range of
catchment stakeholders working between catchment-based
research and real world management decisions. This includes the
farming community, extension officers and farm advisors and
policy makers (Macleod and Haygarth, 2010).
In this paper our focus is on a DSS that aims to help manage risks
arising from agriculture for water quality and in particular, an
integrated research process that has a farmer questionnaire survey
as a central part of the DSS methodology. A parallel argument about
examining the use of questionnaire surveys to elicit specialist
scientific knowledge on land-water management has been dis-
cussed elsewhere (Fish et al., 2009). Questionnaire surveys have
been a key and longstanding element of social science research into
issues of land management and conservation (e.g. Potter and
Gasson, 1988; Morris and Potter, 1995; Fish et al., 2003).
However, one of the emerging challenges of interdisciplinary
research is to relate this type of work to the specific needs of DSS
and to embed such elicitation procedures within a more integrated
framework that involves the participation of the farming commu-
nity from project inception through to legitimisation of the
research product (Barreteau et al., 2010). As a way of eliciting local-
level knowledge, questionnaire surveys can generate large quan-
tities of data from which to generate ‘model’ rules relating to farm
enterprise types and help to discern farming practices likely to
impact on environmental quality. The proposition is that their value
and impact can be magnified considerably through careful coor-
dination of the wider participatory process itself (e.g. Ticehurst
et al., 2011; Selfa et al., 2010; Oliver et al., 2009a).
We begin by providing the empirical context to this paper; an
extensive questionnaire survey and programme of farmer partici-
pation undertaken by the authors in the Taw catchment in North
Devon, UK. The aim of the paper is to highlight an integrated
approach to participatory environmental management and DSS
development as exemplified using the Taw catchment project and
its associated questionnaire survey as a case study. As a result,
a sequential (and iterative) seven-step protocol is presented to
promote farmer participation (through both informal dialogue and
formal survey-based procedures) as an integral part of DSS
evolution.
2. Case study context
This paper bases its principal insights on a questionnaire survey
of 77 farms, and a three year process of farmer engagement with 10
of those farms in the Taw catchment, North Devon. The extended
engagement component was limited to ten farms and built on the
assumption that greater resources of time could be committed
towards nurturing a trusting working relationship with individual
research subjects (c.f. Jakeman and Letcher, 2003). The ten farms
were selected to represent a range of farm types and landscape/
geographical characteristics and were identified in advance of the
completion of all 77 farm questionnaires. The time and cost
constraints of typical 3-year research projects make it difficult to
undertake substantive engagement with a larger subset of farms.
Thus, our study developed an integrated participatory process with
ten farmers and framed the experiences in the wider context of the
77 farm systems. The participatory process is outlined in detail in
Section 4 but briefly it focused on a structured questionnaire con-
ducted face to face and designed to obtain information of on-farm
management practices likely to impact on the risk of microbial
transfers to watercourses from livestock faecal material and
manures deposited on farmed land. In general terms the ques-
tionnaire survey was structured to obtain information relating to
land, livestock and manure management and general
77
characteristics typical of farmed land in order to inform the output
of a risk indexing tool (essentially, a field and farm DSS; Oliver et al.,
2009a, 2010) to identify agricultural land most vulnerable for
contributing towards diffuse microbial pollution of water. Key
phases of farmer recruitment, relationship building, on-farm
monitoring and farmer qualitative validation were essential to
complement the core data obtained by questionnaire survey.
The DSS resulting from this project demonstrates considerable
compatibility with prevailing approaches to environmental risk
management, most notably the source-pathway-receptor model
(e.g. Lytton et al., 2003). The case study of farmer participation for
DSS co-construction therefore has relevance and applicability to
a broad range of diffuse agricultural pollutants that can be
modelled using this generic conceptual framework. While the
questionnaire survey and farmer participation was conducted to
draw on information relating to microbial transfers from land to
water to inform a DSS, the Taw case study analysis below identifies
general issues and complexities, in addition to the advantages and
limitations of the data, that can be of benefit to future participatory
modelling studies.
3. Engaging farmers within DSS: underpinning rationales and
approaches
In this section we explain how the process of designing and
implementing a DSS relates to wider rationales for stakeholder
engagement within environmental management, and the partic-
ular function that questionnaire surveys and wider consultation
serve within this field. In principle, the idea of engagement with
stakeholders - in this case the farming and land management
community - has tended to be based on three main arguments (see
especially Fiorino, 1990):
 Inclusiveness: that individuals have a legitimate right to influ-
ence processes that have a direct bearing on them. In this
sense, engagement with farmers in DSS design should be seen
as a pre-requisite of the research process because the system
implies change;
 Acceptability: that greater trust and legitimacy can be built into
what is being proposed by the environmental management
process. In other words engagement is a better way of
achieving particular ends. Farmers and land management
organisations and groups, will potentially be more accepting of
the DSS if involved in the processes and reasoning that
underpinned it;
 Effectiveness: that outcomes and outputs are realised more
effectively. For example active engagement with farmers
provides a source of practical know-how that can help inform
the DSS and lead to a better product.
In practical terms, it is useful to think of these three rationales in
terms of three main types of engagement (Chilvers, 2009), namely:
 Informing stakeholders: non-interactive engagement designed
to disseminate information to constituencies of interested/
affected people. Thus, engagement techniques are employed to
make farmers and land managers aware of the existence and
utility of the DSS;
 ‘Learning from’ stakeholders: interactive engagement designed
primarily to take account of the views, interests or insights of
stakeholders. This type of engagement is therefore concerned
with drawing out farmer and land manager insight to inform
the DSS;
 ‘Collaborating with’ stakeholders: shared approaches to decision
making among stakeholders based, for instance, on dialogue,
78 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
discussion, deliberation and collective problem framing. In this
case farmers and land managers work directly with specialists
to create the DSS.
A framework for clarifying the role of participation in partici-
patory research has been proposed elsewhere as the basis for
promoting and facilitating quality control in participatory research
(Barreteau et al., 2010). Taking these engagement rationales and
types together, our general proposition is that questionnaire
surveys should primarily be seen as a mechanism for drawing out
farmer knowledge as the basis for improving model effectiveness
but that the methodology needs to be couched within a much
larger programme of engagement (e.g. a shift to ‘collaborating with
stakeholders’) as outlined above.
4. Co-construction of a DSS: The Taw catchment case study
Fig. 1 outlines a protocol for effective farmer-scientist co-
construction of a DSS guided by seven iterative stages as outlined in
the following (4.1e4.7). The co-learning process is an iterative cycle
and Fig. 1 is not dissimilar to the approach of modelling with
stakeholders highlighted in Voinov and Bousquet (2010). Indeed,
approaches to social learning are well documented as being central
to the management of complex environmental issues (e. g.; Pahl-
Wostl et al., 2011, 2010; Jakeman and Letcher, 2003). The cycle
Fig. 1. Seven stage protocol for effective farmer-scientist co-construction of a DSS: key iterative steps needed in the development of DSS for farmers.
proposed here was trialled in the Taw catchment and novel insights
from this case study are detailed within each of the seven steps. A
precursor to the seven iterative steps will always be a clear defi-
nition of the research need and this was established (and evolved)
in conjunction with the farming community. This is an important
first phase for ensuring that any questionnaire survey and pro-
gramme of engagement is structured in order to meet the objec-
tives of the science under investigation and to avoid a mismatch
between the type of engagement desired (i.e. ‘inform’, ‘learn from’
or ‘collaborate’) in the DSS formulation process and the type of
technique used. As we have explained, questionnaire surveys are an
appropriate means of eliciting key information but should be
framed within a more integrated approach to elicitation so that
‘users’ of models become active participants in fostering the
acceptability of DSS at project inception (Oliver et al., 2009b). They
are a ‘learning from’ approach designed to inform the evidence base
behind DSS.
4.1. Establish relationships (Fig. 1 stage 1)
In the Taw case study the farming community were contacted
using a four-pronged approach involving the use of: (i) leaflet
distribution (see Fig. 2); (ii) existing networks; (iii) word-of-mouth;
and (iv) cold-calling. The purpose was to make clear the importance
of farmer opinions and to dispel any suggestion that their
 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85 79

Fig. 2. Leaflet distributed to farmers in the Taw catchment as a method of introducing the project ahead of making contact and engaging in person.

participation was simply a pro-forma exercise (c.f. Glicken, 2000). shot across the catchment and this was followed up with a tele-
The leaflet (Fig. 2) was designed to emphasise the point that the phone call to attempt to arrange an interview. The leaflet proved to
research team needed the assistance of the farming community to be an important component of the Taw study: it introduced the
take this DSS project forward. The leaflet was distributed via mail- project, expressed a call for help from the farming community,
80 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
highlighted the central role of farmers in the project, informed that
we were independent of any government agency, and showed
a photograph of the three researchers who would be undertaking
the project and engaging with farmers and landowners. This
therefore served to introduce the team ahead of the farm visit.
Obtaining lay information from farmers can provide crucial data to
the research community yet is often undervalued at project
inception stages. Appreciating the value of such information, and
anticipating farmer questions of ‘what’s in it for me’, the leaflet also
informed of a free manure or slurry nutrient test to all those who
participated in the survey (and access to the latest advice on how to
manage livestock manures). This not only provided the farmers
with useful information free of charge but also facilitated discus-
sion around land and nutrient management issues and helped to
conceptualise the DSS framework through agreed common needs
(Quinn et al., 2010).
The project team chose to adopt a face-to-face approach to
engagement. This approach allowed an ongoing rather than ‘one-
off’ relationship with the respondent to be established. While
repeat visits are a methodological ‘ideal’ in terms of resource and
time commitments they can also help reduce the uncertainties
associated with the acquisition of farm management data and
unlock the depth of information central to a farmer’s day-to-day
routine (on a limited number of the farm sample). For example,
in the Taw catchment case study, a participating dairy farmer
claimed in the original survey return that he had purchased enough
land (for increased slurry spreading while avoiding overloading
pasture with faecal material and associated contaminants) to allow
for an increase in his dairy herd, and that when he spread slurry to
land he ensured he left a 10 m buffer between the watercourse and
the applied slurry. Further visits to the farm revealed new intents
and attitudinal positions. At a later date the same respondent
quipped once, ‘I get a thrill when I hear muck splash in the water’ and
suggested that ‘I always spread slurry onto the 3 or 4 most convenient
fields next to the slurry lagoon’. Both of these comments contradict
the responses given within the formal questionnaire survey. It is
a debating point in this case regarding which information was more
true or accurate, but certainly our reasoning is that, as formalities
are dropped and confidences built, deeper insights into system
characteristics and behaviour are brought to bear upon the problem
at hand. This may relate to an initial lack of confidence on the
farmers behalf in terms of how survey responses may be inter-
preted, and may possibly have been linked to concern about the
regulatory implications of supplying particular types of manage-
ment information to ‘researchers’. In this sense, it proved especially
important for the project team to stress the funding mechanism of
the project (e.g. non-government, Research Council) and to clarify
the disassociation between the project and regulatory authorities.
In addition, face to face interviews in practical settings provided
opportunities to ‘ground-truth’ claims made in a survey. For
example a participating beef farmer in the Taw Study informed the
project team of his clean and dirty water separation system within
the farm hard-standings that allowed clean water to be channelled
to a clean water drain which in turn diverted the clean water to the
stream. When the interview team finished discussions with the
farmer and left the farm office a farm worker was witnessed hosing
down a tractor that was covered in fresh farmyard manure directly
above the clean water drain that diverted to the stream. This latter
example serves to demonstrate the value of on-farm survey
approaches because of the different levels (and mismatches) of
information that can be obtained from first-hand experiences.
Besides, such information is difficult to detect as a pollution signal
in water quality because of the activity’s incidental nature. This
example is of course coincidental with the timing of the research
team’s visit but the key point to make is that by establishing
relationships, making regular visits and by maintaining commu-
nication there is ample opportunity to undertake verification of
questionnaire survey responses by direct observation as a cross-
comparison to other more formal data sources. So whether coin-
cidental or not, regular ‘checks’ for consistency between on-the-
ground operation and survey responses can be maintained and
reveal interesting supplementary or confirmatory data.
The successful recruitment of farmers for management surveys
and wider engagement can be enhanced through contacts at
respected environmental research institutes and the reputations of
organizations can therefore prove vital for research programmes
that need the help of the farming community. Sterk et al. (2009)
highlight examples of work undertaken in Ecuador whereby
extensive contact networks in environmental research institutes
facilitated positive research activities and enhanced farmer
engagement. Likewise, in the Taw catchment case study, the
grasslands research institute - North Wyke Research - provided
a mechanism whereby respondents could relate to the presence
and reputation of the organization from past and extensive
research undertaken in the local area. Furthermore, local farms
were made aware that this institution is independent of govern-
ment policing agencies linked to land and water quality. These
arguments can work in reverse too; if reputations are not held in
high regard, or seen as threatening, interviewers may be impeded
in their activities (Fairweather, 2008).
However, long-term relationships cannot always be developed
as part of a research project because of a mismatch in funding cycle
duration. This issue is not confined to the UK; Nancarrow (2005)
highlights this as a major problem in Australia because of the
time required to develop a degree of trust and familiarity with the
[farming] community and the race against the three year funding
cycle. Long-term research platforms or ‘catchment observatories’,
with farming communities forming an integral part of the research
network, offer attractive benefits to modelling studies since they
can provide longitudinal data rooted in wider farmer insights and
ownership.
4.2. DSS conceptualisation (embedding expert dialogue into
conceptualisation of DSS; Fig. 1 stage 2)
McCown (2002) suggested that a critical problem with
‘management science models’ is that managers practically never
use them. Others have stressed the overarching need to ask ‘why
are we building a model or DSS?’ (Nancarrow, 2005). If tools are to
be used directly by the farming community or in cooperation with
the regulatory community then there is an imperative for clear
communication between science providers and science users to
maximise utility of products to both parties. Active engagement in
early stages is important to bridge rather than reinforce the
science-stakeholder divide and accommodate knowledge or
understand the concerns of those who need to use emerging tools
(Quinn et al., 2010; Barreteau et al., 2010). By embracing farmers as
experts and encapsulating their detailed local knowledge into DSS
the resulting products are more reflective of the farm system and
perhaps more accessible or appealing to the farming community
because they are seen to support the decision making process rather
than attempt to make the decision.
Whatever the purpose of a DSS there are considerable benefits
to be gained by simply entering into a dialogue with end-users at
early stages (preferably inception) of DSS conceptualisation and
then throughout the design process. This will ensure that their
knowledge and input is included within the design process but
ultimately it also facilitates social learning (van Delden et al., 2011).
Through the formation of trusting relationships with the farming
community such dialogue should take place organically and form
 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
the second iterative step in our seven-stage sequential protocol. In
the Taw catchment case study there were ample occurrences
whereby qualitative information was provided through informal
conversations during routine water quality monitoring that took
place on farms. These points were expanded upon during the more
formal questionnaire survey and allowed for open and purposeful
discussion of the DSS requirements linked to microbial pollution of
watercourses. For example, the importance of ‘within field hot-
spots’ of pollution sources was raised by many of the farmers
during early discussions of the practicality of on-farm monitoring
strategies. This in turn contributed to the development of both
a field- and farm-scale DSS to allow farmers to identify not only the
overarching drivers of risk within their farm systems but also finer
resolution field-by-field vulnerability to microbial loss from land to
water. This latter point was critical in that the resulting DSS
provided farmers with a method to prioritise land for a shift in
management or introduction of a mitigation option.
4.3. Farmer survey & expert elicitation (Fig. 1 stage 3)
There are a number of key criteria to evaluate before under-
taking a questionnaire survey or elicitation procedure with
respondents. Careful consideration must be given to: (i) question-
naire survey design; (ii) sample population; (iii) recruiting
Fig. 3. Map of dairy farm annotated by farmer to show grazing patterns, manure applications, drainage patterns and no-spread zones.
81
respondents; (iv) questionnaire survey conduct; and (v) question-
naire survey analysis. It is not the intention of this study to offer
a roadmap of questionnaire survey design as there is extensive
information available detailing such procedures (e.g. Oppenheim,
2000). From the experiences of the Taw catchment study
whereby the land, manure and livestock management practise of
77 farmers were recorded across a 1200 km2 area, there were clear
advantages in using particular approaches over others. Face-to-face
meetings allowed for a first-hand visualisation of the farm
management regimes providing critical secondary data to
complement the dialogue between farmers and researchers. Closed
questions were used to obtain key numerical data linked to herd
size and tanker volumes, but more open-ended discussion was
promoted around the use of visual maps annotated by the farmers
and through open-ended questions towards the latter stages of the
survey. The face to face approach provided a context in which
different information elicitation techniques could be deployed in
conjunction with the structured survey. For example, in our study
we employed map-based questions as a way of collecting and
visualizing information about the farm landscape (Winter et al.,
2011). Fig. 3 shows an example of data recorded from farmers
using these map-based approaches to elicit knowledge of drainage,
land use, soil properties, grazing regimes and manure applications
associated with their property. In principle, map orientated
82 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
discussions allow the linkage of management approaches and
expert knowledge with spatial coverage of the farm area. This
interactive graphical component in the research process provided
a detailed spatial representation of the farmscape adding subtlety
and depth to the results of the central questionnaire survey
methodology. Our experience was that this mapping exercise also
served to make the farmer feel more comfortable with the
discussion rather than being “interrogated” over management
approaches via a rigid questionnaire. In the Taw study farmers
populated the map using coloured pencils and pens, though it is
possible to envisage more technologically advanced approaches,
such as participatory GIS (Macleod et al., 2007). The questionnaire
surveys were trialled within the research team before being
employed on-farm. A hard-copy survey was preferred over an
electronic version on a laptop simply because of the awkwardness
in loading the computer operating system and the delay in being
able to get started in conversation because of powering-up issues.
It is important to reiterate a number of critical assumptions that
are inherent to the conduct of questionnaire surveys. For example,
a questionnaire survey tends to proceed from the assumption that
interviewees will:
 understand what an interviewer is asking of them (i.e. question
comprehension)
 speak when they are spoken to (i.e. willingness to answer)
 say what they mean when they speak (i.e. capacity to
articulate)
Equally the questionnaire survey format also assumes that
interviewers will:
 understand what the interviewee is saying to them; (i.e.
answer comprehension)
 respond appropriately to points of clarification (i.e. capacity to
articulate)
 conduct the survey in circumstances where completion is
guaranteed.
On this basis it is argued that a survey is amenable to ‘stand-
ardised’ knowledge leading to consistent, comparable results from
which we can begin to generalise. However, it is not surprising to
also identify that with such a range of assumptions the credibility of
the source of data can often be put in jeopardy. It is highly plausible
that erroneous responses will arise (through misunderstanding or
through misdirection) and that the reliability of data may be
dubious through identification of contradictions in survey returns.
In some cases answers may well appear to lack credibility because
responses appear to be improbable. In the context of all of the
assumptions above, questionnaire surveys generally take place in
a less than perfect way. All of these problems can be mitigated to
some extent in advance, such as: trialling questions; training
interviewers; anticipating where reluctance to engage, and
potential confusions, may exist; and being explicit about practical
questionnaire survey needs, such as length of time needed.
However, neither structured nor semi-structured interviews can by
themselves provide a complete insight into farm operations but by
embedding this methodology within an iterative programme as
depicted in Fig. 1 we can begin to cross-interview with a diverse
range of alternative information.
4.4. Expert driven DSS output (Fig. 1 stage 4)
The research community must give careful consideration to the
range of potential errors and misinterpretations of results that may
arise via questionnaire survey return and wider qualitative data.
Otherwise such errors may compromise the conceptualisation and
execution of a DSS reliant on such input data. It is not difficult to
envisage a researcher being unaware of the limitations of survey
returns particularly if s/he was not responsible for the original
survey design, and simply used the template to ‘extract’ informa-
tion from a respondent. This point strengthens the argument made
in Section 4.1 with regard to familiarisation and an evolving rela-
tionship between researcher and farmer in order to reap most
reward. Uncertainties, limitations and sources of error can easily
become disguised, forgotten and embedded within DSS frame-
works without necessary caveats being associated with the source
data. The risk then is that erroneous data and anecdotal evidence
become inherent to the functioning of a DSS (Davern et al., 2008).
Sources of uncertainty can include, for example, aspects of
missing data, potential for contradiction and a lack of confidence in
the reliability of some responses. Certainly, large structured ques-
tionnaires will often comprise multiple sections meaning that
answers to questions in earlier sections may be related to responses
given in later aspects of the survey return. Technology can play an
important role for improving consistency in the logic of answers:
electronic surveys can be programmed to prevent contradictions
being made through the embedding of logic rules into the frame-
work that restrict text entry into response boxes if certain answers
were given previously. Innovation in web-based surveys (e.g.
Fleming and Bowden, 2009) can use visual stimuli to alert the
respondent to missing data or contradictory responses. However,
technology (e.g. computers) can also represent an awkward 3rd
party in questionnaire situations so the Taw catchment survey
work was better conducted using a hard-copy paper survey and
digitised at a later date (though this can allow for potential errors in
data transfer and so double data entry would therefore be good
practice). In the case of paper-based surveys it is important to
understand how best to deal with contradictions raised in ques-
tionnaires. There is a realisation that we need to develop meth-
odologies that can incorporate rather than deny contradictions, but
contradictions raise problems of coherent statistical analysis. This
would suggest an urgency to self-validate questionnaires and
devise strategies to cross-compare data sources (c.f. Voinov and
Gaddis, 2008) from a wider programme of farmer engagement
(e.g. steps 1e3).
4.5. Qualitative validation (Fig. 1 stage 5)
We advocate a drive for increased use of farmers as experts not
just for informing model input and output, but also for qualitative
validation of agricultural DSS prediction (thereby moving toward
a more integrated and collaborative approach). This critical step in
the iterative cycle of Fig. 1 is fundamental as it provides a ground-
level ‘sanity check’ from those who have everyday expertise in their
land. In addition, continued involvement of farmers in the DSS
validation should in turn help to promote subsequent uptake of the
DSS by the farmer community. Validation is therefore not only
about comparing observed and predicted outcomes from DSS but
also about evaluating DSS output against expert (farmer in this
case) opinion (Le Gal et al., 2010) and allows for refinement of the
science by local stakeholders (Raymond et al., 2010). Oliver et al.
(2010) proposed farmer engagement for such qualitative valida-
tion of a risk indexing tool which currently forms a first approxi-
mation for the protection of watercourses from microbial pollution
brought about by farming activity. Qualitative validation removes
a somewhat biased view that DSS will always result in sensible
output when in reality the peculiarity of heterogeneities in the
landscape may result in differing trends. This concept is outlined by
Landry et al. (1996) who suggest that any model that is not legiti-
mate in the eyes of those for whom the DSS (or model) has been
 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
developed will not have much opportunity for acceptance and
wide-spread uptake by that particular community. Legitimacy in
DSS is therefore highly desirable because it reinforces the value of
that particular tool and also receives a seal of approval from end-
users.
A recent study combining farmer and scientist expertise in
discussions of sustainable agriculture drew attention to initial
conflicts in opinion over the value of scientific knowledge versus
farmer knowledge. However, conceptions of scientific knowledge
being ‘superior’ and ‘more valid’ than farmer’s local experience
were abandoned (Eshuis and Stuiver, 2005). Farmer legitimation of
a DSS is important in terms of providing a sense of ownership of the
process but does of course mean that the resulting DSS must be
transparent and straightforward in operation (Voinov and
Bousquet, 2010; Voinov and Brown Gaddis, 2008). Obviously
quantitative validation to complement qualitative validation is
needed, though as noted by Oliver et al. (2010) on-farm verification
of field and farm-scale tools is often difficult to achieve due to
inherent uncertainties associated with pollutant sources and
pathways within farm systems. Indeed, Merot and Bergez (2010)
suggest that it is very difficult to validate decision support tools
because farmers tend not to record their day-to-day activities, and
certainly not on a field-by-field basis.
The Taw case study project produced a briefing document for
each of the 10 farmers who participated in the environmental
monitoring phase of the project. This document was designed to
highlight the key research findings linked to each farm and provide
a spatial risk map for the farmscape. This provided a further
opportunity for feedback and comment as the documents were
delivered in person and immediate comments noted. All farmers
were encouraged to get in touch should further queries or ques-
tions arise from the DSS output that had been provided to them.
Key feedback from the farming community tended to focus on their
desire to be able to generate the DSS output themselves (or via
a farm advisor) and therefore comments were made regarding the
need to refine these DSS into ‘user-friendly’ operating systems (see
Section 4.7). Suggestions of the provision of ‘look-up-tables’ to
accompany the DSS were made because some farmers commented
that if a farm advisor was not in attendance then some terminology
of risk factors would require clarification. The nature of these
particular DSS (see Oliver et al., 2009a, 2010) were regarded as
having strong visual impact and suggestions were made regarding
the coupling of these tools to DSS that consider other pollutants
also. Finally, one farmer queried the use of arbitrary risk scores and
suggested that accompanying textual information may be appro-
priate for linking to each score so that the significance of the results
were made more clear in the absence of a farm advisor.
4.6. Community qualitative validation (Fig. 1 stage 6)
Briefly, as with all models and DSS, their transferability to
different geographical areas is essential for wider, national scale,
uptake. The discussion of Section 4.5 must be applied beyond
individual farm gates and extended to communities of farmers and
landowners to determine the geographical constraints of DSS.
Qualitative validation from researcher, advisor and farmer points of
view following simulations of prospective scenarios in the agri-
cultural environment all provide valuable data with which to
constrain the DSS capability (Vayssieres et al., 2009). In the case of
the Taw study, the project team hosted an open-day at North Wyke
Research and invited the ten core farmers to attend along with
other catchment stakeholders. This was an opportunity for the
farming community to view outputs of the project via poster
presentations and provide feedback and comment on the data thus
far which helped constrain features important to the development
83
of the DSS. This effectively allowed the farming community to seek
clarification as to why certain information may not have been acted
upon, consider the impact of their knowledge that had been
incorporated into the DSS and also provided an opportunity for
farmers to cross-examine other spatial risk maps derived for
differing farm systems and question particular influences on the
DSS output in an informal environment. It is clear that research
processes cannot involve all necessary decision makers in the co-
construction of environmental management tools. Community
qualitative validation therefore provided an additional mechanism
to capture feedback from those who were not involved directly in
the research but who would use the results and outputs of this
research process. This phase allowed for the expression of agree-
ments and disagreements with the proposed DSS to facilitate
collective learning and DSS refinement (Daniell et al., 2010). The
community qualitative validation phase provided a novel and
worthwhile extension to the project and led to clarifications of
terminology used in guidance documents. Other methods for
extending the involvement to additional excluded decision-makers
have been documented in Hare et al. (2003). The management of
participatory validation needs to be undertaken with as much care
and attention as attributed to the environmental monitoring and
modelling procedures if the research tool is to benefit from useful
information from the correct set of actors and for a common
agreement to be reached among all involved. This latter point is
unrealistic of all stakeholders involved in contentious environ-
mental issues but for the purposes of the Taw case study the DSS
was considered legitimate by the farming community involved.
4.7. Graphic user interface (Fig. 1 stage 7)
The apathy of farmers and landowners towards DSS may be
linked to their (lack of) packaging for end-user uptake. A clear and
well constructed Graphical User Interface (GUI) that allows end-
users without specific modelling skills or knowledge of the
modelling system to take advantage of the DSS is likely to yield
higher interest and usage among the farming community than
a complex DSS that is essentially inaccessible to user groups. This
has been effectively demonstrated for data management interfaces
in the drinking water treatment sector (Worm et al., 2010) who
noted that the acceptance and appreciation of a DSS is increasingly
determined by the ‘look and feel’ of the system. The Taw study was
not able to develop a GUI in the project timeframe but additional
funds are being targeted specifically for this purpose. This is seen as
a critical next stage to ensure that the DSS is actually used by the
user-community rather than shelved as an inaccessible product of
academic research deemed ‘unfit for purpose’. It is proposed that an
enhanced user interface will allow extension officers to use the DSS
to demonstrate potential implications of land management
changes for a range of pollutants and scenarios.
5. Conclusions
Valuing farmers’ knowledge reflects a move towards more
integrated approaches to land and water management: with
natural scientists, sociologists, economists, farmers and regulators
all contributing knowledge that can assist in resolving
agriculturally-derived issues (Hewett et al., 2010) Farmer knowl-
edge is a vital form of expert data and can be acquired via a suite of
formal and informal stages throughout the development of
a working relationship between researchers and the farming
community. The analysis presented in this paper complements
previous reviews of DSS co-construction but provides a UK case
study analysis of DSS development through a seven-step iterative
process.
84 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
From the perspective of natural science one of the strongest
benefits of this process was the ability to improve DSS accuracy
through the establishment of a trusting relationship with the
farming community. This was demonstrated best with the subset of
the ten farms and their involvement in the DSS evolution. Without
the building of trust there are clearly opportunities for errors and
uncertainties to be propagated through the elicitation procedure
and into DSS output. As outlined in Section 4.7, it is hoped that
further uptake and use of these DSS can be promoted once proto-
typical versions are refined to allow for greater accessibility by the
farming community. Farmer feedback (as outlined in Section 4.5)
has been encouraging and suggests that such approaches have, in
part, been welcomed as qualitative risk assessment tools.
The cyclical nature of the process is also important to empha-
sise. Although additional iterations are likely to incur further costs
and take place beyond the life-cycle of the initial funding phase it is
important that DSS’s evolve according to emerging research needs
and evidence. This also reinforces a point made by van Delden et al.
(2011): that iterations allow for adaptations to the prototype DSS
once they have been developed and a better understanding of their
possibilities and limitations can then be obtained.
Such adaptations depend on cultivating the long-term buy-in of
experts in the farming community. Farmer relationships are the
‘social infrastructure’ behind DSS evolution and these take time to
nurture. They should therefore be maintained where possible, such
as via longer-term catchment initiatives. Thus rather than finish
abruptly, engagement processes should ideally inform the
continuing refinement of the DSS.
There is little doubt that in recent years we have seen a shift in
farmer-scientist relationships from one of knowledge transfer to
one of knowledge exchange, learning and two-way communication
of information and advice. This is a positive development for
interdisciplinary researchers with ambitions to build credible DSS
and needs to be embraced. Our conclusion is that programmes of
participatory modelling need to provide sufficient opportunity
throughout and beyond the research process to deal with and
accommodate local expert knowledge within new and emerging
DSS.
Acknowledgements
The authors would like to acknowledge the 77 participating Taw
Catchment Farmers for their valuable and insightful contributions
to the wider findings of our research project RES-224-25-0086
which was funded by the Rural Economy and Land Use (RELU)
programme who receive financial support from the Biotechnology
and Biological Sciences Research Council (BBSRC), the Natural
Environment Research Council (NERC), the Economic and Social
Research Council (ESRC), the Department for Environment, Food
and Rural Affairs (Defra) and the Scottish Government. Additional
thanks are extended to Donald Barr for design of the Microsoft
Excel questionnaire template and data extractor. Finally, we thank
the constructive comments of the three reviewers and the editorial
board.
References
Barreteau, O., Bots, P.W.G., Daniell, K.A., 2010. A framework for clarifying ‘partici-
pation’ in participatory research to prevent its rejection for the wrong reasons.
Ecology & Society 15 (online).
Chilvers, J., 2009. Deliberative and participatory approaches in environmental
geography. In: Castree, N., Demeritt, D., Liverman, D., Rhoads, B. (Eds.),
A Companion to Environmental Geography. Blackwell, Oxford, pp. 400e417.
Daniell, K.A., Mazri, C., Tsoukiàs, A., 2010. Real world decision-aiding: a case of
participatory water management. In: Rios Insua, D., French, S. (Eds.), e-
Democracy: A Group Decision and Negotiation Perspective. Springer, Dordrecht,
pp. 125e150.
Davern, M.J., Mantena, R., Stohr, E.A., 2008. Diagnosing decision quality. Decision
Support Systems 45, 123e139.
Eshuis, J., Stuiver, M., 2005. Learning in context through conflict and alignment:
farmers and scientists in search of sustainable agriculture. Agriculture and
Human Values 22, 137e148.
Fairweather, J.R., 2008. The number of farms and farmers in New Zealand: a plea for
researcher access to farm samples and the inclusion of farmer data. New Zea-
land Journal of Agricultural Research 51, 485e488.
Fish, R., Seymour, S., Watkins, C., 2003. Conserving English landscapes: land
managers and agri-environmental policy. Environment and Planning A 35,
19e41.
Fish, R., Winter, M., Oliver, D.M., Chadwick, D., Selfa, T., Heathwaite, A.L.,
Hodgson, C.J., 2009. Unruly pathogens: eliciting values for environmental risk in
the context of heterogeneous expert knowledge. Environmental Science and
Policy 12, 281e296.
Fiorino, D.J., 1990. Citizen participation and environmental risk - a survey of insti-
tutional mechanisms. Science Technology & Human Values 15, 226e243.
Fleming, C.M., Bowden, M., 2009. Web-based surveys as an alternative to traditional
mail methods. Journal of Environmental Management 90, 284e292.
Glicken, J., 2000. Getting stakeholder participation ‘right’: a discussion of partici-
patory processes and potential pitfalls. Environmental Science & Policy 3,
305e310.
Hare, M., Letcher, R.A., Jakeman, A.J., 2003. Participatory modelling in natural
resource management: a comparison of four case studies. Integrated Assess-
ment 4, 62e72.
Hewett, C.J.M., Doyle, A., Quinn, P.F., 2010. Towards a hydroinformatics framework
to aid decision-making for catchment management. Journal of Hydro-
informatics 12, 119e139.
Irwin, A., 1995. Citizen Science: A Study of People, Expertise and Sustainable
Development. Routledge, London.
Jakeman, A.J., Letcher, R.A., 2003. Integrated assessment and modelling: features,
principles and examples for catchment management. Environmental Modelling
& Software 18, 491e501.
Landry, M., Banville, C., Oral, M., 1996. Model legitimisation in operational research.
European Journal of Operational Research 92, 443e457.
Le Gal, P.Y., Merot, A., Moulin, C.H., Navarrete, M., Wery, J., 2010. A modelling
framework to support farmers in designing agricultural production systems.
Environmental Modelling and Software 25, 258e268.
Lytton, L., Howe, S., Sage, R., Greenaway, P., 2003. Groundwater abstraction pollu-
tion risk assessment. Water Science & Technology 47, 1e7.
Macleod, C.J.A., Scholefield, D., Haygarth, P.M., 2007. Integration for sustainable
catchment management. Science of the Total Environment 373, 591e602.
Macleod, C.J.A., Haygarth, P.M., 2010. Integrating water and agricultural manage-
ment under climate change. Science of the Total Environment 408, 5619e5622.
McCown, R., 2002. Changing systems for supporting farmers’ decisions: problems,
paradigms, and prospects. Agricultural Systems 74, 179e220.
Merot, A., Bergez, J.E., 2010. IRRIGATE: a dynamic integrated model combining
a knowledge-based model and mechanistic biophysical models for border
irrigation management. Environmental Modelling and Software 25, 421e432.
Morris, C., Potter, C., 1995. Recruiting the new conservationists: the adoption of UK
agrienvironmental schemes. Journal of Rural Studies 11, 51e63.
Nancarrow, B.E., 2005. When the Modeller Meets the Social Scientist or Vice-versa.
www.mssanz.org/modsim05/papers/nancarrow.pdf (accessed 04.05.11).
Nettle, R., Paine, M., Penry, J., 2010. Aligning farm decision making and genetic
information systems to improve animal production: methodology and findings
from the Australian dairy industry. Animal Production Science 50, 429e434.
Oliver, D.M., Fish, R.D., Hodgson, C.J., Heathwaite, A.L., Chadwick, D.R., Winter, M.,
2009a. A cross-disciplinary toolkit to assess the risk of faecal indicator loss from
grassland farm systems to surface waters. Agriculture, Ecosystems and Envi-
ronment 129, 401e412.
Oliver, D.M., Heathwaite, A.L., Fish, R.D., Chadwick, D.R., Hodgson, C.J., Winter, M.,
Butler, A.J., 2009b. Scale appropriate modelling of diffuse microbial pollution
from agriculture. Progress in Physical Geography 33, 358e377.
Oliver, D.M., Page, T., Hodgson, C.J., Heathwaite, A.L., Chadwick, D.R., Fish, R.D.,
Winter, M., 2010. Development and testing of a risk indexing framework to
determine field scale critical source areas of faecal bacteria on grassland.
Environmental Modelling and Software 25, 503e512.
Oppenheim, A.N., 2000. Questionnaire Design. Continuum, London.
Pahl-Wostl, C., Jeffrey, P., Issendahl, N., Brugnach, M., 2011. Maturing the new water
management paradigm: progressing from aspiration to practice. Water
Resource Management 25, 837e856.
Pahl-Wostl, C., Mostert, E., Tabara, D., 2010. The growing importance of social
learning in water resources management and sustainability science. Ecology &
Society 13, 24 (online).
Potter, C., Gasson, R., 1988. Conservation through land diversion: results from
a survey. Journal of Agricultural Economics 39, 340e351.
Quinn, P.F., Hewett, C.J.M., Muste, M., Popescu, I., 2010. Towards new types of water-
centric collaboration. Proceedings of the Institution of Civil Engineers e Water
Management 163, 39e51.
Raymond, C.M., Fazey, I., Reed, M.S., Stringer, L.C., Robinson, G.M., Evely, A.C., 2010.
Integrating local and scientific knowledge for environmental management.
Journal of Environmental Management 91, 1766e1777.
Selfa, T., Fish, R., Winter, M., 2010. Farming livelihoods and landscapes: tensions in
rural development and environmental regulation. Landscape Research 35,
595e612.
 D.M. Oliver et al. / Environmental Modelling & Software 36 (2012) 76e85
Sterk, B., Leeuwis, C., van Ittersum, M.K., 2009. Land use models in complex societal
problem solving: plug and play or networking? Environmental Modelling and
Software 24, 165e172.
Ticehurst, J.L., Curtis, A., Merritt, W.S., 2011. Using Bayesian networks to comple-
ment conventional analyses to explore landowner management of native
vegetation. Environmental Modelling and Software 26, 52e65.
Van Delden, H., Seppelt, R., White, R., Jakeman, A.J., 2011. A methodology for the
design and development of integrated models for policy support. Environ-
mental Modelling and Software, 266e279.
Vayssieres, J., Guerrin, F., Paillat, J.M., Lecomte, P., 2009. GAMEDE: a global activity
model for evaluating the sustainability of dairy enterprises part I e whole-farm
dynamic model. Agricultural Systems 101, 128e138.
Vignola, R., Koellner, T., Scholz, R.W., McDaniels, T.L., 2010. Decision making by
farmers regarding ecosystem services: factors affecting soil conservation efforts
in Costa Rica. Land Use Policy 27, 1132e1142.
85
Voinov, A., Bousquet, F., 2010. Modelling with stakeholders. Environmental
Modelling and Software 25, 1268e1281.
Voinov, A., Brown Gaddis, E.J., 2008. Lessons for successful participatory watershed
modelling: a perspective from modelling practitioners. Ecological Modelling
219, 197e207.
Winsten, J.R., Kerchner, C.D., Richardson, A., Lichau, A., Hyman, J.M., 2010. Trends in
the Northeast dairy industry: large-scale modern confinement feeding and
management-intensive grazing. Journal of Dairy Science 93, 1759e1769.
Winter, M., Oliver, D.M., Fish, R., Heathwaite, A.L., Chadwick, D., Hodgson, C., 2011.
Catchments, subcatchments and private spaces: scale and process in managing
microbial pollution from source to sea. Environmental Science & Policy 14, 315e326.
Worm, G.I.M., van der Helm, A.W.C., Lapikas, T., van schagen, K.M., Rietveld, L.C.,
2010. Integration of models, data management interfaces and training support
in a drinking water treatment plant simulator. Environmental Modelling and
Software 25, 677e683.