Decision Analysis informs ®

Vol. 4, No. 4, December 2007, pp. 211–226

issn 1545-8490
eissn 1545-8504
07
0404
0211 doi 10.1287/deca.1070.0098
© 2007 INFORMS

e-Participation and Decision Analysis

Simon French
Manchester Business School, University of Manchester, Manchester, M15 6PB, United Kingdom,
simon.french@mbs.ac.uk
David Rios Insua
Statistics and Operations Research, Universidad Rey Juan Carlos, Móstoles, E 28933 Madrid, Spain, david.rios@urjc.es

Fabrizio Ruggeri
National Research Council, Institute of Applied Mathematics and Information Technology, I-20133 Milan, Italy,
fabrizio@mi.imati.cnr.it

D ecision analytic methods are being increasingly used to help to articulate and structure debate and delib-
erations among citizens and stakeholders in societal decisions. Methods vary, but, essentially, a public
authority or agency, when faced with a significant set of issues, may organise one or more workshops with
stakeholders and citizens as participants. Such methods of public engagement and participation are, by and
large, conducted face to face. However, the advent of the World Wide Web brings the possibility of conducting
citizen and stakeholder interactions in a distributed, possibly asynchronous fashion. In this paper we discuss
the challenges that have to be addressed and overcome if such e-participation is to be a valid tool within a
modern democracy. The difficulties are many and varied, but the pressures towards e-government, and better
regulation in general, mean that such methods will be used in the near future. Thus, we outline a program of
research and debate in which we believe that the professional decision analysis community should engage.
Key words: decision analysis; decision conference; deliberative democracy; e-democracy; e-participation;
facilitation; negotiation analysis; public participation; stakeholder workshops
History: Received on September 25, 2007. Accepted by L. Robin Keller on October 23, 2007, after 1 revision.

1. Introduction automate standard democratic instruments. Typical

In this paper we discuss a range of issues that arise
when using Web-based interactions to support par-
ticipation and deliberative democracy, which we call
e-participation and e-democracy, respectively. Many
exploratory studies and experiments are already
underway: See the reviews in French (2003b, 2007a)
and Rios Insua (2007) that stem from the European
Science Foundation program Towards Electronic
Democracy: Internet Based Complex Decision Sup-
port (TED), which has been examining the potential
for decision analytic methodologies to provide frame-
works to support e-participation and e-democracy
(http://www.esf.org/ted or http://infodoc.escet.urjc.
es/ted). Drawing on the experience accumulated
from this program, we concentrate here on how we
might transform democratic processes using Web-
based decision and negotiation analysis to structure
and articulate participative deliberations. This is in
contrast with the bulk of research in the field, which
concentrates mainly on technologies to facilitate or
211
examples include technologies of e-voting, which
refer mainly to facilitating voting through electronic
means (Krimmer 2006), and technologies of e-delib-
eration, which facilitate debate of issues through the
Web; see Davies and Noveck (2007) for a review.
Browning (2002) provides a broad introduction to the
use of information and communication technologies
to support standard democratic instruments.
Recently, Gregory et al. (2005) have described the
use of decision analysis methodologies to structure
and facilitate public and stakeholder participation in
societal decision making and deliberative democracy
in a face-to-face manner. Similar ideas are offered
by, inter alia, Renn (1999), Sheppard and Meitner
(2005), or Winn and Keller (2001). Hämäläinen’s
group at Helsinki has been particularly active in
practical explorations; see, e.g., Mustajoki et al.
(2004) and many other references at http://www.
decisionarium.tkk.fi. Moreover, many applications of
decision analysis to societal issues have involved
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
212
stakeholder workshops or other similar participatory
instruments; see Keefer et al. (2004) or Renn et al.
(1995). The broad motivation for this work is that pub-
lic authorities, when faced with an important set of
issues, should draw together a range of stakehold-
ers in a workshop or a series of workshops and use
decision analysis to explore the issues and formulate
policies. Essentially, the ideas build on and broaden
the concept of a decision conference (Goodwin and
Wright 2003). This use of decision-analytic work-
shops should be seen as part of a much wider series
of moves towards public engagement and partici-
pation in many Western democracies (Beierle and
Cayford 2002, Rowe and Frewer 2005). Here we
explore the natural step forward of implementing
such approaches through the Web, pointing out many
important challenges and questions that need timely
answers from the decision analysis community, to
achieve valid and coherent e-participation processes
in the near future. In summary, we ask: How do we
implement a decision-analytic process over the Web
so that participants are able to input their views,
beliefs, and preferences, are informed by the process,
and are able to aid in developing a recommendation
on a joint choice to be made? Moreover, because of the
likely applications of these tools within modern mul-
ticultural societies, we ask these questions in a context
in which there might be many participants from var-
ied backgrounds and educational experience.
We are all aware of political imperatives towards
e-government and public participation that, together
with pressures from the information and com-
munications industry to sell their wares, mean that
e-participation and e-democracy are coming, and
coming fast, whether or not the processes involved
are meaningful and valid. Thus, we outline an urgent
program of research and debate within the decision
analysis community to identify valid mechanisms of
e-participation.
The paper is structured as follows. First we pro-
vide a brief review of the current state of the art
in the field of public participation and deliberative
democracy. Then, we turn to the subject of group deci-
sion making, reflecting on the implications of Arrow’s
Impossibility Theoremand related results for democ-
racy and participation. We argue that group decisions
are best seen as social processes and discuss how the
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
decision-analytic process is implemented within deci-
sion conferencing. In §4, we suggest what the infor-
mation and communication technologies architecture
of an e-democracy system might look like. We make
no claims that it is the only possibility, but we have
tried to take a comprehensive view of the generic
requirements of such a system. In §5, we develop and
explore our concerns related to the potential lack of
validity of e-participation systems and the research
that is needed, giving substance to the issues raised in
the context of our proposed architecture. We conclude
that there is need for a far-reaching research program
on the use of decision analysis and support within
e-participation.
2. Deliberative Democracy and
Public Participation
The Athenian ideal of deliberative democracy, some-
times called substantive or direct democracy, may
have never really existed. Women and slaves had no
right to vote, and many of the poorer men could not
leave their work to attend meetings (Crick 2002); it is
estimated that, in fact, only about 15% of the popula-
tion was involved. Nonetheless, to many it serves as
a model for societal decision making, in which all cit-
izens are able to input their views and have an influ-
ence on policy.
Across many Western democracies, disillusionment
with central government and regulatory agencies’
handling of issues, now termed the democratic deficit
(Steffek and Kissling 2007), has led to an increas-
ing use of consultation and stakeholder participation
in public decisions. Most politicians’ motivation to
promote this movement towards more participation
probably relates more to gaining greater public accep-
tance of the ultimate decision than to promoting
democratic ideals. However, whatever the cause,
there is undoubtedly wider use of participatory meth-
ods in societal decision making, from citizen juries to
deliberative polls, going through stakeholder work-
shops or participatory budgets, as reviewed in Rowe
and Frewer (2005). Several studies have covered
issues such as their philosophical underpinnings, cat-
egorisation of different types of activities, and case
studies (Beierle and Cayford 2002, Chess and Purcell
1999, Renn 1998, Rowe and Frewer 2005, Slovic 1993,
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
Webler 1999). There is also a relevant literature dis-
cussing public engagement with science and, more
generally, societal risk communication (Bennett and
Calman 1999, Berry 2004, Langford et al. 1999, Leach
et al. 2005). As we have noted, Gregory et al. (2005)
have reviewed the use of decision-analytic methods
within public participation. There are, however, two
areas in which the literature is very sparse. First,
although there have been many empirical investiga-
tions about the advantages and disadvantages of spe-
cific participatory instruments, there are few studies
comparing the relative merits of two or more instru-
ments; see Rowe et al. (2005) for an exception. Sec-
ondly, there is a relatively small literature on how to
design participatory processes appropriate to specific
contexts; see Bayley and French (2007) for a proposal.
Even terminology in the area has yet to become
standard. Indeed, some might say it is somewhat
confusing, with different terms being used by differ-
ent authors for the same entities and, conversely, the
same term being used for different entities by dif-
ferent authors (Rowe and Frewer 2005). We believe
it is important to distinguish a participatory process
from participatory instruments (or techniques). A par-
ticipatory process is the entire series of interactions
between authorities, stakeholders, and citizens from
the initial exploration of issues of concern, up to the
conclusion of the deliberations and resolution of the
matter, through to the making and implementation
of the decision. During the process, several participa-
tory instruments (e.g., stakeholder workshops or open
meetings) may be deployed to enable the participants
to interact. In the specific context of this paper, some
(or all) of the instruments would rely heavily on the
information and communication technologies archi-
tecture to support their operation.
Following Holtzman (1989), we consider a decision
process—and, hence, a participatory process—to be
broadly structured as in Figure 1. The first stage is to
formulate one or more decision models that reflect the
decision makers’ perceptions of the decision problem.
The next stage is to analyse the decision model(s), i.e.,
to explore what insights they bring about the possi-
ble choice and see where the balance of advantages
and disadvantages lies. The third stage is to decide
upon a policy to implement. This involves interpret-
ing the policies recommended by the analysis phase
213
Figure 1 The Phases of Decision Analysis: After Holtzman (1989)
Decision
Recommendations
model
Formulate Analyse Decide
Refine
into real-world actions. There will typically be a need
to assess whether the model(s) have brought enough
understanding to make the decision: i.e., is the analy-
sis requisite, as defined in Phillips (1984)? Thus, there
may be a feedback or refinement path, either after
the analysis or the decision step. A point, often over-
looked, is that the appropriateness of any instrument
will depend upon the phase in which it is deployed.
For instance, some argue against open Web discus-
sion forums because anyone can express a view, and
the instrument therefore fails any test of legitimacy
and fair representation. However, if the site is used in
the problem formulation phase to identify factors that
may have a bearing on the matter or generate poten-
tial alternatives for implementation, representation is
less important than the variety of issues raised.
Last, but not least, there are many variants of partic-
ipation. For simplicity, within our discussion we will
consider just two. In the first, the participation pro-
cess is part of a direct democracy and concludes with
a binding vote between all citizens on what should be
done. In the second, the authority and responsibility
for the decision remains firmly within a government
agency. They will make the decision. However, in the
process leading up to that point, they interact with cit-
izens and stakeholders. In the former case, there is a
clear imperative to ensure fair representation of all cit-
izens in the process and the highest levels of security.
In the latter case, these constraints may be softened
by the concerned agency, modifying the decision if
it feels that there has been systematic underrepresen-
tation of some groups. We recognise, however, that
any such pragmatic modification is hostage to many
questions of democratic ideals, trust, and legitimacy.
3. Group Decision Theory and
e-Participation
We are with Gregory et al. (2005) in believing that
decision-analytic methodologies provide an effective
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
214
and valuable means of articulating and structuring
deliberations within public participation. Moreover,
we agree that the schools of decision analysis stem-
ming from the subjective expected utility (SEU)
model, including multiattribute value and utility
models, provide the most coherent forms of decision
analysis with which to do this (Clemen and Reilly
2005, French and Rios Insua 2000, Keeney and Raiffa
1976). However, below we do note that proponents of
other schools of decision analysis might be effectively
disenfranchised if only SEU approaches are articu-
lated within a participatory process. For the present,
nonetheless, we maintain an SEU perspective.
Participation inevitably means that we are dis-
cussing a group decision context, and the paradoxes,
social traps, and impossibilities that abound in group
decision making and democratic systems are well
known (Bacharach 1975, French and Rios Insua 2000,
Hodge and Klima 2005, Raiffa et al. 2002). The fol-
lowing modes of group decision analysis have been
proposed as follows.
GDMGSEU : Essentially, this assumes that the subjec-
tive expected utility model applies at the group level,
and moreover that group probabilities and utilities are
constructed from those of its members. Thus, elicit
each group member’s subjective probabilities and
utilities. Combine the individual probabilities and
utilities into group probabilities and utilities, respec-
tively. Form the corresponding group expected utili-
ties and choose according to their ranking (Bacharach
1975, French 1985).
GDMvote : Work with each individual and develop
a personal decision analysis to guide their choice. In
the light of this understanding, each individual votes
within the group and a group choice is made accord-
ing to the vote. In variants of this, the numerical val-
ues of the individuals’ expected utilities are used to
indicate strength of preference, and this information
is incorporated into the voting (Rios and Rios Insua
2007).
GDMSupraDM : A supra decision maker is imagined to
exist. He observes the entire elicitation and decision
analysis process for each individual and altruistically
uses this knowledge to construct a single decision
analysis for the group. The choice is made according
to the supra decision maker’s analysis (Keeney and
Raiffa 1976).
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
GDMF ac : Gather the group together in a facilitated
discussion of issues. Through discussion between
participants, seek to agree on group probabilities
and utilities without formally eliciting individual
ones: i.e., seek to elicit group values directly with-
out any intermediate step for the individual mem-
bers. Develop a group analysis and explore areas of
disagreement via sensitivity and robustness analysis.
Seek to reach a decision by consensus without formal
voting (Eden and Radford 1990, French 2003a).
Over the years, the many paradoxes and impos-
sibilities stemming from Arrow’s theorem have led
most decision analysts to doubt the efficacy of
GDMGSEU and GDMvote . Although one can define algo-
rithms to move the numbers and votes around so
that, ultimately, a group ranking is mathematically
defined, an examination of the assumptions under-
pinning the algorithms finds inconsistencies. It should
also be noted that the generality of Arrow’s and sub-
sequent authors’ analyses means that the difficulties
apply not just to SEU approaches, but essentially to
all decision-analytic methodologies.
Approach GDMSupraDM is more promising at first
sight: all interpersonal comparisons are made within
the mind of the supra decision maker, and it is
the issue of defining valid interpersonal compar-
isons objectively that tends to cause the paradoxes
and inconsistencies leading to the problems with
GDMGSEU and GDMvote . In some cases, the supra deci-
sion maker actually exists: there may be an arbiter,
formally responsible for recommending a decision
that balances all stakeholder perspectives. Also, in
circumstances in which a government agency has
the legal responsibility and accountability for mak-
ing the decision, but does want to take into account
the views of citizens and stakeholders, the assump-
tion of the GDMSupraDM approach may become plausi-
ble: the supra decision maker is the agency. In these
circumstances, we can imagine e-participation sys-
tems in which individual citizens interact with a Web-
based decision support system, leaving their personal
preferences and beliefs for the agency to be drawn
together later into an analysis that will drive their
decision. However, in true democratic approaches to
e-participation, the nonexistence of the supra deci-
sion maker remains a problem: it is a fiction that
creates a fatal flaw in this approach. He has to be
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
constructed by agreement within the group, and this
leads to a further group decision, arguably as hard as
the first, and an infinite regress. Thus, most decision
analysts have looked to GDMFac , developing the use
of facilitated workshops or decision conferences in
which behavioural aggregation of individual perspec-
tives is driven by sensitivity analysis on a putative
model (Dryzek and List 2003, French 2003a, Phillips
1984). The approach of Gregory et al. (2005) to public
and stakeholder participation is largely driven by this
viewpoint.
Modes GDMGSEU , GDMvote , GDMSupraDM , and
GDMFac essentially assume that the group wants to
cooperate and reach a consensus. A further mode
recognises that individual citizens and stakeholders
may be more self-serving and wish to negotiate a
good end point for themselves.
GDMNeg : Bargaining, negotiation analysis, and
even, arbitration tools and methods are deployed to
define a process in which the group interacts and dis-
cusses a series of solutions, usually generated to con-
verge to a point on the Pareto boundary that corre-
sponds to a “deal” or “policy” that all participants
find acceptable (Raiffa et al. 2002).
Variants of GDMNeg can lean more to the algorith-
mic or more to a structuring of a softer facilitated
social process that can be run in workshops, in many
ways similar to the GDMFac approach.
We note briefly a number of issues that partici-
patory democracy brings to standard group decision
support. Some of them will be described in further
detail below. First, we have the issue of scalability:
The GDM modes were initially conceived for small
groups of participants, but not for the many thou-
sands that one could expect in a participatory process.
GDMGSEU and GDMvote would scale better to such con-
texts, provided that the analyses were supported by a
system. GDMSupraDM , GDMF ac , and GDMNeg could per-
haps be implemented in a hierarchical fashion, possi-
bly with the aid of several facilitators working with
different subgroups. Second, there is the issue of capa-
bility, because the GDM modes were designed for par-
ticipants who have some analytical inclination and
who are willing to express their judgements numeri-
cally. Note, however, that full analytical sophistication
is only expected of the facilitators and analysts sup-
porting the processes. The idea, therefore, would be to
215
provide user-friendly facilitation. A third issue refers
to the availability of time and the willingness to use it
for a democratic good, because there is a clear under-
lying assumption that citizens will contribute in sub-
stantial ways to the deliberations. This could perhaps
be solved through participation incentives, perhaps at
the risk of biasing the process, or through delegation
of participation to software agents if one assumes that
the latter will achieve sufficient sophistication in the
foreseeable future. We mention, finally, communica-
tion and coordination issues. A question, still largely
unanswered, is how decision analyses should be com-
municated to the general public. Moreover, no coor-
dination approaches are available for such potentially
large groups. In §5, we discuss these issues in greater
detail, but first let us give more substance to the dis-
cussion by considering a possible architecture for an
e-participation system.
4. A Decision-Analytic Based
Architecture for e-Participation
Support
What, then, might a decision-analytic based e-partic-
ipation system look like in terms of the architecture
of modules, databases, etc.? We offer one view here,
recognising that other researchers might offer a dif-
ferent vision, albeit to provide the same functional-
ity. However, we believe that our conception of the
architecture is sufficiently generic to illustrate many of
the points that we wish to make. Moreover, in purely
software-engineering terms, we believe that it could
essentially be constructed with today’s technologies.
Indeed, many components already exist.
The architecture that we sketch is designed to sup-
port decision-analytic based e-participation processes
that fit the GDMFac perspective, but pick up elements
of GDMvote , GDMSupraDM , and GDMNeg where they
make sense. It is generic because it seeks to accom-
modate various participatory instruments by appro-
priate choice of algorithms and definition of processes
within the modules. Different participatory processes
may be accommodated by removing or repeating
some of the steps, drawing on different instruments,
etc. Moreover, at some points different participants
might interact with different instruments to achieve
the same end. It would allow some participants to

216
delegate their participation in some interactions to
an agent, virtual or real. For example, some partici-
pants might interactively explore and perhaps mod-
ify a quantitative analysis, whereas others would just
want to discuss alternatives qualitatively in a discus-
sion forum. Some might contribute detailed judge-
ments through elicitation processes, whereas others
might be interested in delegating their input to a more
generic agent.
The architecture is illustrated in Figure 2 and articu-
lates a participatory process as suggested in Figure 1:
• Formulate. At this stage, the issues to be deliber-
ated would be formulated and the problem structured,
identifying uncertainties, alternatives, their interrela-
tions, constraints, criteria with which to evaluate con-
sequences, and so on. The participants would discuss,
consolidate, and evolve the basic structure, aided by
real or virtual facilitators to promote and enhance
creativity. This stage would lead to a family of deci-
sion models capable of spanning all perspectives and
allowing all views to be articulated within them.
• Analyse. At this stage, the participants’ judge-
ments of uncertainty and value would be elicited.
Through these, the participants could explore person-
Figure 2 A Generic IT Architecture for an e-Participation System
Log Main control
database subsystem
Security manager Kernel subsystem
subsystem
Problem
structuring
Decision-
Census module
making
database
database
Information
Debate manager
resource
manager module
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
alised quantitative analyses of the issues and draw
their own conclusion on what they believe is the way
forward. Moreover, they could see their own posi-
tion against a context of analyses conducted for other
participants, enhancing their understanding of each
other’s positions.
• Decide. This might be done by arbitration, negoti-
ation, or voting, or several possible combinations and
sequences thereof. As an example, we could propose
a process that involves a negotiation, and, if this fails,
undertake a voting session. If arbitration, voting, or
negotiation fails to find a mutually acceptable out-
come, then the process would have to iterate back
through earlier stages, refining the elements of the
model until a further stage finds an acceptable one.
There might also be a need to iterate or introduce
some further discussion and negotiation if the chosen
strategy fails to be Pareto optimal, as can sometimes
happen in some negotiation or voting processes.
The suggested architecture is illustrated in Fig-
ure 2 with a unified modelling language diagram that
reflects its key elements. The system would need elic-
itation interfaces focused on naïve users, maybe using
some of the ideas pioneered in MAUD (Humphreys
Interface
subsystem
Agent
repository
Arbitration Preference
Explanatory
manager modelling
module
module module
Participatory
process control
module
Negotiation
Voting manager
manager module
module module
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
and McFadden 1980). Indeed, a compromise might be
sought between ease of use and understanding and
too much sophistication in the methods deployed.
There might be a need to “lock” parts of the model
so that, say, exploration could be focused on the val-
ues of the participants with subjective probabilities
locked at values set by previous discussion. In other
words, there would need to be model management
systems such as those developed by Nunamaker et al.
(1988).
We now describe the key component databases
and modules, and point out elements already avail-
able that could be drawn on to implement this
architecture.
First, we identify three main databases:
• The census database would contain the data of
all users allowed to participate in the process, their
roles, permissions to take part at various stages, etc.
The system would check against this database when-
ever a participant seeks to undertake a given action
with the system.
• The log database would contain the complete
execution trace of every system interaction, so as to
be verified whenever a political party or an autho-
rized citizen demands it. Individual execution traces
of each subsystem should be constantly recorded here
for verification and validation purposes, as a way to
increase trust in and legitimacy of the system but, of
course, risking perceptions of the loss of anonymity.
• The decision-making database would contain all
data related to each decision-making process, such
as the internal state of the process (debating, negoti-
ating, voting, terminated, etc.); probabilities, utilities,
etc., of each participant; votes cast; messages submit-
ted to debates; alternatives posted; concessions made;
or offers accepted at negotiations, etc.
The architecture includes four main subsystems:
• Interface Subsystem. This part would be used
to interact with and between the users and, possi-
bly, their agents. To mitigate the “digital divide,” it
should be intuitive and easy to use. Depending on
the user sophistication level, we could employ differ-
ent types of interfaces, as required depending on the
stages and tools adopted in the participatory process
at hand. Here, developments in computer-supported
collaborative work (Coakes et al. 2002, Carstensen
and Schmidt 1999) and online debating (Davies and
217
Noveck 2007) are very relevant, as in, e.g., tools to
support distributed synchronous and asynchronous
collaborative writing over the Web. However, most
of this work assumes an already established, rela-
tively small and cohesive group, in contrast with our
context of large disperse groups. We include in rela-
tion with this subsystem an agent repository, which
would provide various types of software agents on
demand, which a participant might download to del-
egate aspects of his participation in some stages,
should he lack time to actually participate.
• Main Control Subsystem. This module would coor-
dinate computationally the rest of the subsystems. It
would receive user requests from the interface sub-
system, query the security manager subsystem to ask
whether the user has sufficient permissions to do it
and, if positive, call the appropriate module within
the kernel subsystem. Finally, when the results were
available, it would submit them to the interface sub-
system to show them to participants. In addition, this
subsystem would periodically obtain the individual
execution traces from the rest of the subsystems, and
record them in the log database. Should the system be
able to offer automatic facilitation, then this subsys-
tem, together with the participatory process control
module described below, would provide this.
• Security Manager Subsystem. Using the census and
decision-making databases, this would authenticate
whether a user has sufficient permissions to do some-
thing he or she has requested and grant the corre-
sponding execution permission. Standard work on
computer and network security (as in Stallings 2005),
and on secure Internet voting (see Krimmer 2006 for
a review), would be relevant. Note, however, that if
users provide their preference information in a utility
function, the information would require more com-
plex treatment than “simple” votes.
• Kernel Subsystem. This would be the system com-
ponent in which requests to vote, negotiate, debate,
structure a problem, communicate preferences, etc.
would be executed, without the need to worry about
security issues, handled by the security manager sub-
system. It would query and update the decision-
making database and provide support for individuals
and groups, as described below.
The kernel subsystem would be divided into sev-
eral modules. All of them would use the decision-
making database and be directly activated by the
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
218
main control subsystem. The main modules that
should be included are:
• Participatory Process Control Module. This mod-
ule would be involved in creating, specifying, and
structuring the participatory process, by defining the
stages and, for each of them, the tools, iterations,
time allowed, and so on. Once the participatory pro-
cess was specified, agreed upon, and scheduled, this
module would be in charge of controlling its correct
execution and its state flow, either manually through
the system administrator or other user requests, or
automatically through an algorithm. Depending on
the specific participatory process, it would have sev-
eral states, such as acquiring information, struc-
turing the problem, debating, negotiating, voting,
etc. In principle, this should support any decision-
making process, whatever the decision domain is
(budget elaboration, law elaboration, etc.). Model and
process management tools would be relevant here
(Nunamaker et al. 1988, Papamichail and Robertson
2005).
• Problem-Structuring Module. This would be used
to support issue formulation and problem structur-
ing (Rosenhead and Mingers 2001, Franco and Shaw
2006) and should allow for various problem struc-
tures from a simple list of alternatives, possibly with
constraints, to an influence diagram or decision tree
(Clemen and Reilly 2005). Formal structuring meth-
ods could be supported. The module might initially
be used by the problem owner and his analysts, but
in later stages it might be used by all participants
to explore and debate their perception of the prob-
lem, assess consequences, etc. There are some tools to
help in this, but many more are needed (Chen and
Lee 2003). Note that many of the soft methods used
in problem structuring also are extremely effective in
communicating issues (French et al. 2005). Thus, in
the consolidation and discussion phase, there might
also be a need to call this module to support debate.
• Preference-Modelling Module. This would be
used, if invoked, to model the participants’ pref-
erences to be used later to find the partici-
pants’ preferred alternatives and support conflict
resolution. It should facilitate various preference-
modelling modes, from value/utility functions as in
the decisionarium (http://www.decisionarium.hut.fi),
to goal-setting methods (http://bayes.escet.urjc.es/
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
refremov/rgmas/pp_gecd/index.html). Some Web-
based equivalent of the gmaa sensitivity analysis sys-
tem (http://www.dia.fi.upm.es/∼ajimenez/GMAA)
should also be devised. In the related area of consumer
preferences for purchases, Butler et al. (2008) describe
the facilitation of e-transactions using multiattribute
utility theory.
• Voting Manager Module. This would manage
all voting processes at the various stages of the
participatory process. It would implement a range of
voting protocols, including some requiring utility
and value functions to be submitted. For example, it
could handle approval voting from a list of candidate
options or majority voting on the current alterna-
tives in contention in a negotiation. Internet voting
has attracted a great deal of attention, and many
implementations of different rules already exist.
Some examples are Opinions-Online (www.opinions.
hut.fi/introduction.html); Vote-pro (http://www.
vote-pro.com/); 2ask (http://www.2ask.net/); the
VoteSecure Project (http://www.votesecure.org); and
KOA (http://sort.ucd.ie/projects/ucdkoa/).
• Negotiation Manager Module. This would manage
all negotiation processes taking place at various
stages of the participatory process. A range of nego-
tiation methodologies and algorithms from simple
(say, negotiation by posting) to sophisticated (say,
negotiation through the balanced increments method)
would be implemented; see Raiffa et al. (2002) for
a review. There is an important body of work on
generic e-negotiation systems (Benyoucef and Verrons
2007). Many examples of Web-based negotiation
systems stem from the Interneg project (http://www.
interneg.org). Other systems include IBM’s SilkRoad
(http://www.zurich.ibm.com/csc/ebizz/oldprojects/
silkroad.html) and Joint Gains (http://www.jointgains.
hut.fi/).
• Arbitration Manager Module. This would be
invoked if arbitration is selected as the conflict reso-
lution method. It should support various arbitration
algorithms (Thomson 1994) and be able to interact
with a real arbiter as well as to generate and sug-
gest arbitrated solutions automatically. Examples of
Web-based arbitration systems may be seen at http://
interarb.com/vl/p806396718.
• Debate Manager Module. This would be used
to handle, generate, structure, and compare user
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
opinions in all phases required by the participa-
tory process. For instance, in the structuring phase
it would support the development, comparison, and
modelling of different perspectives; in the negotia-
tion phase, it would handle discussion about posted
offers. Here there are many noteworthy contribu-
tions in the area of online debate tools, as may be
seen in Davies and Noveck (2007), from which we
could draw upon PHPbb, which is a freeware engine
to implement forums (http://www.phpbb.com/). An
important issue for which there is little work would
be to incorporate tools to detect clusters of users
with similar interests or opinions, to create coali-
tions. A description of such a system may be seen in
Lourenco and Costa (2007).
• Information Resource Manager Module. This would
save and manage information resources related to a
decision-making problem, such as Internet links, book
references, electronic documents, previous related
debates, sounds, e-learning material, etc. It would also
include computer-supported collaborative work tools,
for example, wikis, to share information cooperatively
among several participants, such as Carstensen and
Schmidt (1999). Debates generated in the debate man-
ager module could be related to the information man-
aged here. Therefore, the debate manager module
would use the information resource manager module.
• Explanatory Module. Because the majority of users
would not be able to interpret the output of the anal-
ysis and develop a qualitative understanding of the
factors driving the output, the system could benefit
from an automatic explanation module. Klein (1994),
Papamichail and French (2003), Bielza et al. (2003),
and Geldermann et al. (2007) provide illustrations.
These might further benefit from sensitivity analysis
and robustness tools as described in Rios Insua and
Ruggeri (2000).
We believe the majority of the technology, algo-
rithms, and tools needed to build such a system
exist. However, that does not mean it can be built
instantly—the tools must be interconnected and inte-
grated. At present, this would require a federation of
complex sites involving substantial programming and
technically complicated interfaces, whose use needs
to be transparent to end users. In particular, tests,
case studies, and training examples also need to be
219
compiled to facilitate wider use, together with train-
ing and software manuals aimed at a wide variety
of users of varying levels of sophistication. The total
complexity of the whole system would be compara-
ble to a software suite such as SAS. Even allowing for
the existence of many parts of the system in stand-
alone form, making them work together in a secure,
robust, coherent way would be a task requiring many
person-years of effort.
An example of a site with many potential com-
ponent modules is the decisionarium at Helsinki
University of Technology (Hämäläinen 2003, www.
decisionarium.tkk.fi). It is aimed at somewhat sophis-
ticated users of decision analysis tools. It is built on a
functional model, where the user presses a button to
initiate an operation or analysis, crucially lacking pro-
cess management tools to integrate the components
into a well-structured participatory process. Thus,
the user needs to know which buttons to press and
in what order. For less sophisticated users, a work-
book approach such as the one used by Mathemat-
ica, (Wolfram 1991), leading them through an analysis,
might be more appropriate.
Of course, rather than building a generic tool, we
could try to build specific algorithmic tools to solve
particular instances of specific problems. One exam-
ple is parbud (see Rios and Rios Insua 2007), aimed
at supporting participatory budgeting experiences.
5. Decision Analytic e-Participation:
Some Practical Issues
The previous section suggests that building a system
capable of supporting a range of participatory pro-
cesses securely and robustly is a major task, albeit one
that seems technically achievable. However, building
a system is only part of the task. If we are to create
a valid e-participation and e-democracy system, we
need to use it to implement and support the social
processes of GDMFac or GDMNeg . Although build-
ing the technology will be difficult, recognising and
addressing all the behavioural, cognitive, cultural,
legal, political, and psychological issues that must
be solved to establish valid e-participation will, we
believe, be much harder. Below, we catalogue some
of the most relevant issues that we have discovered—
and there will be more!—using the generic informa-
tion and communication technologies architecture to
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
220
structure and illustrate our points. Unless these points
are successfully addressed, the validity and meaning-
fulness of any e-participation system might be ques-
tionable. We formulate the issues through research
questions that could be addressed by the decision
analysis community within a relevant research pro-
gram, providing pointers to possible answers.
5.1. The Web vs. a Room
How do we extend our knowledge of how to support
group discussion of a decision analysis within face-
to-face environments to support asynchronous explo-
rations of quantitative decision models?
Meeting on the Web is not the same as meeting in
a room. Of course, videoconferencing (Schuman 2003)
or virtual environments—e.g., Second Life (http://
secondlife.com/)—may be used to recreate some ele-
ments of a room, but one of the key arguments
for e-participation is the possibility of asynchronous
usage of the Web. Indeed, this is the first and most
obvious difference between the Web and a room:
Interactions on the Web can be, and usually are, spa-
tially and temporally dispersed. There is much less
chronological ordering of interactions in Web meet-
ings, and many more people can “speak” at the same
time. To some extent, each participant can explore
the material in his or her own chosen order, whereas
in face-to-face meetings all participants hear and see
the same interactions in the same order. Building the
main control subsystem and the participatory process
control module to be sufficiently powerful, secure,
and robust to manage such asynchronous processes
for large numbers of participants may be a challenge
in itself. There are many sites that can manage hun-
dreds and thousands of two-way and group interac-
tions using e-mail, chat, and discussion forums. Such
technologies could support qualitative deliberations,
but we know of none that can handle a similar range
of asynchronous interactions in relation to, e.g., the
exploration of a quantitative decision model. Asyn-
chronous working in large groups also places a sub-
stantial burden on individual participants who truly
wish to deliberate and debate with others. The more
participants, the more time each needs to review inter-
actions and assimilate arguments, points, etc., made
by others. In the context of societal decision making,
it is conceivable that legal requirements may arise as
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
to the minimum time to be given to each stage so that
all participants have both a chance to interact among
themselves and to read the input of others.
Building upon current understanding of the social
processes in group decision support related to small
group synchronous interactions:
—How do such processes work in asynchronous
interactions in much larger groups?
—What methodologies are needed to support
them?
In a Web-meeting, almost everybody has a com-
puter in front of them, and therefore can conduct side
analyses of the issues or use the Web to gather fur-
ther information not available to all. In workshops,
it is unusual to let participants behave that inde-
pendently. The emphasis is on sharing information,
models, and ideas. There is a long and extensive lit-
erature on managing and facilitating such processes
in computer-supported collaborative work and group
decision support (Ackermann and De Veerde 2001,
Coakes et al. 2002, Morton et al. 2003). However, at
the very least, the methodologies would need modifi-
cation to cope with much larger groups. Thus, a key
issue, still to be studied in depth, is that there is no
prima facie reason to expect that the processes, pro-
cedures, or agendas developed for face-to-face par-
ticipation workshops and other implementations of
GDMFac will translate simply in a one-to-one fashion
into processes, procedures, or agendas for e-participa-
tion events.
5.2. The Diversity of the Population
How does one organise, structure, and facilitate
debate around decision models when some of the par-
ticipants share few values with others, or perhaps
even disagree on the underlying facts of the situation?
Decision conferences were developed to support
groups of decision makers faced with a common
set of concerns and issues. Such groups of decision
makers do not form randomly, but rather are already
working together because of some common interests
and objectives. For instance, members of a corporate
board of directors may represent many individual
perspectives, but they will also share many objec-
tives, such as ensuring profitability of the firm. Even
in stakeholder workshops run as part of a public
participation process, participants are not usually as
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
diverse as the whole of society. In our experience,
their willingness to attend such events, their selection
for participation by the authorities, and other factors
imply that they will share many objectives as well
as a willingness to reach consensus. All the GDM
models assume that there is a well-defined group. In
contrast, e-participation events will draw their par-
ticipants from across society and they may be self-
selecting, therefore being much less likely to have
common interests or even common understandings
of science, economics, etc. Consider an e-participation
event designed to discuss medical research fund-
ing. Animal rights activists are unlikely to share any
key objectives with representatives of pharmaceu-
tical companies. In discussions on flood defences,
there may be a minority who do not accept current
views on climate change. In short, the diversity of
the participants’ objectives is likely to be greater in
e-participation than in face-to-face workshops and the
difficulty in exploring decision models and converg-
ing to a consensus may be correspondingly larger.
How does one define interfaces that meet the
diverse skills and cultural understandings in society
in a fair and effective manner?
Societies are becoming more multiethnic and mul-
ticultural. This too has serious implications. Differ-
ent cultures have very different attitudes towards
decision making (Hofstede 1994, Wright and Phillips
1980). Such differences may occur in decision con-
ferences, but previous joint working experience often
softens the edges. Cultural backgrounds also affect
the understanding of words and images. Geograph-
ical information systems (GIS) interfaces are being
used in some explorations of e-participation (Carver
et al. 2001); however, Walsham and Sahay (1998) indi-
cate that even the cultural understanding of maps can
affect the usefulness of GIS-based decision support.
Offence is always possible between different cultures
by the unintentional choice of words. In face-to-face
encounters, body language can help anticipate and
recognise such issues as they happen; in a large Web
meeting there may be fewer safety nets. If the inter-
face subsystem offers different interfaces tailored to
the needs of different user groups, some more tech-
nically sophisticated than others, say, there is still a
question of whether all the interfaces fairly convey
221
the same information, and a need to verify whether
trust is to be engendered in the participants.
How does one deal with participants who reject the
validity of the decision methodology embodied in the
decision models?
Even within decision analysis, there is far from uni-
versal agreement on the validity of the SEU method-
ology. This is not merely an esoteric issue that fuels
academic debates, but may also relate to cultural dif-
ferences. For instance, the differences between the
French outranking school and the primarily Anglo-
American school based on SEU arises, in part, because
of subtle cultural differences in understanding what
a decision is (process versus point of choice) and,
arguably, in the understanding of intentionality (Roy
1996, Belton and Stewart 2002). Thus, structuring
e-participation around a particular decision analy-
sis methodology may effectively disenfranchise those
who find it neither transparent nor, perhaps, ratio-
nal. Therefore, we believe that an important research
issue within this area is to analyze how we may aid in
their decision processes to large diverse groups, pos-
sibly supported by heterogeneous decision-analytic
methodologies. Note that this is acknowledged within
our architecture, because we allow various modelling
approaches within the preference modelling module.
5.3. Communication and Understanding
How does one construct effective communication,
possibly automatically within the interface subsystem?
French (2007b) has argued that a key to making the
SEU paradigm a methodology for decision support in
large groups dispersed over the Web is the develop-
ment of better means of communicating the reasoning
and import of an analysis. Similar arguments may be
made for other decision paradigms. Much of the work
on public engagement has identified the importance
of addressing communication issues, particularly with
respect to risk (Bennett and Calman 1999, Fischhoff
1995, Gigerenzer 2002). Despite an enormous research
effort, much more is understood about the problems
of communication than is known about how to con-
struct effective communication (Maule 2004). True
communication, in the sense of building shared men-
tal models, is difficult even when it is performed face
to face with all sorts of cues passing between listeners
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
222
and speakers. On the printed page, in a media inter-
view, or through a Web-based discussion, it seems
even more difficult.
First, in spite of the work that has gone on over
many years about effective presentation of data and
analyses, see, e.g., Tufte (1997), practice is still far from
perfect. Within e-participation, data presentation will
need to be much better. The Web is essentially graph-
ical, yet one only has to look at a few Websites to
realise how poorly we may use its power. We need to
improve, and not only in terms of data presentation.
Good decision analyses provide understanding, not
simply optimal alternatives (French and Rios Insua
2000, Phillips 1984). While most decision analysts are
adept at explaining an analysis and what it is saying
either at a meeting or in a report, their audiences have
typically been well educated. How decision analyses
should be communicated to the general public is still
largely an unanswered question.
Secondly, words are not used just with their dictio-
nary definitions. They have local and cultural mean-
ings that some participants may not understand. For
instance, while conducting an environmental deci-
sion conference, one of us (SF) had difficulty eliciting
value judgements from a representative of an envi-
ronmental group until he discovered that this person
objected to using the word “scoring” to refer to elici-
tation, because of connotations of success (scoring in a
game) when the environmental impacts being consid-
ered were usually detrimental. In face-to-face discus-
sions, particularly facilitated discussions, the meaning
of words is negotiated. How that is achieved on the
Web is far from clear, particularly when the partici-
pants are drawn from a wide, politically diverse, mul-
ticultural society.
5.4. Facilitation
How does one facilitate large-scale, asynchronous
Web deliberations, both in qualitative discussion and
in exploring and learning from one or more decision
models?
In a very real sense, many of the points made
in §§5.2 and 5.3 relate to the need for facilitators
in a decision conference (Ackermann 1996, Macauley
and Alabdulkarim 2005, Phillips and Phillips 1993).
A facilitator is someone with no responsibility or
accountability for the consequences of the decision
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
per se, who joins the group to structure, smooth,
and enhance the deliberative processes. He should
be skilled in decision-making tools, group dynam-
ics, psychology, cultural issues, and communication,
and he would use this knowledge to support a group
in their work. Facilitation has been fairly well stud-
ied in face-to-face contexts, but is less well under-
stood in Web-based meetings (Niederman et al. 1996,
Macauley and Alabdulkarim 2005). The facilitator’s
art relates to his or her skill in selecting an effec-
tive intervention to move the group’s work on in
a productive manner and draw on content arising
from the group. Shaw (2004) provides one example
of e-facilitation in problem structuring. Asynchronous
involvement of thousands of participants is likely to
be particularly challenging. For instance, one cannot
just watch the effect of a participant’s statements as
they are made, but must maintain a watch for any
effect in the hours and days following.
Apart from the lack of physical cues to help the
facilitator in Web meetings, there is a problem of scale.
Decision conferences and stakeholder workshops typ-
ically have 15 to 50 participants; 100 or more partici-
pants would be exceptional. However, e-participation
could involve thousands of participants. This is sig-
nificant because the potential for misunderstandings
in participant interactions rises as some power of their
number. Moreover, it is unlikely that there would be
funding to maintain a ratio of facilitators to partici-
pants in the order of 1 to 30 or so. Inevitably, one will
probably be working with teams of facilitators, which
brings its own issues of coordination and coherence
of approach.
One possibility is to develop artificial intelligence
or agent technologies to facilitate the process auto-
matically. However, even recent work in collaboration
engineering on describing interventions as thinklets
(see Kolfschoten et al. 2006) has not addressed the
sophistication of facilitation needed for issues as com-
plex as found in societal decisions. Moreover, facili-
tation requires much tacit knowledge that cannot be
easily articulated in software (Polyani 1962, Walsham
2001). Therefore, to develop automated facilitation
and encode it within the interface and control subsys-
tems would require both a major research program
and a substantial software-engineering task.
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
5.5. Legitimacy and Trust
How does one engender trust in e-participation sys-
tems, including
—in the process,
—in the software,
—in the facilitation and analyst team, and
—in the methodology and its simplifications?
Let us suppose that all the issues above are success-
fully addressed and one can create an e-participation
process to the satisfaction of the facilitators and ana-
lysts. Would the public trust it? Would they perceive
it as legitimate? There are several levels of trust that
we should consider. Would they believe that all the
interactions on the Website are genuine? Would they
accept the analyses on the Website as reflecting a set
of beliefs and preferences or would they perceive the
system as distorting or oversimplifying their views?
In a workshop, participants see and hear all the inter-
actions and can count shows of hands themselves;
what would give them the same assurance over the
Web? Any democratic system is subject to some risk
of manipulability (Gibbard 1973, Hodge and Klima
2005). Will citizens trust other citizens to reveal their
preferences honestly in the manner that a decision
analytic approach would assume? Face-to-face inter-
action provides many cues to support trust; the Web
does not provide as many.
There is also a trust issue in the sense of trust-
ing the system to implement an algorithm correctly.
Even in the relatively simple area of counting votes,
debates in the literature of e-voting suggest that guar-
anteeing that the system transparently adds up the
votes correctly is a nontrivial task. How much greater
is the challenge with systems that solve influence
diagrams with complex probability and utility struc-
tures? What quality control is needed to ensure that
the results are trusted? Would the code need to be
published, even when the models rely on copyrighted
or patented decision-analytic software? Open source
code may well be needed, but it takes time and effort
to write and assure quality. In the previous section, we
noted that the scale of an e-participation system is as
great and as complex as some of the largest software
systems developed to date with many person-years of
effort.
Would the facilitation team be trusted by citizens
and politicians? They need to steer the deliberation
223
in some neutral way, discarding irrelevant details
to ensure that the decision analysis is tractable. To
a great extent, they fix the deliberation rules, but
that leaves potential for mistrust. Politicians would
also have to accept that such systems could lead to
changes in political power structures. That may be
a step too far for some. However, like citizens, they
have to trust the processes and systems.
How does one ensure that e-participation processes
are legitimate and perceived to be so?
There is also the issue of legitimacy. Even if society
trusts an e-participation process to reflect the delib-
erations of the participants, it does not necessarily
follow that it will see it as a legitimate reflection of
the whole of society. This relates to issues of the dig-
ital divide (Browning 2002): Does every citizen have
equal access to the Web? We should note, however,
that such issues exist for any participation process.
Many face-to-face interaction processes disadvantage
rural communities because they are generally held
in cities. They also impose costs such as the need
to get time off from work, or travel costs. Therefore,
any participatory process imposes costs and requires
overcoming barriers. There are also issues relating to
“fair” representation and the potential for a pressure
group to hijack an e-participation process. Even if the
e-participation is used to inform an agency’s deci-
sion rather than to determine the decision itself, the
agency must assess whether the deliberations reflect
the views across society. If the agency gives incentives
to promote active engagement, will these further dis-
tort the representativeness of the participants? This
is particularly important because participatory pro-
cesses may shift the focus of deliberation from issues
of science to issues of values: in itself, a move for
the better (Fischhoff 1995), but a move that places
further demands on the representativeness of those
participating.
We partly acknowledge the above problems by
including the log database and the security manager
subsystem.
5.6. Designing the Process Itself
How does one design a participation process that is
either entirely Web based or blends Web and face-to-
face interactions?
We noted earlier the paucity of advice on the
design of participatory processes. This remark applies
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
224
even more to the design of e-participation processes.
Moreover, we have been writing as if a participa-
tion process would be Web based. It is likely, how-
ever, to be a hybrid or blended process arranging a
number of Web-based and face-to-face participation
instruments throughout the deliberation process indi-
cated in Figure 1. We have noted that there have
been few comparative studies on the effectiveness
of different instruments. There have been virtually
no studies investigating interaction effects between
different participation instruments. Does running a
Web discussion before a stakeholder workshop elicit
a broader, more embracing range of issues to be con-
sidered in the deliberations? Once such studies have
been undertaken, how does one use the knowledge
obtained to design an effective process? Bayley and
French (2007) have discussed the issue, offering a
resource allocation model to help structure the design.
Once we have designed the process, we would define
it through the participatory process control module.
6. Conclusions and Recommendations
We are concerned that political and commercial
imperatives towards the adoption of e-participation
might lead to their use before we understand what
they are actually achieving. Thus, our aim here has
been to bring a number of research questions to the
attention of the decision analysis community. These
questions should be addressed before the validity
and legitimacy of decision analytic e-participation
can be assessed. There is a need to develop tools
and methodologies: to explore, formulate, and model
issues; to allow unsophisticated users to conduct, or
at least explore, decision analyses; to discover and
build consensus. We need to understand how to facil-
itate large, diverse, dispersed, multicultural groups,
many of whom may share few common values, may
be interested in different decision methodologies, and
may prefer nonquantitative methods. Without such
developments and the answers to many lesser ques-
tions raised above—and, doubtless, many more ques-
tions that we have missed, it is unlikely that any
deliberative e-democracy or e-participation process
can be deemed valid and worthy of trust. It behooves
the decision analysis community to lead the debate
and conduct the research to address these questions,
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
so that society will use e-participation wisely, possibly
through the type of architecture we have proposed.
Acknowledgments
Much of the thinking in this paper grew from workshops
and discussions within the tedproject: The authors are
grateful to the European Science Foundation for funding
this, and to their many friends and colleagues who took
part. Other relevant funding has been provided by the
UK Research Councils, the Spanish Ministry of Science
and Education, and the government of Madrid, through
the e-Democracia-CM program. Comments on earlier drafts
from Greg Kersten, the editors of Decision Analysis, and
three anonymous referees have been particularly helpful.
The latest version of the paper was prepared while the
second and third authors were visiting the Statistical and
Applied Mathematical Sciences Institute, NC.
References
Ackermann, F. 1996. Participants’ perceptions on the role of facil-
itators using group decision support systems. Group Decision
Negotiation 5 93–112.
Ackermann, F., G.-J. De Veerde, eds. 2001. Special issue: European
research on group decision support systems. Group Decision
Negotiation 10 1–94.
Bacharach, M. 1975. Group decisions in the face of differences of
opinion. Management Sci. 22 182–191.
Bayley, C., S. French. 2007. Designing a participatory process for
stakeholder involvement in a societal decision. Group Decision
Negotiation. Forthcoming.
Beierle, T., J. Cayford. 2002. Democracy in practice: Public partic-
ipation in environmental decisions. Resources for the Future,
Washington, D.C.
Belton, V., T. J. Stewart. 2002. Multiple Criteria Decision Analysis: An
Integrated Approach. Kluwer Academic Press, Boston.
Bennett, P. G., K. C. Calman, eds. 1999. Risk Communication and
Public Health: Policy Science and Participation. Oxford University
Press, Oxford.
Benyoucef, M., M. H. Verrons. 2007. Configurable e-negotiation sys-
tems for large scale and transparent decision making. Group
Decision Negotiation. Forthcoming.
Berry, D. C. 2004. Risk Communication and Health Psychology. Open
University Press, Maidenhead UK.
Bielza, C., J. A. Fernández del Pozo, P. Lucas. 2003. Finding and
explaining optimal treatments. Lecture Notes in Computer Sci-
ence, Vol. 2780. Springer Verlag, Berlin, 299–303.
Browning, G. 2002. Electronic Democracy: Using the Internet to Trans-
form American Politics. Information Today, Medford NJ.
Butler, J. C., J. S. Dyer, J. Jia, K. Tomak. 2008. Enabling e-transac-
tions with multiattribute utility theory. Eur. J. Oper. Res. 186(2)
Forthcoming.
Carstensen, P., K. Schmidt. 1999. Computer supported collaborative
work: New challenges to system design. K. Itoh, ed. Handbook
of Human Factors/Ergonomics. Asakura, Tokyo, 619–636.
Carver, S., A. Evans, R. Kingston, I. Turton. 2001. Public participa-
tion, GIS, and cyberdemocracy: Evaluating online spatial deci-
sion support systems. Environment Planning B: Planning Design
28 907–921.
French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
Chen, J. Q., S. M. Lee. 2003. An exploratory cognitive DSS for strate-
gic decision making. Decision Support Systems 36 147–160.
Chess, C., K. Purcell. 1999. Public participation and the environ-
ment: Do we know what works? Environ. Sci. Tech. 33(16)
2685–2692.
Clemen, R. T., T. Reilly. 2005. Making Hard Decisions. Thomson
Learning, Duxbury, Pacific Grove, CA.
Coakes, E., D. Willis, S. Clarke, eds. 2002. Knowledge Management
in the SocioTechnical World. Computer Supported Co-Operative
Work, Springer Verlag, London.
Crick, B. 2002. Democracy: A Very Short Introduction. Oxford Univer-
sity Press, Oxford.
Davies, T., B. S. Noveck. 2007. Online Deliberation: Design, Research,
and Practice. CSLI Publications/University of Chicago Press.
Dryzek, J. S., C. List. 2003. Social choice theory and deliberative
democracy: A reconciliation. British J. Political Sci. 33 1–28.
Eden, C., J. Radford, eds. 1990. Tackling Strategic Problems: The Role
of Group Decision Support. Sage, London.
Fischhoff, B. 1995. Risk perception and communication unplugged:
Twenty years of process. Risk Anal. 15 137–145.
Franco, A., D. Shaw, eds. 2006. Special issue on: Problem structur-
ing methods. J. Oper. Res. Soc. 57 757–878.
French, S. 1985. Group consensus probability distributions: A crit-
ical survey. J. M. Bernardo, M. H. DeGroot, D. V. Lindley,
A. F. M. Smith, eds. Bayesian Statistics 2. North-Holland, Ams-
terdam, 183–201.
French, S. 2003a. Modelling, making inferences and making deci-
sions: The roles of sensitivity analysis. TOP 11 229–252.
French, S., ed. 2003b. Special issue: The challenges in extending
the MCDA paradigm to e-democracy. J. Multi-Criteria Decision
Anal. 12 61–233.
French, S., ed. 2007a. Special issue: Interface issues in e-democracy.
Internat. J. Tech. Policy Management 7 105–208.
French, S. 2007b. Web-enabled strategic GDSS, e-democracy and
Arrow’s theorem: A Bayesian perspective. Decision Support Sys-
tems 43 1476–1484.
French, S., D. Rios Insua. 2000. Statistical Decision Theory. Arnold,
London.
French, S., A. J. Maule, G. Mythen. 2005. Soft modelling in risk
communication and management: Examples in handling food
risk. J. Oper. Res. Soc. 56 879–888.
Geldermann, J., V. Bertsch, M. Treitz, S. French, K. N. Papamichail,
R. P. Hämäläinen. 2007. Multi-criteria decision support and
evaluation of strategies for nuclear remediation management.
OMEGA—The Internat. J. Management Sci. Forthcoming.
Gibbard, A. 1973. Manipulation of voting schemes: A general result.
Econometrica 41 587–601.
Gigerenzer, G. 2002. Reckoning with Risk: Learning to Live with Uncer-
tainty. Penguin Books, Harmondsworth, UK.
Goodwin, P., G. Wright. 2003. Decision Analysis for Management
Judgement. John Wiley and Sons, Chichester, UK.
Gregory, R. S., B. Fischhoff, T. McDaniels. 2005. Acceptable input:
Using decision analysis to guide public policy deliberations.
Decision Anal. 2 4–16.
Hämäläinen, R. P. 2003. Decisionarium—Aiding decisions, negoti-
ating and collecting opinions. J. Multi-Criteria Decision Anal. 12
101–110.
Hodge, J. K., R. E. Klima. 2005. The Mathematics of Voting and Elec-
tions: A Hands-On Approach. American Mathematical Society,
Providence, RI.
225
Hofstede, G. 1994. Management scientists are human. Management
Sci. 40 4–13.
Holtzman, S. 1989. Intelligent Decision Systems. Addison-Welsey,
Reading, MA.
Humphreys, P. C., W. McFadden. 1980. Experiences with MAUD:
Aiding decision structuring versus bootstrapping the decision
maker. Acta Psychologica 45 51–69.
Keefer, D. L., C. W. Kirkwood, J. L. Corner. 2004. Perspective on
decision analysis applications, 1990–2001 with discussion. Deci-
sion Anal. 1 4–34.
Keeney, R. L., H. Raiffa. 1976. Decisions with Multiple Objectives: Pref-
erences and Value Trade-Offs. John Wiley and Sons, New York.
Klein, D. A. 1994. Decision-Analytic Intelligent Systems: Automated
Explanation and Knowledge Acquisition. Lawrence Erlbaum Asso-
ciates, Hillsdale, NJ.
Kolfschoten, G. L., R. O. Briggs, G. J. de Vreede, P. M. Jacobs,
J. H. Appelman. 2006. A conceptual foundation of the thinkLet
concept for collaboration engineering. Internat. J. Man-Machine
Stud. 647 611–621.
Krimmer, R. 2006. Electronic voting 2006. 2nd International Work-
shop, GI Lecture Notes in Informatics. Castle Hofen, Brĕgenz,
Austria.
Langford, I., C. Marris, T. O’Riordan. 1999. Public reactions to
risk: Social structures, images of science and the role of trust.
P. G. Bennett, K. C. Calman, eds. Risk Communication and Public
Health: Policy, Science and Participation. Oxford University Press,
Oxford, 33–50.
Leach, M., I. Scoones, B. Wynne, eds. 2005. Science and Citizens. Zed
Books, London.
Lourenco, R., J. P. Costa. 2007. Incorporating citizens’ views in local
policy decision making. Decision Support Systems 43 1499–1511.
Macauley, L., A. Alabdulkarim. 2005. Facilitation of e-meetings:
State-of-the-art review. IEEE Internat. Conf. e-Technology, e-Com-
merce and e-Service, Hong Kong.
Maule, A. J. 2004. Translating risk management knowledge: The
lessons to be learned from research on the perception and com-
munication of risk. Risk Management 6 15–27.
Morton, A., F. Ackermann, V. Belton. 2003. Technology-driven and
model-driven approaches to group decision support: Focus,
research philosophy, and key concepts. Eur. J. Inform. Systems
12 110–126.
Mustajoki, J., R. P. Hämäläinen, M. Marttunen. 2004. Participatory
multi-criteria decision analysis with Web-Hipre: A case of lake
regulation policy. Environ. Model. Software 19 537–547.
Niederman, F., C. M. Beise, P. M. Beranek. 1996. Issues and concerns
about computer-supported meetings: The facilitator’s perspec-
tive. MIS Quart. 20 1–22.
Nunamaker, J. F., L. M. Applegate, B. R. Konsynski. 1988.
Computer-aided deliberation: Model management and group
decision support. Oper. Res. 36 826–848.
Papamichail, K. N., S. French. 2003. Explaining and justifying the
advice of a decision support system: A natural language gen-
eration approach. Expert Systems Appl. 24 35–48.
Papamichail, K. N., I. Robertson. 2005. Integrating decision making
and regulation in the management control process. Omega 33
319–332.
Phillips, L. D. 1984. A theory of requisite decision models. Acta
Psychologica 56 29–48.
Phillips, L. D., M. C. Phillips. 1993. Facilitated work groups—The-
ory and practice. J. Oper. Res. Soc. 44 533–549.
Polyani, M. 1962. Personal Knowledge. Anchor Day Books, New York.
 French, Rios Insua, and Ruggeri: e-Participation and Decision Analysis
226
Raiffa, H., J. Richardson, D. Metcalfe. 2002. Negotiation Analysis: The
Science and Art of Collaborative Decision Making. Harvard Uni-
versity Press, Cambridge, MA.
Renn, O. 1998. The role of risk communication and public dialogue
for improving risk management. Risk, Decision Policy 3 3–50.
Renn, O. 1999. A model for an analytic-deliberative process in risk
management. Environ. Sci. Tech. 33 3049–3055.
Renn, O., T. Webler, P. Wiedermann, eds. 1995. Fairness and Compe-
tence in Citizen Participation: Evaluating Models and Environmen-
tal Discourse. Kluwer, Dordrecht, Holland.
Rios, J., D. Rios Insua. 2007. A framework for participatory budget
elaboration support. J. Oper. Res. Soc. Forthcoming.
Rios Insua, D. 2007. Special issue: eDemocracy. Group Decision Nego-
tiation. Forthcoming.
Rios Insua, D., F. Ruggeri. 2000. Robust Bayesian Analysis. Springer,
New York,
Rosenhead, J., J. Mingers, eds. 2001. Rational Analysis for a Problem-
atic World Revisited. John Wiley and Sons, Chichester, UK.
Rowe, G., L. J. Frewer. 2005. A typology of public engagement
methods. Sci. Tech. Human Values 30 251–290.
Rowe, G., T. Horlick-Jones, J. Walls, N. Pidgeon. 2005. Difficulties
in evaluating public engagement initiatives: Reflections on an
evaluation of the UK GM Nation? Public debate. Public Under-
standing Sci. 14 331–352.
Roy, B. 1996. Multi-Criteria Modelling for Decision Aiding. Kluwer
Academic Publishers, Dordrecht, Germany.
Schuman, S. 2003. Information technology for groups. Group Facili-
tation (Spring). http://findarticles.com/p/articles/mi_qa3954/
is_200304.
Shaw, D. 2004. Creativity and learning through electronic group
causal mapping. Internat. J. Innovation Learn. 14 364–377.
Decision Analysis 4(4), pp. 211–226, © 2007 INFORMS
Sheppard, S. R. J., M. Meitner. 2005. Using multi-criteria analy-
sis and visualisation for sustainable forest management plan-
ning with stakeholder groups. Forest Ecology Management 207
171–187.
Slovic, P. 1993. Perceived risk, trust and democracy. Risk Anal. 13
675–682.
Stallings, W. 2005. Cryptography and Network Security. Prentice Hall,
Englewood Cliffs, NJ.
Steffek, J., C. Kissling. 2007. Civil Society Participation in European and
Global Governance: A Cure for the Democratic Deficit? Palgrave
MacMillan, New York.
Thomson, W. 1994. Cooperative models of bargaining. R. Aumann,
S. Hart, eds. Handbook of Game Theory, Vol. 2. North Holland,
Amsterdam, 1238–1277.
Tufte, E. R. 1997. Visual Explanations. Graphics Press, CT.
Walsham, G. 2001. Knowledge management: The benefits and lim-
itations of computer systems. Eur. Management J. 19 599–608.
Walsham, G., S. Sahay. 1998. GIS for district-level administration in
India: Problems and opportunities. Management Inform. Systems
Quart. 23 39–66.
Webler, T. 1999. The craft and theory of public participation:
A dialectical process. J. Risk Res. 2 55–71.
Winn, M. I., L. R. Keller. 2001. A modelling methodology for
multi-objective multi-stakeholder decisions: Implications for
research. J. Management Inquiry 102 166–181.
Wolfram, S. 1991. Mathematica: A System for Doing Mathematics by
Computer. Addison Wesley, Redwood City, CA.
Wright, G., L. D. Phillips. 1980. Cultural variation in probabilistic
thinking: Alternative ways of dealing with uncertainty. Inter-
nat. J. Psych. 15 239–257.