Environmental Modelling & Software 21 (2006) 1695e1710 www.elsevier.

com/locate/envsoft

Distance-based and stochastic uncertainty analysis for multi-criteria decision analysis in Excel using Visual Basic for Applications
K.M. Hyde*, H.R. Maier
Centre for Applied Modelling in Water Engineering, School of Civil & Environmental Engineering, The University of Adelaide, Adelaide, SA 5005, Australia Received 6 January 2005; received in revised form 26 July 2005; accepted 5 August 2005 Available online 21 October 2005

Abstract A program has been developed in Excel and written in Visual Basic for Applications, which enables a decision maker to examine the robustness of a solution obtained when using multi-criteria decision analysis (MCDA). The distance-based and stochastic uncertainty analysis approaches contained in the program allow a decision to be made with condence that the alternative chosen is the best performing alternative under the range of probable circumstances. The uncertainty analysis methodology overcomes the limitations of existing sensitivity analysis techniques for MCDA by enabling all of the input parameters to be varied simultaneously within their expected ranges. The Weighted Sum Method (WSM) and PROMETHEE are the MCDA techniques available for the user to select in the program. The program is illustrated by applying it to a sustainable water resource development problem in the Northern Adelaide Plains, South Australia. 2005 Elsevier Ltd. All rights reserved.
Keywords: Multi-criteria decision analysis; Visual Basic; Uncertainty analysis; Excel; Water resources

Software availability Developers: K.M. Hyde, H.R. Maier Minimum hardware requirements: Intel Pentium II 1.1 GHz, 512 MB RAM Software requirements: Operating system Win XP, Microsoft Excel 2000, @Risk Professional Edition v4.5.2 or above Language: English Size: 15 MB Availability: From the authors upon request for research purposes only

1. Introduction Multi-criteria decision analysis (MCDA) has been utilised to assist in making complex decisions for a number of decades, as it facilitates stakeholder participation and collaborative decision-making, does not necessarily require the assignment of monetary values to environmental or social criteria, and allows the consideration of multiple criteria in incommensurable units (i.e. combination of qualitative and quantitative criteria). The inherent uncertainties and subjectivities of the input parameters to an MCDA model (i.e. criteria weights (CWs) and criteria performance values (PVs)) have been found to have an inuence on the rankings of alternatives (Roy and Vincke, 1981). Despite this, the impact of the variability of the input parameters has been largely overlooked in studies in which MCDA has been applied (see for example Duckstein et al., 1994; Flug et al., 2000; Ulvila and Seaver,1982). The effective incorporation, management and

* Corresponding author. Tel.: C61 8 8303 5033; fax: C61 8 8303 4359. E-mail addresses: khyde@civeng.adelaide.edu.au (K.M. Hyde), hmaier@ civeng.adelaide.edu.au (H.R. Maier). 1364-8152/$ - see front matter 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.envsoft.2005.08.004

1696

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

understanding of uncertainty in the input parameters remain the most fundamental problems in MCDA (Felli and Hazen, 1998). Numerous sensitivity analysis methods designed to quantify the impact of parametric variation on MCDA model output have been proposed in the literature, however, these are mainly focused on the assessment and inuence of the CWs (Barron and Schmidt, 1988; Rios Insua and French, 1991; Wolters and Mareschal, 1995). Few sensitivity analysis methods have been developed to assess the impact of PVs on the ranking of alternatives (Triantaphyllou and Sanchez, 1997) and therefore, the impact of the uncertainty and variability in the criteria PVs is frequently disregarded. The commonly used sensitivity analysis methods are also limited, not only because they are focused on one type of input parameter, but also because predominantly only one parameter is varied at a time, which is inadequate, as it may be the case that the ranking of the alternatives is insensitive to the variations of some parameters in a set individually, but sensitive to their simultaneous variation. In addition, the frequently applied sensitivity analysis methods are generally only applicable to certain MCDA techniques. Distance-based and stochastic uncertainty analysis approaches have been proposed by Hyde et al. (in press, 2004, 2003), Hyde and Maier (submitted for publication), respectively, which overcome the aforementioned limitations of the commonly used sensitivity analysis approaches for MCDA, as both of the proposed uncertainty analysis methods determine how sensitive the ranking of alternatives is to the simultaneous variation of all of the input parameters over their expected range. There have been numerous specially developed computer packages that support the application of MCDA methods e.g. DEFINITE (Janssen, 1996), MULINO-DSS (Guipponi et al., 2003; Mysiak et al., 2005), ASSESS (Hill et al., 2005), VIP (Dias and Climaco, 2000), TOPSIS (Deng et al., 2000), MACBETH (Bana e Costa and Vansnick, 1999), and DECISION LAB (Geldermann and Zhang, 2001). In addition, Podinovski (1999) developed a decision support system (DSS) which helps the user to identify candidate solutions from a nite set of decision alternatives when only the upper and lower bounds of the CWs are specied. Most of these programs, however, are limited in that they only utilise one MCDA technique and the sensitivity analysis (if included) only takes one form of uncertainty into consideration. This paper presents and describes a program that has been developed in Microsoft Excel that incorporates the ability to undertake deterministic MCDA, the distance-based uncertainty analysis approach of Hyde and Maier (submitted for publication) and the stochastic uncertainty analysis approach of Hyde et al. (2004). The choice of uncertainty analysis method may depend on the amount of data available and the output required by the decision maker (DM). The program is illustrated by applying it to a water resource management decision problem which was assessed using MCDA by Fleming (1999). 2. Program description The program is written in Visual Basic for Applications (VBA), which is the programming language incorporated in

Microsoft Excel. The advantage of using Microsoft Excel as a development environment is that it provides capabilities that allow for analysis and manipulation of the data and the visualisation of the results. In addition, Microsoft Excel is familiar, not to mention readily available, to a large majority of people. Consequently, using the program does not necessitate becoming familiar with a new software environment. Help les are included throughout the program, which provide theoretical information on the analysis that is implemented, and information on how to use the program itself. The structure, methodology and use of the program are illustrated in Fig. 1 and described below. 2.1. Decision analysis formulation 2.1.1. Program initialisation The rst stage of any MCDA approach involves the translation of the decision analysis situation into a set of alternatives and criteria, which are generally developed by the actors involved in the decision-making process. The number of alternatives, criteria and actors, which dene the decision analysis problem being undertaken, are entered on the Program Initialisation form, which is shown in Fig. 2. The program is restricted to assessing decision problems with a maximum number of 30 alternatives and 24 criteria. The program currently supports two existing MCDA techniques: the value focused Weighted Sum Method (WSM) (Janssen, 1996), and the PROMETHEE outranking method (Brans et al., 1986), which are utilised to determine the total value of each alternative for the assigned input parameters. The WSM involves calculating an appraisal score for each alternative (V(an)) by multiplying each criterion PV (xm,n) by its appropriate CW (wm), followed by summing the weighted scores for all criteria as follows (Janssen, 1996): Van Z
M X mZ1

wm xm;n

where m is the criterion number, M is the total number of criteria and n is the alternative number. Alternatively, the basic PROMETHEE methods build a valued outranking relation. The preference function associated with each criterion gives the degree of preference, expressed by the DM, for alternative a with respect to alternative b on criterion xj. The function relating the difference in performance to preference is called the generalised criterion function in the outranking MCDA technique, PROMETHEE, and is selected by the DM. The preference functions, according to the PROMETHEE algorithms, are used to compute the degree of preference associated with the best action in pairwise comparisons. Six types of generalised criterion functions have been suggested by Brans et al. (1986) with the aim of realistically modelling the DMs preference, which gradually increase from indifference to strict preference, and to facilitate the inclusion of the inherent uncertainty in the criteria PVs in the decision analysis process. Further details of the PROMETHEE

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1697

Input Data

Program Initialisation (1) Enter title of the project Enter number of alternatives Enter number of criteria Enter number of actors Select PROMETHEE or WSM MCDA method

Alternative (2) & Input Parameter (3) Description Enter the name of each of the alternatives Enter the name of each of the criteria and select a preference direction for each criterion

Enter Input Parameter Values (4) & (5) Enter the criteria performance values. If the WSM is selected in (1) then the user can select from a range of standardisation methods, if required Enter the criteria weights. Select whether the criteria weights sum to 1, 100 or some other amount

Deterministic Analysis Undertaken using selected MCDA technique and input parameters to obtain total values and hence rankings of each of the alternatives using each of the actors CWs

Select Uncertainty Analysis Method

Distance-Based (6) Select pair of alternatives Select which actor criteria weights to be used Select input parameters to vary Select data range or user range for input parameter constraints Select Engine. If Solver selected, enter number of iterations to undertake. If Genetic Algorithm selected, use default parameters or enter values Select distance metric PERFORM ANALYSIS

Stochastic (7) Select Deterministic, Partial Deterministic or Stochastic Analysis Define input parameter distributions (e.g. fitted, uniform or normal) with either user specified or actual data ranges. In addition, select whether to include CW correlations and specify the data type of the PVs (e.g. continuous or discrete) Select whether to input number of Monte Carlo simulations or to allow to run until convergence Select Monte Carlo Simulation sampling method PERFORM ANALYSIS

Distance-Based Output Euclidean distance for each pair of alternatives and actors CWs Critical input parameters

Stochastic Output Probability of alternative rankings Range of total values of alternatives Wilcoxon Rank-Sum Test Spearman Rank Correlation

Note: () refers to form number, which is where the relevant information is entered by the user
Fig. 1. Program structure.

method are contained in Brans et al. (1986). It should be noted that Level 1 generalised criterion functions are utilised in the proposed uncertainty analysis approaches for each of the criteria, therefore, the user does not need to select the

generalised criterion functions or the associated thresholds. This is because uncertainties associated with the criteria PVs are considered elsewhere in the proposed uncertainty analysis approaches (refer to Hyde et al., 2003).

1698

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

The user is required to select one of the techniques described above on the Program Initialisation form (Fig. 2). For a new decision problem, the user must save the le as a unique workbook before continuing, which enables it to be opened and utilised again, if required, following the completion of the analysis. 2.1.2. Alternative and input parameter description A description of each of the alternatives and the criteria can be entered in respective forms. The preference direction (i.e. minimise or maximise) for each of the criteria must also be selected by the user on the Criteria Description form, as shown in Fig. 3. 2.1.3. Input parameter values The next step in the decision analysis process is to assess the alternatives by the criteria that have previously been dened. The PVs may be obtained from models, available data or by expert judgement based on previous knowledge and experience. The type of value assigned to each criterion PV may be quantitative or qualitative. The CWs are elicited from the actors using one of a variety of available techniques. Once the relevant data have been obtained, the PVs and CWs can be entered, or copied from existing les, by the user into the spreadsheets available (see Fig. 4 for an example of the input PVs worksheet). It should be noted that the PVs are required to be standardised to commensurable units when the WSM is used. Therefore, two standardisation methods are available for use if the WSM is the selected MCDA technique and if the PVs are entered in incommensurable units. The only additional information required on the CWs form is the total sum of the CWs.

2.2. Decision analysis The user is asked to save the input data that have been entered, as described above, before continuing with the decision analysis process. A selection is then able to be made by the user between undertaking deterministic analysis, distancebased uncertainty analysis or stochastic uncertainty analysis, as shown in Fig. 5. 2.2.1. Deterministic analysis Deterministic MCDA is the traditional decision analysis method used to determine the total values of the alternatives and hence the ranking of each alternative for each set of actors CWs, using the selected MCDA technique. If the PROMETHEE method is utilised, the user may choose one of a number of generalised criterion functions that have been dened by Brans et al. (1986) for each criterion. This exibility in the program enables a range of analyses to be undertaken, such as comparison of the results of the proposed uncertainty analysis approaches, which only utilise the Level 1 generalised criterion functions, with, for example, existing case studies that have utilised a number of the commonly used generalised criterion functions. A ranking of the alternatives is obtained for each of the actors CWs in addition to the total value of each of the alternatives, which is displayed in tabular and graphical form. Following the deterministic analysis, the user may choose to implement either the distance-based uncertainty analysis methodology or the stochastic uncertainty analysis approach.

Fig. 2. Example of MCDA Uncertainty Analysis Initialisation form.

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1699

Fig. 3. Criteria descriptions and preference directions.

2.2.2. Distance-based uncertainty analysis methodology 2.2.2.1. Methodology. The purpose of the distance-based uncertainty analysis approach developed by Hyde and Maier (submitted for publication) is to determine the minimum modication of the MCDA input parameters (i.e. CWs and PVs) that is required to alter the total values of two selected alternatives (e.g. ax and ay), such that rank equivalence occurs. The minimum modication of the original input parameters is obtained by translating the problem into an optimisation problem and exploring the feasible input parameter ranges. The objective function minimises a distance metric, which provides a numerical value of the amount of dissimilarity between the original input parameters of the two alternatives under consideration and their optimised values. Optimised refers to the set of input parameters that is the smallest distance from the original parameter set, such that when the optimised set is used,

the total values of the two alternatives being assessed are equal. The user may select the Euclidean Distance, de (Eq. (2)), the Manhattan Distance, dm (Eq. (3)), or the Kullbacke Leibler Distance (i.e. relative entropy), dk (Eq. (4)): v u M uX 2 2 2 # # # # Min de Zt w# w# Cxmnli xmnlo Cxmnhi xmnho jmo jmi
mZ1

2 or
M   X    #  # #  wjmi w# C xmnli xmnlo  jmo mZ1

Min dm Z

 #  # C xmnhi xmnho 

Fig. 4. Input criterion performance values worksheet.

1700

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

criterion m of initially higher ranked alternative n, de is the Euclidean Distance, dm is the Manhattan distance, dk is the KullbackeLeibler Distance, M is the total number of criteria, V(ay)opt is the modied total value of the initially lower ranked alternative obtained using the optimised parameters, V(ax)opt is the modied total value of the initially higher ranked alternative obtained using the optimised parameters, LLxl and ULxl are the lower and upper limits, respectively, of the PVs of each criterion for the initially lower ranked alternative, LLxh and ULxh are the lower and upper limits, respectively, of the PVs of each criterion for the initially higher ranked alternative, LLw and ULw are the lower and upper limits, respectively, of each of the CWs for the selected actors CWs, and # refers to the standardised values of these parameters (see Eq. (10)). It should be noted that there is only one term for the CWs in Eqs. (2)e(4) because the CWs are common to all alternatives. To ensure that the scale of the input parameters does not inuence the optimisation, the values used in the distance metric (i.e. Eqs. (2)e(4)) are standardised using the following formula:
# xmnli Z

xmnli sXm

10

Fig. 5. Choice of uncertainty analysis method.

or Min dk Z
M X mZ1

w# ln jmo

w# x# x# jmo # # Cxmnlo ln mnlo Cxmnho ln mnho # # # wjmi xmnli xmnhi

The objective function equations (Eqs. (2)e(4)) are subject to the following constraints:
M X mZ1 M X mZ1

wjmi Z

wjmo

V ay optZVax opt

# where xmnli is the standardised initial PV of criterion m of initially lower ranked alternative n, xmnli is the initial PV of criterion m of initially lower ranked alternative n, sXm is the standard deviation of the set of PVs of criterion m. This formula is also applied to the other parameters in Eqs. (2)e(4), respectively. In some situations, one alternative will always be superior to another, regardless of the values the input parameters take. In this case, the ranking of the alternatives is robust, as it is insensitive to the input parameters. However, in many instances, this is not the case, and a number of different combinations of the input parameters will result in rank equivalence. By determining the smallest overall change that needs to be made to the input parameters (e.g. CWs and PVs) in order to achieve rank equivalence, the robustness of the ranking of two alternatives (ax and ay) is obtained. The total values of the alternatives are determined using the selected MCDA technique (i.e. WSM or PROMETHEE).

LLxl %xmnlo %ULxl for mZ1 to M

LLxh %xmnho %ULxh for mZ1 to M

LLw %wjmo %ULw for mZ1 to M; for actor j and LLw O0 9 where wjmi is the initial CW of criterion m for actor j, wjmo is the optimised CW of criterion m for actor j, xmnli is the initial PV of criterion m of initially lower ranked alternative n, xmnlo is the optimised PV of criterion m of initially lower ranked alternative n, xmnhi is the initial PV of criterion m of initially higher ranked alternative n, xmnho is the optimised PV of

2.2.2.2. Implementation. The user of the program has a number of parameters to input and selections to make before the distance-based uncertainty analysis can be run. The pair of alternatives that the approach is applied to is entered by the user on the Distance Metrics user form (Fig. 6). The actors CWs that are to be utilised for the analysis must also be specied. The main advantage of the proposed distance-based methodology is the ability to simultaneously vary the CWs and PVs within expected ranges of uncertainty. However, exibility is incorporated into the program by allowing the user to select that only the CWs or only the PVs are varied while the other parameters remain xed (Fig. 6). The minimum and maximum ranges of the input parameters also need to be specied by the user, which dene the

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1701

Fig. 6. Example of the Distance Metric form.

feasible range that each parameter is able to be varied between in order to achieve a reversal in ranking (i.e. ay O ax) (Eqs. (7)e(9)). The ranges enable the expected uncertainty and variability in the data to be taken into consideration, therefore, if the PROMETHEE method is selected, the generalised criterion functions are not required to be dened. The range of values (i.e. upper and lower bounds) that must be assigned to each PV represents the set of possible values for that variable, which can either be based upon knowledge of experts (e.g. select the User Range button on the form shown in Fig. 6 and enter the data, as shown in Fig. 7) or the data that are available (e.g. select the Data Range button on the Distance Metrics form (Fig. 6) which uses the minimum and maximum PVs for each criterion based on the values specied for each of the alternatives). Similarly, the CW ranges can be dened by either the DM or actors or, alternatively, actual ranges of the available data can be utilised (i.e. the minimum and maximum values of the CWs elicited from a range of actors involved in the decision process). In order to obtain the robustness of the ranking of each pair of alternatives (i.e. ax and ay) for each actors set of CWs, the optimisation problem given by Eqs. (2)e(10) needs to be solved. Two engines are available for selection by the user to minimise the objective function: Solver and Genetic Algorithm (GA). Solver is a Microsoft Excel Add-In Function, which is based upon the Generalised Reduced Gradient (GRG2) nonlinear optimisation method. GRG2 works by rst evaluating the function and its derivatives at a starting value of the decision vector and then iteratively searching for a better solution using a search direction suggested by the derivatives

(Stokes and Plummer, 2004). The GRG2 optimisation method requires that starting values are specied for the decision variables, therefore, random numbers are generated using the Microsoft Excel RANDBETWEEN function between the specied input parameter ranges for the CWs and PVs to be used as the starting values for the optimisation. GRG2 is not a global optimisation algorithm, therefore, to increase the chances of nding global or near-global optima, the optimisation is repeated a number of times using different randomly generated starting values (which will be referred to as trials for the remainder of the paper). This aims to minimise the impact that the starting values have on the outcome of the analysis. Alternatively, a GA may be selected, which is a heuristic iterative search technique that attempts to nd the best solution in a given decision space based on a search algorithm that imitates Darwinian evolution and survival of the ttest in a natural environment (Goldberg, 1989). An advantage GAs have over traditional optimisation techniques (such as GRG2) is that they do not require the use of a gradient tness function, only the value of the tness function itself. Another advantage of GAs is that they search from a population of points, investigating several areas of the search space simultaneously, and therefore have a greater chance of nding the global optimum. Constraints are unable to be incorporated directly in the formulation of the GA, therefore, they are included in the objective function and multiplied by penalty values to discourage the selection of infeasible solutions by reducing their tness. Default penalty values are provided, however, the values can be changed by the user if desired. The previously

1702

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

Fig. 7. User-dened PV ranges for distance-based uncertainty analysis.

dened objective functions (Eqs. (2)e(4)) are therefore reformulated and dened as:   X  M M X   Minimise P1  wjmi wjmo CP2 d mZ1  mZ1   CP3 Vax opt V ay opt
Fig. 8. Genetic Algorithm Input Parameters form.

11 the default Solver and GA Options and parameter values or input their own. Information that may aid the selection of these parameters is included in the Help le. The Solver trial number or the GA generation number is shown in the Excel task bar during the analysis so that the user may monitor the progress of the analysis. More detailed information on the methodology and the program can be found in Hyde and Maier (submitted for publication, 2004) and Hyde et al. (in press). 2.2.2.3. Interpretation of results. The output of the distancebased uncertainty analysis is the minimum distance metric for each pair of alternatives, which can be summarised in a matrix. A non-feasible or a very large value of the distance metric between two alternatives informs the DM that one alternative will predominantly be superior to another, regardless of the input parameter values selected between the specied ranges. Conversely, if the distance is small, small changes in the input parameters will result in rank equivalence and the ranking of the alternatives can therefore be concluded as being sensitive to the input parameter values. The decision-making process can be improved considerably by identifying critical input parameters and then re-evaluating their values more accurately. The most critical inputs can be identied by examining the relative and absolute change in input parameter values, as follows: Absolute Dxmnl Zxmnlo xmnli 12

subject to the constraints given by Eqs. (5)e(9). P1, P2 and P3 are penalty values and d is the distance metric selected by the user (i.e. de, dm or dk). The code for the GA utilised in this program has been developed by Michael Leonard of the School of Civil and Environmental Engineering, University of Adelaide in 2003 using standard algorithms as described in Michalewicz (1994) and Setnes and Roubos (2000). Specic details of the GA are included in Fig. 8. Both of the engines included in the program have their advantages and disadvantages. The main advantage of using GRG2 is its speed of arriving at a solution, however, its disadvantage is that because it is a gradient method, the chances of a local solution being obtained are high. The advantage of the GA is that it is a global search technique, however, it takes a longer time to converge compared with the GRG2. The processing time of the optimisation methods is dependent on the complexity of the decision problem that is being assessed (i.e. how many alternatives and criteria are involved in the decision problem and the robustness of the ranking of the alternatives). A trade-off is therefore required between the amount of time taken to perform the analysis and the level of certainty that the minimum distance has actually been obtained (see Hyde and Maier, submitted for publication). A number of parameters must be selected/dened before using the GA (see Fig. 8) or Solver. The user may either use

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1703

Relative Dxmnl Z

xmnlo xmnli 100% xmnli

13

probability distributions represents the actual assignment of CWs by the actors. Further details of the methodology are available below and in Hyde et al. (2004). 2.2.3.2. Implementation. Flexibility is a key component of the program, with the user having many options to assess the various sources of input parameter uncertainty using the stochastic uncertainty analysis approach. The user may either select partial deterministic (distributions for either CWs or PVs only) or full stochastic analysis (distributions for both CWs and PVs) on the Stochastic Uncertainty Analysis user form (Fig. 9). In the situation where a relatively large number of actors have been included in the decision process, the CWs of the actors can be considered as a representative sample of the CWs of a population of stakeholders. The user may therefore decide to portray these preferences by tting a distribution to the actual CWs, ensuring that all of the information obtained from the actors is explicitly incorporated in the decisionmaking process. The Microsoft Excel @Risk add-in program (Palisade, 2000) is used to t distributions to the data and goodness of t statistics are reviewed to determine how representative the tted distributions are of the actual sets of CWs elicited from the actors. The user can also elect to undertake a correlation analysis, with the results incorporated in the distributions, to ensure that sampling from the CW probability distributions represents the actual assignment of CWs by actors. In the circumstance where a small number of actors are involved in the decision process, and there are consequently insufcient CWs available to t a representative distribution, the user can select either a normal or uniform distribution to enable uncertainty and subjectivity in the weights to be incorporated in the analysis. The user must then also characterise the uniform distributions, by dening the upper and lower bounds of the CWs using either actor specied limits or bounds based upon the actual CWs available. The uncertainty, imprecision and variability in the quantitative PVs can be represented by continuous probability distributions such as uniform or normal. A range of values (i.e. upper and lower bounds for uniform distributions) must be assigned to each PV by the user, representing the set of possible values for that variable, which can either be based upon knowledge of the experts or the data that are available. A discrete uniform distribution can be utilised for qualitative PVs, which is characterised by dening upper and lower limits of each PV (e.g. a groundwater rise of medium may be between medium low and medium high, or in an integer scale between 2 and 4). The user must also select whether each criterion belongs to a discrete or continuous data type. The user is able to elect whether the Monte Carlo Simulation (MCS) will run until convergence of the input distributions or until a number of user specied iterations are completed. The user may also choose whether random sampling or Latin-Hypercube Sampling is utilised. Information is provided in the Help le to aid the user in making a decision if they are unfamiliar with MCS. The Microsoft Excel @Risk add-in program (Palisade, 2000) is also used to undertake the MCS.

It should be noted that Eqs. (12) and (13) can also be used to determine the most critical PVs of the initially higher ranked alternative, as well as the most critical CWs. The input parameters that exhibit the smallest relative change in value to achieve rank equivalence between two alternatives are most critical to the reversal in ranking. The results therefore provide the DM with further information to aid in making a nal decision. 2.2.3. Stochastic uncertainty analysis approach A shortcoming of the distance-based approach is that it does not consider the likelihood that the input parameter values are changed by a certain amount within their specied uncertainty ranges. If sufcient information is available to dene probability distributions for likely values of each input parameter, the stochastic uncertainty analysis approach can be used to obtain additional information on the likelihood of alternatives achieving a particular ranking. 2.2.3.1. Methodology. As was the case with the distance-based uncertainty analysis approach, the reliability based approach developed by Hyde et al. (2004) enables the DM to examine the robustness of a solution, allowing a decision to be made with condence that the alternative chosen is the best performing alternative under the range of probable circumstances. The proposed stochastic method involves dening the uncertainty in the input values using probability distributions, performing a reliability analysis by Monte Carlo Simulation (MCS) and undertaking a signicance analysis using the Spearman Rank Correlation Coefcient. The approach involves utilising existing MCDA techniques to determine the total value of each alternative, however, the advancement is in the application of Monte Carlo Simulation (MCS) (Kottegoda and Rosso, 1997) to enable repetition of the selected MCDA method with the range of possible input values dened by probability distributions. Three key stages are involved in the proposed reliability analysis process: (i) the input values are randomly sampled from their respective probability distributions, assuming the CW and PV distributions are independent, while maintaining the correlation structure of the CWs, (ii) the randomly drawn vector of CWs is normalised, as the sum of all elements of the weight vector must equal the original total sum of the CWs (e.g. 1 or 100) (Janssen, 1996), and (iii) the selected MCDA technique is applied to determine the total value of each alternative for that realisation (i.e. with the randomly drawn vector of PVs and normalised CWs). The proposed approach provides the benets of jointly varying the PVs and CWs, including any correlations between the CWs, in the analysis and allowing all expected uncertainty and subjectivity in the CWs and PVs to be incorporated so that posterior analysis is not required to be undertaken. CW correlations, obtained through correlation analysis, are incorporated in the distributions to ensure that sampling from the CW

1704

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

Fig. 9. Example of Stochastic Uncertainty Analysis form.

2.2.3.3. Interpretation of results. The results of the reliability analysis provide the DM with valuable information, including distributions of the total values for a single alternative or the difference between total values for competing alternatives. Knowledge of the likelihood of the total value over the entire range of possible input values enables the DM to better assess the risk of an adverse outcome or select an alternative based upon the likelihood that its total value will exceed that of its competitor by a specied amount. The probability that an alternative an receives rank r, based on all probable criteria input parameters, is also available to the DM, in order to assess the robustness of a solution. The DM may also review the input parameter values that result in the probabilities of one alternative outranking another alternative. The signicance of the difference between output distributions is obtained by undertaking the Wilcoxon RankeSum Test (Kottegoda and Rosso, 1997) for each pair of alternatives. To facilitate interpretation of the results, signicance analysis is used to identify the relative contribution that each input parameter (i.e. each CW and PV) has in determining the total

value of an alternative. The most signicant inputs to the analysis are determined using the Spearman Rank Correlation Coefcient, R (Kottegoda and Rosso, 1997). The value of R always lies between 1 and C1, where a value of 1 or C1 indicates perfect association between the parameters, the plus sign occurring for identical rankings and the minus sign occurring for reverse rankings. When R is close to zero, it is concluded that the variable under consideration (i.e. a particular CW or PV) does not have a signicant impact on the ranking of the alternative. Once the most important input parameters are identied from the signicance analysis, resources can be concentrated on characterising their uncertainty if further analysis is required to arrive at a nal decision. 3. Case study The use and the potential benets of the program are illustrated by undertaking analysis of a study carried out by Fleming (1999) and details of the study, the problem formulation and the results are presented below.

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1705

3.1. Decision analysis formulation The Northern Adelaide Plains (NAP) is an important horticultural region on the outskirts of Metropolitan Adelaide, South Australia. The most important water resource in the region is groundwater, however, the groundwater has been used at a rate that far exceeds the rate of natural replenishment, threatening the sustainability of the resource and the horticultural industry that depends upon it. One hundred and thirty-nine actors were involved in elaborating the three development alternatives and 10 evaluation criteria. This information has been entered in the MCDA Uncertainty Analysis Initialisation form, as shown in Fig. 2, in addition to selecting WSM as the MCDA technique to utilise for the analysis, as it is a simple and commonly used MCDA technique. As it is a new decision problem, the relevant data must be entered before the analysis can be undertaken, therefore, after pressing the Input Data button, the Alternatives Description form becomes visible. A description of the three alternative water management options, including Business As Usual (Alternative 1), tertiary wastewater treatment for irrigation of horticulture (Alternative 2) and water conservation and reuse (Alternative 3), is entered on this form. These alternatives were assessed by Fleming (1999) to determine a development strategy that offers a more sustainable future for the area. The next step is to enter a description of the 10 criteria, including environmental, social and economic factors, that were selected to evaluate the three alternatives, on the available form, as shown in Fig. 3. The criteria preference directions are also entered on this form. After pressing the Continue button, the criteria PVs are entered into the worksheet available (Fig. 4), which were evaluated by experts, and involved the interpretation of the documented development alternatives, together with the experts own personal experience and understanding of the issues. A standardisation method is not required, as the PVs are already in commensurable units. In the next window, the CWs are entered for each of the 139 actors, which were elicited via a program of community consultation. This completes the data entry process, and after the Next button is pressed, the user must save the input data before selecting which type of decision analysis to undertake, as shown in Fig. 5. 3.2. Decision analysis Each of the decision analysis methods available for use in the program is described below utilising the case study data that have been entered, as described in Section 3.1. 3.2.1. Deterministic Initially, deterministic analysis is undertaken by clicking the Deterministic button on the form shown in Fig. 5. The program calculates the total value of each alternative and for each actors set of CWs using the WSM. The ranking of the alternatives is also determined. The results of the deterministic analysis are able to highlight the different priorities of the actors, illustrating common agreements and conicts. The results
Total Value

1 0.8 0.6 0.4 0.2 0 Alt 1

Alt 2

Alt 3

Alternatives
Fig. 10. Deterministic MCDA output e total values obtained for each alternative using the CWs elicited from each actor.

of this analysis are illustrated in Fig. 10 and for this case study, it is evident that all of the actors preference values result in the same ranking where Alternative 3 is ranked highest, Alternative 2 is ranked second and Alternative 1 is ranked third. 3.2.2. Distance-based uncertainty analysis approach If the user decides to undertake distance-based uncertainty analysis by clicking the Distance Analysis button on the form shown in Fig. 5, the form shown in Fig. 6 is displayed and the relevant data must be entered by the user before the analysis can commence. The user must enter the pair of alternatives that the analysis will be undertaken for and which set of actor CWs to utilise. To illustrate the methodology, both the CWs and PVs were selected to be altered simultaneously. Therefore, the feasible input parameter range for the actors CWs and the PVs of the alternatives must be specied, as dened by Eqs. (7)e(9). No information was provided by Fleming (1999) on the uncertainty associated with the criteria PVs or the CWs, therefore, the upper and lower limits of the input parameters were assigned to enable the proposed approach to be illustrated. The data entry form for the ranges of the PVs is shown in Fig. 7. A distance metric must also be selected from the three available in the program and for this case study the Euclidean Distance has been chosen as it is one of the most commonly used distance metrics. Once the alternatives and input parameters of interest have been selected, the user can select and execute one of the two available solution methods. The optimisation was initially undertaken using the Microsoft Excel Add-In Solver Function with 100 trial solutions in order to test the complexity of the solution space with the aim of determining whether it is best to use the GRG2 (Solver) or GA to undertake the complete
0.50

Euclidean Distance

0.45 0.40 0.35 0.30 0.25 0.20 0 20 40 60 80 100

Trial Number
Fig. 11. Final Euclidean Distances obtained for each trial number using GRG2 and comparing Alternatives 3 and 2 with Actor 1 CWs.

1706

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

Table 1 Euclidean Distances of pairs of alternatives for Actor 1 CWs using a GA and user ranges for CWs and PVs Initially higher ranked alternatives Alt 3 (R1) Initially lower ranked alternatives Alt 3 (R1) Alt 2 (R2) Alt 1 (R3) e 0.291 0.571 Alt 2 (R2) NA e 0.410 Alt 1 (R3) NA NA e

most signicant input parameters. From Table 2 it can be seen that the input parameters that have the most impact on the analysis are PV3 and PV9 of Alternative 3, as they have the smallest relative change. A larger variation in the CWs is required for them to have a signicant impact on the outcome of the analysis. 3.2.3. Stochastic uncertainty analysis approach If the user elects to undertake an alternative uncertainty analysis approach and clicks the Stochastic Analysis button on the form shown in Fig. 5, then the Stochastic Uncertainty Analysis form, shown in Fig. 9, is displayed. In order to demonstrate the proposed methodology, stochastic analysis is selected and therefore distributions for both the CWs and PVs need to be dened. Decisions are then required to be made by the user, based on the available input parameter data, regarding the types of distribution to utilise. This is undertaken on the form by clicking the appropriate check boxes. One hundred and thirty-nine actors were involved in the decision analysis undertaken by Fleming (1999), therefore, the CWs elicited from the actors could be regarded as a sample of the CWs that would be obtained from the population of stakeholders. It was therefore decided to t distributions to the CWs and the type of distributions obtained for each criterion and the goodness of t statistics are summarised in Table 3. Correlation analysis of the elicited CWs was chosen to be undertaken, by clicking the relevant check box, as shown in Fig. 9, to ensure that the sampling from the CW distributions replicated the preference information provided by the actors. The tted distribution obtained for CW3, compared to the average CW (which was used in the analysis undertaken by Fleming, 1999), is shown in Fig. 12, which demonstrates the range of actor preference information that is disregarded if only the average CW is utilised. As stated above, no information is available on the uncertainty with regard to the criteria PVs, therefore, a uniform distribution was assumed and the upper and lower limits of the distributions were taken to be the same as those utilised in the distance-based uncertainty analysis approach (i.e. between the user specied ranges). The MCS was selected to be run until the input parameter distributions converged (i.e. when the sampled values approximated the distributions of the input values) and the sampling method utilised was Latin-Hypercube

analysis. Different Euclidean Distances were obtained with each of the random starting values when using GRG2 (as illustrated in Fig. 11), which indicates that the solution space is complex and that the starting values have an impact on the solution. Therefore, the GA was selected as the preferred optimisation method, as it is more likely to arrive at a global solution when the solution space is complex. The performance of a GA is a function of a number of userdened parameters, including penalty function values, mutation and crossover probabilities, generation and population numbers (Raju and Kumar, 2004). The optimal values of these parameters to be used during the analysis were obtained by considering suggested values published in the literature (Dandy and Engelhardt, 2001; Mirrazavi et al., 2001; Raju and Kumar, 2004). Sensitivity analysis was also undertaken by varying the size of the population (100e800) and the number of generations (2500e10,000) to ascertain the impact these parameters have on the time taken for the analysis to be undertaken, in addition to their inuence on the nal value of the selected distance metric. The optimal parameters obtained, which were utilised in the analysis, are contained in Fig. 8. The Euclidean Distances obtained that result in rank equivalence for each pair of alternatives using Actor 1 CWs and varying the CWs and PVs are summarised in Table 1. Only the results utilising Actor 1 CWs are presented due to space limitations. The initially lower ranked alternatives are listed down the leftmost column of Table 1, in rank order, while the initially higher ranked alternatives are listed across the top row, in rank order. From Table 1, it can be seen that the second ranked alternative (Alt 2) is the most likely to outrank the highest ranked alternative (Alt 3) when the uncertainty in the CWs and the PVs is considered in the analysis, as it has the smallest Euclidean Distance (i.e. 0.291). An additional output from the distance-based uncertainty analysis approach is the

Table 2 Changes in input parameter values for Alternative 3 Z Alternative 2, CWs of Actor 1 Criteria CWs of Actor 1 wmi 1 2 3 4 5 6 7 8 9 10 0.223 0.173 0.084 0.200 0.189 0.052 0.019 0.010 0.021 0.031 wmo 0.250 0.100 0.050 0.200 0.120 0.160 0.010 0.011 0.090 0.010 Absolute D 0.027 0.073 0.034 0.000 0.069 0.108 0.009 0.001 0.069 0.021 %Relative D 12 42 40 0 36 210 46 16 339 68 PVs of Alt 3 xmi 0.850 0.700 0.750 0.700 1.000 0.600 0.700 0.360 0.460 1.000 xmo 0.740 0.660 0.730 0.620 0.950 0.530 0.700 0.350 0.420 1.000 Absolute D 0.110 0.040 0.020 0.080 0.050 0.070 0.000 0.010 0.040 0.000 %Relative D 13 6 3 11 5 12 0 3 9 0

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710 Table 3 Fitted criteria weight distributions and goodness of t results Criteria CW1 CW2 CW3 CW4 CW5 CW6 CW7 CW8 CW9 CW10 Distribution type Lognorm 2 Loglogistic Weibull Loglogistic Loglogistic Loglogistic Weibull InvGauss Pearson 5 InvGauss Chi-squared statistic 10.7 9.6 2.5 3.6 5.7 5.3 4.2 3.6 4.9 18.2 KolmogoroveSmirnov statistic (KeS) 0.055 0.073 0.032 0.035 0.050 0.043 0.055 0.054 0.055 0.078 Table 4 Probability matrix that Alternative m obtains rank r Rank 1 2 3 Alt 1 (%) 0.00 0.13 99.87 Alt 2 (%) 1.07 98.80 0.13

1707

Alt 3 (%) 98.93 1.07 0.00

Sampling. The analysis commences when the Perform Analysis button is clicked. The MCS ran for 1500 iterations, which was when convergence of the input parameter distributions was achieved. The time taken for the simulations to complete running was approximately 1 h on a 1400 MHz Intel Pentium with 504 MB of RAM. Following completion of the reliability analysis, the DM is able to assess the output and obtain information, such as the probability that each of the alternatives will achieve a ranking of 1e3, with the knowledge that the expected range of values of the CWs and PVs has been incorporated, the input parameters have been sampled from their respective distributions randomly while maintaining any CW correlations, and the input parameters have been varied jointly. The results obtained indicate that there is a 98.9% chance that Alternative 3 is the best alternative when all of the expected values are taken into account (Table 4). The results of the reliability analysis can also be presented as cumulative distributions of the total values of the alternatives over the range of expected input parameters, with Fig. 13 illustrating the probability that an alternative will have a total value less than or equal to any variable total value for each of the alternatives. Fig. 13 also shows the range of possible total values that each alternative may attain, which is demonstrated to be extremely extensive for this case study. The single total value obtained for each alternative using the average CWs is also illustrated in Fig. 13, in addition to a cumulative distribution of the total values of the alternatives obtained by using each of the 139 sets of CWs in the WSM. The cumulative distribution obtained using the deterministic

CWs enables the DM to have more condence in the selection of Alternative 3 as the optimal alternative, compared to only undertaking the analysis using the average CW, as none of the CWs elicited from the actors results in a reversal in the ranking of the alternatives. However, the impact of the uncertainty and variability in the criteria PVs remains unknown, which can be determined by application of the proposed stochastic approach. The signicance analysis was undertaken to determine which input parameters are of most importance to the ranking of each of the alternatives. The results of the signicance analysis for Alternative 3 are shown in Fig. 14, which indicates that CW8, PV9, PV8 and PV7 are the most sensitive input parameters to the ranking of Alternative 3. 3.3. Discussion The use of the program developed in Excel and written in VBA has been illustrated by applying it to the case study originally assessed by Fleming (1999). The overall aim of MCDA is to extract, in a clear and transparent way, the information needed for a sound and responsible decision from a complex conict situation, which this program is able to help facilitate. The decision analysis process has been shown to be expedited and enhanced by its use, with each user form providing a step by step process of elucidating the decision analysis problem. The special emphasis of the program is on the consideration of uncertainty, and application of the program to the case study has demonstrated how valuable information can be obtained through undertaking the two proposed uncertainty analysis approaches that are included in the program. A summary of the results obtained to help inform the decision problem assessed by Fleming (1999) using the program is contained below. Fleming (1999) utilised an average of the 139 actors CWs and deterministic PVs when analysing the three management
Stochastic Deterministic Average CW

8 7 6

1.0

Cumulative Distribution

Weibull Distribution Average CW used in analysis by Fleming (1999)

0.8 0.6 0.4 0.2 0.0 Alt 1 Alt 3

Frequency

5 4 3 2 1 0 0.00 0.05 0.10 0.15 0.20

Alt 2

0.25

0.30

0.35

0.1

0.3

0.5

0.7

0.9

Weight
Fig. 12. Comparison of tted distribution and average actor values for CW3.

Total Value
Fig. 13. Cumulative frequency distribution for results of alternatives.

1708

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710
Alt3CW8 Alt3PV9 Alt3PV8 Alt3PV7 Alt3PV1 Alt3PV6 Alt3PV5 Alt3PV1 Alt3PV2 Alt3PV3 Alt3PV4 Alt3CW6 Alt3CW9 Alt3CW2 Alt3CW4 Alt3CW7 Alt3CW1 Alt3CW3 Alt3CW1 Alt3CW5 -0.5 -0.382 -0.342 -0.337 -0.321 -0.315 -0.285 -0.284 -0.273 -0.253 -0.251 -0.238 -0.199 -0.168 -0.084 -0.050 -0.049 0.075 0.137 0.170 0.319 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4

Input Parameter

Correlation Coefficient
Fig. 14. Spearman Rank Correlation Coefcients for Alternative 3.

alternatives for the Northern Adelaide Plains. In contrast, the deterministic analysis available in the program undertakes the decision analysis for each actors CWs and not just an average CW, which enables the different priorities of the actors to be highlighted, by illustrating common agreements and conicts. In this particular case, each of the actors preferences resulted in the same rank ordering of the alternatives. No analysis of the uncertainty of the input parameters was undertaken by Fleming (1999) and therefore no information is provided to the DM on how robust the ranking of the three alternatives is to changes in the values of the input parameters. Two uncertainty analysis methods are available for use in the program, the selection of which is dependent on the information available and the output information desired by the DM. If limited information is available on the expected uncertainty of the input parameters, the user would select the distance-based approach. The proposed distance-based uncertainty analysis approach determines the parameter combinations that are critical in reversing the ranking of two selected alternatives, thereby allowing the DM to test the robustness of the decision outcomes to variations in the input data. The analysis is somewhat analogous to a traditional sensitivity analysis, where the behaviour of the ranking of alternatives is explored within the expected range of CWs and PVs. The proposed approach, however, provides the benets of jointly varying the CWs and PVs to obtain a single measure of robustness, which is the Euclidean Distance in this case. The relatively large Euclidean Distances obtained for each pair of alternatives suggest to the DM that the ranking of the alternatives is robust and that large changes in the input parameters are required for a reversal in the ranking to occur. The ability of the method to identify the most critical input parameters also provides the DM with valuable information which can provide direction for further analysis (i.e. obtaining more information so that the uncertainty intervals can be rened) or condence that a large change in the input parameters is required before a reversal in the ranking occurs, which is the case for this particular case study. The results of the analysis also illustrate that the criteria PVs play an equally important role in the decision analysis process as the CWs, with the majority of the most critical input

parameters being PVs. This is an important result, as existing sensitivity analysis methods generally ignore the uncertainty in the PVs or consider uncertainty only in CWs or PVs, but not both simultaneously. The implication of this is that a complete and accurate understanding of the uncertainty associated with the ranking of the alternatives is not obtained when existing sensitivity analysis methods are used. A shortcoming of the distance-based approach is that it does not consider the likelihood that the input parameter values are changed by the amounts indicated to require rank reversal. If sufcient information is available to dene probability distributions for likely values of each input parameter, the stochastic uncertainty analysis approach can be used to obtain additional information on the likelihood of alternatives achieving a particular ranking. Another shortcoming of the distance-based approach is that only one set of actors CWs can be considered at a time. A benet of the stochastic uncertainty analysis approach is demonstrated in this case study, as all of the CWs elicited from the actors were able to be included in the analysis by tting a distribution to these values. The expected uncertainty and variability in the PVs were incorporated by assigning uniform distributions to the PVs. The results of the stochastic uncertainty analysis provide the DM with condence that Alternative 3 is the best performing alternative under the range of possible values, as it has a probability of being the highest ranked alternative of 98.9%. This means that there is only a small combination of parameter values within the anticipated range of values that will result in rank reversal between the alternatives, which conrms the output of the distance-based approach, however, providing more information on the likelihood of this occurring. The output of the analysis provides the DM with information to aid discussion regarding the alternatives with the actors and guidance as to the most important input parameters which could be the focus of further data analysis and reassessment, if required. The output of the Spearman Rank Correlation analysis also conrms the output of the distance-based approach, with changes in the PVs found to be the most critical for rank reversal to occur, although this is unlikely to occur in this case study.

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

1709

4. Summary and conclusions The output of MCDA techniques (such as WSM and PROMETHEE) depends critically on the data input, therefore, the values of the input parameters should ideally be precise. However, in reality, available information is often uncertain, which cannot be taken into account in existing MCDA techniques and associated software programs. Existing sensitivity analysis methods developed for MCDA are generally unable to adequately consider the various sources of uncertainty, as they involve the systematic variation of one input parameter at a time, while the remaining parameters are xed and mainly focus on the inuence of the CWs. The main focus of this paper is the introduction of a program, which has been developed in Microsoft Excel using VBA, and extends the analysis of existing softwares for MCDA based on WSM and PROMETHEE. The emphasis in designing the program was to support two proposed uncertainty analysis methods: a distance-based uncertainty analysis approach (Hyde and Maier, submitted for publication) and a stochastic uncertainty analysis approach (Hyde et al., 2004). These two improved uncertainty analysis approaches enable the uncertainty and variability in the input parameters to be included in the analysis concurrently, as it may be the case that a decision is insensitive to the variation of some parameters individually, but sensitive to their simultaneous variation. Flexibility has been incorporated into the program, however, by enabling individual input parameters to be varied, if required. The distance-based uncertainty analysis approach is able to determine the parameter combinations, and the distances between them, that are critical in reversing the ranking of two selected alternatives. Alternatively, the stochastic uncertainty analysis approach allows the DM to gain an enhanced understanding of the risk associated with selecting an alternative and is able to select an alternative based on the likelihood that it will outrank the other alternatives considered when all of the estimated uncertainties are taken into account. The program produces results in a way that is transparent and easy to understand in order to promote acceptance among the actors, as well as the other stakeholders. Information on the robustness of the ranking of alternatives to variation in all of the input parameters is therefore provided to the DM using the program. Help les are included to aid in the utilisation of the program and to inform the user of the various technical aspects of the program, however, the targeted user of the program is the expert MCDA analyst/facilitator. It is believed, though, that by enhancing and extending some features of the program, the program could be utilised by DMs and actors without the aid of an expert to provide enhanced support for decision analysis. The program has been tested successfully on a number of case studies, one of which is presented in this paper (the case study undertaken by Fleming, 1999). As shown, a user who is familiar with other Windows applications can easily use the program. The case study presented demonstrates how all of the available information, including uncertainty, can be incorporated in the analysis concurrently, providing additional information on the robustness of the alternatives to variation

in the input parameters and allowing condence to be placed in the outcome of the analysis. If limited information is available, the user is able to use the distance-based uncertainty analysis method, which demonstrated that large changes in the parameter values are required for a reversal in the ranking of the alternatives to occur. The outcomes of this analysis were conrmed by performing a stochastic uncertainty analysis, which requires distributions to be dened for the input parameters. The output of the stochastic analysis provides the DM with more information than the distance-based approach, as it shows the likelihood that one alternative will outrank another (e.g. 1.07% chance that Alternative 2 will outrank Alternative 3). In addition, the signicance analysis available with both uncertainty analysis methods informs the DM of the input parameters that have the most inuence on the ranking of each alternative and therefore information is able to be provided to the DM to direct future analysis and renement of the input parameter values, if required. The Fleming (1999) case study demonstrated the importance of incorporating the uncertainty of both the PVs and the CWs in the analysis, as the PVs were found, by both uncertainty analysis methods, to have the largest impact on the ranking of the alternatives. The program has been applied to a water resource allocation case study in this paper, but the scope of decision problems that could be analysed using the program presented in this paper is very broad: it can be applied to any kind of usage conicts between natural resources and economic development. Acknowledgements The authors wish to thank the Australian Research Council, Department for Water, Land, Biodiversity and Conservation (DWLBC) and the Ofce of Economic Development, State Government of South Australia for funding to undertake this research. In addition, the authors wish to thank Michael Leonard from the School of Civil and Environmental Engineering at the University of Adelaide for his work on the development and provision of the code for the GA used in the program. Appendix A
Acronyms CWs DM GA GRG2 MCDA MCS NAP PROMETHEE PVs TOPSIS VBA WSM Criteria weights Decision maker Genetic Algorithm Generalised Reduced Gradient nonlinear optimisation method Multi-criteria decision analysis Monte Carlo Simulation Northern Adelaide Plains Preference ranking organisation METHod for enrichment evaluations Performance values Technique for preference by similarity to the ideal solution Visual Basic for Applications Weighted Sum Method

1710

K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710 Hyde, K.M., Maier, H.R., Colby, C.B., 2003. Incorporating uncertainty in the PROMETHEE MCDA method. Journal of Multi-Criteria Decision Analysis 12, 245e259. Hyde, K.M., Maier, H.R., Colby, C.B., 2004. Reliability-based approach to multicriteria decision analysis for water resources. Journal of Water Resources Planning and Management 130 (6), 429e438. Hyde, K.M., Maier, H.R., Colby, C.B. A distance-based approach to undertaking sensitivity analysis of criteria weights for water resources decision making. Journal of Environmental Management, in press. Janssen, R., 1996. Multiobjective Decision Support for Environmental Management. Kluwer Academic Publishers, Netherlands. Kottegoda, N.T., Rosso, R., 1997. Statistics, Probability, and Reliability for Civil and Environmental Engineers. McGraw-Hill, Singapore. Michalewicz, Z., 1994. Genetic Algorithms C Data Structures Z Evolution Programs. Springer-Verlag, New York. Mirrazavi, S.K., Jones, D.F., Tamiz, M., 2001. A comparison of genetic and conventional methods for the solution of integer goal programmes. European Journal of Operational Research 132 (3), 594e602. Mysiak, J., Guipponi, C., Rosato, P., 2005. Towards the development of a decision support system for water resource management. Environmental Modelling & Software 20, 203e214. Palisade, 2000. @Risk e Risk Analysis Add-In for Microsoft Excel. 4.5.2 e Professional Edition. Podinovski, V.V., 1999. A DSS for multiple criteria decision analysis with imprecisely specied trade-offs. European Journal of Operational Research 113, 261e270. Raju, K.S., Kumar, D.N., 2004. Irrigation planning using Genetic Algorithms. Water Resources Management 18 (2), 163e176. Rios Insua, D., French, S., 1991. A framework for sensitivity analysis in discrete multi-objective decision-making. European Journal of Operational Research 54 (2), 176e190. Roy, B., Vincke, P., 1981. Multicriteria analysis: survey and new directions. European Journal of Operational Research 8 (3), 207e218. Setnes, M., Roubos, H., 2000. GA-fuzzy modeling and classication: complexity and performance. IEEE Transactions on Fuzzy Systems 8 (5), 509e522. Stokes, S., Plummer, J., 2004. Using spreadsheet solvers in sample design. Computational Statistics & Data Analysis 44, 527e546. Triantaphyllou, E., Sanchez, A., 1997. A sensitivity analysis approach for some deterministic multi-criteria decision-making methods. Decision Sciences 28 (1), 151e194. Ulvila, J.W., Seaver, D.A., 1982. Decision analysis for water resources planning. Omega 10, 185e194. Wolters, W.T.M., Mareschal, B., 1995. Novel types of sensitivity analysis for additive MCDM methods. European Journal of Operational Research 81 (2), 281e290.

References
Bana e Costa, C.A., Vansnick, J.C., 1999. The Macbeth Approach: Basic Ideas, Software and an Application. Kluwer Academic Publishers. Barron, H., Schmidt, P., 1988. Sensitivity analysis of additive multiattribute value models. Operations Research 36 (1), 122e127. Brans, J.P., Vincke, P., Mareschal, B., 1986. How to select and how to rank projects: the PROMETHEE method. European Journal of Operational Research 24 (2), 228e238. Dandy, G.C., Engelhardt, M., 2001. Optimal scheduling of water pipe replacement using genetic algorithms. Journal of Water Resources Planning and Management 127 (4), 214e223. Deng, H., Yeh, C.H., Willis, R.J., 2000. Inter-company comparison using modied TOPSIS with objective weights. Computers & Operations Research 27 (10), 963e973. Dias, L.C., Climaco, J.N., 2000. Additive aggregation with variable interdependent parameters: the VIP analysis software. Journal of the Operational Research Society 51, 1070e1082. Duckstein, L., Treichel, W., Magnouni, S.E., 1994. Ranking of groundwater management alternatives by multicriterion analysis. Journal of Water Resources Planning and Management 120 (4), 546e565. Felli, J.C., Hazen, G.B., 1998. Sensitivity analysis and the expected value of perfect information. Medical Decision Making 18 (1), 95e109. Fleming, N., 1999. Sustainability and Water Resources Management for the Northern Adelaide Plains, South Australia. The University of Adelaide. Flug, M., Seitz, H.L.H., Scott, J.F., 2000. Multicriteria decision analysis applied to Glen Canyon Dam. Journal of Water Resources Planning and Management 126 (5), 270e276. Geldermann, J., Zhang, K., 2001. Software review: decision lab 2000. Journal of Multi-Criteria Decision Analysis 10 (6), 317e323. Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading. Guipponi, C., Mysiak, J., Fassio, A., Cogan, V., 2003. MULINO-DSS: a computer tool for sustainable use of water resources at the catchment scale. Mathematics and Computers in Simulation 64, 13e24. Hill, M.J., Braaten, R., Veitch, S.M., Lees, B.G., Sharma, S., 2005. Multicriteria decision analysis in spatial decision support: the ASSESS analytic hierarchy process and the role of quantitative methods and spatially explicit analysis. Environmental Modelling & Software 20, 955e976. Hyde, K.M., Maier, H.R., 2004. Distance-based uncertainty analysis for multicriteria decision analysis in Excel using Visual Basic for Applications. In: Managing Uncertainty in Decision Support Models Conference, Coimbra, Portugal, 22e24 September, 2004. Hyde, K.M., Maier, H.R. A distance-based uncertainty analysis approach to multi-criteria decision analysis using Genetic Algorithms. European Journal of Operational Research, submitted for publication.