European Journal of Operational Research 250 (2016) 179–191

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Decision Support

A new method for elicitation of criteria weights in additive models:

Flexible and interactive tradeoff
Adiel Teixeira de Almeida∗, Jonatas Araujo de Almeida, Ana Paula Cabral Seixas Costa,
Adiel Teixeira de Almeida-Filho
Universidade Federal de Pernambuco Cx. Postal 7462, Recife-PE, 50.630-970, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: This paper proposes the Flexible and Interactive Tradeoff (FITradeoff) method, for eliciting scaling constants
Received 4 November 2014 or weights of criteria. The FITradeoff uses partial information about decision maker (DM) preferences to de-
Accepted 31 August 2015
termine the most preferred in a specified set of alternatives, according to an additive model in MAVT (Multi-
Available online 8 September 2015
Attribute Value Theory) scope. This method uses the concept of flexible elicitation for improving the appli-
Keywords: cability of the traditional tradeoff elicitation procedure. FITradeoff offers two main benefits: the information
Multiple criteria analysis required from the DM is reduced and the DM does not have to make adjustments for the indifference be-
MAVT additive model tween two consequences (trade-off), which is a critical issue on the traditional tradeoff procedure. It is easier
Flexible elicitation for the DM to make comparisons of consequences (or outcomes) based on strict preference rather than on in-
Interactive elicitation difference. The method is built into a decision support system and applied to two cases on supplier selection,
Tradeoff elicitation already published in the literature.
© 2015 Elsevier B.V. and Association of European Operational Research Societies (EURO) within the
International Federation of Operational Research Societies (IFORS). All rights reserved.

1. Introduction our research and led to the original achievement proposals to

One of the most relevant issues in using a multicriteria decision
model is probably that of evaluating the weights of criteria (or at-
tributes) in the aggregation procedure. This is particularly relevant
for aggregation using an additive model. In practice, this aggregation
procedure is the most commonly found in a multicriteria decision
model (Spliet & Tervonen, 2014), for instance when selecting suppli-
ers (Xia & Wu, 2007), or planning of metro extension lines (Hurson &
Siskos, 2014). The additive model can be applied under some basic as-
sumptions covered by many earlier studies (Fishburn, 1967; Keeney,
1972; Keeney, 1992; Keeney & Raiffa, 1976). Stewart´s survey on mul-
ticriteria methods shows some useful characteristics for an additive
model (Stewart, 1992). A more recent survey considers eliciting the
weights of criteria as a central issue (Riabacke, Danielson, & Ekenberg,
2012). Eisenführ, Weber, and Langer (2010) give a broad overview on
weights elicitation procedures for additive models.
Previous studies on experimental analysis (Borcherding, Eppel,
& von Winterfeldt, 1991; Weber & Borcherding, 1993) on the main
elicitation procedures for additive models have identified some ma-
jor difficulties and challenges. The results of these studies prompted
∗
Corresponding author. Tel./fax: +55 81 21268728.
E-mail addresses: almeidaatd@gmail.com (A.T. de Almeida), jonatasaa@yahoo.
com.br (J.A. de Almeida), apcabral@hotmail.com (A.P.C.S. Costa), adieltaf@gmail.com
(A.T. de Almeida-Filho).
overcome those issues that this article sets out. First of all, it has
long been held that the tradeoff elicitation procedure (Keeney, 1992;
Keeney & Raiffa, 1976) has a strong axiomatic foundation (Weber &
Borcherding, 1993). Nevertheless, experimental studies have shown
that inconsistencies have been found when applying this procedure
(Weber & Borcherding, 1993).
The method proposed in this paper contributes to overcoming
some of these inconsistencies. This paper proposes a flexible elici-
tation procedure, which collects information from the DM, and eval-
uates this information. The main difference from previous studies is
related to the elicitation process. In flexible elicitation, incomplete or
imprecise information, a priori, is not assumed. Whether the DM is
or is not able to give complete information, this is evaluated in the
elicitation process itself, in a flexible way. For this reason, right from
the start, the flexible process seeks complete information, based on
the tradeoff elicitation procedure. However, at any point further on, it
may consider incomplete information in either of the following two
situations: when a unique solution is found or when the DM is not
able to give additional information.
The method is built into a DSS (decision support system), which
uses a flexible elicitation concept that requires less effort from the
DM (Decision Maker). Before presenting the method proposed and
its DSS, a brief review of the related literature is presented. In or-
der to illustrate how the method named FITradeoff (Flexible and
Interactive Tradeoff) works, the DSS is used on two applications

http://dx.doi.org/10.1016/j.ejor.2015.08.058
0377-2217/© 2015 Elsevier B.V. and Association of European Operational Research Societies (EURO) within the International Federation of Operational Research Societies (IFORS).
All rights reserved.
180 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
dealing with supplier selection problems (Barla, 2003; Xia & Wu,
2007).
2. Literature related to the additive model and the elicitation
of weights
As previously stated, eliciting a criterion weight (ki ) is probably
the main concern of an additive model with regard to aggregating
the value functions vi (xi ) over the consequences xi for all criteria
i (i = 1,…, n), which is represented (Fishburn, 1967; Keeney, 1972;
Keeney, 1992; Keeney & Raiffa, 1976) in (1), usually assuming the nor-
malization in (2).

n
v(x) = ki vi (xi ).
i=1

n
ki = 1 and ki ≥ 0.
i=1
In many studies the use of the term scaling constant for ki is pre-
ferred to weight, considering that these parameters are not related
only to the meaning of the importance of criteria, but include other
issues (Edwards & Barron, 1994; Keeney, 1992; Keeney & Raiffa, 1976;
Kirkwood & Corner, 1993). In this paper, these issues are also con-
sidered, although the term weight is used for the sake of simplifi-
cation. Similarly, the term criteria is applied in the same sense as
attributes.
With regard to the evaluation of criteria weights, many studies
are found in the literature on eliciting scaling constants or weights
of criteria in an additive model obtained from a DM’s preferences
(Eisenführ et al., 2010; Keeney, 1972; Keeney, 1992; Keeney & Raiffa,
1976; Kirkwood, 1997). Several procedures in additive models are dis-
cussed in a recent survey on the procedures for eliciting weights,
which considers both value and utility functions (Riabacke et al.,
2012).
Many of these elicitation procedures are concerned with obtain-
ing complete information, in order to assess the weights. For in-
stance, the swing procedure is one of the procedures applied (Barron
& Barrett, 1996a; Barron & Barrett, 1996b; Edwards & Barron, 1994).
Macbeth (Measuring Attractiveness by a Categorical Based Evaluation
Technique) is a method based on a qualitative evaluation of the dif-
ference of attractiveness (Bana e Costa, De Corte, & Vansnick, 2005),
in which the weights for the additive model are evaluated, thereby
avoiding that the DM has to produce a direct numerical representa-
tion of preferences. Another procedure based on complete informa-
tion is the tradeoff procedure, which considers tradeoffs on criteria
(Keeney, 1992; Keeney & Raiffa, 1976). The next section gives more
details about the tradeoff procedure, on which this paper is based,
and introduces flexible and interactive approaches for using partial
information.
Some elicitation procedures use a complete evaluation (Edwards
& Barron, 1994; Keeney & Raiffa, 1976) and others use partial informa-
tion (Barron, 1992; Barron & Barrett, 1996a; Barron & Barrett, 1996b)
in order to evaluate alternatives. The SMARTER method (Edwards &
Barron, 1994) is one of the propositions based on partial information.
Studies making use of information of ranked weights indicate that
there are two conceptual approaches to selecting the best alternative
(Edwards & Barron, 1994). The first analyzes inequalities for weights,
as shown in (3), in order to eliminate inferior alternatives. The second
uses surrogate weights, which should be consistent with the ranked
weights. Barron and Barrett (Barron & Barrett, 1996a; Barron & Bar-
rett, 1996b) present rank-order centroid weights (ROC), on using the
second approach.
For many partial information procedures, such as for the ROC, the
conditions given in (2) and the information on the ranking of the
weights (k1 > k2 > · · · ki · · · > kn−1 > kn ), lead to the n-dimension
weight space (ϕ n ) given by in (3):

ϕn = (k1 , k2 , k3 , . . . , kn )|k1 > k2 > k3 > · · · > kn ;


n
ki = 1); ki > 0 . (3)
i=1
The centroid consists of calculating the average of the extreme
points of the weight space given by (3). Thus, the ROC weights are
the coordinates of the centroid (Barron & Barrett, 1996a; Barron &
Barrett, 1996b). The problem with such a procedure is that it may not
reflect the DM´s preferences, although previous studies have shown
there are many advantages to using this procedure (Barron & Barrett,
(1) 1996a; Barron & Barrett, 1996b).
With regard to the first mentioned approach for selecting the best
alternative (Barron & Barrett, 1996a), which uses inequalities, it is
(2)
observed that in many situations several alternatives remain which
are not seen as evident choices for the best alternative (Barron &
Barrett, 1996a; Kirkwood & Corner, 1993). Other approaches use par-
tial ranking by clusters, but many alternatives remain in the best clus-
ter (Kirkwood & Sarin, 1985). The PAIRS method (Salo & Hämäläinen,
1992) applies interval judgments, indicating a range for the weights.
These are different approaches, which are unrelated to the context of
the tradeoff elicitation procedure.
Regarding the use of imprecise or partial information, the lit-
erature presents quite a few of these approaches (Barron, 1992;
Hazen, 1986; Li et al., 2012; Lofti, Stewart, & Zionts, 1992; Mármol,
Puerto, & Fernández, 2002; Mustajoki, 2012; Park, 2004; Salo &
Hämäläinen, 2001; Salo & Punkka, 2005; Steuer, 1976). Jiménez,
Ríos-Insua, and Mateos (2003) presented a DSS to enable decisions
with imprecise parameters for additive and multiplicative multi at-
tribute utility functions, admitting imprecision for weights and utili-
ties in terms of ranges.
Thus, eliciting single values may not be an easy task, which has
inspired several approaches such as that presented by Danielson,
Ekenberg, Idefeldt, and Larsson (2007) to deal with decision analysis
problems that require a tool for enabling interval probabilities and in-
terval weights for additive aggregation processes. The Cardinal Rank
Ordering Step (CROC) used by Danielson et al. (2007) is detailed in
Danielson, Ekenberg, Larsson, and Riabacke (2014).
Another kind of procedure is based on the DM making a holistic
evaluation of a few alternatives, which are used to infer the parame-
ters related to the additive model. The UTA (Utilité Additive) method
(Jacquet-Lagréze & Siskos, 1982) is one of these procedures.
Although this section is not intended as an exhaustive litera-
ture review on partial information for building additive models, a
framework is presented in Fig. 1 that summarizes different types of
approaches introduced for partial information in the elicitation pro-
cedure. This framework considers three main steps, preference state-
ments by the DM, forms of partial information and a final synthesis
step for generating the output by dealing with partial information for
screening alternatives. The second step represents an interface be-
tween the two other steps, and the structure of information applied.
Although it is not explicit in Fig. 1, regression analysis may be applied
in the synthesis step, which may be needed for holistic judgments,
which may also apply LPP models.
The procedures mentioned above may be classified based on this
framework. For instance, using the framework of Fig. 1, one can see
that in the SMARTER procedure, the preference statements by the DM
are given based on a structured elicitation process, all of this informa-
tion is given at once, and a fixed process is applied. The form of partial
information used is the ranking of weights. The final step of synthe-
sis which generates the output by dealing with partial information is
based on surrogate weights.
The concept of a dominance relation should be considered
at this point. Let us first consider the concept as given by
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
Fig. 1. Framework with types of approaches for partial information in the elicitation procedure.
Greco, Matarazzo, and Slowinski (2008). They apply the concept of
a dominance relation to additive models, using a set of pairwise com-
parisons on a subset of reference alternatives, such as the preference
information provided by the DM. In another study, Greco, Mousseau,
and Slowinski (1999) apply a different concept of a graded domi-
nance relation in a different context, since it is used to build up a
rough approximation of a preference relation. In that case, a grade
is considered, such that, one can say that x positively dominates y
by degree h with respect to the set of criteria. Weber (1987) defines
several dominance relations in order to approach incomplete infor-
mation in additive models. Yang and Wang (2013) consider a seman-
tic dominance concept for dealing with incomplete preference infor-
mation. Mustajoki and Hämäläinen (2005) also consider the domi-
nance relation concept applied in additive models. They use interval
judgments in order to handle preferential imprecision in an additive
model based on SMART and SWING weighting methods. Eum, Park,
and Kim (2001) considered dominance and potential optimality as
two criteria in order to check whether or not each alternative is out-
performed for a fixed feasible region denoted by constraints.
A few studies were conducted anchored on experimental anal-
ysis of the main elicitation procedures (Weber & Borcherding,
1993). Many other studies consider several situations and make
comparisons between different approaches for eliciting weights
(Borcherding et al., 1991; Kirkwood & Corner, 1993; Salo & Hämäläi-
nen, 1992). Several issues are raised, such as the drawback of tra-
ditional techniques, especially on account of the elicitation process
being tedious and time consuming (Salo & Hämäläinen, 1992). One
of the critical aspects of the tradeoff procedure is that the DM is
required to make the adjustment for the indifference between the
two consequences (Weber & Borcherding, 1993). Experimental stud-
ies have found inconsistencies for 67% of the subjects, who applied
this procedure (Borcherding et al., 1991; Weber & Borcherding, 1993).
An important feature of the FITradeoff elicitation procedure is
to avoid that the DM has to make adjustments for the indifference
between two consequences, which is a critical issue on the tradi-
tional tradeoff procedure (Weber & Borcherding, 1993). This is ac-
complished in the FITradeoff by using partial information. Using the
framework of Fig. 1, for the FITradeoff procedure, the DM gives pref-
erence statements based on a structured elicitation process. All in-
formation is given interactively and a flexible process is applied. The
forms of partial information used are rankings or bounds of weights,
obtained indirectly according to the tradeoff approach. The output is
generated in the synthesis step based on LPP models.
3. FITradeoff: flexible and interactive tradeoff elicitation method
Before introducing FITradeoff, a brief, mostly qualitative, descrip-
tion of its logic and the rationale for using this process is presented,
including a few related issues from previous studies. Then, basic as-
pects are presented of the procedure for eliciting weights employing
tradeoffs (Keeney, 1992; Keeney & Raiffa, 1976), since this is the basis
for FITradeoff.
181
3.1. The concept of flexible elicitation
The concept of flexible elicitation comprises the context of build-
ing a multicriteria value model. This flexibility means that the elici-
tation procedure can be easily changed and adapted to different con-
ditions and circumstances as and when they occur. That is, the way
in which the elicitation procedure is implemented does not follow all
steps that the standard procedure requires. Instead, the steps of the
procedure are chosen according to the different conditions that arise
during the process, in such a way that the DM is expected to provide
less information than in the standard procedure.
The flexibility of the DSS consists of systematically evaluating the
possibility of finding a solution for the problem while the elicitation
process is being conducted. This means that the elicitation procedure
may be suspended as soon as a solution is found with the partial in-
formation obtained during this process. This partial information is
based on preference (P) relations in order to find a solution, which in
most cases can be achieved by using this partial information obtained
from the DM up to that point, which is then applied to solve a linear
programming problem (LPP).
One of the advantages of using the flexible elicitation process is
that it requires less effort from the DM and consequently may lead to
less inconsistency during the elicitation process. Since it is assumed
that as the DM is required to make less effort, this is expected to lead
to fewer errors in the processes for specifying preferences. However,
this has not been confirmed by experimental studies, which should
be conducted in another study. That is, the simplification introduced
with this method is supposed to reduce errors, although this should
be demonstrated based on behavioral decision making experiments.
Some kind of flexibility has been introduced in previous studies.
Dias and Climaco (2000) use tolerance that provides the DM with
a more flexible manner for dealing with preference statements. In
PRIME method (Salo & Hämäläinen, 2001), the DM can revise earlier
preference statements. In RICH method (Salo & Punkka, 2005), after
examining results, the DM may either: choose to accept one of the
alternatives pre-selected, or continue with the specification of fur-
ther preference information. The flexibility proposed in this paper is
consistent with the structure of preference elicitation of the tradeoff
procedure. For this reason there is some kind of fixed pattern to fol-
low. The flexibility uses partial information in a few different ways.
The method may follow most of the classical tradeoff procedure, al-
though this is less likely to occur. This may happen only if the DM
is willing to do that and when it is necessary. The method may use
partial information to give a result completely similar to that of the
classical tradeoff procedure. Another way is the use of partial infor-
mation in the general sense, with partial results, and so forth. Follow-
ing this fixed pattern avoids leaving the DM free to make any kind of
preference statements, which may be inconsistent with the axiomatic
structure of the additive model.
The approach proposed in the FITradeoff method follows the ob-
jectives of that proposed by Holloway and White III (2003) who con-
sidered that there is a cognitive cost for each question answered by
182 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
Fig. 2. Comparison of consequences for eliciting weights by means of indifference relations.
the DM, thus the cognitive effort should be reduced. White III and
Holloway (2008) based on a Markov decision process model for the
elicitation process proved that there are conditions that guarantee
the existence of a question-response policy that enables the choice of
the most preferred alternative in a finite number of questions.
Thus, the aim is to provide an elicitation process that enables a
decision process to be built with less information than is required
by the standard tradeoff procedure. Throughout the FITradeoff steps
a weights vector space is considered, which is updated with DM re-
sponses in order to reduce the subset of potentially optimal alterna-
tives. The concept of a potentially optimal alternative (Park, 2004;
Sarabando & Dias, 2009) is given in Definition 1, according to the con-
text of this study, and it relates to the alternative, the value of which,
is greater than the value of any other alternative, for at least one vec-
tor of weights in the weight space.
3.2. Procedure for eliciting weights using the classical tradeoff
procedure
In the tradeoff procedure, the DM compares consequences, by
considering tradeoffs on criteria (Keeney, 1992; Keeney & Raiffa,
1976). In one of the steps the DM is asked to choose between two
key consequences. Both consequences have the best outcome (bi ) for
one of the criteria and the worst outcome (wi ), for the others.
Let A, B and C be three different consequences. Consequence A has
the best outcome (bi ) for the criteria 1 (represented in Fig. 2 as black
circles) and the worst outcome (wi ), for the others (represented in Fig.
2 as white circles). The value function is set in such way that vi (bi ) = 1
and vi (wi ) = 0. Hence, applying the additive value function given in
Eq. (1) the value of consequence A is given by v(A) = k1 v1 (b1 ) = k1 .
Consequence B has the best outcome for criterion 2. Thus, the value of
consequence B is given by v(B) = k2 v2 (b2 ) = k2 . Now, let us consider
a strict preference relation P, such that for the DM may state either:
A is preferred do B (APB) or B is preferred do A (BPA). Therefore, as-
suming that the DM prefers consequence B to A (BPA) then, k2 > k1 ;
otherwise, k1 > k2 . This part of the procedure obtains the order of the
weights ki , using the preference relation P.
The subsequent steps consist of obtaining indifference relations (I)
in order to find the value of ki . These indifference relations are applied
to pairs of consequences of the kind shown in Fig. 2. Let xi I denote the
outcome of criterion i for which the indifference is obtained. In this
case, consequence C has the outcome x2 I (represented in the Figure as
a grey circle) for criterion 2 and the worst outcome wi for the others
(represented in the Figures as white circles), which means that v(C)
= k2 v2 (x2 I ), based on (1). The DM is asked to inform the outcome
x2 I , in criterion 2, which makes the consequences A and C indifferent
(AIC). Since the DM is able to indicate the value x2 I , then, the follow-
ing relation is established v(A) = v(C). Consequently, k1 = k2 v2 (x2 I ).
Hence, v2 (x2 I ) can be applied to compute the ratio between weights,
k1 /k2 . In addition, it should be noticed that if a non-linear value func-
tion vi (xi ) is considered for computing the weights, then this value
function modifies the ratio ki+1 /ki , compared with the possibility of
having vi (xi ) be a linear function, since this ratio depends on the value
of vi (xi I ). Considering (2), n−1 similar equations are required in order
to find the values of the weights.
The values of x2  and x2  in consequence C are respectively the
upper and lower limit for x2 I , thus, each xi I is limited by a xi ’ and a xi ’’,
which can be assessed from the DM, by the proposed flexible elicita-
tion procedure. This concept will be explored in the next section.
3.3. The concept of flexible elicitation in the tradeoff procedure
The elicitation in FITradeoff seeks to obtain xi  and xi  , based on
the DM´s preference relation P judgments. The proposition of this
method comes from the assumption that either:
• the DM is not able to specify xi I (see Fig. 3), or
• it may not be reliable to assume that this information can be ob-
tained in a consistent way from the DM.
Therefore, the specification of xi  and xi  , as illustrated in Fig. 3,
can be used instead of requiring the DM to specify xi I . Since the value
function vi is assumed to be in the interval (0,1), it can be seen that:
1 = vi (bi ) > vi (xi  ) > vi (xi I ) > vi (xi  ) > vi (wi ) = 0 (4)
From now on, for the sake of simplification of notation, let us as-
sume that the criteria are ordered, so k1 > k2 > · · · ki · · · > kn−1 > kn .
The specification of either xi  or xi  is obtained from the preference
relation P, as shown in Fig. 3. Therefore, the DM can specify xi ’ such
that YPX (Consequence Y is preferred to X). Also, the DM can spec-
ify xi ’’ such that XPZ (Consequence X is preferred to Z). Thus, v(Y) >
v(X) > v(Z). Hence, since that v(X) = k3 , v(Y) = k2 v2 (x2  ) and vi (xi  ) >
vi (xi I ), then, k2 v2 (x2  ) > k3 . Similarly, k2 v2 (x2  ) < k3 .
A more general relation can be obtained for vi (xi  ) and vi (xi  ),
given any criterion i:
ki vi (xi  ) > ki+1 , (5)
ki vi (xi  ) < ki+1 (6)
This means that:
vi (xi  ) > ki+1 /ki , (7)
vi (xi  ) < ki+1 /ki . (8)
FITradeoff criteria weights develop from a more systematic exam-
ination of this information obtained by flexible elicitation, and seek a
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191 183

(a) Consequence X

vj(bj)=1
b3

vj(wj)=0
w1 w2 w4
Criteria: 1 2 3 4

(b) Consequence Y (c) Consequence Z

bi b2 bi b2

xi’ x2’ xi’ x2’

xiI x2I xi I x2I

xi’’ x2’’ xi’’ x2’’

wi w1 w3 w4 wi w1 w3 w4
Crit 1 2 3 4 Crit: 1 2 3 4

Fig. 3. Comparison of consequences for eliciting weights by means of preference relation P.

weight space described by (9). This is a sub-space of (3). compared with each potentially optimal alternative, an alternative
⎧ n ⎫ may be considered dominated, even if it is not dominated by all po-
⎪
⎪(k1 , k2 , k3 , . . . , kn )| ki = 1; ki ≥ 0 ⎪ ⎪
⎪
⎨ i=1
⎪
⎬
tentially optimal alternatives.
 
ϕn = k1 v1 (x 1 ) < k2 < k1 v1 (x 1 ); . . . ; . (9) Definition 3. an optimal alternative, is an alternative, the value of
⎪
⎪ v (  ) < k < v (  ); . . . ; ⎪
⎪
⎪ i
⎩
k 1 x i i+1 k i 1 x i ⎪
⎭
which, as given by (1), is greater than the value of any other alter-
kn−1 v1 (x n−1 ) < kn < kn−1 v1 (x n−1 ) native j from the whole set of alternatives, whatever the vector of
weights in the whole weight space is, given by Eq. (9). The optimal
According to the performance of alternatives in the weights space
alternative dominates each one of the alternatives j from the whole
given by (9), these alternatives are classified into three different situ-
set of alternatives, in the whole weight space in (9).
ations: potentially optimal, dominated, or optimal. Hence, some def-
initions are now given in order to address some preliminary analysis, The DSS seeks an alternative from the set of alternatives that has
assuming that each alternative aj has a consequence vector xj = (x1 , the maximum value given in (1) and taking into account the weight
x2 , … , xi , … xn ), evaluated by Eq. (1). These definitions could be asso- space given by (9). The DSS conducts the elicitation process in a flex-
ciated to decision rules, so that the set of alternatives could be further ible way, so that only essential information is required from the DM.
analyzed. However, this will be let for a future work, since the focus That is, a preference relation P, in order to specify xi  and xi  , in a
of this wok follows another direction. broader range (xi  –xi  ), sufficient enough to obtain a unique solution.
It is worthwhile to observe that, in this method, no question is put
Definition 1. a potentially optimal alternative, is an alternative, the
to the DM directly related to weights, even those regarding the ratio
value of which, as given by (1), is greater than or equal to the value of
between weights or interval for weights. On the other hand, using
any other alternative j from the whole set of alternatives, for at least
the mathematical structure of the tradeoff procedure a ratio relation
one vector of weights in the whole weight space, given by Eq. (9).
between weights can be obtained, based on vi (xi I ). Thus, the limits
In most cases, it is expected that the values of potentially optimal
for this ratio can be determined by vi (xi  ) and vi (xi  ). This leads to the
alternatives are greater than (strictly) any other alternative. Although
possibility of computing an interval for weights. Therefore, this ratio
this does not happen often, they may happen to be equal to any other
(or interval) information on weights is obtained indirectly from the
alternative j, in which case both alternatives are potentially optimal.
DM. The questions directly put forward to the DM involves a pair of
Definition 2. a dominated alternative, is an alternative, the value of consequences, that evaluates the values for xi  or xi  , such as shown
which, as given by (1), will be lesser than the value of at least one of in Fig. 3. That is, the preference relation P is obtained, leading to the
the alternatives from the subset of potentially optimal alternatives, ratio relation formulated by the inequalities indicated in (7) and (8).
whatever the vector of weights in the whole weight space is, given Moreover, direct information on ratio or interval of weights does
by Eq. (9). A dominated alternative is a not potentially optimal alter- not seem to be compatible with the purpose of the tradeoff model.
native. It should be noticed that the usual definition for ’dominance In other words, the tradeoff model takes into account the concept of
relation’ is related to a pairwise relation. Although in this definition, scaling constants rather than weights as degrees of importance (de
the ’dominance relation’ is considered for the alternative in question Almeida et al., 2015). That is, the DM thinks about choices among
184 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
Fig. 4. Procedure for the DSS.
consequences as illustrated in Fig. 3; that is why the method is clas-
sified as an indirect procedure, since the weights are obtained by
the equations system, related to the DM’s preferences on these con-
sequences. On the other hand, information on ratio or interval of
weights is cognitively more compatible with outranking methods
(Roy, 1996), in which the meaning of the structure of the weights is
directly associated with their degree of importance of the criteria (de
Almeida et al., 2015).
An imprecision may be considered in most traditional procedures
for full information elicitation related to additive model, such as:
tradeoff or swing. There is a chain of ratio relation of criteria weights
between every neighboring pair of criteria in the structure of the or-
dered weights. In the classical tradeoff procedure, this information is
based on the indifference relation I, requiring the DM to specify xi  .
The proposed method does not use xi I . It uses only a range for this
relation (xi ’ and xi ’’). Therefore, since these ratios of criteria weights
(ki+1 /ki ), based on xi I is not collected, the proposed method may be
more dependable than those of full information. This is due to the
odds that this interval (xi ’ and xi ’’) may contain the real xi I not col-
lected. In other words, with full information (if exact value for xi I
were collected), the procedure may be inaccurate, since imprecision
may be considered in this process.
The main assumptions of the FITradeoff model are the same as
those of the classical tradeoff elicitation procedure (Borcherding et
al., 1991; Fishburn, 1967; Keeney, 1972; Keeney, 1992; Keeney &
Raiffa, 1976). It is valuable to emphasize that the following assump-
tions and information are required for FITradeoff:
• The weights are normalized, so (2) is applied;
• It is assumed that ranked weights are obtained
• It is assumed that the DM can specify xi  and xi  , as per (7) and
(8).
• Regarding preferences, the model incorporates criteria (or at-
tributes) with consequences xi (outcomes) monotonically either:
increasing or decreasing.
Also, regarding the input information on consequences, the crite-
ria may be either: continuous or discrete.
The combination of both kinds of preferences (increasing or de-
creasing) for xi in (1) is given by vi (xi ), so that whichever is the case:
vi (bi ) = 1 and vi (wi ) = 0. Explicitly, for xi monotonically decreas-
ing, the minimum value of xi = bi (most desirable) and the maximum
value of xi = wi (least desirable). For both monotonically increasing
and decreasing consequence xi , the convention is the same, such that:
xi  P xi  .
For discrete criteria, instead of a value function, vi (xi ) represents
the value of a discrete consequence xi . In these cases few options are
available for xi  , xi  . For instance, let xi be a discrete criterion with five
levels: y1 P y2 P y3 P y4 P y5 . Then, bi = y1 and wi = y5 and only three
options are considered for xi  and x : y2 , y3 , y4 .
The next section shows the elicitation process, in which the
weight space in (9) is obtained.
3.4. Elicitation process
The DSS system works as illustrated in Fig. 4, which presents the
procedure and illustrates how the main issue raised in this method,
associated with its flexibility, works.
3.4.1. Step related to intra-criteria evaluation
Step 1 consists of an intra-criteria evaluation, thereby obtaining
the value functions vi (xi ) for each criterion i. In many cases this may
be a linear function, as suggested by Edwards and Barron (1994). In
that case, it should be ensured that the proper normalization proce-
dure is applied, since the elicitation procedure in an additive model
depends on the scale adopted, which is considered in the equations
of the model for computing the weights, as shown in Section 3.2. For
the procedure applied in this paper the scale is from 0 to 1, in which
0 and 1 respectively represent the minimum and the maximum value
performances in the criterion.
In this Step the output is the value function vi (xi ), obtained as per
procedures well established in the literature (Keeney, 1992; Keeney
& Raiffa, 1976; Riabacke et al., 2012). The input is the outcome or per-
formance of each criterion (xi ) for each single alternative, in which
the value function vi (xi ), may be applied. There are either of two pos-
sibilities for the DSS: (a) it supports the elicitation of vi (xi ); or (b) it
support the input process for parameters of standard vi (xi ) functions.
3.4.2. Steps related to ranking criteria weights
3.4.2.1. Step 2.1. Step 2.1 consists of eliciting the order of values for
ki . The steps 1 and 2.1 are important preliminary parts of the method
proposed, just as they are in many previous methods that have been
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191 185

Step 3.1 Set values for test distribution of weights

No Closer to Yes
Set constraint uniform pattern? Set constraint

Set heuristic Set heuristic

B Computation of value xi for DM

Step 3.2
Ask preference to the DM
No
No answer or See partial
inconsistency Output
DM choice results
partial Finalize?
results
Inconsistency No Answer
Preference Yes
answer
Explanation
Set xi to:
xi’, xi’’, or x1I
Restart
DM Set Step 3.3
choice initialization
Continue Compute
elicitation B
Go back LPP
one step Register
Set previous reason
Set Unique
xi’ or xi’’ No
next solution?
Block pair cycle
B of criteria Yes
Step 3.4
Set next No All criteria Yes
cycle Finalization
blocked?

Fig. 5. Steps of the DSS related to evaluation of weights.

proposed in the literature (Barron & Barrett, 1996a; Barron & Barrett,
1996b; Keeney, 1992; Keeney & Raiffa, 1976).
In step 2.1, ties in the criteria used may be identified. In this case,
one of the criteria should be chosen to be used in the assessment.
The choice of which criterion i is to be used in the assessment is
based on which criterion has an outcome which is easier for the DM
to compare. The DSS presents a note to the DM, asking him/her to
choose one of the criteria. The DM is instructed to consider a few
issues related to the ability of interpretation for each outcome. For
instance, an objective outcome (quantitative; for instance, cost in a
known monetary unit) may be preferred to a subjective one (quali-
tative; for instance, company image). An analyst can give additional
support on this task. If a tie is not identified in step 2.1 (that is, the
DM may not perceive at this step that two criteria have the same
weights), it could be identified in step 3.2. However, it seems that it
would be rare to identify ties in that step, since they may most prob-
ably be found in step 2.1. In any case, the DSS adapts the LPP for this
condition.
3.4.2.2. Step 2.2. Step 2.2 tries to solve the problem with the avail-
able weight space, similar to that shown in (3), but in which strict
inequality is replaced by a greater than or equal condition in the LPP
model. This is equivalent to assuming that vi (xi  ) = 1 and vi (xi  ) = 0
in (9), for all i.
If a unique solution is not found, step 3.1 is started; otherwise,
the process is finalized. In the finalization step, the ranges of weights
supporting the solution is computed and produced in a report with
the final recommendation.
Steps 3.1–3.4 are the main parts of FITradeoff (Fig. 5). Neverthe-
less, the previous steps and step 2.2 play an important role in this
concept of a flexible elicitation process. Although a situation may be
found, in which a unique solution is found in Step 2.2, as shown in
the first example taken from the literature on supplier selection, such
situations are infrequent. Sarabando and Dias (2009) tested several
decision rules to evaluate the performance of these rules in problems
involving MAUT/MAVT with 5, 10 and 15 criteria and 5, 10, 15, 25 and
50 alternatives based on a specific weight vector defined by a ROC ap-
proach. From the study by Sarabando and Dias (2009), it is possible to
infer on how the information elicited from DM to define the space of
weights may perform based on the structure of the set of alternatives
in terms of dominance relations. They found that for problems with 5
criteria more than 50% of the alternatives from their experiment are
dominated on average, and this number increases as the number of
alternatives for the problem increases (5, 10, 15, 25, 50). When the
186 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
number of criteria increases (5, 10, 15), the average number of domi-
nated alternatives decreases.
Thus, Step 2.2 is relevant in order to evaluate a rare situation, just
in case, if there is a problem that has above the average number of
dominated alternatives based on the space of weights defined by the
ranking of the weights. Thus, as more information is elicited in the
next steps, and the space of weights is reduced, more alternatives can
be cleared as dominated alternatives.
3.4.3. Steps related to evaluation of criteria weights
Steps 3.1 to 3.4 are the main parts of the proposed method, which
are illustrated in Fig. 5. In these subsequent steps, the values for xi 
and xi  are elicited. These steps are run until either: a unique solution
is found, or the DM chooses not to give additional information.
3.4.3.1. Step 3.1. Step 3.1 is a preparation for collecting inputs from
the DM. In this step the output is a new set of values for xi  or xi ’’,
which is presented to the DM, in the subsequent step. First of all an
initial set of vectors for xi ’ and xi ’’ has been set at step 2.2, such that
vi (xi  ) = 1 and vi (xi  ) = 0.
In this Step a computation is conducted in order to prepare the
next question to the DM, so as to set a value for xi , in order to eval-
uate the value for xi  or xi  . The procedure for choosing a value for
xi is a heuristic, in which the objective is to minimize the number of
questions to the DM, thus making the procedure easier.
It should be noted that the heuristic depends on the distribution
of weights. Two basic patterns are considered for this distribution: a
modal pattern and a uniform pattern. In the former the weights have
a concentration of first ranked criteria weights with higher values and
then the values decay very much for other criteria, since they are nor-
malized and sum up to 1. In the latter the weights have values closer
to a uniform distribution. For a uniform distribution the weights are
equal to 1/n.
In order to choose the heuristic, the following evaluation of distri-
bution of criteria weights is introduced. The DM is asked to compare
the first and the last ordered criteria weights by choosing between
the following consequences: bn and x1 , such that v1 (x1 ) = 0.5, in a
context similar to Fig. 3 (xi = wi , for any other criteria). Considering
Eqs. (7) and (8), if the DM chooses consequence with bn , it means
that the ratio kn /k1 > 0.5, indicating a distribution of weights with
a uniform pattern. Otherwise, kn /k1 < 0.5 and the distribution has a
modal pattern. For instance, if kn /k1 = 1, then the weights have a uni-
form distribution. At first, this evaluation was introduced to choose
the heuristic. Subsequently, it was found useful to include this infor-
mation with the inequality obtained, as a new constraint in the LPP
model, in order to possibly shorten the process.
Let us consider a heuristic for the modal pattern. This heuristic
considers the range = vi (xi  ) – vi (xi  ), for all criteria. Thus, the crite-
rion with the maximum range is chosen for the subsequent question
which considers the consequence associated with xi such that v(xi )
= (v(xi  )- v(xi  ))/2. This means that this value is the midpoint of the
range for that particular criterion. It is expected to be cognitively eas-
ier for the DM to work with a larger range (xi  –xi  ). It also considers
the fact that the narrower the range (xi  –xi  ), the more probable it
is that a unique solution will be found. At the beginning of this Step,
the range is the same for all criteria. Hence, the DSS begins with the
criteria with the highest weights (i = 1). Therefore, the initial value of
xi is such that v(xi ) = 0.5v(bi ) = 0.5, since v(bi ) = 1.
For the uniform pattern, a more appropriate heuristic is applied,
changing the first two questions. These two questions consider the
value of xi closer to the best consequence bi , which is illustrated in
the second application given in Section 4.2. In that case, in the first
question v(xi ) = 0.75v(bi ) = 0.75. Then, if the DM’s answer confirms
the assumed distribution of weights, then, a second question is put on
the same pair of criteria, such that, v(xi ) = 0.875v(bi ) = 0.875. These
two kinds of questions are subsequently applied to the first half pair
of ordered criteria. After that, the basic rule of the previous heuristic
is applied, considering xi such that v(xi ) = (v(xi  )- v(xi  ))/2. Compar-
ing this procedure with the previous one, it skips the first question
(with v(xi ) = 0.5v(bi )). The analysis of these heuristics and other pos-
sibilities should be the core for another study since they may have an
important impact in order to shorten the process.
The heuristic for a discrete outcome is slightly different. As pre-
viously mentioned in this case there are a few options for xi ’ and xi ’’
and the heuristic applied consists of choosing xi = (xi  –xi  )/2, if the
number of levels in the discrete scale is odd. Otherwise, two options
in the middle are available and the greater value is arbitrarily chosen.
For instance, for a discrete scale with six levels: y1 P y2 P y3 P y4 P y5
P y6 . Then, bi = y1 and wi = y6 and only four options are considered
for: xi  and xi  : y2 , y3 , y4 , y5 . Then, xi = y3 .
The result of Step 3.1 led the process to Step 3.2, which initiates a
new phase in the interaction with the DM.
3.4.3.2. Step 3.2. Step 3.2 continues the process of elicitation by
putting questions that ask the DM to choose between two conse-
quences, taking into account the new consequence xi for evaluation
of xi  or xi  .
The DM may answer in one of the following ways (see Fig. 5):
“Preference answer”, “see partial results”, “No answer”, or “inconsis-
tency”.
For the option “Preference answer”, the DSS shows an illustration
similar to Fig. 2 (in which B is considered instead C), and the DM has
three options:
• Consequence A, in which case the DSS sets xi  = xi ; or
• Consequence B, in which case the DSS sets xi  = xi ; or
• Indifference between A and B, in which case the DSS sets xi I = xi .
If the DM is able to choose one of three options above, which is
expected to happen in most cases, then Step 3.3 is conducted and the
LPP model (10) will run.
If the DM chooses “see partial results”, the intermediate result
given in Step 3.3 (in the previous cycle) is presented. In this option,
the DM can see the subset of potentially optimal alternatives. This
may support the DM in making a choice of either: proceeding with
the flexible elicitation (do not finalize, as per Fig. 5, in Step 3.2) or
reducing the amount of information given, by choosing ‘finalize’, as
per Fig. 5. For instance, if the DM considers that the small subset of
potentially optimal is adequate, at some point thereafter, he/she may
choose to stop the process and accept the recommendation given for
subsequent analysis of these few alternatives.
The DM may give a different output in Step 3.2, choosing: “No
answer”. It should be noted that the DM is asked to answer with a
choice, only if he/she is in no doubt about his/her response on com-
paring the consequences. If the DM hesitates in giving an answer, the
process goes either to Step 3.1 or 3.4. If the DM does not know the
answer for this criterion, the process goes back to Step 3.1 in order
to compute another question, and considers the next set of criteria. If
this happens again for all sets of criteria, then the process goes to Step
3.4, which is regarded as the finalization. This finalization is given
using partial information, considering the subset of alternatives that
can be considered for the present set of weights. Although this may
be considered a rare situation, it still gives a partial solution compat-
ible with the partial information that the DM can provide.
Finally, the DM may choose to give an indication of a possible “in-
consistency”. Given the way in which the questions are designed, an
inconsistent answer by the DM, in this step, is not possible, since the
DSS puts questions that systematically seek to reduce the range =
vi (xi  )–vi (xi  ), in such way that it pursues a specific direction without
redundancy. However, an option for “inconsistency” is included in the
DSS (see Fig. 5), so the DM can evaluate if the questions are failing to
get closer the value of indifference xi I and if they are going in the
wrong direction. This situation may occur, should a previous answer
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
have been wrong. That is, if the DM chooses this option, it is because
she/he feels that she/he gave a wrong answer earlier, in which case,
as shown in Fig. 5, the DM may give some explanation and report this
kind of inconsistency. Also, the DM can be assisted by an analyst in
such a situation.
3.4.3.3. Step 3.3. Step 3.3 consists of trying to solve the problem re-
garding the present weight space, which is based on (9), given the
information so far available; that is, the vectors for xi  and xi  . The
alternatives are compared in order to be classified as per definitions
1, 2 and 3, using (10), as given in the next section. If a unique solution
is found, then the finalization step 3.4 is conducted. Otherwise, the
dominated alternatives are eliminated. Then, all alternatives found to
be potentially optimal are considered in the subsequent steps and the
process returns to step 3.1, until a unique solution is found.
At this point the DSS may run several cycles of Steps 3.1, 3.2 and
3.3, before finding a unique solution in the test conducted in Step 3.3
(see Fig. 5) and then goes to finalization.
3.5. FITradeoff LPP model
In order to analyze the set of alternatives in the weights space at
the Steps 2.2 and 3.3 of the above process, the DSS classifies the al-
ternatives into three different situations: potentially optimal, domi-
nated, or optimal.
The subset of potentially optimal alternatives has a key role in the
flexible elicitation process. During the elicitation process, in which
the weight space is reduced, any one of these alternatives may be-
come either: a dominated alternative or the optimal alternative. As
long as there is more than one potentially optimal alternative, the
elicitation process continues to ask the DM questions, in order to re-
duce the weight space.
Therefore the DSS model consists of reducing the weight space by
asking the DM for new preference relations P as long as necessary
in order to find a unique solution, that is, a situation in which the
optimal alternative is found. In the operational process this solution
is found when the subset of potentially optimal alternatives has only
one element.
In order to do this, the following LPP is applied for each alternative
j, considering consequences xij , for criterion i and alternative j:
n
 many situations a supplier selection problem is found to be associ-
Max ki vi xi j , j = 1, 2, . . . , m
k1 ,k2 ,...,kn i=1
s.t.
n
 n
ki vi xi j ≥ ki vi (xiz ), z = 1, 2, . . . , m, z = j
i=1 i=1 . (10)
ki+1 ≤ ki vi (xi  ) − ε for i = 1 to n − 1
ki+1 ≥ ki vi (xi  ) + ε for i = 1 to n − 1
n Chai, Liu, and Ngai (2013) present a literature review for papers pub-
ki = 1
i=1
ki ≥ 0, i = 1, 2, . . . , n
The inequalities in (5) and (6) are introduced as constraints, with
a constant ε in order to avoid the strict inequality, not available in the
software command applied for this LPP. The constant ε is assigned in
the software and has value as smallest as possible according to the
precision of the computational tool, so that, it should make the con-
straint in (10) equivalent to (5) and (6), considering the numerical
precision of the computational language.
This LPP is applied for each of those alternatives j, in order to find
if there is any vector of weights, in the weight space, in which, the
maximum value of that alternative j is greater than (or equal to) any
other alternative in the subset. That is why the first constraint is in-
troduced in (10). If there is such a vector of weights, then, this alterna-
tive is classified as potentially optimal. It is possible that no solution
will be found for any particular j, which means that this alternative
is dominated. The DSS performs a test to verify when the remaining
187
subset of alternatives has a unique alternative that is potentially op-
timal, in which case, the optimal alternative is found. Otherwise, the
LPP is used itself to classify alternatives as potentially optimal.
The above LPP is applied to the initial set of all alternatives, for
each alternative j. Subsequently, this LPP is applied to the subset of
potentially optimal alternatives, found in a previous step. When ap-
plying this LPP, the following situations may be found:
• Only one potentially optimal alternative is found (unique solu-
tion). Then, this is the optimal alternative and final solution for
the problem.
• More than one potentially optimal alternative is found. These po-
tentially optimal alternatives form a new subset of alternatives
which will be further analyzed, as they are possible solutions
for the problem. All other alternatives are dominated and are no
longer considered.
The above LPP model includes another constraint which may be
either: kn /k1 > 0.5, or kn /k1 < 0.5, depending on DM’s preference. This
preference is evaluated by comparing two consequences, related to
the first and last ordered (k1 and kn ), for choosing the heuristic, as
explained in the description of Step 3.1.
There are two particular conditions that lead to exceptions in (10):
• A circumstance in which the DM is able to specify an indiffer-
ence relation I and identify the consequence xi I , such as illus-
trated in Fig. 2. In this case, the DSS adapts the LPP model for
this condition, considering the constraint ki+1 = vi (xIi )ki instead
of ki+1 ≤ vi (xi  ) ki −ε and ki+1 ≥ vi (xi  )ki + ε .
• A situation in which ties are identified in the criteria used, as de-
scribed in Section 3.4.2.1, leads the DSS to adapt the LPP model for
this condition, considering the constraint k i = ki+1 . This is equiv-
alent to making vi (xi  ) = vi (xi  ) = 1.
4. FITradeoff method applied to supplier selection problems
Supplier selection is amongst the main problems studied in the
management literature, in which many supplier selection multicri-
teria models may be found. As stated by Xia and Wu (2007), sup-
plier selection is a multi-criteria decision making problem, which
includes qualitative and quantitative criteria. Supplier selection is
one of the most important issues in competitive strategies and in
ated with other related problems, such as: contract selection (Brito,
de Almeida-Filho, & de Almeida, 2010; de Almeida, 2001) and out-
sourcing (de Almeida, 2007). Indeed, the supplier selection problem
has been studied extensively in the literature. Ho, Xu, and Dey (2010)
present a literature review for decision models in supply selection
problems, published in international journals from 2000 to 2008.
lished between 2008 and 2012.
In order to illustrate how this procedure works two applications
are given using two characteristic problems found in the literature on
supplier selection.
4.1. Results of supplier selection case 1
Xia and Wu (2007) present a problem with eight criteria and four
alternatives (denoted as Sup, for supplier) and proposed a multi-
objective mathematical model. This paper uses data from one of the
examples they use, considering first an additive model, as shown in
Table 1. The criteria Price, Technical level, Defects and Repair turn
round are to be minimized, whereas Reliability, On-time delivery,
Supply capacity and Warranty period are to be maximized.
The global weights computed by Xia and Wu (2007), are used as
reference values in this paper in order to replicate the elicitation pro-
cess when applying the proposed DSS. Then, by applying the proce-
dure illustrated in Fig. 4, a solution was found in Step 2.2. The solution
188 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191

Table 1
Data for supplier selection case of Xia and Wu (2007).

Supplier(Sup) Price ($) Technical level Defects (rate) Reliability (rate) On-time delivery Supply capacity Repair turn round Warranty period

Weights 0.4321 0.0637 0.1173 0.0536 0.1333 0.1 0.0667 0.0333

Sup 1 55 2 0.04 80 0.85 400 2 4
Sup 2 40 1 0.01 95 0.95 700 1 3
Sup 3 45 1 0.02 90 0.98 600 1 3
Sup 4 50 3 0.06 70 0.9 500 3 4

Fig. 6. Ranges of weights for the Xia and Wu (2007) case.

was the same as that found by Xia and Wu (2007), which is the Sup-
plier 2. This means that a unique solution was found for the space of
weights, in which xi  = 1.0 and xi  = 0, for all criteria.
Fig. 6 illustrates one of the outputs of the DSS, showing the ranges
of feasible weights for which Supplier 2 is the most preferred. This il-
lustrates that this particular case has a very robust result, with a large
range of weights for the main criteria in descending order. The range
of weights for each criterion is: Price (1–0.125); Technical level (0.5–
7.1 × 10−11 ); Defects (0.333 – 0); Reliability (0.25 – 0); On-time de-
livery (0.2 – 0); Supply capacity (0.167 – 0); Repair turn round (0.143-
0); Warranty period (0.125 - 0). These ranges of weights result from
LPs models, considering maximum and minimum weights, subject to
constraints in (10) for the winning alternative.
This example illustrates that a minimal effort would be necessary
using FITradeoff, for some particular cases. Of course this would hap-
pen only in a case where alternative performances per criterion have
a range of data which allows such a situation to arise. This is not al-
ways the case, as shown in the next example.
4.2. Results of supplier selection case 2
Barla (2003) presents another supplier selection problem with
seven criteria and ten alternatives, representing subcontractors (de-
noted by Subc). The data from this problem were used in FITradeoff
and rearranged as shown in Table 2, in which all criteria are to be
maximized. This study recommends Subcontractor 1, after using the
additive model. It can be observed that the alternatives for criteria
“Geographical condition” and “Price” have the same performance and
this problem could be simplified, thus reducing the number of crite-
ria. However, this study kept the example as analyzed in Barla (2003)
in order to facilitate the comparison with the same basis of weights.
In this example the weights have values that are close to each
other, following the uniform pattern for the distribution of weights.
Following the elicitation process as described in Section 3.4, after sev-
enteen cycles, the result is finally found, the recommendation being
alternative Subc 1, as in Barla (2003).
Table 3 shows the cycles and inputs considered in the process for
this case. For each cycle, the second column shows Consequence A
and the third shows Consequence B. Consequence B consists of the
best consequence of the criterion with the least weight in the pair
of criteria being compared. Consequence A consists of the value indi-
cated of xi  or xi  for the criterion with the greatest weight in the pair
of criteria being compared. This value will be assumed as xi  or xi 
depending on the DM’s answer. If the DM chooses A, then the value
is xi  . Otherwise it is xi  .
The last column of Table 3 shows the intermediate results found
during each cycle, which consists of the subset of potentially optimal
alternative, found at the end of Step 3.3. For instance, at the end of
the first cycle, in Step 3.3, this subset has four alternatives. There is
an option in the system in Step 3.2, so that the DM can see this sub-
set and also data of the range of weights for this result, including a
plotted line.
It is important to observe that there is a tie between the criteria of
financial condition and of geographical condition, which could most
probably have been perceived in Step 2.1, as usually happens in the
standard tradeoff procedure. The identification of this tie in Step 2.1
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191 189

Table 2
Data of Barla’s supplier selection problem (Barla, 2003).

Criterion Quality organization Service Capability Financial condition Geographical condition Reliability Price
Weights k1 = 0.17 k2 = 0.16 k3 = 0.15 k4 = 0.14 k5 = 0.14 k6 = 0.13 k7 = 0.11

Subc 1 59.6 64 55 80 100 61 40

Subc 2 60.3 10 70 100 100 83.3 40
Subc 3 51.3 44 55 80 100 66.6 40
Subc 4 57.6 53.5 45 60 100 39.3 40
Subc 5 59.6 22.5 80 100 100 58.3 40
Subc 6 49.6 41 70 80 100 19.3 40
Subc 7 58.3 45.5 51.5 60 100 35.6 40
Subc 8 56.6 71 51.5 60 100 52.6 40
Subc 9 59 71 83 60 100 36.6 40
Subc 10 61.3 57 43.5 80 100 42.6 40

Table 3
Cycles and inputs considered in the process.

Cycle Consequence A Consequence B: Best of DM’s choice Potentially optimal alternatives

1 75 of Quality organization Service B Subc 1, Subc 2, Subc 5, Subc 9

2 87.5 of Quality organization Service B Same as previous cycle
3 75 of Service Capability B Same as previous cycle
4 87.5 of Service Capability B Same as previous cycle
5 75 of Capability Finan. condition B Same as previous cycle
6 87.5 of Capability Finan. condition B Same as previous cycle
7 75 of Finan. Condition Geog. condition B Same as previous cycle
8 75 of Geog. Condition Reliability B Same as previous cycle
9 75 of Reliability Price B Same as previous cycle
10 87.5 of Finan. Condition Geog. Condition B Same as previous cycle
11 87.5 of Geog. Condition Reliability B Same as previous cycle
12 87.5 of Reliability Price A Same as previous cycle
13 93.75 of Finan. Condition Geog. Condition B Subc 1, Subc 2, Subc 5
14 93.75 of Geog. Condition Reliability A Same as previous cycle
15 81.25 of Reliability Price B Same as previous cycle
16 95 of Quality organization Service A Subc 1, Subc 2, Subc 5
17 95 of Service Capability A Subc 1

leads the process to finding the unique solution at Step 14. Table 3
shows the application of FITradeoff considering the worst possibility
in order to illustrate an extreme situation.
Fig. 7 shows the ranges of weights for each criterion, for the
unique solution found in the last run of Step 3.3. The ranges of
weights are: Quality organization (0.19–0.16); Service (0.17–0.15);
Capability (0.16–0.14); Financial condition (0.15–0.13); Geographical
condition (0.15–0.13); Reliability (0.14–0.11); Price (0.12–0.09).
Fig. 7 illustrates that for this particular case, the solution does not
have a similar robust result, like that of the former case. In this case,
although still robust, the solution has small ranges of weights for the
criteria.
4.3. Discussion on the results
The results given for these two cases are consistent with their ref-
erences. This has been checked with other applications and simula-
tions of a random data set. These and other cases have been analyzed
using FITradeoff. The two cases presented above show the key factor
of FITradeoff, which is the flexibility in the elicitation process. This
feature of flexibility enables the DSS to ask the DM for a lesser amount
of information than that required in the classical procedure.
These cases represent two extreme situations. That is why they
were chosen, since they can represent completely distinctive situa-
tions in the elicitation process, when applying the FITradeoff method.
In the Xia and Wu (2007) case, a solution can be found with only the
information for ordering the criteria, in Step 2.1. On the other hand,
in the Barla (2003) case, several cycles are required.
Another distinctive characteristic of these two cases is associated
with the robustness of the results. In the first case, as illustrated in
Fig. 6, there is a broader range of weights, while in the Barla (2003)
case, the range of weights is narrower, as illustrated in Fig. 7. This
does not mean that this result of the latter is not good, since it is in ac-
cordance with the DM’s preference structure. Moreover, it should be
emphasized that this may happen in real situations and under such
conditions we have to face up to this reality. Indeed, this is a second
reason for choosing this case of Barla (2003). Even so, this shows that
by using the FITradeoff system, one can still reduce the amount of
information to be obtained from the DM in the elicitation process,
compared with the classical tradeoff procedure.In addition, the DSS
system provides this information, and is useful for an analysis of the
robustness of the result for each particular situation.
Regarding the establishment of a benchmark for the number of
questions in the elicitation process, an important remark should be
given. The classical tradeoff procedure requires the assessment of
(n−1) indifference relations to establish weights for n criteria. Unfor-
tunately, in many studies, (n−1) has been applied as a usual bench-
mark for the number of questions. However, when assessing DM’s
preferences, in a real situation, a facilitator usually starts by asking
softer questions related to a strict preference relation P. This is done
by putting questions with values of xi  and xi  close to bi and wi  ,
respectively, in order to compare consequences similar to those il-
lustrated in Fig. 2. This is highly recommended, instead of asking the
indifference value (xi I ) straightaway. This may reduce inconsistencies
by calibrating DM and imposing consistency in the DM judgment pro-
cess. Therefore, a reasonable benchmark for a real elicitation situation
would be q(n−1), where q−1 is the number of questions based on re-
lation P, to be put forward before the final question of indifference
relation. The minimum acceptable for this would be two questions,
each one of them close bi and wi ’, which would make a benchmark of
3(n−1).
Using this benchmark for the Barla (2003) case, 3(n−1) = 18, in the
FITradeoff system the amount of question (17) to the DM is reduced,
compared with the classical tradeoff procedure, if this benchmark is
190 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
Fig. 7. Ranges of Weights for the Barla (2003) case.
applied. Also, the kind of question is much softer. Otherwise, apply-
ing the benchmark of (n−1) = 6, although the FITradeoff system has
11 more questions, all those questions are cognitively much easier.
That is, the preference relation P requires less cognitive effort than
the indifference relation, as it has been observed in other studies, in-
cluding Weber and Borcherding (1993). Also, this is associated with
the use of partial information, instead of complete information, since
the former uses inequalities and the latter uses equations, such as in
the classical tradeoff procedure.
With regard to the computational complexity of the system, it has
been observed in all tests using cases from the literature and the ran-
dom data set that the speed at which the interactive process with the
DM was undertaken has been quite sufficient. Conducting Step 3.3
using the LPP model was almost imperceptible to the DM, whose first
version has been implemented with MATLAB (de Almeida, Costa, de
Almeida, & de Almeida-Filho, 2014).
The use of the flexible elicitation method can avoid being tedious
and time consuming and also the possibility of minimizing inconsis-
tencies that may occur in long processes, such as those pointed out
by Edwards and Barron (1994).
Therefore, the DSS proposed can facilitate the use of the classical
tradeoff elicitation procedure, with the novel increments introduced,
while considering other paradigms, such as the use of partial infor-
mation and the use of the concept of flexibility for the elicitation pro-
cess.
5. Conclusions
This paper presents an elicitation method for implementing a
multicriteria additive model, using the concept of flexible elicitation
in order to improve the applicability of the classical tradeoff elicita-
tion procedure. The method is built into a decision support system
(DSS), which can be obtained upon request from the authors. It has
been discussed and illustrated how the proposed method improved
the elicitation process, compared with the classical tradeoff proce-
dure, requiring less information from the DM and possibly providing
a more consistent result.
This paper shows some prospects for the future directions of other
studies. Since the present paper does not aim at optimizing the num-
ber of questions, unlike other studies (Holloway et al, 2003; White
III & Holloway, 2008), a future study could usefully be conducted on
this matter, with a view to seeking to improve the heuristic for com-
puting the value for the DM’s evaluation, regarding xi  or xi  in Step
3.1. Behavioral decision making experiments on this method, sim-
ilar to those performed in other studies (Borcherding et al., 1991;
Poyhonen & Hämäläinen, 2001; Weber & Borcherding, 1993), may
be applied in order to evaluate the cognitive features of the new
method. The use of flexible elicitation may be considered for other
methods such as MAUT, in which choices amongst lotteries would be
considered.
Role of the funding source
This work had partial support from CNPq (the Brazilian Research
Council). The CNPq was not involved in the study or in writing this
paper.
Acknowledgments
The authors would like to acknowledge the Editor and the anony-
mous reviewers for their insightful and positive critique of a previous
version of this paper in which they identified its potential and for
making valuable suggestions by means of which they encouraged the
authors to improve its presentation.
References
Bana e Costa, C., De Corte, J.-M., & Vansnick, J.-C. (2005). On the Mathematical Foun-
dation of MACBETH. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple Criteria
Decision Analysis: State of the Art Surveys (pp. 409–437). New York: Springer.
Barla, S. B. (2003). A case study of supplier selection for lean supply by using a mathe-
matical model. Logistics Information Management, 16, 451–459.
Barron, F. H., & Barrett, B. E. (1996a). Decision quality using ranked attribute weights.
Management Science, 42, 1515–1523.
Barron, F. H., & Barrett, B. E. (1996b). The efficacy of SMARTER – simple multi-attribute
rating technique extended to ranking. Acta Psychologica, 93, 23–36.
Barron, F. H. (1992). Selecting a best multiattribute alternative with partial information
about attribute weights. Acta Psychologica, 80, 91–103.
Borcherding, K., Eppel, T., & von Winterfeldt, D. (1991). Comparison of weighting judg-
ments in multiattribute utility measurement. Management Science, 37, 1603–1619.
Brito, A. J., de Almeida-Filho, A. T., & de Almeida, A. T. (2010). Multi-criteria decision
model for selecting repair contracts by applying utility theory and variable inter-
dependent parameters. IMA Journal of Management Mathematics, 21, 349–361.
Chai, J., Liu, J. N. K., & Ngai, E. W. T. (2013). Application of decision-making techniques
in supplier selection: a systematic review of literature. Expert Systems with Appli-
cations, 40, 3872–3885.
Danielson, M., Ekenberg, L., Idefeldt, J., & Larsson, A. (2007). Using a software tool for
public decision analysis - the case of Nacka municipality. Decision Analysis, 4(2),
76–90.
Danielson, M., Ekenberg, L., Larsson, A., & Riabacke, M. (2014). Weighting under am-
biguous preferences and imprecise differences. International Journal of Computa-
tional Intelligence Systems, 7(1), 105–112.
de Almeida, A. T. (2001). Multicriteria decision making on maintenance: spares and
contracts planning. European Journal of Operational Research, 129, 235–241.
de Almeida, A. T. (2007). Multicriteria decision model for outsourcing contracts se-
lection based on utility function and ELECTRE method. Computers and Operations
Research, 34, 3569–3574.
de Almeida, A. T., Costa, A. P. C. S., de Almeida, J. A., & de Almeida-Filho, A. T. (2014). A
DSS for resolving evaluation of criteria by interactive flexible elicitation Procedure.
In F. Dargam, J. E. Hernández, P. Zaraté, S. Liu, R. Ribeiro, B. Delibasic, & J. Pap-
athanasiou (Eds.), Decision Support Systems III - Impact of Decision Support Systems
for Global Environments (pp. 157–166). Springer LNBIP 184 (Lecture Notes in Busi-
ness Information Processing).
de Almeida, A. T., Cavalcante, C. A. V., Alencar, M. H., Ferreira, R. J. P., de Almeida-
Filho, A. T., & Garcez, T. V. (2015). Multicriteria and multiobjective models for risk,
reliability and maintenance decision analysis. International Series in Operations Re-
search & Management Science: vol 231. New York: Springer.
Dias, L. C., & Climaco, J. N. (2000). Additive aggregation with variable interdependent
parameters: the VIP analysis software. Journal of the Operational Research Society,
51, 1070–1082.
Edwards, W., & Barron, F. H. (1994). SMARTS and SMARTER: improved simple methods
for multiattribute utility measurement. Organizational Behavior and Human Deci-
sion Processes, 60, 306–325.
 A.T. de Almeida et al. / European Journal of Operational Research 250 (2016) 179–191
Eisenführ, F., Weber, M., & Langer, T. (2010). Rational Decision Making. Heidelberg, Ger-
many: Springer.
Eum, Y. S., Park, K. S., & Kim, S. H. (2001). Establishing dominance and potential op-
timality in multi-criteria analysis with imprecise weight and value. Computers &
Operations Research, 28, 397–409.
Fishburn, P. C. (1967). Methods of estimating additive utilities. Management Science, 13,
435–453.
Greco, S., Matarazzo, B., & Slowinski, R. (2008). Ordinal regression revisited: multiple
criteria ranking using a set of additive value functions. European Journal of Opera-
tional Research, 191, 416–436.
Greco, S., Mousseau, V., & Slowinski, R. (1999). Rough approximation of a preference
relation by dominance relations. European Journal of Operational Research, 117, 63–
83.
Hazen, G. B. (1986). Partial information, dominance, and potential optimality in multi-
attribute utility theory. Operations Research, 34(2), 296–310.
Ho, W., Xu, X., & Dey, P. K. (2010). Multi-criteria decision making approaches for sup-
plier evaluation and selection: a literature review. European Journal of Operational
Research, 202, 16–24.
Holloway, H. A., & White III, C. C. (2003). Question selection for multi-
attribute decision-aiding. European Journal of Operational Research, 148(3), 525–
533.
Hurson, C., & Siskos, Y. (2014). A synergy of multicriteria techniques to assess additive
value models. European Journal of Operational Research, 238, 540–551.
Jacquet-Lagréze, E., & Siskos, J. (1982). Assessing a set of additive utility functions for
multicriteria decision making, the UTA method. European Journal of Operational
Research, 10(2), 151–164.
Jiménez, A., Ríos-Insua, S., & Mateos, A. (2003). A decision support system for multi-
attribute utility evaluation based on imprecise assignments. Decision Support Sys-
tems, 36, 65–79.
Keeney, R. L., & Raiffa, H. (1976). Decision making with multiple objectives, preferences,
and value tradeoffs. New York: Wiley.
Keeney, R. L. (1972). Utility functions for multiattributed consequences. Management
Science, 18, 276–287.
Keeney, R. L. (1992). Value-Focused thinking: A path to creative decision making. Cam-
bridge, MA: Harvard University Press.
Kirkwood, C. W. (1997). Strategic Decision Making: Multiobjective Decision Analysis with
Spreadsheets. Belmont, CA: Duxbury Press.
Kirkwood, C. W., & Corner, J. L. (1993). The effectiveness of partial information about
attribute weights for ranking alternatives in multiattribute decision making. Orga-
nizational Behavior and Human Decision Processes, 54, 456–476.
Kirkwood, C. W., & Sarin, R. K. (1985). Ranking with partial information: a method and
an application. Operations Research, 33, 38–48.
Li, J., Chen, Y., Yue, C., & Song, H. (2012). Dominance measuring-based approach for
multiattribute decision making with imprecise weights. Journal of Information &
Computational Science, 9(12), 3305–3313.
Lofti, V. T., Stewart, T. J., & Zionts, S. (1992). An aspiration-level interactive model for
multiple criteria decision making. Computers & Operations Research, 19(7), 671–
681.
191
Mármol, A. M., Puerto, J., & Fernández, F. R. (2002). Sequencial incorporation of impre-
cise information of multiple criteria decision processes. European Journal of Opera-
tional Research, 137, 123–133.
Mustajoki, J., & Hämäläinen, R. P. (2005). Decision support by interval SMART/SWING
- incorporating imprecision in the SMART and SWING methods. Decision Sciences,
36(2), 317–339.
Mustajoki, J. (2012). Effects of imprecise weighting in hierarchical preference program-
ming. European Journal of Operational Research, 218, 193–201.
Park, K. S. (2004). Mathematical programming models for characterizing dominance
and potential optimality when multicriteria alternative values and weights are si-
multaneously incomplete. IEEE Transactions on Systems, Man, and Cybernetics—Part
A: Systems and Humans, 34(5), 601–614.
Poyhonen, M., & Hämäläinen, R. P. (2001). On the convergence of multiattribute
weighting methods. European Journal of Operational Research, 129, 569–585.
Riabacke, M., Danielson, M., & Ekenberg, L. (2012). State-of-the-art prescriptive criteria
weight elicitation. Advances in Decision Sciences, ID276584, 1–24.
Roy, B. (1996). Multicriteria methodology for decision aiding. Series in Nonconvex Opti-
mization and its Applications: vol 12. New York: Springer.
Salo, A. A., & Hämäläinen, R. P. (1992). Preference assessment by imprecise ratio state-
ments. Operations Research, 40, 1053–1061.
Salo, A., & Hämäläinen, R. P. (2001). Preference ratios in multiattribute evaluation
(PRIME) - elicitation and decision procedures under incomplete information. IEEE
Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 31(6),
533–545.
Salo, A., & Punkka, A. (2005). Rank inclusion in criteria hierarchies. European Journal of
Operational Research, 163(2), 338–356.
Sarabando, P., & Dias, L. (2009). Multi-attribute choice with ordinal information: a com-
parison of different decision rules. IEEE Transactions on Systems, Man, and Cybernet-
ics: Part A, 39(3), 545–554.
Spliet, R., & Tervonen, T. (2014). Preference inference with general additive value mod-
els and holistic pair-wise statements. European Journal of Operational Research, 232,
607–612.
Steuer, R. E. (1976). Multiple objective linear programming, with interval criterion
weights. Management Science, 23(3), 305–316.
Stewart, T. J. (1992). A critical survey on the status of multiple criteria decision making
theory and practice. Omega, 20, 569–586.
Weber, M., & Borcherding, K. (1993). Behavioral infuences on weight judgments in mul-
tiattribute decision making. European Journal of Operational Research, 67, 1–12.
Weber, M. (1987). Decision making with incomplete information. European Journal of
Operational Research, 28, 44–57.
White III, C. C., & Holloway, H. A. (2008). Resolvability for imprecise multiattribute al-
ternative selection. IEEE Transactions on Systems, Man, and Cybernetics: Part A, 38(1),
162–169.
Xia, W., & Wu, Z. (2007). Supplier selection with multiple criteria in volume discount
environments. Omega, 35, 494–504.
Yang, W.-E., & Wang, J.-Q. (2013). Multi-criteria semantic dominance: a linguistic deci-
sion aiding technique based on incomplete preference information. European Jour-
nal of Operational Research, 231, 171–181.