Information Sciences 545 (2021) 487–498

Contents lists available at ScienceDirect

Information Sciences
journal homepage: www.elsevier.com/locate/ins

A benefit-to-cost ratio based approach for portfolio selection

under multiple criteria with incomplete preference information
Eduarda Asfora Frej a, Petr Ekel b,⇑, Adiel Teixeira de Almeida a
a
Universidade Federal de Pernambuco, CDSID – Center for Decision, Systems and Information Development, Av. Acadêmico Hélio Ramos, s/n -, Cidade
Universitária, Recife, PE CEP 50.740-530, Brazil
b
Pontifícia Universidade Católica de Minas Gerais, Programa de Pós-Graduação em Engenharia Elétrica. Av. Dom José Gaspar, 500, CEP 30535610 Belo
Horizonte, MG, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: Benefit-to-cost ratio (BCR) measuring is a useful approach for selecting portfolios of pro-
Received 11 May 2020 jects when they are evaluated applying multiple and conflicting criteria. A multiattribute
Received in revised form 20 July 2020 value function can be used to measure the benefit of each project, which allows one to
Accepted 31 August 2020
evaluate the BCRs, rank projects, and select them according to the available budget.
Available online 16 September 2020
However, here, there is a significant difficulty associated with the inaccuracy in the values
of criteria scaling constants, which may not be exactly known by decision makers (DMs).
Keywords:
Considering this, the present work is directed at overcoming this difficulty by developing
Multicriteria decision-making/aiding
(MCDM/A)
a BCR-based model for selecting portfolios under incomplete information about criteria
Incomplete information scaling constants. During the elicitation process, DMs answer questions on preferences
Preferences elicitation by considering tradeoffs amongst criteria. The provided information is converted into
Portfolio selection inequalities forming a space of criteria weights. These inequalities serve as constraints
Projects prioritization for linear programming models, which are processed to find dominance relations between
Criteria scaling constants projects, considering their BCRs. The process is supported by developed computing tools.
The formation of a portfolio of research and development projects, which are to be exe-
cuted by a Brazilian electric energy utility, is presented to illustrate the paper results
and their practical applicability.
Ó 2020 Elsevier Inc. All rights reserved.

1. Introduction

Portfolio selection problems in organizations consist on the selection, out of a previously defined set of projects, a subset
of them. Each of these projects adds a certain value for the organization, but they also have a specific cost of their implemen-
tation. The available organizational budget for investment in such projects is usually limited and predefined according to
each company’s objectives and priorities. Due to the combinatorial nature of portfolio problems, it is usually impracticable
to evaluate the whole set of feasible portfolios and select the one that adds the greatest value for the organization and fits
within the available budget. Therefore, several approaches for modeling portfolio problems were developed in the literature
for dealing with such situations without the need to compute and evaluate all possible portfolios.

⇑ Corresponding author at: Graduate Program in Electrical Engineering, Pontifical Catholic University of Minas Gerais, Av. Dom José Gaspar, 500, 30535-
610 Belo Horizonte, MG, Brazil. Tel.: +55 31 3319 4305; fax: +55 31 3310 4225.
E-mail addresses: eafrej@cdsid.org.br (E.A. Frej), ekel@pucminas.br, petr.ekel2709@gmail.com (P. Ekel), almeida@cdsid.org.br (A.T. de Almeida).

https://doi.org/10.1016/j.ins.2020.08.119
0020-0255/Ó 2020 Elsevier Inc. All rights reserved.
E.A. Frej et al. Information Sciences 545 (2021) 487–498

A widely used approach for project prioritization is its formalization within the well-known Knapsack problem, when an
optimization model is analyzed with the objective of maximizing the overall value of benefit of the portfolio, subjected to a
budget constraint [9,15,20,21]. According to this approach, the best portfolio is not necessarily formed by the best-ranked
projects, due to the combinatorial nature of the problem. The methodology developed in the present paper, however, treats
the portfolio problematic according to the benefit-to-cost ratio approach. This is another way to realize the portfolio analysis
in a more practical manner. This approach relies on ranking projects in a decreasing order of their benefit-to-cost ratio, and,
then, selecting those projects that fit within the available budget [3,7,26,36]. Through this approach, projects with higher
benefit-to-cost ratio are selected to the portfolio, as long as they fit within the available budget. It differs from the classical
Knapsack problem, which selects the best-evaluated portfolio – instead of the best-evaluated projects – according to a com-
binatorial approach to the problem analysis. The benefit-to-cost ratio approach has the advantage of treating the portfolio
selection problem in a more practical manner, without spending much computational efforts in analyzing the combinatorial
optimization model.
Despite of the selected approach for prioritizing projects, the fact that multiple and conflicting organizational objectives
are inherently involved in portfolio selection problems cannot be neglected. This leads to the need of using multicriteria
decision-making/aiding (MCDM/A) approaches for dealing with portfolio selection. Angelou and Economides [1] combine
Real Options analysis with the Analytic Hierarchy Process (AHP) approach and propose the ROAHP method for portfolio
selection of information and communication technologies projects within a decision analysis framework. Okioga et al.
[32] addresses a renewable energy portfolio problem in the United States based on a cost-benefit method with the use of
the AHP approach. Hajkowicz et al. [22] propose a cost utility analysis for selection of an optimal portfolio of water quality
enhancement projects in Australia. Unlike the traditional cost benefit analysis, in the cost utility analysis benefits are mea-
sured through utility scores over multiple attributes under different units. Lourenço et al. [26] propose a new decision sup-
port system PROBE (Portfolio Robustness Evaluation) for evaluating the efficiency of portfolios considering a cost benefit
approach, in which the benefit of each project is obtained through a multicriteria analysis based on the MACBETH method.
Outranking methods were also applied for handling portfolio problems. Basilio et al. [4] used the PROMETHEE II method
for asset ranking for investment portfolio selection in a Brazilian stock market, in which 21 financial indicators were consid-
ered. De Almeida and Vetschera [12] highlighted a problem of scale that arises in the original PROMETHEE V model for port-
folio selection, and then Vetschera and De Almeida [42] proposed a new approach for the PROMETHEE V method by
incorporating the concept of c-optimal portfolios in order to overcome such scaling issues. Approaches applying fuzzy sets
have also been used for portfolio selection as well as for other problematics in a multiattribute environment [13,14,29,34].
Moreover, hybrid approaches considering outranking methods and fuzzy concepts were also developed for ranking projects
under multiple criteria [17]. Finally, proxy variables based approaches were also applied to rank projects in order to con-
struct efficient portfolios [5].
A literature review on MCDM approaches for dealing with portfolio selection was conducted by Aouni et al. [2]. The
authors of this work performed an appraisal of diverse criteria that have been used for handling portfolio problems. More-
over, they indicate which MCDA techniques were applied for solving portfolio problems: the most frequently used multicri-
teria method was AHP, but other methods such as ELECTRE, PROMETHEE, and TOPSIS have also been widely used. Another
systematic literature review on project portfolio selection problems was carried by Mohagheghi et al. [30]. In this review, the
most widely used criteria and modeling tools are highlighted.
A wide variety of methods that works with partial information about DMs’ preferences can be found in the literature, and
these approaches were mainly motivated due to the difficulty of eliciting weights using traditional methods [24,39,43]. Tra-
ditional approaches often require too much specific preference information of the DM, which makes them unusable in prac-
tice, since it is difficult for DMs for specify the detailed information required. This makes even larger the gap between
theoretical methods and practical applications. Therefore, several MCDM methods that work based on partial information
have been developed in order to facilitate the decision making process for DMs. The results of [10,28,31,38–41] are related
to the choice problematic in MCDM; i.e., when the aim of the problem is to choose a unique alternative or a subset of alter-
natives. For the ranking problematic, a few approaches were also developed (see [19,24,25,27,33]). When it comes to the
portfolio problematic, however, there is a gap in the literature of partial information MCDM methods. It should be empha-
sized here that the indicated above reviews of Aouni et al. [2] and Mohagheghi et al. [30] do not describe any MCDM method
that works with incomplete (or partial) preference information from decision makers, related specifically with the portfolio
problematic.
Therefore, in this context, it is necessary to develop and apply methods which work with incomplete preference informa-
tion for handling multicriteria decision problems, since DMs are not always capable and/or willing to provide the necessary
level of preference information required by traditional methods [43]. This paper is directed at filling the indicated gap and
proposes a multicriteria approach for solving portfolio selection problems on the basis of incomplete preference information
obtained from DMs. This is conducted throughout a flexible process, operated by means of a decision support system (DSS),
applying a benefit-cost ratio model. Hence, our paper differs from existing approaches on what comes to the way in which a
BCR-based model for portfolio problematic is treated throughout a multiple criteria decision approach that considers partial
information about the DMs preferences.
The main motivation for the development of this approach relies on making the decision process easier for DMs and lower
the cognitive efforts spent during this process. The motivation for introducing benefit-to-cost ratio approach within the pro-
posed model is to treat the portfolio problem in a more practical way, without the need to spend a high computational effort
488
E.A. Frej et al. Information Sciences 545 (2021) 487–498

in a combinatorial optimization model, and the actual innovative challenge addressed by this paper is how to deal with a
BCR-based model considering partial/incomplete information about preferences. The results of the paper are illustrated
by the detailed consideration and solution of a practical problem of selecting a portfolio of research and development
(R&D) projects which have to be developed by a big Brazilian electric energy utility.
The paper is organized as follows. Section 2 presents the benefit-to-cost ratio model for projects portfolio selection based
on an elicitation, generally, providing incomplete information on the DM’s preferences. Section 3 presents a practical case of
R&D projects selection for a big Brazilian electric energy utility, which illustrates an application of the model discussed in
Section 2. Section 4 presents the results obtained with the application and a discussion is made upon the main points. Finally,
Section 5 presents the final remarks and conclusions of this work.

2. Multicriteria benefit-to-cost approach for portfolio selection under incomplete information

2.1. Benefit-to-cost ratio approach for projects prioritization

Let us assume first that a set of n indivisible projects P ¼ fp1 ; p2 ;    ; pn } is available for selection, in order to somehow
bring a measure of benefit for an organization. Each project pi has a cost associated to its implementation ci . A limited budget
B is defined by the organization as the maximum amount of money available for spending with the projects. The projects are
evaluated applying multiple and conflicting attributes A ¼ fa1 ; a2 ;    ; am g that are relevant for the organization when choos-
ing which projects is to be prioritized. Each attribute aj has a scaling constant (or weight) kj , which represents the tradeoffs or
substitution rates between criteria. These attributes represent measures of benefit for each project, and therefore the overall
benefit of a project pi can be calculated, according to the Multiattribute Value Theory [23], as a sum of the attributes’ values
weighted by the scaling constant values of each attribute kj as follows:
X
m
bðpi Þ ¼ kj v j ðxij Þ ð1Þ
j¼1

In Eq. (1), bðpi Þ is the overall benefit of project pi , xij is the outcome value of project pi in attribute aj , and v j is the value
function of attribute aj , which is obtained based on a scale transformation of the values of xij , in order to make all attributes
evaluated in the same value scale. Value functions are built through an intracriteria evaluation. The values of v j ðxij Þ can be
calculated as a simple normalization of the values of xij in a 0–1 linear scale [11], or they can be elicited based on DMs’ pref-
erences with the bisection method, differences method or construction of a value qualitative scale (for details on intracriteria
evaluation methods, see Belton and Stewart [6]). Given the benefit of project pi , bðpi Þ, and the cost of project pi , ci , it is pos-
sible to calculate the benefit-to-cost ratio BCRi of each project pi , as follows:
bðpi Þ
BCRi ¼ ð2Þ
ci
Based on the benefit-to-cost ratios of the projects, it is possible to rank them in decreasing order of BCR, and, therefore,
the projects are selected to be part of the portfolio according to this rank, until the budget B is beat. This approach is con-
sidered as a practical strategy for prioritizing projects, and it results to a portfolio that produces the highest benefit for finan-
cial resources which are to be spent [26].
However, it is necessary to indicate that benefit-to-cost ratio models for portfolio selection usually assume that attri-
butes’ scaling constants values are known a-priori, or are obtained through a multicriteria method that gives exact values
for these parameters [3]. At the same time, the assignment of the values for attributes weights in additive models is not
a trivial task. These parameters do not represent only the level of importance of the criteria, but a scaling issue is also
involved, and weights have a meaning of substitution rates when used for aggregating criteria [23]. DMs should evaluate
these parameters by considering how much they are ready to refuse a level of satisfying a certain criterion to obtain a better
level of satisfying another criterion.
Establishing such tradeoff points is cognitively demanding and time-consuming [39,43], and sometimes DMs are not
ready and/or willing to provide such information when eliciting preferences. Considering this, approaches dealing with
incomplete (or partial) information related to criteria tradeoffs can serve as useful ways for conducting the elicitation pro-
cess with much less cognitive efforts and time spent by DMs [8,10,18,19]. The following subsections illustrate how it is pos-
sible to conduct the elicitation of incomplete information on preferences provided by DMs and also how it is possible to
elaborate a recommendation for portfolio selection without exact values of weights provided for evaluating projects benefits.

2.2. Elicitation of scaling constants parameters under incomplete information

In order to calculate the values of benefits of each project in accordance with (1), values of attribute scaling constants are
to be elicited. However, as mentioned above, establishing these values is not easy for DMs, and, therefore, the development
of an approach to deal with incomplete information (i.e., without the need to provide exact values of weights) is a construc-
tive way to apply traditional MCDAs with applying elicitation techniques, which permit one to significantly reduce time and
necessary efforts. Taking this into account, below, we discuss the elicitation process generating, generally, weights which are
489
E.A. Frej et al. Information Sciences 545 (2021) 487–498

not exactly specified. The involved DM establishes preference relations between attributes values, which can be converted
into inequalities with respect to attributes scaling constants ratios. The availability of these inequalities permits one to use
linear programming models in order to elaborate the corresponding recommendations.
First, let us assume that attributes weights are ranked by the DM in decreasing order of preference as follows:

k1 > k2 >    > km ð3Þ

Now, since attribute a1 has a higher relative importance – also considering the consequences range – than attribute a2 ,
there should be a certain intermediate (between worst and best) outcome Y T1 of attribute a1 for which the DM is willing to
trade for the best outcome of attribute a2 , with all other attributes in the same level (let us assume they have the worst out-
come). Since the value function of attributes is defined in such a way that the value of the best outcome is set to 1
   
(v j bestj ¼ 1) and the value of the worst outcome is set to 0 (v j worstj ¼ 0), and based on the additive aggregation function
(1), it is possible to write
 
k1 v 1 Y T1 ¼ k2 ð4Þ

Therefore, Y T1 is the tradeoff point between attributes a1 and a2 . If the DM further specifies the tradeoff points between
attributes a2 and a3 , a3 and a4 , and so on, then a simple equation system can be constructed and solved, so that the values of
scaling constants are found. This is how Keeney and Raiffa’s [23] traditional tradeoff procedure works for eliciting weights
values. However, such tradeoff values Y Tj are definitely not easy to establish, and this is what we call here as complete infor-
mation establishment. Therefore, instead of forcing the DM to find his/her tradeoff point between pairs of criteria, the elic-
itation can be conducted by asking preference relations for the DM with respect to different adjacent attribute levels. For
instance, the DM is asked whether he/she prefers an intermediate outcome Y 1 of attribute a1 or the best outcome of attribute
a2 , with all other attributes in the same level (worst level). By setting the value of Y 1 ¼ Y 01 , if the DM prefers Y 01 than best2 ,
then the following inequality is directly obtained based on Equation (1)
 
k1 v 1 Y 01 > k2 ð5Þ

For another value of Y 1 ¼ Y 001 , 00

if the DM prefers best2 than Y 1 , then the inequality
 
k1 v 1 Y 1 < k2
00
ð6Þ

can be written.
Similarly, the DM can also state preference relations for different pairs of adjacent criteria. In a more generic way, the
following inequalities can be obtained for j ¼ 1;    ; m  1:
 
kj v j Y 0j > kjþ1 ð7Þ

 
kj v j Y 00j < kjþ1 ð8Þ

The inequalities (3), (7), and (8) permit one to construct a feasible set of weights vectors, which we call a feasible weight
space.
8
>
> k1 > k2 > . . . > km
>
>  
>
>
> k1 v 1 Y 01 > k2
>
>  
>
> k1 v 1 Y 001 < k2
>
>  
>
>
>
>
> k2 v 2 Y 02 > k3
>
>  
>
>
> k2 v 2 Y 002 < k3
<
x¼  . .. ð9Þ
>
> v 0
> kjþ1
>
> k j j Y
>
>  
j
>
>
>
> kj v j Y j < kjþ1
00
>
>
>
>
>
> ...
>
>  0 
>
> km1 v m1 Y m1 > km
>
>
>
:k v  00 
m1 Ym1 <k
m1 m

The DM does not necessarily have to specify all preference relations for all pairs of adjacent criteria. Equation (9) repre-
sents, in a generic manner, how the weight space can be constructed. The inequalities that form the space of weights will
depend, in each situation, on which information the DM is willing to provide. Sometimes a decision recommendation can
be drawn with few inequalities only. Of course, since the information provided is partial, the decision recommendation is
also partial – for instance, a partial ranking of projects, in which incomparability relations may arise. As long as the DM
490
E.A. Frej et al. Information Sciences 545 (2021) 487–498

is willing to provide additional preferential information, the recommendation becomes more complete. The following sec-
tion illustrates how a portfolio of projects can be selected with incomplete information of criteria weights, generally, repre-
sented by (9).

2.3. Building a portfolio of projects based on incomplete preferential information

The main goal here is to obtain a ranking of projects based on the benefit-to-cost ratio concept, and after that, select the
projects to enter within the portfolio, based on the available budget B. Our greatest challenge, however, is to build a ranking
of projects without exact information provided on attributes weights. Due to the incompleteness level of information pro-
vided on the values of kj , it is not possible to calculate exact values for the benefit of each project bðpi Þ. Instead of exact values
for attributes weights, the information reflected by (9) leads to a range of values between which each attribute weight kj is.
Let us first introduce the concept of benefit-to-cost ratio difference, which can be computed for any pair of projects
ðpi ; pk Þ, denoted by i;kd BCR and calculated as Equation (10).

bðpi Þ bðpk Þ
d
i;k BCR ¼  ð10Þ
ci ck
The meaning of i;kd BCR is quite straightforward, since it represents nothing besides the different between the benefit-to-
cost ratios of projects pi and pk .
With partial information about attributes weights, it is not possible to calculate the exact value of i;kd BCR 8 i; k. Nonethe-
less, the maximum difference between the benefit-to-cost ratios of projects pi and pk can be obtained by analyzing the fol-
lowing optimization model:
!, !,
X
m
  X
m
max d
i;k BCR ¼ kj v j xij ci  kj v j ðxkj Þ ck ð11Þ
j¼1 j¼1

s.t.
k1 > k2 >    > km
 
k1 v 1 Y10 > k2
 
k1 v 1 Y100 < k2
 
k2 v 2 Y20 > k3
 
k2 v 2 Y200 < k3


 
kj v j Yj0 > kjþ1

 
kj v j Yj00 < kjþ1


 0 
km1 v m1 Ym1 > km
 00 
km1 v m1 Ym1 < km

kj  0; j ¼ 1;    ; m

X
m
kj ¼ 1
j¼1

The analysis of (11) is directed at maximizing the value of i;kd BCR, subjected to the constraints that form the weight space
x (9) with the addition of the normalization and non-negativity constraints. The decision variables in (11) are the attributes
scaling constants kj .
Now, in order to build a ranking of the projects with incomplete provided information, we consider here the concept of
pairwise dominance between projects, taking into account their benefit-to-cost ratio.
491
E.A. Frej et al. Information Sciences 545 (2021) 487–498

Definition: Dominance between candidate projects.

 
Let d
i;k BCR be the optimal solution of (11). A project pi dominates another project pk if and only if d
i;k BCR > 0 and

d
k;i BCR  0; i.e., pi dominates pk if the benefit-to-cost ratio of pi can be greater than the benefit-to-cost ratio of pk for at least
one vector of feasible weights within x, but the benefit-to-cost ratio of pk cannot be greater than the benefit-to-cost ratio of
pi for any vector of feasible weights within x.
The linear programming model (11) is run for all pairs of projects ðpi ; pk Þ 8 i; k; and, then, the dominance condition is
tested for each pair. If the dominance condition is not satisfied for projects pi and pk (nor for pk and pi ), then projects pi
and pk are considered to be incomparable for the current level of obtained preference information. With such incompatibility
relations and dominance relations between projects, a partial ranking of projects can be straightforwardly obtained [19]. The
more information given about attributes weights, the more complete is the ranking.

3. R&D projects portfolio selection: a practical case of the electric energy utility

3.1. Problem description

The portfolio selection problem addressed in this paper is associated with forming a portfolio of R&D projects, which are
to be executed by an electric energy utility, one of the strongest and most important in the electric power segment in Brazil.
A group of consultants, together with engineers of the utility, defined a total of fifteen criteria/attributes to be considered
in the evaluation. These criteria were established according to five key strategic lines for the utility. The first group of criteria
is related to impositions made by the Brazilian National Electricity Regulatory Agency (ANEEL). The second group of criteria
embraces strategic objectives prioritized by the utility. In the third group, there are criteria related to the utility’s market.
The fourth group evaluates aspects related to the execution team of the project. Finally, the fifth group embraces criteria
related to the social, economic and environmental impact of each project. Table 1 summarizes the set of criteria of the
problem.
A total of forty six projects were evaluated with respect to those fifteen criteria according to a five-point Likert scale, in
which 1 represents the worst possible performance and 5 represents the best possible performance. In addition, the cost
related to each project’s implementation was also measured. Table 2 summarizes the consequences matrix of the problem,
and the last column indicates the cost, in Brazilian reals (R$), associated to each project.

3.2. Preferences elicitation process

The elicitation of preferences was conducted within an interactive process with the use of the DSS, which is available, for
free by request to the authors, on the website www.fitradeoff.org.
The estimates given in Table 1 serve as an input for the DSS.
The first information requested from the DM is the criteria scaling constants order, which was given as follows:
kA1 > kA2 > kA3 > kA4 > kB1 > kC1 > kE1 > kD1 > kB2 > kC2 > kD2 > kB3 > kE2 > kC3 > kE3:
After ranking the criteria weights, the system started to make questions to the DM. To answer such questions, the DM had
to think about his tradeoffs between different criteria. Fig. 1 illustrates the first question made for the DM by the DSS.

Table 1
Criteria description.

A. Criteria imposed by the Brazilian National Agency A1. Originality of the project
of Electric Energy (ANEEL) A2. Applicability of the project
A3. Practical Relevance of the project
A4. Reasonability of projects cost structure
B. Strategic criteria B1. Compliance with the objectives, assumptions, and expected results of the utility
B2. Contribution to the integration between the utility segments
B3. Degree of coverage of new digital technologies addressed by the project
C. Market criteria C1. Contribution of the project to create new markets or expansion of existing markets for
the utility
C2. Degree of integration between innovation ecosystem agents (universities, R&D
institutions, startups, and technology-based companies)
C3. Degree of replication of project results in the electricity sector as a whole
D. Criteria related to the competence of the project D1. Qualification of the team and executing entities (technical qualification of the team,
execution team previous experience in R&D projects)
D2. Project quality in terms of presentation of key ideas, clarity, conciseness, and text
structure
E. Social, economic and environmental impact E1. Degree of contribution to increase operational efficiency, cost and loss reduction, and
criteria improvement in service delivery and quality
E2. Degree of contribution to training and qualification of people
E3. Degree of contribution to social and/or environmental development

492
E.A. Frej et al. Information Sciences 545 (2021) 487–498

Table 2
Consequences matrix and projects costs.

Projects A1 A2 A3 A4 B1 B2 B3 C1 C2 C3 D1 D2 E1 E2 E3 Cost (R$)

P A1-1 5 4 5 4 5 4 5 4 5 5 5 5 4 5 3 4,000,000
P A1-2 4 4 4 4 4 3 4 3 4 4 5 4 4 4 3 3,850,000
P A1-3 4 5 4 3 4 3 4 4 4 4 4 4 4 4 3 5,400,000
P A1-4 4 4 4 3 3 4 4 3 3 4 4 3 4 3 3 5,200,000
P A1-5 4 4 3 5 2 4 4 3 4 4 4 3 3 4 3 3,600,000
P A1-6 4 3 4 4 3 3 4 3 3 3 3 4 3 4 3 4,100,000
PA1-7 3 3 4 4 3 3 4 3 3 3 4 3 3 3 3 4,300,000
P A2-1 4 4 4 4 5 4 4 3 4 4 4 4 3 4 5 5,000,000
P A2-2 3 4 4 3 3 4 3 3 4 4 3 3 4 3 4 6,200,000
P A3-1 4 5 4 2 4 5 4 4 3 4 4 3 4 3 3 8,500,000
P A3-2 5 4 4 4 4 3 4 3 3 4 3 4 4 3 3 5,800,000
P A3-3 4 4 3 3 3 4 4 3 3 3 4 3 3 3 3 6,600,000
P A3-4 4 4 4 1 3 3 4 3 3 3 3 3 3 2 3 8,400,000
P A3-5 3 3 4 4 3 3 3 3 4 3 3 1 3 2 3 6,000,000
P A4-1 4 4 4 3 4 5 5 4 4 4 4 5 4 4 4 8,400,000
P A4-2 5 4 4 4 4 4 4 5 4 4 4 4 5 4 3 6,800,000
P A4-3 4 4 3 4 4 3 4 4 4 4 4 3 4 3 3 5,800,000
P A4-4 4 4 3 3 4 4 3 4 3 3 4 3 4 3 3 6,400,000
P A4-5 3 4 4 4 1 3 4 3 3 3 3 3 3 3 3 5,500,000
P A4-6 4 1 3 4 3 3 3 4 3 3 2 3 3 3 3 5,400,000
P A4-7 3 3 2 3 4 1 2 3 3 3 2 3 2 3 3 6,300,000
P A4-8 4 2 3 3 3 3 3 3 2 2 3 3 1 2 3 6,400,000
P A4-9 3 3 2 4 4 3 3 2 2 2 3 2 3 1 2 5,400,000
P A4-10 3 3 2 3 3 3 3 2 1 2 1 3 3 2 3 6,200,000
P A4-11 3 3 1 4 3 2 2 3 2 2 2 2 3 2 3 5,200,000
P A4-12 1 2 2 3 3 3 2 1 2 1 2 2 2 2 2 6,200,000
P A5-1 5 5 5 3 4 5 4 4 4 4 5 5 4 5 4 6,500,000
P A5-2 5 5 5 2 5 4 4 4 3 4 5 5 5 4 3 7,800,000
P A5-3 5 4 5 3 5 4 4 3 3 3 5 4 4 3 3 6,400,000
P A5-4 5 4 4 3 5 4 4 3 3 3 5 5 3 3 3 6,200,000
P A5-5 5 4 4 2 4 4 4 3 4 3 5 4 3 4 3 7,600,000
P A5-6 4 5 4 2 4 3 4 3 3 4 4 4 3 4 3 7,700,000
P A5-7 4 4 4 4 3 3 1 3 3 3 4 3 3 2 3 5,600,000
P A5-8 4 4 3 3 3 2 2 3 3 3 3 2 3 2 1 6,400,000
P A6-1 4 4 4 2 4 4 4 3 4 4 3 4 4 4 3 7,700,000
P A6-2 5 4 4 3 4 4 3 4 3 4 4 3 4 4 3 5,500,000
P A6-3 4 4 4 4 4 3 3 4 3 3 4 3 4 3 3 4,900,000
P A6-4 4 4 3 3 4 4 3 4 3 3 3 4 3 3 3 5,700,000
P A6-5 4 3 4 2 4 3 3 4 3 3 4 3 3 4 3 7,800,000
P A6-6 3 4 4 3 4 3 4 3 4 3 3 3 4 3 3 5,600,000
P A6-7 4 3 3 3 3 3 3 4 3 3 4 3 3 3 3 5,400,000
P A6-8 3 3 4 5 4 3 3 2 3 2 3 3 4 3 3 4,800,000
P A6-9 3 3 4 3 4 4 3 4 2 2 2 3 3 3 3 5,700,000
P A6-10 3 2 3 3 3 3 3 3 4 3 3 2 4 3 3 5,500,000
P A6-11 3 2 3 4 3 3 3 3 4 3 2 2 3 3 3 4,700,000
P A6-12 2 2 2 3 3 3 3 3 3 2 2 2 3 4 3 5,500,000

This question relies on a comparison between two hypothetical consequences that the DM should compare, by stating his
preferences considering tradeoffs amongst different criteria. In particular, in Fig. 1 the DSS asked the DM to choose between a
performance equal to 3 in the first-ranked criterion (A1) and a performance equal to 5 (which is the best possible perfor-
mance) in the last-ranked criterion (E3). To do this, the DM considers the comparison of two hypothetical consequences:
consequence A, with all criteria in the worst level, except for the first-ranked criterion (A1), illustrated by the blue bar;
and consequence B, with all criteria in the worst level, except for the last-ranked criterion (E3), illustrated by the green
bar. All the red bars represent the worst outcome of the other criteria. On the right side of the screen, it is possible to see
the answers’ options: consequence A, if the DM prefers the outcome 3 of A1; consequence B, if the DM prefers the outcome
5 of E3; indifference between those two consequences; or no answer, if the DM thinks this question is too hard to be
answered. In the latter case, the DSS generates another question without loss of information. Also on the right side of the
screen, below the answers’ options, it is possible to see the number of levels in the partial ranking of projects obtained based
on the benefit-to-cost ratio model presented in Section 2. It is possible to see that four ranking levels were found according to
the partial information given until that point (which in this case is the criteria weights order). The DM chose preference for
consequence B in the question of Fig. 1, which means that he/she prefers the maximum level of criterion E3 than a perfor-
mance of 3 of A1. At this point, another preference question was generated, as shown by Fig. 2.
The comparison of consequences put by the DSS illustrated in Fig. 2 is analogous of that of Fig. 1; but now the DM is asked
to compare a performance of 4 of the criterion A1 and the best performance (5) of the criterion A2 (second-ranked criterion).

493
E.A. Frej et al. Information Sciences 545 (2021) 487–498

Fig. 1. First elicitation question made by the DSS.

Fig. 2. Second elicitation question made by the DSS.

However, before answering this question, the DM noted that 16 ranking levels were found after he answered the first elic-
itation question. By clicking on the button ‘‘Show Current Results”, the DM visualized the partial ranking obtained until that
point and, even with the fact that the ranking was not complete, he/she decided that it was enough for the aim of the com-
pany, which was to perform a portfolio analysis. The DSS is flexible and allows the DM to stop the elicitation process when-
ever he is satisfied with the partial results provided. The next section shows the obtained results.

4. Results and discussion

As mentioned in the previous section, the DM decided to stop the process by providing a small amount of information:
ranking of criteria scaling constants and answer to one elicitation question. He decided to do it because the obtained partial
ranking had 16 levels and it was enough for the utility’s purposes. Table 3 presents the ranking of projects, which was com-
puted based on the dominance relations found by the benefit-to-cost ratio model described in Section 2.
494
E.A. Frej et al. Information Sciences 545 (2021) 487–498

Table 3
Projects ranked by benefit-to-cost ratio.

Ranking Projects Cost (R$) Cumulative cost (R$)

1 [P A1-1] 4,000,000 4,000,000
2 [P A1-2] 3,850,000 7,850,000
3 [P A1-5] 3,600,000 11,450,000
4 [P A1-6], [P A2-1] 9,100,000 20,550,000
5 [P A1-3], [PA1-7], [P A6-3], [P A5-1] 21,100,000 41,650,000
6 [P A6-2] 5,500,000 47,150,000
7 [P A1-4] 5,200,000 52,350,000
8 [P A3-2], [P A4-2], [P A4-3], [P A5-3], [P A5-4], [P A6-8] 35,800,000 88,150,000
9 [P A6-4], [P A6-6] 11,300,000 99,450,000
10 [P A5-2], [P A5-7], [P A6-11], [P A4-5], [P A6-7], [P A2-2], [P A4-4] 41,600,000 141,050,000
11 [P A4-1], [P A4-6], [P A5-5], [P A6-9], [P A6-10], [P A3-3] 39,200,000 180,250,000
12 [P A3-5], [P A6-1], [P A5-6] 21,400,000 201,650,000
13 [P A3-1], [P A4-9], [P A6-5], [P A6-12], [P A4-11], [P A5-8] 38,800,000 240,450,000
14 [P A4-7], [P A4-8] 12,700,000 253,150,000
15 [P A3-4], [P A4-10] 14,600,000 267,750,000
16 [P A4-12] 6,200,000 273,950,000

The first column of Table 3 shows the ranking position of the projects indicated in the second column. The third column
reflects the cost associated to the projects within the corresponding ranking position, and the fourth column represents the
cumulative cost, which is given by the cost of the current position’s projects and the cost of all upper positions’ projects.
It is important to highlight that a project only occupies a certain position in the ranking if and only if, according to Def-
inition 1, it is dominated by all other projects in the higher positions. For instance, project [P A1-6] is in the fourth position of
the ranking; this means that this project was dominated by projects [P A1-1], [P A1-2], and [P A1-5], which occupies, respec-
tively, the first, second, and third ranking positions. When there are more than one project within the same ranking level,
this means that these projects may be incomparable to each other for the current level of partial information obtained until
that point. For instance, there are two projects in the fourth level of the ranking: [P A1-6] and [P A2-1]; therefore, neither [P
A1-6] dominates [P A2-1] nor [P A2-1] dominates [P A1-6], and so they are incomparable for the level of partial information
provided by the DM until this point.
For portfolio selection purposes, the utility should first define which budget is available for spending with the R&D pro-
jects, and, then, the portfolio can be selected. For instance, if the budget is set to R$50,000,000, then all projects from the first
position up to the sixth position could be selected for implementation. In other case, if the budget is set to R$120,000,000,
projects up to the ninth position could be selected, and a total of R$20,550,000 would still be available. This means that the
DM could analyze the projects that belong to the tenth position and still choose some of them to be part of the portfolio in
order to complete the budget.
As mentioned in Section 2, the exact values of the criteria scaling constants do not need to be found, since the information
provided is partial. However, upper and lower bounds for these parameters can also be calculated on the basis of linear pro-
gramming. Fig. 3 shows a graphic provided by the DSS, which illustrates the upper limit and lower limit for each criterion
weight. The blue line indicated the upper limits of the weights’ values, while the red line reflects the lower limits of weights’
values, and criteria are ordered from left to right. The range of values between the upper and lower limits allow us to perform
a robustness analysis of the resulting ranking shown in Table 3, because for any vector of weights within those ranges (and
summing up to 1), the ranking of Table 3 remains the same. Since the range in Fig. 3 seems quite large, the ranking can be
considered to be relatively robust to changes in the criteria scaling constants values.
Another graphical output of the DSS is a Hasse diagram of the projects, which shows all the dominance relations between
them with the transitivity reduction property, as illustrated by Fig. 4.
In Fig. 4, each circle represents a project, and the directed arrows represent a dominance relation between two projects.
Different positions in the ranking are separated by dashed lines. This diagram is especially useful for portfolio selection,

Fig. 3. Upper and lower limits for criteria scaling constants.

495
E.A. Frej et al. Information Sciences 545 (2021) 487–498

Fig. 4. Dominance relations between projects.

496
E.A. Frej et al. Information Sciences 545 (2021) 487–498

when the available budget does not exactly match the cumulative cost until a certain position. In the example given above, in
which the company would set the available budget in R$120,000,000, all projects up to position nine would be selected, and
R$20,550,000 would still be available for spending with R&D projects of the tenth position. But this position has a total of 7
projects ([P A5-2], [P A5-7], [P A6-11], [P A4-5], [P A6-7], [P A2-2], and [P A4-4]). So, how would the DM choose which of
them is going to be part of the portfolio and which of them are not? This question could be answered with a deeper analysis
of the dominance diagram in Fig. 4.
By analyzing the projects that belong to the position 10, it is possible to see that projects [P A5-2], [P A5-7], and [P A6-11]
dominates at least one project within position 10, but they are not dominated by any project within this position. Therefore,
they could be promising options for including in the portfolio. Now, the cost of those projects should be analyzed to verify
whether all some of them (or even all of them) fits within the portfolio according to the available remaining budget. Accord-
ing to Table 2, projects [P A5-2], [P A5-7] and [P A6-11] cost, respectively, R$7,800,000, R$5,600,000, and R$4,700,000, sum-
ming up to R$18,100,000, which means that all of them still can be part of the portfolio, since the available remaining budget
is R$20,550,000.

5. Conclusions

In this paper, a new model for selecting a portfolio of projects based on the benefit-to-cost ratio with incomplete infor-
mation provided by DMs was presented. The benefit of a project is calculated according to the multiattribute value function,
based on the multiple criteria through which candidate projects are evaluated. Pairwise dominance relations between pro-
jects are computed using the benefit-to-cost ratio concept, in order to rank the projects using incomplete information about
criteria scaling constants values, whose exact values are considered to be unknown. The use of partial/incomplete preference
information in portfolio selection problems can serve as an effective alternative for those situations where it is undesirable or
impossible to spend much time and/or cognitive efforts in the decision-making process. Experiments with neuroscience
tools have been conducted for evaluating partial information decision approaches and to facilitate the whole process for
DMs, based also on the support of graphical visualization [37].
The results of the paper were applied for solving the R&D projects portfolio selection problem of a big Brazilian electric
energy utility, in which forty six candidate projects were evaluated with respect to fifteen criteria, divided in five main
strategic lines. Right after the DM ranked criteria weights and answered a single tradeoff question, a partial ranking with
sixteen levels was achieved. The DM had the option to continue answering questions in order to obtain a more complete
and refined ranking; however, for portfolio selection purposes, the partial ranking obtained until that point was sufficient.
This particular case shows how the approach proposed in this paper can be useful for portfolio-related decision-making with
few information provided from users. Moreover, the DSS applied for preferences elicitation and benefit-to-cost ratio-based
ranking computation is interactive, and enables DMs to visualize partial results during the process. Graphical visualization
for decision aiding and robustness analysis are also useful features provided by the proposed system.
Since the proposed approach has a preference-driven focus (but not data-driven focus), experimental tests with a broader
range of case studies were not conducted in this research. Instead, the proposed methodology was applied to a practical R&D
portfolio selection problem for an electric energy utility whose engineers and consultants were actually interviewed and
their preferences were processed. However, it should be emphasized that the proposed approached can be applied to any
portfolio selection problem (e.g. [16,35]) under conditions that benefits and costs of each project can be estimated and a
DM is available for having his/her preferences evaluated.

CRediT authorship contribution statement

Eduarda Asfora Frej: Methodology, Software, Writing - original draft, Writing - review & editing. Petr Ekel: Conceptu-
alization, Writing - review & editing, Validation, Supervision, Investigation. Adiel Teixeira Almeida: Conceptualization,
Methodology, Supervision, Project administration.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Acknowledgments

The authors are grateful to CNPq (Brazilian National Counsel of Technological and Scientific Development) for the finan-
cial support received.

References

[1] G.N. Angelou, A.A. Economides, A decision analysis framework for prioritizing a portfolio of ICT infrastructure projects, IEEE Trans. Eng. Manage. 55 (3)
(2008) 479–495.

497
E.A. Frej et al. Information Sciences 545 (2021) 487–498

[2] B. Aouni, M. Doumpos, B. Pérez-Gladish, R.E. Steuer, On the increasing importance of multiple criteria decision aid methods for portfolio selection, J.
Oper. Res. Soc. 69 (10) (2018) 1525–1542.
[3] C.A. Bana e Costa, T.G. Fernandes, P.V. Correia, Prioritisation of public investments in social infrastructures using multicriteria value analysis and
decision conferencing: a case study, Int. Transa. Oper. Res. 13 (4) (2006) 279–297.
[4] M.P. Basilio, J.G. de Freitas, M.G.F. Kämpffe, R. Bordeaux Rego, Investment portfolio formation via multicriteria decision aid: a Brazilian stock market
study, J. Model. Manage. 13 (2) (2018) 394–417.
[5] S.S. Bastiani, L. Cruz-Reyes, E. Fernandez, C. Gómez, G. Rivera, Quality-assessment model for portfolios of projects expressed by a priority ranking, in:
Recent Advances on Hybrid Approaches for Designing Intelligent Systems, Springer, Cham, 2014, pp. 555–565.
[6] V. Belton, T. Stewart, Multiple Criteria Decision Analysis: An Integrated Approach, Springer Science & Business Media, 2002.
[7] M.M. Borrego-Marín, J. Berbel, Cost-benefit analysis of irrigation modernization in Guadalquivir River Basin, Agric. Water Manage. 212 (2019) 416–
423.
[8] P.A.A. Carrillo, L.R.P. Roselli, E.A. Frej, A.T. de Almeida, Selecting an agricultural technology package based on the flexible and interactive tradeoff
method, Ann. Oper. Res. (2018) 1–16.
[9] R.T. Clemen, J.E. Smith, On the choice of baselines in multiattribute portfolio analysis: a cautionary note, Decis. Anal. 6 (4) (2009) 256–262.
[10] A.T. De Almeida, J.A. Almeida, A.P.C.S. Costa, A.T. Almeida-Filho, A new method for elicitation of criteria weights in additive models: flexible and
interactive tradeoff, Eur. J. Oper. Res. 250 (1) (2016) 179–191.
[11] A.T. De Almeida, C.A.V. Cavalcante, M.H. Alencar, R.J.P. Ferreira, A.T. de Almeida-Filho, T.V. Garcez, Multicriteria and Multiobjective Models for Risk,
Reliability and Maintenance Decision Analysis, Springer, 2015.
[12] A.T. de Almeida, R. Vetschera, A note on scale transformations in the PROMETHEE V method, Eur. J. Oper. Res. 219 (1) (2012) 198–200.
[13] P. Ekel, I. Kokshenev, R. Parreiras, W. Pedrycz, J. Pereira, Multiobjective and multiattribute decision making in a fuzzy environment and their power
engineering applications, Inf. Sci. 361–362 (2) (2016) 100–119.
[14] P. Ekel, A.C. Lisboa, J.G. Pereira, D.A.G. Vieira, L.M.L. SilvaD‘Angelo, M.F.S.V., Two-stage multicriteria georeferenced express analysis of new electric
transmission line projects, Int. J. Electr. Power Energy Syst. 108 (3) (2019) 415–431.
[15] T. Fasth, S. Bohman, A. Larsson, L. Ekenberg, M. Danielson, Portfolio decision analysis for evaluating stakeholder conflicts in land use planning, Group
Decis. Negot. (2020) 1–23.
[16] E. Fernandez, C. Gomez, G. Rivera, L. Cruz-Reyes, Hybrid metaheuristic approach for handling many objectives and decisions on partial support in
project portfolio optimization, Inf. Sci. 315 (2015) 102–122.
[17] L. Ferreira, D. Borenstein, E. Santi, Hybrid fuzzy MADM ranking procedure for better alternative discrimination, Eng. Appl. Artif. Intell. 50 (2016) 71–82.
[18] E.A. Frej, L.R.P. Roselli, J. Araújo de Almeida, A.T. de Almeida, A multicriteria decision model for supplier selection in a food industry based on FITradeoff
method, Math. Probl. Eng. 2017 (2017).
[19] E.A. Frej, A.T. de Almeida, A.P.C.S. Costa, Using data visualization for ranking alternatives with partial information and interactive tradeoff elicitation,
Oper. Res. (2019) 1–23.
[20] K. Golabi, Selecting a portfolio of nonhomogeneous R&D proposals, Eur. J. Oper. Res. 21 (3) (1985) 347–357.
[21] K. Golabi, C.W. Kirkwood, A. Sicherman, Selecting a portfolio of solar energy projects using multiattribute preference theory, Manage. Sci. 27 (2) (1981)
174–189.
[22] S. Hajkowicz, R. Spencer, A. Higgins, O. Marinoni, Evaluating water quality investments using cost utility analysis, J. Environ. Manage. 88 (4) (2008)
1601–1610.
[23] R.L. Keeney, H. Raiffa, Decision Analysis with Multiple Conflicting Objectives, Wiley & Sons, New York, 1976.
[24] C.W. Kirkwood, R.K. Sarin, Ranking with partial information: a method and an application, Oper. Res. 33 (1) (1985) 38–48.
[25] J.C.L. López, P.A.Á. Carrillo, D.A.G. Chavira, J.J.S. Noriega, A web-based group decision support system for multicriteria ranking problems, Oper. Res. 17
(2) (2017) 499–534.
[26] J.C. Lourenco, A. Mortone, C.A.B. e Costa, PROBE—a multicriteria decision support system for portfolio robustness evaluation, Decis. Support Syst. 54 (1)
(2012) 534–550.
[27] B. Malakooti, Ranking and screening multiple criteria alternatives with partial information and use of ordinal and cardinal strength of preferences, IEEE
Trans. Syst. Man Cybern. A Syst. Hum. 30 (3) (2000) 355–368.
[28] A.M. Mármol, J. Puerto, F.R. Fernández, Sequential incorporation of imprecise information in multiple criteria decision processes, Eur. J. Oper. Res. 137
(1) (2002) 123–133.
[29] M.K. Mehlawat, A. Kumar, S. Yadav, W. Chen, Data envelopment analysis based fuzzy multi-objective portfolio selection model involving higher
moments, Inf. Sci. 460 (2018) 128–150.
[30] V. Mohagheghi, S.M. Mousavi, J. Antuchevičienė, M. Mojtahedi, Project portfolio selection problems: a review of models, uncertainty approaches,
solution techniques, and case studies, Technol. Econ. Dev. Econ. 25 (6) (2019) 1380–1412.
[31] J. Mustajoki, R.P. Hämäläinen, A. Salo, Decision support by interval SMART/SWING – incorporating imprecision in the SMART and SWING methods,
Decis. Sci. 36 (2) (2005) 317–339.
[32] I.T. Okioga, J. Wu, Y. Sireli, H. Hendren, Renewable energy policy formulation for electricity generation in the United States, Energy Strategy Rev. 22
(2018) 365–384.
[33] K.S. Park, S.H. Kim, Tools for interactive multiattribute decision-making with incompletely identified information, Eur. J. Oper. Res. 98 (1) (1997) 111–
123.
[34] W. Pedrycz, P. Ekel, R. Parreiras, Fuzzy Multicriteria Decision-making: Models, Methods, and Applications, John Wiley & Sons, 2011.
[35] F. Perez, T. Gomez, Multiobjective project portfolio selection with fuzzy constraints, Ann. Oper. Res. 245 (1–2) (2016) 7–29.
[36] L.D. Phillipse, C.A.B. e Costa, Transparent prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision
conferencing, Ann. Oper. Res. 154 (1) (2007) 51–68.
[37] L.R.P. Roselli, A.T. De Almeida, E.A. Frej, Decision neuroscience for improving data visualization of decision support in the FITradeoff method, Oper. Res.
19 (2019) 1–21.
[38] A.A. Salo, R.P. Hamalainen, Preference ratios in multiattribute evaluation (PRIME)-elicitation and decision procedures under incomplete information,
IEEE Trans. Syst. Man Cybern. A Syst. Hum. 31 (6) (2001) 533–545.
[39] A.A. Salo, R.P. Hämäläinen, Preference assessment by imprecise ratio statements, Oper. Res. 40 (6) (1992) 1053–1061.
[40] A. Salo, A. Punkka, Rank inclusion in criteria hierarchies, Eur. J. Oper. Res. 163 (2) (2005) 338–356.
[41] P. Sarabando, L.C. Dias, Simple procedures of choice in multicriteria problems without precise information about the alternatives’ values, Comput.
Oper. Res. 37 (12) (2010) 2239–2247.
[42] R. Vetschera, A.T. De Almeida, A PROMETHEE-based approach to portfolio selection problems, Comput. Oper. Res. 39 (5) (2012) 1010–1020.
[43] M. Weber, Decision making with incomplete information, Eur. J. Oper. Res. 28 (1) (1987) 44–57.

498