Accepted Manuscript

A multiobjective DEA approach to ranking alternatives

Marianela Carrillo , Jesús M. Jorge

PII: S0957-4174(15)00824-6
DOI: 10.1016/j.eswa.2015.12.022
Reference: ESWA 10430

To appear in: Expert Systems With Applications

Received date: 2 September 2015

Revised date: 18 December 2015
Accepted date: 19 December 2015

Please cite this article as: Marianela Carrillo , Jesús M. Jorge , A multiobjective DEA approach to
ranking alternatives, Expert Systems With Applications (2015), doi: 10.1016/j.eswa.2015.12.022

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service
to our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and
all legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

Highlights

 We present a new method for ranking that combines DEA and multiobjective
concepts
 Multiobjective DEA is approached from the space of aggregate inputs and outputs
 A computational experiment has been performed with encouraging results
 The new procedure shows a better discrimination performance than two known
methods

T
IP
CR
US
AN
M
ED
PT
CE
AC
 ACCEPTED MANUSCRIPT
2

A MULTIOBJECTIVE DEA APPROACH TO RANKING ALTERNATIVES

Marianela Carrilloa, *, Jesús M. Jorgeb

a
Departamento de Economía Aplicada y Métodos Cuantitativos, Universidad de La Laguna, 38071 La Laguna, Spain
b
Departamento de Ingeniería Informática y de Sistemas, Universidad de La Laguna, 38071 La Laguna, Spain

* Corresponding author. Tel +34 922317031

e-mail addresses: mcarrif@ull.es (M. Carrillo), jjorge@ull.es (J.M. Jorge)

T
IP
Abstract

CR
The application of Data Envelopment Analysis (DEA) as a tool for efficiency evaluation has
become widespread in public and private sector organizations. Since decision makers are often

US
interested in a complete ranking of the evaluated units according to their performance,
procedures that effectively discriminate the units are of key importance for designing intelligent
AN
decision support systems to measure and evaluate different alternatives for a better allocation of
resources. This paper proposes a new method for ranking alternatives that uses common-weight
DEA under a multiobjective optimization approach. The concept of distance to an ideal is
M

thereby used as a means of selecting a set of weights that puts all the decision units in a
favorable position in a simultaneous sense. Some numerical examples and a thorough
ED

computational experiment show that the approach followed here provides sound results for
ranking alternatives and outperforms other known methods in discriminating the alternatives,
therefore encouraging its use as a valuable decision tool for managers and policy makers.
PT
CE

Keywords: Data envelopment analysis; Multiobjective; Optimization; Common-Weight DEA;

Ranking
AC
 ACCEPTED MANUSCRIPT
3

1. Introduction

Data Envelopment Analysis, first introduced in Charnes et al (1978), is a mathematical

programming technique useful for assessing the relative efficiency of a homogeneous set of
decision making units (DMUs) in a production system with multiple inputs and multiple
outputs. Subsequent developments have proved DEA as a valuable tool for performance
evaluation in a wide number of fields, with interesting applications in health care, education,
banking, manufacturing, etc. In the DEA methodology, for each DMU an efficiency score is
computed as a ratio of a weighted sum of outputs to a weighted sum of inputs, with such set of

T
weights found to guarantee the most favorable result for the DMU under evaluation. According

IP
to this score, every DMU is either found to perform efficiently or deemed inefficient, in which
case DEA can find a corresponding set of efficient units to be used as a benchmark for

CR
improvement.

The problem tackled by DEA highly resembles the one studied within Multicriteria Decision

US
Making (MCDM), in which a number of alternatives have to be evaluated and compared in
terms of several conflicting criteria in order to achieve a ranking of the alternatives and/or select
AN
the best option. In fact, ranking a set of alternatives in real-world applications often turns into a
rather overwhelming problem for many decision-makers that may require expertise and
computational support, especially when the number of alternatives and criteria grow, and so the
M

problem has been extensively studied. Particularly, the methodological connections existing
between DEA and MCDM approaches (Belton and Vickers, 1993, Stewart, 1996) that become
ED

clear when we identify alternatives with DMUs and criteria with inputs and outputs, have led
some authors to propose the use of DEA as a tool for MCDM (Doyle and Green, 1993, Sarkis,
2000, Mavrotas and Trifillis, 2006).
PT

However, when using standard DEA techniques with a ranking purpose some difficulties may
arise. First, DEA efficiency scores do not always allow a complete ordering of the alternatives
CE

since many of the DMUs are usually classified as efficient. The lack of discrimination in DEA
applications is well documented, particularly when the number of inputs and outputs is too high
AC

relative to the number of DMUs being evaluated, and a number of techniques have been
proposed to alleviate this drawback (Adler et al, 2002, Angulo-Meza and Lins, 2002,
Hosseinzadeh Lofti et al, 2013). Moreover, it is argued that using different sets of weights is
inappropriate in a ranking context because such flexibility deters the comparison among DMUs
on a common base (Kao and Hung, 2005). Also, it has been noticed that some units classified as
inefficient could in turn be better overall performers than certain efficient ones, possibly
involved in an unreasonable weight scheme induced by the maximization of their own
efficiency (Dyson and Thanassoulis, 1988).
 ACCEPTED MANUSCRIPT
4

The above reasoning suggests that, when DEA is being used with a ranking purpose, a common
set of weights (CSW) is highly recommended in order to fairly expose all the units to the same
light (Despotis, 2002) and thus provide a common base for the ranking. Several approaches
have been proposed in the DEA literature to obtain a common set of weights (see Kao and
Hung, 2005, Liu and Peng, 2008, Wang et al, 2011b and references therein), including methods
based on multivariate statistical analysis, regression analysis or cross-efficiency analysis, among
others. However, the present research is mainly interested in those approaches based on a
multiobjective view since they represent a natural generalization of the traditional DEA

T
approach: the search of a DMU-dependent set of weights that is the most favorable to each
individual DMU is generalized to the search of a common set of weights that is the most

IP
favorable to all the DMUs in a simultaneous way. This idea can be accomplished through the

CR
simultaneous maximization of the efficiency scores of all the DMUs, leading to a multiobjective
programming problem that can be solved using a Compromise Programming approach
(Despotis, 2002, Kao and Hung, 2005, Zohrehbandian et al, 2010). Some other authors

US
approach the computation of the common weights by considering the simultaneous
minimization of the differences between the weighted sum of inputs and the weighted sum of
AN
outputs for each DMU (Chen et al, 2009, Chiang et al, 2011). In this way they obtain a linear
programming problem that is equivalent to the simultaneous maximization of efficiency ratios.
Earlier methods built on a similar rationale are based on maximizing the average of efficiency
M

ratios of all the units (Roll and Golany, 1993) or maximizing the minimum efficiency ratio
(Troutt, 1997).
ED

The objective of this research is to further investigate the application of DEA as a ranking tool
within MCDM and therefore in the following sections a new DEA-based procedure for ranking
PT

alternatives is proposed which combines the CSW concept with the multiobjective paradigm
that characterizes the above-mentioned approaches. More specifically, the concept of distance to
an ideal is used to obtain a common set of weights that is favorable to all the alternatives
CE

simultaneously. The distance to an ideal DMU, defined as a hypothetical unit that consumes the
least inputs to produce the most outputs, has been explored previously in a context of cross-
AC

efficiency evaluation, with each DMU choosing its own set of weights to minimize its distance
from this ideal DMU (Wang et al, 2011a) as well as in a ranking context (Jahanshahloo et al,
2010, Sun et al, 2013). However, the approach followed here differs from these former works
since it does not rely on the definition of such an ideal unit, but on the computation of the ideal
point in a 2-dimension space where the DMUs are mapped when the aggregate input and output
measures are considered.

The rest of the paper is organized as follows. After this introduction, the main topics concerning
MCDM and DEA approaches are briefly reviewed, with special attention to the most relevant
 ACCEPTED MANUSCRIPT
5

CSW-DEA methods. Section 3 introduces a new DEA-based procedure for ranking a set of
alternatives that combines the common-weight concept and the multiobjective methodology.
The last sections illustrate the usefulness of the proposed approach and validate its application
as a ranking tool within a multicriteria decision framework: in Section 4 some numerical
examples are examined and Section 5 summarizes the results of the computational study
performed. Finally, some concluding remarks are provided.

2. Methodological background

T
The term Multiple Criteria Decision Making is used to describe a collection of formal

IP
approaches that seeks to explicitly account for multiple conflicting criteria in the evaluation and

CR
comparison of a number of alternatives. Both MCDA and DEA have been receiving
considerable attention in the specialized literature for the last 30 years, developing
independently to each another during the first decades. However, after several authors

US
established a number of analogies between MCDM and DEA methodologies (Golany, 1988,
Belton and Vickers, 1993, Doyle and Green, 1993, Stewart, 1996), the two fields have been
AN
evolving in a more cooperative mode. In this work both approaches come together to solve a
ranking problem.

First, some of the main concepts and techniques concerning MCDM and DEA are reviewed in
M

this section.
ED

2.1 MCDM preliminaries

The awareness of the multidimensional nature of socioeconomic phenomena and the need to
PT

consider more than a single criterion when judging them have encouraged the interest in the
Multiple Criteria Decision Making paradigm. The multicriteria decision problem is
CE

mathematically defined as

vopt ( z1 ( x), z2 ( x),  zk ( x))

AC

s. t. (1)
x X

where X represents the set of possible alternatives or feasible region, is a n-dimensional

vector containing decision variables, are the objective functions representing the criteria that
the decision-maker wants to attain and vopt stands for the simultaneous optimization of the k
objective functions. In this context an optimal solution, which attains the optimum value of all
the objectives, is generally impossible to find due to the conflictive nature of the criteria. Hence
 ACCEPTED MANUSCRIPT
6

solving a multicriteria problem implies to find an efficient solution, which cannot be altered to
improve one criterion without deteriorating at least another one.

Many different techniques are available for handling an arbitrary multicriteria decision problem
(Steuer, 1986), a number of which essentially combine the multiple objectives into one single
objective. Particularly popular are the weighting method, which attempts the optimization of a
weighted sum of the k objectives, and the Compromise Programming method, where the
solution with minimum Lp ‒ distance to the ideal point, the one that comprises the optimal
outcomes of all the objectives, is selected. This approach perfectly captures what is known as

T
Zeleny's axiom of choice, stating that "alternatives that are closer to the ideal are preferred to

IP
those that are farther away" (Zeleny, 1982).

Then, if z*Rk represents the coordinates of the ideal point, by considering the family of Lp

CR
metrics, the objective of choosing a solution as close as possible to the ideal is accomplished by

(∑ ( US ( )
(
AN

where i is a weight associated with the i-th objective function and the parameter 1 ≤ p ≤  is
M

related to the relative contributions of individual deviations. Particularly, when p = 1 the

distance measures the sum of individual deviations over the k objectives, which is the longest
ED

distance between the two points in a geometric sense, or "Manhattan distance", when p = 2 the
Euclidean distance to the ideal point is considered and ultimately, for p = , the largest of the
deviations completely dominates the distance measure. In this way, p = 1 represents total
PT

compensability among objectives and p =  represents no compensability among objectives.

Since other values of p are not easily interpreted, those are the most commonly used.
CE

2.2 DEA preliminaries

AC

Let us consider n production units or DMUs, each of them being evaluated in terms of r inputs
and s outputs. Let xij and yik be nonnegative values denoting respectively the amount consumed
of the j-th input and the amount produced of the k-th output for the i-th alternative, i = 1,...n,
j = 1,...r, k = 1,..s. Similarly, let xi and yi be, respectively, the r-dimensional vector of inputs of
unit i and the s-dimensional vector of outputs of unit i, as shown in Table 1.

In the original DEA ratio model developed in (Charnes et al, 1978) the efficiency measure is
defined as the ratio of a weighted sum of outputs to a weighted sum of inputs and each
 ACCEPTED MANUSCRIPT
7

alternative is allowed to choose its own weights for inputs and outputs in the most favorable
way as long as the efficient scores of all DMUs calculated from the same set of weights do not

Table 1
Data set notations
DMUs Inputs Outputs
1 x11 … x1r y11 … y1s
2 x21 … x2r y21 … y2s
⁞ ⁞ ⁞ ⁞ ⁞
n xn1 … xnr yn1 … yns

T
IP
exceed one. Therefore, denoting uq and vq the vector input and output weights for each DMU q

CR
under evaluation, the following non-linear programs are formulated, for each q{1,...,n}:

US (
AN
M

where and , and are the unitary vectors of the appropriate dimensions
and is an arbitrarily small positive constant that prevents the obtention of zero weights. As
seen, the efficiency of each unit is rated relatively to the others. Furthermore, it can be said that
ED

if a maximum efficiency rating ( ) is not attained by DMU q through this weight

selection process, then it cannot be attained from any other set of weights. The above problem
PT

can be suitably transformed into an equivalent linear program that is usually known as CCR
model, named after its authors Charnes, Cooper and Rhodes:
CE

(
AC

As seen, problem (4) must be solved n times in order to obtain the efficiency scores for all the
units analyzed. The above approach is particularly useful for the identification of those DMUs
that behave inefficiently, but it has some important drawbacks if the aim of the analysis is
ranking the units according to their performance. Typically, several DMUs will be evaluated as
efficient and the corresponding optimal weights will vary among the different units.
 ACCEPTED MANUSCRIPT
8

Consequently, although DEA methodology is able to classify units into efficient and non
efficient groups, the model does not always provide a ranking within each group (Sarkis, 2000).

Several strategies have been proposed to achieve a better discrimination among the evaluated
units. Within the group of methods not requiring a priori information provided by a decision
maker, Angulo-Meza and Lins (2002) highlight three main approaches: super-efficiency, cross-
evaluation and multiple objective methods. Super-efficiency models allow efficiency scores
greater than one by removing the qth constraint in model (4), but some infeasibility issues may
arise. Cross-efficiency models attempt to complement the self-evaluation view of the DEA

T
model with a peer-evaluation view, in such a way that the efficiency scores are computed

IP
averaging the n efficiency values obtained when the optimal weighting scheme of the n DMUs
are used. The fact that model (4) may have multiple optimal solutions complicates the cross-

CR
efficiency evaluation and therefore a secondary objective needs to be considered to guarantee a
unique set of optimal weights for each DMU, which may assume a benevolent or aggressive

US
performance to the other DMUs. Multiple objective models improve discrimination in DEA by
considering additional objectives in model (4), as initially proposed by Li and Reeves (1999)
and further developed by other authors (Bal et al, 2010, Ghasemi et al, 2014), yet maintaining
AN
the self-evaluation procedure in the calculation of the efficiency scores.

These strategies solve to a greater or lesser extent the discrimination problem but still they are
M

based on a unit-dependent weighting scheme. In order to establish a common base for

comparison and ranking units in a DEA context, some authors hint at the possibility of reducing
ED

the flexibility in the choice of input and output weights by using a common set of weights for
assessing the performances of all the units on a same scale.

2.3 Common-weight DEA models

PT

The idea of common weights within DEA started to receive an increasing research interest in
the late 90’s, mainly after some works evidenced its applicability (Roll et al, 1991, Roll and
CE

Golany, 1993). Since then, research about the common weight approach in DEA has been
developing gradually and the area still continues to be explored due to its interest both from
AC

theoretical and practical viewpoints.

Various different approaches have been investigated along the years: multivariate analysis
techniques, such as correlation analysis or nonlinear discriminant analysis (Friedman and
Sinuany-Stern, 1997, Sinuany-Stern and Friedman, 1998), as well as the concept of cross-
efficiency (Doyle, 1995, Wang and Chin, 2010) have been used to derive common weights for
all DMUs. More recently, Wang et al (2011b) developed a CSW approach based on least
squares regression, Qi and Guo (2014) introduced a CSW method using Shannon's entropy,
Ramón et al (2012) attempted the minimization of the deviations of the CSW from the
 ACCEPTED MANUSCRIPT
9

traditional DEA weights of the efficient DMUs and Wang et al (2009) proposed a procedure
that solves a series of LP problems and obtains a set of weights that “tend to be common”.

Particularly interesting is the line of research that seeks the computation of a CSW by using a
multicriteria paradigm, and this precise field is where this work contributes. The idea of
expressing the DEA problem as a multiobjective program can be credited to Kornbluth (1991)
and its application in a CSW context emerges as a natural, intuitive extension of the traditional
DEA paradigm: instead of allowing each DMU to choose the most favorable set of weights so
as to maximize its own efficiency, a unique set of input and output weights is chosen that

T
benefits all the alternatives simultaneously. This idea leads to the maximization of the n

IP
efficiency ratios, which is formulated as a multiobjective fractional programming problem that
must be solved in order to find the values of the weights. This is the approach put forward by

CR
Kao and Hung (2005) who use the Compromise Programming paradigm to find the common
weights u, v that make the resultant efficiency vector be as close as possible to the traditional
DEA efficiencies, denoted as

(∑ (
US
in the following expression:

) )
⁄
AN
(
M

Zohrehbandian et al (2010) develop over the same idea although they introduce some rewriting
ED

in the model in order to avoid fractional formulations and obtain linear programming problems
instead, at least when p = 1 and p =  norms are used:
PT

(∑( ) )
CE

(
AC

Similarly, Despotis (2002) proceeds from the simultaneous maximization of the n efficiency
scores and then proposes to average, for different values of t[0, 1], mean and maximal
deviations between DEA efficiencies and the calculated common-weight scores as a way to
provide the model with flexibility enough to account for different degrees of compensability
among these deviations:
 ACCEPTED MANUSCRIPT
10

∑ (

Leaving the multiobjective fractional programming problem aside, Chen et al (2009) introduces

T
a multiobjective perspective in their analysis by considering the simultaneous minimization of

IP
the differences ‒ rather than ratios ‒ between weighted sum of inputs and outputs, thus
preventing nonlinearities in the problem formulation. Clearly, when the traditional DEA

CR
efficiency score is 1, the difference between aggregate inputs and outputs is 0 and vice versa. In
this way, these authors simultaneously minimize the n differences between aggregate inputs and

US
outputs by considering the augmented Tchebychev distance (i.e., the L distance with a small
linear part added) to the theoretical ideal point (0,...,0), as shown in (7):
AN
( ) ∑( )
(
M
ED

with being a sufficiently small scalar.

More recently, Chiang et al (2011) also exploited the same idea but using the Manhattan instead
PT

of augmented Tchebychev distance in the simultaneous minimization of the differences between

inputs and outputs. Additionally, the procedure described in Liu and Peng (2008) and further
analyzed by Ramezani-Tarkhorani et al (2014) to evaluate the subset of DEA-efficient DMUs
CE

can also be classified into this category of methods, although it is originally developed from a
different reasoning.
AC

As seen, two main ideas are used to combine the multiobjective paradigm and the CSW-DEA
concept: the simultaneous maximization of the n efficiency ratios (weighted sum of outputs to
weighted sum of inputs) on one hand, and the simultaneous minimization of the n differences
between weighted sum of inputs and outputs, on the other hand. The various approaches within
each category differ in the way the multiobjective problem is tackled, usually by Compromise
Programming with varying Lp norms.
 ACCEPTED MANUSCRIPT
11

In the next section a new approach for deriving common weights in DEA is presented where the
multicriteria paradigm is implemented through the concept of distance to an ideal in a related
biobjective problem.

3. A new common-weight DEA-MCDM approach to ranking

This section presents a new CSW DEA-based procedure for ranking a given set of alternatives.
The proposed method is framed within the group of CSW DEA approaches that rely on a

T
multicriteria paradigm, but it does not fall into any of the two categories mentioned earlier.

IP
Particularly, the rationale for the proposed procedure comes from the exploration of the
interconnections between MCDM and DEA, assuming that if they were made explicit at a first

CR
stage, a multiobjective problem could be formulated considering the minimization of the r
inputs and maximization of the s outputs as associated objectives (Stewart, 1996). From this
starting point, the approach we are about to detail is inspired by the hierarchical approach

US
described in Belton and Vickers (1993), who consider the aggregate input and aggregate output
measures as a pair of high-level criteria that are used to replace the (r + s)-objective problem
AN
with a biobjective problem that allows the characterization of each DMU by a point in IR2 where
their visual approach takes place.
M

In a similar manner, having a pair of given nonnegative vector weights IRr and IRs, and
using and to denote the corresponding aggregate input and output measures for
ED

alternative i, then the maximization of the efficiency ratios can be confronted through
the minimization of and simultaneous maximization of . Note that for any set of input
and output weights each alternative has a ( , ) pair associated in the aggregate
PT

input−aggregate output space, as illustrated in Figure 1. Then, following a Compromise

Programming paradigm, the preferred DMU would be the one mapping onto a point that is the
CE

closest to the ideal point with minimum aggregate input value and a maximum aggregate output
value (left upper corner).
AC

30
DMU Input 1 Input 2 Output 1 Output 2 IDEAL
POINT
D
Aggregate Output

A
A 20 40 35 30
B 15 30 15 25 15 B
C
C 60 30 20 15
D 40 40 60 25
0
10 20 30
Weights: u1= 0,3 u2=0,4 v1=0,2 v2=0,5 Aggregate Input

Figure 1
Projection of the DMUs onto the aggregate Input/Output space
 ACCEPTED MANUSCRIPT
12

Now, as different set of weights would yield different plots with different alternatives being in a
favorable position, we would like to find a set of weights that puts all the alternatives as close as
possible to an ideal situation with minimum aggregate input and maximum aggregate output, in
a Compromise Programming sense. In this way, in each particular
(u, v)-dependent bidimensional space representation, the distance to the ideal point for the
biobjective problem can be considered.

To formalize the above reasoning, let us denote the ideal point in the aggregate input−aggregate
output space for a given set of nonnegative weights as

T
( ( ( ) ( (

IP
and let us consider the distance in Lp-norm between the point ( , ) IR2 corresponding to

CR
the i-th DMU and the ideal point

⁄
( ((
US
( ) ( ( ) )

The approach proposed here suggests searching for a set of weights u, v that places all mapped
(
AN
DMUs as close as possible to the ideal point in the aggregate input−aggregate output space in a
simultaneous way. Notice that it is not the best DMU that we are searching for, but the best set
of weights that makes all DMUs to be as well valued as possible. Consequently, we consider the
M

following multiobjective problem:

vmin (D1(u,v),...Dn(u,v))
ED

s.t. (11)
, for all i = 1,...,n
PT

Among the different methodologies available to deal with multiobjective problems we will draw
CE

on the weighting method that finds the solution that minimizes a weighted sum of the problem
objectives. In this case, we will consider equal weights for the n objectives, as all alternative
AC

DMUs are considered equally important, and so we will solve the following problem:

∑ (

s.t. (12)
, for all i = 1,...,n
 ACCEPTED MANUSCRIPT
13

It can be seen that the current formulation requires nonlinear solution procedures, due to the
minimizing and maximizing functions in the computation of Di. However, a suitable re-
statement of the problem will allow the use of linear optimization techniques, at least for
p = 1 and p =  metrics. In particular, for p = 1 (10) yields

( ( ( ) ( ( ) (

and it is straightforward to see that the problem to be solved (12) is equivalent to the following
linear program with r + s + 2 variables and 3n constraints:

T
∑( ) ( )

IP
s.t. (14)

CR
, for all i = 1,...,n
, for all i = 1,...,n

US , for all i = 1,...,n

AN
, ≥0

Similarly, for the Tchebychev metric (p = ) the distance (10) takes the following expression
M

( ( ( ( ) (
ED

and then problem (12) can be restated as a linear program with r + s + n + 2 variables and 5n
constraints:
PT

∑̅

s.t. (16)
CE

̅ for all i = 1,...,n

̅ for all i = 1,...,n
AC

, for all i = 1,...,n

, for all i = 1,...,n
, for all i = 1,...,n

, , ̅ ≥0 for all i = 1,...,n

In the models described in this section, as in the CCR model (4), the value of is introduced as
a lower bound of the weight values to keep them away from zero. Some research has been made
 ACCEPTED MANUSCRIPT
14

to find the maximum value of that guarantees feasibility in model (4) (see for example
Mehrabian et al, 2000, Amin and Toloo, 2004, Toloo, 2014), although most authors fix a
sufficiently small value to remain computationally significative, usually 10-3.

Rather often in practical applications, some input or output criteria are incommensurate with
others or have different measurement units. In MCDM processes, normalization is usually used
as a data preprocessing task aimed at eliminating the effects of different unit scales and
improving the numerical efficiency of the computations involved. Although in a DEA context
data need not be normalized, it is known that the normalization process does not affect DEA as

T
it is invariant in scale transformations of inputs and outputs. Therefore, in the same line that

IP
other known contributions in a DEA framework (Despotis, 2002, Sarkis, 2000, Wang et al,
2009, Mavrotas and Trifillis 2006), before the calculation procedure described above the first

CR
step is recommended to be the normalization of the alternative scores in the criteria (inputs and
outputs). In this work, original values are rescaled to the interval (0, 1) by dividing each score

̅
US
by the maximal value in its corresponding column, that is:

̅ (
AN
for each j = 1,...,r and k = 1,...,s.
M

To sum up, the proposed procedure for ranking a given set of alternatives has the following
steps:
ED

Step 1: Normalize the data set.

Step 2: Solve model (14) or (16) to obtain the weights associated to the input
PT

(minimizing) criteria and output (maximizing) criteria, .

̅
Step 3: Compute the efficiency scores ̅
and rank the alternatives accordingly.
CE

From a theoretical point of view, the approach proposed benefits from the use of a common-
weight scheme under a multicriteria paradigm. First, using a single set of weights allows a
AC

uniform assessment of all the units that are subsequently evaluated on a same scale. Also,
forcing the units to be simultaneously close to an ideal unit guarantees that no alternative is
being favored in the process, which makes the ranking obtained with this method rest on a more
sound, fairly assessed basis than other methods not using a multicriteria view. Admittedly,
multiobjective optimization theory may be difficult to understand by non-academic managers
who consequently can be unwilling to trust the approach. However, the distance-to-an-ideal
concept is remarkably natural and leads to a Linear Programming formulation, which is much
more appealing to a broad audience. A noticeable shortcoming of models (14) and (16) when
 ACCEPTED MANUSCRIPT
15

compared with models with similar features such as (6) or (8) concerns the size of the problems.
Indeed, the constraint set of model (14) increases by a factor of 3 with the number of
alternatives, the situation even worse for model (16). On the other side, models that require
previous computation of DEA scores, such as (6), are expected to be more computationally
burdensome. Nonetheless, the procedure above proposed will stand as a wise choice for
practical applications since, as will be proved throughout the following sections, it performs
better than alternative approaches in a key aspect, such as discrimination.

T
4. Numerical examples

IP
In this section three numerical examples taken from previous studies are examined using the

CR
above procedure to test its discrimination capabilities and to evaluate the practical application of
the proposed approach for ranking purposes. With that aim, the procedure described in the
preceding section was performed in EXCEL, which is a commercial software available for most

US
people not requiring excessive expertise, and the results are compared to those obtained with
two different methods (Chen et al, 2009, Zohrebandian, 2010).
AN
Table 2
Data of the 17 forest districts in Taiwan (Example 1)
Inputs Outputs
M

Initial Main Soil

Budget stocking Labour Land product conservation Recreation
District (dollars) (m3) (persons) (ha) (m3) (m3) (visits)
ED

1 51,62 11,23 49,22 33,52 40,49 14,89 3166,71

2 85,78 123,98 55,13 108,46 43,51 173,93 6,45
3 66,65 104,18 257,09 13,65 139,74 115,96 0
PT

4 27,87 107,6 14 146,43 25,47 131,79 0

5 51,28 117,51 32,07 84,5 46,2 144,99 0
6 36,05 193,32 59,52 8,23 46,88 190,77 822,92
CE

7 25,83 105,8 9,51 227,2 19,4 120,09 0

8 123,02 82,44 87,35 98,8 43,33 125,84 404,69
9 61,95 99,77 33 86,37 45,43 79,6 1252,62
AC

10 80,33 104,65 53,3 79,06 27,28 132,49 42,67

11 205,92 183,49 144,16 59,66 14,09 196,29 16,15
12 82,09 104,94 46,51 127,28 44,87 108,53 0
13 202,21 187,74 149,39 93,65 44,97 184,77 0
14 67,55 82,83 44,37 60,85 26,04 85 23,95
15 72,6 132,73 44,67 173,48 5,55 135,65 24,13
16 84,83 104,28 159,12 171,11 11,53 110,22 49,09
17 71,77 88,16 69,19 123,14 44,83 74,54 6,14
 ACCEPTED MANUSCRIPT
16

Example 1. The first example considered is by Kao and Hung (2005) who studied the problem
of measuring the efficiency of 17 forest districts in Taiwan using four outputs and three inputs.
Table 2 contains the original data. The outcomes of models (14) and (16) in comparison with
the results originally reported are shown in Table 3, also listing the efficiency scores and
associated rankings obtained for the 17 alternatives by models (6) and (8). A first check at the
results presented in Table 3 would suggest that the procedures proposed here show higher
discrimination properties than the other procedures, all of them presenting multiple ties with
two or more alternatives achieving the higher efficiency score. The next section will investigate

T
the extent to which this preliminary perception can be stated as a general feature of the method.

IP
Table 3
Efficiency scores for Example 1 and associated rankings (in parentheses) computed with different models

CR
DMU CCR p=1 p= Kao and Hung Zhorehbandian Zhorehbandian Chen et al
Model (4) Model (14) Model (16) (2005) model et al (2010) et al (2010) (2009) model
(p=2) model (p=1) model (p=)

1
2
1.000 (1)
1.000 (1)
1.000
0.490
(1)
(6)
1.000
0.408
(1)
(7)
US
1.000
1.000
(1)
(1)
1.000
1.000
(1)
(1)
1.000
1.000
(1)
(1)
0.923
0.439
(3)
(7)
AN
3 1.000 (1) 0.286 (13) 0.558 (3) 0,989 (3) 1.000 (1) 0.743 (9) 0.102 (17)
4 1.000 (1) 0.561 (5) 0.480 (6) 0,993 (4) 1.000 (1) 0.854 (6) 1.000 (1)
5 1.000 (1) 0.611 (4) 0.517 (4) 0,987 (5) 1.000 (1) 0.944 (4) 0.615 (6)
M

6 1.000 (1) 0.702 (2) 0.806 (2) 0,912 (6) 0.965 (6) 0.835 (7) 0.774 (4)
7 1.000 (1) 0.426 (7) 0.363 (8) 0,885 (7) 0.874 (8) 0.711 (12) 1.000 (1)
8 1.000 (1) 0.363 (10) 0.307 (11) 0.871 (9) 0.847 (9) 0.962 (3) 0.268 (12)
ED

9 1.000 (1) 0.636 (3) 0.515 (5) 0.669 (14) 0.678 (13) 0.649 (15) 0.717 (5)
10 0.940 (10) 0.408 (8) 0.339 (9) 0.877 (8) 0.878 (7) 0.899 (5) 0.359 (9)
11 0.935 (11) 0.239 (16) 0.196 (16) 0.652 (15) 0.653 (15) 0.746 (8) 0.196 (13)
PT

12 0.829 (12) 0.384 (9) 0.310 (10) 0.758 (10) 0.717 (11) 0.726 (11) 0.313 (10)
13 0.799 (13) 0.257 (15) 0,219 (15) 0.626 (16) 0.622 (16) 0.692 (13) 0.183 (15)
14 0.773 (14) 0.354 (11) 0.294 (12) 0.714 (12) 0.713 (12) 0.743 (10) 0.278 (11)
CE

15 0.763 (15) 0.297 (12) 0.247 (14) 0.721 (11) 0.721 (10) 0.669 (14) 0.410 (8)
16 0.744 (16) 0.149 (17) 0.188 (17) 0.681 (13) 0.670 (14) 0.555 (17) 0.135 (16)
17 0.687 (17) 0.276 (14) 0.269 (13) 0.607 (17) 0.592 (17) 0.567 (16) 0.185 (14)
AC

In order to verify that the new approach provides sound results, we analyzed the correlation
among the different rankings. The Spearman's rank correlation coefficient was used to see how
well the ordinal ranks of each of the techniques correlated with each other. The results obtained
and displayed in Table 4 provide evidence of statistically significant (at least at a 0.05 level)
positive association of ranks. Consequently, the rankings obtained with the weights computed as
 ACCEPTED MANUSCRIPT
17

in models (14) and (16) seem to correlate well with the other rankings considered, which
validates the consistency of the results of the approach proposed here.

Table 4
Spearman's rho coefficients, and 1-tailed significance in parentheses, between different rankings.
Kao and Hung, 2005 Zhorehbandian et al, 2010 Zhorehbandian et al, 2010 Chen et al, 2009
(p=1) (p=)

Model (14) 0.644 0.674 0.546 0.868

(p<0.005) (p<0.005) (p<0.025) (p<0.001)
Model (16) 0.777 0.801 0.547 0.608

T
(p<0.001) (p<0.001) (p<0.025) (p<0.01)

IP
CR
Table 5
Data of 6 nursing homes (Example 2)

Inputs Outputs
DMU
A
B
x1
1.50
4.00
US x2
0.2
0.7
y1
1.40
1.40
0.35
2.10
y2
AN
C 3.20 1.2 4.20 1.05
D 5.20 2.0 2.80 4.20
E 3.50 1.2 1.90 2.50
F 3.20 0.7 1.40 1.50
M

Table 6
ED

Efficiency scores for Example 2 and associated rankings (in parentheses) computed with different models
DMU CCR p=1 p= Kao and Hung Zhorehbandian Zhorehbandian Chen et al
Model (4) Model (14) Model (16) (2005) model et al (2010) et al (2010) (2009) model
PT

(p=2) model (p=1) model (p=)

1 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1)
CE

2 1.000 (1) 0.698 (5) 0.698 (5) 0.920 (5) 1.000 (1) 0.918 (5) 0.953 (4)
3 1.000 (1) 0.980 (2) 0.980 (2) 0.924 (4) 0.830 (6) 0.926 (4) 0.883 (5)
4 1.000 (1) 0.794 (3) 0.794 (3) 1.000 (1) 1.000 (3) 1.000 (1) 1.000 (1)
AC

5 0.977 (5) 0.782 (4) 0.782 (4) 0.972 (3) 0.977 (4) 0.972 (3) 0.974 (3)
6 0.867 (6) 0.673 (6) 0.673 (6) 0.831 (6) 0.867 (5) 0.830 (6) 0.846 (6)

Example 2. Next a classic example in the literature is being considered, originally posed by
Sexton (1986) and revisited by many authors through the years (Li and Reeves, 1999, Adler et
al, 2002 or Wang et al 2011b among others). The data provided in Table 5 describes a case of
six nursing homes that are evaluated in terms of two inputs (x1: staff hours per day; x2: supplies
 ACCEPTED MANUSCRIPT
18

per day measured in thousands of dollars) and two outputs (y1: total Medicare-plus Medicaid-
reimbursed patient days (104); y2: total privately paid patient days (104)). The results for the
different model tested are reported in Table 6. Note that models (14) and (16) yield the same
results in this case, providing a complete ranking of the alternative set that cannot be achieved
with the other models tested, which stands as an advantage of the proposed approach over the
previous approaches.

Table 7
Data of 28 key Chinese cities (Example 3)

T
Inputs Outputs
District x1 x2 y3 y1 y2 y3

IP
1 483.01 1397736 616961 6785798 1594957 1088699
2 371.95 355509 385453 2505984 545140 835745

CR
3 268.23 685584 341941 2292025 406947 473600
4 202.02 452713 117424 1158016 135939 336165
5 197.93 471650 112634 1244124 204909 317709
6
7
8
178.96
148.04
184.93
423124
367012
408311
US
189743
97004
111904
1187130
658910
993238
190178
86514
1411954
605037
239760
353896
AN
9 123.33 245542 91861 854188 135327 239360
10 116.91 305316 91710 606743 78357 208188
11 129.62 295812 92409 736545 114365 298112
M

12 106.26 198703 53499 454684 67154 233733

13 89.7 210891 95642 494196 78992 118553
14 109.26 282209 84202 842854 149186 243361
ED

15 85.5 184992 49357 776285 116974 234875

16 72.17 222327 73907 490998 117854 118924
17 76.18 161159 47977 482448 67857 158250
PT

18 73.21 144163 43312 515237 114883 101231

19 86.72 190043 55326 625514 173099 130423
20 69.09 158436 66640 382880 74126 123968
CE

21 77.69 135046 46198 867467 65229 262876

22 97.42 206926 66120 830142 128279 242773
23 54.96 79563 43192 521684 37245 184055
AC

24 67 144092 43350 869973 86859 194416

25 46.3 100431 31428 604715 55989 127586
26 65.12 96873 28112 601299 37088 224855
27 20.09 50717 54650 145792 11816 24442
28 69.81 117790 30976 319218 31726 169051

Example 3. A final example, originally drawn from Charnes et al (1989), has also been
examined in further papers by Despotis (2002), Mehrabian et al (2000) and others. This
 ACCEPTED MANUSCRIPT
19

example deals with the efficiency in the economic performance of 28 Chinese cities. Three
inputs and three outputs are involved in the study, whose values are reproduced in Table 7 and
are defined as follows: x1 (Labour, 10000 persons), x2 (Working funds, 10000 rmb), x3
(Investments, 10000rmb), y1 (Gross industrial outputs value, 10000 rmb), y2 (Profits and taxes,
10000 rmb), y3 (Retail sales, 10000rmb). Table 8 shows the efficiency scores and associated
rankings obtained by the various approaches. All the rankings obtained are fairly consistent,
showing statistically significant (at a 0.01 level) Spearman correlation values in all cases, and

T
Table 8

IP
Efficiency scores for Example 3 and associated rankings (in parentheses) computed with different models
DMU CCR p=1 p= Kao and Hung Zhorehbandian Zhorehbandian Chen et al

CR
Model (4) Model (14) Model (16) (2005) model et al (2010) et al (2010) (2009) model
(p=2) model (p=1) model (p=)

US
1 1.000 (1) 0.697 (6) 0.637 (9) 0.824 (8) 1.000 (1) 0.917 (7) 1.000 (1)
2 1.000 (1) 0.557 (11) 0.709 (6) 1.000 (1) 0.705 (12) 0.842 (8) 0.708 (10)
3 0.657 (17) 0.442 (19) 0.421 (20) 0.548 (16) 0.656 (13) 0.609 (14) 0.629 (11)
AN
4 0.522 (26) 0.382 (24) 0.380 (23) 0.439 (23) 0.514 (25) 0.474 (24) 0.466 (23)
5 0.580 (22) 0.412 (21) 0.400 (21) 0.463 (21) 0.538 (23) 0.497 (23) 0.506 (19)
6 1.000 (1) 0.580 (10) 0.567 (11) 0.612 (11) 0.904 (8) 0.754 (10) 0.611 (14)
7 0.488 (27) 0.330 (26) 0.316 (26) 0.345 (28) 0.468 (26) 0.400 (28) 0.370 (28)
M

8 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 1.000 (1) 0.864 (5)
9 0.625 (20) 0.471 (15) 0.489 (14) 0.590 (14) 0.625 (15) 0.611 (13) 0.589 (16)
10 0.538 (24) 0.356 (25) 0.335 (25) 0.374 (26) 0.524 (24) 0.442 (25) 0.415 (27)
ED

11 0.694 (14) 0.463 (17) 0.458 (19) 0.497 (19) 0.653 (14) 0.571 (18) 0.505 (20)
12 0.653 (18) 0.440 (20) 0.468 (18) 0.485 (20) 0.583 (20) 0.532 (20) 0.434 (25)
13 0.453 (28) 0.315 (27) 0.308 (27) 0.388 (25) 0.453 (28) 0.425 (27) 0.426 (26)
PT

14 0.719 (12) 0.513 (12) 0.485 (15) 0.554 (15) 0.718 (11) 0.629 (12) 0.617 (13)
15 0.865 (10) 0.650 (8) 0.655 (8) 0.748 (9) 0.855 (9) 0.799 (9) 0.770 (8)
CE

16 0.598 (21) 0.410 (22) 0.365 (24) 0.424 (24) 0.598 (19) 0.500 (22) 0.510 (18)
17 0.636 (19) 0.465 (16) 0.472 (17) 0.538 (18) 0.625 (16) 0.580 (17) 0.545 (17)
18 0.671 (15) 0.461 (18) 0.480 (16) 0.593 (13) 0.546 (22) 0.571 (19) 0.598 (15)
AC

19 0.660 (16) 0.501 (13) 0.502 (13) 0.598 (12) 0.602 (18) 0.602 (15) 0.620 (12)
20 0.568 (23) 0.393 (23) 0.388 (22) 0.454 (22) 0.561 (21) 0.509 (21) 0.471 (21)
21 1.000 (1) 0.756 (2) 0.824 (3) 1.000 (1) 0.999 (3) 1.000 (1) 0.953 (4)
22 0.788 (11) 0.591 (9) 0.599 (10) 0.702 (10) 0.788 (10) 0.745 (11) 0.719 (9)
23 1.000 (1) 0.692 (7) 0.796 (4) 1.000 (1) 0.942 (7) 0.976 (3) 0.863 (6)
24 1.000 (1) 0.732 (4) 0.739 (5) 0.924 (6) 0.976 (4) 0.952 (5) 1.000 (1)
25 1.000 (1) 0.702 (5) 0.706 (7) 0.900 (7) 0.943 (6) 0.924 (6) 0.986 (3)
26 1.000 (1) 0.748 (3) 0.864 (2) 1.000 (1) 0.949 (5) 0.966 (4) 0.862 (7)
27 0.536 (25) 0.233 (28) 0.225 (28) 0.368 (27) 0.456 (27) 0.435 (26) 0.448 (24)
28 0.704 (13) 0.475 (14) 0.525 (12) 0.547 (17) 0.623 (17) 0.583 (16) 0.469 (22)
 ACCEPTED MANUSCRIPT
20

with a general agreement in the units ranked at first and last positions. Again, models (14) and
(16) produce a complete ranking of the alternatives, whereas the other models show two or more
ties in the first position.

These numerical examples illustrate that the methodological approach described in Section 3
can be successfully used to produce a full ranking of the alternatives in a variety of contexts and
therefore can be considered as an effective decision-making tool for a number of academics and
practitioners in a wide range of fields. With our proposal, an efficiency assessment of the

T
alternatives is undertaken under objective, impartial and fair conditions, therefore improving the
ranking's trustworthiness and leading to more robust decisions.

IP
CR
Since all the methods tested share a multiobjective approach to the common-weight DEA
model, and having all of them a similar linear programming-based nature (excepting Kao and
Hung's model which is of a nonlinear nature), the discrimination power stands as the main

US
advantage of the proposed models as compared to the others. The next section will test this
property more thoroughly, although we will focus our attention on model (14) since in model
AN
(16) the number of variables and constraints can grow significantly as the number of DMUs
considered increases.
M

5. Computational results
ED

In order to investigate the discrimination properties and evaluate the efficiency of the method
proposed, a computational study was conducted. For this purpose, a number of data sets of
different sizes were randomly generated and model (14) described above was solved to obtain
PT

the associated ranking for each instance. In order to gain insight concerning the performance of
the proposed method in comparison with other known methods in the literature, we also
CE

analyzed the ranking obtained for the same data sets by applying model (8) (Chen et al, 2009)
and model (6) (Zhorehbandian et al, 2010) with p = 1, which are the methods that exhibit a
AC

stronger correlation with the former, according to Table 4. The whole experiment was
performed using VBA for MS-EXCEL 2007 on a personal computer running under MS-
Windows 7 Professional, with 8 GB of RAM, and 2.3 GHz Core i3 CPU.

For this experiment ten problem types were defined according to the number of alternatives, n,
the number of inputs, r, and the number of outputs, s, as detailed in Table 9 and 10. For each
problem type, 100 instances were tested with corresponding input and output values being
randomly generated from a discrete uniform distribution U[100, 1000]. Therefore, a total of
1000 data sets of different dimensions were generated and ranked using the three models.
 ACCEPTED MANUSCRIPT
21

Table 9
Comparison of discrimination power
% Full discrimination Average number of ties
n r s
model (14) model (8) model (6) model (14) model (8) model (6)
2 2 70 58 9 1.3 1.42 2.5
15
2 3 65 43 3 1.35 1.58 3
2 2 64 51 4 1.36 1.49 2.68
40 3 4 36 12 0 1.75 2.2 4.71
4 3 60 31 0 1.43 1.89 5.32
3 4 26 7 0 1.92 2.39 4.84

T
80 4 3 41 20 0 1.73 2.15 5.5

IP
5 5 28 2 0 1.94 2.93 7.78
8 8 28 1 0 2.12 3.87 12.89
150

CR
10 10 29 0 0 2.11 3.99 16.31

Table 10
Average time requirements

n r s
USAverage CPU time consumed (secs)
AN
model (14) model (8) model (6)
2 2 0.08 0.06 0.60
15
2 3 0.08 0.06 0.64
2 2 0.16 0.09 2.71
M

40 3 4 0.18 0.10 3.31

4 3 0.15 0.08 3.21
ED

3 4 0.43 0.20 6.93

80 4 3 0.32 0.12 8.76
5 5 0.39 0.17 12.54
PT

8 8 1.07 0.38 18.45

150
10 10 1.12 0.45 40.00
CE

The computational performance of the methods was measured in terms of the level of
AC

discrimination observed in the rankings and the computational time requirements. Tables 9 and
10 summarize the performance results concerning both aspects. First, to account for the
discrimination power of the methods, the percentage of instances with no ties in the first
position of the ranking is given for each of the models tested. It can be seen that model (14)
provides a complete ranking of the alternatives more frequently than model (8), with a notable
improvement as the number of alternatives increase, whereas model (6) reveals a very poor
discrimination. Table 9 also lists the average number of ties in the resulting rankings of the 100
instances in each problem type. Again, it is observed that model (8) yields between 10% and
90% more ties than our procedure, and model (6) performs 2 to 8 times worse. In summary, the
 ACCEPTED MANUSCRIPT
22

results reported identify our model as a more powerful discrimination tool than Chen et al.'s and
Zohrehbandian et al.'s models, with even a higher improvement ratio in those instances having a
large number of alternatives that are precisely the most difficult to handle.

As far as the computational complexity of the procedures is concerned, time requirements of the
methods have been computed. Table 10 reports the average run time (in seconds) taken by
models (14), (8) and (6) to solve each of the 100 instances of each problem type. Again, model
(6) stands out as the worst performer, with notably higher average time requirements than the
other two methods due to the computational effort involved in the calculation of the DMU's

T
efficiency scores. Admittedly, the comparison of the time figures reported for models (14) and

IP
(8) indicate that the better discriminating behavior of the former was only reached at the cost of
a higher computational time, as expected from a more elaborated theoretical construct leading to

CR
a more intricate problem. However, the increase of CPU time seems to remain within reasonable
limits, and therefore this drawback does not hinder the usefulness of our procedure.

6. Conclusions
US
AN
It is known that many relevant decisions, frequently involving investments or allocation of
resources, have to be made over a ranked set of options, and therefore ranking procedures often
become an essential part of the problem-solving toolbox that integrates any intelligent system
M

used in public or private organizations to achieve better decisions. While Data Envelopment
Analysis is established as a powerful approach to analyze the efficiency of organizational units
ED

such as universities, banks, hospitals or countries, traditional DEA approaches fail to produce a
complete ranking of the alternative set and much research has been devoted to tackle this
PT

inconvenience. The present study deals with the application of Data Envelopment Analysis as a
ranking tool within Multiple Criteria Decision Making. Particularly, a new common-weight
based method has been proposed to compute the efficiency scores to be used for effectively
CE

ranking the alternatives. The new procedure seeks to put all the units under a favorable sight in a
simultaneous way in a bidmensional space where the aggregate input and aggregate output
AC

measures are represented, the concept of distance to an ideal being used to this aim. The
consideration of the multiobjective and common-weight paradigms within a DEA framework
strengthens the efficiency analysis with a fair and unbiased assessment of all the units that
naturally extends the traditional approach. Consequently, the rankings obtained are conceptually
more sensible and trustworthy than others based on different paradigms.

In order to validate our analysis, the proposed methodology is illustrated with three numerical
examples extracted from the literature. Since the rankings obtained present a high correlation
with those found with other known methods, it can be said that our approach yields sound
 ACCEPTED MANUSCRIPT
23

results and therefore it can be considered as a suitable option for many studies based on ranking.
Additionally, the computational experiments performed with randomly generated data sets have
proved that the proposed procedure works reasonably well under different experimental
conditions, successfully discriminating the alternatives in significantly more instances than
other previously known methods, especially for the larger instances. This is doubtlessly a major
advantage of the approach that is achieved at a cost of higher (although rather acceptable)
computational time figures. In summary, since the enhancement in discrimination power of such
procedures is an important feature that makes the rankings obtained more useful, the approach

T
proposed here contributes to the development of more efficient decision systems.

IP
CR
7. Acknowledgments

The authors are grateful to two anonymous referees for their useful suggestions, which have
improved the quality of the final manuscript.

US
AN
8. References

Adler, N., Friedman, L., & Sinuany-Stern, Z. (2002). Review of ranking methods in the data
envelopment analysis context. European Journal of Operational Research, 140, 249-265.
M

Amin, G. R., & Toloo, M. (2004). A polynomial-time algorithm for finding  in DEA models.
ED

Computers & Operations Research, 31, 803-805.

Angulo-Meza, L., & Lins, M. P. E. (2002). Review of methods for increasing discrimination in
PT

data envelopment analysis. Annals of Operations Research, 11, 225-242.

Bal, H., Örkcü, H. H., & Çelebioğlu, S. (2010). Improving the discrimination power and
CE

weights dispersion in data envelopment analysis. Computers & Operations Research, 37, 99-
107.
AC

Belton, V., & Vickers, S. P. (1993). Demystifying DEA-A visual interactive approach based on
Multiple Criteria Analysis. Journal of the Operational Research Society, 44, 883-896.

Charnes, A., Cooper, W. W., & Li, S. (1989). Using Data Envelopment Analysis to Evaluate
Efficiency in the Economic Performance of Chinese Cities. Socio-Economic Planning
Sciences, 23 (6), 325-344.

Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making
units. European Journal of Operational Research, 2, 429-444.
 ACCEPTED MANUSCRIPT
24

Chen, Y. W., Larbani, M., & Chang, Y. P. (2009). Multiobjective data envelopment analysis.
Journal of the Operational Research Society, 60, 1556-1566.

Chiang, C. I., Hwang, M. J., & Liu, Y. H. (2011). Determining a common set of weights in a
DEA problem using a separation vector. Mathematical and Computer Modelling, 54, 2464-
2470.

Despotis, D. K. (2002). Improving the discriminating power of DEA focus on globally efficient
units. Journal of the Operational Research Society, 53, 314-323.

T
Doyle, J., & Green, R. (1993) Data Envelopment Analysis and Multiple Criteria Decision

IP
Making. OMEGA, 21(6), 713-715.

Doyle, J. R. (1995). Multiattribute Choice for the Lazy Decision Maker: Let the Alternatives

CR
Decide! Organizational Behavior and Human Decision Processes, 62 (1), 87-100.

Dyson, R. G., & Thanassoulis, E. (1988). Reducing weight flexibility in Data Envelopment

US
Analysis. Journal of the Operational Research Society, 39 (6), 563-576.

Friedman, L., & Sinuany-Stern, Z. (1997). Scaling units via the canonical correlation analysis in
AN
the DEA context. European Journal of Operational Research, 100, 629-637.

Ghasemi, M. R., Ignatius, J., & Emrouznejad A. (2014). A bi-objective weighted model for
M

improving the discrimination power in MCDEA. European Journal of Operational

Research, 233, 640-650.
ED

Golany, B. (1988). An Interactive MOLP Procedure for the Extension of DEA to Effectiveness.
Journal of the Operational Research Society, 39 (8), 725-734.
PT

Hosseinzadeh Lofti, F., Jahanshahloo, G. R., Khodabakhshi, M., Rostamy-Malkhlifeh, M.,

Moghaddas, Z., & Vaez-Ghasemi, M. (2013). A review of ranking models in Data
CE

Envelopment Analysis. Journal of Applied Mathematics, volume 2013, Article ID 492421,

20 pages.
AC

Jahanshahloo, G. R., Hosseinzadeh Lofti, F., Khanmohammadi, M., Kazemimanesh, M., &
Rezaie, V. (2010). Ranking of units by positive ideal DMU with common weights. Expert
Systems with Applications, 37, 7483-7488.

Kao, C., & Hung, H. (2005). Data envelopment analysis with common weights: the compromise
solution approach. Journal of the Operational Research Society, 56, 1196-1203.

Kornbluth, J. S. H. (1991). Analysing Policy Effectiveness using cone restricted Data

Envelopment Analysis. Journal of the Operational Research Society, 42 (12), 1097-1104.
 ACCEPTED MANUSCRIPT
25

Li, X., & Reeves, G. (1999). A multiple criteria approach to data envelopment analysis.
European Journal of Operational Research, 115, 507-517.

Liu, F. H. F., & Peng, H. H. (2008). Ranking of units on the DEA frontier with common
weights. Computers & Operations Research, 35, 1624-1637.

Mavrotas, G., & Trifillis, P. (2006). Multicriteria decision analysis with minimum information:
combining DEA with MAVT. Computers & Operations Research, 33, 2083-2098.

Mehrabian, S., Jahanshahloo, G. R., Alirezaee M. R., & Amin, G. R. (2000). An assurance

T
interval for the non-archimedean epsilon in DEA models. Operations Research, 48, 344-347.

IP
Qi, X. G., & Guo, B. (2014). Determining common weights in Data Envelopment Analysis with
Shannon's Entropy. Entropy, 16, 6394-6414.

CR
Perelman, S., & Santín, D. (2009). How to generate regularly behaved production data? A
Monte Carlo experimentation on DEA scale efficiency measurement. European Journal of
Operational Research, 199, 303-310.
US
Ramezani-Tarkhorani, S., Khodabakhshi, M., Mehrabian, S., & Nuri-Bahmani, F. (2014).
AN
Ranking decision-making units using common weighs in DEA. Applied Mathematical
Modelling, 38, 3890-3896.
M

Ramón, N., Ruiz, J. L., & Sirvent, I. (2012). Commons sets of weights as summaries of DEA
profiles of weights: With an application to the ranking of professional tennis players. Expert
ED

Systems with Applications, 39, 4882-4889.

Roll, Y., Cook, W. D., & Golany, B. (1991). Controlling Factor Weights in Data Envelopment
PT

Analysis. IIE Transactions, 23 (1), 2-9.

Roll, Y., & Golany, B. (1993). Alternate Methods of Treating Factor Weights in DEA.
CE

OMEGA, 21 (1), 99-109.

Sarkis, J. (2000). A comparative analysis of DEA as a discrete alternative multiple criteria

AC

decision tool. European Journal of the Operational Research, 123, 543-557.

Sinuany-Stern, Z., & Friedman, L. (1998). DEA and the discriminant analysis of ratios for
ranking units. European Journal of the Operational Research, 111, 470-478.

Steuer, R. E. (1986). Multiple Criteria Optimization‒Theory, Computation and Applications.

Wiley: New York.

Stewart, T. (1996). Relationships between Data Envelopment Analysis and Multicriteria

Decision Analysis. Journal of the Operational Research Society, 47, 64-665.
 ACCEPTED MANUSCRIPT
26

Sun, J., Wu, J., & Guo, D. (2013). Performance ranking of units considering ideal and anti-ideal
DMU with common weights. Applied Mathematical Modelling, 37, 6301-6310.

Toloo, M. (2014). The role of non-Archimedean epsilon in finding the most efficient unit: With
an application of professional tennis player. Applied Mathematical Modelling, 38, 5334-
5346.

Troutt, M. D. (1997). Derivation of the maximin efficiency ratio model from the maximum
decisional efficiency principle. Annals of Operations Research, 73, 323-338.

T
Wang, Y, M., & Chin, K. S, (2010) A neutral DEA model for cross-efficiency evaluation and its

IP
extension. Expert Systems with Applications, 37, 3666-3675.

Wang, Y. M., Chin, K. S., & Luo, Y. (2011a). Cross-efficiency evaluation based on ideal and

CR
anti-ideal decision making units. Expert Systems with Applications, 38, 10312-10319.

Wang, Y. M., Luo, Y., & Lan, Y. (2011b). Common weights for fully ranking decision making

US
units by regression analysis. Expert Systems with Applications, 38, 9122-9128.

Wang, Y. M., Luo, Y., & Liang, Y. (2009). Ranking decision making units by imposing a
AN
minimum weight restriction in the data envelopment analysis. Journal of Computational and
Applied Mathematics, 223, 469-484.
M

Zeleny, M. (1982). Multiple Criteria Decision Making. McGraw Hill: New York.

Zohrehbandian, M., Makui, A., & Alinezhad, A. (2010). A compromise solution approach for
ED

finding common weights in DEA: an improvement to Kao and Hung's approach. Journal of
the Operational Research Society, 61, 604-610.
PT
CE
AC