Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com

ScienceDirect
Procedia
Procedia Computer
Computer Science
Science 00 (2018)
126 (2018) 000–000
1006–1013
www.elsevier.com/locate/procedia
Procedia Computer Science 00 (2018) 000–000
www.elsevier.com/locate/procedia

International Conference on Knowledge Based and Intelligent Information and Engineering

Systems, on
International Conference KES2018, 3-5 Based
Knowledge September 2018, Belgrade,
and Intelligent Serbiaand Engineering
Information
Systems, KES2018, 3-5 September 2018, Belgrade, Serbia
The improvement of data in pairwise comparison matrices
The improvement of dataJacek
inSzybowski
pairwise comparison matrices
JacekAddress,
AGH University of Science and Technology, Szybowski
al. Mickiewicza 30, 30-059 Kraków, Poland
AGH University of Science and Technology, Address, al. Mickiewicza 30, 30-059 Kraków, Poland

Abstract
Abstract
This article analyzes the question of reliability of entries collected in the pairwise comparison matrices used for prioritization
of alternatives. The inappropriate input data may result in an invalid output hierarchy. Basing on the Geometric Mean Method
This article analyzes the question of reliability of entries collected in the pairwise comparison matrices used for prioritization
introduced by Crawford and Williams and the geometric consistency index defined by Aguarón and Moreno-Jiménez we introduce
of alternatives. The inappropriate input data may result in an invalid output hierarchy. Basing on the Geometric Mean Method
an algorithm which allows to search for the least reliable pairwise comparison matrix elements and improve them by means of their
introduced by Crawford and Williams and the geometric consistency index defined by Aguarón and Moreno-Jiménez we introduce
estimations.
an algorithm which allows to search for the least reliable pairwise comparison matrix elements and improve them by means of their
c 2018 The Authors. Published by Elsevier B.V.
estimations.
© 2018 The under
Authors. Published by
Peer-review
c 2018 responsibility of Elsevier Ltd.
KES International.

This The
is an Authors.
open accessPublished by Elsevier
article under B.V.
the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review
Selection and
Keywords: under responsibility
peer-review
pairwise of KES International.
under algorithm;
comparisons; responsibility of KES International.
inconsistency;
Keywords: pairwise comparisons; algorithm; inconsistency;

1. Introduction
1. Introduction
The method of pairwise comparisons has been widely applied in the multicriteria decision making (MCDM) pro-
cess.
TheComparing
method ofonly two objects
pairwise comparisonsis muchhaseasier
been than
widelytaking intoinconsideration
applied the multicriteriaall the alternatives
decision makingat the
(MCDM)same time.pro-
Several
cess. MCDM methods
Comparing only twouse this is
objects simple
muchfact,
easierforthan
instance: the Analytic
taking into Hierarchy
consideration all theProcess (AHP)
alternatives introduced
at the same time. by
Saaty 1 , The
Several MCDMPreference
methods Ranking
use thisOrganization
simple fact,METHod for Enrichment
for instance: the Analytic of Hierarchy
Evaluations (PROMETHEE)
Process introduced
(AHP) introduced by
by Brans
Saaty 1
, The Vincke 2 , PROMETHEE
andPreference Ranking Organizationfor Sustainability
METHod for Assessment
Enrichment (PROSA) introduced
of Evaluations by Ziemba et al.
(PROMETHEE)
3
, or the
introduced
4,5
Characteristic Objects2 ,Method
by Brans and Vincke PROMETHEE(COMET) forintroduced by Piegat
Sustainability and Sałabun
Assessment (PROSA) . Aintroduced
comparative byanalysis
Ziembaof et these meth-
al. 3 , or the
6
ods can be found
Characteristic in theMethod
Objects paper .(COMET) introduced by Piegat and Sałabun . A comparative analysis of these meth-
4,5

odsHowever,
can be foundindipendent comparisons
in the paper 6
. (even made by the same person), may lead to the inconsistency of data. This
1 7
may significantly
However, affect the
indipendent final result.
comparisons (evenManymade authors,
by the among others Saaty
same person), may lead or toAguarón and Moreno-Jiménez
the inconsistency of data. This,
introduced
may variousaffect
significantly consistency
the final indices which
result. Many allow to judge
authors, how far
among a pairwise
others Saaty comparison
1
or Aguarónmatrix (PC matrix), used
and Moreno-Jiménez 7
,
for examplevarious
introduced in the AHP, is fromindices
consistency perfectwhich
consistency.
allow toAttempts
judge how to far
determine the comparison
a pairwise good properties of such
matrix indices were
(PC matrix), used
8 9 10
made, for example
for example by Brunelli
in the AHP, is fromand Fedrizzi
perfect , Szybowski
consistency. , or Koczkodaj
Attempts to determine andtheUrban .
good properties of such indices were
All these measures, however, do not indicate
8 the source
9 of inconsistency. Reasons
made, for example by Brunelli and Fedrizzi , Szybowski , or Koczkodaj and Urban . 10 for that can be different, de-
pending on how
All these we accumulate
measures, however,data. In most
do not cases,
indicate thepairwise
source ofcomparisons are performed
inconsistency. Reasons for bythat
experts.
can be However,
different,some de-
pending on how we accumulate data. In most cases, pairwise comparisons are performed by experts. However, some
∗ Corresponding author. Tel.: +48-12-617-3587.
∗ E-mail address: szybowsk@agh.edu.pl
Corresponding author. Tel.: +48-12-617-3587.
E-mail address: szybowsk@agh.edu.pl

1877-0509  c 2018 The Authors. Published by Elsevier B.V.

1877-0509 © 2018 The Authors. Published by Elsevier Ltd.
Peer-review under responsibility of KES International.
This is an 
1877-0509 open access
c 2018 article under
The Authors. the by
Published CCElsevier
BY-NC-ND
B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and
Peer-review peer-review
under under
responsibility responsibility
of KES of KES International.
International.
10.1016/j.procs.2018.08.036
 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013 1007
2 Author name / Procedia Computer Science 00 (2018) 000–000

of their judgments may be wrong or subjective. Sometimes, the entries of a PC matrix come from seemingly objective
tests, like the results of sport’s games. Even so, a number of them may also be misleading and not reflect a real strength
of a sportsman or a team, as they may result from a worse disposition of the day, bad weather or other exceptional
circumstances.
That is why it is important to find a method of eliminating such incorrect data. In some sport’s disciplines (for
instance figure skating or ski jumping) where the overall result depends on judges’ evaluation, the extreme rates are
not taken into consideration. The aim of this study was to measure how far the elements of a PC matrix are from
their consistent approximation obtained by a Row Geometric Mean Method (RGMM) introduced by Crawford and
Williams 11 and to decrease this distance by means of a simple algorithm leaving as many original PC matrix entries
unchanged as possible.

2. Preliminaries

An n × n matrix A = [ai j ] with all positive elements is called a pairwise comparisons matrix (a PC matrix) if n ≥ 3.
A PC matrix A is called reciprocal if ai j · a ji = 1 for every i, j = 1, . . . , n (then obviously aii = 1 for every i = 1, . . . , n).
From now on we assume that each PC matrix is reciprocal. A PC matrix A is consistent if ai j · a jk · aki = 1 for all
i, j, k = 1, . . . , n.
In 1985 Crawford and Williams 11 came up with an idea of applying the Logarithmic Least Square Method to an
inconsistent matrix A in order to seek for the closest consistent matrix A . As they proved, the elements ai j of A may
be obtained as the quotients wwij of the geometric means of rows of A:

n
 1
wi = (ai j ) n , (1)
j=1

which are the weights of a priority vector. That was the origin of RGMM, which has become one of the most popular
prioritization methods.
Basing on this method, Aguarón and Moreno-Jiménez (2003), introduced the Geometric Consistency Index:
  
2 wj
GCI(A) = ln2 ai j · , (2)
(n − 1)(n − 2) i< j wi

where wi are given by (1) for i = 1, . . . , n. Obviously, a matrix A is consistent if and only if GCI(A) = 0. The higher
values GCI achieves, the more inconsistent matrices are.
If, for i, j = 1, . . . , n, we make substitutions
wi
ai j = (3)
wj
and   
 w j 
ei j = | ln ai j − ln ai j | = ln ai j , (4)
 wi 
in (2), we obtain an alternative formula:
2  2 2 
GCI(A) = ln ai j − ln ai j = e2 , (5)
(n − 1)(n − 2) i< j (n − 1)(n − 2) i< j i j

which shows that GCI is simply the estimator of the variance of perturbations in RGMM.
Similarly, we can define a Logarithmic Consistency Index (LCI) by the formula:
 
LCI(A) =
2 ln a − ln a  = 2
ei j . (6)
ij ij
(n − 1)(n − 2) i< j (n − 1)(n − 2) i< j

Obviously, A is consistent if and only if GCI(A) = LCI(A) = 0.

1008 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013
Author name / Procedia Computer Science 00 (2018) 000–000 3

3. The input data improvement

3.1. The algorithm body

Let A = [ai j ] be a PC matrix. Fix a positive threshold ε.

The flowchart of the algorithm of the input matrix improvement by its LCI reduction is as follows:

START


FIND p, q
 SUCH THAT p < q
AND e pq = max ei j
i< j


REPLACE a pq
w
WITH wqp
wq
AND aqp WITH wp


IS LCI(A) < ε?


 
NO YES


EXIT

Let us introduce the algorithm in more details:

 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013 1009
4 Author name / Procedia Computer Science 00 (2018) 000–000

ALGORITHM(A: matrix)
begin
repeat
begin
for i := 1 to n
begin
wi := 1;
for j := 1 to n
wi := wi · ai j ;
√
wi := n wi ;
end
LCI:=0; MAX:=0;
for i := 1 to n − 1
for j := i + 1 to n
begin
ai j := wwij ;
AUX:= | ln ai j − ln ai j |;
LCI:= LCI+AUX;
if AUX > MAX then
begin
MAX:=AUX; p := i; q := j;
end
end
2
LCI:=LCI· n2 −3n+2 ;
a pq := apq ; aqp := a1pq ;
end
until LCI < ε
end
Remark 3.1. The idea of the algorithm comes from the algorithm introduced by Holsztynski and Koczkodaj 12 and
developed in works 13,14 , where the most inconsistent triad is replaced in each step with a consistent triad to reduce
the inconsistency index defined by Koczkodaj 15 . A very similar method of the worst triads inconsistency reduction
was also used in the algorithm presented by Koczkodaj et al. 16 . Obviously, neither the presently introduced approach
nor the previous ones depend on the scale used in a PC matrix. The advantage of the new algorithm is that it reduces
the inconsistency resulting from the singular entries so it is faster and the resulting matrix better reflexes the original
input data.
Remark 3.2. At any step of the algorithm it may happen that there are more than one elements a pq such that e pq =
maxi< j ei j . In this case we choose the first found element satisfying this condition.

3.2. Discussion on the algorithm correctness

By A and A∗ denote the PC matrix respectively before and after a single iteration. Let
√
wk = n ak1 · . . . · akn , (7)
and 
w∗k = n
a∗k1 · . . . · a∗kn , (8)
for k = 1, . . . , n.
Let a pq be the element of A for which e pq is maximal. During the iteration of the algorithm it is replaced with
wp
a∗pq = , (9)
wq
1010 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013
Author name / Procedia Computer Science 00 (2018) 000–000 5

while aqp is replaced with

wq
a∗qp = . (10)
wp
The rest of entries remain unchanged, so the values of geometric means differ only for rows p and q.
n ∗
 a pq
w∗p = a p1 · . . . · a p,q−1 · a pq · a p,q+1 · . . . · aqn = √n
n ∗ wp. (11)
a pq
Similarly,
n
a∗qp
w∗q = √n wq . (12)
aqp

Lemma 3.3. If i, j  {p, q}, then e∗i j = ei j .

Proof. The lemma is straightforward since a∗i j = ai j , w∗j = w j and w∗i = wi .

Lemma 3.4.
2
e∗pq = e pq .
n
Proof. From (10), (11) and (12) we get
  √
n a∗

  ∗       
wq    w p √n aqp wq    
qp
w 
∗ 
e pq = ln a pq · ∗  = ln 
∗
· √  = ln n (a∗qp a pq )2  = 2 ln a pq q  = 2 e pq .
 w p    wq n a∗pq    n wp  n
 n a pq w p 
√

Lemma 3.5. If i < p < j and j  q, then

1
e∗ip ≤ eip + e pq
n
and
1
e∗p j ≤ e p j + e pq .
n
Proof. From (11) it follows that
           
 w∗p   wp n w p   wp 1 wq  1
∗
eip = ln aip  
 = ln aip aqp  = ln aip − ln a pq  ≤ eip + e pq .
 wi wi wq wi n wp n

Similarly,            
 w j   wj n wq   wj 1 wq  1
e∗p j = ln a p j ∗  = ln a p j a pq  = ln a p j + ln a pq  ≤ e p j + e pq .
 wp   wp wp wp n wp n

Lemma 3.6. If i < q < j and i  p, then

1
e∗iq ≤ eiq + e pq
n
and
1
e∗q j ≤ eq j + e pq .
n
 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013 1011
6 Author name / Procedia Computer Science 00 (2018) 000–000

Proof. From (12) it follows that

           
w∗q  

e∗iq = ln aiq  = ln aiq wq n a pq wq  = ln aiq wq + 1 ln a pq wq  ≤ eiq + 1 e pq .
 wi   wi wp   wi n wp  n

Similarly,
           
 w j   wj n w p   wj 1 wq  1
e∗q j = ln aq j ∗  = ln aq j aqp  = ln aq j − ln a pq  ≤ eq j + e pq .
 wq   wq wq wq n wp n

Theorem 3.7.
2
LCI(A∗ ) ≤ LCI(A) + e pq .
n(n − 1)
Proof. Let us notice that the set
K := {(i, j) : i < j and i, j  {p, q}}
n2 −n
has 2 − 2n + 3 elements. The sets

L := {(i, p) : i < p} ∪ {(p, j) : p < j and j  q}

and
M := {(i, q) : i < q and i  p} ∪ {(q, j) : q < j}

have n − 2 elements each. Therefore, using lemmas 3.3, 3.4, 3.5 and 3.6, we get
 
2  2       
LCI(A ) = ∗ ∗
ei j =  ∗
e pq + ∗
ei j + ∗
eip + ∗
ep j + ∗
eiq + eq j  ≤
∗
(n − 1)(n − 2) i< j (n − 1)(n − 2) (i, j)∈K (i,p)∈L (p, j)∈L (i,q)∈M (q, j)∈M

 
2  2    1
  
1
   1
   1 
 e pq + ei j + eip + e pq + e p j + e pq + eiq + e pq + eq j + e pq 
(n − 1)(n − 2) n  n n n n
(i, j)∈K (i,p)∈L (p, j)∈L (i,q)∈M (q, j)∈M

2 2
≤ LCI(A) − e pq + (2 + n − 2 + n − 2) e pq ≤
(n − 1)(n − 2) n(n − 1)(n − 2)

−2n + 4n − 4 2
≤ LCI(A) + e pq ≤ LCI(A) + e pq .
n(n − 1)(n − 2) n(n − 1)

Obviously, Theorem 3.7 does not prove the correctness of the algorithm since it does not guarantee that the index
must be reduced in each iteration. Contrary, it allows even its slight increase. However, the estimations in proofs of
lemmas 3.5 and 3.6 seem to be too restrictive. If half the errors ei j does not increase either for i or for j equal to p or
q, then LCI(A∗ ) must be smaller than LCI(A). In fact, simulations show a significant reduction of inconsistency in
every step of the algorithm. Undoubtedly, more theoretical investigation is necessary.
On the other hand, the lack of proof of LCI convergence to 0 is not a hindrance to use the algorithm, as we can
always stop it after an arbitrarily chosen number of iterations. We must have in mind that our aim is not making the
matrix almost consistent but correction of a few worst entries.
1012 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013
Author name / Procedia Computer Science 00 (2018) 000–000 7

4. Examples

In this section we consider two examples of the algorithm’s operation. In the following two examples we list:
a number of  iteration
 N, a corrected PC matrix A, a priority vector w = [wk ] given by (7), a consistent matrix
wi
B= [bi j ] = w j , a matrix of errors E= [ei j ] given by (4) for i < j and LCI(A). We also arbitrarly define a threshold ε.
In each iteration we frame the maximum value of ei j , which determines the choice of the element of A to be
changed.
Example 4.1. Let us consider a PC matrix  
 1 1 4 
 
A =  1 1 1 
1 
4 1 1
and a threshold ε = 0.5.
N A w B E LCI
   2   2 4   
 1 2 3 2 3   0 2 2
 1 1 4   2 3  ln 2 ln 2 
     − 2 2   3 3 
1  1 1 1   1   2 3 1 2 3   ∗ 0 2
ln 2  2 ln 2 ≈ 1.386
1  − 23   −4   3
4 1 1
2
2 2 3 2− 3 1 ∗ ∗ 0
 2   8   10 14   
 2 9   1 2 9 2 9   0 4 4
 1 2 3 4 
 9 ln 2 9 ln 2 

 − 23   2− 29   − 10 4 
1 2 9 
 4
2  2 1 1   2 9  ∗ 0 4  ln 2 ≈ 0.924
9 ln 2 
  2   − 14   3
1
1 1
4 
4 2− 3 2 9 2− 9 1 ∗ ∗ 0
 10   28   38 46   
 1 2 9 4   2 27   1 2 27 2 27   0 8 8 
 27 ln 2 27 ln 2  
 − 10   − 10   − 38 8 
3  2 9 1 1   2 27   2 27 1 2 27   ∗ 0 8
ln 2  8
9 ln 2 ≈ 0.616
 1   −2   − 46 8   27 
4 1 1 2 3 2 27 2− 27 1 ∗ ∗ 0
 38   92   130 146 
 
 1 2 27 4   2 81   1 2 81 2 81   0 16
ln 2 16
ln 2 
 − 38   − 38   − 130 16 
 81 81 
4  2 27 1 1   2 81   2 81 1 2 81  
 ∗ 0 16
ln 2  16
27 ln 2 ≈ 0.411
 1   −2   − 146 16  81 
4 1 1 2 3 2 81 2− 81 1 ∗ ∗ 0
As we can notice, in each step of the algorithm all errors ei j decrease and are equal, so we always correct the element
a12 . The LCI(A) in iteration N + 1 is equal to 23 of LCI(A) in iteration N.
Example 4.2. Now, let us consider a PC matrix
 
 1 2 5 7
 0.5 1 3 4 
A =  
 0.2 0.33 1 1 
0.143 0.25 1 1
and a threshold ε = 0.1.
N A w B E LCI
       
 1 2 5 7  2.893   1 1.849 5.706 6.651   0 0.079 0.132 0.051 
 0.5 1 3 4   1.565   0.541 1 3.087 3.598   ∗ 0 0.029 0.106 
1         0.184
 0.2 0.33 1 1   0.507   0.175 0.324 1 1.166   ∗ ∗ 0 0.154 
   
0.143 0.25 1 1 0.435 0.150 0.278 0.858 1 ∗ ∗ ∗ 0
       
 1 2 5 7 
  2.893   1 1.849 5.490 6.921   0 0.079 0.093 0.011 
 0.5 1 4   1.565   0.541 1 2.970 3.744   
3  ∗ 0 0.010 0.066 
2   
 0.527 
 
 0.182 0.337 1 1.261 
  0.113
 0.2 0.33 1 1.166     
 ∗ ∗ 0 0.079 
 
0.143 0.25 0.858 1 0.418 0.144 0.267 0.793 1 ∗ ∗ ∗ 0
       
 1 2 5.490 7 
  2.961   1 1.892 5.761 7.084   0 0.056 0.048 0.012 
 0.5 1 3 4   1.565   0.529 1 3.045 3.744   ∗ 0

0.015 0.066 
3  
       0.083
 0.182 0.33 1 1.166   0.514   0.174 0.328 1 1.230   ∗ ∗ 0 0.053 

     
0.143 0.25 0.858 1 0.418 0.141 0.267 0.813 1 ∗ ∗ ∗ 0
 Jacek Szybowski / Procedia Computer Science 126 (2018) 1006–1013 1013
8 Author name / Procedia Computer Science 00 (2018) 000–000

It is easy to observe that the maximum errors and LCI significantly decrease, in spite of the fact that two elements of
E slightly increase after the second iteration.

5. Conclusions

We have introduced a new algorithm of the pairwise comparison data improvement. Its main advantage are speed
and simplicity. Its weakness is the lack of formal proof that the inconsistency of subsequent matrices must decrease.
The replacement of the worst PC matrix entries with the quotients of the geometric means of the appropriate rows
seems very natural, however the algorithm should be implemented and carefully tested.
The above method has been invented for the improvement of a few PC matrix entries which could falsify the
hierarchy of alternatives. In fact, its application in order to improve all or most of matrix elements seems to be
pointless. Our main goal is always to obtain a priority vector, which can be done immediately, for example, by means
of the RGMM. However, if we know that most of input data is unreliable, we cannot expect that the output data will
be satisfactory.
The least reliable PC matrix entries elimination also seems to be a perfect idea for the choice of the best matrix
generators introduced by Koczkodaj and Szybowski 17 .

Acknowledgements

This work was supported by the National Science Centre, Poland as a part of the project no.
2017/25/B/HS4/01617, and by the Faculty of Applied Mathematics of AGH UST within the statutory tasks subsi-
dized by the Polish Ministry of Science and Higher Education, grant no. 11.11.420.004.

References

1. Saaty TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 1977; 15(3): 234–281.
2. Brans JP, Vincke PH. A preference ranking organisation method: The PROMETHEE method for MCDM. Management Science 1985; 31(6):
647–656.
3. Ziemba P, Watróbski
˛ J, Zioło M, Karczmarczyk A. Using the PROSA Method in Offshore Wind Farm Location Problems. Energies 2017;
10(11): 1755.
4. Piegat A, Sałabun W. Identification of a multicriteria decision-making model using the characteristic objects method. Applied Computational
Intelligence and Soft Computing 2014; 2014: Article ID 536492, 14 pages.
5. Faizi S, Rashid T, Sałabun W, Zafar S, Watróbski
˛ J. Decision making with uncertainty using hesitant fuzzy sets. International Journal of Fuzzy
Systems 2018; 20(1): 93–103.
6. Sałabun W, Piegat A. Comparative analysis of MCDM methods for the assessment of mortality in patients with acute coronary syndrome.
Artificial Intelligence Review 2017; 48(4): 557–571.
7. Aguarón J, Moreno-Jiménez JM. The geometric consistency index: Approximated threshold. European Journal of Operational Research 2003;
147(1): 137–145.
8. Brunelli M, Fedrizzi M. Axiomatic properties of inconsistency indices for pairwise comparisons. Journal of the Operational Research Society
2015; 66(1): 1–15.
9. Szybowski J. The cycle inconsistency index in pairwise comparisons matrices. Knowledge-Based and Intelligent Information and Engineering
Systems of the 20th International Conference, KES-2016, Procedia Computer Science 2016; 96: 879–886.
10. Koczkodaj WW, Urban R. Axiomatization of inconsistency indicators for pairwise comparisons. International Journal of Approximate Rea-
soning 2018; 94: 18–29.
11. Crawford G, Williams C. A note on the analysis of subjective judgement matrices. Journal of Mathematical Psychology 1985; 29: 387–405.
12. Holsztynski W, Koczkodaj WW. Convergence of inconsistency algorithms for the pairwise comparisons. Inf. Process. Lett. 1996; 59(4):197–
202.
13. Koczkodaj WW, Szwarc R. On axiomatization of inconsistency indicators in pairwise comparisons. Fundamenta Informaticae 2014;
32(4):485–500.
14. Koczkodaj WW, Szybowski J. The limit of inconsistency reduction in pairwise comparisons. International Journal of Applied Mathematics
and Computer Science 2016; 26(3): 721–729.
15. Koczkodaj WW. A new definition of consistency of pairwise comparisons. Mathematical and Computer Modelling 1993; 18(7): 79–84.
16. Koczkodaj WW, Kosiek M, Szybowski J, Xu D. Fast convergence of distance-based inconsistency in pairwise comparisons. Fundamenta
Informaticae 2015; 137: 355–367.
17. Koczkodaj WW, Szybowski J. Pairwise comparisons simplified. Appl. Math. Comput. 2015; 253: 387–394.