See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/266079064

Multi-criteria decision methodology for selecting maintenance key

performance indicators

Article · September 2014

CITATION READS

1 181

1 author:

José Mendonça Dias

New University of Lisbon
18 PUBLICATIONS   127 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

OPTIMIZAÇÃO DO PERÍODO DE SUBSTITUIÇÃO PREVENTIVA DE COMPONENTES EM FUNÇÃO DOS CUSTOS View project

All content following this page was uploaded by José Mendonça Dias on 25 September 2014.

The user has requested enhancement of the downloaded file.

This article was downloaded by: [b-on: Biblioteca do conhecimento online UAlgarve]
On: 12 September 2014, At: 12:07
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer
House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Management Science and

Engineering Management
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/tmse20

Multi-criteria decision methodology for selecting

maintenance key performance indicators
a b
César Duarte Freitas Gonçalves , José António Mendonça Dias & Virgílio António Cruz
b
Machado
a
Departamento de Engenharia Mecânica, Instituto Superior de Engenharia, Universidade
do Algarve, Campus da Penha, 8005-139 Faro, Portugal
b
Departamento de Engenharia Mecânica e Industrial, Faculdade de Ciências e
Tecnologia, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
Published online: 10 Sep 2014.

To cite this article: César Duarte Freitas Gonçalves, José António Mendonça Dias & Virgílio António Cruz Machado (2014):
Multi-criteria decision methodology for selecting maintenance key performance indicators, International Journal of
Management Science and Engineering Management

To link to this article: http://dx.doi.org/10.1080/17509653.2014.954280

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall
not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other
liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 International Journal of Management Science and Engineering Management, 2014
http://dx.doi.org/10.1080/17509653.2014.954280

Multi-criteria decision methodology for selecting maintenance key performance indicators

César Duarte Freitas Gonc alvesa*, José António Mendonc a Diasb and Virgı́lio António Cruz Machadob
a
Departamento de Engenharia Mecânica, Instituto Superior de Engenharia, Universidade do Algarve, Campus da Penha, 8005-139
Faro, Portugal; bDepartamento de Engenharia Mecânica e Industrial, Faculdade de Ciências e Tecnologia, Universidade Nova de
Lisboa, 2829-516 Caparica, Portugal
(Received 11 April 2014; final version received 8 August 2014)

Many key performance indicators (KPIs) can be used for measuring maintenance performance. Companies must implement
a suitable set of KPIs to measure the performance of maintenance services and the influence of practices implemented in
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014

their activities. Establishing a useful set of maintenance KPIs depends on a company’s maintenance objectives and is highly
related to specific business contexts, strategies, processes, and systems. Maintenance managers deal with the complex tasks
of finding the best performance indicators that can help them to achieve goals. This paper presents a new approach to
selecting relevant maintenance KPIs using a methodology based on the original ELECTRE I, which is a multi-criteria
decision making method. The proposed methodology, which involves decision maker’s preference information, determines
a ranking of possible alternatives following its evaluation according to important criteria. To validate the methodology, a
case study is presented. The results show that this methodology is an effective tool to aid maintenance managers in accurate
KPI selection tasks according to maintenance objectives and strategies. The proposed methodology makes the decision
process more explicit, rational, and efficient.
Keywords: Maintenance; KPI; multi-criteria decision making; MCDM; ELECTRE; SRF

JEL Classification: C44; C52; C88; L00; L25

1. Introduction problem. In this paper a methodology is proposed using

Maintenance management is an important activity in
companies; it has to do with ensuring the smooth operation
of facilities, systems, and equipment, and that good
operating conditions with maximum availability are
achieved, all at an optimized overall cost. Companies
must strive to optimize all their activities to obtain
satisfactory levels of productivity in order to stay
competitive in business. Maintenance objectives are
highly related to the company’s specific business context,
strategies, processes, and systems. Managers need
information about maintenance performance for planning
and controlling maintenance activities.
Since measuring maintenance performance has
become an essential element of strategic thinking,
maintenance managers need a good performance track
record vis-à-vis maintenance processes and results, and
this can be achieved through the development and
implementation of a rigorously defined performance
measurement framework with relevant indicators that are
able to assess important elements of maintenance function
performance.
Establishing a useable set of maintenance performance
indicators depends mainly on maintenance objectives and
the company’s goals. Maintenance performance measure-
ment is, in fact, a multidisciplinary process taking into
account multiple aspects of maintenance activities, and for
this reason, the selection of the best maintenance
performance indicators is a complex task, which can be
formulated as a Multi-Criteria Decision Making (MCDM)
*Corresponding author. Email: cgoncal@ualg.pt
q 2014 International Society of Management Science and Engineering Management
MCDM methods that deal with such issues in order to
assist maintenance managers.
ELECTRE is an MCDM method that draws on the
logic of the outranking model (Roy, 1991) and that has
been applied to various types of decision making
problems. Based on the ELECTRE method, the proposed
methodology was developed to help decision makers
resolve ranking problems, by evaluating several conflict-
ing alternatives according to multiple criteria.
This paper describes the methodology developed,
which joins the original ELECTRE I, which is an MCDM
method for choice type problems, and a complementary
analysis to allow the ranking of alternatives. For eliciting
the weights of criteria, the SRF (Simos – Roy –Figueira)
software is proposed, which uses an algorithm of the
revised Simos procedure (Figueira & Roy, 2002).
The methodology is proposed for selecting relevant
key performance indicators (KPIs) regarding maintenance.
Because the decision maker is typically not familiar with
the mathematical procedures of MCDM methods, an
informatics tool was conceived. Using the popular
software MS Excelw, by means of Visual Basicw for
Applications (VBA), the mathematical procedure of the
methodology has been programmed to provide compu-
tational analysis of decision problems.
In order to validate the proposed methodology, a case
study based on reality is presented. A number of possible
KPIs for measuring maintenance service quality in an
airport were evaluated by two expert maintenance decision
makers. The methodology ranked the alternatives, from
best to worst, and consistently selected KPIs with greater
interest to the decision makers.
 2 C.D.F. Gonc alves et al.
The paper is organized into seven sections. After this
introduction, a maintenance KPIs approach is presented in
Section 2. A review of MCDM is made in Section 3, where
some studies are highlighted, relating mainly to mainten-
ance. In Section 4 a methodology for selecting relevant
maintenance KPIs is proposed. The application of the
proposed methodology is undertaken in Section 5. Section
6 reports results and discusses methodology validation to
assist the maintenance managers in the KPIs selection
process. Finally, Section 7 summarizes and makes final
remarks.
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
2. Maintenance Key Performance Indicators (KPIs)
Companies typically use KPIs to measure the success of
the activities in which they are engaged. Maintenance can
serve as a management system of appropriate indicators to
measure its own performance, that is, to measure both the
expected and actual results of its activities.
A performance indicator is defined by the European
Standard EN 15341 (2007) as a measured characteristic (or
set of characteristics) of a phenomenon, according to a
specific formula that evaluates its evolution. EN 15341
provides three categories and three levels of maintenance
performance indicators to appraise and improve efficiency
and effectiveness in the maintenance of technical assets.
This standard also presents some steps as a methodology
for the selection and use of maintenance performance
indicators. In order to select relevant KPIs, EN 15341
prescribes that the first step is to define the objectives to be
reached. The next step is to find KPIs that allow the
measurement of parameters or aspects related to the
objectives defined. Then, in the search for relevant KPIs, a
first approach can be made, by choosing from among the
list of existing indicators, which after analysis prove to be
of interest for measuring the performance of maintenance
of the intended aspects. Other KPIs can be developed for
specific measurement aspects.
A relevant indicator is a key element in the decision
making, which means that its evaluation and its data
should be related to the performance parameter to be
measured and in accordance with the defined objective.
Some authors provide guidelines for choosing mainten-
ance KPIs that seek to align maintenance objectives with
production objectives (Kumar, Soni, & Agnihotri, 2013b;
Muchiri, Pintelon, Gelders, & Martin, 2011) which are
important for the success of performance measurement
and for satisfying customers’ needs (Pacaiova, Nagyova,
Namesanska, & Grencik, 2013).
Regardless of the type of indicators that are used in
measuring the performance of maintenance, there must be
a procedure for their accurate identification, namely an
algorithm that can be used to establish a group of
indicators that are representative in terms of an
explanatory value of a given process (Pacaiova et al.,
2013). The number of indicators used for each department
in an organization should be limited through the
identification of key features or key factors (Kumar,
Galar, Parida, Stenstrom, & Berges, 2013a). Having a
large number of indicators to measure each maintenance
aspect hinders comprehension and the work for which they
are developed.
Different categories of KPIs and distinct frameworks
have been discussed extensively and proposed in the
literature to monitor and control maintenance activities.
However, few publications propose methodologies for
selecting relevant KPIs, especially in the field of
maintenance. Muchiri et al. (2011) mention that the
literature mainly proposes lists of KPIs but lacks
methodological approaches for selecting them. These
authors are also of the opinion that organizations should
resort to suitable models to identify relevant KPIs for a
maintenance function in a certain context. The selection of
KPIs has usually been derived from the needs and
experience of maintenance managers (Kumar et al., 2013b).
3. Multiple-criteria decision making
Multiple-Criteria Decision Making (MCDM) or Multi-
Criteria Decision Analysis (MCDA) are different designa-
tions for the same concept. These well-known terms are
concerned with analysing and solving decision problems
involving multiple criteria, and are considered a sub-
discipline within operations research. Their purpose is to
aid decision makers when multiple alternatives are
evaluated by multiple criteria, which are often conflicting.
Some authors refer to these matters by a third name,
Multiple-Criteria Decision Aiding (MCDA) (Figueira,
Greco, Roy, & Slowinski, 2013) as a result.
An MCDM approach refers to making preference
decisions over the available alternatives that are
characterized by multiple incompatible criteria. MCDM
problems usually involve discrete variables, which are a
finite number of pre-specified alternatives to be evaluated
against the criteria set. In turn, when the number of
alternatives is infinite, a Multiple Objective Decision
Making (MODM) approach is used. In the case of MODM,
multiple objective programming models are employed to
optimize a set of conflicting objectives subjected to a set of
mathematically defined constraints.
In the context of decision aid, Roy (1991) and Figueira
et al. (2013) note three main types of decision problems:
ranking, sorting, and choice. Ranking problems call for
rank ordering all alternatives from the best to the worst, i.e.
alternatives are ordered by decreasing preference; sorting
problems call for assigning each alternative to an
appropriate predefined class and according to the ordered
preference; choice problems call for selecting a single best
alternative or a smallest subset of alternatives considered
as the best or incomparable alternatives. Many authors
favor the high potential of different MCDM methods.
There are numerous MCDM methods and the literature is
extensive in this field.
The acronym ELECTRE stands for ‘ELimination Et
Choix Traduisant la REalité [Elimination and Choice
Expressing Reality]’ and is a well-known MCDM method
introduced by Bernard Roy in the late 1960s. The
ELECTRE approach uses a preference model based on
 International Journal of Management Science and Engineering Management
the construction of ‘outranking relations’ by paired
comparisons between the alternatives using the concept of
concordance and discordance. ELECTRE I was the first
decision aid method using the concept of outranking
relations (Roy, 1991) and was originally designed to lead to
‘choice-type’ results. The ELECTRE approach has evolved
into a number of variants based on the same concept
covering the different types of decision problems described
above, see Figueira et al. (2013). A complementary analysis
to the ELECTRE I method that introduces the calculation of
aggregate differential concordance and discordance values
(net concordance and net discordance values), also exists
(van Delf & Nijkamp, 1976), which allows for the ranking
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
of alternatives from best to worst.
Some approaches in the literature dealing with the
original ELECTRE I method deserved attention in the
current research. For instance, there are authors proposing
methodologies of decision making in supplier selection
(Sevkli, 2010; Hatami-Marbini & Tavana, 2011), evalu-
ation of transport sustainability (Bojković, Anić, & Pejčić-
Tarle, 2010), bioinformatics investigation of human genes
(Ermatita, Hartari, Wardoyo, & Harjoko, 2011), environ-
mental impact assessment (Kaya & Kahraman, 2011), risk
assessment in tunnel construction (Yazdani-Chamzini,
Yakhchali, & Mahmoodian, 2013) and tourism manage-
ment (Botti & Peypoch, 2013). In another approach, the
original ELECTRE I and the complementary analysis
through the net concordance and net discordance values
were used for selecting appropriate materials (Shanian &
Savadago, 2006). With regard to maintenance, there is also
a fuzzy model based on the original ELECTRE I method
for solving maintenance strategy selection problems
(Vahdani & Hadipour, 2011).
Two recent approaches to the application of MCDM
methods are also to be found, both using the Analytic
Network Process (ANP), an extension of the Analytic
Hierarchy Process (AHP) method: one, for choosing the
most appropriate practices and KPIs in a supply chain
(Cabral, Grilo, & Cruz-Machado, 2012); and the other, for
helping the maintenance manager in the definition and
selection of the relevant KPIs (Horenbeek & Pintelon,
2014). However, these methods have some limitations. For
instance, Cabral et al. (2012) mention that the main
disadvantages of their proposed ANP model are the large
number of comparisons needed and the inconsistency
problems. Horenbeek and Pintelon (2014) emphasize that,
in the application of their ANP approach, sometimes the
decision makers encountered difficulty in expressing
preferences using the defined ratio scale. Another remark
of these authors is that an ANP approach may require
considerable effort and time from the decision makers.
Decision making in the selection of indicators for
maintenance performance measurement can be aided by
conducting MCDM ranking methods.
4. The proposed methodology
A methodology is proposed herein that makes use of the
original ELECTRE I method to select the most important
3
key performance indicators for a maintenance manage-
ment framework.
There are several reasons why the ELECTRE I method
is considered suitable for selecting relevant KPIs. First, the
method has been successfully applied to other ranking
problems, as mentioned above. The method is simple to
apply and many authors, such as Yazdani-Chamzini et al.
(2013), consider that the ELECTRE logic is rational and
the computation process is systematic and well organized.
Furthermore, ELECTRE is the most commonly used
MCDM method for ranking alternatives (Zandi &
Roghanian, 2013).
Some authors discuss some advantages of the ELE-
CTRE method when compared to the limitations of other
methods such as AHP and ANP. The ELECTRE family of
methods can deal with heterogeneous scales, i.e. different
scales for each criterion. Whatever the nature of the scale,
the procedure can run while preserving the original
performance of the alternatives on the criteria (Figueira
et al. 2013). Yazdani-Chamzini et al. (2013) observe that,
because there is no need to impose a rating scale, the
alternatives keep their original concrete verbal meaning.
In addition, the decision maker does not make any
pairwise comparison, since all the comparisons are carried
out by the procedure itself (Figueira et al. 2013). The
ELECTRE method does not allow for compensation of
performances among criteria, i.e. the degradation of
performances on certain criteria cannot be compensated
for by improvements of performances on other criteria
(Figueira et al. 2013).
Saaty and Ozdemir (2003) call attention to Miller’s
theory (Miller, 1954), wherein seven plus or minus two
represents the greatest amount of information that an
observer can give about an object on the basis of
judgement. Thus, an upper limit of seven plus or minus
two elements in AHP analyses is suggested. This is indeed
a limitation of the AHP, advising against its use in solving
decision problems with a large number of alternatives and
criteria.
The ELECTRE method, on the other hand, can deal
with a large number of alternatives. Hatami-Marbini and
Tavana (2011) mentioned that ELECTRE is the preferred
method for problems with a large set of alternatives and
few criteria.
Within the ELECTRE family methods, the decision
maker is asked to assign numerical values to the weights of
the criteria. The calculation of the priority vector of the
AHP is mentioned by many authors as a method to be used
for determining the weights of a set of activities or criteria
(Horenbeek & Pintelon, 2014; Kaya & Kahraman, 2011;
Yazdani-Chamzini et al., 2013). However, Bana e Costa
and Vansnick (2008) address a critical problem regarding
the priority vector and the consistency of the ratio of AHP.
The values of the priority vector do not always preserve
the order of the respective intensities of preferences,
although the ratio proposed to assess the consistency of
human judgments does not detect disturbances. In other
words, the weights calculated by the priority vector of the
AHP method may not correctly reflect the intensity of
judgments established by the decision maker by compar-
 4 C.D.F. Gonc alves et al.
ing pairs of criteria. For that reason, the use of AHP is also
not advised for calculating the weights of the criteria.
For assessing those weights, the proposal here is the
revised Simos procedure aided by SRF (Simos –Roy –
Figueira) software (Figueira & Roy, 2002). This method
attributes an intrinsic weight to each criterion without
dependence on either the range of the scale or on the
encoding to express the score on this scale. On the other
hand, the decision maker has the possibility to express her/
his preferences spontaneously and achieve the outputs in a
coherent way without any application of weighted sum
aggregation techniques. This means that the value of each
criterion is fixed without taking into account the impact
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
that other criteria weight can have. Augusto, Figueira, and
Lisboa (2005) and Figueira et al. (2011) used SRF
software to determine the weights of the criteria for the
resolution of different decision problems through different
ELECTRE methods.
Ranking is very different from choice or sorting and
therefore requires the use of specific methods. ELECTRE I
is a simple method for choice type problems (Roy, 1991).
Nevertheless, their outputs can be worked upon to obtain a
ranking of alternatives (Hatami-Marbini & Tavana, 2011;
Shanian & Savadogo, 2006; van Delf & Nijkamp, 1976;
Yoon & Hwang, 1995; Zandi & Roghanian, 2013).
Toward that goal, calculation of the Net Concordance
(NC) and Net Discordance (ND) values is proposed. These
methods make it possible to rank and select the most
relevant maintenance KPIs following their evaluation by a
decision maker.
Methodological procedures are given in Figure 1 in
order to illustrate the steps in the selection of indicators.
The conceptual framework is intended to aid decision
makers using the methodology.
For the selection of relevant indicators, measuring a
particular aspect of maintenance is necessary in order to
1 by another KPI already selected.
Definition of the maintenance
management objectives
2
Specification of the criteria
for evaluation of the KPIs
3 Delf & Nijkamp, 1976; Yoon & Hwang, 1995). The
Determination of the weights of
criteria using the SRF software
4 for the decision making process with the proposed
Pre-selection of KPIs as
competing alternatives
5
Evaluation of the KPIs
according to each criterion
6 realized that there should be a relevant set of indicators for
Ranking of KPIs using ELECTRE I
and calculation of NC and ND values
Figure 1. Conceptual framework of the proposed methodology.
set the evaluation criteria according to the predefined
objectives to be reached by the maintenance department.
It is also necessary to propose a battery of indicators
(KPIs) focused on the maintenance aspect to be measured.
That cluster of KPIs is the set of competing alternatives
which are evaluated by the decision maker against the set
of criteria.
The evaluation of alternatives represents the convic-
tions of the decision maker, who can mirror her/his
preferences by assigning a level of importance for each
alternative. For defining the assessment scale, we followed
the principles presented by Likert (1932) and Miller
(1954). Likert (1932) indicates that the scales to capture
the perception must present symmetrical options, and
Miller (1954) complements these guidelines considering
that in addition to symmetry, the scales should provide five
or nine response options. A Likert scale assessment of nine
response options is proposed with which the decision
maker can specify her/his level of agreement with each
alternative in relation to different criteria. The higher the
response level in a criterion, the more important is the KPI.
The ELECTRE method is applied in combination with
the calculation procedures of the NC and ND values.
Ordering the set of alternatives in descending order of the
NC values, a ranking list of all alternatives from the best to
the worst is obtained. The final selection should satisfy the
condition that its NC value is at a maximum and its ND
value at a minimum (Yoon & Hwang, 1995). The interest
is to find a set of maintenance performance indicators that
satisfy the conditions to be accepted, and that means
having the highest NC values and the lowest ND values
simultaneously.
A sensitivity analysis must be conducted in order to
contrast the position achieved by the different KPIs,
considering their value of discordance. Moreover, the
decision maker must determine if some KPIs that were
ranked as being preferred are still of interest and should be
selected, or if the measuring maintenance aspect is ensured
The procedures to obtain the weights of criteria by the
SRF software are described in the literature (Figueira &
Roy, 2002; Gonc alves, Dias, & Cruz-Machado, 2014). The
original ELECTRE I method and the calculation of NC
and ND values can also be found in the literature
(Gonc alves et al., 2014; Shanian & Savadogo, 2006; van
mathematical procedures were programmed in ‘Excelw
VBA’ code to provide computational results. The main
goal is to provide a practical and effective informatics tool
approach.
5. Application – case study
Suppose a maintenance department of an important airport
maintenance service quality measurement of their
facilities. Two maintenance managers find it difficult to
establish strict preference judgements, due to the vast
 International Journal of Management Science and Engineering Management
number of alternatives. The fundamental questions are:
‘Which KPIs must be selected to meet the maintenance
quality measurement needs?’ and ‘How many KPIs must
be selected to assist maintenance managers?’ The
proposed methodology was applied to select an optimal
set of KPIs that best measures the performance of the
maintenance sector with regard to the quality of their
service and achieving the goals of the organization. The
decision process for resolving the problem is summarized
as follows.
Step 1: Maintenance management objectives. It is
necessary to understand the needs of the organization
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
in order to answer the question of which indicators are
relevant to the business and related to maintenance.
In general, the organization’s strategic goals are the
efficient management of airport infrastructure, ensur-
ing productivity levels and quality of service to
passengers and airlines, continuously improving the
ideal maintenance service, and lowering costs.
The main maintenance objectives were defined by
the maintenance department and can be summarized as
follows: provide continuous improvement of the
facilities; ensure quality of airport service; optimize
maintenance activities; allow proactive management
and agility; achieve high availability and reliability of
physical assets; provide shared vision of maintenance
performance among employees; and provide a safe
work environment with motivated employees.
Step 2: Specification of criteria. The first way to
approach the problem is to generate appropriate
criteria as constraints in evaluating the set of
alternatives. The literature review revealed the
implications in KPI selection. Three maintenance
management experts were invited to identify the
important criteria. In a subsequent analysis with airport
maintenance managers, the focus was on the main
matters for determining the most important criteria in
the context of maintenance service quality measure-
ment. From all of this, five decision criteria were
settled on and are described as follows.
Maintenance quality (C1): to evaluate how the KPI
quantitatively measures the quality aspects of main-
tenance service; how it allows measuring and
comparing maintenance performance, identifying the
strengths and weaknesses of maintenance service.
Maintenance objectives (C2): to evaluate how the KPI
measures the maintenance state in the fulfillment of
established objectives for maintenance and according
to company goals; how the KPI assists the manage-
ment in setting goals in a process of continuous
improvement.
KPI influence (C3): to evaluate how the result of the
KPI can influence decision making in maintenance;
how the KPI helps to guide future decisions, to track
progress and changes over time.
Understanding and interpretation of KPI (C4): to
evaluate how the KPI expresses a measurement that
easily translates an obtained or expected real result; to
evaluate how easily the KPI gives a perception of
5
reality, the data interpretation, and the analysis of
results without compromising the effectiveness of
maintenance management in decision making.
Sharing results (C5): to evaluate how the results of the
KPI are of interest for being shared with other
company sectors, employees, or customers; how the
results of the KPI are of interest for informing and
motivating employees; to evaluate the common
interest between maintenance and other sectors of the
company in sharing the results of the KPI.
Step 3: Criteria weights determination. Maintenance
managers were called upon to elicit the relative
importance of criteria in an indirect way by using SRF
software. The data required for the revised Simos
procedure used by the SRF software algorithm are
given in Table 1. Through this software, the
appropriate values for the weights of the five criteria
were generated, which are also presented in Table 1 as
normalized weights.
Step 4: Pre-selection of KPIs. A set of competing
alternatives, assuming that all are possible best
choices, was pre-selected. The same indicators often
appear repeatedly in the literature. Muchiri et al.
(2011) provide a well organized structure of indicators
identified as key elements to measure different features
of maintenance management. Among them, the
authors and airport maintenance managers pre-selected
the ones important as alternatives. KPIs were also
collected from the standard EN 15341. Airport
maintenance managers have used a pool of KPIs to
measure and manage maintenance activities, from
which some were also collected for the present study.
Additionally, the authors developed four KPIs. Table 2
presents a set of 20 KPIs having possible interest for
measuring maintenance performance in terms of
service quality.
Step 5: Evaluation of KPIs against criteria. The two
maintenance managers of the airport evaluated the pre-
selected KPIs against the criteria together. They
discussed the importance of each KPI and for each
criterion a Likert scale value was assigned. The
performance of the 20 alternative KPIs evaluated
according to the five criteria is also shown in Table 2.
Step 6: Ranking of KPIs. Making use of the
informatics application built into VBA, 20 rankings
of alternatives were provided. The preference rank
Table 1. Weights of criteria generated by RSF software.
Criteria Criteria Blank Non-normalized Normalized
code rank cards weights weights
C5 1 – 1 224
– – 2 – –
C2 2 – 8 1791
C3 2 – 8 1791
– – 1 – –
C1 3 – 1267 2836
– – 0 – –
C4 4 – 15 3358
Total – 3 (Z value ¼ 15) 100
 Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014

Table 2. Evaluation of the proposed KPIs against criteria.

Proposed KPIs (competing alternatives) Evaluation against criteria

N Description Calculation formula Source C1 C2 C3 C4 C5
1 Rate of incidents due to causes imputed to maintenance No. of incidents with causes imputed to maintenance/(No. of Airport 8 9 9 8 7
(per 1000 passengers) passengers/1000)
2 Average response time to emergency calls (incidents) Total response time to emergency calls/No. of emergency calls Airport 7 8 8 7 7
3 Average time to replacement of functionality Total replacement time/No. of incidents Airport 9 8 7 9 5
4 Availability related to maintenance Total operating time/(Total operating time þ downtime due to EN 15341 (2007) 8 7 8 8 8
maintenance)
5 Availability due to planned and scheduled maintenance Total operating time/(Total operating time þ downtime related to EN 15341 (2007) 9 8 7 7 7
the planned and scheduled maintenance)
6 Mean time between work orders that cause downtime Total operating time/No. of maintenance work orders causing EN 15341 (2007) 8 5 1 5 7
downtime
7 Percentage of systems covered by criticality analysis No. of systems covered by a critical analysis/Total no. of systems EN 15341 (2007) 7 4 4 7 6
8 Mean Time To Repair (MTTR) Total time to restore/No. of failures EN 15341 (2007) 8 6 8 7 8
9 Percentage of working time lost due to accidents Man– hours lost due to injuries of maintenance personnel/Total EN 15341 (2007) 5 7 8 7 9
(severity rate) man– hours worked by maintenance personnel
10 Fulfilment rate of work orders No. of work orders performed as scheduled/Total no. of scheduled EN 15341 (2007) 9 8 9 8 9
work orders
11 Percentage of work orders in backlog No. of overdue work orders/No. of received work orders Muchiri et al. (2011) 9 9 9 9 9
12 Percentage of available man– hours used for proactive Man– hours used for proactive work/Total man– hours available Muchiri et al. (2011) 7 8 7 4 7
work
C.D.F. Gonc alves et al.

13 Percentage of work orders requiring rework (quality of No. of work orders to rework/Total no. of work orders Muchiri et al. (2011) 9 9 9 8 9
execution)
14 Rate of planned work (planning intensity) Planned work/Total work done Muchiri et al. (2011) 8 3 4 8 8
15 Quality of planning No. of work orders requiring rework due to planning/Total no. of Muchiri et al. (2011) 9 8 8 8 9
work orders
16 Quality of scheduling No. of work orders with delayed execution due to material or Muchiri et al. (2011) 9 8 9 7 6
manpower/Total no. of work orders
17 Execution Index Hours of work orders planned and executed/Expected hours of Developed 7 9 8 7 7
work orders planned
18 Rate of customer complaints due to causes imputed to No. of complaints with cause imputed to maintenance/(No. of Developed 8 7 9 8 9
maintenance (per 1000 passengers) passengers/1000)
19 Maintenance Quality Index Total hours of maintenance/Rework hours Developed 8 9 8 5 8
20 Maintenance Performance Index Expected hours of maintenance/Hours used for maintenance Developed 3 7 6 5 5
 International Journal of Management Science and Engineering Management 7

Table 3. Ranking of KPIs by concordance in descending order. work done to ensure that maintenance work is not
left to chance. Work orders are reworked as a result
Net values Ranking
Alternative, of low quality work or work that was initially poorly
N NC ND Rank C Rank D done.
. In third place was ranked the ‘fulfillment rate of
11 152,387 2190,000 1 1
13 122,165 2160,161 2 2 work orders’, which is identified by the standard EN
10 100,673 2128,535 3 3 15341 as an organizational indicator. This indicator
3 82,534 2101,294 4 4 provides the percentage of work executed within the
1 77,981 2 94,813 5 5 scheduled time and that also fulfills the explicit
15 77,390 2 91,803 6 6 tasks in the work orders.
16 47,389 2 50,338 7 8
. The ‘average time to replacement of functionality’
18 41,268 2 52,597 8 7
4 15,745 2 19,931 9 9 and the ‘number of incidents due to causes imputed
5 07,988 2 14,539 10 10 to maintenance per each thousand passengers’ were
17 2 23,510 2 11,079 11 11
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014

the KPIs ranked in fourth and fifth places,

19 2 23,654 63,972 12 14 respectively. These indicators were already used in
14 2 34,403 88,398 13 15
8 2 40,222 47,772 14 13 the airport to monitor the efficiency and effective-
2 2 45,002 31,863 15 12 ness of the maintenance department in carrying out
9 2 74,403 103,287 16 16 its job. They can help to manage and optimize the
12 2 109,924 131,710 17 18 planning of maintenance activities, seeking con-
7 2 113,061 124,794 18 17 tinuous improvement of the airport facilities and
6 2 115,444 154,671 19 19
20 2 145,897 168,622 20 20 quality of service to passengers and airlines.
. In the sixth rank was placed the ‘quality of planning’,
which obtains the percentage of work orders requiring
based on the NC is given in Table 3, where the KPIs are rework due to failures in planning. Planning is
presented in descending order of the NC value. essential to maximize the efficiency of maintenance
and to ensure the necessary time to carry out the tasks
The proposed methodology allowed the ranking of according to the required procedures. Managers
KPIs according to the decision makers’ preferences. From report that sometimes teams have to repeat or add
the analysis of the computed results, taking into account tasks as a result of poor job planning. It also happens
the ranking by concordance versus discordance, a set of that there are tasks that have to be repeated owing to
relevant KPIs can be selected. insufficient time having been allocated to them.
According Muchiri et al. (2011), this is an important
indicator for measuring work planning.
6. Results and discussion Note that with only the six highest-ranked KPIs, some
The six top-ranked alternatives are classified with clear important aspects of maintenance quality will not be
evidence, where ‘Rank C’ equals ‘Rank D’. That is, they measured. Based on the experience of the authors and
achieved the same ranking, both in descending order of the airport maintenance managers, it was observed that the
NC value and in ascending order of the ND value. For two subsequent ranked KPIs should be selected consider-
subsequent ranked alternatives, although both classifi- ing the analysis as follows.
cations do not establish the same order, there are no very
significant differences. Thus, it establishes a balanced . The KPI ranked in seventh position is entitled
pattern of preferences in the selection of the main KPIs for ‘quality of scheduling’ and provides the percentage
measuring maintenance service quality at the airport. of delayed work orders due to lack of materiel or
The human brain can only handle four to eight manpower. The maintenance department must
measurements intended to quantify one aspect (Kumar ensure that all the necessary resources are available
et al., 2013a), which suggests that it is reasonable to use six before the work commences. Apart from ensuring
performance indicators for measuring each maintenance that planned activities are realized within the
aspect. Therefore, the six top KPIs were analysed as expected time, it is essential that they be scheduled
follows. in light of the availability of skilled manpower and
the necessary materials for work. According Muchiri
. The top two KPIs are identified by Muchiri et al. et al. (2011), this is a work scheduling indicator that
(2011) as indicators to measure the performance in should not be ignored.
work execution. The ‘percentage of work orders in . It was realized that there should be a measurement
backlog’ and ‘percentage of work orders requiring that relates potential customer dissatisfaction with
rework’ were validated to measure how scheduled maintenance performance. That information can be
activities are carried out within the allocated time given by the eighth ranked KPI, which was developed
and how maintenance work is done effectively. The by the authors in order to provide a rate of customers’
maintenance managers consider important the (passengers and airlines) complaints due to causes
measure of schedule compliance and the quality of imputed to maintenance per thousand passengers.
 8 C.D.F. Gonc alves et al.
The maintenance managers are in agreement that the
first eight KPIs measure the main aspects of maintenance
service quality in the airport and are also linked to the
organization objectives. To ease understanding in
evaluating an aspect of maintenance, Kumar et al.
(2013a) hold that a number of indicators greater than
eight should be avoided. This belief also derives from
Miller’s theory (Miller, 1954).
Therefore, the eight KPIs ranked highest with respect
to the concordance and in which discordance shows no
dispersion in the set were selected as measures. The
selected KPIs cover the main aspects for measuring
maintenance service quality, such as the management and
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
execution of work orders, the quality of work executed, the
control of events (activities, rework, incidents), the quality
of planning and scheduling of maintenance activities, the
use of time and agility of working teams, and the
contribution of maintenance to customer satisfaction.
The methodology quantitatively compares the interest
of KPIs for maintenance, through concordance values and
also by checking the discordance, according to the
preference and experience of managers. This approach is
not intended to group multi-criteria decisions, which
implies the consensus of opinion of maintenance managers
in determining the weights of the criteria and the
evaluation of KPIs.
Facility and consistency were achieved in the
procedures for the evaluation of the criteria and
alternatives according to the preference information of
the decision makers. The procedures of the methodology
were well accepted by maintenance managers and both
informatics tools allowed for the easy collection of data
and also the quick achievement of results.
7. Conclusions
A new approach has been presented in order to aid
maintenance managers in the difficult task of selecting
relevant KPIs for measuring the performance of
maintenance services.
In the literature, few applications are addressed for
selecting KPIs for maintenance. The majority of these
contributions address problems with compensatory
methods such as AHP and ANP, of which considerable
limitations were found.
Some approaches were found that dealt with the original
ELECTRE I, but none for the selection of KPIs. ELECTRE I
is an outranking method for choice type problems that,
combined with net concordance and net discordance values
calculation, can provide ranking results. A simple ranking
MCDM methodology based on the original ELECTRE I
method has been proposed that proved to be a suitable and
efficient tool for dealing with the type of problems presented
in the case study. However, the methodology can be used
even when dealing with ranking problems that are different
from those addressed in this paper.
In the airport case study used herein for exploratory
demonstration, the proposed methodology was applied to
ranking and selecting a set of KPIs with the preference of
maintenance managers, and the results were well accepted.
A ranking list of alternatives was generated, from the best
to the worst, which was considered acceptable by the
maintenance managers of the airport. The KPIs ranked in
the first eight positions meet the needs of intended
measurement, covering the main aspects to be ensured for
maintenance service quality in airports.
The proposed methodology requires little effort and
time from the decision maker. It provides decision support
and guidance for the selection of KPIs aligned with the
organization’s objectives. Moreover, it reduces the level of
subjectivity with which the decision maker reflects her/his
preference of relevant KPIs to measure a particular aspect
of maintenance.
The proposed approach shows advantages when
seeking to rank a wide range of alternatives. The
methodology developed is both practical and adequate
for maintenance managers that find it difficult to establish
direct preference in the selection of KPIs facing a large
number of alternatives. Furthermore, the proposed
approach avoids the limitations found in using AHP and
ANP. The decision maker evaluates the KPIs directly in
relation to criteria. There is no need to draw comparisons
between KPIs, which become ambiguous when the
decision maker has many doubts about them. This makes
the decision process more explicit, rational, and efficient.
The authors believe that the proposed methodology
can support both maintenance managers in industry and
academics interested in this strand of research.
The methodology presented is an innovation in this
proposed application area. In further work, the method-
ology can be extended to the ranking and selection of KPIs
for other maintenance aspects or in different business
contexts. Future work can compare the results obtained in
this study with those achieved through the use of different
MCDM ranking methods.
Funding
This work is supported by a cooperation protocol between
Instituto Superior de Engenharia of Universidade do Algarve
(ISE/UALG) and Faculdade de Ciências e Tecnologia of
Universidade Nova de Lisboa (FCT/UNL). The authors
acknowledge the support of UNIDEMI – R&D Unit in
Mechanical and Industrial Engineering of FCT/UNL.
References
Augusto, M., Figueira, J., & Lisboa, J. (2005). An application of
multi-criteria approach to assessing the performance of
Portugal’s economic sectors – methodology, analysis and
implications. European Business Review, 17(2), 113– 132.
Bana e Costa, C. A., & Vansnick, J. -C. (2008). A critical analysis
of the eigenvalue method used to derive priorities in AHP.
European Journal of Operational Research, 187, 1422– 1428.
Bojković, N., Anić, I., & Pejčić-Tarle, S. (2010). One solution for
cross-country transport-sustainability evaluation using a
modified ELECTRE method. Ecological Economics, 69(5),
1176– 1186.
Botti, L., & Peypoch, N. (2013). Multi-criteria ELECTRE
method and destination competitiveness. Tourism Manage-
ment Perspectives, 6, 108– 113.
 International Journal of Management Science and Engineering Management
Cabral, I., Grilo, A., & Cruz-Machado, V. (2012). A decision-
making model for lean, agile, resilient and green supply
chain management. International Journal of Production
Research, 50(17), 4830– 4845.
EN 15341. (2007). Maintenance key performance indicators.
European Committee for Standardization (CEN). Brussels,
Belgium.
Ermatita, Z., Hartati, S., Wardoyo, R., & Harjoko, A. (2011).
ELECTRE methods in solving group decision support
system bioinformatics on gene mutation detection simu-
lation. International Journal of Computer Science &
Information Technology, 3(1), 40 – 52.
Figueira, J., Almeida-Dias, J., Matias, S., Roy, B., Carvalho, M.,
& Plancha, C. (2011). ELECTRE TRI-C, a multiple criteria
decision aiding sorting model applied to assisted reproduc-
tion. International Journal of Medical Informatics, 80,
Downloaded by [b-on: Biblioteca do conhecimento online UAlgarve] at 12:07 12 September 2014
262– 273.
Figueira, J., Greco, S., Roy, B., & Slowinski, R. (2013). An
overview of ELECTRE methods and their recent extensions.
Journal of Multi-Criteria Decision Analysis, 20, 61 –85.
Figueira, J., & Roy, B. (2002). Determining the weights of
criteria in the ELECTRE type methods with a revised Simos’
procedure. European Journal of Operational Research, 139,
317– 326.
Goncalves, C., Dias, J., & Cruz-Machado, V. (2014). Decision
methodology for maintenance KPI selection – based on
ELECTRE I. 25 – 27 July 2014. In J. Xu, V. A. Cruz-
Machado, B. Lev, & S. Nickel (Eds.), Proceedings of the
Eighth International Conference on Management Science
and Engineering Management. Lisbon, Part V, Paper 87
(pp. 1001– 1014), Vol. 281 of Advances in Intelligent
Systems and Computing.: Springer – Verlag Berlin Heidel-
berg. doi:10.1007/978-3-642-55122-2_87.
Hatami-Marbini, A., & Tavana, M. (2011). An extension of the
ELECTRE I method for group decision-making under a
fuzzy environment. Omega, 39(4), 373– 386.
Horenbeek, A., & Pintelon, L. (2014). Development of a
maintenance performance measurement framework – using
the analytic network process (ANP) for maintenance
performance indicator selection. Omega, 42, 33– 46.
Kaya, T., & Kahraman, C. (2011). An integrated fuzzy AHP–
ELECTRE methodology for environmental impact assess-
ment. Expert Systems with Applications, 38(7), 8553– 8562.
Kumar, U., Galar, D., Parida, A., Stenstrom, C., & Berges, L.
(2013a). Maintenance performance metrics: A state-of-the-
art review. Journal of Quality in Maintenance Engineering,
19(3), 233– 277.
Kumar, J., Soni, V., & Agnihotri, G. (2013b). Maintenance
performance metrics for manufacturing industry. Inter-
View publication stats
9
national Journal of Research in Engineering and Technology,
2(2), 136–142.
Likert, R. (1932). Technique for measurement of attitudes.
Archives of Psychology, 140(1), 5 – 55.
Miller, G. (1954). The magical number seven, plus or minus two:
Some limits on our capacity for processing information.
Psychological Review, 101(2), 343– 352.
Muchiri, P., Pintelon, L., Gelders, L., & Martin, H. (2011).
Development of maintenance function performance
measurement framework and indicators. International
Journal of Production Economics, 131(1), 295– 302.
Pacaiova, H., Nagyova, A., Namesanska, J., & Grencik, J. (2013).
Systematic approach in maintenance management improve-
ment. International Journal of Strategic Engineering Asset
Management, 1(3), 228– 237.
Roy, B. (1991). The outranking approach and the foundations of
ELECTRE methods. Theory and Decisions, 31(1), 49 – 73.
Saaty, T., & Ozdemir, M. (2003). Why the magic number seven
plus or minus two? Mathematical and Computer Modelling,
38(3 – 4), 233– 244.
Sevkli, M. (2010). An application of the fuzzy ELECTRE
method for supplier selection. International Journal of
Production Research, 48(12), 3393– 3405.
Shanian, A., Savadogo, O., & ELECTRE, I. (2006). decision
support model for material selection of bipolar plates for
polymer electrolyte fuel cells applications. Journal of New
Materials for Electrochemical Systems, 9(3), 191– 199.
Vahdani, B., & Hadipour, H. (2011). Extension of the ELECTRE
method based on interval-valued fuzzy sets. Soft Computing,
15, 569– 579.
van Delf, A., & Nijkamp, P. (1976). A multi-objective decision
model for regional development, environmental quality
control and industrial land use. Fifteenth European Congress
of the Regional Science Association. Papers of the Regional
Science Association, 36(1), 35 – 57.
Yazdani-Chamzini, A., Yakhchali, S., & Mahmoodian, M.
(2013). Risk ranking of tunnel construction projects by using
the ELECTRE technique under a fuzzy environment.
International Journal of Management Science and Engin-
eering Management, 8(1), 1 – 14.
Yoon, K. P., & Hwang, C. -L. (1995). Multiple attribute decision
making: An introduction Sage University Paper Series on
Quantitative Applications in the Social Sciences. Thousand
Oaks, CA: Sage.
Zandi, A., & Roghanian, E. (2013). Extension of Fuzzy
ELECTRE based on VIKOR method. Computers and
Industrial Engineering, 66(2), 258– 263.