Multi-Criteria

Decision Analysis
Tools

Developed by Dr. Martyn Jones, The University of

Manchester, January 2016.
 About this module

Content and learning objectives:

This module is an introduction to MCDA. It gives a brief overview of MCDA and two
frequently used methods.
 What is multi-criteria decision
analysis
Multiple-criteria decision-making (MCDM) or multiple-criteria decision
analysis (MCDA) is a sub-discipline of operations research that explicitly
evaluates multiple conflicting criteria in decision making (both in daily life
and in settings such as business, government and medicine). Conflicting
criteria are typical in evaluating options: cost or price is usually one of the
main criteria, and some measure of quality is typically another criterion,
easily in conflict with the cost. In purchasing a car, cost, comfort, safety,
and fuel economy may be some of the main criteria we consider – it is
unusual that the cheapest car is the most comfortable and the safest one.
In portfolio management, managers are interested in getting high returns
while simultaneously reducing risks; however, the stocks that have the
potential of bringing high returns typically carry high risk of losing money.
In a service industry, customer satisfaction and the cost of providing
service are fundamental conflicting criteria.
 What is multi-criteria decision
analysis
In their daily lives, people usually weigh multiple criteria implicitly and may
be comfortable with the consequences of such decisions that are made
based on only intuition.[1] On the other hand, when stakes are high, it is
important to properly structure the problem and explicitly evaluate multiple
criteria.[2] In making the decision of whether to build a nuclear power plant
or not, and where to build it, there are not only very complex issues
involving multiple criteria, but there are also multiple parties who are
deeply affected by the consequences.
Structuring complex problems well and considering multiple criteria
explicitly leads to more informed and better decisions. There have been
important advances in this field since the start of the modern multiple-
criteria decision-making discipline in the early 1960s. A variety of
approaches and methods, many implemented by specialized
decision-making software,[3][4] have been developed for their application in
an array of disciplines, ranging from politics and business to the
environment and energy.
MCDM is concerned with structuring and solving decision and
planning problems involving multiple criteria. The purpose is
to support decision-makers facing such problems. Typically,
there does not exist a unique optimal solution for such
problems and it is necessary to use decision-makers'
preferences to differentiate between solutions.
 What is multi-criteria decision
analysis
Multi-criteria decision analysis (MCDA) is a formal, structured and transparent decision
making methodology. Its ism is to assist groups or individual decision makers to explore
their decisions in the case of complex situations with multiple criteria.

MCDA does not provide the ‘right’ answer.

MCDA does not provide an objective analysis.
MCDA does not relieve decision makers of the responsibility of making difficult judgments.

MCDA assists the decision maker in confidently reaching a decision by:

• enabling decision makers to gain a better understanding of the problem faced;

• organising and synthesising the entire range of information;
• integrating objective measurements with value judgements;
• making explicit and managing the decision maker’s subjectivity; and
• ensuring that all criteria and decision factors have been taken properly into account.
 MCDA vs unstructured decision
making process
All decisions are subjective.
All decision makers are biased.

Avoiding bias and subjectivity may be feasible only if a problem is sufficiently simplified,
such that some numeric objective criteria could be designed:

We may simplify evaluation of a chemical process to an individual

criteria, such as RISK, where risk is a product of HAZARD and
EXPOSURE. If HAZARD is then quantified numerically by using e.g., a
toxicity scale, and EXPOSURE is known from materials flows, than
RISK could be evaluated numerically and the process option with the
minimum RISK value would be considered the best option.

But the process with minimum RISK value is not automatically the
best one, if we consider more criteria: energy, cost, resource
efficiency, etc. Which of these criteria are more important?
 MCDA vs unstructured decision
making process
Through the use of MCDA a larger number, of perhaps individually less important,
indicators do not get ignored in the final decision during a simplification that relies too
heavily on a small number of key criteria.

Unstructured decision making often fails to make use of or consider the uncertainty
surrounding different criteria.

MCDA can be used in conjunction with Monte Carlo simulation to take account of
modelled uncertainty of the criteria values, uncertainty of the subjective criteria
preferences and provide a known level of certainty in the proposed decision, unachievable
in unstructured decision making.
 MCDA methodology

MCDA is an umbrella term for a range of tools and methodologies. The level of complexity,
interaction with the decision maker and level of detail utilised in the decision making
process can vary substantially.

In general the decision maker follows the same process:

1. Identify multiple criteria on which to base their decision;

2. Identify multiple alternative solutions to their decision;
3. Provide (subjective) ranking or weighting of criteria; and
4. Provide values, rankings or weighting of alternatives for each criteria.
 MCDA methodology

The Analytical Hierarchy Process (AHP)

AHP was developed in the late 1970s. Today it is the most widely used MCDA method.

AHP generates all criteria weighting and alternative preference within each criteria by
eliciting these values from the decision maker through a series of pairwise comparisons,
as opposed to utilising numerical values directly.

Thus, a complex decision is reduced to a series of simpler ones, between pairs of

alternative values within criteria or between pairs of criteria. The decision maker’s
preference is always explicit. However, the decision maker may be asked to make very
many small decisions. Hence, it becomes important to generate an optimized hierarchy of
criteria and alternatives, to reduce the number of pairwise decisions.
 AHP

Step 1: Construct the problem hierarchy

Model, usually visually, the problem decision identifying relationships between criteria
and alternatives.

Step 2: Pairwise comparison of criteria

Undertake pairwise comparison between criteria, identifying decision maker preference
for criteria on which alternatives are evaluated.

Step 3: Pairwise comparison of alternatives within each criterion

Undertake pairwise comparison between alternatives based on their performance within
each criterion.

Step 4: Compute the vector of criteria weights

From a matrix of pairwise comparison results AHP utilises a variety of matrix
transformations to calculate criteria weight vectors representing normalised criteria
weightings.
 AHP

Step 5: Compute the matrix of alternative scores

From the results of the pairwise comparisons on alternatives within each criterion a nxm
(where n is the number of criteria and m is the number of alternatives) matrix is
constructed representing the normalised performance (score) of each alternative for each
criteria.

Step 6: Ranking the alternatives

Utilising the vectors of criteria weights and the matrix of alternative scores a global score
and hence ranking for each alternative is calculated using:

where: a is the alternative, c is the criteria, g is the global score of the alternative, w is the
criteria weight and s is the alternative score. A function of the ranking equation,
aggregating across each criteria means that trade-offs between criteria in fundamental to
the final ranking.
 AHP

Problem Hierarchy
The problem hierarchy provides a structured, usually visual, means of modelling the
decision being processed. As the first step in the analytical hierarchy process the creation
of a hierarchy that models the decision problem enables decision makers to increase their
understanding of the problem, its context and, in the case of group decision making, see
alternative approaches to the problem across different stakeholders.

The AHP problem hierarchy consists of a goal (the decision), a number of alternatives for
reaching that goal, and a number of criteria on which the alternatives can be judged that
relate to the goal.

Here as an example a simple

AHP hierarchy was generated
in DECERNS tool.
 AHP

For most realistic analyses criteria are multi-tiered. Criteria weightings are calculated as
pairwise comparisons.

Consider a problem hierarchy

with 12 criteria on which the
three alternatives are to be
judged. For pairwise
comparison the first criteria
will be compared against the
remaining 11, the second
criteria against the remaining
10, the third criteria against
the remaining 9 etc. In total 66
(11+10+9+8+7+6+5+4+3+2+1)
pairwise comparisons will
need to be undertaken in
order to determine the
weighting of the 12 criteria.
 AHP

Pairwise Comparisons

Within AHP pairwise comparison is the process of comparing entities in pairs so as to judge
which is preferred and by how much. Comparisons are undertaken to determine criteria
weighting and also assess the value or score of different alternatives within each criteria.

A 9-point scale to elicit the scale of preference from a decision maker:

the more preferable entity within the pair scores

1 when it is showing no preference

3 when it is showing moderate preference
5 when it is showing strong preference
7 when it is showing very strong preference
9 when it is showing extreme preference.

The less preferable entity within the pair scores the inverse, for example the less preferable
entity where the more preferable entity shows very strong preference would score 1/7.
 AHP

Groups of pairwise comparisons are undertaken between every alternative value within a
single criteria, and every criteria within the goal (or for multi-tier hierarchies within their
parent criteria). For each group a matrix is completed with the results of the pairwise
comparison, such as that shown in the table below, following the example from figure on
Slide 12.

The results of the matrix would provide the normalised criteria weights for criteria A1 to A4.
Similar matrices would be completed for criteria B1 to B4, for C1 to C4 and also one
comparing criteria A, B and C. Finally, pairwise comparisons would be undertaken to fill
matrices for each criteria comparing the performance of each alternative within that
criteria.
 AHP

Consistency across pairwise comparisons

The consistency of the decision maker across a number of pairwise comparisons is a
significant complexity. Consider the very simple comparison of three criteria: A, B and C. If
the decision maker judges A to be more preferable than B, and A to be less preferable
than C then the decision maker must not judge B to be more preferable than C.

In a group that contains a large number of pairwise comparisons or where the difference
is between moderate and very strong preference it can be seen that lack of consistency is
a largely inevitable consequence of complex decision processes within AHP.

The AHP method attempts to address the issue of consistency by implementing a

consistency index that is a function of opposing comparisons. Above a threshold a lack of
consistency is highlighted and no analysis results are presented. An unfortunate
consequence is that decision makers begin to fulfil pairwise comparisons not on their
actual judgements but rather in order to maintain acceptable consistency.

An effective approach to limit the issue of consistency is to utilise a multi-tier hierarchy

thereby reducing the number of pairwise comparisons undertaken within each group.
 AHP

Rank reversal
If the inclusion or exclusion of a non-outperforming alternative, or duplicate alternative
alters the ranking of the remaining alternatives a rank reversal occurs.

AHP method and other MCDA methods are susceptible to rank reversal and experienced
users must be aware of this.
 AHP

Recording the decision making process and decision makers subjectivity

It is useful to have a record of decision making process. This gives some idea of how the
decision was reached. The problem hierarchy gives insight into how the decision was
structured.

Most AHP tools allow to view the pairwise comparison matrices showing the preference
values applied to each pair. However, this does not make explicit the subjectivity inherent
in the judgements made by the decision maker; the reasoning and understanding behind
those simple judgements is lost.

For complex problems where a large number of pairwise comparisons have been
undertaken it is unlikely that enough subjective detail could be recorded at the time of
making the decision to suggest the information available could be considered transparent
or suitable for any form of audit. Similarly it is unlikely that returning to this information in
the future would enable the reasoning behind any mistakes to be observed.
 AHP

Analytic Hierarchy Process (AHP) proposed by [Saaty, 1980] is very popular and
has been applied in wide variety of areas including planning, selecting a best
alternative, resource allocation and resolving conflict.

AHP Applied in
•Engineering
•Management
•Social sciences
•Strategic decisions like facility location
•Merger and acquisition
•Selection of projects in research and development
•Information technology outsourcing decisions
•Operational decisions like software selection
•Supplier selection
•Maintenance
•Logistics
•Engineering education
 AHP

Various steps in AHP Process

1.Develop a model for the business
2.Derive priorities (weights) for the Criteria
3.Consistency check (Weights assigned correct or not ?)
4.Derive overall priorities (Model Synthesis) and Final decision
AHP
AHP
AHP
AHP
AHP
• Multi weighted scoring models;
• A weighted scoring model typically uses several weighted
selection criteria to evaluate project proposals. Weighted
scoring models will generally include qualitative and/or
quantitative criteria. Each selection criterion is assigned a
weight. Scores are assigned to each criterion for the project,
based on its importance to the project being evaluated. The
weights and scores are multiplied to get a total weighted score
for the project. Using these multiple screening criteria,
projects can then be compared using the weighted score.
Projects with higher weighted scores are considered better.
Project screening matrix