The construction

of composite indicators
Dr.ssa Mariateresa Ciommi
Dipartimento di Scienze Economiche e Sociali - UNIVPM
Main
references
Part 1:
Introduction
Definition:
A composite indicator is formed when individual indicators are
compiled into a single index, on the basis of an underlying model of the
multi-dimensional concept that is being measured.
Source: OECD, 2004, “The OECD-JRC Handbook on Practices for Developing Composite Indicators”, paper presented
at the OECD Committee on Statistics, 7-8 June 2004, OECD, Paris.

Definition
Definition:
A composite indicator is formed when individual indicators are
compiled into a single index, on the basis of an underlying model of the
multi-dimensional concept that is being measured.
Source: OECD, 2004, “The OECD-JRC Handbook on Practices for Developing Composite Indicators”, paper presented
at the OECD Committee on Statistics, 7-8 June 2004, OECD, Paris.

Definition
Definition:
A composite indicator is formed when individual indicators are
compiled into a single index, on the basis of an underlying model of the
multi-dimensional concept that is being measured.
Source: OECD, 2004, “The OECD-JRC Handbook on Practices for Developing Composite Indicators”, paper presented
at the OECD Committee on Statistics, 7-8 June 2004, OECD, Paris.

Definition
 CI are mathematical combinations (or aggregations)
of a set of indicators.

#
–𝑥!" : value of the sub-indicator j for
country i at time t, with i=1,…,n and
Definition j=1,…,k
(More # #
–𝐼!" : normalised value of 𝑥!"
technical)
–𝑤!" : weight associated to sub-indicator j,
j=1,…,k and ∑%"$! 𝑤!" = 1
–𝐶𝐼!# : value of the composite indicator for
country i at time t.
 – Composite indicators provide a single measure for a
not directly observable multidimensional
phenomenon, based on indicators or variables.
Before – They are very useful because they can effectively rank
starting: countries, and regions, providing information to both
policymakers and the public since the results are easily
Variable, understood.
Indicator and – However, the idea of summarizing complex
Index phenomena into single numbers is not straightforward.
It involves both theoretical and methodological
assumptions that need to be assessed carefully to
avoid producing results of dubious analytic rigour.
Before – The construction of a composite indicator can be seen
starting: as an obstacle course, from the availability of data to
the choice of the individual indicators to their
Variable, treatment in order to compare and aggregate them.
Indicator and – Therefore, criticisms could grow along with each step
Index of the proces
 – The term “variable” derives from the late Latin
variabilis, meaning “that varies, which tends to vary”.
– In statistics, we mean a characteristic found in one or
Before more statistical units belonging to a population or a
sample as a result of a direct survey.
starting: – Nowadays this definition seems almost reductive since
Variable, we live in the era of the deluge in which we are
Indicator and surrounded by big data or administrative sources used
for statistical purposes and, therefore, not only by
Index direct surveys.
– In more general terms, the variable is defined as a
characteristic associated with a statistical unit such
as income, age or beds in a hospital.
 – The term “indicator” comes from the late Latin
indicator, meaning “who indicates” or “element that
indicates or signals something”.
– In the economic field, we indicate some
Before macroeconomic quantities (employment rate, inflation
starting: rate, GDP per capita, etc.) that are considered
significant for the purpose of evaluating the
Variable, performance of the economy in a given country and in
Indicator and a given time.
– While any normalized variable can be called an “indicator”,
Index we usually mean that the indicator is a ratio and that,
therefore, it is composed of a numerator, which can be
considered the variable that provides the meaning, and a
denominator, which is the variable that allows its
comparability in space and/or over time.
 – Let’s take the example of wanting to compare the
health-care infrastructure of two European countries,
Germany and the Netherlands:
– if we considered the variable “number of hospital beds” it
Before would make no sense to make the comparison because,
obviously, Germany will have more sleeping places than
starting: Holland.
– If, on the other hand, we calculate the indicator “number of
Variable, beds per 1,000 inhabitants”, we obtain a normalized value
according to the population variable and then we can make a
Indicator and comparison between the two countries.

Index – In other words, the presence of a reference

denominator (the population) makes it possible to
transform a variable (the number of beds) into a
relative measure and therefore it becomes comparable
in time and over space.
 – The term “index” derives from the Latin index and
means anything that serves to indicate.
– By index we mean the synthetic expression of the
Before components of a given phenomenon in time and/or
starting: space, or the relationship between multiple
phenomena considered at the same time.
Variable,
– We could therefore define an index as a complex
Indicator and structure in which the components are linked together
Index by mathematical operations.
– For example, the Gini index is a measure of the
inequality in the distribution of a transferable character
(e.g., income) among the N units of a population.
General
scheme
PROS
Can summarise complex, multi-dimensional realities with a
view to supporting decisionmakers.
Are easier to interpret than a battery of many separate
indicators
Can assess progress of countries over time.
Reduce the visible size of a set of indicators without
dropping the underlying information base
Thus make it possible to include more information within
the existing size limit.
Place issues of country performance and progress at the
centre of the policy arena.
Facilitate communication with general public (i.e. citizens,
media, etc.) and promote accountability.
Help to construct/underpin narratives for lay and literate
audiences.
Enable users to compare complex dimensions effectively.
CONS
May send misleading policy messages if poorly constructed
or misinterpreted.
May invite simplistic policy conclusions.
May be misused, e.g. to support a desired policy, if the
construction process is not transparent and/or lacks sound
statistical or conceptual principles.
The selection of indicators and weights could be the
subject of political dispute.
May disguise serious failings in some dimensions and
increase the difficulty of identifying proper remedial action,
if the construction process is not transparent.
May lead to inappropriate policies if dimensions of
performance that are difficult to measure are ignored.
Part 2:
10 steps for the
construction of a CI
 1.Theoretical framework
2.Data selection
3.Imputation of missing data
4.Multivariate Analysis
10 steps for the 5.Normalisation of individual indicators
construction of 6.Weighting and aggregation
a CI 7.Uncertainty and sensitivity analysis
8.Back to the data
9.Links to other indicators
10.Visualisation of the results
Table 1.
Checklist for
building a
composite
indicator
Table 1.
Checklist for
building a
composite
indicator
Table 1.
Checklist for
building a
composite
indicator
Table 1.
Checklist for
building a
composite
indicator
 • Define your aim:
• indicators and sub-indicators;
• Input/output variables
• Check for existing indicators
• Ask to experts
Step 1.
Theoretical
framework
 A composite indicator is above all the sum of its parts

• Selection of the indicators

• Relevance
• Validity (official statistics? Private Stat. Offices?
Survey?)
• Availability (both time and space)
• …
Step 2. • Ridundancy (via correlation) VS loss of information
Data selection (via esplained variance)
• Moreover, be carfully with polarity!
 The idea of imputation could be both seductive and dangerous

• Reduce/erease missing values

• Take note of all the passages!
• Check for «outliers»

Step 3.
Imputation of
missing data
 – Hot deck imputation. Filling in blanks cells with
individual data, drawn from “similar” responding units.
– For example, missing values for individual income may be
replaced with the income of another respondent with similar
Imputation: characteristics, e.g.age, sex, race, place of residence, family
relationships, job, etc.
Implicit
– Substitution. Replacing non-responding units with
modelling. unselected units in the sample.
– For example, if a household cannot be contacted, then a
previously non-selected household in the same housing block is
selected.
– Cold deck imputation. Replacing the missing value with
a value from an external source,
– e.g. from a previous realisation of the same survey.
 – Unconditional mean/median/mode imputation. The sample mean (median,
mode) of the recorded values for the given individual indicator replaces the
missing values.
– Regression imputation. Missing values are substituted by the
predicted values obtained from regression.
– The dependent variable of the regression is the individual indicator
hosting the missing value, and the regressor(s) is (are) the individual
indicator(s), showing a strong relationship with the dependent variable,
Imputation: i.e. usually a high degree of correlation.

Explicit – Expectation Maximisation (EM) imputation. This model

focuses on the interdependence between model parameters and
modelling the missing values. The missing values are substituted by
estimates obtained through an iterative process.
– First, the missing values are predicted based on initial estimates of the
model parameter values.
– These predictions are then used to update the parameter values, and the
process is repeated.
– The sequence of parameters converges to maximum-likelihood estimates,
and the time to convergence depends on the proportion of missing data
and the flatness of the likelihood function.
 Analysing the underlying structure of the data is still an art

Step 4.
Multivariate
Analysis
 Analysing the underlying structure of the data is still an art

Step 4.
Multivariate
Analysis
 Avoid adding up apples and oranges

– Normalisation is required prior to any data

aggregation as the indicators in a data set often have
different measurement units.

– Put all the indicators with a positive polarity.

Step 5. – The composite index should increase if an indicator increases
– Change the polarity before/during the normalization step
Normalization
 Avoid adding up apples and oranges

1. Ranking is the simplest normalisation technique.

This method is not affected by outliers and allows
the performance of countries to be followed over
time in terms of relative positions (rankings).
2. Standardisation (or z-scores) converts indicators to
a common scale with a mean of zero and standard
deviation of one.
Step 5. 3. Min-Max normalises indicators to have an identical
Normalization range [0, 1] by subtracting the minimum value and
dividing by the range of the indicator values
4. Distance to a reference measures the relative
position of a given indicator vis-à-vis a reference
point. This could be a target to be reached in a given
time frame
 Avoid adding up apples and oranges

5. Categorical scale assigns a score for each indicator. Categories can be

numerical, such as one, two or three stars, or qualitative, such as ‘fully
achieved’, ‘partly achieved’ or ‘not achieved’. Often, the scores are
based on the percentiles of the distribution of the indicator across
countries.
6. Indicators above or below the mean are transformed such that values
around the mean receive 0, whereas those above/below a certain
threshold receive 1 and -1 respectively.
Step 5. 7. Methods for cyclical indicators. The results of business tendency
Normalization surveys are usually combined into composite indicators to reduce the
risk of false signals, and to better forecast cycles in economic activities
(Nilsson, 2000).
8. The latter is a special case of balance of opinions, in which managers
of firms from different sectors and of varying sizes are asked to express
their opinion on their firm’s performance.
9. Percentage of annual differences over consecutive years represents
the percentage growth with respect to the previous year instead of the
absolute level.
 Avoid adding up apples and oranges

Step 5.
Normalization
“by equations”
 Avoid adding up apples and oranges

Step 5.
Normalization
“by equations”
Step 5.
Normalization
“by equations”
Step 5.
Normalization
“by equations”
Step 5.
Normalization
“by equations”
Step 5.
Normalization
“by equations”
Step 5.
Normalization
“by equations”
Comparison
between
normalization
methods
Comparison
between
normalization
methods
Comparison
between
normalization
methods
 – The polarity of an indicator is the sign of the
relationship between the indicator and the
phenomenon to be measured.
– For example, in the calculation of a development index,
GDP has positive polarity (+), while the infant mortality
rate has negative (-).
– Similarly, in the calculation of a poverty index, GDP
negative (-), while the infant mortality rate has positive
Polarity polarity (+).
– In order to aggregate correctly a set of indicators,
it is necessary that all have positive polarity.
– The main methods of reversing the polarity is
divided into:
– Linear transformation
– Non linear transformation
 – It's the easiest way to reverse the polarity of an
indicator and it is based on subtracting from the
maximum value the value of each unit:
#
Methods to 𝑥!" = max{𝑥!" } − 𝑥!"
!

invert polarity: max{𝑥!" } is the maximum value of the j indicator.

!
Linear – This operation does not alter the distances
transformation between the different statistical units.
– It is used primarily in the normalizations:
– Ranking
– Indices based on the range (min-max)
– Z-score
 – It consists in calculating the reciprocal of the value
of each unit :
# 1
Methods to 𝑥!" =
𝑥!"
invert polarity:
– This operation allows you to avoid the presence of
Non Linear null values in the distribution of the indicator
transformation normalized, but it alters the distances between the
different statistical units.
– It can be used in the normalization:
– index numbers
– percentage values
 – Making variables comparable (e.g. dividing by
population /income/ populated land area)
– Missing data imputation (e.g. data deletion, mean
substitution, regression, multiple imputation, nearest
Preliminary neighbour, ignore).
data treatment – Logarithms of highly skewed variables (e.g. skewness
measure greater than 5).
– Truncating distributions (e.g. to account for inaccuracy
of data at the extremes, to avoid extreme cases
becoming benchmarks for entire population.
 The relative importance of the indicators is a source of contention

THE CHOICE OF THE WEIGHTS

The weighting system is based on two possible
approachs:
– Subjective approach. The weights are assigned
Step 6. on the basis of a arbitrary judgment by experts (eg.
Human Development Index). In this case, the
Weighting and system of weights does not depend on the
Aggregation observed values;
– Objective approach. The weights are calculated
mathematically through the application of particular
statistical methods (eg. Principal Components
Analysis). In this case, the system of the weights
depends on the observed values.
 The relative importance of the indicators is a source of contention

WEIGHTING METHODS
1. Weights based on principal components analysis or factor
analysis
2. Data envelopment analysis (DEA)
– DEA employs linear frontier that would be used as a
benchmark to measure the relative performance of a
Step 6. country. This requires construction of a benchmark (the
frontier) and the measurement of the distance between
Weighting and countries in a multi-dimensional framework.
Aggregation 3. Benefit of the doubt approach (BOD)
– The application of DEA to the field of composite
indicators is known as the “benefit of doubt approach“
(BOD) and was originally proposed to evaluate
macroenomic performance. In the BOD approach, CI is
defined as the ratio of a country’s actual performance to
its benchmark performance:
 The relative importance of the indicators is a source of contention

– Most composite indicators rely on equal weighting

(EW), i.e. all variables are given the same weight. This
essentially implies that all variables are “worth” the
same in the composite
– Moreover, if variables are grouped into dimensions and those
are further aggregated into the composite, then applying
Step 6. equal weighting to the variables may imply an unequal
Weighting and weighting of the dimension (the dimensions grouping the
larger number of variables will have higher weight).
Aggregation – Weights may also be chosen to reflect the statistical
quality of the data. Higher weights could be assigned
to statistically reliable data with broad coverage.
– However, this method could be biased towards the readily
available indicators, penalising the information that is
statistically more problematic to identify and measure.
 The relative importance of the indicators is a source of contention

– The existing literature offers a quite rich menu of

alternative weighting methods all having pros and
cons.
– Statistical models such as principal components
analysis (PCA) or factor analysis (FA) could be used to
Step 6. group individual indicators according to their degree of
correlation.
Weighting and – Weights, however, cannot be estimated with these
Aggregation methods if no correlation exists between indicators.
– Other statistical methods, such as the “benefit of the
doubt” (BOD) approach, are extremely parsimonious
about weighting assumptions as they allow the data to
decide on the weights and are sensitive to national
priorities
 The relative importance of the indicators is a source of contention

– Aggregation methods also vary.

– While the linear aggregation method is useful when all
individual indicators have the same measurement unit,
provided that some mathematical properties are
respected.
– Geometric aggregations are better suited if the
Step 6. modeller wants some degree of non compensability
between individual indicators or dimensions.
Weighting and – Furthermore, linear aggregations reward base-
indicators proportionally to the weights, while
Aggregation geometric aggregations reward those countries with
higher scores
– In both linear and geometric aggregations, weights
express trade-offs between indicators. A deficit in one
dimension can thus be offset (compensated or partially
compenseted) by a surplus in another.
 The relative importance of the indicators is a source of contention

– To ensure that weights remain a measure of

importance, other aggregation methods should be
used, in particular methods that do not allow
compensability;
– With regard to the time element, keeping weights
unchanged across time might be justified if the
Step 6. researcher is willing to analyse the evolution of a
Weighting and certain number of variables;
Aggregation – The absence of an “objective” way to determine
weights and aggregation methods does not necessarily
lead to rejection of the validity of composite indicators,
as long as the entire process is transparent.
– The modeller’s objectives must be clearly stated at
the outset, and the chosen model must be tested
to see to what extent it fulfils the modeller’s goal.
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods:
Example

In general, we have: M0 £ M1 £ M2
 The relative importance of the indicators is a source of contention

!
– Power means (of r ordine): 𝑀!# = (∑& #
"$% 𝑦!" 𝑤" )
" where 𝑤" is the weight
(0 < 𝑤" < 1) e ∑&
"$% 𝑤" = 1

Some – Wroclaw tassonomic method: 𝑊! =

'
( # )*+#
where 𝑀' is the average, 𝑆' the
Aggregation standard deviation, 𝑑! = ∑& *
"$%(𝑦!" −𝑦,-." ) and 𝑦!" represents the
Methods standardized value and 𝑦,-." is a referent point
)/0
– Mazziotta-Pareto Index: 𝑀𝑃𝐼! = 𝑀1$ ± 𝑆1$ 𝐶𝑉1$
 The relative importance of the indicators is a source of contention

– The method is based on the hypotesis that the

components are not substitutable.
– This involves the introduction of a 'penalty' for the
units that do not present the balanced values of the
Some indicators.
Aggregation – Requirement:
Methods: – Independence from the variability of the individual
indicators and from the unit of measure;
The Mazziotta- – Independence from an ideal unit, since the definition of a
set of values «objective» is subjective, is not unique and
Pareto Method it can vary over time;

– PROS:
– Semplicity of calculation;
– Easy interpretation.
 The relative importance of the indicators is a source of contention

A) Normalization
– Each individual indicators is transformed in a z-score with
mean equal to 100 and standard error equal to 10 (mean=100
Some and s.e.=10); the normalized values will be in the range 70-
130.
Aggregation
– This procedure allows to depurate the indicators both from
Methods: the unit of measure and from the variability and it does not
require the deifnition of a target values (ideal unit), because it
The Mazziotta- replaces the vector with the of average values.
Pareto Method – In this way, it is easy to identify the units with a level of the
phenomenon above average (values greater than 100) and
the units with a level below the average (values less than
100).
Some
Aggregation
Methods:
The Mazziotta-
Pareto Method
 The relative importance of the indicators is a source of contention

B) Penalization
– The aggregation function (arithmetic mean of standardized
Some values) is 'corrected' by a penalty coefficient that depends, for
Aggregation each unit, on the variability of the indicators compared to the
average value ('horizontal variability')
Methods: – This variability, measured by the coefficient of variation (CV),
allows to penalize the score of the units that, with the same
The Mazziotta- arithmetic average, have a higher imbalance between the
Pareto Method values of the indicators.
– The penalty can be added or subtracted, depending on the
type of the studied phenomenon (poverty, development, etc.).
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods:
The Mazziotta-
Pareto Method
 The relative importance of the indicators is a source of contention

Some
Aggregation
Methods:
The Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Mazziotta-
Pareto Method
 • It is variant of MPI, based on a transformation Min-Max instead
in waste z-score
• The transformation Min-Max is based on two goalposts: a
minimum and a maximum that represent the possible range of
Some variation of each indicator throughout the period considered and
all units
Aggregation
Methods: AMPI vs MPI

The Adjusted - MPI the value 100 represents the mean of the values of all units
- AMPI the value 100 represents the mean of the goalposts.
Mazziotta- You can 'fix' the goalposts in order to put equal to a 100 = reference
Pareto Method value (eg., the national average in a given year).

- MPI the normalized indicators have equal variances

- AMPI the normalized indicators have similar variances
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
 PROS AND CONS
– It is applicable to any phenomenon, changing the
sign of the penalty.
Some – The index can be decomposed into two parts: the
Aggregation mean effect (compensatory) and the penalty effect
(imbalance).
Methods:
– In order to calculate the composite index of a
The Adjusted statistical unit, it is not necessary to know the
Mazziotta- values of the other.
Pareto Method – It is possible to make absolute comparisons
between units over time. It does not completely
purifies indicators from the variability effect.
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
Some
Aggregation
Methods:
The Adjusted
Mazziotta-
Pareto Method
 The relative importance of the indicators is a source of contention

Step 6.
Weighting and
Aggregation
 Sensitivity analysis can be used to assess the robustness of composite indicators

1. Inclusion and exclusion of individual indicators.

2. Modelling data error based on the available information on variance
estimation.
Step 7. 3. Using alternative editing schemes, e.g. single or multiple imputation.
Uncertainty 4. Using alternative data normalisation schemes, such as Mni-Max,
standardisation, use of rankings.
and Sensibility 5. Using different weighting schemes, e.g. methods from the
analysis: participatory family (budget allocation, analytic hierarchy process) and
endogenous weighting (benefit of the doubt) and different plausible
Robustness values for the weights
6. Using different aggregation systems, e.g. linear, geometric mean of
un-scaled variables, and multi-criteria ordering.
 Sensitivity analysis can be used to assess the robustness of composite indicators

Step 7.
Uncertainty
and Sensibility
analysis:
 De-constructing composite indicators can help extend the analysis

Step 8.
Back to the
data

Also check for subgroups!

 Composite indicators can be linked to other variables and measures

Step 9.
Link to
other
indicators
 A well-designed graph can speak louder than words

Step 10.
Visualization
of the results
 The European
Statistics Code of
Practice
(Principles 11-15)
focuses on
Part 3: statistical outputs
as viewed by
more on CI users. Six quality
dimensions are
considered:
 1. Relevance
2. Accurancy (credibility, objectivity)
3. Timeliness
4. Accessibility” and Clarity”
Six quality 5. Comparability
dimensions: 6. Coherence
1. Relevance – In the context of composite indicators, relevance
refers to the has to be evaluated considering the overall
purpose of the indicator.
degree to – Careful evaluation and selection of basic data have
which statistics to be carried out to ensure that the right range of
meet current domains is covered in a balanced way.
and potential – Given the actual availability of data, ”proxy” series
are often used, but in this case some evidence of
needs of the their relationships with “target” series should be
users; produced whenever possible.
 In the context of composite indicators, accuracy of basic data is extremely
important.
2. Accuracy
refers to the – The credibility of data products refers to confidence that users place in
those products based simply on their image of the data producer, i.e.,
closeness of the brand image.

computations – One important aspect is trust in the objectivity of the data. This implies

or estimates to that the data are perceived to be produced professionally in accordance

the exact or with appropriate statistical standards and policies and that practices are
transparent (for example, data are not manipulated, nor their release
true values; timed in response to political pressure).
– Other things being equal, data produced by “official sources” (e.g. national
statistical offices or other public bodies working under national statistical
regulations or codes of conduct) should be preferred to other sources.
3. Timeliness – In the context of composite indicators, timeliness is especially
refers to the length of important to minimise the need for the estimation of missing data or
time between the for revisions of previously published data.
availability of the
information and the – As individual basic data sources establish their optimal trade-off
event or phenomenon between accuracy and timeliness, taking into account institutional,
it describes. organisational and resource constraints, data covering different
domains are often released at different points of time. Therefore
“Punctuality”
refers to the time lag special attention must be paid to the overall coherence of the
between the target vintages of data used to build composite indicators (see also
delivery date and the coherence).
actual date of the
release of the data;
 – In the context of composite indicators, accessibility of basic
4.“Accessibility” refers to the data can affect the overall cost of production and updating of
physical conditions in which
users can access statistics: the indicator over time.
distribution channels, ordering
procedures, time required for – It can also influence the credibility of the composite indicator
delivery, pricing policy,
marketing conditions (copyright, if poor accessibility of basic data makes it difficult for third
etc.), availability of micro or
macro data, media (paper, CD-
parties to replicate the results of the composite indicators.
ROM, Internet, etc). – In this respect, given improvements in electronic access to
“Clarity” refers to the statistics’
information environment: databases released by various sources, the issue of coherence
appropriate metadata provided across data sets can become relevant.
with the statistics (textual
information, explanations,
documentation, etc); graphs, – Therefore, the selection of the source should not always give
maps, and other illustrations;
availability of information on preference to the most accessible source, but should also
the statistics’ quality (possible take other quality dimensions into account.
limitation in use);
 – In the context of composite indicators, the wide range of data used
5. Comparability to build them and the difficulties due to the aggregation procedure
refers to the require the full interpretability of basic data.
measurement of the
impact of differences in – The availability of definitions and classifications used to produce
applied statistical basic data is essential to assess the comparability of data over time
concepts and
and across countries (see coherence):
measurement tools and
procedures when – for example, series breaks need to be assessed when composite
statistics are compared indicators are built to compare performances over time.
between geographical
areas, non-geographical – Therefore the availability of adequate metadata is an important
domains or over time; element in the assessment of the overall quality of basic data.
 In the context of composite indicators, two aspects of
6. Coherence coherence are especially important: coherence over
refers to the time and across countries.
adequacy of the – Coherence over time implies that the data are based
data to be on common concepts, definitions and methodology
reliably over time, or that any differences are explained and
can be allowed for. Incoherence over time refers to
combined in breaks in a series resulting from changes in concepts,
different ways definitions, or methodology.
and for various – Coherence across countries implies that from country
uses. to country the data are based on common concepts,
definitions, classifications and methodology, or that
any differences are explained and can be allowed for.
The main
characteristics
of the
measurement
scales.
Part 4:
Example of CI
The BES Project & SDG
 – Environmental Sustainability Index (WEF)
– Air Quality Index (WEF)
– Environment Index (World Travel and Tourism Council)
– Environmental Performance Index (WEF, Yale &
Columbia Universities)
Environment – Living Planet Index (UNEP & WCMC)
– National Biodiversity Index (Secretariat of the
Convention on Biological Diversity)
– Natural Capital Index (RIVM, The Netherlands)
 – Human Development Index (United Nations)
– Health System Achievement Index (WHO)
– Corruption Perceptions Index (Transparency
International)
– World Income Inequality Database:Gini Index (United
Society Nations)
– Wellbeing Index (Prescott-Allen)
– Genuine Progress Indicator (Redefining Progress)
 – Economic Sentiment Indicator (EC)
– Composite Leading Indicators (OECD)
– Internal Market Index (EC)
– Doing Business Indicators (World Bank)
– Index of Economic Freedom (Heritage Foundation)
Economy
– Economic Competitiveness Index (Institute for
Management Development)
– Human Tourism Index (World Travel and Tourism
Council)
 – Summary Innovation Index (EC)
– Innovative Capacity Index (Porter and Stern)
– Investment/Performance in the knowledge based
Innovation economy (EC)
Technology – Technology Achievement Index (United Nations)
Information – The Networked Readiness Index (Harvard University -
Centre for International Development)
– E-Government Rankings (World Markets Research
Centre)
 – Globalization Index (Foreign Policy Magazine)
– World Competitiveness Index (IMD)
– Growth Competitiveness Index (WEF)
Globalization – Current Competitiveness Index (WEF)
– The Globalisation Index (G-Index) (World Markets Research
Centre)
Beyond GDP
 “The gross national product includes air pollution
and advertising for cigarettes and ambulances to
clear our highways of carnage. It counts special
locks for our doors and jails for the people who
break them. GNP includes the destruction of the
redwoods and the death of Lake Superior. It grows
with the production of napalm, and missiles and
nuclear warheads... it does not allow for the health
Beyond GDP of our families, the quality of their education, or the
joy of their play. It is indifferent to the decency of our
factories and the safety of our streets alike. It does
not include the beauty of our poetry or the strength
of our marriages, or the intelligence of our public
debate or the integrity of our public officials. It
measures everything, in short, except that which
makes life worthwhile.”
Corrective to GDP Initiative aiming at meauring the well-being
– Measure of Sustainable Welfare - MEW Nordhaus e Tobin, 1971
2004 - OECD World Forum on Key Indicators.
– Index of Sustainable Economic Well-being – ISEW Daly e Cobb, 1991
Statistics, Knowledge and Policy (Palermo).
2006 - Istanbul Declaretion+ Global Project on
Synthetic indices and rankings
measuring the progress of societies.
– Relative National Standards of Living Bennett, 1937
2007 - WWF, Club of Rome, EP – Beyond GDP
– Level of Living Index UNRISD, 1966
– Human Development Index UNDP, 1991 2008 - Sen-Stiglitz-Fitoussi Commission
– Indice di Qualità della Vita delle Provincie Sole 24 Ore, 1993 2009 - European Commission – “GDP and
– Indice di Qualità Regionale dello Sviluppo - QUARS Sbilanciamoci!, 2006 Beyond: Measuring progress in a changing world”.
– Happy Planet Index NEF, 2006 2011 - OECD Better Life Initiative. “How’s Life”
– Better Life Index OECD, 2011 2011 - Eurostat –Sponsorship Group on
Measuring Progress, Well-being and Sustainable
Set of indicators Development
– Report of the President's Research Committee on Social Trends USA, 1933
– Millennium Development Goals ONU, 2000
– Sustainable Development Indicators EC “What we measure affects
– National Accounts of Wellbeing NEF, 2009 what we do”

Subjective well-being Stiglitz-Sen-Fitoussi

– Life satisfaction /Happiness Layard, 2000s
 – 25 September 2015: UN
General Assembly
adopted the 2030
Agenda for Sustainable
Development, with 17
goals and 169 targets
up to 2030
Sustainable – SDGs apply to all
Development countries, but with
differing responsibilities
Goals
– March 2016: UN
Statistical Commission
agrees 230 global
indicators “as a practical
starting point”
– Report: https://unstats.un.org/sdgs/report/2022/
99
– Data: https://unstats.un.org/sdgs/dataportal
Sustainable Development Goals
The Italian National Council for Economics and Labour (CNEL) and
the Italian National Institute of Statistics (ISTAT) present the second
edition of the “Report on Equitable and Sustainable Wellbeing” (BES
2014) analysing the fundamental dimensions of wellbeing and
progress in Italy and its territories.
With its continued work BES seeks to become a reference point for
citizens, for civil society, media and politicians, providing an overall
view of the main social, economic and environmental phenomena
which characterize our country.
 The individual sphere

The 12
dimensions of
well-being
The context
 Benessere Equo e Sostenibile
Equitable and Sustainable Well-being

For Italy:
BES
Multidimensional aspects of
quality of life, including health Guarantee for next
generations

Focus on distribution of wellbeing: including

inequalities
BES: – Bes for Italy, Macroarea and Regions
for Italy, – https://www.istat.it/it/archivio/269316
macroarea, – Bes at local level (Provinces):
Regions, – https://www.istat.it/en/well-being-and-
Provinces, sustainability/the-measurement-of-well-being/bes-
at-local-level
Municipalities
 – The Law no. 163/2016, which reformed the Italian Budget
law, provided that the Equitable and sustainable well-being
indicators should contribute to define those economic
Bes In The policies which largely affect some fundamental dimensions
Economic And for the quality of life.
– On the 8st of April 2022 Istat provided the 2021 updates of
Financial the complete set of the 12 equitable and sustainable well-
Document being indicators, mainly based on final data but also, when
it was necessary to ensure timeliness, based on provisional
(DEF) data and ad hoc estimate models. .
– See: https://www.istat.it/en/well-being-and-
sustainability/the-measurement-of-well-being/bes-in-the-
economic-and-financial-document
Part 5:
open issues in CI
The normalization
problem
 – Mazziotta, M., & Pareto, A. (2021). Everything you always
wanted to know about normalization (but were afraid to
ask). Rivista Italiana di Economia Demografia e
Statistica, 75(1).
References – Mazziotta, M., & Pareto, A. (2021). Data Normalization For
Aggregating Time Series: The Constrained Min-Max
Method1. Rivista Italiana di Economia Demografia e
Statistica, 75(4).
 the distributions of
indicators
transformed into z-
Comparing scores are ‘centered’
normalization around the origin
method (mean=0) and
‘elongated’ or
‘shortened’ to have
the same variability
(variance=1).
 Re-scaling also makes the variances
more homogeneous (but not equal),
bringing all the values into a common
interval (Figure 2.b). However, the
distributions of indicators are not
‘centered’ and this leads to the loss of
a common reference value, such as the
mean. It follows that equal normalized
values (i.e. balanced normalized
Comparing values) can correspond to very
unbalanced original values. For
normalization example, the normalized value 0.2 for
the Exp indicator corresponds to a
method high original value; whereas for the
Nor and Bet indicators it corresponds
to a very low original value.
Therefore, the use of a simple re-
scaling for aggregating individual
indicators with an unbalance
adjustment method, such as the
geometric mean, can lead to biased
results.
 Indicization with the mean
as a base set to 100 ‘centers’
all distributions around the
mean, but do not ‘normalize’
their variability (e.g., the
Comparing range of the Bet indicator is
normalization very short, whereas the
range of the Exp indicator is
method very large). So, a significant
implicit weighting is
introduced when different
indicators are compared and
aggregated.
 Re-scaling with a
reference is very
similar to
standardization when
the mean is chosen as
a reference. In fact, it
‘centers’ indicators
(like indicization) and
Comparing ‘normalizes’ them
(like re-scaling).
normalization Nevertheless, it
method allows to keep fixed
the goalposts and the
reference when
different periods have
to be compared,
contrary to
standardization
Comparing
normalization
method
The balance of
indicators
 – The components of a composite indicator are called
‘substitutable’ if a deficit in one component may be
compensated by a surplus in another
– Similarly, the components of a composite indicator are called
‘non-substitutable’ if a deficit in one component may not be
Substitutability, compensated by a surplus in another
– (e.g., in a development index, a low value of “Life expectancy at
non-substitutability birth” cannot be offset by a high value of “GDP per capita” and
vice versa).
and ‘balance’ of – Compensability is closely associated with the concept of
indicators unbalance, i.e., a disequilibrium among the individual indicators
that are used to build the composite indicator.
– In a non-compensatory or partially compensatory approach, all
the dimensions of the phenomenon must be balanced and an
aggregation function that takes unbalance into account, in
terms of penalization, is often used.
The aggregation
The – Weighting is one of the most important steps in building
aggregation composite indicators
step: – Its importance stems from the fact that, once the
weighting is done, it is likely to affect the scores of the
horizontal and composite indicators and, consequently, the rankings of
vertical the countries (OECD, 2008)

penalization
A possible
solution:
vertical
weights
 – In this paper, we have introduced a new class of composite indicators for
measuring the well-being at the local level, which takes into account the
variability between and within the local units.
A new class of – This class has two extreme special cases: on the one hand, the Adjusted
Mazziotta Pareto Index (AMPI), which takes into account the unbalanced
CI for well- distribution among the indicators belonging to the same well-being
domain, and, on the other hand, the Gini based weighted average (GW
being and RGW), which depends on the distribution of each indicator across the
local units. Combinations of them are also possible (GAMPI and RGAMPI).
 M1 - (EW)
1a) AMPI

2a) M1 w= Gini (GW) 3a) M1 w= 1/Gini (rGW)

The weights depend on a

coefficient of vertical variability:
a more homogeneous
distribution of the indicator
implies a greater weight
 3a) AMPI w=Gini (GAMPI) 3b) AMPI w=1/Gini (RGAMPI)

The average effect The penalty effect

corrected for the vertical connected to the
variability of the horizontal variability
elementary indicators of the elementary
indicators in each unit

'2
1 )
𝐼!$ = ' · 5 𝑟!"$ · 𝐺."$ − 𝛽 · 𝑆*342 · 𝑐𝑣*342
∑"%&
2
(𝐺 ."$ ))
"%&

A class of CI
 Method α 𝛃 Equation
6%
1
The equally-weighted average 0 0 𝐸𝑊!5 = < 𝑟!"5 >
𝑛5
"$%

The adjusted Mazziotta-Pareto Index 0 1 𝐴𝑀𝑃𝐼!5 = 𝐸𝑊!5 − 𝑆#$% 𝑐𝑣#$%

The Gini - based adjusted Mazziotta- 𝐺𝐴𝑀𝑃𝐼!5 = 𝐺𝑊!5 − 𝑆#$% 𝑐𝑣#$%
1 1
Pareto Index
The Reciprocal Gini - based adjusted 𝑅𝐺𝐴𝑀𝑃𝐼!5 = 𝑅𝐺𝑊!5 − 𝑆#$% 𝑐𝑣#$%
-1 1
Mazziotta-Pareto Index
6%
The Gini - based weighted average 1
1 0 𝐺𝑊!5 = > < 𝑟!"5 > 𝐺."5
𝐺5
"$%
6%
The reciprocal Gini - based weighted 1
-1 0 𝑅𝐺𝑊!5 = 𝐺5 > < 𝑟!"5 >
average 𝐺."5
"$%
A
possible
solution:
horizontal
weighs
Mariani, F., Ciommi,
M., & Recchioni, M. C.
(2022). A new class of
composite indicators:
the penalized power
means. arXiv preprint
arXiv:2206.11216.
 – Ciommi, M., Gigliarano, C., Emili, A., Taralli, S., & Chelli, F. M.
(2017). A new class of composite indicators for measuring
well-being at the local level: An application to the Equitable
and Sustainable Well-being (BES) of the Italian
Provinces. Ecological indicators, 76, 281-296.
References – Mariani, F., & Ciommi, M. (2022). Aggregating Composite
@DISES Indicators through the Geometric Mean: A Penalization
Approach. Computation, 10(4), 64.
– Mariani, F., Ciommi, M., & Recchioni, M. C. (2022). A new
class of composite indicators: the penalized power
means. arXiv preprint arXiv:2206.11216.
Part 5:
working with R