Ecological Indicators 112 (2020) 106061

Contents lists available at ScienceDirect

Ecological Indicators
journal homepage: www.elsevier.com/locate/ecolind

Review

A critical review of studies related to construction and computation of T

Sustainable Development Indices
Swati Kwatraa,b, Archna Kumarb, Prateek Sharmaa,
⁎

a
Department of Energy and Environment, TERI School of Advanced Studies, Plot No. 10 Institutional Area, Vasant Kunj, New Delhi, India
b
Department of Development Communication and Extension, Lady Irwin College (University of Delhi), Sikandra Road, Mandi House, Delhi, India

ARTICLE INFO ABSTRACT

Keywords: Sustainable development is a multi-dimensional concept, which emphasizes integration and striking a dynamic
Sustainable Development balance between economic, social and environmental aspects in a region, to ensure inter-generational and intra-
Composite Index generational equity. One of the major tools to address sustainability concerns is to have indices, which can
Sustainable Development Index measure the performance of a region on various dimensions of sustainable development. The review suggests
Environmental Sustainability
there are various indices that are available at the global scale that compare different countries on different
aspects of sustainability; however, a limited number of studies have been reported at regional scale. Further, the
paper appraises about the various approaches and frameworks used to develop these Sustainable Development
Indices (SDIs). The merits and demerits of these approaches and frameworks have been discussed. The review
finds that top-down approaches have been generally employed for construction of SDIs. However, bottom-up
approaches which are rarely used, also need to be developed in order to construct robust, contextual and
consensus based SDIs. The review shows that composite SDIs are the most common form of framework used
across the globe. Most studies have been found to use 30–60 indicators for development of SDIs. The key steps to
be followed for construction of Composite Sustainable Development Indices (CSDIs) have been reviewed in
detail. The review finds variable processes and steps followed by different studies for development of SDIs. Based
on the review, research gaps have been identified and recommendations on steps to be followed in construction
of new CSDI have been provided. The review also reveals that most studies have focused on assessment of
current state of sustainable development, a limited number where the trends have been studies; no study related
to the future projections/predictions of the SDI has been reported in the literature.

1. Introduction
Although the term ‘Sustainable development’ was first used by IUCN
in 1980, it gained importance after the Brundtland Commission Report
– ‘Our Common Future’. The definition given by the commission is still
the most “common” used definition across the world. It defines sus-
tainable development as development that meets the need of the pre-
sent generation without compromising the needs of the future genera-
tion (World Commission on Environment and Development, 1987).
According to Gro Harlen Brundtland, sustainability can only be assured
when it emphasizes the:
1. alleviation of poverty and deprivation;
2. conservation and enhancement of the resources base, which alone
can ensure that the elimination of poverty is permanent;
3. broadening of the concept of development, so that it covers not only
economic growth but also social and cultural development; and
4. unification of economics and ecology in decision making at all scales
(Pearce et al., 1989).
The commission report gained importance and attracted the atten-
tion of policymakers as well as planners in both developed and devel-
oping nations. Environmental protection agencies have been setup or
reestablished in many countries and have legal powers to examine
developmental project/ proposals on different sustainability dimen-
sions before approving or disapproving them.
In 1992, the Earth Summit in Rio de Janeiro proved to be another
milestone in this direction, wherein a set of 27 principles were adopted
to guide countries in achieving sustainable development for both pre-
sent and future generations. It recognized the need and right of the
present generation to develop, but also emphasized its responsibility to
safeguard the common environment for assuring intra- and inter-

⁎
Corresponding author.
E-mail address: prateeks@terisas.ac.in (P. Sharma).

https://doi.org/10.1016/j.ecolind.2019.106061
Received 22 March 2019; Received in revised form 15 December 2019; Accepted 30 December 2019
1470-160X/ © 2020 Elsevier Ltd. All rights reserved.
S. Kwatra, et al.
generational equity. Since then, sustainable development as a subject
has gained immense popularity across the world (Mebratu, 1998; Kori
and Gondo, 2012).
Sustainable development is a cross-cutting subject as it has three
dimensions – social, economic and environment (Goodland and Daly,
1996). Social sustainability refers to poverty reduction; economic sus-
tainability refers to long-term sustainability of both renewable and non-
renewable resources so that they feed into the production system and
provide long-term economic benefits; and environmental sustainability
refers to preservation and maintenance of the life forms that exist on
earth (Goodland, 1995; Sutton, 2004; Kori and Gondo, 2012). Sus-
tainable development has become a subject of debate among en-
vironmentalists and economists (AtKisson, 2006; Ayres et al., 2001),
and broadly assessed in weak and strong terms. A weak definition of
sustainability is based on the economic value principle, neoclassical
capital theory and is human centric in nature. It considers the capital
value of the natural resources, but completely forgets to take into ac-
count their values in terms of the natural materials and services it
provide. Whereas, strong definitions of sustainability are based upon
biophysical principles and take into account those certain function, that
the environment performs for humans (Hediger, 2006; Nourry, 2008).
For an example, a weak sustainability model will support the idea of
estimating the value of forest land only in terms of the total number of
trees cut and the value it generates for making furniture or paper.
However, a strong sustainability model will estimate the financial
contributions of forest not only in terms of the economic value of the
trees but also its environmental and social values. These trees provide
food and shelter to many animals and humans, help in rainfall, provide
fresh air, and act as carbon sinks, all of which are not taken to any
account in a weak sustainability model. Thus, the weak sustainability
models are based on the philosophy that man-made capital is important
than natural capital, and natural capital can be substituted by manmade
capital. On the other hand, strong sustainability model provides im-
portance to natural/ human capital in comparison to man-made capital.
The strong sustainability models show clean inclination towards reg-
ulating the resource intake by use of advanced technologies, and pre-
serving natural capital in absolute form for the next generation (Pearce
et al., 1993; Neumayer, 2003; Roberts, 2004; Barr, 2008 as citied in
Davies, 2013). Hence, weak and strong sustainability concepts are
different from each other in terms of ethics and philosophical per-
spective.
Based upon the strong concept model of sustainability, the
Brundtland Commission report also urged nations across the world to
follow a path of development that is measured not only in terms of
economic growth, but also takes into account social and environmental
aspects, thus, all aspects must be considered and integrated (Pope et al.,
2004). Many national governments assured their commitments to make
their development sustainable by the early years of the new millen-
nium. As mentioned earlier, Agenda 21 at the Earth Summit in 1992,
the Johannesburg Plan of Implementation (2002), and the Commission
on Sustainable Development (CSD) at its 11th (2003) and 13th (2005)
sessions asserted the need to develop and report indicators that can
depict sustainable development progress made by countries. Inter-
changeably indicator and index has been used in the literature, author
wishes to clarify that in the present article, indicator has been used as a
single indicator whereas index has been used for composite index for-
mulated by the combination of indicators.
Indicators and Indices help provide status reports and suggest di-
rections to policy-makers to formulate policies and programs that
would help a region more towards sustainable development. This paper
focuses on reviewing various indices available to measure sustain-
ability. The purpose of the review paper is to review various SDIs
available and identify the most widely accepted methodology for de-
veloping a SDI based on at least three dimensions of sustainability –
social, economic and environment.
2
Ecological Indicators 112 (2020) 106061
2. Method
Inverted pyramid approach (Maier, 2013) has been used for writing
the present review paper. To meet the present objective of the paper, an
extensive literature review has been carried out of the SDIs at the global
and regional scale. For the purpose of review, various scientific reports
and articles in online journals which have been indexed in recognized
databases (e.g., ScienceDirect and Scopus) have been reviewed. To start
with global SDIs have been studied and thereafter, the studies at-
tempting to apply these indices at regional scale were also reviewed.
This has been undertaken to understand the applicability of global in-
dices at national or regional scale. Thereafter, a comprehensive review
of regional level SDIs has been carried out. Further, approaches (top-
down, bottom-up or mixed) and frameworks used to develop these SDIs
have also been reviewed. A specific review was carried out to collate
steps for constructing a composite SDI in order to present a standar-
dized approach for developing the index. For each step in construction
of SDI, several methodologies have been used in different studies. These
methodologies have been reviewed, and their advantages, limitations
along with application have been discussed along each step.
3. Sustainable development index (SDI) - tool for sustainability
assessment
Sustainability assessment is one of the most complex types of ap-
praisal activity, as it is multi-disciplinary in nature and based on cul-
tural and value-based elements (Sala et al., 2015). The major challenge
in sustainability assessment is to maintain balance between the scien-
tific basis and the policy basis of the index. Sustainability assessment is
an evaluation tool that provides results on the basis of integrated
nature-society systems. These tools can act as guides to the policy-
makers for determining future actions required for a sustainable so-
ciety. These evaluation tools can be used at global as well as local scale,
and results can be interpreted in short-term and long-term perspectives
in relation to sustainability (Ness et al., 2007, cited in Singh et al.,
2012). Parris and Kates (2003) stated four prime purposes of sustain-
ability assessment, namely, 1) decision-making and management; 2)
advocacy; 3) participation and consensus building; and 4) research and
analysis. Tools used for sustainability assessment must play three pri-
mary roles: integrate natural and social systems; capture and show
sustainability trends at a spatial or temporal scale; and assess sustain-
ability both in short-term (< 10 years) as well a long-term (> 10 years)
perspective (Ness et al., 2007). The process of sustainability assessment
must be clean, transparent, inclusive, based on broad participation,
continuous and adaptive so that it can feed into policy decision making
processes on a regular basis (Hardi and Zand, 1997 as cited in Ramos
and Caeiro, 2010). Measurement of sustainability of a system is es-
sential to ascertain the intra-generational and inter-generational equity
in resource allocation (Graymore et al., 2008). It is also useful for op-
timum and efficient utilization of natural resources along with socio-
economic development and enhancement of human well-being (UN,
2007). In order to bring the necessary objectivity to measuring and
understanding sustainable development, indicators, indices and models
act as useful tools. The indicators and indices used to assess sustain-
ability of a system are known as SDIs. SDIs have been used globally to
adequately assess the progress made by a region towards a sustainable
economy, society and environment. SDIs indicate whether progress
made, in the long-term and short-term, is positive, negative, or if there
is no significant progress at all (Macrory, 2015). SDIs have been used to
assess, quantify and evaluate performance of a unit that may be as large
as a country or as small as an industry on their sustainability. Without
indices and indicators, sustainable development policies lack a strong
justifiable foundation to move forward. The SDI assures equity in re-
source allocation at both intra-generational and inter-generational
scales (Graymore et al., 2008). The index must co-produce knowledge
and solutions for attaining sustainability in trans-disciplinary settings
S. Kwatra, et al.
(Sala et al., 2015). Since the Earth Summit in 1992, globally several
efforts have been made to develop SDIs that could gauge the progress
made by nation towards the path of sustainable development (UNDESA,
1992; Moffatt, 1996; Hanley et al., 1999). At present there are many
indices available to measure sustainability at various scales – global,
national, regional, and city; also sector-specific indices are being de-
veloped to measure and indicate the direction of development. Over
500 sustainability indices have been identified on a global scale de-
veloped by various governmental and non-governmental organizations
(Parris and Kates, 2003).
The next sections of the paper discuss the main criteria for for-
mulating an effective SDI, scales of application of SDIs, approaches and
frameworks used for their formulation. Since CSDI are the most widely
used framework to develop SDI across globe, each step carried out in
construction of the CSDI has been reviewed in detail. Finally, the re-
view of literature is provided for the chronological evolution of SDIs in
last few decades.
3.1. Criteria for effective SDIs
There are several important aspects which need to be considered
before formulating an SDI. The SDI must be independent of any over-
laps or correlations among variables used to measure sustainability. It
must be dynamic and sensitive towards temporal, spatial, and structural
changes in social, economic, and environmental dimensions of sus-
tainability. It should be relatively stable to capture long-term sustain-
ability rather than short-term. The index is formulated by aggregation
of indicators; hence, selection of suitable indicators is most important
for a good index (Kondyli, 2010). Different criteria are used worldwide
for selection of appropriate indicators for measuring SD. Important
criteria used for development of SDIs in different parts of the world are
listed below (Harger and Meyer, 1996; Morse, 2013; Redclift, 2005;
Singh et al., 2012; UN, 2007).
1. Representative – The index must include all dimensions of sus-
tainability, and be capable of capturing the multidimensional
nature of sustainable development that it aims to measure.
2. Scientifically valid – The index must be tested and validated to
measure what it is intended to measure.
3. Specific, measurable & quantifiable – There should be a clearly
defined method of measuring the indicator, so that if two people
measure it, they would measure it in the same way. Further, the
method of data collection must be feasible for various indicators in
the index. The indicators used in the index must have the capacity
to be counted, observed, analyzed, tested, or challenged.
4. Reliable – The data sources for the index and the statistical tech-
niques used must be reliable and must show consistency in results,
as the index value is repeatedly calculated using the same proce-
dures and conditions.
5. Based on accessible data – The data for various indicators used in
the index must be accessible within a stipulated time frame and at a
desired scale (global/national/regional/city).
6. Relevant – The index must be relevant in the present context and
capable of capturing the underlying issue. Further, the index must
be flexible and adaptable to future needs.
7. Precise – The number of indicators used in the index must be lim-
ited in number but also be sufficient to measure the multi-dimen-
sional nature of sustainable development. If too many indicators
are used, the results become unwieldy and difficult to interpret.
Only the indicators that are the most important or strategic ones
must be used.
8. Simple, unambiguous, practical & cost-effective - The indicators
selected in the index must be conceptually strong yet simple in
terms of understanding and application, thus, to be free from am-
biguity. Since, to regularly gauge progress, it is necessary to com-
pute and evaluate SDI on regular interval, the data for the indicator
3
Ecological Indicators 112 (2020) 106061
used in the index should not be too difficult to collect. The indicator
should be reasonable in terms of the data collection cost, frequency,
and timeliness for inclusion in the decision-making process.
(Becker, 2004 as cited in Ramos and Caeiro, 2010).
9. User friendly– The index itself, and the results it produces, must be
easy to comprehend and interpret by its users.
10. Show trends overtime – The index must show certain kind of trend
over several years to compare and assess the progress made towards
achieving sustainable development.
11. Sensitive to changing circumstances – The index must be sensitive
and must be able to capture the smallest changes occurring in the
environmental, social or economic aspects of sustainable develop-
ment.
The above section discusses about the criteria of an effective SDI.
Before developing an SDI, it is essential to understand geographical
scale of application (global, national, regional, and city). Based on the
scale of application, next thing is defining an approach (top-down,
bottom-up or mixed) to be used in the development of the SDI. The SDIs
are generally formed using a framework in which indicators are orga-
nized in different ways to assess sustainable development. To create a
relevant SDI, it is essential to have clarity on its geographical scale of
application, approaches and frameworks to be used in construction of
the index. The subsequent three sub-sections discuss them in detail.
3.2. Need and application of SDIs at varying geographical scales
Sustainable development concerns are found to be different at dif-
ferent scales, as demonstrated by studies undertaken to assess sustain-
ability at varied geographic scales including global, national, regional,
and city settings (Moffatt, 1996; Hanley et al., 1999).
Global indices are the ones that are employed to keep track of the
concerns that are relevant at the planetary level, and require global
coordination. These are generally used to track common global issues;
some of these may be applicable even at the national scale for some
countries, but may not be equally relevant to every country. These in-
dices also act as a common and consistent set of metrics employed to
not only assess global progress towards a goal, but also relative per-
formance of the countries. Agencies at the global scale are involved in
developing these indices in consultation with partnering countries and
provide technical support for data collection and reporting at the
country scale (Parris and Kates, 2003 ). Some of the global indices are
Ecological Footprints (EF), Living Planet Index (LPI), Human Devel-
opment Index (HDI), Environment Sustainability Index (ESI), Environ-
ment Vulnerability Index (EVI), Environment Performance Index (EPI),
Happy Planet Index (HPI), United Nations Commission on Sustainable
Develeopment - Sustainable Development Index (UNCSD-SDI), and SDG
Index. Most of these indices do not fit into the 3-pillar model or triple
bottom line approach of sustainable development (i.e., they do not
measure sustainability on all the three dimensions – social, economic
and environment). The one exception is the UNCSD-SDI and SDGI
Index. Each of them focuses on either one or two dimensions (Mori and
Christodoulou, 2012).
National scale indices are required to adjudge the relative perfor-
mance of different states/provinces in a country based on commonly
applicable issues. These indices may not be simply a sub-set of the
global set of indices, as there could be national priority issues that may
not be so important at the global scale. Even within a country, there
could be large variation across different regions (especially in a larger
country such as India) and indices at the national scale may eclipse a
very prominent issue in a particular region in the country, e.g., mining
may not be an issue of concern for the whole nation since it contributes
to the economic leg of the nation as well as of the region where mining
is happening. The indicators used for national level assessment may not
use ‘sustainable mining’ as a priority indicator in the index, whereas, at
the regional scale, unsustainable mining practice will have huge level of
S. Kwatra, et al.
implication on environment and social wellbeing of people especially in
an environmentally fragile area. Hence, National-scale indices if ap-
plied (exactly) at the regional scale, may also elevate problems in sub-
national scales having extremely unsustainable situations (Herrera-
Ulloa et al., 2003; Meadows, 2005 cited in Coelho et al., 2010). Re-
gional scale sustainability issues, if dealt adequately, can eventually
contribute to addressing national scale sustainability concerns. Hence,
it is essential for a region to carry out self-assessment to assess the pace
and direction of their growth while moving on development pathways.
While several indices at global scale have been developed to measure
and compare sustainability of different nations (Singh et al., 2012),
only few efforts have been made to develop indicators at regional scales
(Mascarenhas et al., 2010). It is widely recognized that actions towards
sustainable development are most effective at the regional scale. As
resources are common at regional scales, deploying sustainable in-
itiatives and measuring them at the regional scale is equally essential as
at national scales (Mascarenhas et al., 2010).
City scale indices attempts to compare and assess cities on en-
vironment, social and economic dimension based on the concept of
strong sustainability (Kori and Gondo, 2012). Several efforts are on-
going across the world to develop city scales indices. These include SDIs
such as City Development Index (CDI), Urban Sustainability Index (USI)
and City Sustainability Index (CSI), which estimate and compare sus-
tainability of different cities across the globe in developing or devel-
oped nations. Certain countries are also using city level SDIs to compare
major cities in their own country such as USI developed in 2002 com-
pared 112 Chinese cities (Zhang, 2002 and Xiao et al., 2010).
The review of literature indicates that more initiatives have been
conducted at the global and city scales as compared with the national
and regional scale. While it is important to have indices at a macro scale
(global/national) to suggest a broad trajectory of growth and assess
relative performance of its sub-units (e.g. countries/states), micro scale
(regional/local) indices are also essential to identify local sustainability
concerns and provide corrective measures. Improvements at the re-
gional/local scale are cumulatively expected to enhance sustainability
at the macro scale also. However, there is a need to develop effective
interaction in policies at national and regional scale to attain long-term
sustainability (Coelho et al., 2010). Since, regional scale problems are
different from national scale problems, and thus, require tailor-made
separate regional scale indices apart from the national scale indices
(Mickwitz et al., 2006 as cited in Coelho et al. 2010). Also, to attain
sustainability at the national scale, vertical integration of sustainable
development policies at national and regional/local scales is essential
(Mascarenhas et al., 2010). This requires interdependence among
governments at different scales – national, state/province and local
(Swanson and Pintér, 2004 as cited in Mascarenhas et al, 2010). Fur-
ther, it could be a cause of tension between national and state gov-
ernments, if not integrated, as it is difficult to reconcile the national and
regional priorities (Mascarenhas et al, 2010).
After deciding the scale of application of the study, the second thing
to be assessed is the approach to developing a SDI.
3.3. Approaches to develop SDI
For larger scale contexts such as global and national scale; top-down
approaches are used often, while at smaller scales (regional/urban),
bottom-up methods are employed to take into account local stakeholder
perspectives. Top-down approaches are driven by the opinions of ex-
pert/researchers, through which the framework is designed based on
broad common issues without taking into account specific issues in
different regions. On the other hand, bottom-up approach are partici-
patory whereby view-point of various local stakeholders is taken while
developing a framework to measure sustainable development (Bell and
Morse, 2005; Lundin, 2003; Singh et al., 2012). The top-down approach
is considered to be faster, and hence has been used extensively across
the world for development of indicators. The bottom-up approach is
4
Ecological Indicators 112 (2020) 106061
advantageous in regional contexts where local issues and the view-
points of different stakeholders are prominent enough to be considered
in the analysis. Stakeholder participation in conceptualization and de-
velopment of an index is essential to incorporating their views, values,
concerns and identifying common goals of sustainability (Valentin and
Spangenberg, 2000; Kelly and Moles, 2002; Beratan et al., 2004 as cited
in Mascarenhas et al., 2010). The bottom-up analysis not only addresses
the issue of deeper understanding of the regional systems and their
sustainability issues, but also empowers social learning (Hermans et al.,
2011). Bottom-up analysis evokes active participation of key stake-
holders in the development process. Their collective participation al-
lows addressing the issue of sustainability in holistic manner as ev-
eryone perspective is taken, issues, problems and local solutions to the
problem are thought in a collective manner. This active involvement of
stakeholders helps social learning and promotes deeper understanding
of issues and solutions pertaining to sustainable development in the
region. Moreover, the process also helps in building consensus, and
enhances the acceptance of the SDI in a region as all key stakeholders
were involved in the process. However, it is more time and resource
consuming, and thus, has been used limitedly for certain regions only.
According to Coelho et al. (2010), involvement of stakeholders was
beneficial in regional sustainability assessment as it allows identifica-
tion of major sustainability concerns at the regional scale. Thus, public
participation is an essential component of any regional scale SDI de-
velopment (Coelho et al., 2010). Most regional SDIs are based on triple
bottom- up approach, i.e., dependent on three dimensions of sustain-
ability, namely social, economic, and environment. Some of them also
incorporate specific regional policy topics. It has been observed that the
ratio of social indicators is more in comparison to the economic or
environmental indicators in development of CSDI. (Coelho et al., 2010).
Most of the regional studies have indicators ranging from 30 to 60. Very
few have more than 90 indicators used to formulate SDI.
Boggia and Cortina (2010) developed a regional scale SDI whereby
multi-criteria approach was used to compare the progress made by 92
municipalities of the Umbria region. Nine environmental and 9 socio-
economic indicators were used in the Index, whereby various decision-
makers were involved in the process of assigning weights to different
indicators.
Munda and Saisana (2010) carried out a regional scale sustainability
assessment for selected regions in Spain, Italy and Greece using 29 in-
dicators based on the three dimensions of sustainability. Weights were
assigned to each indicator based on regional preferences. As CSDI aims
to influence policy and social change for sustainable development of the
region, importance of public participation in regional scale assessment
has been highlighted in the study. Graymore et al. (2010) stated that
there are limited tools available for the local or regional governments to
measure sustainability and those that are available do not match the
needs or capacity of local experts. In the study, they developed a tool
‘Sustaining Human Carrying Capacity’ to assess sustainability at the
regional scale, which was based on the concept of human carrying
capacity (Rees and Wackernagel, 1994) using the DPSIR approach.
Using 40 indicators, an aggregated index was formed. They argued that
a mix of top-down and bottom-up approaches will be more successful
for developing sustainability reporting tools by involving both sus-
tainability experts and local stakeholders. Reed et al. (2008, 2006) and
Fraser et al. (2006) also emphasized the need for integrating top-down
and bottom-up approaches at the regional scale SD assessments. SS,
1998, is a unique index that was developed using a bottom-up ap-
proach. The community volunteers of Seattle derived indicators to as-
sess sustainability of Seattle. The United Nations Centre for Human
Settlements at Habitat II conference in 1996 honored Seattle with an
Excellence in Indicators award. Long term health and vitality in cul-
tural, economic, environmental and social well-being are the major
aspects considered in formulation of the index. Environment, Popula-
tion & Resources, Economy, Youth & Education, and Health & Com-
munity are the basic domains used in the Sustainable Seattle Index (SS,
S. Kwatra, et al. Ecological Indicators 112 (2020) 106061

1998).
To summarize, while top-down approach is less time and resource Driving Force
Response
intensive and has been generally applied at larger geographical scale,
the bottom-up approach for indicator selection has been used when the
study domain is small and there are region-specific issues which need
understanding through stakeholder consultations. There is a research
Pressure Impact
gap in development of regional scale composite indices of sustainability
based on the bottom-up approach, wherein the weights to different
aspects are computed based on local knowledge and priorities.
State
3.4. Frameworks for formulating SDIs

Framework is a schematic visual representation of what is to be

measured by the Index. Framework helps identify core indicators re- Fig. 2. The DPSIR framework for reporting on environmental issues. Source:
lated, identify reliable data, analyze and structure information. The Smeets and Weterings (1999).
index frameworks are different ways to organize sets of indicators so
that all aspects are covered with their true representation. Frameworks
addition of Drivers and Impacts. This framework has been adopted
are important as they provide clarity in terms of what to assess, what to
by the European Environmental Agency (EEA) and the European
expect from the assessment, which type of indicators must be selected
Statistical Office in 1997 and also by the Centre for Development
(UN, 2007), and mechanisms by which various stakeholders can posi-
Finance to formulate the Environmental Sustainability Index for
tively contribute in the process (Coelho et al., 2010). Frameworks help
Indian States (Dash et al., 2011). This framework enhances the
in defining the vision, goal, and methodology to be adopted for de-
scope of the index by taking into account the driving forces re-
veloping the index and in selection of indicators for the same (Becker,
sponsible for a certain impact over environment. The framework
2004). Different frameworks have been used across the world to orga-
classifies the indicators into the categories - DPSIR (Driving Force –
nize the SDIs, with each one having its own merits and demerits (Singh
Pressure – State – Impact – Response). Driving force refers to the
et al., 2012). The main SDI frameworks used across the world are:
social processes that lead to certain human activities that have di-
rect impact on environment. Basically, this category helps in iden-
1. Pressure – State – Response (PSR) Framework is based on the con-
tify and address the root causes of the pressures exerted on the
cept of cause-and-effect relationships. This framework was devel-
environment. Pressures refer to both anthropogenic as well as nat-
oped by OECD (1994) to measure environmental dimension of
ural in this framework, unlike the PSR framework. State refers to the
sustainable development. PSR approach primarily identifies the
state of the environment, and impacts refers to the ways in which
anthropogenic pressures on environment, assesses the state of the
changes in the environment influences human well being. A new
environment, and defines the responses taken by the society in re-
dimension introduced in this cause-effect linkage-based framework
action to state of environment (Carr et al., 2007; UN, 2007). Each
has been the response. Response refers to the institutional efforts
indicator in this framework is classified as a pressure, state, or re-
made towards addressing the environmental changes (Fig. 2).
sponse that helps policy-makers and the public to visualize linkages
between economic growth and environmental implications (Fig. 1).
Limitations of DPSIR framework have been similar as for the PSR
framework, including that it is not able to examine and measure com-
The major limitation of the PSR approach is that it focuses only on
plex interlinkages between different categories. Moreover, in all these
the anthropogenic pressures on environment, regardless of the pres-
frameworks, categorisation of indicators under different categories
sures resulting from the natural system. The UNCSD in 1997 addressed
(DPSIR, DSR or PSR) also remains an issue of confusion. Other limita-
this limitation by adopting the “Driving force–State–Response frame-
tions with the DPSIR framework is that the central focus of this fra-
work” instead of “Pressure–State–Response framework” while devel-
mework is either driving forces or pressures. SDI using this framework
oping indicators for sustainable development. In this new DSR frame-
can only be used by experts having good knowledge of the environment
work, driving force was considered as pressure on the environment
and economics, as categorisation of the indicators under different ca-
resulting from both anthropogenic as well as natural causes. The major
tegories –DPSIR has to be carried out by experts in the scientific pro-
limitation associated with this framework is that it is applicable in as-
cess. Local involvement can only be limited to categorization of in-
sessments related mainly to environmental impacts. Further, classifi-
dicators under the impacts or response categories. Finally, the indices
cation of indicators into pressure, state or response is often ambiguous;
based on this framework sometimes fail to adequately highlight gaps at
there are uncertainties over cause-and-effect linkages (Carr et al., 2007;
the policy level, as the results are not able to provide gaps and direc-
UN, 2007).
tions at the sector level. (Bell et al., 2007; Smeets and Weterings, 1999;
Spohn, 2004; UN, 2007).
2. Driving Force – Pressure – State – Response – Impact (DPSIR) fra-
mework is an extension of the PSR and DSR frameworks, with an
3. Issue- or theme-based frameworks group the indicators into dif-
ferent relevant issues or themes of sustainable development based
on policy relevance (Fig. 3). The advantage of using this framework
at national or regional scales is that it easily links indicators with the
existing sectoral policies and targets, which help the policy-makers
to track the progress made towards desired goals and objectives
(Labuschagne et al., 2005 and UN, 2007). This was the core reason
for UNCSD to switch from Driving force–State–Response framework
to theme-based framework. The main disadvantage of this frame-
work is that it provides insights for different sectors individually;
however, it does not carry out the cumulative assessment of dif-
Fig. 1. The pressure-state-response framework. Source: OECD (1998). ferent sectors towards attaining sustainable development objectives.

5
S. Kwatra, et al.
Fig. 3. The United Nations Commission for Sustainable Development (UNCSD) theme indicator framework. Source: Labuschagnea et al. (2005).
4. Capital Framework expresses all forms of capital in common
monetary terms. It calculates national wealth as a function of the
sum of interactions among different kinds of capitals (financial ca-
pital, produced capital goods, human and institutional capital). The
advantage of this framework is that the results are available in
monetary values, which is more widely understandable. The major
drawback of this framework is the requirement for conversion or
substitution of different forms of capital into monetary terms. In
some cases the conversion is straight-forward (e.g., monetary values
of crops); however, in many cases this is not easy and often deba-
table. Accounting for non-financial capital assets into monetary
value has inherent problems. Due to lack of robust markets or proper
evaluation techniques, social and environmental capital can be ea-
sily conceptualized as assets or liability, but cannot be accurately or
adequately priced (SASB, 2013). For example, value of a human life
is a difficult characteristic to be monetarily quantified and generally
remains very controversial. Similarly, quantification of loss in bio-
diversity in monetary terms is extremely difficult. Thus, the results
of the index, which are in monetary terms, sometimes become de-
batable (UN, 2007).
5. Accounting framework generally works on two legs of sustainable
development – environmental and economic, however, it neglects
the social and institutional components which are important in the
sustainable development context (Kee and De Haan, 2004). This
framework takes different indicators from a single database that
allows aggregation of various sectors with the use of same classifi-
cations and definitions. The main advantage of this framework is the
use of consistent databases, allowing for meaningful sectoral and
spatial disaggregation, which means the trajectory to achieve cer-
tain targets and cross-sectoral impacts can be evaluated consistently
(UN, 2007). This framework has been used by System of Integrated
Environmental and Economic Accounting (SEEA).
6. Aggregated framework normalizes and concentrates various in-
dicators into a composite index (CI), which provided results in form
of a single value which is unit-less and comparable, indicating an
overall comprehensive picture of sustainable development. This
framework has been used by the policy-makers to assess present
policies, future plans, and to raise public awareness. The aggregated
value of the CSDI provides relative performances of an area across
years (temporal) or regions (spatial) over the same year. The
6
Ecological Indicators 112 (2020) 106061
methods used for SDI using the aggregated framework must be
simple, transparent, and easy to be summarized, as it is to be used
and interpreted by policy makers. In case detailed analysis of SDI is
required for more in-depth assessment, a clear, simple and trans-
parent aggregation method would help immensely (Fraser et al.,
2006). The aggregated framework has both advantages and limita-
tions. As an advantage, it provides a single composite value and a
clear overall direction of the progress made towards sustainability
goals. The major drawback of this framework is that it faces sig-
nificant challenges in terms of data availability, selection of vari-
ables and relative weights required to be put against each of the sub-
indicators. Also, aggregation sometimes leads to eclipsing, which
implies loss of information. Still this framework is being used widely
for formulation of indices such as EF, 1997; HDI, 2001; ESI, 2002;
EPI, 2006; HPI, 2006; USI, 2002/10/11; and EVI, 2005 (UN, 2007).
In summary, it can be stated that identification of an appropriate
framework is the important step for developing an SDI. The framework
defines the prime objective of an index and its measurement, expecta-
tions, and also the type of indicators to be used for index formulation
(UN, 2007). Since, aggregated framework has been the most widely
used framework for developing SDIs, thus, has been reviewed in more
details for better understanding and its application.
3.5. Composite sustainable development index (CSDI) based on aggregated
framework
CSDI is a multi-dimensional, well recognized tool used for policy
analysis, public communication, and benchmarking of performance
(Nardo et al., 2005; Kondyli, 2010). The major benefit of CSDI is ease of
interpretation, as it can summarize and represents complex issues in a
single value. The prime purpose of the CSDI is not to give values at the
end but to generate discussions at policy level, and attract public and
media attention on relevant issues. Since CSDI provides a single ag-
gregated value, which is easy to interpret and track, it can assist policy-
makers to take informed decision. It benchmarks the performance of a
unit on spatial as well as temporal level. Also CSDI can include more
information within the limited dataset. As a drawback, CSDI can be
misleading, if it is poorly constructed at any step of it’s formulation–e.g.
selection of indicators, normalization, weighting, and aggregation. The
S. Kwatra, et al.
different steps used in construction of CSDI based on the review of
various studies are described below.
4. Steps for constructing CSDI
According to JRC-EC (2008) and Kondyli (2010), building CSDI
requires ten-step process:
1. Conceptualization of the index – The CSDI is formed from compi-
lation of individual indicators into a single index. Thus, it is essential
to clarify and clearly define its theoretical framework (Freudenberg,
2003). The theoretical framework must clearly define the objectives
of CSDI, geographic scale of application, approach, framework as
well as identifying individual themes and indicators required to
construct the index (Mazziotta and Pareto, 2013). It should describe,
provide justification and reasons for selection theme and indicator
used in construction of CSDI. The individual indicators selection
must be on the basis of their desirability and availability. Also, in-
volvement of experts or stakeholders in development of the index
must be determined at this stage (JRC-EC, 2008).
2. Data selection – The selection of variables should be done on the
basis of their accuracy, accessibility, timeliness, and soundness. It is
important to check the quality of each available indicator.
International Monetary Fund (IMF) provides five quality dimensions
– assurance of integrity, methodology soundness, accuracy and re-
liability, serviceability, and accessibility of indicators that should be
examined. For the indicators, European Statistics Code of Practice
provides six quality dimensions – relevance, accuracy, timeliness
and punctuality, accessibility and clarity, comparability, and co-
herence. Hence, the indicator, selected must be relevant and must
meet the current and potential needs of the users. Further, indicators
have been selected in different studies through top-down and
bottom-up approaches. Wood and Garnett (2010) used the top-down
approach to select 10 indicators for sustainability assessment in
northern Australia. In this, they carried out in extensive review of
literature to identify indicators of most relevance for the region.
Boggia and Cortina (2010) also used a top-down approach for se-
lection of 18 indicators based on policy relevance, analytical
soundness, and measurability. On the other hand, few studies have
used a bottom-up approach for selection of indicators. Sardain et al.
(2016) selected 20 indicators out of 341 for developing a SDI
dashboard for Panama, through a participatory approach.
Mascarenhas et al. (2010) selected 20 indicators based on regional
priorities, through local level workshops conducted in the Alagarve
region. In order to maintain simplicity, the number of indicators
used in the index must be kept to the minimal; however, adequate
number of indicators has been ensured to appropriately capture the
performance of SDI (Simon et al., 2016). In literature, most of the
studies have used number of indicators ranging from 30 to 60. There
are very few studies which have used more than 90 indicators to
formulate SDI. The data required for the indicator must be afford-
able in terms of the accessibility and availability. The indicators
should be interpretable; user must be able to understand and use it
properly to analyze the data. The indicators should be coherent i.e.
they should have same meaning world-wide. There should be dif-
ferent terminology but understanding and interpretation should be
same worldwide. In case desired indicators are not available one
may use proxy indicators (UN, 2007). At this stage, quality check of
each indicator must be done by looking at its strengths, weaknesses,
and availability (JRC-EC, 2008)). Correlation analysis of indicators
must also be carried out to understand positive/negative linkages
between them. The indicators which are highly correlated with each
7
Ecological Indicators 112 (2020) 106061
other could be removed in order to remove any redundancy in the
SDI (Gustavson et al., 1999; Salzman, 2003). A detailed summary of
individual indicators used must be laid down indicating their
characteristics, availability and sources.
3. Imputation of missing data – For any CSDI to be robust, the data for
indicators should be complete; therefore, in the case of missing data,
techniques should be used for imputation. Data can be found
missing broadly in three fashions:
a. Missing completely at random (MCAR)– data missing is not de-
pendent on the variable of interest or any other observed value.
b. Missing at random (MAR) – data missing is not dependent on the
variable of interest or depends upon any other observed value.
c. Not missing at random (NMAR) – Missing value depends upon the
value itself.
There is no statistical tool available for imputing values of NMAR.
The main methods used for imputing data are: (a) case deletion, (b)
single imputation or (c) multiple imputation (Nardo et al., 2005).
Single imputation methods are mean/median/mode substitution,
hot–cold deck imputation, unconditional mean imputation, regression
imputation, and expected maximization imputation. A multiple im-
putation method is the Monte Carlo Algorithm (Kondyli, 2010). The
benefit of this step is that it helps to estimate the missing values in the
available data. The imputation method used must be clearly recorded
and described. Also in this step, the presence of outliers in the dataset
must be given due attention. Outliers in dataset must be verified in
terms of accuracy. In case of accuracy of data, identify core reasons
behind such a variation. As presence of outliers may become un-
attended benchmarks in a composite index. The improper selection of
imputation method and outliers in the dataset may result in increasing
the uncertainty of the index (Dempster and Rubin, 1983; JRC-EC,
2008).
4. Multivariate analysis – While forming a CSDI, multi-variate analysis
is an important step to analyze the basic structure of the raw data
and understand the implications of choices such as weighting and
aggregation. But multivariate analysis is not feasible for small da-
tasets (Kondyli, 2010). Multivariate approaches such as Principal
Composite Analysis, Factor Analysis, and Cluster Analysis are used
to:
a. reveal how different variables are related to each other and their
correlations;
b. explore whether different dimensions of the process are statisti-
cally well balanced in the composite indicator or not;
c. classify large amounts of information into manageable datasets;
and
d. disseminate information on composite indicator without losing
track of information in the individual indicators.
5. Normalization – Different individual indicators have varying mea-
surement units. Normalization is an essential step in development of
CSDI as this technique to convert the indicators having different
units into same units or into dimensionless quantities so that they
become comparable. Moreover, in a CSDI, several indicators are
used, with some correlating positively (e.g., income) and some ne-
gatively (e.g. pollution) to the overall SDI value. The normalization
technique converts them into positive indicators so that with an
increase in the value of normalized indicators, the CSDI value also
increases (Mazziotta and Pareto, 2013). The most popular normal-
ization methods used for developing the CSDI are shown in Table 1.
The table also presents a review of studies that have used different
normalization techniques and have also enlisted their advantages
and disadvantages.
S. Kwatra, et al. Ecological Indicators 112 (2020) 106061

6. Weighting and aggregation – Weighting and aggregation of CSDI

Sustainability Performance Index (as cited in Singh et al.,

Developed municipal-level indicators of sustainability for

Composite Sustainability Development Index (as cited in

components is one of the most important steps in the whole process

(Fagerberg, 2001 as cited in JRC-EC, 2008; Singh et al.,

City Development Index (as cited in Singh et al., 2012)

SDI for Coastal City of Yantai, China (Yu et al., 2010)
Information and Communication Technologies Index (Gan et al., 2017). After normalization of raw data, weighting of

Sustainability index for Taipei (Lee & Huang, 2007)

SDI for Islands of the North Aegean region, Greece

Environmental Sustainability Index and Composite
different indicators in the CSDI is carried out to give varying im-
portance to different indicators based on their relevance in the
overall subject of the study. Broadly, weights to a theme/sub-
theme/indicator are assigned by three mechanisms: i) by assigning
equal weights to the indicators, ii) by use of statistical models, and
iii) by using participatory methods (Kondyli, 2010). Most CI, gave
Brazil (Sherbinin, 2003) equal weights to all the individual indicators/themes/sub-themes
giving equal importance to each (e.g., Summary Innovation Index,
HDI, Environment Policy Performance Index, Sustainability Perfor-
Singh et al., 2012)

mance Index, GSI [Singh et al., 2012]). Equal weights are assigned
(Kondyli, 2010) when all the indicators in CSDI are of equal importance or when no
statistical or empirical evidence is available to assign the weights
Applied

2012)

2012)

(Nardo et al., 2005). Differential weights are sometimes chosen ei-

ther to give varying priorities to the different themes of the index or
to prioritize datasets with higher certainties. When statistical
It excludes information about variance of
evaluated as information on levels is lost

models are to be used, weights are assigned to themes/ sub-themes/

Extreme values have greater effect on

Outliers could distort the composite

indicators must be according to their degree of correlation. Some

Based on extreme values that can
sometimes be unreliable outliers
Absolute value, hence cannot be

commonly used statistical models for weighting purposes are Factor

Analysis (FA), Principle Component Analysis (PCA), and Data En-
the transformed indicator

velopment Analysis (JRC-EC, 2008). As PCA/FA reduces the di-

mensionality of the actual datasets (indicators), these techniques are
used when a large number of indicators need to be taken (Gan et al.,
Disadvantages

2017). Also, they reduce the risk of double weighting, which may
indicator

occur in case of the equal weighting method (Yeheyis et al., 2013).

The PCA/FA approach has been used by experts for weighting dif-
CSDI

ferent indicators in CIs studies the Business Climate Indicator, EPI,

and CDI (Singh et al., 2012). Originally, use of PCA/FA was done to
Reference acts as a target or benchmark for

It rewards the best and penalizes the worst

Could widen the range of indicators lying
different units into dimensionless values.

understand the relationships between variables or indicators of CSDI

Simple method to convert quantities in

and not to determine weights (Hermans et al., 2008). The major

drawback of using PCA/FA for estimation of weights is that some-
times unrelated indicators are grouped into the same dimension
because of false correlations. Further, the ideal number of dimen-
Not affected by outliers

sions may vary with usage of varying methods for principal com-
within small intervals

ponents extraction (OECD, 2008), which could lead to unpredictable

rest of the dataset

outcomes (Hermans et al., 2008). The third category of approach

Advantages

that used for weighting is the participatory approach. This approach

is used when SDI is to be used as a guiding tool and when a well
defined basis for policy decision is required (Munda, 2005; Munda
and Nardo, 2007). Budget Allocation Process and Analytical Hier-
It assigns a score (quantitative such as 1, 2 or 3; or qualitative
It measures the relative position of an indicator to a reference

archy Processes (AHP) are examples of participatory methods. Saaty

subtracting the minimum value and dividing by the range.

such as completed, partly complicated, not done) for each

(1980) developed the AHP technique to deal with complex decision

Converting indicators to z-scores with a mean of 0 and

It normalizes indicators to have an identical range by

making. The AHP leads to the best possible decision making by

exposing previously complex decisions to a series of pair-wise
comparisons, and then synthesizing the results. The technique ac-
counts for both subjective as well as the objective aspects of decision
Simple ranking of the indicator values

making. This multi-criteria choice making technique now is being

extensively used for assigning weights to different themes and sub-
themes (indicators) of sustainability while developing a CI. AHP
Normalisation techniques generally used in CSDIs.

now has been extensively used in various CIs for sustainability as-
sessment [Poveda and Lipsett (2013), Qureshi and Harrison (2003),
standard deviation 1.

Yu et al. (2010), Barzekar et al. (2011), Jain et al. (2014), and Panda
et al. (2016)]. Krajnc and Glavi (2005) used AHP for weighting the
indicators in order to formulate an index to assess sustainability of a
Description

indicator

company. Bhatt et al. (2010) also used the AHP technique to carry
point

out sustainable building assessment in an Indian context. Some of

the common indices such as Composite Sustainability Performance
Index, Composite Sustainable Development Index, Environment
Name of the technique

Distance to reference

Quality Index, and Life Cycle Index also use AHP for assigning
Categorical scale

weights to different indicators in the index (Singh et al., 2012). The

Standardization

major drawback of AHP is that it requires a high number of pair-

Min-Max

wise comparisons. Use of too many indicators can lead to higher

Ranking
Table 1

inconsistency in the results (Gan et al., 2017). Response of the sta-

keholders who show high inconsistency (Consistency Ratio greater

8
S. Kwatra, et al.
than 15) then would not be incorporated in the final analysis.
However, there are few studies which have used different ways for
weighing the themes/sub-themes. Google search engine ranking of
various themes of sustainability has been carried out by Hak et al.
(2012) to assess Czech Republic’s Sustainable Development
Strategy. The technique is not relevant where internet usage is low.
Further, Q-sort method has been used by Doody et al. (2009) to rank
order the themes; however, it requires high level of expertise.
After weighting, aggregation is the most important step to formulate
SDI. Aggregation in CSDI is generally done linearly when all indicators
have the same unit of measurement. This is also called additive/ar-
ithmetic average aggregation. In the past, it has been used for assessing
sustainability of islands of the North Aegean region, Greece (Kondyli,
2010), EPI (Esty et al., 2008), ESI, and HDI. The major advantage of
using this is its simplicity and transparency. Sensitivity and uncertainty
analysis become simple in this type of aggregation (which is also the
next step in the CSDI development process). The only requirement is
that the various indicators used in the aggregation process must be
mutually exclusive (Gan et al., 2017). Geometric aggregations use
multiplicative instead of additive functions. Some indices (e.g., Life
Cycle Index and LPI) use geometric aggregation. While the linear
method rewards an indicator in proportion to its weight, geometric
method rewards those with higher scores.
7. Uncertainty and sensitivity analysis– It is important to identify the
propagation of uncertainty from the input factors to the CSDI va-
lues. Sensitivity analysis is carried out to compute the share of
uncertainty caused by individual indicators to the CSDI
(Freudenberg, 2003). Uncertainties in the development of a CSDI
would be due to a number of factors, such as selection of an ap-
propriate framework, selection of appropriate indicators, normal-
ization technique used, imputation of missing data or removing
outliers, assignment of weights to various themes and sub-themes,
and choice of aggregation technique (e.g., additive, multiplicative,
or multi-criteria analysis) (Nardo et al., 2005). Uncertainty analysis
is carried out by including or excluding individual indicators, and
using alternative normalization, weighing, and aggregation
schemes. Use of alternate weights has been used to carry out un-
certainty analysis (Yu et al., 2010; Boggia and Cortina, 2010). Since
composite indices have been used extensively by various countries
to measure or benchmark performances of the countries on sus-
tainable pathways, and these indices have been used to influence
policymakers and masses, it is essential to have check the robust-
ness of the index. Uncertainty and sensitivity analysis increases
transparency and ascertains robustness of the index (Quaddus and
Siddique, 2001; Saisana et al., 2005; Munda and Saisana, 2010).
8. Back to the data – This step is important as the CSDI alone may not
be sufficient to inform the decision-makers comprehensively. This
step of ‘Going back to data’ helps to understand and reveal the core
issues which are impacting the result of the CSDI most significantly.
Cause and effect relationships of different indicator values on the
index could also be studied through several statistical analyses.
Hence, the de-construction of CSDI can really help in extending the
analysis to the sub-themes. This information can thus become the
basis for suggesting future strategies that could help improve sus-
tainability.
9. Links to other indicators – This is an added step for use of the CSDI
by comparing it which other relevant indicators (e.g., the sustain-
ability indicator can be compared with GDP of a country). The
linkage is expected to evolve the explanatory powers of the com-
posite indicator.
10. Visualization of the results – Visualization of results in the CSDI
must clearly, quickly and accurately communicate a story to policy-
makers and the public. The numbers estimated for the CSDI can be
presented in a form (graphics- bar, line, pie charts, spider diagrams,
9
Ecological Indicators 112 (2020) 106061
etc.) that can convey the information effectively to decision-makers
and other important stakeholders. Saisana et al. (2005), Defra
(2007), Sardain et al. (2016) have used various types of charts to
visualize the results of SDIs. However, tables, which provide com-
plete information sometimes, are irreplaceable, as charts may not
be able to show all the datasets accurately or in detail. Several other
interesting ways by which data has been visualized in many studies.
To visualize HPI values, Abdallah et al. (2012) used a traffic lights-
based visualization system, which denotes thresholds of good,
moderate, and bad performers in the form of green, amber and red
colors. This method, when applied at the country scale, showed the
spatial variation of performance on the world map. Porter et al.
(2004) used the change in rank approach (difference in rank be-
tween two timeframes) to visualize results, which helped in iden-
tifying the fastest changing countries in terms of good and bad
performance. The main drawback of ranking-based visualizations is
that they only provide ranks in relative terms and tend to lose in-
formation on the actual value of CI. UNDP (2013) used the ranking
and scoring method to visualize the comparative scores of different
countries for the HDI. This visualization method identified the front
runners and laggards clearly, as the countries could see their ranks
and compare the differences in their scores with other countries
scores. EVI (2005) used a report card-based visualization format
whereby the results of all 50 indicators used to develop the EVI are
rated on a 1–7 scale. This provides a comprehensive view of all the
indicators used; however, it could also be confusing due to overload
of information provided.
Many of the studies reviewed in this section discuss only a few steps
in detail, such as, identification of indicators, normalization, aggrega-
tion, and visualization of the results. Very few studies reviewed dis-
cussed the uncertainty and sensitivity analysis step and process used to
carried out the analysis. This raises questions about their robustness.
The review also reveals that most studies have focused on assessment of
current state of sustainable development, very few studied the past
trends and none of them used SDIs for future projections.
5. Evolution of sustainable development indices
The review of literature shows chronological development of re-
search in developing SDIs. In the past several years, Gross Domestic
Product (GDP) remained as the measure of development across the
world. In late 1980′s, the concept of sustainable development started
budding and emerged as an issue of global concern. The initial sets of
indicators/indices used to measure sustainability have mainly focused
on two dimensions of sustainable development – economic and en-
vironment. They were primarily economic indicators that tried to
measure sustainability of various resources for long-term economic
growth. Index of Sustainable Economic Welfare (ISEW), Genuine
Progress Index (GPI), and Genuine Saving Index (GSI) are the classic
examples of this type. ISEW proposed by Daly and Cobb in 1989 to
assess the proportion of economic activity leading to improvement of
quality of life of people (CES, 2000 as cited in Singh et al, 2012). It is an
adjusted economic indicator replacing GDP, which takes into account
the social and economic well-being within environmental limits. The
index took into account 20 indicators assessing different countries.
Seven of these indicators showed positive impacts on quality of life of
people, namely adjusted consumer expenditure, services from domestic
labor, consumer durables, streets and highways, public expenditure on
health and education, net capital growth, and net change in interna-
tional investment position. On the other hand, the other 13 indicators
showed the aspects which negatively affect quality of life of people,
namely increasing the difference between expenditure and value of
services, costs incurred for commuting, health and protecting in-
dividual humans from various environmental damage caused due to air
pollution, water pollution, noise pollution, climate change, ozone
S. Kwatra, et al.
depletion and loss of natural habitat (Singh et al., 2012). Inspired by the
simplicity and popularity of the ISEW in US, UK, Germany and Austria,
Guenno and Tizzi (1998) reproduced the index to measure sustain-
ability of Italy over the period of 1960–1990 using 21 indicators, 14 of
which were market related and rest 7 were focused on environmental
issues. The major drawback of ISEW was lack of a theoretical basis for
selection of indicators. GPI developed in 1995 and GSI developed in
1997 were attempts in similar direction and modification of ISEW. GPI
was developed by the Government of Maryland (Australia) for national
accounting, which redefined progress by replacing GDP as the measure
of progress with GPI using 26 different indicators from social, economic
and environment dimensions of sustainable development (Hamilton
and Saddler, 1997). GSI is another simple indicator developed by World
Bank to assess the net creation and destruction of national wealth. It is
based on green accounting and social welfare. The index has been cri-
ticized for its theoretical flaws in terms of indicators selection. In-
dicators selected captured only net income generated from the eco-
nomic activities and failed to account for change in human health
capital (Hamilton et al, 1997; Anielski, 2001)
The new indices such as HDI focused upon the economic and social
well-being. In 1990, HDI was proposed by UNDP to measure and de-
velopment across various countries in the world. The index was based
on two aspects of development i.e. social and economic. Four basic
indicators have been used to derive HDI values of different countries.
The four indicators used were life expectancy, adult literacy rate, school
gross enrolment ratio and purchasing power adjusted GDP/capita. HDI
has been criticized as another redundant composite development in-
dicator, as the ranking of the countries provided by GNI/capita and HDI
have been found to be very similar and HDI did not add value to the
existing indices (McGillivray, 1991). Recently, in 2010, the indicators
constituting HDI have been revised. The indicators of adult literacy
rate, school gross enrolment ratio and purchasing power adjusted GDP/
capita have been replaced by mean years of schooling, expected years
of schooling and GNI/capita. HDI is now one of the most widely used
and trustworthy index in the field, however, still there is criticism on
the choice and methodologies used for its constituent indicators (Sever,
2013).
With the outbreak of several natural and man-made disasters
(drought in Africa killing several million, Bhopal Tragedy in India,
Liquid gas tank explosion in Mexico city, Chernobyl nuclear reactor
explosion in Europe, Rhine River tragedy threatening several lives in
Switzerland, Germany and Netherlands, increasing diarrhoeal cases due
to unsafe drinking water and malnutrition across world) the attention of
the world and WCED lead to set up Brundtland Commission to address
this global challenges that not only looks into economic or environ-
mental sustainability but addresses human well-being in totality, thus
considering – social, environmental and economic dimensions to be the
three basic pillars of sustainable development. Further, with the release
of Agenda 21 and inception of UNCSD during the Earth Summit in
1992, a few new indices such as UNCSD indicators, Better Life Index
and Compass Index of Sustainability (CIS) were also developed. These
were based on the 3-pillar model of sustainability – social, economic
and environment. In 1995, UNCSD released its first set of indicators to
measure sustainability. This was based on driving force, state and re-
sponse (DSR) framework, where UNCSD selected 134 indicators to as-
sess sustainability of various countries. Most of the countries found
indicators very elaborate and extensive, in 2001 a revised set of 58
indicators were adopted based on the new thematic framework.
Thematic framework was preferred over the earlier DPSIR framework
mainly because of two reasons. It provided status report as per the
theme so that specific recommendations could be made. Also, it over-
comes the ambiguity of classifying indicators into different categories
as per DPSIR framework. In 2007, new revised set of the 96 indicators
10
Ecological Indicators 112 (2020) 106061
were identified in total, out of which 50 were identified as core in-
dicators. The key themes present under UNCSD index are: poverty;
governance; health; education; demographics; natural hazards; atmo-
sphere; land; oceans, seas and coasts; freshwater; biodiversity; eco-
nomic development; global economic partnership and consumption and
production patterns (UN, 2007). Another index accounting for multi-
dimensional aspect of sustainable development was CIS developed in
1997. It is an aggregated index using 25 different indicators under 4
categories - Nature, Society, Economy, Wellbeing (NSEW - initial letters
of the English-language compass). The innovation of this particular
indicator is that it emphasizes the importance of individual’s well-being
and of human as a basic unit of sustainability. As compass helps in
ascertaining directions, these indicators also direct the path of sus-
tainability and also try to determine the weak and strong aspects of
sustainability in terms of Nature, Society, Economy, and Wellbeing.
Equal weights were given to each indicator for clustering, aggregating,
and scaling processes. The result is interpreted on an absolute 0–100
performance scale. This indicator has a wide application ranging from a
big unit like a country to a smaller unit like an industry (Atkinson et al.,
1997).
At the same time, due to increased environmental degradation,
special emphasis has been given to developing indices that measure
environmental sustainability at global and national levels. This has lead
to emergences of indices such as EF, LPI, ESI, EVI, and EPI. The aims of
these indices were to highlight and emphasize the need for protecting
environment. Majority of these indices provided sustainability reports
at country or global scales. EF is an aggregated CI, which estimates
human pressure on the natural resources on earth. The calculations of
the index is based on total land and water resources consumed by a
country to meet its requirements and the amount of resources required
to dispose waste generated by the country. It does not measure quality
of life and takes into account only the environment dimension of the
sustainable development, altogether neglecting the other two – social
and economic dimensions. The major components of the ecological
footprints include – cropland, forest, build-up area, grazing land,
fishing ground and energy land i.e. land use causing to anthropogenic
GHG emissions (Wackernagel and Rees, 1997). LPI (1998) was devel-
oped by World Wildlife Foundation, Zoological Society of London &
Global Footprint Network and reflects the changes in state of the bio-
diversity & ecosystem health based on trends in about 5000 populations
of nearly 1500 terrestrial, marine and freshwater vertebrate species in
temperate and tropical regions domain. Equal weights were given to all
the indicators. Scaling was done by taking the ratio of the species in the
current year to the previous year (WWF, 2012). ESI, 2002; ESI, 2005;
and EPI, 2006 are the important indicators among several others. The
ESI was developed in 2002 by the Yale Centre for Environmental Law
and Policy for 142 countries to assess their environmental sustain-
ability. It consisted of 76 variables, 21 indicators and 5 components,
namely - environmental systems, reducing environmental stresses, re-
ducing human vulnerability to environmental stresses, societal and in-
stitutional capacity to respond to environmental challenges & global
stewardship. The EVI was developed in 2005 by South Pacific Applied
Geoscience Commission (SOPAC) using 50 indicators for assessing the
vulnerability of the environment of a country to future shocks for 40
Small Islands Developing States (SIDs) countries. Out of these, 32 are
hazard specific, 8 are resistance specific, and 10 are damage specific.
The index uses arithmetic mean as aggregation technique. A scaling of
1–7 is used, where, 1 represents increasing resilience/decreasing vul-
nerability and 7 indicators decreasing resilience /increasing vulner-
ability (South Pacific Applied Geoscience Commission, 2005). The EPI
was built upon historical time series data; it tracks environment per-
formance over the past decade using 16 indicators under the two ca-
tegories of environment health & ecosystem vitality (Esty et al., 2008).
S. Kwatra, et al.
Weights are given using top-down approach i.e. expert based Principal
Component Analysis.
Certain indices started measuring and comparing cities across the
globe on sustainability levels, such as the USI and CDI. CDI was de-
veloped in 1996 by UN-Habitat to rank countries based on the scale of
development in their major cities. CDI was cited as a good index of
urban poverty and governance (UNHABITAT, 2001, 2013). The index
uses a method similar to HDI and considers themes such as infra-
structure, waste management, health, education, and city product. This
was one of the early initiatives to give relative weights to each of these
themes, based on expert opinion. The USI was developed in 2002 to
assess sustainability for various cities in China (Zhang, 2002; Xiao et al.
2010), which aggregated 17 indicators and ranked 112 Chinese cities
on a sustainability level. Generally, simple aggregation, assuming equal
relative weights for the themes/indicators, has been the practice.
Limited research has been constructed on computing weights by taking
into account the local stakeholder viewpoints (Li et al., 2009; Yu et al.,
2010). A sustainability index was developed for Taipei city (the pro-
visional capital of Taiwan) in 2007, using 51 indicators based on the
four sub-themes of sustainable development - social, economic, en-
vironmental, and institutional. In the study, selection of indicators was
done on the basis of discussions with experts, scholars and government
departments (top-down approach) and equal weights were given to
each indicator to develop the composite sustainability index (Lee and
Huang, 2007). In another study, Mori and Christodoulou (2012) de-
veloped City Sustainability Index in 2011, which is based upon the
three dimensions of sustainable development and used a bottom-up
approach by involving various stakeholders in the process. In this,
twelve indicators were used to compare 18 megacities. The major
drawback of the index has been its inclination towards environment,
leaving social and economic dimensions as secondary priorities. Bao
et al. (2011) applied EF as a tool to assess sustainability of Zhejiang, a
province in China, over a period of 17 years (1990–2006). The results
show that the ecological footprint of the city has been increasing over a
period of time, ecological carrying capacity of city remains steady, and
ecological deficit of the city is increasing. The major limitation of the
study is that it looks into the relationship between economic and en-
vironmental dimensions of sustainability and completely neglects the
social dimension. The paper highlights that rapid economic growth in
the province is highly dependent on the environment and resources of
that province. As a limitation of the study, the recommendations pro-
vided are generic in nature rather than specific, which would have been
more useful for making policy decisions. Thus, it provides a broad
analysis; however, it fails to suggest any specific policies for the state/
province to improve its sustainability.
HPI developed in 2006 by the New Economic Foundation (NEF) was
on similar lines and measured three aspects of well-being, life ex-
pectancy and ecological footprints. Life expectancy data of countries is
taken from UNDP Human Development Report. Data on well-being for
each country is drawn from the Gallup World Poll (Gallup, 2018).
Ecological footprints data is taken from Global Footprint Network. Like
any other headline indicator, HPI is also used to provide a board overall
status of sustainability, based on few most important indicators. Simi-
larly, the Better Life Index was developed by OECD, which is a CSDI and
measures 11aspects of well-being namely - housing, income, jobs,
community, education, environment, civic engagement, health, life
satisfaction, safety, and work-life balance (OECD, 2013). It was used for
measuring sustainability of 34 developed countries, which are part of
the OECD region; however, it is considered a complex index than HDI
(Sharpley et al., 2014). The index includes subjective evaluation of
personal well-being, and also it does not provide guidelines for further
improvement. The index evaluates conditions for an average individual
only and doesn’t account inequalities among people within the country.
11
Ecological Indicators 112 (2020) 106061
Equal weights are given to each indicator and the index value ranges
between ‘0′ (for the worst possible outcome) and ‘1′ (for the best pos-
sible outcome).
Recently, to assess the status of different countries SDSN and the
Bertelsmann Stiftung have come out with Sustainable Development
Goal Index (SDG Index) to compare 149 countries on the global scale
(Lafortune et al., 2018). The index showed that Sweden, Denmark and
Norway are the top three in rankings (UNSDSN, 2016). A mixed ap-
proach has been used where stakeholder consultations have been done
to identify final set of indicators to be used in composite SDG Index.
Nearly 86 indicators have been used.
Table 2 provided the summary of the various SDIs, that clearly in-
dicates that most of the indices have been developed based on top-down
approach. Most SDIs available are applicable at global or city scale.
There is clear need to develop SDIs at national as well as regional scale.
Aggregated framework has been most widely used framework to de-
velop these SDIs. The number of indicators used the SDI must be re-
presentative and minimal.
6. Concluding remarks
Indicators and indices act as useful tools to bring the necessary
objectivity to measuring and understanding sustainable development.
Of the indices reviewed, only few of them take into account at least
three basic dimensions of sustainability in assessing progress. Several
frameworks are available to develop SDIs, the aggregated framework,
although sometimes claiming to give an incomplete picture of sus-
tainability, still is the most widely used framework in assessing sus-
tainability of the various regions across the world. This is due to the fact
that the aggregated framework provides a single composite value as a
result, which helps in providing clear direction of progress made to-
wards attaining sustainability. While using the aggregated framework
for developing SDI, it is essential to keep the methods simple, trans-
parent and easy to summarize, as the data are to be used and inter-
preted by policy makers. Broadly, ten steps have been used for devel-
oping a composite SDI based on the aggregated framework; a majority
of the studies reviewed discuss only a few of these ten steps such as
identification of indicators, normalization, aggregation, uncertainty
analysis, sensitivity analysis and visualization of results, and rarely
discuss the other steps.
The review of relevant literature also indicates that both national as
well as regional scale indices are important to ascertain a nations’
progress towards long-term sustainability. SDIs at the national scale are
required to compare the nations’ progress from a macro perspective,
whereas the regional scale indicators are required to reflect community
values, concerns and hopes for the future. Specifically, for nations with
huge regional diversity, it is of utmost importance to understand,
highlight and incorporate regional issues that may have implications on
a nation’s sustainability. Regional scale SDIs provides opportunities for
the regions to introspect their progress on sustainable pathways. Most
of the indicators reviewed such as UNCSD, Better Quality of Life Index
are being used at the national scale in assessing sustainability. A few
city scale indicators are also available to assess the sustainability of a
single city or to compare various cities around the world at their re-
spective country or global scale. USI compare various cities of China
whereas the CDI compares multiple similar cities across the world on a
single index. Only a few indices are available at the regional scale.
Sustainable Seattle is one of them. Regional scale indicies help assess
local issues of sustainability and help state governments formulate
better policies and programmes.
Based on the review of literature, it is clear that there is need to
develop SDI at national and regional level. The performance of the
countries and its sub-units must be appraised regularly. SDI must have a
 S. Kwatra, et al.

Table 2
Summary of development indicators available globally assessing progress towards sustainability.
Indicator, Year Approach* Scale of application ** Framework No of indicators Aspects of sustainability

Gini Index, 1912 Top down Global, National, State Capital & aggregated 2 Economic
Index of Sustainable Economic Welfare (ISEW), Top down Global Capital & aggregated 20 Economic
1989
Human Development Index (HDI), (1990) Top down Global Aggregated 4 Social
UNCSD, 1995; Revised in 2001 & 2007 Top down Global Earlier DSPIR, Presently thematic (2007 134 (in 1995) Social, economic, environment & Institutional
onwards) 58 (in 2001)
50 core (in 2007)
Genuine Progress Index (GPI), 1995 Top down Global Accounting & aggregated 26 Economic, environment
Genuine Saving Index (GSI), 1997 Top down Global Accounting& aggregated 5 Economic, environment
Compass Index of Sustainability (CIS), 1997 Top down Cities / regions Aggregated & aggregated 25 Nature, Economy, Society, Well-Being
Ecological Foot Printing (EF), 1997 Top down National to individual Aggregated & aggregated 6 Nature – water & land resources
Sustainable Seattle, 1998 Bottom-up City Thematic 40 Economic, environmental and social

12
Living Planet Index (LPI), 1998 Top down Global Others 1100 Ecosystem
Better Quality of Life Index Top down Country Accounting 24 economic, environmental and social
City Development Index (CDI), 1996 Revised Top down City Aggregated 11 Economic and governance
2010
Environment Sustainability Index (ESI), 2002 Top down Country Aggregated 68 core-21 Environment
Urban Sustainability Index (USI), 2002/10/11 Top down City Aggregated 17 Social, economic and environmental
Environment Vulnerability Index (EVI), 2005 Top down (40 SIDs - Small Island developing Aggregated 50 Environment
countries)
Happy Planet Index (HPI), 2006 Top down Country Aggregated 3 Social & environment
Environment Performance Index (EPI), 2006 Top down Global Aggregated 16 Environment
Sustainability Index for Taipei, 2007 Top down City Aggregated 51 Social, economic, environmental and institutional
dimensions
SDG Index, 2016 Mixed (Expert consultation) Global Aggregated 86 Social, economic and environmental

*Approach used – Top down, Bottom up and Mixed approach.

* * Scale of application – Global, National, Regional, and City.
Ecological Indicators 112 (2020) 106061
S. Kwatra, et al.
robust methodology, which is simple and easily applicable. The review
shows that CSDIs are the most common form of framework used across
the globe. A combination of aggregation and theme-based framework
for development of SDI will be a good approach as aggregation will
provide overall performance of region on sustainability goals. On dis-
aggregation of the index, theme-based framework helps to connect to
the underlying issues which lead to the overall SDI scores for a region
and hence, can provide inputs to policymakers to identify sectors,
which need more attention for attaining positive sustainability status.
Based on this, the policymakers can then further investigate ways to
improve these sectors, which will eventually contribute to improvement
in the sustainability of the region. Important steps for construction of
composite SDI such as identification of indicators, imputation of
missing data, normalization, weighing, aggregation and uncertainty
and sensitivity analysis must be carried out carefully. Most studies have
been found to use 30–60 indicators for development of SDIs. Indicators
must be relevant, easily and regularly available and must be kept
minimal. Use of bottom-up approaches (involvement of key stake-
holders) in the process of development of SDI must be done at regional
scale as it helps in construction of robust, contextual and consensus
based SDIs. The review also reveals that most studies have focused on
assessment of current state of sustainable development, very few stu-
died the past trends and none of them used SDIs for future projections.
The future projections will assist the policy makers to foresee the future
implication of present policies on long term development.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.
References
Abdallah, S., Michaelson, J., Shah, S., Stoll, L., Marks, N., 2012. The Happy Planet Index:
2012 Report, A Global Index of Sustainable Well-Being. New Economic Foundation,
London, United Kingdom.
Atkinson D., G., Dubourg, R., Hamilton, K., Munasinghe, M., Pearce W., D., Young, C.,
1997. Measuring Sustainable Development: Macroeconomics and the Environment.
Edward Elgar Publishing, England.
AtKisson, A., 2006. Sustainability is Dead – Long Live Sustainability. In: Keiner, M. (Ed.),
The Future of Sustainability. Springer, Dordrecht: Netherlands, pp. 231–243.
Ayres, R., Van den Berrgh, J., Gowdy, J., 2001. Strong versus weak sustainability: eco-
nomics, natural sciences, and consilience. Environ. Ethics 23 (2), 155–168.
Anielski, M., 2001. Measuring the sustainability of nations: The genuine progress in-
dicator system of sustainable well being accounts. In The Fourth Biennial Conference
of the Canadian Society for Ecological Economics: Ecological Sustainability of the
Global Market Place (pp. 1–52).
Bao, H., Chen, H., Jiang, S. & Ma, Y. (2011). Regional sustainability assessment based on
long periods of ecological footprint: A case study of Zhejiang Province, China. African
Journal of Business Management, 5(5), 1774–1780. Available at: www.
academicjournals.org/AJBM.
Barr, S., 2008. Environment and Society: Sustainability. Ashgate, Policy and the Citizen,
Aldershot.
Barzekar, G., Aziz, A., Mariapan, M., Ismail, M.H., Hosseni, S.M., 2011. Using analytical
hierarchy process (AHP) for prioritizing and ranking of ecological indicators for
monitoring sustainability of ecotourism in Northern Forest, Iran. EcologiaBalkanica 3
(1), 59–67.
Becker, J., 2004. Making sustainable development evaluations work. Sustain. Dev. 12 (4),
200–211.
Bell, M.L., Goldberg, R., Hogrefe, C., Kinney, P.L., Knowlton, K., Lynn, B., Rosenthal, J.,
Rosenzweig, C., Patz, J.A., 2007. Climate change, ambient ozone, and health in 50 US
cities. Clim. Change 82 (1–2), 61–76.
Bell, S., Morse, S., 2005. Delivering sustainability therapy in sustainable development
projects. J. Environ. Manage. 75 (1), 37–51.
Beratan, K.K., Kabala, S.J., Loveless, S.M., Martin, P.J., Spyke, N.P., 2004. Sustainability
indicators as a communicative tool: building bridges in Pennsylvania. Environ.
Monitor. Assess. 94 (1), 179–191.
Bhatt, R., Macwan, J.E.M., Bhatt, D., Patel, V., 2010. Analytic hierarchy process approach
for criteria ranking of sustainable building assessment: a case study. World Appl. Sci.
J. 8 (7), 881–888.
Boggia, A., Cortina, C., 2010. Measuring sustainable development using a multi-criteria
model: a case study. J. Environ. Manage. 91 (11), 2301–2306.
Carr, E.R., Wingard, P.M., Yorty, S.C., Thompson, M.C., Jensen, N.K., Roberson, J., 2007.
Applying DPSIR to sustainable development. Int. J. Sustain. Dev. World Ecology 14
13
Ecological Indicators 112 (2020) 106061
(6), 543–555.
CES, 2000. Index of Sustainable and Economic Welfare.
Coelho, P., Mascarenhas, A., Vaz, P., Dores, A., Ramos, T.B., 2010. A framework for re-
gional sustainability assessment: developing indicators for a Portuguese region.
Sustain. Dev. 18, 211–219. https://doi.org/10.1002/sd.488.
Dash, R., Anantram, K., Diwan, M., Venkataramani, V., Sivaprada, C.R., 2011.
Environmental Sustainability Index for Indian States 2011. Retrieved from:. Centre
for Development Finance Institute for Financial Management and Research, Chennai,
India.
Davies, G.R., 2013. Appraising weak and strong sustainability: Searching for a middle
ground. Consilience 10, 111–124.
Defra, 2007. Sustainable Development Indicators in Your Pocket. Department for
Environment, Food and Rural Affairs, U.K Government.
Dempster, A.P., Rubin, D.B.,1983. Introduction pp.3-10, in Incomplete Data in Sample
Surveys (vol. 2): Theory and Bibliography (W.G. Madow, I. Olkin and D.B. Rubin eds.
) New York: Academic Press.
Doody, D.G., Kearney, P., Barry, J., Moles, R., O’Regan, B., 2009. Evaluation of the Q-
method as a method of public participation in the selection of sustainable develop-
ment indicators. Ecol. Ind. 9 (6), 1129–1137.
Esty, D.C., Levy, M.A., Kim, C.H., De Sherbinin, A., Srebotnjak, T., Mara, V., 2008.
Environmental Performance Index. Yale Center for Environmental Law and Policy,
New Haven, USA.
Fagerberg J. (2001), in Lundvall B. and Archibugi D. (eds.) Europe at the crossroads: The
challenge from innovation-based growth in the Globalising Learning Economy,
Oxford Press.
Fraser, E.D., Dougill, A.J., Mabee, W.E., Reed, M., McAlpine, P., 2006. Bottom up and top
down: analysis of participatory processes for sustainability indicator identification as
a pathway to community empowerment and sustainable environmental management.
J. Environ. Manage. 78 (2), 114–127.
Freudenberg, M., 2003, “Composite Indicators of Country Performance: A Critical
Assessment”, OECD Science, Technology and Industry Working Papers, No. 2003/16.
Paris: OECD Publishing. Retrieved from: https://doi.org/10.1787/405566708255.
Gallup, The Gallup World Poll Retrieved from: https://www.gallup.com/analytics/
232838/world-poll.aspx 2018.
Gan, X., Fernandez, I.C., Guo, J., Wilson, M., Zhao, Y.g.B., Wu, J., 2017. When to use
what: methods for weighting and aggregating sustainability indicators. Ecol. Ind. 81,
491–502.
Goodland, R., 1995. The concept of environmental sustainability. Annu. Rev. Ecol. Syst.
26 (1), 1–24.
Goodland, R., Daly, H., 1996. Environmental sustainability: universal and non-negotiable.
Ecol. Appl. 6 (4), 1002–1017.
Graymore, M.L., Sipe, N.G., Rickson, R.E., 2008. Regional sustainability: how useful are
current tools of sustainability assessment at the regional scale?. Ecol. Econ. 67 (3),
362–372.
Graymore, M.L., Sipe, N.G., Rickson, R.E., 2010. Sustaining human carrying capacity: a
tool for regional sustainability assessment. Ecol. Econ. 69 (3), 459–468.
Guenno, G., Tizzi, S., 1998. The index of sustainable economic welfare (ISEW) for Italy.
Nota Di Lavoro 5, 98.
Gustavson, K.R., Lonergan, S.C., Ruitenbeek, H.J., 1999. Selection and modeling of sus-
tainable development indicators: a case study of the Fraser River Basin, British
Columbia. Ecol. Econ. 28 (1), 117–132.
Hak, T., Kovanda, J., Weinzettel, J., 2012. A method to assess the relevance of sustain-
ability indicators: application to the indicator set of the Czech Republic's Sustainable
Development Strategy. Ecol. Ind. 17, 46–57.
Hamilton, C., Saddler, H., 1997. The Genuine Progress Indicator. A new index of changes
in well-being in Australia. Retrieved from: http://www.tai.org.au/sites/defualt/files/
DP14_8.pdf.
Hamilton, K., Atkinson, G., Pearce, D., 1997. Genuine Savings as an Indicator of
Sustainability, CSERGE Working Paper, GEC 97-03, pp. 1–28.
Hanley, N., Moffatt, I., Faichney, R., Wilson, M., 1999. Measuring sustainability: a time
series of alternative indicators for Scotland. Ecol. Econ. 28 (1), 55–73.
Hardi, P., Zand, T., 1997. Assessing Sustainable Development: Principles in Practice.
International Institute of Sustainable Development, Winnipeg, Canada.
Harger, J.R.E., Meyer, F.M., 1996. Definition of indicators for environmentally sustain-
able development. Chemosphere 33 (9), 1749–1775.
Hediger, W., 2006. Weak and strong sustainability, environmental conservation and
economic growth. Nat. Resour. Model. 19 (3), 359–394.
Hermans, E., Van den Bossche, F., Wets, G., 2008. Combining road safety information in a
performance index. Accid. Anal. Prev. 40 (4), 1337–1344.
Hermans, F.L., Haarmann, W.M., Dagevos, J.F., 2011. Evaluation of stakeholder partici-
pation in monitoring regional sustainable development. Reg. Environ. Change 11 (4),
805–815.
Herrera-Ulloa, A.F., Charles, A.T., Lluch-Cota, S.E., Ramirez-Aguirre, H., Hernández-
Vásquez, S., Ortega-Rubio, A., 2003. A regional-scale sustainable development index:
the case of Baja California Sur, México. Int. J. Sustain. Dev. World Ecol. 10 (4),
353–360.
Jain, S., Aggarwal, P., Kumar, P., Singhal, S., Sharma, P., 2014. Identifying public pre-
ferences using multi-criteria decision making for assessing the shift of urban com-
muters from private to public transport: A case study of Delhi. Transport. Res. Part F:
Traffic Psychol. Behav. 24, 60–70.
JRC-EC, 2008. Handbook on Constructing Composite Indicators: Methodology and User
Guide. OECD publishing.
Kee, P., De Haan, M., 2004. Accounting for sustainable development. Statistics
Netherlands, Division of Macro-economic Statistics and Dissemination Development
and support Department. Centraal Bureau voor de Statistiek.
Kelly, R., Moles, R., 2002. The development of Local Agenda 21 in the mid-west region of
S. Kwatra, et al.
Ireland: a case study in interactive research and indicator development. J. Environ.
Plann. Manage. 45 (6), 889–912.
Kondyli, J., 2010. Measurement and evaluation of sustainable development: a composite
indicator for the islands of the North Aegean region Greece. Environ. Impact Assess.
Rev. 30 (6), 347–356.
Kori, E., Gondo, T., 2012. Environmental Sustainability: Reality, Fantasy or Fallacy?. 2nd
International Conference on Environment and BioScience, IPCBEE vol. 44, IACSIT
Press, Singapore, DOI: 10.7763/IPCBEE. 2012. V44. 22.
Krajnc, D., Glavi, P., 2005. A model for integrated assessment of sustainable development
Resources. Conserv. Recycl. 43 (2005), 189–208.
Labuschagne, C., Brent, A.C., Van Erck, R.P., 2005. Assessing the sustainability perfor-
mances of industries. J. Cleaner Prod. 13 (4), 373–385.
Lafortune, G., Fuller, G., Moreno, J., Schmidt-Traub, G., Kroll, C., 2018. SDG Index and
Dashboards—Detailed Methodological Paper. SDSN, New York (US).
Lee, Y.J., Huang, C.M., 2007. Sustainability index for Taipei. Environ. Impact Assess. Rev.
27 (6), 505–521.
Li, F., Liu, X., Hu, D., Wang, R., Yang, W., Li, D., Zhao, D., 2009. Measurement indicators
and an evaluation approach for assessing urban sustainable development: a case
study for China's Jining City. Landscape Urban Plann. 90 (3), 134–142.
Lundin, M., 2003. Indicators for Measuring the Sustainability of Urban Water Systems: A
Life Cycle Approach. Chalmers University of Technology.
Macrory, I., (2015). Sustainable Development Indicators: July 2015, Retrieved from
https://www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/articles/
sustainabledevelopmentindicators/2015-07-13#introduction.
Maier, H.R., 2013. What constitutes a good literature review and why does its quality
matter? Environ. Modell. Softw. 43, 3–4.
Mascarenhas, A., Coelho, P., Subtil, E., Ramos, T.B., 2010. The role of common local
indicators in regional sustainability assessment. Ecol. Ind. 10 (3), 646–656.
Mazziotta, M., Pareto, A., 2013. Methods for constructing composite indices: one for all or
all for one. Rivista Ital. Econ. Demografia Statis. 67 (2), 67–80.
McGillivray, M., 1991. The human development index: yet another redundant composite
development indicator? World Dev. 19 (10), 1461–1468.
Meadows, P., 2005. Towards developing regional sustainability indicators: lessons and
progress. In: Proceedings of the Fourteenth Conference of Commonwealth
Statisticians. Millennium +5: Managing Statistics for more Equitable Societies. Cape
Town, September 5–9.
Mebratu, D., 1998. Sustainability and sustainable development: historical and conceptual
review. Environ. Impact Assess. Rev. 18 (6), 493–520.
Mickwitz, P., Melanen, M., Rosenström, U., Seppälä, J., 2006. Regional eco-efficiency
indicators–a participatory approach. J. Cleaner Prod. 14 (18), 1603–1611.
Moffatt, I., 1996. Sustainable Development: Principles, Analysis and Policies. Parthenon
Publishing Group.
Mori, K., Christodoulou, A., 2012. Review of sustainability indices and indicators: to-
wards a new city sustainability index (CSI). Environ. Impact Assess. Rev. 32, 94–106.
https://doi.org/10.1016/j.eiar.2011.06.001.
Morse, S., 2013. Indices and Indicators in Development: AN Unhealthy Obsession With
Numbers. Routledge.
Munda, G., 2005. , Multi-Criteria Decision Analysis and Sustainable Development. In:
Figueira, J., Greco, S., Ehrgott, M. (Eds.), Multiple-Criteria Decision Analysis. State of
the Art Surveys. Springer International Series in Operations Research and
Management Science, New York, pp. 953–986.
Munda, G., Saisana, M., 2010. Methodological considerations on regional sustainability
assessment based on multicriteria and sensitivity analysis. Reg. Stud. 45 (02).
Munda, G., Nardo, M., 2007. Non-compensatory/Non-Linear composite indicators for
ranking countries: a defensible setting, Forthcoming. Appl. Econ.
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., 2005. Tools for composite indicators
building. pp 85. European Commission, Joint Research Centre, IPSC, Italy, EUR
21682 EN. Retrieved from : http://publications.jrc.ec.europa.eu/repository/
bitstream/JRC31473/EUR%2021682%20EN.pdf.
Ness, B., Urbel_Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools for sus-
tainability assessment. Ecol. Econ. 60, 498–508.
Neumayer, E., 2003. ). Weak Versus Strong Sustainability: Exploring the Limits of Two
Opposing Paradigms. Edward Elgar Publishing.
Nourry, M., 2008. Measuring sustainable development: some empirical evidence for
France from eight alternative indicators. Ecol. Econ. 67, 441–456.
OECD, 1998. Towards Sustainable Development: Environmental Indicators. Organisation
for Economic Co-operation and Development, Paris, France.
OECD, 2013. OECD Better Life Index Country Reports. Retrieved from: https://www.
oecd.org/newsroom/BLI2013-Country-Notes.pdf.
Panda, S., Chakraborty, M., Misra, S.K., 2016. Assessment of social sustainable devel-
opment in urban india by a composite index. Int. J. Sustain. Built Environ. 5,
435–450. https://doi.org/10.1016/j.ijsbe.2016.08.001.
Parris, T.M., Kates, R.W., 2003. Characterizing and measuring sustainable development.
Annu. Rev. Environ. Resour. 28 (1), 559–586.
Pearce, D.G., Turmer, R.K., Duborg, R., Atkinson, G., 1993. The conditions for sustainable
development. Blueprint 3: Measuring Sustainable Development. Earthscan,
London, UK.
Pearce, D.W., Markandya, A., Barbier, E.B., 1989. Blueprint for a Green Economy.
Earthscan, London, UK.
Pope, J., Annandale, D., Morrison-Saunders, A., 2004. Conceptualising sustainability as-
sessment. Environ. Impact Assess. Rev. 24 (6), 595–616.
Porter, M.E., Schwab, K., Sachs, J., 2004. The Global Competitiveness Report 2004–2005.
Palgrave MacMillan, New York, NY.
Poveda, C., Lipsett, M., 2013. Weighting sustainable development indicators (SDIs) for
surface mining operations using the analytical hierarchy process (AHP). Int. J. Anal.
Hierarchy Process. 5, 200–222.
14
Ecological Indicators 112 (2020) 106061
Quaddus, M.A., Siddique, M.A.B., 2001. Modelling sustainable development planning: a
multicriteria decision conferencing approach. Environ. Int. 27, 89–95.
Qureshi, M.E., Harrison, S.R., 2003. Application of the analytic hierarchy process to ri-
parian revegetation policy options. Small-Scale For. Econ. Manage. Policy 2 (3), 441.
Ramos, T.B., Caeiro, S., 2010. Meta-performance evaluation of sustainability indicators.
Ecol. Ind. 10 (2), 157–166.
Redclift, M., 2005. Sustainability: Critical Concepts in Social Sciences Vol. 8 Routledge,
Taylor and Francis group, London, UK.
Reed, M.S., Dougill, A.J., Baker, T.R., 2008. Participatory indicator development: what
can ecologists and local communities learn from each other. Ecol. Appl. 18 (5),
1253–1269.
Reed, M.S., Fraser, E.D., Dougill, A.J., 2006. An adaptive learning process for developing
and applying sustainability indicators with local communities. Ecol. Econ. 59 (4),
406–418.
Rees, W.E., Wackernagel, M., 1994. Ecological footprints and appropriated carrying ca-
pacity: measuring the natural capital requirements of the human economy. In:
Jansson, A.M., Hammer, M., Folke, C., Costanza, R. (Eds.), Investing in Natural
Capital: the Ecological Economics Approach to Sustainability. Island Press.,
Washington D.C, pp. 362–390.
Roberts, J.A., 2004. Environmental Policy. Routledge, London.
Saaty, T.L., 1980. The Analytic Hierarchy Process. McGraw-Hill, New York, USA.
Saisana, M., Saltelli, A., Tarantola, S., 2005. Uncertainty and sensitivity analysis techni-
ques as tools for the quality assessment of composite indicators. J. R. Statist. Soc. Ser.
A (Statist. Soc.) 168 (2), 307–323.
Sala, S., Ciuffo, B., Nijkamp, P., 2015. A systemic framework for sustainability assess-
ment. Ecol. Econ. 119, 314–325.
Salzman, J., 2003. Methodological choices encountered in the construction of composite
indices of economic and social well-being. Centre for the study of living standards.
Sardain, A., Tang, C., Potvin, C., 2016. Towards a dashboard of sustainability indicators
for Panama: a participatory approach. Ecol. Ind. 70, 545–556.
SASB, (2013). Conceptual Framework of the Sustainability Accounting Standards Board,
Sustainability Accounting Standards Board. Retrieved from https://www.sasb.org/
wp-content/uploads/2013/10/SASB-Conceptual-Framework-Final-Formatted-10-22-
13.pdf.
Sever, M., 2013. To what extent is the HDI a good indicator of the relative 'success' or
'failure'of national social policies? Acad. J. Interdiscipl. Stud. 2 (8), 208.
Sharpley, R., & Telfer, D. J. (Eds.). (2014). Tourism and development: concepts and issues
(Vol. 63). Channel View Publications.
Sherbinin, 2003. The Role of Sustainability Indicators as a Tool for Assessing Territorial
Environmental Competitiveness, Presented at the International Forum for Rural
Development 4–6 November. Hotel Grand Bittar, Brasilia, Brazil.
Simon, D., Arfvidsson, H., Anand, G., Bazaz, A., Fenna, G., Foster, K., Nyambuga, C.,
2016. Developing and testing the Urban Sustainable Development Goal’s targets and
indicators–a five-city study. Environ. Urbanization 28 (1), 49–63.
Singh, R.K., Murty, H.R., Gupta, S.K., Dikshit, A.K., 2012. An overview of sustainability
assessment methodologies. Ecol. Ind. 15 (1), 281–299.
Smeets, E., Weterings, R., 1999. Environmental Indicators: Typology and Overview.
European Environment Agency, Copenhagen, pp. 19.
South Pacific Applied Geoscience Commission. (2005). ‘Building resilience in SIDS. The
Environmental Vulnerability Index (EVI). Technical Report, SOPAC, Suva. Retrieved
from: http://islands.unep.ch/EVI%20Final%20Report%202005.pdf.
Spohn, O.M., 2004. Sustainable Development Indicators within the German Water
Industry: A Case Study. Chalmers tekniska högskola.
SS. (1998). Sustainable Seattle Indicator of Sustainable Community – A status report on
long-term cultural, economic, and environmental health for Seattle/King County,
Sustainable Seattle. Washington: USA.
Sutton, P. (2004). A perspective on environmental sustainability. Paper on the Victorian
Commissioner for Environmental Sustainability, 1–32.
Swanson, D., & Pintér, L. (2004). National strategies for sustainable development.
Challenges, approaches and innovations in strategic and co-ordinated action based on
a 19 country analysis. In National strategies for sustainable development. Challenges,
approaches and innovations in strategic and co-ordinated action based on a 19
country analysis. IISD.
UNDESA. (1992). Earth Summit Agenda 21 – The United Nations Programme of Action
from Rio. UN Department of Economic and Social Affairs.
UNDP. (2013). Human Development Report 2013, The Rise of the South: Human Progress
in a Diverse World, United Nations Development Programme. Retrieved from http://
hdrstats.undp.org/en/countries/profiles/IND.html.
UNHABITAT, 2001. The State of the World’s Cities 2001. United Nations for Human
Settlements, Nairobi, Kenya.
UN. Indicators of Sustainable Development: Guidelines and Methodologies Third Edition
2007 United Nation New York, USA Retrieved from http://www.un.org/esa/sustdev/
natlinfo/indicators/guidelines.pdf.
UNHABITAT (2013). State of the world's cities 2012/2013: Prosperity of cities.
Routledge. Retrieved from https://sustainabledevelopment.un.org/content/
documents/745habitat.pdf.
UNSDSN, 2016. New Index Shows Countries Need to Act Urgently to Achieve Sustainable
Development Goals. Retriev*ed from http://unsdsn.org/news/2016/07/20/new-
index-shows-countries-need-to-act-urgently-to-achieve-sustainable-development-
goals/.
Valentin, A., Spangenberg, J.H., 2000. A guide to community sustainability indicators.
Environ. Impact Assess. Rev. 20 (3), 381–392.
Wackernagel, M., Rees, W.E., 1997. Perceptual and structural barriers to investing in
natural capital: economics from an ecological footprint perspective. Ecol. Econ. 20
(1), 3–24.
Wood, R., Garnett, S., 2010. Regional sustainability in Northern Australia—A quantitative
S. Kwatra, et al.
assessment of social, economic and environmental impacts. Ecol. Econ. 69 (9),
1877–1882.
World Commission on Environment and Development, 1987. Our Common Future.
Oxford University Press, Oxford.
WWF, 2012. Living Planet Report 2012 Summary. World Wild Fund for Nature (WWF),
Switzerland.
Xiao, G., Xue, L., & Woetzel, J. (2010). The Urban Sustainability Index: a new tool for
measuring China’s cities. The Urban China Initiative-A joint initiative of Columbia
University, Tsinghua University, and McKinsey & Company. Retrieved from: http://
15
Ecological Indicators 112 (2020) 106061
www.urbanchinainitiative.org/wp-content/uploads/2012/04/2010-USI-Report.pdf.
Yeheyis, M., Hewage, K., Alam, M.S., Eskicioglu, C., Sadiq, R., 2013. An overview of
construction and demolition waste management in Canada: a lifecycle analysis ap-
proach to sustainability. Clean Technol. Environ. Policy 15 (1), 81–91.
Yu, L., Hou, X., Gao, M., Shi, P., 2010. Assessment of coastal zone sustainable develop-
ment: a case study of Yantai China. Ecol. Indic. 10 (6), 1218–1225.
Zhang, M., 2002. Measuring urban sustainability in China. Thela Thesis. the
Netherlands, Amsterdam.