02_IRAS69/3 articles 15/8/03 10:37 am Page 313

Evaluating the quality of public governance: indicators,

models and methodologies
Tony Bovaird and Elke Löffler

Abstract
This article provides an overview for this special issue on the evaluation of the quality
of public governance. It charts the move in the public sector during the 1990s from
concern largely with excellence in service delivery to a concern for good governance.
It examines what we mean by governance and ‘good governance’ and the dimensions
of ‘good public governance’. It demonstrates that there is now widespread interest in
measuring not only the quality of services but also improvements in quality of life and
improvements in governance processes. It discusses how measures of good
governance are being used in different contexts around the world. Finally, it considers
how the measurement of good governance can be encouraged, e.g. through awards,
inspections, setting funding conditions and empowering stakeholders to demand better
evidence.

Introduction
This article provides an overview for this symposium issue on the evaluation of
the quality of public governance. It sets out a conceptual framework within which
international developments in the measurement of public governance can be
understood.
The article begins by charting the move in the public sector during the 1990s
from concern largely with excellence in service delivery to a concern for good
governance. It examines what we mean by governance and ‘good governance’
and the dimensions of ‘good public governance’. It demonstrates that there is now
widespread interest in measuring not only the quality of services but also
improvements in quality of life and improvements in governance processes. It dis-
cusses how measures of good governance are being used in different contexts
around the world. It considers how the measurement of good governance can be
encouraged, e.g. through awards, inspections, setting funding conditions and
empowering stakeholders to demand better evidence. Finally, the authors present
an innovative approach to the evaluation of public governance which is currently
being piloted internationally.

Tony Bovaird is Professor of Strategy and Public Services Management, Bristol Business
School, University of the West of England, Bristol, UK and Elke Löffler is Chief
Executive, Governance International, UK. CDU: 65.012.3.
International Review of Administrative Sciences [0020–8523(200309)69:3]
Copyright © 2003 IIAS. SAGE Publications (London, Thousand Oaks, CA and New
Delhi), Vol. 69 (2003), 313–328; 036081
02_IRAS69/3 articles 15/8/03 10:37 am Page 314

314 International Review of Administrative Sciences 69(3)

From excellence in service delivery to good governance?

Excellence in public service delivery was one of the key themes of international
public sector reforms in the 1980s and 1990s. A large battery of approaches has
been built up to help governments with service improvement. Some of these are
legislative (Bovaird and Halachmi, 2001), e.g. the Government Performance
Results Act (GPRA) in the USA, Best Value in the UK or the Bassanini reforms
in Italy. However, others are essentially voluntaristic — e.g. use of the EFQM
(European Foundation for Quality Management) Excellence model and other
quality management systems.
There is no single view on what ‘quality’ means in this context but a number of
main approaches can be discerned:
• conformance to specification (ISO 9000, contract culture),
• fit for purpose (meeting corporate and social objectives, Balanced Score-
card I),
• aligning inputs, processes, outputs and outcomes (EFQM, Balanced Score-
card II),
• meeting or exceeding customer expectations (the Servqual instrument
decribed in Zeithaml et al. [1990]) and
• producing emotional, passionate commitment on the part of users (Pirsig,
1974).
However, governments now realize that, while excellent service delivery
remains important, it is not sufficient (Bovaird and Löffler, 2002). This is partly
because the application of these models of service excellence has thrown up the
existence of ‘wicked problems’ which do not lend themselves to solution simply
by service improvement in each of the agencies concerned.
For instance, in Germany, many public agencies have had to seek new
approaches to reducing local social welfare expenditure. Because of the effects of
the long-running economic recession in Germany, the ‘lean state’ reform move-
ment, which has introduced more flexible and results-oriented resource manage-
ment systems, has not been enough to contain expenditure within dwindling local
resource levels. Recently, schools in the City of Arnsberg have implemented an
innovative programme with associations of foreign residents to support foreign-
born children and their families in learning German. Local statistics have shown
that foreign-born children are much more likely to leave school without qualifica-
tions than German-born children, which means that they face great difficulty
entering the training system and in finding a job. By working with organizations
in civil society to improve the language skills of children of Turkish origin,
Arnsberg expects that not only will school results improve but also fewer of the
children will later become unemployed or social welfare recipients.
Moreover, it has emerged that even when citizens and other stakeholders
believe services are good, they do not necessarily express greater trust in the
governments concerned — and when services improve, trust does not necessarily
rise (Performance and Improvement Unit, 2002: 22).
02_IRAS69/3 articles 15/8/03 10:37 am Page 315

Bovaird and Löffler: Evaluating the quality of governance 315

As a consequence, it has become clear that a public organization cannot be

judged only on the excellence of its services — it also has to be excellent in the
way it exercises its political, environmental and social responsibilities. As this
has become clear, a new generation of government reforms has already started
which may be labelled ‘public governance’ reforms.

The move to public governance

After the decade of dominance by New Public Management (npm), i.e. the mid-
1980s to the mid-1990s, in which many initiatives were launched around the
world in accordance with its principles, it became evident that there was increas-
ing dissatisfaction with its limited focus. This dissatisfaction arose from several
sources:
• ‘Wicked problems’ were clearly not adequately solved by public sector orga-
nizations, even those which were ‘economical, efficient and effective’ — indeed,
such problems were not even solved when all public services were delivered at
high quality.
• Partnerships with the private sector were increasingly seen as more important
than relationships based on antagonistic contracting procedures. Indeed, success-
ful partnerships working with all organizations, in whichever sector they were
based, were increasingly seen as being reliant on trust.
• The role of citizens, as conceived in npm, was increasingly recognized to be
too thin and ‘consumerist’ — they were expected to be consulted as service users
but their role as members of communities which co-planned, co-designed and
co-managed public initiatives was largely ignored or undervalued.
• In most countries, the major scandals in relation to government performance
were often not about low service performance but about failures in the way
government carried out its tasks (both politicians and officials), for example, in
relation to the extent to which citizens and other stakeholders were fully informed
about the background to decisions made and in relation to the honesty and fairness
of politicians and staff.
• The long-term sustainability of economic, social and environmental policies
was seen to require an alignment of all strategies and policies — not only within
agencies but also between agencies and between sectors. This also meant that all
service decisions had to be evaluated against the priorities of the organization as
a whole and of the partnerships in which it was involved — no longer could
services be designed and delivered simply in order to achieve service-specific
objectives.

What we understand by ‘governance’

It is striking that there are many more definitions of ‘good governance’ than of
the underlying concepts of ‘governance’ and ‘public governance’. While social
scientists continue to argue about an appropriate definition of governance, how-
ever, practitioners have appeared much less concerned — e.g. in the case of npm,
02_IRAS69/3 articles 15/8/03 10:37 am Page 316

316 International Review of Administrative Sciences 69(3)

practitioners generally have ignored the terminology behind the concepts and
simply pushed forward with their practice.
We start from the belief that any definition of public governance has to be
context-specific and is likely, therefore, to differ between stakeholders and
between countries. However, for our purposes in this article, we will use a specific
definition of public governance which makes it particularly easy to explore
how different stakeholders can be involved appropriately in the evaluation of the
quality of public governance (Governance International, 2003). It will become
clear that other definitions of governance will entail some modifications of the
approach outlined here. Later in the article we discuss some examples of this.
We understand public governance to be

the ways in which stakeholders interact with each other in order to influence the out-
comes of public policies.

By ‘good governance’, we mean

the negotiation by all the stakeholders in an issue (or area) of improved public policy
outcomes and agreed governance principles, which are both implemented and regularly
evaluated by all stakeholders.

Measuring ‘good public governance’

At first sight the development of a conceptual framework for the measurement of
governance quality seems like attempting to ‘nail a pudding on the wall’. There
are few terms which are as vague in social science and in practice as ‘govern-
ance’. Yet, this vagueness may also be the source of its current popularity, as
different institutions and individuals all ascribe their own meaning to the term. In
particular, in international organizations the vagueness of the term ‘governance’
has helped to make it acceptable to a wide range of different countries and stake-
holders. However, when we want to measure the quality of public governance we
have to have some understanding of what governance comprises.
If we start from the contrast that excellence in the ‘new public management’
paradigm is about getting the instrument right (‘designing a good piano’), then
good governance is about ensuring that the outcomes are right (‘ensuring that
good pianos play good music’). Following this metaphor, governance evaluation
entails the assessment of the quality of the concert. But while most music lovers
would have no problems on agreeing what constitutes a good piano, it is clear
from the daily writings of music critics that opinions vary considerably about
what makes a good concert. And why should the quality of public governance be
measured? Do we not run the risk of trying to quantify what is not quantifiable?
We do not wish to give a definitive answer to these questions in this intro-
ductory article — indeed, we do not believe it is yet possible. Readers must judge
for themselves the cogency of the arguments presented by the various authors in
this symposium.
Nevertheless, we believe that the potential value of such an evaluation of the
02_IRAS69/3 articles 15/8/03 10:37 am Page 317

Bovaird and Löffler: Evaluating the quality of governance 317

quality of public governance is clear and that many stakeholders — and this
certainly includes the general public in most OECD countries — would like such
an evaluation to be undertaken and would like the results to be publicly available.
However, if this task is to be tackled, it will be important to get it right. In our
view, existing performance measurement approaches are entirely inadequate for
this task — it requires new conceptual frameworks, different assessment models
and methodologies and new sets of indicators.
The starting point is to explore the components of good governance. From our
definition of good governance it is clear that there are two key areas in which
measurement is required (Governance International, 2003):
• improvements in public policy outcomes; and
• implementation by all stakeholders of a set of principles and processes by
means of which appropriate public policies will be designed and put into practice.
We will consider each of these in turn.

Improvements in public policy outcomes

While during the first generation of public sector reforms in the 1980s and 1990s
a lot of lip service was paid to the need to measure outcomes, not much action
really took place on this front. (This even applied in Switzerland, where its varia-
tion of npm was actually called ‘outcome-oriented public management’.) This
measurement gap has been largely due to the fact that outcomes are often con-
tingent on factors outside the direct control of the agency responsible for deliver-
ing particular outputs. Rare attempts to tackle this measurement gap, such as the
Oregon benchmark exercise, showed that the measurement of outcomes requires
an inter-agency effort and inevitably involves data-sharing, which is often not
popular with individual public agencies.
Over recent years, however, there has been a strong resurgence in interest
around the assessment of policy outcomes. This new interest has partly been
driven by the insight that current performance reports which focus mainly on
output measures are of little interest to politicians, to citizens or to the media. As
‘civic literacy’ (Milner, 2002) increases, citizens and other stakeholders are
asking for information on government activities which is relevant to them.
Politicians, in particular, are asking for evidence of whether their policies have
made a difference in terms of achieving desired outcomes (Nutley and Boaz,
2003).
Consequently, there is now an interest in measuring the success of public inter-
ventions in terms of the quality of life changes which they bring about for those
affected by them, rather than the quality of the activities themselves. This means,
for example, that we need to assess:
• the level of security from external attack perceived by citizens, rather than
the quality of the national defence services;
• the level of community safety perceived by citizens, rather than the quality of
police and crime prevention services;
02_IRAS69/3 articles 15/8/03 10:37 am Page 318

318 International Review of Administrative Sciences 69(3)

• the level of income and conditions of working life, rather than the perform-
ance of economic development programmes;
• the level of health and social well-being experienced by citizens, rather than
the quality of health and social care given to service users;
• the level of understanding of citizens, in relation to the issues about which
they wish to know, and their level of competence in tasks which they wish to
undertake, rather than the quality of the education and training services which
they experience;
• the level of access which people enjoy to facilities they wish to experience,
rather than the quality of the roads or public transport services which they use;
• the quality of the environment which people experience rather than the
quality of environmental protection or improvement services;
• the comfort which people enjoy in their homes, rather than the quality of
house building and repair services which are provided; and
• the quality of the leisure and cultural experiences which citizens enjoy, rather
than the quality of the recreation, sports and arts services which are provided.
This does not mean that there is no longer an interest in or a need for measuring
the quality or quantity of services provided by the public sector. However, these
must be seen as purely instrumental, rather than as ends in themselves. As a con-
sequence of this change in orientation, there is now a major challenge to find ways
in which quality of life improvements such as those listed here can be assessed.

Measuring quality of life improvements

Measurement of quality of life performance indicators (qol pis) can be
approached from a number of different standpoints. In Table 1, we distinguish
four different types of qol pis:
• Type 1: overall indicators of the holistic quality of life of individuals (e.g.
percentage of respondents who report that they are ‘happy’ or ‘very satisfied with
the overall quality of their life’);
• Type 2: partial indicators of the holistic quality of life of individuals (e.g. per-
centage of respondents who are seeking to move to another country, region or
town);
• Type 3: overall indicators of the quality of life of individuals in one specific
dimension of their lives (e.g. percentage of respondents who report that they are
happy in their personal relationships or ‘very satisfied with the quality of their
working life’ or the quality of their health); and
• Type 4: partial indicators of the quality of life of individuals in one specific
dimension of their lives (e.g. percentage of respondents who wish to move in
order to improve their housing conditions).
Each of these approaches can be useful in specific circumstances. Clearly it
would be ideal if it were always possible to work with Type 1 qol pis. Conse-
quently, there is increasing international interest in such measures in order to
02_IRAS69/3 articles 15/8/03 10:37 am Page 319

Bovaird and Löffler: Evaluating the quality of governance 319

TABLE 1
Type of quality of life performance indicators (QOL PIs)

Nature of PI
General Specific
Coverage Holistic — all dimensions of qol Type 1 pis Type 2 pis
One or several dimensions of qol Type 3 pis Type 4 pis

produce international league tables such as the annual overall quality of life
survey undertaken by the consulting company Mercer (www.mercer.com/
globalcontent/employeemobility/quality.asp, as of 25 March 2003). However,
there will often be only limited information of this sort, so that it is necessary
to fall back on one of the other types of information. Type 2 qol pis can partly
supplement Type 1 qol pis but they are very heterogeneous by nature, so that it
can be difficult to aggregate them — for example, it is rarely possible to deduce
from several different Type 2 qol pis (such as desire to move to another town,
desire to find a new relationship) whether or not an individual is likely to be con-
tent with his/her overall quality of life.
The most common approach has generally been to use Type 3 or 4 qol pis.
These may provide overall or partial assessments of the quality of life but cover
only a limited number of specific perspectives. It has been particularly common to
develop qol pis from the standpoints of health, social care and the environment.
Examples include the following.
1. Objective improvements to quality of life of users and citizens:
• improvements in health status — overall PI,
• percentage of immobile elderly who have increased visits from mobile ser-
vice providers (e.g. hygiene assistants, chiropodists, library staff) — partial PI and
• percentage of citizens living within five minutes walk of a field, park or other
open space — partial PI.
2. Subjective improvements to quality of life of users and citizens:
• percentage feeling less pain than before — partial PI,
• percentage of frail elderly people who believe that their level of social inter-
action with family and friends is improving — overall PI and
• percentage believing that the quality of their overall environment is improv-
ing – overall PI.
3. Achievement of objectives/targets/standards in relation to outcomes:
• percentage of citizens with lower than World Health Organization recom-
mended levels of obesity — partial PI,
• percentage of people with disabilities who are capable of living independent-
ly (as defined in social care standards) — partial PI and
02_IRAS69/3 articles 15/8/03 10:37 am Page 320

320 International Review of Administrative Sciences 69(3)

• percentage of citizens living in areas where air pollution levels are below EU
maximum standards — partial PI.
While each of the indicators in this list needs careful interpretation (and, there-
fore, is capable of misinterpretation), they do suggest that many aspects of the
quality of life can now be more easily taken into account in decisions made in the
public domain. The challenge is no longer mainly technical, i.e. to demonstrate
that it can be done, rather it is political, i.e. to show that public stakeholders
actually care about demonstrating improvements to the quality of life, as shown
by such indicators.

International approaches to measuring quality of life improvements

In recent years, projects which have sought to measure the quality of life (and
changes to quality of life) have mushroomed, particularly at the local level. In
many cases, the first trigger for the development of qol indicators was the
need to respond to Agenda 21, following the Rio Earth Summit in 1992. As a
consequence, many early qol indicators tended to be strongly focused on
environmental and sustainability issues. In this section we note some of the most
interesting recent international qol projects.
Probably the best known ‘quality of life’ measurement project is the Human
Development Index (hdi) of the United Nations (United Nations, 2002). The
Human Development Report was first launched in 1990 with the simple but ambi-
tious goal of putting people back at the centre of the development process. This
goal had far-ranging implications — going beyond measuring income to assess-
ing the level of people’s long-term well-being. Since the first Report, four new
composite indices for human development have been developed — the hdi, the
Gender-related Development Index, the Gender Empowerment Measure and the
Human Poverty Index. The hdi, published annually by the UN, ranks nations
according to their citizens’ quality of life, rather than strictly by their economic
statistics. The criteria for calculating the rankings include life expectancy, educa-
tional attainment and adjusted real income.
The Compass Project of the Bertelsmann Foundation in Germany (www.
kompass-modellkommunen.de) supports the definition of qol indicators at the
local level with the involvement of key stakeholders. The key objective of this
project, involving the cities of Arnsberg, Celle, Coesfeld, Dortmund, Herford and
the county administrations of Osnabrück, Pinneberg and Soes, is to develop
‘strategic management’ within the local authority, which is defined as consisting
of four steps. First, the local authority has to develop a long-term external vision
and strategic objectives in consultation with citizens and other stakeholder
groups. Second, it has to build up a local qol indicator system which allows
comparisons between the pilots. Third, after primary data collection, including
comprehensive surveys, proposals for improvement are developed from the data.
Finally, the pilots need to evaluate the extent to which the proposals contribute to
the implementation of their overall strategy. Once a local authority has reached
02_IRAS69/3 articles 15/8/03 10:37 am Page 321

Bovaird and Löffler: Evaluating the quality of governance 321

the fourth phase, the strategic cycle starts again. It is planned to extend this project
to other Germany local authorities during 2003.
The Audit Commission in Britain, working with over 70 pilot local authorities,
has developed a portfolio of 38 pis which measure qol in 13 broad thematic areas
(Audit Commission, 2002). There are four pis in the economic field, four pis for
community involvement, 17 other social pis and 12 pis in the environmental field.
Of these 38 pis, 28 can be collected from existing ‘objective’ data sources but ten
pis require surveys of citizens — seven of which attempt to obtain subjective,
perceptual data and three of which are to collect objective data on behaviour.
(Such survey approaches are already common in the UK — citizens are already
surveyed every three years on how satisfied they are with their local council and
the services which it provides.) These qol pis have been produced in order to
encourage local authorities and their partners to consider qol issues in ways
which will benefit local communities. The government regards the use of these
pis as ‘recommended good practice’ in the monitoring of community strategies
but there is as yet no statutory requirement to do this.
The European Urban Audit was launched by the European Commission in
1997 ‘to enable an assessment of the state of individual EU cities and to provide
access to comparative information from other EU cities’ (http://europa.eu.int/
comm/regional_policy/urban2/urban/audit/src/intro.htm). Since then, the 58 pilot
cities have grown to a consortium of more than 200 cities in Europe. The pilot
started with a set of 33 qol pis on five themes including socioeconomic develop-
ment, civic involvement, levels of training and education, environment, culture
and recreation. These 33 qol indicators had to be refined and regrouped into 21
domains in order to allow comparable information to be collected by the pilots.
In the meantime, two large data volumes have been published which allows the
reader to see how a particular city compares with the other Urban Audit cities.
The results of the second Urban Audit will be released in early 2004 — however,
the list of variables has been changed considerably, which will make comparisons
over time difficult.
Although the qol measurement movement is still relatively young, it is possi-
ble to make a preliminary assessment:
• Even though information about qol indicators is primarily geared towards
improving the accountability of public agencies to citizens and community
groups, the way the information is presented is rarely target-group oriented.
Annual reports (which tend to become thicker and thicker every year), reporting
publicly about the qol in a local area or in a wider context, seem to be targeted
more at social scientists and enthusiastic statisticians than to average citizens
(or politicians). They rarely seem to be tailored to the needs of people or groups
interested in specific issues.
• There is little evidence so far that elected politicians make much use of qol
indicators. An initiative by the Canadian House of Commons to examine the
actual and potential impact of societal indicators and outcomes for use by parlia-
02_IRAS69/3 articles 15/8/03 10:37 am Page 322

322 International Review of Administrative Sciences 69(3)

mentarians suggested that one of the main problems is that tracking a society’s
progress is a long-term process, while parliamentarians work on comparatively
short time scales (Bennett et al., 2001).
• In the case of qol projects at the local level, the large bulk of data is still col-
lected by the local authority. Other local agencies, voluntary groups and citizens
are not much involved in the definition of indicators or data collection, which
makes it less likely that they will be interested in the qol reports which ensue.
• There is already a discernible trend to use qol indicators, where they are
available, not only for inter-area comparisons but also for inter-agency bench-
marking. There must be a major question as to whether this makes sense for qol
indicators. Given the differences between local areas and the different configura-
tions of agencies responsible for qol improvements, inter-agency comparisons on
qol indicators are likely to have relatively few benefits. Indeed, they are
likely to have significant disbenefits in driving the whole qol assessment exercise
into a game of proving ‘who is best’, with all the distortions this typically pro-
duces.

Measuring improvements against public governance principles and

processes
Whereas there is a lot of activity in measuring qol outcomes at local and other
levels of government, there are considerably fewer efforts to measure how well
governance principles have been observed in the public domain and how well
governance processes have been observed by the relevant organizations.
In recent years, there have been many attempts to categorize the main public
governance principles and processes (Kooimans, 1993; Rhodes, 1997). Here we
list ten of the characteristics of ‘good governance’ which have recurred frequent-
ly, both in the literature and in political and practitioner debates on the subject:
• citizen engagement,
• transparency,
• accountability,
• the equalities agenda and social inclusion (gender, ethnicity, age, religion,
etc.),
• ethical and honest behaviour,
• equity (fair procedures and due process),
• ability to compete in a global environment,
• ability to work effectively in partnership,
• sustainability and
• respect for the rule of law.
These principles and processes of public governance are not absolute — their
importance can be expected to vary between contexts and over time. Moreover, it
is likely that different stakeholders have differing views on what they mean in
practice. For example, citizens in Scandinavian countries are likely to have higher
02_IRAS69/3 articles 15/8/03 10:37 am Page 323

Bovaird and Löffler: Evaluating the quality of governance 323

expectations about the degree of transparency in relation to decisions made by

civil servants and elected officials than citizens in most other European countries.
This also applies in the international context, as Stuart Weir (Beetham and Weir,
2002: 85) argues:
It is central to the whole idea of a democratic audit that people within any given country
should want to undertake it for themselves, and should see it as a useful project at what-
ever stage of democratization they happen to be. And since they will be taking responsi-
bility for the judgements made, it is essential that they also acknowledge ownership of
the criteria and standards of assessment being used, and should find them appropriate.
At this point substantial differences of situation, experience and political perspective
come into play, which have significant implications for how the audit criteria are
formulated, and appropriate standards of assessment are chosen.

Consequently, the evaluation of the quality of governance principles and pro-

cesses must be stakeholder-specific.
For politicians and government officials, there is another important question
relating to governance: to what extent will strict conformity to governance princi-
ples and high quality achievement of governance processes serve to increase the
trust of stakeholders in government (and will this, in turn, increase the legitimacy
of government and its ability to mobilize resources and effect societal change)?
From the other end of the telescope, to what extent is a high level of trust between
different groups in society necessary for ‘good governance’ to become feasible?
These wider issues are examined in detail in Bouckaert and van de Walle (2003,
in this volume) and are not considered further here.

International approaches to measuring improvements in governance

processes
A number of international projects in recent years have sought to measure some
aspects of the quality of governance processes. Some have been at the national
level, others at regional or local level. Here, we note some of the most interesting
of these projects.
The most widely known governance index is the corruption perceptions index
(cpi) of Transparency International (www.transparency.org). The cpi is a poll of
polls, reflecting the perceptions of business people and country analysts, both
resident and non-resident. First launched in 1995, the cpi in 2002 drew upon 15
surveys between 2000 and 2002 from nine independent institutions. Although it
only included those countries that had featured in at least three surveys, it already
covered 102 countries. More than 70 countries scored less than five, as compared
to the ‘clean’ score of 10. Corruption was perceived to be particularly rampant in
Indonesia, Kenya, Angola, Madagascar, Paraguay, Nigeria and Bangladesh,
countries with a score of less than two. In addition, many countries in the former
Soviet Union remained ‘riddled with corruption’ but some of them — most
notably Slovenia, which had a cleaner score than EU member countries, Italy and
Greece — were perceived to be reducing the level of corruption.
At the other end of the scale, the countries which scored nine or more (i.e. very
02_IRAS69/3 articles 15/8/03 10:37 am Page 324

324 International Review of Administrative Sciences 69(3)

low levels of perceived corruption) were predominantly rich countries: Finland,

Denmark, New Zealand, Iceland, Singapore and Sweden. The UK ranked 10th
(with a score of 8.7) and the USA ranked 17th (with a score of 7.7).
The World Bank ‘second-generation’ governance indicators have emerged
from an initiative to test and develop policy-relevant, politically acceptable, quan-
titative indicators of governance (Knack et al., 2003). Previous work by the
Development Assistance Committee (DAC) on indicators of participatory dem-
ocracy and good governance were reviewed. The project considered potential gov-
ernance indicators, both those already in existence and new ones, in the light of
such criteria as the quality and coverage of data, the replicability and transparency
of the collection process, the level of ownership by and acceptability to govern-
ments, the specificity of the measure to the work of government, and with some
indicators, their relevance to particular development outcomes. Broadly speaking,
the indicators considered in the initiative either measure constraints on the execu-
tive related to accountability (e.g. up-to-date auditing processes, Freedom House
civil liberties score, literacy rate in the population) or else they measure the
administrative capabilities of the executive (e.g. budgetary volatility, average
government wage compared to private sector wage). As the project develops, it is
intended that it will analyse the relevance of second-generation indicators to pro-
poor development outcomes and then add additional indicators to the DAC list.
Democratic Audit is a loosely knit consortium of scholars, lawyers, journalists
and others, organized around the Human Rights Centre, University of Essex, and
the Centre for Democratization Studies, Leeds University (Beetham and Weir,
2002). It was initially established at Essex in 1992 by the Democracy Panel of the
Joseph Rowntree Charitable Trust. Recently, a number of its core members have
been commissioned by the Institute for Democracy and Electoral Assistance in
Stockholm to conduct a global audit of the state of democracy, involving eight
pilot studies in countries from five regions of the world. The UK Department for
International Development (DFID) has also commissioned members of Demo-
cratic Audit to revise its governance criteria, with a specific brief to revise the
assessments of democratic and human rights aspects of governance, to introduce
issues of service delivery and to consider the position of poor and disadvantaged
groups. The revised criteria were tested in Uganda during 2002 and were also used
in consultancy work on consolidating parliamentary government in Namibia and
Zimbabwe. There are four sets of criteria in the audit (Beetham and Weir, 2002):

• first, criteria on ‘free and fair elections’, including the procedural aspects of
electoral process and systems but going further to assess how far the system
enshrines equality between citizens in the value of their votes and their oppor-
tunity to stand for political office;
• second, criteria assessing the degree of openness, accountability and respon-
siveness of government (at different levels);
• third, criteria assessing the degree of protection for civil and political rights
and freedoms; and
02_IRAS69/3 articles 15/8/03 10:37 am Page 325

Bovaird and Löffler: Evaluating the quality of governance 325

• fourth, criteria on democratic society, assessing the vigour of associational

life, the accountability of economic institutions, the degree of media pluralism
and the extent to which citizens are publicly involved and tolerant of diversity.

Multiple stakeholder assessment

In the previous discussion on measuring improvements to the quality of life,
measuring conformity with governance principles and measuring improvements
to governance processes, we have stressed that all performance assessment in the
field of governance must be stakeholder-specific. This begs the question as to
which stakeholders should be involved in such assessments. Clearly, no absolute
list of stakeholders can be drawn up. In every context, different stakeholders will
be able to influence governance outcomes and governance processes. However, it
is obvious that governance evaluations should always be based on multiple
stakeholder assessments rather than self-assessments or external assessment by
some (expert) stakeholder who is not interested or does not care about a specific
governance issue.
One governance measurement application which operates on the basis of a
multiple stakeholder assessment is the GI Governance Health Check developed
by Governance International (www.govint.org). This assessment approach
enables organizations (and their key stakeholders) in the public and non-profit
sectors to evaluate the achievement of key governance principles and policies on
the basis of 360° assessment, based on focus group sessions with the following
key stakeholders:

• citizens,
• politicians representing the specific issue,
• third sector representatives, including community organizations, voluntary
organizations and charities working on the issue,
• business sector,
• the media (press, tv, radio),
• government departments and agencies (officials) and
• other levels of government with responsibilities for the issue.

Some groups fit rather uneasily into these categories — for example, trade
unions representing public sector staff would normally be classified as ‘third
sector’ but they will be viewed by many of their staff as just one more internal
interest group among the category ‘government departments and agencies’.
Again, community groups will often be closer to ‘citizens’ than to ‘third sector’.
Consequently, this classification should be regarded as indicative rather than cut-
and-dried.
In contrast to the other governance measurement projects described earlier,
the aim of the GI Governance Health Check is not only to provide a diagnosis but
to activate the involved stakeholders into taking action to contribute their share
to the improvement of identified governance weaknesses. In particular, the GI
02_IRAS69/3 articles 15/8/03 10:37 am Page 326

326 International Review of Administrative Sciences 69(3)

Governance Health Check is based on a systematic multi-stakeholder approach

and also includes the media, which are not part of other similar projects.

Encouraging the measurement of the quality of public governance

It is one thing to argue that improvements in the quality of life, conformity to
public governance principles and high quality achievement of public governance
processes are important. It is quite another to suggest that they could indeed be put
into practice. What incentives would there be for any stakeholders to become
involved in such an assessment process? After all, just because possible indicators
of achievement exist does not mean that a measurement process would be uni-
versally welcomed or conducted with integrity — and even if it were to take
place, and to be done well, it would not necessarily be used to improve decision-
making in public policies. In this section, we examine some of the options open to
different stakeholders in taking forward the assessment of the quality of public
governance.
The first issue is about incentives — ‘How might an interest in the assessment
of public governance be initiated’? By analogy with assessment of service
quality, it is possible to envisage a spectrum of triggers which might give stake-
holders an incentive to initiate the assessment of quality of public governance,
which would include the following:
• awards: e.g. CAPAM has already made its first awards for excellence in
public governance and the Speyer Quality Award (for German-speaking coun-
tries) already has a category for excellence in citizen engagement;
• inspections: e.g. in ‘the audit society’ of the UK, the Audit Commission
already carries out ‘corporate governance’ inspections of local authorities and
health organizations; and
• placing conditions on loans and grants: e.g. multi-national aid agencies, such
as the United Nations family of regional Development Banks, have already made
a practice of making aid to some countries contingent on improvements to some
aspects of public governance.
The second issue is about which stakeholders should take the lead — ‘Who
should assess public governance’? Traditionally, there has been a fundamental
choice here, between self-assessment and external (independent) assessment.
Self-assessment is usually more knowledgeable and allows those people to learn
the necessary lessons who must later play a key role in the improvement process.
However, it is also potentially myopic and self-deluding — and it is less likely to
be trusted by other stakeholders. Conversely, external assessment by ‘auditors’ or
‘inspectors’ is more likely to be independent but is often not trusted by the agency
— and it often exhibits limited understanding of the context, a tendency to be
simplistic or superficial — and it can be a very expensive process. Finally, there is
the possibility of 360° appraisal by relevant stakeholders — this is likely to be
independent and to exhibit a better understanding of the context and the core
issues (although this may vary between stakeholders). Moreover, its results are
02_IRAS69/3 articles 15/8/03 10:37 am Page 327

Bovaird and Löffler: Evaluating the quality of governance 327

more likely to be seen as important by the agency, given that its own stakeholders
have delivered the judgements — and the results can be embedded into the ongo-
ing learning relationship between the agency and its stakeholders. However, it is
clearly a more complex process, likely to take more time and to be more expen-
sive, unless designed to focus on priority issues.
Clearly, the triggers listed here are not necessarily likely to produce enough
pressure to fire enthusiasm for public governance assessment, particularly in a
360° multiple stakeholder process. However, there is reason to be optimistic that
these triggers might work on occasion, particularly where powerful stakeholders
are discontented with the judgements made about performance on the narrow
front of service delivery. It is then possible that the natural process of competitive
claiming might make it attractive for other organizations, or areas such as regions
or local districts, to join in the comparisons. In this way, the enthusiasm of the
pathfinders may be able to initiate a ‘snowball’ process whereby the most
advanced organizations undertake public governance assessments in order not to
be upstaged by their ‘rivals’ for prestige. At the same time, some organizations
which are conscious of their weaknesses but determined to strive for improve-
ment are likely to join in the game, even though it will result in their appearing in
a disadvantageous position in any ‘league table’ which emerges.
However, it must remain doubtful that this process would ever extend to those
organizations or areas where the results are likely to be embarrassingly bad and
where there is little stomach for the painful process of real improvement. Here, it
is more likely that only insistence by higher levels of government would ensure
that public governance assessment was undertaken.

Conclusions
There is now widespread international interest in measuring not only the quality
of services but also improvements in quality of life, both overall and in specific
dimensions (such as health, social well-being and environment). Moreover, there
have recently been some encouraging developments in the measurement of
improvements in governance processes.
However, the definition of ‘good governance’ is still very much a contested
area, so that measures of ‘good governance’ are used in widely different ways in
different contexts around the world. While this is not a problem in itself, there
would be real benefits from some standardization of approaches, particularly
where they made it possible for comparisons to be made. It would, therefore, be
valuable if the measurement of good governance could be encouraged on a wider
front, e.g. through international awards for ‘best practice’ in ‘good governance’.
However, more headway in this area is probably still contingent upon the
development and testing of more systematic approaches to the evaluation of the
quality of public governance. Only where this has already been undertaken is it
likely to be productive to move to more ‘punitive’ mechanisms for promoting
such evaluations, such as inspectorates (at national level) or the setting of funding
conditions for aid (at the international level).
02_IRAS69/3 articles 15/8/03 10:37 am Page 328

328 International Review of Administrative Sciences 69(3)

References
Audit Commission (2002) Quality of Life: Using Quality of Life Indicators. London: Audit
Commission.
Beetham, David and Weir, Stuart (2000) ‘Democratic Audit in Comparative Perspective’,
in Hans-Joachin Lanth, Gert Pickel and Christian Wegel (eds) Demokratiemessung.
Opladen: Westdeutscher Verlag, pp. 73–88.
Bennett, Carolyn, Lenihan, Donald, Williams, John and Young, William (2001) Measur-
ing the Quality of Life: The Use of Societal Outcomes by Parliamentarians. Ontario:
Centre for Collaborative Government.
Bouckaert, Geert and van de Walle, Stefan (2003) ‘Comparing Measures of Citizen Trust
and User Satisfaction as Indicators of “Good Governance”: Difficulties in Linking Trust
and Satisfaction Indicators’, in this volume.
Bovaird, Tony and Halachmi, Arie (2001) ‘Learning from International Approaches to
Best Value’, Policy and Politics 29(4): 451–63.
Bovaird, Tony and Löffler, Elke (2002) ‘Moving from Excellence Models of Local Service
Delivery to Benchmarking “Good Local Governance”’, International Review of
Administrative Sciences 68(1): 9–24.
Governance International (2003) URL: http//www.govint.org
Knack, S., Kugler, M. and Manning, N. (2003) ‘Second-generation Governance Indica-
tors’, in this issue.
Kooimans, Jan (1993) Modern Governance: New Government–Society Interactions.
London: Sage.
Milner, Henry (2002) Civic Literacy: How Informed Citizens Make Democracy Work.
Hanover, NH: University Press of New England.
Nutley, Sandra and Boaz, Annette (2003) ‘Evidence-based Policy and Practice’, in Tony
Bovaird and Elke Löffler (eds) Public Management and Governance. Routledge:
London.
Performance and Improvement Unit (2002) ‘Social Capital’, Discussion Paper. London:
Cabinet Office.
Pirsig, Robert (1974) Zen and the Art of Motorcycle Maintenance: An Inquiry into Values.
London: Vintage.
Rhodes, Rod (1997) Understanding Governance: Policy Networks, Governance, Reflex-
ivity and Accountability. Buckingham: Open University Press.
United Nations (2002) The Human Development Report. New York: United Nations.
Zeithaml, Valarie, Parasuraman, A. and Berry, Leonard (1990) Delivering Quality Service:
Balancing Customer Perceptions and Expectations. New York: Free Press.