PMRC Government Delivery Index (GDI) Series

A tool for tracking how Government is delivering on its development commitments

Unlocking Zambias potential in Governance | Service Delivery | Poverty Reduction | Social Justice and Equality | Economic Growth and Development | |Mining | Health | Education | Infrastructure| Agriculture

July 2013 1. GDI Series

Prepared by: Salim Kaunda, (Researcher) with the Support of Michelle Morel (Executive Director) Chileshe Chaunga (Research Intern), Isaac Dumbe (Communication Specialist), Brian Sambo Mwila (Communication Specialist), Agatha Siwale (Senior Researcher).

ABBREVIATIONS
GDI MDD GFI JASZ II KPIs APRM CPIA PF PEFA SNDP MTEF Government Development Index Management Development Division Grassroots Focus Index The Joint Assistance Strategy for Zambia II Key Performance Indicators African Peer Review Mechanism Country Policy and Institutional Assessment Patriotic Front Public Expenditure and Financial Administration Sixth National Development Plan Medium Term Expenditure Framework

2.

GDI Series

GOVERNMENT DELIVERY INDEX SERIES

Unlocking Zambias Potential through tracking

how Government is delivering on its development commitments

GDI Series

3.

CONTENTS

1. PMRC Background Note 1

The Need For A Government Delivery Index What do Government Indicators indicate?

2. PMRC Policy Brief 1

Learning From Governance Indices Towards a Government Delivery Measurement tool (Index)

15

3. PMRC Background Note 2

Drawing Lessons from Best Practice Towards Dimensions and Methodology for the Government Delivery Index (GDI) 4. PMRC Policy Brief 2 PMRC Government Delivery Index Series Learning From Renowned Governance Indices to Inform the GDI. Towards Methodological Options

21

30

5. Government Delivery Index (GDI)

PMRC Briefing Document Towards a Consolidated Monitoring and Evaluation (M&E) tool for Government

40

4.

GDI Series

EXECUTIVE DIRECTOR

ear fellow researchers, Policy Monitoring and Research Centre (PMRC), is a think tank committed to promoting public understanding through research and education, encouraging debate on social and economic policy issues critical to the growth of poverty reduction. The vision of PMRC is Unlocking Zambias Potential through timely, relevant evidence based policy analysis and reform proposals. Our aims include: To promote public understanding through research and education. To encourage and facilitate debate on social and economic policy issues critical to poverty reduction. To be a source of quality data relevant for stakeholders in areas of social and economic development To produce quality policy analysis and

To disseminate research based reform proposals. As part of the process of unlocking the resources available to Zambia, we at PMRC recognize that our greatest asset are the minds of the Zambian people. Our role is to engage all Zambians, and inform them of policy, policy changes and the consequences of policy decisions in their day-to-day lives. We look forward to engaging more with all Zambians in the coming period. Thank you for taking time to read this PMRC publication and please get involved and voice your opinions through PMRCs many engagement channels. Yours truly,

Michelle Morel GDI Series

5.

PMRC Background Note 1

THE NEED FOR A GOVERNMENT DELIVERY INDEX What do Government Indicators indicate?

ood governance is critical to long-term economic, social and environmental development. This Background Note outline the scope and functions of Government Indicators, showing exactly what they indicate. It explores the role of Key performance indicators (KPIs) and standards of good governance indicators. It then analyzes the structure of 3 identified Indices (Mo Ibrahim, Commitment to Development and Grassroots focused Index) and hence shows the importance of measuring government performance. This provides a background / best practice for evaluating government Key Performance indicators. The Background Note concludes by outlining the key components of good indicators and why it is necessary to develop a Government Delivery Index. There is an increasing demand from developing country governments, civil society organizations and donor agencies to measure different aspects of democracy, human rights and economic development. This demand has resulted in a tremendous growth in indicator frameworks and indices, which are used to measure the performance of governments delivery, the quality of public institutions, and peoples perceptions on various aspects of governance.

Governance
Governance is defined as the system of values, policies and institutions by which a society manages its economic, political and social affairs through interactions within and among the state, civil society and private sector. It comprises three (3) dimensions interdependent in society, namely, social, economic and political governance. The role of government is to provide a stable political, social and economic environment. Government policies throughout the world aim to promote fiscal responsibility; remove barriers to competition; ensure a legal framework for property rights and regulatory oversight; and ensure transparency of the law and policies.
6.
GDI Series

A question worth exploring therefore is: What are Governance Indicators and what do they really measure?

Indicators
To indicate is to point out something or to measure. Therefore, in simple terms, an indicator is a measure. Indicators are basically stipulated targets of objectives which give the basis of measurement.

Uses for indicators

The value of inductors to policy implementation, monitoring and evaluation is summarized as follows: Providing feedback on system behaviour and policy performance. Improving chances of successful adaptation. Ensuring movement toward common goals. Improving implementation and increasing accountability.

Governance Indicators
A governance indicator is a measure that points out the state of governance in a country. They are intended to inform and are usually narrowed to more specific areas of governance such as electoral systems, corruption, human rights, public service delivery, civil society, and gender equality. (e.g., Ministerial Sectors.) From a development perspective, governance indicators can be used for monitoring and evaluation of governance programmes and projects. These indicators are also often used to establish benchmarks, objectives, targets, and goals in the development context. Indicators, therefore, form targets or goals which the government puts on board as a basis of targeting achievements and how to track progress towards goals. In Zambia, Governance Indicators overall form an analysis of desired areas of performance. The next section explains in detail what Indicators really indicate and measure. This is done by first outlining the characteristics of good governance indicators, after which, Key Performance Indicators (KPIs) will be explained in detail. Lastly, an analysis of 3 Indices will be outlined, clearly showing how they measure performance and delivery.
7.

GDI Series

What are Government Indicators and What Do They Really Measure?

What are Key Performance Indicators?

Key performance indicators (KPIs) are factors or goals, against which the development, performance or position of the government/Institution can be measured effectively. An indicator does not necessarily have to come in numeric form. In the broadest sense, a KPI can be defined as providing the most important performance information that enables government and stakeholders to assess and understand whether the delivery/progress is on track or not, Therefore flagging the need for corrective action. KPIs must be specific, measurable, attainable, realistic and time sensitive (S.M.A.R.T). They usually vary from Ministry to Ministry and also across a detailed timeline. KPIs are so detailed and always must have Baseline Value and a target value. This is done so as to outline where the measurement is beginning from, and where the intended target lies. One of the challenges of selecting good indicators is that it is easier to choose indicators based on ease of measurement or data availability, rather than what needs to be measured.

Index
An index consists of multiple indicators combined into a composite or aggregated unit. While the development of indices is a complex task, indices have the potential to attract the decision maker and media attention. From the information outlined, we would now qualify the formulation of the Government Delivery Index (GDI), as an effective tool to measure the performance of the Zambian Government from the basis of its various stated aims and commitments. The Government Delivery index (GDI) will leverage from renowned indices such as the Mo Ibrahim Foundation Index of African Governance, the Commitment to Development Index (CDI) and the Grassroots Focus Index (GFI).

The Mo Ibrahim Index of African Governance

Established in 2007, the Mo Ibrahim index is the most comprehensive collection of quantifiable data that provides an annual assessment of government performance in any African Country. It is Africas leading assessment of governance. The index provides a framework and tools for citizens, public
8.
GDI Series

authorities and partners to assess progress in governance. It is compiled using 86 Indicators grouped into 14 sub-categories and 4 overarching categories, to measure the effective delivery of public goods and services to African Citizens.

The rationale is that the data gathered comes in different units and scales. Before they can be used in the Ibrahim Index, they are transformed onto a scale on which they can be meaningfully compared and averaged. Once the 86 indicators have been transformed to a common scale, each one is grouped with similar indicators to form 14 sub-categories.

Indices Analysed
1. The Mo Ibrahim Index of African Governance. 2. The Grassroots Focus Index 3. The Commitment to Development Index.

Figure 1: Structure of the Mo Ibrahim African Governance Index

Safety and Rule of the Law

Human Development

Mo Ibrahim Index
Sustainable economic opportunity

Participation and Human right

(Source:- The Mo Ibrahim Index_)(http:www.moibrahimfoundation.org)

The sub-category score is the simple average of all the indicator scores. Sub-categories are then grouped into one of four (4) categories, as shown in figure 1. The category score is the average of subcategory scores. The category scores are then averaged to produce the final Ibrahim Index score. Underlying indicators are combined in a standardized way to provide statistical measure of governance performance.

GDI Series

9.

The Grassroots Focus Index (GFI)

This Composite Index measures the grassroots perception of the impact of development programs. It measures the extent to which African governments and donor partners prioritize the grassroots (the local people) in policy making, policy implementation and performance monitoring. The GFI is constructed on the assumption that they are composite indicators that measure multidimensional concepts (as shown in Figure 2): of development and hence cannot be compared by a single indicator.
Figure 2: Structure of the Grassroots Focus Index

Resource Mobilisation, Management, Utilisation Government/ donor responsiveness Grassroots Influence

Grassroots Focus Index

(Source:- African Monitor Trust (2010), African Monitor, Cape Town)

Methodology- the index captures 21 indicators in its measurement. The composite indicators are based on 3 key pillars: Three best practice principles can be drawn from the GFI structure. These are: a solid theoretical framework; a sound process of construction; a good quality underlying data set;

The GFI has a set of 8 themes in its construction structure: -(a) Responsiveness;(b) Empowerment; (c) Political Governance; (d) Equity (e) Resource commitment and flows; and (f) Participatory budgeting and capacity to influence. The rationale of the GFI is to assess if the grassroots (local people) are considered in policy making, policy implementation and policy monitoring. This is done because the local citizens are the impact bearers of the government decisions and laws.
10.
GDI Series

The Commitment to Development Index

This is a composite index which combines metrics of Aid, Trade, Investment, Migration, Environment, Security and Technology. It however has a method of varying weights to the Indicators. Trade and investment are weighted higher while migration is weighted low. Figure 3 shows the policy areas of the commitment development index.
Figure 3: Policy Areas of the Commitment to Development Index.

Aid

Technology

Trade

Security

Commitment to Development Index

Investment

Environment

Migration

The CDI uses a range of model selection criteria instead of just one standard. The model in selection is not limited to one view of what complexity means or how it should be perceived. The rationale is that, literally, it is not possible to go into the field and measure the theoretical constructs directly in order to determine whether such models are an accurate representation of reality. Thus, critical measurement is necessary.

Lessons and Best Practices for Good Governance Indices

Based on the review of the three indices, several key lessons and best practices for a good indicator framework and indices can be drawn. The following are some of the lessons learnt and best practice. They all should be: Developed within an accepted conceptual framework;
GDI Series

11.

Clearly defined, easy to understand and interpret, and able to show trends over time; Scientific and credible, based on high quality data; Policy relevant; Relevant to users, politically acceptable and a basis for action; Responsive to changes in the environment and related human activities; Able to provide a basis for international comparison by providing a threshold or reference value; Subject to aggregation (from household to community, from community to nation.); Objective (be independent of the data collector); and Have reasonable data requirements;

CONCLUSION
Appropriate indicators and how they tie in to the formulation of indices are vital tools for measuring government performance. Critical analysis is needed in formulating and selecting the type or kind of indicator, as the findings from an index should influence policy formulation process and monitoring criteria.

12.

GDI Series

References and Other Resources for Backgroung Note 1

African Development Bank- Country Strategy paper, (2011) Booysen Friedrick- An Overview and Evaluation of Composite Indicators, (2002) Kaufman, Kraay and Mastuzi, Governance Indicators, (2004) GRZ _ Ministry of Finance and National Planning, (2010) GRZ _ Six National Development Programme (SNDP), Analytical report, (2011). David Roodman The Commitment to Development Index, Washington DC, Centre for Global Development (2011) The World Bank (2010), Global Policy Forum, United Nations Development Programme (UNDP), note for Human Development (2001) United Nations Development Programme (UNDP), Annual progress report, (2011) UNEP http://www.unep.org/ieap GRZ http://www.mofnp.gov.zm The Mo Ibrahim Index_http:www.moibrahimfoundation.org The Grassroots Focused Index_http:www.afdb.org/grassroots focused index. The Commitment to Development Index_http:www.cgdev.org/section/initiatives/_active/cdi

GDI Series

13.

PMRC POLICY BRIEF 1

LEARNING FROM GOVERNANCE INDICES "Towards a Government Delivery Measurement tool (Index)"

ambia does not have one consolidated and widely accepted tool for measuring Government delivery of development commitments and targets. There does exist a list of targets, Key Performance Indicators (KPI) at Ministry level, but KPIs do not measure government delivery on commitments. The brief builds and proposes a single, widely accepted composite index, the Government Delivery Index (GDI). The GDI will create awareness and drive Government to align its various strategies, track and timely review its performance. This Policy Brief builds the case for the Government to develop a standard tool (metric), which will be a guide for monitoring and evaluating performance and progress towards delivering on its commitments. Alignment of key strategic documents is an urgent requirement as it will enhance a harmonized way of monitoring Government delivery.

Summary
This Policy Brief argues the case for Government to develop the Government Delivery Index (GDI) as a performance monitoring, evaluation and learning tool, to support relevant policy and also enlighten the Zambian people on how the Government is delivering on its commitments.

Statement of the problem

Good Policies are pronounced and implemented; some ministries have methods for evaluating indicators but their lacks a defined, standard monitoring and evaluation tool or mechanism that streams across all sectors of government. An opportunity therefore exists to explore the Government Delivery Index (GDI), a standard tool metric for Government to self-assess and provide reasonable feedback to its citizens and stakeholders. This will also provide best practice for Zambia and possibly African Governments for fostering greater transparency and accountability in the development delivery among other gains. This would also upgrade Zambias ranking on Governance Indices such as the Mo Ibrahim
14.
GDI Series

and African Peer Review Mechanism (APRM). In this regard, a standard tool of measuring government performance and delivery is an urgent requirement.

Background
Zambians have high expectations of the government to deliver, based on the commitments of its Partys Manifesto. The Government needs to ensure that it constantly tracks and measures progress on delivery and how aligned it is to the Partys Manifesto commitments. Governments with enhanced measurement systems tend to be more effective and efficient in their delivery. There is confidence in the policies of government, yet it is still a long road ahead to the desired outcomes. This calls for strategising national statistical capacity activity. Notably, there is also an opportunity for Government to review the development of Key Performance Indicators, clearly, defined, detailed and feasible. These observations support the recommendation for Government to work towards a comprehensive Monitoring and Evaluation approach of government commitments.

Statement of the issue to the Government

Zambia abides by the African Peer Review Mechanism (APRM); a mutually agreed instrument voluntarily acceded to by Member states of the African Union (AU) as an African self-monitoring mechanism. The APRM process is based on a self-assessment questionnaire developed by the African Peer Review (APR) Secretariat. It is divided into four sections: democracy and political governance, economic governance and management, corporate governance, and socio-economic development. Its questions are designed to assess states compliance with a wide range of African and international human rights treaties and standards. This mechanism does not fully assist the Government to review or track its performance because it channels focus to the APR secretariat assessments rather than its own localized mechanism that fully covers the governance system in Zambia. This calls for a deliberate policy to be formulated that would allow a standard tool in form of the Government Delivery Index; to be tailored, localized and used in all government sectors against the stated Government Key Performance Indicators (KPIs). This would make office bearers (Ministers / PS) to focus on delivery, consistent to the Party manifestos commitments. The fundamental reason for developing performance measures is to help governments to focus on delivery and results and initiate timely corrective action when there is a mission drift.

GDI Series

15.

Figure 1. Sector Performance (Key Performance Indicators)

100 80 60 40 20 0
>7 5% M S i n ni e ci

a e p nc r Su e t a W

. nd

..

y pl

an

t re ce S en nt ltu alth nan gy e % ism AID m u 5 m rn / r er ic 0% -7 He ver ve iron % Tou HIV <6 Agr En o o 0 G 6 l G Env ca o L
No. of KPI Targets met

d.

..

Total No. KPI Targets Percentage of KPI Targets met

(Source:- (SNDP analytical report, July 2012))

Pre existing Policies

Efforts have been made to improving measuring performance and delivery in the public service, although its effectiveness and efficiency remains to be ascertained. The Country Policy and Institutional Assessment (CPIA) for sub-Sahara Africa was used to asses policies and institutions. The Public Expenditure and Financial Accountability (PEFA) was also developed as a contributing performance measurement framework. The Institutional Assessment and Organizational Development (IA/OD) approach are tactics which have been worked out as pilot programs in 3 ministries: (Education, Energy, Science and Vocational training). This approach is envisaged to fill in the gaps that were identified with the Public Service Reform Programme (PSRP). Another initiative was the Management Development Division (MDD), tied together with the IA/OD plan to provide for a comprehensive analysis of systems, using a variety of analytical tools and techniques. Public service reform programme, (2011, Lusaka)
16.
GDI Series

The approaches clearly havent been effective and thus the need for a very consolidated, detailed and comprehensive Government measurement tool or metric that would run through all sectors of Government. The Annual Sector Performance Analysis (ASPA) was also undertaken to feed into Government reviews and review performance for the previous year. This initiative also failed to fully measure Government delivery and also; a yearly review had short falls of mission drift and denied any steps of corrective action within the year. There is no formal domestication/ localization of the Regional agreements, the Paris declaration, the Accra Agenda and the African Peer Review Mechanism (APRM) to which Zambia is subscribed. Ministry of Finance and National Planning, Paris declaration evaluation, phase II (2011). This has contributed to the Government not to have a standard tool or mechanism; formulated to suit the Governance system in Zambia and help to track the Key Performance Indicators, in terms of delivery aligned to the Party Manifestos commitments.

Policy Recommendation and Procedures

1. Harmonise Key strategic documents: The Medium Term Expenditure Framework (MTEF), The Sixth National Development Plan (SNDP), Ministry Strategic plans and Joint Assistance Strategy for Zambia (JASZ) and the Yellow Book to the Patriotic Front Manifesto.
Figure.2 Alignment of Key Documents

Alignment and Harmonization must be done to improve developmental opportunities and alignment of developmental priorities for the donors and government. It will also promote co-funding,
GDI Series

17.

as there will be one clear focus of operation, defined sets of KPIs and also allow for standard monitoring of delivery due to aligned priorities. This satisfies the agreed development principles that where endorsed by the Accra Agenda and Paris declaration as follows; ownership, alignment, harmonization, mutual accountability, and management of development risks. 2. Develop a common assessment framework, Review existing Key Performance Indicators (KPIs) and the formulation criteria. This is because a framework to facilitate integration does not exist in Zambia. Also, the absence of specific institutions, mandated at promoting Sustainable Development and delivery at sectoral and national levels as well as lack of localized indicators for monitoring progress towards sustainable development and delivery. This would allow the Government to standardize the measurement, evaluation and assessment of all Sector KPIs and thus a clear track of performance will be developed. The Government Delivery Index would be the most effective and reliable metric to use across all sectors and help to measure delivery and performance. 3. Improve Ministerial budgetary allocations; have a human resource development strategy. Progress towards achieving Key Performance Indicators (KPIs) should be monitored over set periods and objective feedback and guidance provided with the view to correcting identified obstacles, where necessary through capacity building. This is especially so for the PF government where some public office bearers are either new to the role or would benefit from updating their skills. Also, an Assessment of the African Peer Review Mechanism (APRM) needs to be carried out so as to identify the existing gaps and as such add up to a standard mechanism, a Government Delivery Index.

CONCLUSION
This is a strong case for government to review and develop a comprehensive assessment of the Monitoring and Evaluating across all sectors. There is a case for urgent consideration of the Government Delivery Index, reviewing of existing Key Performance Indicators and harmonizing of the key national strategic documents. Budgetary allocations determine how many KPIs will be achieved and as such, needs to be increased for Key sectors especially. There is also an urgent need for the Government to develop a standard, comprehensive, widely accepted tool (Government Delivery Index) as a measure of government performance and delivery, in alignment to the Party manifestos commitments.

18.

GDI Series

References and other Resources

GRZ_ Ministry of Finance and National Planning, (MOFNP), Country evaluation of the implementation of the Paris Declaration in Zambia: phase II. February 2011 GRZ_ National Report Zambia, the United Nations Conference on Sustainable Development, June 2012 (RIO+20) Service, PSRP Bulletin, July- December 2011, Lusaka. The Government of the republic of Zambia GRZ, National Report for Zambia, June 2012 The Sixth National Development Plan, Annual Progress Report_ Analytical report, July 17, 2012 The Sixth National Development Plan (SNDP), Annual Progress Report, June 2012, Lusaka. The United Nations Development Programme (UNDP), Country Programme document for Zambia (2011-2015) The United Nations Development Programme (UNDP), Zambia Human Development Report, Service Delivery for sustainable Human Development, 2011. The United Nations Development Programme (UNDP), Assessment of development results-Zambia, 2012 http//web.undp.org.africa/programmedocs/CPD http://aprm-au.org http://web.worldank.org http://datatopics.worldbank.org/cpia http://aprm-au.org http://web.worldank.org http://datatopics.worldbank.org/cpia

GDI Series

19.

PMRC BACKGROUND NOTE 2

Drawing Lessons from Best Practice Towards Dimensions and Methodology for the Government Delivery Index (GDI)

overnance indices have several uses. They are used to; measure and guide budget process, improve service delivery (in terms of delivery modes and functions), enhance accountability and responsibility on the part of the office bearers. They introduce the discipline of relevant benchmarking, and improve the morale of Government officials and personnel, based on a set of standards, incentives, information and sanctions in the event of poor or non-delivery

This Background Note draws lessons for a methodology and process for Government Delivery Index (GDI) based on renowned Governance indices in Africa. It begins by providing a rationale for a single nationally accepted composite index to measure Government delivery. It further outlines sequenced steps for the GDI methodology and process. It concludes by recommending methodology and process options for developing the proposed GDI. Numerous government interventions have taken place towards ensuring delivery on its commitments, however, mechanisms to speed up the process must be in place. The Government Delivery Index, when constructed and implemented; will contribute to timely and relevant monitoring of government delivery commitments and progress, thus enhance transparency and accountability of policy implementation. The GDI will track to fill the gaps observed in the current Key Performance Indicators (KPIs) used by Government. The GDI will be a very useful monitoring and evaluation tool for Government delivery and progress.

The Process of Developing an Index

Lessons from best practice show that the overall process of conceptualizing and developing an index should be systematic, phased and reasonably sequenced.

The following steps are often critical and standard ones followed are:
20.

Conceptualization
GDI Series

Internal brainstorming and initial visioning of core dimensions, subdimensions, indicators and data sources Identifying external resource, high-level technical people with global experience in developing indices to serve on the technical advisory group (TAG- external and professional peer review mechanism) Engage potential TAG and establish the TAG Develop theoretical / conceptual (overall) framework Prototyping and refining ( organizing stakeholder consultative forum Organize and expert consultative meeting Develop and pilot the index Collect data, analyze, and produce the report

Figure 1.

3 Elements essential in constructing indicators for measuring performance of Government

Links Outputs to Costs

Ensures the attainment of organisational objectives/goals

Matches actual to planned Costs and tasks

Illustration, 1 Essential elements (source: PMRC)

GDI Series

21.

Identifying and selecting dimensions

Key Dimensions of an index must be identified, defined and established according to the objective of measurement. The aim is to encompass all relevant dimensions of sectors of the Government delivery areas. A thorough review of core Government delivery sectors and KPI is conducted. The next step breaks down the identified dimensions into Sub Categories or dimensions. Subsequently, a specified number of measurable (SMART) indicators, both essential and recommended are identified and selected.

Identifying and selecting essential Indicators

Indicators are identified through participatory and consultative approaches to enhance their acceptability, wider value and use. This involves partnering with other think-tanks, consultations with key institutions through a series of engagements based on the proposed concept of the Indices and methodology. The creation of partnerships, especially amongst national statistical agencies, NGOs, academic institutions and government, is critical for selecting indicators that are technically sound, operationally feasible and widely acceptable. Governance indicators selected must highlight the potentially different experiences of governance felt by vulnerable and minority groups, including women, youths and the poor in society. The selection criterion is often conducive to develop consensus, legitimacy and broad-based national ownership. Public and professional scrutiny is broadly essential for the credibility of governance indicators.

Towards a Methodology and Dimensions for the GDI

After indicator identification and analysis, the following questions must be answered. Do the dimensions of governance adequately capture performance in attaining the overall development and goal? Do the different result packages satisfactorily support each of the strategic objective?
22.
GDI Series

How well do performance Indicators/ measures relate to each of the result packages?

How do the alternative performances Indicators rank?

Measurement of certain sectors output in other areas like defense, regulatory activities and social welfare programs remain elusive. For each concern (per sector) key measurable variables or indicators must be drawn up. The indicators must be reliable, replicable, and simple to interpret in order to be easily understood by all users, especially the general public. The indicators should focus more on outputs, outcomes and impacts that is; the process of enriching human life rather than just the richness of the economy in which Zambians live.

Developing the Index

Data collection for national governance assessments (both primary and secondary) should consider two issues: 1. Data collection is undertaken in different phases. Data collected at national level through surveys is disaggregated along various lines, such as gender, income, age and region. Disaggregating indicators along these lines helps reveal differences across social groups and is crucial for formulating priorities, identifying target groups and designing specific policies and approaches to reduce inequities. 2. Integrating national capacity development into data collection activities. Using national researchers and Central Statistic Offices (CSO) as leaders on governance assessments may assist in securing sustainability, ownership and critical cultural and socio-political insights.

GDI Series

23.

The Grassroots Focused Index

Figure 2.

Resource Mobilisation, management, utilisation Sub Category Indicator 1. Access to public resources and market based benefits Equity 2. Economic opportunities and social services 1. Mechanism for resource flow to the grassroots 2. Forms of resource flow 1. Involvement in budget allocation

Resource Commitment and Flow

Participatory budgeting

Weighting
Weighting is a mathematical process through which a selected dimensions or component of the index is assigned value. Weightings determine the relative importance of each component of the index. Most composite indicators rely on equal weighting (EW), i.e. all variables are given the same weight. In this approach all components of the index are assigned equal value or weights. Moreover, if variables are grouped into dimensions and those are further aggregated (grouped) into the composite, then applying equal weighting to the variables may imply an unequal weighting of the dimension (the dimensions grouping the larger number of variables will have higher weight). This could result in an unbalanced structure in the composite index. Weights are chosen to reflect the statistical quality of the data. Higher weights could be assigned to statistically reliable data with broad coverage. Aggregation methods vary depending on the model and framework being used. The linear aggregation method is useful when all individual indicators have the same measurement unit, provided that some mathematical properties are respected. To ensure that weights remain a measure of importance, accurate aggregation methods should be use.
24.
GDI Series

Lesson from the Mo Ibrahim Index

It is essential to focus on indicators that are sensitive to policy changes, even if they measure only one aspect of human welfare. Initially, equal weights may be assigned to the different indicators, then subjected to validation and weighting during the technical and consultations sessions. The Ibrahim Index of African Governance is a composite index, constructed by combining underlying indicators in a standardised way to provide a statistical measure of governance performance in all African countries. As data included in the Index come from 23 separate institutions, and are on different scales at source, these raw data are standardised in order to be meaningfully combined. A simple statistical method of data aggregation is applied to the normalize and combine data into a composite index.. Each normalised indicator is assigned to the sub-category in which it best sits conceptually. Sub-category scores are calculated by averaging the scores of the underlying indicators. Category scores are calculated by averaging the 14 sub-category scores, and finally, the overall Index scores are obtained by averaging the scores of the four categories.

Structure: - The Mo Ibrahim Index

The Mo Ibrahim Index, to 4 dimensions, 14 sub categories and 86 (SMART) indicators
Figure 3.
Indicates

14 Sub categorise

4 Dimesnions

MO IBRAIM

(Source:- The Mo Ibrahim Index)

GDI Series

25.

Mo Ibrahim- Governance Dimension and Indicators

Mo Ibrahim Governance Dimension:Sustainable Economic Opportunity Indicators Investment climate Rural Financial services development Budget Management Reliability of financial institutions Telephone and IT infrastructure Agricultural input and produce markets Road and Rail networks.

Lessons from the Grassroots Focused Index

The key methodological contribution is to estimate the index from a Grassroots perspective. Indices are constructed on the assumption that they are composite indicators which measure multi-dimensional concepts that cannot be captured by a single indicator. Each of the indicators determining a component was given equal weighting. The number of indicators in each component influences that component. The number of components in each dimension influences the weighting. The range of the indicators and dimensions falls within the 0 and 100 range, A score of 50 indicates average focus, less than (<50) poor focus, and greater than (>50) above average focus. Data from both primary and secondary sources are ascribed values and then aggregated to compute the sub-indexes. After determining the relevant indicators, an arithmetic mean was calculated to derive the corresponding dimension index. The 3 pillars are essential for the final features and dimensions of an Index. They are the basis upon which a consolidated and widely accepted Governance Index is well formulated and designed. The GDI takes this approach towards a model that will characterise Monitoring and Evaluation of Government Delivery and performance in Zambia and beyond.
Figure 4. Solid Theoretical Framework Good Quality Underlying Data

A Sound Process of Construction

3 Pillars of good Composite Indicator

26.

GDI Series

Figure 5.

GOVERNMENT DELIVERY INDEX

DIMENSIONS DIMENSIONS

DIMENSIONS

SUB-CATEGOTRIES

INDICATIONS
Illustration:- Structure for the GDI.

The structure of the GDI will draw from best practice and identify Dimensions (that encompass government delivery). The dimensions will be broken down into Sub categories (to accommodate different dimensions of government sectors and structures. The indicators shall be drawn that represent the basis of measurement for each of the detailed sub categories identified. Weights shall be attached to the indicators and these will define measurement. Finally, a clear graphical illustration of the GDI will be designed and piloted based on the consolidated formulation base and inclusive criteria.

CONCLUSION
Learning from best practice, the GDI is established on 3 pillars, namely: (a) Solid theoretical/ conceptual framework (b) A solid Theoretical framework (c) Good quality underlying data. It is important that good technical and stakeholder collaboration effects in preparation for the GDI in data collection, weights, and building towards relevance. In the final analysis, a collaborative agreement session involving key national government sectors, the technical groups, NGOs, and local academic institutions is most appropriate. The brainstorming sessions allow the institutions to capitalize on each other's strengths, thus presenting a consolidated, widely accepted Government Delivery Index.
GDI Series

27.

References and Other Resources

Anna Rita, Massimiliano Mascheniri, (2012) Towards a set of composite indicators on flexicuriy; A comprehensive approach, European Commission, 2012 Italy Daniel Kauufmann, Aart Krayy, (2004) Governance Indicators, World Bank, 2004, New York Kazi Iqbal and Anwar Shar (2008) How do worldwide Governance Indices measure up. World Bank, 2011, New York OECD, (2008) Handbook on constructing composite indicators Methodology and user guide, European Commission, 2008, Ispra Rosario G Manasa et al (2000) Developing an index of Governance quality, Philippine Institute for Development Studies, 2000, Manilla UNDESA (2007) Public Governance Indicators, A literature review, 2007, UNDESA, New York UN HABITAT (2009) Urban Governance Index; A tool to measure progress in achieving good urban governance, UN 2009, New York The Mo Ibrahim Index http:www.moibrahimfoundation.org The Grassroots Focused Index http:www.afdb.org/grassroots focused index. The Commitment to Development Index http:www.cgdev.org/section/initiatives/_active/cdi

28.

GDI Series

PMRC POLICY BRIEF 2

PMRC GOVERNMENT DELIVERY INDEX SERIES Learning From Renowned Governance Indices to Inform the GDI.Towards Methodological Options

KEY MESSAGES:

ultiple unaligned Monitoring and Evaluation (M&E) frameworks in Zambia have led to ineffective evaluation, tracking and reporting of Government delivery efforts.

Government and Cooperating Partners (CPs) use different guiding frameworks, basing different projects, programmes and Key Performance Indicators (KPIs).This affects the ability to monitor and track progress made by the government. The Sixth National Development Plan (SNDP) Medium Term Expenditure Framework (MTEF), the Vision 2030, the Yellow Book, Joint Assistant Strategy for Zambia II (JASZ II) and the PF Manifesto currently guide government delivery in Zambia but they are clearly not in alignment and harmony The question is; what document framework is Guiding Government Office bearers? An opportunity exists to utilize a harmonised Plan, as a basis for monitoring government performance. Better decision-making within government is enhanced as a result of monitoring public office bearers. It provides a forum for purposeful and concrete engagement between the executive, the legislatures and civil society around critical choices and outcomes. The Government Delivery Index (GDI) will help track and contribute to consensus based comprehensive monitoring feedback on delivery, and provide Government with the necessary feedback.

Summary
This Policy brief is towards methodological options and technical recommendations for the Government Delivery Index (GDI). Developing the index will draw simple methodological options and easy computation of weights. GDI will be based on assessment areas (commitments) from the PF
GDI Series

29.

governments manifesto. Technical recommendations towards filling policy gaps are then drawn. The GDI is being developed as a delivery monitoring, and progress reporting tool. Citizens have a right to know what the Government is doing in terms of delivery and what decisions the public office bearers make on their behalf towards development objectives and targets. The assessment findings of the SNDP analytical report, supports urgent need for a comprehensive standard M&E tool. For example, by the end of June 2012, the Government had released 76.6% of the Budget allocation with most of the ministries and spending agencies having received above 50% of their allocations, but many ministries did not meet KPI targets. Two questions arise; (1) are we really holding office bearers accountable? (2) Do SNDP KPIs really capture performance targets? It is paramount that a standard and cross-cutting M&E tool is formulated. This will track resource allocation, delivery, projects and policy performance, including support of results-based management. As a standard composite index, GDI will inform the Zambians and other stakeholders on how government is delivering on development commitments it made.

Statement of the Problem

Zambia does not have a single, consolidated M&E tool. The use of multiple unaligned M&E frameworks which focus clearly on input and output assessment, leads to information variations and unclear reporting of Government progress. Bearing that some sectors KPIs take long to measure, the GDI tracks progress using 3 dimensions, of progress, output and outcome. Citizens want to know how money is being utilized and this is a case of transparency and accountability for Government and public office bearers. The major challenge is that framework documents are not aligned and this has led to information variances and inconsistencies in monitoring government performance. The Government is currently formulating the Government wide M&E framework; focusing on input-output analysis however, the PMRC GDI leverages from this initiative and goes beyond to track the whole results chain from input, process, activities, outputs and outcomes. The GDI will move beyond outcomes and assess the impact of the outcomes, to the citizens. The GDI composite tool will thus be an ideal M&E tool and reveal actual progress of government delivery.

Statement of the Problem to Government

Government has appealed to members of the public to take up the civic duty of monitoring programmes financed and implemented using tax payers money. Mr. Chileshe Kandeta (Ministry of Finance Public Relations officer) Zambia Daily Mail 4th October 2012. Governance is not just about how a Government and social organizations interact, and how they relate to citizens. It concerns the States ability to serve
30.
GDI Series

citizens and proper allocation of resources. Policy inclusions must foster citizens participation in decision-making at the national, regional and local levels. Legitimacy goes hand in hand with participation. The GDI, by going beyond input-output analysis, will direct government delivery to enrichment of the citizens. It will provide a sound basis for painting a comprehensive Government delivery picture and timely evidence-based feedback. This initiative recognizes Governments efforts towards developing a Government - Wide M&E framework based on select Key Performance Indicators (KPI). The value addition of GDI is its guide; where Government leadership will be adequately be consulted and engaged in identifying and ranking delivery priorities, consistent with their Party Manifesto and broader long-term vision.

Pre-existing Polices
Within Government, differences exist between the Sixth National Development Plan (SNDP), Medium Term-Expenditure Framework (MTEF), Public Expenditure and Financial Administration (PEFA) and each individual ministrys strategic plans. Individual ministries each have tailored strategic implementation plan.
Figure 1. Zambia Road to Development

Sixth National Development Plan Vision 2030 Patriotic Front maniesto

Joint Assistance Strategy for Zambia Millennium Development Goals Medium Term Expenditure Framework

Zambia's Guiding documents; one focal point but, are they in one alignment?

ZAMBIA ROAD TO DEVELOPMENT

(Source:- Policy Monitoring and Research Centre, 2012)

GDI Series

31.

The multiple M&E frameworks in Zambia have led to ineffective tracking, evaluation and reporting of Government delivery efforts in various sectors. The SNDP does not have an implementation plan. This has led to inconsistencies, as government does not have a standard way of monitoring and tracking progress being made under each ministry. This causes significant differences in data between Government and the Co-operating partners. Government is currently formulating the Government wide M&E framework. This mechanism focuses on input-output analysis in measuring Government delivery. The GDI goes beyond this notion and is concerned on measuring the impact of the government outcome deliverables and measuring the impact thereof, to the intended target groups.

Learning from the Renowned Governance Indices

The Mo Ibrahim Index of African Governance is a composite index, constructed by combining underlying indicators in a standardized way to provide a statistical measure of governance performance in all African countries. The Mo Ibrahim Index was launched in 2007 and has evolved to be the most comprehensive, quantitative assessment of African governance available. A sub-set of the Advisory Council sits on a Technical Committee which, along with the Index Research Team, is responsible for the methodological and statistical robustness of the Index. It can be summarized by four main governance concepts, which form constituent categories; (a) Safety & Rule of Law (b) Participation & Human Rights (c) Sustainable Economic Opportunity (d) Human Development. These four categories are themselves composed of 14 sub-categories. The four Index categories are populated with 86 indicators1. Data utilized in the construction of the Ibrahim Index come from 23 separate institutions and are often produced on different scales at source. Before being included in the Index it is necessary that these raw data are standardized in order to be meaningfully combined.

Data collection, standardization and construction of the Mo Ibrahim

1. The data gathered comes in different units and scales. Before they can be used in the Ibrahim Index, they are transformed onto a scale on which they can be meaningfully compared and averaged. 2. Once the 86 indicators have been transformed to a common scale, each one is grouped with similar indicators to form 14 sub-categories. The subcategory score is the simple average of all the indicator scores. 3. Sub-categories are then grouped into one of four categories; the category score is the average of subcategory scores. 4. The category scores are then averaged to produce the final Ibrahim Index score

32.

GDI Series

Process of data collection, standardization and computation of Grassroots Focused Index (GFI)
Guiding questions for the process of consultations, data collection and weighting To what extent have African governments and donors been able to prioritize Grassroots in development policy priorities and strategies? To what extent (where such prioritization has occurred) has prioritization led to effective channelling of resources to Grassroots development by African governments and donors? To what extent are Grassroots able to meaningfully influence government and donor policies and programs, including resource commitments and flows?

Since the GFI takes Grassroots as the starting point, primary research through the concept mapping methodology was used to engage with Grassroots communities and collect pilot data. The results of the concept mapping exercise were used to construct the components of the GFI, as well as to develop the rapid appraisal instrument that was used to conduct research. Two rapid appraisal instruments were designed one for community respondents and the other for development actors. The data collected from the rapid appraisal exercises was then coded and used to construct the GFI pilot results. The range of the indicators and dimensions falls within the 0 and 100 range, and all indicators were normalized in this way. A score of 50 indicates average focus, <50 poor focus, and >50 above average focus The overall GFI score is obtained by adding the average means of the three dimensions. Based on the lessons from the two indices, the starting point for the GDI concept is the commitments in the PF Manifesto (as basis for measurement) and how these guide Governments policy. The GDI focuses on how the Government is delivering on its Manifesto development commitments.
Figure 2. GFI

GFI

1/3
Government and Donor Responsibility Index

1/3
Resource Mobilisation management and utilization

1/3
Grassroots influence index

(Source:- African Monitor, 2011 Grassroots Focused Index)

GDI Series

33.

The GDI methodology will engage a Technical Advisory Group (TAG) and various stakeholders to enhance its wider appeal and standard. The conceptualisation, data collection and formulation process will obey the following steps: Identify potential members to the GDI Technical Advisory Group (TAG) A gap analysis of the PF manifesto commitments to assess feasibility Planning and conducting a GDI stakeholder consultative workshop Conducting the expert consultative meeting Engage the sector Minister, Permanent secretary or planning officer Pilot the GDI in one ministry Attaching of weights and standardization process Computing the final GDI score Report the GDI score in the respective sector.

Methodological Options:-the GDI proposes the following steps

The visioning of the structure, key components, sub-dimensions and key performance indicators of the GDI will be based on key Government delivery commitments as articulated in the PF manifesto 2011 - 2016. 1. The first step will be to conduct a gap analysis of the PF manifesto commitments in order to assess the political, economic and social feasibility of the commitments based on reasonable sequencing and time frames. This will help PMRC to align its monitoring to realistically sequenced and phased implementation of manifesto and Government delivery promises. 2. The second step will involve planning and conducting a GDI stakeholder consultative workshop. This meeting will target key stakeholder segments who are potential users of data and beneficiaries as follows: Government departments, especially the policy planning, implementation and M&E units Multilateral and bilateral development institutions Government, Local Government and parastatals and agencies

Civil society organisations( research think-tanks, NGOs, Private sector interest organisations, and community-based organisations

34.

GDI Series

Central Statistics Office, as the Government-wide data and information repository Academia ( Statisticians)

3. The third step will be planning and holding an expert consultative meeting, for consultations and sequenced phase projects events. This will involve engagements with the identified Technical Advisory Group (TAG). This will be the advisory team to the PMRC GDI project team. This group will compose renowned professionals who are experienced with the formulation of governance indices. GDI data collection and computation. The GDI data collection phase will engage; the Sector Minister, Permanent secretary/ planning officer, stakeholders, consultations, government and report publications, sector institutions. The computation will be based on adding sub totals of the 3 dimensions weights (validation) Release and dissemination of data; after the computation and analysis of final GDI score, a graphical illustration will be used to represent the findings.

GDI Methodological processes

The GDI will comprise 3 dimensions (Process, output and outcome): This will capture all relevant information needed to fully track government delivery. This takes into account the time lap and various activities that are required before a commitment can be met. An example can be drawn from Health sector in the Manifesto. One of the commitments is, To establish Intensive Care Units that will be fully equipped and staffed for emergencies and specialized cases to drastically reduce the need for referring patients abroad. The GDI, by using 3 dimensions will be able to track the commitment in a step-by-step format and evaluate it based on progress made from any of the dimensions. This will give a true picture of the state of delivery in relation to all assessment areas per sector. The GDI will capture the following information from government; as a way of compiling actual evaluations of government delivery. These will be collected and compiled for every commitment in the Manifesto; narrowed to 3 dimensions (Process, output, and outcome). The data collection will involve (for every Ministry) the Minister, The Permanent Secretary and the Planning officer. Inputs: capturing all the resources that contribute to the production of service delivery outputs. They include finances, human resource, equipment and infrastructure. Activities: will capture the processes or actions that use a range of inputs to produce the desired outputs and ultimately outcomes. In essence, activities describe what is being done.
GDI Series

35.

Outputs: the final products, goods and services produced for delivery. Outputs will be defined as what has been produced. Outcomes: the medium-term results for specific beneficiaries which are the consequence of achieving specific outputs. Outcomes should relate clearly to ministries strategic goals and objectives set out in its plans. Impacts: the results of achieving specific outcomes, such as; reducing poverty and creating jobs. Impacts will capture how the results have influenced communities, general citizenry and target groups.

The Key GDI Guiding Questions

1. How is Government delivering on its Manifesto commitments? 2. To what extent are the PF Manifesto core commitments aligned to and inform Government policies, strategies and programmes? 3. Is quality data to support effective monitoring of progress on Government commitment delivery available and accessible? 4. What stakeholders engagement and feedback mechanisms exist to promote wider public monitoring and understanding of governments delivery goals and targets?

Activity and Process Sequencing

The strategy is to review one ministry per time. The commitments (assessment areas) will be ranked and weightings will be attached to each assessment area based on its overall importance/ rating towards achieving the total objective in that sector. The 3 dimensions for each assessment must add up to 10. The measurement scale is therefore 1-10. For each assessment area, weights will be attached for process, output and outcome. The total GDI score will be computed by adding the sub totals and computing the average. The Minister or Permanent Secretary will be engaged to determine what is being done for every assessment area. After weights have been attached, the GDI will be subjected to a team of experts, to validate and analyse the scores. This will pass through a stakeholders consultative meeting and general public for more views and reviews For every sector, the Manifesto commitments will be clearly outlined (Assessment areas) and each subjected input process, output result and final outcome. The Indicators, for every commitment will be the intended target and outcome measure planned by the Government.
36.
GDI Series

Figure 3. GDI Assesement

GDI ASSESEMENT - PER SECTOR

Assessment Area
Commitment 1 Commitment 2 Commitment 3 Commitment 4 Commitment 5 Commitment 6 Commitment 7 Commitment 8 Commitment 9 Commitment 10 Commitment 11 Commitment 12 TOTAL SCORE

DIMENSIONS

SCORE
OUT OF 10 Total/10

WEIGHTINGS

PROCESS Weight

OUTPUT Weight

OUTCOME Weight

TOTAL

TOTAL

TOTAL

TOTAL

(Source:-Policy Monitoring and Research Centre, October 2012)

Weighting for the GDI will be established after consultations and engagement with the Minister, permanent secretary and technical advisory group. Different indicators will be attached various weights, based on their rating and overall relevance to achieving the delivery goal. The 3 dimensions weights should add up to 10 and the final GDI score will be computed by adding the sub totals and averaging the values.

CONCLUSION
The GDI provides an integrated, standard M&E tool; with principles, practices and standards to be used throughout Government, as a consolidated monitoring and progress tracking tool. Governments major challenge is to become more effective. GDI will assist the public sector in tracking and evaluating its performance and identifying the factors which contribute to its service delivery outcomes. The index is uniquely oriented towards providing users with the ability to draw causal connections between the choice of policy priorities, the resourcing of those policy objectives, the programmes designed to implement them, the services actually delivered and their ultimate impact on communities. The tool will assist to provide an evidence base for public resource allocation decisions and help identify how challenges should be addressed and successes replicated. The case therefore is for the government to adhere to the methodological options outlined in this policy brief, as we build towards a
GDI Series

37.

References and other Resources

African Monitor (2009) Grassroots Focused Index: Towards a more Prioritised and Grassroots focused Development. Briefing Document, Cape Town. European Commission, (2007) Tools for Composite Indicators Building, Joint Research Centre, Spar (Italy) GRZ (2011) The Monthly, Volume 94, January 2011. Lusaka, CSO GRZ (2012) The Monthly, Volume 106, January 2012, .Lusaka, CSO Kaufman, Kraay and Mastuzi, (2004) Governance Indicators, 2004, India Mo Ibrahim Foundation (2010) Technical Document for the Mo Ibrahim Index, Mo Ibrahim (2010) London PMRC, (2012) GDI Briefing Document, Policy Monitoring and Research Centre, Lusaka The Presidency, (2007) Policy Framework for the Government wide Monitoring and Evaluation System. Presidency Publishing , Pretoria GRZ_The Sixth National Development Plan (SNDP), Annual Progress Report, June 2012, Lusaka

38.

GDI Series

GOVERNMENT DELIVERY INDEX (GDI)

PMRC Briefing Document Towards a Consolidated Monitoring and Evaluation (M&E) tool for Government

1.0 Introduction and background

efore the 2011 Parliamentary and General Elections, the Patriotic Front (PF) Party made several commitments, which are outlined in its Party Manifesto 2011-2016. Now, as the Party in Government Zambians expect it to timely and adequately deliver its development commitments. Numerous questions are being asked about how the PF is translating its commitments to tangible policy, programmatic and regulatory reform initiatives to influence Government-wide implementation and achievement of objective targets. In order to regularly obtain feedback on the progress being made towards achieving PF Governments goals and objectives, and timely communicate the achievements, a sound, single, consolidated and widely accepted Monitoring and Evaluation (M&E) tool should be utilized. The focus of every process of development implementation and delivery is allowing timely reflection of critical questions in order to take timely corrective action in the event of a mission-drift: Are we taking the action we said we would take? Are we making progress on achieving the results we said we wanted to achieve? Are the results relevant to the intended target group? What changes, if any, are intended to attribute our actions to? Can these results last long enough to have last effects on the changes made? How do we document these achievements? How do intended beneficiary and stakeholder segments know when progress is made and results achieved?

The standard tool should have wider value and appeal and ensure effective and transparent communication flow of progress and achievements of Government on its manifesto and overall national strategy objectives. This is the basis of
GDI Series

39.

Policy Monitoring and Research Centres (PMRC) support thus managing development results for greater accountability and transparency in public service delivery mechanisms and among the public duty bearers. To realize the vision of a single standard monitoring tool of Government development delivery and marry it to with effective system of timely, transparent communication of progress and achievements, PMRC is in the process of developing a composite Government Delivery Index (GDI). The PMRCs proposal to develop the GDI builds on an identified need for Zambia to narrow information variances brought about by use if multiple M&E framework by developing a single, independent and consolidated Government delivery Monitoring and Evaluation (M&E) framework. Unlike the Government-wide KPI, this proposal seeks to tie together key elements of the results chain-from: Activities Inputs, Processes, Outputs; Outcomes

Government M&E framework focuses on input-output analysis, the PMRC GDI, by going beyond input-output analysis, will include processes, inputs and outcomes. It will provide a sound basis for painting a comprehensive Government delivery picture and timely and relevant evidence-based feedback. This initiative recognizes and builds on the current Governments efforts towards developing a Government - Wide Monitoring and Evaluation System (GWMES) based on select Key Performance Indicators (KPI). The value addition that GDI brings to the table is its niche of giving the opportunity for the Government leadership to adequately be consulted and involved in identifying and ranking delivery priorities consistent with their Party Manifesto and broader long-term vision. The opportunity for developing the GDI is further strengthened by the current need for more harmonized policy strategies and M&E approaches within Government, Co-operating Partners, the Patriotic Front Manifesto and civil society organisations. The starting point for the GDI concept is the commitments in the PF Manifesto (as basis for measurement) and how these guide Governments policy. The GDI focuses on how the Government is delivering on its Manifesto development commitments.

1.1 Statement of Policy Opportunity

The multiple M&E frameworks in Zambia have led to ineffective tracking, evaluation and reporting of Government delivery efforts in various sectors. Within Government, differences exist between the Sixth National Development Plan (SNDP), Medium Term-Expenditure Framework (MTEF), Public Expenditure
40.
GDI Series

and Financial Administration (PEFA) and each individual ministrys strategic plans. The SNDP outlines KPIs for every ministry, but lacks a clear implementation plan of projects and programmes. This affects the ability to monitor and evaluate the progress being made by government. It is not very clear how the PF Manifesto is being interpreted and used to influence and guide Government policies. Similarly, it is perceived that Co-operating partners use their own approaches to benchmark Government delivery. Notably, co-operating partners reference frames are; the SNDP M&E framework and the Joint Assistance Strategy for Zambia (JASZ II). This has resulted in significant differences in data between CSO, Government and civil society organisations. These differences are a significant concern not only for reporting, but also planning and monitoring purposes. This qualifies the urgent need towards harmonized key strategic documents for Zambia (SNDP, MTEF, JASZ, PF Manifesto, and Ministry Strategic Plans), and also the related M&E frameworks. The GDI therefore is proposed as a composite index, whose formulation will be guided by best practice of developing indicator framework. This will enhance and promote its wider appeal, use and sustainability as a tool for tracking government delivery.

1.2 Rationale for the Government Delivery Index

The GDI is being developed as composite policy, strategy implementation and delivery monitoring, evaluation and learning tool. PMRCs rational for this proposal is based on its felt the need to contribute to a more robust, transparent and accountable national governance systems M&E framework which has a much wider public appeal and value. A more consolidated M&E framework for assessing Government delivery is likely to reduce information asymmetries, and foster a more consensus-based progress tracking and feedback system so support effective on-going performance benchmarking of PFs and successive Governments. PMRC preliminary visioning of the GDI is that it will consist of the following 3 primary assessment results-chain variables:(Dimensions) Processes; (policy, strategy and regulatory initiatives); Outputs (delivery tangibles, hospitals; schools; jobs created, teacher trained and recruited etc.); Outcomes Impact (resulting changes, both visible and more intangibles such as change in attitudes, culture of doing things etc.)

GDI Series

41.

The 3 Key GDI Dimensions:

Process

Output

Impact

PMRCs prototyping of the key GDI Assessment Framework components:

1.3 KEY GUIDING QUESTIONS

The following four key questions will guide the process of deciding on the structure and help identify key dimensions, sub-components and essential indicators for the GDI (key assessment areas and indicators). How is Government delivering on its Manifesto commitments? To what extent are the PF Manifesto core commitments aligned to and inform Government policies, strategies and programmes? Is quality data to support effective monitoring of progress on Government commitment delivery available and accessible? What stakeholders engagement and feedback mechanisms exist to promote wider public monitoring and understanding of governments delivery goals and targets?

1.4 Implementation Methodology and process 1.4.1 Methodology

The visioning of the structure, key components, sub-dimensions and key performance indicators of the GDI will be based on key Government delivery commitments as articulated in the PF Manifesto 2011 -2016. I. The first step will be to conduct a gap analysis of the PF Manifesto commitments in order to assess the political, economic and social feasibility of the commitments based on reasonable sequencing and time frames. This will help PMRC to align its monitoring to realistically sequenced and phased implementation of Manifesto and Government delivery promises.

42.

GDI Series

II. The second step will involve planning and conducting a GDI stakeholder consultative workshop. This meeting will target key stakeholder segments who are potential users of data and beneficiaries as follows: Government departments, especially the policy planning, implementation and M&E units Multilateral and bilateral development institutions and agencies Government, Local Government and parastatals Civil society organisations( research think-tanks, NGOs, Private sector interest organisations, and community-based organisations Central Statistics Office, as the Government-wide data and information repository Academia ( Statisticians) A GDI concept note, outlining the key guiding questions, policy rationale and strategy for GDI and its proposed structure, will be developed and presented during the stakeholder consultative meeting as a background and discussion document. The rationale for holding this meeting will be to identify opportunities and potential risks in the wider environment for developing the GDI, promote consensus and gain wider appeal and value for the tool proposed tool. This will ensure it is continuously used and thus; is an important strategy for sustaining the impact and wider use of the tool. The third step will be the expert consultative meeting, for consultations and sequenced phase of events.

1.4.2 Implementation Approach

The key dimension for all sectors will be to evaluate; (1) what has been done and (2) what is being undertaken towards achieving every commitment in the Manifesto. The GDI will take into consideration each sectors unique operation modes and mandates; this will allow the Index to review one sector per time. Importantly, the sector under consideration and evaluation will be given as much consideration as possible and thus cover greater evaluation. For each and every commitment outlined in the Manifesto, three (3) dimensions will be determined; in (1) Process, (2) output and (3) outcome. For each dimension, an overall weight will be attached, data will be collected and the final score will be computed. For different commitments per sector, the weighting system will vary, with regard to the overall delivery process. The GDI will reflect every significant initiative, project and development in a sector aimed at achieving the target. The approach will engage a Technical Advisory Group (TAG) and various Stakeholders for consultation and guide.

GDI Series

43.

1.4.2.1 Weighting
Weighting for the GDI will be established after consultations and engagement with the technical advisory group. Different indicators will be attached various weights, based on their rating and overall relevance to achieving the delivery goal.

1.4.2.2 Dimensions
For every sector, the Manifesto commitments will be clearly outlined and each mandated to input process, output result and final outcome. For the GDI, the dimensions for each sector commitments will be process, output and outcome. The Indicators, for every commitment will be the intended target and outcome measure planned by the Government.
Figure1. (The GDI Framework)

GOVERNMENT DELIVERY INDEX

DIMENSIONS DIMENSIONS

DIMENSIONS

SUB-CATEGOTRIES

INDICATIONS

The structure of the GDI will draw from best practice and identify Dimensions (that encompass government delivery). For Every Sector, the commitments will be outlined and then 3 dimensions defined and analysed. Dimension 1- Process Dimension 2- Output Dimension 3- Outcome

44.

GDI Series

Illustration 2. (The GDI Structure)

Overall target

Current Progress Targets/ Outcomes met

Scale :( 0, 25%, 50%, 75%, 100%) The big outer white circle represents the overall sector. The small black circle shows what has been done. In this example, we see how the GDI in a sector may look like, if no major delivery has taken place. This may be a 10% score The white circle represents the overall sector. The black circle shows what has been done. In this example, we see how the GDI in a sector may look like, if major delivery has taken place. This may be a 98% score.

Illustration Chart of Key GDI Assessment Areas

GDI ASSESEMENT - PER SECTOR

Assessment Area
Commitment 1 Commitment 2 Commitment 3 Commitment 4 Commitment 5 Commitment 6 Commitment 7 Commitment 8 Commitment 9 Commitment 10 Commitment 11 Commitment 12 TOTAL SCORE

DIMENSIONS

SCORE
OUT OF 10 Total/10

WEIGHTINGS

PROCESS Weight

OUTPUT Weight

OUTCOME Weight

TOTAL

TOTAL

TOTAL

TOTAL

(Source:-Policy Monitoring and Research Centre, October 2012)

GDI Series

45.

One ministry or sector will be evaluated per time. Assessment areas will be drawn from the Manifesto, outlined as commitments. The Weightings will be attached to each assessment area based on its overall importance/ rating towards achieving the total objective in that sector. The 3 dimensions for each assessment must add up to 10. The measurement scale is therefore 1-10 For each assessment area, weights will be attached for process, output and outcome. The total GDI score will be computed by adding the sub totals and computing the average. The Minister or Permanent Secretary will be engaged to determine what is being done for every assessment area. After weights have been attached, the GDI will be subjected to a team of experts, to validate and analyse the scores. This will pass through a stakeholders consultative meeting and general public for more views and reviews

1.5 Other considerations

The GDI recognizes the government wide M&E framework being formulated and leverage from it as a data source. The weighting scale for the 3 dimensions will be from 1-10. The GDI progress tracking will be drawn from assessment areas, outlined in the manifesto as commitments, under every sector. The GDI will monitor one sector/ ministry per time. Process here is defined as all activities, initiatives, strategies, projects, collaborative efforts and programmes that are being undertaken towards achieving the commitment as planned by government. Output here is defined as the results from all the areas identified in the process phase. Outcome here is defined as the total result and actual impact of the process and output phase. This goes to determine the impact and any development taking place as a result of processes. This will be used to measure delivery.

46.

GDI Series

GDI Series

47.

Unlocking Zambia's Potential Correspondence on this Series can be sent to: info@pmrczambia.net Policy Monitoring and Research Centre (PMRC) Plot No. 32 Sable Road, corner Bishop and Sable Roads, Kabulonga, Lusaka, Zambia Private Bag KL 10 Tel: +260 211 268 385 | +260 979 015 660 www.pmrczambia.org

48.

GDI Series