Setting Up a Poverty Monitoring and Evaluation System: Experience from Asia and Africa

by Sylvain Lariviere, Ph.D. and Frederic Martin, Ph.D. University Laval and IDEA International Institute

Preliminary Draft

Paper presented at the Regional Conference on Poverty Reduction Strategies

4-6 December 2001

Viet Nam .2 Ha Noi.

including policies and programs. . Identifying a set of priority actions. 2. The starting point is to define poverty (poverty criteria and poverty line) and related concepts of vulnerability 1 A Specific Measurable Objective is a target level at a given date on a cardinal scale for a given indicator. the choice of methodology depends partly on human resource capabilities and institutional relative strengths. and costing 4. PRELIMINARY CONSIDERATIONS Let us start by outlining three points. Second. Finalizing the specification of SMOs for priority actions to ensure consistency between available financial resources and the costs of reaching the SMOs. This implies a strategic planning approach including: 1. there is no unique PME system which is valid for all countries all the time. on the other hand. PME needs assessment Data collection Data analysis Policy recommendations Budget allocation The starting point for a PME system should be a rigorously defined Poverty Reduction Strategy. technical issues are intertwined with institutional issues: on one hand. to reach the target SMOs of the strategic orientations at a given deadline.1. Setting preliminary SMOs for priority actions. 3. describing implementation strategies. First comes the recognition that data collection and analysis are often misguided and useless because they are not guided by a sense of priority PME needs and they are not fed into the decision-making process to become policy recommendations and eventually budget allocations. the choice of institutions and their respective roles in the PME system depends on the technical characteristics of the proposed system. Third. Choosing a reduced number of strategic orientations of poverty reduction and setting Specific Measurable Objectives (SMOs)1 such as the Millenium Summit International Development Goals.

The need to assess the impact of key policies and programs on poverty. These needs have to be assessed much more specifically and a consensus built around the priority needs. 3. as indicated. The need to assess the overall impact of the PRS. development partners) should assess its needs in terms of PME since various needs along with institutional and human capital characteristics will lead to various PME objectives and designs. a good PME system should include three components to address each of those needs (Figure 1): • • • A Poverty Monitoring Component which focuses on poverty evolution (outcome indicators). 2 The result-based management litterature distinguishes 5 categories of indicators which correspond to the sequence of actions in any program or project : Inputs → Activities → Output → Outcome → Impact. Then the government in consultation with other stakeholders (civil society. understand it. The need to monitor the level of achievement of the Poverty Reduction Strategy (PRS) to see if the work plan adopted is actually being implemented according to schedule and if financial resources are allocated according to the set priorities. and identify the poorest areas. focuses on impact indicators. in most cases. NGOs. PROPOSAL OF A GENERIC PME SYSTEM Four types of needs have been expressed in many African and Asian countries regarding poverty monitoring and impact evaluation: 1.4 and inequality to reflect national and regional realities. The need to monitor the poverty situation. a PRS Monitoring Component which concentrates on budgetary (input) indicators and PRS objectives achievement levels indicators (output). 2. 2.2 These three components taken together make it possible to evaluate the global impact of the PRS on poverty. Each component is discussed in the following sections. However. 4. . an Impact Assessment Component which.

5 FIGURE 1 : THE TECHNICAL COMPONENTS OF A GENERIC PME SYSTEM POVERTY MONITORING COMPONENT Population Well-Being related Indicators System • • • • • International and sub-regional environment indicators Macroeconomic indicators Sectoral development indicators Local well-being indicators Vulnerability indicators • • • • • Poor Well-Being related Indicators System Poverty profile indicators Poverty characterization indicators Poverty dynamics indicators Vulnerability indicators Inequality indicators Source : Household survey system Source : Administrative services INPUTS OUTPUTS OUTCOMES IMPACT PRS Monitoring Component Monitoring of budget allocations for priority actions Monitoring of SMO’s achievement levels • • • Poverty Impact Assessment Component Impact of selected macroeconomic policies on poverty Impact of selected sectoral policies on poverty Impact of selected programs on poverty Overall PRS Assessment .

One may focus simply on income poverty criteria (using a moneymetric criteria such as consumption expenditures) or include other dimensions of human poverty as well (nutrition. • Keeping track of the dynamics of poverty over time.e. • Understanding the endogenous and exogenous factors behind the poverty trends. i. supply of social services.) and even human rights criteria. etc. This will also help identify the poorest areas to focus interventions and budget allocations.: GDP per capita) and of factors influencing this well-being level such as growth. why do we observe a reduction or an aggravation of poverty over time? This introduces the causal analysis of poverty.g. i. rainfall.e. • Understanding the endogenous and exogenous factors behind the dynamics of poverty. the influence of shocks and types of capital endowment on poverty flows. They can be sub-categorized as: • • • • • International and sub-regional environment indicators Macroeconomic indicators Sectoral development indicators Local well-being indicators Vulnerability indicators. the analysis of poverty dynamics focuses on the flows in and out of poverty. who is getting into poverty. Indicators of the Poverty Monitoring Component fall into two categories: (i) population wellbeing related indicators and (ii) poor’s well-being related indicators. health and education levels. The first category includes indicators of the well-being of the whole population (e. i. who is remaining in poverty? While the analysis of poverty trends focus on the stocks of poverty. Most of those indicators come from secondary sources and can be usually routinely collected by public services. . other basic needs indicators.6 Poverty Monitoring Component This component may have several objectives: • Keeping track of poverty trends. who is getting out of poverty.. etc.e.

as capacity . others yearly. The analysis must also be broken down by gender since women and men do not have the same opportunities and constraints and poverty levels in most societies. Often. The most efficient way to obtain those data is to set up a household survey system including various surveys with qualitative and quantitative modules. Most of the data needed to calculate these indicators require primary data collection. etc. the higher the requirements in terms of human and financial resources for the PME system. commune) and agro-economic (urban/rural milieus. Let us also note the possible use of proxies to real indicators which might be easier and cheaper to monitor.7 The second category includes indicators on the well-being of the poor and factors influencing it. An example of such a proxy might be the wage rate of unskilled wage labor or the price of a staple food like rice. In a country characterised by significant human capital limits. The analysis must be broken down from the national level towards subregional units. These units may be administrative (province. The more disaggregated and frequent the monitoring. These indicators can be grouped under the following sub-headings: • • • • • Poverty profile indicators Poverty characterisation indicators Poverty dynamics indicators Vulnerability indicators Inequality indicators. some indicators need to be monitored quaterly. district. agro-ecologically homogenous zones). a stepwise approach is suggested whereby the PME system starts with the monitoring of poverty trends with a limited number of poverty indicators and. Various poverty indicators might be monitored with different frequencies because they vary more or less over time. All these analyses must be disaggregated to be meaningful because poverty is unevenly distributed. these indicators are also calculated for the non-poor to serve as a basis for comparison. every 2 or 3 years.

Poverty Impact Assesment Component This component may have two objectives : • Assessing whether a specific macroeconomic or sectoral policy or a specific program (e. vulnerability and inequality (PVI) indicators. . (ii) other modules are progressively added. Setting up this component implies that SMO’s have been specified and validated at the time of PRS formulation. An adequate allocation of funds to the right use at the right time is a precondition to reach SMO’s. (i) more indicators are added to the poverty trend modules.. After defining poverty.8 building and experience increase human capital in the PME team.e.. a set of strategic axes and priority actions. strategic axes and priority activities at different time horizons. one must transform the matrix of poverty oriented macroeconomic and sectoral policies into a set of Specific Measurable Objectives (SMO’s). • monitoring the budget allocations made to reach the SMO’s by priority action of the PRS. For the PRS to become operational and its level of implementation monitored. i. a microfinance program) succeeded in reducing poverty . The indicators monitored therefore relate to the level of achievement of the PRS SMO’s at various time horizons with a calculation of relative gaps and the level of budget allocation to achieve the SMO’s of a given priority action. 3 A Poverty Reduction Strategy usually involves a set of global poverty reduction objectives. PRS Monitoring Component This component usually has two objectives : • Monitoring the level of achievement of the Specific Measurable Objectives (SMO’s) of the Poverty Reduction Strategy (PRS)3 and check whether the work plan adopted is actually implemented according to schedule. a matrix of poverty reduction oriented macroeconomic and sectoral policies. levels of key indicators related to policies. and (iii) the level of disaggregation is increased to lower administrative levels (province → district →commune).g. the second step in the elaboration of a PME system is to agree on a priority set of core poverty. a budget for financing those actions and a calendar of realisation.

However. including the isolation of the impact of the policy or program under scrutiny on poverty from all the other endogenous and exogenous factors that may affect the poverty outcome. could we have achieved higher poverty reduction at a reduced cost. even more useful. In a 4 Selected sectoral policies Selected representative programs A serious ex post impact assessment may require from 1 to 2 years. but rigorous analytical tools such as small empirical trade or budget models.9 • assessing the level of efficiency of specific policies or programs. so that better policy recommendations can be formulated.e. is ex ante impact assessment in which the impacts of a given policy or program are simulated. .g. At the same time. Ex post impact assessment is very useful to understand the past effects of policies and programs and improve upon their future design. This comment actually applies to the whole PME system. children benefiting from an income generation project for women) as well as the medium and long run sustainable effects of the policy or program. i. The notion of impact assessment also implies tracing the indirect effects on non target group well-being (e. Ex ante impact assessment requires a clear and structured analytical framework. impact evaluation is not easy. it is most useful for policy analysis. The suggested PME need assessment exercise should be conducted early in the process of elaboration of this system and should not end up being an endless shopping list of indicators in order to satisfy every user’s whims. and simple. is costly and may require time4. Therefore it is suggested that this component be gradually added to the PME with a careful selection of the policies and programs to be evaluated. a good understanding of the expected behavior of economic agents. faster or in a more sustainable way? Impact assessment implies addressing a series of methodological challenges. a minimum amount of valid and up to date data. Selected macroeconomic policies Ex post evaluation Ex ante evaluation When done seriously...

one should move beyond the quantitative impact measurment of the PRS on living conditions and poverty to also take into account the impact of the process on two key conditions of sustainable human development.e. change their behavior in a sustainable way. putting together and triangulating the results from the three PME system components should provide a pretty good indication of the impact of the PRS and explain the reasons behind the observed performance. outputs. PRELIMINARY LESSONS LEARNT FROM AFRICAN AND ASIAN EXPERIENCES Let us underline the very preliminary nature of lessons learnt since most countries are in the process of designing their PRS and PME system. However. outcomes and impact. priorities have to be established and a gradual increase in depth and breadth of the PME system be considered as capacity building is reinforced. if not impossible. and nationals being better equipped to negotiate with development partners 3. (iv) key policies and programs within the PRS have proven to reach the poor. and resulted in improved living conditions. The overall impact of the PRS cannot be easily measured quantitatively since it is very hard.: (i) capacity building: designing a PRS. a closer articulation between civil servants and civil society. Since there is a logical causal linkage between inputs. (ii) improved planning: The process should improve absorption capacity.10 country faced with strict human and financial capital constraints. involves substantial formal and learning by doing training to put the nationals in a position to master up to date inter-sectoral and participatory strategic planning tools and qualitative and quantitative data collection and analysis methods. including a PME system. However. i. for the PRS to have an impact implies in sequence that (i) inputs have been allocated efficiently according to the set priorities (ii) the SMOs of priority actions and strategic orientations have been achieved (iii) poverty and well-being indicators indicate a better off situation with the PRS compared to a situation without the PRS and finally. . more efficient policies and program design and implementation. to isolate the effect of an inters-sectoral strategy such as the PRS per se from all other exogenous and endogenous variables that affect the population and the poor living conditions.

A lot of energy is still spent on data collection on large samples at the detriment of analysis and policy recommendations. especially in a policy making perspective. • Apart from difficulties already mentioned at the PRS level. and (vi) poorly analyzed. The implication for the PME system is that monitoring the PRS is very hard in such a context. . (v) or at the same time. (iii) with very little quality control. (ii) collected on an irregular basis. among which: Challenges related the PRS. There is much generalities in a number of existing PRS without clear definitions of prority actions. Designing a truly intersectoral PRS requires a change in the traditional way of designing policies. PRS monitoring is faced with the difficulty of obtaining (i) disaggregated budgetary allocations to be able to establish a clear link between financial resources and outputs (ii) the required data to measure the level of achievement of SMOs on a timely and rigorous basis. • Household surveys are still often biased toward large scale. institutional and financial capacities and are therefore conducted on an ad hoc basis. and especially the PRSP process: • • • • Often the time frame does not allow for a full participatory process and an indepth strategic planing exercise. There is often a lack of articulation between the costs of SMOs and available financial resources. Challenges related the PME system: • Routine administrative data are often: (i) not necessarily useful. (iv) not necessarily at the same level of geographic desaggregation. expansive operations that exceed national human.11 A rapid overall assessment of the PRS and PME processes indicates a number of challenges. programs and projects. SMOs and costing of SMOs. Coordination is often the missing link among national structures and among donors.

(iii) the cost of conducting a serious impact assessment. (ii) the time frame required to obtain cardinal measurements of impacts. such as control group design. . in particular related to isolating the impact of the specific policy or program under consideration. This means that the PME requires the design and implementation of a Management Information System (MIS) for greater efficiency in information flows. Above all. (iv) the expertise required in qualitative and quantitative methods and field experience in household analysis. the PME greatest challenge is to build upon existing functional units and ensure efficient co-ordination among national structures and development partners to ensure a timely production of useful and a consensual analysis of results to be fed into the policymaking process.12 • Impact assessment of key policies and programs requires addressing several challenges among which: (i) methodological issues.

Sign up to vote on this title
UsefulNot useful