Participatory Monitoring and Evaluation
TABLE OF CONTENTS
Introduction and Overview • Welcome • Shifts in Development Thinking • Learning Objectives • Methodology • Target Audience • Expected Outcomes • Lesson Plan • Exercise Concepts and Principles of Participatory Monitoring &Evaluation • Definitions • Indicators, Outputs, Outcomes And Impact • Quantitative, Qualitative and Participatory Methods • Principles of PM&E • Role of Gender • Value-Added of PM&E • Exercise The PM&E Process Cycle PM&E Case Studies • Micro or Project Level • Meso or Local Governance Level • Macro or Policy Level Exercise: Group Action Planning Additional PM&E Resources Monitoring Mechanisms References
Section III: Section IV:
Section V: Section VI: Appendix A:
Welcome Most of us would agree that in theory, evaluation is a good thing. But putting this ideal into practice is often seen as too complicated, conflict-ridden, and/or costly. This Module is designed to introduce diverse development stakeholders to Participatory Monitoring and Evaluation in different contexts – what it is and how it adds value to development work at both the project and the policy level. Participatory Monitoring and Evaluation (PM&E) is a tool for bringing diverse stakeholders together, giving them an opportunity to negotiate regarding what success should look like and what indicators should be used. This process of negotiating the definition of success is a first step in institutionalizing participation and voice in development. Whose information is used to assess the success of a policy or project is important if we aim to make projects and policies more grounded in the realities of poor and disenfranchised people. PM&E ensures that all stakeholders have a voice in what indicators are used and how the results are analyzed. Multiple sources of information guarantee the greatest accountability and triangulation of results, while allowing for the monitoring of monitors. There are many potential positive spillovers from PM&E -- for example, experiential learning through working in diverse groups builds trust and the ability to cooperate across sectors/stakeholders for future development endeavors. Shifts in Development Thinking PM&E encompasses some of the major conceptual shifts in development work today. In accordance, this Module is based on the following assumptions: 1. Participation and ownership of all stakeholders is essential for sustainable development. 2. Monitoring and evaluation are critical for holding development actors (all members of society) accountable, measuring our successes and learning from our mistakes. This means going beyond measuring inputs and outputs. 3. Expertise is not the monopoly of technocrats; people at all levels hold important relevant information. 4. Working across groups in an atmosphere of trust (with some healthy skepticism) is important. Experiential learning prepares people to work cooperatively across groups and organizations; people often need spaces and opportunities to build healthy social capital across institutions. Learning Objectives of this Module By the end of this module, participants should be able to: Articulate the importance of monitoring and evaluation – who measures; what goals; and how should it be done; Highlight the value-added of participatory monitoring and evaluation versus traditional approaches; Describe the process cycle for designing a participatory monitoring and evaluation program applicable to various projects, policies and processes; Appreciate the wide variance in evaluation approaches; each approach needs to be tailored to specific context and capacity; Recall model practices and case studies on PM&E in different settings and refer to additional on-line resources; 2
however several face-to-face and/or Distance Learning activities can be added to deepen understanding and skills. meso and macro levels with Learning Activities which allow you to practice different stages of the Process Cycle. Ideally a diverse group of organizations will be represented during the training and have the opportunity tot work together on some of the practical applications during and after this module. Trainers for any of the above mentioned constituencies will also be targeted. and is intended to be adapted to your local needs. • Section III introduces the PM&E Process Cycle which can be modified and applied in a variety of different contexts. Case studies will also be used to help participants apply their learning during the course itself. • Section VI refers you to additional resources (some of them on-line) and opportunities to stay engaged in a PM&E Community of Practice in order to be involved in follow-up learning activities and to request and/or provide technical assistance to and from your fellow participants. private sector. Target Audience The primary audience for this Module is community-based groups. interactive way. Note: this module will offer sufficient background and experiential learning for stakeholders to participate in on-going or future PM&E activities. Participants will also know where to find additional resources and technical assistance via the internet and through the community established during the training. Methodology In order to meet the above objectives. Additional learning will probably be necessary.
. Expected Outcomes By the end of this Module. it will not provide sufficient technical skill-building to enable all participants to design and implement their own monitoring and evaluation. this Module is delivered in a participatory. The minimum time required for delivery of this module is three hours.Collaborate with other groups of stakeholders on PM&E activities. Call upon a community of practice to support one another post-training. participants will hopefully view PM&E as a doable. media. collaborative and worthwhile investment for designing and delivering programs and policies that meet the needs of a wide range of stakeholders. • Section IV provides short Case Studies of PM&E at the micro. based on practice during the module. think tanks and donors organizations. Follow-up activities may involve Distance Learning (computer-based/internet training and video-conferencing) where appropriate. Participants will be familiar with the PM&E Process Wheel and will have practical experience working through some of its stages. Lesson Plan • Section II presents a Conceptual Framework including Definitions and Principles of PM&E. However. local and national government. Other target audiences include representatives from NGOs. • Section V offers space for Action Planning regarding potential PM&E relevant to your own context. especially poor people.
Before You Begin… Before you begin. and to be able to solve them immediately. and that the project outputs and outcomes correspond to the objectives.
1. accountable and transparent system of operating. made good progress. we would like you to reflect a little on your own situation by answering the following questions. Monitoring helps all stakeholders to recognize problems and obstacles. What are two expectations you have for this course/module? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
. and faced any problems. It also considers how to solve such problems. It thus plays an important role to ensure that activities are performed on schedule and within the allocated budget. It enables a more responsive. that all concerned are satisfied. How will we judge success at the end of this module? How will we know if this course has been successful and if it has met your expectations? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
SECTION II: CONCEPTS AND PRINCIPLES OF PARTICIPATORY MONITORING & EVALUATION ===================================================================== Definitions Monitoring examines whether the project has been implemented as planned. What project or policy in your local community or region would you like to see monitored and evaluated using participatory techniques? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
all stakeholders discuss and plan the project together from the outset. For instance. Participation effects monitoring and evaluation in the following ways: • The purpose of the evaluation • The criteria and indictors identified • The design and implementation of the evaluation • Who participates in the monitoring and analysis • How the information is used. resource mobilization. selfreliance. jointly setting the objectives. but to give advice and assistance so that the project can fulfill its objectives efficiently and effectively. and environmental and natural resource conservation. In this process. but also a learning process for all concerned. all interested parties and civil society can play a role themselves and take part in making decisions and planning policies for development and problem solving. Evaluation also reveals the state of changes. Evaluation can be conducted at several levels. ===================================================================== Citizen participation in monitoring the activities of local administrative organizations is an important method for creating good local governance. Notably. and the extent of the benefits from such changes.
Monitoring and evaluation offer not only tools for effective and efficient project implementation. Evaluation helps to target project outcomes. and work process. Monitoring and evaluation yield the greatest benefits when they are performed in a participatory manner. income distribution. The mission of the monitors is not to find fault. Equity: inputs and benefits reach the various stakeholders fairly. targets. the evaluation of a project designed to change unfavorable citizen behavior can be done at three levels. by measuring and comparing the trainees’
. Accountability: actors are held responsible for their contributions by other constituents. Participation ensures four things: • • • • Responsiveness: objectives are met according to the priorities of all stakeholders. It can be done immediately after the training. the persons such changes affect. and the extent and features of project benefits. The evaluation should take into consideration the impact upon individuals and the community in terms of the development of new ideas and quality of life. PM&E is a systematic management tool designed to reveal the degree of effectiveness and efficiency in the achievement of objectives according to the perspectives of various stakeholders. as follows: 2) Primary evaluation or phase 1 is to assess the output or immediate outcome. This makes sure that we measure not just inputs and outputs but what significant changes have taken place.Evaluation systematically judges the value of changes (planned and unplanned) resulting from project outputs and outcomes. Transparency: clarity and openness regarding activities and relationships.
trainers. Output is compared with objectives by following the trainees’ behavior to assess whether and how they change their behavior after a certain period. For instance. It may change the living condition of project participants. • Lower diarrhea results in a lower infant mortality rate. materials. These indicators often require open-ended responses.
Indicators. Impact: the final result of the outcome. Outcome: a result after the use of outputs. The impact might be seen by the end of the project or several years afterwards. and whether the objective will be achieved. workshops. For instance. which assesses whether a change is in the desired direction. A good indicator reflects both quantitative and qualitative changes. Outputs. 4) Tertiary evaluation or phase 3 is to assess the outcome and impact. outcome. • Women’s having more capital to invest results in this impact: higher income and better
. Outcomes And Impact Four key terms tell a monitoring and evaluation group the effects of a project: indicator. and healthier children. the outputs include training documents. • Higher enrollment in a school might have the impact of increased literacy. by assessing how such behavioral changes affect society. For instance: • The availability of clean drinking water in the village results in an outcome of a lower incidence of diarrhea.knowledge before and after the training. Quantitative indicators can be displayed as a ‘single figure’ (such as the amount of produce in a given year). 3) Secondary evaluation or phase 2 is to assess project effectiveness. qualitative indicators may: • Measure behavior in food consumption from weight and height gained • Measure the health situation by the availability of clean drinking water and toilets in a village • Measure poverty by the condition of houses and construction materials Output: the product or service produced in order to achieve a target. Qualitative indicators measure either a behavior or an indirect representation. Indicator: a variable to measure the progress of activities or changes resulting from the project activity. • School construction results in an outcome of higher enrollment. to achieve a target to enable 75% of women to understand laws and the Constitution.) This figure can be in the form of percentages and proportions. and impact. • Provision of loans without collateral results in an outcome of more capital for women to invest. and so on. For instance. output. or a ‘cumulative figure’ (such as the produce over several years combined together.
comparable and ‘generalizable’ to a wider population. unstructured. This allows for in–depth analysis of social. open-ended efforts are incorporated. political and economic issues. structured individual interviews for data collection and subsequent statistical analyses which allow the results to be considered representative. identify one of each of the following: Indicator: _____________________________________________________________________ Output: ______________________________________________________________________ Outcome: _____________________________________________________________________ Impact: ______________________________________________________________________
==================================================================== Monitoring and Reporting
Principles or Characteristics of Participatory Evaluation
The Role of Gender
The Value-Added of PM&E
SECTION III: QUANTITATIVE. 1
Krishna and Shrader. ‘Qualitative’ methods often refer to a spectrum of data collection and analysis techniques where sampling is not necessarily random and more in-depth. QUALITATIVE AND PARTICIPATORY METHODS • ‘Quantitative’ methods often refer to random sampling for survey research. ===================================================================== Exercise: Based on the local project or policy you proposed to monitor and evaluate. 2000
.nutrition for children.
Conduct a readiness assessment.
. The two underlying assumptions of most evaluations are that the people judging a project are a neutral and independent third party. independent evaluations has no claims to
References: Gaventa IDS. Establish goals. Share info/results with all relevant parties. and PRSP Sourcebook): The information in this case study is drawn from Françoise Coupal and Marie Simoneau. establish principles for working together. and that an evaluation without these neutral.html). Set realistic targets. Gather info (quantitative and qualitative). and assessed. Develop indicators. Activities for learning more about PM&E 1. d. f. implemented. it may be more useful to distinguish between contextual versus noncontextual methods of data collection and analysis (Hentschel 1999). project or evaluation is designed.
SECTION IV: PARTICIPATORY MONITORING & EVALUATION PROCESS CYCLE Participatory Monitoring and Evaluation Process Cycle 2 a. c.ca/pehaiti15. A few myths about PM&E 3. Construct baseline. Participation requires that various stakeholders have a voice and some leverage in deciding how a policy.mosaic-netintl. Introduction and the Case of the CCIC in Haiti Evaluations classically exist to ensure accountability. monitored. Decide who participates. l. i. g. j. so that funds are spent as they are intended and so that we learn from our mistakes and build on successes.Given the imprecise divide between quantitative and qualitative approaches. Introduction and the case of the CCIC in Haiti 2. Analyze results including all stakeholders. And both methods can be carried out in participatory ways. However. Take action – use the findings!. e. Narayan. Gather info again (quantitative and qualitative). h. Kusek and Rist. k. “Participatory Evaluation: A Case Study of CCIC Humanitarian Fund Projects in Haiti” (http://www. Start over (making any needed modifications in process…)
SECTION V: THREE CASE STUDIES OF PARTICIPATORY MONITORING AND EVALUATION Case Study 1: Haiti3 This case study is divided into three parts: 1. neither of these approaches are guaranteed to be participatory. b.
“objective” effectiveness. and it seeks quasi-scientific evidence for judging project success. analyze.
. This approach is mainly quantitative because it chiefly measures inputs and outputs. Managers. Many observers have called for a new approach to evaluation.4 which illustrates its added benefits:
Build capacity of stakeholders to reflect. 3. 1997). and take action. UNDP. Accountability to Stakeholders. one that is qualitative and participatory. The advantage of participatory evaluation can be summarized by this diagram.
The diagram is adapted from (and slightly changed) Coupal and Simoneau. and Donors
Various functions of Participatory Evaluation
Development of lessons learned for corrective action by stakeholders
Development of lessons learned which help program staff improve
Improve rapport between government and civil society. Adapted from Who Are the Question-Makers? A Participatory Evaluation Handbook (New York: OESP.
Haiti is the poorest country in the Western hemisphere. The United States invaded Haiti in 1994 and President Aristide was restored to power. Accountability only documents inputs and outputs instead of impact. to credit. The CCIC trained approximately thirty personnel from NGOs in Haiti in participatory methods of monitoring and evaluation. After conducting a four-day workshop and an evaluation process. or inadequate sharing of information on a project Poor management and lack of transparency Poor leadership and paternalistic attitudes Egotism A lack of community participation and communication between project managers and project beneficiaries Poor monitoring and follow-up A few myths about participatory evaluation and monitoring
. NGOs and grassroots organizations began to implement development programs. The centralization of power. the CCIC made a point of using participatory monitoring and evaluation. the CCIC aimed to build capacity among Haitian and Canadian NGOs by training local participants in participatory methods. a coalition of volunteer groups. took the principles of participatory M&E and tested them. in a somewhat volatile and unstable setting: Haiti. ranging from agriculture. in the wake of a political coup. to communications. with notable success. ousting President Aristide and stirring a wave of violence and unrest. As Haiti began to rebuild its fragile civil society. the participants concluded that the success of any development project depended on several factors: • • • • • • • • It must respond to real priorities in the community Training should create and build capacity of organizations and groups Group dynamics should enhance feelings of belonging and of honesty Community participation from the onset is important Good governance and leadership of the project is key Good project structure and follow-up is essential Strategies must be flexible Sufficient resources must be available
The program also found that project aims can be limited or blocked by the following factors: • • • • • • 2. to nutrition. and elected Father Jean-Bertrand Aristide as President.Recently. Haiti held its first-ever democratic elections. an aid organization. After seven months there was a military coup. Some of these personnel were reluctant to participatory methods at first. Its intent was to go beyond the simple accountability function. The CCIC. funded thirty-six Haitian development projects. the Canadian Council for International Cooperation (CCIC). Instead. As part of its broad program. In 1990.
Participation takes time and is costly. the concerns about objectivity and expertise were also found to be unsupported. Project participants cannot “objectively” evaluate projects. signifying indicators of local wealth and poverty. cattle.
Field Insight: Lesotho Community people know well the relative wealth and poverty of other community members. No. rice.
Village women sorted the pictures into three categories of items owned by the rich. (World Bank Working Paper. participatory M&E facilitators found that a core of committed and skilled practitioners can train participants to evaluate projects fairly and effectively. and poor people. that participatory evaluation only works in a project devised with participation as its basic principle. such as maize porridge. Participatory Evaluation. Participatory evaluations can only be done with projects designed in a participatory manner. In total agreement. A few blank cards were provided so that any missing indicators could be added. and they became better skilled in its methods.The CCIC found that its NGO personnel participants brought with them four reservations about PM&E. the women took only a few minutes to rate themselves and neighboring families in the appropriate categories. Time and cost objections were also reversed when the participatory M&E was revealed to (1) cost the same as traditional evaluation. for its time and money investment. a set of thirty small picture cards was drawn. 4. Evaluations require “experts. Based on information from earlier studies. which their experience in the workshop significantly changed: 1. In Lesotho.
. average. a greater impact than a traditional evaluation. 64. failed to hold water. and a nice house. the inclusion of a participatory M&E approach “did not disrupt the status quo. Though many of the CCIC projects were designed traditionally.
Source: Deepa Narayan.
Coupal and Simoneau. 3. 2. they then recategorized the cards cased on the gender of the head of household.”5 In fact participatory approaches to M&E allowed NGO personnel to appreciate its effectiveness in all aspects of project design. Feeling no need for secrecy. 15.” Finally. to respond to donor priorities.” The first of these. thus identifying wealth differences with gender. a way was found to put community knowledge to use without resorting to socioeconomic surveys. Facilitators declared after the participatory M&E training that “it really changed the way that [they] saw development. (2) take the same amount of time. chickens. 207). and (3) purchase. They then picked three indicators for each category of wealth.
Mark “T” next to the ones you consider typical of traditional evaluation.
Activities. What project do you want to monitor and evaluate? What knowledge or expertise of this project do you bring to the table? What knowledge and expertise would you like to see involved in monitoring and evaluation. and “P” next to the ones you consider participatory. and who else might you engage in your M&E process to add this?
Activity 1. 12.
. _____Conducted by experts _____Empowers local people to take action _____Focuses on scientific objectivity _____Open results _____Frequent small evaluations _____Conducted by community members _____Results have limited access _____Indicators set by inputs and outputs _____Procedures simple and adapted to local culture _____Conducted to determine accountability _____Takes place at the end of a project _____Procedures are uniform and complex
Source: Narayan. What are three words that come to mind when you think of evaluation? List them here.
Here is a list of evaluation characteristics.3.
the community. Brazil6 This lesson is divided into two parts: 1. After local meetings are completed in all the regions. and by associations and groups such as trade unions. Introduction and the case of Porto Alegre 2. One of the best examples in terms of participatory budgeting. which may ultimately fail because it does not reflect or respond to local priorities. is a complicated and lengthy process. 1995). Porto Alegre implemented participatory budgeting in 1988. 7 Participatory Budget. The purpose of the small-scale meetings is to elect representatives for regional meetings.
. “Participation. and the Formation of a Modern Polity: The Case of ‘Participatory Budgeting’ in Porto Alegre. participatory budgeting consists of a series of open meetings held March through June by the city. Municipal City Hall of Porto Alegre (Municipal Department of Culture. Introduction and the case of Porto Alegre Setting policy on a broad scale. and to set the budgetary agenda to be promoted by the municipality at the regional meeting.Case Study 2: Porto Alegre. Coordination of Social Communications. at great effort. Activities to become acquainted with participatory monitoring 1.7 Now over ten years old. such as deciding the budget of a region. but also that it improved local conditions. representatives meet as a Council to set the budget. Porto Alegre reports the following successes since the inception of participatory budgeting:
The information in this case study is drawn from Zander Navarro. Porto Alegre’s system is an instrument that combines direct democracy with the empowerment and promotion of parliamentary representation. participatory practices and approaches prevent a local administration from designing a policy. Brazil (198901998)” (Development 41: 3 [Sept ‘98]: 68-71).
Citizen participation in monitoring and evaluating the activities of local administrative organizations helps to ensure four things: Transparency: the clarity and accessibility of local administration Accountability: tasks are done and money is spent as planned Responsiveness: objectives are met Equity: benefits reach the locality fairly
The example of Porto Alegre suggests that participatory monitoring not only relieved local communities from some of the severe effects of the macroeconomic crisis experienced during Brazil’s process of decentralization in 1988. during the first challenging year of the new decentralized administration in Brazil. Democratizing Practices.
to inform the community about their actions. and to receive new requests The Council establishes the next year’s budget and it is approved by elected city councillors
Citizens in the region were able to identify and prioritize their needs. intermediate meetings.
. Confronting a better-informed and more demanding citizenry. and to reclaim political relations in a formerly corrupt system. poor neighborhoods are allocated a greater share of resources. is this: • • • • • • The region is divided into 16 areas for a more representative system The Participatory Budget Council includes two representatives from each area Municipal meetings. By holding open meetings and making the budget every citizen’s business. and thematic meetings.•
Considerable reduction in widespread corrupt behavior and administrative malpractice. co-manage municipal resources. the main source of revenue in the region.e. In effect. i. Porto Alegre. in brief. to discuss and decide public works. the processes of allocation and distribution are less susceptible to bribery. which encourages even greater levels of civic engagement and participation. to allocate funds more fairly overall. wealthier neighborhoods are now taxed at a higher rate than poorer neighborhoods. graft. are held to discuss and decide public works decisions and budget priorities. and transparency. Considerable reduction in clientelism (or patronage). where the municipality should invest The Council delegates attend the Forum of Delegates for the region. More equitable.
The participatory budget practice in Porto Alegre offers a way to redistribute resources to the poor. accountability. Correspondingly. well versed in the new “rules of the game. Open meetings mean open books. or other “backstage” arrangements between civil servants and private interests. in keeping with their greater neediness. Reduced corruption also increases public confidence in local government.” city councilors and would-be candidates notably improved in unfair practices of awarding patronage. “progressive” distribution of taxes and resources.8
Participatory Budget. Property tax. organized by the community. control and evaluate the mayor’s commitments. and other topics raised by community leadership Local delegates attend the Participatory Budget Council to discuss and decide the priorities of the region. and create opportunities for participation. The process. was reassigned on a progressive scale.
___Roads: Paved. traffic lines recently painted.2. or parks within reasonable distance
Which public service received the lowest grade?_______________________ Which public service received the highest grade?______________________ What specific things. cinemas. building and other permits are issued without difficulty ___Lighting: Working street lights ___Sanitation: Streets cleared of litter. trees plentiful and in good condition ___Buildings: No abandoned buildings. in your opinion. recycling pick-up available ___Motor vehicles: No abandoned cars. caused the worst public service to perform poorly? What caused the best public service to perform well?
Using the M&E Learning Wheel. museums. traffic lights work ___Other: Public telephones. no potholes. mailboxes. garbage collected regularly. newspaper machines available and in good working condition ___Safety: police and fire departments come when they’re called. What is the quality of public services in your area? Give each one a letter grade from A to F. parking regulations enforced. crime not severe ___Resources: food and other stores are within reasonable distance ___Amenities: libraries. how would you describe the “baseline” of one of these two public services?
. stray dogs collected regularly. Activities Activity 1.
Do you think a participatory budget process could work in your community? Why or why not?
Consult the M&E Learning Wheel.Activity 2. Which steps of the Wheel would present the greatest challenge for doing participatory budgeting? Which steps would be the most useful?
Although this situation was a disincentive for more committed involvement in the project by both government and communities.Field Insight: Nigeria In Nigeria. the participants were each asked to list who made decisions about resources for different project activities and how those decisions were reached. technicians. and local government management and extension staff.
Source: Narayan. 57. The subsequent discussion revealed that while the project was trying to support greater autonomy on the part of local governments in the planning and implementation of rural water and sanitation systems. In attendance were the project coordinator. the need to account for externally provided public funds and the pressure placed by donors to achieve construction targets in a timely fashion made it difficult for project managers to contemplate an alternative approach. a workshop was held to evaluate the budgetary process for a large externally-financed rural water and sanitation project. In the informal atmosphere. This was done to ensure accountability and to avoid mismanagement of funds. the schedule of project activities was also beyond the control of community committees. in effect the main decisions regarding disbursement of funds were made by the project coordinator.
. Similarly. while the project strove to encourage community participation and self-determination. but it also had the unintended effect of reducing the sense of ownership and control over the project by the local government administration.
Participatory planning and evaluation activities 1.Case Study 3: Sri Lanka9 This lesson is divided into two parts: 1. education. and the introduction of subsidies for certain crops. indicators. and the Second Rural Development Project. especially in setting indicators. and methods.” or top-down. This case study is about a project that had limited success. The example of Sri Lanka offers a lesson in how participatory monitoring and evaluation--which is typically added onto development projects at their conclusion--could have significantly transformed project results and impacts if it had been integrated into the project from the start. nor did beneficiaries necessarily want what was offered. project managers learned that participatory methods would have improved the and outcome of the entire project. Importantly. when the defects of the projects became clear. In Sri Lanka in the 1970s. The disappointment of the rural development aims. during participatory evaluation.org/oed/inter. rural development projects were generally not designed with participatory approaches in mind. who take a greater interest in the continuation and maintenance of the assets and processes that the project seeks to improve. the expansion of credit services to farmers. alongside the successes of participatory M&E.
. The project did not necessarily offer useful tools. (The rates of return on agricultural products were optimistically estimated to be 24 to 34 percent. It can also include investment in local health care. “Sri Lanka: Kurunegala Rural Development Project and Second Rural Development Project” (http://wbln0018. a key component of project design is participation by beneficiaries. or KRDP.worldbank. Such participation develops a sense of ownership among stakeholders. and the evaluation of these projects suggested that the difficulty arose from the use of a “blueprint. transportation. or SRDP).html). according to its traditionally-set goals.)
The information in this case study is drawn from John English. and electrification. water supply. community organizations. Those on the receiving end of rural development see the greatest benefit when a project is planned around their contributions and involvement and they are participants in the whole process. the World Bank funded two major rural development projects (the Kurunegala Rural Development Project. government workers) were consulted only during the evaluation process.nsf/htmlmedia/publications. offer strong evidence of the potential this approach to project design. Introduction and the case of Sri Lanka
Rural development includes projects such as equipment repair and replacement. The two Sri Lankan projects enjoyed mixed successes. but were actually only 10 percent. To make rural development sustainable. Local beneficiaries (farmers. At that time. model for project design. households. Introduction and the case of Sri Lanka 2.
The latter turned out to be far more useful to the community.The Sri Lankan experience provided valuable lessons about project design. Although the project sought to diversify crops in the district and increase agricultural production. When farmers were involved in all stages of irrigation work. again recommended “from outside. roads) were not regularly maintained. Here are some of the findings of the Sri Lankan KRDP and SRDP: • Project planners did not consider how to sustain benefits after projects closed.to two-year rolling plans. rather than those imposed from outside. which have since guided rural development in that country. and raised the average household income some 30 percent—a far more significant impact than the income increases from improved farming output (10 percent).” were not followed by farmers. and schools significantly benefited the community. When they were not. farmers reported that the greatest benefit of the project was the construction of roads.
The Sri Lankan project evaluators drew two principles from these data: • Participation by stakeholders at all stages. and unintentionally provided another. Beneficiaries. User groups. Travel.
. and so the assets created (irrigation systems. medical facilities. User groups succeed when they are initiated by the users themselves. The project had one objective. to keep the project in line. were not part of the project. Differing conditions in a variety of rural areas call for a processoriented. learn-by-doing approach. and did not participate. especially monitoring and evaluation. participatory monitoring is a management tool. replacing footpaths. job opportunities. It is essential that M&E be conducted using truly significant variables. Management practices in irrigation. and conflicts with contractors were minimal. versus the significant impact of building roads) demonstrates the importance of relying on local values for M&E. Flexibility of design. and access to markets. and prevent it from wasting resources on assets and procedures that are not useful to the community. Variation between the intended outcome and actual outcome (the modest increase in agricultural output. structures were properly maintained. following one. who perceived a limited benefit in doing so. who were left out of the planning process. In this approach. quality control suffered. Farmers saw user groups promoted by the government as outside impositions. did not take on a sense of ownership of these assets. which are crucial to operation and maintenance in rural development.
the implementation of one project or goal means that there are fewer resources left for others. clarified ownership rights. the water was flowing again.Field Insight: Indonesia Lack of information.” When this information was conveyed to the senior government officials in the provincial capital. Pak Kake. We have not repaired the pump because we are waiting for the chief administrator to give us permission. Finite resources mean that it is necessary to When a group takes thoughtful and realistic action as a result of M&E. When a participatory evaluation team came to the village. it addresses
. it knows what it is measuring. 62. Given that all activities have their own costs including money. described earlier in this Module. and labor.
Source: Narayan. When a group establishes shared
goals for M&E. He said. and Take action. This section offers an activity for learning and enhancing the common thread of these three steps in the PM&E Wheel: determining and setting priorities. time. verified Pak Kake’s observations. “I have measured with a rope and the water level has fallen to 52 meters. Activity: A tool for participatory planning
The Participatory Monitoring and Evaluation Learning Wheel. problems more effectively. a pump remained broken for six months because the government officials in the provincial capital assumed that the well had run dry. Within hours. and worked with the water group to fix the pump. includes these important steps: Establish goals. an elderly man whose love of tinkering was well known. was asked why the well was dry. it knows what success looks like. confidence. and clarity about who owns facilities can prevent corrective action from being taken. When a group develops shared indicators for M&E. trust.
2. a technical team visited the site. whereas the cylinder hangs at 45 meters. In a rural water project in Indonesia.
and to prioritize them: Goals (fill in what your group is prioritizing ) Beneficiari es (positive and negative) Urgency of goal Resource s required Impact of not pursuing this goal Ranking of this goal
. The following table can be used to compare the merits of various goals of the group.prioritize your goals and your indicators. Important factors in setting priorities include: • the urgency and severity of the problem • the possibility the problem could be solved with existing resources • the number and types of stakeholders who will be either positively or negatively affected by the project or activity • the community’s participation • the budget (amount and source) • the sustainability of the development which would result from the project/activity The aim of this section is to provide trainees with basic knowledge and skills in prioritizing. with emphasis on the participation of the community and stakeholders.
by inserting “actions” into the first column.(The table can be adapted to take action.)
A table for developing indicators might look like this: Indicators Who can and can't use this indicator? How much Resource participati on does indicator call for? s required Is it quantitive or qualitative ? Ranking of this indicator
Make a list of some of the projects in your community that are reported there.4: Prioritization of a project/activity Take a copy of today’s newspaper. In a small group. Use the chart above to evaluate those two projects. choose two projects from that list.
.Activity 7. What did each small group discover about its chosen projects? Report your findings to the class.
Jot down some ideas for steps you could take to complete each phase of the process cycle.ACTION PLANNING Here you have some time to work through the stages of the Participatory Monitoring and Evaluation Process Cycle in the context of a project or policy which affects your community. Consider ideas you have drawn from case studies and adapt them to your own needs/context. • • • • • • • • • • • • Conduct a readiness assessment Decide who participates.SECTION VI: EXERCISE -. establish principles for working together Establish goals Develop indicators Gather info (quantitative and qualitative) Construct baseline Set realistic targets Gather info again (quantitative and qualitative) Analyze results including all stakeholders Share info/results with all relevant parties Take action – use the findings! Start over (making any needed modifications in process…)
SECTION VII: ADDITIONAL RESOURCES • • On-Line Other
Brazil) Household Surveys (Guatemala.APPENDIX A: MONITORING MECHANISMS • • • • Report Cards (Ukraine. Tanzania) Citizen Dialogues/For a (Brazil)
. Philippines) E-Gov (India.
Mosaic Intenrational Training in PM&E Ottawa. UNDP.worldbank. Empowerment Evaluation: Collaboration. New Society Publishers.org/inprint/conference/fetterman. Participatory Evaluation in programmes involving governance decentralization. 1995. 1994. The Action Evaluation Research Institute. Sam Kaner et al. Lyra Srinavasan et al.deakin.org/participation/partme. and a Case Example. PRSP Sourcebook Chapter on Monitoring and Evaluation. PROWWESS/UNDP Technical Series.htm 17. July 1995 http://www. Participatory Monitoring. Evaluation Paradigms: four generations of evaluation (short review of Guba and Lincoln's Fourth Generation Evaluation charting the evolution of evaluation thinking) http://www2. World Bank Participatory Monitoring and Evaluation Website: http://www. Michael Bamberger 2.co. Participatory Evaluation: Tools for Managing Change in Water and Sanitation. Facilitator’s Guide to Participatory Decision-Making. DfID/Actionaid.org/dac/htm/pubs/p-pdggev. 3. Larry Salmen
.htm 18.org/poverty/strategies/chapters/monitoring/moneval. http://www. Action Research. Go to: www. 19993. Community Participation Strategies and Tools: A Trainers’ Manual for the Rural Water Supply and Sanitation Sector in Pakistan. John Gaventa.dfid. Washington D.•
1. World Bank. Lyra Srinavasan. World Bank. DAC http://www. Tools for Community Participation: A Manual for Training Trainers in Participatory Techniques. 13. PACT 21. ODA/DfID. 14. 1990. World Bank 11.oecd.html 5. 9.gov. IDS. 6. 1996. 12. Jesko Hentschel 7. Soniya Carvalho. David Fetterman.uk/public/what/advisory/group7/pdf/sddstak2. http://www. Evaluation of Programmes Promoting Participatory Development and Good Governance: Synthesis Report.htm 15.pdf 4.C.aepro. The Contribution of People’s Participation: Evidence from 121 Rural Water Supply Projects. UNDP and Govt of Pakistan. Stanford University. 1994.au/dcad/FTL/evaluation/paradigms/paradigms. Deepa Narayan. Course for Establishing M & E Systems in Government. Indicators for monitoring poverty reduction World Bank Discussion Paper 254. Guidance note on indicators for measuring and assessing primary stakeholder participation.htm 16.worldbank. 8.mande. Evaluation and Reporting: Online Manual. 2001. 20. Deepa Narayan. Canada 10.uk/guidance. Guidance note on Increasing the Participation of the Poor in the Assessment of the Impact of Development Interventions.htm 19.edu. Kusek and Rist 2001.