Monitoring & Evaluation

what why ways

Monitoring and Evaluation of Development Programs

K.V.Subramanian Prime Consulting Group Bangalore

www.primeconsultinggroup.net +919845274369 +918024647518

Prime Consulting Group, Bangalore

1

Monitoring & Evaluation

what why ways

TABLE OF CONTENTS
1. MONITORING AND EVALUATION.................................. ..........5 1.1. Some Tools, Methods and Approaches for Monitoring and Evaluation.............................................................................................6 2. PERFORMANCE INDICATORS................................................7 2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 4.1. 4.2. 4.3. 4.4. 4.5. 4.6. 5.1. 5.2. 5.3. 5.4. 5.5. 6.1. 6.2. 6.3. 6.4. Uses of Performance indicators ....................................................8 Advantages of Performance Indicators...........................................8 Limitations of performance indicators............................................8 Cost of Monitoring..........................................................................9 Skills Required in Monitoring..........................................................9 Time Required for Monitoring.........................................................9 What can we use it for?..................................................................9 Advantages..................................................................................10 Disadvantages.............................................................................10 Cost..............................................................................................10 Skills Required..............................................................................10 Time Required..............................................................................10 What can we use it for?................................................................11 Advantages..................................................................................11 Disadvantages.............................................................................11 Cost..............................................................................................12 Skills Required..............................................................................12 Time Required..............................................................................12 What can we use them for?..........................................................12 Advantages..................................................................................13 Disadvantages.............................................................................13 Cost..............................................................................................13 Skills Required..............................................................................13 What can we use them for?..........................................................13 Advantages..................................................................................14 Disadvantages.............................................................................14 Cost..............................................................................................14

3. THE LOGICAL FRAMEWORK APPROACH.................................9

4. THEORY-BASED EVALUATION..............................................10

5. FORMAL SURVEYS............................................... ..............12

6. RAPID APPRAISAL METHODS...................................... ........13

Prime Consulting Group, Bangalore

2

Monitoring & Evaluation

what why ways

6.5. Skills Required..............................................................................14 6.6. Time Required..............................................................................14 7. PARTICIPATORY METHODS.............................................. ....15 7.1. 7.2. 7.3. 7.4. 7.5. 8.1. 8.2. 8.3. 8.4. 8.5. 8.6. 9.1. 9.2. 9.3. 9.4. 9.5. What can we use them for?..........................................................15 Advantages..................................................................................15 Disadvantages.............................................................................15 Cost..............................................................................................15 Skills Required..............................................................................16 What can we use them for?..........................................................16 ADVANTAGES................................................................................16 Disadvantages.............................................................................16 Cost..............................................................................................17 Skills Required..............................................................................17 Time Required..............................................................................17 What can we use it for?................................................................18 Advantages..................................................................................18 Disadvantages.............................................................................19 Skills Required..............................................................................19 Time Required..............................................................................19

8. COST-BENEFIT AND COST-EFFECTIVENESS ANALYSIS............16

9. IMPACT EVALUATION.........................................................17

10. EXAMPLES OF IMPACT EVALUATION DESIGNS....................19 11. RAPID EVALUATION .................................................. .......20 12. COMPREHENSIVE SPENDING REVIEWS .............................21 13. UTILITY OF M&E.................................. ............................21 14. PERFORMANCE BUDGETING.............................................23 15. PREREQUISITE TO INSTITUTIONALIZATION OF M&E............25 16. INCENTIVES FOR M&E........................................ ..............27 17. KEY DRIVERS TO SUSTAINABLE M&E.................................28 18. M&E CHAMPION.............................. ................................29 19. OVER-ENGINEERING THE M&E SYSTEM....................... .......30 20. THE VALUE FROM M&E SYSTEMS......................................32 21. STRUCTURAL ARRANGEMENTS FOR M&E SYSTEM..............33 22. HOW TO STRENGTHEN A GOVERNMENT M&E SYSTEM........35 23. GLOSSARY OF TERMS USED..................................... .........35 23.1. Logical framework (Logframe)...................................................35 23.2. Indicator.....................................................................................36

Prime Consulting Group, Bangalore

3

Monitoring & Evaluation

what why ways

23.3. Inputs.........................................................................................36 23.4. Accountability............................................................................36 23.5. Activity.......................................................................................36 23.6. Data collection tools...................................................................36 23.7. Base-line study...........................................................................37 23.8. Benchmark.................................................................................37 23.9. Beneficiaries...............................................................................37 23.10. Effect........................................................................................37 23.11. Effectiveness............................................................................37 23.12. Efficiency..................................................................................38 23.13. Feedback..................................................................................38 23.14. Finding......................................................................................38 23.15. Evaluation................................................................................38 23.16. Goal..........................................................................................39 23.17. Impacts....................................................................................39 23.18. Independent evaluation...........................................................39 23.19. Indicator...................................................................................39 23.20. Inputs.......................................................................................39 23.21. Lessons learned.......................................................................39 23.22. Mid-term evaluation.................................................................40 23.23. Monitoring................................................................................40 23.24. Outcome..................................................................................40 23.25. Outputs....................................................................................40 23.26. Participatory evaluation...........................................................40 23.27. Partners....................................................................................40 23.28. Performance.............................................................................41 23.29. Performance indicator..............................................................41 23.30. Performance measurement......................................................41 23.31. Performance monitoring...........................................................41 23.32. Process evaluation...................................................................41 23.33. Program evaluation..................................................................41 23.34. Project evaluation....................................................................42 23.35. Results......................................................................................42 23.36. Results chain............................................................................42 23.37. Results framework...................................................................42 23.38. Results-based management (RBM)..........................................42 23.39. Target group.............................................................................42 23.40. Triangulation............................................................................43 23.41. Stakeholders............................................................................43

Prime Consulting Group, Bangalore

4

Monitoring & Evaluation 1. MONITORING AND EVALUATION

what why ways

Monitoring is the process of tracking the progress of achievement of results from any developmental initiative. Monitoring encompasses the activities involved in the conceptualization, planning, data collection, analysis and drawing inferences, disseminating the findings and drawing up the action plans to overcome shortcomings that have unfolded from the monitoring process. Monitoring helps in continuously understanding the route to realising the targets and devising appropriate timely measures to ensure end result objectives are achieved. Monitoring is a collective and inclusive process involving the program sponsor, the monitoring experts / agency, the program implementers, the beneficiaries of the program and any other stakeholders. Monitoring is based on qualitative and quantitative data on various parameters of progress defined a priori, interactive processes between the monitoring expert and the stakeholders individually, a collective process of stocktaking of the progress parameters and casual factors, consensus building and drawing up remedial measures. Monitoring is done at periodic intervals. Monitoring will cover both inputs to the intervention, processes as well as the outputs expected, Monitoring calls for clear definition of the intervention, the objective, scope, approach, tasks to be carried out within defined timelines / to achieve certain targets, progressive results to be expected over the duration of the intervention and measured at periodic intervals, measures for results from the intervention, definition of data on various facets (inputs, processes, outputs, results, externalities,…), and analysis to track progress of achievement, collection of the data as per predefined formats and methods, analysis of the data, comparison with pre-stated targets for achievements, assessing progress in terms of shortfalls/exceeding targets, analysis of the causal factors for various levels of performance, discussing and disseminating the remedies and follow the process of tracking the same.

Prime Consulting Group, Bangalore

5

Monitoring & Evaluation

what why ways

Evaluation is normally a one or several time activity at end of certain predefined logical milestones / points of intervention. In contrast to monitoring, evaluation looks at only the achievement of the predefined objectives in terms of outputs and outcomes, and does not address the inputs / processes, that are covered in Monitoring. Evaluation can be a mid course evaluation, end project evaluation or other predefined milestones. While monitoring helps in improving the program from an implementation perspective, evaluation also addresses the strategies adopted to achieve certain outcomes over a period of time. Monitoring looks at the inputs and processes, evaluation only addresses the achievement of initial objectives in terms of outputs and outcomes.
1.1.

Some Tools, Methods and Approaches for Monitoring and Evaluation

Monitoring and evaluation (M&E) of developmental activities provides government officials, development managers, and civil society with better means for learning from the past experience, improving service delivery, planning and allocating resources efficiently, and demonstrating results as part of accountability to key stakeholders. Within the development community there is a strong focus on results, evidence based, which explains the growing interest in M&E. Overview of a sample of M&E tools, methods, and approaches outlined here, include their purpose and use; advantages and disadvantages; costs, skills, and time required for their deployment. The M&E Overview discusses: • • • • • Performance indicators The logical framework approach Theory-based evaluation Formal surveys Rapid appraisal methods 6

Prime Consulting Group, Bangalore

Monitoring & Evaluation • • • • Participatory methods Public expenditure tracking surveys Cost-benefit and cost-effectiveness analysis Impact evaluation

what why ways

This list is not comprehensive, nor is it intended to be one. Some of these tools and approaches are complementary; some are substitutes. Some have broad applicability, while others are quite narrow in their uses. The choice of which is appropriate for any given context will depend on a range of considerations. These include the uses for which M&E is intended, the main stakeholders who have an interest in the M&E findings, the speed with which the information is needed, and the cost.

2. PERFORMANCE INDICATORS Performance indicators are measures of inputs, processes, outputs, outcomes, and impacts for developmental projects, programs, or strategies. When supported with sound data collection—perhaps involving formal surveys—analysis and reporting, indicators enable managers to track progress, demonstrate results, and take corrective action to improve service delivery. Participation of key stakeholders in defining indicators is important, because they are then more likely to understand and use indicators for management decision- making. Performance Indicators can be at very high level, in terms of measuring the government’s performance relative to the (Millenium Development Goals) MDGs or the national development plan, or in terms of ministry and agency activities and outputs. They are useful for setting performance targets, for assessing progress towards achieving them, and for comparing the performance of different organizations. They are a relatively inexpensive means of measuring government performance on a frequent basis. Although performance indicators can be used to identify problems, thus allowing corrective action to be Prime Consulting Group, Bangalore 7

Monitoring & Evaluation

what why ways

taken, a limitation is that they do not usually reveal whether government actions led to improved performance. They can, however, be used to flag the need for a follow-up review or evaluation of an issue. A common danger with performance which can lead to poor-quality data.
2.1.

indicator systems is over

engineering the system by including too many underutilized indicators;

Uses of Performance indicators

Performance indicators are used for • • •
2.2.

Setting performance targets and assessing progress toward achieving them. Identifying problems via an early warning system to allow corrective action to be taken. Indicating whether an in-depth evaluation or review is needed. Advantages of Performance Indicators

Performance indicators enable as tools for • • Effective means to measure progress toward achievement of objectives. Facilitates benchmarking and comparisons among different organizational units, Geographical units (districts), and over time.
2.3.

Limitations of performance indicators Poorly defined indicators are not good measures of success. Tendency to define too many indicators, or those without accessible data sources, making the system costly, impractical, and likely to be underutilized.

• •

Often there is a trade-off between picking the optimal or desired indicators and having to accept the indicators which can be measured using existing data or data that can be generated.

Prime Consulting Group, Bangalore

8

Monitoring & Evaluation
2.4.

what why ways

Cost of Monitoring

Can range from low to high, depending on number of indicators collected, the frequency and quality of information sought, and the comprehensiveness of the system
2.5.

Skills Required in Monitoring

Several days of training may be needed to develop skills for defining practical indicators. Data collection, analysis and reporting skills, and management information system (MIS) skills are required to implement performance monitoring systems.
2.6.

Time Required for Monitoring

Several days to several months, depending on extent of participatory process used to define indicators and program complexity.

3. THE LOGICAL FRAMEWORK APPROACH The logical framework (LogFrame) helps to clarify objectives of any project, program, or policy. It aids in the identification of the expected causal links—the “program logic”—in the following results chain: inputs, processes, data, verifiability, outputs (including coverage or “reach” across beneficiary groups), outcomes, and impact. It leads to the identification of performance indicators at each stage in this chain, as well as risks which might impede the attainment of the objectives. The LogFrame is also a vehicle for engaging partners in clarifying objectives and designing activities. During implementation, the LogFrame serves as a useful tool to review progress and take corrective action.
3.1.

What can we use it for?

The Logframe approach can be used for • Improving quality of project and program designs—by requiring the specification of clear objectives, the use of performance indicators, and assessment of risks.

Prime Consulting Group, Bangalore

9

Monitoring & Evaluation • • • Summarizing design of complex activities.

what why ways

Assisting the preparation of detailed operational plans. Providing objective basis for activity review, monitoring, and evaluation.

3.2.

Advantages Ensures that decision-makers ask fundamental questions and analyze assumptions and risks. Engages stakeholders in the planning and monitoring process. When used dynamically, it is an effective management tool to guide implementation, monitoring and evaluation.

• • •

3.3.

Disadvantages If managed rigidly, stifles creativity and innovation. If not updated during implementation, it can be a static tool that does not reflect changing conditions. Training and follow-up are often required. Cost

• • •
3.4.

Low to medium, depending on extent and depth of participatory process used to support the approach.
3.5.

Skills Required

Minimum 3–5 days training for facilitators; additional facilitation skills required for use in participatory planning and management.
3.6.

Time Required

Several days to several months, depending on scope and depth of participatory process.

4. THEORY-BASED EVALUATION Theory-based evaluation has similarities to the LogFrame approach but allows a much more in-depth understanding of the workings of a Prime Consulting Group, Bangalore 10

Monitoring & Evaluation

what why ways

program or activity—the “program theory” or “program logic.” In particular, it need not assume simple linear cause-and effect relationships. For example, the success of a government program to improve literacy levels by increasing the number of teachers might depend on a large number of factors. These include, among others, availability of classrooms and textbooks, the likely reactions of parents, school principals and schoolchildren, the skills and morale of teachers, the districts in which the extra teachers are to be located, the reliability of government funding, and so on. By mapping out the determining or causal factors judged important for success, and how they might interact, it can then be decided which steps should be monitored as the program develops, to see how well they are in fact borne out. This allows the critical success factors to be identified. And where the data show these factors have not been achieved, a reasonable conclusion is that the program is less likely to be successful in achieving its objectives.
4.1.

What can we use it for? Mapping design of complex activities. Improving planning and management. Advantages Provides early feedback about what is or is not working, and why. Allows early correction of problems as soon as they emerge. Assists identification of unintended side-effects of the program. Helps in prioritizing which issues to investigate in greater depth, perhaps using more focused data collection or more sophisticated M&E techniques.

• •
4.2.

• • • •


4.3.

Provides basis to assess the likely impacts of programs. Disadvantages Can easily become overly complex if the scale of activities is large or if an exhaustive list of factors and assumptions is assembled.

Prime Consulting Group, Bangalore

11

Monitoring & Evaluation •

what why ways

Stakeholders might disagree about which determining factors they judge important, which can be time-consuming to address.

4.4.

Cost

Medium—depends on the depth of analysis and especially the depth of data collection undertaken to investigate the workings of the program.
4.5.

Skills Required

Minimum 3–5 days training for facilitators.
4.6.

Time Required

Can vary greatly, depending on the depth of the analysis, the duration of the program or activity, and the depth of the M&E work undertaken.

5. FORMAL SURVEYS Formal surveys can be used to collect standardized information from a carefully selected sample of people, households, stakeholders, beneficiaries…. Surveys often collect comparable information for a relatively large number of people, in particular target groups.
5.1.

What can we use them for? Providing baseline data against which the performance of the strategy, program, or project can be compared. Comparing different groups at a given point in time. Comparing changes over time in the same group. Comparing actual conditions with the targets established in a program or project design. Describing conditions in a particular community or group. Providing a key input to a formal evaluation of the impact of a program or project.

• • • • • •

Prime Consulting Group, Bangalore

12

Monitoring & Evaluation
5.2.

what why ways

Advantages Findings from the sample of people interviewed can be applied to the wider target group or the population as a whole. Quantitative estimates can be made for the size and distribution of impacts.

• •

5.3.

Disadvantages The processing and analysis of data can be a major bottleneck for the larger surveys even where computers are available. Surveys are expensive and time-consuming. Many kinds of information are difficult to obtain through formal interviews.

• • •

5.4.

Cost

Ranges widely. Costs will be significantly higher if there is no master sampling frame for the country.
5.5.

Skills Required

Sound technical and analytical skills for sample and questionnaire design, data analysis and processing.

6. RAPID APPRAISAL METHODS Rapid appraisal methods are quick, low-cost ways to gather the views and feedback of beneficiaries and other stakeholders. These views provide rapid information for management decision making, especially at the activity or program level. Methods include interviews, focus group discussions, direct observation, and mini-surveys. It can be difficult to generalize the findings from these qualitative methods. They are also less valid, reliable, and credible than formal surveys.
6.1.

What can we use them for?

Prime Consulting Group, Bangalore

13

Monitoring & Evaluation • •

what why ways

Providing rapid information for management decision-making, especially at the project or program level. Providing qualitative understanding of complex socioeconomic changes, highly interactive social situations, or people’s values, motivations, and reactions.

Providing context and interpretation for quantitative data collected by more formal methods.

6.2.

Advantages Low cost. Can be conducted quickly. Provides flexibility to explore new ideas. Disadvantages

• • •
6.3.

• •
6.4.

Findings usually relate to specific communities or localities—thus difficult to generalize from findings. Less valid, reliable, and credible than formal surveys. Cost

Low to medium, depending on the scale of methods adopted.
6.5.

Skills Required

Non-directive interviewing, group facilitation, field observation, notetaking, and basic statistical skills.
6.6.

Time Required

Four to six weeks, depending on the size and location of the population interviewed and the number of sites observed.

Prime Consulting Group, Bangalore

14

Monitoring & Evaluation 7. PARTICIPATORY METHODS

what why ways

Participatory methods provide active involvement in decision-making for those with a stake in a project, program, or strategy and generate a sense of ownership in the M&E results and recommendations.
7.1.

What can we use them for? Learning about local conditions and local people’s perspectives and priorities to design more responsive and sustainable interventions.

• • •
7.2.

Identifying problems and trouble-shooting problems during implementation. Evaluating a project, program, or policy. Providing knowledge and skills to empower poor people. Advantages Examines relevant issues by involving key players in the design process. Establishes partnerships and local ownership of projects. Enhances local learning, management capacity, and skills. Provides timely, reliable information for management decisionmaking.

• • • •

7.3.

Disadvantages Sometimes regarded as less objective. Time-consuming if key stakeholders are to be involved in a meaningful way. Potential for domination and misuse by some stakeholders to further their own interests.

• • •

7.4.

Cost

Low to medium. Costs vary greatly, depending on scope and depth of application and on how local resource contributions are valued. Prime Consulting Group, Bangalore 15

Monitoring & Evaluation
7.5.

what why ways

Skills Required

Minimum several days’ training for facilitators.

8. COST-BENEFIT AND COST-EFFECTIVENESS ANALYSIS Cost-benefit and cost-effectiveness analysis are tools for assessing whether or not the costs of an activity can be justified by the outcomes and impacts. Cost-benefit analysis measures both inputs and outputs in monetary terms. Cost-effectiveness analysis estimates inputs in monetary terms and outcomes in non-monetary quantitative terms (such as improvements in student reading scores).
8.1.

What can we use them for? Informing decisions about the most efficient allocation of resources. Identifying projects that offer the highest rate of return on investment.

• •

8.2.

ADVANTAGES Good quality approach for estimating the efficiency of programs and projects. Makes explicit the economic assumptions that might otherwise remain implicit or overlooked at the design stage. Useful for convincing policy-makers and funders that the benefits justify the activity.

• • •

8.3.

Disadvantages Fairly technical, requiring adequate financial and human resources available. Requisite data for cost-benefit calculations may not be available, and projected results may be highly dependent on assumptions made.

• •

Prime Consulting Group, Bangalore

16

Monitoring & Evaluation •

what why ways

Results must be interpreted with care, particularly in projects where benefits are difficult to quantify.

8.4.

Cost

Varies greatly, depending on scope of analysis and availability of data.
8.5.

Skills Required

The procedures used in both types of analyses are often highly technical. They require skill in economic analysis and availability of relevant economic and cost data.
8.6.

Time Required

Varies greatly depending on scope of analysis and availability of data.

9. IMPACT EVALUATION Impact evaluation is the systematic identification of the effects – positive or negative, intended or not, from the intervention and the environment, caused by a given development activity such as a program or project. Impact evaluation helps us better understand the extent to which activities reach the target segments and the magnitude of their effects on people’s welfare. Impact evaluations can range from large scale sample surveys in which project populations and control groups are compared before and after, and possibly at several points during program intervention; to small-scale rapid assessment and participatory appraisals where estimates of impact are obtained from combining group interviews, key informants, case studies and available secondary data. Impact evaluation focuses on the outcomes and impacts of

government activities. Rapid evaluation methods can be used to estimate impact, but more sophisticated methods of impact evaluation can provide much more reliable and persuasive findings. Such methods entail the comparison of program beneficiaries with control or 17

Prime Consulting Group, Bangalore

Monitoring & Evaluation

what why ways

comparison groups at two or more points in time. Advanced statistical techniques are used to identify the precise impacts of the program on beneficiaries. This type of evaluation is highly demanding in terms of data and can be very expensive; however, there are ways this cost can be reduced significantly. It is necessary to plan such evaluations well in advance of when their findings will be needed, as it is usually not possible to conduct them quickly. Although sophisticated evaluations are often expensive, they can be highly cost effective when they lead to even marginal improvements in program performance.
9.1.

What can we use it for? Measuring outcomes and impacts of an activity and distinguishing these from the influence of other, external factors. Helping to clarify whether costs for an activity are justified. Informing decisions on whether to expand, modify or eliminate projects, programs or policies. Drawing lessons for improving the design and management of future activities. Comparing the effectiveness of alternative interventions. Strengthening accountability for results.

• • • • • •

9.2.

Advantages Provides estimates of the magnitude of outcomes and impacts for different demographic groups, regions or over time. Provides answers to some of the most central development questions – to what extent are we making a difference? What are the results on the ground? How can we do better?

• •

Systematic analysis and rigor can give managers and policymakers added confidence in decision-making.

Prime Consulting Group, Bangalore

18

Monitoring & Evaluation
9.3.

what why ways

Disadvantages Some approaches are very expensive and time-consuming, although faster and more economical approaches are also used. Reduced utility when decision-makers need information quickly. Difficulties in identifying an appropriate counter-factual. Skills Required

• • •
9.4.

Strong technical skills in social science research design, management, analysis and reporting. Ideally, a balance of quantitative and qualitative research skills on the part of the evaluation team.
9.5.

Time Required

Can take up to 2 years or more. Rapid assessment evaluations can often be conducted in less than 6 months.

10.EXAMPLES OF IMPACT EVALUATION DESIGNS Randomized evaluation designs, involving the collection of information on project and results-based management or other ways of using M&E information offer some sort of panacea for improving government performance. M&E is necessary to achieve evidence-based policy making, evidencebased management, and evidence based accountability. This is why most of the member countries of the Organisation for Economic Cooperation and Development (OECD) —the richest countries in the world—place considerable emphasis on having and using M&E information in support of sound governance. Some of the key trends influencing countries to give higher priority to M&E are also discussed in paras below. Growing number of governments are working to improve their

performance by creating systems to measure and help them understand their performance. These systems for monitoring and evaluation (M&E),

Prime Consulting Group, Bangalore

19

Monitoring & Evaluation

what why ways

are used to measure the quantity, quality, and targeting of the goods and services— the outputs—that the state provides, and to measure the outcomes and impacts resulting from these outputs. These systems are also a vehicle to facilitate understanding of the causes of good and poor performance. There is currently a great deal of emphasis on results and on being able to demonstrate performance. One challenge faced by government and donor evaluation officers alike, is to ensure that the evaluations they produce are used intensively. It is now well understood that it is not enough to complete an evaluation report, make it publicly available, and assume that utilization will somehow take care of itself. Instead, individual evaluators and their evaluation offices need to be highly proactive in implementing a detailed strategy for dissemination of not just the evaluation report but its findings as well—to encourage acceptance of the evaluation’s findings and implementation of its recommendations.

11.RAPID EVALUATION This involves a formal review of a government activity or program. For example, rapid evaluations entail desk reviews by external consultants, of a program’s objectives and preparation of a logframe analysis (which maps the causal links between government activities and desired outcomes and impacts). Any available data (including performance indicators) are analyzed to assess the efficiency and effectiveness of the activity. The main strengths of rapid evaluations are their speed and relatively low cost. Their main disadvantages— compared with more rigorous approaches— are their weaker empirical evidence and much weaker data-analysis techniques. They also face difficulty in identifying possible causal relationships between government actions and subsequent improvements in performance.

Prime Consulting Group, Bangalore

20

Monitoring & Evaluation

what why ways

The World Bank uses this approach in many of its ex ante and ex post project evaluations. The Bank also uses a range of additional information in its ex post evaluations, including the findings of supervision missions, key informant interviews, and any primary or secondary data that have been collected.

12.COMPREHENSIVE SPENDING REVIEWS These are a type of policy evaluation. These entail desk reviews of issues of inefficiency and program duplication. The United Kingdom’s biennial spending reviews investigate these issues, as well as program outcomes and government priorities.

13.UTILITY OF M&E M&E can provide unique information about the performance of government policies, programs, and projects. It can identify what works, what does not, and the reasons why. It is tempting—but dangerous—to view M&E as having inherent value. The value of M&E does not come simply from conducting M&E or from having such information available; rather, the value comes from using the information to help improve government performance. As we shall see, there are many governments that systematically use M&E information to improve their performance. Ways M&E information can be highly useful to governments and to others include the following: • To support policy making—especially budget decision making— performance budgeting and national planning. These processes focus on government priorities among competing demands from citizens and groups, in society. M&E information can support government’s deliberations by providing evidence about the most cost-effective types of government activity, such as different types of employment programs, health interventions, or conditional cash transfer payments. Terms that describe the use of Prime Consulting Group, Bangalore 21

Monitoring & Evaluation

what why ways

M&E information in this manner include evidence-based policy making, results-based budgeting, and performance informed budgeting • To help government ministries in their policy development and policy analysis work and in program development. • To help government ministries and agencies manage activities at the sector, program, and project levels. This includes government service delivery and the management of staff. M&E identifies the most efficient use of available resources; it can be used to identify implementation difficulties, for example. Performance indicators can be used to make cost and performance comparisons— performance benchmarking—among different administrative units, regions, and districts. Comparisons can also be made over time that help identify good, bad, and promising practices, and this can prompt a search for the reasons for this performance. Evaluations or reviews are used to identify these reasons. This is the learning function of M&E, and it is often termed results-based or resultsoriented management. • To enhance transparency and support accountability relationships, by revealing the extent to which government has attained its desired objectives. M&E provides the essential evidence necessary to underpin strong accountability relationships, such as of the government to the Parliament or Congress, to civil society, and to donors. M&E also supports the accountability relationships within government, such as between sector ministries and central ministries, among agencies and sector ministries, and among ministers, managers, and staff. Strong accountability, in turn, can provide the incentives necessary to improve performance. These uses of M&E place it at the center of sound governance arrangements as a necessary condition for the effective

Prime Consulting Group, Bangalore

22

Monitoring & Evaluation

what why ways

management of public expenditures for poverty reduction. Thus M&E is necessary to achieve evidence-based policy making, evidence-based management, and evidence based accountability. An emphasis on M&E is one means to achieve a results-oriented and accountable public sector, including a performance culture in the civil service. For this reason, M&E should not be viewed as a narrow, technocratic activity. Within the four broad categories of use of M&E information, there are many specific activities where it can be used. At the same time, M&E is closely related to other aspects of public sector management: • • Budgetary tracking systems and financial reporting Intergovernmental fiscal relations, including government

decentralization, and the extent to which they encompass a focus on government performance • Accountability institutions such as national audit offices

Some countries have created a government M&E system to support performance budgeting, and a growing number of developing countries are following suit. M&E is widely viewed as a useful tool to help governments under fiscal stress reduce their total spending by identifying programs and activities that have relatively low costeffectiveness. Performance budgeting also helps governments prioritize among competing spending proposals. In this way, it is a vehicle to help them achieve greater value for money from their spending. Performance budgeting involves the use of monitoring information and evaluation findings.

14.PERFORMANCE BUDGETING There are three main approaches to performance budgeting. The first is known as direct performance budgeting—where there is a direct, often formula-driven relationship, so that the budget allocation

Prime Consulting Group, Bangalore

23

Monitoring & Evaluation

what why ways

for a program is based on its performance as measured by its results (that is, its outputs or outcomes). An example is university funding based on the number of students who graduate in each discipline, such as medicine or the arts. The second approach is indirect performance budgeting. This is the most common form of performance budgeting. M&E information on program results is an input, but only one input, into decisions on the budget allocation for the program. Other information, as well as the government’s policy priorities (including equity considerations), also influences budget allocations. The third approach is presentational performance budgeting. The government uses M&E information to report the actual (past) or expected (future) performance in the budget documents it sends to the Parliament or Congress. This information may have no influence on budget decision making and is the weakest form of performance budgeting. A common misconception is that evidence of poor performance should lead to a program either having its appropriations reduced or being abolished entirely. Although this might happen with a low-priority government activity, often more money will need to be spent on the program to fix it, at least in the short run. For example, an evaluation finding that primary education or public hospital spending is highly inefficient would not lead to these programs being abolished; they are too important. Instead, it would be necessary to identify the reasons for poor performance—which an evaluation should reveal—and to address them. • Commercialization and private sector (profit and nonprofit) delivery of public services, for example, by contracting out government functions. Success in these activities requires a clear understanding of objectives and actual performance.

Prime Consulting Group, Bangalore

24

Monitoring & Evaluation • •

what why ways

Clarification and public reporting of program goals, objectives, and the strategies necessary for achieving them The setting of explicit customer service standards by service delivery agencies, and monitoring and publicizing the extent to which these are achieved

Civil service reform that focuses on personnel performance, management, and appraisal, including merit-based hiring, promotion, and firing—recognizing the links between individual performance and project or program performance

• •

The quality of the civil service’s policy advice and the extent to which this advice is evidence based (using M&E) Anticorruption efforts—M&E can be used to identify the “leakage” of government funds via, for example, public expenditure tracking surveys (PETS). Community monitoring of donor (or government) projects can also be an effective way to help curb corruption in the implementation of projects.

Participation in civil society—M&E provides a vehicle to magnify the voice of civil society and to put additional pressure on government to achieve higher levels of performance.

Minor adjustment of program, for example, improved processes or information systems. Major adjustment of management processes, for example, changed targeting criteria, or new information systems, Substantial redesign of program or organizational structure, Institutional relocation of program may be the outcome of acceptance of findings from an M&E task.

15.PREREQUISITE TO INSTITUTIONALIZATION OF M&E The first and foremost lesson is that substantive demand from the government is a prerequisite to successful institutionalization. That is, an M&E system must produce monitoring information and evaluation findings that are judged valuable by key stakeholders, that are used to

Prime Consulting Group, Bangalore

25

Monitoring & Evaluation

what why ways

improve government performance, and that respond to a sufficient demand for the M&E function, to ensure its funding and sustainability for the foreseeable future. Efforts to build an M&E system will fail unless real demand exists or can be intentionally created, especially by ensuring that powerful incentives are in place to conduct and use M&E. It is not enough to issue an edict that M&E is important and should be done; this is likely to produce only lip service and is certainly unlikely to produce good quality monitoring information and evaluations. Such efforts at top-down compliance can, unless accompanied by a range of other actions, easily lead to ritual compliance or even active resistance. Achieving substantive demand for M&E is not easy. And a barrier to demand is lack of knowledge about what M&E actually encompasses, particularly where the buy-in of key stakeholders such as government ministers or finance ministries is necessary, before substantive effort will be put into creating and funding an M&E function. So there is frequently a chicken-and-egg problem: There is a lack of government demand for M&E because of the lack of understanding of M&E and what it can provide; there is a lack of understanding because of the lack of experience with it; and there is a lack of experience because of weak demand. The way around this conundrum is to increase awareness of M&E—its range of tools, methods, and techniques—and its potential uses. Demand can be increased, once key stakeholders in a government begin to understand it better, when they are exposed to examples of highly cost-effective monitoring systems and evaluation reports, and when they are made aware of other governments that have set up M&E systems, that they value highly. It can be persuasive to point to the growing evidence of very high returns on investment in M&E. The supply side is also important—provision of M&E training, manuals, and procedures and identification of good M&E consultants, and so forth. Prime Consulting Group, Bangalore 26

Monitoring & Evaluation

what why ways

M&E expertise is certainly necessary if reliable M&E information is to be produced. Those who view M&E in technocratic terms as a stand-alone technical activity tend to focus only on these issues. But the supply side of producing M&E information is less important than demand. If demand for M&E is strong, then improving supply in response can be relatively straightforward, but the converse does not hold. Demand from governments is crucial for the utilization of M&E information and for the ongoing funding of M&E activities. Thus, it is necessary to secure the buy-in of the key stakeholders such as government ministers or finance ministries if substantive effort is to be put into creating, strengthening, or funding M&E functions.

16.INCENTIVES FOR M&E Incentives are an important part of the demand side. There need to be strong incentives for M&E to be done well, and in particular for monitoring information and evaluation findings to be actually used. In other words, strong incentives are necessary if the M&E function is to be successfully institutionalized. This observation is also consistent with the extensive literature on achieving any type of institutional reform, particularly in the context of public sector management and sound governance. Simply having M&E information available does not guarantee that it will actually be used, whether by program managers in their day-to-day work, by budget officials responsible for advising on spending options, or by a Congress or Parliament responsible for accountability oversight. This underscores both the dangers of a technocratic view, which sees M&E as a set of tools with inherent merit, and the fallacy that simply making M&E information available would ensure its utilization. No governments build M&E systems because they have intrinsic merit. Governments build M&E systems because (1) those systems directly support core government activities, such as the budget process; national planning; development of policies and programs; and the

Prime Consulting Group, Bangalore

27

Monitoring & Evaluation

what why ways

management of ministries, agencies, programs, and activities, or (2) provide information in support of accountability relationships. Thus, M&E systems are often linked to public sector reforms such as performancebased budgeting (PBB), evidence-based policy making, results-based management, and the like; such initiatives share a number of common elements. In working to build or strengthen a government M&E system, it helps to start with a diagnosis of what M&E functions currently exist and their strengths and weaknesses—both on the demand .and supply sides. The extent of actual (as distinct from the hoped-for) utilization of M&E information must be clear, as well as the particular ways in which it is being used. Such diagnoses are themselves a form of evaluation, and they are useful not just for the information and insights they provide, but also because they can be a vehicle for raising the awareness of stakeholders in government, civil society, and the donor community about the importance of M&E, and the need to build a new system or strengthen existing systems.

17.KEY DRIVERS TO SUSTAINABLE M&E The key drivers to a sustainable M&E culture and system of

institutionalization are • • • • • • • • • • Substantive government demand Strong role for incentives Diagnosis of existing M&E as first step Recognition of key role of a powerful “champion” Centrally driven by capable ministry No over-engineering the system Reliable ministry / program data systems Utilization as the measure of success Training in M&E and in using M&E Limitations regulations Prime Consulting Group, Bangalore 28 of relying on government laws, decrees, and

Monitoring & Evaluation • • • •

what why ways

Structural arrangements ensure M&E objectivity and quality A long-term effort, requiring persistence Development in a nonlinear, less predictable manner Regular evaluation of M&E system itself.

18.M&E CHAMPION Another dimension to the demand side, and another success factor, is having a powerful champion— a powerful minister or senior official who is able to lead the push to institutionalize M&E, to persuade colleagues about its priority, and to devote significant resources to creating a whole of government M&E system. A champion needs to have some understanding of M&E, in terms of tools and methods, and an appreciation of its potential usefulness for government—for one or more of the four main uses of M&E information. Government champions have played important roles in the creation of some of the more successful government M&E systems. However, powerful champions constitute a success factor; they do not provide a guarantee of success. There are cases of a group of key ministers for M&E substantially frustrated by skeptical mid-level officials. Creating a whole-of-government M&E system— whether focused solely on a system of performance indicators or encompassing various types of evaluation and review—requires a significant effort. It involves recruiting and training staff to conduct or manage M&E and use their findings; creating the bureaucratic infrastructure to decide which government programs should be evaluated and what issues should be addressed in each evaluation; and creating data systems and procedures for collecting, sharing, and reporting M&E information. It also requires active oversight of the M&E system by senior—and usually very busy—officials. Like other systems, in areas such as financial management or procurement, it takes sustained effort over a period of years to make an M&E system operate efficiently. It takes time to develop a performance

Prime Consulting Group, Bangalore

29

Monitoring & Evaluation

what why ways

measurement system and to integrate it into a management system. No country considers developing a performance measurement system as easy. On the contrary, it is perceived as an exhausting exercise which needs constant monitoring and controlling. Thus, another feature of successful government M&E systems is the stewardship of this process by a capable ministry that can design, develop, and manage an M&E system. In many developed and upper middle-income countries, this has meant the finance ministry. It certainly helps to have the institutional lead of an M&E system close to the center of government (for example, a president’s office) or the budget process. There is also the need to have clarity, concerning the roles and responsibilities of key stakeholders A successful M&E institutionalization experience has a powerful

demonstration effect, a successful sector agency can have. The problem is more the poor quality and partial coverage of performance information and its substantial underutilization.

19.OVER-ENGINEERING THE M&E SYSTEM A common mistake is to over-engineer an M&E system. The appropriate number of performance indicators also depends on the number of government programs and services and on the type of performance indicator. Indicators can focus on the various parts of the “results chain” for a program: inputs, activities, outputs, outcomes, and impacts. Senior officials would tend to make use of high-level strategic indicators such as outputs and outcomes. Line managers and their staff, in contrast, would tend to focus on a larger number of operational indicators that target processes and services. The regular production of this kind of detailed performance information is only a first step. Senior officials and managers would usually not have the time to scrutinize raw data. It can assist their work considerably if capable analysts review the data and provide summaries and analyses for senior officials. Ministry data

Prime Consulting Group, Bangalore

30

Monitoring & Evaluation

what why ways

systems are often—and perhaps typically—uncoordinated, so a single ministry can possess several such systems, each with its own data definitions, data sources, periodicity of collection, and quality assurance mechanisms (if it has any). Data are poor partly because they aren’t being used; and they’re not used partly because their quality is poor. In such countries there is too much data, not enough information. Thus, another lesson for the institutionalization of a government M&E system is the need to build reliable ministry data systems—to help provide the raw data on which M&E systems depend. An audit of data systems and a diagnosis of data capacities can be helpful in this situation. It would provide the starting point for any necessary rationalization of data collections or improvements in their quality. It would give the data the credibility necessary to be used. Data verification and credibility is partly a technical issue of accuracy, procedures, and quality control. Related to this issue of technical quality is the need for data to be potentially useful—for information to be available on a timely basis, easy to understand, consistent over time, and so forth. There is also an issue of honesty and objectivity; as performance information becomes important— particularly when it is used for accountability purposes— there will arise incentives to falsify the data.2 This is clearly of concern and requires specific measures to verify the data, such as through independent data audits. Verification could also involve accreditation of an agency’s processes for data verification. Of course, efforts to falsify data can also be taken as a sign of success that the M&E system is starting to have an impact—that it is having real “bite.” Financial data on program spending comprise a fundamental type of information; quality and ready availability must be ensured. This information is supplied by financial management information systems. Sound performance budgeting, program management, and government

Prime Consulting Group, Bangalore

31

Monitoring & Evaluation

what why ways

accountability all require that information on the costs and results of government programs be linked. Ironically, many evaluations then fail to discuss the cost of the programs being evaluated; this makes it harder to gauge the cost-effectiveness of the program.

20.THE VALUE FROM M&E SYSTEMS The objective of government M&E systems is never to produce large volumes of performance information or a large number of high-quality evaluations per se; this would reflect a supply driven approach to an M&E system. Rather, the objective is to achieve intensive utilization of whatever M&E information exists to ensure that the M&E system is costeffective—utilization in support of core government functions, as noted earlier. Utilization is the yardstick of success of an M&E system; conversely, it would be hard to convince a skeptical finance ministry that it should continue to fund an M&E system whose outputs are not being utilized. Such systems would deservedly be regarded as useless. For an M&E system to perform well, it is necessary to have well-trained officials or consultants who are highly skilled in M&E. For this reason, most capacity-building plans place considerable emphasis on provision of training in a range of M&E tools, methods, approaches, and concepts. Those governments that contract out their evaluations, need to ensure that their officials possess the skills and experience necessary to oversee and manage evaluations—this requires a broader set of competencies than the ability to simply conduct an evaluation. They also need to understand the strengths and limitations—the relative costeffectiveness—of various types of M&E. Introductory training in M&E can also raise awareness of and demand for M&E information. Training should also extend to the use of M&E information. Budget analysts and program managers need to be able to interpret monitoring data to understand trends, data definitions, breaks in data time series, and so forth. They also need to be discriminating

Prime Consulting Group, Bangalore

32

Monitoring & Evaluation

what why ways

consumers of evaluations; that is, they must be able to tell when an evaluation’s methodology is flawed and is findings unreliable. Finally, parliamentarians and their staff need to be able to understand M&E information. Another lesson is that there are limitations when a country relies on a law, decree, cabinet decision, or other high-level pronouncement to create an M&E system. The point here is not that a law or decree mandating M&E is irrelevant: on the contrary, these can be useful vehicles for legitimizing M&E, particularly in those countries where the presence of such a legal instrument is viewed as necessary for any government reform to be perceived as worthwhile and taken seriously. But a law or decree on its own does not ensure that the considerable efforts required to build an M&E system will be undertaken.

21.STRUCTURAL ARRANGEMENTS FOR M&E SYSTEM The structural arrangements of an M&E system are important from a number of perspectives. One is the need to ensure the objectivity, credibility, and rigor of the M&E information, the system produces. On the data side, some governments rely on external audit committees to perform this function, some rely on the national audit office and some rely principally on internal ministry audit units. Some rely on the central ministry checking data provided by sector ministries and others have no audit strategy. On the evaluation side, issues of objectivity and credibility are particularly important. By contracting out evaluations to external bodies such as academic institutions and consulting firms; the evaluations are commissioned and managed by the finance ministry rather than by sector ministries, and the process of seeking bids and awarding contracts to conduct the evaluations is entirely transparent. The downside of this approach is a lack of ownership of these evaluation findings by the sector ministries, which do not make much use of the Prime Consulting Group, Bangalore 33

Monitoring & Evaluation

what why ways

evaluations commissioned by the finance ministry. That may not be so great a problem, where the powerful finance ministry is able to use evaluation findings not only to support budget decision making, but also to impose management and program changes on the sector ministries. This centrally imposed system is unique. Building an a government M&E system is a long-haul effort, requiring patience and persistence. It takes time to create or strengthen data systems; to train or recruit qualified staff; to plan, manage, and conduct evaluations; to build systems for sharing M&E information among relevant ministries; and to train staff to use M&E information in their day-to-day work, whether that involves program operations or policy analysis and advice. This is not to say that a slow and measured approach to building an M&E system is appropriate, however. Government champions will eventually depart, and the window of opportunity—indeed, the priority a government gives to any type of public sector reform—can close as quickly as it opened. This suggests that an approach of working in a focused, purposeful, and even intense manner to build various components of the M&E system is necessary, and that governments need to institutionalize them as quickly as possible. The frequency of mid-course corrections as M&E systems are being built indicates another lesson from experience: the value of regularly evaluating an M&E system itself, with the unsurprising objective of finding out what is working, what is not, and why. It is valuable to identify the nature of the roadblocks encountered and to indicate possible changes in direction. Such evaluations provide the opportunity to review both the demand and the supply sides of the equation, and to clarify the extent of actual—as distinct from the hoped-for—extent of utilization of M&E information, as well as the particular ways in which it is being used.

Prime Consulting Group, Bangalore

34

Monitoring & Evaluation

what why ways

22.HOW TO STRENGTHEN A GOVERNMENT M&E SYSTEM • • • • • • Impact Final outcomes Intermediate outcomes Outputs Activities Government Parliament Civil society

M&E is used for government decision making on policies, plans, and budget resource allocation; implementation of and management accounting of of government activities; monitoring activities,

expenditures, evaluation of programs and projects; government analysis and policy review; and government accountability. The action plan leads to the production of a range of outputs, such as number of officials trained in M&E; harmonized data systems; improved quality and focus of available monitoring indicators; improved quality for and range of to evaluations completed; strengthened incentives ministries

conduct and use M&E. A package of activities to strengthen country M&E functions is undertaken by the government and donors, such as national seminars on country M&E systems; diagnoses of national/sectoral M&E functions; audits of data systems; provision of M&E training—including trainer training—or scholarships to officials, NGOs, universities/research institutes, parliamentary staff; various types of evaluation are conducted on pilot/demonstration basis.

23.GLOSSARY OF TERMS USED
23.1. Logical framework (Logframe)

Management tool used to improve the design of interventions, most often at the project level. It involves identifying strategic elements (inputs, outputs, outcomes, impact) and their causal relationships, indicators, and the assumptions or risks that may influence success and

Prime Consulting Group, Bangalore

35

Monitoring & Evaluation

what why ways

failure. It thus facilitates planning, execution and evaluation of a development intervention. Related term: results-based management
23.2. Indicator

Quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor.
23.3. Inputs

The financial, human, and material resources used for the development intervention.
23.4. Accountability

Obligation to demonstrate that work has been conducted in compliance with agreed rules and standards or to report fairly and accurately on performance results vis a vis mandated roles and/or plans. This may require a careful, even legally defensible, demonstration that the work is consistent with the contract terms. Note: Accountability in development may refer to the obligations of partners to act according to clearly defined responsibilities, roles and performance expectations, often with respect to the prudent use of resources. For evaluators, it connotes the responsibility to provide accurate, fair and credible monitoring reports and performance assessments. For public sector managers and policymakers, accountability is to taxpayers.
23.5. Activity

Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilized to produce specific outputs. Related term: development intervention
23.6. Data collection tools

Methodologies

used

to

identify

information

sources

and

collect

information during an evaluation. Note: Examples are informal and

Prime Consulting Group, Bangalore

36

Monitoring & Evaluation formal surveys, direct and participatory

what why ways observation, community

interviews, focus groups, expert opinion, case studies, literature search
23.7. Base-line study

An

analysis

describing

the

situation

prior

to

a

developmental

intervention, against which progress can be assessed or comparisons made.
23.8. Benchmark

Reference

point

or

standard

against

which

performance

or

achievements can be assessed. Note: A benchmark refers to the performance that has been achieved in the recent past by other comparable organizations, or what can be reasonably inferred to have been achieved in the circumstances.
23.9. Beneficiaries

The individuals, groups, or organizations, whether targeted or not, that benefit, directly or indirectly, from the development intervention.
23.10. Effect

Intended or unintended change due directly or indirectly to an intervention. Related terms: results, outcome
23.11. Effectiveness

The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Note: Also used as an aggregate measure of (or judgment about) the merit or worth of an activity, i.e. the extent to which an intervention has attained, or is expected to attain, its major relevant objectives efficiently in a sustainable fashion and with a positive institutional development impact. Related term: efficacy

Prime Consulting Group, Bangalore

37

Monitoring & Evaluation
23.12. Efficiency

what why ways

A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.
23.13. Feedback

The transmission of findings generated through the evaluation process to parties for whom it is relevant and useful, so as to facilitate learning. This may involve the collection and dissemination of findings, conclusions, recommendations and lessons from experience.
23.14. Finding

A finding uses evidence from one or more evaluations to allow for a factual statement.
23.15. Evaluation

The systematic and objective assessment of an ongoing or completed project, program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision–making process of both recipients and donors. Evaluation also refers to the process of determining the worth or significance of an activity, policy or program. An assessment, as systematic and objective as possible, of a planned, ongoing, or completed development intervention. Note: Evaluation in some instances involves the definition of appropriate standards, the examination of performance against those standards, an assessment of actual and expected results and the identification of relevant lessons. Related term: review

Prime Consulting Group, Bangalore

38

Monitoring & Evaluation
23.16. Goal

what why ways

The higher-order objective to which a development intervention is intended to contribute. Related term: development objective
23.17. Impacts

Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.
23.18. Independent evaluation

An evaluation carried out by entities and persons free of the control of those responsible for the design and implementation of the development intervention. Note: The credibility of an evaluation depends in part on how independently it has been carried out. Independence implies freedom from political influence and organizational pressure. It is characterized by full access to information and by full autonomy in carrying out investigations and reporting findings.
23.19. Indicator

Quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor.
23.20. Inputs

The financial, human, and material resources used for the development intervention.
23.21. Lessons learned

Generalizations broader

based

on

evaluation

experiences highlight

with

projects, or

programs, or policies that abstract from the specific circumstances to situations. Frequently, lessons strengths

Prime Consulting Group, Bangalore

39

Monitoring & Evaluation

what why ways

weaknesses in preparation, design, and implementation that affect performance, outcome, and impact.
23.22. Mid-term evaluation

Evaluation

performed

towards

the

middle

of

the

period

of

implementation of the intervention. Related term: formative evaluation
23.23. Monitoring

A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds. Related terms: performance monitoring, indicator
23.24. Outcome

The likely or achieved short-term and medium term effects of an intervention’s outputs. Related terms: result, outputs, impacts, effect
23.25. Outputs

The

products,

capital

goods

and

services

which result

from a

development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes.
23.26. Participatory evaluation

Evaluation

method

in

which

representatives

of

agencies

and

stakeholders (including beneficiaries) work together in designing, carrying out and interpreting an evaluation.
23.27. Partners

The individuals and/or organizations that collaborate to achieve mutually agreed upon objectives. Note: The concept of partnership connotes shared goals, common responsibility for outcomes, distinct accountabilities governments, and civil reciprocal society, obligations. Partners may include non-governmental organizations,

Prime Consulting Group, Bangalore

40

Monitoring & Evaluation universities, professional and business

what why ways associations, multilateral

organizations, private companies, etc.
23.28. Performance

The degree to which a development intervention or a development partner operates according to specific criteria/standards/guidelines or achieves results in accordance with stated goals or plans.
23.29. Performance indicator

A variable that allows the verification of changes in the development intervention, or shows results relative to what was planned. Related terms: performance monitoring, performance measurement
23.30. Performance measurement

A system for assessing performance of development interventions against stated goals. Related terms: performance monitoring, indicator
23.31. Performance monitoring

A continuous process of collecting and analyzing data to compare how well a project, program, or policy is being implemented against expected results.
23.32. Process evaluation

An evaluation of the internal dynamics of implementing organizations, their policy instruments, their service delivery mechanisms, their management practices, and the linkages among these. Related term: formative evaluation
23.33. Program evaluation

Evaluation of a set of interventions, marshaled to attain specific global, regional, country, or sector development objectives. Note: A development program is a time-bound intervention involving multiple activities that may cut across sectors, themes and/or geographic areas. Related term: country program/strategy evaluation Prime Consulting Group, Bangalore 41

Monitoring & Evaluation
23.34. Project evaluation

what why ways

Evaluation of an individual development intervention designed to achieve program. Note: Cost benefit analysis is a major instrument of project evaluation for projects with measurable benefits. When benefits cannot be quantified, cost effectiveness is a suitable approach.
23.35. Results

specific

objectives

within

specified

resources

and

implementation schedules, often within the framework of a broader

The output, outcome or impact (intended or unintended, positive and/or negative) of a development intervention. Related terms: outcome, effect, impacts
23.36. Results chain

The causal sequence for a development intervention that stipulates the necessary sequence, to achieve desired objectives beginning with inputs, moving through activities and outputs, and culminating in outcomes, impacts, and feedback. In some agencies, reach is part of the results chain. Related terms: assumptions, results framework
23.37. Results framework

The program logic that explains how the development objective is to be achieved, including causal relationships and underlying assumptions. Related terms: results chain, logical framework
23.38. Results-based management (RBM)

A management strategy focusing on performance and achievement of outputs, outcomes and impacts. Related term: logical framework
23.39. Target group

The

specific

individuals

or

organizations

for

whose

benefit

the

development intervention is undertaken.

Prime Consulting Group, Bangalore

42

Monitoring & Evaluation
23.40. Triangulation

what why ways

The use of three or more theories, sources or types of information, or types of analysis to verify and substantiate an assessment. Note: By combining multiple data sources, methods, analyses or theories, evaluators seek to overcome the bias that comes from single informants, single methods, single observer or single theory studies.
23.41. Stakeholders

Agencies, organisations, groups or individuals who have a direct or indirect interest in the development intervention or its evaluation

Prime Consulting Group, Bangalore

43

Sign up to vote on this title
UsefulNot useful