9/4/2018

Module Objectives
Uganda Martyrs University
MSc in M&E  On completion of this module, you should:
 Have acquired knowledge in applications of Monitoring
and Evaluation
 Be able to explain the processes of developing the
Module: M&E Principles and Perspectives monitoring applications and frameworks
Topic: Introduction to M&E  Be able to carry out evaluation guided by research
Facilitator: Mugerwa George methods approaches
 Use the different frameworks and tools for their planning,
monitoring, performance determination and analysis

September 4, 2018 mugerwageorge@gmail.com 2

Reading List Reading List

 Louisa Goslin and Mike Edwards (2007); A practical Guide to
Planning, Monitoring, Evaluation and Impact Assessment,
Save the Children
 Harold Kerzner,(2009) Project Management; A Systems
Approach to Planning, Scheduling and Controlling
 Civicus; Monitoring and Evaluation Toolkit; www.civicus.org
 UNDP; 1997; Results Oriented Monitoring and Evaluation; A
Handbook for Program Managers; In Mind Design, L.L.C.
 Bhavesh M. Patel (2000). Project Management; Strategic
Financial Planning, Evaluation and Control; Vikas Publishing
House PVT Ltd
 PACT, (1998): Participatory Monitoring, Evaluation and
Reporting.
September 4, 2018 New York, PACT
mugerwageorge@gmail.com
 Patton, M.Q. (1997) “Toward Distinguishing Empowerment
Evaluation and Placing It in a Larger Context” Evaluation Practice
1997 18(2)
 Pretty, J. (1996): Participatory Learning and Action: A Trainer’s
Guide. London: International Institute for Environment and
Development.
 Jules Pretty, Irene Gujit, Ian Scoones and John Thompson (2002).
Participatory Learning and Action A Trainer’s Guide; International
Institute for Environment and Development (IIED), London, 1995.
 Rugh, J. (2007).Self-Evaluation: Ideas for Participatory Evaluation of
Community Development Projects World Neighbours
 Narayan, D. (2008).Participatory Evaluation: Tools for Managing
Change in Water and Sanitation; Technical Paper Number 207.
Washington, DC: The World Bank.
3 September 4, 2018 mugerwageorge@gmail.com 4

Presentation Outline Session Objectives

 By the end of the session, you should be
 Session Objectives
able to:
 Overview of M&E  Explain the evolution of M&E
 Definition of M&E  Explain the concept M&E
 Rationale for M&E  Explain the rationale for carrying out M&E
 Performance indicators activities
 Demonstrate ability to design project/
 M&E Plan
organizational performance indicators

September 4, 2018 mugerwageorge@gmail.com 5 September 4, 2018 mugerwageorge@gmail.com 6

1
 9/4/2018

Overview of M&E
 As soon as your organization decides to
 The evolution of evaluation in Africa began
engage to have something done it has
three problems: early in the 1950s.
 Get people to do it!  The rapid growth in international
 Monitor whether they do it! development assistance in the 1970s saw the
 Find out whether it was done well!
shoring up of M&E systems, as the need for
the control and accountability increased.
MONITORING  The conventional M&E effort focused on
physical inputs and outputs and financial
EVALUATION expenditure against capital items.

September 4, 2018 mugerwageorge@gmail.com 7 September 4, 2018 mugerwageorge@gmail.com 8

Overview of M&E Overview of M&E

 It relied on objective methods, facilitated by – Justifying activities to external funding bodies and
outside experts and often initiated by donor donors;
agencies. – Ensuring that the group or organization is
 It was by and large concerned with: developing in a sustainable manner;
 Establishing the financial and technical feasibility – Sharing successes and failures with others to build
of the project solidarity; and
 Producing a comprehensive and detailed project – Building community.
design or blueprint

September 4, 2018 mugerwageorge@gmail.com 9 September 4, 2018 mugerwageorge@gmail.com 10

Evolution of M&E Evaluation in OECD Developed Countries

 However in China evaluation can be traced as back as 2200
BC- personnel selection
 In the 19th century western countries adopted performance
measurement in education as a mechanism for evaluation.
 In the 1960s USA - Social programs Great society and War on
Poverty. Optimism of 1960s: evaluation would change the
world
 In the 1970s there was a growing concern for Aid evaluation
(Netherlands, UK, Australia)
 In 1990s we see a new wave for Development-oriented
evaluation.
September 4, 2018 mugerwageorge@gmail.com
 OECD countries have mature M&E systems
 In a recent survey (2013): Australia, Canada, the
Netherlands, Sweden, and the USA had highest
“evaluation culture ranking”
 These countries have developed an evaluation
culture with:
 Professional evaluators in different fields
 Institutional government framework for collecting and
disseminating information. It is a requirement for
institutions to show their worth.
 Evaluation has moved beyond audits to policy and
11
outcomes – e.g. the
September 4, 2018
regulatory impact assessments 12
mugerwageorge@gmail.com

2
 9/4/2018

OECD Approaches Introduction to M&E

 OECD encourages three pronged approach to
evaluation.
 Whole-of-Government: Broad-based,
comprehensive M&E at all levels of government.
E.g. the National Integrated Monitoring and
Evaluation System.
 Enclave: Focused on one part or sector of the
government (one ministry or cabinet). E.g. Local
government monitoring systems
 Mixed Approach: Blended whole of government
and enclave. Some areas have comprehensive
approaches; others
September 4, 2018
more sporadic attention.
mugerwageorge@gmail.com
 Monitoring and evaluation (M&E) of development
activities provides government and development
practitioners with information for :
 Better means for learning from past experience,
 Improved service delivery,
 Planning and allocating resources, and
 Demonstrating results as part of accountability to key
stakeholders.
 Within the development community there is a strong
focus on results—this helps to explain the growing
interest in M&E.
 However there is also some confusion about what M&E
entails e.g. going to field, fault finding, meeting donor requirements etc.
13 September 4, 2018 mugerwageorge@gmail.com 14

The Power of Measuring Results But what is M&E?

 But why measure results?
 If you do not measure results, you cannot tell success
from failure
 If you can not see success, you can not reward it
 If you can not reward success, you are probably
rewarding failure
 If you can not see success, you can not learn from it
 If you can not recognize failure, you can not correct it
 If you can demonstrate results, you can win public
support

September 4, 2018 mugerwageorge@gmail.com 15 September 4, 2018 mugerwageorge@gmail.com 16

M&E
 Monitoring and evaluation are essentially the core
processes for “learning from experience.”
 Learning implies a process of analysis which, in turn,
requires the existence of relevant information or
evidence on which to base the analysis.
 The primary function of M&E is to provide pointers
on how to do things better through a better
understanding of what works and what does not.
M&E
 M&E is the bringing together of information and
learning.
 Data are the raw material used to fuel the
monitoring and evaluation process.
 Access to appropriate data and data sets that can be
processed into usable, timely, and relevant statistical
information is essential for effective monitoring and
evaluation that in turn can lead to a learning
experience.

September 4, 2018 mugerwageorge@gmail.com 17 September 4, 2018 mugerwageorge@gmail.com 18

3
 9/4/2018

Monitoring versus Evaluation Monitoring Evaluation

Monitoring Evaluation  The continuous collection  A systematic process to

A planned, systematic process
of observation that closely
follows a course of
activities, and compares
what is happening with
what is expected to happen
(planned vs actual)
Looks at inputs, processes,
activities and outputs
A process that assesses an
achievement against preset
criteria.
Has a variety of purposes, and
follow distinct
methodologies (process,
outcome, performance,
etc).Looks at relevance,
effectiveness, cost
effectiveness and impact
and review of information on
programme implementation,
coverage and use for
comparison with
implementation plans.
 Open to modifying original
plans during implementation
 Identifies shortcomings
before it is too late.
 Provides elements of analysis
as to why progress fell short
of expectations
determine the extent to which
service needs and results have
been or are being achieved
and analyse the reasons for
any discrepancy.
 Attempts to measure service’s
relevance, efficiency and
effectiveness. It measures
whether and to what extent
the programme’s inputs and
services are improving the
quality of people’s lives.

September 4, 2018 mugerwageorge@gmail.com 19 September 4, 2018 mugerwageorge@gmail.com 20

Why M&E?
Comparison between M&E There are a lot of good reasons for conducting M&E.
M&E is important for:
• Determining whether resources are being expended in
the manner planned and according to the program
budget.
• Getting frequent feedback about how your project is
progressing and making sure that things are on track.
• Improving the effectiveness of projects by allowing for
mid-course corrections if there are aspects that are
not having the desired impact.

September 4, 2018 mugerwageorge@gmail.com 21 September 4, 2018 mugerwageorge@gmail.com 22

Why cont’s Why cont’s?

•Serving as a basis for sound decisions about
whether and how to revise your project.
•Promoting accountability among project staff.
•Demonstrating the impact and success of your
project.
•Identifying lessons learned, enabling
institutional learning and informing decisions
about future programs.
According to the World Bank, effective M&E provides
answers to the following questions:
• Are we doing the right things?
–Are our interventions contributing to the project objectives?
• Are we doing the right things right?
–How effective have we been in achieving expected
outcomes? How efficient have we been in optimizing
resources? Are these results sustainable?
• Are there better ways of doing the right things?
–What are the best practices we’ve identified?

September 4, 2018 mugerwageorge@gmail.com 23 September 4, 2018 mugerwageorge@gmail.com 24

4
 9/4/2018

Emerging Trends & Implications Top of International Development Agenda

 Changes in M&E as a result of emerging Issues
 Globalization
 Growing incidence of conflict
 Terrorism and money laundering
 Widening gap between rich and poor
 More development players
 Drive toward debt reduction
 Focus on improved governance and accountability
 Drive toward comprehensive, coordinated, participatory
development with results
 Sustainable Development Goals (MDG)
 Debt Initiative for Heavily-Indebted Poor Countries
(HIPC)
 The Emergence of New Actors in International
Development Assistance (China, Korea, India, Turkey,
EU, South Africa, Singapore, Libya, Iran)
 Conflict Prevention and Post-conflict Reconstruction
 Governance and accountability

September 4, 2018 mugerwageorge@gmail.com 25 September 4, 2018 mugerwageorge@gmail.com 26

Top of International Development Agenda New Actors in International Agenda

 Anti-Money Laundering and Terrorist Financing
 Workers’ Remittances  Large foundations:
 Gender: From Women in Development (WID) to  The Bill and Melinda Gates Foundation
Gender and Development (GAD)  The Ford Foundation
 Clinton Foundation
 Mainstreaming Private Sector Development
(PSD) and Investment Climate  China –Africanisation and the AID industry
 Environmental and Social Sustainability  All these actors must become part of the
 Global Public Goods process to design and develop evaluations

September 4, 2018 mugerwageorge@gmail.com 27 September 4, 2018 mugerwageorge@gmail.com 28

Trends particularly in the south…. Trends particularly in the south…

 New aid paradigms  New management in states and NGOs: results /
 PRSP December 1999 performance oriented
 MDGs September 2000  New indicators in international monitoring protocols
 Paris Declaration March 2005 e.g.
 Choice of indicators
 New conditionalities
 e.g. enrolment vs. completion rate
 Old-style conditionalities are replaced with conditionalities
 Level of aggregation
that focus on in-country processes - i.e. opening up
 e.g. world vs. regional scale
discussion among stakeholders, ‘ownership’,
 Narrow definition of indicators
 Enhancement of M&E in developing countries, new
 e.g. ownership measured by number of countries with national
instruments, measurability of MDGs “makes developing development strategies (see Paris Declaration)
countries nervous”

September 4, 2018 mugerwageorge@gmail.com 29 September 4, 2018 mugerwageorge@gmail.com 30

5
 9/4/2018

Trends particularly in the south… What do these trend mean to M&E?

 New actors:  Driving developing countries to build
 M&E systems established at central and decentralized evaluation capacity and systems
levels of government- e.g. local government  Donors called upon to provide technical
performance assessments
 Civil society organizations as watchdogs of government
assistance
performance- e.g. parliamentary performance index  Involvement in non traditional areas of
 Civil society organizations to respond to RBM trend development
themselves
 Countries must commit to accountability and
transparency -openness
 Subscribe to aid effectiveness
September 4, 2018 mugerwageorge@gmail.com 31 September 4, 2018 mugerwageorge@gmail.com 32

Where are We Today?

What do these trend mean to M&E?
 The new development agenda calls for broader
understandings of:
 Gender budgeting mandatory
 sectors
 Objective measures of business regulations  countries
and their reinforcement (env’tl degradation)  development strategies
 Corporate Social Responsibility (CSR): actively  policies
taking into account the economic,  Emphasizes learning and continuous feedback at all
environmental and social impacts, and phases of the development cycle
consequences of business activities  Evaluations have moved from traditional
 Evaluation of global public goods in regard to implementation focused evaluation models to results
environment and sustainable development based evaluation models.
September 4, 2018 mugerwageorge@gmail.com 33 September 4, 2018 mugerwageorge@gmail.com 34

Characteristics of M&E practice and

Where Are We Today?
 Emphasis is now being placed on working
together and growing a number of partnerships
 Performance of individual partners now needs
to be evaluated according to their respective
contributions and obligations
 Evaluation has become more difficult to design
with a more demanding, fragmented, and
participatory approach to development
September 4, 2018 mugerwageorge@gmail.com
systems in South (LDCs)
(Khan, 1998:313)
 Evaluation practice is of recent date
 Direct relationship between quality of governance and
the success of M&E
 Weak institutional and methodological capacity
 Most evaluations are donor-driven
 Many are top-down and lack wider participation
 Limited inter-ministerial cooperation and coordination
 Monitoring is first- tracking physical progress
 Feedback system is absent or weak
 Limited budgets for evaluations
35 September 4, 2018 mugerwageorge@gmail.com 36

6

Characteristics cont’
 There are highly placed champions willing to
assume political risks of evaluations e.g evaluation
networks
 Efforts to professionalize evaluations are taking
root
 Some credible institutions
 Availability of basic statistical systems and data
 Sometimes appropriate quality and quantity of
empirical evidence
 Increasing demand from the citizens
September 4, 2018 mugerwageorge@gmail.com
Hierarchy of Performance Concepts
September 4, 2018 mugerwageorge@gmail.com
Where is the relevance of M&E?
 M&E is a key function for service delivery
improvement.
 M&E shows whether a service/program is
accomplishing its goals.
 It identifies program weaknesses and
strengths, areas that need revision, and
areas that meet or exceed expectations.
 To do this, you need to analyse the
program’s domain.
September 4, 2018 mugerwageorge@gmail.com
9/4/2018
How do we execute M&E?
 In executing the M&E functions, M&E
practitioners focus on the performance
indicators.
 These indicators can be at the different
levels of objectivity hierarchy i.e.
 Impact
 Outcome
 Output
 Activity
 Input
37 September 4, 2018 mugerwageorge@gmail.com 38
What are performance indicators?
 Performance indicators are measures of inputs,
processes, outputs, outcomes, and impacts for
development projects, programs, or strategies.
 When supported with sound data collection and
management system, they enable managers to track
progress, demonstrate results, and take corrective
action to improve service delivery.
 It is important to involve the key stakeholders when
defining performance indicators.
 This enables them to have a thorough understanding of
these indicator and the rationale for capturing them.
 PMT can base on them to make management decisions.
39 September 4, 2018 mugerwageorge@gmail.com 40
How does M&E inform Mgt functions?
 M&E helps to make informed decisions
regarding the on-going programs.
 It facilitates effective and efficient use of resources
 It helps to determine whether the program is right on track
and where changes need to be considered
 M&E help stakeholders conclude whether the
program is a success or not.
 M&E preserve institutional memory.
41 September 4, 2018 mugerwageorge@gmail.com 42
7
 9/4/2018

Three Primary Uses of M&E Findings When should we do M&E?

 M&E is a continuous process that occurs throughout the life
 Rendering judgments of a project/ program.
 Summative evaluations of program’s overall effectiveness  But to be more effective, it should be planned at the design
 e.g. audit, renewal, quality control, accreditation stage of a program, with the time, money and personnel that
will be required calculated and allocated in advance.
 Facilitating improvements  Monitoring should be conducted at every stage of the
 Formative evaluation to improve program program, with data collected, analyzed and used on a
 e.g. program’s strengths/weaknesses, progress continuous basis.
 Evaluations are usually conducted at the beginning, in the
 Generating knowledge middle and at the end of programs.
 Conceptual use of findings  Evaluations should be planned for at the start because they
 e.g. generalization, theory building rely on data collected throughout the program, with baseline
data being very important.
September 4, 2018 mugerwageorge@gmail.com 43 September 4, 2018 mugerwageorge@gmail.com 44

END

September 4, 2018 mugerwageorge@gmail.com 45