Development Evaluation Training Programme

in Africa (DETPA)
FOUNDATION TRACK

COURSE 1 F, Module 1.1F

Situating M&E in Development Interventions
13 August 2018

Facilitator: Mike Leslie

Overview of the session
Subject area Start End

Welcome and introductions 11:00 11:15

Introduction and overview 11:15 11:30
Discussion 11:30 11:45
Defining the purpose & 11:45 12:00
scope of M&E
Discussion points 12:00 12:10
Towards and African 12:10 12:30
approach to M&E
Discussion 12:30 12:45
Key definitions 12:45 13:00
Welcome and introductions
• Name
• Official title or designation
• Organisation and country
• Experience with M&E and/or development
interventions
Introduction and
Overview
Why M&E + development?
• Imposed conditions for progress?
• Ad-hoc love/hate relationship?
• Mutually beneficial relationship?
• A bureaucratic appendage?

All of the above, none or something more?

Evaluation and Development
• Evaluation supports learning about achievement (or
not!) of development goals
• Many developed and developing countries have put
M&E systems in place to address development
challenges
• Many are in the process of establishing M&E
systems, with some placing greater emphasis on the
M than the E…
Evolution of M&E in the West
• “Early adopters” – e.g. Australia, Canada, U.S.
• Strong internal pressures (e.g. new socio-economic
programmes, legislative oversight)
• Early adopters spread evaluation culture to other
countries (training, dissemination of information
and ideas etc.)
• “Late adopters” – e.g. Ireland, Italy, Spain
• Strong external pressures (e.g. EU funding
requirements)
A growing demand for M&E in
Africa
• Supply, to a large extent, influenced by donor
demands
• Initial absence of national government demand
• Increasing demands for government accountability
• Changes in type of evidence or information required
• Case-studies of Benin, Ghana, Kenya, Senegal, South
Africa and Uganda (Porter and Goldman, 2013):
• Merging of donor-driven and country-led demands
• Narrow interpretation of M&E, focusing on monitoring
• Less focus on development, more on accountability
A growing demand for M&E in
Africa
• Entry of New LANGUAGE focusing more on results
• “Results-based management” (RBM) & “Managing
for Development Results” (MfDR)
• The Paris Declaration on Aid Effectiveness (2005)
• Ownership, alignment, transparency, results and
harmonization
• Accra Agenda for Action (AAA, 2008) took stock of
progress and accelerated advancement
• Ownership; inclusive partnerships; results; capacity
development
Integrating M&E into Development
Planning and Implementation
• Evaluation and development – symbiotic
relationship
• Evaluation informs BOTH learning & action
• Value is in their combination
• Good evaluation should lead to improved
development
• Evidence-informed development is seldom achieved
But if not evidence, then what informs development?
 Type of evidence most often used for
policy decision-making (int. with 54 SAn SMS)
All
Highest no is 16
Research Synthesis (1/3), and little use
of formal
Scientific Research research/evaluation

Formal Research
More of
Substantive Current

Informal

Opinion

None

0 5 10 15 20 25
Source: Paine Cronin, G & Sadan, M. 2015 ‘Use of evidence in policy making in South Africa…’ African Evaluation Journal 3(1)
11
Integrating M&E into Development
Planning and Implementation
?

? ?

? ?
Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 Integrating M&E into Development
Planning and Implementation
• Planning, monitoring and evaluation should not
necessarily be approached in a sequential manner.
The conduct of an evaluation does not always take
place at the end of the cycle. Evaluations can take
place at any point in time during the programming
cycle.
• Planning for monitoring and evaluation must take
place at the planning stage of a development
intervention

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
Source: https://blog.abglobal.com/post/en/2017/01/UN-sustainability-goals-roadmap-for-imact-investing
How will we know we have arrived?
The Sustainable Development Goals,
National Development Planning and
Evaluation
• The indicators to measure the success of the eight
Millennium Development Goals (MDGs) were an
afterthought devised by UN officials “in a dark room
in New York and thrust down countries’ throats”
(Guido Schmidt-Traub, executive director of the UN Sustainable Development
Solutions Network)

• Indicators are not an exact science, and some

developing countries may struggle to gather the
massive amount of new data needed
Source: http://www.scidev.net/global/mdgs/feature/track-sdgs-
data-development-goals.html#
Ideal developmental agenda
setting and coordination
SDGs

NDPs

MTSPs

SPs
But in practice…
SDGs
NDPs

MTSPs

SPs
Defining the purpose
and scope of M&E
 How can M&E help achieve
development outcomes?

Budgets and Sustainable

Programmes Development
 How can M&E help achieve
development outcomes?
Good planning, monitoring and evaluation
helps to establish clear links between past,
present and future initiatives and development
results.
Discussion point:
In your country/sector, are there examples of good
planning and M&E? Does M&E information play a role in
decision-making and planning?

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 How can M&E help achieve
development outcomes?
Monitoring and evaluation can help an organization
extract relevant information from past and ongoing
activities that can be used as the basis for
programmatic fine-tuning, reorientation and future
planning.

Discussion point:
How “adaptive” do you think your
sector/country is to using information as a
programme is being implemented?

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 How can M&E help achieve
development outcomes?
Without effective planning, monitoring and
evaluation, it would be impossible to judge if work is
going in the right direction, whether progress and
success can be claimed, and how future efforts might
be improved.
Discussion point:
In your experience, have there been
programmes that were clearly
unsuccessful, but were continued anyway?
Would M&E have helped?

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 Select the most appropriate answer
Without proper planning and clear articulation of
intended results…….

• a) it is not clear what should be monitored and how;

hence monitoring cannot be done well.
• b) the necessary data is not collected; hence
evaluation cannot be done well.
• c) the basis for evaluation is weak; hence evaluation
cannot be done well.

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 True or false?

Monitoring is necessary, but not

sufficient, for evaluation

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
 True or false?
Once indicators have been
developed and data collection
systems decided upon, they
should be stabilised, and should
not be changed.

Source: UNDP. 2009. Handbook on Planning, Monitoring and Evaluation for Development Results. New York: UNDP
Towards an African
approach to M&E
Main funders of Humanitarian Aid
Evaluations in Africa (2001-2010)
Who is conducting Evaluations in
Africa?
• International teams and individuals are conducting
evaluations at a ratio of 2:1 to local teams
• Mixed teams are less common than local teams despite
the potential mixed teams present
• The development agenda necessitates growing local
capacity, enhancing buy-in and stimulating demand
• This has implications for ownership, accountability,
alignment and results!
Towards an African approach to
M&E
There are increasing efforts to explore the fusion
between local experiences and knowledge systems
and other systems of ‘knowing’ in order to replace or
enhance current dominant discourses.
African evaluators and other stakeholders
need to own and advance monitoring and
evaluation theory and practice.

Source: The Bellagio Report. 2012. African Thought Leaders Forum on Development Evaluation
Some Key Priorities for the
Evaluation Profession in Africa
• Mastery of critical monitoring and
evaluation approaches and methods
• Innovation in M&E theory and practice
• Positioning the evaluation profession in
Africa and globally

Source: The Bellagio Report. 2012. African Thought Leaders Forum on Development Evaluation
Towards an African approach to
M&E
Small group discussions
• What does an African approach to M&E mean for
you?
• How is it different? How is it similar?
• Are there any aspects of your M&E experience that
you consider distinct in terms of an African-rooted
approach?
Key Definitions
 Development Evaluation
• A sub-discipline of evaluation
• Originates with multilateral development
organisations
• Has evolved over time, acknowledging increased
complexity
• Focuses on evaluation for development results
• Uses a variety of methods and practices
• Mixed methodologies work best for triangulation

Morra Imas,L.G. and Rist, R. 2009. The Road to Results: Designing and Conducting Effective Development Evaluations. Washington: The World Bank
 Monitoring, Research, Auditing,
Appraisal or Evaluation?
A continuing function that
uses systematic collection
of data on specified
indicators to provide
management and the main
stakeholders of an ongoing
development intervention Monitoring
with indications of the
extent of progress and
achievement of objectives
and progress in the use of
allocated funds.
 Monitoring, Research, Auditing,
Appraisal or Evaluation?
The systematic and objective
assessment of an on-going or
completed project, programme
or policy, its design,
implementation and results. The
aim is to determine the
relevance and fulfilment of
objectives, development
efficiency, effectiveness, impact Evaluation
and sustainability. It should
provide information that is
credible and useful, enabling the
incorporation of lessons learned
into the decision–making
process of both recipients and
donors.
Impact, Outcome, Output, Activity
or Input?

The likely or achieved

short-term and medium-
term effects of an
intervention’s outputs. Outcome
Impact, Outcome, Output, Activity
or Input?
The products, capital
goods and services which
result from a
development
intervention; may also
include changes resulting
Outputs
from the intervention
which are relevant to the
achievement of
outcomes.
Impact, Outcome, Output, Activity
or Input?

The financial, human, and

material resources used
for the development
intervention. Inputs
 Impact, Outcome, Output, Activity
or Input?
Quantitative or
qualitative factor or
variable that provides a
simple and reliable
means to measure
achievement, to reflect
the changes connected to Indicator
an intervention, or to
help assess the
performance of a
development actor.
Impact, Outcome, Output, Activity
or Input?
A representation of how
an organization or
initiative is expected to
achieve results and an
identification of the
underlying assumptions Theory of Change
made.
More on Theory of Change

• See also R2R pages 141-171 and Chapter 3 in

Evaluation Management in South Africa and Africa
End of Module 1.1F
LUNCH BREAK