B Sc Honors in Monitoring and Evaluation

Participatory Monitoring and Evaluation

Course Code:HSME: 1204
Participatory Impact Assessment (PIA)

Lecturer: Prosper Mupa

Email: mupa.prosper@gmail.com
Mobile:0772851973 or 0716023844
Work:0284-20539 Home: (09) 214274
 Presentation
• Overview of participation in M & E
• Definition of PIA
• Origins of PIA
• Outline the eight steps of PIA
 Introduction/Background information
• Bottom up approach now replacing top down
approach.
• .The focus has been extended from participatory
programming to P-M & E and PIA
• However development agency remain reluctant to
fully implement participatory approaches
• Reasons cited are: costs, time and lack of skills to
facilitate community participation
• The top down approach used to be the
conventional style of development. However
this had many flaws and was not effective
• Top-down approach raised questions about
whether ‘outsiders” had the right or knowledge
to set the development agenda of indigenise
people. In the 1980’s the development
practitioners then explored more alternatives
that avoided some of these problems
Origins of PM & E
2.
Definitions of PM & E
•Participatory monitoring and evaluation (PM&E) is a
process through which :
• Stakeholders at various levels
• Engage in monitoring or evaluating a particular project,
program or policy
• Share control over the content, the process and the results of
the monitoring and evaluation activity and
• Engage in taking or identifying corrective actions.

• (WORLD BANK 2010)

6
 Rationale for community participation

• Why participation? (RIFKIN and KANGERE 2002)

PARTICIPATION

• People know what works for them and

professionals need to learn from people.
• People make contributions of resources (money, materials,
labour) for these programmes.
• People become committed to activities that they have
helped develop.
• People can develop skills, knowledge and experience that
will aid them in their future work.

7
 What’s Participatory Impact Assessment?

“Involves the adaptation of participatory tools combined with more

conventional statistical approaches specifically to measure the
impact of humanitarian assistance and development projects on
people’s lives.”

Source: Feinstein International Center: Participatory Impact Assessment” Guide for Practitioners

8
 Origins: Development Theory
• Community participation: The involvement of
communities in description and analysis of problems,
and identification of solutions.
• It leads to joint design and implementation of action
• It often requires profound attitudinal change among
professionals
• Professional act as facilitators, co-learners and mentors
• Ultimately, it relates to issues of power, control,
respect and ownership
• Local people have their own perceptions of change, their own
ways of measuring change and their own analysis of project
attribution
• The perceptions not only matters, but within participatory
way of working they are crucial to impact assessment and
refining future interventions
• However poor people or vulnerable people posses specialist
knowledge and skills (human capital). Rather than seeing
people as ignorant, uneducated, irrational, participatory
approaches highlights indigenous knowledge in areas such as
farming methods and livestock husbandry.
 Origins: Process versus Impact
• Two main types of indicators are measured in
aid projects:
– Process indicators:
• measure things being done e.g. quantity of medicines
delivered
– Impact indicators:
• measure the outcome of things being done e.g. changes in
the prevalence of disease
– Assessment of aid projects dominated by process
rather than impact – the “truck and chuck” mentality
 Participatory versus Conventional IE

Conventional IE Participatory IE
Who? External experts, IE specialists Stakeholders, including communities
and project staff; service providers
and users
What? Predetermined indicators, to Indicators identified by stakeholders,
measure impact to measure impact

How? Questionnaire surveys, by outside Simple, qualitative and quantitative

“neutral” evaluators, distanced
Why? To make project and staff
accountable to funding agency
Method? Pre-determined
methods, by stakeholders themselves
guided by facilitator
To empower stakeholders to take
corrective action
Adaptable

12
Source: Adapted from Deepa Narayan, World Bank.
 At the project level-3 key questions…

1. What changes have there been in the

community since the start of the project?
2. Which of these changes are attributable to
the project?
3. What difference have these changes made to
people’s lives?
 Designing and Implementing PIA: Steps
Stage 1: Define Questions
Stage 2: Define the geographical and time-limits of the project
Step 3: Identify locally defined impact indicators
Step 4: Decide on ranking/scoring methods
on and testing sampling methods
Stage 5: Choose Sampling Methods
Stage 6: Assess project attribution
Stage 7: Triangulation
Stage 8: Feedback and verify results with community

Source: Feinstein International Center: Participatory Impact Assessment” Guide for Practitioners
 Stage 1. Identify the key questions
 The most important, and often the most difficult stage of a PIA is
deciding which questions should be answered.
 Who defines the questions?
 If you’ve already worked with communities to identify their impact
indicators at the beginning of a project, the questions will focus on
the measurement of these indicators and assessment of project
attribution.
 If you’re using a retrospective approach, discuss the impact
assessment with community partners / representatives and jointly
define the questions with them.
 Defining the questions in an impact assessment is like defining the
objectives of a project – unless you know, specifically, what you’re
trying to achieve, you’re unlikely to achieve it
 Stage 2. Define the boundaries of the project in space and time

• Define spatial unit of analysis: geographical

boundaries of the project (mapping)
• Aims to ensure that everyone understand the limits of
the area in which impact is supposed to take place.
– Participatory Mapping
– “Interview the map”: When the group finished the diagram
– Cross-check community map with other groups (villages)
and secondary data (such as project document, etc.)
• Define temporal boundaries of the project (timelines)
• Define temporal boundaries of the project
(timelines)
– Sometimes called the “temporal boundary”.
When did the project start, how long has it been
running? Aims to ensure that everyone is clear
about the time period which is being assessed.
Step 2, Tool #1: Defining the project boundary: participatory
mapping

A map of Zipwa Site, Zimbabwe Community members drawing a map in the sand
 Step 2- Tool # 2: Define the project period by timelines
established by the communities

Creating a timeline--
- Identify a Knowledgeable
person (or persons) in a
community
-Ask them to describe the
history of the community.
-- In many rural communities,
such descriptions usually refer
to key events such as drought,
periods of conflict or disease
epidemics
- The project start and end
time should be related to
these key events.
 Stage 3. Identify community-defined indicators of project impact

1. Who identifies the impact indicators?

a) As far as possible, the PIA should use impact
indicators which are identified by the
community or beneficiaries of the project.
i. Local people have their own way of describing
change, and their own priorities for improving their
lives
ii. Participatory impact assessment (PIA) works with
communities to identify locally-defined impact
indicators
2. How can we identify community impact
indicators?
3. Specific versus general indicators
4. Number of indicators
5. Quantitative and Qualitative Indicators
 Stage 3: Indicators-identified by communities themselves…

Ex: Drought projects in Zimbabwe and Niger

Impact indicator by project M&E
specialist
increased crop production
dietary diversity
Impact indicators by beneficiaries
The ability to pay for school fees using project
derived income (education benefits)
The ability to make home improvements
Improved skills and knowledge from the projects
training activities

Improved social cohesion

Tips for practitioners: Make sure to capture the views of different groups of
people within the community. (Women will often have different priorities and
expectations of project impact than men.)
 STAGE FOUR: METHODS
• This section provides both real life and hypothetical
examples of how different methods have been or
might be used to measure project impact on
livelihoods.
• The exact tools used in these examples may or may
not be transferable to other projects or assessments.
• Decide on the participatory methodology you will
use
 Simple participatory methods
• Lifeline / Quality of Life Curve: Community members
rate each year that they can remember on a scale
from 1 (lowest rating) to 5 (highest rating) on a chart.
They then mark the significant events that caused
them to give the ratings (e.g. drought, building of a
borehole, disease outbreak, introduction of farm
subsidies, etc.)
• Road Journey Diagram: Community members draw a
picture of a road that describes changes in their
community over time. Eg, the road may show
pictures of a new school being built, or people
fighting within the village. The road can be extended
into the future.
• Activity List: Community members list all the activities
being run by different organisations in their area. They
then rate activities based on their importance, who
benefited, and how much time and effort they put into
the activities.
• Influence Matrix: Community members list the areas of
their lives that have changed (e.g. income, skills, food
security, etc) down one side of the table. On the other
side of the table they list the activities of the program.
They then rate on a scale from 0 to 10 how much each
activity has influenced each part of their lives.
 others
• The following examples:
– Mapping
– Time-lines
– “Before and after” proportional piling*
– “Before and after” scoring*
• Simple ranking*
– Matrix scoring
– Unserialised posters
– Force-field analysis
• Focus Group Discussions
• Social Maps
• Transect walks
• Role play
• Tree diagrams
• Needs Assessment
• SWOT analysis
• Wealth ranking
• Pair wise ranking
• Historical lines
• Access to resources profile
• Listing
• Ranking
Step 4: Methods for Scoring/Evaluation project
Tool #1: Scoring of Food sources using counters- Evaluating the impact
participants
of a community garden
identify all the
food sources
that contribute
to the
household food
basket.

Practitioner Tips-
Where informants
are literate you
may choose to
simply write the
name of
The indicator on a
card.
Stage 4-Tool # 2: Impact calendars-post-harvest food
balance exercise -done with project participants for
the agricultural year before and after the
Monthly household utilization of the harvested project and again for the agricultural year.
The exercise then repeated with community
maize until depletion (using 25 counters)
members who had not participated in the
project
Timelines
Timelines
Social Mapping
 Mapping of Assets

Luvuluma Village N
To
Ndolwane
Mazwaligwe Makumbi Village
Primary W E
School
Masendu
Tjebhoroma
Central Masendu Makumbi
Primary Village Primary
Village School School S
+ `

Tondimelani
Pre-School

Nopemano
Primary To
School Plumtree Town
Mambo Village
KEY
Thandawani Village School
+ Clinic
River

Tekwane River
To Dam Business
Madlambuzi Major Road Center
33
Minor Road
Graffiti Wall
Opportunity Youth Elderly Cattle Young Unempl Coffee
Men Keepers Women oyed growers

Improve
sanitation in
people’s houses
Add classroom
block to school

Repair the bridge

Develop credit
schemes for
CBOs
Get extension
support for
dealing with
coffee wilt
disease
35
 Stage 5: Measure changes in the impact indicators which
occurred during the project

• If possible, use community-defined indicators

of impact
• “Before and after” scoring methods are useful.
For each indicator,
• What was the situation (score) at the start of
the project?
• What is the situation (score) now? If there is a
difference, why? Probe these reasons
 Stage 6: Assess project attribution
• This stage aims to understand all the project
and non-project factors which contributed to
the changes in the impact indicators – these
factors should be listed.
• Also, it aims to understand the relative
importance of these project and non-project
factors – methods include simple ranking or
scoring methods, and causal diagrams with
scoring of causes.
 Stage 7: Triangulate
• Triangulation is a crucial stage of the
assessment, and involves the use of other
sources of information to cross-check the
results from the participatory exercises.
• Key sources: researches, M & E reports
Stage 8: Validating the finding by the community
• This final stage involves presentation of the
PIA findings back to the community.
• If a CBO or local group is involved in the
project, they should receive a copy of the
results (and the final report).
• This stage is a final opportunity for local
people to verify that the results are correct.
• A “feedback” workshop can also be an
opportunity to plan further work to improve
the project.
 Application of PIA

• PIA is now used worldwide by Government

research institutes, NGOs, and others
– Impact of humanitarian and development projects
– Understanding of resilience to climate variability
and extreme events in coupled social and
ecological systems
 Conclusions
• Participatory impact assessment helps to
bridge two gaps:
• The limited understanding of the livelihoods
impact of relief and development
interventions – implications for organizational
learning and national-level policy reform
• Methodologically, a gap between the ad hoc
use of participatory methods and systematic
research with statistical analysis
• Correct use of the approach requires specialized
training and experience in participatory approaches
and methods
• PIA can be useful because:
– It focus on changes which local people prioritize and can be
measure using their own indicators, methods and resources
– It is based on indigenous knowledge and skills which is
based on continuous monitoring and assessment of
changes
– It uses participatory methods that often describe trends or
compare variables across spatial and temporal scales,
rather than seek to make precise measurements
 Ground Rules when using PIA Tool

• Give everyone the opportunity to participate

• Respect what people say
• Don’t dominate
• Be on the same level as the participants
• Don’t make assumptions
• Don’t rush
• ‘Hand over the pen’
• Learn from mistakes
• Be flexible
• Make it Fun
 Transect Walk
• A location can be divided into sections and shared between several small
groups.
• Explain the purpose of transect walks to participants and together decide
on any key issues to be explored, or key areas to be identified, during the
walks.
• Decide on the location to be 'walked'. Plan, approximately, routes to be
taken.
• Keep routes flexible to allow for the unexpected.
• Decide on any key issues to be discussed with people met during the
walk.
• Keep these plans flexible to enable people in the community to identify
issues of concern. Issues identified by people in the community may be
different from those expected by the group.
• Agree a time and location for groups to come back together to share and
discuss
• what they have learned during the walks. If useful, the information can be
• A variation of a transect walk is historical transects. These
are diagrams showing how the same location has changed
over time.
• The time – of the day, week or season – when a transect
takes place will influence what is learned. It may be useful
to return to particular locations at different times. Transect
walks can include periods of observation – for example,
when having a
• drink in a bar – as well as active walking and exploring.
• If cameras and videos are used, remember to ask
permission from anyone beingphotographed or filmed.
 Timeline Description
• A timeline is a good way to encourage
participants to reflect upon their lives or upon
the history of an area. It can be used as a way
of introducing people in a group, although it
can take a long time! Trends (highs and lows)
can then be added to the line. The creator(s)
of the timeline can explain the key features
and issues raised to the facilitator, with the
focus of attention on the drawing.
 Lifeline
• What is it?
– Lifelines show events and experiences in the
– lives of people that are important to them.
– Lifelines can also show the history of
– organisations or places.
• Why use it?
• help people reflect on their own and other
• people's experience
• •
• organise events and experience in historical order
develop a case study of a person, organisation or
place
• understand how culture, social norms and practices
affect individuals, and compare how people are
affected differently by these
• explore changes in individual needs over time – for
example, the sexual and reproductive needs of
females from childhood to adulthood – and ways in
which these needs can be met.
 Seasonal calendar
• A seasonal calendar is a diagram of changes over the
seasons – usually over the period of about 12 to 18
months.
• Why use it?
– identify seasonal patterns of change – for example, changing
availability of resources
– such as food or income, work and migration
Patterns
• explore relationships between different patterns of change
– for example, the relationship between income levels and
movements of key populations for work
• identify when people may be particularly vulnerable
• explore seasonal patterns of well-being and hardship
and how different people are affected. For example,
when do people have the most income? When do
people have free time?
• identify when people are particularly vulnerable to
infection – for example, around big festivals
• identify times during the year when people are
available for involvement in community action.
 Pocket Chart
• Use: To assist individuals and communities to
identify, assess their and practices. It also
helps them to prioritise their needs. It
stimulates and can also be used as an
investigative tool in carrying out evaluation
Spider Diagrams
 Spider diagram
• What is it?
• This tool involves participants drawing a diagram
like a spider to identify the problems or
opportunities about a subject relating to
HIV/AIDS. This helps with solving problems and
identifying solutions.
• Spider diagrams are also called sunburst
diagrams.
• .
• Why use it?
• Using spider diagrams helps to:
• provide a non-threatening way to identify the key
problems or opportunities about sensitive issues eg
HIV/AIDS
• assess which of those problems or opportunities
are most important, and why begin to identify how
to overcome problems begin to identify how to
make the most of the opportunities
• Mapping can be used as an engaging and fun
way to convey understanding and awareness
of an area. It is a great introductory tool to
generate initial data and is will attract people
and create discussion.
 Graffiti Wall Description
• A graffiti wall allows individuals to write or
draw their comments and suggestions: these
can then be and analysed (clustered, themed
and prioritised) to generate group discussion.
 NB about Graffiti Walls
• Form of visual communication, usually illegal,
involving the unauthorized marking of public space
by an individual or group. Technically the term
applies to designs scratched through a layer of
paint or plaster, but its meaning has been extended
to other markings. Graffiti is widely considered a
form of antisocial behaviour performed in order to
gain attention or simply for thrills. But it also can
be understood as an expressive art for
 H-Form Description
• An H-form considers both the positives and
negatives of an issue, before focusing on
possible solutions to the negative aspects.
While individuals usually write or draw their
views, the format allows for group discussion
and further detail to be brought out.
Force Field Analysis
 Force-Field Analysis Description
• Force-field Analysis considers both the positives
and negatives of an issue, before focusing on
possible solutions to the negative aspects and
ideas. The distance from the centre line shows
how good or bad the issue is, the fatness of the
arrow can be used to denote the importance of
the issue. While individuals usually write or
draw their views, the format allows for group
discussion and further detail to be brought out.
Causal-Effect Diagram
 Causal-Effect Diagrams
• A Causal Impact Diagram is great at looking in
depth at a problem to examine not only the
effects of the problem, but also the causes.
This is important when trying to devise
solutions, as we often only consider solutions
to the effects and tend to ignore the causes.
Bean-Counter Prioritisation
 Bean Counter Prioritisation Description

• A Bean Counter prioritises suggestions and

comments made by the group. Each
participant uses a specific number of dots to
vote for their priorities from a list generated
from their previous comments.
Criteria Ranking
 Criteria Ranking Matrix
• A Criteria Ranking Matrix helps participants
prioritise from lists or options identified
through other tools during earlier exercises in
the participatory process. After identifying
things which they want to rank or prioritise,
normally by using one of the earlier tools,
participants have to come up with criteria by
which all of the options can be ranked
Pair-Wise Ranking
 Pair-wise Ranking
• Pair-wise Ranking can be used to make a final
choice, especially if only one action needs to
be found. Participants choose between the
options on the matrix. This can either be done
by group consensus (especially if the group is
an odd number!) or individual voting.
Impact Ranking
 Impact ranking
• Impact ranking helps participants to judge
between the difficulties of doing something
again the impact. This can be a very helpful
first step when trying to prioritise projects.