Who Are The Question Makers - Undp Participatory Evaluation

Who are the question-makers?
Who Are the Question-makers?

A Participatory Evaluation Handbook
OESP Handbook Series
Editorial Board: Sharon Capeling-Alakija, Carlos Lopes, Abdenour
Benbouali
and Djibril Diallo
Managing Editor: Janet Donnelly
© OESP, 1997
Office of Evaluation and Strategic Planning
United Nations Development Programme
One United Nations Plaza, New York, NY 10017
Foreword
Introduction
● Purpose of the Handbook
● Audience
● Organization of the Handbook
Part One:
Participatory Evaluation: An Overview
● Evolution of the Participatory Approach

● Participatory Action Research
● Rapid Rural Appraisal and Participatory Learning and
Action
http://www.undp.org/eo/documents/who.htm (1 of 4) [29/01/2002 18:59:07]

● Farming Systems Research

● Self-Evaluation and Beneficiary Assessments
Part Two:
Participatory Evaluation
● Functions of Participatory Evaluation

● Differences Between Participatory and More
Conventional Evaluations
● Rationale for a Participatory Approach to Evaluation
Part Three:
Participation within UNDP
● Participation as a Philosophy
● Participation and UNDP Programming
Part Four:
How to do a Participatory Evaluation
● Framework
● Practical Considerations
● Possible Reactions to the Participatory Evaluation
Process
Part Five:
Training Module ''Money and Mambas''

● Introduction
● Purpose
● Audience
● Format.
● Guide to the Facilitator
Money and Mambas or Listening to the People. Case

Study: Participatory Evaluation of a Water and Sanitation
Project
Questions for Group Work
Annexes:
Annex I.
● Glossary
Annex II.
● A Sampling of Tools for a Participatory Evaluation
Annex III.
● Who's Doing What? Organizations Supporting

Participation
Annex IV.
● Resource Persons, Groups and Institutions

Bibliography

Who are the question makers? - Foreword
Foreword
"Who Are the Question-makers?: A Participatory

Evaluation Handbook" is the first in a new handbook
series being launched by the UNDP Office of
Evaluation and Strategic Planning (OESP). This
series aims to provide development practitioners
with tools, examples, exercises and case studies
that will help in translating plans into actions and
theory into practice.
This handbook evolved from work undertaken during

the past seven years in UNDP, first by the Central
Evaluation Office (CEO) and in the past two years by
OESP. Interest in participatory evaluation bas
deepened as UNDP has striven to improve interaction
with and serviced delivered to end-users and
beneficiaries, particularly at the grass-roots
level. Work began in late 1989 with an occasional
paper commissioned by CEO entitled "Participatory
Evaluation: Questions and Issues" prepared by Kim
Forss. This was followed by a research and pilot
test phase during which three project evaluations
were conducted along participatory lines.
While the original intention of undertaking these

pilot exercises was to produce guidelines on
participatory evaluation, as we gained more
experience, we realized that participation in not a
process that can be mandated from above or guided
from the centre. In this realization, we in OESP
travelled the same path as the authors of the World
Bank Source Book on Participations who found that
"the best way of learning about participation is to
experience it directly. The second best way is by
seeing what others have done in the name of
participation, talking to them, and seeking their
guidance".
http://www.undp.org/eo/documents/wfore.htm (1 of 3) [29/01/2002 18:59:12]

The present handbook has been designed to capture

OESP's learning on participation and share it with
you in the hope that you will be motivated to try
the best method (doing something yourself) through
experiencing the second best method (reading about
what others have done)! To achieve this, the
handbook is divided into two distinct sections: the
first provides an overview of participatory
evaluation while the second includes a self-
contained training module consisting of a case study
that documents an attempt at participatory
evaluation, warts and all. This case study has been
successfully used in training courses conducted for
Junior Professional Officers during the past two
years.
A few works of explanation will help you to

understand why we chose the title "Who are the
Question-makers?" As Michael Quinn Patton, one of
the most eloquent and persuasive advocates for user-
focused evaluation, puts it "Language matters. It
simultaneously suggests possibilities and
communicates boundaries." In this instance, the
unfamiliar conjunction of "question maker" as
distinct from the more usual "question asker" was
chosen deliberately because it communicates a more
active involvement of the stakeholder in the process
of evaluation. A question-maker has more
responsibility for seeking the answers than a more
passive question-asker.
This handbook has been a collaborative effort by

many people and we would like to take this
opportunity to acknowledge their contributions: our
thanks to Jennie Campos and Francoise Coupal, who
helped to put the handbook together; to Kim Forss
and Claus Rebien, who put the original case study
together; and to the informal brainstorming and
feedback group of Nurul Alam, Abdenour Benbouali,

Janet Donnelly, Naheed Haq, Peter Hazelwood, Rosein

Herweijer, Mala Liyanage, Rema Pai Nanda, Kaarina
Valtasaari, Samir Wanmali and Rob Work. Carlos Lopes
provided valuable insight and acted as a sounding
board throughout the preparation of the handbook.
Hearty thanks are due to the JPOs who participated
in the induction coursed held in New York from late
1994 through 1996. The detailed feedback we received
from these training sessions was of particular value
to us in preparing section two. We would also lie to
acknowledge Barbara Brewka for her excellent editing
work, enforcing rigour in the way we express
ourselves, and Maureen Lynch for coordinating the
design and layout stages of the publication.
Finally, like all successful projects, this handbook
had its champion. Chandi Kadirgamar's unstinting
devotion to this initiative at every stage from
conceptualization to publication is a clear
manifestation of her commitment to fostering
participatory practices in UNDP.
We hope this handbook proves worthy of its name and

is frequently referred to, not only because it
proves to be a useful resource but also because it
is enjoyable reading. We see this as the first
version of many and would therefore look forward to
hearing your ideas and suggestions on how it can be
improved. Your input in expanding the annex on
resource persons, groups and institutions who have
experience in participation would be particularly
appreciated.
Sharon Capeling-Alakija
Director
Office of Evaluation and Strategic Planning

Who are the question-makers? - Introduction
Introduction:
Experience has shown that participation improves the quality, effectiveness
and sustainability of development actions. By placing people at the centre of
such actions, development efforts have a much greater potential to empower
and to lead to ownership of the results.
The UNDP Office of Evaluation and Strategic Planning (OESP) has been
assessing the value and role of participation as part of its broader effort to
redefine the function and role of evaluation within the organization. During
this process, OESP has had to address several key questions, including:
● How can evaluation be a tool for development?

● How can evaluation build local capacity and contribute to a learning
culture?
● How can evaluation contribute to the achievement of sustainable
human
development (SHD)?
● What is the value added of more responsive evaluation methods and
would such methods require a change in attitudes and skills by
UNDP staff?
In its search for answers to these questions, OESP has been experimenting
with initiatives that involve greater participation of programme stakeholders
and beneficiaries. Commonly referred to as "participatory evaluations", these
experiments challenge the traditional way in which development is viewed
and carried out.
While to some people, the participatory approach may represent a radical

departure from past practices, others see it as a logical step in the evolution
of development thinking and methods. For example, UNDP policies to
promote decentralization, national execution of programmes and
partnerships have all sought to transfer ownership to its partners in
programme countries. Further decentralization is taking place within
developing countries. Grass-roots efforts, bottom-up approaches, initiatives
that empower are all focusing attention on the poor and disenfranchised,
whose opinions and participation are increasingly being sought.
http://www.undp.org/eo/documents/wintro.htm (1 of 4) [29/01/2002 18:59:23]

The growing interest in participatory evaluation parallels the growth of such

concepts as empowerment, democratization, partnership and sustainability.
Each of these concepts attempts in one way or another to give a greater say
to the spectrum of voices in our programming countries-not only to national
governments but also to civil society, communities and municipalities, the
poor and the disenfranchised - who have been the object of development
cooperation and whose voices have not been adequately heard.
Purpose of the Handbook

In view of the growing importance that is being attributed to participation,
this handbook has been prepared to:
● provide UNDP staff with a better understanding of what is meant by a

participatory approach to evaluation and how they can support the
participatory evaluation process;
● help to introduce participatory evaluations into UNDP programming,
thereby enabling the multitude of stakeholders that are central to
UNDP
development efforts-the poor, local communities, Governments-to
have
a much stronger voice through development efforts that are more
responsive to their needs and that contribute to capacity-building at
the
local and national levels;
● strengthen the learning and management culture of UNDP.
Audience
The principal users of the handbook, which has been designed primarily for
UNDP staff, include:
● Resident Representatives and Deputy Resident Representatives, who

over
see the country programmes and make critical decisions about the
allocation of resources, which programmes and projects will be
evaluated

and the approach to be used;

● National and International programme staff and Junior Professional
Officers (JPOs), who are directly responsible for the day-to-day
manage
ment of projects and who appraise, review or evaluate development
activities and need assistance in designing field missions and
preparing
terms of reference (TOR).
UNDP staff may also want to share this handbook with colleagues who are
interested in applying participatory evaluation techniques to their projects.
Thus, it will also be useful for Government counterparts, project leaders and
consultants who need to have a better understanding of how a participatory
evaluation works and how it fits into UNDP programming.
Organization of the Handbook

This volume provides the information needed, and helps to develop the
sensitivity and skills required, to support evaluations that place greater
emphasis on stakeholder participation in the evaluation process. It is divided
into five parts.
Parts one to four, which present an overview of the participatory evaluation

approach, include:
● a brief description of the evolution of the participatory approach;

● a comparison of participatory evaluation with more conventional
evaluation approaches;
● a discussion of the role of participation in UNDP;
● a description of the framework of a participatory evaluation and a
discussion of some of the practical issues involved in doing such
an evaluation.
Part five consists of a stand-alone package developed around the case study
MONEY AND MAMBAS. It describes an attempt at undertaking a
participatory evaluation of a rural water supply and sanitation project and
focuses on the practical aspects of applying participatory evaluation
techniques:

● Pre-planning, including negotiation of the TOR, assessing the

participatory evaluation context and identifying enabling and
inhibiting
factors surrounding that context;
● Collaborative planning with stakeholders;
● Data-gathering and analysis;
● Reflection and follow-up.
This case study is presented as a training module which can be the subject of
a mini-workshop to introduce staff to the practice of participatory evaluation.
We suggest that this exercise can be accomplished within 3 to 4 hours.
A glossary of basic terms, examples of some of the basic tools that can be
used in participatory evaluations as well as lists of manuals and resource
persons, groups and institutions are presented in the annexes.

Who are the questions makers? - Part One
Part One:
Participatory Evaluation: An Overview
Evolution of the Participatory Approach
The emergence of what has become known as the participatory evaluation
approach reflects much wider experimentation in development that has been
taking place in various parts of the world since the 1970s. It has primarily
involved development practitioners and social researchers in a wide variety
of fields, e.g., adult education, sociology, rural development, agriculture and
applied research. Only now has it entered the policy-making spheres of large
development agencies.
What is increasingly being called participatory development began for some

with the critical analysis of society and the inequities it generates, leaving
the poor voiceless and dominated. For others, participatory development is
less ideological or philosophical: it started with the exploration of more
responsive techniques and approaches at the grass-roots level, involving the
poor, project stakeholders and beneficiaries. For those involved specifically
with evaluation, there has been a growing dissatisfaction with conventional
modes of assessment that claim to be scientifically neutral and unbiased yet
have had very little impact on how development activities are carried out.
The following pioneers or schools of thought have contributed to the

emerging field of participatory development and, more specifically, to
participatory evaluation.
Participatory Action Research

Participatory action research (PAR) has its origins in the work of social
scientists from developing countries who have been experimenting with
PAR over the past 20 years. Influenced by such authors as Paulo Freire,
Orlando Fals-Borda and Mohammad Anisur Rahman, the "basic ideology of
PAR is that self-conscious people, those who are currently poor and
oppressed, will progressively transform their environment by their own
praxis. In this process others may play a catalytic and supportive role but
will not dominate" (Fals-Borda, 1991:13). Along similar lines, Paulo Freire,
in his book Pedagogy of the Oppressed, outlines an educational philosophy
that actively involves the poor in critically analysing their social situation,
http://www.undp.org/eo/documents/whop1.htm (1 of 3) [29/01/2002 18:59:47]

thus creating the potential for them to transform their environment. Once
considered radical, the work of these authors is gaining increasing
prominence and is credited with critically challenging mainstream thinking
and influencing the development of participatory development.
Rapid Rural Appraisal and Participatory Learning and Action

Rapid rural appraisal (RRA) first emerged in the late 1970s, spearheaded by
Robert Chambers at the University of Sussex, England, in response to
lengthy assessment methods used in development. RRA enables donors to
seek information and insight quickly from local people about local
conditions. Over time, RRA sought to be less extractive and more
participatory in the collection of information by involving local people in
data-gathering and analysis through the use of popular education methods,
such as mapping; transect walks; scoring and ranking with seeds, stones or
sticks; and institutional diagramming. As the emphasis shifted from
collecting data quickly to the involvement of end-users and learning from the
experience, RRA became known as participatory learning and action (PLA).
PLA activities have been undertaken in over 130 countries by development
practitioners, NGOs and donors.
Farming Systems Research

Research in this field emerged in the 1970s, mainly in response to concerns
about the skewed benefits of the Green Revolution. In contrast to research
station experiments, which were difficult to replicate in the field, systems
research supported farmer-managed trials in which rural people selected
alternatives for experimentation and implementation. It recognized the
breadth of knowledge farmers had of their own interrelated systems of
production and livelihood and supported experiments conducted by
the farmers.

Self-Evaluation and Beneficiary Assessments

The term "self-evaluation" is most often used to describe a process of
permanent, internal evaluation involving staff at all levels or beneficiaries
with a view to generating information that can inform decision-making.
NGOs, such as World Neighbors, academics and donors have been
experimenting with the concepts of self-evaluation and beneficiary
assessment. As a result of its experiences with beneficiary assessments, the
World Bank views them as essential to building programmes that are
responsive and relevant to recipients of Bank loans, providing Bank
managers with the tools to improve the quality of development operations.
As mentioned previously, all of these approaches and schools of thought

have influenced the emerging field of participatory development.

Who are the question-makers? - Part Two
Participatory evaluation, a dimension of participatory development embodying many of the same
concepts, involves the stakeholders and beneficiaries of a programme or project in the collective
examination and assessment of that programme or project. It is people centred: project stakeholders and
beneficiaries are the key actors of the evaluation process and not the mere objects of the evaluation.
Participatory evaluation is reflective, action-oriented and seeks to build capacity by:
● ?providing stakeholders and beneficiaries with the opportunity to reflect

?on a project's progress and obstacles;
● ?generating knowledge that results in the application of lessons learned
?and leads to corrective action and/or improvements;
● ?providing beneficiaries and stakeholders with the tools to transform their environment.
Functions of Participatory Evaluation

Participatory evaluation thus serves four key functions, some of which concern the stakeholders and
beneficiaries while others relate to the funding agencies.
● It helps to build the capacity of stakeholders to reflect, analyse and take

?action. While such analysis should occur throughout the life of a project,
?it is never too late to involve project recipients in evaluations at mid-
?term or even at the end of a project. UNDP staff may also witness their
?own growth and enrichment through their involvement in the evaluation process.
● It contributes to the development of lessons learned that can lead to
?corrective action or improvements by project recipients. When project
?stakeholders are involved in analysing problems, constraints and
?obstacles, they can often propose solutions. Their sense of ownership of
?the process, of final recommendations and of action plans makes them
?much more likely to introduce necessary changes.

● It provides feedback for lessons learned that can help programme staff
?to improve programme implementation. A participatory evaluation not
?only looks into the past but also guides projects into the future.
● It helps to ensure accountability to stake-holders, managers and donors
?by furnishing information on the degree to which project objectives
?have been met and how resources have been used. Answers to these
?questions will help programme managers make critical decisions about
?continuing or terminating a project's funding.

The focus on lessons learned is an essential dimension of participatory evaluations. Such evaluations
should help to guide projects into the future by giving stakeholders the tools with which to take
corrective action. In addition, lessons learned should provide donors with the insight and tools to
improve programme delivery and management
Differences between Participatory and More Conventional Evaluations

Participatory evaluations differ from more conventional evaluations in several critical ways. Figures 1
and 2 illustrate some of these differences.
As shown in Figure 1, conventional evaluations have been more donor focused and donor driven. The
donor is the key client, providing financial support and defining the TOR for the evaluation.
Participation of project stakeholders in the definition of the TOR is minimal. More often than not, the
evaluation is carried out more to fulfil a management or accountability requirement than to respond to
project needs. An outside expert or evaluator is hired to conduct the evaluation.
The evaluator collects the data, reviews the project or programme and prepares a report. In most cases,
stakeholders or beneficiaries play a passive role, providing information but not participating in the
evaluation itself. The process can be considered more linear, with little or no feedback to the project.
In a participatory evaluation, the role and purpose of the evaluation change dramatically. Such an
evaluation places as much (if not more) emphasis on the process as on the final output, i.e., the report.
The purpose of the evaluation is not only to fulfil a bureaucratic requirement but also to develop the
capacity of stakeholders to assess their environment and take action.
Stakeholders and beneficiaries do more than provide information. They also decide on the TOR, conduct
research, analyse findings and make recommendations. The evaluator in conventional evaluations
becomes more of a facilitator in participatory evaluations, animating workshops, guiding the process at
critical junctures and consolidating the final report, if necessary, based on the findings of the
stakeholders. The process is much more circular, as shown in Figure 2.
Participatory evaluations also call into question the notions that only scientific inquiry provides valid
information and that outside experts or those independent of the project or programme somehow hold the
ultimate truth. Participatory evaluations recognize the wide range of knowledge, values and concerns of
stakeholders and acknowledge that these should be the litmus test to assess and then guide a project's
performance.
While the participatory approach to evaluation poses its own challenges, it has the capacity to empower

recipients. The active participation of stakeholders can result in new knowledge or a better understanding
of their environment. It is this new knowledge and understanding that can enable them to make changes
they themselves have discovered or advocated. Stakeholders feel a sense of ownership of the results
which does not come from an outsider or a donor.
LEVELS OF END-USER PARTICIPATION IN EVALUATION
Dimensions of Low Medium High

evaluation/Levels of
participation
Evaluation initiator Commissioned or External evaluator Evaluation in which

obligatory evaluation invites end-users to end-users collaborate
typically part of assist in one or more with external
programme evaluation task(s). facilitator or among
development. Meets themselves to asses,
institutional needs. review and critically
Evaluation done to, on reflect on strategies
or about people. formulated for them.
Purpose Justify or continue Gain insights into Promote self-

funding. Ensure development activity sufficiency and
accountability. Levels from end-users' sustainability by
of funding or sustained perspective. Shift linking end-users to
support. focus from evaluation planning
institutional concerns cycle. Develop
to end-user needs and relevant, effective
interests. programme decision-
making based on end-
user views, opinions,
recommendations.
Increase ownership in
& responsibility for
success-failure of
development
interventions.
Questions-maker(s) ? Agency heads, End-users with End-users, external

administrators, outside external evaluator at facilitator, persons
clientele, persons various stages of most affected by
distances from evaluation generally development
evaluation site. determined by the intervention.
evaluator.

5 Method(s) Established research Qualitative methods Relies on highly

designs, statistical favored but also interactive qualitative
analyses, reliance on includes quantitative methods but does not
various quantitative methods. Values a disregard quantitative
methods. Product process focussed on tools. "The process is
(findings) oriented open-ended inquiries. the product".
(mathematical in Uses methods that Inventiveness and
nature). Dominated by give voice to creativity encouraged
math whiz kids. voiceless. to adapt the methods
to the context being
evaluated.
Evaluator's versus Evaluator takes lead in Evaluator works Evaluator becomes

Facilitator's Role designing evaluation. collaboratively at more of a facilitator.
Formulates various stages with Facilitator acts as
questions/survey end-users. Is partner in catalyst, confidante,
forms with no input evaluation and imparts collaborator. Takes
from those evaluated. evaluation skills. lead from end-users.
Steers overcome by Shares lead with end- Has few if any pre-
setting users. determined questions.
design.Assumes
objective, neutral,
distant stance.
Impact/Outcome Reports, publications Shared data-gathering End-user more capable

circulated in house. but limited of meaningful decision-
Findings rarely participation in data making based on
circulated among end- analysis. End-user effective involvement
users. Findings loop views loop into in evaluation. Findings
into planning stage planning stage. become property of
with little input from Increased end-users or
end-users. understanding of end- community.
user experiences. Participation in
analysis is critical.
The purpose, methods, role of the evaluator and impact of the evaluation will vary considerably
depending on the type of evaluation and the level of participation of donors, stakeholders and
beneficiaries, as shown in the following table. In evaluations with a high degree of participation by
stakeholders and beneficiaries, for example, the stakeholders rather than the donors become the question-
makers and the evaluations are driven by the stakeholders and recipients.
Rationale for a Participatory Approach to Evaluation

All too often conventional evaluation reports sit on shelves or desks and have little or no impact on
project beneficiaries or development practice either in the field or at headquarters. This can be attributed
in part to a lack of input or feedback from those whose lives are affected by a programme or project, who
have their own perceptions of what they need and how things should be done, yet who have little or no
opportunity to make their views known.

Participatory evaluations breathe life into more conventional evaluation approaches by involving project
stakeholders in all aspects of the evaluation: designing the TOR, collecting and analysing data,
formulating recommendations and making changes in the implementation of a project's activities. In
addition, supplementing more formal methods of inquiry, such as standard questionnaires or one-on-one
interviews, with nonformal techniques can yield richer information than the use of only formal
methods.As a result of the active involvement of stakeholders in reflection, assessment and action, a
sense of ownership is created, capacities are built, beneficiaries are empowered and lessons learned are
applied both in the field and at the programme level, increasing effectiveness. There is growing evidence
that sound, sustainable development requires their participation throughout the development process in
project planning, decision-making, implementation and evaluation.

Who are the question-makers? - Part Three
PARTICIPATION WITHIN UNDP

Participation as a Philosophy
It is not enough to advocate participation; one must also be participatory.
Many have argued that to be truly participatory, it is first necessary to look
inward at the attitudes, behaviours and practices that create the culture of an
organization.
What does it mean to be participatory in the workplace, especially in an

organization such as UNDP, which is characterized by a mosaic of cultures,
attitudes and behaviours? Some staff may feel that their work environment is
highly participatory while others experience attitudes and barriers that
prevent their voices from being heard and taken seriously. Bureaucratic
structures may impede participatory processes.
Being participatory involves more than using a particular technique or

approach. It reflects an attitude towards human interaction and the way that
individuals learn from one another that should permeate all levels of an
institution from headquarters to the field and to the project level. It means
exhibiting a willingness to share decision-making, power and perhaps even
resources.
Empowering others often requires letting go of one's own power or at the

very least sharing the power that one already has, a challenge to those
institutions and groups unaccustomed to working in a collaborative or
participatory manner. However, staff who have been involved in
participatory exercises have often commented on feeling empowered and
seeing creativity unleashed either individually or as a group as a result of
that experience.
Creating a transparent environment is an important dimension of being

participatory. Hidden agendas become open agendas where everyone has the
opportunity to influence and shape events. While this may appear as
idealistic to some, others view it as a need that it is now time to fulfil.
Participation and UNDP Programming

How does a participatory approach fit into UNDP programming? As
illustrated in the circular diagram, such programming is traditionally

composed of at least six stages:
● identification of project/programme concept or problem

● project/programme design
● appraisal
● implementation
● monitoring
● mid-term evaluation and/or end-of-project evaluation.
In addition, the Project Performance Evaluation Report (PPER) and the

Tripartite Review (TPR) are undertaken on a regular basis.
Participatory techniques and evaluations can be used at different stages of a

project: in the design stage, where baseline data and needs analysis are
required; at mid-point, to review progress and reorient project activities; and
at the end to assess the project's achievements. In a participatory approach,
key variables include: who identifies the project concept or problem; who
makes the decisions; and who conducts the evaluation. However, use of such
an approach implies greater involvement of the stakeholders in all aspects of
the project cycle.
Identification of the Project and Programme Concept or Problem

Projects and programmes are more likely to be successful if they respond to
a real need identified by the local population, institution or government
ministry. This means involving the key stakeholders in identifying their
needs and assessing the most appropriate options for meeting those needs.
Project and Programme Design

Traditionally, projects and programmes are designed by an outside expert
according to UNDP specifications. The role of the stakeholder is primarily to
provide information that can then be incorporated into the project and
programme. However, projects and programmes based on participatory
approaches involve the stakeholders in fleshing out the design. Through
workshops and focus groups, stakeholders participate in preparing the
logical framework analysis (LFA) (an example of an LFA is contained in
annex II) and the overall framework for the execution of the project and
programme.

Project and Programme Appraisal

An appraisal, which is usually completed once a project and programme
have been designed, can be performed by both the stakeholders in the field
and UNDP staff.
Project and Programme Implementation

A project or programme with a high degree of participation will customarily
involve stakeholders in its implementation. Participation may mean
involving them in decision-making about such issues as the planning and
organization of activities, the use of resources, and the delegation of roles
and responsibilities.
Project or Programme Performance Evaluation Report

The PPER, an instrument for monitoring progress, is usually prepared by
project or programme management once a year, three months before a
tripartite review. It records the activities and progress made in the production
of outputs, describes the activities and results to be produced over the next
12 months, and includes any recommendations for action. Under a more
participatory approach, the PPER should be part of an annual review
undertaken by stakeholders. Ongoing and periodic reviews should be built
into the project from the start. The PPER, which indicates differences
between the original project design and what the project has produced,
should mirror the findings and recommendations of the stakeholders.
Tripartite Review
The TPR is a formal, planned, periodic monitoring mechanism for joint
decisions and implementation which is undertaken at least once a year. The
UNDP country office customarily invites all those concerned to participate
in the TPR and sets the agenda for the issues to be discussed. Under a more
participatory approach, a TPR should be undertaken at the project site and
involve stakeholders in joint decision-making and implementation. Project or
programme stakeholders could play a greater role in the meeting, including
its organization and setting of the agenda. Invitations to attend could be
extended to representatives of NGOs, parastatals and the private sector.
In-depth Evaluation
In-depth evaluations may take place during a project or programme
(ongoing), towards or at its end (terminal) or at least two years after it has
been completed (ex-post). An experienced participatory evaluation facilitator

(see Practical Considerations) will, of course, facilitate the process, but

stakeholders should be involved in defining the aims of the evaluation and
the assessment indicators as well as in analysing the findings.

Who are the question-makers? - Part Four
HOW TO DO A PARTICIPATORY
EVALUATION
For some time now, participatory evaluation practitioners have travelled various
highways and byways in search of the best route to useful participatory evaluations.
However, no single map, no universal set of directions or procedures has emerged to
guide the travellers to that destination.
Effective participatory evaluation practitioners remain undaunted by this situation,

recognizing that participatory evaluations are context-specific, rooted in the concerns,
interests and problems of programme end-users. They know that the complex process of
situating an evaluation in the end-users' immediate reality is what charts the route and
determines the evaluation's purpose and direction. Flexibility is their watchword.
Depending on the particular context in which the participatory evaluation approach is

applied, choices must be made about the degree to which end-users can realistically
participate in the process. The levels will vary, as illustrated in the table on page 14.
Using these levels as benchmarks for an evaluation can help to make it more
participatory.
Framework
The following participatory evaluation framework can be incorporated into UNDP
programming arrangements by programme staff and their collaborators, including
government offices, NGOs and community members. It consists of four basic phases:
1. Pre-planning and preparation

2. Generating evaluation questions
3. Data-gathering and analysis
4. Reflection and action
Pre-planning and Preparation

"In participatory evaluation, it is important not to belabor the issue of whether or not to
try participatory evaluation. Resist waiting for the perfect time. Just take a stab at it! If
you wait for the perfect conditions, you may never do it. All you can do is try and by
trying you will learn from failure: Think small but find a problem that may serve five
people in the world. Then celebrate for having tried to make the investigation more
inclusive of people's voices."
Rajesh Tandon, Participatory Evaluation Conference, University of Massachusetts, Amherst,

Massachusetts, USA, 1985
Typically, the early stage of a participatory evaluation is managed at the institutional

level far from the day-to-day lives of end-users. It should lay the foundation for a
participatory evaluation that is rooted in stakeholder interests.
● In order to establish stakeholder interest in conducting a participatory evaluation,

garner broad-based support by soliciting end-user input and collaboration.
● Since participatory evaluation strives for transparency, openly discuss the purpose,
goals and objectives and the various supporting or competing agendas of the
evaluation. Continually ask the basic question: "Who wants to know what for
what purpose?"
● When stakeholder input is difficult to solicit, review secondary sources, such as
programme documents, to gain an understanding of the context. Refer to
qualitative and quantitative baseline data and consult with stakeholders either
directly or indirectly.
● Address logistical matters, such as drafting and negotiating the TORs, identifying
participatory evaluation facilitators and stakeholders, and other administrative
matters.
Generating Evaluation Questions

When asked what questions she intended to investigate once in the field, the participatory
evaluation facilitator responded with a blank stare. "I have no recipe, no yellow brick
road to follow. In participatory evaluation, I rarely know what questions will be asked;
that is up to end-users. With them, we survey their situation and share our respective
fears, anxieties, hopes and dreams for the participatory evaluation. Then we form
questions worth asking."
J. Campos, UNIFEM Field Notes, La Paz, Bolivia
● Discuss and decide with end-users which data collection methods have a high
probability of yielding data that are useful and relevant to both outsiders and
insiders.
● Assess the current research skills of the persons involved in the participatory
evaluation and provide training as needed.
● Determine whether or not different methods will be needed for collecting various
types of data. Consider a mix of data-gathering techniques.
● Take into account the prevailing socio-cultural and political climate. Political
instability or geographic distances, for instance, can have a bearing on the
logistics of a participatory evaluation effort as well as on which data collection
methods are feasible. Sensitivity to the socio-cultural milieu, indigenous language
issues, gender issues and cultural diversity, especially regarding cultural minority
groups, is key.
● Negotiate evaluation questions with stakeholders. Workshops for large groups or
field visits to the end-users' workplace or home are appropriate venues for contact.
● Negotiate data collection techniques and provide training as needed. In this phase,
the participatory evaluation facilitator or participatory evaluation team works

shoulder-to-shoulder with key actors.
Data-gathering and Analysis

"It is better to have less perfect but more usable data and data that can more easily be
shared than to have a massive amount of data that becomes the private and often
confidential possession of a few expert-specialists."
(Maria-Thérése Feuerstein, 1988)
● Design appropriate venues for meeting with end-users and working with them in a
participatory manner. Workshops for large groups in which a cross-section of
representative end-user groups can work together are effective for dealing with
evaluations that are multi-level or multi-faceted. For more face-to-face contact,
field visits can satisfy the requirement of gathering data from individual sources or
from small groups working as focus groups.
● Provide thorough instruction or training for participatory evaluation team
members who work as data gatherers. Triangulation, a simple research tool in
which evaluation members confer frequently and regularly to cross-check, verify
and validate the process and data, is an effective strategy to use during the data-
gathering phase.
Reflection and Action

By entering another reality it is absolutely necessary for he or she who is going to in
some way perform a very difficult exercise, an almost impossible exercise, that is to
'deknowledge-ize' ourselves. This means to forget the knowledge which we had before
and to begin again. But now this time inside of the new reality or cultural frame of
reference. This way the people can have more power than we do. Always, the best rule is
to know that we do not know that new situation as do the people who live it. It is through
our disempowerment that they are empowered a little.
Paulo Freire, 1985
The final phase of a participatory evaluation is characterized by the creation of solutions

to end-user problems.
● Have the participatory evaluation group begin with the problems or evaluation
questions that were originally defined and articulated by end-users. The goals of
this activity are:
●
● to validate end-user experience by using it as the basis for future action plans
rather than using outsider's plans;
● to motivate end-users to find solutions and act on them rather than avoid them;
and
● to promote a sense of self-determination and sustainability through feelings of

empowerment.
Suggested Steps to Make Each Phase of an Evaluation Participatory
Pre-planning and Preparation
● Outline a conceptual framework based on participatory evaluation principles.

● Define parameters for the participatory evaluation (i.e., what can and cannot be
achieved).
● Assess constraints and resources or enabling and inhibiting factors.
● Identify the participatory evaluation facilitator, team members and stakeholders.
● Negotiate the purpose and objectives of the participatory evaluation with the key
actors.
Generating Evaluation Questions
● Facilitate participatory workshops in, or field visits to, stakeholder workplace or

residence.
● Collectively identify the main focus of the evaluation.
Data-gathering and Analysis
● Provide necessary training in data-gathering methods.

● Gather data collectively.
● Analyse data collectively.
Reflection and Action
● Prioritize problems to be solved or questions to be answered.

● Coordinate resources for resolving problems identified during the evaluation.
● Take collective action.
Practical Considerations
Resident Representatives, Deputy Resident Representatives, National Programme
Officers and JPOs may be confronted with numerous questions prior to funding a
participatory evaluation or in determining whether or not a participatory evaluation is
appropriate.
For Which Type of Project/Programme is A Participatory Evaluation Relevant?

Projects/programmes that have a clearly identified group of end-users and beneficiaries
lend themselves to experimentation with this methodology.
When Should a Participatory Evaluation Be Done?

Participatory evaluations may take place during the course of a project (usually at its mid-
point), towards or at the end, or a significant amount of time (e.g., two years) after a
project has been completed.
Undertaking an evaluation at mid-point offers several advantages. It represents an

opportunity to take stock of a project's progress to date, its achievements and any
obstacles encountered. Lessons learned can be applied and corrective action can be taken
if necessary. Since mid-term evaluations are forward-looking, they can provide
stakeholders with the tools to take different courses of action.
Terminal evaluations, which usually look at the longer-term impact of a project or

programme, tend to be done towards the end of a project whereas ex-post evaluations are
undertaken at least two years after a project has been completed. Since all of the
stakeholders may not be involved or associated with the project or programme after its
completion, the level of their participation in ex-post evaluations may vary considerably.
How Long Does a Participatory Evaluation Take?

The duration will vary with each project, depending on its complexity and the capacity
and availability of stakeholders to participate in all aspects of the evaluation. It may be
more important to view the evaluation as an activity done in phases rather than in one
block of time. For example, the following phases or combination of phases could be part
of the evaluation: defining the parameters of the evaluation; planning; collection of data;
and analysis of data and recommendations. Usually about 10 person-weeks of consultant
services (facilitator(s) should be calculated for a period lasting from three to six months.
Sufficient time should be allowed for pre-planning and setting the stage in the field with
the various stakeholders. The commitment of all stakeholders is critical to a participatory
evaluation and adequate time must be allocated to develop relationships of trust and an
understanding of what participatory processes entail. This could be done in two phases.
Phase one could include setting the stage and defining, with the stakeholders, the
parameters of the evaluation. The second phase could involve the analysis of findings and
proposed changes.
How Much Will a Participatory Evaluation Cost?

As might be expected, evaluation costs will vary depending on the activity, e.g., the level
of expertise that exists within the project (if local experts are unavailable, it will be
necessary to seek external consultants), the amount of baseline data already collected, the
availability of stakeholders to participate in the evaluation process. The participatory
evaluation facilitation team may be composed of national and international consultants.
About ten weeks of consultant services and travel should be estimated. It may be
necessary to factor in more than one trip for the facilitator if the evaluation is to take
place over an extended period.
In general, evaluation costs may include the following:

● consultant services for one or more participatory evaluation facilitators

● travel and per diem costs
● communication costs
● consultant services-nationals
● travel and per diem expenses of Government representatives where the need arises
● other costs related to data-gathering
● translation costs for the final report in the local language.
What Are the Role and Key Skills of a Participatory Evaluation Facilitator?
Hiring (done by UNDP) a knowledgeable participatory evaluation facilitator is critical to
a successful participatory evaluation. Not all evaluators know about or are trained in
participatory methods and techniques.
Without a careful screening of candidates, a participatory evaluation runs the risk of

becoming conventional, with limited stakeholder participation.
Participatory evaluation facilitators usually have an academic background in the social

sciences and typically are social science researchers or development practitioners. They
should have field experience, experience as educators of adults or as informal trainers and
have a reasonable grasp of qualitative methods, such as participatory rural appraisal
(PRA) and group dynamics techniques. They must also have the capacity to:
● listen
● guide and facilitate discussions, helping the group to ask key questions
● encourage trust
● delegate tasks and responsibilities
● plan actions to help bring together the viewpoints of various stakeholders
● create an environment of sharing and reflection.
The facilitator must act as a catalyst or stimulator, managing the evaluation without being
seen as directing it.

What If the Project Was Not Designed with Participation in Mind?

Ideally, participation should be part of the project design from the beginning and embody

the spirit of collaboration and interaction between and among different stakeholders
throughout the course of project or programme execution. However, not all projects or
programmes have been designed with participation in mind. Nevertheless, it is never too
late to build in greater consultation with all stakeholders through field visits and
monitoring trips or during TPRs. Consulting before taking decisions is an important step
towards achieving greater participation in decision-making, in implementation and in the
benefits of development actions. The level and degree of stakeholder participation will
depend on a number of factors:
● context of the project

● degree of willingness and commitment on the part of all stakeholders to
participate in a participatory evaluation exercise
● availability of baseline data
● availability of time and resources to enable stakeholders to collect data
● any external constraints that may impede stakeholder participation.
If the evaluation process is to be meaningful, then at the very least, stakeholders should
participate in defining the parameters of the evaluation, analysing the findings and
proposing solutions. Their involvement in the collection and analysis of data may depend
primarily on the availability of time and resources. Ideally, the evaluation report should
reflect the findings, concerns and recommendations of stakeholders. Where more
conventional evaluations are carried out, external evaluators should share their findings
with the stakeholders to ensure that recommendations are appropriate.
What Is the Role of the Resident Representative, Deputy Resident Representative,

National Programme Officer and Junior Professional Officer?
Participatory evaluations challenge conventional UNDP and technical cooperation
practices. Where Resident Representatives or government ministries or institutions
anticipate that an external evaluator will do everything, there may be some initial
skepticism about embracing participatory methods and practices. As one of the key
stakeholders in the MONEY AND MAMBAS case study remarked:
"I must confess that I was very skeptical about this participatory evaluation thing at first.
But I have been surprised at how well it has worked, and I can see that my staff have
benefitted. Their relations with the community will be better. We all have a better
understanding of the project as well as other issues concerning water in the
communities."
Resident Representatives can play a key role by being supportive of participatory

evaluation processes in the face of such skepticism. Resident Representatives or Deputy
Resident Representatives will also play an important role in determining whether or not
resources will be allocated for a mid-term review or evaluation.
The National Programme Officer or Junior Professional Officer (JPO) plays an important

role in assessing the levels of understanding and commitment that exist concerning
participatory evaluation and what such an evaluation would entail in terms of stakeholder
participation. Often they will be responsible for defining the initial TOR with the
participation of stakeholders. Part of the job of National Programme Officer, JPO,
Resident Representative and Deputy Resident Representative will be to promote a better
understanding of the benefits of participatory approaches, such as capacity-building,
greater ownership of results and more effective programming.
How Should the Terms of Reference (TOR) Be Prepared?

In conventional evaluations, it is customary for UNDP to detail quite extensively the
parameters of the evaluation and the issues that the external evaluator will examine.
For participatory evaluations, it is critical that the TOR provide as much flexibility as
possible.
Since the stakeholders play a key role in defining the parameters of the evaluation, the
TOR should help to launch the evaluation process without any second guessing of the
issues that the stakeholders will place on the table. Thus, consideration should be given to
holding a number of workshops with the key stakeholders:
● a planning workshop where stakeholders can define the parameters of the

evaluation
● a smaller workshop for data collection
● another workshop for the analysis of data and feedback.
The TOR for the evaluation should address issues such as:
● project context
● purpose of the evaluation
● proposed methodology
● preliminary identification of the stakeholders
● an evaluation strategy that outlines the various of the evaluation, such as a
planning workshop, data collection, analysis of data, and a feedback session
● duration of the participatory evaluation exercise as a whole and estimates of the
time required for each phase
● composition of the evaluation team and specification of the expertise required
● responsibilities of the consultants
● resources required.
The initial TOR should be prepared with the participation of key project or programme
stakeholders. The specific issues that will be examined by the stakeholders, however,
should be left for the first phase of the evaluation exercise once a participatory evaluation
facilitator has been identified.

Possible Reactions to the Participatory Evaluation Process

It's so confusing! Some participatory evaluation efforts can appear unorganized, feel
chaotic or take meandering routes in achieving their objectives. However, a participatory
evaluation is an emergent process. It takes its lead from the concerns, issues and
problems that appear during discussion, dialogue and interaction with participatory
evaluation partners. Confusion and/or ambiguity occur early in the evaluation and must
be anticipated since such an evaluation involves collaboration and negotiation to arrive at
mutual agreements between and among persons who probably have never met or worked
together before.
It's out of control ! Ambiguity in the participatory evaluation approach is further

heightened when the outside participatory evaluation facilitator views the participatory
evaluation effort as one that is "out of her/his control". In one sense, this is ideal since
one of the primary goals of a participatory evaluation is to share control of the process in
order to engender ownership. The participatory evaluation process readily becomes clear
once outsiders become conversant with the life struggles, understandings and sensibilities
of those who have been voiceless and excluded from the development process. To give
others control, be "out of control"!
But is this really learning? At its most fundamental level, the participatory evaluation
approach is investigative and educational. It is about systematically involving the least
powerful, least visible and least assertive actors in evaluating development efforts
devised on their behalf. Drawing on their experiences and views exposes the true nature
of the problems as they see them and engages them in a process of creating and
articulating viable solutions to those problems.
The evaluator assumes the role of facilitator, who works to promote learning moments.
The approach supports end-users in their efforts to confront, analyse and find solutions.
Involving end-users means patiently waiting for their questions to emerge, but the wait
can reap long-lasting benefits. Tapping their knowledge and using it to determine future
plans can promote sustainability. Such evaluations can be empowering for both the end-
users and the evaluation facilitators.
The poignant words of a Mayan woman from Guatemala who was involved in a United
Nations-sponsored participatory evaluation demonstrate the value of participation:
"I didn' t think my simple words would be important. I didn't think your big bosses would
accept them. But I know that without my words you would not understand our reality.
You have made me think that what I think in my head is worth the time. You have made
me remember what I had forgotten to think about."

Who are the question-makers? - Part Five
TRAINING MODULE
INTRODUCTION
The case study described in this training module is based on UNDP's effort to
conduct a participatory evaluation in an actual country setting. Participatory
evaluation, an innovative approach to evaluation, can contribute to
sustainable development by involving key stakeholders in assessing
programmes and projects from their perspective. Involving stakeholders at all
levels of evaluation can lead to more comprehensive assessments of
development and can effectively draw upon beneficiary views and opinions
to redirect development planning.
Purpose
The module is designed to:
● help participants become knowledgeable about and conversant with

participatory evaluation as a beneficiary-centred evaluation approach;
● provide participants with a forum for discussing the strengths and
weaknesses of an actual participatory evaluation effort and how
participatory evaluation can complement conventional evaluation
practice;
● provide participants with a forum in which to discuss basic
participatory evaluation tools and the requirements for applying a
participatory evaluation approach.
Audience
This training module targets UNDP personnel, especially Assistant Resident
Representatives, junior-level personnel and programme officers. Other
government or non-governmental groups or agencies involved in UNDP
programme and project planning, implementation, and monitoring and
evaluation are also included.
Format
The case study format used in MONEY AND MAMBAS actively involves
trainees in discussions and analysis of participatory evaluation theory and
practice. Working through the case study helps trainees to understand events
that supported and worked against the participatory evaluation exercise. All
factual details and names of individuals involved in the case study have been

changed to avoid distraction from the learning objectives of the training

module.
Guide to the Facilitator

Pre-workshop Activities
● It is recommended that the facilitator become familiar with the details

of MONEY AND MAMBAS. Further information is available from
OESP, DC-1, 21st floor, One UN Plaza, New York, NY 10017, USA.
● Photocopy and distribute the case study on pages 37-45. Advise
participants to read the case study prior to the training session. While
it is possible to have workshop participants read the case study as part
of small group discussions during the session, their discussions,
analyses and recommendations can be better informed by prior
reading.
Workshop Activities
The module, which has been designed to help groups to analyse the pros and
cons of participatory evaluation during a single training session, is organized
for a three-to-four-hour time frame but can be adapted to meet specific needs.
Below is a suggested schedule with approximate times for each activity.
Introduction (30 - 45 minutes):
● Welcome and introduction by facilitator.

● Facilitator provides a general description of the case study and
explains that small group discussions will address questions related to
the various parts of the participatory evaluation case study.
Small groups (30 - 45 minutes):
● Facilitator divides the group into small groups-no more than 5 persons
per group.
● Facilitator instructs groups to discuss questions related to parts 1 and 2
of the case study ("Questions for Group Work" are included following
the text of the case study).

Break (12 minutes):
● Short break if needed.
Plenary (45 minutes):
● In plenary, groups report their responses to questions relating to parts

1 and 2.
Small groups (30 - 45 minutes):
● Facilitator instructs groups to discuss questions pertaining to part 3.
Plenary (30 - 45 minutes):
● Groups report their responses to the questions pertaining to part 3.
● Facilitator directs the groups' attention to the "Additional Questions".
● Participants discuss how they would design and implement a

Money and Mambas1 or Listening to the

People
A case study of a participatory evaluation of a water and sanitation
project
This case study is based on an actual participatory evaluation effort by UNDP
in Eland. It is presented in three main parts:
1. Pre-planning and preparation

2. Getting started: generating evaluation questions
3. Data-gathering, data analysis and action.

PART 1.
PREPLANNING AND PREPARATION

In the late 1980s, UNDP decided to conduct a number of evaluations with
greater participation by local community and stakeholder groups. After an
extensive review of possible projects for evaluation, the former UNDP
Central Evaluation Office-now reconfigured as the Office of Evaluation and
Strategic Planning (OESP)-settled on a water and sanitation project in Eland.
The project chosen was called "Rural Water Supply and Sanitation". Its
purpose was to supply water to homesteads in rural areas, to provide training
in health and hygiene and to construct latrines.
The project appeared to have the right characteristics: during its design, there
had been an effort to involve the people and when it was submitted to the
programme approval committee in New York, the presenters had emphasized
that it was aimed at the poor in rural areas and that it was built on community
participation. Project work had started in 1990 and was scheduled for
completion at the end of 1995. In addition, an evaluation was planned for mid-
1994. Hence, the timing and subject appeared to be right.

In January, 1994, OESP in New York approached the Resident

Representative of the UNDP office in Eland to ask if a participatory
evaluation approach could be used. If so, OESP would pay the full costs of
the evaluation from its funds instead of using the funds allocated on line 16
of the project budget. The Resident Representative consulted the authorities
in Eland and their positive response made a participatory evaluation
agreeable to all partners.
PART 2.
GETTING STARTED: GENERATING EVALUATION QUESTIONS

OESP initiates a participatory evaluation of UNDP-Eland's community-based
water and sanitation project. All start-up procedures begin. Preparatory work
includes a desk review by Claus and Karl of all project documents and
telephone conferences between Claus and Karl and OESP in New York.
Contractual agreements with the two international consultants are confirmed
by OESP. Additional members of the participatory evaluation facilitation
team are recruited (Lane and Didi), with the inclusion of a female sociologist
(Didi) almost an afterthought.
Standard UNDP procedures require clearly defined terms of reference (TOR),

including precise definition of evaluation questions and data-gathering
methods. In a participatory evaluation, pre-determined evaluation questions
are often considered irrelevant to the immediate needs and concerns of
programme beneficiaries. Participatory evaluation asks one important,
fundamental question: Whose questions are being asked in a particular
evaluation? Therefore, individuals attempting a participatory evaluation are
challenged starting with the TOR. In MONEY AND MAMBAS, drafting the
TOR presents the first major obstacle.The TOR dilemma quickly surfaced:
Bruce North's thoughts:

Headquarters has come up with this experimental project. I've been briefed
on this by Mendisa, our National Programme Officer. Our project is
somewhat on track. Our goals of supplying water to rural home steads,
providing health and hygiene training, and constructing latrines have been
met. The Rural Water and Sanitation Department has put in pipes and
filtration systems to community standpipes. Our expansion to 22,000 water
users is within reach, but one disturbing trend remains: pit latrine
construction (critical to sound water and sanitation) is far below target

because villagers refuse to build latrines according to government

specifications. Their resistance jeopardizes the project. The original project
design was a package in which the provision of water supply systems linked
to rural sanitation through ventilated pit latrines is conditional upon
community construction of ventilated pit latrines. We had villager buy-in at
the start, but particip
One snag has already emerged. The TOR seem to be a problem. It seems that
Claus and Karl (alleged participatory evaluation "specialists") insist that
stakeholders determine the questions to be asked in a participatory
evaluation. A novel and noble idea-but too time-consuming. I wonder about
this participation fetish. Haven't we hired them to do the evaluation?
Claus and Karl in Sweden:

Our first major obstacle came with the TOR. In participatory evaluation, pre-
determined questions are typically irrelevant to stakeholders; they are
questions "outside" of project beneficiaries' immediate needs. Who knows
better than water users what works or doesn't? TORs for participatory
evaluation should include end-users as question-makers, not just question-
answerers. In participatory evaluation there is always "the chicken or the
egg" dilemma: whose questions are more important, ours or theirs?
UNDP-Eland agrees with consultants that a question-generating process

should be part of the TOR.
Because so much of a participatory evaluation is context-specific and cannot

be pre-determined, participatory evaluation facilitators must bring a jack-of-
all-trades tool kit with them when they arrive. Claus, the first consultant to
arrive in Eland, is an experienced participatory evaluator but unaware of the
surprises awaiting him.
March 1994. Claus's introductory meeting with the UNDP Resident

Representative and programme officer was cordial yet distant. The tone of
the meeting seemed non-committal and the meeting ended abruptly. His
introductory meeting with a key Eland actor, Richard Kosi, was even more
perplexing. Claus recorded the following thoughts:
Claus's thoughts:
I am concerned, to say the least. UNDP does not seem to really know much

about the participatory evaluation nor do they seem interested. Frankly, the
apparent lack of genuine buy-in expressed so far tells me they are not
interested because this "was not their baby".
My meeting with Richard revealed that no one has been invited to Workshop
I, which begins in three days. Richard seems to have an exclusive invitation
list which includes certain beneficiaries and excludes others. He has
excluded his mid-level and field staff. What will it take to convince Richard
that all stakeholders need to be represented? Four months in the planning
and I feel like I'm talking to myself. I wonder if anybody in the country office
or the Rural Water and Sanitation Department took the time to read the two-
page briefing note on participatory evaluation and the TOR detailing the
evaluation exercise that were sent ahead of my arrival. I have strong doubts.
Richard's thoughts:
This participation idea seems good. The way I see it, these Claus and Karl
fellows are participatory evaluation experts being paid to carry out the entire
evaluation. They are responsible for all the main aspects and for the final
recommendations and writing up the report. My staff participates by helping.
Claus wants all stakeholders to be involved from the start. How unrealistic
this man is. Surely, only trained evaluators should collect the data. Don't
these fellows get paid enough to handle everything?
Claus's thoughts:
As if all of these setbacks are not enough, the Resident Representative
informed me that neither he nor his deputy will attend Workshop I, but he
will send his programme officer. UNDP's noncommittal stance and Richard's
misunderstanding of participatory evaluation put the effort into question.
Most disturbing are the differences Richard and I have regarding stakeholder
involvement. How could Karl and I have so wrongly assumed that everyone
would jump on our bandwagon? Maybe we should have spent more time
sensitizing people to the value of participatory evaluation?
Despite the initial confusion, Workshop 1 started promptly a few days after
Claus's arrival. The usual official welcomes and introductions were led by
Richard Kosi. The 24 participants in attendance included 16 women and 1
Community Water Committee leader, representing programme beneficiaries;
one Ministry of Health representative; one UNDP representative; three
participatory evaluation facilitation team members; and the senior engineer

of the Rural Water and Sanitation Department and his deputy. The group
focused quickly on the purpose of the evaluation and generated the first
important list of evaluation concerns.
Examples of Problems and Questions Raised by the Stakeholders
● What should be done when handpumps do not function properly?

● What should be done with government workers (e.g., soldiers and
teachers) who use the water but do not pay?
● What should be done with the people who allow their cattle to drink at
the community water taps?
● The elders must be advised not to send young children to fetch water
to avoid their contaminating clean water with dirty hands when they
collect it.
● What should be done if the community wants development and
advancement but the rural health motivators do not meet?
● Chiefs do not understand their role in participation in the project.
● Women's concerns are not taken seriously by community leaders.
● Women are the promoters of development in communities and
homesteads. It would be better if they were represented at the Chief's
inner, highly contentious committee.
● Imported foreign ideas can be a problem (with particular reference to
pit latrines in this case).
● Dirty containers are used to transport water from a clean source.
The issues brought the participatory evaluation into sharp focus from the
beneficiaries' point of view. The women's group was particularly animated
and articulate. They more than anyone else grasped the participatory
evaluation idea. The Rural Water and Sanitation Department's Senior Water
Engineer and his staff did not participate in the lively discussions of the small
groups. They did an excellent job of holding themselves aloof from the
exchange of views.
Claus's thoughts:
Given the aloofness of the Rural Water and Sanitation Department chief
officer and staffers, it appears that the process is either of no interest to them
or it may appear threatening. Could I have been wrong about everyone's buy-
in for the participatory evaluation? Do they fear that it may probe sensitive
management issues that could put them in a bad light? Why aren't water

officials interacting with the actual water users-the women?
Claus, Lane and Didi led the group through a discussion about important
participatory evaluation steps, including data-gathering techniques,
recording data and basic data analysis. At the day's end, Richard mentioned
another problem. He announced that tomorrow's workshop session must end
early to accommodate participants residing in the countryside. The rushed
schedule, although a considerate logistical decision, could potentially
sabotage the participatory goal of stakeholder involvement in planning the
Claus, at the end of Day 1:

Day 1 has finally ended. My thoughts are scattered. The first few days were
full of problems, but this is to be expected. The skepticism shown by the Rural
Water and Sanitation Department officials doesn't help. There is a spirit
lacking here... a core something missing. Many of Kosi's staff, some of the
primary project stakeholders, were absent or were assigned to field duty by
him. So much for stakeholder participation. It seems like a half-hearted
effort. With Karl in Sweden and Didi not expected until tomorrow, even our
facilitation team worked at half strength. What on earth might we expect for
the following phases? Only time will tell.
Surprisingly, Day 2 progressed smoothly and quickly. The discussion in small

groups generated constructive recommendations regarding data-gathering
methods. Questionnaires and a work plan were developed based on questions
that stakeholders had identified regarding issues to be addressed by the
evaluation. The group decided to tailor questionnaires to address the
concerns of particular groups and to distribute them to women homesteaders,
Water Committee officers, community water-minders, and Rural Water and
Sanitation Department field staff, all of whom would be responsible for
gathering the data. Because of time constraints, Claus ended up having to
prepare the questionnaire.
Claus to himself:
A new understanding and interest in participatory evaluation seemed to
emerge today. Almost everyone seemed more comfortable with the concept
and with each other. Perhaps the evaluation sketch that Lane and I enacted
yesterday contributed to this new attitude. Everyone had the chance to
comment on what should be evaluated. There is less skepticism and more

enthusiasm in the air. Tomorrow, I leave Eland. Karl and I will return four
weeks after data have been collected. It is all in their hands now.
PART 3.
DATA-GATHERING, DATA ANALYSIS AND ACTION

April-May 1994.
Four weeks elapsed between Workshops 1 and 2.
In the interim, Lane and Didi conducted the data-gathering phase. All data
were collected by Lane and Didi, who did not include other stakeholders
although their inclusion had been agreed upon in Workshop 1. It seemed that
villagers who represented communities in Workshop 1 and drew up the list of
questions reverted to the traditional role of question-answerers, not question-
makers, during the data-gathering phase instead of collecting information
and asking questions that had been identified during the planning workshop.
When Karl and Claus returned to Eland to review and analyse data, they
learned that only 22 water schemes had been visited, which meant that only
200 beneficiaries had been interviewed. This figure was well below what had
been planned. Group interviews, and not the individual interviews that had
been planned originally, had been conducted.
Karl to himself:
When I first saw the data, I strongly doubted its quality: not enough water
schemes had been included; many interview reports lacked valuable
responses; and the group format may have compromised the reliability. It
was evident that Lane and Didi did not have sufficient background in
working with empirical data and, worse, Claus and I had failed to give them
sufficient long-distance support. Do we go ahead anyway?
The four-week separation had also promoted another unexpected event.

Richard unilaterally decided to reconfigure the workshop participant list. He
invited more senior-level policy-makers, including Government Ministry
representatives, UNDP and Rural Water and Sanitation Department officials.
Principal secretaries of the Ministries of Finance and Economic Planning,
Agriculture, Labour and Health were the new faces that replaced all but four
of the 16 rural village women water users. In addition, a few Chiefs or their
representatives had been invited.

Karl to Claus:
So much for participatory decision-making. The decisions that were
originally made and agreed upon by the group have been changed in our
absence. Is there some hidden agenda here that we are not part of? Whose
participatory evaluation is this anyway? Is this what is meant by "sharing
control" in participatory evaluation? Who wants to control what for what
purpose? What about the women? Will it be men who speak for the women
again?
Despite the changes, Workshop 2 progressed well: 30 people attended,

including four beneficiaries, and OESP's evaluation officer from New York
attended the final session. The group collectively reviewed the data and
quickly categorized the main findings into socio-cultural, technical and
organizational issues. The group review process allowed workshop
participants to agree with or argue the data. The presence of Chiefs
representing numerous traditional community groups was particularly
welcomed.
Participants sped through the final steps and organized plenary

presentations, which focused on problem analysis and recommendations for
future action. The following comments help to convey how some issues were
resolved by the group.
Rural Water and Sanitation Department and Ministry of Health staff:

Our group questioned why villagers refuse to follow our rules of building pit
latrines. Our designs are excellent and have worked in other places. What's
their problem?
End-users:
We object! We object to that conclusion! The Ministry of Health design is no
good for us! We want latrines with walls that allow for air flow and let in the
light. In the Ministry of Health design, the pit is far too deep and the walls
collapse because of imprecise site and soil specifications. Worst of all, the
latrines are dark and wet-perfect housing and nesting ground for the
venomous mamba snake! Let's see you share your latrine with a mamba!
Richard, Senior Rural Water and Sanitation Department Officer:

You think the mamba snake is the worst problem? Not so! The problem has
nothing to do with mamba snakes but has everything to do with not being
able to retain good staff. The Rural Water and Sanitation Department is not a
government ministry with an official budget. The Ministry of Finance
classifies us as an externally funded project dependent on donor offerings.
Our economic uncertainty undermines any long-range planning and does
little to boost staff morale. In short, we cannot attract or retain the best
technicians, much less technicians with a sensitivity to rural villagers. Now,
that is the problem.
Clearly, end-users saw the threat and fear of mamba snakes in the pit latrine
as the major problem while the Rural Water and Sanitation Department saw
money and personnel as the main issues. The discussion continued:
Traditional Chiefs:
All problems would be solved if the Rural Water and Sanitation Department
would work more closely with the traditional power structure through which
we exercise authority. All coordination and entry into community affairs
should go through us. This is strictly a problem of what happens when you
bypass traditional authority.
Various voices:
No, we reject that idea. We object! Yes, we do need closer coordination
between Rural Water and Sanitation Department field activities and the
community groups. Training would be a good place to begin, and maybe the
Chiefs should be the first to be trained. That's right. The Chiefs need
training!
Rural Water and Sanitation Department field staff:

Many Chiefs and their Councils do not understand modern development
needs. Chiefs need education, too, but they are difficult to persuade to come.
If they don't come when invited, they send representatives instead.
Chiefs:
Chiefs could be encouraged to attend training. Just offer incentives for our
time.
Various voices:
We protest, protest, protest...!

End-users:
Why do Chiefs need incentives? Isn't being a chief incentive enough? Why
should Chiefs be paid to take part in training that they need to do their job?
Others don't get any incentives. We don't get incentives, yet we give of our
limited time to attend Water Committee meetings and even these workshops.
Chiefs:
If the Rural Water and Sanitation Department wants to reach the Chiefs and
work with the community through us, it needs to learn to work through
channels it does not normally work through. We would be more likely to
come if we were invited to training via the King's offices, our traditional
power structure. The money is not the issue.
Workshop participants spoke frankly and openly and challenged each other
on important issues. The participatory evaluation facilitators had hoped that
the final session would result in conclusions and recommendations for future
action. This was not accomplished. Facilitators found that the exercise had
generated more data and information than was expected and the final session
did not allow for a thorough discussion of all issues. Yet four clear action
points were identified by the group: (a) the need for user designed latrine
construction; (b) improved community training for water committees; (c)
improved Rural Water and Sanitation Department-community coordination;
and (d) improved relations with traditional power structures.
After the workshop, plans for the facilitation team's remaining days in Eland
were made. The team produced a draft of the final report and circulated it to
the Rural Water and Sanitation Department and UNDP for initial comments.
Commitments were made by both offices to collaborate on the translation of
the report's summary and its circulation to the participants in both
workshops.
Most revealing to Karl and Claus were the lessons learned by their Eland
counterparts:
Richard to Karl and Claus:

I must confess that I was initially very skeptical about participatory
evaluation. I was surprised at how well it worked and I saw that my staff

benefited. I believe their interaction with the communities they serve will
improve. We all have a better understanding of the project and a new
appreciation for the value of soliciting stakeholder input.
Lane:
What a tremendously satisfying experience! The participatory evaluation
often seemed like a logistical nightmare, but it was a revelation to me. Even
after 15 years working on water issues, I was struck by the articulate
precision with which women end-users pinpointed critical water issues that I
had overlooked.
My favourite problem was the one asking that Water Committee members not
drink beer before their meetings.
Didi:
I was amazed at how much I learned about my country. I travelled to rural
areas to see women in their natural surroundings. I was most amazed to find
that the subject of water provided the entry point for discussion of community
problems. Despite the Rural Water and Sanitation Department's many
accomplishments, the crucial problem was simple: water containers had too
small an opening, which precluded cleaning and encouraged algae growth.
We learned that sanitation problems need to focus on water collectors. While
visiting the homesteads, I often saw that kids, the chief water collectors, had
the dirtiest little hands!
The eight-month participatory evaluation process had finally ended. Long

after the final days, questions lingered. Had the process been effective? What
would its long-term impact be? Could the costs be justified? Most important,
what would happen in the coming months in Eland?
Money and Mambas or Listening to the

People
Questions for Group Work
A. Questions about Parts I and 2: Stakeholder Issues
1. Identify the major stakeholders.

2. What steps were taken to include or exclude various stakeholders?

3. What principal roles did they play?
4. What conclusions can you draw about the stakeholders' roles as
question-makers?
PREPARATION ISSUES
1. Analyse the preparatory work for the participatory evaluation.

a. What right and wrong actions were taken?
b. What would you have done differently?
c. What difference(s) might your decisions have made?
d. Discuss the TOR "snag".
B. Questions about Part 3: Data-Gathering Issues
1. Discuss and analyse the strengths and weaknesses of the data-

gathering phase.
2. Discuss the final phase:
a. reporting and writing evaluation findings
b. content and language of the final report
c. use of findings.
PROCESS ISSUES
1. Discuss the overall management of the participatory evaluation. What

can you say about the various elements, including:
a. TOR
b. workshop timing
c. selection of participants
d. scheduling of international consultants
e. group dynamics
f. overall organization?
ACTION ISSUES
1. In your view, what actions might the UNDP country office take after
its involvement in the participatory evaluation?
2. What can be surmised about future actions of the various groups

represented at the workshops, including

3. What possible inter/intra-agency changes might result?
C. Additional Questions: Application Issues
1. What is involved in the TOR for a participatory evaluation?

2. Describe the type of consultant you would hire for a participatoryn
evaluation.
3. How would you ensure that stakeholders are effectively involved in
4. What support training might be needed to undertake a participatory
evaluation?
5. The case study presents a non-traditional approach to evaluation,
which was designed to complement, not replace, traditional
evaluations. How might a similar effort be useful to your country
office?
6. The evaluation described in MONEY AND MAMBAS was not driven
externally by UNDP consultants but rather by stakeholders. How
might such an attempt work in your country context?
7. Contrast the case study with the UNDP basic Programme and
Projects Manual (PPM). How could participatory evaluations be
worked into the PPM cycle?

Who are the question-makers? - Annex 1
Annex I.
Glossary
Baseline data a set of conditions existing at the outset of a programme or

project. Results can be measured or assessed against such data. In
participatory development, it is important that stakeholders participate in
identifying the key sources of information and the indicators required for
measuring performance.
Beneficiary assessment an approach to information-gathering that places the

emphasis on the perceptions of the principal actors. Qualitative research
methods are stressed, including direct and unobtrusive observation methods
and interviewing, including semi-structured and open-conversation
interviews.
Case-study analysis a learning tool used in training or group settings to

stimulate the processes of dialogue, discussion and analysis. Typically based
on real-life incidents, case studies illustrate the range of possible enabling
and inhibiting factors in a given development activity. Case-study group
discussions can increase understanding of broader issues involved in
programme and project development.
Donor the funding organization, whose role in participatory evaluation is to

mobilize resources which can increase beneficiary involvement in the
evaluation and ensure more relevant feedback for the purposes of long-term
planning. Donors in participatory evaluation exercises listen more and
dictate less.
Emergent knowledge data that emerge informally as part of an evaluation

encounter and not as a response to pre-designed evaluation questions.
Identifying emergent knowledge leads to inductive analysis, which is critical
in participatory evaluation. According to Michael Quinn Patton, an
evaluation specialist, inductive analysis means that the patterns, themes, and
categories of analysis emerge out of the data rather than being imposed prior
to data collection and analysis.
http://www.undp.org/eo/documents/whoa1.htm (1 of 3) [29/01/2002 19:02:24]

Evaluator in participatory evaluations, the stakeholders or programme end-

users and/or beneficiaries, who are involved in all stages of the evaluation
process, from defining the terms of reference and collecting and analysing
data to making recommendations and taking corrective action or making
improvements.
Facilitator an outside expert in participatory evaluation methods, who has the

capacity to listen; help the group to ask key questions; guide and facilitate
discussions; encourage trust; delegate tasks and responsibilities; create an
environment of sharing and reflection; and plan actions to help bring
together the viewpoints of the various stakeholders.
Focus group small discussion group that concentrates on a specific topic. A

group facilitator assists in focusing the discussion on strategies for defining
solutions to particular problems. Used in evaluation as a means of starting a
discussion, identifying needs and clarifying key points.
Participatory evaluation the collective examination and assessment of a

programme or project by the stakeholders and beneficiaries. Participatory
evaluations are reflective, action-oriented and seek to build capacity.
Qualitative methods methods that minimize the use of numerical analysis,

such as interviews, observation, testimonials and various PRA methods to
elicit information from stakeholders. Information obtained in this way can
help to illuminate data and numbers.
Random sampling a selection made without method or conscious choice.
Secondary sources sources such as periodic progress reports, annual reports,

memos, sectoral studies and baseline data. They serve as background and
foundation material and resources for an evaluation.
Stakeholder an actor that has a vested interest in a given project, activity, or

issue. Stakeholders may include groups affected by development actions,
such as the poor, women, workers, farmers or the community at large, as
well as other actors that can affect the outcome of a project, i.e., government
officials, institutions, project personnel or the local government. In
participatory evaluations, stakeholders assume an increased role in the

evaluation process as question-makers, evaluation planners, data gatherers

and problem solvers.
Stratified sampling a selection that ensures representation of a cross-section

of a community according to such characteristics as age, gender, social class
or race.
Triangulation a process of cross-checking and cross-validating by talking

with and referring to various sources.

Annex II.
A SAMPLING OF TOOLS FOR PARTICIPATORY EVALUATIONS
Over the years, many tools and manuals have emerged that have contributed to the growing recognition
of participatory evaluation as an evolution in, and a valuable alternative to, conventional evaluation
methods. While the stages in undertaking a participatory evaluation can be clearly delineated, there is no
one model, recipe or tool for such an evaluation. This flexibility has allowed the approach to grow and
evolve.
The tools used depend, of course, on the nature of the project, the context and the stakeholders. Since
each project setting is unique, different tools will be required, based on the particular political, cultural,
economic and social characteristics of the project. Such tools can be used independently or in
combination.
It should also be noted that the use of participatory evaluation methods should not preclude the use of
quantitative methods. In fact, depending on the initiative, it might be beneficial to combine various
methods and approaches. It is the responsibility of the experienced participatory evaluation facilitator to
determine the most appropriate tools, depending on the context and nature of the evaluation.
Some of the most common tools are described in this section. For further information on how to apply
them, a list of key manuals is also provided.
Beneficiary assessment
A beneficiary assessment involves the participation of beneficiaries in evaluating a planned or ongoing
development activity and builds on the experience of participant observation. Assessing the value of an
activity as it is perceived by its principal users, this tool seeks to provide a context for quantitative data
by letting beneficiaries' voices, values and beliefs be expressed. Methods include direct observation,
conversational interviews, and participant observation, which involves the protracted residence of an
outsider in a community for a period ranging from several weeks to two or three months. These methods
should be used by an experienced observer and inquirer.
Notes from a Participant Observer

After more than two years of existence, a fishing cooperative in the state of Rio Grande do Norte had
attracted only about 10% of the fishermen for whom it was intended. To attempt to determine why,
World Bank project officers and local management agreed to try out the participant observer evaluation
method (which has since become known as beneficiary assessment), using host-country observers.
Project officials selected a young man in his twenties who had recently received a university degree. He
was to live in two fishing communities for several weeks each and spend an additional few weeks in and
around the central cooperative in the state capital of Natal. After almost three months in these areas
living with and talking to fishermen, the participant observer made the following observations about the
role of local fish buyers ("intermediaries") in the project:
"The fishermen are less exploited if they deal with the cooperative, yet none of them are conscious of
this, owing to the anticooperative campaign carried out by the intermediaries (many of whom are also
fishermen). The majority of the active (cooperative) members are not individuals who are conscious of
the advantages of cooperativism but fishermen who do not get along well with the intermediaries. On the
other hand, many nonmembers give preference to the intermediaries in order to maintain ties of family
or friendship.
There exists misinformation about the actual prices (offered by the cooperative for fish) such that none

of the nonmember fishermen can say exactly what this price is. The notion that the fishermen have... is
that the price is lower than what the intermediary pays, whereas the price of the cooperative is 20%
higher than the price of intermediaries. This error is daily reinforced by the intermediaries. The
anticooperative mentality is such that the fishermen do not believe that the cooperative is paying a
higher price when they are informed that this is indeed the case".
As a result of the information from the participant observation, the local manager of this Bank-supported
fishing cooperative instituted a comprehensive new promotional campaign to educate the fishermen
about cooperativism, took steps to replace a cooperative administrator seen by the fishermen as cold and
uncommunicative, and redistributed profits to the fishermen, thus redressing one of their major
grievances.
Source: Lawrence Salmen, Listen to the People, 1987:93-94.
Focus group
A focus group brings together a representative group of 10 to 15 people, who are asked a series of
questions related to the task at hand. A facilitator guides discussion. Focus groups, which draw from
local experience and traditions and provide local insight, are useful in project design and in assessing the
impact of a project on a given set of stakeholders. While focus groups are commonplace among North
American advertising agencies, they are being used increasingly in the field to validate project designs or
help to assess project performance.
Seasonal calendar
This calendar uses visual representation to help to identify events that occur seasonally. It is useful for a
wide range of projects and activities (planting, harvesting, identifying seasonal health risks, seasonal
transmission of HIV). It can also highlight such constraints as drinking water availability, drainage
blocks, labour availability, food intake, etc. A seasonal calendar can be created on the ground using
stones, sticks or leaves or on paper.
Logical framework analysis (LFA)

The framework presents, in a nutshell, the core activities and outcomes of the project as well as the key
indicators for monitoring and measuring its results. Customarily, this exercise is undertaken by the
person responsible for the project design with some type of input from the field. However, it is possible
for stakeholders to be the authors of the LFA by holding special workshops. Once the objectives have
been agreed upon, stakeholders can work at defining the respective activities, results and performance
indicators. Not only do stakeholders provide valuable input into the project design, but they also have a
much better understanding of the project.
LFA is used by many donors as a planning tool to summarize the activities, results and verifiable
indicators related to each objective of a project. For project evaluations, the LFA will usually be a key
source of information referred to by the stakeholders since it contains a summary of the purpose of the
project and its expected results. The verifiable indicators will also help stakeholders to measure the
degree to which project results have been achieved. It is important to review the LFA with project
stakeholders to ensure that the indicators are still valid.
LOGICAL FRAMEWORK ANALYSIS
Specific Activities Results Verfiiable Indicators

Objectives

To promote the 1. The technical 1. The daycare 1. Manuals are

psycho social team revises the centers adopt prepared,
and physical child development childcare models distributed and
development of model from and manuals used by the
marginalized various per prepare on daycare centres.
childrentwo to spectives: psycho- technical and 2. 100 per cent of
seven years of motor, cognitive administrative the children are
age and aspects of monitored on a
communication childhood regular basis by
skills, nutrition development and the caregiver.
and health. daycare centres, 3. There are
2. Social workers, respectively improvements in
caregivers and 2. Training is the children's
parents are trained provided on a basis caloric
in childhood quarterly basis to intake.
development. stakeholders. 4. Per cent of
3. Caregivers 3. Caregivers children who
prepare balanced, prepare varied meet standard
nutritious meals. menus for psycho social
4. The growth and children. and physical
development of 4. 100 per cent of development for
children are the daycare their age group
monitored on a centres measure 5. Per cent of social
regular basis. and weigh the workers,
children on a caregivers and
regular basis. parents who
apply training in
childhood
development.
Reduce Two skills training Within three years, 100

employment cources: appliance repair men and women
and telephone installation workers will become 1. Training of the
qualified in two skill level required by
areas- appliance repair prospective
and telephone employers.
120 tranees installation- and will 2. Number of men
find jobs in these fields. and women
For the majority, these placed in jobs for
are expected to be new, which they were
pemanant jobs. The trained, within
training is expected to one year of
permit growth of these training, and
sectors. then three years
later.
3. Number of new

jobs created in
these sectors.
Participants of 1. Leadership 1. Within two 1. Quality of

marginalized training years, supported analysis of the
populations and 2. Assistance with organizations country's
graa-roots strategic planning seeking land political,
organizations in and action plans. reform in economic ans
political 3. Letter-writting country X are social
decisions that campaigns and able to anylise conditions.
affect them. other acts of current 2. Quality of action
solidarity when cordinates and plans.
supported develope action 3. Degree of
organizations are plans. They participation in
harasses. consult with like- consultations on
minded groups. land redorm.
2. Within three 4. Number of
years, supported lobbying
organizations activities and
take on advacacy responses. Media
roles. coverage.
3. Within five 5. Extent of
years, supported representation in
organizations are political
assemblies assemblies.
where
represented in
political
decisions are
taken.
Semi-structured interview
A semi-structured interview, which is less formal than a structured interview, allows for conversation
and the reciprocal transmission of information. Preparation usually involves outlining the broad areas of
inquiry, leaving specific questions to be formulated during the interview itself.
In her book, Participatory Program Evaluation, Judi Aubel highlights a series of questions that were part
of an interview guide for community health nurses in Gambia and Sierra Leone. As she points out, the
questions should be sequenced with the easier questions coming first and less intimate questions coming
before more personal ones. The questions are open-ended and seek to collect in-depth information on
people's attitudes, opinions and knowledge. This allows the interviewer the time to gain the confidence
of the person being interviewed. The questions should also be kept simple.

The following are some of the questions that were used in an interview guide for Community Health
Nurses:
1. What was your role in the Nutrition Education Pilot Campaign (NEPC)?
2. What was the role of the Mothers' Committee in the program?
3. To what extent did they assume that role?
4. Did you receive your fuel subsidy?
5. Was the fuel given adequate for carrying out your NEPC activities?
6. What was your role in monitoring the NEPC activities?
7. Were you trained on how to monitor the NEPC activities?
8. What information did you collect in the monitoring?
9. How frequently did you monitor the activities?
10. Did you encounter any obstacles in monitoring the activities?
11. What did you do with the information provided?
Source: Judi Aubel, Participatory Program Evaluation: A Manual for Involving Program Stakeholdersin the Evaluation
Process, CatholicRelief Services, 1993:38.
Social mapping
This tool, which can be used at various stages of a project, involves community members in drawing
maps of community structures, institutions, associations and resources on the floor, ground or paper.
Mapping can provide insight into the interactions or lack thereof within the community, the resources
that are available and access to them.
Social Mapping: The Importance of Having a Good Cross-Section of Participants

The diagram shown below shows the importance of ensuring a good cross section of participants in a
social mapping exercise and different gender interpretations of one's community.
When men were asked to map their village, they showed their village as a network of roads and services
used by them. They showed the official residences of the chief and chairman, the cotton trees where the
different clans met. As for the location of the school and hospital its proximity to the village did not
seem to be of concern. It was suggested that wells be located near each of the village clans.
When women were asked to map their village, they saw it through their own lenses. Women pointed to
the areas where they collected water and fuel and worked in the fields. Their attention was focused more
on the village per se than the outlying regions. The women specified specific houses inhabited by village
leaders. The women suggested that the hospital and school be located near the village with the well at the
center of the community for the greatest benefit of all.
Social mapping can highlight different perceptions of one's social environment. What is important to one
group may be less important to another. For that reason, it is critical that stakeholders are well
represented and come from a cross-section of the community.
Testimonial
A testimonial records a person's thoughts, feelings and experiences in the first person narrative style. It is
a way of learning about a project or its impact through the voices of participants and stakeholders.
Testimonials can help to reveal the degree of empowerment, the way in which income is used, how
decisions are made or issues tackled. They can also help to corroborate other sources of data and
information and provide a more personal insight into a project's achievements. Usually testimonials are
taped and played back to the participant.

Sample Testimonial:
Ms. Mosammat Jainab Bibi, the Manager of the Shahapur Bittaheen Women's Cooperative Society in the
district of Jamalpur, Bangladesh, joined the Cooperative Society in 1984 as a Manager. She is involved
in paddy-husking and poultry-rearing activities. She received training on Members Education, cow
rearing and poultry. She is also attending the Manager's Training regularly.
"I joined the Society in 1984. Mr. Tara, the local upazila official had distributed 15 wheat feeding cards
among 15 vulnerable female villagers. One day he told to mobilize another 15 women to form a society.
We did it and he helped us to form a BRDB society. We deposited Tk.1 per week as savings. We were not
united then. We did not know each other. When the other 15 women joined us we held a weekly meeting.
We continued it and Mr. Tara would also attend. We generated a little fund and Mr. Tara and we
deposited it at BRDB office. With our consent he formed a BRDB society for us. We deposited Tk.1 or 2
as savings in 1984. We did everything by ourselves like raising savings, depositing them at the bank,
issuing verity vouchers, taking receipts from bank etc. We registered our society on 30.3.85. It is nearly
7 or 8 years that we have been running our society....
We take a loan every year and husk paddy which provided us with some profit. We spend a little of that
for education of our children. Previously, the Railway School was completely reluctant to admit our
children. The directors of BRDB asked us once: "How many are you?" We answered we were 46. They
replied that means at least 46 children and advised us to go and admit our children in the Railway
school and gave us hope that they would help us. We went there, the teachers were in panic. We asked
them: "Why do you not want to admit our children-because we are poor? Since we have no clean
clothes? Why do you admit rich children?" Then the teachers agreed to admit our children.
They told us to pay Tk.10 for each boy or girl as an admission fee. We had protested earlier but realized
very soon that we had to pay Tk.10 because it was compulsory for everyone. They gave us 7 days to
collect the money. We collected the money and admitted our children in the school. That's how we
overcame the problem...We cultivate fish collectively.. We have no pond but we requested one old man to
provide us with his pond for fish culture. "We will cultivate fingerlings or young fish in your pond. We
will sell the fish after 2-3 months regularly and the rest will be yours", we said. We took the pond under
this condition and we earned Tk.880 in 2 months. We have been cultivating fish for 5-6 years and earn
Tk.500-600 each 2-3 months. We maintain the pond and take care of it, catch the fish and sell them. We
do not get the help of any men. We have utilized our training fully."
The above excerpts of a dialogue were part of a broader evaluation exercise of a training component of a
Bangladeshi project. The dialogue method was used to complement and enrich the quantified data
obtained through the interview questionnaire, case studies and file review.
Source: Yusuf Kassam, 1995:6-7.
A Transect Walk Through Mbusyani, Kenya
Soil Loose, deep red soil Sandy soils and small Shallow sandy soils
patches of red soils rocky in most parts

Water About 1/4 of A river infested with Water in Kilindiloni

households have bilharzia, two poorly river salty. River
shallow well area also maintained dams Kathana has bilharzia.
has three dams & one Roof catchment in
spring progress
Vegetation All natural vegetation High proportion of Natural vegetation

cleared to give way to natural vegetation consisting of acacia
settlement mainly acacia lantana, shrubs and grass
canola grasses
Social-Economic 1/2 of household heads 1/2 of household- Mainly grass thatched

Indicators in wage employment corrugated iron roofs, houses
majority have magati 1/2 grass thatched,
roofs brick or stone brick walls
walls
Food Crops Maize, beans, pigeon Maize, beans, a lot of Maize, beans, peas,
peas, bananas sore pigeon millets, bananas, fruits
fruits, bananas
Cash Crops Coffee Coffee Coffee
Achievements (Last 5 Soil conservation, tree Soil conservation, Some soil and water
Years) planting, water water development, conservation
development - wells, dams
roof catchment
Forestry/Agro Forestry Widespread Minimal tree planting Very little tree

agroforestry with but mangoes and paw planting
grevillea, eucalyptus, paws planted
mangoes, and paw
paws
Resources A lot terracing A lot of bench Limited soil

Management embankments terracing conservation
reinforced with multi-
purpose grasses
Problems Inadequate water, Water, famine, Water, Transport and

education and health inadequate education Food
facilities, famines and and health facilities
lack of dip facilities
Opportunities Rehabilitation 3 dams, Water development- Water development-

one spring. External dam, well, roof dams,well, roof
assistance-tools, catchment. catchment. External
market Government assistance assistance

Source: National Environment Secretariat. Participatory Rual Appraisal Handbook: Conducting PRAs in
Kenya,1991:21
Transect walk
A transect is usually a straight cut through the community. A transect walk involves walking through a
community with the local people. It seeks to cover all major ecological, production and socially stratified
zones of that community and usually includes observation, asking questions, pointing and discussing
what is being seen-zones, land, vegetation, local markets, community service centres, schools-and
mapping the areas.
Venn diagram
A Venn diagram, of usually circular areas, can be used to look at relationships within institutions or
relationships between the community and other organizations. It illustrates different participant
perceptions of access to resources or of social restrictions, for example. Circles of various sizes are cut
out of paper and given to participants, who are then asked to allocate the circles to different institutions,
groups or departments. The larger the circle the more important it is. The circles may overlap, showing
the degree of contact between institutions or groups.
A Potato Project in Pakistan:
Participants from headquarters and the regions were asked to identify the different institutions,
associations and target groups related to their project. Participants were asked to select circles of
different sizes to represent the importance of an institution or group. It was noted that the headquarters
staff located in Islamabad mapped a wide range of institutions with which they maintained contact:
donors, regional research agencies, private businesses. Provincial staff closest to headquarters had less
knowledge of international linkages yet knew of many linkages, such as the research agencies, farmers
and different private-sector groups. Those most remote showed even fewer linkages and less interaction
with other institutions.
In barrack, one of the villages, the following indicators were used to distinguish levels of wealth within
the village:
Pile/Category Household No. Indicators

I 43, 42, 38, 36, 34, 28, 41, 35 •mostly landowners
•some houses are made of

permanent materials
•own more livestock (cows,

carabaos, pigs and chickens)
•mostly receiving remittances

from children working in
Manila or abroad
II 39, 37, 27, 22 20, 19, 17, 33 • mostly either tenants or

21,18 tenants and at the same time
andowners of small land
parcels
• mostly own a number of

livestocks (cows, carabaos,
pigs and chickens)
• mostly receiving remittances

from children working in
Manila
III 40, 31, 26, 25, 23, 16, 7, 4, 2, • tenents of small land parcels
30, 24, 15, 1
• hired labourers
• majority don't have caraboas

and other livestock
• old folks dependent upon

children's support
IV 32, 29, 13, 12, 10, 9, 3, 5, 6, 8, • majority are not cultivating

11, 14 any land parcels
• no caraboas (as draft animal)
• dependent mostly on any of

the following sources of
income:
-fishing (small scale)
-tuber gathering (small scale)
-gathering and selling

firewood/charcoal
-hired labour/maids
Source: F.T. Banlina Ly Tung, "Farm Experiences of Wealth Ranking in the Phillippines: Different
Farmers Have Different Needs" in "Special Issues on the Application of Wealth Ranking", RRA Notes,
no. 15, International Institute for Environment and Development, May 1992: 48-50.
Wealth ranking
Wealth ranking uses the perceptions of villagers or the local population to rank households within a
village according to wealth. The local population is involved in listing the criteria to identify the poor
and those who are better off. What emerges is a set of criteria that in the eyes of the local population
indicates what constitutes poverty or wealth. The results can and are many times at odds with
conventional methods of classifying people solely according to income. For example, in a small village
in India, when villagers listed 30 criteria for identifying the poor, land ownership was only one of the
variables. Villagers viewed a widow who had land but could not cultivate it as poor whereas Indian
planners did not consider her to be poor.
Wealth Ranking in the Philippines:

The Farm and Resource Management Institute of the Philippines tested wealth ranking in three villages.
The Institute obtained a list of households from the Barangay captain, the political head of the village.
Cards were prepared for each family (see diagram to the right). Five informants for each village were
identified and asked to sort the cards into three to five piles, indicating the different wealth groups.
Zielorientierte Projektplanung (ZOPP)

(objectives-oriented project planning)
ZOPP is a set of procedures and instruments that seek to integrate stakeholder concerns into project
planning. Introduced by Gesellschaft für Technische Zusammenarbeit (GTZ) (German Agency for
Technical Cooperation) in 1983, it involves a number of stages, including participation analysis, problem
analysis, defining the problem tree, and an analysis of objectives, all of which is undertaken over a one-
week period and at different stages of the project cycle, from project design and management to
evaluation. ZOPP involves participation analysis, which seeks to integrate the interests and expectation
of persons and groups significant to the project.
ZOPP in Action:
The following diagram shows ZOPP in action. Pieces of paper in various sizes and colours have been
prepared in advance. The different shapes represent different ideas, priorities or themes which can then
be reorganized easily on a blackboard. Participants will use a few words to describe their idea on the
piece of paper. ZOPP can be used in evaluation to brainstorm over the key areas of evaluation, indicators
of assessment, and roles and responsibilities for data-gathering.
Best of the Best: Manuals for Doing Participatory Evaluations
1. Participatory Learning and Action - A Trainer's Guide

Authors: Jules Pretty, Irene Gujit, Ian Scoones and John Thompson International Institute for
Environment and Development (IIED), London, 1995.
This manual, which offers a step-by-step approach to participatory learning and action
methodologies, is an excellent resource for trainers and practitioners interested in participatory

development.
2. Participatory Program Evaluation: A Manual for Involving Program Stakeholders in the Evaluation
Process
Author: Judi Aubel Catholic Relief Services, Senegal, December 1993. Available through PACT,
New York.
Basing her work on experiences in Africa, Asia and Latin America, the author details the various
steps involved in participatory evaluation and draws examples from field experience.
3. Partners in Evaluation: Evaluating Development and Community Programmes with Participants
Author: Marie-Thérése Feuerstein MacMillan Education Ltd., London, 1986.
This pioneering work offers a thorough step-by-step approach to participatory evaluation.
Excellent graphics and drawings accompany the text. A must-read for participatory evaluation
facilitators and others interested in participatory evaluation.
4. Participatory Evaluation: Tools for Managing Change in Water and Sanitation
Author: Deepa Narayan
World Bank Technical Paper no. 207, The World Bank, Washington, D.C., 1993.
This paper provides policy-makers, managers and planning and evaluation staff with information
about participatory processes and indicators that can be used to involve stakeholders in
programme evaluation.
5. Poverty and Livelihoods: Whose Reality Counts?
Author: Robert Chambers
Division of Public Affairs, United Nations, Development Programme, New York, 1994.
This policy paper, commissioned by UNDP for the World Summit for Social Development
(Copenhagen, March 1995), presents a new paradigm for assessing the realities of poor people
and deciding what needs to be done.

Annex III.
WHO'S DOING WHAT? ORGANIZATIONS SUPPORTING
PARTICIPATION
Various donors and organizations have been experimenting with the concept
of participation for a number of years. For some, participation is not simply a
technique or approach to be applied at different stages of the project cycle; it
is also a philosophy about how development is approached and implemented.
This section seeks to summarize key organizational thinking on the subject of

participatory development and evaluation. It should be noted that for most
donors, participatory evaluation is relatively new and challenges conventional
modes of evaluation with its emphasis on outside external experts. There is
much work to be done before participatory evaluation methods and
approaches are mainstreamed into the operational processes of organizations.
More time will also be needed to change attitudes, approaches and
institutional cultures.
Canadian International Development Agency (CIDA)

Support for participatory development has been concentrated primarily at the
project level. The creation of umbrella mechanisms or microrealization
projects has permitted the funding of numerous discreet activities at the grass-
roots level that promote and foster participation. Workshops have also been
launched to try to introduce staff to PRA techniques and approaches. The
agency is currently seeking to address participation at the policy and
programme levels in a more consistent manner. CIDA is also exploring, in
cooperation with the World Bank Inter-Agency Group on Participation, the
institutional changes that are required to mainstream participatory approaches
within donor organizations.
Food and Agriculture Organization (FAO)

In 1992, FAO put forward a Plan of Action on People's Participation with the
aim of ensuring the "active participation of people in the achievement of
sustainable development... Without participation, rural development
initiatives are unlikely to be sustainable in the long run and rural inequities
are unlikely to be addressed" (FAO: 1992:5). FAO has established a People's
Participation Service within the Division of Women and Participation. This
service, established in 1992, has built on the work pioneered in the mid-

1970s, initially through the Rural Organization Action Programme and Small
Farmer Development Programmes and in the 1980s through the People's
Participation Programme, which aimed at promoting self-managed and self-
reliant groups at the community level.
Gesellschaft für Technische Zusammenarbeit (GTZ).

Since the mid-1980s, GTZ has spearheaded a number of efforts aimed at
increasing participation. The ZOPP method was introduced into GTZ's
project cycle management approach. This method seeks stakeholder
participation in the planning and monitoring processes of projects and is now
used with other methods and tools, such as PRA. Regional Learning Groups
on Participation have been created with a view toward gaining a better
understanding of what is happening at the local level and how the
organization can be adapted to promote participation at the project and
societal levels. Numerous projects have been initiated that have adopted
bottom-up approaches aimed at initiating ownership and the self-organization
capacity of the community.
International Fund for Agricultural Development (IFAD)

IFAD has been concerned with participation of the rural poor over the past
ten years. People's participation is viewed both as a means and an end to
poverty alleviation. Apart from believing that "investing in the production
potential of the poor can bring high returns", IFAD views participation as a
catalyst for self-supporting and sustainable development. IFAD has invested
over $2.6 billion in self-help, participatory development projects targeted at
small-holder farmers, the landless, rural women, fishermen, nomadic
herdsmen and agro-pastoralists
(Lineberry, 1989).
International Institute for Environment and Development (IIED)

Located in England, IIED is a policy research institute linking environmental
concerns with the development needs of resource-poor people in the South
and with other global environment and development priorities. The Institute
has been instrumental in developing and spreading participatory
methodologies and approaches. Since 1988, it has put out a series of user-
friendly RRA notes, which are now called PLA Notes-Notes on Participatory
Learning and Action. Development practitioners from around the world
contribute to the notes by sharing their experiences, conceptual reflections

and methodological innovations.
Overseas Development Administration (ODA)

For ODA of England, participation is seen as a central element in achieving
effective, sustainable development. ODA is concerned with how to involve
other key secondary and primary stakeholders in the monitoring and
evaluation process and developing systems and procedures for more
systematic assessment of impact. The agency has recently published a
number of guides that look at measuring participation, doing a stakeholder
analysis, and enhancing stakeholder participation in aid activities.
Society for Participatory Research in Asia (PRIA)

PRIA is an NGO based in India that promotes people-centred development,
participatory research and participatory development. As an educational
support institution, PRIA offers training in participatory development and
participatory methodology to grass-roots organizations and to personnel from
bilateral and multilateral organizations and from government and semi-
government institutions. PRIA conducts participatory training of trainers and
participatory evaluations for grass-roots organizations.
United Nations Development Programme (UNDP)

UNDP has defined grass-roots participation as "a process whose objective is
to enable people to initiate action for self-reliant development and acquire
the ability to influence and manage change within their society". The
promotion of participatory approaches and empowerment of people are key
intervention points for UNDP activities. Establishing close partnerships with
NGOs, community-based organizations (CBOs) and other civil society
groups is an essential element of external support. This builds on earlier
efforts to promote regional and global programmes that seek to promote
participation at the grass-roots level.
United Nations Children's Fund (UNICEF)

UNICEF advocates a people-centered, poverty alleviation approach to
environment and development. Participation is seen as key to empowering
people to take control of their lives and to act to meet their own needs.
United States Agency for International Development (USAID)

Participation is a key dimension of the Agency's efforts to redesign

government. A systemic approach is currently being used in redesigning

systems and operations so that they are more responsive to USAID
customers-the poor. Participation is seen as essential for moving from macro-
management, top-down approaches to those that empower and enable.
USAID holds a series of informal, monthly meetings on participatory
development for its staff. Speakers, who come from both inside and outside
the agency, are invited to share their experiences. A Senior Advisor
specifically promotes the integration of participation into Agency activities
and acts as a focal point at USAID.
World Bank
The World Bank defines participation "as a process through which
stakeholders influence and share control over development initiatives, and the
decisions and resources which affect them". A Learning Group on
Participatory Development, established four years ago with representation
from all the Bank's departments, has led the Bank to adopt an action plan that
advocates greater stakeholder participation in all aspects of the Bank's
operations. Participatory development is increasingly being looked at as
essential for achieving greater efficiency and sustainability in Bank
operations. The recent World Bank Participation Sourcebook provides an
overview of the Bank's experience as well as methods and tools that enable
the poor to participate. The Bank has also piloted several beneficiary
assessment and stakeholder evaluations and supported participatory
approaches in Bank operational activities.
At an international level, the World Bank is spearheading an inter-agency

group on participation composed of six sub-groups: institutional changes and
mainstreaming of participatory approaches; project preparation and
implementation; training and learning; monitoring and evaluation;
information dissemination; and country-level capacity-building.
World Neighbors
World Neighbors is a small development agency with an overall budget of
$3.2 million supporting programmes in approximately 22 countries that seek
to "strengthen the capacity of marginalized communities to meet their basic
needs". World Neighbors has been interested in participatory development
and evaluation since the 1980s and published its own handbook on self-
evaluation. More recently, World Neighbors has been concerned with
strengthening its ability to learn from its experience, particularly in terms of

"strengthening community capacity" to meet basic needs, and to use this

experience to influence peer agencies and policy-makers. World Neighbors
has been applying PRA techniques, such as social mapping, wealth ranking,
time-line, household composition and identification of women of
reproductive age, to its development work.

Annex IV.
RESOURCE PERSONS, GROUPS AND INSTITUTIONS
The following individuals, groups and institutions provide training, research
and publications related to participatory evaluation as it is discussed in this
handbook.
Ms. Ariane Berthoin Antal

Director of Ashnage International Institute for Organizational Change
French-Geneva Campus
741 66 ARCHAMPS, FRANCE
TEL: (33-50) 31-5600
FAX: (33-50) 31-5606
Oguz Baburoglu
School of Business Administration
Bilkent University
P. 0. Box 8
06572 Maltepe
Bilkent, Ankara, TURKEY 06533
TEL: (90-31) 2-266-4164
FAX: (90-31) 2-266-4985
E-mail: babur@bilkent.edu.tr
Mr. Tim Baker

University of Taxmania
c/o 85 Berkley Street
Hawthorn, Victoria, AUSTRALIA 31 22
TEL: (61-3) 98192504
FAX: (61-3) 98182504
Dr. Rémy-Claude Beaulier

CIDA - Social Policy Branch
Place du Centre
200 Promenade du Portage
Québec, Canada KI A OG
TEL: (1-819) 953-6376
FAX: (1-819) 953-6356

E-mail: remy-beaulieu@acdi-cida.gc.ca
Prof. L. Dave Brown

Institute for Development Research
210 Lincoln Street
Boston, Massachusetts 02111 USA
TEL: (1-617) 422-0422
FAX: (1-617) 422-0494
E-mail: econet:idr@jsi.com
Mr. Federico Butera

Instituto RSO srl Ricerche sui Sistemi
Org Via Leopardi 1
1-201 23 Milano, ITALY
Dr. Juanita Campos

Adjunct Faculty & Research Scientist
Bureau for Applied Research in Anthropology (BARA)
Anthro Bldg. 317, A
University of Arizona
Tucson, Arizona 85721 USA
TEL/FAX: (520) 743-0622 (home)
TEL: (1-520) 622-5549 (BARA)
FAX: (1-520) 622-5449 (BARA)
Ms. Alice Carloni, Rural Sociologist

Investment Center Division
Technical Cooperation Department
Food and Agriculture Organization (FAO)
Viale delle Terme di Caracalla
1-001 00 Rome, ITALY
TEL: (39-6) 522-55799
FAX: (39-6) 522-54657
E-mail: alice.carloni@fao.org
Hillel Efrat
Kibutz Industries Association
P. O. Box 400012, Tel-Aviv 61400

8 Shderot Shaul Hamelech, ISRAEL

TEL: (972-3) 6955413/4/5
FAX: (972-3) 6951464
Dr. David Evans

Center for International Education
285 Hills House South
University of Massachusetts
Amherst, MA 01003 USA
TEL: (1-413) 545-0465
FAX: (1-413) 545-1263
E-mail: cie@educ.umass.edu
Dr. Rosalind Eyben

Principal Social Development Advisor
Overseas Development Administration
94 Victoria Street
London SW1 E5DH, England
UNITED KINGDOM
TEL: (44-1 71) 917-0566
FAX: (44-1 71) 917-0197
E-mail: esdorje.vs3@oda.gtnet.gov.uk
Prof. Orlando Fals-Borda

Instituto de Estudios Políticos
Universidad Nacional
Bogotá, COLOMBIA
TEL: (1-571) 3681 579
FAX: (1-517) 3687 471
Dr. Marie-Thérése Feuerstein

49 Hornton Street
London W8 7NT, ENGLAND
Mr. Phil Glaser, Director

Radford, Glaser & Associates
22 David Street
Pietermaritzburg 3201, SOUTH AFRICA

TEL: (0331) 428185

FAX: (0331) 425482
Mr. Alfonso Gonzalez

Grupo de Estudios
Allende 7, Colonia
Apartado Postal 76
Mexico City, MEXICO
Mr. Davyd J. Greenwood

John S. Knight Professor of International Studies
Center for International Studies
Cornell University
170 Uris Hall Tower Rd.
Ithaca, New York 14853-7601, USA
TEL: (1-607) 255-6370
FAX: (1-607) 254-5000
Mr. Peter Gubbels, Coordinator

World Neighbors Organizational & Institutional Development
Voisins Mondiaux, OI BP 1315
Ouagadougou OI BURKINA FASO
TEL: (226) 34.55.60
FAX: (226) 34.15.92
E-mail: gubbels@voisins.mondiaux.bf
Mr. Khalid El Harizi

Senior Evaluation Officer
Office of Evaluation & Studies
International Fund for Agricultural Development (IFAD)
107 Via del Serafico
00142 ROME, ITALY
TEL: (39-6) 545-92059
FAX: (39-6) 519-1702
Mr. Michael I. Harrison

Dept. of Sociology & Anthropology
Bar Ilan University

52900 Ramat Gan, ISRAEL

TEL: (972-3) 531 8333
FAX: (972-3) 535 1825
E-mail: bitnetf42166@barilan
Mr. Thomas Kuby

Sr. Planning Officer
GTZ Dag-Hammarskjold-Weg 1-2
Postfach 51 80
D-6236 Eschborn bei Frankfurt/Main, GERMANY
TEL: (49-61) 967-91741
FAX: (49-61) 967-96109
E-mail: tkuby@worldbank.org
Prof. Yvonna Lincoln

Dept. of Educational Administration
Texas A & M University
222 Mt. Harrington Education Center
College Station, TX 77843 USA
TEL: (1-409) 845-2716
FAX: (1-409) 862-4347
E-mail: eioiyl@tamvml.tamu.edu
Dr. Erma Manoncourt

Programme Communication/Social Mobilization
UNICEF
3 United Nations Plaza D-H-40F
New York, NY 10017 USA
TEL: (1-212) 702-7245
FAX: (1-212) 702-7154
E-mail: emanoncourt@icg.apc.org
Dr. Ineke Meulenberg-Buskens

Centre for Science Development
Human Science Research Council
Private Bag 41
Pretoria 0001, REPUBLIC OF SOUTH AFRICA
TEL: (27-12) 202-2604
FAX: (27-12) 202-2421

E-mail: jajmb@gallup.hsrc.ac.za
Mr. Timothy Murphy

European Bank for Reconstruction and Development
One Exchange Square
London EC@A 2EH, UNITED KINGDOM
TEL: (44-171) 338-6020
FAX: (44-171) 338-6848
E-mail: murphyt@ebrd6.ebrd.com.
Mr. Peter Reason

Center for Action Research in Professional Practice
University of Bath
School of Management
Bath BA2 7AY, UNITED KINGDOM
TEL: (011-44) 225-826792
FAX: (011-44) 225-826473
Mr. James Sessions, Director

Highlander Research and Education Center
1959 Highlander Way
New Market, Tennessee 37820 USA
TEL: (1-615) 933-3443
FAX: (1-615) 933-3424
Mr. William Staub

Sr. Social Development Specialist
Social Development Division
Asian Development Bank
Manila, PHILIPPINES
TEL: (63-2) 632-6756
FAX: (63-2) 741-7961
E-mail: wstaub@mail.asiandevbank.org
Prof. Marja-Liisa Swantz

United Nations University
World Institute for Development
Economics Research

Katajanokanlaituri 6B
00160 Helsinki, FINLAND
TEL: (358-9) 615-9911
FAX: (358-9) 693-8548
E-mail: wider@wider.unu.edu
Dr. Rajesh Tandon

Society for Participatory Research in Asia
42, Tughlakabad Institutional Area
New Delhi 110062, INDIA
Mr. Francisco Vio Grossi

Secretary General - CEEAL
Diagonal Oriente
1604, Casilla 6257
Santiago, CHILE 22
Ms. Sonam Yangchen

Programme Advisor NGO Programme
Sustainable Development and Poverty
Elimination Division Bureau for Policy and Programme
Support UNDP
One UN Plaza, DCI Rm. 2058
New York, NY I0017 USA
TEL: (1-212) 906-6029
FAX: (1-212) 906-5857
E-mail: sonamyangchen@undp.org

Who are the question-makers? - Bibliography
BIBLIOGRAPHY
References Mentioned in the Text
Aubel, Judi. Participatory Program Evaluation: A Manual for Involving

Program Stakeholders in the Evaluation Process. Catholic Relief Services,
Senegal, 1993.
Banlina, F. T. and Ly Tung. "Farm Experiences of Wealth Ranking in the

Philippines: Different Farmers Have Different Needs". International
Institute for Environment and Development. "Special Issues on
Application of WealthRanking". RRA Notes, no. 15, May 1992,
pp. 48-51.
Campos, Juanita Diane. Towards Participatory Evaluation: An Inquiry into

Post-Training Experiences of Guatemalan Community Development
Workers.
Dissertation submitted to the University of Massachusetts, School of
Education, May 1990.
Chambers, Robert. "Participatory Rural Appraisal (PRA): Analysis of

Experience", in World Development, vol. 22, no. 9, 1994:1253-1268.
Coupal, Françoise. "Participatory Project Design: Its Implications for

Evaluation. A Case Study from El Salvador". Paper presented at the 1995
Evaluation Conference, 1-5 November, Vancouver, British Columbia.
Dinámica Integral Ltda. El Metaplan-Metodología Eficiente y Eficaz para

Lograr Objetivos con Participación Grupal, Colombia.
Fals-Borda, Orlando and Mohammad Anisur Rahman (eds.). Action and

Knowledge: Breaking the Monopoly with Participatory Action Research.
London, Intermediate Technology Publications, 1991.
Farrington, John and Adrienne Martin. "Farmer Participation in

Agricultural Research: A Review of Concepts and Practices", Agricultural
Administration Occasional Paper no. 9, London, 1988.
http://www.undp.org/eo/documents/whobib.htm (1 of 16) [29/01/2002 19:03:13]

Fernandez, Maria. Participatory-Action Research and the Farming Systems

Approach with Highland Peasants. Colombia, Department of Sociology,
University of Missouri.
Feuerstein, Marie-Thérése. Partners in Evaluation: Evaluating Development

and Community Programmes with Partners. London, MacMillan, 1986.
Freedman, Jim. Participatory Evaluations-Making Projects Work. Technical

Paper no. TP94/2. Calgary, Canada, Division of International
Development, International Centre, University of Calgary, 1994.
Freire, Paulo. Pedagogy of the Oppressed. New York, Continuum Publishing

Company, 1993.
GTZ. ZOPP: An Introduction to the Method. Frankfurt, Germany,

March 1988.
Guba, Ego and Yvonna Lincoln. Fourth Generation Evaluation. England,

Age Publications, 1990.
International Institute for Environment and Development. "Special Issues

on Application of Wealth Ranking". RRA Notes, no. 15, May 1992.
Kassam, Yusuf. "The Combined Use of Participatory and Survey

Methodology in Evaluating Development Projects. A Case Study of a
Rural Development Project in Bangladesh". September 1995. Paper
presented at the 1995 Evaluation Conference, 1-5 November, Vancouver,
British Columbia.
Marsden, David, Peter Oakley and Brian Pratt. Measuring the Process:
Guidelines for Evaluating Social Development. Oxford, 1994.
Narayan, Deepa. Participatory Evaluation Tools for Managing Change in

Water and Sanitation. Washington, DC, World Bank Technical Paper no.
207, 1993.
National Environment Secretariat, Government of Kenya; Clark

University; Egerton University; The Centre for International

Development and Environment of the World Resources Institute.
Participatory Rural Appraisal Handbook: Conducting PRAs in Kenya.
Natural Resources Management Support Series no. 1, 1991.
Overseas Development Administration. Note on Enhancing Stakeholder

Participation in Aid Activities. England, Social Development Department,
ODA, April 1995.
Pretty, Jules, Irene Gujit, Ian Scoones and John Thompson. Participatory
Learning and Action: A Trainer's Guide. London, International Institute
for Environment and Development, 1995.
Rahman, Mohammad Anisur. People's Self-Development. London, Zed

Books, 1993.
Salmen, Lawrence F. Beneficiary Assessment-An Approach Described.

Environment Department Papers no. 23, Washington, DC, World Bank,
July 1995.
UNDP. Building a New UNDP. New York, Office of Evaluation and

Strategic Planning, 1995.
_____. "Report of the UNDP Capacity Development Building Retreat".

New York, Office of Evaluation and Strategic Planning, 1995.
_____. Strategic Management in UNDP. New York, Office of Evaluation

and Strategic Planning, 1995.
_____. UNDP Evaluation Findings in 1994. New York, Office of

Evaluation and Strategic Planning, 1995.
_____. UNDP Evaluation Findings in 1995. New York, Office of

Evaluation and Strategic Planning, 1996.
Uphoff, Norman. "Monitoring and Evaluating Popular Participation in

World Bank-Assisted Projects" in B. Bhatnagar and A. C. Williams,
Participatory Development and the World Bank: Potential Directions for

Change, World Bank Discussion Paper no. 183. Washington, DC, World
Bank, 1992.
Anayanwn, C. N. "The Technique of Participatory Research in

Community Development", Community Development Journal, January
1988, vol. 23:11-15.
Armstrong, John and Michael Key. "Evaluation, Change and Community

Work", Community Development Journal, October 1979, vol. 14:210-223.
Campos, Juanita Diane. Towards Participatory Evaluation: An Inquiry into

Post-Training Experiences of Guatemalan Community Development
Workers.
Dissertation submitted to the University of Massachusetts, School of
Education, May 1990.
Chambers, Robert. "Participatory Rural Appraisal (PRA): Analysis of

Experience" in World Development, vol. 22, no. 9, pp. 1253-1268, 1994.
_____. "Shortcut Methods for Gathering Social Information for Rural

Development Projects" in M. Cernea, Putting People First. Oxford, Oxford
University Press.
Clark, Noreen and James McCaffery. Demystifying Evaluation. World

Education, Boston, Massachusetts, 1979.
Coupal, Françoise. "Participatory Project Design: Its Implications for

Evaluation. A Case Study from El Salvador". Paper presented at the 1995
Evaluation Conference, 1-5 November, Vancouver, British Columbia.
Cuthbert, Marlene. "Evaluation Encounters in Third World Settings" in M.

Q. Patton (ed.), Culture and Evaluation, San Francisco, Jossey-Boss,
pp. 29-35.
DAC Expert Group on Aid Evaluation. Evaluation and Participation. An

Approach Paper. Development Studies Unit, Department of Social

Anthropology, Stockholm University, September 1995.
Derricourt, Nick and Peter Oakley (eds.). "Participation in Evaluation",

Special Issue, Community Development Journal, vol. 23:1, 1988.
FAO. Community Forestry-Participatory Assessment, Monitoring and

Evaluation. Rome, FAO, 1989.
Fernandes, Walter and Rajesh Tandon. Participatory Research and

Evaluation-Experiments in Research as a Process of Liberation. New Delhi,
Indian Social Institute, 1981.
Fetterman, David (ed.). Qualitative Approaches to Evaluation in Education.

New York, Praeger Press, 1988.
Feuerstein, Marie-Thérése. "Community Participation in Evaluation:

Problems andPotentials.", International Nursing Review, 1980, vol. 27 (6),
187-190.
_____. "Evaluation by the People". International Nursing Review, 1978,

vol. 25 (5), 146-153.
_____. "Finding the Methods to Fit the People: Training for Participatory
Evaluation", Community Development Journal, 1988, vol. 23: 16-25.
_____. Participatory Evaluation: by, with and for the People. IDEA:
Reading
Rural Development Communications Bulletin. London: University of
Reading Agricultural and Extension and Rural Development Centre,
April 1982, vol. 14: 18-23.
_____. Partners in Evaluation: Evaluating Development and Community

Programmes with Participants. London, MacMillan Education Ltd., 1986.
Forss, Kim. Report from the Participatory Evaluation of the Royal

Government
of Eland/UNDP Project ELA/90/003-Rural Water Supply and Sanitation.
6 September 1994, UNDP.

_____. Participatory Evaluation: Questions and Issues. A Report

Commissioned by the Central Evaluation Office, Occasional Paper no. 1.
New York, UNDP, October 1989.
Forss, Kim and Claus Rebien. "Evaluation with (and without)

Participation. A Case Study of the Evaluation of the UNDP project
URT/89/003-Strengthening the Ministry of Water-Towards Achieving
IWSSD Objectives". Report. New York, UNDP, 1992.
Freedman, Jim. Participatory Evaluations-Making Projects Work. Technical

Paper no. TP94/2. Calgary, Canada, Division of International
Development, International Centre, University of Calgary, 1994.
Gariba, Sulley. "Participatory Impact Assessment as a Tool for Change:

Lessons from Recent Experience in Poverty Alleviation Projects in Africa".
Paper presented at 1995 Evaluation Conference, 1-5 November, Vancouver,
British Columbia.
Ghai, D. and D. Westendorff (eds.). Monitoring Social Progress in the

1990s: Data Constraints, Concerns and Priorities. UNRISD, Avebury,
Aldershot, 1994.
Guba, Egon and Yvonna Lincoln. "Do Inquiry Paradigms Imply Inquiry
Methodologies?" in David Fetterman (ed.). Qualitative Approaches to
Evaluation in Education: The Silent Revolution. New York, Praeger Press,
1988, 89-115.
_____. Fourth Generation Evaluation. England, Age Publications, 1990.
Hall, Budd, Alan Etherington and Ted Jackson. "Evaluation, Participation

and Community Health Care: Critique and Lessons". Paper presented at the
American Health Association, Toronto, Canada, November 1979.
Jackson, Edward. "Participatory Impact Assessment for Poverty Alleviation

Opportunities for Communities and Development Agencies". Paper
presented at the 1995 Evaluation Conference, 1-5 November, Vancouver,
British Columbia.

Jackson, E. T. and Kassam, Yusuf. Better Knowledge, Better Results:

Participatory Evaluation in Development Cooperation, 1996.
Kassam, Yusuf. "The Combined Use of Participatory and Survey

Methodology in Evaluating Development Projects. A Case Study of a Rural
Development Project in Bangladesh". September 1995. Paper presented at
the 1995 Evaluation Conference, 1-5 November, Vancouver, British
Columbia.
Kensey, David. Evaluation in Nonformal Education. The Need for

Practitioner Evaluation. Amherst, Massachusetts, Centre for International
Education, University of Massachusetts, 1981.
_____. "Participatory Evaluation in Adult and Nonformal Education". Adult

Education, vol. 31:155-168.
Krishnamurthy, Lakshmi. "Exploding Myths: The Participants Evaluate" in

W. Fernandes and R. Tandon (eds.). Participatory Research and Evaluation:
Experiments in Research as a Process of Liberation. New Delhi, Indian
Social Institute, pp. 151-165.
Krueger, R. A. Focus Groups: A Practical Guide for Applied Research.

Newbury Park, Sage Publications, 1988.
Lackey, Alvin, Mack Peterson and Jeff Pine. "Participatory Evaluation: A

Tool for Community Development Practitioners". Journal of the Community
Development Society, 12 pp., 1981.
Marsden, David, Peter Oakley and Brian Pratt. Measuring the Process:
Guidelines for Evaluating Social Development. Oxford, 1994.
Merrifield, Juliet. "Putting the Scientists in Their Place: Participatory

Research in Environmental and Occupational Health". Highlander Center
Working Paper, New Market, Tennessee.
Merschrod, Kris. "Participation in Program Evaluation at the Regional Level

in Honduras". Rural Development Participation Review, vol. 11:18-22,

1980.
Narayan, Deepa. Participatory Evaluation Tools for Managing Change in

Water and Sanitation. Washington, D.C., World Bank, Technical Paper no.
207, 1993.
Oakley, Peter. "Conceptual Problems of the Monitoring and Evaluation of

Qualitative Objectives of Rural Development". Community Development
Journal, vol. 23:3-11, January 1988.
_____. "Evaluating Social Development: How Much or How Good?". IDEA:

Reading Rural Development Communications Bulletin. London, University
of Reading Agricultural and Extension and Rural Development Centre, vol.
14:12-17, 1982.
Pagaduan, Maureen and Elmer M. Ferrer. "Working as Equals-Towards a

Community Based Evaluation System." Community Development Journal,
vol. 18:146-159, 1983.
Participatory Research Network. Report on lnternational Forum on

Participatory Evaluation. New Delhi, India, International Council for Adult
Education, 1988. 82 pp.
Pfohl, Jacob. Participatory Evaluation. A Users' Guide. New York, PACT

Publications, 1986.
Quinn Patton, Michael. Alternative Evaluation Research Paradigm. Frank

Forks, North Dakota, University of North Dakota, February 1965.
_____. Culture and Evaluation. San Francisco, Jossey-Bass, 1985.
_____. "Developmental Evaluation". Evaluation Practice, vol. 15, no.3, pp.

311-319, 1994.
_____. How to Use Qualitative Methods in Evaluation. Newbury Park, Sage

Publications, 1987.

_____. Qualitative Evaluation Methods. Beverly Hills, Sage Publications,

1980.
_____. Utilized-Focus Evaluation. Beverly Hills, Sage Publications, 1978.
Robinson, Sheila and Philip Cox. Process Evaluation in Nepal: Tracking

Capacity-Building in Health Development. Dialogue on Development,
Technical Paper no. TP95/1, Division of International Development,
International Centre, The University of Calgary, Canada, 1995.
Rugh, Jim. Self-Evaluation Ideas for Participatory Evaluation of Rural

Community Development Projects. Oklahoma, World Neighbors, 1992.
Salmen, Lawrence F. Beneficiary Assessment - An Approach Described.

Environment Department Papers no. 23, Washington, D.C., World Bank,
July 1995.
_____. Listen to the People - Participant-Observer Evaluation of

Development Projects. Washington, D.C., World Bank, 1987.
Srinivasan, Viji. "The Methodology of Participatory Evaluation" in W.

Fernandes and R. Tandon (eds.), Participatory Research and Evaluation:
Experiments in Research as a Process of Liberation. New Delhi, Indian
Social Sciences Institute, 1981.
Stecker, Carl. Participatory Evaluation: A Review of the Literature and Its

Implications for Participatory Community Development, Comprehensive
Paper One, University of Massachusetts, School of Education, April 1993.
Swiss Directorate for Development Cooperation and Humanitarian AID

(SDC). Mirror, Mirror on the Wall... Self-Evaluation in Development
Cooperation. Bern, Evaluation Service, August 1991.
Tandon, Rajesh and Walter Fernandes. Participatory Evaluation: Issues and

Concerns. New Delhi, Society for Participatory Research In India, 1988.
_____. Participatory Research and Evaluation. New Delhi, Indian Social

Institute, 1981.

_____. Participatory Research and Evaluation: Experiments in Research as

a Process of Liberation. New Delhi, Indian Social Institute, 1982.
_____. Report of International Forum on Participatory Evaluation, 1-5

March 1988, New Delhi. New Delhi, 1988.
Uphoff, Norman. "A Field Methodology for Participatory Self-Evaluation".

Community Development Journal, vol. 26, no. 4, 1991.
_____. Monitoring and Evaluating Popular Participation in World

Bank-"Assisted Projects" in B. Bhatnagar and A. C. Williams, Participatory
Development and the World Bank: Potential Directions for Change, World
Bank Discussion Paper no. 183. Washington, DC, World Bank, 1992.
_____. "Participatory Evaluation of Rural Development Projects", The

International Fund for Agricultural Development, Monitoring and
Evaluation Division, Economic and Planning Department, 1991.
World Bank. Evaluation and Development, Proceedings of the 1994 World

Bank Conference. World Bank Operations Evaluation Department.
Yacoob, May and Tom Cook. "Evaluating Community Participation".

WASH Working Paper, no. 98, Rosslyn, Virginia, USA, 1991.
Participatory Development
Australian International Development Assistance Bureau. "Social Analysis

and Community Participation-Guidelines and Activity Cycle Checklist".
Appraisal, Evaluation and Sectoral Studies Branch, 1991.
Beaulieu, Rémy-Claude and Violetta Manoukian. "Participatory

Development: A Brief Review of CIDA's Experience and Potential". Policy
Branch, CIDA, February 1994.
Cernea, Michael. Putting People First. Sociological Variables in Rural

Development. World Bank, 1991.

Chambers, Robert. "Paradigm Shifts and the Practice of Participatory

Research and Development, IDS Working Paper #2, England, April 1994.
_____. "Participatory Rural Appraisal (PRA): Analysis of Experience" in

World Development, vol. 22, no.9, pp. 1253-1268, 1994.
_____. "Participatory Rural Appraisal (PRA): Challenges, Potentials and

Paradigm" in World Development, vol. 22, no. 12, pp. 1437-1454, 1994.
_____. Poverty and Livelihoods: Whose Reality Counts?, Institute of

Development Studies, University of Sussex, Development Paper 342,
England, 1995.
_____. Rural Appraisal: Rapid Relaxed and Participatory. Development

Paper #311, Institute of Development Studies, University of Sussex, October
1992.
_____. "The Origins and Practice of Participatory Rural Appraisal" in World

Development, vol. 22, no. 7, pp. 953-969, 1994.
Cohen, John and Norman Uphoff. Rural Development Participation:

Concepts and Measures for Project Design, Implementation and Evaluation.
Rural Development Monograph no.2, Rural Development Committee,
Centre for International Studies, Cornell University, New York, January
1977.
Donnelly-Roark, Paula. "Re-inventing Bureaucracy for Sustainable

Development -Donor Organizations and Participatory Development". New
York, UNDP, November 1993.
Fals-Borda, Orlando and Mohammad Anisur Rahman (eds.). Action and

Knowledge Breaking the Monopoly with Participatory Action Research.
London, Intermediate Technology Publications, 1991.
Glade, William and Charles Reilly. Inquiry at the Grassroots: An Inter-

American Foundation Fellowship Reader. Virginia, 1993.
GTZ. ZOPP: an Introduction to the Method. Frankfurt, Germany, March

1988.
Freire, Paulo. Pedagogy of the Oppressed. The Continuum Publishing

Company, New York, 1993.
International Institute for Environment and Development (IIED). PLA

Notes-Notes on Participatory Learning and Action, IIED, Sustainable
Agriculture Programme, London. Distributed regularly three times a year.
_____. RRA Notes. Special Issues on Applications of Wealth Ranking,

Sustainable Agriculture Programme, United Kingdom, May 1992.
_____. RRA Notes. Special Semi-Special Issue on Participatory Approaches

to HIV/AIDS Programmes, no. 23, June 1995.
Kumar, Krishna. Rapid Appraisal Methods. Washington, DC, International

Bank for Reconstruction and Development, 1993.
Lineberry, William P. Assessing Participatory Development: Rhetoric

Versus Reality. Westview Press, 1989.
Mabey, Christopher and Paul Iles. Managing Learning. Routledge, London,

1994.
Meister, Albert. La participation pour le développement. Paris, Les Editions

ouvriéres, 1977.
Mosse, David. "Authority, Gender and Knowledge: Theoretical Reflections

on the Practice of the Participatory Rural Appraisal". Network Paper no. 44,
Agricultural Administration (Research and Extension) Network, Overseas
Development Institute, London, December 1993.
Narayan, Deepa. The Contribution of People's Participation: Evidence from

121 Rural Water Supply Projects. Environmentally Sustainable
Development Occasional Paper Series no. I, Washington, DC, World Bank,
1995.
Oakley, Peter et al. Projects with People: The Practice of Participation in

Rural
Development. Geneva, ILO.
Overseas Development Administration. Guidance Note on How to Do

Stakeholder Analysis of Aid Projects and Programmes. England, Social
Development Department, ODA, July 1995.
_____. Guidance Note on Indicators for Measuring and Assessing Primary

Stakeholder Participation. England, Social Development Department, ODA,
July 1995.
_____. Note on Enhancing Stakeholder Participation in Aid Activities.

England, Social Development Department, ODA, April 1995.
Rahman, Mohammad Anisur. People's Self-Development. London, Zed

Books, 1993.
Schneider, Hartmut and Marie-Héléne Libercier (eds.). Participatory

Development from Advocacy to Action. Paris, OECD, 1995.
Scoones, Ian. "Investigating Difference: Applications of Wealth Ranking

and Household Survey Approaches among Farming Households in Southern
Zimbabwe". Development and Change, vol. 26 (1995), pp. 67-88.
Smillie, Ian. "NGO Learning, Evaluation and Results: Life in a Three-Ring

Circus". Draft commissioned by the OECD Development Centre, Paris, for a
conference in June/July 1996.
Tripp, Robert. Farmer Participation in Agricultural Research: New

Directions or Old Problems. Institute of Development Studies, University of
Sussex, Discussion Paper no. 256, February 1989.
UNDP. Grassroots Participation: Defining New Realities and

Operationalizing New Strategies. UNDP Discussion Paper.
Uphoff, Norman. "Participation's Place in Rural Development: Seeking

Clarity through Specificity". World Development, vol. 8, pp. 213-235.

Uphoff, Norman, M. L. Wickramasinghe and C. M. Wijayaratna. "Optimum

Participation in Irrigation Management: Issues and Evidence from Sri
Lanka". Human Organization, vol. 49, no. 1, 1990.
World Bank. The World Bank and Participation. Washington, DC, World
Bank Operations Policy Department, September 1994.
Participatory Development and Evaluation manuals
Aaker, Jerry and Jennifer Shumaker. Looking Back and Looking Forward: A
Participatory Approach to Evaluation. New York, Heifer Foundation,
Private Agencies Collaborating Together/PACT, 1994.
American Council of Voluntary Agencies for Foreign Service. Evaluation

Sourcebook. New York, 1983.
Aubel, Judi. Guidelines for Planning and Conducting Studies Using the
Group Interview Technique. Geneva, International Labour 0rganization,
1994.
_____. Participatory Program Evaluation: A Manual for Involving Progra

Stakeholders in the Evaluation Process. Senegal, Catholic Relief Services,
December 1993.
Case, Roland, Mary Andrews and Walter Werner. How Can We Do It? An
Evaluation Training Package for Development Educators. New York,
InterAction American Council for Voluntary International Action, 1988.
CEDPA. Training Trainers for Development. Washington, DC, Centre for

Development and Population Activities, 1995.
Davis-Case, D'Arcy. The Community Toolbox: The Idea, Methods and Tools
for Participatory Assessment, Monitoring and Evaluation in Community
Forestry. Rome, Community Forestry Unit, FAO, 1990.
_____. Participatory Assessment, Monitoring and Evaluation: A Field

Manual. Rome, Community Forestry Unit, FAO, 1989.

FAO. Guidelines for Participatory Nutrition Projects. Rome, FAO, 1993.
_____. Participatory Monitoring and Evaluation. Handbook for Training

Field Workers. Bangkok, Regional Office for Asia and the Pacific, February
1988.
_____. The Community's Toolbox: The Idea, Methods and Tools for
Participatory Assessment, Monitoring and Evaluation in Community
Forestry. Community Forestry Field Manual 2, Rome, 1990.
Gajanayake, Stanley. Community Empowerment: A Participatory Training

Manual on Community Project Development. Office of International
Training and Consultation, Northern Illinois University, PACT Publications,
New York, 1993.
Gosling, Louisa and Mike Edwards. Toolkits. A Practical Guide to

Assessment, Monitoring, Review and Evaluation. Development Manual 5.
London, Save the Children, 1995.
International Institute for Environment and Development. RRA Notes,PLA

Notes. Sustainable Agriculture Programme, London. Published quarterly.
Magnani, David P. (ed). Building Organizational Effectiveness Through
Participation and Teamwork: A Training Manual. New York, PACT

Publications, 1992.
Narayan, Deepa and Lyri Srinivasan. Participatory Development Took Kit.

Washington, DC, World Bank, 1994.
National Environment Secretariat, Government of Kenya; Clark University;

Egerton University; The Centre for International Development and
Environment of the World Resources Institute. Participatory Rural
Appraisal Handbook: Conducting PRAs in Kenya. Natural Resources
Management Support Series no. 1, 1991.
Pretty, Jules. Irene Gujit, Ian Scoones and John Thompson. Participatory

Learning and Action: A Trainer's Guide. London, International Institute for

Environment and Development, 1995.
Srinivasan, Lyra. Options for Educators. A Monograph for Decision-Makers

on Alternative Participatory Strategies. New York, PACT Publications,
1992.
_____. Tools for Community Participation. A Manual for Training Trainers

in Participatory Techniques. Prowess/UNDP Technical Series Involving
Women in Water and Sanitation, 1993.
Audio-Visual Material
PROWESS Video, Audio Plus Video, New Jersey, 1995.
World Vision Australia. The PRA Report: Walking in Their Shoes. Research
and Policy Unit, Melbourne, Australia.
Salmen, Lawrence. Listen to the People, 1987.

Who Are The Question Makers - Undp Participatory Evaluation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Who Are The Question Makers - Undp Participatory Evaluation

Uploaded by

Copyright:

Available Formats

Who are the question-makers?

Who Are the Question-makers?

● Evolution of the Participatory Approach

http://www.undp.org/eo/documents/who.htm (1 of 4) [29/01/2002 18:59:07]

● Farming Systems Research

● Functions of Participatory Evaluation

http://www.undp.org/eo/documents/who.htm (2 of 4) [29/01/2002 18:59:07]

Money and Mambas or Listening to the People. Case

Questions for Group Work

● A Sampling of Tools for a Participatory Evaluation

● Who's Doing What? Organizations Supporting

● Resource Persons, Groups and Institutions

http://www.undp.org/eo/documents/who.htm (3 of 4) [29/01/2002 18:59:07]

http://www.undp.org/eo/documents/who.htm (4 of 4) [29/01/2002 18:59:07]

"Who Are the Question-makers?: A Participatory

This handbook evolved from work undertaken during

While the original intention of undertaking these

http://www.undp.org/eo/documents/wfore.htm (1 of 3) [29/01/2002 18:59:12]

The present handbook has been designed to capture

A few works of explanation will help you to

This handbook has been a collaborative effort by

http://www.undp.org/eo/documents/wfore.htm (2 of 3) [29/01/2002 18:59:12]

Janet Donnelly, Naheed Haq, Peter Hazelwood, Rosein

We hope this handbook proves worthy of its name and

Office of Evaluation and Strategic Planning

http://www.undp.org/eo/documents/wfore.htm (3 of 3) [29/01/2002 18:59:12]

● How can evaluation be a tool for development?

While to some people, the participatory approach may represent a radical

http://www.undp.org/eo/documents/wintro.htm (1 of 4) [29/01/2002 18:59:23]

The growing interest in participatory evaluation parallels the growth of such

Purpose of the Handbook

● provide UNDP staff with a better understanding of what is meant by a

● Resident Representatives and Deputy Resident Representatives, who

http://www.undp.org/eo/documents/wintro.htm (2 of 4) [29/01/2002 18:59:23]

and the approach to be used;

Organization of the Handbook

Parts one to four, which present an overview of the participatory evaluation

● a brief description of the evolution of the participatory approach;

http://www.undp.org/eo/documents/wintro.htm (3 of 4) [29/01/2002 18:59:23]

● Pre-planning, including negotiation of the TOR, assessing the

http://www.undp.org/eo/documents/wintro.htm (4 of 4) [29/01/2002 18:59:23]

What is increasingly being called participatory development began for some

The following pioneers or schools of thought have contributed to the

Participatory Action Research

http://www.undp.org/eo/documents/whop1.htm (1 of 3) [29/01/2002 18:59:47]

Rapid Rural Appraisal and Participatory Learning and Action

Farming Systems Research

http://www.undp.org/eo/documents/whop1.htm (2 of 3) [29/01/2002 18:59:47]

Self-Evaluation and Beneficiary Assessments

As mentioned previously, all of these approaches and schools of thought

http://www.undp.org/eo/documents/whop1.htm (3 of 3) [29/01/2002 18:59:47]

Participatory evaluation is reflective, action-oriented and seeks to build capacity by:

● ?providing stakeholders and beneficiaries with the opportunity to reflect

Functions of Participatory Evaluation

● It helps to build the capacity of stakeholders to reflect, analyse and take

http://www.undp.org/eo/documents/whop2.htm (1 of 6) [29/01/2002 19:00:33]

http://www.undp.org/eo/documents/whop2.htm (2 of 6) [29/01/2002 19:00:33]

Differences between Participatory and More Conventional Evaluations

http://www.undp.org/eo/documents/whop2.htm (3 of 6) [29/01/2002 19:00:33]

LEVELS OF END-USER PARTICIPATION IN EVALUATION

Dimensions of Low Medium High

Evaluation initiator Commissioned or External evaluator Evaluation in which