From Event to Process: Current Trends in Microfinance

Impact Assessment
By Anton Simanowitz1

Abstract
This article gives an overview of the recent developments in microfinance impact assessments. It starts by
reviewing the motivation for impact assessment in microfinance, and the increasing interest of the sector in
doing impact assessment. Particularly important is the lack of credibility given to the notion that client
repayment and retention and organisational financial performance are sufficient proxies for impact.

The article describes the move away from donor-led impact events, towards more practitioner-focused
processes, and the experience of individual organisations and international projects, most notably the
AIMS project, in developing impact assessment that is more responsive to practitioner needs. Practitioner-
focused impact assessment looks at how impact information can feed into management and product design
processes, and provide frequent and timely information. Impact assessment thus provides information that
allows MFIs to improve their services, and thus improve the impact on their clients.

A lot of challenges remain, particularly in terms of ensuring a diversity of approaches and applications for
IA, and the inclusion of a range of stakeholders in defining what should be included in the impact
assessment process, and in the analysis and use of results. Clients in particular have a greater role to play
in this process. These challenges are the focus of a Ford Foundation sponsored action-research
programme, Imp-Act.

Microfinance programmes are one of the most important interventions in developing

country efforts to reduce poverty. Recent years have seen a huge growth of the sector in
terms of numbers and size of organisations, numbers of clients and provision of
subsidised donor funding. A large proportion of Microfinance Institutions (MFIs)
include poverty reduction in their Mission, and donor funding is allocated to
microfinance on this basis. At the most basic level there is a need to understand and
improve the impact of MFIs as a key premise to successful poverty reduction.

There have been considerable recent developments in the field of impact assessment,
particularly in relation to microfinance. These developments relate to the rationale and
conceptualisation of impact assessment, the approaches used and whose needs these
meet, and the actual tools and methodologies adopted. In the microfinance industry there
are three key questions relating to impact assessment that have concerned much of the
recent impact assessment work.

1) Why should impact assessment be done at all? Is this not an inefficient use of
resources where market proxies can be used to determine whether clients are
benefiting from the services provided by the MFI?

1
Anton Simanowitz is Programme Coordinator of Imp-Act (Improving Impact of Microfinance on Poverty;
an Action Research Programme). Further information is available from www.Imp-Act.org or email
a.simanowitz@ids.ac.uk
This paper draws on several papers produced by the Imp-Act programme, including a programme concept
note; the programme proposal submitted to the Ford foundation; and a background paper prepared for a
virtual meeting on impact assessment methodologies. I am grateful to Susan Johnson, Anna Portisch and an
anonymous review for comments on an earlier draft, but views expressed in the paper are mine alone.

1
2) Can impact really be attributed to the work of the MFI? With the complexity of
clients’ livelihoods and the external environment, it is very difficult to prove impact,
and the methods that can be used to this end are time-consuming, costly and
complex. These factors, combined with the perception that adequate market proxies
exist, have led to a strong lobby against the need to do impact assessment.

3) Whose needs are met by impact assessment? In the past impact assessment has
primarily met donor needs for proving impact and effective use of resources. Is it
possible to design impact assessment that better meets practitioner needs for
improved understanding of their clients and how to improve the MFI’s services so as
to improve the impact that they are having? What are the needs of other stakeholders
such as clients?

Considerable debate and work has taken place on these questions, particularly through
the work of the AIMS project, the Microcredit Summit and a number of individuals and
MFIs. Drawing on practical examples from a number of organisations, this paper gives
some background to these questions, looks at how thinking and practical work has
developed, the current challenges, and outlines the future work of the Imp-Act
programme which is exploring a range of possibilities, approaches and uses for impact
assessment.

Why is Impact Assessment necessary?

The debate about whether impact assessment is necessary or not centres on the conviction
that the market can provide adequate proxies for impact, and that impact assessment is
therefore an inefficient use of resources. Based on economic theory of utility, it is argued
that if clients are willing to pay for a service ie. client retention and repayment rates are
good, then it can be assumed that they are happy to pay for this service because it is
doing them good. If this measure is combined with organisational financial performance
it is argued that these two indicators are effective in telling us that we have a strong,
efficient organisation that is providing a service that is needed and has a positive impact.
For example, Rich Rosenburg, a senior advisor to CGAP2 makes the following point:

“if your investee institutions [the MFIs] are pricing their services in a way which covers
ALL of the costs of providing them...and if their clients continue to use these services,
then you have strong evidence from the persons most likely to know that the clients are
deriving benefits whose value exceeds the cost of providing them. Do you really need to
know a lot more than that?” (Rosenburg, 1997)

It is clear from experience, however, that although rational, this argument fails in practice
where a number of factors other than economic utility lead to seemingly “irrational”
decisions by clients. Market proxies mask the range of client responses and benefits to
the financial services. Indeed, the complexity of impact assessment stems from the
numerous inter-related factors that lead to client decisions and actions, and affect the
2
Consultative Group to Assist the Poorest, a consortium of 29 bilateral and multilateral donor agencies who
support microfinance. CGAP’s mission is “ to improve the capacity of microfinance institutions to deliver
flexible, high-quality financial services to the very poor on a sustainable basis”.

2
effectiveness of the organisation in achieving positive impacts. The author’s experience
in Kenya for example, demonstrates the power of the solidarity group methodology to
create a situation where women were prepared, in the midst of near starvation, to sell
food in order to repay their loan.

In 1994 I was working for an International NGO in a semi-arid region of Kenya. The area was
experiencing its third consecutive crop failure as a result of drought, and the community was
reaching crisis situation. Livestock were dying, people were leaving the area in search of work,
malnutrition and hunger were rife, and coping strategies such as selling livestock or assets or
seeking paid work were ineffective as the market prices fell and the loss of income for better-off
families meant that they could no longer afford to employ casual workers.

One afternoon I attended a poverty-focussed group meeting attended by about 10 women, selected
by the NGO using wealth ranking as being from the poorest households in the village. The
meeting lasted about an hour with discussion centring on the crisis and how people were coping
(or not). Energy levels were low and some of the women fell asleep from time to time. One of the
women apologised for this explaining that none of them had eaten that day!

As the meeting drew to a close I witnessed a bizarre scene, as the women started digging in their
pockets and taking out money which they handed over to the NGO field worker. He diligently
recorded the amounts in his ledger, thanked them and walked over to his motorbike, ready to
depart. I was dumbfounded and asked if I could ask a couple of questions. I asked about the
handing over of money, and the field worker explained that these were the loan repayment
instalments due from the group for their solidarity group loans. The women explained that they
had sold a chicken in order to raise the funds to make their loan repayment.

The reasons for continued repayment in this case are not clear. Perhaps it was due to their
perception of future benefits of maintaining a good credit record with the NGO, perhaps
it was the fear of loosing face. As well as demonstrating the weakness of market proxies
for impact, this experience reveals the importance for programme that are working with
very poor and vulnerable clients to be sensitive particularly to the potential negative
impacts of their intervention. Thus the programme must know that it is having a positive
impact, or at the very least that it is not having a negative impact on some people.

Can impact be proven? Methodological issues in impact assessment

A considerable amount of debate in microfinance impact assessment has centred on the

question of attribution, or whether it is possible to demonstrate that it is the particular
intervention of the MFI that has led to a specific change. Client livelihoods and the
communities in which they live are complex. Clients may have multiple income sources,
and the credit provided by the MFI is fungible and is not necessarily used for the purpose
for which it was requested. Attributing impact to the MFI is thus difficult and complex,
particularly given the difficulty of establishing effective control-group mechanisms
which can establish a counter-factual (ie. what would have happened without the MFI
intervention). This complexity creates high costs, and leads many people to question the
cost-effectiveness of this type of study.

“[Impact Assessment studies] are fraught with insurmountable methodological problems

and the costs of doing them usually exceed any benefits they might provide” (Dale
Adams, 2001)

3
At the core of many donor-led impact assessments has been the need understand how
effective microfinance is in lifting people out of poverty, and to compare the relative
costs/benefits of microfinance compared to other poverty alleviation interventions. A
number of large scale, costly studies have been carried out. However, the complexity of
these studies and the range of variables included have often have resulted in inconclusive
results. For example, an impact assessment survey in 1994 by Pitt and Khandkar of
several MFIs in Bangladesh reputedly cost US$0.75 million 3, and produced results which
are challenged by other academics (Hulme, 2000; Kabeer 1998). Although the survey
and the quantitative analysis are seen as sound, it is the basis of understanding around
issues and the conclusions that are then made is questioned.

“The problem with the Pitt and Khandkar study is the problem with much of this genre of
neoclassical analysis: it is strong on highly sophisticated econometric modelling but weak
on the kind of contextual information necessary to interpret their findings” (Kabeer, p16).

The survey produced a huge amount of data on a range of variables. Quantitative

analysis of this data reveals patterns, but value judgements must then be used to make
conclusions. Kabeer compares the analysis of empowerment and gender given in this
study and several others looking at the impact of microcredit in Bangladesh. She is
critical not necessarily of the content of the conclusions, but that the effects reported are
presumed rather than demonstrated. Kabeer shows through her analysis how the
conclusions differ primarily according to the researchers stand-point and frame of
analysis and therefore interpretation of data, rather than observed differences in the
clients studied.

This methodological debate highlights the subjective nature of impact assessment and the
need for in-depth qualitative understanding on which reasonable conclusions can be
made. It also highlights the problem in impact assessment where all research is open to
differing interpretation and criticism.

Making Impact assessment useful: meeting the needs of different stakeholders

At the core of these debates is a question of producing credible results for what purpose.
David Hulme (2000) presents a useful dichotomy of objectives - “proving” and
“improving”. Where it is important to prove impact in order to make resource allocation
decisions, then there is a need for rigorous results that can demonstrate causality. For
MFIs that are primarily concerned with improving their practice, it is still important to
understand impact and the relationship of the organisation and its services to its clients,
but the level of rigour is likely to be much lower, although it should still be credible.

The proving/improving dichotomy masks a complex range of possibilities and differing

needs of an MFI’s stakeholders, and the effect of the relative power of these stakeholders
in determining the priorities and objectives of an impact assessment. These objectives
then define the approach and the tools used in the assessment (see figure one). For
3
Paul Mosley, personal correspondence

4
 example, where there is a need to provide conclusive evidence for donors or policy
makers the IA objectives will be more towards the “proving” end of the scale, and the
approach taken is likely to be externally led and biased towards collecting quantitative
data. Where the main stakeholder is the MFI itself and the objectives tend towards
looking at how to improve practice and develop internal learning systems, the process is
likely to be more inclusive and participatory, with much greater involvement of staff (and
perhaps clients) in defining the IA process and analysing the data gathered.

Figure 1. Approaches to Impact Assessment

Institution profile Objectives of the study

 Mission
 Structure and Organization
 Methodology
Institution needs
 Current problems and needs
 Strategic plans
Context
 Microfinance reglementation
Why we do it and for whom ?
Assessment approach
 Improving methodology /
management  Who do it? ( externally led /
 Internal learning
internally led )
 How? ( participatory/qualitative /
 Internal proving
survey)
 External proving and Advocacy
 How often ? (ad hoc /
continuous )

 Poverty characteristics Findings application

 Socio- economic situtation and  Mission and objectives Tools used
political stability
 Financial Methodology
What mix of tools to use?
 Management
 Quantitative
 Marketing
 Qualitative
 Institutional and geographical structure
cccsssstructure  PRA/PLA
 Impact, outreach and sustainability

Most MFIs seek to meet the needs of multiple stakeholders and to fulfil a number of
different objectives. It is therefore apparent that the “proving” and “improving” agendas
are not mutually exclusive, and most impact assessments will necessarily include a mix
of approaches.

As the impact agenda has opened up to incorporating the needs of a greater range of
stakeholders it has become clear that the impact research can be used for a wide range of
purposes (see figure two). The processes of understanding clients’ livelihoods and their
relationship to the MFI, the needs of clients in terms of their businesses and in reducing
vulnerability and poverty, and the organisational services and structures needed to deliver
them, have a large number of potential benefits. Impact assessment that is more focused
on the needs of practitioners has moved towards looking at how impact information can

5
feed into management and product design processes, and provide frequent and timely
information. Impact assessment thus provides information that allows MFIs to improve
their services, and thus improve the impact on their clients.
Figure 2. Usefulness of Impact Assessment to different stakeholders
Stakeholder
Donors
Donor/MFI staff & board
MFI staff and board
MFI staff and board; clients
MFI management –
product development &
methodology
MFI management –
Staff and product delivery
management
Clients
Usefulness
Allow funders to validate their investments
 Quantitative information about the size of the changes taking
place
Demonstrate meeting Mission and Objectives
 Improve understanding of the causes of changes and the
relation between changes and the work of the MFI
Monitor meeting of Mission and Objectives
 Provide managers with information about performance in terms
of organisational mission and impact
 What are the general directions of changes taking place?
 Are changes different amongst MFI clients compared to the
wider community?
 What impacts or changes are taking place in relationships
between clients and other household members, other MFI
clients, or other community members?
Organisational learning & improving practice
 Improve understanding of individual clients, households,
businesses, and their communities and the inter-relationships
between the different levels
 Improve MFI’s understanding of differing needs of different
sections of the community eg. the very poor.
Provide market research information to improve
products and services
 Improve MFI’s responsiveness to clients’ expressed and
underlying needs
 Provide information which helps managers understand what
services their target clients need
Management tool to manage & improve impact
 Provide managers with information about staff performance
 Improve understanding of client livelihoods and processes of
change which are taking place
Provide accountability to clients
 Clients’ input into improving MFI services so as to better meet
their needs.
Self-analysis, learning and improved practice
 Increase ability to understand and analyse their own situation
6
Practitioner-led IA – What can MFIs do today?

In the development NGO sector internationally there has be an increasing recognition of

the weakness of institutional learning and accountability and “a growing gap between the
rhetoric of agencies and the reality of what they achieve” (Roche, p2). This has led to a
growing pressure on organisations to demonstrate that they are in fact achieving
significant results and impact. At the same time there is interest amongst practitioners in
developing products and services that are more sensitive to clients needs, and recognition
of the need to conduct research that informs these processes. This understanding is
necessary to deal with perceived challenges such as increasing drop-out rates and
competition in many organisations.

Practitioner-focussed impact assessment values internal organisational learning and

product development processes. This has developed in response to the inappropriateness
of many donor-driven studies in providing this type of information to managers. At a
recent workshop at the Asian Regional Microcredit Summit, for example, one
organisation described how they are currently implementing two parallel and separate
impact assessment systems – one a study being conducted by their donors, the other an
internal learning process for their own needs.

New approaches to impact assessment have been developed both within individual MFIs
and NGOs (ActionAid, Simanowitz, Naponen, Hossain, Cheston and Reed), and through
internationally coordinated impact assessment work (Roche, AIMS). Practitioner-
focussed impact assessment emphasises the needs of MFIs rather than those of donors,
and is more integrated into existing work patterns, organisational learning, and tries to
build on existing knowledge and experience, and produce results that can be easily used
by management.

Practitioner-focused impact assessment takes the MFI and its clients and their needs as its
starting point. Its supplements existing information systems to provide additional
information that leads to improved understanding of clients’ constraints and their reasons
for success or failure, or for leaving the programme. This information is directly relevant
to operational design issues, and can feed into the development of products or
improvement of services in a way that not only increases client impact but improves MFI
performance.

For example, many MFIs routinely collect data about their clients on entry which could
effectively be used as base-line information for later impact assessment. However, it is
often not collected in a methodical way, nor is it stored in a way that it can be easily
utilised. Many MFIs therefore do not use this data in the most effective way possible and
find that they do not have adequate base-line information when they do start looking at
measuring their impact. CARD, a Grameen replicant in the Philippines, for example,
collects base-line information about client assets and income as part of their client

7
selection process. This is then followed up at the end of the third loan to track changes in
clients and their businesses, and make basic impact conclusions.

“However, difficulties in retrieving baseline data (destroyed by floods, misplaced when a

branch had to move to another office) compelled the Research Unit to rely on client
recall. CARD recognizes the importance of establishing baseline data on its area of
operation (i.e, number of family members, level of income, type of house, etc.) and
securing the data for future impact assessment. It is taking steps to ensure that branch
baseline data are stored properly in its head office computers.” (CARD, p1)

A first step, then, for all MFIs is to review the basic client information that is collected on
entry and how this information is stored and used. If the information collected is related
to a conceptual understanding of intended programme impacts, then it can be carefully
selected to provide useful impact information. For example, issues such as separating
information about men and women or boys and girls can be very important to future
impact analysis, but can easily be overlooked if the information is not collected with
impact assessment in mind.

A number of MFIs have developed internal impact assessment and monitoring systems,
or initiated impact assessment studies4. It has been the work of USAID’s AIMS project 5,
however, that has been key in creating recognition of the importance of impact
assessment in microfinance, and the usefulness of a practitioner focus. Launched in
1995, AIMS has been influential in changing both perceptions and practice of
microfinance impact assessment.

“At its inception…USAID’s AIMS Project was a minority voice defending the value of
impact assessment in microfinance…Five years later, we are pleased to report a marked
shift in attention to and interest in understanding the actual impact that microfinance
programs are having on clients.” (SEEP, 2000, pi)

In addition to concrete tools and methodological outputs AIMS has demonstrated that
impact assessment can be conducted in a cost-effective way that, in part, meets the needs
of practitioners and provides timely and useful information. In demonstrating that it can
be done, AIMS has also re-emphasised the value in using scarce resources for impact
assessment.

For MFIs to take a more active role in the impact assessment process it is important for
them both to understand the key questions and decisions that need to be made in the
design and implementation process, and to be able to be active in the implementation and
analysis stages. It is this active involvement that leads to learning at an individual client
or staff member, and organisational level. To reach this stage it has been necessary to
think both about the methodological issues involved in impact assessment, so that people
can make informed and justified choices, as well as developing simple tools that can be

4
A list of examples is given in Cheston and Reed
5
The Assessing the Impact of Microenterprise Services Project is funded by USAID’s Office of
Microenterprise Development

8
implemented and analysed by practitioners. Impact assessment needs to be based on a
sound conceptual framework that can be used for developing hypotheses about possible
impact channels, and as a framework for analysis and understanding. Particularly useful
is a livelihoods analysis that helps contextualise the specific interventions of a
development agency in a broader understanding of poverty. AIMS, for example,
developed a conceptual framework termed the “Household Economic Portfolio model”
(Chen and Dunn) which highlights the importance of focusing on the client and her
business in the context of the household, and outlines the different levels at which impact
may occur.

This type of analysis can be useful for example in analysing why clients leave. A client-
exit survey will give “market” information about client’s preferences and expressed
reasons for leaving. Contextualising this study in a deeper understanding will help to
reveal underlying reasons for exit and allow conclusions to be made about impact. For
example, the common response clients give of “resting” might relate to much more
complex issues of intra-household dynamics, or to cyclical or seasonal patterns of income
and expenditure in the household which mean that the need for credit varies through the
year.

AIMS, through a number of background papers and a series of virtual meetings addressed
a range of conceptual and methodological questions, and provides a sound basis for
thinking through the design and implementation of impact assessment 6. These have been
captured in a set of guidelines for microfinance impact assessment (Barnes and Sebstad,
2000). The project also developed, in collaboration with the US based SEEP 7 network, a
set of practitioner impact assessment tools. These are tools that which can be used by
MFIs themselves to track and assess the impact of their programmes. Resulting from this
process has been the development of a tools manual of five assessment tools for
practitioners (SEEP, 2000). The manual gives practical step-by-step guidance which
provides a sound starting point for MFIs to gain a deeper understanding of their clients
and changes in their lives and attitudes towards the services of the MFI.

4. Challenges for the Future – Improving Impact Assessment

The work of the AIMS project and others has had significant impact in terms of raising
the profile of impact assessment in the microfinance industry, and providing the
mechanisms by which it can be done. The opening up of the impact assessment agenda
to include the needs of a range of stakeholders has firmly established the value of
conducting credible impact assessment. A lot of work has gone into developing useful
conceptual frameworks for impact assessment, and in thinking through many of the
methodological problems that in the past have made impact assessment so difficult and
6
Issues addressed include: programme characteristics; assets; diversification; economic, policy and
regulatory environment; income; diversification; debt; measurement of profit and networth; use of control
groups; fungibility; attribution; quasi-experimental design; hypothesis development. All the AIMS
publications can be downloaded from their website www.mip.org
7
Small Enterprise Education and Promotion Network. An association of more than 50 North American
private and voluntary organisations which support micro and small enterprise programmes in the
developing world.

9
costly. Low cost approaches are now available to practitioners that can be implemented
in a timely and non-disruptive way.

Challenges remain in a number of areas, relating particularly to the flexible application of

impact assessment to suit a range or organisational types and objectives, and its
institutionalisation in on-going systems rather than one-off assessments. These issues
form the central questions for Imp-Act8, a three-year programme of action-research
designed to improve the quality of microfinance services and their impact on poverty
through strengthening the development of impact assessment systems.

1) Flexible use of impact assessment

A significant amount has been achieved in improving understanding of impact

assessment pathways, and providing a set of tools and approaches. There is danger,
however, that these tools will be applied blindly and this will not serve the needs of MFIs
or donors, or the needs of the poor. The greatest danger is conformity. No two impact
assessments should be the same unless organisations and the contexts in which they work
are exactly the same. It is too easy to follow good practice that has gone before, and to
replicate the application of impact assessment tools without going through the necessary
process which leads to the decision of how to approach a particular task, and what tools
to select to fulfil our aims. Each organisation should go through a process of thinking
about what approach it wishes to take given its own context and objectives. This will
lead to the selection of a unique mix of tools, methodologies and stakeholders (see figure
one).

There is a need to design impact assessment to suit a plurality of stakeholders, a plurality

of outcomes and purposes, and a plurality of methodologies for doing IA. The challenge
is to ensure that IA is used and applied in appropriate ways. As part of this there is also a
need to more clearly define the range of objectives to be fulfilled and questions to be
answered through impact assessment, and the range of possibilities of approaches and
tools available to meet these objectives. It is important to examine how to apply and
adapt tools and methodologies that have been already been developed, and to use them in
different contexts and for a range of objectives.

2) Institutionalising IA – client monitoring and systems

8
The programme is funded by the Ford Foundation and jointly implemented by a team from three UK
Universities (Bath, IDS and Sheffield). More information is available from the website www.Imp-Act.org.
Imp-Act is a collaboration between some 21 MFIs in five regions on four continents, creating a set of inter-
linking projects, collectively addressing the major challenges and issues facing the microfinance industry in
achieving improved impact on poverty. It is designed to investigate a plurality of approaches with
organisations working in different contexts, with different objectives and using a range of methodologies.
At the core of the programme are individual impact assessment projects run by each MFI according to their
own needs and priorities. From this experience lessons and insights will be shared and synthesised and
guidelines for better practice in impact assessment developed. Imp-Act is explicitly concerned with the
poverty impacts of microfinance, and will contribute to a deeper understanding of the role of microfinance
in securing long term improvements in the livelihoods of the poor.

10
Whilst a lot of work has been done in terms of recognising practitioners interests and
concerns in the impact assessment process, relatively little has been achieved in
integrating impact assessment into existing organisational systems, as part of an on-going
internal learning process.

Microfinance impact assessment is moving away from narrow, donor-focused impact

assessment events, towards more practitioner-focused processes. The challenge is how to
institutionalise IA and develop systems which are on-going and not just one-off studies.
In order for this to take place organisations need to look at their existing systems and
identify opportunities for integrating impact information, without disrupting core
activities, and within resource and time constraints. Information must feedback quickly
and allow management to take operational and strategic decisions. Two key issues for
the practical application of these type of impact questions are whether the benefits
created by organisational learning and improved practice outweigh the costs of
developing and implementing an impact assessment system, and the organisational
capacity and skills that need to be developed.

Impact monitoring and assessment can provide useful information about clients, their
needs and their perceptions. These can assist management of impact and improvement of
products and methodologies. In this realm IA has many similarities with market research,
and much of the information collected can be used to inform market and product
development type questions. Conversely, market research (MR) tools, when applied as
part of an impact assessment system can provide useful information that will support the
information needs of impact assessment. There may well be opportunities for building
upon existing market research processes to provide impact information as well.
However, it is important to maintain a clarity in terms of the differences between impact
assessment and market research in that impact assessment seeks to look at longer term
changes and transformations in the clients and their livelihoods, highlight unintended
outcomes of microfinance interventions, and understand the underlying processes that
determine client choice and “wants”.

Traditional impact assessment studies tend to be one off events, and are narrow in their
focus and objectives. We have seen, however, that impact assessment has been
interpreted in different ways, often much more broadly. If MFIs are to meet a wider
range of objectives then they need to think more in terms of impact assessment systems
rather than one-off studies. It is useful to look at how to combine a range of tools, to
triangulate information, increasing the cost-effectiveness of the work, and enabling
information to be provided for a number of different objectives. Such a system, through
triangulation and provision of longitudinal data can increase the rigour of the IA, and
may provide information that contributes to credible impact results that can be used to
inform donors or for advocacy.

The characteristics of a system may include:

 Using a range of different approaches of tools – mixture of quantitative and

qualitative;

11
 Relying on triangulation of results from different sources to increase the credibility
and reliability of information;

 Including a range of objectives for the impact assessment eg. a focus on improving
practices, but with a secondary objective of providing credible quantitative
information for donors (or vice-versa);

 Thinking about impact assessment as a long-term on-going process, rather than as a

one off event.

A practical starting point for most organisations would be to examine where they are
starting from: what information is currently being collected, for example on initial client
information forms, loan applications, and in the management information system; how
can this existing information be built on, and made more rigorous or comprehensive?;
how can data collection be built into existing work patterns so as not to increase the
burden on staff (even externally run surveys demand considerable staff time in setting up
interviews, helping with sample selection etc.)?; what external sources of information are
useful, for example from Government statistics or University research?

It is necessary to be aware of the costs and benefits of different methods, and accept that
trade-offs have to be made to create something that is manageable and cost-effective.
Each methodology has its advantages and disadvantages and therefore informed choices
need to be made about what approach will generate the information needed in the most
cost-effective way, whilst fitting with the objectives of the impact assessment. For
example, SEF in South Africa, due to its primary objective of understanding processes
and improving its intervention, chose not to use a formal control group. Instead more
resources were devoted to detailed qualitative household interviews which generated
more useful information; this was triangulated by longitudinal data from impact
monitoring (Simanowitz, 1999).

3) Including clients

Although there has been increasing recognition of the need to be more practitioner-led in
impact assessment, few organisations have translated their commitment to being “client-
driven” into their impact assessment processes. Impact assessment can be an important
tool in terms of developing greater accountability to clients. Client involvement in
determining IA process, indicators and analysis can also contribute to the success of IA
work that really understands changes in client livelihoods, and feeds learning into
improved organisational practices and methodology. It is an issue of client influence of
programme design and the analysis of organisational success. Impact assessment that
focuses on client needs has the potential to involve clients in a process of reviewing their
goals and activities, and strengthening their capacity to define the services that they need
from the MFO and more effectively utilise these services to strengthen their livelihoods
and businesses.

12
Participatory methods do exist to facilitate this type of inclusion 9, but are generally
viewed as being too time consuming and expensive to be used by MFIs. However,
participatory approaches can have a significant role in impact assessment in terms of
cost-effectively gathering information. They can also contribute to the process of making
MFIs more sensitive to client needs, more empowering for clients, and ultimately
increasing their effectiveness in terms of impact. One of the objectives of Imp-Act is to
explore ways of involving clients to a greater extent in the IA process, and looking at the
trade-offs and benefits in terms of costs and benefits.

4) Levels of impact

The conceptual framework used by the AIMS project the emphasised need to look at
individual, intra-household, household and business levels. Community-level and wider
sector, regional or national level impacts were not addressed. Whilst MFIs clearly cannot
hope to answer all questions themselves, they do need to assess their role in these broader
relationships, and modify their activities in order to maximise their impact.

Impact assessment is more than just the monitoring of performance outcomes. There are
broader and longer term questions relating to the direct and indirect impacts of
microfinance services on clients and non-clients, their communities and the local and
national economies of the countries in which they live. For example, whilst gender
impacts are often seen as integral to these broader impact assessment questions, it is
important that they are explicitly raised. The question of wider impacts for example links
to the recognition of the “constraints that the wider society imposes in terms of norms of
behaviour, legal rights and perceptions of the value of what women do.” (Johnson, p2).
In terms of the MFI and its methodology the impact assessment process can help in
thinking through how gender factors may influence the operations of the MFIs, its client
selection biases, and its impacts. In the design and implementation of impact assessment
processes there are a number of points that need to be considered to ensure that the data is
open to gendered analysis and credible. These relate for example, to the establishment of
a gender baseline, the type of information collected and indicators used, the approaches
and methods used to collect data, and the way in which data is collected (Johnson, p10-
11).

CONCLUSION
Impact assessment has come a long way from the days when studies were commissioned
by donors, implemented by consultants, and provided little of use for MFIs other than a
pretty report on a shelf. A number of methodological challenges remain, and debates
continue amongst academics as to whether it is possible to prove impact, or if market
proxies are sufficient. Practitioners, however, are recognising the value of increasing
their understanding of their clients in terms of designing better programmes which are
more effective in reaching the organisational mission, at retaining clients, and improving
operational efficiency. Impact assessment studies, impact monitoring, client portfolio
monitoring and market research combine to give managers useful and timely information
9
See discussion on participatory impact assessment in Simanowitz et. al (2000)

13
that can be used to improve their understanding of how the MFI delivers its service and
how this relates to the client needs, and to develop and improve services and products so
as to become a more effective and efficient organisation.

A number of tools and methodologies have been developed and used by MFIs around the
world, and these combined with manuals produced by AIMS and others give a sound
basis for MFIs supported by their own choice of consultants or academics to develop and
implement impact assessment studies or on-going systems. Given the wide range of
contexts, objectives, stakeholders, skills and resources of MFIs it is essential that each
MFI thinks through the impact assessment process for itself. The experience of others
should provide guidance, but never a blue-print.

14
References

ActionAid (1999); ActionAid Impact Assessment Workshop; November 30th – December

4th, 1998, Isle of Thorns, Sussex

Adams, Dale (2001); Debt impact manual; posting to the Dev-Finance email discussion
list, May 1, 2001

Barnes, Carolyn and Sebstad, Jennefer (2000); Guidelines for Microfinance Impact
Assessments; Discussion Paper for the CGAP 3 Virtual Meeting, October 18-19,
1999); USAID AIMS Project

CARD (2000); Center for Agriculture and Rural Development’s Experience on Impact
Assesment: A Note for the Workshop The Ford Foundation Assessment Project, June 22-
24, 2000, IDS, IDS, University of Sussex, UK

Chen, Martha Alter and Dunn, Elizabeth (1996); Household Economic

Portfolios; Submitted to Monique Cohen, USAID AIMS Project

Cheston, Suzy and Reed, Larry (1999); Measuring Transformation: Assessing and
Improving the Impact of Microcredit; paper produced for the Microcredit Summit
Meeting of Councils

Cohen, Monique (2001); The AIMS Tools; posting to the Dev-Finance email discussion
list, May 5, 2001

Copestake, James (2001); Towards a general framework for client monitoring within
microfinance organisations; Imp-Act, UK

Copestake, J, Johnson, S, McGregor, A, Greeley, M, Gaventa, J, Blauert, J, Mosley, P

(1999); Ford Foundation Action-Research Programme Into Assessing The Impact Of
Development Finance: Concept Paper; Imp-Act website www.Imp-Act.org

Dunn, Elizabeth (2001); The Aims of AIMS; posting to the Dev-Finance email discussion
list, May 11, 2001

Dunford, Chris (1998); Exchanging Value for Value, unpublished paper, Freedom from
Hunger, US

Hulme, David (2000); Impact Assessment Methodologies for Microfinance: Theory,

Experience and Better Practice; World Development, Vol. 28, No. 1

Hussain, A.M.Muazzam (2001); Methodological Aspects of Impact Assessment : BRAC’s

Experiences; presentation to Imp-Act Asia Region Workshop on: Micro-finance and
Impact Assessment Methodology

15
Imp-Act (2001); Improving Impact of Microfinance on Poverty: Proposal for three year
Action-Research Programme, Submitted to the Ford Foundation by Centre for
Development Studies, University of Bath, Department of Economics, University of
Sheffield, Institute of Development Studies, University of Sussex, March 2001

Johnson, Susan (2000); Gender impact assessment in microfinance and microenterprise:

why and how; Development in Practice, Vol. 10, No.1, February 2000

Kabeer, Naila (1998); ‘Money Can’t Buy Me Love’? Re-evaluating Gender, Credit and
Empowerment in Rural Bangladesh; IDS Discussion Paper no. 363

Pitt, M and Khandkar, S (1996); Household and intrahousehold impact of the

Grameen Bank and similar targeted credit programs in Bangladesh; World Bank
discussion paper no. 320, Washington, D.C., USA

Rosenburg, R (1997). Impact monitoring system; Posting to development finance email

discussion group, 21 February

SEEP Network (2000); Learning from Clients: Assessment Tools for Microfinance
Practitioners; AIMS

Simanowitz, Anton (1999); Understanding Impact: Experiences and Lessons from the
Small Enterprise Foundation’s Poverty-Alleviation Programme, Tšhomišano; Paper
submitted to the Third CGAP Virtual Meeting on Impact Assessment Methodologies,
October 1999

Simanowitz, Anton et. al (2000); How can IA be made more participatory?; background
thematic paper, Imp-Act www.Imp-Act.org

Simanowitz, Anton (2001); Virtual Meeting on Impact Assessment Methodologies:

Background Paper; Imp-Act www.Imp-Act.org, April 2001

Woller, Gary (2001); AIMS Manual; posting to the Dev-Finance email discussion list,
May 5, 2001

16