Paul Clist and Stefan Dercon have proposed12 principles for Payment by Results, which they offer as guidelines

for
whether PbR is the right route. We believe that their principles are predicated on a model of development cooperation
which assumes a predictable relationship between inputs and outcomes (albeit subject to unexpected shocks and
principal-agent constraints). We set out an alternative 12 principles for payment by results in the real world and list our
principles below, alongside the corresponding principles proposed by Clist & Dercon. Although there are significant points
of agreement between our principles and those of Clist & Dercon, a central difference is that in our view PbR offers a
practical way to manage complex realities in development cooperation. (Some of these ideas are elaborated further in
CGDs work on Cash on Delivery Aid.)

Clist and Dercons 12 Principles for PbR 12 Principles for PbR in the Real World
Preamble: Payment by Results (PbR) gets a lot attention,
and could be misunderstood as a silver bullet to provide the
right incentives for higher impact in development. If it is
implemented in the wrong setting, it will lead to poor
programmes that perform worse than alternative contract
designs. We offer 12 principles to ensure that PbR is used
for the right things, in the right places and in the right way. If
the principles here are not met, PbR is not the right route.
Preamble: There is no silver bullet in foreign aid: different
approaches are needed depending on problems and
circumstances, and more experimentation is needed to
find out what works in particular circumstances. Payment
by Results (PbR), like other aid mechanisms, has
attractive characteristics if it is used well in the right
circumstances, but used badly or in the wrong situation it
could be worse than alternatives. We offer 12 principles to
ensure that PbR is used for the right things, in the right
places and in the right way.
1. The promised benefit of PbR lies in the efficiency
improvement that comes from linking aid transfers to
performance against pre-agreed measures. Recipients must
invest first, and will only be compensated if measured results
are achieved. If a large upfront payment is required to get the
programme started, or if the results cannot be
unambiguously measured, then PbR is unlikely to be the
best solution.
1. The promised benefit of PbR lies in the improvement
that comes from giving recipients greater discretion in how
they use and report on the use of aid transfers, and the
shared commitment to outcomes. Linking aid transfers to
pre-agreed measures for outcomes that recipients
themselves want to achieve allows recipients to focus on
experimenting, learning and adapting rather than focusing
on satisfying funders. (There is now somelimited empirical
evidence of a positive correlation between the use of
results-based approaches and greater autonomy for
implementers: it is this autonomy that may lead to
improved outcomes.) But if results cannot be measured
with sufficient precision, or if funders are certain that
particular inputs will generate the desired impact, then
PbR may well be less useful than a conventional aid
program.
2. The principle factor that should determine whether PbR is
used, and the strength of incentives, is the quality of the
performance measure. It is not sufficient for a performance
measure to be correlated with the underlying variable of
interest ex ante, it must also remain so ex post. A good
performance measure is affected by the same mix of actions,
and to the same degree, as the underlying objective. For
example, teacher absenteeism may be correlated (ex ante)
with poor effort and low motivation of teachers, which in turn
lead to poor learning outcomes. Punitive policies to reduce
absenteeism may not increase learning outcomes as they
leave teacher motivation unaffected. Absenteeism would not
be a good performance measure in this case.
2. A key determinant of the success of PbR is the quality
of the performance measure. It is not sufficient for a
performance measure to be correlated with the underlying
variable of interest ex ante, it must also remain so ex post.
Ideally, the performance measure would be the underlying
objective itself. For example, teacher absenteeism may be
correlated (ex ante) with poor effort and low motivation of
teachers, which in turn lead to poor learning outcomes.
Punitive policies to reduce absenteeism may not increase
learning outcomes as they leave teacher motivation
unaffected. Hence it would be better to link payments to
learning outcomes than to teacher absenteeism.
3. Gaming (deliberately improving the performance measure
through otherwise socially-useless activities) receives a lot of
attention as it is morally objectionable. However, distortion
should be of greater concern as it will be more pervasive. For
example, imagine a case where the measure of completion
is used as a performance measure in education. The optimal
mix of policies for achieving the underlying objective
(increasing the numbers of educated children) may place a
large emphasis on increasing teacher quality and a low
emphasis on stopping excessive repetition. The optimal mix
to improve measured performance may be the same basic
policies, but with the opposite emphases. The use of this
performance measure may lead to an efficiency loss by over-
rewarding action on repetition and under-rewarding action on
teacher quality, distorting incentives.
3. Whenever aid is linked to an input, indicator or output,
there is a possibility of gaming (deliberately improving the
performance measure through otherwise socially-useless
activities) or creation of distorted incentives.
Conventional aid, which is typically focused on inputs, can
also generate a large amount of inefficient behaviour,
whether because of gaming or perverse incentives. In
general, the scope for such distortions is likely to be
reduced by identifying performance measures as far as
possible along the results chain towards measuring the
desired impact. If donors are able to link aid to the
underlying objective (for example, learning outcomes), this
has less scope for distortions and gaming than linking aid
to inputs or outputs. Hence PbR linked to outcomes tends
to reduce the risk of perverse incentives compared to
input-based aid by aligning the incentives of implementers
(organisations and individuals) more closely to the shared
ultimate objectives.
4. Risk transfer is only relevant as the mechanism to
sharpen performance incentives for the implementing
agency. Risk sharing itself is not a rationale for PbR, for two
reasons. First, a bilateral donor will have a widely diversified
portfolio of projects, and will thus be less risk averse than
any implementing agency/recipient country. The greater an
agents risk aversion the less effort she will exert for a given
promised payment, resulting in a decrease in efficiency.
Second, any risk transferred to the recipient will compound
risk, not mitigate it. For example, imagine an incentivised
measure of immunisation rates for a country which is hit by
excessively heavy rains that affect crop yield and transport.
The recipient government may have a) invested heavily in an
immunisation program, b) be adversely affected
economically through misfortune and c) receive no
compensation. Both greater risk aversion and positively
correlated risks suggest a suboptimal level of agent effort in
a PbR contract.
4. Risk transfer itself is not a rationale for PbR. Bilateral
donors have a widely diversified portfolio of projects, and
should act as if they are risk neutral. (This is why donor
agencies do not insure themselves against the failure of
aid projects.) There are many types of risk, and the
allocation of risk matters because it affects incentives for
performance. Donor capriciousness is a risk that donors
control and so they should continue to bear the
consequences. Donors should also not transfer the risk
that their project design preferences might fail to deliver
positive impact. It may make sense to transfer to
implementing partners those delivery risks about which
they have more information than funders and which they
are best able to manage. Although risk sharing is not a
goal of PbR in itself, an appropriate allocation of risks can
improve effectiveness by transferring responsibilities for
risks to the partner best able to monitor and mitigate them.
5. Economic theory states that in the case of full alignment of
objectives (the donor and recipient want the same thing) the
promised benefit of PbR is foregone as there is no additional
effort towards the donors priorities: if we truly want the same
thing, performance incentives are irrelevant. The additional
costs of PbR (relating to verification, risk aversion and costs
of control) are still incurred. From an empirical perspective, it
is an open question whether PbR is more successful in
cases of high or low misalignment, as it is possible that
countries with relatively low misalignment are most receptive
to slight changes in priorities. In other words, little is known in
practice about how alignment relates to the marginal benefit
of measures-based policy pressure but being clear about the
objectives of the recipient/implementer is important.
5. In the case of full alignment of objectives (the donor
and recipient want the same thing), PbR may still be
appropriate - not because it better aligns incentives, which
would not be necessary in this case, but because it can
create more flexibility for experimentation, adaptation and
learning, and because it can force a closer attention to
definition and verification of results. If there is a
misalignment of objectives then neither PbR nor a
conventional aid programme is likely to succeed (there is a
lot of evidence that aid projects only succeed where there
is genuine ownership). Even so, if there is such a
misalignment, then PbR has two potential advantages
over a conventional aid project: first, the misalignment may
be revealed more quickly because it would be more
difficult to negotiate a PbR contract than it would be to
paper over the differences with a conventional aid
programme; and second, if a project were agreed, the
costs of the likely failure would be borne by the
implementing partner, not the donor.
6. PbR is only appropriate where there is low observability
of effort. If effort can be observed (e.g. inputs), these can be
contracted upon directly. Where monitoring of actions/inputs
is difficult or costly PbR has an advantage, otherwise other
contracts are likely to be better.
6. PbR is appropriate where (a) there is uncertainty
about the combination or quality of inputs that is
needed to deliver the required outcomes in the context;
and/or (b) there is low observability of effort or
inputs. If there is a known combination of inputs to deliver
the final objective and if the administration of those inputs
can be fully observed, then alternative approaches may be
appropriate. Very often, a process of experimentation,
including failure, learning and adaptation is necessary and
desirable to find the most effective input mix. In these
cases, PbR contracts may be more effective than
conventional aid programmes.
7. PbR is more appropriate where recipients have a larger
degree of control over the outcomes (a high signal-to-
noise ratio). In cases of a low signal-to-noise ratio, where
random variation or other actors are more important,
recipients are more likely to be punished for misfortune and
rewarded for good luck. This will in turn dampen their
incentives to exert effort. In highly risky environments (where
implementers have low control), PbR would be very costly as
a large prize would be needed to incentivise effort.
7. PbR may be most effective, relative to other
instruments, where there is more uncertainty about the
relationship between the activities and the intended
outcomes, such as where the intervention is affected by
other factors outside the control of implementers, and in
fragile states. In these circumstances all aid projects are
more risky and uncertain than in relatively predictable
situations - this is true for PbR as it is for other aid
modalities. Butthere is some evidence that the failure rate
of projects is higher when uncertainty is greater if the
implementing agents have little autonomy to react to
changing circumstances and new information. If PbR
permits greater autonomy for implementing agents, then
PbR may be preferred over other aid instruments in risky
and uncertain environments or for uncertain projects in
stable environments.
8. The most tangible cost of PbR relative to other forms of
aid is the cost of verifi cation, which may be offset by a
reduction in transaction costs and management time. Both
are likely to be dwarfed in magnitude by other considerations
(see point 3). Results measured in PbR contracts would
need to be extremely hard to dispute, audited and stand up
in court.
8. The most tangible cost of PbR relative to other forms of
aid is the cost of verifi cation. It is an advantage of PbR
that the existence of an explicit contract forces all
stakeholders to agree and implement a robust,
independent mechanism to measure results - which aid
projects typically fail to do and, as a result, miss
opportunities for learning and improvement. By contrast,
detailed tracking of inputs is very costly and often ends up
with little or no evidence of the results that programmes
have achieved. The focus on results need not be more
expensive in terms of staff or money than the detailed
tracking of inputs which it replaces, and because it focuses
on outcomes it may provide much more useful information.
9. Microeconomic research shows that extrinsic motivation
(monetary fines or rewards) rarely simply augments intrinsic
motivation. Most worryingly, if intrinsic motivation is eroded
by fines/rewards, there is evidence that it does not return
once the extrinsic motivation is removed. At present, there is
little relevant empirical evidence in the macro PbR setting,
but we should be wary of settings where fines and rewards
may undermine the personal commitment of implementers to
make a success of the project.
9. Microeconomic research shows that extrinsic
moti vation (monetary fines or rewards) rarely simply
augments intrinsic motivation. We should be wary of
settings where fines and rewards may undermine the
personal commitment of implementers to make a success
of the project. This applies to PbR programs that pay
individuals, particularly when payments are big relative to
a small organizations budget. Such evidence, however,
has little bearing on how payments affect the behaviour of
large organizations, subnational authorities or
governments. If there is a genuine concern that
incentivising implementation staff lowers intrinsic effort, the
contracting party (e.g., governments) can simply choose
not to pass all or some of the financial incentives on to
implementation staff. Whether and how to use financial
incentives for implementing staff is one is likely to be best
decided by managers on the ground, not aid agency
policy-makers.
10. A pragmatic acceptance of donor limitations is useful. If
the donor can only contract over a relatively short time
horizon (say, under 5 years), the control of the recipient may
be low. The possible distortions mentioned in point 3 may
become binding in the case of short time horizons, as the
best actions may only pay-off over a longer time horizon than
the donor is able to commit to. If the donor cannot withhold
aid in the case of non-performance, the incentive effect is
lost: non-payment has to be credible.
10. Some donors may suffer from limitations that may
make PbR less effective. If the donor can only contract
over a short time horizon (say, under 5 years), it may not
be possible for the recipient to achieve and demonstrate
results within the timespan for which the donor can
commit. If the donor cannot withhold aid in the case of
non-performance, the incentive effect is lost: non-payment
has to be credible. But donor constraints also make
conventional aid programmes less effective: for example,
there are significant efficiency costs from uncertain and
volatile grant programmes, which are the result of
constraints on donors from making long term
commitments. By entering into long-term contracts which
pay only when results are proven, PbR may allow donors
to escape from some of the limitations which reduce the
effectiveness of conventional aid.
11. Any evidence of success in PbR that is measured only
in terms of the incentivised performance measure should be
treated with caution. For example, an apparently successful
programme that only reports reductions in teacher
absenteeism may have had no effect on learning outcomes.
Other outcomes should be measured to gauge program
performance.
11. Evidence of success of all aid programmes - whether
PbR or conventional aid - should be framed in terms of the
desired impact. Evidence that points only to intermediate
measures (inputs, activities, outputs) should be treated
with caution. For example, an apparently successful
programme that only reports teachers trained or
reductions in teacher absenteeism may have had no effect
on learning outcomes. Similarly, an apparently successful
program to improve a learning outcome like literacy may
cause a negative externality of lower maths scores
through the reallocation of teacher time. However, this
criticism applies equally to both PbR- and input-based aid,
and does not make a case for or against one approach or
the other. Instead, an advantage of well-designed PbR
programmes over conventional aid is that they will, by
definition, include evidence of impact on desired
outcomes. Even where an outcome has been achieved,
evidence of the success of the program should ideally also
document how the process worked, and ways it may or
may not have been successful beyond changes in the
performance measure: did the recipient innovate to
achieve outcomes? Was it more or less burdensome than
alternative approaches? Were there other effects, such as
improved data quality as a result of the verification
process? Projects funded by PbR may facilitate this form
of learning because they are, by design, rigorous and
transparent about whether- and to what degree- projects
succeed, providing information about variation in
outcomes that may be traceable to variation in inputs and
methods.
12. There are other forms of aid, beyond PbR, that could
also offer performance incentives. Examples are prizes or
competitions which combine possible efficiency
improvements with an element of peer learning. For
example, a competition amongst NGOs to increase bed net
use would publicise the strategies of the winning NGO.
12. There are many forms of aid that could offer
performance incentives. PbR itself encompasses a wide
range of designs, some of which are more like cost-plus
contracts, while others are more like prizes or
competitions. For example, DFID's RBA program to
Ethiopia pays a bonus for each additional secondary
student but that still represents less than 1% of its annual
education budget. In this way, it is more like a subsidy or
prize than a cost-contract. Projects that are not normally
described as PbR, such as Advance Market
Commitments, prizes or competitions, may combine
possible efficiency improvements with an element of peer
learning. For example, a competition amongst NGOs to
increase bednet use would publicise the strategies of the
winning NGO. Where working capital is not available, a
development impact bond is a PbR approach that would
provide pre-financing from private investors, who are paid
back on the basis of results and have a stake in whether
the program is successful.