Platform Synthesis Paper - Donor Methods To Prioritise Investments in Agricultural Research and Development

Donor Methods to Prioritise Investments in Agricultural Research and Development
About the Platform Synthesis Paper

With increasing global demand for nutritious and safe food, it is widely recognized that agricultural research needs additional resources in order to achieve the necessary improvements in agricultural productivity. Donors have to split their limited development resources between research and non-research activities. But which criteria do donors use when they allocate their budgets? This Platform synthesis paper presents a set of tools and methods that are presently used to prioritise the selection of research projects. Members of the Global Donor Platform for Rural Development will benefit from this summary of academic work, learn from case studies of how priorities have been assessed and gain knowledge on how to articulate these processes into existing programming mechanisms. Furthermore, the comparative analysis of various donor approaches is beneficial and can help to promote collaboration and mutual learning. On the basis of the synthesis paper a Platform policy brief will be published that will summarize the main strategies and recommendations for decisionmakers.
This paper is available electronically and can be downloaded from the website of the Global Donor Platform for Rural Development at: www.donorplatform.org/resources/publications Secretariat of the Global Donor Platform for Rural Development, Godesberger Allee 119, 53175 Bonn, Germany Email: secretariat@donorplatform.org The views expressed herein are those of the authors and do not necessarily represent those of individual Platform members. All rights reserved. Reproduction and dissemination of material in this information product for educational or other non-commercial purposes is authorised, without any prior written permission from the copyright holders, provided the source is fully acknowledged. Reproduction of material in this information product for resale or other commercial purposes is prohibited without written permission of the copyright holders. Applications for such permission should be addressed to: Coordinator, Secretariat of the Global Donor Platform for Rural Development, Godesberger Allee 119, 53175 Bonn, Germany, or via email to: secretariat@donorplatform.org. Global Donor Platform for Rural Development 2012
Synthesis Paper Donor methods to prioritise investments in agricultural research and development
01
Contents
FROM THE PLATFORM AR4D WORKING GROUP ACKNOWLEDGEMENTS ACRONYMS EXECUTIVE SUMMARY 1.0 Introduction 2.0 Setting the context of agricultural research for agricultural development 2.1 Investment allocations research versus alternatives 2.2 Theory of decisionmaking under uncertainty 2.3 Summary and implications 3.0 Agricultural research priority setting methods 3.1 3.2 3.3 3.4 The goals of donors Defining the relevant alternatives Projecting impacts of research Comparing alternatives and establishing priorities 3.5 How to use priority setting methods 4.0 Donor consultations 4.1 Which priority setting mechanisms are currently used? 4.2 How donors evaluate priority setting mechanisms according to their ease of use, provision of reliable evidence and persuasiveness in informing decisionmaking? 4.3 Donor perspectives on the analytic methods available in the literature 4.4 Strategies donors employ to inform their practice and learn from each other 4.5 Multi-donor engagement 02 02 03 04 08 5.1 Types of institutions used by donors to invest in agricultural research 5.2 Mechanisms used to invest in agricultural research 5.3 Pros and cons of competitive versus targeted funding 5.4 What comes first: the choice of institutional partner or the choice of research topic? 6.0 Conclusions and recommendations 11 7.0 References 11 12 APPENDIX 1_ Formal approaches for determining the amount to invest in research versus alternative interventions under risk APPENDIX 2_ Description of quantitative methods for assessing impacts of agricultural research APPENDIX 3_ Terms of Reference 15 18 20 AbOUT US 32 24 26 27 5.0 Mechanisms for investments in agricultural research 24
09
28 29
34
12 12 13 14
37 42 45
20
21
22
23 24
02
From the Platform AR4D working group

Agricultural research is essential to achieve the transformative change in agricultural productivity and dietary diversity needed to meet future global demand for nutritious and safe food. Faced with the potential of limited growth in development assistance budgets, it will become increasingly important to improve the effectiveness of investments in agricultural research, including how to prioritise research investments and to compare investments in research with other nonresearch investments. Allocation of development resources among research and non-research activities is better informed when donors have information on norms and best practices to achieve sustainable development outcomes. Though immediate humanitarian outcomes and development goals might be well served with a 100 per cent allocation to non-research programming, long-term sustainability of agricultural production and market systems requires research to address emerging global challenges and apply innovations to local context. Donors allocate resources to agricultural research for development through different funding channels and to achieve a range of objectives. This support helps to build a strong agricultural research and innovation system across all elements of the research and development value chain from research infrastructure, capacity building, training, knowledge generation, dissemination, technology transfer, and extension to research policy, planning and administrative management activities. Where donor support is focused on agricultural research, there is a need to prioritise the allocation of research investments. The private sector offers some useful models for allocating research resources in the context of other business activities. Similarly, public sector agriculture institutions have developed models for optimizing allocations to research versus other activities, and to optimize allocations among research areas to achieve stated goals. The objective of this synthesis paper is to inform Platform members and other relevant organisations about approaches, methods and tools to assess and prioritise research investments. It presents an academic review as well as case studies on how priorities have been assessed and provides knowledge on how to articulate these processes into existing programming mechanisms. The comparative analysis of donor methods to allocate investments in agricultural research is beneficial to present the range of possible approaches and identify strengths and gaps among the greater donor community. Greater collaboration among donors may help to build and promote the use of some objective methods and technical tools for mutual benefit.
Acknowledgement
The Platform wants to thank the authors George Norton and Jeffrey Alwang of Virginia Tech's Department of Agricultural and Applied Economics and the following organisations and individuals for responding to the authors requests for interviews: Gregory Traxler of the Bill and Melinda Gates Foundation, Andr Fabian of BMZ, Nikita Eriksen-Hamel of CIDA, Alan Tollervey of DFID, David Radcliffe of the EC, Wolfgang Kasten of GIZ, Karim Hussein of the OECD and Sahara Moon Chapotin and Jennifer Long of USAID. Comments on an earlier draft are also gratefully acknowledged from David Radcliffe of the EC, Nikita Eriksen-Hamel of CIDA, Clara Cohen of USAID, Gary Jahn and Max Rothschild of USAID and David Nielson of the World Bank and Ute Hbner, who also coordinated.
03 01
Acronyms
AECF AR4D ARD ASARECA BEAF BMZ CG or CGIAR CIDA CRP CRSP DALY DEVCO DFID DG DREAM EC EIARD EIQ EU FARA FGT FtF Gates GCARD GDP GMO GIZ IFAD IFPRI NARS NGO NPV ODA OECD R&D RCT SPIA USAID WFP Africa Enterprise Challenge Fund Agricultural Research for Development Agriculture and Rural Development Association for Strengthening Agricultural Research in Eastern and Central Africa German Advisory Service on Agricultural Research for Development Federal Ministry of Economic Cooperation and Development Germany Consultative Group on International Agricultural Research Canadian International Development Agency CGIAR Research Program Collaborative Research Support Program Disability-Adjusted Life Years Development and Cooperation EuropeAid Department for International Development Directorate General Dynamic Research EvaluAtion for Management European Commission European Initiative for Agricultural Research for Development Environmental Impact Quotient European Union Forum for Agricultural Research in Africa Foster-Greer-Thorbecke Feed the Future Bill and Melinda Gates Foundation Global Conference on Agricultural Research for Development Gross Domestic Product Genetically Modified Organism German International Cooperation International Fund for Agricultural Development International Food Policy Research Institute National Agricultural Research System Nongovernmental Organisation Net Present Value Official Development Assistance Organization for Economic Cooperation and Development Research and Development Randomized Control Trial Standing Panel on Impact Assessment United States Agency for International Development World Food Program
04
Executive summary
One objective of the agriculture research for development working group of the Global Donor Platform for Rural Development (Platform) is to improve knowledge of and harmonize donor methods to prioritise investments in agricultural research and development. This report describes: The context in which investments are selected in agricultural research Reviews methods for agricultural research prioritisation Describes mechanisms donors use to programme agricultural research resources Suggests how donors might adjust their prioritisation strategies to improve coordination and potentially increase impacts of their portfolios The decision to fund agricultural research versus other interventions can be guided by: The need to evaluate donor investments in terms of their expected contributions to goals The comparative advantage of agricultural research compared to other interventions The order of the decision process The complementarity of investments
Review of agricultural research priority setting methods

Agricultural research priority setting by donors involves: Identifying and prioritising donor goals Defining the research alternatives to be prioritised Projecting impacts or contributions (to donor goals) of research on the alternativesconsidering the degree of risk and uncertainty associated with the impacts Comparing the alternatives and establishing priorities once impacts are estimated (projected) These four elements are included in research priority setting regardless of specific methods employed although donors differ in the way they address them.
Goals
An array of donor goals is used in agricultural research priority setting. Agricultural research can be expected to contribute to some goals more than others and differs in its ability to achieve them as compared to the ability of other interventions. Meeting the goal of improved productivity, income and food security also contributes to other goals such as improved nutrition and health or sustainability of the natural resource base.
Research alternatives
Differences in the time periods for investments The riskiness of investments Opinions of key stakeholders are important in defining the appropriate researchable alternatives. Information on what is possible to achieve through research should draw on appropriate, often multidisciplinary, scientific expertise. A participatory structured process that identifies key researchable problems and the potential contributions of different disciplines to them is essential.
05 01
Impact assessment
A key part of research prioritisation is projecting the impacts that can be achieved through investment in different types of research. Quantitative assessments are preferred where confidence can be placed in the results, but the level of confidence differs depending on the nature of the research and the types of impacts. In some cases donor research budgets are largely aimed at strengthening national and international research institutions. Improvements in productivity or efficiency are among the easiest impacts to project quantitatively. Methods for projecting agricultural research impacts on nutritional and human health wellbeing and on the natural resource environment are also available, but due to the multifaceted nature of nutrition, health and the environment, less confidence can be placed in the results. Impact assessment can absorb significant resourcesso choices must be made on the level of rigour and resources to devote to assessing potential impacts of alternative research themes. For each donor goal, a few key factors determine the expected contributions of a research project toward achieving the goal. For the goal of improving agricultural productivity, income growth and food security, key factors include the current value of production, projected impacts of the research on yields and costs, the odds of success in the research, the likely adoption of the technologies and the timing of the benefits to be received. For the nutrition/health goal, the size of the target group, the incidence of the problem, the mortality rate or degree of disability associated with the problem deficiency and the projected effects of the research on reducing the problem drive research benefits. The impact of research on quantities produced and consumed and on prices paid affect nutrient consumption. Thus an economic assessment can also be a starting point for a health/nutrition assessmentthe foods that make up the largest portion of the consumption of malnourished people are those that also may have the biggest impact on nutrition
Key factors influencing poverty effects of research are: The number of poor The depth of their poverty The income provided by the research results or technology The adoption of the technology by the poor Food price changes affecting consumers The environment is multidimensional but a practical method for assessing environmental sustainability is to assess the physical effects of the research on specific environmental categories. Availability of data to support environmental impact assessment varies by category of environmental impact and the data are often rough. Donors must consider the extent to which the research should be a public rather than a private responsibility. Another factor is the extent to which other donors are supporting research on a topic and the comparative advantage of the donor on the research themes.
Comparing alternatives
Tradeoffs must be considered with respect to research contributions to different objectives. The potential impacts of a research portfolio that emphasizes one objective versus another can be assessed and then how much the contribution of the portfolio to one objective would be reduced if the other objective is emphasized to differing degrees. To address the benefits of specific research options under varying levels of funding, research benefits can be projected under a high, medium and low level of funding for each alternative. Research that is already in the pipeline must be compared to new research.
06
Donor consultations
Informal consultations were held with representatives from seven donor organisations to discuss their current priority setting mechanisms. Representatives were interviewed from BMZ, GIZ, CIDA, DFID, EC, OECD, USAID and the Gates Foundation. Most donors reported using informal processes to set priorities because they lacked capacity, resources, or staff time to conduct formal, technical priority setting exercises. Some donors have at times engaged in formal priority setting but not on a regular basis. Donors indicated that the provision of reliable evidence was their most important criterion in evaluating priority setting mechanisms. Several interviewees hinted that easierto-use methods might lead to more widespread formal priority setting. Donors had a favourable impression of analytical methods but none use them on a regular basis. Some stated that increased sharing of experience among donors and formal collaboration with other donors might improve their ability to utilize these methods. Few have in-house capability to conduct the exercises. Some were concerned that the evidence basein favour of agricultural researchwas weak for non-efficiency objectives. They felt that much of the academic priority setting literature focused too closely on efficiency, while many politicians currently favoured non-efficiency objectives. They are concerned about tradeoffs between objectives and would like means of evaluating these tradeoffs when setting research priorities.
Reasons for this dependence on the CG include: Confidence in the quality and relevance of CG-led research Close correspondence between CG goals and donor goals The ability of CG centers and CRPs to link with other actors including NARS and university researchers The ease of contracting with the CG and its ability to conduct large-scale, multidisciplinary research. Donors are also increasingly partnering with applied research entities who can deliver research results to end users
Recommendations
Most donors employ informal, semistructured processes to set agricultural research priorities and lack capacity, resources or time to conduct formal, technical priority setting exercises. However, the following recommendations may help improve the priority setting process.
Recommendation 1 Systematize the prioritisation process

A five-step sequence of practices can be useful for setting priorities for most donors: 1. Identify goals
Mechanisms for investments in agricultural research

Donors engage a variety of institutions to conduct agricultural research for development, including the CGIAR system, universities, government agencies, NGOs, private research organisations and others. While the distribution of resources among institution type varies from donor to donor, it is clear that the bulk of research resources pass through the CGIAR system
2. Specify potential alternatives (topics or institutions) to be prioritised 3. Project contributions of research topics (institutions) to the goals 4. Consider tradeoffs associated with alternative priorities 5. Compare priorities to the current portfolio of topics or institutional investments and vet any changes against political acceptability Projecting contributions of research to specific goals can be accomplished by applying a subset of impact assessment tools, can make use of the results from metaanalyses of previous research on the topics, or can make use of indicators or theory. A formal prioritisation process is not completed each year, but should be undertaken when a new strategic plan is developed or at least every five years. Adjustments to priorities can be made more
07 01
frequently and targeted impact assessments can be undertaken to inform those changes.
Recommendation 5 Use quantitative tools where practical and credible

For the goal of improving agricultural productivity, income growth and food security, economic surplus and benefit cost analyses are time-tested methods that can be combined with other methods to provide donors with projected benefits and tradeoffs. Application of these tools can be externally contracted for during priority setting exercises. For the nutrition/health goal, the size of the target group, the incidence of the problem in the group, the mortality rate or degree of disability associated with the problem deficiency in each group and the projected effects of the research on reducing the problem are key factors determining research benefits. Key factors influencing poverty effects of research on producers are the number of poor, the depth of their poverty, the income provided by the research results or technology and the adoption of the technology by the poor. Factors affecting poverty effects on consumers are the relative importance of the commodity in question in the expenditures of the poor and the expected fall in market prices due to the research. For assessing environmental sustainability, rely heavily on projected physical effects of the research across specific environmental categories. For major environmental topics such as climate change, information from global models can be used to predict the location of heat stress, flooding and other effects.
Recommendation 2 Contract with appropriate groups to project impacts

Most donors lack the internal capacity to directly apply quantitative priority setting tools. However, donors could individually, or acting as a group, contract with certain research groups to apply the tools and supply analyses of estimated impacts. Some impact assessment groups already have models and some of the data.
Recommendation 3 Insist the CGIAR do more priority setting

Because so many donor resources are directed at the CG system, donors might insist that the CG system do a more formal prioritisation itself of proposed research topics. This prioritisation process might be led by the Standing Panel on Impact Assessment (SPIA) and SPIA could contract out as well with appropriate groups as necessary.
Recommendation 4 Make use of accepted theory and available information

Even without quantitative tools, donor priority setting processes could make additional use of available information and theory. Income and productivity effects of research are driven to a large extent by: Base value of production associated with a commodity Expected per unit cost and yield changes if the research is successful The likelihood of research success Projected adoption rate of the results and their timing Poverty is highly geographic and for producers affected by the value of the products produced and by the risk. The urban poor are affected significantly by the price of the food products they consume most. Poor women especially are affected by low food prices. Improvement in one goal can mean sacrifice in another and the tradeoff can be quantified if quantitative tools are used when evaluating impacts, or at least discussed if only qualitative methods are used.
Recommendation 6 Ensure that priority setting methods help lead to tangible results in farmers fields
To ensure that priority setting methods lead to results in farmers fields: Donors can fund a mix of long- and short-term researchso that short-term impacts can be realized without sacrificing long-term gains Coordinate with in-country offices as they prefer to support technology transfer programmes rather than research programmes Obtain input from field offices, NARS, NGOs and other local experts to identify the most pressing researchable problems Insist that research groups funded have a plan for diffusing results Include social scientists in research programmes who study constraints to adoption
08
1.0 Introduction
About the Platform
The Global Donor Platform for Rural Development (Platform) is a network of 34 bilateral and multilateral donors and international financing institutions that share a common vision of the role that agriculture, rural development and food security (ARD&FS) plays in reducing poverty. They are committed to achieving increased and more effective aid for ARD&FS.
Methodology
To meet these objectives: Literature on agricultural research priority setting was briefly reviewed Representatives from several donors that support agricultural research were interviewed Recommendations are made on strategies to improve donor coordination of agricultural research priority setting
About the AR4D working group

One of the objectives of the agriculture research and development working group of the Platform is to improve knowledge and harmonization of donor methods to prioritise investments in agricultural research and development.
Objectives of this synthesis paper

The primary objectives of this paper are to: Demonstrate the context in which investments are selected in agricultural research for development Describe methods for agricultural research prioritisation Describe the mechanisms donors use to programme agricultural research resources Suggest how donors can use prioritisation methods, strategies to improve coordination and information from the priority setting literature Serve as input to the Global Conference on Agricultural and Rural Development in October 2012
09 01
2.0 Setting the context of agricultural research for agricultural development

Agricultural research is an investment in the production of knowledge to achieve donor goals. The investment must compete with non-research activities for scarce resources. Like any investment, research involves a time dimension and a foregoing of current consumption or alternative investments. Though immediate humanitarian outcomes and development goals might be well served with a 100 per cent allocation to non-research programming, long-term sustainability of agricultural productionespecially in a context of highly uncertain future economic, social, climatic conditions, market systems and attainment of development goals, requires research investments. Research and non-research activities differ in their efficiency in achieving specific donor goals. Allocation of development resources among research and non-research activities is best informed when donors are aware of accepted norms and best practices for choosing among alternatives to achieve desired development outcomes in the face of uncertainty. In some research areas the private sector may offer useful approaches for allocating resources for R&D versus other business activities. Similarly, public sector agricultural institutions have developed mechanisms for optimizing allocations to research versus other activities. Either process would lead to the same resource allocation. However, researchand most otherinvestments are characterized by uncertainties, returns to scale in the research process, long lags between investments and usable research outputs, presence of fixed assets and imperfect information. Returns to research investments also depend on other actions; for example, returns to researchno matter how these returns are definedare higher when research is accompanied by investments in infrastructure such as roads and irrigation. This complementarity of alternative investments complicates valuation of the investments and further reinforces the notion that the order of the decision process matters.
box 1. Investment allocation in an optimal world Because the sequence of decisionmaking decision making matters, efficient (least cost of attaining objectives) investment allocation should be in this sequence: 1. The decisionmakers prioritises their own goals 2. Information is obtained on the relative efficiency of alternative investments (R&D, operational, policy, etc.) in meeting the goals. This efficiency depends on historical and current investments among all alternatives 3. Resources are allocated across the alternative investments
2.1 Investment allocations research versus alternatives

It is important to consider the sequence of decisionmaking when examining investments in research compared to other investments. Under complete information, a straightforward research process, perfect markets and no risk, it would not matter whether decisionmakers first decided how to allocate investments between (overall) research and others, or, alternatively, first decided to allocate investments to meet specific goals. That is, the decisionmaker could begin by deciding to allocate a share of the total budgeted resources to research, or could first decide to allocate to research a share of resources devoted to meeting each of its objectives.
If the process described in box 1 is followed, the total amount of resources allocated to research is an outcome of the goal-prioritisation process. In practice, however, the research resource budget is usually set without a thorough analysis of the relative efficiency of research versus other investments in meeting goals. The main reason is the complexity, cost and time required to conduct a thorough analysis of options. Given the relatively rapid pace of political decisionmaking, a complete analysis of returns to alternative investment portfolios is rarely possible. Thus, although several approaches to making decisions across multiple investments are found in the literature, they are in practice not widely used in either the private or public sectors (boxes 2 and 3).
10
box 2. Example of private sector process to determine share of total resources to devote to research The pharmaceutical industry is highly dependent on research resultsR&D as a percentage of sales is the highest of all industries. Large pharmaceutical companies such as Astra-Zeneca make decisions based on historical allocations to R&D versus other investments, on industry norms or what their competitors are doing, and on the research pipeline that is, the resources needed to continue development of promising lines and other ongoing approved research directions. R&D investment decisions are made at the most senior level during periodic reviews (quarterly). Budgets are determined based on how many projects are ongoing and in what stage. Earlier projects require less funding than those in later stages. Companies rarely use project allocation-mix portfolio models or other decision theoretical models to decide these questions.
box 3. Example of processes used by two large scale donors to determine share of total resources to devote to research Large-scale donors to international agricultural research such as Gates and DFID utilize relatively simple decisionmaking approaches when deciding on the overall resource envelope for research compared to alternative investments: Gates began with a research/project mix that considered the relative need for research, its contribution toward development goals, and the ability of the research complex to absorb funding. None of these factors were explicitly quantified in the process. The initial allocation to research was approximately 60 per cent of the agricultural portfolio. As it adjusts funding, Gates does not specifically define a ratio of research expenditures to other investments, but marginal changes to investment allocations are made based on new challenges and opportunities, promising lines of research and absorptive capability. Currently the mix is approximately 70 per cent, and while adjustments are made based on new information and needs, formal decision-theoretic models are not applied. At DFID, the total envelope of resources for research is set at a very high institutional level and this envelope depends on political and other considerations. Once the overall research resource envelope is set, agriculture must compete with other sectors such as health for research resources. At this stage, evidence about the efficacy of alternative research lines in meeting political goals is considered. Formal decision-theory modeling is rarely used, but decisionmakers are presented evidence from literature and other studies about expected impacts of alternative research themes. Current priorities include child health and malnutrition, and decisionmakers note that it is important to demonstrate the effectiveness of agricultural research in meeting these priorities. Priorities themselves change relatively frequently and empirical information on how different broad research themes affect different pathways of impact is needed to inform these decisions.
11 01
2.2 Theory of decisionmaking under uncertainty

Formal models of decisionmaking about investments focus on the relative contribution of specific investments such as research toward meeting firm/donor objectives. In most cases, the chief objective of private firms is profit, while donors often have multiple objectives (discussed below). Other private firm objectives depend on the nature of the industry but may include the ability to obtain strategic advantage in key markets, goodwill and the public perception of the firm and others. Literature on research-related decisionmaking focuses on optimal allocation of resources to R&D where optimality is defined relative to firm/institutional objectives. When multiple objectives are considered, additional tools may be required. The literature stresses that in addition to expected returns, an important factor affecting decisions about long-term investments such as research is risk. In the absence of risk, the appropriate conceptual approach to investment decisionmaking would be to maximize the discounted return among alternative investments. There are five basic types of risk (box 4).
Several important approaches can be used for longterm investment planning under risk. The most common approaches are: Discounted cash flow/net present value, perhaps combined with decision tree analysis Decision analysis Real options (Details in appendix 1) In brief, the traditional approach to valuing uncertain R&D projects is a discounted cash flow-based approach. This method produces an expected net present value (NPV) reflecting the expected discounted returns minus discounted costs. Key criticisms of an NPV approach are: It focuses on a single stream of benefits and costs It assumes that at the start of the project, decisionmakers are completely committed to the project A single stream of benefits and costs is inappropriate because risks differ by investment and different risks imply different cash flows. The assumption of a complete commitment is also inappropriateas a decision may be made to terminate a research project after an initial stage and before significant costs are incurred. Decision analysis can deal with multiple scenarios and incorporate flexibility in decisions. Decision analysis structures choices according to formal rules that flow from decision theory and utility analysis but it requires decisionmakers to be able to articulate the relative importance of different outcomes to their goals so a model can be created. Real option analysis provides a theoretically consistent means of valuing risk and including this value when evaluating investment options. For example, following investment in agricultural variety research, the decisionmaker can decide whether to proceed with licensing, multiplication and distributionsuch pathways are real options. The value of the option arises because there is no obligation to exercise itand therefore the option holder does not suffer a higher loss from increased volatility when the underlying asset loses value. The main challenge with applying a real options approach is that historical recordsand hence measures of volatilitydo not exist for many investment options.
box 4. Types of research-related risk or uncertainty Five types of research-related risks or uncertainty have been identified: Uncertainty about payoffs or the expected contribution of a specific research investment to firm/donor goals Fluctuations in research cost Riskiness of alternative projects Riskiness associated with market acceptance of a particular innovation or research product Risks implicit with long time horizons such as political uncertainty and others. Some of these risks are endemic to any investment and some are specific to R&D Source: Huchzermeier and Loch 2001
12
2.3 Summary and implications

The information required to rigorously apply decision theory approaches to decisions about investments in research versus other investments is substantial. As a result, even private-sector firms that are extremely reliant on R&D for firm viability do not generally apply complex models to this decision. Donors, whether public or private, frequently resort to adhoc decision processes. These processes are increasingly informed by evidence relative to the cost-effectiveness of alternatives, but the underlying decision process differs from the theoretical ideal. Despite these findings, several important insights can be gleaned from the literature on investing in research as compared to alternative interventions: Goals must be set and investments evaluated in terms of their expected contributions to these goals The comparative advantage of agricultural research compared to alternative investments should be consideredthere may be more cost-effective mechanisms than research for attaining specific goals The order of the decision process mattersdecisions that allocate resources to a research envelope first likely differ from those that allocate resources by sector first Investments within and between sectors can be highly complementaryand this complementarity must be considered for optimal decisionmaking Historical processes are importantresearch investments take a long time to gestate and decisions relative to alternative investments are complicated by differing gestation periods Risk is an extremely important considerationand upside risk is especially important The timing of information flows is a strong determinant of the relative desirability of optionsthe earlier information emerges, the more valuable it is
3.0 Agricultural research priority setting methods

Agricultural research priority setting by donors involves several elements: Identifying and prioritising donor goals Defining the alternatives such as research themes, options, institutional investments to be prioritised Projecting impacts or contributions (to donor goals) of research on the alternatives, including the degree of risk and uncertainty associated with the impacts Comparing the alternatives and establishing priorities once impacts are estimated or projected These four elements are included in research priority setting regardless of specific methods employed, although donors may differ in the methods they use to address them. This section describes available methods, including information, time and expertise required. It describes the methods currently used by a sample of donors to set priorities. Finally, it addresses the issue of methods for avoiding disproportionately high donor investments in some areas and underinvestment in others.
3.1 The goals of donors

Donors use an array of goals in their agricultural research priority setting. These goals relate to major development objectives and may include: Improved agricultural productivity, income growth and food security in developing countries Distributional effects in terms of poverty reduction, geographic emphasis and improvements in womens or childrens nutrition and health Environmental sustainability Benefits that revert back to the donor country
13 01
The latter reflects a need to maintain political support for foreign assistance and a desire to sustain donor leverage on the recipient country or organisation. These goals reflect the overall goals of the donor agencyalthough agricultural research can be expected to contribute to some goals more than others based on its ability to achieve them as compared to the ability of other instruments. Contributions to achieving a goal such as improved productivity, income and food security can also contribute to achieving other goals such as improved nutrition and health or sustainability of the natural resource base. For example, research on staples consumed by many poor and nutritionally deprived people may increase productivity but will also lower poverty and lessen malnutrition due to higher incomes and lower food prices. This overlap can reduce the precision of predictions from any priority setting method. In addition to development goals, donor agencies have preferences for the level or scale of impacts such as household, regions within countries, national, regional and global. For example, one donor has prioritised a set of countries as well as regions within those countries. Preferences for regional or global scale impacts reflect the desirability for producing international public goods with high potential for geographic spilloversas compared to benefiting particular countries or regions. Methods for estimating research impacts that primarily affect a narrowly targeted level may differ from those that spill over broadly across levels or geographic areas. Donor research priorities set at a global level may have to be reconciled with priorities set at country or regional levels. Donors may also have preferences for length of time to impacts. Just as poor people place greater emphasis on the present than people who are not living as close to the margin, so too do development agencies. The poor strongly prefer the present because of their immediate needs. Donors respond not only to the needs of the poor but to the reality that governments prefer impacts sooner rather than later, given political cycles. Benefits received sooner may also be more certain than those in the future. Length of run is an important issue in agricultural research priority setting because most research takes timeoften more than five yearsand requires patience, while strategic planning cycles often run five years or fewer. Donors may also express preferences across disciplines but generally speaking disciplinary choice should reflect research requirements for achieving specific development goals rather than being considered a goal itself. In some cases observed preferences for specific disciplines may reflect comparative advantages for the country in conducting certain types of research. For example biotechnology research is most frequently funded by
the United States given the regulatory environment prevailing in many donor countries. Donors may also exhibit different preferences toward risk or the probability that outcomes will be realized. All research involves some degree of uncertainty about success and uncertainty and its time profile may vary by research line. For example, biotechnology research was in its early stages a highly risky endeavour because the basic science was unsettled. The time profile of this risk has changed and the more important risk to biotechnology research now is regulatory whether the regulatory framework and political reality will allow the research to be used. All else equal, donors would likely favour certain investments over uncertain investments but it would be a mistake to abandon risky lines of research that show great upside potential. As noted, real options theory shows that there is an important value to risk when options exist. Donors may need to become more aware of this value.
3.2 Defining the relevant alternatives

An important step in agricultural research priority setting is to obtain the opinions of key stakeholders in defining the appropriate researchable alternatives to be prioritised. Some alternatives may be determined before the decision is made to invest in research as opposed to other donor interventions. However, information on key problems that might be solved through research at the country, regional or global levels can come from local stakeholders. An important stakeholder group is the scientific community. Information on what is possible to achieve through research should draw on appropriate, often multidisciplinary, scientific expertise. Major global issues such as concerns over climate change or growing water scarcity are generally known but key researchable problems in specific countries or regions may not be known even to donor representatives in the field. Even when consensus holds that an issue is important, the solutions and lines of research leading to them may not be clear. For example many paths of solution to water shortages are possible including drought resistant breeding, identification and adaptation of new varieties or even new crops and others. Scientific input into the potential of each researchable problem needs to be considered. A participatory structured process that identifies key researchable problems and the potential contributions of different disciplines to them is essential.
14
3.3 Projecting impacts of research

A key part of research prioritisation is assessment of the impacts that can be achieved through investment in different areas of research. These impacts represent potential contributions of the research to donor goals and objectives. One challenge in predicting research impacts for use in priority setting is to link the expected changes to the research alternatives under consideration. To do so, it is imperative to have a conceptual model of how the research information is expected to generate an impact, i.e. the impact pathway. This pathway guides information collection for priority setting. All the methods mentioned below require information about the functional relationship between the research output and a change in state or capacity, whether that change is in productivity, consumption, health or other. These changes usually occur at the beneficiary level a change in productivity at the farm, more nutrients consumed by the householdand the methods generally involve aggregation of these small changes over many units. While some of the methods propose short-cut techniques to measure the micro-level change and/or techniques for aggregating them, precise quantification of the expected micro-level change is often the most difficult problem in ex-ante assessment. For example, increased productivity leads to income gains at the farm level. Farm-household consumption patterns are likely to change as a resultand these changes induce other changes. The net effect of all the changes is difficult to predict. Even when completed research is evaluated, its impacts can only be truly measured by comparing what actually happened with what would have happened if the research had not been conducted (Khandker et al., 2010). The outcome without the research intervention is the counterfactual but the counterfactual is not observed. Therefore evaluators are left with either trying to compare households before and after the intervention or with and without the intervention. The problem is that other factors change before and after, or differ between groups with and withoutand these changes or differences must be carefully accounted for. Otherwise, there could be systematic differences between adopters and non-adopters of a research intervention. It is especially difficult to identify the counterfactual when projecting impacts before the research has been completed. One must project what might happen with the research and compare it to what is most likely to have happened without it. This comparison becomes more difficult the farther into the future one is making the projection. For example, people adapt to climate change by adopting new cultivation techniques,
switching crops and in many other ways. A true assessment of the likely impact of a particular line of climate change research would have to compare outcomes with the research to whichever of these post-adaptation outcomes would be most likely. The farther one projects into the future, the more one must discount increasingly uncertain benefits. Depending on the types of impacts measured and the methods employed, impacts may be quantitatively or qualitatively assessed. The latter is used when quantitative assessment cannot be performed with enough rigour to impart confidence in the results given time or data limitations or the complexity of the interactions. Quantitative assessments are preferred where confidence can be placed in the results but the level of confidence will differ depending on the nature of the research and the types of impacts. Improvements in productivity or efficiency that might result from research on specific commodities are among the easiest to project quantitatively. Methods such as projecting economic surplus (income) changes and including them in a cost-benefit or rate of return analysis are well established and have been frequently applied (Alston, Norton and Pardey, 1995). They have been used to estimate the benefits of research of plant breeding, pest management, agronomic practices, animal health and many other areas. These methods are most suitable when researchinduced changes in yields and/or production costs can be projected and combined with data on value of production and potential technology adoption to estimate economic benefits. They are also most suitable when the chief source of impact is through markets and when market-level changes affect producers, consumers and other stakeholder groups. In some cases, the distribution of those benefits can also be estimated by household income level, by region, by consumer versus producer and by gender. These methods are less suitable for measuring impacts that occur outside of markets or are not well reflected in market changes, such as nutrition, health and environmental impacts. Quantitative methods are available for and have been applied to project agricultural research impacts on nutritional wellbeing, human health and the natural resource environment. However, due to the multifaceted nature of nutrition, health and the environment, less confidence can be placed in most quantitative assessments of predicted agricultural research impacts on these factors. Examples of the types of quantitative methods that have been used are listed below in table 1 and described in appendix 2. In some cases it can make sense to use these methodsas long as their limitations are recognized.
15
The impacts of research also differ by discipline in ease of impact quantification. For example, impacts of policy research or of genomics research can be harder to predict or even measure than those of plant breeding. If quantifying impacts is stressed too highly, there is danger that research portfolios will be skewed toward topics for which impacts are easiest to measure, even though they may not have the highest payoff. It is easi-
est to look for lost keys under the lamp post but that is not always the most fruitful place to search. A brief description of common methods that donors might consider for projecting impacts of agricultural research is provided in table 1. The types of research and donor goals for which they might be appropriate are discussed along with their strengths and weaknesses.
Table 1. Methods for assessing impacts of agricultural research
Method
Goals most easily addressed All goals Productivity, income, food security Productivity, income, food security Environmental
Ease of use for research impact assessment Medium-high High
Reliability of predicted impacts Medium-low Medium-low
Meta-analysis Congruence analysis
Benefit-cost and economic surplus analysis Non-market valuation methods Disability-adjusted life years Poverty rate analysis Randomized control trials
Medium
Medium
Medium-low
Medium-low
Health and nutrition Distributional Productivity, health, environmental Productivity Income, environmental, distributional All goals All goals
Medium Medium-low Low
Medium-low Medium High
Econometric impact models Simulation/math programming Changes in indicators Scoring indices
Medium-low Medium-low
Medium Medium-low
High High
Low Low
3.4 Comparing alternatives and establishing priorities

The starting point for agricultural research priority setting by a donor is to identify the goals for the research programme. Once the goals are established, potential research alternatives or themes are identified and potential impacts of research are projected for those alternatives or themes. Once the potential impacts are assessed, a research portfolio can be developed that considers the weights placed by the donor on specific goals. Differences in units of expected change can complicate priority setting. Economic surplus and cost-benefit analyses are favoured by economists, in part because
they deal with common units/monetary units. Comparing the expected income change from one research portfolio to another is consistent because the units are all currency units. When comparing across unitssay currency units to disability-adjusted life years weighting becomes complicated and resulting rankings must be treated with care. In addition, all methods possess inherent uncertainties as they all project into the future. The relative confidence in the accuracy of the underlying information should be considered when choosing among alternatives. Impact assessment is itself research and can absorb significant resourcesso choices must be made on the level of rigour and resources to devote to assessing
16
potential impacts of alternative research themes. For each donor goal, key factors or variables largely deter-
mine the expected contributions of a research project toward achieving those goals (table 2).
Table 2. Key factors that determine contributions of a research project toward achieving donor goals.
Goals 1. Agricultural productivity gains, income increases and food security Key determinants of research impacts Value of production Yield and cost changes Odds of research success Likely technology adoption Timing of benefits and costs
2. Nutritional and health improvements
Size of target group Incidence of problem Mortality rate or degree of disability associated with the problem deficiency Effects of research on reducing the specific nutrient problem Effects of research on food prices Effects of research on incomes of poor Number of poor in target audience Depth of poverty Income provided by the research results or technology Adoption of technology by the poor Effects of research on food prices Depends on aspect of the environment, hence on changes in specific physical measures related to pesticide use, soil erosion, carbon sequestration etc.
3. Poverty reduction
4. Natural resource sustainability / environmental improvement
For the goal of improving agricultural productivity, income growth, food security and value of production is one of those factors. Which is one reason why the congruence methodwhich simply allocates the research budget in proportion to value of productionis so popular. Of course other variables are important too, such as the projected impacts of the research on yields and costs, the odds of success in the research, the likely adoption the technologies and the timing of the benefits to be received. The economic surplus/benefit cost method considers those factors. Rather just considering value of production, benefit-cost analysis can be applied for a more rigorous analysis (than congruence analysis) of the economic impacts of research. Fortunately, for the major commodities, data and models such as those in HarvestChoice can be used to readily apply this method for research aimed at major biotic and abiotic constraints. Alternatively, simple spreadsheets can be used to project economic impacts of research using an economic surplus approach.
For the nutrition/health goal, the size of the target group, the incidence of the problem in the target group, the mortality rate or degree of disability associated with the problem deficiency in each target group and the projected effects of the research on reducing the problem are key factors driving research benefits. Notice that the size of the overall impact of research on quantities produced and consumed and prices paid all affect nutrient consumption. Thus an assessment of economic surplus impacts may be a logical starting point for a health/nutrition assessmentthe foods that make up the largest share of the consumption of malnourished people are those that have the biggest impact on nutrition. The DALY approach is widely used because it considers these factors more, and because it has been used in health assessments so that key coefficients have already been estimated in many cases. It also puts changes in health outcomes into common units so that comparison of DALYs across research programmes makes sense. Calculation of
17
DALYs, however, still involves a number of assumptions that are difficult to verify. Key factors influencing poverty effects of research are: 1. The number of poor in a target audience 2. The depth of their poverty 3. The income provided by the research results or technology 4. The adoption of the technology by the poor 5. Food price changes affecting consumers Use of FGT type (Foster-Greer-Thorbecke) poverty indicators is relatively simple but the data necessary to compute the indicators are often absent without additional surveys being conducted. Consequently even simpler indicators than FGT are often used to project poverty impacts such as the importance of a specific food in the diet of the poor. Often staple foods are most important in diets of the poor. Unfortunately these are rough indicators as improvements in high value even non-food commodities can be equally or effective more effective in lifting people out of poverty. Because the environment is multidimensional, a practical method for assessing environmental sustainability is to assess the physical effects of the research across specific environmental categories. In some cases those effects can then be valued using market or non-market techniques. In other cases the cost and imprecision of the methods for valuing environmental impacts makes them not worth the effort. Availability of data to support environmental impact assessment varies by category of environmental impact and the data are often rough. Detailed data from the plot or farm level can be difficult to project to the aggregate level. For major environmental topics such as climate change, global models exist which predict the expected spatial distribution of heat stress, flooding and other effects. The impacts of agricultural research on mitigating those effects must consider the uncertainty involved in predicting both the occurrence of climatic events and in mitigating them through specific research alternatives.
it by making profit from it. In many cases the issue comes down to whether a private firm can control the intellectual property rights through plant breeders rights, patents and so forth. Even in cases where legal protection exists, incentives may not exist if the technology would be too easy to duplicate. For example, a crop such as rice is problematic for the private sector for varietal research because most rice is open-pollinated which implies that farmers can save their seeds from one season to the next. Therefore the company would suffer a reduction in seed sales after the first year of release, making it difficult to recoup its research costs. Most maize varieties however are hybrids for which the seeds cannot be saved from season to season. As a result private companies have more incentive to conduct research on maize than on rice for some technologies for those crops. For other technologies, for example seeds for vitamin-enhanced crops, the private sector may not be able to sell them at a premium and therefore the private sector would not have much incentive even if they could control the intellectual property. Another step or factor for an individual donor to consider is the extent to which other donors are currently or will in the future support research on a topic and the comparative advantage of the donor on particular research themes. For example, regulations in the United States are supportive of genetically modified crops but they are not in Europe. Therefore USAID might be a more logical donor to support GMOtype research than European donors. Major research problems also differ by country and for political and other reasons donors select certain countries to emphasize over others. These factors may affect the selection of research alternatives. Once the public versus private good nature of the research is determined and donor comparative advantage is considered, a method is needed that can evaluate the tradeoffs with respect to research contributions to different objectives. Three primary methods might be considered: 1. Utilize a tradeoffs model that considers the value of research contributions to one goal that would be given up if contributions to another goal are emphasized 2. Use an optimization method 3. Use a scoring approach In all cases significant judgement needs to be applied, as none of the quantitative methods give a complete answer without making assumptions that need to be considered (box 5).
Portfolio analysis
A key step for donors after they project likely impacts of agricultural research by goal is to assess the extent to which the research should be a public rather than a private responsibility. It is possible that research that is projected to make the highest contribution to a donors goals should not be supported by the donor because the private sector has sufficient incentive to undertake
18
box 5. Methods to assess tradeoffs in research contributions to objectives Tradeoffs model The simplest application of the tradeoffs model assesses the potential impacts of a research portfolio that emphasizes one objective versus another and then calculates how much the contribution of the portfolio to one objective is reduced if the other is emphasized to differing degrees. For example, a portfolio could be developed that maximized the total economic value of agricultural productivity growth and another that maximized nutritional and health impacts for children under five years old. The two portfolios may overlap but are not identical. Therefore a model can be constructed to assess how much total economic value is sacrificed to achieve different levels of the second goal or a model can be used to assess how much of an impact on child nutrition is sacrificed to meet varying levels of specific economic value goals. Optimization model The simplest optimization model would be to construct a programming model in which a multiple-goal objective function is maximized subject to research resource constraints with weights on goals allowed to vary. The levels of funding for specific research themes can be easily allowed to vary with this model as well. Contributions to goals could be measured in dollar terms.
Given the difficulty of valuing contributions to non-productivity goals, an alternative objective function could also be used with income growth (goal 1) maximized subject to varying levels of nutritional, poverty, or environmental impacts. The model could be set up for a country, a region(s), or as a global model. The disadvantage of the model would be the need for fairly accurate impact assessments for each research alternative for each goal and also for assumptions about the nature of the relationship of impact on specific goals as research budgets are increased. Mutangadura et al (1999) provide an example of this model for an agricultural research system in a single country but donors would likely find this model too data intensive in practice. Scoring model A scoring model can be used to weight the contributions of research alternatives to various goals. As noted above the weighting should be of goals and not of simple indicators or disaggregated factors before they are combined to measure impacts. Even if scoring is used to weight goals, there is still room for nonsense results from scoring models if contributions to goals are measured in different units. Scoring models are dangerous because they appear simple and yield rankings but the rankings are often based on a meaningless weighting of apples and oranges.
One issue that arises in applying any priority setting model is how to assess the benefits of specific research options under varying levels of funding. One way to address this issue is to project research benefits under a high, medium and low level of funding for each alternative. In some cases there may be returns to scale as funding is increased and in others incremental benefits may decline as funding grows. Another issue is how to consider research that is already in the pipeline as opposed to new research. Once the initial analysis of priorities is completed, new high impact areas are included in the portfolio and existing areas are continued if they continue to be importantor if not they are dropped at the appropriate time such that the returns from past investments are captured. Publically supported research may emphasize highpayoff, high-risk, long-term problems because the private sector is often hesitant to undertake such research. Public research managers are faced with the difficult task of justifying to high-level decisionmakers and politicians a research portfolio that includes risky long-term research themes. However, a well-structured priority setting process may help in that regard.
3.5 How to use priority setting methods

Among the three portfolio models described above the tradeoffs model is likely to be the most useful to donors, given its combination of rigour and ease of use. This section presents basic steps in applying a tradeoffs model so that the priority setting process can be compared with current practice as identified during donor consultations. Basic steps 1. Identify donor goals _ These goals are often established for the agricultural portfolio of the organisation at a higher level than the specific unit responsible for administering the agricultural research portfolio. There are frequently multiple goals that relate to some combination of improved agricultural income (productivity, food security), improved equity (reducing poverty for producers and consumers, assisting smallholders, empowering women), improved nutrition (and health, especially for children and women of child-bearing age), environmental sustainability (reduced erosion and pollution, increased carbon sequestration, increased biodiversity, mitigating effects of climate change, etc.). Weights on these goals may be explicitly determined by the institution, or determined during the priority setting process, but often they are left flexible if there are multiple goals
19
as tradeoffs are assessed later in the process and implications of alternative weights become clear. 2. Identify the research alternatives or options to be prioritised_ These options may be among commodities, non-commodity topics, or a combination of the two, usually in combination with particular geographical foci. They may also involve institutions for strengthening. A range of stakeholders are included in this step. Researchable problems from the country and regional levels can be identified by local scientists, farmers, NGOs and organisation personnel. Scientists in the donor organisation, universities and national and international research organisations can identify what is possible. Scientists, economists and the private sector can identify which alternatives can be excluded because there are adequate private incentives to undertake the research. The list of alternatives may include some research topics with potential short term benefits and others that are long term. It may include some topics that are high risk but high reward and others that are low risk but perhaps low reward. It may include commodities or options that are known to be important to the poor. It may exclude alternatives that are known to be addressed by other donors. Two or three rounds of organized consultations may be required just to define the list of alternatives to be prioritised. 3. Define the measures of contributions of the alternatives to the goals and gather data and other information_ This is the most difficult step. Contributions of some research options for some goals are difficult to measure, some options contribute to multiple goals, projecting research impacts carefully will require resources and time is limited. For some alternatives it is possible to project their Contributions to the agricultural income goalas a whole or for targeted groupsby designing an impact measure that includes the base value of a commodity Expected yield and/or cost change due to the research Projected rate and timing of adoption of the resulting technology or policy change Probability of research success With additional information on market parameters, a ggregate economic return can be calculated. For some commodities, this analysis or the data for it can be obtained from HarvestChoice through the University of Minnesota or the International Food Policy Research Institute. For other research options, data would need to be gathered from scientists and secondary sources and spreadsheets or another programme used to perform the calculations.
Questions to scientists may need to be framed in a way that allows for research funding levels to vary and corresponding benefits to differ depending on the funding. With additional data on income or consumption of representative households, a poverty impact could be projected for the research options using the surplus analysis as a building block. Since many environmental benefits associated with the research options are difficult to project and are often multidimensional, expected physical or biological effects for specific categories of the environment may be obtained from scientist opinions or from existing models on climate change, erosion and so forth. Because of the time and data required to calculate DALYs, for the nutrition/health goal, a simpler approach may be used to assess nutritional impacts that involves gathering information on the total number of people in the target group and the incidence of the problem for that group and then projecting the effects of the research option on reducing the incidence of the problem for the target group. An economic surplus analysis would help frame the analysis of health and nutrition impacts because it provides information on changes in quantities consumed and prices paid for the commodity in question. These changes would be a logical starting point for considering health and nutrition impacts. For environmental and health impacts, meta-analyses of the literature may be a fruitful option. 4. Conduct a tradeoffs analysis once the basic impacts are projected for each research option and relevant goal(s)_ In this analysis, the impacts of each option on one goal are compared and ranked, and a total benefit for the complete set of research options is calculated after allocating budget to each item on the list down to the point that the budget is exhausted. Then the impacts for another goal (on a nutrition measure, poverty measure, or environmental measure) are calculated and the options are ranked for that goal. For example, if the latter ranking is for a nutrition goal, it may imply a different set of priorities from those for the income goal. The total economic benefits for that second list can be calculated and subtracted from the total for the first list to calculate how much income is given up by focusing all the research on nutrition. Calculations can also be made for lists that mix items from the two sets of priorities (or three sets if more than two goals). 5. In the fifth step, a preliminary decision on funding priorities is made based on information from step four. This set of priorities is then compared to the current portfolio to determine where adjustments may need to be made_ At this step, the priorities are shared with decisionmakers at higher levels of the organisation or in the political process to determine what is feasible or desirable from a political standpoint. Given the impact calculations made, it will be possible to demonstrate what is projected to be given up for a particular goal if the final research portfolio differs from the preliminary list.
20
4.0 Donor consultations

Informal consultations were held with representatives from seven organisations to discuss their current priority setting mechanisms. Representatives were interviewed from BMZ, GIZ, CIDA, DFID, EC, OECD, USAID and the Gates Foundation. Individuals were sent emails asking for their participation and a small set of interview questions was sent in advance of the phone interviews. designed to produce shorter-term outputs such as investigation into processes to adapt to climate change. This balancing is not a result of formal modelling but uses expert opinions of technical staff about tradeoffs. USAID also examines its own comparative advantage in doing certain types of research. As a result of this comparative advantage, for example, USAID is a major funder of biotechnology research. Many European donors are prohibited by law from conducting biotechnology research. USAID helps fill the gap. DFID and USAID provided the strongest evidence of use of scientific information in priority setting, but even their processes were ad hoc in nature. For other donors the process of setting research priorities is unstructured and changes depending on immediate needs. CIDA uses an informal process that favors emerging priorities or responds to political direction. Messages about highneed areas and the effectiveness of alternative research themes filter up to senior leadership and among CIDA programming staff. Currently, nutrition is a priority, but with an open ended timeframe this priority could change. CIDA is mainly responsive to country contexts and allows country needs to be considered when selecting its research initiatives. This process is ad hoc due to a lack of technical capacity to undertake a formal priority setting process. GIZ stated that it accepted the CG prioritisation as reflected by the CRPs and other projects and does not conduct additional priority setting. GIZ-supported research projects are selected based on correspondence with development policy principles of BMZ and the ongoing focus of the CRPs. Each year the CG centers submit proposals for funding; these proposals have to correspond to at least one CRP. Prioritisation of proposals and centers for funding depend on performance indicators for centers; most of the agricultural research resources go to well-performing centers. BMZ, the German ministry providing the research funds to GIZ, stated it would like to concentrate its funding on fewer CRPs following analysis of CRP effectiveness and correspondence with German political priorities. BMZ recognized that it was part of a system to fund the CG and it would do its best to align itself within the system. National priorities may change and BMZ would adapt to this change, but by reallocating its funding within the CG framework. The European Commission has a number of Directorates General (DGs). The DG for Research and Innovation supports agricultural researchlargely from a European perspective. DG EuropeAid (DEVCO) has a mandate for
4.1 Which priority setting mechanisms are currently used?

Summary points
Most donors reported using only informal processes to set priorities, explaining that they lacked capacity, resources or staff time to conduct formal, technical priority setting exercises. Some donors with larger technical staffs such as USAID and DFID had at times engaged in formal priority setting but did not do so on a regular basis. No donor uses a standardized procedure.
Details
DFID stated that when setting priorities, it responded to minister (political) goals and attempted to align its agricultural research programme with those goals. The institution uses different forms of analysis to establish which areas of research were likely to contribute to which higher-level goal, and uses technical staff judgment to make tradeoffs across objectives. Most of the analysis is of published research but some consult with experts or with country missions, or both. The respondent stated that history was important as DFID had always invested substantially in the CGIAR and this pattern was not likely to change. DFID liked to maintain high funding levels since it allowed it to exert influence over the CG. This influence was important. USAID followed a detailed priority setting process for its Feed the Future (FtF) programme, which is an agencywide programme. The agricultural research programme within USAID subsequently realized that it had to align agricultural research focal areas with FtF priorities. It used its technical staff to evaluate different lines of research and compared them to prioritised problems to help effect this alignment. An important component of USAID research prioritisation is the balance between short/medium-term and longer term research themes. For example, research on climate change is long-term in nature (example was given of heat stress-resistant wheat) and USAID balances this funding with research
21
development and cooperation and supports agricultural research for development with a primary focus on developing countries. DEVCO divides its programming among global, regional and country levels. Country level research allocations are decided on the basis of a strategy negotiated between the EC delegations and the host country. Most research funding is programmed at the global and regional level. Priorities are set informally but generally correspond to EU policy priorities. They include support for smallholder farmers, research on agro-ecological systems and benefits to women. They also place a high priority on supporting African agricultural research by engaging regional groups such as ASARECA and FARA. Other factors considered during informal research priority setting are historical precedents, administrative ease and performance of research along key dimensions. They coordinate with other donors through various platforms, with a focus on Europe-wide coordination through the European Initiative for Agricultural Research for Development (EIARD). OECD undertakes policy studies in support of agricultural transformation in partnership with research institutions. It also helps coordinate studies funded by donors. The procedure for identifying which studies to undertake varies by directoratethe governance structures for the directorates, such-units of the OECD, define specific work programmes and institution-wide norms for setting priorities do not exist. The Bill and Melinda Gates Foundation examines measures of productivity and performance for different lines of research and makes marginal adjustments to its portfolio based on these measures. The Foundation also engages in consultations with donors, the scientific community and local and regional interest groups when deciding on a research programme, but formal modelling is not used.
research themes needed to show results. Donors noted that they were not insulated from these pressures and used evidence about research effectiveness to set priorities. These priorities had not only to reflect what was scientifically and socially desirable but also what was politically saleable.
Details
DFID commented that research programmes other than agriculture, such as health, had been more effective at showing convincing evidence of impacts and were thus receiving more favourable implicit prioritisation by the political process. If agriculture wanted to continue to receive adequate funding, it had to demonstrate convincing linkages between research and prioritised objectives. DFID uses in-house technical expertise to evaluate potential lines of research and its first consideration about methods is the reliability of evidence. It uses multiple mechanisms for evaluating research lines because a mechanism that works well for one policy priority might not work well for another. USAID expressed a similar perspective, relying on strong in-house expertise, but noted that it lacked technical ability to formally model or quantify tradeoffs among objectives. That is, it used technical expertise to evaluate the likely impact of lines of research on each development objective, but had no mechanism for comparing across research lines or between objectives. Donors also noted that the reliability of evidence tied directly into their ability to persuade politicians that their research programmes are effective. The ability to sell a particular research programme to the political world hinged on perceived reliability of the information. As politically determined objectives change, it was important to have evidence of research effectiveness in meeting a broad menu of objectives. At least one respondent noted that because the CG was self-interested, it was important to have reliable, objective evidence from external evaluators. Several interviewees hinted that more easy-to-use methods might lead to more widespread formal priority setting. Many donors lacked technical expertise to formally prioritise agricultural research. Others noted that the speed with which many of these decisions were made made it difficult to ever envision a regular formal priority setting process. Some stated that increased sharing of experience among donors might improve their ability to accept methods.
4.2 How donors evaluate priority setting mechanisms according to their ease of use, provision of reliable evidence and persuasiveness in informing decisionmaking?
Summary points
Every interviewee responded to this question by stating the provision of reliable evidence was his/her most important criterion in evaluating priority setting mechanisms. Several made the point that it was increasingly important to build an evidence base for investments in agricultural research. Political leadership and the general public demanded accountability, so different
22
4.3 Donor perspectives on the analytic methods available in the literature

Summary points
Donors all conveyed a favourable impression of analytical methods but none use them on a regular, formal basis. A synthesis of responses would include two key points. 1. All donors are open to more frequent use of formal priority setting tools but need collaboration with other donors. Few have in-house capability to conduct the exercises and none stated they were likely to obtain this expertise in the future. As a result it is necessary to leverage the expertise of others. 2. Donors are critically concerned about tradeoffs between objectives and would like means of evaluating these tradeoffs when setting research priorities. Donors have multiple objectives and the weights placed on the objectives vary by country and over time. It is important that priority setting tools have the capability to evaluate impacts on these multiple objectives. Several stated that a weakness of the priority setting literature was that it had focused almost exclusively on efficiency and needs to build a better linkage between efficiency and other objectives. This is an area where donor collaboration and consensus building on appropriate means of including tradeoffs may have a large payoff.
big gap in most of these models is convincing evidence of a causal relationship between a research finding and ultimate impact. More information about what actually works and why would help better prioritise research investments. DFID also expressed a favourable view of analytical methods from the academic literature and would use them when appropriate. DFID echoed the USAID concern that the evidence basein favour of agricultural researchis weak for non-efficiency objectives. It also stated that much of the academic priority setting literature focused too closely on efficiency, while many politicians currently favoured non-efficiency objectives. For example, the evidence supporting a strong linkage between agricultural research and child health and nutrition is considered to be weak and DFID is supporting research looking at that link. DFID and others are strong supporters of improving the evidence base and generally do not think the problem is one of accessibility of different techniques, but is concerned that priority setting research has focused too narrowly on efficiency. Formal models were needed to evaluate impacts of agricultural research on other objectives such as health, nutrition, environmental sustainability and others. BMZ stated that it implicitly relied on findings of research effectiveness from groups such as the CGs Standing Panel on Impact Assessment (SPIA). It presumed that SPIA was actively engaged in evaluating priorities while establishing the CRPs. The EC stated that quantitative methods were acceptable, but they found that the speed with which they made decisions was not amenable to formal priority setting processes. They used evidence from studies, their own experiences in the field and relied on the intuition of their own staff.
Details
USAID planning for Feed the Future came closest to use of analytical methods. USAID began by reviewing literature on the importance of staple crops for achieving food security and alleviating poverty (this research came chiefly from IFPRI). The academic literature pushed it toward a focus on staples as a means of achieving its objectives. This focus included cereal staples but also roots and tubers where these crops represent main sources of food energy. USAID next considered what factors needed to be addressed to improve yields and the scientific (biophysical) literature suggested that abiotic constraints were most important. HarvestChoice evidence on the relative size (value of production) and constraints faced by staple crops was considered. USAID respects formal modelling for priority setting but does not have in-house capability to conduct these exercises on a regular basis. A key concern of the agency is that decisions be based on solid scientific evidence. For example, when considering funding of particular lines of research, the USAID agricultural research office conducts extensive reviews of the scientific literature to ensure that approaches are feasible and in-line with best practices in science. The
4.4 Strategies donors employ to inform their practice and learn from each other
Interviewees noted a high degree of consultation among donors and that this consultation served learning and a coordination functionwith disagreement about which function was more important. For example, GIZ stated that it worked with the Global Donor Platform for Rural Development and other platforms to coordinate actions but felt that there was more need to bring information from the field level as to what works in development. If scientifically sound information were brought forward, consensus among donors could be reached and this information would better inform cross-donor research funding decisions. The Platform needed to be strengthened to better coordinate with donors and the CG system. GIZ supports creation of a menu of best practices to better inform research allocation decisions. OECD echoes this need for a consensus about what best works and a menu of best practices.
23
Donor consultation also exposes information about what has and has not been done and which institutions are doing what. Consultation helps identify knowledge or research gaps and ensures that appropriate entities are working in appropriate areas. USAID noted that an important function of donor consultations was to identify comparative advantage in addressing research problems. This comes through consultations on platforms such as the Global Donor Platform, through feedback and consultations with country missions and stakeholders. The impression was given that USAID would like more coordination among donors but felt that this coordination would lead to more effective research fundingnot more information about what works and what does not. The information about what works and does not should instead come from the scientific community. DFID noted that consultation across donors and recognition of other donor programmes was critical for effective global agricultural research planning. DFID actively consults with donor partners to understand how their investments can best be leveraged. This consideration was based on perceived comparative advantages, institutional interests and knowledge of who has done what over time. The EC noted a similarly high level of consultation with various donor platforms including the EIARD, the CG fund council and other global donor platforms. EC also receives bottom-up input from implementing partners (including African research networks and the CG), information from EC delegations to specific countries and shared information through donor networks. By identifying comparative advantages and gaps, additional consultation could lead to more efficient use of research resources. Donors could bring information from their own systems about perceptions of comparative advantage of global versus field office-funded research in addressing specific problems. In addition to global research support, most donors conducted research through their country offices. Country-office research tended to be closer to end users, while the former tended to focus more on producing global public goods. Over time, political forces had favoured an increased focus on achieving research results to meet political objectives. These types of research results were most frequently seen at the local- or end use-level. As a result, donors perceived fewer incentives from the political system for using research funds to produce global public goods and there was some potential that they would thus be underfunded. Increased coordination and dialogue might help avert this problem. Some donors expressed the idea that the cross-donor research allocation dialogue needed to be better linked to the organisations on the ground who are engaged in
managing programmes for agricultural development. BMZ/GIZ and CIDA were all strong proponents of this view. Enhanced dialogue could help ensure that research would be better programmed to meet the needs of groups such as NGOs, national extension services and organisations engaged in agricultural development. This opinion was not universal. Several donors thought adequate attention has been given to this issue. None of the donors stated that less coordination was desirable. Several noted that increased coordination would allow more effective use of academic models to analyze and prioritise research needs. The exact vehicle for such coordination would need to be considered but alternatives existed. For example, a single global priority setting exercise for agricultural research could be undertaken, possibly informed with information such as that available from HarvestChoice. Of course, knowledge of tradeoffs and ability to achieve other objectives would need to be brought to the table. Such an exercise would set broad parameters about objective relationships between research expenditures and multiple objectives. If the exercise were fully transparent and the donor community was convinced of its objectivity, a consultative process could then be undertaken to identify institutional comparative advantages, political priorities etc. and global efficiency could be attained.
Summary points
1. Donors would like to undertake more formal priority setting, but most lack capacity to do it. The speed of decisionmaking is also a concern, as several donors noted that because research resource allocation decisions had to be made frequently and quickly a formal process would be unwieldy. This is an area where agreement and collaboration among donors might bear fruit. Many of the donors expressed a need for more information to inform their priority setting, but do not have adequate staff resources to utilize a formal systematic process. Establishment of a donor-certified process to systematize information might find wide acceptance. 2. Donors want more information on the impact of agricultural research on non-efficiency objectives. They feel that the evidence base needs to be strengthened. There is a sense that because much priority setting has been guided to meet efficiency objectives, other objectives/ goals may be underappreciated in application of formal models. Several stated that they recognized that the overall size of the economic impact was important, but this information does not stimulate political enthusiasm. More needs to be done to link the overall size to attainment of other objectives such as reductions in poverty. They need this information to incorporate quantitative tradeoffs into their decisionmaking process.
24
4.5 Multi-donor engagement

There is a danger of donors over-investing in some areas and under-investing in others as they all provide their resources for research on the highest priority topics. No matter how important a topic is, there are capacity limitations to work on it. Some topics are important but not as attractive to donors for whatever reason.
This problem can be minimized if: Donors share their plans with other donors in a forum such as the Global Donor Platform or EIARD Donors assess their own comparative advantage for addressing specific topics and countries Donors to the CG system allocate enough of their resources to Window 1 which gives the centers flexibility in allocating their funds to topics Donors communicate with CG centers and NARS to discuss this issue explicitly when they allocate funds to other windows or topics
5.0 Mechanisms for investments in agricultural research

5.1 Types of institutions used by donors to invest in agricultural research
Summary points
Donors engage a wide variety of institutions to conduct agricultural research for development, including the CGIAR system, universities, government agencies, NGOs, private research organisations and others. While the distribution of resources among institution type varies from donor to donor, it is clear that the bulk of research resources pass through the CGIAR system. Reasons for this dependence on the CG vary and include: Confidence in the quality and relevance of CG-led research Close correspondence between CG goals and donor goals The ability of CG centers and CRPs to link with other actors including NARS and university researchers The ease of contracting with the CG and its ability to conduct large-scale, multidisciplinary research fund administered by the World Bank. In 2010, approximately 20 per cent of total funding to the CG was disbursed through the World Bank. The main advantage to members of using the World Banks payment system is a reduction in the number of disbursement transactions. Members make only one transfer to the World Bank trust fund account instead of as many as 15 to individual centers bank accounts. Since 2001, centers entered into formal contractual agreements with the World Bank concerning accountability for the funds they receive. CGIAR funding has been traditionally divided into two broad categories: unrestricted and restricted. Unrestricted (core) funding supports a center as a whole. The World Banks funding for general support is the best example of this type because, within the research agenda, allocation of the funding is totally unconstrained. Recently, about 30 to 35 per cent of funding to the CG have been unrestricted. Centers can allocate unrestricted funds to any programme or cost within the research agenda on the basis of institutional needs and priorities. Three levels of restriction are associated with restricted contributions to the CGIAR: the least restricted are programmes (e.g. Challenge Programs, System Wide and Eco-regional Programs), followed by targeted (e.g. geographic) and the most restricted are projects (usually requiring line item reporting). Even donor funding to the CG, however, is used to engage other institutions in international agricultural research. For example, GIZ, the implementing agency for the German development ministry BMZ, allocates essentially 100 per cent of its research budget through
Details
In the CG, members have the option to disburse their funding either directly to the centers, to CRPs, or through the secretariat using the multi-donor trust
25
the CG. About 25 per cent go directly to unrestricted core funding for the centers and the remainder goes through the German Advisory Service on Agricultural Research for Development (BEAF). This latter 75 per cent are used for project funding in support of the 15 CRPs who partner with German universities, NARS and other researchers to submit research proposals to BEAF. BMZ and GIZ expressed the view that it is important to include German universities as partners in much of the research. This inclusion helped ensure work is scientifically sound and helped better align the research with political priorities. A small portion of this 75 per cent were devoted to small grant funds. Thus, the 100 per cent of BMZ agricultural research funding devoted to the CG engage numerous institutions. GIZ gives three reasons for reliance on the CG: The consistency of their objectives The quality of the research The relevance of the CG research for development Donors with more mixed portfolios also tend to support the CG but mix this support with other institutional arrangements. As an example, DFID currently supports around 40 international agricultural research projects. Approximately 52 per cent of this funding go for core support to the CGIAR or to individual centers or CRPs (DFID is the leading supporter of the Center Fund), with the latter consuming the lions share. Other DFID partners include: Private research councils and organisations (about 15 per cent of funding), some UK-based but many located in developing countries or regions UK and developing country universities (15 per cent) Support for partnerships through research councils and others Support for private research funds in developing countries such as Africa Enterprise Challenge Fund (AECF), Forum for Agricultural Research in Africa Inclusion of universities and other advanced research groups helps build diversity of viewpoints and strengthens scientific rigour. USAID is another institution with a mixed portfolio of institutions. Like DFID, the majority of its research resources flow directly to the CGIAR, with a mix of funding to the CG Funds various windows and direct funding to Centers and CRPs. The second largest
recipient of USAID agricultural research funds is the Collaborative Research Support Programs (CRSPs) whose funds all flow through US universities. In its support for the CG and the CRSPs, USAID indirectly funds other entities including host-country universities and NARS, NGOs and public and private research organisations. Other major research efforts include partnerships with host-country regulators, extension workers, farmers and othersexamples include the Agricultural Biotechnology Support Program, the Farmer to Farmer Program and Support for the International Fertilized Development Center. USAID also funds research directed toward specific regions such as the Middle East Regional Cooperation Program, which focuses on agricultural research involving Arab-Israeli cooperation. Increasingly USAID is funding private sector entities to do specific directed research where those entities have a comparative advantage. The EC also has a mixed portfolio with about 50 percent of their funds going to the CG. The others go through regional research organisations (mainly in Africa) and through open calls. Interestingly, the EC prohibits the CG centers from participating in many of their open calls, partly as a means of ensuring institutional diversity. Donors are increasingly considering partnering with applied research entities who can deliver research results to end-users. For example, CIDA has established several mechanisms for conducting research closer to end users. The Canadian International Food Security Research Fund supports individual research partnerships between Canadian and developing country researchers with grants to develop applied agricultural innovations. CIDAs proportion of its agricultural research portfolio that is devoted to only research generation is decreasing as research investments are integrated as a component of development projects. There is recognition of an explicit need to link research with development programming and to scale up successful research products. GIZ expresses similar ideas and uses its field office presence to help build linkages between researchers and the ultimate users of the technologies. CIDA also supports participatory farmer-led research through civil society partnerships and a university twinning programme that increases capacity of developing country education and training organisations. Donor interviews did not reveal concern about disproportionate investments in certain institutions. If this concern existed, it would have to be related to the CG system which receives by far the largest shares of agricultural research funding.
26
Concern about overinvestment in the CG is muted for several reasons: 1. There is widespread agreement among donors that funding for the CGwhich declined for several years prior to the food crises that began in 2008had reached dangerously low levels. As the magnitude of the crisis became apparent, donors began to reinvest and the infrastructure of the CG is such that it is well-able to absorb resources 2. Major donors have confidence that CG research closely aligns with their own goalsfunding the CG system is the most cost-effective way of meeting research goals 3. Most CG research involves partnerships with other institutions so that the resources that flow to the CG are actually spread across many institutions 4. Individual donors balance their portfolios at the margin in order to achieve objectivesthey support the CG in order to meet broader development objectives and balance these with more targeted funds to meet country- or organisation-specific goals Donors appreciate the efficiency of interactions with the CG. Donors were engaged in an intensive consultation process during development of the CRP approach and the new CG Fund. As a result, donors have confidence in the CG and passing funding to existing CRPs facilitates efficiency and avoids duplication. Potential drawbacks to heavy reliance on the CG include passing a disproportionate administrative burden on researchers and limited diversity in research perspectives. However, CG researchers voice concerns about the growing burden of proposing and managing multiple competitive projects. Donors should consider whether increasing movement toward competitive grants targeted at the CG might unduly impose administrative costs on researchers. The second drawback is not likely to be a serious problem due to diversity of disciplines represented in the CG and the high degree of partnering with outside institutions. However, several donors voiced concern about challenges of partnering with international agencies that are not part of the CG. The facility of working through the CG may lead to underinvestment in alternative regional partners.
5.2 Mechanisms used to invest in agricultural research

As noted, the main mechanisms used to invest in agricultural research include investments in the CG, which usually involves multiple mechanisms, competitive grants, directed research in specific areas, scholarships, and university-focused research and capacity building. Three broad trends are evident: Increasing donor investments in the CG Increasing reliance on competitive grant funding mechanisms Increasing use of targeted research to address specific research questions
CG support
Donor support for the CG appears to be solid and is based on several factors, which include: Quality of CG research Relevance for development and correspondence with donor goals Transparency of CG planning Historical investments in CG infrastructure enhance its ability to conduct a wide variety of research Donors consider the CRP approach to be a major improvement in the way CG research is conducted and several rely on the CRP structure as a guide to their own priority setting. Unfortunately, the CRPs do not reflect any serious priority setting. They are mainly a structure under which prior cross-center research themes have been organized. As a result, evenly distributed funding among the 15 CRPs reflects a failure to prioritise. Similarly, Window 1 investments reflect implicit reliance on the CG prioritisation. All donors interviewed stated that they would like to increasingly concentrate CG-destined resources into the more productive CRPs, but none has formally considered the correspondence between CRP objectives and impacts of CRP research on their own objectives.
Competitive grants
Reliance on competitive grant funding has its own economic logic. The marketplace of ideas may generate approaches or issues that the donor was unaware of and competition among potential researchers will enhance cost efficiency. Proposal calls can also be structured to promote more diversified institutional
27
participation and are especially effective in inducing participation of advanced research institutions. As a result, most donors have moved aggressively toward competitive fundingand even funding spent through the CG system is often put into a competitive bidding process. USAID, GIZ, DFID and CIDA funnel a large proportion of agricultural research resources into competitive processes. The World Bank is increasingly moving toward competitive proposals and Gates funding is largely competitive. Donor interviews indicated strong perceptions that reliance on competitive funding is not likely to decline in the future, but some are increasingly using targeted research to fill gaps.
their large competitive grants. Many of these also promote linkages with other developing-country institutions such as NARS. The USAID CRSPs, for example, provide funding for developing country researchers (often university researchers) and also provide scholarships to US and host-country universities for US or host-country graduate students.
5.3 Pros and cons of competitive versus targeted funding

Competitive funding
Has the advantage of attracting the widest set of potential scientists and institutions. If the subject of the call for proposals is broad enough, the response may generate novel approaches to the problem. They can help scientists to focus their thinking. Scientific rigour can be maintained by peer review of proposals. The calls can be made on topics in response to political pressures and hence maintain support from above for the donor. The biggest disadvantage of competitive grants is the scientist time diverted to proposal preparation when only a fraction of the proposals written will be funded. They also tend to require more donor time than core funding mechanismsbut that difference is dwarfed by the time scientists divert from research to writing proposals and preparing reports. Competitive grants tend to have a shorter time horizon than core funds which may mean the topics are more applied than they should be or research is stopped before it is totally completed.
Targeted research
Several forces have made targeted research more attractive: 1. Since donors rely increasingly on competitive grants, certain areas of research may be neglected. For example, USAID used targeted earmarks for biotechnology research in its CRSP research because the funded proposals did not adequately cover biotechnology. 2. Donors recognize the comparative advantage of certain entities, often in the private sector, to conduct certain types of research. For example, applied biotechnology research is often best done by the private sectorand donors recognize this advantage. Targeted funding can be the most efficient means of meeting objectives when comparative advantage is relatively clear. 3. Research projects tend to be of medium duration and donor priorities may change. The research itself may indicate a particular area of need, or the demand for results may be pressing. Opening a new round of competitive proposals can be costly, time consuming and may not fill focused research gaps. In such cases, targeted research has advantages. Donors aggressively promote participation of their own universities in international agricultural research. This is generally achieved by placing conditions that grant proposals include domestic universitiesall the major donors include these conditions in their major research grants. Some donors, CIDA is the best example, promote partnerships between their domestic researchers and host-country researcher partners, mostly universities. This brings Canadian expertise to bear on particular problems and can be an effective way of building capacity at developing-country research. USAID, DFID and BEAF/GIZ also have language about partnerships with developing country universities in
Institution-based, core funding

Has the advantage of flexibility. It can be used to maintain the basic research infrastructure of an institution, facilitate long-term research programmes and at the same time allow research directors to take risks and move funds to emerging areas that appear important. Research directors and scientists are often closer to the emerging problems on the ground than donors. Scientists have an incentive to perform for their director and directors have an incentive to reward strong scientific performance. This arrangement may lead to better research than one where the reward is based more on research inputs (competitive grants received) than on research outputs.
Targeted research projects

This is a second type of non-competitive funding. These projects have the advantage of being highly responsive to political pressures and short-run needs, can be funnelled to an institution that already has a comparative advantage in undertaking a projectperhaps due to past investment in facilities and people and are generally more efficient to contract than competitive
28
grants. They have the disadvantage of potentially being too influenced by political pressures and may not be awarded to the best institutions or scientists. Many donors blend all three funding mechanisms. Most of the donors who were interviewed provide part of their support to the CGIAR as core support. It reduces their ability to direct those funds to specific research areas beyond the commodity or other research focus of the institutions, but the funding mechanism is efficient and allows those close to the research problems to make key decisions on allocating the funds where they are most needed. Many also fund other projects competitively. To the extent they do fund programmes competitively or target specific topics, an extra onus is placed on them to conduct rigorous priority setting if they hope to achieve the greatest impact with those funds.
5.4 What comes first: the choice of institutional partner or the choice of research topic?
Some donors choose first to support the CGIAR or other partners and then select the research to be funded based on proposals they put forward, or allow the partner institutions to prioritise their own research. Others choose topics and then award them to institutions, competitively or targeted. There are reasons why both or a mix can make sense. GIZ channels almost all of its research funds to the CG centers. Given the size of the donor programme and the confidence the donor has in the quality of the institutions, there is logic to it. Other donors such as USAID award part of their funds to the CG before topics are chosenpart of them to topics first. This mix also makes sense for that donor. Not only because they are a large donor, but because the goals and geographic priorities are set at an institutional level above the level at which research priorities are set. Hence USAID needs flexibility to address the most important topics in a gradually changing set of country priorities and donor goals. The CG centers and a few other institutions are organisations that are uniquely placed to produce major international agricultural public goods. There is a sufficient degree of confidence in their ability to complete meaningful research based on their track record and they have checks and balances in place to ensure appropriate use of their resources. Supporting them first ensures long-term continuity of research effort on topics that require a long term commitment. Some domestic public sectors follow the same model of providing core funding to certain universities and other research institutions first and then choosing topics to award competitively. The US Land Grant System is a good example of this approach to funding.
29
6.0 Conclusions and recommendations

Most donors indicate they employ informal, semistructured processes to set agricultural research priorities. Some donors with larger technical staffs, such as USAID and DFID have at times engaged in formal priority setting, but most lack capacity, resources, or time to conduct formal, technical priority setting exercises. Donors are not alone. Few private companies or public agricultural research systems use formal, technical priority setting methods to any great extent either. One reason is the time and resources that would be required to quantitatively project research benefits in a rigorous manner for a large set of alternatives. However, based on a distillation of the literature and donor consultations, some practical recommendations can be made to improve the quality and efficiency of priority setting procedures used by some donors. These recommendations relate to process and tools. not involve simply identifying research topicsand it is important for the CG system itself to formalize more its system of priority setting, an issue currently under study. Nevertheless, for most donors it is helpful to explicitly identify alternative topics to prioritise. These options may include commodity and non-commodity research topics, along with a geographical focus. Researchable problems from the country and regional levels can be identified by local scientists, farmers, NGOs and donor personnel. Scientists in the donor organisation and at universities and national and international research organisations can identify what is possible. Scientists, economists and the private sector can identify alternatives to exclude because there are adequate private incentives to undertake the research. Some alternatives may be excluded because they are known to be addressed by other donors. Two or three rounds of consultations may be required to refine the list of alternatives to be prioritised. Projecting contributions of research on specific topics to specific goals can be accomplished by applying a subset of the tools discussed above, can make use of the results from meta-analyses of previous research on the topics, or can make use of indicators or theory. Most donors lack the internal capacity to directly apply quantitative priority setting tools. However, they could as individual donors, or as a group (platform), contract with certain research groups to apply the tools and supply analyses of estimated impacts of alternative research themes. HarvestChoice for productivity or income changes, HarvestPlus for micronutrient nutritional impacts and other groups for these and other impacts (e.g. environmental, poverty, gender) could be contracted for specific analyses to feed into priority setting exercises. The environmental goal is broad and might require multiple research groups for impact assessments, but resources could be targeted to a small set of impact assessment groups under a strict timeline. A formal prioritisation process is not something that is completed each year but should be undertaken when a new strategic plan is developed or at least every five years. Adjustments to priorities can be made more frequently and targeted impact assessments can be used to inform those changes.
Recommendation 1 Systematize the prioritising process

A five-step sequence of practices can be useful for setting priorities for most donors. Those practices involve: 1. Identifying goals 2. Specifying potential alternatives (topics) to be prioritised 3. Projecting contributions of research topics to the goals 4. Considering tradeoffs associated with alternative priorities 5. Comparing priorities to the current portfolio of topics and vetting any changes against political acceptability In some cases the goals are specified in the donor organisation above the level responsible for agricultural research. The goals are often multiple but explicitly stating them is essential for priority setting. For some organisations such as GIZ, most of the agricultural research budget is allocated to the CG system, primarily to the core or to the CRPs. Most donors allocate a significant share of their total agricultural research budget to the CG system even if it is not the majority share. Therefore, specifying alternatives may
30
Recommendation 2 Contract with appropriate groups to project impacts

Donors could individually, or acting as a group, contract with impact assessment groups to apply the priority setting tools and supply analyses of estimated impacts. Many of these groups already have models and some of the data. The groups could also be asked to complete meta-analyses.
One of the most important priority steps for a donor to undertake is to carefully compare suggested priorities with the current research portfolio. For political and other reasons, there can be a tendency to stress new programmes, but often the highest payoff can be found in added support to an existing programme. One implication is that additional effort may be needed in ex post evaluation of existing programmes to compare against projected benefits of new ones.
Recommendation 3 Insist the CGIAR do more priority setting

Because so many donor resources are directed at the CG system, donors should insist that the CG system do a more formal prioritisation itself of proposed research topics. This prioritisation process could be led by the Standing Panel on Impact Assessment (SPIA) and SPIA could contract out with appropriate groups as necessary.
Recommendation 5 Use quantitative tools where practical and credible

Impact assessment is itself research and can absorb significant resources. So tools must be chosen wisely and choices made on the level of rigour and resources to devote to priority setting analyses. For the goal of improving agricultural productivity, income growth and food security, economic surplus and benefit cost analysis are time-tested methods that can be combined with other methods to provide donors with projected benefits and tradeoffs. Application of these tools can be contracted for during priority setting exercises. Orwith a small amount of in-house (masters level) expertisesimple spreadsheets can be used to generate rough projections of benefits and these benefits include in a congruence analysis. For the nutrition/health goal, the size of the target group, the incidence of the problem in the target group, the mortality rate or degree of disability associated with the problem deficiency in each target group and the projected effects of the research on reducing the problem are key factors determining research benefits. IFPRI or other groups can be contracted with to apply the DALY approach for certain topics. Key factors influencing poverty effects of research on producers are the number of poor in a target audience, the depth of their poverty, the income provided by the research results or technology and the adoption of the technology by the poor. Factors affecting poverty effects on consumers are the relative importance of the commodity in question in the expenditures of the poor and the expected fall in market prices due to the research. Information is needed on consumption patterns of targeted poor groups to predict these poverty effects. Use of FGT type poverty indicators is relatively simple, but the data necessary to compute the indicators may be absent in the time frame required for the priority setting and poverty effects of agricultural research on consumers are driven in part by food price changes. Consequently even simpler indicators than FGT may be best for projecting poverty impacts such as the importance of a specific food such as staples in the diet of the poor or the potential of improved high-value crops to provide higher incomes to the poor.
Recommendation 4 Make use of accepted theory and available information

Even without using quantitative tools, donor priority setting processes could make additional use of available information and accepted theory. For example, income and productivity effects of research are largely determined by the base value of production associated with a commodity, the expected per unit cost and yield changes if the research is successful, the likelihood of research success, projected adoption rate of the results and their timing. Therefore for research aimed at improving productivity and income, these factors should be explicitly considered. Poverty is highly geographic and, for producers, affected by the value of the products produced and by the risk. The urban poor are affected significantly by the price of the food products they consume most. Poor women especially are affected by low food prices. Tradeoffs can be considered when two or more goals are identified as being important. Improvement in one goal can mean sacrifice in another and the tradeoff can be quantified if quantitative tools are used when evaluating impactsor at least discussed if only qualitative methods are used. Agricultural research is felt by many to have a comparative advantage in meeting some goals such as improved productivity and income more than others such as certain environmental goalsfor which institutional change may have a comparative advantagebut even that conclusion depends on the problem, the type of research that might be undertaken and alternative policy levers.
31
Because the environment is multidimensional, a practical method for assessing environmental sustainability is to rely heavily on physical effects of the research across specific environmental categories. In some cases, those effects can then be valued using market or non-market techniques, but often the cost and imprecision of the methods for valuing environmental impacts makes them not worth the effort. Availability of data to support environmental impact assessment varies by category of environmental impact and the data are often rough. For major environmental topics such as climate change, information from global models can be used to predict the location of heat stress, flooding and other effects.
Recommendation 6 Ensure that priority setting methods help lead to tangible results in farmers fields
One of the biggest concerns voiced by donors is that agricultural research must be prioritised in a way that eventually leads to tangible results in farmers fields. The research needs to be embedded into broader development processes. Research priorities may contain a mix of long- and short-term research so that shortterm development impacts can be realized without sacrificing long-term gains. Country offices of donors are usually more willing to support technology transfer programmes than research, making the programmes of these offices complementary to agricultural research, which is usually centrally supported. With sufficient communication between units that support research and field offices, research programmes can be aligned with priority needs for technology transfer. Complementary investments in infrastructure, training, education and extension can be used to facilitate adoption of research results. When identifying research alternatives to prioritise, it is essential to obtain input from field offices, NARS, NGOsmany of which work with technology transfer programmesand other local experts to identify the most pressing researchable problems. Relieving the most pressing constraints encourages technology adoption. Involving host-country rsearchers particularly on field-level trialscan create a stronger link between the research programme and technology transfer.
It is not necessary or even desirable that agricultural research programmes themselves spend significant resources on diffusion of their results. However, every agricultural researcher and research programme should have a realistic plan for how their results will be diffused to have a development impact. In some countries the entities doing the diffusion may be public extension services, private firms, NGOs or others, but the agricultural research programmes should identify the units and how their results will be transferred to themor to applied researchers in NARS or the private sector in the case of upstream research. Many social scientists are trained in how to assess constraints to adoption of agricultural interventions including technologies and institutional changes. Including them in research can reduce production of research results that just sit on the shelf.
Topics for further investigation

Several issues that were beyond the scope of this report could be useful subjects of future investigation. 1. A more detailed accounting of current allocations by donorsas proportions of their total support for agricultural researchto the CGIAR and other institutions might be helpful 2. It might be useful to collect data on the extent to which donors currently align their investments with each other in national and regional programmes 3. A more detailed accounting of existing research centers that could provide individual donors or a donor group with data and other analyses for priority setting might be useful
32
7.0 References
Alston, J.M., C. Chan-Kang, M.C. Marra, and T.J. Wyatt, (2000) A Meta-Analysis of Rates of Return to Agricultural R&D: Ex Pede Herculum, Research Report 113, International Food policy Research Institute, Washington, D.C. Alston, J.M., G.W. Norton, and P.G. Pardey, (1995) Science under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting, Cornell University Press, Ithaca, New York, 1995. Antle, J. (2012), Parsimonious multi-dimensional impact assessment, American Journal of Agricultural Economics, (in press). Banerjee, A., S. Cole, E. Duflo, and L. Lindebn, (2007) Remedying Education: Evidence from Two Randomized Experiments in India, Quarterly Journal of Economics, (122)(3): 1235-1264. Benbrook, C.M., D.L. Sexson, J.A. Wyman, W.R. Stevenson, S. Lynch, J. Wallendal, S. Diercks, R. Van Haren, and C.A. Granadino, (2002) Developing a pesticide risk assessment tool to monitor progress in reducing reliance on high-risk pesticides. American Journal of Potato Research. (79): 183-199. Black, F. and M. Scholes, (1973) The Pricing of Options and Corporate Liabilities, Journal of Political Economy, (81) (May-June): 637-54. Clemen, R. T., (1996) Making Hard Decisions: An Introduction to Decision Analysis, 2nd ed. Duxbury Press, Belmont, CA. Cuyno, L.C.M., G.W. Norton, and A. Rola, (2001) Economic analysis of environmental benefits of integrated pest management: a Philippines case study. Agricultural Economics (25): 227-234. de Janvry, A., A. Dustan, and E. Sadoulet, (2010) Recent Advances in Impact Analysis Methods for Ex Post Impact Assessments of Agricultural Technology: Options for the CGIAR, Report to the CGIAR Standing Panel on Impact Assessment, October, Berkeley, CA. Dufflo, E., R. Glennerster, and M. Kremer, (2008) Using Randomization in Development Economics Research: A Toolkit. In Handbook of Development Economics, Vol. 4, ed. T.P. Shultz and J. Strauss, pp. 3895-962, North Holland, Amsterdam. Gregory, R., B. Fischhoff, T. McDaniels, (2005) Acceptable input: Using decision analysis to guide public policy deliberations. Decision Analysis (2)(1) 416. Hajkowitz, S., (2002) Regional Priority Setting in Queensland: A Multi-Criteria Evaluation Framework, CSIRO, Australia. Huchzermeier, A. and C. H. Loch (2001) Project Management Under Risk: Using the Real Options Approach to Evaluate Flexibility in R&D. Management Science (47) (1): 85-101. Karlan D. and J. Apple, (2011) More Than Good Intentions: How a New Economics is Helping to Solve Global Poverty, Dutton, New York, New York. Keeney, R., H. Raiffa, (1993) Decisions with Multiple Objectives, 2nd ed. Cambridge University Press, New York. Khandker S.R., G.B. Koolwal, and H.A. Samad, (2010) Handbook on Impact Assessment: Quantitative Methods and Practices, The World Bank, Washington, D.C. Kovach, J., C. Petzoldt, J. Degni, and J. Tette, J., (1992) A Method to Measure the Environmental Impact of Pesticides. New Yorks Food and Life Sciences Bulletin, Number 139, Cornell University, New York State Agricultural Experiment Station, Geneva, New York. Manyong, V, A. Bamire, I. Sanusi, and D. Awotide, (2004) Ex Ante evaluation of nutrition and health benefits of biofortified cassava roots in Nigeria: The DALYs approach. Paper presented at the African Association of Agricultural Economists Association, 6 December, Nairobi, Kenya. Merton, R.C. (1973) Theory of Rational Option Pricing, Bell Journal of Economics and Management Science, (4)(Spring): 141-83. Moyo, S., G.W. Norton, J. Alwang, I. Rhinehart, and C.M. Deom, (2007) Peanut Research and Poverty Reduction: Impacts of Variety Improvement to Control Peanut Viruses in Uganda. American Journal of Agricultural Economics, 89(2)(May): 448-460. Mullen, J. D., G.W. Norton, G.W., D.W. and Reaves, D.W., (1997) Economic analysis of environmental benefits of integrated pest management. Journal of Agriculture and Applied Economics (29): 243-253.
33
Murray, C. and A. Lopez, (Eds.), (1996) The Global Burden of Disease. Volumes I and II. Harvard University Press, Cambridge, M.A. Mutangadura, G. and G.W. Norton, (1999) Agricultural research priority setting under multiple objectives: an example from Zimbabwe. Agricultural Economics (20): 277-286. Neely, J. E. and R. de Neufville. 2001. Hybrid Real Options Valuation of Risky Product Development Projects. International Journal of Technology, Policy and Management Nguema, A., G.W. Norton, M. Fregene, R. Sayre, and M. Manary, (2011) Expected economic benefits of meeting nutritional needs through bio-fortified cassava in Nigeria and Kenya, African Journal of Agricultural and Resource Economics, (6)(1)(March): 70-86. Penrose, L. J., W.G. Thwaite, and C.C. Bower, (1994) Rating index as a basis for decisionmaking on pesticide use reduction for accreditation of fruit produced under integrated pest management. Crop Protection (13): 146-152. Ravallion, M., (1992) Poverty Comparisons: A Guide to Concepts and Methods. Living Standards Measurement Study Working Paper No. 88. Washington, DC: The World Bank. Robin G., B. Fischhoff and T. McDaniels, Acceptable Input: Using Decision Analysis to Guide Public Policy Deliberations, Decision Analysis 2005 2:4-16; doi:10.1287/deca.1050.0035 Santiago L.P. and P. Vakili. 2005. On the Value of Flexibility in R&D Projects, Management Science, (51)(8)(August): 1206-1218 Saaty, T.L., (2001) Fundamentals of Decisionmaking and Priority Theory. RWS Publications, Pittsburgh, Pennsylvania. Stein, A, J. Meenakshi, M. Qaim, P. Nestel, H. Sachdev,and Z. Bhutta, (2005) Analyzing the health benefits of biofortified staple crops by means of the disabilityadjusted life years approach: a handbook focusing on iron, zinc and vitamin A. Harvest Plus Technical Monograph no. 4, International Food Policy Research Institute and International Center for Tropical Agriculture, United States and Colombia.
Swinton, S. M., N.N. Owens, and E.O. van Ravenswaay, (1999) Health risk information to reduce water pollution. In F. Casey, et al (eds.), Flexible Incentives for the Adoption of Environmental Technologies in Agriculture, Kluwer Academic Publishers, Boston, pp. 263-271. Teague, M.L., H.P. Mapp, and D.J Bernardo, (1995) Risk indices for economic and water quality tradeoffs: an application to Great Plains agriculture. Journal of Production Agriculture (8): 405-415. Trigeorgis, L. (1997) Real Options. MIT Press, Cambridge, MA. Zimmermann, R. and M Qaim, (2004) Potential health benefits of Golden Rice: a Philippine case study. Food Policy 29(2): 147-168.
34
Appendix 1:
Formal approaches for determining the amount to invest in research versus alternative interventions under risk
Discounted cash flow/NPV approaches
The traditional approach to valuing uncertain R&D projects is a discounted cash flow-based approach. This method produces an expected net present value (NPV) reflecting the expected discounted returns minus discounted costs. Key criticisms of these NPV approaches are: 1) they focus on a single stream of benefits and costs and 2) they assume that at the start of the project, decisionmakers are completely committed to the project. A single stream of benefits and costs is inappropriate because risks differ by investment and different risks imply different cash flows. The assumption of a complete commitment is also inappropriate; management flexibility should be incorporated into the decision framework. Flexibility comes from the ability of managers to stop unproductive lines of research and development. For example, in pharmaceutical research, the most costly phase of product development occurs when randomized trials are started. Unproductive lines can be ended as information is gained, perhaps about the regulatory environment or about consumer demand for a product. For agricultural variety research, the decision to release a crop variety triggers a series of expenses (seed multiplication, mobilization of outreach channels, advertisement, etc.) that can represent a substantial share of the overall cost of the new innovation. If the major risk occurs before the variety is releasedand before these large expenditures must be madethe value of a particular (risky) research project increases. Decision trees can be used to consider different outcomes and multiple scenarios, can effectively incorporate risk and allow the analyst to include management flexibility (Santiago and Vakili, 2005). They also have the advantage of being relatively simple and intuitive. The combination of NPV approaches with decision trees, however, is not appropriate for projects of long duration, where investments continue over time and decisions to continue or terminate them can be made at multiple points. These characteristics are clearly present in the case of agricultural research where expenditures on licensing, multiplication and outreach come late in the process and can represent important components of the total research budget. The reason for this inappropriateness is that a decision tree analysis assumes that the discount rate is constant over the entire life of the project (Neely and de Neufville 2001). Discount rates should depend on the relative risk associated with a given situation and this risk varies over time, as information becomes available and as markets and other institutions evolve. If a pure decision-tree approach were to be taken, the discount rates would have to be re-calculated at each decision point and for each scenario. Such a process would be unwieldy and too costly to implement annually or whenever major investment decisions are made.
Decision analysis
Decision analysis is an obvious approach to evaluating risky situations; it has the benefit of being able to deal with multiple scenarios and incorporate management flexibility (Keeney and Raiffa, 1993). Decision analysis is a broad area of theory and practice that structures choices in terms that 1 from the formalisms of decision theory and utility analysis (Clemen 1996, Keeney and Raiffa 1993; Gregory, et al. 2005). It begins by requiring that decisionmakers understand and identify issues that most matter to them. They must also be able to articulate the relative importance of different outcomes to their own goals. With this information, a decision model can be created.
Finance theory and real options

Finance theory provides perhaps another way of valuing managerial flexibility and the fact that discount rates change as decisions are made, as information becomes available and as conditions change. Financial option analysis shows that increased variability or volatility leads to a higher option value. If this option value is not considered, conventional analyses will likely undervalue risky investments. The value of the option arises because there is no obligation to exercise an option. Therefore the option holder does not suffer a higher loss from increased volatility when the underlying asset loses value. Increased volatility leads to a higher pay-off when the value of the underlying asset increases and the option holder exercises the option. In finance, the option is an option to buy or sell, but when option theory is applied to investments, the option becomes real. For example, following investment in agricultural variety research, the decisionmaker can decide whether to proceed with licensing, multiplication and distribution; such pathways are real options. Real option analysis provides a theoretically consistent means of valuing risk and including this value when evaluating investment options.
35
In finance theory, Merton, Black and Scholes developed the basic techniques for the valuation of options based on historical risk as reflected in market fluctuations in specific assets relative to the overall market (Merton, 1973; Black and Scholes, 1973). These models can be directly applied to financial assets or commodities for which historical pricing records exist. The main challenge with application of a real options approach is that these historical records (and, hence, measures of volatility) do not exist for many investment options. For example, in agricultural biotechnology research, there are few historical precedents for measuring scientific uncertainty, regulatory uncertainty and market uncertainty, as the products are completely new. Direct use of real options is thus constrained by lack of basic data on relative risks, but insights from this theory have relevance to research decisionmaking. Information about the five risks associated with research reaches decisionmakers at different times. According to a real options approach, each information event increases the value of managerial flexibilitythe ability to abandon research lines or alter the research portfolio. Five basic sources of management flexibility have been identified (e.g., Trigeorgis 1997). A defer option refers to the possibility of waiting until more information has become available. An abandonment option offers the possibility to invest in research in stages, deciding at each stage, based on accumulated information, whether to proceed further. An expansion or contraction option represents the possibility to adjust the scale of the research line as information becomes available. Finally, a switching or improvement option allows the decisionmaker to adjust research methods or the focus within a research line. The value of flexibility depends on when information becomes available relative to when the management decision must be made. If information becomes available after a decision point, the value of flexibility is reduced. These considerations are especially important to decisions about research versus alternative development investments. While the time profile of research costs vary by research programme, in general, the stream of information about potential success grows over time and allows processes to be ended prior to incurring key expenditure outlays. The specific timing of resolution of research uncertainty (compared to alternatives) may help explain under investment in research. Decisionmakers fail to consider the value of these real options and, thus, tend to under-value risky research.
36
Appendix 2:
Description of quantitative methods for assessing impacts of agricultural research
Meta-analysis
Meta-analysis is the analysis of multiple studies to draw inferences about common parameters or results. It can be particularly useful for linking ex-post study results to ex-ante projections. For example, there are many previous estimates of rates of return to agricultural research on different commodities (Alston et al., 2000). Another example is the widespread use of the benefit transfer method in the environmental economics literature(http://www.ecosystemvalution.org/benefit _ transfer.htm). This method uses findings from various studies of values of environmental change to infer that a similar change in a different area will have a similar value (hence the wording, benefit transfer). If a research organisation was interested in, say, the likely environmental impact of research on a new variety, the analyst might ask what impact have similar varieties had in similar or even different environments and use this information to project a value of an environmental impact. Meta-analyses that can draw on many previous studies or on ones that closely match the current research alternative being evaluated are the most useful. analysis assumes that these other factors are less important to maximizing total economic impact than is the initial value of production of the commodities. Alternatively, it might assume that reliable information about these other factors is hard to come by, while the value can be measured with relative certainty. It also is seldom used to assess relative impacts of research contributions to objectives besides maximizing total economic impact. Therefore it congruence may be a reasonable starting point in some situations, but is seldom a sufficient ending point.
benefit-cost and economic surplus analysis

The most widely used method to project impacts of agricultural research investments is to apply benefitcost techniques. Methods to conduct such studies have been refined and tested in many environments. They are widely accepted by the economics profession. Expected changes in yields, net cost reduction per unit of output, or other measures of productivity improvements are combined with projections of the number of units affected over time. Information on these changes may be gathered from various sources. For example, extension agents might project adoption rates over time to generate information on the total number of units affected. Projected probabilities of research success are included in the analysis; these projections might be based on scientist interviews or historical success rates for similar lines of research. Benefits and research costs are projected, year by year, for say 15-20 years and a rate on return on investment is calculated or a net present value of total benefits from the research investment. Because the same benefits received sooner are worth more than those received later, a discount rate is applied to the benefits and costs when the net present value is calculated so that comparisons can made with alternative investments that differ in their expected time frames. If the research impacts are expected to be felt over a wide area, the added production that results from the research might affect the price of the commodity. Therefore the markets themselves can be modeled and changes in economic surplus calculated, to account for this price effect. Economic surplus changes represent income gains (and/or losses) to producers from producing more of the commodity at a lower cost per unit of output but selling this output at a lower price and to consumers from having a greater supply of the product at a lower price. Surplus transfers to consumers from lower commodity prices can be substantial and can lead to large reductions in poverty and
Congruence analysis
Examining the economic importance of commodities is a logical starting point for assessing the aggregate economic impact of an agricultural research programme. Congruence analysis treats economic importance of commodities as a starting and ending point and simply checks to see if research expenditures on commodities occur in proportion to the value of the commodities. In other words, if one commodity has twice the value of production of another, it should have twice the research expenditures. If these proportions are equalized across the entire portfolio, then congruence is attained. There are two main reasons that congruence analysis has been popular. First it is simple. Second, the total economic value of a research programme does depend in part on the economic importance of a commodity. The problem of course is that it also depends on the probability of research success, likely adoption rates, likely research-induced productivity gains, timing of technology release, discounting and other factors. In other words, use of congruence
37
improvements in human nutrition and health. This latter point is often forgotten, but much of the poverty reducing effects of the Green Revolution came through lowering prices of foods thus allowing poor consumers to consume more. Food consumption increases (within reason) can obviously improve human health and nutritionand while surplus changes do not measure these factors, they are important. Calculation of economic surplus changes, rates of return and net present values has become a relatively routine practice for evaluating impacts of agricultural research that result in yield and cost changes. It is used for evaluating completed research and for projecting future research benefits. Numerous examples exist in the literature (see Alston et al., 2000 for a sample list). Many more examples are found in unpublished reports because the approach is so commonly used that it can be difficult to publish the results of such routine analyses in journals. Simple spreadsheet programmes are available for entering the data and doing the calculations. A programme called DREAM can also be freely downloaded from the website of the International Food Policy Research Institute (IFPRI) at http://www.ifpri.org/dataset/dream-dynamic-research-evaluation-management. The advantage of spreadsheet applications is that they be applied with a short learning curve and are relatively transparent. Any application is only as good as the data used in it, but spreadsheets can be used to generate a series of what if results for a variety of alternative and transparent assumptions on yield or cost changes, output prices, probabilities of research success, technology adoption rates, time frames and so forth. One challenge here is that when one projects benefits over many research programmes, it can be difficult to consistently identify the risks associated with them, including correlations of risks across programmes. A disadvantage of the spreadsheet approach is that many spreadsheets may need to be generated for multiple countries or regions. The advantage of the DREAM programme is that it can be used to project impacts of research in several countries or markets at once in a menu driven programme. A joint research project between the University of Minnesota and IFPRI called HarvestChoice has gathered the necessary data for 16 major crops to project and map the impacts of research to relieve major pest and abiotic constraints to production http://harvestchoice.org/. The data either already are or soon will be available from that project for most of the developing world and at a disaggregated level within countries. Donors can tap into this resource to project impacts of productivityenhancing research for 16 crops and 5 livestock groups (barley, wheat, rice, maize, millet, sorghum, cotton, potato, sweet potato/yam, cassava, banana/plantain, su-
garcane, beans, soybeans, coffee, groundnuts, cattle goats, sheep, pigs, and poultry). HarvestChoice tools have been supported and used by the Bill and Melinda Gates Foundation (Gates Foundation) and by the U.S. Agency for International Development (USAID) with its Feed the Future (FtF) programme. The initial prioritisation and targeting of the focal areas and systems for FtF was undertaken using HarvestChoice data and analysis. One limitation of the HarvestChoice data set is its limited set of commodities and its limited ability to project impacts for donor goals beyond productivity gains.
Non-market valuation methods

Many impacts of agricultural research, especially some of those that affect the environment and health are not priced in the market. These impacts can be measured in physical terms and, in some cases, valued using non-market valuation methods. Research can affect many environmental and health sub-components (soil, water, air, biodiversity, mammal, birds, fish and so forth) in the immediate and the long term. Therefore, assessing its impacts first requires projections on specific physical changes that may result from the research. If those changes are significant, other tools then can be applied to value them. Or, without valuation, the decisionmaker can be presented tradeoffs in terms of physical changes. For example, new pest resistant crop varieties may reduce pesticide use. Projections would need to be made about which chemicals would be reduced, how much and where they would be applied. A careful projection of the impact would also consider the toxicity of the chemicals to humans, birds, fish, groundwater and categories of the environment. This information would most conveniently be obtained from a metaanalysis using benefit transfer techniques. If an economic value were to be placed on the projected chemical reduction, values would need to be elicited on willingness-to-pay for the risk reductions, or market information on the values of time or avoided costs might be used in some cases. Notice that these values would be in addition to the cost-reducing impacts of the new varietieslower pesticide use would reduce the unit cost of production and induce market-level changes in producer and consumer surplus. The environmental assessment would thus add to the precision of the market-level analysis of costs and benefits. Economists have developed several methods for eliciting information on willingness-to-pay, with contingent valuation and choice experiments being the most commonly used (http://www.ecosystemvaluation.org/103.htm). Contingent valuation uses a survey to collect data on peoples stated willingness to pay to receive a benefit or their willingness to accept compensation for a loss. Examples applied to agricultural research are
38
provided by Mullen et al., 1997; Swinton, et al., 1999; and Cuyno et al., 2001. The choice method imputes willingness to pay, but does not ask it directly of respondents. In a priority setting exercise, however, such surveys would be costly and it would be best to rely on literature values of potential environmental costs and benefits. Some impact assessments use location-specific models that require detailed field information such as soil type, irrigation system, slope and weatherproduce information on the fate of chemicals applied (Teague et al., 1995). These model results can be valued using damage estimates or willingness-to-pay techniques. Most assessments of health and environmental effects have used non-location specific models that require information only on the type of impact, such as pesticides applied and their method of application amount of soil lost, amount of carbon sequestered and so forth to produce indicators of risks by health and environmental category. Some methods weigh the risks across environmental categories to create an index. Examples of the latter are the environmental impact quotient (EIQ) developed by Kovach et al. (1992), the Pesticide Index (PI) of Penrose et al. (1994) and a multi-attribute toxicity index developed Benbrook and others (2002). Because of the subjectivity of the weights used in multi-attribute indexing methods such as the EIQ, other methods can be used to elicit estimates on individuals willingness to pay for risk reduction for the various health and environmental categories and those estimates can then be used as weights.
Zimmerman and Qaim (2004) calculated the economic value of DALYs saved in the context of rice biofortification in the Philippines. Subsequent studies have applied the method to vitamin A, iron and zinc for several other crops and countries (e.g., Manyong et al., 2004; Stein et al. 2005; Nguema et al., 2011). The number of DALYs lost to disease are calculated as the sum of years of life lost due to preventable death and years lived with illness or disability from a preventable disease or health condition. The calculation of DALYs lost for say a particular micronutrient deficiency in a specific country would require identifying functional outcomes (e.g., night blindness, increased child mortality) associated with the deficiency as well as the affected target groups (e.g., children less than five, lactating women) and the size of those groups. DALYs take in to account the total number of people in a target group, the mortality rate associated with the deficiency in the target group, the average remaining life expectancy for the target group, the incidence of each disease in the target group, or the percent who suffer from the disease or health condition. A disability weight is placed on the disease in the target groupthat is, the associated degree of disability of each health outcome, which can vary from 1 for someone who dies to a small fraction for a somewhat minor disabilityand the duration of disease in the target group. Finally, a discount rate is placed on future life years, which accounts for the fact that losses that occur closer to the present are worth more than those occurring later. The numbers of DALYs currently lost in a country for a particular health problem is calculated and the projected reduction in DALYs lost if the agricultural research succeeds and the intervention is adopted. For example if a new vitamin enriched cassava variety is developed and adopted, the expected percentage reduction in DALYs lost may be calculated. This percentage depends on the assessment of expected incidence for the nutrient deficiency-related diseases. Those expected incidence in turn depend on the nutrient quantity and bioavailability in the new crop varieties, the effect of the added available nutrients on the functional health outcomes and the quantity of the new cassava variety consumed. The health impacts of the new crop depends on the quantity consumed, which is determined by both the quantity of cassava currently consumed and the adoption rates for the new variety. The difference in the number of DALYs lost with and without biofortified cassava represents the health impact of the biofortification; once again, these health impacts would be in addition to market-level projections of consumer and producer surplus changes.
Disability-adjusted life years

Quantifying expected health benefits of agricultural research involves an assessment of the likely changes in health factors and in some cases assessing the economic value of those changes. It is exceedingly difficult to establish a true causal link between research and nutritional outcomes, in part due to the complexity of the impact pathway. Research induces relative changes in prices and incomes and specific information on changes in nutrient consumption is difficult to come by. Economic incentives also affect household adoption of research-based versus alternative nutritional interventions. A common method used to assess the impacts of nutritional and health impacts interventions is an index measure called Disability-Adjusted Life Years (DALYs). The DALY approach was first described by Murray and Lopez (1996) as a means of capturing health effects in a single index that combines the number of years of life lost and the number of years lived with temporary or permanent disability due to a given health problem. Widely used in the health field since the 1990s, DALYs have more recently been applied to evaluate health impacts of agricultural research that affects nutrition, especially intake of micro-nutrients. For example,
39
There are multiple ways to assign an economic value to the DALYs saved through biofortification. Discounting and summing the DALYs saved from the year of variety release until say 20 years hence, and dividing that sum by total expenditures on research and development, provides the cost per DALY saved, which can be compared to alternative means of meeting the nutrient requirements. Alternatively, each DALY saved could be assigned a subjective dollar value in order to determine the total value of lives saved and disability avoided due to the micronutrient enhanced varieties. The advantage of this alternative is that the values could be directly compared to surplus changes. Its main disadvantage is that placing a value on human life is subjective and subject to considerable controversy.
policies on sub-groups (Ravallion, 1992). The FGT index takes into account the number of poor people and also the severity of their poverty. Use of a FGT index is now standard in many poverty assessments around the developing world. Moyo et al. (2007) used this method to assess poverty impacts of improved virus-resistant peanut varieties in Uganda. The main difficulties in using this method for priority setting is predicting how much productivity will change and predicting the patterns of technology adoption. It also requires information on consumption patterns of the poor, as decreases in prices of commodities primarily consumed by the poor will have different poverty effects than other commodity price changes.
Randomized control trials Poverty rate analysis

The question for donors is often not how much a research programme affects income, but how much it affects poverty. New agricultural technologies can lower per-unit costs of production and output prices, increase the supply of food products and may raise incomes of adopting producers. The poor can gain disproportionately as consumers from lower food prices, as they spend a high proportion of their income on food. Poor consumers gain most when prices of goods they consume most, such as staples and highenergy foods, fall. Analyses of predicted changes in poverty resulting from adoption of a new technology involve three main steps: Computing the household-level value of the income or consumption per capita prior to adoption of the technology and comparing it to the poverty line; Determining which households are most likely to adopt the technology and estimating how household income/consumption will change following adoption (note that some of these are farm-household adopters, but much of the reduction in poverty comes through markets as prices of food fall); and Adding up the change in the number of poor people or households resulting from adoption. The household analysis of projected real income changes among adopting households can be used to create an estimate of the aggregate of market-level changes (corresponding to the total change in income for all participants in the market) and of changes in poverty in the population. A measure called the Foster-Greer-Thorbecke (FGT) index is commonly used to add up poverty in a population and is useful because it allows evaluation of impacts of agricultural research and other government As noted above, the impacts of agricultural research can only be truly assessed by comparing what actually happened with what would have happened if the research had not been done. The problem is that other factors change before and after the research intervention or differ between groups with and without itand these changes or differences must be considered. Otherwise, there could be systematic differences between adopters and non-adopters of a research intervention. One suggestion for addressing this problem has been to conduct a randomized control trial (RCT), much like a clinical trial that drug companies do before seeking approval of a new drug. In an agricultural RCT, random farm-households or villages are assigned a new technological innovation while others are not assigned it (de Janvry et al., 2010). A new crop variety or management practice, for example, could be randomly tested on some farms but not others. Then if it works, the release of the technology could be scaled up. This approach is now being widely used for evaluating non-research-based development interventions (Banerjee, et al., 2007; Dufflo, et al., 2008; Karlan and Appel, 2011). It has the advantage of controlling for non-random effects of who receives training about a technology or the hidden characteristics of the adopters themselves that may differ from nonadopters. Unfortunately, it is significantly easier to apply RCTs to development interventions such as credit programmes, direct health interventions, educational interventions, non-research agricultural interventions and others that have well-defined alternatives. Because agricultural research has complex impact pathways, it is difficult to assign a research output to individual end users at random. This is because those who are selected need to be willing to participate and actively undertake actions over multiple seasons, and the large numbers of potential treatments are often not amenable to RCTs.
40
Given the long time period alone for the research, pilot tests are unlikely to yield results in time for research priority setting. Impacts of new technologies often take many years to emerge as market effects play themselves out over time. Also, donors are setting priorities over a large number of alternatives and individual RCTs are each research projects in themselves. It may be possible to use RCTs to explore alternative mechanisms for encouraging farmer adoption of technologies once they exist, but usually not for an impact evaluation per se of the technologies. Results from small-scale RCTs would be useful in providing information for ex-ante analyses of income or productivity changes if some of them could be completed over time to create a bank of results that could be incorporated into surplus models. Meta analyses of RCTs from different regions or countries could then help inform a surplus or other approach to priority setting.
Simulation/math programming models

Many types of simulation models are available to project impacts of agricultural research. They can be econometric-based, mathematical-programming-based and even economic-surplus based. They can include distributions around uncertain parameters. Antle (2012) presents an example of how assumptions or information about distributions can be used to estimate predicted effects of certain types of research. Some of the models can incorporate expert opinion. Others may combine multiple methods. For example, economic surplus results can be included within a math programming model. The models can incorporate multiple objectives and assess tradeoffs among objectives. For example, Mutangadura et al (1999) maximized total income in an agricultural-research-priority-setting model for Zimbabwe and then explored the implications of placing higher weights on benefits going to small communal farms. Because simulation/ mathematical programming models can be used for multiobjective impact assessment, they are discussed further below under the discussion of approaches for developing an optimal research portfolio given multiple objectives.
Econometric impact models

Econometric methods are frequently employed to address the counterfactual problem. These statisticalregression-based methods can be applied to assess impacts of agricultural research ex post and the results used to project which areas have the highest benefits in the future. There are a range of methods, each with their strengths and weaknesses. Propensity Score Matching can be used to compare impacts on those who adopt and do not adopt interventions generated by agricultural research. Adopters are matched with nonadopters based on a range of observed characteristics. It is assumed that selection bias in who adopts a technology is only based on these characteristics. Another method called Difference in Differences assumes that unobserved selection bias is present but that it is time invariant so that the effects of the research can determined by taking the difference in outcomes for adopters and non-adopters before and after the intervention. The most commonly used econometric method is an Instrumental Variables approach in which selection bias on unobserved characteristics is corrected by using a variable (instrument) that is correlated with technology adoption but not with unobserved characteristics affecting the outcome. The instrument is used to predict adoption. Assuming the bias problem is adequately handled, the main advantage of econometric methods is that they can use post-adoption data to assess research impacts and these impacts provide feedback to decisionmakers that may indicate where future research impacts may be highest. Here is an area where meta analysis of previous studies can be used as a guide.
Changes in simple indicators

Changes in simple indicators are frequently used by donors to evaluate the benefits of agricultural programmes including research. Indicators such as number of people affected, number of poor people affected, number of women affected, number of hectares affected, number of children under five affected and many more are frequently used to indicate impacts of programmes during and after the factand they are used to signal which programmes are likely to achieve their objectives. While such indicators may be useful for certain non-research programmes, their usefulness for agricultural research impact assessment is limited. Research takes time and its impacts depend on many factors in addition to research and therefore it is nearly impossible to attribute impacts to simple indicators. Even indicators such as number of technologies developed, which is used by one donor, are misleading because one highly successful technology can be worth many times more than five lowimpact technologies. In an ex-ante priority setting framework, these indicators are even less useful than in ex-post analysis because expected changes in them must be predicted for multiple research themes. Specific assumptions about likely adoption rates and patterns of adoption are needed and these numbers are exceedingly uncertain.
41
Scoring indices
Scoring models are frequently used to combine factors or multiple indicators of impact into one value (Hajkowitz, 2002; Saaty, 2001). They are commonly used in ex-ante settings. The weighting of research programmes across a wide set of criteria can appear rigorous and transparent because it is quantitative and criteria are explicit. Unfortunately, weighting of criteria can mislead because of unit differences and because unless the criteria are combined in specific ways, the result is usually meaningless. For example, the value of production of a commodity, the likely probability of research success, the expected per-unit cost reduction due to a technology and the adoption rate of that technology are multiplicatively related factors. In other words, when they are multiplied together, they give an approximate idea of the value of the research. But weighting these same factors and adding them, which is frequently done, gives completely misleading results. Measures of contributions to individual goals can be weighted once the criteria have been logically combined, but not contributions to disaggregated criteria. Scoring models are discussed again below because they are a common means for developing a research portfolio once individual impacts are assessed for multiple goals.
42
Appendix 3: Terms of Reference

Synthesis Paper on Agricultural Research Prioritisation
bACKGROUND
The Global Donor Platform for Rural Development (Platform) is a network of 34 bilateral and multilateral donors and international financing institutions that share a common vision of the role that agriculture and rural development (ARD) plays in reducing poverty. They are committed to achieving increased and more effective aid for agriculture and rural development. Within the next three to five years, the Platform wants to contribute to: an increase in the share of members ODA going to ARD; tangible progress in the implementation of Paris Declaration and Accra Agenda for Action commitments at country level; an increase in the use of programme-based and sector-wide modalities in ARD programmes; and a rationalisation of ARD support by member agencies in selected countries. The Platform develops knowledge-capture and -exchange initiatives in order to effectively advocate for improved agricultural and rural development assistance by strengthening existing and creating new networks. It seeks to influence all donors engaged in developing country rural development programmes at political and technical levels and will seek to enhance its influence by expanding its membership. The Platforms work is achieved through implementing twin pillars of advocacy and knowledge sharing. The Platform is currently chaired by the International Fund for Agricultural Development (IFAD). The Secretariat of the Platform is hosted by BMZ in Bonn, Germany and is managed by the German Agency for International Cooperation (GIZ). One of the objectives of the Agriculture Research and Development working group of the Platform is to improve knowledge and harmonization of donor methods to prioritise investments in agricultural research and development. To achieve this objective, a synthesis paper on agricultural research prioritisation is to be commissioned whose findings will be used to advocate for improved prioritisation of investments in agricultural research inside as well as beyond the platform network.
DESCRIPTION OF THE ASSIGNMENT

Long-term sustainability of agricultural production and market systems requires research. The synthesis paper on research prioritisation will be framed within the broader development assistance dialogue. Allocation of development resources among research and non-research activities is better informed when donors have information on norms and best practices to achieve sustainable development outcomes. The paper will include a literature review and consultation with donors to present a rich discussion on current and potential methods and to identify forward looking approaches for more effective and more efficient research prioritisation within the context of agricultural development strategies and programmes. It is expected to make this paper ready for dissemination at the second Global Conference on Agricultural Research for Development (GCARD) in late October 2012.
OVERALL ObJECTIVE
Improve knowledge of donor methods to prioritise investments in agricultural research and development
PURPOSES OF THE PAPER

1. To demonstrate the context in which investments are selected in agricultural research for development 2. To describe the methods for agricultural research prioritisation 3. To describe the mechanisms donors use to programme agricultural research resources 4. To make recommendations on how donors can use prioritisation methods, strategies to improve coordination and considerations from the priority setting literature
OUTPUT
1. The major output of the contract will be a synthesis paper (estimated 20-45 pages). Detailed outline of the paper is presented in the Annex. 2. A short policy brief (3-4 pages) will also accompany the paper. The policy brief will summarize the introduction, analysis and recommendations of the paper.
EXPERTS PROFILE
University degree (preferably PhD); At least five (5) years experience in the areas of agriculture, agricultural research, research policy, rural development and/or development cooperation; A sound understanding of the Platforms mandate on agriculture and rural development, knowledge of development donors and current global trends in agricultural research for development.
DURATION OF ASSIGNMENT
Up to 30 days starting as early as 20 March to 18 July, 2012.
43
REPORTING
Content: Final paper not exceeding forty (40) pages (A4). Approximately 600 words per page without graphics/ 450 with graphics. Assistance on graphics and layout etc will be given by staff at Global Donor Platform for Rural Development. Language: Style: Format: Dates: English Sharp, clear and concise writing. Naming the issues and when necessary, explaining the options and/or differences. Electronic in MS Word format Preliminary draft 1 June, 2012 Final draft 6 July, 2012
ANNEX: OUTLINE OF THE SYNTHESIS PAPER ON RESEARCH PRIORITISATION

Estimated length 20-45 pages
Primary objectives of paper:

1. To demonstrate the context in which investments are selected in agricultural research for development 2. To describe the methods for agricultural research prioritisation 3. To describe the mechanisms donors use to programme agricultural research resources 4. To make recommendations on how donors can use prioritisation methods, strategies to improve coordination and considerations from the priority-setting literature
Section 1. Setting the context of agricultural research for agricultural development (2-3 pages)
This study on research prioritisation will be framed within the broader development assistance dialogue. Allocation of development resources among research and non-research activities is better informed when donors have information on norms and best practices to achieve sustainable development outcomes. Though immediate humanitarian outcomes and development goals might be well served with a 100 per cent allocation to non-research programming, long-term sustainability of agricultural production (especially in a context of highly uncertain future economic, social, climatic.. conditions) and market systems requires research. In some research sectors (e.g., crop breeding) the private sector offers useful models for optimal allocations of resources for R&D versus all other business activities. Similarly, public sector agriculture institutions have developed models for optimizing allocations to research versus other activities. To address the main bottleneck of current support to agricultural research for development the study should clearly indicate pathways of technology transfer of achieved research results. This section is a brief reference to the broad and deep literature available on how to allocate budgets to research versus non-research investments. It is not expected to be an analysis of this area itself. I will include a broad overview of the quantitative and qualitative methods employed in this area, limited to some elements of comparison between them (strengths/weaknesses), and will mainly set up the context for research prioritisation"
Section 2. Agricultural Research Priority Setting (10-20 pages)

Description of existing agricultural research priority setting methodologies and information on how donors are using them. a. Defining the Methods: Review of the steps in agricultural research priority setting. I. The first section will describe the array of donor goals that can be used in the research prioritisation process. These might include, for example, the following: degree to which the proposed research contributes to achieving specific development objectives (e.g., womens or childrens nutrition), disciplinary emphasis (e.g., a preferred level of investment in biophysical vs. social sciences), length of time to output (e.g., short term, medium term, and/or long term), geographic emphasis, and scale of impact (e.g., global , with high potential for spillover vs local or household-level impact, with low potential for spillover). II. A second key input to research prioritisation is impact analysis showing the level of impact that can be achieved through investment in different research approaches (e.g., rate of return analysis, cost benefit analysis, qualitative analysis). Impact may be more easily measured for research with a narrow definition of outcomes, (e.g. crop breeding research with a focus on yield improvements) than research conducted at the landscape level, such as natural resources management research where benefits are dispersed and are not easily captured with one metric. Data availability is a key constraint to this step. Thus, multiple approaches for assessing impact may need to be brought together in complementary ways to enable comparisons of options, described in the next step. III. This section will describe how priorities can be established and a portfolio developed once impacts of all potential investments are estimated. To develop a portfolio of research options, analysis methods can explore how different combinations of research investments would achieve multiple goals (e.g., optimizing among poverty alleviation, increased Ag GDP growth, reduction in child mortality, etc.) and/or would integrate a treatment of uncertainty (preparedness for various future scenarios).. Depending on donor goals, a different portfolio of research priorities will emerge, yielding varying returns on investment over varying periods of time. Additionally, within a given research area, such as cocoa research in Cote dIvoire, such methods could
help determine which research areas contribute most to donor goals such as poverty alleviation , maximize export earnings, or contribute to the management of natural resources.
44
b. How to use the Methods: This section will describe what combination of methods donors can use to compose a portfolio and what they need to understand about these methods to use them effectively. It will facilitate understanding of how to use methods in complementary ways. c. Donor consultation: This section will be prepared on the basis of an informal consultation with 3-5 donors on current priority-setting mechanisms. Donors would be invited to come forward on a voluntary basis to provide information through various means (ie: informal interview or semi-structured survey). The following questions will be addressed: I. What priority-setting mechanisms are currently used? II. How do donors evaluate priority-setting mechanisms according to their: 1) ease of use; 2) provision of reliable evidence; and 3) persuasiveness in informing decisionmaking ? III. What are donor perspectives on the analytic methods from the academic literature, and do donors use these methods? If not, what is needed to make them more accessible/relevant/useful? Would multi-donor collaboration on this kind of analysis be useful? IV. How have donors learned from others? What kinds of formal or informal strategies do donors employ to inform their practice (e.g., involvement in the GDPRD agriculture research working group)? d. Multi-donor engagement: What are strategies for avoiding disproportionately high donor investments in some areas and under-investement in others (i.e., the donor pile-on effect). Window 1 funding through the CGIAR could be addressed as an example.
Section 3. Mechanisms for Investments in Agricultural Research (5-15 pages)

Description of mechanisms used by donors to invest in agricultural research a. What types of institutions do donors engage to conduct agricultural research for development, e.g.,university community, government agencies, NGOs and other research actors? How can the risk of disproportionate donor investments with certain institutions be managed? b. What mechanisms are used to invest in agricultural research: competitive grants, directed research, scholarships, university twinning, etc.? c. What are the pros and cons of competitive calls for proposal (open calls) versus targeted funding (institutional core budget or project support)? Can mechanisms be blended, and if so, is there guidance available on best practices? d. What comes first: the choice of institutional partner or the choice of research topic? (A donor may choose first to support the CGIAR or a NARS, and then select the research to be funded based on the proposal put forward by this partner) or a combination of both? Section 4. Conclusions & Recommendations (3-5 pages) This section should reflect a distillation of both the literature and the donor consultation. This will allow for a rich discussion of what is practical based on donor experience and forward thinking on how donor practice can be changed with evidence of more effective and more efficient approaches. It could also include: a. Additional recommendations from outside the ARD donor community including from the academic community or private sector This should be limited to the most recent or most innovative approaches being discussed in current fora. b. Specific attention to forward-thinking recommendations what might/should be different going forward given the goals that have been stated by donors in various fora (LAquila, etc.). c. Who takes responsibility for achieved research results in a way that impact on farmers field is visible?
Tackling rural poverty together
45
About us
The Global Donor Platform for Rural Development is a network of 34 bilateral and multilateral donors, international financing institutions, intergovernmental organisations, and development agencies. Members share a common vision that agriculture, rural development and food security is central to poverty reduction, and a conviction that sustainable and efficient development requires a coordinated global approach. Following years of relative decline in investment in the sector, the Platform was created in 2003 to increase and improve the quality of development assistance in agriculture, rural development and food security.
Prepared by: Platform secretariat Published by: Global Donor Platform for Rural Development Godesberger Allee 119 53175 Bonn, Germany Authors: George W. Norton and Jeffrey Alwang Cover picture: Neil Palmer (CIAT) 123RF, iStock, Fotolia September 2012
donorplatform.org
Contact: Secretariat of the Global Donor Platform for Rural Development, Godesberger Allee 119 53175 Bonn, Germany Phone: + 49 228 24934 166 Email: secretariat@donorplatform.org Website: www.donorplatform.org Publication date: September 2012

Platform Synthesis Paper - Donor Methods To Prioritise Investments in Agricultural Research and Development

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Platform Synthesis Paper - Donor Methods To Prioritise Investments in Agricultural Research and Development

Uploaded by

Copyright:

Available Formats

Donor Methods to Prioritise Investments in Agricultural Research and Development

About the Platform Synthesis Paper

From the Platform AR4D working group

Review of agricultural research priority setting methods

Recommendation 1 Systematize the prioritisation process

Mechanisms for investments in agricultural research

Recommendation 5 Use quantitative tools where practical and credible

Recommendation 2 Contract with appropriate groups to project impacts

Recommendation 3 Insist the CGIAR do more priority setting

Recommendation 4 Make use of accepted theory and available information

About the AR4D working group

Objectives of this synthesis paper

2.0 Setting the context of agricultural research for agricultural development

2.1 Investment allocations research versus alternatives

2.2 Theory of decisionmaking under uncertainty

2.3 Summary and implications

3.0 Agricultural research priority setting methods

3.1 The goals of donors

3.2 Defining the relevant alternatives

3.3 Projecting impacts of research

Table 1. Methods for assessing impacts of agricultural research

Ease of use for research impact assessment Medium-high High

Reliability of predicted impacts Medium-low Medium-low

Meta-analysis Congruence analysis

Medium Medium-low Low

Medium-low Medium High

Econometric impact models Simulation/math programming Changes in indicators Scoring indices

3.4 Comparing alternatives and establishing priorities

2. Nutritional and health improvements

4. Natural resource sustainability / environmental improvement

3.5 How to use priority setting methods

4.0 Donor consultations

4.1 Which priority setting mechanisms are currently used?

4.3 Donor perspectives on the analytic methods available in the literature

4.5 Multi-donor engagement

5.0 Mechanisms for investments in agricultural research

5.2 Mechanisms used to invest in agricultural research

5.3 Pros and cons of competitive versus targeted funding

Institution-based, core funding

Targeted research projects

6.0 Conclusions and recommendations

Recommendation 1 Systematize the prioritising process

Recommendation 2 Contract with appropriate groups to project impacts

Recommendation 3 Insist the CGIAR do more priority setting

Recommendation 5 Use quantitative tools where practical and credible

Recommendation 4 Make use of accepted theory and available information

Topics for further investigation

Finance theory and real options

benefit-cost and economic surplus analysis

Non-market valuation methods

Disability-adjusted life years

Randomized control trials Poverty rate analysis

Simulation/math programming models

Econometric impact models

Changes in simple indicators

Appendix 3: Terms of Reference

DESCRIPTION OF THE ASSIGNMENT

PURPOSES OF THE PAPER

ANNEX: OUTLINE OF THE SYNTHESIS PAPER ON RESEARCH PRIORITISATION

Primary objectives of paper:

Section 2. Agricultural Research Priority Setting (10-20 pages)

Section 3. Mechanisms for Investments in Agricultural Research (5-15 pages)

Tackling rural poverty together