UNIVERSITY OF ANTWERP

INSTITUTE OF DEVELOPMENT POLICY AND MANAGEMENT

Dissertation

THE IMPACT OF AGRICULTURAL IMPROVEMENT

INTERVENTION THROUGH MARKET
DEVELOPMENT ON THE LIVELIHOODS OF RURAL
FARMERS IN ETHIOPIA

A Propensity Score Maching Application

Wole Kinati Wakjira

Master of Development Evaluation and Management

Supervisor: Dr. Ruslan Lukach

Academic Year 2012-202013
ii
 UNIVERSITY OF ANTWERP

INSTITUTE OF DEVELOPMENT POLICY AND MANAGEMENT

Dissertation

THE IMPACT OF AGRICULTURAL IMPROVEMENT

INTERVENTION THROUGH MARKET
DEVELOPMENT ON THE LIVELIHOODS OF RURAL
FARMERS IN ETHIOPIA

A Propensity Score Maching Application

Wole Kinati Wakjira

Master of Development Evaluation and Management

Supervisor: Dr. Ruslan Lukach

Academic Year 2012-2013

iii
Preface

The conception of this dissertation is primarly based on my exposure to impact

evaluation approaches and more specifically to the advanced quantitative techniques
during my stay at IOB. Through the various policy-oriented research assignments,
exercises and theories, I gradually learned more concerning the specificities of
development oriented policy evaluatuation in general and impact evaluation in
particular.

The other reason that initiated me to consider this work is the current expansion in
developmenet oriented projects as a result of the recent rural developement strategy
launched by government in my country. In particular, the close relationship with and
the knowladge I had about one of such projects which I considered for this study
stimulated me to contribute to it based on the knowaldge I gained here.

In doing so, first and for most, I would like to extend my special thanks to the
government of Belgium for giving me the opportunity to persue my MSc study in
Development Evaluation and Management. I am so grateful to the project staff,
district experts and farmers representatives for their cooperation in giving me all the
data used for this study. I would also like to thank my families and friends for their
continued moral support. My special gratitude goes to Dr. Ruslan Lukach who
supervised me throughout the process of my work without whom this dissertation
could have been impossible. Finally, I would like to thank all IOB staff members for
every support they gave me to bring my dream true.

iv
 Table of Contents

List of Tables vii

List of Tables in the Appendices viii

List of Figures viii

List of Figures in the Appendices x

List of Acronyms x

Executive Summary xii

CHAPTER ONE: INTRODUCTION 1

1.1 Background 1

1.2 Statement of the Problem 4

1.3 Research Questions 6

1.4 Significance, Scope and Limitations of The Study 6

1.5 Outline of the Study 7

CHAPTER TWO: LITERATURE REVIEW 8

2.1 Market Development and Participation in the Market 8

2.2 Commercialization and Measuring Commercial Orientation 9

2.3 Market Institutions and Agricultural Marketing 12

2.4 Definitions, Types and Approaches to Impact Evaluation 15

2.4.1 Definitions and objectives of impact evaluation 15

2.4.2 Typologies of impact evaluation 16

2.4.3 Approaches to impact evaluation study 17

2.4.3.1 Experimental method 20

2.4.3.2 Quasi-experimental method 21

2.4.3.3 Choosing an estimator 29

2.4.3.4 Related empirical studies and application of PSM 30

CHAPTER THREE: RESEARCH METHODOLOGY 33

3.1 Description of the Study Area 33

v
 3.2 Data Sources, Sampling and Collection Techniques 35

3.3 Data Analysis Techniques 37

3.3.1 Qualitative and descriptive analysis 37

3.3.2 Empirical Analysis 37

3.3.2.1 Propensity score matching 37

3.3.2.2 Propensity scores estimation using binary response model 41

3.3.2.3 Choice of matching algorithm 42

3.3.2.4 Checking common support condition and testing matching quality 44

3.3.3 Choice of Variables for Logit Model and Indicators of Outcome Variables 45

3.3.3.1 Variables selected for propensity score estimation 45

3.3.4 Outcome variables and their measurement 47

CHAPTER FOUR: RESULTS AND DISCUSSION 49

4.1 Qualitative Approach 49

4.1.1 Organizational and institutional arrangements 49

4.1.2 Enhancing farmers’ capacity 53

4.2 Descriptive and Empirical Approaches 54

4.2.1 Descriptive results 54

4.2.1.1 Characteristics of sample households 54

4.2.1.2 Descriptive results of outcome variables 55

4.2.2 Empirical results 57

4.2.2.1 Estimating propensity scores 57

4.2.2.2 Matching participant and non-participant households 59

4.2.2.3 Choice of matching algorithm 60

4.2.2.4 Testing the matching quality 60

4.2.2.5 Estimating treatment effect on the treated 62

CHAPTER FIVE: CONCLUSION AND RECOMMENDATION 66

BIBLIOGRAPHY 69

APPENDICES 76

vi
 List of Tables

Table 1. Sample size by Kebele 36

Table 2. Observable covariates for selection bias adjustment, their description and
measurement 47

Table 3. Types and roles of stakeholders involved along the value chains of
commodities of intervention 50

Table 4. T-test for equality of means between groups (continuous variables) 54

Table 5. Chi-square test for equality of proportions between groups (dummy

variables) 55

Table 6. t-test for equality of means between groups for outcome variables 56

Table 7. Logit result of program participation 58

Table 8. Distribution of estimated propensity scores 59

Table 9. The balancing test before and after matching 61

Table 10. Estimated ATT for the outcome variables 64

vii
 List of Tables in the Appendices

Table 1. Conversion factor for labor supply (person day equivalent) 76

Table 2. Multicollinearity test for explanatory variables 76

Table 3. Multicollinearity test for explanatory variables 76

Table 4. Performance of different matching estimators 78

Table 5. Propensity score and covariate balance before matching 78

Table 6. Propensity score and covariate balance after matching 79

viii
 List of Figures

Figure 1. The evaluation problem—disentangling project effects from intervening

factors 19

Figure 2. Map of the study area 35

ix
 List of Figures in the Appendices

Figure 1. Kernel density of propensity scores of participant households after matching

for participant households 77

Figure 2. Kernel density of propensity scores of participant households after matching

for non-participant households 77

x
 List of Acronyms

AGP Agricultural Growth Plan

ATARC Adami Tulu Agricultural Research Center
ATT Average Treatment Effect on the Treated
APEDO Arsi zone Planning and Economic Development Office
CIA Conditional Independent Assumption
DD Double Difference
FAO Food and Agriculture Organization
FFS Farmers Field School
IFAD International Fund for Agricultural Development
IV Instrumental Variable
IPM Integrated Pest Management
IHEP Integrated Household Extension Program
GTP Growth and Transformation Plan
KLM Local Linear Matching
MoFED Ministry of Finance and Economic Development
NN Nearest Neighbor
OBoARD Oromia Bureau of Agriculture and Rural Development
ORA Oromia Regional Atlas
OCSSC Oromia Credit and Saving Share Company
RD Regression Discontinuity
PSM Propensity Score Matching
PSNP Productive Safety Net Program
SPSS Statistical Packages for Social Sciences

xi
 Executive Summary

The recent move towards the liberalization of foreign direct investment in developing
countries and the subsequent structural adjustment programs in many Sub-Saharan
African countries have liberalized their economies and formulated poverty reduction
strategies that are intended to open-up new market-led opportunities for economic
growth. Like many developing countries, Ethiopia has undergone several structural
adjustment programs in the past few decades that aimed at bringing economic growth
thus reduce poverty. A pillar of the country’s recent rural development strategy has
been the active promotion of marketing cooperatives as a means of commercializing
smallholder agriculture.

The recently formulated strategy of the country has emphasized on the transformation
of subsistence agriculture into market oriented patterns of production as a basis for
long-term development of the agricultural sector. Currently, several programs are
under way both by government and non-government institutions so as to realize the
strategy. In view of this, the regional research institute and government of Oromia
initiated a joint program on introduction of improved agricultural technologies and
input/output marketing to facilitate the development of marketable commodities. The
project focused on the value chain approach by encouraging the involvement of actors
along the value chains so that the experience generated would be scaled up to similar
areas in the region. Kofale district of the Arsi Zone was selected based on its potential
for the selected commodities of interventions which includes inset and fruit
cultivation, apiculture and fattening implemented as a pilot project primarily to raise
the income level of participant households through market development. In
consultation with the selected kebele administrations and development agents,
participant households were identified based on their willingness and capacity to start
up project activities. Accordingly, the project was implemented from 2009 to 2012
and claimed successful which created an interest among policy makers to quantify the
results.

Based on the request from the regional government, the project staff recently
administered household survey in order to generate information through impact

xii
evaluation regarding whether the project has achieved its ultimate objective of
increasing rural income. Accordingly, in the beginning of 2013, data were collected
from a total of 201 farm households coming from both participating and non-
participating kebeles within the same pilot district having similar socioeconomic and
agro-ecological conditions. For this purpose, 100 participant and 101 non-participant
households were sampled based on probability proportionate to sample size in order
to generate the data. Based on the collected data, the current study aimed answering
the questions of what changes were brought in the organizational and institutional
aspect of the agricultural input/output marketing in the pilot district as a result of the
intervention and what would have been the market orientation, participation,
productivity and income outcomes of participating households if the intervention had
not been in place.

In order to deal with the potential selection bias problems and address causality so
that the above research questions were properly tackled, we adopted propensity score
matching (PSM) as the best alternative analytical method available to estimate the
impact of the intervention based on the available data at hand. This is particularly
because of three reasons. Firstly, baseline date was not available for the program.
Secondly, treatment assignment was not based on random. Thirdly, PSM is assumed
as the second best alternative design available to that of experimental design in
minimizing selection bias (Baker, 2000).

The qualitative analysis indicated that the program has improved the linkages and
interaction among different actors along the value chains of the commodities of
intervention enhancing participant households’ access to different agricultural inputs
and outputs marketing services. On the other hand, after controlling for the initial
differences, the result of our impact estimate revealed that the program significantly
raised the intensity of input use, productivity, commercial orientation and proportion
of sales of most of the commodities of intervention for participant households leading
to a significant increase in cumulative net income by about 10%. Although the results
of our impact estimation proved that the program had succeeded in achieving the
ultimate objective of the program, increasing household income, we recommend
further research with relatively large sample size concerning how the benefits were
distributed among the different categories of participants and the relative contribution
xiii
of each commodity of intervention. These would enable policy makers to consolidate
efforts in the course of scalling up the experiences of the program as the current study
did not capture these due to lack of data.

xiv
CHAPTER ONE: INTRODUCTION

1.1 Background

The recent move towards the liberalization of foreign direct investment in developing
countries, starting with Latin America which latter rapidly spread to Asian and
African countries favored the development of supermarkets. As part of the structural
adjustment program of this move in the 1980s and the 1990s, many sub-Saharan
African countries have liberalized their economies and formulated poverty reduction
strategies that are intended to open-up new market-led opportunities for economic
growth (Shiferaw et al, 2006). On the other hand, rising incomes and fast urbanization
combined with increasing consumer attention to food safety increases the demand for
high value agricultural products stimulating commercial orientation among producers.
It is widely acknowledged that improved access to high value chains by small scale
farmers have a positive impact on farmers’ income level and poverty alleviation
(World Bank, 2008).

Like many developing countries, Ethiopia has undergone several structural adjustment
programs in the past few decades that aimed at bringing economic growth thus reduce
poverty. For example, agricultural extension services in the country until about 2002
were focused mainly on increasing production and productivity in view of reducing
poverty through achieving food security. However, it had become apparent around
1996 that without integrating farmers into the market, sustained growth in the
agriculture sector would not be realized (Gebremedhin, 2006). A pillar of the
country’s recent rural development strategy has been the active promotion of
marketing cooperatives as a means of commercializing smallholder agriculture.

Based on previous lessons and in response to the challenges, the government has
formulated its new development strategy called the Growth and Transformation Plan
(GTP) with a long term vision of transforming the economy from agricultural led
development to industrialization (MoFED, 2010). This strategy has driven the
introduction of several policies to promote a more supportive macroeconomic
framework and development particularly to promote liberalized markets for

1
agricultural products which also includes encouraging a strong extension and credit-
led push for intensification of high value crops and food staples production through
the use of modern agricultural inputs. Under the Agricultural Growth Plan (AGP) of
GTP, the strategy has emphasized on the transformation of subsistence agriculture
into market orientation as a basis for long-term development of the agricultural sector
(ibid).

Commercialization of agriculture and the growth of the non-farm private sector are
among the main pillars of the AGP to accelerate the agricultural growth. The
emphasis is to bring farm specialization both at farm and community level through: a
shift to high-value crops, promotion of niche high-value export crops, a stronger focus
on selected high-potential areas, supporting the development of large-scale
commercial oriented agriculture in areas where it is appropriate, and encouraging the
commercialization of agriculture in general through improved integration of farmers
with functioning markets. In order to enhance commercialization, specific strategies
were put forward to realize the envisaged objectives: realize an effective and efficient
service delivery system through a coordinated public and private actors, increase
availability and utilization of appropriate technologies by farmers at large, improve
institutional competence and performance at all levels, and create sustained demand
for the agricultural outputs (ibid).

In order to realize these policy directions and strategies particularly towards

commercializing the agricultural sector, currently the government has engaged in
implementing several programs/projects. These include increasing famers’ income
from market oriented production, processing, and marketing via improving access to
market information through the wider promotion of farmers’ cooperatives. Unlike the
previous development strategies, although the government has put emphases on
market orientation as a goal, the policies to achieve it is not clearly demonstrated
regarding how the potential benefits of increased commercialization could be
maximized by the small holder farmers while minimizing the potential negative
effects that could arise from such initiatives (ATARC, 2010).

The challenge, therefore, is to integrate small holder farmers who actually make up
the majority of farmers in the country and still remain subsistent oriented patterns of
2
production. In view of this and government initiatives, the regional research institute
and government of Oromia initiated a joint program on introduction of improved
agricultural technologies and input/output marketing to facilitate the development of
marketable commodities early in 2009 so that the experience of the project can be
used in the scaling up process. In the project approach, the important elements that
were given attention were the emphasis on the value chain rather than focussing on
the production alone and encouraging the involvement of actors within the value
chain in the participatory commodity development through capacity building and
knowledge sharing. The project was implemented as a pilot so that the experience
generated would be taken up by the regional government for scaling up in similar
areas (ibid).

At the local level, the program was implemented by involving existing public and
private institutions. From the public, institutions like extension service providers,
agricultural input suppliers, cooperatives and credit institutions were the main actors
considered. Whereas traders involved in agricultural input and output marketing were
the other main actors considered from the private sector. Apart from facilitating target
households’ access to agricultural innovations like improved agricultural
technologies, the project strengthens the capacity of these institutions so that they will
better serve the target households. In involving these institutions at local level where
the project is actually implemented, the project provides them with new ideas and best
practices from within and outside the country in the areas of agricultural service
provision and commodity development (ATARC, 2010; OBoARD, 2010).

The commodities of intervention selected as market oriented commodities by the

project in the pilot districts include inset1and fruit cultivation, apiculture development
and small ruminant fattening. At a higher level, the pilot zone (Arsi) and district
(Kofale) was selected by the regional government in consultation with the regional
research institute because of its potential for the commodities of intervention and
common interest in the area between the two implementing bodies. At the local level,

1
Inset belongs to the species of E.rentricosum (welw.) cheesm. (also known as Enset or “Inset” in
Amaharic, the Ethiopian Official language). It is widely cultivated in Ethiopia for food and fibber mainly
in the high lands of the country between 1600 and 3000m (Bezuxeh and Feleke, 1966).

3
kebeles2 were selected based on certain merits like its accessibility to main road and
agricultural extension services, and its potential opportunity for the specific
commodity of intervention. Similarly, identification and selection of households into
the program was done in consultation with local Development Agents (DAs) and
kebele administrators. Accordingly, willing households who own required resources
to start the selected activities were allowed to participate in the program activities
beginning from the start of the project as participants and thus selection into the
program was based on a non-random placement. Therefore, households who have
been involved in program activities since 2009 were considered as program
participants for this study.

1.2 Statement of the Problem

In the past, although the government of Ethiopia has implemented various programs
to enhance agricultural production, the development of agricultural markets was not
sufficiently emphasized. As a result, according to Puskur et al (2007), weak and
inefficient marketing system generally characterizes the agricultural marketing system
of the country. On the other hand, as to Eleni and Goggin (2006), farmer
organizations are weak, underdeveloped and are not yet commercial oriented.
Although the policy environments are conducive and the involvement of private
sectors in agricultural marketing believed to potentially improve the delivery
efficiency of input/output marketing, their involvement and level of investments is
very low.

Recognizing the situation, the government of Ethiopian has paid attention to address
the problem of the agricultural marketing system. It is evident from the recently
formulated development strategy of the country that agricultural sector particularly
commercialization of the small scale holder farmers via improving the market system
became one of the focal attention of the policy direction. It was believed that
development of agricultural markets contributes towards the improvement of
agricultural production and productivity by revitalizing the agricultural sector
(MoFED, 2010). Several programs are underway by various stakeholders, government

2
Kebele is the smallest administrative structure, which is equivalent to peasant association

4
and non-government actors, among which the current program is one. These programs
are mainly aimed at improving the agricultural marketing systems in order to bring
agricultural development and commercialization so that production growth further
catalyzed and boost rural incomes (ibid).

The current program was implemented as a pilot so that the experience of the program
will be scaled up in similar districts within the region. In this regard carrying out an
evaluation study is indispensable to generate information and prove the successfulness
of the program. In the literature, of course, there are several reasons why ex-post
impact evaluation of any project is required particularly in developing countries where
resources are scarce and every dollar spent should aim to maximize its impact on
poverty reduction. These include, to mention but a few, provision of feedback to the
scientists and policy makers, for accountability (establishing the credibility of the
public sector research and development), to justify for increased allocation of
resources to the sector under consideration, and to learn from and design new but
similar programs (Baker, 2000; Ponniah et al, 1996).

The project under consideration was completed at the beginning of 2013 according to
the project plan and claimed tobe successful on different events. The regional
government was interested to know the impact of the intervention and requested the
implementing institutions to carry out an impact study. Following the request, the
project staff recently conducted a field survey in order to collect data on both targeted
and non-targeted households for the evaluation purpose. The result of the evaluation is
intended to be used as an input in the planned scaling up of the project experience at a
regional level. Due to lack of knowledge in the application of a standard and robust
analytical model by the project staff, they were forced to use simple descriptive
analysis in order to estimate whether the project has brought any improvements in the
livelihoods of the target communities. Therefore, the current work is intended to
complement the ongoing work through employing a more standard and robust
analytical model that fit into the existing data collected by the project staff so that the
quality of information they are trying to generate through evaluation would greatly be
improved. In doing so, the current study tried to address the following research
questions.

5
 1.3 Research Questions

The impact of agricultural improvement intervention through market development on

the livelihoods of rural farmers in the pilot district was carried out to generate
information and estimate its effect of market orientation, participation, productivity
and income of target households. In doing so the following research questions were
formulated and accordingly addressed.

(1) What changes were brought in the organizational and institutional aspect of
agricultural input/output marketing in the pilot district as a result of the
intervention?
(2) What would have been the market orientation, participation, productivity and
income outcomes of participating households if the intervention had not been
in place?

1.4 Significance, Scope and Limitations of The Study

The aim of the study is to contribute to policy lessons generated from the implications
of the results and discussions on the studied variables. The findings from the impact
analysis of this particular intervention could be helpful to the regional government of
Oromia to revisit its policy and design similar programs in the course of expanding
the experiences to other parts of the region. It also adds value to the debate about the
effectiveness of programs happening in the country.

In terms of scope, the current study is limited to an evaluation of certain outcome

variables as the analysis was based on a pre-designed and collected household data on
limited outcome variables. With regards to limitations, since we are confined to the
available data, the sample size is also relatively small to carry out the intended
analysis using Propensity Score Matching (PSM). However, it is very common to find
similar studies available in the literature with a sample size of the same and even
below the one we considered here (for example see Wang et al, 2010; Abebaw et al,
2010). Moreover, the qualitative analysis carried out is not detailed enough. This is
particularly because of the difficulty to conduct a detailed investigation as the

6
qualitative analysis was primarily based on secondary sources backed up with online
communications with limited respondents.

1.5 Outline of the Study

The study was organized in five chapters. Following the introductory chapter, chapter
two provides related concepts and practices of impact study as well as the conceptual
and analytical framework based on the related literature review. The third chapter
discusses the methodological part of the study. Chapter four deals with results and
discussions. Finally, chapter five summarizes the conclusion and policy implication of
the study.

7
CHAPTER TWO: LITERATURE REVIEW

Under this section, the following subheadings were discussed: market development
and participation in the market, commercialization and measuring commercial
orientation, market institutions and agricultural marketing, approaches to impact
evaluation and related emperical studies.

2.1 Market Development and Participation in the Market

Market development has made up a key component of the agricultural growth strategy
recently being promoted by the governments of developing countries. The driving
factor behind is that input and output marketing system triggers the adoption of
modern agricultural technologies that is key for agricultural growth (Bellmare and
Barrett, 2006; Oechmke et al, 1997). Market development, according to Bellmare and
Barrett (2006), involves marketing extension and training, networking market
information and intelligence, grading and standardization of commodities at a
producer’s level, improving in competition and awareness among input and output
market actors, accessibility of marketing finance and credit by both input and output
market actors, and promotion of the commodities by targeting different customers. In
order to bring about these, Eleni and Goggin (2006) argue that interaction of all the
relevant market actors only with key market institutions that govern this interaction is
very important.

In developing countries, market development can benefit, small scale producers and
input suppliers. By strengthening the capacity of small and medium size local traders
through linking them with financing sources and large scale traders at higher levels as
well as by providing them technical supports, input market development can be
possible. Input market development helps suppliers to provide commercial inputs at
relatively lower price by reducing transaction costs involved. This in-turn can create
more demand stimulating traders to supply in bulk which further reducing the relative
prices of such inputs ultimately benefiting producers (Goetz, 1992). Similarly, Mwape
(2009) suggest that output market development can be made by opening up new

8
market opportunities for producers through providing market information, linking
with potential buyers, establishing producers’ cooperatives and more importantly by
engaging the different institutions and organizations along the value chain. By
improving farmers’ access to a more efficient input and output markets, Oechmke et
al (1997) argue that, farmers can engage in a commercial oriented production pattern
and invest in the use of new technologies.

Market development activities that target rural small scale farmers will improve their
market participation. The concept of market participation which is commonly used in
the literature is based on the approach suggested by Goetz (1992). This approach
divides household’s market participation decision into two stages: in the first stage,
households that produce a given commodity decide whether to be net buyer, net seller,
or autarkic in the market for that particular commodity, whereas in the second stage,
net buyers and net sellers determine the extent of market participation. According to
Reardon and Timmer (2005), market participation and economic growth are cause and
a consequence of one another. Improving either of one result in improving the other.
As to Ana et al (2008), higher market participation, for example, could drive
productivity by providing incentives for the farmer while higher productivity could
encourage their market participation since households with higher productivity are
more likely to have surpluses above their immediate consumption needs. As
household income increases so does the demand for more inputs thereby stimulating
demand-side market participation which further enhance the demand for cash and thus
supply-side market participation through increasing production and productivity.

2.2 Commercialization and Measuring Commercial Orientation

Relatively speaking, there is a growing interest and large body of literature available
dealing with the commercialization being stimulated by the current trends towards a
more globalized world. One can find diverse definitions with regard to
commercialization. A closer look at the literature can reveal that there are certain key
components in common which includes the different sides of the market, the different
types of commodities involved, and the decision making behavior of farm households
in the process of commercialization (Berhanu and Dirk, 2009; Alem et al, 2006; Jaleta

9
et al, 2009, Pingali 1997). In the following, among many, few of the definitions
concerning commercialization were reviewed.

Von Braun et al, (1994) states that commercialization is all about increased market
transactions so as to capture the benefits of specialization. Similarly, focussing on the
principles of profit maximization beyond merely the objective of marketing, for
Pingali (1997), commercialization is attained particularly when household product
choice and input use decisions in the process of marketing are made based on the
principles of profit maximization. In support of the above definitions, Berhanu and
Dirk define commercialization based on the objectives of the producers by
distinguishing between the commercial and subsistent oriented farmers. According to
them commercial oriented farmers are those households engaged in production
decisions with the objectives of profit maximization based on market signals and
comparative advantage. Whereas subsistent oriented farmers are those producers
whose primary production decisions are based on production feasibility and
subsistence requirements and usually sell surplus produces after their consumption
demands are met (Berhanu and Dirk, 2009).

From this perspective, it can be stated that commercial orientation can be understood
as a transition from subsistence oriented with the objectives of production feasibility
and subsistence requirements to increasingly market-oriented primarily with the
objectives of profit maximization patterns of production and resource use.

The shift to a wider financial market development (World Bank, 2008) and the recent
move towards market reform especially in developing countries has renewed an
interest in the working of agricultural markets for the poor as a source of income,
employment and food security (Pingali and Rosegrant, 1995). In a place where market
institutions are relatively working better, according to Timmer (1997),
commercialization of surplus output from rural small scale farming households is
increasingly recognized that it is closely linked to higher productivity, greater
specialization and better income. However, the potential benefit from
commercialization greatly affected by the imperfect markets and high transaction
costs that usually characterize market in developing countries (Bernard et al, 2010;
Key et al, 2000). Under such circumstances where there are no mechanisms in place
10
to overcome market imperfection and high transaction costs, small-scale farmers are
unlikely to participate in markets and difficult to realize the full benefits of
participation. According to Jayne et al (2006), these problems are particularly
important in Sub-Saharan Africa where significant proportion of farmers engaged in
subsistence agriculture and those who participate in markets often do so only at the
margins because of the high market risks and associated transaction costs. Moreover,
recent studies revealed that even in the contexts where the formal market institutions
proved relatively to be efficient and reduced transaction costs, market participation by
the poor and marginalized households is highly constrained by social institutions
(Johnson, 2012).

One can notice, in the literature, that there are several related ways of approaching the
measurement of commercialization at the household level. However, according to
Moti et al (2009:1), “[a]lthough the net welfare gain from agricultural
commercialization at the household level is universally accepted, there is no common
standard for measuring the degree of household commercialization”. In spite of the
fact that there is no universally agreed standard measurement for commercialization, a
large body of literature exists in this regard particularly for crop as opposed to the
livestock sector. For example, the household commercialization index developed by
Govereh et al (1999) for a crop is calculated as a ratio of the gross value of all crop
sales per household per year to the gross value of all crop production as an indicator
of the level of household commercialization. On the other hand, commercialization
structure is measured in terms of the degree of allocation of the different resources
such as labor and land to competing agricultural activities and total sales of
agricultural commodities (Keister and Nee, 2001). Similarly, Von Braun et al. (1994)
have used the proportion of land allocated by farmers to commercial crops and the
value of input and output sales and purchases weighted by the value of agricultural
production to measure the level household commercialization. With regard to
livestock subsector, the commercialization process and its measurements have
received little attention and thus hardly possible to find (Jaleta et al, 2009). However,
the net (market) off-take rate, which is more relevant, according to the same author,
can be used in measuring the level of smallholder livestock commercialization which
considers only the sales and purchases of livestock per household per a specific
period.
11
Based on the existing literature, the measurements of commercialization can be thus
expressed broadly as a higher proportion of agricultural input and crop output that is
marketed for cash and similar resources allocated in an increased amount to this
commodity.

2.3 Market Institutions and Agricultural Marketing

Market works as a result of institutions. According to Dorward et al (2005)

institutions are the rules of the game that defines the incentives and sanctions
affecting people’s behavior. Thus, agricultural market institutions constitute rules,
enforcement mechanisms and organizations that promote market transactions (World
Bank, 2002). Hence, it is clear that agricultural institutions provide numerous
functions to markets like transmission of information, mediate transactions between
market actors, facilitate the arrangement and enforcement of property rights and
contracts, manage the degree of competition in the market and etc.

On the other hand, where supporting and important market institutions are absent, “…
asymmetric information, high transaction costs and imperfectly specified property
rights” are reasons responsible for market failure undermining the process of
exchange that affect the market efficiency (Shiferaw et al, 2006: 2). In rural areas
where there are underdeveloped road and communication networks, as to Chowdhury
et al (2005), market problems are more pronounced and characterized by localized
markets with little rural-urban linkages. Under such circumstances, De Janvry et al
(1991) argues that households forced to produce a limited range of goods and services
only for their own consumption since social protection for food security is not
provided through existing markets. As a result market players have no incentives to
undertake market oriented investments for greater profit since the lack of market
coordination further hinders market functions in such contexts (Dorward et al, 2005).
In general, according to Dorward et al (2003), coordination failure along the value
chain may explain constrained agricultural development which is considered to be a
big challenge to policy makers. World Bank (2002) suggests that overcoming the

12
effects of such market imperfections particularly in agricultural input and output
markets would require institutional measures that promote coordination of market
functions, minimize transaction costs, and integrate marketing through strengthening
institutions.

Among the several institutional measures used to better serve the poor particularly
through improving agricultural market functions suggests the use of producer
marketing groups that have been widely recommended to improve the market failures
due to high transaction costs, asymmetric information and imperfectly specified
property rights (World Bank, 2002). According to Coulter et al (1999) and World
Bank (2002), their potential contribution in bridging the gap created as a result of
market failure lies in enabling contractual links to input and output markets, in
leveraging market functions for smallholder farmers and promoting economic
coordination in liberalized markets. Their success in this process, of course, depends
on their ability in conveying market information; coordinating marketing functions;
defining and enforcing property rights and contracts; facilitating smallholder
competitiveness in markets and more importantly in mobilizing their members to
engage in markets.

However, according to Bromley (1998: 87) “[t]he poor remain poor because the
institutional arrangements rendered them poor prior to the development intervention
[…]”. Pointing out the importance of understanding the local context in designing any
interventions that geared towards the benefit of the poor and marginalized parts of the
community. Bastiaensen et al, argues that sustainable poverty reduction requires pro-
poor institutional change apart from anti poverty intervention which is based on
external assumptions and collection of best practices which might not bring change
rather adds another political arena to the local institutional landscape which further be
a cause for the reproduction of poverty (Bastiaensen et al, 2005). This argument is
rooted in the concept forwarded by Long who criticizes externally planned
interventions as a gross simplification of complex realities. He argues that “[t]he
concept of intervention, then, needs deconstructing so that it is seen for what it is- an
ongoing, socially-constructed, negotiated, experiential and meaning-creating process,
not simply the execution of an already-specified plan of an action with expected
behavioral outcomes” (Long, 2001: 25). This is what Bastiaensen et al brings out in
13
their text that “[t]he relationship between outside poverty alleviators and inside actors
is much more interactive, complex, and beyond the control of outside “development
planners” (sic) than some self-acclaimed social engineers have suggested”
(Bastiaensen et al, 2005: 981, emphasis original).

As a result of the recent advancement in social theories, there is an increasing

understanding of how institutions are designed, implemented and differentially affect
the various actors involved. This ever increasing in understanding acknowledges that
the process of socio-technical innovation is a complex process. Douglas and Cook in
their study describe it as a messy do-it-yourself ‘bricolage’ process. As a result
“[n]either researchers nor politicians can determine the direction or pace of change.
But with a deeper understanding of the institutional landscape and social processes,
researchers and politicians can support people to improve their livelihoods and well-
being” (Douglas and Cook, 2012: 16). In arguing against the instrumentalist approach
to institutional design for specific purposes who assumes individual actors as political
and social entrepreneurs who can rationally utilize their social capital in order to craft
institutions in pursuit of optimal resource management, they argue for the concept of
institutional bricolage which signifies the active, conscious creative process of
adapting norms, values and social arrangements to fit new purposes, while also
reflecting and being shaped by deeply embedded unconscious principles. Similarly,
Cleaver further expand the concept of bricolage to explain how institutions formed
and functioning in practice as “[…] the adaptive processes by which people imbue
configurations of rules, traditions, norms and relationships with meaning and
authority. In so doing they modify old arrangements and invent new ones […]”
(Cleaver, 2012: 34).

Furthermore, it is becoming clear that an institutional arrangement for the

management of resources that is based on the interest of the users themselves does not
bring an equitable outcome for all participants. According to Cleaver and Franks
(2008: 161), “[p]articular institutional arrangements for [… ] resource governance
produce differential outcomes of poor people”. The authors argue that instrumental
approaches to the generation of knowledge for institutional design and policy based
on the amalgamation of perceived 'success stories' and 'good practice' does not work
(emphasis original). Therefore, scholars and policy makers need to adopt approaches
14
that attempt to understand institutional arrangements for governance and the likely
outcomes it has particularly for the poor within a wider framework of negotiations
over the allocation of societal resources. Franks and Cleaver, in their earlier study,
pointed out that better understanding of the mechanisms of resource governance, the
particular local structures and processes of decision making, and how the existing
social relationships, norms and daily practices interact with the resource management
systems and eventually how it shape access to resources by the poor is the crucial step
to be undertaken in the first place than institutional crafting for the poor (Franks and
Cleaver, 2007).

2.4 Definitions, Types and Approaches to Impact Evaluation

2.4.1 Definitions and objectives of impact evaluation

In the body of literature one can discover a number of definitions to impact

evaluation. One of the most widely quoted definitions of impact assessment is the one
given by Omoto (2003) and Rover and Dixon ( 2007) that definite impact as a process
of systematic and objective identification of the short and long-term effects either
positive or negative, direct or indirect effect of intervention on the four main areas-
economic, social, institutional and environmental. Such effects due to a certain
intervention may be anticipated or unanticipated, and positive or negative realized at
different levels namely individual, household or organizational level and caused by
on-going or completed development activities such as a project or program.

The goal of an impact evaluation is to know what happened to the target as a result of
an intervention. Therefore, impact evaluation tries to assess the extent to which a
program or project has caused the desired changes on the targeted populations.
Therefore, impact evaluation is concerned with the net impact of an intervention on
target objects at any level as a result of the intervention alone (Baker, 2000). Hence,
impact evaluation consists of assessing outcomes of research and developmental
change resulting from an intervention at different impact levels.

15
Impact evaluation has two main functions-accountability and learning functions.
According to FAO (2000), impact evaluation is carried out for a number of practical
reasons:
(1) Accountability – this refers to both upward accountability (accountability to
donors) in which evaluation is conducted to satisfy the needs of these
stakeholders on what is accomplished in order to secure continued support.
(2) Improving program design and implementation - to draw lessons from own
activities in order to take informed decisions that help improve future
performances and;
(3) For planning and prioritizing purposes- impact evaluation is carried out in
order to generate information that helps to forecast the likely consequences of
an intervention on which the planning and prioritization of resource
investment will be based.

Generally speaking, there are broadly two types of impact assessments based on the
time continuum. Ex-ante and ex-post impact studies. Ex-ante type of impact study
refers to the assessment of an impact of the likely future environments and anticipated
impacts of an intervention. The objectives of conducting such an assessment are
usually to generate information that assists in decision making about approval and
funding of any project. Whereas ex-post impact study refers to impact evaluations
carried out to evaluate performance, achievements and impacts of the past
interventions of a project or program with the aim of generating information used in
accounting for the past use of resources, and also as a useful input for future planning
of an intervention (ibid).

2.4.2 Typologies of impact evaluation

As explained earlier, generally impact evaluation can take place at three main levels,
individual, household and organizational levels. However, the comprehensive impact
evaluation, according to Ponniah et al. (1996), would be undertaken at people
(household) and community levels. Most programs or projects targets certain
segments of the population and thus implemented for specific target communities at

16
household level. Therefore, impact evaluation at this level refers to household level
impact and tries to answer the question of what changes have been brought to the
ultimate users or target group for which the intervention is designed and conducted as
a result of the intervention. Since, in such cases impact assumed to begin to happen
when there is a change in behavior among the target beneficiaries. Therefore, the
household level impact deals with the actual uptake of the intervention and the
subsequent effects on, for example, production, income, environment and/or any other
development objectives (Omoto, 2003). Recently, the need of understanding the
change brought in the broader institutional arrangements as a result of program
intervention is widely acknowledged in the literature. Because of the likely
sustainability of the impact of any intervention lies in the positive changes of this
component of an intervention. The institutional impact component of an intervention
involves the assessment of the impact of an intervention in non-technical areas
beyond the household level (FAO, 2000).

2.4.3 Approaches to impact evaluation study

Evaluation approaches for development programs have evolved greatly over the past
two decades. Impact evaluation is getting attention of donors and governments of
recipient countries more than ever. Recently, literature on impact evaluation is
growing as a result of a widespread encouragement from a diversified actors
interested and involved in funding, implementing and evaluating development
programs. As a result, approaches to impact evaluation of development programs
rapidly evolving.

There are key questions that evaluators need to take into account while approaching
impact evaluation. According to Baker (2000), some of the questions addressed in
impact evaluation include the following:
(1) How did the project affect the beneficiaries?
(2) Were any improvements a direct result of the project, or would they have
improved anyway?
(3) Could program design be modified to improve impact?
(4) Were the costs justified?

17
The above listed questions cannot, however, be simply answered by measuring the
outcome of an intervention. There may be other factors or events that correlate with
the outcomes but are not caused by the project. Baker (2000) argues that estimating
counterfactual for an impact evaluator is a process of ensuring methodological rigor.
That is a process of judging what would have happened had the intervention is not
implemented. However, as stated earlier, it is impossible to observe two outcomes for
an individual at the same time. In other words, one can only observe the factual
outcome. This becomes the basic problem in any social program impact evaluation.

If one could observe the same individual at the same point in time, with and without
the program, this would effectively account for any observed or unobserved
intervening factors or concurrent events and the problem of endogeneity do not arise
(Ravallion, 2005). Since this is impossible in practice, instead something which is
similar need to be constructed by identifying non-participating individuals who are
sufficiently identical in every way to the participating groups that receives the
treatment, except that non treated groups do not receive.

However, development programs operate in a complex situation and thus the effects
of the program are influenced by various factors evolving around the time of the
program’s life span. Baker (2000) argues that estimating the effect of a project
requires separating its effect from other intervening factors which may be correlated
with the outcomes, but not caused by the project itself. Similarly, Ravallion (2005)
and Bryson et al (2002) suggest that the task of separating the effect of the project
from other intervening factors is facilitated if control groups which are more or less
identical to the treatment group are constructed from non-beneficiaries in order to
estimate the counterfactuals. Constructing such group make possible the control of
other factors that affect the outcome (confounding factors). According to Ezemenari
et al (1999), the identification of comparator group correctly is a key to estimating
what would have occurred had the intervention is not there.

18
Figure 1. The evaluation problem—disentangling project effects from intervening
factors, Ezemenari et al, 1999

The process of constructing a comparator group, however, is difficult because of two

reasons. First, program targeting- the situation in which treatment group is selected on
the basis of certain characteristics. According to Ravallion (2005), if these
characteristics are observed then a comparison group with identical characteristics can
be easily selected. Second, the comparison group may be contaminated with several
factors including spill-over effects from the intervention itself or groups in
comparison are exposed to similar intervention (IFAD, 2012). Therefore, if these
differences that could arise due to program targeting and self selection into the
program are not properly accounted for, comparison of outcomes between participant
and comparison group is likely to yield biased estimates of program effect (Bernard et
al, 2010).

In theory, quantitative evaluation approaches can be carried out in two ways in

establishing treatment and control groups. These approaches are known as
randomization (pure experimental) design and non-experimental (quasi-experimental)
design.

19
 2.4.3.1 Experimental method

In theory allocating a program or intervention randomly across a sample of

observations is one solution to avoiding selection bias, provided that program impact
analysis is carried out at the level of randomization. This requires the careful
elimination of drop outs from groups, program participants and non program
participants.

Experimental designs, also known as randomization, are generally considered as the

most robust of the evaluation methodologies (Baker, 2000). By randomly allocating
the intervention among eligible beneficiaries where program participants and
nonparticipants have the same expected outcome of interest in the absence of the
program, the assignment process itself creates a comparable treatment and control
groups that are statistically identical to one another, given appropriate sample sizes
and helps to overcome the problem of selection bias, provided that program impacts
are examined at the level of randomization.

The basic idea behind experimental designs is that the treatment and control samples
are randomly drawn from the same population (Bernard et al, 2010). This enables the
researcher to determine program effect by comparing means of outcome variable for
the two groups. Similarly, Nssah (2006) argues that in a randomized experiment, since
the treatment and control samples are randomly drawn from the same population,
observable and unobservable characteristics do not correlated and as a result the
problem of selection bias will be avoided. This allows the researcher to estimate the
impact of the program by comparing means of outcome variables for the two groups
which yields sufficiently an unbiased impact estimate of the intervention. Bernard et
al (2010) states that random assignment ensures that the two groups are statistically
similar in both observable and unobservable characteristics, thus avoiding program
placement and self-selection biases if implemented appropriately. It also helps to
ensure that potential confounders are balanced across the two groups and therefore
any differences in the outcomes between the two can be attributed to the intervention.

However, according to Shahidur (2010), “[i]f these programs are exogenously placed,
conditional on these observed characteristics, an unbiased program estimate can be
20
made. Despite the clarity of a randomized approach, a number of factors still need to
be addressed in practice. They include resolving ethical issues in excluding areas that
share similar characteristics with the targeted sample, accounting for spill-overs to
non-targeted areas as well as for selective attrition, and ensuring heterogeneity in
participation and ultimate outcomes, even if the program is randomized”. Similarly,
Ravillion (2005), Bernard et al (2010) and Baker (2000) argue that although
experimental designs are considered the best approach to estimate the impact of an
intervention, in practice, it is not feasible in demand-driven programs in which
participants make their own decisions of whether to participate and about the kind of
activities to do in the learning process. Furthermore, individuals in comparison groups
may change certain identifying characteristics during the intervention time that could
contaminate the outcome of an interest. Moreover, experimental designs, under
certain circumstances, are expensive and time consuming.

2.4.3.2 Quasi-experimental method

In situations where setting up a random experimental design is not feasible due to

certain reasons, non experimental designs called quasi-experimental methods are
introduced to carry out an impact evaluation. For development projects that are often
implemented with intentionally, it is not common to see a baseline survey conducted
before the project is implemented (Jalan and Ravallion, 2003). As a result it is,
therefore, necessary to construct a comparison group that is similar to the treatment
group in order to establish counterfactuals through different econometric
methodologies.

As to Bernard et al (2010), when quasi-experimental methods are used, the treatment

and control groups are usually selected after the program is implemented through non-
random methods. In some cases a comparison group can be chosen before
intervention even though the selection is not randomized. Therefore, statistical control
methods must be applied to address differences between the two groups or
sophisticated matching techniques must be used based on the types of quasi-
experimental methods used to construct a comparison group that is as similar as
possible to the treatment group.

21
On the other hand, a quasi-experimental method is the only alternative approach
available when neither a baseline survey nor randomization techniques are feasible
options (Jalan and Ravallion, 2003). The main advantage of quasi-experimental
designs are that they can draw on existing data sources and are thus often quicker and
cheaper to implement, and they can be carried out after a project has been conducted,
given sufficient existing data is available. The main shortcomings of quasi-
experimental techniques according to Baker (1999) are:
(a) The reliability of the estimated results is often reduced as the methodology is
statistically less robust;
(b) The methods are statistically complex and large data demanding; and
(c) A selection bias problem arising from program targeting and self selection.

Thus, the central methodological challenge to quasi-experimental evaluation method

is the process of examining outcome response of an intervention which involves
distilling the impact of intervention from other factors affecting the outcome of
interest (Foster, 2003). In response to this, there are different econometric approaches
developed to deal with this problem. The commonly used methods include double
difference (DD), reflexive comparison, propensity score matching (PSM),
instrumental variable (IV) and regression discontinuity (RD).

Double difference or difference-in-differences (DID) design

Double-difference (DD) methods assume that unobserved heterogeneity in

participation is present—but that such factors are assumed to be time invariant
(Shahidur et al, 2010)). Using data on program participant and non-program
participant observations which taken before and after the program intervention, DD
tries to eliminate this difference due to unobserved heterogeneity. As to Jalan and
Ravallion (1999), in other words, DD compare a treatment and comparison group
before (first difference) and after a project (second difference).

Using this method program impact is estimated by calculating the difference in

outcomes between treatment and control groups after program implementation minus

22
the difference in outcomes between treatment and control groups before the
implementation of the intervention. In this way DD estimates the average effect of a
program in the absence of a randomized design control method.

The strength of the panel data based DD estimator usually comes from its simplicity
of application. That means DD can easily derive an estimate of the impact by
comparing the treatment and control groups using data from post treatment (second
difference), after the pre-treatment data was used to equate treatment and control
groups (first difference). As a result, DD is less subject to selection bias as it
eliminates pre treatment differences between the treatment and comparison groups via
removing the effect of any unobserved time-invariant differences. .

However, there are at least two disadvantages that attached to the very simplicity of
such a panel based approach to impact evaluation. First, using DD to construct panel
data sets can be expensive, time taking, and logistically problematic since we are
required to collect baseline and follow-up data. Second, the design assumes that the
potential selection bias as a result of the program targeting and self selection bias is
assumed to be linear and time invariant such that it can be differenced out in the first
differencing (Jalan and Ravallion, 1999). However, this assumption might be violated
if the time period between two panel data sets is long enough so that the unobservable
characteristics of subjects are altered. Furthermore, the unobservable characteristics
can be changed as the subjects participate in the program which cause the impact
estimated to be biased.

Shahidur et al (2010) suggests that in situations where a baseline data might not be
available in order to employ the DD method, use of a triple-difference method with
exclusively separate control experiment after the program intervention (that is, a
separate set of untreated observations) offers an alternate estimation of the program’s
impact. On the other hand, apart from panel data, DD can be used on repeated cross-
sectional data as well, as long as the composition of treated and comparison groups is
fairly stable over time along with the program intervention period.

23
A reflexive comparison design

A reflexive comparison method can take different forms like a simple before and after
data, panel data and a time series data taken on the same program participants
(Valadez and Bamberger, 1994). But in most cases it refers to the methods in which a
baseline survey of treated individuals is done before the intervention and a follow-up
survey is done after the intervention. Here, treated individuals who received the
treatment are compared to themselves before and after receiving the intervention. The
use of a control group in this case, is not employed as both the treatment and the
control group is the set of treated individuals themselves (ibid).

Propensity score matching (PSM) design

PSM is considered to be the second best impact evaluation method among the quasi
experimental approaches (Baker, 2000). Basically, PSM tries to create the
observational analogue of an experiment in which everyone has the same probability
of participation in the treatment. The main difference here is that in PSM it is the
conditional probability (P(Z)) of observations that is intended to be identical between
participants and nonparticipants as opposed to the case of randomization which
ensures by design that participant and nonparticipants are the same in terms of the
distribution of all characteristics whether observed or not. As a result of this
underlying assumption in PSM design, there are always concerns about the remaining
selection bias (Chaudhury and Parajuli (2006).

As a measure to this problem in PSM, evaluators employ the use of regression

techniques in order to avoid the pre-intervention differences between the two groups.
However, there are limitations involved in such approaches too. In the first case, a
regression technique generally assumes a linear relationship between the independent
(covariates) and dependent (outcome) variables. Secondly, a more subtle problem
involves what is so called common support or distribution of the covariates
(Ravallion, 2005). Not only high and low treatment groups differ in terms of the
means of those covariates, but the distribution of those covariates could overlap
relatively little. Under such situations, regression essentially projects the behavior of

24
individuals in one group outside the observed range to form a comparison for the
other at common values of the covariate. Such projections can be more sensitive to
functional form (Foster, 2003).

With the help of PSM, a sufficiently close comparative counterfactual group can be
constructed from a large enough group of non participants which is similar as much as
possible on the bases of observable characteristics with that of program participants
(Bryson et al, 2002; Caliendo and Kopeinig, 2008). What PSM does for us is that it
matches control groups to treatment groups on the basis of these characteristics in
terms of their propensity scores. In principle, the closer the propensity score between
the two groups are, the better the match can be made. According to Jalan and
Ravallion (1999), good control group can be constructed and the matching estimates
are more sufficiently close to reliable if: (i) treated and untreated groups have similar
distribution of unobserved characteristics; (ii) they have the same distribution of
observed characteristics; (iii) the same questionnaire instruments are administered to
both groups; and (iv) treated and control households are facing the same economic
incentives that drive participation into the program. If these conditions are not
fulfilled, the estimated mean impact of the program as the mean difference in the
outcomes of these participants and the matched non-participants will be biased
estimate of the impact of the program.

PSM compares only observations which are comparable and does not rely on
parametric assumptions, unlike in the case of econometric regression methods, in
order to estimate the impacts of an intervention and avoid assumptions on functional
form and error distributions such as the imposition of linearity, multicollinearity and
heteroscedasticity issues. The emphasis of PSM is achieving a sizable common
support region condition so that bias due to extrapolation to non data region will be
avoided. This enables one to explain the results to peer groups and policy makers
easily since the idea of comparing two similar groups is quite insightful.

However, as to Rosenbaum and Rubin, matching becomes difficult when a multi-

dimensional vector of characteristics exist. But, PSM manages this problem by
creating a single index variable from each pre-treatment characteristics of subjects and
use the propensity score (PS) to match comparable individuals from both groups.
25
Hence, in PSM, the probability of getting into the treatment is conditional on pre-
treatment variables (Rosenbaum and Rubin, 1983). On the basis of propensity scores
calculated based on observable characteristics, participants and nonparticipants whose
propensity scores fall in the common support can be matched using different matching
algorithms. As to Shahidur et al (2010), the most commonly known matching
algorithm once include nearest-neighbor (NN) matching, caliper and radius matching,
stratification and interval matching, and kernel matching and local linear matching
(LLM).

Instrumental variable estimation (IV) design

In approaching impact assessment using estimators that involves micro-econometrics,

the major difficulty that is stressed is the likelihood of inconsistent parameter
estimation as a result of endogenous repressors. In regression estimates we measure
only the magnitude of association between given variables, however, the most
important thing required to be known for policy analysis is not only the magnitude of
the association but also the magnitude and direction of causality. One of the most
widely used impact estimator in Econometrics which provides us a way to achieve
consistent parameter of estimates, in spite of its conceptual difficulty and easily
altered, is an instrumental variable estimator (Alberto, 2003).

The main idea behind the IV is that it involves the identification of an important factor
that determines the receipt of a given project, but which does not affect outcomes of
interest. Then based on the assumption that the take up of the treatment is based on
that factor, simulation will be made about who would have been in the treatment
group and the control group. The impact of the treatment, then, be estimated as the
difference in outcomes between these simulated treatment and control groups.

The IV method allows for endogeneity in individual participation, program

placement, or both in contrast to PSM estimator. In addition to that, with panel data,
IV methods can allow for time-varying selection bias as opposed to the case in DD
estimator which impose a conditional assumption that states pre-intervention
differences between treatment and control groups is time invariant- constant over time

26
even in the absence of the treatment. Moreover, as to Shahidur et al (2010), IV
estimator provides evaluators an alternative means of resolving bias arise as a result of
measurement error.

The IV approach involves finding an instrument that is highly correlated with

participation or program placement but that is not correlated at the same time with
unobserved characteristics affecting outcomes. So the major task of an evaluator is
finding this variable (instrument) that fulfills the above condition in order to apply the
IV approach. In order to identify these instruments, the evaluator has to have a good
knowledge of the way the program placement is made and participation in the
program is determined. In order to minimize bias, instruments must be carefully
selected.

One of the main advantages that come along with IV estimator according to Alberto,
(2003: 232) is that “[…] it allows the researcher to construct estimators that can be
interpreted as the parameters of a well-defined approximation to a treatment response
function under functional form misspecification”. However, if the instruments are
found to be correlated with unobserved characteristics or omitted variables that also
affect the outcome, the identified instrument become weak worsening the bias
(Shahidur et al, 2010).

Regression discontinuity (RD) design

In most cases development oriented programs are designed with specific objectives to
serve specific segments of the population and thus usually targeted. As to Paul et al
(2011: 81), “social programs often use an index to decide who is eligible to enroll in
the program and who is not. For example, antipoverty programs are typically targeted
to poor households, which are identified by a poverty score or index”. Under such
conditions setting experiment easily may not be possible and instead program
eligibility rules can be used in most cases as instruments for exogenously identifying
participants and nonparticipants. Unlike PSM approach, RD accounts for observed as
well as unobserved heterogeneity. In order to establish comparability, one can use
participants and nonparticipants as treatment and control groups within a certain

27
neighborhood of the cutoff point as the relevant sample for estimating the treatment
impact using an approach called a Regression Discontinuity (RD). Although the
eligibility threshold can be defined non-parametrically, the threshold level has in
practice traditionally been defined through an instrument.

Discontinuities in program design and implementation, based on eligibility criteria (or

other exogenous factors), can be very useful in the non-experimental program impact
evaluation. According to Shahidur et al (2010), it is assumed that people above and
below the cutoff point are similar in observed characteristics and can be distinguished
in terms of outcomes. However, in using eligibility criteria, caution must be taken
while sampling since samples across which to compare would have to be sufficiently
close to the eligibility cutoff point to ensure comparability.

Moreover, unobserved heterogeneity may be a factor if people within the eligible

targeting range reveal a discrepancy in actual take-up of the program that may lead to
selection bias. In such case, eligible and non eligible samples of both groups close to
the eligibility cutoff would be taken to ensure the comparability of the average
program effect. RD design is, therefore, similar to instrumental variable (IV) methods
in a way that both introduce an exogenous variable that is highly correlated with
participation. By comparing the outcomes of the groups with a narrow band of units
that are below and above the cutoff point, one can estimate the program’s effect since
households just below and above the cutoff point are likely to be very similar to each
other in both observed and unobserved characteristics (ibid).

In the case of RD, robustness checks can be conducted in order to examine its validity
and unanticipated changes in other control variables at the cutoff point. In addition, it
is also helpful examining the pattern in the covariate determining eligibility condition.
However, concerns with RD approach include the possibility that eligibility rules will
not be remain to consistently, in addition to the likely change of eligibility rules over
time. Moreover, according to Paul et al (2011: 83), “[…] the estimated impact is only
valid in the neighborhood around the eligibility cutoff score”. On the other hand RD
brings evaluators, from an ethical point of view, the advantage of successfully
estimating the effect of a given intervention without excluding any eligible segments
of the population from the intervention.
28
 2.4.3.3 Choosing an estimator

In practice, the choice of any of the above approaches for evaluating a given program
depends on the way the program is implemented, the types of data available, the cost
of study involved, and ethics of the experiment are among the others (Baker, 2000;
Degnet et al, 2010). The program under consideration was designed to improve rural
livelihoods via raising the income level of the farm households in targeted potential
agro-ecological zone. Although specific criteria were not clearly indicated, willing
households who could able to involve in the program activities were allowed to
participate in the program and hence based on non-random placement. The data at
hand for the program is a cross-sectional data collected at the end of the program by
the project staff for the intention of estimating the impact of the intervention on target
households. The choice of our estimator for the current program impact study on the
agricultural improvement intervention through market development was made after
reviewing the relevant impact estimators in the literature as presented in the preceding
sub-sections in line with the nature of the program as described above.

Although the randomized experimental control design is generally considered as the

most robust evaluation methodology (Baker, 2000), it cannot be adopted for the
current study as the program was based on non-random placement. Moreover, the
reflexive comparison approach cannot be followed since panel or baseline data was
not available. Under situation where setting up a random experimental design is not
feasible, the only option available is to adopt quasi-experimental method/s that is/are
appropriate for the available data. This is because of the fact that for development
projects that are often implemented with intent, like the one under consideration, it is
not common to see a baseline survey conducted before the project is implemented
(Jalan and Ravallion, 2003). Hence, quasi-experimental method allows one to
construct a comparison group that is similar to the treatment group in order to
establish counterfactuals through different econometric methodologies. However, the
next question is, among the available quasi-experimental methods, which estimator/s
is/are most feasible for the data at hand.

29
Among the quasi-experimental designs, as far as the current program is concerned,
one cannot go for difference in difference (DID) design since this particular estimator
required a baseline data collected before the implementation of the program.
Similarly, instrumental variable (IV) and regression discontinuity (RD) cannot be
employed for the current impact study as the implementation of both designs are
based on clear identification of an instrument and eligibility condition that can be used
as a cutoff point respectively so that the problem of selection bias would be dealt
with. However, given the nature of the program in targeting households, finding
relevant and exogenous instruments as well as a clear cutoff point for program
participation that enables one to employ these models is very difficult. Therefore, in
order to deal with potential selection bias problems and address causality so that the
research questions were properly tackled, propensity score matching (PSM) seems the
best alternative analytical method available to estimate the impact of the intervention
based on the available data at hand. This is particularly because of the following
reasons. Firstly, baseline date was not available for the program. Secondly, treatment
assignment was not based on random. Thirdly, PSM is assumed as the second best
alternative design available to that of experimental design in minimizing selection
bias (Baker, 2000).

2.4.3.4 Related empirical studies and application of PSM

Although it is not very common to find empirical studies in relation to the subject
under consideration in the study area, application of PSM in order to evaluate
programs is very popular as it offers an alternative for ex-post evaluation, as well as
ways to overcome the selection bias. Since it is difficult to review all the available
related literature on PSM application, however under this sub-heading, few but
relevant empirical evidences was reviewed on the application of PSM as an
alternative approach to evaluation of the impact of programs that were conducted both
outside and within the country.

In assessing the impact of adoption of technological packages in agriculture on the

farming households in Kenya using PSM, Deschamps-Laporte (2013) reported that
the program households were found better in adopting improved technologies, for
example their fertilizer dosage increased by at least 24.91%, although productivity

30
per acre is not affected by the program. Using the same approach, Mendola (2006)
carried out an evaluation of whether adopting a modern seed technology causes
resource-poor farmers to improve their income and decrease the propensity to fall
below the poverty line in rural Bangladesh. The result of his analysis indicated that
there is a robust and positive effect of agricultural technology adoption on farm
household well-being. With the application of PSM to evaluate the impact of a pilot
farmer field-school (FFS) program on farmers’ knowledge of integrated pest
management (IPM) practices related to potato cultivation in the Peruvian Andes,
Godtland et al (2004) find out that farmers who participate in the program have
significantly more knowledge about IPM practices and improved productivity than
those in the non-participant comparison group. In the same way, Davis (2010)
assessed the impact of FFS on agricultural productivity and poverty in East Africa
using PSM investigated that participation in FFSs led to increased production,
productivity, and income of the participant households.

In Ethiopia few empirical studies are available with regards to the application of PSM
in order to evaluate development programs. One of such recent studies is the impact
of participation in the IHEP program on household income, investment and income
diversification conducted by G Egziabhe et al (2013) in the northern part of the
country. The study found that the extension program had increased income of the
participating households with about 10 %. Moreover, the program had a large positive
impact on investment and income diversification as well. Similarly, Gilligan et al
(2008) analyzed the impact of productive safety nets program on food security and
coping mechanisms in Ethiopia using the same approach. The result of their analysis
revealed that participation in the public works component of the program over the first
five months has modest effects. Participants’ food security improved by 0.40 months,
livestock holdings by 0.28 (in Tropical Livestock Units) and the likelihood that a
household is forced to make a distress asset sale by 4.4 % points. On the other hand,
Andersson et al (2009) by applying the model in order to assess the impact of the
Productive Safety Net Program (PSNP) in Ethiopia on rural households’ livestock and
tree holdings, found that there was no indication that participation in PSNP leads
households to disinvest in livestock or trees. The number of trees, in fact, increased
for participant households.

31
As far as commercialization is concerned, finding empirical evidence generated
through the application of PSM model is hardly possible in the study area. The only
study that was reviewed in this regard is the work of Bernard et al (2010) conducted
on the evaluation of the impact of cooperatives on smallholders’ commercialization of
staple crops in which he used output price offered and proportion of output sold as
outcome indicators using PSM. The study revealed that, relative to what member
households would have received had they marketed their output individually,
cooperatives found delivering 7% premium price for their members on average.
However, the quantity of market surplus for member farmers was found to be less
than that of the non-member households.

32
CHAPTER THREE: RESEARCH METHODOLOGY

3.1 Description of the Study Area

The study district, Kofale, is one of the 22 districts of the Arsi zone of the Oromia
Regional State where the pilot project was implemented. It is located in the southern
Arsi zone at 270km away from Addis Ababa, the capital city of Ethiopia. Kofale town
is the administrative center of the district.

Agro ecologically, the district is divided into two namely temperate (Badaa in local
name) and subtropical (Badaa Daree in local name) making up 83.6% and 16.4%
respectively. According to the information from the district administration office, the
district posses a tropical heavy and erosive rainfall characteristic as well as cool to
moderate climate. The mean annual rainfall is about 1200mm and the average annual
temperature is about 14.7oc. In general, the majority of the areas in the district
receives both long (Belg) and short (Meher) rainy seasons which extends from
February to May and June to September respectively (ORA, 2011).

Concerning the demographics of the study area, according to the report from the Arsi
zone Planning and Economic Development Office (APEDO), the total population of
the pilot district is estimated to be 24,5046 in 2010 of which 50.23% constitutes
females. It is estimated that only 7.6% of the total population are living in the town
which indicates poor urbanization in the zone (APEDO, 2010).

Mixed agriculture characterizes the district. Cool temperature crops are the major
annual crops grown in the district during the long rainy seasons in addition to inset
which is widely cultivated by farmers including some vegetables like potato, onion
and the like. On the other hand, hot land crops which include maize and sorghum are
also grown as supplementary crops including fruits which indicates its potential for
theses crops as well. Due to several reasons like shortage of improved technologies,
decline in soil fertility, crop disease and pests, the production and productivity of the
district have shown a declining trend. For example, the average yield of major crops
reported was 16.66 qt/hectare in 1999 and 15.71 qt/hectare in 2003 indicating a

33
declining trend. In the district, livestock agriculture is another key economic activity
for the farmers. The district is known for its cattle production and holds first and
second position in cattle and small ruminant population in the zone respectively.
However, livestock farming is highly constrained by shrinkage in the grazing land,
inadequate feed supply, poor infrastructure, underdeveloped market facilities and
prevalence of cattle and small ruminant diseases among others. Although traditional
oriented practices, apiculture farming is also widely practiced and forms another
source of income for most farmers (ibid).

With regard to institutions and services, according to data from the district
administration, there are about 55 active farmers’ associations with total member
farmers of 21,083 in 2010 and three Service Cooperatives with 3,536 total member
farmers. For the district, agricultural service cooperative is one of the most important
agricultural institutions through which farmers access agricultural inputs and credit
services. These institutions sometimes involve in output marketing of agricultural
products mainly to stabilize the market for member farmers in the case of critical
market problems. However, a report in this regard shows that farmers still suffer from
the high cost of agricultural inputs, lack of supply and dalliance, and lack of access to
financial services. Although it is at an infant stage, small scale industries (mostly
grain mills) and hand craft activities are also exist in the district making another type
of economic sub sector for the community. Data from APEDO show that there is a
road density of 57.7km per 1000 km2 in the district making 0.32 km per 1000 people
(APEDO, 2010).

34
 Map of the district, study area

Control
Controlkebeles
kebeles

Participant
Pilot kebeles
kebeles

Figure 2. Map of the study area

Source: www.oromiyaa.com/english/images/Kofele.pdf: consulted on 20/06/13

3.2 Data Sources, Sampling and Collection Techniques

The study used both qualitative and quantitative data collected from secondary and
primary sources. Secondary data related to the study were collected from project
reports, Zonal and regional offices of Agriculture and Rural Development and
APEDO of the pilot zone. Whereas primary data were collected from randomly
selected farm households from both participants and non-participants using formal
survey by the project staff.

In doing so, all the five pilot kebels where the project implemented and another five
Kebeles outside the project area within the same pilot district were selected as control
based on their similarity in socio-economic and agroecological characteristics with the
target kebeles. Moreover, in selecting the control kebeles, their accessibility to main

35
road and agricultural extension services, and their potential opportunity for the
specific commodity of intervention were considered. In addition, their geographical
location (non-adjacent kebeles) was taken into account so that information flow from
participant to non-participant kebeles minimized. This helps one to reduce the
probability of contamination as a result of the spill-over effect from participant to the
non-participant households. Following the identification of the participant and eligible
non-participant, a total of 201 farm households (100 from participant and 101 from
non-participant households) were randomly drawn based on probability proportional
to size. In the months of February and March, 2013, the project staff administered the
same structured questionnaire to the selected participant and non-participant
households in order to collect data on a variety of topics.

Using the structured questionnaire, information on outcome variables and different

topics that include household resource endowments, access to agricultural and
markets services, and demographic characteristics of the respondents were collected
from both groups using the same questionnaire at the same time. In addition to the
data collected through formal survey by the use of structured questionnaire, informal
communications through telephone and e-mails was made with project staff, district
and kebele experts, and representatives of farmers’ associations in order to generate
additional information that was useful to understand and better describe the overall
project situations. Moreover, this type of information help to broaden the qualitative
data gathered from secondary sources enriching the interpretations of the result of
such qualitative analysis.

Table 1. Sample size by Kebele

Project targeted Kebeles Non-targeted (Control) Kebeles

Sample Participant N % Sample Total eligible N %
Kebeles HHs Kebeles HHs
We/Bendelichu 131 16 16 Garmama 1882 32 32
Ro/Ashoka 270 31 31 Wamanye 1200 21 21
Alkeso
We/Abosa 233 28 28 Quma 1313 23 23
Mamo
We/Wolu 151 18 18 Abosa 918 16 16
Adamonye
Benjo Ashoka 56 7 7 Guchi 544 9 9
Total 841 100 100 Total 5857 101 101

36
 3.3 Data Analysis Techniques

3.3.1 Qualitative and descriptive analysis

The qualitative information generated from secondary sources and informal

interviews on changes in organizational and institutional aspect of agricultural input
and output marketing in the district as a result of the intervention were narrated and
described qualitatively. These results from the qualitative analysis were also used to
supplement the results from the quantitative analysis. Moreover, descriptive statistics
such as mean, standard deviation, percentages and cross tabulations were used in
analyzing the data collected through formal surveys by employing SPSS.

3.3.2 Empirical Analysis

3.3.2.1 Propensity score matching

In order to deal with the research questions at hand, a non-experimental method called
propensity score matching (PSM) was selected as the best alternative analytical
method available to estimate the impact of the intervention based on the available data
at hand. We selected PSM among the available non-experimental methods because of
three reasons: (1) baseline date was not available for the project, (2) treatment
assignment was not based on random, and (3) PSM is assumed as the second best
alternative available to that of experimental design in minimizing selection bias
(Baker, 2000).

In non-experimental methods, the major problem is the presence of selection bias

arising from non-random location of the program and the selection of households into
the program. As a result, three possible sources of bias can be identified. These are
observable and unobservable characteristics at household and community levels and
spell-over effect due to the program on non-participant households (Bernard et al,
2010; Heckman et al, 1998; Ravallion, 2005). Therefore, as a result of these
problems, the differences that can be observed between both groups may, either

37
totally or partially, reflect differences that exist initially between participants and non-
participants rather than the effects of the intervention.

PSM estimates impact by controlling for the households’ observable characteristics

and compare the outcomes of program participants with that of matched non-
participants and thus minimizes the first bias. However, removing bias due to
unobservable characteristic remains the main drawback of this method. The problem
bias due to spill over effect can be tackled through constructing control groups that
share the same socioeconomic conditions and similar agro-ecology that help to ensure
the validity of PSM (Bernard et al, 2010; Heckman et al, 1998).

In spite of its deficiencies, PSM is being observed as the most widely used approach
in the recent literature dealing with development impact evaluations (Baker, 2000). It
is increasingly becoming very attractive to evaluators with time, money and baseline
data constraints as it can be applied to a single cross-sectional data. Following this,
the mathematical specification of the PSM method was presented. According to the
same author, there are several steps in implementing PSM. The main once are
estimation of the propensity scores using binary model, choice of a matching
algorithm, checking overlap (common support) condition between the groups and
testing the matching quality.

Specifying the model

The impact of the intervention under consideration on a given outcome (Y) can be
given as:

τ = Y D =1 −Y D = 0 (1)

Where is the treatment effect due to participation in the program, Yi is the outcome
variable observed on household i , and Di indicates whether household i participated
in the program or not. In this case, however, Y D = 1 and Y D = 0 never be
observed at the same time for a given household. As a result, it is impossible to
estimate the individual treatment effect . Hence, one needs to construct

38
counterfactuals. In principles, there are two types of average treatment effects,
average treatment effect (ATE) and Average Treatment Effect on the Treated (ATT).
In the literature, ATT is preferred as it reports the effect of an intervention for whom
the program was intended and thus useful for policy makers (Heckman et al, 1997). It
can be specified as:
τ = E τ/D = 1 = E Y /D = 1 − E Y /D = 1 2

By doing so one can answer the question of how much did participant households
benefit from the program as compared to what they would have experienced had the
program is not taken place. As mentioned above, only data on / = 1 are
available. The problem to an evaluator is, therefore, to find E / = 1 (i.e., data on
participant households in case what would have they experienced had they not
participated in the program). Hence, one needs to find a proper substitute by using the
mean outcome of comparable individuals through establishing a control group in
order to estimate ATT. However, this can be possible only after dealing with
selection bias through correcting the differences between both groups. Thus, by
subtracting E Y /D = 0 from both sides of the above equation, one can arrive at
equation (3) for ATT.

E Y /D = 1 − E y /D = 0 = τ + E Y /D = 1 − E Y /D = 0 3

Now, both terms in the left hand side can be observed and possible to estimate ATT
given that, if and only if E Y /D = 1 − E Y /D = 0 = 0 (i.e., when there is no self-
selection bias). Under social experiments, however, it is impossible to get rid of
selection bias completely and thus we need to introduce two basic assumptions, the
Conditional Independence Assumption (CIA) and Common Support Condition that
help to deal with this problem. The CIA is given as:

Y ⊥ D/X 4

Where Y stands for non-participants, ⊥ indicates independence and is a set of

observable characteristics. This means that, given a set of observable characteristics (
) which are not affected by participation in the program, potential outcomes are

39
independent of treatment assignment. This implies that the selection is solely based on
observable characteristics ( ), and variables that influence treatment assignment and
potential outcomes are simultaneously observed (Bryson et al, 2002; Caliendo and
Kopeinig, 2008). Thus, after adjusting for observable differences between participant
and non-participant households, the mean outcome will be the same for D = 1 and D =
0, hence, E Y /D = 1, X = E Y /D = 0, X .

According to Rosenbaum and Rubin (1983), instead of conditioning on X, they

suggest conditioning on propensity score (P(X)) when propensity score is defined as
the probability of participation for household i given a set which is households
characteristics (i.e.,P X = pr D = 1/X . Deriving the propensity scores from the
discrete choice model and constructing the comparison groups, through matching
exercises, PSM solves the problem of selection bias. Therefore, the differences
between participant and non-participant households are reduced to the attribute of
treatment assignment and thus unbiased estimation of the impact of an intervention
can be possible (Liebenehm et al, 2009; Rosenbaum and Rubin, 1983).

The second assumption is about the common support region which is the region where
the balancing score has positive density for both groups. This assumption rules out
perfect predictability of D given . That is given by:

0 < pr D = 1/X < 1 5

By excluding the tails of the propensity score distribution, X , this assumption tries to
improve the quality of the matches although it is done at the cost of probability that
sample size might be considerably reduced. Yet, it is only over regions of common
support that non-parametric matching methods can be meaningfully applied (Bernard
et al, 2010; Rosenbaum and Rubin, 1983). Based on the above two assumptions, ATT
can be rewritten as:

τ = E[Y − Y /D = 0, p x ] = E[Y /D = 1, p x ] − E[Y /D = 0, p x ] 6

40
Where p x is the propensity score calculated based on the covariates . Then, the
PSM estimator according to equation (6) can be explained as the mean difference in
outcomes over the common support region appropriately weighted by the propensity
score distributions.

3.3.2.2 Propensity scores estimation using binary response model

Estimation of propensity score can be possible using either logit or probit models so
that the household’s probability of participation is predicted. For this study, however,
logit model was used although both provide similar results (Gujarati, 1999;
Rosenbaum and Robin, 1983). In calculating the propensity scores, observable
household characteristics (covariates) were used for the logit model which are not
affected by the program but they simultaneously affect household’s participation
decisions and the outcome variables. Then, participants with similar propensity scores
were matched with their counterparts from non-participants. The dependent variable
for the logit model, in this case, is participation in the program which takes the value
of 1 if a household participated in the program and 0 otherwise. The specification for
the logit model is:

e) *
P = 7
1 + e) *
Where, , is the probability of participation in the program for the ith household and it
ranges from 0-1, is a function of N-explanatory variable which is also expressed as:

Z = β +∑β x +U 8

Where, = 1, 2, 3, - --, n, is an intercept, is regression coefficients to be

estimated or logit parameter, is a disturbance term, and is pre-intervention

characteristics.

Similarly, the probability that a household belongs to a non participant group is given
as:

41
 1
1−P = 9
1 + e) *
Then, following that, the odds ratio can be formulated as:
P 1 + e) *
= = e) * 10
1−p 1 + e2)*

In the above formula, is simply the odds ratio in support of participating in the

program. Finally, by taking the natural log of the equation (10) the log of odds ratio
can be written as:

E
Pi β < ∑> β A
34 = Ln 7 9 = Ln :e ; =?@ = =* B = Z = β + C βD XD 11
1 − pi
DF

Where G is log of the odds ratio in favor of participation in the program, which is not
only linear in , but also linear in the parameters.

3.3.2.3 Choice of matching algorithm

On the basis of propensity scores calculated based on observable characteristics,

participants and nonparticipants whose propensity scores fall in the common support
can be matched using different matching algorithms. As to Shahidur et al (2010), the
most commonly known once include nearest-neighbor (NN) matching, caliper and
radius matching, and kernel matching. They all provide similar results given the CIA
and the common support condition assumptions are satisfied (Caliendo and Kopeinig,
2008).

According to Caliendo and Kopeinig (2008), the most straightforward matching

estimator is Nearest Neighbor (NN) matching. The idea behind NN is that the
individual from the control group is chosen as a matching partner for a treated
individual that is closest in terms of the propensity score. In doing so, matching can be
done with or without replacement. In the case of matching with replacement, a
comparable individual can be matched to more than one treatment individuals, which
would result in increased quality of matches and decreased precision of estimates
where as in the case of matching without replacement, a comparable individual can be

42
used only once. Matching without replacement increases bias but it could improve the
precision of the estimates. In situations where the treatment and comparison units are
very different in terms of their propensity scores, finding a satisfactory match by
matching without replacement can be very problematic.

As to the same author, NN matching faces the risk of bad matches if the closest
neighbor is far away. As a result, evaluators use the other alternative matching
algorithm called caliper by imposing a tolerance level on the maximum propensity
score distance (caliper) with the aim to avoid the problem of bad match. Caliper
matching means that an individual from the comparison group is chosen as a matching
partner for individual in the treated group that lies within a given propensity score
range (caliper) and is closest in terms of the propensity score. The smaller the size of
the neighborhood the better is the quality of the matches. However, the smaller the
dimension of the neighborhood, the more the probability of getting some of the
treated individuals without matched. One possible drawback in caliper matching as
Smith and Todd (2005) cited in Caliendo (2008) argue is that it is difficult to know a
priori what choice for the tolerance level is reasonable.

The other commonly used matching algorithm is Kernel Matching. Unlike to the
above matching algorithm which uses only a few observations from the comparison
group to construct the counterfactual outcome of a treated individual, kernel matching
uses the weighted averages of (nearly) all individuals in the comparison group to
construct the counter factual outcome. Accordingly, all treated units are matched with
a weighted mean of all controls with weights which are inversely proportional to the
distance between the propensity scores of treated and controls. Kernel weights the
contribution of each control group member so that more importance is attached to
those comparators providing a better match. The difference from caliper matching,
however, is that those who are included are weighted according to their proximity
with respect to the propensity score. According to Caliendo and Kopeinig (2008), a
shortcoming of this approach is that likely bad matches are used as the estimator
includes individual observations from the control group for all treatment observation.
Hence, the proper imposition of the common support condition is of key importance
for the kernel matching method. In kernel matching, setting the tolerance level is often

43
not be obvious, However, the use of kernel matching with a bandwidth of 0.25 is very
common in the literature (Mendola, 2007).

3.3.2.4 Checking common support condition and testing matching quality

According to Bryson et al (2002), imposing a common support condition is to ensure

that any combination of characteristics observed in the participant group can also be
observed among the non-participant group. It is a region between the minimum and
maximum propensity scores of participant and non-participant households
respectively. By comparing households whose propensity scores lies in this area
comparing the incomparable will be avoided. Therefore, checking common support
condition by comparing the minima and maxima of the propensity score in both
groups is an important step in the estimation procedures. In doing so, Caliendo and
Kopeinig (2008) suggest that all observations whose propensity score is smaller than
the minimum and larger than the maximum in both groups must be discarded from the
analysis.

It must be noted that the main purpose of the propensity score matching exercise is
not to perfectly predict the probability of selection into treatment but to balance all
covariates. While differences in covariates between participants and non-participants
are expected before matching, the difference must be removed after matching.
Therefore, it is necessary to test the matching quality once a common support
condition is satisfied. The idea behind the balancing test is to ensure whether the
propensity score is sufficiently balanced or not after matching (Caliendo and
Kopeinig, 2008; Dehejia and Wahba, 2002; Rosenbaum and Rubin, 1983).

Finally, using the propensity score, match pairs are constructed using alternative
methods of matching algorithms. Then the impact of an intervention can be computed
as the difference between the simple mean of the outcome variable of interest for
participant and non participant households. Accordingly, ATT is obtained by
averaging these differences in outcome variable/s ( ) across the k matched pairs of
households as follows:

44
 O
1
ATT = CKY ∈MF − Y ∈MF N 12
K
F

The result computed by equation (12) based on the above procedures can be taken as
the impact of the program and hence a positive or negative value of ATT suggests that
participant households have higher or lower of outcome variable than non-
participants.

3.3.3 Choice of Variables for Logit Model and Indicators of Outcome

Variables

3.3.3.1 Variables selected for propensity score estimation

In general, the choice of covariates for the propensity score model should be based on
theory and previous empirical findings (Black and Smith, 2004; Heckman et al, 1998;
Heckman and Smith, 1999). In the literature, one can find some advice regarding the
inclusion (or exclusion) of covariates in the propensity score model. It is suggested
that only variables that affect simultaneously the treatment status and the outcome
variable need to be included (Sianesi, 2004; Smith and Todd, 2005). Given that
unconfoudedness assumption3 requires the outcome variable(s) to be independent of
treatment assignment conditional on the propensity score and thus one must choose a
set of covariates that credibly satisfy this condition. In other words, only variables that
are unaffected by treatment need to be included in the model. To ensure this,
therefore, variables should either be fixed over time (time invariant) or measured
before participation. For the current study, since there have been no data available
before intervention for the program, only time invariant household characteristics
were feasible to include in the model and hence variables that are fixed over time
were used.

3
The unconfoudedness assumption states that conditional on X, non-participant outcomes have the
same distribution that participants would have experienced if they had not participated in the
programme and vice versa (Caliendo and Kopeinig, 2008).

45
The question, in cases of uncertainty of the proper specification, is whether to include
too many as much as possible rather than too few variables. As to Heckman et al
(1997), omitting important variables can potentially increase bias in resulting
estimates. However, according to Bryson et al (2002) and Caliendo and Kopeinig
(2008), there are two reasons why over-parameterized models should not be used.
Firstly, it may be the case that including extraneous variables in the propensity score
model exacerbate the support problem. Secondly, although the inclusion of non-
significant variables in the propensity score specification will not bias the propensity
score estimates or make them inconsistent but it can increase their variance. On the
other hand, a covariate should only be excluded from analysis if there is agreement
that the variable is either unrelated to the outcome or not appropriate covariate.

However, in case there are doubts about these two points, Rubin and Thomas (1996)
recommend including the relevant variables in the propensity score estimation
procedure. Therefore, as mentioned above, there is a tradeoff involved between the
plausibility of the unconfoudedness assumption and the variance of the estimates.
When using all the available covariates in the estimation procedure, bias arises from
selecting a wide bandwidth in response to the weakness of the common support.
When using a small number of covariates, the plausibility of the unconfoundedness
assumption is a problem but common support is not (Black and Smith, 2004).

According to Anderson et al (2009), there are no general guidelines for which

variables to include in the model. However, as stated above, one needs to be guided
by available economic theory and empirical studies to identify observable covariates
that affect both participation and the outcomes of interest (Bryson et al, 2002). Based
on this fact, for the current study, different socioeconomic, demographic and location
factors were identified as below for our propensity score model.

46
Table 2. Observable covariates for selection bias adjustment, their description and
measurement

Variables Description Types and Measurements

Dependent variables
TYPFRMER Participation in the Dummy, 1 if yes, 0 otherwise
intervention
Independent variables
SEXHH Sex of the respondent Dummy, 1 if male, 0 otherwise
AGEHH Age of the household head Discrete, in years
EDULVLHH Education level of Dummy, 1 if read and write, 0 otherwise
household head
TOTLFMSZ Total family size of the Discrete, number of household members
household
EXPFRMG Farming experience of the Discrete, in years
household head
SZEOWNLN Total land owned Continuous, in hectare
DISTMKTN Distance to the nearest Continuous, in kilometers
market
DISTEXTO Distance to extension office Continuous, in kilometers

3.3.4 Outcome variables and their measurement

In the current study, four variables were considered as outcome indicators. These are
productivity, market surplus, net income and proportion of resources allocated (as
indicators of market orientation).

Productivity is measured by the quantity of inputs used for the market oriented
commodities of interventions (Liverpool-Tasie et al, 2011). The project has facilitated
input supply by engaging and supporting the different actors so that participant
households had access to agricultural inputs which are very important to increase
productivity. Households required extra labor in addition to the family labor for
specific commodities like inset especially during peak season (like harvesting).
Therefore, for this particular commodity, the intensity of labor use was measured

47
using person days. Marketed surplus, according to Wolday (1994) is the quantity of a
commodity actually sold in the market without meeting farmers’ consumptions and
utilization requirement. The project intervention is expected to improve the quantity
of produce taken to the market and this market surplus was measured as the
percentage increase in volume. Household net income is the net income generated as a
result of household’s participation in the program and measured in Birr. It is the
difference between the total revenue and the total cost incurred by the household for
the generation of that revenue for the commodities of intervention. Following the
approaches adopted by Berhanu and Dirk (2008), the proportion of land allocated for
specific commodities, possession of improved bee hives for apiculture and the number
of sheep allocated for fattening were used to estimate the impact of the program on
the market orientation behavior of participant households.

All the required tests were conducted on the data set before the actual estimation was
undertaken. The existence of strong multicollinearity between the explanatory
variables was checked using the VIF4 as suggested by Gujarati (2004). In the same
way, for the dummy variables, contingency coefficient test was employed. On the
other hand, White (1980) argues that the existence of heteroscedasticity creates a bias
in the standard error leading to biased estimate. Therefore, the existence of
heteroscedasticity was checked by using the Breusch- Pagen test (hettest) in STATA.
Ultimately, the impact of the program on the outcomes interests were estimated using
a PSM algorithm (psmatch2) developed by Leuven and Sianesi (2003).

4
VIF stands for Variance Inflation Factor which is a formal method for determining the presence of
multicollinearity. VIF's measure how much variances of estimated regression coefficients are inflated
when compared to having uncorrelated predictors (Gujarati, 2004).

48
CHAPTER FOUR: RESULTS AND DISCUSSION

4.1 Qualitative Approach

Under this section, based on information from different sources like secondary
sources, online communication with district officials, project staff and representatives
of target households, qualitative description were provided on how the program was
organized and carried out to elucidate the effects of the intervention on organizational
and institutional aspects of the pilot district and the zone at large.

4.1.1 Organizational and institutional arrangements

a) Involving and linking value chain actors

Lack of coordination among important stakeholders is believed to be one of the most

factors explaining the low performance of the agricultural sector in Ethiopia (Mulat,
1999; Tsedeke et al, 2011). Similarly, prior to the intervention in the pilot district, the
interaction and linkages among important actors to support the development of the
agricultural sector was observed to be weak. However, after the intervention was
carried out, the approach adopted by the project enables functional interaction among
a number of actors who have directly or indirectly been involved in different stages in
the value chain development of the commodities of intervention (OBoARD, 2011).

The project facilitated and supported the involvement of these actors along the value
chain to enable them play different roles that support target households so that farmers
had improved access to a variety of services as a result their agricultural production
and productivity could be improved. The different stakeholders involved and the roles
they played during the intervention as reported and witnessed by district experts and
farmers' representatives was summarized below in Table 3.

49
Table 3. Types and roles of stakeholders involved along the value chains of
commodities of intervention

Stakeholders Role and responsibilities Types of Level of

commodities thier
supported involvement
Kebele Mobilize farmers, facilitate All target High
administration linkage between farmers and commodities
other supporting actors, support
learning and experience sharing
events
District office of Provide technical support, support All target High
agriculture & other actors involved in the commodities
rural program, facilitate and support
development different activities useful to
farmers
Research centers Provide training, technical All target High
backup, basic improved commodities
technologies and host experience
sharing tours/visits
Livestock Provide technical support and Small Medium
Development market information ruminant
Agency fattening and
apiculture
Concentrate feed Supply concentrates feed to Small High
& by-product farmers engaged in fattening ruminant
suppliers activities fattening
Public & private Provide veterinary services to Small Medium
veterinary service target households facilitated by ruminant
providers development agents fattening
Local saving and Provide financial services to High
credit groups target households and needy All target
(informal) private traders involved in the commodities
value chain facilitated by the
project
Village level Supply locally produced All target High
input suppliers important agricultural inputs commodities
(private) demanded by target households
Local honey Buy honey from target Apiculture Medium
traders households and sell to other
middlemen or supermarkets
Local carpenters Construct improved bee hives and Apiculture High
and foundation produce foundation sheet
sheet sellers supported by experts from

50
 research centers and sell to
beekeepers
Farmer Provide different inputs All target High
cooperatives demanded by farmers & collect commodities
and market products from
members with premium price
Provide credit services and All target Low
OCSSC financial management trainings to commodities
target households
Source: Own based on online consultation with OBoARD and Kebele leaders,
farmers’ representatives and secondary sources

b) Improving input supply system

The low production and productivity in the study area due to lack of adequate and
sustainable input supply was mainly attributed to the coordination failure among the
important stakeholders. Lack of input supply is believed to be a major bottleneck for
the production of market oriented agricultural commodities in the pilot district
(OBoARD, 2011). In order to improve the situation, the project worked on facilitating
and creating linkages among the different stakeholders, establishing and strengthening
private sectors including local traders who wanted to engage in supplying inputs to be
used for market oriented commodities and the supply of all other related inputs and
services.

Accordingly, for example, private food oil industry supplied cotton meal input via
local private traders organized by the project so that small ruminant fatteners had
access to it. By facilitating and assisting the linkage between this supplier and local
traders, the project helped small scale local traders to engage in this new business.
This input was used as supplementary feed for small ruminant fattening by target
farmers engaged in fattening. Moreover, the involvement of private actors such as
traders and entrepreneurs in supplying inputs which were entirely the role of the
public sector before the intervention by opening alternative village shops for
commodities of interventions were very encouraging. These local private inputs
traders were linked at higher levels to input importers at the capital city. For inputs
that can be locally prepared, the project organized interested individuals and engaged
them in producing these at village level in collaboration with research centers

51
(OBoARD, 2011) and then supplied to target households with affordable prices. In
doing so, these newly initiated small scale business beginners were financially
supported through credit services based on group collateral arrangements with both
formal (OCSSC) and informal financial service providers facilitated by the project.

However, though the project creates business opportunities for small scale traders so
that target households’ access to agricultural inputs would be improved, peer group
collateral arrangement hampered revolving of credit funds as a result of delayed loan
repayment by some of the traders. They also not well aware of the working principles
that OCSSC follows in providing credit services. Hence, some of the traders quit their
work affecting farmers’ access to the inputs as one of the district officials witnessed
(online interview, 13 July 2013). Reducing the size of collateral group, looking for
other options as alternative to group lending, for example encouraging and
introducing private saving could be explored. Moreover, in order to improve the
linkage between the organized small scale traders, financial service providers and
other actors in the system, integrating saving and credit and financial management
trainings into rural financial service provision is essential.

c) Improving output marketing system

The other important problem to the development of agricultural sector in the district
was the lack of access to market by households for their produce. Usually market
prices are drastically lower at the time of harvesting for most agricultural products
since there is relatively high supply at local market places. For example, according to
the project document, the price of honey at the local market during the harvesting
period is much less than the prices of the same quality at other times due to high
production and thus more supply (OBoARD, 2011). Taking into account this
situation, the project facilitated the establishment of producer groups so that farmers’
access to market information and bargaining power could be improved thereby
enabling them to fetch a better price for their produce. The information from the
district via telephone has shown that these producer groups have been developed to
primary cooperatives and legally registered by the districts’ cooperative promotion

52
desk which could further strengthen their capacity to better serve their members in
input/output marketing (online interview with one of district officials, July, 13, 2013).

Following the establishment of producer groups, the project has also tried to link them
to potential buyers and processors within and outside their region. Leaders of farmers'
groups and experts from the district witnessed that due to the linkage created with
potential buyers, producers now able to have improved access to better market for
their produce as a result of the intervention through their groups. For example, they
indicated that fattened animals, honey and other product buyers from different places
came several times to make an agreement with producer groups and collect their
produces with better prices. Moreover, farmers were technically supported
particularly to synchronize their fattening cycle with national and religious holidays
where demand for fattened small ruminant is usually high. This helped producers to
get higher prices since the groups had a reasonable stock size that was advantageous
for group to bargain on the price and to reduce marketing costs.

4.1.2 Enhancing farmers’ capacity

In order to enhance the production and productivity of the target households, the
project recognized the importance of continued capacity building through different
means. Accordingly, different trainings on a variety of topics were facilitated and
organized throughout the project life. As part of capacity building, several experience
sharing and study tours were also conducted for selected farm households so that they
were later used as resource farmers for their community to teach other farmers in their
vicinity. Research centers were used as the main actors in this regard. Apart from
providing improved technologies, they involved in the capacity building programs of
the project by sending trainers and technical advisors throughout the training
programs. As a result, target households were benefited from the training programs in
the course of exercising the application and use of new technologies (OBoARD,
2013).

53
 4.2 Descriptive and Empirical Approaches

Under this section, following descriptive analysis using SPSS, propensity score
estimation, matching methods, common support region, balancing test and finally
ATT estimation was provided.

4.2.1 Descriptive results

4.2.1.1 Characteristics of sample households

Table 4 below presents the results of the test for the difference in means between
participant and non-participant households over a set of control variables. Variables
that are assumed fixed over time were used for this study to describe household
characteristics since baseline data is not available. In general, we observe that
participant and non-participant households differ in certain characteristics. Compared
to non-participants, on average, participant households own more land and less family
size. Similarly, non-participant households are living far away from market center and
extension office compared to participating households.

Table 4. T-test for equality of means between groups (continuous variables)

HH characteristics Participant Nonparticipa Difference in t-value

(N=100) nt means
(101)
Mean STD Mean STD Mean STD5
Age of HH head 43.95 8.710 44.090 7.482 -0.139 1.145
-0.121
(years)
Total family size 5.240 2.046 6.390 2.249 -1.146 0.303 -3.78***
Farming experience of 22.71 6.927 22.850 6.842 -0.141 0.971 -0.146
HH head (in years)
Size of ownd land (ha) 1.213 0.950 1.024 0.643 0.194 0.115 1.688*
Distance from market 1.500 1.518 2.376 1.996 -0.876 0.250 -3.51***
center (km)
Distance from 2.306 1.297 3.452 1.722 -1.147 0.215 -5.34***
extension office (km)
Note: *** and** means significant at the 1%, 10% probability levels, respectively (t-
test is used for differences in means)

QRS@T QRSTT
5
STD for mean difference = P +
U@ UT

54
On the other hand, the chi square test for dummy variables as presented in the
following table indicated that there is no significant difference between the two
groups in terms of sex composition. However, the result showed that there is a
significant difference in terms of educational achievement between them indicating
that participant households are better educated as compared to their counterparts.

Table 5. Chi-square test for equality of proportions between groups (dummy

variables)

HH Participant Nonparticipant Total X2

characteristi Category N % N % N %
cs
Sex of HH Male 90 90 84 84 174 87 2.017
head Female 10 10 17 17 27 13

Educational Can read & 94 94 87 87 181 90 3.47***

level of HH write
head Cannot read 6 6 14 14 20 10
& write
Source: Own calculation based on the survey data

4.2.1.2 Descriptive results of outcome variables

In this subsection, we presented an overview of the raw effect of the program on

certain outcome variables for which data were available. The ultimate objective of the
program was to increase rural incomes particularly for targeted households. The
preliminary analysis of the effect of the intervention on different outcome variables
was given in Table 6. When the unmatched effect of the program on a set of outcome
variables is compared for participant and non-participant households, it is clear that
participation in the program is associated with sharp differences in most of the
outcome variables computed. A simple comparison of means, for example, reveals a
large difference in total household income: 5,778.02 Birr for participant households
compared to 5,196.06 Birr for non-participant households. This is a difference of
about 11 percent, which is significant and relatively large in the context of the study
area.

55
Table 6. t-test for equality of means between groups for outcome variables

Participant Nonparticipant
Outcome variables households households t-test
(N=100 ) (N=101 )
Mean SE Mean SE t-value
Intensity of input use
Intensity of labor use
(PDa) for inset 107.4900 2.9200 91.2400 4.6900 2.937***
Additional farm tools
used for inset
cultivation (No.) 3.3100 0.1270 1.1300 0.0820 14.457***
Value of input used for
apiculture (Birr) 122.5300 24.9840 65.4000 25.6580 1.5950
Quantity of cotton meal
used for fattening(Kg) 89.4900 8.5810 4.1600 1.5870 9.779***
Value of fattening input
use (Birr) 776.8820 81.7204 78.2720 26.3108 8.137***
Value of input use for
fruit (Birr) 68.4800 19.3130 58.6100 17.2420 0.3810
Productivity
Productivity of
inset (Quintal/ha) 1435.740 58.7620 1291.030 33.3900 2.141**
Productivity hives
(Kg/hive) 28.2400 0.7870 25.5800 0.8610 2.278**
Commercial
orientation
Land allocated for inset
(%) 0.6694 0.0301 0.5988 0.0192 1.979**
Possessions of
improved hives (%) 0.4200 0.0500 0.3000 0.0460 1.823*
Sheep allocated for
fattening (No.) 5.3500 0.2570 3.5700 0.3380 4.208***
Proportion of sales
Sale of inset (%) 0.8235 0.0081 0.7853 0.0070 3.556***
Sale of honey (%) 0.7186 0.0255 0.5839 0.0225 3.959***
Sale of fattened small
ruminant (%) 0.94333 0.01177 0.75148 0.03587 5.082***
Sale of fruit seedling
(%) 0.08 0.02 0.09 0.021 -0.255
Average net income
Total net income 5778.02 47.58 5196.06 66.67 7.105***
Note: ***, ** and * means significant at the 1%, 5% and 10% probability levels,
respectively (t-test are used for differences in means), a person days

56
The descriptive results in the above table suggest that participant households have a
larger income than non-participant households. The superior performance of
participant households in terms of income including most of the other outcome
variables as indicated in the table above are in line with what one would expect given
household’s participation in the program. However, given the non-random distribution
of the program, these results do not say anything about the impact of the intervention
as a result of participation in the program. From the presented descriptive statistics, it
remains unclear whether the observed differences in the outcome variables listed are
attributable to the impact of the program or to differences in the underlying
characteristics of participant versus non-participant households. Hence, further in-
depth econometric analysis was carried out using PSM in order to address this
problem.

4.2.2 Empirical results

The descriptive analysis of the effect of the program on outcome variables suggests
that participant households performed much better in most of the outcome variables.
However, similar analyses of household characteristics presented in the previous
section revealed that there are also substantial differences in the underlying
characteristics of participant versus non-participant households. Therefore, based on a
simple comparison of means without controlling for these differences, it is impossible
to identify causality and to attribute the observed differences in all the outcome
variables to the impact of the program. Hence, in the subsequent sections we
presented an econometric analysis that was used to estimate the causal impact of
participation in the program mainly on household productivity, market orientation and
participation, and income outcomes.

4.2.2.1 Estimating propensity scores

In order to estimate the propensity scores, logistic regression was used. Our dependent
variable for the logit model is a binary that indicate households’ participation decision
in the program which takes the value of 1 if a household participated in the program
and 0 otherwise.

57
The logistic regression model was tested for the existence of strong multicollinearity
and found to be weak (Table 11) indicating that our model is not suffering from the
existence of strong multicollinearity. As a result there were no explanatory variables
dropped from the estimated model. Furthermore, the White’s and Breusch-pagan tests
for heteroskedasticity found to be insignificant to reject the null hypothesis suggesting
the non-existence of a serious heteroskedasticity problem (P=0. 5928, see Table 2 in
the Appendix).

According to the results presented in Table 7 based on the output of logit regression,
the estimated model appears to perform well in the subsequent intended matching
exercises. The pseudo-R2 value is about 0.19 suggesting that program households do
not exhibit much different characteristics overall and thus finding a good match
between program and non-program households would not be difficult.

Table 7. Logit result of program participation

Covariates Coef. Std. Err. Z

AGEHH .0164093 .0358498 0.46
SEXHH 2822197 .4748278 0.59
EDULVLHH .6421835 .6015255 1.07
TOTLFMSZ -.3143394 .0852972 -3.69***
EXPFRMG .0015008 .0405158 0.04
SZEOWNLN .4398058 .231271 1.90*
DISTMKTN -.207486 .0940409 -2.21**
DISTEXTO -.4669114 .1126572 -4.14***
_cons 1.485161 1.331961 1.12
Number of obs 201
LR chi2(8) 53.78
Pseudo R2 0.1930
Log likelihood -112.42862
Prob > chi2 0.000
Note: ***, ** and * means significant at the 1%, 5% and 10% probability levels,
respectively
Source: Own calculation based on the survey data

The estimated coefficients as indicated in the above table revealed that four
explanatory variables (i.e. family size, owned land, distance to the nearest market and

58
extension office) significantly influence household participation in the program.
Households with relatively better land ownership, living nearer to market centers and
extension office are more likely to participate in the program. On the contrary,
households with larger family size are less likely to participate in the program than
households with smaller family size.

4.2.2.2 Matching participant and non-participant households

As discussed earlier, imposing a common support condition is one of the most

important steps in the estimation procedure in order to ensure that any combination of
characteristics observed in the participant group can also be observed among the non-
participant households in the control group. For the current data, this was done by
taking the region between the minimum and maximum propensity scores of
participant and non-participant households respectively. In doing so all observations
whose propensity score are smaller than the minimum and larger than the maximum
in both groups were discarded from the analysis.

According to the result from Table 8 below, the estimated propensity scores vary
between 0.134 and 0.970 for program participant and 0.025 and 0.863 for non-
program participant households. Hence, according to Bryson et al (2002) and
Caliendo and Kopeinig (2008), the common support region would then lie between
0.134 and 0.863. Therefore, 11 participants and 14 non-participant households whose
propensity score is smaller than the minimum and larger than the maximum were
discarded from the analysis in order to avoid comparing the incomparable once during
the analysis. Therefore, checking common support condition by comparing the
minima and maxima of the propensity score in both groups is an important step in the
estimation procedures.

Table 8. Distribution of estimated propensity scores

Variable Obs Mean Std. Dev. Min. Max.
Total 201 0.498 0.247 0.025 0.970
Participant 100 0.617 0.210 0.134 0.970
Nonparticipant 101 0.379 0.220 0.025 0.863

59
 4.2.2.3 Choice of matching algorithm

On the basis of propensity scores calculated, participant and non-participant

households whose propensity scores fall in the common support need to be matched
with the appropriate matching algorithm. According to Dehejia and Wahba (2002),
choice of feasible matching algorithm is guided by three indicators namely the
balancing test (test of equality of means), pseudo-R2 and matched sample size. They
state that a matching estimator is preferred if it able to balance all explanatory
variables (i.e., insignificant mean differences between the two groups), bears a low R2
value and results in large matched sample size.

Accordingly, the results of our tests of matching algorithms suggest that kernel
matching with a bandwidth of 0.25 is found fulfilling the guiding criteria and thus
thought to be the best estimator for the data at hand (see Table 3 in the Annex).
Therefore, in the subsequent analysis, our estimation results and discussions are the
direct outcomes of this matching algorithm based on a bandwidth of 0.25. Heckman et
al (1998) argues that kernel matching has got an advantage of lower variance since
more information is used. It associates the outcome of the participant household with
matched outcome given by a kernel-weighted average of all households in the non-
participant group that is used to construct the counterfactual outcome.

4.2.2.4 Testing the matching quality

Using the selected matching algorithm based on the guiding principles provided by
Dehejia and Wahba (2002) the balancing of propensity score and covariates were
checked. This time, reduction in the mean standardized bias after matching and the
equality of means between the matched sample households would be checked. In
doing so, t-test and chi-square test for joint significance of the variables were used as
indicators for assessing the balancing of propensity scores and covariates after
matching (Caliendo and Kopeinig, 2008).

The test for matching quality has shown that in the present matching models, the
mean standardized bias difference in X before matching is in the range of 0.1% to
31.2% in absolute value (Table 4 in the Annex). However, after matching, the
60
remaining standardized difference of X for all covariates lies between 1.0% and
24.9% (Table 5 in the Annex). Here we observe a reduction in mean standardized
bias. The t-test result after matching in Table 9 below show that the chosen covariates
that exhibited statistically significant differences before matching turned out to be
insignificant and thus all the variables are balanced. Similarly, the low pseudo-R2
value and the insignificant likelihood ratio tests support the hypothesis that states both
groups have identical distribution in covariates X after matching suggesting that the
matching exercises was effective in balancing the characteristics in the participant and
matched non-participant group so that we can safely proceed to the estimation
exercises. Therefore, these results were used to evaluate the effect of the program
among groups of households having similar observed characteristics.

Table 9. The balancing test before and after matching

Variable Before matching After matching

Treatme Control t-value Treatme Control t-value
nt nt
Continues Variables
_pscore .62652 .62166 0.17 .58011 .48711 1.23
Age of HH head 43.911 43.376 0.47 43.292 43.568 -0.14
(in years)
Total family size 5.2376 5.0891 0.44 5.4607 5.9287 -0.51
(No.)
Farming experience 22.723 22.713 0.01 22.191 22.441 -0.07
of HH head (years)
Size of owned land 1.2122 .95911 2.15** 1.0873 1.0602 -0.20
(ha.)
Distance from the 1.4851 1.4554 0.13 1.5899 1.9489 -0.63
nearest market
center (km)
Distance from 2.298 2.296 0.01 2.4674 2.8464 -0.85
extension office
(km)
Dummy Variabels Treatme Control X2 Treatme Control X2
nt nt
Sex of HH head (%) .90099 .86139 0.87 .89888 .86293 0.26
Educational level of .94059 .9505 -0.31 .93258 .92962 -0.11
HH head (%)
Note: ** means significant at the 5% probability level
Source: Own calculation based on the survey data

61
 4.2.2.5 Estimating treatment effect on the treated

The results of our estimate of the impact of agricultural improvement intervention

through market development after controlling for observable confounding factors was
presented in Table 10 below. The results suggest that the program was found
improving intensity of input use, productivity, commercial orientation and proportion
of sales of most of the commodities of intervention for participant households
eventually increasing their income. Specifically, we found a statistically significant
program effect for intensity of input use for inset cultivation and small ruminant
fattening (P<0.01). This result is in line with the findings of Deschamps-Laporte
(2013) and Mendola (2006). In the context of the study area, input used for these
program activities, inset cultivation and small ruminant fattening, was mainly
accessed by program participant households through marketing from other sources
facilitated by the program. This means that the program has improved the average use
of these inputs for participant households. Moreover, as argued by Goetz (1992), is
being a net buyer in the market determine the extent of once market participation.
Higher market participation drives more productivity by providing incentives for the
households (Reardon and Timmer, 2005).

Similarly, the empirical analysis for the productivity of commodities of intervention

for which data is available suggests that there is statistically significant difference
between participant and non-participant households. The program was found
improving the productivity of inset and honey by about 12% and 9% for participant
households respectively (P<0.01 and P<0.1). The result for inset productivity is, of
course, in line with one’s expectation as the previous results of input use for this
particular commodity is significantly higher for participant than that of non-
participant households. However, the striking result for the productivity of honey for
which there is no statistically significant difference in terms of input use between the
groups might be attributed to the difference in management practices between them as
a result of the simultaneous capacity development intervention for participant
households as discussed under the qualitative analysis section.

In the same way, our analysis also clearly demonstrated that the program had a
significant and posetive effect on the quantities of produce taken to the market for
62
inset, honey and fattend small ruminants. More specifically, participant households
increased the proportion of their produce marketed on average by 4%, 12% and 18%
respectively for the indicated commodities of intervention. This implies that the
program enhenced target household’s participation in the market as a result of higher
productivity.

Based on the concept of commercialization measurement which states that

commercialization can be measured in terms of the degree of allocation of the
different resources such as labor and land to competing agricultural activities and total
sales of agricultural commodities (Keister and Nee, 2001), we measured the effect of
the program on the level of households’ commercialization by comparing these
indicators between the two groups. Accordingly, our estimate revealed that the
program has achieved improving the levels of resource allocated to and possession of
market oriented commodities. More precisely, the program helped participant
households to increase the allocation of land and small ruminant for market oriented
activities by about 14% and 44% respectively while improving the possession of
improved beehives by about 30%. The result is also significant at a 1% and 10%
probability levels respectively (Table 10). Households’ decision to increase resource
allocation and possession of commercial oriented resources, in this case, could be
stimulated due to market signals as a result of their improved access to market
facilitated by the program (Berhanu and Dirk, 2009; Pingali, 1997). Moreover, the
involvement of relevant actors across the value chains as depicted by the results of our
qualitative analysis could also be part of the reason.

63
Table 10. Estimated ATT for the outcome variables

Nonparti
Outcome variables Participant cipant Difference S.E t-value
Intensity of Input use
Intensity of labor use for
inset cultivation (PD) 108.6773 89.6455 19.0319 6.363 2.99***
Additional tools used for
inset cultivation (No.) 3.3596 1.0484 2.3112 0.159 14.51***
Value of input used for
apiculture (Birr) 62.1930 56.5350 5.6580 28.56 0.20
Quantity of cotton meal
used for fattening(Kg) 97.6136 3.7869 93.8268 8.984 10.4***
Value of input use for
fattening (Birr) 847.024 83.6408 763.3839 91.13 8.4***
Value of input use for
fruit(Birr) 61.4944 56.2478 5.2466 28.44 0.18
Productivity
Productivity of inset
(Quintal/ha) 1413.148 1242.827 170.3203 70.21 2.43**
Productivity of improved
hives (Kg/hive) 28.4688 25.7934 2.6754 1.477 1.81*
Commercial Orientation
Land allocated for inset
(%) 0.6713 0.5754 0.0960 0.040 2.39***
Possessions of improved
hives (%) 0.4270 0.2987 0.1283 0.074 1.73*
Sheep allocated for
fattening (No.) 5.2830 2.9711 2.3120 0.427 5.41***
Proportion of sales
Sale of inset (%) 0.8232 0.7812 0.0420 0.012 3.41***
Sale of honey (%) 0.6981 0.5832 0.1149 0.040 2.89***
Sale of fattened small
ruminant (%) 0.9433 0.7573 0.1827 0.052 3.54***
Sale of fruit seedling (%) 0.5815 0.5702 0.0112 0.278 0.04
Average net income
Total net income 5757.384 5200.388 556.9964 94.65 5.9***
Note: ***, ** and * means significant at the 1%, 5% and 10% probability levels,
respectively
Source: Own calculation based on the survey data

As stated earlier, the ultimate objective of the current program under consideration
was to improve the income of participant household from increased participation in

64
the production and marketing of commercial oriented agricultural inputs and outputs.
Based on the available data, we measured the impact of the program on the income of
participant households. Accordingly, the result of our estimation indicated that the
program has significantly increased the cumulative net income of households who
participated in the program by about 10% (P<0.01). This finding was consistent with
the findings reported by Egziabher et al (2013) who evaluated the impact of the IHEP
program on the income of participant households in the northern part of Ethiopia and
reported similar results. The effect of the program on net income of participant
households is, of course, in line with one’s expectation since the program succeeded
in significantly improving the levels of input use, resource allocation, productivity,
and market participation of its target households.

65
CHAPTER FIVE: CONCLUSION AND RECOMMENDATION

This study provides crucial insights into and important evidence on the impact of
agricultural improvement interventions through market development on the
livelihoods of farmers in rural Ethiopia using cross sectional data collected in the
beginning of 2013. Using matching estimator, the study evaluated the program
implemented jointly by the Regional Bureau of Agriculture and Research Institute
between 2009 and 2012. The primary objective of the current study was to provide
robust evidence for the regional government of Oromia on whether the program could
contribute to household productivity and income so that the information generated
will be used in the course of scaling up of the program experience to similar areas.

The main research question of the study was what would have been the productivity,
market participation and income outcomes of participating households if the
intervention had not been in place. Answering this question requires observing
outcomes with-and-without the program for the same household. Nevertheless, it is
impossible to observe the same object in two states simultaneously thus results in a
missing data problem. Thus, non-participant households without the program were
constructed to be used as the control group for this reason.

Taking into account the agro-ecological criteria, assumed difference in terms of

distance to minimize spill-over effect and accessibility of the kebeles, target and non-
target kebels within the pilot district were identified for the study. A random sample
size of 100 program participant and 101 non-program participant households were
randomly selected using the techniques of probability proportionate to sample size.
Accordingly, the primary data for the study was, thus, collected from a total of
201farm households coming from the same pilot district using the same instruments.
The study also examined the process of household targeting and availability of
baseline data and found that household assignment to the program was based on a
non-random selection process and baseline data were not available. Therefore, the
propensity score matching technique was found to be the most suitable estimator for
the data at hand as it is capable of extracting a comparable pair of participant-

66
nonparticipant households in a non-random program setup and absence of baseline
data (Dehejia and Wahba, 2002).

The qualitative analysis has shown that the program has improved the linkages and
interaction among different actors along the value chains of the commodities of
intervention. By doing so, participant households’ access to different agricultural
inputs and outputs marketing services have been enhanced although the level of
involvement and service delivery by some of the actors (like OCSSC) are not
satisfactory as pointed out by respondents as a results of the institutional arrangements
in place. On the other hand, the empirical findings, as expected, revealed that
participation in the program is significantly determined by a combination of factors
like family size, size of owned land, distance to the nearest market and extension
office. Households with relatively better land ownership, living nearer to market
centers and extension office are more likely to participate in the program. By contrast,
households with larger family size are less likely to participate in the program than
households with smaller family size.

After controlling for these initial differences, our impact estimate using PSM
suggested that the program significantly raised the intensity of input use, commercial
orientation, productivity, and proportion of sales of most of the commodities of
intervention for participant households eventually increasing their income. More
specifically, allocation of land and small ruminant for market oriented activities was
increased by large which is about 14% and 44% respectively while improving the
possession of improved beehives by about 30%. Similarly, the program was found
improving the productivity of inset and honey by about 12% and 9% respectively.
Moreover, the results have shown that participant households increased the proportion
of their produce marketed by 4%, 12% and 18% for inset, honey and fattend small
ruminant respectively increasing thier market participation. Similarly, it was evident
from the findings that the program managed to increase the cumulative net income of
participant households by about 10%.

The current study is restricted to an evaluation of certain outcome variables as the

analysis was based on a pre-designed and collected household data on limited
outcome variables. Relatively speaking, the sample size used is also small although it
67
is very common to find similar studies in the literature with a similar sample size.
Moreover, the qualitative analysis carried out is not detailed enough as it was
primarily based on secondary sources backed up with online communications with
limited number of respondents.

Although the results of our impact estimation confirmed that the program had
succeeded in improving the income level (the ultimate objective of the program) of
participating households, further evidences are required that elucidate how the
benefits were distributed among the different categories of participants like poor, rich
and women within the targeted households. As evidences from literature pointed out
that there are firm criticizes about the popular understanding that assumes
participation of the poor in development interventions ensures equitable benefit
(Cleaver and Hamada, 2010). Similarly, it is also interesting to investigate through
further research the reason behind why households with relatively large family size
are less likely to participate as this study found out. Moreover, it is also apparent that
further study is required with sufficient sample size to explore the relative
contribution of each commodities of intervention that might help the regional
government in order to consolidate efforts in the course of extending the experience of
the program as the current study did not capture these due to lack of data. Finally, it is
also worthy revisiting the institutional arrangements adopted by rural financial
institutions like OCSSC to further enhance households’ access to financial services.

68
BIBLIOGRAPHY

Abebaw, D., Fentie, Y. and Kassa, B. (2010) “The impact of a food security program
on household food consumption in North western Ethiopia: A matching estimator
approach”, Food Policy, 35(4): 286–293.

Alberto, A. (2003) “Semi parametric instrumental variable estimation of treatment

response models”, Journal of Econometrics, 113 (2003): 231 – 263.

Ana, R., William R., Masters A. and Shively, G.E. (2008) Linkages between market
participation and productivity: results from a multi-country farm household sample.
Prepared for presentation at the American Agricultural Economics Association
Annual Meeting, July 27-29, Orlando, FL.

Andersson, C., Alemu, M. and Stage S. (2009) Impacts of the productive safety net
program in Ethiopia on livestock and tree holdings of rural households, Discussion
Paper Series, Environment for Development (EfD) Initiative, DP 09-05.

APEDO (Arsi Zone Planning and Economic Development Office) (2010) Atlas of
Arsi Zone, http://www4.fao.org/cgibin/faobib.exe?rec_id=588193&database=faobib&
searchtype=link&table=mona&back_path=/faobib/mona&lang=eng&format_name=E
FMON (last consulted: 21 July 2013)

ATARC (Adami Tulu Agricultural Research Centre) (2010) “Annual report of

research activities”, Oromia Agricultural Research Institute, Addis Ababa, Ethiopia.

Baker, J.L. (2000) Evaluating the impact of development projects on poverty: A

handbook for Practitioners, Washington D.C., World Bank.

Bellemre, M.F and Barrett, C.B. (2006) “An Ordered Tobit Model of Market
Participation: Evidence from Kenya and Ethiopia”, American Journal of Agricultural
Economics 88(2): 324-337.

Bezuxeh, T. and Feleke, A. (1966) “The production and utilization of Ensete in

Ethiopia”, Economic Botany, 20(1): 65-70.

Bryson, A., Dorsett, R. and Purdon, S. (2002) “The use of propensity score matching
in the evaluation of labour market policies” Working Paper No. 4, Department for
Work and Pensions.

Bernard,T., Spileman, D.J., Seyoum, A. and Gabre-Madhin, E. (2010) “Cooperatives

for staple crop marketing: evidence from Ethiopia” IFPRI research monograph, 164.

Gebremedhin, B. and Dirk, H. (2009) “Staple food crops turning into commercial
crops: Case studies of Teff, Wheat and Rice in Ethiopia”, Ethiopian Journal of
Economics, ILRI, Addis Ababa, Ethiopia.

69
Bastiaensen, J., De Herdt, T., D’Exelle, B. (2005) “Poverty Reduction as a Local
Institutional Process”, World Development, 33 (6): 979-993.

Bromley, D.W. (1998) “Property regimes in economic development: Lessons and

policy implications”, in: E. Lutz (Ed.), Agriculture and the environment: Perspectives
on sustainable rural development, World Bank, Washington, DC: 83–91.

Black, D.A. and Jeffrey A.S. (2004) “How Robust is the Evidence on the Effects of
College Quality? Evidence from Matching", Journal of Econometrics, 121(1-2): 99-
124.

Caliendo, M. and Kopeinig, S. (2008) “Some practical guidance for the

implementation of propensity score matching” IZA Discussion Paper No. 1588,
University of Cologne.

Coulter, J., Millns, J., Tallontire, A. and Stringfellow, R. (1999) “Marrying farmer
cooperation and contract farming for agricultural service provision in liberalizing
economies in Sub-Saharan Africa” Natural Resources Perspectives, No. 48, ODI,
London:

Chowdhury, S., Negessa A. and Torero M. (2005) “Market institutions: Enhancing the
value of rural-urban links” MTID Discussion Paper 89, International Food Policy
Research Institute,Washington DC.

Chaudhury, N. and Parajuli, D. (2006) “Conditional Cash Transfers and Female

Schooling: The Impact of the Female School Stipend Program on Public School
Enrolments in Punjab, Pakistan” Policy Research Working Paper 4102(9), The World
Bank.

Cleaver, F. and Hamada, K. (2010) “‘Good’ Water Governance and Gender Equity: a
Troubled Relationship”, Gender and Development, 18(1): 27-41.

Cleaver, F. (2012) Development through Bricolage: Rethinking Institutions for

Natural Resource Management, Routledge.

Cleaver, F. and Franks, T. (2008) “Distilling or Diluting: Negotiating the Water

Research-Policy Interface”, Water Alternatives, 1(1): 157-177.

Alemu, D., Gabre-Madhin, E. and Dejene, S. (2006) “From farmer to market and
market to farmer: Characterizing smallholder commercialization in Ethiopia” Paper
submitted for ESSP Policy Conference on Bridging, Balancing, and Scaling up:
Advancing the rural growth agenda in Ethiopia, Addis Ababa, Ethiopia.

De Janvry, A., Fafchamps, M. and Sadoulet, E. (1991) “Peasant household behaviour

with missing markets: Some paradoxes explained” Economic Journal, 101:1400-
1417.

Dorward, A., Kydd, J., Morrison, J. and Poulton, C. (2005) “Institutions, markets and
economic development: Linking development policy to theory and praxis”,
Development and Change, 36(1): 1-25.

70
Dorward, A., Poole, N., Morrison J., Kydd, J. and Uray, I. (2003) “Markets,
institutions and technology: Missing links in livelihoods analysis”, Development
Policy Review, 21(3): 319-332.

Douglas, J.M. and Cook, S. (2012) “Fostering Institutional Creativity at Multiple

Levels: Towards Facilitated Institutional Bricolage”, Water Alternatives, 5(1). 1-19

Davis, K., Ephraim, N., Edward. K., Daniel, M.A., Martins, O., Richard, M. and
Jackson, N. (2010) Impact of Farmer Field Schools on Agricultural Productivity and
Poverty in East Africa. IFPRI Discussion Paper 00992.

Abebaw, D., Fentie, Y., Kassa, B. (2010) “The impact of food security program on
household food consumption in North Western Ethiopia, A matching estimator
approach”, Food Policy 35 (2010) 286-293

Dehejia, R.H. and Wahba, S. (2002) “Propensity score matching methods for non
experimental causal studies”, The Review of Economics and Statistics, 84(1):151-161.

Deschamps-Laporte, J.P. (2013) “The impact of extension services on farming

households in Western Kenya: A propensity score approach” Economics Department,
Örebro Business School, Örebro University, Örebro.

Gabre-Madhin, E. and Goggin,I. (2006) “Does Ethiopia need a commodity exchange?

An integrated approach to market development” Paper presented at the Ethiopian
Strategy
Support Program 2006 Policy Conference, Addis Ababa.

Ezemenari, K., Rudqvist, A. and Subbarao, K. (1999) “Impact evaluation: A note on

concepts and methods” poverty reduction and economic management network, World
Bank.

Franks T., and Cleaver, F. (2007) “Water governance and poverty: a framework for
analysis”, Progress in Development Studies, 7(4): 291–306.

FAO (Food Agriculture Organization), (2000) “Impact assessment of agricultural

research: Context and state of the art” Revised version of a paper prepared by the
Impact Assessment and Evaluation Group (IAEG) of the Consultative Group on
International Agricultural Research (CGIAR) for the ASARECA / ECART / CTA,
Workshop on impact assessment of agricultural research in Eastern and Central
Africa, Rome, Italy.

Foster, E. (2003) “Propensity score matching: An illustrative analysis of dose

response”,
Lippincott Williams & Wilkins, Inc. Journal of Medical Care, 41(10):1183-1192.

Egziabher, G.K., Mathijs, E., Deckers, J., Gebrehiwot, K., Bauer, H. and Maertens,
M. (2013) “The Economic Impact of a New Rural Extension Approach in Northern
Ethiopia” Bioeconomics Working Paper Series, Department of Earth and
Environmental Sciences, University of Leuven.

71
Gujarati, D.N. (1999) Essentials of Econometrics, 2nd edition, Mc-Graw-Hill
Companies.

Gujarati, D.N., (2004) Essentials of Econometrics, 4th edition, Mc-Graw-Hill

Companies.

Govereh, J., Jayne, T.S. and Nyoro, J. (1999) “Smallholder commercialization,

interlinked markets and food crop productivity” Cross-country evidence in eastern
and southern Africa.

Goetz, S.J., (1992) “A Selectivity model of household food marketing behaviour in

Sub-Saharan Africa”, American Journal of Agricultural Economics, 74(2): 444-452.

Gebremedhin, B., Hoekstra, D. and Tegegne, A. (2006) “Commercialization of

Ethiopian agriculture: Extension service from input supplier to knowledge broker and
facilitator” IPMS, Working Paper 1. ILRI, Nairobi, Kenya.

Godtland, E.M., Sadoulet, E., Janvry, A., Murgai, R. and Ortiz, O. (2004) “The
Impact of Farmer-Field-Schools on Knowledge and Productivity: A Study of Potato
Farmers in the Peruvian Andes” Development Economics Research Group, The
World Bank.

Heckman, J.J., Ichimura, H. and Todd, P. (1997) “Matching as an econometric

evaluation estimator: evidence from evaluating a job training programme”, The
Review of Economic Studies, 64:605-654.

Heckman, J.J., Ichimura, H., Smith, J. and Todd, P. (1998) “Characterizing selection
bias using experimental data”, Econometrica, 66: 1017–1098.

IFAD (International Fund for Agricultural Development) (2012) “Methodologies for

Impact Assessments for IFAD” Rome, Italy.

Jayne, T., Zulu, B. and Nijhoff, J. (2006) “Stabilizing food markets in Eastern and
Southern Africa” Food Policy 31(4): 328–341.

Jalan, J. and Ravallion, M. (2003) “Estimating the benefit incidence of an antipoverty

program by propensity-score matching”, Journal of Business and Economics
Statistics, 21(1): 19-30

Johnson, S. (2012) “From Microfinance to Inclusive Financial Markets: The

Challenge of Social Regulation”, Oxford Development Studies, Routledge, London,
UK.

Keister, L.A. and Nee, V.G. (2001) “The rational peasant in China. Flexible
adaptation, risk diversification and opportunity”, Rationality and Society, 13 (1).

Key, N., Sadoulet, E., and DeJanvry, A. (2000) “Transactions costs and agricultural
household supply response”, American Journal of Agricultural Economics, 82(2):
245-259.

72
Liebenehm, S., Affognon, H. and Waibel, H. (2009) “Impact assessment of
agricultural research in West Africa: An application of the propensity score matching
methodology” Paper presented at the International Association of Agricultural
Economics Conference, Beijing, China.

Liverpool-Tasie, L.S., Kuku, O., Ajibola, A. (2011) “A Review of Literature on

Agricultural Productivity, Social Capital and Food Security in Nigeria” IFPRI, NSSP
Working Paper 21.

Long, N. (2001) Development Sociology: Actor Perspective, London and New York.

Mendola, M., (2007) “Agricultural technology adoption and poverty reduction: A

propensity score matching analysis for rural Bangladesh”, Food Policy, 32(3): 372-
393.

MoFED (Ministry of Finance and Economic Development) (2010) “Growth and

Transformation Plan (GTP)” 2010/11-2014/15 Draft, Addis Ababa, Ethiopia.

Mulat, D. (1999) “Agricultural Technology, Economic Viability And Poverty

Alleviation In Ethiopia”, Department of Economics, Addis Ababa University,
Presented to the Agricultural Transformation Policy Workshop, Nairobi, Kenya.

Mwape, F. (2009) “Paper presented at the opening speech of Towards Priority

Actions for Market Development for African Farmers” ILRI Conference, Nairobi,
Kenya.

Jaleta, M., Gebremedhin, B. and Dirk, H. (2009) “Smallholder commercialization:

Processes, determinants and impact” Discussion Paper No. 18, Improving
Productivity and Market Success of Ethiopian Farmers Project, ILRI, Nairobi, Kenya.

Nssah, B. (2006) “Propensity score matching and policy impact analysis” Poverty
reduction
Group, World Bank, Wasihngton, D.C.

Omoto, W. (2003) “Impact assessment of urban agriculture research and development

in Nairobi” Kenya.

Oechmke, J.F., Anandajayasekeram, P., Masters, W.A. (1997) “Agricultural

Technology Development and Transfer in Africa Impacts Achieved and Lessons
Learned” USAID, Office of Sustainable Development, Bureau for Africa, SD
Publication Series, Technical Paper No. 77, Washington, D. C.

OBoARD (Oromia Bureau of Agriculture and Rural Development) (2011) “Annual

Report on the Progress of Agricultural Improvement Intervention through Market
Development Intervention in Kofale Woreda” Arsi Zone, Oromia National Regional
State.

OBoARD (Oromia Bureau of Agriculture and Rural Development) (2013) “Annual

Report on the Progress of Agricultural Improvement Intervention through Market

73
Development Intervention in Kofale Woreda” Arsi Zone, Oromia National Regional
State.

ORA (2011):http://oromiyaa.com/index.php?option=com_content&view=article&id=
189%3Akofele-woreda-profile&Itemid=611 (last consulted 06 June 2013).

Ponniah, A., Martella, D.R. and Rukuni, M. (1996) “A training manual on R & D
evaluation and impact assessment of investments in agricultural and natural resources
research” SACCAR, Gaborone, Botswana.

Puskur, R., Ponniah, P., Berhe, K., Hoekstra, D. (2007) “Partnerships for enhancing
market-led Innovation processes – Experiences and lessons from IPMS Ethiopia”
Paper presented at the Innovation Africa Symposium, Kampala.

Pingali, P. (1997) “From subsistence to commercial production systems: The

transformation of Asian agriculture”, American Journal of Agricultural Economics,
79(2): 628–634.

Pingali, P. and Rosegrant, M. (1995) “Agricultural commercialization and

diversification:
Processes and policies”, Food Policy, 20(3): 171–185.

Paul, J.G., Sebastian M., Patrick P., Laura B.R., Christel, M.J.V. (2011) Impact
Evaluation in Practice, The World Bank.

Ravallion, M. (2005) “Evaluating anti-poverty programs” Policy research working

paper 3625, World Bank, Washington D.C.

Reardon, T. and Timmer, C.P. (2005) “Transformation of Markets for Agricultural

Output in
Developing Countries Since 1950: How Has Thinking Changed”, in: Evenson, R.E.,
Pingali, P. and Schultz, T.P. (eds) Agricultural Development, Amsterdam, 135-149.

Rosenbaum, P.R. and Rubin, D.B. (1983) “The central role of the propensity score in
observational studies for causal effects”, Biometrika, 70(1): 41-55.

Rover, T. and Dixon, R. (2007) “Operational guidelines for assessing the impact of
agricultural research on livelihoods: Good practices from CIMMYT” Impact
Targeting and Assessment (ITA) Unit, Mexico.

Rubin, D. B., and Thomas, N. (1996) “Matching Using Estimated Propensity Scores:
Relating Theory to Practice”, Biometrics, 52(1), 249-264.

Sianesi, B. (2004) “An evaluation of the active labour market programs in Sweden”
The Review of Economics and Statistics, 186(1):133-155.

Shahidur, R.K., Gayatri, B.K., Hussain A.S. (2010) Handbook on Impact Evaluation:
Quantitative Methods and Practices, World Bank, Washington, DC.

74
Shiferaw, B. Obare, G. and Muricho, G. (2006) “Rural institutions and producer
organizations in imperfect markets: Experiences from producer marketing groups in
semi-arid eastern Kenya” Socio-Economics and Policy Working Paper 23, ICRISAT,
Patancheru, India.

Timmer, C.P. (1997) “Farmers and markets: The political economy of new
paradigms”, American Journal of Agricultural Economics, 79(2):621–627.

Tsedeke, A., Shiferaw B., Gebeyehu S., Amusable B., Negash, K., Assefa, K., Eshete,
M., Aliye, S., and Hagmann J. (2011) “A systems and partnership approach to
agricultural research for development – Lessons from Ethiopia”, Outlook on
Agriculture Journal, 40(3): 213-220.

Valadez, J. and Bamberger, M. (1994) “Organizational and Management Issues in

Program Evaluation”, in: Valadez J. and Bamberger M. (eds.) Monitoring and
Evaluating Social Programs in Developing Countries: a Handbook for Policymakers,
Managers and Researchers, Washington DC, The World Bank.

Wolday A. (1994) “Food grain marketing development in Ethiopia after the market
reform 1990, A case study of Alaba Siraro district” PhD Dissertation, Veriog Koster,
Berlin, Germany.

75
APPENDICES

Table 1. Conversion factor for labor supply (person day equivalent)

Age group (in years) Male Female

<10 0.0 0.0
10-13 0.35 0.35
15-50 1.0 0.80
>50 0.55 0.5
Source: Storck et al., 1991

Table 2. Multicollinearity test for explanatory variables

. vif

Variable VIF 1/VIF

agehh 3.24 0.308263

expfrmg 3.02 0.331141
totlfmsz 1.19 0.837137
szeownln 1.17 0.853262
distmktn 1.06 0.940920
distexto 1.03 0.968362

Mean VIF 1.79

Table 3. Multicollinearity test for explanatory variables

Breusch-Pagan / Cook-Weisberg test for heteroskedasticity

Ho: Constant variance
Variables: fitted values of typfrmer

chi2(1
1) = 0.29
Prob > chi2 = 0.5928

76
 Kernel density of propensity score distribution

2
1.5
Density
1 .5
0 Kernel density estimate

0 .2 .4 .6 .8 1
psmatch2: Propensity Score

Pscore of Participant in common support after matching

Participant households
Participant household in common support
kernel = epanechnikov, bandwidth = 0.0753

Figure 1. Kernel density of propensity scores of participant households after matching

for participant households

Kernel density estimate

1.5 1
Density
.5
0

0 .2 .4 .6 .8 1
psmatch2: Propensity Score

Pscore of Non-participant in common support after matching

Non-participant households
Non-participant household in common support
kernel = epanechnikov, bandwidth = 0.0793

Figure 2. Kernel density of propensity scores of participant households after matching

for non-participant households

77
Table 4. Performance of different matching estimators

Matching Performance criteria

estimator Balancing test Pseoudo-R2 Matched sample
size
Nearest-Neighbour
(NN) Matching
NN(1) 6 0.139 176
NN(2) 6 0.119 176
NN(3) 6 0.124 176
NN(4) 6 0.113 176
NN(5) 6 0.104 176
Caliper & Radius
Matching
0.10 8 0.034 168
0.25 6 0.133 190
0.50 5 0.186 200
Kernel Matching
with band width of
0.10 6 0.111 176
0.25 8 0.007 176
0.50 7 0.067 176

Table 5. Propensity score and covariate balance before matching

. pstest _pscore agehh sexhh edulvlhh totlfmsz expfrmg szeownln distmktn distexto,sum

Mean t-test
Variable Treated Control %bias t p>|t|

_pscore . 6 2 6 52 .6 2 1 6 6 2.2 0 .1 7 0 .8 6 9
agehh 4 3 . 9 11 43 . 3 7 6 6.6 0 .4 7 0 .6 3 6
sexhh . 9 0 0 99 .8 6 1 3 9 11 . 6 0 .8 7 0 .3 8 7
edulvlhh . 9 4 0 59 .9505 -3 . 3 - 0 .3 1 0 .7 5 8
totlfmsz 5 . 2 3 76 5. 0 8 9 1 6.9 0 .4 4 0 .6 6 0
expfrmg 2 2 . 7 23 22 . 7 1 3 0.1 0 .0 1 0 .9 9 2
szeownln 1 . 2 1 22 .9 5 9 1 1 31 . 2 2 .1 5 0 .0 3 3
distmktn 1 . 4 8 51 1. 4 5 5 4 1.7 0 .1 3 0 .8 9 5
distexto 2 . 2 98 2.296 0.1 0 .0 1 0 .9 9 2

Pseudo R2 LR chi2 p>chi2 MeanB MedB

0.0 31 8.6 1 0.47 4 7 .1 3.3

78
Table 6. Propensity score and covariate balance after matching

. pstest _pscore agehh sexhh edulvlhh totlfmsz expfrmg szeownln distmktn distexto,sum

Mean t-test
Variable Treated Control %bias t p>|t|

_pscore .5 801 1 .48 711 4 3.0 1 .23 0 .22 1

agehh 43 .29 2 43. 568 - 3.4 -0 .14 0 .89 1
sexhh .8 988 8 .86 293 1 0.5 0 .26 0 .79 2
edulvlhh .9 325 8 .92 962 1.0 -0 .11 0 .91 4
totlfmsz 5. 460 7 5.9 287 -2 1.8 -0 .51 0 .61 2
expfrmg 22 .19 1 22. 441 - 3.6 -0 .07 0 .94 1
szeownln 1. 087 3 1.0 602 3.3 -0 .20 0 .84 3
distmktn 1. 589 9 1.9 489 -2 0.3 -0 .63 0 .53 2
distexto 2. 467 4 2.8 464 -2 4.9 -0 .85 0 .39 6

Pseudo R2 LR chi2 p>chi2 MeanB MedB

0. 007 1. 79 0.99 4 14. 6 1 0.5

79