World Development 112 (2018) 136–149

Contents lists available at ScienceDirect

World Development
journal homepage: www.elsevier.com/locate/worlddev

NGOs and international development: A review of thirty-five years of

scholarship
Jennifer N. Brass a,⇑, Wesley Longhofer b, Rachel S. Robinson c, Allison Schnable a
a
School of Public and Environmental Affairs, Indiana University, Bloomington, 1315 E. 10th Street, Bloomington, IN 47405, United States
b
Goizueta Business School, Emory University, 1300 Clifton Road NE, Atlanta, GA 30322, United States
c
School of International Service, American University, 4400 Massachusetts Avenue, NW, Washington, DC 20016-8071, United States

a r t i c l e i n f o a b s t r a c t

Article history: Since 1980, the number of non-governmental organizations (NGOs) in developing countries has
Accepted 31 July 2018 exploded. Published research on NGOs has paralleled this growth, yet there exists scant synthesis of
Available online 22 August 2018 the literature. This article presents a synthesis, while also introducing a collaborative research platform,
the NGO Knowledge Collective. We ask four questions: first, who studies NGOs, and how do they study
Keywords: them? Second, what issues, sectors and places are studied when NGOs are the focus? Third, what effect
NGOs (non-governmental organizations) do NGO activities have on specific development outcomes? And fourth, what path should the NGO
Civil society
research agenda take? To answer these questions, we conduct a mixed-method systematic review of
Development outcomes
Health
social science publications on NGOs, which includes computer-assisted content analysis of 3336
Governance English-language journal articles (1980–2014), alongside a close, qualitative analysis of 300 randomly
Systematic review selected articles. We find, first, that interdisciplinary journals dominate NGO publishing, that research
on NGOs is more qualitative than quantitative, and that practitioners publish, but Northern academics
create most published knowledge. Second, we find the literature is framed around six overarching ques-
tions regarding: the nature of NGOs; their emergence and development; how they conduct their work;
their impacts; how they relate to other actors; and how they contribute to the (re)production of cultural
dynamics. Articles also focus disproportionately on the most populated and/or politically salient coun-
tries, and on the governance and health sectors. Third, we find that scholars generally report favorable
effects of NGOs on health and governance outcomes. Fourth, we propose a research agenda calling for
scholars to: address neglected sectors, geographies, and contextual conditions; increase author represen-
tativeness; improve research designs to include counterfactuals or comparison groups; and better share
data and findings, including results from additional, focused NGO-related systematic reviews.
Implementing this agenda will help reduce bias in decisions by donors, governments, and other develop-
ment actors, which should improve development outcomes.
Ó 2018 Elsevier Ltd. All rights reserved.

1. Introduction
The meteoric rise of non-governmental organizations (NGOs) in
international development over the past three decades is well-
documented. Such growth in NGOs is evident in places like Kenya,
where registered NGOs grew from several hundred in 1990 to
more than 8000 by 2012, and Bangladesh, where by 2005, 90% of
villages were home to at least one NGO (Brass, 2012, 2016; Gauri
& Galef, 2005). Growth has also occurred globally; the Union of
International Associations estimates 37,500 active international
⇑ Corresponding author.
E-mail addresses: brassj@indiana.edu (J.N. Brass), wesley.longhofer@emory.edu
(W. Longhofer), robinson@american.edu (R.S. Robinson), schnable@indiana.edu
(A. Schnable).
NGOs in 2018, and more than 1000 new international NGOs are
incorporated each year in the U.S. alone (Union of International
Associations, 2018; Schnable, 2015). Simultaneously, NGOs have
come to be deeply incorporated into the development process writ
large. The World Bank reports that nearly 90% of currently financed
projects involve the participation of an NGO or other civil society
organization, compared to just 21% in 1990 (World Bank, 2018a).
And, according to the OECD, more than 20% of all bilateral aid flows
through NGOs (OECD, 2015).
The rise of NGOs has been fueled by an often-implicit assump-
tion that private actors, including private not-for-profits, are effec-
tive in expanding democracy via a robust civil society, providing
services when national governments lack sufficient resources or
political will, and reducing global poverty through participatory
and market-based methods (Brass, 2016). Donor mistrust of state

https://doi.org/10.1016/j.worlddev.2018.07.016
0305-750X/Ó 2018 Elsevier Ltd. All rights reserved.
 J.N. Brass et al. / World Development 112 (2018) 136–149 137

intervention and a growing concern over corruption, as well as 4500

widespread resistance to structural adjustment programs, further 4000
amplified the NGO promise in the late 20th century (Banks, 3500
Hulme, & Edwards, 2015). Yet mounting skepticism has met the 3000
rise of NGOs. The cholera epidemic that followed the 2010 earth- 2500
quake in Haiti called attention to the failure of many NGOs that 2000
responded (Schuller, 2012), and the diffusion of policies banning 1500
foreign funding for NGOs suggests a further backlash from devel- 1000
oping country governments (Dupuy, Ron, & Prakash, 2016). 500
Debates over NGOs are not unique to development practition- 0
ers. Concurrent with the NGO boom, an interdisciplinary academic
field drawn from a wide range of scholarly traditions has emerged.
Fig. 1 illustrates the concurrent growth of academic articles on Number of NGOs (with UN Consultancy Status) Total Arcles
NGOs (see ‘‘Data and Methods”) and the number of NGOs that
received consultancy status with the United Nations (Willetts, Fig. 1. Growth in NGOs and Articles about NGOs, 1980–2014.

2015) from 1980 to 2014. Some of this growth reflects trends dri-
ven by prominent academics like Robert Putnam (2000), who were
Third, we focus in on the most commonly asked question in the
instrumental in placing concepts like social capital – of which
literature: what effects, if any, do NGO activities have on specific
NGOs are thought to be a key component—onto the World Bank’s
development outcomes? Although we cannot answer the question
development agenda in the early 2000s. In contrast, Banks et al.
outright, we provide a synthesis of findings reported in published
(2015; see also Edwards and Hulme (1996)) have criticized the
studies. By development outcomes, we follow Sen’s (1999) defini-
impact NGOs have had (or not had) on development, calling atten-
tion of development as expansions in freedoms—not only political
tion instead to their dependency on donors, fragile roots in civil
freedoms, but also social opportunities, protective securities, eco-
society, and technocratic approaches to complex developmental
nomic opportunities and transparency guarantees. The outcomes
challenges.
associated with these freedoms include improvements in health,
Despite (or perhaps because of) such conflicting views of NGOs
education, sanitation, and democratic governance, among others.
among scholars, few have made efforts to systematically aggregate
Our goal in this analysis is to show the possibilities (and limits)
findings from the more than three decades of research that now
of a systematic aggregation of research findings. Specifically,
exists. This paper represents a step toward accumulated knowl-
through a mixed-method analysis of a subset of our corpus, we
edge, using a ‘‘systematic review”1 methodology that provides a
summarize the conclusions authors have drawn about how NGOs
level of rigor that most review articles do not achieve. The paper is
affect specific health and governance outcomes, noting that
part of a larger research agenda and data platform on which all
authors usually report NGOs as having favorable effects.
scholars can collaborate, the NGO Knowledge Collective (www.
Fourth, what path should the ‘‘NGO research agenda” take?
ngoknowledgecollective.org). Based on computer-assisted content
Given the descriptive ‘‘lay of the land” of NGO scholarship, we pro-
analysis of 3336 journal articles about NGOs published between
vide a research agenda for the future based on four needs: to fill
1980 and 2014, alongside a random sample of 300 articles from this
gaps in knowledge, particularly at the intersection of sectors and
corpus, we assess four broad questions.
geographies; to incorporate more scholars from the global South
First, who studies NGOs, and how do they study them? Here,
and from NGOs themselves; to develop and implement better
our analysis reveals a truly interdisciplinary field that has pro-
research designs, particularly those that include true counterfactu-
duced a tremendous number of insights. At the same time, we
als, organizational comparisons, and analysis of contextual condi-
identify potential sources of bias that may explain some of the
tions; and finally, to share data and synthesize findings in more
knowledge gaps that exist. For example, we note a high level of
and better ways.
author affiliation with the NGOs they write about, as well as dis-
For the purposes of this research, we define ‘‘NGO” broadly as
proportionate numbers of articles written by authors at Northern
any nonprofit, non-governmental organization that works in the
institutions. Knowing whether such biases exist in the literature
development, humanitarian, advocacy, or civil society sector in
matters—if the knowledge of NGOs stems from a non-
any country considered a developing country at some point during
representative sample of countries or sectors, we must take care
the period of study (1980–2014).2 These organizations are known
in applying that knowledge to policy and program decisions affect-
by a number of terms in addition to NGOs, such as civil society orga-
ing broader geographic regions or different sectors. Moreover,
nizations (CSOs), community based organizations (CBOs), grassroots
determining what has been researched extensively allows us to
organizations, private voluntary organizations, or faith-based orga-
identify what is not known, a key precursor to designing useful
nizations (FBOs). They can be international, regional, national, or
new research.
local. Included are foundations and religious development organiza-
Second, what is studied when NGOs are the object of analysis?
tions, but excluded are universities, hospitals, churches, political
Specifically, what geographies or sectors are studied, and what
parties, and unions.3
research questions or topics do scholars focus on? We first present
data showing that governance and health are the most common
topics, and that populous locations have many NGOs. We then
2. Theory: existing research and ‘‘the NGO Debate’’
identify a set of six, broad research questions and topics that the
NGO literature has posed and briefly summarize some findings.
Despite thousands of articles related to NGOs and development,
We find that scholars have carried out significant work on a range
syntheses of the literature remain sparse. Early reviews, notably
of places, issues, and questions, but that clear gaps remain.
2
We define a country as developing if it was not a ratified member of the OECD in
every year between 1980 and 2014.
1 3
A ‘‘systematic review” is a specific type of research product designed to Although these latter types of organizations are generally included as ‘‘nonprof-
systematically synthesize literature to answer particular questions (Petticrew & its” in studies of Western countries, they are usually not included in studies of
Roberts, 2006). developing countries. We follow previous scholarship in this decision.
138 J.N. Brass et al. / World Development 112 (2018) 136–149
Fisher (1997, p. 441), responded to largely ideological essays on
what scholars believed about NGOs, and called for better conceptu-
alization of NGOs and more empirical evidence on their benefits or
drawbacks. Similarly, Clarke (1998) focused on the political role
that NGOs can play, but like Fisher (1997), concluded that more
and better research was needed. More recently, reviews have
tended to focus on relatively narrow questions. Kareithi and
Lund (2012), for example, reviewed 31 articles about factors affect-
ing NGO performance, and found a severe lack of voices—both
authors and beneficiaries—from the global South. McLoughlin
(2011) and Brass (2016, p. 44–56) both reviewed existing literature
to assess determinants of NGO-government collaboration or con-
flict, and Brass found that variation in organizational attributes,
NGO regulations, the impetus for and contours of NGO-
government interactions, and the level of interest alignment
among the organizations shape outcomes (2016, p. 45).
Other broad-reaching analyses of NGOs provide theory that we
use to interpret our findings. For example, Watkins, Swidler, and
Hannan (2012) characterized NGOs as organizations that must
manage uncertainty in a variety of forms. They also highlighted
the ways that NGOs can serve as bearers of a dominant world cul-
ture, or ‘‘world society,” spreading prevailing norms throughout
the world. Along these lines, Schofer and Longhofer (2011) found
that countries with ties to international NGOs and donor organiza-
tions tend to have more local NGOs (see also Longhofer & Schofer,
2010). International NGOs, moreover, help connect local NGOs into
broader networks that can confer money, legitimacy, motivation,
and many other valuable resources. Connections to international
NGOs are not equally distributed across countries, however, but
instead vary primarily based on a country’s wealth, with a stark
divide between the global North and South (Beckfield, 2003;
Hughes, Peterson, Harrison, & Paxton, 2009; Smith & Wiest,
2005). Related to this, scholars of development have also noticed
the way that donors and NGOs focus on sectors of activity central
to the world polity. HIV/AIDS, for example, is over-represented
among NGOs in Africa (Morfit, 2011).
In addition to research on the role of NGOs in global processes
and their locations and spheres of work, a basic debate persists
across disciplines about whether NGOs do more harm than good
(see Fisher 1997). On one hand, NGOs are lauded—often in compar-
ison to their public counterparts—and are presumed to be more
efficient, effective, flexible and innovative than government; to
be more other-oriented and ideologically committed to democracy
and participatory pro-poor development; and to be more account-
able and transparent (Bratton, 1989; Fowler, 1991; Owiti, Aluoka,
& Oloo, 2004; World Bank, 1989). The World Bank specifically
highlights the role that non-governmental actors should play in
decision-making about public services in order to make those ser-
vices more democratic, accountable and transparent (World Bank,
2011). Others in this camp argue that NGOs’ participatory
approach mobilizes people, encourages increased information
sharing, fosters alternative political ideas, and empowers the dis-
advantaged (Boulding & Gibson, 2009 citing Putnam, 1993;
Bratton, 1989; Fowler, 1991).
Some of the same authors who note NGOs’ positive aspects,
however, also see the harm they can cause. For example, because
NGOs are able to pay higher salaries, they can draw the most com-
petent employees out of the public sector (Chege, 1999; Schuller,
2009), reducing public sector productivity and morale (Pfeiffer,
2003). Likewise, NGOs can threaten to undermine government
legitimacy (Fowler, 1991). In providing services and encouraging
participatory development and empowerment, NGOs can put pres-
sure on extant political authority, power, and order, which can lead
to backlash (Bratton, 1989).
NGOs, moreover, are frequently critiqued for promoting
governance-related strategies that they don’t actually use them-
selves and may not improve the lives of targeted populations. For
example, critical scholars point to NGOs’ emphasis on participatory
development, but note that many NGOs ‘‘perform” participation to
reinforce donor interests (Mercer, 2003) or they ‘‘abuse” notions of
participation for their own interests (White, 1996). Others critique
NGOs for failing to influence policy change to address deep
inequalities (Banks et al., 2015), or for falling prey to elite capture
(Dill, 2009). Ethnographers have offered a number of critical por-
trayals of NGOs as part of an ‘‘aid machine” that operates mainly
to perpetuate itself (Campbell, 2003; Mosse & Lewis, 2005). Schol-
ars working from this perspective tend to fault NGOs for failing to
deliver the empowerment they promised (Banks et al., 2015), or
having worsened lives they aimed to help (de Waal, 1997).
We note the philosophical and normative preferences built into
these debates. If NGOs can be idealized both as efficient private
alternatives to government and as grassroots challengers to
entrenched power structures, they may be viewed as failures from
one perspective even if they succeed in the other. One of the con-
sequences of the expansion of the NGO literature is that those with
different normative assumptions about the roles of NGOs have
been able to create parallel conversations. A synthesis of the liter-
ature may not resolve the debate—and our positivist approach may
be antithetical to some scholars—but our approach, at a minimum,
identifies lines of inquiry that have diverged, and moves towards
an aggregation of published findings.
3. Data and methods
3.1. Dataset creation
To identify the relevant literature for analysis, we first created a
list of keywords that would allow us to locate pertinent articles
(see Appendix). These keywords relate to organizational terms
(e.g., non-governmental organization, community-based organiza-
tion) and to international development (e.g., names of developing
countries, foreign aid terms). We used this list to search the Aca-
demic Search Premiere Database of EBSCO, which produced a
group of 11,373 articles whose title, abstract or keywords matched
at least one organizational and one international development key-
word. Next, to eliminate false positives, two research assistants
read the title, abstract and keywords of each article to ensure that
it was a research article in which NGOs were an object of analysis.
In the case of disagreement or uncertainty among the research
assistants, one of the authors decided. This process produced a cor-
pus of 3402 articles. We then located a PDF version of 3336 articles
for use in computer-assisted content analysis.4 Finally, we ran-
domly selected 300 articles from the corpus for use in the closely-
read qualitative analysis. (For additional information, see the
Appendix.)
Although systematically constructed, our database of 3336 arti-
cles has limitations. First, scholarly books and gray literature are
excluded, even though such sources contain germane research
about NGOs. Given search limitations and organizational restric-
tions on sharing data, it is likely impossible to identify and include
all relevant gray literature.5 Second, journals exist that meet our cri-
teria but which EBSCO does not include and which we did not
search. We believe the number of relevant articles in such journals
to be relatively limited, and that the more ‘‘mainstream” journals
we capture likely replicate their content. Third, articles plausibly
exist that either our search in EBSCO did not identify or that we
wrongfully excluded in our search for false positives, likely because
4
The remaining 66 articles could either not be located or converted into a format
suitable for text analysis.
5
In future work, the NGO Knowledge Collective hopes to identify a clearly
delimited sub-set of gray literature to compare with the scholarly literature.
 J.N. Brass et al. / World Development 112 (2018) 136–149 139

they had brief (or poorly-written) abstracts and/or irrelevant key- 3.2.2. Coding of 300 articles for quantitative and qualitative analysis
words. We believe the number of such articles to be relatively lim- Second, we randomly selected a sample of 300 articles in the
ited. Fourth, although we know experimental research published corpus, first, to validate the findings from the computer-assisted
by development economists, political scientists, and other research- content analysis, and second, to identify author locations and arti-
ers often examines NGO program outcomes, much of this research cle topics, sectors, research questions, and effects of NGOs. This
does not appear in our corpus simply because ‘‘NGO” or related orga- sample contains approximately 10% of the corpus and is represen-
nizational words tend not appear in the title, abstract or keywords, tative of it in terms of geography, sector, and journal, the features
and may only appear once or twice within the text of the article. This that we can compare between the sample and the corpus. We cre-
means that the corpus likely under-reports the existence of quanti- ated a quantitative data set containing a wide range of features of
tative analysis of outcomes from programs implemented by NGOs. each article in the sample by asking coders a series of questions in
Finally, the articles in our database are of varying quality (discussed Qualtrics (a survey software package) about each article. Although
in the next section). most elements of the Qualtrics coding guide created quantitative
data, some questions were open-ended to facilitate qualitative
analysis, such as that on the effects of NGOs, a summary of the
3.2. Methods
research, and key research questions. We then conducted exten-
sive qualitative analysis of the questions asked in all of the sample,
Methodologically, our approach is most similar to a systematic
and of the reported effects of NGOs in the 113 articles on health
review, which is a specific form of literature review that aims to
and governance contained in the sample.
‘‘identify, appraise and synthesize all relevant studies . . . in order
to answer a particular question or set of questions” (Petticrew &
Roberts, 2006, p. 9). Given that the vast majority of published arti- 4. Findings I: who studies NGOs, and how do they study them?
cles on NGOs comprise small samples and/or case studies, a sys-
tematic review is an appropriate method (versus a meta-analysis, Results from these analyses facilitate the first descriptions of
which requires independent quantitative studies of large sample the literature as a whole. We begin with high-level descriptive
sizes). In nonprofit studies, systematic reviews have been used information drawing on the entire corpus (indicated by n = 3336
by Gazley and Guo (2015) to analyze nonprofit collaboration and or ‘‘corpus”) and the sample (n = 300 or ‘‘sample”), focusing on
by Gugerty and colleagues (2018) to assess the success of self- where NGO research is published, with what methods, and by
help development groups. Characteristically, a systematic review whom.
is ‘‘a systematic search that attempts to identify all studies” that
meet pre-specified eligibility criteria (Liberati et al., 2009). Most
review articles are not systematic in this way. 4.1. Interdisciplinary journals dominate NGO publishing
Our approach does not perfectly match a systematic review,
however. First, the gold standard for systematic reviews is to Table 1 provides a list of journals that appear most often in the
include criteria for the quality of an article. Because we are inter- corpus, accounting for approximately 30% of articles. Articles about
ested in the arguments that scholars make in published work on NGOs appear most commonly in Development in Practice, perhaps
the topic of NGOs, and because it is difficult to establish criteria the most applied peer-reviewed journal on international develop-
for ‘‘quality” in such an interdisciplinary body of research, we ment, and World Development, the leading academic development
chose not to limit inclusion in the corpus in this way. Second, sys- journal. Among the top five most-common journals are also lead-
tematic reviews tend to synthesize findings from a relatively small ing journals in nonprofit studies, Nonprofit & Voluntary Sector Quar-
number of studies (almost always fewer than 100) on a narrow terly and Voluntas, as well as Public Administration and Development,
question. Although our future work will examine a number of which likely reflects the integration of NGOs in state provision of
specific questions about NGOs, our goal in this article is to describe public services in developing countries (Brass, 2016).
the breadth of existing research.
In this study, our findings come from computer-assisted con-
Table 1
tent analysis of the corpus of 3336 articles, alongside both quanti- Top journals represented in corpus.
tative and qualitative analysis stemming from a close read of 300
Journal Total articles (% of corpus)
randomly sampled articles from the corpus.
Development in Practice 217 (6.50%)
Voluntas 113 (3.39%)
3.2.1. Computer-assisted content analysis World Development 94 (2.82%)
First, we conduct computer-assisted content analysis of 3336 Public Administration & Development 62 (1.86%)
Nonprofit & Voluntary Sector Quarterly 53 (1.59%)
articles in our analysis. Specifically, we analyzed keyword frequen-
Third World Quarterly 40 (1.20%)
cies based on a dictionary of 1073 keywords compiled by the Disasters 39 (1.17%)
authors, including country and region names (i.e., Bolivia, sub- Development 37 (1.11%)
Saharan Africa), development sectors (i.e., sanitation, microcredit), Journal of Civil Society 35 (1.05%)
analytic topics (i.e., service provision, civil society), and journal Gender & Development 34 (1.02%)
Social Science & Medicine 33 (0.99%)
titles. Research assistants converted all PDFs to text files and International Journal of Not-for-Profit Law 31 (0.93%)
removed the references using a Python script or by hand. We used Development & Change 28 (0.84%)
PowerGREP in SAS to calculate keyword frequencies within each AIDS Care 22 (0.66%)
article. We then merged information from each article’s biblio- Human Rights Quarterly 20 (0.60%)
Democratization 19 (0.57%)
graphic record (e.g., the year of publication, journal title) with
International Journal of Educational 19 (0.57%)
the keyword frequencies. We also used topic modeling techniques Development
to develop analytic topics in the corpus as a whole.6 Reproductive Health Matters 18 (0.54%)
African Journal of AIDS Research 17 (0.51%)
Society & Natural Resources 16 (0.48%)
Global Governance 16 (0.48%)
6
The authors have a paper in progress, which uses topic modeling to identify Total 963 (28.87%)
trends in this literature.
140 J.N. Brass et al. / World Development 112 (2018) 136–149
Although many articles are published in discipline-specific jour-
nals, we find most research on NGOs appears in interdisciplinary
development journals. Fewer than a handful of the top journals have
a clear disciplinary home, and those that do fall in the latter half of
the table. For example, Democratization is a political science journal,
and several journals on AIDS or reproductive health are public
health journals, but even these journals are more interdisciplinary
than most in their respective fields. The ‘‘big tent” approach to inter-
disciplinarity that nearly all of the journals represent may explain
part of the reason that NGO studies have not been sufficiently syn-
thesized: doing so requires integrating work from a considerable
range of scholarly traditions, norms and interests. Moreover, a total
of 950 journals are represented in our corpus, but more than half of
the journals published just one article on NGOs, further illustrating
the fragmented nature of the field as a whole.
4.2. Research on NGOs is more qualitative than quantitative, and has
become more rigorous over time
Analysis of the research methods in the 300-article sample illu-
minates how aggregating research on NGOs is difficult, not just
because NGOs work in diverse geographic contexts, but also
because researchers use a host of methods to study them. Sixty-
three percent of articles in the sample used qualitative data and
analysis exclusively. But the dominance of qualitative work has
eroded since the 1980s, following the trend toward quantitative
analysis in political science and other social science disciplines
(Sigelman, 2006). For articles published in 2010 or later, roughly
a third use quantitative data exclusively or combined with qualita-
tive data.
The sample also suggests that articles on NGOs have also
become more empirical in general in the last 35 years. Using nei-
ther qualitative nor quantitative data—relying simply on anecdote
or argument—was common in the 1980s and 1990s, as reflected in
Fisher’s critique in 1997. But the share of articles characterized by
coders as ‘‘merely descriptive” dropped from 25% in the 1980s to
10% in the 2010s, while the share using a social science framework
increased from 0% in the 1980s to 66% in the 2010s.7
Case study approaches are dominant in the sample. We find that
54% of articles are studies of a small number of places or organiza-
tions. Case studies may be fruitfully used to study a variety of devel-
opment interventions, management practices, and governance
questions. Yet we find that for half of all case studies, the author
provided no clear rationale for choosing that particular case.
As the low frequency of quantitative data would suggest, the
use of any statistical analysis is the exception rather than the rule.
Descriptive statistics were presented in only 16% of articles.
Regression analysis is rare; cross-sectional models appear in 10%
of papers, and longitudinal models are in only 5% of articles, and
only two articles in our sample use randomized controlled trials
(RCTs). The paucity of RCT studies may reflect NGOs’ infrequent
involvement in RCTs compared to other development agencies,
NGOs being involved in RCTs but not explicitly mentioned in arti-
cles, or a buzz about randomization that is disproportionate to its
actual use as a research tool.
In sum, the methods used in a random sample of 300 articles
are consistent with a literature whose aims are not simply to ana-
lyze causal effects, but to provide thick descriptions of particular
places and projects, to advance theories and ideas about best prac-
tices, and to offer social critiques. The methodological trends in
social science disciplines in the last four decades are visible in
7
The figure for the 1980s should be interpreted with caution given the small
number of articles within the n=300 sample that were published in the 1980s. We
define a social science framework broadly as a process of asking research questions,
developing a theory, collecting data, analyzing, and drawing conclusions.
the NGO literature, but weakly. Given that qualitative work is apt
to remain vital to NGO literature, efforts to publish and share qual-
itative data (such as the Qualitative Data Repository at Syracuse
University) will improve rigor and aid scientific progress.
4.3. Practitioners publish, but Northern academics create most
published knowledge
Fundamental to this analysis is the issue of who creates the
knowledge. We coded the 300-article sample for author’s institu-
tional affiliation and country for authors based at universities.8
The topic of NGOs draws a large share of analysts from outside aca-
demic institutions. About one-third of articles had at least one non-
academic author; more than half of such authors were employees of
NGOs. The high representation of non-academic authors in this sam-
ple is notable because this analysis includes only academic journals,
excluding the grey literature where non-academics often publish
research.
But the modal author in this sample is an academic based at a
university in the global North. Nearly half of academically-
affiliated first authors are from the United States or Canada. Europe
is the next most common region, dominated by universities in the
U.K. Only 16% of articles included an author based at a university in
Asia, Africa, or Latin America. The dearth of Latin American authors
is especially dramatic—in 300 articles, there were only four articles
by scholars based in Latin America. These trends are shaped by
sampling English-language publications, but other factors are
likely at play; India is one of the most-studied countries in this cor-
pus, but only two articles were written by scholars at Indian uni-
versities, where English is spoken. These findings provide
preliminary evidence that supports concerns about North/South
knowledge gaps. What is being written about Southern countries
is mostly by Northern-based scholars, and what Southern-based
scholars are writing about their own countries is not appearing
in the English-language journals that Northern scholars are most
likely to read (see also Kareithi and Lund (2012) on this issue).
There are some indicators of difference in type and quality of
research published by academics and non-academics. We assessed
whether articles had a clear argument; used systematic evidence;
and/or were ‘‘merely descriptive.” Articles written by first authors
with non-academic affiliations were almost twice as likely to be
characterized as ‘‘merely descriptive” compared to those by aca-
demic authors (23% vs. 13%; p < .05), and more likely to lack sys-
tematic data to support their claims (48% vs. 29%; p < .01). The
differences in these measures may be partly explained by the fact
that many of these articles are case studies of the authors’ own
organizations. Thirty-four of 101 articles in the sample with non-
academic authors are about the organizations that fund or employ
the authors. These articles may offer important insider knowledge,
but they may also present a biased view, given that authors may be
unable or unwilling to criticize an NGO that employs them. Many
of these articles are also written by authors at very large, interna-
tional organizations. The NGO insider knowledge that passes into
academic journals thus likely better reflects knowledge of large,
professionalized development NGOs than other kinds of
organizations.
5. Findings II: what do we study?
We turn to questions of what NGO scholars study. We first
assess the sectors and geographies studied. We then address the
overarching research questions and topics analyzed based on a sta-
8
Institutions and departments for up to four authors were recorded if noted in the
article.
 J.N. Brass et al. / World Development 112 (2018) 136–149 141

tistical and qualitative assessment of the 300 randomly-sampled Table 2

articles. Sectors and development priorities represented in corpus.

Sector Total articles (% of Largest keyword

5.1. Governance and health are the most prominent sectors corpus)*
Governance 458 (13.73%) civil society (23.9%)
Table 2 reports the relative prevalence of various sectors— Health 386 (11.57%) health (45.2%)
development priorities or fields in which organizations Gender/women 306 (9.17%) women (83.2%)
Conflict 301 (9.02%) humanitarian (24.2%)
work—studied in the corpus, as defined through a set of keywords.9 People and culture 292 (8.75%) children (24.5%)
We identified these keywords based on common usage by NGO and Environment 289 (8.66%) conservation (25.8%)
other international development practitioners and academics (c.f. Economic development 251 (7.52%) microfinance
Oxfam America, 2018; World Bank, 2018b). Governance—which institution (20.0%)
Education 204 (6.12%) education (75.4%)
includes the keywords governance, corruption, democracy, policy,
Agriculture 197 (5.91%) farm (38.4%)
and human rights—comprised the most common sector (13.7% of Infrastructure 149 (4.47%) urban (46.1%)
all articles), followed by health. These sectors are also the most com- Water, Sanitation, and 102 (3.06%) water (79.5%)
mon sectors in the n = 300 sample. Hygiene (WASH)
There has been variation in sectors studied over time. Fig. 2 *
Cutoff based on proportion of sector-based text that falls one standard devia-
reports the percentage of articles that examine governance or agri- tion over the corpus mean. Percentages do not sum to 100 because an article can be
cultural sectors. Articles on governance grew rapidly after 2000, about multiple sectors, and not all articles are focused on a sector.
compared to a relative plateau of articles on agriculture. By 2014,
governance comprised more than 17% of all articles published; in
35
contrast, agriculture comprised less than 3%, having fallen from a
third of all articles in 1990. 30

25
5.2. Geographic focus is on the most populated countries, with some

Percentage
20
exceptions
15
Table 3 reports the 20 most-prevalent countries in the corpus. 10
We code an article as focused on a given country if the country
5
name appeared at least ten times in the main text. The top two
countries—India (5.8% of articles) and China (4.9%)—are also the 0
most populous developing countries and so their strong represen-
tation in the corpus is expected. The first sub-Saharan African
Governance Agriculture
countries to appear in the list, however, are South Africa, Kenya,
and Uganda, and each has more articles than the most populous Fig. 2. Percentage of articles published on governance and agriculture sectors.
countries in Africa, Nigeria and Ethiopia. Notably, 100 countries
have fewer than five articles mentioning them at least ten times,
and 55 countries have zero articles. Table 3
Limiting this analysis to English-language articles likely biases Top twenty countries represented in corpus.

these results somewhat, but the countries reported in Table 3 still Country Total articles (% of corpus)
reflect the relative geopolitical importance of some countries ver- India 193 (5.79%)
sus others. It is also possible that an article may not be about a China 163 (4.89%)
country if it mentions it ten times, such as articles containing South Africa 129 (3.87%)
cross-national statistical analyses of a large number of countries, Bangladesh 118 (3.54%)
Mexico 78 (2.34%)
but articles of this type generally do not mention any one country
Russia 72 (2.16%)
more than ten times. Based on the n = 300 sample, we know that Israel 67 (2.01%)
some articles are not ‘‘about” any country or countries, or they Kenya 64 (1.92%)
compare many countries. Brazil 63 (1.89%)
Uganda 63 (1.89%)
Nigeria 58 (1.74%)
5.3. Sector-specific research is concentrated in particular geographies Tanzania 52 (1.56%)
Ghana 50 (1.50%)
We also examined whether existing published research maps Pakistan 50 (1.50%)
onto regional development challenges, and conversely, whether Indonesia 49 (1.47%)
Afghanistan 48 (1.44%)
gaps in our knowledge on sectors in particular geographies exist.
Thailand 44 (1.32%)
Table 4 reports how five sectors (governance, health, gender, envi- Cambodia 43 (1.29%)
ronment, and agriculture) cluster with the geographies repre- Philippines 38 (1.14%)
sented in the corpus. We categorized the countries into six Turkey 37 (1.11%)
regions based on World Bank geographic classifications. Then, we Total 1,474 (44.18%)

determined the regional focus by calculating whether any country Note: Number of articles based on at least ten mentions of
country in article text. Articles may feature more than one
9
country or none, so percentages do not sum to 100.
The authors identified 11 sectors and groups of keywords associated with each
sector. (A list of keywords and umbrella categories is available via request.) We
calculated the total word count of all sector-related keywords for each article, and
then standardized it by the article’s total word count. We defined an article as being in the region was mentioned at least ten times in the article. Each
‘‘about” a sector if the proportion of sector-related words fell one standard deviation
or more above the corpus mean. To validate this categorization, we randomly
cell in Table 4 thus represents the number of articles that focus on
sampled ten articles ‘‘in” and ‘‘out” of each sector, and determined that the vast a specific sector (i.e., governance) in a specific region (i.e., sub-
majority were categorized in the proper sector. Saharan Africa).
142 J.N. Brass et al. / World Development 112 (2018) 136–149
Table 4
Sectors and geographies represented in corpus.
Governance (N = 458) Health (N = 386)
Sub-Saharan Africa (N = 909) 118 175
East Asia (N = 588) 83 62
Latin America (N = 533) 58 45
Middle East & North Africa (N = 220) 43 6 24
South Asia (N = 477) 36 63
Central Asia (N = 331) 88 15
*Sector focus is based on the proportion of sector-based text that falls one standard deviation over the corpus mean. Region focus determined by ten or more mentions of a
country included in the region. Columns and rows do not sum to the total for each region or sector because an article can be about multiple sectors and/or regions (or none).
Several patterns emerge. First, countries in sub-Saharan Africa
are represented the most across all sectors except the environment
(which co-occurs most frequently with East Asia). Within the
sub-Saharan African subset, health is the most common sector;
sub-Saharan African countries are also commonly associated with
articles on HIV/AIDS. Despite comprising only 30% of all articles
with a regional focus, sub-Saharan Africa encompasses more than
45% of all health-related articles. The largest sector across East
Asia, the Middle East and North Africa, and Central Asia is gover-
nance. For example, Central Asia, which includes some Eastern
European countries, is positively associated with articles on cor-
ruption, presumably capturing work on democratic transitions in
the former Soviet Union (Zaloznaya, 2017). South Asia is also asso-
ciated with mentions of gender, perhaps due to the female-focused
microcredit movement that originated with the Grameen Bank in
Bangladesh, and was later associated with BRAC in Bangladesh
and self-help groups in India.10
5.4. The literature addresses six key research questions
What questions do scholars of NGOs seek to answer in their
research? Table 5 provides a set of six over-arching research ques-
tions that characterize most of the published literature, with exam-
ples of more specific research questions asked in individual papers.
We use the n = 300 sample to create this set of questions; as part of
the close reading and qualitative coding, we summarized each arti-
cle, identified authors’ research question(s),11 and coded each arti-
cle for related analytic topics, such as what NGOs are doing (service
provision, advocacy, etc.) and who their relationships are with
(states, donors, etc.). While doing so, we inductively translated par-
ticular questions (e.g., what effect did a particular NGO program in
Malawi have on infant mortality?) to a set of broad questions that
subsume the particular ones (e.g., what effect, if any, do NGOs have
on specific development outcomes?). Here, although some authors
take clear normative stances—often embedding critique or praise
of NGOs in their research question—we have tried to maintain a neu-
tral, social scientific articulation for the over-arching questions.
Table 5 thus presents a stylized portrait of the major research
questions posed in published articles about NGOs. The first two
questions (What is the nature of NGOs? and What factors lead to
the emergence, development or evolution of NGOs or the NGO sec-
tor?) examine foundational issues about the nature of NGOs and
the NGO sector. As a result of articles asking these two questions,
the definition and conceptualization of ‘‘NGOs” and related terms,
including descriptions of the contours of the NGO sector globally
and in particular places, has grown more precise over time—re-
10
In other analyses not reported but available upon request, we conducted
statistical analyses using region-specific word counts to predict the word counts of
specific sub-sector topics. Results from these regression analyses (as well as count
models) reveal similar patterns: African countries are positively associated with
mentions of HIV; South Asia is positively associated with mentions of microcredit;
Central Asia is positively associated with mentions of corruption; and Latin America is
associated with mentions of participation.
11
Authors did not always state their question in the form of a question.
Gender (N = 306) Environment (N = 289) Agriculture (N = 197)
77 71 91
47 72 23
53 70 38
6 1
73 22 22
31 18 8
flected in the definition of NGOs used in this paper. Scholars
explain the sector’s rise and evolution as a result of weak states
in developing countries, a donor ideology of liberalization and civil
society promotion, and growing pressure for services and democ-
racy from within countries.
Questions 3 (How do NGOs carry out their work?) and 4 (What
effect (if any) do NGOs have on specific development outcomes?)
then examine how NGOs complete their activities—looking at orga-
nizational processes, structure, management, mission, and devel-
opment theory—and to what effect. These articles often assess
why NGO efforts fail or succeed in reaching an objective of interest,
and tend to highlight the importance of organizational autonomy,
truly participatory development, and successful partnerships with
other actors as reasons for success, rather than technical con-
straints. Authors show that ‘‘good” NGO practices are those that
align with local norms, establish trust, or encourage involvement
in development projects. As the next section of the paper illus-
trates, articles asking about NGO effects also tend to report favor-
able effects of NGOs on target populations, suggesting that NGOs
on balance do more good than harm.
Question 5 (How do NGOs interact with other actors in their
environments?) explores the dynamics of NGO relationships with
other actors, sometimes as a descriptive question, but other times
associated with development outcomes. Within the sample, 23% of
articles feature NGOs’ relationships with donors; 33% analyze
NGOs’ relationships with government; 30% examine NGOs’ rela-
tionships with fellow aid actors; and 29% look at how NGOs inter-
act with citizens of the countries where they work.
As resource dependence theory predicts, many articles analyze
the effects of donors on NGOs. These articles generally confirm that
donors influence program choices of individual organizations as
well as the contours of the NGO sector at the national level, and
most depict NGOs’ fears or complaints about donors. Articles dis-
cussing relationships between NGOs and developing country gov-
ernments vary somewhat depending on the type of NGO and the
country features. Most often, when NGOs provide services, they
ally with the government on implementation. But there is also
sometimes tension between NGOs and government—especially in
relief, conflict, or post-conflict settings, but also in non-
humanitarian zone settings, as when NGOs struggle in vain for gov-
ernment support of new service areas (e.g., disability) or new
methods of provision. Nevertheless, even when NGOs act in more
adversarial civil society roles, they tend to advocate for particular
policies or engage in soft methods of enforcing accountability
(Routley, 2012), rather than directly confronting the government.
Papers analyzing NGO-government relationships are often evalua-
tive in nature, and consider the consequences of NGO-government
interactions for a particular policy or development outcome. But
for roughly every three papers that evaluate the consequences of
these relationships, two papers neutrally describe NGOs’ relation-
ships with other actors.
Articles that depict NGOs interacting with fellow NGOs or other
aid actors are most often case studies of collaboration for broad or
sector-wide programs; advocacy coalitions; or the influence of glo-
 J.N. Brass et al. / World Development 112 (2018) 136–149 143

Table 5
Research Questions Addressed in Studies of NGOs.*

Major research question Example questions asked by authors

1. What is the nature of NGOs?
2. What factors lead to the emergence,
development, or evolution of NGOs or
the NGO sector?
3. How do NGOs carry out their work?
4. What effect (if any) do NGOs have on
specific development outcomes?
5. How do NGOs interact with other actors
in their environments?
6. In what ways do NGOs contribute to the
production or reproduction of cultural
categories or power dynamics?
*
Our definition of NGO is expansive and includes civil society organizations, faith-based organizations, and community-based organizations, among others (see Appendix).
 How should we define, conceptualize, or typologize NGOs?
 How are NGOs different than states, certain international actors, or other forms of social organizing? Under what con-
ditions do NGOs take on the characteristics or functions of these other forms?
 How have legal or regulatory environments shaped the emergence of the NGO sector?
 What caused the proliferation of specific types of NGOs, such as fair trade organizations, women’s CBOs, or environ-
mental NGOs?
 How has the NGO sector evolved in a specific place or development sector?
 Why do we see more robust civil society in some places versus others?
 How does organizational change occur within an NGO or the NGO sector?
 What are the mechanisms by which NGOs implement activities? e.g. How do NGOs (try to) implement participatory
development?; What do NGOs do to (try to) increase the political voice of particular actors?
 How do internal characteristics of organizations shape NGO work? e.g. How does organizational design shape the pro-
cesses NGOs use?
 How can NGO programs be improved?
 What is the range of development theories or models that NGOs embrace and employ?
 What are the underlying principles on which NGOs base their work? e.g. How do changing global norms influence how
NGOs do their work?
 What effect does service provision by NGOs have on outcomes?
 What effect does NGO activism have on development outcomes? e.g. What effect do NGO lobbying efforts have on pol-
icy uptake?
 Do NGOs contribute to democratization, broadly conceived? Under what condition, if any, does participatory develop-
ment by NGOs increase local level democracy, in terms of empowerment, accountability, transparency or voice?
 What are reasons that NGOs fail to achieve outcomes?
 How do national laws or regulations influence NGOs?
 Under what conditions do NGOs and governments collaborate or conflict?
 How do NGOs get involved in national or local government policymaking?
 How do global actors affect local NGOs? e.g. How do local NGOs respond to international NGO networks?
 How have networks of NGOs gained traction on the international stage/in international diplomacy or negotiations?
 How do donor priorities and NGO knowledge on the ground inform one another?
 How are concepts such as development, participation, gender, or sustainability understood in NGO programs, by NGO
workers and/or those affected by such categories?
 How might NGOs’ ontological assumptions undermine their goals for human capability and self-determination?
 In what ways do NGOs reinforce existing power inequalities, either globally or locally?

bal or northern NGOs or movements on NGOs in the global South.
These papers find that effective conditions for NGO-NGO collabora-
tion include shared objectives, lack of intense competition for
resources, and relatively open information sharing.
Finally, articles that address Question 6 (In what ways do NGOs
contribute to the production or reproduction of cultural categories
or power dynamics?) reflect critically on cultural categories and
power dynamics. These articles demonstrates the ways that NGOs
reinforce or reify existing power imbalances at the local, national
or international level, and in that way, somewhat mitigate the find-
ings about positive effects of NGOs on target populations.12 Fre-
quently NGOs fail to understand and fit programs to local cultural
contexts to deliver development outcomes, but other times NGOs
successfully navigate these contexts, or establish relationships with
local elites to get ‘‘buy-in.” In so doing, however, NGOs can repro-
duce power asymmetries.
6. Findings III: NGOs have largely favorable effects: a synthesis
of the most common question in the literature using articles on
the most common sectors
Following on this summary of the six key areas of NGO research
inquiry, here we rigorously and systematically synthesize answers
to the research question most frequently addressed—what effect
do NGOs have on development outcomes? In so doing, we demon-
strate a model for how to carry out such syntheses of published
conclusions. And we call for others to use the NGO Knowledge
Collective database of published research to systematically review
the answers to the remaining overarching questions. We do not
seek to assess NGOs’ effectiveness or their relative efficiency in
implementing activities; we specifically review and summarize
scholarly publications’ findings on the effects of NGOs on specific
development outcomes.13 We find that published studies usually
report favorable effects of NGOs on target communities.
In the 300-article sample, NGOs are depicted providing services
(104 articles) or engaging civically (105 articles), but rarely doing
both (17 articles). Specifically, in roughly one-third of articles in
our sample, NGOs provide some sort of service—most often health
care, education, or water—in lieu of or in partnership with govern-
ment. Service provision by NGOs is an area where academic litera-
ture is dominated by a handful of country cases: of the 86 articles
in our sample that analyzed service provision in specific countries,
34 articles referred to Uganda, South Africa, India, or Bangladesh.
In another one-third of articles in our sample, NGOs play civil
society roles: engaging directly in advocacy, or mobilizing citizens
to improve accountability, empowerment or government perfor-
mance in society. Forty percent of civil society articles feature
direct engagement with government, including NGOs advocating
for policy. Participatory development is a major theme, and much
research is devoted to the ways that NGOs increase the political
voice of marginalized people, or fail to do so. The literature depicts
tensions between the goals of global activists and the preferences
of local actors, and between the unfettered ability of grassroots
organizations to speak and the professionalizing tendencies of
NGOs.

12 13
NGOs can have favorable effects on their target populations while simultaneously Discussing relative efficiency would have been particularly difficult, as there are
reinforcing existing inequalities, as when their programs fail to assist the worst-off in very few articles that explicitly compare NGOs to other, comparable actors (such as
target communities. the government).
144 J.N. Brass et al. / World Development 112 (2018) 136–149
Table 6
Reported effects of NGOs on health and governance outcomes, from n = 300 sample.
Number of articles about sector
Articles showing an effect of NGOs in sector
Articles with clearly measured indicator
Articles showing favorable effect of NGOs
Articles showing unfavorable effect of NGOs
Articles showing mixed effect of NGOs
Articles showing null effect of NGOs
*
Except where noted, percentages indicate the percent of articles that report an effect of NGOs in the sector.
We address the findings on this most commonly asked research
question for the two most prevalent sectors in the literature, health
and governance. These sectors demonstrate NGOs’ role in service
provision and civil society, respectively. For the 113 articles in
the n = 300 sample that discuss the impact of NGOs on some health
or governance outcome, we summarize authors’ statements on
whether NGOs have favorable, unfavorable, mixed, or null effects
on a group of people or on society.14 Here, some authors address
the effects of particular programs on specific communities, while
others make comments about effects of NGOs at a more systematic
level.
To synthesize these findings, we double-coded each article (to
address concerns of inter-coder reliability). We identified whether
the authors reported that NGOs had an effect on some outcome
(see Table 6). Then we identified and described the reported rela-
tionship between NGO action and the outcome in three to four sen-
tences. Identifying an effect of NGOs was not always the primary
goal of the author, but our close reading of articles revealed very
clear statements of NGO effects. For articles in which authors dis-
cussed NGOs’ effects, we also identified whether the outcome was
clearly measured with an indicator (e.g., infant mortality rates or
voter turnout). Notably, however, many articles attributed out-
comes to the efforts of NGOs without a clear indicator, and with
varying levels of evidence used to support their claims. Analysis
of the article data on NGO effects was thus complicated by the
infrequent measurement of defined indicators, and the predomi-
nance of qualitative, historical or case study data to make claims.15
We analyze these issues more in our Discussion.
6.1. Reported effects of NGOs on health sector outcomes
Health is the leading service-provision sector in the NGO liter-
ature. Within the corpus as a whole, 11.5% of articles contain
above-average numbers of words about health (see Table 2). Yet
trends in published topics within health articles have ebbed and
flowed over time. Fig. 3 reports the annual frequencies (per million
words) and overtime trends of several health terms in the corpus.
The most obvious trend is the sharp rise in frequency of HIV men-
tions since the early 2000s (in addition to a peak in the mid-1990s),
which coincides with the massive increase in donor funding for
HIV that occurred at the same time through the President’s Emer-
gency Plan for AIDS Relief (PEPFAR) and the Global Fund to Fight
AIDS, Tuberculosis and Malaria. Other health sectors show more
stability and, in the case of family planning, even decline over time,
reflecting how HIV detracted attention from family planning both
globally and within countries (Robinson, 2017).
Turning to our sample, 54 articles, or 18%, are about health, and
80% of the health articles contain some argument about the effect
14
Three of the 113 articles discussed both health and governance outcomes for a
total of 116 assessed outcomes.
15
Here, we specify various types of data to be clear that we consider qualitative and
historical data to be data; we are not only looking for mathematical or statistical data.
Health Governance
54 62
43 (80% of health articles) 44 (71% of governance articles)
26 (60%)* 7 (16%)
27 (63%) 23 (52%)
2 (5%) 1 (2%)
11 (26%) 19 (43%)
3 (7%) 2 (5%)
of NGOs on a health-related outcome. Twenty-six of the 43 articles
that report an effect, or 60%, do so with a specific, measurable indi-
cator, which is possible because the health field offers a number of
relatively easily conceptualized and measurable outcomes. Pub-
lished results on health outcomes are overwhelmingly positive:
out of 43 articles reporting any effect, 27 report a favorable effect
of NGOs, 11 report a mixed effect, two report a negative effect,
and three report null effects.
Close examination of articles that measure the effect of NGOs
on some health outcome shows that favorable outcomes are most
commonly defined as improvements in health behaviors or atti-
tudes in a limited population, and over a relatively short duration.
For example, three articles reported increased contraceptive use by
rural women in selected regions of predominantly Muslim coun-
tries (Amin & Li, 1996; Huber, Saeedi, & Samadi, 2010; Khurram
Azmat et al., 2013). Six of the articles reporting favorable outcomes
addressed measures related to HIV or other sexually transmitted
infections, with two reporting improvements in service access
(Hershey, 2014; Kokku, Mahapatra, Tucker, Saggurti, & Prabhakar,
2014), two showing increased knowledge about HIV generally
and HIV status specifically (Dalrymple, 2006; Loukid et al., 2014),
one looking at reductions in country-level HIV rates (Shircliff &
Shandra, 2011), and one analyzing NGO workers’ perceptions of
stigma reduction (Agbonyitor, 2009). An additional three favorable
outcomes related to increases in vaccine uptake due to NGOs
(Amin & Li, 1996; Berti, Mildon, Siekmans, Main, & MacDonald,
2010; Ladner, Besson, Rodrigues, Audureau, & Saba, 2014). Among
other articles showing favorable outcomes, outcome measures
included cases of diseases detected and/or treated (e.g., tuberculo-
sis, sleeping sickness, cataracts, and cholera), health care provision
(e.g., clinics built), and more macro outcomes (e.g., political pro-
cesses, legal reform).
An example of an article reporting a negative effect of an NGO
(Kwiatkowski, 2005) analyzed how competing global and local dis-
courses led to unintended negative effects of the NGO on the com-
munity it intended to serve, reinforcing Western healthcare
practices and denigrating indigenous ones. The articles showing
null or mixed effects usually have difficult-to-measure dependent
variables, such as decentralization of HIV services at the municipal
level (Le Loup et al., 2009), priority setting within global health ini-
tiatives (Cohn et al., 2011), and perceived collective efficacy among
sex workers (Chakravarthy, Joseph, Pelto, & Kovvali, 2012).
Eleven health articles did not report any effect of NGOs on a
particular outcome. Authors of these articles instead looked pri-
marily at processes internal to NGOs or relationships of NGOs with
other organizations, reflecting Questions 3 (How do NGOs carry
out their work?) and 5 (How do NGOs interact with other actors
in their environments?) from Table 5. Examples of the former
include an analysis of women’s experiences as health promotion
volunteers with NGOs in Peru (Jenkins, 2009) and a calculation of
cost recovery within an NGO medical facility in Bangladesh
(Alam & Ahmed, 2010); examples of the latter include analyses
of public–private partnerships between NGOs and corporations
 J.N. Brass et al. / World Development 112 (2018) 136–149 145

1000
900
800

Frequencyy per million words

700
600
500
400
300
200
100
0
1980 1985 1990 1995 2000 2005 2010

HIV Maternal mortality Malaria

Family planning Reproducve health

Fig. 3. Mentions of five health terms in the corpus.

to address food insecurity (Kraak et al., 2012) and NGOs’ positions centage than articles about health. Yet far fewer authors identify
within aid chains and other transnational AIDS networks (Barnes, and measure a clear indicator: only 16% of governance articles, ver-
2008). sus 63% in health. Clearly measured indicators include the propor-
tion of electoral vote share for left-leaning parties (Brown, Brown,
6.2. Reported effects of NGOs on governance sector outcomes & Desposato, 2002), country-level human rights performance (Bell,
Murdie, Blocksome, & Brown, 2013), or whether countries adopted
Fig. 4 depicts how the focus of people writing about governance specific policies or types of policies (Ringius, 1997; Welch, 2003).
has shifted over time. Specifically, it shows the annual frequencies Within the 44 articles that reported an effect of NGOs (with a
(per million words) of six governance terms in the corpus. Men- measured indicator or not), slightly more than half reported a
tions of all terms together increased over time, mirroring a focus favorable effect of NGO activity, and provided at least some evi-
on these governance issues in the global system, particularly in dence. The favorable effect percentage in governance was similar
the 1990s after the fall of the Soviet Union (the 1993 spike in to, but slightly lower than, in health (60%), and as in health, almost
‘‘democracy” is likely directly related). But research on NGOs in no articles reported either a null or unfavorable effect on gover-
development is more focused on human rights and democracy nance outcomes. A representative article that reported a favorable
than the other four terms. This is not surprising, as democracy effect of NGOs on governance argued that NGO efforts persuaded
and human rights are more institutionalized as fields of academic the Pakistani government to better acknowledge and enforce the
study, and the remaining terms could be considered subsets of Convention on the Elimination of All Forms of Discrimination
them. Against Women (Afsharipour, 1999).
Unlike research on NGOs in the health sector, studies of NGOs More than health articles, however, authors of governance arti-
and governance rarely measure outcomes, although authors do fre- cles were likely to present mixed results of NGO programming—
quently identify an outcome and attribute it to NGOs in some way. 43% of the governance articles in the sample that report any effect
Slightly more than 20% of articles in the sample, or 62 of the 300, reported mixed effects, versus 26% in health. These mixed findings
are about governance. Of those articles, roughly 70% of authors simultaneously reflect a clear attempt to report nuances in NGO
associate NGOs with a governance outcome, a slightly lower per- effects, and are the necessary result of writing on a sector that gen-
erally lacks easily conceptualized and measurable outcome indica-
tors.16 One article reporting mixed effects of NGOs, for example,
showed that human rights NGOs have increased awareness of
1600
human rights issues in African countries, but have not successfully
1400
Frequency per million words

halted the decline of the rule of law, nor achieved prosecutions of

1200
human rights abuses in regimes undisposed to such cases (Welch,
1000 2003). Another article examined whether participation in transna-
800 tional networks leads to civil society empowerment and sustainabil-
600 ity for AIDS CBOs in Mexico, and found that paradoxically,
400 participation leads to increasing resources going to some organiza-
200 tions at the expense of others, thus reproducing, in some cases,
0
power asymmetries among local organizations (Barnes, 2008).17

16
Indeed, even the term ‘‘governance” itself is notoriously difficult to conceptualize
Corrupon Accountability Transparency and measure (Rhodes, 2000).
17
Empowerment Democracy Human Rights Note that this article is an example of an article in the sample coded as being
about more than one sector – in this case, both health and governance. For the health
Fig. 4. Mentions of six governance terms in the corpus. sector, this article did not count as analyzing an effect of NGOs.
146 J.N. Brass et al. / World Development 112 (2018) 136–149
The governance sector encompasses a broad range of activities.
Most of the governance articles considered NGOs in relation to
democracy, meaning that they focus on one or more of the follow-
ing: accountability, empowerment, corruption, representation,
political rights, civil liberties, regime type, or the development of
the rule of law. Such a focus on democracy may reflect or stem
from the World Bank and other donors’ interest in NGOs as agents
of civil society. For example, Kanyinga (2011) described the medi-
ation role that CSOs played in stopping post-election violence in
Kenya in 2008 and in democracy promotion in the country gener-
ally. Roughly a quarter of articles relate to NGOs’ involvement in
making or influencing policy. For example, Mundy, Haggerty,
Sivasubramaniam, Cherry, and Maclure (2010) discussed the role
of CSOs in education policymaking in four African countries. And
a sizable number of articles discuss NGO activities related to
human rights or humanitarian relief. Other governance issues
addressed in these articles include nation-building, state-
building, sovereignty, or diplomatic processes. Many articles
addressed more than one of these sub-topics, such as the article
on human rights and the rule of law in African countries men-
tioned above (Welch, 2003).
Articles pertaining to governance for which NGOs were not an
independent variable discussed a range of factors, such as why so
many NGOs work in humanitarian relief (Borton, 1993), state reg-
ulation or interaction with NGOs (Moran, 2006), or inter-NGO col-
laboration (Saab, Tapia, Maitland, Maldonado, & Tchouakeu, 2013).
Many articles relating to governance examined NGO relationships
with other actors.
7. Discussion and recommendations for the future of NGO
research
We structured our systematic review of the NGO literature
around four broad questions. First, who studies NGOs, and how
do they study them? Second, what is studied when NGOs are an
object of analysis? Third, what effect, if any, do NGO activities have
on specific development outcomes? This final section of the paper
summarizes our answers to these questions, discusses some impli-
cations, and answers our fourth question—what is the path for-
ward for NGO research?
Based on computer-assisted content analysis of 3336 corpus
articles and mixed-method analysis of 300 randomly sampled arti-
cles, we are able to describe quite clearly the contours of published
research on NGOs. We find that most research is published in inter-
disciplinary journals that either explicitly address development
(e.g., Development in Practice, World Development, and Public
Administration and Development) or the broader nonprofit sector
(Voluntas and Nonprofit & Voluntary Sector Quarterly). Thus, while
producers of NGO-related scholarship may have disciplinary train-
ing and/or disciplinary homes, their audience comes from across
the academic and practitioner communities. We also find an
increasing quantification of the NGO literature, which, although
still reliant on case studies and qualitative methods, has increas-
ingly used quantitative data and advanced statistical approaches
over time.
Most articles on NGOs address one or more of a set of six
research questions that consider the nature of NGOs; the factors
that lead to their emergence and development; how they carry
out their work; their impacts; how they relate to other actors in
their environments; and how they contribute to the production
or reproduction of cultural categories and power dynamics. We
briefly summarized findings to each of these, and then provided
a rigorous synthesis of findings for one of these questions—what
effect do NGOs have on specific development outcomes?—as it per-
tains to the articles in our sample about health or governance. We
find that articles that address outcomes associated with NGOs’ pro-
grams and activities overwhelmingly report positive results (as
opposed to mixed, null, or negative results). Articles in the health
sector associated NGOs with changing certain types of health
behaviors, especially related to contraceptive use, specific diseases,
and vaccine use. Fewer articles tackled NGOs’ ability to shape
broader population health and to improve public health and health
care systems, perhaps because those questions are harder to eval-
uate and require longer time frames. Within governance studies,
articles reported positive effects of NGOs on accountability, citizen
representation, policy making, and human rights.
We note reasons for caution in interpreting this synthesis to
mean that NGOs always have positive effects on development out-
comes. Substantively, our approach narrowly addresses published
findings related to singular development outcomes, health or gov-
ernance, with the goal of aggregating reports of favorable, mixed,
or unfavorable outcomes. This approach necessarily required sim-
plification, but real-world dynamics are complex. Future research
on this question might be structured differently to better capture
the complexity, for example, by creating an index of whether
NGO programs are reported to be efficient, effective, and participa-
tory, and if so, under what conditions and in what sectors.
Methodologically, moreover, it has been well documented that
there exists bias against publishing null results, particularly in the
social sciences (Franco, Malhotra, & Simonovits, 2014). We suspect
that this bias influences the articles from the health sector in par-
ticular, since more health than governance articles presented mea-
surable indicators. The overwhelming reporting of positive results,
however, has serious implications for the study of NGOs and the
design of health programs. Specifically, it makes it challenging to
draw firm conclusions given that we know little about interven-
tions that had no effect. Furthermore, NGO studies often have no
counterfactual or control group, and alternate causal paths to the
outcome are not considered at length, particularly in the gover-
nance sector. Because of these issues, it remains unknown whether
similar outcomes could have been achieved through alternative
pathways or would have happened even in the absence of the
NGO. In particular, NGOs are often credited for improvements
related to democracy or empowerment that may have occurred
whether or not the NGO was present.
7.1. Moving NGO research forward
Despite 35 years of research and over 3000 academic articles,
there is considerable work to be done in at least four areas. First,
we call for NGO research to address a number of geographic and
sectoral gaps. To date, research has tended to go ‘‘where the action
is;” that is, countries with geopolitical importance, large popula-
tions, and, presumably, the most NGOs. As a result, we know very
little about NGOs that may be operating in some regions or coun-
tries. Fifty-five countries did not have a single article that men-
tioned them ten times or more.18 Many of these countries are
small island nations vulnerable to climate change, such as the Mal-
dives, yet we know very little about NGOs that may be working on
this crucial issue in those locations. Others are countries with violent
conflict—Yemen and the Central African Republic each had fewer
than five articles mentioning them at least ten times. Although we
18
These countries/regions include: American Samoa, Andorra, Antiqua and Barbuda,
Aruba, Bahamas, Barbados, Bermuda, Brunei, Cayman Islands, Central African
Republic, Comoros, Curacao, Djibouti, Dominica, Equatorial Guinea, Faeroe Islands,
French Polynesia, Gabon, Greenland, Grenada, Guyana, Isle of Man, Kiribati, Kyrgyzs-
tan, Macao, Maldives, Marshall Islands, Mauritania, Micronesia, Monaco, Montenegro,
New Caledonia, Northern Mariana Islands, Oman, San Marino, Sao Tome and Principe,
Seychelles, Somaliland, St. Kitts and Nevis, St. Lucia, St. Martin, St. Vincent and the
Grenadines, Suriname, Timor-Leste, Togo, Trinidad and Tobago, Turks and Caicos,
Tuvalu, United Arab Emirates, Virgin Islands, and Western Sahara.
 J.N. Brass et al. / World Development 112 (2018) 136–149
appreciate that these are places where researchers may be reluctant
to work, the implication is that scholarly conclusions about NGOs in
conflict areas may not actually apply outside of well-studied con-
flicts, such as Afghanistan and Israel-Palestine.
Likewise, articles on microfinance disproportionately address
South Asian countries, despite the fact that MIX (2017) reports that
only 16% of microfinance institutions are in South Asia (compared
to 25% in Africa and 30% in Latin America and the Caribbean). This
clustering suggests that much of what we know about NGOs and
microcredit, for example, is also specific to particular contexts,
leaving significant blind spots. The agriculture sector also appears
under-represented in general. Although nearly half of South Asians
(44%) and more than half of Africans (55%) work in agriculture
(World Bank, 2018c), less than six percent of our corpus focused
on that sector. These gaps in geography and sector suggest several
paths forward for future research. Certainly there is benefit to
studying underrepresented intersections of geography and sector.
But scholars working in well-trod areas can also make their work
more useful by giving deliberate attention to the contextual vari-
ables that affect their findings. Understanding the conditions under
which various types of findings occur is both a good of itself, and it
also facilitates scholars working to extend research to under-
studied locations or sectors.
Second, we encourage efforts to bring into scholarly conversa-
tions more researchers based in the global South and more devel-
opment workers. Most of the published academic literature on
NGOs in international development originates from scholars work-
ing in countries in the global North. We support efforts to aid
southern scholars in publishing in peer-reviewed journals and to
bring scholarly eyes to the places where these southern scholars
are already working. One such effort would be for Northern schol-
ars, particularly established ones, to purposively seek out collabo-
rations with scholars based in the countries they study. Another
means to increase southern voices would be to increase the mean-
ingful involvement of NGO employees in high quality research on
NGOs. We recognize that professional incentives for those in the
academy and those working in NGOs differ, and that NGO-based
author bias is possible. Yet research leaves for NGO employees
(once a common feature of large NGOs) would do a good deal to
bring NGOs’ own knowledge to academic researchers. Such collab-
orations would likely also improve scholarly access to data, such as
NGOs’ internal process and impact evaluation data and findings.
With this sort of data, academics could more meaningfully con-
tribute to understanding of what does and does not work in devel-
opment interventions.
Third, we call for research designs that allow findings to be bet-
ter compared and synthesized. We began this project over three
years ago with the goal of finding new kinds of data on NGOs.
We hoped that a large-scale systematic review would allow us to
speak, for example, to the question of NGO effectiveness in service
delivery. But we soon learned that even the latest bibliographic
software and huge amounts of human labor could not synthesize
articles where conclusions about NGO effects were often offered
without clear outcomes or indicators, and usually without system-
atic data collection or a counterfactual. Our ‘‘deep dive” into a sin-
gle research question in the health and governance sectors
provides one possible model for additional focused synthetic
research. However, simply put, the quality of research on the
effects of NGOs on development outcomes must improve. Compar-
ison and synthesis will also become easier if scholars interested in
the effects of NGOs build in the question ‘‘compared to what?”
Explicit comparisons between NGOs and other actors with similar
goals (e.g., states in service provision; religious groups or social
movements in advocacy) are especially needed, as are studies of
the conditions under which outcomes of interest do or do not
occur.
147
The six broad research questions outlined in this review, and
the motivations for this review itself, show that NGOs operate in
diverse contexts and are a ‘‘case of” many things. But because
the role and success of NGOs is a question with high stakes for
development, we urge scholars in this field not to lose the forest
for the trees. It is too simple for a scholar working on one major
NGO research question to negate the others, even when those
other questions are critical to the organizations or populations
they are analyzing. We urge scholars to pay more attention to
the borders of their research; to be explicit about the questions
that are being asked and which are being left unexamined. We
emphasize this in light of the high level of specialization within
the literature and the fragmentation in publication outlets, which
make it difficult to identify or develop general theories of NGOs
or move to general statements on development practice. We hope
that our six broad research questions will be an orienting device to
mitigate this issue, and that additional, focused systematic reviews
can address these questions.
Finally, scholars can aid comparison and synthesis with new
ways of sharing data and research. Our own contribution to this
effort is in progress: the NGO Knowledge Collective will be a plat-
form that catalogues all 3336 articles referenced in this paper, and
will be updated periodically. Unlike library-based bibliographic
systems, this database will allow users to sort articles using topics,
geographies, and sectors assigned by topic models and keyword
frequencies. This tool will help increase the odds that those design-
ing research on NGOs have access to as much of the relevant liter-
ature as possible, thus allowing them to carry out projects that
truly fill gaps in knowledge.
8. Conclusion
The findings above are the broadest conclusions generated from
an analysis that systematically assesses 35 years of published arti-
cles on NGOs. As expected, scholars draw from a wide range of
fields to cover the activities of NGOs in the largest countries and
the most prioritized development sectors. Furthermore, we know
what we know because of very particular sets of methodologies
and analytic approaches, namely qualitative historical and case
study approaches, which are becoming increasingly refined over
time. And, while not conclusive, our findings suggest that NGOs
are likely to be associated with positive intended outcomes.
So where do we go from here? Our attempt to systematically
review a mammoth corpus of academic literature yields insights
that, to some, may come across as obvious. Still, we contend that
in revealing the obvious, new research directions reveal them-
selves. Indeed, we know much more about particular sectors (gov-
ernance, health, gender) and particular places (India, Bangladesh)
than we do about others. Thus, this review (and the broader NGO
Knowledge Collective project) can serve as an intellectual map of
sorts to help scholars identify new geographies, topics, and sectors
to investigate. Much of what is unknown lies at the unexplored
intersections of these axes.
We also think there is a larger implication of this project. Devel-
opment scholars are quick to point out that development interven-
tions rarely scale well from one context to another. We should
assume the same about the role of NGOs in development given that
our own NGO scholarship clusters in particular research questions,
development sectors, geographies, and methodologies at the
expense of others. This is incredibly important for scholars inter-
ested in examining whether NGOs truly advance development.
We call for carefully crafted future research that more systemati-
cally develops an understanding of the conditions under which
these organizations can be effective instruments for improving
economic, political and social development outcomes.
148 J.N. Brass et al. / World Development 112 (2018) 136–149
Acknowledgements
We gratefully acknowledge financial support from Indiana
University (School of Public and Environmental Affairs, Lilly Family
School of Philanthropy, and Social Science Research Commons);
Emory University (Goizueta Business School); and American
University (Center on Health, Risk, and Society and School of Inter-
national Service). A team of research assistants also provided valu-
able work, particularly Anthony DeMattee, and also Briana Bardos,
Emily Besinger, Alix Collins, Andrew Dam, Julia Fischer-Mackey,
Andrew Guenther, Sher Khashimov, Georgia Kossoff, Diana Lara,
Ariana Leonelli, Alana McGinty, Clayton Morrison, Lina Ochoa,
Aimee Seligstein, Andrew Shermeyer, and Ellie Symes. Likewise
Ron Harris and Christina Sheley provided invaluable research
support. Thanks are also due to the students of the Spring 2016
American University course ‘‘NGOs in International Affairs.” We are
also grateful for feedback from presentations at the International
Society for Third-Sector Research, the Indiana University Lilly
School of Philanthropy, the West Coast Nonprofit Data Conference,
the African Studies Association, the American Sociological Association,
and the World Society Mini-Conference. All errors are our own.
Declaration of interest statement
The authors of this paper have no known conflicts of interest
that could inappropriately bias the research presented in this
paper.
Appendix A. Supplementary data
Supplementary data associated with this article can be found, in
the online version, at https://doi.org/10.1016/j.worlddev.2018.07.
016.
References
Afsharipour, A. (1999). Empowering ourselves: The role of women’s NGOs in the
enforcement of the women’s convention. Columbia Law Review, 99(1), 129.
Agbonyitor, M. (2009). Home-based care for people living with HIV/AIDS in Plateau
State, Nigeria: Findings from qualitative study. Global Public Health, 4(3),
303–312.
Alam, K., & Ahmed, S. (2010). Cost recovery of NGO primary health care facilities: A
case study in Bangladesh. Cost Effectiveness & Resource Allocation, 8, 12–21.
Amin, R., & Li, Y. (1996). Women’s credit programs and family planning in rural
Bangladesh. International Family Planning Perspectives, 22(4), 158.
Banks, N., Hulme, D., & Edwards, M. (2015). NGOs, states, and donors revisited: Still
too close for comfort? World Development, 66, 707–718.
Barnes, N. (2008). Paradoxes and asymmetries of transnational networks: A
comparative case study of Mexico’s community-based AIDS organizations.
Social Science & Medicine, 66(4), 933–944.
Beckfield, J. (2003). Inequality in the world polity: The structure of international
organization. American Sociological Review, 68(3), 401–424.
Bell, S. R., Murdie, A., Blocksome, P., & Brown, K. (2013). Force multipliers:
Conditional effectiveness of military and INGO human security interventions.
Journal of Human Rights, 12(4), 397–422.
Berti, P. R., Mildon, A., Siekmans, K., Main, B., & MacDonald, C. (2010). An adequacy
evaluation of a 10-year, four-country nutrition and health programme.
International Journal of Epidemiology, 39(2), 613–629.
Borton, J. (1993). Recent trends in the international relief system. Disasters, 17(3),
187–201.
Boulding, C. E., & Gibson, C. C. (2009). Supporters or challengers?: The effects of
nongovernmental organizations on local politics in Bolivia. Comparative Political
Studies, 42, 479–500.
Brass, J. N. (2012). Why do NGOs go where they go? evidence from Kenya. World
Development, 40(2), 387–401.
Brass, J. N. (2016). Allies or Adversaries: NGOs and the State in Africa. New York:
Cambridge University Press.
Bratton, M. (1989). The politics of government-NGO relations in Africa. World
Development, 17(4), 569–587.
Brown, D. S., Brown, J. C., & Desposato, S. W. (2002). Left turn on green? The
unintended consequences of international funding for sustainable development
in Brazil. Comparative Political Studies, 35(7), 814.
Campbell, C. (2003). Letting them die: Why HIV/AIDS prevention programmes fail.
Bloomington, IN: Indiana University Press.
Chakravarthy, J. B. R., Joseph, S. V., Pelto, P., & Kovvali, D. (2012). Community
mobilisation programme for female sex workers in coastal Andhra Pradesh,
India: Processes and their effects. Journal of Epidemiology & Community Health,
66, ii78-ii86.
Chege, S. (1999). Donors shift more aid to NGOs: But Cooperation between
governments and NGOs is critical for greater effectiveness. Africa Recovery, 13
(1), 6.
Clarke, G. (1998). Non-governmental organizations (NGOs) and politics in the
developing world. Political Studies, 46(1), 36–52.
Cohn, J., Russell, A., Baker, B., Kayongo, A., Wanjiku, E., & Davis, P. (2011). Using
global health initiatives to strengthen health systems: A civil society
perspective. Global Public Health, 6(7), 687–702.
Dalrymple, L. (2006). Has it made a difference? Understanding and measuring the
impact of applied theatre with young people in the South African context.
Research in Drama Education, 11(2), 201–218.
de Waal, A. (1997). Famine crimes: Politics and the disaster relief industry in Africa.
Bloomington: Indiana University Press.
Dill, B. (2009). The Paradoxes of Community-Based Participation in Dar es Salaam.
Development and Change, 40(4), 717–743.
Dupuy, K., Ron, J., & Prakash, A. (2016). Hands off my regime! Governments’
restrictions on foreign aid to non-governmental organizations in poor and
middle-income countries. World Development, 84(Supplement C), 299–311.
Edwards, M., & Hulme, D. (1996). Too close for comfort? The impact of official aid on
nongovernmental organizations. World Development, 6(961–973).
Fisher, W. F. (1997). Doing good? The politics and antipolitics of NGO practices.
Annual Review of Anthropology, 26, 439–464.
Fowler, A. (1991). The role of NGOs in changing state-society relations: perspectives
from eastern and southern Africa. Development Policy Review, 9, 53–84.
Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social
sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505.
Gauri, V., & Galef, J. (2005). NGOs in Bangladesh: Activities, resources, and
governance. World Development, 33(12), 2045–2065.
Gazley, B., & Guo, C. (2015). What do We Know about Nonprofit Collaboration? A
Comprehensive Systematic Review of the Literature. Paper presented at the
Academy of Management.
Gugerty, M. K., Biscaye, P., & Anderson, C. L. (2018). Delivering Development?
Evidence on Self-Help Groups as Development Intermediaries in South Asia and
Africa. Development Policy Review. https://doi.org/10.1111/dpr.12381.
Hershey, M. (2014). Measuring the success of HIV/AIDS NGOs among Nairobi’s
youth. Development in Practice, 24(1), 51–62.
Huber, D., Saeedi, N., & Samadi, A. K. (2010). Achieving success with family
planning in rural Afghanistan. Bulletin of the World Health Organization, 88(3),
227–231.
Hughes, M. M., Peterson, L., Harrison, J. A., & Paxton, P. (2009). Power and relation in
the world polity: The INGO network country score, 1978–1998. Social Forces, 87
(4), 1711–1742.
Jenkins, K. (2009). ‘‘We have a lot of goodwill, but we still need to eat...”: Valuing
women’s long term voluntarism in community development in Lima.
Voluntas: International Journal of Voluntary & Nonprofit Organizations, 20(1),
15–34.
Kanyinga, K. (2011). Stopping a conflagration: The response of Kenyan civil society
to the post-2007 election violence. Politikon: South African Journal of Political
Studies, 38(1), 85–109.
Kareithi, R. N. M., & Lund, C. (2012). Review of NGO performance research published
in academic journals between 1996 and 2008. South African Journal of Science,
108, 36–44.
Khurram Azmat, S., Tasneem Shaikh, B., Hameed, W., Mustafa, G., Hussain, W.,
Asghar, J., ... Bilgrami, M. (2013). Impact of social franchising on contraceptive
use when complemented by vouchers: A quasi-experimental study in rural
Pakistan. PLoS One, 8(9), 1.
Kokku, S. B., Mahapatra, B., Tucker, S., Saggurti, N., & Prabhakar, P. (2014). Effect of
public-private partnership in treatment of sexually transmitted infections
among female sex workers in Andhra Pradesh, India. Indian Journal of Medical
Research, 139(2), 285–293.
Kraak, V. I., Harrigan, P. B., Lawrence, M., Harrison, P. J., Jackson, M. A., & Swinburn,
B. (2012). Balancing the benefits and risks of public–private partnerships to
address the global double burden of malnutrition. Public Health Nutrition, 15(3),
503–517.
Kwiatkowski, L. (2005). NGOs, power, and contradiction in Ifugao, The Philippines.
Urban Anthropology & Studies of Cultural Systems & World Economic Development,
34(4), 385–436.
Ladner, J., Besson, M.-H., Rodrigues, M., Audureau, E., & Saba, J. (2014). Performance
of 21 HPV vaccination programs implemented in low and middle-income
countries, 2009–2013. BMC Public Health, 14(1), 2175–2195.
Le Loup, G., De Assis, A., Costa-Couto, M.-H., Thoenig, J.-C., Fleury, S., De Carnargo, K.,
Jr, & Larouzé, B. (2009). A public policy approach to local models of HIV/AIDS
control in Brazil. American Journal of Public Health, 99(6), 1108–1115.
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., et al. (2009). The
PRISMA statement for reporting systematic reviews and meta-analyses of
studies that evaluate healthcare interventions: explanation and elaboration.
PLoS Med, 6(7), e1000100.
Longhofer, W., & Schofer, E. (2010). National and global origins of environmental
association. American Sociological Review, 75(4), 505–533.
Loukid, M., Abadie, A., Henry, E., Hilali, M., Fugon, L., Rafif, N., ... Préau, M. (2014).
Factors associated with HIV status disclosure to one’s steady sexual partner in
PLHIV in Morocco. Journal of Community Health, 39(1), 50–59.
 J.N. Brass et al. / World Development 112 (2018) 136–149
McLoughlin, C. (2011). Factors affecting state–non-governmental organisation
relations in service provision: Key themes from the literature. Public
Administration and Development, 31(4), 240–251.
Mercer, C. (2003). Performing partnership: Civil society and the illusions of good
governance in Tanzania. Political Geography, 22(741–763).
MIX. (2017). The MIX market. Retrieved from http://www.themix.org/mixmarket.
Moran, D. (2006). Comparing services: A survey of leading issues in the sectoral
literatures. Public Administration and Development, 26(3), 197–206.
Morfit, N. S. (2011). ‘‘AIDS is Money”: How donor preferences reconfigure local
realities. World Development, 39(1), 64–76.
Mosse, D., & Lewis, D. (2005). The aid effect: Ethnographies of development practice
and neo-liberal reform. London/New York: Pluto.
Mundy, K., Haggerty, M., Sivasubramaniam, M., Cherry, S., & Maclure, R. (2010). Civil
society, basic education, and sector-wide aid: Insights from Sub-Saharan Africa.
Development in Practice, 20(4/5), 484–497.
OECD. (2015). Aid for CSOs. Retrieved from https://www.oecd.org/dac/peer-
reviews/Aid%20for%20CSOs%20in%202013%20_%20Dec%202015.pdf.
Owiti, J., Aluoka, O., & Oloo, A. G. R. (2004). Civil society in the new dispensation:
Prospects and challenges. In D. Okello (Ed.), Civil society in the third republic.
Nairobi: National Council of NGOs.
Oxfam America. (2018). Our Work. Retrieved from https://policy-practice.
oxfamamerica.org/work/.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A
practical guide. Malden, MA: Blackwell.
Pfeiffer, J. (2003). International NGOs and primary health care in Mozambique: The
need for a new model of collaboration. Social Science & Medicine, 56(4),
725–738.
Putnam, R. (1993). Making democracy work: Civic traditions in modern Italy.
Princeton, NJ: Princeton University Press.
Putnam, R. (2000). Bowling alone: The collapse and revival of American community.
New York: Simon & Schuster.
Rhodes, R. A. W. (2000). Governance and public administration. In J. Pierre (Ed.),
Debating governance: Authority, steering, and democracy (pp. 54–90). Oxford:
Oxford University Press.
Ringius, L. (1997). Environmental NGOs and regime change: The case of ocean
dumping of radioactive waste. European Journal of International Relations, 3(1),
61–104.
Robinson, R. S. (2017). Intimate interventions in global health: Family planning and
HIV prevention in sub-saharan Africa. New York: Cambridge University Press.
Routley, L. (2012). NGOs and the formation of the public: Grey practices and
accountability. African Affairs, 111(442), 116–134.
Saab, D., Tapia, A., Maitland, C., Maldonado, E., & Tchouakeu, L.-M. (2013). Inter-
organizational coordination in the wild: Trust building and collaboration
149
among field-level ICT workers in humanitarian relief organizations. Voluntas:
International Journal of Voluntary & Nonprofit Organizations, 24(1), 194–213.
Schnable, A. (2015). New American relief and development organizations:
Voluntarizing global aid. Social Problems, 62(2), 309–329.
Schofer, E., & Longhofer, W. (2011). The Structural Sources of Association. American
Journal of Sociology, 117(2), 539–585.
Schuller, M. (2009). Gluing globalization: NGOs as intermediaries in Haiti. PoLAR
Political and Legal Anthropology Review, 32(1), 84–104.
Schuller, M. (2012). Killing with kindness: Haiti, International Aid, and NGOs. New
Brunswick: Rutgers University Press.
Sen, A. (1999). Development as freedom. New York: Random House Inc.
Shircliff, E. J., & Shandra, J. M. (2011). Non-governmental organizations, democracy,
and HIV prevalence: A cross-national analysis. Sociological Inquiry, 81(2),
143–173.
Sigelman, L. (2006). The coevolution of American political science and the
‘‘American Political Science Review”. The American Political Science Review, 100
(4), 463–478.
Smith, J., & Wiest, D. (2005). The uneven geography of global civil society: National
and global influences on transnational association. Social Forces, 84(2), 621–652.
Union of International Associations. (2018). The Yearbook of International
Organizations. Retrieved from https://uia.org/yearbook.
Watkins, S. C., Swidler, A., & Hannan, T. (2012). Outsourcing social transformation:
Development NGOs as organizations. Annual Review of Sociology, 38, 285–315.
Welch, C. E. Jr., (2003). Human rights NGOs and the rule of law in Africa. Journal of
Human Rights, 2(3), 315–327.
White, S. C. (1996). Depoliticising development: The uses and abuses of
participation. Development in Practice, 6(1).
Willetts, P. (2015). The Growth in the Number of NGOs in Consultative Status with
the Economic and Social Council of the United Nations. Retrieved from http://
www.staff.city.ac.uk/p.willetts/NGOS/NGO-GRPH.HTM#graph.
World Bank (1989). Sub-Saharan Africa: From crisis to sustainable growth.
Washington, D.C.: The World Bank.
World Bank (2018c). World Development Indicators. Washington, DC: World Bank.
World Bank. (2011). Civil Society. Retrieved from http://go.worldbank.org/
19WRCK3AY0.
World Bank. (2018a). Civil Society. Retrieved from http://www.worldbank.org/en/
about/partners/civil-society#2.
World Bank. (2018b). Projects and operations by sector. Retrieved from http://
projects.worldbank.org/sector.
Zaloznaya, M. (2017). The politics of bureaucratic corruption in post-transitional
eastern Europe. New York: Cambridge University Press.