1

The Changing Field of ICTD: Content analysis of research published in selected journals and conferences, 2000-2010
Ricardo Gomez
University of Washington rgomez@uw.edu

Luis F Baron
University of Washington lfbaron@uw.edu

Brittany Fiore-Silfvast
University of Washington fioreb@uw.edu

ABSTRACT
In this study, we report the results of a content analysis of 948 papers from selected peer reviewed journals and conferences published between 2000 and 2010 in the academic literature on the interdisciplinary field of Information and Communication Technologies for Development (ICTD or ICT4D). Results indicate that the majority of the literature focuses on business and empowerment as the primary domains of ICTD work, and on ICT in general and on information systems as the most common technology objects of analysis, with a growing trend toward mobile phones. Furthermore, most of the literature consists of studies of individual countries or of organizations, and the most frequent contributions are field studies and best practices, with a growing trend towards contributing theory and policy recommendations. This is the first-ever comprehensive analysis of the ICTD literature across multiple sources over ten years; it offers important insights about the trends and directions of research in the ICTD field. Further analysis will explore additional dimensions, such as the goals of development, the relationship between ICT and society, and the epistemological stances in the research, in order to offer a better understanding of the changes over time and the differences across the different journals and conferences studied.

Lavelle, Megan Willan, Melissa Gill, Meredith Slota, Michael Carpenter, Monica Caraway, Muzammil Hussain, Mychal Ludwig, Patty Northman, Richard Counsil, Ruixue Xia, Sarah Caldwell, Sarah Ganderup, Sarah Wang, Tess Cradick, Tod Robbins, and Whitney Chamberlin, who helped code the 948 papers used in this study.

1. BACKGROUND
The field of Information and Communication Technologies for Development (ICTD or ICT4D) is the successor of the early uses of print, radio, photography and television as media for community development, what was known in the 1970s and 1980s as Development Communication, or DevCom. But in the 1990s, computers and the Internet made possible a new wave of communication technologies for development and the introduction of community access centers, generically called telecenters. In 2000, the Digital Opportunities Task Force was established at the G8 Summit in Okinawa, Japan to promote the use of ICT as a tool for Digital Opportunities for All. By the time of the World Summit of the Information Society (WSIS) in Geneva (2003) and Tunis (2005), ICT was used to describe computer-related technologies, increasingly with a focus on providing access to infrastructure and strengthening the technical dimension of the ICT. In addition, in the 2005 Summit in Tunis, ICT shifted slightly to include novel applications and more in-depth discussions about ICTD impact. In the 2007 Third Global Knowledge Conference GK3 in Malaysia, Kuala Lumpur, the last of the large ICTD international events of the decade, a dozen different sub-themes were tackled, including access, readiness, availability, financing, governance, ICT applications, uptake, and impact [1]. At the end of the first decade of the millennium, DevCom work with grassroots uses of community radio, popular video or other communication technologies had been mostly abandoned in the field of ICTD, replaced by computers, mobile phones, and related infrastructure and services. The magnitude of the growth in the field of ICTD is unquestionable. Between 2000 and 2010 several new journals were created which focused on research about the use of ICT for development, and hundreds of conferences have taken place to discuss progress in this field. Patra and colleagues offer an overview of some of the salient trends in the field based on an extensive literature review and a survey of 50 researchers and practitioners. They conclude that the field of ICTD is reaching maturity, focused mostly on the domains of agriculture, education, (technical and social sciences) communication, governance, healthcare, design, business of

General Terms
Management, Measurement, Human Factors

Keywords
ICTD, content analysis, developing countries, domain, theory, meta analysis

Acknowledgements
Authors wish to acknowledge the contribution of Adam Taplin, Andrew Brink, Bridget Nowlin, Bryan Dosono, Carl Burnett, Claire Imamura, Deirdre Coyle, Edgar Sioson, Elaine Richman, Elly Krumwiede, Emily Cunningham, Erica Leigh Nelson, Erin Briggs, Erin Mettling, Freeda Brook, Gary Gao, Heather Gillanders, Joel Turner, Jung-Ho Ryu, Kate MacBain, Marissa
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICTD '12, March 12 - 15 2012, Atlanta, GA, USA Copyright 2012 ACM 978-1-4503-1045-1/12/03$10.00.

2
ICTD, and general ICTD [2]. Toyama analyzes the evolution and growth in the ICTD field, from before telecenters to the current enthrallment with mobile phones, and discusses opportunities for mutual learning with the field of Human Computer Interaction (HCI) [3]. Heeks suggested there was a 2000% increase in publications about ICTD between 1999 and 20081, concluding that the ICTD research area is growing significantly faster than other cognate areas. He also described the need for a new ICTD paradigm, which he labeled ICT4D 2.0, to overcome the limitations of the early models. ICT4D 2.0 is one in which new technologies, new applications, new innovation and implementation models, and new viewpoints are required to truly harness the potential of ICT for development [4]. Furthermore, he posits that the ICTD outputs to date reflect: (i) a bias to action and not a bias to knowledge, (ii) a preference for what is narrowly descriptive, and (iii) a field that is not analytical enough. Others suggest that shortcomings of this research area include a lack of: theory, conceptual definition, interdisciplinary approach, qualitative research and longitudinal research [5, 6]. In this context, this paper seeks to understand some of the characteristics of the research literature on ICTD in the first decade of the millennium. How are the different journals and academic conferences describing and analyzing the field of ICTD? How are they similar or different across different journals and conferences, and how have they changed over the decade? In particular, this paper addresses the following questions: what are the domains, level of analysis, technology objects of study, and main contribution to the field in the dominant academic literature about ICTD? How have these changed over the past decade? Future studies can go further back in time and include the 1990s, which saw the birth of the ICTD field, or they can extend this type of study to include books and book chapters, posters and demos, additional journals, other non-academic sources such as development agency reports, or analyze practitioner conferences and publications for a more full comparison of this applied and interdisciplinary field of inquiry and intervention. The remainder of the paper presents the research methods employed to collect and analyze the data, a description of the most salient findings pertinent to the research questions stated above, and a discussion of the implications of these findings for the field of ICTD. education, social work, management, and public administration, to name a few. We selected a small number of journals and conferences that draw scholars from different fields of inquiry and that are focused on ICTD at large. Five Journals: The sample included all research papers (excluding editorials, book reviews, notes from the field and other non-research contributions) from the top three journals according to Heeks rankings2 (Information Technologies & International Development ITID, Electronic Journal of Information Systems in Developing Countries EJISDC, and Information Technology for Development ITD). Two additional journals were retained for their importance to the field, although one is too new to have a meaningful comparative rank International Journal of Information and Communication Technologies for Human Development IJICTHD, and the other one Journal of Community Informatics JoCI- is not ranked by Heeks). Journals ranked fourth to seventh were excluded because they are topic or region specific (Asia, Africa, Education, and Health, respectively). Additional journals (i.e., Information Technology & People, or Information Development) would be relevant to this study, but they were left out given limited resources: they are ranked lower, they cover a wider variety of topics than ICTD, and they both started before 2000 and include a high number of papers published during the decade. Two Conferences: In addition to the journals, papers from the two leading ICTD conferences (once again, according to Heeks rankings3): the International Conference on Information and Communication Technologies and Development (ICTD), which came together four times during the decade (2006, 2007, 2009 and
Acro nym ITID EJIS DC ITD JOCI IJICT HD Rank 1 2 3 NA 13 (Too new) Top Full name Information Technologies & International Development Electronic Journal of Information Systems in Developing Countries Information Technology for Development Journal of Community Informatics International Journal of Information and Communication Technologies for Human Development International Conference on Information & Communication Technologies and Development International Conference on Social Implications of Computers in Developing Countries Active since: Published 2000-2010

2003 2000 1986 2004 2009

26 issues 194 papers 44 issues 270 papers 7 issues 149 papers 18 issues 115 papers 8 issues 36 papers 4 conf, (2006, 07, 09, 10) 140 papers 4 conf, (2002, 05, 07, 09) 160 papers

ICTD

2006

2. METHODS
Between January and June 2011 we analyzed the content of 948 out of 1064 published peer-reviewed academic papers from the ICTD field. We focused on papers that appeared in five peerreviewed journals and two conference series between 2000 and 2010 (see details in figure 1). The selection of journals and conferences places an important boundary on the findings of this study. ICTD is an interdisciplinary field, with scholars, activists and practitioners publishing in numerous, different fields including sociology, anthropology, communications, rural development, computer science, information science, humancomputer interaction, engineering, telecommunications, health,
1

IFIP WG 9.4

Top

1998

Figure 1: 2000-2010 ICTD Journals and Conferences Studied

Richard Heeks, ICT for Development Research, Size and Growth, http://ict4dblog.wordpress.com/2010/02/08/ict-fordevelopment-research-size-and-growth/

Richard Heeks, Table of ICTD Journal rankings: http://ict4dblog.wordpress.com/2010/04/14/ict4d-journal-rankingtable/ 3 Richard Heeks, Table of ICTD Conference rankings: http://ict4dblog.wordpress.com/2010/04/28/ict4d-conferencepapers-impact-and-publication-priority/

3
2010); and the International Federation of Information Processing Working Group 9.4, Social Implications of Computers in Developing Countries (IFIP WG9.4), which also convened four times during the decade (2002, 2005, 2007 and 2009). Figure 1 summarizes the journals and conferences included in the study. After all papers were identified, each paper was coded using an online coding instrument designed for this study to identify title, author(s), abstract, and keywords (if available); to categorize from a non-exclusive, multiple choice list the domain of development work, technology object of study, level of analysis, and contribution to the field; and to assess each papers epistemological stance [based on 7], type of research methods used, and relation between technology and society. In addition, the research questions, purpose of development, and key findings and recommendations were extracted (if explicitly stated) for future secondary analysis. Given the magnitude of the sample we had to limit the coding variables, and we excluded the geographic location covered in the papers, and the country of origin or discipline of the authors, among other interesting variables that could have been analyzed as well. Coders were asked primarily to take into consideration title, abstract, introduction, and conclusions in order to inform their choices in the codebook. On some occasions only abstracts considered, because full text was not available (e.g., many IFIP papers). In addition, some sources included only protected PDF files that did not allow for text to be copied and pasted (e.g., ITD, IJICTHD), so codebook sections on research questions, research findings or recommendations, and notion of development are excluded for those sources. The code book is appended. A total of 41 coders participated in the coding (23 papers per coder on average), mostly consisting of graduate students from the authors university. The majority of coders participated in a two-hour training session, and all received one-on-one coaching and mentoring as needed. Intercoder reliability was assessed by randomly assigning 51 papers, mixed across all sources, to two different coders. The rate of agreement for all coding decisions was 88%, with the lowest ones being for epistemological stance and relation between ICT and society (68% and 66%, respectively), which are analyzed elsewhere. This result indicates that the coding scheme was trustworthy for the issues analyzed in this paper. The intercoder reliability is not as high for questions requiring more interpretation and conceptual background on the part of the coders, which may require further research and validation. Figure 2 summarizes the results of the intercoder reliability tests:
Coding Category Title Author(s) Journal/Conference Year Abstract Keywords Domain Object of Study Level of Analysis Contribution Research Questions Key Findings Goal of Development Relation ICT & Society Intercoder agreement rate 100% 100% 100% 100% 100% 80% 92% 82% 82% 73% 92% 92% 94% 66% Epistemology Research Methods Total Score 68% 82% 88%

Figure 2: Results of Intercoder Reliability Tests While the use of non-exclusive categories to code the papers does not give a definitive picture, the identification of salient trends is more significant than the potential for error or interpretation in the coding and analysis process.

3. FINDINGS
The total production of academic papers in the ICTD field has dramatically grown between 2000 and 2010, and there are now more venues available for publication than there were before. Only two of the sources studied existed before 2000 (IFIP and ITD). There is a noticeable growth trend over the decade, with a dip in 2008, likely explained by the fact no ICTD or IFIP conferences were held that year. The highest peak was reached in 2009, with 165 papers (17.4% of the total sample) published that year. The lowest point in published record was in 2001, when only EJISDC published any issues. Both ICTD and IFIP conferences seemed to move toward fewer papers accepted or presented over time, while journals seem to have settled into a more or less constant pattern of publication flow (IFIP 2009 and ITD 2010 show some attrition in the coding, as more papers were actually published than coded for those sources in those years). The overall coding results of the papers published between 2000 and 2010, broken down by source, are displayed in Figure 3: Total Papers Analyzed, by Source and by Year.
00 EJISDC ICTD IFIP IJICTHD ITD ITID JoCI Total 19 01 13 02 23 30 21 17 10 13 53 62 03 35 04 20 05 17 47 23 12 11 11 0 18 26 10 13 3 06 36 43 07 24 36 26 20 17 17 14 0 08 20 09 34 43 13 24 14 17 19 16 4 10 32 19 13 5 25 18 11 2
Total % of total

40

16 12 48

27 13 13 73

273 141 116 37 145 136 100 948

29% 15% 12% 4% 15% 14% 11% 100 %

Figure 3: Total Papers Analyzed, by Source and by Year

3.1 ICTD Domains: Focus on Business and Empowerment

Based on an analysis of the different domains of work reported in the ICTD literature between 2000 and 2010, we concluded that there was a very strong overall focus on business (including ecommerce, entrepreneurship, employment, industry, or microfinance: 36%), empowerment (including issues of community development, citizen participation, or social capital: 31%), and education (including literacy and science: 22%), followed in fourth place by e-government activities (including politics, democratization, government services, or corruption: 18%). Other possible domains listed in the literature (in order of decreasing proportion) are the following: Minorities (11%); Health (11%), Other (11%); Gender (7%); Agriculture (6%); Youth (3%); Environment (3%), Relief (2%); Disabilities (1%).

4
In addition, 4% of the papers were not applicable (NA) to code for domain, and 0.8% were coded as not sure (NS) about the domain. These are non-exclusive categories (any given paper can touch on multiple domains simultaneously (except for NA or NS), and analysis is based on all categories having the same weight, with no indication of a predominant or most important domain ascribed to each paper). Even though there is some variation across specific journals or conferences (described below), the averages tend to follow the same distribution and ranking order of domains of work, with only minor variations. The overall domains of work, for all papers and across all sources published during the decade, are summarized in Figure 4.
Overall Domains of Work in the ICTD Literature, 2000-2010
36% 31% 22% 18% 11% 11% 11% 7% 6% 3% 3% 2% 1% 4% 1%

areas such as e-government, which have several domain-specific journals not included in this study. When comparing the two top conferences in the field, IFIP has a stronger concentration on gender issues (10%), while ICTD tends to have a higher concentration on minorities (15%). Otherwise, the distribution of the domain in the papers presented at these conferences is quite similar. There is no noteworthy variation over time, as opposed to the total picture of the decade, in the predominance of the different domains of work in the ICTD literature. The proportions of papers touching on different domains, by source, are presented in Figure 5.
EJIS DC ICTD IFIP IJICT HD ITD ITID JoCI

Figure 4: Overall Domains of Work in the ICTD Literature, 2000-2010 (note: non-exclusive category, does not add up to 100%) It is striking that the top four domains (business, empowerment, education, and e-government) are the top four for all journals and conferences in the study, albeit with slight differences in their order. The sources with a stronger emphasis on business are IJICTHD (51%4); ITD (42%) and IFIP (39%), all of which have an emphasis on business higher than the 38% mean across all sources. On the other hand, the sources with a stronger emphasis on empowerment are IFIP (37%), ICTD (35%) and JoCI (34%), all of which are above the 31% mean for empowerment across all sources. It is noteworthy that IFIP is above average (more concentrated), and EJISDC is below average (more distributed), for both business and empowerment, the two top domains overall. Furthermore, IJICTHD has a very high proportion of papers focused on e-Government (27%, vs. 19% mean), while JoCI has a very high proportion of papers focused on education (33%, vs. 23% mean). Both ITD and ICTD have higher concentrations of papers that touch on minorities (19% and 15%, vs. 11% mean). All sources have a notoriously low proportion of papers that touch on the domains of health (11%), agriculture (6%), or youth (3%) even though all of these domains are central to the Millennium Development Goals (MDG): three of the eight MDG have to do with health! It is possible that health-related research and literature in ICTD is being published in more specialized venues such as the Journal of Health Informatics in Developing Countries (ranked #7), reducing the magnitude of this area of work in the more general ICTD literature. The same may be true for other

Business Empowerment Education E-Gov Minorities Health Gender Agriculture Youth Environment Relief Disabilities

31% 28% 22% 17% 10% 12% 7% 7% 3% 4% 1% 2%

32% 35% 16% 18% 15% 13% 5% 3% 4% 2% 1% 1%

39% 37% 21% 17% 4% 11% 10% 7% 3% 3% 4% 0%

51% 27% 24% 27% 5% 5% 5% 14% 3% 0% 3% 3%

42% 30% 24% 21% 19% 8% 9% 7% 3% 5% 2% 1%

36% 27% 21% 19% 10% 13% 6% 7% 6% 4% 1% 0%

38% 34% 33% 16% 12% 10% 3% 2% 2% 2% 1% 3%

Empowerment

Disabilities

Youth

Other

Relief

NA

Environment

Agriculture

Education

Minorities

Not Sure

Business

E-Gov

Gender

Health

Figure 5: Domains of Work in the Published ICTD Literature, by source: 2000-2010

3.2 Technology Objects of Study: from fuzzy ICT in General to focus on mobile phones
During 2000 and 2010 the single most common technology object of study in the academic literature on ICTD was the rather fuzzy ICT in General (48%); followed only from afar by Information Systems (including Information Technology Industry, Management Information Systems (MIS), or Decision Support Systems: 26%); and trailed by Software (including software applications and software design: 14%). More than 70% of all the published literature covered in this analysis includes at least one of these objects of study. Although ICT in General is, by definition, fuzzy and unfocused, preliminary tests of the coding instrument revealed that many papers did not really have a clear object of study corresponding to the available options, so this fuzzy object was added. The result is astounding: almost half of all the papers include the fuzzy label of ICT in General as their object of study, and more than half of these (53%) have it as the ONLY technology identified in the paper. After ICT in General, Information Systems, and Software, other types of technologies most often studied in the ICTD literature are Telecenters (including different non-profit variations such as community technology centers (CTC), multipurpose community centers (MCT) and others, defined as non-profit venues and with an intent to contribute to people development; 11%); Mobiles (mobile phones, smart phones, text messaging and other mobile applications: 10.4%); and Infrastructure (including network infrastructure, technical connectivity, and things generally dealing with cables, circuits and machines: 10.1%). The remaining

Note that EJISDC is the newest journal in the ICTD field, with only 8 issues in 2009 and 10 for consideration in this study.

5
technologies receive a small proportion of the attention in the ICTD literature, as follows: Other (most of other indicates technologies related to geographic information systems (GIS): 7%); School Labs (including computers in schools and other education settings: 5%); Computers (including PC, laptop, OLPC, tablet PC, etc., and not including shared access computers such as telecenters, cybercafs, libraries or school labs: 5%); Cybercaf (including variations of for-profit shared access computers such Internet cafs, kiosks or other for-profit venues: 4%); Media (including radio, video and other media: 3%); Not Sure (coder not sure of which technology to choose: 2%); and Libraries (including public libraries and community libraries: 1%). Technology Objects of Study in ICTD Research
48%

Figure 6 summarizes the distribution of the technologies most studied in the ICTD literature. It is noteworthy that Libraries receive the least attention of all (1%), even less than other media such as radio or video, or than the occasions when coders were not sure which technology was being studied in a given paper (3% and 2% respectively). Changes over time: Studies about the newest player in the ICTD technology world, the mobile phone, jumped in popularity starting in 2006 (the first year that the ICTD conference was held: one third of the papers from that conference touched on mobile phones). Mobiles continued to receive increasing attention in the following years, especially in ICTD conference, in ITID journal and, more recently, in EJISDC journal as well. In 2010, Mobiles were the second most common object of study (19%) after ICT in general (44%), followed by Other (notably GIS: 13%) and by Information Systems, Telecenters, and Software (all at 10%). Papers on ICT in general continue to be the most numerous and continue to go up, while the frequency of papers that touch on Information Systems and on Software has tended to go down. This important shift in focus toward mobile phones and away from Information Systems or Software is remarkable in the data, as displayed in Figure 7. Differences across sources: Furthermore, across sources (journals and conferences), salient differences in the relative frequency of appearance of different technology objects studied between 2000 and 2010 occur. IFIP has the highest proportion of studies that touch on Information Services (58%), while JoCI and ICTD have the lowest (5% and 12%, respectively). ICTD has the highest proportion in software (21%), mobiles (23%), computers (11%), and cybercafs (8%). IFIP is lowest in mobiles (5%). Both ICTD and ITID share a strong predominance of studies touching on mobiles, cybercafs, and media. IFIP seems to be more clearly focused around ICT in General and Information Systems; it has a

26%

14%

10%

10%

9%

7%

5%

5%

4%

3%

2%

1%

Figure 6: Technology Objects of Study in ICTD Research, 2000-2010

Trends in Technology Object of Study

60%

50%

40% Averages per Year

ICT in general Information Systems Software

30%

Mobile Linear ( ICT in general) Linear (Information Systems)

20%

Linear (Software) Linear (Mobile)

10%

0% 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Figure 7: Trends over Time in some of the Technology Objects of Study

6
lower proportion of papers touching on the other types of technology objects. IJICTHD has the highest proportion of studies that deal with school computers (but bear in mind it is the newest journal, with shortest track record). Finally, JoCI is highest in ICT in general (57%), telecenter (26%), infrastructure (18%), and library (3%) as objects of study. The technology objects of study in the ICTD literature, distributed by source (journals and conferences) is summarized in Figure 8.
Tech Object of Study ICT in general Information Systems Software Telecenter Infrastructure Mobile School computers Computers Cybercaf Media Library EJISDC 48% 27% 14% 5% 5% 8% 3% 4% 3% 2% 1% ICTD 33% 12% 21% 16% 11% 23% 9% 11% 8% 6% 0% IFIP 41% 58% 14% 6% 3% 3% 5% 2% 3% 1% 0% IJICTHD 43% 24% 19% 5% 11% 5% 11% 8% 0% 3% 0% ITD 57% 39% 12% 6% 8% 6% 3% 2% 1% 3% 1% ITID 54% 16% 15% 13% 12% 17% 6% 7% 5% 6% 1% JoCI 57% 5% 2% 26% 18% 6% 6% 3% 6% 3% 3%

Scope of Analysis in ICTD Research

41%

26%

22% 16% 15% 8% 8% 2% 1%

Figure 9: Overall Scope of Analysis in ICTD Research, 20002010 When comparing across sources, it is noteworthy that all conferences and journals except for JoCI present country, organization, and multiple countries as the most frequent levels of analysis. Only IJICTHD shows organization as a more frequent level of analysis than country (IJICTHD organization is 43% and country is 38%). JoCI, on the other hand, has its highest proportion of papers focused on neighborhoods (38%, highest across all sources, vs. mean 17%); also, JoCI has the highest concentration of analysis at the city level (17%, vs. mean 11%), although still smaller in proportion than country (19%). If we had excluded EJICTHD and JoCI from the study (with the argument that they are too new or not ranked) the predominance of country, organization, and multiple countries (in that order) for scope of analysis would have been absolute. ICTD, ITID and JoCI all show a broader distribution of the different scopes of analysis, while the other sources (EJISDC, IFIP, IJICTHD, ITD) all show a relatively more concentrated focus on fewer options. The breakdown of the proportions of scope of analysis by source is summarized in
Level of Study Individual Family Neighborhood Organization City Social Networks Region in country Country Mult. countries
EJISDC

Figure 8: Technology Objects of Study in ICTD Research, by source: 2000-2010

3.3 Level of Analysis: Preference for country and organizational levels

The country level is the most common unit of analysis in the published ICTD literature between 2000 and 2010: at 40%, country analyses far outnumber the studies that look at organizations (including non-profit organizations, universities, businesses: 29%) or at multiple countries in a single study (24%)5. Neighborhood or community-level analysis and regions within a country are both less frequent units of analysis overall (16% and 15%, respectively), followed by individuals or cities (both around 8%), and trailed by social networks or families as units of analysis (2% and 1%, respectively). We did not analyze the geographic distribution of the countries studied across the entire sample. The overall distribution of the levels of analysis is summarized in Figure 9.

ICTD 18% 1% 27% 21% 13% 0% 28% 28% 22%

IFIP 4% 2% 11% 34% 3% 3% 17% 55% 22%

5% 1% 8% 19% 4% 1% 14% 45% 18%

IJICT HD 3% 0% 11% 43% 0% 3% 5% 38% 35%

ITD 1% 0% 5% 33% 4% 2% 6% 50% 28%

ITID 13% 2% 19% 24% 11% 1% 17% 43% 29%

JoCI 13% 2% 38% 24% 17% 6% 14% 19% 13%

Figure 10.
Level of Study Individual Family Neighborhood Organization City Social Networks Region in country Country Mult. countries
EJISDC

ICTD 18% 1% 27% 21% 13% 0% 28% 28% 22%

IFIP 4% 2% 11% 34% 3% 3% 17% 55% 22%

Note that this study does not judge whether the research papers considered are successful at analyzing the country or organizational levels effectively, reliably or credibly. The quality of ICTD research is a different matter that needs to be assessed separately.

5% 1% 8% 19% 4% 1% 14% 45% 18%

IJICT HD 3% 0% 11% 43% 0% 3% 5% 38% 35%

ITD 1% 0% 5% 33% 4% 2% 6% 50% 28%

ITID 13% 2% 19% 24% 11% 1% 17% 43% 29%

JoCI 13% 2% 38% 24% 17% 6% 14% 19% 13%

7
Figure 10: Scope of Analysis in ICTD Literature, by source: 2000-2010 When looking at conferences alone, IFIP shows a clear predominance of country-level analyses over organization or multiple countries (55%, 34% and 22%, respectively), while ICTD shows more variety, with a smaller concentration on single country, more on multiple countries, and a smaller concentration on organizational analyses (43%, 24% and 29%, respectively), in addition to more on region within country, neighborhood, individuals, and cities (28%, 27%, 18% and 13%, respectively). When looking at changes over time, the general predominance of country, organization, and multiple countries as main units of analysis are sustained. Nonetheless, 2006 marks the beginning of a growing attention to neighborhoods as a unit of analysis, which was virtually nonexistent before. This started with ICTD and continued in JoCI. of research contributions, except for an inverse relationship between theory and design: while ICTD appears to be high on design (38%) and low on theory (14%), IFIP appears to be the opposite, high on theory (41%) and low on design (9%). Moreover, IJICTHD displays the highest proportion of contributions around best practices (49%; close behind are ITID with 39% and EJISDC with 34%), and for policy (41%). IJICTHD and ICTD have the highest proportion of design contributions (30% and 38%, respectively). On the other hand, IFIP and JoCI have the highest proportions for theory contribution (41% and 39%, respectively). ICTD has highest proportion of contributions around field experience and design. Finally, ITD has the highest proportion of testing theory (24%). The details of the research contributions, disaggregated by source, are summarized in Figure 12.
Contribution the Field to

EJISDC 34% 28% 22% 12% 7% 8% 5%

ICTD 24% 38% 22% 14% 38% 13% 10%

IFIP 23% 44% 23% 41% 9% 7% 12%

IJICTHD 49% 14% 41% 22% 30% 8% 0%

ITD 26% 30% 31% 26% 6% 24% 10%

ITID 39% 27% 26% 17% 13% 17% 20%

JoCI 24% 30% 18% 39% 8% 12% 10%

3.4 Contribution to the Field: Best practices and field experiences

The top four research contributions across all sources are Best Practices (including lessons learned, or success factors: 31%); Field Experience (including description, evaluation or analysis of an experience or project: 30%); Policy Recommendations (26%); and Theory (including formulation of theory, conceptual framework or typology: 24%). These are followed by a less pronounced proportion of studies that contribute Design (including description, creation, evaluation, or testing of artifact, software or hardware: 16%); Testing Theory (including validation of existing typologies, theories or frameworks: 13%); and Methods (including novel methods or approaches to collect or analyze data: 10%). The overall distribution of contributions to the field in the ICTD literature between 2000 and 2010 is summarized in Figure 11.

Best Practices Field Experience Policy Theory Design Testing Theory Methods

Figure 12: Contributions to the Field of ICTD, by source: 2000-2010 A shift over time is noticeable in the data of 2000-2010. The decade has seen an upward trend in the proportion of research papers that contribute design, policy recommendations or theory (especially formulation, but also testing and validation), while best practices and methods remain flat over time, and the proportion of field experiences decreases. The trends over time with more significant changes over the course of the decade are displayed in Figure 13.

4. DISCUSSION
Contributions to the Field in ICTD Research
31% 30% 26% 24% 16% 13% 10%

These findings are important for understanding where the field of ICTD is coming from and where it might be headed. There was an unquestionable growth of intellectual production in the field of ICTD during the first decade of the millennium, with a proliferation of recognized venues for publication and discussion of its advances. There appears to be some overlap between conferences and journals, with conference papers frequently becoming journal contributions, either as part of special issues that emerge from the conference or as stand-alone papers that are published, presumably after further refinement and work.

Figure 11: Contributions to the Field of ICTD, 2000-2010 Despite this overall picture, each source appears to have a unique character in the type of research contributions it favors. The two top conferences in the field appear to be very comparable in terms

8
Trends in Research Contribution
60%

50%

40%

Theory Policy Field Experience

30%

Design Linear (Theory) Linear (Policy) Linear (Field Experience)

20%

Linear ( Design)

10%

0% 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Figure 13: Trends over time in Contribution to the Field, 2000-2010.

4.1 ICTD Domain: Business and Empowerment

Is the field of development concerned with creating new business opportunities for economic development or with empowering people and communities to foster social development? This is the underlying question that is evidenced by the dynamic tension between the focus on business and on empowerment, the two most important domains of work that emerge in the analysis of the ICTD literature between 2000 and 2010. Overall, Business (36%) receives a slightly greater focus than Empowerment (31%). Little variation over time occurs in the course of the decade; nonetheless, some journals and conferences tend to have a stronger concentration of one or the other, as described above. The tension between economic development and people empowerment has long been a matter of debate, because some favor economic growth and business opportunities as measures of development [for example, 8, 9, 10], while others emphasize participation and empowerment as key conditions for development [for example, 11, 12, 13]. Although a growing trend to integrate both perspectives exists, highlighting the importance of both economic development and peoples empowerment [14], we found that only 10% of all the papers include both Business and Empowerment as their domains of intervention. There is more room to grow in integrating both economic growth and empowerment as important dimensions of development in the ICTD literature and practice.

Apart from the predominance of Business and Empowerment as preferred domains in the ICTD literature, the proportion of papers focusing on minorities, health, gender, agriculture, youth, or the environment is relatively low. All are generic areas of importance to development and central to the Millennium Development Goals (MDG). Additionally, a very low proportion of papers focused on relief efforts or people with disabilities, specialized domains that are gaining much attention in the field of development. Nonetheless, the cumulative presence of all these domains in the ICTD literature continues to be very small.

4.2 ICTD Objects of Study: the emergence of Mobile

The analysis of the different technology objects of study offers the most variation across different journals and conferences and demonstrates important shifts over time. Although studies continue to deal predominantly with ICT in General, the proportion of studies about Information Systems is dropping significantly over time, and studies about Mobile phones are gaining momentum. If they continue growing at the current pace, they will outnumber studies of all other technologies put together, except perhaps ICT in General. The growth in interest in Mobile has already created a new hype, with a discourse reminiscent of the early euphoria about the Internet as the new agora of electronic democracy [15], or of

9
Telecenters as the magic formula for community development [16]. Influential researchers promote Mobiles as the new paradigm in what is called ICTD 2.0 [4]. The speed and extent of mobile phone adoption is unprecedented, but it may not be replacing other technologies. Most common in developing countries and underserved communities in developed countries are very basic phones: they are generally shared among family or community members, they rely mostly on prepaid airtime, they make very little use of text messaging, and the use of mobile data (Internet) is extremely rare [17]. Is the euphoria about mobile phones growing, has it peaked, or is it declining? A quick look at the 2011 issues available for the first half of 2011 in the journals included in this study show no papers touching on mobile phones published during 2011 (JoCI had not yet published a 2011 issue in the first semester of the year), which may be an indication that the Mobile euphoria has peaked and could be on the decline. Studies of public access or shared computing environments have been primarily focused on telecenters and school labs, although there have been small proportions of studies on cybercafs and even a few on libraries. Together, shared computing environments continue to be of importance in developing countries as well as among underserved populations in developed countries. The role of libraries in particular is very understudied in the ICTD literature, a gap that calls for further research [18]. On the other hand, studies of Information Systems have declined over time; further research may help to explain the waning interest on Information Systems in the ICTD field. contributions (generating new or validating existing theory) and the decreasing proportion of (descriptive) field studies are both noteworthy findings that suggest a maturing field of ICTD. Likewise, the growth in interest about policy recommendations (half of which are targeted at the national, country-level, discussed above) suggests a growing awareness of the importance of influencing policy as part of the success of development work, informed by more scalable, national-level research and analysis. The degree to which this growing intent to influence policy is translated into actual policy changes should be further researched. The growth in design contributions in the ICTD literature is also most interesting, reflecting a stronger interest in design and testing of technological artifacts as an important part of the ICTD field. The relevance or usefulness of such design contributions would benefit from additional research.

5. CONCLUSIONS
This study uncovered important trends in selected published literature in the field of ICTD between 2000 and 2010. Drawing from a content analysis of 948 peer-reviewed papers from five journals and two conferences, we found an unprecedented growth in the quantity and diversity of published research papers and to their valuable contributions to knowledge in the interdisciplinary field of ICTD. The published literature exhibits a dynamic tension between a focus on business and economic development and a focus on empowerment and community development, the two most salient domains of work in the field. While most of the literature seems to analyze ICT in General, there is a decreasing interest in Information Systems, a sustained interest in telecenters and other public access venues (although very little research is reported about libraries), and a fast growing interest in mobile phones as a new technology with much promise for ICTD. Most research is focused on particular countries or organizations, with some indication that this may be a reflection of a maturing field that tries to go beyond the field experiences of individual projects and scales up with best practices and recommendations for national policies. Finally, the recent growth of papers that contribute theory formulation or testing or artifact designs points to a changing field of ICTD. As a field ICTD appears to be shedding its a-theoretical past and laying a more solid foundation of theories and conceptual frameworks for its work. In addition, the field is no longer just using existing technologies for development but helping shape the design of novel technologies and applications for developing world contexts. These findings can be further explored by doing more in-depth analysis of selected papers relevant to each topic. Also, a broader analysis of papers in other disciplinary journals and conferences, books and book chapters, as well as government and non-governmental agency reports can help strengthen or complement this study. The field of ICTD is growing, consolidating, and evolving, and it is more diverse and interdisciplinary than what is represented here. Ongoing analysis of this study is exploring the nature of the key findings and recommendations, as well as the types research questions and the way researchers define the goals development in the published literature. Finally, further analysis will explore the nature of the relationship between ICT and society, the preferred research methods employed, and the epistemological stance implied in the literature (forthcoming).

4.3 Level of Analysis: Country and Organization

Most ICTD research focuses on a country or an organization as the unit of analysis. Nonetheless, the degree to which results are actually representative of a particular country is uncertain. A cursory glance at papers focused on particular countries reveals that most tend to focus on an experience, community, or organization within that country, with very little evidence of research that aims at being representative of the diversity and richness of the country as a whole. We note a quick jump from the description or analysis of a field experience, project or organization to conclusions or recommendations that are aimed at the national, country level. Our data reveals that the country or national level is the preferred scope for policy recommendations and best practices, even though much of the research is actually at the more limited level of an organization, project, or experience. In fact, about 25% of the papers coded as country are also coded as organization, the highest overlap in these nonexclusive categories. It would be valuable to conduct further studies to ascertain whether ICTD research is in fact moving toward higher levels of generalization, from individual case studies to national-level policy recommendations, as the preliminary findings of this study suggest.

4.4 Contribution: building stronger theoretical base

The field of ICTD has been criticized for being focused too heavily on action and too little on knowledge and for being too descriptive and insufficiently analytical. While the overall picture of the ICTD literature between 2000 and 2010 would seem to corroborate this idea, the growing attention to theoretical

10

6. APPENDIX: CONTENT ANALYSIS CODE BOOK

Field 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Title Author(s) Journal / Conference name Year Abstract Keywords or Index Terms Domain of development work Technology object of study Scope, level of analysis Main type of contribution to the field of ICTD Research Question(s) Key findings or recommendations Goal of Development Relation between ICT and Society Epistemological Stance Research Methods Type of field copy & paste copy & paste single choice single choice copy & paste copy & paste multiple choices (non-exclusive) multiple choices (non-exclusive) multiple choices (non-exclusive) multiple choices (non-exclusive) copy & paste copy & paste copy & paste single choice single choice single choice

12. 13.

14.

15.

16.

17.

18.

7. REFERENCES
1. Heeks, R., Do information and communication technologies (ICTs) contribute to development? Journal of International Development, 2010. 22(5): p. 625-640. Patra, R., J. Pal, and S. Nedevschi. ICTD state of the union: Where have we reached and where are we headed. in Information and Communication Technologies and Development (ICTD), 2009. 2009. Doha: IEEE. Toyama, K., HumanComputer Interaction and Global Development. Foundations and Trends in Human Computer Interaction, 2010. 4(1): p. 1-79. Heeks, R., The ICT4D 2.0 Manifesto: Where next for ICTs and International Development?, 2009, Institute for Development Policy and Management: Manchester, UK. p. 33. Gomez, R. and S. Pather. ICT Evaluation: are we asking the right questions? in International Development Informatics Association conference IDIA. 2010. Cape Town, South Africa: School of Information Technology, Monash South Africa. van Dijk, J., Digital divide research, achievements and shortcomings. Poetics, 2006. 34(4-5): p. 221-235. Guba, E.G., The Paradigm dialog1990, Newbury Park, Calif.: Sage Publications. Easterly, W., The elusive quest for growth : economists' adventures and misadventures in the tropics2001, Cambridge, Mass.: MIT Press. xiii, 342 p. Hoeven, R.v.d. and A.F. Shorrocks, Perspectives on growth and poverty2003, Tokyo ; New York: United Nations University Press. xxvi, 302 p. Sachs, J., The end of poverty : economic possibilities for our time2005, New York: Penguin Press. xviii, 396 p. Melkote, S.R. and H.L. Steeves, Communication for Development in the Third World. Theory and Practice for Empowerment. 2nd ed2001, Thousand Oaks: Sage. 422.

Crewe, E. and E. Harrison, Whose development? : an ethnography of aid1998, London ; New York Arce, A. and N. Long, Anthropology, development, and modernities : exploring discourses, counter-tendencies, and violence2000, London ; New York: Routledge. xvi, 232 p. Unwin, T., ed. ICT4D: Information and Communication Technology for Development. 2009, Cambridge University Press: Cambridge. Gore, A., Address to the International Telecommunication Union, 21 March 1994, 1994: Buenos Aires. Gomez, R. and A. Ospina, The Lamp Without a Genie: using the Internet for development without expecting miracles. Journal of Development Communication, 2001. 12(2). Chigona, W., et al., Can Mobile Internet Help Alleviate Social Exclusion in Developing Countries? Electronic Journal of Information Systems in Developing Countries, 2009. 36. Gomez, R., ed. Libraries, Telecentres, Cybercafes and Public Access to ICT: International Comparisons. 2012, IGI Global: Hershey, PA. 537.

2.

3.

4.

5.

6. 7. 8.

9.

10. 11.