Nordlund 01 Samantha Nordlund Last Mile Project; Advisory Report 13 November 2013 Using Technology to Monitor & Evaluate

Project Collaborators
DAT: Ja mi e Pfa ff, Andrew Whi tti ngton, Woohyun Chun Independent International Consultant: Gopa l a n Ba l a gopa l Indian Institute of Management, Kozhikode: Pra na v Mehta , Durga l a ks hmi , Sudha ns u Dutta , Di pa nka r Bi s wa s , a nd Sha heen Ma l i k

Macro-- Team Research: Assessments on experiences in other parts of the world in reaching hard to reach communities. Micro-- My Research: International examples where modern technology (i.e. internet and cell phones) is used to monitor and track progress of social sector projects in hard to reach communities. Context: The topography of the tribal communities in Wayanad are mountainous and forested, making monitoring and evaluation difficult. Wayanad has 177 Anganwadi Centres through which the child nutrition program is extended into the community, and there is information that needs to be regularly collected from these centres (e.g. how many beneficiaries of Anganwadi Center services, performance of the preschool education component, administrative and logistical issues affecting the Center, etc). Our international consultant on the ground, Gopalan Balagopal, explains that all reporting that flows from [...] centres is paper based and is consolidated at the Project, district, state and national levels. This is inevitably a slow process and the use of the consolidated information for analysis and action at the Project and district level is marginal. He remarks that reporting is transitioning to an electronic database though the systems are yet to be fully established. As the centres in Wayanad and across Kerala look for new ways to collect information, the Notre Dame DAT has been asked to provide research of international examples of how modern technology can be used for monitoring and evaluation of social sector projects in similar hard to reach communities. Key terms useful for further research:
M&E: Moni tori ng a nd eva l ua ti on of s oci a l s ector projects eHealth/e-Health: Hea l thca re pra cti ce s upported by el ectroni c proces s es a nd the i nternet mHealth/m-Health/mobile health: A s ubs egment of eHea l th, mHea l th i s more s peci fi ca l l y the us e of mobi l e devi ces i n col l ecti ng a ggrega te a nd pa ti ent l evel hea l th da ta mDevelopment: The us e of mobi l e devi ces i n the devel opment s phere ICT/ICT4D: Informa ti on a nd communi ca ti on technol ogy/i nforma ti on & communi ca ti on technol ogy for devel opment TEME: Technol ogy Ena bl ed M&E

CASE 1: CLEAR South Asia Report In conjunction with the Abdul Latif Jameel Poverty Action Lab (J-PAL), Innovations for Poverty Action (IPA), Fieldata, and IFMR Trust, the Regional Center for Learning on Evaluation and Results of South Asia (CLEAR - South Asia) has put together a publication by the name Mobile-Based Technology for Monitoring and Evaluation. In the words of CLEAR, this publication is intended to inform would-be users on technology options and act as a reference guide for understanding the application and operationalization of TEME to their projects and programs.

Nordlund 02 The guide highlights the latent potential of TEME to improve data quality, speed and transparency, leading to better implementation of projects, evidence-based decision making, and eventual development outcomes.1 I have included reference to this report for a number of reasons. Not only does it boast a number of internationally esteemed research institutions as contributors, the CLEAR report is wonderfully thorough, informative, and concise. It is broken down into three sections: A Quick Start section on the applicability of mobile technology to projects and how to initiate mobile-based monitoring and data collection tools; a section on Mobile Technology Options and Opportunities which lists the many components of a mobile-based data collection system, including the possible devices, features and functionalities, from low-end phones to smartphones and tablets; a final section on Implementing Mobile Technology in M&E which details implementation issues, including but not limited to estimating costs, planning timelines, training field staff and ensuring data quality. CLEARs report also mentions practical considerations that can dampen the success of digital data collection, such as limited network coverage, low bandwidth difficulties, literacy issues, dialects, and gender disparity. The publication supports research with case studies that highlight TEME initiatives in various countries and contexts. One such case study details the use of mobile phones by the Community Assistants Initiative in Ghana, launched by the Ghana Education Service in 2009. The project was undertaken to improve educational quality in primary schools, and entailed the tracking of about 25,000 students over the course of two years. Over one hundred field enumerators were equipped with phones to photograph and track the children each year. In an effort that resonates with the Last Mile Project, a database with individual pictures of the students by school was collated in real-time on a web-server. The list with photographs became an audit as well as tracking tool for the project.2 This 2009 initiative in Ghana has components similar to the Last Mile Projects child nutrition programs in Wayanad, which hopes to regularly collate beneficiary information from Anganwadi Centres scattered over the district. Furthermore, the 2013 CLEAR report would be an integral resource and guide should the Last Mile Project choose to move forward in implementing mobile technology in its M&E.
Li nks : http://www.theclearinitiative.org/mobile-basedtechnology.html http://www.theclearinitiative.org/mobile-based-tech.pdf

CASE 2: MOTECH The Mobile Technology for Community Health Project (MOTECH) in Ghana is an initiative to decrease the infant mortality rate in Ghana through mobile technology. In addition to sending messages to pregnant women throughout their pregnancy and during the childs first year, the service enables community health workers to use mobile phones to enter information about the patients they have seen and the services they have delivered, increasing efficiency and accuracy on the healthcare side of the equation.3 The program now boasts over 25,000 people registered for the service, and about 300 community health workers equipped with phones to track patients. Through CAREs programs in Bihar, India, pregnant women and children under one are enrolled in the MOTECH system and tracked along the continuum of care. Mobile phones also provide built-in checklists to help [field workers] follow the most appropriate protocol during
1 2

http://www.theclearinitiative.org/mobile-basedtechnology.html http://www.theclearinitiative.org/mobile-based-tech.pdf 3 The Stanford Social Innovation Review http://www.ssireview.org/blog/entry/scale_can_happen_the_motech_experience

Nordlund 03 their interactions with women. Furthermore, mobile phones of health workers in the same catchment area are synchronized with each other, enabling information sharing and improved coordination. The application generates a list of patients in each health workers catchment area who are due for immunization, and supervisors receive real-time reports on services delivered by health workers.4 The MOTECH intervention is notable for its eye toward long-term scale and replication, and emphasis on building components that could be reused in other geographies and other health domains. The Grameen Foundation notes that there is nothing maternal-health or Ghana-specific about the technology components we are building, and worked with Dimagi, InSTEDD and other organizations that had complimentary [sic] technologies to make our services interoperable,5 all of which resulted in the MOTECH Suite. The Suite maintains core funding from the Bill and Melinda Gates Foundation, as well as project-specific funding from organizations such as Johnson & Johnson and USAID. This suite of technology services has expanded the use of MOTECH to other areas of Ghana, as well as development initiatives across India. MOTECH also notably helps health workers track clients in World Vision programs in seven countries, from Afghanistan to Zambia.6 The technology can be locally customized as needed, and supports multiple devices (from low-budget phones to tablets and PCs) as well as message formats (SMS, voice, data),7 suggesting the possibility for easy incorporation into the Last Miles projects in India. The preceding information was found in part through the Stanford Social Innovation Review, which has a special series on scaling up mobile health initiatives that can be accessed here.
Li nks : http://www.s s i revi ew.org/bl og/entry/s ca l e_ca n_ha ppen_the_motech_experi ence#rel a ted_s tori es http://www.i mpa ti entopti mi s ts .org/Pos ts /2013/07/Ma ki ng-a -Di fference-wi th-Mobi l e-Technol ogy http://www.motechs ui te.org/ http://www.motechs ui te.org/i ma ges /s tori es /MOTECH_Sui te_Overvi ew_Nov2012.pdf

CASE 3: Water for People and Akvos FLOW Water for People and Akvo have joined forces to address the issue of monitoring development projects through the creation Akvo FLOW (Field Level Operations Watch) in 2010. Created for the purpose of addressing the need to replace cumbersome paper- based monitoring surveys and the delay in manually compiling the information, FLOW was designed to provide accountability and transparency to donors and the public through fast data collection, survey flexibility, analytical tools for data- driven decision making, and map-based reporting of results.8 Through using Android cell phones, GPS and Google Earth software, FLOW allows volunteers, partners, and community members to record vital data from water points around the world. This data includes location, pictures, and the service status of the wells/pumps. This information is then displayed online, signaling whether a project is functional, broken, or in need of maintenance. FLOW maintains partners such as the World Bank, Water for People, A Childs Right, and the International Water and Sanitation Center IRC, and is currently used around the world in 17

4 5

http://www.motechsuite.org/images/stories/MOTECH_Suite_Overview_Nov2012.pdf http://www.ssireview.org/blog/entry/scale_can_happen_the_motech_experience 6 http://www.impatientoptimists.org/Posts/2013/07/Making-a-Difference-with-Mobile-Technology 7 http://www.motechsuite.org/images/stories/MOTECH_Suite_Overview_Nov2012.pdf 8 http://www.waterforpeople.org/flow-mapping/

Nordlund 04 countries: Ethiopia, Ghana, Liberia, Malawi, Rwanda, Senegal, Sierra Leone, Uganda, Nepal, India, Bolivia, Dominican Republic, Ecuador, Guatemala, Honduras, Nicaragua, Peru.9 The FLOW project map is impressive, as one can view approximately 40 data pieces about a single well point in Bolivia, from numerical factors (e.g. How many people use this water source?...98) to the type of community or name of local contact. The format of FLOW is worth looking into for disseminating information regarding the Last Mile Projects interventions in Wayanad, Palghat and Idukki. So far, the primary applications of FLOW have been in the sanitation, hygiene and water sector, but smaller projects outside the sector have demonstrated its flexibility and potential for diverse applications, potentially opening the door for the Last Mile Projects type of nutritional interventions in India.10
Li nks : http://www.wa terforpeopl e.org/fl ow-ma ppi ng/ http://wa terma pmoni tordev.a pps pot.com/ http://www.wa terforpeopl e.org/a s s ets /pdfs /fl ow/a kvo-fl ow.pdf http://www.wa terforpeopl e.org/everyone/moni tori ng-eva l ua ti on.html

Additional Resources for Further Research: A 2013 World Bank Study: Challenges and Opportunities of Mobile Phone-Based Data Collection: Evidence from South Sudan The 2012 Journal of Health Informatics in Developing Countries report titled A Review on Challenges in Implementing Mobile Phone Based Data Collection in Developing Countries A 2013 paper published by the Global Impact Investing Network in partnership with Acumen, the Aspen Network of Development Entrepreneurs, Echo Mobile, and Grassroots Business Fund entitled Collecting Impact Data Using Mobile Technology The United Nations Global Pulse Project, an online directory of projects that use mobile phones for data collection. The inventory exists as a living document that is regularly updated, accessible in a Google Spreadsheet here.

10

http://www.waterforpeople.org/assets/pdfs/flow/akvo-flow.pdf http://www.waterforpeople.org/assets/pdfs/flow/akvo-flow.pdf