Computer Science Research for Global Development

Kentaro Toyama Assistant Managing Director Microsoft Research India University of Washington March 12, 2009 – Seattle

Technical Research

Global Development Multidisciplinary Approach

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Photo: courtesy of Infosys

Infosys campus, Bangalore, India

Photo: Nimmi Rangaswamy

A small Internet café on a market street in a town near Bombay

Photo: Kentaro Toyama

Rural village with a VSAT Internet connection near Bhopal, Madhya Pradesh

“Kids in the developing world need the newest technology, especially really rugged hardware and innovative software.”
– Nicholas Negroponte (OLPC website, 2005)

“The world's poorest two billion people desperately need healthcare, not laptops.”
– Bill Gates (WRI Conference, Seattle, 2000)

Technology for Emerging Markets
Microsoft Research India Understand potential technology users in developing communities Design and evaluate technology and systems that meet needs and aspirations of potential users Impact communities worldwide through partnerships with Microsoft groups and non-Microsoft organizations

Computer-skills camp in Nakalabande, Bangalore (MSR India, Stree Jagruti Samiti, St. Joseph‟s College)

Multidisciplinary Research
Social science
Aishwarya Lakshmi Ratan
– International Development Public Administration Communications

Technology for microfinance

Jonathan Donner

Mobile banking
Technology in slums

Mobile phones in developing countries PCs usage under free access Middle-class consumption

Nimmi Rangaswamy
– Social Anthropology

Indrani Medhi


– –


David Hutchful
Human Computer Interaction

Mobile-phone interfaces Telecentres Technology in healthcare

UIs for non-literate users
Computers in education DVD multimedia

Kentaro Toyama (Group Lead)


– – –

Computer Science

Saurabh Panjwani
Computer Science

Bill Thies
Computer Science

Technology for agriculture

Rikin Gandhi
– Astrophysics

Video and mediated instruction

Sample Projects
Collaborations with UW
Warana Unwired

Research on annotated video (Natalie Linnell, Richard Anderson)
Digital Green

Mobile phones and young adults (Carolyn Wei)
Mobiles in the Developing World

User studies with CAM (Tapan Parikh)

Technology for Microfinance

Virtual keyboards for MultiPoint (Saleema Amershi)

Advising on IPAI research grant (Chris Coward)
Rural Kiosks

Support through incubation period (Tom Anderson, Paul Javid, Kurtis Heimerl)
Digital StudyHall

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

– – – – – – – – – – – – – – – – – Saleema Amershi Sukumar Anikar Ravin Balakrishnan Abigail Cauchi Jennifer Fenech Rahul Gupta David Hutchful Divya Kumar Andrea Moed Neema Moraveji Merrie Morris Miguel Nussbaum Owen Otto Joyojeet Pal Udai Singh Pawar Bhagya Rangachar Sushma Uppala

Photo: Udai Pawar

Udai and Rahul with schoolchildren

Education in India
300M children aged 6-18 Household income <$5 per day Government spends <$100, per student, per year No end to difficulties…
Photo: Kentaro Toyama

Primary school in Tamil Nadu with minimal infrastructure

Sources: Ministry of Human Resource Development, Gov‟t of India, 2005-6, “Selected Educational Statistics”

Problems in Education
Child labour Parents uninvolved Teachers multitasking No toilets

Frequent maintenance needs of technology

No permanent building No textbooks Irrelevant curriculum No walls Intermittent electricity Poor pay for teachers No supplies Terrible student-teacher ratio UPS broken Heat Caste discrimination Teacher absenteeism Low attendance Teachers not Many children per computer computer literate Religious discrimination Student illness Students hungry

Photo: Kentaro Toyama

Young children not attending school in the middle of a weekday near Jhansi, Uttar Pradesh

Photo: Joyojeet Pal

Mid-day meal in Pondicherry

Mid-day meal in Ghana
Photo: Colleen Foley, Elisia Carlson

PCs in Schools
Strong anecdotal evidence that children attend school more, if they have an opportunity to interact with PCs.

• PC labs keep students occupied

• Excited by opportunity to interact with PCs

• Want children to learn about PCs

Governments and Administrators
• Eager to put PCs in schools • Constrained by limited budgets

Photo: Leba Haber

A Shanti Bhavan 6th grader, and potential computer engineer, with her mother

Sources: Pal, J., M. Lakshmanan, and K. Toyama, My Child Will be Respected': Parental Perspectives on Computers in Rural India, Proceedings of ICTD2007. Various field notes by U. S. Pawar, D. Hutchful, S. Panjwani, L. Micallef, K.Toyama, 2005-2008

Value of Computer Literacy
Evidence that entry-level white-collar jobs are possible with secondary education and PC literacy [data for India]
• PC literacy as proxy for other skills

• Increased confidence • Strong interest in white-collar jobs

IT Training Centers
• Consistent demand from young adults
Photo: Aishwarya Ratan

Office service staff at MSR India using a freely provided PC

Caveat: English ability and “soft skills” valued more than PC literacy

Sources: Ratan, A., Satpathy, S., Zia, L., Toyama, K., Blagsvedt, S., Pawar, U.S., Subramanian, T. Kelsa+: Digital Literacy for Low-Income Office Workers, to be published in Proceedings of ICTD2009. Discussions with Hope Foundation India, Microsoft Learning, MS Unlimited Potential Group, 2007-2008

Photos: Joyojeet Pal

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Non-Tech Solution

Source: Pal, J. Computer Aided Education in India: A survey of the Azim Premji Foundation‟s junior school initiatives, 2005.

Techno-Centric Solution
A $100 laptop for every child (now $200) Non-profit organization Bulk sales to governments Marketing, distribution, training, and support by governments or volunteers
One Laptop Per Child (OLPC): “$100” laptop

Learning by self-driven, constructivist paradigm

Source: One Laptop Per Child.

HW/SW is not the main cost!
Conservative, back-of-the-envelope calculations for actual costs per child, based on one laptop per child at $100 lasting 5 years, and looking at cost on a per-year basis, amortized over 5 years. Hardware/software Distribution $20 $25 $100 / 5 years Low estimate

“Losses” in distribution
Breakage, theft, unintended sale Connectivity and power System administration, maintenance Teacher training

$20 $15 $100 $50

Conservatively, 20%
e.g., 1 in 5 each year Low estimate = $10,000/yr / 100 kids

Maine laptop project cites 1/3 total cost for teacher training --------------------------------------------------------------------------------------------------------------------Total Indian gov‟t spend on public education $250 $1250 <$100 per child, per year cost per child, every five years per child, per year

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Photos: Joyojeet Pal


Provide a mouse (and cursor) for every student

Sample MultiPoint Game

The first MultiPoint alphabet-learning game

Technical Considerations
Basic approach:
• • • • • • Avoid kernel and driver modifications Hijack mouse-event callbacks Handle mouse commands separately for each mouse ID Hide regular cursor and redraw one cursor per mouse Package functionality as a dynamic link library Expose same programming model as for regular GUI programming

• • • Extra work to handle mice plug-in and unplug events “Lost” mouse events in some environments Doesn‟t apply immediately to most existing applications

Preliminary Trials
Everyone wants a mouse.

Young children understand MultiPoint immediately.
Before MultiPoint

All students more engaged for longer periods of time.

Children like it!

After MultiPoint

Formal Evaluation
Can students learn as much with MultiPoint, compared with singlemouse configurations?

What designs encourage more learning?
Photo: Udai Pawar

Children crowding around a laptop screen, using MultiPoint

What designs encourage collaboration?

Formal Evaluation

Choice of Task
Desired characteristics for evaluation task:
– Quantifiable and objective metrics for learning – Measurability in short term – Consistency regardless of degree of PC usage – Generalizability to many educational domains – Practical educational value – Comparability – allows “apples to apples” comparisons between multiple mice and single mouse

English vocabulary learning task
– Match images with words



Software Configurations

SS: Single User, Single Mouse
Learning phase = testing phase
– Learn by trial and error

Multiple choice questions
– Feedback on „correct‟ or „incorrect‟

Word delivery gradually introduces new words to maximize learning
Iterative design in the early preparatory phases
Photo: Udai Pawar

Software Configurations

MS: Multiple User, Single Mouse

Software exactly the same as SS! Five children share one PC and one mouse.

Photo: Udai Pawar

Software Configurations

MM-R: Multi-User, Multi-Mouse Racing
Competitive in nature Interactivity based on SS configuration Every child has own mouse, cursor, and equal on-screen capability. Screen change occurs as soon as one player clicks on correct answer.

Photo: Udai Pawar

Software Configurations

MM-V: Multi-User, Multi-Mouse Voting
Collaborative in nature Interactivity allows multiple students to click on the same button. Every child has own mouse, cursor, and equal on-screen capability. Screen change occurs only if all players click on correct answer.

Photo: Udai Pawar

Formal Evaluation

Summary of Configurations
SS: Single user, single mouse MS: Multiple user, single mouse MM: Multiple user, multiple mouse
• MM-R: MM racing (competitive) configuration • MM-V: MM voting (collaborative) configuration

Note: All configurations reduce to SS when there is only one student.

Formal Evaluation

Experimental Set-Up
Four configurations:
– – – – SS MS MM-R MM-V

Randomized assignment to configurations
– 7 minutes pre-test – 30 minutes PC usage – 7 minutes post-test

– – – – 11-12 yrs; 6-7th grades Very basic English ability Some exposure to PCs Rural government schools

– Change in vocabulary – All on-screen activity

Subject grouping:
– Mixed groups (some all male, some all female) of 5 each – 238 subjects total

All comments recorded; some trials video-recorded

Formal Evaluation

Quantitative Results
Number of words learned under MM-V roughly the same as with SS (no statistically significant difference) MM-V unique among non-SS configurations in showing equal learning
Average No. of Words Learnt

3 2.5

4.11 4.3


3.7 2.93 2.8

4.53 4.4

MS okay, but not with boys


3.76 3.6


Strong gender effects:

3 1.5 2

– All-girl groups do better in all 1 0.5 multiple person configurations. 0 0 – Boys learn much less in BOYS GIRLS 1 ALL STUDENTS25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 5 9 13 17 21 competitive scenarios; AverageRate of clicks overlearned during PC usage number of words time (blue line), rampant clicking.
for one group of boys in MM-R configuration



Formal Evaluation

Qualitative Observations
On the whole, more positive collaboration with multiple mice. Engagement and interest greatest in MMR, then MM-V, then MS. Interactions not always positive:
– E.g., “I‟ll kill you if you don‟t click”

Dominance roles not eliminated – linked to initiative and knowledge legitimacy Role of teacher/supervisor still critical
– – Handling disruptive behavior Content creation or matching to curriculum

Photo: Udai Pawar

Spontaneous “cross-mouse” usage

Moed, A., Otto, O., Pal, J., Pawar, U.S., Kam, M., and Toyama, K. (2009) Reducing Dominance Behavior in Multiple-Mouse Learning Activities, to be published in Proceedings of Computer Supported Collaborative Learning CSCL 2009.
Pawar, U.S., Pal, J., Gupta. R., and Toyama, K. (2007) Multiple Mice for Retention Tasks in Disadvantaged Schools, In Proceedings of ACM CHI 2007. Pal, J., Pawar, U.S., Brewer, E., and Toyama, K. (2006) The case for multi-user design for computer aided learning in developing regions, In Proceedings of WWW 2006. Pawar, U. S., Pal, J., and Toyama, K. (2006) Multiple mice for computers in education in developing countries, In Proceedings of IEEE/ACM Int’l Conf. on Information & Communication Technologies for Development, ICTD 2006. Pawar, U.S., Pal, J., Uppala, S., and Toyama, K. (2006) Effective Educational Delivery in Rural Computer Aided Education: Multimouse. In Proceedings of Digital Learning DL 2006.

Beyond Research
Microsoft MultiPoint SDK 1.0 released June, 2007
Regular use in ~170 schools
– – – – Thailand: 140 schools Vietnam: 20 schools Philippines: 5 schools Malaysia: 3 pilots + content development – Chile: 1 pilot + content development – Indonesia: content dev

Ongoing marketing and product development (SDK 1.1 coming soon)

Related Work
Multiple Mice
• Bier (1991), Hourcade (1999)
– – Technical issues of multiple mice “Single Display Groupware”

Computers for Education in Developing World
• Mitra (1999)
– “Hole in the Wall” – free access to PCs for children in low-income areas Self-taught computer literacy

Inkpen et al. (1995)
– – 2-student education scenario Cursor control toggles between two mice

Azim Premji Foundation (2002)
– – Computer labs in primary schools Culture-localizable educational games

Bricker (1998)
– – 3-person collaborations Color-matching task

Negroponte et al. (2004)
– – One Laptop Per Child Low-cost laptops and constructivist education

• •

Druin, Bederson, et al. (2003)
– 2-person browsing task

Greenberg et al. (2004)
– Multiple mice for collaborative work

AstraLab (2007)
– – PC and projector Multimedia content pre-loaded

MultiPoint Characteristics
Simple solution for a real need Cost and “sustainability”
– One computer + 5 mice equals ~$100 per child – If PCs exist, more mice is easy

Stakeholder alignment
– – – – – Government / administrators Teachers Students Parents Content creators

– Rich avenue for further exploration

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Further Research with MultiPoint

Mitigating “Dominance” Behavior
Work by Andrea Moed, Owen Otto, Joyojeet Pal, Matthew Kam, Udai Pawar, Kentaro Toyama Can we combine the best aspects of competitive and cooperative play through team games? Challenges:
– – – Mouse as a text-entry device Restricted screen real estate Occlusion among cursors

Status: studies completed; paper accepted to CSCL 2009

Further Research with MultiPoint

Whole-Class MultiPoint
Ongoing work by Miguel Nussbaum, Heinz Susaeta, Kentaro Toyama; related efforts by Neema Moraveji, Taemie Kim

What kinds of educational games can be effective for 20-40 children and multiple mice? Challenges:
– – – Restricted screen real estate Varying distance to screen Pedagogical model

Photo: Miguel Nussbaum

Status: Prototypes built; studies in Chile begun; planning comparative studies in India

Further Research with MultiPoint

Text Entry
Ongoing work by Saleema Amershi, Merrie Morris, Neema Moraveji, Ravin Balakrishnan, Kentaro Toyama What‟s the best way to enter text in multiple-mouse scenarios? Challenges:
– – – Mouse as a text-entry device Restricted screen real estate Occlusion among cursors
Screenshot: Saleema Amershi

Status: studies completed; writing paper

Further Research with MultiPoint

Collaborative Activities
Work by Abigail Cauchi, Jennifer Fenech, Kentaro Toyama
How should non-competitive activities be designed to maximize collaboration? Challenges:
– –
Screenshot: Abigail Cauchi Photo: Miguel Nussbaum

Weaker goals, softer metrics Different classroom “cultures” and student personalities

Status: Prototypes built; studies in three different schools in completed; writing papers

Further Research with MultiPoint

Considering the Teacher
Work by David Hutchful, Ashish Sharma, Kentaro Toyama How can we improve the experience for teachers interested in using MultiPoint? Challenges:
– – – Teachers want to customize content Limited time for class preparation Moderate PC literacy among teachers
Photo: Saurabh Panjwani

Potential solution: Encapsulate interaction into templates, while making content trivially editable Status: prototypes being built

Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Continuum of PC Sharing
Shared processor, monitor & keyboard Shared PC Shared processor & monitor

Nothing shared

Shared processor

True personal computer

Personal mouse, keyboard & monitor (Multi-console, Thin client)

Nothing personal

Personal mouse (MultiPoint)

Personal mouse & keyboard (Split Screen)

Split Screen
Two users, two mice, two keyboards, two instances of the desktop, but only one monitor

Are there problems with reduced screen size, distraction, or ergonomics? What sort of collaborative behaviors occur naturally?
Photo: Kentaro Toyama

Two young adults learning with Split Screen

What sort of collaborative behaviors can be encouraged?

Other CS Opportunities
in Education for the Developing World

“Featherweight Computing”
– Ultra-low-cost electronics for multimedia content
Not in school Occasional attendance Regular attendance Secondary school Formal work

Modeling of educational states and transitions
– Stochastic modeling applied to human states – Links between micro- and macro-scale understanding – Collaborations with cognitive psychologists and economists

Digital Slates in Microcredit
Technology for easing the burden of digitizing records in microfinance transactions

Secure Mobile Banking
Security for mobile banking, especially where transmission channels are flakey

Accent-Robust Speech
Speech recognition that is robust to differences and accents and dialects

Embedded Systems

Cryptography and Security

Speech Recognition

Paper-and-Digital Forms
Tools to support generation of easy-to-use forms that can also be easily digitized

SMS Server Toolkit
Information systems that deliver content over SMS textmessaging

„Tooning for Text-Free UIs
Creating cartoons from photographs to support creation of UIs for the non-literate

Machine Learning, Vision, HCI

Mobility and Systems

Vision and Graphics

Increasing Online Donations Gaudy Photo Editing
Can sites such as increase online donations through design tweaks? Photo-editing tools designed for a culturespecific aesthetics

Cost-Aware Data Transfer
rem aini ng dat a


p1 p2
0 t

Cost-aware transfer of data across heterogeneous channels, e.g., for mobiles

HCI, Social Computing

Computer Vision


Introduction Case Study
– – – – Problem Context Two Possibilities? Solution and Evaluation Ongoing Research

An Appeal to Computer Science Conclusion

Qualitative Social Science Design Computer Science & Engineering Typical Strengths
Micro-scale understanding; attention to human issues Adaptive innovation; parsimonious design Technology; problem-solving optimism

Typical Weaknesses
Bias towards critique; weak understanding of technology Technologylimited; subjective evaluation Techno-centrism; blindness to nontechnology issues

Role in MultiPoint
Identification of problem



Solution concept

Technical innovation

Software innovation

Quantitative Social Science

Randomized control trial

Evaluation and cost-benefit analysis

Black-box analysis; weak understanding of technology

Confirmation of benefit


…from computer science and engineering research

Openness to true, multidisciplinary effort

Willingness to start with simple solutions
Persistence in identifying technical challenges Flexibility about matching problems with technologies

Realism about potential impact


…from non-technology disciplines

Openness to true, multidisciplinary effort

Forgiveness for technologists doing self-evaluations
Constructive critiques of potential or actual impact Guidance on human issues of technology deployment

Pragmatism and emphasis on primary explanations

Technical Research

Global Development Multidisciplinary Approach

Thank you!