Kentaro Toyama - Computer Science Research For Global Development

Computer Science Research for
Global Development
Kentaro Toyama
Assistant Managing Director
Microsoft Research India
University of Washington
March 12, 2009 – Seattle
Technical Research
Global Development
Multidisciplinary Approach
Outline
Introduction
Case Study
– Problem Context
– Two Possibilities?
– Solution and Evaluation
– Ongoing Research
An Appeal to Computer Science
Conclusion
Outline
Introduction
Case Study
– Problem Context
Conclusion
Photo: courtesy of Infosys
Infosys campus, Bangalore, India
Photo: Nimmi Rangaswamy
A small Internet café on a market street in a town near Bombay
Photo: Kentaro Toyama
Rural village with a VSAT Internet connection near Bhopal, Madhya Pradesh
“Kids in the developing world need the
newest technology, especially really rugged
hardware and innovative software.”
– Nicholas Negroponte (OLPC website, 2005)
“The world's poorest two billion people

desperately need healthcare, not laptops.”
– Bill Gates (WRI Conference, Seattle, 2000)
Technology for Emerging Markets
Microsoft Research India
Understand potential technology

users in developing communities
Design and evaluate technology

and systems that meet needs and
aspirations of potential users
Impact communities worldwide

Computer-skills camp in Nakalabande, Bangalore through partnerships with Microsoft
(MSR India, Stree Jagruti Samiti, St. Joseph‟s College)
groups and non-Microsoft
organizations
Multidisciplinary Research
Technology
Aishwarya Lakshmi Ratan for microfinance
Social science
– International Development
Public Administration Mobile phones in
Jonathan Donner Mobile developing countries
– Communications banking
PCs usage under
Nimmi Rangaswamy free access
– Social Anthropology
Technology
in slums
Indrani Medhi Middle-class
consumption
Design
– Design
Mobile-phone
David Hutchful interfaces UIs for
– Human Computer Interaction
non-literate users
Kentaro Toyama (Group Lead) Telecentres
– Computer Science Computers
Technology
Technology in education
Saurabh Panjwani
in healthcare
– Computer Science DVD
Bill Thies Technology multimedia
– Computer Science for agriculture
Rikin Gandhi Video and
– Astrophysics
mediated instruction
Sample Projects
Collaborations with UW Research on
annotated video
(Natalie Linnell,
Warana Unwired Richard Anderson)
Digital Green
Mobile phones
and young adults User studies
Technology for
(Carolyn Wei) with CAM Microfinance
Mobiles in the (Tapan Parikh)
Developing World
Support through
Advising on IPAI incubation period
research grant (Tom Anderson,
Virtual keyboards for (Chris Coward) Paul Javid,
MultiPoint Rural Kiosks Kurtis Heimerl)
(Saleema Amershi)
Digital StudyHall
MultiPoint
Outline
Introduction
Case Study
– Problem Context
Conclusion
Collaborators
– Saleema Amershi
– Sukumar Anikar
– Ravin Balakrishnan
– Abigail Cauchi
– Jennifer Fenech
– Rahul Gupta
– David Hutchful
– Divya Kumar
– Andrea Moed
– Neema Moraveji
– Merrie Morris
– Miguel Nussbaum
– Owen Otto
– Joyojeet Pal
Photo: Udai Pawar
– Udai Singh Pawar
Udai and Rahul with schoolchildren
– Bhagya Rangachar
– Sushma Uppala
Education in India
300M children aged 6-18
Household income <$5 per day
Government spends <$100,

per student, per year
No end to difficulties…
Primary school in Tamil Nadu
with minimal infrastructure
Sources: Ministry of Human Resource Development, Gov‟t of India, 2005-6, “Selected Educational Statistics”
Problems in Education
Child labour Parents uninvolved Teachers multitasking
Frequent maintenance
No toilets
needs of technology
No permanent building No textbooks
Irrelevant curriculum
No walls
Poor pay for teachers Intermittent electricity
No supplies
Terrible student-teacher ratio
UPS broken
Heat
Caste discrimination
Teacher absenteeism
Low attendance
Teachers not
Many children per computer computer literate
Religious discrimination
Student illness
Students hungry
Young children not attending school in the middle of a weekday near Jhansi, Uttar Pradesh
Photo: Joyojeet Pal
Mid-day meal in Pondicherry
Mid-day meal in Ghana

Photo: Colleen Foley, Elisia Carlson
PCs in Schools
Strong anecdotal evidence that children
attend school more, if they have an
opportunity to interact with PCs.
Teachers
• PC labs keep students occupied
Children
• Excited by opportunity to interact with PCs
Parents
• Want children to learn about PCs
Photo: Leba Haber
Governments and Administrators
A Shanti Bhavan 6th grader, and potential
• Eager to put PCs in schools
computer engineer, with her mother
• Constrained by limited budgets
Sources: Pal, J., M. Lakshmanan, and K. Toyama, My Child Will be Respected':

Parental Perspectives on Computers in Rural India, Proceedings of ICTD2007.
Various field notes by U. S. Pawar, D. Hutchful, S. Panjwani, L. Micallef, K.Toyama, 2005-2008
Value of Computer Literacy
Evidence that entry-level white-collar
jobs are possible with secondary
education and PC literacy [data for India]
Employers
• PC literacy as proxy for other skills
Employees
• Increased confidence
• Strong interest in white-collar jobs
IT Training Centers
• Consistent demand from young adults
Photo: Aishwarya Ratan
Office service staff at MSR India using
a freely provided PC
Caveat: English ability and “soft skills”
valued more than PC literacy
Sources: Ratan, A., Satpathy, S., Zia, L., Toyama, K., Blagsvedt, S., Pawar, U.S., Subramanian, T. Kelsa+:
Digital Literacy for Low-Income Office Workers, to be published in Proceedings of ICTD2009.
Discussions with Hope Foundation India, Microsoft Learning, MS Unlimited Potential Group, 2007-2008
Photos: Joyojeet Pal
Outline
Introduction
Case Study
– Problem Context
Conclusion
Non-Tech Solution
Source: Pal, J. Computer Aided Education in India: A survey of the Azim Premji Foundation‟s junior school initiatives, 2005.
http://research.microsoft.com/en-us/um/india/projects/computeraidedlearningsurvey/Presentation.ppt
Techno-Centric Solution
A $100 laptop for every child

(now $200)
Non-profit organization
Bulk sales to governments
Marketing, distribution, training,

and support by governments or
volunteers
Learning by self-driven,
One Laptop Per Child (OLPC): “$100” laptop constructivist paradigm
Source: One Laptop Per Child. http://laptop.org/en/

HW/SW is not the main cost!
Conservative, back-of-the-envelope calculations for actual costs per child,
based on one laptop per child at $100 lasting 5 years, and looking at cost on a per-year
basis, amortized over 5 years.
Hardware/software $20 $100 / 5 years
Distribution $25 Low estimate
“Losses” in distribution $20 Conservatively, 20%
Breakage, theft, unintended sale $20 e.g., 1 in 5 each year
Connectivity and power $15 Low estimate
System administration, maintenance $100 = $10,000/yr / 100 kids
Teacher training $50 Maine laptop project cites 1/3

total cost for teacher training
---------------------------------------------------------------------------------------------------------------------
Total $250 per child, per year cost

$1250 per child, every five years
Indian gov‟t spend on public education <$100 per child, per year
Outline
Introduction
Case Study
– Problem Context
Conclusion
Photos: Joyojeet Pal
MultiPoint
Provide a mouse (and cursor) for every student

Sample MultiPoint Game
The first MultiPoint alphabet-learning game

Technical Considerations
Basic approach:
• Avoid kernel and driver

modifications
• Hijack mouse-event callbacks
• Handle mouse commands
separately for each mouse ID
• Hide regular cursor and redraw Issues:
one cursor per mouse
• Package functionality as a • Extra work to handle mice plug-in
dynamic link library and unplug events
• Expose same programming model • “Lost” mouse events in some
as for regular GUI programming environments
• Doesn‟t apply immediately to most
existing applications
Preliminary Trials
Everyone wants a mouse.
Young children understand MultiPoint

immediately.
Before MultiPoint
All students more engaged for longer

periods of time.
Children like it!
After MultiPoint
Formal Evaluation
Questions:
Can students learn as much with

MultiPoint, compared with single-
mouse configurations?
What designs encourage more

learning?
Photo: Udai Pawar
Children crowding around a laptop

screen, using MultiPoint What designs encourage
collaboration?
Formal Evaluation
Choice of Task
Desired characteristics for English vocabulary learning task
evaluation task:
– Match images with words
– Quantifiable and objective
metrics for learning
– Measurability in short term
– Consistency regardless of
degree of PC usage “bull”
– Generalizability to many
educational domains
– Practical educational value
– Comparability – allows “apples “tiger”

to apples” comparisons between
multiple mice and single mouse
Software Configurations
SS: Single User, Single Mouse
Learning phase = testing phase

– Learn by trial and error
Multiple choice questions

– Feedback on „correct‟ or „incorrect‟
Word delivery gradually introduces new

words to maximize learning
Iterative design in the early preparatory

phases
Photo: Udai Pawar

MS: Multiple User, Single Mouse
Software exactly the same as SS!
Five children share one PC and one

mouse.
Photo: Udai Pawar

MM-R: Multi-User, Multi-Mouse Racing
Competitive in nature
Interactivity based on SS configuration
Every child has own mouse, cursor, and

equal on-screen capability.
Screen change occurs as soon as one

player clicks on correct answer.
Photo: Udai Pawar

MM-V: Multi-User, Multi-Mouse Voting
Collaborative in nature
Interactivity allows multiple students to

click on the same button.
Every child has own mouse, cursor, and

equal on-screen capability.
Screen change occurs only if all players

click on correct answer.
Photo: Udai Pawar

Formal Evaluation
Summary of Configurations
SS: Single user, single mouse
MS: Multiple user, single mouse
MM: Multiple user, multiple mouse

• MM-R: MM racing (competitive) configuration
• MM-V: MM voting (collaborative) configuration
Note: All configurations reduce to SS when

there is only one student.
Formal Evaluation
Experimental Set-Up
Four configurations: Randomized assignment to
– SS configurations
– MS
– MM-R
Task:
– MM-V
– 7 minutes pre-test
– 30 minutes PC usage
Subjects:
– 11-12 yrs; 6-7th grades – 7 minutes post-test
– Very basic English ability
– Some exposure to PCs Measured:
– Rural government schools – Change in vocabulary
– All on-screen activity
Subject grouping:
– Mixed groups (some all male, All comments recorded; some
some all female) of 5 each
trials video-recorded
– 238 subjects total
Formal Evaluation
Quantitative Results
Number of words learned under MM-V roughly the same as with SS
(no statistically significant difference)
MM-V unique among non-SS

configurations in showing 3
equal learning
5 2.5 MM-R SS MS
4.56
MM-R MM-V
4.5 4.53 4.4
Average No. of Words Learnt

4.3
4.11 4.1
MS okay, but not with boys 4 2 3.76 3.6 3.7
2.93 2.8
3 1.5
Strong gender effects:
2 1
MM-R
– All-girl groups do better in all
MM-V
1 0.5
MS
SS
multiple person configurations.
0 0
– Boys learn much less in
1 ALL
5 STUDENTS BOYS
9 13 17 21 25 29 33 37 41 GIRLS
45 49 53 57 61 65 69 73 77 81 85
competitive scenarios;
rampant clicking. AverageRate number of words
of clicks overlearned
time (blue during
line),PC usage
for one group of boys in MM-R configuration
Formal Evaluation
Qualitative Observations
On the whole, more positive collaboration
with multiple mice.
Engagement and interest greatest in MM-

R, then MM-V, then MS.
Interactions not always positive:

– E.g., “I‟ll kill you if you don‟t click”
Dominance roles not eliminated – linked to Photo: Udai Pawar

initiative and knowledge legitimacy Spontaneous “cross-mouse” usage
Role of teacher/supervisor still critical

– Handling disruptive behavior
– Content creation or matching to curriculum
Publications
Moed, A., Otto, O., Pal, J., Pawar, U.S., Kam, M., and Toyama, K. (2009) Reducing
Dominance Behavior in Multiple-Mouse Learning Activities, to be published in
Proceedings of Computer Supported Collaborative Learning CSCL 2009.
Pawar, U.S., Pal, J., Gupta. R., and Toyama, K. (2007) Multiple Mice for Retention Tasks
in Disadvantaged Schools, In Proceedings of ACM CHI 2007.
Pal, J., Pawar, U.S., Brewer, E., and Toyama, K. (2006) The case for multi-user design
for computer aided learning in developing regions, In Proceedings of WWW 2006.
Pawar, U. S., Pal, J., and Toyama, K. (2006) Multiple mice for computers in education in
developing countries, In Proceedings of IEEE/ACM Int’l Conf. on Information &
Communication Technologies for Development, ICTD 2006.
Pawar, U.S., Pal, J., Uppala, S., and Toyama, K. (2006) Effective Educational Delivery in
Rural Computer Aided Education: Multimouse. In Proceedings of Digital Learning DL
2006.
Beyond Research
Microsoft MultiPoint SDK 1.0 released
June, 2007
Regular use in ~170 schools

– Thailand: 140 schools
– Vietnam: 20 schools
– Philippines: 5 schools
– Malaysia: 3 pilots + content
development
– Chile: 1 pilot + content
development
– Indonesia: content dev
Ongoing marketing and product

development (SDK 1.1 coming soon)
Related Work
Multiple Mice Computers for Education in
Developing World
• Bier (1991), Hourcade (1999)
– Technical issues of multiple mice • Mitra (1999)
– “Single Display Groupware” – “Hole in the Wall” – free access to
PCs for children in low-income areas
• Inkpen et al. (1995) – Self-taught computer literacy
– 2-student education scenario
– Cursor control toggles between two • Azim Premji Foundation (2002)
mice – Computer labs in primary schools
– Culture-localizable educational
• Bricker (1998) games
– 3-person collaborations
– Color-matching task • Negroponte et al. (2004)
– One Laptop Per Child
• Druin, Bederson, et al. (2003) – Low-cost laptops and constructivist
– 2-person browsing task education
• AstraLab (2007)
• Greenberg et al. (2004)
– PC and projector
– Multiple mice for collaborative work
– Multimedia content pre-loaded
MultiPoint Characteristics
Simple solution for a real need
Cost and “sustainability”

– One computer + 5 mice equals
~$100 per child
– If PCs exist, more mice is easy
Stakeholder alignment
– Government / administrators
– Teachers
– Students
– Parents
– Content creators
Research
– Rich avenue for further exploration
Outline
Introduction
Case Study
– Problem Context
Conclusion
Further Research with MultiPoint
Mitigating “Dominance” Behavior

Work by Andrea Moed, Owen Otto,
Joyojeet Pal, Matthew Kam, Udai
Pawar, Kentaro Toyama
Can we combine the best aspects of

competitive and cooperative play
through team games?
Challenges:
– Mouse as a text-entry device
– Restricted screen real estate
– Occlusion among cursors
Status: studies completed; paper

accepted to CSCL 2009
Whole-Class MultiPoint
Ongoing work by Miguel Nussbaum,
Heinz Susaeta, Kentaro Toyama;
related efforts by Neema Moraveji,
Taemie Kim
What kinds of educational games can

be effective for 20-40 children and
multiple mice?
Challenges:
– Varying distance to screen
Photo: Miguel Nussbaum – Pedagogical model
Status: Prototypes built; studies in

Chile begun; planning comparative
studies in India
Text Entry
Ongoing work by Saleema Amershi,
Merrie Morris, Neema Moraveji, Ravin
Balakrishnan, Kentaro Toyama
What‟s the best way to enter text in

multiple-mouse scenarios?
Challenges:
– Mouse as a text-entry device
– Occlusion among cursors
Screenshot: Saleema Amershi
Status: studies completed; writing

paper
Collaborative Activities
Work by Abigail Cauchi, Jennifer
Fenech, Kentaro Toyama
How should non-competitive activities

be designed to maximize
collaboration?
Challenges:
– Weaker goals, softer metrics
– Different classroom “cultures” and
student personalities
Screenshot: Abigail Cauchi
Photo: Miguel Nussbaum
Status: Prototypes built; studies in
three different schools in completed;
writing papers
Considering the Teacher

Work by David Hutchful, Ashish
Sharma, Kentaro Toyama
How can we improve the experience

for teachers interested in using
MultiPoint?
Challenges:
– Teachers want to customize content
– Limited time for class preparation
– Moderate PC literacy among teachers
Photo: Saurabh Panjwani

Potential solution: Encapsulate
interaction into templates, while
making content trivially editable
Status: prototypes being built

Outline
Introduction
Case Study
– Problem Context
Conclusion
Continuum of Nothing
shared
PC Sharing Shared
processor
Shared
Shared processor &
processor, monitor True
monitor & personal
keyboard computer
Shared PC
Personal
mouse,
keyboard
& monitor
(Multi-console,
Thin client)
Personal
mouse & keyboard
(Split Screen)
Personal
Nothing mouse
personal (MultiPoint)
Split Screen
Two users, two mice, two
keyboards, two instances of the
desktop, but only one monitor
Are there problems with reduced

screen size, distraction, or
ergonomics?
What sort of collaborative

behaviors occur naturally?

What sort of collaborative
Two young adults learning with Split Screen
behaviors can be encouraged?
Other CS Opportunities
in Education for the Developing World
“Featherweight Computing”
Not in Occasional Regular Secondary Formal
school attendance attendance school work
– Ultra-low-cost electronics for
multimedia content
Modeling of educational states

and transitions
– Stochastic modeling applied to

human states
– Links between micro- and
macro-scale understanding
– Collaborations with cognitive
psychologists and economists
Digital Slates in Microcredit Secure Mobile Banking Accent-Robust Speech
Technology for
easing the Security for mobile Speech recognition
burden of banking, especially that is robust to
digitizing records where transmission differences and
in microfinance channels are flakey accents and
transactions dialects
Embedded Systems Cryptography and Security Speech Recognition
Paper-and-Digital Forms SMS Server Toolkit „Tooning for Text-Free UIs

Tools to support Creating
generation of Information
systems that cartoons from
easy-to-use photographs to
forms that can deliver content
over SMS text- support creation
also be easily of UIs for the
digitized messaging
non-literate
Machine Learning, Vision, HCI Mobility and Systems Vision and Graphics
Increasing Online Donations Gaudy Photo Editing Cost-Aware Data Transfer

Can sites such Photo-editing rem
aini
Cost-aware
ng
as Kiva.org tools dat
a
transfer of data
increase online designed for a 1 across
p1
donations culture- heterogeneous
through design specific p2 t channels, e.g.,
0
tweaks? aesthetics for mobiles
HCI, Social Computing Computer Vision Networking

Outline
Introduction
Case Study
– Problem Context
Conclusion
Multidisciplinarity
Typical Typical Role in
Methodology
Strengths Weaknesses MultiPoint
Qualitative Micro-scale Bias towards

understanding; critique; weak Identification of
Social Ethnography attention to understanding of problem
Science human issues technology
Adaptive
Technology-
innovation; Solution
Design Design parsimonious
limited; subjective
concept
evaluation
design
Computer Technology; Techno-centrism;

Technical Software
Science & innovation
problem-solving blindness to non-
innovation
Engineering optimism technology issues
Quantitative Black-box
Evaluation and
Randomized analysis; weak Confirmation of
Social control trial
cost-benefit
understanding of benefit
Science analysis
technology
Wanted
…from computer science and engineering research
Openness to true, multidisciplinary effort
Willingness to start with simple solutions
Persistence in identifying technical challenges
Flexibility about matching problems with technologies
Realism about potential impact

Wanted
…from non-technology disciplines
Openness to true, multidisciplinary effort
Forgiveness for technologists doing self-evaluations
Constructive critiques of potential or actual impact
Guidance on human issues of technology deployment
Pragmatism and emphasis on primary explanations

Technical Research
Global Development
Multidisciplinary Approach
Thank you!
http://research.microsoft.com/research/tem
kentoy@microsoft.com

Kentaro Toyama - Computer Science Research For Global Development

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Kentaro Toyama - Computer Science Research For Global Development

Uploaded by

Copyright:

Available Formats

Computer Science Research for

An Appeal to Computer Science

An Appeal to Computer Science

“The world's poorest two billion people

Understand potential technology

Design and evaluate technology

Impact communities worldwide

An Appeal to Computer Science

300M children aged 6-18

Household income <$5 per day

Government spends <$100,

Mid-day meal in Pondicherry

Mid-day meal in Ghana

Sources: Pal, J., M. Lakshmanan, and K. Toyama, My Child Will be Respected':

An Appeal to Computer Science

A $100 laptop for every child

Bulk sales to governments

Marketing, distribution, training,

Source: One Laptop Per Child. http://laptop.org/en/

Hardware/software $20 $100 / 5 years

Distribution $25 Low estimate

“Losses” in distribution $20 Conservatively, 20%

Breakage, theft, unintended sale $20 e.g., 1 in 5 each year

Connectivity and power $15 Low estimate

System administration, maintenance $100 = $10,000/yr / 100 kids

Teacher training $50 Maine laptop project cites 1/3

Total $250 per child, per year cost

An Appeal to Computer Science

Provide a mouse (and cursor) for every student

The first MultiPoint alphabet-learning game

• Avoid kernel and driver

Everyone wants a mouse.

Young children understand MultiPoint

All students more engaged for longer

Children like it!

Can students learn as much with

What designs encourage more

Children crowding around a laptop

– Comparability – allows “apples “tiger”

SS: Single User, Single Mouse

Learning phase = testing phase

Multiple choice questions

Word delivery gradually introduces new

Iterative design in the early preparatory

Photo: Udai Pawar

MS: Multiple User, Single Mouse

Software exactly the same as SS!

Five children share one PC and one

Photo: Udai Pawar

MM-R: Multi-User, Multi-Mouse Racing

Interactivity based on SS configuration

Every child has own mouse, cursor, and

Screen change occurs as soon as one

Photo: Udai Pawar

MM-V: Multi-User, Multi-Mouse Voting

Interactivity allows multiple students to

Every child has own mouse, cursor, and

Screen change occurs only if all players

Photo: Udai Pawar

MS: Multiple user, single mouse

MM: Multiple user, multiple mouse

Note: All configurations reduce to SS when

MM-V unique among non-SS