Human Computation

CSC4170 Web Intelligence and Social Computing

Tutorial 7

Tutor: Tom Chao Zhou

Email: czhou@cse.cuhk.edu.hk
Outline

 Definition & Early Work

 Human Computation:
 Case Study: reCAPTCHA
 Game with a purpose:
 Case Study: ESP Game
 Q&A
 Reference
Definition & Early Work

 Human-based Computation:
 Human-based Computation is a computer science technique in
which a computational process performs its function by
outsourcing certain steps to humans.
 This approach leverages differences in abilities and alternative
costs between humans and computer agents to achieve
symbiotic human-computer interaction.
Definition & Early Work

 Early Work:
 Interactive evolutionary computation.
 The idea is proposed by Richard Dawkins (http://en.wikipedia.or
g/wiki/Richard_Dawkins).
 The preference of a human experimenter is used to guide the ev
olution of two-dimensional sets of line segments.
 This program asks a human to be the fitness function of an evolu
tionary algorithm, so that the algorithm can use human visual per
ception and aesthetic judgment to do something that a normal ev
olutionary algorithm cannot do.
Human Computation Case Study: reCAPTC
HA
 What is CAPTCHA?
 A CAPTCHA (completely automated public Turing test to tell
computers and humans apart) is a program that can tell whether
its user is a human or a computer.
 Colorful images with distorted text at the bottom of Web
registration forms
Human Computation Case Study: reCAPTC
HA
 What is CAPTCHA?
 It was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas H
opper and John Langford of Carnegie Mellon University.
 Applications of CAPTCHA:
 Preventing Comment Spam in Blogs.
 Protecting Website Registration.
 Online Polls.
 Preventing Dictionary Attacks. CAPTCHAs can also be used to prev
ent dictionary attacks in password systems.
 Search Engine Bots. Preventing search engine bots to crawl the pag
es.
Human Computation Case Study: reCAPTC
HA
 Statistics about CAPTCHA and some thought.
 About 200 million CAPTCHAs are solved by humans around the
world every day.
 In each case, roughly ten seconds of human time are being spen
t.
 Individually, that's not a lot of time, but in aggregate these little p
uzzles consume more than 150,000 hours of work each day.
 What if we could make positive use of this human effort?
Human Computation Case Study: reCAPTC
HA
 What is reCAPTCHA?
 reCAPTCHA does exactly that by channeling the effort spent sol
ving CAPTCHAs online into "reading" books.
 reCAPTCHA is a free CAPTCHA service that helps to digitize bo
oks, newspapers and old time radio shows.
Human Computation Case Study: reCAPTC
HA
 What is reCAPTCHA?
 The old books are being photographically scanned, and then tra
nsformed into text using "Optical Character Recognition" (OCR).
 However, OCR produces images, which cannot be searched.
 reCAPTCHA improves the process of digitizing books by sendin
g words that cannot be read by computers to the Web in the for
m of CAPTCHAs for humans to decipher.
Human Computation Case Study: reCAPTC
HA
 Ideas of reCAPTCHA:
 Each new word that cannot be read correctly by OCR is given to
a user in conjunction with another word for which the answer is a
lready known.
 The user is then asked to read both words. If they solve the one
for which the answer is known, the system assumes their answe
r is correct for the new one.
 The system then gives the new image to a number of other peop
le to determine, with higher confidence, whether the original ans
wer was correct.
Human Computation Case Study: reCAPTC
HA
 Performances of reCAPTCHA:
 reCAPTCHA is able to achieve over 99.5% transcription accurac
y at the word level.
Game with a purpose (GWAP) Case Study:
ESP Game
 GWAP:
 A game with a purpose (GWAP) is a game played on a compute
r that serves some purpose for the people setting up the game b
y harnessing human abilities in an entertaining setting.
 There are many activities that are almost trivial for humans to pe
rform that have stymied computers, e.g., labelling image.
Game with a purpose (GWAP) Case Study:
ESP Game
 ESP Game:
 An effort in human computation originally conceived by Luis Von
Ahn of CMU to label images.
 Two players attempt to assign the same labels to an image.
 The game records the results of matches as image labels and th
e players enjoy the encounter because of the competitive and ti
med nature of it.
 To ensure that people do their best to accurately label the image
s, the game requires two people who have only the image in co
mmon, to choose the same word as an image label.
Game with a purpose (GWAP) Case Study:
ESP Game
 ESP Game:
Game with a purpose (GWAP) Case Study:
ESP Game
 ESP Game:
 Demo:
 http://www.gwap.com/gwap/gamesPreview/espgame/
 Play as a guest.
 Questions?
Reference

 reCAPTCHA:
 http://recaptcha.net/
 http://www.cs.cmu.edu/~biglou/captcha_cacm.pdf
 http://www.cs.cmu.edu/~biglou/captcha_crypt.pdf
 http://recaptcha.net/reCAPTCHA_Science.pdf
 ESP Game:
 http://www.gwap.com/gwap/gamesPreview/espgame/
 Homepage of Luis Von Ahn
 http://www.cs.cmu.edu/~biglou/