You are on page 1of 132


ACM OF THE 08/2010VoL.53No.08

The Singularity
Memory Models
Predicting the Popularity
Of Online Content
CTOs on Network
Has China Caught Up in IT?
Mechanism Design
Meets CS

Association for
Computing Machinery

Springer eBooks
Supporting You in Your Research

7 9,800 eBooks in Computer Science

7 More published every day
7 Unrestricted printing and
7 Unlimited access to
reference works,
textbooks, monographs,


communications of the acm

Departments News Viewpoints

5 JACM Editor’s Letter 11 Mechanism Design 24 Economic and Business Dimensions

JACM at the Start of a New Decade Meets Computer Science Is the Internet a Maturing Market?
By Victor Vianu A field emerging from economics is If so, what does that imply?
teaming up with computer science By Christopher S. Yoo
6 In the Virtual Extension to improve auctions, supply chains,
and communication protocols. 27 Education
7 Letters To The Editor By Gary Anthes Preparing Computer Science
CS Expertise for Institutional Students for the Robotics Revolution
Review Boards 14 Looking Beyond Robotics will inspire dramatic
Stereoscopic 3D’s Revival changes in the CS curriculum.
8 BLOG@CACM Researchers working in vision and By David S. Touretzky
The War Against Spam; and More graphics are attempting to develop
Greg Linden asks if spammers have new techniques and technologies 30 Emerging Markets
been defeated; Michael Bernstein to overcome the current limitations Has China Caught Up in IT?
discusses Clay Shirky’s keynote in stereoscopic 3D. An assessment of the relative
speech at CSCW 2010; and By Kirk L. Kroeker achievements in IT infrastructure,
Erika S. Poole writes about how the firms, and innovation in China.
digital world can help parents cope 17 Making Sense of Real-Time Behavior By Ping Gao and Jiang Yu
with the death of a child. Data captured by sensors worn on
the human body and analyzed in 33 Kode Vicious
10 CACM Online near real-time could transform Presenting Your Project
Print is Not Just Ink Anymore our understanding of human The what, the how, and the why of
By David Roman behavior, health, and society. giving an effective presentation.
By Sarah Underwood By George V. Neville-Neil
37 Calendar
19 Celebrating the Legacy of PLATO 35 Privacy and Security
125 Careers The PLATO@50 Conference Remembrances of Things Pest
marked the semicentennial Recalling malware milestones.
of the computer system that was By Eugene H. Spafford
Last Byte the forerunner of today’s social
media and interactive education. 38 Viewpoint
128 Puzzled By Kirk L. Kroeker Rights for Autonomous
Figures on a Plane Artificial Agents?
By Peter Winkler 21 Gödel Prize and Other CS Awards The growing role of artificial
Sanjeev Arora, Joseph S.B. Mitchell, agents necessitates modifying
and other researchers are legal frameworks to better
recognized for their contributions address human interests.
to computer science. By Samir Chopra
By Jack Rosenberger
41 Interview
An Interview with Edsger W. Dijkstra
The computer science luminary,
in one of his last interviews
before his death in 2002, reflects
on a programmer’s life.
By Thomas J. Misa

Association for Computing Machinery

Advancing Computing as a Science & Profession

2 communicaT io nS o f T he ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

08/2010 vol. 53 no. 08

Practice Contributed Articles Research Highlights

48 Software Development 104 Technical Perspective

with Code Maps Attacks Target Web Server Logic
Could ubiquitous hand-drawn And Prey on XCS Weaknesses
code map diagrams become By Helen Wang
a thing of the past?
By Robert DeLine, Gina Venolia, 105 The Emergence of Cross
and Kael Rowan Channel Scripting
By Hristo Bojinov, Elie Bursztein,
55 Moving to the Edge: and Dan Boneh
A CTO Roundtable on
Network Virtualization
Leading experts debate how 114 Technical Perspective
virtualization and clouds impact Large-Scale Sound and
network service architectures. Precise Program Analysis
By Mache Creeger By Fritz Henglein

63 Seven Principles for Selecting 72 The Singularity System 115 Reasoning About the Unknown
Software Packages Safe, modern programming in Static Analysis
Everything you always wanted to languages let Microsoft rethink By Isil Dillig, Thomas Dillig,
know but were afraid to ask about the architectural trade-offs in its and Alex Aiken
the decision-making process. experimental operating system.
By Jan Damsgaard and Jan Karlsbjerg By James Larus and Galen Hunt
Virtual Extension
Articles’ development led by 80 Predicting the Popularity of Online Content as with all magazines, page limitations often
Early patterns of Digg diggs and prevent the publication of articles that might
YouTube views reflect long-term otherwise be included in the print edition.
to ensure timely publication, aCM created
user interest.
Communications’ Virtual extension (Ve).
By Gabor Szabo and Ve articles undergo the same rigorous review
Bernardo A. Huberman process as those in the print edition and are
accepted for publication on their merit. these
articles are now available to aCM members in
Review Articles the digital library.

90 Memory Models: A Case for Intelligent Service Machine

Rethinking Parallel Languages Wei-Feng Tung and Soe-Tsyer Yuan
and Hardware
Solving the memory model problem Thinkflickrthink: A Case Study
will require an ambitious and cross- on Strategic Tagging
about the cover: disciplinary research direction. Eugenio Tisselli
the singularity system,
Microsoft research’s effort By Sarita V. Adve and Hans-J. Boehm
to build a microkernel- Plat_Forms: Is There One Best
based operating system, is
the focus of this month’s Web Development Technology?
cover story as told by the Lutz Prechelt
projects’ lead researchers
James larus and Galen
IllustratIon by st udIo to nne

Hunt. the system draws How a Service-Oriented Architecture

parallels to the singularity
model where physical laws May Change the Software
as we know them are no
longer valid. rendering this
Development Process
concept for the cover is Marc N. Haines and
Paul Farrington from studio tonne, an illustrative agency
based in brighton, england. For more on the studio’s work,
Marcus A. Rothenberger

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 3
communications of the acm
trusted insights for computing’s leading professionals.

Communications of the ACM is the leading monthly print and online magazine for the computing and information technology fields.
Communications is recognized as the most trusted and knowledgeable source of industry information for today’s computing professional.
Communications brings its readership in-depth coverage of emerging areas of computer science, new trends in information technology,
and practical applications. Industry leaders use Communications as a platform to present and debate various technology implications,
public policies, engineering challenges, and market trends. the prestige and unmatched reputation that Communications of the ACM
enjoys today is built upon a 50-year commitment to high-quality editorial content and a steadfast dedication to advancing the arts,
sciences, and applications of information technology.

ACM, the world’s largest educational STA F F eDITOr IAl bOA rD

and scientific computing society, delivers
resources that advance computing as a DIRECTOR OF GROUP PU BLIS h I n G E DITOR- I n- C hIE F
science and profession. ACM provides the scott e. delman Moshe y. Vardi ACM Copyright notice
computing field’s premier Digital Library Copyright © 2010 by association for
and serves its members and the computing Executive Editor nE W S Computing Machinery, Inc. (aCM).
profession with leading-edge publications, diane Crawford Co-chairs Permission to make digital or hard copies
conferences, and career resources. Managing Editor Marc najork and Prabhakar raghavan of part or all of this work for personal
thomas e. lambert Board Members or classroom use is granted without
Executive Director and CEO Senior Editor brian bershad; Hsiao-Wuen Hon; fee provided that copies are not made
John White andrew rosenbloom Mei Kobayashi; rajeev rastogi; or distributed for profit or commercial
Deputy Executive Director and COO Senior Editor/news Jeannette Wing advantage and that copies bear this
Patricia ryan Jack rosenberger notice and full citation on the first
Director, Office of Information Systems Web Editor VIE W P OIn TS page. Copyright for components of this
Wayne Graves david roman Co-chairs work owned by others than aCM must
Director, Office of Financial Services Editorial Assistant susanne e. Hambrusch; John leslie King; be honored. abstracting with credit is
russell Harris Zarina strakhan J strother Moore permitted. to copy otherwise, to republish,
Director, Office of Membership Rights and Permissions Board Members to post on servers, or to redistribute to
lillian Israel deborah Cotton P. anandan; William aspray; lists, requires prior specific permission
Director, Office of SIG Services stefan bechtold; Judith bishop; and/or fee. request permission to publish
donna Cappo Art Director stuart I. Feldman; Peter Freeman; from or fax
Director, Office of Publications andrij borys seymour Goodman; shane Greenstein; (212) 869-0481.
bernard rous Associate Art Director Mark Guzdial; richard Heeks;
Director, Office of Group Publishing alicia Kubista rachelle Hollander; richard ladner; For other copying of articles that carry a
scott delman Assistant Art Director susan landau; Carlos Jose Pereira de lucena; code at the bottom of the first or last page
Mia angelica balaquiot beng Chin ooi; loren terveen or screen display, copying is permitted
ACM CO U N C I l Production Manager provided that the per-copy fee indicated
President lynn d’addesio in the code is paid through the Copyright
Wendy Hall Director of Media Sales P R AC TIC E Clearance Center;
Vice-President Jennifer ruzicka Chair
alain Chesnais Marketing & Communications Manager stephen bourne Subscriptions
Secretary/Treasurer brian Hebert Board Members an annual subscription cost is included
barbara ryder Public Relations Coordinator eric allman; Charles beeler; david J. brown; in aCM member dues of $99 ($40 of
Past President Virgina Gold bryan Cantrill; terry Coatta; Mark Compton; which is allocated to a subscription to
stuart I. Feldman Publications Assistant stuart Feldman; benjamin Fried; Communications); for students, cost
Chair, SGB Board emily eng Pat Hanrahan; Marshall Kirk McKusick; is included in $42 dues ($20 of which
alexander Wolf George neville-neil; theo schlossnagle; is allocated to a Communications
Co-Chairs, Publications Board Columnists subscription). a nonmember annual
Jim Waldo
ronald boisvert, Holly rushmeier alok aggarwal; Phillip G. armour; subscription is $100.
Martin Campbell-Kelly; the Practice section of the CaCM
Michael Cusumano; Peter J. denning; editorial board also serves as
Carlo Ghezzi; ACM Media Advertising Policy
shane Greenstein; Mark Guzdial; the editorial board of .
anthony Joseph; Communications of the ACM and other
Mathai Joseph; Peter Harsha; leah Hoffmann; aCM Media publications accept advertising
Kelly lyons; Mari sako; Pamela samuelson; in both print and electronic formats. all
bruce Maggs; Gene spafford; Cameron Wilson advertising in aCM Media publications is
al aho and Georg Gottlob
Mary lou soffa; Board Members at the discretion of aCM and is intended
Fei-yue Wang C O N TACT P O IN TS to provide financial support for the various
yannis bakos; Gilles brassard; alan bundy;
SGB Council Representatives Copyright permission activities and services for aCM members.
Peter buneman; Ghezzi Carlo;
Joseph a. Konstan; Current advertising rates can be found
andrew Chien; anja Feldmann;
robert a. Walker; Calendar items by visiting or
blake Ives; James larus; Igor Markov;
Jack davidson by contacting aCM Media sales at
Gail C. Murphy; shree nayar; lionel M. ni;
Change of address (212) 626-0654.
sriram rajamani; Jennifer rexford;
Marie-Christine rousset; avi rubin;
Co-Chairs Letters to the Editor Single Copies
abigail sellen; ron shamir; Marc snir;
ronald F. boisvert and Holly rushmeier single copies of Communications of the
larry snyder; Veda storey;
Board Members Manuela Veloso; Michael Vitale; ACM are available for purchase. Please
Jack davidson; nikil dutt; Carol Hutchins; W e b S IT e contact
Wolfgang Wahlster; andy Chi-Chih yao;
ee-Peng lim; Catherine McGeoch;
Willy Zwaenepoel
M. tamer ozsu; Vincent shen; COMMU N ICATION S OF THe ACM
Mary lou soffa; ricardo baeza-yates AU T H O r g U ID e l IN eS RES E A R C h hIGhLIGh TS (Issn 0001-0782) is published monthly Co-chairs by aCM Media, 2 Penn Plaza, suite 701,
ACM U.S. Public Policy Office david a. Patterson and stuart J. russell new york, ny 10121-0701. Periodicals
Cameron Wilson, director A DVe rT IS IN g Board Members postage paid at new york, ny 10001,
1828 l street, n.W., suite 800 Martin abadi; stuart K. Card; deborah estrin; and other mailing offices.
Washington, dC 20036 usa ACM ADVERTISInG D EPARTM E n T shafi Goldwasser; Monika Henzinger;
t (202) 659-9711; F (202) 667-1066 2 Penn Plaza, suite 701, new york, ny Maurice Herlihy; norm Jouppi; POSTMASTer
10121-0701 andrew b. Kahng; Gregory Morrisett; Please send address changes to
Computer Science Teachers Association t (212) 869-7440 Michael reiter; Mendel rosenblum; Communications of the ACM
Chris stephenson F (212) 869-0481 ronitt rubinfeld; david salesin; 2 Penn Plaza, suite 701
executive director
lawrence K. saul; Guy steele, Jr.; new york, ny 10121-0701 usa
2 Penn Plaza, suite 701 Director of Media Sales
Gerhard Weikum; alexander l. Wolf;
new york, ny 10121-0701 usa Jennifer ruzicka
Margaret H. Wright
t (800) 401-1799; F (541) 687-1840
Association for Computing Machinery Media Kit
James landay and Greg linden
2 Penn Plaza, suite 701
Board Members SE
new york, ny 10121-0701 usa A

Gene Golovchinsky; Jason I. Hong;



t (212) 869-7440; F (212) 869-0481


Jeff Johnson; Wendy e. MacKay Printed in the u.s.a.





4 communicaTio nS o f The ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

jacm editor’s letter

DOI:10.1145/1787234.1787235 Victor Vianu

JACM at the Start of a new Decade

It has been almost a year since I assumed the
editor-in-chief role for Journal of the ACM (JACM).
The move coincided with the start of a new decade.
For both reasons, it seems the right time to share
some thoughts on how ACM’s old- published in this period, and none were us say, hypothetically, that we aim to
est publication is doing and where it accepted in software engineering. Even publish annually the three best papers
might be headed. areas with very strong theoretical sides in 10 subfields of computer science. At
First published in January 1954, have minimal representation, includ- an average of 40 pages per paper, this
JACM plays a special role among ACM ing cryptography, logic in computer sci- would quickly consume the current an-
publications. Transactions publish ence, machine learning, and computer- nual page budget of 1,200 pages. This
high-quality research in a specific sub- aided verification. makes the notion of forgoing the print
field of computer science, aiming for There are several possible expla- edition in favor of an online-only publi-
comprehensive coverage of the area. nations for the difficulty in attracting cation increasingly tempting. However,
Communications of the ACM provides top-quality papers in some areas. Con- this remains a controversial idea.
magazine-style content appealing to ference publications are increasingly fa- JACM papers are currently published
all ACM members—academics and vored over journal publications in many in e-form in ACM’s Digital Library. Be-
practitioners—and includes select re- subfields. There also seems to be a (mis) sides advantages of cost and scalability,
search articles showcasing the “best of perception that some topics are not wel- the DL provides substantial added val-
the best” results originally presented come to JACM. One way to counteract ue in the form of cross-links and search-
in ACM proceedings. JACM fulfills a this is to ensure visible representation able metadata. Another advantage is the
complementary role by publishing in a of such areas on the editorial board. ability to post additional content on
single, highly selective venue, the best Indeed, several editors have now been home pages of articles, such as errata,
research in all areas of computer sci- appointed representing such areas as appendices, notes, even slides or videos
ence, allowing researchers to keep up bioinformatics, Web systems and algo- of talks. I believe such content is appeal-
with the state of the art across the entire rithms, software engineering, and com- ing to both readers and authors, and we
discipline. As such, JACM is the flagship putational economics. will aim to provide it on a regular ba-
scientific publication of the ACM. A proactive approach to ensuring sis. JACM’s new information director,
Overall, I believe JACM is going top-quality representation from a wid- Pierre Senellart, will play a central role
strong. It is a widely respected publi- er spectrum of areas, initiated by Prab- in shaping JACM’s online presence. A
cation; according to at least one bib- hakar Raghavan, consists of inviting a new Web site for JACM has also been
liometric authority (http://www.eigen- small number of papers selected from launched (, it is the top-ranked top conferences in targeted subfields. One of the often-cited drawbacks of
journal in computer science. Yet JACM We currently have or are exploring journals versus conferences is the long
is facing nontrivial challenges. With the such arrangements with annual sym- wait from submission to publication.
field expanding and becoming increas- posiums, including STOC, FOCS, Prin- In the case of JACM, the publication
ingly diversified, its charter of publish- ciples of Database Systems (PODS), queue has hovered around three issues,
ing the best research across computer Principles of Distributed Computing or six months, for quite some time. This
science—as broadly construed—is a tall (PODC), Principles of Programming is reasonable, since it is considered
order. Much of JACM’s focus has been Languages (POPL), Logic in Computer risky to have much fewer papers in the
on theory of the flavor found at the Sym- Science (LICS), and conferences such publication pipeline. I am working with
posium on Theory of Computing (STOC) as Research in Computational Mo- the board to keep the processing time
and Foundations of Computer Sci- lecular Biology (RECOMB) and Com- under a year for most papers eventually
ence (FOCS). Past editors-in-chief have puter Aided Verification (CAV). This accepted, and shorter for papers that
worked to expand the scope of JACM be- approach is appealing because it no are eventually rejected.
yond this core. However, publications in longer leaves coverage of underrepre- In summary, JACM is facing some
some of the emerging or cross-boundary sented areas entirely up to the chance growing pains but it is thriving. I am
areas have been slow to follow. Of the 93 of spontaneous submissions. confident the journal will pursue its up-
articles published in JACM over the past An orthogonal obstacle to compre- ward trajectory in the coming years.
three years, approximately 35 are still in hensive coverage is simply due to the
Victor Vianu ( is a professor of
core algorithms and complexity. In con- proliferation of areas to be covered, computer science and engineering at university of
trast, only one bioinformatics paper and coupled with the physical limitations California, san diego.
one computer architecture paper were and cost of JACM as a print journal. Let © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 5
in the virtual extension


in the Virtual extension

Communications’ Virtual Extension brings more quality
articles to ACM members. These articles are now available
in the ACM Digital Library.

intelligent Service machine Plat_forms: is There one Best

DOI:10.1145/1787234.1787268 Web Development Technology?
Wei-Feng Tung and Soe-Tsyer Yuan DOI:10.1145/1787234.1787271
Machine is a metaphor that can be used to Lutz Prechelt
expand the capability of service systems and Plat_Forms is a contest in which
‘think’ for innovative service system design. three-person teams of professional
In this article, the notion of service machine programmers competed to implement
is defined as a socio-technical system with the same requirements for a Web-based
the shared reality of customer and provider system within two days, using different
aiming for the joint optimization of technology platforms. Three teams
productivity and satisfaction. An intelligent used Java EE, three used Perl, and three
service machine (ISM) moves beyond service used PHP. The resulting systems were
machine by modeling and automating thoroughly evaluated with respect to many
the cognitive process and knowledge criteria, such as completeness (reflecting
representations of the machine’s embodied productivity), maintainability, robustness
theory, enabling a systematic and (hinting at security), and size. This article
quantitative delivery of service operation reports on the setup of the contest and
using self services. The authors present some results of this study. Readers should
a machine-aware service-system design expect to see some prevalent prejudices
ACM’s framework (iDesign) and an ISM-supported confirmed and others firmly refuted.
service system to demonstrate the notion
interactions and the framework. how a Service-oriented
magazine explores architecture may change the
critical relationships Thinkflickrthink: a case Study Software Development Process
on Strategic Tagging DOI:10.1145/1787234.1787269
between experiences, people, DOI:10.1145/1787234.1787270
Marc N. Haines and
and technology, showcasing Eugenio Tisselli Marcus A. Rothenberger
emerging innovations and industry A tag can be created and disseminated The service-oriented approach to IT
for strategic purposes, including online architecture has become an important
leaders from around the world protest. The research presented in this alternative to traditional software
across important applications of article analyzes one particular protest development. Adding to this impetus are
strategy adopted by a number of users the efforts related to open standards and
design thinking and the broadening of Flickr: the use of anti-censorship tags open source products. But one key question
field of the interaction design. to make the protest visible within the remains: Are service-oriented architecture
site itself. The study of the dynamics (SOA) adopters ready for this change and
Our readers represent a growing of uncoordinated semantic strategies will they be able to provide a technical
community of practice that within dense online communities is of and organizational environment in which
enormous importance to gain a greater SOA-related technologies can be leveraged
is of increasing and vital understanding of how social and linguistic to their full potential? This article explores
interaction takes place in a networked the development process and methodology
global importance. environment, and how it can augment the that may require adjustments in order to
users’ potential for direct action. provide a good fit for SOA development.

Coming Next Month in COMMUNICATIONS


Medical Informatics: Point/Counterpoint on the


Why So Slow? Future of Internet Architecture


Research in Computing: Injecting Errors


The Myth of Rapid Obsolescence for Fun and Profit


Performance Evaluation and Thinking Clearly


Model Checking Join Forces About Performance


And the latest news on computational neurobiology, applying technology

to education, and MIT’s Big Wheel and smart bikes.

6 comm unicaTio nS o f T he ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

letters to the editor


cS expertise for institutional Review Boards

RBs n e e d com pu teR scien- Two such anecdotes involved research find some of them. Four to eight par-
tists, a point highlighted by on phishing, an intrinsically decep- ticipants should be included if the aim
the Viewpoint “Institutional tive phenomenon. Deception research, is to drive a useful iterative cycle: Find
Review Boards and Your Re- long used in social sciences, typically serious problems, correct them, find
search” by Simson L. Garfinkel takes longer to review because it runs more serious problems.
and Lorrie Faith Cranor (June 2010). counter to the ethical principle of Never expect a usability test to find
Not just over the nature of certain CS- “respect for persons” and its regula- all problems. CUE-studies1 show it
related research but because social sci- tory counterpart “voluntary informed is impossible or infeasible to find all
entists (and others) administer online consent.” Before developing a techni- problems in a Web site or product;
surveys, observe behavior in discus- cal solution to perceived IRB delays, the number is huge, likely in the thou-
sion forums and virtual worlds, and the typical causes of delay must be sands. This limitation has important
perform Facebook-related research. In established. Possibilities include inef- implications on the size of a test group.
this regard, the column was timely but ficient IRBs and uninformed and/or So go for a small number of partici-
also somewhat misleading. unresponsive researchers. Moreover, pants, using them to drive a useful iter-
First, the authors created a dichot- as with any deception research, some ative cycle where the low-hanging fruit
omy of computer scientists and IRBs, proposals may just be more ethically is picked/fixed in each cycle.
saying IRB “chairs from many institu- complex, requiring more deliberation. Finally, the number and quality of
tions have told us informally that they michael R. Scheessele, South bend, IN usability test moderators affects re-
are looking to computer scientists to sults more than the number of test
come up with a workable solution to the participants.
difficulty of applying the Common Rule authors’ Response: In addition, from a recent discus-
to computer science. It is also quite Scheessele is correct in saying an increasing sion with the authors, I now under-
clear that if we do not come up with a number of social scientists use computers in stand that the published research in the
solution, they will be forced to do so.” their research and is yet another reason IRBs article was carried out in 2004 or earlier
However, any institution conduct- should strive to include a computer scientist and the article was submitted for pub-
ing a significant amount of human- as a member. Sadly, our experience is that lication in 2006 and accepted in 2008.
subjects research involving computing most IRBs in the U.S. are understaffed, lack All references in the article are from
and IT ought to include a computer sci- sufficient representation of members with 2004 or earlier. The authors directed
entist on its IRB, per U.S. federal regu- CS knowledge, and lack visibility among CS my questions to the first author’s Ph.D.
lations (45 CFR 46.107(a)): “Each IRB researchers in their organizations. dissertation, which was not, however,
shall have at least five members, with Simson L. Garfinkel, Monterey, CA included in the article’s references and
varying backgrounds to promote com- Lorrie faith cranor, Pittsburgh, PA is apparently not available.
plete and adequate review of research Rolf molich, Stenløse, Denmark
activities commonly conducted by the
institution. The IRB shall be sufficient- how many Participants Reference
1. Molich, r. and dumas, J. Comparative usability
ly qualified through the experience and needed to Test usability? evaluation (Cue-4). Behaviour & Information
expertise of its members…” No usability conference is complete Technology 27, 3 (May 2008), 263–281.

Though CS IRB members do not without at least one heated debate on

have all the answers in evaluating hu- participant-group size for usability correction
man-subject research involving com- testing. Though Wonil Hwang’s and “CS and Technology Leaders Honored”
puting and IT, they likely know where Gavriel Salvendy’s article “Number of (June 2010) mistakenly identified the
to look. It would also mitigate another People Required for Usability Evalu- American Academy of Arts and Sci-
problem explored in the column, that ation: The 10±2 Rule” (Virtual Exten- ences as the American Association for
“many computer scientists are unfamil- sion, May 2010) was timely, it did not the Advancement of Science. Also, it
iar with the IRB process” and “may be address several important issues con- should have listed Jon Michael Dunn,
reluctant to engage with their IRB.” In- cerning numbers of study participants: Indiana University, as one of the com-
deed, if an IRB member is just down the Most important, the size of a par- puter scientists newly elected as an
hall, computer scientists would likely ticipant group depends on the purpose American Academy 2010 Fellow. We
find it easier to approach their IRB. of the test. For example, two or three apologize for these errors.
Second, the authors assumed the participants should be included if the
length of the IRB review process rep- main goal is political, aiming to, say, Communications welcomes your opinion. to submit a
resents a problem with the process demonstrate to skeptical stakeholders letter to the editor, please limit your comments to 500
words or less and send to
itself though offered only anecdotal that their product has serious usabil-
evidence to support this assumption. ity problems and usability testing can © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 7
The Communications Web site,,
features more than a dozen bloggers in the BLoG@cacm
community. in each issue of Communications, we’ll publish
selected posts or excerpts.

follow us on Twitter at


The War against Spam; point that even massive and complicat-
ed spam efforts like the Storm botnet

and more
generate surprisingly low revenues for
what appears to be the work required.
What is your own experience with
email spam? Do you think the spam
Greg Linden asks if spammers have been defeated; Michael Bernstein war been won? Or are these declara-
discusses Clay Shirky’s keynote speech at CSCW 2010; and tions of victory premature?
Erika S. Poole writes about how the digital world can help parents
cope with the death of a child. Reader’s comment:
I would argue that your declaration of
victory is premature, but not for the obvious
Greg Linden’s “has the trying to customize each email sent, reason that spam in the inbox has been
Spam War Been Won?” and many other tricks to evade detec- reduced. In that respect, spam in my inbox, tion, but their increasingly complicat- and my spam folder as well, has gone way
blogs/blog-cacm/78121 ed efforts have not been able to outwit down in recent years, and that is welcome.
A decade ago, email the filters. However, two problems remain. First
spam was a dire problem. My own experience is that email and foremost, there is still the problem
Annoyances flooded most inboxes. spam has become a non-issue. Despite of false positives. I still have to check
Any attempt to read your email started prostituting my email addresses un- my spam folder because the filters will
with deleting the crud that had leaked disguised across the Internet, despite occasionally falsely flag a legitimate email.
through your defenses. receiving hundreds of spam messages Once I do find a desired email, I can flag it
Many predicted the problem would daily, nearly zero make it to my inbox. as non-spam to teach the filter and add the
only get worse. A few predicted that The ones that I do see typically are sender to a white list, but that is reactive. If
email would be dead in just a few years, borderline spam, from companies and you’re receiving hundreds of spam emails
the filters would be overwhelmed, the small businesses sending to a small a day, as you wrote, I imagine more than a
war would be lost, and email readers list rather than the mass splattering of few false positives slip by you.
would be buried under an avalanche true email spam. Additionally, as an entrepreneur, sending
of spam. Amazingly, the drop in response email from a new company like mine that
Today, email spam appears to be a rates from 2003 to 2008 may be close hasn’t established itself with the many
solved problem. A 2003 study put re- to making spam an unprofitable en- filters out there [is] very time consuming
sponse rates at 0.005%. A 2008 study terprise. There is a substantial amount and inefficient. If there was a proactive way
where the authors infiltrated a major of effort required to attack and man- for a legitimate sender to register itself and
spam botnet found response rates age a botnet of 1 million compro- either post a bond or pay e-postage, I think
had fallen to under 0.00001%, with mised machines that can cheaply send that would clean up a lot of email inboxes.
only 28 sales out of 350 million mes- 12 million messages per day. Huge I know e-postage proposals haven’t
sages sent. Spam filters appear to have email campaigns that attempt to work gotten very far in the past, but if the spam
forced down response rates three or- around spam filters require sophistica- response rate is now down to 0.00001%
ders of magnitude in five years. Spam- tion to devise and run. Email address then the postage can be a lot lower as well.
mers have fought back with misspell- lists have to be purchased and main- The second reason we can’t declare
ings, adding additional text to emails, tained. It appears to be getting to the victory just yet is the very fact that the

8 communicaTio nS o f T he acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8


spammers and their resources, both botnets um, an event in which new scholars dis-
and humans, remain alive and well. If email cuss their work with a panel of experts.
spam continues to become less and less “usability is an In addition to being a great opportunity
profitable, then they will simply send spam important refinement for students, the Doctoral Colloquium
in other forms such as on Twitter, Facebook, highlights some of the most exciting
etc. Individual computers continue to technique when you work in the field from promising young
get infected and people still foolishly have a good idea, scholars. In particular, I couldn’t help
click on requests from Nigerian princes. but notice that the students invited to
Unfortunately, we have to continue to apply but it is a horrible this year’s event presented work high-
technological fixes to our networks and determiner of utility lighting the deeply human side of infor-
teach people not to be so gullible. mation technology.
Believe me, I wish we could declare on a grander scale.” For example, imagine you’re a par-
victory, but we’re not there yet. ent who has suffered the unthinkable:
—Will Hartmann Your child has died. How do you cope
with such a traumatic, painful, and
michael Bernstein’s disorienting experience? For some
“clay Shirky: Doing parents, information technologies can
work, or Doing Work?” tivated to do it in the first place?” Excel play an important role in the grieving needs usability testing because people and mourning process. Yet how are be-
blogs/blog-cacm/72609 are forced to use it for Work; technology reaved parents using technologies to
MSN usability research- for work instead needs to understand grieve and mourn? If we were to design
ers were stumped. Their usability lab users’ underlying motivations. technologies that help people cope
had tested just about every aspect of Extrapolating on my own here: with grieving and loss in meaningful
its MSN portal and had been pleased to Usability is an important refinement and respectful ways, what would they
find that it consistently scored higher technique when you have a good idea, look like?
than its competitors. Yet a user base but it is a horrible determiner of utility I had the opportunity to speak with
didn’t flock to MSN—the portal simply on a grander scale. (Sure, pay me $10 Mike Massimi, a Ph.D. candidate at
could not attract and retain as many for a lab study and I’ll use anything University of Toronto who’s examin-
users as it wanted. Then Clay Shirky for an hour!) Usability is a local hill- ing these questions for his thesis work.
relayed the million-dollar question: climbing algorithm. We need tech- To understand how technology plays a
Were these tasks that users actually niques to make and evaluate that mi- role in modern grieving, Mike is work-
wanted to do? Or were these highly us- raculous motivational leap, whether ing extensively with two community
able aspects of the site going to remain it’s derived from the design process or organizations in the Ontario area that
unused because nobody wanted to use social science. Develop that and you provide social support to parents who
them? There was a gaping hole be- could save thousands of man-hours have suffered the loss of a child. His
tween usability and usefulness. developing tools that nobody will ever next step is to create meaningful, ap-
In a keynote delivered to this year’s want to use. propriate, and respectful technologies
ACM Conference on Computer Sup- that help bereaved parents mourn and
ported Cooperative Work (CSCW), Reader’s comment: remember their lost children.
author and academic Clay Shirky cap- Jared Spool, for years, talked about Death and dying are experiences as
tured this question in a distinction be- compelled shopping tasks in which people old as humanity, and have been stud-
tween Work and work. Work (Capital W) were actually given money to buy things ied by scholars in other disciplines for
is what we have considered for years; on Internet sites, and could not. He really centuries. Yet technology researchers
your boss tells you to do something, wanted to solve the usability problem, and designers are just now starting
you do it, and you get paid. By contrast, but also realized that decoupling the to come to grips with how to design
work (little w) is motivated by inherent motivational issues with usability is difficult. with end-of-life experiences in mind.
interest and is generally unpaid. Think Little ‘w’ work vs. Big ‘W’ Work suggests If you’re interested in learning more
of the difference between an Encyclope- that we are going to have to dig much about this topic, Mike co-hosted the
dia Britannica editor doing Work and deeper into this issue than we had before. first workshop (ever!) focused on death
a Wikipedia editor doing work during —Ed H. Chi and the digital world at the ACM CHI
spare hours. Big Work drives the econ- 2010 conference. You can see more
erika S. Poole’s
bernsteIn PH otoGra PH by Jason d orF M an

omy; little work drives the Internet. Big info about the workshop at http://www.
Work builds skyscrapers; little work “Death and the
generates a half-million fan fiction sto- Digital World”
ries about Harry Potter. Greg Linden is the founder of Geeky Ventures. Michael
Clay argued that user-testing tech- blogs/blog-cacm/72837 Bernstein is a Ph.d. student in the Computer science and
artificial Intelligence lab at Massachusetts Institute of
niques developed over the past 25 years At the Computer Support- technology. Erika Shehan Poole is an assistant professor
for Work no longer apply for work. We ed Cooperative Work 2010 conference, at the school of Information sciences and technology at
Penn state university.
shouldn’t be asking, “Can you com- 13 Ph.D. students received invitations
plete the task?” but rather “Are you mo- to participate in the Doctoral Colloqui- © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 9
cacm online

DOI:10.1145/1787234.1787239 David Roman WiLLiam J. DaLLy WinS
eckeRT-mauchLy aWaRD

Print is not Just ink anymore Acm and Ieee

society jointly
presented the
The world of Communications is not contained in the pages of a monthly maga- Award to
William J. dally,
zine. Like other publications, Communications has expanded over time into a va- chief scientist and senior vice
riety of electronic formats for e-connected members and readers. Each format de- president of research of nVIdIA
livers something its counterparts do not. Digital Editions ( corp., for his innovative
communications) present complete issues with familiar, flipable pages, but on contributions to the architecture
of interconnection networks and
full-screen and mobile systems. The Web site ( moves maga- parallel computers. dally, who is
zine content into HTML, also the Willard R. and Inez Kerr
and adds other articles, Bell professor of computer
science and electrical
daily news, blogs, plus ac-
engineering at stanford
cess to ACM’s abundant university, developed the system
member services. Articles and network architecture,
from Communications’ signaling, routing, and
synchronization technology that
Virtual Extension (VE) are is found in most of today’s large
available from the Web parallel computers.
site and ACM’s Digital Li- dally discussed his current
brary (; research in an email interview,
saying, “At present I’m working
the print edition publishes on low-power computer
only their summaries. Digi- architecture. It’s interesting to
tal Editions, introduced in find out where the power actually
goes in a modern computer, and
January 2008, have cleared it’s very exciting that there are
the way for mobile apps opportunities where innovation
Communications is going mobile.
and a mobile Web site, now can make a large difference
in development, that will tailor content to handhelds. The goal of each format is in efficiency. It’s particularly
rewarding that this work is likely
to give users the content they want, where, when, and how they want it. to have a measurable positive
Communications’ brand began taking e-steps before the relaunch of the Com- impact on the environment.
munications Web site in April 2009. The concept of the VE, in fact, was introduced “I really enjoy seeing how
parallel computing enables
in 1996, first as a biannual collection of articles available only in e-format—a pio-
new applications that weren’t
neering step in publishing circles back then. Originally conceived as an outlet for possible before. some of the
articles that did not fit into page-constraints of the print edition, the VE is coming most exciting of these involve
into its own, having evolved as a monthly editorial fixture since September 2008. better human-computer
interfaces and interactions
Like Communications’ other formats, it will continue to evolve, and may become with the physical world. A great
a component of a digital-first publishing strategy. The VE’s status is evidenced by example is augmented reality—
the readership of its most popular articles listed here, which is on par with and in where parallel computing
some cases exceeds that of print issue cover stories. The VE is establishing itself enables realtime computer
vision to interpret the image
as a destination for authors and readers. you see, query a database, and
annotate the image with useful
article uRL
Asked about important
Principles for effective Virtual Teamwork cs issues, dally singled out
“cs education, particularly
Capstone Programming Courses Considered Harmful education about parallelism.
Number of People required for Usability evaluation: 10±2 rule We aren’t producing enough
cs graduates, and the ones we
PHotoGra PH by a ndrIJ bo rys

An Overview of IT Service Management do produce don’t understand

Why Did Your Project Fail? parallel programming. teaching
parallelism isn’t just an add-on
The requisite Variety of Skills for IT Professionals to the existing curriculum; every
course needs to be redesigned
A Holistic Framework for Knowledge Discovery and Management
around parallelism.”
—Jack Rosenberger

10 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


Science | DOI:10.1145/1787234.1787240 Gary Anthes

mechanism Design
meets computer Science
A field emerging from economics is teaming up with computer science
to improve auctions, supply chains, and communication protocols.

h e Bo u n d ARIe s se pARAtIn g
computer science and other
disciplines are blurring at an
accelerating pace. As work in
computers, biology, the phys-
ical and social sciences, and econom-
ics becomes more complex, so does
the motivation for practitioners to seek
help from each other.
Mechanism design, which emerged
from economic game theory in the
1970s, is now shaking hands with in-
formation technology. Built on a formal
mathematical base, mechanism design
expresses ideas that are elegantly sim-
ple, yet tricky to apply in the real world:
people in competition will act “rational-
ly” to meet their own selfish goals; they
have private information, and may act
in ways that can’t be observed; and they in the DaRPa network challenge, teams used mechanism design and social networking tech-
may lie. The central goal in mechanism niques to locate the defense agency’s 10 geographically dispersed, red weather balloons.
design is to devise a system by which
those people will tend to act in ways that nology, from network design to distrib- all 10 balloons.
benefit the owner of the system, or soci- uted computing to operating systems. Success would depend on a competi-
ety at large. The DARPA Network Challenge tor’s ability to assemble a large group
Information technologists are turn- from the Defense Advanced Research of geographically dispersed volunteers.
PHotoGra PH Court esy oF da rPa

ing these concepts into such disparate Projects Agency (DARPA) offers one ex- “We wanted to understand how you
applications as auction management, ample, with a social networking twist. could rapidly mobilize a very large team
supply chain optimization, and the Last December, DARPA tethered 10 red to solve a hard problem, and how to do
matching of organ donors and recipi- weather balloons at undisclosed loca- that in an adversarial environment,”
ents. Meanwhile, mechanism design is tions across the continental U.S. DAR- says Peter Lee, a DARPA director. Also,
enabling advances in information tech- PA’s challenge: Be the first team to find DARPA wanted to learn how teams

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 11

would use social networks and crowd- 5,000 people signed up to assist the MIT um price, and the auction would simply
sourcing as this might aid in military team, and estimates that two million find it. But more recently he’s worked
intelligence gathering. people received email, Twitter, or Face- with combinatorial auctions, where
The DARPA experiment was not book requests for help. there often is no such single price and
conceived specifically as an exercise in The second-place team from Georgia the computation of efficient outcomes
mechanism design, but many of the Institute of Technology did almost as is generally NP-hard. Managing the re-
teams, including the winning one from well, finding nine balloons in a largely sulting complexity demands the appli-
Massachusetts Institute of Technology more traditional way, via old media, in- cation of computer science techniques.
(MIT), profitably employed those prin- cluding an article in The Wall Street Jour- Milgrom says he’s working on dy-
ciples, Lee says. For example, the MIT nal and an interview on National Public namic resource allocation auctions in
team found all 10 balloons in less than Radio. That put Georgia Tech near the which, for example, a seller might have
nine hours by devising a clever way to top of the results in a Google search to decide to accept a bid now or wait for
motivate a large number of volunteers for “DARPA Red Balloon,” says Ethan a better one tomorrow. More generally,
through a recursive incentive scheme. Trewhitt, a research engineer at Geor- it’s the challenge of efficiently allocat-
An obvious financial incentive would gia Tech. “The crowdsourcing thing ing a given resource, such as communi-
be to promise anyone who found and was not as big a thing as the old media, cation bandwidth, repeatedly over time.
reported a balloon to the winning team which are machines built for this pur- Buyers, or users, learn more about the
a part of the $40,000 prize money. Some pose,” he says. resource’s value by using it. “Some of
teams did just that. But the MIT team the best work in this area has been done
knew it not only had to motivate people making auctions Pay by computer scientists,” Milgrom says.
to look for and report balloons, but also Tuomas Sandholm, director of the Milgrom’s work with mechanism
to find lots of additional people to help Agent-Mediated Electronic Market- design, stretching back some three de-
them locate the balloons. places Laboratory at Carnegie Mellon cades, clearly is not at an end. Last year,
The MIT team recruited an initial University, has taken the basic concepts a company he started, Auctionomics,
cadre of volunteers via email, then en- of mechanism design and extended and won an NSF grant to develop auction
couraged each volunteer to establish implemented them in novel ways. For software that allows bidders to specify
a personal chain of participants: A re- example, he has pioneered automated budget constraints. “Multi-item auc-
cruits B, who recruits C, who recruits mechanism design, by which complex tion design has been at the frontier of
D, and so on. The first person to report mechanisms are devised by computers, research in economics and computer
a balloon gets $2,000, and more pay- not humans. science over the past 15 years,” NSF
ments propagate through the chain Many real-world applications of notes. “Yet no existing mechanism en-
back from that person. D finds a balloon mechanism design are complex indeed. ables effective competition when bid-
and gets $2,000, C gets $1,000, B gets For example, Procter & Gamble (P&G) ders face serious budget constraints.”
$500, and A gets $250. seeks to optimize its supply chain by Meanwhile, mechanism design is
“There were people who actually getting annual bids for trucking ser- also being used to solve problems of
made $1,000 for posting a tweet,” says vices across North America. P&G uses network congestion. TCP/IP commu-
Galen Pickard, a computer and cogni- a mechanism design-based combina- nication protocols are based on the as-
tive scientist at MIT. “Everyone was torial auction, designed by Sandholm, sumption that when a computer sees
incentivized to get the word out, and in which participants can bid for indi- congestion, it will temporarily delay
that’s mechanism design.” Pickard says vidual items or packages of items, in sending data. “But when TCP competes
combinations specified by them. “It’s a for bandwidth with other ‘non-polite’
very complex optimization problem for protocols, such as UDP, it ends up being
The miT team located both parties, with lots of constraints,” squeezed away completely,” says Noam
all 10 red balloons Sandholm says.
Likewise, Paul Milgrom, an econo-
Nisan, a computer scientist at Hebrew
University in Jerusalem.
in less than nine mist at Stanford University, co-designed Nisan says that all work on commu-
hours by devising the simultaneous ascending auction
used by the U.S. Federal Communica-
nication protocols today should consid-
er the fact that computers are connected
a clever way to tions Commission in its sale of radio yet controlled by “different, selfish enti-
motivate a large spectrum licenses in the mid-1990s. Ac-
cording to the National Academy of Sci-
ties.” Mechanism design can provide a
framework for optimizing those proto-
number of volunteers ences, “The auction broke all records cols, he says. For example, he cites re-
through a recursive for sale of public property and has been
widely copied in other countries,” while
cent research—namely, “Interdomain
Routing and Games” by economist Ha-
incentive scheme. the National Science Foundation (NSF) gay Levin and computer scientists Mi-
hailed it as a “victory for the field of chael Schapira and Aviv Zohar from the
game theory.” Hebrew University of Jerusalem—that
Milgrom’s early work, in the 1980s, shows that the Border Gateway Protocol
mostly dealt with situations in which (BGP), which is used for Internet rout-
there was a market-clearing or equilibri- ing among competing domains, could,

12 com municaTio nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


with certain security enhancements, be regarded as fake because real sight-

made “incentive-compatible.” That is, ings from multiple people always have
users would no longer have any reason Procter & Gamble some slight variation. “So there is this
to deviate from BGP. uses a mechanism general concept of being able to use
Nisan is a pioneer in the field of al- relatively straightforward data-mining
gorithmic mechanism design, by which design-based techniques to quickly derive the char-
systems find good approximations combinatorial acteristics between good and bad infor-
of optimal solutions in computation- mation,” Lee says.
ally hard problems, such as an auction auction in which Finally, says MIT’s Pickard, the most
where instead of a single point bid from participants can bid elegant of ideas can have unintended
each bidder, one gets an entire “graph” consequences. “We spent four days
of bids and preferences. That approach for individual items winning the DARPA Network Challenge
introduces “an interesting twist,” Nisan or packages of items, and about two months working out how
says. “Many of the results that we got for to pay people,” he says. “Working with
free from economic theory stop work- in combinations lawyers is a lot harder than making Web
ing when there are approximations.” specified by them. sites.”

Potential Pitfalls
Further Reading
To be sure, there are pitfalls in the appli-
cation of mechanism design ideas. For Feigenbaum, J., Papadimitriou, C.,
Sami, R., Shenker, S.
example, Robert Kleinberg, a computer A BGP-based mechanism for lowest-cost
science professor at Cornell University, one of P&G’s combinatorial auctions, in routing. Distributed Computing 18, 1, July
says mechanisms may be so compli- isolation, as much of economic theory 2005.
cated that users don’t understand them assumes. But if an auction is repeated, Nisan, N.
and hence won’t participate. It’s not suf- buyers and bidders learn something Algorithmic mechanism design, Google
ficient to tell them, “don’t worry, I have about each other, and that knowledge Tech Talks video, August 15, 2007,
proved a theorem that the best thing for can lead to behaviors that are both un-
you to do is such-and-such,” Kleinberg desirable and not predicted by theory.
says. Similarly, notions of rationality may Royal Swedish Academy of Sciences,
A related issue is that the mecha- be mathematically pure in econom- Prize Committee
Mechanism design theory. Oct. 15, 2007.
nism and the problem it is trying to ics, but “actual human behavior is a lot
solve may together be so complicated more complicated and superficially ‘ir- economics/laureates/2007/ecoadv07.pdf
as to be computationally intractable. rational’ than the predictions made by Sandholm, T.
The computational ability of partici- theoretical models,” Kleinberg says. Computing in mechanism design. The New
pants and their incentives are inter- And there’s that nasty bit about ly- Palgrave Dictionary of Economics (2nd ed.),
twined in complex ways. And the cost ing. Of the 200 red balloon sightings re- Palgrave Macmillan, Basingstoke, U.K., 2008.
of information gathering in order to bid ported to the MIT team, 80% were false. Varian, H.
can distort results, because the choice DARPA’s Lee says teams detected false Designing the perfect auction. Communications
of how much to invest in bid prepara- reports in some clever ways, with some of the ACM 51, 8, August 2008.
tion is not a choice that is included in teams even automating their detection.
most theoretical models of behavior. For example, clusters of reported bal- Gary Anthes is a technology writer and editor based in
arlington, Va.
Sandholm says another potential loon sightings that contained identical
pitfall is evaluating one event, such as coordinates, to the third decimal, were © 2010 aCM 0001-0782/10/0800 $10.00


How Top ISPs Could Reduce Spam

the zombie computers university of technology in the into a large network of remotely their paper, “the Role of Internet
responsible for sending more netherlands. controlled machines known as service providers in Botnet
than half of the world’s spam eeten and his colleagues a botnet. migration: An empirical Analysis
reside on the networks of the analyzed more than 63 billion “the top 50 Isps account for Based on spam data,” at the
leading 50 Isps, and if these unsolicited email messages sent over half of all [spam] sources Workshop on the economics of
Isps would shut down or block from 2005–2008 and discovered worldwide,” the researchers Information security at harvard
these compromised computers, that some 138 million unique note. “In light of the fact that university, and are now working
it would drastically curtail the Ip addresses were linked to the there are 30,000 [Autonomous with the dutch government
delivery of spam, according to messages’ delivery. often, these system numbers] and anywhere to create metrics of Isps’
a team of researchers led by unsolicited messages are mailed between 4,000 and 100,000 Isps, efforts to detect and shut down
michael van eeten, a professor by computers that have been this is a remarkable finding.” compromised computers.
of public administration at delft hijacked by criminals and turned the researchers presented —Jack Rosenberger

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 13

Technology | DOI:10.1145/1787234.1787241 Kirk L. Kroeker

Looking Beyond
Stereoscopic 3D’s Revival
Researchers working in vision and graphics are attempting to develop new
techniques and technologies to overcome the current limitations in stereoscopic 3D.

t e R e o s c o p I c 3 d I s experi- many people I’ve spoken with did ex-
encing a strong resurgence, perience discomfort, or were annoyed
with moviemakers no lon- kurt akeley is by certain cinematic techniques, such
ger using the technique pri- experimenting with as the limited depth of field in many
marily as a gimmicky audi- scenes.”
ence-draw consisting of objects poking an approach related There are several kinds of depth
from the screen into the theater space. to light-field theory cues that researchers working in this
In today’s cinema, stereoscopic 3D is area are actively studying to improve
being used more subtly as an aspect of in which the display such stereoscopic 3D experiences. For
storytelling to enhance immersion into is replaced with example, one kind of cue is motion
environments that appear to invite the parallax, which conveys depth through
viewer inside. The film Avatar is a testa- a volumetric light apparent object movement. When
ment to this shift in how moviemakers source so light comes looking out the side window of a mov-
now use stereoscopic 3D, and yet the ing vehicle, for instance, objects be-
movie industry is not alone in embrac- directly from the side the road appear to move past the
ing the technique. simulated distance. window more quickly than objects in
Television manufacturers and the distance. Currently, while movies
broadcasters have fallen under the can render parallax for camera motion
spell of the third dimension, with ste- correctly, they cannot create parallax to
reoscopic 3D TVs and Blu-ray play- account for a viewer’s head movement.
ers now widely available, and new 3D After all, everybody in an audience sees
products expected this year from major attempting to develop new techniques the same image on the screen, despite
manufacturers such as LG, Panasonic, to overcome the limitations associated head movement and regardless of seat
and Sony. ESPN and other broadcast- with traditional stereoscopic 3D strat- position in the theater.
ers are rolling out dedicated 3D cable egies, many of which have remained While it might not be difficult to
channels. Also, the market for stereo- unchanged since the 19th century. New imagine movie theaters one day track-
scopic 3D computers is expected to research has found, for example, spe- ing head movement to render viewer-
grow rapidly, with one million units cific physiological reasons for the visu- based motion parallax correctly, sev-
shipped this year and 75 million by al fatigue that viewing stereoscopic 3D eral fundamental depth-cue issues
2014, according to Jon Peddie Research media sometimes causes. And while have yet to be resolved. One of these is-
(although most of these computers the technology for creating such me- sues is how the human brain perceives
will be stereoscopic 3D-capable due to dia has become more sophisticated, simulated 3D differently from how
their graphics processors, they’ll still the content remains costly to produce it perceives the natural world. In the
require a special monitor, glasses, and and cumbersome to consume, requir- physical world, the distance at which
content). And mobile device-makers ing special cameras, projectors, and each eye’s line of sight must converge,
have begun to incorporate 3D technol- glasses. called the vergence distance, and the
ogy into their handhelds, with the most Kurt Akeley, a principal researcher distance at which the eyes must focus,
recent example being the Samsung at Microsoft Research Silicon Valley, called the focus distance, are the same.
SCH-W960, a smartphone designed to says that while stereoscopic 3D tech- Converging the eyes drives focus to a
convert 2D content automatically into niques and technologies are growing nearer distance, while focusing to a
stereoscopic 3D. more sophisticated, they remain far nearer distance drives the eyes to con-
While the stereoscopic 3D resur- from mature. “I enjoyed viewing Ava- verge, which means that vergence dis-
gence continues to have a powerful tar, and I experienced no discomfort tance and focus distance are coupled
impact on consumer culture, distinct during the three-hour showing, which in the brain.
challenges remain. Researchers work- is a big improvement over previous Stereoscopic 3D media requires that
ing in this area—a field that draws on cinematic experiences,” says Akeley, viewers fix their eyes at simulated dis-
vision science, display technology, vi- who cofounded Silicon Graphics and tances but still focus on the display’s
sualization, and cognitive science—are led the development of OpenGL. “But fixed distance. This disparity causes a

14 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


physiological disconnect that can lead to focus at one distance while looking “We think this is potentially a serious
to headaches and even nausea. To ad- at an object at a different, simulated problem with the distribution of ste-
dress this issue, Akeley has been ex- distance. Still, Akeley remains optimis- reoscopic media, particularly when the
perimenting with an approach related tic about such research. “I’m hopeful viewer’s distance is likely to be short, as
to light-field theory, which he says has that this virtuous cycle of researchers with small TV screens viewed at a short
the potential to lead to new strategies using industry-created equipment to distance,” he says. “We still have lots to
for dealing with this disparity. The idea probe human visual mechanisms and learn about how stereoscopic signals
is to replace the display with a volumet- create useful feedback for industry will affect how people perceive things.”
ric light source so light comes directly accelerate as stereoscopic viewing be- Banks is currently working on how
from the simulated distance, essen- comes the standard,” he says. the presentation of information over
tially eliminating the gap between ver- time affects the perception of motion
gence distance and focus distance. understanding Depth cues and depth cues. In stereoscopic 3D
Despite the promise of using light- Another researcher focused on depth cinema, images are presented to the
field theory to make stereoscopic 3D cues in stereoscopic 3D is Martin left and right eye at 72 cycles per sec-
more comfortable for viewers, the idea Banks, a professor of vision science ond. While the images are presented
has proven to be difficult to implement at the University of California, Berke- in counter-phase to the two eyes, each
in practical applications outside the ley. Banks has conducted widely cited image is shown three times before it
lab. The prototype systems are mainly studies showing how this conflict be- is updated. The update rate is only 24
used to help understand human per- tween fixed display depth and vergence cycles per second, a coarse approxima-
ception and the effects of forcing users distance causes visual discomfort. tion of what it would be in the natural
figure 1a and 1b. a pair of custom-designed dynamic lenses constructed with birefringent material. The lenses are used to create a
volumetric stereoscopic 3D display with four apparent image depths. Rendering illuminates pixels in inverse proportion to their distance
from the simulated distance, creating a seamless sense of depth.




Lens Power = 6.89 D

Focal Demand = 0.7 D


Lens Power = 5.09 D
Lens Power = 6.29 D Focal Demand = 2.5 D
Focal Demand = 1.3 D



Lens Power = 5.69 D

Focal Demand = 1.9 D

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 15

figure 2. in the natural world, focus distance (the distance to which the eyes must focus to only will eliminate the need to wear
make an image sharp) and vergence distance (the distance at which the eyes’ lines of sight special glasses, but also will have the
converge on an object) are the same. however, most stereoscopic 3D displays require view- ability to track head movement to ren-
ers to point their eyes at simulated distances while still focusing on the display’s actual
der motion parallax accurately. And he
fixed distance. This incongruity can cause headaches and even nausea.
predicts the proliferation of more pow-
erful content-creation technologies,
Real world Stereo display
such as movie-production systems that
can render a scene from different view-
points without reshooting it, and an
overall better understanding of vision
fatigue related to focus and depth cues.
With these and other technological ad-
vergence distance

vergence distance
vances, 3D viewing experiences will be
focal distance

focal distance
surface greatly improved, whether they occur
on big screens or small ones.

Further Reading
Akeley, K., Watt, S.J., Girshick, A.R.,
and Banks, M.S.
A stereo display prototype with multiple
focal distances. ACM Transactions on
world. Banks is studying how the visual other depth cues, but also to under- Graphics 23, 3, August 2004.
system can tolerate such slow updates stand how changes in that relationship
Hoffman, D.M., Girshick, A.R., Akeley, K.,
and how viewers perceive such signals affect human perception. The results and Banks, M.S.
to be smooth and convincing. of such investigations could influence Vergence-accommodation conflicts hinder
“For these studies, it would be use- the design of content for stereoscopic visual performance and cause visual fatigue.
ful to have faster display technology 3D cinema and television. Journal of Vision 8, 3, March 28, 2008.
than we currently have,” Banks says. Banks predicts that, despite the Love, G.D., Hoffman, D.M., Hands, P.J.W.,
“With such technology, we would be abundance of unanswered questions Gao, J., Kirby, A.K., and Banks, M.S.
able to better understand the conse- about how human perception works high-speed switchable lens enables the
development of a volumetric stereoscopic
quences of using different temporal with simulated 3D, the technique will display. Optics Express 17, 18, August 2009.
protocols in the presentation of stereo- continue its momentum in movie-
Mendiburu, B.
scopic video.” making. He also predicts that display
3D Movie Making: Stereoscopic Digital
In a related project, Banks is study- update rates will improve to the point Cinema from Script to Screen. Focal Press,
ing how blur affects the perception of where motion looks truly smooth, and Burlington, MA, 2009.
distance and size. Conventional opti- vision sciences will continue to im- Watt, S.J., Akeley, K., Ernst, M.O.,
cal devices, such as eyes and cameras, prove colors so they look more like the and Banks, M.S.
can be focused only on one distance natural world. “We’re a long way from Focus cues affect perceived depth. Journal
at a time, which makes objects blurry achieving these goals,” he says. “But of Vision 5, 10, December 15, 2005.
when they are farther from or nearer to once we do, the experience of watching
the focus distance. Banks is conduct- video will be truly breathtaking.” based in los angeles, Kirk L. Kroeker is a freelance
editor and writer specializing in science and technology.
ing studies to determine the relation- Microsoft Research’s Akeley, for his
ship between depth-of-field blur and part, predicts future 3D displays not © 2010 aCM 0001-0782/10/0800 $10.00


Automated Debugger for Parallel Programs

purdue university researchers, puter code for complex parallel inefficient to locate the bug. the primary developers of the
collaborating with high- programs. Automaded’s approach program are Lawrence Livermore
performance computing experts International treaties forbid involves grouping a large number scientist greg Bronevetsky
at Lawrence Livermore national the detonation of nuclear of processes into a smaller and purdue doctoral student
Laboratory, have created an test weapons, so certification number of “equivalence classes” Ignacio Laguna, and the paper,
automated debugging program is performed with complex with similar traits, which keeps “Automaded: Automata-Based
for the simulations used to simulations. these simulations the analysis simple enough so debugging for dissimilar parallel
certify nuclear weapons. can last several weeks, and it it can be performed when the tasks,” was presented at the 40th
called Automaded (for is common for an error in the simulation is running. It also Annual Ieee/IFIp International
automata-based debugging for simulation code to not become operates by splitting a simulation conference on dependable
dissimilar parallel tasks), the evident until long after it occurs, into numerous windows of time, systems and networks in chicago.
program finds errors in com- which makes it difficult and called phases. –Jack Rosenberger

16 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


Society | DOI:10.1145/1787234.1787242 Sarah Underwood

making Sense of
Real-Time Behavior
Data captured by sensors worn on the human body and analyzed in near real-time
could transform our understanding of human behavior, health, and society.

s t h e p RoLIFe RAtI on of
smartphones and net-
works puts a mobile sen-
sor in many people’s
pockets, more intimate
wearable sensors are playing a leading
role in understanding human behav-
ior to an extent that has not previously
been possible.
The sensors can be integrated into
a variety of wearable items, such as
badges, plastic tags on lanyards, and
sticky plasters, but their commonality
is in collecting fine-grained data that
can be computed to visualize real-time
behavior. With in-depth behavioral
knowledge, applications can be devel-
oped that make the best use of behav-
ioral patterns or monitor behavior to
ensure and potentially improve human alex Pentland, right, and a team of researchers with reality-mining devices at massachusetts
well being. institute of Technology’s media Lab.
Alex “Sandy” Pentland, a professor
at Massachusetts Institute of Technol- voice, body language, and location data. office layout and recognize desirable
ogy’s Media Lab who specializes in The real-time data would be sent to a changes, such as moving employees or
computational social science, suggests central server and analyzed to detect knocking down walls to improve com-
real-time collection and analysis of data patterns of mistakes, perhaps identify- munication.
about people, a discipline he calls “re- ing stress in the tone of voice as a pre- Pentland acknowledges that such
ality mining,” could transform the way cursor to mistakes. data-capture schemes could face op-
we understand ourselves and society A similar solution can be used to im- position from staff, but says that most
(although he adds the caveat that reality prove employee productivity. Pentland buy in if their boss is included and they
mining will raise the bar on data privacy points out that as corporations strive for can view the data. Pentland goes one
and ownership). greater productivity they often structure step further by giving staff and manag-
Pentland started research on real- staff time to reduce personal conversa- ers control over the data. “It’s spying
ity mining in the mid-1990s, leading a tions. “At more than a dozen companies that people don’t like,” he says. “If their
team that used large wearable devices where we have used wearable sensors, information is open to them and they
to track human behavior. “Reality min- social interaction has been found to have control of it, people are generally
ing is about understanding people and be an important element in productiv- much happier.”
situations very rapidly,” he explains. “As ity,” says Pentland. “Staff who socialize
computational social scientists we can trade information about how to do their Popular clubs and events
build a better model of how people be- jobs better. People who are cut off aren’t Beyond Pentland’s work, reality min-
have than psychologists or sociologists wise in the ways of the company and ing is reaching the market through
because we can actually see what peo- don’t have social support.” companies such as Hitachi Consulting,
PHotoGra PH by rICK F rIed Ma n

ple are doing.” Having discovered the business ben- which is using smart badges on client
Pentland suggests that one of reality efits of socializing, companies typically projects similar to those undertaken by
mining’s uses could be to reduce hu- reorganize, structuring rest periods Pentland, and through MIT Media Lab
man errors in hospitals. Every day, hos- and coffee breaks to support social in- spin-offs. The latter includes Sense Net-
pital staff would wear sensor-equipped teraction. Data from this kind of reality works, which analyzes mobile location
name badges that capture their tone of mining can also be analyzed to evaluate data from cellphones, taxis, cameras,

AU g U ST 2 0 1 0 | VOl. 53 | N O. 8 | c om m u n ic aTion S of T he acm 17


and GPS devices to create a real-time into human health as the vital sign data
map of popular clubs and events to captured in real time from patients
provide answers to consumers’ ques- Digital plasters will could be correlated to show trends in
tions, such as “Where is everyone this allow “patients to patient health that cannot be readily
evening?” understood using traditional methods
Another spin-off is Cogito Health, leave the hospital, of data capture that are bulky and have
which is developing Pentland’s re- but continue to be limited portability. Correlation could
search to extract meaning from speech also provide an extra dimension to
behavior. One application that uses monitored by alerts, as the analysis of multiple, vital
Cogito’s vocal signaling platform is a healthcare sign data could predict adverse patient
depression-monitoring service that effects that may occur in hours or even
automatically processes care-manage- professionals,” says days.
ment telephone calls and allows health- chris Toumazou. As trials of the digital plaster con-
care professionals to proactively iden- tinue at St. Mary’s Hospital, further ap-
tify patients who may need support for plications of the Sensium technology
clinical depression. are anticipated, with Toumazou and
An MIT Media Lab spin-off is nTag, a his team at the Institute of Biomedical
company founded in 2002 that fell into Engineering currently working with the
bankruptcy before being acquired by Al- technology to create a diabetes manage-
liance Tech in March 2009. lege, London, has developed the tech- ment system that, like the digital plaster
Alliance Tech focuses on market- nology behind a digital plaster concept technology, is expected to be licensed to
ing intelligence at events such as trade that is being used in patient trials at St. a commercial partner.
shows and conferences. It offers event Mary’s Hospital in London. Realizing
organizers sensor-laden handheld de- the constraints of existing wireless tech-
Further Reading
vices that can be used by event partici- nologies, such as Bluetooth and Zigbee,
pants. The devices are worn on lanyards particularly their high-power require- Burdett, A., Lakhanpal, A., McPartland, R.,
Nunn, C., McDonagh, D., Silveira, M.H.
and support not only real-time attendee ments and that any device built using
Key considerations and experience using
tracking, but also social networking. them would be bulky, obtrusive, and the ultra-low-power Sensium platform in
Conference attendees, for example, possess a short operating life, Toum- body sensor networks. Sixth International
can exchange contact information by azou set out to create a low-power tech- Workshop on Wearable and Implantable
pressing a button on their nTag device nology that could capture data from a Body Sensor networks, Berkeley, CA, June
3–5, 2009.
and later access that information on the body, even if it was ambulatory.
event’s Web site, or they can be alerted “We have commercialized ultra low- Olguin Olguin, D., Pentland, A.
Sensible organizations: a sensor-based
by an nTag vibration if they are close to power wireless systems and signal pro-
system for organizational design and
someone who, according to preselected cessing, putting them together on a engineering. International Workshop on
criteria, they want to meet. For event single chip,” says Toumazou. “We call Organizational Design and Engineering,
organizers, real-time data capture and this the Sensium technology platform Lisbon, Portugal, December 11–12, 2009.
analysis means changes can be made to which sensors can be attached. This Olguin Olguin, D., Gloor, P., Pentland, A.
on the fly during an event, the success is the basis of the digital plaster.” Wearable Sensors for Pervasive healthcare
of the event can be measured, and pa- The digital plaster technology, Management. Third International
perwork can be reduced by using nTag which has been developed by Toumaz Conference on Pervasive Computing
Technologies for healthcare, London, U.K.,
for activities such as attendee feedback. but is expected to be licensed and com- April 2009.
The nTag solution uses proprietary mercialized, includes a plaster or patch
Pappas, C.
hardware, but Alliance Tech CEO Art that sticks to the body and captures vital Three problems, one solution: attendee
Borrego says future products will move sign data from patients. The data is for- surveillance. Corporate Event Magazine,
away from such technology. “Why in- warded to a hospital information sys- Summer 2009.
vest in more hardware when we could tem, where it is analyzed, interpreted, Pentland, A.
now use what is in people’s pockets?” and delivered to a nurse or doctor. Honest Signals: How They Shape our World.
Borrego asks. “Our next generation “Because of the economies of scale MIT Press, Cambridge, MA, 2008.
technology will use smartphones and a of semiconductors we can drive down Wong, A.C.W., McDonagh, D., Omeni, O.,
micro browser.” cost and make digital plasters dispos- Nunn, C., Hernandez-Silveira, M., Burdett, A.J.
As smartphones become the sensor able, avoiding problems such as infec- Sensium: An ultra-low-power wireless
body sensor network platform: design &
for many reality-mining applications, tion, the need to sterilize plasters, or
application challenges. Proceedings of the
wearable sensors are likely to prevail in the need to recharge their batteries,” Annual International Conference of IEEE
the medical field where the automated explains Toumazou. “Ultimately, this Engineering in Medicine and Biology Society,
analysis of real-time data captured by technology will allow patients to leave Minneapolis, Mn, September 3–6, 2009.
sensors could prove transformational. the hospital, but continue to be moni-
Chris Toumazou, CEO of Toumaz tored by healthcare professionals.” Sarah Underwood is a technology writer based in
teddington, u.K.
and a professor at the Institute of Bio- Continuous ambulatory monitoring
medical Engineering at Imperial Col- could also provide important insights © 2010 aCM 0001-0782/10/0800 $10.00

18 comm unicaTio nS o f T he ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


Conference | DOI:10.1145/1787234.1787261 Kirk L. Kroeker

celebrating the
Legacy of PLaTo
The PLATO@50 Conference marked the semicentennial
of the computer system that was the forerunner of today’s
social media and interactive education.

n eARLy June, the Computer His-
tory Museum hosted the PLA-
TO@50 Conference to mark
the 50-year anniversary of the
computer system that many
credit with presaging the networked
world of social media and interactive
education that has become a mainstay
of contemporary culture. More than
400 computing enthusiasts attended
the event, the highlight of which was
a moderated discussion between Don
Bitzer, the inventor of PLATO, and Ray
Ozzie, Microsoft’s chief software ar-
The event included several pre-
sentations and a reunion of PLATO
alumni from across the U.S. As a spe-
cial part of the conference, function-
ing PLATO terminals were set up in an
interactive “PLATO playground” for
use by conference participants. The
terminals included many of the origi-
nal PLATO games, such as Avatar and
Empire, and an array of social media
content preserved from the original
PLATO databases.
“PLATO@50 was an important
event for the museum to convene,”
says John Hollar, president of the
Mountain View, CA-based museum.
“It traced the history of a system that
produced the forerunners to an amaz-
PHotoGra PH Court esy oF C arIna sW eet, CoM Puter H Isto ry M useuM

ing array of technologies—hardware,

software, and applications—that live
on today, either in original or deriva-
tive form.” The conference, says Hol-
lar, highlighted how PLATO in the
functioning PLaTo terminals were set up for use by PLaTo@50 conference participants.
Computer-based Education Research The terminals included many of the first PLaTo games and an array of social media preserved
Laboratory (CERL) at the University from the original PLaTo databases.
of Illinois at Urbana-Champaign
produced an accomplished group of In addition to keynote speeches and conference paid tribute to a culture of
alumni, including Ray Ozzie, whose moderated discussions, PLATO@50 innovation and openness fostered by
initial design for Lotus Notes derived documented an array of events that Don Bitzer that clearly had an impact
from PLATO Notes, and Marc An- occurred at CERL from its inception in on a large number of engineering and
dreessen, the co-author of Mosaic and 1967 to its closing in 1994. “That story computer science majors who passed
founder of Netscape. is important to tell,” says Hollar. “The through CERL and the University of

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 19

mated Teaching Operation, began as in 1972. “Nobody else had that,” he

a project at the University of Illinois says. “PLATO was ahead of everyone
“exploring the at Urbana-Champaign and was even- by years, and deserves far more credit
history of PLaTo tually commercialized in the 1970s than it has gotten.”
by Control Data Corporation. Over Sheldon Hochheiser, institutional
and the wild story the years, the system developed into historian for the IEEE History Center
of the people who a sophisticated, networked comput- at Rutgers University, offers a similar
ing infrastructure consisting of smart perspective. “Bitzer’s work was, to my
designed, built, and terminals that could run games, chat mind, a true innovation and far ahead
used it will enrich rooms, and courseware. It was used of its time,” says Hochheiser. “He put
for education and training by many together a package of existing and in-
everyone’s overall schools and universities, corpora- vented technology with educational
perspective on the tions, and by several branches of pedagogy to develop a new way of in-
the U.S. government, including the struction, whose descendents have
world’s embrace military. spread far and wide.”
of the internet and Brian Dear, the conference’s pri- While the mainstream technical
mary organizer, worked on PLATO as a media focuses on the history of online
computers today,” programmer, analyst, and courseware communities largely through study-
says Donald Dear. designer from 1979 to 1984. “I was ing the birth of the Internet by way of
amazed by the system and completely ARPANET and its descendents, Dear
ignored the microcomputer revolu- maintains that approach misses an
tion going on at the time because here extraordinary segment of the com-
was a computer that was all about puter history timeline. “PLATO for
connecting people, whereas micros years served as an excellent predic-
were lonely islands of BASIC program- tor for how the Internet would evolve,
Illinois.” One such alumnus is Marc mers and spreadsheet enthusiasts,” and its impact is everywhere,” he says.
Andreessen, who considers himself he says. “I loved Steve Levy’s Hackers: “Exploring the history of PLATO and
fortunate to be a product of that envi- Heroes of the Computer Revolution, but the wild story of the people who de-
ronment. where is volume two about the other signed, built, and used it will enrich
In computing history, the story of heroes of the computer revolution— everyone’s overall perspective on the
this educational computer system is namely, the PLATO people?” world’s embrace of the Internet and
little known outside of the circles of Dear, whose book on PLATO, titled computers today.”
people who either directly used the The Friendly Orange Glow, is due later
system or helped to build it. Invented this year, says PLATO was far ahead based in los angeles, Kirk L. Kroeker is a freelance
editor and writer specializing in science and technology.
by Bitzer in 1960, PLATO, an acro- of its time with gas-plasma, flat-pan-
nym for Programmed Logic for Auto- el displays sporting touch screens © 2010 aCM 0001-0782/10/0800 $10.00

Artificial Intelligence

Computer Scientists Beat U.S. Stock Market

A pair of computer scientists an enormous computational amount of data.” six of the 10 leading quantitative
has developed an artificial advantage over a lone human AZFintext’s technique funds. schumaker and chen also
intelligence program that, in analyst with its ability to scan involves surveying financial used quantitative strategies to
simulated trading, outperforms large volumes of financial news news and stock prices, and produce a stock portfolio and
the standard & poor’s (s&p) and analyze the words in the text. buying or shorting stocks it employed AZFintext to make
500 Index and six of the top 10 current computer-aided believes will increase by more trades. With this quantitative
quantitative funds. quantitative funds analyze than 1% in the next 20 minutes. approach, they realized a return
developed by Robert p. numerical data, not text, and AZFintext sells the stocks after in excess of 20%.
schumaker, an assistant schumaker and chen’s program 20 minutes. “When you do long- chen expects AZFintext to be
professor of information systems is unique in that it takes a term predictions, there are many deployable for real-world trading
at Iona college, and hsinchun human analyst’s approach of variables,” chen said. “But … you in two to five years.
chen, mcclelland professor deciphering financial news. can have an advantage if you look the researchers have
of management Information In terms of its ability to at five minutes, 10 minutes.” expanded their field of research
systems and director of the analyze one news article, schumaker and chen tested and are now devising systems
Artificial Intelligence Lab at AZFintext “won’t be as accurate AZFintext with five weeks of that can analyze financial
the university of Arizona, the as an individual analyst,” chen data from the fall of 2005, which writers’ sentiments, and are
Arizona Financial text system, told The Wall Street Journal in a included more than 9,000 news analyzing not just traditional
or AZFintext, analyzes the text recent interview. “the computer articles and 10 million stock financial news outlets, but blogs
of financial news in the same is maybe 80% to 85% accurate prices. AZFintext produced and employee and investor
fashion as a Wall street analyst. when analyzing text, but it can an 8.5% return on its trades, forums.
however, AZFintext boasts read maybe 100,000 times the outperforming the s&p 500 and —Jack Rosenberger

20 com municaTio nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


Milestones | DOI:10.1145/1787234.1787267 Jack Rosenberger

Gödel Prize and

other cS awards
Sanjeev Arora, Joseph S.B. Mitchell, and other researchers
are recognized for their contributions to computer science.

h e e u R o p e A n A s s o c I At I o n
for Theoretical Computer
Science (EATCS) and the
ACM Special Interest Group
on Algorithms and Compu-
tation Theory (SIGACT), the British
Computer Society, and other organi-
zations recently honored select scien-
tists for their contributions to com-
puter science.

Gödel Prize
In recognition of their outstanding pa-
pers in theoretical computer science,
EATCS and ACM SIGACT awarded the
2010 Gödel Prize to Sanjeev Arora, a
professor of computer science at Princ-
eton University, and Joseph S.B. Mitch- Gödel Prize winners Sanjeev arora, left, and Joseph S.B. mitchell.
ell, a professor in the Department of
Applied Mathematics and Statistics ter for Field Robotics at the University ematical Sciences, to receive the 2010
at the State University of New York at of Sydney, was honored for his major Alan T. Waterman Award. Considered
Stony Brook, for their concurrent dis- contributions to robotics, in particu- the NSF’s most prestigious honorary
covery of a polynomial-time approxi- lar to the fields of sensor data fusion award since its establishment in 1975,
mation scheme for the Euclidean Trav- and of autonomous vehicle navigation. it is given annually to an outstanding
eling Salesman Problem. Georg Gottlob, a professor of comput- researcher under the age of 36 in any
ing science at the University of Oxford, field of science and engineering sup-
Roger needham award was honored for his fundamental con- ported by NSF. A theoretical computer
and Lovelace medal tributions to both artificial intelligence scientist, Khot works in the area of
The British Computer Society (BCS) and database systems. computational complexity and seeks
presented the Roger Needham Award to understand the power and limits of
to Joël Ouaknine of the Oxford Univer- Gerhard herzberg medal efficient computation.
sity Computing Laboratory in recogni- The Natural Sciences and Engineer-
PHotoGra PH by ( rIGH t) tony sCarl ato s, ( leF t) F roM ItC s, tsInGH ua un IVersIt y

tion of his seminal and mathematical ing Research Council of Canada be- Benjamin franklin medal
contributions to the field of timed stowed the Gerhard Herzberg Canada The Franklin Institute presented the
systems modeling and analysis. BCS’s Gold Medal, the nation’s top medal 2010 Benjamin Franklin Medal in
Lovelace Medal was presented to John for science and engineering, to Gilles Computer and Cognitive Science to
Reynolds, a professor at the School of Brassard, Canada Research Chair in Shafrira Goldwasser, RSA Professor of
Computer Science, Carnegie Mellon Quantum Information Processing at Computer Science and Engineering at
University in recognition of his work the Université de Montréal. Brassard is Massachusetts Institute of Technology
of the last four decades and his contri- one of the inventors of quantum cryp- and professor of computer science and
bution to the theory of programming tography and a pioneer in the field of mathematics at Weizmann Institute of
languages. quantum information science. Science, for her fundamental contribu-
tions to the theoretical foundation of
Royal Society fellows alan T. Waterman award modern cryptography.
The 44 newly elected 2010 Fellows of The U.S. National Science Foundation
the Royal Society include two comput- (NSF) selected of Subhash Khot, an Jack Rosenberger is senior editor, news, of
er scientists. Hugh Francis Durrant- associate professor at New York Uni-
Whyte, director of the Australian Cen- versity’s Courant Institute of Math- © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VOl. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 21

ACM, Advancing Computing as
ACM, Advancing
a Science and Computing
a Professionas
a Science and a Profession
Dear Colleague,
Dear Colleague,
The power of computing technology continues to drive innovation to all corners of the globe,
bringing with it opportunities for economic development and job growth. ACM is ideally positioned
The power of computing technology continues to drive innovation to all corners of the globe,
to help computing professionals worldwide stay competitive in this dynamic community.
bringing with it opportunities for economic development and job growth. ACM is ideally positioned
to help computing professionals worldwide stay competitive in this dynamic community.
ACM provides invaluable member benefits to help you advance your career and achieve success in your
chosen specialty. Our international presence continues to expand and we have extended our online
ACM provides invaluable member benefits to help you advance your career and achieve success in your
resources to serve needs that span all generations of computing practitioners, educators, researchers, and
chosen specialty. Our international presence continues to expand and we have extended our online
resources to serve needs that span all generations of computing practitioners, educators, researchers, and
ACM conferences, publications, educational efforts, recognition programs, digital resources, and diversity
initiatives are defining the computing profession and empowering the computing professional.
ACM conferences, publications, educational efforts, recognition programs, digital resources, and diversity
initiatives are defining the computing profession and empowering the computing professional.
This year we are launching Tech Packs, integrated learning packages on current technical topics created and
reviewed by expert ACM members. The Tech Pack core is an annotated bibliography of resources from the
This year we are launching Tech Packs, integrated learning packages on current technical topics created and
renowned ACM Digital Library – articles from journals, magazines, conference proceedings, Special Interest
reviewed by expert ACM members. The Tech Pack core is an annotated bibliography of resources from the
Group newsletters, videos, etc. – and selections from our many online books and courses, as well an non-
renowned ACM Digital Library – articles from journals, magazines, conference proceedings, Special Interest
ACM resources where appropriate.
Group newsletters, videos, etc. – and selections from our many online books and courses, as well an non-
ACM resources where appropriate.
Timely accessAN
relevant YOU RECEIVE:
Communications of the ACM magazine • ACM Tech Packs • TechNews email digest • Technical Interest Alerts and
Timely access to relevant information
ACM Bulletins • ACM journals and magazines at member rates • full access to the acmqueue website for practi-
Communications of the ACM magazine • ACM Tech Packs • TechNews email digest • Technical Interest Alerts and
tioners • ACM SIG conference discounts • the optional ACM Digital Library
ACM Bulletins • ACM journals and magazines at member rates • full access to the acmqueue website for practi-
tioners • ACM SIG conference discounts • the optional ACM Digital Library
Resources that will enhance your career and follow you to new positions
Career & Job Center • online books from Safari® featuring O’Reilly and Books24x7® • online courses in multiple
Resources that will enhance your career and follow you to new positions
languages • virtual labs • e-mentoring services • CareerNews email digest • access to ACM’s 34 Special Interest
Career & Job Center • online books from Safari® featuring O’Reilly and Books24x7® • online courses in multiple
Groups • an email forwarding address with spam filtering
languages • virtual labs • e-mentoring services • CareerNews email digest • access to ACM’s 34 Special Interest
Groups • an email forwarding address with spam filtering
ACM’s worldwide network of more than 97,000 members ranges from students to seasoned professionals and
includes many renowned leaders in the field. ACM members get access to this network and the advantages that
ACM’s worldwide network of more than 97,000 members ranges from students to seasoned professionals and
come from their expertise to keep you at the forefront of the technology world.
includes many renowned leaders in the field. ACM members get access to this network and the advantages that
come from their expertise to keep you at the forefront of the technology world.
Please take a moment to consider the value of an ACM membership for your career and your future in the
dynamic computing profession.
Please take a moment to consider the value of an ACM membership for your career and your future in the
dynamic computing profession.

Alain Chesnais
Alain Chesnais
Association for Computing Machinery
Association for Computing Machinery

Advancing Computing as a Science & Profession

Advancing Computing as a Science & Profession

membership application &
Advancing Computing as a Science & Profession
digital library order form
Priority Code: AD10

You can join ACM in several easy ways:

Online Phone Fax +1-800-342-6626 (US & Canada) +1-212-944-1318
+1-212-626-0500 (Global)
Or, complete this application and return with payment via postal mail

Special rates for residents of developing countries: Special rates for members of sister societies:
Please print clearly
Purposes of ACM
ACM is dedicated to:
1) advancing the art, science, engineering,
and application of information technology
2) fostering the open interchange of
Address information to serve both professionals and
the public
3) promoting the highest professional and
City State/Province Postal code/Zip ethics standards
I agree with the Purposes of ACM:
Country E-mail address


Area code & Daytime phone Fax Member number, if applicable ACM Code of Ethics:

choose one membership option:

o ACM Professional Membership: $99 USD o ACM Student Membership: $19 USD

o ACM Professional Membership plus the ACM Digital Library: o ACM Student Membership plus the ACM Digital Library: $42 USD
$198 USD ($99 dues + $99 DL) o ACM Student Membership PLUS Print CACM Magazine: $42 USD
o ACM Digital Library: $99 USD (must be an ACM member) o ACM Student Membership w/Digital Library PLUS Print
CACM Magazine: $62 USD

All new ACM members will receive an payment:

ACM membership card. Payment must accompany application. If paying by check or
For more information, please visit us at money order, make payable to ACM, Inc. in US dollars or foreign
currency at current exchange rate.
Professional membership dues include $40 toward a subscription
to Communications of the ACM. Member dues, subscriptions, o Visa/MasterCard o American Express o Check/money order
and optional contributions are tax-deductible under certain
circumstances. Please consult with your tax advisor.
o Professional Member Dues ($99 or $198) $ ______________________

o ACM Digital Library ($99) $ ______________________

o Student Member Dues ($19, $42, or $62) $ ______________________
Association for Computing Machinery, Inc.
General Post Office Total Amount Due $ ______________________
P.O. Box 30777
New York, NY 10087-0777

Questions? E-mail us at Card # Expiration date

Or call +1-800-342-6626 to speak to a live representative

Satisfaction Guaranteed! Signature


DOI:10.1145/1787234.1787243 Christopher S. Yoo

economic and
Business dimensions
Is the Internet
a maturing market?
If so, what does that imply?

Wo conceRns domInAte the holds are not subscribing to broad- Interestingly, management litera-
current debates over U.S. band even when it is available. ture exists suggesting that both devel-
Internet policy. The first is The second is the debate over net- opments may simply reflect the ways
the relatively low level of work neutrality. Network providers are the nature of competition and innova-
U.S. broadband adoption. experimenting with a variety of new tion can be expected to evolve as mar-
Although the U.S. once ranked 4th business arrangements. Some are of- kets mature. If applicable to the Inter-
among industrialized nations in the fering specialized services that guar- net, this literature has the potential to
percentage of residents subscribing antee higher levels of quality of service provide new insights into how to craft
to broadband, it has currently slipped to those willing to pay for it. Others are broadband policy and what steps busi-
into 15th place. Concerns that the U.S. entering into strategic partnerships ness managers might take to prepare
may be losing its leadership position that allocate more bandwidth to cer- for the future.
in this key industry have spurred a se- tain sources and applications.
ries of governmental initiatives to ad- Demand-Side considerations:
dress the problem. The stimulus pack- Product Life cycle Theory
age enacted during the initial days of The real question The best-known theory of market mat-
the Obama administration dedicated uration is known as the product life
$7.2 billion for new investments in is not if the nature cycle. A central feature of every leading
broadband infrastructure. It also re- of competition and marketing textbook, product life cycle
quired the Federal Communications theory examines how the pattern of
Commission to prepare a national innovation will demand growth affects the nature of
broadband plan, which the agency change, but rather competition over time. Empirical re-
released to much fanfare this past search has confirmed that many, if not
March. The plan is designed not just how and when. most, markets follow the pattern pre-
to ensure that broadband is available dicted by product life cycle theory.
and affordable to all Americans, but The predominant version posits
also to devise ways to address the fact that new product markets pass through
that a surprising number of house- four distinct stages shown in the prod-

24 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


en t
lo p m
D e ve
ar t in

uct life cycle figure here. During the in- nological features and to deemphasize are not yet in the market. Instead, firms
troduction stage, the product’s novelty product quality and price, once again, focus on finding ways to deliver greater
dictates that sales are small and grow as theory predicts. value to customers who are already in
relatively slowly. If a market for the As the market transitioned into the the market.
new product develops, this initial stage growth phase, firms began to target According to this theory, there is
gives way to the growth stage, during the mass market and to compete to nothing surprising about the preva-
which sales grow rapidly. Over time, attract new customers who are not yet lence of offering a more complex ar-
market saturation causes the prod- being served. Price and quality took on ray of services and price points. These
uct to enter the maturity stage, during greater importance. In order to keep firms are trying to increase revenue in
which sales growth flattens. Eventual- production processes simple and to their primary market and set them-
ly, the product enters the decline stage, make the product easy for customers selves up to offer new services that gen-
as technologically superior substitutes to understand, firms typically offered erate more revenue. That also explains
emerge. The nature of competition a single product designed to appeal to why industry leaders such as Yahoo,
changes as the market advances from the broadest possible audience. Google, Apple, and Microsoft are be-
one stage to the next. After enjoying an extended period coming more aggressive about invad-
Internet usage over the last two de- of rapid growth, there are some indi- ing territory traditionally controlled by
cades fits comfortably into this pattern. cations that the market is on the cusp other leading firms. It is a natural out-
During the introduction stage of the of entering the maturity phase. U.S. growth of maturity and the natural in-
broadband Internet during the mid-to- Internet penetration has leveled off at crease in rivalry that results when firms
late 1990s firms focused on inducing approximately 75%. When one focuses compete in a market that is no longer
early adopters to try the product, as the solely on broadband, data collected growing as fast as it once did.
IllustratIon by da rrel rees

theory predicts. Early adopters tend to by the FCC suggests that the growth
be technologically sophisticated, risk curve has passed the inflection point Supply-Side considerations:
tolerant, and price insensitive, which marking the transition from growth to Dominant Design Theory
describes the typical Internet user circa maturity. As the market enters the ma- A parallel line of research in this
15 years ago. This focus in turn caused turity phase, revenue growth no longer framework explores the supply-side of
firms to emphasize cutting-edge tech- depends on attracting customers who market maturation. Called dominant

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 25

The product life cycle. This theory also suggests that poli-
cymakers should be careful not to lock
the Internet into any particular archi-
sales tecture or to reflexively regard devia-
introduction Growth maturity Decline
tions from the status quo as inherently
anticompetitive. Such measures would
reinforce the obstacles to architectural
innovation that already exist. Instead,
they should focus on creating regula-
tory structures that preserve industry
participants’ freedom to experiment
with new solutions and to adapt to
changing market conditions. Any other
approach risks precluding the industry
from following its natural evolutionary
path and rendering the obstacles to ar-
chitectural innovation that already ex-
design theory, it posits that when a inconsistent with the existing archi- ist all but insuperable.
technological breakthrough first oc- tecture and can delay or prevent new Applying market maturation theory
curs, uncertainty fosters lack of prod- architectures from evolving. to the Internet comes with a number
uct standardization, which provides What does that have to do with the of limitations. Although the pattern
little incentive to invest in advanced Internet? A growing number of tech- of sales growth predicted by product
production processes. At some point nologists have noted the core architec- life cycle theory is the most common,
the basic product features and techno- ture for the Internet, built around TCP/ empirical research indicates that oth-
logical characteristics coalesce into a IP and its many extensions, is several er patterns exist as well, which leads
dominant design. Innovation becomes decades old. They suggest the new de- some to question the theory’s general-
less driven by trial and error and in- mands being placed on the network ity. Others condemn these theories as
stead becomes more systematic and are creating the need for fundamen- self-fulfilling prophecies, as their wide-
incremental. Other scholars have ex- tally different design architecture. And spread acceptance leads firms to man-
tended this analysis, suggesting that as this theory would predict, they are age their products in ways that cause
technological guideposts or paradigms finding that the standardization on a these patterns to come true. Moreover,
emerge that direct research along par- certain approach combined with the while key turning points are easy to
ticular avenues or trajectories. These interconnected nature of the technolo- identify in retrospect, they have proven
technological trajectories frame the gies comprising the architecture is lim- quite challenging to anticipate far in
way each field determines which prob- iting the Internet’s ability to evolve to advance.
lems are worth solving and which tech- meet these new demands. Even if it is not always possible to
nological solutions are likely to be the anticipate precisely how the nature
most promising. This impetus toward Significance for internet Policy of competition and innovation will
certain trajectories becomes more pro- and Business Strategies change, that both will change over time
nounced if a technology is embedded The implications are myriad. The trans- is a given. The real question is not if the
in a web of interdependent technologi- formation of the Internet from an ex- nature of competition and innovation
cal processes. The presence of such a perimental testbed into a mass-market will change, but rather how and when.
design hierarchy establishes a techni- platform has made major architectural Business managers and IT profession-
cal agenda that channels subsequent change more difficult, just as design als must not take for granted that the
innovation along particular lines. It hierarchy theory would predict. The competitive dynamics and the technol-
also obstructs innovations that are flattening of revenue growth inevitably ogy underlying the industry today will
gives network providers incentive to ex- still be in place tomorrow. Instead,
periment with increasingly specialized they should look for indications that
applying market equipment, both to lower costs and to the market may be reaching saturation
offer services targeted at particular sub- and plan for how their strategy and
maturation theory groups of customers, just as product those of their customers and competi-
to the internet life cycle theory would predict. The de- tors are likely to change as these phase
sire to provide greater value to custom- transformations occur.
comes with a number ers is creating greater interest in facili-
of limitations. tating content providers’ long-standing Christopher S. Yoo ( is Professor
of law, Communication, and engineering and director of
interest in monetizing content streams. the Center for technology, Innovation, and Competition
At the same time, market maturation is at the university of Pennsylvania. For a more extensive
presentation of these ideas, see “Product life Cycle
causing firms to place greater empha- theory and the Maturation of the Internet,” Northwestern
University Law Review, 104:2 (forthcoming 2010).
sis on capturing a bigger fraction of the
dollars that are available. Copyright held by author.

26 com municaTio nS o f T h e acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8


DOI:10.1145/1787234.1787244 David S. Touretzky

preparing computer
science students for
the Robotics Revolution
Robotics will inspire dramatic changes in the CS curriculum.

egInnIng In the 1970s, a se-
ries of technological ad-
vances in computing has
repeatedly reshaped the
undergraduate computer
science curriculum. Affordable bit-
mapped displays brought GUI inter-
faces into widespread use, gave us the
e n t
new field of human-computer interac-
tion, and led CS departments to intro-

e ve lo
in D
duce courses in computer graphics and

HCI. The maturation of networking
technology that led to the Internet and
the Web also spawned a whole spec-
trum of new courses, from the nuts
and bolts of network protocols to the
social impacts of online communities.
The microprocessor that launched the
personal and then wearable computer
revolutions, and in conjunction with
the growth of wireless networks, pro-
duced new types of platforms that are
always on and always with us, has led
to courses targeting smartphones and
PDAs instead of conventional com-
PHotoGra PH by et Ha n J. t Ira- tH oMPson/Ca rneGIe Mel lo n

puters. And when inexpensive graph-

ics processors and sound cards grew
electronic gaming into a multibillion-
dollar business with revenues compa-
rable to the film and music industries,a
CS departments responded by intro- calliope: a prototype create/aSuS robot with a pan/tilt camera and gripper arm.
ducing a variety of multidisciplinary
courses in game design.9 Robotics is the leading candidate parable to what microprocessors did
for the next dramatic change in the for computing three decades ago. In
a Estimated 2008 revenues from
CS curriculum. Advances in sensing, a 2007 Scientific American article, Bill
motion pictures $33 billion; music $15 billion; actuator, and power technologies are Gates drew a parallel between today’s
computer and electronic games $12 billion. fueling an explosion in robotics com- robotics industry and the computing

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 27

industry at the start of the PC revolu- Technology maturation curve.

tion.2 He compared today’s state-of-
the-art industrial robots—priced at
tens to hundreds of thousands of dol-

Product Refinement
lars—to 1970s era mainframes, while 2010 Technology maturity

consumer robots resemble 1970s mi- Laptops

crocomputers: crude, underpowered,

and of interest mainly to hobbyists
who enjoy tinkering with technol- Smartphones
ogy for its own sake. Today’s consumer robots
robots include a variety of kits (Lego computing
Mindstorms, VEX Pro, Robotis Bio-
loid), limited but intriguing toys (Wow-
ee’s Robosapien, Pleo from Innvo Labs, Time Since introduction

Penbo from Bossa Nova Robotics, and

a dozen others; many more in Japan),
and one astonishingly successful vac- computer scientists. The first is mis- made accessible to undergraduates.
uum cleaner: the Roomba; more than conceptions held by some about the na- The time to do that is now.
five million Roombas have been sold. ture of the subject. Robotics cannot be The second impediment to be over-
Advances in robotics are reported taught in CS1. The use of simple robots come is the lack of suitable robot plat-
weekly at technology news sites such to teach basic programming concepts forms for undergraduate instruction.
as, while the popular maga- dates back to Papert’s Logo turtle of The devices used in CS1 courses typi-
zines Robot and Servo are energizing the the 1970s5 and Pattis’ Karel the (simu- cally have no camera and can’t even
robotics hobbyist community the way lated) Robot in the 1980s.4 More recent drive in a straight line without drifting.
Byte and Dr. Dobbs’ Journal once nur- examples include a Python-based pro- The Lego Mindstorms kits used in many
tured amateur computing enthusiasts. gramming course using the Parallax current college courses are no better.
Meanwhile, more than 40 nations now Scribbler,1 and a variant of Alice (http:// On the other hand, the groundbreaking
have military robotics programs.3 that can both simulate Sony AIBO robot dog was an excellent
We see glimmers of our robotic fu- an iRobot Create and teleoperate a real instructional platform due to its pow-
ture in today’s self-parking cars, cam- one via a bluetooth dongle.8 CS edu- erful MIPS processor, rich sensor suite
eras that recognize human smiles, and cators must understand that while it (including a color camera, stereo micro-
flying devices ranging from micro-scale might be a good idea to use simple ro- phones, and multiple IR range finders
robot bees to the airliner-size Eitan UAV. bots to teach students about variables, and accelerometers), and sophisticated
But the robotics revolution will be far- procedure calls, and while loops, this is servos with position and force feedback.
ther ranging—and a lot more weird— not the same as teaching them robot- The AIBO’s $2,000 price tag was compa-
than most of us can envision now. Who ics, any more than making a penguin rable to high-end laptops of its day. But
in 1971 would have looked at the first In- move around in Alice counts as teach- when Sony abruptly exited the robotics
tel microprocessor and predicted eBay, ing computer graphics. market in 2006, the AIBO had not yet
Wikipedia, Google Earth, or “sexting”? High school robotics contests such caught on as a teaching platform, and
as US FIRST, which emphasize the me- that market niche has yet to be filled.
impediments to Progress chanical engineering aspects of the field In 2007 the RoboCup Federation,
We can help speed the revolution by at the expense of computer science, are which oversees robotic soccer com-
introducing our undergraduates to another source of misconceptions. The petitions worldwide, selected the Nao
state-of-the-art robotics hardware and public doesn’t always appreciate that humanoid from Aldebaran Robotics
software. But three factors have sty- the elaborate hardware platforms stu- to replace the AIBO in the Standard
mied progress in robotics education for dents construct must be primarily tele- Platform League. A few schools are
operated because students aren’t being now teaching robotics courses using
taught the kind of software that would Naos, but at a retail cost of approxi-
Real robotics allow their robots to act autonomously. mately $16,000, the Nao will remain
Real robotics involves deep, compu- out of reach for most educators. There
involves deep, tationally demanding algorithms. Ma- are some who believe that education-
computationally chine vision, probabilistic localization al robots should cost no more than a
and navigation, kinematics calcula- Lego Mindstorms kit: only a few hun-
demanding tions, grasp and path planning, multi- dred dollars. They’re right, but it will
algorithms. robot coordination, and human-robot be a while before the economies of
interaction (face tracking, speech and mass production can do for AIBO- or
gesture recognition) are core technolo- Nao-type robots what they’ve already
gies. Today these are found mainly in done for laptops and smartphones.
advanced research labs and graduate- Meanwhile, Mindstorms’ widespread
level robotics courses, but they can be and growing use in high schools and

28 communicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


even middle schools underscores my Early graphics programming was done

point that undergraduates require by turning pixels on and off, just as
something better. They need robots Who in 1971 would early robot programming was done by
that can see, with processors that can have looked at turning motors on and off. But graph-
run the sophisticated algorithms com- ics has developed into a wonderfully
puter scientists should be studying. the first intel rich field that includes specialties such
As illustrated in the figure on the microprocessor as Web design, game design, and sci-
preceding page, educational robot- entific visualization, where the focus
ics is entering the unstable region of and predicted eBay, is on principles of visual aesthetics or
the technology maturation curve. For Wikipedia, Google the graphical presentation of informa-
just a few more years, computer sci- tion, not low-level details of rendering
entists will build their own platforms earth, or “sexting.” algorithms or GPU programming. Web
for education and research. In less and game designers rely on computer
than a decade this will become infea- scientists for the tools of their trade,
sible, for the same reason that no in- but they have different skill sets and
dividual today builds their own laptop are not themselves computer scien-
or cellphone. But since highly capable tists. The applications of computer
robots are not yet mass-produced con- dent software modules responsible for graphics have outgrown the confines
sumer products, today’s educators controlling various types of hardware of a single discipline.
must innovate. Several colleagues in or providing services such as localiza- I believe our notions about robot
the U.S.b have found a good solution by tion. Both support a wide variety of programming will likewise broaden in
mounting a laptop or netbook atop an platforms and devices. And both are de- the coming years. Our students will cre-
iRobot Create. The Create—a Room- signed primarily for research, although ate the technologies that make this pos-
ba without the vacuum—provides an Player/Stage in particular has been sible. Better algorithms for perception
inexpensive mobile base with a few widely used for education. Modules can and manipulation, and high-level frame-
simple sensors, while the laptop pro- be written in any of several languages; works for robot instruction will enable
vides a Webcam for vision, a WiFi con- the frameworks themselves make no robotics application development by a
nection, a speaker, and plenty of com- assumptions about representation. diverse population of users and innova-
puting power. The total parts cost can tors, some of whose job descriptions are
be as low as $600. Anyone who thinks another approach as unforeseeable today as “Web design-
these platforms are too expensive Tekkotsu, developed with Ethan Ti- er” was in 1971. That will be one unmis-
should recall what schools were pay- ra-Thompson in my lab at Carnegie takable sign that the robotics revolution
ing for workstations a few years ago. Mellon University, takes a different has arrived. Let’s get started.
Readers who would like to put one of approach. It is implemented in C++
these robots together themselves, or and makes heavy use of abstraction References
purchase a pre-assembled version facilities such as templates, multiple 1. balch, t. et al. designing personal robots for education:
Hardware, software, and curriculum. IEEE Pervasive
from a commercial vendor, can find inheritance, polymorphism, functors, Computing 7, 2 (Feb. 2008), 5–9.
all the necessary information at http:// and namespaces. It offers a common 2. Gates, b. a robot in every home. Scientific American
(Jan. 2007), 58–65. An en- representation scheme for vision, 3. levinson, C. Israeli robots remake battlefield. The Wall
Street Journal (Jan. 12, 2010), a12.
hanced version with a pan/tilt camera navigation, and manipulation tasks.6,7 4. Pattis, r.e. Karel the Robot: A Gentle Introduction to
and an arm with gripper is presently The idea is to provide a unified frame- the Art of Programming, Second Edition, Wiley, new
york, 1995.
under development (see the image on work that undergraduates can master 5. solomon, C. logo, Papert, and Constructionist learning,
the first page of this column). in two-thirds of a semester and then 2010;
6. touretzky, d.s. et al. dual-coding representations for
The final impediment to be over- move on to working on an interesting robot vision in tekkotsu. Autonomous Robots 22, 4 (apr.
come is the lack of easy-to-use soft- final project. Tekkotsu does not strive 2007), 425–435.
7. touretzky, d.s., and tira-thompson, e.J. the tekkotsu
ware. The three major open source for universal hardware coverage; in- “crew”: teaching robot programming at a higher level.
frameworks for robotics application stead it provides well-tuned primitives In Proceedings of the AAAI Symposium on Educational
Advances in Artificial Intelligence (eaaI-10), (atlanta,
development are Player/Stage (http:// for a small number of educational Georgia, July 13–14, 2010)., ROS platforms, including the AIBO and the 8. Wellman, b., davis, J., and anderson, M. alice and
robotics in introductory Cs courses. In Proceedings
(, and Tekkotsu Create. Other groups are developing of the Fifth Richard Tapia Celebration of Diversity in
( Player/Stage Tekkotsu support for additional plat- Computing Conference (Portland, or, 2009), 98–102.
9. Zyda, M. Computer science in the conceptual age.
and ROS have similar philosophies. forms, including the Nao and various Commun. ACM 52, 12 (dec. 2009), 66–72.
Both provide a general communication robot arms. But for all three frame-
framework for a collection of indepen- works, more work remains to be done David S. Touretzky ( is a research
professor in the Computer science department and the
to make advanced robotics technolo- Center for the neural basis of Cognition at Carnegie Mellon
gies easy for non-experts to use. university in Pittsburgh, Pa. He was named a distinguished
b Jeff Forbes at Duke University, Chad Jenkins scientist by aCM in 2006.
at Brown University, Monica Anderson at the
University of Alabama, and Zach Dodds at Har-
our Robotic future research funded by national science Foundation award
vey Mudd College have been at the forefront of To predict the future of robot software,
this work. look at the history of graphics software. Copyright held by author.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 29

DOI:10.1145/1787234.1787245 Ping Gao and Jiang Yu

emerging markets
has china
caught up in It?
An assessment of the relative achievements in IT infrastructure, firms,
and innovation in China.

h I n A I s e m e Rg I n g as a new mobile Internet connectivity, and the
economic superpower, but like.
is it also emerging as an IT China has also pushed ahead with
superpower? Certainly it national informatization programs
has gone a long way toward initiated in 1994 with the creation
catching up and even leading in some of the National Information Infra-
fields. We can understand its “tech- structure Steering Committee. This
no-nationalism” as the foundation. launched various “Golden” projects
But—as recent difficulties with Google such as Golden Bridge, Golden Card,
illustrate—it must also address ongo- and Golden Customs, which have built
ing challenges before the superpower a national platform for providing com-
label can be applied. mercial Internet services, constructed
a national credit card network, and
understanding catch-up linked customs points through a na-
IT infrastructure and services. When tional EDI system, respectively. These
China started economic reform in the are now just one small part of an over-
early 1980s, its information and com- all drive for e-business, e-commerce,
munications infrastructure was weak. and e-government that has created
In 1980, the Chinese mainland had more than 100 national information
just four telephones per 1,000 citi- systems that provide a true “digital
zens. In contrast, just over the border internet users in china. economy” infrastructure. Annual
in Hong Kong—still a British colony trade in China’s B2C market, for ex-
at that time—there were approxi- deployed in 1994. By 2001, China was ample, is now around $3 billion (to-
mately 460 phones per 1,000 citizens.1 already the world’s largest mobile tal e-commerce is approximately 100
Even in the 1990s in Beijing, a two- to telecommunications market. The lat- times greater) and is growing at a rate
three-year wait was required to have est figures show more than 640 mil- of about 50% per year.
a telephone line installed in a typical lion subscribers in 2008, and a 19% All of this has been built around a
household. From that point, though, annual growth rate during the 2000s.5 fast-growing e-infrastructure: by 2009,
a combination of very strong demand, Contrast this with the U.S., which had China had the world’s largest popula-
heavy state investment plus supply- 270 million subscribers and an 11% tion of Internet users (more than 300
PHotoGra PH by IM aGIneCHIna VIa a P IMaGes

side support from government to the growth rate, or Japan with 110 mil- million) and the largest population
monopoly supplier ensured a major lion subscribers and a 5% growth of broadband users (more than 100
change. The most recent figures indi- rate. During this period, China’s mo- million). For a long time that popula-
cate the number of fixed-line subscrib- bile telecommunications networks tion was restricted to large cities and
ers had reached more than 310 mil- upgraded to GPRS and EDGE, and are coastal provinces. But the 2004 Village
lion; or 230 per 1,000 population.7 now adopting 3G technology. As For- Access project ensured that, by 2007,
The mobile telecommunications tune 500-listed companies, the three 99.5% of all administrative villages
market has similarly gained rapid mo- operators own nationwide networks were connected to the telecommunica-
mentum since 2G digital systems were and provide value-added services, tions infrastructure.6

30 comm unicaTio nS o f Th e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


Of course, China will win most abso- companies in 2008.3 Huawei achieved
lute IT numbers counts because it has a significant milestone in 2009 when
the world’s largest population. Take Some chinese iT it won 4G roll-out contracts in Norway
relative figures—for example, 48% mo- and Sweden that beat rivals Nokia and
bile device penetration compared to firms have evolved Ericsson in their home markets.
87% in the U.S.; 6% broadband penetra- from this domestic IT innovation. As noted earlier, be-
tion compared to 23% in the U.S.—and fore the 1990s China mainly depended
the country is no longer the world lead- platform to innovate on foreign technology. Since then the
er, but catching up. Considering rela- enough to compete nexus of government and Chinese
tive growth rates, one should modify firms has adopted a twin-track ap-
that to “catching up quickly.” And globally. proach to creating domestic technolo-
taking into account the availability of gies: boosting supply by investing in
services and standards then at least R&D, and steering local demand by
for a significant proportion of China’s buying locally innovated products. To-
population, the message is “already day, in areas such as PCs and consum-
caught up.” er electronics, Chinese technology
IT firms. In 2006, China Mobile be- was thus forced to look to foreign part- dominates the domestic market. Con-
came the world’s largest mobile op- ners, with imported technology domi- struction of the nation’s IT infrastruc-
erator, with the world’s biggest mo- nating. As a result, nearly all leading ture has been predominantly based on
bile subscriber base. The Financial IT firms have a joint venture or foreign indigenous products.
Times ranked it fifth in a 2007 list of independent investment operation Some Chinese IT firms have evolved
the world’s most valuable brands,4 re- in China. Initially a sign of weakness, from this domestic platform to inno-
flecting the broader globalization of though, China has managed these rela- vate enough to compete globally. Hua-
Chinese telecommunications firms. tionships, and ensured the large influx wei can again provide an illustrative ex-
This has seen international strategic of foreign capital, technology, and ex- ample: FastCompany rated it among the
alliances set up between China Mobile pertise has acted as an impetus rather world’s top five most innovative compa-
and Vodafone, and China Unicom with than a restraint to local firms. From be- nies in 2010, along with Facebook, Ama-
Spain’s Telefónica. And ambitions have ing a stunted consumer market, China zon, Apple, and Google.2 That rating re-
gone beyond just alliance: in 2007, Chi- has grown to be one of the world’s larg- flected Huawei’s relentless investment
na Mobile purchased Pakistan’s Paktel est producers of IT equipment, includ- in technology R&D; spending some 10%
and used this as the launching pad for ing mobile systems and computers. In- of revenue on R&D centers based not
its own international brand, ZONG. digenous manufacturers have become just in China but now in India, Europe,
In IT services, local firms are the key international players. and the U.S. It is the single largest ap-
dominant force in the domestic mar- Perhaps most notable has been the plicant under the international Patent
ket. Shenzhen-based Tencent, for ex- performance of Lenovo and Huawei, Cooperation Treaty.
ample, has over 500 million registered set up in the 1980s. Lenovo achieved Government has also played a role:
users and a 78% share in the instant global attention and reach through its through investment and policy deci-
messaging market. Some firms have 2005 purchase of IBM’s PC division. sions China has moved to sixth in the
been able to grow even further. Chi- This purchase was an iconic statement global patents league table. The qual-
na’s is the global leader of changing fortunes that led the firm— ity and utilization of such patents out-
in the B2B market with 47 million us- still one-quarter-owned by the Chinese side China is sometimes limited, but
ers covering virtually every country government—to be the world’s fourth- the Chinese government is also play-
in the world. Its 2007 $1.7 billion IPO largest PC maker in 2009 with approxi- ing a higher-level game. Understand-
was the largest for any Internet firm mately $15 billion in sales. ing the importance of standards in
since Google. Google, like Amazon and Huawei’s rise has been even more the internationalization of IT, it has
other major Internet firms, has set up dramatic given its beginnings with a pushed to have Chinese-developed
in China; such firms often create lo- staff of 14 and some $3,000 of capital standards recognized and used. In
cal joint ventures that help develop lo- distributing imported telephone ex- this endeavor, it has had some suc-
cal partners. (At the time this column changes. By 2009, it had 90,000 employ- cess in breaking beyond the confines
was written, Google relocated part of ees and worldwide sales of more than of the domestic market. The Inter-
its search services under the domain $21 billion covering mobile, networks, national Telecommunication Union
name to Hong Kong, but terminals, and value-added services. In has ratified three main 3G standards
maintained its sales group and R&D just two decades, it has not just caught based on CDMA. China’s TD-SCDMA
functions in mainland China and has up, but in some areas taken over. It is now competes with W-CDMA (origi-
not closed its site in China.) the world’s number-two firm in tele- nating from Japan and Europe) and
Foreign firms have also been impor- communications equipment and CDMA2000 (originating from the
tant in IT manufacturing. In the early number one in subfields such as mo- U.S.). The TD-SCDMA standard is
1980s, when China was first targeting bile switches and next-generation net- widely used in China but is also start-
IT, its manufacturing industries were work technology. BusinessWeek rated it ing to experience international de-
poor in both quality and innovation. It one of the world’s 25 most influential ployments: first in Ghana but also

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 31

planned for South Korea and some behind the U.S., Western Europe, novation requires direction from gov-
Eastern European countries. and neighbors like Japan and South ernment. Indeed, the “Great Leap For-
Korea. Its national policies, though, ward” leitmotif in China’s history is
explaining catch-up will no doubt allow the “catching up one that not just encourages this type
We answer the question posed in the quickly” to continue. For example, of “big push” approach; it shows it is
headline for this column with a mixed China has invested extensively in required for development. But gov-
but generally positive response. In IT higher education over the next few ernments may push the wrong way.
infrastructure, China is catching up years, coinciding with cutbacks in The Chinese government’s attempt to
quickly. In some areas of IT services many Western nations. have indigenous WLAN Authentica-
and innovation it has caught up. A More difficult will be China’s aim tion and Privacy technology replace
few IT firms and technologies are now for innovation leadership through Wi-Fi in the local market did not suc-
world-leading. How did this happen in techno-nationalism. One focus of the ceed. And, it forced China Mobile to
a country that, less than three decades 2006 15-year Science and Technology deploy the TD-SCDMA standard, even
ago, was decidedly “Third World”? Plan is building “indigenous innova- though this may be against the firm’s
China’s general economic growth tion.” This was made possible due to own economic interests and wishes.
has helped. So has its large size. But what had already been achieved since Only time will validate whether it was
government and its policy of “techno- the 1980s; it was desirable because right to override other voices, and
nationalism” has been a significant in- of what had yet to be achieved. But whether this standard will truly suc-
fluencing factor with a series of policies it faces three challenges. Tight gov- ceed internationally.
that made IT—especially local IT capac- ernmental control and censorship of In sum, the changes since the 1980s
ity—a priority. The general approach information, media, and the Inter- mean China’s techno-nationalism is
has been heavy control followed by net in the name of national security now a high-wire act—balancing be-
liberalization, but varied from sector inhibits innovation. For example, as tween the economic and the political,
to sector. an important propaganda channel, the global and the local, the state and
For example, in telecommunica- the broadcasting sector is tightly con- business. Its current model will re-
tions services, infrastructure was very trolled by the state; as such, broad- main central to ongoing IT catch-up
weak in the 1980s but there was some cast networks cannot converge with and leadership, with major national
local capacity. The Chinese govern- telecommunications and Internet initiatives already under way in the ar-
ment decided to retain the state mo- networks to offer new information eas of nano-electronics and 4G mobile
nopoly and granted it tax and invest- services. networking. China must hope these
ment privileges. Only as the market Indigenous innovation also faces future large-scale initiatives do not
grew was competition introduced to challenges from overseas. China’s lead it to slip off the high-wire arrange-
boost growth, moving from monopoly relations with foreign IT firms have ment it has constructed.
to duopoly in 1994, and then opening been volatile. Sometimes they seem
to further competition in 1998 and needed but not wanted; sometimes References
1. China Telecommunications Over 50 Years. Ministry of
2008. In telecommunications equip- vice versa. Sometimes they are at- Information Industry, beijing, 1999.
ment, local capacity was too limited tacked in private but soothed in pub- 2. Fast Company. the world’s most innovative
companies, (Feb. 2010);
and risked stunting wider growth. lic; sometimes vice versa. Google, for mic/2010
Government therefore opened the example, has been welcomed, hacked, 3. Gibson, e. and McGregor, J. the world’s most
influential companies, BusinessWeek (aug. 8, 2008);
market to foreign firms in 1982. It partnered, criticized, and tolerated—
used the lure of China’s 1.3 billion often simultaneously—all in a short most_influential/index.htm
4. Global brands. Financial Times, london, 2007; http://
potential consumers as leverage to time span.
5. ICT-Eye. International telecommunication union,
encourage Western firms to compete Google is somewhat unusual— Geneva, 2010;
among themselves in offering the best China does not rely on it (the Chinese Indicators/Indicators.aspx
6. Ministry Report News. Central Government of the
technology transfer to China, and the firm Baidu dominates the search People’s republic of China, beijing, 2008; http://www.
greatest support to local innovation. market) and Google does not rely on
7. Telecommunication Industry Statistics. Ministry of
The Chinese government has also China (revenue from within China Industry and Information technology, beijing, 2009;
created and supported a series of R&D represents approximately 1% of total).
programs that bring firms, research But Google’s reaction of pulling in U.S. 8. Wray, r. and stewart, H. Western business struggles
institutes, and universities together government assistance is not unusual. to break Chinese barriers. The Observer, (Mar. 28,
2010), 38–39
to work on R&D for key technologies. One-quarter of U.S. high-tech firms
Creation of the TD-SCDMA standard in- feel they are losing out due to the 2006
Ping Gao ( is a lecturer in
volved just such a consortium. Togeth- Plan.8 Accusations of Chinese protec- development informatics at the Institute for development
er, these initiatives and actions mean tionism and calls for U.S. Congressio- Policy and Management, and Centre for development
Informatics, university of Manchester, u.K.; http://www.
China now has a drastically improved nal countermeasures are increasing., http://www.manchester.
national innovation system. Together with fallout from the Google
Jiang Yu ( is an associate
case, this may damage foreign trade professor in the Institute of Policy and Management,
challenges and future Trends and investment and, hence, harm in- Chinese academy of sciences, China. He is the editor of
the Journal of Science and Technology Policy in China.
In IT infrastructure and services novation in China.
penetration, China is still some way Thirdly, Plan-based indigenous in- Copyright held by author.

32 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


DOI:10.1145/1787234.1787247 George V. Neville-Neil

Article development led by

Kode Vicious
presenting your project
The what, the how, and the why of giving an effective presentation.

Dear kV,
I have a nontechnical question for you.
I’ve been asked to make a presenta-
tion on the project my team has been
working on for the past year. Actually, I
wasn’t really asked so much: the eight
of us drew straws, and I got the short
one. Other than trying to imagine my
audience in their underwear—a piece
of advice I have now been given at least
five times—what would you suggest?
Stage frightened

Dear Stage,
I am almost positive that “Imagine your
audience in their underwear” is just
about the worst piece of advice anyone
has ever given to a prospective speaker.
Instead, there are practical steps you
can take in order to ensure your presen-
tation is well received. The most impor-
tant step is to be prepared, which, as
Tom Lehrer has pointed out, is the Boy
Scout marching song. Unless you are
very good at giving presentations, and abstract to the specific. What the best “what,” or even “what” and “how,” usu-
from what you say above you’re proba- speakers are able to communicate is ally put audiences to sleep.
bly not, you need to know your material not just the “what” of a system—that In preparing a presentation you
backward and forward before trying to is, the specific details—but also the should start with an outline. I realize
explain it to a large group. You might “how,” which is a higher-level descrip- many people will say, “But that’s ob-
be an expert in the nitty-gritty details of tion, and most importantly, the “why.” vious,” but it turns out it’s not. Many
the system you’re describing, but that’s The most memorable presentations people write presentations by start-
not good enough—in fact, it can be a are the ones that succeed on all three ing with a title slide and then adding
drawback. No one, no matter how tech- levels: “Here is what the system does, one slide after another until they feel
nical, wants to listen to someone drone here is how it works, and here is why they have enough to cover the mate-
PHotoGra PH by sH enG Ha n

on about the low-level details of some it’s built that way.” If you keep these rial. Writing a presentation in this way
system. You need not only to know the three questions in your head while inevitably leads to a wandering style
low-level details but also to be able to writing your outline and slides, then it of presentation that is more like a tor-
talk about the system you are describ- is far more likely your presentation will tured walk through a maze of twisty
ing at many different levels, from the go over well. Presentations full of only paths, all different.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 33

I have a template for my presenta- ences, particularly in Asia, do not ask

tions, which includes sections for In- questions during a presentation but
troduction, History, and Conclusion, a presentation only at breaks or after the presentation.
and I always end with a Questions should not be a If you call for questions two or three
slide. Even having just that much ini- times and receive no response, then
tial structure will help you to focus your one-way broadcasting stop asking and wait until the end of
thoughts on what you’re trying to com- of information but the presentation. There are few things
municate to your audience. more annoying than having the speak-
Although this is necessary boiler- rather a conversation er whine about how no one is asking
plate, it is not sufficient. You need to between the speaker questions.
pick three to five important points, de- When someone does ask a question,
pending on how much time you have and the audience. unless you’re presenting to a group of
allotted to present your material, and fewer than 10 people in a conference
make those important points the foci room, make sure to repeat the ques-
of the internal sections of the talk. tion. The person asking the question
Make sure to introduce each point; the is very likely facing you, and not the
points are not supposed to be surprises audience, and it’s your responsibility
you foist upon the audience at the end When speaking in public it is im- to keep the audience involved through-
of each section. Make sure each point portant to remember you are trying to out the entire talk. If you don’t know
covers the what, how, and why before engage the audience. A presentation the answer to a question, then do not
moving on to the next point. should not be a one-way broadcasting of try to make it up as you go along. Some
You should think of your outline as a information but rather a conversation people are able to make things up as
way to brainstorm on the presentation. between the speaker and the audience. they go along, but this is not what I’d
Put everything you can think of in as a You are telling the audience a story, and recommend for someone doing their
potential slide title, leave out the de- as such you should try to be engaging first, or even their tenth, presentation.
tails, and then go back later and edit out and to keep the audience interested. You may also find there are people
the things that don’t work. It is always Do not speak in a monotone, and don’t who are asking questions in order to
easier to cut material at the last minute read the slides to the audience. Most prove how smart they are, rather than
than it is to create something new. audiences are literate enough to read because they are interested in the an-
One common issue all novice your slides; you don’t have to do it for swer. If you have one of these people in
speakers have is figuring out how long them. The slides should act as a prompt your audience, simply say, “That’s very
the final presentation will run. I have for you to explain the what, how, or why interesting, and I’d like to talk about
found that, on average, most people of some point, but they are not lines on that after the presentation.” After a few
do well with two minutes per slide. Al- a teleprompter for you to read. such rebuffs they usually quiet down.
though some people speak very quick- If at some point during the presen- Finally, a piece of very basic, and
ly—in fact, I know an excellent speaker tation you find yourself writing on a perhaps base, advice: Eat lightly be-
whose talk was once described as “be- chalkboard, whiteboard, or other vi- fore your talk and make sure to use the
ing shot with a machine gun”—and sual aid, do not speak to the board. bathroom, whether you think you need
others quite slowly, the reason for two One of my professors in college would to or not, before you enter the room
minutes per slide is to keep the audi- stand at the chalkboard and read the and begin your presentation.
ence paying attention. When you first textbook to it while writing out notes kV
display a slide people will, believe it or on the board; he barely, if ever, spoke
not, actually read it. If you put up one to the class. I can tell you that I tried
slide and then speak for an hour, then increasingly high levels of caffeine Related articles
it had better be the most interesting to keep myself awake, but no matter
slide that has ever existed. Very few how much I took I could never remain Comparing and Managing Multiple Versions
of Slide Presentations
one-slide presentations are effective. awake through the full hour. Steven M. Drucker, Georg Petschnigg,
Conversely, if you are flipping slides It’s important to pace yourself as Maneesh Agrawala
every 15 to 30 seconds you will lose you speak. Remembering to breathe
your audience’s attention. is quite important, and something cfm?id=1166263
Now that you have your slides to- that new speakers can actually forget Code Spelunking Redux
gether, it’s time to go over a few impor- to do. Panting at the end of each slide George V. Neville-Neil
tant points about actually giving the is a good indication that you ought to
presentation. Many people practice be breathing a bit more. Many speak-
their presentations in front of a mir- ers like to ask for questions from the George V. neville-neil ( is the proprietor of
neville-neil Consulting and a member of the aCM Queue
ror, or a very kind colleague, friend, or audience throughout the talk, in part editorial board. He works on networking and operating
loved one. I recommend avoiding tor- to keep the audience awake but also as systems code for fun and profit, teaches courses on
various programming-related subjects, and encourages
turing your significant other with your a way of pacing themselves. While the your comments, quips, and code snips pertaining to his
Communications column.
presentation unless he or she is also “Any questions?” tactic can be useful,
involved in the project. it must be used sparingly. Some audi- Copyright held by author.

34 communicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


DOI:10.1145/1787234.1787246 Eugene H. Spafford

privacy and security

of things pest
Recalling malware milestones.

nnIVeRsARIes pResent An
opportunity to reflect.
Sometimes we celebrate
anniversaries (birthdays,
graduations, some mar-
riages), and sometimes we grieve
(deaths, disasters, and some marriag-
es). There are also anniversaries when
we compare what might have been
against what has actually happened (all
the above, and more).
˲˲ The first use of the term “computer
virus,” occurred 40 years ago in Ven-
ture Magazine, in a science fiction story
by Gregory Benford, involving com-
puter code and a corresponding vac-
cine program.a Benford’s friend, David
Gerrold,b later incorporated these ideas
into his novel, When HARLIE Was One.
˲˲ Next year is the 25th anniversary of
the first widespread PC (MS-DOS) com-
puter virus (known as Brain, Lahore, or
Pakistani). By 2000 there were 40,000
families (code variations) of viruses for
Microsoft-based operating systems; a
few score viruses existed for other sys-
tems, including the Macintosh, Amiga, ˲˲ Last November was the 21st an- By 2005, malware existed that
and Atari systems.c niversary of the Internet Wormd that spread by Web pages, email, and other
brought malicious software (malware) network services. “Blended” threats
a Personnel communication, later confirmed to the fore after it spread in part of the were common, including components
in a letter to the editor of the New York Times, early Internet over several days. Next spread by inadvertent user activation.
published in 1994.
b David Gerrold is perhaps best known to many
year is the 10th anniversary of its con- Malware developers quickly overcame
IllustratIon by Gordon stud er

as the author of The Trouble With Tribbles sto- ceptual descendent, Nimda, which af- new defenses as they were devised,
ry that was made into a much-beloved “Star fected hundreds of thousands of Win- deploying alteration of OS functions,
Trek” episode. dows systems worldwide in less than code to disable security mechanisms
c Early history of computer viruses can be found 30 minutes. and antivirus programs, and self-modi-
in many references, including “Virus” by E.H.
Spafford in Internet Besieged: Countering Cy-
fication to foil pattern-based detection.
berspace Scofflaws; D. Denning and P. Den- d See Communications of the ACM 19, 1 (Jan. Some malware applied vendor soft-
ning, Eds., Addison-Wesley, 1997. 1989) for several analyses and views. ware patches to prevent other malware

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 35

from displacing it: ironically, that mal- detection indefinitely (the APT or Ad- costs and responsibilities for dealing
ware performed better at maintaining vanced Persistent Threat; see http:// with malware, leading to a culture of
systems than their owners! “add-ons” for security and skewed ex-
Malware now includes “social en- apt-hacks/), and some botnets whose pectations;
gineering” components to entice the origins cannot be traced may include ˲˲ Periodic software patching and
careless, unprotected, and unwary. millions of compromised hosts (for production use of beta products are
Phishing, botnets, cross-site script- example, Conficker). viewed as the norm rather than as un-
ing and SQL injection have become The science fiction story of 40 years usual exceptions;
commonly known terms. There have ago is now a scourge causing huge ˲˲ Law enforcement has not been giv-
been many notorious uses of mal- global losses and evolving as a new en sufficient resources, support, or pri-
ware, including political action in tool of organized aggression. The pub- oritization to pursue malware authors
Estonia, supporting military actions lic is beginning to realize what special- and operators;
against the country of Georgia, and ists have known for years: these prob- ˲˲ Research has been funded mostly
spying on human rights activists and lems are getting worse. How did this to respond to current threats rather
the Dali Lama. happen? And what can we do about it? than to devise disruptive but safer re-
Early malware was developed for placements for current systems; and
bragging rights or out of curiosity; to- factors ˲˲ Issues involving pricing and li-
day’s malware is often written by crim- In no particular order, some of the censing software in a diverse, global
inals—including organized crime—to most notable factors contributing to marketplace have led to numerous,
commit fraud, distribute spam email, the proliferation of malware have in- unauthorized copies that may be ineli-
obtain identity and account data, and cluded: gible for patches, and whose operators
steal proprietary commercial informa- ˲˲ Software is usually produced using cannot afford security add-ons.
tion. Malware-generation tools have error-prone tools and methods, includ-
proliferated, including some posted ing inadequate testing. Well-estab- Remediation
online for anyone to use. Globally, an- lished security principles are ignored— We can reduce the malware problem by
nual losses from malware may total if the developers even knew about actions on four major fronts.
in the tens of billions of dollars (or them. Too many people believe narrow Economics. The economics of secu-
more)—and how do we put a price on “secure programming”f approaches are rity need to be changed. This includes
the loss of national defense informa- the solution, and equate penetration ef- increasing our understanding of the
tion, or the safety of activists opposed forts with security assurance; long-term risks and cost-effectiveness
to oppressive regimes? ˲˲ The market often rewards first-to- of security-related choices to enable
Tens of thousands of new instances sell and lowest cost rather than extra better choices by system owners and op-
of malware appear daily,e although it time and care in development; erators; reducing the barriers to compe-
is impossible to get a precise count be- ˲˲ Vendors favor producing large, tition that might lead to safer products
cause of their often-polymorphic na- all-in-one products to minimize de- such as by embracing vendor-neutral,
ture: a “new” version is created each velopment and marketing costs, but open standards to improve portability;
time a download occurs. Of those, these have larger attack surfaces and and reexamining those parts of regula-
only a fraction is detected because of more options to misconfigure and tory and intellectual property regimes
built-in stealth techniques and poor misunderstand; that interfere with research and (re)use
security practices by the victims. Cur- ˲˲ Vendors pursue upgrades and new of sound security features. Judicious
rent malware may remain without releases as a means of maintaining rev- use of rewards and penalties for prod-
enue streams, but backward-compati- uct quality might help. Changes to lia-
e Personal communications from Vesselin bility and new features both contribute bility protections for vendors, ISPs, and
Bontchev, John Thompson, and John Viega. to new vulnerabilities; end users could also encourage more
˲˲ Customers in industry and govern- proactive actions by all involved.
ment have placed more emphasis on Milieu. The public needs basic edu-
early malware acquisition cost than on total cost of cation about good security and priva-
operation, risk, and quality; cy practices to make better-informed
was developed for ˲˲ Feature lock-in (product and choices. Where private owners cannot
bragging rights or out training compatibility) coupled with afford necessary upgrades or services
a lack of good metrics on security and to “disinfect” and reconfigure their
of curiosity; today’s safety have hindered innovation and systems, public “computing health”
malware is often competition;
˲˲ Insufficient
organizations should be created: con-
diversity enables taminated clients are a threat to the
written by criminals. “write-once, run everywhere” attacks; community as a whole. Although not
˲˲ The end user is burdened with the without their own problems, some
uses of virtualization and software as
f “Secure programming” is writing code with-
a service (SaaS) present opportunities
out certain features that have been frequently for migration of end users away from
exploited. poorly maintained systems.

36 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


There must be a change in the atti-

tude that end users are solely respon-
current and
sible for their systems’ security. Cus-
tomers are not to blame that systems
are shipped without appropriate safe-
past methods of Events
guards, nor should they be forced to employed against August 16–17
buy and maintain a large (and grow- malware have creativity and Innovation
in design,
ing) set of additional protections to
use their systems safely. Additionally, perhaps slowed Aarhus, denmark,
contact: christensen Bo,
everyone should learn that patching the growth of email:
a system is not security, and penetra-
tion testing is no substitute for proper the problem but August 16–20
designing Interactive systems
design and development. certainly have conference 2010,
Technology. As a field, we should Aarhus, denmark,
reexamine construction of smaller, not stopped it. contact: olav W. Bertelsen,
more protected systems and applica-
tions. Known, effective techniques August 18–20
such as putting code in read-only International symposium
devices, code whitelisting, integrity on Low power electronics and
monitoring, and better separation Austin, tX,
of privileges could all play a role if evolve into the current worldwide infra- sponsored: sIgdA,
used integrally rather than as add- structure. Malware and automated at- contact: Vojin g. oklobdzjia,
ons. Tools, programming languages, tacks have also been evolving, and the email:
and platforms in use should also be result is an increasing, usually unno- August 19–20
reexamined from the perspective of ticed drag on our innovation and econ- International conference
how to build functional, safe systems omy. We are now at a point where it is on Intercultural collaboration
cost-effectively rather as instruments becoming an existential issue for some 2010,
copenhagen, denmark,
perpetuating legacy decisions. Test companies and even governments. sponsored: sIgchI,
methods, including some that were Current and past methods em- contact: Anne-marie
previously considered to be too com- ployed against malware have perhaps soederberg,
plex to be practical, should be recon- slowed the growth of the problem
sidered given our continually advanc- but certainly have not stopped it. If August 25–27
ing capabilities.g we simply continue to do more of the european conference on
Law. Most malware is a law en- same we will continue to be victim- cognitive ergonomics,
delft, netherlands,
forcement issue, not a military one; ized, and the problem will get worse. contact: neerincx mark,
it is cybercrime, not cyberwar. Po- The longer we wait, hoping that piece- email:
lice need tools, trained personnel, meal and uncoordinated responses
authority, and a clear mandate to will be enough, the more difficult (and August 30–september 3
Acm sIgcomm 2010
pursue the authors and operators expensive) it will be to address the conference,
of malware. This will require a con- problems when we finally attempt to new delhi,
certed international effort—but the do so. sponsored: sIgcomm,
trends are clear that people in every Change requires resources, will, contact: shivkumar
country are at risk if effective actions and time. We do not need to do every- phone: 518-782-7875,
are not taken. Perhaps, with some thing everywhere at once—but we do email:
creativity, approaches other than tra- need to start. Unfortunately, some of
ditional criminal statues might be those who are in the best positions to August 31–september 4
International conference on
employed, akin to using tax law vio- make changes are also under the most distributed smart cameras,
lations to convict Al Capone. Authors pressure to defer change precisely be- Atlanta, gA,
and operators of malware presented cause it requires resources and dis- sponsored: sIgmm, sIgBed,
contact: marilyn claire Wolfe,
with a significant risk of substantial ruption of the status quo. It is up to all
phone: 404-894-5933,
penalties might instead choose to of us to facilitate the changes that are email:
pursue more legitimate professions. needed—before too many more anni-
versaries pass us by. september 1–3
symposium on solid and
physical modeling,
It has taken decades for computing to Eugene h. Spafford ( is a haifa, Israel,
professor of computer science and the executive director
of the Center for education and research in Information contact: Anath Fischer,
assurance and security (CerIas) at Purdue university. phone: 972-482-93260,
g This is a special case of what I described in email: mereanath
“Rethinking computing insanity, practice and
research” available at
thinking. Copyright held by author.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 37

DOI:10.1145/1787234.1787248 Samir Chopra

Rights for Autonomous
Artificial Agents?
The growing role of artificial agents necessitates modifying
legal frameworks to better address human interests.

t Is A commonplace occur- questions must be addressed with
rence today that computer respect to artificial agents.4 So, what
programs, which arise from The artificial agent place within our legal system should
the area of research in ar- is better understood these entities occupy so that we may do
tificial intelligence known justice to the present system of socio-
as intelligent agents, function au- as the means by economic-legal arrangements, while
tonomously and competently;1 they which the contract continuing to safeguard our interests?
work without human supervision,
learn, and, while remaining ‘just offer is constituted. The contracting Problem
programmed entities’, are capable of Discussing rights and responsibilities
doing things that might not be antici- for programs tends to trigger thoughts
pated by their creators or users. of civil rights for robots, or taking them
In short, leaving philosophical to trial for having committed a crime or
debates about the true meaning of something else similarly fanciful. This
‘autonomy’ aside, they are worthy is the stuff of good, bad, and simplistic
of being termed ‘autonomous artifi- questions. If it is a reasonable as- science fiction. But the legal problems
cial agents’.a And on present trends, sumption that the degree of their au- created by the increasing use of artifi-
we, along with our current social and tonomy will increase, how should we cial agents today are many and varied.
economic institutions, will increas- come to treat these entities? Consider one problem, present in e-
ingly interact with them. They will buy Societal norms and the legal system commerce: If two programs negotiate
goods for us, possibly after carrying constrain our interactions with other a deal (that is, my shopping bot makes
out negotiations with other artificial human beings (our fellow citizens or a purchase for me at a Web site), does
agents, process our applications for people of other nations), other legal that mean a legally binding contract is
credit cards or visas, and even make persons (corporations and public bod- formed between their legal principals
decisions on our behalf (in smarter ies), or animal entities. There are, in (the company and me)?
versions of governmental systems parallel, rich philosophical discussions A traditional statement of the re-
such as TIERS2 and in the ever-increas- of the normative aspects of these inter- quirements of a legally valid contract is
ing array of systems supporting legal actions in social, political, and moral that “there must be two or more sepa-
decision-making3). As we interact with philosophy, and in epistemology and rate and definite parties to the con-
these artificial agents in unsupervised metaphysics. The law, taking its cues tract; those parties must be in agree-
settings with no human mediators, from these traditions, strives to pro- ment i.e., there must be a consensus
their increasingly sophisticated func- vide structure to these interactions. It ad idem; those parties must intend to
tionality and behavior create awkward answers questions such as: What rights create legal relations in the sense the
do our fellow citizens have? How do promises of each side are to be enforce-
a Jim Cunningham has pointed out that a cer- we judge them liable for their actions? able simply because they are contractu-
tain degree of autonomy is present in all When do we attribute knowledge to al promises; the promises of each party
programs; consider Web servers or email
daemons for instance. One might think of in-
them? What sorts of responsibilities must be supported by consideration
telligent agents as a move toward one end of can (or should) be assigned to them? i.e., something valuable given in return
the spectrum of autonomy. It is becoming increasingly clear these for the promise.”5

38 com municaTio nS o f T h e acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8


These requirements give rise to dif- artificial agents as Legal agents It will enable us to draw upon a vast
ficulties in accounting for contracts One possible solution, which would re- body of well-developed law that deals
reached through artificial agents and quire us to grant some legal standing with the agent-principal relationship,
have sparked a lively debate as to how to the programs themselves,7 would and in a way that safeguards the rights
the law should account for contracts be to treat programs as legal agents of of the principal user and all concerned
that are concluded in this way. Most their principals, empowered by law to third parties. Without this framework,
fundamentally, doctrinal difficulties engage in all those transactions cov- neither third parties nor principals are
stem from the requirement there be ered by the scope of their authority. adequately protected. Instead, we find
two parties involved in contracting: as We would understand the program as ourselves in a situation where increas-
artificial agents are not considered le- having the authority to enter into con- ingly sophisticated entities determine
gal persons, they are not parties to the tracts with customers, much as human the terms of transactions that affect
contract. Therefore, in a sale brought agents do for a corporate principal. others and place constraints on their
about by means of an artificial agent, Some of its actions will be attributed actions, though with no well-defined
only the buyer and seller can be the to its corporate principal (for instance, legal standing of their own. Viewing a
relevant parties to the contract. This the contracts it enters into), while program as a legal agent of the employ-
entails difficulties in satisfying the those outside the scope of its authority er could represent an economically ef-
requirement the two parties should will not. The ‘knowledge’ it acquires ficient, doctrinally satisfying, and fair
be in agreement, since in many cases during transactions, such as customer resolution that protects our interests,
one party will be unaware of the terms information, can be attributed to the without in any way diminishing our
of the particular contract entered into corporate principal, in the way that sense of ourselves.
by its artificial agent. Furthermore,
in relation to the requirement there
should be an intention to form legal
relations between the parties, if the
agent’s principal is not aware of the
particular contract being concluded,
how can the required intention be at-
en t
Legal scholarship has suggested
lo p m
a variety of solutions,6 ranging from
the idea programs should be treated
D e
as “mere tools” of their principals to
those suggesting programs be grant-
ed full legal personhood in order to ar t in
grant legal efficacy to the deals entered
into by them. Some of the suggested
solutions struggle to solve this prob-
lem when: protocols between buyers
and sellers (and their agents) are not
specified in advance; the terms of use
governing individual transactions are
not specified; the terms of a contract
are not finalized via human review; or knowledge of human agents is. Lastly, Rights and Legal Personhood
when agents capable of determining the established theory of liability for for artificial agents
the terms of contracts are employed. principal-agent relationships can be There are two ways to understand the
In these settings, agents might arrive applied to this situation. The details of granting of rights, such as legal agen-
at negotiated or reasoned decisions, this solution aside, the most important cy, to artificial agents. Rights might be
which their principals might not have aspect here is that, unlike a car, a pro- granted to artificial agents as a way of
agreed to had they been given the op- gram is neither a thing nor a tool; rath- protecting the interests of others; and
portunity to review the decision. Given er, it is an entity with legal standing in artificial agents might interact with,
this fact the agent cannot just be un- our system. and impinge on, social, political, and
derstood as a ‘mere tool’ or ‘means In granting the status of a legal legal institutions in such a way that
of communication’ of the principal; agent to a computer program, we the only coherent understanding of
rather, the artificial agent is better un- are not so much granting rights to their social role emerges by modifying
IllustratIon by Jo Hn H ersey

derstood as the means by which the programs as protecting those that their status in our legal system—per-
contract offer is constituted.b employ and interact with them. Un- haps treating them as legal agents of
derstanding appropriately sophisti- their principals, or perhaps treating
b This discussion is considerably oversimpli-
cated programs as legal agents of their them as legal persons like we do cor-
fied but I hope the outlines of the legal prob- principals could be a crucial step to porations or other human beings. And
lem are clear. regulating their presence in our lives. when they enjoy such elevation, they

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 39

we attended to the interests of humans.

Artificial agents have a long way to
artificial agents go before we can countenance them
have a long way to as philosophical persons. But their
roles in our society might grow to a
go before we can point where the optimal strategy is to
countenance them grant them some form of limited legal
personhood. Until then, we should ac-
as philosophical knowledge their growing roles in our
persons. lives and make appropriate adjust-
ments to our legal frameworks so that
our interests are best addressed. In-
deed, this area requires an internation-
al legal framework to address the ubiq-
uity of artificial agents on the Internet,
must conform to the standards ex- and their deployment across national
pected of the other entities that enjoy borders.d I have merely scratched the
standing in our legal system. surface of a huge, complex, multidisci-
The question of legal personality plinary debate; in the years to come, we
suggests the candidate entity’s pres- can only expect that more complexities
ence in our networks of legal and so- and subtleties will arise.
cial meanings has attained a level of
significance that demands reclassifica- d The work being done on the “Alfabiite” proj-
tion. An entity is a viable candidate for ect (at Imperial College London, NRCCL Oslo,
legal personality in this sense provided CIRFID Bologna) may be of interest in provid-
ing guidance in this regard.
it fits within our networks of social, po-
litical, and economic relations in such
a way that it can coherently be a subject References and Further Reading
of legal rulings. Thus, the real question 1. the fast-growing literature on agent technologies is
truly gigantic; for introductions, see M.J. Wooldridge,
is whether the scope and extent of ar- Reasoning about Rational Agents, MIt Press,
tificial agent interactions have reached Cambridge, Ma, 2000, and n.r. Jennings and M.J.
Wooldridge, eds., Agent Technology: Foundations,
such a stage. Answers to this question Applications and Markets, springer Verlag, 1998.
2. see
will reveal what we take to be valuable tIers.shtml
and useful in our future society as well, 3. the Proceedings of the International Conferences on
Artificial Intelligence and Law (
for we will be engaged in determin- past-icail-conferences/index.html), and the journal
ing what sorts of interactions artificial Artificial Intelligence and Law are rich sources of
information on these systems.
agents should be engaged in for us to 4. a very good source of material on the legal issues
be convinced that the question of legal generated by the increasing use of artificial agents
may be found at the law and electronic agents
personality has become a live issue. Workshops site:
While the idea of computer programs 5. Halsbury’s Laws of England (4th edition) Vol.
9 paragraph 203; c.f. restatement (second) of
being legal persons might sound fanci- Contracts, § 3
ful, it is worth noting the law has never 6. there is a large amount of literature in this area; some
very good treatments of the contracting problem may
considered humanity a necessary or be found in: t. allan and r. Widdison, “Can computers
sufficient condition for being a person. make contracts?”, Harvard Journal of Law and
Technology 9 (1996), 25–52,; a. bellia Jr., “Contracting
For example, in 19th century England, with electronic agents,” emory law Journal 50,
women were not full persons; and, in the 4 (2001), 1063; I. Kerr, “ensuring the success of
contract formation in agent-mediated electronic
modern era, the corporation has been commerce,” electronic Commerce research 1, (2001),
183–202; e. Weitzenboeck, “electronic agents and the
granted legal personhood.c The decision formation of contracts”, International Journal of Law
to grant personhood to corporations is and Information Technology 9, 3 (2001), 204–234.
Various international trade agreements such as those
instructive because it shows that grant- formulated by the unCItral or national legislations
ing personhood is a pragmatic decision such as the uCIta have not as yet resulted in clarity
in these areas.
taken in order to best facilitate human 7. s. Chopra and l. White, “artificial agents—Personhood
commerce and interests. In so doing, we in law and philosophy,” in Proceedings of the European
Conference on Artificial Intelligence, 2004 and s.
did not promote or elevate corporations; Chopra and l. White, A Legal Theory for Autonomous
Artificial Agents, university of Michigan Press, to be

c In his Max Weber Lecture, “Rights of Non-hu-

mans? Electronic Agents and Animals as New Samir Chopra ( is an
associate professor in the department of Philosophy at
Actors in Politics and Law,” Gunther Teubner
brooklyn College of the City university of new york.
notes that animals were often treated as legal
actors including being brought to trial. Copyright held by author.

40 com municaTio nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


DOI:10.1145/1787234.1787249 Thomas J. Misa, Editor

An Interview with
edsger W. dijkstra
The computer science luminary, in one of his last interviews
before his death in 2002, reflects on a programmer’s life.

he chARLes BABBAge InstItute
holds one of the world’s
largest collections of re-
search-grade oral history
interviews relating to the
history of computers, software, and
networking. Most of the 350 inter-
views have been conducted in the
context of specific research projects,
which facilitate the interviewer’s ex-
tensive preparation and often sug-
gest specific lines of questions. Tran-
scripts from these oral histories are a
key source in understanding the his-
tory of computing, since traditional
historical sources are frequently in-
complete. This interview with pro-
gramming pioneer Edsger Dijkstra
(1930–2002) was conducted by CBI
researcher Phil Frana at Dijkstra’s
home in Austin, TX, in August 2001
for a NSF-KDI project on “Building a
Future for Software History.”
Winner of ACM’s A.M. Turing
Award in 1972, Dijkstra is well known
for his contributions to computer
PHotoGra PH Court esy oF t H e unIVersIt y oF texas at au st In

science as well as his colorful assess-

ments of the field. His contributions
to this magazine continue to enrich
new generations of computing scien-
tists and practitioners.
We present this interview post-
humously on the eighth anniver-
sary of Dijkstra’s death at age 72 in
August 2002; this interview has been
condensed from the complete tran-
script, available at http://www.cbi.umn.
—Thomas J. Misa

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 41

how did your career start? Sydney or Melbourne. The final part of
It all started in 1951, when my father the journey was on an F27 to Canber-
enabled me to go to a programming i had never ra. And we arrived and I met my host,
course in Cambridge, England. It was used someone whom I had never met before. And he
a frightening experience: the first time was very apologetic that this world
that I left the Netherlands, the first time else’s software. traveler had to do the last leg of the
I ever had to understand people speak- if something journey on such a shaky two-engine
ing English. I was all by myself, trying turboprop. And it gave me the dear
to follow a course on a totally new topic. went wrong, i had opportunity for a one-upmanship that
But I liked it very much. The Nether- done it. and it was I never got again. I could honestly say,
lands was such a small country that “Dr. Stanton, I felt quite safe: I calcu-
Aad van Wijngaarden, who was the di- that unforgivingness lated the resonance frequencies of the
rector of the Computation Department that challenged me. wings myself.” [laughter]
of the Mathematical Centre in Amster- In 1956, as soon as I had decided to
dam, knew of this, and he offered me a become a programmer, I finished my
job. And on a part-time basis, I became studies as quickly as possible, since I
the programmer of the Mathematical no longer felt welcome at the univer-
Centre in March of 1952. They didn’t sity: the physicists considered me as
have computers yet; they were trying to a deserter, and the mathematicians
build them. The first eight years of my friends, because if you asked them were dismissive and somewhat con-
programming there I developed the what their professional competence temptuous about computing. In the
basic software for a series of machines consisted of, they could point out that mathematical culture of those days
being built at the Mathematical Cen- they knew everything about triodes, you had to deal with infinity to make
tre. In those years I was a very conser- pentodes, and other electronic gear. your topic scientifically respectable.
vative programmer. The way in which And there was nothing I could point to!
programs were written down, the form I spoke with van Wijngaarden in there’s a curious story behind your
of the instruction code on paper, the 1955, and he agreed that there was no “shortest path” algorithm.
library organization; it was very much such thing as a clear scientific compo- In 1956 I did two important things,
modeled after what I had seen in 1951 nent in computer programming, but I got my degree and we had the festive
in Cambridge. that I might very well be one of the peo- opening of the ARMAC.c We had to have
ple called to make it a science. And at a demonstration. Now the ARRA, a few
When you got married in 1957, you the time, I was the kind of guy to whom years earlier, had been so unreliable
could not enter the term “programmer” you could say such things. As I said, I that the only safe demonstration we
into your marriage record? was trained to become a scientist. dared to give was the generation of ran-
That’s true. I think that “program- dom numbers, but for the more reliable
mer” became recognized in the early What projects did you work on in Am- ARMAC I could try something more am-
1960s. I was supposed to study theoreti- sterdam? bitious. For a demonstration for non-
cal physics, and that was the reason for When I came in 1952, they were computing people you have to have a
going to Cambridge. However, in 1955 working on the ARRA,a but they could problem statement that non-mathema-
after three years of programming, while not get it reliable, and an updated ver- ticians can understand; they even have
I was still a student, I concluded that the sion was built, using selenium diodes. to understand the answer. So I designed
intellectual challenge of programming And then the Mathematical Centre a program that would find the shortest
was greater than the intellectual chal- built a machine for Fokker Aircraft route between two cities in the Nether-
lenge of theoretical physics, and as a Industry. So the FERTA,b an updated lands, using a somewhat reduced road-
result I chose programming. Program- version of the ARRA, was built and in- map of the Netherlands, on which I had
ming was so unforgiving. If something stalled at Schiphol. The installation selected 64 cities (so that in the coding
went wrong, I mean a zero is a zero and I did together with the young Gerrit six bits would suffice to identify a city).
a one is a one. I had never used some- Blaauw who later became one of the What’s the shortest way to travel
one else’s software. If something went designers of the IBM 360, with Gene from Rotterdam to Groningen? It is the
wrong, I had done it. And it was that un- Amdahl and Fred Brooks. algorithm for the shortest path, which
forgivingness that challenged me. One funny story about the Fairchild I designed in about 20 minutes. One
I also began to realize that in some F27: On my first visit to Australia, I flew morning I was shopping in Amsterdam
strange way, programs could become on a big 747 from Amsterdam to Los with my young fiancée, and tired, we sat
very complicated or tricky. So it was Angeles, then on another 747 I flew to down on the café terrace to drink a cup
in 1955 when I decided not to become of coffee and I was just thinking about
a physicist, to become a programmer whether I could do this, and I then
instead. At the time programming a Automatische Relais Rekenmachine Amster-
dam = Automatic Relay Calculator Amsterdam.
didn’t look like doing science; it was b Fokker Electronische Rekenmachine Te Am- c Automatische Rekenmachine MAthematische
just a mixture of being ingenious and sterdam = Fokker Electronic Calculator In Centrum = Automatic Calculator Mathemati-
being accurate. I envied my hardware Amsterdam cal Centre

42 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


designed the algorithm for the short- would write down the formal specifi- the os/360 monitor idea would have
est path. As I said, it was a 20-minute cation of the machine, and all three of never occurred to a european?
invention. In fact, it was published in us would sign it with our blood, so to No, we were too poor to consider it
1959, three years later. The publication speak. And then our ways parted. All the and we also decided that we should try
is still quite nice. One of the reasons programming I did was on paper. So I to structure our designs in such a way
that it is so nice was that I designed was quite used to developing programs that we could keep things under our in-
it without pencil and paper. Without without testing them. tellectual control. This was a major dif-
pencil and paper you are almost forced There was not a way to test them, so ference between European and Ameri-
to avoid all avoidable complexities. you’ve got to convince yourself of their can attitudes about programming.
Eventually that algorithm became, to correctness by reasoning about them.
my great amazement, one of the cor- A simple writing error did not matter how did the notion of program proofs
nerstones of my fame. I found it in the as long as the machine wasn’t there yet, arise?
early 1960s in a German book on man- and as soon as errors would show up In 1959, I had challenged my col-
agement science—“Das Dijkstra’sche on the machine, they would be simple leagues at the Mathematical Centre
Verfahren” [“Dijkstra’s procedure”]. to correct. But in 1957, the idea of a with the following programming task.
Suddenly, there was a method named real-time interrupt created a vision of Consider two cyclic programs, and in
after me. And it jumped again recently a program with non-reproducible er- each cycle a section occurs called the
because it is extensively used in all trav- rors, because a real-time interrupt oc- critical section. The two programs
el planners. If, these days, you want to curs at an unpredictable moment. My can communicate by single reads
go from here to there and you have a hardware friends said, “Yes, yes, we and single writes, and about the rela-
car with a GPS and a screen, it can give see your problem, but surely you must tive speeds of the programs nothing
you the shortest way. be up to it…” I learned to cope with it. I is known. Try to synchronize these
wrote a real-time interrupt handler that programs in such a way that at any
When was the “shortest path” algo- was flawless and that became the topic moment in time at most one of them
rithm originally published? of my Ph.D. thesis.3 Later I would learn is engaged in its critical section.d I
It was originally published in 1959 that this would almost be considered an looked at it and realized it was not
in Numerische Mathematik edited by un-American activity. trivial at all, there were all sorts of side
F.L. Bauer. Now, at the time, an algo- conditions. For instance, if one of the
rithm for the shortest path was hardly how was the computing culture in programs would stay for a very long
considered mathematics: there was a America different? time in its noncritical section, the
finite number of ways of going from A Well, the American reaction was very other one should go on unhampered.
to B and obviously there is a shortest different. When IBM had to develop the We did not allow ‘After-you-after-you’
one, so what’s all the fuss about? It re- software for the 360, they built one or blocking, where the programs would
mained unpublished until Bauer asked two machines especially equipped with compete for access to the critical sec-
whether we could contribute some- a monitor. That is an extra piece of ma- tion and the dilemma would never be
thing. In the meantime I had also de- chinery that would exactly record when solved. Now, my friends at the Math-
signed the shortest sub-spanning tree interrupts took place. And if something ematical Centre handed in their so-
for my hardware friends. You know, went wrong, it could replay it again. So lutions, but they were all wrong. For
on the big panel you have to connect a they made it reproducible, yes, but at each, I would sketch a scenario that
whole lot of points with the same cop- the expense of much more hardware would reveal the bug. People made
per wire because they have to have the than we could afford. Needless to say, their programs more sophisticated
same voltage. How do you minimize the they never got the OS/360 right. and more complicated. The construc-
amount of copper wire that connects tion and counterexamples became
these points? So I wrote “A note on two even more time-consuming, and I had
problems in connection with graphs.”2 in the mathematical to change the rules of the game. I said,
Years later when I went to my ophthal- “Sir, sorry, from now onward I only ac-
mologist—he did not even know that culture of those cept a solution with an argument why
I was a computing scientist—he said, days you had it is correct.”
“Have you designed the algorithm for Within three hours or so Th. J.
GPS?” It turned out he had seen the Sci- to deal with infinity Dekker came with a perfect solution
entific American of November 2000.10 to make your and a proof of its correctness. He had
analyzed what kind of proof would be
how could you tell if early programs topic scientifically needed. What are the things I have to
were correct? respectable. show? How can I prove them? Having
For those first five years I had always
been programming for non-existing
machines. We would design the instruc- d This is an implementation of the mutual ex-
clusion problem, which later became a corner-
tion code, I would check whether I could stone of the THE multiprogramming system
live with it, and my hardware friends [THE = Technische Hogeschool Eindhoven
would check that they could build it. I (Technical University Eindhoven)].

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 43

settled that, he wrote down a program ing into a rigorous discipline, no longer who had to agree with the report, none
that met the proof’s requirement. You a lot of handwaving. But perhaps more saw that sentence. That’s how recur-
lose a lot when you restrict the role of important, it made compiler writing sion was explicitly included.
mathematics to program verification and language definition topics worthy
as opposed to program construction or of academic attention. It played a ma- Was this called recursion at that time?
derivation. jor role in making computing science Oh yes. The concept was quite well
Another experience in 1959 was at- academically respectable. The third known. It was included in LISP, which
tending the “zeroth” IFIP Congress novelty was the introduction of the was beginning to emerge at that time.
in Paris. My international contacts type “Boolean” as a first-class citizen. We made it overlookable. And F.L.
had started in December 1958, with It turns the Boolean expression from a Bauer would never have admitted it in
the meetings for the design of ALGOL statement of fact that may be wrong or the final version of the ALGOL 60 Re-
60. My boss, Aad van Wijingaarden, right into an expression that has a val- port, had he known it. He immediately
had had a serious car accident, and ue, say, “true” or “false.” How great that founded the ALCOR Group. It was a
Jaap Zonneveld and I, as his immedi- step was I learned from my mother’s group that together would implement
ate underlings, had to replace him. reaction. She was a gifted mathemati- a subset of ALGOL 60, with recursion
Zonneveld was a numerical analyst, cian, but she could not make that step. emphatically ruled out.
while I did the programming work. For her, “three plus five is ten” was not
The ALGOL 60 meetings were about a complicated way of saying “false”; it What were other novelties in ALgoL 60?
the first time that I had to carry out dis- was just wrong. A fifth novelty that should be men-
cussions spontaneously in English. It Potentially this is going to have tioned was the block structure. It was a
was tough. a very profound influence on how tool for structuring the program, with
mathematics is done, because math- the same use of the word “structure”
you’ve remarked that learning many ematical proofs, can now be rendered as I used nine years later in the term
different languages is useful to pro- as simple calculations that reduce “structured programming.” The con-
gramming. a Boolean expression by value-pre- cept of lexical scope was beautifully
Oh yes, it’s useful. There is an enor- serving transformations to the value blended with nested lifetimes during
mous difference between one who “true.” The fourth novelty was the in- execution, and I have never been able
is monolingual and someone who at troduction of recursion into impera- to figure out who was responsible for
least knows a second language well, tive programming. Recursion was that synthesis, but I was deeply im-
because it makes you much more con- a major step. It was introduced in a pressed when I saw it.
scious about language structure in sneaky way. The draft ALGOL 60 report Finally, the definition of the seman-
general. You will discover that certain was circulated in one of the last weeks tics was much less operational than
constructions in one language you just of December 1959. We studied it and it was for existing programming lan-
can’t translate. I was once asked what realized that recursive calls were all guages. FORTRAN was essentially de-
were the most vital assets of a compe- but admitted, though it wasn’t stated. fined by its implementation, whereas
tent programmer. I said “mathemati- And I phoned Peter Naur—that call with ALGOL 60 the idea emerged that
cal inclination” because at the time it to Copenhagen was my first interna- the programming language should be
was not clear how mathematics could tional telephone call; I’ll never forget defined independent of computers,
contribute to a programming chal- the excitement!—and dictated to him compilers, stores, etc.; the definition
lenge. And I said “exceptional mas- one suggestion. It was something like should define what the implementa-
tery” of his native tongue because you “Any other occurrence of the proce- tion should look like. Now these are
have to think in terms of words and dure identifier denotes reactivation five or six issues that for many years
sentences using a language you are fa- of the procedure.” That sentence was the United States has missed, and I
miliar with. inserted sneakily. And of all the people think that is a tragedy. It was the ob-
session with speed, the power of IBM,
how was ALgoL 60 a turning point? the general feeling at the time that
Computing science started with AL- all the programming programming was something that
GOL 60. Now the reason that ALGOL 60 should be doable by uneducated mo-
was such a miracle was that it was not i did was on paper. rons picked from the street, it should
a university project but a project cre- So i was quite not require any sophistication. Yes…
ated by an international committee. false dreams paralyzed a lot of Ameri-
It also introduced about a half-dozen used to developing can computing science.
profound novelties. First of all, it in- programs without
troduced the absence of such arbitrary When did you understand that pro-
constraints as, say, ruling out the sub- testing them. gramming was a deep subject?
scripted subscript, the example I men- I had published a paper called “Re-
tioned. A second novelty was that at cursive Programming,” again in Nu-
least for the context-free syntax, a for- merische Mathematik.8 In 1961, I was
mal definition was given. That made a beginning to realize that programming
tremendous difference! It turned pars- really was an intellectual challenge.

44 comm unicaTio nS o f Th e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


Peter Naur and I were main speakers On the blackboard he wrote wall-to-
at a workshop or a summer school in wall formulae and I didn’t understand
Brighton, England; there were quite Thanks to a single word of it. But there were many
a number of well-known British sci- my isolation, people that joined the discussion and
entists in that audience. In the audi- posed questions. And I couldn’t under-
ence was Tony Hoare, but neither of i would do things stand those questions either. During
us remembers that. I don’t remember differently than a reception, I voiced my worry that I
him because he was one of the many was there on false premises. “The first
people in the audience, and he doesn’t people subjected speaker, I did not understand a word
remember it because in his memory to the standard of it.” “Oh,” he said, “none of us did.
Peter Naur and I, both bearded and That was all nonsense and gibberish,
both with a Continental accent, have pressures of but IBM is sponsoring this, so we had
merged into one person. [laughter] We conformity. to give the first slot to an IBM speaker.”
reconstructed years later that we were Well, that was totally new for me. Let’s
both there. i was a free man. say that the fence between science and
In 1962, my thinking about program industry, the fence around a university
synchronization resulted in the P- & V- campus, is here [in the U.S.] not as high
operations. The other thing I remem- as I was used to.
ber was a conference in Rome on sym-
bol manipulation, in April or so. Peter how did go to become ‘harmful’?
Naur was there, with his wife. There was [laughter] the darkest week in my In 1967 was the ACM Conference
were panel discussions and Peter and I professional life. In a NATO Confer- on Operating Systems Principles in
were sitting next to each other and we ence on Software Engineering in 1969 Gatlinburg. That, I think, was the first
had all sorts of nasty comments, but in Rome,11 I characterized the Russian time that I had a large American audi-
we made it the rule that we would go decision to build a bit-compatible copy ence. It was at that meeting where one
to the microphone in turn. This had of the IBM 360 as the greatest Ameri- afternoon I explained to Brian Randell
gone on for an hour or so, and van Wi- can victory in the Cold War. and a few others why the GO TO state-
jngaarden, my boss, was sitting next to Okay now, 1964–1965. I had general- ment introduced complexity. And they
an American and at a given moment ized Dekker’s solution for N processes asked me to publish it. So I sent an ar-
the American grabs his shoulder and and the last sentence of that one-page ticle called “A Case Against the GO TO
says “My God! There are two of them.” article is, “And this, the author believes, Statement” to Communications of the
[laughter] This may be included in an completes the proof.” According to ACM. The editor of the section wanted
oral history? It’s not mathematics, it Doug Ross, it was the first publication to publish it as quickly as possible, so
isn’t computer science either, but it is of an algorithm that included its cor- he turned it from an article into a Let-
a true story…. rectness proof. I wrote “Cooperating ter to the Editor. And in doing so, he
In September 1962, I went to the first Sequential Processes,” and I invented changed the title into, “GO TO State-
IFIP Congress, in Munich, and gave an the Problem of the Dining Quintuple, ment Considered Harmful.”4 That title
invited speech on advancing program- which Tony Hoare later named the became a template. Hundreds of writ-
ming. I got a number of curtain calls: Problem of the Dining Philosophers.5 ers have “X considered harmful,” with
clearly I was saying something unusu- X anything. The editor who made this
al. Then I became a professor of Mathe- When did you first visit the u.s.? change was Niklaus Wirth.
matics in Eindhoven, and for two years My first trip to the U.S. was in 1963.
I lectured on numerical analysis. By That was to an ACM Conference in Why is “elegance” in programming im-
1963–1964, I had designed with Carel Princeton. And I visited a number of portant?
S. Scholten the hardware channels and Burroughs offices; that was the first 1968 was exciting because of the
the interrupts of the Electrologica X8, time I met Donald Knuth. I must al- first NATO Conference on Software
the last machine my friends built, and ready have had some fame in 1963, Engineering, in Garmisch. In BIT I
then I started on the design of THE because there was an ACM workshop published a paper, on “A Constructive
multiprogramming system. with about 60 to 80 participants and I Approach to the Problem of Program
Of course, 1964 was the year in was invited to join. And they paid me Correctness.”1 1968 was also the year
which IBM announced the 360. I was $500. I didn’t need to give a speech, I of the IBM advertisement in Datama-
extremely cross with Gerry Blaauw, didn’t need to sit in a panel discussion, tion, of a beaming Susie Meyer who
because there were serious flaws built they just would like me to be there. had just solved all her programming
into the I/O organization of that ma- Quite an amazing experience. problems by switching to PL/I. Those
chine.7 He should have known about were the days we were led to believe
the care that has to go into the design What about your first two trips to Amer- that the problems of programming
of such things, but that was clearly not ica surprised you about the profession? were the problems of the deficiencies
a part of the IBM culture. In my Tur- Well, the first lecture at that ACM of the programming language you
ing Lecture I described the week that I workshop was given by a guy from IBM. were working with. How did I char-
studied the specifications of the 360, it It was very algebraic and complicated. acterize it? “APL is a mistake, carried

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 45

through to perfection. It is the lan- was a very heavy use of anthropomor-

guage of the future for the program- phic terminology, the “electronic
ming techniques of the past: it creates brain” or “machines that think.” That
a new generation of coding bums.” is absolutely killing. The use of anthro-
I thought that programmers should pomorphic terminology forces you
Content Written not be puzzle-minded, which was one linguistically to adopt an operational
of the criteria on which IBM selected view. And it makes it practically impos-
by Experts programmers. We would be much bet- sible to argue about programs inde-
ter served by clean, systematic minds, pendently of their being executed.
with a sense of elegance. And APL, with
its one-liners, went in the other direc- Is this why artificial intelligence re-
tion. I have been exposed to more APL search seemingly doesn’t take hold in
than I’d like because Alan Perlis had europe?
an APL period. I think he outgrew it be- There was a very clear financial
Blogs fore his death, but for many years APL constraint: at the time we had to use
was “it.” the machines we could build with the
Articles available stuff. There is also a great
Roundtables Why did your “structured program- cultural barrier. The European mind
ming” have such impact? tends to maintain a greater distinction
Case Studies In 1969, I wrote “Notes on Struc- between man and machine. It’s less in-
9-513 tured Programming,”6 which I think clined to describe machines in anthro-
owed its American impact to the fact pomorphic terminology; it’s also less
2.25 X 4.75
that it had been written at the other inclined to describe the human mind
side of the Atlantic Ocean; which has in mechanical terminology. Freud
two very different sides. I can talk never became the rage in Europe as he
about this with some authority, hav- became in the United States.
ing lived here [in the U.S.] for the bet-
ter part of 17 years. I think that thanks you’ve said, “the tools we use have a to the greatly improved possibility of profound and devious influence on
communication, we overrate its im- our thinking habits, and therefore on
portance. Even stronger, we underrate our thinking abilities.”
IMAGINE... the importance of isolation. See, look
at what that 1963 invitation to the ACM
The devious influence was in-
spired by the experience with a bright
a graduate computer science program workshop illustrates, at a time when I student. In the oral examination we
that offers you the convenience and access of online had published very little. I had imple- solved a problem. Together we con-
learning, combined with the benefits of participating
in live classroom discussion and interaction.
mented ALGOL 60 and I had written a structed the program, decided what
real-time interrupt handler, I had just had to be done, but very close to the
It’s here... the Brooklyn Campus become a professional programmer. end, the kid got stuck. I was amazed
of Long Island University is offering Yet I turned out to be quite well known. because he had understood the prob-
a NEW BLENDED LEARNING program How come? Thanks to my isolation, I lem perfectly. It turned out he had to
that fuses online learning with traditional classroom would do things differently than peo- write a subscripted value in a subscript
studies, significantly reducing the amount of time ple subjected to the standard pres- position, the idea of a subscripted sub-
you’ll spend on campus and maximizing interaction sures of conformity. I was a free man. script, something that was not allowed
with faculty members and fellow students. in FORTRAN. And having been educat-
What were other differences between ed in FORTRAN, he couldn’t think of
M.S. in Computer Science europe and the u.s.? it, although it was a construction that
Brooklyn Campus One of the things that saved Europe he had seen me using at my lectures.
Information Session was that until 1960 or so, it was not con-
Wednesday, August 18, 6:00 p.m.
Saturday, August 21, 10:30 a.m.
sidered an interesting market. So we so the use of FoRtRAn made him un-
718-488-1011 • were ignored. We were spared the pres- able to solve that?
sure. I had no idea of the power of large Indeed. When young students have
companies. Only recently I learned that difficulty in understanding recursion,
in constant dollars the development of it is always due to the fact that they had
the IBM 360 has been more expensive learned programming in a program-
than the Manhattan Project. ming language that did not permit it.
I was beginning to see American If you are now trained in such an op-
publications in the first issue of Com- erational way of thinking, at a given
munications of the ACM. I was shocked moment your pattern of understand-
by the clumsy, immature way in which ing becomes visualizing what happens
greater access to excellent education
they talked about computing. There during the execution of the algorithm.

46 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


The only way in which you can see the pressure on the universities to supply
algorithm is as a FORTRAN program. them, even if the university did not
in many places, quite know how. In many places, de-
And what’s the answer then for our fu- departments of partments of computer science were
ture students to avoid the same trap? founded before the shape of the intel-
Teach them, as soon as possible, a computer science lectual discipline stood out clearly.
decent programming language that ex- were founded before You also find it reflected in the
ercises their power of abstraction. Dur- names of scientific societies, such as
ing 1968 in Garmisch I learned that in the shape of the the Association for Computing Ma-
the ears of the Americans, a “math- intellectual discipline chinery. It’s the British Computer Soci-
ematical engineer” [such as we educat- ety and it was the Dutch who had Het
ed in Eindhoven] was a contradiction stood out clearly. Nederlands Rekenmachine Genootsc-
in terms: the American mathematician hap; without knowing Dutch, you can
is an impractical academic, whereas hear the word “machine” in that name.
the American engineer is practical And you got the departments of Com-
but hardly academically trained. You puter Science. Rather than the depart-
notice that all important words carry ment of computing science or the de-
different, slightly different meanings. partment of computation. Europe was
I was disappointed in America by the of the population it is supposed to ad- later, it coined the term Informatics.
way in which it rejected ALGOL 60. I dress have changed radically. That al- Tony Hoare was a Professor of Compu-
had not expected it. I consider it a trag- ready started in the 1970s. So whatever tation.
edy because it is a symptom of how I say about the [European] university
the United States is becoming more is probably idealized by memory. Yes. “Information” came a bit later on?
and more a-mathematical, as Morris But a major difference was that the It was the French that pushed in-
Kline illustrates eloquently.9 Precisely fence around the university campus formatique. Today the English prefer
in the century which witnesses the was higher. To give you an example, Information Technology, IT, and In-
emergence of computing equipment, when we started to design a comput- formation Systems, IS. I think the tim-
it pays so much to have a well-trained ing science curriculum in the 1960s, ing has forced the American depart-
mathematical mind. one of the firm rules was that no in- ments to start too early. And they still
dustrial product would be the subject suffer from it. Here, at the University
In 1963 peter patton, in Communica- of an academic course. It’s lovely. This of Texas, you can still observe it is the
tions of the ACM, wrote that european immediately rules out all Java courses, Department of Computer Sciences. If
programmers are fiercely independent and at the time it ruled out all FOR- you start to think about it, you can only
loners whereas Americans are team TRAN courses. We taught ALGOL 60, laugh, but that time there were at least
players. or is it the other way? it was a much greater eye-opener than as many computer sciences as there
At the Mathematical Centre, we FORTRAN. were professors.
used to cooperate on large projects
and apply a division of labor; it was Is there a relationship between the References
something of a shock when I went to curriculum and the nature of funding 1. dijkstra, e.W. a constructive approach to the problem
of program correctness. BIT 8, 3 (1968), 174–186.
the Department of Mathematics at of universities? 2. dijkstra, e.W. a note on two problems in connection
Eindhoven where everybody worked Yes. It has the greatest influence on with graphs. Numerische Mathematik 1 (1959),
all by himself. After we had completed the funding of research projects. Quite 3. dijkstra, e.W. Communication with an automatic
the THE System, for instance, Nico regularly I see firm XYZ proposing to computer. Ph.d. dissertation, university of
amsterdam, 1959.
Habermann wrote a thesis about the give student fellowships or something 4. dijkstra, e.W. Go to statement considered harmful,
Commun. ACM 11, 3 (Mar. 1968), 147–148.
Banker’s Algorithm, and about sched- and then, somewhere in the small 5. dijkstra, e.W. Hierarchical ordering of sequential
uling, sharing, and deadlock preven- print, that preference will be given to processes. Acta Informatica 1 (1971), 115–138.
6. dijkstra, e.W. notes on structured programming.
tion. The department did not like that students who are supervised by pro- In o.-J. dahl, e.W. dijkstra, and C.a.r. Hoare, eds.,
because it was not clear how much he fessors who already have professional Structured Programming. academic Press, london,
1972, 1–82.
had done by himself. They made so contact with the company. 7. dijkstra, e.W. over de IbM 360, eWd 255, n.d.,
much protest that Cor Ligtmans, who circulated privately; users/
should have written his Ph.D. thesis on Why do computer science depart- 8. dijkstra, e.W. recursive programming. Numerische
another aspect of THE System, refused ments often come out of electrical Mathematik 2 (1960), 312–318.
9. Kline, M. Mathematics in Western Culture. Penguin
to do so. engineering in the u.s.—but not in eu- books ltd., Harmondsmorth, Middlesex, england,
rope? 1972.
10. Menduno, M. atlas shrugged: When it comes to online
Is the outcome of the curricula differ- A major reason is timing. For fi- road maps, why you can’t (always) get there from
ent in europe and America? nancial reasons, Europe, damaged by here. Scientific American 283, 11 (nov. 2000), 20–22.
11. randell, b. and buxton, J.n., eds., Software
I must be very careful with answer- World War II, was later. So the Ameri- Engineering Techniques: A Report on a Conference
ing this because during my absence, can computing industry emerged ear- Sponsored by the NATO Science Committee (rome,
Italy, oct. 1969), nato, 1970.
the role of the university, the financ- lier. The computing industry asked
ing of the university, and the fraction for graduates, which increased the Copyright held by author.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 47
Doi:10.1145/ 1787234.1787250
Partway through this code-under-
Article development led by
standing task, there’s a knock at the
door. It’s Joe, the newest member of the
team. He is working on a bug and is con-
Could ubiquitous hand-drawn code map fused about how one of the product’s fea-
diagrams become a thing of the past? tures is implemented. As the team histo-
rian, Jane is accustomed to this type of
By RoBeRT DeLine, Gina VenoLia, anD kaeL RoWan question. They start the conversation
by looking at an architectural diagram

tacked to the wall near Jane’s computer.
To get into specifics, Jane draws a ver-
sion of the diagram on the whiteboard,
sketching only the relevant parts of the

architecture but in more detail than
the printed diagram. As she talks Joe
through a use case, she overlays the dia-
gram with arrows to show how different
parts of the system interact. From time

with code
to time, she brings up relevant code in
her development environment to relate
the diagram back to the code.
After several minutes, Joe feels con-

fident he understands the design and
heads back to his office. Jane goes back
to her own work. Between exploring the
search results and answering Joe’s ques-
tions, Jane’s development environment
now has dozens of open documents.
Jane tries to resume her task but cannot
find where she left off in all the clutter.
She closes all open documents, reissues
her original search, finds her place in
soFtWARe deVeLopeRs ReguLARLy draw diagrams of their the search results, and carries on ex-
ploring the dependency on the unsup-
systems. To get a sense of how diagramming fits into ported library.
a developer’s daily work, consider this fictitious, but This story illustrates the wide range
representative story: of diagramming practice. The dia-
grams range in quality from sketches
Jane is a developer who has been a member of her team to high-quality posters; in formality,
so long that everyone calls her the team historian. Since from idiosyncratic scribbles to well-
defined notations; in longevity, from
the product just shipped a few weeks ago, Jane is finally the duration of a single task to an
getting around to some code cleanup she had planned entire release cycle; and in audience,
for ages—namely, dropping a dependency on a library from solo use, to anchoring a conver-
sation, to communicating with the
that is no longer supported. Jane uses her development whole team or user community.
environment to search for all the places where her Although the practice illustrated
in the story is widespread and useful,
PHotoGra PH deta Il oF F IG ure 1 .

product uses the unsupported library. She clicks through there are a few downsides where soft-
the results one by one and reads the code to understand ware could make an improvement.
how it uses the library. As she jumps around the code First, the diagrams are typically not
tied to the code. To go from architec-
base, she sketches a class diagram on a notepad to ture-level to code-level discussions re-
capture the architectural dependencies she discovers. quires switching tools—for example,

48 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

en t
lo p m
D e ve
ar t in
Cred It t K

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 49

figure 1. four diagrams of the oahu system.

from the whiteboard to the develop- reproduce parts of the poster on the no visual transition to show where
ment environment. This separation whiteboard in order to answer Joe’s the jump landed. The more she navi-
also causes poor task support. For specific questions. Third, there is a gates, the greater the pileup of docu-
example, Jane’s code search and note- lost opportunity to help deal with the ment tabs. These navigation steps are
pad diagram are not tied together in disorientation Jane feels when con- not just for editing the code. Software
any way. The two can easily get out of fronting all the open documents on developers also have a frequent need
sync and cannot be stored or retrieved resuming her task. for information during their program-
together, as when Jane’s diagram was Getting lost in a large code base is ming tasks.7,8 To try to find answers,
available during task resumption but altogether too easy. The code consists they browse around the code and other
her search results were gone. of many thousands of symbols, with documents, which adds both relevant
Second, there is no transition be- few visual landmarks to guide the eye. and irrelevant documents to the envi-
tween team-level documentation As a developer navigates the code, she ronment’s working set. With so much
and task- or conversation-specific follows hyperlinks, such as jumping discontinuous navigation, a developer
diagrams. For example, Jane has to from a method caller to a callee, with can easily become disoriented.
Better support for code diagrams
Table 1. information needs that are related to diagramming behavior. in the development environment
could support code understanding
and communication, and could serve
1. What code could have caused this behavior? 1. What is the purpose of this code? as a “map” to help keep developers
2. What is statically related to this code? 2. What is the program supposed to do? oriented. The software visualization
3. What code caused this program state? 3. Why was this code implemented this way? community has previously explored
4. What are the implications of this change? different types of maps, such as zo-
omable box-and-line diagrams9 and
cityscapes,10 for tasks such as pro-
gram understanding and analyzing

50 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


project data. These maps are typically derstanding existing code and design-
designed to supplement a standard ing/refactoring often involved pairs or
development environment. Our goal small groups.
is to integrate maps into the develop- In a separate study we sought to
ment environment such that develop-
ers can carry out most tasks within Better support for understand developers’ information
needs while carrying out their devel-
the map. code diagrams in opment tasks.7 We observed 17 devel-

the development
To address these issues, using a us- opers at Microsoft for approximately
er-centered approach we are design- 90 minutes each, manually recorded
ing an interactive code map for devel-
opment environments. In preparation
environment could their activity minute by minute, and
coded these logs into 334 instances of
for our initial design we conducted support code information-seeking behavior. From
a series of field studies at Microsoft
Corporation. We interviewed devel-
understanding and this data, we identified 21 general in-
formation needs, clustered into seven
opers to find how and why they draw communication, work categories. Consistent with the
diagrams of their code, and we col-
lected many example diagrams along and could serve previous study, we found many of
their information needs fell into the
the way.3 We also directly observed as a “map” to help categories of understanding execution
developers at work to watch their
information-seeking behavior and to keep developers behavior and reasoning about design,
see Table 1.
catalog their information needs.7 Fi-
nally, we did a participatory design of
oriented. In our observations, ad hoc com-
munication with coworkers was a
a paper-based code map to allow a de- common way of addressing a variety
velopment team to design its content of information needs. Table 2 shows
and appearance and to witness how it the information needs that were most
supported their conversations.1 Using frequently addressed by talking with
insights from these three studies, we coworkers. This reliance on conver-
are actively prototyping Code Canvas, sations with coworkers corresponds
a Microsoft Visual Studio plug-in that with the ad hoc meeting scenario from
replaces the tabbed documents with a the diagramming study.
zoomable code map.5 From these two studies we know
that developers have frequent, spe-
how and Why Developers Diagram cific information needs when trying
To better understand how profession- to understand existing code and plan-
al software developers use visual rep- ning code changes, and they often use
resentations of their code, we inter- diagrams when looking for answers.
viewed nine developers at Microsoft to This suggests the plausible utility of
identify common scenarios, and then a code map that answers these needs
surveyed more than 400 developers to either directly or through interaction.
understand the scenarios more deep- We also know that developers often
ly.3 The three most frequently men- turn to coworkers to find the answers
tioned scenarios were: they need, and they create diagrams to
˲˲ Understanding existing code. Ex-
amining source code and its behavior Table 2. Top information needs for
which software developers turned to
to develop an understanding of it. their coworkers.
˲˲ Designing/refactoring. Planning
how to implement new functionality,
fix a bug, or make the program struc-
1. What have my coworkers been doing?
ture better match its functionality.
˲˲ Ad hoc meetings. Asking a cowork- 2. What are the implications of this change?

er to explain existing code, vet a deci- 3. Is this problem worth fixing?

sion, or help work through a problem. 4. What is the program supposed to do?
Developers rated these three
5. In what situations does this failure occur?
among the most important to their
job functions. More than half of sur- 6. How have resources I depend
on changed?
vey respondents indicated that dia-
7. What code could have caused
grams were important in these sce- this behavior?
narios. Most ad hoc meetings were
small, involving two or at most five
people. While typically done solo, un-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 51

supplement their conversations. This between two entities. Generally, boxes it, and then revised the drawing based
suggests that the code map should be were arranged so that relationships on their feedback. At their request,
shared among teammates so they have flowed in a more-or-less orderly di- we incorporated types and method
a common spatial frame of reference. rection, top to bottom or left to right. signatures reverse-engineered from
High-level groupings were indicated their code, using a tool we developed
Designing a code map by surrounding boxes or curves, or by for the purpose.
The question remains, what should dividing lines. These visual conven- Through this process we arrived at
the code map look like? We collected tions suggest a starting point for the a design (Figure 2) that represented
many examples of developers’ visual design of a code map. the code in a way that was meaningful
representations of their code and Armed with this general knowl- to the team. The final design was ba-
identified the visual conventions they edge, we worked closely with a soft- sically an architectural layer diagram
used.3 These ranged from sketches ware development team called Oahu sprinkled with types (white boxes)
on whiteboards to diagrams care- (a pseudonym) to develop a paper containing method signatures. It
fully made using a drawing tool. We prototype of a code map.2 The Oahu closely followed the visual conven-
also looked at the visual conventions team consisted of a few dozen people tions we found in the earlier study. It
used by developers when represent- working on an incubation project of also included some features that are
ing code.2 Box-and-arrow diagrams around 75,000 lines of C#. We first not typical in architectural diagrams,
were by far the most common repre- had each developer separately sketch such as representations of planned,
sentation, where each box represent- the Oahu project on a large piece but nonexistent code (for example,
ed some kind of software entity and of paper. Four of these sketches are the empty white box beneath the mo-
each arrow indicated a relationship shown in Figure 1. Next we synthe- bile phones), colorized identifier frag-
between two entities. Boxes were typi- sized into a master drawing the com- ments to aid visual searching (which
cally labeled, but arrows almost never mon features and interesting excep- we call concept keyword colorization),
were. tions that appeared in the sketches. and the vertical banding representing
Some of these diagrams made casu- For several weeks we repeated a daily concepts that cut across architectural
al use of visual notations, such as UML cycle where we printed this drawing, layers.
(Unified Modeling Language), but this hung it in the developers’ offices, From these studies we learned that
was uncommon. Adjacency was some- interviewed the team members for it is possible to design a code map
times used to indicate a relationship changes and reports of how they used from a simple set of visual conven-

figure 2. The oahu code map designed through team participation, with a callout showing
the map at full resolution. The dashed rectangle is the map region shown in figure 3.

52 comm unicaTio nS o f T he acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8


figure 3. The code canvas version of the oahu map, focused on the upper left corner of the ui layer. The map includes two overlays: search
results in yellow and an execution trace as a series of red arrows. The callout shows the result of zooming into a method to edit its code.

tions. The Oahu code map showed relying on tabbed documents and hi- tioned in Code Canvas in the same lay-
that a single map could represent an erarchical overviews to navigate and out as the Oahu map.
entire software project in a way that edit the code, Code Canvas places all An important lesson from the
was meaningful to all the developers of a project’s documents (code files, Oahu research is that developers as-
on the team. The team’s response to icons, user interface designs, among sign meaning to the spatial layout of
the map was mixed. Two new hires on others) onto a panning, zooming code the code. Code Canvas therefore takes
the Oahu team used the map exten- map. The user can zoom out to get an a mixed initiative approach to layout.
sively as part of their “onboarding” overview of the project’s structure and The user is able to place any box on
process, studying and annotating it of- zoom in to view or edit code and other the map through direct manipulation,
ten. Other team members had several documents. (A video of Code Canvas is but Code Canvas also uses the MSAGL
criticisms, all stemming from the lack available at (Microsoft Automatic Graph Layout)
of interaction. They wanted to tailor watch?v=tsFfyli2Y9s.) engine (
the level of detail and the element po- We designed the look of the Code com/msagl) to provide an initial lay-
sitions to the needs of the discussion Canvas map based on our experience out for new code maps and to prevent
to change the content for the task (for with the Oahu team. Figure 3 shows occlusion and maintain relationships
example, add call graphs to the map). the Oahu project loaded into Code as the user makes subsequent chang-
We were able to address all these con- Canvas, in particular the upper left- es to the layout.
cerns in our Code Canvas. hand corner of the UI layer (the area Code Canvas uses a technique
indicated with a dashed rectangle in called semantic zoom to show differ-
maps at the center of the Figure 2). Using the visual conventions ent levels of detail at different levels
Development environment from the Oahu map, the Code Canvas of zoom. At the 10%-level of zoom, the
We’ve incorporated insights from map shows types as white boxes, with code itself is invisible because its size
these studies into a prototype user the identifiers labeled using concept is less than a pixel per line, but the type
interface for Microsoft Visual Studio, keyword colorization, and with types names and member names are shown
called Code Canvas, which makes a organized into labeled bands (Popup at a readable size. The callout in Fig-
code map the central metaphor of the Menu, Reminders, among others). ure 3 shows the 100%-level of zoom,
development experience.5 Rather than The type and concept boxes are posi- where the code file itself is displayed

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 53

using the standard editor, which pro- stack. She zooms out to give Jim a tour References
vides the usual syntactic formatting of the parts of the code involved in the 1. bragdon, a., reiss, s.P., Zeleznik, r., Karumuri, s.,
Cheung, W., Kaplan, J., Coleman, C., adeputra,
and coloring and standard editor fea- feature. When Joe asks detailed ques- F., laViola Jr., J.J. Code bubbles: rethinking the
tures such as code completion. At in- tions about the algorithms, Jane zooms user interface paradigm of integrated development
environments. In Proceedings of the 32nd
termediate levels of zoom the code be- in on the relevant code. When the con- International Conference on Software Engineering
comes visible, first in a skeletal form versation with Joe is over, Jane simply (2010).
2. Cherubini, M., Venolia, G., deline, r. building
(in the style of Seesoft,6 a well-known closes the new tabs and returns to the an ecologically valid, large-scale diagram to
software visualization tool from the one where she was working, which looks help developers stay oriented in their code. In
Proceedings of the IEEE Symposium on Visual
early 1990s), then as readable text. exactly as she left it. Languages and Human-Centric Cowmputing
(sept. 2007).
For a tour of Code Canvas’s fea- In short, Code Canvas provides ex- 3. Cherubini, M., Venolia, G., deline, r., Ko, a.J. let’s go
tures, let’s replay our initial story. plicit task support through multiple to the whiteboard: How and why software developers
use drawings. In Proceedings of the SIGCHI
Jane’s development environment canvases and uses stable, spatial lay- Conference on Human Factors in Computing Systems
shows an overview map of the whole outs to keep users oriented. These (May 2007).
4. deline, r., Czerwinski, M., Meyers, b., Venolia,
project, called the HOME canvas. Its design goals are shared by the Code G., drucker, s., robertson, G. Code thumbnails:
layout is as familiar to her as her home- Bubbles project at Brown University.1 using spatial memory to navigate source code.
In Proceedings of the IEEE Symposium on Visual
town, since she has been moving around Code Bubbles’ strategy is to start with Languages and Human-centric Computing (2006).
both of them for years. To start her task an empty canvas and add items as the 5. deline, r., rowan, K. Code Canvas: Zooming toward
better development environments. In Proceedings
of understanding her project’s depen- user searches and browses the proj- of the International Conference on Software
dency on the unsupported library, she ect. In contrast, Code Canvas starts Engineering (New Ideas and Emerging Results).
May 2010.
searches for uses of the library. The with an overview and allows users to 6. eick, s.C., steffen, J.l., sumner Jr., e.e. 1992.
search results are overlaid on the map filter down to items of interest. In our seesoft: a tool for visualizing line-oriented
software statistics. IEEE Transactions on Software
in yellow boxes (as shown in Figure 3) future work, we will explore hybrids of Engineering 18, 11 (1992), 957-968.
in addition to being listed in a separate the two approaches. 7. Ko, a.J., deline, r., Venolia, G. Information needs
in collocated software development teams. In
window. She immediately sees the two Proceedings of the 29th International Conference on
conclusion Software Engineering (May 2007).
parts of the code that depend on the li- 8. sillito, J., Murphy, G. C., de Volder, K. 2008. asking
brary. She zooms into one of them to Based the work practices we observed and answering questions during a programming
change task. IEEE Transactions on Software
look closer at exactly which classes are in our field studies, we believe making Engineering.
implicated and then clicks on an indi- a code map central to the user inter- 9. storey, M.a., best, C., Michaud, J., rayside, d.,
litoiu, M., Musen, M. sHriMP views: an interactive
vidual search result to look at the code face of the development environment environment for information visualization and
itself. promises to reduce disorientation, an- navigation. In Proceedings of the Conference on
Human Factors in Computing Systems (May 2002).
After exploring this way for a while, swer common information needs, and 10. Wettel, r., lanza, M. Visualizing software systems
she decides to focus on just the relevant anchor team conversations. Spatial as cities. In Proceedings of the IEEE International
Workshop on Visualizing Software for Understanding
code, so she creates a new “filtered can- memory and reasoning are little used and Analysis (2007).
vas” in a new tab that contains the subset by software developers today. In a lab-
of the code containing the search results, based evaluation of a previous version Robert DeLine (
maintaining the spatial relationships of our code-map design, we showed is a Principal researcher at Microsoft research, working
at the intersection of software engineering and human-
that help her stay oriented. As on the that developers form a reliable spatial computer interaction. His research group designs
development tools in a user-centered fashion: they
HOME canvas, the code on the filtered memory of a code map during 90-min- conduct studies of development teams to understand their
canvas is shown inside boxes represent- ute sessions of programming tasks.4 work practice and prototype tools to improve that practice.
ing the relevant classes and interfaces. By exploiting these cognitive resourc- Kael Rowan ( is a
This filtered canvas acts as the class dia- es, code maps will allow developers senior research software design engineer at Microsoft
research, focusing on the next generation of software
gram Jane previously drew on her note- to be better grounded in the code, development including software visualization and
pad, except here the search results and whether working solo or collabora- spatial layout of source code. His background has gone
from operating system internals and formal software
class diagram are automatically kept in tively. We believe this will fundamen- verification to modern user interfaces and HCI.
sync and are persisted together. tally change and improve the software
Gina Venolia ( is a
Joe knocks on the door and asks Jane development experience. senior researcher with Microsoft research in the Human
a question. She clicks over to the HOME Interactions of Programming group. Her research focuses
on building systems that make knowledge flow more
canvas tab, zooms out, and points at freely among people. she is studying distributed software
parts of it to support what she’s saying. teams and developing systems that exploit spatial
Related articles memory to support navigation and team awareness.
The HOME canvas is shared among all
team members, precisely to provide a
common ground around which the team
Code Spelunking Redux
can have discussions. To explain the George V. Neville-Neil
feature that is puzzling Joe, Jane sets a
debugger breakpoint and runs the pro- Visualizing System Latency
gram. When the breakpoint is reached, Brendan Gregg
Code Canvas shows the call stack us-
ing a series of red execution arrows, like The Woes of IDEs
those in Figure 3. Jane then creates a sec- Jef Raskin
ond filtered canvas, focused on this call © 2010 aCM 0001-0782/10/0800 $10.00

54 com municaTio nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

Doi:10.1145/ 1787234. 1 78 72 5 1

Article development led by

Leading experts debate how virtualization and

clouds impact network service architectures.
By mache cReeGeR

moving to
the edge:
a cTo
on network
the geneRAL It community is just beginning to digest
how the advent of virtual machines and cloud
computing are changing their world. These new
technologies promise to make applications more
portable and increase the opportunity for more
flexibility and efficiency in both on-premise and

outsourced support infrastruc- In this ACM CTO Roundtable, lead-

tures. Virtualization can break ing providers and users of network
long-standing linkages between virtualization technologies discuss
applications and their support- how virtualization and clouds impact
ing physical devices, however. network service architectures, both in
Before data-center managers can take their abilities to move legacy applica-
advantage of these new opportunities, tions to more flexible and efficient vir-
they must have a better understanding tualized environments and to enable
of service infrastructure requirements new types of network functionality.
and their linkages to applications. —Mache Creeger

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 55

clockwise from top left: Surendra Reddy, mache creeger, martin casado, and charles Beeler.

Participants able. I would like each of you to com- cause the core can now handle so many
Simon Crosby is the CTO of Virtu- ment on the challenges and opportuni- workloads, dedicated network devices
alization and Management Division, ties people will face in the next couple are not being asked to solve the same
Citrix Systems. of years as the world progresses with problem. Networks in the past have re-
Oliver Tavakoli is the CTO and VP these new platform architectures. inforced the concept that user equals
of SLT Architecture and Technology cRoSBy: Virtualization challenges device, equals port, equals location.
Group, Juniper Networks. the binding of infrastructural services With virtualization, those identity re-
Lin Nease is director of Emerging to physical devices. One can no lon- lationships are now dead. Networking
Technologies for ProCurve Network- ger reason about the presence or the will need to evolve as a result.
ing, Hewlett-Packard. utility of a service function physically caSaDo: Networks have always been
Martin Casado is VP, CTO, and bound to a device and its relationship built in support of some other high-
founder of Nicira, Inc. to a specific workload. Workloads now er-priority requirements. As a result
Surendra Reddy is VP of Cloud Com- move around, based on demand, re- people have never been required to
puting, Yahoo! sponse time, available service capac- produce good stand-alone network
Charles Beeler is General Partner of ity, resource prices, and so on. While architectures. If I’m building an oper-
El Dorado Ventures. the networking industry was founded ating system that people use as a plat-
Steve Bourne is Chair of ACM’s Pro- on a value proposition tied to a specific form for applications, I must have nice
fessions Board; CTO of El Dorado Ven- physical box, virtualization as a separa- application platform abstractions.
tures; and past president of ACM. tion layer has introduced a profound Networks have never had to do that.
Mache Creeger (Moderator) is Princi- challenge to that premise. Moreover, Originally, the leverage point was
pal of Emergent Technology Associates. given the progress of Moore’s Law and in the network because it was central.
the large number of VMs (virtual ma- Because of this, networks have always
cReeGeR: Our discussion will focus chines) we can run per server, the im- been an obvious place to put things
on how virtualization and clouds im- plicit change to networking is that the such as configuration state. Now the le-
PHotoGra PHs by toM uPton

pact network service architectures, last-hop switch is necessarily a feature verage point is at the edge because the
both in the ability to move legacy appli- of the hypervisor or hardware of the semantics there are very rich. I know
cations to more flexible and efficient server and not a traditional hardware where a VM is, I know who’s on it, and I
virtualized environments and what switch in the physical network. know when it joins and when it leaves.
new functionality may become avail- neaSe: We’ve broken a paradigm. Be- As a result, I don’t require traditional

56 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


clockwise from top left: oliver Tavakoli, Lin nease, Simon crosby, and Steve Bourne.

service discovery and often don’t need where the semantics are. work that satisfies the requirement. I
multicast. Because the leverage point cRoSBy: Networking vendors sell dif- don’t care if it’s Cisco, Juniper, or what-
is at the edge, the dynamic changes ferentiated networking value proposi- ever. What I want is a service provider
completely; and because the semantics tions to their customers. As IaaS (Infra- that constructs and delivers the network
are now more interesting at the edge, structure as a Service) comes into wider that is required. As the end user, I care
you have a clash of paradigms. use, APIs will shift. If I have an invest- about only the above-the-line result.
neaSe: We’ve seen the same process ment in IT skill sets to manage Juniper cRoSBy: Deciding whether to buy an
take place with blade servers. When equipment in my private data center, HP or a Juniper switch is a localized
we started centralizing some of the can I use those skills, configurations, problem of scale, complexity, IC inte-
functions that used to be distributed and policies off-premise in the cloud? gration, and so forth. To outsource that
among many servers, we could easily IaaS challenges the traditional ven- work, I will have to go to a large num-
and authoritatively know things that dor/customer roles for networking ber of cloud vendors and attempt to de-
used to be difficult to obtain. Things— equipment. It may be that the cloud termine for my combined networking
for example, in station state, address, vendor purchased equipment from a and compute problem how to choose
or user—became much easier to deter- specific vendor, but there is no way for the best vendor. That’s way too hard if
mine because the new blade architec- that vendor to surface its unique value I already have a strong preference for
ture made it very convenient. proposition to the IaaS customer. Does and investment in a particular vendor’s
caSaDo: Most scalable clouds are sub- this necessarily force commoditiza- equipment, value, and management.
networked. They are architected to not tion in network equipment? I think it Alternatively, if we pursue a “ge-
deal with VLANs (virtual LANs) or do flat does. Google, for example, reportedly nericized” feature list on the part of
anything and are not going to try and do already builds its own networking gear virtualized networks, we make it diffi-
one big L2 domain. It’s all subnet’ed up- from industry-standard parts. cult for application owners to support
front with the cloud overlaid on top. ReDDy: In the next two to three years their rich needs for control. After all,
Networks do not have virtualization our goal is to make the building of an the richness of a switch/router feature
built in, and VLANs are not virtualiza- application, its packaging, and deploy- set is designed specifically to address
tion in a global sense. It’s easier just to ment completely transparent. I want to customer needs. If we genericize those
treat the entire network infrastructure specify SLA (service-level agreement), features, we may not be able to support
as a big dumb switch and hold the in- latency, and x-megabit-per-second features in the cloud rich enough to
telligence at the edge because that is throughput and receive a virtual net- meet customer needs.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 57

neaSe: That’s looking at cloud com- “god box” in the middle.

puting in its infancy. How does some- neaSe: We are talking about taking a
one decide on HP versus Juniper? Two network function, decomposing it into
or three vendors will come in and say, constituent parts, and choosing a differ-
“I know the environment and I know Simon cRoSBy ent place to implement those parts. One
what you’re trying to run. Based on
what you need, here are the things I’d
if i have an of the parts is “figure out where to send
this,” and that part gets separated into
recommend.” investment in iT multiple parts depending on the specif-
Cloud vendors are not going to be-
come a monopoly overnight. It will
skill sets to manage ic attributes of your existing physical in-
frastructure. Owning your own assets is
evolve over some meaningful period of Juniper equipment still going to make sense for some time
time, and eventually a few major win-
ners will emerge, depending on area. in my private to come because the complexity of your
existing data-center infrastructure will
caSaDo: We all agree that if you take
a slider bar and move it fully to the right
data center, can restrict what you can actually hire some-
one else to do for you as a service.
to “future,” you’re going to have some i use those skills, TaVakoLi: There are solutions to-
massive consolidation, with a few large
vendors left standing. The question is
configurations, and day that will work in a garden-variety
5,000-square-foot data center. They
how long will that slider bar take to get policies off-premise take a general approach and are not as
to the end result?
cRoSBy: I talk to CIOs who are al-
in the cloud? concerned about things such as end-
to-end latency. That’s one approach.
ready telling their employees there You can also take a more specialized
will be no new net servers, and any approach and address customers that
new server purchases will require their have very specific needs such as latency
sign-off. This has motivated operations sensitivity.
teams to find ways to rent server cycles ReDDy: There are two management
by the hour. perspectives in addressing this issue.
A key opportunity arising from vir- One is at the infrastructure level: the
tualization and the cloud is to enable service provider who cares about the
CIOs to address the labor challenges of networks. The other is about the ser-
today’s owned infrastructure. CIOs will vices received over the wire. They don’t
absolutely take advantage of every op- care about the network; they care about
portunity to outsource new workloads service availability and response time.
to hardware they do not have to pur- From the service virtualization perspec-
chase and is automatically provisioned tive, I need to see everything in a holis-
and managed without expensive labor tic way: network, storage, computing,
costs, provided that key enterprise re- service availability, and response time.
quirements—SLAs and security and caSaDo: As you consume the net-
regulatory compliance—can be met. work into the virtualization layer, you
TaVakoLi: One of the things that vir- lose visibility and control of those com-
tualization buys you is homogeneity. ponents. We are just figuring out how
When you run on top of a hypervisor, to get it back. Networks are being con-
you don’t really care what the drivers sumed into the host and they’re losing
are; you are relying on the system. We control. We have a set of practices and
have to get to the same degree of homo- tools that we use to monitor things,
geneity on the network side. The ques- provide security, and do trending, and
tion is both economic and technical: that information is now more acces-
Who is best positioned to solve the mas- sible to the guy who runs the host than
sive network management problem? to the guy who runs the network.
You could take virtual switches and, cRoSBy: Let’s make it real. In a me-
as Arista has done, stitch them into dium-size enterprise I have a LAN seg-
your existing environment, while leav- ment called LAN A with an IDS (intru-
ing the virtual switch in place. You can sion detection system), and a different
take the Cisco approach of replacing LAN B segment with no IDS. If I have an
that virtual switch with your own virtu- application running in a VM and move
al switch and pull everything back to an it from a server residing on LAN A to a
aggregation point. At Juniper, we want server on LAN B, then I have a problem.
to build what is in effect a stateless, neaSe: No, you move it only to a seg-
high-capacity, 100,000-port switch but ment that supports the physical service.
without backhauling everything to the That’s how networks are rendered.

58 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


cRoSBy: The key point is that you caSaDo: You already have the model hop switch and how they participate
don’t have the luxury of being asked of slicing, so you already have virtu- in the value chain. Cisco, for example,
when a VM moves; you are told. The ar- alization; thus, nothing changes in via its Nexus 1000V virtual switch, is
gument that Lin (Nease) makes is that complexity. You have the exact same already staking a claim at the edge and
we would never move a thing to a LAN complexity model, the exact same protecting its customers’ investments
segment that is not protected. People management model. in skill sets and management tools.
usually don’t understand the infrastruc- neaSe: No, if I can get problems from BeeLeR: If I manage the network
ture at that level of detail. When the IT more than one place, something has within an enterprise and I’m told we
guy sees a load not being adequately ser- changed. Think of virtual switching as just virtualized all our servers and are
viced and sees spare capacity, the ser- a distributed policy enforcement point. going to be moving VMs around the
vice gets moved so the load is adequate- It is not true, however, that distributed network to the best host platforms,
ly resourced. End of story: it will move to stuff is equal in cost to centralized then as network manager, since I do
the edge. You are not asked if the move stuff. If distributed stuff involves more not have a virtualized network, this
is OK, you are told about it after it hap- than one way that a problem could oc- causes me problems. How do I address
pens. The challenge is to incorporate cur, then it will cost more. that? How do I take the IDS that I have
the constraints that Lin mentions in the caSaDo: It would have to be distrib- on my network today, not of the future,
automation logic that relates to how/ uted on the box. If you’re going to in- and address this problem?
when/where workloads may execute. ject it to one or more logical topologies, caSaDo: You either take advantage
This in turn requires substantial man- then you will have the same amount of of the dynamics of the cloud, which
agement change in IT processes. complexity. You’ve got logically isolat- means you can move it and you do scale
TaVakoLi: You can have a proxy at the ed components, which are in different out, or you don’t. In this case you can’t
edge that speaks to all of the function- default domains. do these VM moves without breaking
ality available in that segment. If people want the dynamics and the IDS. The technology simply dic-
cRoSBy: The last-hop switch is right cost structure of the cloud, they should tates whether you can have a dynamic
there on the server, and that’s the best either not invest in anything now and infrastructure or not.
place to have all of those functions. wait a little while; or invest in a scale- ReDDy: My server utilization is less
Moving current in-network functions to out commodity and make it look than 10%. That number is not just CPU
the edge (such as, onto the server) gives like Amazon. If they do not take one utilization—memory and I/O band-
us a way to ensure that services are pres- of these two paths, then they will be width are also limited because there
ent on all servers, and when a VM exe- locked into a vertically integrated stack are only two network cards on the
cutes on a particular server, its specific and the world will pass them by. each server box. All my applications
policies can be enforced on that server. cRoSBy: The mandate to IT is to vir- are very network-bandwidth intensive
caSaDo: This conversation gets tualize. It’s the only way you get back and saturate the NICs (network inter-
much clearer if we all realize there are the value inherent in Moore’s Law. face cards). Moreover, we also make a
two networks here: a physical network You’re buying a server that has incred- lot of I/O calls to disk to cache content.
and one or more logical networks. ible capacity—about 120 VMs per serv- Though I have eight cores on a box, I
These networks are decoupled. If you er—that includes a hypervisor-based can use only one, and that leaves seven
do service interposition, then you virtual switch. You typically have more cores unutilized.
have to do it at the logical level. Other- than one server, and that virtual switch neaSe: It seems like you would ben-
wise you are servicing the wrong net- is the last-hop point that touches your efit from an affinity architecture where
work. Where you can interpose into packets. The virtual switch allows sys- the communicating peers were in the
a logical network, at some point in tems administrators to be in charge same socket, but that sometimes re-
the future these services will become of an environment that can move quires gutting the existing architecture
distributed. When they become a ser- workloads on the fly from A to B and to pull off.
vice, they’re part of a logical network requires the network to ensure that TaVakoLi: From our perspective,
and they can be logically sized, parti- packets show up where they can be you want a switch without those affin-
tioned, and so on. consumed by the right VMs. ity characteristics so you don’t have to
Today, services are tethered to a neaSe: The people who will be left worry about “same rack, same row” to
physical box because of the sales cycle, out in the cold are the folks in IT who achieve latency targets. You really want
because someone comes in and says, have built their careers tuning switch- a huge switch with latency characteris-
“I’ve got a very expensive box and I need es. As the edge moves into the server tics independent of row and rack.
to have high margins to exist.” As soon where enforcement is significantly ReDDy: It is more of a scheduling is-
as you decouple these things, you have improved, there will be new interfaces sue. It’s about getting all the data into
to put them into the logical topology that we’ve not yet seen. It will not be a the right place and making best use
or they don’t work. Once you do that, world of discover, learn, and snoop; it of the available resources. We need to
you’re untethered. will be a world of know and cause. schedule a variety of resources: net-
neaSe: But once you distribute cRoSBy: The challenge for network- work, storage, computational, and
them, you have to make sure that you ing vendors is to define their point memory. No algorithms exist to opti-
haven’t created 25 things to manage of presence at the edge. They need to mally schedule these media to maxi-
instead of one. show what they are doing on that last- mize utilization.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 59

TaVakoLi: You want an extremely ho- tion to the problem of large data cen- want to be able to dictate how packets
listic view. You are asking where you ters and virtualization in the cloud. If move. To implement this, you will need
put a VM based on the current runtime routing intelligence needs to move up to control your flow table.
context of the hypervisor so you can the stack to layer-7, then by definition neaSe: The issue here is that a net-
maximize utilization and minimize you’re going to disenfranchise layers work is a single shared system, and it
contention for all aspects of data-cen- 2 and 3 from a bunch of policy deci- won’t work if an individual constituent
ter operations, such as CPU, network, sions. As you move up the stack it be- tries to program it. It has to be the cen-
storage, and others. comes more of a general-purpose kind ter that is programmed. Effectively, it
neaSe: You have to understand that of application, with general-purpose comes down to replacing the vendor’s
the arrival of demand is a critical com- processors being better suited for that view of the protocols of importance.
ponent to achieving this optimization, type of work. cRoSBy: Hang on. You intend to sell a
and it is not under your control. If a decision point requires that you switch to cloud vendors? If that is true,
ReDDy: At Yahoo! we built a traffic pick something specific out of an XML every single tenant has a reasonable ex-
server proxy that is open sourced and schema or a REST (representational pectation that they can program their
has knowledge and intelligence re- state transfer) body, then the intelli- own networks—to implement policies
garding incoming traffic from the net- gence needs to be there. The distribu- to make their applications work prop-
work edge. The proxy characterizes and tion of enforcement needs to be closer erly and to protect them.
shapes incoming traffic, and routes it to the edge for it to scale. neaSe: No, it’s the service provider
appropriately. Where precisely that edge is, wheth- that programs the network on behalf
caSaDo: This approach works best er it’s on the last-hop physical switch, of the tenants.
when linked with pretty dumb com- the NIC, or the vSwitch, is almost beside cRoSBy: If I’m a tenant of a virtual
modity switches, high fan-out, and the point. With something like VEPA private data center, I have a reasonable
multipath for load balancing. Then (Virtual Ethernet Port Aggregator), you right to inspect every single packet that
they build the intelligence at the edge. could aggregate that up one hop and crosses my network. Indeed, I might
This is the state of the art. it would not significantly change the have a regulatory obligation to do pre-
It does not matter where the edge is argument. The issue is about what you cisely that or to run compliance checks
in this case. Whether you enforce it at can ascertain from layers 2 and 3 versus or network fuzzing to check that my
the vSwitch, the NIC, or the first-hop what you need to ascertain from a much systems are secure.
switch, the only thing that matters is higher context at the application level. caSaDo: This gets to become a red
whose toes you step on when you exer- ReDDy: This was the best we could do herring. People who are building effi-
cise control. The definition for the edge given our current level of virtualization cient data centers today are overlaying
is the last piece of network intelligence. infrastructure. on top of existing networks, because
How that translates to a physical de- How do I get from where I am to take they can’t program them. That is just
vice—an x86, a NIC—depends on how full advantage of the current state of a fact.
you want to set up your architecture. virtualization? I want to move my distri- TaVakoLi: We have done an imple-
ReDDy: When a Yahoo! user sends a bution function from where it currently mentation of Open Flow on our MX
request from Jordan, a domain address resides at the edge of the network to its routers that allows for precisely what
maps to his IP address. Once he lands core. I want to build a huge switch fab- you’re talking about. Though not a sup-
on the server, DNS (Domain Name Sys- ric on the hypervisor so I can localize the ported product, it does provide a proof
tem) is used to determine that he is routing process within that hypervisor. of concept of an SDK (software devel-
coming from Jordan. I can then map If you look at the enterprise ap- opment kit) approach to programming
this traffic to the edge server located plications, there is a relationship be- networks.
in the data center serving the Middle tween multiple tiers. We built a de- neaSe: There’s a contention over
East. That’s your entry point and that is ployment language that characterizes who’s providing the network edge in-
your edge router. the relationship between each tier: side the server. It’s clearly going inside
The traffic then goes to the Apache the application tier, which is latency the server and is forever gone from a
proxy, which is a layer-7 router that sensitive; the application-to-database dedicated network device. A server-
we built. It determines that since the tier, which is throughput sensitive; based architecture will eventually
user is coming from Jordan, we should and the database-storage tier, which emerge providing network-manage-
route service to our data center in Sin- is again throughput sensitive. We then ment edge control that will have an
gapore, or in Switzerland. The traffic internally built a modeling architec- API for edge functionality, as well as an
never comes to the U.S. ture to characterize this information enforcement point. The only question
Depending on how I want to do traf- so that it can be used effectively dur- in my mind is what will shake out with
fic shaping, this architecture allows me ing operations. NICs, I/O virtualization, virtual bridges,
to change my routing policies dynami- caSaDo: Where you enforce is a com- among others. Soft switches are here to
cally and route traffic to the U.K., Swit- plete red herring. What this says to stay, and I believe the whole NIC thing
zerland, or Taiwan as needed. I can do me is that because Surendra’s (Reddy) is going to be an option in which only a
all this through layer-7 routing (Apache data-path problems are being driven few will partake. The services provided
proxy layer). by applications, he really has to open by software are what are of value here,
TaVakoLi: This is a different solu- them up. To solve this problem, you and Moore’s Law has cheapened CPU

60 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8


cycles enough to make it worthwhile to that is the only way you can get these
burn switching cycles inside the server. types of flexibility and per-port costs.
If I’m a network guy in IT, I better It would be interesting to compare
learn the concept of port groups, how a vertically integrated enterprise with
VMware, Xen, and others work much mache cReeGeR something like Amazon EC2 (Elastic
more intensely, and then figure out
how to get control of the password and
mass interest in Compute Cloud) in terms of cost per
port and efficiency.
get on the edge. Those folks now have virtualizing the data BeeLeR: The guys who run infrastruc-
options that they have never had before.
The guys managing the servers are
center is breaking ture for Google told us that the differ-
ence between running their own infra-
not qualified to lead on this because a lot of traditional structure and running their stuff on
they don’t understand the concept of
a single shared network. They think in physical versus Amazon was small enough that it really
made them think about whether they
terms of bandwidth and VPLS (virtual logical bounds. you wanted to continue to do it themselves.
private LAN service) instead of think-
ing about the network as one system need to concentrate caSaDo: We have seen close to two
orders of magnitude difference be-
that everybody shares and is way over-
on what you’re tween a vertically integrated solution
and something like EC2.
ReDDy: We are moving to Xen and trying to achieve BeeLeR: The relevance here is that
building a new data center architec-
ture with flat networks. We tried to use
in your data center while these issues may not affect you
today as a practitioner, you should un-
VLANs, but we have taken a different and what key derstand them because they will affect
approach and are going to a flat layer 2
network. On top of this we are building properties you’re you tomorrow. In this way you can make
intelligent investments that will not
an open vSwitch model placing every- trying to preserve. preclude you from taking advantage of
thing in the fabric on the server. these kinds of benefits in the future.
My problem is in responding to caSaDo: The leverage point for verti-
the service requirements of my appli- cal integration has always come from
cations and addressing things such the networking vendors. It was lost a
as latency and throughput. The data long time ago on the servers. Someone
needed to address these issues is not who offers a full solution is going to be
available from either a network virtual- a networking vendor. If you’re making
ization solution or the hypervisor. a purchasing decision, then you don’t
Also, my uplink from the switches is have to blindly follow the legacy archi-
10 gigabits per second or multiple 10 tectures.
gigabits per second, but my NICs are I do not believe that owners of ex-
only one gig. If I run 10 VMs on a box, isting network infrastructure need to
then all of the bandwidth aggregates worry about the hardware they already
on one or two NICs. have in place. Chances are your exist-
neaSe: You guys are cheap. If you ing network infrastructure provides
went to a backplane, then you would adequate bandwidth. Longer term,
get a lot more bandwidth out of those networking functions are being pulled
servers. A KR signal on a backplane is into software, and you can probably
how you get a cheap copper trace for keep your infrastructure. The reason
10-gigabit service. you buy hardware the next time will be
caSaDo: Going forward, a new net- because you need more bandwidth or
work layer is opening up so you can less latency. It will not be because you
take advantage of virtualization. Tradi- need some virtualization function.
tional networking vendors certainly do TaVakoLi: We get caught up on
not control it today and may not con- whether one is implementing a new
trol it in the future. The implications data center from scratch or is incre-
are that it may not matter what net- mentally adding to an existing one.
working hardware you purchase, but it For a new data center, there are sev-
may matter much more what network eral things to keep in mind. Number
virtualization software you choose. one, if you’re planning for the next five
If you like the cost point and the ser- years, understand how you are going
vice and operations model of the cloud, to avoid focusing on “rack versus row
then look at Eucalyptus, Amazon, Rack- versus data center.” Architect the data
space, and so forth, and see how they center to minimize location-depen-
build out their infrastructure. Today dent constraints but still be able to

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 61

take advantage of location opportuni- of new, more efficient and functional

ties as they arise. architectures as they gain broader ac-
Also, have a strategy for how you can ceptance. The advice here suggests that
still obtain a top-level view from all the you keep things simple. Avoid invest-
independent edge-based instances. maRTin caSaDo ing in vendor-proprietary functions,
This is especially critical in areas such
as security, where you need a global view
if people want the and wait to see what new architectures
emerge. Once you identify these new
of a multiple-point attack. If there’s an dynamics and cost architectures, invest conservatively as
attack that consists of multiple events
that are all below individual thresh-
structure of the they gain acceptance.
cRoSBy: The key point is to be aware
olds, then there’s still some correlation cloud, they should that your networking competence in-
required up top to be able to recognize
it as an attack. You cannot get away with either not invest in vestments are going to shift radically.
The new network will be automated,
saying that these are distributed, inde- anything now; or aware of the current locus of the work-

invest in a scale-
pendent problems at the edge and that load, and dynamically reconfigure the
no correlation is required. infrastructure as the workload mi-
neaSe: You will never remove the
concept of location from networking. It
out commodity grates or scales elastically.
TaVakoLi: An opportunity exists to
will always be part and parcel of the val- and make it look implement very fine-grained, high-
ue proposition. Bandwidth consump-
tion will be rarer the farther you span,
like amazon. if quality enforcement at the very edge
of the network, including on the host.
and latency will be shorter the closer they do not take That will have to be stitched into your
you are. Physical location of network-
ing resources never completely goes one of these two service model. You can scale and dis-
tribute your control to the very edge
away. The network is never location paths, then they of the network, which now is on the
independent and always has a compo-
nent of geography, location, and phys- will be locked into a hosts. The question is, who ends up
driving the overall policy decision?
ics. You cannot separate them.
cReeGeR: The issues involved in net-
vertically integrated neaSe: If you’re a network person
and you’re not touching VMware and
work virtualization are moving quickly. stack and the world find yourself not needed, you have to
Mass interest in virtualizing the data
center is breaking a lot of traditional
will pass them by. ask yourself whether or not your skill
set is not needed as well. The network
physical versus logical bounds. You edge has moved, and if you are not ar-
need to concentrate on what you’re try- chitecting the network inside the serv-
ing to achieve in your data center and er, then your skill set may not matter.
what key properties you’re trying to BeeLeR: The good news is that some
preserve. If you do decide to virtualize, systems administrators don’t have a
do an internal cloud, or subcontract clue about networking. This is an oppor-
out to an external cloud vendor, then tunity for network engineers still to add
you need to parallel your architecture value in the new virtualized world.
closely to industry leaders such as Am-
azon so you keep close to current ac-
cepted practices. Additionally, to avoid Related articles
breakage between physical and virtual on
devices, you want to minimize func- network Virtualization:
tionality and performance investments Breaking the Performance Barrier
that require device-specific configura- Scott Rixner
tion of the physical infrastructure. As
virtual devices become more prevalent, CTO Roundtable: Virtualization
Mache Creeger
those device-specific configurations
will become more of a burden.
Meet the Virts
Some network vendors are offer-
Tom Killalea
ing products under the banner of net-
work virtualization that provide virtual
implementations of their physical de- Mache Creeger ( is a technology
vices. I believe they are being offered industry veteran based in silicon Valley. along with
being a columnist for ACM Queue, he is the principal
to preserve a customer’s investments of emergent technology associates, marketing and
in legacy infrastructure. By preserving business development consultants to technology
companies worldwide.
the status quo, however, it will be that
much more difficult to take advantage © 2010 aCM 0001-0782/10/0800 $10.00

62 com municaTio nS o f T h e acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8

Doi:10.1145/ 1787234 . 1 78 72 5 2

Everything you always wanted to know but

were afraid to ask about the decision-making
By Jan DamSGaaRD anD Jan kaRLSBJeRG

for Selecting
ARound the mId-1950s, in the early years of commercial
use of computers, all software systems were developed
in-house. There was no software industry in existence
at that time.15 As the software industry formed over
the next few decades, many organizations outsourced
their software development to specialized software

suppliers. Most software products ware a way to capture and black-box

were, however, still developed as best practices by embedding it into
unique systems for each organization; the standardized components of the
that is, there was little standardization. systems.16 Next in the standardization
The next step occurred when that soft- process was a step away from propri-
ware producers developed their own etary standard systems that essentially
proprietary software in order to cap- locked customers to a single software
ture economies of scale in developing producer to open software standards.37
the software once and then selling it to In principle, software built on open
multiple customers.2 This standardiza- standards allowed customers to source
tion process also benefitted software from any supplier that could supply
buyers by lowering transaction costs software in accordance with open stan-
and risks, as it was now possible to dards (for example, Java- and XML-
choose among a proven set of applica- based systems).
tions. Moreover, standardization gave Open standards meant that prices
both producers and buyers of soft- dropped and functionality was en-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 63

hanced, which resulted in a mass

market for many software application
types. In addition, software producers
had enough resources to make their
software even more general-purpose
oriented with larger feature sets that
were organized into a product.8 Soft-
ware became even more standardized,
and in the process, many local markets
were annexed into global markets. For
example, word processing software
was no longer produced specifically
for a particular profession or industry
or nation;26 instead, an almost uni-
versal office suite emerged, such as,
Microsoft Office. The generalized soft-
ware products could be configured in
various ways (for example, program
parameters, macro functionality, lan-
guage support, and so on) to suit spe-
cial needs among customers. These
highly configurable general purpose
software products came to be known
as software packages.23
Until recently in the IS academic
community, there has been a ten-
dency to focus on traditional studies
of software development and imple-
mentation of large custom-made sys-
tems.20,24 This has been despite the
leading trend that organizations use
“shrink wrapped” systems31 where the
core functionalities of the software are implementations are essentially iden- tions means the choice of software
identical across all implementations tical; that is, the main functionalities package has wide ripple effects for
in dozens, thousands or even millions are common to all adopters. While other parts of the organization whose
of different organizations.15 When the core components of a package software packages, implementations,
it comes to managing the process of are identical across all user organiza- and interests may not originally have
identifying and evaluating packages, tions, the implementation into an in- been identified or considered in the
the IS academic community has been dividual organizational information decision process regarding a new soft-
almost silent. infrastructure is usually configured in ware acquisition.
The aim of this article is to provide some manner to fit the requirements Packages are often referred to as
practitioners with a grounded set of of the organization.1,17,22 For the pur- “commercial off-the-shelf” software,31
principles to guide the selection of pose of this article, we define a stan- but open source systems (for example,
software packages. By principles, we dard software package as: a collection Open Office) or other types of nominal-
mean a set of fundamental ideas that of software components which when ly free software, for example, Firefox21
summarize important insights in soft- combined perform a set of generalized or Internet residing systems (for exam-
ware package acquisition that are not tasks that are applicable to a range ple, Google Apps) are other examples
(yet) embedded into the practice of of users. As a package is adopted by of packaged software. Some standard
buying software. The principles are in- many, it forms a standard because the software packages require little adjust-
terdependent and together they form a core components are identical across ment on the part of the user before they
whole that is larger than the sum of the all of its installations. The software can perform (for example, Internet Ex-
parts. Similar to Klein and Myers’ argu- package may be configured or custom- plorer), while other software packages
ment,19 the use of all principles is not ized to make it fit with specific require- are mere tools or platforms on top of
mandatory, but in each case it must be ments unique to the concrete imple- which specific functionalities required
judged whether, how, and which prin- mentation. This is accomplished by by the user can be implemented (for
ciples apply to a specific situation. setting program parameters, install- example, ERP systems).8 Some setups
ing add-on modules, or building in- of parameters may be common among
Packaged Software terfaces with other software systems. several customers, in which case the
Packaged software is a category of Within an organization, the growing producer can offer standard solutions
information systems for which all importance of system interconnec- on top of which only site-specific con-

64 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


figurations need to be made.35 For ex- field study was conducted using semi- their knowledge and experience with
ample, the ERP producer SAP provides structured interviews. The persons corporate intranets. During the three
more than 25 industry solution portfo- interviewed were five senior directors years the nature of intranets changed
lios for large enterprises that embed with knowledge of—and some power from being home grown, to a situation
best practice (for example, SAP for oil to influence—software acquisition. To where a few local software companies
and gas). broaden our knowledge base, we also vigorously fought over market share,
carried out 34 interviews in 13 other to a situation where intranets were
Seven Guiding Principles for organizations. The interviewees were built upon international standards
Selecting Software Packages CIOs, CFOs, and general managers, and readily available from multiple
Here, we present the guiding prin- and were deliberately chosen because software houses. The seven principles
ciples for making a better informed of their high experience with software have now been presented and critically
choice when selecting software pack- package selection processes. All inter- reviewed at numerous IT managers’
ages. The first principle we label the views were recorded and transcribed conferences, and we are indebted to
founding principle because it is fun- and thematically coded. The longitu- the participants for many of the exam-
damental to the other six. For each dinal approach meant that a theme ples that illustrate the principles.
principle we provide examples that il- identified in one interview could be
lustrate its importance. further investigated and validated in The fundamental first Principle:
The seven principles were derived subsequent interviews. When you buy packaged software
empirically from a field study and A second source of inspiration was you join its network.
from our understanding of software information about particular software Prior to the emergence of packaged
acquisition. The field study approach standards and packages, vendors, software, any organization that was us-
provided us with in-depth knowledge historic data about system compat- ing software in effect committed itself
IllustratIon by JaM es Ga ry

of a number of standards decisions ibility, market shares, and mergers not only to a software product but also
made by actual organizations. The fo- and acquisitions. Yet another source to a particular software producer’s con-
cal company had more than 50,000 of inspiration came from participat- tinued ability to deliver new function-
employees, and we followed its soft- ing in an industry network in the late alities, as organizational requirements
ware acquisition processes and stan- 1990s where representatives from 60 evolved and new technology became
dard choices over three years. The companies met bimonthly to share available. In the present day, most of

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 65

these commitments and dependen- ful users has repeatedly postponed the
cies have evolved from local software sunset date of Windows XP.
producers to global standard software Open source packages, on the con-
packages that can be sourced from, trary, are not owned by a single en-
and configured by, many independent
software vendors with the necessary The users tity; instead, the software is designed
specifically to promote shared own-
competences and technical skills.
The users and producers of a soft-
and producers ership.25,29 Open source software can
appear unattractive and risky to some
ware package constitute a network of of a software because there is no central point of
parties that share a common interest in
its destiny.34 The network is virtual, in
package constitute control from which advice about the
software package and its future devel-
the sense that the members probably a network of opment can be sought. Others view
do not know each other but neverthe-
less share a common interest in pro-
parties that share these properties as strengths since
they protect the standard package
tecting their investments and ensuring a common interest from the opportunistic actions of prof-
the continued evolution of the pack-
age. The network indirectly also has in its destiny. it maximizing software producers. We
shall not conclude the heated debate
other interests in common; for exam- over open source here, but merely em-
ple, the training and education of per- phasise that organizations adopting
sonnel.34 An organization’s purchase a software package need to be alerted
and implementation of a particular to the intimate connection between
software package thus means that the a software package and its associated
organization has joined the network network.
associated with the software package,
and the level of commitment is equal Principle Two:
to the size of the investment (buying Take a long-term perspective:
and configuring the software and the Look ahead but reason back.
training of personnel, and so on). To a Many choices made in the early stages
large extent, the investment represents of an organization’s use of computers
sunk costs,10 which make risk mitiga- have turned out to have surprisingly
tion activities even more central. long-lasting consequences, as both
The network around the package software and data standards have been
has implications for the purchasing shown to be very persistent.20 Many
decision and has to be considered as application types have historically de-
part of the investment decision. Be- veloped in an evolutionary manner,
yond the immediate network of users where the first simple implementa-
and producers, the extended network tions were custom built by innovators,
includes vendors, standard setting and then spread to a small number
institutions, government authori- of early adopters. As the application
ties and other compatible software type benefited its adopters, competing
products. It is imperative to choose to systems became available on the mar-
participate in the network that is per- ket, and finally the application type
ceived to provide the best long-term became a commodity, possibly to be
benefits as the organization, the net- bundled with other software applica-
work and package co-evolve. In the tion types into larger software pack-
network the distribution of power and ages. A similar evolution trajectory will
influence depends chiefly on who con- likely describe the development of fu-
trols the package and thereby its evolu- ture application types that first appear
tion. In the case of most software pack- as isolated systems. As a consequence,
ages, the producer wields the greatest organizations must take a long-term
power over the proprietary software perspective and envision a more com-
network, as they own the rights to the plex and connected future, or else they
package outright and thus control its risk implementing tomorrow’s legacy
further development. The producer’s systems.
power can be challenged if users unite We emphasize this long-term per-
to influence the producer or even chal- spective of software packages. As the
lenge the producer’s ownership; for pace of change in the computer in-
example, by reverse-engineering the dustry reduces the effective lifespan of
package’s functionality. As an example most hardware and software to a few
of influence, the pressure from power- years, the organizational data and the

66 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


standards that define them are more perpetually, giving organizations the usability, etc. Most often, however,
durable.5 An organization’s standard choice of staying with the incumbent compatibility is not a clear binary issue.
package choice therefore involves par- producer or giving them time to look As standards and packages evolve and
ticipation in networks that may last a for migration paths toward a standard producers compete against each other,
decade or often longer. Shapiro and Var- package with more perceived vitality.17 packages may converge or diverge on
ian34 argue that when buying standard The one-way street scenario describes some features, such as, reaching or
technology we should look ahead but the situation where the organization is breaking compatibility.33 Of course,
reason back, noticing the network and left with little choice when it comes to this development can be caused by le-
the evolution process that produced it. buying upgrades or expansions to the gitimate technical design and imple-
We applaud and echo this advice that package. This is the case when the pur- mentation decisions, but it may also
is valid also when selecting packaged chase of a particular package in effect be caused by the producer’s perceived
software. This principle is useful to in- obliges the organization to place future advantage in changing the degree of
clude when comparing a proprietary purchases with the same software fam- compatibility or interoperability with
software package from a local vendor ily because the product has low compat- competing packages.
with that of a software package built ibility with other families of software A producer may differentiate its
upon an open global standard. or packages. In this situation, the orga- package from the competition by add-
nization may find itself chained to the ing proprietary features and unwar-
Principle Three: producer because the costs involved in ranted proprietary extensions to an
When choosing packaged software, switching to another package are pro- open standard. There are some calls for
there is safety in numbers. hibitively high, and the organization is the execution of this predatory business
One route to mitigating the perceived in effect locked-in.11 This is quite com- technique of “embrace, extend, and ex-
risk in purchasing packaged software is mon for ERP systems where once the tinguish,” and often Microsoft is associ-
to choose a package based on its histori- initial choice between (for example, ated with an almost flawless execution
cal and current success, as measured Oracle Financial Systems and SAP), has of the technique. Only the law suits that
by the financial success of the software been made, it becomes prohibitively doubtlessly follow spoil the perfection.
package’s producer and the size of the expensive to switch. Sometimes, a pack- One historical example is the fight be-
associated network. Flocking behavior age may be so successful in the market tween Sun and Microsoft over Java and
is a low risk strategy that is worth pur- that there are few—if any—viable alter- extensions to Java.32 The practice of add-
suing for software support of non-core native products available to the organi- ing proprietary extensions to an (open)
functionality and for companies that zation, an example of which is the cur- standard is successful when some
consider themselves followers. Below, rent choice of operating systems for PCs adopters find the proprietary features
we describe two scenarios representing being limited to Microsoft Windows, attractive and implement them. Howev-
opposite outcomes of a competition be- creating a near monopoly. However mo- er, it is important to be aware that pro-
tween software packages; namely, blind nopolies are constantly challenged and prietary features that might be useful
alleys and one-way streets.12 they are often short lived in the software for the singular adopter are in fact false
The blind alley scenario refers to the business, as reported by Chapman7 who gold for the network at large. Every time
situation where an organization has narrates the story of how WordPerfect a proprietary feature is implemented it
adopted a package that is losing its lost its near monopoly and how other adds to the switching costs, meaning
market share to competing packages. software packages such as Netscape that it will be harder to pull away from
David12 uses the term “angry orphan” and dBase lost their lucrative position the software package that embeds the
to describe the situation of the los- in the market. proprietary extensions.34 For the net-
ing package. He points out that such work, it means that proprietary fea-
products often show a sudden rapid Principle four: tures become entrenched as de facto
development when they are losing the focus on compatibility standards, and for the community in
battle. For example, the greatest speed and beware of false gold.a general, it becomes an insurmountable
of innovation in sail ships happened as Because of the long life expectancy of barrier to change, thus diminishing the
the steam engine challenged the sail as organizational data stored in some (of- value of a standard.
the leading propelling technology on ten proprietary) format (see Principle The break-down of open standards
sea voyages. Despite the sudden and Two), backward compatibility between happens in many cases where there is
remarkable development, sail boats software systems becomes a major fac- no central governance of a standard
never really challenged the inevitable tor when organizations consider new by a central institution or authority,
change to steam engine boats. In a software investments. Sometimes soft- and even if such governance does ex-
similar manner, the losing software ware adheres to one common standard, ist, standards often break down anyway
package might undergo rapid develop- enabling user organizations to choose as competitors extend the limits of the
ments, but shrinking network effects among competing packages based on standard.9 One example that we claim to
make the downward spiral inevitable. features such as price, performance, be false gold comes from the company
In a special case of the blind alley sce- Linksys (owned by Cisco) which has ex-
nario, the losing package manages to a By false gold or fool’s gold we mean something
tended its wireless network equipment
capture a niche market network where that appears attractive but in reality is not valu- with proprietary protocols, thus dou-
it may sustain itself for years—or even able at all. bling the throughput of the non-propri-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 67

etary protocol IEEE 802.11b. While the but for several reasons, could turn out Producers employ various strat-
products are still backward compat- to be false gold.20,23,35 First of all, the egies for ensuring a pool of knowl-
ible with the open standard backed by customization is often expensive and edgeable users for their software.11
the IEEE, Linksys gives users a strong represents sunk costs that, in practice, One strategy is to produce free or
incentive to use Linksys hardware ex- limit the choices when the package or low cost versions so that interested
clusively. Another large manufacturer, service contract is up for renewal.5,20 people will be more likely to sample
D-Link, does exactly the same thing; Second, when upgrading the software it. Another variation is to make “aca-
however, the proprietary extensions to the next version, usually all custom- demic versions” of the software pack-
of D-Link and Linksys are not compat- izations have to be re-implemented. In age available as free downloads, or to
ible. For the community, the danger addition, the new features of the next bundle the package with textbooks
of proprietary extensions is that it may version are obviously not part of the cus- used in educational institutions. The
not be compatible with the next genera- tomization that was implemented from process of institutionalizing skills is
tion of the open standards (in this case the previous version.8 Beatty and Wil- more complex for packages based on
IEEE 802.11n), and if the proprietary liams5 recommend “un-customizing open source (sendmail, emacs, Linux,
extensions have become entrenched, customizations” before any upgrade is among others), where there may be no
none is willing to adopt the next open attempted because they are found to single trusted certifying institution
standard version. Thus, the network form major technical obstacles and are corresponding to the owner or vendor
has moved from a situation where or- the main threat to achieving a Return of a package. Instead, other forms of
ganizations could choose to buy open on Investment. Instead, Beatty and Wil- legitimization are used, such as a per-
standard compatible equipment from liams5 propose that an upgrade is an son’s rank in recommender-systems
a number of independent suppliers to a opportunity to review critically existing such as discussion Web sites. Such on-
situation where standard evolution has customizations in order to determine line networks also make it possible to
stopped and there is only one supplier whether they are really needed, and if determine the contributions of a par-
of a proprietary de facto standard. In so, to determine if they are supported ticular member, enabling potential
fairness, it should be noted that neither in the new version and eligible for elim- employers to retrieve an account of a
D-Link nor Linksys has been successful ination. In line with this, we advocate person’s skills in regard to a particular
in their effort to manifest their propri- avoiding any comprehensive custom- software package.
etary extensions as de facto standards; ization of packaged software, unless The co-development of the two
however, the risk remains. absolutely necessary. networks (that of the producers and
Organizations should keep their that of the users) has high path de-
options open by buying packaged Principle five: pendence to the point of being quasi
software that is close to compatible choose a software package irreversible.11 For a new competing
standards; and if they are already us- with accessible knowledge. software package that starts with es-
ing proprietary standard packages, When an organization chooses to use sentially no network; the existing net-
they should keep their eyes open for custom-built software, it must carry work forms a formidable entry barrier
gateway standards as a way to break an the entire burden of training and re- that is difficult to break.6 If the new
existing lock-in to a proprietary exten- taining personnel to develop the nec- package is proprietary and the own-
sion.13 At the very least, organizations essary skills to use the software. The ers are willing to invest, one way for
should be conscious of the adoption use of packages, however, promises the new standard package to achieve
of proprietary extensions, document access to knowledge of the package’s a critical mass of users is for the own-
their use in the organization, and con- application and implementation. Ide- er to bear some or all of the costs for
sider which steps will be necessary to ally, the network of organizations us- the organizations willing to switch.33
discontinue their use in the future; that ing a package is matched by a network An alternative approach is to invest
is, a viable exit strategy. of individuals competent in configur- in building gateway features into the
Generic software packages do not ing and using it, but often the supply new standard package, thus easing
meet all the requirements of an organi- and demand of certain skills is not the transition from an incumbent
zation;8,28 there are therefore plenty of aligned, as is pointed out by Light.23 package.13 When Microsoft Word was
options offered as part of the package If there is an unmet demand for winning over the majority of the word
to configure it as needed.16,30 Often local knowledge and skills, both user and processing market from WordPerfect
practices or cultural issues add to the producer organizations suffer. One in the first half of the 1990s, Microsoft
desire to customize or localize the pack- historical example of misaligned net- sought to circumvent the knowledge
age.5,22 Customization is different from works is that of ERP systems, where barriers by providing WordPerfect us-
configuration in that customization the number of people with knowledge ers an easy passage. Microsoft Word
is more radical and adds functionality and skills of the configuration of SAP featured two gateways: an alterna-
that was not an intended generic feature systems is far less than the demand tive user interface where Microsoft
in the original package. Customization from user organizations. The result is Word could be made to emulate the
is more lucrative for local software ven- disproportionately high costs for the keyboard shortcuts of WordPerfect,
dors compared to selling the package it- people component of SAP implemen- and “Help for WordPerfect users”
self. For the adopting organization, the tations and delayed projects with re- where the use of Microsoft Word was
option to customize may appear shiny, duced or poor functionality. explained in terms that WordPerfect

68 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8


users were accustomed to. We suggest two reasons: seeking backward com-
using this principle to assess the avail- patibility with data stored in legacy
able knowledge base for the software systems, or seeking to ensure access
package. to the data from other information

Principle Six: By choosing systems in the future; that is, forward

compatibility. By choosing an open
choose packaged software with
the right type of standardization.
an open standard, standard, an organization can usually
choose between numerous compat-
Standardization can be achieved at an organization ible software packages, thus bringing
various levels and in many forms in
packaged software. Here, we provide
can usually choose the simple advantage of choice. The
disadvantage is that the user organiza-
an overview of the most common types between numerous tion must abstain from using any pro-
of standardization because it is impor-
tant to choose the type that is right for
compatible prietary features or extensions of the
packages chosen (the false gold men-
the particular organization, accord- software packages, tioned in Principle Four) in order to
ing to its available resources and con-
straints. thus bringing the maintain strict data standardization.
Examples of data standards with wide
Standardization of user interface is simple advantage vendor support are the all-purpose in-
a common strategy employed to limit
the need for user training. After some of choice. formation formatting languages XML
and the database query language SQL,
experimental implementations of in- although both are also subject to stan-
formation systems of a particular type, dard deviations among the implemen-
a dominant design typically emerges, tations from various producers.
resulting in striking similarities of More advanced modes of standard-
user interfaces among different soft- ization of data interfaces include inter-
ware systems. Referring to Web site de- connectivity and interoperability.4 In-
sign guru, Nielsen,27 users spend most teroperable information systems are
of their time on other sites, and there- able to communicate during the exe-
fore prefer new Web sites to be de- cution of a particular task. An everyday
signed similar to the sites with which example is that the functionality of an
they are familiar. Dominant designs electronic spreadsheet program can
sometimes become static and end up be employed by a word processing pro-
as anachronisms when the surround- gram to perform a calculation inside a
ings change. For example, the diskette text document. More advanced imple-
icon featured in most software appli- mentations allow interoperability be-
cations invokes the “save” function, tween software running on separate
even though no files are ever saved to computers - even in different locations
diskettes and personal computers no or organizations such as most Web
longer have disk drives. services organized in serviced oriented
In standardization of output, the architectures (SOA).14 Features such
software package’s only compatibil- as these will have far-reaching impli-
ity restraint is that it must produce an cations for the implementation of
output that can be used by recipient standard software packages and inter-
users or software. One example is that organizational information systems in
of Web page production, where dif- the coming years.
ferent departments in an organiza- Organizations may choose stan-
tion may use very different production dardization of skills by employing only
techniques as long as their Web pages people with a particular education or
satisfy agreed-upon requirements. skill set, or if necessary, to carry the
This standardization strategy has the cost of training new employees to some
strength of allowing users greater free- formalized level of training (see Prin-
dom to optimize and personalize their ciple Five). Organizations can choose
production methods. The strategy also to standardize two types of skills: ge-
has serious drawbacks if the users ever neric or specific skills. Generic skills
need to share intermediate data; we are skills that are acquired through
would thus not recommend this strat- education, such as critical thinking,
egy for most organizational standard- programming, business knowledge,
ization issues. and so on Specific skills encompass
An organization might choose stan- a user’s qualifications with a particu-
dardization of data structure for one of lar software package, and these may

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 69

be certified by the product’s producer adopt its predecessor today by joining

or a trusted third party (see Principle its network. Being part of the network
Five). Every major vendor in the pack- will also ensure that special needs are
aged software market has such certi- noted and incorporated into the next
fication programs, and many are even
updated on a continual basis, forcing We promote a version of the package.

certificate holders to take new exams

in order to preserve their status.
view of buying conclusion
Software packages are replacing cus-
One might argue that if all are using software as a tom built software at a fast pace. Yet,
the same standard software package,
where does competitive advantage
continuous process there is little available advice on how to
evaluate and choose among the offered
come from? As a rule of thumb, we rec- of constantly packages. This article highlights seven
ommend organizations to follow and
standardize in all non-core areas to
trying to match principles that are related to selecting
and assessing software packages. The
bring down costs, and in order to dif- available packages principles extend beyond the two ob-
ferentiate themselves, organizations
must be prepared to lead (be an early with a base of vious but narrow factors of price and
immediate features, to include a wider
adopter) and tolerate a higher degree already installed networked and multilateral view of soft-
of standard uncertainty in core areas.
We will return to the issue of competi- information ware packages. We promote a view of
buying software as a continuous pro-
tive advantage in the conclusion. systems, while cess of constantly trying to match avail-
able packages with a base of already
Principle Seven: anticipating future installed information systems, while an-
all journeys start with
a first step.
organizational ticipating future organizational needs
and advantages in technology. Compa-
In a market of fast update cycles and needs and nies should seek to select the package
many options, some buyers may as-
sume a wait-and-see position, while advantages in that fits their situation. However, this is
not a unilateral decision, as other com-
they let the rest of the market test out technology. panies’ actions also contribute to the
competing products, determine the destiny of the package. Software pack-
necessary feature sets, and so on.3,20 Of ages are networked and built around
course, this strategy will mitigate the standards that allow (and disallow) con-
risks of investing time and money in nection to other software systems and
a software package which later loses these considerations must be added to
in the market, but we advise organiza- the equation, too. It is therefore neces-
tions not to fall into the wait-and-see sary to adopt a multilateral approach
trap for the following two reasons. that asserts the benefits of participation
First, a winner will only emerge when from as many parties as possible in the
organizations actually buy software, selection process.
so an organization stands a greater The proposed principles are useful
chance of finding software that fits its in several ways. First, they form a refer-
needs if it plays an active role in the se- ence point for IT managers when en-
lection process (invest in the package). gaging in software acquisition. Second,
Second, the further development without the principles, IT managers
of packages is inevitable, and thus it would have to spend much time con-
is very likely that while an organiza- densing these foundations from avail-
tion is waiting for a package to appear able theoretical and empirical sources.
in the marketplace for a perfect fit, its Third, the principles help IT managers
requirements may have changed. In ensure that vital aspects of the soft-
fact, it may never be possible to find ware package acquisition process have
a perfect match.36 After a prolonged not been left out or neglected. Finally,
sampling process and the organization the set of principles is an invitation to
finally selecting a software package, formulate a disagreement and start a
activities such as conversion of legacy discussion on what constitutes sound
data may turn into considerable tasks, software acquisition practices.
as there may be no personnel with ex- Here is a checklist that IT managers
pertise in both the legacy system and can consider in addition to the usual
the new software package.20 There- technical features and price, when eval-
fore, the best strategy to ensure that a uating a software package purchase:
better package is there tomorrow is to ˲˲ What companies are involved in

70 communicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8


producing the package? but after more than a year of attempt- Policy 3 (1988), 165–202.
14. erl, t. Service-Oriented Architecture: Concepts,
˲˲ How many companies are already ing to implement the new ERP system Technology, and Design. Prentice Hall, upper saddle
using the package? the manufacturer had to revert to its old river, nJ, 2005.
15. George, J.F. (ed.) The Origins of Software: Acquiring
˲˲ How many software companies ERP system. The skill set and knowl- Systems at the End of the Century. Framing the
can configure the package? edge base built around the former ERP Domains of It Management: Projecting the Future
through the Past. Pinnaflex educational resources,
˲˲ What is the history of the package? system in practice inhibited a switch. Inc., Cincinnati, oH, 2000.
˲˲ Is the package built upon open Returning to the competitive ad- 16. Howcroft, d. and light, b. reflections on issues of
power in packaged software selection. Information
standards? vantage discussion initiated earlier Systems Journal 16, 3 (2006), 215–235.
˲˲ How is the fit with other packages? 17. Karlsbjerg, J. staying outside the mainstream:
and playing the devil’s advocate, one an empirical study of standards choices. Hawaii
˲˲ What kind of standardization does might argue that if everybody were International Conference on System Sciences, Hawaii,
the package represent? using the same software packages, 18. Karlsbjerg, J., damsgaard, J., and scheepers, r. a
˲˲ Is customization of the package where would competitive advantage taxonomy of Intranet Implementation strategies:
to make or to buy? Journal of Global Information
necessary? in the form of differentiation come Management 11, 3 (2003), 151–165.
˲˲ Is there an accessible knowledge from? Succinctly put as a paradox, “In 19. Klein, H.K. and Myers, M.d. evaluating interpretive field
studies. MISQ 23, 1, 67–94
base for the implementation and ex- the world of software packages, ad- 20. Khoo, H.M. and robey, d. deciding to upgrade packaged
ploitation of the package? vantage comes from having the same software: a comparative case study of motives,
contingencies and dependencies. European Journal of
˲˲ What are the costs of switching to packages as everybody else before Information Systems 16, 5 (2007), 555–567.
an alternative package? they do.” Thus, competitive advan- 21. Klein, b. 1998. Microsoft’s use of zero price bundling
to fight the browser wars. The Progress & Freedom
˲˲ What are the implications of post- tage is gained from being able to spot Foundation. Kluwer academic Publishers, Washington,
poning a decision to adopt? and adopt the packages of the future dC, 1998.
22. Kutar, M. and light, b. exploring cultural issues in the
In what direction is the package before they have become the de facto packaged software industry: a usability perspective.
evolving? And is our company headed standard packages, and to identify In Proceedings of the 13th European Conference on
Information Systems (regensberg, Germany, 2005).
the same way? and phase out the packages of the past 23. light, b. Potential pitfalls in packaged software
adoption. Commun. ACM 48, 5 (May 2005), 119–121.
The principles can be used prior to before they become legacy systems. 24. light, b. and sawyer, s. locating packaged software
making an investment and be used to in information systems research. European Journal of
Information Systems 16, 5 (2007), 527–530.
monitor the vitality of existing packag- this research was in part supported by the danish 25. ljungberg, J. open source movements as a model for
es. To illustrate, when a university built research Foundation, grant number 331958. organizing. European Journal of Information Systems.
(dec. 2000).
a new campus building it came with a 26. Moore, G.C. end user computing and office automation:
free proprietary facility management 1. adam, a. and light, b. selling packaged software: an
a diffusion of innovations perspective. Infor 25 (1987),
system with the new building already ethical analysis. In Proceedings of the 12th European 27. nielsen, J. user interface directions for the Web.
Conference on Information Systems. (turku, Finland,
encoded. However, using the seven 2004).
Commun. ACM 42, 1 (Jan. 1999), 65–72.
28. Pollock, n., Williams, r., and Procter, r. Fitting standard
principles, the university management 2. attewell, P. technology diffusion and organizational software packages to non-standard organizations: the
learning: the case of business computing. Organization
decided that even though the package Science 3, 1 (1992), 1–19.
‘biography’ of an enterprise-wide system. Technology
Analysis and Strategic Management 15, 3 (2003),
itself was free of charge, the supporting 3. au, y.a. and Kauffman, r.J. should we wait? network 317–332.
externalities and electronic billing adoption. Hawaii 29. raymond, e.s. The Cathedral and the Bazaar:
network around the package was too International Conference on System Sciences. (Hawaii, Musings on Linux and Open Source by an Accidental
local and too small for the university to 2001). Revolutionary. o’reilly, 1997.
4. bailey, J., McKnight, l., and bosco, P. the economics 30. sawyer, s. effects of conflict on packaged software
invest in encoding the remainder of its of advanced services in an open communications development team performance. Information Systems
buildings into the package. infrastructure: transaction costs, production costs, and Journal 11, 2 (2001), 155–178.
network externalities. Information Infrastructure and 31. sawyer, s. Information systems development: a
Another example of the application Policy 4 (1995), 255–277. market-oriented perspective. Commun. ACM 44, 11
of the principles was the company in 5. beatty, r.C. and Williams, C.d. erP II: best practices (nov. 2001), 97–102.
for successfully implementing an erP upgrade. 32. shankland, s., Kanellos, M. and Wong, W. sun and
the field study mentioned earlier. The Commun. ACM 49 (Mar. 2006), 105–109. Microsoft settle Java suit., 2001.
company used the principles to annu- 6. besen, s.M. and Farrell, J. Choosing how to compete: 33. shapiro, C. and Varian, H.r. the art of standards wars.
strategies and tactics in standardization. Journal of California Management Review 41, 2 (1999), 8–32.
ally monitor the decision to stay with Economic Perspectives 8, 2 (1994), 117–131. 34. shapiro, C. and Varian, H.r. Information Rules: A
7. Chapman, M.r. In Search of Stupidity: Over 20 Years of
a package that had been dominant but High-Tech Marketing Disasters. apress, berkeley, Ca,
Strategic Guide to the Network Economy. Harvard
business school Press, boston, Ma, 1999.
was losing market share. The question 2003. 35. sia, s.K. and soh, C. an assessment of package-
8. Chiasson, M.W. and Green, l.W. Questioning the It
was straightforward: Was the network artefact: user practices that can, could, and cannot be
organization misalignment: Institutional and
ontological structures. European Journal of
of users around the software package supported in packaged-software designs. European Information Systems 16, 5 (2007), 568-583.
Journal of Information Systems 16, 5 (2007), 542–554.
sufficiently large to provide the pack- 9. damsgaard, J. and lyytinen, K. the role of
36. truex, d.P., baskerville, r., and Klein, H. Growing
systems in emergent organizations. Commun. ACM 42,
age owner with revenue that allowed intermediating institutions in diffusion of electronic 8 (aug. 1999), 117–123.
data interchange: How industry associations in the 37. West, J. the economic realities of open standards:
it to invest in developing the package? grocery sector intervened in Hong Kong, Finland, black, white and many shades of gray. Standards
For a number of years the answer was and denmark. The Information Society 17, 3 (2001), and Public Policy. s. Greenstein and V. stango (eds.).
195–210. Cambridge university Press, Cambridge, Ma, 2007.
positive, but when the network was 10. david, J.s., schuff, d., and louis, r.s. Managing your
deemed inadequate, it was decided to total It cost of ownership. Commun. ACM 45, 1 (2002),
switch to the dominant package.17 An Jan Damsgaard ( is director and a
11. david, P.a. Clio and the economics of Qwerty. The
professor at the Center for applied ICt, Copenhagen
illustration of Principle Five and Six is American Economic Review 75, 2 (1985), 332–337.
business school, denmark school of Information
12. david, P.a. narrow Windows, blind Giants, and
as follows: One large manufacturer had systems, Curtin university of technology, australia.
angry orphans: the dynamics of systems rivalries
already implemented one ERP system and dilemmas of technology Policy. technological
Jan Karlsbjerg ( is an associate
Innovation Project (no. 10), stanford university, Ca,
when a vendor offered a competing product manager at active Community solutions,
Vancouver, Canada.
13. david, P.a., and bunn, J.a. the economics of Gateway
ERP system at a very competitive price. technologies and network evolution: lessons from
The manufacturer attempted to switch electricity supply history. Information Economics and © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 71
contributed articles
Doi:1145/ 1787234.1787253
computer users and uses were also very
Safe, modern programming languages let different; the small group of people
with access to computers understood
Microsoft rethink the architectural trade-offs the technology and tolerated its short-
in its experimental operating system. comings. Though computers were in-
creasingly important in business, and
By JameS LaRuS anD GaLen hunT thus operated in secure environments,
they were not central to anyone’s per-

sonal life. None of these characteristics
is true today.
Construction of the Singularity
operating system began in 2004 with

three design principles:
Use safe high-level programming lan-
guages to the greatest extent possible.
They prevent entire classes of critical

errors (such as those enabling buffer
overrun attacks) while facilitating de-
velopment and use of accurate and ef-
ficient software-development tools;
Software failure should not lead to
system failure. Despite advances in pro-
gramming languages and tools, per-
fect software remains a vision for the
future. However, robust system archi-
at Microsoft Research
t h e s Ingu LARIt y pRo Je c t tecture can limit the consequences of a
failure and give a system the ability to
began by asking what modern operating-system respond and recover without having to
and application software would look like if it were reboot; and
designed with modern software-engineering practices Systems should be self-describing at
all levels of abstraction. Specification
and tools.9 Answering is important, since almost and verification are increasingly com-
every system today shares a common intellectual mon for language features and library
interfaces. However, as systems consist
heritage with the time-sharing systems developed in of many components, most are never
the 1960s and 1970s. Computers and the computing formally described. Introducing speci-
environment have changed dramatically since then, fications at the boundaries of compo-
nents describes both their dependen-
but system software has evolved much more slowly, cies and their contributions to the
leaving a wide gap between system requirements and system, enabling principled decisions
about system architecture.
In the 1960s, computers were limited, expensive key insights
devices used only by small groups of highly trained new demands on computer systems
experts. Their limited speed, memory capacity, require rethinking assumptions
concerning language, operating system,
and storage forced designers and programmers to and system architecture.

be parsimonious with resources. Applications and Safe modern programming languages

IllustratIon by st udIo to nne

promise significant benefits for

systems were generally written in assembly language, constructing high-performance
not in high-level programming languages, as they Systems must be self-describing at
are today. Extensive sharing of code and data was all levels of abstraction for building
automatic tools that verify and validate
essential for efficient use of scarce memory. Moreover, their correctness and integrity.

72 comm unicaTio nS o f T he ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

Cred It t K

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 73
contributed articles

figure 1. Structure of a Singularity system.


content extension HTTP server TCP/IP stack network driver

processes ext. class app. class subsys. class driver class
library library library library

runtime runtime runtime runtime

kernel aBi

security subsystem

memory manager
directory service
manifest binder

debugger and
i/o manager

kernel class
kernel library

hardware abstraction layer runtime

Singularity differs in significant gate new OS abstractions. Although the the additional information provided by
ways from most previous operating Singularity kernel includes many fea- manifests and specifications, Singular-
systems, pointing the way to systems tures found only in production OS ker- ity is able to detect and avoid conflicts
better able to respond to future com- nels (such as multiprocessor support, among components and prevent or iso-
puting requirements. Unlike Microsoft full-feature kernel debuggers, and sup- late the use of unsafe code.
Windows and Unix systems, it follows port for hardware standards like ACPI),
a microkernel design philosophy in Singularity is not a replacement for Safe Programming Languages
which much of a system’s functional- Windows or Linux, as it has no GUI and Modern programming languages (such
ity, including its device drivers and ma- only a sparse set of user applications. as C# and Java) are type and memory
jor subsystems, resides in processes Unlike in other systems, processes safe. Safety ensures a program applies
outside the kernel; Figure 1 outlines in Singularity are software-isolated pro- only operations appropriate to a partic-
the architecture of a Singularity sys- cesses, or SIPs, that rely on language ular type of object to instances of that
tem. Unlike other microkernel sys- safety, not hardware mechanisms, to object, a program does not create or
tems, most Singularity code is written isolate system software components modify memory references, and mem-
in safe high-level Sing#, a dialect of from one another. SIPs provide isola- ory is reclaimed only when no longer in
C#.a Moreover, also unlike other sys- tion and failure containment at far less use. These properties, not present in C,
tems, all user code in processes—out- performance cost than hardware mech- C++, and other languages, help detect
side the OS-supplied runtime—must anisms, so they can be used in more programming errors that could have
be written in a type- and memory-safe places than conventional processes. serious consequences; for example, in
language (such as Sing#, C#, F#, or Due to the lower cost of isolation, a safe language, input that overwrites
even Visual Basic). Singularity can require an extension a string buffer causes a runtime excep-
Conceived as an extensible home (“plug-in”) to reside in its own SIP that tion, rather than silently failing and
server, Singularity has been used pri- prevents the extension’s failure from permitting an attacker to inject mali-
marily as a research vehicle to investi- affecting its host SIP. (We describe later cious code. In addition, safe languages
how hardware protection can be com- rely on garbage collection to reclaim
a The hardware abstraction layer in Singularity bined with SIPs in Singularity to pro- memory, relieving programmers of
consists of 21.5KLOC but only 1,700 lines of
vide multiple layers of protection.) Sin- having to devise and enforce conven-
unsafe Sing# and 350 lines of assembly code.
The counterpart hardware-abstraction layer gularity also assumes more authority to tions concerning when an object is no
in Windows includes 25KLOC of unsafe C and decide which system components can longer in use and which component
assembly. be safely loaded and executed. Due to has the obligation to free the object.

74 com municaTio nS o f T h e acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

Furthermore, because safe languages in memory footprint Bartok achieves Other systems, including Cedar/
have a fully defined semantics, unlike for a variety of programs. Much of the Mesa, Lisp Machines, and Java, were
languages like C, with one seman- code and data “shaken” out of these written in higher-level languages and
tics if a program obeys the language programs comes from the unused por- depend on language safety to isolate
rules and no guarantees if they don’t, tions of general-purpose libraries. different computations running in
program-analysis tools are not put in Language safety is another founda- the same address space. While the
the untenable position of assuming a tion of Singularity’s SIPs, which consist SPIN operating system uses traditional
buggy program plays strictly according of memory pages holding the objects a page-based hardware protection be-
to the language definition. process can access (see Figure 2). Singu- tween processes, it also depends on
Safe languages are far more popular larity enforces the invariant that a refer- language safety to isolate OS exten-
since the introduction of Java but are ence manipulated by process P1 cannot sions running in the kernel’s address
generally considered inappropriate for point to a page belonging to process P2, space.4 Singularity’s approach differs
systems code, which is usually written where P1 ≠ P2. A process might try to in that it isolates a process’s objects
in a low-level, glorified assembly lan- violate this invariant in two ways: by memory pages, rather than allocat-
guage like C or its more sophisticated Create a new reference or modify an ing them in a common address space.
cousin C++. The common belief is that existing reference to point to another pro- When a process terminates, Singularity
safe languages are inefficient, due in cess’s page. Language safety guarantees quickly reclaims the process’s memory
part to the size and complexity of their that code running on Singularity cannot pages, rather than turning to garbage
runtime systems and reliance on gar- perform either of these operations; and collection to reclaim memory. Beyond
bage collection. Pass a reference to another process’s the performance benefits of improved
Singularity’s Bartok compiler pro- page. This operation is prevented by memory locality and a simplified gar-
vides language safety without the typi- Sing#’s type system for inter-process bage collector, the isolation invariant
cal performance penalty by compiling communication. is far easier for the operating system to
C#’s Microsoft Intermediate Language
representation to native (x86, x64, or Table 1. memory footprint for “hello World” process (in kilobytes).
ARM) code at installation time rather
than at runtime. Bartok also links com-
Singularity freeBSD 5.3 Linux 2.6.11 (Red hat fc4) Windows XP (SP2)
piled code to a small runtime consist-
c - static lib — 232Kb 664Kb 544Kb
ing of only a class library and a garbage
c++ - static lib — 704Kb 1,216Kb 572Kb
collector, not a large runtime environ-
c# - w/ Gc 408Kb — — 3,750Kb
ment like the Common Language Run-
time (the virtual machine component
of Microsoft .NET) and the Java Virtual
Machine. The Shared Source Common
Language Infrastructure runtime and Table 2. memory-footprint reduction due to tree shaking.
class library are more than five times
larger than the Bartok runtime (64
thousand lines of code, or KLOC, vs. code (Total) code (Tree Shake) % Reduction
350KLOC), roughly the same as the C Singularity kernel 2,371 Kb 1,291 Kb 46%
runtime in the latest version of Win- Web Server 2,731 Kb 765 Kb 72%
dows (72KLOC). Moreover, Bartok is a SPecweb99 Plug-in 2,144 Kb 502 Kb 77%
highly optimizing compiler that gen- iDe Disk Driver 1,846 Kb 455 Kb 75%
erates high-quality code and reduces
memory use through extensive tree
shaking to discard unneeded class vari-
ables and method definitions. figure 2. Singularity process objects reside on a dedicated collection of pages.
Table 1 emphasizes this point by
outlining the memory footprint for a
small program written in C, C++, and Process 1 Process 2 Process 3

C# running on several different oper-

ating systems. The program outputs
“Hello World” using the standard I/O
libraries and APIs for each system— channel
printf for C and C++ and Console.
WriteLine for C#. The C# code on
Singularity is smaller than for all but
exchange heap
one other system—the statistically
linked code on Free BSD—in some
cases half to one-third the size of C++
code. Table 2 outlines the reduction

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 75
contributed articles

enforce at a process level, rather than Table 3. Basic cost (in cPu cycles) of common operations between isolated processes on
word level. an amD athlon 64 3000+ system.
Singularity also provides flexible
hardware-based process isolation as Singularity freeBSD 5.3 Linux 2.6.11 (Red hat fc4) Windows XP (SP2)
a secondary mechanism. A Singular- Process create 353,000 1,030,000 719,000 5,380,000
ity hardware-protection domain is an and start
address space holding one or more minimum 91 878 437 627
SIPs. Domains can run in either user kernel aPi call

or kernel mode (ring 3 and ring 0 on Thread 346 911 906 753
context switch
an x86 processor). At runtime, the sys-
message 803 13,300 5,800 6,340
tem-configuration manifest specifies request/reply
which SIPs reside in which domains.
Domains allow untrusted code to be
isolated behind conventional hard-
ware-protection mechanisms while figure 3. hybrid hardware-software isolation using SiPs and domains.
more trusted code resides in the same
address space, benefiting from faster
communications and failure isolation
Unsigned Protection domain
(see Figure 3). App2 ring 3
Domains also enable Singularity de- ring 0
velopers to run a series of experiments
comparing the execution overheads of
software and hardware isolation.1 The
basic cost of software isolation is the Signed Unsigned
App1 extension extension
runtime checks for null pointers and
array accesses (4.7% of CPU cycles). By
contrast, hardware isolation similar to
conventional operating systems (sepa-
rate address spaces and protection do- Unsigned Signed
mains) incurred a cost of up to 38% of Driver Driver Kernel

CPU cycles (see Figure 4).

modular System architecture

Unlike many systems, Singularity as-
sumes that software contains bugs checkers) share their host process’s compatibility as a system evolves. Un-
and consequently is likely to fail oc- address space and have unconstrained less the system formally specifies the
casionally. Singularity’s architecture access to its code and data structures. interface between a plug-in and its
aims to contain the consequence of a An extension’s failure typically causes host, seemingly unrelated changes to
failure within a fault-isolation bound- the host to fail as well. Considerable the host can affect the plug-in and pro-
ary, thereby allowing the system to de- evidence shows that extensions are duce many failures despite extensive
tect the failure and recover by restart- less reliable than host code; for ex- testing regimes.
ing the failed component. Although ample, Orgovan and Tricker reported11 The Singularity architecture avoids
less intellectually appealing than flaw- that approximately 85% of the Win- many of these problems. For example,
less operation, most complex artifacts dows XP kernel crashes they studied is SIPs are sealed processes that prohibit
share this paradigm and most pro- caused by device drivers, and Chou et shared memory, in-process code gen-
grammers are comfortable with it; for al. reported that the Linux drivers they eration, and dynamic code loading. A
example, a car does not stop running studied have up to seven times the bug process that wishes to invoke an exten-
when a headlight burns out or a tire density of other kernel code.5 sion starts the extension code running
goes flat. Plug-in architectures also involve in a separate SIP. If the extension fails,
Tight coupling between compo- other disadvantages: First, code exten- its process terminates, but the par-
nents in monolithic software systems sions can subvert modularity and en- ent process continues and can recover
routinely means the failure of one gineering discipline. A plug-in can use from the error. Moreover, the exten-
component can bring down an appli- any data structure or procedure it can sion is limited to the functionality ex-
cation and, in the worst case, the sys- discover. Most of a host’s functional- plicitly provided by the parent process.
tem itself. The epitome of this prob- ity may be private or inappropriate for This recovery is feasible in many cases
lem is the common plug-in software plug-ins, but the host has no way to pre- because of three built-in Singularity
architecture that allows extensions to vent its use, except, perhaps, by hiding design decisions:
be dynamically loaded into a host’s ad- names and documentation. Moreover, SIPs are inexpensive.10 The cost of
dress space. Plug-ins (such as device a plug-in that uses undocumented creating a SIP and communicating be-
drivers, browser extensions, and spell functionality can frustrate backward tween two SIPs is low in terms of CPU

76 comm unicaTio nS o f Th e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

cycles, thus reducing the overhead of the first process left the shared struc- it (see Figure 5). This semantic pre-
this isolation mechanism and allow- ture in an inconsistent state.7 Shared vents SIPs from sharing the memory in
ing it to be used at finer granularity memory further opens each process a message while allowing for efficient
than a conventional process. The high to spontaneous corruption of shared communications, as code cannot dis-
cost of processes on other systems en- state at any time by an errant or mali- tinguish communication in which a
courages monolithic software archi- cious peer. By forbidding shared mem- message is copied from communica-
tectures and plug-ins to extend system ory, Singularity ensures that process tion in which a pointer to the message
behavior. On Singularity, program- state is altered by only one process at is passed among the SIPs. The receiv-
mers are able to encapsulate small a time; and ing SIP should still validate message
extensions to existing applications or Communication between SIPs pass- parameters but need not worry about
to the system itself in their own sepa- es through strongly typed channels.6 A their asynchronous modifications.
rate SIPs. Table 3 summarizes the cost channel is a pair of bounded message Each channel is annotated with a
in terms of CPU cycles of a variety of queues between two SIPs. A message specification, or “contract,” of the con-
systems for creating a process and is a structure consisting of scalar types tent of each message and the allowable
communicating with the kernel and (such as integers, float, and strings), sequence of messages. For example,
another process. These operations are arrays of structures, and pointers to the following code is part of the con-
far less costly on Singularity; other structures sent in the same send tract for a channel to Singularity’s TCP
SIPs do not share memory. Data operation. Messages are allocated service, defining the legal messages
structures shared between two pro- in a special area of memory—the Ex- that can arrive at the service when a
cesses provide a simple, high-band- change Heap—with programs access- socket is connected:
width communication mechanism ing it through a special Sing# type
requiring little forethought on the part system that permits at most one out- public contract TcpSocketCon-
of the host. However, when a process standing reference to a data structure. tract {
fails, the shared structure couples the When a SIP sends a message across a ...
failure to the other process, support- channel, it relinquishes ownership of state Connected : {
ing the conservative assumption that the message and can no longer access Read? -> ReadResultPending;
Write? -> WriteResultPending;
figure 4. normalized execution time comparing the overhead cost of software and hardware
GetLocalAddress? ->
process isolation mechanisms for a Web server running on Singularity. our experiments ran
on a 1.8Ghz amD athlon 64 3000+ system, starting with a pure software-isolated version of IPAddress! -> Connected;
Singularity, progressively adding hardware address-space protection. GetLocalPort? -> Port! ->
DoneSending? -> ReceiveOnly;
unsafe code Tax
DoneReceiving? -> SendOnly;
1.40 +33.0% Close? -> Closed;
+18.9% Abort? -> Closed;
1.20 Safe code Tax
+6.3% }
1.00 state ReadResultPending : {
Data! -> Connected;
NoMoreData! -> SendOnly;
0.60 RemoteClose! -> Zombie;
0.40 ...

0.00 If, for example, the service receives

SiPS without SiPS in SiPS in one Web server in Web server in all SiPS in a Read message from a client, the
runtime physical virtual memory separate ring 3 separate
checks memory address space address space address space address spaces contract transitions to the ReadRe-
sultPending state, where the service
is expected to respond with a packet
of data or a status or error indication.
figure 5. message exchange across a channel; message ownership passes from Process 1 Singularity’s compiler statically checks
through a channel to Process 2.
the code that sends and receives mes-
sages on a channel, ensuring it obeys
Process 1 Process 2 Process 3 the contract.
One objection to SIPs and chan-
nels is they make writing software
more difficult than shared data struc-
tures and procedural APIs. Channel
… Page 23 Page 24 Page 25 Page 26 Page 27 … contracts clearly require forethought
for designing and specifying an inter-
face, which is a good thing. In practice,

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 77
contributed articles

programming language support for manifests.12 A Singularity device driver

communications, explicit contracts, specifies the underlying hardware re-
and compiler checking reduces the sources (such as memory mapped I/O
burden of this style of development. registers) it can access.
As an experiment, we removed one of
the Bartok compiler’s most complex unlike in other Depending on hardware support,
Singularity may corroborate only a
components—its register allocator—
and ran it in a separate SIP. It shared
systems, processes subset of this information, but it uses
the declared information in the fol-
code for 156 classes with the rest of the in Singularity are lowing ways to ensure correct system
compiler, running every time a func-
tion is compiled. Because its interface
software-isolated configuration:
Look for conflicting claims. When a
originated in a shared address space, it processes, or driver is loaded, Singularity looks for
passes a large amount of data—50KB–
1.5MB—at every invocation, much
SiPs, that rely on conflicting claims on hardware re-
sources. If a new driver uses the same
of which is the same across allocator language safety, I/O registers as an existing driver, then
invocations (such as the machine de-
scription). Nevertheless, we were able not hardware Singularity avoids a conflict by refusing
to load the new driver; and
to run the allocator in a separate SIP mechanisms, to Incorporate declared resources. If the
system detects no conflicts, then Sin-
by changing 508 lines of code (0.25%
of the compiler), and the modified isolate system gularity incorporates its declared re-
compiler ran only 11% slower while
compiling the Singularity kernel. De-
software sources into the system manifest used
to configure the boot process. When
signing the interface to the allocator components from starting up, the Singularity kernel starts
appropriately in the first place could
reduce the communications cost and
one another. each device driver in its own SIP. It also
creates in-process I/O objects for ac-
overhead penalty. Still, the experiment cessing the I/O registers and interrupt
shows the practicality of partitioning lines used by the driver. These pre-pop-
even a complex interface so it works ulated I/O objects simplify driver ac-
across channels. cess to hardware while simultaneously
providing low-cost access to hardware
Self-Describing Systems resources with language safety.
For the past 10 years, software-develop- Singularity demonstrates that
ment tools based on formal methods lightweight specifications are valu-
have become increasingly sophisti- able if closely connected to the under-
cated and available8 for comparing a lying system and offers a value greater
specification of the intended behavior than the additional burden they im-
of a system component against the pose. Specifications may be closely
component’s actual code, pointing tied to the actual code. Documenta-
out discrepancies between the behav- tion grows stale in the absence of sys-
iors. Such tools, including SLAM2 and tematic tools to detect discrepancies
Boogie,3 generally check the behavior between a description and the related
of procedure and method boundaries. code. On the other hand, specifica-
While the proper use of these inter- tions that drive tools remain closely
faces is central to writing correct soft- linked to code and must meet only the
ware—and strongly supported by Sin- lower bar of providing sufficient util-
gularity, including language support ity to justify learning a new language
for the Boogie verification system— and unfamiliar tools.
systems provide many other abstrac-
tions. The correctness of a system de- Discussion
pends on them, as well as on low-level The Singularity project is first and
interfaces. foremost an experiment in build-
Singularity follows this paradigm ing from scratch a nontrivial system
of specification and checking at many (approximately 250KLOC) using a
different levels of system structure, in- safe language. Much of what we have
cluding for purposes other than static- learned may be of value in other sys-
defect detection. Channel contracts, tems, and many ideas have been trans-
described earlier, capture the behavior ferred into Microsoft products. Ben-
of Singularity’s primary communica- efits include SIPs for encapsulating
tion mechanism. Another example of program components, configuration
high-level specification is device-driver of system components by manifest,

78 comm unicaTio nS o f Th e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

and a lightweight, compiled runtime Finally, C#, like many modern lan- zel, Steven Levi, Nick Murphy, Mark
system for safe code. Like any system, guages, does not provide convenient Aiken, Derrick Coetzee, Ed Nightin-
Singularity also has its rough spots, mechanisms for manipulating bit- gale, Brian Zill, and Richard Black
and future research should aim to level formatted data and inlined arrays built portions of the operating sys-
help resolve three troubling issues: found in device-control registers and tem. Ted Wobber, Martin Abadi, An-
the garbage collector in the kernel; the network packets. Not adding this func- drew Birrell, Ulfar Erlingsson, and
inconsistencies between Sing#’s two tionality to Sing# early in the Singular- Dan Simon developed the security ar-
type systems; and C#’s incomplete ity-development project was an omis- chitecture. In addition, more than 30
type system. sion that continues to incur a penalty. interns contributed heart, mind, and
Despite early concern in the project hands to the project.
and ongoing external skepticism, our conclusion
experience shows that high-perfor- Singularity is a small operating system References
1. aiken, M., Fähndrich, M., Hawblitzel, C., Hunt, G.,
mance system software can be built we and a group of our colleagues at Mi- and larus, J.r. deconstructing process isolation.
in a garbage-collected language. Sin- crosoft Research built to demonstrate In Proceedings of the ACM SIGPLAN Workshop on
Memory Systems Performance and Correctness (san
gularity performed much better on a nontrivial change in the standard Jose, Ca, oct.). aCM Press, new york, 2006, 1–10.
basic micro and macro benchmarks practice of designing and construct- 2. ball, t. and rajamani, s.K. the slaM toolkit. In
Proceedings of the 13th Conference on Computer-
than we originally anticipated, and ing software. On today’s fast comput- Aided Verification (Paris, July). springer, 2001,
when failing to perform well, prob- ers, it is no longer necessary to design 260–264.
3. barnett, M., Change, b.-y.e., deline, r., Jacobs, b.,
lems were seldom attributable solely systems around the lowest common and leino, K.r. boogie: a modular reusable verifier
for object-oriented programs. In Proceedings of the
to garbage collection. Our experience denominator of assembly language Fourth International Symposium on Formal Methods
confirms wisdom in the Java and Com- or C, seeking performance to the det- for Components and Objects (amsterdam, the
netherlands, nov.). springer, 2005, 364–387.
mon Language Runtime communities riment of essential system attributes 4. bershad, b.n., savage, s., Pardyak, P., sirer, e.G.,
that garbage collection obviates the (such as modularity and reliability). Fiuczynski, M., becker, d., eggers, s., and Chambers,
C. extensibility, safety and performance in the sPIn
need for strict memory accounting Singularity shows that modern, safe operating system. In Proceedings of the 15th ACM
but does not eliminate the need for programming languages enable new Symposium on Operating Systems Principles (Copper
Mountain resort, Co, dec.). aCM Press, new york,
carefully managing memory in high- system architectures that not only 1995, 267–284.
performance code. improve robustness but perform bet- 5. Chou, a., yang, J., Chelf, b., Hallem, s., and engler,
d. an empirical study of operating systems errors.
The design of an optimal garbage ter in many circumstances than tradi- In Proceedings of the 18th ACM Symposium on
collector for an OS kernel is an open tional approaches. Operating Systems Principles (Chateau lake louise,
banff, Canada, oct.). aCM Press, new york, 2001,
question. The assumptions underlying The lessons of Singularity are appli- 73–88.
generational collectors do not agree cable far beyond the ground-up design 6. Fähndrich, M., aiken, M., Hawblitzel, C., Hodson, o.,
Hunt, G., larus, J.r., and levi, s. language support
with the lifetime of many kernel ob- of new systems; for example, mani- for fast and reliable message-based communication
jects that persist as long as the system fests could be used in more traditional in singularity os. In Proceedings of the First ACM
SIGOPS/EuroSys European Conference on Computer
or process exists. Reference counting, operating systems to describe depen- Systems (leuven, belgium, apr.). aCM Press, new
york, 2006, 177–190.
despite trade-offs involving cost and dencies, cross-process communica- 7. Flatt, M. and Findler, r.b. Kill-safe synchronization
the inability to reclaim cyclic struc- tion, and hardware access. Likewise, abstractions. In Proceedings of the 2004 ACM
SIGPLAN Conference on Programming Language
tures, is common in conventional oper- replacing in-process plug-ins with Design and Implementation (Washington, d.C., June).
ating systems and deserves reexamina- components in separate processes aCM Press, new york, 2004, 47–58.
8. Hinchey, M., Jackson, M., Cousot, P., Cook, b., bowen,
tion as a garbage-collection technique would improve the resilience of any J.P., and Margaria, t. software engineering and formal
for safe kernels. system. Gradually incorporating safe methods. Commun. ACM 51, 9 (sept. 2008), 54–59.
9. Hunt, G. and larus, J. singularity: rethinking the
Sing#, the language of Singular- languages, software isolation, and in- software stack. ACM SIGOPS Operating Systems
ity, supports two type systems: C# and creased specification into existing sys- Review 41, 2 (apr. 2007), 37–49.
10. Hunt, G., aiken, M., Fähndrich, M., Hawblitzel, C.,
data passed between processes. Data tems offers cost-effective incremental Hodson, o., larus, J., levi, s., steensgaard, b., tarditi,
in a process is conventional C# ob- improvement. d., and Wobber, t. sealing os processes to improve
dependability and safety. In Proceedings of the
jects, but data passed along channels Source code for the Singularity sys- Second ACM SIGOPS/EuroSys European Conference
lives in a distinct type system, limited tem is available for noncommercial on Computer Systems (lisbon, Portugal, Mar.). aCM
Press, new york, 2007, 341–354.
to structs, not objects, and is governed use at 11. orgovan, V. and tricker, M. An Introduction to Driver
by strict rules restricting references. gularity. Quality. Microsoft WinHeC 2004 presentation
ddt301, new orleans, la, 2003.
This system allows static verification 12. spear, M.F., roeder, t., levi, s., and Hunt, G. solving
the starting problem: device drivers as self-describing
of channel contracts but exacts a price acknowledgments artifacts. In Proceedings of the EuroSys 2006
in programmer frustration and addi- Singularity was the work of large Conference (leuven, belgium, apr.). aCM Press, new
york, 2006, 45–58.
tional code for marshalling, unmar- team of dedicated individuals: David
shalling, and operations on the structs. Tarditi, Bjarne Steensgaard, Qun-
Increased interoperability or, better, yan Mangus, Mark Plesko, and Juan James Larus ( is director of
research and strategy in the extreme Computing Group
a unified type system would simplify Chen built the Bartok compiler and at Microsoft research, redmond, Wa.
the code for creating and manipulat- runtime. Manuel Fähndrich, Song-
Galen hunt ( is principal
ing messages. In addition, the channel tao Xia, Sriram Rajamani, Jakob Re- researcher in the Microsoft research operating systems
contracts we used were not expressive hof, Herman Venter, Rebecca Isaacs, Group and leads the Menlo project and the singularity
project at Microsoft research, redmond, Wa.
enough to describe asynchronous in- and Tim Harris worked on Sing# and
teractions between processes. tools. Orion Hodson, Chris Hawblit- © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 79
contributed articles
Doi:10.1145/ 1787234.1787254
channels. Examples of services that
Early patterns of Digg diggs and YouTube have made the exchange between pro-
ducer and consumer possible on a
views reflect long-term user interest. global scale include video, photo, and
music sharing, blogs, wikis, social
By GaBoR SzaBo anD BeRnaRDo a. huBeRman bookmarking, collaborative portals,
and news aggregators, whereby con-

tent is submitted, perused, rated, and
discussed by the user community.
Portals often rank and categorize
content based on past popularity and

the Popularity
user appeal, especially for aggregators,
where the “wisdom of the crowd” pro-
vides collaborative filtering to select
submissions favored by as many visi-

of online
tors as possible. Digg is an example,
with users submitting links to and
short descriptions of content they have
found on the Web and others voting on

them if they find them interesting. The
articles attracting the most votes are ex-
hibited on the site’s premiere sections
under headings like “recently popular
submissions” and “most popular of
the day.” This placement results in a
positive feedback mechanism leading
to rich-get-richer vote accrual for the
very popular items, though the pattern
pertains to only a small fraction of the
t h e e Ase oF producing online content highlights submissions that rise to the top.
Besides Digg, anyone with Internet
the problem of predicting how much attention any access can watch YouTube videos, reply
of it will ultimately receive. Research shows that user to them through their own videos, and
leave comments. The way the online
attention9 is allocated in a rather asymmetric way, with ecosystem has developed around You-
most content getting only some views and downloads, Tube videos is impressive by any stan-
whereas a few receive the most attention. While it dard, and videos that draw millions of
viewers are prominently displayed on
is possible to predict the distribution of attention the site, like stories on Digg.
over many items, it is notably difficult to predict the Content providers, Web hosts, and
advertisers all would like to be able to
amount that will be devoted over time to any given predict how many views and downloads
item. We solve this problem here, illustrating our individual items might generate on a
approach with data collected from the portals Digg given Web site. For example in advertis-
( and YouTube (, key insights
two well-known examples of popular content-sharing-
Site administrators, advertisers, and
and-filtering services. providers would all find it useful to be
able to predict content popularity.
The ubiquity of Web 2.0 services has transformed
Prediction is possible due to the extreme
the landscape of online content consumption. With regularity with which user attention
focuses on content.
the Web, content producers can reach an audience early patterns of access indicate long-
in numbers inconceivable through conventional term popularity of content.

80 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

Sports Gaming Technology

offbeat Science World & Business

Sunday monday Tuesday Wednesday Thursday friday Saturday entertainment Lifestyle

chris harrison’s Digg Rings visualization plots the top 10 most-dugg stories by days of the week may 24, 2007 to may 23, 2008 (bottom) and
all stories (close up) Dec. 1, 2004 to may 23, 2008 (top) rendered as a series of tree-ring-like visualizations moving outward in time.

ing, if popularity count is tied directly 0.3% to images. Submitted content is of pride in the Digg community and a
to ad revenue (such as with ads shown placed by the submitters on Digg in main motivator for repeat submitters.
with YouTube videos), revenue might the so-called “upcoming” section, one The exact algorithm for promotion is
fairly accurately be estimated ahead of click from the site’s main page. Links not made public to thwart gaming but
time if all parties know how many views to content are provided, along with is thought to give preference to upcom-
the video is likely to attract. Moreover, surrogates to the submission (a short ing submissions that accumulate diggs
in content-distribution networks, the description for news, a thumbnail im- quickly from diverse neighborhoods in
computational requirements for band- age for images and videos) intended to the Digg social network,7 thus modu-
VIsua lIZatIon by CHrIs H arrIson, CarneGIe Mellon unIVersIt y

width-intensive new content may be entice readers to peruse the content. lating the influence of very popular
determined early on if the hosting site Digg functions as a massive collabora- submitters with hundreds of followers.
is able to extrapolate the number of re- tive filtering tool to select and share the Digg’s social-networking feature lets
quests the content is likely to get by ob- most popular content in the user com- users place watch lists on other users
serving patterns of access from the mo- munity; registered users thus digg sub- by becoming their fans. Fans are shown
ment it was first posted. missions they find interesting. Digging updates on which submissions are
Digg allows users to submit links increases the digg count of the submis- dugg by these users; the social network
to news, images, and videos they find sion one digg at a time, and submis- therefore plays a major role in making
on the Web and think will interest the sions that get enough diggs in a certain upcoming submissions more visible to
site’s general audience. Based on data amount of time in the “upcoming” sec- a larger number of users. Here, we con-
we collected from Digg in the second tion are shown on the Digg front page, sider only stories that were promoted,
half of 2007, 90.5% of all uploads were or, per Digg terminology, “promoted.” since we were interested in submis-
links to news, 9.2% to videos, and only Promotion is a considerable source sions to which many users had access.

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 81
contributed articles

We used the Digg application pro- YouTube is the apex of the Web’s added” videos, it also offered listings
gramming interface (http://apidoc. user-created video-sharing portals, based on such YouTube-defined selec- to retrieve all diggs made with (as of 2008) 65,000 new videos up- tion criteria as “featured,” “most dis-
by registered users from July 1, 2007 loaded and 100 million viewed daily, cussed,” and “most viewed.” We chose
to December 18, 2007. This data set in- implying that 60% of all online videos the “most recently uploaded” list to
cluded approximately 29 million diggs were watched through YouTube.3,6 It give us an unbiased sample of all vid-
by 560,000 users on approximately 2.7 was also the third most frequently ac- eos submitted to the site or complete
million submissions, a number includ- cessed site on the Web, based on traf- history of the view counts for each vid-
ing all past submissions receiving any fic rank.1 Beginning April 21, 2008, we eo during its lifetime. YouTube’s API
digg, not only the submissions during collected view-count time series on (
the six months. The number of submis- 7,146 selected videos daily in the por- overview.html)10 provided program-
sions was about 1.3 million, of which tal’s “recently added” section, carrying matic access to several video statistics,
about 94,000 (7.1%) were promoted to out data collection for the next 30 days. with view count at a given time being
the front page. Apart from the list of “most recently one of them.
However, due to the fact that the
figure 1. average normalized popularity of submissions to Digg and youTube by individual view-count field of a video did not ap-
popularity at day 30. The inset is the same measurement for the first 48 digg hours
of Digg submissions.
pear to have been updated more often
than once a day by YouTube, we were
able to calculate only a good approxi-
youtube mation of the number of daily views.
1.2 Worth noting is that while the over-
average normalized popularity

whelming majority of video views was

initiated from the YouTube Web site
itself, videos might have been linked
0.8 1 from external sources as well, appear-
ing as embedded objects on the refer-
0.6 0.6
ring page; while 50% of all videos in
2007 were thought to be linked exter-
nally, only about 3% of the views came
0.2 0 10 20 30 40 from these links.2
Time (digg hours)

0 Popularity Growth
0 5 10 15 20 25 30 By “popularity” we mean number of
votes (diggs) a story collected on Digg
Time (days)
and number of views a video received
on YouTube, respectively. Figure 1 re-
flects the dynamics of content popular-
figure 2. Daily and weekly cycles in the hourly rates of digging activity, story submissions, ity growth on both portals, showing the
and story promotions, respectively. To match the different scales, we multiplied the rates
for submissions by 10 and the rates of promotion by 1,000. The horizontal axis represents
average normalized popularity for all
the week august 6, 2007 (monday)–august 12, 2007 (Sunday). The tick marks are midnight submissions over time; we first deter-
on the respective day, Pacific Standard Time. mined the popularity of each individ-
ual submission at the end of the 30th
submissions * 10 day following its submission, dividing
promotions * 1,000 their popularity values before that time
by this final number. For each submis-
12000 sion, we obtained a time series of pop-
ularities that monotonically increased

from 0 (at submission time) to 1 at day
8000 30. By thus eliminating the prevailing
differences in content-specific inter-
6000 estingness among the submissions
(one submission might get only a few
views over its lifetime, while another
2000 gets thousands or even millions), we
averaged overall submissions of the
0 normalized popularities.
Mon tues Wed thu Fri sat sun Mon
An important difference between
Time the two portals is that while Digg sto-
ries saturate fairly quickly (about a day)
to their respective reference populari-

82 communicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

ties, YouTube videos keep attracting figure 3. correlation of digg counts on the 17,097 promoted stories in the data set older
views throughout their lifetimes. The than 30 days. a k-means clustering separates 89% of the stories into an upper cluster; the
rate videos attract views may naturally other stories are a lighter shade of blue. The bold line indicates a linear fit with slope 1 on
differ among videos, with the less-pop- the upper cluster, with a prefactor of 5.92 (Pearson correlation coefficient of 0.90).
ular likely marking a slower pace over a
longer time. 104
These two notably different user-
popularity patterns are a consequence
of how users react to content on the two

Popularity after 30 digg days

portals. On Digg, articles quickly be- 103
come obsolete, since they often link to
breaking news, fleeting Internet fads,
or technology-related themes with a
naturally limited time for user appeal. 102

However, videos on YouTube are mostly

found through search, since, with the
sheer number of videos constantly be-
ing uploaded, it is not possible to match
101 102 103
Digg’s way of giving each promoted sto-
ry general exposure on a front page. The Popularity after one digg hour

quicker initial rise of video view counts

can be explained through the videos’ ex-
posure in YouTube’s “recently added”
figure 4. Popularity of videos on the 30th day after upload vs. popularity after seven days.
section, but after leaving it, the only way The bold line with gradient 1 is fit to the data.
to find them is through keyword search
or when displayed as related videos next
to another video being watched.
The short fad-like popularity life
cycle of Digg stories (a day or less) sug- 105

gests that if overall user activity on Digg

Popularity after 30 days

depends on time of day, a story’s popu-
larity may grow more slowly when fewer
visitors are on the site and increase
more quickly at peak periods. For You-
Tube, this effect is less relevant, since
video views are spread over more time, 101
as in Figure 1. Figure 2 outlines the
hourly rates of user digging, story sub- 100
mitting, and upcoming Digg story pro- 100 101 102 103 104 105
motions as a function of time for one
Popularity after seven days
week, beginning August 6, 2007. The
difference in rates may be as much as
threefold; weekends showed less activ-
ity, and weekdays appeared to involve digging activity at night but might have ries. We count diggs only on promoted
about 50% more activity than weekends. initially penalized interesting stories stories because this section of the por-
It was also reasonable to assume that that were otherwise likely to be popu- tal was our focus, and most diggs (72%)
besides daily and weekly cycles, such ac- lar. For instance, an average story pro- were to promoted stories anyway. The
tivity also involved seasonal variations. moted at 12 p.m. received approximate- average number of diggs arriving at
Moreover, in 2007, Digg users were ly 400 diggs in the first two hours and promoted stories during any hour day
mostly located in the UTC-5 to UTC-8 only 200 diggs if promoted at 12 a.m. or night was 5,478 when calculated
time zones (the Western hemisphere). That is, based on observations made over the full six-month data-collection
Depending on the time of day a sub- after only a few hours after a story was period; we define one digg-hour as the
mission was made to the portal, stories promoted, a portal could misinterpret time it takes for so many new diggs to
differed greatly in the number of initial the story’s relative interestingness if it be cast. As discussed earlier, the time
diggs they received. As we expected, did not correct for the variation in daily for this many diggs to arrive took about
stories submitted during less-active user-activity cycles. three times longer at night than during
periods of the day accrued fewer digs Since digging activity varies by time, the day. This “detrending” allowed us
in the first few hours than stories sub- we introduce the notion of digg time to ignore the dependence of submis-
mitted during peak hours. This was a measured not in seconds but in num- sion popularity on the time of day it
natural consequence of suppressed ber of diggs users cast on promoted sto- was submitted. Thus, when we refer to

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 83
contributed articles

the age of a submission in digg hours more slowly than the popularity of the
at a given time t, we measure how many majority of submissions; by the end
diggs were received on the portal be- of the first hour of their lifetimes, they
tween t and the promotion time of the had received most of the diggs they will
story, divided by 5,478 diggs. ever receive. The difference in popular-
Similar hourly activity plots were While Digg ity growth of the two clusters is perceiv-
not possible for YouTube in 2008,
given that video view counts were pro-
stories saturate able until approximately the seventh
digg hour, after which the separation
vided by the API approximately only fairly quickly vanishes due to digg counts of stories
once a day, in contrast to all the diggs
received by a Digg story. Moreover, we
(about a day) mostly saturating to their respective
maximum values, as in Figure 1.
were able to capture only a fraction of to their respective A Bayesian network analysis of sub-
the large amount of traffic the You-
Tube site handled by monitoring only reference mission features (day of the week/hour
of the day of submission/promotion,
the selected videos in our sample. popularities, category of submission, number of
diggs in the upcoming phase) reveals
Predicting the future youTube videos no obvious reason for the presence of
Here, we cover the process we used to
model and predict the future popular-
keep attracting clustering; we assumed it arises when
the Digg promotion algorithm mis-
ity of individual content and measure views throughout judged the expected future popularity
the performance of the predictions:
First, we performed a logarithmic
their lifetimes. of stories, promoting stories from the
“upcoming” phase unlikely to sustain
transformation on the popularities of user interest. Users lose interest much
submissions. The transformed vari- sooner in them than in stories in the
ables exhibit strong correlations be- upper cluster. We used k-means clus-
tween early and later time periods; on tering, with k = 2 and cosine distance
this scale, the naturally random fluctu- measure to separate the two clusters,
ations can be expressed as an additive as in Figure 3, and discarded the sto-
noise term. We call reference time tr ries in the lower cluster.
the time at which we intend to predict Trends and randomness. Our in-
the popularity of a submission whose depth analysis of the data found strong
age with respect to its upload (promo- linear correlations between early and
tion) time is tr. By indicator time ti we later times of the logarithmically trans-
mean when in the life cycle of the sub- formed submission popularities, with
mission we performed the prediction, correlation coefficients between early
or how long we can observe submis- and later times exceeding 0.9. Such a
sion history in order to extrapolate for strong correlation suggests the more
future popularity (ti < tr). popular submissions are at the begin-
To help determine whether the ning, the more popular they will also
popularity of submissions early on is be later on. The connection can be de-
a predictor of later popularity, see Fig- scribed by a linear model:
ures 3 and 4, which show the popular-
ity counts for submissions at the ref- ln N (tr) = ln [r(ti, tr)N(ti)] + ξ(ti, tr)
erence time tr = 30 days both for Digg = ln r(ti, tr) + lnN(ti) + ξ(ti, tr),
and YouTube vs. the popularity mea-
sured at the indicator times ti = 1 digg where N(t) is the popularity of a par-
hour and ti = 7 days for the two portals, ticular submission at time t; r(ti, tr)
respectively. We measured the popu- accounts for the linear relationship
larity of YouTube videos at the end of between the log-transformed populari-
the seventh day, so the view counts at ties at different times; and ξ is a noise
that time ranged from 101 to 104, simi- term (describing the randomness we
lar to Digg in this measurement. We observed in the data) that accounts
logarithmically rescaled the horizontal for the natural variances in individual
and vertical axes in the figures due to content dynamics beyond the expected
the large variances present among the trend in the model and is drawn from
popularity of different submissions, a fixed distribution with mean 0. It is
which span three decades. important to note that the noise term
Observing the Digg data, we noted is additive on the log-scale of populari-
the popularity of about 11% of the sto- ties, justified by the fact that we found
ries (lighter blue in Figure 3) grew much the strongest correlations on this

84 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

transformed scale. In light of Figures 3 ness of submissions should be seen tisements and number of comments
and 4, the popularities at tr also appear early on, up to a variability accounted the content is expected to generate
to be evenly distributed around the lin- for by the noise terms. on the community site. For this rea-
ear fit, taking only the upper cluster in Popularity prediction. To illustrate son we measured the performance of
Figure 3 and considering the natural how a content provider might use the the predictions as the average relative
cutoff y = x in the data for YouTube. We random logarithmic growth model of squared error over the test set or as
also found that the noise term (given content popularity on Digg and You- the expected difference of a predic-
by the residuals after a linear fit in both Tube, we performed straightforward tion from the actual popularity, in per-
the YouTube and the Digg data) is well extrapolations on the data we col- centages. For a reference time of tr to
described by a normal distribution on lected to predict future access rates. predict the popularity of submissions
the logarithmic scale. If submissions do not get more or less we chose 30 days after submission
However, there is also an alternative attractive over time as they were in time. Since the predictions naturally
explanation for the observed correla- the past, we expect their normalized depend on ti and how close we are to
tions: If we let ti vary in the model just popularity values to follow the trends the reference time, we performed the
described we see that the popularity at in Figure 1. The strong correlation be- parameter estimations in hourly in-
the given time tr should be described tween early and later times suggests tervals starting immediately after the
by the following formula, assuming the a submission that is popular at the introduction of a submission. The
noise term in the model is distributed beginning will also be popular later parameter values for the predictions
normally (t0 is an early point in time af- on. The linearity of popularity accrual (ln r(ti, tr) in the log-linear model dis-
ter submission/promotion): with a random additive noise on the cussed earlier) can be obtained with
tr logarithmic scale also allows us to ap- maximum likelihood fitting from the
lnN(tr) = lnN(t0) + ∑ η(τ). proximate the number of views/diggs training-set data.
at any given time in the future; they are The errors measured on the test set
η(τ) is a random value drawn from an predicted to be a constant product of (see Figure 5) show that the expected
arbitrary, fixed distribution, and τ is the popularity measured at an earlier error decreases rapidly for Digg (neg-
taken in small, discrete timesteps. time. However, the multiplier depends ligible after 12 hours), while for You-
The argument for this process is as on when the sampling and the predic- Tube the predictions converge more
follows: If we add up a large number tion are performed. slowly to the actual value. After five
of independent random variables, In order to perform and validate the days, the expected error made in esti-
each following the same given distri- predictions, we subdivided the sub- mating the view count of an average
bution, the sum will approximate a mission time series data into a training video was about 20%, while the same
normal distribution, no matter how set and a test set. For Digg, we took all error was attained an hour after a Digg
the individual random variables were stories submitted during the first half submission. This is due to the fact that
distributed.5 This approximate normal of the data-collection period (July to Digg stories have a much shorter life
distribution is the result of the central mid-September 2007) as the training cycle than YouTube videos, and Digg
limit theorem of probability and why set and the second half as the test set. submissions quickly collect many
normal distributions are seen so often On the other hand, the 7,146 YouTube votes right after being promoted.
in nature, from the height of people to videos we followed were submitted at The simple observation that the
the velocity of components of atoms in about the same time, so we randomly popularities of individual items are lin-
a gas. If we consider the growth of sub- selected 50% of them as training and early related to each other at different
mission popularity as a large number the other 50% as test; the table here times enables us to extrapolate to fu-
of random events increasing the loga- outlines the numbers of submissions ture popularities by measuring content
rithm of the popularity by a small, ran- in the two sets. The linear regression popularity shortly after the content is
dom amount, we arrive at the log-linear coefficients between ti and tr data were introduced. However, the detailed pa-
model just described. determined on the training set, then rameter-estimation procedure strongly
What follows from the model is used to extrapolate on the test set. depends on the idiosyncrasies of the
that on the natural, linear scale of Content popularity counts are of- random multiplicative model and the
popularities we must multiply the ac- ten related to other quantities, like type of error measure we wish to mini-
tual popularity by a small, random click-through rates of linked adver- mize (such as absolute and relative), so
amount to obtain the popularity for the
next timestep. This process is called Partitioning the collected data into training and test sets, we divided the Digg data by time
and chose the youTube videos randomly for each set, respectively.
“growth with random multiplicative
noise,” an unexpected characteristic
of the dynamics of user-submitted Training set Test set
content.9 While the increments at each Digg 10,825 stories 6,272 stories
timestep are random, their expectation (7/1/07–9/18/07) (9/18/07–12/16/07)
value over many timesteps adds up, ul- youTube 3,573 videos 3,573 videos
timately to ln r(t0, tr) in the log-linear randomly selected randomly selected

model. Thus the innate differences

among the user-perceived interesting-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 85
contributed articles

figure 5. Prediction performance is based on the logarithmic growth model It was known that the Digg social
measured by the average relative squared error function for (a) Digg and (b) youTube, network plays an important role in
respectively. The shaded areas indicate one standard deviation of the individual making a story visible and popular
submission errors around the average. when the submission is still in Digg’s
“upcoming” section, with new stories
appearing at the top of the “upcoming”
page on average every ninth second, as
in Figure 2, with about 400 new sub-
missions an hour in 2007. Though all
new submissions are shown in the “up-
Relative squared error

coming” section, the list is updated so

quickly that entries left the first page in
about two minutes. The most effective
0.2 way to discover new stories should thus
be through the social network, where
0.1 recent diggs of a user’s idols are visible
for more time on the user’s personal
0 page. To what extent then, do diggers
0 5 10 15 20 pay attention to what their idols al-
Digg story age (digg hours)
ready dugg?
To see how Digg social networking
functioned we took all submissions
for which we had data for at least 12
hours after promotion and measured
the fraction of diggers with at least
one digger among their idols and who
had already dugg the same story. In es-
Relative squared error

0.6 sence, this measurement is the prob-

ability that a new digg is made by users
0.4 who may have seen the story through
their social networks. We normalized
the times of diggs with respect to the
promotion time of the individual sub-
missions, so for diggs made before pro-
motion, time is measured backward.
0 5 10 15 20 25 30
Results are outlined in Figure 7, where
youTube video age (days) about 20% of diggers have an idol who
(b) dugg the same story before they did,
when it was still in the “upcoming”
phase. However, this figure drops con-
these constraints must be considered 2008; however, they might have seen siderably (to 7%) after promotion; most
to achieve the minimum error possible if friends recently uploaded videos. diggs are cast by users who could not
allowed by the model. Due to the limited nature of social-net- have seen the submission in their so-
working options on YouTube in 2008, cial network before. This falloff in peer
Social networking we focus on the network of Digg users. following supports the assumption
Social networking features in Web 2.0 Together with content-popularity data, that stories are found through the so-
services are so ubiquitous it is almost we also collected link information us- cial network in the “upcoming” phase,
mandatory for a site to offer them to its ing the Digg API. Figure 6 shows a typi- but once they are promoted to the front
users. For example, Digg’s approach to cal snapshot of the Digg social network page and exposed to a diverse audience
social networking is to make it possible in 2007, with about 260 users and 550 for a longer time, the effect of the so-
for users to be fans of other users, after links, where a link represents whether cial network becomes negligible.
which they are able to see what stories a particular user is a fan of another While users are about three times
their “idols” submit or digg. This is user. Users who dugg a particular story more likely to digg a submission their
essentially a restricted form of collab- are in red, with no apparent clustering idols dugg in the “upcoming” phase
orative filtering, but users themselves among them. However, these users are than after it was promoted, the mea-
select the peers they wish to follow. A relatively dense in the neighborhood of surement only intuitively suggests that
similar kind of social network is active the small social graph in Figure 6, since users pay attention to the activities of
in YouTube, though the feature that al- the story attracted nearly 15,000 diggs their peers. To determine whether dig-
lowed users to follow the videos their altogether, considerably more than the gers are truly influenced by their so-
friends were watching was nascent in average submission at the time. cial peers, the null hypothesis for user

86 communicaT io nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

contributed articles

diggs would be a scenario in which sions but browse the Digg main page to extended (or even any) social network.
users pick stories randomly, never be- see what other users found interesting Consequently, the observed probability
ing influenced by what their idols did (making up the bulk of the user base), of peer influence is diminished.
before them. If the observed fractions and, though they do not digg often, The beneficial effect of a social net-
substantially exceed the random ex- their compounded activity dominates work on content popularity is therefore
pectation, we can safely say that users the diggs a story gets at this stage. At the confined to less active periods of the
indeed pick the same submissions as same time, they are unlikely to have an content’s life cycle; that is, it matters
their peers.
We were able to test whether users figure 6. Representative example of a Digg-user social network. We randomly selected
a user as origin and included every other user in the social graph with snowball sampling
digg stories according to the random up to distance four from the user following breadth-first search. Diggers of a particular
null hypothesis by randomly shuffling story are in red; non-diggers are in green.
their activities. We simulated a scenar-
io whereby users made their diggs at
exactly the same times as they would in
real time to mimic the sessions when
they’re logged onto Digg. However, we
let the agents representing the users
digg any story present in the system,
rather than what users actually dugg.
This approach ensured that the simu-
lated agents picked a random story
from among all the stories available to
them. We maintained (important) the
agents’ social links and correspond-
ing user links, so we could observe the
presence or lack of a social-network
effect. After the agents’ selections
were randomly made, we performed
the same measurement as in Figure 7
to determine how agents might have
been influenced by their idols to digg
the same stories as their idols.
In Figure 7, the difference between
the random model (green line) and the
observed digging pattern (blue line) is
obvious: Users digging stories in the
“upcoming” phase were more than
twice as likely to digg what their idols figure 7. Probability that a digger of a story is a fan of a digger who dugg the same story
dugg than they would if there was no (blue line) as a function of the time of the digg. Time is relative to the promotion time
social network—the same as picking a of the story, with the average calculated over all diggs on all stories. The vertical red line
marks time 0 (promotion time), and negative times refer to the “upcoming” phase.
story randomly. However, and most im- The green line is the same measurement but with diggs randomly shuffled.
portant, stories late in the “promoted”
phase get diggs from users who do not
watch their links at all; the random hy- random hypothesis
pothesis delivers the same fractions as 0.25

the real observations after about a day.

However, right after promotion, users 0.2
fraction of new diggers who

seem to do the opposite of their peers:

are fans of earlier ones

The probability that a new digger is a fan

of a previous digger of a story is signifi-
cantly less than one would expect from
random choice. The controversy in this 0.1

result might be resolved if we consider

that once a submission is promoted 0.05
to the front page (shortly after time 0
in the figure), it gains tremendous vis- 0
ibility compared to the “upcoming” –2 0 2 4
phase and is exposed to many casual
Popularity after seven days
Digg users. These users do not actively
participate in discovering new submis-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 87
contributed articles

only when its visibility is minuscule We thus based our predictions of fu-
compared to its other stages, and the ture popularity only on values measur-
highest number of diggs accrues when able at the time we did the study and did
the social-network effect is nonexis- not consider the semantics of popular-
tent. We therefore do not consider this
feature (otherwise deemed important) in the presence ity and why some submissions become
more popular than others; however, this
a main contributor from a prediction
point of view in terms of total popular-
of a large user base, semantics of popularity may be used to
predict click-through rates in the ab-
ity count. predictions can sence of early-access data.8 In the pres-

be based on ence of a large user base, predictions
can be based on observed early time
In this article we have presented our observed early series, while semantic analysis of con-
method for predicting the long-term
popularity of online content based on
time series, while tent is more useful when no early click-
through information is available.
early measurements of user access. semantic analysis However, we could not explore sev-
Using two very popular content-shar-
ing portals—Digg and YouTube—we of content is more eral related areas here. For example,
it would be interesting to extend the
showed that by modeling the accrual
of votes on and views of content of-
useful when no analysis by focusing on different sec-
tions of the portals (such as how the
fered by these services we are able to early click-through YouTube “news & politics” section
predict the dynamics of individual
submissions from initial data. In Digg,
information is differs from the YouTube “entertain-
ment” section). We would also like to
measuring access to given stories dur- available. learn whether it is possible to forecast
ing the first two hours after posting al- a Digg submission’s popularity when
lowed us to forecast their popularity 30 the diggs come from only a small num-
days ahead with a remarkable relative ber of users whose voting history is
error of 10%, while downloads of You- known, as it is for stories in Digg’s “up-
Tube videos had to be followed for 10 coming” section.
days to achieve the same relative error.
The differing time scales of the predic- References
1. alexa Web Information service;
tions are due to differences in how con- 2. Cha, M., Kwak, H., rodriguez, P., ahn, y.-y., and Moon,
tent is consumed on the two portals; s. I tube, you tube, everybody tubes: analyzing the
world’s largest user-generated content video system.
Digg stories quickly become outdated, In Proceedings of the Seventh ACM SIGCOMM
while YouTube videos are still found Conference on Internet Measurement (san diego, oct.
24–26). aCM Press, new york, 2007, 1–14.
long after they are submitted to the 3. Cheng, x., dale, C., and liu, J. statistics and social
portal. Predictions are therefore more network of youtube videos. In Proceedings of the
16th International Workshop on Quality of Service
accurate for submissions for which at- (enschede, the netherlands, June 2–4, 2008),
tention fades quickly, whereas predic- 229–238.
4. digg aPI;
tions for content with a longer life cycle 5. Feller. W. An Introduction to Probability Theory and
are prone to larger statistical error. Its Applications, Vol. 1. John Wiley & sons, Inc., new
york, 1968.
We performed experiments showing 6. Gill, P., arlitt, M., li, Z., and Mahanti, a. youtube traffic
that once content is exposed to a wide characterization: a view from the edge. In Proceedings
of the Seventh ACM SIGCOMM Conference on
audience, the social network provided Internet Measurement (san diego, oct. 24–26). aCM
by the service does not affect which us- Press, new york, 2007, 15–28.
7. lerman, K. social information processing in news
ers will tend to look at the content, and aggregation. IEEE Internet Computing (Special Issue
on Social Search) 11, 6 (nov. 2007), 16–28.
social networks are thus not effective 8. richardson, M., dominowska, e., and ragno, r.
promoting downloads on a large scale. Predicting clicks: estimating the click-through rate
for new ads. In Proceedings of the 16th International
However, they are important in the Conference on the World Wide Web (banff, alberta,
stages when content exposure is con- Canada, May 8–12). aCM Press, new york, 2007,
strained to a small number of users. 9. Wu, F. and Huberman, b.a. novelty and collective
On a technical level, a strong linear attention. Proceedings of the National Academy of
Sciences 104, 45 (nov. 2007).
correlation exists between the logarith- 10. youtube aPI;
mically transformed popularity of con- overview.html

tent at early and later times, with the re-

Gabor Szabo ( is a research scientist in
sidual noise on this transformed scale the social Computing lab at Hewlett-Packard labs, Palo
being normally distributed. Based on alto, Ca.
our understanding of this correlation, Bernardo A. huberman ( is
we presented a model to be used to pre- an HP senior Fellow and director of the social Computing
lab at Hewlett-Packard labs, Palo alto, Ca.
dict future popularity, comparing its
performance to the data we collected. © 2010 aCM 0001-0782/10/0800 $10.00

88 com municaTio nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

Call for Nominations
The ACM Doctoral Dissertation Competition

Rules of the competition Publication Rights

ACM established the Doctoral Dissertation Award Each nomination must be accompanied by an assignment to
program to recognize and encourage superior research ACM by the author of exclusive publication rights. (Copyright
and writing by doctoral candidates in computer science reverts to author if not selected for publication.)
and engineering. These awards are presented annually
at the ACM Awards Banquet. Publication
Winning dissertations will be published by Springer.
Nominations are limited to one per university or college, Selection Procedure
from any country, unless more than 10 Ph.D.’s are Dissertations will be reviewed for technical depth and
granted in one year, in which case two may be nominated. significance of the research contribution, potential impact
on theory and practice, and quality of presentation.
Deadline A committee of five individuals serving staggered five-year
Submissions must be received at ACM headquarters by terms performs an initial screening to generate a short list,
october 31, 2010 to qualify for consideration. followed by an in-depth evaluation to determine the winning
Each nominated dissertation must have been accepted The selection committee will select the winning dissertation
by the department between October 2009 and in early 2011.
September 2010. Only English language versions will
be accepted. Please send a copy of the thesis in PDF award
format to The Doctoral Dissertation Award is accompanied by a prize
of $20,000 and the Honorable Mention Award is accompanied
Sponsorship by a prize of $10,000. Financial sponsorship of the award
Each nomination shall be forwarded by the thesis advisor is provided by Google.
and must include the endorsement of the department
head. A one-page summary of the significance of the
dissertation written by the advisor must accompany for Submission Procedure
the transmittal. See
review articles
Doi:10.1145/ 1787234.1787255

Solving the memory model problem will

require an ambitious and cross-disciplinary
research direction.
By SaRiTa V. aDVe anD hanS-J. Boehm

a case for
and hardware
most pARALLeL pRogRAms today are written using threads potentially provides a performance
advantage; for example, by implicitly
and shared variables. Although there is no consensus sharing read-mostly data without the
on parallel programming models, there are a number space overhead of complete replica-
tion. The ability to pass memory refer-
of reasons why threads remain popular. Threads
IllusratIon by G Wen Va nH ee

ences among threads makes it easier to

were already widely supported by mainstream share complex data structures. Finally,
operating systems well before the dominance of shared-memory makes it far easier to
selectively parallelize application hot
multicore, largely because they are also useful for other spots without complete redesign of data
purposes. Direct hardware support for shared-memory structures.

90 communicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

e n t
e lo pm
e v
art in D

The memory model, or memory to adjacent fields in a memory location gram (written in a high-level, byte code,
consistency model, is at the heart of at the same time? Must the final value assembly, or machine language) or any
the concurrency semantics of a shared- of a location always be one of those writ- part of the language implementation
memory program or system. It defines ten to it? (including hardware) without an unam-
the set of values that a read in a pro- The memory model defines an in- biguous memory model.
gram is allowed to return, thereby de- terface between a program and any A complex memory model makes
fining the basic semantics of shared hardware or software that may trans- parallel programs difficult to write, and
variables. It answers questions such as: form that program (for example, the parallel programming difficult to teach.
Is there enough synchronization to en- compiler, the virtual machine, or any An overly constraining one may limit
sure a thread’s write will occur before dynamic optimizer). It is not possible to hardware and compiler optimization,
another’s read? Can two threads write meaningfully reason about either a pro- severely reducing performance. Since

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 91
review articles

it is an interface property, the memory this experience has made it clear that
model decision has a long-lasting im-
key insights solving the memory model problem will
pact, affecting portability and maintain- memory models, which describe the
require a significantly new and cross-
ability of programs. Thus, a hardware semantics of shared variables, are disciplinary research direction for par-
architecture committed to a strong crucial to both correct multithreaded allel computing languages, hardware,
memory model cannot later forsake it applications and the entire underlying and environments as a whole.
implementation stack. it is difficult
for a weaker model without breaking bi- to teach multithreaded programming This article discusses the path that
nary compatibility, and a new compiler without clarity on memory models. led to the current convergence in mem-
release with a weaker memory model after much prior confusion, major ory models, the fundamental shortcom-
may require rewriting source code. Fi- programming languages are converging ings it exposed, and the implications
nally, memory-model-related decisions on a model that guarantees simple for future research. The central role of
interleaving-based semantics for
for a single component must consider “data-race-free” programs and most
the memory model in parallel comput-
implications for the rest of the system. hardware vendors have committed to ing makes this article relevant to many
A processor vendor cannot guarantee support this model. computer science subdisciplines, in-
a strong hardware model if the mem- This process has exposed fundamental cluding algorithms, applications, lan-
ory system designer provides a weaker shortcomings in our languages and guages, compilers, formal methods,
model; a strong hardware model is not a hardware-software mismatch. software engineering, virtual machines,
Semantics for programs that contain
very useful to programmers using lan- data races seem fundamentally difficult, runtime systems, and hardware. For
guages and compilers that provide only but are necessary for concurrency practitioners and educators, we pro-
a weak guarantee. safety and debuggability. We call upon vide a succinct summary of the state of
software and hardware communities
Nonetheless, the central role of the to develop languages and systems
the art of this often-ignored and poorly
memory model has often been down- that enforce data-race-freedom, and understood topic. For researchers, we
played. This is partly because formally co-designed hardware that exploits and outline an ambitious, cross-disciplinary
specifying a model that balances all supports such semantics. agenda toward resolving a fundamental
desirable properties of programmabil- problem in parallel computing today—
ity, performance, and portability has environments that addressed the is- what value can a shared variable have
proven surprisingly complex. At the sue with relative clarity,40 but the most and how to implement it?
same time, informal, machine-specific widely used environments had unclear
descriptions proved mostly adequate in and questionable specifications.9,32 Sequential consistency
an era where parallel programming was Even when specifications were relative- A natural view of the execution of a
the domain of experts and achieving the ly clear, they were often violated to ob- multithreaded program operating on
highest possible performance trumped tain sufficient performance,9 tended to shared variables is as follows. Each
programmability or portability argu- be misunderstood even by experts, and step in the execution consists of choos-
ments. were difficult to teach. ing one of the threads to execute, and
In the late 1980s and 1990s, the area Since 2000, we have been involved in then performing the next step in that
received attention primarily in the hard- efforts to cleanly specify programming- thread’s execution (as dictated by the
ware community, which explored many language-level memory models, first thread’s program text, or program or-
approaches, with little consensus.2 for Java and then C++, with efforts now der). This process is repeated until the
Commercial hardware memory model under way to adopt similar models for C program as a whole terminates. Effec-
descriptions varied greatly in precision, and other languages. In the process, we tively, the execution can be viewed as
including cases of complete omission had to address issues created by hard- taking all the steps executed by each
of the topic and some reflecting ven- ware that had evolved without the ben- thread, and interleaving them in some
dors’ reluctance to make commitments efit of a clear programming model. This way. Whenever an object (that is, vari-
with unclear future implications. Al- often made it difficult to reconcile the able, field, or array element) is accessed,
though the memory model affects the need for a simple and usable program- the last value stored to the object by this
meaning of every load instruction in ev- ming model with that for adequate per- interleaved sequence is retrieved.
ery multithreaded application, it is still formance on existing hardware. For example, consider Figure 1,
sometimes relegated to the “systems Today, these languages and most which gives the core of Dekker’s mutual
programming” section of the architec- hardware vendors have published (or exclusion algorithm. The program can
ture manual. plan to publish) compatible memory be executed by interleaving the steps
Part of the challenge for hardware model specifications. Although this from the two threads in many ways. For-
architects was the lack of clear memory convergence is a dramatic improve- mally, each of these interleavings is a
models at the programming language ment over the past, it has exposed fun- total order over all the steps performed
level. It was unclear what programmers damental shortcomings in our parallel by all the threads, consistent with the
expected hardware to do. Although languages and their interplay with hard- program order of each thread. Each ac-
hardware researchers proposed ap- ware. After decades of research, it is still cess to a shared variable “sees” the last
proaches to bridge this gap,3 wide- unacceptably difficult to describe what prior value stored to that variable in the
spread adoption required consensus value a load can return without com- interleaving.
from the software community. Before promising modern safety guarantees or Figure 2 gives three possible execu-
2000, there were a few programming implementation methodologies. To us, tions that together illustrate all possible

92 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

review articles

final values of the non-shared variables Second, while sequential consisten- neous access to variables by different
r1 and r2. Although many other inter- cy may seem to be the simplest model, threads. If we require that these be used
leavings are also possible, it is not possi- it is not sufficiently simple and a much correctly, and guarantee sequential
ble that both r1 and r2 are 0 at the end of less useful programming model than consistency only if no undesirable con-
an execution; any execution must start commonly imagined. For example, it current accesses are present, we avoid
with the first statement of one of the two only makes sense to reason about in- the above issues.
threads, and the variable assigned there terleaving steps if we know what those We can make this more precise as
will later be read as one. steps are. In this case, they are typically follows. We assume the language allows
Following Lamport,26 an execution individual memory accesses, a very low- distinguishing between synchroniza-
that can be understood as such an in- level notion. Consider two threads con- tion and ordinary (non-synchronization
terleaving is referred to as sequentially currently assigning values of 100,000 or data) operations (see below). We say
consistent. Sequential consistency gives and 60,000 to the shared variable X on that two memory operations conflict if
us the simplest possible meaning for a machine that accesses memory 16 bits they access the same memory location
shared variables, but suffers from sev- at a time. The final value of X in a “se- (for example, variable or array element),
eral related flaws. quentially consistent” execution may and at least one is a write.
First, sequential consistency can be be 125,536 if the assignment of 60,000 We say that a program (on a particu-
expensive to implement. For Figure 1, occurred between the bottom and top lar input) allows a data race if it has a
a compiler might, for example, reorder half of the assignment of 100,000. At sequentially consistent execution (that
the two independent assignments in a somewhat higher level, this implies is, a program-ordered interleaving of
the red thread, since scheduling loads the meaning of even simple library op- operations of the individual threads)
early tends to hide the load latency. In erations depends on the granularity at in which two conflicting ordinary op-
addition, modern processors almost al- which the library carries out those op- erations execute “simultaneously.” For
ways use a store buffer to avoid waiting erations. our purposes, two operations execute
for stores to complete, also effectively More generally, programmers do “simultaneously” if they occur next to
reordering instructions in each thread. not reason about correctness of parallel each other in the interleaving and corre-
Both the compiler and hardware opti- code in terms of interleavings of indi- spond to different threads. Since these
mization make an outcome of r1 == 0 vidual memory accesses, and sequential operations occur adjacently in the inter-
and r2 == 0 possible, and hence may re- consistency does not prevent common leaving, we know that they could equally
sult in a non-sequentially consistent ex- sources of concurrency bugs arising well have occurred in the opposite or-
ecution. Overall, reordering any pair of from simultaneous access to the same der; there are no intervening operations
accesses, reading values from write buf- shared data (for example, data races). to enforce the order.
fers, register promotion, common sub- Even with sequential consistency, such To ensure that two conflicting ordi-
expression elimination, redundant read simultaneous accesses can remain dan- nary operations do not happen simulta-
elimination, and many other hardware gerous, and should be avoided, or at neously, they must be ordered by inter-
and compiler optimizations commonly least explicitly highlighted. Relying on vening synchronization operations. For
used in uniprocessors can potentially sequential consistency without such example, one thread must release a lock
violate sequential consistency.2 highlighting both obscures the code, after accessing a shared variable, and
There is some work on compiler and greatly complicates the implemen-
analysis to determine when such trans- tation’s job. figure 1. core of Dekker’s algorithm.
can r1 = r2 = 0?
formations are unsafe (for example,
Shasha and Snir37). Compilers, howev- Data-Race-free
er, often have little information about We can avoid both of the problems initially X=y=0
sharing between threads, making it mentioned here by observing that: Red Thread Blue Thread
expensive to forego the optimizations, ˲˲ The problematic transformations X = 1; Y = 1;
since we would have to forego them ev- (for example, reordering accesses to r1 = Y; r2 = X;
erywhere. There is also much work on unrelated variables in Figure 1) never
speculatively performing these optimi- change the meaning of single-threaded
zations in hardware, with rollback on programs, but do affect multithreaded
detection of an actual sequential con- programs (for example, by allowing figure 2. Some executions for figure 1.
sistency violation (for example, Ceze both r1 and r2 to be 0 in Figure 1).
et al.14 and Gharachorloo et al.21). How- ˲˲ These transformations are de-
execution 1 execution 2 execution 3
ever, these ideas are tied to specific im- tectable only by code that allows two
X = 1; Y = 1; X = 1;
plementation techniques (for example, threads to access the same data simul-
r1 = Y; r2 = X; Y = 1;
aggressive speculation support), and taneously in conflicting ways; for ex-
vendors have generally been unwilling ample, one thread writes the data and Y = 1; X = 1; r1 = Y;

to commit to those for the long term (es- another reads it. r2 = X; r1 = Y; r2 = X;

pecially, given non-sequentially consis- Programming languages generally // r1 == 0 // r1 == 1 // r1 == 1

tent compilers). Thus, most hardware already provide synchronization mecha- // r2 == 1 // r2 == 0 // r2 == 1

and compilers today do not provide se- nisms, such as locks, or possibly trans-
quential consistency. actional memory, for limiting simulta-

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 93
review articles

the other thread must acquire the lock continue to be allowed for ordinary ac- order requirement of sequential consis-
before its access. Thus, it is also possi- cesses—care must be taken primarily tency. For example, Sparc’s TSO guar-
ble to define data races as conflicting ac- at the explicitly identified (infrequent) antees that a thread’s memory accesses
cesses not ordered by synchronization, synchronization accesses since these will become visible to other threads in
as is done in Java. These definitions are are the only ones through which such program order, except for the case of a
essentially equivalent.1,12 optimizations and granularity consid- write followed by a read. Such models
A program that does not allow a data erations affect program outcome. Fur- additionally provide fence instructions
race is said to be data-race-free. The da- ther, synchronization-free sections of to enable programmers to explicitly im-
ta-race-free model guarantees sequen- the code appear to execute atomically pose orderings that are otherwise not
tial consistency only for data-race-free and the requirement to explicitly iden- guaranteed; for example, TSO pro-
programs.1,3 For programs that allow tify concurrent accesses makes it easier grammers may insert a fence between
data races, the model does not provide for humans and compilers to under- a thread’s write and read to ensure the
any guarantees. stand the code. (For more detail, see our execution preserves that order.
The restriction on data races is technical report.11) Such a program-orderings + fences
not onerous. In addition to locks for Data-race-free does not give the style of specification is simple, but
avoiding data races, modern program- implementation a blanket license to many subtleties make it inadequate.1,2
ming languages generally also provide perform single-threaded program op- First, this style implies that a write is
a mechanism, such as Java’s vola- timizations. In particular, optimiza- an atomic or indivisible operation that
tile variables, for declaring that cer- tions that amount to copying a shared becomes visible to all threads at once.
tain variables or fields are to be used variable to itself; such as, introducing As Figure 3 illustrates, however, hard-
for synchronization between threads. the assignment x = x, where x might ware may make writes visible to differ-
Conflicting accesses to such variables not otherwise have been written, gener- ent threads at different times through
may occur simultaneously—since they ally remain illegal. These are commonly write buffers and shared caches. Incor-
are explicitly identified as synchroniza- performed in certain contexts,9 but porating such optimizations increases
tion (vs. ordinary), they do not create a should not be. the complexity of the memory model
data race. Although data-race-free was for- specification. Thus, the full TSO speci-
To write Figure 1 correctly under da- mally proposed in 1990,3 it did not see fication, which incorporates one of the
ta-race-free, we need simply identify the widespread adoption as a formal model simplest atomicity optimizations, is
shared variables X and Y as synchroni- in industry until recently. We next de- much more involved than the simple
zation variables. This would require the scribe the evolution of industry models description here. PowerPC implements
implementation to do whatever is nec- to a convergent path centered around more aggressive forms of the optimiza-
essary to ensure sequential consistency, data-race-free, the emergent shortcom- tion, with a specification that is com-
in spite of those simultaneous accesses. ings of data-race-free, and their implica- plex and difficult to interpret even for
It would also obligate the implementa- tions for the future. experts. The x86 documentation from
tion to ensure that these synchroniza- both AMD and Intel was ambiguous on
tion accesses are performed indivisibly; industry Practice and evolution this issue; recent updates now clarify
if a 32-bit integer is used for synchroni- Hardware memory models. Most hard- the intent, but remain informal.
zation purposes, it should not be visibly ware supports relaxed models that are Second, in well-written software, a
accessed as two 16-bit halves. weaker than sequential consistency. thread usually relies on synchronization
This “sequential consistency for da- These models take an implementation- interactions to reason about the order-
ta-race-free programs” approach allevi- or performance-centric view, where the ing or visibility of memory accesses on
ates the problems discussed with pure desirable hardware optimizations drive other threads. Thus, it is usually overkill
sequential consistency. Most important the model specification.1,2,20 Typical to require that two program-ordered
hardware and compiler optimizations driving optimizations relax the program accesses always become visible to all
threads in the same order or a write ap-
figure 3. hardware may not execute atomic or indivisible writes. pears atomic to all threads regardless of
the synchronization among the threads.
Assume a fence imposes program order. Assume core 3’s and core 4’s caches have X and Y. Instead, it is sufficient to preserve order-
The two writes generate invalidations for these caches. These could reach the caches in a different ing and atomicity only among mutually
order, giving the result shown and a deduction that X’s update occurs both before and after Y’s. synchronizing threads. Some hardware
implementations attempt to exploit
initially X = y = 0 this insight, albeit often through ad hoc
core 1 core 2 core 3 core 4 techniques, thereby further complicat-
X = 1; Y = 1; r1 = X; r3 = Y; ing the memory model.
fence; fence; Third, modern processors perform
r2 = Y; r4 = X; various forms of speculation (for ex-
Can r1 = 1, r2 = 0, r3 =1, r4 = 0, violating write atomicity? ample, on branches and addresses)
which can result in subtle and complex
interactions with data and control de-
pendences, as illustrated in Figure 4.1,29

94 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

review articles

Incorporating these considerations in a The Java memory model. Java pro-

precise way adds another source of com- vided first-class support for threads
plexity to program-order + fence style with a chapter specifically devoted to
specifications. As we discuss later, pre- its memory model. Pugh showed that
cise formalization of data and control
dependences is a fundamental obstacle hardware this model was difficult to interpret
and badly broken—common compiler
to providing clean high-level memory
model specifications today.
memory model optimizations were prohibited and in
many cases the model gave ambiguous
In summary, hardware memory specifications or unexpected behavior.32 In 2000, Sun
model specifications have often been
incomplete, excessively complex, and/
have often been appointed an expert group to revise the
model through the Java community
or ambiguous enough to be misinter- incomplete, process.33 The effort was coordinated
preted even by experts. Further, since
hardware models have largely been
excessively through an open mailing list that at-
tracted a variety of participants, repre-
driven by hardware optimizations, they complex, and/or senting hardware and software and re-
have often not been well-matched to
software requirements, resulting in in- ambiguous enough searchers and practitioners.
It was quickly decided that the Java
correct code or unnecessary loss in per- to be misinterpreted memory model must provide sequen-
formance, as discussed later.
High-level language memory mod- even by experts. tial consistency for data-race-free pro-
grams, where volatile accesses (and
els. Ada was perhaps the first widely locks from synchronized methods
used high-level programming lan- and monitors) were deemed synchro-
guage to provide first-class support for nization.
shared-memory parallel programming. However, data-race-free is inad-
Although Ada’s approach to thread syn- equate for Java. Since Java is meant to
chronization was initially quite differ- be a safe and secure language, it cannot
ent from both that of the earlier Mesa allow arbitrary behavior for data races.
design and most later language designs, Specifically, Java must support untrust-
it was remarkably advanced in its treat- ed code running as part of a trusted ap-
ment of memory semantics.40 It used a plication and hence must limit damage
style similar to data-race-free, requiring done by a data race in the untrusted
legal programs to be well-synchronized; code. Unfortunately, the notions of safe-
however, it did not fully formalize the ty, security, and “limited damage” in a
notion of well-synchronized and left un- multithreaded context were not clearly
certain the behavior of such programs. defined. The challenge with defining
Subsequently, until the introduc- the Java model was to formalize these
tion of Java, mainstream programming notions in a way that minimally affected
languages did not provide first-class system flexibility.
support for threads, and shared-mem- Figure 4(b) illustrates these issues.
ory programming was mostly enabled The program has a data race and is bug-
through libraries and APIs such as Posix gy. However, Java cannot allow its reads
threads and OpenMP. Previous work de- to return values out-of-thin-air (for ex-
scribes why the approach of an add-on ample, 42) since this could clearly com-
threads library is not entirely satisfac- promise safety and security. It would,
tory.9 Without a real definition of the for example, make it impossible to
programming language in the context guarantee that similar untrusted code
of threads, it is unclear what compiler cannot return a password that it should
transformations are legal, and hence not have access to. Such a scenario ap-
what the programmer is allowed to as- pears to violate any reasonable cau-
sume. Nevertheless, the Posix threads sality expectation and no current pro-
specification indicates a model similar cessor produces it. Nevertheless, the
to data-race-free, although there are sev- memory model must formally prohibit
eral inconsistent aspects, with widely such behavior so that future specula-
varying interpretations even among ex- tive processors also avoid it.
perts participating in standards com- Prohibiting such causality violations
mittee discussions. The OpenMP mod- in a way that does not also prohibit
el is also unclear and largely based on other desired optimizations turned out
a flush instruction that is analogous to to be surprisingly difficult. Figure 5 il-
fence instructions in hardware models, lustrates an example that also appears
with related shortcomings. to violate causality, but is allowed by the

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 95
review articles

figure 4. Subtleties with (a) control and (b) data dependences. behaved executions, and the to-be com-
mitted write is not dependent on a read
that returns its value from a data race.
It is feasible for core 1 to speculate that its read of X will see 1 and speculatively write Y. Core 2
similarly writes X. both reads now return 1, creating a “self-fulfilling” speculation or a “causality
These conditions ensure that a future
loop.” Within a single core, no control dependences are violated since the speculation appears correct; data race will never be used to justify a
however, most programmers will not expect such an outcome (the code is in fact data-race-free since speculative write that could then later
no sequentially consistent execution contains a data race). Part (b) shows an analogous causal loop
justify that future data race.
with data dependences. Core 1 may speculate X is 42 (for example, using value prediction based on
previous store values) and (speculatively) write 42 into Y. Core 2 reads this and writes 42 into X, thereby A key reason for the complexity in
proving the speculation right and creating a causal loop that generates a value (42) out-of-thin-air. the Java model is that it is not opera-
Fortunately, no processor today behaves this way, but the memory model specification needs to reflect tional—an access in the future can de-
this property.
termine whether the current access is
initially X=y=0 initially X=y=0 legal. Further, many possible future
core 1 core 2 core 1 core 2
executions must be examined to deter-
r1 = X; r2 = Y; r1 = X; r2 = Y;
mine this legality. The choice of future
if (r1 == 1) if (Y == 1) Y = r1; X = r2; (well-behaved) executions also gives
Y = 1; X = 1; some surprising results. In particular,
Is r1 = r2 = 1 allowed? Is r1 = r2 = 42 allowed? as discussed in Manson29 if the code of
(a) (b) one thread is “inlined” in (concatenated
with) another thread, then the inlined
code can produce more behaviors than
the original. Thus, thread inlining is
generally illegal under the Java model
figure 5. Redundant read elimination must be allowed.
(even if there are no synchronization-
and deadlock-related considerations).
For thread 1, the compiler could eliminate the initially X=y=0 In practice, the prohibited optimiza-
redundant read of X, replacing r2=X with r2=r1.
This allows deducing that r1==r2 is always original code tions are difficult to implement and this
true, making the write of Y unconditional. Then Thread 1 Thread 2
is not a significant performance limi-
the compiler may move the write to before
r1 = X; r3 = Y;
tation. The behavior, however, is non-
the read of X since no dependence is violated. intuitive, with other implications—it
r2 = X; X = r3;
Sequential consistency would allow both the
reads of X and Y to return 1 in the new but not
if (r1 == r2) occurs because some data races in the
Y = 1; original code may no longer be data rac-
the original code. This outcome for the original
code appears to violate causality since it seems after compiler transformation es in the inlined code. This means that
to require a self-justifying speculative write of Y. Thread 1 Thread 2
It must, however, be allowed if compilers are to
when determining whether to commit a
Y = 1; r3 = Y; write early, a read in a well-behaved ex-
perform the common optimization of redundant
r1 = X; X = r3;
read elimination.
r2 = r1; ecution has more choices to return val-
if (true); ues than before (since there are fewer
Is r1 = r2 = r3 = 1 allowed? data races), resulting in new behaviors.
More generally, increasing synchro-
nization in the Java model can actually
result in new behaviors, even though
common compiler optimization of re- cuted (the execution where both reads more synchronization conventionally
dundant read elimination.29 After many of X return 0). For Figure 4(b), there is constrains possible executions. Recent-
proposals and five years of spirited de- no sequentially consistent execution ly, it has been shown that, for similar
bate, the current model was approved where Y=42 could occur. This notion of reasons, adding seemingly irrelevant
as the best compromise. This model whether a speculative write could occur reads or removing redundant reads
allows the outcome of Figure 5, but not in some well-behaved execution is the sometimes can also add new behaviors,
that of Figure 4(b). Unfortunately, this basis of causality in the Java model, and and that all these properties have more
model is very complex, was known to the definition of well-behaved is the key serious implications than previously
have some surprising behaviors, and source of complexity. thought.36 In particular, some optimiza-
has recently been shown to have a bug. The Java model tests for the legal- tions that were intended to be allowed
We provide intuition for the model be- ity of an execution by “committing” by the Java model are in fact prohibited
low and refer the reader to Manson et one or more of its memory accesses at by the current specification.
al.26 for a full description. a time—legality requires all accesses to It is unclear if current hardware or
Common to both Figure 4(b) and commit (in any order). Committing a JVMs implement the problematic op-
Figure 5 are writes that are executed write early (before its turn in program timizations noted here and therefore
earlier than they would be with sequen- order) requires it to occur in a well- violate the current Java model. Cer-
tial consistency. The examples differ behaved execution where (informally) tainly the current specification is much
in that for the speculative write in the the already committed accesses have improved over the original. Regardless,
latter (Y=1), there is some sequentially similar synchronization and data race the situation is still far from satisfac-
consistent execution where it is exe- relationships in all previously used well- tory. First, clearly, the current specifi-

96 comm unicaTio nS o f T he ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

review articles

cation does not meet its desired intent face of increasing doubt that a Java-like pushes Java’s issues with causality into
of having certain common optimizing memory model relying on sequential a much smaller and darker corner of
transformations preserve program consistency for data-race-free programs the specification; exactly the same is-
meaning. Second, its inherent com- was efficiently implementable on main- sues arise if we rewrite Figure 4(b) with
plexity and the new observations make stream architectures, at least given the C++ atomic variables and use low-
it difficult to prove the correctness of specifications available at the time. level memory_order_relaxed op-
any real system. Third, the specification Largely as a result, much of the early dis- erations. Our current solution to this
methodology is inherently fragile— cussion focused on the tension between problem is simpler, but as inelegant
small changes usually result in hard-to- the following two observations, both of as the Java one. Unlike Java, it affects a
detect unintended consequences. which we still believe to be correct given small number of fairly esoteric library
The Java model was largely guided existing hardware: calls, not all memory accesses.
by an emergent set of test cases,33 based ˲˲ A programming language model As with the Java model, we feel that
on informal code transformations that weaker than data-race-free is probably although this solution involves compro-
were or were not deemed desirable. unusable by a large fraction of the pro- mises, it is an important step forward. It
While it may be possible to fix the Java gramming community. Earlier work10 clearly establishes data-race-free as the
model, it seems undesirable that our points out, for example, that even core guarantee that every programmer
specification of multithreaded program thread library implementors often get should understand. It defines precisely
behavior would rest on such a complex confused when it comes to dealing ex- what constitutes a data race. It finally
and fragile foundation. Instead, the sec- plicitly with memory ordering issues. resolves simple questions such as: If x.a
tion entitled “Implications for Languag- Substantial effort was invested in at- and x.b are assigned simultaneously,
es” advocates a fundamental rethinking tempts to develop weaker, but compa- is that a data race? (No, unless they are
of our approach. rably simple and usable models. We do part of the same contiguous sequence
The C++ memory model. The situa- not feel these were successful. of bit-fields.) By doing so, it clearly iden-
tion in C++ was significantly different ˲˲ On some architectures, notably tifies shortcomings of existing compil-
from Java. The language itself provided on some PowerPC implementations, ers that we can now begin to remedy.
no support for threads. Nonetheless, data-race-free involves substantial im- Reconciling language and hard-
they were already in widespread use, plementation cost. (In light of modern ware models. Throughout this process,
typically with the addition of a library- (2009) specifications, the cost on oth- it repeatedly became clear that cur-
based threads implementation, such as ers, notably x86, is modest, and limited rent hardware models and supporting
pthreads22 or the corresponding Micro- largely to atomic (C++) or volatile fence instructions are often at best a
soft Windows facilities. Unfortunately (Java) store operations.) marginal match for programming lan-
the relevant specifications, for example This resulted in a compromise mem- guage memory models, particularly in
the combination of the C or C++ stan- ory model that supports data-race-free the presence of Java volatile fields
dard with the Posix standard, left sig- for nearly all of the language. However, or C++ atomic objects. It is always pos-
nificant uncertainties about the rules atomic data types also provide low-lev- sible to implement such synchroniza-
governing shared variables.9 This made el operations with explicit memory or- tion variables by mapping each one to
it unclear to compiler writers precisely dering constraints that blatantly violate a lock, and acquiring and releasing the
what they needed to implement, result- sequential consistency, even in the ab- corresponding lock around all accesses.
ed in very occasional failures for which sence of data races. The low-level op- However, this typically adds an over-
it was difficult to assign blame to any erations are easily identified and can head of hundreds of cycles to each ac-
specific piece of the implementation be easily avoided by non-expert pro- cess, particularly since the lock accesses
and, most importantly, made it difficult grammers. (They require an explicit are likely to result in coherence cache
to teach parallel programming since memory_order_ argument.) But they misses, even when only read accesses
even the experts were unable to agree on do give expert programmers a way to are involved.
some of the basic rules, such as whether write very carefully crafted, but portable, Volatile and atomic variables are
Figure 4(a) constitutes a data race. (Cor- synchronization code that approaches typically used to avoid locks for exactly
rect answer: No.) the performance of assembly code. these reasons. A typical use is a flag that
Motivated by these observations, Since C++ does not support sand- indicates a read-only data structure has
we began an effort in 2005 to develop boxed code execution, the C++ draft been lazily initialized. Since the initial-
a proper memory model for C++. The standard can and does leave the seman- ization has to happen only once, nearly
resulting effort eventually expanded to tics of a program with data races com- all accesses simply read the atomic/
include the definition of atomic (syn- pletely undefined, effectively making it volatile flag and avoid lock acquisi-
chronization, analogous to Java vola- erroneous to write such programs. As tions. Acquiring a lock to access the flag
tile) operations, and the threads API we point out in Boehm and Adve12 this defeats the purpose.
itself. It is part of the current Commit- has a number of (mostly) performance- On hardware that relaxes write ato-
tee Draft24 for the next C++ revision. The related advantages, and better reflects micity (see Figure 3), however, it is often
next C standard is expected to contain existing compiler implementations. unclear that more efficient mappings
a very similar memory model, with very In addition to the issues raised (than the use of locks) are possible; even
similar atomic operations. in the “Lessons Learned” section, it the fully fenced implementation may
This development took place in the should be noted that this really only not be sequentially consistent. Even

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 97
review articles

on other hardware, there are apparent surprising results later during program nating the debugging issues associated
mismatches, most probably caused by execution, possibly long after the data with data races.
the lack of a well-understood program- race has resulted in corrupted data. Al- Unfortunately, these both take us to
ming language model when the hard- though the usual immediate result of active research areas, with no clear off-
ware was designed. On x86, it is almost a data race is that an unexpected, and the-shelf solutions. We discuss some
sufficient to map synchronization loads perhaps incomplete value is read, or possible approaches here.
and stores directly to ordinary load and that an inconsistent value is written,
store instructions. The hardware pro- we point out in prior work12 that other implications for Languages
vides sufficient guarantees to ensure results, such as wild branches, are also In spite of the dramatic convergence
that ordinary memory operations are possible as a result of compiler opti- in the debate on memory models,
not visibly reordered with synchroniza- mizations that mistakenly assume the the state of the art imposes a difficult
tion operations. However it fails to pre- absence of data races. Since such races choice: a language that supposedly has
vent reordering of a synchronization are difficult to reproduce, the root cause strong safety and security properties,
store followed by a synchronization of such misbehavior is often difficult to but no clear definition of what value a
load; thus this implementation does identify, and such bugs may easily take shared-memory read may return (the
not prevent the incorrect outcome for weeks to track down. Many tools to aid Java case), versus a language with clear
Figure 1. such debugging (for example, CHESS30 semantics, but that requires abandon-
This may be addressed by translat- and RaceFuzzer35) also assume sequen- ing security properties promised by lan-
ing a synchronization store to an ordi- tial consistency, somewhat limiting guages such as Java (the C++ case). Un-
nary store instruction followed by an ex- their utility. fortunately, modern software needs to
pensive fence. The sole purpose of this Synchronization variable perfor- be both parallel and secure, and requir-
fence is to prevent reordering of the syn- mance on current hardware. As dis- ing a choice between the two should
chronization store with a subsequent cussed, ensuring sequential consis- not be acceptable.
synchronization load. In practice, such tency in the presence of Java volatile A pessimistic view would be to
a synchronization load is unlikely to fol- or C++ atomic on current hardware abandon shared-memory altogether.
low closely enough (Dekker’s algorithm can be expensive. As a result, both C++, However, the intrinsic advantages of a
is not commonly used) to really con- and to a lesser extent Java, have had to global address space are, at least anec-
strain the hardware. But the only avail- provide less expensive alternatives that dotally, supported by the widespread
able fence instruction constrains all greatly complicate the model for ex- use of threads despite the inherent
memory reordering around it, includ- perts trying to use them. challenges. We believe the fault lies not
ing that involving ordinary data access- Untrusted code. There is no way to in the global address space paradigm,
es, and thus overly constrains the hard- ensure data-race-freedom in untrusted but in the use of undisciplined or “wild
ware. A better solution would involve code. Thus, this model is insufficient shared-memory,” permitted by current
distinguishing between two flavors of for languages like Java. systems.
loads and stores (ordinary and synchro- An unequivocal lesson from our ex- Data-race-free was a first attempt to
nization), roughly along the lines of periences is that for programs with data formalize a shared-memory discipline
Itanium’s ld.acq and st.rel.23 This, races, it is very difficult to define seman- via a memory model. It proved inad-
however, requires a change to the in- tics that are easy to understand and yet equate because the responsibility for
struction set architecture, usually a dif- retain desired system flexibility. While following this discipline was left to the
ficult proposition. the Java memory model came a long programmer. Further, data-race-free
We suspect the current situation way, its complexity, and subsequent by itself is, arguably, insufficient as a
makes the fence instruction more ex- discoveries of its surprising behaviors, discipline for writing correct, easily
pensive than necessary, in turn moti- are far from satisfying. Unfortunately, debuggable, and maintainable shared-
vating additional language-level com- we know of no alternative specification memory code; for example, it does not
plexity such as C++ low-level atomics or that is sufficiently simple to be consid- completely eliminate atomicity viola-
lazySet() in Java. ered practical. Second, rules to weaken tions or non-deterministic behavior.
the data-race-free guarantee to better Moving forward, we believe a critical
Lessons Learned match current hardware, as through research agenda to enable “parallelism
Data-race-free provides a simple and C++ low-level atomics, are also more for the masses” is to develop and pro-
consistent model for threads and complex than we would like. mote disciplined shared-memory models
shared variables. We are convinced it The only clear path to improvement that:
is the best model today to target during here seems to be to eliminate the need ˲˲ are simple enough to be easily teach-
initial software development. Unfortu- for going beyond the data-race-free able to undergraduates; that is, mini-
nately, its lack of any guarantees in the guarantee by: mally provide sequential consistency to
presence of data races and mismatch ˲˲ Eliminating the performance moti- programs that obey the required disci-
with current hardware implies three vations for going beyond it, and pline;
significant weaknesses: ˲˲ Ensuring that data races are never ˲˲ enable the enforcement of the disci-
Debugging. Accidental introduction actually executed at runtime, thus both pline; that is, violations of the discipline
of a data race results in “undefined be- avoiding the need to specify their be- should not have undefined or horren-
havior,” which often takes the form of havior and greatly simplifying or elimi- dously complex semantics, but should

98 com municaTio nS o f T h e acm | AU g U ST 201 0 | VO l . 5 3 | NO. 8

review articles

be caught and returned back to the pro- written for performance have determin-
grammer as illegal; istic outcomes and can be expressed
˲˲ are general-purpose enough to ex- with deterministic algorithms. Writing
press important parallel algorithms and such programs using a deterministic
patterns; and
˲˲ enable high and scalable perfor- Data-race-free environment allows reasoning with se-
quential semantics (a memory model
mance. provides a simple much simpler than sequential consis-

and consistent
Many previous programmer-produc- tency with threads).
tivity-driven efforts have sought to raise A valuable discipline, therefore, is to
the level of abstraction with threads; for
example, Cilk,19 TBB,25 OpenMP,39 the
model for threads provide a guarantee of determinism by
default; when non-determinism is in-
recent HPCS languages,28 other high- and shared herently required, it should be request-
level libraries, frameworks, and APIs
such as java.util.concurrent and the C++
variables. We ed explicitly and should not interfere
with the deterministic guarantees for
boost libraries, as well as more domain- are convinced the remaining program.7 There is much
specific ones. While these solutions go
a long way toward easing the pain of that it is the best prior work in deterministic data paral-
lel, functional, and actor languages. Our
orchestrating parallelism, our memory- model today to focus is on general-purpose efforts that
models driven argument is deeper—we
argue that, at least so far, it is not pos- target during continue use of widespread program-
ming practices; for example, global
sible to provide reasonable semantics
for a language that allows data races,
initial software address space, imperative languages,
object-oriented programming, and
an arguably more fundamental prob- development. complex, pointer-based data structures.
lem. In fact, all of these examples either Language-based approaches with
provide unclear models or suffer from such goals include Jade34 and the recent
the same limitations as C++/Java. These Deterministic Parallel Java (DPJ).8 In
approaches, therefore, do not meet our particular, DPJ proposes a region-based
enforcement requirement. Similarly, type and effect system for determinis-
transactional memory provides a high- tic-by-default semantics—“regions”
level mechanism for atomicity, but the name disjoint partitions of the heap
memory model in the presence of non- and per-method effect annotations
transactional code faces the same is- summarize which regions are read and
sues as described here.38 written by each method. Coupled with
At the heart of our agenda of disci- a disciplined parallel control structure,
plined models are the questions: What the compiler can easily use the effect
is the appropriate discipline? How to summaries to ensure that there are no
enforce it? unordered conflicting accesses and
A near-term transition path is to the program is deterministic. Recent
continue with data-race-free and focus results show that DPJ is applicable to a
research on its enforcement. The ideal range of applications and complex data
solution is for the language to elimi- structures and provides performance
nate data races by design (for example, comparable to threads code.8
Boyapati13); however, our semantics dif- There has also been much recent
ficulties are avoided even with dynamic progress in runtime methods for deter-
techniques (for example, Elmas et al.,17 minism.4,5,16,31
Flanagan and Freund,18 or Lucia et al.27) Both language and runtime ap-
that replace all data races with excep- proaches have pros and cons and still
tions. (There are other dynamic data require research before mainstream
race detection techniques, primarily for adoption. A language-based approach
debugging, but they do not guarantee must establish that it is expressive
complete accuracy, as required here.) enough and does not incur undue pro-
A longer-term direction concerns grammer burden. For the former, the
both the appropriate discipline and its new techniques are promising, but the
enforcement. A fundamental challenge jury is still out. For the latter, DPJ is at-
in debugging, testing, and reasoning tempting to alleviate the burden by us-
about threaded programs arises from ing a familiar base language (currently
their inherent non-determinism—an Java) and providing semiautomatic
execution may exhibit one of many pos- tools to infer the required programmer
sible interleavings of its memory ac- annotations.41 Further, language anno-
cesses. In contrast, many applications tations such as DPJ’s read/write effect

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T he acm 99
review articles

summaries are valuable documenta- ware technology makes this a particu-

tion in their own right—they promote larly opportune time to embark on such
lifetime benefits for modularity and an agenda. Power and complexity con-
maintainability, arguably compensat- straints have led industry to bet that fu-
ing for upfront programmer effort. Fi-
nally, a static approach benefits from no We believe that ture single-chip performance increases
will largely come from increasing num-
overhead or surprises at runtime.
In contrast, the purely runtime ap-
hardware that bers of cores. Today’s hardware cache-
coherent multicore designs, however,
proaches impose less burden on the takes advantage are optimized for few cores—power-ef-
programmer, but a disadvantage is
that the overheads in some cases may
of the emerging ficient, performance scaling to several
hundreds or a thousand cores without
still be too high. Further, inherently, a disciplined software consideration of software requirements
runtime approach does not provide the
guarantees of a static approach before
programming will be difficult.
We view this challenge as an op-
shipping and is susceptible to surpris- models is likely to portunity to not only resolve the prob-
es in the field.
We are optimistic that the recent be more efficient lems discussed in this article, but in
doing so, we expect to build more ef-
approaches have opened up many than a software fective hardware and software. First,
promising new avenues for disciplined
shared-memory that can overcome the oblivious approach. we believe that hardware that takes
advantage of the emerging disciplined
problems described here. It is likely software programming models is likely
that a final solution will consist of a ju- to be more efficient than a software-
dicious combination of language and oblivious approach. This observation
runtime features, and will derive from a already underlies the work on relaxed
rich line of future research. hardware consistency models—we
hope the difference this time around
implications for hardware will be that the software and hardware
As discussed earlier, current hard- models will evolve together rather than
ware memory models are an imperfect as retrofits for each other, providing
match for even current software (data- more effective solutions. Second, hard-
race-free) memory models. ISA changes ware research to support the emerging
to identify individual loads and stores disciplined software models is also
as synchronization can alleviate some likely to be critical. Hardware support
short-term problems. An established can be used for efficient enforcement
ISA, however, is difficult to change, es- of the required discipline when static
pecially when existing code works most- approaches fall short; for example,
ly adequately and there is not enough through directly detecting violations of
experience to document the benefits of the discipline and/or through effective
the change. strategies to sandbox untrusted code.
Academic researchers have taken Along these lines, we have recently
an alternate path that uses complex begun the DeNovo hardware project at
mechanisms (for example, Blundell et Illinois15 in concert with DPJ. We are
al.6) to speculatively remove the con- exploiting DPJ-like region and effect
straints imposed by fences, rolling annotations to design more power-
back the speculation when it is detect- and complexity-efficient, software-
ed that the constraints were actually driven com munication and coher-
needed. While these techniques have ence protocols and task scheduling
been shown to work well, they come mechanisms. We also plan to provide
at an implementation cost and do not hardware and runtime support to deal
directly confront the root of the prob- with cases where DPJ’s static informa-
lem of mismatched hardware/software tion and analysis might fall short. As
views of concurrency semantics. such co-designed models emerge,
Taking a longer-term perspective, ultimately, we expect them to drive the
we believe a more fundamental solu- future hardware-software interface in-
tion to the problem will emerge with cluding the ISA.
a co-designed approach, where future
multicore hardware research evolves conclusion
in concert with the software models re- This article gives a perspective based
search discussed in “Implications for on work collectively spanning approxi-
Languages.” The current state of hard- mately 30 years. We have been repeat-

100 co mm unicaT io nS o f T h e ac m | AU g U ST 201 0 | VO l . 5 3 | NO. 8

review articles

edly surprised at how difficult it is to el; Lawrence Crowl, Paul McKenney, Implementation, 2009.
19. Frigo, M., leiserson, C.e. and randall, K.H. the
formalize the seemingly simple and Clark Nelson, and Herb Sutter, for the implementation of the Cilk-5 multithreaded language.
fundamental property of “what value a C++ model; and Vikram Adve, Rob Boc- In Proceedings of the ACM Conference on Programming
Language Design and Implementation, 1998, 212–223.
read should return in a multithreaded chino, and Marc Snir for ongoing work 20. Gharachorloo, K. Memory consistency models for
program.” Sequential consistency for on disciplined programming models shared-memory multiprocessors. Ph.d. thesis, 1996,
stanford university, stanford, Ca.
data-race-free programs appears to and their enforcement. We thank Doug 21. Gharachorloo, K., Gupta, a. and Hennessy, J. two
be the best we can do at present, but it Lea for continuous encouragement to techniques to enhance the performance of memory
consistency models. In Proceedings of the Intl. Conf.
is insufficient. The inability to define push the envelope. Finally, we thank Vi- on Parallel Processing, 1991, I355–I364.
reasonable semantics for programs kram Adve, Rob Bocchino, Nick Carter, 22. Ieee and the open Group. IEEE Standard 1003.1-
2001. 2001.
with data races is not just a theoretical Lawrence Crowl, Mark Hill, Doug Lea, 23. Intel. Intel Itanium Architecture: Software
shortcoming, but a fundamental hole Jeremy Manson, Paul McKenney, Bratin Developer’s Manual, Jan 2006.
24. Iso/IeC JtC1/sC22/WG21. Iso/IeC 14882,
in the foundation of our languages Saha, and Rob Schreiber for comments Programming languages - C++ (final committee draft)
and systems. It is well accepted that on earlier drafts. papers/2010/n3092.pdf.
most shipped software has bugs and it Sarita Adve is currently funded by 25. reinders, J. Intel Threading Building Blocks: Outfitting
C++ for Multi-core Parallelism. o’reilly, 2007.
is likely that much commercial multi- Intel and Microsoft through the Illinois 26. lamport, l. How to make a multiprocessor computer
threaded software has data races. De- Universal Parallel Computing Research that correctly executes multiprocess programs. IEEE
Transactions on Computers C-28, 9 (1979), 690–691.
bugging tools and safe languages that Center. 27. lucia, b. et al. Conflict exceptions: simplifying
seek to sandbox untrusted code must concurrent language semantics with precise
hardware exceptions for data-races. In Proceedings
deal with such races, and must be given of the International Symposium on Computer
semantics that reasonable computer 1. adve, s.V. Designing Memory Consistency Models
Architecture, 2010.
28. lusk, e. and yelick, e. languages for high-productivity
science graduates and developers can for Shared-Memory Multiprocessors. Phd thesis.
computing: the darPa HPCs language project.
university of Wisconsin-Madison, 1993.
understand. 2. adve, s.V. and Gharachorloo, K. shared memory
Parallel Processing Letters 17, 1 (2007) 89–102.
29. Manson, J., Pugh, W. and adve, s.V. the Java memory
We believe it is time to rethink how consistency models: a tutorial. IEEE Computer 29, 12
model. In Proceedings of the Symp. on Principles of
(1996), 66–76.
we design our languages and systems. Programming Languages, 2005.
3. adve, s.V. and Hill, M.d. Weak ordering—a new
30. Musuvathi, M. and Qadeer, s. Iterative context
Minimally, the system, and preferably definition. In Proceedings of the 17th Intl. Symp.
bounding for systematic testing of multithreaded
Computer Architecture, 1990, 2–14.
the language, must enforce the absence programs. In Proceedings of the ACM Conference on
4. allen, M.d., sridharan, s. and sohi, G.s. serialization
Programming Language Design and Implementation,
of data races. A longer term, potentially sets: a dynamic dependence-based parallel execution
2007, 446–455.
model. In Proceedings of the Symp. on Principles and
31. olszewski, M., ansel, J., and amarasinghe, s. Kendo:
more rewarding strategy is to rethink Practice of Parallel Programming, 2009.
efficient deterministic multithreading in software.
5. berger, e.d., yang, t., liu, t. and novark, G. Grace: safe
higher-level disciplines that make it multithreaded programming for C/C++. In Proceedings
In Proceedings Intl. Conf. on Architectural Support
for Programming Languages and Operating Systems.
much easier to write parallel programs of the Intl. Conf. on Object-Oriented Programming,
Mar. 2009.
Systems, Languages, and Applications, 2009.
and that can be enforced by our languag- 6. blundell, C, Martin, M.M.K. and Wenisch, t. Invisifence:
32. Pugh, W. the Java memory model is fatally flawed.
Concurrency-Practice and Experience 12, 6 (2000),
es and systems. We also believe some Performance-transparent memory ordering in
conventional multiprocessors. In Proceedings of the
of the messiness of memory models to- Intl. Symp. on Computer Architecture, 2009.
33. Pugh, W. and the Jsr 133 expert Group. the Java
memory model (July 2009); http://www.cs.umd.
day could have been averted with closer 7. bocchino, r. et al. Parallel programming must be
deterministic by default. In Proceedings of the 1st
cooperation between hardware and Workshop on Hot Topics in Parallelism, 2009.
34. rinard, M.C. and lam, M.s. the design,
implementation, and evaluation of Jade. ACM
software. As we move toward more dis- 8. bocchino. r. et al. a type and effect system for
Transactions on Programming Languages and
deterministic Parallel Java. In Proceedings of the
ciplined programming models, there is Intl. Conf. on Object-Oriented Programming, Systems,
Systems 20, 3 (1998), 483–545.
35. sen, K. race directed random testing of concurrent
also a new opportunity for a hardware/ Languages, and Applications, 2009.
programs. In Conf. on Programming Language Design
9. boehm, H.-J.. threads cannot be implemented as a
software co-designed approach that re- and Implementation, 2008.
library. In Proceedings of the Conf. on Programming
36. sevcik, J. and aspinall, d. on validity of program
thinks the hardware/software interface Language Design and Implementation, 2005.
transformations in the Java memory model. In
10. boehm, H.-J. reordering constraints for pthread-style
and the hardware implementations of Proceedings of the European Conference on Object-
locks. In Proceedings of the 12th Symp. Principles and
Oriented Programming, 2008, 27–51.
all concurrency mechanisms. These Practice of Parallel Programming, 2007, 173–182.
37. shasha, d and snir, M. efficient and correct execution
11. boehm, H.-J. threads basics. July 2009; http://www.
of parallel programs that share memory. ACM
views embody a rich research agenda
Transactions on Programming Languages and
12. boehm, H.-J. and adve, s.V. Foundations of the
that will need the involvement of many C++ concurrency memory model. In Proceedings
Systems 10, 2 (apr. 1998), 282–312.
38. shpeisman, t. et al. enforcing isolation and ordering
computer science sub-disciplines, in- of the Conf. on Programming Language Design and
in stM. In Proceedings of the ACM Conference on
Implementation, 2008, 68–78.
cluding languages, compilers, formal 13. boyapati, C., lee, r., and rinard, M. ownership types
Programming Language Design and Implementation,
methods, software engineering, algo- for safe programming: Preventing data races and
39. the openMP arb. openMP application programming
deadlocks. In Proceedings of the Intl. Conf. on Object-
rithms, runtime systems, and hardware. Oriented Programming, Systems, Languages, and
interface: Version 3.0. May 2008; http://www.openmp.
Applications, 2002.
40. united states department of defense. Reference
14. Ceze, l et al. bulksC: bulk enforcement of sequential
acknowledgments Consistency. In Proceedings of the Intl. Symp. on
Manual for the Ada Programming Language: ANSI/
MIL-STD-1815A-1983 Standard 1003.1-2001.
This article is deeply influenced by a Computer Architecture, 2007.
springer, 1983.
15. Choi, b. et al. denovo: rethinking Hardware for
collective 30 years of collaborations and disciplined Parallelism. In Proceedings of the 2nd
41. Vakilian, M. et al. Inferring method effect summaries
for nested heap regions. In Proceedings of the 24th
discussions with more colleagues than Workshop on Hot Topics in Parallelism, June 2010.
Intl. Conf. on Automated Software Engineering, 2009.
16. devietti, J. et al. dMP: deterministic shared memory
can be named here. We would particu- processing. In Proceedings of the Intl. Conf. on
larly like to acknowledge the contribu- Architectural Support for Programming Languages Sarita V. Adve ( is a professor in the
and Operating Systems (Mar. 2009), 85–96. department of Computer science at the university of
tions of Mark Hill for co-developing 17. elmas, t., Qadeer, s. and tasiran, s. Goldilocks: a race Illinois at urbana-Champaign.
the data-race-free approach and other and transaction-aware Java runtime. In Proceedings
of the ACM Conference on Programming Language hans-J. Boehm ( is a member of
foundational work; Kourosh Ghara- Design and Implementation, 2007, 245–255. Hewlett-Packard’s exascale Computing lab, Palo alto, Ca.
18. Flanagan, C. and Freund, s. Fasttrack: efficient and
chorloo for hardware models; Jeremy precise dynamic race detection. In Proceedings of
Manson and Bill Pugh for the Java mod- the Conf. on Programming Language Design and © 2010 aCM 0001-0782/10/0800 $10.00

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T h e acm 101


The ACM Magazine for Students

XRDS delivers the tools, resources, knowledge, and connections

that computer science students need to succeed
in their academic and professional careers!

The All-New XRDS: Crossroads is the official

magazine for ACM student members featuring:
� Breaking ideas from top researchers and PhD students
� Career advice from professors, HR managers, entrepreneurs, and others
� Interviews and profiles of the biggest names in the field
� First-hand stories from interns at internationally acclaimed research labs
� Up-to-date information on the latest conferences, contests, and submission
deadlines for grants, scholarships, fellowships, and more!

Also available
The All-New is the new online hub of XRDS
magazine where you can read the latest news
and event announcements, comment on articles,
plus share what’s happening at your ACM chapter,
and more. Get involved by visiting today!
research highlights
P. 104 P. 105
Technical The Emergence of
attacks Target Cross Channel Scripting
Web Server Logic By Hristo Bojinov, Elie Bursztein, and Dan Boneh
and Prey on XcS
By Helen Wang

P. 114 P. 115
Technical Reasoning About the Unknown
Perspective in Static Analysis
Sound and Precise By Isil Dillig, Thomas Dillig, and Alex Aiken
Program analysis
By Fritz Henglein

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T h e acm 103

research highlights
Doi:10.1145/ 1787234.1 78 72 5 6

Technical Perspective
attacks Target Web
Server Logic and Prey
on XcS Weaknesses
By Helen Wang

A syst em I ssecure only if the entire sys- ample, several NAS appliances both sumption that the global state (such as
tem is secure. expose a Web interface for system the file system) is shared with others.
While this may sound obvious, management and allow file uploading To make things worse, if any of the
achieving total security throughout a through FTP, SMB, or a P2P service. services has a security hole, all servic-
system is rarely trivial when you con- Because the file system is shared be- es can be affected. Here, the authors
sider many real-world systems are con- tween the Web server and these other talk about reverse XCS vulnerabilities
stantly evolving. In the following pa- file uploading services, an attacker can where a Web server’s XSS vulnerability
per, “The Emergence of Cross Channel upload a file with a specially crafted can cause private data from other ser-
Scripting” (XCS), Hristo Bojinov, Elie file name that contains a malicious vices to be leaked.
Bursztein, and Dan Boneh highlight JavaScript. Later, when an administra- Based on these keen observations,
this problem. tor of the device configures the device the authors uncovered real-world XCS
The systems examined in the paper through the Web interface, the mali- vulnerabilities in a slew of embedded
are embedded Web servers that have cious file is loaded in the address bar systems, including several NAS appli-
become prevalent for system manage- and the malicious JavaScript executes. ances, lights-out management sys-
ment and configurations of consumer For a device with such co-location tems (LOM), and photo frames. The
electronic devices like digital photo of services, even if each of the services authors also explore cellphone-based
frames, wireless routers, and network- is secure on its own, running them to- XCS where the Palm Pre is vulnerable
attached storage (NAS) appliances. gether creates new security holes be- to an XCS attack that injects its payload
Web applications have long suf- cause each service has not had the as- through a calendar title or content.
fered cross site scripting (XSS) vul- The authors note some initial di-
nerabilities. XSS vulnerabilities of a rections for defending against XCS,
Web application allows an attacker to The cross site mostly along the lines of preventing
inject attacking scripts into the Web information leakage by restricting the
application and then the attacking scripting (XSS) destinations of the outgoing network
scripts execute with the privilege of problem is amplified messages. This is indeed an interest-
the Web site on browsers. A particu- ing direction. The heart of the prob-
larly damaging type of XSS is persis- in the embedded lem here is to construct such a policy.
tent XSS in which the injected script Web server setting To me, a more fundamental solution
persists beyond a browsing session would be to address the root cause of
and across different browsing us- where servers the problem and eliminate any state
ers. For example, the infamous Samy co-locate with other sharing of independently designed ser-
worm exploits an XSS vulnerability in vices. If cross-service sharing is need- and the attacker (Samy, services, sharing ed, the service designer must enable
in this case) injected a script as part of the underlying device such sharing explicitly.
Samy’s (persistent) user profile. Peo- By highlighting the vulnerabilities
ple who viewed Samy’s profile found resources, like that still exist, this paper offers a valu-
their profiles infected as did the view- the file system. able lesson in—and interesting read
ers of their profiles, and so on. about—system security.
The XSS problem is amplified in the
embedded Web server setting where helen Wang ( is a senior
Web servers co-locate with other ser- researcher leading the security and privacy research
group at Microsoft research, redmond, Wa.
vices, sharing the underlying device
resources, like the file system. For ex- © 2010 aCM 0001-0782/10/0800 $10.00

104 comm unicaTio nS o f Th e ac m | AU gU ST 201 0 | VO l . 5 3 | NO. 8

Doi:10.1145/ 1787234 . 1 78 72 5 7

The Emergence of
Cross Channel Scripting
By Hristo Bojinov, Elie Bursztein, and Dan Boneh

abstract figure 1. embedded Web servers will soon outnumber generic Web
Lightweight, embedded Web servers are soon about to servers on the internet.
outnumber regular Internet Web servers. They reside in
devices entrusted with personal and corporate data, and Growth
are typically used for configuration and management. We 300
reveal a series of attacks on consumer and small office elec- Internet Embedded (NAS and photo frame only)

tronics, ranging from networked storage to digital photo 225

frames. The attacks target Web server logic and are based

on a new type of vulnerability that we call cross channel
scripting (XCS). XCS is a sophisticated form of cross site 150
scripting (XSS) in which the attack injection and execution
are carried out via different protocols.

1. inTRoDucTion 0
Current consumer electronic devices often ship with an Data :
2008 2009 2010 2011 2012 2013
embedded Web server used for system management. The -Parks associates
benefits of providing a Web-based user interface are two-
fold: first, the user does not need to learn a complicated
command-line language, and second, the vendor does not
need to ship client-side software. Instead the user interacts HTTP response. This script can then take over the page and
with the device through a familiar browser UI. Market data perform arbitrary actions on behalf of the attacker. A Type
confirms the success of the browser-based device man- 2 (persistent) XSS enables the attacker to inject a malicious
agement paradigm: even when considering only network- script into persistent storage at the victim site. When an
attached storage (NAS) and digital photo frame products, unsuspecting user views a page that contains the script, the
embedded Web servers are on track to surpass in number script can take over the page. For example, Type 2 XSS can
general-purpose Web servers on the Internet (Figure 1). affect message boards; an attacker can post a message con-
While browser-based device management is a cost- taining a script that is later executed by the browser of every
effective and convenient approach, it can introduce con- user that happens to view the attacker’s post. A recent exam-
siderable security risk due to the large number of potential ple of such an attack is the XSS Twitter worm that struck in
vulnerabilities in a weak Web application. Moreover, secur- the middle of April 2009.13
ing Web applications on a consumer electronics device can cross channel scripting: Many of the embedded devices we
be difficult due to the large number of supported network examined were vulnerable to a type of persistent XSS that
protocols and the interactions between them. For example, we call cross channel scripting, or XCS. In an XCS attack a
a user might upload a file to a network storage device by non-Web channel, such as SNMP or FTP, is used to inject
using the SMB protocol, manage its permissions through a persistent XSS exploit which is activated when the user
the Web interface, and eventually share it with his friends connects to the Web interface. For example, several NAS
through FTP. The overall opacity of both the software that devices we examined allow an attacker to upload a file with
runs on embedded systems and any state they store further an almost arbitrary filename via SMB. The attacker takes
adds to the security risk, as it effectively prevents security advantage of this lack of restrictions and crafts a filename
products from scanning such systems and reporting on vul- that contains a malicious script. When the NAS adminis-
nerabilities or attacks in progress. trator views the NAS contents through the Web interface,
In this complex environment, it is not surprising that many the device happily sends an HTTP response to the admin’s
embedded devices are vulnerable to Web attacks. In fact, all browser containing a list of file names including the mali-
the 23 devices we evaluated3 were vulnerable to several types cious filename, which is then interpreted as a script by the
of Web attacks, including cross site scripting (XSS),5 cross
site request forgeries (CSRF),2, 12 and many others. The original version of this paper appeared in the Proceed-
Recall that in a Type 1 (reflected) XSS attack, the user fol- ings of the 16th ACM Conference on Computer and Commu-
lows a malicious link to a victim site. A vulnerability in the nications Security (Chicago, IL, Nov. 9–13, 2009), 420–431.
site causes an attack script to be embedded into the resulting

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aTion S of T h e acm 105

research highlights

browser. The script executes on the admin’s browser giving figure 2. overview of the XcS attack.
the attacker full control of the admin session. In Section 3,
we present the most interesting XCS attacks we discovered. Alternate
We also found a related class of attacks in which a Web Web
vulnerability is used to attack a non-Web channel. We refer
to this as a reverse Xcs vulnerability. We give examples in
Section 4. Attacker Device User
XCS and reverse XCS are more likely to affect embedded
devices than traditional Web sites because these devices
often provide a number of services (e.g., Web, SNMP, NFS,
P2P) which are cobbled together from generic components. Injection Reflection
The interaction between the components may not be com-
pletely analyzed, leading to an XCS vulnerability. In contrast,
many Internet Web sites only provide a Web interface and
hence are less likely to be affected by XCS. Interestingly, code on the server. In the second step the malicious content
large Web sites, such as Facebook and Twitter, provide non- is sent to the victim by the Web application. As soon as the
Web cloud APIs for third-party applications which present victim accesses the malicious content via her browser, it is
XCS opportunities, as discussed in Section 5. executed with her permissions. While an XCS exploit is a
Detecting an XCS or reverse XCS vulnerability can be dif- form of persistent (Type 2) XSS, we argue that a distinction
ficult because these attacks abuse the interaction between between the two should be made for two reasons.
the Web interface and an alternate communication chan- First, XCS vulnerabilities are harder to detect since they
nel. Simply inspecting the Web application code and the involve multiple protocols. Static analyzers used to detect
other service code is not enough to detect the vulnerability. XSS (such as Pixy8) do not detect XCS because their taint
The Web application and the other service, such as an FTP analysis assumes that the user input is stored in global vari-
server, can be completely secure in isolation and become ables. Using taint analysis to detect XCS is difficult because
vulnerable only when used in conjunction. of the large number of possible tainted data sources. For
An XCS exploit can be used to carry out a variety of attacks example, for PHP, in addition to the obvious file( ), and other
including file related functions, many other protocol specific func-
tions need to be considered. This includes every SNMP func-
• exfiltrating sensitive data, such as NAS-protected files tion that reads data, such as snmpget( ), ftp_nlist( ) which
or a user’s keystrokes lists an FTP directory, and of course database functions that
• Redirecting the user to a drive-by-download site10 or a return a result set, such as mysql_ fetch_object( ). Even if all
phishing site the functions were correctly enumerated, the number of
• exploiting the user’s Ip address for DDoS9 or for false alarms would be overwhelming. Current research on
proxying the attacker’s traffic static analysis shows promise in improving the situation in
the near future.1
On consumer electronic devices, an XCS exploit can be a Second, XSS defenses that sanitize data at input time
stepping stone toward a larger attack on the user’s LAN that are unlikely to protect against XCS. These mechanisms are
aims to assimilate home machines into a botnet4 or to break mostly applied to data acquired from Web traffic, while
into the user’s corporate network. For instance, a reverse in XCS the attack vector is presented through a non-Web
XCS can be used to reboot a switch and therefore shutdown channel which is unlikely to sanitize for Web exploits. This
an entire LAN. difficulty of detecting and preventing XCS vulnerabilities
organization: In the remainder of this article, we define XCS explains why in every embedded device we examined we
in more detail, then present real-world XCS attacks and dis- were able to uncover XCS problems.
cuss their impact. Afterward, we introduce the concept of
reverse XCS and present examples from practice. We also 3. ReaL-WoRLD XcS
demonstrate that reverse XCS is a general powerful attack by We present four case studies illustrating different types of
showing how Restful API-based RXCS can be used to attack real-world XCS vulnerabilities in popular embedded devices
very popular Web sites. Finally, we briefly cover defenses and mobile phones. The first example uses file transfer pro-
against XCS and refer the readers to our original paper for a tocols such as FTP to inject a script into persistent storage,
more detailed discussion. the second uses P2P networks, and the third injects a script
into log files. The final example uses the calendar protocol
2. cRoSS channeL ScRiPTinG in a nuTSheLL to infiltrate the Palm Pre.
An XCS attack is an attack where a non-Web channel is used
to inject a script into Web content running in a different 3.1. a two-stage XcS exploit
security context. NAS appliances are lightweight servers that provide data
An XCS attack comprises two steps, as shown in Figure 2. storage services to other devices on the network. The low-
In the first step the attacker uses a non-Web channel such end NAS market is very active with over 50 vendors offering
as an FTP or SNMP service to store malicious JavaScript products, including Apple, Buffalo, Dell, Lacie, and Linksys.

106 com municaTio nS o f T h e ac m | au Gust 2 0 10 | Vol. 53 | no. 8

Since NAS appliances need to be managed over the net- figure 3. Result from the two-stage XcS attack on a naS appliance.
work, most vendors build a Web server into them for this
purpose. NAS devices inherently support multiple inter-
faces and thus are primary candidates for XCS exploits.
Moreover, the market pressure to quickly add new features
(e.g., P2P file downloads and RSS flux) gives ample oppor-
tunities for implementation oversights that will turn into
XCS exploits. We evaluated five NAS devices from different,
well-known vendors and found multiple XCS vulnerabili-
ties in all of them. All five products support the FTP and
SMB (CIFS) file transfer protocols.
Three of the products we examined suffer from the most
prevalent XCS attack: a file is created with a filename spe-
cifically crafted to contain malicious payload that gets exe-
cuted when the admin uses the Web interface to view NAS
contents. In Figure 3, the administrator has just accessed
some shared storage where the attacker has planted a file
with a specially designed name. As a result, instead of show-
ing the name of the file, the browser executes a script which
accesses a restricted area of storage from the administrator’s perform a more complex encoding.
session, but without his or her approval (a carefully designed To avoid the filename length restriction, there are two
attack script would also cover its tracks, showing the direc- possible methods. The first method is to keep the second
tory contents that the administrator expects). stage payload short by loading the full exploit from an
The first step in the attack is a payload injection into the external script on the Internet. We were able to use this
NAS, where the attacker uploads a file with a malicious file- approach on the three devices: in all cases the remote
name. Uploading a file into the NAS can be done using a script invocation fit within the necessary length restric-
public directory (a public FTP directory is often configured tion. Nevertheless, this method can be prevented by con-
by default). Payload injection through file transfer protocols figuring a firewall to block requests from the NAS to the
is a little tricky due to two restrictions enforced by the FTP external network (though this may interfere with the soft-
and SMB protocols: ware update process on the NAS). The second method for
overcoming filename length restrictions is to simply divide
1. Filenames have bounded length. up the second stage exploit across multiple filenames.
2. Filenames cannot contain a ‘/’ and as a result we can- Each filename contains an encoded slice of the second
not embed an HTML closing tag in them. Therefore it stage payload. The first step payload is used to read all the
is not possible to load an external script directly. filenames and recompose the payload.
NAS XCS attacks can be harmful. For example, an
To overcome the second limitation, we designed a two attacker can inject a malicious filename that, when viewed
stage payload using “JavaScript packing”: we encode (pack) by the NAS admin, will take over the admin’s browser ses-
our second stage payload, using HTML escaping, so that sion. This can be used to exfiltrate protected files on the
the packed string does not contain a ’/’ and we use the first NAS, steal the admin’s password, or infect the admin’s
stage (unpacker) to write into the HTML page. For instance machine with malware.
against one of the devices we used the following two-stage
payload: 3.2. XcS from a P2P channel
Another NAS product had a more subtle and potentially
more potent XCS exploit hidden in its P2P (Peer-to-Peer)
“<iframe onload=’javascript:document.write(’ feature. The appliance in question allows the user to
&apos;<html><head><&#47;head><body><script src download BitTorrent files directly by providing an embed-
=&quot; http&#58;&#47;&#47;;t2.js&quot;> ded client. This client is controlled through a Web inter-
<&#47; script><&#47;body><&#47;html>&apos;);’ face available on an alternate port (8080): for example,
src=’index.htm’>” users can add torrents by supplying .torrent files.
A BitTorrent file is basically a list of files to download,
along with their hash and tracker URLs that are used to
find peers.
The first stage (unpacker) bypasses the charset restric- To exploit the XCS vulnerability, an attacker constructs a
tion by avoiding the use of <script></script> to run torrent containing a file with a filename that acts as a mali-
JavaScript code. Instead, we use the onload event of an cious payload. As soon as the user downloads the torrent
iframe to execute the code as soon as the iframe is loaded. file, the Web interface displays the list of files in the torrent
When the HTML encoding is not sufficient to build an causing the browser to execute the payload embedded in
acceptable second stage payload, we use JavaScript eval( ) to the malicious filename (as shown in Figure 4).

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aT ion S of T h e acm 107

research highlights

In more detail, the attack, depicted in Figure 5, proceeds Moreover, because the torrent actually contains the real
as follows: movie, if the payload is sufficiently stealthy the user might
never know that an infection took place.
step 1: The attacker creates a .torrent which contains
a popular movie and an additional file that will have the 3.3. Log-based XcS
malicious payload in its filename. Lights-out management systems: When an operating sys-
step 2: The attacker seeds and uploads the .torrent tem crashes or becomes corrupt, administrators typically
to a popular tracker such as The Pirate Bay. This gives the need local access to the console to reboot or reconfigure
attacker access to over 14 million potential victims. the machine. This situation arises both in the data center
step 3: Lured by the torrent name, many users will fetch the and on personal computers, where the admin must walk
.torrent and once that is opened in the NAS Web interface up to the corrupt machine to diagnose and reboot it. The
the attacker gains control of the browser session. need of physical intervention is problematic, in particular
when there is a service level agreement in place, because it
In this attack, the user has no way of knowing that the drastically increases downtime. To address this issue, all
torrent contains a malicious payload before the torrent is the major hardware vendors have developed firmware com-
fetched. The torrent name by itself is perfectly reasonable and ponents called lights-out management systems (LOM) that
there is nothing to alert the user that it contains a malicious can be remotely accessed by an administrator, no matter
file. how corrupt the software on the machine becomes. LOM
As soon as the torrent is fetched the attack begins. is found on servers, desktops, and laptops (every computer
that uses an Intel Core2 chipset has one, in the form of the
figure 4. Result from the P2P XcS attack. The payload writes “XcS Intel vPro technology). Most LOMs provide a Web interface
attack” in the page. for the administrator to remotely manage the computer.
Lom Vulnerabilities: We examined the Web interface on
four widely used LOM systems, and found several XCS vul-
nerabilities on all of them. We note that these vulnerabili-
ties are compounded by the fact that the LOM Web site
cannot be monitored or filtered by the OS or any software,
such IDS, firewall, or antivirus running on top of it. (The rea-
son for this is to prevent a misconfigured OS from disabling
the LOM system, as that would defeat the purpose of LOM.)
Lom security mechanisms: Vendors took various secu-
rity measures to prevent unauthorized access to the LOM
system. These measures include, among other things,
the use of SSL to protect against network attacks, several
forms of user authentication, and extensive logging of
user activity. Ironically it is the interaction between the
logging facility and the Web interface which is respon-
figure 5. The P2P XcS attack overview. sible for the worst example of XCS we found. The attack,
which applies almost identically to the products of two
different vendors, is possible by simply accessing the
Tracker Web interface on the affected system. There is no need
for an authenticated session.
Abusing the Logging Facility: This XCS uses log injection6
1. The attacker uploads 2. Users download
to inject a script into persistent storage on the device. The
the malicious torrent the torrent attack works as follows:

step 1: The attacker attempts to log into the LOM Web

Attacker site served by the managed machine. Instead of trying to
guess the login, he inputs a malicious payload as the user
name. For example, the malicious payload might first close
the function invocation where the user name is passed as
an argument, then close the current script tag, and finally
3. The attacker controls
the admin browser inject an invocation of the attacker’s script, fetched from a
remote URL. The whole sequence is very compact:
r”,“”,“”);\\/––></script><script src=“http://xxx”></script>

step 2: The logging facility will record this username as-is

into the LOM log file on the machine. The logging facility

108 comm unicaTio nS o f Th e acm | au Gust 2 0 10 | Vol. 53 | no. 8

does not escape data written to the log file to prevent Web using the Web server embedded in a photo frame. The sec-
attacks, despite the fact that the log file can be viewed via the ond combines XCS and reverse XCS to exfiltrate protected
Web interface. data stored on a NAS through a P2P network.
step 3: The malicious payload is executed by the LOM ad-
min’s browser when she views the log. The malicious pay- 4.1. The ghost in the photo frame
load can be used to add a rogue administrator account to the A photo frame built by a major consumer device vendor has
LOM and thus grant full access to the attacker. The attacker an embedded Web server on port 5050, with a default pass-
can also infect the administrator’s computer by directing word. As most embedded devices that we evaluated, the
the browser to a malware site.10 photo frame is vulnerable to CSRF and XSS attacks. More
precisely, in the settings page it is possible to use the frame
The result of this XCS attack on the Dell Remote Access name input to inject and store a nonescaped payload: our
Controller is shown in Figure 6 where an image is injected “ghost.” The ghost will be reflected on the photo frame
onto the administration page. main page that displays the current photo and provides
controls to change it.
3.4. cellphone-based XcS Figure 7 depicts how the attack works: first the attacker
XCS attacks are not limited to Web management inter- injects malicious code to a site that the user will visit. Then the
faces. Modern smartphone platforms such as Google’s user browser runs the malicious code and infects the photo
Android and Palm’s WebOS use HTML and JavaScript to frame with the ghost (see Figure 8). Finally, each time the user
build application views. On the Palm Pre, for example, the visits the photo frame Web server, the ghost executes and exfil-
entire GUI is built using JavaScript and HTML on top of trates the current photo which is stored on an SD card.
Webkit. Given the number of services and protocols sup- Note that once again firewalls can’t prevent this kind of
ported on these elegant devices, XCS is an important con- attack as the user browser is used to infect the frame and
cern. Indeed, a recent report7 shows that the Palm Pre is exfiltrate the data. As presented in Figure 8, the attack can
vulnerable to an XCS attack that injects its payload through be broken into two phases: infection and execution.10 Figure 9
a calendar title or content. shows the ghost in action. We added a visible debug trace at
the bottom of the interface.
4. ReVeRSe XcS: DaTa eXfiLTRaTion anD BeyonD Infection: The infection phase aims to store the ghost into
A reverse XCS attack uses the Web interface to eventually the photo frame. To do so, three steps are required. First,
attack a non-Web channel. The main application for this the malicious code performs a port scan to detect if the
class of attacks is to exfiltrate data that is not supposed to photo frame is present in the user LAN. To do so, it tests
be shared either because it is protected by an access con- whether port 5050 is open on a set of probable internal
trol mechanism or because it is not supposed to be shared IPs:, for instance. Since the port used by the
at all. photo frame is unusual, then there is a good chance that, if
We illustrate reverse XCS using two real-world vulner- this port is open, a photo frame is present. Second, for each
abilities. The first exfiltrates photos stored on an SD card by open port found a CSRF attack is used to log in using the
default password. Finally a second CSRF attack is used to
figure 6. The result of the Lom log injection attack. inject the ghost into the photo frame name. Since it might
happen that the user is already logged in, a more robust
technique is to first do the CSRF used to inject the ghost
then try to log in and finally re-inject the ghost. In the worst
case, this way we only end up overwriting our ghost which is
not an issue—and we are able to infect frames with custom
passwords as long as the user is already logged in to them.

figure 7. The ghost in the photo frame overview.


Step 1 Iframe injection

Infected site Infected site

Step 2 User browsing Step 3 User browser injects the ghost


Firewall Device

Step 4 Ghost abuses the browser to stealth data

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aT ion S of T h e acm 109

research highlights

figure 8. The ghost attack executed on a photo frame.

2. Javascript errors: The code must not trigger a single
JavaScript error, otherwise the browser will stop the
Phase 1: infection execution, preventing the exfiltration.
3. Fetching data: We had to find a way to fetch binary
data. This is not supported directly by
Port scan Login CSRF Ghost injection XMLHTTPRequest.
4. exfiltrating data: Once the data was loaded in memory,
we had to exfiltrate it while keeping the regular frame
code running.
Phase 2: execution

The first challenge was addressed by using a loader: the

Inject payload Fetch data Post data injected code is not the ghost itself but rather a payload
that will ask the browser to load the ghost as an external
The second challenge was more difficult because the
injected ghost is reflected in the middle of a JavaScript func-
tion in the variable name. Therefore the following payload
figure 9. The ghost in action: a photo has just been exfiltrated. was injected to the frame:

name “; }</script>
<script src=“http://www/g.js”>
</script><script> function n() {var frame Name =”

This payload is designed to close the variable, the func-

tion and the script, request the ghost as a new script and
resume the function. Resuming was required because oth-
erwise the frame control would have been broken.
To deal with the third and fourth challenges which
are closely related, we had to come up with a new
method that uses AJAX tricks and a manipulation of the
XMLHTTPRequest object in a novel way. The sketch of the
code used as a ghost is depicted below:

data = fetch(page);
data = decode(data);
data = rencode(data);

This code works as follows: first it injects in the page an

invisible form named f (used to post exfiltrated data) and an
iframe named uploadtarget into the Web page (line 1). This
iframe is used to take advantage of the ability to control
through JavaScript the iframe in which the form f action will
be executed. Accordingly the second step of the ghost (line
2) is to redirect the form f action to our invisible iframe by
using the following JavaScript command:
= ‘upload_target’;. Posting into the iframe is mandatory to
prevent the redirection of the entire page that will break
the exfiltration loop and alert the user. Note that the same
execution: The four challenges we faced in implementing a origin policy—the mechanism which protects the user’s
ghost designed to exfiltrate data were: session to a legitimate Web site from being exploited by a
different, malicious Web site11—is not an issue here as post-
1. payload size: The size of the payload that can be ing data from the legitimate site to the malicious one goes
injected is limited. in the opposite direction and is currently fully unrestricted:

110 co mm unicaTio nS o f T h e acm | au Gust 2 0 10 | Vol. 53 | no. 8

such posting is assumed to be under the control of the the attacker to hide his malicious activity. By playing with
request originator. the CSS display attribute the attacker can mask his
At this point, the problem is to acquire the data that will be malicious torrents and display only those requested by
exfiltrated. The standard way to post a file is to use a file input the user. So unless the user views the page source, he will
field that the user will use to select which file to post. Of course be completely unaware of the attack. The key challenge
in our case, we need to find an alternate method as we don’t was to find a way to allow the ghost to control which files
have the user’s cooperation, and, moreover, it is not possible need to be seeded. To achieve this, we used an externally
to manipulate the file input with JavaScript for obvious secu- loaded JavaScript that keeps track of the current files
rity reasons. Therefore we had to come up with an alternative seeded by reading the client page and comparing it to a
approach. Based on the observation that the files we want list supplied by the attacker. If one file is not seeded, then
to exfiltrate are located in the same domain as the ghost, we the JavaScript adds it by hijacking the Web function used
came up with the idea of using an XHR (XMLHTTPRequest) to to add a torrent file. Note that again a firewall cannot pre-
load the data inside a JavaScript variable (line 3). vent this attack since the client is authorized to download
The same origin policy is once again unable to prevent its own torrents.
this behavior because our ghost acts here as an autoim-
mune disease: an infected page attacks the rest of the same 4.3. Bypassing cSRf defenses
Web site. One difficulty with using this method is that the Another application of reverse XCS, which is a natural
XHR object is not designed to fetch binary data—only text, extension of previous attacks, is to use the infected page
and so using solely XHR is not sufficient. To go around to attack the same site using XHR or CSRF. This combina-
this issue, we came up with the idea of changing the http tion allows to bypass current CSRF defenses because they
request header and more precisely the Mimetype encoding. all rely on the same origin policy in one way or another. In
In Firefox, for example, it is possible to override the mime other words, the same origin policy does not apply to our
type used in an XHR and request a custom charset encoding attack because we use an infected page to attack other
by using the method: pages within the same domain.
The two prominent defenses against CSRF2 are to verify
overrideMimeType(“text/ the HTTP header referer/origin and to use a hidden secret
plain;charset=x-user-defined”) token. Checking the HTTP header is useless in the context
of XCS because the request comes from the same domain.
Using the XHR object with this override allows the ghost to The use of secure tokens can be defeated by sending an
fetch any type of file and load it into a JavaScript variable. XHR request to the page, reading its result and extracting
The rest of the ghost code is straightforward: it is used to the token value to construct dynamically the form that
decode the XHR custom encoding (line 4), re-encode the file will be used to perform the CSRF attack. The direct con-
in base64 (line 5), post it (line 6) in the iframe, and reload clusion of this is that any device subject to an XCS is also
the interface to exfiltrate the next photo. subject to CSRF attacks regardless of the CSRF defense it
The last problem we had to deal with was the reload implements. Moreover, since the XCS injection vector is
timer used in the photo frame: every 500 ms the page was not Web based, pure Web defense mechanisms have no
reloaded. Of course this behavior was breaking the ghost impact on XCS attacks.
activity so a part of the ghost is used to override the time-
out value with a huge number and when the photo is exfil- 5. Back To The WeB: ReSTfuL RXcS
trated the reload method is explicitly called by the ghost. Up to this point we have intentionally avoided direct refer-
In this way the ghost is able to transparently accommo- ences to product vendors. In this section, we address the
date any upload speed. challenges posed by two specific social networking sites;
however, the problem discussed has a much wider scope.
4.2. The ghost in the P2P client We feel that being concrete will help illustrate the prob-
Recall from Section 3.2 that some devices have an lems in each environment without having to point at any
embedded P2P client. Besides being an XCS injection third-party application developers.
vector, this client can be abused by a reverse XCS to seed In this section, we present RXCS vulnerabilities that
illegal data and exfiltrate data. The idea behind the attack make use of the APIs provided by large social networking
is as follows: the attacker, who has control over the Web Web sites. RESTful APIs are becoming the ubiquitous way to
interface, uses it to insert torrents that he wants the NAS interact with cloud services. Many popular cloud services,
to seed for him. including Twitter, Facebook, E-bay, Google and Flickr, offer
This can have two purposes: on the one hand he can use this kind of API.
the NAS capacity and the user bandwidth to seed illegal files For example, the Twitter RESTful API allows anyone to
on his behalf. Combined with the P2P XCS approach from query user profiles in XML format by issuing the following
Section 3.2, this is a way to seed illegal data on a massive call:
scale. On the other hand, the attacker can use the P2P client
to exfiltrate NAS content through the P2P network.
The user is oblivious to the attack because, as in the
photo frame case, having full control of the page allows

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aT ion S of T h e acm 111

research highlights

This call will return the following formatted data that can provided to third-party applications is HTML escaped.
be subsequently processed by the third-party application If an application wants to deal with “raw data” it has to
to compute statistics or store in a database for later use. unescape the data before processing it. Of course, when
the application wants to output the data, it has to reescape
the data in its own way. This unescape, reescape process
<user> is tedious and error-prone. Indeed, it is not difficult to
<id>57142771</id> find a Twitter application that is vulnerable to RXCS injec-
<name>Elie Bursztein</name> tion. In Figure 10, one such application is asked to search
<screen_name>elie</screen_name> for a string, which is the label of a message planted by the
<location>Palo Alto</location> attacker. When the application finds the string (and thus
<url></url> the planted message), it shows the message in the browser
<protected>false</protected> after processing it and the embedded payload is inadver-
… tently executed.
<text> As one can see, interactions with cloud services rely on
Second time I see the SMS fuzzing talk, many assumptions that are not properly formalized. In par-
I am still loving it :) #woot ticular, understanding the trust model behind this exchange
</text> and how to combine filtering policies are open questions
… that we want to address in future work.
6. DefenSeS aGainST XcS
One defense against XCS is to ensure that all data sent to the
The problem here lies in the fact that there is implicit user’s browser is properly sanitized. In principle, static analyz-
trust between the third-party application and the cloud ser- ers can perform flow analysis to detect potential XCS.1, 14 This
vice. Third-party application developers assume the cloud approach must taint all input channels into the Web applica-
service provides “safe” data. However, defining what safe tion, including all persistent data on the device, and raise an
data means is far from obvious and each cloud service has alarm if tainted data is displayed in a Web page without first
its own sanitization policy which is often not explicitly docu- being sanitized. Tracking data life cycle can easily miss some
mented. This inconsistency between expected data and sup- XCS channels or fail to taint XCS content. Another popular XSS
plied data can result in RXCS. We give two examples. defense, used by Twitter for instance, is to sanitize all user data
at input time, before it is written to persistent storage at the
5.1. facebook RXcS site. This is unlikely to mitigate an XCS vulnerability because
Facebook escapes at display time which means that the the malicious content is injected via a non-Web channel,
data provided to third-party applications is not escaped. which usually does not sanitize for Web exploits. Moreover,
Facebook’s terms of service say that third-party applica- this defense fails to work directly on non-Web raw data, such
tions are not supposed to directly output the data fetched as plain text event logs. This is problematic if these data are
from the API but rather use the Facebook output functions. also used by other applications that are not Web based such as
Similarly, applications are not supposed to store any user a back-end statistics analyzer or an IDS.
data. However, it is likely that some applications will display In our original paper on XCS,3 we proposed a defense
the data or store it, even if Facebook may monitor API usage called SiteFirewall, and compared it to other current devel-
to prevent terms of service violations. opments in Web security. SiteFirewall, CSP, SOMA, and
To give an example, we point out that attacking a vulnera- other mechanisms generally attempt to block a Web site
ble third-party application can be done by noting that all the running in a browser context from executing operations that
profile details from interests to music and movies are not involve other Web sites and could potentially either access
escaped. Consequently, it is sufficient to add the <script> or exfiltrate private data.
tag to them to get that text reflected to an application. In While recent work on XCS defenses has focused primar-
theory, this might be used to bypass Facebook security ily on the browser, we have only begun to address some
policies. The source of the problem is that Facebook trusts inherently server-side problems. First, device vendors must
third-party applications to properly handle API data, while
application developers often make assumptions about the
figure 10. a Twitter third-party application attack illustrated.
safety of that data.
Suppose you have an application that displays informa-
tion about Facebook users’ favorite movies. It is sufficient
to add malicious payload in the movie profile data: this pay-
load can then get reflected to all Facebook users that use the

5.2. Twitter RXcS

Twitter has the opposite filtering policy compared to
Facebook: escaping is performed at input time so all data

112 comm unicaTio nS o f Th e ac m | au Gust 2 0 10 | Vol. 53 | no. 8

exercise greater care in selecting Web server implementa- J. robust defenses for cross-site e. Pixy: a static analysis tool
request forgery. In Proceedings of for detecting Web application
tions and demand better security from external embedded ACM CCS '08 (2008). vulnerabilities. In IEEE Symposium
Web server developers and their own engineering staff. 3. bojinov, H., bursztein, e., boneh, on Security and Privacy (2006).
d. xCs: cross channel scripting 9. lam, V.t., antonatos, s., akritidis, P.,
Second, complex embedded application logic and state and its impact on Web applications. anagnostakis, K.G. Puppetnets:
need to be more visible, which would enable vulnerability In CCS ‘09: Proceedings of the 16th Misusing Web browsers as a
ACM Conference on Computer and distributed attack infrastructure.
scans either from the same host (in the case of LOM) or over Communications Security (new york, In Proceedings of the CCS (2006).
the network (for appliances not running a general-purpose ny, usa, 2009), aCM, 420–431. 10. Provos, n., Mcnamee, d.,
4. dagon, d., Gu, G., lee, C., lee, W. Mavrommatis, P., Wang, K.,
OS). Third, the Web community needs to recognize that a taxonomy of botnet structures. Modadugu, n. the ghost in the
embedded Web sites can have fundamentally different use In Proceedings of the 23 Annual browser analysis of Web-based
Computer Security Applications malware. In Proceedings of
models from those usually seen on the Internet: we have to Conference (ACSAC) (2007). HotBots’07 (2007).
enable these two paradigms to coexist securely. 5. Fogie, s., Grossman, J., Hansen, r., 11. ruderman, J. Javascript security:
rager, a., Petkov, P. XSS Exploits: same origin, august 2001. http://
Cross Site Scripting Attacks and
Defense. syngress, 2007. components/same-origin.html.
7. concLuSion 6. Grzelak, d. log injection attack and 12. stuttard, d., Pinto, M. The Web
Networked appliances are not as secure and harmless as defence, 2007. Application Hacker’s Handbook:
assets/downloads/sIFt-log-Injection- Discovering and Exploiting Security
they are often assumed to be. The advent of browser-centric Intelligence-report-v1-00.pdf. Flaws. Wiley, 2007.
Web 2.0 computing has amplified the scope of attacks possi- 7. Harris, t.l., Palm. software update 13. twitter worm. http://www.
information for palm pre sprint
ble via embedded devices, giving rise to XCS. There is much p100eww, august 2009. Web: http:// hit-by-stalkdaily-worm/.
work to be done before hardware vendors start to routinely 14. xie, y., aiken, a. static detection of
pre/p100eww/sprint/solutions/ security vulnerabilities in scripting
design and test for security, Web browsers are capable of article/50607_en.html. languages. In In Proceedings of the
dealing securely with different classes of Web applications, 8. Jovanovic, n., Kruegel, C., Kirda, USENIX Security Symposium (2006).

and users are enabled to make and execute decisions about

managing their networked private data. We hope that we hristo Bojinov (, Dan Boneh (,
stanford university, stanford, Ca. stanford university, stanford, Ca.
have at least made the first step toward that goal. Elie Bursztein (,
stanford university, stanford, Ca.
1. balzarotti, d., Cova, M., Felmetsger, sanitization in Web applications. In this work is supported by the nsF, dHs, and the Packard Foundation.
V., Jovanovic, n., Kirda, e., Kruegel, IEEE Symposium on Security and
C., Vigna, G. saner: Composing static Privacy (2008).
and dynamic analysis to validate 2. barth, a., Jackson, C., Mitchell,
© 2010 aCM 0001-0782/10/0800 $10.00

Announcing ACM’s Newly Improved

Career & Job Center!
Are you looking for your next IT job? Do you need Career Advice?
Visit ACM’s newly enhanced career resource at:
◆ ◆ ◆ ◆ ◆

The ACM Career & Job Center offers ACM members a host of benefits including:
➜ A highly targeted focus on job opportunities in the computing industry
➜ Access to hundreds of corporate job postings
➜ Resume posting keeping you connected to the employment market while letting you maintain full

control over your confidential information

➜ An advanced Job Alert system that notifies you of new opportunities matching your criteria

➜ Career coaching and guidance from trained experts dedicated to your success

➜ A content library of the best career articles complied from hundreds of sources, and much more!

The ACM Career & Job Center is the perfect place to

begin searching for your next employment opportunity!
Visit today at

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aT ion S of T h e acm 113

research highlights
Doi:10.1145/ 1787234.1 78 72 5 8

Technical Perspective ternal choices: the abstract model just

performs the internal choices without
Large-Scale Sound and external control. This abstraction is fa-
miliar from software model checking.
Precise Program analysis The authors contend it is important
for practical precision to model the re-
By Fritz Henglein sults of internal choices as unknowns:
once a choice is performed the result
you ARe gIVen a program. Will it crash? against arbitrary programs thrown is unknown, but every time we need
Is it subject to a spoofing, buffer over- its way it forces the programmer to the value it is the same. Their key in-
flow, or injection attack? Is this part of provide what amounts to a checkable sight is we are typically only interested
it dead code? Can I replace that code proof that the program has the desired in whether an abstracted function has
fragment with a more efficient one? property. Very often we need to analyze a property for all possible sequences
Which exceptions can arise from call- general-purpose programs without of internal choices in its body or for
ing this function in that context? Does the privilege of foresight provided by some particular sequence, or indeed
this variable always contain the same the programmer and enforced by the both. This is captured by logical con-
value? All of these questions require programming language, however. This straints for the input-output relation
program analysis: We want to know leaves only the third possibility: of a function where all the internal
whether program code (the input to the ˲˲ Allow approximate answers; that choices are universally, respectively
analysis) has a specified property. is, allow the program analysis to return existentially bound.
Program analysis is inherently dif- “don’t know” answers or to run forever The paper outlines conceptually
ficult. Take the following ingredients: (which is practically tantamount to independent and generally applicable
˲˲ A Turing-complete programming “don’t know”). Since a “don’t know” steps assembled into a solution pipe-
language such as C, Java, Haskell;a answer constitutes a false positive line: quantifier elimination, symbolic
˲˲ A nontrivial behavioral program whenever the (desirable) property re- fixed-point computation to compute
property; for example, does a program ally holds, a key challenge is designing closed forms for recursive constraints,
de-reference a null pointer, is a particu- a program analysis to be sufficiently and finally constraint solving, bring-
lar fragment dead code; and precise for its intended application so ing together and leveraging techniques
˲˲ Require an exact answer: yes if the it does not drown its user (whether hu- ranging from the foundation of (Bool-
property holds, no if it doesn’t. man or another system such as a com- ean) logic to state-of-the-art SAT-solv-
The requirements look reason- piler) in false positives: An analysis re- ing. Their execution requires care, as
able: Given an input program we want turning 1,000 warnings (“don’t know”) the authors explain aided by their se-
to check whether it has some desired of which only 10 turn out to be bona- lection of an illustrative fragment of
“runtime” (behavioral) property. But fide bugs may not be practically useful. C and a running example developed
they constitute an explosive mix: Any The following paper provides a pow- from source code to analysis result.
such problem is undecidable—no pro- erful sound static analysis framework The challenge of bug hunting in
gram can solve it.1 Consequently, we for C programs and nontrivial behav- large C code bases has seen the rise
need to give up at least one of these re- ioral properties. The authors show that and application of techniques that
quirements to escape undecidability. a precise compositional program anal- are neither sound nor complete nor
˲˲ Remove Turing-completeness: ysis that is context-sensitive and path- even stable: their result may depend
Languages of restricted expressive sensitive is possible for huge code bases on the machine load, other programs
power may have decidable nontrivial of central importance to our day-to-day analyzed before or simultaneously, and
behavioral properties; for example, computing infrastructure. In particu- other factors completely extrinsic to the
equivalence of regular expressions, lar, they show that null derefence bugs, program at hand. A bug is a bug, which-
which allows guaranteed optimization which require interprocedural data ever way you catch it, the argument
of regular expression matching. flow analysis, can be found without goes. This paper illustrates that the
˲˲ Make the property trivial; that is, drowning them in false positives. days of feasible, well-specified, sound,
make it (or its negation) hold for all pro- Their starting point is an abstract and sufficiently precise static program
grams. In other words, build it into the model of a program where infinite do- analysis for bug hunting even in huge
programming language. This is what mains are finitely approximated and code bases are not numbered.
static type systems and emerging tech- certain steps are replaced by nondeter-
nologies such as proof-carrying code ministic steps resulting in unknown Reference
1. Gordon rice, H. Classes of recursively enumerable
can provide. They constitute a cunning results; for example, the integers may sets and their decision problems. Transaction of the
turning of the tables: Instead of pro- be replaced by a single abstract value American Mathematical Society 74 (1953), 358–366.

gram analysis fighting an uphill battle (representing all integers) and the re-
Fritz henglein ( is a professor of Cs
sult of an arithmetic comparison may and head of the algorithms and Programming languages
a A programming language is Turing-complete
be approximated by a nondeterminis- Group, department of Computer science, university of
Copenhagen (dIKu), denmark.
if it is possible to write an interpreter for Tur- tic choice of a Boolean value for the re-
ing Machines in it. sult. We can think of these steps as in- © 2010 aCM 0001-0782/10/0800 $10.00

114 co m municaTio nS o f T h e acm | AU gU ST 201 0 | VO l . 5 3 | NO. 8

Doi:10.1145/ 1787234 . 1 78 72 5 9

Reasoning About the Unknown

in Static Analysis
By Isil Dillig, Thomas Dillig, and Alex Aiken

abstract arise from approximating program behavior. A static analy-

Static program analysis techniques cannot know certain sis cannot simply carry out an exact program simulation; if
values, such as the value of user input or network state, nothing else, we usually want to guarantee the analysis ter-
at analysis time. While such unknown values need to be minates even if the program does not. Thus, static analysis
treated as nondeterministic choices made by the program’s always has some imprecision built in. For example, since
execution environment, it is still possible to glean very use- lists, sets, and trees may have an unbounded number of
ful information about how such statically unknown values elements, many static techniques do not precisely model
may or must influence computation. We give a method for the data structure’s contents. Reading an element from
integrating such nondeterministic choices with an expres- a data structure is modeled as a nondeterministic choice
sive static analysis. Interestingly, we cannot solve the result- that returns any element of the data structure. Similarly, if
ing recursive constraints directly, but we give an exact the chosen program abstraction cannot express nonlinear
method for answering all may and must queries. We show arithmetic, the value of a “complicated” expression, such as
experimentally that the resulting solved forms are concise coef*a*b+size, may also need to treated as an unknown by
in practice, enabling us to apply the technique to very large the static analysis.
programs, including an entire operating system. The question of what, if any, useful information can be
garnered from such unknown values is not much discussed
in the literature. It is our impression that if the question is
1. inTRoDucTion considered at all, it is left as an engineering detail in the
Preventing software errors is a central challenge in software implementation; at least, this is the approach we have taken
engineering. The many tool-based approaches to the prob- ourselves in the past. But two observations have changed our
lem can be grouped roughly into two categories. Dynamic minds: First, unknown values are astonishingly pervasive
analysis techniques discover properties by monitoring pro- when statically analyzing programs; there are always calls
gram executions for particular inputs; standard testing is to external functions not modeled by the analysis as well as
the most commonly used form of dynamic analysis. In con- approximations that lose information. Second, in our expe-
trast, a static analysis discovers properties that hold for all rience, analyses that do a poor job handling unknown values
possible inputs; a sound static analysis concludes a program either end up being unscalable or too imprecise. For these
is error-free only if the program indeed has no errors. reasons, we now believe a systematic approach for dealing
Unlike dynamic analyses, sound static analyses have the with unknown values is a problem of the first order in the
advantage of never missing any potential errors, but, unfor- design of an expressive static analysis.
tunately, there is no free lunch: Soundness usually comes at We begin by informally sketching a very simple, but
the cost of reporting false positives (i.e., spurious warnings imprecise, approach to dealing with unknown values in
about error-free code) because static analyses must approxi- static analysis. Consider the following code snippet:
mate some aspects of program behavior. This approxima-
tion is inevitable as analyzing even very simple properties of
programs’ behavior is undecidable. Hence, a key challenge 1: char input = get_user_input();
for static analysis techniques is achieving a satisfactory com- 2: if(input == ’y’) f = fopen(FILE_NAME);
bination of precision, soundness, and scalability by report- 3: process_file_internal(f);
ing as few false positives as possible while still being sound 4: if(input == ’y’) fclose(f);
and scaling to real systems.
This goal of obtaining satisfactory precision is further
complicated by the fact that certain values are simply Suppose we want to prove that for every call to fopen,
unknown statically: For example, if a program queries the there is exactly one matching call to fclose. For the
user for an input, this input appears as a nondeterminis- matching property to be violated, it must be the case that
tic environment choice to the static analysis. Similarly, the the value of input is ’y’ on line 2, but the value of input
result of receiving arbitrary data from the network or the
result of reading operating system state are all unknowns The original version of this paper is entitled “Sound,
that need to be treated as nondeterministic environment Complete, and Scalable Path-Sensitive Analysis” and was
choices by the analysis. published in the Proceedings of Programming Language
Even in the special case where all program inputs are Design and Implementation (PLDI) 2008, ACM.
known, static analyses still need to deal with unknowns that

au G u st 2 0 1 0 | Vo l . 5 3 | n o. 8 | c om m u n ic aTion S of T h e acm 115

research highlights

is not ’y’ on line 4. Since the value of the input is unknown, (abbreviated T) in any calling context, is then given by the
one simple approach is to represent the unknown value constraint:
using a special abstract constant é. Now, programs may
P.b = (a = T) Ù (b = ’y’Ú (¬(b = ’n’) Ù P[T/a] = T)) (*)
have multiple sources of unknown values, all of which are
represented by é. Thus, é is not a particular unknown but This formula is read as follows. The term a = T captures that
the set of all unknowns in the program. Hence, the predi- the function returns true only if feature_enabled is true
cates é = ’y’ (which should be read as: ’y’ is equal to (line A). Furthermore, the user input must either be ’y’ (term
some element of values represented by é) and é ’y’ b = ’y’ and line C) or it must not be ’n’ (term ¬(b = ’n’) and
(which should be read as: ’y’ is not equal to some element line D) and the recursive call on line G must return true (term
of values represented by é) are simultaneously satisfiable. P[T/a]). Observe that because the function is recursive, so is
As a result, program paths where input is equal to ’y’ at the formula. In the term P[T/a], the substitution [T/a] models
line (2), but not equal to ’y’ at line (4) (or vice versa) can- that on the recursive call, the formal parameter a is replaced by
not be ruled out, and the analysis would erroneously report actual parameter true. Finally, the binding P.b reminds us
an error. that b is a choice variable. When the equation is unfolded to per-
A more precise alternative for reasoning about unknown form the substitution [T/a] we must also make the environment
values is to name them using variables (called choice vari- choice for b. The most general choice we can make is to replace
ables) that stand for a single, but unknown, value. Observe b with a fresh variable b¢, indicating that we do not know what
that this strategy of introducing choice variables is a choice is made, but it is potentially different from any other
refinement over the previous approach because two dis- choice on subsequent recursive calls. Thus, P[T/a] unfolds to:
tinct environment choices are modeled by two distinct
(T = T) Ù (b ¢ = ’y’Ú (¬( b¢ = ’n’) Ù P[T/a]
choice variables, b and b¢. Thus, while a choice variable b
may represent any value, it cannot represent two distinct While the equation (*) expresses the condition under
values at the same time. For instance, if we introduce the which query_user returns true, the recursive definition
choice variable b for the unknown value of the result of the means it is not immediately useful. Furthermore, it is easy
call to get_user_input on line 1, the constraint charac- to see that there is no finite nonrecursive formula that is
terizing the failure condition is b = y Ù b π y, which is unsat- a solution of the recursive equation (*) because repeated
isfiable, establishing that the call to fopen is matched by unfolding of P [T/a] introduces an infinite sequence of fresh
a call to fclose. The insight is that the use of choice vari- choice variables b ¢, b², b¢², . . . . Hence, it is not always possi-
ables allows the analysis to identify when two values arise ble to give a finite closed-form formula describing the exact
from the same environment choice without imposing any condition under which a program property holds.
restrictions on their values. On the practical side, real programs have many sources of
While this latter strategy allows for more precise reason- unknowns; for example, assuming we do not reason about
ing, it leads to two difficulties—one theoretical and one the internal state of the memory management system, every
call to malloc in a C program appears as a nondeterminis-
tic choice returning either NULL or newly allocated memory.
bool query_user(bool feature_enabled) {
In practice, the number of choice variables grows rapidly
A: if(!feature_enabled) return false;
with the size of the program, overwhelming the constraint
B: char input = get_user_input();
solver and resulting in poor analysis scalability. Therefore,
C: if(input == ’y’) return true;
it is important to avoid tracking choice variables whenever
D: if(input == ’n’) return false;
they are unnecessary for proving a property.
E: printf(“Input must be y or n!
Our solution to both the theoretical and the practical
F: Please try again.\n”);
problems can be understood only in the larger context of why
G: return query_user(true);
} we want to perform static analysis in the first place. Choice
variables allow us to create precise models of how programs
interact with their environment, which is good because we
practical—that the simpler, but less precise, strategy does never know a priori which parts of the program are impor-
not suffer from. Consider the following function:a tant to analyze precisely and so introducing unnecessary
Suppose we want to know when query_user returns imprecision anywhere in the model is potentially disastrous.
true. The return value of get_user_input is statically But the model has more information than needed to answer
unknown; hence it is identified by a choice variable b. The most individual questions we care about; in fact, we are usu-
variable feature_enabled, however, is definitely not a ally interested in only two kinds of 1-bit decision problems,
nondeterministic choice, as its value is determined by the may and must queries. If one is interested in proving that a
function’s caller. We represent feature_enabled by an program does not do something “bad” (so-called safety prop-
observable variable, a, provided by callers of this function. erties), then the analysis needs to ask may questions, such as
The condition, P, under which query_user returns true “May this program dereference NULL?” or “May this program
raise an exception?”. On the other hand, if one is interested in
While this function would typically be written using a loop, the same prob- proving that a program eventually does something good (so
lem arises both for loops and recursive functions, and we use a recursive called liveness properties), then the analysis needs to ask must
function because it is easier to explain. questions, such as “Must this memory be eventually freed?”.

116 co m municaT io nS o f Th e acm | auGust 2 0 10 | Vol. 53 | no. 8

May questions can be formulated as satisfiability que- can be composed using the standard Boolean connectives, Ù,
ries; if a formula representing the condition under which Ú, and Ø. In this language, we model unknown values by refer-
the bad event happens is satisfiable, then the program is not ences to unbound variables, which are by convention taken to
guaranteed to be error-free. Conversely, must questions are have a nondeterministic value chosen on function invocation.
naturally formulated as validity queries: If a formula repre- Thus, any free variables occurring in a function body are choice
senting the condition under which something good hap- variables. Observe that this language has an expressive set of
pens is not valid, then the program may violate the desired predicates used in conditionals, so the condition under which
property. Hence, to answer may and must questions about some program property holds may be nontrivial.
programs precisely, we do not necessarily need to solve the To be specific, in the remainder of this paper, we con-
exact formula characterizing a property, but only formulas sider the program properties “May a given function return
that preserve satisfiability (for may queries) or validity (for constant (i.e., abstract value) Ci?” and “Must a given func-
must queries). tion return constant Ci?”. Hence, our goal is to compute the
The key idea underlying our technique is that while choice constraint under which each function returns constant Ci.
variables add useful precision within the function invoca- These constraints are of the following form:
tion in which they arise, the aggregate behavior of the func-
Definition 1 (Constraints).
tion can be precisely summarized in terms of only observable
variables for answering may and must queries. Given a finite
abstraction of the program, our technique first generates a
recursive system of equations, which is precise with respect
to the initial abstraction but contains choice variables. We
then eliminate choice variables from this recursive system
to obtain a pair of equisatisfiable and equivalid systems over
only observable variables. After ensuring that satisfiabil- Symbols s in the constraint language are abstract values
ity and validity are preserved under syntactic substitution, Ci, choice variables b whose corresponding abstract values
we then solve the two recursive systems via standard fixed- are unknown, and observable variables a representing func-
point computation. The final result is a bracketing con- tion inputs provided by callers. Because the values of inputs
straint áfNC, fSCñ for each initial equation, corresponding to each function f are represented by variables a, the con-
to closed-form strongest necessary and weakest sufficient straints generated by the analysis are polymorphic, i.e., can
conditions. be used in any calling context of f. Constraints F are equalities
We demonstrate experimentally that the resulting brack- between symbols (s1 = s2), constraint variables with a substi-
eting constraints are small in practice and, most surpris- tution P[Ci /a], or Boolean combinations of constraints. The
ingly, do not grow in the size of the program, allowing our substitutions [Ci /a] on constraint variables are used for the
technique to scale to analyzing programs as large as the substitution of formals by actuals, and recall that the vector
entire Linux kernel. We also apply this technique for finding of choice variables named with the P variable is replaced
null dereference errors in large open source C applications by a vector of fresh choice variables in each unfolding of
and show that this technique is useful for reducing the num- the equation. More formally, if P. = F, then:
ber of false positives by an order of magnitude.
P[Ci /a] = F [Ci /a][ / ] ( fresh)
2. fRom PRoGRamS To conSTRainTS This renaming is necessary both to avoid naming collisions
As mentioned in Section 1, static analyses operate on a and to model that a different environment choice may be
model or abstraction of the program rather than the pro- made on different recursive invocations. Constraints express
gram itself. In this paper, we consider a family of finite the condition under which a function f with input a returns a
abstractions where each variable has one of abstract val- particular abstract value Ci; we usually index the correspond-
ues C1, . . . , Ck. These abstract values can be any fixed set of ing constraint variable Pf, a, C for clarity. So, for example, if
predicates, typestates, dataflow values, or any chosen finite there are only two abstract values C1 and C2, the equation
domain. We consider a language with abstract values C1, . . .
, Ck; while simple, this language is sufficiently expressive to
illustrate the main ideas of our techniques:
describes the function f that always returns C1, and

describes the function f that returns C1 if its input has abs-

tract value C2 and vice versa. As a final example, the function
define f(x) = if (y = C2) then C1 else C2
where the unbound variable y models a nondeterministic
Expressions are true, false, abstract values Ci, variables x, choice is described by the equation:
function calls, conditional expressions, let bindings and com-
parisons between two expressions. Boolean-valued expressions

AU g U ST 2 0 1 0 | VO l. 53 | N O. 8 | c om m u n ic aT ion S of T h e acm 117

research highlights

figure 1. inference rules.

an expression e evaluates to true or false in environment A.
Rules 6–11 prove judgments A Ci e : F that give the constraint
under which expression e evaluates to Ci. Finally, rule 12 con-
structs systems of equations, giving the (possibly) mutually
recursive conditions under which a function returns each
abstract value.b
We briefly explain a subset of the rules in more detail. In
Rule 3, two expressions e1 and e2 are equal whenever both have
the same abstract value. Rule 8 says that if under environment
A, the abstract value of variable x is represented by constraint
variable a, then x has abstract value Ci only if a = Ci. Rule 11
presents the rule for function calls: If the input to function
f has the abstract value Ck under constraint Fk, and the con-
straint under which f returns Ci is Pf,a, C , then f (e) evaluates to
Ci under the constraint Fk Ù Pf,a, C [Ck/a].

Example 1. Suppose we analyze the following function:

define f(x) = if ((x = C1) Ú (Y = C2)) then C1 else f(C1)
where y models an environment choice and the only abstract
values are C1 and C2. Then

is the equation computed by the inference rules. Note that the

substitution [C1/a] in the formula expresses that the argument
of the recursive call to f is C1.
We briefly sketch the semantics of constraints. Constraints
are interpreted over the standard four-point lattice with ⊥ £
true, false,  and ⊥, true, false £ , where Ù is meet, Ú is join,
and ¬⊥ = ⊥, ¬ = , ¬true = false, and ¬ false = true. Given
an assignment q for the choice variables b, the meaning of a
system of equations E is a standard limit of a series of approxi-
mations q (E0), q (E1), . . . generated by repeatedly unfolding E.
We are interested in both the least fixed point (where the first
approximation of all P variables is ⊥) and greatest fixed point
(where the first approximation is ) semantics. The value ⊥
in the least fixed point semantics (resp.  in the greatest fixed
point) represents nontermination of the analyzed program.

2.1. Reduction to Boolean constraints

Our main technical result is a sound and complete method
Note that b is shared by the two constraints; in particular, for answering satisfiability (may) and validity (must) queries
in any solution b must be either C1 or C2, capturing that a for the constraints of Definition 1. As outlined in Section 1,
function call returns only one value. the algorithm has four major steps:
Our goal is to generate constraints characterizing the
condition under which a given function returns an abstract • Eliminate choice variables by extracting strongest nec-
value Ci. Figure 1 presents most of the constraint inference essary and weakest sufficient conditions
rules for the language given above; the remaining rules are • Rewrite the equations to preserve satisfiability/validity
omitted for lack of space but are all straightforward analogs under substitution
of the rules shown. In these inference rules, an environ- • Eliminate recursion by a fixed point computation
ment A maps program variables to variables a, b in the con- • Finally, apply a decision procedure to the closed-form
straint language. Rules 1–5 prove judgments A b e : F where equations
b Î {true, false}, describing the constraints F under which
Because our abstraction is finite, constraints from Def-
Note that rules 3, 10, 11, and 12 implicitly quantify over multiple hypoth- inition 1 can be encoded using Boolean logic, and thus our
eses; we have omitted explicit quantifiers to avoid cluttering the rules. target decision procedure for the last step is Boolean SAT. We

118 com municaTio nS o f T h e ac m | auGust 2 0 10 | Vol. 53 | no. 8

must at some point translate the constraints from Figure 1 is extracting necessary/sufficient conditions from a system
into equivalent Boolean constraints; we perform this transla- of constraints E. The necessary (resp. sufficient) conditions
tion first, before performing any of the steps above. should be satisfiable (resp. valid) if and only if E is satisfiable
For every variable j (j Î{a, b}) in the constraint lan- (resp. valid). This section makes precise exactly what neces-
guage, we introduce Boolean variables ji1, . . ., jin such that sary/sufficient conditions we need; in particular, there are
jij is true if and only if ji = Cj. We map the equation variables two technical requirements:
Pf, a, C to Boolean variables of the same name. A variable
Pf, a, C represents the condition under which f returns Ci, • The necessary (resp. sufficient) conditions should be as
hence we refer to Pf, a, C ’s as return variables. We also trans- strong (resp. weak) as possible.
late each s1 = s2 occurring in the constraints as: • The necessary/sufficient conditions should be only
over observable variables.

In the following, we use V + (f)