In the

News
ALSO FEATURED THIS ISSUE

“SASSY” CHATBOT WINS WITH WIT

Features Editor: Dennis Taylor
dtaylor@computer.org

another pattern recognition system based

Practical Pattern Matching on graphs. A data mining system that repre-
sents data as a collection of nodes and links
between the nodes, Subdue works by search-
Danna Voth ing through graphic data using a heuristic
based on the notion of compression.
After researchers input a big graph into

H umans are fascinated by patterns, and they can spot them well—in fact, that’s one
area where humans excel over computers. But research is producing interesting
competition as scientists discover and employ new methods of automated pattern recogni-
the system and run a search, Subdue finds
a pattern that has several instances in the
graph. The system then replaces all of those
tion. Practical applications include finding genes, detecting cancer-causing chemicals in instances by a single node, making the graph
smaller. The larger the pattern, and the more
molecules, searching out potential terrorists, and predicting terrorist threat levels, as well instances it has, the more compression you
get. “The more it compresses, the more we’re
as recognizing speech patterns and creating In the factor graph, each variable is a interested in it,” Holder says.
a nanotechnology resource library. node. The scoring function comprises many A practical application of Subdue exam-
local scoring functions that look for a small ines a chemical structure to determine
No new genes? number of variables. For that small set of whether it causes cancer. The system repre-
At the University of Toronto, Brendan variables, it finds a score for each configu- sents the chemical in terms of its atoms and
Frey is leading a group of scientists who are ration of those variables. The local scores’ the bonds between them (the atoms are
using AI techniques to analyze molecular- sum is the total score. “It’s a nice way to nodes in the graph, and the bonds are links).
biology data. One of their projects involves decompose a very complex problem into a For the system to learn, researchers input
using a factor graph they developed called whole bunch of simpler problems,” Frey many cancer-causing chemicals as graphs,
GenRate to discover and evaluate genes in says. The scientists then compare the factor which the system searches to find recurring
mouse tissues. Factor graphs let researchers graph data to known gene patterns. patterns, or subgraphs. It then searches
describe a system with complex variables, Because the factor graph provides a com- through the space of subgraphs to find a
such as gene location in DNA as well as gene putational framework for vetting the best pattern that shows up a lot. That pattern is
length and function. configuration of variables as well as discov- then matched against the new chemical’s
“What a factor graph is useful for,” says ering them, the team came up with surpris- structure.
Frey, “is describing a scoring function that ing results that led to a major revision of the “The interpretation would be if this sub-
tells you how good each setting of the vari- view of the mammalian genome. Although molecule shows up in 90 percent of these
ables is.” some research claims many genes are left to chemicals that cause cancer, then it may be
Using samples from over 1 million probes discover, Frey’s team has shown that might predictive,” Holder says. So, if a new chem-
along DNA in 37 different mouse tissues, not be true. “Beyond the genes we found,” ical contains the subgraph, he says, “you
the scientists used their factor graph to deter- Frey says, “we don’t believe there exists might predict that this chemical may cause
mine which bits of DNA are expressed, or many new protein-coding genes.” cancer, and you may want to go off and test
activated to read protein. In some tissues, the it in the laboratory.”
DNA is expressed; in others, it might not be. Cancer detection Holder has tested the system in the Amer-
DNA parts that have no function are never At the University of Texas at Arlington, ican Cancer Institute’s predictive-toxicology
activated. Lawrence Holder has developed Subdue, challenge. The Institute releases information

4 1541-1672/06/$20.00 © 2006 IEEE IEEE INTELLIGENT SYSTEMS

Published by the IEEE Computer Society
on both a set of chemicals that it has deter-
mined to be cancer causing and a set that
isn’t. Participants speculate which chemi-
cals cause cancer, and the one with the
most correct guesses wins the challenge.
Holder won the competition in 2000.
Terrorists targeted
Subdue is also useful for detecting pat-
terns of potential terrorist activity and locat-
ing potential terrorist networks. Holder
trained his system on simulated data that
the US Air Force’s Evidence Assessment,
Grouping, Linking, and Evaluation program
created. The domain simulates the evidence
available about terrorist groups and their
plans before they put them into action.
Following a general plan of starting a
group, recruiting members, acquiring
resources, communicating, visiting targets,
and transferring resources between actors,
groups, and targets, the domain contains
numerous concepts. The concepts include
threat and nonthreat actors and threat and
nonthreat groups.
Trained on patterns that give examples
of threat potential, Subdue searched the
simulated data to find similar types of pat-
terns. The system achieved 78 to 93 percent
accuracy discriminating threat from non-
threat groups.
Language learning
Cornell University professor Shimon
Edelman, in collaboration with colleagues at
Tel Aviv University, has created a program
that can discover patterns in languages,
learn them as grammars, and then generate
sentences of its own in that language. The
system is called ADIOS (automatic distilla-
tion of structure), and it has been tested on
both natural languages, such as English and
Chinese, and artificial grammars, such as
those in DNA and music.
“You can only recognize patterns if you
have the right primitives, the right features,”
Edelman says. “It’s like having the right
glasses.”
Language contains patterns that on its
face are invisible, and it’s generally thought
to possess structure beyond just the serial
order of words, a grammar. “The true struc-
ture of the sentence is a kind of tree,” Edel-
man says. ADIOS combines statistics and
rules applied to a body of text in a language
to discover the grammar. The system can
then generate sentences in that language.
“It can do things like assign structure to a
JANUARY/FEBRUARY 2006
R
Ga O C
O Ga
C O
C O
O Ga
Ga
R
The Subdue pattern-recognition system searches through graphic data to find a
pattern that has several instances. In this chemical-structure example, the atoms are
nodes in the graph and the bonds are links in the graph.
new sentence,” Edelman says. “It’s not such
a big deal to recognize a pattern on which
you’ve trained your system.” The team is
patenting the technology, and Edelman
wants to put it to commercial use. One possi-
ble arena is speech recognition technologies.
Patterns in theory
Research on the theoretical aspect of
pattern discovery is also generating useful
applications. At the University of Califor-
nia, Davis, Jim Crutchfield has leveraged
his interest in “what a pattern is” to apply a
pattern’s abstract definition—which he
calls a causal state—to different kinds of
processes. He defines causal states as groups
of histories that lead to the same knowledge
about the future.
The mathematical theory that defines the
causal states leads to a small number of
possible ways to find the causal states. The
mathematical definition of being in the
same state of predictability about the future
means that, by looking at data, you can
estimate and make predictions about the
future on the basis of different points in
time having different histories. From that
definition, Crutchfield derived an algorithm
that describes how to group histories that
provide knowledge about the future when
those histories are basically predictively
equivalent. “We just apply this in these
different domains,” he says, “whether it’s a
spatial pattern, like cellular automata, or time
series, or looking at complex materials.”
Crutchfield has applied his causal-state
www.computer.org/intelligent
R
O Ga
C
O Ga O CH CH
O
R O C
R C O R
O Ga O CH
C C
Ga O
O C R
R
R
O Ga O
O
C Ga
C O
O
R
definition to examining dynamic systems,
irregular crystals, hidden Markov models,
and cellular automata. One field of applica-
tion is quantum computation.
“A current proposal for implementing
molecular computers is to look at very long
chain molecules and to design the interac-
tions between the atoms in the molecular
chain so that they implement various of
these cellular-automata rules,” Crutchfield
says. “So this pattern discovery system that
we have for cellular automata is making a
catalog of all the possible kinds of interac-
tions and what sorts of information storage
structures they can produce, and how those
information storage structures can be moved
around and interacted, and how they interact
to process information.”
Crutchfield is working on developing a
library called the Encyclopedia of Cellular
Automata. “It will be a resource for people
working in nanotechnology,” he says, “to
look at how to design molecular systems that
have only local interactions but that will pro-
duce in their behavior large-scale structures
that can be used for doing computations.”
A ccording to Nello Cristiani, associate
professor of statistics at UC Davis, people
have always been attracted to patterns and
pattern recognition. “In a way, this is the
essence of science and most cognitive
processes, such as generalization,” Cristiani
says. “Now this activity has been automa-
5
 IEEE

tized, and we rely heavily on it as a society. seen a revolution in pattern recognition

“There would be no genome project, no technology. Machine learning algorithms
speech recognition, and probably no credit are now faster, simpler, and more accurate
card system without it. The last decade has in generalization.”

IEEE Computer Society

Publications Office
10662 Los Vaqueros Circle, PO Box 3014
Los Alamitos, CA 90720-1314

“Sassy” Chatbot Wins

STAFF
Lead Editor
with Wit
Dennis Taylor
dtaylor@computer.org Benjamin Alfonsi
Group Managing Editor
Crystal R. Shif

cshif@computer.org
Senior Editors
Shani Murray, Dale Strok, and Linda World
Staff Editor
Rita Scanlan
Editorial Assistants
Brooke Miner and Molly Mraz
Magazine Assistant
Hilda Hosillos
Contributing Editors
Keri Schreiner and Joan Taylor
Design Director
Toni Van Buskirk
Layout/Technical Illustrations
Carmen Flores-Garvey and Alex Torres
Publisher
Angela Burgess, aburgess@computer.org
Assistant Publisher
Dick Price
Membership/Circulation Marketing Manager
Georgann Carter
Business Development Manager
Sandra Brown
Senior Production Coordinator
Marian Anderson
Submissions: For detailed instructions and formatting,
see the author guidelines at www.computer.org/intelligent/
author.htm or log onto IEEE Intelligent Systems’ author
center at Manuscript Central (www.computer.org/mc/
intelligent/author.htm). Visit www.computer.org/intelligent
for editorial guidelines.
Editorial: Unless otherwise stated, bylined articles as well as
products and services reflect the author’s or firm’s opinion;
inclusion does not necessarily constitute endorsement by
the IEEE Computer Society or the IEEE.
W
hen you sit down for an online
chat with computer scientist
Rollo Carpenter, you’re not quite sure if
it’s him on the other end or his virtual alter
ego, a chatbot named George. And that, in a
nutshell, is the point of Carpenter’s research.
Carpenter’s work is inspired by Japanese
roboticist Masahiro Mori’s Uncanny Valley
theory. In 1970, Mori asserted that, as robots
become increasingly human-like in appear-
ance, movement, and behavior, they will
illicit emotional responses from human
beings that border on human-to-human
empathy levels.
In discussing digital immortalization and
a cultural renaissance by way of virtual
reality, Carpenter views advanced chatbots
as harbingers of a new era in machine learn-
ing. “We intend to get very close indeed to
the Uncanny Valley,” he says.
Bringing home the bronze
During the 2005 Loebner Prize contest
(www.loebner.net), a panel of judges found
George to be the most convincing conver-
sationalist of the four chatbot participants,
which included reigning three-time cham-
pion Alice. The contest, launched in 1990
by Hugh Gene Loebner and touted as “the
first formal instantiation of a Turing Test,”
gauges the contestants’ “intelligence” levels.
In the contest’s 15 years, no contestant
has won a silver or gold medal, awarded
for convincing at least half of the judges
that a text-based program or virtual per-
sona is actually real. However, every year a
bronze medal and a cash prize have gone to
the most human-like program. This year,
Carpenter’s Jabberwacky program, which
hosts George, brought home the bronze.
Now, George has caught the interest of the
computer science community as well as
thousands of visitors to the Jabberwacky
site (www.jabberwacky.com).
Considering context
Many of the site’s visitors find it hard to
believe George isn’t human after convers-
ing with him. Even some of the contest’s
four judges were fooled—at least initially.
“It [becomes] crystal clear within a couple
of lines of communication who is human
and who is bot,” according to one judge, Lila
Davachi, assistant professor of psychology
at New York University. “However, I found
Jabberwacky to be the most interesting bot
by far because it displayed some very human
qualities. It was sassy and playful.”
What makes George seem so human? It
appears George is different from other
chatbots not only because of his personal-
ity but also because of how he learns. Car-
penter says machine learning is trending
toward being statistical and probabilistic,
relying on analyzing significant volumes
of data.
“Most of the chatbots that exist today
work in a hard-coded, entirely predictable
fashion,” he says. “A series of ‘if’ (or equiv-
alent) statements created by the program-
mer evaluate the input and return known
results, either as whole sentences or as
modifications of the input.”
Although George is also statistical, prob-
abilistic, and data intensive, the bot pro-
gram is also something else: chaotic. “It
never turns probabilities into numbers,”
explains Carpenter. “It avoids looping
through data, summing up an estimate of
‘fitness for purpose.’”

6 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS


For George, context is key. Output is
based on an interpretation of the current
context—taking the current conversation
into account, while comparing it to past
conversations.
“Techniques for finding context within
complete conversations are the real key to the
AI’s success to date,” says Carpenter. “The
context finding tends to do a lot inaccurately,
allowing influences from all sorts of quarters
even if they may be individually only par-
tially relevant and often even irrelevant.”
He says tiny differences in context, how-
ever seemingly unconnected, can give the
program a reason to choose one thing over
another, and those differences are infinitely
better than randomness.
Many people believe George is human
because he mimics human behavior, which
is contextual, rather than human responses,
which might or might not be. For example,
someone testing the program might ask,
“What color is a red apple?” A programmer
could easily add a rule that deals precisely
with that question, but George formulates
the answer to such a question on the basis
of what past users have said to him. Most
humans would find the question inane and
might respond sarcastically, “Pink with
yellow and white polka dots.” So, this is the
kind of answer that Carpenter’s program is
likely to give—what some users see as wit.
Just as humans typically increase in intel-
ligence from infancy through adulthood,
learning from each person with whom they
come into contact, so does George—who
his creator says is still “a child.”
“As the data set grows,” Carpenter says,
“there are ever-improved chances of accurate
overlaps between the current conversation and
its predecessors, so it becomes increasingly
able to make intelligent connections.” And as
George’s conversational level improves, peo-
ple are more willing to engage in sensible,
consistent, and valuable dialogue. The result?
Data continues to improve over time.
Looking to the future
In discussing how George is in many
ways his thumbprint, Carpenter suggests
that advancements in robotics might take
the cloning debate out of the biology labs
and into the computer labs. Whatever soci-
etal and sociological implications would
exist in the wake of widespread prolifera-
tion of highly evolved chatbots, the computer
scientist is confident about the advancements
in AI they represent.
JANUARY/FEBRUARY 2006
IEEE
How to Reach Us
Writers
For detailed information on submitting articles, write
for our Editorial Guidelines (isystems@computer.org)
or access www.computer.org/intelligent/author.htm.
Letters to the Editor
Rollo Carpenter’s chatbot, George, took Send letters to
home the bronze medal in the 2005
Dennis Taylor, Lead Editor
Loebner Prize contest.
IEEE Intelligent Systems
10662 Los Vaqueros Circle
“Ultimately, this is a form of digital im- Los Alamitos, CA 90720
mortalization, clearly not a complete one, dtaylor@computer.org
yet much more immediate and closer to a Please provide an email address or daytime phone
person’s ‘being’ than the more classic tech- number with your letter.
nique of writing an autobiography,” he
says. “Only one person trained George—
me. So, George does gradually become my On the Web
reflection, or a reflection of the persona I Access www.computer.org/intelligent for information
project, in terms of speech patterns, charac- about IEEE Intelligent Systems.
teristics, and interests.”
But, Carpenter says, anybody can create
Subscription Change of
his or her own chatbot at Jabberwacky, just
as he has—hence his prophecy that Mori’s Address
Uncanny Valley is approaching. Send change-of-address requests for magazine subscrip-
tions to address.change@ieee.org. Be sure to specify
IEEE Intelligent Systems.
Membership Change of
C arpenter has partnered with Televirtual
in creating a 3D animated character with
Address
Send change-of-address requests for the membership
voice input and custom voice output. (The directory to directory.updates@computer.org.
partnership has spawned the current image
of George.) He has also commenced col-
laborative research with other computer Missing or Damaged Copies
scientists to construct a highly lifelike robotic If you are missing an issue or you received a damaged
head that imitates those who interact with copy, contact membership@computer.org.
it in voice, movement, and facial expres-
sion. Think of it as George, version 3.0.
Reprints of Articles
“Imagine a Jabberwacky with millions
of characters like George in place of For price information or to order reprints, email
today’s handful, operating with Google- isystems@computer.org or fax +1 714 821 4010.
scale serving capacity,” says Carpenter.
“Almost certainly it will be shockingly Reprint Permission
realistic, whether or not it passes a formal
To obtain permission to reprint an article, contact
Turing Test.”
“Shockingly? I doubt it,” says Davachi. William Hagen, IEEE Copyrights and Trademarks
“But that is an empirical question; I’ll wait Manager, at copyrights@ieee.org.
and see.”
www.computer.org/intelligent 7