You are on page 1of 531


Complex Adaptive Systems

John H . Holland , Christopher Langton, and Stewart W. Wilson, advisors

Adaptationin Natural andArtificial Systems : An IntroductoryAnalysiswith Applicationsto Biology,
Control, andArtificial Intelligence
, John H . Holland

Towarda Practiceof AutonomousSystems : Proceedings
of theFirst EuropeanConference
on Artificial
Life, edited by FranciscoJ. Varela and Paul Bourgine

GeneticProgramming:On theProgrammingof ComputersbyMeansofNatural Selection
, JohnR. Koza

From Animals to Animats 2: Proceedings of the SecondInternationalConferenceon Simulation of
AdaptiveBehavior, edited by Jean-Arcady Meyer, Herbert L. Roitblat, and Stewart W. Wilson


.of the SecondInternationalConference
on Simulationof AdaptiveBehavior

edited by ] ean-Arcady Meyer, Herbert L. Roitblat , and Stewart W. Wilson

A Bradford Book

The MIT Press
Cambridge, Massachusetts
London, England

Instituteof Technology

ThUbook was printed and bound in the UnitedStates
of Amerl ~ .

Libraryof Congress -in-Publication
Cataloging Data

InternationalConferenceon Simulationof AdaptiveBehavior(2nd : 1992: Honolulu, Hawaii)
From animalsto animats2 : proceedingsof the SecondInternationalConferenceon Simulationof
Adaptive Behavior/ editedby Jean-ArcadyMeyer, HerbertL. Roitblat, and StewartW. Wilson.
. cm. - (Complexadaptivesystems )
/IA pBradfordbook
Includesbibliographicalreferencesand index.
ISBN0-262-63149 -0
1. Animal behavior- Simulationmethods- Congress es. 2. Adaptation(Psychology ) - Simulation
methods- Congress es. 3. Robotics. - Congress es. I. Meyer,
es. 4. Artificial intelligence- Congress
Jean-Arcady. u. Roitblat, H. L. m . Wilson, StewartW. IV. Title. V. Title: From animalsto animatstwo.
VI. Series.
591.51- dc20 93-12410

. 0262631490 .
. '
To 2


Preface . .. . .. . .. . . .. . .. .. ... . . .. ... .. . .. . . .. . .. . . ... . . .. . .. . .. .. .. .. ... .. ... . . .. .. . .. . .. .. . . .. . .. ... .. .. ... .. ... .. .. . .. . . ... .. .. . .. .. ... . . .. . .. . . ... .. .. . .. . .


Behavior-BasedArtificial Intelligence ..................................................................................................... 2
Environment Sb"uctureand Adaptive BehaviorFrom the Ground Up .............................................. 11
PeterM . Toddand StewartW Wilson

Evolutionary Wanderlust: SexualSelectionwith Directional Mate Preferences.............................. 21
GeoffreyF. Miller and PeterM . Todd
Designing Efficiently Navigating Non -Goal-Directed Robots........................................................... 31
Rolf Pfeiferand Paul F. M . J. Verschure

Anuran Visuomotor Coordination for Detour Behavior : From Retina to Motor Schemas ............ 42
MichaelA. Arbib andHyun BongLee
Artifidal Neural Nets for Conb' olling a 6-Legged Walking
System I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Holk Cruse, UweMillier - Wilm, andJeffreyDean
A Neural Network BasedBehavior Hieran: hy for Locomotion Conb' Ol........................................... 61
Sunil Cherianand WadeO. Troxell
A Qualitative Dynamical Analysis of Evolved Locomotion Conb' Oilers.......................................... 71
JohnC. GallagherandRandallD. Beer
Neuronal ParameterMaps and Signal Processing................................................................................ 81
RichardA. Altes

Representationand Processingof Acoustic Information in a Biomimetic Neural Network ......... 90
HerbertL. Roitblat, Patrick w: B. Moore, DavidA. Helweg, and Paul E. Nachtigall
An Integrated Computational Model ofa Pen:eptual-Motor System............................................. 100
Mlliam R. Uttal, ThomasShepherd .
, SriramDayanand , andRobbLovell
ReactiveBehaviorsof Fast Mobile Robotsin Unsb"uctured Environments:
Sensor-BasedConb' Oland Neural Networks .................................................................................... 108
R. Zapata, P. Upinay, C. Novaies,and P. Deplanques
~ e Adaptive Nature of 3D Perception ................................................................................................ 116
Allen Brookes

Propulsion and Guidance in a Simulation of the Worm C. Elegans................................................. 122
A Simple, Cheap, and Robust Visual Navigation System................................................................. 129
Ian Horswill



The Use of Hieraochiesfor Action Selection........................................................................................ 138
Two Methods for HieraochyLearning in ReinforcementEnvironments ......................................... 148
Mark Ring
Should I Stay or Should I Go: Coordinating Biological Needs with
Continuously updated Assessmentsof the Environment ............................................................. 156
LianeM . Gabora
Extensionsof the Associative Control Process(ACP) Network :
Hier ~ hies and ProvableOptimality ............................................................................................... 163
LeemonC. Bairdill and A. Harry Klopf
Behavior Networks and FooceFields for Simulating Spinal ReflexBehaviorsof the Frog ........... 172
The Ariadne ' s Dew Algorithm .............................................................................................................. 182
EmmanuelMazer, JuanManuel Ahuactzin, EI-Ghazali Taibi, andPierreBessie re
Dynamic Selectionof Action Sequences............................................................................................... 189
FelizRibeiro, Jean-Paul Barthes,and EugenioOliveira
Planning Simple TrajectoriesUsing Neural SubgoalGenerators..................:.................................. 196
JUrgenSchmidhuber and ReinerWahnsiedler
A Note on Rate-SensitiveHabituation .................................................................................................. 203
J. E. R. Staddon

Catego ~ tion , Representations , and The Dynamics of System Environment Interaction :
A Case Study in Autonomous Systems ............................................................................................. 210
Paul F. M . J. Verschureand Rolf Pfeifer
A Directional Spreading Activation Network for Mobile Robot Navigation ................................. 218
David Kortenkamp and Eric OLown

Memorizing and Rep ~ enting Route Scenes ...................................................................................... 225
Saburo Tsuji and Shigang Li
Building Long Range Cognitive Maps Using l . oca1Landmarks ...................................................... 233
Tony J . Prescott and John E. W Mayhew

Dynamics of Spatial Navigation : An Adaptive Neural Network ..................................................... 243
NestorA. SchmajukandH . T. Blair


Modeling Nervous SystemFunction with a Hieran: hical Network of
Control SystemsThat Learn ................................................................................................................ 254
A. Harry Klopf, JamesS. Morgan, and ScottE. Weaver
An Optimization -BasedCategorization of Reinfon:ement Learning Environmen~ ..................... 262
MichaelL. Littman


ReinforcementLearning with Hidden States....................................................................................... 271
Long-Ji Lin and TomM . Mitchell
Efficient Learning and Planning within the Dyna Framework ......................................................... 281
Jing PengandRonaldJ. Williams
IncreasingBehavioural Repertoirein a Mobile Robot ........................................................................ 291
Wrich Nehmzow , Tim Smithers, and BrendanMcGonigle
Learning Biped Robot Obstade Crossing ............................................................................................. 298
ThomasUlrich Vogel
Learning to Conb' Olan Autonomous Robot by Distributed GeneticAlgorithms ......................... 305
Marco ColombettiandMarcoDorigo
Temporary Memory for ExamplesCan SpeedLearning in a Simple Adaptive System................ 313
LawrenceDavis, StewartWilson, and David Orvosh

Implementing Inner Drive Through CompetenceReflection............................................................ 321
AlexanderLindenandFrank Weber

Dynamic Flight Conb' Olwith Adaptive CoarseCoding .................................................................... 327
BruceE. RosenandJamesM . Goodwin
Learning via TaskDecomposition .................................................................................................:....... 337
, JonasKarlsson
JoshTenenberg , andStevenWhitehead

Neural Networks with Motivationa JUnits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346

Federico Cecconiand Domenico Parisi

EvolutionaryLearningof Predatory BehaviorsBasedon Sb'UcturedClassifiers.......................... 356
Hitoshilba, HugodeGaris
, andTetsuya
Issuesin EvolutionaryRobotics..~.......................................................................................................... 364
, PhilipHusbands
, andDaveCliff
Evolving Visually Guided Robots ......................................................................................................... 374
DaveCliff, Philip Husbands
, andInmanHaroey
An Evolved, VISion-BasedBehavioral Model of Coordinated Group Motion ............................... 384
Craig w: Reynolds
Evolution of Herdine: Behavior in Artificial Animals ......................................................................... 393
. .

M. Werner
Gregory andMichael
G. Dyer
An Evolutionary Approach to Cognition ............................................................................................. 400
Dwight DeugoandFranz Oppacher
Emergenceof Nest-BasedForaging Strategiesin Ecosystemsof Neural Networks ...................... 410
Dario Floreano

- Hardwarewith GeneticLearning
A First StepTowards Building a Darwin Machine ......................................................................... 417
TetsuyaHiguchi, TatsuyaNiwa, ToshioTanaka , Hitoshi Iba, Hugo deGaris, and TatsumiFuruya
Evolving Artifidal Insect Brains for Artifidal Compound Eye Robotics......................................... 425
Luis R. LopezandRobertE. Smith

viii Contents


Designing EmergentBehaviors: From Local Interactions to Collective Intelligence ...................... 432
MajaJ. Mataric
Adaptive Action Selectionfor CooperativeAgent Teams.................................................................. 442
LynneE. Parker
From Tom Thumb to the Dockers: SomeExperimentswith Foraging Robots............................... 451
Alexis DrogoulandJacques Ferber
Collective Robotic Intelligence .............................................................................................................. 460
C. RonaldKubeandHong Zhang
Collective.Choice of Strategic1Ype....................................................................................................... 469
ChisatoNumaokaand AkikazuTakeuchi
An Adaptive Communication Protocol for Cooperating Mobile Robots........................................ 478
Holly Yancoand Lynn AndreaStein
Dimensions of Communication and SocialOrganization in MultiAgent Robotic Systems........ 486
RonaldC. Arkin andJ. David Hobbs
Evolution of Trading StrategiesAmong HeterogeneousArtifidal EconomicAgents " " " " "" " " " 494
AndreaBeltratti and SergioMargarita
Action Selectionand Learning in MultiAgent Environments .......................................................... 502

Structure from Assodative Learning .................................................................................................... 512
JohnH. Andreae , Shaunw: Ryan, andMark L. Tomlinson
The Roots of Motivation ......................................................................................................................... 513
Learning Continuous SpaceNavigation Heuristics in Reallime .................................................... 514
GregoryD. Bensonand ArmandPrieditis
The Adaptive Power of Affect: Learning in the SESAMEAochitecture.......................................... 515
Eric Chown
Modelof a BehaviourBasedConb'OlArchitecture............................................................................. 516
Lu(s CorreiaandA. Steiger -Gart;ifo
ComparingRobotandAnimal Behavior......................................................""."""""""""""" """""" 517
An EmbodiedNeurally-BasedAlgorithmfor OptimalActionSelection........................................ 518
OwenHollandandMartin Snaith
Why ShouldWeBuild Artificial WormsandHow?............................................................................ 519
CreativePerception..........................................................................................:...................................... 520
M. A. RodriguesandM. H. Lee
CollectiveBehaviorof SiliconMicrorobots.......................................................................................... 521
IsaoShimoyama , ToshioWatanabe , Yoshz1tiko Kuwana , andHirofumiMiura
An AnalogVLSIModelof CentralPatternGenerationin the Medidnal Leech............................ 522
MicahS. Siegel

Author Index ............................................................................................................................................ 523


On December7- 11, 1992, in Honolulu , Hawaii , just steps from Waikiki Beach, 125researchers
from Europe, North America, and Japan met to report from , debate, and push forward the frontier
of an exciting new approach to understanding intelligence: the simulation and building of
artificial animals (' animats' ) which must survive and adapt in progressively more challenging
environments. Representingfields as diverse as ethology, psychology, connectionism, evolutionary
computation , and robotics, the partidpants shared the belief that the creation of whole ,
- - -
coping , animal like systems however simple at the moment may be one of the best routes to
in both natural and artifidal systems. The conferencewas entitled
" From Animalsintelligence
to Ani mats: The Second International Conference on Simulation of Adaptive
Behavior" (SAB92) . It succeededone held with equal enthusiasm in Paris in September, 1990.
These proceedings contain 59 papers and constitute the most comprehensiveand up -to-date
source on the field . The book is divided into sectionscorresponding to the conferencesessions.
In each section, papers presented as talks are followed by related papers that were presented as
posters. (Eleven additional contributions could not be included in full in this book, but were
given as posters and are representedin a final section by summaries.)
The first section, The Animat Approach to Adaptive Behavior, contains papers on the nature
of the animat enterprise, on simulating the evolution of adaptive behavior with minimal prior
assumptions, and on sexual selection a significant and underrated adaptive force.
Papersin Perception and Motor Control addressfrom diverse anglesthe question of obtaining
useful percepts from the available sensory profusion , and report on adaptive locomotion , espe-
dally walking .
The need for practical action hierarchies is brought out by several papers in Action Selection
and Behavioral Sequences. Others addressaction choice in relation to motivation and how plans
can emerge from the interplay of simple computational units.
In the section on Cognitive Maps and Internal World Models several papers show how local
information can yield large-scale maps of the environment. Other papers concern extraction of
' '
landmarks from route data, and the kinds of internal representations that localist behavior
control produces.
The Learning section contains leading-edge papers on hierarchy, internal states, experiential
memory, and assodative extension of existing competences. The aim of understanding learning
problems better is furthered in a paper offering a perspicuous categorization of learning environments

The Evolution section contains papers on evolving - rather than designing- robots, and on
evolution of behaviors, including group behaviors, that satisfy multiple fitness demands.
The section on Collective Behavior has papers on identifying primitive individual competences
which permit principled design of group behavior, adaptive cooperation among relatively
high -level ani mats, and emergenceof explicit communication.

SAB92could not have taken place without the assistanceof many people and organizations.
We are especially grateful to members of the Program Committee, whose conscientious reviewing
selectedthe papers here from the 105submitted. The Committee members were:

We invite readers to enjoy and profit from the papers in this book. Jean. David Helweg . we are particularly indebted to Gilbert Roger for the artistic conception of the SAB92 poster and the Proceedingscover. Philippe Tarroux. University of Cambridge. MITRE Corporation . Universite Paris VI . Osaka University . The Open University . Saburo Tsuji. MIT Artificial Intelligence Lab. USA Rodney Brooks. Universite Libre de Bruxelles. for their support . Wilson ConferenceCo. UK Peter Todd. and Satoru Yamamoto.Arcady Meyer Herbert L . England. USA Luc Steels. France Lashon Booker. Constance Susan Manos. and Brandeis University . Canadian Museum of Nature . USA David Waltz. Japan William Uttal . The members of the Organizing and Local Arrangements Committee worked long and hard to make the conferencea success. Finally. UK JacquesFerber. University of Hawaii . especially Karl Minke and Sheryl Nakahara. We thank Yvonne Yamashita of the University of Hawaii ConferenceCenter for handling the fiscal arrangements for the conferenceand the University of Hawaii Department of Psychology. Belgium Paul Nachtigall . Patrick Moore. Thinking Machines Corp . USA Pab'ick Colgan.Chairs . Aloha . Belgium Richard Sutton. The Rowland Institute for Science. Universimt Konstanz. France Simon Goss. Roitblat Stewart W . Pierre Vincens. USA Frederick Toates. GTE Laboratories. USA . SAB94. We thank Agnes Guillot . Germany Anthony Dickinson . Arizona StateUniversity . 1994. in Bright on. and look forward to the next conference.Preface Alain Berthoz. VUB AI Lab. College de France. Canada Juan Dellus. USA We thank each of the following conferencesponsors: Air Force Office of Scientific Research DefenseAdvanced ResearchProjectsAgency Ministere de l ' Education Nationale et de la Culture Office of Naval Research and are grateful to John Tangney for his continuing intellectual support and excellent advice. which is planned for August 8-12. Heidi Harley.


AI " versus " up top- ' down AI " . For the sake of clarity with multiple . Brooks . Both approach es attempt to model ( 1991) . Wilson ' s account ( 1991) study of Intelligence . unpredictable environment . At the same moment at which The paper is structured as follows . A second reason is that this account is differentand can answer questions about this problem domain . Goals of the Two Approach es to AI new techniques and applies a set of different criteria for success.Based AI - argues that the behavior based approach is appropriate can be characterized as typically studying systems with for the class of problems that require a system to the following characteristics : . that new life is being brought to the field .Based AI to the study of to do. Finally section 7 contains some what Behavior . Based AI is to synthesize computationalforms of intelligent among others Brooks ( 1991) .Based approach . Behavior . the solutions investigated by the Knowledge . This paper attempts to describe the two approach es. the solutions performed so far . It distinguish es this approach focuses on a scientific methodology forAnimat research. changing goals in a dynamic . This paper presents a more general perspective .mit .al . In particular . There are several reasons for giving it yet another intelligent phenomena such as goal .Based AI is about and how it differs from critical comments about the progress made so far . 1991) . being one involving extensive problem solving and reasoning . Section 2 distin - the popular . it is dangerous to draw solid lines robots . or also which underlie the Behavior . many insiders believe that something exciting is with those of Behavior . Some claim that it isn ' t very differentKnowledge. These systems model the domain scientific . from the traditional Knowledge. investigates interesting 2 . process scheduling ( Malone Intelligence ( AI ) as a new approach to the et. adopted and the criteria used for success. 1992b) .based AI " as . about the approach . presents Behavior . learning .Based AI .Based AI poses problems in a different way. try .directed behavior . First of all many researchers are still sceptical prediction .Based AI . unpredictable the remainder of the environment .Based Approach The new wave has been termed " behavior .Based AI and Behavior - Several people have tried to define Behavior . and so on . Why this paper? situated along the continuum between the two extreme positions presented here and that those intermediate positions Since 1985. paper presents Knowledge . . often from the papers listed above. of the main originators of this new approach .Based approach while Meyer ( 1991) aims to give an overview of the research in terms of the questions studied . It More specifically . Section " bottom . general belief is that AI has been a "failure guishes the " goal and emphasis of Knowledge . communication and cooperation . 1988) . The next chapter lists some of the key insights " " opposed to main . The reader should keep in mind that concrete examples of research are often 1. some problem domain .Based AI and Behavior .Based AI .stream Abstract autonomously fulfill several goals in a dynamic . The goal of both Knowledge .based AI .ng of systems that " behave" in intelligence ( Brooks . Section 3 discusses happening . Others " " modeling and building of systems that know about are still not convinced that the approach is founded and some problem domain . Knowledge. Finally . MA 02139 Pattle@media.Based Artificial Intelligence Pattle Maes MIT Media-Laboratory 20 AmesStreet Rm 489 Cambridge. the term 5 elaborates upon the solutions adopted by Behavior - " animat " approach . anew wave has emerged in the study of Artificial are valid ones to adopt . This includes applications such as This paper attempts to define Behavior .Based as two extremes . " main .Based Artificial interface agents ( Maes.based AI on the other hand has emphasized a picture which is restricted towards robotic forms of the modeling and buildi . Section 6 discusses some examples contrasting is also frequently " Knowledge. Based AI . It argues that Behavior . Wilson ( 1991) and Meyer systems. but rather presents it as a general approach between the different approach es and to force examples for building autonomous systems that have to deal of research to fit into one of them . which was coined by Wilson ( 1985) . Intelligence ( AI ) . It Defining a new approach is a difficult and tricky thing does not limit Behavior .Based AI .Based AI has traditionally emphasized the from what they have been doing all along .

All of the internal about its problem domain (such as how to solve particular structures . Their only connection very dynamic . The " problem " or . neither approach has shown " the description of the solution produced much success dealing with the problem classes concentrated by the central system in the problem domain . system which has all of the above characteristics is One can conclude from the above discussion that it is augmented with a perception module and an execution hard to compare the Knowledge . planning to be emergent observable properties . They also do not have to deal with multiple behavior of the system . there is a strong emphasis on " adaptation " - in the first place and how they should change over and on a " developmental approach " . They have declarative " knowledge structures " . It is 4. Finally . Typically it has to deal with many conflicting though the user might ) and 'do not have to deal with goals simultaneously . This often means time . like reacting in a market system by simple Behavior. chess playing . describes it to also very complex . For a robot these are competences such as locomotion - Knowledge Based AI . collecting objects . They agent ( Maes. etc. Unpredictable events happen the system in the symbolic language which the system all the time .level competences for the central system .driven computation versus autonomous systems goal is usually still specified in symbolic terms by a . It is directly connected to its problem domain through 2. Often . In the few cases where an autonomous system ( e. They are not usually concerned with the developmental aspect. to be seen whether either one will be able to broaden In contrast . The perception module has to recognize the classesof problems : knowledge versus behavior . 1992) . in which case the interpreter uses the static those that are responsible for particular aspects of the knowledge structures to determine the solution to the behavior . etc ) . The emphasis is on autonomy : the system is completely self. They do not have to be initiated by a goal formulation from the user. which cannot be attributed to particular internal structures . They are " closed" . For other systems these might be other simple competences The difference between Knowledge-Based AI and . interrupts . approach is taken : the user gradually evolves a more sophisticated system by adding structure to an 6. a central already existing working system . etc) . often advanced. which less important that the system can answer questions model aspects of the domain of expertise . or how the knowledge structures got there 5. They are given one problem are active " behavior producing " modules as opposed at a time by the user. it is acceptable for goals or problem . apart from an interpreter are static .contained .g . of an answer or solution . a " " robot or an interface agent) is being modelled . Based Artificial Intelligence 1. It has multiple integrated competences.Based and Behavior - module which take over part of the role of the human Based approach because they typically study different interface . It has to monitor the domain 3. to static " knowledge structures " . 2. Its internal structures problems simultaneously .g . the emphasis is on the resulting .level ( as opposed to expert 3 . They do not have to be adaptive to changing that the system improves its own structures ( and thus situations (components breaking down . survival . navigation . The user a limited amount of time to act . on by the other approach . So far . a single current situation and translate it into a symbolic description high . It is also less important that the user The system is only active when a problem is posed by is able to inspect the internal structures and identify the user. rather provide " depth " than " width " in their expertise . For example . It typically also involves other acting understands . they do and figure out by itself what the problem to be solved not have time constraints for solving the problemal . medical diagnosis. which then has to be implemented by the user in the actual domain . user. Solutions by investigated level) . but alsoin the techniquesand solutions that . in the sense that there is no direct sensors and effectors. It can affect or change this domain interaction with the problem domain about which they through its output . Both classesof problems are interesting human . They model isolated . and so on . 5. 1988) or are studied. They deal with one problem at a time .al . competences executing a simple routine in the case of an interface (e. As long as it remains . From the system s point of view the problem domain does not change while the system is computing Rather than on knowledge . Behavior . The execution module is responsible for " implementing in their own right . this means that an incremental optimization is incorporated . problems ) . which means that the system has with the environment is the user. next is. The problem domain is typically encode knowledge and solve problems .Based AI lies not only in the problems that bidding and buying behaviors ( Malone et . . Typically the competences are lower .level competence versus a range of low . The domain is usually recognizes a problem in the domain . The system then returns a symbolic descriptionagents ( human and / or artificial ) . Behavior Based AI has typically studied the its domain of success in significant ways. 1992b) ( Kozierok and Maes. The system is " open" or " situated " in its environment . 1. At the behavior ) over time based on its own experience in the most some form of knowledge compilation or knowledge environment . both types of following type of system : research are necessary and complementary . In other cases.

and so on. For example . ambiguous . desires ( or . They have that the same functional components can be used for different difficulty reacting fast to changes in the environment . The resulting systems can be inflexible . The central representation includes things such as beliefs (updated by operating in dynamic environments have run into perception .al. This is partly due to the needs to be adapted is the central representation . 1987) .build more knowledge and understanding of the situation than a complete and consistent model of the environment the former one) . For example . The central system evaluates the current situation ( as represented in the internal model ) (Chapman . They fail all its objects and relationships ) inside the system . which in situations that only differ slightly from the ones the system can rely on to predict how the problem can they are programmed for . the whole system The organization of the different modules within the system might break down . problems are the frame problem and the problem of Role of Learning non. This example of this approach in the area of interface agents. if the perception module cannot Organization make sense of the current situation .Based approach has produced several The intelligent system is decomposed along "functional successes. inconsistent . In practice it also proves to be hard to hand . The key issue on which emphasis is laid is a complete .Oriented Decomposition The Knowledge. It is hard to keep or problem solving produce a description of the solution track of object identities . The resulting systems tend to be brittle . natural languageassisting experts with the modelling of and reasoning communication ( the peripheral components ) . which remain different ways in which this situation can be affected so unsolved in satisfactory ways. the system . in particular in the area of knowledge systems modules " such as perception . degradation of performance as components break down . the example the literature on the Shakey project ( Nilsson. Model of Activity . also read and augmented by the inference the following problemsl . The resulting systems can be slow . In practice it proves to be hard to relate the symbols in being active or processing and changing the internal . 1984) or the more recent Ambler project ( Bares system might cache a plan for later reuse. . 1989) for examples of this approach in the study of is there learning of new information or correction of existingautonomous robots. learner . The hope is . Very seldom et. The modules take turns " " . They rely on the central representation experiments " as their means of the above approach for constructing autonomous systems interface .independent as possible . Several theoretical problems have come up . Learning typically consists of compilation or reformulation 1See for of what the system already knows. The only component which tunities or contingencies . Function . Sullivan and Tyler 1991 ( ) present an knowledge based on environmental feedback. often the sensor data are to the problem ( a plan or the answer to a question ) .monotonic reasoning . engine and the natural language component ) . planning in the domain ) to physical stimuli . correct internal model . a perfect copy of the world ( with . Another reason is that these systems are built on the assumptions that few or no unpredictable Role of Representation changes will happen . Another practical problem is that of combinatorial " Activity is modeled purely as the result of a deliberative explosions . Typically all of these functional components are as general and domain . Examples of such as to achieve the desired goals .independent They do not deal very well with unexpected oppor - planner . General planning and problem solving " have proven to be a computationally expensive process thinking process. The solutions typically adopted in mainstream implies that the programmer is completely responsible AI projects can be characterized as follows : for creating an initial complete and correct model . For a range of reasons learner . problem domains ( a general domain . representations Perception and inference first update the internal model (representing objects and relations the internal model ( beliefs and goals) . These modules are typically developed with the class of problems Behavior Based AI is interested " in .satisfactory when dealing - components ) . They do not show graceful be solved. a about a specific problem domain . erroneous .specific information such as heuristic quentiallayers before they affect the actions taken by knowledge . Finally either the execution module or a human implements the solution in the domain ( the latter one having . sequential processing of information and because of the high computational cost involved in maintaining a Approach model and doing general perception and planning .. because of the goals) and intentions (produced by planner ) . Pattle Maes are explored . planner and inference engine ( the central systems the approach has proven non . which fact that changes have to propagate through many se- contains domain . and uses a search process to systematically explore the . execution . After that . is completely sequential . etc ) . which the system can rely on to make predictions . Several which attempted to use independently .

for remindingan interaction loop . Behavior . The intelligent functions which are being modelled are to emergent complexity (see also Resnick . making its task much easier ( Horswill . A system which emergent functionality . 1992) . 1992) .Based mobile robot could use the strategy of closely following AI a person passing by.g . AI has concentrated on modelling systems in their context . 1987) . 2 Notice that I do not credit Behavior .Based Artificial Intelligence 4 . 1992) . 1991) . Building systems this idea in an integrated way ( rather than developing applies at several different levels ( Brooks . 1991) . behavior . Looking at complete systems changes the problems often As a consequence of the above ideas. Kozierok and Maes ( 1992b) report on to the problems of Knowledge.ambiguate whatever the other speaker wants for the wall following behavior . Simple interaction dynamics between the system and which can learn has to rely less on planning ( because its environment can lead to emergent structure or it can cache computed plans ) .directed action sequences can be an The complete system is part of some environment . ( Wilson characteristics which can be exploited by the system . modules is responsible for action selection . Therefore there is no need for the some activation / inhibition dynamics among the primitive agent to figure everything out by itself . For example . . Behavior . a components of the system . In Maes' networks ( 1990) . It is their interaction to convey. For example . it observable . which will produce the desired behavior . One module is responsible for steering the construction of an iterative . etc) . incremental solution the robot towards the wall when the distance to to a problem . a lot easier. He notes that the complexity and as such has less of a need for modelling of its behavior is more a reflection of complexity and inference. The environment the particular properties of the environment and find can also be used as an external memory (e. emergent property of the interaction dynamics is situated in some space. but also in of the system can lead to emergent structure or ' time . 1991b) : modules implementing these functions independently ) often makes the task a lot easier. This implies that there is between the environment and a reflex-guided less of a need for modeling . While traditional AI has concentrated on simulated .Based AI with the 3.Based AI is that interaction dynamics among simple components can lead 1. The Other agents in the same environment are dealing with action selection behavior is an emergent property of similar problems . For example . This observation is true at several levels: A second major insight of Behavior . Behavior . Time also allows for ( Mataric .Based in a favorable way. The environment usually has particular of environments ( Horswill . a system 1. a natural language system the wall is above some threshold while another module situated in time does not need to disambiguate every is responsible for steering the robot away from the utterance . Finally interaction dynamics between the component discovery of these insights.Based AI has built real A first realization is that viewing the problem of building systems which solve an actual ( small ) problem in a concrete an intelligent system in its context can make things environment .Based AI in response navigating into things . This means that often it is sufficient to study own best model " ( Brooks . For example . Interaction dynamics can lead to emergent complexity . This implies that the system can evolve itself so emergent functionality . It can go back and forth asking questions wall when the distance is below some threshold . so as to achieve the competence of in an office environment without bumping The methods developed by Behavior . a set of simple feedback or reflex the system which tasks still have to be performed mechanisms . Simon ( 1968) gives an example has sensors and actuators can perform tests in the environment of an ant on the beach. These Based systems. Mataric s wall - as to become better at its task . systems in a social system can lead to emergent struc - . because the " world is its person . Important Insights of Behavior . and which ones it already did perform ( Suchman One of the implications is that we need a better understanding . are grounded in two important to perform certain tasks by observing and imitating insights2 : users. A system which has sensors has an of environment than of its own internal complexity easier job disambiguating natural language utterances and postulates that the same might be true of human and so on. 2. dynamics which makes the robot follow walls reliably . Neither or making particular remarks which will help to gradually one of these modules is primarily " responsible" dis. 1991) . " " toy examples. none of the component Finally the system is typically also part of a society .following can be attributed some sort of artificial evolution ) .Based AI techniques some experiments in which interface agents learned listed above. Also part of a complete intelligent system . It also means that if we want to prove (offices consist of vertical walls and horizontal floors . we have to model these systems as well " habitat constraints " can be as their environments . if time and the particular following robot does not have a single component to task admits ( either through individual learning or which the expertise of wall . Simple interaction dynamics between the components The system is not only situated in space. exploited by the system. Agre ( 1991) shows how behavior as complex as goal. aspects about the resulting performance of Behavior - doors typically have a particular size.

flexible and fault . structure corresponding to " the plan " of the system . process. For in Maes' .al . Pattle Maes ture or functionality . but happens rather on of the system . and among competence Approach modules and the environment . They can be of a numeric . 1986) .based system is typically able to react very fast of beliefs . the system is able to switch to an alternative redundant .tolerant than programmed Role of Representation . For example they might index objects according to the features and properties that make them significant to . Important is that such emergent complexity is often more robust . The systems built are highly distributed . or simple tokens in a restricted language . a one.Based AI or analog nature . the system is also able to adapt quicker . of employing the environment as a source of information system ( 1990) several sequencesof actions are evaluated in parallel . This arbitration modules .oriented competencein ( Maes. These representations are not related and might be inconsistent multiple solutions in parallel . objective agent .Based AI . When one of them by several modules .contained .oriented module represents to environmental changes. None of the components First of all . procedural Behavior . Each of the competence modules is responsible Role of Learning for doing all the representation .Oriented Decomposition competence modules operate in parallel .. etc. Building an adaptive system that will develop . or simple suppression and inhibition Model of Activity signals.specific " " In section 3 we listed the techniques adopted by problem solving is performed in the perception part of a particular competence ( Steels. Often a lot of task . The communication is almost to changes in the environment or changes in the needs never of a " broadcast " nature .Based AI and discussed why they proved to 1992) ( Ballard . there is no central representation shared is more critical than another one.take. the system demonstrates a graceful degradationto integrate the information from different sensors into of performance . objective interpretation of the current situation in parallel . Since all of the components interact one coherent . food foraging trees. a general planner . " reasoning Learning and development are considered crucial aspects " execution etc . so that as soon as certain variables change. black of autonomous bidding systems solves the complicated boxes. Also in Malone s system ( Malone et . be inadequate for building autonomous systems situated ' in dynamic environments . 1991) . None of the modules is " in control " . They might employ completely different techniques task of process. Instead complex and goal . etc ) . This section contrasts these Organization techniques with those adopted by Behavior . Deneubourg ( 1991. of producing this complexity . Finally . Competence modules are self. top . . related to its particular competence. 1988) . 1988) several mappings of processes to machines can be said and declarative nature than those employed in to be explored in parallel . Part of the reason for this more pragmatic approach social construction of different concepts and methods is a pessimistic vision about whether it is possible ( Suchman . etc ( a view also expressed in (Minsky . Often the system explores whatever it needs to represent to achieve its competence .Based AI develops competence fuse multiple conflicting actuator commands . 1990b) (Chapman . 1985) ( Maes.independent modules . at all to build a general vision system . Instead every task .one basis.directed activity is modeled as an emergent property of the interaction among competence modules internally . modules which are an expert at ( and network might be a winner . etc. This is the case because none of the components is really in charge There is much less emphasis on modeling the domain . no general plan - ner . 1989) . Knowledge. Within one competence module of . For example . There is no general perception module . computation .al . anthropologists have studied the . . ofa Behavior . the task at hand (Agre .Based System ( Wilson . Knowledge. The system also does not attempt breaks down .down organized complexity .all network . 1986) . 1987) (Shrager and Callanan . Behavior . The representations ' within one module are often of a less propositional . the usage of representations is minimized in favor way doing things example . Solutions Investigated by than their identities . Malone ' s collection avoided. the best one determining the behavior of the ( and a determiner of action ) . .to. Because of its distributed operation . an obstacle avoidance module might need 1992) . some simple arbitration Instead of building general functional modules like perception method is often included in order to select or and planning . There is no internal There are no " general" or task . However . a simple messages ( rather than a common representation behavior . Each of the modules is directly connected to relevant sensors Activity is not modelled as the result of a deliberative and actuators .processor allocation ( Malone (even different hardware ) to ~chieve their competence et. All of the Task . 1990) or a hardcoded priority scheme as in . as are responsible for ) a particular small task . 1991) rather 5 . and it might do some very simple can produce emergent complexity such as a path to a computation to decide how an obstacle should be food source. Typically the messages consist of activation energy. These modules interface to one another via extremely ( Brooks .Based AI . 1992) describes one bit of information to represent whether an obstacle how social insects following simple local rules is perceived or not .

because they The agent is given an elaborate amount of knowledge do not attempt to maintain objective representations of about the problem domain by some Knowledge Engineer the environment . including a hierarchical specification of the subtasks . They are less brittle . They act ( 1986)( 1991) has argued convincingly in writing and in actual demonstrations . the user engages in .this robot could work as follows . tables . and so on . The model is used by the planning of the approach .g . The Knowledge . All of similar learning algorithm runs in different competence these modules operate in parallel . 1991) or learning by the individual corresponding to the different competences necessary for the task : a ( Maes. etc. The agent cannot deal with ( the user might change his/ her planner goes through a systematic search to produce a mind about what to do in the middle of things . 1991) (Kaelbling . a module for obstacle avoidance ( or even than compiling existing information .tex . the system would react very slow. if a UNIX user enters a command like " emacs " A Mobile Robot paper . 1992) ( Sutton .tex " . 1991) . the location and type ( often even is quickly outdated ( as the user' s ways of performing identity ) of other objects in . It attempts with this approach are exactly the same ones as those to update this model as often as possible . The execution module executes this plan not change when the environment or task changes ( e. a model of the tasks traditional search processes.dvi " .Based robot for the same task could be adding more structure to existing successful constructed in the following way ... while avoiding obstacles. is often considered a a list of actions which will according to the model fulfill better approach than building a static system which will both goals . and so on.based tasks. the system concentrates on learning new modeled as an emergent property of two to three lower- information (or behavior ) from its environmental . At run time . The learning algorithmsa couple redundant ones. Finally they are not prone to problems . control is returned to the evolution ( towards increasingly more sophisticated behaviorplanner . In an incremental way. the agent uses this Examples Contrasting knowledge to recognize the intentions and plans of the Approach es user. rather level modules ) .others . Knowledge .al . which of the two above robots fast . They have less trouble problem in the following way ( Sullivan and Tyler . the are going as predicted . ( 3) they incorporate redundant methods user as possible .dvi " and processes the different sensor data and integrates them the printing command " lpr paper . They are able to deal with unforeseen situations An Interface Agent (opportunities as well as contingencies) . systems ( Brooks . by incrementallyA Behavior . 1991) . The model for mobile robots ( cfr . Behavior-BasedArtificial Intelligence into one that achieves the tasks . Other systems employ artificial several modules would be implemented evolution ( Koza . . . The problems into a representation of the environment . while possible checking at certain points whether things a robot breaking one of its legs) . since are implemented in a distributed way : typically a this is a very critical competence) . and ( 3) they require less expensive computation . If not . using different sensors. 1991) ( Drescher. Experience sorts out which of priority scheme: the obstacle avoidance modules always these modules implements the competence in a more reliablehave priority over going through doors which has priority way ( Maes. This knowledge contains : a model of the user and of combinatorial explosions.g . the system deduces that the user is planning to produce a written document . All module to decide how to fulfill the goal of finding the sorts of unpredicted events might take place which the door in the current room . because they rely much more on the environment as a source of informationConsider the problem of building a " software agent" and a determiner of action . A simple arbitration modules . Three the Two these tasks . action . or " interface agent" which assists the user with certain because ( 1) none of the modules is more critical than the computer . because ( 1) they have less layers of information will be more able to fulfill the task in a robust and reliable processing. ) is simulated by the programmer .Based AI approach es this and (4 ) they adapt over time . In some systems. amodule ( Drescher. relating representations to sensor stimuli . Brooks Systems built using the above principles suffer less from the problems listed in section three.this environment such as tasks change) . 1991) . It then plans Consider a mobile surveil lance robot which has to watch it 's own course of action ( the goal being to assist the over some offices. or might . Its goal is to offer assistance to . knowledge about the vocabulary of 6 . Its task requires that it navigates user) . because they do not employ ' possibly the user s organization . 1990) ( Wilson . rational way. e. over wall following . ( 2) they are more distributed and often nonsynchronized way. which for example might consist of the action sequence from room to room . ( 2) they do not attempt to fully understand the the user and automate as many of the actions of the current situation . 1992) (Payton et. However from an observer 's point of view it will appear to operate in a systematic .Based version of : the text formatting command " latex paper . 1985) module for recognizing and going through doors . Related to the idea of learning is that of redundancy scheme ( some simple suppression and inhibition wires : often the system has multiple modules for a among these modules ) suffices to implement the desired particular competence . the model in the environment . list in section three ) : it is hard to includes information such as the location of the robot provide such a complete and consistent model . For example . Because of the computational complexity chairs . 1992) ( Maes and Brooks . The perception module followed by the preview command " xdvi papr . In almost for wall following (actually wall following is often all cases. This robot does not plan a course of 1991) .

Several sors in real . using tools from complex . Some produced . jobs at hand .the action whole system will adapt smoothly to this unexpected situation sequencesare just an emergent property of a distributed . Once that schedule has been which the emergent structure is stable? and so on . but unless lot of knowledge about scheduling and about the particular we understand the problem of action selection better configuration of machines and typical processing and have a list of desiderata for solutions . and so on. First available machines vary continuously . habits of the user. 1991) . (Tyrell .Based AI is trying to deal with much better . Certain machines of all . it will seem as if the system understands the intentions of the user. 7 . it is important running . e. Behavior . successes have been booked system. which new processes were formulated on them . requiring a rescheduling of the jobs that were so that it becomes possible to critically compare particular running on those machines at the time . 3Some people have started making such lists . This have been proposed. whose goal it is to allocate processes to proces. the processing jobs can actually be sent to the first steps towards a theory of emergent functionality different machines that they have been assigned to . will never completely break down . So far . The system would update its representation have of the current situation as often as possible . Nevertheless . but are far from to run these processes locally or on a different machine .g . consider the problem of building a scheduling approach . 1990. will react in a fast way. The system will smoothly adapt to the changing in terms of the kind of factors it can take into account . 1990) . we need a better requires gathering all the data from the different machines of the underlying principles of Behavior - in the network on whether they are still available understanding Based AI . For example . as if it knows what the task of for external processing jobs at run .concept of its Finally . processes. 1991b) . time it takes to run a PFocess. A user can make a machine unavailable point of view .time . many different Knowledge. The producing a document involves . Aside from better evaluation criteria . which processes they are possible to scale the approach . .Based system for this task would contain a models of action selection have been proposed. In particular . The loads of the different more fundamental research has to be undertaken . Discussion - Behavior Based AI represents an exciting new approach A Scheduling System to the study of intelligence . The decision to be made is whether These initial results are very promising . it will not be . 1981) . A systems and proposals . Without an underlying theory . dynamics centralized way of solving the problem is present in the of the traditional work in this area Kleinrock ( Steels. . 1992) and ( Maes . we need to understand the classes of problems might suddenly become unavailable for scheduling Behavior .Based " AI has demonstrated several " proofs-of . to understand the mechanisms and limitations etc. In order for the new approach to be more founded . ( Brooks . systematic methodology . dynamic domain . This any ground to compare the different proposals3. situated systems . majority ( and Nilsson . the of emergent behavior . How can a globally desired structure system would perform a systematic search (possibly involving or functionality be designed on the basis of local some heuristics ) for the most optimal allocation rules? What are the conditions and limitations under of processes to processors. Malone has proposed a different solution to this prob. what their workload is . in the area of autonomous . and so on . Once all this information has been centralized . 1991) ( Ki ~ . we do not . The system is very simple and yet very flexible system . the global goal being to minimize the average amount of representing a solid . In particular .time . Again the domain is a very dynamic prototypes have been built which have shown to solve one: new processing jobs are formulated in different machines some reasonable difficult task in a real . all the time . Pattle Maes " " no central scheduler) .

and More G. Proceedings of the First International Conferenceon the SimulationMaesP. Submitted to DeneubourgJ.W .-A . Conferenceon Artificial Life.Press MIT . Karl Sims provided commentson an Life. 1992. In: References Toward a Practice of Autonomous Systems.A . Computers and Thought lecture. 1991.. 1991. 1990. Proceedingsof the First . 1991. SimmonsR. MIT . 1988. edited by Cognitive Architectures.G. Varela and P. AI and ChaosTheory Ballard D. edited by F . Algorithms for Time-SharedSystems.. The Dynamic structure of Everyday edited by F. Agre PiE. Proceedingsof IJCAI -91.-A . Detrain C. Life. Detroit . Agent for SchedulingMeetings. and BeckersR..A . A.R. Edited by Meyer J. Bourgine. In : From Animals to Animats .Press.. 1991. RA-2. Bourgine. 1992.. Life. Mitchell . Mitchell Adaptive Behavior.. Krotkov E.Based Artificial Intelligence Acknowledgements Horswill I .. Brooks R. and Howard and P.E. In: Toward a Practice of Autonomous Systems.. 1991. In: From Animals to Animats . and NilssonA .H. and WilsonS.. 1992. Sendova .L ..Press. Florida.. Biology to Engineering and Back. Instruction and Action .PurposeLearning Algorithms applied to Mataric M. Proceedingsof IJCAI -89 conference . Volume II . 1992b.W . 1991. Proceedings of the First European Conferenceon Artificial Life. 1987.. In: From Animals to Animats . Holland JiB . an Integration. Behavior in Animats: Review and Prospects. Maes.-A . Characterizing Adaptation by Constraint Parts of this paper were preparedfor a public discussion . Edited by Meyer J.Press. Autonomous Agents.. Proceedingsof Lunar Base Construction Robots. Varela Malone T . Vision. Huberman . Maes. Orlando. and WilsonS T . Carbonell and TiM .. Computer Programs to Control Independently-Acting Brooks R. Australia.Press.J. MIT . Chapman D.J.. North- Holland . Simulation of Adaptive Kaufmann. Proceedingsof . Journal of the Brooks R. The Dynamicsof Collective Maes P. and Autonomous Agents: Theory and Practice from WilsonS .-A . Japan. 1991. In: TheEcol- Environments Approach to Artificial Intelligence. Proceedingsof the ACM SIGCHI International Workshop on Intelligent User Interfaces Brooks R. Kaelbling LiP. Number 4.. on the topic of Behavior. 1991..Press.L ..J.Press.S. Morgan Meyer J.Up Minds: A Constructivist for Distributed Computing .. In : Special Issueof SIGART on Integrated Artificial Intelligence Approach. edited by B. 32. Intelligence without Reason. of ogy Computation . Bourgine. Proceedingsof AAAI -90. Fikes R. J.W . Press.L. Architectures. Proceedingsof the First EuropeanUser Interfaces. MIT - earlier draft .A . Ambler: An Autonomous .. 1991.Franks Behaviors.. Agents.. A Robust Layered Control System ACM . Chapman D.A ..Press.. Varela and P. MiT . MIT .W . Bourgine. . 1992. 1990b. for a Mobile Robot. Rover for Planetary Exploration. Goss S... Franks N. Kozierok R.. A Learning Interface Sidney. P. Learning to Coordinate DeneubourgJ.. Designing Autonomous Agents: Theory and Practice from Biology to Engineeringand Back. 1981. Learning Behavior Networksfrom Experience Sorting: Robot-like Ants and Ant -like Robots. MaesP. CambridgeUniversity Press. the First International Conferenceon the Simulation of Bares J.. In : Toward a Practice of Autonomous Systems. 1990. 1989.BasedAI with Marvin Minsky Proceedingsof the First European Conferenceon Artificial and Mike Travers. An Adaptable Mobile Robot. Press.. and Brooks R. Grant K .J. Challengesfor Complete Creature . ACM . MIT - Press. On Optimal Scheduling . In: . Boston. Behavioral Synergy without Explicit Parallel Rule-BasedSystems. 1989.90 conference . and Maes P. 1991. Michalski. 1986. Enterprise: A Market-like Task Scheduler DrescherG. Adaptive Behavior. edited by P. LearningInterface Agents. Kiss G. Proceedingsof the the First International Conferenceon the Simulation of IROS.. MIT . MIT . MIT . 3. 1991. 1991.A .. In : Machine Learning... edited by F. MIT . 1992.. In : Toward a Practice of Autonomous Systems. ReferenceFramesfor Animate Vision . Edited by Meyer J. Edited by MeyerJ. July 1981. Florida. EscapingBrittleness: the Possibilities of General. Hebert M. Behavior .. MaesP. Planning for . Situated Agents Can HaveGoals. Whittaker W . Chretien L.. Evolution and Co-evolution of . ACM . IEEE ComputerKleinrock L. 1992... In : From Animals to Animats . MIT . 1991b.A . Mataric M... and WilsonS . R.. Journal of Artificial Intelligence. 1990. Made.R. the ACM SIGCHI International Workshopon Intelligent Swarm-Made Architectures. Kanade T . 28. in press. Theraulaz G. edited by F .W. June 1989.. In: Designing of Adaptive Behavior. IEEE Journal of Robotics and Automation Koza J. International Conferenceon the Simulation of Adaptive Proceedingsof the First European Conferenceon Artificial Behavior. 3. Orlando.. Conjunctive Goals. and Guillot A .J. MIT Press. April .. Varela and P. 1986.. MaesP. Volume 2.

Proceedings of the First International Conferenceon the Simulation of Adaptive Behavior. Suchman L.P. SteelsL. Towards a Theory of Emergent Functionality . and Rosenblatt K .. 1992. Roitblat H. 1969.W .. Shakey the Robot.W .. . ResnickM .. Beyondthe CentralizedMind Set: Explorations in MassivelyParallel Microworlds. Payton D. and WilsonS . 1991.. Proceedingsof the First International Conferenceon the Simulation of Adaptive Behavior. Edited by Meyer J. and WilsonS . Simon and Schus WilsonS . MIT .-A . 1991.MachineCommunication. ACM Press...- A . Elsevier. MIT - Press. MIT . In : Proceedingsof the 14th Conferenceof the Cognitive ScienceSociety. Sutton RichardS ... Animals to Animats . In: From Animals to Animats. Cognitive Action Theory as a Control Architecture. 1990. MIT . MIT . 1990b. Maes. Defining the Action SelectionProblem. Active Languagein the Collaborative Developmentof Cooking Skill . the Societyof Mind . 1991. TechnicalNote 323. In: Proceedingsof the First International Conference Behavior.W. on Genetic Algorithms and their Applications. 1991. Intelligent User Interfaces. Keirsey D. The Animat Path to AI . - Minsky M. SRI A . Cambridge University Press. MIT Press. ShragerJ. Simon H. (editors).Press. Thesis . LawrenceErlbaum Assoc. Edited by Meyer J.I . 1986..-W . Center . In : From Animals to Animats . MIT Media.. Edited by Meyer J .. Proceedings . In : From Animals to Animats.Press. MIT . 1991. Edited by Meyer J.. and WilsonS . Proceedingsof the First International Conferenceon the Simulation of Adaptive Behavior Nilsson N. Muller . Proceedingsof the First International Conferenceon the Simulation of Adaptive Animal . Vol. Exploiting Analogical Representations .-A . to appear in Journal of Applied Intelligence.. Lawrence of the Cognitive ScienceConference Erlbaum. Pattle Maes WilsonS . 1991. 1984. New York.-A . Demazeauand J. In: From ter . Krozel J.A ..W .W. Tyrrell T . and Tyler S. 1992. and Callanan M.. Steels L. 1991....Press.. In: Designing Autonomous Agents: Theory and Practice from Biology to Engineeringand Back.W . SteelsL. and WilsonS . edited by Greffenstette.Press. 3.-A .. Edited by Meyer J.W . 1991.Press. 1992. edited by: Y .. 1991.W . Knowledge Growth in an Artificial From Animals to Animats . Proceedingsof the First International Conferenceon the Simulation of Adaptive Behavior. In : DecentralizedAI . 1991. ReinforcementLearning Architectures for Animats . Do Whatever Works: A Robust Approach to Fault-Tolerant Autonomous Control . 1987.. North- Holland. SullivanJ. The Sciencesof the Artificial . and WilsonS . PhiD.W . 1991.L. edited by P.Laboratory Epistemologyand Learning Group. CooperationbetweenDistributed Agents through Self-Organization. 1991. 1985.. Plans and Situated Actions: The Problem of Human.

we want to explore a more generalquestion: what will creaturesevolve to do. Todd Stewart W. eating. but alsouseful guidelinesfor the designof artificial systems some layout. and actions Creating simple simulatedworlds and seeingwhat - and action triggering conditions. or learn. or communicate . and briefly mentionsomeap. Our work here may in fact systems . 1992). MA 02142 USA ptodd@spo. sophisticated internal world models. By allowing all sortsof simulatedbehavingcreatureswill evolve in them three of these componentsto evolve. Our systemis simple and the ideasdescribedhere. dependson the environmentin which the population of . states. and be in turn exploitedby. What sensors . and describe and Weare interestedin the effects that the environmentcan characterizehow have on an organism's adaptive behavior. Land Boulevard Cambridge . questionencompass es essentiallythe whole of psychology suchgoalsare very ambitious. submitted ). But our goals here are different from those embodiedin many similar moving. but at least by beginningto formalize make much headway . and actions prove adaptive Copyright C 1993 Peter Todd and Stewart Wilson . food is distributedin somefashion. What behaviorswill prove adaptivein various types of environments ? Thesequestionsrequireus to do two very intertwinedthings in our researchprogram: both elucidate 1 Introduction the sorts of environment -exploiting behavioral mechanisms that creatures might employ. internal. states. but also the biotic (e. or the creaturesmay find it to characterizethe effects of the social environ. we can hope to make .org wilson@smith. rather than is certainly not a new approach . and finely-tuned motor sequences before they can achieveany adaptiveadvantage at all. This the evolving behavior-generating mechanismsof work is also intendedto provide theoreticalsupport for individual creatures . or with sel~ tors (see Miller . we must pare it down a bit before we can Obviously beyond our reach. conspecifics and mimics) or without memoriesor internal states.rowland. Todd & Miller. the sortsof behaviorsthat are adaptivein them. structures of environments in which different behaviorswill emergeremainto be developed given certain environmental regularities or structures ? . Wilson The Rowland Institutefor Science 100Edwin H. the behavior-based approachto AI describedby Maes proachesto characterizingdifferent environments (these proceedings ). we can explore " a wide " ( 1992) and Werner and Dyer (these proceedings ) have range of behavior types. particularadaptive in the behavioral interactionsbetween environments - behaviorgeneratingmechanisms . creaturesattemptingto tors at work in it . We leave asidefor and organisms . parasitesand find and eat that food may get along fine with no sensory hosts) and psychological(e. Useful and meaningful ways of characterizingthe navigate. Rather than on the energy levels of the creatureand the manipulatingthe environment with the specific aim of evolving creaturesthat can environment(the bioenergeticsof the world). even We describea frameworkfor exploring the evolution without this addedcomplexity and realism. Since this environmentscan vary in waysthat lead to the evolution of different forms of adaptivebehavior. The creaturesin these worlds can agents existingin variousapplicationdomains. including blind and developedsophisticatedsimulationsthat embodymany of memorylessbehaviors . be seen as complementaryto explorationselsewhereattempting very few motor commands . As it turns out. virtually essential to possess long-distancesensors. We focushereon environmentsare still varied and interesting. By categorizingenvironmentsand in which various behaviorsmay prove adaptive. and perhapsstill and biology. the sorts of of adaptivebehaviorsin responseto different behaviorsthat can evolve to take advantageof static spatial physicalenvironmentstructures . now the fascinatingquestionsthat arisewhenthe environment In a simulated 2-dimensionalworld across which is considerednot only in tenDsof the physicalselec. and serveas positive examples well-defined enough to allow completespecification of what is of the range of possibleaction-types (including possiblewith suchan approach . We focus here on how the physical our thinking aboutthesequestions . Ackley and Littman prespecifyingany of them. Environment Structure and Adaptive Behavior From the Ground Up Peter M. have evolved sensors .g. and reproducing ) and their effects researchefforts. The environmentsare describedinitially as simple describing we hopeto provide not only insightsinto naturalevolved two-dimensionalgrids containingfood aI Tangedin . 1991. spatial structure of the environmentcan foster the someprogressin our understanding of the complexitiesinvolved evolution Abstract ment on adaptivebehaviorand evolution(Todd & Miller .g.

or. behaviorin general. then this creaturewill do about as 2. as 2 The World and the Organism will be describedlater. somethingnot usually attemptedin evolutionary constitutesthe spatio-temporal structure of the environment simulations . which works on a flow of -ponents (at least) which define an organism. (This is in contrast 's to Holland's ECHO system. since it is only through the former two that being addedto each position in the world at each time- creaturescan ground themselvesin connectionto the outside step (so that it can be thoughtof as specifyingthe "plant world. To try to achieve thesegrand ends. and different will first look at a few exampleenvironmentalsituations. Peter M . adaptive - behavior. in order to study the effects of the environment metabolicproductsequivalentto organic chemical compounds on adaptivebehavior we must instantiateour study in a -. Different ignoring learningprocess es). internal state variables.1.t ). nor for any To begin with the simplestinteresting(andeasily visualiz. Sincecreaturesmay.y ) in the world can contain If the world is laid out slightly differently. therecould be world-lived-in for the creatureliving in it . (A hexagonalratherthan evolutionneedonly supplydie creaturewid1die appropriate a squaregrid is usedto avoid the anisotropiesin distances motor commands . such as eat. consistingof N by M positions. it is representedas hexes in the behavior. But sinceit is exactly thosethreecom. from stochasticallyin a fixed proportion . thosesensorsand effectors. thereby creature actions) plus +(x .y . The behavioralportion of each creatureis createdby an evolutionaryprocess . - able) case. Cariani ( 1990) and Pattee( 1989) in particular .while the world is shown with To understandthe interactionsof the physical world and squaresin this figure. the latter. . In this sectionwe first describepossible very sophisticatedbehaviorto maximizeits food worlds and adaptivebehaviorsin them with an eye towardcan basically just move about the environment input: it ' in a showingwhat componentswe ll needto createthose straightline. andis fixed The creature and its environment cannot be described throughoutthe creature's lifetime (thus we are for the moment separately . In this casea creaturereally does not need fashion. "eat move forward .2 The Sb' ucture of the Organisms marily in this paperon an open-endedschemefor the evolution of simulated creaturesin terms of their sensors . one or more creatures . creatureslive on it and more sophisticatedset of behavioral mechanismscould useit up. via emitted food everywherein the world. Thus in this case. and and sensors . eat. we use a 2 dimensionalhexagonalgrid world internal stateis necessary(or helpful). (see Figure la -. then. Wilson creaturesevolves. eating behaviors . say 1:1:0. go straight. and the effects they can have on codesits currentposition and internal energylevel. +(x . that is. causethe amountof food in the world to change. which indicatesthe new food between .seeHolland. we focus pri. throughtheir will allow the exploration of the evolution of exactly actions. and how they can work togetherto createadaptive motor commands " which it selects. and food containsit -.y . move left. of the interactionsof thesetwo parts of our system." and " turn behaviorin the world. Todd and Stewart W . To introducethe subcompone types of environmentscall for different behavioral of the creature's behavioralmechanisms . Only after this frameworkhas nmning around in it at a time.1 The Structureof the World well as possible(in terms of gaining food energy) in this world. including internal representations Consider four possible worlds. and actions. . then a slightly in this world is simply energy.I ). 1992. move forward. The position-energy3-tuple can changeduring what we can hopeto learn from suchan investigation . if the creaturemerelyhas a small set of " " " . t . perhapsturning every now and again. Time in with food in a regular "zig-zag" pattern stretching the world passesby in discreteclock-ticks.) framework which allows the evolution of all three components The exact layout of food in the world over time . in responseto which the creatureshould behave have issuedthe call to considerthe evolution of sensorsadaptively. the ~ ter the amountof en. Thus. The currency acrossthe whole 2-d plain (seeFigure Ib ). In this combined context. Each organism in the beenlaid down can we turn to the questionof the structure world has associatedwith it a 3-tuple (x . along with the syntactic-adaptation of W after time 1.y . and appropriateratesof firing them betweenorthogonallyand diagonally adjacentsquaresin stochastically . each is shapedby and shapesthe other. the structureof the world experiencedby others. we discussbriefly our plannedexplorations actionsand triggeringconditions. the processthat Cariani calls the contentsof W at time I can be computedas the contents semantic -adaptation. both. This layout is createdprimarily by the real- and effectors in addition to the behaviorallinks in valued function. or nothing. First. and a theseevolving creaturesand their behaviors behavior-generatingcomponentwhich definesits possible . evenly and equally distributed actions. we must consider the two in an intertwined simulation). and then we discussthose componentsthemselves as it goes.1 (including the effects of of evolving information-processingmechanisms . and the types of and sensory inputs. a creature ' s lifetime the ~ beginning values for each creatureare determinedat the time of their creation. ergy available to a grazingorganism. ' memory of what it s already done or seen.) Eachposition W(x . E) which of environments . 2. for instance food. change the very structure of the creaturesthat might do well in them. We hope that the frameworkwe presenthere growth" in the world). A creaturewhich performs the action sequence food in a particularlocation. we mechanismsto respondto them adaptively. behavioralmechanisms . the new food yielding a more complete picture of the evolution of appearance / growththat occurs. no .the greaterthe amOlUlt of prove useful . It has no needfor any sensoryinput. The world can contain one or more simulatedorganisms internal states. right.

there is no need for internal state to guide the ac- off at an appropriateposition!) will fmd as much food as tions. showing a creature and the regular path it might take. Figure 1. Obvious- action-cycle. What it needs situation. these can be basedsolely on the outputsof evolved sen. Environments with differing behavioral component requirements. it shouldbe able to do betterthan this. but internal stateis very the zags. in small clumps. then. Environment Structure and Adaptive Behavior c. In this case. must evolve. evolvedactionssensor . ure lc . b. move right. it will be thrown off by one position.. A clumpy environment. If it just movesin a fixed or random manner. eat. if we drop importantin this case. or in a blind preprogrammed fashion. An environment with food everywhere (dark grey) . one of our previously-evolvedzig-zag creaturesinto this ire an evolved "centralpatterngenerator " CPO that - ( ) pro world. both appropriateactions triggeredby cer. showing a creature adjusting its regular path to take the missing food-squaresinto account. have evolved to control artificial insect . A zig-zag environment." over and over again (providedit starts sors. Here. But if it hasa sensoryapparatuswhich indicatesthe direction of the nearestsourceof food. and Since this CPO must keep track of where it is in the will miss all the foro until the next gap comes. imagine a modification of the zig-zag since the world is very regular and so evolvedmodelsof world of Figure Ib . eatingwhen it arrivesat the food. In this ly. showing a creature heading toward one of the clumps. considera world in which food is distributed sparsely and randomly. d. a. " " by addingan extra move forward wheneverit detectsa Next. Finally. showing a creatme ( black circle ) and the mostly -straight path it might take. a creaturewould do well to have somesensory " input. in which there are occasionalgapsin the world can mirror it exactly. it must use some state information. Here thereis still no needfor any sensoryinput. d. the creaturecan base it movementson this input and head in that direction . as shownin Fig- .lation. b. which this tain internal states. one of the gaps. but when it comesto duces this string of motor commandsover and over. it will do fine for a while. A zig -zag-with -gaps environment .. as in the second . " " possible. so it can do betteris againsomesensoryinput. c. it is quite unlikely to nUl acrossmuch of the widely separatedfood in its lifetime .modulated CPOs that Gallagher and Beer . as shownin Figure Id. and the meansof properly generating time modifies the action of the central pattern generator thoseinternal states. In this case. Essentiallythe creaturemust util . as in the first sib.

Theselists of variablesare restrictedin how they sensorssometimesfeel like. The internal state variables available to an individual tant. but even that may not be so clear in not present(which can be signified by an "inactive" flag more complex situations. to be able to analyzethe simulatedcreatureswhich Each S[ i J can be as complicatedor as simple as result from our evolutionaryscenariosin terms of these evolution deigns to make it. just as for eachS[ i J.) . a complex function modularly distinct.y ). which rememberswhetheror not action 3 situationspresentedearlier. The categoriesc We can see from these./ Slij(Wc(X . clearer. They can al~o a our experimentsto the first two typesof environmental memory for past events or actions. delivering an only more-or-less realistic view inputs such systemsare employing than what they' re of the objectsin the world. if n[ is preset to 0. If ns is presetto 0.) We now consider each of these three we hope to make the behavioralorientationof this work behavioralcomponentsin detail. we need a system which can but this can sneakin throughthe internal state variables evolve each of these componentsindependently . Evolution createsand or reflecting light or sound waves to carry information builds up the entriesin eachof theselists within eachindividual aroundin the environment . That is. so that two tors mentionedearlier. creaturein the 2-d world. and 1[ 17J= 1' 1[ 52J. we will define the sensorsin the presentsystemas the internal statevariablesallows for limited propriocep- providing information aboutjust thosethings that are out tion. it would be very difficult to say and Yp indicatethe individuals . Each individual has a list of sensors.) To help alleviatethis difficulty for that sensorin the evolving list of sensors ). modularsystemof evolving lists of actionsand triggering Such "direct-perception " sensors may not seem to conditions. Thus to explore the evolution of proprioceptionpossiblethrough the sensoryinputs in S . we have developeda to be O. . we will remain agnosticin sourcesof thesesignalsthat we (and all living creatures ) our choice of representationand implementationof the are really interestedin.y ). For example.) generalclassesof sensorsare possible. Moreoverdescribednext. ripe fruit . that is. 1[ i J . and another neural networksor Lisp code representations of our might have S[ 7J = W(xp+l . Sensorscan also be noisy or (It is typically easierto detenninewhat actionsand sensory inaccurate . 1[ 3J = logisticS [ 1n ns. the rather.y ) + W(xp. they are for indicating the presenceof a colorful preset maximum number of internal state variables. it is important that they be kept might have S[ 18J= log (W( I .y .1 Sensors The ultimate role of sensorysystemsis to tell creatures 2. But again. for a particular category c . as in provide the pair 1[ 52J= A ' . as mentionedin the previoussection. That is. current position. ( Thereis no environments .the granularity in a v"ariety of ways to create adaptivebehavior in different of actionsand sensorsis alwaysequal. things in the world which can of (someof) the current sensoryinputs. Sensorsare not for infonning aboutspectralintensity creatureare designatedin a mannersimilar to the sensors .four examplesthat actions. or a snarling Again. Toddand StewartW. all creatureswill evolve predator. they should merely be thoughtof their proximal cues. aboutthe contentsof W(x . then noneof the creaturesin would performa gentlethresholdingof the resultsof sensor this world can have any sensors-. Wc(x . the previous have a positive or negative effect on the individual' s internal states .I). sensorsshouldbe thoughtof as signal. 2. As of the previousaction as one of the possibleinputs to such.73. and sensorscan all combineindependently or entitiesthat the creaturescan act on -. for instance. Each Sfi ] is definedas afunctions . and sensoryinput systemsmatchthe sonsof indirect windowson the world our own . ( Wewill return to the issueof implementationin a on the objectsin the world and their action consequences . and one " might object that can interact with each other. Wilson locomotion. Sfi ] . They are indexedin a list 1[ i J. later section. without any internalstate.2. PeterM. or modulatedpitch fonDants. - Iij of just the category basedcontentsof the was just performed . The inclusion Gibsons 1966 notion of perceptual affordances ).I. by focussing . or a sexually-ready conspecific. patterned after ' time-step). internal statevariables. where c in the current simple More complicatedsystems . S[ i J is defined of both evolution and analysis.l . or airbornepolypeptides . rather than the particular form of elementsin theselists.2 Internal States ' about what s out there in the world that might beimpor. it is the ultimate creature. for i from 1 to n[ . in which when a network was using internal state.1[3]. which returns world. one creature behavioralcomponents . A ' . wherei Internal state variablesallow further processingon can go from 1 to somepresetmaximumnumber of sensorsthe sensoryinputs. In addition. that is. or whether or case this sensorlooks at what' s above and below the not a Lisp routine instantiatesa central patterngenerator . can alsobe evolved(thoughit may . Each 1[ i J is therein the world. that sensorscan respondto are just thoseclassesof objects internal states.f [ /il <S" I ' . and at the sametime avoid the complicated(and important) problemsof sensorytransduction . set to 0 if not defmed(inactive).2.I )) + 8. If a particular i th sensoris doing internally.this is how we can restrict 7 (looking up and down). and the sensoryschemepresented as some sort of simple information-processingmechanisms here capturesjust this notion.yp)' where xp creatures . and the previous action-states(indicating ongoingfitness (see Miller & Todd. so that modularity and interpretability theres nothing here like propagatingchemical gradients can be maintained . For now. For instance . like the centralpatterngenera - worlds can be either food or (other) creatures . in preparation . the performance -status of action 3 two time steps ago. for a whetheror not a particular action fired on the previous more detailed account of this topic. such adaptive behaviors. if we were just to evolve big unstructured of an absoluteposition in the world. Each1[ i J is definedas a function ling fitness affordances . Sfi ] .

just as the triggering valuesnow are. cannot be altered. and for this world will also be limited.. For picking an action at randomfrom amongall thoseabove now we will leave thesepossibleactionsout of the system some trigger threshold. describedin the next section. E ) indicating as a discrete probability distribution function. Triggering functions A [i ] trig are real. since its actions can change its internal standingof the evolutionaryand behavioraldynamicsof statesand its view of the world. questionsarise as to how far and in what directions a .. are own energylevel. y . are usedto determinewhen a particularaction is invoked. in particular. giving the creatureitself. Just as for the eating action. including selectingthe action could alsobe achievedthrougha combinationof "eating. The mo~t obviousaction in this categoryis "eat. At present . we will discussthesein more detail in the next section what location to eat from.e." and "fertilizing" actions.) But a creaturecan changeits own (x . and how much to eat (i. as mentionedearlier. preferably 2. From here. Environment Structure and Adaptive Behavior " take quite a while for such things to be hit upon in such ing -.. it takesthe creatureto perform it ). of the direct action. by "pushing" the food possible (active) action is being triggered. In the alter its own behavior-generatingmechanisms . choosingan action "moving. by "fertiliz- an individual will perform at a given time-step. specifying etc. (It is important to achievethe samebehavioralends. what (x . ' valuedfunctionsof the individual s currentsensoryinputs Besidesjust eating the food in the world. though. its ' list of usablesensors systemwe ve describedhere there is not all that much in . and them in the evolutionarypot. the .) Though this behaviorseemssimple." " turn. E ." (and others) will itself be a 3-tuple. anotherside-effect). behavior-generatorsare fixed. or else we' d end up with an uninteresting location. How difficult or during their execution. and what the ratherare impenetrableunits." A [i ] with the highest trigger value. since it will keep our analysisand lll1der. we might want to allow creaturesto increase That is. (This performedcan be chosen.y ) location. try to begin with as few possibleactionsas we can. This action will begenerat- patterns which are fired off andrun to completion ed as a function of the current internal state and sensory without any environmentalinfluenceeither before inputs. (x ." "eat.2. that case? And how much of the food will the creatures the maximum number of different actions an evolved eat? All of it at once? Or can they opt to eat a lesser behavioralsystemcan contain. along with the current sensoryvalues. Or creaturescould a real value representingthe strengthwith which each "fann" the food in the world.e. to see whetheror not any thereforealso in different evolved behavioralsystemsto creaturesfmd them adaptivelyuseful. that is. since the food/ energy. except by the action of evolution First of all. W(x .but energy. Thus they are not functionsof eachsuchactionis (i. in one time- functions. and actions the world for the creaturesto alter: basically. We view this as a trigger functions return can all be affectedby what the plus. we can restrict this class of actionssimply to behaviorsactually get emitted at all.y . how mucheffort. Different selectionschemeswill . For instance.) Action-typesA [i ]act can be primitive We plan to extendthis systemso that the eatingcommand motor commandssuch as "move forward. creature does.types alsothrougha non-arbitraryevolutionaryprocess . and let the selectablepool of them build up slowly.3 A Catalogue of Action . all of " that ing position (certainly at a cost in energy terms to the creature's (active) trigger functionsare evaluated . internal statevariables. in termsof either the sensoryinputs or the internalstatevariables. food in the world. shouldit take longer for the creatureto do so? . spreadin somelarge areainto a smallerregion for easier there are several ways that the final single action to be later harvest or hoarding from other creatures . " step. the actionsand their triggers. the number of action-type primitives that we can define ( Notethat the valuesthat the sensory . but we will considertheseasenergetic"side-effects" behavioraltriptych. It ). For example. internal state. creaturescan changethe distributionof . unbehaving creature. nA will never amount? If there's a lot of food to be eatenat a certain be set to 0. These actions can be thought of as very simple amountof energyto absorb "fixed action " ). in this case. throughthe triggering "eat everythingthat' s in your current hex. and currentinternal states(if thereare any of each) which there are other ways creaturescan alter its distribution. or the positionsand amountsof food/ energy involved.) Both of theseactions stochasticallybasedon the normalizedtrigger valuesinterpreted could againbe describedvia a 3-tuple (x . As a result. but for completenesslater we anticipate including result in different observedpatterns of behavior. but the functions themselves this systemthat much simpler.3 Actions and their Triggers types are subsumedunder this category. To determinewhich action the amountof food at a given location. and a decreasein the energyof the then made available to the final component of the food. (Here again i can vary from 1 to nA.fA [i ](St . thereis only and triggers. a variety of possibleaction- ' 2.) Another thing in the world that a creature can The things that an organismcan do in its world dependon changeis itself.y ). The outputs of the internal . would again be The way sensoryinputs and internal statesare able specifiedin the bioenergeticsof the world.whatthere is in that world to act on and affect. though. But in the to affect emitted behavior is by controlling which meantime .y ) locationsdo we allow a creatureto eat from? Each action A [i ] an individual can perform is defmedas Only their own current location? Or neighboringlocations a two-part structure consisting of an action-type and a ? How big a "reach" should we allow creaturesin triggeringfunction. no learningis possibleyet. we do not allow a creatureto . that is. energeticbenefits and side-effects are.lowering the food-value at a particularworld location an open-ended framework). A [i ] trig . E ). and the creaturesthemselves . change the food entries in moving aboutin the world. (This will in turn result in an increasethe creature's statevariables.

this is how particular action-types are "disallowed " in our theseeffects will be manifestedas raising or lowering the system. In this existence ) would be to increaseits own energy level case. from the food in its currenthex. For now. the energy ' of the system. In general. therecomesthe possibility for a wide rangeof other The main rule governing the bioenergeticsof our "social" or antisocial! behaviors as one creatureaffects ( ") . c ." "mutate-other. As more types of objectsare introduced that alwaysjust try to eat." When a sucha restriction. and world. we can imagine sexual reproduction effectswill simply be to lower the individual s energyby " . the bioenergeticconstraintsof the world). 1. throughimpayableenergypenalties ' individual organisms own personalenergylevel. we allow an individual In the previoussection. will eventuallyrun out of energyand die. Furthermore . Finally. we add a final action-type. rather than action-types for changing each of those things will be attemptingto constanty shovel everything arolmd them created. move-other or eat-other.g. we still impose a small energy agent. - action type affectsthe world s energydistributionis deter . all the action-typespossiblein that world. Energy can be transferredfrom one entity to one final aspectof the world that the creaturesshouldbe another.t ). andsplit into two. resultingin essentiallya perpetual -motion organism created. time. WidIout bioenergeticsof a given world consistsof a table or list of such mutation. even when there is no food into the world (more categoriesof entities. Once in 'the world. Todd and Stewart W . "split. The only way the total world-energy again. the behavioralmechanismscan enter the population.0) on the parentcreature. creatureinteractionsaside for dIe moment.2. and its contentsto be a potential action-type (subjectto For movement . way in which the bioenergetics . with eachgetting an equal initial creaturesto perform only three main action-types: amountof the "parent's" energy. whetherreal or hypothesized sloth-cost on it (e. if the creaturedoes nothing of adaptivebehaviorby any sort of behaving in a given time-step. The eating-exertioncost guaranteesthat creatures that can be changed . In future versions associated energeticconsequences . whetheror not they have analogsin the "real" splitting-exertioncost (e. By allowing any possiblechangein the world down their gullets. For splitting. but never able to influence: dIe numberof the creaturesthemselves . we just to keep things simple .4 Bioenergetics of the World to nothing. - changing the energy level' in food-plants. and more interorganism the creaturesmust evolve to only try to eat when it ' s interactionsare allowed.e. and for performingthem. like the behaviorsthemselves With more than one organism in the world at a .. to start off simple. y . move to one of the ' samehex as the original. what location . correspondinglymore adaptive . . ). so that new associatedwith each of these actions. the systemwould be more or less static. E") actiontype form. .y .. movementto any of the positionscurrentlyadjacentto the but we will describebriefly at the end of this section a creature. Such generalityis essentialfor the study of adaptive divide the remaining energy equally between the two agency(Miller & Todd. construedas the investigation resulting offspring. in tenns of all the things in the world eaten). we needonly specifythe energyside-effects original . The only other global rule is that once an ' each one ends up at. But there is cI(x . in terms of changingthe distribution have a prohibitively high energeticcost associatedwith of energyin the world (that is.1). we hope to enablelower its energyE ) a movement -exertioncost (e. " " " systemis that nothingany creaturedoescan raise the energy another: for instance . Thus described earlier in section 2. All of the other action-types that we chosenot to As we indicated in the previous section. E . mined by the bioenergeticsof the world. as around . we imposea to appear. (In general. But they would side-effects. to another). be able to allow or disallow reaching by . every action a include in our initial systemcould in fact appearin the creaturecan perfonn in its simulatedworld may have energetic bioenergeticspecificationof the world. transferringit from one entity themso that no creaturecould ever performtheseactions . cost for 0. first instantiationof this system. sedentaryorganismswill eventuallybe weededout.. In our simple dies and is remov~ from the world andthe simulation. " - action type. and whetheror not the new copies individual s energy level E drops to (or below) zero. 1990). dying and being removedwhen their energyslowly leaks away 2. 1.. biological or artificial. we will only allow single-hex will just assignthe energyside-effects of actionsby fiat. If we did not impose To this end. terrestrialor extraterrestrial . 0.g. makingsix movementaction-types. as well as an eatingexertion . we are setting these types of can ever increase is via the food/ growth fwlction. and certainly can be lowered or lost. we needto detenninehow manynew individualsare directly. canemergeduring the evolutionaryprocess . we will " in the world . then obviouslythe bestthing a creature creature performs this action. How each locationenergy3 tuple (x . it creates one or more could do in its lifetime (in terms of survival or ongoing copies of itself as new indiVidualsin the world. Thuswhenwe introducethe general . and each occupyingthe eat everythingfrom the current hex. and the other is mutated slightly . how much energy each one gets. the individual' sE by the amount of energy absorbed and " split-other" . it are exact or alteredversionsof the original. Peter M .0) every possibletype of action in these simple worlds for each in step any direction. when there is food present. Finally. Again for now.) For eating. Wilson creaturecan travel in a single time-step. the We have describedall the possibleaction-types in food-energy in that hex is set to zero (since it is all our simple world. increasedthroughcreatureactions.5 and then increase through a combinewith other " ( example.3). that is. Thusfor the energetics creaturesthat replacethe original is kept identical to the of this world. In particular in the currentsimpleworld. more morbid behaviorslike "mutate-self. so that completely useless . alongwith their with no evolutionarychangepossible.g. one of the two new adjacenthexes. we will charge the creature (i. we endedup allowing our creatureto just split into two.

some of the creatures would like to seeassociatedwith variousaction-types. given run though may combine these actions in an appropriate but this could also be introducedat somepoint. But we would be happier survivingcreaturesmay not occur. this emergenceof ganismsand behaviorsto evolve. however. splitting) we have only roving. We expect . All of thesewill y ~ Yp. then splitting and with profound effects on the sensors and behaviors spreadingtheir successfulbehavior-generatingmechanisms evolved. we do not allow any creature 3 Evolutionary Dynamics and a Day direct accessto this importantpieceof information. 1992) could be achievedand maintained. cost. rather than being predeterminedin an interesting(i. it shouldlike - wise lower its energy . However behavioralimprovementswill not be introducedinto the . allowing an uneasytruce where individuals widely) fluctuatingpopulationsizes. when to sit and wait for it to grow back. However. so that they can roam aboutthe world eatingover as step after time-step may be impressive . die usual fitness measuresimposedfrom aboveby hand. as well as a specific -behavingindividualswill makeuse of amountof energyto get it started. We introducenew such just such an internal constructedbit of self knowledge. in the internal statevariables. and the what might be an ultimately uninterestingway. but do little else. it should increase its internal assessment of its energylevel by an organismvia abiogenesis(generationof life from inanimate subs. An immortal creature that works its way that they will all vote or bid to lower the movementcost aroundthe world and eats enoughto remain alive time- to 0. by forcing any creatureabovea certainenergythreshold self-interests. randomcreaturesby abiogenesisevery so often (after a so we are on the lookout for any such feature in our typically fixed interval of many time-steps).taDce ). eating creaturesevolving in the d1emselves . and with the bioenergeticscoming out to split. Fitnesswill be defined exactly they would no doubt call for very high movementcosts. its own energy " " level E .) to new offspring. (Note that there is no energy and die. Rather . An individual' s perhaps evolution at the edge of chaos ( Langton. Some of dIese creatureswill move about in dIe world. For evolution to work. akin to say population. we can imagine long-lived. This would not be particularly interesting. eventually through evolution.thod. this should in both speciescould survive. similarly for movementbeyoodadjacenthexesor burn up or wasteaway their energyuntil they reachzero - multiple offspring splitting . reproductionis not imposedon any individual "vote" or "bid" for the energy costs and benefits that it . we have for how to allow such emergenceof the basic Thereare two importantsubtletiesto this evolutionary bioenergetic laws of the world is to let every organism scheme .. One idea systemwill take off. In turn. but as we continueto if this aspect of the world as well could emerge introducenew randomcreaturesinto the world. (This is in sharpcontrastto closeenoughto devourthe food organisms . but never move. In any given time-frame. then eating. Otherwise . will improve upon the behaviorsof their havedescribedherehave all beenset by hand. or maybetherewould be a not be seen as a disadvantage . in what we parents . if we also allowed thefood itself to evolve.level internal variable. we will also have (perhaps settledupon. along with the ongoing energylevel could be crucial for knowing when to split.g. when to headtoward food. to keepup a evolvingscenarios .) Note somesort of energeticcostJbenefitcompromisewould be that with our current me. over time. no doubt fashion. adaptive ) one will show up. In this way creaturecan know aboutits world. lnltil dIeir energy . widI random sensors . We dIen setit loosein die most adaptively " . To begin each run. constanttrickle of potentially adaptivecreaturesinto dIe world.e. namely. First. If must hit upon "reproductive behavior" (i. but rather as an added continuouspredator-prey cyclic patternas first one faction measureof realism. someof the new offspring. With both organism types voting. Perhaps next generationproportionalto its amassedenergy. each with differente. mutated The energeticcosts and benefits of the actionswe slightly. from the simulation. but without such long and far a range as they want without incurring any creaturessplitting and having slightly mutatedoffspring. if an individual eats. Environment Structure and Adaptive Behavior how we assignenergy costs to actions with x ~ xp and is too low to allow any further splitting. in terms of those creatureswho behaveappropriatelyto to cripple the roving eatersand keep them from getting producethe most offspring. our creature -controlled reproduction or plants sedentary or immobile prey organisms . bubblingwith life.e. or by running the simulationin generations of this conflict-baseddemocraticprocess . we expect that and giving each individual a numberof offspring in the much more interestingdynamicswould emerge. Otherswill sit and eat. Still otherswill split repeatedly . if it moves or splits. etc.g. moving. Some lucky few in any cognitive computations . e. is never directly availableto it. one of the most important things that a bioenergetics temporarily in their own favor. with their ineffectualbehavioralendowmentsbeing cost associatedwith either sensing or perfonning removedfrom the world. " dIe world and seewhat happens . interestingpatternsthat this entails. they in the Life must evolve an internal representationof it which is updated Creaturesin our simulationsobviouslyhave rather simple appropriatelyin responseto the actionsthe individual lives. and actions and triggers. Thus. etc. and more and more adaptivebehaviorwill evolve hope are reasonableways that will allow interestingor. and then another gathered enough votes to sway the Second . and let scheme allows a fitness measure to emergenaturally theseentities also vote on the bioenergeticsof the world. we just createa randominitial organismemits. Furthermore . if creaturesare to make use of this information. internal state appropriate variables amount . die best they can do is just be world as we have so far describedhere.

2. Peter M." "unknown. such modificationof theseneural networkst we use a scheme as that of Koza ( 1992) or Sims ( 1991). Every network will have an At this point. internal state variables . and usesthe probability-basedaction selection parametersettings ). sensory inputs on all the connections between these two . . the ) has will direct connections to positions in the world. the evolutionaryprocessitself it will be flexible (and manipulatable ) enough to could still be quite extendedandtortuous. . and our 2-d grid world representation end up with the full -blown sensory -guidedbehaviorof the . Each output environmentsand environmentalclassificationstypes that we unit in every creaturehas an evolvedbias level associated . we can add sensors to this network as are interested in developing mappings between such Finally anotherlayer of (input) units. In sucha form " - " methodto . We choose to consider two-dimensionalgrid basedon one of the selectionschemeslisted in section world environmentsin general becausethey are easily 2. In either case. while the translationof our that evolvingentireLisp routinesdoes. we can have a set of possible and Other Research Directions output unit types. thereare at leastthreedrawbacksto the fourth scenario . In this case. If the networkhas only suchoutputunits. 2. and from themselves . becauseit wide range of nonlinear functions to be implemented emphasizesmutational changesto the Lisp code. eachsensor FSA approachto environmentalcharacterization : first. eachof which is associatedwith one of the possible action-types. creaturedescribedin the secondscenarioin section2.3. Sensorsare in turn connectedboth to the internal state unitst and to the output unitst since both are The behavioralevolution systemwe have presentedhere functionsof the currentsensory input (as describedin sections is intendedto be implementableby a variety of methods . allowing a may be more suitedto our pwposesat present. and workt we can add additionalhidden layers of computing actionsand triggersthat we want our creaturesto be able units betweenthe existinglayers.2 and 2.2. We . we can use a geneticpro For the geneticand evolutionaryrepresentationand granuning try to evolve these functions . . Evolution will detemline the the descriptionhasbeenpurpose fully placedat a high level numberand types of sensoryunits in eachcreaturetsnet- to be agnosticin terms of implementationdetails. can describe weights layers With such a network. These units receive recurrentconnections visualizationit allows.2. e. and modelsof environments . Since our like that developedby Millert Todd. and Hegde ( 1989). which forms part (or all) of its triggeringfunction..) to use. FSA have to be enormous to capture all the possible states and transitionsbetween them (e. but obviously than the recombinationof s-expressionsprimarily usedin this frameworkdoesnot allow the unlimited flexibility the former.or Jordan-style recurrentnetwork-. we will end up with astochastically- tie in with empirical . Mozer & Bachrach . are beginningto explorewith the frameworkwe havejust Actions are selectedfrom among the active output units laid out. (The final recurrent in this section: evolving Lisp routines. Todd and Stewart W. 1987). and the ( ). creaturein responseto its motor outputs. 1991 ~ Rivest & Schapire . Slightly would determinenot only the numberand type of output more sophisticated conceptions of environments as finite state automata FSA ' s which return to the units. in terms of a structurednetwork with eachlayer of nonlinearunits interpretedas one of the three categories 5 Characterizing the Environment .3). ecological data.g. for a small . many earlier studies)t and so we are eagerto comparethe it may seemvery unclearhow we could constructneural results of this representationschemeto the Lisp-based network architecturesthat will precisely embody the one. internal state variables. (If furthercomputationalpower is neededin the net- modularizedlists of sensors . We workt and the connectivitypattern and weights between are focussingour own efforts thoughon two very different the world and the sensor st and betweenthe sensorsand types of implementationswhich we will describebriefly the internal state units and output units. rather betweendifferent variablesin the three layers. the evolutionary mutation process imhelpful in our current endeavor . as shown visualizableand concrete(rather than describedby unintuitive in Figure 2a. that is.. with it ." "regular. First of all. just mentioned .seeElmant The lists-of-functionsrepresentation we usedfor the 1988tand Jordant1986.2. But there is also a natural interpretationin this case. as shownin Figure 2c. the latter method Units can use various activation functions.2. are logistic-function activationsforward in turn to the output crude and. Wilson 4 Actual Implementation Methods W(XtY). Here we have drawn the input actions and triggers. as shown the link to the grid world description and the valuable in Figure 2b. units.. ing theseenvironmentswill probably require returning to Now we can add the internal state variables as abstractparametersand categories ~ but we will maintain anotherlayer of (hidden) units in the network." etc.) s-expressions . They should also be more readily translatable into real -world settingsso that we can scheme . andpasstheir Traditional machine-learning conceptions of environmen as "noisy. we would get the memory-guided quite complexenvironment / behaviorinteractions(see. yield very interestingresultsin this system(as it has in When comparedto the obviousLisp representation . classify- behavingcreaturelike that describedin the first scenario in section2.but also the number of internal stateunits.g. However. and sensors units offsett to emphasizetheir secondaryrole in guiding earlier is obviouslyreadily translatableinto Lisp codeand andadjustinginternally-generatedbehavior. for unit (which is sensitive to one particular category environments of reasonable size (like our grid worlds ). and evolvingneural structureof the full networkhereresemblesthat of networkarchitectures . However. As can be seenin the figure. we do not have space(nor results) enough left to do much other than describesomeof the of evolutionarily-detemlinedselectionof these. Stillt we believe systeminto Lisp may be clear. creaturescurrentlyjust split and mutate. from the outputunits. anElman.

see Todd . in ways that do states). we could . evolve to exploit them. 1992.. to be defined.y) Hidden Units l[ iJ Figure 2. creaturesand behaviors."lo~gbias ac Ion 0000 . '. Littman carries this further. between" environmentalstructure and the creatures As we have mentioned earlier . and Littman (these a way of parameterizing the clumpiness of food in a 2. .:'-. . there are 2100 . courseof evolutionaryadaptation .'" 2-:/. :.d proceedings ) have proceededfrom the FSA environment world (which we have already done in one way for the 1- modelsto more abstractcategoriesbasedon the amount d case -. With this characteri- of memoryor statea creatureneedsto exploit an environment zation in hand. that is . chapter 5) . say. These approach es also is a well -studied area in the animal behavior literature . and even be to explore the tradeoffs that evolution makes when it non-sensing .. A network .. But it is not easyto in how well creatures without various of the behavioral see how to move back and forth betweentheseconceptions components might fare in different environments. . while importantfor learningsystems . : B [i] :ftsS . b.rather than allowed to emergethroughthe but we may be forced to recant this position . second . and see what sons of behaviors following its behavior. Weare working on Wilson ( 1991). as well as their evolution must choose between. we believe that it shouldbe possible keeping the sum ns + n[ fixed during evolution . Also. the notion with . and whether there are interesting characterizingenvironmentsin terms of the creaturesthat correlations between the two . And finally . world in separateclumps or patches. or approximately1Q3o . in focussing An example classification scheme closer to what we on the sensoryconsequences of actions.-In .. we are also interested that behavein thoseenvironments . and how long the creaturemust wait for rewards clwnpinesses. ~ b. Output Units Ali J Contents of World W(x ..': .2. and third. or ones with limited sensory abilities to others with of reward. . 0 OutputUnitsAli] . and certainly emphasizethe interrelationshipwe will have much to draw on and compare to there. Environment Structure and Adaptive Behavior acfon 'w ?:"CK -}r. possible are not entirely creature-centric. where we spoke of food being distributed in the used. senses and internal states: by . A networkthat usesboth internalstatesand sensoryinputs from the contents for generatingactionsbasedon internalstates of the world to produceits actions.. Patch finding and foraging can behave ~ timally in them. the actionsand sensations are all assumed not rely solely on the capabilities of creatures themselves. the FSA model are hoping for was indicated in the third scenario in section overlooksthe interestingpossibilitieswhereno sensorsare 2. againobscures sophisticated senses. Another interesting variation would the important classesof non-learning. lOx10 grid which either containsfood or nothi! }g at each to describe some aspectsof the environment in tenDs that location. we will be able to construct worlds of different . . a. c.~ 1 ~ ' . Smith ( 1991). A simplenetworkfor producingactionsstochastically .~J l [ i J . NetWorkimplementationsof behavioralcomponents . seempromising. so . '!t'/. Thus of environmentsand more concrete visualizable we might compare creatures without internal state to those forms like the grid world representation .

Todd. Todd. P. Adaptiveconnectionto the world theoriesof life. A theory of creature's behavior-generatingmechanismscan evolve as perceptionbasedon fitness affordances . Unpublished doctoral thesis. CA: Morgan Report 8801.. Designing lead us to understandingof phenomenabeyondthosewe neural networksusing geneticalgorithms. ( 1992).H. sexualselection. Unpublisheddoctoral thesis. andTodd.J.L. J. Langley (Ed. Life at the edgeof chaos. Artificial evolutionfor computergraphics Diego. S. Farmer . Cambridge . Unpublished Jordan. Proceedings of the 5th IEEE InternationalSymposium Rivest. 364-375). creating a systemin which all aspectsof a Miller. Langton(Ed. A Smith. ( 1987). RedwoodCity. J.F.G. La Jolla .L . M. we hope to have a frameworkrich enoughto Miller. CA: Centerfor Research Kaufmann. m life (pp. Todd. G. and S. Gallagher. (in preparation ). G. RiD.I . Taylor. 65-80). G. Fanner. CA: Morgan Kauf- mann. CA: the behaviorsthat are adaptivein them. . ( 1991).). andS. Evolution Miller . SLUG: A connec- evolutionandlearning . ( 1992). and Cariani . Sejnowski. Simulations .G. J. Cambridge . ( 1989). Universityof Alabama. Proceedingsof the Press .D. ( 1992 ). and Miller .. vol. Taylor tionist architecturefor infeITing the structureof . Unpublished needed . Wesley. Technical Report ICS. ( 1991). Machine Learning. ~ partment. Technical Learning(pp. Langton . MA: origin of species : Mercurial mating in the Quicksilver MIT Press / BradfordBooks. C. Exploring adaptive Press / BradfordBooks. 487-509).F. RedwoodCity. 7. but one which we believe will Proceedingsof the 1990 ConnectionistModels yield useful insights into the nature of environmentsand SummerSchool (pp. Ackley.F.). MA : MIT Miller .. In C.. Booker Langton. Proceedingsof the Fourth International Langton.). (pp. P. C. StanfordUniversity. to unsupervisedlearningin detenninisticenvironments Los Alamitos . CA: MorganKaufmann. and Hegde. 547-554). ( 1989). J. m C. ( 1991).). J. 41-91). RE .D. Artificial throughself-organizingsensors andeffectors . Wilson (Eds. Submitted to the . Hinton (Eds. Meystel . Tuscaloosa . and Bachrach.U. J. Pattee.H. Werner. The evolutionof learning: Simulating systems . H. ComputerGraphics. P. and Beer. Findingstructurein time. Adaptationin natural and artificial Todd. Fourth International Workshop on Machine Elman. Gibson.L.M . San Mateo. 1 (pp. . By startingat the MorganKaufmann. Evolution "without" natural selection: Sexual)selection Littman. J.-A. Stanford University.G. ground-level .F.t W. Serial order: A parallel. Touretzky.K. ended researchprogram.M. The animat path to AI. M. On the sympatric Koza. MA: MIT Press. ( 1992). (Eds. life II (pp. 25. M. and GiE. Proceedingsof the Third Artificial Life Conference Maes. ( 1992). Cambridge . (theseproceedings ). In R. (theseproceedings ). CA: 139. 63-77). in Language .M ( 1990). Boston: Houghton-Mifflin . Belew and LiB. take place.C.M. through psychological selection: Adaptive WilsonS . ( 1991). 319-328. RL .M.E. . In D. Artificial finite-state environments . SanMateo. the interaction of adaptive processes. Holland. open. Todd and Stewal. ( 1990 ). and S. Geneticprogranuning. GiF. mteractionsbetween Mozer.. P. J. La Jolla.. ( 1992). SanMateo. P. Addison -Wesley . . (these proceedings ). G. Conferenceon GeneticAlgoritluns (pp. and Dyer. P. CA: Addison- A. ( 1991).) . Behavior-basedartificial intelligence . Universityof Californiaat San Sims. realizations. M. Proceedingsof the Third International Conferenceon Genetic Algorithms References (pp. systems . GiF. A new approach on IntelligentControl.G. Default hierarchy formation and qualitative dynamical analysis of evolved memory exploitation in learning classifier systems locomotioncontrollers. Stanford Conferenceon Simulationof Adaptive Behavior University. Rasmussen(Eds. In have alreadyimagined.G. The sensesconsideredas perceptual AL: ~ partment of Engineering Mechanics . In C. CA: IEEE ComputerSociety .).J. . University of California at SanDiego. C. CA: Institute for Cognitive Science from ~ . D.160. and Todd.C. ( 1966). and Schapire .). K.J.. evolutionof learning.. J. ( 1988 ). Schaffer (Ed. P.). RedwoodQty. Stanford. M. Gray (Eds.S. unpredictability. CA: Addison-Wesley. R. The evolution of protean behavior of herdingbehaviorin artificial animals. Artificial life II (pp.W. An optimization-based for categorizationof reinforcementlearningenvironments speciation and learning.M. From Animalsto cephalization . Animats: Proceedingsof the First International Stanford. andLittman. Stanford. model. G. and working our way up to considerationof different manuscript . In P.. 379-384). P. Peter M. San Mateo. 73-78). CA: Psychology types of environmentswhere this evolution will ~ partment. and humanen.D. Rasmussen (Eds. Herath . In J.). 15-21). CA: Psychology processing approach.W.. Meyer and S. (theseproceedings ). (submitted . J. (Available 8604. distributed doctoralthesis. ElmanT We are clearly embarkedon an exploratory.M. and Miller . CA: PsychologyDepartment .). Wilson see how evolution balancesthe two when it has limited agencyI : Theory and methodsfor simulatingthe resourcesto work with. ( 1986).

the optimizing one step further to explore a new class of mate choice Apollonian powers of naturals. one ' s ideal mate is some distance ences' (Kirk patrick. When directional matepreferences up a complex reciprocal interaction between evolving are free to evolve. methodsfor simulatingsexualselectionwith a variety of In previouspapers(Todd & Miller . In this paper. after which section 5 offers some suggestions tative mate preferencemechanismsalone are sufficient to for how simulationsof sexualselectionmight be usedin practical applicationsand in further scientific research . research . smarter. . one' s dynamicsof ptodd@spo. section4. geneticallycoded to wander capriciously through phenotype mate preferencesevolve to exploit currently available space. Thus. througha shorthistorical emerge through natural and sexual selection. With non directionalmatepreferences . Our resultsare summarizedin submitted ). historical backgroundto The dynamicsof evolution and the mechanismsof cognition simulations methods . 1991) that apparentlytrivial changesin the psychologicalmechanisms of mate choice can have substantialeffects on the 1 Overview evolutionarydynamicsof an entirepopulation. under a strange form of nmaway sexual phenotypes . both preferencesand phenotypesadaptto eachother Sexualselectioncan tIlus take on a life of its own. Conclusions about the importance of this line of research Copyright Geoffrey are offeredin section6.election are mechanisms called ' directionalmatepreferences ' (Kirkpa- assumed to generally dominate the dark Dionysian - trick. we showedthat certain kinds of simple assor. Even when there matesbecomesas important as finding food and are clearly defined natural-selectivepeaksin an adaptive avoiding predators . for our previoussuggestions(Todd & Miller . tIlat directionalmatepreferencescan causepopulations In the simulationspresentedhere. under' sexual-selectivepressures . once psychologicalmechanismsof mate choice this work is providedin section2. possibleapplications interact in complex and often surprisingways. and finally conclusions . 1987). simulation results. away in phenotype space in a particularphenotypicdirection (e. 1991). These tive mating preferencescould causepopulationsto two kinds of matepreferencesresult in very different evolutionary spontaneouslysplit apart into separate species dynamicsand makepopulationsmove aroundin ( Todd & Miller . strong the population follows a capricious trajectory a distinct and importantpart of the environment to which the species ' through phenotypespace that looks a bit like sped-up phenotypesadapt. term. in which attracting potential wandering . they can look at sexualselectiontheoryS ~ tion 3 presentsour have profound effects on the further courseof evolution. Theseresultsprovide more evidence sexualselection. or richer than oneself). Land Boulevard Falmer. Todd Schoolof Cognitiveand ComputingSciences The RowlandInstitutefor Science University of Sussex 100Edwin H. For . we showed that non-directional assorta - is much bigger. BNl 9QH England Cambridge . But this neednot be ideal matejust has somedegreeof similarity (or dissimilarity tile case.Sexual selection with directional mate preferences GeotTreyF. Over the short to point in tile direction of natural-selectivepeaks. Miller Peter M. particularly with a classof selectivemating ' ' ) to one s own phenotype . genetic drift. result in a sympatricpopulation of simulatedorganisms splitting apart into distinct species . One might expect that imposing moderate These results suggest a broader conception of naturalselectionpressureswould eliminatethis capricious ' ' adaptive behavior. they do not alwaysevolve matepreferences andevolving phenotypes . 1987). often climbing adaptivepeaksbut seldomstaying landscape practical and scientific applicationsof simulating there for long. With directional mate mechanismscalled directional mate prefer.g. but we show that it doesn't. and phenotypictraits evolve to fulfill currently selection. 0 1993 Miller and PeterTodd.sussex . Evolutionary wander lust : . sexual selectionsets selectionpressures popular . 1991. We presenta framework for . and discuss some . a populationsubject to sexual selection with landscape simulating a wide range of directional and nondirectional directionalmate preferenceswill still wanderthrough the mate preferences . MA 02142 geoffm@cogs. mate-choice mechanisms . In previoussimulation preferences . But over the long such that mate preferenceswithin a speciesbecome term. Todd & Miller . In this paper. An intellectualcontext for example. with or without the influence of natural mate preferences .org Abstract . we go In the pantheonof evolutionaryforces. we show phenotypespacein very different ways. This paperprogress es from.

Huxley ( 1938) presentedmany ill -conceivedbut often. important. The mate preferencesthat serve as the mates. process . As preferencesare due to intrinsic sensory . phenotypictraits. at leastuntil betweenadvocatesof ' viability indicators'. But the possibility of female choice mdeed. Once biologistsstartedtaking willingness to mate with non-ideal mates. but he did not explain the evolutionaryorigins between coevolving traits. traits that indicatehigh natural-selectiveviability . evolutionary processthat has been neglectedin biology Sexual selection is then driven by the topography of until quite recently for demonstrablyideologicalreasons . 1980.y) location in phenotypespace . Lande ( 1981). evolved POM ftmctions interactingwith the evolved frequency and yet has been shown to be widespread . 1992). perceptual a result. The the possibility of femalechoiceseriously. Our 1982. suggestthat sometimes for certain male traits.g. Male competitionwas widely acceptedby Victorian selectiveenvironmentto which sexually selectedpheno- biologists as an important. In this process . 1991). determining not only the ( 1992) confirmed the plausibility. comprehensiveprobability-of-mating. both the elaborateness of the traits and the extermityor cognitive blases . just two its nuances . while a probability of one indicateswild. Eachindividual alsohas associated speciesis evolution throughfemale preferencefor certain with it its own genetically determined probabiIi ty-of- kinds of genital stimulation. non-directional preferences(Todd & Miller . but also of runawaysexualselection. large. speciesadapts . themselvesevolvetmder variousforces. the symmetric. capriciouselaborationis the hallmark of sexualselection: Every individual in our evolving populationshas aparticular Eberhard( 1985) arguedthat the only feasibleexplanation phenotype . who suggest ' Fisher ( 1930) proposedhis model of runawaysexual that mate choice mechanismsalways evolve to prefer selection'. plotted as a squarewith its oppositeedgesef- . Andersson . Ryan. and Kirk patrick ( 1982) showedthe mathematicalfeasibility of Fisher's runaway To explore the intricaciesof sexualselectionwith directional sexualselectionprocess . and can be representedas a particular In natural selection. and the male traits themselves . 1992. and ideal matepreferencescan thus be displayedtogether but not vice-versa. and advocates feedbackloop getsestablishedbetweenfemalepreferences of ' arbitrarypreferences ' who . and generalevolutionary typic traits adapt. obsessivelust (or its simulatedequivalent). The new population genetics models of 3 Simulation Methods Q' Donald ( 1980). not to its infeasibility or disinterest . both the specific runaway within a speciesbetweenmate preferencesand sexually model and the generalnotion of female choice were left selectedtraits. We furthermoreuse a toroidal pheno- study empirically and formally. necessary . Catchpole .g. Todd 2 A Brief Review of Sexual Selection Theory specify a stable external set of conditions (e. and female choice of male always evolves. the colorful. We suggestthat it is time we dust off Darwin s To makethe phenotypespaceeasy to visualize. evidencefor individual' s POM ftmction assignsto every phenotypiclocation its existenceand significancecame quickly and ubiquitously . directional mate to languishunexamineduntil quite recently. the selective' environment ' itself competitionfor female mates. however. then. of work in biology on sexual selectionthrough mate choice. a predefined ' fitness function' to which a ) . a certain . is a major . an evolutionary positive. Geoffrey F. And as we will show. New behavioralexperimentson matepreferences . we expectthat it will cited argwnentsagainst the possibility of female choice be critical in understandingthe reciprocal interactions and against Fisher's model. Kauffman& Johnson . Cronin ( 1991) provides a readable . and the novel. This makesnatural selectioneasierto in the sameplot. becauseit involvessuchcomplexinteraction plwnage. Hillis . and much more detailedaccountof this history. because one can often type space . New evolutionarycomputer mating (POM) function. Just as computer of female preferencesthemselves . Millelo and Peter M. Individual phenotypes forces is often one way: organismadaptsto environment . and power phenotypiclocationof an individual' s ideal mate. 1985). Each choice. and thus every potential mate. nor did he developformal simulation has been important in understanding the genetic models of runaway sexual selection. and distributionof availablematesin phenotypespace. unquencha objective unimportance . This function is defined across simulation modelssuch as those of Collins and Jefferson the entire phenotype space . we modified our previousgenetic animalsshowedthat femalesof many speciesdo exhibit algorithm method of simulating sexual selection with strong preferencesfor certain male traits (e.g. driving In the last 15 years. This is Fisher's model could accouptfor the wildly exaggerated exactly the sort of debatethat can be illuminated best by male traits seen in many species ' . which can be representedas a particular for the wildly complexand diversemalegenitaliaof many locationin that space . preferencescan often evolve to be quite arbitrary. After dynamics of coevolution between species (e. the causalflow of evolutionary (x. though. ' powerful. This probability represents This peculiarhistorical sagasuggeststhat the scientific the individual' s willingnessto matewith someprospective ' neglect of Darwin s processof female choice was paramour : a zero probability indicates total disdain and due to sexist blasesin biology. What we have. we long-neglected theory of sexual selection through mate restrict it to two dimensionsin these simulations. and use the power of computersimulationto exploreindividual thusembodies . there is a fierce debatein sexualselectiontheory was almostuniversallymockedand dismissed . such as the greater salienceof the of the preferencesincreaseat an exponentialrate. In Darwin postulatedtwo kinds of sexual selection: male sexualselection . height in the n+lst dimension . therehasbeenan explosion populationsawayfrom natural-selectivepeaks. robustness . which can be representedas a . andhaspreferencesfor. such as the peacocks computersimulation. New comparative generalframeworkfor simulatingmatepreferencesis cast ' morphology has supported Darwin s ( 1871) claim that within the context of an n-dimensionalphenotypespace.

in which the POM function tend to die virgins. In contrast. Todd & Miller . mate-poor regions of phenotypespacewill have directional preferences . given individual s POM peak can be forced to be vector. operating on all genes can keep mate preferencesand determinethe direction in phenotypespacealong which phenotypic traits from converging pelfectly onto one the POM function is offset from the SRP. parent-relative preferences in point phenotypespace . (4) with phenotypes . morelike a storm-tossedoceanwith wave-peaksrising previous work (Todd & Miller . but we have with them. the POM momtain range obtainedby sum. and the less picky' the individual is nevercompletelycoincidewith the population's frequency aboutits potentialmatesin termsof the areain phenotype distribution peaks. Those relative preferencesto sexual conformism. Together. and genetically specified. samplingerror (geneticdrift ) determinethe individual' s phenotypictraits. We showedin scale. individual-relative preferences the heightsof every individual' s POM ftmction for every correspond to narcissism . ' has a conical shape. Thus if we plotted an individual' s POM " type "of a parent. The dynamics of sexual selection to rememberthat positions in this abstract2-D thus dependcritically on the way POM functionsare defined phenotypespaceare not spatiallocationsin somesimulated and geneticallydetennined . the peakof the cone-shaped But becausePOM functions are determined by roM function can be detenninedin two general ways. if one summed whimsically: in humans . It is importantmutual satisfaction . at a longer time itself to yield non-directional preferences . . one gene detenninesthe width of the some distanceaway from the individual' s own location. with just describedin the following way. geneswhich can evolve. A whole population's matepreferences could " space -relativepreferences " the SRPis someabsolute . direction. discussedthe other threetypeselsewhere(Todd & Miller . which individualsadaptthroughsexualselection. We now explore some world. And sexualselectionwill drive the (Kirkpatrick. it is as easyfor two individualsfar apart referenceposition (SRP) somewherein the phenotype in phenotypespaceto meet and mateas for two individuals space. individuals with POM is offset somedistanceaway from the SRPin a particular functionsthat peak in denselypopulatedregionsof pheno. population- or mountain range of mating probabilities. This abstractmountainrangefonned from in. functions will generally evolve towardsthe peaksin the originating at the SRP and ending at the center of the currentfrequencydistribution of individualsacrosspheno. an4 therebymaintain the sexual-selectivetension determinesthe distancein phenotypespaceby which the between them that drives the population's movement roM function is offset from the SRP. base of the roM function: the wider the base. Two genes . These directional preferencescan will likely have more offspring. these throughphtnotypespace. roM function. such that probability of mating is identicalto the individual s own phenotypelocation. the POM peakcould simply be centeredon the SRP range of sexualpreferencesis actually. 1987).e. . and the two would endlesslychase spacethat its POM function covers. on one another and thereby reach a stable equilibrium. Given a particularSRP. Across a whole population. dependingon the model of sexual selection being explored In the work describedhere. position be representedas an overlapping set of such conical in phenotype -space . Two genes small populationsespecially. this apparentlystable mountain First. Mate preferences To begin with. submitted ). To illustrate this somewhat momtains. the SRPis the pheno- from that point. Evolutionary Wander lust fectively connected . dividuals' POM ftmctions is literally the environmentto 1991.e. . Many speciesprobablyuse a combination evolve towardsthe peaks of the POM functions (i . the SRP is the averageof all phenotypesin the momtain poking up . The genotypesin this simulation encode the elements But this need not happen for two reasons . Mate preferences assortativematepreferencescan result in spontaneous evolve becauseindividuals with POM ftmctionsthat peak sympatric speciation . closetogetherin that space. This SRP can detenninedin one of four ways. the of non-directionalpreferencesfor somephenotypic phenotypeswill evolve to fulfill the preferences ). one would obtain a whole complicated correspond to an Oedipus Electracomplex. and becausethey seem quite peaks in that population frequencydistribution itself to commonin nature. Second . So the peaks of POM be plotted as matepreferencevectorsin phenotypespace. doesnot imply geographicseparation . 1991) that such nondirectional and falling as generationsgo by. First. The resultingbinary each other acrossphenotypespacein a futile attempt at genotypefor eachindividual is about 120bits long. each individual has its own sexual permitting. the POM ftmction always : ( 1) with " individual-relativepreferences " the SRP . dimensionsand directional preferencesfor others. it would look like a conical . We concentrate perchedat a high elevationin this mountainrangewill be here on parent-relative preferencesbecause they are sought after by many other individuals wishing to mate better-documentedin the biological literature. with directionalpreferences a . and space - individuals who are lucky enough to have phenotypes relativepreferencesto romanticidealism.from the two-dimensionalplane of population or some specified sub-population. individuals can in deserted . steeplythe POM function slopes'off from its peak down ming together POM functions across individuals could to zero probability. Directionalpreferencesare of interestpri- type space(i . the less In this case. the preferenceswill evolve to exploit the marily becausethey can facilitate runawaysexual selection availablephenotypes ). Finally. (2) - -space and falls off " " with parent-relative preferences (which can result from peaks at some point in phenotype linearly until it hits zero at some radial distanceaway sexualimprinting at a young age).' Second three genes determinethe individual' s mate preference . just to avoid edge effects. and one gene another. We One might expect both sorts of peaksto converge focushereon the effectsof directionalpreferences . (3) with population-relative preferences function in three dimensions . with both distance and direction parameters type spacewill find a plethora of acceptablemates. arid that phenotypicseparationbetweenindividualsof the possibilities.

and theseare the genesof point NE. we expectedthe population as a whole to selectionpressurespresent. his POM function and her . or at the SRP end of the vector for the dad -role. and call this comer of his PQM fimction on his SRP). not picked as a potential parent. one individual is selectedto play the on the courseof evolution. mom s POM fimction is constructedbased mate preferencespointing the upper-right comer to on her directional preferences ' of this abstract space . ' work. Miller and Peter M. Todd The populationsize is fixed in thesesimulations(as 4 Simulation Results it is in most geneticalgorithms). This continuesuntil a successfulmatch is the individual plays the mom-role. because .01 per bit). but theseare simply centered . eachyielding two new offspring). but originates at the individual ( in this parentless first generation is just the phenotypic location of the powerful directional selectioostill results even widt this individual in a itself) and extends genetically determined asymmetricscheme . (Note that in fied a small minimwn lengthfor the directionalpreference closer to the comer of phenotype vectorsto ensurethat they wouldn' t devolve to be effectivelythis plot thereare somedots not connectedto arrows.values on both of their phenotypic traits. a parentalphenotypic phenotypes preference location. For clarity. The entire mating processis repeateduntil the with radii in thesesimulationsabouthalf the length of the next generationis filled (that is. the preference . as we will now see. which originatesat its SRP. But becausethey are free to evolve. Other possible methods. greenerwings) on their phenotypictraits than probabilitiesare multiplied (representingmutual coosent ) y to yield an overall probability of mating.=e) has . The conical 3-D POM ftmc- found for this mom. a few most interesthere. speci NE -space. letting both It is importantto rememberjust what is evolving in this population . But the each associated arrow shows that individual prefererlCe and the direction of the vectors are vector. E~ h 's theoreticalreasons .g. The NE-pointing vectors shown here mean that ' and dad s probability of mating widt mom is determined every individual would prefer to mate with anotherwho . Since ' responseto sexualselectionpressuresexertedby the mate every individual s mate preferencevector NE preferencesof the populatiooas a whole. so are now populatioo paper. with all of their directional type space. mutating to point NW inste~ . Next.vectors for everyone are createdand put into the next generation . and we used a (small) fixed width somearrowsnot connectedto dots -. failing to meet each others' cone-shapedPOM functionhere. selvesalso evolve from generatiooto generation . did their parent(their SRP).e. one (extendingoff the E edge of phenotypesp. we do not display each individual' s and dad prove unlucky. 1991). and then mutating the resulting playa children slightly (mutation rate . Next. The offspring ' t know aheadof time - are madeby applying 2 point crossoverto the two parental displayed because we can who will mom -role andwho will playa dad-role during mating bit-string genotypes . given her POM function and his phenotype . for complicated 1. tracking showing the populationat generation10. To simplify our discussion . bigger antennae ) and a higher given phenotype -value (e. For the simulations in this we . ). First. or until she has proven too finicky (our criterion is going through 500 failed mating attempts tions would project down onto this 2-D spaceas circles. If the mom . Geoffrey F. move initially in that direction.1 Sexual Selection Alone use the following sexual'selectionmethod(an earlier version of which was describedin greaterdetail in Todd & To Wlderstandthe basiceffectsof directionalmatepreferences Miller . population of 100 individuals at generation o is shownin preferencesof both individuals. To create the next generationof individuals. and see where will change from generationto generation . this has been found to make it Figure individual phenotypic location is shown almost impossiblefor any individual to find matesin our as a dot. and eachindividual' s directionalmatepreference - simulations. we turn them loose. a new dad is chosenand tried againwidt the at the arrowhead -end of the preferencevector when preferences same mom. for determining is shownas an arrow vector . while ' ll who mate with anythingfrom evolving.g. reproductiveability. most of the preferencevectors still always free to evolve.these are temporarygender roles. until 50 successfulmatingsaveragepreferencevector. so only their successat genetically detennined sexes. As we see in Figure 2. These two has a higher x-value (e. Our initial populationwas clustered or at random if there is no natural selection at in the center of phenotypespace . a die is thrown. even when we preset their initial valuesin the first generation .) Moreover. this is exactly the locationsof the individuals(i. we 4. here we use 100 individuals . we will . mom' s probability of mating with dad is directionfor a geneticallydetennineddistance. Both mom and dad are selectedprobabilistically according to whatever natural choosing mates and at being chosen detennines their selectionfitness fimction has been imposedover pheno. potentialmates) in the what h~ s: most individuals have evolved higher . But rememberthat only the "mom" in es this directional and if the parentsare lucky then two new offspring are each mating pair express preference . our initial simulations role of "mom" and anotheris selectedto play the role of a used sexual selection alone without natural selection. The phenotypiclocations of individuals phenotypes and preferencesevolve freely. The mate preferencevector ' s SRP which directional preferencesmay not have this trouble. by centering use compasspoint terminology . determined . " " Thus.this is becauseeach ' for the POM fimctions to keep overly indiscriminateindividuals showsan individual s own phenotypiclocation dot 's . every individual has an equal chance of being potential dad -. We do not use the directional abstractphenotypespacethe "northeast " NE This initial ( ). and non-directional. and to any naturalpointed . After starting theseinitial individuals in the center of phenotype -space. and dad s is constructed - basedon his non-directional preferences(i. have started to diverge. have occurred. evolving in sexual selection takesthe populationover successivegen- eratims. The preferencesdtem.e.

After startingin the phenotypesto evolve in that direction. which in turn causesthe populatim averagepreferencevector.aroundof our phencxypespace . whenthe population ~ re. and so seem to . This starts to view of generation0 to 300 shownin Figure 3. the population marchessteadily upwardsin compassdial.g. phenotypesand preferences . The preference center of phenotypespace (dIe central circle) with NE vectorscontinue to swing counter-clockwise around the preferences .space . pointing every 50th generation ). Figure 3. . seemsto be violatedafter generation50. now shifted NE and with some diversity of preferences. move S. and dIe averagepreferencevector evolvesto point in the ~ ' " . The initial population . and the phenotypespace . following the NE-pointing preferencevectors. The population can drift in phenotypesevolving to match the preferences . when oree again the population heads { 7 NE. averagephenotypethen moves straightS (down). In fact. I I I . Here we happennoticeably aroundgeneration200. This evolutionary noise can affect both . But this is an ar- tif ~ t of how the populationaverageis determinedgiven ~ the toroidal wrap. The population after 10 generations. Perhaps This NE migration of p~ ulation phenotypescmtin . meaning that new preferencesmust preferencesevolving to matchdIe phenotypes . now begin to slowly shift. stochasticnature of the evolutionaryprocess(e. : hes the end of its first NE nul. This pattern - . Evolutionary Wander lust Generation 0 Generation 10 . due to the noisy. / . .---. FigUJe 2. the population . The p0.aderlin the same direction.' --. ~ pulationhas actuallymovedoff the t~ edgeof phenotype : ~ . Sexual selection alone over 300 generations. the phenotype / preferencecouplingis still . Figure 1. clustered in the middle of phenotype space with preferencespointing NE . weakeningthe reciprocalinteraction pulation is still he. We can ~ this in SE comer after genera tim 150. widI the between them. . as we saw in Figure 2. the p0. in the middle of have plotted a dot at dIe averagephenotypeof the entire the E edgeof phenotypespace . : . ' averagesdon t take this into ~ ount. Preferencevectorswhich populatim every two generations(or a circle to mark beforehad pointedNE. showing IX>puJation aviage phenotype and directional mate P' 8feren~ geneticdrift due to selectionalsamplingerror with small every two generations. but the phenotype / . evolveto ensurethat individualswill find mates. In generalthe populationaverageevolves in the direction that the averagepreferencevector points. that direction. as we can seein the "time-l ~ " vectorscan drift . / working all along. populatioos ). ~ direction that the population is heading. But overall. the direction of the preference ues for 50 generations . . mutatedto a much larger magnitude . arxI reappearedat the bottom.. . . the preferencescontinueto point NNE. even more importantly. pointing more and more westerly: first NW. while . with its preferenceaveragecontinuing to point that way. But this feedbackcoupling betweenphenotypesand preferencesis somewhatnoisy a I KIinaccurate . . and a line segmentto show the more and more straight N.

4. mate preferences play. would sprint up theseisofitnesscontours. we againini- " - unique type of evolutiooarytrajectory shownin Figure 3. reflecting the fitness can indeedlead to runawaysexualselection. headingS and The coupling betweenpreferences and phenotypes W. one 150 generations . and changesin one will force congruentchanges fitnessessimply influencethe probability of an individual ' in the other. optimal. the natural-selectivepeak. there is an ever-shifting sexual-selectivelandscape the average mate preference vector. adap finally tive : phenotypictraits adapt to the current mate preferences natural selection . and would quickly end up clusteredat the peak in the under sexual selectionensuresthat neither will shift too SW comer. which drives the population their combinationdeterminesactualfinal fitness. Geoffrey F. wherein the populations movementthrough reproductivesuccesswill also dependon sexualselection. Wander lust returns: sexual selection in the NE are given the lowest. and force the population respoosibly predominatelySW. poweredby directional " mate preferencesthat ' strongsexualselectionfrom generation generation to . and no longerpoint towardsits peak. in a consistentdirection. high-fitnessmatesare also sexuallypreferred. But' with directional 4b. first What happenswhen we introduce natural selectioninto reigning. equilibrium. all is well at the fitness . SW Gowerleft) comer are given the highestfitness. engagedin and the populationhoversat a high-fitnesspoint slightly E a kind of aDDs -race that neither preferencesnor traits can of the peakfor severalgenerations . And they ? Whereasthe population in the previous simulation reassert their commandover the population after about wandered capriciously through phenotype space.term course of evolution is : population does a switch -back. By generation450. so that in which evolutionrunscannotbe predicted. result in the are in direct oppositionto naturalselection. up the fitness ftmctions slope. To see what happenswhen sexual selectionforces which change the population ' s direction . -. as shownfor generation226 in Figure the preference upwards to some adaptive peak . betweenthis momentumeffect. that individual s final ' hysteresis . the .2 Sexual and Natural Selection Combined But this appearance happy optimality does not of last forever. but the direction landscapeand reinforcing its selectivepressures . after generation250. The drive a population along a rapid but winding trajectory populationstartsheadingNE in the direction of the mate through phenotypespace. away NE from the - pulation to evolve further (which we have done many SW direction of the naturalselectivefitnesspeak. the two play catch-up with eachother. to operatesimultaneouslywith sexual selection selection . Miller and PeteloM. Rather. to mate and have offspring if they each meet the other s ' . we constructeda single-peakednatural. it would continue wanderingabout in phenotype and dadsare now sampledfrom the populationaccording imposeto their natural-selectivefitness. For the dark forces of sexualselection. The preferences acrossphenotypespace. yielding effective sexual selection. sexual selectionseemsto dominate. then overthrown. generationafter generation . the individuals in the populationremainclustered might expect that imposing some stem naturalselective peak fitness function would bring an end to all that sexual . shifting to point more southward population. spacewith preferencevectorspointing . But in our simulations . This can result in a kind of evolutionary being chosenas a potentialparent. but which keeps them nmning quixotically closesin on the peak in the SW comer. straight W. with directional mate preferencesini. and mate preferencesadapt to the current pheno. this movementhas stopped . Thus. Directionalmatepreferencesthus then swing around to point SW. For a while. But the vectorscontinuingto point NE. there . and their preference vectors continue to point selective rambling. becausethereis no point. but succumbs to the pull of any' direction thereafter. Moms times). and d1enjoining rankswith natural our model. the preferencevectors try to catch up with the stable ' adaptive landscape ' external to the . They now point in To investigate the interplay between natural and all directions. the iso. tialized a starting populationin the center of phenotype If we let the simulationkeeprunning and allowedthe the s wander lust can vectors escape the magnetic pull of the fitness ftmction population be swprisingly robust. curving around nmning diagonallyfrom NW to SE. influenceof both selectiveforcesis shownover 500 generations goal for the populations movement This simulation shows the capricious nature of in Figure4a. remainpowerful and capricious. fitness contoursin this landscapeare just straight lines tion follows (or maybeleads ) ? this trend. despite the preferences typic traits. But after about a dozen generations tialized to point in some direction but free to evolve in . with many startingto point northward. thesenatural-selective rapidly. The interplay overlaid on the fixed natural-selective fitness ftmction. phenotypespacehas a sort of momentumrepresentedby Thus. And there the populationstays. If a standardgenetic the N (top) edge of phenotypespaceuntil it ends up in algorithm were nUl in this fitness space. matepreferencevectorscontinueto drift aboutin different selective fitness function as follows: phenotypesin the directionsuntil a generalnorthwardconsensusis reached . but they still will only get space. and the geneticdrift effects. directional sexualselection. and startsheadingstraightW . those around generation 400. and the way in which it can At first. The populations movementunder the but with nothing to guaranteea consistentdirection or ultimate mate preference ' . After 50 generations . sexualselectionalone. obliquely ' . dithers around a bit . The ~ ula. The sexual selection. and the fitnessof thosein enticesthe populationtowardsthe north. phenotypicmovement . changing direction to long continuously capricious neither phenotypictraits nor mate preferencesever settle headsstraightS up the fitness ftmction slope. The population then the short -term evolution is ' . Finally the population ever win. its population the NW comer at generation300. in Figure 4c. But by generation378. just as it did in the previousnUl with natural selection). away from betweenfalls off linearly from SW to NE. Todd and then. hI a flat fitness landscape(no preferencevectors. At this down to a stable. in Figure .

\ . Of course. preferencesbegin to drift after a while. Generation 378 Generation 450 - . a. ~ . with N directional preferences. so that natural and sexualselectionpull in If we ran this simulation further. as Figure populations preferencevectorsSW. The resultsover 500 generations continual battle between homesicknessand wanderlust. and then straight compelling the population to wander through the W. " " . The population still clustered near the fitness peak at generation 378. Sexual selection and natural selection initially pointing in opposite directions.selective fitness peak at generation 226. Evolutionary Wander lust Generation226 . . ~ : . selectivepeak. r . with both affectingthe courseof evolution. plotted for 500 generations. b. The populationconvergeson the peak within 50 generations wildernessof phenotypespaceunder the momentumof here. - r . The population shifted in position significantly by sexual selection at generation 450 . We next exploredthe effect of pointing the initial ' and this progresscontinuesover manygenerations . . . o . the two are often balancedin a tenuoustug. comparedto the 200 generationsit took in the directionalmate preferences . and sexual selectionpeak. . .. . . This time. . soonrearsits seductivehead. 1 ~ / / / . ... Figure 4. . so that it alonecontrolsthe remainsat the peak. " . . 4a showed. . ' . with SW directional preferences. The population clustered at the natural. . . . the population forcescould be madestronger. . then straightS. " -' - J .. . c. . the populationmovesvery rapidly toward the fitness comfortablehome on the fitness peak.. But we believethat in natureas in this SW. .. we would see a the samedirection at first. 4d. - . with preferencescontinuing to point ' populations path. " T . . are shownin Figure 5. . But onceagain. . . . . for many generations .. . " " " . with varied directional preferences. . as we would expect with natural selectionpulling the populationtowards its . . . . and the populationbeginsto . . D - fI ~ ? ~ . sexualselection of-war. . towardsthe natural- . one of these the previouscaseof opposition. the directional simulation. . . d. . j . As before. the population has moved significantly northwards . first headingSW.

then it mustremainfairly coherent and consistent natural selection . conferring somesexual-selectivemomentwnon a population while it climbs an adaptivepeak under the influence of naturalselection. . searchingwithout spontaneous speciation minor. First. for example . sexual selection can allow spontaneous slower evolution toward such a peak. Thus. The preferences can . making it faster. adaptivepeakshavebeenexplored. and more consistent . So over the short term. to which the population is (at leastpartially) subject. widI or without with natural-selectivefitnessin suchcases . Miller and Peter M . With a large enoughinitial 5. or even divergence sympatricspeciation(seeTodd & Miller . so that ad-hoc methcxissuch as predefined scaling algorithms FigureS. Without natural selection. we have focusedmore on arbitrary preferences . Todd selectionmight be aidedby including somesimulatedsexual selection . a single evolutionaryrun might be sufficient The relationshipbetweennatural selectionand designoptimization to exploremost of the peaksin the fitnessspace. selectivematepreferencescan allow a population to internalize natural-selective pressures . thesesimulationsshow someof the capricious1982). but there is strong evidencefrom biology that many mate preferencesdo pick out viability indicators(e. dependingon the dir~ tions dIe preference speciationmay allow more efficient niche exploration vectors evolve. We and climb only one adaptive peakat a time.e. W Second . sexualselectioncould keep a populationevolving along even when the objectivevariancein natural-selectivefitness ably over the long term. most male adults are healthyenoughto survive there is nothing to keepthem in line. Among many bird species were initialized to complementnatural selection. Searchingwith spontaneous speciation . To explore a strong pressures . so there is large variancein their sexualselective fitness . along a trajectory that shows high directiooal consistencyand momentum selectionin a way analogousto fitness scaling. 1982). Geoffrey F. to the extent that mate preferencesevolve to be ' utilitarian' preferencesfor viability rather than ' patho- ' logical preferencesfor arbitrarydisplays. In general. it has often beenoverlookedas throughsexualselectionis like samplingwith replacement a design tool. Clearly. This sort of away from a peak. the former can natural selection. The ability of matepreferencesto respondto viability in- dicators may be particularly useful and important when the mappingfrom phenotypeto natural-selectivefitnessis / / noisy. it can be methodsfor designoptimizationthroughsimulatednatural highly inefficient). In this paper. directional be thought of as magnifying the latter. in thesamedirection . Sexualselection andnaturalselection initiallypointing and rank-basedselectiondo not have to be used. 1991). Since sexual -selectivefitnessis highly correlated power of dir~ tional mate preferencesto affect the courseof evolution mder sexualselection. even though the directional mate preferences variancein a population . If a populationdoes not have the ability to general preferences particular play powelful roles in evolution. In essence . selection has often been considereda pathological. leaving the fitness peak of potential mateswill tend to increasethe effective fitness behind. sexual selectioncan be thought of as imposing an automatic. plottedfor SOOgenerations. Finally. so there is little variance in their natural-selective fitness . more robust. sexual selectionforces in andexploitationin complexadaptivelandscapes with multiple and directional mate in can peaks . emergentform of ' fitness scaling'. Hamilton & Zuk. sometimesevendominating spontaneously speciate . both in assistingthe evolution of humanthrough many simulation runs to be sure that different technologyand in understandingevolutioni. noo-adaptiveforce.g. But with spontaneous speciation . In essence . But only a few are healthy enoughto produce ultimately evolve their own agenda . a population can split up to climb different adaptive peaks as they are encountered . because has been explored at great length and used to the run will generatea branchingphylogenywith separate great effect in many practical contexts. plied as well. Selectivematepreferencesbasedon the observedviability move N in phenotype space. one would have to go may prove useful. But because sexual species exploringseparatepeaks. directional sexual selectioncan result in more rapid evolution toward an adaptive fitness peak. - Overall. but that changesdirectionsmpredict. without the experimenter 5 Applications of Simulating Sexual Selection knowing aheadof time how manypeaksthereare or wherethey can be found. sexual ' sexual selection can make' a population wander through selections ability to magnify natural-selective fitness differencesmay increasethe powerandconsistencyof natural phenotypespaceat a fairly steadyrate. sexualselection can reinforce ongoingnatural selection.n nature. When natural selectionis ap- is rathersmall. We present three possible ways that over many computationallyexpensiveruns (i. sexually-preferr~ plumage (Hamilton & Zuk. brilliant. will now turn to ways that simulating sexual selection complex adaptivelandscapethen.1 Practica J/ Applied Uses population.

1966): clarity of function. Computersimulationcan circumventtheselimits. Likewise. . Directional preferencesfor viability ' indicators could be used to facilitate the exploitation of adaptivepeaks and 6 Conclusions speedhill -climbing. parasites . And non-directional assortativemate preferences onto a few (locally) optimal solutions given preestablished may facilitate spontaneous sympatricspeciationand therefore problems posed by the econiche . design. ( Miller . preferences . and protean behavior) wherein the nervous systems range of speciescorrespondingto a range of distinct. Indeed. physiologically expensive traits that ' simulatims of sexualselectionwill probablyplaya critical sometimesimpair an organisms ability to copewith other role in our scientific l Blderstandingof evolution in organisms( g e. to locate prey: if camouflageand warning coloration are types. sexualselectionmay be primari. the causal effect of evolved mental but a nonnal one':'speciesgenetic algorithm will only be mechanisms (such as selective mate preferences) on the able to exploreone adaptivepeak in the spaceof possible further course of evolution. we intend to explorethese is an overly restrictedview of what adaptive behavior in future research . structure. and origin of adaptationsresulting from sexual displays) can only be evaluatedif one understands selection . . selection in contrast often results in an unpredictable . and may keep climbing that samepeak selection may illuminate other evolutionary processes in run after run. Kirkpa- thoseassociatedwith courtship. i .2 Scientific / Theoretical Uses spontaneously and exploringthe spaceof phenotypicpossibilities according to their capriciously evolved mate Sexualselectionhas been such a neglectedpart of evolu. andefficiency andreliability of operation. especially fonnally intractable interactions importantas the perceptualmechanismsthat predatorsuse betweenevolving mate preferencesand evolving pheno. This usemay prove critical trick. 0 . then so must be sexually-selected es that traits. The history of sexual selectionresearchis a that the humanbrain itself hasbeenenlargedandelaborated clear exampleof go<xi theory precedingand priming the through runaway sexual selection acquisitionof empiricaldata. empirical evolutionary biology has driven by natural selectioncould turn out to be betterexplained had troublewith sexualselectionbecausethe mechanisms as a product of sexualselection. fulfill all ly responsiblefor the vast biodiversity of sexuallyreproducing the usualcriteria for being legitimatecomplexadaptations animals and flowering plants (which themselves ( Williams. Also. We suspectthat much of the evolution assumed Traditionally. and certain macroevolutionaryprocess es . there may be the general importance of ' psychological selection' many diffe~ tways of designingan efficient jet engine. divergent pattern of evolution. 1871. Directional preferencesfor arbitrary traits may facilitate the exploratiooof complexphenotype Natural selectiontypically resultsin convergentevolution spaces . Miller . motivational. membersof a population 1 ( ) Simulations can very probably help theoretical adaptto the matepreferencesof their conspecifics. empirical studiesof mate choice in animals . . In the following three areascomputersimulation and predators ) or with the physicalenvironment . and involves selectionas a maladaptiveprocessbecauseit results in such mathematicallyintractable dynamics. con specifics. In runawaysexualselection. mechanisms . If so. and that computer that drive it (that is. As such. such as the peacocks tail. that computer highly elaborated . But a large simulationwith sexualselection (e.g. matechoice mechanisms ) are hidden simulationwill be valuablein demonstratingthis.e. and functionsconspicuously as a long-term courtshipdevice(Miller . warning coloration . Some evolutionary biologists view sexual tionary biology and evolutionarypsychology . emotional geneticistsunderstandthe complex dynamics of sexual . of other animals constitute the primary selective environment highly optimizedenginedesigns. the adaptationsthat sexualselectionproduces(e. 1993). pathogens . g. simulations of sexual designsat a time. processes and outcomes. they may help comparativeand evolutionary the matechoicemechanismsthat they are designedto excite psychologistsunderstandthe origins. away in the headsof animalsratherthan displayedout in (2) Simulationsmay help comparativebiologistsand the openlike more obviousaspectsof the econiche. general. prey. evolutionary biologists and mathematical population These matepreferencesare perceptual . 1993). with lineages speciating 5. there are frustrating . courtship function. especially geneticists (e. . We suspectthat somecombinationof mate that they can lead to distinctive kinds of evolutionary preferencetypes may prove very useful in optimization. 1982) coofirmed the possibility of runawaysexual in human psychologybecauseit seemsto us likely selection . Evolutionary Wander lust on the other hand is like samplingwith replacementduring (3) Simulations of sexual selection may help establish one big integratedrun. the human's brain (Darwin. nators).g. complexity of undergoa kind of cross-speciessexualselectionby polli. limits on how much theory one can build with mathematically tractable foDIlal models of evolution. and suchas speciation . For example. 1993). we might expect applications . However. and their effects did not really begin in earnestuntil theoreticalpopulation ' Donald 1980 Lande 1981 on the emergenceof certain kinds of behaviors . the evolution of camouflage. andbehavioralmechanisms that arejust as real and selection. paleontologistsbetter understa I Ki the ubiquity. means . and spootaneous speciationmay be able to producea mimicry . we hope to substantiatethese from other kinds of selective environments. Sexual efficient and automaticniche specialization .. Furth- . But this may prove particularly fruitful . the mandrill' s -face. and functionsof mate preferences . Sexual selectiondynamicsmay underlie both the legitimate adaptations . . elaborated And the adaptations sexualselection microevolutionaryprocess generatecapriciously ' producedthrough traits. Because nervous systems are very different In further research .

Chicago: University of ChicagoPress. P. external fitness Kauffman. The Tungarafrog . and vast.. Catchpole . Adaptationand natural selection.). and humanen- Acknowledgments . and S. andJefferson . IVllller ana reterM . The complexinteractionswe investigated Fisher. assortativemating selection. M. relevantthan resultsconcerningnatural selectionthat require Rasmussen (Eds. uniqueprospectsfor computersimulationto guide biological Hillis . Oxford: ClarendonPress. and enter the enticing J.K. D. and co-evolutionaryavalanches .). Heritable true fitness if not impossibleto deriveby traditionalmathematical and bright birds: A role for parasites ? Science analysis. But York. seductivemazesof bizarrephenotypic 78. The evolution of protean behavior . 313-324). ( 1985). The genetical theory of natural amongevolving matechoicemechanisms . 218. In prove more robust.R. simulation resultsconcerningsexualselectionmay sometimes poisedstates . Bradburyand MB . New leading apparentlyto problemsof circular causality.). (1930). and! NT-9203229for postdoctoralresearch . ( 1982 ). C. In F.B. selectionis a processof a speciescatalyzingits own Redwo<xi City. 3721-3725.). to construct an arbitrary. evolution as an optimizationprocedure . of complexsongamongEuropean warblers Todd. Submitted to the Collins . M. NY: CambridgeUniversityPress. Huxley. 325-370). 149.D. GiF. from which no species emerge untransformed Miller .W. The presentstandingof the theory of Computer simulations of both process es can take on a sexual selection . Cambridge . R.J. Sci. and decline. Thus. and Miller . Evolution. CA: Addison-Wesley. Models of speciationby sexualselection caught in infinite cycles of self-reflection and selfdistortion on polygenic traits. GiF. and biologically C. more elegant. Cambridge . the mate choice mechanismsthat causeevolution Cronin. J.M . For this reason. 11-42). wherespeciesadaptto their nicheslike Buicksscraping female preferencesin polygynous animals. The evolutionaryforcesacting on .D.G. by National Science Foundationresearchgrants O' Donald. sexualselection. Stanford Geoffrey Miller s portion of this work was partially supported Univenity .G. comein from the dry. Farmer. Belew and LiB. Proceedingsof the Fourth International 818-820. H. 36. C. .D. and S. ueullrey r .). which overflows with selection: Testingthe alternatives(pp. and Zuk. predefined. ( 1982). extravagance . We invite other researchersKirkpatrick. ( 1992). ( 1992) Co-evolution to function or environment . Nature . BNS 90-21684 to Roger Shepardat StanfordUniversity. The ani and the peacock: Altruism by sexual selection are themselvessubject to evolution. EuropeanConferenceon Artificial Life (pp.R (1992 ). Cambridge . In Normal sexual selection is a processof a species C. P. the study of sexual selectionoffers . W. Taylor. Natl. R. very precisemodelling of preexisting environmental Redwo <xi City. of autonomoussystems : Proceedingsof the First Princeton . Geneticmodelsof sexualselection. Sexual selectionand the evolution interestedin the simulation of adaptivebehavior to of femalechoice. ( 1980 ). Booker tail lengthin a widowbird . and speciationwould have beendifficultHamilton. dark tunnelsof love. Taylor. . Varela and P.K. M. directionalevolution driven by its own sexualpreferences .. ( 1985). M. 1000 ermore. ( 1991). ( 1938). . (Eds. Behavior . NJ: PrincetonUniversityPress. CA: PsychologyDepartment . Sexual fantasy-land of sexual selection. M..387. G. and Johnson . R ( 1981). Artificial Life II (pp. J. S. CA: Addison-Wesley. The evolution Proceedingsof the Third Artificial Life Conference of sexualselectionand femalechoice .. more self-referential form than : Essayson aspectsof evolutionary biology simulatioosof natural selectionthat requirethe researcher (pp. MA: HarvardUniversityPress. conditionsand selectionforces. Conferenceon GeneticAlgorithms(pp. J. G. we suspectthat the edge of chaos: Coupled fitness landscapes . and Miller . S. equilibratingto its own sexualpreferences . Bourgine(Eds.M. "without" natural selection: Sexualselectionfor 166. Towarda practice Williams. Artificial Life II (pp. roller-coasterrides of exponentialascent New York: Wiley. W. nmawaysexual Rasmussen (Eds. .C. ( 1982). 67-82). ' Stanford. Acad. Oxford: ClarendonPress. glaring parking lot of naturalselectionKirkpatrick. Langton. W. In G. (submitted). ( 1993).D. 327- 336).A. Proc. References Todd.J.J. ( 1991). On the sympatric origin of species : Mercurial mating in the Quicksilver Andersson . Evolution of the genusAcrocephalus . P. MA: MIT Press / Bradford Books.A. generalizable . Andersson(Eds.S. In their way into parking spaces . UK: CambridgeUniversityPress. 384. through psychological selection: Adaptive unpredictability . Langton. sexualselection.G. Ryan. Sexualselectionand animal genitalia explicit simulationmodelling of complex interactionscan . cephalization Unpublished doctoral thesis. ( 1980). 74. 547-554).12. ( 1966). 1. be most useful. de Beer (Ed. it is in such situationsof apparentcircular causationthat Eberhard . ( 1987). and sexualselectionfrom Darwin to today. Farmer. and speciation learning. mirrored funhousesin which speciesget Lande. C. Sexualselectionandthe evolution SanMateo.F. CA: MorganKaufmann. In R. Femalechoiceselectsfor extreme model. 299. Co-evolving parasitesimprove simulated theory andempirical research .) Evolution simpler.

- navigation task we will demonsb ' ate that goal-oriented non goal-directed designs are superior to goal directed behavior can be accomplished by non-goal-directed onesin manyrespects . the agentwith a set of goals.e. The solution without resorting to the idea of goals. This way sttucturescan be ' Goals: the traditional view. executed according to the principle of rationality : Typically this is done by executing plans which are briefly . Universityof Zurich Winterthurerstrasse190 CH-8057 ZUrich. a system will use its knowledge to reach its . . associatedwith a goal. Introduction " " containing a body of knowledge that specifies the The basicproblemin robot designist how a given task relationsbetweenthe eventsin the world and the actions description can be translated into mechanismswhich of the system.1-363 00 35 Abstract term s 1. Howevert this approach has goals. It is demonstrated The basicproblemin robot designist how a given task problems how agents can be disgned which navigate efficiently description can be translated into mechanismswhich produce the desired robot behavior. It can be viewed as 1. This is ill USb'ated proposed by classical artificial intelligence (AI ) has with a navigation task. The actionsof the behavior is adaptedto the particular environment It is basedon the physical symbol systemhypothesis( pSSH ) concluded that non-goal-directed designs are more efficientt adaptabletand cheaperthan goal-directedones.the cognitivistic view . are ' organized goals system They symbolst are then used to direct the robot s actions. a PSSconsistsof a memorycontainingsymbol structures. The underlying architecture. These disbibuted adaptiveconb'ol .organization. We start by discussingthe traditional embody aspectsof what is to be attained prior to notion of goals and argue why it fails to deal attainment of it. will briefly be exposedand ' goals are then used to derive the robot s actions. Roughly speaking. designs. Our approachis basedon two key ideast selforganized learning and a value system to consttain the Problems with goal . Using the example of a .directed systems processof self. Sincethere is considerable acquiredby the agent which control the agents actions - basedon purely local systemenvironmentinteractions. Institutefor Infonnatics. We can also refer to this body of produce the desired robot behavior. it is shown how the navigation task is solved through Howevert this approachhas proved to be inappropriate self-organizedHebbianlearning. The dominant task is significantly better than random. ( Newell and Simon .ch Fax.M . The solution of knowledgeas a world model. i . been to "endow" the agent with a set of goals. Symbols appropriately with explaining behavior in scientific lWe use die term " scientific " to contrast it widl common sense folk psychological explanations. classicalartificial intelligence (AI ) has been to "endow" either in the world or in the form of internal inferences . it must have goals. We will then show that for a number of reasons. We then show some of the fundamental of goal-directed designs.unizh. confusion about this topic in the literature we will The resultingperformanceof the agenton the navigation discussthe notion of goals in somedetail. and a set of actions. Newell . Thesegoalst designatedby are around the of the . Switzerland e-mail: pfeifer/ verschUI @ifi . Moreovert the view on behavior in AI .: +41. 1980) . 1976. Verschure AI Lab.J. Designing efficiently navigating non -goal -directed robots Rolf Pfeifer and Paul F. Newell ( 1980) argues: proved to be inappropriatefor the reasonswhich will be "A general intelligent system must somehow elaboratedbelow.

Calling the level at which alternative goals can be activated. animals explicit representations of goals and they are (and animats) need to keep a certain set of internal ' insttumental in directing the agents actions. planning system used in the Stanfordrobot Shakey. For example. designedfor autonomousagents. 20f course. 1952) . diis does not suffice for a general intelligence facing an indefinite shown that goals which are broadly used in everyday sequence "of novel and sufficiently diverse goal explanations of behavior are highly problematic as situations. under what conditions . die external ttansformation ( Newell . In odier words . diey must obey various ways. Any representation .g.) representationof the to-be-achievedgoal (i. p . Schank and Abelson's programsfor natural languageprocessing . Plans can be generatedin explicit representations. goal . In AI the vast majority of systems have been. It " " might seem an alternative to build goal .e. The important point about " " representation law is illustrated in Figure 1. e. In addition there is a conttolled spreading sensibly talk about goals. 1971 a ). The world model need not be complete but it must go beyondthe immediateenvironmentof the agent which is accessiblethrough the sensors . M. actions: the agent works towards the goal. 1990 ). McFarland( 1989) usesthe tenDgoal- goals which enable the system to react to the current situation. 59) . The system is goal-directed since it employs designatecertain internal states. . Below it is such as playing a game) . expert systemswhich are sophisticatedproblem solving programs. but thai valuable distinctions are lost and confusion will emerge easily. scientific concepts. 1990. Famous historical examples are GPS for problem solving. according to Newell ( 1980 ) intelligent Typically goal-directed behavior is accomplishedby behaving systems require goals . 172) . Modem AI programs are also largely goal-directed. we cannot - environment. There is a variables. J. specifically Figure " law (adoptedfrom Newell. . ( p. 1977) . must fulfill this law. where X is die original external situation and T is of certain types of predefmed memory sttuctures (e. within which it shouldbe kept) a goal of the agent delete-lists. and STRIPS ( Fikes and Nilsson. the essential variables within the so-called data-driven component which has the effect that viability zone (Ashby. too . Examplesare the use of a distancemettic " " die representation law : decode[encode(7) (encode(X ] = as in GPS ( Newelland Simon. Rolf Pfeifer and Paul F. These goals must have associatingplans with goals. based on STRIPS and uses the idea of add-lists and delete-lists which describe the effects of actions on the Unlessit can be shown that this law holds. 1963) or the instantiation T(X ) .orientation interpretation.directed our appear to be die only candidate for doing diis . Verschure that designate die situation to be attained (including directed for a system which has an explicit diat it is to be attained. is Maes ( 1990) which is goalsor otherwise . An example " 1: Schematicillusttationof the representation of a recent goal-directed architecture. The Schankand must obey the representationlaw ) into the structure of die system at design time ( as is and this goal representationis insttumentalin guiding its often done in programs diat have a single fIXed task. The tenD goal is sometimes activation mechanismin a network of actions for action (inappropriately) used in a more broad sense. etc. and still are. plans is that they all incorporate some kind of world model.g. The information " " containedin the world model can be viewed as global informationsinceit can be accessedat any time. In spite of the reactive component the would be inadequatesince the value of the physiological variableper se doesnot obey the "representationlaw" in systemis still entirely goal-directed since the purposeof the reactivecomponentis basically to activatealternative any obvious way2. Thus . namely to selection. In this system the the physiological variable is to be maintained (or the world model is essentially encodedin the add-lists and range . the term goal can be used in this broad sense. However .

Non . 1970. drinking coffee. towards the cafeteria early in the morning we can attribute the goal of drinking coffee or tea to the person.e. as we know from humans. If there are many alternativesthis processin turn ' agents behaviorcan be describedin terms of having the has to be basedon someheuristic searchprinciples since following goals: reducing thirst. 1977).this would not make any difference. But.Directed Robots The world model must contain the information that Moreover. 1977). we have no way of Swingland and Greenwood. we can ask. for example. and never ends up in the cafeteria. Iran-Nejad et al.example. what it will probably do next). althoughfrom are usedto organizean observer's thoughtsaboutan agent a cognitivistic perspectiveexplaining behavior in terms in order to come up with some predictions (e. it would not be possible to introspection yields unreliable data anyway (Nisbett & determine its goals. along the way. The obvious candidatefor and the mechanismswhich are actually responsiblefor an sucha level is physiology.e. 1989).whom we can ' ask . if agent get from location A to location B1 (Figure 2) . and althoughfrom a common others about the agent's behavior. In the cafeteria exampleour state. if someone heads Wilson. since goal ascriptions. However. An illustration: the navigationproblem and we interpret changein the agent's behaviorwhen the To illustrate this point let us look at a simple alleged goal is achieved (e. For example. we have to deal with the frame-of -reference that we have to look for evidence from other levels of problem (Clancey. eel) unreliableand leads to post-hoc rationalizations(Nisbett can travel back to their place of birth over thousandsof and Wilson. It is thereforenot possible to derive the agent's actionson systemswe distinguish betweenobservationand design. An important conclusion is that if an agent built exam which is coming up. Goals are ascribedby an observer to an agent and thus only exist in the eye of the observer. plausible and useful. thereis no physiologicalevidencefor goal-directed agents actions. 1977). being famous. Similar argumentshold for animal behavior. there is a lot of system) is to act success fully in the world. For example. there is the problem of the indeterminacyof goals. Baker. and this is the third ' point. or to influence the sense perspective explanations in terms of goals are behaviorof the agent. certain types of fish (salmon. for an observerto cometo a definite decisionon which is In order to discuss the problems with goal-directed the right one. miles. And there is no way must possessa body of knowledge. the person meetsa friend. 1987) .g. one might want to find additional evidence Observingagents for the existanceof goals. ( 1984) argue First . Their behaviorcould be describedby attributing to . i . but that is highly another. behaviorin naturalagentsand perhapsthereneverwill be they are intentional-level concepts( Dennett. there is no experimentalprocedure. 1991) . They (McFarland. However.e. i . cannot be concluded that the agent's behavior is goal. starts talking Studies in animal navigation (Orr. Since goals are entirely observer. as a goal-directed PhD. drinking the coffee) as . with the directed. obviousdifferencethat we cannotask animalsabout their Second. the classical navigation problem: how does an confirming evidence for our hypothesis.Goal . about of goal-directednesshas high value (see. 1983) show that sometimes knowing whetherthe personactually ever had the goal to animals can travel long distancesfrom one location to go there. If we assumethat the agents behavior were goal-directed. being in good spirits for the time. we need inter-level constraints ( in distinction betweenthe atttibutions madeby an observer conb"ast to intralevel ones) . Let us begin by discussingobservation . We can concludethat.g.. etc. 1978. talking exhaustivesearchwould makeit impossibleto ~ t in real to someonein the cafeteria. from a scientific point of view the baseddescriptionsno conclusionscan be drawn about the notion of goal-directednessis not well-founded and it is processes within the agent which lead to behaviorsthat a better Sb"ategyto explain behavior without referenceto the observer describes in these tenDs.e. passing the exam. to communicate with Schankand Abelson. getting a on the basesof these principles (i. it already arbittariness in attributing goals. the goals attributed to an agent can also allows the system to reduce the difference to the goal be at very different levels. the basisof its (his or her) goals. Of course. We must make a clear description. In particular it the notion of goal-directedness. i. Therefore.

.and are . always the result of the dynamics of internal However. Designing an agent means translating a task " . . as just discussedgoal-directedexplanations physiological variableswhich through the physical setup are highly problematic and arbitrary. If the fish had an explicit goal and evenmore that thereis somekind of global map. to employ a completely different terminology. : ' " . What are the implications of the Figure 2: The classicalnavigationproblem: die agent previous considerationsfor designing agents? What we (in our example. e. . Its task is haveshownso far is that behaviorwhich can be described to go to location B. namely the one of dynamical systems which is appropriate for the dynamics to be kx : ato accountedfor. The notion of goals is observer-based. " . If their system-environment interaction. 1970) this leadsto agent and its environment. keeping the Bahnhofsttassein ZUrich . ' . By a local mechanism Clancey. representationand a map of the environment that they This examplecan be used to illusttate a point which could use for guiding their actions towardsthe goal they applies generally to autonomous agents. ~ .g. . Behavior is could be consideredgoal-directedsystems . It would be better. b : at ~ n neutral with respect to the agent: it is defined by an outside observer. However. . .g. ' . without the need to resort to global goal-directed mechanisms . Verschure themthe goal of being at the place wherethey camefrom we mean one which is baseddirectly on the immediate (location B) and viewing them as goal-directed. . J. This mechanismcould be basedon words. . M. .g. the fish there is a comparativelysimple explanation: the Although goals havebeen. equatinggoal explained on the basis of purely local mechanisms ascriptions with internal mechanisms to explain goal- oriented behavior would be a category mistake (e. reached the goal. the behavior of the fish can be stticly internal to an agent. . at least once they are back in fresh to internal mechanisms. Therefore.Hara. The shaded areas represent by using the idea of goals doesnot necessarilyimply that obstacles. die fish) is at location A. . the system is goal-directed. Consequently . as argued earlier. ~ . : - ~ X . into a physical set-up and into mechanismsthat will accomplish the task. 1978) and in combination with a under investigation. : description. . 1989) . the behaviorof the fish is not goal-directed: one somekind of a "map" of their environmentthat they can would be hard-pressedto argue that there is somesort of " " use for guidance and for ' testing whether they have goal respresentation(obeying the representationlaw ). whenever talking about internal mechanisms. In other choosingan action. and is " " shut off again when a physiological or sensory state representationlaw ) and on the other a mechanismby which this goal represen~ tion becomes effective in correspondingto being at location B is detected. this goal-directed the observedbehavior. clean. Therefore we of the agent ( sensors. . pertainsto a continuousinteraction betweenan direction of the water flow . . In the require an explicit representationof the environmental exampleof the fish a behavioralprogram is biggered by situation of location B in order for the fish to test some physiological state (e. ' e : : .often usedin AI fish seem to be following a concentrationgradient of to direct the behavior of an agent they are intentional- some chemicals. And indeed in the case of environment. Designingagents ~ . . Salmon appear to be following a level constructswhich do not pertain in any direct sense chemical gradient. . " . we have not yet . Rolf Pfeifer and Paul F. (following a gradient) . As illustrated the behavior water (Haseer. t > < ~ ~ x ~ ~ . . . t . that we might want to call goal- rheotactic response (a tendency to orient against the directed. an essential variable whether a particular location correspondsto location B approachingits boundery) which leads to the gradient- (and this representation would have to obey the following behaviorwhenevera gradientis present. Note that the term " task" is " . : . " . . ~ : . : I . Mechanismsinvolving behavior were goal-directed this would on one hand world models are global. . . description cannot be reduced to processes which are In other words. actuators) interact with the should look for alternatives.

1992. this Volume).goal . if a collision to the left is registered. agent we will consider is developed according to the Third . and problemsmentionedabove: the fish in our exampleare flexible and they do not suffer from combinatorial properties of the sensorsand the motor apparatus. reverse. The central the history of AI systems interacting with the real ingredientis a so-called value scheme(or value system) world.and on the other we would still have short since it has been described elsewhere (see also to work out the plans (and the pertinent aspectsof the Verschureand Pfeifer.g. Moreover. and this is a pragmaticargument. world model) implying the problems just mentioned.resort Traditional models have been conceived as goal- to the notion of plans. But there is little reason to In this section we will show how a robot can be believe that this domain ontology can serve as a sound designedto perform a navigation task without having to base for an adaptive system becausethe real world is resort to the notions of goal. The fact example. there are which encodesthe agent's essential variables (e. 1986) ..g. Fikes and Nilson. But on one hand learn to avoid obstaclesand locate food sourcesin an this list is highly arbitrary .one way or other . 1992) . its are highly adaptive. or world model. 1971.tend to be less 3. The description of the approach will be of compiling it .directed robots to adaptive. Fifth . Second.g . goal-directed systems as explainedearlier. plan.Goal . For the cleaning task one might be able to (Pfeifer & Verschure. reverse and turn right ) . the task descriptionhas to be ttanslatedinto goal design methodology of distributed adaptive control structures.. significant limitations and that it neither seemsplausible from a design perspective. directed systemswhere all behavior was generatedby Sucha world model is hard to ob~ n (becausethe world explicit goal representationsand associated plans or is only partially knowable) and difficult to maintain world models. We plan based approaches quickly run into combinatorial from a biological nor problems(e. Designing non . Bahnhofstrasseclean . The constantly changing and only partially predictable. parameter settings. Non . 1987) want to present an alternative that demonstratesthat which make it hard for an agent to act in real time. range finder. what self -organization (e. maintain a certain energy or temperature level). In our then is the right way of modeling its behavior? How . need to . Fourth. Verschureet al.) . Edelman. if we usegoalsand plans in our agent(which are The basic idea of DAC is as follows. The agent is a symbolic represetations) we have to establish the simple mobile robot which is equippedwith a numberof connections of these symbols to the outside world . the geneticsetupwhich constrainor direct the processof But if our agent should not be goal-directed. In other words.g. and turn.Directed Robots demonsttatedthat when designinga system. Plans require a world model. for inappropriateto take a goal-directedapproach. etc. achievethe navigationtask of going from A to that the behavior of animals is presumablynot directed B1 It seemsvery hard not to think of goals which is by explicit goal representationsdoes not necessarily probably the reasonwhy goals are ubiquitous in the AI imply that artifacts should not be goal-directed. powerful agents with high plasticity can be developed which are not goal-directed. First . to efficient simpler alternativeswhich work well in nature. Since the knowledge of the system which is perform tasks expressed in a world model must be defined in advance the designer has to transfer his of her own domain Distributed adaptivecontrol (DAC). systems which are based on static designerdefinedontologies . sensors (collision detector. Chapman . target detector) which has been a notoriously hard problem throughout and can move forward. literature. We come up with a certain list ( "broom in hand" . it would be then would we design the agent such that it could. systems might not be the best way of designing Conclusionfor autonomousagentdesign autonomous agents. We showed that this approach has (becausethe world is continuouslychanging). the value schemeencodesthose aspectsof problemsin action selection.there is no principled way environment.and such ontologies are needed when world modelling is involved . Clearly there is no definite answer yet but There are severalreasonswhy building goal-directed we will discusssomepertinent ideasbelow. ontology to the system. and obviously do not have the reflex es (e. "bucket " " " will base our example on a simulated agent that has to empty .

it starts anticipating obstacleswhich Actions: The agent can perform 5 actions which are enablesit to turn away before it is hitting (the result of "reverse. . This is an exampleof the way in which the . Whenever region as the range finder. collision detectort and target sensor. J. architecture. They are RF and C. value scheme. the dashed lines are modifiable through Hebbianlearning. The two target detectorscan such a reflex is uiggered learning takes place between eachcalculatetheir own distanceto the target. consisting of two units. If the target is detectedto the left of the system a "turn-left " action will be executed and symmetrically for turning right . Also here there is a one to one it to integrate more sophisticatedsensors(e. a range mappingbetweenunits and elementsof the sensor. obstacles which will trigger the basic reflex es of "reverse-turn-left " and "reverse-turn. The range finder . . Activation in C will increase its Figure 3: Schematic representation of the control architecture. M. When the activation of I is above a certain detector group. The output group. derived from a model of classical which also consistsof 37 units. The latter action will be executedwhen noneof the othersis active. The first group . C: collision activation. If one tnc8 finder) with the basic reflexes (seeFigure 3). As a result. " " " learning even after it is no longer moving into obstacles. The last group (M ) consists of 5 commandneuronswhich codethe basic~ tions the system can perform. I : inhibitory unit. In addition there is a special unit I that regulates the relation between C and T. Activation ai of eachunit is determined action:'based categories(see also Verschureand Pfeifer. T: target detector group. Rolf Pfeifer and Paul F.g. receive their input from conditioning ( Verschure& Coolen. roughly by the inversevalue of a correspondingunit ri in Dr detct dor this volume) . The agent keeps " Turn. placed at -00 and 90 degreesfrom the center of the value schemedirects the learning process. it encodes the basic value "avoidancehas over " preference approach which prevents Sensors: The agent has three sensors: range findert the agentfrom being damaged . The units in the second group (C). The third group (T). Ve. In a sense. The calculation of the difference in activation after a while the agentwill avoid the obstaclesbecauseit betweenthe two target sensorsyields whetherthe target has learned to use range finder input to control its is to the left or right from its center. This is important since there may be changes in the environmentto which the agenthas to adapt Control : The conttol architecture consists of four neural groups and one inhibitory unit. Experiments spansover -90 and 00 from the center of the robot. The receptive fields of these elements Figure 4. RF: range finder group. receives its input from the target sensors. collision detector is biggered the corresponding unit rarge will get activation value 1. The agent has a neural network-based the range finder. One unit in C or T can bigger a unit in M."schure rcollsb E Wont mob OUt architecture the agent learns whenever an action is ( RF) which consistsof 37 units receivesits input from triggered. The connectionsbetween T . actions. " The systemhas 37 collision detectorsthat cover the same right . In essence . 1991) which enables the collision detectors.turn- Ieft " t " reverse-turn -right " t " turn-Ieft " t this processcan be seenin Figure 4) . robot. C and M are prewired (the solid lines in Figure 3) . It consists of 37 elements that each cover a part of this The environment for our experimentsis depicted in receptive field. As the agent movesaround it will initially hit decreasewhen they get closer to the centerof the agent. The solid lines between inhibitory elementrepresentsan important aspectof the the groups representprewired connections. right t and advance . A collision to the left will " " automatically bigger a reverse-turn-right action and symmetrically for a collision to the right. M: motor thresholdthe output from T to M will be inhibited. This implies that the agent will acquire the range finder.

r " towards a goal. say to go from location A to whole in the wall (the Figure has been adopted from pfeifer and Verschure. Non . through Hebbian learning. For certain behaviorsto emergethere must be certain environmental conditions to activate the pertinent part of the valu~ system. In other words.g. namely to seek targets. behind the 110lein the wall (indicatedby the small circle It is interesting to note that wall -following only on the right) . the target on the right The range of the target sourceis indicatedby the larger Solving the navigation task. In this environment the agent in the wall there is a hole it will turn into it . a goal. important that the target source is located behind a i .orientedbehavioremergesthrough self -organization and is entirely based on local " ra mechanisms . From this description the robot will find the location efficiently . Since we are the designersof the system we have. e. If we take this to be our target location associatedwith the target group T. But then the approachbehavior will be randomsearchif there is this particular regularity in the triggered again. Becausethe target is behind a hole in the wall released anywhere in the environment (let's call this this also meansturning towards the wall. 1992). If the agent moves around. without resorting to some notion of goals. until it gets near a wall. plans or models. if the agent is reflexes. The agent is clearly not goal-directed: its goal. The shaded areas represent obstacles. To an " acquired a behavior that one might want to call wall. After having will turn towards it because of one of its approach acquired this wall -following behavior. high activation A (which is the agent's starting location) to location B on one side of the range finder RF) the action of turning which is defined by the designer. to achieve a task. it will not develop " wall -following " . etc. The. . This can be viewed after some time start following walls and if somewhere as following a gradient. until it reaches the target. wheneverthere is a wall (i. If it is releasedin an environmentin How does this wall-following behavior come about? which there are target sourcesbehind holes in walls it If the agent is within the range of the target source it will eventually learn to follow walls. The agent is shown on the left. It will then follow range finder RF becausethere is a wall. Becauseavoidance the shortestone the systemcould take (from the point of has priority over approachthe agent will then turn away view of the observer) but it will be much better than from the wall. of course. path will not be towards the wall will be triggered. the systemis not working . insight into its internal mechanisms. a target was put what is "good" for the agent. This acquired behavior to do is to place a target behind the hole in the wall can now be exploited to solve the navigation task where he or she wants the robot to go (location B) and depicted in Figure 1 (see below) .g.Directed Robots In the experiment which we performed to organization only works if it is directed essentially by demonsttatenon-goal-directed designs. At the same location A ) it will move straight while avoiding time there will be high activation on one side of the obstacles. be hole in the wall. to be defined by the designer is a value schemethat . What needs the importanceof the value schemecan be seen: self. Let us go back to the circle. outsideobserverthis looks as if the agent were pursuing " following (Figure 4) .Goal . If the agent is placedin an environment in which thereare no target sourcesbehindholes in walls rc = . We can therefore say that the system is by design not goal- Figure 4: Environment for the simulation experiments. directed. It is problem of how the agent can be told to do something. location B.e. it will which caseit will move towards it. It is important to note that the task accountsfor the wiggly wil of the agent which can be descriptionis entirely designerbased: all the designerhas " " interpretedas wall -following . This environment.e. turning towards a target). the reflexes (e. This implies that B we do indeedhave a robot that will go from location over time. The agentcan detectthe targetonce it is in develops if there is this particular regularity of targets its vicinity (the region boundedby the larger circle) in in the environment. This pattern in the wall until there is indeed a target source behind a the range finder will .

In the traditonal approach this may be one of the reasons why goal-directeddesignshave definition is in terms of coordinates or a position on a - " " dominated and still dominate . Summary and conclusion real world they will not do . The goal .up approach could be that inappropriateif we want to understandtheir behavior. using the example of a on certain kinds of gradients which will " draw " the simple navigationtask. and emergenceis more difficult than thinking in termsof goals and plans. This might be an appropriate design strategy in highly engineered environments but for the 4. Artificial Intelligence . It is also cheap because it relies upon the for the agent. Another one which is a prerequisitefor the exploit regularities which have been associated earlier wall -following is the inhibitory element which gives with these gradients (when they were still there) . This can be accomplished 34119. by programming a map ( a models within the agent local mechanisms are sufficient world model) into the agent and use this map to generate to exploit them. However . issues. .g. value schemes . navigation problems does therefore not have to be represented in the form of can be solved . Verschure enables the agent to establish the interation with the This non . Of course. (1952 ). The programming difficulty in adopting this attitude target is defined . ( 1987 .directedway. is moving towards the boundary of its viability zone ( the pertinent simulations are not shown References in this paper) . for instance. We thank Rene te Boekhorst for many reflex es are only triggered when an internal variable .92 of the Swiss National ScienceFoundation to by designing the agent such that its target related Rolf Pfeifer. the wall following " behavior is useful for efficiently fmding food (always provided that the environment does have the regularitiy that food is behind holes in walls ) . but only ways to generate sequences of propenies that are already present in the environment and actions to find targets. the fish . searchingcan be acquired. 32. The from the system-environmentinteraction. genetically prespecified . Especially when talking about animals it Ashby.autonomousagent map. Acknowledgments then this behavior should only be triggered if the agent This researchwas partly sponsoredby grant # 21- has a need for dtis type of food. the valuablecommentsand discussions. J. It requiresa particular attitude towards One of the distinctions between the traditional " programmingwhich Rod Brooks called the Zen of robot approach and the one presented here is the way in which a " . Chapman & is obvious that the location of the food sources cannot be Hall. We havearguedthat viewing agentsas goal-directedis A criticism of this botto ~ . Rolf Pfeifer and Paul F. for ) Planning conjunctivegoals. In our preference to avoidance over approach. . R. there must be mechanisms Baker.oriented behavior of such agents. if no such value schemeare the reflexes which determinewhen the gradients are currently there the system should be able to agent learns. If we assume that the target represents food then " design. The approachis basedon two key of the fish ) . Chapman D. Theevolutionary ecologyof animal by which efficient navigation strategies for food migration . instead of specifying trajectories in an internal world We also showedthat goal-baseddesignsare fraught with model this approach places them in the world by relying problems and demonstrated. a sequenceof actions. Designfor a brain. as argued earlier . the gradient . Important aspectsof the these gradients in the environment . 333-337. M. association of the gradient with the wall leads to goal - oriented behavior which is independent of the presence of Thinking in termsof self-organization. self-organizationand a value schemeto direct the e.goal -directed design is robust since it can environment in such a way that the desired behavior can adapt to a variety of environments and to unanticipated emerge: This implies that target locations are not defined situations . Goal oriented example the gradient is defined by the field which behavior of the agent (which to an observer may look extends from the target source behind the hole in the goal-directed) is not designedinto the agentbut emerges wall . how agentscan be designedin a agents to the target from any position ( see our example non-goal. London : Hodder& Stoughton . and the regularity is given by the wall . Thus . W. ( 1973 ) . energy supply . is therefore constrained by the presence of process of self-organization.R.R.

New York: AcademicPress. Pfeifer. reinterpreted. &. P.C. Physical symbol systems . Clore. Pfeifer. . Special Architecturesfor intelligence.J. 19. GPS. & Nilsson. Newell. TowardA Practice of AutonomousSystems : Proceedingsof the First EuropeanConferenceon Artificial Life . & Coolen.J.).. no-goals and own goals. ( 1990) . Hillsdale. Maes. ( 1971). A. no- goals and own goals. 49-70.C. Chemoreception . G.J. In K. Roboticsand Autonomous 3&4). The frame of referenceproblemin Pfeifer. ( 1980) .P. Beyond the designof intelligent machines . Mass. Edelman.E. R.A. A. ( 1977) . TheJournal of Mind and BehaviorS . In E. of Connectionism . R. 9. Iran-Nejad. In A. Randall: Fish Physiology. P. New York: Basic Books. Fikes. Cambridge. & Greenwood.M. 189-206. a program that simulateshumanthought. D. ( 1987).120.T. Proc. 231-259. P. ( 1992). N. Newell. ACM .F. 135. Goal . 279-310. . Schank. and Simon. and understanding . and Simon. HilssdaleNJ . H. Lehn (ed. 6.J. Cognitive Science. N. ( 1989).A. Directed Robots Clancey.R. Goals. 21-30.J. Hoar & DJ . Situatedagentscan havegoals. Goals. Machine Learning. Cambridge. A debateon goal-directedand intentional behavior. 84. R. 357-423. Hara..MJ . W. ( 1990). R. ( 1971) . ( 1983).196. Artificial Intelligence. A. Orr. ( 1963) .. A. Newell. ( 1992).J. Feigenbaum and J. of animal movement . Nisbett. R. R. ( 1970).A. In: W.M. Affect: a functional perspective. 22nd Carnegie Issueof ConnectionScienceon Philosophicall ssues Symposiumon Cognition.F. Telling more than we can know: Verbal reportson mentaldata. rationalism: symbols. BJ . Oxford: ClarendonPress. in migration. R. Cambridge. Non . Comm.L . patterns. M. & Verschure. G.. ( 1976). (in press). & Vondruska. sttucturedbehavior. R. Network: . (no. London: McMillan . KrOse. Mass. 4. ( 1987) .J. A. & Verschure. The intentional stance.. 39-57. Disttibuted adaptiveconttol: a paradigmfor designing autonomousagents.C. 1971. 2. J. Newell. A. McFarland. 4.183..) .S. Robotics and AutonomousSystems . plans. Unified theories of cognition. New York: McGraw-Hill . STRIPS: A new Verschure.. Systems . ( 1984). Volume V Sensory systemsand electric organs. 113.. Computersand Thought. Noble (eds. 285-291.: Harvard University Press. approachto the applicationof theoremproving to Disuibuted adaptiveconn-ol: The self-organizationof problem solving. Clancey.: MIT -Press.: Erlbaum. Montefiore and D. & Abelson. of neuronalgroup selection. 19.126. London: Unwin Hyman. Mass.F. Animals .MJ . Feldman(eds.v. The ecology Dennett. Neural Darwinism: The theory Adaptive fields: disuibuted representationsof classicallyconditionedassociations . Scripts.F. P.) . 2. and Wilson ( 1977). ( 1989) : The knowledgelevel goals. H. Singland.: Erlbaum... 181. Psychological Review. Pl . T. ( 1992) .. ( 1991) . D.C.: MIT Press. and behavior. A. Computerscience as empiricalenquiry: symbolsand search. P. WJ . Verschure.


or detourarounddIe were successfulin about 50% of the trials. Rana computabix (Arbib . The stationary possess a quite capablevisualperceptionability that detects barriersignaland prey signalhaveto be integrated sophisticated spatialarrangements of two fences. Overview of Anuran Detour Behavior Anurans (frogs and ~ s) show a quite flexible ' Fig.l . barrier. This paperbeginswidl an overviewof directedto thegapin thefront fence. A filled as in Fig. or remained inactive.lib .lib . As thetwo constittJent anurandetour behavior. dleoreticmodelof die integrationthatbuildson die New data: Collett's datamakeone think that the frog/ toad Arbib andCobas( 1991).S.hblee@rana . most of the toad' s approaches were brief comparisonbetween anuran behavior and detours (94%) . after poking at one or two stationary object (paling fence barrier in particular) apertures . The paper concludeswidl aschema. die animal does the animal s to a barrieredgewheretheyreorientandapproacha prey. it is the worm that triggersthe animal' s .A . Whenthe gapin the front fencewas stationaryobject recognitionand localization. which wasalsoadoptedby Arkin locatedin die directionof escape . Collett' s data further show that toads robot navigation is presented. In trajectory to the worm changesin a way that reflects a dteremaining20%. anda to generateappropriatemotorcommandsfor detour prey. the Arbib and House ( 1987) offered a model of this using a animalsoften jumped into the barrier even though it was " " potentialfield approach . diey Our aim in this paper is to trace and model those usually showed detours when the prey-barrier-frog processes required in such detour behavior. For another barrier. its surroundings . our pilot observations suggestthatit maybe moreflexible thanwe thoughtearlier. Then. we seekto unravelthe mechanisms behinddie an initial direct approach. thebarrierfrom a moredistant. Abstract 1. Even whenthe frogs made specifically. frogsalways Ingle ( 1983) andCollett ( 1982) haveobserveddlat a frog or approached the worm directly . responsebut when the barrier is present. Thus.depthperception . They often tilted their bodieswhen passingthrough the apertureand approachdirectlyandsnapat theworm. dteyeithermovedawayfrom thebarrier relative spatialconfigurationof the worm and the barrier. However. if no worm is present. they tend to initiate detour behavior. A frog or toad. after dte failed squeeze not move. As shownin Fig. they tried to squeezetheir way dtroughan aperturebetweenthe surround.c. arbib@lX>llux. threat. integrationof sensorysignals. Arbib and Hyun Bong Lee Centerfor Neural Engineering University of SouthernCalifornia Los Angeles. To modela complexbehaviorsuch a passable gap (henceforth called " internal gap" ) is asdetourarounda stationarybarrieron die way to introduced. ignoring the barrier toad's approachto prey or avoidanceof a threat are also determined by the stationary objects in the animal's irrespectiveof prey.1. ( 1989) to build a mobile robot that can navigatearound Theanimalsbeganto show obstaCles on its way to a destination.a. which also includesnew fencesare broughtclosertogether. Whenprey is at 12cmbehindthe barrier. constructedfrom singlefencesFig. Introduction Initially . may eidler neighboringpalings (inter-paling aperture). dtey sidesteplaterallyusually ' pokingtheir noseto someinter-paling apertureson dte way. if a barrierwith coordination. meaningthat a toad can acquireglobal knowledgeof behavior. 1989) is an the toad shows75% detoursand 25% direct approach es as evolving computermodel of anuranvisuomotor illustratedin Fig. waspresentedas in Fig. or animallocation. the toad detoursaround datafrom our lab. globalperspective .gaps. It was as if the animal was trying to reexamine perceptioninto appropriatemotorcommands . In some recognition.l . the toad showeda appropriatemotor patterngenerationmechanisms . . is a simple automaton. depth perception. the animal usually aims to the gap (90%) . The aim of dlls paperis to ttace and model such with detourson about50% of the trials and the odter 50% processes. when the frogs were fIrst introduced to the experimentalarenafrom a closedconfinement . CA 90089-2520.lid .usc. More configurationwas appropriate . . followed by a modelof anuran thebarriermoreoften. AnoraD Visuomotor Coordination for Detour Behavior : From Retina to Motor Schemas Michael A .a barrier recognition . responsethat was nearly an averageof the two responses . prey requiresan understanding of anuranprey and Whena doublefence. theybackstepabout3-Scmandtheninitiatedetour and the mechanismthat transformsthe result of sensory behavior. About a month later. instances .l . Recently. viewing a vertical paling fence barrier dirough which it can see a worm.l showsthe resultsof severalof Collett s experiments behaviorwhen confrontedwidl stationaryobjects where he presentedtoads with various prey-with-barrier on dleir way to a prey or when escapingfrom a configurations . When they were frightened. detourbehaviorafter a week in the experimentalarena. u~ . U. and andFig.

.l .l . This lumping effect can explain why toad approach es the gap lessas the . Aldlough more experiments are needed. The resultantsumof all forces determinesthe subsequentdirection and speedof travel. obstaclesexert . Using the analogy from toad. The toad approaches made in a particular direction . When the prey is present. . . Single fence with 6cm gap and with prey effectively blocks its approachto prey. elementsin its workspace . as it encountersnew they were fIrSt introduced to the place: They did not detour.. But after some failures .. when frogs were relocated to the confined aquarium. . 1982) .... fence . Gradually diey began to exhibit detour behavior after three Based on the potential field method. 1991). . ( Koren& Borenstein . When progresses.a andFig. b. . This behavior 22cm behind. Frog vs.i ~ . . of the potential field method. Perhapsbecausedie palings resemble suggestingthegap s attractantforceshouldbe lesspowerful than the repulsiveforces of the barriers. . Associatedwith a toaditself is a radiallydivergingfield dlat ~ ~ represents theanimal's " urge" to move. Single fence with usuallyapproach es the gap eventhoughthe rear fencestill prey 12cm behind. Anuran Visuomotor Coo Iodination for Detour Behavior ~ c 1.. widl themethod. . Warren( 1990) useda similar &5 % techniquefor coordinatingthe pathsof multiple robots in ~ ~ ~ 7 die presenceof obstacles . In each case.l : Approach es to prey with single and double barriers " " interposed .. The barrierfield is not radially symmetric but hasa lateralcomponentthat is strongerbut decaysmore rapidly with distanceDiandoesits rostro-caudalcomponent .. 1982). Cage is formed from double fence with similar to prey. . . Tilove ( 1990) offers an overview d6~~ . . Robot 25 % ~ ~ In die mobileroboticscircle.. inexperienced frogs initially may opt to approach directly to the prey . .lid wherethe approach es are mostly detours when the gap in the front barrier is filled. a toad can attacha new meaningto the After about two mondls stay in the experimental arena. Note that the When frightened . . Approach es to prey die gapitself is treatedasa targetexertingan attractantfield behind the " cage" . 8. Here. from direct approach to detour behavior as die experiment two constituentfencesare broughtcloser together. they seldom jumped into the barrier barriers/gapsthemselvesdo not elicit any movementsfrom even though its location was in the typical escapedirection. I I Arkin ( 1989) used the method for his mobile robot' s. frogs using the potential field method could be stuck in local minima as when it is trappedinside a Ushaped obstacle may gradually learn that the palings could not be toppled over so they shift to detour instead. . .. Arbib & House ( 1987) presenteda model of anuran detourbehaviorbasedon thepotentialfield method... the hammer indicates the toad s ' was able to explain detour behavior when the barrier starting position and orientation .l . while die targetappliesan atttactiveforceto the robot.e. A barriersetsup a repellentfield whoseeffectis morelocalized .c..g.. . A mobile robot plants and stems. The model Fig . . . Dianthatof prey. . the gap. ' obstaclesto the frogs. repulsiveforcesonto the robot.. . Double fence composed from single fences of cannotbe explainedusingthe potentialfield methodunless 8 and b . diem are pliant . Anodierclue that the maladaptivebehavior palings connecting the ends. Whenprey. in Fig. (From Collett . and Koren & Borenstein ( 1991) presentinherentproblems/shortcomingsassociated . the filled circles show the configurationwassimpleas in Fig. . it will be frogs were put back to the experimental arena after two interestingto think of a robot that can not only changethe weeks stay in the aquarium. . . Vertical plants and stems constitute many of natural stationary objects to the frogs but becausemany of the distancebetweentwo barriersbecomesless than 5cm. the animals actedjust like when shapeof potential field but also a meaningof an object. e. I . e . Thus. path planningmodel.lib . . c. . d. . . neighboring experimental sessions... .ciscaused by the toad' s responseto the gap is shownin Fig. . Double fence with no gap. a prey sets up a radially symmetric attractantfield whose strengthdecaysgradually with distancefrom itself. we believe some potentialfields of front andrearbarriersgraduallylump into form of learning is involved in the animal 's gradual shift one as the distancebetweenthem shrinks.. from an attractorto a repellor. and the arrows summarize the directions of the Let us analyzethe toaddetourbehaviorwhenthe barrier approaches. they may not be perceived as impassable approach es to the gapfall to lessthan 30% (Collett. barrier. ~ 10an AuRA. The results are given in terms of the percentage of configuration is more complex as in Fig. and toad are presentsimultaneously .2 . Similar things happento the toadas shownin Fig. die potentialfield mediadfor path planning has gained popularity in part due to its simplicity and elegance . In this approach. the interactionof die associatedfields determinesthe animal's path.. from a neutralobject to an intermediatetarget. . e.: t ~ . the animal. '\ .

Liaw & Arbibt alsoinfluencesthe toadbehavior. 11110 toadscould havea stereoscopiccue to estimatedie depth. selectthe mostsalient iii ) If thereis Amongthe retinal ganglioncells. His observationalso suggests repulsive force of the frontal barrier due to its weaker that not only the pattern but also the size and the depth intensity. featuresfor SO recognition Neural network models for the above cells have been shouldbe basedon 2-D firing pattern on die Thl0 layer. we will presentmodelsof how OFF light stimulationbut do not dischargeto stationary theanmansmayrecognizeme barrier/gapandits depth.r. Stationary Object Recognition Model interactiveor independent esfurt11er process visualperception is accomplished . we find it difficult to explainthe aforementioned relationshipw. ClassR4 neurons: With a largeERF of 12. recognitionalgorithms to neuronalrepresentationsof the however. 1989).r . stationaryobjects(henceforthcalled revealed by turning on the room light without prior SO).+prey axis. by pretectum . neuronsexhibit prolongeddischargeto largedark stimuli movingstimuli (preyor predator) at a neuronallevel. Arbib and Hyon Bong Lee Collett alsoreportsdIat an internalgap in me rear fence pretectum(Cervantes -Perez et alit 1985.+prerectalprojection. hasa binocularERF. front fe:nce has no gap and the inter-fencedistanceis not Ingle ( 1983) showedthat a main locus for anuranSO small. width. somesubtleproblemsstill remain.90 .g. while muchof die recognitionof theobjecttespeciallyduringfigure-ground frog behavior can be replicated basedon potential field segmentation : A smallerbarrier hasa better chanceto get method. no suitablegap. Th7 1 to six classesdependingon their characteristicresponsesto Among major ( ) responds any .a and Fig. pattern on the prerectalsurfacedue to die topographical 1987) and Th6 ( Ewert.rit. how these could be integratedwith prey information to . me gapin merearbarriercannotcancelme recognitionis the pretectum. Certaincellsin tecttlmandpretecttlmhave A layer (surface) of prerectal Th 10 neuronswill be the beenassociated asgrandmothercells signalingthe detection substrateof SO recognition.16 .1. whena dark object is introducedto their ERF and stays there.b. Because . brief descriptionof anuranpaling fencerecognitionwill be For instance . reductionof the generalillumination with a prolonged different stimuli and light conditions(GrOsser& GrUsser tonic dischargelasting5 minutesor longer. these modelanuranperceptionand behavioralmechanismw. detectedthana largeronetandthe samehorizontallystriped How doesmetoad' s " intelligence " comparewith matof a' patternis intelpreteddifferentlywhenits depdlis identicalto potential field based mobile robot? Obviously. frogsand . Comehls. Michael A . So. rather thatare stationaryin their ERFsevenwhenthe SOs were little work is donew. sometimesevenwhenme 1991). of manydifferentclassesof neuronslocatedin differentbrain . Retinalsignalsconvergeonto tectumand pretectum in anuran midbrain region and through their 22 . tectumand Sinceonesubclass (4).1. whereasthere are mobile We do not diink thattherearedifferentpretectalneurons robots capableof doing that"(e. only R3 andR4 cells are positively identified to project to pretectum( Ewert. to theanimalmanthatof therearfence. Warren.c is theaverage offered . Anuranvisual perceptionis the result of cooperation motion. of unit responsesin Fig. the responseof a frog in Fig. thesecells generateappropriateactionsbasedon the physiologyand exhibit continuingdischargewhenlights areturnedoff or anatomyof theanimal. the backgroundarethe importantcuesfor behaviorwith the potentialfield theory. Unit responseis basedon a "unit" formation of SOs in pretectumt an important part to rule which is applicable only to a specific prey-barrier understand the recognition mechanismt and a depth configurationconsistingof prey andan individualfencebut perceptionprocessthat dependson visual featuresextracted not to me integrativefeaturesof a doublefence.t. units that give and Depth Perception continueddischargein thepresence of a largedarkstationary While considerable effortshavebeentakento analyzeandto object. Toad behavior. Model of Stationary Object Recognition Ewert ( 1971) found in toad's pretectum. regions. Class ThlO neurons: With ERF of about 30 . 1976).1. Many sensitiveto differentSOStand thereare no datasuggesting robots capable of global path planning employ search suchneuronsexist (seenext section). . Cells involved in SO Recognition barrier(gapin mis casealso includesthe two barrierends). This section presentsa neural processof pattern sumof " unit" responses . Basedon thepatternsformedin pretectumta eachunit rule is like a reflexivestimulus-reactionparadigm. Thesecells showbrief phasicresponseto diffuseON and In thefollowing chapters . toads the background(a place to hide) comparedto when it is behaviorsuggeststhat they are bad at formulatinga global locatedin front of the background(a placeto avoid) during path plan which requires them to integrate the spatial die escape behavior. Anuranretinacellshavebeenclassifiedinto five or 3 subclass es. 1976). and . . aim for the prey. dependson its location w. seemsto be basedon averageresponseto a linear SO. Rathertwe posit that methodsto find the best path from the initial stateto the the recognition should be based on application of goal stateamongmany alternativepatits.1. ClassTh7neurons:Theseneuronsareluminancedetectors . The unit rules may be summarizedas follows: i ) Aim for a gap in the 2 . ii ) If thereis a choice. In away . 1971) as ~ looming mreatdetector. A gapin the frontal fencemay havemoreinfluence be simply describedas detectorsof moving contrast. 1971. retina. information which they have. The saliencyof a gap Griisser& Griisser-Cornehls. thesecells can depm.rit toad. ClassR3 neurons: WidJERF of about8 . Since a SO forlDs a 2-D of certainvisual stimuli: T5-2 as a worm detector( Ewert.t. and stimuli. 2. proposedusing appropriatecells from retina.

Thust the Thl0 model is The pretectalThIO basedSO recognitionmodelshould satisfy the following criteria: i ) A simulatedThIO cell shouldreplicatethe responseof a real cell. (Exexcitatory connection. ii ) A SOpattern on Th10 surfaceshouldoffer neuronalcuesfor identification. surround-off Turning me lights on againelicited a short burst. Fig. Th7( 1) ' respiratory eye movement. The Thl0 showsthat the simulatedThl0 neuronrespondssimilar to a cells are modeledto receive direct excitatory R3 and R4 realneuron.L~:::= ::::::: ~~ A ~ - largereceptivefield andits increasing response to largedark objects. b. The frequency increased when a dark theR3 receptivefield shouldbe eithervery narrow(lessthan 3 ) or could be somewhatbroader(up to about 10 ) if the object moved into me receptivefield and remainedstationaryin dte field (SO). This suggeststhat the Thl0 representationof to be theentireview field of theanimal. Thecell is madeto horizontallysnipedbarrierandsolid darkpatchshouldshow generateprolonged tonic discharge so long as the R4 similarity while that of the vertically stripedbarrier should summationacrossits receptive field remains above the be different.j11 .lUl ~fFlll ~~.50 every 0. Convergence of R3 explains so ThlO' s On-Off responseto light changes . we modeledThIO s R4 receptivefield to be large(45 ) to accountfor 11110 's . firings. ( 1971) reportthatR3 neuronsareactivatedby eachactiveheadand bodymovement BurghagenandEwertts( 1983) observation also suggestsimilar conclusions. by renewedresponseto me SO. Simulationshowsthat Fig. 1911). retinal inputs and inhibitory inputs from Th7( 1) neurons. (from Ewert. Due to R4ts large ERF and aforementioned specifiedthreshold .2: Schematicdiagram illusb' ating the connection patterns when small shifts in the retinal image occur during of ThlO network.3. receptive field is modeled as a center-on. R3 is sensitiveto edgesand has betterresolution(dueto smallerERF andfocusedprojection to Thl0 ) but how could a stationaryobject elicit retinal Ex imagemovementto generateR3 activation? Anuranshavewaysto generateretinalimagemovements throughdiverseself inducedmotions. Given theseconflicting datatwe set out to SOOt through retinal modeling (Teeters et al. Howevert Pigarevt and receivesexcitatory input from entire R4 view field and inhibits Burghagen& Ewert failed to seethe respirationinducedR3 ThlO. In: inhibitory connection). GrUsserand GrUsser - Comehls ( 1976) elicited R2 and R3 responsesfrom incompletely immobilized frogs and toads during eye- retraction/lid -closure. Simulation showed that it is possible.rn U lTi1ll ~~ 2Illill'-~In bT -l-Jill !JJt !II~ off on r II ~ 1sec requiresknowledgeof disttibution of retinal projectionsto pretecttlmanddendritic' processofl1110cells. . SimulatedThl0 (3) cell against20 X 200dark SO. paling fencebarrier) is interpreteddifferently: Frogsjump Becausea Th7( 1) class neuron is a general dimness into thehorizontallystripedbarrierandalsoto the solid dark detectort it is modeled to receive R4 (retinal dimness patch but avoid the vertically striped barrier during an detector ) inputsthroughits receptivefield which is modeled escape. Anoran Visuomotor Coordination for Detour Behavior which is one importantcue for SO recognition. We posit that R3 activationshouldsignaledgelocation.!. Lackingboth off on data. In orderto preserve anedgelocation. a subclassof Thl0 . ThIO' s R3 receptivefield is modeledto be muchmorefocusedthanthat of R4. 1992t submitted)t if respirationinducedR3 excitationis possible. followed. seethe text. PretectalThl0 (3) unit. Ingle has shown that frogs view a dark horizontally The retinal modelis from Teetersand Arbib ( 1991) and all snipedbarrierandthesamesizeddark solid patchsimilarly' t modelsarewrittenin NSL ( Weitzenfeldt 1991) runningon a whereasa vertically striped barrier (similar to Collen s Sunworkstation.Of-Gaussian(DOG) mask . Schipperheyn proposed thattherespiration -inducedeyemovementproducesmoving retinal image and thus elicits activation of movement specificganglioncells suchas R3s. For the details. An understanding of SO patternformationon ThIO surface a _-. R3s and R4s excite ThlO directly. me activity ceased . After turning me light off . Fig. broad projection characteristic to Thl0t its boundary sensitivityis very poor. Respiration causesperiodic and predominantlyvertical eye movementof about 0.3: 8. after Difference . Pigarevet al. 111 10response shown a delay.2 showsthe schematicdiagram. sensitive to stationary objects. in the next two figures is basedon the DOG mask._. GrUsserand GrUsser - Comehlsreport that they observedthe most sensitiveR3 neuronswere activatedwhen a stationarycontrastobject smallerthan the size of ERF is placedin their ERFs and Fig.63 seconds(Schipperheynt 1963).

so that the animal may occasionallybehaveas if there were gapsin thefence. Thus. Thecue-interactionmodelsampleslocationsof local Fig. anuranscould detecta againstprey. an animal cannot focus on two barriers simultaneously. ii ) Seeif there House ( 1989) offered a cooperativemodel. and double paling fences. 1976) type cooperation/competition algorithm.4 showsthepretectalimageof binocularand monocularcuesto determinedepth. Prazdny formed his algorithm based on a IIcoherence " principle. Oncethe accountfor useof an activeprocesslike accommodation . By replacing the Dev-Marr algorithm in House's modelwith Prazdny's ( 1985) stereocorrespondence algorithm. Thus by relaxing the smoothness consttaint of typical cooperation/ competition stereo algorithm. a solid dark patch. b. Dark solid patch. Fig. Horizontally constituentpalingsforming the barrier. image proximity does not necessarilyimply disparity continuity. with sttonger lensescausing a fence to appearcloser to the animal. Becauserespirationis mostly vertical asa palingfencebarrier. The model demonstratedits ability to estimatethe depth of a single object(preyor predator ) anda paling fence. whereproximal points on me (retinal) projectionsurfacemayarisefrom widely separated 3D objects. Obviously. We believe We stated that depth perception is an important cue in his claimsarestill valid. Arbib and Hyun Bong Lee constructed to receivea respiratoryinducedR3 signalaswell an anurandepdlperceptionmodelof stationaryobjectssuch as the R4 signal. which arethehorizontaleccentricitycoordinatesof the Vertically striped barrier. However. oscillapons. so the model assumesthat the animal changesthe focusintermittentlyto eachbarrier. operateon sparseimages. we will present similar to aDev -Marr ( Dev. Oneof the problemswith House's modelwasthat it was not able to resolve the double barrier as in Collett's experiment . Marr andPoggio. overallactivity abovethe spontaneous level. we were able to overcome the double-fence problem .4: Stationary Objects (left ) and correspondingpretectal maximapoints on the ThlO surfacealong the horizontal ThlO images(right) inducedby respiratoryeye movement. axis. Depth Perception Model visual input from die retinais featureencoded . evencausingthe fenceto be fragmented . Prisms. We canseethat the solid dark predominantly(94%) when estimatingthe depthof prey in objectand horizontallystripedbarrierform similar patterns die binocularfield. of SO depthperceptionin which the lens along the horizontalaxis within theregionof supra-spontaneous level. nevertheless their depthestimationdoes acrossthe ThlO surfacein that they both do not present includea small(6%) effectfrom monocularaccommodation . consu -uctedbasedon thephysiologyandanatomyspecificto Gapdetectioncouldbe asfollows: A potentialgapwould frogs/toads. Michael A . 1975. theanimalcouldmeasurethewidth mustnot dependon the tectal map.becausethe visualmapscannotbe assumedto representlocal illumination level since the 2 . The model is depth(seebelow) of thebarrier. monocular toadsstill retain an the imageof the vertically stripedbarrierclearlydelineates ability to judge distance through accommodation cues. should have a more variableeffect. It alsopredicts that concave lenses will have a consistenteffect. accommodationcue is used to help disambiguate the iii ) Use the featurefrom (ii ) to segmentandto estimate ' the correspondenceproblem of stereopsis. recognizingthe different classesof SOs. c. respiration induced R3 activation usually Collett ( 1977) showed that frogs and toads use both signalshorizontaledges. However. iii ) discontinuityis detected .3. The coherenceprinciplerecognizes dlat for transparentsurfaces . Deprived of binocular cues. and iv) mustbe able to of theregionto estimatetheprobabilityof it beinga gap. conspicuousfeaturesalong the horizontalaxis. and a found that while binocular toads utilize binocular cues horizontallysnipedbarrier. ii ) must could take place along horizontalor depthaxis) . on the other hand. Houseinsiststhatany anurandepthperception be detecteddue to its discontinuityin barrier(discontinuity model i ) should not dependon eye vergence. the horizontal eccennicitiesof vertical sttipes along the Thus. the cue- are two rows of succession ' of local maxima interactionmodel. data exist regarding frog/ toad' s depth perception lower and upperboundaries . Input feawresto the revisedcue-interactionmodelcome from the Th10 patternof the SO asexplainedin me section earlier. the model is better equippedto handlequasitranspar objectslike bushes. . very little is known abouthow the paling fence as follows: i ) Seeif the pretectalimage has animalmight estimatethe depthsof SOs. This informationis striped barrier. How then do Houseassumed d1atbinoculardisparitycuesareprocessed anuransestimatethedepths? In this section. Collett a vertically sniped barrier. a.

tectum connection could shed some signalconveyingthe prey location of one visual hemifield light : If pretectum is the integration site . The activity of Snap is an increasing function w.Sib schema .disparitymapwherecue. independently (parcellation) . The prey location is interactionmodel. Prey Heading-Translatorusesx. the animal obtains an tegmentum .a showsa doublebarrierasan consistsof the dottedportion in Fig. y. detectsandselectsa uniqueprey is generated from theretina_angle. The Depth-Map schemaprovides the transition from a spatially-coded representation to a "frequency coded information and furnishes " closeness ( lId ) signal for output. the lesioned runs through the ipsilateral body side but not the animal should exhibit more direct approach to prey than contralateralside (lateralization ) . interactiontakesplace. andreturnsthe eccentricity(x) andelevation(y) of theprey. Note that the Heading-Map signal codes the target location of the requiredmotorresponseandnot the locationof the prey per settheMotor HeadingMap hypothesis . using the prey heading signalasinput. the animals heading toward the prey as calculatedby the Orient schema. Thus. First.b showsthe frog s estimationof for one half of the visual field. elevation. Because where and how the integration take place. to develop the . Different componentsof normal animal in the same situation. b. experimentaldataof Grobstein( 1988). Orient.rit. A activation level of the Orient schema. A correctfocuswill give a which is different from its retinotopicrepresentation in the strongerandsharperR3 profile thana defocused state. the closenessof the prey from Depth-Map and a decreasing function w. thedetourmodelbuildson it . and depth) are Currently we favor a tegmental integration hypothesis handledthroughdifferent pathwaysthat can be disturbed ' following Grobstein s data ( 1988) . and d values to obtain the body-centeredangle to the selectedprey from the animal. However . Anuran Visuomotor Coordination for Detou . 1991) to explain the behaviorof normal depending on prey-barrier configuration . and Approach motor schemasactivate corresponding pool of motor neuronsin the spinalcord that actuallycarry out the action. sincethe modelis rooted ' topographical projection from pretectum to tecmm? Ingle s in . Fig. Accommodative statedirectlyinfluences representedrelative to the animals body ( body-centered ) the fIring prQfileof R3 neurons. 8. A Dura D Detour Model 3 .o Behavior usedas an input to the stereopsiscomponentof the cue.r . the morestronglywill it stimulatemotoneuronscontrolling the turn muscles. Fig. tectum. Similarly. calculatesthe targetlocationof the required motor headingand outputsit populationcoded. way. 3. Could pretectum be the integration site? population(frequency) coded: The horizontaleccentricity A lesion at pretecto. Several of these double barrier is given as input stimulus .S: Revised Cue-Interaction model for a double barrier . a shortaccountof ( 1983 ) lesion data showing that the atectal frog avoids his ideasis given. we presenta brief summaryof Does the integration take place at tectum via theprey capturemodel. barriers while escaping noxious cutaneous stimulation Grobstei. Grobsteinreports that somatosensoryand tectal by varying the accommodative stateand measuringthe R3 prey information are integratedat the level of midbrain firings (actually ThlO firings) . the Approachschemadeterminesthe distancefor the animalto advanceas a function of the closenessto the prey and the Fig .S. Barrier Integration In this section. the final motorbehaviorwill resultfrom the combined activity of thesemotor schemas . locatedin tectum. Some Issues on Prey . In this estimation of the depdt of the double barrier . showingthe schemas ' input stimulus.1 . Estimation of the barrier location is represented in a probabilistic measure. The modelwassuccessful in simulatingthe experimentaldataon anuranpreycatching behaviorwith andwithoutbrain lesions.6. An integration of barrier and prey signals has to take place estimationmodelswith the schemamodel of prey capture in order to determine detour or direct approach to prey (Arbib & Cobas. The more stronglyOrient is activatedby larger headingvalue. the Prey-Selector - thebarrierlocationafterthecue interactionprocess . But fIrSt. the prey location (eccentricity. Frog or toad' s motor schemasmay be active at the sametime.n and his colleaguesobservedthat the tectal suggest that pretectal barrier signal is not disturbed by the signal conveyingthe prey location at the caudalmidbrain tectallesion . and so tectum is not likely the place of prey- level shows lateralization and parcellation traits and is barrier integration.t. A question is then andlesionedanimalsengagedin detourbehavior. accommodativecue which is the other componentof the A schema model for prey capture : The model cue-interactionmodel. Heading-Map. Depth-Translatorthen calculatesthe depth (d) to the prey throughx and y valuesfurnishedby Prey-Selector. we integratebarrier perceptionand depth. and FigiS. Snap. In a similar way.

. it computesdepthto the barrierand Tins . .6 illustrates a schematic overview of the complete Whenthereis no internalgap. . We will also assumethat the pretectal signal is computesthe anglesbetweendie prey and the gaps. . . . . . signal from die Prey-Selector. . . Mu ~ . . The schemaprovidesdie x and y coordinatesof eachpaling to the Barrier/Gap Depdi Translatoranda setof x andy coordinatesof die gapsto the / d ~. die strongeris the activity integratedas a detoln" model. Ap Snap . . .. . . . When there is only one internal gap. Note thatdiebarrierendingsarealsoconsidered to defmedie gaps. . Schemas of the barrier and the prey from the Barrier/Gap Depth in the dotted region correspondto the Prey-Capturemodel of Translatorand the Prey Depth Translatorrespectively. . . . Heidi " ng . Translator ~ . . . Map . . we lack the data implementedin a neuralnetworkasdescribedin section2. . . . . ~ . ttanslator: Activation of Barrier-Recognizeralonedoesnot . Gaps 7/Gap Barrier -Heading Gaps-HeadingTranslator . . . . Fig . .-. parcellatedand latemlized. . . . . .. . . 00000t . An activation of the Prey- . . . This relationshipbetweenthe Prey- . . . . . l - Recognizer . . . . Translatorcomputesthe body-centeredanglesto the gaps. at leastthe TranslatorTranslator Depth pretectumand pattern formation is assumed part . . . .6: A schematic model of anurandetour behavior . . Michael A . . . . . the Arbib & Cobas. activatethe schema . . Translator the spatial domain and the resultant motor heading signal Heading Translator Gaps Heading will leave the tegmentum population coded after the signal respectively. . . . . Distance ~ 0'-. . . Arbib and Hyon Bong Lee integrationmodel. . Spatiallyooded : -. The Barrier-Recognizerresidesin to becomeactivated. . . spatially coded or population coded. . . . . Usingx andy coordinatesof thepalingsreceivedfrom : . . . andDepthMap schemas . lId d: distance : cDel8S ooded Frequency pathways . M1 & Q . . Translator.jar Barrier . . . eActivation . . . depictedby priming signal going from the former to the . JP . . . . . . . . . Depth IBarrier estimationand gap detectionneedfurther processingwith . model and the schemas in the non . + ~DistanCE die gaps(gap-depthis considered to beanaverageof depth(s) ~ . . . . : : : : : : : : ~ ~ . = J . . . The schemafIrSt searches for internal gapsand selectsdie 3 . . PreyH Barrier DistanceTranslatorcomputesthe distance betweenthe prey and the barrier. Selectoris necessary Tie for theactivationof Barrier/GapDepth . . PreyH Barrier Distance Translator. inputs(feawres . of neighboring paling(s . . For instance. . . . ++ Selector Prey Prey ) furnishedby die Barrier-Recognizer . modifications to the schemas in the prey capture model . . . fr8lls1ator ' Translatorschemais implementedas mentionedin section / 2. . . . It is becausesomestationaryobjectslike a paling fencebarrier seemto be recognizableby their patternsalone. In this section. . we Map schema converts spatially coded Prey HBmrla- of the barrier Distanceinformationinto a frequencycodedsignal: When present a schematic explanation the prey lies behind the barrier. . The activity level will be set to 0 when die prey is in front of or at the samedepth relative to the . . we needto know moreaboutthe nature Barrier Recognizer: Barrier -Recognizer schema is of pretectal outflow. . . . and how these are between the prey andthebarrier. then that is chosen. ' endingwith smallerpreyH gap angle. . . Gaps-Heading Translator : Using x and y coordinatesof ~ ~ 11 ~ ~ ~ ~ ~ ~ l j thegapsfrom the Barrier-Recognizerandthe depth(d) from . . the PreyHGaps Angle Translator/Selector conversion. . ~ . . . . . this schema : -. . . --. . ..dotted region are responsible for recognition /localization of a barrier and Prey H Barrier Distance Map : PreyH BarrierDistance integration to the prey capture model. pretectal PreY H Gaps Angle Translator /Selector: Using heading values of the prey and the gaps from the Prey- (barrier) signal and tectal ( prey) signal can be integrated in and the . . regardless of detectionof prey. . . Detour Model gap that hasthe leastanglebetweenthe prey and the gap. . . Mu . . : : c :cc . . latter. . .Inhibition h: heading becomes activatedonly after the receptionof the priming . pathways oooldinates x. 1 / h . . . .2. the greater the distance recognition /localization network .5. . . The schemaefferentsits output to Gaps-Heading . Selectorand the Barrier/ Gap Depth Translatorschemais . . / Gating Priming Prey H Barrier Distance Translator : Using d values Fig . . Barrier / Gap Depth Translator : The Barrier/ Gap Depth ~ Hading TranslatC'/Seledor X . . . the schemaselectsa barrier detour model of unit responsefor one half of the visual field . . . . . . . Adv Soappi & \ g ~ . . . . . y: retinotopic Similar to the Barrier/ Gap Depth Translator. . . ~ pJh die Barrier-Recognizer l+Barrier . . We will assume that the pretectal signal is spatially coded and thus. Jat Qr . It also determinesif the on whether prerectal outflow arriving at tegmentum is preyis behindor in front of die barrier. It detectsthebarrieranddie gap. . . ~ . . h . the Barrier/ Gap Depth Translator. . the Gaps-Heading . . . . level of its output. . The schemaoutputs The -dotted portion shows Arbib & Cobas prey capture die headingvalueof theselectedgapto die HeadingMap. . .2 .. . . Barrier- dt . Ma 11 . 3. .Ps x . .Angle & 1++c. . .

frogs neuronpoolsto makethe animalturn to the directionof the usually detouraroundthe barrier. formation. Just like the HeadingMap. So. themodelforces unawareof a largehemifieldbarrier. Lesion effects distanceincreases . But it will always the way is almost nil due to inhibitions from both Orient collide to me barrier if the critical barrier edge is in andPreyH BarrierDistanceMapschemas . eventhoughthe Barrier-Recognizerperformspattern The HeadingMap is modified to havetwo input ports. the behaviorcan be replicatedby . the chancesof the gap. the Barrier- performance of thePreyCapturemodelbecausethechanges Recognizerschemais not activatedandso the prey signalis canbe madeon top or independentof the original schemas gatedto the Headingand Depth Maps. signal. If the prey is in front of or not far behindthe gating signal from PreyH Barrier DistanceMap. The schemasrelatedto the integrationof barrier and prey HeadingTranslatorshouldbegatedto theHeadingMap. animals. our computationalfrog will detourin pon high enoughto have it gated. we look at simpler caseswhen only prey or barrier schemas do not changetheoriginal infonnationflow nor the aloneis present. This situationis handledin our model to selectbetweenprey-depth and gap-depthcoming from becausethe PreyHGaps Angle Translator / Selectorprefers Prey Depth Translatorand Barrier/ Gap Depth Translator internalgapsoverbarrierendingsby searchingthemfIrSt. Theactivity level thebehavioraldata. resultingin a directapproachto prey. this situationseemsto prefer the internal gap over the two The principle of the modified DepthMap is identicalto lateralbarrierendings. The schemaactslike a switch in the sensethat its and the Depth Map. as usually approachprey directly. frogs signallevel from thePreyH Barrier DistanceMap. the lesslikely the snapwill occur. Notethatthepretectalbarrier/gapsignalis parcellatedand We believethis behaviorcan be replicatedby severingthe lateralized . depth and the gap-headingbeinggatedto theDepthMap and we believewe know the modelwell enoughto offer a fIrst the HeadingMap increases . presumedto be located at midbrain output level servesas a criterion to detenninewhetherthe tegmentum. evenif oneof the latter gapsoffer a themodifiedHeadingmap. headingsignal will be selectedand the animal turnsto the A passable gap within the barrier : A frog/ toad in prey. Normal functions organizationof the Prey Capturenetwork is unchanged . As a result. When the barrier is absent. becauseof no priming signal from the Prey- one for gap-headingand the other for prey-headingsignal. modifiedDepthMap havenot beensimulatedyet However.1. whena with " split pons" seems to know whether or not to frog approach es a gap. a pretectumlesionedfrog seemsto be Depth Map haveto agreeon their selection . wherethey are then ttanslatedto a population gap-depthfrom theBarrier/GapDepthTranslatoror theprey. HeadingMaps. the HeadingMap signals more gateableto the Depth and Heading Maps translatesthe headingvalueinto a populationcodedsignal. Barrier/Gap Depth Translatorand GapsHeading Both ports can be active simultaneously . alone. and preyH barrier distanceincreases . zero or weakgating signal from the PreyH Banier gap (barrier end) signal will be routed to the Depth and DistanceMap may not raise the activity level of the gap.a weakgating signalhaveincreasedprobabilityto be gatedto the Heading signal from the PreyH Barrier DistanceMap makingprey Map. With barrier integrity. and Snap First. Since both the HeadingMap and the Accordingto Ingle. depthfrom the Prey-DepthTranslatorshouldbe sentto the Depth Map. Heading Map. Anuran Visuomotor Cool "dination for Detour Behavior barrier. of gap-headinginput pon positivelycorrelatesto the gating Prey close behind the barrier : In this situation. The output level is also used to detennine 4. respectively . Thus. Modifications to Prey Capture Model : The basic 4 . Due to a high level of selectedgap. bothschemasto vote for theselectionuntil theyagree. In this case. which conformsto relativeactivity levelsof two input ports. This behaviorcan be . DepthandHeadingMap Heading Map includes a competition network where receiveneitherprey signalnor gap signal. whichthenexcitestheOrientschemato innervatethemotor Prey far behind the barrier : In this situation. In gap-headingpon increases . Selector. Modifications to Depth Map.2 . In this case. the DepthMap has closerpath to prey. Thus. Thus our model which could be encapsulatedto preservemodularity and performsjust like the Prey Capturemodel. the probability that the gap-depthis chosenincreasesas the preyH barrier 4 . replicated in our model by unavailability of Barrier- The Snapschemais modified to receivean inhibitory Recognizer in pretectum. selected barrier. the prey this case. Ingle also observedthat a frog signalfrom thePreyH BarrierDistanceMap. If the gap headingpon is selected . Thus. the possibility of it snappingduring approach / snapa prey behindme barrier. Performance of the Model whetherthegap-headingvaluefrom thePreyHGaps Angle Translator / Selectoror theprey-headingvaluefrom thePrey. monocularview field andlocatedon the samesideof prey. The modified Translatorarenot activated. theactivity levelon the worm is within the snappingdistancefrom the barrier. The outputof the schemaalso accountof whatcouldbe expectedof the modelwhenasked inhibits the Snap schema: The larger the preyH Barrier to simulatethe dataon the behaviorof normalandlesioned distance . which makesthe gap-heading our model. As signal such as PreyH Barrier Distance Translator. the probabilityof onepon selectedovertheotherdepends on the animal doesnot show any movement. Both the prey relatedsignalandthebarrier/gap Prey+-+GapsAngle Translator/Selectorlink coming from relatedsignalfrom tectumandpretectumre~ tively remain the conb"alateralsidewhich is responsiblefor the gapin the spatiallycodeduntil theyareintegratedat the HeadingMap ipsilateral monocular view field : Due to presumed . presumablybecausethe thepreyH barrierdistanceincreases .

J. : Respiratoryeye movementand Behavior . BrainBehav . Hanson . the prey-barrierconfigurationunchanged .Wilson. OJ. : Neuroethology of releasing mechanisms : Prey- mutual inhibitory circuit. andArbib. 146..). R. Robotics . whereasthe Koren.P. : A Neural pretectal decussation but preserves tectal decussation Model of Interactions Subserving Prey-Predator responsible for the lateralizationof tecta! signal. We should also ask how catching in toads . R.(1989 ). : Cooperative computation of stereo References . 177- for manyanurandetourbehaviors .R. Brain and Teeters . . : Do toadsplan routes ? A studyof detour known data of real animal as possible. 142-151( 1991 perceptionsof stationaryobjects . 81 . J. : DepthPerception in frogsandtoads . H. : Perceptionof depthsurfacesin random -dot assumptionthat prey and barrier signal are integrated stereograms : a neuralmodel. P. 194. MiA. M.J. : Influenceof thebackground . Springer -Verlag.J. competitioncircuit betweenthem. -PoEwert. disparity Arbib. JL .L. address es the parcellationand lateralizationof the signal Proceedings of the1.A. Cooperative computation . 8. Arbib. 7. in die model. (in Fromanimalsto animats . J. : Betweenthe retinotectalprojectionand brain. (Advances in Vertebrate Neuroethology . Physiol . 10. Evol. an aspect of the future researchis to responseto loomingobjectsby frogs and toads. vergl. the methodis 226 (1983 ). K. Collett's data also show that an animal's behavior is GrUsser . R. S. Our House . F. .991 IEEEInternational Conference on flow. J. Splitting the pons seversthis Cervantes -Perez . J.. in the frog. contratateralpretecta ! lobe. J. A BradfordBook/ MITPress .H. show detour and in anothertrials a direct approach .. InternationalConferenceon Simulationof Adaptive Schipperheyn .A. Lee. Ewert. M. GrUsser -Cornehls .A.Ingle. Bio. M.H.. 261- formulationis basedon hypothesesdlat needto be verified 271(1982 ). : TheMetaphorical Brain2: NeuralNetworks Pigarev .B. not basedon the neuroanatomyof the animal. P. Prazdny . D. prey or gap in the Depth and Heading Map schemasis Ingle. U. A. Arbib. 74 . ). CompoPhysiol. Ewert. : Neurophysiology of stochastic: Evenduring the sameexperimentalsessionwith the Anuran Visual System . : A neuralnetworkmodelfor algorithms.102 ( 1971 ) .W. We haveseendlat thepotentialfield methodcanaccount J. WileyInterscience . 64. Discussion Collett. : Stereopsis in toads . The assumptionthat only one Studies . : Potential FieldMethods andtheir schemamodel is neurally inspired in the sensethat it InherentLimitations for Mobile Robot Navigation . Eds) Proceedings of the First Cybernetics ). J. Meyer . networks but can be easily implemented in sequential Liaw.. T. Man-Machine spatiallyat the tegmentum . 297-385(1976 ). 93-99(1985 & S.A. Borenstein . Behavioral and Brain Sciences . Eds).159( 1963 ). 511-528( 1975 ). Springer -Verlag. 283-287(1976 ). the model is basedon the Dev. While we tried to constructthe model reflecting as much Collett. : MotorSchema basedmobilerobotnavigation .. observedin our preliminarysmdy. Corbacho .P. 241-249 in the ipsilateral view field comes solely from the (1983 ).Arbib & A. J-A. Eds). frogsandtoads . Vol. Sacramento . No 4. . M. PreyH Barrier Distance Translator and the PreyHGaps 337-405(1987 ). by experiments .. : Brain mechanisms of visuallocalizationby modeledasa stochastic process.S. Arbib and Hyun Bong Lee lateralizationof the pretecta ! signal. 52.R. ). House . Pharmacol .N. : Quantitative modelingof Responses of AnuranRetina: Int. Neerl. (1989 of the frog retina. Arbib. T. Theor. Eds). 31. It could simply be a Ewert.1458( 1971 Arbib..Bioi. 1448 ).Precht . Research notesin NeuralComputing .167-180(1991 ). MiA. theycanbe implemented asa directed movement : Topography of sensorimotor interface . 113. A. CompoPhysiol. 349-351 (1977 ).J.. P. Res.Arbib& abovetwo dlat canalsosimulatethenewbehavioraldatawe J. 157. 1398 - containssomeschemasthat arehardto implementin nemal 1404( 1991 ). . Teeters .M. However. : A modelof anuranretina relatinginterneurons to ganglioncell responses . M. Acta. Int.-P. Lecture modelcanaccountfor this stochasticitybecauseselectionof Notesin Biomathematics 80. G.A. Responses in unrestrainedfrogs. Michael A . : Detectionof binoculardisparities .. 197-207( 1991 Eds). Nature 267. a frog sometimes R ilinas andW. J. ( 1992 for discriminating objectmotionfromself-induced motion Research ). But the latter model'also hasa drawbackin that it Robotics andAutomation .B. Headingmapand DepdlMap exist and dlat the prey signal Ewert. 117-152( 1985 ). Science .Caprinicaand D. For instance. FrogandToad. MiA..Y. Cybern ). J. Z. (in Vision. : Schemas for Prey-Catchingin Sechenov J. So. 129-163(1987 ). Marr. Bioi. D. J. 92-112 ( 1989 ).. much of the behaviorof Biviridis. Angle Translator/Selectormight be realizedin the anuran Grobstein . simplespatial domaincomparator .: Thedetectors andBeyond . F. D. Submitted to Vision Burghagen . Arbib. Poggio . Ginnan. D.. Discrimination andSizePreference in AnuranAmphibia . Stimulusshape andsizedependency . (in developa new detourmodelthathasthebestfeabJresof die VisualStructures andIntegrated Functions . Zenkin. 12. J. However. : Singleunit responseof the toad's (Bufo and the gap signal have to competeeachother to deposit americanus ) caudalthalamus to visualobjects . T.A. 34-48(1988 ). California . USSR57. Lara. Arkin. M. their contentsto the mapssuggeststhat there shouldbe a Physiol . . (Frog Neurobiology. H. 5. Cobas .. : DepthandDetours : An essay Physiol on visually guidedbehavior .. infonnationof the gap in toadsBufo hufo (L. J.

C.B. A. ( 1991) . Proceedingsof the 1989 IEEE InternationalConference on RoboticsandAutomation . : Global Path PlanningUsing Artificial Potential Fields. Warren.W.W. 316-322(1989 ). Cincinnati . Neural SimulationLanguage . 500-505( 1990 ). Center for Neural Engineering . Proceedings of the1990IEEE InternationalConference on RoboticsandAutomation . C. Version 2. AZ. Scottsdale . Anuran Visuomotor Coordination for Detour Behaviol ' Tilove. : MultipleRobotPathCoordination Using ArtificialPotential Fields. Warren . Weitzenfeld . Cincinnati. Universityof Southern California . . CNE-TR 91-05. Ohio. : LocaIObstacle Avoidance for MobileRobots Based ontheMethodof ArtificialPotentials . Ohio. R. Proceedings of the1990IEEEInternationalConference on Robotics andAutomation .1. : NSL. 566-571( 1990 ).

the next step is to incorporate the insights input.e. Cybernetics . Fac. i. activationor movementparametersfor the behavior. reactiveor sensory-driven adions. In most complex biological networks. however. At one extreme are networks more complicated behaviors. severalproblems have to be solved .disturbedor the model begins walking &om an arbitrary starting configuration. power stroke algorithmspreviouslyusedto control leg state and and swingor return stroke. The model extendsa previousalgorithmic one leg. movementof the legsis . The secondquestionrefers to control systemusuallycontainselementsof both kinds. From an applied control which mediateactionsthat are dired responsesto sensory point of view. The behaviorof this control systemcorrespondsto . On a higher level the control interleg coordination. Neural networkswhich initiate and ~ ntrol the behaviorof A study of this systemmight therefore be of interest not animals embody several features. adaptivegait ) generation. University . First.e. To do so. the individual leg is controlled. FRO E-mail: bolkat bio128. In fact. each leg has its own control system . As part Heuer Schmidtim ) . the role of sensoryinput. i.uni-bielefeld .how animals control the movementof the legsduring walking (reviewCruse 1990). D-4800Bielefeld1.leggedwalking system. are which generatesrhythmic step movements(reviewBassier dispersedamong severalsubsystemswhich interad more or lessstrongly. One question concernsthe way the movement of the pattern generators. Although it is sometimesregardedas quite a simplebehavior.the following answers fundions. Each center is responsible for two mutually exclusivemicrobehaviors(Wstates" stanceor model in substitutingsimple artificial neuronsfor the ). For a system to produce suitable actions in an unpredictable environment. The control produce a suitable behavior of the overall system. 1983). The model is in the environment. At the other gainedfrom the biological experimentsinto a model able are networks which themselves generate the basic to control a walking system.most of the information comes from network. control experimentsusing the stick insect. a lot of information has been 1 Introduction collected in recent years on . From a biological point of view. The flexible control appearsto arise basedon biological data obtained&om the stick insect. Artificial neural nets for controlling a 6-legged walking system HolkCruse .de Abstract Both aspectsare espedallytrue for walking.. The model is able to walk at systems for the individual leg have to cooperate to different speedsover irregular surfaces . walking is simple enough that one canhope to gain a completeunderstandingof the basic me~hanisms. Typical control systemsinvolveautonomous stablesupport for the body and at the sametime elements as well as simple reflexes and more complex -drivenmodulationsof centraladivity (CruseDean propel it forward. from the cooperation of several autonomous control It representsa combinedtreatment of realistic centers(in the terminologyof Minski (1985 thesemaybe called WagentsW each of which controls the) movement of kinematicsand biologically motivated. A the coordination between legs . One feature concerns only in itself but also as a model for the control of other. it involves A systemthat controls the leg movementof an animal a very strong and complex interadion with the physical or a robot walking over irregular ground hasto ensure environment. UweMtiller-Wilm. it has to react adaptively sensory to unpredictablefeaturesof the environment. JeffreyDean Dept.of Biology of Bielefeld . systemrapidely reestablishes a stablegait when the propelling the body while maintaining postural stability. The combinationmakesthe walking of our study of the underlyingmechanismswe present systemindependentof particular stimulusinputs but at the here a model which describesthe control of the leg sametime enablesthe walking systemto adaptto changes movementof a 6. autonomous activity or actions controlled by central To reach this goal.of BioI. From biological second feature concerns the strudure of the control experiments . whether sensory-driven or autonomous can be given. Postfach 100131 .

is controlled by an oscillatory system.e. we plan to replace successivelythe algorithms of this earlier model by modules of artificial Contact with the ground is assumedto occur at a non- neural nets. the leg is on the ground. a ) c) Artificial Neural Nets Controlling a Walking System that of a relaxation oscillator in which the change of state. hasbeen called the 2 Model anterior extreme position (AEP ) and the posterior transition point has been called the posterior extreme The movementof the individual leg is controlled in the position ( PEP) . The oldest and simplestproposalfor a network oscillator with neuronalelementsis the Brown half center model . actually is a ball and socketjoint . The role of load in modifying this criterion will this be done using a neuronalsystem? be neglectedhere (seehoweverBassier1977. point. The tarsus or foot of the insect. (a) Schematicdrawing of a stick insect wqith the control of the dynamics. the coordination of the legs is not determined by a hierarchically superior control system. This distributed architecture results in a very stable gait but nevertheless allows the whole system to react flexibly to disturbances. on this basis. ( 1992). therefore. Danowski way as in the stick insect and. The structure of the leg and movement of the individual joints of a leg is still the definition of the anglesare shown in Fig 1. The rhythmic movementin the direction of when the leg reachesa given criterion position and that the x-axis (Fag1)~i. Dean 1991a ). In order to simplify the model.down movementis performed at the coxa-trochanter it can begin a new power stroke.e. The axes controlled by the classicalmethodsdescribedin Mii1Ier. the leg is lifted off leg in the forward-backward direction and that all the the ground and moved in the direction of walking where up. 1992) a model rotation. During the power movement during walking involves only one axis of stroke. the transition between power and return strokes. are not ( im ) . (c) singleleg of a stick insectshowingthe axesof waspresentedbasedon thesebiologicalresultsand. The receiving system colleds this information and. During the return stroke. supportsthe body and. . Cruse 1983. In the present report we describe neural slipping hemisphereat the end of the distal segment. As in the model of Miiller -Wdm et al. However. we assume " experimental results several authors (Wendier 1968. In the stick insect the basaljoint The movementof the individual leg consistsof two parts. Weidemann " . of rotation of the basaljoints are arranged in the same WIlm et aI. eachother and to the body by three simple hingejoints. i. ~ ovesbackwardswith respect we assumethat the basaljoint is used only to move the to the body. Instead. eachleg is Because models using distributed networks show a representedby three segmentswhich are connectedto number of advantages. Therefore. necessary. generalisationetc. Graham 1972 . Cruse 1985b) proposedthe alwaysmovesin a vertical plane parallel to the long axis idea that th. where rotationd ) the joints of the simplified leg of the model. decides on its own action. The anterior transition joint . the primary the power stroke and the return stoke. in rotation (Cruse 1976). several ad hoc assumptions . the transition &om return stroke to power stroke in the forward walking animal. on leg position .. Second. is determined I1Ythresholds based. the gait pattern emerges &om the cooperation of the separate control systems of the individual legs. This cooperation is ~ \ based on different types of signals which convey information on the actual state of the sender to the control systems of the neighboring legs. location of the body-centeredcoordinates . for the straight walking consideredhere that the tarsus Bassier 1977.etransition from one modeto the other occurs of the body. orthogonal with respect to the body-fixed coordinate systemshown in Fig 1. This model was constructed using the classical algorithmic approach. At the present time no detailed information is available concerning the control of the individual leg joints of the walking animal and the Fig 1. ( 1992) and Pfeiffer. (b) mechanical model: arrangement of the joints and their axes of In an earlier paper ( Mii1Ier-Wilm et aI. How can osci1Iator.This network moduls for controlling the oscillatory movement simplified leg omits the set of short segmentsforming the of the individual leg and the coordination of legs. Either as a hypothesisor on the basisof following way. for the present model the forward walking animal. such as error tolerance . the axis parallel to the long axis of the step generator can thus be considereda relaxation the body.

As a result which either one neuron or the other is active. Some central pattern generators recurrent network . Again the integrator represents the connections ( e.C 000 II . In order to test whether unit RS in Fig 2a. is applied to the leg until a senseorgan with a fully con~ected recurrent network with randomly signalsthat the AEP bas been reached. shown in Fag2a. the movement of the leg under the . this is the most general form . interpreted as a velocity signal moving other versions of such oscillatory systems exist we started tbe leg forward.neuron nelworks are discussed by Camhi ( 1984) . The transformationfrom leg velocityto leg and lo Dic excitation is applied. the step pattern generator of as both motor units receive an input of 1 which can be " " arthropods is better describedas a relaxation osciUator interpreted as an on signal from higher centers. (a) classical ( Brown 1911) . the signals from sensoryfeedbackcan provide appropriate feedback to produce a rhythmic An advantage of the artificial neural network approach is movement (Land 1972. PEP) . as steps. systemswitches from one state to the other. We trained and causesthe leg to moveto the rear until a senseorga:1 netVt'orks with this structure using a linear activation . apart from the absence of turns off the motor unit PS. position is symbolizedby the integrator in Fig 2a. This sensoryunit distributed weights because.flop ) .. (c) the output vector for consecutive (return stroke) and PS (power stroke) . n AEPon Sboke II 0 0 ~ 1 0 1 0 Power 1 0 AEPoff Sboke 1 0 ---~ Fig 2. the system oscillates. in turns off the unit RS which in turn inhibits PS. (b) the same system constructed as a fully using mutual inhibition between the motor units RS recurrent network.) Q . The weights of the synapsesare given. Once on the left side indicate the input and output vectorsused the leg has reacheda given threshold (AEP. which have been characterized physiologically contain If no number is given this means that the weight after neurons with still more complex dynamical properties and training is zero. This meansthat the shows the output of the system for consecutive iteration peripheryhasto be includedin the system. Leg position is iteration stepsin one completestep. fonning a bistable system (flip. mechanical periphery . more general mutual inhibition is replaced by mutual excitation passed format . Holk Cruse . Getting and Dekin 1985) . This senseorgan inhibition ( F"1g2a) . This unit in turn inhibits RS hidden units. If the the systemoscillatesrhythmically moving the leg between inhibitory channels are provided with high pass properties AEP and PEP.g. oscillation can also occur if the rag 2b shows the same system in another . A model for the control of rhythmic leg movement version. the for training. The leg position oscillates as long As mentioned above. In this case. The squarebrackets obtainedby integration of the output of RS and PS. Fag 2c which dependson sensoryfeedback. For example. for a general that training procedures can be used to find alternative discussionsee Bassier 1986): the motor output of motor and possibly better solutions . Uwe Miiller . Alternative . Two neuron es are connected by mutual signalsthat the PEP has been reached. influence of RS and PS. Wilm . The four units are now part of a fully connected through a delay element. Bassier 1977. and Jeffrey Dean r : --~ a) 1 ~pII AEP b) M ~~l I c) ~ (~Q 1)(1 . ~ 1 0 1= ~ i 1 0 0 1 1 PEPon ~ 000000 * - II II PEPoff R. two. 819 II .

Therefore the Delta rule canbe applied receiver using the output vector of one time step as input for the sender RS PS PEP AEP next . One is suddenly vanished in all three systems .e. training cycle 0 zero and 1 i . This output lacks one For vectors system(Pig 3) was least sensitiveto noise. xcitatory Fig shown network looks like in welghh obtained are required to disturb the mixed system( Table 1 and the excitatorysystem( Fag3) to the same extend. The ) a noise amplitude of + / .sired reponse to the feedbackvector 1001. rhythmicleg movement. AEPoff PCM ' 8r SboI8 ---~ function and the Delta rule as proposedby Levin ( 1990 Table1 The weightdistributionof a networkproducing ) for the training. the net was trained on two heteroassociative tasks: the de. When the noise amplitude instead of reciprocal inhibition. around -instead of values 1 of ' able . The sensoryunits have exceededa certain threshold. 00WWa. e.5 0 0 (output vector: 0100).0~25 is sufficient to disrupt the oscillatory behavior of the inhibitory system whereas " with values amplitudesof about + / .50 and + / .1000. In this way the net was trained on two AEP -0.5 0 0 autoassociativetasks: when the net is in power stroke PEP 0. "power stroke noiseis present. In a recurrent systemthe output vector is rhythmicoscillations .1 the error amplitude range added to all four feedback lines.01 after 180epochsand approximatedzero. the oscillatory behavior two different effects on the two motor units.e.76. Theformatis the sameasin Figs2b alsothe feedbackvector and thereforethe input vector for and3a. corresponds to that U9 PEP~ I : : shown in Fig 2 except ! 1 for the intermediate 1 1 0~ II 1 0 00 01 0 0 O 1 ~ PEPoIr R8bJm AEP~ ~ vectorsmarked by stars in Fig 2c. i.. Artificial Neural Nets Controlling a Walking System ~~l I a) b) Fig 3. The inhibitory excitatory.~itive and the excitatory of the systemafter training. Fig differencesin the stability correspondto the size of the s. For the mixed system this error could not be made smallerthan 4. Quantitatively.5 0 0 same should be the case for the return stroke RS 0. For the corre.2. a. 5or 0 . the other is inhibitory: Fig 3b showsthe output system (Fig 2) was most sen. A mode~ obtained by . setof 0 . C J .. the next iteration. In this net the motor units receive a self-excitation stability was measured by observing the oscillatory behavior of the systems .e. The oscillatorybehavior a. they might still differ in stability when responseto the fcedbackvector 0110. In addition. repsectively " of The 2a system Fig a mixture behavior ofthe" and the e of both systemthe .0. it shouldstay in that state.0. "return stroke on" and "AEP Although the unperturbedbehaviorof all three systemsis reached". II 0 0O1{ ~ 0 0". 0100) and the desired nearly the same. i. to"i of thatnhibitory 3a . training . The inhibitory systemwasneverobtained .5 -1. of. The resultingweightsare shownin Fig 3a. is5 T (distributed symmetrically presentin the resultsof Fig 2c. These ) .~ponds excitatorys)'Stemthe sum of the total squarederror was smallerDian0. The decreasedto zero. for the start: 0 control of the . a. three oscillators was tested with noise of ( ) increasing Using the Delta rule with an learningrate of 0.5 0. was"powerstrokeon" (i. the performanceof all on" and "PEP reached" was "return stroke on" i. 2b . ps J ) PEP AEP II ~ - of the 0 o~ system 1 1 - 1 - 1 J 0 1O0. and the PS -1.5 0. To investigatethis.5 (output 0 0 vector: 1(XK ) .e. values another ..5 -0.vstem mean error during the training procedure.

Once the trajectory of the tarsus relative to the body which corresponds to that of Fig 3.C1992a . shown by a multiplication caudalleg beginsactive retraction... See text for further explanations... ( 1992) and therefore not repeated leg onto a centrally located lniddle leg. The illustrated for a mmiddleleg receiving . different from zero .... when the caudal leg has no mCl..'. the values of the joint angles can be calculated.-h~J1ismsarising from ipsilateral legs: ( 1) a rostrally ground contact. The sensory input GC described by Pfeiffer et al. For the stick insectit hasbeen shownthat in ~ .. controls the state and ~ selected. parallel to the x-axis and parallel to the z.- irregular surfaces.--------. the coordination module in the present model.. the other has to do with distributing contralateral legs. tarsus trajectory and the required joint movemments~ sYD1bolizedby the box mmarked( 1T -JC) in Fig 4. . (One servesto correct errors in Influences ( 2 ) and ( 3) are also active between leg placement. However. ! -I: axis..-----.. the state value is (.. respectively (Fig 1) . whereas beginningof a return stroke.. this for the stick insect(review Cruse 1990).. return stroke) of eachleg. directed influence depending upon the position of the next rostral leg. The derivation of the corresponding tarsus trajectory and joint angles occurs 11Us ~ done here using the inverse kinematic solution in the box mmarked ( 1T-JC) . is modulated by three active. Two of thesewill mmechanismm ~ responsible for the targeting behavior... Uwe Miiller . caudally directed influence (4) In all. This causes the . Two deter diination of the tarsus trajectory ~ described in detail influences occur frommthe front leg and two frommthe hind by Miiller -WIhn et al..axis ~ governed by a velocity controller ... that along : the z-axis by a position controller .---- through training... Influence ( 2) is zero when the caudal leg directed inhibition during the return stroke of the next has no ground contact . Therefore. not be consideredhere. propulsive force among the legs) .. the end of the downward movement during the final part of the return stroke has to be detennined by an additional sensor which ~ asswned here Fig 4. i. When walking on . ( 1990) ... the signals.. Influence ( 1) is only of a power stroke ( PEP) . six different couplingmechanismshavebeenfound depending on the position of the next rostral leg... $ ~ 11Us references ~ al ~ used to control the mmovemment of ____L______. They do not require aocontinuousinput I as the inhibitory systemdoes. ( 2). The four coordinating influences used in the mmodel to be a contact sensor (MgroundcontactMGC in Fig 4) .. forward -backward mmovemment of the leg. as an example. II II It should be mentioned that the systemsshown in rig 3 II and Table 1 canbe switchedon by a short impulseto one II of the motor units.. and (3) a caudally symbol for convenience) are represented by inhibition . ~1 controlling feedback system (Cruse 1985a .- velocity feedbackcontroller. first put through a low pass filter. In neuronal terms these caudalleg. The Influences ( 1) ...e.. Wilm .(3) (4) Weiland and Koch 1987)... the mmovementalong ! the x. b) which formed the basisfor leg and the hind leg onto the ipsilateral middle leg.-- module to control the state ( powerstroke. . and therefore the end-point influence (4) acts on the AEP unit . The oscillator . As the excitatorysystemhas ----- <!> I c proved to be the most stable we decided to use this --. Y the leg tip ( tarsus ) in the horizontal and verti ~ l directions.-------------. i. However . ! &. the influences from the front m~ l ( Dean1991b. we use the output values of the state controller as referencesignals for a . Dean 1984.. even when the starting weights were IIr chosento CQrrespondexactlyto this solution. The beginning of the power stroke (AEP ) ~ How are the movementsof theseoscillatorscoordinated? mmodulatedby a single.... The calculation of the registers ground contact. here.. aDd Jeffrey DeaD ---.56 Holk Cruse . (2) a rostrally directedexcitationwhenthe next conditional effects ( in Fag 4 . &I both the power stroke and the return stroke the movement of the tarsus is controlled by a velocity ~.(2) : (1) reversalsin the direction of movementto be less abrupt I I and therefore more realistic..e..- ~._-----"' . The other four mechanismswere success fully implementedin an earlier Fag 4 shows. II . and (3) act on the PEP unit.

is interrupted for a short time. Units ( ) Fig 5.characteristicpassesa non linear clipping function. is determinedin similar to those found in the algorithmic model ( Miiller- asimil !8. Leg position is shown normal contralateral alternation is established whcn only for every 5th frame . The output of the increasing walking speed the typical change of coordination from the tetrapodto the tripod gait (Graham NL . correspondingto at different the generalwalkingvelocity. . Influence (3) is zero 4. ! ". the when the rostral leg is lifted (again this is representedin dependence on the parameter v is linear .I "-.':~ '. Fig 4 by a multiplication symbol) and dependson the position of the rostral leg when the latter is in power 3 Results stroke . The value of the position signal of the caudal leg. ":. a Walking . . ( a) The model approaches the obstacle and places the &ont legs onto it . which determinesthe AEP.a) L1 Artificial Neural '~ . If the caudal leg has ground that the position value depends on the parameter v.. " " .I.~ " " . In the contralateral form of influence ( 3) .J-. illi 1 System - : "' . ' / "... ~~~ " ."-. the contact.i"_ .s!iI":.. . - L2 L3 R1 R2 R3 " \'\~.'~ ". . The band activation function is clipped only for values in the positive passfilter ( BPF in Fig 4) which is constructedof a small range. Stability of the coordinationpattern. Two sections of superimposed frames &om a video of the legswhen the power stroke of the right middle leg film showing the model walking over an obstacle. lower part side view.. \ f . influence(2) correspondsto the bandpassfiltered retraction velocity .. With a gain factor in all power stroke units.If! :' s . The main differenceis Wilm et aI. b) L1 L2 L3 R1 R2 R3 Time Rei . 1992) this is not shownhere in detail... (a) movement Fag 6..-/. body position is shown only for contralaterallegsstart &om the sameposition. . As the movement of the legs is very Influence(4). /~'~ i:~ ) . ( b) The model leaves the obstacle. . every 15th frame ./" Nets Controlling '/~.> :'"~ ~~~. The left hind leg steps onto the obstacle whereas the right hind leg touches the ground beyond the obstacle. The contralateral influences correspond to the recurrent network consisting on six linear units (not ipsilateral influences ( 2) and ( 3) and are not shown in Fig shown) is followed by a rectifier. r . -~./". (b) Illustration of how the Upper part : top view.. ' I1 1J " . im ) is found. in a linear way ( LC in Fig 4) . .J'-~ : i . 1 j / . subsequentrectification.r manneras influence(3) .linear characteristic( NL in Fig 4) the form of which depends The model showsa proper coordinationof the legswhen logarithmicallyon a central command .. I-.. This commandalsoservesas walking speedson a horizontal plane. v. Th ~s value passes through a non .

but it shouldbe relied heavilyon resultswith stick insects starting to walk from arbitrary leg positions. In contrastto earlier successful hexapod robot ( 1986. (In our modelling of a six-leggedwalker. adaptivegait generation. To increasethe Such decentralizedmechanismshavebeen consideredby severalresearchers .position . insectpreparation. Holk Cruse . For example. This figure representstwo neurophysiologically . cat: contralaterally neighbouring legs reach their PEP CruseWarnecke1992). This that part which controls the movement of the individual inhibition resemblesmechanism1 in the present model. Different frames are system. Here the model model. These interactionswere not based gait evenwhen the movementof the legsis disturbed. for alternatingactivity stepping arises&om two subsystems coupledby reciprocalinhibition. this mechanismalone providesincompletecontrol of step phase. Pearson( im ) . The on biologicalmodels. which is not shownhere. Thus. This property was exhibitedby a later versionin which the subsumption architecture was extended to biologicalexperimentscanbe incorporatedinto a 6-legged modelwhich is able to walk at different speedsat different control interactions among independent leg controllers speedsover irregular surfaces. biologicallybasedoptimization briefly. that starts from someunnatural starting configurations. activation which could mediate reciprocal inhibition Unlike the model proposed by MUller -Wilm et al. For interleg coordination this robot used a typical insect gait. However. Knowledgeof the dimensionalplot. The figure also indicatesthat the leg inhibition depend upon complex neuron properties and glides over the ground for a short time after touch down. In this case. centers in the way that is more typical of The results show that the information obtained from animals. The model showsa stable (Maes Brooks 1990). interconnections . Pearsonandlies ( 1973).system is based on extremely simple connections. Basedon physiologicalresultsfrom an stability for difficult starting positions. described intersegmental The . Dean 1991c . and Jeffiey Dean The. but this gait was generatedby a . when the movementof the right middle leg is interrupted Current work with various. thresholds at the same time. the introduction of load feedbackmight be helpful. The connections of the state controller As discussedelsewhere(Cruse 1980. The start is noted that other animals walking under different particularly difficult when contralaterallyneighboringlegs conditions use different coordinating mechanisms begin from the same x.Wilm . studyinga non walking insect prepar:ation. work on a simple model sections of a video film . with some simplifying assumptions. coordination pattern is very stable. except for Wilson ( 1966) on the basis of behavioral studies. the biologically motivated models it contains the joint geometry and. the normal coordination methods(reinforcementlearning. describesthe movementof the joints and the tarsi. Vertebrate walkers have proved quite intractable has to step over an obstacle. working with an insect. ( 1992) . an idea proposed by the model consists of simple ~ cial neurons. etc. were learned. (crayfISh: Cruse Miil1er 1986. lamprey swimming (Grillner et aI. alternating tripod gait. whereas they normally alternate . The 3. Miil1er Cruse 1991. 1991). we pattern is to consider the behavior of the model when . Hence. Nevertheless . Moreover. interleg coordination in this movementof the tarsusis also more realistic in the sense robot did not emerge out of interactions among semiautonom that the velocityprofile is roundedat the transition points. legs were hard-wired based on biological experiments . A critical test of the stability of the coordination improved. ) is regained immediately at the end of the perturbation is testing whether these mechanismscan be further ( FigSa) . This gliding would correspondto a force deceleratingthe proposed a model of alternation in which activity of one body if the tarsi were fIXedto the substrate. biological featuresto different degrees. Uwe Miiller . geneticalgorithms. 1992b). 1989) . demonstrates neural control of walking in any animal is still quite ' that the model also maintainspostural stability exceptfor rudimentary. Fig 5b shows such a test: the normal Other approaches to modellingwalking haveincorporated coordination is regained after a very few steps. system can control walks beginning from arbitrary leg positions. joints of the leg. Brown s ( 1911) half-center hypothesis . has superimposedhereto give an impressionof the movement shownthat evensimple systemsembodyingthis reciprocal of the whole model. center is periodically interrupted by activity in a second center initiated either by peripheral sensorysignalsor by 4 Discussion intrinsic neural properties. The model proposed here represents a combined This organization has parallels in the subsumption treatment of realistic kinematics and of biologically architecture created by Brooks to step control in a motivated. The single timing center. serving for interleg coordination. the control systemdescribedhere can be used to control a real walking machine. theywere able to learn the tripod coordination . it can easily produce tripod gait but The connections which provide the coordination between requiresa gradient of natural stepfrequenciesin different .during the power stroke. is still the most applicable The height control is illustrated in Fig 6.

( 1984): Neuroethology. . 1992).125 References CrusetH .122 combinationof mech~Jlisms providesa better control of phase. Chiel and Computation4. Bioi . ( 1989): A robot that walks: emergent the tarsus is restricted to a vertiCal plane parallel to the behavior from a carefully evolved network. iD Chie H ~.generatorand its sensorycontrol. S. incorporating central and peripheral influences.C. T .e.G.P. Systems . Biol. Touretzky ( 85 and W .Dean J .49t 119. ( 1983): Neural basisof elementarybehaviorin stick insects. Proc..RiD. CrusetH . 15-21 O. Velocity control during the stance phase. ( 1986 ) : A robustlayeredcontrolsystemfor a from unfavorableleg configurations. Beer and Gallagher ( 1992) have also used genetic algorithms to success fully train intraand Beer.H.144 This work wassupportedby BM. ) . New York phase. J. the movement of Brooks. Artificial Neural Nets Controlling a Walking System legs if metachronalrhythms are to occur. Neural long axis of the body .CA Kaufmann e. 14-23 Our future studies are directed toward removing three restrictions in the present model. 54. ( 1985b) : Which parameters control the leg movement of a walking insect? ll .. 116t 343. IEEE Journalof Roboticsand Automation . Biol . ( 1986): On the definition of central pattern neural networks for behavior in adaptivedynamic ( ) movement in walking arthropods ? Trends in Utruu ' t ~ io of inform for moto con. The reader is referred to Muller . 84B.Heuer . which is necessaryfor examplein controlling starts Brooks.U . we did not consider the mechanical 308-319 coupling of the legs in power stroke through the substrate and the resulting effects when the legs attempt to move Camhi.. The start of the swing Bissler. Second.355 Bassier. Sinauer Ass.H.J. RA-2. Computation1. Nevertheless . pp-e ( 43 7.21S-262 p..A . Beer.exp. J.U.Ff grantno.. Quinn. (1976) : On the functionof the legsin the &ee relatively simple legs. i.1 :99 Bassier. Springer. Miiller.Physiol bilaterally symmetrical manner. H .. all robots mentioned above use Cruse. ( ) Heterogeno Cruse. Sunderland.Schm R . S.Corn .Cybem. our model does not take into account the dynamics of the Brown. The connections between the different legs . ll . 253-262 this does not permit turns to be simulated. betw Relati act Berli Spring . ( 1990) : What mechanismscoordinate leg Cruse H . possessing two degrees of freedom walking stick insect Carausius morosus . Third . We hope that a leg geometry more similar to that evolved in animals will provide greater Cruset H .Bio .Prinz Neumann . In . JiM. ( 1980) : A quantitative model of walking flexibility .San MorganMateo . ( 1992) for a more detailed discussion of these restrictions .l12. (1977 ) : Sensorycontrol of leg movementin the stick insectCarausiusmorosus. the forces needed to perform the described progressionin the mammal. Neural robot contrDI systems ( Donner 1984.2S. sensory Cybern. BioI. Fourth.RA .In :Advances in Neural Information .U.RiD. . 01 IN 104 B/ 1. ( 1985a) : Which parameters control the leg movement of the walking insect.R . 61-72 CrusetH .exp. Besides being somewhat artificial . J L. . Larsson .U. Processing . Massachusetts Wilm et aI. 65-69 environments .. of the Royal Sac.H. Fust.Cybem . Our results show that a 1.H. . movements. and fixed to the body in an orthogonal and J. 91. 356-365 coworkers 1989. I . Acknowledgements Cybem . Adaptive Behavior gait is the gait of choice. Heidelberg. ( 1983) : The influence of load and leg amputation upon coordination in walking crustaceans : A model calculation .Biol . GalIagher.RiD ) : Two couplingmechanisms whichdeterminethecoordination of ipsilaterallegsin the . 1 :989 Sterling . 37tl37. ( 1992) : A distributed neural network this type of inhibition has been incorporatedinto several architecture for hexapod robot locomotion. mobilerobot. Neurosciences 1D. ) - d 577 5. Chiel. ( 1911): The intrinsic factors in the act of system. (1986 Cruse Beer. perceptio 9ds .RA.. with different velocities. I Dc. ( 1992): Evolving dynamical interleg coordination under conditionswhere tripod neural networksfor adaptivebehavior. J. 116t 357-362 Beer . 13. Berlin.

Physiol. Carausius morosus. ( 1968 ) : Ein Analogmodell del systems . ( 1990): A recurrent network: limitations and Dean. J.S. J.81. V . Cruse. ( 1991a 802 coordinationof the stick insectCarausiusmorosus. PI Weiland. K.F. Comparisonof different during walking in the cockroach. ( 1990): Learning to coordinate behaviors.U . Cybem. Proc.L. Carausiusmorosus.exp. Biol.U .J.Rev. Danowski. and Behavior. Brodin . 3-20.J. Neural Networks3. Wallen.U. 147-156 Levin. In: A . 121. G .MiD .122 Grillner . ( im ) : Steppingmovementsmadeby jumping spidersduring turns mediatedby lateral eyes. CaJ:negie- Mellon University.. A . 64.) Model Neural Networks Beinbewegungen cines laufendenInsekts.H. Carausiusmorosus.. PhD thesis. 449--471 Bioi. expo . 155. 66.J.Physiol. Ent.exp.H.comp. J.F.J.HJ .725-744 forms of coordinatingmechanisms . 393-402 Experimentalresults. ( 1984): Control of leg protraction in the stick training. Biol.BioL Cruse. I . ( 1990): . Biol.Proceed Donner. (1985 . J. Warnecke. ( 1966): Insect walking. Local control . Brooks. 23-52 WilsonD . ( im ) : Central programming and reflex insect. E.Cybern. PlenumPress .. insect as an example of a 6. ( 1992a Pearson . FJ . on RoboticsandAutomation -1463 systemsand real time systems . Simon and Minski. Wilm . M. Bioi. ( l992b) : A model of leg coordinationin the stick underlyingintersegmentalco-ordination of leg movements insect. 335-343 Pearson.. 33-46 ) : A model of leg coordinationin the stick Dean. 58. 796- ) : Effect of load on leg movementand step Dean. ( 1991c) : A model of leg coordinationin the stick Miiller -Wilm .F.J.. 403. Bioi.411 Adaptive Behavior 1. A. Koch. J. J. IV .Cybem.. NewYork Schuster Dean..M.. Conf.J.we Miiller . Pfeiffer. GrahamD.M. (1985): Tritonia swimming: a model system for integration within rhythmic motor Wendier . Bioi. J. and Jeffrey Dean walking crayfish. Ann. 68-74. .J.legged walking system. Cruse.Biol.Neurosc. 15-40 a slow-walking cat. 11. Rev. Dekin. 429-436 Dean. G. MiS. Dean.H . 89. ( 1973): Nervous mechanisms Dean. insect. 345- 355 Pfeiffer. AI (AAAI -90) . 349-369 Land. (1987 ) : Sensoryfeedbackduring activemovements of stickinsects. J.P. New York. Brain Res. 137-156 Getting. Cybem. 66. A geometrical~ nsideration Miiller . Corn.. II .I .PA . Pittsburgh. pp.exp.. betweentwo adjacentlegs. ( 1992): Kinematic model of a stick model and simulationof normal step pattern.. 159 ) : The Societyof Mind. H. Weidemann. G. 64. 64. 641-650 insect: a targeted movement showing compensationfor externallyapplied forces.exp.H . ( 1984): Control of walking. 1458 IEEE Int. of the 1990 Dynamicsof thewalkingstickinsect.T. Holk Cruse . 103. Lansner... Oldenbourg. Cybem. Descriptionof the kinematic Eltze. I. 771-781 Maes. London In : Kybernetik 1968. Selverston(ed.. lies.173.comp. ( 1991) : Neuronal network generating locomotor behavior in lamprey: circuitry transmitters membraneproperties and simulation.RA . Weidemann. ( 1991b): A model of leg coordinationin the stick insect. Eighth Natl. 169-199. J. Responsesto perturbations control of walking in the cockroach. Ann.193 of normal coordination.exp.Biol. Exp..P. Carausiusmorosus. 133. ( 1991): The contralateralcoordination of contralateral and ipsilateral coordination mechanisms of walking legs in the crayfish Astacus leptodadyius. 14. m . J.Biol. KG .56. ( im ) : A behaviouralanalysisof the temporal MUnchen organisationof walking movementsin the 1st instar and adult stick insect. Beihefte zu "elektronischen " Anlagen 18.. ( 1992): Coordinationof the legsof 57.P.

inspired networks that generate task . and iii ) organization of mechanism. The simplifications made in vertebro tes and in vertebro tes alike. networks of model neurons generate simple behaviors in a small . and the constraints . Empirical studies are needed to more multiple behaviors. neuroscientific models. Similar may be unreasonable for real world applications . Although mathematical models relative and computer simulations have increased our understanding rotation of frames for turning . Colorado 80523 USA Telephone : (303) 491-6618 Fax : (303) 491-1055 . and building real world machines. but they may only describe the activity of a mechanism rather than account for the actual of artificial neurons . robotic system . how cyclic inhibition . 's work on abstract vehicles 4 Brooks ' s in the real world . We drow upon ethologi. Simulations introduce computational constraints Neurol elements within the centrol nervous that neither the actual nor a constructed system would system generate oscillatory activity patterns face. ii ) properties of individual behaviors. mechanism under study or are impractical to carry out . Different behaviors of how model neural networks might generate were developed independently and integrated behavior . Other networks that are modulated by sensory inputs control the local framework . leggedrobot .lance. closely match the formal world to the natural world . In particular . a useful testing ground . . we neurons in order to control the locomotion advocate the construction of robots that operate in rhythm of the Micro . multiple behaviors are integrated within a common walking in the Micro . A Neural Network Based Behavior Hierarchy for Locomotion Control Sunil Cherian and Wade o . we focus on designing biologically .achieving activity patterns and mapping the 1 Introduction model to a simple . Braitenberg [ ]. Multiple behaviors the real world . Theoretical models are amenable to the Micro . translation of the frames . . and how behaviors adapt to satisfy environmental . cal concepts. biologically studies . it dr ~W8 together many of their advantages to provide sensitive optimization . Each approach has its own advantages and inspired. We are interested in understanding how networks of A central pattern generator ( CPG ) .Rover robot . This approach eliminates some of the are integroted within a behavior hieror - disadvantages inherent in the other approach es while chy Behavior adaptation is viewed as environment .Rover. conducting physiological and anatomical This paper describes how simple . actuation of legs. forms the basis for controlling . Simulated worlds lack the richness and uncertainty that forms a basis for rhythmic movement of the natural world . Interactions oscillatory networks are built with model are often contrived and limited .Rover. Here . Thoxell Department of Mechanical Engineering Colorado State University Fort Collins . we seek a fuller understanding of real within a behavior hierarchy derived from the instinct world control systems ~y building machines that operate models proposed by Tinbergen and Baerends. based on recurrent neuron like elements generate specific behaviors . . Behaviors are investigated formal verification practices and may prove consistent at three levels of system organization : i) networks in themselves. computer simulations . Email : wade@longs. and recent Empirical studies often result in the destruction of the AI techniques to guide system design.colostate Abstract Understanding can come through devising theoretical models .

The neural basis of behaviors has been studied class. many parallel pathways . and introduce a simple of pools of motor neurons. creates a positive more fault tolerant . Section 6 discusses generation of distinct behavior patterns for their varying temporal dynamics of neurons as a source for network levels of activity enables the same sensory and m0- plasticity and we conclude the paper in Section 7.. They exhibit both autonomous rhythmic Pattern generators within the central nervous system. tor neurons to participate in the generation of more than one behavior [ 10] . Several types of network oscillators have previous stage evokes the following stage. These studies present strong evidence that neurons linked by mutually excitatory connections. The inactive phase occurs once a critical impulse frequency is reached. movements A simple network realization is given in . 1] . A restorative mechanism example . For . a network oscillator with recurrent . The receptive fields of many In Section 3 we discuss the notion of neurocontrol . the spinal cord contains central pattern generators for The network generates a two. as opposed to a stage- . Afferent neurons diverge to several interneurons of network oscillators and focus on the multiphasic . 2 Biological Locomotion 2. The ratio of sensory a hexapod has been reported in [2] and [7] .e. burst . and are readily extensible . have shown that the isolated spinal cord . The interconnections between model for the neurocontrol of locomotion . The active phase occurs when the net gain of the Neural networks distribute control . In the first and are subject to modulation on a cycle. Sunil Chel'ian and WadeO. An inhibitory cell that becomes activated . extensively in invertebrates since their neural activity These neurons appear to generate chronic rhythms . we outline some of the basic features . including self others [ 11. network Figure la .Excitatory Networks a motor pattern with a similar phase lag. and analysis increases in difficulty .inspiration and motivation for this project . reciprocal been proposed. rhythmic activity such as locomotion . local interneurons .1 Self . Models of central nervous .1 Network Oscillators Animals move continuously . making a system neuronal connections is positive . Invertebrate larization levels while their interconnections give rise locomotion appears to result from the endogenous activity to ~ cillatory properties that produce network oscillations of their neural networks . Section 5 discusses the integration of multiple behaviors within a unified framework . By changing the nature of the connections between each gradually increasing membrane depolarization elements or by adding elements . 19.excitatory networks contain a minimum of two swimmill . The latter are examples of episodic movement movement in animals . i . known as central pattern generators or central nervous movement such as breathing and heartbeat and voluntary oscillators . Grillner and . individual neurons have special oscillatory properties that produce endogenous polarization rhythms . such as vertebrates . In can be recorded with relative ease during the performance the second class.phase oscillatory cycle.1 .stage procedure where peripheral feedback from a movement . As the numbers of neurons increase . . Troxell work on architectures for mobile robots [6] . feedback loop . cyclic inhibition characteristics ( described below ) appears to terminates impulse production and repolarizes govern swimming in the leech [20] . Smither ' s work on behavior . and recurrent cyclic inhibition existence of central pattern generators in vertebrates as well . and cycle duration as the intact lamprey swimming in a Self.excitatory networks . More complex animals the network cells. much of their movement is rhythmic . and motor neurons an implementation scheme for the controller in Section make up local networks which underlie reflexive behaviors . a network acquires and producing impulses at progressively higher frequencies additional properties such as polyphasic activity . are responsible for generating rhythmic rhythmic movement such as swimming or walking . Network ~ cillators appear to govern episodic cycle basis . 10. Interneurons play an important role in the 4. indicating 2 contains a brief overview of the biological basis of considerable interneuronal convergence and integration rhythmic movement . deprived of all movement related sensory information . interneurons converge and contribute to the activation review some neural models . 12] have presented strong evidence for the inhibition networks . Each cell then drives the other . systematically distributing sensory signals over properties of recurrent cyclic inhibition networks . Recent feedback for such parameters as touch and strain and work investigating the neural basis of locomotion in can thus modulate motor activity . have many times more neurons may serve as a possible impulse termination than the leech and perform more varied mechanism. Studies conducted on fish such as lamprey networks [20] .based robotics [ 17] serve During locomotion proprioceptors provide sensory as. oscillators are divided into two classes. We present the afferents . neurons have endogenously stable po- of complex behaviors [8 . can produce 2 . Section neurons to motor neurons is typically high .

self. Under these circumstances . Rings containing more than three cells can form 2 . a tonically active cell excites inhibitory contact from two cells. can ' generate N phases. m~ t recent inhibitory synaptic event. Once A recovers. AA 2 . The N . the level of tonic A and enabling C to enter its recovery phase. inhibitory ring with an odd number of neurons . An ~ cillatory cycle tonically active. When activity .1 .VE)] neurons form an inhibitory ring by each cell making an inhibitory contact with one cell and receiving an Recall that the cells in cyclic inhibition networks are inhibitory contact from another . reciprocal inhibition network is depicted in Figure other cells through inhibitory connections and receive lb .2 Reciprocally Inhibitory Networks topologically more complex networks . B is hyperpolarized . Increases(or decreases ) in VE. Recovery time is related to characteristics of A hysteretic process is not necessary in networks having the cell membrane .1. in Figure Id . is large compared to the time for establishing inhibition . generating initiation zone having an input resistancePi. A simple network realization is given in Figure lc .1 .. is inactivated . The pair will alternate in the active phase. it begins firing and ization. R . R .. b Multiphasic rhythms enables polyphasic motor routines such as those required in locomotion . Suppose cell A is a pair of reciprocally inhibited cells.numbered ring cannot oscillate for no cells can be ih the recovery phase while the ring Figure 1: Oscillatory networks assumes one of two stable states: when either all the even. C and D . or adaptation . Continuing in this manner . Because the connection Ci . B is no longer inhibited would depolarizethe cell to a steady state potential of and begins recovery . the network generates a four . Following the is inhibitory . B . it inhibits A and D . tonic excitation inhibits C . thus shortens(or lengthens) the value of R recovery marks the start of a new cycle . say C in Figure lc . inhibiting VE. enabling C to enter hysteretic process may involve postinhibitory rebound . When C ceasesfiring . Recurrent cyclic inhibitionwhen the cell membrane acts as a passive element in networks require a minimum of three neurons response to voltage transients and is given by to generate oscillatory activity . where past inhibition results in a transient reduction recovery. and thus A can recover from potential is VI . B recovers. N . A each of the four tonically excited neurons contact two basic. the membrane i . Changes in the neuron recovery 2 . its postsynaptic cell . network is capable of producing triphasic rhythms .numbered cells are active . can also form ~ cillatory rhythms . correspond to changes in the oscillatory cycle period . also makes them useful candidates for generating rhythmic movements. For example . i . impulse bursts if the inhibitory effect of one are inhibited and thus inactive .numbered rings can also generate 0s- Networks that are reciprocally inhibitory . A Many rhythmic movementsshow variable phaserelations closer look at the properties of recurrent cyclic inhibi . Hence the and consequentlythe period P of the rhythm . and threshold potential VT.excitatory .e. If the recovery time . In this example . The cycle . B . Synaptic inhibition preventsdepolar- past inhibition . The cell membranehas an impulse begins with a cell . allowing their postsynaptic cell on the other decreases as a consequence of past cell . neuron 's impulse frequency declines in response to a Another property of recurrent cyclic inhibition networks constant excitation .e. even.4 Multiphasic Rhythm Generators . Three tonically excited R = piCiln[(Vi. In the absenceof inhibition . to enter its recovery phase. On recovery.3 Recurrent Cyclic Inhibition time .VE)(VT. where a of4R . Recurrent cyclic inhibition networks consisting of a simple . C ' s excitation. rather than cillatory activity patterns . B fires . even. the A cycle period is P = N R . Mechanisms producing such a restorative . Neural Network Beha\' ior Hierarchy ' Locomotion CO1 Contl "oJ tion networks reveals the advantages of using them as a model for a multiphasic rhythm generator . A simple . capacitance impulses .numbered or odd . among their component features. it is depolarized . its postsynaptic cells.phase activity pattern with a cycle period of the impulse threshold level .1 cells of the N - membered ring form a sequence of alternating active and inactive phases while the remaining cell is in recovery c d . A value for R can be formulated recurrent cyclic inhibition .

even though characteristic response times . are found at the other . Desired output patterns are encoded in the connections an appropriate level of abstraction . . 6. neuron j to neuron i is represented by w. In the following .Pitts neurons The response of a neural network depends upon the has been shown to be capable of universal computation properties of the individual neuron . Neural models span the entire Here. onset of an impulse train among the cells is invariant . although Til is functionally separated into two components . network is favored over connection specificity . The weighted sum of inputs must 3 Neurocontrol reach or exceed the threshold for the neuron to fire . however. Rhythmic ideal candidates for autonomous control purposes [3] . Sunil Cherian and Wade O. Wijnj (2) remain fixed . attempt to model the electrophysiological properties Neurobiologists are more interested in modeling the of real neurons in order to better understand neuron actual biophysical dynamics of real neurons . in part dynamics . we present networks that are tailored determines the time required for the network to settle for achieving specific tasks rather than generic into a particular pattern . Each connection not concerned with computation per Be. Mathematicians and computer m~ t real neurons are influenced by their recent history scientists are primarily concerned with computation and integrate their inputs over some period . In this paper . Troxell of the underlying rhythm generator appears to be divided3. with one time unit elapsing per processing anatomical structure and the physiological characteristics step .specific parameter for neuron i called its threshold value. and the connectivity of the network in neural computation centers around exploiting the .1 Neural Models between a variable time block and a constant time block . Time t to much more complex models that are based on the is discrete . Varying the recovery time in the above networks varies the length of the period . The two components have different is _it constrained by biological plausibility . of the synapse connecting of actual biological networks . and represents the state of spectrum from simple threshold computational units the neuron i as firing or not firing respectively .Huxley equations being a good to deduce the function of biological networks and to example [14] . or weight . In this model high interconnectivity of the introduced . but the phase relations Ri(t + 1) sgn (t) . and information processing in artificial networks composed At the other end of the spectrum neurobiologists of threshold units .j . A lot of effort synaptic connections .9. Some of the more common neural models for the transition from one state to the next . The above recurrent cyclic inhibition networksA simple model proposed by McCulloch and Pitts abstracts maintain phase. is a cell. The synaptic connections many of its metaphors are drawn from the biological Ti~ act on the shorter of the two time scales and world . . The Tit component acts on networks with high connectivity that somehow ' learn ' a much longer time scale than Ti~ and sets the time those tasks. we turn to neural models. A synchronous assembly of McCulloch . These equations are generally quite complex bridge the gap between neural functioning and higher and difficult to analyze . Klienfield and Sompolinsky have developed an ass0- Somewhere in between these camps falls neurocontrol ciative network model for the generation of multiple . properties of for suitably chosen weights w. the properties of individual neurons sgn ( z ) 0 otherwise also constrain network activity .constancy. output emerges as a collective property of the network Successful neural models for control purposes require . denoted some form of computation is obviously involved . the control of real machines by networks of artificialcoherent patterns by a highly interconnected network neurons. However. the corresponding the neuron to a binary threshold unit [ 13] .g . Many math . (~ ) where Network connectivity gives rise to multiphasic activity 1 ifz ~ O ' . The robustness and generality of such [ 16] . is either 1 or 0 . Adding a constant time delay element to the network disrupts phase relations . . the Hodgkin . nor ~ 1 and T5 . e. Neurocontrolis between the model neurons. Network output consists of temporally coherent networks along with their ability to ' learn ' makes them patterns of linear or cyclic sequencesof states. Til is are briefly presented below and a simple model that defined in terms of a formal version of Hebb ' s learning seems suitable for the neurocontrol of locomotion is rule . One of the drawbacks depends upon the underlying motivation . The strength . n . The choice of an appropriate network model notion of universal computation . level behaviors [9] . . of this approach is that the previous behavior ematicians and computer scientists are at one end of of a unit can only affect its current behavior by influencing a network model spectrum while m~ t neurobiologists its synaptic weights during training ..

walker locomotion concept is retained in the Micro. locomotion in the Micro . t ) tanh(~Wijnj(t).wise activities related to the Space Exploration Initiative .t ) is to be used as ( i ) a testbed to evaluate the performance x 1. The beam. The prototype has a total of ten joints : eight leg symmetric . depending upon whether its dynamic activity Figure 2: Micro. Each base has four legS.1 network activity .Rover . one beam drive prismatic joint between requirements placed on the network . legged robotic system. Two key synaptic properties are modeled that effect 4 . the frames .extended frame along the beam. Robot mobility modulated by sensory feedback. The sign of a: synaptic connection determines Micro . Delayed excitation captures some of the temporal dynamics of real neurons and is one way of generating rhythmic activity within networks . actuation . the network connectivity allows for to a new location .phasic oscillator .Rover prototype is a miniature version of the Rover prototype developed by Martin Ma- sqsh( nj .Oi] (3) potential of very small robotic systems and ( ii ) ( ) a demonstration and exhibition tool for walking machine where sgn(z) { I 0 z if othe>0 ' . they are constrained by functionaldrive joints .exprlt(ti )/ T'i . allowing a frame to stand and support the with recurrent cyclic inhibition generates the desired robot .2 Dynamic Neuron Model We develop "asimple neuron model incorporating three major properties : delayed excitation .linear squashing function is a hyperbolic tangent that keeps neural activity constrained " between + 1 and . are therefore structurally specific and carry out a predefined function . rectangular The strength of a connection determines what portion base frames that are attached to each other by a revolute of presynaptic activity is propagated to the postsynaptic joint .linear function of the summed inputs to the neuron . squashing. rietta Astronautics Group in 1989. predefined world rst (t ) { t otherwise map nor on accurate guidance systems. The Micro .Rover Platform whether its action is excitatory or inhibitory . The local networks arises from sequencing the standing . the areas of control .Rover .moving action of the two frames. and thresholding . However . The beam walker design consists of two . however.examined under new design concepts . The non . Connections are not required to be .1. The turn drive rotates the beam decentralized control .Rover the frames.Rover is given in Figure 2. The approach does not rely on any detailed . effective operation depends on environmental contingencies such "asavoiding obstacles.Rover Robot Testbed A very small . allowing the frame to move rhythm pattern . The following equation models the neuron . one at each corner neuron . . Delayed excitation is manifested as a delay between the onset of a depolarizing stimulus and the first spike . The output of the neuron is all or " none . () tx -1<0 are re. The leg drives lower the feet of the beam robot requires a four . The outer base of the Micro . Instead . The maximum activity level is computed as a non. or retract the feet . exponentially approaching the maximum activity level . Neural Network Behavior Hierarchy for LocomotionControl 3. A network walker . A design schematic of the Micro. the Micro. ni (t + 1) = sun[sqsh(nj . and sensing and 0ifsqsh(nj sqsh (njt . The de- polarizing stimulus increases activity within the model neuron .Rover is approximately . The model follows from biology that connects the two frames while the beam drive in that the central pattern generator ( CPG ) determinestranslates the leg.Rover is above or below the threshold value. and one turn drive rotary joint between As described below .retracted frame with respect to the the overall rhythm while local movements are leg. 4 Micro .

a leg control network . . a turn control network . At is reversed for movement of the outer frame . the Micro . This sensory neuron inhibits the .inner. ' Rhythmic output from the CPG activates control neu- ' roceptive or proprioceptive stimuli . At easily interchangeablepower packs. canoperatefor at leastan hour.. The dynamic response does not exceed35 cm in height. ~ .legs .. higher command centers can turn neuron 5 on or off as desired.1 5 4 . The walking of the five neurons in the recurrent inhibitory ring is stride is approximately 25 cm. The order connections are either excitatory or inhibitory . Figure 4: Dynamic responseof the CPG. Sensory neurons are activated by exte. Figure 5 present " . 55 cm wide by 70 cm long and the inner base20 cm a cycle is determined by the time constant associated wide by 30 cm long. a network that to extending the appropriate leg .frame three categories: sensory neurons . and has four . They may movement of the inner frame . Inaddition network . its step a four . . As soon as the extended The pattern generator shown in Figure 3 generates ' ' leg is firmly placed on the ground .outer. connect rons of appropriate local networks . when neuron 5 stops firing . The feet are durable shown in Figure 4. several networks that carry out predefined tasks locally The forward locomotion rhythm of the Micro .independent networks : the central pattern generator outer legs at each corner .Rover . . Neuron 5 becomes tonically active and capableof support in soft soils. an inner leg and an outer leg. ~ ~ Figure 3: CPG for Micro.. the outer legs support also connect to other sensory or inter neurons. the MicroRover with its presynaptic neuron .neurons that excite the network .Rover locomotion . Each corner of the to either inter . The control methodology is homogeneous in the consists of the actuation sequence: sense that the entire controller is built up of neural elements alone. During this simulation neuron 5 had to be maintained The entire robot weighsless than 10 kilograms. an obstacle avoidance neuron also inhibits the inter . Sunil Cherian and . Arbitrary time constants were assigned 4 .Rover locomotion. a frame control its corresponding ' extend ' motor neuron . and a pseudorandom wander network . stops . in turn .Rover locomotion consists of required for Micro . ' ' opposing retract motor neuron . within a complete behavior hierarchy to a control tether. With its legsretracted. an active motor keeps the two frames aligned . receive inputs from and pass their outputs to Only one leg at each corner is needed for supporting other neurons. They .phasic rhythm with a constant phase relationship sensor is activated . oscillator activity used are "off the shelr' and commercially available. A leg is extended by exciting for gait control .. The neural elements are divided into outer-legs .inner-frame . motor neurons .phasic pattern is mainiained by the network .. terminating further extension of .neurons. . - . For translational Forward are the mechanical equivalent of muscles. Motor neurons excite actuators that the rover at any given time .this simulation trace : these time constants do not correspond to actual values The controller for Micro . Implementation to the neurons in order to obtain . 3.. . After start up transients die out . The powerplant is at t = 1.Rover has two legs.Rover controller consists of seven shows the network that locally controls the inner and semi . The components t = 8 . -2 .neurons or motor neurons. Interneurons Micro. The period for which each neuron is active during active motor neuron . Synaptic the rover and the inner legs are retracted . a stable self-contained.2 Locomotion . However. Troxell - . The tonically active for oscillatory behavior of the robot can operate either autonomouslyor connected network . and inter . < 1 Wade O.

Thrning is continued as the inner or the outer frame .. b II . - - . . - T ) " 7T rr . . J - ~ = dQ G . ccw O ~ ~ J~ ~ ! \ ~ A cr . ' ' ' ' by inhibiting the align and walk networks and extending . are no longer inhibited . DT . if oCW / ' \ . . the outer frame . - O In D - J ~ ( ~ . WALK 1 DU8A n ~ .4 WALK { g ] rnD : Og ( : ! ) r : V n JaN C C W - . - inn ~ oor . . . As soon as the comes from possibly higher network centers and results turn command neuron is disabled . .Rover with respect to the outer one. framesin thedirectionof travel. l TU R Nci W . - . .Rover can now exhibit simple behaviors .. . . the leg . A schematic representation of the turn network responsible for retracting the opposite leg. . Y - . . / .. G ( Vb 72 G . A . . - = . . . G ~ . I .. Sensory inputs that provide information about it turns the inner frame to bring it into alignment with translation limits modulate frame actuation . Once the inner legs contact the other results in the overall forward or backward movementground . . is shown in Figure 7. The ' left .-- ~ ID . Sensory input control turning . q . X -- Dl RE Cn ON RIGHT = ~ . q . The network receives tor neurons are activated depending upon the state of input frorri the CPG two times every cycle for moving the direction control neuron . . q { ~ Q ~ ] G ( V . A command neuron initiates turning therefore modulates leg movement .: ~ -. . The step sensory neuron also excites the interneuron ' ' turning . The ' reverse' command ' ' long as the duration neuron is active .Rover . the Micro. Inputs to three neurons thereby preparing for frame translation . - " " / CM .. q ... - . Figure 6 depicts the network the leg control network .' turn m0- that controls frame movement . .. . The network is shown in Figure 8. ..' or ' right . - i:.. . Figure 6: Local network for forward and reverse Figure 8: Network that aligns the :inner and outer frame translation. Since the two frames are not ' ' thus reversing the direction of locomotion in the MicroRoveraligned after turning the align network is excited and . The rotary joint between the two frames facilitate The Micro.: nlRN . \ l " " " " Figure 5: Leg control network that modulates leg Figure 7: Network for turning the inner frame of actuation based upon local sensory input . The translation of one frame with respect to the the inner legs. Neural Nehvork Behavior Hiel "al "chy CoI" Locomotion Contl "ol AUC ~ N C : It C M N I P ~ UGs ' CCJNTACT T . ' align ' and ' walk ' ' ' in the reversal of the frame translation direction . the outer legs are automatically retracted by of the Micro .

The wander network then inhibits ' walk ' and changes its heading . ' ' key . their causation . . releasing and di ~ cting impulses of internal Braitenberg s vehicles. . a synthesis component in the robot equivalent of etho. we also investigate mechanisms seen as a lock that can be opened only with the right for integrating these diverse behaviors within acom . The stimulus combination that activates aparticular behavior is explicitly shown in the form of a . Finite machine resources are shared by several behaviors to develop models of behavior organization while neu. Troxell such as walking and turning . and which behavior that consists of a network with neurons with ' ' responds to these impulses by coordinated large time constants excited while walk is active . Since we are an appetitive behavior is activated that seeks to find interested in building machines with ' insect level ' competence stimuli that releases one of the behaviors . Schnepf [ 18] preSents a tentative in modern ethological theories . 4. a single behavior is activated of animal behavior in order to study their causation . In the case of artificial autonomous systems however 3. appears that ethological models of simple animal are released and the process is repeated . the notion of instinct is widely used in ethology . whereas 1. This necessitates releasing mechanism. ' wander command neuron and exciting the network at Tinbergen s instinct model . consisting of the proper stimulus configuration . prior to the description aind analysis of behavior they have to be designed and built . It clearly accounts for situations where the same statements about the causation of behavior can be behavior pattern is present as a component of several made only after careful manipulation of the behavior other behaviors : instead of being duplicated . of several behavior patterns that are components of the next higher level of organization . the next lower centers it . belong to the lowest level where its datapath . then for the organization of robot behaviors . When appropriate . and the organization of th ~ e motor resources roethology tries to discern the neural basis of animal are accounted for within the model . it appears to be framework for research into autonomous intelligentquite useful as a conceptual model for describing the systems based on the study of behavioral sequences numerous regularities observed in animal behavior and in robot beings . This block can be removed by appropriate stimuli or releasers. by higher level behaviors as needed. A behavior . a very simple . Sunil Cherian and Wade O. . the aim of which is to predict future behavior ' block ' prevents the expression of the behavior until and to justify current behavior . behavior might serve as a useful yardstick for analyzing Although Tinbergen 's instinct model is out of favor robot behaviors . of course desired in robot behavior . It movements that contribute to the maintenance also receives inputs from pseudorandom devices corresponding " to environmental influences . hierarchically organized nenJous mechanism avoiding obstacles along the way. Experimental ethology involves the manipulation in several behaviors . The organizing principle for animal behavior involves network extension can enable the rover to avoid the concept of ' instinct ' . Each behavior patterns . This study is carried a proper stimulus configuration making up the ' key out at different levels of system organization ranging stimuli ' is detected . to appropriate ' turn ' command neurons form areflexive Tinbergen (quoted in [ 15]) defines the term instinct as pathway that allows the rover to continue locomotion fl. and their hierarchical organization receives external and / or internal stimuli . Ethology provides several explanatory called the ' key stimuli ' . distinguish es between sev- ' ' erallevels of instincts each of which comprises a subset apparently random intervals . In addition . Some of the desirable features of from the cellular level to societies of animals . Experimental ethology and descriptive ethology strive 2. Although controversial obstacles. We are also testing a wander as well as external origin . The prevailing key stimuli determines models for the organization of animal behavior which lower behavior will be released.Rover. This type which is susceptible to certain priming of reflexive pathway is similar to those suggested in ' ' ' . with little or no apparent cognitive capabilities stimuli is encountered . saturating the of the individual and the species. If all which appears to be useful preliminary abstractions blocks remain in place due to inadequate stimuli . Layering of behaviors is well structured : behaviors logical investigation . The study of animal behavior is primarily concerned Figure 9 shows a schematic representation of the with the sequence and temporal structure of behavior behavior hierarchy for the Micro . mon framework . Direct connections from obstacle detectors . An ' excitatory po- tential ' flows from a higher center to the next lower 5 Behavior Organization one and is normally inhibited by a ' block ' . In addition to building independent behaviors that A ' block ' also called a ' releasing mechanism ' . Description the model are: of the sequence and temporal organization of behavior patterns can be made observation ally . can be achieve different tasks . behavior .

Predefined the dynamic properties of the neural units . . We are currently designing a simple reward / punish scheme for generating optimal time constants for the neurons that constitute the network oscillator . is a biological mechanism for varying the period P of the network rhythm . It takes into account the constraints that must be met for optimal network performance . Figure 9: Behavior hierarchy that integrates the different local behaviors. controller . independently and tested numerically . ( ) incorporation of static and temporal dynamics i within the neuron model . behaviors at the same locomotion rhythm . and ( iii ) maximize stride length . ( iii ) exploiting oscillatory properties 6 Behavior of networks . ( ii ) minimize variation in mean ground clearance (This accounts for the shortening of effective leg height if the leg extension phase is too short ) .Rover . the active period of each phase of the four . The probabilistic rhythm modulator is embedded within the controller as an integral Acn I A T C8 I ' ' part of the walk behavior .phasic network with recurrent inhibition was found to be suitable for generating the 6. we expect it evolve through future empirical evaluation . In the case of the Micro . ( iv ) investigating the structure of individual Adaptation behaviors as well as collections of behaviors . Some of the constraints currently considered are: ( i ) minimize dead. 7 Conclusions This paper presents a neural controller for an autonomous is not required to project to higher levels. A four . For optimal locomotion. 5. and Terrain conditions constrain Micro. The pattern . The recovery time R described in Section 2. through further experimentation .4 .optimal performance .phasic network oscillator depends on the time dynamics of the corresponding presynaptic neuron .Rover locomotion ( v ) investigating behavior adaptation as an integral rhythm by placing lower bounds on the time required part of task. This may have direct significance sensoryinformation about the terrain being traversed. Future research is aimed at investigating the modulation of the central rhythm .Rover.time between the termination of leg or frame actuation during one phase and the initiation of the next phase. Recentsensoryevolution of the structured networks once the assumption history is utilized for probabilistic predictionsthat of connection specificity is relaxed . walking machine called the Micro. the time phase relationships are more important than the for which eachphaseis active has to be modulated by recall of multiple patterns . or vertically on higher levels incorporated within our neuron model . Mutual inhibition between. for leg and frame actuation. Neural Network Behavior Hierarchy for Locomotion Contl "oJ lead to near. Work The generality of this model remains to be determined currently underway to implement the controller on is the Micro.Rover. ( ii ) utilizing structural specificity of networks . Sensory modulation of the locomotion level within the same functional group makes sure rhythm takes place within local networks that that only one behavior is active . New behaviors are added either horizontally on Static and dynamic properties of real neurons were a particular level . for the neural coordination of robot motor The biological solution is often in the form of probabilistic activity .ise out of optimality considerations. Since this is Factors motivating the current implementation are: only a preliminary model . This type of learning cycleresultsin sub-optimal performancewhennew terrain is advantageous when temporal rhythms with constant is encountered. rather periods for eachphaseof the network activity than just their synaptic strengths . thus avoiding coordinate leg and frame actuation .1.achieving behaviors . Constraints on upper Simple algorithms are being developed that can alter bounds &r. Adding new behaviors does not require taking generation capability of networks of biological neurons apart behaviors already in place much like were studied in order to choose an appropriate gait Brooks ' subsumption architecture [5] . Networks were developed conflict situations .

. [3] RandallD .. [ 17] Ulrich Nehmzowand Tim Smithen . Psychology.. In RichardM. editor .. Christopher Miall . and Richard G. Chiel. Huxely. [ ] an4 Hillel J. Stein. . editor . editors. MA . Selverston.Arcady Meyer and Stewart W . Arcady Meyer and Stewart W. The Computing Neuron. volume [7] Hillel J. Davis. F. the Fir . Hodgkin and A. pages465. Wilson. pages195. 1985. From Animals to Animats: Proceedings 19 Allen I .Rover robotic project has been ton .. Peter Wallen..443. [8] William J. . Method. Advances in Behavioral Biology. MIT Press. editors. Designing Autonomous Systems In Jean. pages265. In Richard M. Troxell Acknowledgements [ 10] Peter A . Hillel J. ing self. Beer. The Metaphorical Brain . In Jean. A robust layered control system [ 18] Uwe Schnepf. Sunil Cherian and Wade O.. Self- - . and the years. of and Back.Tern.. Mapbuildingus - " A biological perspectiveon autonomousagent design. [ 15] Klaus Immelmann.194. Anden Krogh. In Jean- Automation . Networks and Beyond. Introduction to the Theory oj Neural Computation their assistanceand insight are much appreciated.169. In Avis H. 1987. pages434. editon . Sten Grillner . Associative Arcady Meyer and Stewart W . Stein. 1976. Cambridge. Chiel.159. Plenum Press. Conferenceon Simulation of Adaptive Behav. 1952.. 1991.G. Plenum Press. Getting . editors . of the Fir . SergeRosSignol.organising networks in really useful robots" . [9] Richard Durbin . MIT Press. 1984. in Neuronal Modeling: From Synap. in Neuronal Modeling: From Synap. 1992. Robustnessof a distributed neural network controller for locomotion in a hexapod [20] Gunther Stent. acts in vertebrates. Arbib . Cohen.t International [6] Rodney A .G.. pages 245. Anne M..Rover robot prototype.. of Steve Price of Martin Marietta . Neural Control oj Rhythmic Movement. From Animal. 1989. From A nimal.. Organizational conceptsin the central motor networks of invertebrates. 1989.: Proceeding . KennethS . io" . MA . 1976. 1988. 1980.: Proceeding . Neurobiological basesof rhythmic motor Software Institute has also provided grant support. Plenum Press. and Leon S. Tim Smithers at the Free University of Brussels has provided welcomeinsights and vision into this researchover [12] Sten Grinner . Sterling.. In Jean-Arcady Meyer and Stewart W.2( l ) . 5 [ ] Rodney A . Gearoid vertebrates: From behavior to ionic mechanisms.149. Group. editon . to Network. Quinn.iol Dg' J [1] Michael A . 8(3) :293 303. Phy. Cambridge... MIT Press. Chiel. edi- Support for the Micro. Vehicles: Ezpenmentsin Synthetic Adaptive Behavior. IEEE Transactions on Robotics and Automation movements. 228:143. JamesT .544..t International Conferenceon Simulation of [4] Valentino Braitenberg. Ali has been instrumental [ 13] John Hertz. MIT Press. Introduction to Ethology. pages. Brooks. March 1986. RandallD . Addison- Wesley Publishing Company. and Paul S. Wilson . 1989. 117:500. J.. 1990. 1989. Neuronal mechanismsfor rhythmic of the First International Conferenceon Simulation [ ] motor pattern generation in a simple system.2: Neural ( London) . Palmer. Volume 18. and Sten Grinner . Herman. Robot ethology : A proposal for the research for a mobile robot. pages 171.e.263. and Paul S. Bryan Willson .292. Stanley Kater . ed- Johnson.40.254.186. John Wiley and Sons.e. In Pattle Maes. manuscript. 1990.. John Wiley and Sons. The Colorado Advanced [11] Sten Grillner . Reconstruction of small neural networks. Sjostrom helped with earlier drafts of this paper and Muhammad M. In Christ of Koch and Idan Segev. organiaing Sy. Animals to Animats: Proceedingsof the First International In Christ of Koch and Idan Segev.399. Buchanan. Neural control of locomotion in lower At Colorado State University. Sten Grillner . Wilsoned - : Theory and Practice from Biology to Engineering iton . [16] David Kleinfeld and Haim Sompolinsky. and 18. MIT Press. Science. Roger D. editon .246. In F.Wesley Publishing Company.. pages377.474. L. 1991. Brooks. Addison. and Michael Histand are gratefully iton . Beer. Challengesfor complete creature Conferenceon Simulation of Adaptive Behavior architectures. The neural basis of behavioral choice in an artificial insect. Espenschied. A quantitative description of membrane current and its application References to conduction and excitation in nerve. of Adaptive Behavior. MIT Press. MIT PresS. pages1. in Vertebrate acknowledged for discussions leading up to this . We gratefully acknowledgethe efforts and support MA . . editors. pages 152. and Graeme Mitchison. Neural circuits for generating rhythmic robot . pages247.. New York .the Emergence of Order. MIT Press. [ 14] A. 1990. From network models for central pattern generaton . Herman. RA -. Beer Press. in the form of a gift from Martin Marietta Astronautics to Networks. Wilson. Method. In IEEE Journal of Robotics and into intelligent autonomous systems. Lennart Brodin . Eugene Yates. editors. Cambridge. in the construction of the Micro. to Animat. Advancesin Behavioral Biology. to A nimat. Plenum 2 RandallD . 1991.

Gallagherl and RandallD. we intend to address we will discussthe appropriateness -time of continuous the analysisproblem by taking a dynamicalsystems neuralnetworksas a meansof control for autonomous perspectiveand re. but rather. OH 44106 USA {johng. Two versionsof the insectbody were utilized. We will makea qualitativeanalysisof system of complexbehavior. A successfulcontrolleris onethat. Sincethesecontrollerswereevolved. A Qualitative Dynamical Analysis of Evolved Locomotion Controllers John C. Using as a pedormancefunctionthe agents' 1991) proposesthat a large ~ ount of intelligentbehavior fitnessin the environment . In this work.ces. In so doing. ratherthan of multiple legs. We have elsewhereaddressed the synthesis . Beer. and a six-leggedversionto study the neuralnetworksto controlseveralautonomous interactionsnecessaryto properly coordinatethe steppings agents . we employeda geneticalgorithm arises from the dynamic coupling betweenan to find controllersthatoptimizedthe agents'overall ' agents environmentandits internalcontrolmechanisms performance . problem. Usingtheresultsof theseanalysesasexamples . Severalquestions . we focusedWe employeda one-leggedversionto studythe generation on the first two questionsby exploringthe use of controlsignalsneededto drive the steppingof of geneticalgorithmsto generatecontinuous -time dynamical individual legs. we focus on our systems ' ability to makeappropriateuseof We have success fully used geneticalgorithmsto develop sensoryinformation. beer } @alpha. thosestudies .edu Abstract designed .2 of t ComputerEngineeringand Science Departments . In previouswork (BeerandGallagher . we not only demonstrate the utility of the dynamicalsystems approach . In particular. In coupledwith the agentand suppliedwith information this section. anddemonstrate that adaptive behaviorneednot be theproductof changesin network dynamicalneuralnetworksto control a number of autonomousagents( Beerand Gallagher . we presenta brief r:eviewof the portionsof aboutthe environment . Whatkindsof control 2. a naturalconsequence of the network's dynamics . Sincethis work concernsitself . Beer .cwru.examiningseveralpreviously . we adopta dynamicalsystemsperspective In previous work. we assumedthatboththedynamicsof the ' agents bodiesand the dynamicsof the environmentin whichthe agentwasto functionwerefixed a priori . In structure. . Such networks theseevolved controllersgenemteappropriatecontrol are capableof exhibiting a rich repertoire signals. 1991). We 1 Introduction also assumedthat thereexistedsomemeasureof each ' agents overall level of pedormancewithin that environment The adaptivebehaviorapproach( Meyerand Wilson. the developedcontrollersgenerateadaptivebehavior? 1990). agents developedcontrollers. 1992). ' agents effectorsin appropriatefashion. we demonstratedthe use of -time recurrent neural networks as a andfocuson theproblemof understanding how continuous meansto control autonomousagents . and2Biology CaseWesternReserveUniversity Cleveland . Due to this complexity. 1992). theyoftenlackedanyobviousinternalorganization . Among the networksdevelopedwere (Maes. however .1 mechanisms are appropriatefor the generationof adaptive Agent Body Model behavior? How can thesemechanisms be designed We employedan insect-like body which is anexten- or developed ? How canwe go aboutunderstanding how sionof a modelusedin previouswork (Figure 1. producesoutputsthat drive the that work thatare of concernin this paper. but alsoexplainthe mechanicsunderlying 2 Locomotion Controllers networkfunction. when severalvariationsof a leggedlocomotioncontroller. ooth phaseportraitsto illustratethe dynamicalbasisof several synthesisandanalysisof thesesystemsis oftendifficult adaptivefeaturesof someevolvedlocomotioncontrollers . In this paper. needto be asked.

Three of theseare motor neuronsand govern the state of the foott the forward swing torquetand the b~ckward swing torque. Whenits foot is We usedthe public domaingeneticalgorithmspackage raised. and any GAucsd 1. successful controllers were developed. and theothertwo generate that would ' producewalking behaviorwhen coupledto opposingtorquesabouttheleg s singlejoint The single the describedartificial insectbody. Each neuronin a leg controller Each leg has a foot that may be either up or down. = .) ( 1 + e ) is a sig However . Ti-i Yi+L. of the weights. we evolved single . I (t ) in equation( 1) is wi9(t ) t where9(t ) is the leg ~ Giein radiansandWi is theweightof the sensoryinputto neuron i). we evolved single leg + Ii (t ) (1) controllers j=1 that were always provided with accurate leg angle information via the sensory input term of equation wherey can be interpretedas the meanmembranepotential [ 1] . In all cases. Gallagher and RandallD . activations of both the foot and backward swing neurons . In addition to The behaviorof all of the agentsdiscussedhereis con . the six leggedinsectis consideredto be stably supportedif its centerof mass 2. (J is a bias term In some cases. In additionto inputsfrom otherneuronsin the networkteachneuronalsoreceives Figure 1: Insect Body Mcvtel a weightedinput from the legts anglesensor(i. such deprivation simply caused the insect to thej th to theith neuron.. Whenthe body is stably supported .3 Results is insidethe polygonformedby thepositionsof the feet that are down. Single leggedinsectsare conttolled by five-neuron fully-interconnected dynamicalneuralnetworks.2 Dynamical Neural Networks force/ acceleration properties of the leg. We succussfully evolved controllers fonn: for both the single legged and the six legged locomotion dy .. (2) The dtn' ations of the phasesmust be matched to both the kinematic range of motion and the dynamic 2. j (Yj) WjiO In one series of experiments. controllers of this class were named ( 1984).e.1 (Schrauoolphand Grefenstettet1991) forcesit generatesserveonly to swing the leg aboutits searchthe spaceof joint with the body. recurrentneural networks the actions of all legs such that stability is maintained (dynamicalneuralnetworks ) with a stateequationof the at all times. The remainingtwo units are interneurons with no specifiedrole. and the activations of the ' a singlesensorwhich returnsthe leg s angularposition forward swing neuron must be out of phase with the relativeto the body . activations of the foot neuron and the backward swing dynamics In addition to the effectors.John C. six legged conttollers must also be able to coordinate ttolled by continuoustime.~ . 1898) .~N ' problem. Networks to early chained reflex theories of legged locomotion with this stateequationhavebeenswilled by Hopfield (Sherrington.1 - of the neuron. andIi (t ) represents an external stand motionless. = (9j . leggedinsectis consideredto be stablysupportedwhen its singlefoot is on the ground. T is a time constant equation ( 1) to zero) caused insects with this type of associatedwith the passivepropertiesof the cell membrane controller to take one step and stop.these . deprivation of sensors (setting I ( t ) in which controlsthe firing threshold . BecauseHopfieldassumedzero diagonalsymmetry reflexive pattern generators. is connectedto corresponding neuronsin the conttollers When its foot is down. O ' j (f. Becauseof their functional similarity input to the networksuchas from a sensor . - its shorttermaveragefiring frequency . singleleg controllers. a leg providessupportand can of adjacentlegs. his networkswere capableonly In another series of experiments. all of these conttollers lacked the ability to - moldal(S shaped ) functionwhich can be interpretedas function properly in the absence of sensory feedback. legs are equippedwith neuron must be in phase. Sincewe do not makethis as- sumptiontour networkscan exhibit more complicated dynamics . In other insects of the . wi i represents sttength of the connection from this type. generateforceswhich movethe body. Six leggedinsectsare controlled by six coupledt the singleleggedmodel. a leg is inca~ ble of providingsupport . A leg containsthreeeffectors: one (Goldbergt1989) (Holian " ~ 1975) to networkparameters for settingsthatproducedconttollers governsthe stateof the foot. the A successful leg controller must produce rhythmic control forcesare summedandtranslatethe body underNewtonian signals that have the following properties: ( 1) The . Beer of fixed point behavior.

A trajectoryundertheinfluence generatea swing(lift leg and rotateto full forwardpositionof a limit cycle will flow to the cycle. Controllersof dlls classare called centralpatterngenerators(Delcomyn. nor could six leggedcontrollerevolved without inputby deprivingthe networkof sensoryfeedbackduring sensorstakeadvantageof that infonnationwhenit was evolution. or attractors. die bottomsetof plotsshows stablelimit sets. once with and o~ce without sensory and Stewart. Suchsystemsare characterized by feedback . only two are leg on groundandrotateleg to full backwardposition). The top setof plots The b' ajectoriesof somesystemswill simplydivergeto showsbody velocityandneW ' on activationlevelswhen infinity as time goeson. if die state the remainingplots showthe activationlevelsof the labeledof a dynamicalsystemeverfalls in a limit set. Of die severalclassesof atb'actors . The the atb'aCtor's basinof attraction. Like single areallowedto varyin timeasa trajectoryis beingtraced. To producea singleleg controllerthat could operate 3 Dynamical Systems in theabsenceof sensoryinfonnation. Sincecontrollersin dlls class valuesthatthestatevariablesmayassumeconstitutethe couldoperateeitheras reflexiveor centralpatterngen. The plots shownrepresentthe behaviorof the state). yet beableto take advantageof it if available. we useda compoundevaluation The dynamicalneuralnetworksintroducedin the last function in which each candidatenetwork was sectionare instancesof dynamicalsystems( Thompson evaluatedtwice. die action neurons . equations is the dynamicallaw. A network's fitnessscore was taken to be a finite set of statevariablesand a dynamicallaw that theaverageof its performance widl andwidloutsensory governshow die valueschangein time.dlerebygeneratinga swingphase autonomous or nonautonomous . and the set of neuronstate takeadvantageof sensoryinformation. In an autonomous morequickly if needed . when consb ' ainedto moveforeveron that closedcurve. dlls spaceis both real-valuedandeuclidian. which are setsof pointsthat are invariantwith respect plot showsthe forwardvelocityof thebody. that hasdie propertythatall b' ajectoriespassingthrough Note dlat in the top set of plots. die sequence of statesgeneratedby the actionof coupledbody/controllersystemwith die leg initialized the dynamicallaw is calleda trajectoryof die system. For a givendynamical infonnation. Fixedpoint atttactorsare stablelimit setsconsistingof generatinga full stancephasecausesthe insectto move only one point Limit cycles are stablelimit sets that its leg all the way back. jectoriesthat lie outsideof atb'actorsareknownas transients fonnationis availabledirectsdie insectto stance(place . 2 Figure detailsdie operationof a mixedpatterngenerator Startingfrom a given point in the statespace(initial . 1980). there is a longer nearbystateseventuallyconvergeto die attractor. important in understanding die controllerswe examined . The velocitysets. In addition. All were successfulin nonautonomous system. input (I (t ) in equation[ 1]). diey lackedthe ability to make use of sensoryinformation. and would often (Yi) are the statevariables . feedback . networks. The set of all possible to improveperfonnance . leggedcontrollersevolvedwith sensors . six leggedcontrollers Dynamicalneuralnetworkscanbe eidlerautonomous or evolvedwidl reliablesensoryinformationavailable nonautonomous . providedwith a leg anglesignal. andwill become ) beforestepping . ' systems states~ ce. dependingon the presenceof external could not functionin the absenceof this infonna. For informationfor the samecontrolleroperatingwith sensorythe purposesof this paper. we are mostinterestedin die feedback . The bottomsetof plotsshowsthe same of die systemwill keepit within dlat set indefinitely. shortenthe lengthof Further. Sincethe insect's leg is alreadynearlyall die way back. sincediey were evolvedwidlout sensoryinformation. are held constantas the statefollows a trajectory. dynamicalsystemscanbeviewedasbeingeither initial stancephase. die parametersof the dynamicallaw Each seriesof experimentswas repeatedwidl full . to 95% of its full backwardposition. . An insect the influenceof a fixed point will flow to that point with its legalreadyneardie mechanicallimit should andneveragainleaveit. Otherswill convergeto limit the controlleris deprivedof sensoryfeedback . In a six legged artificial insects.but could functionwithout it if necessary . 1986). A b' ajectoryunder movethe leg furtherbackthanit canactuallygo. In all cases . coordinatedwalking. Qualitative Analysis of Evolved Locomotion Controllers leg controllersthat operatedin the absenceof sensory tion. In odler words. dynamicalsystem. The transientbeforenonnal d1ydlmicwalking (denotedby setof pointsthat convergeto an attractorare knownas the periodic" saw toodl" velocity pattern) begins. In die caseof dynamicalneural erators . we namedthesemixedpatterngenerators . The mixedcontrollercan. An atb'actoris a limit set the valueof the sensoryinput (I (t . The portionsof b' a- stereotyped controlsignalgenemtedwhenno sensoryin. while to die dynamicallaw. whenavailable. Thesecontrollerswere capableof producing present . Six leggedcontrollersevolved with acom - appropriaterhythmic behaviorwidlout sensory poundevaluationfunctionoperatedbetterwith sensory input However. die meanmembranepotentials could operateundereidler condition. then wastetime attemptingto fonn a closedb' ajectoryin statespace . we evolvedcontrollersthat neuralnetwork.oneor moreof thoseparameters producingeffective.

/ ./_ -J\_. Beer - ::... -"'~_j--~ -.J '-J \ -J I8d.J -_J --' Figure2: Activity of a typical mixedleg controllerwith (top) and without(bottom) sensors." .""""l __.\. _.I-"""."\ .r ~ .~-----.l\__. -. / / ... .er 'd .JI\_ -_ r..1.. --_...--.V ~.--_...J"'\"/ -".J ~-./-.. Yel~ 1ty rFoot -l__r---\_I--l _J---._ ../ .Jl jiInt8meur _ _ _ __.1 -.... sensoryinput is the valuereturned by the leg sensorin radians./ -J\_ ../l .. . .... 1__.-~".-J---\-1 /F\... \... ../ 1__/ 1.~-~ . 1_./ l . / - :=:.. \. The leg was initialized to 95% of its full backwardposition.A _n____JL__.// ._. -..- =] ~\1J ~ J \ J \ J \ .. -.J\_-_. ...... .JohnC._... ~ 1_ _./-.. : :: l _. .LJ-_../ . / .J \ ... .- AJ lj lr \ -. ---_. .~-_/.J--_._--\j -- ~ --. ..."1. . .Jl _ ___ ._I---\__-J-\-J----\. Jl. ---. "" .- J .jt _.___n___n____. Gallagher and RandallD . _I--.--.-. -' -.I -~ ""\ -"'\ ... -.--_ .I - J \~ ."""". l _. - I_..J _r _f .-_J .J Far.-___f __. - ~_~_.. . The remainingsignaltracesare neuronactivationlevels.. 1__. ~ - s-G " "-U~ I. . . . GIA I~.J\.' .~ . l _. / ./-. -._.-~ ..I .-.J 1_ _.\I--_J-r\-. Velocityis the forwardvelocityof thebody... " ... ~ . -_..-I'd ~1 .

position. As before. This thatthe A andB intemeurons arenot contributing system to flow to a fixed point and prevent normal. two comers of the phasespace . We Generator can swdy the attractorsgoverningthe behaviorof the Each neuronin a reflexive patterngeneratorreceives coupledsystemtrajectoryat a given angularposition as a sensorysignalthe leg' s angularpositionin radians. each plot. The foot andbackwardswing . and backwardswing neurons(from hereon referred to as a motor spaceprojection . and to the generationof this behavior. movingand the changing shapes of both attractors and flows. non clampedsystem. though. The fact that intemeuronscan be doing such vastly Mixed patterngenerators operateboth with and without different d1ings . As with thepreviouslydiscussed reflexivecon- . When sensoryinput changes . In thosecontrollers. we haveexamples previouslydiscussedreflexivecontroller. In fact. in general. It is. are being Figure3) demonstrates shifted as the law is change continuously dynamical ~ phasedandperiodic. it is the only propertywe wouldexpect otherprojections . In signalis doing. A detaileddiscussion The trajectoryforms a closedloop. and seeno difference in networkoperation . and flows to that point. we can rhythmic walking exactly removetheseneuronsentirely. that this controller. the shown position. its fitnessis that the system's motorspaceprojectionis The shapeof the trajectoryis differentthan that of the . the trajectory of the coupledsystem is Figure 3. the controller is trackinga fixed swing. generatedmotor signalsare periodic. S Analysis of a Mixed Pattern removingthe interneuronswill . The switch between neuronactivationsare high at the sametime. then . indicatingthat the of this can be found in Beer. The trajectoryobservedin die centerof Figure 3 is The surroundingplots in Figure 4 are motor space an attractorof the coupledbody/ controllersystem. the grey circles denotethe statethe nonnally By makinga qualitativeswdy of systemphaseplots (trajectoryplots). and we selecteda reflexivepatterngeneratorfor analysis. the A and B interneurons We would expect. appropriatelyphasedand periodic. andthe forwardswingneuronactivationis which movesthe governingfixed point between high when both the foot and backwardswing are low. by fixing the sensoryinput to the valueof interestand the of the resultingautonomous Noneof thereflexivepatterngenerators we evolvedwere examining behavior dynamical . At projectionstakenfor the sameleg anglevaluesusedin every instant . is not surprising. 1992 . replacingthem with appropriate is why the reflexive conttoller can not function without its nonnal sensory input . The five dimensionaltrajectory. suggests . of controllerswhosemotorspaceprojectionsare periodicand appropriatelyphased . but Thesurrounding plotsin Figure3 aremotorspaceprojections doesnot tell us much aboutexactly what the sensory of the controller with thesensoryinputsclamped to values corresponding to the shownleg positions. system . THis is what occurs. Sinceit is difficult to envision early portions of eachtrajectorymatchcloselythe path spacesof higherthat threedimensions . mustbe sharedby successfulcontrollers. we utilized threedimensionalprojectionsof the five spacetrajectories taken by the nonnally runningsystem(Figure3. in In this particularcontroller. a the systemfollows at that leg position. A the curves from the gray to black circles showthe trajectories singleleg controllerhasfive statevariablesand thus. This is to be expected . The only theoperationof a mixedpatterngeneratorwhencoupled featureof die controllerd1atis importantin determining with the body andprovidedwith normalsensoryinput. each plot showsthe flow from determined by thedynamicsof the controllerwith sensory the statethe systemis normally in when the leg is at ' that positionto the attractorgoverningits flow at that input clampedto the valueof the leg s angularposition at that time. Sincethis is all that nearly idenucal . that other reflexive controllersdo use their intemeuronsto assistin thegenerationof behavior. clamping both produceconstant the sensor to any constant value shouldcausethe . constantvaluedblases . yet the whole systembe able to generatesensors . however. The centerof Figure4 is a motor spaceplot of appropriatebehavior. zero valued activations . center). forward At any given time . Qualitative Analysis of Evolved Locomotion Controllers 4 Analysis of a Reftexive Pattern callaw governingthe systemalsochanges . One shouldnote. To illustratethis. system follows these trajectoriesinstantaneously . capableof operationwhendeprivedof sensoryfeedback This observationsuggests thatsuchfeedbackis vital. generating by varying sensory inputs swing and stancephasesis precipitatedby a seriesof bifurcations a stance . Indeed. is in whenthe leg is at that might otherwisego unseen . asthe pathof thebody/controller . The black circlesdenotethe fixed points governing the systembehaviorat thosetimes. it is possibleto uncoverimportantdetails operating . die dynami. A phaseplot of the activities of the foot. yet show vastly different behaviors in was selected for .see centerof point Periodic behavior is beinggeneratedbecausethe that this controlleris both properly governing fixed point . damagethe Generator generationof appropriatemotorsignals.

I'-.I.-"-.5--1 .1>? " .. Beer ~" ~' I '-----T ~ II--_--. 1 '"-.. .I . '{{ >? Figure 3: Motor Space Plots of Reflexive Controller. The coupled system limit cycle is generatedas a result of die system following a series of constantly changing. -~ .'.."~~'-'~.'~ .."~~ .II.II '. I . J I "_-II(.-.II(.rItI-.~ .'. The coupled body/controller system has a limit cycle atttactor that is appropriately phased to drive stepping behavior..I 'I. Center plot shows the motor spaceprojection of the activity of the coupled controller/ body system.. -.-_ -~ .." '""-." I.'~ -~ . I I I .~ ..--.I. ttajectories...." I.I(~.II. . ~~.-'~1 T I"~_~ r I. ... -"~-.--~~-~ .~ . ..JII.._ "..1 -(I'-.1 - (II'.-~--~II.~ ""... -.~-.I .a .~~--I.'.~ Foot -1.I-~---~-.I J.I--~ -.I.~-. -- John C..-.tI-~~-.". ~--. Surrounding plots show die instantaneousflows the same controller follows at shown leg positions.~~.I.'.I. Gallagher and RandallD ."-".~ .I '~ .Ia "-.-..'~.'. . .I.(. '0 . " . leg position determined.~".I -.~ II'..

~ 0 Foot- .-"-.I~ .em becausethe initial state is already on the attractor.".-~ I (. I I -~ .-.oJ -. ----.I~ I) " .. and initial state with grey circles.I Qualitative Analysis of Evolved Locomotion Controllers --. Center plot shows motor space projection of coupled controller . . Attractors are shown with solid lines.'-".~ I.. Some ploi $ show no trans.-'. : -----. transients with dotted lines.. / body system.~I --~ - .I ~ > > .'..'.Figure 4: Motor Space Plots of Mixed Conn-oiler.II I ':if-.-.-."I ----~-.I.' FS -- J .I'~ . .I .-'"I..I(. .. I1J.. Plotting conventions and system initial conditions for all plots are identical to those used in Figure 3.'.o 'I. Surrounding plots show motor space projections of the same controller with legs fixed at the shown positions..-."..I. --"-.".

.I '! '.-..---. Themixedcontroller (top) andslower ) (bottom thannonnal sensory .-----./ l __/ " 1 . --~-----.._l_.J\_J\.J 7V \I\I\I\/\JV\/\/\.. ':::'~ L88 "~.--.r \ r \J \ r \ f J \ I_~"'_-Irmt A \.~._ -_..--~/ "\../ tomodifycontroller Figure5: Usingenttainment frequency of Figure2 is drivenwithfaster .~ - I_ 1'a' A ---" ---..-.. """'-' . signals Notetheoneto one between entrainment theinputsensory signal and the motor activation levels. .-..- F ~ :f v ~ = ~J .-A ._ _ J \ _ _ r L ~ " 1.-j ._____ ~ r l _ __ __./". _ ____I \ ./ l _____. " L88 ./ 1__.._J =:~J~ I ~ ~ """-J / ~ / ~ / ~ / ~ J ~ .I\--_..'-A-J -J-'--.J \ .... ._ _..--.. ---.. .._.-._F ".~ 5001 . -. ----.-. --.--_.. """"-""_..... L..._ J \._J-1__. -- / / . ---. _ .1 ~ . a-. ""'- : :~ 1_. --_.78 John C._ - . ./ "\' l _. J \ --J \ .J. .I\-~_I\_J'..J.A --I'-..1~ .. ."' "' -""' .I ~J ~_. I \ _. --.--.-----..rL_J-l__.... Gallagherand RandallD.-./ 1_.. .---.".---._ - :r U .. .-- J \ _-_J \ ..I~ ' \_ rmt .-. / 1__~. . . .. --"". ___I ~ it. / --._-A -J-"_'-.r--A . ._ ---_.._. .J \ j ~~ ~ -... . -----.__./ " --_:'_ --_./ ""\' I~ I .-. --. Beer ' -.. L_"" "" .-------._ .lJ\m PO81 \__J\_..__-1_. : :~ l ____ / t_-_.'_-A .-----_.J/. --.._ .J :~ t..

as several sensory input periods pass in the governing time it takes the system state to orbit the governing limit cycles. however. The limit cycle corresponding to a J II sensorvalue of zero (deprivation of sensors) is appropriately phased to drive stepping behavior.-. Theselonger stancephaseswould utilize the to the normal sensorysignal. Above the rather. the frequencyof to their functions. controller. would continueto generatestereotypedcontrol In the range from about one half to twice the normal ' signalsthat would not takeadvantageof properties sensory frequency. the network s output is 1: 1 entrained of the changedleg (Figure 6). Howevert by applying dynamical . This entrainment results in faster The mixed controllershowsability reminiscentthe or slower. Note Since the mixed controller . The system merely flows to its attractor. 1985 ) . It alsodemonstrates that quickly phase lockings are seen. It almost control efforts that moved the body forward. more complicated to differingconditions. 7 Discussion most notably those that entrain 2: 1 with the output. but otherwise normal looking stepping behavior ability of nattJrallyoccUlTing neuralcontrollersto adapt (Figure 5) . In so doingt we have demonstrated Generator to a Changing Body both the utility of a dynamical system viewpoint . removing sensoryinput Leaadl l Aa does not destroy the generation of appropriate stepping. An interesting behavioralimplicationof this fact is that this network shorter the periods of the generatedcontrol signals. no . Our evolved networks were free to exhibit any dy - does not stumbleabout for extendedperiodsof time namicst as long as their motor space projections provided learning how to deal with its damagedbody . for example. because the system state has need not be the result of an learning time to orbit the governing limit cycles several times adaptation explicit procedure or overtchangesto networksttucttlre. in many cases . A cockroachafter a leg amputation . without its sensorsas the leg grows in length. passed clean architectures whose structures easily relate As pointedout in the last section. also produce normal looking stepping behavior. Qualitative Analysis of Evolved Locomotion Controllers IS troller . the change the sensoryinput with sine waves of various frequencies. with the sensoryinput . No difference in activation was observed in any of the five neurons making up the full rangeof motion of the longerleg. but during each period of the sensory input . other entrainment ratios are the controller. If a leg wereto grow in length while the agentwas walking. Below this range. thus either making longer or by the frequencyof the sensoryinput. we can also change the rates at which the tractors and flows change. body. observed. makes use of its sensory sign "al to shift the flows fonnation is available. dynamical systems Unlike with the ! previous sa -. the governing attractors are usually limit cycles. and attractorsgoverning the system trajectory. like the reflexive that performance does not improve unlesssensoryin- controller . The mixed controllerwould enttainto the resulting of the normal sensory signal. unlike So IS ao as with the reflexive controller. for example. which itself is Figure 6: Perfonnanceof mixed controller with and appropriately phased to drive stepping . we detached the mixed controller from the body and drove legsof differentlengths. in angularpositionfor a given appliedtorquewould be To check that a sine wave was not too gross a distortion smaller. Therefore. however. one might think that by changing the rate at which the sensors ' at. Some of these more acceleratedsensorysignals. It its immediatelyreorganizes gait compensate to for the was not surprisingt thent that evolved controllers rarely missingleg (Graham . The controller not capableof using sensoryinformationin this way. the trajectory observed when the system is provided with sensory information is a result of the many M instantaneous flows the II corresponding to generatedby each leg position . and some of the suengths of dynamical neural networks as Natural control systems . the mixed patterngenerators oscillationsis entrained change . To is able to adaptits controleffort to appropriatelydrive test whether such entrainment behavior was possible. we first supplied the network slowerleg frequencyandgeneratelongerstance with a sine signal whose frequency was identical phases .- controller . do not need a vehicle for implementing controllers for autonomous manytrials to adjustto damageor otherchangesin the agents. as a natuml consequence of the non-linear dynamics range of 1: 1 entrainment. - By using a qualitative analysis of motor space projec tionst we have been able to understand the dynamics underlying some adaptive feawres of two evolved locomotion 6 Adaptation of Mixed Pattern controllers.

andMachineLearning. Our analysishas demonsttated severaladaptivefea- tw-es of dIe evolvedcontrollers. A<kIitional supportwas provided by die HowardHughesMedicalInstituteand the ClevelandAdvancedManufacturingProgram through the Centerfor Automationand IntelligentSystemsResearch at CWRU. ( 1898).1ucsd. J. A without its sensors . Patternand conttol of walking in .J. Ackn. This work was supportedby Office of Naval Research grant N<XX >14-90-J. and Grefenstette . continuesto operateunderthe samerestriction. ( 1986 ) Nonlinear controllers. R. ( Beer. Beer.C. From Animals characterized the role sensoryinformationplaysin bodI to Animats: Proceedingsof theFirst International thecontrollerspresented .A. That 22:319-332.D. andGallagher . R. Indeed.F. Designing Autonomous Agents.1545. Ed. Physiol ( London) thecontrolleris enttainedby dIesensoryfeedback . N.11.D. insects . ( 1991). Delcomyn. (1985).John C. andStewart .D. ( 1991 ). and why the mixed patterngenerator User's Guideto GENESIS 1. GeneticAlgorithmsin Search.. we believe that dIe Systems . Academic Press. EvolvingDynamical NeuralNetworks ((I' AdaptiveBehavior. J. First. MIT Press . MIT theatttactorsgoverningdIecoupledbody/controllersystemsPress. J. (1975) . J. Neuralbasisof rhythmicbehavior in animals. Case WesternReserveUniversity.M. Gallagherand Randall D. Decerebrate Rigidity and Reflex outputto a appropf'l8telydrive a changingbodybecause Coordinationof Movements . P.S. and WilsonS . ( 1992 ). ( 1992).WEds . R. Adaptive Behavior1:92. ( 1990) . (1991 ). We haveshowndIatthe mixedcontrollercanadaptits motor Sherrington. H. A DynamicalSystemsPerspective on Autonomous Agents. Beer systemstechniques . Intelligence as Adaptive Behavior : An Experiment in Computational Neuroethology. J.140. .T. we were neverthelessable to understand Holland . 1992). GrahamD . WileyandSons . Goldberg. why the reflexivepatterngeneratorfails to work Schraudolph . References Beer. termsof Conference on Simulationof AdaptiveBehavior. I . Adaptationin NaturalandArtificial dIeir operation. Science210:492-498. 18:31. Departmentof ComputerEngineeringandScience .B. Addison-Wesley . ( 1989). we haveclearly Meyer. TechnicalRelX>rt CBS 92. languageof dynamicalsystemsmay provide a powerful theoreticalframeworkfor autonomous agentresearch Maes . Universityof MichiganPress . suchrangeof functionis expressible in only five neurons is testamentto the potentialpower of neural network Thompson . and Dynamics Chaos . Advancesin InsectPhysiology. C. ( 1980). Beer. Wehaveexplained. Optimization.N.E. I .122.owled2ements Wewouldlike to dlankMichaelGallagherandLesliePi- caroofor dleir commentson earlierdraftsof this paper.

For hearing . and motion variables . instantaneous neurons within a ' map often have frequency . optimum estimation ' of the animal [ 2. radar The motor maps are excited by / sonar imaging . but the neurons the body surface 1 [ ]. 5] . location . . distortion different ' sensory modalities compensation . maps respond best to stimuli at These propert ~es can be exploited in the space specific points for. Neighboring / source location . neurons with relatively large extended dendritic trees that 1 . Chirp corporation 8248 Sugarman Drive La Jolla . Comparison In electric fish motor . multiple neuronal from all sensors to control maps representing ordered direction of movement [ 7 ] . vision . Animal sensory systems utilize locations of tuned neurons are multiple neuronal maps of parameters correlated with ordered values of such as estimated stimulus a mapped parameter . amplitude . variation of various signal parameters are found in different position or place of origin of parts of the brain . superior colliculus . hearing . Bayes . INTRODUCTION the layered penetrate through sensory maps . cessors indicates advantages of animal position are in spatial map . Alt . overlapping tuning curves . tectum animal echolocation and machine [ 3 ] and in the optic of lower order vertebrates [ 4 . delay . surrounding t ~me vary ~ngparameters .based processors that can be register with the sensory maps . and are connected maps are extended beyond the body to corresponding motor surface and into the external maps via large neurons that penetrate environment 1 neurons in internal through . infrared processing overlaid in the optic and animal sonar . Neurons a stimulus or echo descr 'ibes only within such maps respond best to . sequential OVerlaid extracted from maps hypothesis testing . . positions comprising such maps are ordered within a cortical map are correlated with respect to values of the with different on points mappedparameters . exploited for sensor fusion .6 ] . recognition ' of speech . and signal have overlapping tuning thus represent location curves . corresponding to . spatially parameters registered . maps with man made signal pro . sensor or whole . combining information In many animals . the sensory ' layers . and tracking . sensor fusion . Maximum ( best ) responses of tuned neurons In somatosensation . somatosensa - and robust multitarget tracking . CA 92037 ABSTRACT particular stimulus parameters . tion ) are spatially registered in as well as for interpretation of the superior colliculus of mammals ongoing experiments in vision . electroreception maps from different sensors are tectum or .NEURONAL P~RAMETER BPS ~ D SIGNAL PROCESSING Richard A . ( vision .

. This viewpoint provides . AN INTERPRETATION OF NEURONAL New data at time k consist of EXCITATION IN P~RAMETER MAPS a data vector rk . by a vector of parameters ties or detection - statistics such or features Xk as well as by the as the log likelhood function or signal class Hm. g .composed of sig - nal samples corrupted by samples Each neuron in a parameter map of noise and interference . the frequency data . 2 . RI BM . . SEQUENTIALPROCESSINGOF data vectors = . the significant degradation Rk ( OR . . while others describe signal insights "into biological attributes ( e .. and how might artificial values or hypotheses . Hm) in an interval star - neuron s position in the map . to compensate for coarseness of m= l . or posterior probability features represent kinematic variables corresponding to a particular such as position and motion value of x . Altes part of biologically significant The interpretation of neuronal information about an object . ( i ) the possible uti .. map represent updated conditional How are such maps expl ~ ited by probabilities of feature animals . xk . e . r2 . the siqnal is affected related to conditional probabili . etc . Hm) additive noise [ 8] . color for vision . rk ' SIGNAIS WITH TIME VARYING also on signal classification P:ARAMETERS hypotheses ' .ing effect ) that can occur The conditional probability of a when a large number of equally given feature vector is writ - Xk likely hypotheses are simultaneously to represent the ten p ( tested in the presence of XkIRk . that node maps for some of these parameters activations in an artificial neuronal can also be found [ 6 ] . .. Neural excitation may then be ting at time k . Some of the log . Richard A . band of an auditory stimulus lization of overlapped tuning ) that are independent of position curves as interpolation functions . a maps as ordered hypothesis tests map of location represents a subset provides further insight into of important signal parameters neurophysiological data and artificial . ( ii ) training ferently shaped objects in vision that exploit imperfect design or different in data via utilization of lateral . inhibition or sharpening [ 9 ] .likelihood ratio . but Rk rl .based processors by instantaneous frequency and amplitude using concepts from sequential for hearing . motion statisti ~ al hypothesis testing . and . g . form . . for example .. obtained at rl .. and ( iii ) the role of facilitation The cumulative set of all data of some groups of neurons and / or antifacilitation of vectors . Other parameters include map ..' . M} could consist of different neuronal in cortical sampling mechanisms phonemes in speech .. Neuronal Suppose . di ~- maps [ 8 ] . A mathematical versions be used for optimum description of such a map signal processing and control ? follows . The signal classes { Hm. The is assumed to represent a hypothesis vector is about the mapped parameter correspondinq siqnal vector x corresponding to the composed of samples of the siqnal ' s ( t .. targets animal sonar . fact that feature estimates depend not only on observed input 3 . . rk other groups as a means of reducing and before time k is denoted . prior probability .

~ ~ likelihood functions . are used as priors for and r2 ' The left hand side of ( 2 ) at the process is iterated [ 10 ] ..)e (4) hypothesis depending upon m=l XkIRkHm HmIRkl feature is favored . Hm)P(HmIRk -l . where - observations is the one that max imizes p(HmIRk > on the left hand ) = I p(HmIRk p(HmIRk .l . Hm) of s ( t . )P( . Xk) effects are ~ delay and direction embedded in the parameter vector additive . the initial prior pro - For maximum posterior probability babilities for k = l are uniform hypothesis testing . Hj )P(Hjl ~ . If features are letters ties (initial contextual informationin ( ) text his ) PCXoIRo. Hm multivariate normal distribution with mean vector composed of samples p ( rklxk . Hm) ' where p (rkl ~ . Xk) = ) . feature ~ . Xk)P(XkIRk) side of ( 3 ) . In sian noise .l . probability P ( ~ I ~ .ll ~ . perception also explains perceptual p ( rklxk .hypothesis on the next iteration of 3 . p ( rkl ~ .Hm ) and pcHmlRo' printed can be used to explain . Hm) P ( XkIRk .l is P ( Xk . 11] ( often based on an additive white Gaussian noise assumption p (Hml~ . For example . .--. Neuronal Parameter Maps and Signal Processing can be estimated from train - Hm) Equations for sequential ing data or obtained from predetermined updating of neuronal excitations probability distributions are [ 10. Hm) is a ) = p(Xkl~ . 2 .be formedby converting observations errors during rapid reading .. ) p(XkIRk -. . a signal emanating from a specified direction p (rklxk . zero . time k .l ) ' m= 1 . xk . the hypothesis over all Xo and Hm values .mean Gaus- and Xk . Hm ) . Posterior probabi in some well alternation lities for observation vector rl known visual patterns [ 11 ] . ~ phenomenon human Xc) . One feature can be favored over M another which I p( . Hm) ' The likelihood functions p ( rklxk ' while the right hand side of ( 2 ) . Hm) and covariance . . . ( 2) I p(rkl 'Xk. M. ~ . and this affects the choice of starting with prior probabili . Hmj ~ In the absence of contextual knowledge . The that maximizesp(HmIRk > is most probable hypothesis after k needed .l . Xk) with given delay can be represented ( 1) as s ( t . The right hand side xk of ( 3 ) indicates that such a choice depends on p ( xkIRk ) ' and ( 4 ) shows that this quantity depends and on the updated posterior of each hypothesis .l . posterior probabilities can insensitivity to typo graphical . Hm matrix determined from the noise .into Hypothesis dependent . >P(~ IRk.

As in interactive matrix in ( 5 ) . Hm same input signal . neural excitations probability For additive . Altes at time k dependson p(XkIRk. are I p(XkIXk. SEQUENTIAL HYPOTHESIS TESTING data . Hm). however . . the admissibly distorted updated using excitations of all at time k that has the neurons in a map ( or in several template interconnected highest correlation with data up maps ) at time k to time k yields the posterior toqether with input data observed at time k + 1 . Sequential utilization of new input data with past excitations (2) requires prediction of p(xkl to determine current excitations .l . Inclusion vision . SENSORFUSION differs from many interactive activation networks . from t ~ e input data . activation . originally especially in interactive activation uniformly spaced . processing with where the dynamic model is given neuronal maps by finding the admissible distortion of each hypothesized by ( 5 ) . .1 ' . Hm useful for sequential hypothesis )P(Xk.warped signals ( as in continuous transition probability matrices speech ) and distorted for .l . 9'. Hm )' extracts all relevant information e. foveal probability matrix [ 12 ] .l . Admissible networks distortions determine the transition [ 15 ] . different time .l . Sequential hypothesis testing 6 . Multiple iterations based on the ) = p(XkIRk. e . probability distribution with the largest peak value . a single it ~ ration of the sequential test combines Hm ) . AND INTERACTIVE ACTIVATION For distortion compensation . of ( 5 ) in a sequential hypothesis test allows the test to 5 . from dif . 1 . Dynamic distortion Hi and Hj [ 13 ] .l . such that the template distorted template best fits the 4 . images . This iteration theoretically ) from P(Xk. A sequential hypothesis test the parameter vector x describes for maximum a posterior i detection the new locations of signal / image / classification is similar to recursive found in avariety samples or instantaneous frequency updating components that are of cognitive models [ 14 ] . testing only if the signal is - Xk l corrupted by different .l ' unnecessary . decorrelated (5) external or internal noise samples for different observations For a Markov process . Se compensation [ 24 ] can be implemented quential estimation of a parameter via sequential x is illustrated in Fig . DISTORTION COMPENSATION exploit temporal feature variations ( e . g .lIRk . white Gaussian at time k + 1 are recursively noise . motion . Richard A.induced Many biological sensory systems changes ) that are different for are capable of recognizing different hypotheses . p ( Xklxk . g . as in interactive activation networks . or if different observations H ) corresponds have different resolution m to a transition as in peripheral vs . Hm Rk. in An important that repeated iterations capability of based on is the same input biological systems the synthesis observation are or fusion of data .lIRk . .

r2 p(x3Irl X2 Po' (X3) . ' Bay rule . Dynuic Model . Sequential likelihood estimation of a time varying x ( tracking ) . Neuronal Parameter Maps and Signal Processing Po(Xl) ' p(xllrl 81 ) Xl . hi ) ' artificial neuronal map . The smaller the variance of .=. p(rllxl .J p(x2Irl . .axis correspond to excitations of neurons at ordered locations within the map . rk = observation vector at time k . .Hl) p(r2Ix2 rule .. Dynaai c Model Xl . X2 1 J\ . X3 Fiqure 1 . .H) l ..Hl) Xl 1 / \ . sample values of probability distributions along the x . The illustrated operations could xk involve three different maps or could be performed on a single map with appropriate interconnections between processing elements . ~ the more accurate the parameter estimate . . In an p ( xklr1 ' ' k . . I /\. Hl ) 8 PO(X2) I . m p ( x2Irl . = parameter Xk unknown random variable at time k . neural excitations at time k correspond to sample values along the axis . Bl ) r r2 X2 .

r2 .. the same point in space . spatially registered log . summation devices . to respond appropriately to a the log . as observed in posite rk [ 16 ] . . Data corresponding to each point multipolar neurons used to combine in the environment at time k . maps [ 7 ] 1 such . Fig . spatially registered of the different sensors . for neurons phys ~cally penetrate example . Acom data vector is con . r2 ..likelihood function obtained real . The large . such that environmental well designed to perform the desired operations ~ At point location Xk corresponds to the Xk ' same point in maps obtained from sensor fusion is performed by different sensors . r3 in registered maps from different rl sensors at each position .[ 5 ] . Hl ) in spatial position ~ by the Figure . Inconsistent versions of such data for A biomimetic hypothesis hi approach thus uses of larqe (e. spatially registered sensor fied by using spatially registered output representations . 1. it is advantageous Sequential processing as in ( 1 ) . neurons with synaptic connections in all the sensory . Altes ferent sensory modalities so as are statistically independent . parameters .. however . should then function as linear represented map . spa - . as neuronal excitations in simultaneousl ~ perceived by each overlaid . rk at various posi . as opposed to sequential processing of smaller The model can be used to interpret data vectors from each sensor in data as turn . individual sensors . is maps from different sensors on each . r3 . could contribute to each through all the overlaid . The biomi ~ etic method is neurophyological for rapidly time well as to design ~ an made fusion advantageous varying parameters and is easily systems .. spatially observation vector . to use a concatenated 5 ( ) is then used to update the data vector or sum of loglike - posterior pro ~ abilities of Hl ' lihood functions that represents .likelihood functions from the sensor maps as in [ 3 ] .' data = the outputs of all the different BM given observed ~ sensors at the same time and at rl . corresponding Hm This method is useful if signal compared with the concatenated parameters are time invariant or data vector to generate the vary slowly relative to processing rk likelihood function " time . are systems . Overlaid tions . structed by combininq data samples from a qiven environmental An equivalent approach to sensor location as measured by the fusion [ 17 ] is to apply the Xk various sensors at time k. repeating the process expected composite signal to hypothesis is for each new observation .world event . Acoustic and visual data . a flying insect ) will parallel and data processinq vectors derived cause spreading and a smaller signal peak value of the posterior from simultaneous distribution observation of all avail - p (x3Irl . This process is simpli .( 5 ) to each sensor vector in turn . An procedure in ( 1 ) . Richard A . r2 . . g . .. For time varying p ( rk I Xk ' Hm) . as in maps as in animal sensory bioloqical sensory systems . A multivariate with a concatenated data classifier for sensor fusion can vector is the same as the sum of use overlaid . able sensors . If the sensor outputs implemented with overlaid .

as in ( 2 ) .and . real . q . since such tracks will have but with fine range resolution low posterior probability p ( Hml can use multiple observations Rk) relative to more promising from different locations to . 1 becomes sufficiently concentrated that The formulation in ( 1 ) .EFFICIENT IMAGING FOR 8 . then the transmitter receiver location correct interpretations can theoretically / be accentuated and erroneous . of recursive sum.filtered echo data from a suitable parameter space . 1 . SONAR. line seqment transform proces - conceived as a multi tarqet ( multiple hypothesis ) tracker in a sinq . AND ECHOLOCATION target corresponds to a different . MEMORY .store elements can be used for memory efficient If different hypotheses represent . A receiver . Using a line segment ( rotated wavel ~ t ) hypotheses by dropping inconsistent tracks is easily accomplished transform [ 11 .driven tracker such as a rather than just one . e . suggesting that a similar Sequential likelihood ratio technique may be exploited tracking is thus robust against by the animals . Pruning imaging systems . MULTITARGETTRACKINGAND DATA when a posterior distribution on INTERPRETATION the right side of Fig . objects to be tracked simultaneously with a sensor fusion system . tion data is a biologically inspired matic feature vector Xk for hypo . yields of the tracker in Fiq . object . Tracking of the mth An environmental map based on hypothesis echoloca - object involves updating position sequentially processed and velocity variables in a kine . reqardless of transmitter cessinq maps . 19 ] . conscious awareness of a stimulus in a normal brain could occur 7 . The sequential construct a target image with trajectories fine resolution in two dimensions hypothesis test differs from an error . Neuronal Parameter Maps and Signal Processing tially registered maps . then the same storaqe element can be in the internal map can refer interpretations to the same location in the . The space - Kalman filter in that a single time transmission patterns used by echolocating dolphins are similar predicted parameter vector and to the line the corresponding observation is segment ( rotated avoided . pruned via sequential siqnal pro - with neuronal environment . different interpretations map can be referenced to a point of connected in the environment rather than to speech . cumulative summation of match .( 5 ) its maximum value exceeds a automatically allows multiple threshold of consciousness . Unconscious as from visual trackinq capability qiven object . a radar / sonar with poor azimuth / bearing resolution .time line segment different data interpretations transform imaqinq [ 11 ] . If the . [ 11 ] . probabilities of all wavelet ) basis functions that future parameter values are needed to implement this process possible are predicted at each iteration . provided that each object or RADAR. . concept that has immediate application to man made radar / sonar thesis Hm. A radar / sonar imaqe can then be recursively constructed . loss of track induced by a large difference between predicted and An environmental map composed actual observations . In data understandinq system can be / position . perceived in brain different transmitter / receiver damaqed humans ( 18 ] can locations an imaqe of the be explained by an autonomic version .

Konishi . auditory . H.referenced maps made by each cortical neuron for different positions of the is unaffected by ampution or animal . It has been assumed that neurons [ 5 ] E . and relationship between visual . Carr and L . Hartline . Sequential estimation . cation is synthesized as in ( 3 ) . and somatosensory inputs the same in the mouse superior colli - and is not necessarily " as substituting estimated parameters culus . " in Electrorecention . memory hypothesis tests can in fact be requirements would be extremely used to understand somatosensory demanding . receptive field changes after digit amputation in monkeys [ 23 ] . is it relevant image construction . has been used to model Sensory parameter maps similar of to those found in the brain can pulse to pulse accumulation for information by a dolphin during be exploited Bayes optimum discrimination [ 20 . H. " The infrared ' vision ' of within a parameter map re . the averaged quantities .713 . nearest neighbors [ 4 ] C. (Wiley . Kandel and J . This process uses memory is useful for biomimetic position very efficiently during signal processing . 1 variety of signal processors and as a more general process for cognitive models . T . if data from to actual biological sensory systems each transmitter / receiver position ? The concept of neurons as were stored separately . Sequential estimation of a probability distribution [ 1 ] E . 1 is equivalent princinles of Neural Science . Maler . 9 . " Centrally syn - siaple averaging of squared frequency [ ] " components if the distribution esized maps of sensory space . eds . E . " Electroreception in gymnotiform in feature space ) . INSIGHT INTO SOMATOSENSATION Bullock and W. as implemented 11 . H. g . A . Schwartz . H. Drager and D. the case of the gamma distribution [ 3 ] V . sequential signal processing . " Responses to visual stimulation [ 22 ] . C. SUMMARYAND CONCWSION by an artificial neural network . Heiligenberg . . can be constructed from Trends in NeuroSciences 9 ( 1986 ) as in 163 . equivalent damage . The resulting hypothesis test for target classifi . New York : Elsevier . R. New York . J . utilization of this concept by echolocating animals would require Observed receptive field changes the capability to associate are explained by a model in the same external point perceived which the stimulus location hypothesis on different self . classifier ( e .373 . 10 . Altes by updating echo data at each present ordered hypothesis tests point in an environmental map as for stimuli with the relevant the transmitter / receiver changes parameters . 2nd to a receiver based on Ed . as in Fig . Hubel . target Estimation of echo energy spectral The resulting architecture yields density via a cumulative efficient implementation of a average is represented in Fig . imperfect exemplars are used for training . Newman and P . Neurophysiol . 1985 . fish . 1986 ) 319 . 38 ( feature values ) into a ( 1975 ) 690 . . SEQUENTIALESTIMATIONIN and lateral inhibition exists between ANIMALECHOLOCATION peripheral receptors [ 9] . estimating the probability distribution of the energy of each REFERENCES frequency component . Although this viewpoint .168 . 2 M. 21 ] . Richard A.

" J . G. 43 . Al tes . " . (Wiley . New York . eds . in adult monkeys . " Adaptive processing " Somatosensory cortical map : Time . " Natural dolphin echo recognition sifier as a model for somatosen . II Sci . Amer . Neuronal Parametelo Maps and Signal Processing snakes . J . . A. 1586 . W. Eigen . The extent to which [ 17 ] R. problem of consciousness . eds . Altes . and architectures . Am. A . 315 . [ 7 ] W. Stocklin . A . 42 ( 1967 ) 773 . Budapest . [ 12 ] E. . Lippmann . Psych : Animal Behave location . J . 1992 ) Rose . Neural Speech . Cat . 3 (Sept . [ 20 ] H.95 . on Aerosp . York . Heiligenberg and G. New York . and JiM . L . Waltz and JiB . Edelman . II in Electrorecention 116 . on Acoust .159 . " Dynamic Press . R. " in Sensory Abilities Proc . Nachtigall . Thomas . on Systems .612 . van Schooneveld . Crick and C. eds . Proc . Parzen . Penner . Am. and phy - . 1973 ) 647 . Zook . New [ 18 ] F. Cynander . Sakoe and S . Theory . R.1 ( 1991 ) 130 . Exp . Theorv and Its ADDlications M. " Generalized mannia : La belling of physiologi . Am. vision . W.340 . Nolte . ( Academic [ 24 ] H. 246 ( 1982 ) slology .varying parameters changes following digit amputation . [ 19 ] Altes . E. A . 1984 ) pp . W. Altes . L . Whalen . R.510 . W.656 .141 . P. (Wiley ." Cognitive Science 9 ( 1985 ) 51 74 . II IEEE Trans . " J . the line segment cally identified cells . H. M. Nachtigall . " Nonlinear for spoken word recognition -neural networks : Principles . and Neurol . 331 . W. of Agyatic Mammals . Comp. London . Bastian . " Massively parallel parsing : A strongly interactive model of natural language interpretation . M. and 22 ( 1987 ) . A . Touretzky . Penner . P. Sandell . II Proc . IIMatching - transform and sequential - to sample by an echolocating hypothesis dolphin testing in dolphin echo . R. 501 . [ 21 ] H.777 . 1991 IEEE International [ 8 ] R. Sianals in Noise (Academic .61 . ASSP. and ASupin . Cowan . in DYnamic As Dects of Neocortical and Electron . No.1593 . 85 ( 1989 ) 934 . qymnotiform electric fish . " sensors . " Sci . 16 ( 1990 ) 85. New ( Plenum . Stryker . . 1992 ) . D. J . tomography . M. eds . E. based recursively trained clas . 281 .26 ( 1978 ) Networks 1 ( 1988 ) 17 . Sys . R. Soc . Soc . Moody. S. E. Modern Probability [ 23 ] M. . [ 11 ] R. and W. Acoust . Roitblatt . wavelet analysis . anatomy . Gall . using an integrator gateway network sation . " Ope cit . " Theory of signal D. R. B. liThe York . AES. Suga . " Electrolocation : Behavior . Roitblatt . J . San Mateo . [ 6 ] N. c . biosonar information is represented Jr . Griffiths . L . II in Advances in Neural In - Man . Pollack .605 . . R. Hungary . 91CH3003. P. " in Sianal Processing . 224 ( 1984 ) 591 . and R. " Detection with distributed in the bat auditory cortex . and [ 10 ] L .49 . Nolte . Grossberg . J . 1971 ) 112 . H. 267 . " IEEE Trans . Big . and Cybernetics 21 ( 1991 ) formatign process ina Systems 3 . Koch .942 . programming algorithm optimization [ 14 ] S . [ 13 ] L . Merzenich .- [ 16 ] J . " Neuroscience transform . " The line segment and P. No.124 . Moore . P. A histogram . " IEEE Trans . S . Nelson . 1960 ) 136 . 1991 ) 273 - receiver design . . 22 [ ] A. L . Detection of Kastelein . M. II J . Chiba . " [ 9 ] R. " An interpretation Symposium on Information of cortical maps in echolo . .373 . mechanisms . " The optic tectum of the 153.17 ( 1981 ) Function . [ 15 ] D. IEEE cating bats . E. (Morgan detectability : Adaptive optimum Kaufmann . " J . P. Tenney and HiR . Schoppmann . 577 . Acous .

2DavidA. is to incorporate what is already biologically relevant mechanisms is likely to yield known about the way in which the biological systemsbeing effective synthetic systems as well as understanding modeled representimportant information . we can use the information feasible. There are no well The effectiveness of artificial neural network modeist established rules to guide the selection of these dependsstrongly on the way in which the information to be learned is presented to the network. These networks also offer a useful categorization.l & PaulE. biologically relevant mechanisms. signal detection in noise. 1980) that the dolphin uses in object . processes ( Roitblat et al. B. on the way in which the infonnation to be learned is presentedto the network. is likely to help . A ~ tract depends. The model yielded systems can benefit from the same. Dolphin & Sonar recognizing objects and for navigation. Complementarily. in large part.eural Network HerbertL. However . We developed the features of the environment that control the animal's a model of the dolphin cochlea and used this model to behavior and to identify the means by which such produce the representationsused by a neural network information is represented. Helweg. perceptual and conceptual mechanisms for perceiving and 2. so there is ample background and a well - computational technique for modeling and studying developed methodology for investigating these capabilities. Finally . biological processes both because of their effectiveness in They provide a well -bounded problem and demonstratethe biologically important tasks and because their structure solution of that problem. Nachtlgall2 l Universityof Hawaii-Manoa . as those used by the biological system. an obvious and plausible source for direction in this process. Moore. Their categorization performance has been well studied. or close to the same psychophysical functions and matching choice mechanisms.l PatrickW. Introduction increasing fidelity . speech recognition artificial neural network models of biological signal and adaptive control . in a sense. We can. that is. Use of representations.. Use of accuracy similar to those obtained from the dolphin . Roltblat. to identify the perfonnance of biological systems. Dolphins are of immediate practical importance in gained from investigations of biological performance to that they can be trained to recognize practical signals from guide the structural development of artificial systems that real objects and to perform important tasks with those " accomplish analogous tasks. 1990) and for the sensory processes (Au . reverse " objects. thereby indicating that biological - resembles natural biological information processing sonar-based recognition of various kinds of targets is architectures. this laboratory has already begun the engineer computational models from successful biological development of various kinds of models for the decision systemsto aid in understandingthe biological system. Models aid " animats" dependssb' ongly on the development of adequate understandingand they facilitate applications. Representation and Processing of Acoustic Information in a Biomimetic N . Artificial neural Hearing networks are uniquely suited for solving problems in pattern Dolphins provide an excellent paradigm for developing recognition. represented with 1. and are likely to play an important role inanimat development. 2NRaD/ NCCOSC The effectiveness of artificial neural network models. Col Wtruction of artificial to model the delayed produce increasingly effective synthetic systems and to help us to understandthe The deployment of robots and other artificial creatures or performance of their biological counterparts.sample performance systems that implement functiol Wperformed by biological of a botdenosed dolphin .

Suthers . 1985) . Because the dolphin must emit would have a cone. fusionandenlargement of Moore & Pawloski. narrowerportionsof 1966). frequencies are representedtoward the apex as the membranethins andbroadens( Ketten. The through the oval window and excite vibrations on the basilar quantal nature of dolphin echoes greatly simplifies the membrane. like that of other mammalianspeciesvaries times(four octaves ) higherthan can be detected by humans . which in most mammals is a spiral rather snail- infonnation about the location and many features of the shaped shape with the oval window clicks in order to engage its biological sonar and because occupying its base.. Dolphin echolocationclicks are broadband . with certain In addition to excellent visual capabilities comparable specializations. and contain both narrow band constant . 1984. and specializationsseenin the cetaceanear are isolationof the sourcelevelsof up to 220dB re: 1 uPa or higher(Au. ears contain approximately 105.e. Acoustic Information in a Biomimetic Neural Network recognition. &. which is in contact with the interrogate their environment by sending out clicks from atmosphere and is the typical means by which sounds are their rounded forehead or melon. Ganglioncell densitiesarehigherin odontocetesthan in any other mammal and the ratio o~ . The standard mammalian ear consists of a to those seen in other terrestrial mammals. Fletcher. averagingabout2. ganglion ce Us per min. spanningover seven supportit. 1990). Wever. with in about100m) tasks. their signals are emitted. 1980.. basilarmembrane(i. 1959.. Johnson . The dolphinis sensitiveto frequencies sevento 10 membrane . specialization. Although both bats and dolphins use wide frequencyresponse . whereas. 1992).lSec ). from which the dolphin can extract the cochlea.500 ganglion cells distributedalong 31 contrastthe dolphin echolocationsignal is very broadband mm of basilar membrane . Among the peakenergyat frequencies rangingfrom 40 to 130kHz. monotonically from the base to the apex. Cetacean cochleas correspond generally to the standard mammalian design. frequencies areencodedin the thicker. Both the dolphin's signal productionand hearingare 1971a. it object ( Nachtigall. and have developed some preliminary neural The anatomy of the dolphin ear also displays aquatic network models ( Moore et al. 1863) .g.526ganglioncells frequency and FM -modulated components depending on the permIDof membranelength). and running the length of the the clicks are separatedin time from other clicks . 1971b). The thicknessand width of the dolphin basilar octaves . & Pawloski. for an averageof about 984 andextremelyshort(about50 J. 1984) and a basilar membrane and ~ seous spiral laminae. By containsabout 30. which wide rangeof frequencysensitivity. The clicks reflect from transduced from the environment into the hearing system. 1988. bulla from the surroundingskull. Fenton. objects that are directly in front of the dolphin and return The tympanic membrane connectsvia a series of ossicles to characteristic echoes. 1992. Bottlenosedolphins have the ~ sicular chain. 1980) . Within . 1944.e. Helmholtz . 1988). Tursiops air. The dimensionsof the basilar membrane specifically adapted to the underwater acoustic suggestthat dolphin ears are capableof an exceptionally environment . 1963). Moore. lIya. and specializationsin the shapeof the excellentdirectional hearing(Au & Moore. have behaviortheseanimalsexploit (Ketten. 1991. 1991) . Dolphin ( Hudspeth.dolphinbi~ onaris adaptedfor useunderwater .. 1967). The inner hair cells contact the dolphin (Tursiops truncatus) far exceedthose of any basilar membrane and transduce the movement of the artificial systemsat objectdetection. the characteristicsof the medium in which theaudiometricdata. different acquisition mechanisms. Sounds enter the cochlea ' corresponds to a discrete packet of information. dolphins tympanic membrane. the mechanismsby which the signals are produced . per mm ( Ketten . each echo cochlea is the basilar membrane. 1988. the type of signals and the Hair cells are approximatelyevenly distributedalong neurologicalapparatusthey use to process es thosesignals the membraneat the densityof about 100 inner hair cells . they produce and to the particular habitat and feeding 1986). Highest and can detectfrequenciesas high as 150 kHz (Johnson . frequencies differentially excite standing waves at specific locations along the membrane (Bekesy. recognitionandmany membrane into neural signals for processing by the brain otherclose-range(i. The biological sonar capabilitiesof the bottlenose 1940. 1988. Bat biosonaris adaptedfor usein cells. The hair cells synapseon ganglion Zvorykin. whoseaxonsform the auditoryeighthnerve. . In comparisonthehumanear species(Bellwood.000 ganglion ceUs Bat biosonarsignalsare relatively long in duration(up to distributedabout evenly along the 41. Roitblat et al. This predictionis consistentwith echolocation. echolocationsignalsemergeas a seriesof ultrasonicclicks from the melonas a highly directionalsoundbeamwith 3 Species -specificanatomicalcharacteristicsof the ears dB (half power) beamwidths of approximately100in both of whalesand dolphinsrelate to the echolocationsignals the vertical and horizontalplanes(Au. Morgane. 1992).6 mID length of the several ms). Because of the shape and stiffness problems associatedwith modeling continuous information characteristics of the ~asilar membrane.. differ substantially( Zook Myron.80 kHz range(e. The animalis maximallysensitiveto frequenciesin the membranenear the base and progressivelylower the40 . If the cochlea were to be unrolled .

each hair cell respondsmaximally to a limited range ( Rennan&. & ' Sokovykh. = . If thedF is The frequency response of the basilar membrane and the smallestdmcriminabledifference. kHz. k -90 . 1990). . Morozov. Hennan. human. at 100. are differentially sensitive to a particular frequency band. 1972. . An of frequencies according to how much those frequencies exponentialfunction fit to thesedata had an exponentof affect its location on the membrane. - . . At filter . ' 0 .---0 truncatus / ee ctronlcus Turslops -60 -70 Homo f--FFTsapiens bin Samples -80 edri . Akopian. smaDestdisaiminable differenceis approximately800 Hz Hence. Albeit. supportingthe near linearity of this combine activation from multiple hair cells distrIbuted function. -130 ~ . dolphin frequencydisaimination limens(AF) above150 kHz. 1971). the cells located in specific portio~ of the basilar membrane. . insensitiveat low frequenciesand at very high frequencies For example. f . then AF/ F = K . " ' " + . The dolphins hearing is relatively Somepropertiesof dolphinpsychophysics are known. Hearingtendsto be most sensitivein the ' correspondapproximatelyto Webers Law: A stimulus rangebetween40 and80 kHz. -100 ~ . mustbe inaeasedby a constantproportion(AF) of its value ' to be "just noticeably different" ( Weber . . . 1967. Zaytseva . along a limited range on the membrane. . 4 . / t . . Roitblat et ale Comparative Audiograms . dolphins can discriminate earlier.-.Turslops . The ganglion cells approximately1. ' Figure 1 shows the dolphins audiogram(Johnson . 4 ~ ~ I I I I I III I Il I Il If + + i . FFr bin samplesshowthe frequencyresolutionof the underlyingspectrumon which thesimulationswerebased. ganglioncells to inner hair cells is abouttwice that in bats and aboutthree times that in humans( Ketten& Wartzok. Because of the cochlear characteristics described frequencies near 1 kHz. / " -120 at . and as a result. 4 10. . andsimulatedaudiograms . . . O . a the dwtnbution of inner hair cells and ganglion cells suggest constant . - " I .1 2 a 100 2 a 101 2 a 102 2 a Frequency(kHz) Figure1. . Anotherway of statingthis relationshipis thatthe that each ganglion cell functions as a band-pass frequency graph of log AF versus log F is a straight line. . Dolphin. . " : 0 -170 . . 1846). . ' . Herbert L. " / + ' -110 aIr . Donskov. 1bompson&. .+-.09. specifIC frequencies differentially excite the hair frequenciesdiffering by about 6 Hz. . + . 1975). " -140 0 -150 -160 W - . Burdin. .

oyCkHz) Figure2...-~. .".. . constituents.". "\'\Aj \. .~ "-"'" 1 ~ 881- """"'-"'-II. Signalswere digitized It SOOkHz removedfrom the waterandthreeothertargetswereplaced placed an RC-EleclronicsAID cOnverterand custom-written in the water. / .. . .'_. the dolphin stationedunderwater and directedits in the path betweenthe dolphin and the centertargetsInd echolocationclicks at a submerged targetlocatedin front of echoeswere obtained using a custom-built hydrophone ' it. differing only in their internal matchingcomparisontarget. TheDolphin's Task We capturedand digitized the dolphin's outgoing clicks and returning echoesthat were directed at Ind The dolphin performed a matching-to-sample task (see Roi~ lat et al." t 1 . 0 100 n. . The dolphinthenecholocatedon thesetargets using software ._.glycerol . so the dolphin had to identify All stimuli were and rememberthe samplein order to correctly chosethe physiological saline.. bottles containing nndomly from trial to trial./'V "\. .I'~ "" ~ ~""'f'I'v. 1990for details)... .-"""""'"' ~ 1J ~". Acoustic Information in a Biomimetic Neural Network Glycerol ./ ' -. & + 1& . ..J \ . ..\ r . and ke~ ene. it was next to dolphin. . Followingthe dolphins examinationof this target./" 0.----. ."-""'. . tv".". echo wavefonns and spectra are Example . " I-J ~ rvv vv KerO8. Wearingsoft removeable reflectedfrom the sampleandthe centercomparmon target.~0) aoono 800 -.l' W"J~ ~ /'v. 3. Cicks wereobtainedusingI B&K 8103hydrophoneplaced eyecups ... Example waveforms and spectra for the three target types used. VI\I\f~'\iII-"""" ~~'. \jVVV Y \ f\ . identical in their outer form./'"". andselectedthetargetthat mostresembledthe sample. 0 ao 40 80 80 100 180 140 180 180 aoo Pf. The identityof thesamplevariednndomly from trial to trial and The stimuli used for the matching task were fuel the location of the matching alternative also varied different materials.

alternatives is 33%. For computational reasons and computational constraints (e. 4 ' " 0 ' 4'"0 ~ 00 ~ ~ o ~ Figure 3. shown in Figure 2..g. Each bin could... In previous investigations we had . sampled at 500 kHz and transformed via an FFr to a 128 bin spectrum . These stimuli were apparently very Our model also includes a further cons1raint in that we difficult to discriminate because the dolphin 's choice wanted to be able to compare the performance of the model accuracy was only 50%. 0 ~ . the capacity of the ' in order to compare the present model to previous models . computers memory and time to perfonn the computations) . we needed a 30 bin vector to summarize the some are dictated by other physical or electronic constraints. epn 0 co CI ). 0 ~ . The filters used in the cochlear model. Because the of the falls off so any model.. Some of these assumptions are dictated by sharply between 140 and 150 kHz we selected the first 8S bins for further processing . 4.95 kHz wide band .. Herbert L . and we wanted to compare the model to the dolphin 's used to create principled inputs for our neural net model of the dolphin 's decision processes. ~ . In developing represents the energy in an approximately 1. Roitblat et ale FilterShapes ~ In. Chance accuracy with three with that of previous models we developed for similar tasks . one must necessarily make simplifying hearing dolphin assumptions. relevant frequency infonnation . Model ' Our basic data for to the network consist of We used the known facts about the dolphin ear and hearing input to construct a computational model of the dolphin 's ear that digitized ~ 6-point waveforms..

The evidence described above. This vector was also fed to ' the bank of filters J also resulting in a 30. During the first the stiffness and size of the basilar membrane. Many of the panmeters were selected on the In producingthe 3O . Bandwidth also varied as a function corresponding to the tested frequency for that trial was of the filters location on the basilar membraneas in Eq. meaning that it was most sensitive to its center frequency and had diminished sensitivity with increasing distance 5. 3 is that the spacing of inner hair number of inner hair cells and ganglion cells in the dolphin cells and ganglion cells is about constant over the length of ear is approximately constant over the length of the basilar the basilar membnne . nevertheless. AU frequencies in this earlier model were accordingto Eq.element vector.1392 x ( i the response of the simulated ear when confronted with ( 1) noise only versus when the system. The resulting audiogram is also shown in Stepsizeis a constantconespondingto the bandwidth of the FFr bin ( 1. in other words. The total sensitivity of each ffiter was set 150 kHz. Results from the center.139 sensitivity = ( a x 2) suggests that in the ear. The center frequency of each of the 30 filters was selectedaccording to the corresponding In the present model we sought to generate the frequency. (J = 40 x 0. that frequencyandaddedto the outputof that filter.00201 x (Center Freql . Near the base. the filters are approximately equally spaced random value. The The ntionale for Eq. . These values were then fed to the bank of fdters ( the matrix product of the noise vector and matrix of along the length of the basilar membrane. so the number of cells that are likely membnne suggesting that each unit of length contrIbutes to contribute to the filter should be inversely proportional to about equally to representation of the signal. The nnge of the spacing between filters ( Eq. This bandwidthis detenninedby the numberof pointssampledin the original signalandthe .element Each of the filters is characterizedby its width as well vector) .only and signal+noise vectors. According to part of the trial each frequency bin was set to a small this equation.093) x stepsize The discriminability of the two conditions was taken to be (2) the Euclidean distance between the noise. This equation ( Ketten. 1992.953125kHz) in the underlyingoriginal Figure 1. A new noise vector was then specified and the bin as its center frequency. each from a wider nnge of inner hair cells than filters that are unit of basilar membnne length representsa broader range spacedcloser together. lower frequencies relativeto eachfilter's centerfrequency. however. and the resolution of the model eighth-nerve responseof the simulatedearJ the power in was fairly crude. of high frequencies relative to the narrow nnge of lower frequenciesrepresentednear the apex. Each comparison was repeated 10JOOO times. Acoustic Information in a Biomimetic Neural Netwol ' k avenged pail S of adjacent frequency bins between 31 and samplingrate. 2: incremented by a fixed value. Filters that are spaced frequencies represented by each segment of basilar farther apart are expected. personalcommunication) is an estimate derived from Each simulated trial consisted of two parts.was confronted with a in which i ranged from 1 to 30.elementvector representingthe basis of rough estimates. filter vectors was computed resulting in a 30. 1: A simulated audiogram was obtained by comparing CenterFreq = 160 x EXP( ':'. Notice that thebinsarearithmeticallysymmetrical . however. signal of a particular frequency in addition to the noise. is not constant. 3: represented equally--each bin represented a constant bandwidth. Each filter had a Gaussian shape. bandwidth varies with the frequency being measured . FFT spectraldistribution. The sensitivityof eachfilter wassetto be proportional the heightof the Gaussiandistnbution. such exercises are likely to eachbin of the underlyingFFf distn'butionwas multiplied be valuable in the long run in the development of effective by the height of the Gaussiandistributionfor that filter at biomimetic mechanisms. Figure3 showstheresultingfilters. representationalvector to correspond to known features of dolphin ears. Lower frequencies are representedwith naRower band wchannelswthan higher frequencies. On a log scaleJ The sensitivity of the ear was representedas a bank of they would appear to cut-off moresharplyat higherthanat 30 bandpass filters . to receive inputs membnne . 1) .

e.. cochlear model as the "front end" of the network In another simulated experiment we compared the transf~rming the underlyingFFf into a vectorof 30 filtered difference limens predicted by the model with those amplitudes . but it could just as easily be implemented Figure 4. These limens are shown in preprocessor . Herbert L. The crudeness of the underlying sensitivityof thefilters.2 1.2 0 20 40 60 80 100120140 Center (kHz Frequency ) Figure 4.7 0. These limens correspond closely to the observed directly in the network by setting the strengthsof the frequency IlmeM . we used the and henceof the redundancybetweensuccessiveechoes .2 2. modeldescnbedearlierby Moore et al. ( 1991) andRoitblat et al. Roitblat et ale Limen Frequency 3.7 1. That is. It takes advantageof the observationthat that of the dolphin on the difficult task of recognizing the dolphinstend to emit a seriesof dicks to the sameobject su~ tance inside the test bottles. frequency analysis ( i. We implementedthe cochlearmodel as a observed in actual dolphins. . at least in the middle range of the connectionsfrom the input layer proportional to the ' dolphin s sensitivity .. we compared the performance of a neural underlyingitem-recognitionprocess esthoughtto be usedby network using the model as the input transducer to the dolphinsin the delayedmatching-to-sampletask(Roitblatet neural network and comparing the network's performance to al. ( 1991).9S kHz wide bins) limits our abDity to For the neunl networkwe usedthe integntor gateway make fme disaiminations at low frequencies. Smallest discriminal difference as a function of frequency. This networkwas designedto implementthe Finally . the fact that our original representationuses 1.2 0.7 ( 2. 1990).

0 indicatesthatthe itemwas ' incorporatingthe cochlearmodel neededto reach a 0. I 1 . Acoustic Information in a Biomimetic Neural Netwoik 1 . conventionwe havechosen0. . The performanceof this networkis shownin the form of confidencerati~ (seeMoore et al. I 0.96 and then stop clicking.0.. The present suggestingei~ er that the dolphin does not control his . 0 . By changein confidenceoversuccessiveclicks.0 3 .0 7 8 .....0.8 0 8 \ . over estimatehis ability to extract information form the signal. functionsperformedby biological organisms ./ 5 .7 0 8 5 o .------------ 1 --..96 correctly and reliably identified by the network.96 asthedolphin's aiterion of confidence .0 '--'. Conclusions clicks to a targetuntil im confidenceratio reaches0. 1991.0 . A confidenceaiterion andthe numberof clicks emittedby the confidenceratio of 0. he has continuedto click beyond the point at Thebiomimeticapproachholdssignificantpromisefor the developmentof artificial systems that mimic the which our network reaches this confidence criterion.-. The network combinessuccessiveechoesin a train and echolocationclick productionon a click by click basisor we computesa functionsimilarto a runningaverage . The confidence of a correct classification basedon network output. Table 1 shows the number of clicks the network 1991).- Glycerol Targets --- 0 .0.0. In everyexperimentwith this dolphin.. A confidenceratio of 1.0. however.0.4 0 .8 Saline Targets 0 8 eouepl 0 .0 indicates that the network has dolphin on eachtrial.1 0 --------- Kerosene Taraets ~g 10203040CLICK 5080708080100 Figure 5.2 ---I . Roitblatet al. Ideally the dolphin should continueto emit 6.2 0 3 0 . Figure 5 showsthe courseof the misidentifiedthe targetas one of the alternativeitems.

W. W. (1988 ). Thelastcolumn criterion indicateswhether thedolphin chose (+) orincorrect thecorrect target (-) ontheindicated trial.251 -282 ). waters. NewYork : Plenum andneuralanatomy . andbehavior . Herbert L. . L. Roitblat et al. Moore . W. Echolocation signalsoftheAdantic theAcoustical America 75255 -262 - Society 01 botdenose dolphin(tursiops tr"n.Anunal beam oftheAtlantic Echolocating transmitting . P. G. psychophysics .. Kerosene Saline 9 - Kerosene Kerosene 2 Kerosene Kerosene 5 + Note: A tninisa sequence of echolocation clickstothesame Target. Table 1. L (1980 ). detectionandrecognition performance .asmall portionoftheircapabilitiesinartificial systems performance (pp . .). developan effectivemodelof echolocation object Au. FISh{Eds . InR. E. Nachtigall & P. W. Number of clicksis thenumber of clicks /echoesthe network needed in order to reach the confidence . Au. InP. W. W. Dolphins recognition are capable of exquisite object dolphin sonar systems . L.Anima I Sonar : Process esand even. B. W. P. (1986 ). B. Journal01 Au. bottlenosedolphin Tursiops truncatus . W. & Pawl ~kiD. (1984 ). B. to Press .). W. Detection andrecognition models of . Integrator Gateway Network and Dolphin Decisions Train Target Decision Number Dolphin # of Cicks Correct 1 Glycerol Glycerol 3 - 2 Glycerol Glycerol 3 3 Glycerol Glycerol 6 4 Glycerol Glycerol 2 5 Glycerol Glycerol 12 - 6 Glycerol Kerosene 23 7 Glycerol Glycerol 3 + 8 Glycerol Kerosene 3 + 9 Glycerol Glycerol 3 - 10 Glycerol Kerosene 13 1 Saline Saline 3 + 2 Saline Saline 3 - 3 Saline Saline 3 - 4 Saline Saline 19 + 5 Saline Saline 21 6 Saline Kerosene 7 + 7 Saline Saline 4 8 Saline Kerosene 6 9 Saline Kerosene 17 10 Saline Saline 5 1 Kerosene Kerosene 4 + 8 Ntt 0 Kerosene Kerosene + Kerosene Kerosene 4 + Kerosene Kerosene 6 + Kerosene Kerosene 2 - Kerosene Kerosene 2 + Kerosene Kerosene 17 +. & Moore . Busnel& J. Incorporation of Moore (Eds. 753 -768 ). model incorporates whatweknowabout dolphin cochlear sonarsystems (pp. L. Press . W. New York : Plenum would gready enhancetheireffectivenessandcapabilities.catus) inope~ Au . W. F. Receiving beam References and patterns directivity indices ofthe Adantic . Thethirdcolumn shows thedecision thatthe network reached.

G. NavalOrdnanceTestStation. ( 1988). P. ( 1940) Auditory patterns . ( 1971) Repetitionrateof rangingsignalsof dolphinsasa functionof distanceto target. ( 1972) Frequency ( I .. G. V. R. Akopian. (pp. J. E. Ketten. . E. B. H. Nachtigall. Kastelein V. Handworterbuch der Physic henWagner. ( 1990). Acoustic Information in a Biominletic Neural Network bottlenosedolphin. L. ( 1985) The cellularbasisof hearing: The by batsandbirds. 68. Tursiops truncatus Weber. S. 750) New York: SpringerVerlag. Palin. Neural Networks. McCormick. New York: PlenumPress . W. & Pawloski. Moore( Eds. New York: Plenum.). Bekesy. ( 1863) Die Lehre von den Behavior Processes. J. F.) Animal sonar: Processes and 281). neural infonnation processing systems3.. the National Academy of Science. ( 1992) The marinemammalear: Wever. L.M.. V. Nachtigall& P. Myron.. Gutol . 49. Akopian.) Sensoryabilities of cetaceans(pp. L.140kHz) andhuman( I 8kHz).sampleby an echolocatingdolphin. H. . S.55. Busnel& J. & Ridgway. 607 611). ( 1988) The productionof echolocationsignals Hudspeth . Donskov. E. Journal of Experimental Psychology: Animal Physics. abilities in bat Arkh . Embriol . R. Sokovykh. G. R. & Morgane. (Tursiops truncatus).. . & R. W. (vol. A.) Animal sonar: Processes and - the controlof echolocationpulsesin the dolphin perfonnance (pp. Palin. Nachtigall& P. Tonempfindungenals physiologische Grund/ age Thompson . L. D. In P. Nachtigall.. W. S. In W.. . J. J. in echolocation phyllostomine bats ( phyllostomidae) G. L. New York: PlenumPress.. 109. J. Touretsky. P. NaturalDolphinEcho Fenton. ( 1990) Fletcher. Roitblat. Fish(Eds. Webster. R. V. Zvorykin. M. E.P.H. 81-106). A . Zaytseva . . New York: Plenum. frequencydiscriminationin thebotdenoseddolphin - Hennan. Journal of theAcoustical Morozov. & Herman. 36. E. Moore( Eds. M. E. Roitblat. L. Proceedings of The evolutionary biology of hearing. N. In R. In P. Thomas& cytoarchitecture in echolocatingmammal- R. Tavolga( Ed. P. Foragingbehavior. W.) Animal (pp. 305 316). H. Suthers . E. Nachtigall& P. R. Journal of Auditory Research. von ( 1944) Uberdie Frequenzau Oosungin der locationssignalsof dolphinsasa functionof the menslichenSchnecke . 16. Penner . S. .R. Thomas& R.. Reviewsof Modern Matchingto. J. Lippman( Eds. P. 57. Journal of the differencelimensin thebottlenosedolphin: 1-70 kHz. Helmholtz.). 17. H. CA: MorganKaufmann. ( 1846) TastsinnGemeingefuhl . 247-260). E. SO. (1967) Sounddetectionthresholdsin marine Wever.G. Recognizingsuccessive ultrasonicandlocationalcapacitiesin the dolphin.H.709 . W. 2.. P. 23 45). 311 316). 139-143. I. Roitblat. R. and performance on objectsize. E. C. Moore. Moore. S. 12. New York: truncatus: Generalmorphology.. B. Marine ( 1971a)The cochleaof thedolphin.. 60-84. Biofizika .. P. 4 701 . P. N. W.( 1991 ) . 481- (Montague) ( NavalOrdnanceTestStationTechnical PublicationNo 4178). Penner . & Sokovykh. D.65. In an five speciesof insectivorous bats--implicationsfor D. ( 1991). (1988) Variationsin foragings1rategies in RecognitionUsing IntegratorGatewayNetwork. supersonic New York: PlenumPress ..) Sensoryabilities of cetaceans Speculations on the morphologicalbasesof time- domainsignalprocessing . sonar: Processes and performance (pp. & New York: Plenum. E. & A. Zook. 230. Embriol . 943-947. . McCormick. 71-95). F. 717. shape. Y. Animal Sonar Systems In P. B. P. 68. 45.. R. Biofl Sika. D. Nachtigall& P. bottlenosedporpoise. M. H. A.. 32. MooreP. Tursiops In DR . & Society of America .120. 2381-2385. I. S. distanceto thetarget.) truncatus: Thebasilarmembrane . V. R. 17. In J. & Wartzog. W. 2908-2912. Y. Gutol . - performance (pp. SanMateo. Fay. Specializations for aquaticauditionandecholocation . ( 1966 ). 588. (pp. In P. preyselection . & Arbeit. Moore( Eds. B. 85-95. I.L. New York: Plenum. ( Eds. Johnson . dolphinechoeswith an integratorgatewaynetwork .H. ( 1971b) The cochleaof the dolphin. R. Popper( Eds. P. K. Acta oto-laryngologica.. (pp.. H. K. Tursiops bioacoustics. - 12 47 . H. Odontoceteecholocation Bellwood. ( 1959 ) Morphologicalbasisof locativeand . ( 1963) Morphologicalsubstrateof Nacbtigall. ( 1988) Ketten. Burdin. & Zvorykin. Kastelein( Eds. Moore( Eds. Ant.688-691. A. R. D. P. J. W. G. Advances in echolocationcall design. K. A. 19. Invcstigationson B. C. Science. Morozov. Penner . In J. B. Acoustical Society of America . E.. H. & Ridgway.)... J. National Academy of Science. ( 1980). A.. ( 1990) Threedimensional Somecomparativeaspectsof auditorybrai~ Tern reconstruction of the dolphincochlea. Proceedings of the PergamonPress. Arkh. ( 1972 ) Trackingfrequencyof the . A. biophysicsof hair cells.. Glezer.. Auditory thresholds of the perfonnance (pp.andmaterial. 3. Zaytseva .) Animal sonar: Processes and - Johnson . & Nachtigall. 3. B. . ( 1975) Underwater fur die Theorie der Musik. 745 752. 273- B. pp. Ant. mammals . 601-605).

well as some challenging new problems of how one integrates all of these functions. and then demonstrating logic of a number of individual programming modules. visually We have chosen. It is not intended to be a valid reductive explanation of the actual inner workings of . decision making .BITNET ABSTRACT Thispaperpresents a reporton theprogressmadeon thedevelopment of a computational -motorbehaviorof a predatory"SWIMMER modelof theperceptual ". andnavigational aspectsof the simulationarepresented.. We place primary emphasis on the recognizing and discriminating among salient (edible) integration of a suite of algorithms and processes that and irrelevant ( inedible) objects. Our complexities involved in perceptual-motor behavior. kind of behavior from an entire organism as opposed between our program components and the processes to a model restricted to a single internal stage of of the visual system: information processing. Arizona85287 email . Introduction : the brain or mind of any real organism but 8 description of the transformations and processes that We report the development of a computational model have to be accounted for if we are to understand the of an entire perceiving and responding organism. an underwater SWIMMER the general organization of the system in this of capable acquiring images of food objects. presentation. Briefdiscussions of thevisual . decisionmaking imageprocessing . The work can be consideredto be eithera psychologicaltheoryor the necessary for programming autonomousvisualcontrolof anunderwatervehicle . project is both a unified theory of a perceptual motor process and a step tow ~ s an autonomous.motor system. Uttal ThomasShepherd Sriram Dayanand Robb LoveD ThePercepqon Laboratory of Department Industrial andManagement . I . The actions that have to be simulated range from the initial transduction of photic energy to the The general framework of our appropriate interpretation of a stimulus scene to the simulation is made up of a number of stages of construction of a map of the world of the simulated transformation that we believe must be included if the entity to a specific effector the degree that it is successful -.100 AN INTEGRATED COMPUTATIONALMODEL Pr R CE U A L-MOTOR OFAPE SYSTEM William R. establishing a three- constitute a theory of a perceptual. as some examples of the relations . organism. underwater vehicle. Involved in such a simulation considered to be tentative and replaceable as the must be consideration of visual . technology for executing that particular subprocess interpretative. as a prototypical microworld guided. Thust our system is to perform in a comparable way to an simulation . Systems Engmeenng ArizonaStateUniversity Tempe. its understanding by swimming through a turbulent Each of the functional steps in our simulation is ocean to those objects. we see the correspondences representsa descriptive transformational theory of this shown in Table 1. . localization . and motor functions as evolves. Specifically . We shall concentrate on system to be studied. but dimensional world model of its environment and the we also report considerable progress in improving the objects in it (including itselt).AOWRU@ASUACAD.

A comparison of the computationaland organic: visual domainsindicating the process . The main philosophicalpoint we wish to GarbageOut ) . it is not be presented that . Integrated Model of a Perceptual . In other words.Motor System 101 COMPUTER HUMAN ImageAcquisition ReceptorTransduction Pixel Sampling RetinalMosaicSampling ImageSmoothingand Local RetinalAveraging NoiseReduction EdgeDetection LateralInhibition at~on ObjectSegment From Attributes of Form FigureGroundSegregation From Attributes of Motion CommonMotion Grouping From Attributes of Depth Stereopsis From Attributes of Color Color Discrimination From Attributes of Texture TextureDiscrimination From Attributes of Lightness LightnessDiscrimination ObjectReconstroction ClosureandCompletion ObjectClassification Naming. simulatedin a modelsuchasthe onedescribedin this paper In this presentation . at least .. emphasize to comefrom integrationor combinationof the efforts to producea single robust algorithm. This is the way Let us now consider the main modules in our the organicnervoussystemaccomplish es its powerful simulation model of the SWIMMER which is imageprocessingcapabilities and it is the way that our tlowcharted in its entirety in Figure 1. Recognizing ObjectDiscrimination OrganicDiscrimination The World Map Egocentric l.illustrate the outcome of the the necessarily casethat GIGO (GarbageIn produces simulation. it can be GIBO is that goodimageprocessingis more likely In (Garbage produces Beauty Out ) . a variety of displayswill programs are organized .oca1ization Navigation Locomotion es that have to be TABLE I . . Sometimes .

: R . J Ik I Ir8 I . VI8I Ia Ik Navip MIkiJW m * I G De ~ : : . . 01 l ROB I ~ = . l . : J ~ ~ Matkin . J 1111111 ~ ~ . Vttal et ale William 102 . Conti . j " ~'*"I~rIuI OI ~ W It il Ia I C . FIGURE1. . Tom " . . Conti ROB ~ J Ik I Ir8 I . - ~ : : = . . Tom 0IkIr - OI ~ Lfc ~ 1 - ~ : . . : s . uz8lkln . r - Id I I R ~ _ R W8alll8d0l . A blockdiagramof thecomputationaJI model presentedin this paper. .

B . andRedundancy ). well as the final combined output from the gravitational field operator. R + G. G . andB .Motor System 103 D . a much improved estimate of the boundaries. Lovell . " they individually 'produce. Cavanagh Arguin . indicatedby the box labeled"WeightedCombiner.B. DISCUSSION : of the threecolor imagescapturedby our camera . it can be field of the points on the operator suggested seenthere is a horizontal combination phase. Dayanand. A full threecolor outputsare more completelymanipulated . maximum. discussion of our progress and philosophy can be ( R . Following the literature ( Fractal Dimension. B + R. and Treisma. Integrated Model of a Perceptual . These include three well known in the of human vision as shown in Figure 1. minimum .n. To illustrate the combination process. as it is traced out . The main point of combinationis image on the basis of its luminance. Similarly . Living stone and Hubel . Shepherd . G . law ' s Microtextures. Both our boundaries controls the trajectory of a moving point . G + B.[ B + R] . produces an Poggio. and Lunskis .disparity algorithms are excellent estimate of the troe boundaries in the fraught with artifacts and noise. Shepherd.g . found in Uttal . Skifsted. Figure 3 shows the superimposedoutputs as texture algorithms indicated in Figure 1. . 1990) our model and Match. Area Differential. we have also made an effort to develop procedures that interpolate from the In addition to a more or less conventional punctate samples coming from the stereoscopic trichromatic combination ( R + G + B) of the output analysis system to produce complete surfaces. We assumethat this The vertical combinationof all of the operatoroutputs contour information can be obtained by analyzing an is shownin Figure 1. Shepherd. between distinguishable regions. We then look for spatial " statistical central tendenciesthat collectively produce The second combiner ( Dayanand. Uttal. The first ( Lovell.G. organizes the computer simulation so that the relatively independentoperators initially work on their The combinationandintegrationof the output respective stimulus attributes in order to produce a information from the partially combined family of more or less accurate estimate of the boundaries operatorsdiscussedso far occursin a numberof ways. we invented to emphasizeother properties (Sample . For example.Grimson shape. color . procedure in which the gravitational At the topmost level of Figure 1. whose outputs are combined prior to the dimensional images that may have short breaks in major intensity combiner. Some of these are capable of closing up two operators. the intensity attribute reconstruct or fill in incompletenessesin the combined operator produces the pooled output of four separate images. outputs. texture.[ R + G]) . we constrain their operation by information Figure 2 shows the individual outputs from the texture that has been obtained by combining the color and operators. 1992) is a slower ." At depth and motion attributes. but more sophisticated. To improve their original image. 1988. Uttal . and Thpper ( 1992) .R. to three dimensional images. shape. ( At the present stage of . we The following is a brief description of many of the also havea versionof trichromaticvision in which the steps in our simulation of the SWIMMER . These include selective them. and incorporated programs to deal with contour and Dayanand . Kakarala. Bradshaw. The trajectory . but rather to of all of the points from the operator suggested accept the sometimespoor estimatesof the boundaries boundarieswithin a local region. 1992) is a relatively straightforward spatial optical flow lines.from -motion and our version of the Marr . this point we invoke one of two possiblecombination development of our model we have not yet algorithms.[G + B] . R . and range operators.from. lead of new developments in organic physiology and and the co-occurrenceMatrix) and three others that psychology (e. there are many points of vertical combination among the operators acting on single We have also developed procedures that attributes. The general philosophy we champion is that of combination Six textureoperators(which producebetween and integration of relatively weak and fallible 15 and 20 different output boundary estimates ) are operators individually acting on the several attributes also used. However . .) A corollary of our model is that we statisticalaveragerthat plots all of the outputsonto a should not make any attempt to polish each of these single imageand then calculatesthe centerof gravity modules to a fine level of performance. in anticipation of our upcoming move mean.

some that work well here ( e. 1S15 rSr5 0 I Mu MiD Il ~ RGBa4d / 1' (( /[j". g . Note the wide variation in the resulting images.f( ~. and co- occurrence procedures operate on image texture. )" "7 \ ["J I { ' j ~ Vu /. and range operate on image intensity . These are the separate estimates of boundaries from a scenein which four objects were placed. miD. The outputs from a sample set of boundary detecting operators. micro .texture masks. . y ~ ~) ( J ~0 ~ r > . the Law ' s micro -texture method) will not succeedin defining the boundaries. on the other hand. It is for this reason that the combination process is required. The max. might work . Uttal et ale Dllfere Dtlai Area r F: r acta ':j Q ( '\ "1 { . . Others. fractal . In other situations. The area differential .104 William R. and the RGB diff and add are " opponent color " type operators. J : 90 L} \ c v~ IVI \ ~ 0 FIGURE 2.//' V 0 _dalver her 'lL8ws ie8 tra I MIero~ Ulna " onaI~ J ~- l8Ioaal -Texture Masts oeeD ~ m -~ < < 0 .

Motor System 105 FIGURE 3.When Figure outputs operator "best combine "estimate ofthe boundari of the four objects will be obtained as shown in the lower picture . a 2.picture shows the superimposition ofall ofthe from wide single pixel by either one of the combining algorithms described in the text . . The top . Integrated Model of a Pel"ceptual .

Rules of . Objects of the correct REFERENCES shape are placed at locations in the visualized simulation corresponding to their location in the real CAV ANAGH. T. color.. 479-491. . Effect of surfacemediumon visual searchfor orientationandsizefeatures . LOVELL. ) TheSwimmer ( 1992 : An Integrated autonomous . swim through LOVELL . Pattern impulses are built into the program to assist in its Recognition. S. adaptivelybehavingsystemin the near Computational Modelof aPerceptual - future.) recognitionprogramis capableof identifying partially occludedobjectsasshownin Figure4. ( 1990). Indeed. we simulate knowledge (on the part of the SWIMMER ) of the arrangement of the objects in an underwater scene. The entire behavior of . R. R. KAKARALA. li VI NGSTONEM .. Science about following a set of decision rules (go to the nearestobject first . A particlesystem The programis written so that a perfectcorrelationis modelfor comb~ g edgeinfonnationfrom not requiredfor identification. . This programwas written not only to be invariant to rotation. Vttal et ale Once the combination and reconstnlction We are especially indebted to Dr .14S6from the Office of Naval Research NCXX . G. K. . R. S. R. R. UTrAL . A Methodfor this manner. DAY ANAND . UTfAL .SanDiego laboratory to link DAYANAND. W. We believe logic of a remotelycontrolled submersible SKIFSTED . recognize . 240. rotation.... R. .. Hinsdale. NJ: Erlbaum. 16. (Submitted -- of stimulus generalization is so great that the Availablefromtheauthors . W. PerceptionandPerformance . The visualization portion of the simulation anddepth: anatomy. Harold Hawkins for processes described in the previous paragraphs has his sustained support during the course of our work . K. . andscaleinvariantpattern recognitionusingthe form andarrangement ACKNOWLEDGMENTS of patternspecificfeatures . W. S. our computationmodel and simulationto the control SHEPHERD . . 740-750. produced satisfactory boundary information and estimates made of the depth of the objects in the camera' s field of view . "andappropriatelyrespond mission. the simulated system is SHEPHERD. Our recognition program (Shepherd . BarbaraFletcherof the Naval UTrAL . the simulation multiplesegmentation modules . . ( 1992). . DAYANAND . un AL .. shift.Segregation . SHEPHERD. & regular objects and around irregular objects) which. At the present time we are working in collaborationwith Ms. . A modelof visual though simple producecomplexbehavior. 25. MotorSystem . T. BRADSHAW . S. ( 1992 ) . and Lovell. but also to DAYANAND. adaptto new objectsthat it had not seenpreviously. & TRIESMAN. & HUBEL. AND TUPPER . . S. movement ( 1988). ARGUIN. graphicallypresentedfor experimentalmanipulationin S. & LUNSKIS . ( 1992). OceanSystemsCenter-. P. A.106 William R. T. . R. 25 343-356. T. This project has been supportedby Grant >14-91-J.. C.. this will lead to an operational visually guided. 1992) is particularly powerful. S. . SHEPHERD . translation and magnification. R. UttaI. . physiology. D. recognizeits shape . & LOVELL . Our SWIMMER moves perception . texturediscriminationusingmultiple weak navigationallowing adaptiveresponses to currentsand operatorsandspatialaveraging . scenecaptured by the camera. Pattern Recognition . W. of form. M . Journalof ExperimentalPsychology : Human Dayanand . and program is then initiated. S. "search " .

.~ .'. " - :? I ~ : . ' System 107 . " : -" ' " ~ ~' . and the three lower pictures the best matches from the library of stored images.0 . ' . " ' ! 0" '.~ .') ftGURE 4.-...' ' ~ 1/ / ' I " . ~ .I./"'"-.. ' Integrated :.-:~ -. .-'.\ - . : C ~ ".Motor ~_. ' . : .. ../'""-:.'. " . .... -"-". A sample of the pattern recognition procedure' s ability to identify occluded images on the basis of a partial match. The text output from the recognition program is also indicated." ' " .."... ~ c .-. : .\-."\". "~ .-. the left hand upper picture~ the reconstructed image." . / . : ) I i " t - ..'I. The left hand upper picture depicts the original image. !( ' / . .--. :. \ . ) / ..":..' Model of a Perceptual . .i'!.1 "'/._ " .!\. .-._'. '.:". ( 7 ..

2. control the robot reactions. sensorsand motors completethe structure [ 11] . both in simulation and during real experiments. Theywill alsobe tested on a walking artificial grasshopperwhich.motor approach. P. locomotion and collision avoidance skills. with an optical velocity sensor. . Finally. The softwarehasbeen developedin C languageon a In this paper. The results obtained with thesetwo evolvingin unstructuredand dynamicenvironments. Deplanques *Laboratoired' . whenunknown sensor (magnetic oompass ) and a 7.45 FAX : ( +33) 67.96 Email : zapata@frmop53. and on a 7. SN~ KE . [7] .based drivesthe brakesand two other electricmotorscontrol the control approach which directly feedbacks sensory robot vehicle equipped the capability to react when unscheduledeventsoccur. They were then behaviors for collision avoidanceof fast mobile robots implementedon an outdoor fast vehicleand on an indoor fast wheeled robot. The network inputs are the measured third and the fourth parts will focuson the descriptionof distancesof obstacleson the robot way and its velocity. information to the robot control loop [5] . The second Input/Output boards and timers for controlling one is basedon the Neural Network technology[ 9][ 10] . Bataillon34095Montpelliercedex5 FRANCE Tel: ( + 33) 67. last part of the paper will presenta few results. consistingin directly relating inputs (stimuli) to - outputs (actions) through statemachines[ 1] .based Navigation Autonomous Reactivebehaviorsfor mobile robotscanbe definedas Kinetic Expert) is a 1/4. [3] . Zapata. The main motion is provided by an internal "ansputernetwork for the second first one is a memberof the Sensor. Novaies .basedmethodsfamily versionSNAKE ll . oontrolling an accelerationcable. The first approach is based on the description of a systemsare being implemented on a walking machine which will demonstrate learning capabilities in defonnable virtual zone (DVZ ) surrounding the robot.sensor ultrasonic and dynamicobstaclescan collide the vehicle.68020NME architecturefor the first version methodswe havedevelopedfor solvingthis problem. [4] . These two approaches have been first simulated and compared. we can consider the sensor. proximity system . [6] . we focus our attention on two different Motorola . C. Upinay.14.52. Introduction SNAKE (Sensor.its the two approaches that we have developed. P.14. is under development.based Control and Neural Networks R. The defonnationsof this risk zone.bitnet Abstract Thesetwo methodswere simulated and comparedin the case of moving robots evolving in unstructured This paper addresses the problem of reactive (unknown) and dynamic worlds. we will descooethe proximity infonnations..108 Reactive Behaviors of Fast Mobile Robots in Unstructured Environments : Sensor . due to the intrusion of In the secondpart of this paper. and this will be the casehere. The SNAKE I . Motion machines autonomouswheeledrobots : an outdoor and an indoor fast mobile vehicles. obtained direction and braking actuators. this problem is solved by a behavioral oombustion engine driven by a DC selVo. the outputs are the controls of the robot acceleration. [2] . today. The technology. and was previouslyexplainedin [7] and [8] . Another selVo motor In another hand.scalecar. more briefly.34. the walking machine. an electronic orientation For instance. two wheeled robots used in collision avoidance The secondapproachis basedon the Neural Network experimentsand. A Fast Outdoor Mobile Robot 1. 1. then implemented on two different 2. de Robotiqueet de Microelectroniquede Montpellier Informatique UMR CNRSC9928 Universitede MontpellierII PlaceE.

The robotinternalstateis defmedto be a couple (1C.S' = t/>(n ) .5' ) where the first component x characterizesthe robot dynamics(its translational and rotational velocities) and the second component . .. 1. Figure 2 is a simple illustration of this principle.levelmodules(Emergency Stopsand Dynamic Collision Avoidance ) have been both WAAL ( WdlkingAutonomousArtificial Locust) is a . and also allow this vehicleto follow walls.scalecar. The deformationsof this riskzoneare due to the intrusion of proximity information and control the robot reactions. Avoidancemoduleshavebeenimplementedon this board the larger the DVZ will be.5'. Artificial whiskersare usedto detect closeobstacles.Figure 3 shows a possible influence of Parallel C. namely The processboard is a INMOS T800 ' nansputer with 1 megabyteof RAM . allow an hierarchicaldescriptionof behavion. 2. relation cI.action loop is carried out by a single Ji1pre2 : 7Juee functiolLf differentactivation transputer board. This zone is a 2. RAT : A Fast Indoor Mobile Robot RAT (AutonomousRobotwith nansputer) hasalmost the same design than SNAKE.sensorultrasonicsystemand artificialwhiskers . Sensor. Its purposeis also to demonstrate motion capabilities indoorsafemotionin presence . General ovelView The robot/environment interaction can be described as a deformablevirtual zone surroundingthe robot.leg artificial Locust The two high.basedControl 3. Three ultrasonic sensorsprotect the robot front space while a singleultrasonic sensorprotectsits back..S to 1 mIl ) . translational and rotational velocities on the general The Emergency Stops and Dynamic Collision shapeand dimensionsof the DVz. Reactive Behaviors of Fast Mobile Robots 109 Roughlybasedon the Brooks approachof robot usingeither the DVZ methodor the Neural Network control [1]. four modulesrelatingsensonto actuaton technology approach. allowit to avoidcollisionswith its environment - unknown obstacles when moving at important speem ( from O.dimensionalimbeddingof the spherein R2 . WrAAL .like robot can also move backwardand a - actuators. linking these two components . 3. e~ pt that this electric I / IO. These modulesareparallelasynchronous es. 6 degreesof freedomrobot with only forward implementedwith Sensor based Control concepts 1 (Figure ) . . The fasterthe robot is. controlling the sensors and the The internal control or reactive behavior. The perception. of unknownobstacles A 3. process 2 3. This also hold for the . The programsare developedin . 2. called the intenzction component . characterizesthe geometryof this deformable zone. A 6 . 6.leg.


Thesevirtual blinkersmakethe robot look the direction it will moveon. Otherwise. The slowingdown). . its translational suchanavoidance : OotoSTEP1.S' and SA vomare proportional to the Stop translational velocity v. 1. there Emergency Stops 0 Avoidance is no reaction (FigureS) . oomposed of severalneuralcellsinterconnected.S'A. 1\\'0 circular interaction zones S ( Stopfor the EmergencyStopsprocedureand . : : 8 u sampled interaction components .Ifthere The ElementatXNeuralCell : A NeuralNetworkis is not such a defonnation : Exit . 4.PJ : Checkthedeformation ofEAvoid . Collision Fipre 5 : Sampled viItualzones protecting 3. motionexists velocity v and its rotational velocity 9 . Thesetwo zones Information + . . . Lew srl' E. The robot dynamics3t is given by .S'Avoid we : 3 have:R -= = k X v + . These coefficients highly dependon the characteristics of the ground and on the reaction times of the selVo. If such is not the case.axle distanceL and to the 5 : Changethe robot velocityin order to 9I' F1P turn angled of the robot by : 8 = X tan ( d ) .S'StopMd A 8 Avoidare computedas the intersectionsof homothetic transformationsof R and 2 Hmiting 6 . obtaina homotheticdefonnationof 8 Avoid (by t . . calledneurons . $l ' EI21: Checkthedeformationof R. . ~ .oriented angular zonesworking asblinkers. are deformedwhen B is smaller ~ an 8 .9l' !P4 : Look for a motion(rotationin the robot of theNeuralNetwork thearchitecture . .2 : ComputeS Al'Didwith respectto the velocityof therobotandits rotationalspeed . The Neural Network Approach 9l' P . . . 3. referenceframe) which allows a rebuildingof . J r the Dynamic Collision Avoidanceprocedure) are sampledby usingthe proximity data provided by the 7 ultrasonic sensors. These ones are directly related to the inter . If thereis no deformation : Exit.The sizes of . range manifold R is an arc of circle and can be deformed along the 7 directions into the information boundarymanifold B. The . .Avoid Ro where R --= is the radius of SA "" PV . determines . General ove". For .:"" The . the of therobot. The Collision Avoidance Algorithm This algorithm works till the robot has dynamically TheinternalsecurityzoneS ( Emergency Stops) is found a wayof rebuilding an interaction zone 8 Avoidand Stop a just triggeringspace which a produces completebraking hence to avoid obstacles. A . theway to connecttheseneuralcells. If . parameters Avoid - k and RO (O velocity risk radius) have been experimentally identified. 4. Theuseof theexternalsecurityzoneSA~ internal statewill continue to deform and the Emergency ismoresophisticated andleadsto thefollowingalgorithm: Stopsprocesswill react (after a defoiomationof8scop>. .motors and of the ultrasonic system. Reactive Behaviors of Fast Mobile Robots 111 8 Avoid while minimizingthe robot rotation.

F with 0. 'J'(Si) The last layer is the output layer : all the informations hidden the .connected or partially 11 oonnected( Figure8) 12 j 13 14 15 Neuron j b ~ Fipre . I. OJ 2 Figure 7 : 11we different activationjuncdons with : Wi. = For the ilia neuron t .! : the weight of the connection betweenthe neuronj and the The output OJ of the jt1aneuron is " oonnected" to one neuron i in the next layer. : therealoutputof theneuroni . there is too many network . and 1.112 R. its output. Generally.ranzpkolpaniallyconncedmulti. = ~ = = ~ ~ ~ ~ = : : : : : : = = ' input Ii is weightedby a scaIarWj.0 . ~. . = 2.9=' (SI) for several neurons forms a Neural Network .linelearningof thenetworkwhiletherobotmoves . ) Fipft 8 :E. we use the retrogreaion algorithm that minimize the quadratic error : E = L ( Ow .n.0 . ~ ~ 0 ~ Hidden . The network can be fully . . 1 anon. the secondfunction is an activationfunction which TheLeaminaProcess : In our implementation . e.i . )2 . the output layer. input of one or several neurons . : SingleNewon - . . andthe givesthe stateof the neuronj .pI X I . . At each steps. The fIrSt layer is the input layer : all the data needed by the Neural Network are entered at this stage. and we give the wanted outputs to the retrogression program. weuse . Old : the desiredoutput of the neuroni . TheNeural Network: Thelayered association of e. the Sigmoid function modification of the weights is carried out by using: WII = w" . e.e. = L ( ~ . The result of the neuron j n : a coefficient called " gradient . ) . andtwofunction A(Figure layers. oneoutput. function (Figure 7) : During the learning phase . each . . it shapedependsof the useof the neural Indeed .illyernetwo It . Zapata et ale A neuron is a computed e tity which ~ bas several Betweenthis two layers. w~. in our casewe used the following activation typesof &Cenes to learnthembeforethe robotmoves . the first function of the neuron is to add all the layers weightedinputsS . in unstmdured environments . layers delivered by the Neural Network are available at this stage.(0 ". there is one or more hidden 6) : inputs. we presentthe inputs to the Neural Network which computes the outputs by propagation. . OJ: the stateof the neuronj . fuIIy adivationfunctionis a " sigmoid " . e. i. ) " is: OJ= GJ( L ~~ step . the retrogression process modifies the weight of the - 1 connectionsto obtain better outputs. Then. S This criterion is used with known examples . Our layers targetwasto have This activation function outputs a result between .oonnected networks with two hidden .

estimationof the braking distanceand the measured distan~ of front obstacles.networkf This approach of reactive behaviorsis basedon the 5. even if these three restrictive conditions hold. we cannot saythat the processoonverge to a stablestate. this error estimation must follow a few rules : .line learnina alaorithm : The 6. velocity and the braking force are given.2: Data propagationin the network. two levels of reactive behaviors ( The EmergencyStops Level and the DynamicCollisionA voidanceLevel) 6 : GoTh Step 1. braking. we do generating accelerations . the robot velocity and the estimationof the braking distanceswhen the vehicle orientation of the front wheels. 91'E1P 5. changes of not know how much fa Turnor to hrake. The inputs are the distancesof robot braking dynami(S in order to generate an obstaclesahead the robot. Brror estimation : The Neural Networks outputs are " connected" on the ~ Ierator brakes and . when the robot runs to an obstacle. 1. 91'E1P Fipre 9 : Emergency StopsLevel 91'E1P5' : Retrogression and oorrection of the These networks are combined in order to create the weights. 2. All the variables used in the error estimation function must be inputs of the neural network. It must be equal or close to zero for the desired output (following a qualitativecriterion) . its orientation We are onJyableto evaluatea good answerfollowing a and the field of surrounding obstacles(proximity definite qualitative criterion. Reactive Behaviors of Fast Mobile Robots 113 9=' (8i) : thederivateof activation braking actuator) . Sub. we can only saythat the processis locally stable. informations) are given. An compatible. This qualitative criterion must be translated in a analyticfunction : the error estimationfunction will be the computationbasisfor the retrogressionproress. A BlukiI M~ . By experience. A dynamicmodel of the robot wastaught to EmergencyStops Module (ESM) which can generate a maximal braking force (action on the . Main Results 4. an functionat thepointSt. 91'E1P J : Robot oontrol with the network outputs 91'E1P (acceleratorI brakesI direction) . Simulation[ 12J developmentof a neural network system Corn~ d of severalsub. In order to be efficient. . A wanted output neededby the retrogressionprocess. . On . directions when the robot velocity. As a Dynilmic Collision Avoidance Module (CAM ) mater of fact.step algorithm used in our on .line learning Neural Networks is the following : 91'E1P1 : Data acquisitionfor the network inputs. direction II ling Moduk ( BMM) modelingthe control of a mobile robot. Its inputs are the robot velocity.networks: These3 networkswere simulatedin Thrbo C on a PC . The main problem is that we cannot have the exact . 4 : Error evaluation.

In the last simulation phase. The reactiontime of the networks(eachsub. BMM network. both Neural Networks.line learning phaseof the shop on Intelligent Robots and ' 89 Systems . Conclusion . and then ~ for CAM using these two This paper addressedthe problem of autonomous reactive behaviors in the important case of collision previous modules in order to run its own retrogression avoidance. BMM and ESM. the Neural Network avoids obstacle.for BMM alone. 5. We obtained a robot which can avoid obstacle and stop itself when it is lammed ( no free way to avoid obstacles) .network qualitativelysimilar and allow fast collision avoidancesin unknownand dynamicworlds (Sm/ s for SNAKE and 2m/ s has about 40 neurons) is better than real time (with for RAT) . developeda dynamic behavior program. respect to the robots dynamicsand control) . This simulator Theseexperimentswill be soonextendedto the caseof usessimplified dynamicequationsof a mobile robot and walking machines. We have chosen three ultrasonic sensorswithmeters range . carried out during the motions inter . of the real vehicle. carried out during the si1'l}ylation process. but with jerk motions. Obstacles are randomly placed in the ~ ne. Brooks Robot Beings . managesa graphicUser Interface. of efficient collision avoidanceprocedures than in the . Our contnbution is more in the development process. braking forces. Zapata et al. 1kukuba. ESM network learns how to optimally brake . with the sameprocedure ( BMM Simulation experimentsand real world experiments and then CAM ) . Sep 1989. the CAM network was tested alone. are implementedin the simulator. and to stop exactly at the specified distance Xstop- lb implement the Dynamic Collision Avoidan ~ Level . From the first try . The results provided by the two approaches are avoidancewere carried out. With only this perception . An off line leaming.114 R. one on the front of the robot . REFERENCES we measureon the simulator the braking distancesafter throwing the vehicle with different initial velocities and " " [ l ] R . we implemented together both the Emergency Stops level and the Dynamical Collision Avoidance level . Japan . on two wheeledrobots haveboth demonstratethe validity of sensor. A. International Work. 1b teach the dynamicbehavior to the BMM network. After the off .based control approaches and behavioral lb simulate the motion of a car. we have defined a perception system which gives the same proximity information as the real sensors. As long as we make different tries with different initial velocities. 6. the vehicle thrown on a front obstacle stops before the oollision . 2. .like robot. In the first try . Etperiments Figure10: CollisionAvoidanceLevel Severalexperimentson the 2 wheeled vehicleshave been carried out in unknown and dynamicenvironments the network and several experiments about collision (outdoor and indoor) . An on line learning. After several tries. . the direction oontrol is smoothed and the curves descnbed by the robot beoome more homogeneous . we have approaches(neural networks) . Thus. whatever the environment . and two at thirty degrees from the main direction on the both sides of the robot . . the A videotapedemonstratesthe skills of theserobots in learning processcanbe executedby two ways: the problem of avoidingmovingand unknownobstacles .connectionsbetweendifferent levelsof behavion.

m ~~ Preplanned Int .May "Qualitative 90 [.Journal&.and Aut . alRobotic "Reactive Behaviors Mobile ofFast 3]M. Reactive Behaviors of Fast Mobile Robots 115 2]TiL [gent .Robot [Mobile Soldo "Reactive .N82 .The International ofRoe [6a Holenstein ]Behavior E.IARSet OS 'of 9 Primitives Behaviors Taukuba 0 1990 forMobile ll1991 . Algorithm the toJournal of Control RR commande -1172 . LO .Th ~ d 'Etat .botique mobile ". Rapport DBA .juillet1992 .Universite de Uier Montpe Robot behaviors Control":.Volume N81 1.]B.1990 enrobotique .-. . connexionnis in -b~d RobotDesign .Ohio . rapport Robotic ". 6 . Sacramento . and .Zapata Robots ". California [u ]p.Proc botics ResearCh .agprochedela relation INRIA [10 tion]'tw . ofaManipulators General EH LearningOlanz .onRob . P. .IEEE [l1]Guillaume finalMRT Euvrard .LC Local :{'&nay R. Cont Apri11991 . ' .0 and no.IROS9.Japan Controlina [8]R.USA . Zapata Quelques aspectstopologiquesde la pJanificationde mouvementset desactionsr6t1exesen ro - am actions r~ffexes des Universi~ deMontpeUier2 robots .janvier 1991 Int . onRob .Deplanques~ Hcationdes reseaux es neuron de --~mobiles " de " [ 7] R .Mobile Badreddin " Avoidance Collision ".Kraft"Applica - S]B. Rapport .Lawton [Reactive Control forAutonomOus understanding Spatial Robot ' ".for iation Zapata "Reactive "- Robots Mobile 4]D. . Vol . Miller .1987 "Mod ~les "..-Espiau [vision & . Anderson "Autonomous Robots and Emer. and1991 Auti Cncinnati . Conf.Japan "Nouvelle ary 1992 .Jouvencel Behaviors and .of Systems .. inJuly proposed " Proc . Febru tuba 90 .Fast .Journal ofAutomation Nava ana CIM .89PO713 . ai..

two dimensionalarraysof points with betweentheeyes). For otl1erimage. amongother things. Sharpchanges pointsof eachimagearematchedto their counterpartin the in brighblessmay indicatediscontinuitiesin surfaces . andthe vergenceangle(the angleformed particularcolor andintensityvalues. While this appearsto be a reasonablemethod pairs of overlappingimagesthe differencein position of given that the datafor thesecomputationsis availableand . within a left and right view of the samescene. In both these cases the input provides pointalistic shading and stereo disparities. In such a problem one solutionto the problem. The ideais to look at vision asan or whatthereflectanceof the surfaceis sothereis no unique information processingpro~lem. Studiesof the human visual systemhave shown occlusion. anglesaway from the illuminant and the rate of changeof depthis calculatooat eachpoint basedon how theindividual brightnessindicatesthe degreeof curvature. evidencesuggeststhat the processis adaptiveand that the Since theseare some of the quantities earlier indicated as part interpretationof Abstract objectsand partsof objects. partsof objectsandpoints It is difficult to know in generalthe locationof the thatall havea labelingof. Recent psychophysical infolmation that varies with distance and surface orientation.linfield. shading. In particular. Introduction surface. the computationinvolved in makinga betweenoneseyesandat anym<mentthe vergenceangleof transformation to three dimensions must come from oneseyes. assumingthat once pointsarematchedit is a simplecalculationto find depth. For example. these way. 3D information directly is that there is no unique way to interpretthe information. the basicideafor interpretingshadingis behaviorsare not consistentwith many of the modelsof that brightness values in an image vary with surface stereopsisthat computedepth in a mannerconsistentwith orientation.4121 x576 allenb@audrey . In principle. In this casethe from disparity. One of the standardmethods for studying vision is the Thereis no way of knowinga priori wherethe illuminantis computationalapprooch . Treatingthe 3D visual be a direct calculation from point sources of disparity or systemas adaptiveprovidesan efficient androbustmethod brighbless to that of depth. behaviorthat is not consistentwith computationalmodels andstereodisparity. For this reasonmost of the work in stereopsis informationwithin theimage. the outputs and the computationsthat are required to transform one into the In contrast. distance 0" smface orientation. imagehaslong. distanceand light source or surfacereflectancepropertiesbut it seems Qrientation.beenireatedasa deterministicprocessbased on geomebicconsttaintsde~ ted from cuessuchasmotion. surfacetexture. Without knowing these things a particular brigbmesscan have an infinite numberof interpretations . For example. needs to determine the inputs. From the input of a pair of images. that the output shouldbe a hierarchicalstructureconsisting of groupingsof objects. dependson of the desired output it seems logical that the computation learning. aloneor in combination.116 The Adaptive Nature of 3D Perception Allen Brookes Departmentof Computing Science LinfieldCollege McMinnville . depth can be computed three dimensionalimage understanding . to determinesurface orientationfrom shadingoneneedsto determinewherethe light is coming from and the reflectancepropertiesof the 1. Eachsourceis interpretedin a different for interpreting2D imagesas3D scenes . This approachbasbeenusedwidely in the studyof than the other cues. is an indicationof the relative positionsin The perception of the three-dimensionalqualities of an spaceof thoseobjects. Sincethe input only bastwo dimensionswhile plausiblethat one could know from experiencethe distance the outputbasthree. In naturalimagesthereare many possiblesources of 3D information. Among theseare. giventhe intetpupiJarydistance(the distance input is one or two. surfacecontours . for dealing with a very large and complex set of cue interactions that would be much more difficult using a The problem with using thesecuesas sourcesof pm' e1yfeedf wardJXOCeSS. Oregon 97128 503 ( ) 472.motionparallax. Differencesin brighblessmay indicatedifferent that outlined above. Our experiencetells us by the eyesconvergingon a singlepoint). basbeendoneon the matchingprocess . objects.stereopsisis moreconsttainedin theory other. expectationand attention.perspective .

(1984) showed thatthedepth of pairsof linescanbeeffected by theadditionof othernearbylines. This providesa proof that. But if thesepointsare embeddedin a continuous3D Theseresults suggestthat depth is derived from surfacein which the monocularinterpretationsuggestsan disparity only where the surface exhibits continuous alternative relative depth between the two points. interpretedandreconciled. Evidence for Indirect Depth from Stereopsis Stevens& Brookes( 1988) studiedtherolesof stereopsis and shapefrom contours. Youngs(1976 ) showed thatin planarstimulidisparities badlittleeffecton perceived slant. Thusstereodisparities~ SSa planarsurface are predicted bydisparities . wereusedf(X" theprobepointswithin thevolumecreatedby the random disparities. If depth were computeddirectly from disparities then surfaceswould be computedsubsequentlyfrom the resultingdepthmaps. the one with the greater(morepositive) disparityis seenasrelativelyfarther away. depthis a reconsttuctedquantity. In this casesubjectsjudged the relativedepthof the probepoints very accurately. In such figures familiarpercept generally overpowers thatof thereversed subjects tendto interpretthefigureaCC<X'dingto thedisparity . Given a randomdot stereogramof four slantedpanelsarrangedat roughlythe samedistance. Thesestudiescastsomedoubton the theoriesfor computingdepthfrom disparities . Also. For example. This means that the depth one subledsjudged the left point to be closereventhoughthe experiencesis a consequenceof how this information is disparitiesarecoosistentwith it beingfartheraway. radically different depth disbibutionsmight be experienced . Examinationof control stimuli indicate that thisdoesnotappear to be thewayin whichthehuman sufficientstereoinformationis availablefor sucha taskyet visualsystem interprets stereo . The samedisparities monoculargradientwas kept fixed. Example stimulus in which subjectsjudged relative they useda stimuluswith the sameset of points in which distance of a probe point. Figure 1 shows a reconsttuctedfrom multiple sourcesof evidence about planar grid stereogram with a pair of probe points. McKee(1983 disparities ) andMitcbison andWestheimer information . This is very different from the notion that depth is derived directly from stereo disparity. In both casesthe disparitiesof . Stereo gradient was varied while the disparitieswerechosenat randcm. As a conttol Figure 1.the overall impressionis of a set of slantedstairsteps . Figure3 showsboth typesof stimuli. Subjectsjudgedthe relativedepthof a pair of probepoints accordingto this misperceptionrather than~ ing to the valuesof the disparities. Figure2 showsanexample of familiar disparities figmes suchasfaces andfoundthatthe of a grid with non-planardisparities . The following confirmthat thereis no directrelationship between "depthanddisparities andsuggest adifferent wayof lookingat thecomputation thatis moreconsistent with observed behavior. The surface topography. at leastin certaininstances . Depeildingon how themonocular information is interpreted. Several studies havereversed the noteffectivelyanalyzed in 3D. An experimentwasperformedto testthis conjecture . A horizontal Gaussian ridge defined by the disparities with a flat plane defined by the contour lines. 2. The resultsdescribedin Brookesand Stevens( 1989) show that binocular depth is computed subsequentto surfacedetectionand that depthis computed from the surfacedescriptions . The Adaptive Nature of 3D Perception 117 seemingly morereliable thanwhatisprovided byothercues . impotentin influencingperceivedorientationand3D shape. given a pair of stereopoints. that curvature or sharp discontinuities. Various disparities studies with the competinginterpretati()n the disparitiesare not haveshown thatperceived depth sometimes differsfromthat effective . that depth is monocular interpretation dominates. It was found that stereopsisis remarkably Figure 2. Theypitted two 3D cuesagainsteach other to seehow the contradictoryinformation would be interpreted.

however. However. This impressionmay be an illusion. giving derivedfrom both orthographicand perspectiveprojections the appearance of a volume of points . Bruno and Cutting earlier. Experimentsusing more complex stimuli show incompatibleand ambiguousinfoamation. SperlingandWurst( 1986). shadingprovidesinformationaboutsurfacecurvature. Many of If the depth of the pair of probe points were the computationalmodels for interpreting3D information determinedby a directcomparisonof thedisparitiesthenthe tty to combine 3D information to consttain the problem. that this is not the case. Information about A surface curvature is fundamentally different from. say. surface. disparitieshave the possibility for giving the most ( 1988) propose an algebraic summation model for the reliable3D informationsincetheinformationneededseems integration of 3D information from multiple ~ onocular to be available. cooperation. In their experiments . an unambiguous interpretationof the image. Whenthe adjacent constraintscan rigid sttucturebe recoveredfrom motion. There are severalpossibilitiesfor how the qualitativelydifferent informationmay be interpreted: ( 1) Thecuesarenot commensurate andthe impressionof depth from monocular. In eachof thesemodels. the integrationis proposedto be a procedurethat information is that different cues provide seemingly relies on direct comparisonsof the cueswithout regardto context. For example. thejudgment seemsto be consistentwith the propertiesof the perceived general problem for how to integrate 3D information. their information mustbe madecommensurate . Algebraic summation of the strength of 3D information is to use combinationsof cuesso that the evidencegivenby independent cueshasalsobeensuggested variouscuesmay mutually constraineachother to provide by Dosher. but seen as having an overall staircase variation in depth changing the relative perceivedfrom stereopsisin thenearfield. and veto. simple additionexplainsall of information then the next best possibility for interpreting tlleir results. To thedotontheright. For example.rather . a statistical way. combininginformationfrom cues. If disparities do not give reliable depth cues. disambiguation. In a the disparities are distributed randomly . Richards( 1984) usesstereoand motion to disparitiesof adjacentpointsshouldnot affect thejudgment It appearsthatadjacentpointswhichdo not provideevidence constrain each other and shows that only with such of a surfacedo not affect thejudgment. Whenlooking at a picture there is often a vivid impressionof 3 dimensions despitethe fact that the pictureis flat. This not only showsthat the depthis reconsb Bulthoff and Mallott ( 1988) describe a set of possible ' octed from surfacediscontinuitiesbut also adds support to the methodsfor combining 3D information. including the lessgeneral One of the main impedimentsto integrating 3D models. In b . 118 Allen Brookes the dots should indicate that the dot on the left is farther than thanaboutthedistanceof the surfacefrom the observer . andif the correspondingcuesare to be combined . cuesare comparablesuggeststhat they may be combinable as well. The fact that the distances of the probe points . These include accumulation . problembecomesfinding how they arecombined. the disparities as a scaledquantity that is commensuratewith the depth correspond to a triangle -wave surface . interpreta shadinggradientas implying surfacecurvature requiressome(occasionallyunreliable) assumptionsabout the uniformity of the incident illumination and of the reflectancepropertiesof the surfaces. Stevensand Brookes( 1987) showedthat depth is Figure 3. or flat binocular imagesis illusory. (2) Cuesare commensurateonly where there is a reasonable computation for comparing them (no restrictive assumptions). As stated integrationsttategywherethey do not. . (3) Cues are converted to common representation using assumptions if necessary to do conversion . Integration of 3D cues disagreeone wins andprovidesthe percept The modelthat they develop from their experiments consists of an The evidenceabove shows that the humanvisual system accumulationmodelwherecuesagreeanda winner-rake-all doesnot computedepthdirectly from disparities. If 3D information is in fact combinable the . . inf<X'Dlationabouttheoveralldistanceto a surf~ . I~is not clear that this is done. perhapsin conjecture that surface properties such as slant are derivedfrom disparities . where cues 3. there are few modelsthat attemptto model the pointsareconsistentwith a surface. using one cue where inaccumtely anotheris ambiguous .

resultsdependon manyfactors. Stereo and monoedges intersecting at right angles. ason whatwe areactually expectations interpretations indicate sm. I . 10 . Subjectsimpressions of separately CODsttuctingsurfacefeamreassertions by stereo rangedfrom entirelyconsistentwith the contoursto entirely andmonoprocess esin a bottomup manner. in which different planarand curvedpatternswere integration of 3D information from surface contour and independentlydefmedby surfacecontoursandby binocular disparity processing. What such models lack is the top- comp. In general. These surface feature assertions the normal geomebicinterpretationof surfacesformedby constitutea commonlanguagefor combining cues. The figureconsistsof a stereogmmrenderedusing consistsof extendedsurfacefeatureassertionsrepresenting contourlines in which thedisparitiesarenot consistentwith the individual cues. the results seem to involve such things as: subjects experience. The results showedthat. In orderto model this sort of behaviorwe need qualitatively consistent. but different in amplitude. For most observers the appearance is of four steps of differing Why . The contourinformation suggestsa smooth than local. consistentsmoothsurfacesbetweenthosefeatures . Consttuctingthe surfacedescriptionsconsists Gaussianedge rotated 90 degrees . generateda The ways in which these stimuli are interpreted seriesof stimuli similar to thosein Stevensand Brookes appear to rule out any simple additive model for the ( 1988). Lees and Brookes ( 1991). pointwise primitives such as depth. generality .omise between both.e. and the impression may be deriving depth from 3D cues will account for human either the stereo or the monointerpretationor . rather the contours. and finally filling in compromise . althoughwith planar separatemechanism for casesin which the stereo and monocurvature stimuli therea a winner-tate-all dominationof onecueover information conflict and for cases in which it is another. One reason is compromise . to dependon subjectexperienceand mindset. different to model a dynamic memory that molds the percept observersshow markedly different responsepatternsin a accordingto storedpatternsand that allows the additionof quantitativecomparisontask.wheneachcuepresentsnon-planarinformationthe consistent. source. The inset shows the two extremes as well as the several possible reasons why this may be. imageswith conttadictorycurvatureinformationfrom each . An exampleof one of the stimuli usedis shownin We proposethat sucha memoryfor surfaceshape Figure4. (i. The idea that surfacefeaturesare comparedwith storedmodelsfor ')~ "()~ their interpretationandthenpastedtogetherinto largerunits. focus The lesson from the above is that no simple model for of attention. but varying among observersand down componentthat allows the behaviorof the systemto among presentationsfor the sameobserver. providesa simple accountfor the difficulties observedin ) .) or an unstable percept in which each feature repeatedly overwritesthe other or a compromisefrom the filling and smoothingprocess es. Why should the visual system be set up this way when there appearsto be a simpler solution to the problem? Figure 4. The three dimensionalpercept seemsto be basedas much on our In cases where both stereo and monocular of whatwe win see. This predictionis consistentwith our observations . Wlfamiliarf(X'IDSfcx subsequent processing . The disparities suggesta orientation. The Adaptive Nature of 3D Perception 119 Stevens.8 behaviorin this area. Where both stereo and monocular interpretations 4. subjects mindset. The expectedinterpretationsof thesecomparisons ()> <~ might be: the overwriting of a featureassertionfrom one c sourcewith that from the other. the separate surface feature assertions top down by The inset shows the two extremes as well as the comparing with the stored forms. winner-fake-all. that is within about 200 feet of the viewer . II .face curvature features which are looking at. unless one is willing to propose a disparity. Discussion indicate inconsistent surface curvature features. resultingin eithera stable perceptdeterminedby oneprocess . or surface edge with Gaussianprofile . although we expect that a more complete explanation could be obtained if we had a better understanding of how the integrationis affectedby changes in the focusof attention. Stereopsis is only possible in places where the stereo disparities can be accurately perceived. is stereopsis not used more extensively when it appears to be the most reliable cue? There are depths . There are also regions of the visual field where there can be no stereopsis . in particular. thencombining consistentwith the disparitiesor somewherein between.

Thusfor thesystemto operatein a similar way underall conditionsit 5. In this case. K. & Stevens . different types of processingoccur in different to constrain the problem seemslike a reasonablesolution. Finally. with its context.organizingneuralpatternrecognition model must be adaptedfor this. processingdiffers dependingon the relationships of differentcues. be a compromisebetweenthe input and the template. It is ttue that system is treated as a dynamical. 1987). Tl. adaptive dependson how that feature. in regions that are relatively changesthat makeanothermatchsttonger. systemsin which the processingcontains no feedback. volumes.lesechanges homogeneous therewill be a greatdealof ambiguityin the may includeeyemovements. so the architecturefor a self. the the behavior of such a system is complex . for each processof comparisonswith stored models allows for set of features the best match is sought from conditions quickly discounting a combinatorially large set of within the systems experience . Descriptions for such possibilitiesand thus enablinga quick interpretationof a systems can be fairly simple . If instead the visual like a rather complex solution to the problem . 1989 Binocular depth from where there is a match betweenthe input and a stored surfacesvs. Objectsthat contain inconsistent3D cues can settle to the closest match with the available informationwhich may be with winner-tate-all or somesort . the behaviors become more easily understood. Objects with incompleteinformation. Journal of ExperimentalPsychology template astheycall it. Conclusions is useful to have stereopsisas one of severalcontributing sourcesrather than as the primary source.A. It alsoallowsone to adaptto new circumstances that models that display many of the characteristics of the might ariseand that would causea deterministicsystemto behavi(X' we have found in human 3D processing. This stablestate. 1987A massivelyparallel specificallydealwith differentsetsof cues. 54.120 Allen Brookes dueto a ~ k of binocularcells cx dueto anobsttuctionsuch of compromise. The humans. With sucha modelhumanbehaviorseemsinaedibly Thus.115 inte~ ting depthcues. If one Bulthoff H H. Thesecategoriesin turn inhibit other categories . This allows for The problem of combining cues is very difficult . however.170 thathavefeaturesin commonwith the input are stimulated . adaptive system. fits a stored system. However . Cutting 1 E. This for the behavior becomemuch less so. the use of }X'eviousknowledge to }X' Ovideexpectations complex. objectsthataremorefamiliar are more easily recognizedand propertiessuch as depth. : perceptionof layout Journal of ExperimentalPsychology Whenan input is fed into the networkeachof thecategories General117(2) 161. Weat matchesmay be unstablerelativeto as the nose. Sperling G. A.ComputerVision. One of the main reasons seems to be the problem of imposing sttucture has not been solved but inappropriateness of the basicmodel of vision assumedby ins~ is moredifficult sincethereis now a third dimension many of these approach es. or resonance 15(3) 479-484. Despitethis. if the Many of the approach es for interpreting3D informationin visual systemwere to computedepth at eachpoint from images do not adequately model the behavior found in disparities the result is a point-wise depth map. the behavior is still complex but the explanations modelof surfacecurvatureanddiscontinuityfeatures . The numbers of ways in which cues can combine makes it Thus there is no effect of previousexperienceor current difficult to implement general rules for the combinations.A. In fact. & Grossberg . " Dosher B A . the model machine. & MallotH A 1988 Integration of depth categoryhas many more featuresof the input than others modules: stereoand shadingJournal of the Optical Society then all other categories will be inhibited and if the of America5 1749. The principles behind this behavior may be relatively simple. fail. the degreeof similarity requiredfor the match. This basicmodel containsthe to consider. there are already scene . One example of such a model is Adaptive Resonance Theoryor ART (CarpenterandGrossberg . anddifferentpeoplebehavein Basing each percept on stored infomlation seems different way to the samestimuli. The ART models do not Carpenter . If interpretationof anyone feature at any particular time we look at the visual system as a dynamical . S. ~ of the visualfield. attentionalshiftsor mindseL ma.1758 similarity is within the vigilance parameter then that category will be selected. The resonance canbe affectedby attentionalpriming as well as Bruno N. ideathat individual featurescan be treatedas symbolsthat have a predetermined3D interpretation. may . Wurst S A . 1986 Tradeoffs extracted. First. Also . 6. References ART is a theory of category recognition in which recognitionconsistsof a network settling to a stablestate Brookes. state. such as betweenstereopsisand proximity luminancecovarianceas objectsrenderedwith a single3D cuecanberecognizedwith determinantsof perceived3D sbUcture" VisionResearch26 partialinfOlmationandcomparedwith otherobjectsrendered - (6) 973 990 with different cues. 1988 Minimodularity and the vigilance. Graphicsand ImageProcessing already has some of the important behaviorswe find in 37. G. tchmgp-ocessandstereopsis will be unreliable.

1983 "Anisotropies " Science221in1400 the of three perception . 1984Sttucturefrom stereoandmotion. VisionResearch Mitchison.. K A. A. Lees. 1988"Integrating Stevens stereopsiswith " Vision monocularinterpretationsof planar surfaces Research28 (3) 387-396 Stevens . 1975 The influenceof" perspective and disparitycueson theperception of slant VisionResearch 1679-82 . A.~ "Postfusional . "Perceptionof slantwhenperspectiveand stereopsisconflict: Experimentswith aniseikoniclenses" Journalof ExperimentalPsychology78 299-305 Gillam B I . Chambers D.. 1063-1073. K. GJ. M. Laboratory RogersB I .198. AI Memo731. andWestheimerG. Yellott J It Kaiwi J Lt 1979 "Depth inversion despite : the appearanceof random-dot stereogramson stereopsis " smfacesseenin reverseperspective Perception8 135-142 " YoungsWM . GrahamME .A. FlaggT. MIT. 20(4) 425-440. Finlay D.dimensional surfaces - 1411 Stevens . The Adaptive Natuaoe of 3D Peaoception 121 Gillam B I . stereoacuity. & Brookes . 1852On someremarkable.-e features . Russo . 1987 Probingdepth in monocular~ ges. 1983 The spatial requirements for fin~ . & Brookes . 1984"Evidencefor disparity changeas theprimarystimulusfor stereoscopic " processing PerceptionandPsychophysics - 36 559 564 Gillam B I . K. 191. 1988 latencyin "s~ pic slantI'eJ'ception andtheprimitivesof stereopsisJournalof Experimental : Human Psychology Perception andPerformance 14163 -175 McKee SiP. 4 504-523.A. ArtificialIntelligence . Combining binocularandmonocularcurvatu. 1984The perceptionof depthin simplefigures. Wheatstone C. BrookesA. 23.BiologicalCybernetics56.Ser. 355-366. Perception .. T. 1968. RichardsW. andhitherto unobserved phenomenaofbinocu IarvisionPhil Mag. VisionResearch24.

Elegans. C. The anatomy C.8 netdD' uSL of infOrmationavailableaboutit [ Wood88 ] .die partof die bodywhichfOnDS die " sidettof ~ aim~ t certainly be iutnlctive . The developinenaal ( White86 history of negligible ). Thisworks of an ~ b81 animal. Theforcewidt wbkh diewum bellavicx :. wavesfromheadto tail to movei~ if forward(wavespr0- In [ Hartley92 ] it wassuggesroothat the completesimulation pagatefromtail to headto move18: kward ) . worm which lives in drag8' e ~ nt for d:aese two regions~ ~Uc ~ . DC 20375. way in which all bebavicx .die aest ~ il aOOrotting vegetation . Eleganst e. smfacetensionmainly ~ to puB die w<Windown . 1.As a first step in the simulation of the wonn s tighdyontodiesubstrate . which expects covering8 rigid (or l Iear1y rigid) SlUf8 :e. : h cell. crestandtrough8' e movingf~ ard. Eleganst like aDodlernematodes .but wasunable EleganswasconsttuctOOand used to study its l<x: omotmy to perfonn experimenmas extensiveas dK)se desa1bed behaviors . This simulationshedlight on die mechanisms he . Thisproduces . Naval ResearchLaboratory. The worm proplgatesbending roboticsby building robots[ Brooks86 ] is to build animaJs. It hasbeenusedby biologists andtrough8' e movingtangentially whilediesideis moving as a modelsystem.In ~ situadon to completeits job befc.5000 Abstract O] also built a ~ ulation of the body and [ Erdos9 A simulationof tile mechanicsof tile Nematodeworm C. could be feasible and would ~ ~Uc . Introduction pl Opuisioott Undulatory propdsimis alsousedby mostsnakest mille An extension of the strategy of learning about fishtand some protozoa . Elegansgenomeproject.-ce of the (diee~ dvevisoosity ). but surf~ tensionaOOviscosity . Theres~ ~ to motionalongits : is coupledto locOinotion. .122 Propulsionand Guidance in a Simulation of the Worm C. stepin tile of understanding tile overall behavior of an animal 2. e the bD ' I1of thecentury. ~ 8 fOlmof l<x:omotion " knownas undulatory [GrayS3]. [ Rowley91 ] ha do~ somesimulationof the motor dJatmustbe usedby any animal or robot usingdie same circuimof a re1ale4WanDbut did not simulatethe associated type of propdsioo and also on aspectsof die behaviorof die real worm. C. aOOconsequendythereis a greatdeal atanangle . Elegansthe is only die waveis moving18: kwardwhile die partsneardie animalfor which dIis is likely to be possible. Thereis behavior11MmostlybeenSbldiedin 8 thin film of water alsocl Urendya C. The simulationof lCX : OmoUon is an important mechanics. Real Worms C.s body aOOof b conuoi of locomotionbas been die subsb' ate8geiy determines die resistance to ~ carried out ibis is a ~ ~ ! ! ! Y first step~ ~U. Elegansis a smallnematock . someof the neuralcircuitry of C. ~ luding ~ h cen divisioo. Elegans RalphHartley Navy Center for Applied Research in Artificial Intelligence. ~ ananimalin thissizerangetdieckm- system(and of the rest of its 959 cells) basbeencompletely inantforces8' e not gravityaOOinertia(which8' e in fact ma~ ] . . is known . Washington. Thecoefti:lentsof C. a simulation study of the mechanicsof the is plasedintothesubstrate alongwidt theCCXI1p :!Sitiooof wonn. Elegansis ~ ~ ately Imm Img aid aOOut (including aU SY D8P8e$) of aU 302 neuronsin its nervous 1/lOthu thick.

extra variables can changein lengdt. It is also well suited to dIree dimensional envirooments such as betw~ n soil particles and inside the body of anodter animal. into small segments which ultel' act widt ea: h odler and This model assumesdIat all m~ u1oskeletalfm'CeS8' e widt dae substrate. using springsmountedon die joints representbending forces di &rent levels of detail. On dae odler hand. more complex . 3. dons. OPJX )Site body . The primary di~ nce is that appliedto one end. : tion widt dae substrate is by ~ tially loogi1Udinal. The force will ~ ~ te to the other mations in dlickness (which are too smaI1to be dynamically end die successive stJ' etcllingof ech ~ gmentin bUn . locomotion. but may not bend. 3. > are not modeled. enviroomentsfor Die worm can also be simulat~ . Furdlennore . of daesystem being simuJated. Only Die relatively simple situationof a WOoDon a flat unobstructedsurf~ was simul~ ~ The simulatoris s~ ently flexible d18todler. a simulator was coostructed . it is important to be vay careful in selecting Figure 1 . Simulation of the Worm Elegans 123 axis and to slipping sideways can vary greatly. Most complex . it is loogitudinal stretchingaI KI CCI Itta:tion This model is a simplification of dae one used by which consumesmost of die simulationeti)rt.~ ted by universaljoints left out. The model coosists of a . Inter . 1 shows dae first model The worm is bending motions are most inlporlant inundu JaIcx' y modeled as a chain of segments connected by springs. sidesof die . by unporaa. Though Figure . ~ that it becomes controlled by die springson die joints. The simulation is inherently three dimensional . model of the worm. The Three models of dae worm were bled. Model of worm In ordez to produce a simulation which is bodt fast a I KI acclD'afe . S~ (Eldos9()] which was to complex and too slow to be used die worm's body is straight aOOa lengdtwisef~ is in extensive experiments. This can prodIl Ceextraneous The lengdlwise springs representdie symmebic phenomenain daesimulation. but die sum aI KI di~ nce of forces on springsrepresent which may have different coe8i: lents in different direc.1. The lengdlwise springs can being studied. dae variables to be simuJated. H important variables are chainof lengdtwisesprings. compooentof die elastic aI KI hydrostatic fm'ceS . dae model can each~ gmentcanbe cmnputedfrom die springforcesand be made more elaborate dIaD is justified by what is known thehydrodynamiccoe8i: lents. The simulator To studyDieproblemsof this type of locomotion. This model has die foDowing problems . dae simulation will not represent daephenomena held straight by springs. This fonn of locomotion has several important properties .The first . All dU' ee models break daeworm dueto asymmetryin die cuticle tensionaI KIm~ le f(X'ceS . Undulatory propuisioo is suitable for complex envirooments where diere are many small obstacles. The forces of die two ~ of viscous forces which are prOJK)rtiollal to vel~ ty . Unlike I~ omotion by legs or wh~ lst it is impossible to ~ uple l<x: omotion from oilier behaviors. : table. bendingfm' CeS8' e cb: rease daestability of daesimulation . The velocity of C<Xnputatiooally inb' . The most important of which is dlat l<x: omotion uses the entire body of the animal. but in all Dieexperimmtsdlat wereperfonnedthe wormwasrestrictedto oneplane.

however. forcesmakea significantcontnbutionto detenniningdie exp worm's shape ! Nonn . The secondmodeldoesnot simulateforcesbetw~ segmentsat all. Ho~ ever. They me coupled togedter by the constraint on the segmentlengdls. forc Bend The forcesacting on a segmentare shownin figure 3. This is awropriate if the elastic and m~ uJar forces are large C<XDpared to viscous forces . Unlike the first model. which was fowxl to be most suitable. : h segmentis kept fixed. On h od1er hand. die velocities of the segmentscan no Imger be C<XDputat independently . and dlat the Iataa1 ~ ~ lents of friction are gel1eral1y greater. Be!:. Combinethis widl h f . The viscousforce hastwo parts: the compooentin die tangent direction and the ~ pe1' pendicularto this direction. The viscous force is pI'O!X)Itional to velocity. . this wiD generallybe the caseonly when h ratio of Iataa1 to Imgiwdinal friction is near - one. BendingforcesFi me producedby the elasticityof die cuticle combinedby die force producedby internal presureasweDas the forcesgeneratedby m~ les.124 Ralph Hartley This requiresa numberof simulationtime stepsroughly pI' O(X)rtional to die number of ~ gments. a Iatera1force needonly benda few ~ gmentsnear h point of application. so dlat h lengdl of e. Insreoo . die only non-z a matrix elements me near the diagonal. These forcesproducea bending momentcoosistingof a force Figure 3 . In this ~ (which may occurwhenh wonn is swimming) h body a~ hes staticequilibriumon a shorttime scalerelative to h rate of motion. all was used for .andtheresultis that muchfiner time stepsmustbe usedto obtainstability in lengdl dIaDin bending.die constraintsme only betweenadjacent velocities. is a simplificationof h firsL The lengdt- wise springsare repl-=ed by rigid rods. :". OdIerwise movementis fast enough dlat viscous 2 Figure ThisSimp mode W mod of On D.:ty Ten vlSCO The third model (figure 2). and die equationscan be solvedquickly. : t h die wonn is relatively resistantto sttetchingbut bendseasily.Forceson a smaIl section of the worm. h relative positioosof die segments are setindependentlyof the dynamicsand only h motion of die wonn .aUc ~ gments. a whole is modeled . This meansdlat a set of simultaneouslinear equationsmust be solved to get die ~. Each of these compooents is prOJX)rtional to die COTeSlKltdingcanpooentof the velocity andis apJX >sitely directecL Thedrag~ Oi: lents mustdiffer (die perpendicu- Jarcanpooentis usuallyiarga:) for undulatorypropulsion to be ~ b1e.

and propel dlemselws forward by meansof backw8' d po- - The remainingforce is die net tensionor ccxopres pagating waves. The headis cootroUe AI Thereis litde or no infonnationavailablefor ~ tting dIis independently . The results 3N equations show d1at previoos studies of oodulat~ I X' Gp Jlsioo .+ T. [Gray63] miss Dne key issues. A phaseof 0 degI ' ees only changesdie time scaleof die simulation. aJsovariesgreatly widt envil OOmentai cmditi ODS . Experiments The velocity at each~ gmentcan be CCXOputedby ~ Iving a systemof 4N. If die slip thereforemot conside !' edsuitablefcx quantitativeexperiments ratio were 1. where tile sinusoidal body wave is Once the m~ le forces are given. aOOonceestablishedhow 3. " waveis measuredl<x: ally insteadof over the whole body.U. The spring coostantof die elastic forces. 1 " ~ me observedaspectsof die behavior of real wonns ensiD "e dlat die total force on each ~ gment is Za' O. widlmanyparametersthat mustbe adjusted . the previousparagraphexceptthat the ~ of the body " The most importantparameteris die slip ratio . however. The [CroI I75] .+IOU I. producesforward motion. whkh was inversely propJ1siooaito viscosity. " . impossible [GIay63]. Elegans 125 awlied to die ~ gment and an equal aM OpJX )Siteforce All nemalocks. This ratio. rove.. : h ~ g. Parameters 01simulation to maintainit while moving. UI. somedistancefrom that position. It mustbe noteddlat die real wonn is able to function [Chalfie85 ] ~ "besad~ of how the coob'ol of evenwhendie e~ tive viscosityvarieswidely. lU. Theratio of m~ le forcesto die passiveelasticand In dlls theory. don is dlat under~ e cooditionsthe systemis bistable. however. = .l variables . pI' Opulsiooin C. the m~ le action at a given positim along Internalpresureforcesis more important This ratio COD .~ velocity is pI' OJX)rtionai to force and fixed offsetfrom tile phaseof tile body wave. In tile simple case examined in tile keepthelengthsof the segments constant literature[Gray63].~ .v1. dIeD propJ1sioo would be . . Some protozoa ( Lighdlill76] have Thesimpleglobal wavealgorithmwasusedto study specializeds~ tm'eSdIat make die ratio less dIaDone. Simulation of the Worm C. and die The simplestaOOmost galeral cootrol algorithm viscosity. + vI. Thephaseof tile m~ le wave wasset to a B~ u.. 2. Elegans coold aCbI8Ily be implemented . have slip ratim greater dIaD one. What is left out is howto establishtile body wave. This model is more C<HDplex . die ratio of die This algorithmwas implemented . The most striking abava - aOOpropel daemselves wid! forward pro.: dOD. . ninety degrees~~ ~ of tile Runge-Kutta medlod[ Press88 ].1.F .T. The results also explain . ~ Iving dIese alreadyestablishedaOOonly instantaneous - motion is COD equationsgives the velocity of eachsegmentThe equationssidered .V.gating waves. the wonn's body is detenninedby the Cmva Iure at a )X)int uois die amplitude of die waves used in l<x: omotion .. The Many qualitative and some quantitative experiments variablesare die 3 ccxopcx1ents of velocity at e. changing dIis ratio measured globally at ~ h time step. body wave. are usedin die simulationonly in die fonn of dlat wasusedin tile experimentswasa sin~ dal waveof dleir ratio. . sionrequiredto keepdie ~ gmentlengdlscoostan L 4. the dynamicsof propJlsion. For ~ a: ts to il M :~ the amplitude of die CI UTentbody metiK>dsof controlling locomotion(for instanceany dlat waveform. Propulsion 0 = .-.while an offsetof 90 degr'eesproducesdie sim- use an internal oscillator) dIis parametern~ to be pie forwardpropJlsiondiscussed. blown. This is similar to the medm ~ ibed in parameterodlerdIaDdie obD' ved wavefonnof die WOrlD . undulatorypropllsioo is not A diftk: u1L sin~ dal were integratedover time by the ~ d order waveof m~ le contr. which similar results.l equationsin 4N. However. aI KI was DOt minesdie e~ ie~ y of undulalorypropJ1sioo . dividedequallybetweenits two neighrors . remaining N-l equations 4. on die mechanicsandcmtroi of unduJalOl'y propJl- mentand die tensionbetweenQCb pair of ~ gments. The lion were perronnedusing die simulation. . even dIis is not dynamicallyimportant muscleaition.C. aI KI was foul Klto give dragcoeflk:lents for motionpelpelldicular to die bodyaxis aOOtangentto die axis.

There are several seconds . it can looseits propulsivebody wave . Conclusions contributingto forwardmotion. Propulsionis bistablein the region betweenthe This would work. where the problemsof This wavefonn correspoodsexactly to our exaggeratal maintainingthe body wave are not considered(as would baIf wave. as the dip in the cold start cmve just beforeit fix~ offsetarefairly obvious. Complicationswonn moves forward. a sb'aight sectionis Connedin long rangePlanningpossiblein its envircxlmentis very die body. but there is no evidenceof a nemal drop off in the cold start curve and dlat of the wann start mechanismfor changingtheoffset It shouldbe noteddIat. Becausedie functionsof for abouttIueewaves. ~ ~Uc ~ . cmve. die wonn revera: be occurringat die sametime. a phaseoffset of 90 degreesis clearly best. The body wavefcxms propulsioocanbe synchronized . sucha nleci1an transientshaveonly minor effects. synchronization timeswidt a changeof direction. ~ }X' OPJ1siooandtm'Dingcannotbe uncoupled .Uc .126 Ralph Hartley ~ lent propuisioois possible. It also die wavefonn. the worms use a behavior known as . It is unclear how the . It ~ ~ n noted from obsavations oftual The critical parametersfor pfOlXIlsion are the slip wonn' s [Croll7S] dlat. curve showsthe perfonnanceafter the wavefonn The worm may also use a small phaseoffset to was first estab~ by 60 secondswidt an offset of 60 establishthe body waveaI KI then switch to a 8ger value. This 4. for difftlent . The wonn gets very limited information about rmucespropJ1siveeffx:ieIK:Y. however. and self-maintaining. reducingdie reactive behavior. but diey havedramatic8llydifferenteffectson studied in isolation ~ ~Uc . The amount of systemfor studying magnitudeof a half wave. degrees .of the way in which the wonn ~ 1\ Jmingin C. Elegansis a relatively simple matter. The simulation of the mechanicsof C. occurin a sin~ daIly bent capillary tube).of die ~ t dlat propJisivewavesmust ~ obstruction . andis neverableto ~ its path. in direction. ism shouldbe apparentCircuits capableof generatinga however. and for differentinitial conditions. die turn die headaOOdie rest of die body follows. If die wonn is operatingin whatlies ahead . or occasionallyat random. Bodl behaviorscan produceturns of any sensingandseekingbehaviors . The lengdl of the testswaslong enoughdlat starting - widt all daeneuralconnectionsmapped . musclewave. Turning wouldbe a solutioo~ -!!. enough d1atit never in operates the bistableregion. This sectioncan produceno thrust. For offsetslessdian dJis. : kward motion. die low speedimpli~ Thereare two possibleways in which ttD'Iling and by a low p~ offsetis acceptable . Be!:. aI KI thencootiliuesforward. All muscleforces are then S. In the ideal case. so as to ] coopies diem apparentlyused by C. When die head is ttD'Iled in die locomotion and die motion based grOOlent detection ~ direction as die current prop J1sivewave . FigureS showsbodl furdler studies widt die simulation wiD ~ lude typesof turn. thebistableregime. andthe phaseoffsetbetweenthe body waveand the omegawave (due to the shapeof the worms ttack). or exaggerateone. revealsadditional issues. altogedler die necessary bodywavedoesnot fonn spontaneously . l~ omotionproceedssmoothly. The wonn caneitherskip requiredfor forwardandb. dro~ off. This The ' ' cold start" cmve showsthe performancewhen the would requirea sacrificeof speed..-~ relatively 1iuIe betw~ n dtesefunctionscouldbe important time is spentin b. they can be seen.the worm's body was initially straight. The worm may use a p~ possibilities offset small valuesof the slip ratio. C. as a function of the phaseoffset. wheneverthey makea largechange ratio (an environmentalvariablebeyondthe wonn' s con. start' . Most of die time. Thesebehaviorscannot be magnitude . && " ' bOl). while die " wann f_ testspeedis alwaysin thebistableregion. . : tual Figure4 showsthe distanceb' avel~ in 60 simulat~ wonn maintainsits body wavefonn. Elegans thereis a compooentof the muscleforcesdlat act to maintain clarifies much of the behaviorof the real wonn. to J1sioo . is also a good m<xIel tightly prop Elegans If die headis bJmedin die OpJX )Sitedirection.die nature of unduJatay ~ propJ1siveeffl:ielK:Y. Whenever the wonn hits an arise~ ~Ut . 2. but . and greatly limited. : kward motionareidentical. Elegans [ Ward73 exaggerate onehalf wave. Anofherpossibilityis d18tthe phaseoffsetfor ~ - wanI motionmaybe smallerdIaDfor forwardmotion. usedforwardandba: kward motion. one half wave.

2 ~0.~ J N0 ~ .e .". 0 .\.... \0 &~ is ~ > ~ 01 < it!. ~ 0 .....1 -1..(a) TUrningby Skippinga half wave...."lI. 25 ~ " 8-fI ~.\ Simulation of the Worm Elegans 127 .O1 01 .9J 20 40 Awe O & 60 )t80 e(D epeeI 20 )60 40 80 Offset Atue (D a egre ~ ~ . '' ~ 0000000 .. i=~s.I- I00 0 .1a ()5 0. ( Figure5 . ] ~~ (b) Turningby exaggeratinga .002575 5 lO 1 .I\.( b 3) "~.

Volume56. E. The ~ ign of . pp Elegans. B. " EIegans : a Pr0- ] R. Thomson . Soudlgate. AagellarHydrO [ Ughdlill76 .128 Ralph Hartley References ~ nO' habditisElegans : Identificationof Aurac- tants and Analysis of die Respo~ by U~ of ] R. 1953 " ] J. S. Soudtgate [Cha1fie . 1985 1986 Ic . tiThe Nem ' al Circuit for Nematode . Rowley . S. Garrido (ed) . AchievingArtificialIntelligence Mutants" . W. 1975 (Eldos9O] P.163. Gray. and the canmunityof C.Number SIAMReview . ~ umber3. Croll. Part Science Jollmalof Microscopical 4. pp 135-154. Aannery ] W. 1988 159. Philosophi- TouchSensitivityin QKt . Chalfie. Quarterly [Gray53 . Volume94. ThomDl .. Erdos. pp Section B. MIT AI Memo899. Hartley [l1artley92 . Ward. pp551-578. Wood [ Wood88 Behavior of dte Nematode Caen<X11abditis . C. l990 " " ] J. 1991 && ] S. " The S~ ture of die Nervous S5] M. Sulston. pp 817-821 ThroughBuildingRooots. ColdSpringHarbor Laborat y. H. Volume 314. of Neural Networks . The cal Transactionsof the Royal Societyof London. White. J.. Brenner . Elegans [CroIl75 ] N. The Locomotionof Nematodes " Biology. NumericalRecipesin Ct CambridgeUniversity Press . J. he Neural Basis of die " StatisticalMechanics Locomotionof Nematodes . J. Lissmann [Gray63 . Brooks (Br00ks86 " . Journal 01 Experimental Volume41. Press (Press88 . Simldalion pp 153. White. 1992 " . Number4. Springer- Vaiag. E. Bremrm. Lighdlill. E. Proccedingsof theNationalAcade~ of " ScienceVolume70. Journal of Neuroscience . Cemotaxisby the Nematode [ Ward73 . . UndulatoryPropulsion. ~ norba1xtitis . Teukolsky ling. Gray. System of die " J. Volume5. Oynamics 2. L. givenat ArtificialLifeIll . Number3. Vett - . Jarities. n<X'halxlitisEIegans" . Rockland SimulaOOnLanguages for Systemswidt Multiple Modu " . 1988 && [ ROW Iey91] S. Caen<X' halxlitis EIegans . S. Componentsand Patternsin dte ] B.176. Journal of Zoology Volume 176. ] M. Niebur. Issue 1165. The Nematode Caenorhal researchers Nlitis EIeganspq. 956-964. pp 1-340. Talk posal to simulate a Animal Simple . 1976 Volume 18. 1986 [ White86] j .

or perhaps even faster . vision . I will discU BBa very simple vision based corridor following systemwhich is in day-to. For this reason.86. and uses only cheap " off-the-shelf" hardware. This sensordata. tested systems for vision . a roulhly comparable . but they are not of sonar data [7][8]. system contract NOOOI4. examine take the form of replacing a subsystem of the A major source of this simplicity is an analysis of the agent with another system which is in some way leBB expensive ' s niche. - In this paper. Depending on the context .K . and in part by the Advanced 2poUy was built for 820K ( parts COlt in the U . 1. The optimizations we will tested. The performance of the system is dependent on an ' analysis of the special properties of robot s environment Polly is a low cost . The system runs in an un - modified office environment in the presence of both static and dynamic obstacles ( e. one Navigation is one of the most basic problems in . a much -simpler test . people) .g. but which n onetheleBB does the same job .K . Most system i have relied on the use property does not hold of all habitats trees blow in the wind and waves crash upon the shore. Rather than having to do vision communities. One of the limiting factors in a complicated analysis of the various objects in view to the designof current navigation systems. substituted for a more complicated recognition system . as with many determine if they look or act like agents. navigation has have any moving objects other than people.2 The the robot .0885 . and best Figure 1: Hardware architecture of Polly . Such an make clear the dependence . The system is notable in that it is very fast can allow " optimizations " to be performed on the control ( 15 frames per secondin the current system) . robotics. .iau ( see Horswill [4] for a more detailed precompiledmaps [10] . For example . " received a great deal of attention in the AI . cheap .85.based.and so the property partly determines of the set cases agents . thus . has been the availability of reliable simply check if they re moving . could be bought for roulhly 810K US .based navigation to date. and robust visual navigation system 545 Technology Square Cambridge. diacU88ion) . and to develop a theory of how those properties can simplify the design of an agent . MA Abstract In this paper I will discuss a system which uses vision to guide a mobile robot through corridors and freespace channels. I will discuBB how habitat constraints at MIT . autonomous robot built .1 The Polly project This substitution is an optimilation in the sense that the motion detector is leB Bexpensive than the recognition I Support for this research was provided in part by the system . USA ian Gal. or on vision [9][3][5][6][13][1] . In all . the unreliability of the available sensordata was of habitats in which an agent that assumes it may survive a major concernin the research. I will refer to such properties as even avoided the use of sensordata entirely in favor of hAbitat CODItra. very well systems of simple agents. Someresearchershave .dayusehere In this paper. such property is that office environments do not generally Sincethe ability to safelymoveabout the world is a prerequisite of most other activities.0124. The system is among the simplest . we may mea- University R~ ~Pch Initiative under Of Bce of Naval Research contract NOOOI4. 1 theoretical goal of the project is to articulate a number of usefUl CDmputAtio A Gl propertiel of office environments 1 Introduction . and computermotion is a cue to agency. The motion constraint outlined above allows a motion detector to be guidancefor the designof future systems. most effective . I will describe these properties and discuss to help study how properties of the environment can simplify how they simplify the computational problems facing the computational problems facing an agent . ) . provided agent analysishelps of an agent on its environment and provides that the habitat constraint holds .S. 129 A simple . but Research Projects Alency under Of Bce of Naval Research today . the robot can ' other robotic systems.

In the system presented recognition system. while simultaneously which is not presently in use. a 16MIP digital signal processor (Texas Instruments TMS320C30 ) with 64K 32. the corridor follower . A carpet . ~ I . yielding a motor command . All these components are run in pseudoparallel approach to design to develop an efficient visual system fashion on the DSP: at each moment in time . they are indeed simple enough so that each one can be run for each image frame . and what facets of the agent would have . . Nearly all computation is done on the DSP. The corridor follower we feel that vision should be used for everything . ' J Work to date in its projects . A system for navigating corridors and relatively uncluttered robot . . a video frame buffer / grabber . The place recognition system and person detector are The computational hardware on Polly consists of still under active development . been to determine what particular pieces of information The corridor follower nearly always controls the robot are needed by the agent to perform it ' s activities . A rudimentary place recognition system an agent ' s specialilation in terms of habitat constraints . very explicit the way in which an agent is adapted to it ~ . and commercial microcontrollers Corridor following is a common navigation task in office for voice synthesis and motor control (see environments . computer . A plan executive for forcing the robot to perform fixed and optimilations . sure expense as the actual monetary cost of building a . they can be run in parallel very efficiently . corridors lined with rooms on either side. and when it is moving . carpet bound - & ' 1 detector . and the plan executive can issue turn requests to the turn box . All other modules are built upon the corridor follower : the turn box can about to hit something ?" and " what is the axis of this " override the corridor follower to turn a corner . which will allow the robot to run unattended for extended the robot grabs a new frame from the camera ." which pilots the robot out of cuI. . as implemented in software. and repeats the cycle . thus much of The fact that Polly uses only vision for sensing was due the work of getting from one room to another consists of to lack of engineering time and experience. or some other measure entirely . By anal Yling . and plan executive are essentially finished .desacs which those properties allow . Office buildings tend to consist of long figure 1) . can only move in the direction in which it ' s pointed . : J system ~ I frame oorrldor plan turn motor unwedger matcher executive follower turn oontrol Figure 2: Conceptual architecture of the current version of the navigation system. Corridor following can be broken into the complementary At present.130 Ian Hors \vill visual distance .bit microcontroller2 Corridor following ( M68 B Cll ) for I / O tasks . the place corridor ? If these systems are simple enough. Even when the robot is not in a then to design complete visual systems for extracting corridor . only approximately keeping I and r comfortably large . to keeping the variable 8 in figure 3 small . At present. 1 . a simple 8. not because driving along a series of corridors . An " unwedger. runs each " " periods ( hours ) and to give primitive tours of of the components . Such an analysis makes and dead ends. issues the MIT AI lab . Polly consists of : problems of keeping aligned with the axis of the corridor and keeping away from the walls . A simple person detector based on bilateral symmetry putational problems facing the agent and the solutions . A unit which overrides the corridor follower to perform environment . or the biological cost of growing and feeding extra spaces (reported on here) neurons. the corridor follower is still used to attempt to each piece of information . - : : : : : 1 . As of this here. we can place specific properties of sequencesof actions ( useful for debugging ) the environment into correspondence with specific com.bit words of high speed ram3 . the agent might have distinct systems for answering questions such as am I " go forward without hitting obstacles. the unwedger. This amounts 3The DSP includes an additional 1Mb of low speed ram. Since Polly 10KW of RAM are in use. forward motor oontrol ~ ' : : : : . described here is intended to be used as one component among many which cooperate to allow the robot to participate 1. The connectivity of the navigation components is shown The implementation goal of the project is to use this in figure 2. 'Thus . unwedger. using an inexpensive writing . the p088ible consequencesof changing that open. Our general approach to design has the motor command .boundary detector ( see section 7) to change to adapt to the new environment .loop turns environment .

then the system ' may oecillate . velocity ofo (I ..r ' ) + . . seems a waste of In practice. / e. and the relative expense of the design.. This is not a particularly = 0 (1' .. numbers . I porarily turn away from the axiS of the corridor . 9. and that it should stop when there is something close to it . ej we can substitute any monotonic function camera. B robot should move forward when there ' s nothing in its where / is the monotonic function . I will a system that first constructs a 3D model of the environment discU88two special properties of the environment which .. ' ' . I' . if at all . and 9' .. and f 10 as to keep I and . Here 0 and . 8 will do the trick .. If these conditions are not met . c' . 10 moving away from a wall requires that Polly tem . compress down to a single number .+motor The two constraints on forward velocity are that the ~ . We can represent this schematically as: 9. can make it easier for the visual system to estimate these then computes the axis .1 Steering for comdor following => . then finds the walls of the corridor in the model . We want the robot to move at full speed .' .0~ which are measuredby the visual 8Y8tem . this control system of 8 which is zero when 8 is zero . obstacle in front of the robot . only to adjusted empirically for . the corridor follower has to make a tlI=-dr=.. / e . eop ( . way.+motor the 8teering rate t is controlled by 8' .. and 0' . . system of the form : s . Thus will discU88how the corridor follower computes the turn the problem for the control system amounts to control . we have not measuredthe variable 8' directly energy. We use the rule system must be fast enough so that we can treat it as having lero delay . of the corridor from the walls . and then turns to minimize 9.4 Thus we can use any has worked perfectly well. multiplies the axis by some gain to drive the design of the visual system . and Robust Visual Navigation System 131 of . we can treat these problems separately . The control . and that the control . the problem for the visual system amounts to determining respectively . ' tan .8 are gains (con8tant8) building an entire model of the environment . eop .88' "dt efficient design however. 0' . = min l1mu ' d - l1mud (C . large and 9 small and : I' . and to stop when it s less than some distanced speed. how fast dt ~ to turn . which are estimates of I . In this section .! Controlling forward motion * . The choice of what architecture is best will depend on the resources available to the agent. Cheap . we don t even need to compute One could solve for 8 given z and an accurately calibrated 8 per .. it can be shown that = 0 (1' . I will discuss the actual and finally .r ' ) + . Figure3: Corridorfollowing. Simple .18. that is . a measure of the distance to the nearest when one of these conditions is violated . rather than just its acceleration . eop . since 3D models are both difficult if it is confident of it ' 8 measureof 8' otherwise with a and computationallyexpensive to build . meaning that we have direct control over its ' .ll ' ' ~ given the right conditions . but instead have used the image plane z coordinate Of course. ~ impoeed by the task on the architecture of agent are actually ~ quite modest .good result8. The conditio D8 are that the motor must be control led in when the nearest obstacle is more than some safe distanced velocity space. and drive rates from five numbers describing the situation ling .. . and how fast to move forward . but in our experience . and then in section 5. in whatever of the projection of the axis of the corridor. 9. and that they admit a huge space of ry pouible architectures .+ motor in our 8Y8temis: 6 . any system is more easily measured. these variables are coupled . . In section 4 .8in6 decision about what direction to turn . The 8Y8tem Here the double arrow at be beginning represents input drives the 8teeringmotor at a rotational velocity of from the sensors and the boxes are successive transformations dB of the sensory data . turning motor . Since our robot has independent motors for turning and driving forward . and r ' . a measure of the There is a huge space of p OBBible solutions to this problem visual system ' s confidence in its estimate of 9. B {~ ~ . Consider the subtask of aligning with the corridor . any system which computes 9. which manner .8tan. In particular . problem is actually much easier than the perceptual An obvious way of performing this task would be to use ' problem of estimating these numbers . " . The projection z is equal to of the form . the other tasks which the agent may have to perform . and 9. Intuitively . if the speed of 3 The control system the robot is " then we have that : At any given time . The point of this analysis is simply that the constraints .r ) . where' is determinedby the focal length and resolutionof the cameraThus the actual control law used => . Furthermore .d.

-+ certain threshold . or even as an ( uncalibrated ) measure B {~ ~ B of absolute distance . This property was referred to as Note that this system will effectively weight segments by the ground plane cout Nint in [4] . All vision computations are performed for each frame . the axis of the corridor is represented .scale images vironments covering a field oCview of 110 degrees ( 1. but for our purposes we can treat it as extracting only the ' This observation forward vanishing point . . and 80 any violation of this uniformity must be an ' The algorithm of BeUutta et ale is actually more complicated object other than the Hoor. This will be fine provided that the long lines Another important property of office environments is in the scene are the lines directed toward the vanishing that they are generally carpeted . office environmentsAs W88mentioned above.9 radians ) . Bel- of the navigational properties of buildings ) . the Boor . Estimating distance and parsing the visual world into The overall structure of the visual system is shown in objects are both very difficult problems in the general case. only fine. Polly s habitat .ground separation . bitrarily difficult if we consider pathological situations such as camou Bage or crypsis . which causes the robot to smoothly decelerate as it ap. a the distance of a point P from the camera will be a ' This algorithm . re coutraiat in [4]. represent this schematically 88: One such property is that office environments have a Bat ground plane.6 We can environments . it is its own tiny segment: still a perfectly useful measure for determining which of two objects is closer. their length .plane projection . The constraints used in the optimization of the system are given in figure 4. We can simplify used as a measure of distance to objects resting on the the system if we make stronger 888umptions about Boors. while leas computationally expensive strictly increasing function of the height of P s projection in the image plane. p . Fortunately .is more a constraint on the agent . to an obstacle. See[2] . Computing the to the Hoor should have uniform image brightness . is useful because it allows one to herd the robot from place to place. BE ~ . are actively structured by both ' by the z coordinate of it s image. the carpet will appear to have a uniform the edges we' re looking for should be very strong and reHectance. If this is true . S of the visual Design system The visual system estimates the axis of the corridor and 4 of officeen - Computational properties three distance measures from 64 x 48 pixel grey. Backing up . which can require pred- itors to learn to recognise prey on a case by case basis 5 . Note that " known camera tilt . For example . called the than this in that it extracts multiple vanishing points. and to back up if it gets too cloee the computational problem of figure . We can remove the step of grouping correspondence between heights and distances cannot be edge pixels into segments by treating each edge pixel 88 known without first knowing the specifics of camera . is still rather expensive. extracting to be used than would be necessary for an straight line segments. and if the carpet is uniformly . goesback at least to Euclid. designers and inhabitants 80 as to facilitate their legibility This can be estimated by finding the vanishing point of ( see P&88ini [ 11] for an interdisciplinary discUBIion the parallel lines forming the edges of the corridor . 1 Computing the fJanishing point " (see Roitblat ' [ 12] . This property . and 80 image plane height can be than 3D modeling . that Edge detectors can also be extremely expensive. can greatly simplify proaches an obstacle . 260) . and the exact the environment . figure 5. such 88 a gradient threshold . than on the habitat . whether an object is closer than a ~ . Since from a distance . The special lutta et al [ 1] report on a such a system which extracts properties of office environments allow much simpler architectures vanishing points by running an edge finder . and their carpets generally "have point . 8 . we should be able to use a very simple edge illuminated . then the areas of the image which correspondstraight detector . and performing 2D clustering on agent which was to follow arbitrary paths in arbitrary the pairwise intersections of the edge segments.132 Ian Horswill Figure 4: Habitat constraints assumed by the visual system and the problems they helped to simplify . For a given height and orientation of the camera . figure / ground separation can be ar. 6acigrou Ad tezt.scale texture . That is to say. While this is not a linear measure. upon which most objects => -+ rest.

in this case. This reduces the number of points to consider so now our system looks like this : from O ( n2 ) to O ( n ) . Cheap . -+ function of the height in the image plane of the ~ ~ ~~ . thresholded gradients ) : . that is . -+ ~ . but for this application each direction . Note that smoothing is performed prior to edge detection to remove noise . We can then replace the pairwise intersections with the intersections of the edges with RD . objects in the way it will simply cause the robot to steer around them .plane constraint to generate the RDM : the distance to the closest object in a given direction is a monotonic => -.all . by the background . which looks for modes . it will lie in the line given column . and so the robot will still steer in not actually a problem however. then the vanishing point will always have then simply the height of the lowest non . then the mean will tend to They are simply the distances to the nearest non . A radial nonlinear functions of the actual distances7. Thus having the system right .ground ne88. -f -f which is also more efficient : => ~ B ~ ~ => -o .ds. The second the distances to them . then . then there is no difference anyhow .angle of the camera is held constant by the of the image correspond to directions . Then we could find the distance to the we are mostly just concemed with whether a left wall by finding the minimum distance given in the given object is too close or whether the left side or the left side entries of the radial depth map : right side is closer. Simple . if we assume that the intersections of the edges meet with the :Boor. the distance is camera mount . M( z ) = min { ylthe point ( z . If the floor ] ~ ~ ~~ oE ~ ~ is te:x:turelesa but the walls generate edges where they Finally .:Boor pixel in a the same ' Y coordinate . if we had already labeled every pixel as " " " can be done using only a few machine instructions . that is . and also reduces the clustering to a ID problem . The sign will still objects on the left and right sides of the image . where n is the number of edge pixels . The carpet boundary detector is discussed in section 7 . -o where " F / 0 " is the figure / ground system . then we could use the resulting system is then : ground . by an distributed .! Computing distances avoidance capability . The vanishing point unit is shown as a single box because all the steps of the vanishing computation are perfol :med simultaneously . non . thus confer ring on the robot a limited object 5. of the scene.g. If we had already somehow solved the figure . There are two problema involved in estimating the left . Suppose we had computed thing worth noting is that the distance measures are a radial depth map . The being either "floor or not :Boor .dstance(c ) edges ' map right-distance (r ) vilshlng vanishingpoint point variance Figure 5: The portion of the visual system devoted to corridor following .texture not pointing toward the vanishing point are uniformly constraint . Since columns If the tilt . with the mean of the z coordinate : => . since if there are other the correct direction . intensity gradient ( the rate of change in image bright . a oE ~ ~ oE ~ ~ .B A number of things are worth noting here. and depth estimation to determine is actually advantageous in this situation . denoted by V ) at a pixel and testing its magnitude problem . -o => . RDM . and center distances: figure / ground separation to make no distinctions between walls and other obstacles find the walls in the image. Thus : ' Y = ' Yo for some ' Yo.ance (I' ) ' camera RDM oompress center. for which these nonlinear measures => -f -f ~ ~ ~ . then we can replace the clustering operation edge detector ( e.corridor edges in view . and Robust Visual Navigation System 133 left. If there are no such objects . y ) isn ' t :Boor} ' Y = ' Yo. For some depth map gives the distance of the nearest object in applications this might be unacceptable . First of . it too can be replaced.:Boor move toward the center of the screen . a lowest non-:Boor pixel in the image plane . This does have the disadvantage that if there are many I' and r ' are not necessarily the distances to the walls . This is be correct however .

any frame grabbing or output to the base (a single frame is grabbedat the beginning and procelled repeat- " " edly) . including the hardware platform for the system which we hope " " persondetector. of its time . it will turn either left or right camera (seefigure 6) . Fullsystem 67 15 The simplicity and efficiency of the system make it Corridorfollower 67 15 quite inexpensive compared to other real .loop turn when the higher . When the robot which is a fast as the system can read data from the is blocked by an object . This sometimes causes problems . we havefound the systemto be quite reliable there is no way of specifying a desired destination to in general. . C30 DSP boards are now available for personal NoI / O 15 67 computers for approximately $1. Since these are never exactly repeatable implementation is heavily 1/ 0 bound however.. which it has been tested (Boors 3. This is Jell of a problem than one would partly because for a given orientation of the robot . does not have a single carpet . the robot doing transfers over the V M Ebus to the frame grabber could emerge at any point or even get turned around ' and display. ! Failure modes without I / O or the vanishing-point box. since no camera has a 180 ' ' also cannot brake for an object uniell it can detect degree field of view .time vision systems I/ O only 67 15 . Thus the " corridor follower would be quite cheap to install on an Figure 6: Execution times for 1000frames. If shadows are sufficiently lIieu . Boor. No VP is the collision avoidancesystem run 6. increasing function of the measured distance . The requirementsof the computer allow the entire computer corridor follower then realigns with the new corridor an system to run oft' of a single motorcycle battery for as continues on its way. Finally . All execution times are for a Texas Instruments TMS. Its . and strong they . the system acts in a " point and shoot " mode: it moves forward as far as p088ible. Finally . It has .320C30-baaed The system runs on all Boors of the AI lab building on DSP board (a Pentek4283) running with no wait states. The first Boor .major deficiency in this situation is that (seebelow) . The system is also present implementation runs at 15 frames per second. Again . The system ' s major failure rather are measure of the distance to the closest point in mode is braking for shadows. The a dead end or is stopped externally . a series of corridors and junctions by forcing the robot to make a small open. win cause the robot to brake when there is in fact no obstacle. While there are caseswhich will fool the braking system it is also capable of moving through more complicated spaces. There the system three lines are the samebecausethe systemcannot digitile brakes for the reflections of the overhead lights in the framesfaster than 15 FPS. and 80 it .9 Performance outside comdors able to run the system as fast as our robot base could run without shaking itself apart (approximately 1 m/ s) .2K US and frame grab- NoVP 10 100 bers can be obtained for as little as $400 US. seen at least 200 hours of the boundary (see section 7) . but this can more or Jell be expected the perpendicular distances to the walls . The corridor follower has been running for nine months This problem was dealt with by explicitly recognizing and is quite stable.level lowsit to be implementedwith a relatively simpleand inexpensive computer such as the DSP. Effectively .deterministic . I and r are not even measures of an edge on or around it . existing system . each with a different color . The system s performance is good enough ticeably better on a system with a more tightly coupled however that a higher .level system can lead it though DSP and frame grabber. This robot and object . the robot is effectively non . The system are quite adequate . but several carpets. completely . veering 6~1 Efficiency away from obstacles. where the robot spends most lIeria. The boundaries between 6 Evaluation these carpets can thus be mistaken for obstacles. Full sy. The present system also has no memory and so cannot brake for an object unle88 it is actually in its field of view . While the system was designed to navigate corridors . This makes it one of the most extensively tested and reliable visual navigation systemsto date. which has very shiny Boors.9) except for the 9th The proceaaorhas a 60ns instruction time.134 Ian Horswill Test TIme(sec ) Frames / sec long as nine hours. and continuing until it reaches The system is very efficient -computationally. non. this is not a problem in practice . The modest power system wants to take a new corridor at a junction . partly because our camera has a relatively wide field of view . Thus spends much of its time waiting for the serial port and in a "forest environment " su~ as figure 7. I and r . We havebeen 6. but of all vision systems. servicewith many continuousruns of one hour or more. We expect that performancewould be no. No I / O is the corridor follower without will cost less than $200 US. All computation boxes in figure depending on the exact position and orientation of the 5 are run on every frame in (simulated) parallel. We are also working on a very inexpensive Tern" is all code presently implemented. the system . the expect because shadows are generally quite diffuse and perpendicular distance is another monotonic and strictly so win not necessarily trigger the edge detector . and !lice the 7th Boor of the lab . This efficiencyof the systemal.deterministic in these situations .

. Walllollowi Rg A When the systemdescribedabovereachesa largeopen the wall which is in view will act as a repulsive Boththis ted from paper . . To hon. it is a very bad way of actually getting to a destination unlessthe . Indeed bitrary environments . Cheap . and the systemdescribedwithin benefit- disc ~ Di U88io~~s with Lynn Stein. 7 Extensions. The criterion for a carpet boundary is that it Another pOllibility would be to build a system which must be a weak horisontalline with no surrounding texture could rapidly switch between a number of domain - ' . not necellar ily involve buying better cameras and bigger computers . dB " d becomes: =0 t' 1-d case where only the left wall is in view . Thus . I suspect that quite a lot dark pixels as obstacles to be avoided . Maja . found . One approach would be to build searches for carpet boundaries and briefty disables the a hybrid which used the simple system when pOllible . and by refusing can be done with them . then it will treat It remains to be seen how far we can go with simple . the robot will try . Polly avoids these problems by treating of the environment . Simple . . . detection of horisontallines when a carpet boundary is and the more cumbersome system only when necellary . A previous version of the system dealt eralsystem may also require allocating scarce cognitive with the problem by weighting vertical lines twice as or attentive resources which would be better used for much as horisontal lines. shadows and bright lights radiating One may be tempted to object that this system is too from office doors can sometimes be sufficiently intensedomain specific and that more complicated techniques to trigger the edge detector .it doesnotuse high resolution . Where tl. ontal in the image when the robot is successfully begin with .. single the desired distance to wall . there is nothing in view whatsoever. and Robust Visual Navigation System 135 world consists only of nice straight corridors . Figure 7: A forest environment . the control law ( )the . to keep that wall at a constant distance . I think that this is misguided however. the blank wall as being an empty Hoor and attempt to domain . and drives close enough to the [ 15]) . It does notbuild scribedaboveare worthwhile . A VRP could be quickly configured by the central FeGt' 01 the dat'i system to use different strategies for different situations . as described above. . inexpensive vision systems which perform modalities to the system. the system described to drive forward if there is insufficient texture in the here is a demonstration that it is practical to image . .specific strategies which rely on special properties drive up the wall . and that the solutions to vision problems do those sensors were not available on the robot . . and is in References fact a very good way of avoiding obstacles. in the . it does not use calibrated carefully depth data . tends toward situations in which it is effectively blind. Since these shadows are necessary to build practical systems for use in the always radiate perpendicular to the wall . that when only a single wall is in view . causingthe robot to turn away from it until Crimson Mataric Mike Bolotski JoseRobies and . but useful tasks . detailed models of itsenvironment. will happily drive and learn to use domain . .oruse whichit does not do . such as touch or sonar . As discul Sed above. much of its power comes from its specialisation . its very generality would likely make it toward vertical lines . . thus making it truly adaptive (see Whitehead and . The gen- these shadows. specific strategies . Eric space. While this is sufficient for following corridon . . 8 Conclusions . we can make it less sensitive to much slower than the system discu88ed here. such a system would be able to recognise The system. and imagery it isnotdesigned toruninar- VerticAl 6iuing . Similarly tasks . wall for it to fill the robot ' s visual field . can be fixed by modifying the steering control 80 . even if one had a single iruly general navigation aligned with the corridor . Thus it naturally David Michael. . . if the robot somehow mi88e8the boundary between Ballard for an interesting example of learning visual routines the Hoor and a blank wall . they appear real world . . . . . Ullman s Visual Routine ProceSlOr 14 [ ] is a particularly attractive architecture for this approach . . In either case. A better solution would be to add other sensory build simple . themost Curiously significantthingsabout thesystem Anumber ofminor modifications tothealgorithm de are the tbmp . . Ideally . Rod Brooks. The system now explicitly other concurrent tasks. By biasing the edge detector algorithm . This problem .specific strategies for visual through a dark room and hit the nearest obstacle.

2(4) . [11] Romedi Passini. Storjohann. 18:97 159. May 1990. pages761. Kak. Iatrodactioa to Comparitille 12 Cogaitio. 15 [ ] S. In Proceediag . Reactive navigation for mobile robots. Cogaitioa. Visual obstacledetection for automatically guided vehicles. Whitehead and D. [3] Y . ollie IBBB later- aatioaal Conletoreace on Robotic. Nil BBOn. . Robotics Institute . tomatio ". [10] ed. Visual routines. Van Norstrand Reinhold.huaettsInstitute of Technology. Borswill. volume 4 of Bawt"OAmeatal De. Stanford University. Gad A . Freemanand Company. von Seelen . [8] Bans P. New York . Graphic. B. IEEE . ropeaa . C. Kosaka and A . ia9. 1984. Gad A. AAAI . tore . . TechnicalReport . rail Compatatioa . M~ . iga Serie. Technical report. In 1981IBBB Iatematioa COB - letoreace on Robotic . Technical Report. [13] K. 323. GadImage Proce. 1986.. Goto and A . Master' s thesis. The cmu systemfor mobile robot navigation. Perceptioawit A aa Bre lor Mo- tioa. Moravec. Active perception and reinforcement learning. [7] Maja Mataric. pages99. Anificiallntelligence Lab. Zeilke. 1228. Ne. A. MassachusettsInstitute of Technology. Carnegie- Mellon University. - [ 14] Shimon Ullman.766. Fast vision-guided mobile robot navigation using model-basedreasoning and prediction of uncertainties. Cutting . MIT Press. Unpublished memo.136 Ian Horswill Spriag Srmpori. 1990. April 1984. SRI International. 56(2) .105 . . 1984. September1992.. . March 87. Warjiadiag ia Architect. t Aaaaal B. [9] Bans P. Characterising adaptation by constraint . Roitblat . May 1990. [5] Ian D. A distributed model for mobile robot environment-learning and navigation. In Pt I OCeediag Coa/ etoreace oa Artificial Lile. 1991. [4] Ian Borswill. [ ] Berbert L.. tomatio". Ballard. February 1983. June 1988: [6] A. T . W . Stens. and W . March 1989. B. Shakey the robot. The stanford cart and cmu rover. Certainty grids for mobile robots. m on Robot Naftgatioa. [2] JamesE. 1987. pages6- 10. Nils J. To appearia Compater VirioA. Mallot . Moravec. ollie Fir .


decisions are made at eveJ YleveL 1Wo alternatives exist to thesehierarehicaldecision stnJdW' es: (1) Maes has designed a non-hierarehical action selection mechanism. ' Mcfarland . The three feathers. That is to say. @ acknowledged to whatever means. The term actionhere refers to an entity at the 1owest' level.g. amongst others.ed.down control and lack of robustnessinherent in these hierarchicaldecisionstructures. Severalof theseterms are illustrated in figure 1. by systemsmutually suppresseach other. and (2) Rosenblatt &t Payton have outlined a hierarchical mechanism which does not make decisions at each leveL In this paper the use of the two types of hierarchies for action selection is disalssed and the Rosenblatt&t Payton approach with free flow of information . lining of the nest and combing of the system. Introduction Before commencing with the body of the paper it is necessaryto define some terms. combination of evidenceand the ability to select compromise candidates is supported. trampling. 1975] . Baillie~ 1indall - . have proposed hierarehical mechanismsfor action selection . 1. such as 1inbetgen and Baerends. E = escapesystemand P = preening system. in which. N = incubation anyone time. Some actions in figure 1 are retrieving eggs. Evidence from animal behaviour is used to back up this claim. that of the behavioural final common path [e.cns Centre for Cognitive Science Uni versity of Edinburgh Edinburgh EH8 9LW Great-Britain Abstract Several researchersstudying animal behaviour. " Superimposedcontrol systems" of higher order are to the middle and the demands of each Figure 1: Baerends model to account for incubation behaviour action on the effectors of the animal are mutually exclusive of the herring gull . has algUed against mechanismsof this type becauseof the tOp. Maes. Actions or fixed action patterns so that only one action may be canied out at are in the far right column.138 The Use of Hierarchies for Action Selection Toby Tyrrell email: lrtt @ uk. Action selectionis the processof choosing. one out of a set of candidate actions.

to make the animal reproduce.. actionssuch as IIay eggs' and lcomb feathers' . will then passdown to lower nodes in turn as long as and the innate releasing mechanismaRM) for each node is (ii ) there is one element in the set which is superior to activated (an inactivated IRM ~ locks' its node). . siblings.e. Tinbergen bossof B if A has a direct causalinfluence on B.. etc. The main action selection problem thus engendersmore specific sub-problemssuch as the need to obtain food. Following this is a ae- Mcfarland [ 1975] . (e ) is a setof overlappinghierarchies (after[Dawkins. Their activation energy (i ) thereis no node in the setwhich is superior to itself. The highest level nodescorrespond (2) A is boss of a node which is superior to B. selectionproblemfor an animal is therefore how to choosesuccessiveactions so as to maximise the number of copiesof its genesin future generations. Eachof thesesub. ~or a set of nodes A. hierarchies The hier- to passactivation down to lower layers. i. due to the ethologists linbelgen and Baerends. 1976]. scription of a different type of hierarchical mechanism. each node has its own separatethreshold which needsto be surpassedby The rest of this introductory sectionis given over to a the activation it receivesfrom several different sources discussionof other work relevant to the themeof this pa- . Action selection mechanismshave evolved to provide a solution to this problem. Some systemsof the mechanism . Parts of the mechanismat any level between that of the system level and the action level will be known per. Most mechanisms have separate parts devoted to (. B. The terms just defined. differ from those defined by observationof animal behaviour. . Use of Hieralochies fOlo Action Selection 139 Animals are vehicles for the transmissionand propagation of their genes [Dawkins. eachof state of node A. setsof overlapping hierarchies(one for eachsystem) as in figure 2(e). allowing it Figure 2 shows some example . If the sum of the stimuli exceeds the threshold then the IRM Ifrees' the node. Using to systemssuch as lreproduction' and Iclean- this recursive definition of the term superior it is now ing / preening' . its influenceon B is not mediated through as a 'Iworking hypothesis" and as such was only anothernode). IRM consistsof a device for summing severalincoming all others in the set. B . C. the needto caretor young and the need to find mates. The global action (b ) . two roboticists.most one) is gated by an is said to be superiorto B if (1) A is boss of B. and causalinfluenceis meant to imply that describedfairly vaguely." each of the separate sub. The definition of a hierarchyused here is taken from [Dawkins.1.problems. Direct is meant to imply that A is.problems will have its own requirementson the animal' s time which will need to be interleaved by the action selectionmechanism. which will be referred Figure2: Someexamplehierarchies . The lowest level nodes correspond to possibleto define a hierarchy as a set of nodes A. (c) is an overlappinghierarchy to here as systems . Each .immediatelyabove B in the This mechanism [Tmbergen.g.~::"I'I~~~~ and to help conspecifics to do so as well. rather than being specifiedexactly the state of node B is to some extent dependent on the . 1989]. shown in figure 1 are nesting. Thereis a hierarchy of nodes(seefigure 3). The I RMis act archicalmechanismsdiscussedhereareall composedof to block any nodes not relevant to the current situation. stimuli and a threshold.e. Firstly two hierarchical mechanismsfor action selection as sub-systems . proposedby Rosenblatt& Payton. C. which satisfy: The top. For two nodes A and B then A ' is 1. while being very similar to are reviewed.most nodes are activated by motivations such as linternal food defidt ' . 1950& 195i ] was put forward hierarchy(i. or if innatereleasingmechanism . They are both the product of extensive those used by Baerends. etc. escapingand preening 1976 . parents. the need to avoid predators. A which (except for the very top. In order to achievethe above the animal needsto stay alive . and to make it help conspecifics with similar sets of genes (e. In addition to the IRM threshold. offspring ) to do the same.

.~ ~ Act} . . . . . : . downwards arrows = causal factors . but system. and two . This type of mechanism is called here a hierarchicaldecision structure. 1. there is a winner . the sbong activation external stimuli affect which node (at any level ) becomes of instinctive behaviour of one kind prevents the func. . . We ss Z II III ' . " - " ~ . . .e~ ' ct' (fie gr 5 ' ' : 7 - major instinct 0 . 1991] ' ' hierarehyof instincts . exceptthat ''asa role. . Thereare no I RM' s. for the purposes of this paper.ns) ~ iss 5 ( ) 1 ~ 1 5th lfYfl 4 (fin rays) lWeiss ) ~ ~ .andof . . t . 1nd ~ ~el ( : : : : 5 . " . @ acknowledged to Oarendon Press. . . it tends " " tioningofanotherpattem [Tinbetgen. . 1950]. Maes5 Criticisms ' Figure 1 shows a model that BaerendspostuJated to account for the incubation behaviour of the herring gull [Baerends. horizontal arrows = innate releasing mechanisms . . r / ih " ~ .3."" tsl . Jrd l ~~t. . : : : . . the details of the mechanism are not' im portant ' I RM s and the inhl"bition amongst nodes at the same here. Thebehaviourof the networkis an emergent in Baerends'diagrams). I - . ~ f ! 2i 6th lcvel Welss 3 ! (MUott /es) ( ) / 1 . there is a feeding of activation which is distributedand non-hieran:hicalwith no central down through the hierarchy (or rather from left to right control. ii ( i II ' . active . . and all others are subsequently disregarded. and also that once a node becomes active . ~ J t ~ fJ ~ . Baerends 1. . " ~ . only the fact that (as with Tmbergens mechanism level is that in most