You are on page 1of 5

Nonlinear Dynamic Systems for Autonomous Agents Navigation

Siome Goldenstein and Dimitris Metaxas Edward Large

Computer and Info. Science Department Complex Systems and Brain Sciences
University of Pennsylvania Florida Atlantic University
200 South 33rd Street 777 Glades Road, P.O. Box 3091
Philadelphia, PA 19104 Boca Raton, FL 33431

Abstract tion and dynamic techniques (Grzeszczuk & Terzopou-

los 1995).
We present a method for the generation of real-time In our dynamic system approach to this problem, the
navigation of dynamic autonomous agents in game en- early methods were restricted to the generation of in-
vironments. The method is based on the use of dy- dividual behaviors such as navigation toward a
namic systems theory which allows the development of
sets of dierential equations that exhibit distinct be- goal. Recently, however, methods have been developed
haviors. The dierential equations are a carefully de- to allow an agent to arbitrate among a large number
signed set of attractor and repeller elds. Coupled to- of potential behaviors, and to generate complex se-
gether with a \behavioral" selection of the relevant con- quences of activity in a manner that is robust, yet ex-
tributions at each time instant are capable of exhibit- ible in the face of a changing environment (Steinhage
ing useful steering behaviors in complex environments. & Schoner 1997 Large, Christensen, & Bajcsy" 1999
To avoid local minima, carefully designed noise terms Goldenstein, Large, & Metaxas 1999). To achieve this
are added. Using this approach we are able to demon- result a second dynamical system is de
ned that op-
strate in real time behaviors such as single/multiple erates in the space of task constraints. This dynamic
target tracking with stationary and moving obstacles. approach, forces task constraints to compete for repre-
Keywords: Digital Agents, Game Animation, Motion
Planning, Dynamical Systems. sentation at the behavioral level. Thus, at any given
time the behavioral vector
eld (and the observed be-
havior) comprises a subset of possible task constraints.
Introduction The parameters of the dynamical system are chosen in
such a way that the agent's behavior is appropriate to
The importance of game and simulation applications the current situation.
grows everyday, as does the need for animated agents In this paper, we discuss an alternative methodol-
that operate autonomously in these environments. ogy that has its roots in behavior-based robotics and
These agents must be able to exhibit certain behaviors is based on a novel way of combining dierential equa-
autonomously, without user intervention. Among the tions exhibiting particular behaviors. According to this
various methods higher levels of behavior, and move- methodology, one de
nes a representation whose dimen-
ment decisions were investigated
rst in the pioneering sions correspond to agent behavior. Using this type of
work by Reynolds (Reynolds 1987), and then in work approach, Schoner and colleagues have developed a dy-
by others (Bates, Loyall, & Reilly 1992 Noser & Thal- namical system for robot path planning and control. In
mann 1993 Reynolds 1993 Tu & Terzopoulos 1994 this system a set of behavioral variables, namely head-
Noser et al. 1995). AI approaches (Lethebridge & C ing direction and velocity, de
nes a state space in which
1989 Funge, Tu, & Terzopoulos 1999) are capable of a dynamics of robot behavior is described (Schoner,
generating autonomous behavior, but typical such tech- Dose, & Engels 1996). Path planning is governed by
niques require complex inferencing mechanisms. This a nonlinear dynamical system that generates a time
may require considerable computational resources, rais- course of the behavioral variables. The system dy-
ing the question of scaling up such systems as the num- namics are speci
ed as a nonlinear vector
eld, while
ber of independent agents grows, or when each agent the task that the agent will execute depends upon the
has a completely dierent goal and behavioral direc- task constraints. Task constraints are modeled as com-
tives. In addition to these factors, agents must be able ponent forces, de
ning attractors and repellers of the
to interact with real-time moving objects that might ei- dynamical system. The individual constraint contribu-
ther contribute to or compromise the
nal goal. Other tions are additively combined into a single vector
approaches to this problem employ learning, percep- which determines the observed behavior.
Copyright c 2000, American Association for Articial In- Here we adapt the above methodology to develop au-
telligence ( All rights reserved. tonomous dynamic behaviors for games. In particular,
we devise a set of time adaptive dierential equations targets (Section . In our formulation, the heading speed
to rule the heading angle and forward speed of a given is modi
ed by either the heading angle information or
digital autonomous agent. Based on a principled com- the relative location of the obstacles (Section ).
bination of these equations we create a whole set of Based on our formulation, an agent ignores targets
relatively complex \low-level" behaviors which are re- or obstacles, depending on the scene geometry around
active in nature. To avoid unstable
xed points in the the agent at each time instance. It is modeled based
dierential equations1 we add a Gaussian noise term in on another type of nonlinear dynamical system, run-
each equation. Using this system decisions are made ning on a more re
ned time scale. This system out-
on-line and do not require any previous memory, train- puts weights that linearly combine the dierent attrac-
ing or global planning. The set of targets and obstacles tor and repeller contributions as calculated by the
can change during the course of the simulation, since system. An important aspect of our methodology is
the agent is able to make \smart" local decisions based that it scales linearly with the number of obstacles and
on its current global knowledge of the dynamic envi- targets in the environment.
ronment it is situated. An example of such a behavior In the following we present the details of each of each
is that the agent will temporarily disregard a target if of the two dynamical systems.
there is an unsurpassable moving/stantionary obstacle
immediately between them. It will then focus like a
human would do to
rst avoid the obstacle and then
The Basic Movement Dynamics
refocus on the target. The
rst dynamical system models the control of the
Our system allows single/multiple target tracking in basic movement of each autonomous agent. The move-
the presence of multiple static/moving obstacles. The ment is de
ned by a 2D vector representing the agent's
design of the dierential equations allows the tracking of heading angle and forward speed.
targets whenever their position is within the visible cone The heading angle of a given agent is controlled by
of an agent requiring only the estimation of its current a dynamical system of the type:
position. However, obstacles are processed in a local _ = f (env) (1)
fashion based on their relative location to the agent
and the target. Given our applications, in our current where env is the vector of variables which models the
implementation our agents are memoryless, reactive in environment (e.g., the geometry and position of the ob-
nature and depending on the situation (emergence of stacles and targets) and we describe in detail below.
new obstacles and/or targets) their movement can be According to our dynamical system formulation each
discontinuous. element of the environment can \attract" or \repel" an
In the following sections, we present previous related agent. We will therefore use attractors to model targets
work in the area, the design of our system and the series and repellers to model objects that should be avoided.
of real-time experiments. We model an attractor as
Movement Dynamics ftar = a sin( ; ) (2)
In our methodology we combine two distinct dynamic where  is the angle of the target's location relative to
systems to model the movement and behavior of each the agent's location and a is a constant parameter.
autonomous agent. The
rst system controls the move-
ment of the agent. The state space of this system is two In order to model complex environment obstacles, en-
dimensional, the
rst parameter represents the head- emies or hazards are distinct entities. Fire-pits, for ex-
ing direction, while the other speci
es its velocity. The ample, are clearly more dangerous than a large wall.
second system controls the agent's movement decision Therefore the repeller de
nition should have enough
making, i.e., its behavior. The state space of this sys- parameters to model the dierent types of objects. We
tem is the space of the agent's behaviors. The param- achieve this by de
ning a repeller to be the multiplica-
eter values of the state vector components determine tion of three dierent functions, Ri  Wi  Di , which result
which \elements" of the environment (e.g., obstacles, in being able to model the type of repeller, its distance
targets) will be used in the calculation of the agent's to the agent and the extent of its inuence to the envi-
movement and therefore behavior. ronent. We therefore repeller as
Each autonomous agent movement is described in po- fobsi = Ri Wi Di : (3)
lar coordinates. It consists of a heading direction and
a forward velocity v. The heading angle is controlled by Function Ri models a generic repeller, and is con-
a one dimensional non-linear dynamical system, which structed as:
consists of \repellers" placed in the subtended angle of ;i 
the obstacles, and attractors in the subtended angles of Ri = ( ;i ) e 1; i  (4)
In dierential equation terminology a xed point is a
point where the derivative is zero and acts like a trap re- where i is the angle of obstacle i and i is the angle
sulting in the agent not to be able to move. subtended by it.
The second function, Wi , is responsible for limiting graphic) and obstacle function (middle right graphic).
the angular range of the repeller's inuence in the en- The presence of two
nal attractors, indicated by the
vironment and is modeled as two arrows in the lower right graph, show the two pos-
sible obvious ways to get to the target and avoid the
Wi = 12 tanh(h1 (cos( ; i ) ; obstacle.
cos(2i + ))) + 1] (5) Configuration Vector Field Components

which models a window-shaped function and h1 is re-


ψ ≅0 −π −π/2 0 π/2 π

sponsible for the inclination of the window's sides and



is modeled by

F obs

ψ −2
obs −π −π/2 0 π/2 π

h1 = 4=(cos(2) ; cos(2 + )): (6) 2

Summed Vector Field

Here  is a \safety margin" constant.


−π −π/2 0 π/2 π

The third and last function, Di , models the inuence


of the obstacle to the environment by taking into ac- Figure 1: Attractor and Repeller interaction.
count the distance of the obstacle from the agent and A second more complex example consists of the agent
is modeled as facing two dierent obstacles located side by side. If
Di = e; d0i 
(7) the obstacles are too far apart, the agent should be
able to pass between them, otherwise it will have to
where ri is the relative distance between them, and d0 go around them. This decision is taken automatically,
controls the strength of this inuence as the distance as it can be seen in Fig. . Fig. 2(a) depicts the case
changes. where two obstacles are too close, Fig. 2(b) depicts the
The resulting inuence on the agent from all obstacles case where the distance between the obstacles is exactly
i = 1 : : :  n, is the sum of the respective repellers
fobs =
Xn f obsi : (8)
equal to the size of the agent, a critical condition, and
Fig. 2(c) depicts the case when the obstacles are far
apart to allow the easy passage of the agent between
i=1 them. For this simple case (no target and two obstacles)
Therefore, the de
nition of the dynamical system we have plotted at the bottom of each
gure (9) as a
controlling the heading angle in (1) is obtained as: function of the angle between the agent orientation and
the y axis (assuming that the noise term n is zero).
_ = f (env) == jwtar jftar + jwobs jfobs + n: (9) These functions clearly show that the dynamical system
exhibits the correct behavior in terms of the value of the
The weights wtar and wobs are intended to eliminate _ . For example in Fig. 2(a) _ = 0 depicts an unstable
spurious attractors that can occur by the direct sum-
xed point which would result in the agent trying to
ming of the nonlinear functions modeling the vari- go through the obstacles. However, the insertion of a
ous obstacles and targets in the environement. These small amount of noise n will overcome this situation
weights are obtained through a \constrain competi- easily given the function diagram.
tion", the second dyanamical system mentioned previ-
ously and described in details in Section . They are the
essence of the \low-level" behavior modeling. Finally,
the noise term n is an extremely important factor. It
allows the system to escape from unstable
xed points
in the de
nition of (9) (e.g., the \center" of a repeller,
where _ = 0, but any slight diplacement would make it 1 1 1

escape from such a situation such as the situation of a

0.5 0.5 0.5

ball situated on the crest of a hill).

-3 -2 -1 1 2 3 -3 -2 -1 1 2 3 -3 -2 -1 1 2 3

-0.5 -0.5 -0.5

All the above functions are carefully designed so that -1 -1 -1

certain expected actions will appear in the

nal system.
rst consider a simple example (Fig. ), the result (a) (b) (c)
of a simple interaction between a target, an attractor,
and an obstacle, a repeller. Let's also take the simple Figure 2: Interaction of repellers of two obstacles.
case that the location of the obstacle is close to the
straight line between the agent and the target.
It is then clear that the agent will have to go around Constrain Competition
the obstacle in order to hit the target. In this case, Individually, the attractors and repellers de
ned in sec-
the modeling of the agent's heading direction _ based tion work well, but because of their non-linear char-
on (9) is shown in the lower right graph of Fig. . It acteristics their direct sum might not always yield the
is the composition of the target function (upper right expected results. For instace, in the example shown in
Configuration Vector Field Components
leads to a situation where the target and the obstacle
contributions in (9) are turned o. Obviously this case


should be avoided, because the agent would move in

−π −π/2 0 π/2 π

an unpredictable way. To avoid this situation, both i


should always be greater than zero.

−π −π/2 0 π/2 π

Summed Vector Field
The second case (wtar  wobs ) = (0 1) occurs when

the target's contribution is turned o (like in the case

of Fig. ). It is stable as long as obs,tar > tar .
−π −π/2 0 π/2 π

Figure 3: Attractor of the angle dynamical system. The third case (wtar  wobs ) = (1 0) happens when
obstacles are ignored. This may occur, for example,
when there are no obstacles near the target. This case
Figure , the sum of one attractor with the two repellers is stable when tar,obs > obs .
propose an impossible path inbetween the two obstacles The last case is when the values of both weights
{ they are too close from each other to allow the agent's are nonzero, (wtar  wobs ) = (Atar  Aobs ), also known
passage. as the \averaging" solution. The following two con-
To avoid this kind of problem, the composition of the ditions have to be satis
ed for this case to be stable
attractors and repeller functions is not obtained by a di- obs > tar,obs and tar > obs,tar . This is de
netely a
rect sum, but though an weighted average by weights desirable situation.
wi . These weights are the result of the second dynam- It is important to note that conditions two and three
ical system, which runs at a more re
ned time scale are not mutually exclusive, and they can happen simul-
wrt the dynamical system in (9). This second system taneously. In this case we have a situation of bistability,
is modeled as X
w_ i = i wi (1 ; wi2 ) ; j i wj2 wi  (10)
where the stable condition that will prevail depends on
the initial conditions. In this case there can be constant
j 6=i alternation between behaviors. A possible solution to
avoid this problem is to give a \hysteresis" to the chang-
where in the simple case where only obstacles and ing of the weights.
targets are modeled, the state space (wi ) consists of Based on the above, the design of i and ij should
(wtar  wobs ), as used in (9). create the dierent stable points according to the en-
This system is completely de
ned by the parameter vironment parameters. This process is described with
functions i , termed competitive advantage, the param- details in (Large, Christensen, & Bajcsy" 1999), and
eter functions ji , termed competitive interaction, and the functions for this two-dimensional case are:
the initial value of its state space. At each time in- e;c2Ptar Pobs
stant these parameters will be computed according to obs tar = tar obs = 0:05
the geometry of the environment, and through (10) we
obtain the weights to be used in (9). At a given time
instance in the computation of (9), the computation of tar = atar
obs = tanh
Xn Di
the weights based on (10) is not done in one step but i=1
in multiple steps. This is in order to ensure that the where Ptar and Pobs are:
computed weights result in a stable
xed point of (10)
as we will explain below. This explains why the whole Ptar = sgn( dFdtar )ec1 jFtar j (11)
system runs in multiple time scales { (10) is evaluated
at a much faster rate compared to (9).
The correct design of the parameter functions i and Pobs = Wobs sgn( dFdobs )ec1 jFobs j (12)
ji will provide the desired low level behaviors. There- and also atar is such that whenever there is competition
fore it is important to understand the stability of this among targets and obstacles, targets will loose, but it
system (for more details see (Perko 1991)), and incorpo- will always be active if there is only a \background"
rate the geometry of the environment in the \low-level" noise. This is set here to be 0:4(1 ; obs ). Di (7) is
behaviors. Table 1 shows the stability analysis for (10): the function used in the distance contribution of each
wtar wobs Stability obstacle repeller, and their sum gives a good estimative
of the concentration of obstacles around and near the
0 0 Unstable tar  obs > 0 agent.
0 1 Stable obs,tar > tar
1 0 Stable tar,obs > obs Modeling the Agent's Velocity
Atar Aobs Stable obs > tar,obs Many dierent approaches can be used for modeling
and tar > obs,tar the forward velocity. A possible approach is to assign a
constant value to the forward velocity. This approach
Table 1: Stability Analysis. has drawbacks in a real time changing environment: if
There are four distinct cases each one related to a an obstacle is suddenly in front of the agent, there might
dierent behavior. The
rst case, (wtar  wobs ) = (0 0) not be enough time for the agent to change direction
and will result in a collision. A better approach is to There is a whole set of parameters to control the ex-
have the agent move faster when there are no objects pected low-level behavior of the overall system. Unfor-
around and slower in a crowded area. The agent should tunately this set is not intuitive for an animator. We
also retreat when it is too close to an obstacle. An are currently working towards making the whole pro-
equation for the forward velocity that satis
es the above cess of modeling a behavior both more high-level and
design criteria is the following \user-friendly" as well as exible enough for dierent
applications. New functions are being analyzed in or-
v = rmint2;c d1  (13) der to achieve a signi
cantly larger and more complex
set of behaviors.
where rmin is the distance to the closest obstacle, d1 Acknowledgments
the safety distance and t2c is the time to contact. This
method basically applies a constant time to contact ap- The
rst author was supported by a Ph.D fellowship
proach. If the closest obstacle is far then the forward from CNPq, Brazil. The second author was partially
velocity is large. Also if the closest obstacle is at a dis- supported by an ONR YIP and an NSF Career Award.
tance smaller than d1 , then the resulting forward veloc-
ity will be negative, meaning that the agent will retreat. References
Note that only obstacles in front of the agent should be Bates, J. Loyall, A. and Reilly, W. 1992. An architecture
considered for this calculation. We have used the above for action, emotion, and socialbehavior. In Proceedings of
method in all our examples. the Fourth Europeans Workshop on Modeling Autonomous
Agents in a multi Agents World.
Experimental Results Funge, J. Tu, X. and Terzopoulos, D. 1999. Cognitive
The system was implemented in C, using lua modeling: Knowledge, reasoning and planning for intelli-
(R. Ierusalimschy & Celes 1996) as an extensible em- gent characters. Proc. of SIGGRAPH 99 29{38.
bedded language to describe both the scene and the Goldenstein, S. Large, E. and Metaxas, D. 1999. Non-
target(s)/agent(s) movement. linear dynamical system apprach to behavior modeling.
The constant a in (2) was set to 1 and the The Visual Computer 15:349{369.
safety margin  in (6) was set to 0.8. The Eu- Grzeszczuk, R., and Terzopoulos, D. 1995. Automated
ler integration time step was 0:25 and all the sim- learning of Muscle-Actuated locomotion through control
ulations run in faster than real time. All ex- abstraction. In Proc. of SIGGRAPH.
periments described below can be found on-line at Large, E. Christensen, H. and Bajcsy", R. 1999. Scal- ing the dynamic approach to path planning and control:
In the
rst experiment we used a single static target Competition among behavioral constraints. Int. Journal
of Robotics Research 18(1).
and a series of static obstacles between it's location and Lethebridge, T., and C, C. W. 1989. A simple heuristically-
the target's initial position. Note that in this case d0 based method for expressive stimulus-response animation.
was 3:0. Computers and Graphics 13(3).
In the second experiment the scene is composed of Noser, H., and Thalmann, D. 1993. L-system-based be-
one static target and multiple moving obstacles. The havioral animation. In Proc. Pacic Graphics.
agent avoids colision by changes of direction and some- Noser, H. Renault, O. Thalmann, D. and Thalmann.,
times by a velocity reduction or even a complete stop. N. 1995. Navigation for digital actors based on synthetic
In this simulation d0 was set to 2:0. vision, memory and learning. Computer and Graphics.
In the third experiment there is a group of static Perko, L. 1991. Dierential Equations and Dynamical Sys-
obstacles and a moving target. The agent successfully tems. Springer Verlag.
reaches the target and avoids the moving obstacles. In R. Ierusalimschy, L. F., and Celes, W. 1996. Lua - an
this case d0 was set to 0:8 and the
nal velocity was the extensible extension language. Software: Practice & Expe-
result of the method scaled by 0:8. rience 26(6).
In the last experiment we illustrate the exibility of Reynolds, C. 1987. Flocks, herds, and schools: A dis-
our method by showing multiple moving and static tar- tributed behavioral model. In Proc. SIGGRAPH '87.
gets together with moving and static obstacles . The Reynolds, C. 1993. An evolved, vision-based behavioral
constant d0 was set to 1:0. model of coordinated group motion. In Proc. 2nd Int. Conf.
In the videos all experiments appear rendered based on Simulation of Adaptive Behavior.
on the use of the rendering package Pov-Ray. Schoner, G. Dose, M. and Engels, C. 1996. Dynamics of
Conclusions behaviour: theory and applications for autonomous robot
architectures. Robotics and Autonomous Systems 16(2{4).
We have presented a technique to model autonomous Steinhage, A., and Schoner, G. 1997. The dynamic ap-
agents navigation for game environments. Using a dy- proach to autonomous robot navigation. In Proc. IEEE
namical system approach we control the agent's head- Int. Symposium on Industrial Electronics.
ing direction and its velocity. We have demonstrated Tu, X., and Terzopoulos, D. 1994. Articial shes: Physics,
natural low-level agent behavior in envirnments with locomotion, perception, behavior. In Proc. of SIGGRAPH.
multiple targets and stationary/moving obstacles.