WORKSHOP ON

MULTIROBOTIC SYSTEMS FOR SOCIETAL

APPLICATIONS

Editors
K. Madhava Krishna
Assistant Professor
Robotics Research Lab
IIIT Hyderabad, India.

Bipin Indurkhya
Professor
IIIT Hyderabad, India.
 MAIN MENU

¾ FOREWORD

¾ ORGANIZING COMMITTEE

¾ PROGRAM COMMITTEE

¾ WORKSHOP PROGRAM

¾ TABLE OF CONTENTS

I
 FOREWORD

Multirobotic applications have become increasingly popular due to their obvious merits
of:
• Sharing computational burden.
• Increased fault tolerance.
• Study of emergent behaviors.
• Augmented capabilities.

The workshop has focused on Multirobotic systems and applications that have
potential utilities for society. The workshop has attracted papers which describe new
ideas, formulations & proof of concept demonstrations. Particularly the area of multi-
robot localization including SLAM has attracted lot of attention and nearly 50% of the
workshop papers have been in this area. A paper on localizing multiple spills in a factory
like setting using biologically inspired behaviors imitated on a swarm of robots has been
particularly motivating.

Other papers have targeted the use of swarm for communication of emergencies in
hazardous situations, automated coverage, and learning by imitation & new theoretical
formulation for multi-agent state representation that actuates coordination in a team only
when necessary.

We would like to express our most sincere appreciation and thanks to all Organizing
Committee members, the Program Committee members, local arrangements chair Dr.
Rajeev Sangal & Gen. Bagga, our students DVK Viswanath and Amit Kumar Pandey for
working hard to make this workshop successful.

K. Madhava Krishna
Bipin Indurkhya

Organizing Committee

II
ORGANIZING COMMITTEE

K Madhava Krishna
Assistant Professor
Robotics Research Lab
IIIT Hyderabad, India.
Email: mkrishna@iiit.ac.in

Bipin Indurkhya
Professor
IIIT Hyderabad, India.
Email: bipin@iiit.ac.in

III
 PROGRAM COMMITTEE

K Madhava Krishna
Asst Professor
IIIT Hyderabad
Gachibowli
Hyderabad - 500 032
AP, India
Phone: 91-40-2300 1967 (Extn: 269)
Email: mkrishna@iiit.ac.in
URL: http://mail.iiit.ac.in/~mkrishna

Bipin Indurkhya
Professor
IIIT Hyderabad
Gachibowli
Hyderabad - 500 032
AP, India
Phone: 91-40-2300 1967 (Extn: 182)e
Email: bipin@iiit.ac.in
URL: http://iiit.net/faculty/bipin.php

Rachid Alami
Directeur' de'Recherche
LAAS-CNRS
7 Avenue du Colonel Roche
Tolouse
Cedex – 31077
France
Phone: 33-561-33 63 46
URL: http://www.laas.fr/~rachid/

Raja Chatila
Directeur' de'Recherche
LAAS-CNRS
7 Avenue du Colonel Roche
Tolouse
Cedex – 31077
France
Phone: 33-561-33 63 44
URL: http://www.laas.fr/~raja/

IV
 Thierry Simeon
Directeur' de'Recherche
LAAS-CNRS
7 Avenue du Colonel Roche
Tolouse
Cedex – 31077
France
Phone: 33-561-33 63 49
URL: http://www.laas.fr/~nic/

Elena Messina
Intelligent Systems Division
National Institute of Standards & Technology
100 Bureau Drive, Mail Stop 8230
Gaithersburg, MD 20899-8230
USA
Email: elena.messina@nist.gov
Telephone Number: (301) 975-3510
FAX Number: (301) 990-9688
Secretary: Karen Hollingsworth

Dr. Raj Madhavan

Research Staff Member
Computational Sciences Division
Oak Ridge National Laboratory
Oak Ridge, TN 37831.
Guest Researcher at:
Intelligent Systems Division
National Institute of Standards & Technology
100 Bureau Drive, Mail Stop 8230
Gaithersburg, MD 20899-8230
E-mail: raj.madhavan@nist.gov
Telephone Number: (301) 975-2865
FAX Number: (301) 990-9688
Secretary: Karen Hollingsworth

V
 WORKSHOP PROGRAM

9:00am Inauguration of Workshop

9:10am Paper Presentation: Exploiting Factored Representations for
Decentralized Execution in Multi-agent Teams
Maayan Roth, Reid Simmons, Manuela Veloso.
9:40am Paper Presentation: GUARDIANS: a Swarm of Autonomous Robots for
Emergencies
Jacques Penders, et. al
10:10am Paper Presentation: Learning by Imitation: A Scheme for Offline
Learning of Robot Control through Demonstration by Other Robot
Umar Suleman, Rauf Baig, Mian M. Awais
10:40am Paper Presentation: Controlled Deployment of Sensor Networks for
Coverage Optimization
Sameera Poduri, Gaurav S. Sukhatme
11:10am Snacks & Tea
11:30am Paper Presentation: Glowworms-inspired multi-robot system for multiple
source localization tasks
P. Amruth, K.N. Krishnanand, D. Ghose
12:00pm Paper Presentation: Efficient Communication Strategy for Cooperative
Multirobot Localization
Valguima Odakura, Anna Helena Reali Costa
12:30pm Paper Presentation: Active Localization of Multiple Robots by Selection
of Best Frontiers
Rakesh Goyal, K Madhava Krishna, Shivudu Bhuvanagiri
1:00pm Paper Presentation: Multi-Robot Marginal-SLAM
Ruben Martinez-Cantin, Jos´e A. Castellanos & Nando de Freitas

1:30pm Lunch
2:30pm Panel Discussion: Have Multi-Robotic systems reached the society:
What needs to be done.
4:00pm Tea Break

4:30pm Visit to IIIT Robotics Research Lab.

VI
 TABLE OF CONTENTS

1. Exploiting Factored Representations for Decentralized Execution

in Multi-agent Teams
Maayan Roth, Reid Simmons & Manuela Veloso………………………1
2. GUARDIANS: a Swarm of Autonomous Robots for Emergencies
Jacques Penders, et. al ………………………………8
3. Learning by Imitation: A Scheme for Offline Learning of Robot
Control through Demonstration by Other Robot
Umar Suleman, Rauf Baig & Mian M. Awais…………………………..17
4. Controlled Deployment of Sensor Networks for Coverage
Optimization
Sameera Poduri & Gaurav S. Sukhatme ………………………………24
5. Glowworms-inspired Multi-Robot System for Multiple Source
Localization Tasks
P. Amruth, K.N. Krishnanan & D. Ghose …………………………….32
6. Efficient Communication Strategy for Cooperative Multirobot
Localization
Valguima Odakura & Anna Helena Reali Costa ………………………38
7. Active Localization of Multiple Robots by Selection of Best
Frontiers
Rakesh Goyal, K Madhava Krishna & Shivudu Bhuvanagiri…………..45
8. Multi-Robot Marginal-SLAM
Ruben Martinez-Cantin, Jos´e A. Castellanos & Nando de Freitas……52

VII
 Exploiting Factored Representations for Decentralized
Execution in Multi-agent Teams
∗
Maayan Roth Reid Simmons
Robotics Institute Robotics Institute
Carnegie Mellon University Carnegie Mellon University
5000 Forbes Avenue 5000 Forbes Avenue
Pittsburgh, PA, USA Pittsburgh, PA, USA
mroth@andrew.cmu.edu reids@cs.cmu.edu
ABSTRACT
In many cooperative multi-agent domains, there exist some
states in which the agents can act independently and oth-
ers in which they need to coordinate with their teammates.
In this paper, we explore how factored representations of
state can be used to generate factored policies that can,
with minimal communication, be executed distributedly by
a multi-agent team. The factored policies indicate those
portions of the state where no coordination is necessary, au-
tomatically alert the agents when they reach a state in which
they do need to coordinate, and determine what the agents
should communicate in order to achieve this coordination.
We evaluate the success of our approach experimentally by
comparing the amount of communication needed by a team
executing a factored policy to a team that needs to commu-
nicate in every timestep.
Categories and Subject Descriptors
I.2.11 [Artificial Intelligence]: Distributed Artificial In-
telligence—Multiagent systems
General Terms
Algorithms, Design, Performance
Keywords
decentralized execution, multi-agent MDP, communication
1. INTRODUCTION
The problem of coordinating teams of cooperative agents
operating under uncertainty is a difficult one that has re-
ceived a great deal of recent attention. This problem is a
challenging one because, while each agent observes only its
own local point of view, the agents must reason about both
the states and actions of their teammates in order to act in
∗The first author is a student.
1
Manuela Veloso
Robotics Institute
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA, USA
veloso@cs.cmu.edu
coordination. Several extensions of the Markov decision pro-
cess have been proposed to enable decision-theoretic model-
ing of multi-agent teams. Decentralized MDPs (Dec-MDP)
can be used to model teams that, like the ones addressed
in this paper, operate under collective observability [1]. In
a collectively observable domain, while each agent is unable
to independently identify the world state, the union of all
teammate observations at a given timestep renders the state
fully observable.
Although Dec-MDPs provide a comprehensive semantics for
modeling multi-agent teams in collectively observable do-
mains, their use in planning for such teams is limited by the
intractability of generating their policies. Even for teams of
as few as two agents, the problem of generating an optimal
policy for the Dec-MDP representing that team is NEXP-
complete [1]. One approach to overcoming the high complex-
ity of planning for Dec-MDPs has been the identification of
classes of domains that are easier to solve. One such class,
transition-independent Dec-MDPs are domains in which the
global state is partitionable into local states for each agent.
This local state depends only on the individual actions of a
single agent, rather than on the joint actions of the team.
However, because the team shares a joint reward that de-
pends on the joint state and action, a transition-independent
Dec-MDP cannot be solved as a set of independent MDPs.
Policy-generation for a transition-independent Dec-MDP is
NP-complete [6], a significant reduction in complexity from
the problem of solving a general Dec-MDP. Unfortunately,
not all multi-agent domains are transition-independent.
Factoring is another approach that has been applied to the
problem of planning for multi-agent teams. Factoring has
been applied successfully to the problem of speeding up
policy computation for large MDPs (e.g. [2, 7]). Unlike
standard MDP representations which enumerate all possi-
ble states, factored MDPs take advantage of conditional in-
dependences among state variables to allow more compact
problem representations, and in some cases, more compact
policies. The presence of an additional type of indepen-
dence, called context-specific independence, can lead to even
greater savings in the size of problem representation and
the efficiency of planning [3]. Work has been done in ap-
plying factoring to decentralized planning for multi-agent
MDPs [8]. In that approach, it is assumed that agents have
full observability of all relevant state variables and the chal-
lenge is to enable those agents to plan independently without
reasoning over the full set of joint actions.
When agents must coordinate without communication, gen-
erating a policy for the team is NEXP-complete. However,
if it is assumed that agents will be allowed to communi-
cate with their teammates at every timestep during exe-
cution, then a centralized policy, which we refer to as the
free-communication policy, can be generated in polynomial
time, as if the entire team were being modeled by a single-
agent MDP [12]. In general, though, communication is not
free, and it is necessary to minimize the use of communica-
tion resources during execution. Like Roth et al. [13], we
assume that each agent in our decentralized team can inde-
pendently compute the free-communication policy, and does
so at plan-time. In this paper, we explore the challenge of
decentralized execution of this centralized policy while min-
imizing the use of communication resources.
In our work, we employ techniques for solving factored MDPs
to generate the centralized, free-communication plan for our
team. We believe that there is a natural synergy in applying
factored representations to cooperative multi-agent teams.
When represented for centralized planning, even assuming
the presence of free communication, multi-agent teams can
quickly form large domains with many relevant state vari-
ables and joint actions. Factored representations and solu-
tion techniques can speed up policy-generation in large do-
mains, but only if those domains contain significant amounts
of conditional and context-specific independence. It is our
claim that many multi-agent domains exhibit a large amount
of context-specific independence, allowing them to be repre-
sented compactly and solved efficiently using factored repre-
sentations. Although there will be times when agents need
to coordinate their actions or gain knowledge of their team-
mates’ local state variables, multi-agent domains also con-
tain situations in which agents can act independently for
long periods of time. The challenge is to identify the in-
dependences present in any given domain, and thus enable
agents to coordinate only when necessary. Through address-
ing this challenge, factoring can enable the efficient solution
of and execution in some Dec-MDP domains.
In the remainder of this paper, we show how factored policies
can enable a team of cooperative agents to execute indepen-
dently, with each agent choosing when and what to commu-
nicate if it is necessary for team coordination. We introduce
an algorithm for transforming a centralized factored policy
into a distributed policy, and evaluate the success of our
approach experimentally.
2. FACTORED DEC-MDP MODEL
The Dec-MDP model is a Markov decision process extended
to handle joint actions and observations by a cooperative
multi-agent team [1]. A Dec-MDP is composed of the tu-
ple hα, S, A, P, Ri where: α is the number of agents on the
team, S is a finite set of world states, A is the set of possible
joint actions, with P defining a transition function that spec-
ifies the likelihood of the team transitioning from one state
to the next given a particular joint action, and R specifies
the joint reward function of the team, given the state and a
joint action.
2
The state, S, is comprised of n state features or variables
X = hX1 . . . Xn i, where each state si is an assignment of
values to each feature in X . The features may be binary
or multi-valued, or both. While in the most general case,
a Dec-MDP model also contains a set of joint observations,
Ω, we limit ourselves to domains in which the state features
are can be partitioned into α sets of local state features, one
for each agent, with each agent observing its local features
perfectly, and a set of joint features that are fully observable
by all of the agents.
Each agent i has a set Ai of possible individual actions. Ev-
ery joint action in A is comprised of one individual action
for each agent on the team. Because our model is factorable,
the relationship between state features and joint actions can
be represented as a Dynamic Decision Network [4]. For each
joint action, there is a Bayes network (such as the example
network shown in Figure 1) that specifies the relationship
between state variables at time t and time t + 1. State fea-
tures at time t+1 are dependent on the values of their parent
features in time t and are conditionally independent of all
other features. Associated with each variable is a conditional
probability table describing the probability of that variable
given the different possible values of its parents. In many
cases, a variable will depend on certain parents in some con-
texts but not others. When this is the case, the conditional
probability table can be represented more compactly as a
tree [3]. For example, in the conditional probability tree for
variable X2 at time t + 1, shown in Figure 1, X2 depends on
the value of X3 at time t when X1 has the value 1, but is
independent of X3 otherwise. This type of independence is
referred to as context-specific independence [3].
X1 X1
X1
0 1
p1 X3
0 2
X2 X2 1
X2 p2 X2
0 1 0 1
p3 p4 p5 p6
X3 X3
Figure 1: Example action network for a joint action
ak ∈ A, with tree-structured CPT for X2 .
Our use of a factored state representation in modeling our
multi-agent teams enables us to generate tree-structured
policies like the ones described in Boutilier et al. [2]. These
factored policies take the form of a decision tree, where the
internal nodes are state variables and the leaves are joint
actions. Structured Policy Iteration (SPI) is an algorithm
that can be used to compute decision tree-structured poli-
cies for factored MDPs. We refer you to [2] for the details
of the algorithm, but point out that it requires the MDP to
be represented as a Dynamic Decision Network, with con-
ditional probability tables for the variables represented as
trees, and that the reward function, also represented as a
tree, depend only on state features and not on the selected
action. This constraint is easily met by adding additional
state variables to the domain representation. We use SPI to
generate a factored free-communication policy for our team.
Figure 2 shows an example of one such factored policy. The
actions at the leaves are joint actions, and V1 and V2 are
state variables.
V1
ax V2
ay cz
Figure 2: A factored policy over joint actions.
3. DECENTRALIZED EXECUTION OF AN
INDIVIDUAL FACTORED POLICY
In order to execute distributedly, each agent on our team
must have a policy that maps assignments of state features
to individual actions. In Section 4 we detail an algorithm for
converting a factored joint policy generated by SPI into in-
dividual factored policies. For now, we note that an agent’s
individual policy may depend on state features that it can-
not observe directly. For example, in the policy for agent
i shown in Figure 3, the feature labeled X j a local feature
of agent j. Leaves in the policy that depend only on agent
i’s features indicate portions of the state space where that
agent has context-specific independence from its teammates.
Communication is needed to facilitate the execution of those
portions of the policy without this context-specific indepen-
dence.
i
X 3. if teammate features is empty
a X
j for the values of those variables.
a b c
Figure 3: A factored policy over individual actions.
Previous approaches to communication for Dec-MDPs and
Dec-POMDPs has utilized either the tell paradigm of com-
munication, in which each agent reasons about its local
knowledge and decides whether to send information to its
teammates (e.g. [13]), or the synchronize paradigm, in
which if at least one agent initiates communication, all the
teammates broadcast their full observation histories (e.g. [11,
5]). In this paper, we consider a different option, query
communication. Instead of trying to predict if its informa-
tion will be useful to its teammates, each agent asks its
teammates for information when needed. In order to query,
agents must reason both about when communication is nec-
essary, as well as what information is needed. Unlike other
approaches which use broadcast communication, query com-
munication is peer-to-peer, with each agent asking for infor-
mation only from the teammate who possess it.
3
Table 1 shows how communication enables the distributed
execution of a factored policy. Each agent observes its lo-
cal state features, as well as any global state variables that
are observable to the team, and traverses the policy tree ac-
cording to the values of those features until it either reaches
a leaf, whereupon it executes the individual action at that
leaf, or until it reaches a decision point in the policy that
depends on a feature to which it does not have local ac-
cess. When this happens, the agent knows it must ask its
teammate to communicate the feature value. The amount
of communication needed during execution measures of the
degree of context-specific independence among agents in a
domain. Agents executing in domains where they are often
independent of their teammates will need to communicate
less than agents in domains where they must coordinate fre-
quently.
Table 1: Recursively execute a factored policy, com-
municating when necessary.
function Execute(policy, features, teammate features)
returns an action
input: policy, a tree-structured policy for agent i
features, the current values of i’s state features
teammate features, a set of teammate features
for which values are known (from
previous communication)
1. if policy is a leaf return policy.action
2. if policy.variable is in features
a. x ← features[policy.variable]
b. return Execute(policy.childx , features,
teammate features)
a. Find all the variables in policy.
b. Build teammate variables by asking teammates
4. x ← teammate features[policy.variable]
5. return Execute(policy.childx , features,
teammate features)
To avoid the possibility that an agent will have to communi-
cate more than once per timestep, when an agent reaches a
decision point where it discovers that it needs a piece of in-
formation local to one of its teammates, the agent attempts
to predict what other information it may need to know. The
agent searches for all of the variables contained in the sub-
tree of its policy rooted at this decision point and asks its
teammates for all of those feature values. This means that,
in some cases, the agent will be requesting information that
it later discovers is irrelevant. However, we believe that in
most domains where it is necessary to conserve communica-
tion resources, it is desirable to minimize the total number
of messages sent, not the size of those messages. Only a
minor change is required in the execution algorithm to ac-
commodate domains in which it is desirable to minimize the
total number of features communicated.
4. GENERATING INDIVIDUAL FACTORED
POLICIES FOR MULTI-AGENT TEAMS
Structured Policy Iteration relies on several tree operations
defined by Boutilier et al., namely the ability to Merge mul-
tiple policy trees such that the resulting tree contains the
maximum policy attainable among the trees, and to Sim-
plify trees to reduce redundant branches and subtrees. In
order to convert a factored policy for a centralized team into
factored policies for individual agents, we extend the policy
tree representation to allow for multiple actions at any given
leaf, indicating the presence of a tie, and introduce two addi-
tional tree operations, Intersect, which simplifies a tree by
combining branches whose leaves have non-empty action-set
intersections, and Independent, which computes, for a sin-
gle leaf, the set of individual actions that can be performed
without considering the action choices of a teammate.
The ability to detect and propagate ties between joint ac-
tions in a policy improves the discovery of context-specific
independences among agents. If there is a portion of the
state where an agent i can act independently, without need-
ing to coordinate with its teammates’ actions, the values
of the joint actions composed of i’s optimal action and all
possible actions that i’s teammates could choose will be the
same. To this end, we modify Merge, which in its original
form picks one joint action at random when it encounters a
tie, to instead build trees where leaves may contain a set of
optimal joint actions. There is now an additional simplifi-
cation that can be performed on a policy tree, Intersect,
detailed in Table 2 and in the example in Figure 4. At each
non-leaf node in the policy tree, Intersect is recursively
called on the node’s children. After this, if every child of
the node is a leaf, and the action sets of the node’s chil-
dren have a non-empty intersection, the node is redundant
and can be replaced with a new leaf containing the intersec-
tion. If all children but one are leaves, a node may still be
redundant. This redundancy is detected by examining the
subtree that is not a leaf. If all of the actions at the leaves
of that subtree are present in the intersection of the node’s
leaf-children’s action sets, the node may be replaced by its
non-leaf child.
Table 2: Simplify a policy tree through intersection
of subtrees that contain the same actions.
function Intersect(policy) returns a policy
input: policy, a tree-structured policy
1. if policy is a leaf return policy
2. Recursively call Intersect on each child in policy.
3. if everyT child of policy is a leaf
return i childi .actions for each childi of policy
4. if exactly one child, childNL , of policy is a non-leaf
a. get AL , the intersection of the leaf-children’s
action sets
b. get ANL , the set of all actions at the leaves of
childNL ’s subtree
c. if ANL is a subset of AL return childNL
5. return policy
An individual action is considered Independent for an agent
i in a particular leaf of a policy tree if the action is optimal
when paired with any of the other individual actions that
its teammates may choose at that leaf. Figure 5 shows an
4
example of a leaf in which an Independent action can be
discovered for agent 1, and one in which no Independent
action exists.
ax ax
ay {a} by {}
bx cz
(a) (b)
Figure 5: Independent action examples. (a) In this
leaf, action a is can be performed independently by
agent 1, since it forms an optimal joint action with
x and y, all the possible actions that agent 2 might
perform at this leaf. (b) There is no Independent
action for agent 1 in this leaf.
The full process of transforming a factored joint policy into
factored individual policies for each agent can be found in
Table 3. First, the joint policy is re-written for each agent
so as to move the state features visible to that agent to
the upper-most nodes of the tree. This is done to increase
the likelihood that, during execution, an agent will be able
to traverse portions of its policy without requiring commu-
nication. This rewritten tree is processed to discover the
Independent actions for the agent at each leaf. Leaves con-
taining independent actions are labeled with those actions,
while those where the set of independent actions is empty
retain their joint action sets. If any ties remain among the
joint actions, they are now broken according to a predeter-
mined canonical action ordering. This is necessary to avoid
equilibrium-selection mis-coordinations like the one shown
in Figure 6. At this point, the joint actions in the policy
tree are converted to individual actions. Finally, Intersect
is performed and the tree is simplified one final time.
Table 3: Transform a factored joint policy into fac-
tored individual policies for each agent..
function generateIndividualPolicies
for each agent:
1. Make a joint policy for that agent, with its state
variables at the root.
2. For each leaf in the policy, find Independent
actions
3. Break ties among remaining joint actions
using canonical action ordering.
4. Convert joint actions to individual actions.
5. Intersect and Simplify.
5. EXPERIMENTAL RESULTS
We demonstrate the effectiveness of our approach on an ex-
ample problem, a Meeting-Under-Uncertainty domain. In
this problem, two agents must meet at a predetermined lo-
cation in a grid world, and when both are at the goal lo-
cation, simultaneously send up a Signal. The other indi-
vidual actions available to the agents are North, South,
East, West, and Stop, with movement actions succeed-
ing with 0.9 probability. The agents receive a reward of
+20 for signaling together, and receive penalties for either
 V1 V1 V1

bx V2 bx V2 bx V3

ax ax ax ax ay
V3 V4 ay V3 ay ay
cz cz

ax ay ax ax ax ay
ay ay
cz

(a) (b) (c)

Figure 4: Intersect example. Agent 1 can take individual actions {a, b, c} and agent 2 can take actions {x, y,
z}. (a) The node labeled V4 is simplified by Intersect because all of its children are leaves. The intersection
of the leaves’ action sets is {ax, ay}. (b) V2 is simplified because it has only one child that is not a leaf (the
subtree rooted at V3). Its non-leaf children’s action sets have the intersection {ax, ay}, which includes all of
the actions at the leaves of the non-leaf subtree. (c) No further intersection is possible.

mis-coordinating their Signal actions or signaling from the 1

X
wrong location. The team incurs a cost of -1 for each timestep
that it takes them to reach the goal and Signal. Each EAST WEST

agent observes its own location (X 1 and Y 1 for agent 1, and 1

Y
X 2 and Y 2 for agent 2) but does not know the position of
2
its teammate. This is a domain that admits a high degree X

of factorability and which allows for considerable context-

specific independence among the two agents. While the SOUTH SOUTH

agents are moving toward the goal location, they can act in- STOP
2
Y STOP
dependently. They only need to coordinate once they have
reached the goal.

We applied our algorithm to grid worlds of various sizes. STOP STOP SIGNAL

Figure 8 shows a factored individual policy for agent 1 op-

erating in the 3-by-3 grid world shown in Figure 7, with Figure 8: A factored policy for Agent 1 in a 3-by-3
the goal at location (2,3). A visual inspection of the policy Meeting-Under-Uncertainty problem.
shows that, as expected, for most of the state space, agents
can act independently. We verified this by measuring the
amount of communication needed in an average run of the
problem.
sults for a 3-by-3 grid world are shown in Table 4.

Our factored algorithm achieves the same average reward

* specifically designed to minimize communication usage, is
Figure 7: Agents in a 3-by-3 grid world. The goal is
at (2,3).
We compared our execution method to a team operating
with free communication and to Dec-Comm, an execution-
time communication algorithm that chooses joint actions by
requiring agents to maintain estimates of the possible joint
beliefs of the team [13]. We ran 1000 trials of each method,
starting the agents in random locations. The complete re-
as a team communicating at every timestep, but requires
significantly fewer instances of communication. This con-
firms our intuition that the Meeting-Under-Uncertainty do-
main admits a great deal of context-specific independence
between agents. The Dec-Comm algorithm, an approach
able to communicate much less frequently than a team ex-
ecuting a factored policy, but does so by sacrificing some
amount of expected reward.
Figure 9 shows the amount of communication used by a team
executing a factored policy, as a percentage of the amount
of communication used by a team with full communication
for grid worlds of increasing size, from 3-by-3 (82 states)
up to 7-by-7 (2402 states). As the problem size increases,
in this domain, the number of states in which agents can
act without coordinating increases, leading to even greater
communication savings over a fully-communicating team.

5
 V1 V1
0 1 0 1

V2 ax a a
V2
0 1 0 1

V1 cx bx c b
0 1 az ay a a

V2 ax

0 1
V2 V2
cx bx 0 1 0
az ay

V1 V1 V1 V1 x

0 1 0 1 0 1 0 1

cx ax bx ax x x x x
az ay z y

(a) (b) (c)

Figure 6: An example of mis-coordination that can occur if ties are not broken using a canonical action
ordering. V1 is a state feature local to agent 1, and V2 is local to agent 2. (a) An individual policy is
generated for each agent, with its local variable at the root. (b) The policies are converted into policies over
individual actions, without an intermediate tie-breaking step. (c) The resulting simplified policies indicate
that agent 1 should always perform action a and agent 2 should always perform x, making the joint action
ax. This is an uncoordinated action whenever V1 = 0.

0.5
Table 4: Complete results for the 3-by-3 Meeting-
Under-Uncertainty domain. 0.45

Mean Mean Mean

0.4
Reward Messages Variables
Sent Sent
(σ2 ) (σ2 ) (σ2 ) 0.35

Free 17.484 7.032 14.064

0.3
Communication (1.069) (1.069) (1.069)
Factored 17.484 3.323 6.646
Execution (1.069) (1.050) (4.199) 0.25
3−by−3 4−by−4 5−by−5 6−by−6 7−by−7
16.378 1.153 2.306
Dec-Comm (1.234) (1.153) (0.498)
Figure 9: Communication usage as a function of
problem size.

One major challenge posed by multi-agent problems is the

exponential growth in the number of states and joint actions
as the number of agents increases. However, as the agents
may be independent of their teammates for large portions
of the state in some multi-agent domains, the single-agent
factored policies needed to control the agents may be very
compact. Table 5 shows that the number of leaves in each
agent’s factored individual policy grows linearly as the num-
ber of agents increases from two to five in a 3-by-3 Meeting-
Under-Uncertainty domain.
6. CONCLUSION
In this work, we show how factored policy representations
can be used for coordinated decentralized execution of a
multi-agent team, providing an efficient answer to the ques-
tions of when and what a team should communicate and al-
lowing agents to act independently of their teammates un-
til they reach a state in which potential mis-coordination
could take place. We provide a post-processing technique
for transforming a centralized factored policy into individ-
ual factored policies for each member of a team, and show
how this technique discovers context-specific independences
between agents. Our preliminary experimental results are
promising, showing that our work enables agents to act inde-
pendently and without communication under favorable state
conditions, and discovers automatically those conditions in
which agents must coordinate with their teammates.
Although the Structured Policy Iteration algorithm that we
discuss in this paper generates optimal policies for MDP
domains, it is important to note that our approach does
not require optimal joint policies as input, nor does it de-

6

Table 5: Policy-tree size grows linearly with the
number of agents in a 3-by-3 Meeting-Under-
Uncertainty domain, even as the number of states
and joint actions grow exponentially.
Policy
Joint Tree
States Actions Leaves
2 agents 82 36 9 [7] C. Guestrin and G. Gordon. Distributed planning in
3 agents 730 216 13
4 agents 6562 1296 17
5 agents 59050 7776 21
pend on the use of one particular MDP solution technique.
The algorithms that we present in this paper can be applied
to transform a joint policy of any quality into single-agent
policies with equal expected reward. All that is required is
that the joint policies be transformable into decision trees
over state variables. For example, the SPUDD algorithm
efficiently solves Markov decision processes in which rela-
tionships among state variables are represented as algebraic
decision diagrams (ADDs) [10], which can easily be trans-
formed into trees. Thus, our work can build on any advances
that arise in the area of generating tree-structured policies
for MDPs to solve increasingly large problems.
There are several directions in which our work can be ex-
tended. Currently, our execution algorithm does not take
into account the cost of communication. An analysis of
the value of information could be integrated into the rea-
soning to allow agents to decide if the cost of a poten-
tial mis-coordination is high enough to justify communi-
cation. Additionally, our work focuses on collectively ob-
servable multi-agent domains modeled by Dec-MDPs. It is
possible that work on factored POMDPs (e.g. [9] could be
applied in a similar fashion to the decentralized execution
of Dec-POMDPs, modeling teams with collective partial ob-
servability. There is also significant work to be done in char-
acterizing multi-agent domains that exhibit a high degree of
context-specific independence between teammates, as those
are the domains that would benefit most from the applica-
tion of our techniques.
[5] C. V. Goldman and S. Zilberstein. Optimizing
information exchange in cooperative multi-agent
systems. In International Joint Conferences on
Autonomous Agents and Multi-agent Systems, 2003.
[6] C. V. Goldman and S. Zilberstein. Decentralized
control of cooperative systems: Categorization and
complexity analysis. Journal of AI Research, 2004.
hierarchical factored MDPs. In Uncertainty in
Artificial Intelligence, 2002.
[8] C. Guestrin, S. Venkataraman, and D. Koller. Context
specific multiagent coordination and planning with
factored MDPs. In AAAI Spring Symposium, 2002.
[9] E. Hansen and Z. Feng. Dynamic programming for
POMDPs using a factored state representation. In
International Conference on AI Planning Systems,
2000.
[10] J. Hoey, R. St-Aubin, A. Hu, and C. Boutilier.
SPUDD: Stochastic planning using decision diagrams.
In Uncertainty in Artificial Intelligence, 1999.
[11] R. Nair, M. Roth, M. Yokoo, and M. Tambe.
Communication for improving policy computation in
distributed POMDPs. In International Joint
Conferences on Autonomous Agents and Multi-agent
Systems, 2004.
[12] D. V. Pynadath and M. Tambe. The communicative
Multiagent Team Decision Problem: Analyzing
teamwork theories and models. Journal of AI
Research, 2002.
[13] M. Roth, R. Simmons, and M. Veloso. Reasoning
about joint beliefs for execution-time communication
decisions. In International Joint Conferences on
Autonomous Agents and Multi-agent Systems, 2005.

7. REFERENCES
[1] D. S. Bernstein, R. Givan, N. Immerman, and
S. Zilberstein. The complexity of centralized control of
Markov decision processes. Mathematics of Operations
Research, 2002.

[2] C. Boutilier, R. Dearden, and M. Goldszmidt.

Stochastic dynamic programming with factored
representations. Artificial Intelligence, 2000.

[3] C. Boutilier, N. Friedman, M. Goldszmidt, and

D. Koller. Context-specific independence in Bayesian
networks. In Uncertainty in Artificial Intelligence,
1996.

[4] T. Dean and K. Kanazawa. A model for reasoning

about persistence and causation. Computational
Intelligence Journal, 1989.

7
 G UARDIANS: a Swarm of Autonomous Robots for Emergencies
Jacques Penders
Materials and Engineering Research Institute
Sheffield Hallam University, UK
J.Penders@shu.ac.uk
Ulf Witkowski
Heinz Nixdorf Institute
University of Paderborn, Germany
witkowski@hni.uni-paderborn.de
Jeremi Gancet
Space Application Services
Belgium
leif.steinicke@spaceapplications.com
Veysel Gazi
Dept. of Electrical and Electronics Engineering
TOBB University of Economics and Technology, Turkey
vgazi@etu.edu.tr
Abstract
The G UARDIANS1 are a swarm of autonomous
robots applied to navigate and search in particular
an industrial warehouse in smoke. A time consum-
ing and dangerous job; toxics may be released and
as senses may be impaired humans may get lost.
The robots warn for toxic chemicals, provide and
maintain mobile communication links, infer local-
ization information and assist in searching. They
enhance operational safety and speed and thus in-
directly save lives. This paper presents the state-
of-the-art and the aims of the project in the areas
of swarm robotics, chemical sensors, and commu-
nication strategies. Though still at the beginning
of the project, a real fire-fighting scenario is envi-
sioned, and previous results by the project partners
are expected to be merged and enhanced, in order
to bring together new potential impact in the use of
teams of robots in emergency setups.
1 Introduction
The G UARDIANS (Group of Unmanned Assistant Robots De-
ployed In Aggregative Navigation by Scent) are a swarm of
autonomous robots applied to navigate and search an urban
1
Group of Unmanned Assistant Robots Deployed In Aggregative
Navigation by Scent; EU-project contract IST-045269; G UARDIANS
is part of the EC Cognitive Systems initiative.
8
Enric Cervera
Robotic Intelligence Lab
Jaume-I University, Spain
enric.cervera@icc.uji.es
Lino Marques
Institute of Systems and Robotics
University of Coimbra, Portugal
lino@isr.uc.pt
Pierre Bureau
K-Team
Switzerland
pierre.bureau@k-team.com
Roberto Guzman
Robotnik Automation
Spain
rguzman@robotnik.es
ground. Several robot platforms are used, off-the-shelf mini-
robots as well as special purpose middle sized robots. The
robots are provided with an artificial nose.
The project’s central use-case scenario is an industrial
warehouse in smoke, as proposed by the South Yorkshire Fire
and Rescue Service. Searching the warehouse is time con-
suming and dangerous; toxics may be released and humans
senses can be severely impaired. Humans become disoriented
and may get lost. The robots navigate autonomously, individ-
ually but also as a coherent swarm. Apart from the swarming
technology, a novelty of the project is that the swarm is able
to adequately assist and safeguard a human squad leader. The
robots therefore:
1. provide and maintain mobile communication links,
2. infer localization information,
3. detect and warn for toxic chemicals and
4. assist in searching.
Thus, they enhance operational safety and speed up the
search job and thus indirectly may save lives.
The robots connect via a wireless network to the squad-
leader and the control station. The autonomous swarm oper-
ates in communicative and non-communicative mode. The
communication network is self-organizing, and automatic
service discovery is applied: the robots find peers to help
them. The aim is for flexible and seamless switching between
these modes to compensate for loss of communication sig-
nals. An artificial nose enables the robots to apply olfactory-
based navigation and toxic plume detection, as well as the
swarm as a collective .
The wireless network also enables the robots to support a
human squad-leader operating within close range, they main-
tain communication connectivity, provide localization infor-
mation and warn for and try to avoid chemicals. Toxic plume
detection by the robots allows safe trespassing for the human
squad-leader.
The rest of the paper is organized as follows. Section 2
describes the major use case of the project, and the over-
all components of the proposed solution. Sections 3, 4,
and 5 respectively present the state-of-the-art, and expertise
of the project partners in the key areas addressed by the
project, namely: robot swarms, ad-hoc mobile networking,
and olfactory-based mobile robots. The ultimate integration
of such components is provided by a specific human - robot
swarm interface, whose main functionalities are outlined in
Section 6. To conclude, Section 7 analyses the potential im-
pact and outcomes of the project, in the addressed key areas.

2 Fire ground search technology

The major use-case scenario and test case is proposed by the
Fire and Rescue Services (Fig. 1) and consists of searching
an industrial warehouse or basement. As part of the economic
developments industrial warehouses are dramatically increas-
ing in size. However, the standard procedures to search a
warehouse date back to the Second World War. In recent
years, five fire fighters died in the UK, as they got disori-
entated in a warehouse search [London Fire Brigades Union,
1991]. The firefighters have a maximum of 20 minutes to en-
ter the fire ground, search and exit the building safely. Any
review of the procedures and even small improvements are
desperately needed. The consortium believes that the appli-
cation of a swarm of ad-hoc networking robots will lead to
very significant improvements.
Before human beings are to enter a fire ground, the hu-
man commander has to decide whether the ground is (rel-
atively) safe and what equipment and gear to use. To en-
hance and speed-up decision making the Fire and Rescue
Services urgently require sensors or other devices to assess
the fire ground for the presence of flammables, explosives,
leaked chemicals, high radiation level, etc. The G UARDIANS
robots are to access and assess scenes for dangers to human
beings, they search for toxic chemicals under extreme condi-
tions: poor visibility, radiation etc. The robots are equipped
with a range of sensors for the registration of temperature, at-
mospheric pressure, and the presence of different chemicals.
The robot swarm can be applied with or without a hu-
man squad leader. Depending on the configuration the robot
swarm has the following roles:
Without a human squad leader:
• search and navigate through the warehouse
• maintain communication connections
• designate one or more robots as positioning beacons
• exchange position data with the base station
• detect and possibly locate toxic agents
Figure 1: South Yorkshire Fire and Rescue Service (project
partner). Up: incident support unit. Down: training setup.
• possibly detect casualties
With a human squad leader the additional roles are:
• navigate the squad leader through the warehouse
• exchange squad leader’s position data with the base sta-
tion
• warn for toxic agents
• call the squad leader’s attention to objects of interest
• maintain the communication link between the fire fighter
and the control base
The abundance of robots in the work space enables them to
switch between jobs without loss of overall functionality. For
instance, at first all robots might be involved in searching but
when the swarm is advancing several robots might stay be-
hind as positioning beacons and to maintain communication
connectivity. Thus the project delivers:
• a mobile adhoc communication network
• a mobile positioning system, carrying its own beacons
• a mobile toxic alarming system
• robot search assistants
3 Robot swarming
The G UARDIANS swarm will consist of many simple off-the-
shelf mini robots; they are 13cm in diameter, able to move on
floor surfaces with minor obstacles such as doorsteps. The

9
swarm will also comprise a few larger middle-size robots:
USAR (Urban Search and Rescue) Robots, which will be as-
sembled in the project (see Fig. 2). The project will equip
the mini robots with special task sensors (electronic nose).
The middle size robots in addition carry more navigation sen-
sors and a large array of gas and vapor sensors. The robots
are to communicate wirelessly with peers and a base station,
and may use Wireless LAN, Bluetooth and ZigBee and may
switch between these technologies.
Figure 2: Small, middle-size, and large robot platforms to be
used in G UARDIANS.
During the operation the robots have to keep track of their
positioning and maintain their communication link. However,
while advancing, communication failures might be caused
by loss of connectivity, or even the loss of a robot. When
the wireless signal is lost, the swarm and robots will au-
10
tonomously search to restore the wireless connection; data
transmission being deferred until contact is restored. More-
over, while the swarm is advancing, some robots may remain
behind as position beacons and/or as communication nodes.
The swarm will also apply a service discovery architecture
where robots assist each other in executing a task or limit
themselves to a particular job. The swarm can be guided from
the base-station and monitored by a central control unit (with
optional human supervision).
The autonomous swarm operates in communicative and
non-communicative mode. In non-communicative mode,
when a communication link is failing, the individual robots
make navigation decisions based only on sensor feedback.
Swarm behavior results from the robots reacting to each other
whilst applying artificial potential functions [Gazi, 2005a;
Gazi, 2005b].
Swarm research and swarm robotics is a relatively new
field that has gained momentum in recent years. Since the pi-
oneering work by [Reynolds, 1987] on simulation of a flock
of birds in flight (using a behavioral model based on few sim-
ple rules and only local interactions), the field has witnessed
many developments. Several different approaches have been
considered for swarm aggregation, navigation, coordination
and control. While initial research has focused on centralized
[Latombe, 1991; Liu et al., 1989; Barraquand et al., 1992]
and leader-based [Desai et al., 2001] approaches, most of
the recent research has been concentrated on decentralized
approaches, which offer reduced computational complexity
and robustness to failures. Such approaches include behav-
ioral [Balch and Arkin, 1998], artificial potential functions
[Reif and Wang, 1999; Egerstedt and Hu, 2001; Gazi and
Passino, 2003; Gazi and Passino, 2004b; Gazi and Passino,
2004a; Gazi, 2005a; Gazi, 2005b], virtual agents or virtual
structures [Bachmayer and Leonard, 2002; P. Ögren et al.,
2002], probabilistic [Stilwell et al., 2005], and others [Tan-
ner et al., 2003a; Tanner et al., 2003b]. The open-loop ap-
proaches use optimization criteria from game theory or for
navigation control [Wangermann and Stengel, 1999] or dis-
tribution or area coverage criteria [Cortes et al., 2004]. There
are also works dealing with improving systems performance
through adaptation and learning [Patnaik et al., 2005; Uchibe
et al., 1999; Asada and Hosoda, 1999]. Some of these works
use global information while others are based on local in-
teractions and rules. Moreover, besides bio-inspired mod-
els, control-theoretic approaches are currently researched as
well. Some surveys/collections on recent advances and the
state of the art in swarms can be found in [Dorigo and Sahin,
2004; Sahin and Spears, 2005; Kumar et al., 2005] and a web
database on swarm robotics related literature is currently be-
ing compiled at http://swarm-robotics.org/.
Swarms are being investigated in many research projects,
most of which with a focus on robot design an im-
plementation: the Swarm-bots project (http://swarm-
bots.org/)the I-Swarm: Intelligent Small World Autonomous
Robots for Micro-manipulation project; (http://www.i-
swarm.org/) the Open-Source Micro-Robotic project
(http://www.swarmrobot.org/); the SWARMS project (Scal-
able sWarms of Autonomous Robots and Mobile Sensors)
(http://www.swarms.org/) and the CentiBOTS project at SRI.
 Despite significant advances, the basic mechanisms of
swarm behavior have not been fully elucidated [Penders et
al., 2004]. Many of the above projects are hardware based
with the objective to develop hardware platforms for swarm
systems. However, a problem of equal importance is to de-
velop mechanisms for coordination and control of swarms.
The G UARDIANS project aims to contribute to better under-
standing of swarming behavior. Moreover, the objective is not
only to contribute to the swarm robotics literature but also to
apply the developments in a real-world application.
The G UARDIANS project will build upon the previous re-
search on swarm robotics and develop it in two new direc-
tions:
1. the swarm has to be able to navigate the area and build up
and maintain the communication channel with the out-
side world.
2. the swarm has to support a human squad leader in
searching a hazardous environment in which the human
senses partly fail;
3.1 Non Communicative swarming
The basic operations of the G UARDIANS-swarm are navigat-
ing and searching. The robots will be able to successfully
carry out these operations without direct communication be-
tween each other. Each robot agent will process informa-
tion locally, but due to the nature of algorithms that can be
developed, the robots will seem to perform their tasks col-
lectively. The searching operations are implemented as very
basic behaviors using decentralized swarming methods. The
G UARDIANS initial approach is artificial potential functions
based. Though various other methods for searching and ef-
fective coverage of the area including heuristic/ad-hoc, gradi-
ent based, or probabilistic approaches, will be developed and
implemented as well.
The control model The control model of the robots are the
mathematical equations governing the robots and the swarm.
We relate our model to those of [Reif and Wang, 1999; Ya-
maguchi, 1999] and the more recent models of [Gazi, 2005a]
−
→
and [Gazi, 2005b]. The model is given by equation (1); Fa
is the basis for the new velocity vector of the robot a and
depends basically on four terms. The first two terms, repre-
−→
sent the external influences. EA(G,a) is the attraction of the
−−→
goal G on robot a and ER(o,a) is the repulsion caused by the
unidentified obstacle o ∈ O on robot a. The second pair of
terms in (1) consists of the internal forces, which originate
amongst the robot in the swarm Sw. They are the attraction
−→ −→
IA(r,a) and repulsion IR(r,a) between the swarm members
r on a. The attraction points directly towards the source ob-
ject and the repulsion points into the opposite direction, away
from the source object.
Sw Sw
−
→ −→ X −−→ X−→ X −→
F a = EA(G,a) + ER(o,a) + IA(r,a) + IR(r,a)
o∈O r6=a r6=a
(1)
−→
The internal attraction IA(r,a) and internal repulsion
−→
IR(r,a) are sometimes called the artificial social potential
11
functions [Reif and Wang, 1999], as their combination in-
duces coherence in the swarm. The functions are chosen such
−
→
that on large distances the attraction IA dominates, on short
−→
distances repulsion IR dominates, and at a particular distance
attraction and repulsion balance; this is called the equilibrium
distance [Reif and Wang, 1999]. Note that in the case when
only internal attraction applies but no repulsion, the robots
will chase each other; if only repulsion applies the robots will
disperse. In the last two sums, only the neighbors of robot a
could be used (i.e. those robots which can be sensed without
any communication).
Special applications. Non-communicative swarming is
considered a very basic ability of the swarm, which the swarm
must be able apply when communication fails. Also if tasks
are achievable by non-communicative swarming, the com-
munication can be dedicated to other purposes. The control
model (1) is intended for general navigation, however it can
be adjusted to special purposes, such as recovery from a dis-
rupted communication network and olfactory based naviga-
tion.
In case all communication links are failing the robots have
to search individually for a communication signal. This be-
havior is achieved by manipulating the first two terms of
−→
the control model (1): EA(G,a) , the external attraction and
−−→
ER(o,a) the external repulsion.
The robots are individually able to apply olfactory-based
navigation. For the swarm of robots non-communicative
olfactory-based navigation is also obtained by manipulating
−→
the first two terms of the control model (1): EA(G,a) , the
−−→
external attraction and ER(o,a) the external repulsion.
3.2 Communicative swarming
Basically in what we are calling communication-based
swarming the robots use a communication channel to ex-
change symbols; as opposed to non-communicative swarm-
ing which is based on physical actions (usually movements)
which change the environment and reactions to these changes.
In the context of robot swarms this distinction is also referred
to as between explicit and implicit communication [Parker,
2000]. In communication based swarming several approaches
can be found in the literature, with an abundance of multi-
agent based approaches, refer to [Farinelli et al., 2004] for a
recent overview.
Whereas searching and navigation can be performed with-
out the individuals communicating, cooperation with com-
munication can dramatically improve the performance of the
system. Moreover, several other tasks essentially require
communication; examples are building and maintaining an
overview of the terrain, localization and position detection.
More importantly, the swarm can make use of communica-
tion channels to effectively build up a division of labour, with
some robots only acting as a positioning beacon and commu-
nication node, while others continue pursuing multiple tasks
(refer to service discovery below).
For the G UARDIANS project we distinguish two distinct
swarm configurations, a locally communicating swarm and a
coordinator based configuration. In the locally communicat-
ing swarm the robots are considered to be peers that exchange
data amongst themselves but each robot makes its own deci-
sions. In the coordinator based configuration the robots ex-
change data with the base station. The base station thus ac-
quires an overview from the data and draws conclusions at the
swarm level. The G UARDIANS project will have both config-
urations operational and develop seamless switching between
them.
3.3 Combining Communicative and
Non-communicative swarming
Basic swarming behavior such as aggregation and coordi-
nated motion, navigation, or even specialization through
learning [Li et al., 2004] can be achieved without com-
munication. However, as mentioned above communication
can drastically improve the performance of the system. In
G UARDIANS each agent will be capable of performing both
communicative and non-communicative swarming and based
on the availability of the communication channels will be op-
erating in either mode. This will not only help to enhance the
performance, but will also improve the robustness of the sys-
tem and potentially decrease robot casualties. While operat-
ing in non-communicative mode each robot will continue col-
lecting data about the environment; which, as soon as com-
munication is established it will share with the other swarm
members, the squad leader, and the base station and will re-
quest help or other tasks. A novel aspect for swarming re-
search (to the best of our knowledge not been done before)
is that in order to maintain communication connections a
swarming algorithm has to be built to make the robots keep
track of the strength of the communication signals, and when
necessary search for a signal or a stronger signal.
4 Ad-hoc mobile networking and service
discovery
The mobile robots will form a wireless ad hoc network
for communication. Various wireless communication stan-
dards have been established in recent years including Wire-
less LAN, Bluetooth and ZigBee. They differ in network
size, radio range, data rate and power consumption. Wireless
LAN is suited to high data rate and high range communica-
tion at the cost of high power consumption. Bluetooth has
a lower data rate and transmission range but in turn signifi-
cantly lower power consumption. ZigBee is highly scalable
with even lower power consumption but with a trade off for
lower data rates.
Mobile ad hoc networks are wireless networks consisting
of mobile nodes with a typical local (decentralized) organi-
zation and potentially high dynamics of the network struc-
ture, they have a dynamic or adjustable topology. In a swarm
and in contrast to a small group, communication by simply
broadcasting becomes infeasible in large-scale networks due
to scalability problems and the limited range of communica-
tion devices. The G UARDIANS project utilities the flexible
topology to deal with communication barriers that are very
likely present in the warehouse; to our knowledge the project
is the first to apply ad-hoc networking techniques for this pur-
pose.
12
One major challenge in wireless ad hoc networks, partic-
ularly in mobile ad-hoc networks, is the design of efficient
routing algorithms. The communication protocol should be
resilient to node failures, scalable, efficient - particularly re-
garding the transmission bandwidth, and secure. Two kinds
of communication take place: First short range communica-
tion within a group of robots and second middle range com-
munication between networks of robots, static activated sen-
sors, humans and the base station.
Generally, topology control can be regarded as the task
of constructing a (connected) subgraph with certain required
properties from a given graph representing the communica-
tion network. Topology control can be used to reduce the
energy consumption and/or interference in large networks by
adapting the transmission ranges of the network nodes or cre-
ating a connected dominating subset of nodes to which regu-
lar nodes connect. The risk is, however, the loss of connec-
tivity in the network through the removal of connection links.
In the scenario we consider here, the availability and robust-
ness of network connection is the most important property
and energy consumption and scalability are only secondary
concerns.
In communicative mode the robots in the swarm can co-
ordinate their actions, for instance by negotiating as a multi-
agent system. Important when applying the swarm in rescue
operations is that in this mode the swarm applies automatic
service discovery that is: the robots find peers to help them to
solve a local problem [Du et al., 2005].
The problem of efficient service discovery in mobile ad-
hoc networks is still unsolved. Known solutions for service
discovery (like Jini, JXTA, SLP, UPnP, Bluetooth SDP or
Salutation) were not designed with such kind of highly dy-
namic networks in mind.
Robots need the ability to mutually offer each other ser-
vices in an open, heterogeneous network and they must be
able to efficiently discover and use those services (Fig. 3).
Only this feature will allow them to cooperate flexibly and
dynamically to solve tasks. Thus, service discovery allows a
constant, task-oriented self-configuration of the overall sys-
tem to achieve required goals.
Figure 3: Basic idea of service discovery.
Besides the autonomous operation modes, the swarm
can also guide the human squad-leader meaning that a
semi-autonomous mode is applied, where the human being
makes the overall decisions, while supported by mostly au-
tonomously operating robots. The aim is for flexible and
seamless switching between the various modes in order to
compensate for loss of network signals and to maintain sup-
port of the squad-leader.
5 Olfactory-based robot swarm navigation
The swarm will navigate autonomously through the terrain
using ’conventional’ SLAM sensors. An additional naviga-
tion mode is based on data from the olfactory sensors.
The possibility to measure the gas concentration with a
mobile robot enables a broad range of applications, rang-
ing from surveillance of environmental pollutants [Marques
et al., 2003] and the detection of hazardous gases, to self-
produced odors to aid navigation [Larionova et al., 2006]. So
far, the main problem studied in olfactory navigation has been
the tracking and localization of static odor sources [Almeida
et al., 2004]. In real situations, this problem can be broken
into three sub-problems: finding traces of the chemical of in-
terest; tracking the respective odor plume; and identifying the
odor source.
Figure 4: Electronic nose, and search path of a swarm.
Our focus is on swarming using olfactory-based naviga-
tion. Efficient swarming algorithms for terrain surveying
based on spatial concentration of odor fields are developed.
Particularly the effects of airflow and diffusion need to be
addressed in the swarming algorithms. Both olfactory based
communicative swarming -enabling efficient coverage of the
13
area- and non-communicative olfactory based swarming -
robust fall-back option- are developed.
6 Human robot swarm cooperation
The G UARDIANS are a swarm of autonomous robots, a fact
with far stretching consequences for the human-robot inter-
face. The human swarm interface is essentially very differ-
ent from the human-robot interfaces applied in telerobotics.
In telerobotics (refer to PeLoTe project, Project IST-2001-
38873, or View-Finder FP6-045541) several humans may op-
erate one robot, in G UARDIANS however, it is basically one
human being cooperating with several robots.
6.1 Human squad leader
The objective of providing a human control over the swarm of
robots is to add some qualities which are not inherently avail-
able in the robotic swarm. Human control in swarm robotics
allows for dynamic authoritative control of specific swarm ac-
tivities based upon local circumstances and human expertise.
In addition, feedback from local robots can enhance the task
performance of the human being.
The competences and capabilities of individual robots and
humans vary. In an emergency situation a robot’s sensors,
position awareness and access will be better than those of the
human, by contrast the humans problem solving ability, inter-
pretative powers and authority will exceed those of a robot.
Having a human add some qualities which are not inherently
available in the robotic swarm, e.g. opportunism, inductive
reasoning, improvisation on unexpected events, learning from
experience. However, no matter how intelligent the robotic
system is, as demonstrated by a lot of remarkable results in
autonomous robotics, still human supervision is necessary,
especially for some critical and sophisticated tasks like search
and rescue (SAR).
Swarm guidance algorithms are built based on the au-
tonomous operations of the robots (distributive) to which
(human originating) tactical planning instructions might be
added. The swarm will be monitored by human supervisors.
At this level there are several new problems to be solved.
One is to interweave human control with the (mostly) au-
tonomously operating swarm and to enable the operator to
single out individual sensor-carriers and take control over
them.
6.2 Base station
The swarm of mobile robots can also be accessed via the
base station. The base station is the central point for data
collection and incorporates a decisive controller function for
the entire operation. It operates in automatic mode, in semi-
automatic mode or in operator mode while accessed via the
user interface. The base station provides global instruction
to the robots. The base-station allows point-to-point con-
nections between robots and the operator as well as point-to-
multipoint and broadcast connections to communicate to the
group of robots. The overall operations of the robot swarm
will be monitored via the base station.
The base station incorporates:
1. A location mapping unit
 2. A human supervised operation command post.
The location mapping unit up-dates maps with position in-
formation received from the robots and, when applied, the hu-
man squad leader. At the base station a mixture of local -robot
centered- information and global information (risk maps, ob-
stacle maps, etc.) is available. From this mixture, an overview
of the data is compiled, mainly presented as maps. Overview-
ing the swarm at group-level conclusions may be deducted,
for instances about areas considered cleared or possible loca-
tions of the chemical plume source. These conclusions might
be translated into sets of instructions to the swarm.
Whenever possible, the robots act autonomously and are
only monitored by the supervisor. However for complex
group operations, and even though the robots have as high
a level of autonomy as possible, it is still necessary to em-
power one or more operators to control them, especially in
critical tasks like cooperated search for explosives or chem-
ical agents, and rescue missions. This is due to the fact that
the robots have limited resources, meaning that there are con-
ditions in which the robots do not know what to do. Inter-
ference of human operators will solve this problem, and thus
support the robots to accomplish the mission. Via the user
interface the supervisor can adjust the degree of autonomy of
the robots.
The base station assists the supervisor in controlling the
swarm; it displays maps, and visualizes the group behavior
and automatically derived conclusions. The human operator
is supported in the overall functions of strategic and contin-
uous tactical planning for the swarm of robots. The operator
can also acquire detailed information about the actions of the
individual robots. The Base station human interface provides
functions for group instructions as well as for direct (robot)
task allocation.
7 Potential impact
Many current swarm-based projects seek to investigate the
effect of swarming and learning in theoretical and controlled
environments. The G UARDIANS project intend to take this
in the next level by offering a practical and real-world appli-
cation for such research. The research in the G UARDIANS
project impacts on the following innovations:
1. Advancing fire ground search technology
2. New algorithms and methods for robot swarming tech-
nology
3. Multi-sensor data fusion in a swarm of robots
4. Ad-hoc communication and networking methods and
service discovery for mobile robots
5. Olfactory-based navigation
6. Human-swarm cooperation
7.1 Fire ground search technology
South Yorkshire Fire and Rescue Service (SyFire) proposes
the use-case scenario of searching an industrial warehouse
covered in smoke. Performing research in this given real-
world scenario will deliver several important innovations and
results:
14
• A review of the currently applied search strategy and
procedures, dating back to the Second World War.
• From the robotics viewpoint completely new strategies
may arise, that may be useful even without robots being
applied.
• Applying robots even when they are only partially
successful- may enhance this time consuming search
task.
• A continuously evolving ad-hoc wireless network will
be developed, which can be applied on robots as well as
on human fire fighters.
• Several solutions for localization will be developed,
some based on sensors others on communication; each
of these solutions might aid the current orientation prob-
lem of the fire fighters.
• Robot-nose and array of chemical sensors, devices to
speed-up the identification of possible toxic gases that
enhance decision making by the commander on whether
and how the ground can be entered by a human fire
fighter
• Enhance safety and efficiency of Search and Rescue op-
erations.
7.2 Communicative and non communicative robot
swarming
Methods for robot swarming are currently an important topic
of research. The G UARDIANS project is one of the first to
apply swarm technology in a real life scenario, which calls
for a mixture of jobs. The importance of the jobs varies with
the situation the robots find themselves in. The strength of
applying a swarm is that each of the individual robots can
dynamically be assigned to any of these tasks.
Autonomous swarming: two approaches are explored in
the project, one without explicit communication and one with
communication; the latter includes service discovery as well.
The approaches are based on studying how multiple individ-
ual behaviors merge to become group behavior, and the group
may accomplish higher-level tasks. This approach has deep
implications on how cognitive systems can be designed and
implemented.
Decentralized control as the project aims at (usually the
swarm operates non-supervised) is a relatively new concept
and largely unexplored although a lot of research is ongoing.
The algorithms developed in the G UARDIANS project will be
general enough to render them applicable in other applica-
tions and contexts as well. These applications include ev-
eryday surveillance tasks as well as other search and rescue
operations. This project will contribute to advancing the state
of the art in the subject field. The proposed architecture might
be used in other applications such as Intelligent Vehicles and
Highway System (IVHS).
7.3 Multi-sensor data fusion in a swarm
Research into the fusion of data from multiple sensors is a
topic of current research in single robot systems. The problem
of data fusion in a swarm environment is at least an order
more complex. The research in G UARDIANS will address the
fusion of data from multiple robot sources. This in turn will
govern the individual behaviors of agents in the swarm.
In a way the G UARDIANS robots, together, will act as one
sensor network that can be used to identify safe passages and
the location of human targets in unsafe environments due to
chemical spillage or fire.
Techniques for target tracking, data association and identi-
fication have to be addressed and developed. In this context,
multi-modal statistical techniques will be used to develop the
framework for fusion processing. Delivery of this objective
will offer new and novel methods that can combine sensor
and classifier information in a distributed manner.
7.4 Ad-hoc mobile networking and service
discovery
At a built-up terrain, and in an industrial warehouse in par-
ticular many metal obstacles may be present that pose several
communication and connection problems, therefore the wire-
less communication network is typically self-organizing and
aims at ad hoc modifications ranging from single point-to-
point connections between local peers via local group con-
nections to centralized connections with the base station.
The Fire and Rescue services are desperate for wireless
mobile communication systems that do not require that net-
work nodes have to be positioned previous to the network
being operational. The ad hoc network, the nodes of which
are in the project carried by the robots, would be an ideal so-
lution for the communication problem; implemented with or
without robots.
The wireless network provides also one of the (rough)
means to locate the robots and the human fire fighter on the
map available at the base station. In applications this func-
tionality allows that the operational command can keep track
of the fire fighter even when the person is not or no longer
responding.
7.5 Olfactory-based Robot swarm navigation
The G UARDIANS project also addresses olfactory-based nav-
igation, and chemical sensors. Their use in fire fighting is
explained above. The application of these sensors on mobile
robot swarms opens up new directions for various applica-
tions beyond the rescue services.
Robot swarms are particularly apt for source detection.
First, as safety requirements for a robot are much lower then
for a human being the robots might be applied in high risk
environments, seconds, because the swarm spreads out they
quickly provide geographical data from which the position
source might be inferred.
7.6 Human Robot Swarm cooperation
The research on the human squad interface has to tackle sev-
eral very new problems. The major requirement is that the
human squad leader interface creates a feel of confidence to
the human. The robot swarm is to help the human to navigate
where the human senses are failing. Feedback from the robots
enhances the task performance of the human being. The inter-
face has to be effective but simple, as any additional complex-
ity of working with a robot swarm could detract the human
being from the potential utility of the swarm. The project has
15
to develop a language to maintain control and co-ordination
between the robots and the human. Moreover, given to con-
ditions on the fire ground, the smoke and the noisy breath-
ing equipment, the project cannot fully rely on the commonly
used audio-visual communication means and is forced to look
at other means to establish communication between the hu-
man and the robots.
A solution that meets all or most of these requirements will
be extremely novel and results might easily be transferable to
many areas where a bond between a human and one or more
robots is desirable. Applications for this innovation may vary
from supporting humans operating in hazardous or danger-
ous environments (regularly experienced by rescue workers)
up to supporting audio-visually handicapped people in their
everyday lives.
Supporting human control: complex tasks are still in the
realms of human reasoning and will be so for quite some
time. It is important that while the proposed robotic system
is able to operate as independently as possible in automatic
and semi-automatic ways it does allow and support full hu-
man operator control. The project investigates ways in which
the system can effectively interact with humans in ways that
support rapid decision making by providing full information
in easily accessible formats. In particular, the outputs of the
map building activity and what they might mean, with uncer-
tainties highlighted. Such communication strategies are rele-
vant to all applications where Geographical data are deployed
from simple home tasks to space exploration.
Acknowledgments
Support from EU-VI Framework Programme (IST-045269) is
acknowledged. G UARDIANS is part of the EC Cognitive Sys-
tems initiative. Jaume-I University is also funded by Spanish
Ministry of Education-CICYT (DPI2005-08203-C02-01) and
Generalitat Valenciana-IMPIVA (IMCOIC/2006/48).
References
N. Almeida, L. Marques, and A.T. de Almeida. Directional
electronic nose setup results on the detection of static
odour sources. In Proc. IEEE Int. Conf. on Sensors,
Wien, Austria, 2004.
M. Asada and E. Uchibe K. Hosoda. Cooperative behavior
acquisition for mobile robots in dynamically changing
real worlds via vision-based reinforcement learning and
development. Artificial Intelligence, 110(2):275–292,
June 1999.
R. Bachmayer and N. E. Leonard. Vehicle networks for gra-
dient descent in a sampled environment. In Proc. Conf.
Decision Contr., pages 112–117, Las Vegas, Nevada,
December 2002.
T. Balch and R. C. Arkin. Behavior-based formation control
for multirobot teams. IEEE Trans. on Robotics and Au-
tomation, 14(6):926–939, December 1998.
J. Barraquand, B. Langlois, and J. C. Latombe. Numeri-
cal potential field techniques for robot path planning.
IEEE Transactions on Systems, Man, and Cybernetics,
22(2):224–241, 1992.
J. Cortes, S. Martinez, T. Karatas, and F. Bullo. Coverage
control for mobile sensing networks. IEEE Trans. on
Robotics and Automation, 20(2):243–255, 2004.
J. P. Desai, J. Ostrowski, and V. Kumar. Modeling and con-
trol of formations of nonholonomic mobile robots. IEEE
Trans. on Robotics and Automation, 17(6):905–908, De-
cember 2001.
M. Dorigo and E. Sahin. Special issue on swarm robotics.
Autonomous Robots, 17(2-3), September 2004.
Jia Lei Du, Stefan Rührup, Ulf Witkowski, and Ulrich
Rückert. Resource and service discovery for large–scale
robot networks in disaster scenarios. In Proceedings of
the IEEE International Workshop on Safety, Security and
Rescue Robotics (SSRR2005), Kobe, Japan, june 2005.
M. Egerstedt and X. Hu. Formation constrained multi-agent
control. IEEE Trans. on Robotics and Automation,
17(6):947–951, December 2001.
Alessandro Farinelli, Luca Iocchi, and Daniele Nardi. Multi-
robot systems: A classification focused on coordina-
tion. IEEE Transactions on System Man and Cybernet-
ics, part B, pages 2015–2028, 2004.
V. Gazi and K. M. Passino. Stability analysis of swarms.
IEEE Trans. on Automatic Control, 48(4):692–697,
April 2003.
V. Gazi and K. M. Passino. A class of attraction/repulsion
functions for stable swarm aggregations. Int. J. Control,
77(18):1567–1579, December 2004.
V. Gazi and K. M. Passino. Stability analysis of social forag-
ing swarms. IEEE Trans. on Systems, Man, and Cyber-
netics: Part B, 34(1):539–557, February 2004.
V. Gazi. Formation control of a multi-agent system using non-
linear servomechanism. International Journal of Con-
trol, 78(8):554–565, 2005.
V. Gazi. Swarm aggregations using artificial potentials and
sliding mode control. IEEE Transactions on Robotics,
21(6):1208–1214, 2005.
V. J. Kumar, N. E. Leonard, and A. S. Morse, editors. Coop-
erative Control: 2003 Block Island Workshop on Coop-
erative Control, volume 309 of Lecture Notes in Control
and Information Sciences. Springer-Verlag, 2005.
S. Larionova, L. Marques, and A.T. de Almeida. Olfactory
coordinated mobile robot area coverage. Autonomous
Robots Journal, Special Issue on Mobile Robot Olfac-
tion, 2006.
J. C. Latombe. Robot Motion Planning. Kluwer Akademic
Publshers, 1991.
L. Li, A. Martinoli, and Y. S. Abu-Mostafa. Learning and
measuring specialization in collaborative swarm sys-
tems. Adaptive Behavior, 12(3–4):199–212, 2004.
Y.-H. Liu, S. Kuroda, T. Naniwa, H. Noborio, and S. Arimoto.
A practical algorithm for planning collision-free coordi-
nated motion of multiple mobile robots. In Proc. IEEE
International Conference on Robotics and Automation,
volume 3, pages 1427–1432, 14-19 May 1989.
16
London Fire Brigades Union. Fire Brigades Union Investiga-
tion into fatal fire at Hays Business Services., 1991.
L. Marques, N. Almeida, and A. T. de Almeida. Olfactory
sensory system for odour-plume tracking and localiza-
tion. In Proc. IEEE Int. Conf. on Sensors, Toronto,
Canada, 2003.
P. Ögren, E. Fiorelli, and N. E. Leonard. Formations with
a mission: Stable coordination of vehicle group maneu-
vers. In Symposium on Mathematical Theory of Net-
works and Systems, August 2002.
L. E. Parker. Current state of the art in multi-robot teams. In
Distributed Autonomous Robotic Systems 4, pages 3–12,
2000.
S. Patnaik, A. Konar, and A. K. Mandal. Improving the multi-
agent coordination through learning. IETE JOURNAL
OF RESEARCH, 51(5):395–406, September-October
2005.
J. Penders, L. Alboul, and M. Rodrigues. Modelling in-
teraction patterns and group behaviour in a three-robot
team. In Proceeding of Taros 04, Colchester; Tech-
nical Report Series, CSM-415 University of Essex,
ISSN 1744-8050. http://cswww.essex.ac.uk/technical-
reports/2004/csm415/, 2004.
J. H. Reif and H. Wang. Social potential fields: A distributed
behavioral control for autonomous robots. Robotics and
Autonomous Systems, 27:171–194, 1999.
C. W. Reynolds. Flocks, herds, and schools: A distributed
behavioral model. Comp. Graph., 21(4):25–34, 1987.
E. Sahin and W. M. Spears, editors. Swarm Robotics, A State
of the Art Survey. Lecture Notes in Computer Science
3342. Springer-Verlag, Berlin Heidelberg, 2005.
D.J. Stilwell, B.E. Bishop, and C.A. Sylvester. Redundant
manipulator techniques for partially decentralized path
planning and control of a platoon of autonomous vehi-
cles. IEEE Transactions on Systems Man and Cybernet-
ics Part B-Cybernetics, 35(4):842–848, 2005.
H. G. Tanner, A. Jadbabaie, and G. J. Pappas. Stable flock-
ing of mobile agents, part i: Fixed topology. In Proc.
Conf. Decision Contr., pages 2010–2015, Maui, Hawaii,
December 2003.
H. G. Tanner, A. Jadbabaie, and G. J. Pappas. Stable flocking
of mobile agents, part ii: Dynamic topology. In Proc.
Conf. Decision Contr., pages 2016–2021, Maui, Hawaii,
December 2003.
E. Uchibe, M. Nakamura, and M. Asada. Cooperative behav-
ior acquisition in a multiple mobile robot environment
by co-evolution. In Lecture Notes in Artificial Intelli-
gence 1604, pages 273–285, 1999.
J. P. Wangermann and R. F. Stengel. Optimization and co-
ordination of multiagent systems using principled nego-
tiation. Journal of Guidance, Control, and Dynamics,
22(1):43–50, 1999.
H. Yamaguchi. A cooperative hunting behavior by mobile-
robot troops. The International Journal of Robotics Re-
search, 18(8):931–940, September 1999.
 Learning by Imitation: A Scheme for Offline Learning of Robot Control through
Demonstration by Other Robot

Umar Suleman
Lahore University of Management Sciences
Computer Science Department
Lahore, Pakistan
SULEMAN@LUMS.EDU.PK
Rauf Baig
FAST - National University of Computer and Emerging Sciences
Computer Science Department
Islamabad, Pakistan
DR.RAUF.BAIG@NU.EDU.PK

Mian M. Awais
Lahore University of Management Sciences
Computer Science Department
Lahore, Pakistan
AWAIS@LUMS.EDU.PK

Abstract
This contribution proposes a machine learning
scheme inspired from learning by imitation and
demonstration in nature where demonstrations per-
formed upon a robot is used to program another ro-
bot’s control program. The proposal is kept generic
and open to deal with a variety of robotics’ prob-
lems. An evolutionary learning algorithm as in
[Surmann, 2000] has been used to implement the
core fuzzy system learning phase. Thus, offline
learning of a robot controller program has been
shown using AIBO robot in a foot-landing task.
The data used for learning in the given experiment
is generated by a human expert controlled perform-
ance over another robot.
1 Introduction
Interaction and imitation are the fundamental mechanisms
for shaping both individuals and societies in terms of infor-
mation, skills and behavior. Learning by imitation has been
regarded as a key source of ‘social’ learning in biological
forms [Noble and Todd, 1999]. Neuroscience experiments
on monkeys have revealed a very tight coupling of percep-
tion of motion and motor neurons, strengthening the belief
in existence of imitation in nature as a basic form of learn-
ing. Some research works have tried to model such coupling
[Mataric, 2002], and there is an ongoing quest by research-
ers for a clearer understanding of this phenomenon and its
reproduction in robots and other artificial and virtual forms
[Alissandrakis et al., 2001].
1.1 Learning by Imitation in Robots
Learning is essential in robotics, especially for modern
autonomous robots with social and/or dynamic orientation.
Learning by imitation is an instance of social and continu-
ous interaction and has been successfully employed for a
wide range of autonomous robotics applications. Learning
by imitation in robots has shown tremendous potential and
productivity [Herrera, 1998]. Many novel representations of
learning systems are proposed to deal with imitation of
varying degrees of complexity. Initially social learning in
robots was employed on groups of similar robots where
robots learned navigation and other movements of one
leader robot. In later works such learning evolved into inter-
personal communications of robots, learning from slightly
different specification robots and then to imitation of hu-
mans and other intelligent embodiments [Billard, 2000].
Robot learning by imitation ranges over various levels
of complexities. A primitive level learning would cover
motor tasks observations alone. In contrast, a higher-level

17
instance would be focusing beyond just motor actions, on
identification of the underlying motivation and goals, of
observed actions and then accomplishments of these goals
[Maistros and Hayes, 2001]. Consequently, high-level envi-
ronment modeling and perception, cognitive modeling,
complex goal identification and planning, and interactive
social imitation are all goals of research in learning by imi-
tation. Some contributions in learning by imitation in robot-
ics have realized imitation as more of a goal-oriented exer-
cise rather than action mimicry, while others have focused
on imitation as reproduction of actions alone, ignoring out-
come and success [Mataric, 2002]. Selection and design of
primitives is naturally a very difficult task in implementing
learning by imitation in robots. Different contributions have
suggested collaborative use of imitation learning constructs
and other machine learning techniques for improvement in
results [Hayes and Demiris, 1994].
Learning by imitation in robots has been extended to
dissimilar robot, and human demonstrators. It has also been
shown that high level tasks learning is poor with a robot
demonstrator as compared to learning of the same tasks with
human demonstrator, however the results are inverse when
dealing with low level task learning or homogenous action
robot demonstrator [Nicolescu and Mataric, 2001].
The employment of vision as the medium of observa-
tion has been the choice of the majority of work in the field.
Besides vision, there are other mediums of observation that
can potentially produce notable enhancement in the technol-
ogy. In the scenario where robots acts as demonstrators for
other robots to learn, the communication mediums and ob-
servation horizon is broadened to new artificial means like
electric signals, digital communication etc.
1.2 Fuzzy Systems Learning and Robotics
Robots are essentially control systems. The internal logic of
any contemporary generic control system such as a robot is
provided very often in the form of a fuzzy system or a fuzzy
logic controller. Fuzzy systems are rule-based systems ca-
pable of handling uncertain situations. Their robustness is
dependant on the appropriateness of the rule base [Takagi
and Sugeno, 1985]. The rules are often provided by experts
and are mostly incomplete, especially when the system is
dynamic and complex. Sometimes, a thorough, costly and
complex analysis is required for the design of the system at
hand. In practice inappropriate rule bases are made appro-
priate by means of trial and error.
The alternative to convention fuzzy controller devel-
opment is empirical automated modeling or simply data-
based learning which, if successful, diminishes the disad-
vantages of the former methodology. The learning of fuzzy
systems has two crucial ingredients, structure i.e. the data
base and parameters i.e. the rule base. While majority of the
work has been concentrated on parametric and partial tuning
[Castellano and Fanelli, 2000], a few have worked on com-
plete knowledge base system learning, all of which have an
incremental procedure involving little or none simultaneous
18
knowledge base learning [Linkens and Chen, 1999]. Over
the time four categories have emerged signifying different
generic approaches in generation or learning of fuzzy sys-
tems.
Most techniques actually generate only the rule-base,
given a data base in advance like [Jeong and Oh, 1999].
However some studies showed optimal data base to be of
much greater impact on the robustness and simplicity of
final fuzzy system then to an optimal rule base. To this rea-
son, there are in fact schemes that optimize the data base as
well after a rule base has been generated [Karr, 1997]. This
kind of optimization poses constraints on the structure of
data base and therefore only the membership functions can
be optimized, which do not completely exploit the potential
benefits. Another such category is of simultaneous rule base
and data base generation, which has shown com-parable to
better results but has undesirably long computational re-
quirements. A natural adjustment to the former approach is
more informed scheme with focused optimization of data
base, followed by rule base derivation. Contributions in the
last approach are beginning to emerge and have shown im-
proved results as compared to the other three [Surmann,
2000; Surmann and Selenschtschikow, 2002].
Many learning technologies have evolved for fuzzy sys-
tems generation from data, some even allowing integration
of the available expert model knowledge in the generated
system. The major methods amongst them include Fuzzy
Inductive Reasoning (FIR) [Mugica and Nebot, 1996],
fuzzy-rule oriented statistical analysis (fuzzy-ROSA)
method [Krone and Kiendl, 1996] and other statistical
methods, neuro-fuzzy [Figueiredo and Gomide, 1999] and
genetic-fuzzy identification [Surmann, 2000]. There is a
variety of other approaches as well that do not fall under any
particular category [Wang and Mendel, 1992].
Application of genetic algorithms, specifically on fuzzy
systems, is an emerging area at present. One interesting ap-
plication is the supervised learning of fuzzy systems using
genetic algorithms.
Learning by imitation has yet to play a vital rule in ro-
botics and control systems in general. This work will focus
on imitation of the trivial-moderate degree where neither
human nor robot demonstrators have shown convincing
superiority over the other. The scope of this work spans
homogeneous (exactly same anatomy of the demonstrator
and learner robots) environment learning.
2 Learning Scheme
The learning scheme proposed here requires a demonstrator
robot to perform the action of interest and a learner robot to
learn the demonstrated action as its control program. In a
homogenous environment, where the demonstrator robot
and the learner robot are exactly same, this learning effort
seems useless in practice because the control program of the
demonstrator could be copied verbatim to the learner robot
to achieve perfect results. The utility of this scheme comes
into perspective when the demonstration is actually per-
formed by a human expert or Expert Human Controller
(EHC) by controlling the demonstrator robot in some intui-
tive manner. Thus a control program could actually be pro-
duced by this scheme when there is no real control program
in the demonstrator robot at all. And that is in fact the real
strength of this scheme. Therefore from an architectural
viewpoint the demonstrator robot in the proposed scheme is
void of a high-level control program. There is only the very
basic inventory of get and set functions that expose the
minimal interface required by EHC to use all the sensors
and actuators of the demonstrator robot respectively. EHC
interact with the get and set functions through the controller
interface as shown in the figure 1.
Figure 1. Schematic view of learning environment setup on
the demonstrator robot
Observer module listens on all the communication of
get and set functions during demonstrations performance
through the Controller Interface and stores it in the Demon-
stration Log. When the required demonstrations are stored
the log is passed on to the learner robot to learn a control
logic equivalent to the control logic of EHC. Arguably, the
medium of communication and observation appropriate for
inter-robot learning should ideally be one that is most in-
formative and has the least of problems of its own. Hence
we chose raw digital data i.e. log. We believe that vision is
not an ideal medium as the mere interpretation of a visual
observation is a hard and open problem to date and a full-
fledged field in its own right to work on. So simply stated
the approach we have identified is basically using the dem-
onstration data stored in the said log to learn EHC equiva-
lent autonomous control system which is finally installed in
the learner robot to deal with real situations.
The whole scheme is broken into a sequence of generic
phases as shown in figure 2. The output of each phase is
also shown. The remainder of this section covers each of the
phases briefly.
19
Environment Setup
EHC Ready
Demonstration Performance
Demonstration Log
Observations Pre-processing
Training Data
Fuzzy System Learning
Fuzzy System
Post-processing
Refined Fuzzy System
Deployment
Figure 2. Phases of proposed scheme
2.1 Environment Setup
The prerequisite environment for the EHC to perform dem-
onstration over the demonstration robot cannot be set up
with any fixed universal formula. However we have ex-
tracted guidelines to carry out the setup phase. The objective
is to engineer an intrinsic computable medium to float the
demonstration forward to the learner robot. First of all the
system to be modeled is analyzed to enumerate all the inputs
to the system actuators i.e. commands, and outputs of the
system i.e. sensor readings etc. In addition the maximum
range of values each of inputs and outputs is calculated.
The second task is to get the demonstrator robot ready
for perform the target action. The demonstration system is
set up according to figure 1. The basic task in this is to, im-
plement the set and get functions to control actuators and
read sensors respectively, put on the controller interface and
the observer modules to record communication between
EHC and underlying hardware of the demonstration robot
and implement the tools for EHC that are intuitive for a hu-
man to use for control. The latter could be implemented in a
variety of ways ranging from simple joystick hardware
setup to sophisticated software cum wearable sensors instal-
lation.
2.2 Demonstration Performance
EHC performs the demonstration of interest multiple times
over the demonstration robot, thus generating demonstration
log. All principles of generating a good supervised learning
data apply. The number of performances should be suffi-
cient enough so that each important situation to be handled
is encountered. Consequently it should be observed that with acceptable solution(s) in the population, as a best effort
each performance deals with new situation. approach, tries to improve the structural elegance of ac-
The Observer module is responsible to sniff all the data ceptably fit individuals, i.e. subsuming similar membership
passing through the Controller Interface and log them. The functions and removing unused rules and membership func-
scheme proposes a simple relational format for the demon- tions called the closing step. Note that when the closing step
stration log consisting of name-value pairs prescribing the cannot reduce an acceptable fuzzy system at all it is as-
actuator commands and sensor readings at each instance of sumed that the most compact system found, hence the algo-
time during demonstrations. rithm exits with success. Third it falls in the most effective
and generic class of learning techniques where the both
2.3 Observations Pre-processing membership functions as well as the rules are learned simul-
Raw demonstration log is processed in the pre-processing taneously.
phase to prepare it for the learning phase to use as supervi-
sory training data. In pre-processing the demonstration log
is processed depending upon the actual learning algorithm
to follow. As an example our current implementation uses
simple data normalization on sensor readings and actuator
values.
The option to cleanse the recorded data off noise and
outliers is consciously rejected on the basis of the fact that
the noisy records are bound to re-occur on the learner robot
platform because of the same reason they occurred at the
time of demonstration. Keeping the learning system in igno-
rance of the potential noise can contribute greatly in produc-
tion of a non-robust control system.

2.4 Fuzzy System Learning

The crux of this scheme is the fuzzy system learning phase.
Given the pre-processed data the goal is to produce a fuzzy
system which can mimic the control logic of the EHC i.e.
the fuzzy system can continually produce the right set of
actuator commands given the sensor readings to achieve the
goal.
The criteria for selection of a particular method for this
phase are generality, robustness and completeness. Genetic
learning of the fuzzy system is one good option for this
phase (see section 1.2) which our current implementation
employs based on works of Herrera and Surmann [Herrera,
1998; Surmann, 2000]. It is implemented as an offline sys- Figure 3. Genetic Algorithm for Fuzzy System Learning
tem which takes all the observation records as training sam-
ples in order to produce a reasonably fitted fuzzy system. When the genetic algorithm returns an acceptable fuzzy
The logical flow of the genetic algorithm implementation is system, it is further optimized by a simple numerical tech-
given in Figure 3. nique i.e. membership functions parameters are fine tuned
To understand the real implication of the used algo- with increments/decrements of reducing magnitude to the
rithm one thing should be kept in mind that the individuals point where no more improvement is possible on the train-
of the population optimized by this genetic algorithm are ing data. This technique provides the fine granular steps to
complete fuzzy systems, with membership functions and get to the real maxima, where the gross granular genetic
rules and the algorithm tries to find an optimum fuzzy sys- steps are unable to reach.
tem with an acceptable MSE over the given training data.
The complete fuzzy system is encoded into two strings i.e.
one for each of rules and membership functions. The mem- 2.5 Post-processing
bership functions used are asymmetric Gaussian functions. The output fuzzy system of the learning phase is not de-
This technique has been selected by virtue of some very ployed as such. The fuzzy system is de-normalized back in
useful features. First it provides recovery from persisting exactly the reverse manner to the initial normalization in the
stagnation in improvement of individuals in the population pre-processing phase. All the membership functions, for
by mutating portions of unfit individuals denoted as the example, are de-normalized to deal with actual sensor val-
open population step. Second when it reaches any maxima

20
ues. The fuzzy system thus constructed is depicting the real
logic i.e. it can process real inputs and render real outputs.
2.6 Deployment
The learner robot is now ready to switch control to the
learned fuzzy system controller which is done in a straight-
forward manner. The demonstration setup is reused in the
robot with some precise alterations as depicted in Figure 4.
Fuzzy
Inference Fuzzy
Driver System
Controller Interface
Set Get
Functions Functions
Actuators Sensors
Figure 4. Schematic view of learner robot system
First the complete system is copied on to the learner ro-
bot and then few alterations follow. The only significant
alteration is the replacement of EHC with fuzzy inference
driver module. The fuzzy inference driver, coupled with the
learned fuzzy system, replaces the EHC. The fuzzy infer-
ence driver is a very lean piece of logic that receives the
inputs of the sensors via the controller interface, passes it to
the fuzzy system for inference, and returns the fuzzy sys-
tem’s outputs to the controller interface, destined towards
the set functions to control actuators. Note that the observer
is no longer required and is thus removed from the system.
3 Experimentation
The proposed scheme is being implemented on a simple
problem using Sony’s AIBO® robot dog as the learner and
demonstrator robot. In order to test the genetic fuzzy im-
plementation of the learning scheme AIBO ERS-220 is pro-
grammed using Open-R SDK version 1.1.4.
The target action is simple yet non-trivial right fore-leg
movement to land it on the ground properly starting from an
arbitrary position. Note that the learner robot and the
scheme will be said successful if it can properly land it’s
arbitrarily positioned right front leg (only) on the ground
below at a particular distance. Proper foot landing on
ground means to make foot contact with the flat surface
below such that the leg is not applying any undue force on
the surface. It is assumed that robot main body is firmly
suspended i.e. managing balance and the three remaining
limbs are out of scope of this experiment. The complete
experiment was repeated multiple times according to the
21
proposed scheme. An account of each phase of experiments
has been laid down as below.
For Environment Setup, the inputs of the target system
are analyzed and thus six inputs are selected for the test,
namely, right front leg joints i.e. RFLEG_J1, RFLEG_J2,
RFLEG_J3, right front leg paw switch i.e. RFLEG_SW,
PSD, and frame i.e. system timer. Note that frame is not a
literal sensor, but as time can possibly be a factor of the
target system, frame is included in the inputs list for the
sake of completeness. The outputs of the system are
RFLEG_J1, RFLEG_J2, and RFLEG_J3 i.e. the three joints
of the front leg.
The mechanism developed for EHC to perform the
demonstration is very intuitive. The EHC can simply hold
the leg of the robot, and maneuver it manually to land it on
the surface below as if the demonstrator robot was landing
the foot. The controller interface is developed such that it
can reverse engineer the commands necessary to perform
the forced movements by the EHC, and pass it to the Ob-
server for logging. The get functions module is implemented
by a single Open-R object to simply pass on the read values
from the sensors to the controller interface. The Observer
module is implemented in the same Open-R object, to store
the received observations, remotely on a desktop computer
host.
The readings from all the identified inputs, namely
RFLEG_J1, RFLEG_J2, RFLEG_J3, RFLEG_SW, PSD,
and frame are taken and passed on the Observer module.
The Demonstration is performed by the EHC multiple times
in a row at different initial orientation of the right leg.
For Observations Preprocessing the complete observa-
tional data is passed on to normalization routine to convert
all the field values onto the normal scale between 0 and 1.
This data now enters the Fuzzy System Learning phase
where the genetic algorithm searches for a fuzzy system
with MSE less than the configured acceptable MSE (e.g.
0.02). Once such a system is learned its membership func-
tions are denormalized to depict real situation and matching
response as explained in 2.5.
The learned system is manually deployed in the learner
robot. The controller interface is slightly modified to create
a command object from the output of the inference driver
and pass it to the Set functions module. The learner robot is
then rebooted such that the new fuzzy system controller
takes over the control and performs in real environment.
3.7 Results Analysis
All experiments eventually generated a fuzzy system to
mimic the control logic of the EHC by running for a few
hours. The fuzzy system generated each time was slightly
different from the rest. However the common observation in
every learned fuzzy system is that it is compact with only
two rules and often with a total of eleven membership func-
tions for the 3 input and 3 output variables. The remaining
three input variables PSD, PAWSW and FRAME have no
membership function, which means that each time the learn-
ing algorithm has discarded these three variables as non-
contributing inputs. Almost every time the system is put to
real action in the learner robot it is able to land the foot
starting from any arbitrary initial position with realistic ele-
gance as judged by the naked eye. Thus a fuzzy system with
an acceptable behavior and a very compact structure has
been generated to depict the target action logic.
Taking one of the experiment’s results as example
where a population size of 20, an acceptable MSE value of
2%, and initial rule count of 10 was used. An acceptable
fuzzy system was returned after 43125 generations and 316
stagnations as given in Figure 5.
Figure 5. Learned Fuzzy System
Thus the implicit expert control rules have been eluci-
dated in a human readable format by the learning algorithm
using the proposed scheme. The dependency, in develop-
ment of a fuzzy membership functions and rules, to control
a robot, on a real expert has been eliminated. A repeatable
process is given that can produce new fuzzy system-based
controller in reasonable time successfully.
4 Conclusion
The scheme has been highlighted in a number of ways.
First, genetic algorithm based fuzzy system learning, a soft-
computing technique has been applied in robot control
learning through robot demonstration. Second, the complete
implementation of the genetic fuzzy learning algorithm pro-
posed in [Surmann, 2000] has been done and tested. Third, a
simple generic process and control architecture is proposed
which is used in two steps i.e. firstly implementation of raw
interfaces to link hardware with a human/program controller
through a controller interface, and secondly to replace the
controller with the learned fuzzy controller. The architecture
allows any form of human or program controller for carry-
22
ing out demonstration e.g. human control through a joystick
setup, an ageing robot controller, etc.
The scheme presented and implemented has success-
fully combined good human control capabilities and soft-
computing based learning algorithm to program intelligent
robotic systems through demonstration over a robot. It cov-
ers the end-to-end details of the process and is intentionally
kept generic.
The experimentation conducted so far is focused on the
proof of the concept only. There is room for further en-
hancements, most exciting of which is to mature and de-
velop soft-computing centric techniques for heterogeneous
imitation i.e. learning robot control programs from human
and dissimilar robot demonstrations.
References
[Noble and Todd, 1999] J. Noble, and P. M. Todd. "Is it
really imitation? A review of simple mechanisms in so-
cial information gathering." The Symp. on Imitation in
Animals and Artifacts, Artificial Intelligence and the
Simulation of Behaviour Convention, Edinburgh, April
1999.
[Mataric, 2002] M. J. Mataric. "Sensory-motor primitives as
a basis for imitation - linking perception to action and
biology to robotics". Imitation in Animals and Artifacts,
pp. 391-422, MIT Press, 2002. ISBN 0-262-04203-7.
[Alissandrakis et al., 2001] A. Alissandrakis, C. L. Nehaniv,
and K. Dautenhahn. "Through the looking-glass with
ALICE - Trying to Imitate using correspondences".
Proc. First Intl. Workshop on Epigenetic Robotics, Mod-
eling Cognitive Development in Robotic Systems, pp.
115-122, Sweden, 2001.
[Herrera, 1998] F. Herrera. "A Learning Process for Fuzzy
Control Rules using Genetic Algorithms". Fuzzy Sets
and Systems, vol.100, Issue 1-3, pp. 143-158, November
1998. ISSN 0165-0114.
[Billard, 2000] A. Billard. "Imitation - a means to enhance
the synthetic proto-language in an autonomous robot". In
C. Nehaniv, and K. Dautenhahn, editors, Imitation in
Animals and Artifacts, MIT Press, 2000.
[Maistros and Hayes, 2001] G. Maistros, and G. Hayes. "An
imitation mechanism for goal-directed actions". Proc.
TIMR, 2001.
[Hayes and Demiris, 1994] G. Hayes and J. Demiris. "A
robot controller using learning by imitation". Proc. the
Intl. Symp. On Intelligent Robotic Systems, pp. 198-204,
France, 1994.
[Nicolescu and Mataric, 2001] M. N. Nicolescu and M. J.
Mataric. "Experience-based representation construction -
learning from human and robot teachers". Proc.
IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems,
IROS, USA, 2001.
[Takagi and Sugeno, 1985] T. Takagi, and S.M. Sugeno.
"Fuzzy identification of systems and its application to
 modeling and control". IEEE Trans. Syst., Man, and Cy-
bernetics, vol. 15, 1985.
[Castellano and Fanelli, 2000] G. Castellano, and A. M.
Fanelli. "Fuzzy inference and rule extraction using neu-
ral networks". Neural Network World Journal, no.3, pp.
361-371, 2000.
[Linkens and Chen, 1999] D.A. Linkens, and M.Y. Chen.
"Input selection and partition validation for fuzzy model-
ing using neural network". Fuzzy Sets and Systems, vol.
107, 1999.
[Jeong and Oh, 1999] J. Jeong, and S. Oh. "Automatic rule
generation for fuzzy logic controllers using rule-level co-
evolution of subpopulations". Proc. The Congress of
Evolutionary Comp., 1999.
[Karr, 1997] C. L. Karr. "Fuzzy-evolutionary systems". In
T. Back, D. B. Fogel, and Z. Michalewicz, editors,
Handbook of Evolutionary Comp., Oxford University
Press, 1997.
[Surmann, 2000] H. Surmann. "Learning a fuzzy rule based
knowledge representation". Proc. 2nd ICSC Symp.. on
Neural Comp., NC'2000, 23-26, pp. 349-355, 2000.
ISBN 3-906454-22-3.
[Surmann and Selenschtschikow, 2002] H. Surmann, and A.
Selenschtschikow. "Automatic generation of fuzzy logic
rule bases: examples 1". Proc. First Intl. ICSC Conf. on
Neuro-Fuzzy Technologies, NF'2002, Cuba, 2002. ISBN
3-906454-29-0.
[Mugica and Nebot, 1996] F. Mugica, and A. Nebot. "A
specialization of k-nearest neighbor classification rule
for the prediction of dynamical systems using FIR". Ad-
vances in Artificial Intelligence and Engineering Cyber-
netics, vol. III, Baden-Baden, Germany, August 1996.
[Krone and Kiendl, 1996] A. Krone, and H. Kiendl. "Rule-
based decision analysis with fuzzy-ROSA method".
EFDAN’96, Dortmund, Germany, 1996.
[Figueiredo and Gomide, 1999] M. Figueiredo, and F.
Gomide. "Design of fuzzy systems using neurofuzzy
networks". IEEE Trans. on Neural Networks, vol. 10, no.
4, 1999.
[Lin and Lu, 1996] C. Lin, and Y. Lu. "A neural fuzzy sys-
tem with supervised learning". IEEE Trans. Syst., Man,
and Cyb., part B, vol. 26, 1996.
[Wang and Mendel, 1992] L.X. Wang, and J. Mendel.
"Generating fuzzy rules by learning from examples".
IEEE Trans. Syst., Man, and Cyb., vol. 22, 1992.

23
 Controlled Deployment of Sensor Networks for Coverage Optimization∗

Sameera Poduri, Gaurav S. Sukhatme

University of Southern California
Computer Science Department
spoduri, gaurav@pollux.usc.edu

Abstract
Controlled deployment of sensor networks is fea-
sible when the positions of individual nodes can
be controlled. In contrast to random, high den-
sity deployment of nodes, which is a common as-
sumption in sensor network literature, controlled
deployment is better suited to a large number of
applications and is significantly more efficient than
random deployment. The problem of controlling
the position of nodes to optimize sensing coverage
has been studied in classic geometric optimization
problems such as the art-gallery problem, disk cov-
ering, voronoi tessellations, etc. For sensor net-
work applications, there are additional challenges
because of communication constraints and lack of
global knowledge of node locations and obstacles.
This paper surveys recent research to address these
challenges for sensing models of varying complex-
ity. We summarize our results on two aspects of the
problem 1) local geometric conditions for deploy-
ment that can guarantee global coverage and con-
nectivity, and 2) self deployment for mobile sensor
networks with guaranteed connectivity, and 3)anal-
ysis of coalescence for mobile networks.
1 Introduction
Hundreds of temperature sensors distributed over a forest to
detect fires and help control them, seismic sensor-actuator
systems embedded in buildings to sense and counteract earth-
quakes, camera units manipulating low resolution images to
check for intruders in a military location, chemical sensors
mounted on buoys and boats monitoring water quality, are
a few examples from the wide spectrum of applications of
Wireless Sensor Networks (WSNs). A typical WSN consists
of a large number of short-range sensors equipped with mod-
est resources for computation, power, and wireless commu-
nication, that cooperate to infer global information about the
∗
This work is supported in part by the National Science Foun-
dation (NSF) under grants 0347621, 0325875, IIS-0133947, EIA-
0121141 and grants CCR-0120778, ANI-00331481 (via subcon-
tract). Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the authors and do not neces-
sarily reflect the views of the National Science Foundation.
Figure 1: (blanket) Coverage configuration with minimum
density of nodes
physical environment that they are embedded in. The den-
sity and locations of sensors will control the accuracy with
which a phenomenon is captured. An important question to
address is how to deploy these networks. The common de-
ployment techniques are nodes being dropped by flying vehi-
cles, planted by robots or humans, or mobile nodes self de-
ploying.
Several Wireless Sensor Network (WSN) deployments will
involve control over positions of nodes. Following are some
example scenarios:
1. Urban WSNs of sensors like cameras, seismic sensors,
temperature sensors, etc., that must be carefully im-
planted on the walls or ceilings of buildings, and other
structures, so as to be unobtrusive.
2. Sensors in water bodies that are either achored to stay in
place or that are mounted on boats that can control their
location.
3. Sensors that are not inexpensive enough to afford high
redundancy.
4. Small and medium scale networks that are maintained
by a robot or human. For example, in the NIMS
project [Kaiser et al., 2004], mobile nodes can control
the deployment locations of static nodes.
5. Mobile sensor networks.
Position controlled deployment can significantly improve
the performance and lifetime of a sensor network. The ad-
vantage over random deployment can be illustrated by the
following example. Consider sensors that can sense a cir-
cular disk of area S around them. If deployed randomly, the

24
density, λ, required to guarantee that each point in the do-
main is sensed (with high probability) by at least one sen-
sor, λ ≥ −log(0.05)
S ≈ S3 which is very high compared to a
hexagonal placement (fig. 1) where the density ≈ 1.2S . In sev-
eral applications, the expected lifetime of the network is very
long compared to the deployment phase and the effort put in
during deployment can lead to significant gains. Therefore
whenever possible, position control must be exploited. This
work considers the cases where position control is available
and addresses the problem of finding sensor locations to op-
timize sensing coverage.
Outline
Section 3 surveys recent research to address these challenges
for sensing models of varying complexity. The coverage
problem is closely related to well studied problems in geome-
try like facility-location, art-gallery problem, packing and fill-
ing of space. Sections 4-6 present our results on three aspects
of the problem 2) self deployment for mobile sensor networks
with constrained node degree [Poduri and Sukhatme, 2004],
2) local geometric conditions for deployment that can guar-
antee global coverage and connectivity [Poduri et al., 2005],
and 3) analysis of coalescence in mobile networks [Poduri
and Sukhatme, 2007]. The next section defines the coverage
problem in the context of sensor networks. Since maintaining
communication connectivity is a necessary requirement for
WSNs and communication links depend on node locations,
we call this the problem of constrained coverage.
2 Constrained Coverage
Given a bounded domain, where should n sensors be located
such that the coverage is maximized subject to some connec-
tivity constraints?
Coverage
Depending on the application, the desired sensing coverage
can be characterized in one of 3 ways [Gage, 1992]:
1. Blanket Coverage:
(a) Maximize the area sensed by a fixed number of sen-
sors, i.e., maximize the number of detections per
time.
(b) Find the minimum number of sensors such that
each point in the domain is covered, i.e., minimize
the number of missed detections per area.
2. Barrier Coverage: Find an arrangement of a barrier of
static nodes that minimizes the probability of the target
penetrating an area undetected.
3. Sweep Coverage: Find a motion strategy for a group of
mobile sensors such that the number of event detection
per time is maximized or the number of event misses
per area is minimized. This can be treated as a moving
barrier.
Sensing Model
Let Q be a bounded convex region and p = {p1 , p2 , ..., pn }
locations of n sensors. The sensing performance of each sen-
sor i at point q is given by f (i, q). Depending on the nature
of f (i, q), we can classify sensors as follows:
25
• Binary Disk Model: f (i, q) = 1 if ||pi − q|| ≤ Rs (i),
0 otherwise. Rs (i) is called the Sensing range of sensor
i.
• Binary Cone Model: The sensors are not isotropic.
If the sensors are facing {θ1 , θ2 , ...θn } directions, then
f (i, (q)) = 1 if ||pi − q|| ≤ Rs (i) and angle(i, q) ≤
φi . and 0 otherwise.
• Continuous Model: f (i, q) = g(||pi − q||) where g is
a monotonically decreasing continuous function.
A point in the domain, q is set to be covered if it is within
the sensing region of at least one node, i.e., if f (i, q) ≥ α
for at least one i, where α is a minimum threshold. Similarly,
a point qin the domain is k−covered if it is simultaneously
within the sensing region of k nodes. For sensors (like tem-
perature sensors) that measure at a point, the sensing range is
a measure of sampling granularity.
Communication Model
The nodes communicate over wireless. The probability that
a message from node i reaches node j can be represented as
P RR(i, j). In general, P RR(i, j) 6= P RR(j, i).
• Binary Disk Model: P RR(i, j) = 1 if ||pi − pj || ≤
Rc (i), 0 otherwise. Rc (i) is called the Communication
range of sensor i.
• Quasi Disc: P RR(i, j) = 1 if ||pi − pj || ≤ rc (i)
and P RR(i, j) = 0 if ||pi − pj || > Rc (i) where
rc (i) ≤ Rc (i) are the minimum and maximum trans-
mission ranges of node i. For j such that rc (i) <
||pi − pj || ≤ Rc (i), P RR(i, j) is not specified. This
model is close to realistic links [Zuniga and Krishna-
machari, 2004].
• Probabilistic Model: In complex domains, P RR(i, j)
does not have a parametric form and can be estimated as
a distribution. [Cerpa et al., 2005].
3 Survey of Related Work
The coverage problem is closely related to several optimiza-
tion problems in geometry. We will first review key results
from these areas, discuss their applicability to sensor net-
works and then focus on recent research on the coverage
problem in the context of sensor networks.
3.1 Locational Optimization
Locational optimization deals with finding optimal locations
for resources (like mailboxes) that can provide local services.
A general problem can be formulated as follows. Find loca-
tions of n facilities in a bounded domain, such that the aver-
age distance to the nearest facility is minimized [Okabe and
Suzuki, 1997]. The facilities could be point-like, line-like, or
area-like and the distance measure can also vary. The case
with point-like facilities and euclidean distance measure is
similar to the sensor network coverage problem without con-
nectivity constraints.
Given a fixed arrangement of sensors, their Voronoi di-
agram represents the partition of the domain such that the
Voronoi cell of a sensor contains all points that it ‘ser-
vices’ or covers. We can compute the maximal breach
path [Meguerdichian et al., 2001] which is the path in the
domain that maximizes the distance between a moving target
and its nearest sensor and thus captures the worst case cover-
age of the network. The optimal locations of sensors can be
found using a gradient descent approach called Lloyd’s de-
scent with the average sensor distance as the objective func-
tion [Cortés et al., 2004]. This can be used to find a dis-
tributed control law for a group of mobile nodes. The result-
ing configuration is the Centroidal Voronoi partition. A fun-
damental drawback of this method is that it does not consider
obstacles in the environment.
3.2 Art-Gallery Problem
Art-gallery is closely related to centralized deployment of
line-of-sight sensors. It deals finding locations for line-of-
sight sensors such that every point inside a polygon (that
could possibly be non-convex and have holes) is visible from
at least one sensor [O’Rourke, 1987]. The sensors are as-
sumed to have infinite range and 360o field of view. For
simple ploygons, b n3 c sensors are sufficient and sometimes
necessary. However, finding the minimum number of sensors
is NP-Hard. For non-isotropic sensors the problem is more
complex.For the case when the cameras have limited field of
view and/or limited sensing range, a number of algorithms
have been proposed to find approximate solutions [Kazazakis
and Argyros, 2002]
3.3 Packing and Filling
Packing is the problem of placing solid shapes in a bounded
domain such that no overlap is permitted. The problem is
to find the placement that minimizes uncovered area. In 2D,
only regular hexagons, squares and triangles can cover a do-
main completely.
Filling is the problem of placing solid shapes such that a
given bounded domain is completely covered. In sensor net-
works, we are interested in finding the placement of the min-
imum number of sensors that can cover the domain. The op-
timal configuration of binary disk sensors is shown in 1.
3.4 Challenges in Sensor Network Placements
Most solutions for the above mentioned problems involve
centralized solutions and do not deal with communication
constraints. In the context of sensor networks these problems
become challenging because of
1. Connectivity Constraints: there are constraints on the
connectivity of the network.
2. Environmental Complexity: the environment can have
complex shaped obstacles
3. Centralized vs Distributed Approaches the deploy-
ment algorithm does not have global knowledge of the
environment or positions of other nodes.
3.5 Coverage in Sensor Networks
Topology Control: Topology Control is considered a canon-
ical problem in static and mobile ad-hoc networks (see [Santi,
26
2003] and [Li, 2003] for detailed surveys). The primary ap-
proaches used, power control and sleep scheduling, operate
on a given well connected network and provide ways of prun-
ing the network’s communication graph to save power. They
do not leverage possible control on node positions. While
sensing coverage cannot be altered by power control, it is of-
ten the objective of sleep scheduling algorithms.
Sleep scheduling assumes an initial high density deploy-
ment and seeks to activate only a fraction of the nodes.
Several researchers have studied the problem of finding
the smallest set of active nodes that can simultaneously
achieve coverage and connectivity where complete coverage
is defined as every point in the network domain being
within the sensing range of at least one active node (Blanket
Coverage (b) ). Under this definition, coverage and connec-
tivity are not opposing goals; in fact, if the sensing range
is at least twice the communication range, then complete
coverage implies connectivity [Zhang and Hou, 2005;
Wang et al., 2003a]. In contrast we focus on the problem
of maximizing sensing coverage given a fixed number
of nodes so that higher coverage, in most cases, implies
poorer connectivity. OGDC [Zhang and Hou, 2005] and
CCP [Wang et al., 2003b] are sleep scheduling protocols
based on geometric conditions. While OGDC seeks to
minimize the number of active nodes, CCP can provide
different degrees of coverage and connectivity. Interest-
ingly, the geometric optimality conditions presented in
[Zhang and Hou, 2005], as pointed out by the authors, can
be attained exactly if the positions of nodes can be controlled.
Controlled deployment of a Static Network De-
ployment and repair of static networks by mobile robots
has been presented in [Batalin and Sukhatme, 2004;
Corke et al., 2004]. Typically, such algorithms have focussed
on the control strategies for the robot rather than properties
of the network. By providing conditions that are distributed
and can be easily combined with robot controllers, our work
provides a means of integrating the two objectives. For
a similar setting, Bredin, et. al., [Brendin et al., 2005]
have recently proposed a network repair strategy that finds
additional node locations required to ensure k-connectivity
in any given network. The algorithm is proven to be within
a constant factor of the absolute minimum number of addi-
tional nodes. In the context of repairing a network, the NET
condition provides a solution with a very low computational
cost but without guarantees on minimizing the number of
additional nodes.
Self Deployment of Mobile Robots: Potential field based
deployment algorithms also maximize coverage [Howard
et al., 2002b]. An incremental deployment algorithm that
ensures line-of-sight connectivity has been presented in
[Howard et al., 2002a]. The above algorithms, require
information about angle and distance to neighbors. They
maximize coverage but do not guarantee network connectiv-
ity.
Rendezvous: Rendezvous is the problem of multiple
mobile robots meeting at a point in an unknown environ-
ment. Several distributed algorithms have been proposed. In our experiments, each node begins with more than K
Some of these assume connectivity [Ando et al., 1999; neighbors and repels all of them using Fcover till it has only
Lin et al., 2003]. If the communication graph is an RNG then K left. The resulting neighbors are called the node’s critical
rendezvous algorithms have been proposed that can deal with neighbors and the connections between them and the node
partial communication loss over some of the links [Cortes are called critical connections. It then continues to repel all
et al., in submission]. In the absence of connectivity, the its neighbors using Fcover but as the distance between the
robots can explore the environment building a map of the node and its critical neighbor increases, kFcover k decreases
landmarks [Roy and Dudek, 2001] where other robots are and kFdegree k increases. As a result, at some distance ηRc ,
likely to visit. This is similar in spirit to the Coalescence where 0 < η < 1, the net force kFcover + Fdegree k between
problem where the base station is a landmark whose lo- the node and its neighbor is zero. At this distance, the node
cation is unknown. However, our focus is on forming a and its neighbor are in equilibrium with respect to each other.
connected component while allowing the robots to remain Fig. 2 shows a node with > K and exactly K neighbors the
”spread out” so that the time for coalescence is reduced. corresponding force profiles.
Several algorithms have been proposed to maintain connec- Mathematically, the forces can be expressed as follows.
tivity of a wireless network [Zavlanos and Pappas, 2005; Consider a network of n nodes 1, 2, 3, . . . , n at positions
Spanos and Murray, 2004] given that the initial network is x1 , x2 , . . . , xn respectively. Let ∆xij represent the Eu-
connected. clidean distance between nodes i and j, i.e. ∆xij = kxi −xj k
Fcover and Fdegree are defined as follows.
4 Coverage with degree constraint 
−Kcover

xi − xj

Fcover (i, j) =
Evidence from the theory of random networks indicates that ∆xij 2 ∆xij
global network connectivity is strongly dependent on node (  
Kdegree xi −xj
degree [Gupta and Kumar, 2003], a local constraint. In- 2 if critical connection;
Fdegree (i, j) = (∆xij −Rc ) ∆xij
spired by this result, we developed a simple self deployment 0 otherwise.
algorithm for a group of mobile nodes that maximizes sens-
ing coverage with the constraint that each node has at least K where Kcover and Kdegree are the force constants. The resul-
neighbors, where K is a parameter to the algorithm. tant force between the nodes i and j is
F(i, j) = Fcover (i, j) + Fdegree (i, j)
and node i will experience a net force of
X
Fi = F(i, j)
all neighbors j

The equation of motion for node i is formulated as:

 
Fi − ν ẋi
ẍi (t) =
m
Non-Critical connection Critical Connection
where ν is a damping factor and m is the virtual mass of
the node which is assumed to be 1.
Virtual Forces

4.1 Simulation Results

Virtual Forces

0 0

Simulations were conducted in Player/Stage [Gerkey et al.,

Fcover F
2002]. Fig. 3 shows the initial and final configurations for a
cover

0.25 η Rc
Fdegree
0.25 ηR
Fdegree
network of 64 nodes and the variation in coverage and node
Rc c Rc
distance between nodes distance between nodes
degree for different values of K. The algorithm converges
Fcover Fcover + Fdegree within a minute and we did not observe any oscillations. Fig.
4 shows that the coverage obtained is significantly better than
Figure 2: Force Profiles that of a random network.

Our approach to the problem is based on virtual potential 5 Local Geometric Conditions to guarantee
fields. We construct virtual forces between nodes so that each global Coverage and Connectivity
node can attract or repel its neighbors. The forces are of two In this section, we present a position control based topology
kinds (fig 2). The first, Fcover , causes the nodes to repel construction mechanism that has two key features:
each other to increase their coverage, and the second, Fdegree
constrains the degree of nodes by making them attract each • the required conditions are simple, local and geometric,
other when they are on the verge of being disconnected. By and
using a combination of these forces each node maximizes its • varying a single parameter (θ) provides tunable network
coverage while maintaining a degree of at least K. coverage and connectivity.

27
 50
110

100 K=0 45

90 40

K=2
80 35

Average Degree of Nodes

70
30

%Coverage
60
25

50
K=5
20
40
K=8 15
30
K=8
10
20
K=5
5
10 K=2

K=0
0
0 0 50 100 150 200 250 300 350
0 50 100 150 200 250 300 350
Time in seconds Time in seconds

(a) (b) (c) (d)

Figure 3: Snapshots of simulation in Player/Stage with 64 nodes. Blue circles indicate sensing ranges. (a) Initial Configuration
and (b) Final Configuration. Variation of (c) Coverage and (d) Node degree with time for different values of K.

110 16

100
14
Ti+1
90

12
80

70 10
Average degree
%Coverage

60
8
Yi+1
50

40 6
Ii
30
θi
4
20 Tiled Network

θi
Potential Field Network
Tiled Network
10
Random Network 2 Potential Field Network
Random Network
X
0
Ti
−2 0 2 4 6 8 10 12 0
−2 0 2 4 6 8 10 12
K K desired
Yi
(a) (b) 2R c

c
R
Figure 4: Coverage and node degree comparison with optimal
and random networks

Figure 6: Coverage with k nodes placed on the communica-

X
tion perimeter of node X. The shaded area is the total cover-
Rc Rc ≤θ age.
>θ

have been well studied and are known to be desirable global

(a) (b) topologies for wireless communication networks including
guaranteed connectivity. Given a binary disk communication
Figure 5: Illustration of the NET condition. In a) X does not model, we can prove the following 3 properties:
satisfy NET condition. It has one sector greater than θ with no
neighbor. In b) X satisfies NET condition, the largest sector 1. If each node has at least one neighbor in every 2π
3 sector
with no neighbors is smaller than θ. of its communication disk, the communication graph is
a supergraph of RNG. Moreover, 2π 3 is the largest angle
that satisfies this property.
This mechanism is based on the Neighbor-Every-Theta
(NET) condition.
We define the Neighbor-Every-Theta (NET) condition for a 2. If each node has at least one neighbor in every
node as requiring at least one neighbor in every θ sector of its θ = 2 arccos( Rrc ) sector of its communication disk, the
communication range (fig. 5). For finite networks, boundary communication graph is a supergraph of GG.
nodes or nodes on the network edge cannot satisfy such a
condition. For a given θ, we define boundary nodes as those
which have no neighbor in at least one θ sector of infinite 3. If each node has at least one neighbor in every
radius. A NET graph is one in which every node except the θ = 2 arccos( Rrc ) sector of its communication disk, the
boundary nodes satisfies the NET constraint for a given θ. communication graph is a supergraph of DelG.
5.1 Connectivity Analysis of NET Graphs
Proximity graphs, such as the Relative Neighborhood Graph It is our conjecture that for θ < π the edge connectivity of
(RNG), Gabriel Graph (GG) and the Delaunay Graph (DelG) the NET graph is at least b 2πθ c.

28
 Tiling type Degree K-Conn No. of edges
Line 2 1 Θ(n)(≈ n)
Hexagonal 3 3 Θ(n)(≈ 23 n)
Square 4 4 Θ(n)(≈ 2n)
Triangle 6 6 Θ(n)(≈ 3n)
Table 1: Properties of Tiling Configurations
5.2 Coverage Analysis of NET Graphs
Suppose that in order to satisfy the NET sector conditions,
a node must have k neighbors. Assuming a circular sensing
range of radius Rs , we can show that
For Rs = Rc , the area coverage is maximized when the
k ≥ 3 nodes are placed at the edges of k disjoint 2π
k sectors
of sensing disk (fig: 6)
Based on this theorem, we can compute an upper bound on
the maximum achievable coverage of a NET graph as a func-
tion of k. This gives a guideline to locally maximize cover-
age while satisfying the NET condition. For k = 3, 4, 6, it
is possible to place nodes such that each node has its neigh-
bors placed symmetrically on its communication range. The
resulting communication graph will be a tiling of the space:
hexagonal, square and triangle for k = 3, 4 and 6 respec-
tively. In these cases, since each node maximizes coverage
locally, the total coverage of the network will also be the max-
imized. Due to their symmetry, tiling graphs possess some
other desirable properties (Table 1).
6 Deployment of NET graphs
NET conditions can easily be incorporated into deployment
of a static network by mobile agents. The agent can incre-
mentally add nodes by picking the best location of the next
node based on the geometry of the existing network. We can
also incorporate these rules into self-deployment algorithms
for mobile nodes based on potential fields.
In a typical mobile deployment scenario, all nodes start
in positions close to each other so that the initial network is
highly well connected and the NET condition is satisfied. Ev-
ery node applies a force of Frepel + Fattract on each its
neighbors. As a result of these forces, each pair of nodes
moves as far away from each other as possible without los-
ing connectivity with each other. After every time step, each
node evaluates the contribution of each of its edges towards
satisfying the NET condition. An edge that is not necessary
to satisfy the condition of either of the nodes it connects is
broken by turning off Fattract . As a result of these forces the
nodes spread out as far as possible till no more edges can be
broken.
Fig. 7(a) shows that the distributed algorithm has a lower
coverage but greater connectivity as compared to the incre-
mental approach. The edge connectivity is greater than b 2π θ c
which is in agreement with our conjecture.
For θ = 2π 3 , the communication graph contains the RNG
(fig. 8). In contrast to the incremental approach, we observe
that there are no edges at the boundary that are present in the
RNG and are not contained in the communication graph. This
is because more boundary nodes satisfy the NET condition
29
(a) t = 0 (b) t = HT1 (c)
(d) t = HT2 (e) t = HT3 (f) t = HT4
Figure 9: Illustration for Coalescence for N = 4. The blue
cone represents the base station, red circles represent robots
and the grey discs are the communication areas. The dotted
circles show robot positions in the previous time step. As
robots join the base station’s network, the area of the con-
nected component grows and the robots that are still search-
ing have a better chance of hitting it.
compared to the incremental approach. Even in the presence
of boundary nodes in real networks, these results validate the
assertion in Theorem 1 that the communication graph will
contain the RNG for θ ≤ 2π
3 .
7 Coalescence
For self deployment of mobile nodes, maintaining connec-
tivity (and other network constraints) is the key challenge.
In the above algorithms, this was achieved by starting out
with a network that satisfied the constraint and only allowing
constraint-preserving steps. If at some point during the de-
ployment, the constraint is broken (because of reasons such
as node failure), it is difficult for the network to recover. One
repair mechanism could be that the nodes stop the deploy-
ment algorithm as soon as connectivity is broken and start
searching for disconnected nodes. Once every one is con-
nected, the deployment restarts. Coalescence is the problem
of isolated robots independently searching for each other and
forming larger connected components.
We are interested in understanding the time taken for nodes
to regain connectivity. As an extreme case, we consider N
disconnected nodes searching for other nodes and forming a
single connected component. We assume that there exists a
single ”special” node called the base station that is known to
all nodes. Nodes perform independent random walks and stop
their search as soon as they are connected to the base station
(fig. 9).
Let HT1 ≤ HT2 ≤ · · · ≤ HTN be the 1st , 2nd , · · · , N th
hitting times i.e., the times at which the 1st , 2nd , · · · , N th
nodes “hit” or join the connected component of the base
station. As nodes join the base station, the communication
spread of the connected component increases and as a result
the probability of a searching node finding the component in-
creases. It can be shown that the time for coalescence de-
creases as N1 with the number of nodes N .
 (a)
Figure 7: Performance of the Incremental and Distributed deployment Algorithms for a deployment of 100 nodes in terms of
(a) Coverage and (b) Edge Connectivity
Student Version of MATLAB
(a) (b)
Figure 8: Comparison of communication graph and corresponding RNG[Distributed Deployment, 100 nodes, θ = 2π
communication graph b) RNG c) Dashed lines indicate edges in RNG that are not in communication graph. Dark lines indicate
edges in communication graph that are not in RNG.
8 Conclusions
This paper presents an extensive survey of research on Cover-
age optimization in Sensor Networks. This problem is related
to several problems in computational geometry like facility-
location, art-gallery problem, packing and filling, etc. We
argue that several sensor network deployments will involve
careful placement of nodes as against random high density
deployments. For such networks we show that efficient net-
works can be constructed by exploiting placement control.
We present our recent results on 3 fronts - 1) Self-deployment
of mobile sensor networks 2) Local conditions to guarantee
global coverage and connectivity 3) analysis of coalescence
for mobile networks.
References
[Ando et al., 1999] H. Ando, T. Oasa, J. Suzuki, and M. Ya-
mashita. Distributed memoryless point convergence al-
30
(b)
Student Version of MATLAB Student Version of MATLAB
(c)
3 ] a)
gorithm for mobile robots with limited visibility. IEEE
Transactions on Robotics and Automation, pages 818–
828, Oct. 1999.
[Batalin and Sukhatme, 2004] M. Batalin and G. Sukhatme.
Coverage, exploration and deployment by a mobile robot
and communication network. In Telecommunication Sys-
tems Journal, Special Issue on Wireless Sensor Networks,
26(2):181–196, 2004.
[Brendin et al., 2005] J. Brendin, E. Demaine, M. T. Haji-
aghayi, and D. Rus. Deploying sensor nets with guaran-
teed capacity and fault tolerance. In ACM Intl. sympo-
sium on Mobile ad hoc networking and computing, Mobi-
Hoc’05, Urbana-Champaign, IL, May 2005. to appear.
[Cerpa et al., 2005] Alberto Cerpa, Jennifer L. Wong,
Louane Kuang, Miodrag Potkonjak, and Deborah Estrin.
 Statistical model of lossy links in wireless sensor net-
works. In IPSN, 2005.
[Corke et al., 2004] P. I. Corke, S. E. Hrabar, R. Peterson,
D. Rus, S. Saripalli, and G. S. Sukhatme. Autonomous
deployment and repair of a sensor network using an un-
manned aerial vehicle. In IEEE Intl. Conf. on Robotics
and Automation, pages 3602–3609, Apr 2004.
[Cortés et al., 2004] J. Cortés, S. Martı́nez, T. Karatas, and
F. Bullo. Coverage control for mobile sensing net-
works. IEEE Transactions on Robotics and Automation,
20(2):243–255, April 2004.
[Cortes et al., in submission] J. Cortes, S. Martinez, and
F. Bullo. Robust rendezvous for mobile autonomous
agents via proximity graphs in d dimensions. IEEE Trans-
actions on Automatic Control, in submission.
[Gage, 1992] D. W. Gage. Command control of many-robot
systems. In AUVS, pages 28–34, Huntsville, AL, 1992.
[Gerkey et al., 2002] B. Gerkey, R. T. Vaughan, A. Howard,
G Sukhatme, and M. J. Mataric. Most valuable player: A
robot device server for distributed control. In IEEE/RSJ
Intl. Conf. on Intelligent Robots and Systems, IROS’02,
pages 299–308, 2002.
[Gupta and Kumar, 2003] P. Gupta and P. R. Kumar. To-
wards an information theory of large networks: An achiev-
able rate region. IEEE Transactions on Information The-
ory, 49(8):1877–1894, Aug. 2003.
[Howard et al., 2002a] A. Howard, M. J. Mataric, and G. S.
Sukhatme. An incremental self-deployment algorithm for
mobile sensor networks. Autonomous Robots, Special
Issue on Intelligent Embedded Systems, 13(2):113–126,
2002.
[Howard et al., 2002b] A. Howard, M. J. Mataric, and G. S.
Sukhatme. Mobile sensor network deployment using po-
tential fields: A distributed, scalable solution to the area
coverage problem. In Distributed Autonomous Robotic
Systems, pages 299–308, 2002.
[Kaiser et al., 2004] W. Kaiser, G. Pottie, M. Srivastava,
G .S. Sukhatme, J. Villasenor, and D. Estrin. Networked
infomechanical systems (nims) for ambient intelligence.
Ambient Intelligence, Springer-Verlag, 2004.
[Kazazakis and Argyros, 2002] G. D. Kazazakis and A. A.
Argyros. Fast positioning of limited-visibility guards for
the inspection of 2d workspaces. In IEEE International
Conference on Intelligent Robots and Systems, 2002.
[Li, 2003] X. Li. Topology control in wireless ad hoc net-
works. Book Chapter in ”Ad Hoc Wireless Networking”
Kluwer, edited by XiuZhen Cheng, Xiao Huang and Ding-
Zhu Du, 2003.
[Lin et al., 2003] J. Lin, A. S. Morse, and B. D. O. Ander-
son. The multi-agent rendezvous problem. In Interna-
tional Conference on Decision and Control, pages 1508–
1513, 2003.
[Meguerdichian et al., 2001] S. Meguerdichian, F. Koushan-
far, M. Potkonjak, and M.B. Srivastava. Coverage prob-
31
lems in wireless ad-hoc sensor networks. In IEEE Info-
com, 2001.
[Okabe and Suzuki, 1997] A. Okabe and A. Suzuki. Loca-
tional optimization problems solved through voronoi dia-
grams. Eur. Journal of Operationa Research, 98(3):445–
456, 1997.
[O’Rourke, 1987] J. O’Rourke. Art Gallery Theorems and
Algorithms. Oxford University Press, 1987.
[Poduri and Sukhatme, 2004] S. Poduri and G. Sukhatme.
Constrained coverage for mobile sensor networks. In IEEE
Intl. Conf. on Robotics and Automation, pages 165–172,
May 2004.
[Poduri and Sukhatme, 2007] S. Poduri and G. S. Sukhatme.
Latency analysis of coalescence in robot groups. 2007.
[Poduri et al., 2005] Sameera Poduri, Sundeep Pattem,
Bhaskar Krishnamachari, and Gaurav S. Sukhatme. A
unifying framework for tunable topology control in sensor
networks. Technical report, University of Southern
California, Center for Robotics and Embedded Systems
Technical Report, CRES-05-004, 2005.
[Roy and Dudek, 2001] N. Roy and G. Dudek. Collabora-
tive exploration and rendezvous: Algorithms, performance
bounds and observations. Autonomous Robots, 11(2):117–
136, Sept. 2001.
[Santi, 2003] P. Santi. Topology control in wireless ad hoc
and sensor networks. Technical Report IIT-TR-04/2003,
Institute for Informatics and Telematics, Pisa, Italy, 2003.
[Spanos and Murray, 2004] D. Spanos and R. M. Murray.
Robust connectivity of networked vehicles. In Interna-
tional Conference on Decision and Control, pages 2893–
2898, 2004.
[Wang et al., 2003a] X. Wang, G. Xing, Y. Zhang, C. Lu,
R. Pless, and C. Gill. Integrated coverage and connec-
tivity configuration in wireless sensor networks. In First
ACM Conf. on Embedded Networked Sensor Systems, Sen-
sys’03, Nov 2003.
[Wang et al., 2003b] X. Wang, G. Xing, Y. Zhang, C. Lu,
R. Pless, and C. Gill. Integrated coverage and connec-
tivity configuration in wireless sensor networks. In First
ACM Conf. on Embedded Networked Sensor Systems, Sen-
sys’03, Nov 2003.
[Zavlanos and Pappas, 2005] M. M. Zavlanos and G. J. Pap-
pas. Controlling connectivity of dynamic graphs. In In-
ternational Conference on Decision and Control, pages
6388–6393, 2005.
[Zhang and Hou, 2005] H. Zhang and J. C. Hou. Maintain-
ing coverage and connectivity in large sensor networks.
The Wireless Ad Hoc and Sensor Networks: An Intl. Jour-
nal, 2005.
[Zuniga and Krishnamachari, 2004] M. Zuniga and B. Kr-
ishnamachari. Analyzing the transitional region in low
power wireless links. In First IEEE Intl. Conf. on Sen-
sor and Ad hoc Communications and Networks (SECON),
Santa Clara, CA, Oct 2004.
 Glowworms-inspired multi-robot system for multiple source localization tasks
P. Amruth, K.N. Krishnanand, and D. Ghose
Guidance, Control, and Decision Systems Laboratory,
Department of Aerospace Engineering,
Indian Institute of Science, Bangalore.
Abstract
We address the problem of multiple source lo-
calization where a collection of mobile robots is
used to localize multiple radiation sources such as
nuclear/hazardous aerosol spills, origins of a fire
calamity, and hostile transmitters/sensors scattered
on a spatial landscape. Our approach consists of a
distributed algorithm based on a glowworm swarm
optimization (GSO) technique that finds solutions
to multimodal function optimization problems. We
present a multi-robot system - a set of four wheeled
robots called glowworms - that can be used to per-
form multiple source localization tasks. Implemen-
tation of the glowworm algorithm requires each
robot, during each cycle, to perform tasks such
as computation of local-decision domain, identi-
fication and localization of neighbors, and selec-
tion of a leader among the current neighbors dur-
ing each iteration. Preliminary experiments in this
regard validate each robot’s capability to perform
the above basic algorithmic primitives. Finally, we
describe the results of a sound localization experi-
ment with a set of four glowworms.
1 Introduction
Localization of multiple emission sources using networked
mobile robots has received some attention recently [Cui et
al., 2004] in the collective robotics community. In particular,
the goal of multiple source localization is to use a collection
of mobile robots to localize multiple sources of a general nu-
trient profile that is distributed spatially on a two dimensional
landscape. The problem is representative of a wide variety
of applications that include detection of multiple radiating
sources such as nuclear/hazardous aerosol spills, origins of
a fire calamity, and hostile sensors/transmitters that are scat-
tered over a region. We assume that the nature of the nutri-
ent profile is such that the concentration of the nutrient peaks
at the source location and monotonically decreases with dis-
tance from the source. We also assume that the concentra-
tion levels at different source locations need not necessarily
be equal. The above problem is non-trivial for a couple of
reasons that can be summarized as follows:
• Non availability of a priori information.
32
• Interference between sources: A source may subsume
the influence of another source whose concentration
level is relatively small.
• Environment infested with forbidden regions/obstacles.
• Lack of gradient information.
• Complex source placements.
• Discontinuities in the nutrient profile.
• Noisy measurements.
Different aspects of the same problem belong to topics
such as multi-agent rendezvous, multimodal function opti-
mization, and bio-inspired algorithms.
1.1 Multi-agent Rendezvous
The theoretical aspects of the problem are related to problem
of consensus (agreement) in multi-agent networks [Olfati-
Saber and Murray, 2004] where agents transit from an ini-
tially random state to a final steady state where they agree
upon their individual state values. The state could be ei-
ther position, heading angle, frequency of oscillation, etc.
The agreed upon quantity is usually called the group deci-
sion value. Multi-agent rendezvous [Ando et al., 1999] is
a specific instance of the consensus problem where the goal
is to design control laws that enable a group of randomly de-
ployed mobile agents to steer towards, and meet at, a common
destination. Multiple source localization is a variation of the
above problem as it involves co-location of agents at multiple
positions instead of convergence to a single location. Kaipa
and Ghose [2006b] report the theoretical foundations of the
above problem where they address the rendezvous of visibil-
ity limited mobile-agents at multiple locations.
1.2 Multimodal Function Optimization
Multimodal function optimization has been addressed ex-
tensively in the recent literature [Parsopoulos and Vrahatis,
2004], [Kaipa and Ghose, 2006a]. Most prior work on this
topic focussed on developing algorithms to find either the
global optimum or all the global optima of the given multi-
modal function, while avoiding local optima. However, there
is another class of optimization problems which is differ-
ent from the problem of finding only the global optimum
of a multimodal function. The objective of this class of
multimodal optimization problems is to find multiple op-
tima having either equal or unequal function values [Kaipa
and Ghose, 2006a]. In particular, we are interested in de-
veloping an algorithm which, while serving for numerical
multimodal function optimization on one hand, could be di-
rectly used in a realistic collective robotics task of simulta-
neously localizing multiple sources of interest like nuclear
spills, aerosol/hazardous chemical leaks, and fire-origins in
a fire calamity. For instance, several forest fires at different
locations give rise to a temperature profile that peaks at the
locations of the fire. Similar phenomenon can be observed in
nuclear radiations and electromagnetic radiations from signal
sources. In all the above situations, there is an imperative
need to simultaneously identify and neutralize all the multi-
ple sources using a swarm of robots before they cause a great
loss to the environment and people in the vicinity.
1.3 Bio-inspired Algorithms
Population-based algorithms could be broadly divided into
two categories: evolutionary computation (EC) techniques
[Goldberg and Richardson, 1987] that are based on evolution-
ary mechanisms encountered in natural selection and swarm
intelligence (SI) methods [Dorigo and Stutzle, 2004] that are
inspired by flocking mechanisms found in biological swarms
like ants, termites, bees, wasps, and bacteria. The intelligent
group behavior exhibited by these social species is a result
of actions performed by relatively simple individuals that are
solely based on neighbor-interactions and local information
from the environment inhabited by the agent-collective. Usu-
ally, the emergent behavior is utilized in an appropriate man-
ner in order to perform certain complex tasks. The above
behavioral metaphor offers an insight into the basis to devise
distributed algorithms that solve complex problems related
to diverse fields such as optimization, multi-agent decision
making, and collective robotics. Recent literature abounds
with examples of such biomimetic algorithms including ant
colony optimization (ACO) techniques, particle swarm opti-
mization (PSO) algorithms, and several swarm based collec-
tive robotic algorithms. While the application of evolution-
ary approaches is largely limited to numerical optimization
problems, the particle-nature of individuals in swarm based
optimization algorithms enable their application to realistic
collective robotics tasks.
In this paper, we present a multi-robot system that imple-
ments a distributed algorithm to achieve multiple source lo-
calization tasks. The algorithm is based on a novel glow-
worm swarm optimization technique (GSO) originally devel-
oped by Kaipa and Ghose [2006a], to find solutions to mul-
timodal function optimization problems. The GSO algorithm
[Kaipa and Ghose, 2006a] is in the same spirit as the ACO
[Dorigo and Stutzle, 2004] techniques but is different in many
aspects that help in achieving simultaneous detection of mul-
tiple local optima of multimodal functions. Generally, ACO
techniques are used for locating global optima. However, the
objective of GSO is to locate as many of the peaks as possi-
ble. In the GSO algorithm, the agents are initially deployed
randomly in the objective function space. The agents carry
a luminescence quantity called luciferin along with them and
are thought of as glowworms that emit a light whose intensity
33
is proportional to the associated luciferin. Each glowworm
uses the luciferin to (indirectly) communicate the function-
profile information at its current location to the neighbors.
The glowworms depend on a adaptive local-decision domain,
which is bounded above by a circular sensor range, to com-
pute their movements. Each glowworm selects a neighbor
that has a luciferin value more than its own, using a prob-
abilistic mechanism, and moves towards it. That is, they
are attracted to neighbors that glow brighter. These move-
ments, that are based only on local information, enable the
glowworms to split into subgroups and exhibit a simultane-
ous taxis-behavior towards the optimum locations leading to
the capture of multiple optima of the given objective function.
Concepts underlying the basic GSO algorithm and prelimi-
nary real-robot implementation results can be found in [Kaipa
and Ghose, 2005] and [Kaipa et al., 2006], respectively. This
paper presents crucial modifications such as:
1. The neighbor-selection probability is made a function of
relative luciferin (luciferin-difference), instead of abso-
lute luciferin values.
2. The adaptive local-decision update rule is modified with
improvements in performance by a factor of approxi-
mately two.
3. The robots used in this paper use a circular ring of 16 in-
frared transmitters, as opposed to a combination of four
vertical transmitters and a conical reflector [Kaipa et al.,
2006], to achieve a near-circular pattern of illumination
around the robot.
We present results of preliminary experiments that are con-
ducted to test each robot’s capability to implement the various
subtasks of the algorithm. Later, we demonstrate a real-robot-
implementation of the glowworm algorithm in the context of
a sound localization task.
2 Glowworm Algorithm
The GSO algorithm starts by placing the glowworms ran-
domly in the workspace so that they are well dispersed. Ini-
tially, all the glowworms contain an equal quantity of lu-
ciferin (`j (0) = `0 , ∀ j). Each iteration consists of a
luciferin-update phase followed by a movement-phase based
on a transition rule [Kaipa and Ghose, 2006a].
Luciferin-update phase: The luciferin update rule is given by:
`j (t + 1) = (1 − ρ)`j (t) + γJj (t + 1) (1)
where, ρ is the luciferin decay constant (0 < ρ < 1) and γ is
the luciferin enhancement constant and Jj (t) represents the
value of the objective function at agent j’s location at time t.
While the first term in (1) represents the decaying nature
of the luciferin that indirectly allows the agents to escape in-
ferior regions and move towards promising regions of the ob-
jective functions space, the second term represents the fitness
of the agent j’s location at time t + 1.
Movement-phase: During the movement-phase, each glow-
worm decides, using a probabilistic mechanism, to move to-
wards a neighbor that has a luciferin value more than its own.
 3 3
2 2
1 1
0 0
Y
Y
−1 −1
−2 −2
−3
−3 −2 −1 0 1 2 3
−3
−3 −2 −1 0 1
X X
(a)
Figure 1: Emergence of solution after 200 iterations for the
following cases: a) The decision range is kept constant with
rdi = 2 (only one peak is captured). d) Decision range is made
adaptive according to (4), with rdi (0) = 3 (all three peaks are
captured).
For each glowworm i, the probability of moving towards a
neighbor j is given by:
(`j (t) − `i (t))
pj (t) = P
k∈Ni (t) (`k (t) − `i (t))
where, j ∈ Ni (t), Ni (t) = {j : di,j (t) < rdi (t); `i (t) <
`j (t)}, t is the time (or step) index, di,j (t) is the euclidian
distance between glowworms i and j at time t, `j (t) is the lu-
ciferin level associated with glowworm j at time t, rdi (t) is
the variable local-decision range associated with glowworm i
at time t, and rs is the radial range of the luciferin sensor. Let
the glowworm i select a glowworm j ∈ Ni (t) with pj (t)
given by (2). Then the discrete-time model of the glowworm
movements can be stated as:
µ ¶
xj (t) − xi (t)
xi (t + 1) = xi (t) + s
kxj (t) − xi (t)k
where s is the step size.
Local-decision range update rule: Since we assume that a
priori information about the objective function is not avail-
able, in order to detect multiple peaks, the sensor range must
be made a varying parameter. For this purpose, we associate
each agent i with a local-decision domain whose radial range
rdi is dynamic in nature (0 < rdi ≤ rsi ). A suitable function is
chosen to adaptively update the local-decision domain range
of each glowworm. This is given by:
rdi (t + 1) = min{rs , max{0, rdi (t) + β(nt − |Ni (t)|)}} (4)
where, β is a constant parameter and nt is used as a thresh-
old parameter to control the number of neighbors. The
advantage of using an adaptive local-decision domain, in-
stead of a constant decision domain for capture of multiple
peaks, is illustrated in the simulation example shown in Fig-
ure 1. The following multimodal function, which has three
peaks at locations (−0.0093, 1.5814), (1.2857, −0.0048),
and (−0.46, −0.6292), is chosen for the purpose:
J(x, y) = 3(1 − x)2 exp(−(x2 ) − (y + 1)2 )
− 10(x/5 − x3 − y 5 ) exp(−x2 − y 2 )
− (1/3) exp(−(x + 1)2 − y 2 )
Luciferin receiver
Sweep platform
Luciferin transmitter
Intensity sensor
Infrared transmitter
2 3
(emission in a
near circular region)
(b)
Sound pick-up sensor
Differential mobile
platform
Figure 2: Wheeled-mobile robot platform used for multiple
source localization experiments
(2)
A set of 50 glowworms are randomly deployed in a two-
dimensional workspace of size 6X6 square units. When the
decision-range is kept constant (rdi = 2), only the global
peak is captured (Figure 1(a)). Even though not shown, for
rdi = 1.8 and 1.5, two and three peaks are captured, respec-
tively. However, when the decision-range is made adaptive,
all the peaks are captured (Figure 1(b)) irrespective of the
chosen value of rdi (0).
3 Glowworms
(3) Four small-wheeled robots christened Glowworms (named
after the glowworm algorithm) were built to conduct our ex-
periments. Implementation of each sensing-decision-action
cycle of the GSO algorithm requires the glowworms to per-
form subtasks such as relative localization of neighbors, se-
lection of a leader among current neighbors, updating of the
associated luciferin and local-decision range, and making a
step-movement towards the selected leader. The various hard-
ware modules of each glowworm that are used to achieve the
above tasks are shown in Figure 2.
3.1 Relative Localization of Neighbors
A circular array of sixteen infrared transmitters placed ra-
dially outward is used as the glowworm’s beacon to ob-
tain a near circular emission pattern around the robot. A
photodiode-pair mounted on a rotary platform perform a 180o
sweep and records the intensity of the received infrared light
as a function of the angle made by the direction of the source
with the heading direction of a Glowworm. Local peaks in the
intensity pattern indicate the presence and location of other
glowworms. The received intensity of infrared light follows
a inverse square law with respect to distance which is used to
compute range information to others. The angular position of
the peak is approximated as the relative angle to a neighbor.
(5) Even though the Glowworm locates all others within the per-
34
 3.5
90
Case 1 50
120 60

Standard deviation of intensity ( σ (θ) )

40
3

I
30
150 30
Case 2 20

10 2.5

180 0

210 330
1.5

240 300
270
1
0 50 100 150 200 250 300 350
θ (degrees)

Figure 3: Polar plot of the intensity pattern for the two cases Figure 5: Plot of standard deviation of received intensity as a
of ambient light conditions function of angle for the second case

3.5 90
200
120 60
Standard deviation of intensity ( σI (θ) )

150
3
150 100 30

50
2.5

180 0

210 330
1.5

240 300
270
1
0 50 100 150 200 250 300 350
θ (degrees)

Figure 4: Plot of standard deviation of received intensity as a Figure 6: Intensity pattern of a glowworm when a single
function of angle for the first case neighbor is placed in its vicinity. Ambient light conditions
of case 2 are considered.
ception range of the distance-sensor (excepting those that are
eclipsed by other glowworms), it identifies them as neighbors bor. While the peak intensity is used to approximate distance
only when they are located within its current variable local- to the neighbor, the angular position of the peak gives the
decision domain. angle to the neighbor. Figures 7 and 8 show the intensity pat-
Ambient light is a major source that alters the received in- terns when a glowworm consists of two, three, four, and five
tensity pattern of a glowworm from the infrared beacon rings neighbors, respectively. We observe that detection of neigh-
of its neighbors. Therefore, we initially characterize the in- bors becomes difficult when the number of neighbors is more
tensity pattern of a glowworm in the absence of neighbors and than six as the situation leads to merging of neighboring lobes
presence of only ambient daylight. A set of 30 experimental in the intensity pattern. However, the above limitation doesn’t
trials is carried out and the average behavior of the intensity affect the performance of the algorithm since the parameter
pattern is shown as a polar plot in Figure 3 (case 1). Figure nt (number of threshold neighbors) is always chosen as less
4 shows the standard deviation of intensity as a function of than five in number.
angle. The average intensity and standard deviation averaged
over all angles are 32.94 and 1.97, respectively. Later, we 3.2 Luciferin glow/Reception
minimize the amount of ambient light by using black walls The glow consists of an infrared light modulated by an 8-
around the boundary of the workspace and by conducting the bit serial binary signal that is proportional to the glowworms
same experiment during night conditions (Figure 3, case 2). luciferin value at the current sensing-decision-action cycle.
Note that there is a substantial reduction in the intensities in Four emitters that are mounted symmetrically about the glow-
all directions (Iav (case2) = 3.56 ≈ 0.11 × Iav (case1)). worms central axis are use to broadcast luciferin data in the
Therefore, all further experiments are conducted under ambi- Glowworms neighborhood. Two infrared receivers mounted
ent light conditions of case 2. Figure 5 shows the standard de- on a sweep-platform are used as luciferin receptors. In order
viation of intensity as a function of angle for case 2. Note that to avoid problems due to interference between data signals
the average standard deviation is 1.97, which is the same as coming from different neighbors, the receiver sweeps and
that of case 1. Figure 6 shows the intensity pattern of a glow- aligns along the line-of-sight with a neighbor before reading
worm when a single neighbor is placed in its vicinity. The the luciferin data transmitted by it. Using the above scheme,
presence of a single peak in the intensity pattern enables the a minimum threshold separation of 10 cm between neighbors
glowworm to detect the condition that it has a single neigh- was observed to be sufficient in order to distinguish data com-

35
 90 90 150
200
120 60 120 60
150
100
A B
150 100 30 150 30
50
50

180 0 180 0

C G2
210 330 210 330
G1 A
240 300 240 300
270 270

(a) (b)

Figure 7: Intensity pattern of a glowworm when two and three

neighbors is placed in its vicinity. Ambient light conditions
of case 2 are considered.
Figure 9: A preliminary experiment to demonstrate the
120
90 150
60 120
90 150
60
sensing-decision-action cycle of a glowworm
100 100
150 30 150 30
50 50
con A in finite number of step-movements.
180 0 180 0

4.2 Sound Source Localization

210 330 210 330
In this experiment, the glowworms localize a sound source
240 300 240 300 which is a loud speaker activated by a square wave signal of
frequency 28 Hz. A microphone based sound sensor enables
270 270

each Glowworm to measure the intensity of sound at its cur-

(a)
Figure 8: Intensity pattern of a glowworm when four and five
neighbors is placed in its vicinity. Ambient light conditions
of case 2 are considered.
ing from different glowworm neighbors.
4 Robot Experiments
4.1 Sensing-action-decision Cycle
We describe a preliminary experiment which demonstrates
the basic behavioral primitives used by a Glowworm to taxis
towards source locations. A set of three static beacons (la-
belled as A, B, and C in Figure 9) and two glowworms (la-
belled as G1 and G2 ) are used for this experiment. The pur-
pose of a static beacon is to emulate the presence of a neigh-
bor that emits a luciferin value associated with its location
in a near circular region. The static beacons A, B, and C
emit luciferin values of 120, 48, and 80, respectively. The
glowworms G1 and G2 are associated with constant luciferin
values of 90 and 32, respectively. For simplicity, we use a
maximum-neighbor selection rule in which a Glowworm se-
lects to move towards a neighbor that emits maximum lu-
ciferin. The hierarchy of agents (beacons/glowworms) with
respect to their luciferin values is given by `G2 < `B < `C <
`G1 < `A . Since beacon A emits the maximum luciferin
and G1 is next in the luciferin-hierarchy, it makes determin-
istic movements towards the beacon A. At initial placement,
the glowworm G2 has G1 , B, and C as neighbors within its
decision-range. The luciferin value associated with G2 is less
than that of static beacons B and C. However, it selects to
move towards G1 , which is its neighbor with maximum lu-
ciferin. Therefore, both the glowworms reach the static bea-
(b) rent location. We place a Glowworm (A) near the source and
a dummy Glowworm (B) away from the source which is kept
stationary but made to emit luciferin proportional to the inten-
sity measurement at its location. Since A is already located at
the source, it doesn’t get a direction to move. Therefore, it re-
mains stationary at the source location. Initially, since B is in
the vicinity of C (while A is not), it moves towards B. How-
ever, as it reaches closer to B it senses A and hence, changes
direction in order to move towards A. Since D is closer to A,
it makes deterministic movements towards A at every step. In
this manner, the glowworms localize the sound source even-
tually. Snapshots from a video of the above experiment are
shown in Figure 10.
A dummy glowworm was placed at the source location
only for the purpose of demonstration and is not a neces-
sary requirement for the algorithm working. According to
the glowworm algorithm, a glowworm with the maximum lu-
ciferin at a particular iteration remains stationary during that
iteration. Ideally, the above property leads to a dead-lock situ-
ation when all the glowworms are located such that the peak-
location lies outside the convex-hull formed by the glowworm
positions. Since the agent movements are restricted to the
interior region of the convex-hull, all the glowworms con-
verge to a glowworm that attains maximum luciferin value
during its movements within the convex-hull. As a result, all
the glowworms get co-located away from the peak-location.
However, the discrete nature of the movement-update rule
automatically takes care of this problem which could be de-
scribed in the following way. During the movement phase,
each glowworm moves a distance of finite step-size s towards
a neighbor. Hence, when a glowworm i approaches closer to
a glowworm j that is located nearest to a peak such that the
inter-agent distance becomes less than s, i crosses the posi-
tion of j and becomes a leader to j. In the next iteration, i re-

36

D A
Sound source
C B t=0 t=7
t = 10 t = 21
t = 28
t = 32
t = 36 t = 41
Figure 10: Demonstration of a sound-source localization task
mains stationary and j crosses the position of i thus regaining
its leadership. This process of interchanging of roles between
i and j repeats until they reach the peak. This phenomenon
was supported by simulations in [Kaipa and Ghose, 2006a].
However, since real robots cannot move over each other, the
robot’s behavior should be modified such that it emulates the
above phenomenon when it encounters another robot.
Real-robot-experiments are in progress and we will report
more results in the final version of the paper.
5 Concluding Remarks
We describe a multi-robot system that consists of four
wheeled-robots called glowworms that have the necessary
hardware potential to conduct multiple source localization
experiments. As inexpensive sensors are used for neighbor
localization and communication tasks there is a need to con-
duct the experiments with sufficient care in order to improve
the quality of the experimental results. For instance, we have
shown through experiments that, in spite of using cheap in-
frared based sensors, we are able to achieve good neighbor
localization by proper conditioning of the ambient light con-
ditions. The preliminary sound localization experiment vali-
dates the suitability of the glowworm algorithm and the multi-
robot system for source localization tasks. Ongoing work in-
cludes construction of twenty six miniature-sized glowworms
and implementation of localization tasks with multiple sound
sources.
37
References
[Ando et al., 1999] H Ando, Y Oasa, I Suzuki, and M Ya-
mashita. Distributed memoryless point convergence al-
gorithm for mobile robots with limited visibility. IEEE
Transactions on Robotics and Automation, 15:818–828,
1999.
[Cui et al., 2004] X Cui, C T. Hardin, R K. Ragade, and A S.
Elmaghraby. A swarm approach for emission sources lo-
calization. In Proceedings of 16th International Confer-
ence on Tools with Artificial Intelligence, pages 424–430,
Boca Raton, Florida, November 2004.
[Dorigo and Stutzle, 2004] Marco Dorigo and T Stutzle. Ant
Colony Optimization. MIT Press, Cambridge, Mas-
sachusetts, 2004.
[Goldberg and Richardson, 1987] D E Goldberg and
J Richardson. Genetic algorithms with sharing for multi-
modal function optimization. In Genetic Algorithms and
their Applications ICCGA87, pages 41–49, 1987.
[Kaipa and Ghose, 2005] Krishnanand N. Kaipa and Deba-
sish Ghose. Detection of multiple source locations us-
ing a glowworm metaphor with applications to collective
robotics. In Proceedings of IEEE Swarm Intelligence Sym-
posium, pages 84–91, Pasadena, CA, June 2005.
[Kaipa and Ghose, 2006a] Krishnanand N. Kaipa and Deba-
sish Ghose. Glowworm swarm based optimization algo-
rithm for multimodal functions with collective robotics
applications. Multi-agent and Grid Systems, 2:to appear,
2006.
[Kaipa and Ghose, 2006b] Krishnanand N. Kaipa and Deba-
sish Ghose. Theoretical foundations for multiple ren-
dezvous of glowworm-inspired mobile agents with vari-
able local-decision domains. In Proceedings of American
Control Conference, pages 3588–3593, Minneapolis, Min-
nesota, June 2006.
[Kaipa et al., 2006] Krishnanand N. Kaipa, Amruth Put-
tappa, Guruprasad M. Hegde, Sharschchandra V. Bidar-
gaddi, and Debasish Ghose. Glowworm-inspired robot
swarm for simultaneous taxis towards multiple radiation
sources. In Proceedings of IEEE International Conference
on Robotics and Automation, pages 958–963, Orlando,
Florida, May 2006.
[Olfati-Saber and Murray, 2004] Reza Olfati-Saber and
Richard M. Murray. Consensus problems in networks of
agents with switching topology and time-delays. IEEE
Transactions on Automatic Control, 48:1520–1533, 2004.
[Parsopoulos and Vrahatis, 2004] K E Parsopoulos and M N
Vrahatis. On the computation of all global minimizers
through particle swarm optimization. IEEE Transactions
on Evolutionary Computation, 8:211–224, 2004.
 Efficient Communication Strategy for Cooperative Multirobot Localization∗
Valguima Odakura
Anna Helena Reali Costa
Laboratório de Técnicas Inteligentes - LTI
Depart. de Eng. de Computação e Sistemas Digitais - PCS
Escola Politécnica da Universidade de São Paulo - EPUSP
{valguima.odakura, anna.reali}@poli.usp.br
Abstract
The cooperative multirobot localization problem
consists in localizing each robot in a group within
the same environment, when robots share infor-
mation in order to improve localization accuracy.
This paper proposes a communication strategy to
be used in a cooperative multirobot localization ap-
proach aiming at reducing the amount of informa-
tion shared among robots without, degrading the
accuracy of their pose estimation. The key idea is
that a robot should only communicate when it ac-
quires useful information to provide for the other
robots in a group or when a robot ask for extra in-
formation to be able to localize itself. Experimental
results show that it is possible to reduce the amount
of data exchanged among the robots and still obtain
accurate localization.
1 Introduction
Research in multirobot systems aims at providing robots with
capabilities to perform tasks that could be dangerous, need
very accurate results or are time consuming for human be-
ings. The possibility to have a group of robots performing
tasks such as search and rescue or surveillance has motivated
developments in cooperative multirobot localization, since lo-
calization is one of the main abilities an autonomous mobile
robot should have.
Cooperation in multirobot systems can be achieved by
transferring information across multiple robots, which tackles
some issues such as: what kind of information is important to
send to other robots, how to incorporate the information re-
ceived into its own belief about its state, and when to send
this information. In the multirobot localization research area
there are some works about the first two issues, but there is not
much about the third. The information shared among robots
in a group is their pose beliefs and the location of another
robot relative to its own, and this information is integrated
into their pose beliefs by using probabilistic approaches [Fox
et al., 2000], [Roumeliotis and Bekey, 2002], [Madhavan et
al., 2002] and [Howard et al., 2003]. The best time for a
∗
This work was supported by Fapesp grant 01/14246-4, Capes
grant BEX 0803/03-0, CNPq and Capes/Grices 099/03.
38
robot to send an information is related to the time a new in-
formation is available to each robot, which means that a great
amount of information can be shared among them.
In this paper we propose a communication strategy that
aims at reducing the amount of information transferred across
robots without reducing the accuracy of their pose estimation.
The key idea is that a robot should only communicate when it
acquires useful information to provide for the other robots in
a group or when a robot ask for extra information to be able
to localize itself.
This paper is organized as follows. Section 2 presents the
basic statistical mechanisms for cooperative multirobot local-
ization. Section 3 shows some useful information that can be
shared among robots in order to improve the localization ac-
curacy. Our strategy for using less data to share among multi-
ple robots is presented in Section 4. Experimental results are
reported in Section 5. Finally, our conclusions are derived in
Section 6.
2 Cooperative Multirobot Localization
The ability to cooperate by exchanging information during
localization is particularly attractive for two main reasons: it
may be easier and less expensive to use several simple co-
operative robots than a single powerful one to accomplish a
difficult task, and it may be more efficient and faster for a
group of robots to cover vast areas than a single one [Cao et
al., 1997]. In order to accomplish tasks more quickly and ro-
bustly than a single robot, multiple robots can be used, which
poses the multirobot localization problem.
The cooperative multirobot localization problem consists
in localizing each robot in a group within the same environ-
ment, when robots can detect each other and exchange infor-
mation in order to improve their localization accuracy.
Representative recent works in cooperative multirobot lo-
calization are from Roumeliotis and Bekey (2002), and Fox et
al. (2000), and they use Kalman filter and Particle filter as lo-
calization algorithms, respectively. Probabilistic approaches
have been used with success in the localization problem.
Markov localization (ML) was initially designed for a sin-
gle robot [Fox et al., 1999], and an extension of ML that aims
at solving the multirobot problem is presented by Fox et al.
(2000).
In ML, p(xt = x) denotes the robot’s belief over its pose
x at time t, where xt is a random variable representing the
robot’s pose at time t, and x = (x, y, θ) is the pose composed
of the robot’s (x, y) position and its heading θ. This robot’s
pose belief represents a probability distribution over all the
space of xt .
ML uses two models to localize a robot: a motion model
and an observation model. The motion model is specified as
a probability distribution p(xt = x|xt−1 = x0 , at−1 ), where
xt is a random variable representing the robot’s pose at time t,
at is the action or movement executed by the robot at time t.
The movement can be estimated, for example, by odometers
on the wheels. In ML the update equation is described as:
X 2002], [Fox et al., 2000]. Every time one robot detects each
p̄(xt = x) = p(xt = x|xt−1 = x0 , at−1 )p(xt−1 = x0 ),
x0
(1)
where p̄(xt = x) is the probability density function before
incorporating sensor measurements derived from the obser-
vation of the environment at time t.
The observation model is used to incorporate data captured
by exteroceptive sensors, such as proximity sensors and cam-
eras, and it is expressed as p(xt = x|ot ), where ot is an ob-
servation sensed at time t. The update equation is described
as:
p(ot |xt = x)p̄(xt = x)
p(xt = x) = P , (2)
x0 p(ot |xt = x )p̄(xt = x )
0 0
where p(xt = x) is the probability density function after in-
corporating observations of the environment at time t.
At the beginning, p(x0 = x) is the prior belief over the
initial pose of the robot. If the initial pose is unknown,
p(x0 = x) is initialized by a uniform distribution.
In order to accommodate multirobot cooperation in ML it
is necessary to add a detection model to the previous obser-
vation and motion models. The detection model [Fox et al.,
2000] is based on the assumption that each robot is able to de-
tect and identify each other and furthermore, the robots can
communicate their probabilities distributions to other robots.
Let’s suppose that robot n detects robot m, m 6= n, and that
it measures the relative distance between them, so:
p(xnt = x) = (3)
X proximity sensor to measure the relative distance.
p̄(xnt = x) p(xnt = x|xm 0 m 0
t = x , rn )p̄(xt = x ),
x0
where xnt represents the pose of robot n, xm t represents
the pose of robot m and rn denotes Pthe measured distance
n m
between robots. The calculation x0 p(xt = x|xt = nicated by robot n, it communicates back its updated prob-
0 m 0
x , rn )p̄(xt = x ) describes the belief of robot m over the
pose of robot n. Similarly, the same detection is used to up-
date the pose of robot m.
To summarize, in the multirobot localization algorithm
each robot updates its pose belief whenever a new informa-
tion becomes available: Equation 1 is used when a motion
information is available; Equation 2 is used when an envi-
ronment observation occurs; and Equation 3 is used when a
detection happens.
Once a detection occurs, the two robots involved in the de-
tection share their probabilities distributions and relative dis-
tance, and update their pose beliefs according to the detection
model. Fox et al. (2000) included a counter that, once a robot
39
has been detected, it blocks the ability to see the same robot
again until the detecting robot has traveled a pre-defined dis-
tance. This communication significantly improves the local-
ization accuracy, if compared to a less communicative local-
ization approach.
3 Refining Localization Estimates
Cooperative multirobot localization takes advantage of robot
detection measurements by exploiting relative information
between different robots’ poses [Roumeliotis and Bekey,
other, they communicate and share their pose beliefs. Fox
et al. (2000) propose the use of the detection model to per-
form pose update only for the two meeting robots. However,
another way to improve the localization accuracy would be
by broadcasting detection information provided by the two
meeting robots to other robots in the group; and by using
negative sights, i.e., events where a robot does not see an-
other robot. Another situation occurs when a robot see more
than one robot at the same time, the multidetection situation.
In the next section these techniques to refine localization es-
timates are detailed.
3.1 Broadcasting Detection Information
The key idea in broadcasting detection information to other
robots in a group is that all of them can benefit from the
shared information derived from a single detection (when
robot n meets robot m) [Odakura and Costa, 2005].
Suppose robots m, n and k in a group. When robot n meets
robot m, if they transfer their pose estimates to robot k, the
latter can use this information to constrain its pose estimate,
by excluding the belief of being at m and n position, since
only one robot can occupy a location in the environment at
a time. Robot k can also conclude that its pose is not in a
location between the two meeting robots, otherwise, robot n
would have detected robot k instead of robot m. It is sup-
posed that the detection sensor can sense robots in front of
the detecting robot. For example, the detection sensor could
be a camera (pointing forward) to identify the robot and a
The broadcasting of the probabilities distributions of the
two meeting robots to other robots in the group can be per-
formed by the robot n, the one that detected robot m. When
robot m updates its pose based on the information commu-
ability distribution, pm . Robot n then calculates a probabil-
ity distribution that will be communicated to the non-meeting
robots:
pd (xkt = x00 ) =1 − (pn (xnt = x) + pm (xm 0
t = x )+
pnm (xnt = x, xm 0
t = x )), (4)
where pd (xkt = x00 ) is the detection information communi-
cated to the non-meeting robots, pn (xnt = x) is the probabil-
ity distribution of robot n, pm (xm 0
t = x ) is the probability
distribution of robot m and pnm (xt = x, xm
n 0
t = x ) is the
probability distribution of a robot being in a location between
the two meeting robots.
 Whenever a detection between robot n and robot m occurs,
the probability distribution function of each robot k can be
updated, with k 6= n and k 6= m, in the following way:
pd (xkt = x)p̄(xkt = x)
p(xkt = x) = P k k
. (5)
x0 pd (xt = x )p̄(xt = x )
0 0
where xk , for k = 0, · · · , n, represents all robots which do
not take part in a detection.
Figure 1 illustrates an example of this situation. In the ex-
ample there are 3 robots, R1, R2 and R3 in their actual poses.
The robots have different knowledge about their poses: R1 is
sure of its pose, R2 is completely unsure and R3 is in doubt
about 2 poses, represented as dashed cells. If R1 meets R2,
R2 becomes certain about its pose and R3 keeps its previ-
ous pose knowledge. However, if the detection information is
transferred to R3, it becomes certain about its pose, because
the possibility of being between R1 and R2 is eliminated.
The example shows the improvement in group localization
obtained if detection information is propagated to all robots
in the group.


 
R1 R2



R3
Figure 1: Example of a situation with three robots: R1 is sure
of its pose, R2 is completely uncertain and R3 is in doubt
about its pose (the two dashed cells).
3.2 Negative Information
Negative evidence is gathered when a robot fails to detect a
data that it expects to see. Most techniques for state estima-
tion update state belief by using a sensor model every time a
sensor reports a measurement. However, it is possible to get
useful state information from the absence of sensor reports
relative to expected landmark measurements.
Negative information has been applied to target tracking by
using the event of not detecting a target as evidence to update
the probability density function [Koch, 2004]. In that work,
a negative information means that the target is not located in
the visibility area of the sensor and since the target is known
to exist it is certainly outside this area.
In the robot localization domain, Hoffmann et al. (2005)
consider as negative information the absence of landmark
sensor measurements. Occlusions are identified by using a
visual sensor that scans colors of the ground to determine
if there is free area or obstacle. The environment is a soc-
cer field in green with white lines. So, if a different color is
identified, it means that an obstacle could be occluding the
visibility of a landmark.
In the cooperative multirobot localization problem negative
information can also mean the absence of detections (in the
case that a robot does not detect another one), which config-
ures a lack of group information [Odakura and Costa, 2006b].
40
In this case, the negative detection measurement can provide
the useful information that a robot is not located in the visibil-
ity area of another robot. In some cases, it can be an essential
information for improving the robot’s belief of its pose when
it is located at a part of the environment with few discriminant
landmarks.
Consider two robots within a known environment. If robot
1 does not detect robot 2 at a given time, a negative informa-
tion is reported, which states that robot 2 is not in the visibility
area of robot 1, as depicted in Figure 2(a).
(a) (b)
Figure 2: (a) Negative information. (b) Occlusion in the field
of view.
The information gathered from Figure 2(a) is true if we
consider that there are no occlusions. In order to account for
occlusions it is necessary to sense the environment to iden-
tify free areas or occupied areas. If there is a free space on
the visibility area of a detection sensor, than there is not an
occlusion. Otherwise, if it is identified as an occupied area it
means that the other robot could be occluded by another robot
or an obstacle. In this case it is possible to use geometric in-
ference to determine which part of the visibility area can be
used as negative detection information (gray area in Figure
2(b)).
Let’s suppose that robot m reports a negative detection.
The negative detection model, considering the visibility area
of the robot and the occlusion area, becomes:
m m
p(d− t |xt = x, v, obs)p(xt = x)
p(xm−
t = x) = P − m m
,
x0 p(dt |xt = x , v, obs)p(xt = x )
0 0
(6)
where d− t is the event of not detecting any robot and x m cor-
responds to the pose of robot m, the robot that reports the
negative detection information. The variables v and obs rep-
resent the visibility area and the identified obstacles, respec-
tively.
Whenever a robot m does not detect another robot k, the
probability distribution function of each robot k can be up-
dated, with k 6= m, in the following way:
p(xm− = x)p̄(xkt = x)
p(xkt = x) = P t
m− , (7)
x0 p(xt = x0 )p̄(xkt = x0 )
where xk , for k = 0, · · · , n, represents all robots which were
not detected.
The consequence of using the negative detection model is
that it can solve certain localization problems that are unsolv-
able for a group of robots that relies only on positive detection
information. A typical situation is the case that the robots are
in different rooms, in a way that one robot cannot detect each
other. In this situation, each robot can constrain its pose belief
every time a negative detection is reported.
3.3 Multidetection
A multidetection situation occurs when one robot detects two
or more robots at once or one robot sees other robot that sees
another one [Odakura and Costa, 2006a], as shown in Figure
3.
R2
R2
R1
R0
R1
R3 R0 R3
(a) (b)
Figure 3: Multidetection: (a) Robot R0 detects R1, R2 and
R3 (lines surrounding robots R1, R2 and R3 represent R0
field of view); (b) R0 detects R1, R1 detects R2, and R2
detects R3 at the same moment.
The key idea to solve the multidetection situation is that
robots more confident about their poses have more useful in-
formation to share, so they should communicate first, propa-
gating their knowledge to the rest of the group.
A measure that can be used to estimate the uncertainty of
a robot’s pose belief is the expected entropy value H. The
entropy measures the amount of information provided by the
probability density function, and is defined as [Russell and
Norvig, 1995]:
X date order is (R0, R1) and (R2, R3) and each robot updates
H =− p(xt = x) log(p(xt = x), (8)
x It is worth noting that in this case it is not necessary to update
where p(xt = x) denotes the robot’s belief that it is at pose x.
The entropy value H is zero if the robot is perfectly localized
in one pose, and the entropy value achieves maximal value
if the robot is uncertain about its pose, which means that the
probability density function is uniformly distributed.
The information gained from the meeting of two robots
is given by the maximum value of the entropy between the
robots, Hij = max(Hi , Hj ), where Hij represents the en-
tropy value of a meeting of robots Ri and Rj .
Let’s consider a group of robots taking part in a multide-
tection configuration. This configuration can be represented
as a graph G = (V, E), where V is the set of nodes and E
is the set of directed edges. Let each robot be represented
as a node, and each detection be represented as a directed
edge, linking the two meeting robots, pointing to the detected
robot. In this way, the configuration that each robot detects
all the others can be represented as a complete directed graph.
All edges (Ri , Rj ) in E are labeled by a weighted value that
corresponds to the entropy Hij .
If the graph presents bidirectional edges it means that two
robots are detecting and being detected by each other at the
same time. In this case, two types of robots should be consid-
ered: (1) homogeneous or (2) heterogeneous, regarding their
exteroceptive sensors capabilities, i.e., the error introduced in
their relative distance measurements. In the first case, we just
eliminate one direction of the bidirectional edge; in the latter,
we just keep the direction pointing to the less accurate robot,
41
i.e., the sensor reading from the most accurate sensor will be
used to update the poses beliefs of the robots.
Given a weighted graph G, the multidetection problem can
be formulated as finding a subset of edges E 0 from E, in a
way that all nodes of V are incidents on edges in E 0 , and
G0 = (V, E 0 ) has the minimum total weight.
At any moment it is possible to represent a detection con-
figuration of the robots by a graph G = (V, E). Given G, the
following steps should be performed in order to update the
robots’ poses:
1. Reduce all bidirectional edges to unidirectional ones.
2. Weight each edge (Ri , Rj ) by Hij .
3. Choose a subset of edges E 0 ⊆ E, so that G0 = (V, E 0 )
has minimum weight.
4. Sort the edges of E 0 by their minimum weight.
5. Update robots’ beliefs following the ranked edges of E 0 .
Following this procedure it is possible to find a detection
update order to deal with multidetection. As an example, con-
sider the robot configuration depicted in Figure 3(b). What is
the update order that obtains the best information gain? If we
suppose that the minimum entropy values for the meetings
are, in order, (R1, R2), (R0, R1) and (R2, R3), then, the up-
date order should be (R1, R2), (R0, R1) and (R2, R3). So,
each robot pair updates their poses beliefs (Equation 3), fol-
lowing the order presented, as shown in Figure 4(a). Now,
suppose that the minimum entropy values for the meetings
are, in order, (R0, R1), (R2, R3) and (R1, R2), then, the up-
its pose belief using Equation 3, as can be seen in Figure 4(b).
(R1, R2), once R1 and R2 are already updated in (R0, R1)
and (R2, R3).
2 nd 1st 3 rd
R0 R1 R2 R3
(a)
1st 2 nd
R0 R1 R2 R3
(b)
Figure 4: Poses beliefs update order: (a) 1st : (R1, R2), 2nd :
(R0, R1), and 3rd : (R2, R3). (b) 1st : (R0, R1) and 2nd :
(R2, R3).
4 Communication Strategy
Communication among robots in localization task is a way to
increase the robots’ perception about their states in the world.
It is achieved by using observation from other robots to better
estimate their poses. Once the aim of communication in robot
localization is to contribute to a more accurate pose estima-
tion, it is possible to use a communication strategy to increase
the localization accuracy and at the same time to diminish
the amount of information communicated and the computa-
tion time. The computation resource is related to the task of
updating the robot’s pose by using the new information re-
ceived.
The communication strategy should address questions such
as what to communicate, which robots should participate in
the communication and when to do it. The question what to
communicate refers to the content of the message, and in lo-
calization task it is the robot’s pose belief. The answer to
which robots should communicate is determined by the oc-
currence of a detection, so all the robots involved in a de-
tection should communicate. Finally, the answer to when to
communicate can be guided by a new evidence; when robots
detect each other they can start communication.
However, in some situations, a communication among the
robots do not contribute to improve their poses estimations.
For example, when two robots are certain about their poses,
this communication does not improve their poses beliefs, or
when two robots are completely lost and exchange messages,
the communication does not help them to localize themselves.
We contribute a communication strategy which is based on
the utility of a message exchange among the robots. A com-
munication between two robots is useful if it improves the
localization accuracy of at least one of the robots. Thus, the
criteria to reduce communication are related to how certain
robots are about their poses.
To reduce the amount of information communicated
among robots, and as a consequence, the processing time, it
is possible to avoid some of the communication situations.
Each robot can decide on the basis of its own knowledge if it
will take part in a communication or not.
The utility of an information to share is given by H, which
is the entropy value of a robot. If H ≤ ST then the robot con-
cludes that it could send an useful information to the group,
where ST is the sender threshold. The robot that receives a
message will accept the information received if H ≥ RT ,
where RT is the receiver threshold.
The proposed communication strategy is fully decentral-
ized once each robot can decide based on its own pose belief
if it takes part in a communication or not. If a robot receives
a message and it is certain about its pose, it will not update its
pose belief with the received information, reducing compu-
tation time, and it will not proceed with the communication
process, avoiding messages that are not necessary.
The communication strategy can be used in different situa-
tions in cooperative multirobot localization: positive or nega-
tive detection, broadcasting and multidetection, in order to re-
duce the amount of messages transmitted, defining conditions
for the information exchanges. And, as a consequence, re-
ducing the amount of computation performed by robots when
updating their poses beliefs.
In the multirobot cooperative localization when a multide-
tection situation occurs, it is represented by a graph, where
its edges guide the messages exchange among the robots. So,
it is needed that all robots represented in the graph commu-
nicate with each other in order to find the update order in the
graph. The graph can be composed in a distributed manner,
if each robot communicate to the others short messages con-
taining their entropy values, so each robot can independently
42
compose the graph and sort the edges.
Once the edges of the graph are sorted, the robots assume
the role of sender (or receiver) and can start the communica-
tion process to exchange their poses beliefs and they can ap-
ply the communication strategy to decide if they would accept
a sender or a receiver role and take part in the communication.
Communication Robots Conditions for
Situation Involved Communication
Single Detection n and m if (Hn ≤ ST ) and
(Hm ≥ RT )
Detection all if (Hn ≤ ST )
Broadcasting for all k
(k 6= n) and (k 6= m)
if (Hk ≥ RT )
Negative all if (Hm ≤ ST )
Detection for all k, (k 6= m)
if (Hk ≥ RT )
Multidetection M robots according to the robot rank
if (Hs ≤ ST ) and
(Hr ≥ RT )
Table 1: Communication strategy.
The proposed communication strategy is summarized in
the Table 1. The first column refers to the communication
situations identified, where the single detection situation cor-
responds to the detection situation described by Fox et al.
(2000). Every time one of these situations occurs, the com-
munication strategy is used to minimize the communication
requirements while obtaining as much information as possi-
ble. In the multidetection situation, M represents a group of
robots that detect each other simultaneously.
5 Experiments
We perform four experiments by using simulated robots in or-
der to evaluate the communication strategy proposed in this
paper. Each robot is equipped with a proximity sensor to mea-
sure the distance to the walls in the environment, and a detec-
tion sensor that can identify other robots and measure their
relative distance. All robot’s sensors are assumed to be cor-
rupted by Gaussian noise. The robots know the environment
model and they do not know their initial poses in the environ-
ment. The robots move in the environment and collect infor-
mation in order to improve their localization beliefs. For each
experiment several runs are performed and in each run robots
start in a different initial pose. A run finishes when all robots
achieve 0.9 of certainty about its poses. The robot environ-
ment model is a grid with 0.2 × 0.2 meters cells, and angular
resolution of 90 degrees. In the experiments the thresholds
RT and ST have the same value, denoted as EV . We tested
two values for the threshold EV , one is the maximum en-
tropy value and the other is an entropy value near zero, which
means that the robot is almost certain about its pose.
The first experiment uses eight robots in the environment of
Figure 5(a). The test environment has dimensions of 6×6 me-
ters and is a symmetric environment with four corridors, A,
B, C, and D. All robots move clockwise at the same pace, fol-
lowing the corridors. The graph in Figure 6(a) plots average
entropy values over all robots as function of steps, averaged
over eight runs for the communication situation of detection
broadcasting. Entropy values are zero if the robot is perfectly
localized in one pose, and the entropy value achieves max-
imal value if the robot is uncertain about its pose. At first
the robots do not know their poses and the expected entropy
has the maximal value. The entropy value drops as soon as
the robot updates its pose with the information reported from
the environment. Each step means an update step by the in-
corporation of some measured information (observation of
the environment, movement, detection or propagation). The
solid line represents a more communicative approach, where
EV = 1.00 is the normalized maximum entropy value, that
corresponds to the complete uncertainty about robot’s pose,
i.e., in this case the robots communicate every time a new
information is available. The dashed line represents a less
communicative approach, where EV = 0.03. The N C is the
7(a). The normalized maximum entropy value is EV = 1.00
and is compared to EV = 0.03. The reduction in N C is
from 60 to 34, without a significant loss in the localization
accuracy. The maximum difference between the entropy val-
ues for the two approaches is less then 0.1.
The last experiment uses the same environment shown in
Figure 5(c) and ten robots. The result depicted in Figure 7(b)
shows multidetection localization with ten robots, averaged
over four runs over all robots. The reduction in N C is from
43 to 36, without a significant loss in the localization accu-
racy. The maximum difference between the entropy values
for the two approaches is less then 0.2.
1
EV: 0.03 − NC: 46.0
EV: 1.00 − NC: 135.0
0.9
0.8

number of communication events that occurs during the ex- 0.7

periment. The reduction in N C is from 135 for EV = 1.00 0.6

Entropy
to 46 for EV = 0.03, without a significant loss in the lo- 0.5

calization accuracy. The maximum difference between the 0.4

entropy values for the two approaches is less then 0.1. 0.3

0.2

B 0.1

0
0 20 40 60 80 100 120 140 160 180
Number of steps
A C

(a)

D 1
EV: 0.03 − NC: 40.0
0.9 EV: 1.00 − NC: 88.0
(a) (b)
0.8

0.7

0.6
Entropy

0.5

0.4

(c) 0.3

0.2

Figure 5: Test environment (white represents free area): (a) 0.1

Hallway with four corridors denoted by A, B, C and D. (b) 0

0 20 40 60 80 100 120 140 160 180
Open environment. (c) Rooms and corridor. Number of steps

The second experiment uses the environment depicted in
Figure 5(b), which has dimensions of 4 × 4 meters. Figure
6(b) shows entropy values as function of steps, averaged over
six runs over all robots for the communication situation of
negative detection. The solid line represents a more commu-
nicative approach, where EV = 1.00 is the maximum en-
tropy value. The dashed line represents a less communicative
approach, where EV = 0.03. The reduction in the num-
ber of communications N C is from 88.0 to 40.0, without a
significant loss in the localization accuracy. The maximum
difference between the entropy values for the two approaches
is less then 0.1.
The third experiment uses the environment depicted in Fig-
ure 5(c), which has dimensions of 8 × 3.6 meters. Results for
the single-detection multirobot localization with ten robots,
averaged over four runs over all robots, are shown in Figure
(b)
Figure 6: Comparison between less and more communica-
tive approaches to: (a) Detection broadcasting. (b) Negative
detection.
6 Conclusions
In this paper we propose a unified approach defining different
communication situations, which incorporates single detec-
tion [Fox et al., 2000], broadcasting detection [Odakura and
Costa, 2005], negative detection [Odakura and Costa, 2006b],
and adds the multidetection situation. In this unified approach
we define the order the robots should communicate so that
the amount of information communicated is reduced. In this
case, the multidetection approach outperforms the approach

43
 1
[Hoffmann et al., 2005] Jan Hoffmann, Michael Spranger, Daniel
EV: 0.03 − NC: 34.0
EV: 1.00 − NC: 60.0
Gohring, and Matthias Jungel. Making use of what you dont
0.9
see: Negative information in markov localization. In IEEE/RSJ
0.8
International Conference on Intelligent Robots and Systems -
0.7 IROS’05, pages 854–859. IEEE, 2005.
0.6 [Howard et al., 2003] Andrew Howard, Maja Mataric, and Gaurav
Entropy

0.5 Sukhatme. Putting the ’i’ in ’team’: an egocentric approach to

0.4
cooperative localization. In IEEE International Conference on
Robotics and Automation - ICRA’03, pages 14–19. Kluwer Aca-
0.3
demic Publishers, 2003.
0.2
[Koch, 2004] Wolfgang Koch. On negative information in track-
0.1
ing and sensor data fusion: discussion of selected examples. In
0
0 10 20 30 40 50 60 70 80 90 100
Seventh International Conference on Information Fusion, pages
Number of steps 91–98, 2004.
(a) [Madhavan et al., 2002] Raj Madhavan, Kingsley Fregene, and
Lynne E. Parker. Distributed heterogeneous outdoor multi-robot
1 localization. In IEEE International Conference on Robotics and
0.9
EV: 0.03 − NC: 36.0
EV: 1.00 − NC: 43.0
Automation – ICRA’02. IEEE Press, 2002.
0.8
[Odakura and Costa, 2005] Valguima Odakura and Anna H. R.
Costa. Cooperative multi-robot localization: using communica-
0.7
tion to reduce localization error. In International Conference on
0.6
Informatics in Control, Automation and Robotics - ICINCO’05,
Entropy

0.5 pages 88–93, 2005.

0.4 [Odakura and Costa, 2006a] Valguima Odakura and Anna H. R.
0.3 Costa. Multidetection in multirobot cooperative localization. In
0.2
First IFAC Workshop on Multivehicle Systems - MVS’06, pages
19–24, October 2006.
0.1

0
[Odakura and Costa, 2006b] Valguima Odakura and Anna H. R.
0 10 20 30 40 50 60 70 80 90 100
Number of steps
Costa. Negative information in cooperative multirobot localiza-
tion. In Advances in Artificial Intelligence – IBERAMIA-SBIA
(b) 2006, volume LNAI 4140, pages 552–561. Springer, 2006.
[Roumeliotis and Bekey, 2002] Stergios I. Roumeliotis and
Figure 7: Comparison between less and more communicative George A. Bekey. Distributed multirobot localization. IEEE
approaches to: (a) Single detection. (b) Multidetection. Transactions on Robotics and Automation, 18(2):781–795, 2002.
[Russell and Norvig, 1995] Stuart Russell and Peter Norvig. Artifi-
cial Intelligence: A Modern Approach. Prentice-Hall, 1995.
proposed by [Fox et al., 2000] that uses a set of single de-
tections procedures when a detection involves more than two
robots. The use of the multidetection approach reduced from
N C = 60 for the approach of [Fox et al., 2000] to N C = 36.
We also propose a communication strategy that defines
conditions for the information exchanges that should be sat-
isfied aiming at reducing the amount of messages transmitted
without loss in the localization accuracy. As a result the num-
ber of communication events are reduced in all situations.
The results are very encouraging since they illustrate the
efficiency and effectiveness of the proposal.

References
[Cao et al., 1997] Yu Uny Cao, Alex S. Fukunaga, Andrew B.
Kahng, and Frank Meng. Cooperative mobile robotics: an-
tecedents and directions. Autonomous Robots, 4:1–23, 1997.
[Fox et al., 1999] Dieter Fox, Wolfram Burgard, and Sebastian
Thrun. Markov localization for mobile robots in dynamic en-
vironments. Journal of Artificial Intelligence Research, 11:391–
427, 1999.
[Fox et al., 2000] Dieter Fox, Wolfram Burgard, H. Kruppa, and
Sebastian Thrun. A probabilistic approach to collaborative multi-
robot localization. Autonomous Robots, 8(3):325–344, 2000.

44
 Active Localization of Multiple Robots by Selection of Best Frontiers
Rakesh Goyal, K Madhava Krishna and Shivudu Bhuvanagiri
IIIT Hyderabad, Gachibowli, Hyderabad – 500 032, INDIA (mkrishna@iiit.ac.in)
Abstract
We envisage a multi robotic scenario where several
robots are in ambiguity about their states and
require help of other robots to overcome their
ambiguity. In such a scenario the method presented
here moves the robots to locations where
probability of eliminating several ambiguous states
among multiple robots is a maximum. There are
two essential aspects to this problem. In the first
aspect all robots move to their best frontiers from
their current hypothesis state. The best frontiers are
those that have the highest probability of achieving
a unique hypothesis if the robots were to arrive
there. In the second aspect robots that do not have
any ambiguity about their state are dispatched to
locations where they can assist a maximum of
robots to overcome their ambiguity. Ambiguity
here is used in the sense of more than one
hypothesis of a robot’s state. The method presented
has been tested in both simulation and real-time on
robots and its efficacy verified.
1 Introduction
Localization has proven to be the most critical aspect of
mobile robot navigation. Generally indoor localization is
achieved with the aid of maps over and above dead
reckoning. However in instances where the initial state of
the robot is unknown maps can lead to conflicting
hypotheses of a robot’s state. This is a pathological problem
in global localization and is well documented [Fox et. al,
1998]. In such a case the robot moves to locations where the
configuration of obstacles enables zeroing onto a single
hypothesis from the earlier multi-hypotheses situation often
called as active localization. This paper explores the idea of
using robots apart from obstacles as hypotheses resolving
agents in such situations. Robots that are in ambiguity of
45
their states can help in resolving each others ambiguity if
they arrive at the right positions. These positions are
frontiers in the sense used in [Yamauchi, 1997]. Since the
robots are already in ambiguity about their states they can
only reach a frontier area with respect to their current
visibility in a probabilistic sense. The algorithm computes
the probability of robots reaching or occupying those
frontier positions that would lead to a unique hypothesis for
a set of robots in the first phase. Once shepherded to these
positions in a probabilistic sense some of the robots could
end up attaining a unique hypothesis. In the second phase
such localized robots are then sent to locations where they
could help in localizing as many robots as possible. In
principle a localized robot can localize all others if it is able
make a measurement on them. The algorithm alternates
between the two phases till ambiguity in states of all robots
is removed. This paper presents the theory and formulation
for only the first phase of the method evidently the more
interesting phase due to strict page limits. The second phase
is only cursorily mentioned for sake of completion.
The authors argue that the novelty of this work lies in
that it is the first such work to be reported where multiple
hypothesis states in more than one robot is resolved in an
active localization sense. The method presented here would
find utility in several multi-robotic applications where
localization issues dominate. It is useful when robots
brought alive need to estimate their initial state. It is useful
when several robots tend to get confused about their states
due to large localization errors and need to globally localize
once in a while in the map provided or built so far. It is once
again useful if one or more robots in a multi-robotic
endeavor could get kidnapped by malicious agents. While
the context mentioned here is resolving multiple hypotheses
among multiple robots this effort can also be seen in the
larger context of cooperative localization where robots also
serve the function of mobile landmarks and come together
to estimate their state whenever required.
 the robots in a partition all share the same virtual frontiers.
Frontiers of different robots are shared if they are close in an
2 Related Work euclidean sense, their distance less than a threshold. The
In general work on active localization has tended to be associated frontier set is denoted by F1, F2, …, Fn with
limited when compared with passive localization. The cardinality n fi . The base pairs are then
pioneering work in this area has been from [Burgard et. al,
1998] and [Dudek et.al, 1998]. In [Burgard et. al, 1998] a
(n f 1 , n R1 ), (n f 2 , n R 2 ),..., (n fn , n Rn ). The partitioning reduces
method of active localization based on maximum the number of combinations to be considered while moving
information gain was presented.. Dudek and others [Dudek to best frontier locations. Figure 1 shows a robot with two
et.al, 1998] presented a method for minimum distance hypotheses states, two distinct frontiers (one on top and one
traversal for localization that works in polygonal at bottom) with two virtual frontiers each.
environments without holes that they show to be NP-Hard.
A randomized version of the same method was presented in
[Dudek et.al, 2005]. In [Kaelbing et.al, 1995] an approach
for guiding the robot to a target location is proposed when
its current position is not known accurately.
As far as authors knowledge goes there has been no
reported work dealing active localization of several robots.
In the context of multi robotic localization Fox and others
extended their earlier global localization methods to a multi-
robotic setting in [Fox et.al, 2000]. A cooperative Figure1 A robot with two hypotheses states H1 and H2. The distinct
localization approach was also presented based on Monte- frontiers for each state are two one on top and other at bottom
Carlo methods in [Howard et.al, 2003]. In [Rekelitis et. al, denoted as ft1 and fb1 for state H1 at the boundary of the visibility
1997] a method is presented in which in which team polygon. There are two virtual frontiers (ft1, ft2) for every distinct
members carefully coordinate their activities in order to frontier due to two hypotheses states.
reduce cumulative odometric errors. However these
methods did not consider active localization in a rigorous 3.1 Some preliminaries
sense of guiding robots to positions. Let f ij (rij ) denote the event that a robot ri belonging to
set R j reaches the frontier f i of set F j for the base pair
3 Methodology
Consider a workspace W populated by robots, NR in (n fj , n Rj ). Then P( f ij (rij )) is the probability associated with
that event. Let f ij (rij ), f mn (rmn ) denote the intersection
number, each of them having multiple hypotheses of their
states. The problem is to move these NR robots to frontier
locations such that each of the robots localizes to a unique event that ri reaches fi and rm reaches f m .
state with minimal number of frontiers visited. One such ( )
P f ij (rij ), f mn (rmn ) denotes the probability of that event or
robot r possesses distinct and possibly virtual frontiers. The
distinct frontiers are those mentioned in [Yamauchi, 1997] the occupancy probability of the frontiers f i and f m . We
and computed from a hypothesis position of r. The distinct compute
{ } ( ) (
Pm n Rj ,1 Pm n Fj − 1, n Rj − 1 )
frontiers are denoted as f1d , f 2d ,..., f md , the superscript d
( )
P f ij (rij ) =
(
Pm n Fj , n Rj )
.......... (1)
signifying its distinctness. If r has n f hypotheses locations
of its state, each of its distinct frontiers would have Pm(x, y ) denotes the permutation of x things taken y at a
n f copies that are called virtual frontiers. A set of robots R ( )
time. P f ij (rij ), f mn (rmn ) is the probability of two
with cardinality nR all sharing the same n f virtual frontiers independent events since the robots belong to different sets
is defined as a base pair and denoted by (n f , nR ) . ( )
and hence given by P f ij (rij ) P( f mn (rmn )) . However
Henceforth the term frontier refers to a virtual frontier ( )
P f ij (rij ), f kj (rkj ) is the probability of events that are not
unless explicitly mentioned as distinct. Figure 1 depicts the independent and given by
distinct and virtual frontiers for a robot that has two ( )
hypotheses states H1 and H2. ( ( ) ( ))
P f ij rij , f kj rkj =
Pm n Rj ,2
( ) Pm(n Fj − 2, n Rj − 2) ...... (2)
Pm n Fj , n Rj
The NR robots are partitioned into mutually exclusive
sets R1, R2,…, Rn where the cardinality of set Ri is nRi and Hence P ( f ij (rij ), f kj (rkj )) is the probability that at-least one
∑ nRi = N R . The partition is done in a manner such that of the frontiers fi or f k is left unoccupied. Then
i

46
 ( ( ) ) ( ( ) ( )) evaluates to Intersection
P f ij rij , f mn (rmn ) I P f kj rkj , f pn r pn terms of order greater than 3 are not considered

P ( f (r ), f (r )) + P ( f (r ), f (r ))− P ( f (r ), f (r ))
in (4b) due to their negligible contributions.
−1 −1 −1 −1

D2. I P ( f , r , q ;1)I P( f , r , s; l ) is then given by a similar

l
kj kj pn pn kj kj pn pn kj kj pn pn
i
Here the superscript -1 indicates that the cardinality of the
i =1
frontier set is reduced by 1 while distributing robots among
union of terms as in (4b) where each term is
the frontiers of that set. In other words

( ( ))
⎛ t
( ⎞
) ( ( ))
Ti = ⎜ ∏ P n f 1i , n R1i ,−l1i ⎟ P f i −21l 21 (ri 21 ), K , f j−2ls2 s r K (5)
) ( )
Pm ( r ,1)
P f kj−1 rkj =
kj
Pm n Fj − 2, rkj − 1 ..... (3) . ⎜ ⎟
(
Pm n Fj − 1, rkj ⎝ i =1 ⎠
Notations Here the frontier sets associated with the second term are
1. P ( f , r ,2; l ) : The probability of l intersection terms where F21 , F22 , K , F2 s and the number of frontiers to be unfilled
each intersection term is the probability of two robots in those sets are l 21 , l 22 , K , l 2 s . The frontier sets associated
reaching two frontiers such as in (2). with the first complemented term are F11 , F12 , K , F1t and
2. P ( f , r , q; l ) : Same as notation 1 except that each term
the frontiers to be in unfilled in those given by
computes the probability of q robots reaching frontiers.
l11 , l12 , K , l1t
3. P( f , r ,2; l ) : Complement of notation 1. Each term
D3. Counting frontiers that need to be left unfilled: Figure
signifies the probability that at-least one of the two
2 shows three frontier sets F1 , F2 , F3 . The frontiers in each
frontiers is left unfilled.
4. P( f , r , q; l ) : Complement of notation 2. Each term set are abstracted as circles and labeled f 11 , f 21 etc. Arrows
signifies the probability that at-least one of the q frontiers between two frontiers indicate at-least one of them is left
is left unfilled. unfilled. A dashed arrow indicates that both of them need to
(
Pm n f − l , n R ) be occupied. We proceed with the frontier having maximum
(
5. P n f , n R ,−l = ) (
Pm n f , n R )
. This is used in difficult connections in a particular set and make it unoccupied. In
F1 , f 31 is made unoccupied and marked as 1. Then
computations below. f 11 , f 41 , f 31 can be occupied. Since f 11 can be occupied
6. A measurement between two frontiers is denoted by f 21 needs to be unoccupied and labeled 2. Similarly f 23
(d , θ ) and measures the distance and angle between the
needs to be unoccupied and marked 1 in set F3 . Similarly
two. A unique measurement is one that is unique in terms
of either d or θ between a pair among all pairs of f 51 , f 43 needs to be occupied entailing unoccupancy of
frontiers considered f 12 , f 53 and they are labeled 1, 2 in F2 , F3 . Finally f 32
Difficult computations.
( )
needs to be unoccupied and labeled as 2 in F2 . In the above
D1. P f , r ,2; l I P( f , r ,2;1) : There are l + 1 terms in the the only candidate for alternate combination is 2,2,3 unfilled
overall intersection. Let the number of frontiers associated frontiers in F1 , F2 , F3 obtained by having f 33 unoccupied at
be t in number. The number of frontiers required to be start. This is however subsumed in the existing 2,2,2
unfilled in each of this set be l1 , l 2 , K , l t . Let the frontier combination that turns out be the unique combination.
sets associated with P ( f , r ,2; l ) computation be Fm , Fn . If

)⎟⎟ P( f im−l (rim ), f jn−l (r jn ))K (4a )

⎛ t ⎞
T1 = ⎜
⎜ ∏ (
P n fi , n Ri ,−l i m n

⎝ i =1,i ≠ m,n ⎠

( )
p
then P f , r ,2; l I P( f , r ,2;1) = U Tk K (4b ) , where each
k =1
Tk in (4b) is of the form taken by T1 in (4a) corresponding
to every alternate combination of numbers frontiers that are
to be unfilled for each frontier set. The intersection terms
that result from the computation of (4b) is also of the form Figure 2: Counting frontiers that need to be unoccupied. Three
(4a). The only point to note is how unfilled frontiers are frontier sets F1, F2 and F3 with their associated frontiers. Frontiers
allotted to frontier sets in these intersection terms. A second that are counted have their counts shown inside the circle
order intersection term with two combinations of unfilled
frontiers l1 , l 2 , K , l t and m1 , m 2 , K , mt allocates Uniqueness of hypothesis and belief updates.
max(l i , mi ) to Fi . Similarly a p order intersection term
allocates maximum of those p values to each frontier set.

47
For obtaining a unique hypothesis for a certain nR robots
capable of moving to and occupying n f frontiers it is
enough to ensure a pair of frontier locations results in a
unique measurement. If this pair is visible then the states of
the robots occupying the frontiers can be uniquely
hypothesized as a unimodal distribution through update of
each others beliefs. The belief equations are as mentioned in
[Fox et.al.2000] If they are not visible directly but their
presence known through other robots their beliefs can still
be updated to result in a unique hypothesis. This is done by
maintaining a visibility graph where nodes of the graph are
robots and a link between two nodes signifies their mutual
visibility. By updating beliefs appropriately along the links
of the graph it can be ensured the robots at the unique pair
of frontiers have unimodal beliefs (unique hypothesis). The
belief update along a graph is not always trivial and
involves certain care but we do not elaborate further for
space constraints. The frontier sets are always exclusive.
When robots from different base pairs share the same
frontier then the frontier is separated by some distance to
avoid robots reaching the same place. This also makes
frontier sets mutually exclusive.
Cases: Cases depicting computation of probability of robots
occupying frontiers that lead to a unique hypothesis is
discussed here. We proceed from specific to general
situations.
Frontiers lie along a straight line equidistant from one
another: This case typically occurs while moving in
corridors with doors or rooms equally spaced along the
corridor. If the frontiers are from a single base pair there is
only one pair of frontiers that have a unique measurement.
These are the frontiers at the endpoints of the line. The
( ( ))
probability of an unique hypothesis is
P (UH ) = P( f , r ,2;1) = P f i1 (ri1 ), f n f 1 r j1 . This is
calculated as in (2). Here the second index of all subscript is
1 since there is only one robot and frontier set associated to
it. P(UH) denotes the probability of an unique hypothesis.
Frontiers scattered about everywhere: The pairs of frontier
locations that have unique measurements are counted and
enumerated in a set as
{( )( ) ( )}
Fun = f i1 , f j1 , f i 2 , f j 3 ,..., f pm , f qn . Then
P(UH ) = P ( f i1 (ri1 ), f j1 (r j1 )) U K U P ( f pm (rxm ), f qn (r yn ))L (6 )
Each of the union is a term of the form P ( f , r ,2;1) . Write
( A1 U ... U An ) = A1 + (A1 )I A2 + K + ⎜⎜ I (Ai )⎟⎟ I An K (6a ) .
⎛ n −1 ⎞
⎝ i =1 ⎠
By associating probabilities for all terms we find that every
summation term on the right hand side of (6a) can be
correlated to the difficult computation elaborated in D1 and
hence computed as specified there. A1 in (6a) denotes the
( )
event f i1 (ri1 ), f j1 r j1 , or the occupancy of f i1 , f j1 in (6).
48
Visibility restriction between frontiers: If in the set Fun , a
{ }
pair of frontiers f i1 , f j1 is not directly visible then its
occupancy probability ( ( ))
P f i1 (ri1 ), f j1 r j1 ought to be
multiplied by the visibility probability P(V s ) while
computing (6). Here P(V s ) is the probability that two
frontiers not visible directly but can become visible
indirectly through the presence of other robots. A visibility
{ }
graph is constructed between f i1 , f j1 . Let the number of
paths leading from f i1 to f j1 be denoted by the set
{
PATH = P1 , P2 , K , Pp } and the nodes of each path
Pi ∈ PATH be Vi = {V1i , V 2i , K , V vi } . Note that Vi is
essentially a set of frontier locations. The probability that
p
f i1 and f j1 are visible is given by P(V s ) = U P (Vi )K (7 )
i =1
( )
v
and P(Vi ) = I P V ji K (8) . By writing (7) as in (6a) we
j =1
find that each summation term corresponds to the
computation described in D2. Since occupancy of frontiers
and their visibility are independent events their probabilities
get multiplied while computing (6). This is because if a pair
of unique frontiers is directly visible then their visibility is
not hindered by presence of other robots in between because
update of beliefs would any way happen through the
intermediaries. In such cases P(V s ) = 1 always. P(V s ) is
computed only when robots at two frontiers are not directly
visible due to farness or presence of an obstacle and
mandates the presence of other robots at other locations.
Then we compute the probability of those robots occupying
those locations.
3.2 Choosing best combination: A base pair n f , n R ( )
is henceforth denoted as fr . Given the set of all base pairs
{ fr1 , fr2 , K , frm } its power set is obtained. From the power
set a set VAL of valid strings are generated as
{{ fr1 , fr2 , K , frm }, { fr1 , ( fr2 , K , frm )}K , {( fr1 , fr2 , K , frm )}}
A valid string is one that has all the base pairs occurring
exactly once albeit in various combinations. For example
{ fr1, ( fr2 , K , frm )} is a combination of one base pair fr1
considered separately and the remaining considered
together. From VAL a set of strings that can actually occur
in W is extracted and denoted as GVAL, the strings with
ground truth value. For example a string {( fr1, fr2 , K , frm )}
that signifies the combination of all base pairs considered
together belongs to GVAL only if at-least one frontier of
every base pair is visible to at-least one of all other base
pairs. For each of the strings in GVAL an objective function
is evaluated. The string with the highest evaluation is
chosen and the robots dispatched to those frontiers.
Objective function: Given a string from GVAL such as moving to the frontiers. The number of frontiers to be
{( fr1, fr2 ), ( fr3 , K , frm )} the objective function is of the form visited by a robot on an average before localizing is nearly 2
in such a case, whereas by considering robot-robot
⎛ n R1 n R 2 ⎞ ⎛ m ⎞
⎜ + ⎟ P (UH ( fr1 , fr2 )) + ⎜ ∑ n Ri ⎟ P(UH ( fr3 , K , frm )) detections localization is achieved by moving to the first
⎜ n f1 n f 2 ⎟ ⎜ ⎟
⎝ ⎠ ⎝ i =3 n fi ⎠ selected frontier for all the robots. Similar decrease in
( )
P UH ( fri , fr j ) is the probability of obtaining a unique frontiers to be covered before localizing to a unique
hypothesis is obtained in various simulations that are not
hypothesis through a combination of frontiers in base pairs reported here for conciseness.
(
fri and fr j . The multiplicative term n Ri n fi ) biases the Figure 5 shows the graph of probability of obtaining a
unique hypothesis versus number of unique measurement
evaluation to locations where the occurrence of number of
pairs for a fixed number of frontiers (here 12) for various
robots is high to enable localizing a large number of robots
number of robots occupying those frontiers. The probability
at the same time.
increases as the number of unique measurement pairs
4 Simulation Results increase for a given number of robots. Also across the
Figure 3 shows 4 suites of 3 neighboring rooms labeled robots the probability increases as more number of robots
S1,S2,S3 and S4 from top to bottom. The initial positions occupies the same space. Hence the rationale behind the
of the robots are at the center of the rooms. All the robots objective function (section 3.2) that tends to select
shown have three hypotheses of their states initially and all combinations where more robots come together. This graph
of them possess two distinct frontiers one at the top and one sums up the essence of this effort, it gives a bird’s eye view
at the bottom at each of their hypothesis position. Each of of where the robots should move to maximize probabilities
the two distinct frontiers has 3 copies of virtual frontiers of obtaining a unique hypothesis.
corresponding to the three hypotheses states for each robot.
When robot hypotheses of states overlap over certain grids
they are represented as r1/r2, r4/r5 etc suggesting that both
r1 and r2’s hypotheses share some of the grids.
There are eight base pairs with the superscript t and b
indicating the top and bottom distinct frontiers in the
{ }
set fr1t , fr1b , fr2t , fr2b ,K, , fr4t , fr4b . Robots r1 and r2
constitute the base pairs fr1t , fr1b , r3 forms fr2t , fr2b , r4 and
r5 with fr3t , fr3b and r6 and r7 the last two.
Based on the above enumeration, the power set, the set
VAL and the set GVAL are formed as mentioned in 3.2. The
{( ) } {( ) }
combinations fr1b , fr2t , fr3t , fr4t and fr1b , fr2b , fr3t , fr4t
have the highest objective functions. Ties are broken
randomly and the latter combination is chosen. The arrows
show the direction in which each robots moves to reach its
frontier. After reaching their respective frontiers all but
robots r6 and r7 have localized to a unimodal hypothesis of
their states in figure 3b. In the second phase robot r3 is sent
to the frontier set shared by r6 and r7. When r3 nears the Figure 3: 4 suites of 3 rooms labeled S1, S2, S3 and S4 with
center of gravity of that frontier set r6 and r7 are localized to robots at their centers initially. Arrows show the movement of
a unique hypothesis distribution. This has not been shown robots from all its hypothesis states to corresponding frontiers.
in the form of figure due to constraints of space.
Figure 4 tabulates some of the combinations in string
VAL for a different scenario consisting of 6 base pairs of the
{ }
form fr1t , fr1b , ,K, , fr3t , fr3b . The corresponding number of
unique measurements between frontier pairs, probabilities of
unique hypotheses and the objective function evaluation is
also shown. From this the top most combination that can
actually occur in the workspace is selected. This figure
makes clear much of the details in section 3, especially 3.2.
Analysis: If robot-robot detections were not considered
in figure 3b only robots r3 and r5 would have localized after

49
 robot on the top half and 2 for the one at bottom. The
combination { frb1 , frt 2 } yielded the maximum objective
function. The robots are dispatched to the corresponding
frontiers and upon reaching detect one another in figure 6c.
The unique hypothesis states corresponding to figure 6c is
shown in 6d for the two robots.

6 Conclusions:
This paper has presented a novel method of
shepherding robots to locations for rapid elimination of
multiple hypotheses among them. A set of robots Ri with
cardinality n Ri and sharing the same frontier set Fi is
( )
represented by the base pair n Ri , n fi also denoted as fri in
the paper. A combination of several such base pairs that are
Figure 4: Tabulation of only some of the combinations in both syntactically and semantically valid is stored in the set
string VAL for a scenario consisting of 3 base pairs. Other GVAL. The combination with the highest evaluation of
combinations not shown due to space. objective function is chosen and robots are shepherded to
those locations to eliminate maximum number of conflicting
5 Implementation hypotheses. The strategy has been tested both in simulation
and real robots and its efficacy verified. This method finds
The method was verified on a pack of Amigobots utility in several multi-robotic scenarios, where robots are
equipped with 8 sonar transducers. External hardware in the
form of IR transceiver circuit was interfaced to the serial
buffer of the Amigobot’s controller board to facilitate easy
detection of one robot by other. Each transceiver transmitted
a unique pulse code corresponding to a particular robot. It
also received similar codes corresponding to other robots.
The hardware apart from detecting a robot was capable of
measuring the bearing between the detected and detector
robot. Sonar sensors are then fired along the detected
direction to measure distance between the two. Figure 6a
shows a corridor studded with cardboards somewhat akin to
situations in warehouse with two Amigos. Figure 6b shows
the hypotheses returned by the global localization procedure
resulting in two distinct frontiers (one at the top and one at
bottom) for each robot. Each frontier has two copies of
Figure 5: Graph showing probability of obtaining a unique
virtual frontiers corresponding to the two hypotheses states hypothesis versus number of unique measurements for a fixed
for a robot. The number of base pairs is thus four number of frontiers for different numbers of robots
namely { frt1 , frb1 , frt 2 , frb 2 } where the suffix 1 is for the not clear about their state, have multiple hypotheses and

Figures 6a-6d. 6a: Two robots (indicated by white arrows) in an environment studded with cartons. 6b: The hypotheses states corresponding
to 6a. 6c: The robots detect one another through IR transceivers upon nearing the frontiers. 6d. The hypotheses states corresponding to 6c
with each robot having now singularized to a unique state

50
require the assistance of other robots to resolve conflicts as
well as to refine their states to precise coordinates.
References
Dudek et. al, 1998] G. Dudek, K. Romanik, and S. Whitesides.
“Localizing a robot with minimum travel”., SIAM J. Computing
27(2), 1998.
[Fox et. al, 1998] D. Fox, W. Burgard, and S. Thrun,. “Active
markov localization for mobile robots,” Robotics and Autonomous
Systems, 25 (1998), 195-207
[Fox et. al, 2000] Fox et. al, “A Probabilistic Approach to
Collaborative Multi-robotic Localization”, Autonomous Robots,
8(3), 2000
[Howard et. al, 2003] A Howard, M J Mataric and G S Sukhatme,
“Putting the I in the team.: Ego-centric approach to multi-robotic
localization”, IEEE ICRA, 2003.
[Kaelbing 1995] L. P. Kaelbing, M.L. Littman, and A.R.
Cassandra “Planning and acting in partially observable stochastic
domains”, TR, Brown University, 1995.
[Rao et. al, 2005] M Rao, G Dudek, and S. Whitesides,
“Minimum Distance Localization for a Robot with Limited
Visibility”, Proc of ICRA, 2005
[Rekelitis et. al, 1997] I. M. Rekleitis, G. Dudek, and E. Milios.
“Multi-robot exploration of an unknown environment: efficiently
reducing the odometry error.” 2: 1340–1345, IJCAI 1997
[Yamauchi 1997] Yamauchi, “A Frontier-Based Approach for
Autonomous Exploration”, IEEE International Symposium on
Computational Intelligence in Robotics and Automation,: 146-151.

51
 Multi-Robot Marginal-SLAM
Ruben Martinez-Cantin and José A. Castellanos
Dept. Informática e Ingenierı́a de Sistemas
Inst. de Investigación en Ingenierı́a de Aragón (I3A)
University of Zaragoza
rmcantin@unizar.es
Abstract
This paper has two goals. First, it expands the pre-
sentation of the marginal particle filter for SLAM
proposed recently in [Martinez-Cantin et al., 2006].
In particular, it presents detailed pseudo-code to en-
able practitioners to implement the algorithm eas-
ily. Second, it proposes an extension to the multi-
robot setting. In the marginal representation, the
robots share a common map and their locations are
independent given this map. The robot’s relative
locations with respect to each other are assumed
to be unknown. The multi-robot Marginal-Slam
algorithm estimates these transformations of coor-
dinates between the robots to produce a common
global map.
1 Introduction
For many years, Simultaneous Localization and Mapping
(SLAM) has occupied the center-stage in robotics re-
search [Durrant-Whyte and Bailey, 2006; Thrun, 2002]. The
introduction of particle filters (PFs) gave researchers the
power and flexibility to handle nonlinearity and non-Gaussian
distributions routinely [Fox et al., 2001]. Moreover, it en-
abled researchers to exploit conditional independence, us-
ing the Rao-Blackwellized particle filtering (RBPF) variance
reduction technique, to obtain more efficient Monte Carlo
schemes. RBPFs were applied to dynamic maps [Doucet
et al., 2000] and subsequently to static maps with the cel-
ebrated FastSLAM algorithm [Montemerlo et al., 2003].
The application of RBPF to dynamic maps is only sensi-
ble inasmuch as one has a good model to describe the evo-
lution of the dynamic map. On the other hand, the appli-
cation of RBPF to static maps has come into question. It
has become popular knowledge that the approach can di-
verge [Bailey et al., 2006]. In loose terms, learning static
variables (the map) by conditioning on increasing histories
of the state variables results in an accumulation of Monte
Carlo errors and explosion of variance. We will elaborate
on this later on and present further arguments. Heuristic ap-
proaches to ameliorate the situation [Stachniss et al., 2005;
Elinas et al., 2006] have been proposed, but these do no solve
the fundamental problem at hand. We note that the problem
of PF divergence resulting from learning fixed variables by
52
Nando de Freitas
Dept. of Computer Science
University of British Columbia
nando@cs.ubc.ca
conditioning on increasing paths was already described in as
early as 1999 [Andrieu et al., 1999].
The same concerns apply to the multi-robot setting. The
problem is further exacerbated since we typically do not know
the relative position of one robot with respect to the others. It
is important to estimate this quantity in order to produce a
global map. Some authors simplify the problem by only car-
rying out the map merging step when two robots make “eye-
contact” [Howard, 2005]. In general, however, the only infor-
mation available is the data association between common fea-
tures in both maps. In this setting, some authors have adopted
Monte-Carlo localization using joint path sampling [Fox et
al., 2006]. However, as we have argued earlier, this approach
could easily result in degeneracy of the filter. We refer the
interested reader to [Fox et al., 2006] for background on the
multi-robot SLAM problem.
This paper presents a marginal filtering approach, where
the Monte Carlo integration with respect to the robot state
(pose) happens in the marginal space. The approach treats
static maps as parameters, which by necessity are learned
using maximum likelihood (ML) or maximum a posteriori in-
ference. The idea of treating maps as parameters is not new.
It has been central to the incremental ML method [Thrun,
1993]. However, this method resorts to an ML estimate of
the state and hence fails to manage the uncertainty in the ro-
bot state properly, as elaborated in [Thrun, 2002]. A varia-
tion based on learning the distribution of the robot states and
using an ML estimator of the map as a function of the exist-
ing and growing state trajectories was proposed in [Thrun,
2001]. This approach unfortunately suffers from the same
consistency problems as FastSLAM. Various EM approaches
using forward-backward state smoothing and map estimation
in the M step have also been proposed; see [Thrun, 2002] for
a survey. However, this approach only applies when learn-
ing the map off-line. Moreover, it is only very recently that
particle smoothing has become feasible [Klaas et al., 2006].
(As an aside, we note that two-filter smoothers do better than
forward-backward smoothers in the PF context.)
In the marginal approach to SLAM, we are able to com-
pute the filter derivative on-line. This extremely crucial for
on-line static parameter estimation, but has only become pos-
sible very recently following new advances in particle simu-
lation [Klaas et al., 2005; Poyadjis et al., 2005a]. Another
advantage of working on the marginal space is that we can
assume independency of the robot locations given the shared goes to infinity. Unfortunately, one cannot easily sample
map. Therefore, an independent sample set can be used for from the marginal distribution p(xt |y1:t ) directly. Instead,
each robot; thereby reducing the dimensionality of the sam- we draw particles from p(x1:t |y1:t ) and samples x1:t−1 are
pling space. That is, the filter derivatives and mapping up- ignored. This is a valid way to draw samples from a marginal
dates are independent despite the fact that the map parameters distribution and is at the core of most Monte Carlo statistical
are shared. methods. The unknown normalizing constant precludes us
Sections 2 and 3 of this paper are a review of the method from sampling directly from the posterior. Instead, we draw
proposed in [Martinez-Cantin et al., 2006]. The new exten- samples from a proposal distribution q and weight the parti-
sion to multiple robots and new experiments are presented in cles according to the following importance ratio:
Sections 4 and 5.
p(x1:t |y1:t )
wt (x1:t ) =
2 Problem Formulation q(x1:t |y1:t )
We present a general formulation of the problem that is ap- The proposal distribution is constructed sequentially
plicable to both feature-based maps and grid-based maps. A
more specific and detailed model appears in the experimen- q(x1:t |y1:t ) = q(x1:t−1 |y1:t−1 )q(xt |yt , xt−1 )
tal section. The unknown robot pose (location and heading), and, hence, the importance weights can be updated recur-
xt ∈ X , is modelled as a Markov process of initial dis- sively in time
tribution p (x1 ) and transition prior p ( xt | xt−1 ). The ob-
servations, yt ∈ Y, are assumed to be conditionally inde- p(x1:t |y1:t )
pendent given the process {xt } and of marginal distribution wt (x1:t ) = wt−1 (x1:t−1 ).
p(x1:t−1 |y1:t−1 )q(xt |yt , xt−1 )
p θ (yt |xt ), where θ is a vector describing the elements of the (1)
map. Hence, the model consists of the following two distrib- (i)
Given a set of N particles {x1:t−1 }i=1 , we obtain a set of
N
utions: (i) (i)
particles {x1:t }N
i=1 by sampling from q(xt |yt , xt−1 ) and ap-
p(xt |xt−1 ) plying the weights of equation (1).
p θ (yt |xt ) t ≥ 1. The familiar particle filtering equations for this model are
obtained by remarking that
We denote by x1:t
{x1 , ..., xt } and y1:t
{y1 , ..., yt },
respectively, the robot state and the observations up to time t, 
t
and define p(x1 |x0 )
p(x1 ) for notational convenience. p(x1:t |y1:t ) ∝ p (x1:t , y1:t ) = p(yk |xk )p(xk |xk−1 ),
In this formulation, we have used θ to describe the map k=1
only. However, θ could be used to include other parameters in given which, equation (1) becomes
the transition and measurement models, as well as, to include
data association variables [Thrun, 2002]. For simplicity of (i) (i) (i)
(i) p(yt |xt )p(xt |xt−1 ) (i)
presentation, we will assume that the associations are given. t ∝
w (i) (i)
t−1 .
w
Our objective is to compute sequentially in time the filter- q(xt |yt , xt−1 )
ing distribution p ( xt | y1:t ) and point estimates of the map
This iterative scheme produces a weighted measure
θ. (i) (i) (i) (i)  (j)
{x1:t , wt }N i=1 , where wt =w t / j w t , and is known
3 Particle Methods as Sequential Importance Sampling (SIS).
It has been proved [Doucet, 1998] that the variance of the
3.1 The Joint Path Space Approach importance weights in SIS increases over time. This causes
In classical Rao-Blackwellized particle filtering, one first most particles to have very small probability. A common
notes the following decomposition of the joint posterior: strategy to solve this degeneracy, consists of using a resam-
p(θ, x1:t |y1:t ) = p(θ|x1:t , y1:t )p(x1:t |y1:t ). pling step (SIR) after updating the weights to replicate sam-
ples with high probability and prune those with negligible
Consequently, given the state path x1:t , we can solve for the weight [Doucet et al., 2001].
map θ analytically. This leaves us with only having to carry This is the procedure in common use by practitioners. It
out particle filtering to compute the posterior distribution of can be deceptive: although only the state xt is being updated
the robot state p(x1:t |y1:t ). every round, the algorithm is nonetheless importance sam-

N
If we had a set of samples (or particles) xt
(i)
from pling in the growing joint path space X t .
i=1 Formally, the resampling step should be done along the full
p(xt |y1:t ), we could approximate the distribution with the (i) (i)
Monte Carlo estimate path {x1:t , w1:t }Ni=1 . Since dynamic systems forget the past
exponentially fast, several authors carry out resampling over
1 
N (i) (i)
p(dxt |y1:t ) = δ (i) (dxt ), the marginal space {xt , wt }N i=1 . This would be fine if, for
N i=1 xt example, we were inteterested in tracking dynamic maps.
Static parameter estimation and model selection do not
where δx(i) (dxt ) denotes the delta Dirac function. This esti- necessarily exhibit the exponential forgetting behavior. For
t
mate converges almost surely to the true expectation as N example, static maps depend on the whole state trajectory.

53
Resampling over the joint path space is guaranteed to de- Using equation (3) and Bayes rule, the unnormalized filtering
plete the past in finite time. Alternatively, resampling from distribution can be expanded as follows:
the marginal space still leaves us with an accumulation of 
Monte Carlo errors over time. Some implementations have ξ θ (xt , y1:t ) = p θ (yt |xt ) p(xt |xt−1 )p θ (xt−1 |y1:t−1 )dxt−1 .
introduced artificial dynamics or Markov chain Monte Carlo
(MCMC) rejuvenation steps to reduce the severity of the The integral in equation (3) is generally not solveable
problem, but these approaches do not overcome the problem. analytically, but since we have a particle approximation of
(i) (i)
In conclusion, whether we resample or not, learning the p(xt−1 |y1:t−1 ) (namely, {xt−1 , wt−1 }), we can approximate
map as a function of a growing path in Monte Carlo simula- (3) as the weighted kernel density estimate
tion is a bad idea. 
N
(j) (j)
The same degeneracy problem arises if we try to obtain p
θ (xt |y1:t−1 ) = wt−1 p(xt |xt−1 ).
estimates of the filter derivative ∇θ p θ (xt |y1:t ) for recursive j=1
(online) map estimation. To see this, let ∇θ p θ (x1:t |y1:t ) de- While we are free to choose any proposal distribution that
note the gradient vector of the path posterior with respect to has appropriate support, it is convenient to assume that the
the map. Then, we have marginal proposal takes a similar form, namely
∇θ p θ (x1:t |y1:t ) 
N
∇θ p θ (x1:t |y1:t ) = p θ (x1:t |y1:t ) (j) (j)
p θ (x1:t |y1:t ) q θ (xt |y1:t ) = wt−1 q θ (xt |yt , xt−1 ). (5)
j=1
and, consequently the filter derivative, necessary for online
map learning, is given by: We can easily draw particles from this proposal using
 multinomial or stratified sampling. Note the importance
∇θ p θ (x1:t |y1:t ) weights are now defined on the marginal space:
∇θ p θ (xt |y1:t ) = p θ (x1:t |y1:t )dx1:t−1
X t−1 p θ (x1:t |y1:t ) p θ (xt |y1:t )
(2) wt = .
q θ (xt |y1:t )
This equation says that when using standard particle filters
to approximate the filter derivative we are implicitly carrying The Filter Derivative
out importance sampling on a vast growing space with pro- In order to obtain the gradient vector with respect to the map
posal p θ (x1:t |y1:t ) and weight ∇pθ θp(xθ (x1:t |y1:t )
. This should variables, we apply standard differentiation rules to equa-
1:t |y1:t ) tion (4), yielding:
be enough reason to call for a new approach. Yet, the problem
∇θ ξ θ (xt , y1:t )
is even worse. ∇θ p θ (xt |y1:t ) =
The filter derivative is a signed-measure, and not a standard ξ θ (xt , y1:t )dxt

probability measure. It consists of positive and negative func- ∇θ ξ θ (xt , y1:t )dxt
tions over disjoint parts of the state space and it sums to zero − p θ (xt |y1:t ) . (6)
ξ θ (xt , y1:t )dxt
over the entire state space. A serious problem, when carrying
out classical particle filtering to estimate this signed-measure, Similarly, using the expansions for the derivatives of logs, the
is that particles with positive and negative weights will cancel gradient of ξ(·) can be written as follows:
each other, say, in parts of the space where the derivative is ∇θ ξ θ (xt |y1:t ) = p θ (yt |xt )∇θ log p θ (yt |xt )
close to zero. This is wasteful and statistically harmful. See 
Figure 1 of [Poyadjis et al., 2005a] for a beautiful depiction × p(xt |xt−1 )p θ (xt−1 |y1:t−1 )dxt−1
of this problem. 
The technique presented in the following section over- + p θ (yt |xt ) p(xt |xt−1 )∇θ p θ (xt−1 |y1:t−1 )dxt−1 (7)
comes these deficiencies.
Note that the importance sampling process now happens in
3.2 The Marginal Space Approach the marginal space. The last integral in equation (4) can be
Marginal Particle Filtering expanded using the score identity:
To eliminate the problems discussed in the previous section, 
we will perform particle filtering directly on the marginal dis- p(xt |xt−1 )∇θ log[p θ (xt−1 |y1:t−1 )]p θ (xt−1 |y1:t−1 )dxt−1
tribution p(xt |y1:t ) instead of on the joint space [Klaas et
al., 2005; Poyadjis et al., 2005b; 2005a]. To do so, we be- That is we sample from the marginal filtering distribution
gin by noting that the predictive density can be obtained by and weight with β
∇θ log[p θ (xt−1 |y1:t−1 )]. Contrast this
marginalization: with equation (2).
 The other thing to note, as pointed out in [Poyadjis et al.,
p θ (xt |y1:t−1 ) = p(xt |xt−1 )p θ (xt−1 |y1:t−1 )dxt−1 (3) 2005a] is that the marginal filter derivative allows us to obtain
a particle approximation of the Hahn-Jordan decomposition.
To simplify the exposition later on, we introduce the follow- This implies that we can surmount the problem of particles
ing notation [Poyadjis et al., 2005a]: of opposite signs cancelling each other out in infinitesimal
neighborhoods of the state space. Once again we refer the
ξ θ (xt , y1:t ) reader to Figure 1 of [Poyadjis et al., 2005a] for an illustration
p θ (xt |y1:t )
(4)
ξ θ (xt , y1:t )dxt of this phenomenon.

54
Monte Carlo Implementation A detailed analysis is presented in [Tadic and Doucet, 2005].
We are now ready to present the particle algorithm of [Poy- The only remaining detail is to describe the Monte Carlo ap-
adjis et al., 2005a] for approximating the filter derivative ef- proximation of the gradient of the predictive distribution. It
ficiently. Assume that at time t − 1, we have the following can be derived as follows:
approximations of the filter and its gradient:

∇ θ p θ (yt |y1:t−1 )

N ∇θ log p
θ (yt |y1:t−1 ) =
(i) p
θ (yt |y1:t−1 )
p
θ (xt−1 |y1:t−1 ) = wt−1 δx(i) (xt−1 )
i=1
t−1
∇ θ ξ θ (xt , y1:t )dxt
=

N
(i) (i) ξ
θ (xt , y1:t )dxt


∇θ p θ (xt−1 |y1:t−1 ) = wt−1 βt−1 δx(i) (xt−1 ). N (j)
i=1
t−1
j=1 ρ t
=
N (j)
(8)
Then, we can sample from the proposal in equation (5) and j=1 w t
compute the new unnormalized importance weights:
(i) N (j) (i) (j)
Pseudo-Code for Marginal Slam
(i) p θ (y t |x t ) w
j=1 t−1 p(x t |x t−1 ) The Marginal-SLAM algorithm is depicted in Figure 1. Note
wt = (i) that it is linear in the number of features. It has an O(N 2 )
q θ (xt |y1:t )
complexity in terms of the number of samples, but this can be
(i)  (j) (i) (j) (i) (j)
(i) p θ (y t |x t ) w
j t−1 p(x t |x t−1 )[∇ θ log p θ (yt |x t )+βt−1 ] reduced to O(N log N ) using the fast multipole expansions
ρt = (i) and metric tree recursions proposed in [Klaas et al., 2005].
q θ (xt |y1:t ) The marginal particle filter is an old idea [Fearnhead, 1998;
These weights lead to the following approximations: Pitt and Shephard, 1999]. Yet, because of its large computa-
tional cost, it was not fully explored until the introduction of
1  (i)
N
ξ
θ (xt , y1:t ) = t δx(i) (xt )
w fast methods [Klaas et al., 2005]. When using the transition
N i=1 t prior as proposal, the marginal filter and classical particle fil-
ter are equivalent [Khan et al., 2004], but this is no longer
1  N
(i) true when computing the derivative of the filter as outlined in
∇ θ ξ θ (xt , y1:t ) = ρ δ (i) (xt )
N i=1 t xt [Poyadjis et al., 2005b] and this paper.
Finally, normalizing the weights and substituting the above 4 Multi-robot Implementation
Monte Carlo estimates into the expression for the derivative
of p θ in terms of ξ θ , we obtain: Hereinafter, we focus on feature-based representations be-
cause our goal is to apply the method to visually guided mo-

N
(i) (i) bile robots [Lowe, 2004; Bay et al., 2006; Newman et al.,


∇θ p θ (xt , y1:t ) = wt βt δx(i) (xt )
t 2006]. The robot motion model is based on simple differen-
i=1
tial drive vehicle
where,
(i)
 (j) Xt = Xt−1 + dt cos(ψt−1 )
(i) (i) ρ (i) t
jρ
wt βt =  t (j) − wt  (j) Yt = Yt−1 + dt sin(ψt−1 )
jwt t
jw ψt = ψt−1 + αt ,
On-Line Map Learning where xt = [Xt , Yt , ψt ] denotes the robot pose and heading
Armed with Monte Carlo estimates of the filter derivative, and ut = [dt , αt ] denotes the translation and rotation motion
we can now attack the problem of developing recursive map commands (odometry) at time t with corresponding Gaussian
estimators. Here, we choose to maximize the predictive like- noise vt ∼ N (0, σR I), where σR = [σd , σα ]. The transition
lihood (also known as the innovations or evidence): model is Gaussian in terms of ut , but it involves a nonlinear

transformation between ut and xt . Specifically, it is given by
p θ (yt |y1:t−1 ) = p θ (yt |xt )p(xt |xt−1 )p θ (xt−1 |y1:t−1 )dxt−1:t the following expression:
 
To accomplish this, we adopt the following stochastic approx- 1 (dt − dˆt )2 (αt − α̂t )2
imation algorithm: p(xt |xt−1 , ut ) = exp + ,
2πσd σα −2σd2 −2σα2
θt = θt−1 + γt ∇θ log p
θ (yt |y1:t−1 ) (9)
Provided that the step size γt satisfies standard stochastic ap- where
proximation criteria, see for example [Bertsekas and Tsitsik-
ˆt =
d d2x + d2y
lis, 1996; Spall, 2005], it can be shown that θt converges to
the set of global or local maxima of l(θ), where [Poyadjis et  
−dx sin(ψt−1 ) + dy cos(ψt−1 )
al., 2005a] α̂t = arctan .
dx cos(ψt−1 ) + dy sin(ψt−1 )
1 
k
l(θ) = lim log p θ (yt |y1:t−1 ) with dx = Xt − Xt−1 and dy = Yt − Yt−1 . In the work,
k→∞ k + 1 we will use this transition prior as the proposal distribution.
t=1

55
 ture is:
 
Marginal-SLAM
1 (ρt − ρ̂t )2 (φt − φ̂t )2
p(yt |xt ) = exp + (10)
• For i = 1, ..., N , sample the robot state from the pro- 2πσρ σφ −2σρ2 −2σφ2
posal
N
The overall observation likelihood is given by a product of
(i)
xt ∼
(j) (j)
wt−1 qθ (xt |yt , xt−1 ) the individual likelihood terms. The gradient of the log-
j=1
likelihood can be obtained by simple differentiation:
 


∆x (ρt −ρ̂t ) ∆ (φ −φ̂ )
• For i = 1, ..., N , evaluate the importance weights 1  σρ2 − y ρt tσ2 t
∇θ log(p(yt |xt )) = .


φ
∆y (ρt −ρ̂t )
N ρt + ∆x (φt −2 φ̂t )
(i) (j) (i) (j)
(i) p θ (yt |xt ) j=1 wt−1 p(xt |xt−1 ) 2
σρ ρt σφ
wt = (i)
q θ (xt |y1:t ) (11)

(i)
ρt = (i)
(i)
p θ (yt |xt )
q θ (xt |y1:t )
·
One of the main SLAM difficulties is partial observability of
the parameters. The choice of learning rates γt is affected by
this partial observability. In our case, we chose to implement
N
(j) (i) (j)
separate learning rates for each landmark. Each individual
· wt−1 p(xt |xt−1 ) · rate depends on the number of times its corresponding feature
j=1
is observed. In addition, when there is no observation, the
(i) (j)
· [∇θ log p θ (yt |xt ) + βt−1 ] likelihood function of the unseen features is assumed to be
uniform over the whole space, then ∇θ log(p(yt |xt )) = 0.
• Normalise the importance weights



(i) 4.1 Non-Stationarity and Estimate Recovery

(i) wt



wt = (j) Although our goal has been to develop a method for static




j wt
maps using decreasing learning rates, it is possible to adopt



(j)
(i) (i) ρt
(i)
(i) j ρt small constant learning rates to estimate time-varying para-
wt β t = (j)
− wt (j) meters (maps). This enables us to attack maps that contain
j wt j wt
some moving parameters, such as chairs and doors [Martinez-



• Update the map vector Cantin et al., 2006].
The same strategy for dealing with non-stationarity can



(j)
j ρt also be adopted to correct for modelling errors. For exam-
θt = θt−1 + γt (j) ple, the ability to track moving features enables Marginal-
j wt
SLAM to recover from errors in data association, provided
• Update the learning rate γt . that the correct correspondences are obtained in subsequent
steps. This is crucial because in the early steps of the al-
gorithm there aren’t enough observations to guarantee global
map consistency. Figure 6 in the experimental section, illus-
Figure 1: The Marginal-SLAM algorithm at time t. trates how Marginal-SLAM is robust with respect to reason-
able errors in matching.

We do this for both Marginal-SLAM and FastSLAM so as 4.2 Map Merging

to provide a fair comparison. One can do better however by Our goal is to merge the map estimates of each robot into
adopting approximations of the optimal proposal distribution a single global map. When the relative robot locations are
that account for the latest observations. Typical approxima- known, the problem is simple and reduces to standard mono-
tions of this type include the extended and unscented Kalman lithic SLAM. In practice, however, we do not know the trans-
filters. formations of coordinates that would allow all the robots to
We assume a simple range and bearing sensor yt = [ρt , φt ] have the same reference frame for the global map. We must
with a point feature detector: therefore estimate the parameters of these linear transforma-
tions (rotation and translation) between the poses of the vari-
ous robots.
ρ̂t = ∆2x + ∆2y Some authors simplify this problem by only carrying
  out the map merging step when two robots make “eye-
−∆x sin(ψt ) + ∆y cos(ψt )
φ̂t = arctan , contact” [Howard, 2005]. In general, however, the only infor-
∆x cos(ψt ) + ∆y sin(ψt ) mation available is the data association between common fea-
tures in both maps. In this setting, some authors have adopted
where ∆x = θx − Xt and ∆y = θy − Yt , with [θx , θy ] de- Monte-Carlo localization using joint path sampling [Fox et
noting the feature location. The sensor has white Gaussian al., 2006]. However, as we have argued in earlier sections,
noise vt ∼ N (0, σI) in the measurement space, where σ = this approach could easily result in degeneracy of the filter.
[σρ , σφ ]. Hence, the observation likelihood for a single fea- We point out in passing that map merging is only one of the

56
 Here, we adopt standard vision algorithms to compute the
Multi-robot Marginal-SLAM at step t correspondences between features [Hartley and Zisserman,
1. For each robot, compute their pose and map:
2000; Brown and Lowe, 2005]. In order to simplify the pre-
sentation, let us for the time being consider only two robots:
• For i = 1, ..., N , particles R1 and R2 . When the two robots share a common set of map
1. Sample new particles from the prior mixture pro- features for which we have computed the correspondences,
posal: we apply standard algorithms to estimate the transformation
(a) Select mixture components using stratified re- of coordinates between the poses or R1 and R2 . (We refer the
sampling:
 x
t−1 ,
(i) 1
 N 
← xt−1 , wt−1
(i) (i)
 N
reader to either [Castellanos and Tardós, 1999, Appendices A
and B] for a detailed explanation or [Brown and Lowe, 2005]
for a brief yet comprehensive explanation on how to carry out
N i=1 i=1
feature matching and estimation of transformations between
(b) Simulate the stochastic dynamics for each se- different reference frames.)
lected component p(xt |ut , x
t−1 ):
(i) Once we know the transformation of coordinates, we map
R2 ’s pose to the reference frame of R1 . We now have a com-
(i) (i) (i) (i)
[dt , αt ] = [dt , αt ] + vt ; vt ∼ N (0, σI) mon reference frame for the poses of both robots. We also
(i)
(i) (i)
(i) transform the estimates of the location of map features to this
Xt = Xt−1 + dt cos(ψt−1 )
common reference frame. Finally we merge the estimates of
(i)
(i) (i)
(i)
Yt = Yt−1 + dt sin(ψt−1 ) the map feature locations of both robots in order to produce
(i)
(i) (i) the common global map.
ψt = ψt−1 + αt
As mentioned previously in Section 4.1, we noticed that
2. Evaluate the importance weights using the expres- Marginal-SLAM was able to converge empirically to the true
sions in equations (9), (10) and (11) for the transi- map, even in the presence of errors in the estimates of the

tion prior (odometry), likelihood and gradient: relative transformations.
The multi-robot Marginal-SLAM algorithm is detailed in



(i) (i)
wt = p θ (yt |xt )
(i) (i) (i)
figure 2. For clarification, the pseudo-code is based on the
ρt = wt ∇θ log p θ (yt |xt ) robot and sensor models presented in this paper, but it can be
p θ (yt |xt ) N

(i) (j) (j) (i) (i) (j)
j=1 wt−1 βt−1 p(xt |ut , xt−1 )
easily modified to treat other models.
+ N (j) (i) (i) (j)
j=1 wt−1 p(xt |ut , xt−1 )
5 Experiments
• For i = 1, ..., N , normalise the importance weights:







In this preliminary work, experiments are based on simula-



(j)
(i) wt
(i)
(i) (i) ρt
(i)
(i) j ρt tions in a controlled environment, where all the error sources
wt = (j)
; wt βt = (j)
−wt (j) and data associations are known. This simulated environment
j wt j wt j wt
allows us to carry out experiments for one and multiple robots


• Update the individual map vector: using different sensor and motion noise models.




We first present a brief comparison of Marginal-SLAM and
(j)
j ρt FastSLAM. For a more detailed comparison, see [Martinez-
θt = θt−1 + γt (j) Cantin et al., 2006]. Figure 3 shows a comparison be-
j wt N
tween the number of effective particles Nef f = 1/ i=1 wi2
• Update the learning rates γt . using Marginal-SLAM and FastSLAM. This quantity mea-
2. If maps share features: sures the degeneracy of sequential Monte Carlo algorithms.
To obtain a fair comparison, we use the same transition
• Compute the relative linear transformation between the
coordinates of both robots.
prior p(xt |ut , xt−1 ) as the proposal distribution in both algo-
rithms. Clearly, the marginal particle filter reaches a steady
• Transform all maps to a common reference frame using state, but FastSLAM loses particle diversity at a very fast rate.
the estimated linear transformations. These results are based on the following parameter settings:
• Place all transformed features in a common global map. L = 200, σd = 0.1m, σα = 0.5deg, σρ = 0.025m, σφ =
All robots use this map at step t + 1. 3deg, where L denotes the number of landmarks.
In the multi-robot case, we used only 50 landmarks to en-
sure that we have a demanding testing scenario. That is, the
Figure 2: The multi-robot Marginal-SLAM algorithm at time robots share very few features in common. For simplicity, we
t. This version uses stratified resampling to sample from the use only two robots, but the algorithm extends naturally to
mixture proposal. The proposal in this case is simply a mix- more robots.
ture of transition priors. We assume that the initial relative robot location is un-
known. As shown in Figure 4, we have to learn the linear
transformation (rotation and translation) of coordinates be-
components of the more general approach presented in [Fox tween the two robots using standard geometry. When the ro-
et al., 2006]. bots detect common features, the estimated linear transforma-

57
 80 120
120
MarginalSLAM
FastSLAM
70 FastSLAMt
100
100 First steps inset
60
80 80

50
Effective particle

Efficient particle

Efficient particle
s

s
60 60
40

40 40
30

20 20
20

0 0
10

0 −20 −20
0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000 0 50 100 150 200 250 300
Steps Steps Steps

Figure 3: Average number of effective particles Nef f , with corresponding confidence intervals, for 25 simulations. The number
of particles is 200 in both cases. The left plot corresponds to Marginal-SLAM, where the filter achieves a steady state with
a 25% effective number of particles. The middle plot shows the same experiment for FastSLAM. The effective sample size
quickly drops and FastSLAM fails to maintain particle diversity after 200 steps. The right plot is a zoomed view of the middle
plot.

Step 200 Step 250 Step 4536

40 40
40

35 35
35

30 30
30

25 25
25

20 20
20

15 15
15

10 10 10

5 5 5

0 0 0

−20 −10 0 10 20 30 −10 0 10 20 30 40 −45 −40 −35 −30 −25 −20 −15 −10 −5 0 5

Figure 6: Loop closing in Multi-robot Marginal-SLAM with two robots. Despite the error introduced during map merging,
the robot is able to close the loop (left plot). Although the map estimate after loop closing is still inaccurate (middle plot),
Marginal-SLAM is able to minimize the average error and converge to the true solution. The right plot displays the estimate
and true maps map after 10 loops. The blue stars represent the true map and the black circles correspond to the most recent
map estimate. The red diamonds are the current observations.

tion is used to represent both maps with respect to a common in the loop closing area for this robot. The convergence of the
reference frame (robot R1 ’s position in our case). The two filter is empirically verified in this last experiment.
maps are then merged into a single global map. This is illus- We close this section by stating that Marginal-SLAM is
trated in Figure 5. In this figure, the estimated transformation able to build accurate maps with large range and bearing
is biased, because the estimates of the maps have not con- noise. However, very large sensor noise (for example sonar)
verged and the observations are very noisy. Therefore, the demands a high number of particles to ensure convergence.
map merging step introduces some error in the global map. In addition, many update steps are required to obtain reason-
However, as discussed in Section 4.1, Marginal-SLAM is able able errors. We believe that data driven proposals and adap-
to recover from these errors. Figure 6 reveals how R1 and R2 tive stochastic approximation could be applied in the future
close the loop and keep updating the shared map, while re- to confront this problem and speed up the convergence rate.
ducing the error introduce in the map merging step. The plots
are with respect to R1 ’s location. There is no need for ab- 6 Conclusions
solute location or reference information. The experiments and arguments in this paper and [Martinez-
Finally, Figure 7 shows the average mean Euclidean dis- Cantin et al., 2006] indicate that Marginal-SLAM is an im-
tance of the landmarks estimate with respect to the true loca- portant new direction in the design of particle methods for
tion after ten repetitions of the experiment. We use a robocen- SLAM. It is clear that algorithms designed to work on the
tric representation based on the expectation of the first robot’s marginal space behave better than the ones designed to work
location. The peaks represent points with high heading error on the path space. They also lend themselves naturally to the

58
 Step 1 Step 45
50
40

45
35
40

30
35

25
30

20 25

20
15

15
10
10

5
5

0 0

−5 0 5 10 15 20 25 30 35 40 45 −30 −20 −10 0 10 20 30

Step 1 Step 46

40 40

35 35

30 30

25 25

20 20

15 15

10 10

5 5

0 0

−5 0 5 10 15 20 25 30 35 40 45 −30 −20 −10 0 10 20

Figure 4: In the first iteration of the multi-robot experiment,
the unknown relative position of the robots is modelled in
terms of an unknown linear transformation. The upper plot
represents the true robot locations. The lower plot shows the
robot locations, while emphasizing that we don’t know the
linear transformation of coordinates at this early stage. The
blue stars represent the true map and the black circles corre-
spond to the most recent map estimate. The red diamonds are
the current observations.
Figure 5: Map merging in multi-robot Marginal-SLAM. The
unknown relative robot location is recovered using the es-
timated transformation between common features in both
maps. In the lower plot, maps have been merged. The re-
sulting map is biased because of the poor relative location es-
timate at this early stage. Again, the blue stars represent the
true map and the black circles correspond to the most recent
map estimate. The red diamonds are the current observations.

global map estimation in the multi-robot setting.
Marginal-SLAM seems to be robust with respect to errors
introduced during the map joining step. This is important
as computer vision matching algorithms are not perfect. We
are planning to deploy Marginal-SLAM using the visual fea-
tures and matching procedure presented in [Brown and Lowe,
2005]. This will enable us to obtain 3D maps of the environ-
ment.
In future work, we plan to adopt better proposal distrib-
utions using extended and unscented Kalman filters and to
implement Hessian updates so as to potentially maximize the
predictive likelihood more efficiently.
7 ACKNOWLEDGMENTS
We are very thankful to Arnaud Doucet for many thought-
ful discussions. We are also indebted to Frank Dellaert and
Tim Bailey for valuable feedback on an earlier version of this
paper.
This work was supported in part by the Dirección General
de Investigación of Spain under project DPI2003-07986 and
by NSERC.
References
[Andrieu et al., 1999] C Andrieu, N de Freitas, and A Doucet. Se-
quential MCMC for Bayesian model selection. In IEEE Higher
Order Statistics Workshop, pages 130–134, Caesarea, Israel,
1999.

59
 4 [Castellanos and Tardós, 1999] J.A. Castellanos and J.D. Tardós.
Mobile Robot Localization and Map Building. A Multisensor Fu-
3.5 sion Approach. Kluwer Academic Publishers, 1999.
[Doucet et al., 2000] A Doucet, N de Freitas, K Murphy, and
average eclidean distance of features S Russell. Rao-Blackwellised particle filtering for dynamic
3

Bayesian networks. In C Boutilier and M Godszmidt, editors,

2.5 Uncertainty in Artificial Intelligence, pages 176–183. Morgan
Kaufmann Publishers, 2000.
2 [Doucet et al., 2001] A Doucet, N de Freitas, and N J Gordon, ed-
itors. Sequential Monte Carlo Methods in Practice. Springer-
1.5 Verlag, 2001.
[Doucet, 1998] A Doucet. On sequential simulation-based methods
1
for Bayesian filtering. Technical Report CUED/F-INFENG/TR
310, Department of Engineering, Cambridge University, 1998.
0.5
[Durrant-Whyte and Bailey, 2006] Hugh Durrant-Whyte and Tim
Bailey. Simultaneous Localisation and Mapping (SLAM): Part
0
0 500 1000 1500 2000 2500 3000 3500 4000 4500 I The Essential Algorithms. Robotics and Automation Magazine,
steps June 2006.
1.4
[Elinas et al., 2006] Pantelis Elinas, Robert Sim, and James J.
Little. sigmaSLAM: Stereo vision SLAM using the Rao-
1.2 Blackwellised particle filter and a novel mixture proposal dis-
tribution,. In IEEE International Conference on Robotics and
average euclidean distance of features

1
0.8
0.6
0.4
0.2
0
46 66
Figure 7: Evolution of the average map error in 10 runs. The
error is measured in terms of the Euclidean distance between
the robocentric map and the ground truth map. Therefore, the
robot location error is implicitly included in this plot. The
top plot shows that the algorithm converges asymptotically.
The bottom plot is a zoomed version of the top plot with 95%
confidence intervals, which indicate the robustness of the al-
gorithm.
[Bailey et al., 2006] Tim Bailey, Juan Nieto, and Eduardo Nebot.
Consistency of the FastSLAM algorithm. In IEEE International
Conference on Robotics and Automation, 2006.
[Bay et al., 2006] H. Bay, T. Tuytelaars, and L. Van Gool. Surf:
Speeded up robust features. In Proceedings of the ninth European
Conference on Computer Vision, May 2006.
[Bertsekas and Tsitsiklis, 1996] D P Bertsekas and J N Tsitsiklis.
Neuro-Dynamic Programming. Athena Scientific, 1996.
[Brown and Lowe, 2005] M Brown and D Lowe. Unsupervised 3D
object recognition and reconstruction in unordered datasets. In
5th International Conference on 3D Imaging and Modelling, Ot-
tawa, Canada, 2005.
Automation, 2006.
[Fearnhead, 1998] P Fearnhead. Sequential Monte Carlo Methods
in Filter Theory. PhD thesis, Department of Statistics, Oxford
University, England, 1998.
[Fox et al., 2001] D Fox, S Thrun, W Burgard, and F Dellaert. Par-
ticle filters for mobile robot localization. In A Doucet, N de Fre-
itas, and N J Gordon, editors, Sequential Monte Carlo Methods
in Practice. Springer-Verlag, 2001.
[Fox et al., 2006] D. Fox, J. Ko, K. Konolige, B. Limketkai,
D. Schulz, and B. Stewart. Distributed multi-robot exploration
and mapping. Proc. of the IEEE, 2006.
[Hartley and Zisserman, 2000] R Hartley and A Zisserman. Mobile
86 106
steps
126 146 166 186 Robot Localization and Map Building. A Multisensor Fusion Ap-
proach. Kluwer Academic Publishers, 2000.
[Howard, 2005] A. Howard. Multi-robot simultaneous localization
and mapping using particle filters. In In Proc. Robotics: Science
and Systems Conference, 2005.
[Khan et al., 2004] Z. Khan, T. Balch, and F. Dellaert. A Rao-
Blackwellized particle filter for EigenTracking. 2004.
[Klaas et al., 2005] Mike Klaas, Nando de Freitas, and Arnaud
Doucet. Toward practical n2 Monte Carlo: The marginal particle
filter. In Uncertainty in Artificial Intelligence, 2005.
[Klaas et al., 2006] Mike Klaas, Mark Briers, Nando de Freitas, Ar-
naud Doucet, Simon Maskell, and Dustin Lang. Fast particle
smoothing: If i had a million particles. In International Confer-
ence on Machine Learning, 2006.
[Lowe, 2004] D.G. Lowe. Distinctive image features from scale-
invariant keypoints. In International Journal of Computer Vision,
volume 60, pages 91–110, 2004.
[Martinez-Cantin et al., 2006] R. Martinez-Cantin, N. de Freitas,
and J.A. Castellanos. Marginal-SLAM: A Convergent Parti-
cle Method for Simultaneous Robot Localization and Mapping,
2006.
[Montemerlo et al., 2003] M. Montemerlo, S. Thrun, D. Koller, and
B. Wegbreit. FastSLAM 2.0: An improved particle filtering al-
gorithm for simultaneous localization and mapping that provably
converges. In Proceedings of the Sixteenth International Joint

60
 Conference on Artificial Intelligence (IJCAI), Acapulco, Mexico,
2003. IJCAI.
[Newman et al., 2006] Paul Newman, David Cole, and Kin Ho.
Outdoor slam using visual appearance and laser ranging. In
International Conference on Robotics and Automation (ICRA),
Florida, 2006.
[Pitt and Shephard, 1999] M K Pitt and N Shephard. Filtering via
simulation: Auxiliary particle filters. JASA, 94(446):590–599,
1999.
[Poyadjis et al., 2005a] G. Poyadjis, A. Doucet, and S.S. Singh.
Maximum likelihood parameter estimation using particle meth-
ods. In Joint Statistical Meeting, 2005.
[Poyadjis et al., 2005b] G. Poyadjis, A. Doucet, and S.S. Singh.
Particle methods for optimal filter derivative: Application to pa-
rameter estimation. In IEEE ICASSP, 2005.
[Spall, 2005] J C Spall. Introduction to Stochastic Search and Op-
timization. Wiley, 2005.
[Stachniss et al., 2005] C. Stachniss, G. Grisetti, and W. Burgard.
Recovering particle diversity in a Rao-Blackwellized particle fil-
ter for SLAM after actively closing loops. In Proc. of the IEEE
Int. Conf. on Robotics & Automation (ICRA), pages 667–672,
Barcelona, Spain, 2005.
[Tadic and Doucet, 2005] V B Tadic and A Doucet. Exponential
forgetting and geometric ergodicity for optimal filtering in gen-
eral state-space models. Stochastic Processes and Their Applica-
tions, 115:1408–1436, 2005.
[Thrun, 1993] Sebastian Thrun. Exploration and model building in
mobile robot domains. In E Ruspini, editor, Proceedings of the
ICNN, pages 175–180, 1993.
[Thrun, 2001] S. Thrun. A probabilistic online mapping algorithm
for teams of mobile robots. International Journal of Robotics
Research, 20(5):335–363, 2001.
[Thrun, 2002] S. Thrun. Robotic mapping: A survey. In G. Lake-
meyer and B. Nebel, editors, Exploring Artificial Intelligence in
the New Millenium. Morgan Kaufmann, 2002.

61