Professional Documents
Culture Documents
Springer Tracts in Advanced Robotics Vol
Springer Tracts in Advanced Robotics Vol
Volume 11
Soccer Robotics
13
Professor Bruno Siciliano, Dipartimento di Informatica e Sistemistica, Università degli Studi di Napoli
Federico II, Via Claudio 21, 80125 Napoli, Italy, email: siciliano@unina.it
Professor Oussama Khatib, Robotics Laboratory, Department of Computer Science, Stanford University,
Stanford, CA 94305-9010, USA, email: khatib@cs.stanford.edu
Professor Frans Groen, Department of Computer Science, Universiteit van Amsterdam, Kruislaan 403,
1098 SJ Amsterdam, The Netherlands, email: groen@science.uva.nl
STAR (Springer Tracts inAdvanced Robotics) has been promoted under the auspices of EURON (European
Robotics Research Network)
Authors
Dr. Jong-Hwan Kim Dr. Dong-Han Kim
Dept. of Electrical Engineering Dept. of Electrical Engineering
and Computer Science and Computer Science
Korea Advanced Institute of Korea Advanced Institute of
Science and Technology (KAIST) Science and Technology (KAIST)
373-1 Gusong-dong, Yusong-gu 373-1 Gusong-dong, Yusong-gu
Daejeon 305-701 Daejeon 305-701
Republic of Korea Republic of Korea
ISSN 1610-7438
ISBN 3-540-21859-9 Springer-Verlag Berlin Heidelberg New York
Library of Congress Control Number: 2004104485
This work is subject to copyright. All rights are reserved, whether the whole or part of the mate-
rial is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication
of this publication or parts thereof is permitted only under the provisions of the German Copyright
Law of September 9, 1965, in its current version, and permission for use must always be obtained
from Springer-Verlag. Violations are liable to prosecution under German Copyright Law.
Springer-Verlag is a part of Springer Science+Business Media
springeronline.com
© Springer-Verlag Berlin Heidelberg 2004
Printed in Germany
The use of general descriptive names, registered names, trademarks, etc. in this publication does
not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
Typesetting: Digital data supplied by authors.
Data-conversion and production: PTP-Berlin Protago-TeX-Production GmbH, Germany
Cover-Design: design & production GmbH, Heidelberg
Printed on acid-free paper 62/3020Yu - 5 4 3 2 1 0
Editorial Advisory Board
EUROPE
Herman Bruyninckx, KU Leuven, Belgium
Raja Chatila, LAAS, France
Henrik Christensen, KTH, Sweden
Paolo Dario, Scuola Superiore Sant’Anna Pisa, Italy
Rüdiger Dillmann, Universität Karlsruhe, Germany
AMERICA
Ken Goldberg, UC Berkeley, USA
John Hollerbach, University of Utah, USA
Lydia Kavraki, Rice University, USA
Tim Salcudean, University of British Columbia, Canada
Sebastian Thrun, Carnegie Mellon University, USA
ASIA/OCEANIA
Peter Corke, CSIRO, Australia
Makoto Kaneko, Hiroshima University, Japan
Sukhan Lee, Sungkyunkwan University, Korea
Yangsheng Xu, Chinese University of Hong Kong, PRC
Shin’ichi Yuta, Tsukuba University, Japan
Foreword
been the ‘examination’ ground for the testing of new techniques and tech-
nologies integrated in the game of robot soccer, and has provided much ex-
citement and entertainment for all those who participated.
This new book Soccer Robotics is intended to be a comprehensive intro-
duction to the field of soccer robotics, emphasizing breadth of coverage and
accessibility of the material to readers with possibly different backgrounds.
Its key feature is the emphasis placed on a robot soccer-programming frame-
work that integrates all the key areas of robot technology. Until now, these
areas had been treated mainly in separate books or in research literature
only, outside the arena of soccer robotics.
A substantial portion of this book is based on the first author’s lectures
EE006 Robot Soccer System at the Korea Advanced Institute of Science
and Technology in the period July 13 - August 14, 1998. The material on
robot soccer originated with the KAIST postgraduate theses of Dong-Han
Kim (2003,1998), Yong-Jae Kim (2003), Hyun-Sik Shim (1998), Mun-Soo
Lee (2000), Heung-Soo Kim (1997) and others, together with joint publica-
tions with the first author. The experimental robot soccer system program
for Small League MiroSoT that supplements this book has been developed
with the help of many students in the first author’s Robot Intelligence Tech-
nology (RIT) Lab at KAIST, while the simulator package for Large League
SimuroSoT has been developed by Bing-Rong Hong’s research team at Harbin
Institute of Technology, P.R. China. Both are available for free download from
the FIRA website http://www.fira.net.
Soccer Robotics is written as a textbook for practical courses at the un-
dergraduate level, and up to the first-year graduate level. It is useful for
researchers and practising engineers interested in trying out new techniques
in the domain of robot soccer. This book is also suitable for anyone interested
in learning and developing robot soccer systems for edutainment purposes.
For those interested in participating in either the MiroSoT or SimuroSoT
categories of the annual FIRA Cup and other robot-soccer championship
events, the material in this book will provide a firm foundation for the devel-
opment of robot soccer systems to competitive standards. The book will be of
interest to scientists, engineers and students in a variety of disciplines besides
AI Robotics, where the use of robot soccer as a test bed is relevant: sensors
(including computer vision), control, communication, multiagent systems and
artificial intelligence.
To review the chapters briefly:
Chapter 1 defines the multi-agent framework of soccer robotics in terms of
the three commonly accepted primitives of AI robotics, namely, SENSE,
DECIDE and ACT. The goals of soccer robotics in research and educa-
tion of intelligent multi-agent robotic systems are explained. The various
categories of robot soccer created by FIRA, an international regulating
body for robot soccer, are described. The classification of robot soccer
systems for MiroSoT is also examined.
Preface XI
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IX
1. Soccer Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Agents, Multi-agent Systems, and AI Robotics . . . . . . . 1
1.1.2 Cooperative Robot Teams . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.3 Domain Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.4 Robot Soccer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 The Goals of Soccer Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 Test Bed for Robotics Research and Development (R&D) 8
1.2.2 Educational Tool for AI Robotics . . . . . . . . . . . . . . . . . . . 9
1.2.3 FIRA Robot World Cup . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.4 Technology Transfer to New Useful Tasks . . . . . . . . . . . . 10
1.3 Fundamental Motion Benchmarks for Robot Soccer . . . . . . . . 11
1.3.1 Striking the Ball . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.2 Passing the Ball to Another Robot . . . . . . . . . . . . . . . . . 12
1.3.3 Striking a Moving Ball . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.4 Passing a Moving Ball to a Moving Robot . . . . . . . . . . . 12
1.3.5 Dribbling the Ball Past Obstacles . . . . . . . . . . . . . . . . . . 12
1.4 Categories of Robot Soccer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.4.1 MiroSoT and NaroSoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.2 SimuroSoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.4.3 RoboSoT and KheperaSoT . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4.4 HuroSoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5 The MiroSoT Robot Soccer System . . . . . . . . . . . . . . . . . . . . . . . 19
1.6 Classification of MiroSoT Robot Soccer Systems . . . . . . . . . . . . 22
1.6.1 Command-Based Robots . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.6.2 Action-Based Robots .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
XIV Contents
3. How to Sense?
Use Computer Vision Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2 Vision Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2.1 Computer Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2.2 Vision System Operations . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.2.3 Sampling, Pixel, and Quantization . . . . . . . . . . . . . . . . . . 74
3.2.4 Gray Scale, Binary, and Colour Images . . . . . . . . . . . . . . 74
3.2.5 Colour Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3 Binary Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3.1 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.2 Computing Geometric Properties . . . . . . . . . . . . . . . . . . . 80
3.3.3 Labelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.3.4 Labelling Algorithm 1: Recursive . . . . . . . . . . . . . . . . . . . 87
3.3.5 Labelling Algorithm 2: Sequential, 4-Connectivity . . . . 87
3.3.6 Size Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.4 Vision System For MiroSoT Robot Soccer . . . . . . . . . . . . . . . . . 90
3.4.1 System Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.4.2 Image and Physical Coordinates on MiroSoT Playground 91
3.4.3 Example 1: System Hardware . . . . . . . . . . . . . . . . . . . . . . 93
3.4.4 Example 2: Vision Processing . . . . . . . . . . . . . . . . . . . . . . 94
3.4.5 Example 3: Information Extraction . . . . . . . . . . . . . . . . . 97
3.4.6 Example 4: Window Tracking for Fast Vision Processing 100
Notes on Selected References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Contents XV
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
List of Figures
4.1 A hybrid control architecture for robot soccer (MiroSoT Category) 105
4.2 Situational problems encountered by role-level assigner . . . . . . . . . . 107
4.3 Basis areas of manoeuvre for zone defence strategy (Small League
MiroSoT Category) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.4 Formations for Middle League MiroSoT . . . . . . . . . . . . . . . . . . . . . . . 110
4.5 Wandering action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.6 SweepBall action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.7 Shoot action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.8 Cannon Shoot action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.9 Position To Shoot action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.10 PushBall action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.11 Position To PushBall action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.12 BlockBall action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.13 Block diagram representation of a PID controller . . . . . . . . . . . . . . . 118
4.14 Robot soccer situation: A robot (in white) should kick the ball
(round) avoiding a opponent robot (in grey) . . . . . . . . . . . . . . . . . . . 120
4.15 Robot modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.16 Available velocity region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.17 Component forces for generating a potential field . . . . . . . . . . . . . . . 127
4.18 A potential field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.19 A univector field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.20 Univector field for obstacle avoidance by a point object . . . . . . . . . . 129
4.21 Modified univector field for obstacle avoidance by a robot while
moving towards a target point g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.22 Phase portrait of a limit-cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
4.23 Navigation using the limit-cycle method . . . . . . . . . . . . . . . . . . . . . . . 133
4.24 Multiple obstacle situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.25 Decision of rotational direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.26 Navigation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.27 Local minima with two overlapped obstacles . . . . . . . . . . . . . . . . . . . 138
4.28 Extended navigation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.29 Robot soccer example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
A.1 Using CCP in PWM mode for motor driving and USART for data
reception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
A.2 Switch for assigning Team ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
4.1 Robot primitives and hierarchy levels defined for robot soccer
(MiroSoT Category) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.1 Program functions for robot soccer system (MiroSoT category) . . . 210
1. Soccer Robotics
1.1 Introduction
Soccer robotics is an emerging field that combines artificial intelligence and
mobile robotics with the popular sport of soccer. In essence, it studies how
mobile robots can be built and trained to play a game of soccer. It arises
out of a need to find a domain that can serve as an integrated framework
for the complementary purposes of research and education in multi-agent
robotics. Indeed, robot soccer, a roboticized version of human soccer, has
gained worldwide acceptance as an interesting and challenging domain for
studying and investigating a large spectrum of issues of relevance to the
development of complete autonomous robots in multi-robot systems.
Since its official founding in 1997, the Federation of International Robot-
soccer Association (FIRA, http://www.fira.net), an international non-profit
regulating body for robot soccer, has been actively promoting the devel-
opment of soccer robotics. This chapter presents the motivations and an
overview of the FIRA framework for soccer robotics as a subfield of AI
robotics. More specifically, it presents the ‘what’ of intelligent multi-agent ro-
botic systems and robot soccer systems. The basic terminology and concepts
used in AI robotics and multi-agent systems are defined. The domain charac-
teristics inherent in robot soccer, and of significant relevance to the study of
multi-agent robotic systems, are examined. The goals of soccer robotics in re-
search and education of intelligent multi-agent robotic systems are explained.
The various categories of robot soccer created by FIRA are described. The
classification of robot soccer systems for the Micro-Robot Soccer Tournament
(MiroSoT) is examined. MiroSoT is an important category of robot soccer,
especially from the perspective of edutainment, and is the focus of this book.
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 1-26, 2004
Springer-Verlag Berlin Heidelberg 2004
2 1. Soccer Robotics
teams. Ideally, a robot team should satisfy the system design requirements of
robustness and fault tolerance, reliability, flexibility or adaptability, coherence
and scalability.
In this section, we examine each requirement to emphasize its significance
in robot teams.
Robustness and Fault Tolerance. Robustness refers to the ability of a
system to gracefully degrade in the presence of partial system failure. The
related notion of fault tolerance refers to the ability of a system to detect and
compensate for partial system failures. To achieve robustness and fault tol-
erance, cooperative teams need to minimize their vulnerability to individual
robot outages.
To achieve this design requirement, first, one must ensure that critical
control behaviors are distributed across as many robots as possible rather
than being centralized in one or a few robots. This complicates the issue of
action selection among the robots, but results in a more robust multi-robot
cooperation system since the failure of one robot does not jeopardize the
system’s objective entirely.
Second, one must ensure that an individual robot does not rely on orders
from a higher-level robot to determine the appropriate actions it should em-
ploy. Relying on one or a few coordinating robots makes the team much more
vulnerable to individual robot failures. Instead, each robot should be able to
perform some meaningful tasks, up to its physical limitations, even when all
other robots have failed.
And third, one must ensure that robots have some means for redistrib-
uting tasks among themselves when some robots fail. This characteristic of
4 1. Soccer Robotics
thus the team composition can vary from one team to another. The aim is
to have these robots perform reasonably well in their teams the very first
time they are grouped together, without requiring any robot to have prior
knowledge of the abilities of the other team members. However, over time, we
want a given team of robots to improve its performance by having each robot
learn how the presence of other specific robots on the team would affect its
own behavior. For example, a robot that prefers to clean floors but can also
empty the garbage, should learn that in the presence of another robot that
is only capable of cleaning the floors, it should automatically take the role of
emptying the garbage.
Coherence. Coherence refers to how well the team performs as a whole in
terms of whether the actions of individual agents are purposefully combined
toward some unifying objective. Typically, coherence is measured by criteria
such as the quality of the solution or the efficiency of computing the solution.
Efficiency considerations are particularly important in teams of heteroge-
neous robots whose capabilities overlap, in that different robots are able to
perform the same task but with quite different performance characteristics.
In a highly efficient team, the team robots select tasks such that the overall
team performance is as close to the optimal as possible.
A team in which robots pursue conflicting actions or duplicate one an-
other’s actions cannot be considered a highly coherent team. A coherent
team, however, need not be totally free of all possible conflicts. Rather, the
team robots must be able to resolve conflicts as they arise. As a simple ex-
ample, conflicts do occur whenever multiple robots physically share the same
workspace. Although they have the same high-level objective, they may at
times try to occupy the same position in space, giving rise to positioning
conflicts that need to be resolved.
Clearly, robot teams exhibiting low coherence are of limited use in solv-
ing practical engineering problems. Achieving high coherence is therefore an
important design objective in building cooperative robot teams.
Scalability. Scalability refers to the ease with which a team can admit more
team members so as to improve the overall team performance in working on
its problem tasks. As an example of a scalability problem, consider a situation
in which two robots, r1 and r2 , clean up some toxic wastes. During execution,
two robots, r3 and r4 , are added. The extra robots, r3 and r4 , know about the
existing robots, r1 and r2 . However, the existing robots do not know about
these extra robots. Thus, several problems can occur, such as a degradation
in team efficiency due to the interference among the robots and an increase
in both communication collisions and job reallocations. To cope with these
problems, considerations for scalability are necessary. In particular, the issues
of complexity that arise as the team size increases must be mitigated if one
is to produce a highly scalable team.
6 1. Soccer Robotics
Through the game of robot soccer, soccer robotics studies how multiple
soccer-playing robots on each team could be built to cooperate in an ad-
versarial environment to achieve specific objectives. The domain of robot
soccer has the following characteristics:
• Independent agents with the same well-defined high level objective of scor-
ing as many goals to win the match: teammates.
• Agents with conflicting well-defined high-level objective of counter-scoring
as many goals to win the match: opponents.
• A need for real-time decision-making.
• Sensor and actuator noise.
Note that in a competitive setting, the intermediate or low-level objectives of
teammates and opponents can differ indeterminately. The different low-level
objectives are usually associated with different roles such as defending and
attacking. A teammate or opponent can dynamically assume these roles in
accordance to its own team’s strategy.
1.2 The Goals of Soccer Robotics 7
The robots are assumed to have, at their disposal, the following resources:
• Sensors that provide partial, noisy information about the environment.
• The ability to process sensory information in order to update a world model
(of the environment).
• Noisy actuators that affect the environment.
• Low-bandwidth, unreliable (wireless) communication capabilities.
In order to cooperate well in such a domain, soccer robots must perform
real-time visual recognition and tracking of moving objects, collaborate with
teammates (to decide their roles of attack or defence in a dynamic game
situation), navigate in coordination with teammates and in counteraction
against opponents, and strike the ball in the goal-ward direction. All these
demand robots that are efficient (functioning under time and resource con-
straints), reactive and proactive (deciding what actions to take based on
strategic reasoning of the game situation, and perhaps learning and adapting
from experience), communicative (as part of collaborating and coordinating
with one another when deciding what actions will accomplish the low-level
objectives that are beyond individual’s capabilities) and autonomous (sens-
ing, deciding and acting as an independent system). The point to emphasize
is that all these capabilities must be well integrated into a single and complete
robot soccer system. Soccer robotics studies how such integrated robots can
be built, using different approaches from those employed in separate research
disciplines.
This book provides course material for developing a soccer team of inde-
pendent robots that can cooperate and work towards the ultimate objective
in a complex, real-time, noisy, and adversarial environment. As a quick in-
troduction, the system set-up considered is a robot soccer team that consists
of micro-robots, a global vision system, a communication module and a host
computer. Fig. 1.1 shows the hardware composition of the robot soccer team.
More information on this game is given in Section 1.4.1. The rules of the game
are given on the FIRA website http://www.fira.net. To build such a system
successfully, this book uses a common architecture and basic robot hardware
design, but emphasizes a robot soccer-programming framework on which to
learn how to write programs to incorporate intelligence into the system, i.e.,
to integrate the abilities in a soccer team of robots to play different roles and
utilize different strategies and control techniques in their behavior.
The goals of soccer robotics are closely linked to those of the Federation of
International Robot-soccer Association (FIRA), officially founded in June
19971 . FIRA actively promotes the game through organizing tournaments,
1
FIRA started as an organizing committee of the first Micro-Robot World Cup
Soccer Tournament held in November 1996 at KAIST, Daejeon, Korea.
8 1. Soccer Robotics
The current study and research into building intelligent multi-agent robotic
systems necessitate an integrated approach to problem solving in comput-
ing, science and engineering. Students in this area come from a broad base of
computer, control and electrical engineering, computer science, mathematics,
biology, physics and biomedical engineering and neuroscience. Robot soccer
not only provides an experimentation test bed for multi-agent robotic sys-
tems research, but also a useful education tool for practical courses in this
area. The domain of robot soccer is sufficiently complex yet accessible, with
standard game rules well-defined and regulated by FIRA. Technically, the
game of robot soccer makes heavy demands in all the key areas of robot
technology, namely, mechanics, control, sensors, communication, and intelli-
gence. It thus provides a sound educational, integrated project to progress
students to real-world problem solving. Students will have the unique oppor-
tunity to focus on an easily understood standard problem where a wide range
of these technologies would need to be developed, examined and integrated
in a multidisciplinary and teamwork setting.
The four key education areas involved are discussed below.
Integrated knowledge. Students become quickly aware in a project on ro-
bot soccer that specialist intellectual knowledge from many pure discipline
areas, such as mathematics, physics, electronics, computing, etc, must be
brought together into an integrated whole. Complex systems are indeed com-
plex, and to study their evolution and control requires an investment and
application of a vast amount of multidisciplinary knowledge, forcing a team-
work approach.
Teamwork. A project on robot soccer is undertaken typically by interdis-
ciplinary teams, rather than by a single individual. This introduces partici-
pating students to a teamwork approach to problem solving which is quite
lacking in many current educational systems that encourage individual prob-
lem solving within specific discipline areas.
Real world issues. Through their involvement in the development of their
multi-robot system, students are quickly brought into the realities of working
with real-time evolution of complex, nonlinear physical systems.
Unlike the many textbook problems studied and examined by students
in an undergraduate curriculum, either by theory or through computer sim-
ulations where information and modelling are exact, real world problems
are inherently complex systems where the modelling process is never exact,
input and output data have a certain degree of uncertainty, parameter mea-
surements may be imprecise and uncertain, and definitive control evaluation,
for example, is impossible to achieve.
10 1. Soccer Robotics
Computer programs that run simulation models with exact input and
output data very well may not, for example, run on a micro-robot due to its
limitations in onboard processor and memory requirements.
Students must learn to overcome these difficulties in practical applications
and students lacking this hands-on experience greatly underestimate these
real world issues.
Critical thinking and creativity. The involvement and participation in
building a robot soccer system brings to the student a sense of creativ-
ity and critical thinking necessary for student transition to the professional
worker/researcher in our technological world.
One basic goal of soccer robotics is to take the spirit of science and technology
in AI robotics to the laymen and the younger generation, worldwide. In line
with this, FIRA has its flagship event, the FIRA Cup, held annually since
1998. FIRA Cup has its predecessor in the Micro-Robot World Cup Soccer
Tournament, held in 1996 and 1997 at KAIST, Korea, and is an international
competition that seeks to fulfill the dual purposes of research evaluation and
edutainment (education plus entertainment). Current information on this
event can be found on the FIRA website http://www.fira.net.
The game of robot soccer allows researchers to discover and learn how to
get a team of robots to sense with acuity, decide collaboratively and act in
coordination within the limited context of a soccer game. The hope is that
it will be possible to use or modify the same techniques and technologies to
build robots that carry out other more useful tasks in industries. This should
extend eventually to robots capable of working for or with humans in their
envisaged roles as personal, service or field agents.
Below, we enumerate these roles in various human-oriented applications
which are currently subjects of intensive research and development, but which
could potentially develop into gigantic enterprises, as personal computer busi-
nesses are today, to serve the needs of our 21st century society:
1. Personal Robots: homeostasis and utility oriented, e.g. as household ser-
vants, subject tutors (education) and pets (entertainment).
2. Service Robots: occupation oriented, e.g., as hospital nursing and surgery
assistants, museum tour guides, restaurant waiters and service-on-demand
driverless taxis (intelligent transportation).
3. Field Robots: intensive-labour or hazardous-mission oriented e.g., as con-
struction workers, farmers and military agents.
1.3 Fundamental Motion Benchmarks for Robot Soccer 11
Central to these human-oriented applications is the need for ease and safety
in operating these robots after switching them on. They should be com-
manded intuitively even by non-experts via multi-modal interfaces such as
voice, speech, graphics and vision, and would dynamically adapt to ever-
changing environmental conditions. Personal and service robots should be
fail-safe while field robots should be survivable.
As a test bed that even laymen can easily identify with, we believe that
robot soccer can help fire the imagination in terms of scientifically imitating
human abilities, both mental and physical, leading to the creation of power-
ful technological ideas initially directed at developing a soccer team of fully
autonomous humanoid robots capable of winning against the human world
soccer champion team. It is envisaged that the technological know-how devel-
oped in the process could culminate in overcoming challenging issues facing
the design and development of personal, service and field robots. A lot of the
accumulated technology would have high potential for transfer, to eventually
realizing robotic agents capable of carrying out other more useful tasks in
human-oriented roles.
In other words, the robot should be able to start from anywhere to strike
a ball placed anywhere to make it go in any specified direction. This is a
basic requirement to play soccer competently. It is necessary for the most
elementary tasks such as kicking off, taking goal kicks and taking penalties.
The Moving Ball-Striking test requires a robot to move from (x, y, θ) to strike
a moving ball at (x′ , y ′ , θ′ ) to make the ball pass through (x′′ , y ′′ ).
This test reflects a real game situation, in which the ball is moving most
of the time, and sometimes it is moving quite fast, exceeding 1m/s. To hit a
moving ball, the vision system must be working well at the highest possible
image sampling rate, since the decision-making module has to know the speed
of the ball to predict where it will be when the robot hits it.
The Passing a Moving Ball test requires a robot at (x, y, θ) to strike a moving
ball at (x′ , y ′ , θ′ ) to be struck at (x′′ , y ′′ ) by another robot starting from
(x′′′ , y ′′′ , θ′′′ ) to pass through (x′′′′ , y ′′′′ ).
Perhaps the most challenging, this test requires a robot to manoeuvre with
the ball past a series of obstacles, without colliding with any one of them.
The robot needs to plan and move with the ball through a zigzag or winding
course that avoids obstacles. These obstacles simulate the opponent team
robots and they could be stationary or moving.
A basic test set-up, as shown in Fig. 1.2, has one test robot, one ball and
two stationary obstacles. One obstacle is placed directly behind the other, at
(xo1 , y) and (xo2 , y), with x01 < x02 . The Cartesian x-distance in between the
obstacles is just long enough to place two imaginary robots, rotating about
their individual centres, to form a straight line (through the centres of these
1.4 Categories of Robot Soccer 13
Obstacle1
(x 01, y)
Robot Obstacle2
(x, y, θ) (x02, y)
0 X
Fig. 1.2. Basic set-up for the ‘dribbling the ball past obstacles’ test
four objects). The test robot is placed at (x, y, θ), in front of the obstacle at
(xo1 , y), with x < xo1 ; the Cartesian x-distance in between them is just long
enough to place one imaginary robot rotating about its own centre to form a
straight line. All imaginary robots replicate the test robot. The ball is placed
directly in front of the test robot. The test then requires the robot at (x, y, θ)
to push the ball around and through the two obstacles in an ‘S’-like path to
a good posture behind the obstacle at (xo2 , y).
(a) MiroSoT
(b) NaroSoT
1.4.2 SimuroSoT
(a) SimuroSoT
(i.e., the coordinate positions and directions of move) of the robots and the
ball are passed as input to the client programs which then execute the nec-
essary team game strategy, and pass control information back to the server
computer that updates its monitor display accordingly.
Fig. 1.4 shows the screen snapshot of a simulation game in Middle League
SimuroSoT. A computer simulation game, this category is decidedly a com-
petitive test of complex strategy development (the DECIDE component)
using advanced AI techniques.
The Robot Soccer Tournament (RoboSoT) made its debut in 2001. Each
team can consist of one, or up to three robots of which one can be the
goalkeeper. The size of each robot is limited to a rectangular box with a
square base of 20cm × 20cm and a height of 40cm This differentiates from
a similar but one-a-side tournament, the Khepera robot Soccer Tournament
(KheperaSoT), which made its debut in 1998. Khepera is the name of a
commercially available robot; it is much smaller, has a vertical cylindrical
shape, and has also been used for other research and education purposes in
autonomous mobile robotics. Although KheperaSot is primarily intended for
Khepera robots, any similar cylindrical robot is allowed as long as its base
diameter does not exceed 6 cm. Fig. 1.5 shows the respective snapshots of a
game of RoboSoT and KheperaSoT.
A major difference of these two categories from the MiroSoT/NaroSoT
categories is that in the physical set-up, the overhead camera is abandoned in
favour of vision processing systems onboard each team robot. With this on-
board / distributed vision-based set-up, the environment becomes inaccessible
1.4 Categories of Robot Soccer 17
(a) RoboSoT
(b) KheperaSoT
in that each robot will always only have a partial view of the environment.
Therefore, this set-up offers a unique opportunity to promote research and
development in distributed (or local) vision processing as an equally impor-
tant aspect of robotic agents. These categories also require the team robots
to be endowed with the local capabilities to decide and act. In other words,
the SENSE-DECIDE-ACT components are (required to be) distributed
in the team robots, making them truly autonomous as individual agents.
Because of increased complexity, the level of play in these two categories,
as demonstrated in past international tournaments thus far, has been inferior
to the MiroSoT/NaroSoT categories, but they are much closer to the research
goals of intelligent multi-agent robotic systems.
Below, we briefly discuss some non-trivial problems of local sensing and
local decision-making problems not found in centralized vision-based systems
that teams in the FIRA RoboSoT and KheperSoT categories must contend
with.
18 1. Soccer Robotics
1.4.4 HuroSoT
Fig. 1.6 shows some robot participants at the first Humanoid Robot Soccer
Tournamnent (HuroSoT). A category at the stage of infancy, HuroSoT made
its debut in 2002. It is the only category in which the robots assumes the
primitive form of humans, from which the term humanoid is derived. These
robots do not move on wheels, but are biped (i.e., ‘two legged’), admitting
critical problems of dynamic motion control and balancing not found in the
other categories. The HuroSoT initiative aims to stimulate and promote re-
search and development in humanoid (biped) robotics.
Because of the current state of the art, this game is still quite far away
from an actual soccer game. Presently, the competition is organized as a series
1.5 The MiroSoT Robot Soccer System 19
(a) HuroSoT
of tests, including robot dash, penalty kick and obstacle avoidance. Robot dash
is a sprint event; in penalty kick, the robot participants have to kick a ball
into an empty goal and in obstacle avoidance, they have to avoid obstacles
simulating stationary opponent players.
These tests are to be seen as preparations for humanoid robot soccer in
subsequent years of development. The format of the HuroSoT will actively
evolve in tandem with the state of the art developments in humanoid robotics.
Although still a distant dream, it is hoped that the game will eventually evolve
into one with two competing humanoid robot teams playing a full game of
soccer.
Fig. 1.7 shows the general set-up for the Micro-Robot Soccer Tournament
(MiroSoT). The set-up is a combination of mobile (small-sized) micro-robots,
a vision camera overlooking the playground connected to a centralized host
computer, and a wireless communication module connecting the host com-
puter to the robots.
The fact that visual sensing is achieved by a video camera that over-
looks the complete playground offers an opportunity to get a global view of
the dynamic game situation. This set-up may have simplified the sharing of
information among multiple robots, but it still presents a real challenge for
reliable and real-time processing of the movement of multiple moving objects,
namely, the ball, the team robots as well as the robots on the opposing team.
In a well-defined processing cycle, the global vision system perceives the dy-
namic game situation and processes the image frames, giving the postures
20 1. Soccer Robotics
of each robot and the ball (the SENSE functionality); the decision-making
program module, given this information, uses its strategic knowledge to de-
cide what action each robot has to take next (the DECIDE functionality);
the intelligent control module, based on the selected action or sensed infor-
mation, determines the actuation commands which are communicated to the
individual robots that translate them into physical robot motion using their
resident actuation routines (the ACT functionality).
To adapt to the dynamic game situation, the team robots under con-
trol should always change their desired postures2 , and reactively determine
the appropriate paths towards these desired postures. In order not to miss
critical visual information that helps the team robots adapt in time to the dy-
namic game situation, the moving objects in the playground must be tracked
as closely and as accurately as possible, and this necessarily requires fast
sampling of the image frames of the game situation captured by the vision
camera. Clearly, the processing cycle time (of sensing, deciding, controlling
and communicating) must not exceed this sampling time in order not to miss
any image frame. To satisfy this, the processing cycle time must be kept
correspondingly short for a fast (image frame) sampling rate. Keeping this
processing cycle time short while attempting to increase the overall intelli-
gence of the system is a challenging constraint to handle in robot soccer, and
in real-time multi-agent robotic systems in general.
Table 1.2 maps the robot primitives of SENSE-DECIDE-ACT onto
the robot soccer domain and the possible system (hardware) entities; the
2
Of course, what is ‘desired’ is decidedly a subjective opinion of the system de-
signer. In general, such an opinion is coded in the system as a strategy formulated
in terms of the positions of the other team robots, the opponent team robots
and the ball.
1.5 The MiroSoT Robot Soccer System 21
Table 1.2. Robot primitives and system entities for robot soccer (MiroSoT Cate-
gory)
wireless communication that always exists between the host computer and
the micro-robots is implicit. Fig. 1.8 shows a typical block diagram of an
organization relating these primitives. There are many ways to design and
implement this organization. The flexibility is due to one salient feature of
the organization, namely, it does not enforce the cyclic operational sequence
of SENSE, DECIDE, ACT, but allows DECIDE to modify the SENSE
and ACT couplings as needed.
It is easy to see from the block diagram why intelligent control is some-
times called outer-loop control and actuation is sometimes called inner-loop
control. In outer-loop (robot) control, regardless of the techniques used, there
are conceptually two major steps, namely, desired posture generation and
posture control. In inner-loop control, encoders (together with some auxil-
22 1. Soccer Robotics
Intelligent Control
- -
REAL
Wheel Motor WORLD
velocities rotations
Encoders
Status of actions
iary devices) are needed to quantify the physical motion of the wheels (input)
and measure their current velocities (output) as feedback information; these
devices would be covered in Chapter 2.
Strictly speaking, an encoder is a motion sensor and thus comes under the
SENSE primitive. However, for conceptual neatness as laid out in Table 1.2,
and as it is better to present motion sensors as an integral part of a MiroSoT
robot hardware, we classify motion sensing as part of actuation under the
ACT primitive. The SENSE primitive is viewed exclusively as sensing the
real outer-world or environment in which the motors run.
The actual entity for each primitive depends on the system of play used
that we will elaborate next.
Command
Action
Posture
2.1 Introduction
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 27-70, 2004
Springer-Verlag Berlin Heidelberg 2004
28 2. Hardware Components
(c) Omni-directional
Fig. 2.1. Different types of ‘move by rolling’ mechanism for a mobile robot
tends to slip easily because of the small contact areas its wheels make with
the floor surface. Fig. 2.1(b) shows a caterpillar-based mechanism. A robot on
this mechanism can move in a straight line easily, but is unable to negotiate
sharp bends. Fig. 2.1(c) shows a 3-wheel mechanism, with the rim of each
wheel lined with small bar rollers placed parallel to the rotation axis. Unlike
the previous two mechanisms, a robot on this mechanism can move freely
in any direction with proper rotation of the three wheels. A robot with this
characteristic mechanism is called an omni-directional mobile robot. Such
a movement mechanism is, however, difficult to construct due to its struc-
tural complexity. The mechanisms shown in Fig. 2.1(a) and Fig. 2.1(b) are
the most commonly used in general, and are also popular with designers of
soccer robots for the FIRA tournament series (except HuroSoT).
Wheel Mobile Robots. In this section, we analyze how different wheel
assembly designs for a wheel mobile robot can affect the turning ability of
a robot that has a rigid body. A wheel assembly is a movement mechanism
or device which provides or allows motion between its mounted robot and
the surface, on which each wheel is intended to have a single point of rolling
contact. The mechanical flexibility (to move and turn) determines if the robot
is capable of attaining an intended posture, depicted in Fig. 2.2 and defined
by P as follows:
2.2 Mobile Robots 29
x
P = y , (2.1)
θ
where (x, y) is the coordinate position of the robot’s centre in the Cartesian
X1-Y1 frame, and θ, called its heading angle, is the angle of orientation of the
robot in X1-Y1 frame, defined by the angle the robot’s heading in the X2-
direction makes with the X1-axis. By convention, the heading angle increases
counter-clockwise.
Y1
Y2 X2
θ
X1
Fig. 2.2. The robot’s posture
In Fig. 2.3(a) is a robot with a unique ICR. The robot’s wheel assembly
is similar to that of an automobile; the robot can turn left about the unique
ICR-axis which exists on its left-hand side.
Fig. 2.3(b) shows a robot on two parallel wheels, with the centres of
these wheels aligned, resulting in a common surface normal. For this wheel
assembly, the unique ICR can exist at any point on this line, and is determined
(marked off on the line) by the turning speed induced by the difference in
the velocities of the two wheels.
2.2 Mobile Robots 31
Fig. 2.3(c) shows a robot with no unique ICR. Any robot with this wheel
assembly cannot move in any direction and always stays ‘locked’ in any po-
sition it is placed.
Fig. 2.3(d) shows an assembly similar to that of Fig. 2.3(b) but with
the wheels’ centres not aligned. The unique ICR in this case is, in principle,
located at infinity since the two parallel wheel surface normals are deemed to
intersect there. This implies that the robot cannot turn, but move forward.
When building a more compact robot (e.g. a NaroSoT robot), the wheel
assembly shown in Fig. 2.3(d) may be chosen. Robot turning is possible in
practice because the two wheels attached to the same mechanical body always
create some slip. The slip is due to ‘pull and push’ that result whenever the
wheels move at different velocities. But since the left side is not symmetrical
to the right side, an analysis to determine the unique ICR is difficult.
VL + VR ωL + ω R
ν= =r ;
2 2
(2.3)
VR − VL ωR − ω L
ω= =r ,
L L
where L is the distance between the two wheels. [x y θ]T 1 and [ν ω]T
have the following relationship:
1
For notational convenience, we also denote it by (x, y, θ).
32 2. Hardware Components
Y VL v
VR
θ
y
0 x X
Fig. 2.4. Kinematics of a robot
ẋ cos θ 0
ν
Ṗ = ẏ = sin θ
0 . (2.4)
ω
θ̇ 0 1
ν
U= (2.5)
ω
ẋ
H · Ṗ = sin θ − cos θ = ẋ sin θ − ẏ cos θ, (2.6)
ẏ
H · Ṗ = 0. (2.7)
2.2 Mobile Robots 33
ẏ
tan θ = . (2.8)
ẋ
Now, Eq. (2.4) has 3 component equations but only two input control
variables ν and ω (obtainable from Eq. (2.3), given the related wheel velocities
VR and VL ). The holonomic constraint is not an independent equation as it
can be obtained by combining the first two component equations of Eq. (2.4).
This explains why, in general, no control solution is guaranteed to move the
robot from a given posture (x, y, θ) to a desired posture (x′ , y ′ , θ′ ).
R
VL VR
ICR
R1 L
R2
VL = R2 ω, VR = R1 ω. (2.9)
Since L is the distance between the two wheels, it is easy to deduce from
Fig. 2.5 that:
L L
R1 = R − ; R2 = R + , (2.10)
2 2
where R is the turning radius of the robot.
Substituting R1 , R2 of Eq. (2.10) into Eq. (2.9) and eliminating ω, we
obtain the following formula for turning radius R:
L VL + VR
R= . (2.11)
2 VR − VL
The robot moves in a straight line if VL = VR , implying from Eq. (2.11) that
R −→ ∞. It turns about its own centre if R = 0, implying, from Eq. (2.11),
that VL = −VR .
34 2. Hardware Components
Consider the two-wheel robot shown in Fig. 2.6. Suppose we want the robot
to move from one position to another along a circular path. To analyze circu-
lar motion control, it is convenient to denote the relative coordinate positions
of the robot by (R, 0) and (R, ϕ), which specify its current and desired coor-
dinate positions respectively. ϕ is called the robot’s turning angle about its
unique ICR-axis.
D
R
ϕ
ω ω
ωR
ωL
t0 t1 t2 t3 Time t0 t1 t2 t3 Time
t3
D= ν dt
t0
t3
VL + VR
= dt
t0 2
t3
ωL + ω R (2.12)
= r dt
t0 2
t3 t3
1
= r ωL dt + ωR dt
2 t0 t0
1
= r (ωL + ωR ) (t3 − t0 + t2 − t1 ) .
4
By Eqs. (2.9) and (2.11), the turning radius R is given by:
L VL + VR L ωL + ω R
R= = . (2.13)
2 VR − VL 2 ωR − ω L
Hence, the desired turning angle ϕ (in radians) is determined by:
D r
ϕ= = (ωR − ωL )(t3 − t0 + t2 − t1 ). (2.14)
R 2L
Note that only when the ratio of the two wheel velocities is constant, such as
if t0 = t1 and t2 = t3 , will the robot move in a perfectly circular path.
As a final remark, cirular motion control is fundamental in some navi-
gation methods such as a recently proposed limit-cycle method; the essence
of this method comes from the observation that when a robot is moving to-
wards a specified position, it can avoid colliding with any obstacle by turning
clockwise or counter-clockwise around the obstacle. More will be said of this
method in subsequent chapters.
36 2. Hardware Components
a one-way link with a transmitter at the host computer. The rechargeable bat-
tery supplies power to the motor driving and other onboard logic circuitry.
The (gear box accompanying the) motor used for this robot design has a gear
ratio of 130 : 1 (motor rotations : wheel rotations).
2.3.1 Microcontroller
This section focusses on the microcontroller which is the main processing unit
in the robot hardware. We shall discuss the following aspects of a microcon-
troller:
• basic functions,
• supporting architecture,
• data and address buses,
• chip selector,
• clock and
• interfacing.
Finally, we briefly describe a microcontroller chip, and elaborate the use of
the microcontroller’s on-chip PWM (pulse width modulation) for actuating
the motors, and its on-chip serial communication interface for receiving data.
Basic Functions. There are 4 basic functions in a microcontroller, namely
reset, instruction fetch, instruction execution and service interrupts.
1. Reset: This function sets the program counter (PC) to the starting ad-
dress held in the reset vector. You may think of an address as a pointer
to a memory location. The PC holds the physical address of the next
instruction to be read from memory after the current instruction is exe-
cuted. It also initializes the other internal registers to default values.
2. Instruction Fetch: The number of machine operations required for a single
instruction fetch varies, depending on the type of microcontroller. In
general, fetching an instruction is divided into 4 basic operations. First,
the function writes the current PC value into the memory address register
(MAR), and hence onto the address bus. Second, it sends out a read
control signal on the control bus, upon which the requested data from
the addressed memory is output on the data bus. Third, it checks the
memory buffer register (MBR) to see if the requested data has been
latched into it. Fourth, it reads the data from the MBR. Each machine
operation takes up one timing state, with a duration of 2 or 3 clock pulses.
3. Instruction Execution: This function stores the instructions read from
MBR (as done by the instruction fetch function) in the instruction reg-
ister (IR). It then decodes and executes the opcode contained in the IR.
An opcode is a machine code executable by the processor in the micro-
controller to carry out a corresponding instruction.
4. Service Interrupts: This function checks for interrupt requests. An inter-
rupt request occurs if a higher priority task arrives. Upon such a request,
it suspends the running program by storing the program’s states (current
memory address held in PC and other data) in a stack and then setting
the PC with the data in the interrupt vector; this data is the starting
physical address of the interrupt handling routine. After the interrupt
handling routine has been executed, this function restores the suspended
program’s states from the stack, which enable the instruction fetch and
execution functions to continue on the original program.
The cycle in which the basic functions except service interrupts are carried
out in a microcontroller is shown in Fig. 2.10.
Instruction Execution
Internal
registers’ Instruction Decode Execute
Reset Fetch
values
initialized
Restores
all program
states from
the stack,
Stores all Executes and
Requests program Gets start address of interrupt continue
interrupt states in interrupt handling routine handling with
service stack from interrupt vector routine program
execution
DataBus
ROM RAM
Microcontroller
Read/Write
AddressBus
Chip-
selector
AD0 A0 DataBus
AD1 A1
AD2 A2
AD3
8 bit A3
AD4 A4
AD5 Latch A5
AD6 A6
AD7 A7
Address Bus
Microcontroller ALE
A8
A9
A10
A11
A12
A13
A14
A15
To elaborate, in this scheme, when the ALE signal goes active, the 8-bit
latch is enabled such that its output signals, labelled A7-A0, ‘follows’ the
signals on the bus AD7-AD0 that contains the lower-byte address. When the
ALE signal turns inactive thereafter, the latch output holds this lower-byte
address; during this time, the upper-byte address remains on the bus A15-A8,
and the data bits are put on the AD0-AD7 bus, hence separating the data
signals from the address signals that ‘point’ at a memory location where the
data signals are being read from or written into.
Chip Selector. As mentioned, a chip selector enables only one device at
any instant, and disables all the other devices connected to it. Depending on
the microcontroller used, a chip selector need not be added externally in the
microcontroller board. Microcontrollers such as the AM188 and 80296 chips
do have a chip selector built in.
2.3 A Two-Wheel Command-Based Soccer Robot 41
Fig. 2.14 shows the inputs and outputs of a chip selector. As shown,
the chip selector receives at its inputs, control signals such as Read/Write
(RD/WD) and the address signals, usually the upper-byte A15-A8, from the
microcontroller, and generates chip-select (CS) digital signals at its output.
Each CS output is connected to exactly one device chip. So, for the ar-
chitecture in Fig 2.12, two CS signals are needed, one each for the RAM and
ROM. A CS output signal can be made to go active (in order to enable the
connected device) according to a logic CS equation expressed in terms of the
logic variables for the inputs. An example of a CS equation is shown in Fig.
2.15. The CS output signal is said to be active-low if it is functional (with
respect to exclusive chip select) at logic 0, and active-high if it is functional
at logic 1.
ANDoperator
!CNT1_CS = !A15 & A14 & A13 & A12 & A11 & A10 & A9 & !A8 & !RD;
0 1 1 1 1 1 1 0 RD
Active-low
= 7E00H~7EFFH
To ensure that only one CS signal is active at any instant, the logical
‘AND’ing of any two CS equations must always be at logic 0. As a design
principle, the CS equation, being expressed partly in terms of the upper-
byte address bits A15-A8, should define a unique upper-byte address or a
unique set of upper-byte addresses that enables the connected device such
as a ROM, RAM, PPI (programmable peripheral interface) or counter, and
thus uniquely situates the device in a block of full addresses A15-A0 in the
system (memory) address map (0000H-FFFFH). An example of an address
map ‘partitioned’ by a set of CS equations is given in Fig. 2.16.
The set of CS equations has to be designed and programmed to configure
the internal hardware logic of the chip selector. Generally speaking, a decoder
may be used as a chip selector but programmable array logic (PAL) and
42 2. Hardware Components
000H
ROM
PPI Chip-select
7D00H
7E00H
Counter1 Chip-select
7F00H
7FFFH
Counter2 Chip-select
RAM
FFFFH
Equations
!RAM_CS = A15;
!PPI_CS = !A15 & A14 & A13 & A12 & A11 & A10 & !A9 & A8;
!CNT1_CS = !A15 & A14 & A13 & A12 & A11 & A10 & A9 & !A8 & !RD;
!CNT2_CS = !A15 & A14 & A13 & A12 & A11 & A10 & A9 & A8 & !RD;
!ROM_CS = !A15 & PPI_CS & CNT1_CS & CNT2_CS;
generic array logic (GAL) are preferred because they are programmable and
physically more compact. Note however that a PAL can be programmed only
once, whereas a GAL is re-programmable. Programming methods for PAL
and GAL may differ depending on the compilers used.
Clock. The operations in a microcontroller are synchronized by running dig-
ital signals known as the clock. As a result, the processing speed of a micro-
controller depends on the frequency of the clock. There are two methods to
generate clock signals, as shown in Fig. 2.17.
C2
XTAL2 NC XTAL2
GND GND
The first method is a circuit (shown on the left-hand side) that consists
of a crystal used to drive the internal oscillation circuit. The frequency of
the crystal determines the clock speed. The second method (shown on the
right-hand side) uses an external oscillator as the microcontroller’s clock; this
method is used when a clock is needed to synchronize the operational timings
of two or more IC chips.
Interfacing. To add a peripheral device chip, the following aspects need to
be considered:
• number of address and data bits required,
• whether or not a device’s chip select signal (CS) is active-high or active-low,
• whether or not a device’s reset signal is generated by software or hardware.
Data Bus
Chip Output
RD
WR
CS
A0
A1
Reset
Address Bus
Consider the example shown in Fig. 2.18. This example shows a device
chip interfaced with an 8-bit data, a 2-bit address, two control signals RD
and WD, the reset signal and a chip select signal CS. The CS signal is from
the chip selector, and reset signal, used to initialize the device, as well as all
the other signals, are from the microcontroller.
As a note of precaution, care must be taken when interfacing with ana-
log devices because most analog devices tend to draw large instantaneous
currents. Drawing a large instantaneous current from a digital device could
damage the device. It is highly advisable to use only the driver (logic) circuits
recommended by the manufacturer for interfacing analog devices such as DC
motors.
Microcontroller PIC16C73/73A. A microcontroller chip is typically a
central-processing unit (CPU) integrated with modules for motion control
44 2. Hardware Components
N S
l
I
Stator Stator
F
Rotor
Fig. 2.21. Working principle of a DC motor
F = B · I · l. (2.15)
The direction of the force F follows Fleming’s left-hand rule: Stretch out the
thumb, first and second fingers of your left hand to be mutually perpendicular;
then, with the first Finger pointing in the direction of the magnetic Field and
the seCond finger in the direction of the Current, the Thumb is pointing in
the direction of the Thrust or force.
The opposite forces F rotate the rotor that ‘cuts’ the magnetic field con-
tinually, inducing, according to Faraday’s law, a back-emf2 in the rotor, which
can be shown to be given by
E = kbemf · ωM , (2.16)
where,
E: back emf,
kbemf : back emf constant, ωM : rotational velocity of (motor) rotor.
Hence, by Kirchoff’s voltage law,
V = E + I · ra , (2.17)
where
V : voltage applied,
I: rotor current, ra : armature resistance.
V is the voltage of the amplified PWM signals output by the motor driver;
the motor driving circuit will be discussed in Section 2.3.3.
Finally, the torque generated, i.e., the ‘angular’ force due to the opposite
forces F about the (rotor’s) axis of rotation, and the current flow I are related
by
2
The electromotive force ( abbreviated ‘emf’ ) is an archaic term for an induced
electric potential (i.e., an induced voltage).
2.3 A Two-Wheel Command-Based Soccer Robot 47
(τM + τr ) = kT · I, (2.18)
where,
τM : motor torque, kT : torque constant,
τr : frictional torque.
The parameters of Eqs. (2.16) to (2.18) are documented in the manufac-
turers’ catalogues on DC motors.
Torque. Following the combination of Eqs. (2.16) and (2.17) and Eq. (2.18),
we get two graphs of rotational velocity ωM versus motor torque τM and I
versus τM , as shown in Fig. 2.22.
Current (mA)
I
Velocity (rpm)
ωM
Torque (mNm) τM
The graphs show that there is a tradeoff between motor torque and ve-
locity; increasing the drive current I leads to an increase in the motor torque
τM , but a decrease in the motor velocity ωM , and conversely.
Most soccer robots move on two wheels. In order to drive the wheels, a
force FG is needed against a frictional force Fr , as depicted in Fig. 2.23.
ro G
Wheel shaft
Fr
The formulae for the wheel-driving force FG and frictional force Fr are as
follows:
48 2. Hardware Components
Fr = µ · m · g,
ro (2.19)
FG = Fr · ,
r
where
µ: frictional constant, m: mass of robot,
g: gravitational constant,
ro : radius of the shaft, r: radius of the wheel.
The torque τM generated by the motor and that τG applied to the wheel
(attached to it) are related by
τM 1
= , (2.20)
τG N · ηG
where
N : 1: gear ratio, ηG : gear (‘torque-transfer’) efficiency.
1 µ·m·g
τM = · ro . (2.22)
2 N · ηG
Eq. (2.22) shows that the motor torque τM is directly proportional to the
radius ro of the wheel shaft.
Power Consumption. It is important that the power supplied by the bat-
teries can last through one half of the game since it is convenient to change
batteries only during the half-time interval.
The battery power that is consumed by a DC motor is the product of the
voltage V across the motor and the current I flowing through it. Rewriting
Eq. (2.18), the current I is given by
τM + τr
I= . (2.23)
kT
Define Io by
τr
Io = . (2.24)
kT
Then Eq. (2.23) becomes
2.3 A Two-Wheel Command-Based Soccer Robot 49
τM
I= + Io . (2.25)
kT
Io is known as the no-load current as I = Io when there is no load, i.e.,
τM = 0.
Some DC motor catalogues do not specify the τr values; in such cases, the
no-load current Io can be easily obtained (by measurement) for Eq. (2.25).
Linear and Rotational Velocity. The gear ratio N : 1 relates the veloci-
ties of the motor and the wheel as follows:
νG ωG 1
= = , (2.26)
νM ωM N
where
νM : motor linear velocity, νG : wheel linear velocity,
ωM : motor rotational velocity, ωG : wheel rotational velocity.
Combining Eqs. (2.16) and (2.17) and rearranging, the rotational velocity
ωM in rotations per minute (rpm) is given by
1
ωM = · (V − I · ra ) rpm. (2.27)
kbemf
Since ωM is in rpm and not rad/s, the wheel (linear) velocity νG in cm/s is
given by
2 · π · ωG 2·π·r
νG = r · = · ωG cm/s, (2.28)
60 60
where r in cm is the radius of the wheel.
Substituting Eq. (2.26) into Eq. (2.28), we have
2·π·r 2·π·r
νG = · ωG = · ωM cm/s. (2.29)
60 60N
Size. As explained, DC motors and their compatible gearboxes and encoders
are individually available, but in only a few fixed sizes. As a result, placing
the motors, gearboxes and encoders within a confined cubic space of 7.5cm ×
7.5cm × 7.5cm for a MiroSoT robot becomes a challenging mechanical design
problem. No systematic approach exists, and this design is, at best, done with
a lot of engineering ingenuity. Fig. 2.24 shows two sample designs that meet
the size specifications for MiroSoT. The design on the left-hand side uses
tendons (timing belts or chains) wrapped around a pair of wheels on each
side; the design on the right-hand side uses gears to connect the gearbox (or
gear head) to the encoder on each side.
50 2. Hardware Components
Criterion Requirement
Let’s consider Escap’s 16C18-205 DC motor and B16C27 gearbox that have
the characteristics as shown in Table 2.1.
Measuring Voltage V 12
No-load Speed rpm 17300
Stall Torque mNm 1.2
Average No-load Current Io mA 9
Typical Starting Voltage V 0.15
Max. Continuous Current A 0.16
Max. Continuous Torque mNm 0.96
Max. Angular Acceleration 103 rad/s2 59
Back-emf Constant kbemf V/1000rpm 0.66
Torque Constant kT mNm/A 6.3
Terminal Resistance ra ohm 65
Ratio N 27
Efficiency ηG 0.73
Length (with 16C18) Lm 33.7mm
Mass m 6g
required torque is also much less than the stall torque of 1.2mNm, defined as
the minimum torque at or beyond which the motor will stall. Thus, in this
aspect of torque generation, the 16C18-205 DC motor is a good choice.
Using a B16C27 gearbox, the minimum current I|min required to produce
τM |min is
τM |min 0.1061mNm
I|min = + Io = + 9mA = 25.8mA.
kT 6.3mNm/A
This current of 25.8mA is much less than the maximum of 125mA that can
be supplied, and much less than the maximum continuous current of 160mA
allowed. Thus, in this aspect of current drive, the 16C18-205 DC motor is a
good choice.
With this 16C18-205 DC motor, the maximum motor rotational velocity
ωM |max is
1
ωM |max = · (V |max − I|min · ra )
kbemf
1
= · (12 − 25.8mA · 65ohms)
0.66V/1000rpm
= 15641rpm.
ωM |max
ωG |max =
N
15641rpm
=
27
= 579rpm.
This means that the robot can move up to the maximum value of 579rpm,
which is beyond the required maximum of 480rpm.
By the selection criteria, the combination of a 16C18-205 DC motor and
a B16C27 gearbox is an acceptable choice for the MiroSoT soccer robot.
The PWM (Pulse Width Modulation) method is one of the many methods
used to drive DC motors. Referring to Fig. A.1, the amplified PWM signal
V output by the motor driver can rotate (the rotor of) the DC motor at a
velocity ωM that is proportional to the applied voltage V , according to Eq.
(2.27).
Why is a motor driver needed ? To serve two purposes, as explained below.
2.3 A Two-Wheel Command-Based Soccer Robot 53
Vcc
+ -
NOT Gate
M
SW4 SW3
GND
V {vu {vmm
Average Voltage
Vp STOP
0 time
-Vp
{w~t
V Average Voltage
{vmm
{vu
Vp CLOCKWISE
0
time
-Vp
V {vu {vmm
Average Voltage
COUNTER-CLOCKWISE
Vp
0 time
-Vp
Vcc Vcc
+ - + -
M M
GND GND
Referring to Fig. 2.26 again, the average voltage V̄ of the amplified PWM
signal V is
2.3 A Two-Wheel Command-Based Soccer Robot 55
(n+1)T
1
V̄ = V dt = · (TON − TOFF ) · Vp
nT TPWM (2.30)
= (2d − 1) · Vp ,
TON
for an arbitrary n ≥ 0, where d = , 0 ≤ d ≤ 1, is the duty cycle
TPWM
of the PWM signal. Note that ideally, Vp = Vcc . This average voltage V̄
applied across the motor, with an average current I¯ flowing through it,
determines the resultant DC motor rotational velocity ω̄M according to
the following formula.
1
ω̄M = · V̄ − I¯ · ra rpm. (2.31)
kbemf
This formula follows directly from Eq. (2.27). Since |V̄ | >> |I¯ · ra |, V̄
(dominantly) determines, by its magnitude, how fast the DC motor ro-
tates, and by its sign (+ or -), determines its direction of rotation (clock-
wise or counter-clockwise, respectively).
Substituting Eq. (2.30) into Eq. (2.31), we obtain
1
ω̄M = · (2d − 1) · Vp − I¯ · ra rpm
kbemf
(2.32)
1
≈ · (2d − 1) · Vp rpm.
kbemf
Vp
ω̄G ≈ ωG |max · (2d − 1), where ωG |max = . (2.33)
N · kbemf
TON
d=
TPWM
(2.34)
W
= , where A = (PR2) + 1.
4A
With 0 ≤ d ≤ 1, and by substituting Eq. (2.34) into Eq. (2.33), we obtain
W
ω̄G = ωG |max · −1 rpm, where 0 ≤ W ≤ 4A. (2.35)
2A
−ωG |max if W = 0,
ω̄G = 0 if W = 2A,
ωG |max if W = 4A.
(W + △W )
ω̄G = ωG |max · −1 rpm, where 0 ≤ W ≤ 4A. (2.36)
2A
W △W
ω̄G = ωG |max · − 1 + ωG |max · rpm. (2.37)
2A 2A
specified
actual error
Dead Zone and Saturation. There is a dead zone range of ±zd , where
zd ≥ 0, at around W = 2A, within which the rotational velocity ω̄G is 0.
Motor saturation occurs at |ω̄G | = ωsat ≤ ωG |max . Both phenomena are due
to the inherent motor characteristics. Incorporating these into Eq. (2.36), we
have a piece-wise function for ω̄G (in rpm) as follows:
+
W − zd
if W + ≥ (2A + zd ),
min ω |
G max · − 1 , ωsat
2A
+
ω̄G = 0
+ if |W − 2A| ≤ zd ,
W + zd
max ωG |max ·
− 1 , −ωsat if W + ≤ (2A − zd );
2A
(2.38)
ω̄G +
min 2A · + 1 + zd , 2A + Wsat if ω̄G > 0,
ωG |max
W+ = 2A if ω̄G = 0,
ω̄
G +
max 2A ·
+ 1 − zd , 2A − Wsat if ω̄G < 0;
ωG |max
(2.39)
+ ωsat
where Wsat = 2A · + zd .
ωG |max
The graph for Eq. (2.38) or Eq. (2.39) is shown in Fig. 2.28.
The host computer program for a command-based robot soccer system
needs to compute W + using Eq. (2.39) and send W as wheel velocity data
to a respective team robot for its motor actuation.
2.3.5 Communication
ωG
ωGmax
M
W_S`
V
4
W+
V
OV M Tc VcBM V
W_S`
-ωGmax
+ +
Note: Only the range [2A − Wsat , 2A + Wsat ] of W + is valid.
Fig. 2.28. Graph showing the relationship between the wheel rotational velocity
ω̄G and the PWM data W + required to attain it
based and robotic agents communicate with one another across the medium.
When these agents communicate with one another, they exchange a series of
messages. To understand and act on these messages, the agents must agree
on what a message means. A protocol has its rules described in terms of the
format that a message must take and the systematic procedure in which agents
must exchange messages within the context of a particular activity, such as
sending and receiving messages across the medium. The message exchange
procedure attempts to ensure the electronic messages are correctly formatted
and transmitted from the originating agent to the destination agent. Agents
of different types are able to communicate with one another on a certain
activity - in spite of their differences - when they agree to use an appropriate
communication protocol that offers a standard format and message exchange
procedure.
IR: Communication Circuit and Protocol. To use IR as the means
of communication, two methods are available. One is the ASK (Ampli-
tude Shift Key) method and the other is the base band method. These
methods follow the standards set by the Infrared Data Association (IrDA,
http://www.irda.org/). IrDA is an international non-profit organization that
creates and promotes interoperable, low cost infrared data interconnection
standards.
In the ASK method, data is transmitted on a carrier signal and in the
base band method (IrDA1.0), it is transmitted by switching the transmitter
on-and-off. Fig. 2.29 illustrates how the serial data is transmitted using these
methods.
vGG
{
W X W W
k
\WWo¡
zGhzrG
Z
{
pkhXUWG X]
Referring to Fig. 2.29, when the data bit is logic 0, the ASK method
generates and transmits the corresponding digital signals at a frequency of
500kHz; when the data bit is logic 1, it does not generate any signal. In
the base band (IrDA 1.0) method, when the data bit is logic 0, the method
3
generates a corresponding pulse for the first of the one bit time Tb ; when
16
the data bit is logic 1, it does not generate any pulse.
In the rest of this section, we focus on the base band (IrDA1.0) method.
IR Receiver
(PIN)
IR Transmitter
(LED) Serial Input
Serial Output
Receiver Circuit
(Amplifier and Quantizer)
Transmitter Circuit
(LED DRIVER)
Decoder
(Edge Detector
and
Encoder Pulse Width Demodulator)
(3/16 th Pulse Width Modulator)
Fig. 2.30. Module block diagrams implementing the base band method for IR
communication
Fig. 2.30 shows the block diagrams of the transmitter and receiver
modules that implement the base band method. These module block dia-
grams, which are self-explanatory, can be realized using a HSDL7001 en-
coder/decoder IC chip and a HSDL1000 LED driver/receiver IC chip in a
generic module for data transmission (at the host computer) or reception (at
the team robot), as shown in Fig. 2.31.
We turn our attention to the experimental robot soccer system. This
system uses the base band method for IR communication. As explained on
page 280 in Section A.2, without carrier signals at different frequencies to
identify the teams, the two teams need to share the same IR communication
2.3 A Two-Wheel Command-Based Soccer Robot 61
g
Qf
vQt
Be
VCC
VCC VCC
U3
1 11 TXD
JP1 JP2 JP3 X1 15 16XCLK IR_TXD 10 RXD JP4 R1
X2 14 OSCIN IR_RCV
OSCOUT
JUMPER
JUMPER
JUMPER
JUMPER
TXDATA 2 R2 U1
RXDATA 3 TXD
RCV TXD 6 8 LEDA
13
RXD 4 TXD LDEA 7
9 POWERDN 7
RXD LEDC
NRST CLK_SEL 12
4 PULSEMOD 1 3
A0 2 CX1 VCC 5 VCC
5 16 CX2 GND
6 A1 VCC 8 VCC
A2 GND C1 C2 C3 HSDL1000
HSDL7001
R3 R4 R5
GND
C7
X1
Y1 R6 cBBB\
C8
vzfcvcB_`B
BBvfBBB
X2
cB
BB\
tzfcvcB^_B
BBtf BB
PC 1 v
v
PC 2
v c v d
kktBvBoBBB
Fig. 2.32. A game set-up for teams using IR base band communication
RNR
SUR
v
{
UW
NUW
UW
N[W
θ t Nθ r \x
θ t
θr
SWR
t
z
The simple procedure of the broadcast protocol used for data send and
receive is as follows:
• The team host computer transmits the messages continually to the team
robots through the transceiver.
• The IR receiver module onboard each of the 3 team robots receives and
decodes the messages as they arrive; the microcontroller stores these mes-
sages in its memory automatically via its configured hardware logic. The
microcontroller program examines every message stored for the following
conditions.
1. Start of message: the 0th -2nd bytes each contains the value 0xFF;
2. Team ownership: the 3rd byte contains its team ID;
3. Data delimiter: the 6th , 9th and 12th bytes each contains the value 0xAA;
• If all the conditions are true, it proceeds to extract the left-wheel and right-
wheel velocity data from the (3i + 4)th and (3i + 5)th bytes respectively,
where i ∈ [0, 2] is the unique robot ID.
Data Communication from Host Computer to Team Robots. Now,
consider a command-based robot soccer system with the host computer pro-
gram continually deciding and sending the velocity data W (for each wheel
of each PIC16C73/73A microcontroller-based robot) through the send and
receive communication protocol introduced above. Recall that W determines
the duty cycle as in Eq. (2.34).
In the experimental robot soccer system, the duty cycle of the PWM
signal has a resolution of at most 10-bits. i.e.,
v
tuOTUTeBnBf
jB uB
u JoczTUTK uB jB
e e
JvBcK JvBdK
u u
v v
r r
JSXeWWR wctvK JSXeWWR wctvK
o
JkBZRWSK
r
v
u
JSXeWWR wctvK
ktBv
o
Format Definition
• The first 3 bytes of 0xFF ( ‘0x’ denotes Hexadecimal) indicate start of message.
• Byte TID contains a team ID 0x0F or 0xF0 that denotes Team A or Team B,
respectively.
• Byte HLi and byte HR i
, for 0 ≤ i ≤ 2, contain, respectively, the unsigned velocity
data for the left and right wheels of team robot with robot ID i.
• Byte 0xAA indicates the end of velocity data pair for each team robot.
In other words, the velocity data W needs at least 10-bits but the protocol
defined admits only one byte for velocity data. The simple approach used is
for the host computer to send byte data H , given by
W
H= , rounded to the nearest integer ≤ 28 − 1 (i.e., 255).
4
64 2. Hardware Components
Data
FSK
Ao cos(2πf1 t) if data(t) = 1,
s(t) = (2.40)
Ao cos(2πf2 t) if data(t) = 0,
f1 = fc + △f and f2 = fc − △f
Fig. 2.37 show the appearance and physical dimensions of the real prod-
uct.
The pin configuration is listed in Table 2.3.5.
32.00mm
27.94mm
2.54mm
RXD RXE
TXD TXE
GND
22.86mm
30.00mm
ARFM-4XX
VCC ANT
3.57mm
GND AGND
GND AGND
2.03mm
16.2mm
2.5mm
0.8T
6.2mm
The choice of a power source is crucial from the hardware point of view.
Batteries constitute a significant percentage of the total weight of a MiroSoT
soccer robot. On the one hand, larger batteries usually supply higher voltages
(and hence currents), but this may be at the expense of attaining good speed
as they might contribute excessive weight and size to the overall mechanical
2.3 A Two-Wheel Command-Based Soccer Robot 67
design of the robot. On the other hand, smaller batteries might not be able to
supply the robot with a constant current that is sufficient to drive its logic and
motor circuitry, or that lasts long enough to sustain the required autonomy
68 2. Hardware Components
dummy header 0
HL 0
HR mode-0 1
HL 1
HR mode-1 2
HL 2
HR mode-2
(during the game). Besides, an under-weight robot that results might not
be favourable strategically since the fairly rugged nature of the game means
that it can be easily ‘pushed off’ its posture during the game. As a guideline
from the viewpoint of sustained robot autonomy, the batteries must supply
sufficient current through one half of the two-half game duration.
Functionally, a battery converts chemical energy into electrical energy.
There are basically two types of battery, namely, the rechargeable and non-
rechargeable (i.e., ‘used once’) type. For a robot soccer system, it is econom-
ically cheaper to use rechargeable batteries.
A: 1.8
Excellent energy density, high unit
Lithium No 300 3.0 C: 5 0.3
cost
D: 14
AA: 0.5
Low internal resistance, available
NiCd Yes 38 1.2 C: 1.8 0.009
from many sources
D: 4
AA: 1.1 Better energy density than NiCd,
NiMH Yes 57 1.3
4/3A: 2.3 expensive
High energy density but not widely
Zinc-air No 310 1.4
available, limited range of sizes
Carbon-
No 75 1.5 D: 6 Inexpensive but obsolute
zinc
Table 2.3.6 lists some commercially available batteries and their charac-
teristics. The capacity (or energy stored) of a battery is quantified in terms
of amp-hours (Ah) or milliamp-hours (mAh). For example, a battery with
500mAh means that it can supply 500mA for one hour. Therefore, if a supply
of 300mA is needed for 5 hours, a battery with 1500mAh is appropriate.
When the motors start up or reverse direction, they could draw large
transient currents from the power system, leading to a possibly large instan-
Notes on Selected References 69
taneous drop in the supply voltage. Thus, in using a common power system
for the logic circuits and the motors of a soccer robot, batteries with the right
capacities need to be chosen to ‘withstand’ the maximum transient currents
that the motors might draw. As a guideline, a rechargeable battery needs to
have a capacity at least one third of the (total) maximum instantaneous cur-
rent that can be drawn. For example, if the maximum instantaneous current
is 3000mA, the battery capacity must be at least 1000mAh. Otherwise, the
battery supply voltage will drop instantaneously and excessively when such
a current occurs, leading to battery ‘breakdown’ and/or logic circuit reset.
Overall, in selecting the batteries for a soccer robot’s power system, one
needs to balance the economic cost with the capacity (in mAh) and total
weight of the batteries.
Power Regulation. It is uncommon to always use IC chips such as those
of the 74HC series that can operate over a wide (discrete) range of voltages
(i.e., the ‘Vcc ’s). Most IC chips are powered by applying a fixed Vcc voltage.
Thus, for an electrical circuit system containing several IC chips powered by
different voltages (e.g. 5V and 12 V), power regulation is needed.
Two types of power regulator are commonly used; as illustrated in Fig
2.41, a linear regulator ‘steps down’ a supply voltage while a DC-DC converter
‘steps up’ the voltage; a DC-DC converter can also be used to reverse the
polarity of the voltage. Besides voltage level transformation, importantly, a
power regulator ensures its output voltage is constant regardless of some
inevitable fluctuations in the supply voltage or the load (i.e., the electrical
currents drawn by the system circuitry) due to, for example, the start up or
direction reversal of the motors in the soccer robot. The logic circuits need
to operate under a constant applied voltage, and hence this voltage is best
provided for by the regulator output (see Fig. 2.8).
The two companion books [14, 15] provide a reasonably good reference on
PIC microcontrollers and the auxiliary devices for various applications.
70 2. Hardware Components
VCC
7805
7~35 volts
Supply
IN OUT
GND
10u 47u
Battery Supply
POWER SW
470uH
5 volts
LX VOUT
MAX631 100u
2~5 volts
LBI GND VFB
GND
3.1 Introduction
In a robot soccer game, real-time information about the robots’ and the
ball’s dynamically changing coordinate positions and directions of move is
vital. In a MiroSoT set-up, such information is obtained using a real-time
vision system which consists of a vision algorithm continually processing the
digital images captured by a vision board that receives the analog images from
a video camera overlooking the complete playground. The FIRA MiroSoT
rules specify well defined colours for different objects in the playground, and
these are used as major cues for object detection. Vision processing in a robot
soccer system is therefore colour-based.
The performance of a vision system for robot soccer is gauged in terms
of its rate and accuracy in determining the objects’ coordinate positions and
directions of move in the playground. Assuming that a good and fast enough
vision processing algorithm exists, the former is limited to the image sampling
rate of its vision processing board (also called an image frame grabber); the
latter is limited by the quality of the digital images it processes.
Most commercial vision boards provide about 30 image frames per sec-
1
ond, implying that new image frames are sampled once in every 30 s (or about
33.3ms). Thus, in a robot soccer system, if the system processing cycle time
(of vision processing, deciding, controlling and communicating) does not ex-
ceed this frame sampling time, the maximum of 30 actuation commands per
second can be received by the team robots. But each sampled image frame
1
is interlaced with an even and an odd field, and each field is captured at 60 s
(about 16.7ms) intervals. So, if individual image fields are processed instead,
and the system processing cycle time does not exceed this field sampling
time, the maximum rate is doubled to 60 actuation commands received per
second. But this higher rate is achieved at the cost of lower accuracy because
individual image fields processed are clearly of lower resolution than an image
frame.
This chapter focusses on the (visual) SENSE primitive; it covers the
basics of vision processing systems most relevant to the study of robot soccer
systems, and presents how the postures of target objects in robot soccer
can be computed using centralized vision techniques. Real examples then
follow to study several specialized aspects of real vision systems as used by
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 71-101, 2004
Springer-Verlag Berlin Heidelberg 2004
72 3. How to Sense?
previous FIRA Cup MiroSoT teams. These examples highlight the practical
considerations in building a good vision system for a MiroSoT team.
rays under computer control. The host computer program then calls upon a
resident vision algorithm to process the stored digital images.
pixel a[1,1]
Pixel Column j m Columns
Row
Pixel a[i, j]
n Rows
Visible light has a wavelength ranging from 400nm to 700nm. Colours are
created by mixing different visible lights. A colour model (or colour space) is
a way of representing these colours and their relationship to one another.
The RGB Colour Model. The RGB colour space consists of the three
additive primaries: red, green, and blue. Spectral components of these colours
combine additively to produce a resultant colour.
In what follows, the colours are normalized (i.e., their values lie between 0
and 1.0). This is easily accomplished by dividing the colour by its maximum
76 3. How to Sense?
White
White = (1,1,1)
Magenta = (1,0,1)
Green = (0,1,0)
Black = (0,0,0)
Red = (1,0,0)
Yellow = (1,1,0)
Y 0.299 0.587 0.114 R
I = 0.596 −0.275 −0.321 G . (3.3)
Q 0.212 −0.528 0.311 B
Y 0.299 0.587 0.114 R
U = −0.169 −0.331 0.500 G . (3.4)
V 0.500 −0.419 −0.081 B
Y 0.299 0.587 0.114 R
Cb = −0.299 −0.587 0.886 G . (3.5)
Cr 0.701 −0.587 −0.114 B
Binary images have only 2 (gray level) intensity levels, 0 and 1. Compared to
gray scale level or colour images, binary images require less memory storage
and can be processed more quickly, but clearly contain a lot less information
of the scenes they represent.
However, many techniques developed for binary vision systems are also
applicable to vision systems which use gray scale or colour images. A con-
venient way to represent an object in a gray scale or colour image is to use
its mask. The mask of an object is a binary image in which the pixel values
5
Phase Alternating Line, another set of colour television standards used in West
Germany, The United Kingdom, parts of Europe, South America, parts of Asia
and Africa.
78 3. How to Sense?
of the object are 1 and those of the background are 0. After an object has
been ‘separated’ from the background, its geometric properties such as size,
position and orientation may be required for decision making. These features
can be computed from its binary image.
In other words, the many basic concepts and processing techniques of
computer vision are found in the simpler domain of binary image processing.
This section introduces and explains these concepts and techniques through
the processing of binary images.
3.3.1 Thresholding
1 if C(F [i, j]),
FT [i, j] = (3.6)
0 otherwise.
Depending on the application knowledge, the criterion function C(F [i, j]) can
take one of the following formulae.
1. F [i, j] ≤ Tu , for some upper bound threshold Tu .
This is used to separate darker colour objects (lower gray levels) from
the lighter colour background (higher gray levels).
2. F [i, j] ≥ Tl , for some lower bound threshold Tl .
This is used to separate lighter colour objects (higher gray levels) from
the darker colour background (lower gray levels).
3. T1 ≤ F [i, j] ≤ T2 , for some threshold range [T1 , T2 ].
This is used to separate objects with intensity values in the range [T1 , T2 ]
from the background known to be outside this range.
4. F [i, j] ∈ Z, where Z is a union of several disjoint threshold ranges.
This is a general thresholding scheme used to separate objects with in-
tensity levels that may come from several disjoint ranges from the back-
ground known to be outside all these ranges.
3.3 Binary Image Processing 79
Fig. 3.4 shows a gray level image and its resulting binary images obtained by
using different thresholds.
(a) Image
Fig. 3.4. A gray level image and its resulting binary images using different thresh-
olds
distribution plot showing the number of image pixels for each gray scale level.
The MiroSoT robot appears as a bright square in the image and it lies in the
gray level range [140, 180]. From the ‘valley’ between the two peaks in the
histogram, it is thus clear that setting the threshold T (either lower or upper
bound) to 135 will quite distinctly separate the square from the background.
\WWW
[WWW
ZWWW
YWWW
XWWW
W
WGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGXWWGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGX\WGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGYWW
One should note that although the thresholding techniques available are
useful tools, a proper threshold value is usually selected on the basis of human
experience with the application domain.
n
m
A= B[i, j]. (3.7)
i=1 j=1
to enclose the object image’ may be used, the centre of area of the object
image is a point and is relatively insensitive to noise in the whole image.
The following equation provides formulae to compute the centre position
(x̄, ȳ) of the object in binary image B with respect to the image Cartesian
x-y plane.
n m
i=1 j=1 j · B[i, j]
x̄ = ,
n mA (3.8)
i=1 j=1 i · B[i, j]
ȳ = .
A
To illustrate, consider the image in Fig. 3.6. Note the origin of the image
Cartesian x-y plane and the directions of the image x-y axes.
T
T
T
(x, y)
Fig. 3.6. An example showing the centre position (x̄, ȳ) of an image
In this example, A = 13. The centre position (x̄, ȳ) is calculated using Eq.
(3.8) as follows:
5×4+6×5+7×4
x̄ =
13
= 6.0.
4×2+5×3+6×3+7×3+8×2
ȳ =
13
= 6.0.
82 3. How to Sense?
In general, the calculated x̄ and ȳ may not be integers, and usually lies
between the column and row indices of two pixel positions. It does not imply,
however, that the calculated position is better than the resolution of pixel
positions.
Orientation. Calculating the orientation of an object is a little more com-
plicated than calculating its position. For an object which is circular, its
orientation is not unique. An object’s orientation is unique if it is elongated,
such as that shown in Fig. 3.7.
Line equation:
Array row i
Orientation:
x
Array column j
The angle θ between the thick line and x-axis in Fig. 3.7 is defined as the
object’s orientation. This thick line, called the object’s orientation line, is
the least squares fit of the positions of all the pixels of the object (or simply
called object points) in binary image B, i.e., it is the line that best fits the
object points in that the sum of the squared distances between the object
points and the line is minimized.
Formally, to find the equation of such a line from which the orientation
information about the object can be obtained, minimize λ2 , the sum of the
squared perpendicular distances of all object points from the line given by
n
m
λ2 = d2ij · B[i, j], (3.9)
i=1 j=1
where dij is the perpendicular distance from an object point [i, j] to the thick
line. To avoid numerical problems when the line is nearly vertical, represent
the line in polar coordinates:
3.3 Binary Image Processing 83
Referring to Fig. 3.7, θo is the orientation of the normal to the thick line with
the x-axis, ρ is the normal (and of course, shortest) distance between the line
and the origin.
The normal distance d between an arbitrary Cartesian coordinate (x, y)
and the line characterized by Eq. (3.10) satisfies the following equation.
Plugging Eq. (3.11) for every image point into Eq. (3.9) (the minimization
criterion), we get
n
m
λ2 = (xij cos θo + yij sin θo − ρ)2 · B[i, j], (3.12)
i=1 j=1
where (xij , yij ) is the Cartesian coordinate point of pixel position [i, j] (i.e.,
pixel at the i-th row and j-th column). The characteristic equation model
(ρ, θo ) of the line that best fits the object points can be obtained by mini-
mizing λ2 , done as follows:
In Eq. (3.12), set the derivative of λ2 with respect to ρ to zero. Then
solving for ρ yields
which shows that the regression line passes through the centre (x̄, ȳ) of the
object points.
Define (x̃ij , ỹij ) = (xij − x̄, yij − ȳ). Substituting these definitions and Eq.
(3.13) into Eq. (3.9), we get
where
n
m
a= (x̃ij )2 · B[i, j],
i=1 j=1
n m
b = 2· x̃ij ỹij · B[i, j],
i=1 j=1
n
m
c= (ỹij )2 · B[i, j].
i=1 j=1
3.3.3 Labelling
Given a binary image, we need to group all spatially close pixels of value
1 into connected components that distinctly represent the different objects.
This is done using a component labelling algorithm, which finds all connected
components in the image and assigns a unique label, usually an integer, to
all pixels in the same component. Fig. 3.8 shows an example of component
labeling of a binary image derived using some thresholding technique. The
dark pixels (see Fig. 3.8(a)) have ‘1’ values and the others have ‘0’ values.
There are a total of 4 connected components and they are given labeling
values of 1, 2, 3 and 4 (see Fig. 3.8(b)) through some labeling algorithm.
Computing the geometric properties (such as size, position and orienta-
tion) of each object, as has been covered in Section 3.3.2, is usually made as its
component is labelled. Other properties that can be computed are perime-
ter (number of pixels at perimeter) and compactness of the object image.
Compactness is defined by
4π · Area
Compactness = ,
Perimeter2
where ‘Area’ refers to the object image area defined by A in Eq. (3.7). A
circular object has a compactness value of 1; usually, objects with more com-
plex shapes have smaller values. If the shapes of objects are known, as in
robot soccer, compactness and perimeter values are helpful in finding and
recognizing them.
Computing object properties can be easily integrated into the labelling
algorithm. However, in order not to clutter the idea of component labelling,
this section only presents the intrinsic labelling algorithms.
The notion of spatial proximity has to be made precise before two labelling
algorithms can be clearly presented. For this purpose, some definitions are
introduced first.
3.3 Binary Image Processing 85
5555
555
666
66
777 88
7777 8888
7777 8888
i, j i, j
Following, we present two basic algorithms for finding and assigning con-
nected components in a binary image. One is recursive while the other is
sequential. A recursive algorithm is very time inefficient on a sequential
processor, and so is usually implemented on parallel processors. A sequential
algorithm takes less computation time and memory.
1. Initialize label L = 1.
2. Scan the binary image to find an unlabelled pixel p ∈ S and assign it
label L.
3. Recursively assign the label L only to all pixels q ∈ S that p ∈ S is
connected to.
4. Set L := L + 1.
5. Go to Step 2.
a) If one has been assigned6 with label (say, L1 ) and the other is not
labelled, assign label L1 to the pixel at [i, j].
b) If both have been assigned and with the same label (say L2 ), assign
label L2 to the pixel at [i, j].
c) If both are assigned but with different labels, then assign the label
of pixel at [i − 1, j] to that at [i, j], and record the two labels in the
equivalence table (as equivalent labels).
d) If both the pixels are not labelled, assign a new label L to the pixel
at [i, j] and record it in the equivalence table.
3. If not all pixels in S are labelled, go to Step 2.
4. Determine the lowest-valued label for each equivalent-label set in the
equivalence table.
5. Scan the image again and replace each label by the lowest-valued label
in its equivalent-label set.
Equivalent labels are different label values assigned to pixels of the same
connected component. Equivalent labels constitute an equivalent-label set,
and these sets constitute an equivalence table.
The algorithm requires two scans of the image. In the first scan (Steps
1-3), the connected components are found, and all those labels of the pixels
in one component are put into its equivalent-label set. In the second scan
(Steps 4-5), each label assigned to a pixel in the image is replaced by the
lowest-valued label in its equivalent-label set.
Given below is a pseudocode that implements Step 2 of the algorithm,
but with details of the equivalent label recording in Step 2 abstracted away.
It assumes that the elements of array label are all initialized to 0; this means
that all the pixels of a binary image are initially unlabelled.
// Begin
L=1;
If p[r][c] = 1
{
if(label[r-1][c] > 0 && label[r][c-1] = 0)
label[r][c] = label[r-1][c];
6
Note that only pixels which have unity values are labelled.
3.3 Binary Image Processing 89
Fig. 3.11. An example illustrating the workings of the sequential connected com-
ponents algorithm on an image
Fig. 3.12. A noisy binary image and its resulting image after application of a size
filter (Af = 8)
1. Calibrate the vision system for colour recognition of target objects with
respect to a colour model (we shall assume that the Y U V colour model
is used in our description):
• This amounts to determining and setting the intensity ranges [Ymin ,
Ymax ], [Umin , Umax ] and [Vmin , Vmax ] of every colour used for the
target objects under the Y U V colour model.
According to the FIRA MiroSoT rules, a different robot soccer team is
distinguished by a different team colour of either yellow or blue patches
placed on top of its team robots. Additional colours can be placed to
uniquely identify each individual robot of a team. The target objects in
robot soccer are the robots and the ball; thus the Y U V intensity ranges
of each different colour for the team ID patch, robot ID patch and the
ball need to be determined and set.
2. Run the vision system
When the vision system is running, it performs the following steps in a
cyclic fashion.
a) Obtain a binary image of target objects from a captured Y U V -colour
image.
• By applying a thresholding technique against the Y U V intensity
ranges.
b) Obtain a labelled connected components image from the binary im-
age.
• By applying a labelling algorithm to the binary image.
c) Remove noise in the labelled connected components image.
• By performing size filtering of the labelled connected components
image.
d) Determine the (centre) positions of the remaining connected compo-
nents.
• By applying Eq. (3.8) to each connected component (with other
components ‘virtually’ removed).
e) Recognize target objects from all the remaining connected compo-
nents and compute the postures of the target robots.
• Through various approaches that exploit the geometry of the target
objects (i.e., the robots and the ball) and the layout of the colour
patches on top of each robot.
During a soccer match, Steps 2a - 2e of vision processing (the SENSE
functionality) is executed continually and as shown in the flow chart of
Fig. 3.13, it works in a close loop in conjunction with the DECIDE and
ACT:Control functionalities.
Start_Game
VISUAL SENSE
Grab_Image
Scan_Image
Set_Colour_Ranges Label_Image
Locate_Objects
Loop
Stop_Game Interrupt ?
NO YES
DECIDE
and Stop_AllRobots
Generate CONTROL
Send_Command
Fig. 3.13. A high-level flow chart showing vision processing as a software compo-
nent of a robot soccer host-system program
and the corresponding point (xj , y i ) which is the image coordinate of the
pixel at [i, j] (i.e., in the i-th row and j-th column) in its n × m image (i.e.,
image of n rows and m columns of pixels). Note that in the image coordi-
nate point (xj , y i ), xj and y i correspond to column j and row i of the pixel,
respectively.
The physical and image coordinate frames are depicted in Fig. 3.14. The
two frames are displaced such that the top-left corner of the playground has
the image coordinate point (Jmin , Imin ), Jmin ≥ 0 and Imin ≥ 0; and the
bottom-right corner has the image coordinate point (Jmax , Imax ), Jmax ≤ m
and Imax ≤ n. Then, for an arbitrary image coordinate point (xj , y i ); the
corresponding physical coordinate point is (x, y), given by
xj − Jmin
x= × 150cm,
Jmax − Jmin
(3.19)
Imax − y i
y= × 130cm.
Imax − Imin
The intra-pixel spatial resolution of the image is defined by
3.4 Vision System For MiroSoT Robot Soccer 93
Image X - axis
[1,1]
j
I mage Y-- axis
( J min, I min)
Physic al Y - a xis
(0,
(0,130)
180)
( xj , yi )
180cm ( x,y )
( J max ,I max )
(220,
(150,0)0)
220cm
Physical X-- axis
[ n,m]
i
Fig. 3.14. On mapping the image and physical coordinate points in the playground
Lg
cm per pixel (column-wise),
Jmax − Jmin
(3.20)
Bg
cm per pixel (row-wise),
Imax − Imin
where the length Lg and breadth Bg of the playground are 150 cm and 130
cm, respectively. In general, the lower this intra-pixel spatial resolution is,
the higher the accuracy of the computed physical position (x, y).
Note that elsewhere in this book, we rely on context rather than notations
to indicate if a coordinate (x, y) is an image point or a physical point.
Most participating teams of the previous FIRA Cup’s used similar vision
hardware, differing only in the various vision software algorithms for comput-
ing the robots’ postures. There are many commercially available vision boards
that support very high sampling rates, but these teams used the relatively
cheaper boards that sample at 30 image frames per second.
94 3. How to Sense?
This example illustrates the steps of vision processing for a MiroSoT team,
up to and including Step 2d (see Section 3.4.1). The hardware in Example 1
is used.
1. Calibrate the vision system for colour recognition of target objects.
• The calibration steps of determining and setting the Y U V intensity
ranges for the different colours of the team ID patch, robot ID patch
and the ball are as follows:
a) Store a sampled image into memory, and display it on the host mon-
itor screen.
3.4 Vision System For MiroSoT Robot Soccer 95
b) Do a scan of the area where an object (team robot or ball) with the
target colour is.
c) Scrutinize the Y , U and V constituent colour values of pixels in that
area to determine the maximum and minimum values of each con-
stituent colour for the target colour. To lessen any deviation due to
noise, these maximum and minimum values are adjusted and read-
justed several times, possibly on a different team robot patch, until
each and every image pixel of the object (or that part of the object)
which has the target colour appears on the monitor screen in that
colour.
d) Store the finalized maximum and minimum values in computer mem-
ory.
e) Repeat Steps 1c-1d for each different target colour.
2. Run the vision system
a) Apply a thresholding technique against the Y U V intensity ranges.
In this process, the Y , U and V constituent values of every pixel in the
2-D colour image frame are checked if they fall in the corresponding
constituent ranges of any target colour. If so, the value of the pixel
is changed to a unity value, and its position is stored in association
with the matched target colour.
b) Apply a labelling algorithm to the binary image.
The labelling algorithm used modifies Step 2 of the sequential la-
belling algorithm introduced in Section 3.3.5. The modified step is
given below.
Image X - axis
[1,1]
j
Image Y - axis
[3,3]
Component 1
[4,13]
Component 2
Component 3
[9,5] [10,16]
Component 4
[13,21]
i : Representative pixel of each component
: Component pixel
3×3+4×3+5×2
x̄j1 =
8
= 3.875.
3×2+4×3+5×3
ȳ1i =
8
= 4.125.
The image pixel resolution is n × m, where n = 13 rows and m = 21
columns. Suppose that Imin = Jmin = 0, and Imax = 13 and Jmax =
21, i.e., the playground and its image boundaries coincide. Then the
physical coordinate (x̄1 , ȳ1 ) of the centre position of component 1 is
given by
3.875
x̄1 = × 150cm
21
= 27.67 cm.
13 − 4.125
ȳ1 = × 130cm
13
= 88.75 cm.
Robot ID
Robot ID
colour
colour
Team ID
colour
Team ID
colour
Two commonly used colour patch layouts for the top of a robot are as
shown in Fig. 3.17.
In Fig. 3.17(a), the placement of the robot ID colour square-patch on
the upper left quadrant (on the square top of the robot), with the team ID
98 3. How to Sense?
Y (220cm,180cm)
Playground π
p
Heading direction
4
( xr ,,yyr ) θqo θq
( xt , yt )
( x,yy))
(x, Orientation line of
colour
colour patches
(0.0) X
For the layout in Fig. 3.17(b), the robot’s orientation line can be drawn
by applying the general technique introduced in Section 3.3.2 (on page 82)
to the team ID colour hexagon-patch (placed obliquely on the square top
of the robot). The placement of the robot ID colour triangle-patch on the
upper right-hand corner, with the ‘base’ of the triangle-patch parallel to the
orientation line, then provides a means to help visually set the robot’s heading
direction.
In this example, the layout as shown in Fig. 3.17(a) is used.
The target objects are recognized from the labelled connected components
image as follows:
1. The Ball
Identified as the connected component with the most number of pixels.
3.4 Vision System For MiroSoT Robot Soccer 99
2. Team Robot
Identified from a pair of connected components, each representing a team
ID colour patch and a robot ID colour patch.
Let Dr and Dt contain the labels of components with robot ID colours
and team ID colours, respectively, and do be the intra-object distance;
do is user-specified. Then, the pairing is done using ‘Nearest Neighbour
Association’, as follows:
After associating the (labelled) connected components with the target ob-
jects, each target robot’s posture can be computed. Referring to Fig. 3.18,
the image coordinates (xr , yr ) and (xt , yt ) are, respectively, the centre points
of the robot ID and team ID square colour patches that uniquely identify the
team robot. The image coordinate (x, y) of the robot’s centre point can be
calculated as follows:
xr + xt
x= ,
2 (3.21)
yr + y t
y= .
2
To find the heading angle θ, the orientation angle θo should be computed first;
this can actually be determined based on the general technique introduced
in Section 3.3.2 (on page 82) by treating the team ID and robot ID colour
patches as one whole, and using some means for direction indication. In this
example, a simpler and perhaps more practical method is used by computing
θo as follows:
△yrt
θo = tan−1 , (3.22)
△xrt
where
Note that △yrt and △xrt can be positive or negative, and their signs together
define the heading direction of the robot.
Fixing θo ∈ (−π, π],
2π + (θo − π4 ) if θo ∈ (−π, − 43 π],
θ= (3.23)
θo − π4 otherwise.
Y rBBB
y
X X
W
W
V
V
U U
S T T
S
R
current frame, as in the basic method. This modification has been found to
be very effective.
The vision requirements for robot soccer have also been examined by other
researchers [17, 18, 19, 20].
Computer vision has been an area of active research. The book [21] pro-
vides a good introduction to vision. Aspects of digitization are covered in de-
tail in books on image processing [16]. There are several excellent references
on image processing [16, 22] and computer graphics [23, 24] that contain in-
formation useful to computer vision. The reader interested in knowing more
about pattern recognition should refer to [25]. There are also many good
books on artificial intelligence, including [26, 27, 28].
4. How to Decide and Act?
Use Intelligent Systems and Control
Techniques
4.1 Introduction
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 103-140, 2004
Springer-Verlag Berlin Heidelberg 2004
104 4. How to Decide and Act?
fgekfg
t
xkuwcnBugpug
c
cev
d
g
g
Fig. 4.1. A hybrid control architecture for robot soccer (MiroSoT Category)
In this level is a role assigner that chooses a role and an area of manoeuvre
in the playground for each robot as per the game situation and the role-level
strategy.
Two self-explanatory but significant situations that a MiroSoT robot sys-
tem designer would need to consider are depicted in Fig. 4.2.
The team robots should take appropriate actions according to their different
roles as attacker, defender and goalkeeper. At this level is an action selection
mechanism that selects an appropriate action for each team robot in an as-
signed role, according to the online game situation and adopted action-level
strategy.
The action-level strategy for each role can be designed as a supervised
discrete-event system modelled by a formalism such as Petri nets or automata.
Each state in the modelled strategy is a ‘decision point’ that should capture
106 4. How to Decide and Act?
Table 4.1. Robot primitives and hierarchy levels defined for robot soccer (MiroSoT
Category)
Defender Defender
Opponent team
Opponent team
Attacker Attacker
This level implements velocity control that drives the motors of each team
robot.
A two-wheel MiroSoT robot exercises a move behaviour by driving its
motors through a skilful combination of left-wheel and right-wheel velocities
input from the behaviour level. These possibly different and dynamically
changing reference wheel velocities continuously modify the directional turn
and motion of the robot. The robot is said to be proactive when it exercises
a move behaviour towards a desired posture, and reactive when it also has to
move in avoidance of an obstacle in its path.
Left-wing zone
Opponent goal
Team goal
Common area
Right-wing zone
Fig. 4.3. Basis areas of manoeuvre for zone defence strategy (Small League
MiroSoT Category)
the ball away from the keeper zone, while the other team robots could either
help with the defence by blocking the ball shot at goal or moving away in
cooperation so as not to hinder the action of the goalkeeping robot, thereby
reducing the risk of conceding a goal.
To reiterate, at the role level is a role assigner that determines a role and
a zone in the playground for each robot as per the game situation and the
role-level strategy.
Here, we consider a role-level strategy (used by the role assigner) that
addresses the problems of fault occurrence in a team robot and robot-blocking
by opponent robots. It is designed to work with the example action-level
strategy that we have described in the preceding section.
The strategy obeys the following general rule:
Whenever a team robot malfunctions or gets blocked by an opponent,
1. re-draw the zone boundaries;
2. assign a suitable role and a re-defined zone to each of the remaining
two robots.
Consider, for instance, a critical situation when the goalkeeping robot mal-
functions due to weak batteries. In this case, one of the two remaining robots
should be assigned the role of goalkeeping covering the keeper zone, while
the other assumes the role of an attacker covering both the left-wing and
right-wing as its re-defined zone.
Other related role-level strategies that can be incorporated include forma-
tion strategies that affect how the roles and variable zones are assigned to the
team robots; for example, in a Middle League (i.e., 5-a-side) MiroSoT game,
a ‘1-0-4’ formation strategy (a goalkeeper and four attackers) can be used
against a weak opponent team; other formations are ‘1-2-2’ (a goalkeeper,
two defenders and two attackers), ‘1-1-3’ (a goalkeeper, a defender and three
attackers) and ‘1-3-1’ (a goalkeeper, three defenders and an attacker). These
formations are depicted in Fig. 4.4.
In general, the selection of role-level strategies is decided by the human
manager. In MiroSoT, this can be done by implementing a ‘strategy data-
base’, from which appropriate strategies are selected and loaded prior to the
game and during the half-time interval, depending on how the human man-
ager reads the game.
and stop, pass, dribble and shoot for the other players. Of course, given that
the MiroSoT robots move on wheels, ball passing, dribbling and saving are
done by ‘bumping’ and blocking against the ball respectively, and these are
behaviorally less skilful (or more naive) than a human player does, or what
a humanoid player might be able to do in the future.
This section presents a set of basic actions organized into the base class
and three other classes according to the roles, namely, attacking, defending
and goalkeeping. What each action does is given, without any specific on how
it could do perform its function(s) for which there are possible many ways.
The purpose of this section is really to give the reader conceptual ideas of
the game-specific actions.
As the word base suggests, the actions in this class are meant for every team
robot, regardless of its role.
4.4 Design of Robot Soccer Actions 111
Stop Action and Wandering Actions. The Stop action is selected to halt
a team robot.
Opponent goal
Team goal
Action trajectory
The Wandering action is selected for a team robot to move to and fro
in parallel to the length of the playground (i.e., the x-axis) as depicted in
Fig. 4.5, whenever the robot has no other appropriate action to perform; this
happens especially when one other robot has been deemed more suitable to
go for the ball.
SweepBall and Position To SweepBall Actions. The SweepBall action is se-
lected for a team robot to attempt to take out a ball bumped into a corner
of the playground. The general direction of ball sweeping at each of the four
corners is depicted in Fig. 4.6.
If the robot’s posture does not permit it to perform the action, then the
Position To SweepBall action is selected first to put the robot in a new posture
in anticipation of performing the SweepBall action subsequently.
The actions in this class are defined for an attacking robot. These actions
bring about the robot behaviours of ball shooting.
Shoot Action. This action is selected for a team robot to strike the ball
through a given target point towards goal; the basic idea is to get the robot
move along a trajectory towards a desired point, with the ball in the trajec-
tory path, as depicted in Fig. 4.7. The desired robot posture (i.e., the desired
point facing an appropriate heading direction) would need to be determined
first. This action is selected for the robot whenever it is deemed to be in a
112 4. How to Decide and Act?
Opponent goal
Team goal
Action trajectory
TP
DP
Shootable area
Action trajectory
Cannon Shoot Action. This action is similar to Shoot, but is selected for a
team robot to strike the ball towards goal without a given target point. This
4.4 Design of Robot Soccer Actions 113
action is selected whenever the robot is facing (a sufficiently wide gap in) the
opponent goal, with the ball directly in-between, as depicted in Fig. 4.8.
Opponent goal
DP
Action trajectory
Position To Shoot Action. In attacking, the Shoot and Cannon Shoot ac-
tions are given the highest priorities, and so the conditions to select either
one are always checked for first. Whenever none of them can be selected, the
Position To Shoot action is selected for the team robot to move to a suitable
posture in anticipation of performing one of the shoot actions subsequently.
This is depicted in Fig. 4.9.
Opponent goal
TP
DP
Action trajectory
kw
TR i
Pushable area
Action trajectory
Fig. 4.10. PushBall action
TR
DP
B
Action trajectory
The actions of this class are defined for the goalkeeping robot to bring about
its behaviours of ball blocking, usually within the penalty box.
BlockBall Action. In goalkeeping, this action is of the highest priority, and
is selected for the goalkeeping robot to move parallel to its own team goal
line towards a desired point, whenever the ball is struck towards the team
goal. The ball velocity (i.e., speed and direction of move) has to be estimated
first from the images of previous and current ball positions over an image
sampling time period. Assuming that this ball velocity is constant, the time
required or the robot to move to the desired point to intercept the ball can
116 4. How to Decide and Act?
Team goal
DP
BIP
Keeper
B
be calculated, using which the desired robot velocity can be computed. The
ball intercept point is at the intersection of the predicted ball trajectory and
the dotted line next to the one along which the goalkeeping robot moves, as
depicted in Fig. 4.12.
KDefaultPosition and KDefendGoal Actions. The KDefendGoal and KDe-
faultPosition actions are selected for the goalkeeping robot to hold its position
against the attack of the opponent robots.
KDefendGoal is selected for the robot to move parallel to its team goal line
to a position that has the same y-coordinate as the predicted target point of
the ball, whenever the x-distance between the team goal and the ball is in a
predetermined range (for example, [30cm, 50cm]).
KDefaultPosition is selected for the robot to move parallel to its team goal
line to a predetermined position, whenever the x-distance between the team
goal and the ball is less than the lower bound of a predetermined range for
selecting the KDefendGoal action (for example, 30cm).
Keeper Attack and KNeedEscapeGoal Actions. The KNeedEscapeGoal ac-
tion is selected for the goalkeeping robot to exit from inside its own goal
whenever it enters it due to inexact visual information or pushing by an
opponent robot.
4.5 Control Basics 117
The Keeper Attack action is selected for the goalkeeping robot to push
the ball by itself whenever the ball has stayed close to the team goal for
a sufficiently long time in which no other team robot could push it away
towards the opponent’s half of the playground.
This section presents the classical techniques that can be applied at the be-
haviour and execution levels.
At the behaviour level, the reference (or setpoint) input is the desired
posture of every team robot; in other words, this level implements robot
posture control, i.e., the control purpose is to (try to) bring each robot to
its referenced position and heading angle by computing and recomputing the
desired wheel velocities of the robot. This is navigation control. It implements
the ‘Intelligent Control’ block of Fig. 1.8.
At the execution level, the reference inputs are the desired left-wheel
and right-wheel velocities of every team robot; in other words, this level
implements velocity control, i.e., the control purpose is to drive the motors
of each robot accordingly to attain the desired wheel velocities. This is motion
control. It implements the ‘Actuation’ block of Fig. 1.8.
The PID controller is the most common form of feedback in use today. The
structure of a PID controller is simple. PID stands for
P : Proportional control,
I : Integral control,
D : Derivative control.
These three terms treat the current control error (P), past control errors
(I), and predicted future control errors (D). To understand the operation of
a PID feedback controller, the three terms should be considered separately.
Proportional Control. Proportional control is a pure gain adjustment act-
ing on the error signal to provide the driving input to the process known
commonly as the plant. The P term in the PID controller is used to adjust
the speed of the system.
Integral Control. Integral control is implemented through the introduction
of an integrator. Integral control is used to provide the required accuracy for
the control system.
118 4. How to Decide and Act?
PID controller
R
Reference e(t) u(t)
r(t) y(t)
R Σ I
R
Σ Plant
T R
y(t)
Sensor
t
u(t) = KP · e(t) + KI · e(τ ) dτ + KD · ė(t), (4.1)
o
KP
KI = , where Ti is the integral (or reset) time constant,
Ti
3. KD is the derivative (D) gain given by
x(k) − x(k − 1)
ẋ ≈ , (4.2)
TS
where
• k is an integer and k ≥ 1,
• TS = tk − tk−1 (the sample interval in seconds),
• tk = kTS (for a constant sample interval),
• x(k) is the value of x at tk ,
• x(k − 1) is the value of x at tk−1 .
This approximation can be used in place of all derivatives in the controller
differential equations to convert them to a set of difference equations that
can be solved repetitively with time steps of length TS by a digital computer.
Differentiating the continuous PID control law (i.e., Eq. (4.1)) with re-
spect to time, we get
Thus, applying Eq. (4.2) to the I term (once) and D term (twice) of Eq. (4.3),
we get the digital form of the PID control law as the following difference
equation:
u(k) =u(k − 1) +
TS Td Td Td
KP 1+ + e(k) − 1 + 2 e(k − 1) + e(k − 2) .
Ti TS TS TS
(4.4)
vicinity of its final value; settling time refers to that needed to settle, i.e., for
the transients to decay away; and overshoot refers to the maximum amount
the system exceeds its final value divided by its final value.
It has been understood, that perhaps with the sharing of knowledge and
experience among many MiroSoT participants, such controllers have often
been tuned by trial and error, with reasonable success. However, it is often
best to resolve this matter as systematically as possible. Systematic methods
for automatic tuning are available, and we refer the reader to the textbook
[29] and the journal issue [30] for an excellent exposition to these methods.
Fig. 4.14. Robot soccer situation: A robot (in white) should kick the ball (round)
avoiding a opponent robot (in grey)
4.6 Unified Navigation Control 121
ẋ
Ṗ = ẏ = J(θ)U. (4.5)
θ̇
Y VL v
VR
θ
y Y
θe
θd
θ
0 x X X
(a) Kinematics modeling (b) Angle error
θe = θd − θ. (4.8)
ω = KP · θe + KD · θ̇e ,
4.6 Unified Navigation Control 123
where KP and KD are the proportional and the derivative gains for the
robot’s turning velocity ω, respectively, the following PD control law can be
used to move the robot:
VL = ν − KP′ · θe − KD
′
· θ̇e ,
′ ′ (4.10)
VR = ν + KP · θe + KD · θ̇e ,
where
L L
KP′ = · KP ′
and KD = · KD .
2 2
Now, a velocity input [VL , VR ]T inherently generates a centrifugal force on
a turning robot. A robot can slip or overturn if the centrifugal force generated
exceeds the limit of the frictional forces between the wheels and their points
of contact with the surface. To prevent such mishaps from happening, the
robot must satisfy the following kinematics constraints:
|VL | ≤ Vm ,
(4.11)
|VR | ≤ Vm ,
|νω| ≤ Rm , (4.12)
where Vm is the maximum speed for each wheel and Rm quantifies the max-
imum turn allowed. By restricting |νω|, the robot can be controlled with the
centrifugal force not exceeding the frictional forces.
Constraints (4.11) and (4.12) map out what is called the available velocity
region in the (VL , VR ) space, and is shaded grey as shown in Fig 4.16. Any
velocity input (VL , VR ) outside the grey region could cause the robot to slip
or overturn.
With velocity input P1 , a robot would move straight with maximum
speed; with input P2 , it would rotate with maximum angular velocity without
changing (the centre of) its position.
A straightforward approach to prevent slipping or overturning is to re-
strict the desired wheel velocities, computed in accordance to the PD control
law of Eq. (4.10), to within the square inside the grey region, as shown in
Fig 4.16. But this unnecessarily limits the speed performance of the robot.
Besides, when the robot is executing a turn at a relatively high speed, due
to PD control not following (or tracking) the desired heading angle on time,
subsequent ‘corrective’ control results in its rate of desired angle θd varying
significantly with alternate sign changes, thus causing the robot to oscillate
as it moves.
The next section presents a new control law that extends the maximum
speed limit of a robot to fully cover the available velocity region, without
causing the robot to slip or overturn, or to oscillate.
124 4. How to Decide and Act?
VL
P2 (V R2 , VL2 ) P1 (V R1 , VL1 )
|VL | <Vm
0 VR
|vω| <Rm
|VL| <V m
A New Approach. The error in angle θe between the robot’s current head-
ing angle θ and the orientation of the field univector θd is given as follows:
θe = θd − θ. (4.13)
The derivative of θe is
θ˙e = θ˙d − ω. (4.14)
By Eq. (4.5), θ˙d can be expressed by
∂θd ∂x ∂θd ∂y
θ˙d = + ,
∂x ∂t ∂y ∂t
∂θd ∂θd
= cos θ v + sin θ v, (4.15)
∂x ∂y
= φv (x, y, θ)v,
∂θd ∂θd
where φv = cos θ + sin θ.
∂x ∂y
The following control input component ω is considered:
ω = φv ν + KW sgn(θe ) |θe | (4.16)
Then, θe will become zero within the time T ≥ 2 θe (0)/KW [31]. It means
that once the robot’s heading direction has converged to the univector direc-
tion, the angle error θe becomes zero.
To satisfy the constraints to prevent slipping and overturning, the other
control input component ν should be bounded. From Eq. (4.16) and Con-
straint (4.11), the inequality is
L
|ν ± ω| ≤ Vm . (4.18)
2
Substituting Eqs. (4.16) to (4.18), the inequality becomes
L L L
|ν ± ( + KW sgn(θe ) |θe |)| ≤ (1 + |φv |)|ν| + KW sgn(θe ) ≤ Vm .
2 2 2
(4.19)
Solving the last two terms of Eq. (4.19), a sufficient condition satisfying
Constraint (4.11) can be derived as
2Vmax − LKW |θe |
|ν| ≤ . (4.20)
2 + L|φv |
From Constraint (4.12), the following inequality is obtained similarly.
|νω| = |ν(φv ν + KW sgn(θe ) |θe |)| ≤ |φv ||ν|2 + KW |θe | ≤ Rm . (4.21)
A sufficient condition satisfying Constraint (4.12) is
KW 2 |θe | + 4Rm |φv | − KW |θe |
|ν| ≤ . (4.22)
2|φv |
If the robot moves with velocity ν satisfying Conditions (4.20) and (4.22),
it can satisfy the Constraints (4.11) and (4.12), i.e., can move without slipping
or overturning.
Consequently, the control law is
ν =min(v1 , v2 , v3 ),
ω =φv ν + KW sgn(θe ) |θe |,
2Vm − LKW |θe |
where v1 = , (4.23)
2 + L|φv |
KW 2 + 4Rm |φv | − KW |θe |
v2 = ,
2|φv |
v3 = Kd ||p − g||.
In Eq. (4.23), v3 is used for the translational velocity ν to slow down to
zero near the destination g.
In the following, two navigation methods are introduced, with emphasis
on the generation step, i.e., on how the desired direction θd is determined.
126 4. How to Decide and Act?
The potential field method is a unified method for real-time robot naviga-
tion control [32]. This section provides a brief overview of this method before
presenting, in more detail, the univector field method that modifies and sig-
nificantly improves upon the potential field method.
The Vector Field Method. Fig. 4.18 shows a potential field. Graphically,
it consists of dash lines connected to equally-spaced points. Each line is a
field vector indicating a direction that points outward from the point it is
connected. In the potential field navigation method, these lines indicate the
directions in the vector field that a robot should follow under control. As
depicted in Fig. 4.18, the direction of each field vector is generated in pro-
portion to a resultant force, with an attractive force component from a target
position and a repulsive force component from an obstacle to avoid.
The component forces, F t and F io , exerted at an arbitrary point due to a
target position and one of the obstacles, respectively, are determined by the
following equations:
where
• dt is the distance between the point and the target position,
• nt is the unit vector in the direction from the point to the target position,
• Kt is a coefficient parameter for the target position,
• dio is the distance between the point and the (centre of the) i-th obstacle,
• nio is the unit vector in the direction from the i-th obstacle to the point,
and
• Koi is a coefficient parameter for the i-th obstacle.
The resultant force F can then be obtained by vector addition, as follows:
Resultant force :F = F t + F io . (4.25)
all i
Fig. 4.17 depicts an example of component forces at a point (where the centre
of the robot is) due to a target position (to move to) and an obstacle.
This method is simple and helps to control a robot in real time. However, it
has been found that when the robot’s constant velocity cannot be maintained
and the obstacle to avoid is too big, the robot under control is liable to break
into oscillations and its heading direction cannot be guaranteed at the target
point [33, 34].
4.6 Unified Navigation Control 127
Obstacle
do
Ft
Fo dt
F
Target
position
Fig. 4.17. Component forces for generating a potential field
nφ
g r
xbound
the circular boundary of the obstacle should be orthogonal to, and points
directly away from the boundary. A modified univector field satisfying such
requirements is shown in Fig. 4.21. Ro is the radius of the obstacle and M is
the width of the boundary margin.
Fig. 4.21. Modified univector field for obstacle avoidance by a robot while moving
towards a target point g
Note that the centre of a robot in univector field navigation would never
enter the margin surrounding a stationary obstacle. But should the robot
finds its centre position within the margin of a moving obstacle, due, for
instance, to its relatively lower speed against the oncoming obstacle, the
robot may be able to move away from the obstacle by following a sequence
of univectors within the margin, as depicted in Fig. 4.21. In other words,
the margin of width M is said to provide an anti-collision buffer against the
moving obstacle that it surrounds. Other than that the width of the margin
M must be set wider than half of the robot’s width L to account for the
robot’s size, the actual setting is decidedly a subjective opinion of the human
designer.
The resultant univector field for real-time robot navigation can be ob-
tained using optimization techniques. We defer to Section 5.6.4, in the next
chapter, a learning approach using evolutionary programming technique that
generates a sub-optimal univector field.
The essence of this technique comes from the observation that to avoid an ob-
stacle while moving towards a target posture, a robot could negotiate around
the obstacle, either clockwise (CW) or counter-clockwise (CCW). The ba-
sic idea is to continually adjust and readjust the radius of the motion circle
4.6 Unified Navigation Control 131
around any detected obstacle and decide the direction of obstacle avoidance,
considering also its target posture. This technique reactively generates a tra-
jectory path that a robot follows to avoid possibly multiple moving obstacles
while navigating in real time towards a target posture, using the limit-cycle
characteristics of a 2nd -order nonlinear function that models its motion.
Formally, consider the following 2nd -order nonlinear system [35]:
The derivative of V (x) is positive for V (x) < 1 and negative for V (x) > 1.
Hence, on the level surface of V (x) = c1 with 0 < c1 < 1, all the trajectories
will be moving outward, while on the level surface of V (x) = c2 with c2 > 1,
all the trajectories will be moving inward. This shows that the annual region
M = {x ∈ R2 |c1 ≤ V (x) ≤ c2 }
Then, all the trajectories will be moving inward, as shown in Figure 4.22(b).
The general form of Eq. (4.27) can be derived by replacing 1 with r as follows:
1
X2
0
x2
-1
-2
-3
-3 -2 -1 0 1 2 3
x1
X1
1
x2 2
0
X
-1
-2
-3
-3 -2 -1 0 1 2 3
x1
X1
The derivative of V (x) is positive for V (x) < r2 and negative for V (x) > r2 .
The general form of the limit-cycle is thus derived, using which we can ad-
just the radius and the direction of the limit-cycle while maintaining motion
stability. Importantly, it provides an easy programming basis for implemen-
tation, as would be elaborated in the next section.
The limit-cycle method generates a local navigation plan by applying the
limit-cycle characteristics exhibited in Eq. (4.31). The plan thus shows an
efficient way by which the robot can avoid obstacles without having to move
far away from them.
The Limit-Cycle Local Navigation. Figure 4.23 depicts the limit-cycle
method which can drive a robot towards the desired direction and avoid an
obstacle. At this time, the direction, either clockwise or counter-clockwise,
should be decided. Fig. 4.24 shows a situation in robot soccer where the
rightmost robot needs to avoid three obstacle robots A, B and C in moving
towards the target (ball).
Robot
Desired
direction Obstacle
rv
A
r n1
O d2
T21 O n1
C T11 l
Target
T22 B
rd1
T12
Od1
ax + by + c = 0. (4.32)
rv
(Q x, Qy) l
d (Rx, Ry)
Target
(Gx, Gy)
0 X
Fig. 4.25. Decision of rotational direction
rv = rr + ro + δ, (4.35)
136 4. How to Decide and Act?
where rr and ro are the radii of the robot and the obstacle, re-
spectively. Here, ro = r, where r is the radius of the real obstacle
if the obstacle is a disturbing one; ro = 0 if it is non-disturbing
or virtual. δ is a safety margin for collision avoidance. As the ro-
bot moves, the line l varies. So, repeat Steps 2 ∼ 4 until the the
destination is reached. Note that to obtain rv experimentally, we
enclose the obtained 2D image of the obstacle within an appro-
priate circle. The radius of the circle is ro . With rv and δ given
and ro measured, rv is obtained using Eq. (4.35).
For example, suppose, as shown in Fig. 4.24, there are three obstacles between
the robot and the target. The robot should move towards the target avoiding
these obstacles which are marked as A, B and C. First, a line l can be marked
from the robot to the target (Step 1). This line goes through two obstacles
B and C, so they are considered as Od1 , Od2 , respectively and obstacle A as
On1 (Step 2). Using the direction of line l through Od1 , the robot decides the
direction in which it should avoid obstacle B. The counter-clockwise direction
is chosen (Step 4) as shown in Fig. 4.26(a). It follows the chosen directions
until it avoids obstacle B. Once the robot passes obstacle B, the line l ceases
to go through obstacle B. Thus, obstacle B becomes On1 and obstacles C and
A become Od1 and On2 , respectively, (Steps 1 ∼ 2). Applying the limit-cycle
method again (to avoid obstacle C), the navigation path thus generated is as
shown by the solid line in Figure 4.26(b).
Extended Limit-Cycle Local Navigation. The robot in Fig. 4.27 is
avoiding obstacle A counter-clockwise by the limit-cycle navigation method.
Later, however, it moves clockwise beside the obstacle B and hence it will
get stuck in a local minima between obstacle A and obstacle B. To overcome
this problem, we have to add the following rule to Step 4.
The distance d from the centre of the obstacle Od to the line l can
be calculated as
aQx + bQy + c
d= √ . (4.36)
a2 + b2
If more than two variable obstacles are overlapped, they can be re-
garded as one obstacle and a new central position of obstacles can
be defined as
n n
1 1
Qx = Qxk , Qy = Qyk (4.37)
n n
k=0 k=0
where Qxk and Qyk are the x-y coordinate values of centre position of
the overlapped obstacles. With this (Qx , Qy ), new distance dfor all
overlapped Od ’s can be calculated. In the global coordinate OXY ,
the desired direction of the robot at each position can be calculated
from
4.6 Unified Navigation Control 137
A
Od2
O n1
C T11 4 1
l
Target
B
T12 rd1
Od1 2
A
Od1 2
O n2
T11
C 1
Target l
T12
4 B
rd1
O n1
d
ẋ = y + x(rv2 − x2 − y 2 ),
|d|
(4.38)
d
ẏ = − x + y(rv2 − x2 − y 2 ),
|d|
On1
Target A
O d1
So, with rv and d, navigation plan can be provided to avoid the obstacle
by Eq. (4.38). Finally, the robot can move towards the target avoiding three
obstacles as shown in Fig. 4.28.
On1
rv
Od1 B
Target A d
(Qx, Qy)
On2
Od2 B
Od1
A
Od2 B
Od1
A
O d3
On1
−1 (CCW ) if a virtual variable obstacle is on the left of the ball,
d=
1 (CW ) if a virtual variable obstacle is on the right of the ball.
(4.39)
For example, a situation as in Fig. 4.29(a) is modified to a situation where
it is assumed that two virtual variable obstacles are on either side of the ball
as in Fig. 4.29(b). Thus, the modified target is on the extended line from
the opponent goal to the ball and is adjacent to virtual variable obstacles
140 4. How to Decide and Act?
with the minimum variable radius rvmin . In the robot soccer, however, the
robot moves as fast as 150cm/s. It is meaningless to calculate the minimum
variable radius without considering the centrifugal velocity. The following
equation shows the non-slippery minimum radius of the robot:
mvc2
rvmin ≤ (4.40)
Fc
where m is the mass of the robot, vc is the centrifugal velocity and Fc is
the frictional force of the robot. It should be noted that the upper limit
of the minimum variable radius is constrained by the centrifugal velocity
and the frictional force of the robot. Since the frictional force is fixed and
measurable, the minimum virtual radius can be decided if the centrifugal
velocity is known.
In Fig. 4.29(b), On1 is on the left side of the ball, so d = −1 by the above
rule, for Od3 , d = 1. Then, the robot moves using the limit-cycle navigation
plan as shown in Fig. 4.29(b). Without modification, if there is no obstacle
A in Fig. 4.29(b), d is negative by the former limit-cycle navigation method.
So the robot may kick the ball to home side. Since the limit-cycle naviga-
tion method does not calculate all the trajectories in the current situation,
but only calculates the next trajectory of the robot using the robot’s cur-
rent relative positions of the target and any obstacle, this method generates
the navigation plan incrementally and “adapts” to the dynamically changing
environment.
The limit-cycle navigation method, as already described, can adjust the
direction and the safety distance for obstacle avoidance, and so is applicable
to robot soccer.
The book [29] provides an excellent coverage of PID control. Aspects of digital
PID and its implementation can be found in the books [29, 36]. A special
journal issue on PID control [30] focusses on the design methods and future
potential of PID control.
For an introduction to robot navigation, refer to the textbook [37]. The
univector method originated with the work of [38]. The limit-cycle navigation
method originated with the work reported in [39]. Several other navigation
methods have been reported for robot soccer, including [40] that proposes
an optimal path generating navigation method using a combination of a
geometric method and a fuzzy logic method optimized using evolutionary
programming.
The limitations of potential field navigation methods are discussed in [33,
34]. Earlier unified navigation methods include motor-schema [3], navigation
templates [41, 42] and artificial potential functions [43].
5. How to Improve Intelligence?
Use Soft Computing Techniques
5.1 Introduction
The field of robot soccer provides numerous opportunities for the application
of AI methods for game strategy development. As mentioned in the previous
chapter, good strategies are needed to decide the roles and actions of team
robots during the game. Chapter 4 has introduced a hybrid control archi-
tecture in which these strategies can be organized or integrated for proper
management and control. In general, building a proper strategy is best guided
by the intelligence aspects of search and evolution, knowledge representation
and inference and learning and adaptation. In this chapter, these aspects of
intelligence as needed by the DECIDE and ACT primitives and their im-
portance are first discussed. The basics of some widely known soft-computing
paradigms that make concrete (at least one of) these abstract aspects are then
introduced. They include the formalisms of Petri nets, Q-learning, neural net-
works, evolutionary programming and fuzzy logic. Along which, the use of
each paradigm for formulating strategies in robot soccer is motivated through
simplified examples taken from previous FIRA Cup MiroSoT teams that
demonstrate and emphasize its applicability in control, either at high-level
(also called supervisory) or low-level. More specifically, for each paradigm,
one or two examples are provided that address some key issues at specific
hierarchical levels of the hybrid control architecture introduced in Chapter
4. By this, however, we do not imply that these paradigms cannot be applied
at the other levels.
The reader interested in the performance evaluations of the example tech-
niques presented should consult the research papers referenced therein. As a
note of caution though, the performance evaluation results are often inconclu-
sive and are frequently based on limited empirical testing. As each paradigm
is an elaborate field in itself, the reader interested in these paradigms in
general should consult the many textbooks referenced.
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 141-204, 2004
Springer-Verlag Berlin Heidelberg 2004
142 5. How to Improve Intelligence?
We refer the reader to the book [28] for the many search algorithms using
various basic strategies.
Evolution. In the literature, a class of stochastic search methods inspired
by the process of natural evolution have emerged. These are called evolution-
ary algorithms, and are distinguished from classical search algorithms by the
evolutionary paradigm wherein a population of candidate solutions undergoes
iterative operations of variation and selection. There are three predominant
EAs, namely, evolutionary programming, evolution strategies and genetic al-
gorithms; more will be said about them in Section 5.6 in the next chapter.
A drawback of any evolutionary algorithm is that a solution is ‘better’
only in comparison to other, presently known solutions; such an algorithm
actually has no concept of an ‘optimal solution,’ or any way to test whether
a solution is optimal. For this reason, evolutionary algorithms are best em-
ployed on problems where it is difficult or impossible to test for optimality.
This also means that an evolutionary algorithm never knows for certain when
to stop, aside from the length of time, or the number of iterations or candidate
solutions, that is initialized allow it to explore.
Learning is perhaps the only way an agent can acquire what it needs to know
in the absence of complete knowledge about the environment that the agent
designer can build into the agent. Learning provides autonomy, and helps
the agent improve its behaviour through diligent study of its own experi-
ence. Here, no explicit distinction is made between adaptation and learn-
ing; instead, it is assumed that adaptation is covered by learning, in that
according to common usage, the term adaptation is only applied to those
self-modifications that enable an agent to survive in a changed environment.
There is a great variety in the possible forms of learning for an agent in
a multi-agent environment, and there are several key criteria that may be
applied in order to structure this variety. Two standard examples of such
criteria, which are well known in the field of machine learning (ML), are the
following:
1. The learning method or strategy used by a learning entity (a single agent
or several agents). The following methods are usually distinguished.
• rote learning (i.e., direct implantation of knowledge and skills without
requiring further inference or transformation from the learner);
• learning from instruction and by advice taking (i.e., operationalization
- transformation into an internal representation and integration with
prior knowledge and skills - of new information like an instruction or
an advice that is not directly executable by the learner);
• learning from examples and by practice (i.e., extraction and refinement
of knowledge and skills like a general concept or a standardized pat-
tern of motion from positive and negative examples or from practical
experience);
• learning by analogy (i.e., solution-preserving transformation of knowl-
edge and skills from a solved to a similar but unsolved problem);
• learning by discovery (i.e., gathering new knowledge and skills by mak-
ing observations, conducting experiments, and generating and testing
hypotheses or theories on the basis of the observational and experi-
mental results).
A major difference between these methods lies in the amount of learning
efforts required by them (increasing from top to bottom).
2. The learning feedback that is available to a learning entity and that
indicates the performance level achieved so far. This criterion leads to
the following usual distinction.
5.2 Intelligence Basics 145
• all available agents are involved, and the learning steps are ‘maximally’
distributed and parallelized. Of course, the degree of decentralization
may vary for different learning processes.
3. An agent’s involvement in a learning process. With respect to the impor-
tance of involvement, one can identify the following two extremes:
• the involvement of the agent under consideration is not a necessary
condition for achieving the pursued learning goal (e.g., because it can
be replaced by another equivalent agent);
• the learning goal cannot be achieved without the involvement of exactly
this agent.
Other aspects of involvement that could be applied in order to refine
this criterion are its duration and intensity. It also has to be taken into
consideration that an agent may be involved in several learning processes,
because it may pursue several learning goals.
4. The agent-agent and agent-environment interaction required for realizing
a learning process. Two obvious extremes are the following:
• learning requires only a minimal degree of interaction;
• learning would not be possible without extensive interaction.
This criterion could be further refined with respect to the frequency,
persistence, level, pattern and type of interaction.
Many combinations of different values for these criteria are possible. For
instance, one might think of a small group of agents that intensively interact
(by discussing, negotiating, etc) in order to understand why the overall system
performance has decreased in the past, or of a large group of agents that
loosly interact (by sometimes giving advices, sharing insights, etc) in order
to enhance the knowledge base of one of the group members.
The above criteria characterize learning in multi-agent systems at the
single-agent and the total-system level, and they define a large space of pos-
sible forms of multiagent learning. Each point in this space represents a form
of multiagent learning having its specific characteristics and its specific de-
mands on the skills and abilities of the individual agents.
(a) Structure
p2 t5 p6
t1
p1
p3 p4
t3
t2 t4
p5
(b) Graph
the supervisor, according to the game situation, assigns the role of attacking
or defending to each of the other two robots. To read the game situation, the
supervisor continually receives feedback information on the robots’ postures
and ball’s position.
Supervisor Model. The following places and transitions are defined for a
Petri net model C s of the supervisor:
T1
P1 T2 P2
P3
P5 P6
T3 T4 T5 T6
P4 P8 P7
T1
P1 P2
T2
P3 P5
P6
T3 T4 T5 T6
P4 P8 P7
(a)
T1
T1
P1 P2 P1 P2
T2 T2
P3 P3 P6
P5 P6 P5
T3 T4 T5 T6
T3 T4 T5 T6
P4 P8 P7 P4 P8 P7
(b) (c)
Fig. 5.3. A Petri-net supervisor for role assignment: transition firings and token
redistributions
152 5. How to Improve Intelligence?
towards it, it can attack more effectively than the other robot, and should
change its role to attacking if it is not already in. For the Petri net supevisor,
this has to be formalized as ‘liveness’ conditions for transitions T1 and T2 .
To do so for transition T1 , with i ∈ {1, 2}, let
1. di be the distance between robot i and the ball;
2. θi be the angle between the heading direction of robot i and the ball.
This is depicted in Fig. 5.4. Then, an example of the ‘liveness’ condition for
Robot 1
d1
θ1
Opponent goal
d2 θ2
Robot 2
The condition for T2 is similar, but with the two robot IDs swapped in
(5.1). All other transitions can fire immediately once enabled.
Supervisor
Fig. 5.5. A role-action structure for Petri net supervision and control
The example Petri net controllers for the defending and goalkeeping ro-
bots are given.
Defending Robot Controller. The principle to apply is simple and not
unlike human soccer play: the robot assigned to defend should kick the ball
away from its own team goal before any opponent robot can kick it.
Assuming that the left half of the playground is the opponent side, it is
reasonable that the defending robot must move to the right side of the ball
as soon as possible, so as to get behind the ball. Four simple situations are
assumed for the defending robot (controller):
(a) defending robot behind the ball,
(b) defending robot kicks the ball,
(c) self goal position, and
(d) defending robot in contact with ball and behind.
Fig. 5.6 depicts the four key situations. In situation (a), the defending
robot is in a probable position to kick, in situation (b) it is kicking the ball,
in situation (c), it is in front of the ball and facing its own team goal (self
goal position), so needs to be careful to avoid a self goal, and in situation
(d), it is in contact with the ball.
Defence Control. ‘Angle’ is used to refer to the angle between the heading
direction of the defending robot and ball. ‘Distance’ is used to refer to the
distance between the defending robot and ball in pixels. In what follows, the
values for ‘Angle’ and ‘Distance’ are as fixed by the designer.
In situation (a), the defending robot controller should command the robot
to move to the ball and kick it. In situation (b), if ‘Angle’ is above 45◦ , or
154 5. How to Improve Intelligence?
Team goal
Team goal
Ball
Ball
Defending Defending
robot robot
Team goal
Team goal
Ball
Ball Defending
Defending
robot robot
T6 : in self goal position, and then in contact with the ball and behind, and
T7 : away from the ball and behind, but still in a self goal position.
It should be clear that the purpose of such a controller is to keep track of and
influence the situational occurrences as modelled by transitions, according
to situations modelled by (tokenized) places that the defending robot finds
itself in. With the above defined places and transitions, the Petri-net graph
for the defending robot controller is arrived at, as shown in Fig. 5.7. The
‘liveness’ condition for each enabled transition can be easily formulated from
the descriptions above.
P1
P5
T1
P7 T3 T4
T2
P6
P2 T5 P3
T6
P8
T7
P4
Note that in the control execution of this Petri net controller, a token
would be put in an auxiliary place Pi , i ∈ {5, 6, 7, 8} provided the visual
feedback information asserts that the situation (or condition) the place char-
acterizes (or is predicated on) becomes true. Such places are said to be con-
trollable.
Goalkeeping Robot Controller. The principle of goalkeeping also follows
human soccer play: the goalkeeping robot should block or kick the ball away
from its own team goal. In this example, the goalkeeping controller is designed
to react to situations characterized according to the distance between the
team goal and the ball. The three situations considered are
(a) far distance,
(b) medium distance, and
(c) in goal area.
5.3 Petri Nets 157
In the following, the distance values for these situations are as fixed by the
designer.
‘Far distance’ means the distance between the goal and the ball is above
60 pixels, ‘medium distance’ means it is above 20 pixels but within 60 pixels,
and ‘in goal area’ means, it is below 20 pixels. A subroutine predictor()
predicts the ball direction using a linear equation. It is assumed that the
ball always moves in a straight line. Using the past four positions and the
current position of the ball, the predictor can derive an equation for the line
of movement of the ball using a curve fitting algorithm. Fig. 5.8 illustrates
the role of predictor().
Position “TARGET”
Goal keeper
Goalkeeping Control. The goalkeeping robot would only move within the
goal area, along a line parallel to the team goal line. The goalkeeping robot
controller performs the following:
(a) It commands the goalkeeping robot to move to the centre of the goal for
‘kick off’ when the game commences/resumes.
(b) It commands the goalkeeping robot to guard the ‘TARGET’ position by
moving to and staying in between the ball and the ‘TARGET’ position,
called the ‘TARGET’-intercept position, when the ball is in the oppo-
nent’s half of the playground, i.e., at a far distance from the team goal.
158 5. How to Improve Intelligence?
The ‘TARGET’ position is the point on the goal line where the predicted
path of the moving ball intersects. The predicted path is determined by
predictor().
(c) It commands the goalkeeping robot to move to the ‘ASSUME’ position
when the ball is in the team’s half of the playground but outside the
goal area, i.e., at a medium distance from the team goal. The ‘ASSUME’
position refers to the point of intersection of the goalkeeping robot’s pre-
determined path (which is parallel to the goal line) and the straight line
that passes through the current position of the ball and the centre of the
goal (line).
(d) It commands the goalkeeping robot to (try to) kick the ball away from
the team goal when the ball is in the goal area.
Goalkeeping Control Model. The following places and transitions are de-
fined for a Petri net model C g of the goalkeeping robot controller.
P5
T1 P2
P6
T2 T7
T3
P1 P7 P3
T4
T8
T5
P8 P4
P9
T6
5.4 Q-Learning:
A Model-Free Reinforcement Learning Method
Reinforcement learning is the problem faced by an agent that learns how to
act through trial-and-error interactions with a dynamic environment. It can
160 5. How to Improve Intelligence?
P5
T1 P2
P6
T2 T7
T3
P1 P7 P3
T4
T8
T5
P8 P4
P9
T6
(a)
P5
T1 P2
P6
T2 T7
T3
P1 P7 P3
T4
T8
T5
P8 P4
P9
T6
(b)
Fig. 5.10. A Petri-net control for robot goalkeeping: a transition firing and token
redistributions
5.4 Q-Learning: A Model-Free Reinforcement Learning Method 161
Environment
s i
Input
a
Agent
Reward r
(that is, i = s, implying the agent perceives the exact state of the environ-
ment).
The agent’s job is to find a policy π, mapping states to actions, that max-
imizes some long-run measure of reinforcement. Notice that after choosing
an action, the agent is told the immediate reward and the subsequent state,
but is not told which action would have been in its best long-term interests.
It is necessary for the agent to gather useful experience about the possible
system states, actions and rewards actively to act optimally.
5.4.2 Q-Learning
The task facing the agent is to determine an optimal policy, one that maxi-
mizes the cumulative discounted expected reward Rst (at ) for performing an
action at ∈ A(st ) at every state st ∈ S, modelled by
∞
Rst (at ) = E γ j rt+1+j , (5.3)
j=0
∗ π∗ ∗
V (s) ≡ V (s) = max Rs (a) + γ Pss′ [a]V π (s′ ) . (5.5)
a
s′
In other words, the Q(s, a) value is the expected discounted reward for exe-
cuting an action a at state s and following policy π thereafter. The objective
in Q-learning is to estimate the Q values for an optimal policy. For conve-
∗
nience, define these as Q∗ (s, a) ≡ Qπ (s, a), ∀s, a. It is straightforward to
∗
show that V ∗ (s) = max Q (s, a) and that if a∗ is an action at which the
a
maximum is attained, then an optimal policy can be formed as π∗ (s) ≡ a∗ .
Herein lies the utility of the Q values - if an agent can learn them, it can
easily decide what it is optimal to do. Although there may be more than one
optimal policy or a∗ , the Q∗ values are unique.
In Q-learning, an agent learns through its experience which consists of a
sequence of distinct stages or episodes (also called ‘trials’). In the nth episode,
n ≥ 1, the agent
• observes its current state st ,
• selects and performs an action at ,
• observes the subsequent state st+1 ,
• receives an immediate payoff rt+1 and
• adjusts its Qn−1 (s, a) values at state st+1 , using a learning factor αn and
discount factor γ, according to
(1 − αn )Qn−1 (s, a)+ if s = st and
αn [rt+1 + γmax{Qn−1 (st+1 , at+1 )}] a = at ,
Qn (s, a) = at+1 (5.7)
Qn−1 (s, a) otherwise.
Note that strictly speaking, the notations st and at should be replaced by st,n
and at,n , respectively, to more appropriately denote the state and action at
time instant t of episode n, and rt+1 should be replaced by rt+1,n to denote
the reward at a next time instant t + 1 of episode n, when the agent has just
entered a new state st+1,n due to taking action at,n at state st,n .
max{Qn−1 (st+1 , at+1 )} is the best the agent thinks it can do from state
at+1
st+1 . Of course, in the early stages of learning, the Q values may not accu-
rately reflect the policy they implicitly define. The initial Q values, Q0 (s, a),
for all states s ∈ S and actions a ∈ A are either set to zero or assumed to be
known.
164 5. How to Improve Intelligence?
o
θ6 =30
o o
θ 0 =30 θ 5 =45
o
θ1 =30 r a4 >40cm
r a3<40cm
ra2 <30cm
o ra1 <20cm o
θ2 =45 θ4 =90
ra0<10cm
o
θ3 =90
3. The distance between the defending robot and the ball is classified into
3 states (see Fig. 5.13) : rd0 , rd1 , rd2 .
4. The position of the ball in the playground is classified into 9 states (see
Fig. 5.14) : b0 , b1 , b2 , b3 , b4 , b5 , b6 , b7 , b8 .
Assuming that the two team robots involved are technically identical, then
5 × 7 × 3 × 9 = 945 (composed) states (sm , m = 1, 2, · · · , 945), would need
to be considered. Each state is denoted by (raia , θja , rdid , bib ), where a and
d indicate the roles of attacking and defending, respectively, 0 ≤ ia ≤ 4,
0 ≤ ja ≤ 6, 0 ≤ id ≤ 2 and 0 ≤ ib ≤ 8. In each state, the following two
decision actions are considered.
5.4 Q-Learning: A Model-Free Reinforcement Learning Method 165
r d2>40cm
rd1 <40cm
r d0<20cm
b0 b3 b6
vB
b1 b4 b7
b2 b5 b8
• Action a0 : change of roles between the attacking robot and the defending
robot.
• Action a1 : no change of roles between the attacking robot and the defending
robot.
Fig. 5.15 shows one of the states. If the supervisor chooses the action of
not changing roles, the attacking robot would keep moving behind the ball.
The time taken τ by one of the two robots that moves towards and kicks
the ball first is determined by the product of the image frame sampling time
and the number of frame samples taken that captures the robot movements
from its current position to the kick position. This is deemed as the complete
effect of the decision action taken that brings the transition from one state to
another. The reward for taking an action at at state st , recorded at a next
state st+1 , is then computed as an inverse proportion to time τ , given by
β1
rt+1 (st , at ) = , (5.8)
τ + β2
where β1 and β2 are constants.
166 5. How to Improve Intelligence?
Robot 2
Team goal Robot 1
Ball
Fig. 5.15. A Q-learning state (ra3 , θ1 , rd2 , b5 ) of a role-level strategy for robot
soccer
1
Learning factor: αn =
n
Discount factor: γ = 0.3
Constants : β1 =2
β2 =1
Image sampling period: TS = 33ms
Initial Q-values for all s ∈ S: Q0 (s, a0 ) = 0.5 (for role change)
Q0 (s, a1 ) = 0.6 (for no role change)
2
r1 (s617 , a0 ) =
0.891 + 1
= 1.058.
Thus, applying Eq. (5.7), Q1 (s617 , a0 ) = 1.058 + (0.3 × 0.6) = 1.238. This
concluded the first episode of learning when action a0 at state s617 was taken.
Repeating for 29 more episodes, Q30 (s617 , a0 ) = 1.616.
Consider taking the decision action a0 (role change) at state s617 . Then,
in the 1st episode, robot 2 kicked the ball, it took 23 sampling instances
(0.759s) to reach the ball and kick it. The reward for taking decision a1 was
2
r1 (s617 , a1 ) =
0.891 + 1
= 1.137.
Applying Eq. (5.7), Q1 (s617 , a1 ) = 1.317. This concluded the first episode of
learning when action a1 at state s617 was taken.
Repeating for 29 more episodes, Q30 (s617 , a1 ) = 1.024.
If we base on these (not necessarily optimal) Q-learning results, then since
Q30 (s617 , a0 ) > Q30 (s617 , a1 ), π(s617 ) = a0 , that is, the supervisor would
exchange the robots’ roles in state s617 .
! n
"
y = f (w0 u0 + w1 u1 + w2 u2 + · · · + un wn ) = f wi ui , (5.9)
k=0
168 5. How to Improve Intelligence?
u1
w1
w2
x
u2 f y
.
. w0
. wn
un
u0
where the numbers wi are called weights, w0 = 1 and u0 is the threshold. The
function f is a so-called sigmoid function, an example of which is illustrated
in Fig. 5.17.
-1
-4 -2 0 2 4
for
! n
"
x= wi ui ,
i=0
(λ1 − 1) (λ1 + 1)
f (x) ∈ , . (5.11)
λ2 λ2
This model of a neuron is thus simply a non-linear function. Some special
classes of functions can be approximated by Eq. (5.9).
u1
u2 y1
u3
u4 y2
u5
5.5.3 Learning
Notice that there are many parameters (weights) in a neural network. As-
suming that there are n neurons in a layer, if all neurons are connected, n2
parameters are then required to describe the connections between two layers.
Another interesting property of a neural network is that there are so-called
learning procedures. This is an algorithm that makes it possible to find para-
meters (weights) so that the function matches given input-output values. The
parameters are typically obtained recursively by giving both an input value
to the function and the desired output value. The weights are then adjusted
so that the data is matched. A new input-output pair is then given and the
parameters are adjusted again. The procedure is repeated until a good fit has
been obtained for a reasonable data set. This procedure is called training a
network. A popular method for training a feedforward network is called back
propagation [46, Ch. 6]. For this reason, the feedforward net is sometimes
called a back-propagation network.
1 if action i is feasible,
fij (k) =
0 otherwise.
From the above, Ajit = {aji | i ∈ Ijit }, Ajit ⊂ Aj , and Ijit ⊂ Ija . If the
interrupt for Internal Motive module is necessary, IT j (k) = 1. Otherwise,
IT j (k) = 0. The Intervention module interrupts the Internal Motive
module only when φjl j
i (k) is more than θi , action i is feasible and allowed
j
by the Supervisor module. IT (k) can be obtained through the following
calculations involving several parameters:
where
x if x > 0,
u(x) =
0 if x ≤ 0.
1 if M ax η j (k) > 0,
IT j (k) =
0 if M ax η j (k) = 0.
Fig. 5.20 shows how to select aji f s with this order of priority.
daji f s (k) corresponding to aji f s is compared with t daji f s and A F S j (k)
is finally determined as follows:
174 5. How to Improve Intelligence?
j aji f s if daji f s (k) > t daji f s ,
A F S (k) = j
A F S (k − 1) otherwise.
will be said of this later. The inputs ui to the neural network are situation
variables that characterize a game situation at each time instant k. The 10
input variables used are as follows:
• the ball’s velocity; the opponent robot’s velocity;
• the four variables characterizing ball possession (depicted in Fig. 5.22),
defined by
– θBR : angle between the team (or home) robot’s heading direction and
its direction towards the ball,
– θBO : angle between the opponent robot’s heading direction and its di-
rection towards the ball,
– DBR : distance between the team robot and the ball,
– DBO : distance between the opponent robot and the ball;
• the two variables representing the risk level of conceding a goal (depicted
in Fig. 5.23), defined by
– DBRG : distance between the ball and the team (or home) goal,
– DIRG : distance between the centre of the goal and the intersection point,
of the team goal line and the line passing through the opponent robot
and the ball;
• and the two variables representing the team robot’s winning score against
the opponent robot (depicted in Fig. 5.23), defined by
– DBOG : distance between the ball and the opponent goal,
– DIOG : distance between the centre of the goal and the intersection point
of the opponent goal line and the line passing through the team robot
and the ball.
176 5. How to Improve Intelligence?
Ball
DBOG
DBRG
Team goal
DIRG
DIOG
Fig. 5.23. Four situation variables representing the team (or home) robot’s winning
score against the opponent robot and the risk level of conceding a goal
1
f (x) = ∈ (0, 1).
1 + e−ax
2
Note that this sigmoid function can be obtained by setting λ1 = 1, λ2 = 2, and
a = 2σ in Eq. (5.10).
5.5 Neural Networks 177
In other words, the output of each simple neuron in the net represents the
activation level of one of the two actions in Ajit , where SweepBall is an action
done to kick away the ball and BlockBall is done to block the ball and avoid
conceding a goal.
Thus, the feedforward neural network to be set up for the Intervention
module has 10 inputs ui and 2 outputs yj . In [47], 2 hidden layers were
used, with the first and second hidden layers consisting of 12 nodes and
6 nodes, respectively. The complete net built was a 10 input, 2 output, 2
hidden layer, fully-connected, feedforward neural network. The error back
propagation algorithm [46], one of the supervised learning methods, was used
to train the network depicted in Fig. 5.24.
// Begin EP
// End EP
where
– xij , x′ij , ηij and ηij ′
denote the j-th parameter of the vectors xi , xi ′ ,
′
ηi and ηi , respectively;
– N (0, 1) denotes a normally distributed one-dimensional random
number with mean 0 and standard deviation # √ 1. $
−1 √
−1
′
The factors τ and τ are commonly set to 2 n and 2n .
′ ′ ′
• evaluate P (t) for each individual (xi , ηi ), ∀i ∈ {1, · · · , µ};
• P (t + 1) := survive P (t), P ′ (t);
In this step, the following are carried out:
a) Pairwise comparisons over the union of parents (xi , ηi ) and offsprings
(xi ′ , ηi′ ), ∀i ∈ {1, · · · , µ}, done as follows: for each individual,
– q opponents are chosen uniformly at random from all the parents
and offsprings;
– if the individual’s fitness is no smaller than opponent’s, it re-
ceives a ‘win’.
b) Selection of µ individuals, out of (xi , ηi ) and (xi ′ , ηi′ ), ∀i ∈ {1, · · · , µ},
that have the most wins, to be parents of the next generation.
• t := t + 1; } // end while
6. End EP
182 5. How to Improve Intelligence?
There are two important ways in which EP differs from GAs, though they
are merging together in a unified algorithm.
Representation: There is no constraint on the representation of individuals
in a population. The typical GA approach involves encoding the problem
solutions as a string of representative tokens, the genome. In EP, the
representation follows from the problem. A neural network can be repre-
sented in the same manner as it is implemented, for example, because the
mutation operation does not demand a linear encoding. (In this case, for
a fixed topology, real-valued weights could be coded directly as their real
values and mutation operates by perturbing a weight vector with a zero
mean multivariate Gaussian perturbation. For variable topologies, the
architecture is also perturbed, often using Poisson distributed additions
and deletions.)
Genetic Operators: While crossover and mutation operators are needed in
GA, the mutation operation in EP simply changes aspects of the indi-
vidual according to a statistical distribution which retains the behavioral
linkage between parents and their offsprings. Further, the severity of mu-
tations is often reduced as the global optimum is approached. There is a
certain tautology here: if the global optimum is not already known, how
can the spread of the mutation operation be damped as the solutions
approach it? Several techniques have been proposed and implemented
which address this difficulty. The most widely studied one is the Meta-
Evolutionary technique in which the variance of the mutation distribution
is subject to mutation by a fixed variance mutation operator and evolves
along with the individual. EP uses stochastic competition while GAs use
the Roulette wheel method for selection; it is noted that the stochastic
characteristics of the selection methods are similar.
5.6.3 EP and ES
This example illustrates how EP can be used to train univector fields for
robot navigation [38].
The idea of generating univector fields for robot navigation has already
been introduced in Section 4.6.2 of the previous chapter. To exploit the uni-
vector field F to achieve higher performance in robot control, the field has to
be optimized. For this purpose, a grid net and a function approximator are
developed. To start with, in general, a grid of size b × a is located within the
workspace3 as shown in Fig. 5.26(a). The shape and density of the grid net
can be varied in accordance with the application and the desired accuracy. A
node represents the point of intersection of the grid lines. Denser grid implies
a larger number of nodes. pi,j is the (coordinate) position of node (i, j) and
Fi,j represents the field vector at pi,j .
The set of angles of univectors Fi,j forms an b × a matrix, which is defined
as univector field matrix Φ as follows:
where ψi,j is the angle of vector Fi,j . To determine the field vector at an
arbitrary position p, interpolating operation is adopted. At first, the operator
finds the four neighbouring nodes at positions pi,j , pi+1,j , pi,j+1 , and pi+1,j+1
3
In robot soccer, the playground is the workspace.
184 5. How to Improve Intelligence?
Fi,j
(a) Grid
F i,j Fi,j+1
F(P)
Fi+1,j+1
Fi+1,j
(b) Interpolation
with
(db dc dd ) Fi,j + (da dc dd ) Fi,j+1 + (da db dd ) Fi+1,j + (da db dc ) Fi+1,j+1
F = ,
db dc dd + da dc dd + da db dd + da db dc
||F || = Magnitude of F ,
where
5.6 Evolutionary Programming 185
Fig. 5.27 (a) shows the field vectors represented by the matrix Φ and Fig.
5.27 (b) is the univector field calculated using Eq. (5.13). In the next section,
(a) (b)
where ts is the elapsed time and θd is the desired heading angle at point
g. This evaluation function is used for optimizing the univector field matrix
{Fi,j |1 ≤ i ≤ b, 1 ≤ j ≤ a}.
The first term in the evaluation function is to ensure quick reachability of
the desired point g and the second term forces the robot to converge to the
desired heading angle θd at point g. The third term ft (p) makes the robot
move to the desired point g:
0 if arrived at point g within allowable error bound,
ft (p) = (5.16)
Tp + mint∈[0,ts ] ( |p(t) − g| ) otherwise,
where p(t) is the position of the robot centre at time t, point g is the desired
position, and Tp is a penalty value that is added when the robot does not
arrive at point g. If the robot does not reach the desired position as indicated
in (5.16), the distance from the robot centre to the desired point, the cor-
responding value mint∈[0,ts ] ( |p(t) − g| ), and Tp are used to obtain ft (p) as
a penalty. The fourth term fo (p) prevents the robot from colliding with an
obstacle and assumes the following values:
0 if no obstacle collision,
fo (p) = (5.17)
Bp + maxt∈Ω ( |p(t) − pb | ) otherwise,
where Bp is a penalty parameter, Ω ⊂ [0, ts ] refers to the time interval during
which the robot is within an obstacle boundary, and pb is the closest point on
the obstacle boundary from the robot centre. When the robot collides with an
obstacle, the function fo (p) is calculated by projecting the robot trajectory
nearest to the obstacle center. The shortest distance of such a point from
the periphery of the obstacle is used for getting the value of the function
fo (p). The last term fa (ω̇) prevents the robot angular acceleration, ω̇, from
exceeding its limit αmax :
0 when ω̇ is within the limit αmax ,
fa (ω̇) = (5.18)
Ap + maxt∈[0,ts ] ( |ω̇(t) − αmax | ) otherwise.
In the computer simulation [38], the scaling factor kt and kd are taken as
1 and 5, respectively. The penalty values Tp , Bp , and Ap are taken as 500 cm,
100 cm, and 50 rad/s2 , respectively. The value of Tp is set to be greater than
the sum of the other two terms Bp and Ap in evaluation function. The terms
ft (p), fo (p), and fa (w) are made to satisfy the constraints, which ensure
that the robot reaches the desired position without collision and exhibits
movements without ripples. The remaining terms kt ts and kd | θc (ts ) − θd |
of (5.15) are used as fine tuning for short navigation time and desired heading
angle at point g. Once the univectors are optimized properly, the values of
ft (p), fo (p), and fa (w) are all zeros.
The penalty parameters Tp , Bp , and Ap determine the properties of opti-
mization progress. At the first stage of optimization, the individuals that fail
5.6 Evolutionary Programming 187
to drive the robot to the desired position are weeded out because Tp is the
strongest penalty. But if all the individuals violate this constraint, the term
on the right-hand side of Tp in (5.16) becomes meaningful. By this term the
individuals are inclined to approach the desired position. The terms Bp and
Ap in Eqs. (5.17) and (5.18) are used in a similar manner. The penalty values
of Tp , Bp , and Ap can be varied to suit the designer’s intentions.
1. Initialization
a) t ←− 0
b) Initialize population P0
2. While (not termination condition) do
a) t ←− t + 1
b) q ←− 0
c) While (not q = np ) do
i. q ←− q + 1
ii. Mutate q-th univector field matrix
iii. S(q) ←− 0
iv. k ←− 0
v. While (not k = ns ) do
A. k ←− k + 1
B. Simulate the robot navigation
C. Calculate evaluation function f
D. S(q) ←− S(q) + f
d) Select the best np candidates using S() for population Pt+1 .
3. End
1. the target point g and guidance point r of the subfield for a target posture
coincides respectively with the actual target point and guidance point in
the playground;
2. the obstacle point (and there are 5 of them for Small League MiroSoT) of
each duplicate subfield for obstacle avoidance coincides with the actual
centre of the obstacle in the playground.
All the 6 subfield univectors whose positions coincide at the actual robot’s
position p are then mathematically ‘combined’ to yield the univector F (p),
and hence the desired heading angle F (p), as discussed in Section 4.6.2 and
depicted in Fig. 4.21.
Fuzzy control is a control paradigm that has received a lot of attention re-
cently. In this section we will give a brief description of the key ideas. We
will start with fuzzy logic, which has inspired the development.
5.7 Fuzzy Logic and Control 189
Ordinary Boolean logic deals with quantities that are either true or false.
Fuzzy logic is an attempt to develop a method for logic reasoning that is less
sharp. This is achieved by introducing linguistic variables and associating
them with membership functions, which take values between 0 and 1. In
fuzzy control, the logical connectives ‘and’,‘or’ and ‘not’ are operators on
linguistic variables. These operations can be expressed in terms of operations
on the membership functions of values of the linguistic variables. Consider
two linguistic values, A and B, of a linguistic variable x, with the respective
membership functions, µA (x) and µB (x). The logical operations are defined
by the following operations on the membership functions.
1 if x = x0 ,
µA (x) = (5.22)
x0 .
0 if x =
Assume for example that we want to reason about temperature. For this
purpose we introduce the linguistic values cold, moderate, and hot, and we
associate them with the membership functions shown in Fig. 5.28. The mem-
bership function for the linguistic values cold and moderate and cold or mod-
erate are also shown in the figure.
0.5
0
-10 0 10 20 30 40
1
cold and moderate
0.5
0
-10 0 10 20 30 40
1 cold or moderate
0.5
0
-10 0 10 20 30 40
Fig. 5.28. Illustration of membership functions for fuzzy values. The upper diagram
shows the membership functions of cold, moderate, and hot. The middle diagram
shows the membership functions for cold and moderate; the lower diagram shows
the membership functions for cold or moderate.
ce
Inference u
Fuzzyfier Defuzzifier
e
engine
Fuzzy rule
base
give the rules for a PD controller where the error e and its derivative ce are
each characterized by three linguistic values (N, Z, P ) and the control input
u is characterized by five linguistic values (N B, N M, ZE, P M, P B).
Rule 1: If e is N and ce is P , then u is ZE.
Rule 2: If e is N and ce is Z, then u is N M .
Rule 3: If e is N and ce is N , then u is N B.
Rule 4: If e is Z and ce is P , then u is P M .
Rule 5: If e is Z and ce is Z, then u is ZE.
Rule 6: If e is Z and ce is N , then u is N M .
Rule 7: If e is P and ce is P , then u is P B.
Rule 8: If e is P and ce is Z, then u is P M .
Rule 9: If e is P and ce is N , then u is ZE.
These rules can also be expressed in table form, see Table 5.2.
u ce
e P Z N
N ZE NM NB
Z PM ZE NM
P PB PM ZE
If there are several rules, as in the description of the PD controller, each rule
is evaluated individually. The results obtained for each rule are aggregated
using the ‘or’ operator. This corresponds to taking the maximum operation of
the membership functions obtained for each individual rule. Similarly, instead
of the maximum operator max(.), called a T-conorm operator, any other T-
conorm operator for implementing ‘or’, such as algebraic sum, bounded sum
and drastic sum, can be used.
Fig. 5.30 is a graphical illustration for the case of the first two rules of the
PD controller. The figure shows how the so-called qualified (induced) con-
sequent membership function corresponding to each rule is constructed, and
how the overall output membership function representing the control input
is obtained by taking the maximum of the membership functions obtained
from all rules.
The inference procedure described is called ‘min-max.’ This refers to the
operations on the membership functions. Other inference procedures are also
used in fuzzy control. The ‘and’ operation is sometimes represented by taking
the product of two membership functions and the ‘or’ operator by taking an
algebraic sum. Combinations of the schemes are also used. In this way, it is
possible to obtain ‘min-max’ and ‘min-sum’ inference.
5.7 Fuzzy Logic and Control 193
w1
w2
e ce
Fig. 5.30. Illustration of fuzzy inference with two rules using the min-max rule.
5.7.4 Defuzzification
Fuzzy inference results in a control input expressed as a linguistic set and de-
fined by its membership function. To apply a control input to the real system,
we must have a real value. Thus, the linguistic set defining the control input
must be converted to a real number through the operation of ‘defuzzifica-
tion.’ This can be done in several different ways. Consider an overall output
linguistic set C with the membership function µC(u). Defuzzification by the
‘the centroid of area’ method gives the value
%
uµ (u) du
u0 = % C . (5.25)
µC (u) du
This example implements a navigation controller (at the behaviour level) for
a Shoot action using fuzzy logic control [40].
Allowed
Not allowed
Fig. 5.31. Shooting from the left side when the line connecting the ball to the
opponent goal is on the right
d
qB
ϕ
z
t
Fig. 5.32. Variables for relative posture characterization
Fig. 5.33. Overall fuzzy control structure for the Shoot action
The input, output and rules for the destination block are defined as fol-
lows:
1. Input space (ρ, ϕ), relative position of the robot to the ball:
ρ ∈ [0cm, 60cm],
ϕ ∈ [0, 180 deg.].
2. Output space (θd ), desired heading angle:
θd ∈ [-180 deg., 180 deg.] (indicated by arrows → in Fig. 5.36).
3. Rules
49 rules are obtained for θd at sampled positions as shown in Fig. 5.36,
with ϕ and ρ each characterized by seven linguistic values (N B, N M, N S,
ZE, P S, P M, P B). Since the input space is uniformly divided, the rules
are ‘sampled’ at the centre of each input region. The resultant rules for
the destination block are represented in Table 3.
5.7 Fuzzy Logic and Control 197
Obstacle Block. This block modifies θd using offset angle θf if there is any
obstacle nearby. Four variables, namely, velocity Vr , direction Dr , distance
dr , and position Pr (positive if the obstacle is in front, negative otherwise.),
all relative to the robot, are utilized to obtain θf in the presence of obstacles.
Those relative quantities are necessary to obtain the escape radius Rs so as
to avoid any obstacle that is either stationary or moving, as shown in Fig.
5.37. The input variable membership functions are depicted in Fig. 5.38 and
Fig. 5.39.
198 5. How to Improve Intelligence?
1.2
NB NM NS ZE PS PM PB
1
0.8
Membership value
0.6
0.4
0.2
0
−10 0 10 20 30 40 50 60 70
rho
(a) For ρ
1.2
NB NM NS ZE PS PM PB
1
0.8
Membership value
0.6
0.4
0.2
0
0 20 40 60 80 100 120 140 160 180
phi
(b) For ϕ
The input, output and rules for the obstacle block are defined as follows:
1. Input space(Vr , Dr , dr , Pr ), the velocity, direction, distance and position
of an obstacle relative to the robot:
Vr ∈ [-0.5, 1.5],
Dr ∈ [0 deg., 180 deg.],
dr ∈ [0 cm, 90 cm],
Pr ∈ [-0.5, 1.5].
5.7 Fuzzy Logic and Control 199
60
50
40
30
20
10
0
-6 0 -4 0 -2 0 0 20 40 60
Fig. 5.36. θd sampled at each input region in the vicinity of the ball at (0, 0)
do
Resc
s
θd'
θs
dr Robot
θd
Obstacle
v
1.2
NB ZE PB
1
0.8
Membership value
0.6
0.4
0.2
0
−0.5 0 0.5 1 1.5
relative velocity
(a) For Vr
1.2
NB ZE PB
1
0.8
Membership value
0.6
0.4
0.2
0
0 20 40 60 80 100 120 140 160 180
relative direction
(b) For Dr
Table 5.4. Rules for the obstacle block, FLC1 (left) and FLC2 (right)
Rs Dr wo dr
Vr NB ZE PB Pr ZE PS PM PB
NB 20 20 20 NE 0.8 0.7 0.6 0.0
ZE 20 25 30 ZE 1.0 1.0 0.9 0.0
PB 20 35 40 PO 1.0 1.0 1.0 0.0
5.7 Fuzzy Logic and Control 201
1.2
ZE PS PM PB
0.8
0.6
0.4
0.2
0
−10 0 10 20 30 40 50 60 70
(a) For dr
1.2
NE ZE PO
1
0.8
0.6
0.4
0.2
0
−40 −30 −20 −10 0 10 20 30 40
(b) For Pr
Fuzzy Posture Controller. In the overall structure of Fig. 5.33, the fuzzy
posture controller block receives θd from the fuzzy planner block and the ρ-
part of robot posture information from the vision processing system. Then the
posture controller block generates the appropriate left-wheel and right-wheel
velocities to make θ follow θd at non-zero linear speed before ρ diminishes. So
the posture controller is only concerned with directing the robot’s heading
angle θ to follow θd at positive linear velocity. For this conventional problem
of mobile robotics, the following heuristics are incorporated:
202 5. How to Improve Intelligence?
Vr Rs θs
FLC1 Tan-1( )
Dr
θf =woθs
dr wo
FLC2
Pr
• If ρ big → VL , VR big.
• If |θe | = |θd′ − θ| big → |VL − VR | big.
The input variable membership functions are depicted in Fig. 5.41.
The input, output, and rules for the posture controller block are defined
as follows:
1. Input space (ρ, θe ), posture error of the robot to the ball and the path:
ρ ∈ [0cm, 60cm],
θe ∈ [-120 deg., 120 deg.].
You would have noticed that all the rule tables contain real-value entries
instead of the linguistic values for the respective control variables, namely,
5.7 Fuzzy Logic and Control 203
1.2
NB NM NS ZE PS PM PB
1
0.8
Membership value
0.6
0.4
0.2
0
−10 0 10 20 30 40 50 60 70
Distance
(a) For ρ
1.2
NB NM NS ZE PS PM PB
1
0.8
Membership value
0.6
0.4
0.2
0
−150 −100 −50 0 50 100 150
Theta error
(b) For θe
6.1 Introduction
The software complexity of a real-time robot soccer system calls for a struc-
tured development methodology and framework. In this chapter, we describe
a host software model for the development of a robot soccer system. This
model emphasizes modularity of design. An overview of the programming
framework for robot soccer is then presented, in which a number of the robot
soccer concepts described in earlier chapters are illustrated through example
‘C’ programs. These programs implement the key functions of a command-
based robot soccer system for MiroSoT. An overview of the functions, cate-
gorized into basic and applied skills, is given below.
1. Basic skills
For a specified soccer robot,
a) Velocity() sets its left-wheel and right-wheel velocity data;
b) Angle() sets its desired left-wheel and right-wheel velocity data to-
wards achieving a specified turning angle;
c) Position() sets its left-wheel and right-wheel velocity data to move
towards a specified position; and
d) Shoot(), similar to Position(), but additionally sets the desired
angle at which the robot should arrive at the specified position (where
the ball is). The purpose is to direct the robot to hit the ball in the
intended direction.
2. Applied skills
These are more advanced functions that consider various strategic game
situations. For a specified robot,
a) Kick() implements a strategic process of ball kicking by pushing, as
described by a state machine;
b) Goalie() implements a strategic process of goalkeeping, as described
by IF-THEN rules; and
c) AvoidBound() implements an auxiliary strategy to prevent the robot
from getting ‘stuck’ at the side-wall. The strategy is also described
by IF-THEN rules.
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 205-256, 2004
Springer-Verlag Berlin Heidelberg 2004
206 6. Robot Soccer Programming
Several game and robot navigation strategies are also suggested. For the
overall game, a simple zone-defence strategy is explained and its ‘C’ program
code is given. The univector field and limit-cycle navigation methods, studied
in Section 4.6, are good alternative strategies for implementing the functions,
Kick() and Position(). The ‘C’ codes of the essential component functions
for implementing each method are also given and explained.
To build a solid MiroSoT team, these example and generic codes could
be modified or expanded to incorporate other ideas and techniques, many of
which have been introduced in earlier chapters.
User interface
Message
Vision Strategy
transmission
Image Game
Calibration data recorder data recorder
Real time
Data
monitor display
User commands
Critical data
The key system modules are the vision, strategy and message transmis-
sion modules; they implement the functionalities of SENSE, DECIDE and
ACT:Control, and ‘interface’ between ACT:Control and ACT:Actuation,
respectively.
As this is a real-time system, data is continually being generated. A mod-
ule implementation should therefore adhere to the following rules necessary
to achieve fault-tolerance in real-time:
1. It should be able to act only on the most current set of data and not
waste time processing old data.
This implies that the system as a whole should not have any queuing.
2. It should be able to function even if the module ‘listening’ to (i.e., re-
ceiving) its output becomes inoperative.
In other words, sending its output data to an inoperative or nonexistent
module will not cause it to stall, a feature called non-blocking write.
3. It does not re-transmit and must communicate with other modules using
only atomic messages (i.e., messages that cannot be fragmented).
4. It needs to be able to tolerate the failure of a module that it receives its
data set from, and has code to handle such events.
For example, the message transmission module monitoring the command
data from the strategy module would need to tell the team robots to halt
if the strategy module fails to send the command data within an alloted
time, to prevent these robots from crashing onto the side walls.
208 6. Robot Soccer Programming
The vision, strategy and message transmission modules of the host software
model of Fig. 6.1 are implemented as Find_Object(), My_Strategy(), and
Send_Command(), respectively, in the program structure depicted in Fig. 6.2.
2Q5HDG\ )LQGB2EMHFW
0\B6WUDWHJ\
6HQGB&RPPDQG
Fig. 6.2. Overall program structure
void My_Strategy()
{
switch(m_nGameMode) {
case GAME_STARTKICK:
Kick_Off(); // Strategy for kick-off
break;
case GAME_PENALTY:
Penalty_Kick(); // Strategy for penalty-kick
break;
. . . .
. . . .
. . . .
6.3 Programming Framework: An Overview 209
.LFNB2II
0\B6WUDWHJ\ 3HQDOW\B.LFN
)UHHB.LFN
)UHHB%DOO
*RDOB.LFN
1RUPDOB*DPH
case GAME_NORMAL:
Normal_Game(); // Strategy for game
break;
}
}
The game mode is selected via the system GUI, and the program invokes
accordingly via switch-variable m_nGameMode.
The key robot soccer functions/procedures needed for each mode of the
game are listed in Table 6.1. They are grouped into basic and applied skill
functions. The next two sections describe how they are implemented as pro-
gram codes for the experimental robot soccer system; in this system, only the
proportional (P) control law is used to compute the desired wheel velocities,
and then close-loop (actuation) control is implemented to achieve the com-
manded or desired wheel velocities in each team robot. Besides, the P gain
values used in all the program codes are empirical values. The microprocessor-
based hardware and firmware for a soccer robot as introduced in Chapter 2
are assumed.
The following are definitions of some programming constants, variables
and arrays needed to understand the program code following the description
of each robot soccer function:
1. Constants
HOME1 : team robot ID 0.
HOME2 : team robot ID 1.
HGOALIE : team robot ID 2.
M_PI : π (3.14).
210 6. Robot Soccer Programming
Table 6.1. Program functions for robot soccer system (MiroSoT category)
Mode Functions
Normal_Game()
2. Variables
whichrobot : robot ID.
d_e error in distance between robot’s current
: position (x, y) and desired position (xd , yd ).
theta_d : desired angle θd (in degrees).
theta_e : error in angle θ e .
vL : desired (PWM-based) velocity data HL
for left-wheel of robot.
vR : desired (PWM-based) velocity data HR
: for right-wheel of robot.
Recall that data H ∈ [0, 255] is a PWM integer data (see Section 2.3.5,
page 63).
3. Arrays and functions
• PositionOfBall[0]:
the x-coordinate value of the ball’s position.
• PositionOfBall[1]:
the y-coordinate value of the ball’s position.
• AngleOfHomeRobot[whichrobot]:
the heading angle θ (in degrees) of robot with ID whichrobot.
6.4 Basic Skill Functions 211
• PositionOfHomeRobot[whichrobot][0]:
the x-coordinate value of the position of the robot with ID whichrobot.
• PositionOfHomeRobot[whichrobot][1]:
the y-coordinate value of the position of the robot with ID whichrobot.
• atan2(arg1,arg2):
arg1
tan−1 (in radians).
arg2
6.4.1 Velocity()
To actuate the motors, PWM (Pulse Width Modulation) is used (see Section
A.1, from page 273 onwards). As explained in Section 2.3.5 (from page 63),
the velocity data H is a PWM data sent by the host computer and converted
to the actual PWM data W by the receiving robot for motor actuation. The
required PWM data H for a desired wheel (rotational) velocity ω̄G can be
1
computed using Eq. (2.39) for W and H = W .
4
For a specified robot, the Velocity() function sets the H data vL and
vR, given the respective ‘normalized’ velocity data vl and vr defined by
1 ω̄GL 1 ω̄GR
vl = A , vr = A , (6.1)
2 ωGL |max 2 ωGR |max
where ωGL and ωGR denote the rotational velocities - averages (overhead bar
¯) and maximum constants (|max ) - of the left and right wheels, respectively.
Equivalently, we get
1 VL 1 VR
vl = A , vr = A , (6.2)
2 VL |max 2 VR |max
where VL |max and VR |max are constants denoting the linear maximum ve-
locities of the left and right wheels, respectively. Motors of the same driving
capacity are used, thus VL |max = VR |max = Vmax .
Hence, the ‘PWM-version’ of Eq. (4.9) is
vl = Kν · ν − Kω · ω,
(6.3)
vr = Kν · ν + Kω · ω,
where
1 L
Kν = · A and Kω = Kν · .
2Vmax 2
212 6. Robot Soccer Programming
// in forward direction
if( vl > 106 ) vl = 106; // For left wheel
if( vr > 106 ) vr = 106; // For right wheel
In the code, the global array command[] holds 10 one-byte elements. The ex-
perimental host program uses this array to send data by IR communication
to the team robots, on the message format shown in Fig. 2.35. The elements
command[i], for which i = 4,5,..,9, are the variables that store the respective
velocity data H for the team robots. The other elements, for which i = 0,..,3,
are the variables that store information in accordance to the IR communi-
cation protocol. The communication function used is the Send_Command()
function given below:
void Send_Command()
{
m_pComm.WriteCommBlock((LPSTR)command, 10);
// Sends data in command array of size 10 bytes
}
Note that the byte 0xAA is not included in the command[] array; the trans-
mitter automatically adds these bytes during the actual transmission. Before
Send_Command() can be used, a serial port must be selected; details of port
selection are, however, omitted.
Velocity() is the most fundamental function. The next two basic skills
are formulated in terms of this function.
6.4.2 Angle()
This function sets the desired velocity data vL and vR for a specified robot
towards achieving a desired turning angle theta_d.
Referring to Fig. 6.4, the x-distance and y-distance between the position
(x, y) of the robot and the desired point are |dx| and |dy|, respectively; the
desired angle theta_d is thus given by
dy
theta_d = tan−1 .
dx
214 6. Robot Soccer Programming
ڴ dx
Desired
point
dy θe
θe = θd −θh
θh
θd
ڳ
For a specified robot, Angle() first uses a control law to set the ‘normal-
ized’ velocity data vl and vr, given a desired angle theta_d, and then uses
Velocity() to generate the velocity data vL and vR. The specified robot
that receives the data will rotate its two wheels accordingly; for example, the
robot depicted in Fig. 6.4 would rotate its left-wheel forward and right-wheel
backward to face the desired direction.
Consider proportional (P) control for ω. Then
ω = KPa · θe , (6.4)
where KPa is the proportional gain. Because Angle() is concerned with turn-
ing motion only, ν = 0. Substituting Eq. (6.4) and ν = 0 into Eq. (6.3), we
get the ‘PWM-version’ of P control law to move the robot:
vl = −ka · θe ,
(6.5)
vr = ka · θe ,
where ka = Kω · KPa .
The robot’s heading angle theta is obtained in real-time through a routine
called AngleOfHomeRobot[]. Using this routine and Eq. (6.5), the program
code for Angle() follows:
As shown in the above code, three ranges of angle errors are handled, namely,
when it is greater than 50◦ , between 20◦ and 50◦ , and between 0◦ and 20◦ .
This is done to enable the suitable application of different proportional gains
ka ’s. It has been found that if the smaller proportional gain used for bigger
angle errors is used for smaller angle errors, the velocity data values sent to
the robot are smaller than the required values.
When called periodically with the same theta_d, the above program
Angle() directs the specified robot to turn to the desired angle.
6.4.3 Position()
This function sets the velocity data vL and vR for a specified robot to move
towards a desired position (x_d, y_d).
For a specified robot, Position() first sets the ‘normalized’ velocity
data vl and vr, given a desired position (x_d, y_d) to reach, and then uses
Velocity() to generate the velocity data vL and vR.
216 6. Robot Soccer Programming
ڴ dx
Desired
point
(xd, yd)
dy θe
θh
θd
(x, y) ڳ
Referring to Fig. 6.5, to reach the desired point (xd , yd ) from the robot’s
online position (x, y), the errors in distance de and angle θe to correct are
de = dx2 + dy 2 , θe = θd − θ.
ν =KPl · de ,
(6.6)
ω =KPa · θe ,
where KPl and KPa are the proportional gains for ν and ω, respectively.
Substituting Eq. (6.6) into Eq. (6.3), we get the ‘PWM-version’ of P control
law to move the robot:
vl = kl · de − ka · θe ,
(6.7)
vr = kl · de + ka · θe ,
dx = x_d - PositionOfHomeRobot[whichrobot][0];
6.4 Basic Skill Functions 217
dy = y_d - PositionOfHomeRobot[whichrobot][1];
When called periodically with the same (x_d, y_d), the above program
Position() directs the specified robot to move to the desired position.
218 6. Robot Soccer Programming
However, the code does not address two practical problems. First, as the
specified robot approaches the desired point, it slows down and eventually
stops at the desired point. This means that if this desired point is the position
of the ball, the robot stops when it reaches (or is close to) the ball. In other
words, it cannot kick the ball. Second, as depicted in Fig. 6.6, when the angle
error theta_e is at +90 ◦ or −90◦ , the robot experiences a swinging motion
as it continually switches its heading to either the conventional forward or
backward direction.
To elaborate, Position(), called when θe = +90◦, will direct the robot to
move forward (to the right) as shown in Fig. 6.6(a); but in so doing, the angle
error θe increases marginally above 90◦ by the second call to Position().
In this case, Position() will switch the robot’s heading to the conventional
backward direction, in the opposite direction as indicated by the thick ‘Mov-
ing direction’ arrow. Following which, the robot moves in the new heading
direction (to the left) as shown in Fig. 6.6(b), causing the angle error θe to
decrease marginally below −90◦ by the third call to Position(). The condi-
tion leads Position() to switch the robot’s heading back to the conventional
forward direction, as indicated by the thick ‘Moving direction’ arrow. This
cyclic pattern continues, resulting in oscillatory motion that the robot will
experience. Assuming a stationary ball, this motion can continue for several
cycles before the robot can reach the ball.
Vc
40
20
0
1cm 5cm
de
Fig. 6.7. Graph of Vc against de
to keep the robot’s velocity ν at a certain positive limit when it reaches the
desired position.
To overcome the second problem (the oscillation problem), the code for
Position() needs to be revised to switch robot’s heading direction to the
opposite direction if the angle error θe is greater than (90◦ + β + ) or less than
−(90 + β − ). Besides, under the condition of θe ∈ [−(90◦ + β − ), −(90◦ − β − )]
or θe ∈ [(90◦ − β + ), (90◦ + β + )], it needs to set the velocity data vl and vr
for robot turning only in order to exit this condition swiftly.
Incorporating the aforementioned considerations with vo = 70, ε1 = 3 and
ε2 = 0.3 for Eq.(6.9) and β + = β − = 5, the program code for Position() is
revised as follows:
220 6. Robot Soccer Programming
// manage an exception
if(d_e<5.0 && abs(theta_e)<40) ka = 0.1;
}
else if(theta_e < 85 && theta_e > -85) {
// manage an exception
if(d_e<5.0 && abs(theta_e)<40) ka = 0.1;
6.4 Basic Skill Functions 221
}
else { // if magnitude of angle error is within [90-5,90+5]
// calculate vr and vl of robot (turning only)
vr = (int)(+0.17*theta_e);
vl = (int)(-0.17*theta_e);
}
With the revised code for Position(), the robot can continue moving
once it has reached the desired position, rather than slow down to a halt as
in the previous code. Governed by Eq. (6.9), when the robot is far away from
the desired position which, say, is the position of the ball, the velocity ν will
be set higher, to drive the robot more quickly towards the ball, but will be
set gradually lower as it approaches the ball, until a specified limit of 0.2vo
(according to selected parameter values of vo = 70, ε1 = 3 and ε2 = 0.3 for
Eq. (6.9)), thus improving the accuracy of the robot hitting the ball, and at
a positive lowest-limit velocity.
Although the program code has been tested to work reasonably well, no
claim is made that it is the best. By adjusting the parameters in Eq. (6.9)
for the Position() function, the human designer can increase the robot’s
velocity ν when the robot is at a further distance from the ball, and decrease
it more sharply as it reaches the ball. Additionally, if the robot is at an ideal
position for shooting, the designer can set the lowest velocity-limit higher for
a stronger shot of the ball at goal. In general, the designer can always improve
the program and experiment with different value settings for the parameters
used.
6.4.4 Shoot()
When called periodically with the same (x_d, y_d), the Position() function
directs the specified robot to move to the desired position. But notice that the
function does not consider the desired heading angle at the desired position
(x_d, y_d). This consideration is important for the Shoot() function, which
is also concerned with the desired angle at which the robot should come into
contact with the ball at (x_d, y_d). The purpose is to direct the robot to
shoot the ball in the intended direction.
222 6. Robot Soccer Programming
Path 2
Desired direction 1 q2
q1
Ball
Path 1
Desired direction 2
Robot
Fig. 6.8. Different paths of the robot for shooting the ball in different desired
directions
θd = 2φ2 − φ1 , (6.10)
where
dy1 dy2
φ1 = tan−1 , φ2 = tan−1
dx1 dx2
and
dx1 = xt − xb , dy1 = yt − yb ,
dx2 = xb − x, dy2 = yb − y.
Ball
θe
dx1
yb θd
θ dy2 φ1
φ2
y
dy1
Robot
Opponent goal
dx2 Desired heading direction
yt
Target Point
x xb xt X
Fig. 6.9. Geometric relationships for calculating the robot’s desired heading angle
θd at its online position (x, y)
θe = θd − θ.
ω =KPa · θe , (6.11)
where KPa is the proportional gain. Substituting Eq. (6.11) into Eq. (6.3),
we get the ‘PWM-version’ of P control law to move the robot as follows:
vl = Vc − ka · θe ,
(6.12)
vr = Vc + ka · θe ,
// calculate phi1
if(dx1 == 0 && dy1 == 0) phi1 = 90;
else phi1 = (int)(180/M_PI*atan2((double)(dy1), (double)(dx1)));
// calculate dx2
dx2 = x_b - PositionOfHomeRobot[whichrobot][0];
// calculate dy2
dy2 = y_b - PositionOfHomeRobot[whichrobot][1];
// calculate phi2
if(dx2==0 && dy2==0) phi2 = 90;
else phi2 = (int)(180/M_PI*atan2((double)(dy2),(double)(dx2)));
When called periodically with the same position inputs, the above program
Shoot() should direct a specified robot to bump against the ball at the
desired heading angle.
However, the code does not address the technical problem of robot chat-
tering. Ideally, Shoot() should direct the specified robot at high velocity to
negotiate any bend smoothly, and then approach the ball along a straight
line. But due to ‘overshoot’ in negotiating the bend at high speed, there is
subsequently a continual angle-error correction in the robot’s heading as it
approaches the ball, resulting in a chattering motion trajectory as depicted
in Fig. 6.10. To elaborate, when the fast-moving robot ‘cuts’ the straight
line from below, Shoot() would set the robot’s desired heading angle θd to
negative. But by the next call to Shoot(), the fast-moving robot would have
‘cut’ the line from above, so Shoot() would set the robot’s desired heading
angle θd to positive. This cyclic pattern continues rapidly, resulting in the
robot chattering as it approaches the ball.
Robot
Ball
Other techniques, such as the two methods detailed in Section 4.6, are
suitable for implementing the Shoot() function. In particular, we highlight
226 6. Robot Soccer Programming
oB
oB d
d
0.5∆
0.5∆
Fig. 6.11. A modified univector field for solving the problem of chattering
6.5 Applied Skill Functions 227
6.5.1 Kick()
This function uses Position() and Angle() to implement the process of ball
kicking by a specified robot. This function is an alternative to the Shoot()
function.
In the following, the pseudocode for Kick() is first given. We then ex-
plain how this function can strategically direct the specified robot to kick
the ball towards the opponent goal, taking the borders (or side-walls) of the
playground into consideration.
Overall Pseudocode.
switch(flag){
case 0: // state S0
Go behind the ball (i.e., into ‘shootable area’)
by using Position();
If the robot is near the specified shooting
position, switch to state S1 by setting flag = 1;
break;
case 1: // state S1
Turn to face the ball by using Angle();
If the robot’s direction is towards the ball,
switch to state S2 by setting flag = 2;
break;
case 2: // state S2
Kick the ball by using Position();
If the robot is out of the shootable area,
switch to state S0 by setting flag = 0;
break;
}
228 6. Robot Soccer Programming
VZLWFK
KIIODJ
^^
FDVH
H
VWDWH6
'R
R$$
,I
I&&
WWKHQ
QIO
IOD
DJ
EUHDN
&
FDVH
H
VWDWH6
'R
R$$
,I
I&&
W
WKHQ
QIO
IOD
DJ
EUHDN 6
$
FDVH
H
VWDWH6
'R
R$$ & &
,I
I&&
W
WKHQ
QIO
IOD
DJ
EUHDN
` 6 6
$ & $
&
&
S0 : Far from the ball A0 : Move behind the ball C0 : When robot is near
the specified shooting
position behind the ball
We now present the codes with explanation for the various sections of the
Kick() program.
Set Kicking Direction. In this code section, ideally, the function should
direct the specified robot to kick the ball towards the opponent goal, but to
6.5 Applied Skill Functions 229
prevent the robot from colliding with any playground border or side-wall, or
miskicking towards its own team goal, the kicking direction has to be planned
according to the position of the ball. The plan considered for this example
program is shown in Fig. 6.13, where the direction of ball kick has to be
aimed towards the goal if the ball is in subarea A, but otherwise modified as
depicted in the figure.
Y '
B
B
'
'
D
D
C
C
laog tnenoppO
AA
laog maeT
C
C'
D
D'
B
B' X
It should be noted that the values of 150 and 130 in the code refer to the
dimensions (in cm) of a rectangular playground, and need to be replaced
accordingly for 5-a-side MiroSoT.
In State S0: Do A0: Check C0. In this code section, to direct the specified
robot to a shooting position (pos[0], pos[1]) in the shootable area (behind
the ball), as depicted in Fig. 6.14, A0-action is implemented by the position
function Position( whichrobot, pos[0], pos[1]) to move the specified robot
to the point.
From Fig. 6.14, pos[0] and pos[1] are calculated as follows:
case 0:
// get the position behind the ball
pos[0] = PositionOfBall[0] - D_h*cos(theta_d*M_PI/180.);
pos[1] = PositionOfBall[1] - D_h*sin(theta_d *M_PI/180.);
6.5 Applied Skill Functions 231
Dc
dy'
OWSXP
dx '
break;
In State S1: Do A1: Check C1. The specified robot is behind the ball,
and so in this code section, to direct the robot to turn towards the ball at the
desired angle theta_d (‘the kicking direction’), the function uses A1-action
Angle(whichrobot, theta_d) to set the desired velocity data (for the two
wheels) that should turn the specified robot to face the ball.
To prepare for an appropriate action in the next call to Kick() for the
same robot, the function checks if the robot’s heading direction is towards
the ball; and if so, switches to state S2. Referring to Fig. 6.16, the robot is
deemed to be heading towards the ball if C1-condition given by
is true, i.e., the magnitude of the angle error θe between the robot’s heading
angle and the desired angle θd at which the robot would directly face the ball
is less than some angle constant Ad set by the designer.
The code for this section follows:
case 1
// rotate to the goal
Angle(whichrobot, (int) theta_d);
Ad
θe
Ball
Robot
Fig. 6.16. In state S1: to switch to state S2 if the angle error is less than a specified
value Ad . Note that in this figure, the illustrated C1-condition ‘|θe | < Ad ’ is false.
break;
In State S2: Do A2: Check C2. In this code section, to direct the
specified robot to kick the ball towards goal through an auxiliary position
(pos[0], pos[1]), as depicted in Fig. 6.17, the function uses A2-action
Position(whichrobot,pos[0], pos[1]) to set the desired velocity data that
should move the specified robot to shoot the ball towards goal.
From Fig. 6.17, pos[0] and pos[1] are calculated as follows:
pos[0] =PositionOfBall[0] + dx ′′ ,
(6.16)
pos[1] =PositionOfBall[1] + dy ′′ ,
Fig. 6.17. State S2: the desired point (pos[0], pos[1]) the specified robot should
reach in order to move through the ball’s position at a certain positive velocity.
is true, i.e., the magnitude of distance between the robot at position (x, y)
(denoted as point c) and the ball at position (xb , yb ) (denoted as point b) is
longer than some constant Df set by the designer, or the ball is behind the
→ → →
robot, i.e., |θd − bc | < 90◦ , when bc and (θd − bc) are kept in the
range (−180◦, 180◦ ].
The code for this section follows:
case 2:
// get the position in front of the ball
pos[0] = PositionOfBall[0] + D_v*cos(theta_d*M_PI/180.);
pos[1] = PositionOfBall[1] + D_v *sin(theta_d*M_PI/180.);
θd
Df
Jxb , ybK
uB
Fig. 6.18. In state S2: to switch to state S0 if the robot is at less than a distance
of Df from the ball, or is behind the ball.
break;
6.5.2 Goalie()
This function uses Position() and Angle() to direct the goalkeeping robot
to block the ball.
Ball
Team goal
Goalie
Goalie
Near Middle
Fa
Far
r
Y
Ball
PositionOfBall[1]
dy
estimate_y
∆y
dx
∆x
G_OFFSET 65cm
15cm Far-distance
- area
estimate_x PositionOfBall[0] X
Ball
estimate_y
PositionOfBall[1]
G_OFFSET
Middle-distance area
15cm
PositionOfBall[0]
estimate_x X
Y
Near-distance area
PositionOfBall[0]
Ball
estimate_y
PositionOfBall[1]
G_OFFSET
estimate_x X
when the robot comes into contact with a side-wall, directing it to turn
immediately away towards a desired position (usually the ball’s position)
is not possible when the wall is obstructing ‘on the spot’ turning of the
robot, or we say the robot is ‘stuck at the wall’. This happens when
the (smaller) angle between the line along the robot’s heading direction
and the normal to the side-wall it comes into contact with is within a
certain (maximum) angle bound Ab set. In any such situation, the robot
would need to be turned away from the wall first at a suitable non-zero
translational velocity. Exception handling is done to prevent the robot
from getting stuck at the wall in various situations, by applying the P
control law of Eq. (6.12) with appropriate values for Vc and ka .
Five exception-situations depicted in Fig. 6.24 are considered for the
goalkeeping robot. In the figure and program code that follows later,
angle bound Ab is denoted by G_ANGLE_BOUND. To elaborate, consider
exception-situations 2 and 3 in Fig. 6.24. Set angle bound Ab = 60o .
In the latter situation, the goalkeeping robot is facing rightward at a
heading angle of between −60o and 60o, and should be directed to make
6.5 Applied Skill Functions 239
{
{
d
wB
*B/
*B /(1*
(1*7+
7+
nB
d
*B 2) )6 (7
*B 2) )6 (7
z z
a left turn towards the goalie area. In the former situation, the robot
is heading leftward at a heading angle of between 120o and 240o , and
should be directed to make a reverse left turn towards the goalie area.
In the next section, a similar problem for the other team robots will be
addressed, using an Avoidbound() function.
The following program code for Goalie() implements the suggested strategy:
m
5
m
G_BOUND
3
m
m
G_ANGLE_BOUND
2
Fig. 6.24. Numbered exception-situations and the desired turning directions for
the goalkeeping robot
dx = estimate_x - PositionOfHomeRobot[whichrobot][0];
dy = estimate_y - PositionOfHomeRobot[whichrobot][1];
while(AngleOfHomeRobot[whichrobot] >180)
AngleOfHomeRobot[whichrobot] -= 360;
while(AngleOfHomeRobot[whichrobot] <-180)
AngleOfHomeRobot[whichrobot] += 360;
6.5.3 AvoidBound()
with appropriate values for Vc and ka as set by the designer. In the figure and
program code that follows later, angle bound Ab is denoted by ANGLE_BOUND.
To elaborate, consider exception-situation B1 in Fig. 6.25. Set angle
bound Ab = 60o . In this situation, the robot is heading towards the bot-
tom wall at a heading angle of between −150o and −30o, and should be
directed to make a reverse turn, away from the wall.
T1 ANGLE_BOUND T2
R1
DISTANCE_BOUND ANGLE_BOUND
L1
ANGLE_BOUND
ANGLE_BOUND
ANGLE_BOUND
R2
ANGLE_BOUND
L2
L4 ANGLE_BOUND ANGLE_BOUND R4
ANGLE_BOUND
ANGLE_BOUND
DISTANCE_BOUND ANGLE_BOUND
L3 R3
DISTANCE_BOUND B1 ANGLE_BOUND
B2 DISTANCE_BOUND
#define ANGLE_BOUND 60
#define DISTANCE_BOUND 9
void AvoidBound(int whichrobot)
{
A-BLOCK1; // for top and bottom walls
A-BLOCK2; // for left wall
244 6. Robot Soccer Programming
dx=PositionOfBall[0]-PositionOfHomeRobot[whichrobot][0];
dy=PositionOfBall[1]-PositionOfHomeRobot[whichrobot][1];
theta = AngleOfHomeRobot[whichrobot];
theta_e = theta_d - AngleOfHomeRobot[whichrobot];
// right side-wall
if (m_theta<ANGLE_BOUND &&
PositionOfHomeRobot[whichrobot][0]>150.0-DISTANCE_BOUND &&
PositionOfHomeRobot[whichrobot][1] > 85.0)
{ // right wall: case R1
vr = (int)(-14+19.0/90.0*theta_e);
vl = (int)(-14-19.0/90.0*theta_e);
Velocity(whichrobot, vl, vr);
}
else if (m_theta>180-ANGLE_BOUND &&
PositionOfHomeRobot[whichrobot][0]>150.0-DISTANCE_BOUND &&
PositionOfHomeRobot[whichrobot][1]>85.0)
{ // right wall: case R2
vr = (int)(14 +19.0/90.0*theta_e);
vl = (int)(14 -19.0/90.0*theta_e);
Velocity(whichrobot, vl, vr);
}
else if (m_theta<ANGLE_BOUND &&
PositionOfHomeRobot[whichrobot][0]>150.0-DISTANCE_BOUND &&
PositionOfHomeRobot[whichrobot][1] < 45.0)
{ // right wall: case R3
vr = (int)(-14 + 19.0/90.0*theta_e);
vl = (int)(-14 - 19.0/90.0*theta_e);
Velocity(whichrobot, vl, vr);
}
else if (m_theta>180-ANGLE_BOUND &&
PositionOfHomeRobot[whichrobot][0]>150.0-DISTANCE_BOUND &&
PositionOfHomeRobot[whichrobot][1]<45.0)
{ // right wall: case R4
vr = (int)(14 +19.0/90.0*theta_e);
vl = (int)(14 -19.0/90.0*theta_e);
Velocity(whichrobot, vl, vr);
}
6.6 Game Strategy Development 247
Fig. 6.26. Team robots’ assigned areas according to the zone-defence strategy
6.6 Game Strategy Development 249
void Stop_AllRobots()
{
command[0] = 0xFF; // first 3 bytes of FFH
command[1] = 0xFF; // indicate start of
command[2] = 0xFF; // message
command[3] = 0x0F; // team ID = 0x0F
command[4] = 0x7F; // robot1’s left PWM data
command[5] = 0x7F; // robot1’s right PWM data
command[6] = 0x7F; // robot2’s left PWM data
command[7] = 0x7F; // robot2’s right PWM data
command[8] = 0x7F; // robot3’s (goalie’s) left PWM data
command[9] = 0x7F; // robot3’s right PWM data
}
250 6. Robot Soccer Programming
#define D 2.
#define N 8.
rx = bx + D*cos(btheta);
ry = by + D*sin(btheta);
r1 = atan2(by-y, bx-x);
r2 = atan2(ry-y, rx-x);
r3 = atan2(ry-by,rx-bx);
dist = sqrt((bx-x)*(bx-x)+(by-y)*(by-y));
phi = r3-r1;
while( phi > PI ) phi -= 2.*PI;
6.6 Game Strategy Development 251
theta_d = r1 - N*phi;
return theta_d;
}
2. N_Obstacle()
This function modifies the desired heading angle theta_d and returns it
as F (p), using the subfield univector generated for avoiding a circular
obstacle, as depicted in Fig. 4.21.
(x,y) and (ox,oy) denote the robot’s current position p and the position
of an obstacle’s centre, respectively. Ro and M are the radius of obstacle
and the boundary margin, as shown in Fig. 4.21. With these denotations,
the program code for N_Obstacle() follows:
/sqrt(dist*dist-(Ro+M)*(Ro+M))))+angle;
}
}
}
return theta_d;
}
Using the above two functions, function Uvect(), which returns the desired
heading angle (in radians) of the robot at position p, can be written for
a specified target position g in the presence of several circular obstacles. In
robot soccer, a robot can be treated as (an object placed within the boundary
of) a circular obstacle. There are perhaps several ways to program it; one such
way is reported in [63]. Programming this function is left as an exercise for
the reader.
We now present the program code for Control() which reads the robot’s
current posture (x,y,q), uses Uvect() and implements Eq. (4.23) to compute
and output the control input [v, w]T .
// approximate phi_v
phi_v = phi_v/dl;
254 6. Robot Soccer Programming
a_phi_v = fabs(phi_v);
// calculate theta_e
theta_e = theta_d - q;
a_theta_e = fabs(theta_e);
// calculate v
v1 = ( 2.*V_M - L*K_W*sqrt(a_theta_e) )/( 2. + L*a_phi_v );
if( a_phi_v> 0) {
v2= sqrt(K_W*K_W*a_theta_e + 4*R_M*a_phi_v)-
K_W*sqrt(a_theta_e)
/(2*a_phi_v);
}
else{
v2 = V_M;
}
*v = v1<v2 ? v1:v2 ;
// calculate w
if( theta_e>0.){
*w = *v * phi_v + K_W*sqrt(a_theta_e);
}
else{
*w = *v * phi_v - K_W*sqrt(a_theta_e);
}
}
The generic ‘C’ codes of the two essential program functions for implement-
ing the strategy of limit-cycle navigation for a robot are presented in the
following.
1. TurnClockwise()
Towards attaining the robot’s target posture, this function uses Eq. (4.27)
to generate and return a clockwise field, where (x,y) denotes the position
of the centre of the turning circle and r denotes the radius of the circle.
With these denotations, the program code for TurnClockwise() follows:
6.6 Game Strategy Development 255
ddx = dy + dx * (r - dx * dx - dy * dy);
ddy = -dx + dy * (r - dx * dx - dy * dy);
return theta_d;
}
2. TurnCounterClockwise()
Towards attaining the robot’s target posture, this function uses Eq. (4.29)
to generate a counterclockwise field, where (x,y) denotes the position
of the centre of the turning circle and r denotes the radius of the circle.
With these denotations, the program code for TurnCounterClockwise()
follows:
Using the above two functions, function Limitcycle(), which returns the de-
sired heading angle (in radians) of the robot at its current position (x,y), can
be written for a specified target position in the presence of several obstacles.
Programming this function is left as an exercise for the reader.
To compute and output the control input [v, w]T , the program code for
Control() that implements Eq. (4.23) can be used by replacing Uvect()
with Limitcycle().
The host software model is adapted from the work of other researchers [19].
For beginners of ‘C’ programming, the textbook [64] is a good starting
point.
7. Simulated Robot Soccer
7.1 Introduction
This chapter complements the real-system programming framework presented
in the previous chapter with a computer-simulated system programming
framework.
The simulation framework enables the development of robot soccer sys-
tems for the FIRA Simulated-Robot Soccer Tournament (SimuroSoT), which
essentially is MiroSoT played in a computer simulator. Without the need for
robot and vision hardware, the problems of sensing and acting are reduced to
non-issues, and it becomes possible to focus on game strategy development
for the two bigger leagues in SimuroSoT, namely the middle league (5-a-side)
and the large or full league (11-a-side). This opens up the challenges of using
advanced AI techniques to develop complex game strategies.
This chapter describes the core simulator system for Large League Simuro-
SoT in terms of its architecture, system requirements and general operating
procedure. It then presents the essentials for programming a game strategy.
To illustrate the programming framework, example program fragments that
implement some strategies and related actions are provided. These example
codes could be modified or expanded to incorporate other new ideas and
techniques.
J.-H. Kim, D.-H. Kim, Y.-J. Kim, K.-T. Seow: Soccer Robotics, STAR 11, pp. 257-271, 2004
Springer-Verlag Berlin Heidelberg 2004
258 7. SimuroSoT
eB m
B
pB
B
pB
B
B
ugtxgt enkgpv
Fig. 7.2. Internal architectue of client-server based simulator for robot soccer pro-
gramming
The server consists of five modules, as shown in Fig. 7.2. The function of each
module is described in the following:
• Network communication module
1. It receives the updated information of the ‘virtual’ field from the display
module, and sends it to each client. The field information includes the
coordinate positions and heading directions of the robots and the ball.
2. It receives control information (namely, the desired wheel velocities for
each team robot) from each client and sends it to the kinematics module.
• Kinematics module
1. It computes the latest field information according to some kinematics
models and the latest control inputs from the clients, and sends this
information to the collision test module.
2. The kinematics models employed are given in Section 7.3.
7.2 Client-Server Architecture 259
The client consists of two modules, as shown in Fig. 7.2. The function of each
module is described in the following:
• Network communication module
1. It receives the latest field information from the server and sends it to
the strategy module.
2. It receives the latest control information from the strategy module, and
sends it to the server.
• Strategy module
1. Based on the field information (or game situation) received, it strategi-
cally decides and generates the latest control information for its team
robots.
2. This is the only user-defined module of the simulator; it requires client
programming (in Visual C++ ) to build user-defined action-functions such
as Position(), based on a system-defined function Velocity(), as well
as the game strategy using these basic skill functions. More on this would
be discussed in Section 7.5.
260 7. SimuroSoT
The simulator applies the following model to generate the straight-line motion
of the ball:
where
• af is the acceleration caused by friction,
• k is an integer and k ≥ 1,
• νb (k) and νb (k − 1) refer to the velocities of the ball at time tk and tk−1 ,
respectively,
• (xb (k), yb (k)) and (xb (k − 1), yb (k − 1)) refer to the coordinate positions of
the ball at time tk and tk−1 , respectively,
• θb (k − 1) refers to the ‘heading’ angle of the ball at time tk−1 , and
• time interval △t = tk − tk−1 .
The simulator applies the following model to generate the motion of each
robot:
ν = Rω,
& &
& &
& V
& M &
& (7.2)
ω=& &
& L&
&R + &
2
and
△θ
x(k) = x(k − 1) + R sin △θ cos θ(k − 1) + ,
2
(7.3)
△θ
y(k) = y(k − 1) + R sin △θ sin θ(k − 1) + ,
2
where
• ν, ω and L are the robot’s translational velocity (at its centre), turning
velocity and physical width, respectively,
7.4 How To Run the Simulator 261
L(VM + Vm )
R= ,
2(VM − Vm )
• k is an integer and k ≥ 1,
• ω(k − 1) refers to the robot’s turning velocity ω at time tk−1 , and time
interval △t = tk − tk−1 , such that
△θ = ω(k)△t,
• θ(k) and θ(k − 1) refer to the heading angles of the robot at time tk and
tk−1 , respectively, such that
• (x(k), y(k)) and (x(k − 1), y(k − 1)) refer to the coordinate positions of the
robot at time tk and tk−1 , respectively.
The simulator package for Large League SimuroSoT (in Visual C++ ) is avail-
able for download from website http://www.fira.net.
Fig. 7.3 shows a screen display of the simulation environment when the
SimuroSoT server program is started. The team which is connected to the
server first is the left-team. The server program will function normally only
when both the two teams are connected.
The server program provides a clock, score record, team name, control
commands and robots.
The control commands are displayed at the top of the simulation display
screen. Control commands can be selected via the keyboard; an activated
command is displayed in red. To execute the selected command, press the
‘Enter’ key on the keyboard.
When two teams are connected, their team names are displayed. Following
the Kick-off command, the game starts. To pause the game, use the Break
command. In this case, the clock is paused; following the Start command
the game resumes. If the Stop command is executed, the clock is reset to
0:00. To replay, use the Repeat command.
In the server program, there is an auto-referee which checks for fouls
automatically in accordance to the game rules. Referee displays the current
refereeing decision; for example, if the decision is a penalty-kick, then Referee
displays Penalty-Kick and the game is paused automatically. To resume the
game, use the Start command. The following are useful command keys (from
the keyboard):
7.4 How To Run the Simulator 263
Key : Function
‘p’, ‘l’, ‘;’,‘” : direction keys for positioning the ball or robots,
‘b’ : select the ball,
‘t’ : select the team,
‘s’ : select the robot in order.
For example, to move the 3rd robot in the first team, select the team by
pressing key ‘t’, select the robot by pressing key ‘s’ twice (now the 3rd robot
in the first team is selected). Now you use the direction keys (‘p’, ‘l’, ‘;’,‘”)
to reposition the robot as desired.
In this section, we explain the basic client program structure and client sim-
ulator programming. The client program uses the information about the ball
and robots from the server. The programming environment is MicroSoft Vi-
sual C++ 6.0.
The client program for simulating the motion of the team robots has to be
developed entirely within the system-defined class name CStrategySystem.
When the game starts, the strategy begins at the system-defined C++
method name or function name Action() of the class CStrategySystem.
The following shows an example of a program structure for the func-
tion Action(); it simply calls the user-defined function Think(). Function
Think() consists of several NormalGame()’s, but latest version of the sim-
ulator provides a system-defined variable m_nStrategy which is either 0 or
1. By the structure of function Think(), the user’s game strategy should be
written in the user-defined function NormalGame().
void CStrategySystem::Action()
{
Think();
}
void CStrategySystem::Think()
{
switch(m_nStrategy)
{
7.5 Client Programming 265
case 0:
NormalGame();
break;
case 1:
NormalGame();
break;
case 2:
NormalGame3();
break;
case 3:
NormalGame4();
break;
case 4:
NormalGame5();
break;
}
}
The field information, i.e., the coordinate positions and heading directions
of the robots and the ball, as well as the size of the playground, are, for
programming purposes, obtained via system-defined variables given below:
266 7. SimuroSoT
Posture of Robot.
1. For an opponent robot j, j ∈ [1, 11]:
a) opponent.positionj.x ,
b) opponent.positionj.y .
Note that opponent robot 11 is the goalie.
2. For a team (or home) robot i, i ∈ [1, 10]:
a) homei.angle,
b) homei.position.x,
c) homei.position.y.
3. For the team goalie:
a) hgoalie.angle
b) hgoalie.position.x
c) hgoalie.position.y
In the case of opponent robots, the heading angle information is not given.
The centre of a robot is defined as its position, and is given in real pixel
values. The pixel range of the x-coordinate is [0, 1030] and that of the y-
coordinate is [0, 818]. The heading angle has a range of (−180◦ , 180◦] and
it increases clockwise, a convention which is opposite to that used in the
MiroSoT programming framework.
Position of Ball. The coordinate position (xb , yb ) of the ball is given by
following variables
1. Ball.position.x,
2. Ball.position.y.
Size of Playground. Information on the boundaries of the field gives the
convenience for determining an absolute position. In programming, the con-
stants boundRect.left and boundRect.right are x-boundary values; the
constants boundRect.top and boundRect.bottom are y-boundary values.
The following inequalities relate these system-defined constants:
and
It simulates ‘driving the left and right wheels of a specified robot to attain
the respective input velocities vl and vr. The velocity inputs are ‘normalized’
PWM velocity data lying in the range [−128, 128], where a negative value
‘drives’ the wheel backward, a zero value ‘drives’ it to a halt and a positive
value ‘drives’ it forward.
Position(). This is a user-defined basic skill function given by
switch(whichrobot){
//which means the number of robot, HOME1 is matched to 1
case HOME1:
robot = &home1;
break;
case HOME2:
robot = &home2;
break;
case HOME3:
robot = &home3;
break;
. . .
. . .
case HOME10:
robot = &home10;
break;
268 7. SimuroSoT
case HGOALIE:
robot = &hgoalie;
break;
void CStrategySystem::NormalGame()
{
CPoint target;
int dx,dy;
. . . .
. . . .
. . . .
. . . .
}
pass the ball to some other home robot; otherwise, it would return to its base
position, as follows:
void CStrategySystem::NormalGame()
{
CPoint pos1,pos2,pos3,pos4,pos5,pos6,pos7,pos8,pos9,pos10;
int dx,dy;
pos1.x = boundRect.right
- 3*(boundRect.right - boundRect.left)/8;
pos1.y = boundRect.bottom
- (boundRect.bottom - boundRect.top)/3;
pos2.x = pos1.x;
pos2.y = (boundRect.top + boundRect.bottom)/2;
pos3.x = pos1.x;
pos3.y = boundRect.bottom
- 2*(boundRect.bottom - boundRect.top)/3;
pos4.x = boundRect.right
- 2*(boundRect.right - boundRect.left)/8;
pos4.y = pos1.y;
pos5.x = pos4.x;
pos5.y = pos2.y;
pos6.x = pos4.x;
pos6.y = pos3.y;
pos7.x = boundRect.right
- (boundRect.right - boundRect.left)/8;
pos7.y = boundRect.bottom
- (boundRect.bottom - boundRect.top)/8;
pos8.x = pos7.x;
pos8.y = boundRect.bottom
- 3*(boundRect.bottom - boundRect.top)/8;
pos9.x = pos7.x;
pos9.y = boundRect.bottom
- 5*(boundRect.bottom - boundRect.top)/8;
pos10.x = pos7.x;
pos10.y = boundRect.bottom
- 7*(boundRect.bottom - boundRect.top)/8;
// for robot 1
// compute distance between
// home robot 1’s base position and the ball’s
dx = ball.position.x - pos1.x;
dy = ball.position.y - pos1.y;
Notes on Selected References 271
// for robot 2
// similar as for home robot 1
dx = ball.position.x - pos2.x;
dy = ball.position.y - pos2.y;
// for robot 10
dx = ball.position.x - pos10.x;
dy = ball.position.y - pos10.y;
In the program code above, the Pass() and Goalie() functions must be
implemented with sound strategies. This is left as an exercise for the intersted
reader.
This appendix contains two sections that detail the use of Capture/ Com-
pare/ PWM (CCP) and Universal Synchronous Asynchronous Receiver Trans-
mitter (USART) modules of a PIC16C73/73A microcontroller for, respec-
tively, motion control and communication by a command-based soccer robot
(see Fig. 2.8).
This appendix is to be read in consultation with the PIC16C7X manual,
available for download from website http://www.microchip.com.
To move, a soccer robot needs to generate control signals to turn its motors
(and hence its wheels). The on-chip PWM (pulse width modulation) of a
PIC16C73/73A microcontroller can provide this signal. In PWM, a digital
signal of fixed frequency is generated, and can be used for driving the motors
by connecting up the CCP pins (pin 12 and 13) as shown in Fig. A.1.
Wireless Receiver
Module Power Supply
RX
Motor Gear Box
Motor Gear Box Wheel
PIC16C73/73A
Microcontroller
CCP1
PWM Signals
CCP2 Motor Driver
Motor Driver
Fig. A.1. Using CCP in PWM mode for motor driving and USART for data
reception
274 A. Programming the PIC16C73/73A Microcontroller
To set the CCPx pins, x = 1 or 2, to PWM mode, clear the TRISC2 bit.
The general notation REGi, 0 ≤ i ≤ 7, refers to the i-th bit of an internal
register named REG.
The frequency FPWM of PWM signals output at each CCPx pin is
1
. The PWM period is specified by setting the register PR2.
(PWM period)
The formula for the PWM period in terms of the value (PR2) set in register
PR2 is
where
• TOSC is the time period of the oscillator (system clock generator), and
• TMR2 prescale value for Timer2.
TON
The PWM duty cycle is defined as , i.e., the proportion of time TON
TPWM
within the PWM period TPWM that the PWM signal is at logic 1 (or high).
For CCPx, it can be specified by setting the internal register CCPRxL and
the CCPxCON5 : 4 bits, where the general notation REG i : j refers to
the i-th and j-th bits in an internal register named REG. This forms a 10-bit
value denoted by CCPRxL: CCPxCON5 : 4, where the CCPRxL contains
the eight most significant bits (MSbs) and CCPxCON5 : 4 contains the two
least significant bits (LSbs). The following equation is used to calculate the
duty cycle of the PWM signal output at CCPx, x = 1 or 2, but conventionally
in terms of the time TON as the PWM period TPWM is known and set.
The maximum PWM resolution Rmax (in bits) (for setting the PWM duty
cycle) at a given PWM frequency FPWM is
FOSC
log FPWM
Rmax = bits, (A.3)
log 2
1
where FOSC , equal to , is the frequency of the oscillator.
TOSC
To illustrate, consider an example with FOSC = 20MHz and TMR2
prescale = 1. If the desired PWM frequency FPWM = 78.125kHz, then
A.1 On-chip PWM Programming for Robot Motion Control 275
1 1
= [(PR2) + 1] · 4 · ·1
78.125kHz 20MHz
12.8µs = [(PR2) + 1] · 4 · 50ns·1
(PR2) = 63;
• the maximum PWM resolution Rmax is calculated using Eq. (A.3) as fol-
lows:
1 1
= 2Rmax ·
78.125kHz 20MHz
12.8µs = 2Rmax · 50ns
256 = 2Rmax
Rmax = 8 bits.
This means that with a clock at 20MHz, the duty cycle of a PWM signal
at 78.125kHz can have a resolution of at most 8-bits. i.e.,
So, any such value greater than 255 will result in a 100% duty cycle.
To summarize, the following steps configure the microcontroller for PWM
operation.
1. Set the TMR2 prescale value and enable Timer2 by setting the T2CON
2 bit.
2. Configure CCPx as an output pin by clearing the TRISC2 bit.
3. Set (to fix) the desired PWM period by setting register PR2 with a value
obtained using Eq. (A.1).
4. Set (to initialize or change) the desired PWM duty cycle by setting
CCPRxL:CCPxCON5 : 4 with a value obtained using Eq. (A.2).
The following is a program code fragment that implements these steps.
...
...
...
}
A soccer robot needs to receive data from the host computer via asynchronous
communication. In particular, it needs to receive the desired wheel velocity
data and generate the appropriate PWM signals for motor actuation. The
on-chip USART of a PIC16C73/73A microcontroller provides this means of
data communciation.
The desired communication baud rate is selected by setting the register
SPBRG with a value X, 0 ≤ X ≤ 255, and the BRGH bit of the Transmit
Status and Control register TXSTA, accordingly as follows:
A.2 On-chip USART Programming for Robot Communication 277
FOSC FOSC
Baud Rate
64(X + 1) 16(X + 1)
Using the formulae tabulated above, the value of X can be computed easily
for a desired baud rate. However, since X has to be set as an integer value in
register SPBRG, the actual baud rate selected may deviate from the desired.
It is a good practice to verify that the actual baud rate set is acceptable.
To illustrate, consider the following example with a clock frequency FOSC of
16MHz and a desired baud rate of 9600 bps (bits per second). With BRGH
= 0, we have
16MHz
9600 =
64(X + 1)
X = 25.042
≈ 25.
16MHz
The actual baud rate set is = 9615. Hence the deviation or
64(25 + 1)
error is
• To select a desired baud rate, set the register SPBRG with an appropriate
value X, and the corresponding High Baud Rate Select bit BRGH of the
Transmit Status and Control register TXSTA.
• Under polling,
– to transmit 8-bit data, set the bits of the Transmit Status and Control
register TXSTA as follows:
– to receive 8-bit data, set the bits of the Receive Status and Control
register RCSTA as follows:
To receive data only as in the soccer robot considered, we only need to connect
up the RX pin (pin 10) as shown in Fig. A.1.
Shown below is a program code fragment that configures the microcon-
troller for sending and continuously receiving data via polling at a baud rate
of 19200 bps under a clock frequency of 11.0592MHz, using X = 8 with BRGH
= 0.
...
main()
{
unsigned char config; /* Declare the variable */
config = RX_INT_OFF &
TX_INT_OFF &
USART_ASYNCH_MODE &
USART_TX_RX_MODE &
USART_LO_SPEED &
USART_EIGHT_BIT &
USART_CONT_RX; /* Set config value to 0b00011001 */
...
if (config.0) // CONTINUOUS RX
RCSTA.CREN = 1;
else
RCSTA.SREN = 1; // SINGLE RX
if (config.3) // TX & RX
{
280 A. Programming the PIC16C73/73A Microcontroller
TXSTA.TXEN = 1;
TRISC.TX = 0;
}
TRISC.RX = 1;
if (config.6) // TX INT
PIE1.TXIE = 1;
if (config.7) // RX INT
PIE1.RCIE = 1;
Logic 1
RB7 Switch
GND
PIC16C73/73A
Microcontroller
...
...
Each of the three robots has a unique robot identification number (ID),
0, 1 or 2, assigned similarly using switch (binary) settings at the two inputs
PORTB5 : 4.
Available on website http://www.fira.net is a full microcontroller program
code that incorporates all the program code fragments given above for a team
robot of the experimental system.
B. Reference Manual
for an Experimental MiroSoT System
3. Use the slide bar or type the values for Hue, Saturation, Brightness and
Contrast (see Fig. B.5).
286 B. Reference Manual for an Experimental MiroSoT System
Fig. B.6 shows a default camera image. In order to acquire the optimum
screen display from the camera, adjust the above values. The screen dis-
play is optimum if it displays the image with the colour as is perceived
by the human eyes. Adjust Hue, Saturation, Brightness and Contrast ac-
cordingly, since the fluorescent lights or halogen lamps in the surrounding
are not likely to illuminate every corner of the playground. The following
provide some usual guidelines to follow under normal situations.
a) Decrease Brightness to clearly display the patch colour of the ground.
Fig. B.7 shows an image acquired by adjusting Brightness.
b) Adjust Hue and Saturation to obtain a clear screen display of the
team ID colours (yellow and blue). Note that depending on the Hue
value, yellow may become red or purple. Fig. B.8 shows an image
acquired by adjusting Hue.
1
Company website is at http://www.yujinrobot.com/soccer/soccer-e.htm.
B.1 Vision System: Set-up and Initialization 287
Following the image setting for the camera, set the ball colour as follows:
1. Click [SetColour].
B.1 Vision System: Set-up and Initialization 289
B.1 Vision System: Set-up and Initialization 291
After selecting the ball colour, select the robot team ID colour as follows:
1. Click [Set Colour].
2. Click on a robot on the real-image screen as shown in Fig. B.13. This will
display a three-times enlarged view of the robot on the upper right-hand
side of the screen.
3. Select a square inside the robot team ID colour (yellow or blue) patch
on the enlarged view with the left mouse button. This will pop up the
[YUV Ranges of Colours] dialog box, as shown in Fig. B.14.
4. Select [Team Colour] in [Colour Select], and adjust the YUV to set
the team ID colour in the same way that the ball colour was set.
5. Adjust gradually the Y Min lower and Y Max higher.
If the green part of the screen remains the same without noise as the
Y value is being changed, then change the U value and the V value,
sequentially, to set the required colour. Because each colour has its own
YUV range, simply decreasing Min and increasing Max will not lead to
the optimum colour selection.
6. Click [OK] once a satisfactory result is obtained.
Fig. B.14 shows the colour of the team ID set in this manner.
292 B. Reference Manual for an Experimental MiroSoT System
After setting the team ID colour, set the robot ID colours in the same manner
that the colours of the ball and the robot team ID were set, as follows:
1. Select colour for robot ID1 in [Colour Select].
Fig. B.15 shows the colour range when selecting a robot ID colour.
After setting the ball colour, the robot team ID colour and the robot ID
colours, set the playground boundary as follows:
1. Select the [Set Boundary] button or the Set Ground Area button.
2. Set the boundary in the [Boundary of Field] dialog box (see Fig. B.16).
Prior to setting, apply the colour patch to robots, and then place the
robots on the edge of the ground. Although the robots are placed inside,
on screen, they seem to be out of the boundary. Therefore, the boundary
should be set a little wider.
In changing the range of Top, Left, Bottom and Right, the blue square
is also changed. Take special care to set the boundary size correctly. Note
that the boundary does not include the goal area. It includes just front
294 B. Reference Manual for an Experimental MiroSoT System
line of the goal area. If the area of the boundary is not accurate, the
robot position would be miscalculated.
Fig. B.18 shows the screen display for clicking the [Set Pixel Size] button.
On the [Adjust UB & LB of Pixel] dialog box, the minimum and max-
imum pixel value for each colour (ball colour, team ID colour and robot ID
colour) can be set. When selecting a robot or the ball, the upper left-hand
B.1 Vision System: Set-up and Initialization 295
side of the screen shows ‘number of pixel = XXX ’ with the numeric figure
XXX in green. This value should indicate the pixel size when the pixel size
is set. LB indicates the minimum value, and if the pixel (for the ball or a
robot colour) is smaller than that, the program cannot recognize the object.
When the program fails to identify between the ball and the patch with a
similar colour, adjust this value. Note however that in most cases, there is
no need to adjust these values. (This function is needed only for advanced
colour setting.)
To import the previous settings saved in a file, click the [Open Vision File]
button, and the window as shown in Fig. B.20 would pop up.
b) Click the left mouse button on the team ID colour (yellow or blue)
in the enlarged view on the right-hand side of the ground screen (see
Fig. B.21).
c) Click the mouse on the desired colour in the same enlarged view.
The selected part would be seen changing to green. On the same
dialog box as in [Set colour], only the team ID colour is displayed
on the dark screen.
Fig. B.22. Setting the team ID colour with auto colour setting
d) Check the team ID colour and then click [OK] (see Fig. B.22).
3. To set a robot ID colour, do the following:
a) Set the ID1 colour to green. In the enlarged view on the right-hand
side of the ground screen (see Fig. B.17).
b) Click the left mouse button on green.
The team ID colour would be observed changing into green. In the
same dialog box as in [Set colour], only the team ID colour is dis-
played on the dark screen.
c) Set the ID on the ground screen.
d) Click the [Ok] button once the robot ID colour has changed to green.
With the [Auto Colour Set] button, the program sets the robot team ID
colour and robot ID colour automatically.
1. Select the [Enable] checkbox on [Auto Set Mode].
2. Click the [ID1 Robot] option button for Robot1.
B.1 Vision System: Set-up and Initialization 299
Fig. B.26 shows the menu for changing colours. To change colour, click on
[Change Colour]; a dialog box would be displayed and the screen would
turn black (see Fig. B.27).
B.1 Vision System: Set-up and Initialization 301
302 B. Reference Manual for an Experimental MiroSoT System
Change Colour.
1. MyVisionFrame
Syntax :
void MyVisionFrame (void *data);
Return Value :
NULL
Description :
This system calls a function when a message occurs indicating the
frame grabber memory storage is full. Then it calls the function of
MyVisionFrame().
Example :
digHookFunction (m_hDig, MV_HOOK_FRAME, MyVisionFrame, this)
;
See Also :
MV_HOOK_FRAME, MV_HOOK_FIELD_EVEN, MV_HOOK_FIELD_ODD
2. DisplayDD
Syntax :
void CvisionView::DisplayDD(HBUF hBuf);
Return Value :
NULL
Description :
This function displays the real time image data stored in display
memory; it first receives from the camera via the frame grabber (in
HBUF format) and stores it into the display memory.
3. DisplayOverlay
Syntax :
void CvisionView::DisplayOverlay();
Return Value :
NULL
Description :
Draw an outline of the playground on the display screen, using play-
ground data stored in computer memories in order to prevent display
flicking.
304 B. Reference Manual for an Experimental MiroSoT System
1. OnLButtonDown
Syntax :
void CvisionView::OnLButtonDown (UINT nFlags, CPoint point)
Return Value :
NULL
Description :
This function implements a Window Event , and is used for setting
up the position of each robot in SetColor and its size. It facilitates
saving or setting up of a selected point.
2. Zoom
Syntax :
void CvisionView:: Zoom(CPoint point);
Return Value :
NULL
Description :
This function saves the image data of a selected area into a buffer,
and then reads from the buffer to display it 3-times enlarged on the
screen. It facilitates the initialization of colours for the ball and robot
ID patches.
3. OnMouseMove
Syntax :
void CvisionView:: OnMouseMove(CPoint point);
Return Value :
NULL
Description :
This function draws a square box on the screen, as the mouse moves,
to select an area. It continually updates the colours within the se-
lected area.
4. MinMaxOfRGB
Syntax :
void CvisionView:: MinMaxOfRGB ();
Return Value :
NULL
Description :
This function acquires the image data from a selected area, converts
it to YUV format, and sets the YUV Min & Max values.
306 B. Reference Manual for an Experimental MiroSoT System
5. OnTimer
Syntax :
void CSettingColor::OnTimer();
Return Value :
NULL
Description :
This function displays the data stored in the image buffer. It displays
a currently selected range, and updates/refreshes the screen display
once per 1,200 milliseconds.
6. OnHScroll
Syntax :
void CSettingColor::OnHScroll (UINT nSBCode, UINT nPos, CScroll-
Bar* pScrollBar);
Return Value :
NULL
Description :
This function updates the values of YUV Min and Max thresholds
in the case of scrolling the image on the screen via the scroll bar on
the keyboard.
7. UpdateLUTcolor
Syntax :
CVisionView::UpdateLUTcolor();
Return Value :
NULL
Description :
This function updates the YUV threshold values in the look-up table
as RGB values.
See Also :
void UpdateLUTAllColor();
1. OnWholeFind
Syntax :
CVisionView::OnWholeFind();
Return Value :
NULL
Description :
This function finds the coordinate positions of the ball and robots
on the (whole area of the) playground.
B.2 System Library 307
2. GlobalLabelingPartialImage
Syntax :
CVisionView::GlobalLabelingPartialImage (long OffsetX, long Off-
setY, long ImageSizeX, long ImageSizeY, short object);
Return Value :
NULL
Description :
This function labels all objects with designated colours on the play-
ground, and compute the coordinate values of their centre positions.
It then saves the number of labels and all centre position values. It is
used when searching for the positions of objects such as BALL, HOME1,
HOME2, HGOALIE, and OPPONENT in the whole area. Note that the ac-
308 B. Reference Manual for an Experimental MiroSoT System
3. WholeFind....
Syntax : CVisionView::WholeFind.... ();
Return Value :
NULL
Description :
To be used after executing GlobalLabelingPartialImage, this func-
tion searches for the positions of the ball or robots over the whole
playground area.
Example :
size = PixelSizeOfComponent[BALL_COLOR][whichcomponent];
if(size > maxsize && size > 5 &&
size < UpperBoundOfBallSize/5)
{
maxsize = size;
candidate = whichcomponent;
}
Note that the size would be the number gathered in swarms of size-
filtered pixels.
See Also :
FindBall(), FindOpponentRobot().
4. SearchFindOpponentRobot
Syntax :
CVisionView::SearchFindOpponentRobot ();
Return Value :
NULL
Description :
This function finds the positions of all opponent robots on the play-
ground. It is to be used after checking for the presence of any oppo-
B.2 System Library 309
if((bFlagOpponentFound[RobotID] == FALSE)
&& Size > distance) {
findFlag=TRUE;
RobotCnt=RobotID;
Size=distance;
}
}
5. OpponentColorLabeling
Syntax :
CVisionView::OpponentColorLabeling(long OffsetX, long OffsetY,
long ImageSizeX, long ImageSizeY);
• OffsetX, OffsetY: size of pixel labelling
• ImageSizeX, ImageSizeY: size of screen displaying
Return Value :
NULL
Description :
This function selects the positions of the opponent robots based on
their positions in the whole area moving a position to the whole area
and finds if the opponent robots are in that position.
6. FindOpponentRobot
Syntax :
CVisionView::FindOpponent(short whichRobot, short OffX, short
OffY);
• (OffX , OffY ) : starting point for labelling on the screen.
• WhichRobot : ID number of a specified opponent robot.
Return Value :
NULL
310 B. Reference Manual for an Experimental MiroSoT System
Description :
After size-filtering, this function assigns a number to the group of
pixels that is larger than the preset filter size, and saves the position.
See Also :
FindBall(), FindOpponentRobot().
7. FindObjects
Syntax :
CVisionView::FindObjects
Return Value :
NULL
Description :
This function calculates the color tones based on the locations of
all objects on the playground, and obtains the exact position and
heading angle of each object.
8. DetermineOffsetXY
Syntax :
CVisionView::DetermineOffsetXY(short rx, short ry, short size,
long *offx, long *offy);
• (rx, ry) : coordinate point of the area for labelling
• size : size of the area for labelling
• *offx : data of point X which was converted as a point in the
position of the area in the field
• *offy : data of point Y which was converted as a point in the
position of the area in the field
Return Value :
NULL
Description :
This function keeps the area for labelling within the selected area of
the playground.
9. LabelingPartialImage
Syntax :
CVisionView::LabelingPartialImage(long OffsetX, long OffsetY, long
ImageSizeX, long ImageSizeY, short object);
• NumberOfComponent[COLORID] : number of components
• Component[COLORID][component][pixel][0/1] : component of
label not size-filtered
• PixelSizeOfComponent[COLORID][component] : size of compo-
nent
• CenterXOfComponent[COLORID][component],
CenterYOfComponent[][]: center point of component
• SumOfX[COLORID][component], SumOfY[][] : centre point of pixel
B.2 System Library 311
// Size of component
PixelSizeOfComponent[COLORID][component]
10. FindHomeRobot
Syntax :
CVisionView::FindHomeRobot(short whichRobot, short OffX, short
OffY);
• whichRobot : HOME1, HOME2, HGOALIE
• OffX, OffY : positions of each point started labelling
• PositionOfHomeRobot[][] : position of the robot/(logical coor-
dinate system)
• AngleOfHomeRobot[] : angle of the robot/(logical coordinate sys-
tem)
Return Value :
NULL
312 B. Reference Manual for an Experimental MiroSoT System
Description :
By first locating both the team ID color and robot ID color, this
function determines the position and heading angle of the specified
robot.
Example :
DetermineOffsetXY(PositionOfHomeRobot[HOME1][0],
PositionOfHomeRobot[HOME1][1],
PARTIAL_IMAGE_SIZE,
&OffsetX, &OffsetY);
LabelingPartialImage(OffsetX, OffsetY, PARTIAL_IMAGE_SIZE,
PARTIAL_IMAGE_SIZE, HOME1);
FindHomeRobot(HOME1, (short)OffsetX, (short)OffsetY);
See Also :
wholeFindHomeRobot.
1. SetComPort
Syntax :
CComm::SetComPort(intport,DWORD rate,BYTE bytesize, BYTE
stop,BYTE parity);
• int port : port number
• DWORD rate : baud rate
• BYTE bytesize : capacity of data to be transmitted
• BYTE stop : set up Stop Bit
• BYTE parity : set up parity bit
Return Value :
NULL
Description :
This function sets up the communication port via object ComPort.
Example :
m_pComm.SetComPort(1,19200,8,0,0)
// Port no. 1, BAUD RATE 19200 bps, 8-bit data,
// No Stop bit, No Parity bit
2. CreateCommInfo
Syntax :
CComm::CreateCommInfo ();
Return Value :
NULL
B.2 System Library 313
Set up Comport
SetComPort()
Initialize Comport
CreateCommInfo()
Open Comport
OpenComPort()
Description :
This function implements the TransmitEvent that occurs in ComPort.
In setting up ComPort, make sure that the 3 functions are set in
order, as in the example.
Example :
m_pComm.SetComPort(1,19200,8,0,0);
m_pComm.CreateCommInfo();
m_pComm.OpenComPort();
3. OpenComPort
Syntax :
CComm::OpenComPort ();
Return Value :
NULL
Description :
This function opens ComPort.
314 B. Reference Manual for an Experimental MiroSoT System
4. WriteCommBlock
Syntax :
CComm::WriteCommBlock( LPSTR lpByte , DWORD dwBytesToWrite);
5. CloseConnection
Syntax :
CComm::CloseConnection();
Return Value :
NULL
Description :
This function closes ComPort.
1. Velocity
Syntax :
void Velocity(int whichrobot, int vl, int vr);
Return Value :
NULL
Description :
This function converts and sets the required left-wheel and right-
wheel PWM velocity data for a specified robot. See Section 6.4.1.
B.2 System Library 315
2. Position
Syntax :
void Position(int whichrobot, double x, double y);
Return Value :
NULL
Description :
This function moves a specified robot to a specified position. See
Section 6.4.3.
3. Angle
Syntax :
void Angle(int whichrobot, int desired_angle);
Return Value :
NULL
Description :
This function rotates a specified robot to a specified heading angle.
See Section 6.4.2.
4. Angle2
Syntax :
void Angle2(int whichrobot, double x, double y);
Return Value :
Description :
A variant of Angle, it rotates a specified robot to face directly towards
a desired point instead.
5. AngleOfPosition
Syntax :
void AngleOfPosition(int whichrobot, double x, double y);
• intWhichrobot : HOME1, HOME2, HGOALIE
• doublex : X-coordinate of desired point
• doubley : Y-coordinate of desired point
Return Value :
NULL
Description :
A variant of Angle, this function rotates the goalkeeping robot to a
specified point (x,y).
6. bool DefencePosition
Syntax :
verb bool DefencePosition(int whichrobot, double x, double y);
• int whichrobot : HOME1, HOME2, HGOALIE
• double x : X-coordinate value of specified target position (x,y)
• double y : Y-coordinate value of specified target position (x,y)
316 B. Reference Manual for an Experimental MiroSoT System
Return Value :
• TRUE, if the robot arrived at the selected position
• FALSE, if the robot did not arrive at the selected position
Description :
A variant of Position, this function performs positioning for a spec-
ified robot and in addition, returns a boolean value to indicate
whether or not the robot arrived at the specified position.
7. bool GoaliePosition
Syntax :
bool GoaliePosition(int whichrobot, double x, double y);
• int whichrobot : HOME1, HOME2, HGOALIE
• double x : X-coordinate of a specified target position
• double y : Y-coordinate of a specified target position
Return Value :
• TRUE, if the goalkeeping robot arrived at the specified position
• FALSE, if the goalkeeping robot did not arrive at the specified po-
sition
Description :
Another variant of Position, this function performs positioning for
the goalkeeping robot and in addition, returns a boolean value to
indicate whether or not it arrived at the specified position.
8. bool Goalie
Syntax :
void Goalie(int whichrobot);
Return Value :
NULL
Description :
This function implements the strategy of goalkeeping presented in
Section 6.5.2.
9. Attack
Syntax :
void Attack(int whichrobot1, int whichrobot2);
Return Value :
NULL
Description :
This function implements a two-robot attack strategy via zone-
defence. See Section 6.6.1.
B.2 System Library 317
10. AutoPosition
Syntax :
void AutoPosition();
Return Value :
NULL
Description :
This function makes each robot move to a pre-programmed position
for the FreeBall and KickOff modes. It is invoked via the system
user interface.
12. AvoidBound
Syntax :
void AvoidBound(int whichrobot);
Return Value :
NULL
Description :
This function gets a specified robot to come off the side-wall of the
playground. See Section 6.5.3.
13. AKick
Syntax :
void AKick(int whichrobot);
Return Value :
NULL
Description :
A kick function used for attacking, it implements the univector field
navigation method (see Section 4.6 and Section 6.6.2) that gets a
specified robot to attempt to run through the dynamically changing
ball position.
1. StopRobot
Syntax :
void StopRobot(void);
Return Value :
NULL
Description : This function halts all robots.
2. WhichRobotStop
Syntax :
void WhichRobotStop(int whichrobot);
Return Value :
NULL
Description :
This function halts a specified robot.
B.2 System Library 319
3. Send_Command (void)
Syntax :
void Send_Command(void);
Return Value :
Description :
This function transmits data saved as a command to COMport .
References
15. Myke Predko, PICmicro Microcontroller Pocket Reference, McGraw Hill, New
York, 2002.
16. A. K. Jain, Fundamentals of Digital Image Processing, Prentice Hall, Inc.,
Englewood Cliffs, New Jersey, 1989.
17. Michael K. Sahota, Alan K. Mackworth, Rod A. Barman, and Stewart J. King-
don, “Real-time control of soccer-playing robots using off-board vision : The
Dynamite Testbed,” in Proceedings of IEEE International Conference on Sys-
tems, Man and Cybernetics, October 1995, pp. 3690–3693.
18. Randy Sargent, Billy Bailey, Carl Witty, and Anne Wright, “Dynamic object
capture using fast vision tracking,” AI Magazine, vol. 18, no. 1, pp. 65–72,
Spring 1997.
19. Karl E. Nelson, Jeffrey W. Collins, Michael A. Soderstrand, and Tien C. Hsia,
“Vision systems and software design for soccer playing mobile robots,” In-
telligent Automation and Soft Computing (Special Issue on Soccer Robotics :
MIROSOT’97), vol. 6, no. 1, pp. 19–32, 2000.
20. Randy Sargent, Billy Bailey, Carl Witty, and Anne Wright, “The importance
of fast vision in winning the first micro-robot world cup soccer tournament,”
Robotics and Autonomous Systems (Special Issue on First Micro-Robot World
Cup Soccer Tournament, MIROSOT’96), vol. 21, no. 2, pp. 139–147, September
1997.
21. Ramesh Jain, Rangachar Kasturi, and Brian G. Schunck, Machine Vision,
McGraw-Hill Inc., 1995, McGraw-Hill Series in Computer Science.
22. W. K. Pratt, Digital Image Processing, Wiley, New York, Second Edition,
1991.
23. J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer Graphics:
Principles and Practice, Addison Wesley, Second Edition, 1990.
24. D. F. Rogers and J. A. Adams, Mathematical Elements for Computer Graphics,
McGraw Hill, New York, 1976.
25. R. O. Duda and P. Hart, Pattern Recognition and Scene Analysis, Wiley, New
York, 1973.
26. E. Rich, Artificial Intelligence, McGraw Hill, New York, 1983.
27. P. Winston, Artificial Intelligence, Addison Wesley, Reading, Massachusetts,
Third Edition, 1992.
28. Stuart J. Russell and Peter Norvig, Artificial Intelligence : A Modern Approach,
Prentice Hall, Inc., Upper Saddle River, New Jersey, 1995.
◦
29. Karl Johan Aström and Tore Hägglund, PID Controllers : Theory, Design and
Tuning, Instrument Society of America. Second Edition, 1995.
30. Alf Isaksson and Tore Hägglund, Eds., IEE Proceedings on Control Theory and
Applications (Special Issue on PID Control), Vol. 149, No. 1, The IEE Press,
U.K, January 2002.
31. J. Gulder and V. I. Utkin, “Stabilization of non-holonomic mobile robots using
lyapunov functions for navigation and sliding mode control,” in Proceedings of
the IEEE International Conference on Decision and Control, 1994, pp. 2967–
2972.
32. J. Borenstein and Y. Koren, “Real-time obstacle avoidance for fast mobile
robots,” IEEE Transactions on Systems, Man and Cybernetics, vol. 19, no. 5,
pp. 1179 –1187, 1989.
33. J. Borenstein and Y. Koren, “Potential field methods and their inherent limi-
tations for mobile robot navigation,” in Proceedings of the IEEE International
Conference on Robotics and Automation, April 1991, pp. 1398–1404 (vol. 2).
34. J. Borenstein and Y. Koren, “The vector field histogram - fast obstacle avoid-
ance for mobile robots,” IEEE Transactions on Robotics and Automation, vol.
7, no. 3, pp. 278 –288, 1991.
References 323
35. Hassan K. Khalil, Nonlinear Systems, chapter 7, pp. 289–312, Prentice Hall,
Inc., Englewood Cliffs, New Jersey. 2nd Edition, 1996.
36. Gene F. Franklin, J. David Powell, and Michael L. Workman, Digital Control
of Dynamic Systems, Addison Wesley Longman, Inc., Reading, Massachusetts,
Third Edition, 1998.
37. Phillip John McKerrow, Introduction to Robotics, Addison-Wesley, New York,
1991.
38. Yong-Jae Kim, Jong-Hwan Kim, and Dong-Soo Kwon, “Evolutionary
programming-based uni-vector field navigation method for fast mobile robots,”
IEEE Transactions on Systems Man, and Cybernetics, Part B, vol. 31, no. 3,
pp. 450–458, 2001.
39. Dong-Han Kim and Jong-Hwan Kim, “A real-time limit-cycle navigation
method for fast mobile robots and its application to robot soccer,” Robotics
and Autonomous Systems, 2002, In print.
40. Moon-Su Lee, Myung-Jin Jung, and Jong-Hwan Kim, “Evolutionary
programming-based fuzzy logic path planner and follower for mobile robots,”
in Proceedings of The Congress on Evolutionary Computation, San Diego, USA,
July 2000, pp. 139–144 (Vol. 1).
41. Marc G. Slack, “Navigation templates: Mediating qualitative guidance and
quantitative control in mobile robots,” IEEE Transactions on Systems Man,
and Cybernetics, Part B, vol. 23, no. 2, pp. 452–466, 1993.
42. Erann Gat, “Navigation templates: Enhancements, extensions and experi-
ments,” in Proceedings of the IEEE International Conference on Robotics and
Automation, Atlanta, GA, USA, May 1993, pp. 541–547 (Vol. 1).
43. E. Rimon, “Exact robot navigation using artificial potential functions,” IEEE
Transactions on Robotics and Automation, vol. 8, no. 5, pp. 501 – 518, 1992.
44. Dong-Han Kim, Yong-Jae Kim, Kwang-Choon Kim, Jong-Hwan Kim, and
Prahlad Vadakkepat, “Vector field based path planning and petri net based
role selection mechanism with Q-Learning for the soccer robot system,” In-
telligent Automation and Soft Computing (Special Issue on Soccer Robotics :
MIROSOT’97), vol. 6, no. 1, pp. 75–87, 2000.
45. Leslie Pack Kaelbling, Mitchael L. Littman, and Andrew W. Moore, “Rein-
forcement learning : A survey,” Journal of Artificial Intelligence Research, vol.
4, pp. 237–285, 1996.
46. Laurene V. Fausett, Fundamentals of Neural Networks : Architectures, Algo-
rithms and Applications, Prentice Hall, Inc., Englewood Cliffs, New Jersey.,
1994.
47. Myung-Jin Jung Heung-Soo Kim, Hyun-Sik Shim and Jong-Hwan Kim, “Ac-
tion selection mechanism for soccer robot,” in Proceedings of the IEEE interna-
tional Symposium on Computational Intelligence in Robotics and Automation,
Monterey, USA, July 1997, pp. 390 – 395.
48. Ernst Mayr, Toward a New Philosophy of Biology: Observations of an Evolu-
tionist, Belknap Press, Cambridge, MA, USA, 1988.
49. Thomas Bäck and Hans Paul Schwefel, “An overview of evolution algorithms
for parameter optimization,” Evolutionary Computation, vol. 1, no. 1, pp. 1–23,
1993.
50. Jong-Hwan Kim and H. Myung, “Evolutionary programming techniques for
constrained optimization problems,” IEEE Transactions on Evolutionary Com-
putation, vol. 1, no. 2, pp. 129–140, July 1997.
51. James Lyle Peterson, Petri Net Theory and the Modelling of Systems, Prentice
Hall, Inc., Englewood Cliffs, New Jersey, 1981.
52. Richard S. Sutton and Andrew G. Barto, Reinforcement Learning : An Intro-
duction, MIT Press, Cambridge, Massachusetts, 1998.
324 References
Vol. 1: Caccavale, F.; Villani, L. (Eds.) Vol. 6: Jarvis, R.A.; Zelinsky, A. (Eds.)
Fault Diagnosis and Fault Tolerance for Mechatronic Robotics Research { The Tenth International Symposium
Systems: Recent Advances 580 p. 2003 [3-540-00550-1]
191 p. 2002 [3-540-44159-X] Vol. 7: Boissonnat, J.-D.; Burdick, J.; Goldberg, K.;
Vol. 2: Antonelli, G. Hutchinson, S. (Eds.)
Underwater Robots: Motion and Force Control of Algorithmic Foundations of Robotics V
Vehicle-Manipulator Systems 577 p. 2004 [3-540-40476-7]
209 p. 2003 [3-540-00054-2] Vol. 8: Baeten, J.; De Schutter, J.
Vol. 3: Natale, C. Integrated Visual Servoing and Force Control
Interaction Control of Robot Manipulators: 198 p. 2004 [3-540-40475-9]
Six-degrees-of-freedom Tasks Vol. 9: Yamane, K.
120 p. 2003 [3-540-00159-X] Simulating and Generating Motions of Human Figures
Vol. 4: Bicchi, A.; Christensen, H.I.; 176 p. 2004 [3-540-20317-6]
Prattichizzo, D. (Eds.) Vol. 10: Siciliano, B.; De Luca, A.; Melchiorri, C.;
Control Problems in Robotics Casalino, G.
296 p. 2003 [3-540-00251-0] Advances in Control of Articulated and Mobile Robots
Vol. 5: Siciliano, B.; Dario, P. (Eds.) 259 p. 2004 [3-540-20783-X]
Experimental Robotics VIII
685 p. 2003 [3-540-00305-3]