State Machine for Arbitrary Robots for Exploration and
Inspection Tasks
Marco Steinbrink Philipp Koch
Technical University Bergakademie Nuremberg Institute of Technology
Freiberg,Germany Georg Simon Ohm,Germany
marco.steinbrink@doktorand.t philipp.koch@th-nuernberg.d
u-freiberg.de e
Bernhard Jung
Technical University Bergakademie Freiberg,Germany
jung@informatik.tu-freiberg.de
ABSTRACT
In this paper, a novel state machine for mobile robots is described
that enables a direct use for exploration and inspection tasks. It
offers a graphical user interface (GUI) to supervise the process
and to issue commands if necessary. The state machine was
developed for the open-source framework Robot Operating
System (ROS) and can interface arbitrary algorithms for
navigation and exploration. Interfaces to the commonly used ROS
navigation stack and the explore_lite package are already included
and can be utilized. In addition, routines for mapping and
inspection can be added freely to adapt to the area of application.
The state machine features a teleoperation mode to which it
changes as soon as a respective command was issued. It also
implements a software emergency stop and multiplexes all
movement commands to the motor controller. To show the state
machine’s capabilities several simulations and real-world
experiments are described in which it was used.
CCS Concepts
• Computer systems organization → Robotic autonomy •
Software and its engineering→State systems •
Human-centered computing → Graphical user interfaces
Keywords
State Machine, Exploration, Inspection, Mobile Robotics
ACM Reference format:
Marco Steinbrink, Philipp Koch, Stefan May, Bernhard Jung and
Michael Schmidpeter. 2020. State Machine for Arbitrary Robots
for Exploration and Inspection Tasks. In Proceedings of 2020 4th
International Conference on Vision, Image and Signal Processing
(ICVISP 2020), December 9-11, 2020, Bangkok, Thailand. ACM,
New York, NY, USA,6 pages.
https://doi.org/10.1145/3448823.3448857
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice and
the full citation on the first page. Copyrights for components of this work owned
by others than the author(s) must be honored. Abstracting with credit is permitted.
To copy otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee. Request permissions from
Permissions@acm.org.
ICVISP 2020, December 9–11, 2020, Bangkok, Thailand
© 2020 Copyright is held by the owner/author(s). Publication rights licensed to
ACM.
ACM ISBN 978-1-4503-8953-2/20/12…$15.00
https://doi.org/10.1145/3448823.3448857
Stefan May
Nuremberg Institute of Technology
Georg Simon Ohm,Germany
stefan.may@th-nuernberg.de
Michael Schmidpeter
Nuremberg Institute of Technology Georg Simon
Ohm,Germany
michael.schmidpeter@th-nuernberg.de
1. INTRODUCTION
In ever more diverse areas, mobile robots are deployed to reduce
risks for human workers and automate tasks. Different robotic
layouts and handling renders their control difficult and a task only
suited for specialists. The remote control of complex robots
requires a skilled operator that is alert at all times. Replacing parts
of the control by autonomy reduces fatigue of the operator which
can lead to more efficiency when autonomy may fail and the
operator is required to execute tasks manually.
The novel state machine described in this paper, hereinafter
referred to as Robot Statemachine (RSM), offers an aided control
for arbitrary mobile robots that can be used by simple GUI
commands. It enables sending the robot to explore or follow given
waypoints autonomously and execute special routines when
reaching a goal. These routines can be implemented through
plugins and range from informing the operator to carry out a
routine, to autonomously sweeping a camera and identifying
certain objects to be mapped.
It can be adapted for different mobile robots and tasks through the
implementation of plugins for navigation, exploration and
inspection tasks. Because of these plugins, the RSM is applicable
to 2D and 3D environments, but the following paper will only
discuss the 2D environment in which it was successfully tested
already.
The operations enabled by the RSM can be applied to industrial
settings, underground mining, agricultural areas and many more in
which an initial map has to be extracted at first and repeating
routines executed at specified locations afterwards. Also, in rescue
scenarios simplified control and supervision of robots could
enhance and alleviate their use.
The organization of the paper will be described in the following.
First, recent works regarding state machines for controlling robots
are put into context with this work. Afterwards, the
implementation of the RSM is described as well as the
possibilities offered by the GUI. Interfaces to some existing ROS
packages, for example navigation and explore_lite, are also
explained. Simulations and experiments are described to show use
cases for the RSM and verify it is working. For example, its
application on the robot Schroedi during the RoboCup Sydney
2019, as can be seen in figure 1.

(a)
(b)
Figure 1. Robot Statemachine being used on the robot Schrödi
for exploration in a secluded area during RoboCup Sydney
2019 in (a) and the Robot Statemachine’s GUI embedded in
RViz in (b).
2. RELATED WORK
The RSM uses the previously mentioned ROS framework which
was introduced by Quigley et. al [9] and is one of the most-used
frameworks in robotics research nowadays. It became so popular
because it is available as open-source and enables its worldwide
community to participate on features to be added next. It also
offers many packages which can be used for commonly applicable
tasks like Simultaneous Localization and Mapping (SLAM),
navigation and exploration as well as interfaces to a lot of much
used sensors.
In most cases, to operate a robot using ROS, a multitude of ROS
packages which offer different functionalities must be started
using specifically designed files or commands issued in the
terminal. A change of robot behavior therefore requires a skilled
operator who mostly has to have knowledge about the
programming as well. There currently is a selection of available
state machines in ROS which will be compared to the RSM in the
following.
One of those is SMACH which was developed by Boren and
Cousins [1] and offers a finite state machine which can be realized
with Python code and supervised through a provided GUI. Many
additions to SMACH were published which for example enable
preconfigured, repeatable behaviors [8], decouple design from
programming by providing a description language for the
respective states [4] or introduce core templates and automatic
code block generation [10]. SMACH and its additions allow to
implement complicated behaviors whereas the RSM is more
focused on usability instead of complexity. Therefore it is not as
adaptable as SMACH but can be deployed with less effort for
mobile robots.
FlexBE is also based on SMACH and provides flexibility through
enabling changes on the fly by the operator. It introduces
behaviors for pre-specified tasks which are also adjustable [11].
Even though it offers many possibilities and great flexibility, it
also requires an operator with programming experience which the
RSM tries to render unnecessary.
An addition to FlexBE furthermore integrates its own plugins for
navigation, enabling more interaction with the underlying classes
of the navigation stack[3]. The necessary part of the extended
interaction for mobile robots is also possible with the RSM.
The package ros_control is offering a framework to control all
robot joints and the robot’s movement [2]. It rather aims at
controlling hardware and therefore is not tailored for exploration
and inspection unlike the RSM which offers control of the robot’s
movement as well.
Another alternative is the decision_making_tools package1 which
allows construction of arbitrary behaviors for robots in ROS. But
it is targeted at a lower level control while the RSM targets
high-level autonomy for mobile robots.
Even though the above-mentioned state machines provide
adaptable and extensive frameworks, the RSM aims to offer a less
complex solution for exploration and inspection with mobile
robots while focusing on usability for operators.
3. STATEMACHINE CONCEPTS AND
IMPLEMENTATION
The RSM was written in C++ and follows the UML state pattern
[5]. The class responsible for state transitions only allows volatile
states which makes an additional class for handling data necessary.
All implemented states need to inherit from a virtual base state
which class diagram is shown in figure 2 together with the
surrounding structures. Each state’s base functions are called by
the class handling state transitions. To obtain data necessary for
processing, each state has to interact with the data handler class
through services and topic subscriptions.
The RSM features some integrated states offering basic
functionalities which are listed below:
• Boot State
• Idle State
• Teleoperation State
• Emergency Stop State
• Waypoint Following State
The Boot State is always the first state to be called when the RSM
is started and simply subscribes to a service that should inform
about the boot-up process. When the process finishes, the Boot
State will initiate a transition to the Idle State. The latter only
awaits input from the operator to transition to a desired state. One
of these is the Teleoperation State which becomes active as soon
as a teleoperation command is issued to the robot. The Emergency
Stop State is activated when the operator pushes the software stop
button in the GUI. A transition to the Idle State is only initiated
when the button is released again. The Waypoint Following State
is called when waypoint following is started from the GUI. It
manages the list of waypoints and forwards the next waypoint as a
navigation goal.
To realize exploration and waypoint following, the three plugins
listed below need to be implemented with interfaces to arbitrary
packages:
• Calculate Goal State
• Navigation State
• Mapping State
1
www.wiki.ros.org/decision_making_tools
 Figure 2. RSM partial class diagram showing the most important attributes and methods of each class.
The Calculate Goal State is responsible for extracting a most
suitable exploration goal to go to next. Therefore, it should
interface a package calculating exploration goals. The data handler
class expects a list of possible exploration goals to evaluate if the
current goal is still viable or has already been explored while
moving. This is only used when exploration is run in interrupting
mode, in which navigation is aborted as soon as the current goal
becomes obsolete. The extracted goal from Calculate Goal State is
then forwarded to the Navigation State which needs to implement
an interface to a designated navigation package. This interface
should be able to retrieve the current status of the navigation and
move forward and in reverse. The Mapping State can implement
behaviors to rigorously scan the surrounding by moving the
robot’s sensors. The Navigation State is also used for reaching
waypoints while waypoint following is active.
Waypoints can be linked to certain routines that are implemented
as Routine States. These are called when the respective waypoint
was reached and can execute arbitrary behaviors. Routine States
are not mandatory to implement, but there can be up to ten
different ones.
Figure 3. State diagram showing an exemplary
implementation with normal states, additional plugin states
(bold border) and transitions.
An intended process for exploration would be that the Calculate
Goal State forwards a goal to the Navigation State which then tries
to reach said goal. If it succeeds, the Mapping State is called and
after finishing, transitions back to Calculate Goal State to find a
new exploration goal. If navigation fails, the goal is blacklisted
and Calculate Goal State called to find a new exploration goal. A
state diagram showing these transitions can be seen in figure 3.
Waypoint following works in a similar fashion. The Waypoint
Following State extracts a navigation goal from the list of
waypoints and forwards it to the Navigation State. If the
Navigation State succeeds in reaching the goal, the respective
Routine State is called. When the routine is finished or if there
was no routine set for this waypoint, the Waypoint Following
State is called again to determine a new navigation goal. The
respective waypoint’s status is then set to visited. If navigation
fails, its status is set to unreachable and the Waypoint Following
State is called to provide the next navigation goal.
The RSM also features a class to control the command velocity
output to the motor controller interface. It either maps the
command velocity from teleoperation, from autonomous operation
or a stop command to the motor controller interface. The
command forwarded can be set through the GUI or in case of
teleoperation also automatically by issuing a teleoperation
command. This only works when the software emergency stop is
not pushed in the GUI. If it is, only the stop command is sent to
the motor controller interface and the robot will not move.
A sample implementation provided in the rsm_additions package
features plugins for all of the above states. The Calculate Goal
State interfaces the ROS package explore_lite which is based on
the frontier exploration algorithm introduced by Yamauchi [12].
The plugin subscribes to explore_lite’s visualization that shows
frontiers on a 2D map and extracts the closest frontier center point
to the robot as a navigation goal. The Navigation State realizes an
interface to the navigation ROS package [7]. It forwards received
goals to the navigation stack and also gets feedback from it
regarding the progress. There are two implementations for the
Mapping State where the first one is merely a dummy that
transitions back to the Calculate Goal State. The second one is
meant for an included simulation that will be explained later. This
Mapping State will swivel a depth camera from one side to the
other and back to create a dense 3D pointcloud. A Routine State
called Reversing Routine State is also included and toggles the
reverse mode when the routine is executed. This means the robot
is driving in reverse now when it was going forward before and
vice versa. An additional data handler class is provided that adds
services to interface the explore_lite and navigation packages.
A secondary sample implementation realizes an interface to the
RoNa2 navigation package which is available as open source. This
secondary sample only offers a Navigation State plugin and
henceforth still requires the other plugin states.
To apply the RSM with an arbitrary mobile robot, f.e. the popular
TurtleBot, a user would need to configure the navigation stack
including running SLAM. With these prerequisites the RSM can
be executed using the plugins from rsm_additions to explore or
inspect the area.
4. GUI IMPLEMENTATION
The RSM can be operated through a GUI that enables the use of
all its core functionalities. The GUI is depicted in figure 4 and can
be integrated into RViz or rqt which can be used in ROS for
visualization. It always shows which state is currently active and
gives the user the options explained below.
The GUI offers control over the class handling the command
velocities forwarded to the motor controller interface. This
includes the software emergency stop as well as choosing
autonomy, teleoperation or stopped. When the software
emergency stop is active, the other choices are disabled and the
command velocity is set to stopped until the software emergency
stop is released again.
Figure 4. GUI used to control the Robot Statemachine.
The exploration can be started and stopped by using the respective
buttons in the GUI where the exploration mode can be set as well.
This mode can either be finish or interrupt where the former lets
the robot reach each goal before transitioning to the Mapping State
while the latter starts the transition as soon as the current goal is
no longer listed as an exploration goal.
Waypoint following can also be started and stopped through the
respective buttons. Furthermore, when waypoint following is
stopped, there is the option to reset the current progress and
restore all waypoints to their initial values. It is possible to set the
waypoint following mode to one of the following three options:
single, roundtrip and patrol.
2
www.github.com/schmiddey/rona
The single mode lets the robot start from the first waypoint and
then moves to all consecutive ones in order. Upon reaching the
last one it stops. In roundtrip mode, after reaching the last
waypoint, all waypoints are reset and it starts anew from the first
waypoint. This is repeated until manually stopped. Patrol mode
works in a similar fashion. After reaching the last waypoint, all
waypoints are reset and it starts again in reverse order. The first
and last waypoints are only targeted once and their attached
routines executed only once as well. It can only be stopped
manually.
The GUI also offers the possibility to set a waypoint at the robot’s
current location, with the robot’s current orientation and a desired
routine.
Furthermore, a checkbox enables setting the reverse mode
manually. When the box is checked, the robot moves in reverse.
Figure 5. Waypoints as interactive markers in RViz that can
be manipulated by dragging and their routines changed with
the attached menu.
When using RViz, waypoints can be placed in the map by utilizing
the Set Waypoint Tool. The color of the marker corresponds to
the waypoint’s status: blue is the default color, green means the
waypoint has been visited and red that it is unreachable.
The displayed markers are interactive and can be seen in figure 5.
Their position and orientation can be changed. Furthermore, they
offer a menu where the routine to be executed when reaching the
waypoint can be set and the waypoint deleted.
Also, simple navigation goals can be sent in RViz using the 2D
Nav Goal Tool. The execution of these goals can be interrupted
with the respective stop button in the GUI.
5. SIMULATIONS AND EXPERIMENTS
To test the RSM, two different simulations were realized. The first
one is a 2D simulation to get familiar with the GUI and the RSM’s
principle while the second is a full 3D simulation with physics.
The 2D simulation was realized with the stdr_simulator package
and features a simple circular robot with a 360 degree laser
scanner. A sample launch file for ROS is included in the
rsm_additions package which is ideal for systems with weaker
hardware.
The second simulation is built in gazebo[6] and uses the openly
available husky_simulator package as a base. Through setting the
respective parameters, the robot which is a Husky UGV from
Clearpath Robotics, can be equipped with a 270 degree 2D laser
scanner and a Kinect depth camera. The launch file provided by
the rsm_additions package also features an additional joint that
enables to move the Kinect horizontally around its axis by
approximately 75 degrees in each direction. This will be used by
the Mapping State which moves the camera from left to right and
back to the center. With this, the exploration can be used to create
a dense point cloud. Exemplary screenshots from both simulations
can be seen in figure 6 using the navigation package.
Both simulations were rigorously tested and verified the RSM is
working as intended. Exploration and waypoint following in their
different modes ran as expected and could be interrupted by
teleoperation or hitting the software emergency stop.
The RSM was also tested on two real robots. The first is called
Georg and is a self-built, skid-steer drive, four-wheel ground
vehicle with a Velodyne laser scanner attached to its top. It was
run with ROS navigation configured and also with RoNa
navigation. It was used in indoor office environments for
(a)
(c)
Figure 6. Robot Statemachine simulations with gazebo in (a) and stdr_simulator in (c) showing the simulation view and the
respective RViz view in (b) and (d).
6. CONCLUSION
The RSM was developed to provide a user-friendly interface for
exploration and inspection tasks with arbitrary mobile robots. The
implementation is available as an open-source package3 in ROS
and provides exemplary simulation implementations that can be
used to try out the RSM and get familiar with the GUI and
handling.
It enables the inclusion of arbitrary plugins for exploration,
navigation, mapping and inspection routines which makes it
adaptable to all kinds of mobile robots. The package already
features plugins suited for two different navigation packages and a
set of plugins for running it out of the box. The GUI can be used
as an addition for RViz or rqt in ROS.
Therefore, the RSM is a novel ROS state machine that can be
adapted using C++ and which offers simulations and
implementations that can be executed without further adjustment
effort. This enables testing new algorithms for navigation,
3
www.http://wiki.ros.org/robot_statemachine
exploration and waypoint following and successfully passed all
given tasks.
The second robot is called Schroedi of the AutonOHM team from
Nuremberg Tech and participated in the RoboCup Rescue Robot
League in Sydney 2019. The RSM with RoNa navigation as a
navigation plugin was used for autonomy in some challenges.
Schroedi scored the fifth place and can be seen in figure 1 doing
an autonomous exploration in a secluded area during the
RoboCup.
Unfortunately, the RSM failed in the exploration final of the
RoboCup because the robot’s 2D lidar scanner was not able to
map obstacles attached to a wall slightly above ground. This led to
Schroedi getting stuck repeatedly. Therefore, plugins for 3D
environments will be future work for the RSM.
(b)
(d)
exploration or specific inspection tasks without having to work on
the surroundings first.
The GUI offers basic functionalities and information that are
relevant to the operator which should separate programmer and
operator role to enable more widespread use of robots. It is not
meant for overly intricate tasks but as an easy-to-use state machine
for more inexperienced operators that are not familiar with
programming. This distinguishes the RSM from the already
available state machines based around SMACH.
Future work will be developing and adding plugins suitable for 3D
environments, as the RSM was only successfully utilized in 2D
scenarios by now. Its implementation allows it to be used for 3D
exploration with the respective plugins. Also, self-defined
interrupts could be introduced to enable a state-transition on
currently unsupervised, user-defined events. This will enable a
broader application with tasks unforeseen by the authors.
ACKNOWLEDGMENTS
This activity has received funding from the European Institute of
Innovation and Technology (EIT), a body of the European Union,
under the Horizon 2020, the EU Framework Programme for In 2004 IEEE/RSJ International Conference on Intelligent
Research and Innovation. Robots and Systems (IROS), Vol. 3. IEEE, 2149–54.
https://doi.org/10.1109/IROS.2004.1389727
REFERENCES [7] Eitan Marder-Eppstein, Eric Berger, Tully Foote, Brian
[1] Jonathan Boren and Steve Cousins. 2010. The SMACH
Gerkey, and Kurt Konolige. 2010. The office marathon:
high-level executive. IEEE Robotics and Automation
Robust navigation in an indoor office environment.
Magazine 17, 4 (dec 2010), 18–20.
Proceedings - IEEE International Conference on Robotics
https://doi.org/10.1109/MRA.2010.938836
and Automation (2010), 300–7.
[2] Sachin Chitta, Eitan Marder-Eppstein, Wim Meeussen, Vijay https://doi.org/10.1109/ROBOT.2010.5509725
Pradeep, Adolfo Rodríguez Tsouroukdissian, Jonathan arXiv:1402.5188v1
Bohren, David Coleman, Bence Magyar, Gennaro Raiola,
[8] Hai Nguyen, Matei Ciocarlie, Kaijen Hsiao, and Charles C
Mathias Lüdtke, and Enrique Fernandez Perdomo. 2017.
Kemp. 2013. ROS commander (ROSCo): Behavior creation
ros_control: A generic and simple control framework for
for home robots. In Proceedings - IEEE International
ROS. The Journal of Open Source Software 2, 20 (2017), 456.
Conference on Robotics and Automation. 467–74.
https://doi.org/10.21105/joss.00456
https://doi.org/10.1109/ICRA.2013.6630616
[3] David C. Conner and Justin Willis. 2017. Flexible Navigation:
[9] Morgan Quigley, Ken Conley, Brian P. Gerkey, Josh Faust,
Finite state machinebased integrated navigation and control
Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng.
for ROS enabled robots. In Conference Proceedings - IEEE
2009. ROS: an open-source Robot Operating System. ICRA
SOUTHEASTCON. IEEE, 1–8.
Workshop on Open Source Software. 679–86.
https://doi.org/10.1109/SECON. 2017.7925266
https://doi.org/10.1109/IECON.2015.7392843
[4] Michalis Foukarakis, Asterios Leonidis, Margherita Antona, arXiv:1106.4561
and Constantine Stephanidis. 2014. Combining finite state
[10] Barry Ridge, Timotej Gaspar, and Ales Ude. 2017. Rapid
machine and decision-making tools for adaptable robot
state machine assembly for modular robot control using
behavior. In Lecture Notes in Computer Science (including
meta-scripting, templating and code generation. IEEE-RAS
subseries Lecture Notes in Artificial Intelligence and Lecture
International Conference on Humanoid Robots (2017), 661–8.
Notes in Bioinformatics), Vol. 8515 LNCS. 625–35.
https://doi.org/10.1109/HUMANOIDS.2017.8246943
https://doi.org/10.1007/978-3-319-07446-7_60
[11] Philipp Schillinger, Stefan Kohlbrecher, and Oskar Von Stryk.
[5] Erich Gamma, Richard Helm, Ralph E. Johnson, and John
2016. Human-robot collaborative high-level control with
Vlissides. 1995. Creational Patterns. In Design Patterns:
application to rescue robotics. In Proceedings - IEEE
Elements of Reusable Object-Oriented Software.
International Conference on Robotics and Automation, Vol.
Addison-Wesley, Reading, MA, 305–13.
2016-June. IEEE, 2796–802.
http://119.235.48.116/E-Resources/CMRITCSE/A.Y-2018-19
https://doi.org/10.1109/ICRA.2016.7487442
/III/III-II/DP/DP-Textbook.pdf
[12] B Yamauchi. 1997. A Frontier Based Approach for
[6] Nathan Koenig and Andrew Howard. 2004. Design and use
Autonomous Exploration. Proceedings of IEEE CIRA (1997).
paradigms for Gazebo, an open-source multi-robot simulator.