9th

9th IFAC
IFAC Symposium
Symposium on on Intelligent
Intelligent Autonomous
Autonomous Vehicles
Vehicles
9th
JuneIFAC
29 --Symposium
July 1, on
2016. Intelligent
Messe Autonomous
Leipzig, Germany Vehicles
9th IFAC
June
9th IFAC
29 Symposium
July 1,
Symposium on
2016.
on Intelligent
Messe Autonomous
Leipzig,
Intelligent Germany
Autonomous Vehicles
Vehicles
June 29 - July 1, 2016. Messe Leipzig,Available
Germanyonline at www.sciencedirect.com
June 29 - July 1, 2016. Messe Leipzig, Germany
June 29 - July 1, 2016. Messe Leipzig, Germany

ScienceDirect
IFAC-PapersOnLine 49-15 (2016) 266–271
Context-Based
Context-Based
Context-Based Adaptation
Adaptation
Adaptation of
of
of In-Hand
In-Hand
In-Hand Slip
Slip
Slip
Context-Based Adaptation
Detection for Serviceof In-Hand
Robots Slip
Detection
Detection for
for Service
Service Robots
Robots
Detection for Service Robots
∗ ∗
Jose
Jose Sanchez ∗∗ Sven
Sanchez Sven Schneider
Schneider ∗∗ Nico Hochgeschwender ∗∗∗
Nico Hochgeschwender
Jose
Jose Sanchez
Gerhard
Sanchez ∗ Sven
∗ K.
Sven Schneider
Kraetzschmar
Schneider Nico ∗ Nico
∗ ∗
Nico
∗ Paul Hochgeschwender
G. Plöger
Hochgeschwender ∗
∗ ∗
∗
Jose Sanchez
Gerhard
Gerhard Sven
K.
K. Schneider
Kraetzschmar
Kraetzschmar ∗ Paul
Paul Hochgeschwender
G.
G. Plöger
Plöger ∗
∗
∗ Paul G. Plöger ∗
Gerhard K. Kraetzschmar
Gerhard K. Kraetzschmar Paul G. Plöger ∗
∗
∗ Department of Computer Science, Bonn-Rhein-Sieg University,
∗ Department of Computer Science, Bonn-Rhein-Sieg University,
∗ Department
∗
Department Germany of
of Computer
(e-mail: Science,
{jose.sanchez, Bonn-Rhein-Sieg
sven.schneider, University,
Department Germany
Germany of Computer
Computer
(e-mail:
(e-mail:
Science,
Science, Bonn-Rhein-Sieg
{jose.sanchez,
{jose.sanchez, Bonn-Rhein-Sieg
sven.schneider,
sven.schneider,
University,
University,
nico.hochgeschwender,
nico.hochgeschwender,Germany (e-mail:
Germany gerhard.kraetzschmar,
(e-mail: {jose.sanchez,
gerhard.kraetzschmar, paul.ploeger}@h-brs.de).
{jose.sanchez, sven.schneider,
sven.schneider,
paul.ploeger}@h-brs.de).
nico.hochgeschwender,
nico.hochgeschwender, gerhard.kraetzschmar,gerhard.kraetzschmar, paul.ploeger}@h-brs.de).
gerhard.kraetzschmar, paul.ploeger}@h-brs.de).
paul.ploeger}@h-brs.de).
nico.hochgeschwender,
Abstract:
Abstract: Mobile
Mobile manipulators
manipulators are
are intended
intended to
to be
be deployed
deployed in
in domestic
domestic and
and industrial
industrial
Abstract:
environments
Abstract: Mobile
where
Mobile manipulators
they
manipulators will are
carry
are intended
out tasks
intended to
that
to be
be deployed
require
deployed in
physical
in domestic
domestic and
interaction
and industrial
with
industrial the
Abstract:
environments
environments Mobile
where
where manipulators
they
they will
will are
carry
carry intended
out
out tasks
tasks to
that
that be deployed
require
require in
physical
physical domestic and
interaction
interaction industrial
with
with the
the
surrounding
environments
environments
surrounding world,
where
where
world, for
for example,
they
they
example,will
will picking
carry
carry
picking out
outor
or handing
tasks
tasks
handing that
that over
over fragile
require
require
fragile objects.
physical
physical
objects. In-hand
interaction
interaction
In-hand slippage,
with
with
slippage, i.e.
the
the
i.e.
asurrounding
grasped
surrounding world,
object
world, for
moving
for example,
within
example, picking
the robot’s
picking or
or handing
grasp,
handing is over
inherent
over fragile
fragileto objects.
many
objects. of In-hand
these
In-hand tasksslippage,
and
slippage, i.e.
thus,
i.e.
surrounding
aa grasped
grasped world,
object
object for
moving
moving example,
within
within picking
the
the robot’s
robot’sor handing
grasp,
grasp, is
is over
inherent
inherent fragileto
to objects.
many
many of
of In-hand
these
these tasks
tasksslippage,
and
and i.e.
thus,
thus,
a robot’s
grasped
aa grasped
robot’s ability
object to
object
ability to detect
moving
moving
detect a
within
within
a slippage
the
slippage is
the robot’s
is vital
robot’s
vital for
grasp,
grasp,
for executing
is inherent
is
executinginherent a
a manipulation
to many
to many of
manipulation of thesetask
these
task successfully.
tasks
tasks and thus,
and
successfully. In
thus,
In
this
a robot’s
paper,
robot’s ability
we
ability to
develop
to detect
detect a a
slip
a slippage
detection
slippage is
is vital
approach
vital for
for executing
which
executing is aa manipulation
based on the
manipulation robot’s task
task successfully.
tactile sensors,
successfully. In
Inaa
a
thisrobot’s
this paper,
paper, ability
we
we to
develop
develop detect a
a a
slip
slip slippage
detection
detection is vital
approach
approach for executing
which
which is
is based
baseda manipulation
on
on the
the robot’s
robot’s task successfully.
tactile
tactile sensors,
sensors, In aa
force/torque
this
this paper, we sensor
paper, wesensor
force/torque develop
develop and
and a a combination
slip
a aaslip detection thereof.
approach
detection approach
combination thereof. The
which
which
The evaluation
is based
is based of
evaluation of
on
onourour
the approach,
robot’s
the approach, carried
tactile
robot’s tactile carried out
sensors,
sensors,
out on
on a
force/torque
the Care-O-bot sensor3 and combination thereof. The evaluation of our approach, carried out on
force/torque
force/torque
the
the Care-O-bot
Care-O-bot
sensor
sensor33 platform,
and a
and
platform,
platform,
highly
a combination
combination
highly
highly
suggests
suggests
suggests
that
thereof.
thereof.
that
that
the
Theactions
The
the
the
evaluation
evaluation
actions
actions
and
and
and
motions
of our
of
motions
motions
performed
our approach,
approach,
performed
performed
by
carried
carried
by
by
the
the
the
robot
out
out
robot
robot
on
on
during
the
during grasping
the Care-O-bot
Care-O-bot
grasping 3should
3 platform,
platform,
should be
be taken
highlyinto
highly
taken account
suggests
suggests
into accountthatduring
that the actions
the
during slip
actions
slip detection
and motions
and
detectionmotions for
for improved
performedperformance.
performed
improved by the
by the robot
performance. robot
during
Based
during grasping
on this
grasping should
insight,
should webe
be taken
propose
taken into
an
into account
in-hand
account during
slip
during slip
detection
slip detection
architecture
detection for
for improved
that
improved is performance.
able to
performance. adapt
during
Based
Based grasping
on
on this
this insight,should
insight, we
webe taken
propose
propose into
an
an account
in-hand
in-hand during
slip slip
detection
slip detection detection
architecture for
architecture that improved
that is performance.
able
is able to adapt
to adapt
to
Based
Based
to the
the current
on
on this
current robot’s
this insight,
insight,
robot’s actions
we
we propose
actionsproposeat
at run
runantime.
an in-hand
in-hand slip
time. slip detection
detection architecture
architecture that that is is able
able to to adapt
adapt
to the
to the current
the current robot’s
current robot’s
robot’s actions actions
actions at at run
at run time.
run time.
time.
to
© 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Keywords: Manipulation
Keywords: Manipulation tasks, tasks, slipslip detection,
detection, sensor
sensor fusion,
fusion, tactiletactile sensing,
sensing, force force sensing,
sensing,
Keywords:adaptation
run-time
Keywords: Manipulation tasks,
Manipulation tasks, slipslip detection,
detection, sensor
sensor fusion,
fusion, tactiletactile sensing,
sensing, force force sensing,
sensing,
Keywords:
run-time
run-time Manipulation
adaptation
adaptation tasks, slip detection, sensor fusion, tactile sensing, force sensing,
run-time
run-time adaptation
adaptation
1. INTRODUCTION
1. INTRODUCTION tighter
tighter to avoid
to avoid loosing
loosing the the object.
object. Contrary,
Contrary, when when aa robot
robot
1. INTRODUCTION
INTRODUCTION tighterover to avoid
avoid loosing the object.aContrary,
Contrary, whenreaction
1.
1. INTRODUCTION hands
tighter
tighter
hands to
to
over an
avoid
an object
loosing
loosing
object to
to aa human,
the
the object.
object.
human, a more
Contrary,
more suitable
suitablewhen
when aa
a
robot
robot
robot
reaction
Mobile manipulators such as ARMAR (see Asfour et al. hands
is to
hands over
release
over an
the
an object
grasped
object to
to a human,
aaobject
human, so a
thatmore
aa morethe suitable
human
suitable reaction
can take
reaction
Mobile manipulators
Mobile manipulators
manipulators such
such 3as as
as(seeARMAR
ARMAR (see
(see Asfour
Asfour et al.
et are
al. is hands
is to
is to
toobject.over
release
release the an
theobject
grasped
the grasped to
grasped object human,
object
object so so that
so that more
that thethe suitable
human
the human
human can reaction
can take
can take
take
(2006))
Mobile
Mobile
(2006)) or the
manipulators
or the Care-O-bot
Care-O-bot such
such 3as ARMAR
ARMAR
(see Reiser
Reiser et
(see
(see
et al. (2009))
Asfour
Asfour
al. (2009)) et
et al.
al.
are the
is
the to release
release
object. the grasped object so that the human can take
(2006))
intended orto thebe Care-O-bot
deployed in 3 (see Reiser et al. (2009)) are the object.
(2006))
(2006))
intended or
orto the
thebe Care-O-bot
Care-O-bot
deployed in 33domestic
(see
(see
domestic Reiser
Reiser and
and et
et industrial
al.
al. (2009))
(2009))
industrial envi-
are
are
envi- the
the
The object.
object.
contributions of this paper are three-fold:
intended to
ronments
intended to
to bebe deployed
support
deployedhumans in domestic
in domestic
in their
their and and industrial
work.industrial
Even though, envi- The contributions of this paper are three-fold:
though,
envi-
intended
ronments to
to be deployed
support humans in domestic
in and
work. industrial
Even envi- The The contributions
contributions of of this
this paper
paper are are three-fold:
three-fold:
ronments
these
ronments to
to support
environments
support humans
are
humanshighly in
in their
dynamic,
their work.
work. Even
the
Even though,
robots
though,are The •• contributions
We develop of this
three types paperof are detectors
slip three-fold: based on the
ronments
these to
these environments support
environments
environments humans
are
areandhighly
highly in their
dynamic,
dynamic, work. Even
the
thea wide though,
robots
robots are
are We
• We develop
We develop
develop three types
three measurements
types of of slip
of slip detectors
slip detectors
detectors based
based as on the
onwell
the
required
these
these
required to
to successfully
environments
successfully are
areandhighly
highly robustly
dynamic,
dynamic,
robustly perform
perform the
thea robots
robots
wide range
are
are
range • • tactile
We
tactiledevelop and
and force
three
three
force types
types
measurementsof slip of the
detectors
of the robot
based
based
robot on
on
as the
the
well
required
of tasks.
required to
The
to successfully
first
successfullystep and
to
and dealrobustly
with
robustly perform
such
perform a wide
environments
aa wide range
range is tactile
as aa fusion and force
of measurements
these. of the robot as well
required
of tasks. to
The successfully
first step and
to dealrobustly
with perform
such wide
environments range is tactile
tactile
as and
and
fusion force
force
of measurements
measurements
these. of
of the
the robot
robot as
as well
well
of tasks.
tasks. The first
first step
step toto deal with robots,
such environments
environments is as aa fusion
fusion of these.
these.
their
of
of
their
their
perception.
tasks. The firstTherefore,
The
perception.
perception. step to deal
Therefore,
Therefore, dealmodern
modern
modern
with such
with robots,
robots,
as
as
as
the
such environments
the
the
ones
ones
ones
in-
is
is
in-
in- ••• as
Then,
as
Then,
Then, a fusionwe experimentally
we
we
experimentally
of
of these.
experimentally
show how
show
show
how the
how
the performance
thetask
performance
performance
troduced
their
their before,
perception.
perception.
troduced before, are equipped
Therefore,
Therefore,
are equipped with
modern
modern
with a
a multitude
robots,
robots,
multitude as
as of
the
the
of sensors,
ones
ones
sensors,in-
in- of
•• Then,
Then,
of each
each we
weslip detector
experimentally
experimentally
slip detector varies
variesshow
show with
with how
how the
thethe
the task currently
performance
performance
currently
troduced
not only
troduced before,
before, are
exteroceptive,
are equipped
but
equipped with
also
with a multitude
proprioceptive
a multitude of
of sensors,
ones
sensors, to of each
executed slip
by detector
the robot. varies
In with
this the
context, task thecurrently
perfor-
troduced
not only before, are
exteroceptive, equipped
but with
also a multitude
proprioceptive of sensors,
ones to of
of each
each
executed slip
slip
by detector
detector
the robot. varies
varies
In with
with
this the
the
context, task
task thecurrently
currently
perfor-
not
create
not only
a
only exteroceptive,
coherent
exteroceptive, but
representation
but also
also of proprioceptive
their environment
proprioceptive ones
ones andto
to executed
mance of by
the the
slip robot.
detectors In this
is context,
evaluated on the
a perfor-
Care-O-
not
createonly exteroceptive,
aa coherent but alsoof
representation proprioceptive
their environment onesand to executed
executed
mance of by
by the
the the
slip robot.
robot.
detectors In
In is this
this context,
context,on
evaluated the
the perfor-
perfor-
aa Care-O-
create
detect
create coherent
external
aa coherent representation
disturbances.
representation The of
of their
major
their environment
problems
environment here and
are
and mance
bot 3 of the
platform, slip detectors
where we is
measureevaluatedthe on Care-O-
robustness by
create
detect coherent
external representation
disturbances. The ofmajor
their problems
environment here and
are mance
mance
bot of
of the
the slip
33 platform, detectors
slipwhere
detectors we is
is evaluated
measureevaluatedthe on
on aa Care-O-
Care-O-
robustness by
detect
1) the
detect external
limitations
external disturbances.
inherent
disturbances. to The
the
The major
sensors;
major problems
2)
problems the here are
different
here are bot
comparing platform,
the where
number we
of measure
successful the
and robustness
unsuccessful by
detect
1) the external
limitations disturbances.
inherent to The
the major
sensors; problems
2) the here are
different bot
bot 3
comparing platform,
3 platform, the where
where we
number we
of measure
measure the
successful the
and robustness
robustness
unsuccessful by
by
1)
1) the
the limitations
characteristics
limitations and inherent
modalities
inherent to
to the
of
the thesensors;
data
sensors; 2) the
measured
2) the different
by the
different comparing
slip detection the number
results. of successful and unsuccessful
1) the limitations
characteristics and inherent
modalities to the
of thesensors;
data 2) the
measured different
by the comparing
comparing
slip detection the
the number
number
results. of
of successful
successful and
and unsuccessful
unsuccessful
characteristics
sensors; and
characteristics 3) and
the
and modalities
fusion
modalities of the ofmeasurements
theof the data
the data measured by the
considering the slip detection
•• slip
Finally, detection
we results.an adaptive slip detection ap-
propose
characteristics
sensors;
sensors; and
and 3)
3) and
the
the modalities
fusion
fusion of
of the the data measured
ofmeasurements
measurements measured by
by the
considering
considering • slip
Finally,
Finally, detection
we
we
results.
results.an
propose
propose adaptive
antheadaptive
adaptive slip
slip detection
detection ap-
ap-
the previous
sensors;
sensors;
the and
and 3)
previous two
3)
two theproblems.
fusion
fusion of
theproblems. of thethe measurements
measurements considering considering proach
•• Finally,
Finally,
proach which
we
we propose
which enables
propose
enables an
anthe run-time
adaptive
run-time selection
slip
slip detection
detection
selection of
of slip
ap-
ap-
slip
the
the previous
previous two
two problems.
problems. proach
detectors which
suitable enables
for the
the run-time
current task.selection of slip
the previous
Especially, two
highly problems.
delicate manipulation and grasping proach
proach
detectors which
which
suitable enables
enables
for the
the
the run-time
run-time
current task.selection
selection of
of slip
slip
Especially, highly delicate manipulation and grasping detectors suitable
detectors suitable for for thethe current
current task. task.
Especially,
tasks that
Especially, highly physical
require
highly delicate interaction,
delicate manipulation
manipulation such and
andas grasping Following
handling
grasping detectors this suitable
introduction, for theSection
current2task. describes the slip
Especially,
tasks
tasks that
that highly
require
require delicate
physical
physical manipulation
interaction,
interaction, such
such andas grasping
handling
as handling
handling Following
Following used this introduction,
this introduction,
introduction, Section
Section 2 describes
2 describes
describes the the slip
the slip
slip
of
of food
tasks
tasks
food that
thatand
and fragile
require
require
fragile goods
physical
physical
goods or
or handing
interaction,
interaction,
handing over
oversuch
such objects
as
as
objects to
handling
to hu-
hu- detectors
Following
Following
detectors this
this
used in this
introduction,
in this work.
work. The
Section
Section
The context-adaptive
2
2 describes
context-adaptive the ap-
slip
ap-
of
of food
mans,
food and
demand
and fragile
the
fragile goods
precise
goods or
or handing
reaction
handing to over objects
disturbances
over objects to
at
to hu-
run
hu- detectors
proach and used
the in this
associated work. The
architecture context-adaptive
is proposed in ap-
Sec-
of food
mans, and
demand fragile
the goods
precise or handing
reaction to over objects
disturbances to
at hu-
run detectors
detectors
proach and used
used
the in
in this
this
associated work.
work. The
The
architecture context-adaptive
context-adaptive
is proposed in ap-
ap-
Sec-
mans,
time.
mans, demand
One
demand suchthe the precise
disturbance,
precise reaction
which to to disturbances
wedisturbances
investigate in at
in run
this proach
tion 3. and
Section the associated
44 presents architecture
the related is
workproposed
and in Sec-
discusses
mans,
time.
time. demand
One
One such
such the precise reaction
disturbance,
disturbance, reaction
which
which to
we
we disturbances
investigate
investigate
at
in
run
run proach
at this
this proach
tion
tion 3.
3.
and
and
Section
Section
the
the associated
associated
4 presents
presents
architecture
architecture
the
the related
related
is
is proposed
work
workproposed
and
and
in
in Sec-
Sec-
discusses
discusses
paper,
time.
time. Oneis in-hand
such slippage,
such disturbance, i.e.
which a
aa grasped
we
we investigate object moves
in this
this tion the approach.
3. SectionFinally,
3. Section 44 presents our conclusions
the related
related work are summarized
and
and discusses in
paper,One
paper,
within
paper,
is
the
is
in-hand
is in-hand
in-hand
robot’s
disturbance,
slippage,
slippage,
grasp.
slippage, To
which
i.e.
i.e.
detect
i.e. a
grasped
grasped investigate
in-hand
grasped
object
object
slippage,
object
in
moves
moves
our
moves
tion
the approach.
the approach.
Section approach.
5.
presents
Finally,
Finally, our the
our conclusions
conclusions work
are summarized
are summarized
summarizeddiscusses in
in
paper,
within is
the in-hand
robot’s slippage,
grasp. To i.e.
detect a grasped
in-hand object
slippage, moves
our the
the
Section approach.
5. Finally,
Finally, our
our conclusions
conclusions are
are summarized in
in
within
robots
within the
are
the robot’s
equipped
robot’s grasp.
with
grasp. To
To detect
tactile
detect in-hand
sensors
in-hand in slippage,
their
slippage, our
hands,
our Section 5.
within
robots the
are robot’s
equipped grasp.
with To detect
tactile in-hand
sensors in slippage,
their our
hands, Section
Section 5.
5.
robots
but
robots are
alsoare equipped with
force/torque
equipped with
sensors tactile
tactilein theirsensors
their
sensorsarms. in However,
in their hands,
their hands,
the
robots
but
but also
also are equipped with
force/torque
force/torque sensors
sensors tactilein
in sensors
their arms.
arms. in However,
their hands,
However, the
the
involved
but
but also
also
involved sensors
force/torque
force/torque
sensors are
are affected
sensors
sensors
affected by
by the
in
in
the actions
their
their
actionsarms.
arms. which
which the
However,
However,
the robot
the
the
robot
involved
performs,
involved sensors
for
sensors are
instance,
are affected
a
affected by the
force/torque
by the actions
actions which
sensor
which is the robot
sensitive
the robot
involved
performs,
performs, sensors
for
for are
instance,
instance,affected
a by the
force/torque actions which
sensor is the robot
sensitive 2. SLIP DETECTION
to motion
performs,
performs,
to motion whereas
for
for a
instance,
instance,
whereas a aa
tactile
a
tactile
force/torque
sensor
force/torque
force/torque
sensor might
might
sensor
not
sensor
sensor
not
is
be
is
is
be
sensitive
affected
sensitive
sensitive
affected
2. SLIP
2. SLIP DETECTION
SLIP DETECTION
DETECTION
to
at motion
all. Also whereas
Alsowhereas a
the reactions tactile
reactions sensor
tosensor might
accommodate not be
for affected
detected 2.
2. SLIP DETECTION
to
to
at motion
motion
all. whereas
the aa tactile
tactile to sensor might
might
accommodate not
not be
be
for affected
affected
detected
at
at all.
slippage
all. Also
are the reactions
context-dependent. to accommodate
For instance, for detected
when the In this section we describe three slip detectors using tactile
at all. Also
slippage
slippage Also
are the
are the
reactions
reactions to
context-dependent.
context-dependent. to accommodate
accommodate
For instance,
For instance,
instance,
for detected
forwhen
detected
when the
the In In
In thisthis section
this section
section we describe
we describe
describe three
three slip detectors
slip detectors
detectors using tactile
using tactile
tactile
robot
slippage
slippage
robot carries
are
are
carries an
an object
object and
context-dependent.
context-dependent.
and feels
feels slippage,
For
For
slippage, it
instance,
it should
when
when
should grasp
the
the
grasp sensing,
In this
sensing, force
section
force sensing
we
we describe
sensing and
and a combination
three
three
a slip
slip
combination detectorsof
of them.
using
using
them. tactile
robot
robot carries an object and feels slippage, it should grasp sensing, force sensing and a combination of them.
robot carries
carries an an object
object andand feels feels slippage,
slippage, it it should
should grasp
grasp sensing,
sensing, force force sensing
sensing and and aa combination
combination of of them.
them.
Copyright
2405-8963 ©
Copyright © 2016,
2016 IFAC
IFAC (International Federation of Automatic Control)266 Hosting by Elsevier Ltd. All rights reserved.
Copyright ©© 2016
2016 IFAC
IFAC 266
266
Peer review
Copyright © under
2016 responsibility
IFAC
Copyright © 2016 IFAC of International Federation of Automatic
266Control.
266
10.1016/j.ifacol.2016.07.765
IFAC IAV 2016
Jose Sanchez et al. / IFAC-PapersOnLine 49-15 (2016) 266–271
June 29 - July 1, 2016. Messe Leipzig, Germany 267

2.1 Approach apT qT b

f lowx = 
s , f lowy = 
r (3)
pi qi
Based on the study of human tactile sensing, Howe (1993) i=1 i=1
proposed to equip robot manipulators with sensors able Where a and b defined as: a = [−(n − 1), . . . , −(n − s)]
to perceive different signals (e.g. vibration, contacts). For T
instance, a tactile sensor estimates a pressure distribution and b = [−(m − 1), . . . , −(m − r)] , and represent the
while a force/torque sensor is able to measure external cell positions in the X and Y direction of the pressure
forces and torques. Furthermore, Melchiorri (2000) showed matrix, respectively. Furthermore, p and q are vectors
that slip detection can be performed with a combination of representing the mean value of columns and rows of the
force/torque and tactile sensors using a Coulomb friction convolution matrix C[k], respectively. Defined formally as,
 r 
model. However, it requires knowledge of the friction 1
coefficients which might not be available when handling p= cij f or j = 1, . . . , s (4)
r i=1
unknown objects.  
1  s 
We propose an approach to detect slippage of a grasped
q= cij f or i = 1, . . . , r (5)
object that does not require a priori information about the s 
j=1
object being manipulated. In this context, a slip is defined
as the object being translated within the grasp (e.g. if the Finally, the tactile flow is found using the results of
object is pulled down this will result in a downwards slip, equation 3,
see Fig. 1 and Fig. 2). Torque and tactile sensors are used f lowtactile = f low[k] − f low[k − 1]2 (6)
to compute signals that indicate a possible slippage.
with f low = [f lowx , f lowy ]. This computation can be
Based on the torque sensor in each joint, the KUKA applied to tactile sensors of different shapes, provided
Lightweight Robot 4 (LWR4), see Bischoff et al. (2010), their output is a two dimensional array. Having N tactile
can estimate the wrench (force and torque) applied to the sensors, we define sliptactile as:
arm’s end-effector. The wrench is measured at a rate of 50 N

Hz. For grasping and manipulating objects, the robot is sliptactile = E [P[n]] · f lowtactile [n] (7)
equipped with the three-fingered Schunk dexterous hand n=1
SDH-2. Each finger has two tactile sensors built by Weiss Where E[P[n]] is the pressure average and f lowtactile [n]
Robotics (2015) to measure pressure caused by contacts. is the tactile flow of the n-th tactile sensor. This linear
The tactile sensors operate at an average rate of 30 Hz. combination allows tactile sensors with higher pressure
values to contribute more information regarding how an
Force slip detection We assume a slip occurs whenever a object is slipping from the grasp, since sensors with lower
force is exerted in the right direction (e.g. downwards with intensity values, arising from spurious contacts, might not
respect of the grasp frame). A fdirection signal is computed provide an accurate measure of slippage. Note that the
as follows: f lowtactile , as defined in equation 6, is an absolute value
and thus the direction of the slippage is not considered to
sensor
fgrasp = Rgrasp · fsensor (1) produce the sliptactile (see Fig. 1b).
T
fdirection = fgrasp · (fx , fy , fz ) (2)
Combined slip detection The slipcombined is computed
Where fsensor is the force measured w.r.t. the sensor by combining both tactile and force slip signals as
sensor 
frame, and Rgrasp represents the orientation of the grasp  slip up if (sliptactile ≥ thresholdtactile ) ∧
w.r.t. the sensor frame. The orientation depends on the 


 (slipf orce ≥ thresholdf orce )
hand and grasp type. (fx , fy and fz ) selects the direction 
in which an object can slip up or down within the hand. For slip = slip if (sliptactile ≥ thresholdtactile ) ∧


example, as shown Fig. 2 this vector is then set to (1, 0, 0). 
 downwards (slipf orce ≤ −thresholdf orce )


Note that the torque components, for this particular setup, n/a otherwise
are ignored since their measurements were in the range of
Where the thresholdtactile and thresholdf orce , represent
noise level (see Fig. 1a). However, the force components
were sufficient to detect a slip. numerical values 1 for detecting a slip based on the
sliptactile and slipf orce , respectively. Both of these thresh-
olds are chosen experimentally.
Tactile slip detection To compute the sliptactile signal
we apply the algorithm proposed by Alcazar et al. (2012) 2.2 Experiments & Evaluation
to each tactile sensor, which estimates the tangential force
on the sensor caused by a sliding pressure (e.g. a grasped The three slip detectors were evaluated using two different
object slipping). Specifically, a two-dimensional convolu- grasp shapes, namely a grasp that uses all three fingers
tion is computed between a tactile sensor’s pressure matrix and one that only uses two fingers. The thresholdf orce
P[k] of size (m × n), and its previous pressure matrix, and thresholdtactile were set to 1.5 and 5e−3 , respectively.
P[k − 1]; the output is the convolved matrix, C[k], of size Three different objects were used in the experiments. A
(r × s), with r = (2m − 1) and s = (2n − 1). coffee paper cup, an empty Pringles can and a Sprite
We then compute the tactile flow in each axis using the bottle. Seven actions were performed on each object ten
following equations, adapted from Alcazar et al. (2012), 1 These values can be chosen to increase the sensitivity of the slip

as: detectors, e.g. lower values result in higher false positives.

267
IFAC IAV 2016
268 Jose Sanchez et al. / IFAC-PapersOnLine 49-15 (2016) 266–271
June 29 - July 1, 2016. Messe Leipzig, Germany

(a) Force/torque sensor raw output.

(b) Tactile slip signal computed using Equation 7.

Fig. 2. Robot platform used for the evaluation.
Fig. 1. A slip downwards occurring at approximately 2.5 Table 1. Performance rate of slip detectors in
seconds. (a) shows the raw output of the force/torque detecting a true slippage.
sensor and (b) presents a computed signal based on
the tactile array sensors. Slip down perceived by the False positives
force (a) and tactile (b) sensors. Tactile Force Combined
grasp 60/60 10/60 0/60
times, producing 60 tests per action. The seven actions are move base 1/60 51/60 0/60
described as follows: release 50/60 1/60 1/60
rotate counter-clockwise 39/60 6/60 1/60
grasp: the fingers of the gripper close to hold the object rotate clockwise 50/60 9/60 4/60
move base: the robot’s base moves back-and-forth ap-
proximately 20 cm while holding the object Table 2. Performance rate of slip detectors in
release: the fingers of the gripper open to release the detecting a true slippage.
object
rotate counter-clockwise: the grasped object was ro- True positives
tated counterclockwise within the grasp Tactile Force Combined
rotate clockwise: grasped object was rotated clockwise slip downwards 53/60 60/60 49/60
within the grasp slip up 50/60 60/60 47/60
slip downwards: the grasped object was pulled down to also showed that the slip detectors achieve a similar
simulate the object slipping down (shown in Figure 2) performance (±5%) regardless of the object and grasp.
slip up: the grasped object was pulled up from the grip- Already in the few experiments we have performed with
per. the inclusion of actions (i.e. grasp, release and move base),
Only slip downwards and slip up were considered as a we can see that the action has a major influence on the
slip. The results for the three detectors are summarized in performance of the slip detection.
Table 1 and Table 2.
3. ROBUST SLIP DETECTION VIA
2.3 Preliminary Results & Discussion CONTEXT-BASED ADAPTATION

The performance of each slip detector varies considerably
depending on the action, e.g. the tactile slip detector
outputs a slip whenever grasping an object, as shown
in Table 1. Contrary, the force slip detector achieved a
perfect accuracy for detecting actual slips as displayed
in Table 2, however its performance was poor when no
slippage occurred, particularly in the move base action
(see Table 1). The combined slip detector had the lowest
accuracy for actual slips, but its performance was by far
the best when no slippage occurred. The experiments
The evaluation of the stand-alone slip detectors in the
previous section indicates that the robustness of slip de-
tection clearly benefits from the run-time adaptation of
the manipulation architecture based on the action that
the robot performs. At the core, the proposed architec-
ture (see Fig. 3) consists of a control loop where a grasp
controller commands the robot’s hand to grasp an object.
As input, the grasp controller receives, on the one hand,
the grasp to be executed (e.g. desired joint values) and,
on the other hand, a signal if the object slips within the

268
IFAC IAV 2016
Jose Sanchez et al. / IFAC-PapersOnLine 49-15 (2016) 266–271
June 29 - July 1, 2016. Messe Leipzig, Germany
Fig. 3. The core of the architecture consists of the slip detector (red) to analyze the robot’s state and the grasp controller
(green) which commands the robot. At run time, a specific slip detector is selected by the detector selection pipeline
(violet). Similarly, a grasp controller can be selected online, given the robot’s task context (yellow).
hand. As described in the previous section, the slip signal
is derived from the data provided by the robot’s tactile
sensors and the force/torque applied to the robot’s arm. To
accommodate for the different robot’s actions, we propose
the design of multiple slip detectors and grasp controllers
which are in turn selected at run time with the knowledge
of the robot’s action and task context.
3.1 Representation and Storage of Slip Detectors
To make different slip detectors available during run time,
we need to model and store them in an explicit manner.
We employ the Robot Perception Specification Language
(RPSL) introduced in Hochgeschwender et al. (2014) to
model the various slip detectors and their corresponding
properties. The RPSL is a domain-specific language (DSL)
which provides suitable abstractions to model robot per-
ception systems such as those introduced in this paper (see
Fig. 4). In particular, with RPSL we can model multi-stage
slip detectors in the form of directed acyclic graphs where
sensor components (e.g. tactile and force sensors) are root
nodes and processing components are leaf nodes. Further,
RPSL provides means to model the data in- and output
types employed in the slip detectors such as pressure ma-
trices of the tactile sensors. Beyond structural information
the RPSL enables the attachment of contextual informa-
tion about the appropriateness of a slip detector for a cer-
tain context. In this work, the appropriateness is expressed
as a ranking of symbols where each symbol represents a
motion context such as grasp, move base and release.
Similarly to our previous work (see Hochgeschwender et al.
(2015) and Blumenthal et al. (2015)), the attachment of
contextual information is performed during design time
and after some preliminary experimentation when more
domain knowledge is available.
3.2 Monitoring Motion
To retrieve the current motion context we perform
symbol grounding by employing the Conceptual Space
(CS) knowledge representation framework, as proposed
269
269
by Gärdenfors (2004). The CS framework is supported by
RPSL and allows to ground different motion contexts such
as grasp, move base and release through the notion of
concepts, domains and dimensions. In the CS framework,
concepts are convex regions in a set of domains which
are composed of measurable dimensions. In the context of
this work the measurable dimensions are the joint values
(velocities) for each joint. This allows us to model for each
concept a set of prototypes encoding typical values. For
example, the prototype for the move base concept has
only zero-values for each arm joint whereas the base joints
are non-zero. During run time, the motion monitor then
computes for each joint state sample the closest match-
ing prototype by employing the Euclidean distance as a
metric.
3.3 Run-time Selection of Slip Detectors
To select a slip detector which is appropriate for the
current motion context we apply a simple, yet powerful
rule-based approach. During design time we devise a set
of decision rules where the motion context is part of the
condition and the selection of a slip detector is part of the
rule body. The slip selector also performs the activation
of slip detectors on an implementation level. This happens
only, if the selected slip detector is not already activated.
One might argue that activating and deactivating compo-
nents is unfeasible from a timing perspective. However, as
we have shown in Gherardi and Hochgeschwender (2015)
activating and deactivating system components in a time-
critical robotic application context is feasible.
3.4 Task-based controller adaptation
In parallel to the adaptation of the slip detection pipeline
it is also beneficial to select specific grasp controllers
during the robot’s run time. Here, we build upon our
previous work in Sanchez et al. (2015), where we proposed
a grasp controller that reduces the exerted force on the
grasp object. Obviously, the exerted force is not the only
criterion to be optimized in grasping, but also the stability
IFAC IAV 2016
270 Jose Sanchez et al. / IFAC-PapersOnLine 49-15 (2016) 266–271
June 29 - July 1, 2016. Messe Leipzig, Germany
rpsl . sensor_compon e n t do
name " force_sensor "
add_port : out , " out_port " , " wrench "
end
rpsl . p r o c e s s i n g _ c o m p o n e n t do
name " slip_d etectio n "
add_port : in , " in_port " , " wrench "
add_port : out , " out_port " , " force_slip "
end
rpsl . perception_g r a p h do
name " f o r c e _ s l i p _ d e t e c t o r "
connect " force_sensor " , " out_port " ,
" sl ip_dete ction " , " in_port "
attach_prototyp e " task_context " , " move_base "
end
Fig. 4. An excerpt of the domain model of the force-based
slip detector represented in RPSL (see Hochgeschwen-
der et al. (2014)). Two atomic components are mod-
eled, namely a force sensor providing wrench data
(out port) and a slip detection component de-
manding wrench data (in port) and providing a slip
signal (see Sec. 2). Both components are connected
in the force slip detector yielding a structurally
complete specification of the force-based slip detec-
tor. Note, the attached task context expressing the
suitability of the force slip detector for scenar-
ios when the base is moving. Due to space rea-
sons the specification of the data types (e.g. wrench,
force slip and task context) is not shown.
of the object within the hand should be taken into account.
This is achieved by feeding the signal produced by the
slip detector back into the grasp controller. In a pick
and place task the feedback signal should be represented
numerically so that the grasp controller can adapt the
grasp accordingly. However, for a hand-over task, it is
already sufficient to provide the grasp controller with a
symbolic signal, where upwards slip indicates that the
robot should release the object, whereas downwards slip
means that the robot should grasp the object more tightly.
Further task-specific controllers are, for example, required
in tactile exploration as proposed by Sommer et al. (2014),
measuring object’s in-hand motion (see Alcazar et al.
(2012)) or reactively placing a grasped object as shown
by Romano et al. (2011).
In order to select an appropriate grasp controller, the robot
requires knowledge about the individual grasp controllers
which we also represent in RPSL. This is possible because
both, slip detectors and grasp controllers, share structural
properties which are general enough to be modeled with
RSPL. In addition the selector requires knowledge of the
task which the robot is about to perform. This task context
is represented symbolically, e.g. as hand over, pick or
place. In contrast to the slip detector selection, the task
context does not have to be derived from the robot’s
motions. Instead, for instance, a knowledge base of the
robot can be queried for information about the current
task or a task planner, situated above the manipulation
pipeline in the robot control architecture, can provided
this information directly.
270
4. RELATED WORK AND DISCUSSION
4.1 Use cases
The ability to detect in-hand slippage can be helpful in
different use cases. One such use case is that of a grasp
controller (see Sanchez et al. (2015) and Romano et al.
(2011)), where the grasp force exerted on an object can
be computed proportionally to a continuous slip signal.
A continuous slip signal is also required for tasks such
as measuring the in-hand motion of a grasped object
(see Alcazar et al. (2012)). On the contrary, a hand-over
task might only require a discrete slip signal, for instance,
to detect when a person is pulling the grasped object from
the robotic gripper. Similarly, a robot placing an object
can benefit from a discrete event signaling a slippage, i.e.
the object is in contact with a support surface and is thus
slipping from the grasp.
4.2 Limitations
As part of a critical discussion we also would like to outline
the main limitations of our approach:
• Up to now, we have not evaluated in how far the run-
time adaptation of the grasp controller and slip de-
tector influences the stability of the core control loop
stability. Should this turn out to be a critical issue, it
seems worthwhile to identify further knowledge and
annotate the model repository of slip detectors and
grasp controllers accordingly.
• Additionally, the task context is only represented
symbolically. Here, a more in-depth analysis of the
relationship between the task context and the grasp
controller selection could reveal further information
that allows to improve the selection.
• Even though, the current chosen rule-based approach
is simplistic in structure, we do not depend on it.
In fact, the adaptation mechanism could be replaced
with other adaptation methods (e.g. constraint satis-
faction etc.) as the knowledge to represent the context
and the slip detectors would remain the same.
4.3 Related Work
Although tactile sensing in robotics has been researched
for over two decades (see Howe (1993)), it has recently
been applied to very diverse manipulation tasks. For
instance, Hsiao et al. (2010) use tactile information to
execute corrective actions on a PR2 that improved the
robot’s ability to grasp an object when greater positional
errors are present. Although usage of tactile information
improves the sensing capabilities of a robot, it is the
addition of different sensing modalities that has proven
to be a go-to solution to increase a robot’s robustness
in an uncertain and dynamic environment (see Luo et al.
(1988)). Prats et al. (2009) present an example of multi-
sensor fusion, where they integrate tactile sensing with
vision and force signals to improve their robot’s ability
to physically interact with the environment (e.g. sliding
a door open), demonstrating the increase of robustness
when using the three modalities combined. Slip detection
has been another application where tactile sensing plays
a major role; Romano et al. (2011) increase the grip force
IFAC IAV 2016
Jose Sanchez et al. / IFAC-PapersOnLine 49-15 (2016) 266–271
June 29 - July 1, 2016. Messe Leipzig, Germany
of the robot when a slippage is detected. Furthermore,
Romano et al. propose a phase-based architecture to
address the task of grasping an object and placing it in
another location, where the phases are detected based on
information produced by tactile and force sensors.
5. CONCLUSIONS
This paper presented an approach for run-time selection of
a slip detector with the best performance for the current
task being executed by the robot. More specifically, we
implemented three slip detectors based on tactile, force/-
torque signals and the fusion of these signals. A prelimi-
nary evaluation provided insights on how the performance
of the slip detection approaches depends on the action that
the robot executes. Based on these insights, we proposed
a context-adaptive architecture that improves the robust-
ness of the slip detection and is also able to select an
appropriate grasp controller based on the required task.
Future work will be focused on quantitatively evaluating
the run-time adaptation of the grasp controller, as well as
developing controllers required for different tasks as men-
tioned in Section 4. Furthermore, we would like to evaluate
different robot actions (e.g. arm motions) to observe their
impact on the developed slip detectors.
ACKNOWLEDGEMENTS
The authors gratefully acknowledge the on-going support
of the Bonn-Aachen International Center for Information
Technology. Nico Hochgeschwender and Sven Schneider
received a PhD scholarship from the Graduate Institute
of the Bonn-Rhein-Sieg University which is gratefully
acknowledged.
REFERENCES
Alcazar, J., Barajas, L.G., et al. (2012). Estimating object
grasp sliding via pressure array sensing. In Robotics and
Automation (ICRA), 2012 IEEE International Confer-
ence on, 1740–1746. IEEE.
Asfour, T., Regenstein, K., Azad, P., Schröder, J., Bier-
baum, A., Vahrenkamp, N., and Dillmann, R. (2006).
Armar-iii: An integrated humanoid platform for sensory-
motor control. In Humanoid Robots, 2006 6th IEEE-
RAS International Conference on, 169–175. IEEE.
Bischoff, R., Kurth, J., Schreiber, G., Koeppe, R., Albu-
Schäffer, A., Beyer, A., Eiberger, O., Haddadin, S.,
Stemmer, A., Grunwald, G., et al. (2010). The kuka-
dlr lightweight robot arm-a new reference platform for
robotics research and manufacturing. In Robotics (ISR),
2010 41st international symposium on and 2010 6th
German conference on robotics (ROBOTIK), 1–8. VDE.
Blumenthal, S., Hochgeschwender, N., Prassler, E., Voos,
H., and Bruyninckx, H. (2015). An Approach for a Dis-
tributed World Model with QoS-based Perception Al-
gorithm Adaptation. In 28th IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS).
IEEE, Hamburg, Germany.
Gärdenfors, P. (2004). Conceptual spaces as a framework
for knowledge representation. Mind and Matter, 2, 9–27.
Gherardi, L. and Hochgeschwender, N. (2015). RRA:
Models and Tools for Robotics Run-time Adaptation. In
28th IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS). IEEE, Hamburg, Germany.
271
271
Hochgeschwender, N., Olivares-Mendez, M.A., Voos, H.,
and Kraetzschmar, G.K. (2015). Context-based se-
lection and execution of robot perception graphs. In
Emerging Technologies Factory Automation (ETFA),
2015 IEEE 20th Conference on. doi:10.1109/ETFA.
2015.7301631.
Hochgeschwender, N., Schneider, S., Voos, H., and Kraet-
zschmar, G.K. (2014). Declarative specification of robot
perception architectures. In D. Brugali, J.F. Broenink,
T. Kroeger, and B.A. MacDonald (eds.), Simulation,
Modeling, and Programming for Autonomous Robots,
volume 8810 of Lecture Notes in Computer Science, 291–
302. Springer International Publishing. doi:10.1007/
978-3-319-11900-7 25. URL http://dx.doi.org/10.
1007/978-3-319-11900-7_25.
Howe, R.D. (1993). Tactile sensing and control of robotic
manipulation. Advanced Robotics, 8(3), 245–261.
Hsiao, K., Chitta, S., Ciocarlie, M., and Jones, E.G.
(2010). Contact-reactive grasping of objects with partial
shape information. In Intelligent Robots and Systems
(IROS), 2010 IEEE/RSJ International Conference on,
1228–1235. IEEE.
Luo, R.C., Lin, M.H., and Scherp, R.S. (1988). Dynamic
multi-sensor data fusion system for intelligent robots.
Robotics and Automation, IEEE Journal of, 4(4), 386–
396.
Melchiorri, C. (2000). Slip detection and control using
tactile and force sensors. Mechatronics, IEEE/ASME
Transactions on, 5(3), 235–243.
Prats, M., Sanz, P.J., and Del Pobil, A.P. (2009). Vision-
tactile-force integration and robot physical interaction.
In Robotics and Automation, 2009. ICRA’09. IEEE
International Conference on, 3975–3980. IEEE.
Reiser, U., Connette, C.P., Fischer, J., Kubacki, J.,
Bubeck, A., Weisshardt, F., Jacobs, T., Parlitz, C.,
Hägele, M., and Verl, A. (2009). Care-o-bot
R 3-creating
a product vision for service robot applications by in-
tegrating design and technology. In IROS, volume 9,
1992–1998.
Romano, J.M., Hsiao, K., Niemeyer, G., Chitta, S., and
Kuchenbecker, K.J. (2011). Human-inspired robotic
grasp control with tactile sensing. Robotics, IEEE
Transactions on, 27(6), 1067–1079.
Sanchez, J., Schneider, S., and Plöger, P. (2015). Safely
grasping with complex dexterous hands by tactile feed-
back. In RoboCup 2014: Robot World Cup XVIII, 332–
344. Springer.
Sommer, N., Li, M., and Billard, A. (2014). Bimanual
compliant tactile exploration for grasping unknown ob-
jects. In Robotics and Automation (ICRA), 2014 IEEE
International Conference on, 6400–6407. IEEE.
Weiss Robotics (2015). Tactile sensor module,
type: Dsa 9205. http://www.weiss-robotics.
de/en/tactile-sensing/tactile-sensors/
tactile-sensor-dsa-9205.html. [Online; accessed
13-November-2015].