Professional Documents
Culture Documents
Mechatronics
journal homepage: www.elsevier.com/locate/mechatronics
a r t i c l e i n f o a b s t r a c t
Keywords: The greatest limitation for an amputee subject who uses a prosthesis having no sensory feedback is the difficulty
Slippage detection to manage unexpected events in an autonomous way. In grasp and manipulation tasks, the possibility of object
Touch detection slippage is high. For this reason, it is necessary to detect the beginning of the slippage and provide the control
Force sensing resistor sensor
with a fast contrast action. In this work, a touch-and-slippage detection algorithm for effective grasp control of
Normal force component
a prosthetic hand embedding monoaxial, low-cost sensors is proposed. The algorithm has a low computational
Prosthetics
burden, and is capable of detecting slippage events using only the normal force component. Three experimental
sessions were performed to validate the touch-and-slippage algorithm. In the first one, the variation of the normal
force component provided by a Force Sensing Resistor (FSR) sensor was compared with the tangential force
component measured by a triaxial force sensor. In the second one, the threshold for detecting slippage events
was obtained and applied to study the dependence of the number of slippage events on sliding velocity and force
applied on the sensor. Finally, the algorithm was tested on a real system made of the IH2 Azzurra robotic hand
provided with FSR on the fingertips. The obtained results are illustrated in terms of false positives and negatives
and true positives and negatives. The low computation time of the algorithm, i.e. 4.9 ms, and the high success rate
in terms of detected slippage events (99.4%) suggest that the algorithm applicability in real systems is feasible.
1. Introduction the wrist of a robotic gripper. In [2], Coulomb friction model has been
applied on the output of three different sensors, i.e. a 10N single-axis
Over the years growing interest has been addressed to grasping and tension load cell sensor, a load cell and an optical encoder. In [3], the
manipulation actions performed by robotic and prosthetic hands. One of torque necessary to contrast the slippage event has been computed from
the main aspects to be taken into account in the evaluation of grasping the ratio between the tangential force and friction coefficient. In order
and manipulation is the stability of the grasped object which could be to identify the slippage onset and to filter the high frequency noise, the
guaranteed by estimating and preventing object slippage. residual of a Kalman filter has been added to the obtained torque. In [4],
Different approaches have been proposed in the literature to obtain a 6 axis Force/Torque sensor has been mounted on a parallel gripper and
information about object slippage during grasping and manipulation ac- a cascade of machine learning algorithms has been adopted to control
tions. which parts of the grasped objects are in contact with the instrumented
It is possible to distinguish among approaches that use sensors pro- surface of the gripper. The object slippage has been provided as a binary
viding the three force components, solutions utilizing the information value by using a Support Vector Machine algorithm. In [5], the slippage
about vibration, and methods relying only on the normal force com- is detected when the normal force value is lower than a threshold that
ponent. All these approaches have been proposed for commercial and depends on the tangential force, the torque and the translational and
ad-hoc developed sensors. rotational friction coefficients. A silicon-based tactile sensor has been
To detect both the tangential and the normal force components, proposed in [6] to measure normal force and tangential force simulta-
complex sensors such as load cells or 6-axis force/torque sensors have neously. A pre-sliding is detected when the tangential force, measured
been largely adopted. In these cases, the methods for slippage detection by the sensor positioned on the hand fingertips, is smaller than the sur-
mainly rely on Coulomb friction model, Kalman filters, machine learn- face shear traction. Contrariwise, a gross slip is identified. In [7], a Fast
ing, Fast Fourier Transform and threshold models. In [1], the Coulomb Fourier Transform (FFT) has been used to examine the slip and nonslip
friction model has been applied to tangential and normal forces obtained events from six strain gauges located on the thumb of the Motion Control
from a 6-Degree of Freedoms (DoFs) force/torque sensor positioned on
∗
Corresponding author.
E-mail addresses: c.gentile@unicampus.it (C. Gentile), f.cordella@unicampus.it (F. Cordella), cesar@ieee.org (C.R. Rodrigues), l.zollo@unicampus.it (L. Zollo).
https://doi.org/10.1016/j.mechatronics.2020.102402
Received 12 September 2019; Received in revised form 24 June 2020; Accepted 3 July 2020
0957-4158/© 2020 Elsevier Ltd. All rights reserved.
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
Hand. It has been verified that, during object slippage, the vibrations in tive slippages and false positive slippages, iii) the adoption of a current
the range 25 - 60 Hz are amplified. loop both for conditioning the signal originating from resistive tactile
The main drawbacks of the sensors providing the three force compo- sensors in order to cancel noise and as a proof of concept for the future
nents are their expensiveness, complexity (especially the electronics for implementation of an integrated circuit [23], iv) the introduction of a
signal acquisition and processing) and susceptibility to noise [8]. Fur- systematic way for validating the proposed slippage detection, general
thermore, the approaches adopted with these sensors for detecting slip- enough to be used for evaluating other approaches. The steps performed
page events are characterized by a high computational cost (e.g. Fourier to develop a robust slip detection method will be described in detail to
or wavelet transform have to be applied to big sample windows of the provide general methods for quantitatively evaluating system perfor-
force signals [9]). mance and establishing performance comparisons.
Other solutions use the information about vibrations induced by slip- The paper is structured as follows: in Section 2 the conditioning cir-
page itself for detecting it. A characterization of vibrations caused by cuit, the Receiver Operating Characteristic (ROC) curve for determining
slippage events has been performed in [10], where it has been found the threshold, the touch-and-slippage detection algorithm and the ex-
that vibrations are stochastic, non-Gaussian white noise processes and perimental setup and protocol are presented. Experimental results are
that their magnitude depends on the relative velocity of sliding. Start- illustrated and discussed in Section 3 and Section 4, respectively. Dif-
ing from these findings, in [11] an algorithm based on the covariance ferences with respect to other similar works proposed in the literature
among the vibration information from the tactile sensors placed on a are outlined in Section 5. Finally, conclusion and future work are given
gripper has been developed. Slippage detection has been associated to in Section 6.
a threshold, i.e. when the covariance is smaller than a threshold, slip-
page is not present, contrariwise, object slippage happens. In [12], the 2. Materials and methods
pressure response from a biomimetic tactile sensor array [13] has been
analyzed with a Short Time Fourier Transform (STFT) with a time win- In this section, an algorithm able to detect the touch and the slip-
dow of 80 ms, to mimic biological grip reflex response times [14]. The page from information about the normal force component measured by
range of frequencies between 30 Hz and 200 Hz has been considered a FSR and a conditioning circuit are described. The proposed approach
since it has been verified that slippage events are located in this range. and conditioning circuit are applied in the following to the output of an
In [15], the ThimbleSense has been adopted to detect slippage from FSR, but they can be used with several kinds of sensors providing infor-
vibrations by using discrete Wavelet Transform in static conditions. In mation about the normal force component, such as resistive, inductive
[16], a method that uses a biomimetic fingertip that integrates piezore- and capacitive sensors.
sistive MEMS sensors to detect, from the vibrations measured from the
normal force component, the ON/OFF signal related to the presence or
absence of slippage has been developed. 2.1. Conditioning circuit
Other methods use low cost commercial sensors, such as the FSR. In
[17] two methods for detecting slippage have been proposed and tested The Anderson loop [24] was chosen for conditioning signals from
on a tactile sensor array attached to gripper fingers under silicone rub- the monoaxial tactile sensor. Similarly to the conventional Wheatstone
ber. The first method uses the rank correlation for detecting both trans- bridge, the loop subtracts sensor voltage drops from those measured on
lational and rotational slippage, the second one adopts the 2-D cross a reference resistor. However, while subtraction is performed actively
correlation to detect slippage velocity. In [18], the forces acquired from in the Anderson loop, it is a passive process in the bridge. Although
a FSR sensor are used to build a dynamical model of initial slip on the the resulting response is less linear with respect to the conditioning cir-
basis of the theory of friction-induced vibration at near-zero slip veloc- cuit suggested by the sensor manufacturer, the Anderson loop provided
ity [19,20]. The high-frequency component of the grasping force signals a substantial reduction of vibration noise, which was the main source
is extracted by wavelet transform and the change of wavelet coefficients of errors observed in the previous experiments. Noise reduction is ob-
is used to detect the onset of slip. Fuzzy logic controllers have been also tained with an FSR identical to all others, lodged in the palm, as the
applied to the outputs of a FSR [21] for discriminating among “slow reference resistor Rref . Since it is affected by the same background vibra-
slip” (i.e., the controller detects a slow object slipping), “fast slip” (i.e., tions present at fingertips, this noise is simultaneously subtracted from
the controller detects a rapid change in slip values), and “null” (i.e., the the tactile signals acquired from sensors (RS1 -RS5 ). On the other hand,
object is either grasped or it has slipped and fallen beyond controller as the touch-and-slippage detection algorithm takes the linearized force
action). as input, a resulting non-linear static response at the conditioner output
An approach characterized by a lower computational cost with re- has little impact on classification performance.
spect to the previous ones is proposed in [22]. The method is based on The current reference circuit, shown in the blue rectangle of Fig. 1,
computing the absolute value of the voltage signal derivative, obtained consists of a reference voltage (OP1 ) that sets the input for a second
from a FSR, and comparing it to an empirically determined threshold, operational amplifier (OP2 ). OP2 establishes 𝐼𝑟𝑒𝑓 = 𝑉𝑟𝑒𝑓 ∕𝑅𝑠𝑒𝑡 throughout
to generate a binary output slippage/steady holding. its feedback loop (pink background area) containing six resistances (Rref ,
The main drawbacks of the above methods are i) uncertainty about and RS1 -RS5 ). Each resistance is an FSR linearized with a parallel 10 kΩ.
the slippage output when the absolute value of the signal derivative Two layers of active subtractors, depicted in the green area, were
is evaluated (i.e. a force increment could be confused with a slippage implemented with instrumentation amplifiers (AD623). The first layer
event), ii) the presence of additional noise due to the computation of (IA0 to IA5 ) amplifies total voltage drops across each of the FSR sensors
the signal derivative, which could imply the impossibility of recognizing (RS1 -RS5 ), including the reference one (Rref ). The second layer produces
slippage with respect to noise, iii) the confusion between initial touch the output signals (Vo1 to Vo5 ), which are proportional solely to the
and slippage. resistance variation ΔRSi on each sensor (𝑉𝑜𝑖 = 𝐼𝑟𝑒𝑓 .Δ𝑅𝑆𝑖 , 𝑖 = 1, … , 5).
This paper aims at overcoming the limitations highlighted in the lit-
erature by proposing and validating a fast and efficient algorithm, able 2.2. Threshold detection for the algorithm output
to detect the touch and the onset of the slippage event by using only the
normal component of the acquired signal (i.e. force or voltage). The ap- The ROC curve analysis method originated from the theory of signal
proach is grounded on the detection of the vibrations due to the sliding detectability [25] has been used in many fields, such as medical diag-
between two contact surfaces, as in [22]. The main improvements with nosis or algorithm validation [26–28].
respect to the author’s previous work [22] are: i) the introduction of Let us first recall some definitions: true positive rate (TPR), or sensi-
the touch identification stage, ii) the discrimination between true posi- tivity, is the probability that slippages are correctly classified based on
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
the threshold; true negative rate (TNR), also called specificity, is the prob-
ability that no-slippages are correctly classified; false positive rate (FPR)
is the probability that no-slippages are falsely classified; false negative
rate (FNR) is the probability that slippages are falsely classified.
The hit rate of a classification process is highly dependent on the
function chosen to separate results into each class. In this work, where
simple thresholding is adopted, the ROC curve analysis was used for de-
termining the threshold in two confused states yielding maximum sensi-
tivity. In particular, for the slippage and no-slippage state classification,
a threshold (or cut-off point) was determined to discriminate the algo-
rithm output.
In the ROC curve coordinate system, the x and y of the cut-off point
are represented by FPR and TPR, respectively. The area under the ROC Fig. 2. Workflow of the touch-and-slippage detection algorithm.
curve (AUC) can be used as an index to evaluate detection capacity of a
specific feature. The detection accuracy increases with the AUC increase.
by avoiding an increase of the computation time) acquired with a fre-
If AUC is greater than 0.9, the test is highly accurate [27]. Then, the cut-
quency of 2 kHz
off point will be in the upper left corner of the ROC curve [26].
1∑
5
𝑣𝑚𝑒𝑎𝑛 = 𝑣 (1)
2.3. Touch-and-slippage detection algorithm 5 𝑖=1 𝑖
Considerable researches have been performed to understand sliding The 1st stage of the algorithm is devoted to the identification of the
process. The stick-slip model welldescribes the behavior of the sliding first touch between the sensor and the object surface. The steps of the
between two contact surfaces: a stick phase, where the system accumu- touch identification procedure are the following:
lates potential energy, is alternated with a slip phase, where the energy 1) Computation of the average value on 10 samples (acquired with
is transformed into movement, causing vibrations [29–32]. a frequency of 2 kHz) of the voltage signal at resting (calibration) pe-
This model is also used to describe the slip coding in epicritic tactile riod. In the calibration period, no force variation is detected, and vrest
perception (relating to mediating cutaneous sensory reception) because represents the mean of background noise magnitude.
the slip appears discontinuous in space and in time [33]. The vibrations 2) Comparison of vrest with vmean , to obtain a mean voltage error
due to the sliding between two contact surfaces [10] represent the key- Δ𝑣 = 𝑣𝑟𝑒𝑠𝑡 − 𝑣𝑚𝑒𝑎𝑛 (2)
point of the proposed algorithm.
The algorithm is able to detect i) the first touch between the sensor The touch with the object is detected if Δv is greater than the mini-
and an object and ii) the slippage events. The functioning principle is mum voltage variation (𝛿) measured by the sensor when pressed
shown in the workflow in Fig. 2.
Δ𝑣 > 𝛿, 𝑡𝑜𝑢𝑐ℎ = 1; (3)
With the aim of removing the electronic noise from setup compo-
nents, mean value vmean of the FSR conditioned signal (i.e. force or volt- In the algorithm 2nd stage, the value 𝑣′𝑚𝑒𝑎𝑛 , i.e. the voltage signal pos-
age), vi , has been computed for every 5 samples (this value has been ex- itive derivative, is computed. Only the positive derivative is considered
perimentally obtained and seemed to be adequate to decrease the noise, since the negative value corresponds to a pressure increment on the FSR,
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
as shown in the first and the second subplots in Fig. 3 (to obtain the force
value corresponding to the measured voltage, a static characterization
of the sensor was performed, as described in Section 2.4.1)
𝑑 𝑣 (𝑖) − 𝑣𝑚𝑒𝑎𝑛 (𝑖 − 1)
𝑣′𝑚𝑒𝑎𝑛 = 𝑣 = 𝑚𝑒𝑎𝑛 (4)
𝑑𝑡 𝑚𝑒𝑎𝑛 𝑐 𝑦𝑐 𝑙𝑒 𝑡𝑖𝑚𝑒
Fig. 5. a) Initial position, b) Final position for the Kuka-probe. The reference Fig. 6. The IH2 robotic hand grasping the cube equipped with the M-IMU from
system is for the JR3 sensor. the moving-object system.
Table 1
Parameters used in the first experimental session:
forces and time needed by the robot arm to pass coupled and activated simultaneously by the same motor) and Abduc-
over the FSR sensor surface. tion/Adduction (A/A) of the thumb. The fingers were ad-hoc developed
A B C
with a flat surface in order to minimize the problems due to the non
perfect contact between the fingers and the object but the FSR can also
Force [N] 1.2 3.6 4.7 work on a curved surface, as demonstrated in the experimental study in
Time [s] 5 3 1
[41].
The robotic hand was controlled to grasp a cubic object (33 x 33
x 33 mm, weight = 15.32 g) equipped with a magnetoinertial sensor
where 𝑛 + 1 is the order of the polynomial, n (1 ≤ n ≤ 9) is the degree (M-IMU, XSENS MTw) to measure the acceleration components caused
of the polynomial. The order indicates the number of coefficients to be by slippage. The optoelectronic system BTS SMART-D Motion Capture
fit, and the degree provides the highest power of the predictor variable. System (www.btsbioengineering.com/it/) was used to track markers po-
To obtain the vector FJR3 of the generalized forces (i.e. force and sitioned both on the robotic hand index and on the cube in order to
torque) from the voltage values acquired by the JR3 sensor, the JR3 measure the object displacement due to slippage. Displacement was cal-
calibration matrix M, provided by the producers, was multiplied by the culated as the euclidean distance between the marker positioned on the
sensor voltage vector VJR3 cube and the marker positioned on the robotic hand index finger. The
𝐹𝐽 𝑅3 = 𝑀 ∗ 𝑉𝐽 𝑅3 . (8) BTS SMART-D Motion Capture System is a 7-camera motion analysis
with an acquisition rate of 60 Hz and an accuracy less than 0.1 mm
The proposed touch-and-slippage detection algorithm, implemented over a 2 x 2 m area. The information provided by the M-IMU and by the
in C programming language under Windows 10, was applied to the nor- optoelectronic system were used to verify the algorithm performance in
mal force acquired by the FSR sensor and was tested within its force terms of false positives and negatives and true positives and negatives.
range. To ensure repeatability of the experimental conditions, a moving-
The robotic arm is characterized by 7 active DoFs and is provided object system was developed (Fig. 6). The system is a manipulator with
with joint position and torque sensors. It was controlled with a parallel 4 passive DoFs and a gripper like end-effector. In this way, the hand can
force/position control to pass over the entire surface of the FSR sensor hold the grasped object always in the same position; then, the gripper
by applying a constant force and simulating a slippage. releases the object and changes its orientation without interfering with
𝑣′𝑚𝑒𝑎𝑛 was compared with the tangential force component Fy (Fig. 5) the task. After each activity, the moving-object system comes back into
acquired by the JR3 sensor during the probe sliding. The aim is to verify the initial position.
if the method is able to detect a normal force component variation, due The KUKA robotic arm provided with the probe was used to apply
to vibrations caused by the slippage event, when a tangential force (used an impulsive force to the grasped object and induce slippage (Fig. 7).
as a ground truth) is present. Data acquired by the M-IMU and by the optoelectronic system represent
To verify the correctness of the method in different conditions of our ground truth.
forces and sliding velocities, three force values exerted by the robot arm To determine the threshold 𝛼 (Eq. 5), different values of 𝛼 with the
and 3 durations of the probe sliding were chosen, as shown in Table 1 ROC curve were tested and the corresponding parameters described in
(i.e. Force A Time A, Force B Time C, etc.). Section 2.2 (i.e. TPR, TNR, FPR, FNR) were evaluated. The threshold
The robot arm was controlled to move the probe in contact with the was determined on data acquired by the FSR sensor attached on the in-
FSR sensor surface (Fig. 5 a) and apply one of the force values shown in dex fingertip of the prosthetic hand during the pinch grasp of the cube.
Table 1. When the cube was firmly grasped, 30 slippage events were induced
by the probe attached to the robotic arm end-effector. The ROC curve
2.4.2. Second experimental session was constructed on the acquired data and the value that yields to the
Two FSR sensors were placed on the thumb and index fingertips largest AUC was chosen as the threshold 𝛼. Furthermore, the obtained
of the IH2 Azzurra biomechatronic hand (Prensilia srl) [40] and cov- threshold value was applied to the data acquired in the first experimen-
ered with a glove. The hand has 5 fingers and 5 active DoFs, i.e. tal session to analyze the dependency of the output of the algorithm
Flexion/Extension (F/E) of the five fingers (ring and little fingers are 3rd stage (i.e. slip) on applied force and sliding velocity. The obtained
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
Table 2
Objects grasped in the third experimental session.
Fig. 8. From left to right: big parallelepiped, big cylinder and tennis ball, that
were grasped with a power grasp; small parallelepiped, small cylinder and small Fig. 10. Force B Time B combination for the first experimental session.
ball, that were grasped with a pinch grasp.
Fig. 11. Force C Time C combination for the first experimental session.
Table 3
Average and standard deviation of the number of the slips for the
tasks performed in the first experimental session and calculated in
the second experimental session, as a function of the force and time.
Force\Time 5s 3s 1s
Fig. 17. Validation in real scenario. A. Pinch grasp of SB. B. Power grasp of TB.
From data analysis, it emerged that there is a trend that binds the
number of slips with the change of the force applied on the sensor and Table 4
the sliding velocity of the probe. In particular, it is possible to notice that Sensitivity, specificity and accuracy parameters for the 40 task performed with
the number of slips decreases when speed increases, keeping the force the robotic hand for the pinch grasp.
constant, whereas it increases when force increases, at a constant veloc-
Pinch grasp
ity. Indeed, when velocity increases there is a loss of contact between the
SC SP SB
surfaces. Hence, vibrations decrease because the friction coefficient de-
creases. Contrary, when the force increases, the friction coefficient and 40 total tasks 40 total tasks 40 total tasks
the vibrations increase [41]. Since the proposed approach grounds on 39 tasks 1 tasks 38 tasks 2 tasks 40 tasks 0 tasks
variations, a decrease in the number of vibrations implies a decrease in
Sensitivity 100% 80% 100% 80% 100% -
the number of slippage events. In Table 3, the average and the standard Specificity 100% 100% 100% 100% 100% -
deviation of the number of slippages for each force-time combination Accuracy 100% 99.53% 100% 99.53% 100% -
are reported.
The algorithms computation time, i.e. the time from the acquisition
of the first voltage sample to the binary output (i.e. 1 slip, 0 no slip), is Table 5
4.9 ms. This time was obtained using an Intel Core i5-4570 @ 3.20 GHz Sensitivity, specificity and accuracy parameters for the 40 task performed with
processor with 4 GB of RAM. By using a dedicated device a lower com- the robotic hand for the power grasp.
putation time could be obtained.
Power grasp
BC BP TB
3.3. Third experimental session
40 total tasks 40 total tasks 40 total tasks
To test the approach on a real scenario, the IH2 robotic hand was 37 tasks 3 tasks 39 tasks 1 tasks 38 tasks 2 tasks
used to grasp different objects (robotic hand index and each object were
Sensitivity 100% 80% 100% 80% 100% 80%
equipped with markers) from the moving-object system and the slippage Specificity 100% 100% 100% 100% 100% 100%
events were simulated by applying impulsive forces to the object surface Accuracy 100% 99.53% 100% 99.53% 100% 99.53%
by means of the robotic arm.
For the sake of brevity, in Fig. 17 the output of 5 slippages induced
by the robotic arm are shown for a single task both for pinch and power
grasps. Subplot a) in A. and B. depicts the forces applied by the fingers. a variation of the A/A angle of the thumb joint are evident when slippage
Subplots b) and c) show the touch and the slippage events detected occurs (in the red box of the subplots d) of Fig. 17 an enlargement of
by each finger. In the subplot e), the displacements due to the slippages this behavior is shown).
are presented. Finally, to verify the correctness of the use of the touch-and-slippage
It is possible to observe that the sensor on the thumb detected one detection algorithm with the force-and-slippage control, the sensitivity,
slippage event in A. and two in B., whereas the one on the index detected specificity and accuracy (the capacity of the algorithm to detect the slip;
five slippage events for both cases. This information is enough for the it is obtained by the ratio between the correct classified samples and the
force-and-slippage control strategy adopted for controlling the robotic total samples) parameters for the 40 tasks for each of 3 objects for the
hand since it requires that only one of the fingers detect the slippage, 2 grasp types, were computed and are listed in Tables 4 and 5. On the
in order to compensate it. In subplots d), the finger joint angles are total 240 tasks, the true positive rate is 96,25%, the true negative rate is
shown. An increasing of the F/E angles of the involved fingers joints and 100%, the false positive rate is 0% and the false negative rate is 3,75%.
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
Fig. 19. a) Force signal, b) the positive derivative of the voltage and c) ob-
Fig. 18. a) Force signal, b) the derivative of the signal and c) slippage behaviour
ject displacement when 50 impulsive external disturbances were induced on
when the maximum force (i.e. 20 N) measured by the sensor is applied.
the cube.
5. Comparison with other approaches made it possible to define general methods for quantitatively evaluating
system performance.
With respect to methods where the threshold for detecting slippage It is possible to test the algorithm with other control laws in different
is empirically derived [12,42] or is identified with a demanding off-line scenarios, such as assistance to disabled and elderly people [44,45] or
training [43], the proposed method has the advantage to have a short to return slippage sensations to amputees in order to finely control a
training time. In fact, a small number of acquisitions, i.e. less than 30 prosthetic hand during manipulation tasks [46].
events, need to be available to determine the optimal threshold for a Future work will be devoted to test the algorithm on a prosthetic
wide range of measures (in Section 4, it has been demonstrated that the hand during its active control by a patient through peripheral neural
obtained threshold is valid in the whole functioning range of the FSR signals [47] also using other type of sensors. Furthermore, a miniatur-
sensor). Information about the initial touch is not differentiated from ized version of the electronic circuit will be developed by using TSSOP
slippage in no one of the reviewed paper. For instance, in [16], touch components. It will have dimensions of a few centimeters (about 30 x
is considered as false positive, but it is not distinguished from the other 30 mm for this circuit) and could be integrated into the socket of a hand
false positives. prosthesis (wrist diameter ~ 50 mm).
Although some literature approaches demonstrated to obtain good
results, no one of them adopts a rigorous performance evaluation of the Declaration of Competing Interest
proposed method. Although papers [16] and [17] evaluate true and false
positives (in [16] true positives were detected in 100% of the cases and The authors whose names are listed immediately below certify that
false positives in no more than 1.01%; in [17] 100% of true positive rate they have NO affiliations with or involvement in any organization or en-
and 93% of true negative rate were obtained) they do not adopt a rig- tity with any financial interest (such as honoraria; educational grants;
orous method to evaluate algorithm performance. In fact, they do not participation in speakers bureaus; membership, employment, consultan-
adopt a ground truth to recognize effective slippage events, therefore cies, stock ownership, or other equity interest; and expert testimony or
the reader cannot verify the correctness of the obtained results. In this patent-licensing arrangements), or non-financial interest (such as per-
work, not only the performance is evaluated in terms of false positives sonal or professional relationships, affiliations, knowledge or beliefs) in
and negatives and true positives and negatives outperforming the values the subject matter or materials discussed in this manuscript.
obtained in the literature (the true positive rate is 96.25%, the true neg-
ative rate is 100%, the false positive rate is 0% and the false negative CRediT authorship contribution statement
rate is 3.75%), but the correctness of the results is guaranteed by the
adoption of a ground truth obtained with an optoelectronic system. Cosimo Gentile: Formal analysis, Funding acquisition, Writing -
One significant contribution of this paper is to provide a systematic original draft, Writing - review & editing. Francesca Cordella: For-
way for validating the proposed slippage detection approach, by deter- mal analysis, Writing - original draft, Writing - review & editing.
mining the threshold with the ROC curve, by adopting a probability- Cesar Ramos Rodrigues: Supervision, Writing - original draft, Writ-
based approach to determine false positives and negatives so as true ing - review & editing. Loredana Zollo: Supervision, Writing - review
positives and negatives, and by introducing different ground truths to & editing.
verify the correctness of each part of the method.
Acknowledgment
6. Conclusions and future work
This work was supported partly by the Italian Institute for Labour
In this paper, a touch-and-slippage detection algorithm for Accidents (INAIL) with RehabRobo@work (CUP: C82F17000040001)
robotic/prosthetic hands has been presented and experimentally vali- and PPR AS 1/3 (CUP: E57B16000160005) projects, partly by the ANIA
dated. The proposed approach relies only on the normal force compo- Foundation with Development of bionic upper limb prosthesis charac-
nent (or on the corresponding voltage raw signal) to detect the slippage terized by personalized interface and sensorial feedback for amputee
events. The touch with the object surface has been also computed and patients with macro lesion after car accident project, partly by the Cam-
adopted as a trigger to start the slippage detection. pus Bio-Medico University Strategic Projects (Call 2018) with the SAFE-
It has been verified that the 2nd stage of the developed algorithm, MOVER project, and partly by the Italian Ministry of University and
based on the signal variation, produces an output only in presence of Research (PON project research and innovation 2014–2020) with the
a tangential force component (measured by a JR3 Multi - Axis Force - ARONA project (CUP: B26G18000390005).
Torque sensor).
A ROC curve has been adopted for the detection of the threshold in References
order to generate a value equal to 1 only if the output of the 2nd stage
(𝑣′𝑚𝑒𝑎𝑛 ) is higher than the threshold 𝛼. [1] Francisco E Viña B, Bekiroglu Y, Smith C, Karayiannidis Y, Kragic D. Predicting slip-
From data analysis, it is possible to conclude that the number of page and learning manipulation affordances through Gaussian Process regression.
IEEE-RAS International Conference on Humanoid Robots 2013.
slips is dependent on the change of the force applied on the sensor and [2] Dzitac P, Mazid AM, Ibrahim MY, Appuhamillage GK, Choudhury TA. Optimal sens-
on the sliding velocity of the probe. Further, the developed algorithm ing requirement for slippage prevention in robotic grasping. IEEE International Con-
works independently by the force applied on the sensor and the sliding ference on Industrial Technology 2015.
[3] De Maria G, Falco P, Natale C, Pirozzi S. Integrated force/tactile sensing: the en-
velocity of the probe. abling technology for slipping detection and avoidance. IEEE International Confer-
The results obtained in a real scenario, the low computation time of ence on Robotics and Automation 2015.
the algorithm, i.e. 4.9 ms, the high success rate in terms of both detected [4] Moreira E, Rocha LF, Pinto AM, Moreira AP, Veiga G. Assessment of robotic picking
operations using a 6 axis force/torque sensor. IEEE Rob Autom Lett 2016;1(2).
slippage events (99.4%) and compensated slippage events (100%) make
[5] Teshigawara S, et al. Highly sensitive sensor for detection of initial slip and its appli-
the approach feasible for online application on a prosthetic hand con- cation in a multi-fingered robot hand. IEEE Conference on Robotics and Automation
trolled by an amputee. 2011.
[6] Ma Y, Huang Y, Mao L, Liu P, Liu C, Ge Y. Pre-sliding detection in robot hand
The results obtained in papers presenting alternative algorithms,
grasping based on slip-tactile sensor. IEEE International Conference on Robotics and
mentioned in our literature survey, were compared with the ones ob- Biomimetics 2015.
tained with our method, demonstrating the high performance reached [7] Engeberg ED, Meek SG. Adaptive object slip prevention for prosthetic hands through
and the rigorous evaluation procedure adopted. proportional-derivative shear force feedback. IEEE/RSJ International Conference on
Intelligent Robots and Systems 2008.
Each step performed to develop the approach proposed in this paper [8] Francomano MT, Accoto D, Guglielmelli E. Artificial sense of slip - a review. IEEE
has been described in detail and validated in a rigorous manner. This Sens J 2013;13(7):2489–98.
C. Gentile, F. Cordella and C.R. Rodrigues et al. Mechatronics 70 (2020) 102402
[9] Engeberg ED, Meek SG. Adaptive sliding mode control for prosthetic hands to simul- [46] Zollo L, DiPino G, Ciancio AL, Ranieri F, Cordella F, Gentile C, et al. Restoring tactile
taneously prevent slip and minimize deformation of grasped objects. IEEE/ASME sensations via neural interfaces for real-time force-and-slippage closed-loop control
Trans Mechatron 2013;18(1):376–85. of bionic hands. Sci Rob 2019;4(27):eaau9924.
[10] Qiao S, Beloiu D, Ibrahim R. Deterministic and stochastic characterization [47] Noce E, et al. Grasp control of a prosthetic hand through peripheral neural signals.
of friction-induced vibration in disc brakes. Nonlinear Dyn 2004;36:361– Journal of Physics: Conference Series 2018;1026(1). IOP Publishing.
378.
[11] Stachowsky M, Hummel T, Moussa M, Abdullah HA. A slip detection and Cosimo Gentile obtained his B.Sc. in Computer Engineering
correction strategy for precision robot grasping. IEEE/ASME Trans Mechatron from Università degli studi del Sannio, Benevento, Italy and
2016;21:2214–26. M.Sc. in Automation Engineering from University of Naples
[12] Fishel JA, Santos VJ, Loeb GE. A robust micro-vibration sensor for biomimetic Federico II, Naples, Italy, in 2012 and 2015, respectively. He
fingertips. IEEE/RAS-EMBS International Conference on Biomedical Robotics and earned his Ph.D. in Bioengineering at Università Campus Bio-
Biomechatronics 2008. medico di Roma, Rome, Italy in 2020. His research interests
[13] Wettels N, Santos VJ, Johansson RS, Loeb GE. Biomimetic tactile sensor array. Adv centre around the human-robot interaction. His current re-
Rob 2008;22(7):829–49. search is focused on the design and the development of control
[14] Johansson RS, Westling G. Signals in tactile afferents from the fingers eliciting adap- schemes for prosthetic robotic hands.
tive motor responses during precision grip. Exp Brain Res 1987;66:141–54.
[15] Ajoudani A, Hocaoglu E, Altobelli A, Rossi M, Battaglia E, Tsagarakis N, Bicchi A.
Reflex control of the pisa/IIT softhand during object slippage. IEEE International
Conference on Robotics and Automation 2016.
[16] Romeo RA, Oddo CM, Carrozza MC, Guglielmelli E, Zollo L. Slippage detection with Francesca Cordella obtained the Laurea degree in Electronic
piezoresistive tactile sensors. Sensors 2017;17(8):1844. Engineering and the Ph.D. in Computer and Automation Engi-
[17] Cheng Y, Su C, Jia Y, Xi N. Data correlation approach for slippage detection in neering both at the University of Naples Federico II. She was
robotic manipulations using tactile sensor array. IEEE/RSJ International Conference visiting student at the Institut fur Robotik und Mechatronik,
on Intelligent Robots and Systems 2015. Deutsches Zentrum fur Luft-und Raumfahrt, where she investi-
[18] Zhang Y, Duan X-G, Zhong G, Deng H. Initial slip detection and its application in gated the human grasping behavior and developed techniques
biomimetic robotic hands. IEEE Sens J 2016;16(19):7073–80. for motion analysis. From 2013 to 2018 she has been Assistant
[19] Ko PL, Taponat MC, Pfaifer R. Friction-induced vibration - with and without external Researcher at the Campus Bio-Medico University of Rome.
disturbance. Tribol Int 2001;34(1):7–24. Since 2018 she is Assistant Professor with the Unit of Biomed-
[20] Owen WS, Croft EA. The reduction of stick-slip friction in hydraulic actuators. ical Robotics and Biomicrosystems of the same university. Her
IEEE/ASM Transaction on Mechatronics 2003;8(3):362–71. research interests are mainly in the field of biomechanics and
[21] Sriram G, Jensen AN, Chiu SC. Slippage control for a smart prosthetic hand proto- biomedical robotics, with special reference to the following
type via modified tactile sensory feedback. IEEE International Conference on Elec- topics: bio-inspired grasping strategies for robotic/prosthetic
tro/Information Technology 2014. systems, robot-aided rehabilitation, robot path planning, mul-
[22] Cordella F, Gentile C, Zollo L, Barone R, Sacchetti R, Davalli A, Siciliano B, timodal interfaces for assistive robotics, upper-limb prosthe-
Guglielmelli E. A force-and-slippage control strategy for a poliarticulated prosthetic ses, human-machine interaction, biomechanical analysis of
hand. IEEE International Conference on Robotics and Automation 2016. the human being, user state and intention detection, vision-
[23] Anderson KF. The new current loop: an instrumentation and measurement circuit based strategies for arm-hand pose and motion estimation.
topology. IEEE Trans Instrum Meas 1997;46:1061–7. doi:10.1109/19.676711. She is member of the Technical Program Committee of sev-
[24] Anderson KF. NASA’s anderson loop. IEEE Instrumentation & Measurement Maga- eral International Conferences and Workshops. She is member
zine 1998;1(1):5–15. 30 of the Editorial Board of the Journal of Biomedical Science
[25] Peterson W, Birdsall T, Fox W. Theory of signal detectability. Transactions of the & Applications. Since 2015 she is an expert and reviewer for
IRE Professional Group on Information Theory 1954. the European Commission within H2020- ICT. She has been
[26] Wan P, Wu C, Lin Y, Ma X. On-road experimental study on driving anger iden- involved in several EU-funded and national projects in her
tification model based on physiological features by ROC curve analysis. IET Intel fields of interest. She is project manager of the national Project
Transport Syst 2017. RehabRobo@Work. She has authored/coauthored more than
[27] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating 35 peer-reviewed publications appeared in international jour-
characteristic (ROC) curve. Radiology 1982. nals, books and conference proceedings.
[28] Swets JA. Measuring the accuracy of diagnostic systems. Science, New Series
1988;240(4857):1285–93. Cesar Ramos Rodrigues (M95) received the B.Sc. degree in
[29] Bowden FP, Tabor D. Friction and lubrification. Oxford University Press 1954. electric engineering from Federal University of Santa Maria
[30] Rabinowicz E. Stick and slip. Sci Am 1956;194(5):109–18. (UFSM), Santa Maria, Brazil. The M.Sc. and the Ph.D. degrees
[31] Rabinowicz E. The intrinsic variables effecting the stick-slip process. Proceedings of in electronics were received from State University of Campinas
the Physical Society 1958;71:668–75. (Unicamp), Brazil, in 1990, and 1993, respectively. He joined
[32] Liberto F, Balzano E, Serpico M, Peruggi F. Dinamica stick-slip: oscillazioni con at- UFSM as a Teaching and Research Assistant in 1997, where he
trito. La fisica nella Scuola 2006. started the Microelectronics group (Gmicro). In 2013, he was a
[33] Cornelius Schwarz. The slip hypothesis: tactile perception and its neuronal bases. visiting researcher with the Graduate School of Biomedical En-
Trends Neurosci 2016. gineering at UNSW (Sydney, Australia). In 2016 he was a vis-
[34] Butcher JC. Numerical methods for ordinary differential equations. J Wiley 2016. iting researcher with the Campus Bio-medico di Roma (Rome,
[35] Datasheet F. S. R.. Model 400. Interlink Electronics, Inc. Italy). After that, he started to focus on integrated circuits and
[36] Datasheet JR3. Multi - Axis Force- Torque Sensor. artificial intelligence techniques applied to biomedical engi-
[37] Albu-Schaffer A, Haddadin S, Ott C, Stemmer A, Wwimbock T, Hirzinger G. The DLR neering, and moved (2019) to University of Santa Catarina
lightweight robot: design and control concepts for robots in human environments. (UFSC), Florianpolis, Brazil. Presently he is a professor with
Industrial Robot: An International journal 2007;34(5):376–85. the Institute of Biomedical Engineering (IEB) at UFSC.
[38] Romeo RA, Cordella F, Zollo L, Formica D, Saccomandi P, Schena E, Carpino G,
Davalli A, Sacchetti R, Guglielmelli E. Development and preliminary testing of Loredana Zollo, M.S. 2000, Ph.D. 2004, is Full Professor of
an instrumented object for force analysis during grasping. International Confer- Biomedical Engineering at UCBM. In 2000 she joined ARTS
ence of the IEEE Engineering in Medicine and Biology Society 2015:6720–3. Lab (Advanced Robotics Technology and Systems Laboratory)
doi:10.1109/EMBC.2015.7319935. of the Scuola Superiore Sant Anna. In 2003 she was visiting
[39] Leone F, Gentile C, Ciancio AL, Gruppioni E, Davalli A, Sacchetti R, et al. Simul- student at the Laboratory of Neuroscience INSERM483, at the
taneous sEMG classification of wrist/hand gestures and forces. Front Neurorobot University Pierre et Marie Curie in Paris. In 2008 she was ap-
2019;13(42):13–42. pointed assistant professor at UCBM. Her research interests are
[40] Datasheet IH2 azzurra robotic hand, prensilia s.r.l. mainly in the fields of neuro-robotics and biomedical robotics.
[41] Adams MJ, Johnson SA, Lefèvre P, Lévesque V, Hayward V, André T, Thonnard J-L. In 20122015 she was member of the editorial board as Asso-
Finger pad friction and its role in grip and touch. Journal of The Royal Society ciate Editor of the IEEE Robotics and Automation Magazine.
Interface 2012. Presently, she is associate editor for the International Journal
[42] Romano J, Hsiao K, Niemeyer G, Chitta S, Kuchenbecker K. Human-inspired robotic of Advanced Robotic Systems, Journal of Healthcare Engineer-
grasp control with tactile sensing. IEEE Trans Rob 2011;27:10671079. ing, IEEE Transaction on Medical Robotics and Bionics. She
[43] Goeger D, Ecker N, Woern H. Tactile sensor and algorithm to detect slip in robot is member of the editorial board of the book series Springer
grasping processes. Proceedings of the 2008 IEEE International Conference on BIOSYSTEMS & BIOROBOTICS (BioSysRob) and was co-chair
Robotics and Biomimetics 2008. of the IEEE Robotics and Automations Technical Committee
[44] Badesa FJ, Morales R, Garcia-Aracil N, Sabater JM, Casals A, Zollo L. Auto-adaptive on Rehabilitation and Assistive Robotics. She is expert and
robot-aided therapy using machine learning techniques. Comput Methods Programs reviewer within the H2020 research program and has been
Biomed 2014. involved in many EU-funded and national projects in her ap-
[45] Zollo L, Laschi C, Teti G, Siciliano B, Dario P. Functional compliance in the control of plication fields. She has authored/coauthored more than 130
a personal robot. IEEE International Conference on Intelligent Robots and Systems peer-reviewed publications appeared in international jour-
2001. nals, books and conference proceedings and 5 patents.