EMG-based Simultaneous Estimations of

Joint Angle and Torque during Hand

Interactions with Environments

Journal: Transactions on Biomedical Engineering

Manuscript ID TBME-01171-2020.R1

Manuscript Type: Paper

Date Submitted by the

06-Jul-2021
Author:

Complete List of Authors: Kim, Dongwon

Koh, Kyung; University of Maryland, Baltimore,
Oppizzi, Giovanni; University of Maryland Baltimore
Baghi, Raziyeh; University of Maryland Baltimore
Lo, Li-Chuan; University of Maryland Baltimore
Zhang, Chunyang; University of Maryland Baltimore
Zhang, Li-Qun; University of Maryland Baltimore
Page 1 of 16
Kim et al. TBME-01171-2020

1
2
3
4 EMG-based Simultaneous Estimations of
5
6
7
Joint Angle and Torque during Hand
8
9
Interactions with Environments
10
11 Dongwon Kim1, Kyung Koh1, Giovanni Oppizzi1,3, Raziyeh Baghi1, Li-Chuan Lo1, Chunyang Zhang1, and Li-Qun Zhang1-3
12
1
13 Department of Physical Department of Physical Therapy and Rehabilitation Science, University of Maryland, Baltimore, MD, USA,
2
14 Department of Orthopaedics, University of Maryland, Baltimore, MD, USA
3
Department of Bioengineering, School of Engineering, University of Maryland, College Park, MD, USA
15
16 Abstract—It is necessary to control contact force through Index Terms—Human-machine interaction, Electromyography
17 modulation of joint stiffness in addition to the position of our limb (EMG), Decoding, Machine learning, Prosthesis.
18 when manipulating an object. This is achieved by contracting the
19 agonist muscles in an appropriate magnitude, as well as, balancing
20 it with contraction of the antagonist muscles. Here we develop a I. INTRODUCTION
decoding technique that estimates both the position and torque of

I
21
a joint of the limb in interaction with an environment based on N usual interactions with an object, human behaviors
22 activities of the agonist-antagonistic muscle pairs using are described in neither an isometric manner nor an
23 electromyography in real time. The long short-term memory isotonic manner. When grasping or manipulating an
24 (LSTM) network that is capable of learning time series of a long-
25 time span with varying time lags is employed as the core processor object, the human regulates the position of the limb, while
26 of the proposed technique. We tested both the unidirectional maintaining appropriate contact force or stiffness to
27 LSTM network and bidirectional LSTM network. A validation perform a given task. Torque to position the limb toward
was conducted on the wrist joint moving along a given trajectory the target is produced by differences between contractions
28 under resistance generated by a robot. The decoding approach
29 provided an agreement of greater than 93% in kinetics (i.e. torque)
of agonist and antagonist muscles about a joint. Joint
30 estimation and an agreement of greater than 83% in kinematics stiffness increases when both agonist and antagonist
31 (i.e. angle) estimation, between the actual and estimated variables, muscles are simultaneously activated. It may be necessary
32 during interactions with an environment. We found no significant to consider both the position and stiffness (or contact
33 differences in performance between the unidirectional LSTM and force) in controlling robot manipulators or prosthetic
bidirectional LSTM as the learning device of the proposed
34 decoding method. devices that replace the user’s limb.
35
36 A few studies focused on simultaneous estimation
37
schemes for joint angle and torque (stiffness) based on
38
Surface electromyography (EMG) is recognized as a EMG signals. The majority of the simultaneous
39
preferred interface versus other input media including a estimation schemes rely on mathematical models based
40
41
joystick or a haptic device with which mechanism that the on the intuitive fact [12-14]. These models predict the
42 user should be acquaint [1-5]. Indeed, frequent attempts position of a joint based on the difference between flexor
43 to use surface EMG to decode limb kinematics or kinetics and extensor activations, while estimating stiffness using
44 have been made through pattern recognition or regression simultaneous activations of the flexor and extensor.
45 (mapping EMG to angle or force) for controlling Though these methods are capable of efficiently
46 prosthetic devices or teleoperated robots [6-11]. Despite deciphering the user’s intention, the prediction accuracy
47 the need of taking both kinematics and kinetics in design of these methods is relatively low [12-14]. A significant
48 of interactive devices into account, only the isometric improvement was achieved in a study that adopted hidden
49 condition or isotonic condition is considered in most variables to model unobserved and intrinsic system states
50 EMG-based control applications. Kinematic data or between muscle activation and limb kinematics/kinetics,
51 kinetic data were obtained with no resistance to motions linking the hidden variables with muscle activation and
52 of participants involved or with the joint angle entirely limb kinematics/kinetics using an optimization technique
53 restricted. It may be thought to be less useful or less [15].
54 realistic for applications in interactive environments
55 which typically drive force exertion as well as motion. Nowadays, machine learning is one of the exploding
56 application fields that enables one to predict outcomes of
57
interest in a robust and accurate way based on a trained
58
59 This work was supported in part by the National Institute on Disability, Independent Living, and Rehabilitation Research (90DP0099 to L-QZ) and the University
60 of Maryland Baltimore, Institute for Clinical & Translational Research (ICTR)/Clinical and Translational Science Award (CTSA, 5TL1TR003100-03 to the first
author DK’s postdoctoral training). Corresponding author: Li-Qun Zhang.
 Page 2 of 16
Kim et al. TBME-01171-2020

1
black box that maps the inputs to the outputs. Certainly, procedure, respectively. This effort would provide an
2
machine learning techniques have been a tool in insight into the predictability of unlearned motions by the
3
4
developing EMG-based decoding methods. Several proposed technique based on learned motions. A
5 studies have demonstrated the excellent performance of comparison with a decoding method that possesses no
6 machine learning techniques in myoelectrically driven device for time dependency of data demonstrates the
7 applications using surface EMG [10], [16-21]. Still these proposed method’s advantageous capability of learning a
8 are restricted solely to decoding either kinematics or long time-span data for EMG-based decoding of
9 kinetics only, however. The purpose of this study is to kinematics and kinetics. Part of this study was presented
10 develop a method for EMG-based decoding of both joint in a conference [31].
11 kinematics and kinetics utilizing the power of machine
12 learning. II. METHODS
13
A. Participants
14 The time dependency of EMG signals is high;
15 instantaneous EMG is also informative. It is a typical way The experiment was approved by the Institutional
16 to extract features from EMG signals with a time window Review Board at the University of Maryland. All
17 of samples in a range between 50 ms and 500 ms [19], participants gave written informed consent. A total of 8
18 [22-24]. Storage of history of the related information is volunteers (4 female, 4 male), ranging in age from 23 to
19 required for processing EMG accordingly. The recurrent 41 years (33.25 ± 7.94 standard deviation), took part in
20 the study. All participants reported that they had no
neural network (RNN) is known as a suitable network for
21 neurological or motor deficit and that they were right-
processing sequential data by transferring information
22 handed.
23
across time steps via hidden states, which enables the
24 history of a sequence to be stored in the networks.
25 Traditional RNNs, however, expose a disadvantage of the B. Apparatus
26 requirement for time lags to be predetermined to learn A one degree-of-freedom robot for the wrist joint was
27 temporal sequence processing [25]. RNNs may not be developed. The motor (LS Mecapion, APM-SA01ACN-
28 fully appropriate to capture human movements that are 8) embedded an encoder of a resolution of 2048 pulses per
29 often aperiodic in daily life, though they are a revolution and was connected with a Harmonic Drive gear
30 reciprocating action. As a variant of RNN, the long short- with a ratio of 50:1. A torque sensor (Transducer
31 term memory (LSTM) neural network was devised to Techniques, TRT-200) was installed between the
32 learn time series with a long time span and determine Harmonic Drive and the hand plate. A custom Advanced
33 optimal time lags for prediction [26]. Indeed, LSTM Motion Controls (AMC) drive was used to convert the
34 neural networks have produced successful applications in motor command into the current to the motor.
35 various areas with strong time dependency, including
36 speech recognition [27] and human action recognition
37 Two wireless pre-amplified surface EMG electrodes
[28-30]. LSTM neural networks are an excellent (Delsys, Trigno) were used to acquire EMG signals from
38 candidate to decode human motion based on EMG
39 the flexor carpi radialis (FCR), and extensor carpi radialis
signals. Recently, an EMG-based kinematics decoding (ECR) muscles.
40
method that combined LSTM neural network with
41
convolutional neural network was proposed [19]. A monitor was placed in front of the participant with a
42
43 distance of about 1 meter to display wrist flexion
44 The proposed EMG decoding method of both joint trajectories to follow. Fig. 1. displays the schematic of the
45 kinematics and kinetics stands on unidirectional LSTM experimental setup.
46 and bidirectional LSTM neural networks. We evaluate its
47 performance in estimating the wrist angle and torque Data processing including feature extraction as well as
48 based on surface EMGs of a pair of wrist flexor and data acquisition and motor control were conducted in a
49 extensor muscles. While several magnitudes of resistance LabVIEW environment through a data acquisition card
50 are imposed on the wrist joint by a robot, participants are (Texas Instruments, PCI-6225). The sampling rate for all
51 asked to move their wrist joint according to given data was set at 1000 Hz.
52 trajectories. To examine the efficacy of LSTM neural
53 networks in learning time series with long time spans The EMG-based decoder was developed and performed
54 using varying time lags for prediction of the angle and
55
utilizing Matlab Deep Learning Toolbox™ (MathWorks,
torque of the wrist joint, we present different trajectories Matlab 2018b). EMG features calculated in the LabVIEW
56 with irregularity and aperiodicity to the participants for
57 program were exchanged with estimated angle and torque
the training and evaluation phases of the decoding
58
59
60
Page 3 of 16
Kim et al. TBME-01171-2020

1
2
3
4
5
6
7
8 Fig. 2. The architecture of the proposed decoding strategy. The proposed
9 decoder consists of multiple steps between the input (EMG features) and
output (joint angle and torque); EMG features go through the sequence of
10 input layer, LSTM neural network, fully connected layer, dropout layer, fully
11 connected layer again, and regression layer. The angle and torque estimations
12 are achieved through this procedure.
13
14
15 During the training phase, the decoder was trained with
16 the EMG features as the input and the profiles of joint
17 angle and torque as the output. These features (2 RMS
Fig. 1. Experimental setup: A participant sits on a chair with a specific arm
18 configuration on the right side: 45 degree abduction and 45 degree flexion at values +12 coefficients of AR models) were extracted
19 the shoulder, 50 degree flexion at the elbow. Two wireless surface EMG from EMG signals with a time window, while the profiles
electrodes are placed on the FCR and ECR muscles. A monitor displays the of joint angle and torque were averaged over the time
20 wrist flexion trajectories for the participant to follow.
21 window. During evaluation, the trained decoder estimated
22 produced in Matlab through UDP (User datagram the profiles of joint angle and torque according to the
23 protocol) communication. EMG features that were extracted from EMG signals with
24 the time window.
25 C. Signal Processing
26 In this study, the time window was set as 100 ms. In a
27 Raw EMG signals were differenced prior to feature
preliminary study we found that the time window of 100
28 extraction, since differencing enables EMG signals to be
ms provided the best performance of the proposed method
29 more stationary than original signals [30]. We extracted for prediction accuracy. The number of hidden units in the
30 the features of EMG signals using root mean square LSTM neural network was set to be 200. No further
31 (RMS) values and the coefficients of 6th-order notable improvement in estimation accuracy was seen
32 autoregressive (AR) models [32]. beyond the number.
33
34
35 D. Decoder Development 2) LSTM Neural Network
36 A unidirectional LSTM neural network consisted of an
1) Decoder Architecture
37 input layer, LSTM layer (hidden layer) and output layer
The architecture of the proposed decoder was formed
38 (see Fig. 3). The LSTM layer was filled with multiple
with the sequence input layer, LSTM neural network,
39 LSTM blocks that included a cell with self-connections
fully connected layer, dropout layer, again fully
40 connected layer, and regression layer in turn. The LSTM block
41 Input gate Output gate
sequence input layer transferred the sequence data (EMG i o
42
features) to the LSTM network. A delay of 50 msec was Cell
43 𝒉(𝑘 − 1) 𝒉(𝑘)

44
incorporated with the EMG signals to account for the c

45 electromechanical delay from muscle activation to the

f
46 resulting force production. The two fully connected layers 𝒚(𝑘 − 1) Forget gate

47 linked every output element in the previous step with Output layer 𝒚(𝑘 − 2) 𝒚(𝑘 − 1)
𝒙(𝑘)
𝒚(𝑘)

48 every input element in the next step. The number of the ⋯ 𝜎 𝜎 𝜎 ⋯

49 first fully connected layer was set as 150. The dropout

LSTM layer
50 layer set input elements to zero randomly, with a 𝒉(𝑘 − 3) 𝒉(𝑘 − 2) 𝒉(𝑘 − 1) 𝒉(𝑘)
⋯ ⋯
51 probability of 0.5. The regression layer dealt with the LTSM block LTSM block LTSM block

52 regression problem between the actual and predicted

53 values of angle and torque. The number of the learnable Input layer ⋯ 𝒙(𝑘 − 2) 𝒙(𝑘 − 1) 𝒙(𝑘) ⋯
54 parameters was estimated as 136. The architecture of the Fig. 3. A schematic of an (unidirectional) LSTM network. A relationship
55 proposed decoding strategy is presented in Fig. 2. between the input vector x and output layer y is formed by activation vectors
produced by the cell, and the input, output and forget gates. The input gate (i)
56 modulates the input to the cell (c), the forget gate (f) allows the LSTM block
57 to forget the previous memory, and the output gate (o) determines the output
58 of the hidden layer.
59
60
 Page 4 of 16
Kim et al. TBME-01171-2020

1
for storing temporal states and a pair of adaptive,
2
multiplicative gating units. The cell (c) was responsible where σ(∙) is the sigmoid function and the operator ⊚ is
3
for tracking the dependencies between the elements in the the scalar product of two vectors. i, f, o, and c ∈
4
5 input sequence. The forget gate (f) was responsible for 𝐑 denote the activation vectors of the input, forget,
6 removing information from the cell state once the output, and cell gates, respectively. h ∈ 𝐑 denotes the
7 information was out of date. The input and output hidden state of the hidden layer. The input weight
8 activations were controlled by the input gate (i) and output matrices W ∈ 𝐑 × quantifies the connection of the input
9 gate (o). The input gate governed a new value that flows vector x with the input state i, forget state f, output state
10 into the cell, while the output gate governed the value in o, cell state c, or output y. The recurrent weight matrices
11 the cell that was used to compute the output activation. U ∈ 𝐑 × quantifies the connection of the hidden state h
12 with the input gate i, forget gate f, output gate o, cell state
13 While unidirectional LSTM neural networks learn the c, or output vector y. The vector b ∈ 𝐑 denotes the biases
14 time dependencies of the input data that are of these connections.
15 chronologically ordered in a positive direction from time
16 step k−1 to time step k, bidirectional LSTM neural
17 E. Procedure
networks additionally learn the dependencies of the The experiment was composed of 3 stages in each of
18 reverse-chronological ordered input data to exploit more
19 which different magnitudes of resistance were imposed
information. Fig. 4 depicts a bidirectional LSTM network on the wrist joint by the wrist robot. In the first stage, the
20
that consists of a backward layer as well as a forward layer robot was designed to produce minimized resistance
21
to learn the backward and forward dependencies of the (Soft), using an impedance control technique derived
22
23 from the one (rigid mode) presented in [32], while
24 Output layer 𝒚(𝑘 − 2) 𝒚(𝑘 − 1) 𝒚(𝑘)
resistance with a stiffness of 0.005 Nm/deg (Less Hard)
25 ⋯ 𝜎 𝜎 𝜎 ⋯ and 0.03 Nm/deg (Hard) was imposed for the second and
26 third stages, respectively, using proportional control (P
27 Backward LSTM layer
control, see Fig. 5). The resistive torque was 0 at the
28 𝒉(𝑘 − 3) 𝒉(𝑘 − 2) 𝒉(𝑘 − 1) 𝒉(𝑘)
⋯ ⋯
LTSM block LTSM block LTSM block
neutral position of the wrist joint. To overcome the
29 resistance by the robot, the participants need to generate a
30 Forward LSTM layer
sufficient magnitude of torque at the joint through
31 ⋯ 𝒉(𝑘 − 3)
LTSM block
𝒉(𝑘 − 2)
LTSM block
𝒉(𝑘 − 1)
LTSM block
𝒉(𝑘)
contraction of the agonist muscles. As well, they are
32
required to modulate joint stiffness to follow the given
33 Input layer ⋯ 𝒙(𝑘 − 2) 𝒙(𝑘 − 1) 𝒙(𝑘) ⋯ trajectory as accurately as possible through co-contraction
34
35 Fig. 4. A schematic of a bidirectional LSTM network. A bidirectional LSTM
network consists of a backward layer and a forward layer to learn the backward
36 and forward dependencies of the input data.
37
input data.
38
39
40 Through the sequence input layer, EMG features (vector
41 x ∈ 𝐑 ) were admitted by the LSTM neural network and
42 outputs (vector y ∈ 𝐑 ) were processed that led to the Fig. 5. Schematics of wrist joint movements in the presence of (a) zero
43 corresponding angle and torque through the fully resistance, (b) resistance with a stiffness of 0.005 Nm/deg and (c) resistance
44 connected layer, dropout layer, and regression layer. The of 0.02 Nm/deg. The resistive torque is 0 at the neutral position (black thick
line) of the wrist joint.
45 dimensions 𝛼 and 𝛽 denoted the number of input features
46 (14: 2 RMS values and 12 coefficients of AR models) and of the agonist-antagonist muscles. By imposing different
47 number of hidden units (200), respectively. A relationship magnitudes of resistance, we assess the performance of
48 between vector x and vector y at time point k of the LSTM the proposed technique to decode joint torque as well as
49 neural network was identified using the following joint angle, and accordingly, joint stiffness. Each stage
50 equations: consisted of a training phase and evaluation phase, and
51 each phase lasted about 80 seconds. A break time of 2
52 𝒊 = 𝜎(𝑾 𝒙 + 𝑼 𝒉 + 𝒃 ), (1) minutes was provided after each phase to mitigate the
53 𝒇 =𝜎 𝑾 𝒙 +𝑼 𝒉 +𝒃 , (2) fatigue effect.
54 𝒐 = 𝜎(𝑾 𝒙 + 𝑼 𝒉 + 𝒃 ), (3)
55 𝒄 =𝒇 ⊚𝒄 + 𝒊 ⊚ tanh(𝑾 𝒙 + 𝑼 𝒉 + 𝒃 ), (4) All participants sat on a chair in a comfortable posture but
56 𝒉 = 𝒐 ⊚ tanh(𝒄 ), (5) with a specific arm configuration on the right side: 45
57 𝒚 =𝑼 𝒉 +𝒃 , (6) degree abduction and 45 degree flexion in the shoulder,
58
59
60
Page 5 of 16
Kim et al. TBME-01171-2020

1
50 degree flexion in the elbow. We set the neutral position
2
of the wrist joint as 0 degrees. Participants were asked to To assess similarity in each participant between the data
3
4
move their right wrist joint, tracking a respective given acquired during training and the data acquired during
5 trajectory for each phase (Fig. 6). These trajectories were evaluation, we utilized the 12 coefficients of the AR
6 generated using the X signal of a chaotic system to be model of EMG signals. Different inputs meant different
7 irregular, aperiodic and of wide bandwidth [33]. The outputs (i.e. angle and torque). By comparing the values
8 chaotic system is equated as: of the coefficients of EMG signals for each time window
9 during training with those of the corresponding
10 𝑥̇ = a(y − x), coefficients for every time window during evaluation, we
11 𝑦̇ = −xz + cy, (7) examined whether or not the data used to train the decoder
12 𝑧̇ = 𝑦 − 𝑏𝑧, were used for the evaluation phase. Similarity can be
13 Where the parameters were selected as a = 35, b = 3, and indexed as the minimum value of the summations of the
14 differences in the values of the 12 coefficients throughout
15 (a) all time windows between the training and evaluation
16 phases. While the difference in the values for each
17 coefficient was ranged within 0 and 1, the index of 0
18 indicated that at least one same input of EMG existed in
Angle (deg)

19 both the training and evaluation phases.

20
21
To evaluate the performance of the proposed decoding
22
23 (b) method with the unidirectional LSTM and bidirectional
24 LSTM, a single decoder for each individual was trained
25 with EMG features as the input and with joint angle and
26 torque data as the output acquired throughout the training
27 phase. The performance of the decoding strategy was
28 evaluated by examining how accurately the joint angle
29 and torque were predicted, respectively, by the trained
30 decoder based on EMG signals during the evaluation
31 phase. The variance accounted for (VAF) between the
Fig. 6. Examples of trajectories given for each phase: (a) the training phase
32 and (b) evaluation phase. Different trajectories are generated using a chaotic measured and predicted joint angle and between the
33 system by modulating its initial values. An angle of 0 degrees indicates the measured joint torque and predicted joint torque was
34 neutral position of the wrist joint. An angle of positive values is during computed, respectively. The VAF is defined as
extension, whereas an angle of negative values is during flexion.
35
36 c = 25. VAF = 1 −
( )
(8)
37 ( )
38 where 𝑦 denotes measured values, while 𝑦 denotes
The amplitude of the trajectories was bounded at ±50 estimated values.
39
degrees and their back-and-forth movement frequency
40
was within about 0.6 Hz. The range of the amplitude For a comparative study, RNN and a general linear model
41
covers the range of motion of the wrist joint for activities (GLM) [24] for the regression problem were selected.
42
43 of daily living [34], the designed motion speed, which While the same feature extraction (explained above) was
44 varied with time, can be regarded modest in daily life, used, the raw EMG signals were bandpass filtered (200-
45 based on [35]. 400 Hz) for a better performance of GLM. The time
46 window was set as 10 ms or 20 ms, across subjects, for
47 F. Performance Evaluation the RNN approach, while it was selected as 300 ms for
48 the GLM approach, which produced the best
49 First, we analyzed the outcomes of the experiment.
Means by each case across trial of the RMS values of the performance. The number of hidden units in the RNN
50 decoder was selected as 50. No further improvement in
51 angle and torque calculated over each time window were
evaluated to compare wrist joint displacements and torque estimation accuracy was observed with more hidden units
52
generation according to the extent of resistance. To for RNN.
53
54 estimate changes in wrist joint stiffness according to the
55 increases in resistance by the robot, the averaged For the ease of data analysis, we first finished collecting
56 instantaneous stiffness (partial derivative of the torque all data and simulated the real-time circumstance to
57 versus angle curve) was calculated. evaluate the online performance of the decoding
58 technique. For evaluation, the acquired EMG data during
59
60
 Page 6 of 16
Kim et al. TBME-01171-2020

1
2
3
4
5
6
7
8
9
10
11
12 Fig. 7. Means by each case across trial of the RMS values of (a) the angle and (b) torque calculated over each time window, and (c) means by each case of
13 instantaneous stiffness during the trial.
14 the evaluation phase were read and processed to the EMG ANOVA on the mean across trial of the RMS values of
15 features in the LabVIEW program. The EMG features the torque calculated over each time window reported a
16 were then transferred via UDP to the decoder that had significant main effect of case [F(2,14)=1272.344, p <
17 been trained in the secondary PC with the data acquired 0.001 ]. Pairwise comparisons revealed significant
18 during the training phase, and the decoder sent out the differences between case Soft and case Less Hard (p <
19
predicted values back to the LabVIEW program. A 0.001) and between case Less Hard and case Hard (p <
20
comparison between the actual and predicted values 0.05). These results suggested that participants generated
21
(angle and torque) was made then. greater joint torque in case Hard to overcome the
22
23 resistance by the robot than in case Less Hard that caused
24 G. Statistical Analysis greater torque in comparison to case Soft.
25 A repeated-measures analysis of variance (ANOVA),
26 ANOVA on stiffness revealed a significant main effect of
with case (three levels: Soft, Less Hard and Hard) as a case [F(2,14)=97.040, p < 0.001]. Pairwise comparisons
27
within-subjects variable, was used to evaluate the degrees revealed significant differences between case Soft and
28
of movement, contact force and stiffness across repeated case Less Hard (p < 0.001) and between case Less Hard
29
30
measurements. A mixed-design ANOVA, with case as a and case Hard (p < 0.001). These results suggested that
31 within-subjects variable and decoding method as a participants employed stiffer wrist joint as the resistance
32 between-subjects variable, was employed to compare the by the robot was greater.
33 performances of decoding approaches. If the sphericity
34 assumption in ANOVAs was violated, then Greenhouse- The similarity index for each participant was identified as
35 Geisser adjusted p-values were used. Bonferroni post-hoc a value above 0.276. These results indicated that none of
36 tests were conducted if pairwise comparisons were the inputs used to train the decoder of each participant was
37 needed. The statistical analyses were performed with the same as those used for evaluation.
38 SPSS version 20.0 (SPSS Inc., Chicago, USA) and the
39 significance level was set at 0.05. All analyses were With the new inputs that were not observed in the data
40 preceded by Shapiro–Wilk tests of normality and their
41
sets for decoder training, the proposed decoding method
results were employed only when normality was not in general showed an agreement of greater than 0.83 in
42 violated.
43 VAF for angle estimation and an agreement of greater
44 than 0.93 for torque estimation, between the actual and
III. RESULTS
45 estimated variables, during interactions with an
46 First, we evaluated our experimental design that was environment (cases Less Hard and Hard).
47 aimed at eliciting different magnitudes of contact force
48 and joint stiffness across cases while maintaining similar The proposed decoding technique incorporated with a
49 magnitudes of the angular displacement of the wrist joint unidirectional LSTM network produced a VAF of 0.8307
50 (see Fig. 7). and 0.9105 in average during cases Less Hard and Hard,
51 respectively, while it produced a VAF of 0.8083 in
52 It was reported that the mean across trial of the RMS average during case Soft, for angle estimation. For torque
53 values of the angle calculated over each time window had estimation, a VAF of 0.9297 and 0.9552 in average was
54 no significant main effect of case, implying that seen during cases Less Hard and Hard, respectively.
55 participants made similar amounts of movement during VAFs between the measured and predicted joint torque
56 trials in each case. for case Soft were not computed, since the noise
57
component was dominant in measured joint torque for this
58
59
60
Page 7 of 16
Kim et al. TBME-01171-2020

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22 Fig. 8. Results by the unidirectional LSTM network of Subject 3. The Fig. 9. Results by the bidirectional LSTM network of Subject 3. The trajectories
23 trajectories of the measured and predicted joint angle (a) and the trajectories of of the measured and predicted joint angle (a) and the trajectories of the
24 the measured and predicted joint torque (b) for each case are displayed. The measured and predicted joint torque (b), and the corresponding raw EMG
corresponding raw EMG signals of the ECR muscle (c) and FCR muscle (d) are signals of the ECR muscle (c) and FCR muscle (d) are exhibited. The technique
25 presented. The proposed decoding method predicts the trajectory of the shows a VAF of 0.9284 and 0.9152 in the cases of Less Hard and Hard,
26 angle/torque averaged over the time window of 100 ms based on EMG signals, respectively, while it shows a VAF of 0.7416 in the Soft case, for position
27 and the predicted trajectory (solid line) closely follows the trajectory (dashed estimation. For torque estimation, a VAF of 0.9305 and 0.9683 are recorded in
line) of the original angle/torque (dotted line) averaged over the time window of the cases of Less Hard and Hard, respectively.
28 100 ms. The original trajectory and averaged trajectory closely overlap. The
29 technique shows a VAF of 0.8845 and 0.9427 in the cases of Less Hard and For RNN, a VAF of 0.3953 and 0.6316 in average for
30 Hard, respectively, while it shows a VAF of 0.8351 in the Soft case, for position
angle estimation was seen during cases Less Hard and
estimation. For torque estimation, a VAF of 0.9459 and 0.9614 are recorded in
31 the cases of Less Hard and Hard, respectively. Hard, respectively. For torque estimation, a VAF of
32 0.7124 and 0.7246 in average was observed during cases
33 case where the measured torque was assumed to be
Less Hard and Hard, respectively.
34 around zero. The noise component is difficult to be
35 predicted by the proposed decoding method. Fig. 8 shows
For GLM, a VAF of 0.2451, 0.3635 and 0.4924 in average
36 the results of a representative participant with the
for angle estimation was recorded during cases Soft, Less
37 unidirectional LSTM. The trajectories of the original
38
Hard and Hard, respectively. For torque estimation, a
angle and torque and the corresponding trajectories
39 VAF of 0.6755 and 0.7324 in average was seen during
averaged over the time window of 100 ms closely
40 cases Less Hard and Hard, respectively.
overlapped with each other. This implies that beside the
41 50ms electromechanical delay, there is no extra delay in
42 ANOVA reported a significant main effect of decoding
estimating outputs using the EMG features that were
43 method for angle estimation [F(3,28) = 60.140, p < 0.001,
computed based on the past 150 to 50 ms EMG signals
44 𝜂 =0.866]. Pairwise comparisons showed that there were
within the time window of 100 ms.
45 no significant differences in performance between the
46 unidirectional LSTM and bidirectional LSTM as the
47
For the performance created by the bidirectional LSTM,
learning device of the proposed decoding methods,
48 a VAF of 0.8414 and 0.8700 in average for angle
whereas these methods outperformed the methods with
49 estimation was recorded during cases Less Hard and
RNN and GLM (p < 0.001). A significant main effect of
50 Hard, respectively. For torque estimation, a VAF of
decoding method was observed for torque estimation
51 0.9335 and 0.9515 in average was seen during cases Less
[F(3,28) = 12.067, p < 0.001, 𝜂 =0.564]. Pairwise
52 Hard and Hard, respectively. The decoder with the
comparisons showed that there were no significant
53 bidirectional LSTM produced a VAF of 0.7231 in average
differences in performance among the unidirectional
54 for position estimation during case Soft. Fig. 9 displays
LSTM, bidirectional LSTM and RNN as a learning device
55 the results of the representative participant with the
56 while the method with GLM marked the lowest accuracy
bidirectional LSTM.
57 (p < 0.005). Significant differences between decoding
58 methods for each case are displayed in Fig. 10.
59
60
 Page 8 of 16
Kim et al. TBME-01171-2020

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 Fig. 10. A comparison of the proposed decoders incorporated with the unidirectional LSTM and bidirectional LSTM with the decoders based on RNN and GLM.
VAFs averaged across all participants for each case and the corresponding standard deviations are presented. VAFs between the measured and predict ed joint torque
29 for case Soft are not evaluated; the noise component that is difficult to be predicted is dominant in measured joint torque for CASE 1 where the measured torque is
30 nearly zero in the ideal condition.
31 In this study, an EMG-based decoding method was
32 ANOVA revealed a significant main effect of case for proposed that predicts the position and torque of the wrist
33 angle estimation [F(1.492,41.773) = 14.481, p < 0.001, joint simultaneously. Numerous attempts have been made
34 in an effort to decipher the user’s intention through EMG
𝜂 =0.341] and torque estimation [F(1,28) = 7.550, p <
35 interfaces [3], [10], [16], [36], [37], but the majority of
0.05, 𝜂 =0.212]. These results suggest that greater
36
accuracy appears in case Hard than in case Less Hard and these techniques tends to be concentrated on human
37
greater accuracy appears in case Less Hard than in case movement decoding, despite the need of consideration in
38
39 Soft for angle estimation while greater accuracy is design of interactions with environments. Relatively few
40 exhibited in case Hard than in case Less Hard for torque studies have been launched to develop decoding strategies
41 estimation. that predict both kinematics and kinetics (or stiffness).
42
43 IV. DISCUSSIONS In our experiments, three levels of resistance were
44 imposed on the wrist joint, while participants tracked the
45 When we pick up either a tomato or an apple, different given trajectories with similar amplitudes. As resistance
46 magnitudes of contact force are required, relying increased, it was required for participants to generate
47 primarily on somatosensory feedback. If contact force is greater torque to overcome resistance through contraction
48 excessive, the fruit can burst. If contact force is of the agonist muscles. Meanwhile, it was required to
49 insufficient, it can be dropped from the hand. All the modulate joint stiffness to follow the given trajectory as
50 while, movements of the fingers are involved in holding accurately as possible through contraction of the
51 the object. Though it is possible for a robot manipulator antagonist muscles, balancing it with contraction of the
52 or a computer-controlled prosthesis to deal with this agonist muscles [38-40]. Therefore, the experiment
53 problem through position/force hybrid control or design is thought to provide an insight into human motor
54 control in terms of joint movement as well as joint
impedance control, both 'contact force' and 'position'
55
must be in control of the user when the operation is driven stiffness during a given task. Indeed, the results showed
56
by the user’s biological signals. that torque production increased as greater resistance was
57
58
imposed on the wrist, while the amplitudes of joint
59
60
Page 9 of 16
Kim et al. TBME-01171-2020

1
movement remained the similar across cases. The suitability and superiority of the proposed method
2
Correspondingly, joint stiffness increased to counteract that employs LSTM neural networks in EMG-based
3
4
the increased resistance by the robot. decoding tasks are pronounced from the comparisons
5 with the decoders that employ RNN and GLM. RNN is
6 Overall, the proposed decoding method produces an characterized by fixed time lags in training. Lower
7 agreement of greater than 93% in torque estimation and accuracy of the RNN decoder implies that varying time
8 an agreement of greater than 83% in angle estimation, lags in training is more efficient in learning biological
9 between the actual and estimated values, during signals than fixed time lags. The results of the decoder
10 interactions with an environment. The proposed decoder with GLM indicates the lack of the capability in the
11 based on (unidirectional or bidirectional) LSTM neural regression-based method to deal with the time
12 networks is superior in estimation performance to the dependency of EMG signals. This method searches the
13 decoders based on RNN and GLM, as presented in Fig. 9. coefficients of the regression model through the
14 In particular, the decoders with LSTM neural networks relationships of the input and output at the same time
15 produce significantly greater accuracy in angle point.
16 estimation, in comparison with the decoders based on
17 RNN and GLM (p < 0.05). We showed no significant differences in performance
18 between the decoder with the unidirectional LSTM and
19 Accuracy is greater in estimations of the angle and torque the decoder with the bidirectional LSTM. Unidirectional
20
during case Hard, followed by cases Soft and Less Hard LSTM neural networks make use of only the forward
21
in turn (p < 0.05). It may be because stronger resistance dependency of time series data which are chronologically
22
23
causes more muscle activation while achieving arranged. Meanwhile, bidirectional LSTM neural
24 movements of the same amplitude, which makes clear the networks make use of both the forward and backward
25 difference between the phases of contraction and non- dependencies of data. It was reported that in the case that
26 contraction of the target muscle. the time series data show periodicity and regularity,
27 bidirectional LSTM neural networks outperform
28 The proposed decoding method employed a machine unidirectional LSTM neural networks [42]. The fact that
29 learning technique. Machine learning has recently our data are highly irregular and aperiodic primarily leads
30 attracted extraordinary attention as a tool highly efficient to no significant differences in performance between the
31 to relate two parties that look difficult to be related. two decoders.
32 Several studies demonstrated the superiority of neural
33 networks in myoelectrically driven applications using
34 surface EMG [10], [16-20]. In particular, LSTM neural V. CONCLUSION
35 networks are able to automatically determine the optimal We have proposed an EMG-based decoding technique for
36 time lags and learn the time series with longtime spans. simultaneous estimation of the position and torque of a
37 Indeed, our proposed method estimated participants’
38 joint of the limb in interaction with environments,
movement and force with an excellent agreement, though employing LSTM neural networks. The proposed method
39
the decoder was not trained with the similar trajectories that admits the advantages of LSTM networks showed
40
used for evaluation. As observed in Fig. 5, the two superior performance in simultaneous estimation of the
41
trajectories for the training and evaluation phases, position and torque by dealing with high time dependency
42
43
respectively, are highly irregular, aperiodic and different of EMG inputs. The decoding results showed an
44 from each other. The two trajectories lead to inputs for the agreement of greater than 93% in torque estimation and
45 training phase that are not shared with those for the an agreement of greater than 83% in angle estimation,
46 evaluation phase. These results imply the strength of the between the actual and estimated values, during
47 proposed method that is based on what the decoder is interactions with an environment. Through a comparative
48 trained using varying time lags in estimating the angle and study, we demonstrated the suitability and superiority of
49 torque with new inputs resulting from a series of different the proposed method based on LSTM neural networks
50 trajectories of motion [41]. In our related study, we with the capability of learning a long time-span data with
51 demonstrated that the decoding strategy with LSTM varying time lags.
52 neural networks is superior in coping with the data with a
53 long-time span [31]. This feature is advantageous in that
54 unlearned trajectories of motion can be predicted from REFERENCES
55 learned trajectories of motion.
56 [1] A. T. Au and R. F. Kirsch, "EMG-based prediction of
57 shoulder and elbow kinematics in able-bodied and spinal
58
59
60
 Page 10 of 16
Kim et al. TBME-01171-2020

1 cord injured individuals," IEEE Trans Rehabil Eng, vol. [15] P. K. Artemiadis and K. J. Kyriakopoulos, "EMG-based
2 8, pp. 471-80, 2000. position and force estimates in coupled human-robot
3 [2] D. Song, P. Hendrickson, V. Z. Marmarelis, J. Aguayo, systems: Towards EMG-controlled exoskeletons," in
4 J. He, G. E. Loeb, and T. W. Berger, "Predicting EMG Experimental Robotics, 2009, pp. 241-250.
5 with generalized Volterra kernel model," Conf Proc [16] S. Muceli and D. Farina, "Simultaneous and proportional
6 IEEE Eng Med Biol Soc, vol. 2008, pp. 201-4, 2008. estimation of hand kinematics from EMG during
7 [3] C. L. Pulliam, J. M. Lambrecht, and R. F. Kirsch, mirrored movements at multiple degrees-of-freedom,"
8 "Electromyogram-based neural network control of IEEE Trans Neural Syst Rehabil Eng, vol. 20, pp. 371-8,
9 transhumeral prostheses," J Rehabil Res Dev, vol. 48, pp. 2012.
10 739-54, 2011. [17] N. Jiang, S. Muceli, B. Graimann, and D. Farina, "Effect
11 [4] P. Liu, L. Liu, F. Martel, D. Rancourt, and E. A. Clancy, of arm position on the prediction of kinematics from
12 "Influence of joint angle on EMG-torque model during EMG in amputees," Medical & biological engineering &
13 constant-posture, quasi-constant-torque contractions," J computing, vol. 51, pp. 143-151, 2013.
14 Electromyogr Kinesiol, vol. 23, pp. 1020-8, 2013. [18] M. Atzori, M. Cognolato, and H. Müller, "Deep learning
15 [5] X. Li, H. Shin, P. Zhou, X. Niu, J. Liu, and W. Z. Rymer, with convolutional neural networks applied to
16 "Power spectral analysis of surface electromyography electromyography data: A resource for the classification
17 (EMG) at matched contraction levels of the first dorsal of movements for prosthetic hands," Frontiers in
18 interosseous muscle in stroke survivors," Clin neurorobotics, vol. 10, p. 9, 2016.
19 Neurophysiol, vol. 125, pp. 988-94, 2014. [19] P. Xia, J. Hu, and Y. Peng, "EMG‐based estimation of
20 [6] D. Staudenmann, I. Kingma, D. F. Stegeman, and J. H. limb movement using deep learning with recurrent
van Dieen, "Towards optimal multi-channel EMG convolutional neural networks," Artificial organs, vol.
21
electrode configurations in muscle force estimation: a 42, pp. E67-E77, 2018.
22
high density EMG study," J Electromyogr Kinesiol, vol. [20] M. Zia ur Rehman, A. Waris, S. Gilani, M. Jochumsen,
23
15, pp. 1-11, 2005. I. Niazi, M. Jamil, D. Farina, and E. Kamavuako,
24 [7] A. M. Simon, K. Stern, and L. J. Hargrove, "A "Multiday EMG-based classification of hand motions
25 comparison of proportional control methods for pattern with deep learning techniques," Sensors, vol. 18, p.
26 recognition control," Conf Proc IEEE Eng Med Biol Soc, 2497, 2018.
27 vol. 2011, pp. 3354-7, 2011. [21] Y. Fang, D. Zhou, K. Li, Z. Ju, and H. Liu, "Attribute-
28 [8] B. Peerdeman, D. Boere, H. Witteveen, R. H. in 't Veld, Driven Granular Model for EMG-Based Pinch and
29 H. Hermens, S. Stramigioli, H. Rietman, P. Veltink, and Fingertip Force Grand Recognition," IEEE Trans
30 S. Misra, "Myoelectric forearm prostheses: state of the Cybern, 2019.
31 art from a user-centered perspective," J Rehabil Res Dev, [22] L. H. Smith, L. J. Hargrove, B. A. Lock, and T. A.
32 vol. 48, pp. 719-37, 2011. Kuiken, "Determining the optimal window length for
33 [9] J. M. Hahne, F. Biessmann, N. Jiang, H. Rehbaum, D. pattern recognition-based myoelectric control: balancing
34 Farina, F. C. Meinecke, K. R. Muller, and L. C. Parra, the competing effects of classification error and
35 "Linear and nonlinear regression techniques for controller delay," IEEE Trans Neural Syst Rehabil Eng,
36 simultaneous and proportional myoelectric control," vol. 19, pp. 186-92, 2011.
37 IEEE Trans Neural Syst Rehabil Eng, vol. 22, pp. 269- [23] R. Menon, G. Di Caterina, H. Lakany, L. Petropoulakis,
38 79, 2014. B. A. Conway, and J. J. Soraghan, "Study on interaction
39 [10] J. Liu, S. H. Kang, D. Xu, Y. Ren, S. J. Lee, and L. Q. between temporal and spatial information in
40 Zhang, "EMG-Based Continuous and Simultaneous classification of EMG signals for myoelectric
41 Estimation of Arm Kinematics in Able-Bodied prostheses," IEEE Transactions on Neural Systems and
42 Individuals and Stroke Survivors," Front Neurosci, vol. Rehabilitation Engineering, vol. 25, pp. 1832-1842,
43 11, p. 480, 2017. 2017.
[11] W. Sun, J. Zhu, Y. Jiang, H. Yokoi, and Q. Huang, "One- [24] J. Liu, R. Yupeng, D. Xu, S. H. Kang, and L. Q. Zhang,
44
45
Channel Surface Electromyography Decomposition for "EMG-based real-time linear-nonlinear
Muscle Force Estimation," Front Neurorobot, vol. 12, p. cascade regression decoding of shoulder,
46
20, 2018.
47 [12] S. Rao, R. Carloni, and S. Stramigioli, "Stiffness and
elbow and wrist movements in able-bodied
48 position control of a prosthetic wrist by means of an persons and stroke survivors," IEEE transactions
49 EMG interface," Conf Proc IEEE Eng Med Biol Soc, vol. on Biomedical Engineering, In press.
50 2010, pp. 495-8, 2010. [25] F. A. Gers, J. Schmidhuber, and F. Cummins, "Learning
51 [13] E. Hocaoglu and V. Patoglu, "Tele-impedance control of to forget: Continual prediction with LSTM," 1999.
52 a variable stiffness prosthetic hand," in 2012 IEEE [26] S. Hochreiter and J. Schmidhuber, "Long short-term
53 International Conference on Robotics and Biomimetics memory," Neural Comput, vol. 9, pp. 1735-80, 1997.
54 (ROBIO), 2012, pp. 1576-1582. [27] A. Graves, A.-r. Mohamed, and G. Hinton, "Speech
55 [14] C. W. Antuvan and L. Masia, "Position and stiffness recognition with deep recurrent neural networks," in
56 modulation of a wrist haptic device using myoelectric 2013 IEEE international conference on acoustics,
57 interface," IEEE Int Conf Rehabil Robot, vol. 2017, pp. speech and signal processing, 2013, pp. 6645-6649.
58 734-739, 2017.
59
60
Page 11 of 16
Kim et al. TBME-01171-2020

1 [28] A. Graves and J. Schmidhuber, "Framewise phoneme [42] C. Cui, G. Bian, Z. Hou, J. Zhao, G. Su, H. Zhou, L.
2 classification with bidirectional LSTM and other neural Peng, and W. Wang, "Simultaneous Recognition and
3 network architectures," Neural Netw, vol. 18, pp. 602- Assessment of Post-Stroke Hemiparetic Gait by Fusing
4 10, 2005. Kinematic, Kinetic, and Electrophysiological Data,"
5 [29] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. IEEE Transactions on Neural Systems and
6 Baskurt, "Sequential deep learning for human action Rehabilitation Engineering, vol. 26, pp. 856-864, 2018.
7 recognition," in International workshop on human
8 behavior understanding, 2011, pp. 29-39.
9 [30] J. Kumar, R. Goomer, and A. K. Singh, "Long short term
10 memory recurrent neural network (lstm-rnn) based
11 workload forecasting model for cloud datacenters,"
12 Procedia Computer Science, vol. 125, pp. 676-682,
13 2018.
14 [31] D. Kim, K. Koh, G. Oppizzi, R. Baghi, L.-C. Lo, C.
15 Zhang, and L.-Q. Zhang, "Simultaneous Estimations of
16 Joint Angle and Torque in Interactions with
17 Environments using Electromyography," presented at
18 the International Conference on Robotics and
19 Automation (ICRA), Paris, 2020.
20 [32] D. Kim, K. Koh, G.-R. Cho, and L.-Q. Zhang, "A Robust
Impedance Controller Design for Series Elastic
21
Actuators using the Singular Perturbation Theory,"
22
IEEE/ASME Transactions on Mechatronics, vol. 25, pp.
23
164-174, 2020.
24 [33] D. Kim, P. H. Chang, and S.-h. Kim, "A new chaotic
25 attractor and its robust function projective
26 synchronization," Nonlinear Dynamics, vol. 73, pp.
27 1883-1893, 2013.
28 [34] A. K. Palmer, F. W. Werner, D. Murphy, and R. Glisson,
29 "Functional wrist motion: a biomechanical study,"
30 Journal of Hand Surgery, vol. 10, pp. 39-46, 1985.
31 [35] K. A. Mann, F. W. Wernere, and A. K. Palmer,
32 "Frequency spectrum analysis of wrist motion for
33 activities of daily living," Journal of Orthopaedic
34 research, vol. 7, pp. 304-306, 1989.
35 [36] R. Song, K.-y. Tong, X. Hu, and L. Li, "Assistive control
36 system using continuous myoelectric signal in robot-
37 aided arm training for patients after stroke," IEEE
38 transactions on neural systems and rehabilitation
39 engineering, vol. 16, pp. 371-379, 2008.
40 [37] A. Smith and E. E. Brown, "Myoelectric control
41 techniques for a rehabilitation robot," Applied Bionics
42 and Biomechanics, vol. 8, pp. 21-37, 2011.
43 [38] P. L. Gribble, L. I. Mullin, N. Cothros, and A. Mattar,
"Role of cocontraction in arm movement accuracy,"
44
Journal of neurophysiology, vol. 89, pp. 2396-2405,
45
2003.
46
[39] L. P. Selen, P. J. Beek, and J. H. Van Dieën, "Can co-
47 activation reduce kinematic variability? A simulation
48 study," Biological cybernetics, vol. 93, pp. 373-381,
49 2005.
50 [40] L. P. Selen, P. J. Beek, and J. H. Van Dieën, "Impedance
51 is modulated to meet accuracy demands during goal-
52 directed arm movements," Experimental Brain
53 Research, vol. 172, pp. 129-138, 2006.
54 [41] S. Hochreiter and J. Schmidhuber, "Bridging long time
55 lags by weight guessing and “Long Short-Term
56 Memory”," Spatiotemporal models in biological and
57 artificial systems, vol. 37, p. 11, 1996.
58
59
60
 Page 12 of 16

1
2
3 Reviewer: 1
4
1. The proposed research is important and relevant to Biomedical Engineering. The presented
5
6 work would have been an excellent endeavor, had the author addressed the following issues.
7
8 Answer: We really appreciate the evaluation.
9
10 2. There is no attempt to link the research to a real life application. For the proposed technique to
11 be a robust one, the model should be able to fit into patients need. The requirement of one patient
12
13
is very different from another patient. So what is required by the robot to assist patient is subject
14 to identification. You can imagine your model is something like,
15 Torque or angle or stiffness = f (patients age, weight, height, fatigue, muscle group, type of
16 contraction, EMG)
17 Then EMG = f (angular position, type of contraction, angular velocity, muscle length, load).
18 So one can imagine hundred different cases of trajectories may be required by the patient. There
19
is a direct or indirect effect of each one of the above arguments of the function in the ultimate
20
21 prediction of joint torque or stiffness or angle. In the absence of any specific parameter whose
22 effect is not considered, the trained model will produce high degree of model deficiency or
23 uncertainty. So author is encouraged to work within a well-defined goal or trajectory rather than
24 vague target.
25
26 Answer: Thank you for the comment. The decoder contains these models in the deep learning
27
algorithms, though it does not have the explicit models. The advantage of use of deep learning is
28
29 that deep learning has numerous parameters that can potentially describe each effect that is
30 mentioned above. The deep learning does not provide explicit relationships regarding the
31 arguments though. In this study, we demonstrated the efficacy of the use of deep learning on
32 estimating joint angle from EMG, without information of the aforementioned arguments.
33
34 3. The paper also deals with patient-robot interaction. In any human-machine interaction control,
35
36
there is interaction between intention of human and intention of the robot. If there is a conflict of
37 intended action between human and robot, then this may lead to an undesirable interaction. To
38 avoid such phenomena happening, robot can be made to mimic human intention which is
39 extracted from muscle EMG (this part is addressed in the paper to some extent). But in case,
40 robot has not learnt to mimic human intention completely, then there will be strong interaction
41 force at the human-robot interaction point. In such cases, robot has to learn new trajectory at that
42
instant online while interacting with the environment. In most robot rehab devices, the patient is
43
44 able to generate a small spectrum of torque or stiffness until the patient improves to switch to the
45 next phase of the rehabilitation with new set of torque-EMG data. The embedded model in the
46 robot to predict joint torque is updated in this new phase of rehabilitation. This is the reason why
47 the author is encouraged to work in the narrow spectrum of torque rather than three distinct
48 torque levels (Soft, Hard and very Hard) which are quite far apart and unnecessary in the
49 perspective of real life application.
50
51
52 Answer: Thank you for the valuable comment. The robot was used to simulate a passive object,
53 with a stiffness of Soft, Hard and very Hard. We selected representative stiffnesses. Our study
54 covers the soft to very hand range of interactions, which may cover the needs of different
55
56
57
58
59
60
Page 13 of 16

1
2
3 patients and different levels of impairment. When needed, we believe the developed approach
4
can be similarly applied to the narrow spectrum of torque.
5
6
7
8 4. Rehab devices can be categorized into mainly two types, one used as assistive devices
9 (prosthetic) where robot will follow the human intention (Extraction from muscle EMG). Second
10 is used in rehabilitation purpose. Even some haptic rehab robot has the element of “assist as
11 needed”. In Either case robot must have the complete knowledge of the environment while
12 interacting which make the training and evaluation of the proposed model a daunting task.
13
14
Rather than experimenting with three far apart EMG signal of soft, hard and very hard stiffness,
15 the author could primarily work in a narrower band of stiffness or torque such as less soft, soft
16 and very soft. In this case, EMG patterns of less soft, soft and very soft will almost look similar
17 and the classification becomes difficult. By using more sophisticated feature extraction of EMG
18 signal these almost similar EMG pattern can still be separated. The point I am trying to make
19 here is that author can make the classification or regression process little more challenging than
20
what it is in current scenario looking towards the perspective of real life application. Also any
21
22 test to the validity of decoder will be relatively more complicated and difficult.
23
24 Answer: Thank you for the comment. Although EMG patterns of less soft, soft and very soft may
25 look similar and the classification becomes difficult, the co-contraction levels of each condition
26 may be different. Signal/system features like frequency components may be different. And the
27 decoder estimates the joint angle and torque differentiating each condition, which means that the
28
classification is made well by the decoder with the machine learning technique.
29
30
31 5. In this paper there is no description of protocol used to the applied trajectory. Once again, the
32 proposed model is trained to predict soft, less hard and hard without any indication of the desired
33 velocity and acceleration at which the participant completed the recording. Until and unless a
34 trajectory is channeled through clear instruction of data recording protocol, robot will only learn
35 a model with some degree of uncertainty. In addition to that, type of electrodes used to collect
36
37
data plays an important role to the quality data. Array electrodes are very effective in this regard.
38 Apparently, this may not seem important. But in reality, if a smooth human-robot interaction is
39 required, then a more complex protocol to trajectory and data recording is required. Otherwise
40 robot will learn something and some motion will be left unlearnt. The author is encouraged to
41 take LSTM to deal with EMG which is rather difficult to do regression (When EMG signals are
42 very similar).
43
44
45 Answer: We appreciate the comment. The applied trajectory was a chaotic trajectory that was
46 irregular with wide bandwidths. The equation of the trajectory has been added in the manuscript
47 (Eq. 7). The objective of the use of this kind of trajectory is to make the decoder learn motions
48 with various velocities and accelerations within a certain period of time. If a regular trajectory is
49 given, decoders usually can predict regular motions only. So we chose an irregular trajectory.
50 We totally agree that a more complex protocol to trajectory and data recording is required.
51
However, in this study, we simulated contacts with a passive object with various stiffness, not a
52
53 learning robot. Also, we considered the minimum number of electrodes to estimate the wrist
54 flexion/extension motion.
55
56
57
58
59
60
 Page 14 of 16

1
2
3 6. In the performance analysis of the LSTM, the author should focus more on how good is the
4
LSTM to predict torque or stiffness or angle from muscle EMG signal when compared with
5
6 actual measured torque or stiffness or angle. ANOVA test is good enough when LSTM
7 performance is compared with RNN and GLM. But I feel cross validation is more useful to
8 verify the performance of LSTM itself across wide range of test data.
9
10 Answer: We really thank you for this comment. As cross validation, we used VAF (Eq. 8) to
11 quantify the difference between the predicted torque and angle and actual measured torque and
12
13
angle, using fresh segments of data not used in building the model.
14
15
16
17 Reviewer: 2
18
19
Comments to the Author
20
21
22 Comment 1: Generally, this paper is well-written, with comprehensive description of the relevant
23 literature and experimental setup. Main contribution of this paper is utilizing LSTM network to
24 estimate kinetics and kinematics during interactions with an environment. Experimental results
25 illustrate that although the training phase signal and inference phase signal are completely
26 different, the proposed methods can estimate the signal with high accuracy.
27
28
29 Answer: We really appreciate your evaluation. The main contribution of this study is to estimate
30 kinetics and kinematics during interactions with an environment using the machine learning
31 technology.
32
33 Comment 2. (Methods) This paper would have been stronger if the paper provides more details
34 of the proposed network architecture as a figure including sequences of input layers -> LSTM ->
35
36
fully connected layer -> dropout -> fully connected layer -> regression layer. Without having
37 decoder architecture figure, it is always difficult to understand proposed network architecture.
38 Current Figure 2 and 3 only explains LSTM blocks.
39
40 Answer: Thank you for pointing it out. We have added the relevant network architecture as the
41 new Fig. 2.
42
43
44
45 2. Time window set as 100ms in Page 4. So, input signal is 100 x 1 vector that each input value
46 is sampled in every 1ms? Or 200 units are utilized to be matched with 200 hidden units in
47 LSTM?
48
49 Answer: Thank you for the good question. All data were collected at the sampling frequency of
50 1kHz. Because data collected during 1 ms are not informative (especially EMG data during 1 ms
51
do not contain enough information), we averaged the data over 100 ms, which provided the
52
53 optimal performance.
54
55 3. (Page 6, Line 30) What 12 coefficients you utilized for AR model of the EMG signals? Does
56 EMG signals originally have 12 coefficients?
57
58
59
60
Page 15 of 16

1
2
3
4
Answer: To extract features of EMG signals, we used the 6th-order autoregressive (AR) model.
5
6 Here is the AR model.
7
8
9
10
11
12 X denotes the averaged EMG signal; c a constant; coefficients of the model, where p
13 is the order of the model. We chose the 6th order AR model for each EMG signal source. We
14 found that selecting the number of coefficients as 6 gives the optimal estimation accuracy. As we
15 had 2 EMG signal sources (FCR and ECR muscles), 12 coefficients were used.
16
17
18 4. (Results) (Page 6, Line 35) What is this F() stands for? What F(2,14) stands for? Please define
19 it explicitly before use it.
20
21 Answer: Thank you for the question. F stands for F-test that can assess the equality of variance.
22 The numbers in the parentheses indicate the numbers of degrees of freedom (df, computed from
23 number of subjects and number of groups). Expression F() is a convention that we typically use
24
25
to express statistical results. Please note that it is not necessary to present its definition.
26
27 5. In Figure 6 (b), what do the bold lines with p<0.001, p<0.05 mean? Similarly, Figure 6 (c).
28 Need to explain more of those lines. These are applied to Figure 9 too. Especially, Figure 9 first
29 row with multiple lines with p-value are difficult to understand what they mean.
30
31
Answer: Thank you for the question. The bold line indicates the comparison between the two
32
33
quantities located below the line’s ends. p<0.05 indicates that there is a statistically significant
34 difference between the corresponding two quantities. p<0.001 indicates stronger statistical
35 difference between two quantities, in comparison to p<0.05.
36
37 6. VAF is one metric to evaluate the proposed method’s inference results in this paper. It would
38 be great to explain what it is with equation.
39
40
41 Answer: Thank you. We have added the definition of VAF.
42
43 7. (Page 7, Line 3) The numbers show here are different from Figure 7's caption. The number in
44 Figure 7's caption is only for subject 3 and this is for averaging up across all subjects?
45
46 Answer: Thank you for the question. Figure 7 presents the results of the representative
47
48
participant (Subject 3) for reference. Thank you
49
50 8. (Page 7, Line 54-55) Figure 8 caption is the same as Figure 7 caption especially VAF
51 numbers. Is it correctly reported?
52
53 Answer: Thank you for pointing it out. The VAF numbers have been corrected in the new Fig. 9.
54
55
56
57
58
59
60
 Page 16 of 16

1
2
3 9. (Page 9, Line 50, second column) When the author used RNN, and GLM to perform
4
comparison, how did the author use it? Simply change LSTM block to RNN block? Or other's
5
6 methods?
7
8 Answer: We really appreciate this comment. For RNN, the LSTM block has been replaced with
9 the RNN block. For GLM, we adopted our previous decoding methodology presented in Liu et
10 al. "EMG-based real-time linear-nonlinear cascade regression decoding of shoulder, elbow, and
11 wrist movements in able-bodied persons and stroke survivors." IEEE Transactions on
12
13
Biomedical Engineering 67.5 (2019): 1272-1281.
14
15 Overall, this paper presents interesting and unique ideas to utilize LSTM network to predict
16 angle, torque, and stiffness. Additionally, the experimental results illustrate that the proposed
17 methods are promising. This paper could be solid if the author addresses the weaknesses
18 mentioned above.
19
20
21 Answer: Thank for the comment.
22
23 Minor Comments:
24 a. Manuscript ID number should be presented on the top
25 b. Page3, Line 11: The time dependency of EMG signals is high: Written two times
26 c. Page 5, Line 26: “input data” should be placed before Figure 4
27
d. Page 5, Line 48: Please use \sigma(.) as a sigmoid function to avoid confusion between \alpha
28
29 and \alpha(.)
30 e. Page 7, below discussions section: Unnecessary large blank area.
31 f. Page10, Line 19: “As observed in Fig 4” -> This seems Fig 5.
32
33 Answer: Thank you so much for the critiques and suggestions. We have responded to all the
34 comments
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60