Virtual Rehabilitation System Using Electromyographic

Sensors for Strengthening Upper Extremities

Andrea Sánchez Z.1, Santiago Alvarez T.2, Roberto Segura F. 2, Tomás Núñez C.3,
Pilar Urrutia-Urrutia1, Franklin Salazar L.1[0000-0002-3404-5202], Santiago Altamirano1 and
Jorge Buele1[0000-0002-7556-0286]
1 Universidad Técnica de Ambato, Ambato 180103, Ecuador

{ap.sanchez, elsapurrutia, fw.salazar, santiagomaltamirano,

jl.buele}@uta.edu.ec
2 Instituto Tecnológico Superior Guayaquil - Ambato, Ambato 180205, Ecuador

{ salvarez, rsegura}@institutos.gob.ec
3 CELEC EP., Baños 180250, Ecuador

leandro.nunez@celec.gob.ec

Abstract. This work presents a virtual system for the rehabilitation of the upper
extremities, using the MYO Smart Band device for the acquisition of electro-
myographic signals produced by the user. Processing and managing of these
signals are done through the SDK provided by the manufacturer of the bracelet
which is compatible with the MATLAB software. The virtual environment is
developed in the Unity 3D graphics engine, in which three-dimensional objects
that were previously designed in the 3DS MAX software are implemented. The
application presents the user with a virtual scenario set in a natural landscape, in
which there is a van that must be driven on a certain path (the complexity is in-
creasing). The videogame is of low complexity, since it seeks to avoid situa-
tions of stress while the rehabilitation process takes place. Each task in the ap-
plication is associated with a hand and forearm movement of the user, it means,
the patient is given an alternative tool that allows him/her to perform exercises
that improve his/her extremity active mobility, mitigating the routine effects of
a conventional session. To validate this proposal, it is tested by 5 retired mili-
tary personnel in passive state, to whom the Using Task Ease (SEQ) usability
test is applied. The result is (58,8 ± 0,27), which shows that this interactive in-
terface has a good acceptance when being in the range between 40 and 65.

Keywords: Rehabilitation, Virtual Reality, Upper Extremities, Electromyo-

graphic Sensor.

1 Introduction

Military confrontations and wars are considered extreme and challenging situations
that a human being must endure [1]. It is subjected to a multitude of psychomotor,
sensory, cognitive and psychological tests that exert great pressure on military per-
sonnel [2]. Some get to lose their senses or develop some type of trauma or disability
2

[3]. Among the main intellectual/cognitive disabilities are learning disorders, general-
ized developmental disorders, attention deficit disorders and behavior [4]. In the sen-
sory field there are visual, auditory, language, speech and voice disabilities [2-3].
Additionally, in the physical aspect, reference is made to motor alterations at osteoar-
ticular level, nervous system, amputations and disability in the extremities [5]. The
most common injuries occur in both the lower and upper extremities [6-7]. In the
upper ones there is ankylosis, joint stiffness, sequelae of fractures or traumatisms and
complete paralysis of fingers or hands. That is why, with the aim of preventing or
mitigating any of these illnesses, a physical therapy process must be initiated [8].
Through a cycle of relearning promote and stimulate muscles and joints to regain their
functionality [9].
The rehabilitation process must be evaluated by a qualified physiotherapist who
provides the patient with the tools to accelerate this process. Previously, routine coor-
dinated physical exercises were developed to produced boredom and fatigue during
the sessions [10]. But this reality has changed when merging these procedures with
advanced technological tools. A clear example of this is the use of compatible devices
with virtual reality (VR) interfaces and auditory accompaniment. Systems involving
VR are an advanced form of human-machine interaction (HCI), which allows the user
to experience extraordinary situations, without the need to move to another site. These
systems achieve a higher level of immersion with the proper use of sensors, devices
and peripherals that enrich the experience. All this stuff, focused on the rehabilitation
processes, gives the patient the opportunity to perform exercises in a different way
and with greater encouragement [8]. Especially when it comes to military personnel
who have been subjected to stress and have certain post-war psychological disorders.

2 State of Art

The implementation of systems that include VR in their architecture for rehabilitation

processes has been gaining relevance in recent years and that is why several related
researches have been developed. A treatment focused on counteracting the incidence
of post-traumatic stress disorder (PTSD) of military personnel using VR is described
in the proposal developed by Achanccaray et al. [11]. A virtual environment inspired
by Iraq/Afghanistan has been developed and includes the dangers found in these plac-
es. This proposal has provided initial reports with positive results, when tested by
active staff and veterans.
Regarding upper extremity treatment, the development of new sensors and devices
allows the development of several options when it comes to rehabilitation. In the prior
research carried out by Levin et al. [12] VR is presented as a tool used in rehabilita-
tion processes of upper extremities. Virtual environments that optimize learning and
motor skills recovery are managed. Based on the respective experimental tests, the
limitations of current technologies are discussed with respect to their effectiveness
and feedback in the process of physical and sensory learning. For its part, Liu et al.
[13] present a rehabilitation system of upper extremities based on electromyographic
sensors (EMG) and portable accelerometers (ACC) for children with cerebral palsy
 3

(CP) is explained. Using the Android platform, games are developed that seek to im-
prove motor function under the guidance of a doctor. The proposal has been validated
by three test subjects who have improved their reaction capacity by interacting with
this motivating and user-friendly interface. Similarly, López et al. [14] presents an
interactive system that uses myoelectric sensors to strengthen upper limbs in children.
Unity 3D software develops interactive interfaces (games) that promote the realiza-
tion of therapies by patients. This prototype is tested by 5 users (3 boys and 2 girls)
with ages between 6 and 12 years where respective System Usability Scale (SUS)
usability test is presented.
As can be seen, there are many researches related to this topic, but not focused on
the military field as required. In this context, this work proposes a virtual rehabilita-
tion system for upper extremities that interacts intuitively with the patient, in this
specific case of military personnel who suffered a combat injury. Data acquisition for
the interface configuration as well as the reading of electromyographic signals are
done using a low-cost electronic device with which the physical gesture that the per-
son is making can be recognized. Three-dimensional objects have been made in 3DS
MAX software and imported into the Unity 3D software, where they are given hierar-
chy and animation. In this context, the aim is to present the patient with an interactive
tool that contributes to a comfortable and accelerated rehabilitation process, mitigat-
ing the post-traumatic stress disorders that they develop after their exposure to stress
situations.
This work is composed of 5 sections, including the introduction in section 1. The
works related to the subject are described in section 2 and the methodology used in
section 3. In section 4 the tests and the results obtained are presented and finally, in
section 5 the conclusions and future work.

3 Methodology

This section details the proposed system and the elements that comprise it. The gen-
eral diagram is described in Fig. 1.

Fig. 1. General diagram of the implemented system.

4

3.1 I/O Device Configuration

The MYO bracelet is used as the input device and the HTC VIVE virtual reality
glasses and binaural headphones as output devices. Next, the respective configuration
of these hardware elements is presented.

MYO Smart Band. To perform the digital processing of acquired signals, this device
is connected to the MATLAB software through the Myo SDK MATLAB MEX
Wrapper library. Through the SDK, information can be accessed provided from the 8
electromyographic sensors and the inertial measurement unit (IMU). Among the most
relevant data are those obtained from the three-axis gyroscope (angular velocity) and
the three-axis accelerometer (linear acceleration).

HTC VIVE. The connection of this device (virtual reality glasses) and the environ-
ment is generated natively in UNITY 3D thanks to the SteamVR plugin. In addition,
reorientation and movement scripts have been made due to the fact that HTC Control-
lers are not used. In this device, the teleportation function is activated by default in
standard mode, which is replaced by signals from the Myo bracelet.

Binaural Headphones. They allow to reproduce the spatial sound that is generated in
the virtual scene and in which the user can spatially identify the origin of the sound.

3.2 Signal Acquisition

The signal acquisition process is performed based on the flow diagram described in
Fig. 2. The Myo Armband device from Thalmic Labs integrates a set of eight non-
invasive sensors, both electromyographic (EMG) and inertial. All sensors collect
information that is processed in a central module (sensor number 4), which includes
batteries and a communication device based on Bluetooth. Through libraries devel-
oped in C++, it has been possible to use it without restrictions letting know the devel-
oper certain parameters such as the measurement of each one of the electromyograph-
ic sensors, processed and unprocessed information of the inertial sensors, transmis-
sion periods, the gesture made by the patient, the values of accelerometers and gyro-
scopes, properties to eliminate the connected device, transmission rate, among others.
Fig. 3 shows a capture of the eight sensor’s signals in a 10 seconds period executing
the open hand gesture. The amplitude units of each of the sensor signals are denoted
here as maximum factory value (MFV), a dimensionless value used by the developers
to represent the minimum and maximum amplitudes (-1 to 1) that the signal can have,
given the internal preprocesses and validators (within the SDK).
 5

Fig. 2. Flow chart of the algorithm used for signal acquisition.

Fig. 3. EMG signal acquisition in 10 seconds of the gesture: open hand.

3.3 Interface Development

The interface design is made based on the flowchart of the interaction of programs,
which also includes the configuration in MATLAB and the database presented in Fig.
4. Using Unity 3D software, a virtual environment has been developed, in which ob-
jects that simulate an open field with a road across it are placed. The aim in this appli-
cation is to drive a van-type vehicle and completing a trajectory that is previously
6

defined by itself. Tasks have been carried out in a simple way so as not to produce a
stressful situation in patients, but instead they are entertained while performing their
rehabilitation exercises. Script programming allows to identify 5 gestures that patients
can perform: (i) fist, (ii) open hand, (iii) palm inward, (iv) palm outward and (v) join
thumb and middle fingers.
The three-dimensional objects design has been made in 3DS MAX software. Once
all the necessary objects have been modeled in 3D, their compatibility is checked, the
rotation points are located, hierarchies are established and the axes of each element
are oriented. After that, models are imported from 3DS MAX using a format with
extension *.fbx. Generally, the model of the imported objects is in gray color, or it
can have certain properties of previously created materials but commonly the colors
and textures are eliminated. In this way, within the Unity 3D videogame engine, col-
ors and textures are assigned to the 3D model, depending on the material and graphic
quality required. In addition, the different tools offered by the palette of this software
are used to incorporate the required grass and road, as shown in Fig. 5. Additionally,
the respective environmental sounds are coupled, which complement the immersive
experience of the patient who can feel how the car drives in the middle of nature.

Fig. 4. Flowchart of the interaction of programs.

 7

Fig. 5. Design of the interface in Unity 3D software.

4 Tests and results

4.1 Test

When executing the application, the patient must perform various gestures with his
hand and forearm in order to perform the tasks entrusted, as shown in Fig. 6a and Fig.
6b. When the fist gesture is made, the vehicle is turned on and you can drive on the
road. Once you are in driving mode, moving the palm of the hand outward produces a
turn to the right, depending on the intensity of the gesture, the rotation is performed.
The movement of the palm of the hand inward indicates that it is going to turn to the
left. When you open your hand, it indicates that you have reached the desired point
and so the vehicle must stop. In case that a bad maneuver has been made, the reverse
option is incorporated by joining the thumb and the middle finger. Patients performed
the entrusted exercises in 30 minutes’ sessions, 2 times a week. This treatment lasted
4 weeks and was carried out by 5 retired soldiers in a passive state (it must be differ-
entiated that 2 of them required rehabilitation in both arms), with an age range be-
tween 48 and 63 years. The supervision of the therapist is very important, since it
must corroborate that patient performs the exercises correctly so that there is an im-
provement in the treatment.
8

a)

b)
Fig. 6. a) Tests performed by a patient. b) Virtual environment seen by the patient.

4.2 Results

At the end of the application, each patient data and the values obtained in the video-
game are saved in a text file that is automatically generated with the user's name. To
determine the level of acceptance of this proposal, the usability test SEQ, has 14 ques-
tions, of which 13 have the values of 1 to 5 points according to the following scheme:
The first seven questions (Q1-Q7) are related to the level of acceptance and immer-
sion after the user experienced the virtual environment. The following four questions
(Q8-Q11) are related to the effects and discomfort that the system could cause: nau-
sea, disorientation or ocular discomfort. The next two questions are related to the
difficulty encountered when performing the tests. The last question is open, so the
user can indicate if there is any discomfort when using the virtual system and its rea-
sons. If the result obtained is in the range of 40 to 65, the implemented system is con-
sidered acceptable. The questions asked to the users about the virtual system and the
results of the SEQ usability questionnaire are shown in Table 1.
The results of the SEQ test performed by 5 users after using the virtual system are:
(58,8. ± 0,27). The score obtained is greater than 40. Therefore, it is determined that
patients feel fully comfortable with the developed interface, so it represents an ideal
tool to contribute to the rehabilitation process which requires an extended period of
application.
 9

Table 1. Results of the SEQ test applied to users.

Result (N = 5)
Question
Mean SD
1. How much did you enjoy your experience with the system? 4,8 0,43
2. How much did you sense to be in the environment of the system? 4,8 0,43
3. How successful were you in the system? 4,4 0,83
4. To what extent were you able to control the system? 3,2 0,71
5. How real is the virtual environment of the system? 5 0
6. Is the information provided by the system clear? 4,8 0,43
7. Did you feel discomfort during your experience with the system? 4 0,43
8. Did you experience dizziness or nausea during your practice with the
4,8 0,43
system?
9. Did you experience eye discomfort during your practice with system? 4,6 0,87
10. Did you feel confused or disoriented during your experience with the
5 0
system?
11. Do you think that this system will be helpful for your rehabilitation? 4,2 0,71
12. Did you find the task difficult? 4,8 0.43
13. Did you find the devices of the system difficult to use? 4,4 0,43
Global score (total) 58,8 0,27

5 Conclusions

The experimental results after the implementation of this system show that it is a valid
proposal for rehabilitation of upper extremities. With this, it is pretended to give the
patient a new tool to replace the conventional method; immersing his/her in a virtual
world (fusing the sense of sight and hearing) a wide range of possibilities for physical
rehabilitation is presented. An important fact to note is when dealing with users who
were in active service and were subjected to stress situations, the execution of simple
tasks and the simulation of an environment with nature, help them to mitigate possible
side effects of a psychological nature. The application of the usability test SEQ shows
that the interface presented to the user is intuitive, easy to use and does not cause
discomfort. It should be pointed out that this research has been evaluated from the
point of view of the acceptance generated by this system, but not with a medical ap-
proach, given that the results are not conclusive when requiring a longer evaluation
period.
In this context, the authors of this research propose as future work the execution of
this application in civil patients, in order to make a comparison and determine factors
of incidence. In addition, the development of a system that allows the rehabilitation of
the lower limbs is proposed, taking advantage of the already established design. Final-
ly, these authors propose the use of a control algorithm to expand the number of ges-
tures presented in this work and thus allow a better recovery process.
10

Acknowledgments. To the authorities of Universidad Técnica de Ambato (UTA),

Dirección de Investigación y Desarrollo (DIDE), Instituto Tecnológico Superior
Guayaquil - Ambato and CELEC EP., for supporting this work and future research.

References
1. Drakos, N. D., et al.: In good conscience: developing and sustaining military combat trau-
ma expertise. Journal of the American College of Surgeons 227(2), 293-294 (2018).
2. Russell, C. A., Gibbons, S. W., Abraham, P. A., Howe, E. R., Deuster, P., Russell, D. W.:
Narrative approach in understanding the drivers for resilience of military combat medics.
Journal of the Royal Army Medical Corps 164(3), 155-159 (2018).
3. Wells, T. S., Seelig, A. D., Ryan, M. A., Jones, J. M., Hooper, T. I., Jacobson, I. G., Boy-
ko, E. J.: Hearing loss associated with US military combat deployment. Noise & health
17(74), 34-42 (2015).
4. Writer, B. W., Meyer, E. G., Schillerstrom, J. E.: Prazosin for military combat-related
PTSD nightmares: a critical review. The Journal of neuropsychiatry and clinical neurosci-
ences 26(1), 24-33 (2014).
5. Stinner, D. J.: Improving outcomes following extremity trauma: the need for a multidisci-
plinary approach. Military medicine 181(4), 26-29 (2016).
6. Wilken, J. M., Roy, C. W., Shaffer, S. W., Patzkowski, J. C., Blanck, R. V., Owens, J. G.,
Hsu, J. R.: Physical Performance Limitations After Severe Lower Extremity Trauma in
Military Service Members. Journal of orthopaedic trauma 32(4), 183-189 (2018).
7. Galarza E.E. et al. Virtual Reality System for Children Lower Limb Strengthening with the
Use of Electromyographic Sensors. In: International Symposium on Visual Computing,
ISVC 2018, pp 215-225. Springer (2018).
8. Gelman, D., Eisenkraft, A., Chanishvili, N., Nachman, D., Glazer, S. C., Hazan, R.: The
history and promising future of phage therapy in the military service. Journal of Trauma
and Acute Care Surgery 85(1), 18-26 (2018).
9. Cools A., Whiteley R., Kaczmarek P.K.: Rehabilitation of Upper Extremity Injuries in the
Handball Player. Handball Sports Medicine, 433-459, Springer Germany (2018).
10. Quevedo W.X. et al.: Assistance System for Rehabilitation and Valuation of Motor Skills.
In: International Conference on Augmented Reality, Virtual Reality and Computer
Graphics, AVR 2017, pp 166-174. Springer (2017).
11. Achanccaray, D., Acuña, K., Carranza, E., Andreu-Perez, J.: A virtual reality and brain
computer interface system for upper limb rehabilitation of post stroke patients. In: 2017
IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1-5. IEEE (2017).
12. Levin, M. F., Weiss, P. L., Keshner, E. A.: Emergence of virtual reality as a tool for upper
limb rehabilitation: incorporation of motor control and motor learning principles. Physical
therapy 95(3), 415-425 (2015).
13. Liu, L., Chen, X., Lu, Z., Cao, S., Wu, D., Zhang, X.: Development of an EMG-ACC-
based upper limb rehabilitation training system. IEEE Transactions on Neural Systems and
Rehabilitation Engineering 25(3), 244-253 (2017).
14. López, V. M., Zambrano, P. A., Pilatasig, M., Silva, F. M.: Interactive System Using My-
oelectric Muscle Sensors for the Strengthening Upper Limbs in Children. In: International
Conference on Augmented Reality, Virtual Reality and Computer Graphics, AVR 2018,
pp 18-29. Springer (2018).