See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281620705

Sign to Letter Translator System using a Hand Glove

Conference Paper · April 2014

DOI: 10.1109/ICeND.2014.6991369

CITATIONS READS
12 3,425

5 authors, including:

Abdallah Kassem M. Hamad

Notre Dame University Notre Dame University
75 PUBLICATIONS   341 CITATIONS    18 PUBLICATIONS   37 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

FYP Student projects View project

Small-signal analysis and modeling of ultra-high-speed VCSELs View project

All content following this page was uploaded by Abdallah Kassem on 09 September 2015.

The user has requested enhancement of the downloaded file.

 Sign to Letter Translator System using a Hand
Glove
Hanine El Hayek and Jessica Nacouzi,
Department of Electrical, Computer and Communication
Engineering
Notre Dame University-Louaize,
P.O.Box: 72 Zouk Mosbeh,
Zouk Mikayel, Lebanon
{Haninehayek ; jessica.nacouzi}@hotmail.com
Abdallah Kassem, Mustapha Hamad and Sami El-Murr
Faculty of Engineering Department of Electrical,
Computer and Communication Engineering
Notre Dame University-Louaize,
P.O.Box: 72 Zouk Mosbeh,
Zouk Mikayel, Lebanon
{akassem; mhamad; selmurr}@ndu.edu.lb

Abstract— A sign to letter translator system using a hand
glove is proposed in this paper. The glove can be used by any
deaf and/or mute person to communicate with people that do not
understand sign language. The performed signs, with the hand
wearing the glove, are displayed as alphabetic letters on the LCD.
This glove can be very useful, efficient, and helpful for solving
the communication issues that arise between the people suffering
from hard of hearing and the people that do not. It consists of
flex resistors that sense the change in gestures, a microcontroller
that gathers, processes, and sends data to the LCD which shows
the respective resulting letter. The Sign to Letter Translator is
designed to facilitate the communication between the transmitter
and the receiver so that the meaning of the message is properly
reached and completely understood.
Keywords—Communication; Sign to Letter Translator;
Electronic Hand Glove; Flex resistors; Microcontroller; LCD
gloves into letters displayed on the LCD or computer screen to
allowing users to transmit their intended message in the
language other people know [4, 5, 6].
The concept of the Sign to Letter Translator (S2L) is
introduced in section II. In section III, related work is
presented. The hardware of the proposed glove is described in
section IV. The system implementation is described in section
V, followed by future plans and a brief conclusion in section V.
II. OVERVIEW OF THE SIGN TO LETTER TRANSLATOR
DESIGN
Figure 1 shows the proposed system for the Sign to Letter
Translator (S2L). The system consists of a glove holding flex
sensors, discrete components, a microcontroller, and an LCD.

I. INTRODUCTION
Communication is the activity of conveying information
through the exchange of thoughts, messages, or information as
by speech, visuals, signals, writing, or behavior.
“Communication is easily overlooked, but the ability to
communicate effectively is necessary to carry out the thoughts
and visions of an organization to the people. The importance of
words, through a paper or a speech voice in a communication
medium, is to convey directions and to provide
synchronization” [1]. According to the Word Federation of the
Deaf, there are around 72 million deaf, mute, or deaf-mute
people in the world today [2]. And while many of them can
Fig. 1. S2L Proposed System
communicate with each other through sign language, there is a
linguistic wall between them and people who can speak and The glove is lined with six flex sensors, thin strips that
write but do not know signs. Furthermore, most people
detect changes in resistance indicating when a finger is bent.
suffering from hearing impairment prefer sign language [3].
Sometimes some sounds cannot be differentiated from the lip- On the back of the hand lies a micro-controller circuit, the
reading for instance “ice cream” and “I scream”, “hear” and heart of the system that analyzes all these incoming signals
“here”, “bo” and “po”, etc. Moreover, another example is the and transmits them to a Mini-LCD in order to display the
word “lie”, this word has more than one meanings and could resulting letters.
easily be misinterpreted. Therefore, the use of sign language in Because the glove should be placed in the hand of the deaf
the life of deaf and/or mute people becomes essential. New person; the design should fulfill the following requirements:
electronics technology can serve to enhance their lifestyle.
x The system provides accurate data collection and data
Sign to Letter Translator is an electronic device to help deaf
output.
people communicate with others who do not have knowledge
x The system is portable.
in sign language [3]. It transforms the hand gestures using

ISBN: 978-1-4799-3166-8 ©2014 IEEE 146

 x The system is easy to install. training, and a computer including a camera with high
x The system is easy to use. resolution.
x The system is safe to use.
IV. HARDWARE IMPLEMENTATION OF THE
x The system is durable.
PROPOSED SYSTEM
III. RELATED WORK The block diagram of the proposed (S2L) system is shown
M. Borghetti et al [7] presented a system that depicts the in figure 2. It is mainly composed of the flex resistors, a 9-
fingers’ positions of one hand in order to monitor the Volts power supply, and a hardware circuit. The inputs are the
rehabilitation movements of the hand. This device consists of signs from the flex sensors that are connected to the hardware
a glove that has flex sensors on the fingers connected to an circuit and the power supply. The hardware circuit includes a
electronic unit. However, to perform the bending tests of the microcontroller, discrete components, and an LCD.
fingers, only one flex resistor was measured at a time knowing
that there are ten flex sensors, two on each finger, of the glove
leading to a long working time that is why low power
consumption was a necessity. The system requires a huge
hardware, in addition to the instrumented glove holding the
ten sensors and the circuit unit that contains the
microcontroller, the battery, the transmitting antenna, the
connectors, the transceiver, a computer, a receiver circuit, a
DAQ card, and an optical system are needed to perform the
experiment.
Arabic sign language recognition technique using glove and
support vector machines is presented in [8]. This technique
system still requires a computer to analyze the fingers’
positions and to display the letter.
Fig. 2. Block diagram of the proposed S2L
A relevant paper introduced a new system designed to
enable speechless people to communicate with the world using The flex resistors illustrated in Figure 3 are essential
an electronic glove and a screen [9]. It presented a system that elements that play the role of getting the signs which are the
enables letter input matrix system, meaning a faster letter inputs from the user while sensing the change in the amount of
keying based on photodiodes and photo-detectors in addition bend applied on the flex [11, 12]. They convert the change in
to the study of the distance between joints. A predetermined bend to electrical resistance - the more the bend, the more the
chip is used for subsequent digital signal processing; the resistance value. They are used in a voltage divider circuit to
meaningful information will show up on a screen for new supply the data (analog signal) to the microcontroller after
responses. The logic of the system lies on the bended angel of digitized into 10 bits of precision to get processed.
the finger; It used the intensity of the LEDs that is evaluated in
order to reflect the letter needed "the voltage can be used to
represent the degree of finger bending". The detected light
was further digitized into 8-bit signals via using the 0-255
grey scale. The letters are picked from a virtual keyboard and
the thumb is used to confirm the letter. This implies the need
of a wide screen that displays a full keyboard and a text input
row. The tests were made on photo-detectors of 4.5 and 6 cm
of diameter. This means that this system needs large photo-
detectors which might be unpractical in usage. Moreover the
system has to re-programmed (the software has to be altered)
for each case in order to meet the user disabilities.
Other techniques used of the recognition and translation of
the alphabets especially in the Arabic sign language were Fig. 3. Flex Sensors
developed using five phases; Pre-processing phase, Best-
frame Detection phase, Category Detection phase, Feature The formulas used to get the voltages and the 10-bits are
Extraction phase, and finally Classification phase. This respectively shown in equation (1) and (2).
technique used the images of bare hands that can translate the ୖ
movement to Arabic Letters using minimum distance  ൌ ൈͷ (1)
ሺୖାହሻ
classifier (MDC) and multilayer perception (MLP) neural
networks [10]. This system requires a huge software program, Where R is the measured resistance.

ISBN: 978-1-4799-3166-8 ©2014 IEEE 147

 ଵ଴ଶସ
 ൌ ‡…‹ƒŽ˜ƒŽ—‡ሺͳͲ„‹– െ ‡“—‹˜ƒŽ‡–ሻ ൌ  ൈ ሺ ሻ (2)
ହ

The ranges taken were approximately close for the different

fingers and same position. The different ranges were entitled
as follows: OPEN, MID1, MID2 and CLOSE. Table I
illustrates the different values for each flex.

TABLE I. FLEX RANGES ANGLES FOR DIFFERENT POSITIONS

OPEN (00) MID1 (01) MID2 (10) CLOSE (11)

Flex

F1 X<670 669< X<715 714<X<738 X>737

F2 X<675 674<X<760 759<X<808 X>807
F3 X<662 661<X<700 699<X<740 X>739
F4 X<740 739<X<795 794<X<840 X>839
F5 X<700 699<X<750 749<X<790 X>789
F6 X>600 X<600
Where X is the digitized input voltages for flexes F1 till
F5 connected to 5 fingers shown in figure 1, and flex F6
placed on the wrist to detect its movement(if it is straight
or bent).
Based on the data of table I and the list of signs of each letter,
Table II was filled with zeros and ones corresponding to each
letter for the five flex sensors.
Fig. 4. S2L Hardware System (a): Side view, (b): Back view
TABLE II. BINARY OF EACH LETTER FOR EACH FLEX
SENSOR Looking inside the Sign to Letter Translator hardware system
shown in figure 4, the microcontroller is the central unit. It is
F1 F2 F3 F4 F5
operated with a clock of 4MHz to provide enough data to its
A 00 10 10 10 10
connected components and meanwhile to receive accurate
B 00 00 00 00 00
data. It has many special features from which are a Central
C 01 01 01 01 01
Processing Unit and a 10-bit, 8-channel A/D Converter. In
D 01 00 01 01 01
addition to the microcontroller, the PCB contains discrete
E 10 10 10 10 10
components, an oscillator, and a voltage regulator that
F 01 01 00 00 00
converts the 9 Volts supplied by the battery to 5 Volts needed
G 00 01 10 10 10
to operate the microcontroller circuit.
H 10 00 00 10 10
I 01 10 10 10 00 Figure 5 shows the whole system including the glove.
J 00 10 10 10 00
K 00 00 00 10 10
L 00 00 10 10 10
M 10 01 01 01 10
N 10 01 01 10 10
O 10 01 01 01 01
P 00 00 00 11 11
Q 00 00 11 11 11
R 01 01 00 10 10
S 10 11 11 11 11
T 01 01 10 10 10
U 01 00 00 10 10
V 01 00 00 11 11
W 01 00 00 00 01
X 10 01 11 11 11
Y 00 10 10 10 00
Z 10 00 11 11 11
Fig. 5. S2L system including Glove

ISBN: 978-1-4799-3166-8 ©2014 IEEE 148

 V. SYSTEM IMPLEMENTATION
After building the hardware, the software is developed to
display the output letters corresponding to the input
information coming from the sensors. The flowchart presented
in Figure 6 demonstrates the steps in the program. In the
initialization phase, the code tables were defined. The program
starts by reading the analog inputs of origin the flex sensors
(F1 ÆF6) and reserving two bit positions for each flex value.
The code is built by concatenating the binary representations
of all flexes (F1 till F2) as shown in figure 7. Next, the code
undergoes some if conditions then calls a function in order to
reach the final output, the adequate letter after which the few
last steps will be repeated through a loop to get the new data.

Fig. 7. Bits representation of letter A

The user wearing the glove can use sign language simply by
moving the appropriate fingers of his/her hand. Since each
flex sensor has a range of resistances, each sensor was tested
apart and its voltage values were taken for the twenty-six
letters. While testing, the values altered, so the problem was
solved by taking different ranges for each flex position. Figure
8 below shows the sign representation for letter “A” and Table
III illustrates the different resistances, voltages tested for the
six flex sensors for this letter, 10-bit representation, finger
positions (either open, mid1, mid2, or close), and the binary
representation for these positions.

Fig. 8. Sign of Letter “A

TABLE III. EXAMPLE OF LETTER “A” VALUES

Resistance Voltage Decimal Finger Binary
(kΩ) (V) 10-bits position
F1 9.9 3.3221 680.38 Mid1 01
F2 15.3 3.7685 771.78 mid2 10
F3 13.8 3.6702 751.66 mid2 10
F4 10.9 3.428 701.99 mid2 10
F5 14.5 3.718 761.44 mid2 10
F6 8.57 3.158 646.7 open 00

This system achieved an average accuracy of 94% when

trying to display all letters. Figure 9 shows the average
accuracy values for each letter.
Fig. 6. Flowchart of the S2L program

ISBN: 978-1-4799-3166-8 ©2014 IEEE 149


Fig. 9. Accuracy of S2L System
VI. CONCLUSION AND FUTURE PLANS
The Sign to Letter Translator system (hardware and
software) is described in this paper. The objective of this
system is to detect any change regarding the gesture of the
hand which provides sufficient data or information to allow
the system to translate the sign into letter. This system was
found to be accurate, reliable, and affordable enough which
gives the chance for every person suffering from hearing
impairment to use it anytime he or she needs to communicate
with other people. Systems, similar to this design which
employs advanced communication technologies and sensors,
open new horizons for improving our lifestyles.
Future plans are being set in order to enhance the proposed
system. First, higher quality of flex sensors or gyroscopes
and/or accelerometers will be implemented in this system in
order to improve its performance and reduce errors. Second,
the system will become wireless to transmit data to a smart
phone, this should eliminate the use of an LCD, and will take
advantage of the text-to-voice applications available in smart
phones.
ISBN: 978-1-4799-3166-8 ©2014 IEEE
View publication stats
ACKNOWLEDGMENT
The authors would like to acknowledge the valuable
assistance of the ECCE laboratory staff.
REFERENCES
[1] Kam, David, "The Importance of Communication," Marketing Deviant -
Marketing Business Strategies, June, 2009, http://marketingdeviant.com/
the-importance-of-communication
[2] World Federation of the Deaf (WFD), http://wfdeaf.org
[3] Su-Jing Wang; De-Cai Zhang; Cheng-Cheng Jia; Na Zhang; Chun-
Guang Zhou; Li-Biao Zhang, "A Sign Language Recognition Based on
Tensor," Second International Conference on Multimedia and
Information Technology (MMIT), vol.2, pp.192-195, April 2010
[4] Allen, J.M.; Asselin, P.K.; Foulds, R., "American Sign Language finger
spelling recognition system," 2003 IEEE 29th Annual Proceedings of
Bioengineering Conference, pp.285-286, 2003
[5] Khambaty, Y.; Quintana, R.; Shadaram, M.; Nehal, S.; Virk, M.A.;
Ahmed, W.; Ahmedani, G., "Cost effective portable system for sign
language gesture recognition," IEEE International Conference on
System of Systems Engineering, 2008. SoSE '08, pp.1-6, June 2008
[6] Yu-Fen Fu; Cheng-Seen Ho, "Development of a Programmable Digital
Glove," International Conference on Machine Learning and Cybernetics,
2007 , vol.4, pp.1948-1953, Aug. 2007
[7] Borghetti, M.; Sardini, E.; Serpelloni, M., "Sensorized Glove for
Measuring Hand Finger Flexion for Rehabilitation Purposes," IEEE
Transactions on Instrumentation and Measurement, vol.62, no.12, pp.
3308-3314, 2013
[8] Mohandes, Halawani and Al-Baiyat, Okaka, "Arabic Sign Language
Recognition Using An Instrumented Glove And Support Vector
Machines" Prince Salman Center for Disability Research, 7/2003-7/2006
[9] Yu-Fen Fu; Cheng-Seen Ho, "Development of a Programmable Digital
Glove," International Conference on Machine Learning and Cybernetics,
2007 , vol.4, no., pp. 1948-1953, 2007
[10] Nashwa El-Bendary et al., “ArSLAT: Arabic Sign Language Alphabets
Translator”, International Journal of Computer Information Systems and
Industrial Management Applications, vol. 3, pp. 498-506, 2011
[11] Rebecca Bray, Claire Mitchell and Melissa Holtz, "Flex Sensors,"
Spring 2006, Spring 2012,
http://itp.nyu.edu/physcomp/sensors/Reports/Flex/
[12] Saggio, G., "Bend sensor arrays for hand movement tracking in
biomedical systems," 4th IEEE International Workshop on Advances in
Sensors and Interfaces (IWASI), 2011, pp.51,54, 28-29 June 2011
150