I Can Hear, But
I Cant Understand
Visual Systems for Face-to-Face Communication
V isual systems have been developed
to ensure an effective delivery of
complete information to people whose
hearing loss could significantly impact
on the reception of the information.
Note-Taking
One of the most commonly used
strategies has been the use of note-
takers. However, the ability of note-
takers to effectively capture the infor-
mation can vary greatly. In the business
world, this can vary from effective note-
taking by secretaries who have been
trained in shorthand (almost obsolete
today) to colleagues doing their best
to take notes.
In school settings, note-takers can
be teacher aides or designated fellow
students. The ability of note-takers to
capture all the presented information
is limited. Instructors generally speak
170-220 words per minute, while note-
takers generally write 20-30 words per
minute or up to 50-60 words per
minute if they type the notes. This
might allow for the capture of key con-
tent but it does not allow for enough
information to be captured for real-
time interaction.
Although note-taking is one
of the least costly ways to transcribe
orally presented information to written
text, it is an ineffective way to follow
information in classes, meetings, or
one-to-one exchanges.
Computer-Assisted Note-Taking
This consists of a typist synthesizing
the key information and typing into
a computer connected to an LCD
projector or other monitor. One
28 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter
Have you found yourself in a
situation where you have to
understand information being
presented in a lecture-style
format? Even if you were to hear
most of the information while
using a personal listening or
sound field speaker system, the
effort over a sustained period of
time could lead to fatigue and a
decreased ability to process and
absorb the information.
advantage of this method is that it can
be used for one individual (using a
small desk-type monitor or tablet) or
for many. Another advantage is that it
is not as costly as realtime captioning,
although it only provides just enough
information to stay on topic.
Voice-to-Text Options
Communication Access
Realtime Translation (CART)
CART encompasses realtime caption-
ing and involves the use of intensively
trained stenographers.
Stenographers have
typically been used in
courtroom settings
where their role is to
Communication
Access Realtime
Translation (CART)
is used at all HLAA
Conventions. Right:
Richard Einhorn spoke
at Convention 2014 in
Austin, TX.
By Larry Medwetsky
capture spoken words, although more
recently they have also been used to
provide realtime captioning of the
news, sports and celebrity events.
CART writers use a stenotype
machine to capture verbatim what has
been said. In order to pass the United
States Reporter test, a trained court re-
porter or captioner must type at a speed
approximating 200-225 words per
minute. The stenotype machine consists
of phonetic-based keys which means
that the keys represent the sounds
heard, not the alphanumeric characters
representing those sounds. Multiple
keys can be pressed simultaneously
(initial and final consonant keys as
well as vowels) allowing for sound
combinations, and in turn, allowing
the stenographer to spell out syllables,
whole words, or phrases.
Modern stenotype keyboards
have more in common with computers
in that they contain microprocessors
and are connected to computers with
software that can convert the stenotype
code to English, and stored onto user-
specific dictionaries. That is, CART
captioners must build up their own
vocabularies. This is the reason why
it is beneficial to provide text of a
presentation prior to an event since this
enables the CART provider to enter
special words ahead of time into the
dictionary stored on their computer.
If there is no matching word in the
computer dictionary, the software will
try and make the best phonetic match
possible which often leads to some of
the comical errors that we might see in
realtime captioning of the news, sports
or celebrity events.
The captioned text is conveyed
onto a computer projection panel,
projecting onto a screen or combined
into one unit (merged with other
media, such as a television). Realtime
captioning can be done either with
the CART provider located on-site
or off-site. If the provider is doing
remote captioning, there is an online
connection to the Internet that captures
the transcribed text and interfaces with
the projector panel in the setting where
the presentation is taking place.
However, the CART captioner
must be able to hear the presenters
voice. This is done by having the pre-
senter talk into a microphone, his voice
is transmitted to the on-site computer
and, in turn, transmitted via the
Internet to the captioners computer
whereby he is able to hear what is being
said. The captioner then types what he
has heard via the stenotype machine
which is connected to the computer,
with the transcribed captions being
relayed back to the on-site computer
and projected for everyone to see.
The obvious advantage of
CART is capturing verbatim what has
been said. The disadvantage is that
by involving highly-trained CART
captioners, CART is costly (ranging
from $60 to as high as $350 an hour).
The party responsible for payment
depends on the situation. In some
cases, if used in a school setting the
costs might be covered by Section
504 of the Rehabilitation Act of 1973
or the Individuals with Disabilities
Like HearingLossAssociation on Facebook March/April 2016 29
Dragon Dictationvoice recognition comes to the iPhone, allowing for a voice-to-text message
Education Act (IDEA). In other
situations, the costs might be covered
under the Americans with Disabilities
Act. In addition, because there is not an
abundance of these trained individuals
they might not always be available for
conferences or presentations unless
reserved ahead of time.
Recently, voice-to-text technology
has become refined enough to use with
realtime captioning (e.g., captioned
telephone services). In order to achieve
high accuracy, this presently requires
that the speaker train the software
to recognize his or her voice. Obvious
advantages to this technique are that
the costs are significantly less than
when using stenographers; one is also
not constrained by their availability.
Although this technology has improved
significantly, it still is not as accurate
as when one uses CART providers.
Thus, for professional settings, CART
is preferred.
Text Interpreting
Unlike CART, text interpreting
involves capturing the key content
of what is being presented. This
service is specifically designed for the
educational setting. Text interpreting
is an electronic note-taking system
that is designed to provide meaning-
for-meaning transcription of spoken
words into text, versus the verbatim
transcription provided by CART
stenographers. The text interpreter
listens to the presenter, condenses
the text to derive the key meaning,
then types in the text via a shorthand
method. The student and the text
interpreter usually have their own
laptops when the service is provided
on-site.
There are a number of available
options such as C-Print and TypeWell.
C-Print was developed by the National
Technical Institute for the Deaf
(NTID). In this case, the C-Print
captioner (usually located in the same
class as the student, but can be done
remotely) uses a software application
such as the C-Print Pro, which uses
an abbreviation system based on
phonetics. The C-Print captioner
condenses and summarizes the key
meaning and content in realtime and
types into his computer, which is then
relayed to the students computer.
When a student has a question, he can
type the question which is then relayed
to the C-Print captioner who then
verbalizes the questions to the class.
NTID indicates C-Print is
for students whose preferred mode
is English; who have a hearing loss
significant enough to make it difficult
to follow spoken English in the
classroom; whose reading level allows
for reading the text of the lesson at least
at a fourth grade reading level; and,
who know little or no sign language.
Because training for text interpreting
can be done online and requires only
a few months of training, C-Print
or TypeWell are less expensive than
realtime captioning. The disadvantage
of text interpreting when compared
to CART is that only the meaning is
conveyed, thus, relying on the typists
derivation and interpretation of the key
content. continued on page 30
Face-to-Face Communication
continued from page 29
Automatic Speech Recognition
Development of voice-to-text software
has allowed for significant innova-
tions for use by people with hearing
loss. Examples are Dragon Speech
recognition software, Speechlogger,
Crescendo Speech Processing. Go
to capterra.com/speech-recognition-
software/ for a review of some of
the best speech recognition software
products on the market. These soft-
ware products can be downloaded
onto computers and mobile devices.
Voice-to-text products can be
used in various situations depending
on the specific goal of the software.
One possible application is for those
who wish to avoid typing. In addition,
there are a number of applications that
have been developed for people whose
hearing might not allow them to easily
follow spoken language. Depending on
the specific product, the voice-to-text
software could allow the individual to
engage in one-on-one conversation or
follow a presentation. A remote mic
worn by the presenter can be used
to transmit the speakers voice to an
intermediary device connected to a
computer with voice-to-text software
installed which, in turn, allows the
user to easily follow the presenter. The
accuracy of the transcribed text will
be greatest if the presenter has trained
the software to recognize their voice.
However, in reviewing online a number
of voice-to-text products, a number of
providers have expressed that they can
provide high quality text without much
training.
Here are a few examples:
Speech Assistant a speech
recognition software for enterprises,
allowing for communication between
customers and employees
Dragon Naturally Speaking
used by professionals, students for
dictation and transcription
30 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter
FaceTime, Google
Hangout, and Skype
use a webcam and a
desktop, laptop, table
or smartphone to
see the other party.
This allows for voice,
speechreading and
sign language.
RogerVoice an individual with
hearing loss calls or receives calls
on a smartphone that are instant
transcriptions of what the other party
is saying, regardless of the spoken
language (this software is meant to
work with any voice, thus errors can
occur in the transcription)
Embedded Voice Recognition
programs included with Windows 10,
Google Chrome, Apple Siri, which can
be used to dictate information into
Word software or to control various
computer programs
An important application of
voice-to-text software in recent years
has been in use with Voice Carry Over
(VCO) relay services as provided by
CapTel or CaptionCall. In this case,
people with hearing loss conduct
telephone calls using a relay service.
The person with hearing loss speaks on
the phone to a hearing person at the
other end. What the hearing person
says is not only transmitted directly
but is also relayed to a third party, a
Communication Assistant (CA) via the
Internet. Rather than typing what the
hearing person has said and relaying
it to the individual with hearing loss,
voice recognition software has been
adapted for use to relay the text. Prior
to being able to serve as a CA, the
individual must train the voice-to-text
software to recognize his or her voice
to the point that minimal transcription
errors occur, which takes many hours.
In addition to training the software,
the CA must also be trained to shadow
(repeat) in realtime what he or she
hears. Once the CA has trained the
voice-to-text software and is able to
shadow other talkers at an acceptable
level, he or she is ready to serve as a CA.
Video Face-to-Face
Telephone Services
FaceTime, Google Hangout, and Skype
use a webcam and a desktop, laptop,
tablet or smartphone to see the other
party, thereby allowing for either voice
plus speechreading or for those who
use sign language.
Summary
It is amazing how technological ad-
vances in visual systems have allowed
people with all levels of hearing loss
to overcome obstacles that previously
made communication in meetings,
classes, or on the phone to be so
difficult that they were discouraged
from participating. I hope I have
provided information of use to the
reader. Heres to seeing a brighter
future. HLM
Larry Medwetsky,
Ph.D., associate
professor, can
be contacted
at Gallaudet
University at larry.
medwetsky@
gallaudet.edu.