See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/355348465

REAL-TIME SPEECH TO BRAILLE CONVERTER FOR PEOPLE WITH

AUDITORY AND VISUAL IMPAIRMENTS

Technical Report · July 2021

DOI: 10.13140/RG.2.2.15941.58085

CITATIONS READS

0 16

3 authors, including:

Md. Nasim Afroj Taj Samit Hasan

Bangladesh University of Engineering and Technology Bangladesh University of Engineering and Technology
15 PUBLICATIONS   0 CITATIONS    9 PUBLICATIONS   0 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Hex-Password Based Lock Security System View project

AM Radio Receiver & Transmitter Circuit Design & Build View project

All content following this page was uploaded by Md. Nasim Afroj Taj on 16 October 2021.

The user has requested enhancement of the downloaded file.

EEE 416 – Microprocessor and Embedded Systems Laboratory
Jan 2020 Level-4 Term-I Section A
Final Project Demonstration

REAL-TIME SPEECH TO BRAILLE

CONVERTER FOR PEOPLE WITH AUDITORY AND VISUAL IMPAIRMENTS

SUBMITTED BY – GROUP A2.3

Md. Nasim Afroj Taj Md. Sultan Mahmud Samit Hasan Md. Obaidur Rahman
1606037 1606050 1606056 1606057
Department of Electrical and Electronics Engineering 1
Bangladesh University of Engineering and Technology
 Outline
◦ Summary
◦ Background
◦ Methods
◦ Simulation
◦ Experimental Demonstration
◦ PCB Layout
◦ Photo Gallery
◦ Future Outlook

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 2
 Summary
Our main target was to convert speech to braille. We did it in two significant parts for
both simulation and experimental demonstration.

In simulation, we used a Virtual Serial Port Driver to connect with the Bluetooth module
in our Proteus simulation circuit and send the text input (presented as the decrypted
output of Google’s Speech-to-Text). Then using the input an STM32 chip gives us the
output in LED arrays (presented to pretend as the dot matrix of braille[1]).

In experimental setup, at first, using Google’s Speech-to-Text we converted speech

input into text output and we sent it to an STM32 chip using a Bluetooth module. Then
the STM32 chip drives LED arrays where they act as the dynamic dot matrix of braille
which can be implemented using some piezoelectric devices or motors replacing the
LEDs.

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 3
 Background
Braille is a form of written language for blind people, in which characters are
represented by patterns of raised dots that are felt with the fingertips. It is traditionally
written with embossed paper. Braille users can read computer screens and other
electronic supports using refreshable braille displays. [2, 3]

We are going to use an ARM cortex m series board & Google's API to convert voice
signal (speech) into braille so that a visually as well as auditorily impaired person could
understand conversations in real time.
EEE 416 (2020) – Final Project Group A2.3
 Methods
◦ At first, speech is taken as an input
◦ Then it is analyzed using Google’s Speech to text SPEECH Braille

◦ Then we get the textual output of the input

◦ Using a Bluetooth module it is sent to STM32
Board Google API ARM cortex M Series
◦ Then the STM32 is programmed to convert the
text input into an electric signal that can
operate some mechanical entities.
◦ Then we used the electrical signal to run LED TEXT Bluetooth Module
arrays (which can be replaced with
piezoelectric devices or motors so that they can
be felt through touch) to represent the braille
output.

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 5
 Simulation
Process[4] Demonstration

Explanation Of STM32 Config Proteus Circuit & Demonstration

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 6
 Experimental Demonstration

1.Experimental Setup
2. Explanation of Code
3. Demonstration

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 7
 Photo Gallery (Simulation)
“F”
SETUP

“a”

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 8
 Photo Gallery (Hardware)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 9
 PCB Layout

EEE 416 (2020) – Final Project Group A.XY Title of the Project 10
 Future Outlook
◦ Our plan is to replace the LEDs with either piezoelectric devices or
small motor parts that can be felt with touch for auditorily and
visually impaired persons.
◦ For commercialization we will also build a good case around it for
protection and beautification.
◦ Also we will reduce the latency enough to feel like real time
conversation.
◦ Finally, we plan to use less costly components so that it is
affordable to more people.

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 11
 Conclusion
We had a lot planned for this project. But due to COVID-19 situation, we
had to compensate a lot. Though in the end, we managed to pull
through everything.
At first, we completed our simulation. We learned a lot of new things
along the way. It took some time getting used to STM32Cube IDE, Virtual
Serial Port Driver & their performance together to execute the
functionality in our Proteus circuit. Our setup worked flawlessly for any
input and perfectly showed outputs for them.
We faced more obstacles doing our hardware setup as most of our
members lived outside Dhaka. At first we couldn’t execute it, but finally
we managed to get it to work. But our plan was to use something more
physical but due to lockdown we couldn’t get them for testing. But we
are pleased with our final outcome & hope one day we will manage to
follow through our plans with this project.

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 12
EEE 416 – Microprocessor and Embedded Systems Laboratory
Jan 2020 Level-4 Term-I Section A
Final Project Demonstration

Additional Slides

Department of Electrical and Electronics Engineering 13

Bangladesh University of Engineering and Technology
 References
[1]
https://www.pharmabraille.com/pharmaceutical-braille/the-braill
e-alphabet/?fbclid=IwAR3SS8YF3_kZux5v6bnNMNo-wnpGoh1GB50
j5e2kXFpupc16m87o4xzOChY
[2] https://en.wikipedia.org/wiki/Braille
[3]
https://libbraille.org/how-alphabets-and-numbers-are-written-in-th
e-script/
[4] https://www.st.com/resource/en/data_brief/stm32cubeide.pdf

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 14
 Bill of Materials
No. Component Name Quantity Cost per unit Total Cost
1 STM32 Blue Pill Board 1 400 400
2 USB to Serial Converter 1 200 200
3 HC-05-Bluetooth Module 1 250 250
4 Jumper Wire (M-F, M-M) 2 sets 120 240
5 Bread Board 2 sets 50 100
6 LED 6 5 30
TOTAL 1170

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 15
 Detailed Methods
Software Tutorials:
Proteus STM32 Cube IDE:
https://www.youtube.com/playlist?list=PLNyfXcjhOAw
STM32 Cube IDE
O5HNTKpZPsqBhelLF2rWQx
Easy EDA
https://deepbluembedded.com/stm32-hc-05-blueto
Virtual Serial Port Driver oth-module-examples/?fbclid=IwAR2DEOYd8GEuMa6
Arduino IDE WEQhz04Jdw4QjB6hdxuavV0oCaudUNFTaevzEha8pe
5I
Android Meets Robot
Proteus:
Google Speech to text
https://www.youtube.com/watch?v=MSfemfrm8t4
https://www.youtube.com/watch?v=4clPGiZ8jAc

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 16
 Circuit Diagram

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 17
 Source Codes (Proteus Hex File)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 18
 Source Codes (Proteus Hex File)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 19
 Source Codes (Proteus Hex File)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 20
 Source Codes (Proteus Hex File)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 21
 Source Codes (Arduino code for STM32)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 22
 Source Codes (Arduino code for STM32)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 23
 Source Codes (Arduino code for STM32)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 24
 Source Codes (Arduino code for STM32)

EEE 416 (2020) – Final Project Group A2.3 REAL-TIME SPEECH TO BRAILLE CONVERTER 25
 Difficulties
◦ COVID-19
◦ Lockdowns & Shutdown
◦ Online Medium for Lab
◦ Every Member Being in Different Districts
◦ Most of the Members Being Outside Dhaka
◦ Unavailability of components.

EEE 416 (2020) – Final Project Group A2.3

View publication stats
REAL-TIME SPEECH TO BRAILLE CONVERTER 26