Design, Simulation, and Implementation of

connected IoT for Disabled People

Hazem Hesham1, Loujayne Khalil2, Abdelrahman Hafez3, and Mohamed Hussein4
1,2,3
Department of Electronics and Communication Engineering Misr International University, Cairo, Egypt
4
Military Technical Collage, Cairo, Egypt

Abstract: Taking care of the elderly is difficult,
especially when they have chronic diseases. It's
difficult to focus and address the challenges
experienced by differently-abled persons, especially
if our patient is visually, audibly, or vocally
disorderly. The project's goal is to design and
implement a wearable device solution that is simple,
quick, accurate, and cost-efficient; through the
benefit of the Internet of Things (IoT) concept with
a mobile application to facilitate the patient's
caregivers. In this project, there are two cases,
wherein in both cases, the patients are diagnosed
with Alzheimer's disease, where they will be
wearing a health band to detect and record their
health vitals and allocate their position. The first one
is that the Alzheimer’s patient is audibly and vocally
impaired. There is an implemented program within
the wearable band and the mobile application to
detect the voice and convert speech-to-text (SST)
and convert text-to-speech, in which the speaker
outputs the speech. In the second case, the patient
has lost the sense of sight. There are two solutions,
face recognition using a camera and the third eye
using an ultrasonic sensor.
Keywords: IoT, Atmega Microcontroller,
Healthcare System, wearable Device, Alzheimer,
Deaf-Dumb-Blind people, speech recognition, face
recognition, third eye, mobile application.
1. INTRODUCTION
The technologies that support the Internet of
Things are making significant progress in major IoT
technical sectors, becoming more accessible,
available, and diverse every day, enabling quicker
expansion of the things connected via the Internet.
These technological advancements provide human
and machine data communication capabilities,
opening a new universe of diverse ways to
communicate. Businesses and institutions use the
underlying IoT technology to improve their internal
processes and services. IoT technologies are
growing into new application fields as we move
closer to a fully connected world. Technical
improvements for people with special needs, for
example, can help them with their daily activities
and thereby improve their quality of life. This may
be accomplished with the help of well-adapted and
unique IoT technology for this type of person;
nevertheless, while we can anticipate these scenarios
soon, their execution today constitutes a significant
technological leap. From a more technical
standpoint, we can state that what is required is:
1) The data acquisition system senses changes
in the physical status of things; that is
subsequently saved and communicated.
2) Embedded systems provide the potential
for things to communicate and interact to
achieve a primary goal for a user
application, along with nanotechnology, to
communicate and interact to achieve a
primary goal for a user application.
3) Use a cost-effective method such as RFID
technology and face recognition to link
ordinary objects and technologies to the
Internet.
4) Create software that hides heterogeneity,
such as hardware and operating systems,
from the application layer.
5) A series of data analyses, big data, and
semantic-based enablers enables obtaining
information from enormous datasets.
6) Finally, applications that connect people to
things.
2. LITERATURE
2.1 Disabled people
Disabled persons used to have a lot of difficulties
going about their daily lives and
accomplishing anything. Over one billion people are
believed to live with a disability, which equates to
around 15% of the world's population. Year after
year, the number of people living with impairments
rises, also, as the rise of chronic ailments. As a
result, a special needs person's family is torn
between leaving them alone or going out alone. [1]
2.2 Alzheimer’s disease
Alzheimer's disease AD is the most frequent
cause of dementia and a degenerative brain illness
[2]. According to the World Health Organization,
the number of persons with dementia is expected to
quadruple in the next 30 years [3]. Around 50
million individuals are estimated to be living with
dementia at this time. WHO estimates that by 2050,
the number will have risen to 152 million. [4].
Furthermore, one new case of Alzheimer's develops
every 33 seconds, resulting in 1 million new cases
every year [4]. According to several pieces of
research, the inability to identify familiar faces has
generally been referred to as the underlying memory
impairment in Alzheimer's disease. Deficits in both
anterograde episodic memory of faces and
retrograde semantic memory of famous people are
both present in Alzheimer's disease, it is assumed to
be the cause of difficulty recognizing recognizable
faces [5] [6].
2.3 Deaf and Dumb
Hearing loss affects almost 5% of the world's
population or 466 million individuals [7]. Hearing
loss that is disabling is defined as a loss of more than
40 decibels (dB) in the better hearing ear in adults
and more than 30 dB in the better hearing ear in
children [8]. Deaf persons might communicate using
sign language. However, this approach is inefficient
because the abled person does not need to master
sign language to communicate effectively. Another
disadvantage of adopting sign language is that both
parties must be visible. The drawback of ineffective
communication between deaf and hearing people is
addressed by our suggested solution.
Microcontroller, voice recognition module, GSM
Module, and Cloud Server form the basis of our
system. Our system's mobility is improved by using
the AVR Atmega.
2.4 Blind
According to the World Health Organization
study, there are 285 million people with vision
impairment, 39 million of them are blind, and 246
million people have impaired vision [7]. With the
progress of contemporary technology, there is a
range of solutions for the mobility of visually
impaired and blind people that is accessible.
Electronic Travel Aids (ETAs) is the name given to
these devices (ETAs). The ETAs contain sensors
that warn the blind of impending danger through
noises or sensations. The development of Electronic
Traveling Aids has increased the safety and self-
confidence of the blind [8] [9].
3. Previous works
• The Raspberry Pi serves as the device's
support system, connecting the camera,
microphone, speaker, and LCD display.
The device is useful for blind people
because the camera takes a picture of the
document and the output is in audio format
through the speaker, the audibly impaired
because the microphone takes the spoken
words as input and displays them as text on
the LCD display, and the vocally impaired
because the user types the message in the
LCD and the speakers output it as audio
[13].
• Smart home control systems use wireless
modems and the internet to allow humans
to operate their home electrical items such
as televisions, lights, and fans using simple
voice commands. This system will allow
people with disabilities to turn on and off
electrical equipment without moving [14].
• For handheld GPRS modules that require
data transmission and data processing
multitasking, an embedded design is used.
AVR is used to retrieve sensor data and
wirelessly transfer it to handheld embedded
devices or specially built blind mobile
phones [15].
4. DESIGN
4.1. Proposed System
We must go through a series of steps to reach our
desired system.
Cloud
Admin
Pharma
Doctor
Database
Nurse
Figure 2. Proposed system design
A wearable device, a linked app running on the
user's smartphone, and a cloud server service are all
part of the usual system architecture. Figure 1 shows
an example of a wearable system architecture, in
which the architecture can be both simpler and more
complicated than the one displayed. Some advanced
navigators used by mountain hikers, for example,
link directly to a GPS, eliminating the need for a
smartphone. On the other hand, a physical activity
tracker might provide data to an insurance company
to improve health problem prediction and
personalize insurance rates for a specific consumer.
The functionality is divided over many devices with
varying capacities, which influences the design. It
is critical to examine the usability of individual
devices and their "interusability" [10]. That means
the creation of a usable experience across several
devices. In addition, these designs have a physical
space distribution of system nodes. As a result, both
human and technical limitations exist. The context
must be carefully examined during the design phase
to optimize the user's engagement. Consider the
wireless network visibility range as an example of
the first sort of constraint. To optimize the
interaction with a user, an example of the second
type of restriction must carefully consider the
context of the design.
4.2. System Architecture
The wearable gadget is intended for health
Wi-Fi/GPS/GSM
Speaker Camera
AVR
Micro-
phone Atmega LCD
Vibration
Sensor Ultrasonic
sensor
HR/BP/Temperatu
re sensors
Figure 1. System Architecture
monitoring and assisting people with disabilities.
Our solution consists of five main phases: firstly,
a biomedical kit with sensors, Wi-Fi/GPS/GSM
module that will be utilized to track the patient's vital
signs and location. Secondly, a voice recognition
microphone that will convert speech-to-text, will be
used for people with hearing loss to communicate.
Also, the device contains a speaker to output verbal
sounds that have been created by transforming text-
to-speech conversation. Thirdly, ultrasonic sensors
are known as the third eye for people with blindness.
Finally, a vibration sensor that will vibrate to inform
the patient that the circuit of speech recognition is
operating. Lastly, the Atemga microcontroller
receives collected data from the sensors and the
modules, which will process it before uploading it to
a database, where it will be kept and retrieved at any
time via the mobile application.
 4.3. Software system
The software part of our system will include a
database to collect all the acquired data from the
patient, and a mobile application used by the
medical team to access this data.
A. Database
A database is a collection of data that is
specifically structured so it can be easily accessed or
edited. It can be created for any type of data that
needs to be stored for future use or reference. In our
case, the database will be used to save any data
regarding the person. From the complete personal
history to the constantly updated person’s
recordings.
There exists a special type of database called a
real-time database designed specially to contain
information that is needed for real-time processing
to handle workloads whose state is constantly and
quickly changing. There are many ways to design a
database such as; Microsoft Azure and Amazon web
services (AWS) but in our project, we are going to
use Google’s Firebase service to design our real-
time database.
Google’s Firebase can be easily configured to
the mobile application through the android studio.
For our system, it will be an easier and more
efficient choice since we will be designing android
mobile applications to access our patient’s data.
Inside firebase there are two types of storage
systems, firstly is the Firestore database which more
closely resembles a standard database with some
optimizations for mobile applications, the other is a
real-time database that can be updated instantly by
any data uploaded through the application or the
wearable device, such as recordings, prescriptions,
or a piece of general personal information about the
patient.
B. Mobile application
Mobile applications are a new and quickly
developing Segment of the global Information and
Communication Technology. They are easy, user-
friendly, inexpensive, downloadable, and able to run
on any platform. Therefore, we chose it to be our
user interface.
Our mobile application will include a registration
page used to enter new patients and members of the
medical team. An administrator will oversee
assigning patients to each physician or nurse in case
there are several of each. Next, each member of the
medical team will have a page specific to them
where they can access the information that pertains
to them for each of their patients. This will include
three pages: the first contains the patient’s general
information, the second contains the diagnosis and
prescription, and the list will display the recordings
which will be updated as soon as the medical team
member opens the application.
4.4. Hardware system
The various components to be used in the
system are:
A. AVR-Atmega
AVR-Atmega is a microcontroller developed by
Atmel for many applications as an embedded
system. They have adapted an 8-bit RISC single-
chip microcontroller to the Harvard architecture and
use on-chip flash memory for program storage. The
GPIO pins available on the board, shown in Figure
3, are used for input/output purposes and can be
programmed as needed. The proposed system uses
AVR-Amega32. The AVR-Atmega and the IoT
have proven to be pioneers in innovative technology
in health care.
B. Temperature Sensor
The LM35 sensor was used to measure
temperature. It is designed to measure temperature
using a temperature proportional analog output.
The temperature sensor outputs a voltage signal
proportional to the Celsius temperature. The LM35
sensor is the ideal Kelvin-calibrated linear
temperature sensor because it does not require a
large constant voltage to be removed from the output
to obtain a reading in degrees Celsius. These great
features of the LM35 sensor make it very easy to
connect to any type of circuit.
C. Heart-rate and SpO2 Sensor
The MAX30100 is a sensor that combines pulse
oximetry and a heart rate monitor. It detects pulse
oximetry and heart rate signals using two LEDs, a
photodetector, improved optics, and low-noise
analog signal processing. The IR LED emits infrared
rays, which are reflected off the surface. The voltage
that passes across the resistor increases as the
intensity of the light beam increases. The tip of the
index finger is placed on the sensor to determine
heart rate. The LED will begin blinking in time with
the pulse as soon as the circuit detects one. The
output is delivered to the microcontroller, which
measures the patient's heart rate in BPM and blood
oxygen level.
D. Camera Module
The ESP32-CAM module is a cost-effective and
powerful platform that requires you to develop
intelligent projects with a microcontroller. This
module sends data through Wi-Fi and Bluetooth.
The ESP32 has a dual-core 32-bit processor clocked
at 80, 160, or 240 MHz the microcontroller has a
unique cable to connect to the camera, a slot for a
microSD memory card, and the ESP32S chip itself.
The image is saved in JPEG format on the SD card.
A UART to USB adapter is required to connect the
module to your computer. Thanks to a
microcontroller that operates at a low-level protocol,
the camera can receive and analyze images of
objects that reside in the room. For that reason, this
module is as popular as the ESP8266 or
ESP32CAM. This module consists of a cable for a
direct connection to the camera sensor. The camera
sensor has a wide-angle lens with a viewing angle of
160 °.
E. Voice Recognition Module
Voice Recognition Module that incorporates an
excessive sensitivity microphone. It is simple to
govern and helps as much as eighty voice
instructions at once. Around seven voice
instructions may want to work on the equal time.
Any sound of any language (English, Arabic,
French) can be skilled as a command. Users want to
familiarize the module first earlier than letting it
apprehend any voice command. Speech recognition
modules are used in lots of programs along with
domestic automation for humans with special needs.
F. Vibration Motor
is a low-cost vibration motor. The PWM controls
the vibration intensity of the sensor, which can
easily convert electrical signals into mechanical
vibrations. Wearable devices suitable for
manufacturing vibration-sensitive interactive
products vibrate and call small vibration motors. It
serves as an inaudible indicator. If the input is high,
the motor will vibrate as it receives signals from the
speech recognition module.
5. Proposed Scenarios
5.1. Healthcare and Alzheimer’s Disease
Our completed setup of the wearable device will
consist of the AVR-Atmega microcontroller
connected with the HR sensor, temperature sensor,
gyroscope/accelerometer, and piezo sensors, which
will be powered by an external rechargeable battery
of roughly 10 Amp that will keep our system
running for 56 hours. The vital signs data log from
the kit will be collected by the AVR-Atmega
microcontroller, which will subsequently be
uploaded to the real-time firebase database. On the
microcontroller, high-level and low-level alarms for
all vital signs, as well as a fall detection system, will
be installed to alert the medical staff instantly if any
strange readings occur. All the data collected from
our wearable devices, such as the patient's history,
diagnosis, and prescription, will be available to the
medical team in charge of the case via the mobile
application user interface.
Alzheimer's patient's family will allocate the
patient's location using the GPS/GSM module. The
GPS module will collect the calculated parameters,
the longitude and latitude position, and send them to
the mobile application through SMS using GSM.
5.2. Deaf and Dumb
A. Deaf Module
When a person calls the patient by his name, the
wearable band in the patient’s hand will start to
vibrate now of hearing their name using a speech
recognition microphone. Therefore, the patient will
enable the STT to work and communicate with that
person through the mobile application as shown in
the diagram below Figure 9. The wearable device
also will notify the person with some critical words
that he needs to listen like:
• Dangerous if any dangers threaten his life.
• Prayer for the prayer time.
• Phone if someone calls him or any kind of
massages.
Since he is an Alzheimer’s patient, he also needs
to hear some words like:
• Medicine reminds him to take the
medicine.
• Food to remind him for food time.
• Exercise to remind him to do some
exercises.
STT App
Audio Display
Mobile
Mic Text Display
Figure 3. Working of the STT
B. Dumb Module
On the other hand, when the patient wants to text
back, the mobile application will convert Text-to-
Speech, as shown in Figure 10, where the output
voice will come from the speaker.
App TTS
Text
Keyboard Text Speaker
Figure 4. Working of Text-to-speech
5.3. Blind
The two solutions for a blind person:
A. Face recognition
The patient will be wearing a camera module,
which will take the image of the unknown person.
The camera will send the image to the phone and
will compare the picture with the data stored in the
database. After recognizing the person, it will send
a voice message to the patient telling them the name
of the person (Figure 11).
For Alzheimer’s patients, we can locate their
position through GPS on the wearable device and
send it to the mobile application.
The main purpose of this work is to assist the
communication for the deaf, dumb, and blind people
so that they can easily communicate with each and
everyone around them and easier their life.
Audio
CONCLUSION
To sum things up, we would like to do an
associated framework in IoT-HealthCare for
disabled people, that is easily checked and
controlled through a versatile application and
coordinated wearable devices. The coordination
between the application and the wearable devices
will allow the recording of numerous data about the
patient’s vitals, assist the Alzheimer’s community,
and aid the visually, vocally, and audibly disabled
person.

Camera Cloud App Audio

Figure 5.Working of Face Recognition

B. Third eye
The third eye for people who are blind is an
innovation that helps blind people to navigate with
speed and confidence by detecting the nearby
obstacles using the help of ultrasonic waves and
notifying them with vibration and buzzer. They only
need to wear this device as a band.

6. Results

The connected Atmega with GSM module, LCD,

medical sensors, and microphone, when turned on,
the desired solution can be selected by using the
switch.

Medical sensors sent the data through the mobile

application to be easily monitored. For deaf people,
voice recognition helps them to hear important
words around them. If he wants to communicate
with the outer world, he will enable speech to text
on the mobile application. For dumb people, they
can enable text to speech also on the mobile
application. For blind people, the ultrasonic sensor
will sense the barriers around them. The camera
recognizes the peopled saved on the database.
REFERENCES
[1] World Health Organization and World
Bank Group, World Report On Disability,
New York, 2013.
[2] R. S. Wilson, E. Segawa, P. A. Boyle,
S. E. Anagnos, L. P. Hizel and D. A.
Bennett, "The Natural History of
Cognitive Decline in Alzheimer's
Disease," American Psychological
Association, vol. 27, 2012.
[3] World Health Organization, Global
action plan on the public health response
to dementia 2017-2025, Geneva, 2017.
[4] Alzheimer's Association, 2017
Alzheimer's Disease Facts and Figures,
Chicago, 2017.
[5] M. M. Lavallee, D. Gandini, I.
Rouleau, G. T. Vallet, M. Joannette, M.-J.
Kergoat, T. Busigny, B. Rossion and . S.
Joubert, "A Qualitative Impairment in
Face Perception in Alzheimer’s Disease:
Evidence from a Reduced Face Inversion
Effect," Journal of Alzheimer’s Disease ,
pp. 1225-1236, 2016.
[11] A. Sharma, K. A, M. Pandya and D.
Garg, "IoT based Assistive Device for
Deaf, Dumb and Blind People," Procedia
Computer Science, vol. 165, pp. 259-269,
2019.
[12] World Health Organization, "
Deafness and hearing loss Fact sheet," 1
April 2021. [Online]. Available:
https://www.who.int/en/news-room/fact-
sheets/detail/deafness-and-hearing-loss.
[Accessed 6 December 2021].
[13] A. Karmel, A. Sharma, M. pandya and
D. Garg, "IoT based Assistive," 2019.
[14] Isyanto, Haris; Arifin, Ajib Setyo;
Suryanegara, Muhammad (2020). [IEEE
2020 International Conference on Smart
Technology and Applications (ICoSTA) -
Surabaya, Indonesia (2020.2.20-
2020.2.20)]
[15] Zhou, Mingyong & Li, Wenyan & Zhou,
Bo. (2017). An IoT System Design for
Blind. 90-92. 10.1109/WISA.2017.72.222

[6] M. Roopaei, . P. Rad and J. J. Prevost,

"A Wearable IoT with Complex Artificial
Perception Embedding for Alzheimer
Patients," Texas, 2018.

[7] World Health Organization, Universal

Eye Health: Global Action Plan 2014-
2019, Geneva, 2014.

[8] A. A. Nada, M. A. Fakhr and A. F.

Seddik, "Assistive Infrared Sensor Based
Smart Stick for Blind People," in Science
and Information Conference, London,
2015.

[9] Y. Rohilla, V. Parihar and K. K

Rohilla, Ultrasonic Sensor based Smart
Cap as Electronic Travel Aid for Blind
People, 2020.

[10] R. E. Wendrich and R. Kruiper, "Keep

It Real: On Tools, Emotion, Cognition and
Intentional Design," in INTERNATIONAL
DESIGN CONFERENCE , Dubrovnik,
2016.