2017 9th IEEE-GCC Conference and Exhibition (GCCCE)
Ambient Intelligence in Automated Houses
Omar M. Abbas, Amr A. Lotfy, Ahmed N. Abdul-Basset, and Mohammed El - Abd
Electircal and Computer Engineering Deparmtnet
American University of Kuwait
{s00023456,s00023405,s00024127,melabd}@auk.edu.kw
Abstract— Smart houses are invading nearly all modern
companies under the flag of the booming field of home automation.
This field is being implemented with the highest technologies to be
integrated in daily life and be a non-neglect able part of everyone’s
life style. This field is growing at a fast pace as the field of smart
devices was years ago. The main purpose of smart houses and
home automation is to provide a more luxurious lifestyle to its
users. However, throughout the span of the past years, there was
not as much attention paid to the possibility of dangerous
situations as the attention paid to providing luxurious welfare. The
proposed home automation system in this work provides both
luxurious and emergency features. There are six features
discussed throughout this paper that shows how a house can deal
with home tenant and the surrounding environment in case of
emergency or dangerous situation.
Keywords— ambient intelligence; smart homes; Kinect; home
automation; artificial intelligence; skeleton detection; facial
recognition; hotspots Introduction
I. INTRODUCTION
In a world where everything is moving fast and
becoming smarter, we are determined to revolutionize the daily
lifestyle. Ambient Intelligence in Automated Houses (ALOHA)
is going to be the modernized smart house that will solve many
problems, make life easier and more luxurious. Nowadays, we
hear a lot of stories on the news or read on newspapers of
horrific accidents that occur because of silly mistakes. For
example, irresponsible residents that cause small errors such as
fire, lack of care for the elderly and children as well, or even
homicides and other crimes that violate the basic rights of
residents. What if there was an entity watching over your
residence, securing you from all the points that are mentioned
above and designed to prevent them. Not only that, but it also
provides you with a luxurious lifestyle.
We propose a home automation system (ALOHA) having
two sides, just like the human sympathetic and parasympathetic
nervous systems; it can detect both dangerous and peaceful
situations. ALOHA approaches safety by detecting dangerous
behaviors and symptoms. By keeping an eye out for elder
people’s behaviors such as falling and calling for help and
acting accordingly by calling the ambulance and alerting other
residents. It can also check up on the movement of a child if
they come near any of the house’s hotspots such as electric
plugs or windows. In addition, it can detect fire or smoke,
calling the fire department and notifying other residents to
evacuate the place. Moreover, ALOHA can recognize faces for
security issues to distinguish between residents and outsiders.
978-1-5386-2756-3/17/$31.00 ©2017 IEEE
So, in case of burglary or violence acts by intruders ALOHA
will call the Police and alerts the rest of the residents.
ALOHA’s luxurious services include remote control of
lighting, AC, door locks, and TV all by simple hand gestures
and voice commands. The users can get notifications from the
system by a simple user-friendly application that will be
provided for smart phones and tablets. Apart from all the
previous features, ALOHA is made environmental friendly by
having the ability to alternate between the main power supply
and alternative power resources depending on the environment
of the house itself.
ALOHA is designed and built using various technologies.
These technologies are the Kinect sensor, microcontrollers, and
computational power represented by PCs. Some additional
circuit designs will be required to utilize ALOHA’s features.
The renewable or alternate power source could be wind, water,
or solar.
In overall, ALOHA is very user friendly as the users interact
with it with simple hand gestures and voice commands, while it
responds back through an application easily accessible by its
users. ALOHA will make daily lifestyle easier and more
luxurious. Most importantly it will preserve the residents’
safety and it will act against dangerous signs to keep the
residence’s environment as safe as possible.
The rest of the paper is divided as follows: Section II
presents the research and implementation phase, the system
performance is evaluated in Section III. Finally, the paper is
concluded in Section V.
II. LITERATURE SURVEY
In this paper, we researched similar systems to ALOHA and
compared it to our work. We compared the technologies used
as well as the features available in those other systems. Below
are the researched projects:
A. Non-Commercial:
1) In July 2014, [1] enclosed their system in the context of
intelligence systems and technology in the environment to
enrich the capability of recognizing people and the system
adapting itself to user specific needs and preference by using
the sensors of a Kinect. This system also includes user
recognition and environment adaptation while performing daily
activities.
2) In April 2013, [2] designed and implemented tracking
and user identification system for use in a smart environment.
Their system makes users to issue commands via their smart
 2017 9th IEEE-GCC Conference and Exhibition (GCCCE)

home giving it nonintrusive voice commands. Biometrics and home or not. This enhances the security of the house and
facial recognition, user tracking, Active/Passive Infrared family members. The fourth feature is sharing access to non-
Tracking and RFID chip are all ways to help their system family members. This gives some flexibility to frequent
function. The technology used for their system was Microsoft guests and close friends.
Kinect. 4) Z-Wave is a Danish leading company in-home
3) In [3] they developed a smart home system that helps post- automation. Although they are not using Kinect sensors, but
stroke patients complete activities of daily living (ADL) they are using communication protocols between house
independently while saving their money, time and effort. This appliances. Their technologies can sure be integrated in our
system uses the Kinect technology to detect and monitor work and this will help us support thousands of devices that
patient’s intake gestures including fine cutting, loading food, use Z-Wave. The special thing about Z-Wave is their Portal,
and maneuvering the food to the mouth. The position of the which enables the user to interact with the house even if
wrist, elbow and shoulder are measured. In addition, the angular he/she were outside it. Our work has a portal similar to the
displacements at the elbow and shoulder joins are detected. one Z-Wave. [8]
Moreover, the Kinect is monitoring the acceleration of the 5) CastleOS started off as a system with the same purpose
spoon/fork/cup, which are held by patents. Also, it aims to of our work. [9] They are using Kinect to interact with
monitor their skeleton abnormal movements and ask for help various devices but they only use the voice commands
when needed. supported by Kinect APIs neglecting the other capabilities of
4) In [4] their system idea is to control home devices via the Kinect. Their system has a lot of features such as security,
voice commands in a wireless system, such as light, fan and climate control, energy management and many others.
music. They used XBEE technology to avoid distance CastleOS does not have emergency protocols as our work.
problems and signal interference and it is more flexible. The
C. Survey Tables:
system consists of human voice, microcontrollers as
Table I:
ATmega328 and ATmega1284, microphone, LCD and
ALOHA’s features compared to other systems.
devices, which are, the fan, lights and music stereo. However, System# Facial Skeleton Security Renewabl Luxury Emergenc
this system didn’t use a Kinect. Recognitio
-n
Recognitio
-n
Encryptio
-n
-e Energy Protocol
-s
-y
Protocols

B. Commercial:
A.1 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
1) Apple designed a home kit application [5] that links your A.2 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
apple smart device to the various appliances and accessories A.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
in the house. This application acts like a portal to remotely A.4 ‫ݲ‬
control the house easily. The house has a hub that identifies B.1 ‫ݲ‬ ‫ݲ‬
B.2 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
each smart device and their users to preset rooms or B.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
environments according to their preference without the user’s B.4 ‫ݲ‬ ‫ݲ‬
command. The house can also be controlled if the user is B.5 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
outside. The house can also sense the location of the smart ALOHA ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
device so for example if the user is approaching the house, the Table II:
ALOHA’s technologies compared to other systems
lights will go and so on so forth. Similar to other systems, the System# Kinect Sensors Portal Hub Microcontrollers Remote
Apple home kit has no emergency protocols; but only luxury (Smart Controllers
App)
protocols are provided.
2) The Technology: Insteon: [6] the technology makes A.1 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
products for monitoring and controlling a house. Insteon A.2 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
A.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
operates without a hub and has phone applications for all A.4 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
operating systems. It has sensors like water leakage sensors B.1 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
and motion sensors, it can turn on and off lights, it can also B.2 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
detect smoke. Insteon can control climate as it can also B.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
monitor energy. It can do other features but mostly B.4 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
B.5 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
emergency features. Insteon have not concentrated on
luxurious features that we have in our work.
ALOHA ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
3) Human-Computer Interaction Institute (HCI) with D. Similar systems in Kuwait and the G.C.C.
collaboration with the German company Bosch have created 1) Envision Kuwait [10] is a home automation system
and developed a secure smart house called Haven. [7] KIN is implemented and designed in Kuwait established in 2010.
a proposed family-aware mobile system to integrate Bosch Their system is basically controlling various parts of a house
into the smart houses. The system focuses on four main or a building using an app on a smartphone or a tablet. This
aspects that help one family to control data in and outside app can control the lightings, switch on and off the T.V,
their smart house. First aspect is, managing devices and data. switching on and off the sound system as well as opening and
Second aspect is, creating routines. This allows family closing the windows. However, our peak on this system is
members to be reminded by the devices to do every day’s that they have no emergency protocols and their luxurious
routine as watering the plants, feeding the pets, putting protocols are very limited, as they don’t have any hand or
snacks in kid’s bags...etc. The third feature is to monitor the facials gestures as they do not have a facial or skeleton
house while being away. A camera is being installed at the recognition.
door that sends notifications and video streams to family 2) ICAS [11] is a company based in Kuwait established in
members on their phones. Moreover, the sensors located in 2007. This system is basically a remote control that controls the
the house can detect if family members have gone outside A/C, lightings, sound systems, curtains, security systems and
 2017 9th IEEE-GCC Conference and Exhibition (GCCCE)

much more. They use a technology called “JUNT” and more within ALOHA [14]. Once our skeleton detection detects
technologies stated on their website. This system does not have someone, it will start monitoring the head and knee joints. The
emergency protocols and it does not detect your face and head and knee joints are monitored using the Kinect 2 Body
skeleton like our work. Frame. Kinect 2 body frame sends 30 frames per second and
returns all body joints in 3D space which consists of the X, Y
3) E-home automation international [12] is a system based in
and Z components. We discard all joints and points but consider
Dubai, UAE. This system creates powerful touch buttons like
the head and knee joints and their Y axis points only. After that,
“reading”, movie time” and “relaxing” which adjust the
we start comparing the Y axis points of both head and knee
environment accordingly. Control A/C, fans, heaters, curtains
joints, if the result of the comparison was that the Y axis points
and more. It calls you and your neighbors when someone
of the head joint was below the Y axis points of the knee joint,
intrudes your house. This system keeps an eye on your property
falling would be detected and further steps would follow to
when you’re away.
make sure emergency falling is being detected. [15]
Table III:
ALOHA’s features compared to other systems in Kuwait and the G.C.C.
System# Facial Skeleton Security Renewable Luxury Emergency
Recognition Recognition Encryption Energy Protocols Protocols

D.1 ‫ݲ‬ ‫ݲ‬

D.2 ‫ݲ‬
D.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
ALOHA ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
Table IV:
ALOHA’s technologies compared to other systems in Kuwait and the G.C.C. Fig. 1. Kinect snapshots with the user in the normal standing up condition.
System# Kinect Sensors Portal Hub Microcontrollers Remote Those voice commands are being recognized by ALOHA
(Smart Controllers
by using multi-language speech recognizers supported by
‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
App)
D.1 Microsoft. Those multi-language recognizers are useful as it is
D.2 ‫ݲ‬ ‫ݲ‬ not compulsory to use a specific language. For ALOHA to
D.3 ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬
ALOHA ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ ‫ݲ‬ check the fallen person and waits for ten seconds for a reply and
then either notify the ambulance or go idle, each task runs on a
III. RESEARCH AND IMPLEMENTATION separate thread. If all tasks ran on one thread, the system will
go into an infinite loop and eventually crash.
ALOHA has many features implemented within it. The
whole idea of our work evolves around interactions between
humans and computers. The user will feel that the system is
embedded within its surrounding which is the user’s house. An
AI system can interact with both the user and the house and
make decisions based on specific circumstances it detects. The
features we discuss in this paper are mainly related to that AI
system.
Fig. 3. Kinect snapshots with the user in the falling down condition.
The features to be discussed in this paper are the following:
Notice that the skeleton of the user looks unusual in figure
A. Human Falling Detection and Rescue 3, however, keep in mind that the positions of the joints are still
One main emergency feature covered in ALOHA is to being read and taken correctly but drawing them becomes a
detect if someone has suddenly fallen to secure house residents problem when the skeleton falls down. The reason behind that
in case of faint. For example, if someone suddenly falls and is because some joints may be in front of other joints. The
needs help, and cannot call for help, ALOHA checks the fallen accuracy of this feature depends on the response of our system
person. If there was no interaction detected from the fallen and the multi threads to the Body Frame returned by the Kinect
person for an amount of time or they said they’re not ok, 2 sensor. The error rate can be found in Table V.
ALOHA would notify other residents and the ambulance that B. Smoke Detector
there is someone who needs help as shown in figure 1.
We analyzed and evaluated several types of gas/smoke
sensors, however, we decided to use a gas/smoke detector of
type MQ-2. We chose this type of sensor because it is a
sensitive model compared to other models. MQ-2 gas sensor
has a high sensitivity to a lot of substances such as propane,
Fig. 1. SMS sent to notify users. hydrogen and others. It can detect a range of 300-10000 Parts
In case the fallen person replied that they’re ok, again per Million (ppm) of combustible gases. It has a long life and
ALOHA would be idle. The interaction between users and low cost and is portable.
ALOHA happens wirelessly, using voice commands. This After choosing this type of gas/smoke detector we had to
feature is very important in most residential buildings, study its specifications and functions. Every type of gas has its
hospitals, schools, offices, labs...etc. because it monitors own different concentration, and based on this we can control
seniors and children to rescue them in case of an emergency. the type of gas/smoke we detect. First, we connected the smoke
The main scope in this feature is to monitor elderly people as detector to an Arduino UNO and we observed the outputs
they are more vulnerable to falling [13] . depending on the atmosphere [16]. Firstly, when the
The implementation of Human Falling Detection and atmosphere around the sensor was clear, the Arduino sends via
Rescue is divided into three main parts. First, someone needs to Bluetooth a value of nearly 0.00 volts. However, we observed
be identified and detected by the skeleton detection supported that when there was minor smoke in a room, the Arduino sends
 2017 9th IEEE-GCC Conference and Exhibition (GCCCE)
via Bluetooth a value of around 2.00 volts. We then observed
when the concentration of smoke increased in the room the
value went up to 3.00 volts.
We also wanted to test other types of gases to see the
difference of concentration. Therefore, we tested butane gas
which is used in lighters. We released a small amount of butane
near the smoke/gas detector and the value returned was around
3.60 and above. We observed those several types of gases to see
the concentration difference between different types of gases.
Therefore, based on the numbers we collected, we observed that
when the sensor detects a value of over 2.00 volts, it means that
either smoke or gas has been detected in the room.
Consequently, we programmed our system so that whenever
the sensor detects a value above 2.00 volts a notification is sent
to the fire department informing them that there is smoke/gas
detected in the specific house stating the address of the
house/apartment. A notification is also sent to the residents of
the house for them to take care if they’re not aware of it, to
avoid danger. A buzzer is also turned on whenever the sensor
detects a value of above 2.00 volts. Initializing the buzzer and
the smoke/gas detector is done in a separate thread avoid
lagging the whole program. To know the accuracy of our
feature we conducted a lot of tests to observe how good our
smoke/gas detector systems and how good our systems
response to the smoke detector. The error rate of this feature can
be found in Table V.
C. Hotspot
A hotspot is a dangerous place in the house that you do not
want your child to be around. For example, a hotspot can be an
electric plug, a heater, a window or a house’s door. To save a
place where there is a hotspot, an authorized person must stand
in front of the Kinect where the hotspot is located. After
standing in the location of the hotspot, the authorized person
must give a voice command saying “hotspot”. Our program can
detect all the joints of a person and can save their location in
terms of X, Y and Z-axis. However, this command saves the X-
axis and the Z-axis of the person’s spine base where the
authorized person said the command.
Our program can save up to six hotspots per room. We can
also delete a hotspot in a room by giving a voice command
saying, “remove hotspot”. Also, your child won’t be authorized
to give these voice commands. So, whenever a child gets near
a hotspot of the specified X-axis and Z-axis of a radius of 0.5
meters a buzzer is turned on to alert parents that their child is in
a dangerous spot. In order to know the accuracy of our feature
we conducted a lot of tests in order to observe how good our
hotspot function and how our system responds to it.
D. Authorization by Skeletal Identification
This feature is vital in ALOHA; as only authorized users
have the ability to control their houses. For example, a child
should not have the authority to turn off smoke/gas detectors,
or a guest should not have the ability to lock or unlock the main
door. This feature was implemented through several
procedures, one of them is the skeletal identification using
Kinect sensors [17].
The first procedure is through saving the skeletal
information of the user and this is done by the Kinect 2 sensor.
The Kinect 2 sensor can return the positions of 26 joints in 3D
space by providing their X, Y, and Z components through its
Body Frame. The saving process uses only 20 joints and
calculates the distances between several joints as shown in
equation (1) to calculate the length of the arm, leg, torso, length,
hip, shoulders and then it sums them all up in what is called the
Body Index using equation (2).
݀ ൌ  ඥሺ‫ݔ‬ଵ െ ‫ݔ‬ଶ ሻଶ  ൅ ሺ‫ݕ‬ଵ െ  ‫ݕ‬ଶ ሻଶ  ൅  ሺ‫ݖ‬ଵ െ  ‫ݖ‬ଶ ሻଶ  (1)
‫ ͳݏ‬ൌ  σଵଽ
ேୀ଴ ඥሺ‫ݔ‬ே െ ‫ݔ‬ேାଵ ሻ  ൅ ሺ‫ݕ‬ே െ ‫ݕ‬ேାଵ ሻ  ൅ ሺ‫ݖ‬ே െ  ‫ݖ‬ேାଵ ሻ 
ଶ ଶ ଶ (2)
A saving process is also done by the Kinect 2 sensors
which saves the face of the user. This is done by the HDFace
frame which provides the X, Y, Z components of 3D points
(1347 points to be exact) located on the detected face of the
user. The distances between these points are calculated as well
and summed up in what is called the Face Index which is a
unique number for each face as shown in equation (3).
‫ ʹݏ‬ൌ  σଵଷସ଺
ேୀ଴ ඥሺ‫ݔ‬ே െ ‫ݔ‬ேାଵ ሻ  ൅ ሺ‫ݕ‬ே െ ‫ݕ‬ேାଵ ሻ  ൅ ሺ‫ݖ‬ே െ  ‫ݖ‬ேାଵ ሻ 
ଶ ଶ ଶ (3)
Note that the numbers saved from the user’s skeleton and
face are in meters. The skeleton saving process is done in
separate thread where the Kinect 2 sensor reads 450 frames of
the user’s skeleton and calculates the average of the numbers
calculated in these frames, beside taking the user’s name and
whether he/she is authorized or not. Also, note that all the data
is saved in a database. Put in mind that the Kinect sensor runs
by 30 Frames per Second (fps), so taking 450 frames per second
and having delays of 35 milliseconds between each frame will
result in a total time 15 seconds to save the user’s data. The 35-
millisecond delay in the thread was necessary to ensure that
each frame will be read without missing any frames for the next
450 reading.
Same process is done during saving the user’s face. After
saving the users’ data in the database, it becomes possible to
identify them. The identification process which comes after
saving users systems in the same exact way, however, 150
frames is taken in a separate thread by the Kinect 2 sensor to
identify the user which takes less time than the saving process,
taking a total time of 5 seconds to be exact. After reading 150
frames the average is calculated and the notation of the number
is enlarged. Enlarging the data’s notation is done by multiplying
them by 10000 and then the decimals are removed to ease the
process of comparing the users’ data. After that, we subtract
numbers and if the difference is less than 100 then points are
added to the user who has that difference. In the end, the user
with the highest points is returned and identified. The third
procedure is checking whether that user is authorized or not by
checking his Authorized property in the database.
The last procedure is done in the speech recognizer which
takes voice commands from the user and acts accordingly. The
user must be tracked and identified in order to give a voice
command and once he/she does, the system checks whether this
voice command requires authority, if yes then it checks if the
user is authorized or not.
The accuracy of this feature mainly depends on the accuracy
of the skeletal saving and identifying functions. If the person is
saved and identified correctly, then the system would easily
know if he/she is authorized or not. The accuracy and error rates
of our features are found in Table V.
E. Intruder Detection
Another feature also based on the saving and identifying
functions is the intruder detection feature. This feature also uses
the Body Frame provided by the Kinect 2 sensor which is
discussed in the previous feature. As the name of the feature
implies, this feature is implemented to tell whether an
individual is an intruder or not. If the identifying function did
not identify the person who is being tracked, then it assumes
 2017 9th IEEE-GCC Conference and Exhibition (GCCCE)

that the person was not saved in the database from the beginning around, and when it is quite and the results were used to
and considers him an intruder right away. calculate the error rate of this feature which can be found in
Once an intruder is detected the system looks for an Table V.
authorized user around. If an authorized user was around, then
the system is idle and acts normally waiting for further IV. EVALUATION AND ERROR RATE
commands. However, if the presence of an authorized user was A. Evaluation
not in the house then the system turns on the buzzer and alerts
Simply, before any implementation of ideas and coding, we
the authorized users right away by notifying them that an
had to verify that our planned solution will be able to develop
intruder has been detected within their house. This feature can
customers and generate general satisfaction by the community
be applied in other environments besides houses, such as stores
we are living in.
or supermarkets to detect theft or burglary.
Our evaluation process comprises potential customers, and
The accuracy of this feature also depends on how accurate
for that sake we performed a representative round of survey for
users are saved and identified by our functions; however, the
professionals and youngsters. We collected results of total 200
error rate here is less than the authorization feature as this
people.
feature detects intruders as they are not saved in the database.
Due to that, it is easier for the system to tell if a person is not
saved than looking up a user in multiple users saved. The error
rate can be found in Table V.
F. Voice Commands
Voice commands are the main way of interaction between
Fig.4. Survey results, age.
the user and the system. For the user to control his/her Approximately 49% were in the range of 22-27 years old that
surrounding in the house, voice commands have to be said live with either elderly people or children.
according to the action it carries out. The way voice commands
were implemented in the system was through Microsoft’s
speech recognition libraries. Speech runtime SDK, and
language packs must be installed for the system to have a
speech recognizer [18].
The process of which speech is recognized is done through
the microphone array on the Kinect 2 sensor. Due to the
Fig.5. Survey results, showing whether the people we surveyed live with
limitation of Microsoft’s speech recognition library, a grammar old people or children or not.
file had to be written. Grammar files are used to contain the The range of 22-27 years old usually takes care of parents,
commands which the recognizer should expect from the user. grandparents or children; thus using hotspots would be very
In the speech recognizer itself, there is a factor called the helpful. In addition, 94% of our sample think that they would
Confidence Threshold. This factor has a value from 0.0 to 1.0 feel safer if their house can detect intruders and automatically
set by the developer. The higher this factor is the stricter the call for help
recognizer will be to the commands take, and the lower it is the
more lenient it will be. For example, if the Confidence
Threshold was set to 1.0 then the user must make sure it is quite
around the Kinect’s microphone array and has to say the
command clearly. Otherwise, if the Confidence threshold is set
to 0.0 then the speech recognizer will take anything said by the
user as a command even if it is not. A moderate value to the Fig.6. Survey results, showing whether people we surveyed would feel
factor with a reasonable error rate was 0.5. safer if their house can detect intruders.
After taking the voice command the system checks what this That is why using skeleton detection and database saving is
voice command triggers and performs the action accordingly. important. Also, being authorized to be able to control own
For example, if the user says “Turn the lights on”, the system houses by being identified by skeleton is important as almost
will be triggered and the lights will turn on. There are multiple 82% would feel safe if their own houses can identify them.
types of voice commands. Some voice commands are simple
that is limited to turning on or off devices. Some voice
commands require authorized users for the system to work.
Other voice commands are complex that performs multiple
actions at once. An example of those complex voice commands
is “Good Morning”, this voice command turns the lambs off,
opens the shades, and turns on the water heater. Another special
voice command is the “SOS”. Once the user gives this Fig.7. Survey results, shows whether people we surveyed would feel safe
or intimidated if the house can identify them.
command the system will turn on a buzzer, and immediately
notify the police department that there is a dangerous situation. In order for people who are living alone to be safe during an
emergency as falling, 91.4% think that they would feel safer if
In addition, there are also commands that enable the user to
switch between power sources manually only if the power in their houses are watching over them and calling for help; thus
the alternative source is not depleted. [19] falling detection and SMS features are a perfect solution
The accuracy of this feature depends on the accuracy of the
speech recognizer and the Confidence Threshold factor. We
conducted tests in multiple situations, where there is noise
 2017 9th IEEE-GCC Conference and Exhibition (GCCCE)

V. CONCLUSION
After illustrating the selected features of ALOHA, we want
to mention other features we also implemented like he portal
which is an android application we designed to control houses,
facial detection using Kinect 2 sensors, and speech synthesizers
Fig.8. Survey results, shows whether people we surveyed that elder used within our system. After testing the implementation of the
people living with you are safer if your house is watching over. functions we illustrated in this paper, we are determined to
Also, controlling house verbally is a luxurious feature that enhance them and sharpen their accuracy by interpreting errors
78.7% of our sample would like that their own houses to and minimizing them. Moreover, we are thinking of creating
understand them. new features to make our system smarter and more aware. In
order to do that, we need to study the technologies we are using
deeper and most importantly we need to be creative.
VI. REFERENCES
[1] Ferrera, M., Franco, A., Maio, D. July 2014. “On the use of the Kinect
sensor for human identification in smart environments.” Journal of
Ambient Intelligence and Smart Environments. vol.6, no.4, pp.435-446,
Fig.9. Survey results, shows the number of people we surveyed that 2014.
would like if their house could understand them. [2] Fitzpatrick, M., Matthiopoulos, N. April 2013. “Real Time Person
Finally, taking care of gas leakage and smoke detection is one tracking and Identification using the Kinect sensor.” Retrieved from:
of the main features that 93.6% of our sample would feel safer https://www.wpi.edu/Pubs/E-project/Available/E-project-042513-
if implemented in their own houses, especially in our region 081838/unrestricted/Real_Time_Person_Tracking_and_Identification_u
where the weather is very hot and humid. sing_the_Kin ect_sensor.pdf
[3] Hondori,H., Khademi,M.,Lopes,C. November 2012. “Monitoring Intake
Gestures using Sensor Fusion (Microsoft Kinect and Inertial Sensors) for
Smart Home Tele-Rehab Setting”. IEEE HIC 2012, Houston
[4] Wang.J., Zhang,S. 2012. “Smart Home System”. Retrieved from:
http://people.ece.cornell.edu/land/courses/ece4760/FinalProjects/f2012/j
w937_sz369/j w937-sz369/jw937_sz369.html
[5] https://developer.apple.com/homekit/ [Online]
Fig.10. Survey result, shows whether the people we surveyed would feel [6] http://www.insteon.com/technology/#technologycompared/ [Online]
safer if their house can detect smoke. [7] http://yieric.com/portfolio/haven/ [Online]
B. Error Rates [8] http://www.z-wave.com/what_is_z-wave#z-wave_at_home [Online]
[9] http://www.castleos.com/ [Online]
The table below shows the results of testing the features
[10] http://www.envisionway.com/
illustrated earlier. The results are used to conduct an error
[11] http://www.icas-kuwait.com/index.html
percentage to measure the accuracy of the features’
[12] http://www.e-homeautomation.com/solutions.php
functionality.
[13] Mundher, Zaid A., and Zhong Jiaofei. "A Real-Time Fall Detection
To test the functionality of each feature we ran it multiple System in Elderly Care Using Mobile Robot and Kinect Sensor."
times. The readings of 40 persons were taken to test the International Journal of Materials, Mechanics and Manufacturing
functionality of skeletal identification. 20 tests were also IJMMM 2.2 (2014): 133-38. Web. 29 May 2016.
conducted for each function as well. By dividing the number of [14] Gasparrini, Samuele, Enea Cippitelli, Susanna Spinsante, and Ennio
Gambi. "A Depth-Based Fall Detection System Using a Kinect® Sensor."
failed tests over the successful ones, we come up with an error Sensors (2014): 2754-775. Web. 29 May 2016.
rate for that function. All the error rates found in Table V are [15] A. Wickramasinghe, et al., "Recognition of falls using dense sensing in
based on the same approach. an ambient assisted living environment", Pervasive and Mobile
The Authorization feature however depends mainly on Computing, Vol. 34, Jan. 2017, pp14-24
skeletal identification more than the facial identification due to [16] 0HOJDU (QULTXH 5DPRV &LULDFR &DVWUR 'tH] DQG 3U]HPHN -DZRUVNL
the fact that the error rate of skeletal is less than the facial. Arduino and Kinect systems: Design, Build, Blow Their Minds. New
York, NY: Apress, 2012. Print. 29 May 2016.
Also note that the confidence threshold for the voice
[17] Aniruddha, Sinha, Kingshuk Chakravarty, and Brojeshwar Bhowmick.
commands is set to 0.5. "Person Identification Using Skeleton Information from Kinect." ACHI
TABLE V 2013: The Sixth International Conference on Advances in Computer-
Error Rates Table Human Interactions (2013): 101-08. Web. 29 May 2016.
Feature Error Rate [18] Giorio, Clemente, and Massimo Fascinari. Kinect in Motion Audio and
Falling Person 10% Visual Tracking by Example: A Fast-paced, Practical Guide including
Examples, Clear Instructions, and Details for Building Your Own
Smoke Detector 15%
Multimodal User Interface. Birmingham, UK: Packt, 2013. Print. 29 May
Hotspot Less than 5% 2016.
Authorization 5% [19] G. Biagetti, et al. “Distributed Speech and Speaker Identification System
for Personalized Domotic Control”, in Mobile Networks for Biometric
Intruder Detection Less than 5% Data Analysis, (Series of Lecture Notes in Electrical Engineering), Vol.
Voice Commands (with noise) 20% 392, Part III, Springer 2016, pp. 159–170.
Voice Commands (Quite) Less than 5%