See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/312484107

Smart home environment — Agent-based models with scenarios

implementation support

Conference Paper · October 2016

DOI: 10.1109/GIIS.2016.7814949

CITATION READS

1 200

4 authors:

Petr Blecha Peter Mikulecky

University of Hradec Králové University of Hradec Králové
6 PUBLICATIONS 39 CITATIONS 70 PUBLICATIONS 292 CITATIONS

SEE PROFILE SEE PROFILE

Petr Tucnik Jan Matyska

University of Hradec Králové University of Hradec Králové
62 PUBLICATIONS 188 CITATIONS 15 PUBLICATIONS 97 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Informační, kognitivní a interdisciplinární podpora výzkumu View project

Knowledge-based support of water reservoir dispatchers View project

All content following this page was uploaded by Peter Mikulecky on 13 February 2018.

The user has requested enhancement of the downloaded file.

 Smart Home Environment –Agent-Based Models
with Scenarios Implementation Support

Petr Blecha, Peter Mikulecky, Petr Tucnik, Jan Matyska

Department of Information Technologies
University of Hradec Kralove
Czech Republic
{petr.blecha, petr.tucnik, peter.mikulecky, jan.matyska}@uhk.cz

Abstract—The agent-based frameworks could be used either
for modeling or real control of the smart home environments and
stand as a useful tool allowing deployment of digital layer
overlaying hardware/network infrastructure which is then used
for data input processing, decision making, and action execution.
Since adaptability and efficient control of such system is
expected, parallel goals pursue must be used to meet the needs of
all subjects in it. However, it is not always clear which goals have
priority over others and for this reason, various scenarios are
executed (with limited variability) in a virtual model of such
environment in order to clarify normal functioning of such
system on the everyday basis. This paper is focused on various
issues related to the implementation of scenarios in the agent-
based model(s) representing the smart interior environment of a
small flat.
Keywords—smart homes; multi-agent based models; scenarios;
ambient intelligence
I. INTRODUCTION
This paper is focused on utilization of pre-designed
scenarios for the creation of more precise virtual models of the
intelligent (ambient) environment (AmIE). The scenarios are
used to describe typical interactions of the environment and its
inhabitants, and point out important functionalities that are
expected (or intended) to be implemented. The advantage of
such approach is that scenarios might be scalable, with a
customizable level of abstraction according to designer needs,
and could serve in design stages of design of AmIE in a
similar way as UML use-case scenarios. Since an
understanding of scenarios does not require any special
technical knowledge or background, it can be used by
potential customers/users to clarify their expectations and
needs. The output of such processes might be a list of
functional and non-functional requirements of such system.
The use of scenarios in ambient intelligence is not novel,
as a well-known representative example might be mentioned
the famous ISTAG report [1]. In spite of this fact, scenarios
might still prove to be a useful tool for calibration of virtual
representation (model) of the smart environment. However,
according to modern trends in wireless networks (e.g. Internet
of Things or concept of Industry 4.0), there seems to be a
strengthening trend leading to the establishment of the “virtual
layer” (which is considered a model, in this context) over real-
world infrastructure. This is one of the reasons for
incorporation of the model layer as an integral
component/layer in our AmIE framework.
One of the fundamental problems with such virtual
representation is difficult modelling of user’s behaviour which
is generally expected to be reasonably reliable even if the real-
world data are unavailable. This issue is sensitive since
unreliable model could compromise other stages of the AmIE
development. In order to avoid excessively demanding
development of the simulation of the user`s behaviour,
scenarios are used to capture standard situations, with a
limited variability if it is desirable. Set of scenarios might be
diversified and consulted continuously as needed. This is
important since the hardware used in the system is often
customized/adjusted to serve the user`s needs and real-world
implementation and its virtual representation might be
developed in parallel. This shortens the time for
implementation of such system and improves the involvement
of environment`s future user in the process of its development.
Our research is primarily focused on the smart solutions for
offices and other workplaces, homes, and domain of Ambient
Assisted Living (AAL). Therefore, only scenarios from these
areas will be mentioned within the scope of this paper.
II. RELATED RESEARCH AND STATE OF THE ART
The first and essential usage of scenarios in the area of
intelligent environments was undoubtedly the ISTAG report
[1]. The scenarios presented there are up to now quite often
used as starting point for many further researches in the area
of intelligent environments. Scenarios, as descriptions of
typical interactions of users with the respective environment,
seem to be a usable tool for initial steps in any contemplation
about functionalities of an intelligent environment. Let us
mention a couple of recent publications devoted to this area.
A recent paper [2] can be understood as in a sense new
ISTAG report [1]. It outlines a set of visionary scenarios on
how the European society could develop by 2030 by using
advanced ICT tools and modelling techniques and integrating
them into governance processes and policy making
mechanisms. The four scenarios proposed here are oriented
much broader as the original ones from [1]. Namely, they are
focused on ICT for governance and policy making to support
the building of an open, innovative and inclusive Digital
Europe in 2030.
 As it is stressed in [3], there are many ways of modelling
the dynamic behaviour of reactive systems. The most popular
approaches are scenario-based and state-based modelling.
There is a plenty of notations for scenario-based as well as
model-based modelling. Among the scenario-based notations
is probably the best known the UML Sequence Diagrams,
among the model-based notations the Petri Nets achieved a
notable popularity. Scenarios are useful not only for the
requirement phase, but also for the design and implementation
phases of various kinds of systems, conclude the authors of
[3].
An approach to plan and produce scenario traces as
examples for the usage of intelligent environments is
presented by Buchholz and Forbrig [4]. The implemented
application allowed an efficient and intuitive way of preparing
and describing a sequence of actions that form a scenario
further on. This scenario could be than used then as a basis for
an intelligent environment under development. This approach
enables to record user behaviour without the need of real users
performing in the real environment.
According to Sutcliffe, scenarios are instances of actual
experience with a system captured from users [5]. For the
purposes of his research, Sutcliffe defined scenarios as facts
describing an existing system and its environment including
the behaviour of agents and sufficient context information to
allow discovery and validation of system requirements. This
definition can be considered as a bit overcome now, as the role
of scenarios and scenario – based development seem to be
much stronger in development of future systems, especially
future complex intelligent environments.
A very interesting and useful analysis of a number of
existing scenarios used in Ambient Intelligence for design and
development of intelligent system was performed by
Friedewald and his colleagues [6, 7]. Both papers address
issues of privacy and security of intelligent environments by
analysing scenarios for ambient intelligence applications that
have been developed over the years up to 2006, but the results
could be considered as valid also in the present time. The main
message from the published research was, that ambient
intelligence technology frequently goes beyond most of
currently existing privacy-protecting borders, and that changes
in people’s expectations of privacy will happen more slowly
than technology capabilities grow. Therefore the scenarios
used for requirements analysis for design and development of
new intelligent environments should reflect these findings
without any doubts.
Le Guilly with colleagues in a recent paper [8] proposed a
system that enables specification of constraints restricting the
control commands that can be used inside user-defined
scenarios. A formal framework is proposed, based on timed
automata for modelling smart home environments and
scenarios that can be formally verified against formal
requirements representing constraints on possible control
commands sent to the system. Among other features, the
system proposed and implemented by Le Guilly and his
colleagues enables automatic extraction of device models,
automated verification of scenarios as well as their execution
[8].
A special importance for the area of scenario – based
prototyping for ubiquitous computing and intelligent
environments has the work [9] by Coroama. His doctoral
thesis, although eight years old now, introduced new and
usable scenario – based prototyping method, which could be
still used for efficient prototyping of ubiquitous environments
and other ubiquitous applications, especially from the area of
assistive technologies.
III. STANDARD FUNCTIONALITIES OF THE ENVIRONMENT
Following the list of functionalities in the system might be
used for a scenario development. These features are fully
implementable in the environment, depending on the user`s
preferences. The features, listed below, are divided into
several groups according to the area of interest they belong to.
The list serves as a set of examples, it is not complete or
exhausting all possible options, but still gives the basic idea
what are fundamental building blocks of the AmIE system’s
functionality.
Security – electronic security system, central security
console connection, biometric access, camera surveillance and
recording system, infrared lighting, simulated presence, fire
alarm system, flooding detection system, detection of exterior
movement, detection of door/windows opening, infrared/laser
barriers, tremor detection, assistive technologies for elderly
(AAL technologies).
Economic functioning – heating control (floor heating,
radiator heating control, fan coil control, recuperation control),
humidity conditioning, air conditioning, energy consumption
monitoring, temperature monitoring and control, integrated
meteorological station, heating of eaves and drains, access
areas heating, swimming pool temperature control, whirlpool
control, sauna & steam control, backup generator, GSM
communication, IP communication, HDO signal integration.
Ecology and power saving – heat pump integration,
garden pond/water facilities integration, solar panels
integration, photovoltaic integration, wind power plant
integration, water treatment integration.
Comfort –touch panels, iPad, iPhone, tablet, smart phone
control, intercom, home intercom, door control, layout plan
navigation, lighting control, light scenes, shading control, plug
control, gate control, centralized time and alarm, water level
monitoring, central vacuum cleaner integration, wine cellar
integration, motor control of doors and windows, white
electronics control, bathtub control, irrigation system, mailbox
signalization, customizable user interface, system diagnostics,
SMS notification, remote access.
Entertainment – TV, satellite receiver, game console, Hi-
fi stereo, and Blu-Ray disc integration, home cinema, multi-
room audio, projector and screen control, multimedia storage,
digital HDMI matrix, video, photos, and audio sharing, TV
adjustment, and retractable TV/projector control.
Communication – wireless connection, internet
connection, an optical connection, structured cabling, VoIP
technology, WPN, WI-FI.
 Although the list could be longer and could include more
items, this provides the general idea about the system’s
functionalities. The user can decide which are to be
included/excluded based upon his/her individual preferences.
This will constraint set of actions the AmIE would be capable
of as well as number of sensors that are to be incorporated in
the environment, both on the real (hardware infrastructure)
and virtual model layers.
IV. BEHAVIOUR MODELLING
As was already mentioned in the introductory sub-section
of this paper, one of the fundamental problems with virtual
representation layer is that it is quite difficult to create
plausible model of user’s behaviour (with primarily humans or
animals in mind, at this point). From the development
perspective, this issue is one of the most difficult to address
but also very important. We can determine how the sensors or
actuators would react in the environment to different
situations. The behaviour is already defined in the system, it is
a matter of how it was designed and constructed. But how will
Fig. 1: The virtual smart flat representation in the Anylogic platform.
persons react every day in standard (easier case), or unusual
situations (more difficult issue), is very individual.
Within the research project focused on smart flat which is
currently under development (using Java based platform
Anylogic v7.x, see flat illustration on the Fig. 1), the scenario
describing everyday usage of the flat was used.
As it is illustrated in the Fig. 1, the model contains 3D
graphics for better representation of the environment itself,
and – maybe even more importantly – its behaviour. The two
agents represent residents of the smart flat and are situated in
the virtual model. These residents use flat for ordinary
activities like cooking, sleeping or relaxing, etc. The
behaviour of flat environment is represented by sensors
(humidity, motion, temperature…) and data obtained are
processed by the control unit. Control unit evaluates data and
decides what actuators (window opener, alarms, electrical
switches…) should be used at the given moment.
Data evaluation/analysis is necessary for the practical
usage of any sensors or actuators. The event could occur when
the resident is going to the bathroom and then taking the
shower. The humidity sensor reports increasing humidity in
the bathroom and sends data to the central unit. But the central
unit detects there is a resident in the bathroom and that
probably means the resident taking a shower. In this case,
central unit will not execute any actuators (alarms or actuator
used to turn off the water) and decide to “do nothing”. In case
when the resident is out of the bathroom, the central unit could
activate a silent alarm (by sending a message to the smart
phone or terminal) and inform the resident. If the resident
would not be present in the flat the central unit could also stop
the water by sending signals to actuators in the bathroom.
The model uses time units for effective timing of actions.
The Anylogic platform has integrated time system which
could be set by user to various levels of precision – simulation
could proceed in second, minutes, hours or even years (which
in this context is only theoretical since it has limited practical
value). This allows observation of the behaviour of the whole
model and adapts behaviour of particular agents to act towards
optimal performance. The model of the smart flat is designed
to use seconds as default units of measurement in order to
process ordinary activities in the model timeframe close to
real-time. The Anylogic could accelerate the timescale of
simulation runtime with multiplier over 500x, therefore the
one simulation year could occur during approximately one
minute of the real time. It is dependent on the complexity of
behavior or on the number of agents in the model which
setting will be used.
The average resident in the model sleeps every day about
eight hours and about the same amount of time is spent in the
work. Eight hours in the work is generally regular but sleeping
time is to a certain degree random because the resident does
not go to sleep every day at the same time and random
generator is used.
Outdoor environment has its own temperature and air
humidity. The house has its own heating, air conditioning and
windows which regulate the house stats according to the needs
of its residents.
There are certain smart components used in the house. For
example, smart fridge which could remind the upcoming date
of expiration of products stored within, a system to remind
medication (in the given period) and the system monitoring
residents’ health with various indicators (by the smart watches
or motion sensors, or fall detection system).
Residents in the model have some attributes which are
related to whole model behaviour. The non-smoker resident
with a lower level of cholesterol will probably live for a longer
time than smoker resident with a higher level of cholesterol.
The main attributes are defined on the Tab. 1.
Tab. 1: Resident’s parameters in the model of Smart Flat.
PARAMETER DESCRIPTION
AGE Actual age of the resident
SEX Sex of the resident
PENSION DATE Date when the resident retires
CHOLESTEROL Cholesterol level of the resident
PARAMETER DESCRIPTION
BLOOD PRESSURE Blood pressure of the resident
SMOKER Is the resident a smoker
HEALTH LEVEL % health level of the resident
(more the healthier)
ENERGY % energy level of the resident
CLEAN LEVEL % clean level of the resident
HUNGRY LEVEL % hungry level of the resident
BATHROOM NEED % need of a bathroom
INFARCT RISK Level of infarct risk (0…1)
There is a summary of ordinary resident day behaviour and
some principles used in the virtual model of smart flat:
If the resident is being hungry, she will probably go to the
kitchen to do something to eat – if there are no more important
needs to be satisfied (need of medical aid for example). The
same principle is applied with needs of personal hygiene (go
to the bathroom) or sleeping (needs energy - go to the
bedroom). Personal hygiene occurs at least in the morning
after the resident gets up and in the evening before the resident
goes to sleep.
The energy parameter is replenished also by eating and if
the resident has a very low-level energy during a day he could
getting rest to the bedroom and he does not have to wait until
evening. If the resident is sick he is tired more often.
If the resident is seriously ill, he normally stops going to
work and stays at home. The recovery time in the house
environment is shorter than the option of going to the work
with illness.
In the case the resident is getting a heart attack, the health
level decrease. There is a simulation of the incoming
ambulance with doctor and transport resident to the hospital.
The first aid arrives in 5 – 15 minutes to the flat. But the
resident may not survive the infarct. There are two
possibilities when this event may occur:
 In the case of very intensive heart attack, the resident
dies immediately
 Opportunity the first aid will not arrive in the time
(but the first aid should be called in time, smart flat
should recognize heart attack by the smart watches)
If all of the residents of the smart flat died there will be no
entities to observe and collect data so the simulation should
stop.
If we consider the 8 hours resident spend in the work and
on average 8 hours spend by sleeping there are next 8 hours
left for a resident on his free time.
During the spending free time, there are some options
which the resident could make:
 Watch TV
 Read the book
 Using PC
 Making meal
 Using bathroom
 Work on the garden
 Relax in the garden (pool, gardening, car washing…)
  Go to the visiting
During these mentioned activities the resident still has to
follow his needs. There is a possibility that the resident will
swim in the pool which makes him tired. After such activity,
he has to go to eat or take a rest. But if the need of bathroom
use will be higher or the same, he will prefer the opportunity
to go to the bathroom and after that, he will go to the kitchen
or take a rest.
The model layer is still in the development. As can be seen
in Fig. 1, the visual model of the smart flat is finished,
consisting of reusable building blocks which allows
redesign/remodelling as needed. The sensors are completely
prepared and assigned to the flat. Behaviour of the residents of
flat is described in various scenarios and soon will be
implemented to the model. The simple proposal and style of
resident behavior are symbolized at Fig. 2. It is a state
behavior representation of Anylogic platform.
Fig. 2: Preliminary simple propose of resident behavior.
As it has been described in this sub-section, scenarios
consist of smaller behavioural building blocks which together
allow modelling of resident’s behaviour. There is a system of
preferences used to distinguish between more and less
important sets of activities (such as handling life threatening
conditions). Obviously, some of the situations occur less
frequently in the real life – such as the mentioned heart attack
– but even these critical situations should be handled by the
AmIE efficiently. The general goal is to gradually create more
plausible model of the residents’ behaviour through
incorporation of a large number of behavioural building
blocks and their customizable sequencing.
V. CONCLUSIONS
Scenario – based design and development of intelligent
environments as well as scenario – driven prototyping have
View publication stats
already proven their importance and usefulness. In the scope
of the research project one of its research lines is focused on
smart flat which is currently under development. A scenario
describing everyday usage of the flat was used and visualized
using Java based simulation platform Anylogic v7.x. The
usefulness of this simulation tool for verification of various
scenarios created on the way towards new smart homes, flats,
or offices development seems to be high as our experience
signalized. In the paper it is demonstrated on a couple of first
steps towards a smart flat, as an example of presented
approach.
ACKNOWLEDGMENT
The financial support of the Czech Scientific Foundation
scientific project GA15-11724S DEPIES - Decision Processes
in Intelligent Environments is gratefully acknowledged.
REFERENCES
[1] K. Ducatel, M. Bogdanowicz, F. Scapolo, J. Leijten, and J.C.
Burgelman, Scenarios for ambient intelligence 2010, ISTAG report,
European Commission. Institute for Prospective Technological Studies,
Seville, November 2001, available from: ftp://ftp. cordis.
lu/pub/ist/docs/istagscenarios2010. pdf.
[2] G. Misuraca, D. Broster, and C. Centeno, "Digital Europe 2030:
Designing scenarios for ICT in future governance and policy making,"
Government Information Quarterly 29, 2012, pp. 121-131.
[3] H. Liang, J. Dingel, and Z. Diskin, "A comparative survey of scenario-
based to state-based model synthesis approaches," in Proceedings of the
2006 international workshop on Scenarios and state machines: models,
algorithms, and tools. ACM, 2006, pp. 5-12.
[4] G. Buchholz, and P. Forbrig, “From Collaborative Scenario Recording
to Smart Room Assistance Models”, in International Conference on
Distributed, Ambient, and Pervasive Interactions, Springer International
Publishing, August 2015, pp. 262-273.
[5] A. Sutcliffe, "Scenario-based requirements analysis", Requirements
Engineering 3,1, 1998, pp. 48-65.
[6] M. Friedewald, E. Vildjiounaite, Y. Punie, and D. Wright, “The brave
new world of ambient intelligence: An analysis of scenarios regarding
privacy, identity and security issues”, in International Conference on
Security in Pervasive Computing, April 2006, Springer Berlin
Heidelberg, pp. 119-133.
[7] M. Friedewald, E. Vildjiounaite, Y. Punie, and D. Wright, "Privacy,
identity and security in ambient intelligence: A scenario analysis,"
Telematics and Informatics 24,1, 2007, pp. 15-29.
[8] T. Le Guilly, M.K. Nielsen, T. Pedersen, A. Skou, J. Kjeldskov, and M.
Skov, “User constraints for reliable user-defined smart home scenarios”,
Journal of Reliable Intelligent Environments, 2016, pp. 1-17.
[9] V. Coroama, “Scenario-driven prototyping for ubiquitous computing”,
Doctoral Dissertation, Eidgenossische Technische Hochschule ETH
Zurich, Nr. 17850, 2008.