2020 International Conference on Smart Grids and Energy Systems (SGES)

B2B B2C Architecture for Smart Meters using IoT

and Machine Learning: a Brazilian Case Study
Victor Takashi Hayashi Reginaldo Arakaki Tiago Yukio Fujii
Computer Engineering and Digital Computer Engineering and Digital Computer Engineering and Digital
Systems Department Systems Department Systems Department
Polytechnic School of University of São Polytechnic School of University of São Polytechnic School of University of São
Paulo Paulo Paulo
São Paulo, Brazil São Paulo, Brazil São Paulo, Brazil
victor.hayashi@usp.br reg@usp.br tiago.fujii@usp.br
Khalil Ahmad Khalil Fabio Hirotsugu Hayashi
2020 International Conference on Smart Grids and Energy Systems (SGES) | 978-1-7281-8550-7/20/$31.00 ©2020 IEEE | DOI: 10.1109/SGES51519.2020.00152

Computer Engineering and Digital Didactic Dry Laboratories

Systems Department Federal University of ABC
Polytechnic School of University of São Santo André, Brazil
Paulo fabio.hayashi@ufabc.edu.br
São Paulo, Brazil
kha@usp.br

Abstract—In this paper, fault-tolerant architecture with a
focus on usability integrates Internet of Things, Mobile, Cloud,
and Machine Learning technologies to smart meter devices.
Based on a survey with end-users, we developed a smart meter
with an integrated display, mobile application, and a cloud-
based data lake. After deployment in four households with
different profiles in Brazil, energy state labeling with Hidden
Markov Models and daily energy prediction with Machine
Learning algorithms are implemented based on collected data.
Prediction and information extraction from smart meter data
can create value for both businesses and consumers.
Keywords—Smart Meter, IoT, Machine Learning, Prediction,
HMM
I. INTRODUCTION
Energy flow, from generation to home/business
consumption, is formed by a remarkable ecosystem that
impacts our daily life. That is transport, entertainment,
health, home places, public services, and so on. There a long
list of participants of this context that can be classified by the
following: personal and family in a home building,
organizations within their facilities, government with
regulations, generation, transmission and distributing
organizations, and digital organizations using IoT and
artificial intelligence for energy operations [1, 2].
One additional reference regards energy consumption and
production. Considering multi-source energy generation, by
wind, by solar, and using biogas, a user profile can change
from consumer to “prosumer”. There isn’t an absolute
function: sometimes it produces energy, sometimes it
consumes, and once in a while, it acts as producer and
consumer at the same time [3]. The billing of energy
consumption, tariff plans, and sustainability aspects need
data-powered decision making that could benefit all energy
cycle. And digital platforms bring excellent alternatives from
artificial intelligence to improve precision and prediction
using real-time data [4].
In this new context, an essential condition emerges –
how to measure energy consumption in real time while
dealing with a faulty Internet connection? Considering some
participants as mentioned, an innovation platform using
smart meter would bring some values to the energy
ecosystem, as related in the following [2]:
a) Scenery 1: Consumer and Prosumer are profiles of
people, independently of home energy consumption or of a
business building, the understanding about how to deal with
sustainability, how to use energy reasonably involves
qualified information of energy consumption like as how
much, what is the power demanded by electric devices
categorized by places and equipment;
b) Scenery 2: Enterprises doing business for energy in
high volumes need data to plan those operations. How much
energy will be necessary for organizations like public spaces
of the city, for the industries, for the households? Real-time
data with information of amount and historical behavior is
the way to use intelligent algorithms to get high accuracy of
estimates;
c) Scenery 3: Regulation from the public government can
bring some values by flexible taxes organized by time
windows of a day, of a week, of a month, made possible by a
digital platform with real-time data;
d) Scenery 4: Digital connecting people, homes, places,
business buildings, hospitals, using cloud computing, smart
meters connected by the Internet of Things create
opportunities to innovate in B2B, B2C, G2B, G2C projects
(B2B – Business to Business, B2C – Business to Consumer,
G2B – Government to Business, G2C – Government to
Consumer). The energy ecosystem of Fig. 1 involves the
digital connection of people, businesses, and organizations,
creating network effects from rational use of energy and
integration of other convenience services.
Fig. 1. Energy Ecosystem.

978-1-7281-8550-7/20/$31.00 ©2020 IEEE 826

DOI 10.1109/SGES51519.2020.00152

Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.
 This paper presents some recent results from the
academic research and engineering project of a smart meter
architecture. The solution is capable of collecting energy data
in real-time from a personal or business place, process, and
synchronize data to a cloud database. The data allows
specialist algorithms to predict and disaggregate energy
consumption patterns employing machine learning models.

II. ENGINEERING CONCEPTS
Engineering requirements guided the development of IoT
smart meter architecture to implement differentiated aspects
like cost, fault tolerance, internet connection, digital and
social network, user journey mappings with energy
consumption, and exponential scalability of real-time data
and processing. The engineering aspects of this requirement
are described [5, 6]:
a) Open Platform: This solution focus on the integration
of a multi-source of the hardware of data collecting devices,
including IoT meter with digital connectivity by TCP/IP,
Bluetooth protocols. Besides the native IoT data collection,
other hardware and software can be integrated directly into
the solution, with no particular black-box components or
services [1];
b) Fault Tolerance: Local processing of data, remote
synchronizing method, intelligent algorithm to improve
sensors precision, data processing e and internet connections
[2]; and
c) User Experience: Integration to UX Devices like
Alexa, Google Assistant, and Android application to create
gamification experiences to deal with energy consumption.
That is, instead of using electrical measures (like watts.hour,
“white tariff” regulation tables), the system adopts
information of expenses and prediction using money
information with stimulating dashboards to engage users.
The engineering methods to create and improve the
solution are the references ATAM [6], ISO 25010 [7], ISO
10746 [8], and Architectural Tactics [6].
III. PRELIMINARY SURVEY
Questions regarding general information and energy
awareness integrated a preliminary survey with end-users.
Each inhabitant of selected households answered the survey
and provided input at household and individual granularity
levels. The survey rationale followed the existing Energy
Feedback Systems study in Europe [9].
The first aspect is the General Information. Individual
characteristics are age, marital status, education level, and
gender, and household information is income and the total
number of inhabitants.
Fig. 3. Energy Awareness aspects considered in the preliminary survey.
The second group compromises energy awareness
perceptions. One regards motivation: one can raise energy
awareness because of financial or environmental triggers.
The last aspect regards the monitoring and notification
interfaces preferred.
As shown in Fig. 4, we installed smart meters in four
households. Households A and C have just one inhabitant
each, while household B has two inhabitants, and household
D has four inhabitants.
Regarding general information, 25% of participants have
18 to 24 years old, 37.5% has 25 to 34 years old, 12.5% has
45 to 54 years old, and 25% has 55 to 64 years old. Only
25% of all eight people are married, and exactly half of the
participants are male. In the educational aspect, 50%
completed college, 25% completed high school, 12.5%
completed postgraduate studies, and 12.5% have a Ph.D.
Total family income varied among households: 12.5% have
1 to 2 minimum wages, 50% have from 4 to 5 minimum
wages, 12.5% have from 6 to 7 minimum wages, and 25%
are above 8 minimum wages. A minimum wage is a financial
value in Brazil, equivalent to 200 dollars, approximately.
Regarding motivations for energy awareness, Fig. 5
depicts household profiles. More than 50% of participants
agree that mobile and specific display interfaces must be
used to monitor energy consumption. More than 75% of
participants agree that notifications must be received through
a mobile application. Other interfaces, such as website, e-
mail, smart speakers (e.g., Alexa), smartwatch, and smart TV
have not attracted much interest from end-users.
Fig. 4. Number of Residents by Household.

Fig. 2. General Information aspects considered in the preliminary survey.

Fig. 5. Energy Awareness motivation profiles by household.

827

Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.

Fig. 6. Most interesting functionalities in end-user perspective
IV. SMART METER
Based on survey results, functional, non-functional
requirements and restrictions were specified. Intending to
build an architecture that allows value creation for both
residential users and businesses, the first step was to design a
smart meter with autonomy and granular data collection.
Autonomy integrates fault tolerance, offline functionalities,
and local communication aspects: the user must be able to
visualize and interact with information directly with the
smart meter module, with no dependence on Internet
connectivity to do so. Granular data collection allows users
to analyze their energy consumption patterns in detail and
support use cases based on specific algorithm developments.
As shown on the survey results of Fig. 6, the most
exciting functionalities for household inhabitants are access
to individual appliance energy consumption and next month's
energy bill estimate. Based on desired functions, the smart
meter must be able to measure energy consumption, collect
historical values, and present this information to the end-
user. Additionally, temperature data must be acquired to
provide external features for prediction algorithm
development.
The non-functional requirements are based on autonomy
and usability aspects. As stated earlier, the autonomy
requirement is related to offline functionality and standalone
operation. Data collection and information interface access
must happen even on Internet unavailability. Usability
aspects are related to providing energy consumption
information in displays integrated into the smart meter and
on the mobile interface to support different user-profiles and
moments.
Some restrictions considered are ease of installation,
energy, size, and cost constraints. The smart meter module
must be small, easy to install, must not have significant
energy consumption, and must be as cost light as possible.
Smart meter development was based on previous work.
In 2017´s Computer Engineering capstone project, Internet
of Things smart home architecture with fault tolerance and
offline operation was developed. Another capstone project
from 2019 used the fault-tolerant architecture to specialize in
energy awareness, and further extended the work by
integrating it with conversational interfaces and prediction
algorithms. In 2020 the smart meter pilot was rolled, with the
development of smart meter and deployment on four
households, named A, B, C, and D households. Additionally
to prediction algorithms, energy disaggregation explorations
were added to the project.
828
Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.
Fig. 7. Smart meter development timeline
Data collection of consumer electronics devices
characteristics was performed with a specialized module
based on Arduino, with a non-invasive current sensor. This
data was used in energy disaggregation experiments, as
further detailed. The specialized module for the take-off data
collection schematic was created in Fritzing Open Source
Printed Circuit Board (PCB) design tool and is present in
Fig. 8. The module was installed in one household (other
than A, B, C, and D).
The smart meter deployed on households A, B, C, and D
is named RM8 (Real Meter 8). It can measure eight current
inputs, with a measurement rate of one sample per second,
and collects energy consumption data with hour granularity.
The files are stored in an integrated filesystem, and each day
results in one file. It has an autonomy of one-month data
storage with hour granularity. RM8 also estimates the energy
consumption bill by using the average network voltage level
and the average cost per energy consumed (note that we have
a flat energy tariff in Brazil).
RM8 has the following differentials: architecture with
two processors with serial communication, and integrated
standalone interfaces that provide information to end-users
within the household regardless of Internet availability. The
main processor is ESP8266 Wemos D1 Mini, an IoT
development board with integrated WiFi communication.
The coprocessor used is the Arduino Nano, a microprocessor
based on ATMega 328. Both processors and main
characteristics are summarized in Table S.
TABLE I. MICROCONTROLLER (MCU) COMPARISON
Wemos D1 Arduino Nano
mini ESP8266 ATMega 328
Flash Memory 4 MB 32 K
Clock 80/160 MHz 16 MHz
Analog Inputs 1 8
Fig. 8. Specialized module for take-off using Arduino Mega.
 One can observe that ESP8266 has better processing and
memory capabilities than ATMega 328. Another ESP8266
advantage is its integrated WiFi communication, but the
ESP8266 module only has support for one analog input. On
the other hand, Arduino Nano ATMega 328 has support for 8
analog inputs.
RM8 architecture uses both ESP8266 and Arduino Nano
ATMega 328: Arduino Nano ATMega 328 is responsible for
sensor reading, and Wemos is responsible for data Fig. 10. The final version of RM8 installed in household A.
processing, data storage with an integrated filesystem,
information display to end-user, and external wireless
communication through an integrated webserver.
RM8 peripherals are detailed below:
a) Power Supply Module: responsible for converting
110V AC from the network to 5V DC necessary for low
voltage components like Arduino;
b) Real-Time Clock Module: it frees up the processing
of parts and facilitates synchronization, also allowing
autonomous circuit operation by including an internal
battery;
c) Temperature Sensor: to trace the correlation between Fig. 11. RM8 installed in households B (left) and C (right).
energy expenditure and temperature, which affects the use of
air conditioning equipment, such as air conditioning, heater
and shower, and electric taps;
d) Clamp Meter Sensor: eight non-invasive sensors
responsible for reading current;
e) TFT Display: to access consumption information
directly on the module;
f) Push Button: on the front of the module, with factory
reset function, also allowing change between the views of the
integrated display. Fig. 12. RM8 installed in household D.
Four RM8 smart meter modules were deployed in four Fig. 11 depicts the RM8 smart meter installation process
Brazilian households in the first semester of 2020. Fig. 9 in households B (left side) and C (right side). RM8 smart
depicts the first version of the RM8 smart meter deployed in meter was also deployed in a home of four residents. Fig. 12
household A. Eight current sensors are installed near the shows RM8 in household D, with customized daily
electric distribution board. The distribution of energy consumption patterns displayed in the smart meter.
consumption by sector and historical trend are integrated into
the smart meter display. The final version of the RM8 smart To illustrate the RM8 usability aspect, we highlight some
meter is present in Fig. 10. A more straightforward but useful functionalities implemented in a web application hosted in
display was customized for the user after initial usability the smart meter itself. The web interface depicted in Fig. 13
feedback. allows change between total and specific sector power
consumption, real-time power consumption, the
configuration of graph time granularity, accumulated energy
usage, and historical temperature graph. All aggregated data
is stored locally in the RM8 filesystem, and data older than
one month with hour granularity is deleted in a First In First
Out (FIFO) rationale. Note that aggregated data in month
granularity is stored up to 12 months.
The web interface also allows the end-user to compare total
accumulated values and distribution between sectors. All this
information is available offline (i.e., the end-user can access
the interface with a direct WiFi connection, without the need
of Internet nor local WiFi network, as the RM8 module
becomes a WiFi access point itself).
The web interface is integrated with the mobile interface,
Fig. 9. The initial version of RM8 with integrated display that extends the functionalities by allowing data submission
to a data lake hosted in a cloud computing environment, as
detailed in Section V.

829

Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.

Fig. 13. Web interface accessible by desktop or mobile client.
V. ANDROID APPLICATION
The data acquisition has three main cores: a physical
module in the facility where the energy is being measured, a
mobile application, and a cloud service. The first consists of
sensors, an electrical circuit attached directly to the facility’s
electric control panel, and a computational power that can
deliver the data acquired. The second is an application that
runs on a mobile device, connects to the module locally,
downloads the data, sends it to the cloud, and presents it to
the user. The third one, the cloud, is where the data is stored
and is made available to other processes. Fig. 14 shows an
overview of the system.
Fig. 14. Overview of the mobile application for data collection
The mobile app has three main functions, one is to obtain
data from the local module, the other is to show the data to
the user also locally, and finally, to provide those data to the
cloud.
The module does not need to be connected to the
Internet, as it can make a WiFi hotspot, to which the mobile
app connects locally, so the module is not constrained by
connectivity. It can be installed in a place without an Internet
signal, still allowing the user to check the data and upload it
to the cloud. A diagram of the mobile application and cloud
communication is shown in Fig. 15.
Fig. 15. Data Collection Sequence Diagram for Android Application
830
Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.
Fig. 16. Accumulated Daily Files in Data Lake by Household
The application runs on the Android operating system
version 6 (Marshmallow) or higher with WiFi capability and
broadband Internet, for example, a residential WiFi or 4G.
The smartphone must be in the module hotspot range to
collect the data, which is up to 5 meters. Once harvested, to
upload the data to the cloud, the physical distance to the
module is irrelevant.
The project uses Firebase cloud service, a Google
solution to cloud computing. This service is elastic both on
storage and computation, so it assures scalability not only on
both areas but also on income and outcome bandwidth on
servers as necessary.
By the end of August 2020, the cloud has 337 files,
which means there are 337 days of household consumption
data available to be studied. They come from 8 different
users in 4 different households in diverse locations. The data
started to be gathered in June of 2020, which gives an
average of 84,25 days of data for each home, but as each file
has around 2KB, space needed so far reached ~1,6MB (data
has hour granularity). Fig. 16 depicts the evolution of data
collected in the four Brazilian households through the mobile
application.
VI. NON-INTRUSIVE LOAD MONITORING USING HMM
Non-Intrusive Load Monitoring (NILM) tries to identify
individual appliance loads from the total residential load
[10], with the advantages of being less intrusive compared to
the individual load monitoring.
NILM systems can be separated as low and high-
frequency hardware [11]. While high-frequency data, with a
sampling rate greater than 1 kHz, can detect more granular
features leading to better results, it raises new challenges of
data transmission and price accessibility. Low-frequency
data with a period from 1 to 60 seconds, although
problematic when disaggregating appliances with variable
loads or multiple ones with similar consumption [9], can still
be used to disaggregate in real-time using low-cost hardware
[12].
We used Hidden Markov Models (HMM) to classify
power consumption of on-off state appliances as defined by
[10] with sampling rates of 1 and 40 Hz, showcasing the
applications of the low-frequency data while still keeping
the price of the hardware accessible. As a proof of concept,
we used the energy consumption of a fan with sampling
rates of 1 and 40 Hz, switching it on and off multiple times.
In both cases, the disaggregation was successful, as can be
seen in Fig. 17, where the state of the appliance - 0 as turned
off and 1 as on - correctly follows the current power
consumption.
 VIII. CONCLUSION
An end-to-end solution integrating smart meters to
Cloud, Mobile, IoT, and ML technologies is presented. The
survey highlighted household inhabitants' desire to monitor
appliances and predict the value of the next bill. Eight
Fig. 17. Automatic fan take off data labeling using Hidden Markov Model sensors smart meter with offline functionality is developed
and deployed in 4 households in Brazil. Mobile application
VII. SHORT-TERM ENERGY FORECASTING USING ML harvests the data and sends it to the cloud, where algorithms
To forecast the short-term (24 hours horizon) energy are used for data labeling and daily energy prediction. The
consumption, we benchmarked some Machine Learning most promising prediction algorithm is XGBoost with
(ML) algorithms: long short-term memory (LSTM), Support temperature, date, and past consumption features.
Vector Regressor (SVR), and Extreme Gradient Boosting
(XGBoost) model using date, past consumption, and ACKNOWLEDGMENT
temperature as features, downsampled to the hourly The authors thank “Amigos da Poli” endowment fund.
frequency and taken from 4 households. The horizon of 24
hours was chosen to be significant enough to be informative
to the user as opposed to only one hour in the future. REFERENCES
[1] Arakaki, Reginaldo, Victor Takashi Hayashi, and Wilson Vicente
Long Short-Term Memory (LSTM) uses a recurrent Ruggiero. "Available and Fault Tolerant IoT System: Applying
neural network architecture and can maintain and forget Quality Engineering Method." 2020 International Conference on
memory of important or insignificant information, being Electrical, Communication, and Computer Engineering (ICECCE).
successfully used for household forecasting [13]. Extreme IEEE, 2020.
Gradient Boosting (XGBoost) is a gradient boosting library [2] V. Hayashi, R. Arakaki and R. M. Andrade, “OKIoT Open
Knowledge IoT Project: Smart Home Case Studies of Short-term
based on decision trees ensembles [14], meant to be a Course and Software Residency Capstone Project”, IoT BDS 2020 –
scalable and sparsity aware implementation. 5th International Conference on Internet of Things, Big Data and
Seccurity, Czech Republic, May 2020.
To evaluate the forecasts, one of the metrics used is the
[3] RATHNAYAKA, AJ Dinusha et al. A methodology to find
adjusted error proposed by [15]. While traditional metrics influential prosumers in prosumer community groups. IEEE
such as the mean squared error (MSE) help standardize the Transactions on Industrial Informatics, v. 10, n. 1, p. 706-713, 2013.
results between different papers, they fail to account for [4] Islam, Syed. "Digital applications in implementation of smart grid."
displacements in time. We used the mean squared error 2016 International Conference on Accessibility to Digital World
(MSE), normalized mean squared error (NMSE), and 4-norm (ICADW). IEEE, 2016.
adjusted error [15] with adjustment window w of 2. [5] S. Saad and R. Arakaki, “An event processing architecture for
operational risk management in an industrial environment”, DEBS
The summarized results are shown in Table II. Among 2014 - Proceedings of the 8th ACM International Conference on
households A, C, and D, XGBoost performed better in Distributed Event-Based Systems, 2014.
adjusted error metric. Fig. 18 shows a comparison of the [6] L. Bass, P. Clement and R. Kazman, “Software Architecture in
forecasts of each model and the real value of the hourly Practice”, SEI Series, Addison Wesley, 2013.
energy consumption of the household A. Even using only [7] ISO/IEC 25010, Software Product Quality Model, ISO/IEC, 2011.
data from 24 hours ago, the forecasts are still adequately [8] ISO/IEC 10746, Open Distributed Processing – Reference Model:
Architecture, ISO/IEC, 2009.
accurate.
[9] T. Serrenho, P. Zangheri, and B. Bertoldi. "Energy Feedback
TABLE II. SHORT-TERM FORECASTING SUMMARIZED RESULTS
Systems: Evaluation of Meta-Studies on Energy Savings through
Household Algorithm MSE NMSE Adjusted Feedback." Science for Policy Report by the Joint Research Centre
Error (JRC), the European Commission’s Science and Knowledge service.
XGBoost 0.0013 0.1611 0.0390 Luxembourg: Office of the European Union, 2015.
A LSTM 0.0025 0.2997 0.0567 [10] G. W. Hart, “Nonintrusive Appliance Load Monitoring,” Proc. IEEE,
SVR 0.0019 0.2247 0.0507 vol. 80, no. 12, pp. 1870–1891, 1992.
XGBoost 0.3390 0.2681 0.2630 [11] M. Zeifman and K. Roth, “Nonintrusive appliance load monitoring:
B LSTM 0.0420 0.3319 0.3037 Review and outlook,” IEEE Trans. Consum. Electron., vol. 57, no. 1,
SVR 0.0325 0.2572 0.2727 pp. 76–84, 2011.
XGBoost 0.0403 0.5484 0.3956 [12] S. Makonin, F. Popowich, I. V. Bajic, B. Gill, and L. Bartram,
C LSTM 0.0330 0.4489 0.3765 “Exploiting HMM Sparsity to Perform Online Real-Time
SVR 0.0394 0.5350 0.3813 Nonintrusive Load Monitoring,” IEEE Trans. Smart Grid, vol. 7, no.
XGBoost 0.1267 0.4183 0.5360 6, pp. 2575–2585, 2016.
D LSTM 0.1518 0.5011 0.6577 [13] F. T. Romero, J. D. C. J. Hernandez, and W. G. Lopez, “Predicting
SVR 0.1389 0.4585 0.6312 electricity consumption using neural networks,” IEEE Lat. Am.
Trans., vol. 9, no. 7, pp. 1066–1072, 2011.
[14] T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting
System”, 22nd SIGKDD Conference on Knowledge Discovery and
Data Mining, pp. 785–794, 2016.
[15] S. Haben, J. Ward, D. Vukadinovic Greetham, C. Singleton, and P.
Grindrod, “A new error measure for forecasts of household-level,
high resolution electrical energy consumption,” Int. J. Forecast., vol.
30, no. 2, pp. 246–256, 2014

Fig. 18. Daily Energy Consumption Forecast for Household A using ML

831

Authorized licensed use limited to: BOURNEMOUTH UNIVERSITY. Downloaded on June 21,2021 at 18:35:13 UTC from IEEE Xplore. Restrictions apply.