Machine Translated by Google

2021 14th IEEE International Conference on Industry Applications Mo5Track C.3

Challenges for the Application of MLOps in Forecasting

Energy consumption
Tiago Yukio Fujii Wilson Vicente Ruggiero Haroldo LM do Amaral
Department of Computer Engineering Department of Computer Engineering Intelligent Power Systems and Techniques
and Digital Systems and Digital Systems Laboratory
Polytechnic School of USP Polytechnic School of USP Paulista State University -
Sao Paulo, Brazil Sao Paulo, Brazil UNESP
tiago.fujii@usp.br wilson@larc.usp.br Sao Paulo, Brazil
haroldo.amaral@alumni.usp.br

Victor Takashi Hayashi Reginald Arakaki

Department of Computer Engineering Department of Computer Engineering Khalil Ahmad Khalil
and Digital Systems and Digital Systems Department of Computer Engineering
Polytechnic School of USP Polytechnic School of USP and Digital Systems
Sao Paulo, Brazil Sao Paulo, Brazil Polytechnic School of USP
victor.hayashi@usp.br reg@usp.br Sao Paulo, Brazil
kha@usp.br

Abstract—Models for forecasting residential energy consumption in the
short term (horizon of hours or days) allow its users to plan and make
decisions in order to reduce their consumption. Meanwhile, much of two
academic jobs
This area uses offline environments for experimentation, disregarding the
challenges of automation, monitoring and updating of models in online
environments. This article presents the challenges and solutions in a case
study, detailing the implementation in a real scenario online of consumption
forecasting models in 4 Brazilian residences in the period of 2020. It is
concluded that the use of best practices and metrics for the development in
online environments will not only increase
Accuracy of forecasts, as well as facilitating the development of two models
and helping in the speed of experimentation and reproducibility of results.
The forecast of residential energy consumption can be used to
assist residents in decision-making and cost-conscious planning [9]
and energy concessionaires in medium-scale and large-scale
forecasting and detection of customer consumption anomalies [10].
We can also facilitate energy transactions between prosumers in
point-to-point (P2P) energy markets [11], [12], promoting the efficient
use of the electrical network. Home Energy Management Systems
(HEMS) can use consumption prediction as input for predictive
control models[13], helping to plan the use of controlled applications
such as washing machines, air conditioning systems and electric
vehicles, in order to optimize the use of energy consumption of
energy and financial economy for the user in the variable tariff
scheme.

Palavras-Chave—Machine Learning, Model Update, Residential Energy

Consumption, Consumption Forecast.
I. INTRODUCTION
The increase in the non-use of smart energy consumption meters
makes it possible to monitor and analyze the usage habits of
household appliances that were previously impossible. Coupled with
direct feedback, personalized applications to the user, such as
forecasting and classification of energy consumption, allow them to
educate and reduce their electricity bill [1].
The installations of the future, not only for residences, but also
for the industrial sector, should comprise a consumption chain in
which the behavior of energy use in real time will be made possible
by digital platforms based on measurements of residential or business
units. These aggregated data will allow analyzes of consumption,
seasoning, costs and planning in terms of generation, transmission
and distribution capacity [2], [3]. Com isso, or scene two digital data,
ready to be processed
by platforms of algorithms and artificial intelligence, it is quite
consistent with the innovations of products and services in this area.
Machine learning (ML) techniques for forecasting [4] and
classifying [5] energy consumption are widely used both in academia
and in industry, applying different learning models, such as neural
networks [6], support vector machines [7 ] and gradient boosting [8].
Meanwhile, academic research on consumption forecasting
models focused on static environments, analyzing the degradation
of the accuracy as time due to unexpected changes in behavior of
the time series
(concept drift) [14], to the sensitivity of the manual configuration of
hyperparameters and the training and prediction times of two models.
Although they have standardized metrics to measure the
accuracy of models, there is no consensus on the use of metrics to
measure the fitness of machine learning systems in terms of their
operation in online environments, making comparisons between
solutions difficult.
These jobs are considered real scenarios with data on residential
energy consumption, which have great dependency in relation to the
season of the year and temperature [4], [7], causing the concept drift
not analyzed in experiments in static environments.
The main contributions of work are in consideration of two
problems arising from or considering online environments for ML
systems [4], [7], além da
application of methodologies for the development and retraining of
models to attenuate these effects in a
case study using real data of energy consumption
of 4 Brazilian residences in the period of 2020 in order to approximate
a real scenario found in the industry.
The sections to follow, the problems encountered during the
development of an online solution for the forecast of

978-1-6654-4118-6/21/$31.00 ©2021 IEEE 455 ISBN 978-1-6654-4118-6

Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google
Residential energy consumption is discussed, and its solutions
are presented with the objective of not only improving the
accuracy of the model, but also the reproducibility, speed of
experimentation, and operability of the system.
II. MLOPS
In order to integrate the stages of software development
and information technology systems operations, the DevOps
culture uses test automation, monitoring and integration,
infrastructure management as code, among other techniques,
allowing assim deliveries
and continuous implantation of the system [15].
The application of the culture of DevOps in ML systems,
known as MLOps [16], seeks to adapt the DevOps techniques
to the area, distinguishing the practices used in traditional
software systems due to their dependence on the quality of
data by means of correct extraction and processing, its
exploratory character during the development, with the test of
different configurations, model architectures and feature
generation , and its
monitoring of only errors derived from the programming
wrong system, but also caused by obsolete or shipped models
and training data.
Thus, testing systems before introducing them into
production environments and monitoring their performance is
considered a good practice not to develop and operate software
systems. However, due to its nature
prediction, these practices are difficult to define and implement
in ML systems [17].
Google Research uses 28 metrics to measure the readiness
of ML systems in production [18]. These metrics involve tests
related to 4 categories, related to the input given, to the model
used, à
infrastructure, and system monitoring year, listed in TABLE I.
TABLE I. METRICS FOR THE PREPARATION OF ML SYSTEMS IN
PRODUCTION. SOURCE: [18]
Dices
1 Feature expectations are captured in data schemes
two
All the features are beneficial to the model's accuracy
3 Features are not too expensive in memory usage
4 Features in addition to business requirements
5
Pipeline has appropriate privacy control
6 New features can be added quickly
7 Code for creation of features is tested
Models
1 Changes in specifications are revised and versioned
two
Offline metrics correlate with real online impact
3 All hyper parameters are adjusted
4 The aging effects of the model are known
5 A simpler model is not better than the current one
6 The quality of the model is sufficient in all parts of the data
7 Model was tested considering issues of inclusivity
456
Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
infrastructure
1 The training of the reproducible model
two
Specification of the model goes through unit tests
3 All stages of the pipeline pass through integration tests
4 Quality of the model is validated before use
5
It is possible to debug the model
6 New models are gradually introduced to users
7 New models can be reversed quickly and safely
Monitoring
1 Changes of dependencies of the system are monitored
two Invariants two dice are kept both in the offline environment
how much online
3 Features are calculated equally during training and prediction
4 Models are not old too
5 The model is numerically stable
6 The model does not obtain significant changes at the speed of
training or delivery latency
7 There was no detriment to the quality of prediction in the online environment
Besides these metrics, another good practice in ML projects
is the separation of its stages in pipelines [19], in order to
facilitate the integration of the different stages, the scalability
of the system and the reproducibility of two results.
One of the two differences of online systems is the need
for continuous training of their models to avoid the occurrence
of concept drift. In [20] a strategy has been defined for
simulating and evaluating two effects of periodic training in
temporary series, finding the seasonality of two input data and
updating the model at each seasonal cycle, using training and
validation data that reflect the cycle more recent
III. FORECAST OF RESIDENTIAL ENERGY CONSUMPTION
With the implementation of the use of smart meters in the
residential sector, given on the individual consumption of
greater granularity, it allows new applications and discoveries
to help in education and economy by its user.
Among these applications, there is forecasting of residential
consumption, helping in the planning of expenses and decision-
making for the consumer [9], forecasting in the medium and
large scale by energy distributors [10], or assistance in the
energy transaction between prosumers (consumers who also
generate electricity on a small scale) in peer-to-peer markets
(energy transactions carried out directly between consumers)
[12], [21], and in predictive control models of Home Energy
Management Systems
[13].
Different energy consumption on a medium and large
scale, or hourly individual consumption, presents greater
volatility, with daily consumption peaks that may occur at
different times.
Due to this feature, traditional metrics to measure forecasts
such as the mean absolute error (MAE)
Machine Translated by Google

We end up measuring just accurately point by point, not analyzing with an intermediary for temporary storage of two data [3], [25].
temporary or formal errors.

Fig. 1 shows an example in which a constant forecast (F1), which
does not introduce a significant value to its user, has a point-to-point
error less than a forecast of behavior closer to the real one, more
dislocated in time (F3 ). Regarding the forecast F1 possui um
MAE of 0.82, forecast F3 possui a MAE of 0.99.[18][19][20]
The data of hourly consumption and internal temperature of the
residence are sent to a remote data bank, which are used for the
proposed solution of energy consumption forecast for three ML
models.
Currently, based on data provided by information in the period from
January 2020 to February 2021.

Each stage of the pipeline has multiple steps, as shown in Fig.

For a satisfactory analysis of two forecasting models, it is
necessary to use metrics that consider shape and time errors such
as Dynamic Time Warping, Move-Split-Merge [22], DILATE [23], or
adjusted error [24].
3. In the offline environment, important tests have been carried out
to prototype the initial model and experiment with new functionalities.
The online pipeline, for its time, simulates the behavior of the
offline environment, with differences in relation to the degree of
automation, execution time restrictions and error handling.

The first stage of the pipeline, which occurs exceptionally in

online environments, is performed through an automated data search,
either in internal databases or through external interfaces, being
necessary or correct treatment of unavailability or transfer errors.

The next stage of pre-processing included the cleaning and

engineering of features, as well as the treatment of anomalies and
missing values performed manually in offline environments.

In offline environments, an exploratory analysis of data is

Fig. 1 Four different predictions F1, F2, F3 and F4 (dotted lines) performed, in which occurs familiarization of two data, detection of
compared to the real value (solid lines). Source: [24]. anomalies, and analysis of the distribution and correlation between
features, in order to refine or advance the processing step.
Among the main features used to improve the accuracy of
consumption forecasts in the short term (horizon of the next few
hours or days) are the climatic data, such as temperature,
precipitation, or wind speed, and references to the calendar, such as
time of day, day of the week, or occurrence of holidays [4], [7], [14].

Most of the two experiments carried out, meanwhile, were in

offline environments, not giving importance to the treatment of
erroneous or incomplete data, apart from the degradation of the
accuracy as the time shifted [14].

IV. PROPOSED SOLUTION

The project is based on the architecture of energy consumption
data collection in [2], implemented at the beginning of 2020 in 4
Brazilian residences. A Fig. 2 shows two implanted gauges.

Fig. 3 Pipeline of online and offline environments for machine learning.

In the next stage, the model is built, defining its hyperparameters,

either manually or automatically by means of a grid search, and
Fig. 2 Smart meter used for data collection installed next to the circuit training according to the available data.
breaker panel. Source: [2].

Finally, in the evaluation of the model and its accuracy

The meters have a data collection system that is tolerant to
measurement, the hyperparameters that optimize the defined metric
connection failures, guaranteeing the integrity of two data during
are selected. Thus, it is important to analyze and choose
network unavailability through the connection

457

Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

which will be the most adequate and relevant metrics to the To analyze or win the introduction of new features
problem. (Test Dice 2: all the features are beneficial and test Dice 3:
Features are not too expensive in memory use), a base reference
In order to measure the preparation of the solution for
model was used, or one that hardly uses or consumes 24 hours
environments in production, we use the metrics provided by Google
ago and data referring to the calendar: time , day of the week and
Research, being the implementation to satisfy each metric
of the month, month and year.
explained in the following sections.
This reference model was compared with other models with
A. Loading two dice
additional features other than those used in the reference model
The input data are received from the storage service in the (test Model 5: a simpler model is not better than the current one),
cloud, or which storage or total energy consumption and by as shown in TABLE II.
sectors, apart from the internal temperature of the residence, with
a sampling frequency of one hour.
Cross-validation was used for each residence, obtaining the
Every hour, searches are made for files in the cloud, if files are mean of the mean quadratic error (MSE) and the error adjusted
found that are not present in the local files, they are downloaded with a 2-hour window and norm 4 [24] of all the residences. TO
from these local directories. TABLE II. also show the reductions
Percentages of the MSE and the adjusted error in relation to the
B. Pre-processing reference model.
During or pre-processing, the presence of missing hours and
gross anomalies is verified. To be considered an anomalous value,
TABLE II. ANALYZE THE ADDITION OF FEATURES IN THE ACURÁCIA DO
the consumption must be less than 0, and the temperature with a MODEL
variation of more than 10 ºC in relation to the previous value, Model MSE Adjusted Error
satisfying the test Data 1: expectations of the features are captured
Reference Model 0.0479 (0%) 0.2535 (0%)
in data schemes [18].
Reference + 1st Derivative 0.0476 (-0.62%) 0.2529 (-0.24%)

There were occurrences of temperature anomalies, in which Reference + 0.0502 (+4.80%) 0.2568 (+1.30%)
its variation in relation to the previous hour exceeded 20 ºC, apart Internal temperature
from moments with missing temperature and consumption readings. Reference + Consumption 0.0415 (-13.36%) 0.2043 (-19.40%)
These cases were attributed as a reading error, discarding these of 25 and 23 hours ago
values.
The addition of the internal temperature of the residence as a
For the definition of features, we have added three
feature of the model ended up reducing its accuracy, while using
Past hourly consumption, referring to 25, 24 and 23 hours ago in
the 1st derivative of energy consumption, obtaining slightly
relation to the moment to be forecast, in addition to attributes
related to the calendar, such as the time of day, day of month, and significant gains. Due to these results, tais features
not foram considered not final model.
month of year to be forecast. Check out these features
For the final model of prediction, it was carried out in the stage of We observed a low weekly correlation for all the residences,
exploratory analysis of two data. not having great variation between days of the week and the end
of the week, as shown in Fig. 4 for one of the residences. It is
Meanwhile, new features can be added by altering two input
necessary to note that the consumption figures refer to the year
files, such as by adding the internal temperature of the residence,
2020, and this low variation may be related to the quarantine period
or also generated by modifying the code-source at the pre-
during the Covid-19 pandemic.
processing stage, such as by adding gives 1st derivative

hourly energy consumption. In this way, the other stages of the

pipeline do not need new modifications (test Dice 6: new features
can be added quickly).

C. Exploratory analysis of two data

In [2], we developed energy consumption forecasting models
using Extreme Gradient Boosting (XGBoost) architectures, long
short-term memory (LSTM) neural networks, and support vector
machines (SVM). The results will show that the XGBoost
architecture obtains better accuracy in most of the monitored
residences, being this architecture chosen to be used in this work.

The XGBoost is an open source ML library for regression

models and classification by means of ensembles
of decision trees [26]. Its implementation allows the training of
models in a parallelized and distributed way.
The models also accept the existence of missing values given to Fig. 4 Box diagram of total consumption per day of week of one of the
us from the start both in the training stage and in the prediction residences
stage.

458

Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

Fig. 5 shows the autocorrelation function of energy parameters, the one with the smallest mean square error is
consumption shown with hourly frequency for one of the monitored chosen.
residences. Not graphic, a higher value not on the ordinates
indicates a high correlation between the temporal series and the
non-temporary series in k units, with k represented by the axis of
the abscissas. It can be observed autocorrelation peaks for delays
of 24 in 24 hours, evidencing daily seasonality.

Fig. 7 Importance of the features for the forecast model of one of the
residences of the project

Fig. 5 Autocorrelation function of hourly energy consumption (band of
significance of 95%)
Fig. 6 shows the 1st decision tree of the model, being possible
to observe the relevance of the time of day in the prediction of the
model, while Fig. 7 shows the importance of the features
for one of the residences, informing the relative contribution of
each feature in the XGBoost tree breeding process. The possibility
of obtaining information related to the internal structure of the
model is important because it allows us to debug the operation
and investigate performance problems or instability (Test
Infrastructure 5: it is possible to carry out the debugging of the
model).
The models are also endorsed with extreme tickets
or even invalid, endorsing its robustness (Monitoring test 5: the
model is numerically stable). The inputs tested are: consumption
equal to zero, negative, infinity and with missing values.
D. Model Training and Prediction
The model uses the XGBoost library to predict the hourly
consumption of the next 24 hours, being trained with the features
of the consumption of 23, 24 and 25 hours ago, at the hour of the
day, day of the week, day of the month, day of the year and month
To perform or adjust two hyperparameters of XGBoost, a
search was performed in the grade (grid search) with cross
validation with partitioning in 4 subsets for each residence (test
Model 3: all hyperparameters are adjusted), varying the
parameters of size of trees, learner taxa, and objective function to
be minimized. apos
or thirty two models for each hyper combination
A random seed used by XGBoost is fixed
automatically, guaranteeing the reproducibility of two results (Test
Infrastructure 1: the training of the model is reproducible).
Both the data and the code are versioned by means of Git
and DVC version control systems (test Model 1: changes are
revised and versioned).
Fig. 8 shows an example of the forecast of energy consumption
for one of the residences made during the month of July, being
possible to observe the daily seasonality of energy consumption.
E. Assessment
The evaluation of the accuracy in a static environment was
carried out by means of the proposed method in [20]. This method
is trained multiple models, each one based on training dice of
different instants, in order to reflect the change of new dice in an
online environment (test Model 4: the aging effects of the model
are known and test Model 6: a quality of the model is sufficient in
all parts of the data).
It is considered a daily training schedule, with data separation
of 80% for training and 20% for tests. The hyper parameters are
defined by means of search in grade, as stated in Section IV.D.
The adjusted error is used to compare the updated model
with the previous one, being used the one with less error
(Infrastructure test 4: the quality of the model is validated before
serving it and Monitoring test 4: models are not old too).

Fig. 6 Decision tree of two forecasting models of XGBoost

459

Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google
Fig. 8 Forecast (vermelho) and real value (preto) of energy consumption for the full month
of July 2020 and for the 1st week of July 2020
The system in an online environment was implemented
as a Flask application on an Apache2 server hosted on
Amazon Elastic Compute Cloud (EC2), periodically retreining
every 24 hours and allowing calls to be received in REST API
format to forecast the consumption of the monitored residences.
The API can be used by other systems for consultation
gives forecast of the consumption of two users. Fig. 9 shows
an example of an application, in which a website was
developed on the Dash platform [27] to make consumption
forecasts in time periods customized by the user.
The calls made to API are monitored at the time of
training, being saved in log files (Test Monitoring 6: the
model did not obtain significant changes in the speed of
training or latency of delivery).
When an abnormal value is found as defined in Section
IV.B (negative consumption or temperature variation greater
than 10 ºC), an alert is added to the log files (Monitoring
test 2: two data invariants are kept both online and offline) .
V. RESULTS
A TABLE III. It shows the average of two results obtained
for all the residences in relation to the MSE metrics and the
adjusted error. In order to compare the effectiveness of the
model update, the metrics are also calculated considering
460
Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
static models without retreat. The values in parentheses refer
to the reduction of the error of the model with updating in
relation to the model without retraining.
Fig. 9 Website to visualize the forecasts made
One of the difficulties in comparing the results between
different works is the use of different methodologies, which
use various databases for model training, methods for defining
hyperparameters and metrics for accuracy analysis. Due to
the scope of work in analyzing or gaining accuracy in using
practical tools for the development of online models, we did
not bother to make a comparison with other jobs in relation to
accuracy, which requires the reproduction of the different
methodologies used for a real analysis , demanding or
retreinamento in new data bases, out of scope of the project.
A TABLE IV. shows the tests performed by the system
following the metrics defined in [18] automatically (A),
manually (M), not performed (-), or not applicable (N/A).
Tests Data 4 and 5 are not applicable to the project in its
current state because it has not collected personal data that
allows us to identify them, due to concerns related to the
privacy of two users. Just test Model 2 is not applied because
there are no online metrics monitored at the moment. The
Infrastructure 6 test is not applicable due to the insufficient
number of users to launch gradual new versions (rollouts) ,
and the Monitoring 3 test has no differences between the
offline and online training data.
Due to the relative low complexity of the pipeline and low
cost of training, no integration tests (test Infrastructure 3) and
rollback tests (test Infrastructure 7) have been performed.
Given that the XGBoost library performs a large series of unit
tests to guarantee the correct execution of the code for
training and prediction of two models, the verification of the
specification of the model was considered as part of the scope
of the project (Infrastructure 2).
TABLE III. METRICS FOR STATIC AND DYNAMIC ENVIRONMENTS
MSE adjusted error
Model without update 0.0495 0.3472
updated model 0.0486 (-1.82%) 0.3089 (-11.03%)
weekly
Machine Translated by Google
TABLE IV. TESTS RELATED TO GIVEN
Test No.
1 3 4 5 6 7
Set
two
Dices AMMN/AN/AAM
Model AN/YYYMM -
Infrastructure A- - AMN/A -
Monitoring - AN/AAMA
Due to having a low number of users at the moment, or the
project still does not pay careful attention to questions of social
inclusion of the system (test Model 7). When new users are
invited to participate, questions of representativeness of the
Brazilian population will be important so as not to be sent to the
system.
At the moment there were no changes in the structure of two
input data, so that the monitoring of changes (test Monitoring 1)
is not currently carried out, but in future stages, in the case of
new features obtained from external sources, such as forecast of
time, maybe inserted, this test will become more important.
Given that the model forecasts consumption for the next 24
hours, or real-time monitoring of the quality of the forecasts made
(Test Monitoring 7) was not done, once its accuracy can only be
measured 24 hours after the forecast.
The exploratory data analysis proved to be of extreme
importance, satisfying various tests (Data 2, Model 5, Infrastructure
5, Monitoring 5) that were previously performed manually, can be
reused in the future for addition to the pipeline, being executed
automatically.
One of the main objections to the project start-up was related
to the large number of functional changes due to the project start-
up, on the assumption that the tests implemented at this stage
would quickly become obsolete. However, this belief was
unfounded, once the simple definition of two tests did not just
verify the correct execution of the code, but also the development
process, following the test-driven development technique (TDD)
[28].
The versioning of data was shown to be important not only
for the online environment, but also in experimentation during the
exploratory analysis of two data, guaranteeing the reproducibility
of experiments carried out in previous versions.
During the development it was necessary to balance the
delivery of results with the execution of tests, so that in defining
priorities it was extremely important not to run the project. A
selection of priorities was calculated according to the probability
of related problems occurring, as well as the impact of these
problems on the system.
Another key point for the execution of two tests was the
modularization of the stages of the pipeline. By accurately defining
its functionalities, expected inputs and outputs, it becomes easier
to alter the source code and experiment with new configurations,
visualizing more clearly the impact of these changes in the project.
SAW. CONCLUSION
Neste artigo foram presented the details of implementation,
the difficulties encountered in evaluating
Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
metrics proposed in the literature, and the positive and negative
points in their due use, not development of systems
of online machine learning, through a case study using real data
of energy consumption of 4 residences in the period of 2020 in
order to approximate a real scenario found in the industry.
-
The results also show the difficulty in defining metrics that
capture the relevant characteristics in the domain of the energy
consumption prediction problem, especially the need to consider
both spatial and temporal errors. Other difficulties encountered
during the development of the system were also considered, and
the impact of these difficulties were quantified from experiments
in dynamic scenarios, we were able to analyze the impact of the
model training, reducing the adjusted error by 11.03% in relation
to a model static.
As future work, the application of the system in a larger
number of users can help in the validation of its scalability, also
taking into account the representation of the Brazilian population
in the selection of new testers to analyze the performance of two
models for different consumption profiles. Além of the residential
scope focused on this work, the application of the architecture of
MLOps proposed power
also contribute to the prediction of consumption in industrial or
commercial establishments, being a possible future scenario to
analyze its viability, gains for the client, consumption peculiarities
and make a comparison
com or residential sector.
ACKNOWLEDGMENTS
The authors thank the patrimonial fund “Amigos da
Poli” for financial support.
REFERENCES
[1] K. Carrie Armel, A. Gupta, G. Shrimali, and A. Albert, “Is
disaggregation the holy grail of energy efficiency? The case
of electricity,” Energy Policy, vol. 52, p. 213–234, 2013.
[two] V. Hayashi, R. Arakaki, T. Fujii, K. Khalil, and F. Hayashi, “B2B
B2C Architecture for Smart Meters using IoT and Machine
Learning: a Brazilian Case Study,” International Conference on
Smart Grids and Energy Systems, p . to be published, 2020.
[3] V. Hayashi, T. Fujii, R. Arakaki, H. Amaral, and A. Souza, “Boa Energia:
Public Database of Residential Consumption with Data Quality,” 2020.
[4] S. Humeau, TK Wijaya, M. Vasirani, and K. Aberer, “Electricity
load forecasting for residential customers: Exploiting aggregation
and correlation between households,” 2013 Sustainable Internet
and ICT for Sustainability, SustainIT 2013, 2013.
[5] M. Martins PB, RGD Pinto, and SP Bittencourt, “Load Disaggregation of
Industrial Machinery Power Consumption Monitoring Using Factorial
Hidden Markov Models,” The International Workshop on Non-Intrusive
Load Monitoring (NILM), p. 6, 2018.
[6] W. Kong, ZY Dong, Y. Jia, DJ Hill, Y. Xu, and Y. Zhang,
“Short-Term Residential Load Forecasting Based on LSTM
Recurrent Neural Network,” IEEE Transactions on Smart Grid,
vol. 10, no. 1, p. 841–851, 2019.
[7] P. Lusis, KR Khalilpour, L. Andrew, and A. Liebman, “Short
term residential load forecasting: Impact of calendar effects
and forecast granularity,” Applied Energy, vol. 205, no. March,
p. 654–669, 2017.
[8] S. ben Taieb and RJ Hyndman, “A gradient boosting approach
to the Kaggle load forecasting competition,” International Journal
of Forecasting, vol. 30, no. 2, p. 382–394, 2014.
[9] P. Serrenho, T., Bertoldi, “Smart home and appliances: State of
the art,” Luxembourg, 2019.
[10] HLMD Amaral, JAG Maginador, RMJ Ayres, AN de Souza, and DS
Gastaldello, “Integration of consumption forecasting in smart meters and
smart home management
461
Machine Translated by Google
systems,” SBSE 2018 - 7th Brazilian Electrical Systems Symposium,
pp. 1–6, 2018.
[eleven] Y. Wang, Q. Chen, T. Hong, and C. Kang, “Review of Smart Meter
Data Analytics: Applications, Methodologies, and Challenges,”
IEEE Transactions on Smart Grid, Vol. 10, no. 3, p. 3125–3148,
2019.
[12] W. Tushar, TK Saha, C. Yuen, P. Liddell, R. Bean, and HV
Poor, “Peer-to-Peer Energy Trading With Sustainable User
Participation: A Game Theoretic Approach,” IEEE Access, vol. 6, no.
October, p. 62932–62943, 2018.
[13] A. Pratt, D. Krishnamurthy, M. Ruth, H. Wu, M. Lunacek, and P.
Vaynshenk, “Transactive Home Energy Management Systems: The
Impact of Their Proliferation on the Electric Grid,” IEEE Electrification
Magazine, vol. 4, no. 4, p. 8–14, Dec. 2016.
[14] A. Gerossier, R. Girard, A. Bocquet, and G. Kariniotakis, “Robust day-
ahead forecasting of household electricity demand and operational
challenges,” Energies, vol. 11, no. 12, 2018.
[fifteen] M. Senapathi, J. Buchan, and H. Osman, “DevOps Capabilities,
Practices, and Challenges,” in Proceedings of the 22nd International
Conference on Evaluation and Assessment in Software Engineering
2018, Jun. 2018, pp. 57–67.
[16] D. Sculley et al., “Hidden technical debt in machine learning systems,”
Advances in Neural Information Processing Systems, vol. 2015-
Janua, pp. 2503–2511, 2015.
[17] Google Cloud, “MLOps: Continuous delivery and automation pipelines
in machine learning,”
learning/mlops
https://cloud.google.com/solutions/machine-
pipelines-in-machine 2020.
continuous-delivery-and-automation-
learning.
[18] E. Breck, S. Cai, E. Nielsen, M. Salib, and D. Sculley, “The ML test
score: A rubric for ML production readiness and technical debt
reduction,” in 2017 IEEE International Conference on Big Data (Big
Data ), Dec. 2017, vol. 47, no. 3, p. 1123–1132.
462
Authorized licensed use limited to: LIVERPOOL JOHN MOORES UNIVERSITY. Downloaded on August 13,2022 at 06:16:40 UTC from IEEE Xplore. Restrictions apply.
[19] P. Sugimura and F. Hartl, “Building a reproducible machine learning
pipeline,” arXiv, 2018.
[twenty] JA Guajardo, R. Weber, and J. Miranda, “A model updating strategy
for predicting time series with seasonal patterns,” Applied Soft
Computing Journal, vol. 10, no. 1, p. 276–283, 2010.
[twenty-one] Y. Wang, Q. Chen, T. Hong, and C. Kang, “Review of Smart Meter
Data Analytics: Applications, Methodologies, and Challenges,”
IEEE Transactions on Smart Grid, Vol. 10, no. 3, p. 3125–3148, 2019.
[22] A. Stefan, V. Athitsos, and G. Das, “The move-split-merge metric for
time series,” IEEE Transactions on Knowledge and Data Engineering,
vol. 25, no. 6, p. 1425–1438, 2013.
[23] V. le Guen and N. Thome, “Shape and Time Distortion Loss for
Training Deep Time Series Forecasting Models,” no. NeuroIPS, pp.
1–13, 2019.
[24] S. Haben, J. Ward, D. Vukadinovic Greetham, C. Singleton, and P.
Grindrod, “A new error measure for forecasts of household level, high
resolution electrical energy consumption,”
International Journal of Forecasting, vol. 30, no. 2, p. 246–256, 2014.
[25] R. Arakaki, VT Hayashi, and WV Ruggiero, “Available and Fault
Tolerant IoT System: Applying Quality Engineering Method,” 2nd
International Conference on Electrical, Communication and Computer
Engineering, ICECCE 2020, no.
June, 2020.
[26] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,”
Proceedings of the ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining, vol. 13-
17-August, pp. 785–794, 2016.
[27] Plotly Technologies Inc., “Dash.” Plotly Technologies Inc., Montreal,
QC.
[28] K. Beck, Test Driven Development: By Example, 1st ed. Addison
Wesley Professional, 2002.