2014 IEEE International Conference on Big Data
On the Impact of Socio-economic Factors on Power Load Forecasting
Yufei Han Xiaolan Sha
GridPocket Eurecom
Abstract
In this paper, we analyze a public dataset of elec-
tricity consumption collected over 3,800 households
for one year and half. We show that some socio-
economic factors are critical indicators to forecast
households’ daily peak (and total) load. By using a
random forests model, we show that the daily load can
be predicted accurately at a fine temporal granularity.
Differently from many state-of-the-art techniques based
on support vector machines, our model allows to derive
a set of heuristic rules that are highly interpretable and
easy to fuse with human experts domain knowledge.
Lastly, we quantify the different importance of each
socio-economic feature in the prediction task.
1. Introduction
Electricity load forecasting is one of the main
challenges in smart meter technology. Accurate load
forecasting at various temporal granularity can help
utilities design flexible energy supply strategies (e.g.,
by applying load switch schemes [1]) to meet the
supply-demand balance, and directly benefit individual
households by offering a transparent view on energy
consumption, thus enabling learning mechanisms to
compare and eventually reduce energy waste [2], [3],
[4], [5], [6].
Energy load forecasting requires understanding
individual behavioral patterns, to build models that
predict accurately future energy consumption. Many
studies used time series modeling techniques [2], [5],
[7], [8], [9], [10] to achieve such goal, while little
attention has been devoted to understand the impact
of socio-economic factors on the forecasting task. In
this paper, we address this challenge, and explicitly
introduce socio-economic factors into the forecasting
task to overcome the limitation of existing approaches.
Indeed, time series modeling techniques often require
978-1-4799-5666-1/14/$31.00 ©2014 IEEE
Etta Grover-Silva Pietro Michiardi
GridPocket Eurecom
a large number of data points to make statistical
inference, which hinders the forecasting task due to
the lack of historical information for households that
are not equipped with smart metering solutions. We
note that energy consumption is a direct product of
(collective) human behavior, which in turn depends on
several characteristics including demographic informa-
tion [11], [12], [13], and socio-economic factors.
In this work, we focus on two load forecasting
tasks: 1) daily peak load and 2) daily total electricity
consumption. We first propose a variety of predictors
ranging from meteorological to socio-economic fea-
tures in Section 3. We integrate such features in a
novel approach to the forecasting task, which is based
on random forests. Then, we compare in Section 4
our approach to a state-of-the-art modeling technique
based on support vector machines (SVM). Our results
pinpoint an improved prediction accuracy, and illus-
trate that random forest models allow deriving rule sets
that can be easily interpreted and fused with human
domain knowledge.
2. Background and Related Work
Most research work on load forecasting models can
be categorized into three groups – time-series based
models, end user models and econometric models.
Time-series models are often designed to identify
temporal causality of the energy demand between
past and future [4], [5], [6], [7], [10], [14], [15],
[16]. With such models, the task of forecasting is
accomplished by following the learned functional map-
ping between past and future energy usage patterns.
Popular machine learning algorithms are applied in
these models, such as autoregressive integrated moving
average model (ARIMA) [16], support vector machines
(SVM) [4], [15] and neural networks [10], [14]. These
algorithms serve as black-box tools to capture the
temporal dynamics in energy consumption with high
accuracy. However, they often offer no insights on the
742
underlying phenomenon behind the time evolution of
energy demand, and are difficult to generalize.
To overcome lack of interpretability, an alternative
is to build end-user models [1], [2], [3], [5], [9],
[17]. The principle of such models is to disaggregate
daily energy consumption into elementary components
– including heating/cooling, water usage, cooking and
other behaviors – which are used to interpret the
temporal variation of the energy demand from each
household. End-user models can easily explain the re-
lation between predictions and household information.
However, their performance highly depends on the
quality of available information, which makes them
sensitive to noise.
Another group of popular models to forecast energy
load are econometric models [2], [3], [5], [12], [13],
[17], which combine the two techniques described
above. These models identify the main factors which
influence the consumption behavior of households,
and learn the mapping between key factors and the
energy consumption profiles. Though these models are
appealing, they require human intervation to tune the
input factor set.
To the best of our knowledge, the integration and
ranking of socio-economic features for load forecasting
through automatic learning from energy usage data has
remained elusive in the literature. In this paper, we
design a methodology to automatically select socio-
economic factors to build an energy forecast model.
In doing so, we unveil which are the most relevant
features to accurate predictions, and which can be
safely dismissed as they are of little contribution to
the predictive power of our model.
Our forecasting model is based on random forests,
which we briefly overview here. The central idea of
random forests is to construct a set of independent
decision trees by bootstrapping training data. The
output of a random forest is obtained, e.g., through
majority voting or with a simple average of the output
of each individual tree model. Although bootstrap
sampling introduces random bias into each decision
tree as byproduct, the voting scheme reduces variance
of tree output, which compensates the random bias thus
improves the model fitting accuracy. Each decision tree
in a random forest is a tree-like rule chain based on
input variables for classification or regression. Each
rule is a branch-split operation comparing one input
variable with a predefined threshold. The hierarchical
split-branch operations form a coarse-to-fine white box
model, which can explain explicitly how input factors
743
are combined to achieve a complete decision making
procedure.
3. Load Forecasting using Socio-
economic Factors
We now present the public dataset used in this work
and describe the three types of features that are used in
our forecasting models – temporal, environmental and
socio-economic factors. As a general remark, we note
that the volume of the available training samples in our
dataset offers a large coverage of energy and socio-
economic factors, which improve the generalizability
of our model.
The dataset. The CER ISSDA dataset is a publicly
available energy consumption trace [18], containing
electricity consumption data of 4,225 private house-
holds and 485 small / medium enterprises; the trace
covers 1.5 years (from July 2009 to December 2010).
For each customer, the daily load curve is sampled
every 30 minutes and comprises a series of surveys
describing their socio-economic factors. 41 survey
questions belonging to six categories were carefully
selected as a complete list of features for our statistical
model. They cover household profiles, chief owner
income level, residential behavioral related features,
housing conditions, properties of main electrical appli-
ances (heating and cooking devices) and information
about other auxiliary electrical appliances, which are
commonly recognized as influencing factors in energy
consumption behavior. After data curation, our dataset
covers 3,822 households.
Predictors used in the forecasting task. In addi-
tion to the socio-economic factors outlined above, we
include environmental features, which are significant
for energy consumption profiling [14]. The first two
are heating degree days and cooling degree days,
evaluating quantitatively the needs to start heating
or air conditioning appliances to keep an adequate
environmental temperature. We also consider the daily
average humidity, which represents the average air
humidity level during a one-day interval. Finally, we
include month of year, day of week and holiday index:
the first feature represents the seasonal weather change,
directly and strongly affecting energy usage profiles of
all involved customers; the others features differentiate
occupancy patterns of residential customers.
Statistical profile of the load forecast task. We
now build the statistical profile of our dataset, using
well-known feature ranking statistics to provide formal
 Prediction of daily peak loads
Total number of occupants
Number of occupants > 15
years old
Whether there is a dishwasher
Whether there is a tumble dryer
Number of occupants < 15
years old
Whether there is an electric
cooker
Number of bedrooms
Type of the cooking appliances
Average age of occupants
Employment status
Prediction of daily total en-
ergy usage
Total number of occupants
Number of occupants > 15
years old
Whether there is a dishwasher
Number of bedrooms
Whether there is a tumble dryer
Number of occupants < 15
years old
Whether there is a stand-by
freezer
Average age of occupants
Type of the house
Employment status
indicates that house category and usage of stand-alone
freezer devices are key factors for the daily total energy
consumption, but are weakly related to the daily peak
load. Such results follow common sense: the type of
the house is related to the thermal performance and
inertia of the house, which crucially affects the total
amount of energy – required to maintain a desirable
temperature in the house. Next, we show that a data-
driven model based on socio-economic factors can
indeed find associations between user profiles and their
consumption behavior.

Table 1: Top 10 factors in prediction tasks using mutual 4. Experimental Evaluation

information criterion

We now report our experimental results, where we

grounding to the key idea of this work, i.e., that socio-
economic factors are important predictors for a fore-
casting model. Thus, we use the mutual information
criterion1 [19] to compute the correlation between each
input factor and the forecasting target: the larger the
mutual information, the higher the correlation between
the two random variables. In practice, we use the
Kullbeck-Leiber divergence of the product of marginal
distributions of two random variables x and y from
their joint distribution:
 
XX p(x, y)
I(X; Y ) = p(x, y) log (1)
p(x) p(y)
y∈Y x∈X
Tab. 1 reports the top-10 factors (sorted in de-
scending order of mutual information), that are mostly
related to daily peak loads and daily total energy
usage respectively. Occupancy attributes, the number
of bedrooms, age of the clients, employment status
of the clients, usage of tumble dryer and usage of
dishwashers, are the common key elements influencing
both peak loads and total amount of energy consump-
tion, which are related to clients consuming capacities,
potential electricity needs to keep proper temperatures
and daily housework.
For the purpose of the forecasting task, our analysis
indicates that cooking appliances – because they con-
sume a lot of energy – largely contribute to the mag-
nitude of the daily peak energy consumption. Instead,
such appliances only marginally affect the the daily
total energy consumption, as their usage is restricted
to a small interval of time. In contrast, our analysis
1. Note that the mutual information is not used to rank predictors
for the purpose of feature selection.
evaluate the performance of our random forests model
to address the load forecasting task, and compare it to a
state-of-the-art technique based on SVM. In addition,
we present an analysis of the importance ranking of
socio-economic and environmental factors.
4.1.Comparison of Forecasting Models
For each of the 3,822 users and for the 1.5 year
duration of our dataset, we extract daily peak load
measurement (in KiloWatt) and daily total electricity
consumption (in KiloWattHour) to build the target
features of our model. Input predictors are the 41
features described in Sec. 3. Therefore, we obtain
1,615,541 training samples, containing the input-output
pairs for each of the forecasting tasks.
Metrics. The forecasting accuracy is measured using
the coefficient of determination R2 [19] defined by the
ratio between the sum of square regression residual
error and total sum of squares of the forecasting target.
Large values of R2 indicate an agreement between the
model and the underlying real output, which translates
in higher accuracy. Note that for really biased predic-
tions, the coefficient can be zero or negative, while it
is upper bounded by 1.
Forecasting models. The main parameter of random
forests is the number of trees constructed in the
ensemble model to perform a final vote. We vary
this parameter in the range {50, 100, 200, 300, 400},
to study the stability of forecasting performance. Once
the random forest model is constructed, the out-of-
bag error [19] of the random forest is used directly
as the estimation of the model generalization error.
Note that the split-branch operation in the random

744
 Model Peak Load: R2 Total Load: R2
Peak load level in KiloWatts
RF(50) 0.5214 0.7151
RF(100) 0.5240 0.7261
RF(200) 0.5127 0.7236
RF(300) 0.5208 0.7100
RF(400) 0.5157 0.7070
SVM 0.4816 0.6572
Table 2: Generalization error of peak load and total
electricity consumption prediction, with the Random
Forest Model (RF) and SVM. Results for the RF model
include values for the tree parameter (in parenthesis)
we used.
forest constructs a piecewise linear regression model,
which can handle non-linearly distributed data without
introducing additional algorithmic components.
Since the training set consists in more than 1.8
million records, a standard approach such as SVM
can not afford the construction of a huge kernel
matrix to enforce a non-linear regression in memory:
hence, we use a linear kernel SVM [20]. The training
configuration of SVM is carefully selected through a
5-fold cross-validation. After fixing the configuration
parameter, the SVM training ends by providing a linear
regressor: 5-fold cross-validation is employed again to
estimate its generalization error.
Overall, the computational complexity of building
the random forest model is O(mn log(n)), where m is
the number of trees, n is the number of bootstrapped
training samples to construct each tree in the model.
Considering that m is usually much smaller than n,
the complexity of random forest is comparable to or
smaller than the quadratic programming cost of SVM,
which is between O(n2 ) and O(n3 ).
Results. Tab. 2 summarizes the generalization error
for our method and SVM. In general, our modeling
approach outperforms SVM: the highest accuracy is
achieved with an ensemble of 100 trees, for both
forecasting tasks. Indeed, although more trees can
reduce variance, they might increase the model bias.
There are two main reasons for the superiority
of our approach. First, SVM treats categorical inputs
as numerical variables, which loses the categorical
information. Furthermore, due to the computational
complexity and memory constraints, the selected SVM
is linear, thus it cannot capture the non-linear relation
between socio-economic features and energy consump-
tion profiles. In contrast, the random forest method
does not suffer from such limitations.
745
Peak loads prediction Daily total consumption prediction
Daily total consumption in KWH
10 15 20 25 30 35
6
5
4
3
2
1
5
0 5 10 15 20 25 30 0 5 10 15 20 25 30
Day Day
(a) Prediction of daily peak (b) Prediction of daily to-
load of one user during one tal consumption usage of one
month. user during one month.
Figure 1: An example of energy consumption indicator
forecasting.
For illustrative purposes, Fig. 1a and 1b show two
instances of the forecasting task for daily peak load
and daily total consumption respectively: the solid line
corresponds to the ground truth of daily peak load
or daily total consumption measurements, while the
dashed line represents the estimated values using the
random forest model. These figures reveal that our
method achieves higher predictive power for the total
daily consumption than for the daily peak load, which
confirms our findings in Tab. 2. Indeed, peak loads are
easily affected by householders’ occupancy duration,
appliances’ working status, and in general, time-of-day
related behavior. Such effects inject randomness into
daily peak load patterns, which are hardly reflected
comprehensively in the questionnaire. In contrast, the
daily total consumption, as a cumulated sum of energy
consumption measurements, is less prone to random
fluctuations due to time-of-day behavior. As a conse-
quence, the model built with the approach contains a
strong association to the static socio-economic features
available in the dataset.
4.2.Importance Ranking of the Socio-economic
and Environmental Factors
Our method generates a quantitative score for each
input factor to evaluate its association importance with
daily energy usage profiles. All input factors are ranked
by the importance score in a descending order. The
mean, along with standard deviation of each feature
importance score is estimated. For all top 10 factors,
the standard deviations are at least three times smaller
than the mean values. This indicates the ranking result
is statistically stable.
Our experimental results are summarized in Tab. 3,
which reports the top-10 input factors for predicting
 Prediction of daily peak loads
Total number of occupants
Number of occupants > 15
years old
Type of the cooking appliances
Whether there is an electric
cooker
Average daily heating degree
days
Whether there is a tumble dryer
Whether there is a dishwasher
Type of the appliances to heat
water in the house
Whether there is an instant elec-
tric shower facility
Whether there is an electric
heater
Table 3: Top 10 factors in prediction tasks.
Prediction of daily total en-
ergy usage lack of accurate temperature data that affects our study,
Total number of occupants since (due to privacy reasons) it was not possible to
Number of occupants > 15 join our temperature dataset to specific geographical
years old coordinates corresponding to the households in our
Average daily heating degree
days study. Overall, the ranking produced by the random
Number of bedrooms forest method is in line to that obtained through a
manual inspection of the data by domain experts.
Whether there is a dishwasher
Month of year
Whether there is a tumble dryer
Whether there is an electric
heater 5. Conclusion
Whether there is a stand-by
freezer
Type of the appliances to heat
water in the house
In this paper, the issue of forecasting daily elec-
tricity usage patterns and evaluating the influence of
clients’ socio-economic factors on energy consumption
has been addressed.

peak loads and total energy consumption. The ranking
obtained is consistent to that obtained using the mutual
information criterion in Sec. 3. The number of occu-
pants is ranked as the most important feature, affecting
both daily peak load level and the total amount of
energy consumption, which follows the intuition of
a direct relation between household size and energy
consumption. Main appliances, such as cooking and
heating devices, are common features for the two
forecasting tasks. Indeed, they contribute to a large
extent to instantaneous peaks in energy consumption,
and are generally responsible for aggregate energy con-
sumption as well. Appliances that have a daily cycle
(e.g. freezers) and average daily heating degree days
are specially useful for estimating daily consumption,
rather than representative of peak demand.
We remark that, for peak load forecasting, the
random forest method boosts the importance of the
water heating appliances, while it completely neglects
indirect factors, such as the number of bedrooms,
social class of the chief income and employment status.
For total daily energy consumption, the random forests
method assigns higher importance scores to factors
related to heating usage – including environmental
heating degree index, month of year, the year when
the house is built, the number of bedrooms – while
it decreases the importance of social class, age and
employment status of the owner. Interestingly, environ-
mental temperature seems not play an important role
in both forecasting tasks, which is counter intuitive.
A possible reason for this result is related to high
heating loads at all times due to Irish weather status.
Furthermore, such results can also be explained by the
A forecasting method based on random forest that
seamlessly incorporates both clients’ socio-economic
factors and environmental factors has been proposed,
resulting in an ensemble of split-branch decision rule
chains, whereby a voting mechanism is used to achieve
a stable prediction. The rule-chain based structure
enabled the explaination of how each input features
contributes to energy consumption forecasting, thus
unveiling the underlying physical association between
predictors and the daily energy usage pattern. These
learned associations can either be used to discover
unknown factors of energy consumption behaviors,
or they can be applied as a complementary decision
support to human experts.
Experimental results based on a large-scale energy
consumption dataset showed that our methodology
dramatically outperforms state-of-the-art approaches,
both in terms of prediction accuracy and in terms
of scalability. Additionally, our work evaluated the
importance of socio-economic factors in the fore-
casting tasks. Our result can guide the construction
of socio-economic surveys without requiring human
intervention, leading to a decreased intrusiveness of
measurement campaigns.
As shown in the experiments conducted in this
work, daily peak load forecasting is a difficult task,
compared to aggregate energy consumption. As such,
an extension of this work will build detailed usage
patterns of large power appliances that play a key role
in determining peak loads and will integrate historical
peak load information to extend the input set of the
forecasting model.

746
References
[1] Aman, S., Simmhan, Y., and Prasanna,
V.K.,“Improving Energy Use Forecast for Campus
Micro-grids using Indirect Indicators”, in Proceedings
of the International Workshop on Domain Driven
Data Mining (DDDM), pp.1-9, 2011.
[2] Tennesee Valley Authority, “Energy Vision 2020: In-
tegrated Resource Plan/Environmental Impact State-
ment”, Chapter 6, December, 1995.
[3] Kolter, Z. and Ferreira, J., “A large-scale study on
predicting and contextualizing building energy usage”,
in Proceedings of the Twenty-Fifth AAAI Conference
on Artificial Intelligence, pp.330-338, 2011.
[4] Centra, M., “Hourly Electricity Load Forecasting: An
Empirical Application to the Italian Railways”, World
Academy of Science, Engineering and Technology,
vol.5, pp.888-895, 2011.
[5] Hesham K.A. and Nazeeruddin, M., “Electric load
forecasting: literature survey and classification of
methods”, International Journal of Systems Science,
vol.33, pp.23-34, 2002.
[6] Humeau, S.F.R.J., Wijaya, T. K., Vasirani, M. and
Aberer, K.,“Electricity Load Forecasting for Residen-
tial Customers: Exploiting Aggregation and Correla-
tion between households”, Sustainable Internet and
ICT for Sustainability (SustainIT), pp.1-6, 2013.
[7] Aung, Z., Touky, M., Williams, J.R., and Herrero,
S., “Towards Accurate Electricity Load Forecasting
in Smart Grids”, in Proceedings of The Fourth In-
ternational Conference on Advances in Databases,
Knowledge, and Data Applications, pp.52-57, 2012.
[8] Miranda, V., and Monteiro, C., “Fuzzy Inference Ap-
plied to Spatial Load Forecasting”, in Proceedings of
International Conference on Electric Power Engineer-
ing, pp.35-40, 1999.
[9] Fu, C.W., and Nguyen, T.T, “Models for long-term
energy forecasting”, in Proceedings of IEEE Power
Engineering Society General Meeting, pp.235-239,
2003.
[10] Kermanshahi B. S., and Iwamiya H., “Up to year 2020
load forecasting using neural nets”, Electric Power
System Research, Elsevier, vol.24, pp.787-797, 2002.
747
[11] McLoughlin, F., Duffy, A., and Conlon,M., “Char-
acterising domestic electricity consumption patterns
by dwelling and occupant socio-economic variables:
An Irish case study”, Energy and Buildings, vol.48,
pp.240-248, 2012.
[12] Beckel, C., Sadamori, L., and Santini,S.,“Towards
automatic classification of private households using
electricity consumption data”, in Proceedings of the
Fourth ACM Workshop on Embedded Sensing Systems
for Energy-Efficiency in Buildings, pp.169-176, 2012.
[13] Beckel, C., Sadamori, L., and Santini, S., “Auto-
matic socio-economic classification of households us-
ing electricity consumption data”, in Proceedings of
the fourth international conference on Future energy
systems, pp.75-86, 2013.
[14] Taylor, J.W., and Buizza,R., “Neural Network Load
Forecasting With Weather Ensemble Prediction”,
IEEE Transactions on Power Systems, vol.17, pp.626-
632, 2002.
[15] Mohandes, M., “Support vector machines for short-
term electrical load forecasting”, International Journal
of Energy Research, vol.26, pp.335-345, 2002.
[16] El Desouky, A., and Elkateb, M., “Hybrid adaptive
techniques for electric-load forecast using ANN and
ARIMA”, in Proceedings of Generation, Transmission
and Distribution, vol.147, pp.213-217, 2000.
[17] Abreu, J., and Pereira, F., “Household Electricity
Consumption Routines and Tailored Feedback”, in
Proceedings of ACEEE Summer Study on Energy
Efficiency, pp.193-206, 2012.
[18] “Eletricity Customer Behaviour Trial”, Commission
for Energy Regulation, Ireland,2011.
[19] Bishop, C.M., “Pattern Recognition and Machine
Learning”, Spring Verlag, 2006.
[20] Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R.,
and Lin, C.J., “LIBLINEAR: A library for large
linear classification”, Journal of Machine Learning
Research, vol.9, pp.1871-1874, 2008.