2018 1st International Conference on Data Intelligence and Security
Detection of Electricity Theft in Customer
Consumption using Outlier Detection Algorithms
Jaime Yeckle
Department of Electrical and Computer Engineering
Mississippi State Unviersity
Mississippi State, MS
jhy31@msstate.edu
Abstract—Advanced Metering Infrastructure (AMI) is a core
part of Smart-grid, which is responsible for collecting, measuring
and analyzing energy usage data of customers. The development
of this network has been possible thanks to the emergence of new
information and communication technologies. However, with the
arrival of these technologies, new problems have arisen in the
AMI. One of these challenges is the energy theft, which has
been a major concern in traditional power systems worldwide.
To face these challenges, datasets of electricity consumptions are
analyzed to detect intruders. Traditional techniques to detect
intruders include the use of machine learning and data mining
approaches. In this paper, we analyze the feasibility of applying
outliers detection algorithms for enhancing the security of AMI
through of the detection of electricity theft. We explore the
performances of various existing outlier detection algorithms on
a real dataset (consumer energy usage). The results show the
feasibility of use outliers algorithms in the security of AMI and
also the effectiveness of the use of these methods in the electricity
consumption datasets for theft detection.
Index Terms—Outliers detection, electricity theft detection,
advanced metering infrastructure, smart grid
I. I NTRODUCTION
Smart-grid incorporates computer intelligence and sensors
into the power system. This includes the use of smart meters
in the AMI subsystem. However, with the intelligence incor-
porated, new security concerns are raised. One of them is the
electricity theft that has been the long-standing issue to utilities
as it can cause billions of dollars worth in financial losses [1].
The methods of stealing electricity include tampering meters
[2] and connecting unregistered appliances to the power grid.
Many efforts are committed to addressing electricity theft
issues using a variety of approaches. Traditionally, pattern
recognition and data-mining techniques are used over histori-
cal electricity usage to detect electricity theft [3] [4].
In this paper, we analyze the feasibility of applying out-
liers detection algorithms for enhancing the security of AMI
through the detection of electricity theft in a variety of types.
Our focus in this study is to detect normal and malicious
customers based in the analysis of consumption patterns in the
AMI subsystem. In the past, many classification algorithms
were used to train a classifier based on a sample database,
which is then utilized to find abnormal patterns, but, to our
978-1-5386-5762-1/18/$31.00 ©2018 IEEE
DOI 10.1109/ICDIS.2018.00029
Bo Tang
Department of Electrical and Computer Engineering
Mississippi State Unviersity
Mississippi State, MS
tang@ece.msstate.edu
best knowledge, multiple outliers detection algorithms have
not been used. Our approach includes the use of seven outliers
detection algorithms to detect abnormal pattern consumption;
also, a preprocessing of the data using k-means clustering
algorithm is performed with the objective of reducing the
number of measurement samples. The validation is performed
by analyzing the electricity consumption of five customers,
which include seven different types of electricity theft. After
comprehensive experiments, a feasibility study is performed to
analyze the use of these existing outliers detection algorithms
as an improvement to AMI security. The study will help
future researchers to apprehend and extend existing outliers
detection algorithms for building robust IDSs, which are used
in various components of AMI and other critical systems. The
contributions of this research are as follows:
1) We have conducted a set of experiments on a public data
set using state-of-the-art outliers detection techniques
and compared their performances.
2) Seven types of electricity theft are generated to validate
the outlier detection algorithms.
3) We have performed a feasibility study of applying these
outliers algorithms in the detection of electricity theft in
AMI.
II. R ELATED W ORK
In recent years, the emergence of Smart-grid has motivated
research into a variety of intrusion detection techniques. One
approach is exploring Machine-Learning Methods (MLMs)
[5] to detecting anomalies, such as neural networks, support
vector machines, K-nearest neighbor and Hidden Markov
model. However, inaccurate models can lead to false alarms
and/or missed detections. Also, in the literature we find the
outliers detection techniques which can be applied to detect
malicious attempts to manipulate the data from most nominal
cases. Outliers detection techniques can be categorized into
four groups: statistical, distance based, clustering methods
and density-based approaches. In statistical techniques [6]–
[8], the data points are typically modeled using a stochastic
distribution, and points are labeled as outliers depending
on their relationship with the distributional model. Distance
based methods [9]–[11] use solely the distance space to flag
outliers. The cluster-based methods [12], [13] detect outliers
135
in the process of finding clusters. In density-based methods,
an outlier is detected when its local density differs from its
neighborhood. Different density estimation methods can be
applied to measure the density. Local Outlier Factor [17] is
an outlierness score which use a nave density estimation. A
modification to the density estimation with a variable kernel to
yield a robust local density estimation is proposed in [18]. In
[19] the authors use a classic kernel density estimation directly
instead of experimenting with non-standard kernels. Kernel
also is used in [21] but there the authors include the k-nearest
neighbors, reverse nearest neighbors and the shared nearest
neighbors as an improvement in the density estimation.
On the other hand, many approaches are proposed to detect
energy theft using data mining and machine learning algo-
rithms. For example, in [1], [14], the authors use historical
consumption data along with SVM classifier to detect abnor-
mal behaviors. Also, the use of SVM and parallelization to
reduce the detection time is used in [15]. In [16], a neural
network model was incorporated to estimate SVM parameters
in order to detect energy theft attacks that result in zero usage
reports. In this paper, we use outlier detection techniques
(OD) in the detection of electricity theft in AMI system. To
our best knowledge, OD techniques have not been applied to
evaluate the performance in the detection of energy theft using
consumption data of smart meters in AMI system.
III. A LGORITHMS E VALUATED
A. Outliers Detection Algorithms
We implemented seven state-of-art traditional outliers de-
tection algorithms which are described in this section.
Local Outlier Factor (LOF) [17]. This algorithm takes into
account both the density of the given instance and the density
of the data instances in the k-nearest neighbor set of point. The
outliers are detected by comparing the local density of each
point to the local density of its neighbors. We must highlight
that the output factor depends on the choice of k and also that
the LOF computation performance is high.
Local Density Factor (LDF) [18]. This technique merges
LOF concepts and kernel density estimation. To estimate the
density, use non-standard kernels to yield a robust local density
estimation. However, we must highlight, that LDF ignores the
theoretical foundations of kernel density estimation, which in
some cases is a disadvantage.
Flexible Kernel Density Estimates (KDEOS) [19]. This
method estimates the density using standard kernels and
applying the z -score transformation for score normalization.
The result is a combination of existing experience in kernel
density estimation with the ideas of local and density-based
outlier detection methods, preserving the strengths of both.
Influenced Outlierness (INFLO) [20]. This technique pro-
poses a simple but effective measure on local outliers based on
symmetric neighborhood relationships. The proposed measure
considers the k-nearest neighbors and reverse neighbors of an
object when estimating its density distribution.
Relative Density-based Outlier Score (RDOS) [21]. This
method is a combination of KDEOS and INFLO. When the
136
density is estimated, the method considers the use of standard
kernels and (1) the k-nearest neighbors (knn), (2) the reverse
nearest neighbors (rnn), and (3) the shared nearest neighbors
(snn). The snn of a object ’p’ are those objects who share one
or more nearest neighbors with ’p’.
Mutual k-nearest neighbor (MNN) [12]. This approach
studies the relationship between the connectivity of a mutual
k-nearest-neighbor graph and the presence of clustering struc-
ture and outliers in the data. Each connected component is
considered as a cluster if it contains more than one vector and
an outlier when the connected component contains only one
vector.
Indegree Number(ODIN) [13]. ODIN is similar to MNN
but with the characteristic that a vector is defined as an outlier
if it participates in at most T neighborhoods in kNN graph,
where threshold T is a control parameter. ODIN classifies a
vector as outlier on basis of its in degree number in the graph.
B. K-means Clustering Algorithm
In this work, we use k-means clustering to find hours with
similar pattern of consumption. In this form, we reduce the
consumption readings (features) summed up the in-between
samples. K-means is a popular algorithm to clustering data
through a certain number of clusters (assume k clusters) fixed
a priori. The main idea is to define k-centers (one for each
cluster) and take each point and associate it to the nearest
center. In other words, the algorithm aims at minimizing an
objective function know as squared error function given by:
Ci
C 

(|xi − vj |)2 (1)
i=1 j=1
where |xi − vj | is the Euclidean distance, Ci is the number
of data points in it h cluster and C is the number of cluster
centers. In addition to k-means, we use Silhouette plots to
detect the suitable number of consumption patterns (clusters)
in one day. The silhouette value is a measure of how similar
an object is to its own cluster (cohesion) compared to other
clusters (separation).
IV. E VALUATION
A. Dataset
We used the smart energy data from the Irish Smart Energy
Trial [22] in our tests. This data includes the electricity usage
reports of Irish homes and businesses during 2009 and 2010.
We select five customers who participated in the trial and had
a smart meter installed in their homes. Each file (customer)
contains 90 instances(days) by 24 components(meter measure-
ments).
B. Synthetic malicious energy consumption
In this research, we generated seven types of energy theft
based on the dataset of normal samples [23]. To implement
these attacks, first, we obtained the normal patterns using the
historical consumption, and then we applied the following
changes to create the seven types of electricity theft:
 TABLE I C. Experiment Design
DATASET USED IN OUR EXPERIMENTS
The methodology has two steps. In the first step, we
Features performed the preprocessing of the data; this process includes
Dataset
original clustering
Customer-1 24 3 tasks such as (1) format and normalize the data from the raw
Customer-2 24 3 file (2) select a season period(90 days) with 24 meter readings
Customer-3 24 13 by day,(3) create a synthetic data set of malicious samples
Customer-4 24 9
Customer-5 24 3
which contains seven different types of electricity theft, and
(4) perform k-means in conjunction with Silhouette plots to
reduce the number of meter readings by day which sums up
the in-between samples. In the second part, we tested the
seven detection algorithms for the five customers. For every
customer we tested seven files, which correspond to seven
types of electricity theft, making 35 files processed in total.
We use the Area Under ROC Curve (AUC) as a metric
for measuring the performance of the algorithms. In addition,
we tuned the parameter values with the help of corresponding
literature. The process included a test to find the bests ’h’
values (kernel width) for the algorithms LDF, KDEOS, and
RDOS.

Fig. 1. Example of the types of electricity theft generated (attacks)
• Type-1: Scaling down the amount of consumption by a
constant ratio, i.e. reduce the consumption every hour by
the same constant
• Type-2: It generates an interruption in the transmission of
usage reports by smart meters during a random duration
of the day
• Type-3: Scaling down the amount of consumption with
a different random value for every hour
• Type-4: Scaling down the average consumption of the
day every hour for a random value
• Type-5: This represents a constant consumption all day.
The value of the constant is the average consumption for
that day
• Type-6: Reverses the order of readings. In this way, the
illegal customer pays less because the price of the hours
with high demand is more expensive than hours with low
demand
• Type-7: Scaling down the amount of consumption. The
consumption will be between zero and the minimum
value for that hour.
An example of the daily consumption of a customer with
the seven types of synthetic electricity theft generated is shown
in Fig. 1. In this figure, the normal consumption is colored
with black; the rest of the colors correspond to the attacks
generated.
V. R ESULTS
The results of applying the k-means clustering algorithm
over the five customer are shown in the Table I. We can see that
in the majority of the cases, the features were reduced to three
meter readings by day. In other words, this corresponds to
three patterns of consumption during one day: low(morning),
medium(afternoon) and high consumption(night). Customer-
3 and Customer-4 have more patterns of consumption in one
day. This would probably correspond to commercial customers
who have many patterns of consumption during the day.
A. AUC performance by seven Algorithms
As result of the k-means clustering, we have customers with
different meter readings by day. In Fig. 2, RDOS, LDF and
INFLO obtain the best performance for customers with three
features. For data sets with electricity theft type-4 to type-
7, the best results are over 90 percent AUC. Customer-3 and
Customer-4 are shown in Fig. 3 and Fig. 4. In both cases,
the better performances are obtained with KDEOS, INFLO,
LDF and RDOS. Also we observed that for electricity theft
5,6,7 the AUC is over 75 percent while in the rest of the
performance is lower. In addition, we can appreciate that
Customer-3 and Customer-4, in the majority of the methods,
the AUC is lower than Customer 1, 2, 5 (low features). This
leads us to conclude that the increase in the features affects
the performance of some algorithms. For example, the method
RDOS dramatically decreases its performance when it runs
dataset with high features.
B. Results by types of electricity theft
Table II shows the results over the full data sets, i.e. for
the five customers with seven types of electricity thefts for
each one. We observed that the majority of algorithms obtain
good results for all types of electricity thefts (only the type-2
obtains a low AUC). Table II also shows the best methods by

137
 Fig. 2. AUC of customers 1,2,5 (3 features) Fig. 4. AUC of customer-4 (9 features)

Fig. 5. General Time taken by algorithms over all data set

Fig. 3. AUC of customer-3 (13 features) type, which shows that INFLO, RDOS, LDF have excellent
results.
In Fig. 5, we can see the result for the average time over the
ve customer. MNN and KDEOS, obtain the best time while
TABLE II LDF and LOF are the worst performer. We must highlight that
D ETECTION LEVEL FOR TYPE OF ELECTRICITY THEFT (AUC) all methods output similar values for all type of the electricity
Theft-Type AUC Methods theft. Fig. 6 show the Standard deviation obtained over the full
1 0.81 INFLO, LDF, RDOS data set. RDOS and INFLO are the methods with the lowest
2 0.65 KDEOS, LDF, INFLO standard deviation.
3 0.91 RDOS, INFLO
4 0.94 INFLO, LDF, RDOS
5 0.95 RDOS, KDEOS
C. Effect of the clustering
6 0.95 RDOS, LDF Due to space limitations we show the results of features
7 0.94 KDEOS, INFLO
reduction only for customer-2 in Fig. 7 and 8. In these figures,
we observed the effect on the AUC performance using 24-
features vs 3-features. The figures show that for the majority

138
 Fig. 6. General Standard deviation over all data set
Fig. 7. AUC with 24 features for customer 2
of the top algorithms, the performance are increased when
reducing the metering reading from 24 to 3. In other words,
when we reduce the consumption readings (sum between in
samples) we increase the performance of intruders detection
stealing energy. On the same manner, identical results are
obtained on the standard deviation.
D. Feasibility of using outliers detection for AMI network
Based on the results obtained, we observed that the use
of outliers methods produce excellent results in the detection
of electricity thefts over consumer consumption. In Table II,
we can see that all types of electricity theft are detected
efficiently except for type-2. We can also see that not all
methods produce efficient results. The INFLO, RDOS, and
LDF methods obtained the best AUC values while MNN and
139
Fig. 8. AUC with 3 features for customer 2
ODIN methods are the worst. In general, we can say that
the methods that combine the nearest neighbors, the use of
a kernel and symmetry between neighbors to estimate the
density, are excellent ways to detect electricity theft based on
the analysis of consumer consumption.
VI. C ONCLUSION
In this paper, we analyzed the feasibility of applying out-
liers detection algorithms for enhancing the security of AMI
through the detection of electricity theft in a variety of types.
Our focus in this study was to detect normal customers and
malicious customers based on the analysis of consumption pat-
terns in the AMI subsystem. We used seven outliers detection
algorithms to detect abnormal pattern consumption; also, a
preprocessing of the data using k-means clustering algorithm
was performed with the objective to reduce the numbers of
measurement samples by day over the data. The results showed
that when the numbers of metering reading are reduced (using
clustering), the AUC performance is increased. In addition,
INFLO and RDOS show prominent results (AUC and STD) in
the detection of electricity theft. Finally, the best performance
was obtained with three features in comparison with nine and
thirteen features.
R EFERENCES
[1] J. Nagi, K. Yap, S. Tiong, S. Ahmed, and M. Mohamad,
“Nontechnical loss detection for metered customers in power
utility using support vector machines,” IEEE Transactions on
Power Delivery, vol. 25, no. 2, pp. 1162–1171, 2010, cited By
104. [Online]. Available: https://www.scopus.com/inward/record.uri?
eid=2-s2.0-77950188492&doi=10.1109%2fTPWRD.2009.2030890&
partnerID=40&md5=70a42c7ec2f63999ad88f40a06ef15d0
[2] S. McLaughlin, D. Podkuiko, and P. McDaniel, “Energy theft in the
advanced metering infrastructure,” Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture
Notes in Bioinformatics), vol. 6027 LNCS, pp. 176–187, 2010, cited
By 99. [Online]. Available: https://www.scopus.com/inward/record.uri?
eid=2-s2.0-77955027376&doi=10.1007%2f978-3-642-14379-3 15&
partnerID=40&md5=c558ba061d1e4a45a868b5bc4b52211f
 [3] G. Tsekouras, N. Hatziargyriou, and E. Dialynas, “Two- no. 2, pp. 93–104, 2000, cited By 1763. [Online]. Available:
stage pattern recognition of load curves for classification of https://www.scopus.com/inward/record.uri?eid=2-s2.0-0039253819&
electricity customers,” IEEE Transactions on Power Systems, partnerID=40&md5=8237238cd72e69d886fad873ae89c433
vol. 22, no. 3, pp. 1120–1128, 2007, cited By 122. [Online]. [18] L. Latecki, A. Lazarevic, and D. Pokrajac, “Outlier detection
Available: https://www.scopus.com/inward/record.uri?eid=2-s2. with kernel density functions,” Lecture Notes in Computer
0-34548048165&doi=10.1109%2fTPWRS.2007.901287&partnerID= Science (including subseries Lecture Notes in Artificial
40&md5=13b0469f43888be7d57410f728775244 Intelligence and Lecture Notes in Bioinformatics), vol. 4571
[4] Y. Zhang, W. Chen, and J. Black, “Anomaly detection LNAI, pp. 61–75, 2007, cited By 33. [Online]. Available:
in premise energy consumption data,” 2011, cited By 12. https://www.scopus.com/inward/record.uri?eid=2-s2.0-37249036471&
[Online]. Available: https://www.scopus.com/inward/record.uri?eid= partnerID=40&md5=bf02ef283a026cf9a22e35f95194b686
2-s2.0-82855163956&doi=10.1109%2fPES.2011.6039858&partnerID= [19] E. Schubert, A. Zimek, and H.-P. Kriegel, “Generalized outlier detection
40&md5=65a396de5313a3981c18b939f44e8570 with flexible kernel density estimates,” vol. 2, 2014, pp. 542–550, cited
[5] S. Dua and X. Du, Data Mining and Machine Learning in Cybersecurity, By 22. [Online]. Available: https://www.scopus.com/inward/record.
1st ed. Boston, MA, USA: Auerbach Publications, 2011. uri?eid=2-s2.0-84958543874&doi=10.1137%2f1.9781611973440.63&
[6] V. Barnett and T. Lewis, Outliers in Statistical Data, ser. Wiley partnerID=40&md5=ac27f26d093348cccba3db76e25f0405
Series in Probability & Statistics. Wiley, 1994. [Online]. Available: [20] W. Jin, A. Tung, J. Han, and W. Wang, “Ranking outliers using
https://books.google.com.pr/books?id=B44QAQAAIAAJ symmetric neighborhood relationship,” Lecture Notes in Computer
[7] N. Billor, A. Hadi, and P. Velleman, “Bacon: Blocked adaptive Science (including subseries Lecture Notes in Artificial Intelligence and
computationally efficient outlier nominators,” Computational Statistics Lecture Notes in Bioinformatics), vol. 3918 LNAI, pp. 577–593, 2006,
and Data Analysis, vol. 34, no. 3, pp. 279–298, 2000, cited cited By 117. [Online]. Available: https://www.scopus.com/inward/
By 154. [Online]. Available: https://www.scopus.com/inward/record. record.uri?eid=2-s2.0-33745772192&doi=10.1007%2f11731139 68&
uri?eid=2-s2.0-0034282347&doi=10.1016%2fS0167-9473%2899% partnerID=40&md5=b49bc5895564746a45af3007544b77d2
2900101-2&partnerID=40&md5=79b7dd5e97236600daaec721ec1ff5d7 [21] B. Tang and H. He, “A local density-based approach for outlier
[8] E. Eskin, “Anomaly detection over noisy data using learned probability detection,” Neurocomputing, vol. 241, pp. 171–180, 2017, cited
distributions,” in Proceedings of the Seventeenth International By 1. [Online]. Available: https://www.scopus.com/inward/record.
Conference on Machine Learning, ser. ICML ’00. San Francisco, CA, uri?eid=2-s2.0-85015321791&doi=10.1016%2fj.neucom.2017.02.039&
USA: Morgan Kaufmann Publishers Inc., 2000, pp. 255–262. [Online]. partnerID=40&md5=072e890f93ba2dd8e30a6b2946fae646
Available: http://dl.acm.org/citation.cfm?id=645529.658128 [22] I. C. for Energy Regulation. (2018) Irish social science
[9] E. M. Knorr and R. T. Ng, “Algorithms for mining distance-based data archive. [Online]. Available: http://www.ucd.ie/issda/data/
outliers in large datasets,” in Proceedings of the 24rd International commissionforenergyregulationcer/
Conference on Very Large Data Bases, ser. VLDB ’98. San Francisco, [23] P. Jokar, N. Arianpoo, and V. Leung, “Electricity theft detection
CA, USA: Morgan Kaufmann Publishers Inc., 1998, pp. 392–403. in ami using customers’ consumption patterns,” IEEE Transactions
[Online]. Available: http://dl.acm.org/citation.cfm?id=645924.671334 on Smart Grid, vol. 7, no. 1, pp. 216–226, 2016, cited By 29.
[10] C. Aggarwal and P. Yu, “Outlier detection for high dimensional [Online]. Available: https://www.scopus.com/inward/record.uri?eid=
data,” 2001, pp. 37–46, cited By 433. [Online]. Available: 2-s2.0-84960349614&doi=10.1109%2fTSG.2015.2425222&partnerID=
https://www.scopus.com/inward/record.uri?eid=2-s2.0-0034832620& 40&md5=862fb157e50102b2d986b413beed9e29
partnerID=40&md5=838180bd526a93462c9547680d306d01
[11] S. Ramaswamy, R. Rastogi, and K. Shim, “Efficient algorithms for
mining outliers from large data sets,” SIGMOD Record (ACM Special
Interest Group on Management of Data), vol. 29, no. 2, pp. 427–438,
2000, cited By 862. [Online]. Available: https://www.scopus.com/
inward/record.uri?eid=2-s2.0-0039845384&doi=10.1145%2f335191.
335437&partnerID=40&md5=d0562a1553e880767d386344c4682eec
[12] M. Brito, E. Chvez, A. Quiroz, and J. Yukich, “Connectivity
of the mutual k-nearest-neighbor graph in clustering and
outlier detection,” Statistics and Probability Letters, vol. 35,
no. 1, pp. 33–42, 1997, cited By 89. [Online]. Available:
https://www.scopus.com/inward/record.uri?eid=2-s2.0-0031571391&
partnerID=40&md5=4016de803c224e486b5e279444f2ca13
[13] V. Hautamki, I. Krkkinen, and P. Frnti, “Outlier detection using
k-nearest neighbour graph,” vol. 3, 2004, pp. 430–433, cited
By 77. [Online]. Available: https://www.scopus.com/inward/record.
uri?eid=2-s2.0-10044269754&doi=10.1109%2fICPR.2004.1334558&
partnerID=40&md5=c2079e09347211036c9fcd2e25070210
[14] S. Depuru, L. Wang, and V. Devabhaktuni, “Support vector machine
based data classification for detection of electricity theft,” 2011, cited
By 54. [Online]. Available: https://www.scopus.com/inward/record.
uri?eid=2-s2.0-79958816472&doi=10.1109%2fPSCE.2011.5772466&
partnerID=40&md5=36a976d4bac1ea0db3acb2b5f5007ac0
[15] S. Depuru, L. Wang, V. Devabhaktuni, and R. Green, “High performance
computing for detection of electricity theft,” International Journal of
Electrical Power and Energy Systems, vol. 47, no. 1, pp. 21–30, 2013,
cited By 24. [Online]. Available: https://www.scopus.com/inward/
record.uri?eid=2-s2.0-84870315054&doi=10.1016%2fj.ijepes.2012.10.
031&partnerID=40&md5=e6fde4f7bd1a5b9c7fab2666de820387
[16] S. Depuru, L. Wang, V. Devabhaktuni, and P. Nelapati, “A
hybrid neural network model and encoding technique for enhanced
classification of energy consumption data,” 2011, cited By 20.
[Online]. Available: https://www.scopus.com/inward/record.uri?eid=
2-s2.0-82855182204&doi=10.1109%2fPES.2011.6039050&partnerID=
40&md5=7677b27c8553df10a26d29389d7175c4
[17] M. Breuniq, H.-P. Kriegel, R. Ng, and J. Sander, “Lof:
Identifying density-based local outliers,” SIGMOD Record (ACM
Special Interest Group on Management of Data), vol. 29,

140