Data mining
secondary. That is, the data was usually collected to
Introduction
Data mining is a new discipline which has sprung up
at the confluence of several other disciplines, driven
chiefly by the growth of large databases. The basic
motivating stimulus behind data mining is that these
large databases contain information which is of value
to the database owners, but this information is con-
cealed within the mass of uninteresting data and has
to be discovered. That is, one is seeking surprising,
novel, unexpected, or valuable information, and the
aim is to extract this information. This means that
the subject is closely allied to exploratory data anal-
ysis. However, issues arising from the sizes of the
databases, as well as ideas and tools imported from
other areas, mean that there is more to data mining
than merely exploratory data analysis.
Perhaps the main economic driver to the develop-
ment of data mining tools and techniques has come
from the commercial world; the promise of money to
be made from data processing innovations is a famil-
iar one, and commercial databases are now rapidly
growing in size, as well as in number. However, there
is also substantial scientific interest; philosophers of
science have remarked that advances and innovation
often occur when a mismatch between the data and
the predictions of a theory occurs, and nowadays
to detect such mismatches often requires extensive
analysis of large data sets. Examples of areas of sci-
entific applications of data mining include astronomy
[1, 2] and molecular biology [3]. Environmental
applications include such things as detecting the
impact of pollutants on people or (see ecosystems)
before these effects become readily apparent, moni-
toring changes due to urbanization, and monitoring
and detecting climate change. Data sets for these and
other environmental data mining exercises are col-
lected from a wide range of modalities, including
high altitude and satellite imagery, as well as surface
measuring instruments, both terrestrial and marine.
Apart from the sizes of the data sets, one of
the early distinguishing features of data mining was
Based in part on the article “Data mining” by
David Hand, which appeared in the Encyclopedia of
Environmetrics.
Encyclopedia of Environmetrics, Online © 2006 John Wiley & Sons, Ltd.
This article is © 2013 John Wiley & Sons, Ltd.
This article was published in Encyclopedia of Environmetrics Second Edition in 2012 by John Wiley & Sons, Ltd.
DOI: 10.1002/9780470057339.vad002.pub2
that the data to which it was applied were typically
answer some other question, or perhaps collected sec-
ondarily in the course of pursuing some other issue
(rainfall measurements may be collected to study
climate phenomena, but then used as the input to
a study of groundwater distribution). However, data
mining is a rapidly evolving technology, and increas-
ing numbers of data sets are now collected with the
specific objective of trawling through them seeking
interesting or unusual configurations. Examples are
particle physics experiments and many commercial
applications, especially involving the World Wide
Web. In the latter applications, in particular, formal
adaptive experimental designs may be used. Novel
applications of data mining specifically aimed at
using the web include such things as early detection
of epidemics and the spread of disease (see Epidemic
models)from the purchase patterns of pharmaceuti-
cals or from Twitter communications.
The excitement of data mining is also partly a con-
sequence of this secondary nature; it suggests that
there is valuable information concealed within the
data one already has, simply waiting for someone to
tease it out. Unfortunately, the “simply” part of this
exercise is rather misleading. One of the problems is
that large data sets necessarily have a great deal of
structure in them, but this structure has three major
sources in addition to the target one of “important,
real, undiscovered structure.” These three sources are
data contamination, chance occurrences of data, and
structure which is already known to the database
owner (or, if not explicitly articulated as known, suf-
ficiently obvious once it has been pointed out to be
of no genuine interest or value, such as the fact that
married people come in pairs). The first and second
of these are sufficiently important to warrant some
discussion.
Challenges of Data Mining
It is probably not too much of an exaggeration to
say that all environmental data sets are contaminated,
though with small data sets this may be difficult to
detect. With large data sets, it means that the data
miner may triumphantly return an unusual pattern
which is simply an artefact of data collection, record-
ing, or other inadequacies. A postgraduate student
discovered some curious anomalies in wind speed
2 Data mining
records, in which abnormally high recordings occa-
sionally occurred. Closer examination revealed that
these all occurred at midnight, coincidentally, at the
same time at which the recording machine automati-
cally reset itself. A (small) data set giving times and
durations of geyser eruptions which show anoma-
lies is described in Ref. 4. Again, closer examination
revealed that these were not caused by departures
from the underlying common physical mechanism,
but instead by recording errors. Digit preference
is another cause of such curiosities, and is only
detectable in large data sets.
In statistics, one is often able to cope with data
inadequacies by extending the model to cope with
them. Thus, for example, one may handle distorted
sampling by including a model for the case selec-
tion process, and incomplete vectors of measurements
may be handled via the EM algorithm. However,
such strategies can only be adopted if one has some
awareness (and understanding) of the data contamina-
tion mechanism. In data mining with secondary data,
this is often not the case. Often, such problems are
ignored, with obvious potential for misleading con-
clusions.
Statistical hypothesis and significance tests
explore the possibility of spurious patterns arising
in the data using tools that estimate the probability
of such structures arising by chance – merely as
a consequence of random variation. Unfortunately,
with large data sets, and when large sets of possible
patterns are sought, the opportunity for the discovery
of apparent structures is clearly great; in particular,
issues arising from multiple testing are a real
potential problem. This means that a formal statistical
approach may not always be readily applied. Instead,
data miners often simply define score functions
(for the interestingness, unusualness, or some other
characteristic of a pattern), without any probability
interpretation, and pass those patterns which show the
largest such scores over to an expert for evaluation.
This description reveals the process nature of data
mining. Data mining is not a “one-off” exercise, to
be done and finished with. Rather, it is an ongoing
process; one examines a data set, identifies features
of possible interest, discusses them with an expert,
goes back to the data in the light of these discussion,
and so on.
Encyclopedia of Environmetrics, Online © 2006 John Wiley & Sons, Ltd.
This article is © 2013 John Wiley & Sons, Ltd.
This article was published in Encyclopedia of Environmetrics Second Edition in 2012 by John Wiley & Sons, Ltd.
DOI: 10.1002/9780470057339.vad002.pub2
Data Mining and Statistics
The score functions mentioned in the previous section
may be the same as the criteria used in statistical
model fitting, without the probabilistic interpretation,
or they may be other criteria. An illustration of the
differences in perspective is given by regression anal-
ysis. A statistician may find the maximum likelihood
estimates of the parameters, assuming a normal error
distribution. In contrast, a data miner may adopt the
sum of squared residuals as a score function to use in
choosing the parameters. Since maximizing the likeli-
hood based on a normal error distribution leads to the
sum of squares criterion, these two approaches yield
the same result, but they start from different posi-
tions. The statistician has a formal model in mind,
while the data miner is simply aiming to find a good
description of the data. Of course, the distinction is
not a rigid one; there is an overlap between the two
perspectives.
This example does show how central is the con-
cept of modeling to the statistician. In contrast, data
miners tend to place much more emphasis on algo-
rithms. Given the essential role of computers in data
mining, this algorithmic emphasis is perhaps not sur-
prising. Moreover, when data sets are very large, the
popular statistical algorithms may become imprac-
ticable (tools which make repeated passes through
the data, for example, may be out of the question
with a billion data points). Computers are, of course,
also important for statistics, but many statistical tech-
niques can be applied on small data sets without
computers; in fact, many were originally developed
that way. One consequence of this is that it may be
difficult to describe exactly what model is being fit to
the data in a data mining exercise. For example, asso-
ciation rules are a type of local structure which has
been explored in the data mining rather than the sta-
tistical literature. They are small local models or pat-
terns, describing co-occurrences of particular values
of discrete variables. Originally developed for study-
ing purchase behavior patterns in supermarkets, they
have been applied much more widely–for example,
to studying local collocation patterns in ecological
data sets. The key issue in developing association
rules is not how to define the small local models,
which is straightforward, but rather how to cope with
the potentially massive search problems, and some
sophisticated algorithms have been developed by the
data mining community.

The difficulty of describing exactly what model
is being fitted can have adverse consequences. For
example, cluster analysis is widely used in data
mining, but without careful thought about the nature
of the procedure it can be difficult to be clear
about what sort of “clusters” are being found. Thus,
compact structures may be appropriate in some
situations (e.g., to produce compact summarizing
descriptions, with the clusters being represented by
“central” points), while in others elongated shapes
may be desirable, in which neighboring points in the
same cluster are similar but distant ones are not (e.g.,
animals of a given species may differ more in some
dimensions than in others). Without an awareness
of the type of structure that the method reveals,
inappropriate conclusions could be drawn; a species
could be incorrectly partitioned on a dimension in
which it has substantial variability.
In the above, we have used words such as model,
structure, and pattern without defining them. In
Ref. 5, a model was defined as a large-scale sum-
mary of a set of data (that is, as the standard statistical
notion of a model), and a pattern as a small-scale local
structure. A Box–Jenkins decomposition of a time
series is a model, whereas a conjunction of values
which occasionally repeats itself (for example, in an
electroencephalogram (EEG) trace) is a pattern. Mod-
els are the staple of statistics, and patterns are some-
thing with which statistics has generally not been con-
cerned. An examination of the data mining literature
shows that both models and patterns are important,
but narrow views of data mining sometimes fail to
recognize the diversity of the tools used. Thus, for
example, it is sometimes claimed that data mining is
merely the application of recursive partitioning meth-
ods (e.g., tree classifiers;), but this is a parody of the
breadth of the field. Likewise, the viewpoint some-
times proposed in the econometric literature, that data
mining is merely an elaborate and extensive form
of model search, fails to recognize the various other
kinds of data mining activities which go on.
Data Mining Tools
A large number of different kinds of tools are used in
data mining, reflecting the eclecticism of its origins.
Some recent ones in pattern recognition and detec-
tion, culled from the data mining literature with no
particular objective other than to indicate the diversity
Encyclopedia of Environmetrics, Online © 2006 John Wiley & Sons, Ltd.
This article is © 2013 John Wiley & Sons, Ltd.
This article was published in Encyclopedia of Environmetrics Second Edition in 2012 by John Wiley & Sons, Ltd.
DOI: 10.1002/9780470057339.vad002.pub2
Data mining 3
of different kinds of method, are tools for characteriz-
ing, identifying, and locating patterns in multivariate
response data; tools for detecting and identifying
patterns in two-dimensional displays (such as finger-
prints and meteorological charts); identifying sudden
changes over time (such as those induced by chem-
ical leakages or intrusion detection in computer sys-
tems); detecting changes in behavior (animal migra-
tion routes; fraud detection is a common application
in various domains, including commercial, scientific,
and governmental – such as electoral fraud); and
identifying logical combinations of values which dif-
fer between groups. Some examples of important
tools in model building in data mining (again cho-
sen with no particular aim other than to illustrate the
range of such methods) include recursive partitioning,
cluster analysis, regression modeling, segmentation
of time series into a small number of segment types,
techniques for condensing huge (tens of billions of
data points) data sets into manageable summaries, and
collaborative filtering, in which transactions are pro-
cessed as they arrive so that future transactions may
be treated in a more appropriate manner.
Much statistical theory is aimed at producing valid
inferences from a sample to some population (real
or notional) from which the sample has been drawn.
This might be so that one can make comparative
statements about the populations, or for environmen-
tal forecasting, or for other reasons. These methods
are also appropriate in data mining, provided one has
a sample and that it has been drawn in a probabilis-
tic way (so that one knows the probability of each
object appearing in the sample; see Environmental
tobacco smoke (ETS)). Going further than this, in
many data mining applications one has available data
on the entire population (for example, all chemical
molecules in a particular class), and then, in model
building data mining applications, analyzing a sample
from the data set may be a sensible way to proceed.
In contrast, however, in a pattern detection exercise
it will typically be necessary to analyze the entire
data set; if one is seeking those data points which are
anomalous, there is no alternative to examining every
data point.
It is clear that data mining will be of increasing
importance as time progresses. However, the impor-
tance should not conceal the difficulties. Finding
unsuspected structures in large data sets, and iden-
tifying those which are due to phenomena of genuine
4 Data mining

interest and not merely arising from data contami-
nation or due to chance, is by no means a trivial
exercise. Issues of theory, of data management, and
of practice all arise. General descriptions of data min-
ing are given in Refs 6, 7.
References
[1] Way, M.J., Scargle, J.D., Ali, K., & Srivastava, A.N.
(2012). Advances in Machine Learning and Data Mining
for Astronomy, Chapman and Hall, London.
[2] Zeljko, I. (2011) Data Mining and Machine Learning
in Astronomy – A Practical Guide, Princeton University
Press, Princeton.
[3] Gustafson, J.P., Shoemaker, R., & Snape, J.W. (2011).
Genome Exploitation: Data Mining the Genome, Springer,
New York.
[4] Denby, L. & Pregibon, D. (1987). An example of the use
of graphics in regression, The American Statistician 41,
33–38.
[5] Hand, D.J., Blunt, G., Kelly, M.G., & Adams, N.M.
(2000). Data mining for fun and profit, Statistical Science
15, 111–131.
[6] Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., &
Uthurusamy, R., eds, (1996). Advances in Knowledge
Discovery and Data Mining, AAAI Press, Menlo Park.
[7] Hand, D.J., Mannila, H., & Smyth, P. (2001). Principles
of Data Mining, MIT Press, Cambridge.
DAVID J. HAND

Encyclopedia of Environmetrics, Online © 2006 John Wiley & Sons, Ltd.

This article is © 2013 John Wiley & Sons, Ltd.
This article was published in Encyclopedia of Environmetrics Second Edition in 2012 by John Wiley & Sons, Ltd.
DOI: 10.1002/9780470057339.vad002.pub2