FULL PAPER WWW.C-CHEM.
ORG
An Enhanced Monte Carlo Outlier Detection Method
Liangxiao Zhang,*[a,b,c,d] Peiwu Li,*[a,c,d,e] Jin Mao,[a,b,c] Fei Ma,[a,d] Xiaoxia Ding,[a,c]
and Qi Zhang[a,e]
Outlier detection is crucial in building a highly predictive
model. In this study, we proposed an enhanced Monte Carlo
outlier detection method by establishing cross-prediction
models based on determinate normal samples and analyzing
the distribution of prediction errors individually for dubious
samples. One simulated and three real datasets were used to
illustrate and validate the performance of our method, and the
results indicated that this method outperformed Monte Carlo
outlier detection in outlier diagnosis. After these outliers were
Introduction
Outlier detection is a primary step in data modeling and is
important in identifying and subsequently eliminating atypical
observations from a given set of data during the establish-
ment of a high-performance model.[1,2] Both univariate meth-
ods and multivariate methods can be used for outlier
detection.[3] Most early univariate methods for outlier detec-
tion were designed on the assumption of underlying identical
and independent data distribution, such as the Chauvenet’s
criterion, Peirce’s criterion, Grubbs’ test,[4] Tietjen–Moore test,[5]
and generalized extreme Studentized deviate test.[6] These
methods are unsuitable for high-dimensional datasets and
arbitrary datasets without prior knowledge of underlying data
distribution.[7]
Multivariate outlier detection methods include statistical and
data mining methods. Statistical methods aim at identifying
the observations relatively far from the center of data distribu-
tion.[8] In this method, the Mahalanobis distance is used as a
well-known criterion depending on the estimated multivariate
normal distribution parameters.[9] The detection result is satis-
factory for datasets with only one calibration outlier but not
for those with multiple outliers because outliers could distort
the mean value (MV) or covariance matrix to generate a mask-
ing effect, which makes the hat matrix leverage unattainable.
To mitigate the masking effect, many methods have been
proposed to detect outliers, including minimum volume ellip-
soid,[10] ellipsoidal multivariate trimming,[11] minimum covari-
ance determinant,[12] resampling by half-means and smallest
half volume.[13] The key to these methods is to find out the
main body of an observation matrix and identify the outliers
significantly different from the majority of the dataset.
Data mining methods are designed to manage large data-
bases from high-dimensional spaces, including distance-based
methods, clustering methods, and spatial methods. To effec-
tively use dependent variables and detect more outliers, the
Monte Carlo outlier detection (MCOD) method was developed
1902 Journal of Computational Chemistry 2015, 36, 1902–1906
removed, the value of validation by Kovats retention indices
and the root mean square error of prediction decreased from
3.195 to 1.655, and the average cross-validation prediction
error decreased from 2.0341 to 1.2780. This method helps
establish a good model by eliminating outliers. V
C 2015 Wiley
Periodicals, Inc.
DOI: 10.1002/jcc.24026
as a feasible way to detect different kinds of outliers by estab-
lishing many cross-prediction models.[2,14,15] The core idea of
an MC outlier detector is that the predictive results for the X
outlier far from the center of the sample space are consider-
ably variable by Monte Carlo sampling subset predictive mod-
els while predicting the y outlier is usually difficult. Thus, the
distribution of predictive errors could be used for samples in
multiple outlier detection. However, due to the masking effect,
the boundary between normal and abnormal samples is
unclear in the plot of variance of residuals versus mean of
residuals.
In this study, we proposed a new strategy to detect outliers
using MCOD to identify normal samples from the plot of var-
iance versus mean of residuals and then individually checking
the dubious samples. As there is no more than one outlier in
the dataset, the masking effect does not exist, and therefore,
it is easier to detect outliers. In addition, if the dubious
[a] L. Zhang, P. Li, J. Mao, F. Ma, X. Ding, Q. Zhang
Oil Crops Research Institute, Chinese Academy of Agricultural Sciences,
Wuhan 430062, China
E-mail: liangxiao_zhang@hotmail.com (or) peiwuli@oilcrops.cn
[b] L. Zhang, J. Mao
Key Laboratory of Biology and Genetic Improvement of Oil Crops,
Ministry of Agriculture, Wuhan 430062, China
[c] L. Zhang, P. Li, J. Mao, X. Ding,
Laboratory of Risk Assessment for Oilseeds Products (Wuhan),
Ministry of Agriculture, Wuhan 430062, China
[d] P. Li, F. Ma, L. Zhang,
Quality Inspection and Test Center for Oilseeds Products,
Ministry of Agriculture, Wuhan 430062, China
[e] P. Li, Q. Zhang
Key Laboratory of Detection for Mycotoxins, Ministry of Agriculture,
Wuhan 430062, China
Contract grant sponsor: National Key Technologies R&D Program;
Contract grant number: 2012BAK08B03; Contract grant sponsor:
National Nature Foundation Committee of P.R. China; Contract grant
number: 21205118; Contract grant sponsor: Earmarked Fund for China
Agriculture Research System; Contract grant number: CARS-13
C 2015 Wiley Periodicals, Inc.
V
WWW.CHEMISTRYVIEWS.COM
 WWW.C-CHEM.ORG
Figure 1. Flow chart for enhanced Monte Carlo outlier detection.
samples are normal, the prediction errors decrease to an
acceptable range with no influence from the outliers.
Theories and Methods
Dataset
Dataset 1, a simulated dataset, was designed (X (100 3 10)
and y (100 3 1) samples with normally distributed noise) to
illustrate our method. In these examples, matrix X contains
independent columns (X) meaning molecular descriptors and a
dependent column (y) being related to X by the equation:
y 5 f(X). Then, two types of outliers (y and X) are added into
this dataset as follows: (1) 20 additional y outliers with three-
fold noise are added to this dataset. The independent varia-
bles of these y outliers are derived from the main body of the
100 normal samples and (2) 20 additional X outliers with a
large Mahalanobis distance (twice larger than the average
leverage value from the normal samples) are added to this
dataset. The X outliers have the same functional relationship
as the 100 normal samples.
Dataset 2, stack loss plant dataset for oxidation of ammonia
to nitric acid, provides operational data of a plant, which
includes 21 observations on three independent variables (cool-
ing air flow, cooling water inlet temperature, and acid concen-
tration) and a dependent variable of stack loss.[16,17] Among all
the samples, the outliers are No. 1, 3, 4, and 21, and No. 2 is a
good leverage point.
Dataset 3, Hawkins–Bradu–Kass Data, is another classic data-
set for outlier detection and robust regression. The first 14
observations out of 75 are outliers of this dataset.[18]
Dataset 4, a dataset of Kovats retention indices, was chosen
according to.[19–21] A total of 177 methylalkanes comprised a
range of different carbon chain lengths. Only the last two dig-
its of the KI were recorded in Ref. 21, which were obtained by
subtracting the number of carbons from the main chain 100.
FULL PAPER
MC outlier detection
Traditional outlier detection methods usually analyze the distri-
bution of samples in the sample space. The purpose of outlier
detection is actually to build the best prediction model, and
the model-based method is, therefore, more effective. Recently,
Monte Carlo cross-validation was developed to detect outliers
by studying the distribution of prediction errors of each sample
obtained from the original dataset.[2] The hypothesis of this
outlier detection method is that the normal samples at the
center of the sample space are less influenced by fluctuant
model parameters. As the detailed algorithm was described
elsewhere,[2] this study only gives the outline. Figure 1 (the left)
presents a flow chart for the complete algorithm. The number
of principle components was firstly determined by cross-
validation in partial least squares or principal component
regression modeling. Then, the whole dataset was randomly
divided into training and validation sets. The prediction model
was built using the training set and used to predict the valida-
tion set so as to obtain the prediction error for each validation
sample. After N cycles, the prediction error distribution was
obtained for each sample. Then, the MV and standard deviation
(STD) of the prediction error distribution for the samples were
used to detect outliers. The distribution of the predictive errors
generated by many models contains more sample information
about whether a sample is an outlier or not. The error distribu-
tion of a normal sample is less likely varied when normal sam-
ples are the main body of the whole dataset. The predictive
residuals of a y outlier, however, have a large expectation value,
while an X outlier (good leverage point) far from the main
body of all the samples possesses a small expectation value of
predictive residuals but a large STD. According to the hypothe-
sis of this outlier detection method, normal samples have small
MVs and STDs for their prediction errors and, therefore, lie on
the lower left of the MV/STD plot; the upper left lies the sam-
ple outliers that have small MVs but large STDs; the lower right
displays the y outliers or model outliers that have large MVs
but small STDs. This MV/STD plot could obviously provide vis-
ual diagnosis for direct outlier detection. Being validated by
several datasets, this method was proven effective for outlier
detection.[2]
Enhanced Monte Carlo outlier detection
As the outliers in a calibration dataset increase, the probability
of selecting at least one outlier observation is relatively small.
In this case, the masking effect existing in the interaction of
outlier observation makes it quite unclear to divide outliers
from normal samples. A visual diagnostic for the distribution
of prediction errors is, therefore, insufficient and complex. To
overcome the masking effect, an enhanced Monte Carlo out-
lier detection (EMCOD) method was developed to obtain bet-
ter outlier detection results. This method is designed based
on the fact that normal samples with the smallest MV and
STD of prediction errors are easily determined. As shown in
Figure 1, the EMCOD procedure is similar with MCOD, and the
procedure contains the following steps: (1) use MCOD to
obtain the prediction error distribution for each sample; (2)
Journal of Computational Chemistry 2015, 36, 1902–1906 1903
 FULL PAPER
Figure 2. Mean/STD plot of prediction errors for Dataset 1: Enhanced Monte Carlo outlier detection (left) and Monte Carlo outlier detection (right).
select 40–60% of the samples with the smallest MV and STD
of prediction errors, and determine the remaining samples as
dubious samples; (3) randomly divide the selected data (Ns)
into training and validation sets; (4) after the number of prin-
ciple components is determined by cross-validation, build the
prediction model with the training set and use it to predict
the samples in the dubious samples in the validation set to
obtain the prediction error; (5) after N cycles, obtain the pre-
diction error distribution for the dubious samples; and (6) use
the MV and STD of the error distribution on the dubious sam-
ples to test whether the dubious samples are outliers. Accord-
ing to the hypothesis of this outlier detection method, the
MVs and STDs of their prediction errors decrease, while those
of the outliers increase to some extent. As the masking effect
could be eliminated by EMCOD, it could provide better results
than MCOD.
Data processing and analysis
All programs used were coded in MATLAB 2011a for Win-
dows and all calculations were performed on a personal com-
puter. The MATLAB implementation of EMCOD is available
from http://www.mathworks.com/matlabcentral/fileexchange/
52023-emcod.
Figure 3. Mean/STD plot of prediction errors for Dataset 2: Enhanced Monte Carlo outlier detection (left) and Monte Carlo outlier detection (right).
1904 Journal of Computational Chemistry 2015, 36, 1902–1906
WWW.C-CHEM.ORG
Results and Discussion
Enhanced Monte Carlo outlier detection method
Outlier detection is an important step in building a highly pre-
dictive model. MOCD was recently developed to provide a fea-
sible means of detecting different kinds of outliers by
establishing many predictive models and a MV/STD plot of
prediction errors for all samples. This outlier detection method
depends on the graphic MV/STD plot, so the key is to deter-
mine the visualized boundary between normal and abnormal
samples.
To illustrate our method, a simulated dataset was designed,
which contains 100 normal samples, 20 X outliers, and 20 y
outliers. MCOD was initially conducted to detect the outliers.
As shown in Figure 2, two kinds of outliers have a clear tend-
ency to separate from the normal samples. The y outliers have
larger prediction errors than normal samples while X outliers
(good leverage point) have large STD values than normal sam-
ples. However, the boundary between the outliers and the
normal samples is indistinct, making it difficult to determine
whether a sample far from the original point is an outlier or
not. In this case, EMCOD was performed to detect the outliers
in this simulated dataset. As an enhancement of MCOD, the
MVs and STDs of prediction errors acquired from MCOD were
used to select out 60 normal samples with the smallest MVs
WWW.CHEMISTRYVIEWS.COM
 WWW.C-CHEM.ORG FULL PAPER

Figure 4. Mean/STD plot of prediction errors for Dataset 3: Enhanced Monte Carlo outlier detection (left) and Monte Carlo outlier detection (right). [Color
figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

and STDs. When the number (N) of Monte Carlo models and
sampling ratio are, respectively, set to 10,000 and 0.8, the MVs
and STDs of prediction errors could be used to determine
whether the dubious samples are outliers. As shown in Figure
2, the samples in this simulated dataset were noticeably classi-
fied into four groups. The distances between outliers and nor-
mal samples significantly increase, and 20 X outliers and 20 y
outliers could be easily identified from the MV/STD plot of pre-
diction errors. It is absolutely obvious that EMCOD could pres-
ent a better result in correctly detecting outliers.
Method validation
Dataset 2 is the stack loss dataset of a plant. In MCOD, the
number (N) of Monte Carlo models and sampling ratio are set
to 10,000 and 0.8, respectively. The MV/STD plot of the predic-
tion errors for 21 samples was shown on the right of Figure 3.
Lacking the information about this commonly used dataset, it
is hard to determine the boundary for outlier detection. To
obtain a clearer result, enhanced MOCD was proposed and
used to detect outliers in this dataset. As shown in Figure 3,
the samples including 20, 5, 16, 18, 19, 13, 14, 8, 15, 10, and
17 were normal samples, which had the smallest mean and
STD values. We established MC prediction models using these
11 samples and used these models to observe other samples.
The number (N) of Monte Carlo models and sampling ratio are
also set to 10,000 and 0.8, respectively. According to the
hypothesis that the models built with merely normal samples
provide lower prediction errors for normal samples but higher
prediction errors for outliers, the distances between normal
samples and outliers should be longer. The result is shown in
Figure 3 (left), which illustrates that EMCOD has a better result
as the outliers have correctly been detected.
With the help of an MC outlier detector, normal samples
with the smallest MVs and STDs of prediction errors could be
easily detected, even though it was hard to determine the
boundary between normal samples and outliers. We selected
some normal samples with the smallest MVs and STDs of pre-
diction errors and then determined whether other samples
were outlier one after another.
Dataset 3 represents the Hawkins–Bradu–Kass data. As
shown on the right of Figure 4, the M/SD plot indicates that
14 samples (No. 1–14) are outliers. 52 samples with the lowest
STDs of prediction errors (<0.5) were selected as normal

Figure 5. Mean/STD plot of prediction errors for Dataset 4. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Journal of Computational Chemistry 2015, 36, 1902–1906 1905

 FULL PAPER WWW.C-CHEM.ORG
samples. Other 23 samples were then detected and tested one
by one by the MC prediction models of the dataset estab-
lished with the 52 samples. As shown on the left hand of Fig-
ure 4, the prediction errors of 9 normal samples decrease and
the prediction errors of 14 outliers greatly increase. The distan-
ces between normal samples and outliers significantly increase.
Moreover, this method is insensitive to the number of normal
samples used to build the prediction models (data not
shown).
Dataset 4 is the dataset of the Kovats retention indices. The
detailed information was provided in the previous stud-
ies.[19–22] EMCOD was conducted on Dataset 4. As shown in
Figure 5, the M/SD plot indicates that 18 samples are far from
the original point and can be regarded as outliers. The models
built by all 177 samples were compared with those built by
159 samples, and the result showed that the root mean square
error of prediction decreased from 3.195 to 1.655. After these
outliers were removed, the accuracy of the model significantly
improved. The average cross-validation prediction error also
dropped from 2.0341 to 1.2780, which was obviously better
than the previous study (4.6 and 4.3, respectively).[21]
Conclusion
In this study, we proposed EMCOD by establishing cross-
predictive models using determinate normal samples and indi-
vidually analyzing the distribution of prediction errors for dubi-
ous samples. Four datasets were used to illustrate and validate
our method. The results indicated that EMCOD could increase
the distances between outliers and normal samples, making it
easier to detect outliers.
Keywords: outlier detection
enhanced Monte Carlo outlier
detection
validation
1906 Journal of Computational Chemistry 2015, 36, 1902–1906
How to cite this article: L. Zhang, P. Li, J. Mao, F. Ma, X. Ding,
Q. Zhang, J. Comput. Chem. 2015, 36, 1902–1906. DOI:
10.1002/jcc.24026
[1] R. Todeschini, D. Ballabio, V. Consonni, F. Sahigara, Anal. Chim. Acta
2013, 787, 1.
[2] D. S. Cao, Y. Z. Liang, Q. S. Xu, H. D. Li, X. Chen, J. Comput. Chem.
2010, 31, 592.
[3] G. J. Williams, R. A. Baxter, H. X. He, S. Hawkins, L. Gu, In IEEE Interna-
tional Conference on Data-Mining (ICDM’02), CSIRO Technical Report
CMIS-02/102; Maebashi City, Japan, CSIRO Technical Report CMIS-02/
102, 2002.
[4] F. Grubbs, Technometrics 1969, 11, 1.
[5] W. Stefansky, Technometrics 1976, 14, 469.
[6] B. Rosner, Technometrics 1983, 25, 165.
[7] C. Zhu, H. Kitagawa, S. Papadimitriou, C. Faloutsos, J. Intell. Inf. Syst.
2011, 36, 217.
[8] E. M. Knorr, R. T. Ng, In Proceedings of the VLDB Conference; New
York, 1998; pp. 392–403.
[9] I. Ben-Gal, Outlier detection, In Data Mining and Knowledge Discovery
Handbook: A Complete Guide for Practitioners and Researchers; O.
Maimon and L. Rockach, Eds.; Kluwer Academic Publishers, Dordrecht,
the Netherlands, 2005, ISBN 0-387-24435-2.[WorldCat]
[10] R. Gnanadesikan, J. R. Kettenring, Biometrics 1972, 28, 81.
[11] D. M. Rocke, D. L. Woodruff, J. Am. Stat. Assoc. 1996, 91, 1047.
[12] P. J. Rousseeuw, V. D. Katrien, Technometrics 1999, 41, 212.
[13] W. J. Egan, S. L. Morgan, Anal. Chem. 1998, 79, 2372.
[14] D. S. Cao, Y. Z. Liang, Q. S. Xu, Y. F. Yun, H. D. Li, J. Comput. Aid Mol.
Des. 2011, 25, 67.
[15] H. D. Li, Y. Z. Liang, Q. S. Xu, D. S. Cao, J. Chemometr. 2009, 24, 418.
[16] K. A. Brownlee, Statistical Theory and Methodology in Science and
Engineering; Academic: New York, 1965; pp. 491–500.
[17] R. A. Becker, J. M. Chambers, A. R. Wilks, The New S Language; Wads-
worth & Brooks/Cole, Pacific Grove, California, 1988.
[18] D. M. Hawkins, D. Bradu, G. V. Kass, Technometrics 1984, 26, 197.
[19] D. A. Carlson, U. R. Bernier, B. D. Sutton, J. Chem. Ecol. 1998, 24, 1845.
[20] Y. V. Kissin, G. P. Feulmer, J. Chromatogr. Sci. 1986, 24, 53.
[21] A. R. Katritzky, K. Chen, U. Maran, D. A. Carlson, Anal. Chem. 2000, 72, 101.
[22] Y. Z. Liang, D. L. Yuan, Q. S. Xu, O. M. Kvalheim, J. Chemometr. 2008, 22, 23.
Received: 18 June 2015
Accepted: 1 July 2015
Published online on 31 July 2015
WWW.CHEMISTRYVIEWS.COM