Improving Time-complexity of k Nearest Neighbors
Classifier: A Systematic Review
Syed Ali Qamber
National University of Computer and Emerging Sciences, Karachi, Pakistan
k180889@nu.edu.pk
Abstract—k Nearest Neighbors is one of the most simple
and effective classification algorithms in Machine Learning.
In comparison to other classification algorithms, the superior
stability of kN N makes it an ideal choice for dynamic data or
data streams. However, being a lazy learner, there are also some
limitations to the kN N that make its use with big data difficult
and very time consuming. In this paper, we look at what makes
the kN N so effective, and what are the causes of its high time
and computational complexity. We group the various attempts
made by researchers at optimizing the kN N in terms of its’ time
complexity, into two broad categories: reduction of the training
set size, and speeding up or approximating the neighbor search.
We look at how all these attempts compare with each other and
try to predict which will perform best with big data based on
the information provided in the literature being reviewed.
Keywords: k Nearest Neighbor; kN N ; Time Complexity;
Classification;
I. I NTRODUCTION
The k Nearest Neighbors classifier is a non-parametric Ma-
chine Learning algorithm that can be used for both classifica-
tion and regression. kN N is one of the oldest algorithms being
used in ML. The origins of the approach have been traced
back to Alhazen’s works as early as the eleventh century,
where the medieval Islamic scholar introduced the concept
of visual recognition based on similarity examination [1]. The
recognized formulation of the classifier in 1967 by Cover and
Hart [2] was a watershed event in the ML timeline, and
marked the beginning of basic pattern recognition.
The traditional kN N algorithm can be broken down into
three simple steps for any training data set D (with n training
samples and d dimensions or number of features) and a test
sample a:
1) The distance between a and each training sample in D
is computed
2) The k nearest neighbors of a (the k points which have
the smallest distance from a) are selected
3) a is classified into the class which has the simple
majority in the k selected samples i.e. the most common
class of its’ k nearest neighbors
The success of this algorithm lies in its’ simplicity, effective-
ness, and its ease of use. As can be seen from these steps,
kN N does not need any prior training before classification.
All these steps are only started when a test sample is received
and asked to be classified. This approach is known as lazy
learning in ML, and it means that once you have the training
data, you can get started doing classification straightaway
without wasting any time in training your model.
As kN N does not need to train a model before starting
to classify data, it can successfully classify test data points
that may be difficult to be modeled by classifiers like Support
Vector Machines or Decision Trees. This is due to the fact that
some test data points may be situated in those parts of the data
space where the decision boundaries are not easily modeled.
Hence, eager learners (classifiers that train a model first before
classifying test data) may have a difficult time creating a model
that can classify these peculiar data points correctly. kN N ,
on the other hand, can deal with these difficult data points
in isolation and hence, classify them without worrying about
decision boundaries or generalizing data.
Another advantage the kN N classifier holds over other
classification algorithms is its robustness and stability. This
means that kN N is less prone to changing its output in
response to slight changes in the training or test data. In
contrast, classifiers like Decision Trees or even stochastic
models like Naı̈ve Bayes can be prone to small changes in
input data. Add that to the fact that almost all eager learners
work in batches, i.e. they classify a batch of training data at
a time, and kN N becomes an optimal choice for incremental
learning or data streams, as it is a lazy learner and is not
model-based.
Moreover, the error rate of the kN N classifier has been
established, after extensive research, to be really low. In fact,
the kN N classifier, in comparison to its competitors, has
been proven to give test error rates closest to that of the
Bayesian classier [3], which is the gold standard in supervised
classification problems. All the advantages listed above make
kN N one of the best classifiers to be used with data of
any type. However, there are limitations to the algorithm that
provided complications in its usage with real-life data.
The organization of this paper is as follows. In Section II,
we first look at the limitations of the k Nearest Neighbors
classifier; specifically, its time complexity, and give a general
discussion on the attempts to improve it in past works. In
Section III, we elaborate the methodology used to search for,
and then select the articles to be reviewed here. In Section
IV, we look at how various researchers have decreased the
size of the training sets to improve the time complexity of
kN N . In Section V, we analyze the attempts at speeding up
neighbor searching and their effectiveness in lowering the time
consumption of the subject algorithm. Finally, in Section VI,
we conclude our findings, and look at the future avenues of
research with reference to the topic.
II. L IMITATIONS OF THE kN N C LASSIFIER
While lazy learning does have initial advantages, here, it
comes at a cost in two forms. Since the kN N classifier needs
to store the entire training set in memory, it can be very
memory intensive. Also, as can be seen in the step-by-step
breakdown of the algorithm in Section I, the algorithm has
to loop through all the training examples to first calculate the
distance from the test sample and then select the k nearest
neighbors. This long calculation and exhaustive searching
method means that the algorithm can be very time-consuming.
Both these problems become significantly pronounced as the
number of samples or dimensionality, or both, of the data set
increases.
The effect of the increase of dimensionality is arguably
much greater. This is because in most cases, the number of
samples is much higher than the number of features of the data.
Hence, for a data set with n number of samples and d number
of features, with every unit increase in the dimensionality,
there is an increase of n values in the data set. Likewise, with
the addition of every training sample, there is an increase of
d values in the data set. Since n >> d, we can see that the
effect of an increase in dimensionality is much greater. This
phenomenon is also known as the ‘curse of dimensionality’
in ML and can be visualized in Figure 1. We can clearly
see that the classification time grows exponentially as the
dimensionality of the data increases. This is a major reason
why traditional kN N is generally accepted to be unsuitable
for large data sets.
Fig. 1. Classification Time for 50000 queries using kN N against Number
of Dimensions
In practice, the high memory requirements are of minor
concern thanks to advances in hardware technology and feature
extraction and transformation methods. However, for reducing
its computational and time complexity, numerous modifica-
tions have been suggested to the kN N classifier which have
allowed it to be used with big data. These can be divided into
two broad categories.
The first approach is to decrease the size of the training
set in such a way that the smaller subset obtained is an
apt representation of the larger original set. This reduction
in the training data has to be done while ensuring that the
accuracy of the classifier is not affected significantly, while
the decrease in the size of training set is noteworthy enough
to warrant the use of the technique itself. An example of this
kind of approach would be the identification of dense regions
of shared identity to remove redundant training samples in the
Condensed Nearest Neighbor rule [4].
The second approach is to speed up the search for the
closest neighbors by using either faster searching algorithms
or approximating the nearest neighbors instead of finding the
exact ones. This approach was used by Chen, Guo, and Wang
in their work Nearest Neighbor Classification by Partially
Fuzzy Clustering [5]. Both approaches have found notable
success in the research field, and allowed the implementation
of the k-NN classifier on big data with plenty of success.
III. L ITERATURE S EARCH M ETHODOLOGY
Since this is a Systematic Literature Review, it is necessary
that the methodology for the search of the literature to be
considered in the beginning, and the eventual selection of the
articles that were included in this review, both are outlined in
detail to ensure the removal of any bias associated with the
reviewer. Following are the guidelines that were followed for
the search and selection of literature to be reviewed in this
survey.
A. Search Specifications
The search terms were used as they appear in the topic of
this article, with the exception that ‘reducing’ was also used
in addition to ‘improving’ with the help of Boolean operators
(AND and OR) available in the Advanced Search option of the
IEEE Xplore digital library. This was done to ensure that no
research work on the concerned topic may be overlooked. The
search was carried out only on IEEE Xplore for simplification
and time-saving purposes. The range of time period for the
search was chosen to be the maximum possible available in
the IEEE Xplore library.
The specifications are listed below:
• Search Term(s): (((improving) OR (reducing)) AND time
complexity of ((k Nearest Neighbours) or (kN N ))
• Database(s) Searched: IEEE Xplore Digital Library
• Search Date: 8th March, 2019
• Search Duration: 2004 - 2019
• Search Filters: None
B. Publication Selection Criteria
1) Inclusion Criteria
 •Studies that have proposed such novel modifications
to the kN N classifier that improve or reduce its’
time complexity.
• Studies that have only applied the kN N classifier
for a specific task in their research, but while
proposing novel modifications to the algorithm that
particularly reduce its time complexity.
2) Exclusion Criteria
• Studies that have simply applied the kN N classifier
(without any novel modification) for a particular
task in their research, for example, Jun et al. used
the kN N (in addition to other ML algorithms)
for the purpose of internet traffic identification and
classification [6].
• Studies that have proposed modifications to kN N
classifier which improve it in ways other than re-
ducing or bettering its time complexity.
IV. D ECREASING THE T RAINING S ET S IZE
The most time-consuming part of the kN N classification
is the calculation of the distance of the test data point to each
point in the training data. For a data set D with n training
samples and d dimensions, the distance calculation is bound
to take θ(nd) time. Here, we will look at a various number
of methods that attempt to reduce this time by reducing the
number of training samples n and thus shortening the time
for distance calculations. The following notation will be used
throughout this section; the original large training data set is
referred to as D, while the new reduced data set to be obtained
is referred to as S.
Some of methods that propose the reduction in the size of
training sets have not been specifically devised to improve
the time or computational complexity of the kN N classifier.
Instead, they are general-purpose approaches designed to deal
with big data that would otherwise take too much space
to store and too much time to train a model on. These
approaches are referred to, in some literature, as instance
selection techniques, and are widely used to preprocess huge
data sets before being used for ML applications.
One of the oldest of these approaches was the Condensed
Nearest Neighbor (CNN) rule [4] introduced by Hart, one of
the two founders of kN N . The CNN works by removing those
points from the data set that share a dense region of the data
space with a lot of other points that belong to the same class
as the subject data point. The algorithm starts out by moving
one random data point of each class from D to S. Then all the
remaining points in D are classified using kN N over S. Every
point in D that is misclassified is moved to S. This process
is repeated until there are no misclassifications in one whole
iteration through D, i.e. all the remaining points in D can be
successfully classified using kN N over S. Then S is used
as the new reduced data set for test data classifications. The
CNN algorithm has been shown to be very effective where the
underlying densities of the different classes have little to no
overlap. It condensed a data set provided by IBM from 6295
to 197 training samples. However, the error rate resulting from
the classification using the condensed data set was 1.28, which
was higher than other classifiers [4]. This was attributed to the
fact that CNN is prone to noise as it tends to always move
outliers to the selected data set S.
A complement to the CNN rule was proposed in the form
of the Reduced Nearest Neighbor (RNN) rule [7]. Instead
of choosing one sample in the reduced set S and working
upwards (incremental search), RNN employs a decremental
search. After copying all data points in D to S, RNN removes
all those points from S, that can be removed without a
misclassification being introduced into S. Results have shown
that using RNN results in S sets that are smaller in size than
those produced through CNN. Moreover, RNN is less sensitive
to noise, as it successfully removes outliers from S. Both CNN
and RNN have been known to give either a similar or smaller
error rate on test data than kN N [7], so it can be safely
surmised that the improvement in time and space complexity
does not come at the cost of accuracy.
Similar modifications to the kN N classifier are the Edited
Nearest Neighbor (ENN) [8] algorithm, and its refined version
AllKNN [9], which are both decremental search algorithms.
After copying all data points from D to S, ENN removes
those points that are mislabeled when classified using the
kN N classifier. AllKNN uses all values of k from 1 up to a
preassigned number to check for misclassified examples before
removing them. Although both ENN and All k-NN deal well
with noise by removing outliers, they are not as effective as
CNN [4], or RNN [7] in reducing data set size, as they mostly
retain the points that are in dense regions of shared identity
(i.e. have same class). However, ENN may also remove those
points that lie on the decision or class boundaries, and hence,
may result in oversimplification of the data set.
The Variable Similarity Metric (VSM) [10] is another
decremental search method, and was an important variation on
the RNN rule. VSM proposed to remove all those data points
from S, whose k nearest neighbours, all have the same class.
The removal criteria does not take the class of the subject point
into account, and hence, results in the removal of both outliers
and points that are placed in the middle of dense regions of
shared identity. VSM lies between CNN/RNN [4], [7] and
ENN [8] on the spectrum of effectiveness in reducing data set
size.
A series of similar algorithms, with minor changes, were
introduced by Aha et al. [11], [12] in their work on Instance-
based (IB) learning algortihms. The IB1 and IB2 are analogous
to the CNN, except that IB2 does not need to start with a
random point from each class in S, and it only has one iteration
through D. The IB3, IB4, and IB5 are consecutive improve-
ments on IB2, but not in the domain of time complexity, so
they shall not be discussed here.
Another series of reduction techniques are the Decremental
Reduction Optimization Procedure (DROP) algortihms [13],
that can be seen as an improvement to RNN. They start with
DROP1, which is different to RNN only on the basis that the
accuracy check for the removal criteria is done on S instead of
D. In DROP2, the removed data points are also included into
the removal criteria instead of just the ones remaining in S.
DROP3 improves on its’ predecessor by removing noisy data
points from S using ENN. DROP4 and DROP5 are further
modifications but do not add much to the cause of reducing
time complexity.
Hit Miss Networks (HMN) have also been used in reduction
techniques [14] and employ decremental search methods.
HMNs can be defined as graphs that have edges drawn from
each point in the data set to all of its nearest neighbors. An
edge going to a neighbor that has the same class as the subject
point is defined as a hit edge, while a miss edge is an edge
going to a neighbor of a different class. Hit and miss degrees
are calculated for every point based on the lengths of the edges,
and these degrees are then used to make multiple rules that
become the criteria for the removal of data points from S.
The Modified Nearest Neighbor Classifier Based on Cluster-
ing (MNNCBC) [15] employs the clustering technique to re-
move parts of a data set instead of just data points. MNNCBC
first divides the training set into n number of partitions. By
setting a tight bound c < n < 2c, the method ensures that the
clusters are representatives of the inherent classes within the
data. Each cluster is labeled with the class of the majority of
its members. Then, these clusters are only included in S, the
reduced data set, if they increase the classification accuracy of
the training and test data sets.
In a duet of works on data set reduction [16], [17], the
Local Density-Based Instance Selection (LDIS) and the Cen-
tral Density-Based Instance Selection (CDIS) algorithms were
introduced. In these methods, the focus is shifted from trying
to retain the class boundary to finding dense regions of shared
identity (data points with same class). This is similar to CNN,
but LDIS does this using local search (only within each class)
instead of global search (in the whole data set) [16]. Every data
point is assigned a density value based on its distance from
other members of its class. Then all data points that satisfy
a preset standard of density are included in S, and the rest
are rejected. CDIS is mostly similar to LDIS, except that it
changes the calculation of the density value to be based on the
distance from a centroid in a partial k-member-neighborhood,
instead of distances from all class member points [17].
A pre-classification based approach to reducing data set size
devised specifically for improving kN N s’ time complexity
incorporated the use of decision trees [18]. With all data points
falling in at least one leaf node of the data set, the method
sets the following bounds for p, the classification probability
(probability of a data-point belonging to the positive class);
p < α and p > (1 − α) where α is a predetermined constant
within the range 0 < α < 0.5. Only the data points that
satisfy the aforementioned bounds for p are included in S.
This ensures that the noisy data points that are lying on the
decision boundaries are eliminated. Like ENN [8], this may
result in an oversimplification of the data set.
The effectiveness of some of the aforementioned algorithms
can be seen in the table below which consolidates their
reduction and accuracy percentages on the Iris data set, where
available.
TABLE I
T HE R EDUCTION AND ACCURACY % OF VARIOUS DATA S ET S IZE
R EDUCTION A LGORITHMS WHEN IMPLEMENTED ON IRIS ( FROM [13],
[14]
Algorithm Reduction (%) Accuracy(%)
kNN 0 94
CNN 87.26 90
ENN 5.26 95.33
IB3 80.22 94.67
DROP3 85.19 95.33
HMN-EI 95.4 75.2
MNNCBC N/A 97.06
LDIS 87 93
CDIS 87 94
V. ACCELERATING OR A PPROXIMATING THE N EIGHBOR
S EARCH
While the most time consuming part of kN N is the distance
calculation, the search for the nearest neighbors can be an
equally tedious process for large data sets. For traditional
kN N , which employs an exhaustive search method to find the
k nearest neighbors, the time complexity of the subject step is
θ(nk). Reducing data set size automatically reduces the time
complexity for this step also. However, there are methods that
employ techniques other than data set size reduction to either
speed up the search for the nearest neighbors, or approximate
their location instead of finding the exact points.
One example of this would be the kNNModel based ap-
proach [19], which creates representative (rep) portions of
the data set based on the point di which has the largest
neighborhood containing the same class members as itself.
Each rep has the following properties; Cls(di ); the class label
of di , Sim(di ); the point within rep with the largest distance
to di , Num(di ); the number of data points in rep, and Rep(di );
an identifier of rep itself. Once all data is broken down into
reps, the classification of a test point a is made on the basis
of the following inequality: dist(a, di ) < Sim(di ). If the
inequality is satisfied only for one rep, a is assigned to that
reps’ Cls(di ). If it is satisfied for multiple reps, a is assigned
to the Cls(di ) of the rep with the largest Num(di ). While
the kN N Model-Based approach retains almost all the data
points of the original training set, it speeds up the neighbor
search by looking for the nearest representative portions of the
data instead of the nearest neighbors. A further modification
was proposed to this approach with the introduction of e-
kNNModel [5], where the rep is represented by the mean of all
the points inside it, instead of di , the point which was the basis
of its formation. Using the centroid of the data points in each
rep make it a better representative of the actual distribution of
the data points.
A series of algorithms proposed by various authors [20],
[21], [22] used the Haar Wavelet transform to decompose the
feature vectors of the training data. To begin, k random data
points are inducted into the set of the nearest neighbors K,
of a test sample a. The possibility of a training sample di
being one of the k nearest neighbors of a, is rejected if Di
(the Wavelet Coefficient Vector (WCV) of di ) has a lower
similarity rating with A (the WCV of a), than all of the
data points already in set K. Hence, the need for calcula-
tion of the distance between a and di was eliminated. This
was the principle introduced in the WKPDS algorithm [20]
that rejected points based on partial distance searching. The
WKENNS algorithm [21] further refined the idea by using
the approximate coefficients of the fully decomposed feature
vectors as a means to establish the rejection criteria for the
training data points. The WKEENNS algorithm [22] added
variance calculations to the filtering criteria in addition to the
approximate coefficients. This resulted in the rejection of even
more training data points before the actual distance calculation,
thereby reducing time complexity even further.
While proposing a couple of new algorithms for recom-
mender systems for both new and old users, the authors
employed Singular Value Decomposition (SVD) to increase
time efficiency of kN N which was being used to classify old
users [23]. SVD is used to decompose the similarity matrix
into factors with latent feature space before using kN N to
identify the nearest neighbors. The method is analogous to
other dimensionality reduction techniques and helps reduce the
distance calculations needed to identify the nearest neighbors.
Many of the methods for speeding up neighbor searching
combine clustering with kN N to identify a particular area
of the data set in which to look for the k nearest neighbors.
One such method was employed in [24] where the k-means
algorithm is first used to partition the training data set into kc
clusters, where k is a non-negative integer and c is the inherent
number of classes in the training data set. For a test sample
a, the criteria for rejecting a cluster is as follows: d(a, ci ) >
dmax , where ci is the respective cluster center, and dmax is
the distance from the cluster center to the farthest data point
in that cluster. Once the nearest cluster(s) are identified, the
search for the k nearest neighbors is carried out only inside
that cluster, reducing the time complexity significantly.
A similar implementation of clustering to find relevant
search areas was introduced where an enhancement was made
in the form of assigning weights to each attribute in proportion
to its information (entropy) [25]. After dividing the training
data into clusters using any standard clustering algorithm,
the nearest cluster to a test point, a, is chosen based on the
Euclidean distance to each of the cluster centers, ci . Then the
k nearest neighbors of a are identified in that cluster based
on Weighted Euclidean distances between a and each cluster
member. Again, clustering ensures that a specific search area is
first identified before looking for the nearest neighbors, thereby
saving significant time and computation.
The MNNCBC [15] algorithm mentioned in Section IV also
employs the same approach by classifying a test data point
based on the class label of the nearest cluster instead of the
nearest neighboring data points.
A unique approach was adopted when utilizing the emerg-
ing technique of Stochastic Computing (SC) for the specific
reduction of the kN N s hardware cost [26], which indirectly
also resulted in an improvement in its time complexity. A
modified version of the SC technique was used in the dis-
tance matrix calculations required for identifying the nearest
neighbors. Further, dimensionality reduction techniques were
also employed to increase the accuracy of the SC computations
as they are adversely affected by high dimensions.
Another improved and scalable version of kN N titled BAL-
LKNN [27] was proposed for improving the time complexity of
kN N using a subtraction-and-comparison approach to reduce
the distance computations required to identify the nearest
neighbors. BALLKNN starts by inducting k training samples
into the set of nearest neighbors S0 , of a test sample a. It
also stores radius; the Euclidean distance of a to sf , the
point in S0 farthest from a, and max; the square of radius.
Then, for every remaining data point di in the training set,
feature-wise distances between a and di are compared to the
radius. If all of the feature-wise distances are lesser than
the radius, only then is the distance of a to di calculated
and compared to radius to see if it needs to replace sf in
S0 . Otherwise, the point is discarded from the considerations
and no more calculations are required, resulting in saving
significant time and computation. Experiments indicated that
BALLKNN scaled well with an increase in dimensions while
performing as accurately as kN N with much lower time
complexity.
VI. C ONCLUSION AND F UTURE W ORK
Many techniques have been proposed to reduce or improve
the time complexity. As we looked at many of those techniques
in this review, we saw that they are quite effective at making
use of various methods to counter the curse of dimensionality
and big data. This makes the application of the kN N classifier
possible on bigger data sets when it would otherwise have
not been possible. When looking at methods that attempt to
reduce the size of training sets, it can be seen from Table I
that DROP3 is one of the best performers, as it is able to
balance the reduction-accuracy trade-off in the best manner,
while others like the HMN-EI or the ENN, either have too
small an impact on the data set size, or lose too much accuracy
while attempting the reduction.
An interesting thing to note is that the recently introduced
MNNCBC algorithm achieves the highest accuracy on the Iris
data set. While its’ reduction percentage is not available, the
fact that it employs clustering to further speed up the neighbor
search makes it a very tempting choice to use with big data.
Clustering, or similar partitioning of data, seems to be the go-
to approach when attempting to speed up or approximate the
neighbor searching process, as it has a drastic effect on the
time complexity. However, there is scant work on determining
which clustering approach is the best for both the efficiency
and the accuracy of the kN N , and this should be explored
further.
While there are many methods available to modify kN N
for use with big data, as presented in this review, there is
no way to compare them, as some are just proof-of-concepts
without any testing data, and others that have been tested have
been done on either small data sets like Iris or different data
sets thus making the studies incomparable. Hence, there is
a need for a meta-analysis that compares the effectiveness [24] H. Hong, G. Juan, and W. Ben, “An improved knn algorithm based on
of these approaches on actual big data of varying types and adaptive cluster distance bounding for high dimensional indexing,” in
3rd Global Congr. Intell. Syst., Nov 2012, pp. 213–217.
constitutions that would give a clearer picture on what are the [25] S. Taneja, C. Gupta, K. Goyal, and D. Gureja, “An enhanced k-nearest
best approaches to implement kN N on big data. neighbor algorithm using information gain and clustering,” in 4th Int.
Conf. on Adv. Comput. Commun. Technol., Feb 2014, pp. 325–329.
[26] D. Cannisi and B. Yuan, “Design space exploration for k-nearest neigh-
R EFERENCES bors classification using stochastic computing,” in IEEE Int. Workshop
on Sign. Process. Syst. (SiPS), Oct 2016, pp. 321–326.
[1] M. Pelillo, “Alhazen and the nearest neighbor rule,” Pattern Recognition [27] P. Lv, P. Yang, Y. Dong, and L. Gu, “Ballknn: An efficient and scalable
Letters, vol. 38, p. 34–37, 03 2014. knn based on euclidean similarity,” in Int. Joint Conf. Neural Netw.
[2] T. Cover and P. Hart, “Nearest neighbor pattern classification,” IEEE (IJCNN), July 2016, pp. 5141–5148.
Trans. Info. Theory, vol. 13, no. 1, pp. 21–27, January 1967.
[3] L. Gyorfi and Z. Gyorfi, “An upper bound on the asymptotic error prob-
ability on the k-nearest neighbor rule for multiple classes (corresp.),”
IEEE Trans. Info. Theory, vol. 24, no. 4, pp. 512–514, July 1978.
[4] P. Hart, “The condensed nearest neighbor rule (corresp.),” IEEE Trans.
Info. Theory, vol. 14, no. 3, pp. 515–516, May 1968.
[5] L. Chen, G. Guo, and S. Wang, “Nearest neighbor classification by
partially fuzzy clustering,” in 26th Int. Conf. Adv. Information Netw.
and Appl. Workshops, March 2012, pp. 789–794.
[6] L. Jun, Z. Shunyi, L. Yanqing, and Z. Zailong, “Internet traffic classi-
fication using machine learning,” in 2nd Int. Conf. Commun. and Netw.
in China, Aug 2007, pp. 239–243.
[7] G. Gates, “The reduced nearest neighbor rule (corresp.),” IEEE Trans.
Info. Theory, vol. 18, no. 3, pp. 431–433, May 1972.
[8] D. L. Wilson, “Asymptotic properties of nearest neighbor rules using
edited data,” IEEE Trans. Syst., Man, and Cybern., vol. SMC-2, no. 3,
pp. 408–421, July 1972.
[9] I. Tomek, “An experiment with the edited nearest-neighbor rule,” IEEE
Trans. Syst., Man, and Cybern., vol. SMC-6, no. 6, pp. 448–452, June
1976.
[10] D. G. Lowe, “Similarity metric learning for a variable-kernel classifier,”
J. Neural Comput., vol. 7, no. 1, pp. 72–85, Jan 1995.
[11] D. W. Aha, D. Kibler, and M. K. Albert, “Instance-based learning
algorithms,” J. Mach. Learn., vol. 6, no. 1, pp. 37–66, Jan 1991.
[Online]. Available: https://doi.org/10.1007/BF00153759
[12] W. Aha, “Tolerating noisy, irrelevant, and novel attributes in instance-
based learning algorithms,” Int. J. Man-Mach. Studies, vol. 36, pp. 267–
287, 02 1992.
[13] D. R. Wilson and T. R. Martinez, “Reduction techniques
for instance-based learning algorithms,” J. Mach. Learn.,
vol. 38, no. 3, pp. 257–286, Mar 2000. [Online]. Available:
https://doi.org/10.1023/A:1007626913721
[14] E. Marchiori, “Hit miss networks with applications to instance selec-
tion,” J. Mach. Learn. Research, vol. 9, pp. 997–1017, 06 2008.
[15] M. M. Samadpour, H. Parvin, and F. Rad, “Diminishing prototype size
for k-nearest neighbors classification,” in 14th Mexican Int. Conf. Artif.
Intell. (MICAI), Oct 2015, pp. 139–144.
[16] J. L. Carbonera and M. Abel, “A density-based approach for instance
selection,” in IEEE 27th Int. Conf. Tools Artif. Intell. (ICTAI), Nov 2015,
pp. 768–774.
[17] ——, “A novel density-based approach for instance selection,” in IEEE
28th Int. Conf. Tools with Artif. Intell. (ICTAI), Nov 2016, pp. 549–556.
[18] H. Xie, D. Liang, Z. Zhang, H. Jin, C. Lu, and Y. Lin, “A novel pre-
classification based knn algorithm,” in IEEE 16th Int. Conf. Data Mining
Workshops (ICDMW), Dec 2016, pp. 1269–1275.
[19] G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer, “Knn model-based
approach in classification,” in On The Move to Meaningful Internet
Systems 2003: CoopIS, DOA, and ODBASE, R. Meersman, Z. Tari, and
D. C. Schmidt, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg,
2003, pp. 986–996.
[20] W.-J. Hwang and K.-W. Wen, “Fast knn classification algorithm based
on partial distance search,” Electron. Lett., vol. 34, pp. 2062 – 2063, 11
1998.
[21] J.-S. Pan, Y.-L. Qiao, and S.-H. Sun, “A fast k nearest neighbors
classification algorithm,” IEICE Trans. Fundam. of Electron., Commun.
and Comput. Sci., vol. E87A, pp. 961–963, 04 2004.
[22] Y.-L. Qiao, J.-S. Pan, and S.-H. Sun, “Improved k nearest neighbor
classification algorithm,” Tien Tzu Hsueh Pao/Acta Electron. Sinica,
vol. 33, pp. 1101 – 1104 vol.2, 01 2005.
[23] L. Xiong, Y. Xiang, Q. Zhang, and L. Lin, “A novel nearest neighbor-
hood algorithm for recommender systems,” in 3rd Global Congr. Intell.
Syst., Nov 2012, pp. 156–159.