K-Nearest Neighbor

L9
 Machine Leaning Model

2
Source: https://www.mathworks.com/help/stats/machine-learning-in-matlab.html
 Four main families of machine learning
algorithm

• Similarity-based learning (e.g., K-NN)

• Information-based learning (e.g., Decision Tree)
• Probability-based learning (e.g., Naïve Bays)
• Error-based learning (e.g., Logistic Regression)

PP
K-Nearest Neighbors Algorithm

• The k-nearest neighbors (KNN) algorithm is a simple,

easy-to-implement supervised machine learning
algorithm.

4
 K-NN: Example

Source: https://people.revoledu.com/kardi/tutorial/KNN/KNN_Numerical-example.html
5
 Supervised machine Learning: K-Nearest
Neighbors (K-NN)
The K-NN working:
Step-1: Select the number K of the neighbours
Step-2: Calculate the Euclidean distance of new
data point to training data points
Step-3: Take the K nearest neighbours as per the
calculated Euclidean distance.
Step-4: Among these k neighbours, count the
number of the data points in each category or class.
Step-5: Assign the new data points to that category
for which the number of the neighbour is
maximum.
6
Source: https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning
K-NN: Example (Contd..)

7
K-NN: Example (Contd..)

8
K-NN: Example (Contd..)

9
K-NN: Another Example
You decided to purchase a personal computer (C) online from
Flipkart. You take decision based on K-Nearest neighbour
classifier. Training data set is available in the following table in
which each data point (each computer) has two features; first,
grade of computer based on user review (P1) and second, price of
the computer (P2). What will be the decision if one computer has
P1=5 and P2=2. Follow step-by step procedure to decide.
Assume that number of nearest neighbour, K=04.
Computer P1 P2 Decision

C1 6 7 Yes

C2 8 5 Yes

C3 4 6 No

C4 10 9 No

C5 9 7 Yes

10
 About K-NN Classifier
Advantages of KNN
• No Training Period: KNN is called Lazy Learner (Instance based
learning).
• New data can be added seamlessly which will not impact the
accuracy of the algorithm.
• KNN is very easy to implement for multi-class problem. There are
only two parameters required to implement KNN i.e. the value of K
and the distance function (e.g. Euclidean or Manhattan etc.)
Disadvantages of KNN
• K-NN slow algorithm
• Optimal number of neighbors
• Does not work well with large dataset.
• Does not work well with high dimensions
• Need feature scaling
11
http://theprofessionalspoint.blogspot.com/2019/02/advantages-and-disadvantages-of-knn.html
12