k-nearest neighbor

© All Rights Reserved

0 views

k-nearest neighbor

© All Rights Reserved

- The Lion Way Machine Learning Plus Intelligent Optimization
- MANU B.N Visualcv Resume
- licensePlateSegment
- Proc Means
- PyMVPA: A Python toolbox for multivariate pattern analysis of fMRI data
- A Survey of Data Mining Techniques for Socialmedia Analysis
- ClementineUsersGuide_10.0
- (Www.entrance-exam.net)-PTU MCA 3rd Semester Sample Paper 3
- Copy of PHY 532 Computational Physics Assignment
- Vol5I1p9
- Lagrange Newton Polynomial Code
- es_sem04_paper_04307909
- j 25043046
- Orange
- Improving classification Accuracy of Neural Network through Clustering Algorithms
- tu3 weka tutorials
- A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
- 111 1460444112_12-04-2016.pdf
- Applying k-Nearest Neighbour in Diagnosing Heart Disease Patient
- Minimal Realizations Lecture Notes Linear System and Control

You are on page 1of 19

Learning

Classification is the process

of classifying the data with the

help of class labels. On the other

hand,Clustering is similar

to classification but there are no

predefined class

labels. Classification is geared with

supervised learning. As

against, clustering is also known as

unsupervised learning

Nearest Neighbor Classifier

Nearest-neighbor classifiers are

based on learning by analogy, that

is, by comparing a given test tuple

with training tuples that are similar

to it.

Nearest Neighbor Classifier

The training tuples are described by

n attributes. Each tuple represents a

point in an n-dimensional space. In

this way, all of the training tuples

are stored in an n-dimensional

pattern space.

Nearest Neighbor Classifier

When given an unknown tuple, a k-

nearest-neighbor classifier searches

the pattern space for the k training

tuples that are closest to the

unknown tuple. These k training

tuples are the k “nearest neighbors”

of the unknown tuple.

Closeness

defined in terms of a distance

metric, such as Euclidean distance.

The Euclidean distance between two

points or tuples, say, X1 = (x11,

x12, … , x1n) and X2 = (x21, x22,

... , x2n) is

Class Label

For k-nearest-neighbor

classification, the unknown tuple is

assigned the most common class

among its k nearest neighbors.

When k = 1, the unknown tuple is

assigned the class of the training

tuple that is closest to it in pattern

space.

1-Nearest Neighbor

3-Nearest Neighbor

K-Nearest Neighbor

An arbitrary instance is represented by

(a1(x), a2(x), a3(x),.., an(x))

ai(x) denotes features

Euclidean distance between two instances

d(xi, xj)=sqrt (sum for r=1 to n (ar(xi) -

ar(xj))2)

Continuous valued target function

mean value of the k nearest training

examples

K-Nearest Neighbor

Here is step by step on how to compute K-

nearest neighbor algorithm:

1. Determine parameter K= number of nearest

neighbors

2. Calculate the distance between the query-

instance and all the training samples

3. Sort the distance and determine nearest

neighbors based on the K-th minimum distance

4. Gather category Y of nearest neighbors

5. Use majority of the category of nearest

neighbors as prediction value of the query

instance

Example

We have data of two attributes to classify

whether a special tissue is ‘good’ or ‘bad’.

X1 = Acid X2 = Strength Y=

Durability (kg/ Square Classification

(Seconds) meter)

7 7 Bad

7 4 Bad

3 4 Good

1 4 Good

Example

Now the factory produces a new tissue that pass

laboratory test with X1 = 3 and X2 = 7. Without

another expensive survey, can we guess what

the classification of this new tissue is?

1. Determine parameter K= number of nearest

neighbors. Suppose use K = 3

Example

Now the factory produces a new tissue that pass

laboratory test with X1 = 3 and X2 = 7. Without

another expensive survey, can we guess what

the classification of this new tissue is?

1. Determine parameter K= number of nearest

neighbors. Suppose use K = 3

Example

2. Calculate the distance between the query-

instance and all the training samples

Durability (kg/ Square query instance (3, 7)

(Seconds) meter)

7 7 (7-3)2 + (7-7)2 = 16

7 4 (7-3)2 + (4-7)2 = 25

3 4 (3-3)2 + (4-7)2 = 9

1 4 (1-3)2 + (4-7)2 = 13

Example

3. Sort the distance and determine nearest

neighbors based on the K-th minimum distance

Durability Strength (kg/ query instance (3, 7) minimum included

(Seconds) Square distance in 3-

meter) nearest

neighbors

?

7 7 (7-3)2 + (7-7)2 = 16 3 Yes

7 4 (7-3)2 + (4-7)2 = 25 4 No

3 4 (3-3)2 + (4-7)2 = 9 1 Yes

1 4 (1-3)2 + (4-7)2 = 13 2 Yes

Example

4. Gather category Y of nearest neighbors

X1 = X2 = Rank Is it Y=

Acid Strength minimu include Categor

Durabili (kg/ m d in 3- y of

ty Square distance nearest nearest

(Secon meter) neighbo neighbo

ds) rs? rs

7 7 3 Yes Bad

7 4 4 No -

3 4 1 Yes Good

1 4 2 Yes Good

Example

5. Use majority of the category of nearest

neighbors as prediction value of the query

instance

We have 2 good and 1 bad. So test tuple is

included in good class.

Exercise

Find class label of the values (Age = 26, Income

= 18)

($/hr)

25 15 Yes

20 10 No

30 20 Yes

35 25 No

- The Lion Way Machine Learning Plus Intelligent OptimizationUploaded byivansanc
- MANU B.N Visualcv ResumeUploaded bysubashdas1990
- licensePlateSegmentUploaded byapi-26400509
- Proc MeansUploaded bykunal2301
- PyMVPA: A Python toolbox for multivariate pattern analysis of fMRI dataUploaded byreynolds534
- A Survey of Data Mining Techniques for Socialmedia AnalysisUploaded byapkarthick
- ClementineUsersGuide_10.0Uploaded byAndris_Sushchi_2405
- (Www.entrance-exam.net)-PTU MCA 3rd Semester Sample Paper 3Uploaded byMangesh Malvankar
- Copy of PHY 532 Computational Physics AssignmentUploaded bysanahalpapei
- Vol5I1p9Uploaded byJournal of Computer Applications
- Lagrange Newton Polynomial CodeUploaded byJoy Bhadhan Roy
- es_sem04_paper_04307909Uploaded byruhuna013
- j 25043046Uploaded byAnonymous 7VPPkWS8O
- OrangeUploaded byjamesbor
- Improving classification Accuracy of Neural Network through Clustering AlgorithmsUploaded byseventhsensegroup
- tu3 weka tutorialsUploaded byborjaunda
- A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDFUploaded bySaurabh Bhardwaj
- 111 1460444112_12-04-2016.pdfUploaded byEditor IJRITCC
- Applying k-Nearest Neighbour in Diagnosing Heart Disease PatientUploaded byHeshan Rodrigo
- Minimal Realizations Lecture Notes Linear System and ControlUploaded byBHAVNA AGARWAL
- #Review_mishra2014.pdfUploaded bySlamet Kacung
- pr_l1-Introduction to Pattern Recognition.pdfUploaded bycooldoubtless
- Mgw External Alarm InputsUploaded byFahmi Yasser
- Getting Started With Machine LearningUploaded byJohn Murtagh
- Pengumpulan Data LapanganUploaded byBahtiar Afandi
- TEXT EXTRACTION FROM AN IMAGE BY USING DIGITAL IMAGE PROCESSINGUploaded byIRJCS-INTERNATIONAL RESEARCH JOURNAL OF COMPUTER SCIENCE
- Piotr Kulczycki, László T. Kóczy, Radko Mesiar, Janusz Kacprzyk (Eds.) - Information Technology and Computational Physics (2017, Springer)Uploaded byNos
- Gomez Jorge ProjectUploaded byJorge Luis Gomez Ponce
- 1-s2.0-S095741741830215X-mainUploaded byAnees Malik
- Burnap Et Al (2015)Uploaded byfahrizal

- Control of Memory and Action in MinecraftUploaded bymanantomar
- Adaline,Madaline,Widrow HoffUploaded byhenrydcl
- AISCUploaded byenjoy friends
- Ex- Sloution CompendiumUploaded byAli Jaber Al Qattan
- dataUploaded byJohn Tony
- flight control, using a modified backstepping approach and a nonlinear observerUploaded bysaman749
- Performance Analysis of Local and Global Feature Selection3Uploaded bysajid
- Robust Pole Placement using Linear Quadratic Regulator Weight Selection AlgorithmUploaded byijsret
- The Origin of Art and the Destination of Thinking 1967Uploaded byAndrew Lloyd
- Intelligent Control of Robot Arm Using Artificial Neural NetworksUploaded bySergio Coco
- cse402Uploaded byAtif Rahman
- 02-HistoryofAutoCtrl---Uploaded byMelissaTaan
- Fuzzy LQR ControllerUploaded byĐỗ Nhật Minh
- Summer Report-ANN-Slope StabilityUploaded byAbhijeet Dutta
- MLP and RBFN as ClassifiersUploaded byCHAITALI29
- Leadership: The Core of Leadership, Influence, FunctionsUploaded bySAKSHI
- hw4_wi14Uploaded byAdam Hicks
- IJCSET12-03-04-002.pdfUploaded bySerag El-Deen
- AUTOMATION OF UMPIRING IN CRICKET USING FUZZY LOGICUploaded byBharadwaj Santhosh
- Competence PerformanceUploaded bySusana
- deep learningUploaded byHamed Rahimi
- 1. What is CommunicationUploaded byBuhroy R Buhroy
- Assignment RoboticsUploaded byKesava Shankar
- Systems Archetypes Basics WB002EUploaded byKaline Girão
- Nr-410408-Digital Image ProcessingUploaded byvasuvlsi
- Distilation ControllerUploaded byLaila Kusmiati
- ControlExperiments - Precision Modular ServoUploaded byNachoSainz
- PID Loop ShapingUploaded byanoopkntpc
- Experiments on using fuzzy clustering for fuzzy control system design.pdfUploaded byOualid Lamraoui
- Swarm Inteligence Seminar ReportUploaded byAbhilash Nayak

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.