Hc My

(Machine Learning)

Thn Quang Khot

khoattq@soict.hust.edu.vn
Vin Cng ngh thng tin v Truyn thng
Trng i hc Bch Khoa H Ni
Nm 2015

Ni dung mn hc:

Gii thiu chung

Cc phng php hc c gim st

Hc da trn cc lng ging gn nht

(Nearest neighbors learning)

nh gi hiu nng h thng hc my

Cc phng php hc khng gim st

Hc My IT 4866

Hc da trn cc lng ging gn nht

Mt s tn gi khc ca phng php hc da trn cc lng

ging gn nht (Nearest neighbors learning)
Instance-based learning
Lazy learning
Memory-based learning

tng ca phng php

Qu trnh hc

(n gin l) lu li cc v d hc
Khng xy dng mt m hnh (m t) r rng v tng qut ca
hm mc tiu cn hc

i vi mt v d cn phn loi/d on

Gi tr ca hm mc tiu (mt nhn lp, hoc mt gi tr thc)

c suy ra t cc hng xm ca n.
Hc My IT 4866

Hc da trn cc lng ging gn nht

Biu din u vo ca bi ton

Mi v d c biu din l mt vect chiu trong khng gian
cc vect
= (1, 2, , ), trong l mt s thc

C th p dng c vi c 2 kiu bi ton

Bi ton phn lp (classification)
Hm mc tiu c gi tr ri rc
u ra ca h thng l mt trong s cc gi tr ri rc xc nh
trc (mt trong cc nhn lp)

Bi ton hi quy (regression)

Hm mc tiu c gi tr lin tc
u ra ca h thng l mt gi tr s thc

Hc My IT 4866

V d: bi ton phn lp

Xt 1 lng ging gn
nht
Gn z vo lp c2

Xt 3 lng ging gn
nht
Gn z vo lp c1

Xt 5 lng ging gn
nht
Gn z vo lp c1

Hc My IT 4866

Lp c1

Lp c2
V d cn
phn lp z

Gii thut k-NN cho phn lp

Mi v d hc c biu din bi 2 thnh phn:

M t ca v d: = (1, 2, , ), trong
Nhn lp : , vi l tp cc nhn lp c xc nh trc

Giai on hc
n gin l lu li cc v d hc trong tp hc:

Giai on phn lp: phn lp cho mt v d (mi)

Vi mi v d hc , tnh khong cch gia v
Xc nh tp () cc lng ging gn nht ca
Gm k v d hc trong gn nht vi tnh theo mt hm
khong cch

Phn vo lp chim s ng (the majority class) trong s cc lp

ca cc v d trong ()
Hc My IT 4866

Gii thut k-NN cho hi quy

Mi v d hc c biu din bi 2 thnh phn:

M t ca v d: = (1, 2, , ), trong
Gi tr u ra mong mun: (l mt s thc)

Giai on hc
n gin l lu li cc v d hc trong tp hc

Giai on d on: d on gi tr u ra cho v d

i vi mi v d hc , tnh khong cch gia v
Xc nh tp () cc lng ging gn nht ca
Gm k v d hc trong gn nht vi tnh theo mt hm khong
cch

D on gi tr u ra i vi :
Hc My IT 4866

()
7

k-NN: Cc vn ct li

Suy
ngh
khc
nhau!

Hc My IT 4866

k-NN: Cc vn ct li

Trng
thi
no
ging
ti
by
gi
nht?

Hc My IT 4866

k-NN: Cc vn ct li

Hm khong cch

Mi hm s tng ng vi mt cch nhn v d liu.

V hn hm!!!

Chn hm no?

Hc My IT 4866

10

k-NN: Cc vn ct li

Chn tp lng ging ()

Chn bao nhiu lng ging?

Gii hn chn theo vng?

Hc My IT 4866

11

Mt hay nhiu lng ging gn nht?

Vic phn lp (hay d on) ch da trn duy nht mt lng

ging gn nht (l v d hc gn nht vi v d cn phn
lp/d on) thng khng chnh xc
Nu v d hc ny l mt v d bt thng, khng in hnh (an
outlier) rt khc so vi cc v d khc
Nu v d hc ny c nhn lp (gi tr u ra) sai do li trong
qu trnh thu thp (xy dng) tp d liu

Thng xt (> 1) cc v d hc (cc lng ging) gn nht

vi v d cn phn lp/d on
i vi bi ton phn lp c 2 lp, thng c chn l
mt s l, trnh cn bng v t l cc v d gia 2 lp
V d: = 3, 5, 7,
Hc My IT 4866

12

Hm tnh khong cch (1)

Hm tnh khong cch d

ng vai tr rt quan trng trong phng php hc da trn cc
lng ging gn nht
Thng c xc nh trc, v khng thay i trong sut qu
trnh hc v phn loi/d on

La chn hm khong cch d

Cc hm khong cch hnh hc: Dnh cho cc bi ton c cc
thuc tnh u vo l kiu s thc ()
Hm khong cch Hamming: Dnh cho cc bi ton c cc
thuc tnh u vo l kiu nh phn ({0,1})

Hc My IT 4866

13

Hm tnh khong cch (2)

Cc hm tnh khong cch

hnh hc (Geometry distance
functions)

Hm Minkowski (-norm):

n
d ( x , z ) xi z i
i 1

1/ p

Hm Manhattan ( = 1):

d ( x, z )

xi zi

i 1

z
i i

Hm Euclid ( = 2):

d ( x, z )

Hm Chebyshev ( = ):

n
d ( x , z ) lim x i z i
p
i 1

i 1

1/ p

max x i z i
i

Hc My IT 4866

14

Hm tnh khong cch (3)

Hm khong cch
Hamming
i vi cc thuc tnh u
vo l kiu nh phn ({0,1})
V d: = (0,1,0,1,1)

Hc My IT 4866

d ( x, z )

Difference ( x i , z i )

i 1

1, if ( a b )
Difference ( a , b )
0 , if ( a b )

15

Chun ha min gi tr thuc tnh

Hm tnh khong cch Euclid:

d ( x, z )

z
i i
i 1

Gi s mi v d c biu din bi 3 thuc tnh: Age, Income (cho

mi thng), v Height (o theo mt)
= (Age=20, Income=12000, Height=1.68)
= (Age=40, Income=1300, Height=1.75)

Khong cch gia v

.

(, ) = 20 40 2 + 12000 1300 2 + 1.68 1.75 2 0 5

Gi tr khong cch b quyt nh ch yu bi gi tr khong cch (s khc
bit) gia 2 v d i vi thuc tnh Income
V: Thuc tnh Income c min gi tr rt ln so vi cc thuc tnh khc

Cn phi chun ha min gi tr (a v cng mt khong gi tr)

Khong gi tr [0,1] thng c s dng
i vi mi thuc tnh i: = / max( )
Hc My IT 4866

16

Trng s ca cc thuc tnh

Hm khong cch Euclid:

d ( x, z )

z
i i
i 1

Tt c cc thuc tnh c cng (nh nhau) nh hng i vi gi tr

khong cch

Cc thuc tnh khc nhau c th (nn) c mc nh hng

khc nhau i vi gi tr khong cch
Cn phi tch hp (a vo) cc gi tr trng s ca cc thuc tnh
trong hm tnh khong cch
n
l trng s ca thuc tnh i:

d ( x, z )

wi xi z i

i 1

Lm sao xc nh cc gi tr trng s ca cc thuc tnh?

Da trn cc tri thc c th ca bi ton (vd: c ch nh bi cc
chuyn gia trong lnh vc ca bi ton ang xt)
Bng mt qu trnh ti u ha cc gi tr trng s (vd: s dng mt tp
hc hc mt b cc gi tr trng s ti u)
Hc My IT 4866

17

Khong cch ca cc lng ging (1)

Xt tp () gm v d hc gn
nht vi v d cn phn lp/d on
Mi v d (lng ging gn nht) ny c
khong cch khc nhau n
Cc lng ging ny c nh hng nh
nhau i vi vic phn lp/d on cho
? KHNG!

V d cn
phn loi z

Nn gn cc mc nh hng (ng
gp) ca mi lng ging gn nht ty
theo khong cch ca n n
Mc nh hng cao hn cho cc
lng ging gn hn!

Hc My IT 4866

18

Khong cch ca cc lng ging (2)

Gi l hm xc nh trng s theo khong cch

i vi mt gi tr (, ) khong cch gia v
(, ) t l nghch vi (, )

i vi bi ton phn lp:

c ( z ) arg max
c j C

v ( x , z ). Identical

( c j , c ( x ))

x NB ( z )

1, if ( a b )
Identical ( a , b )
0 , if ( a b )

i vi bi ton d on (hi quy):

v ( x , z ). f ( x )

f (z)

x NB ( z )

v( x, z )

x NB ( z )

La chn mt hm xc nh trng s theo khong cch:

v( x, z )

d ( x, z )

v( x, z )

[ d ( x , z )]

Hc My IT 4866

v( x, z ) e

d ( x,z)

19

Lazy learning vs. Eager learning

Lazy learning. Vic nh gi hm mc tiu (target function)

c hon li cho n khi xt v d cn phn loi/d on
nh gi (xp x) hm mc tiu mt cch cc b (locally) v ring r
(diferrently) cho mi v d cn phn loi/d on (ti thi im phn
loi/d on ca h thng)
Tnh ton nhiu ln cc xp x cc b ca hm mc tiu
Thng mt thi gian lu hn a ra kt lun (phn lp/d on), v
cn nhiu khng gian nh hn
V d: Nearest neighbor learner, Locally weighted regression

Eager learning. Vic nh gi hm mc tiu c hon thnh

trc khi xt n bt k v d cn phn loi/d on
nh gi (xp x) hm mc tiu mt cch tng th (globally) i vi ton
b khng gian cc v d (ti thi im hc ca h thng)
Tnh ton mt xp x duy nht ( mc tng th) ca hm mc tiu
V d: Linear regression, Support vector machines, Neural networks, ...
Hc My IT 4866

20

k-NN: u nhc im

Cc u im
Chi ph thp cho qu trnh hun luyn (ch vic lu li cc v d hc)
Hot ng tt vi cc bi ton phn loi gm nhiu lp
Khng cn phi hc b phn loi cho lp

Phng php hc k-NN ( >> 1) c kh nng x l nhiu cao

Phn loi/d on c thc hin da trn lng ging gn nht

Rt Linh ng trong vic chn hm khong cch.

C th dng tng t (similarity): cosine
C th dng o khc, chng hn Kullback-Leibler divergence,
Bregman divergence,

Cc nhc im
Phi la chn hm tnh khong cch (s khc bit) thch hp vi bi ton
Chi ph tnh ton (thi gian, b nh) cao ti thi im phn loi/d on
C th cho kt qu km/sai vi cc thuc tnh khng lin quan
Hc My IT 4866

21