H v Tn: V Vn Thnh

M s: CH0601071
Dng thut tan ID3 v Nave Bayes tm lut phn lp trong bng sau y.
T
T
1
2
3
4
5
6
7
8

Mu
tc
en
en
Rm
en
Bc
Rm
Rm
en

Chiu cao

Cn nng

Tm thc
Cao
Thp
Thp
Tm thc
Cao
Tm thc
Thp

Nh
Va phi
Va phi
Va phi
Nng
Nng
Nng
Nh

Dng
thuc?
Khng
C
C
Khng
Khng
Khng
Khng
C

Kt qu
B rm
Khng
Khng
B rm
B rm
Khng
Khng
Khng

So snh kt qa.
Bi gii:
1. Thut ton ID3
Bc 1:
Cc thuc tnh v min gi tr tng ng baogm:
Thuc tnh Mu tc c min gi tr {en, Rm, Bc}
Thuc tnh Chiu cao c min gi tr {Cao, Tm thc, Thp}
Thuc tnh Cn nng c min gi tr {Nng, Va phi, Nh}
Thuc tnh Dng thuc c min gi tr {C, Khng}
Thuc tnh Lp c min gi tr {P, N} (P ng vi khng b rm v N l ngc
li)
Khi lng thng tin cn thit quyt nh mt mu ty c thuc v lp P hay
N hay khng l:
I(p,n) = -(p/(p+n))*log2(p/(p+n))-(n/(p+n))*log2(n/(p+n))
I(5,3) = -(5/8)* log2(5/8)-(3/8)* log2(3/8) = 0,954
Tnh Entropy cho thuc tnh Mu tc
Mu tc pi
ni
I(pi,ni)
en
2
2
1
Rm
3
0
0
Bc
0
1
0
Ta c:
E(Mu tc) = (4/8)*I(2,2) + (3/8)*I(3,0) + (1/8)*I(0,1) = 0,5
Do :
Gain(Mu tc) = I(5,3) E(Mu tc) = 0,954 0,5 = 0,454
Tng t:
Tnh Entropy cho thuc tnh Chiu cao

Chiu cao
Cao
Tm thc
Thp

pi
2
1
2

ni
0
2
1

I(pi,ni)
0
0,918
0,918

E(Chiu cao) = (2/8)*I(2,0) + (3/8)*I(1,2) + (3/8)*I(2,1) = 0,689

Gain(Chiu cao) = 0,954 0,689 = 0,265
Tnh Entropy cho thuc tnh Cn nng
Cn nng pi
ni
I(pi,ni)
Nng
2
1
0,918
Va phi
2
1
0,918
Nh
1
1
1
E(Cn nng) = (3/8)*I(2,1) + (3/8)*I(2,1) + (2/8)*I(1,1) = 0,939
Gain(Cn nng) = 0,954 0,939 = 0,015
Tnh Entropy cho thuc tnh Dng thuc
Dng thuc pi
ni
I(pi,ni)
C
3
0
0
Khng
2
3
0,970
E(Dng thuc) = (3/8)*I(3,0) + (5/8)*I(2,3) = 0,607
Gain(Dng thuc) = 0,954 0,607 = 0,347
Chn thuc tnh c li thng tin ln nht l thuc tnh Mu tc, ta c cy c
dng:

Bc 2:
Trong cy ny ta thy ng vi mu tc en cn 2 phn t c tr P v 2 phn t c
tr N. Tip tc p dng ID3 cho nt con ny cho n khi t n nt l hoc nt c
entropy=0. Ta c tp d liu (con) ng vi mu tc en nh sau:
Chiu cao

Cn nng

Tm thc
Cao
Thp
Thp

Nh
Va phi
Va phi
Nh

Dng
thuc?
Khng
C
Khng
C

Kt qu
B rm
Khng
B rm
Khng

Cc thuc tnh v min gi tr tng ng baogm:

Thuc tnh Chiu cao c min gi tr {Cao, Tm thc, Thp}

Thuc tnh Cn nng c min gi tr {Va phi, Nh}

Thuc tnh Dng thuc c min gi tr {C, Khng}
Thuc tnh Lp c min gi tr {P, N} (P ng vi khng b rm v N l ngc
li)
Khi lng thng tin cn thit quyt nh mt mu ty c thuc v lp P hay
N hay khng l:
I(p,n) = -(p/(p+n))*log2(p/(p+n))-(n/(p+n))*log2(n/(p+n))
I(2,2) = 1
Tnh Entropy cho thuc tnh Chiu cao
Chiu cao
pi
ni
I(pi,ni)
Cao
1
0
0
Tm thc
0
1
0
Thp
1
1
1
E(Chiu cao) = (1/4)*I(1,0) + (1/4)*I(0,1) + (2/4)*I(1,1) = 0,5
Gain(Chiu cao) = 1 0,5 = 0,5
Tnh Entropy cho thuc tnh Cn nng
Cn nng pi
ni
I(pi,ni)
Va phi
1
1
1
Nh
1
1
1
E(Cn nng) = (2/4)*I(1,1) + (2/4)*I(1,1) = 1
Gain(Cn nng) = 0,954 1 = -0,046
Tnh Entropy cho thuc tnh Dng thuc
Dng thuc pi
ni
I(pi,ni)
C
2
0
0
Khng
0
2
0
E(Dng thuc) = (2/4)*I(2,0) + (2/8)*I(0,2) = 0
Gain(Dng thuc) = 0,954 0 = 0,954
Nh vy thuc tnh Dng thuc c li thng tin ln nht c dng phn
lp, ta c cy quyt nh do thut ton ID3 to ra nh sau:

Nh vy cc lut c to ra nh sau:

IF (Mu tc = Rm) THEN Khng rm nng

ELSE IF Mu tc = en AND Dng thuc = C THEN Khng rm nng
ELSE IF (Mu tc = Bc) THEN Rm nng
ELSE Rm nng
Hay rt gn lut nh sau:
IF (Mu tc = Rm) OR (Mu tc = en AND Dng thuc = C) THEN Khng
rm nng
ELSE Rm nng
2. Thut ton Nave Bayes
Da vo bng d liu ta c th tnh cc xc sut sau:
- Xc sut lp dng (khng rm nng): P(p) = 5/8
- Xc sut lp m (rm nng): P(n) = 3/8
Mu tc
P(en | p) = 2/5
P(Rm | p) = 3/5
P(Bc | p) = 0
Chiu cao
P(Cao | p) = 2/5
P(Tm thc | p) = 1/5
P(Thp | p) = 2/5
Cn nng
P(Nng | p) = 2/5
P(Va phi | p) = 2/5
P(Nh | p) = 1/5
Dng thuc
P(C | p) = 3/5
P(Khng | p) = 2/5

P(en | n) = 2/3
P(Rm | n) = 0
P(Bc | n) = 1/3
P(Cao | n) = 0
P(Tm thc | n) = 2/3
P(Thp | n) = 1/3
P(Nng | n) = 1/3
P(Va phi | n) = 1/3
P(Nh | n) = 1/3
P(C | n) = 0
P(Khng | n) = 3/3

Xt mt mu X c mu tc Rm (khng quan tm cc thuc tnh khc)

P(Rm | n) = 0 => P(X|n)*P(n) = 0
P(xi | p) > 0 => P(X | p) > 0
Suy ra mu X thuc lp P (khng rm nng)

Xt mt mu X c mu tc en v c Dng thuc (khng quan tm cc thuc

tnh cn li)
P(C | n) = 0 => P(X|n)*P(n) = 0
P(xi | p) > 0 => P(X | p) > 0
Suy ra mu X thuc lp P (khng rm nng)

Hai phn lp trn ph hp vi lut c suy ra t gii thut ID3. Tuy nhin, xt
mu X = <Bc,Cao,Va phi,C>, ta c:
P(Bc | p) = 0 => P(X|p)*P(p) = 0
P(Cao | n) = 0 => P(X|n)*P(n) = 0

=> Khng th xc nh X thuc lp no!