Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

M hnh cy quyt nh
Hong Th Lan Giao1
1 HKH

Hu

Ngy 15 thng 3 nm 2014

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Outline

Cc khi nim c bn

Cy quyt nh
Khi nim
Thut tan CLS xy dng cy quyt nh

Xy dng cy quyt nh da vo Entropy

Entropy
Information gain

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

H thng thng tin

A = (U, A)
Bng quyt nh
T = (U, C D)
Bng quyt nh nht qun: C D

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Li v tp rt gn
R D;

Core =

RRED

Lut quyt nh
^
(ai , vi ) (d , dj )
i

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

V d
H thng thng tin bnh nhn
U
u1
u2
u3
u4
u5
u6

au u
C
C
C
Khng
Khng
Khng

au c
C
C
C
C
Khng
C

Thn nhit
Bnh thng
Cao
Rt cao
Bnh thng
Cao
Rt cao

Cm cm
Khng
C
C
Khng
Khng
C

Cc lut quyt nh c th c:
(au c, Khng)

(Thn nhit,Cao) (Cm cm, Khng)

(au c, C)

(Thn nhit,Cao) (Cm cm, C)

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Khi nim

nh ngha
Cy quyt nh l cu trc biu din di dng cy, mi nt trong
biu din gi tr quyt nh ca thuc tnh, nhnh biu din u ra
ca quyt nh v l biu din cc lp quyt nh.
Cy quyt nh c th c dng phn lp bng cch xut pht
t gc ca cy v di chuyn theo cc nhnh cho n khi gp nt
l. Trn c s phn lp ny chng ta c th chuyn i v cc lut
quyt nh.

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Thut tan CLS xy dng cy quyt nh

Thut ton
Gi s tp gi tr ca thuc tnh quyt nh l {Y , N}.
1

To nt T bao gm ton b tp hun luyn (i tng),

Nu tt c cc mu (i tng) ca T u c gi tr quyt
nh l Y th to nt Y con ca T v dng,

Nu tt c cc mu (i tng) ca T u c gi tr quyt
nh l N th to nt N con ca T v dng

Chn mt thuc tnh X c gi tr v1 , v2 , , vi v chia T

thnh cc tp con T1 , T2 , , Ti tng ng vi cc gi tr
ca chng trn X , to i nt T1 , T2 , , Ti con ca T vi
X = vj l nhn ca nhnh t T n Tj

For Tj do: T Tj v quay li Bc 2.

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Vn : Chn thuc tnh no phn nhnh?

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Entropy

nh ngha
Cho bng quyt nh T = (U, C D). Khng mt tnh tng qut,
ta c th xem D = {d }, |Vd | = k. Phn hoch ca U i vi thuc
|Ui )|
tnh d gm cc lp: U1 , U2 , , Uk , t pi =
, i = 1, , k.
|U|
X
Entropy (U) =
pi log2 pi

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

nh ngha
Cho a C ,
Gain(U, a) = Entropy (U)

X |Uv |
Entropy (Uv )
|U|

v Va

Trong , Va l tp gi tr ca thuc tnh a, Uv l mt lp trong

phn hoch ca U i vi thuc tnh a, tng ng vi gi tr v .

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Chn thuc tnh phn nhnh

B1. V:= U;
B2. Tnh gi tr ca Entropy (V );
B3. For a C do Tnh gi tr ca Gain(V , a);
B4. Chn thuc tnh a c gi tr Gain(V , a) ln nht lm thuc
tnh phn nhnh;
C := C \ {a};
B5. For v Va
B5.1. V := Uv
B5.2. if |Vd | > 1
Hong Th Lan Giao
H h tr quyt nh

Thc hin li cc B2, B3, B4;

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

V d
U
u1
u2
u3
u4
u5
u6
u7
u8

Weight
low
low
medium
high
high
low
high
low

Door
2
4
4
2
4
4
4
2

Size
compact
sub
compact
compact
compact
compact
sub
sub

Cylinder
4
6
4
6
4
4
6
6

Mileage
high
low
high
low
low
high
low
low

C = {Weight, Door , Size, Cylinder }; D = {Mileage}.

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Phn hoch U/D

U1 = {u1 , u3 , u6 };

U2 = {u2 , u4 , u5 , u7 , u8 }

3 5
5
3
Entropy (U) = ( log2 + log2 )
8
8 8
8
Xt thuc tnh Weight
Phn hoch U thnh cc lp Uhigh = {u4 , u5 , u7 }, Umedium = {u3 }
v Ulow = {u1 , u2 , u6 , u8 }
Gain(U, Weight) = Entropy (U)
1
4
3
( Entropy (Uhigh ) + Entropy (Umedium ) + Entropy (Ulow ))
8
8
8
Hong Th Lan Giao
H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Xt thuc tnh Door

Gain(U, Door ) = Entropy (U)
3
5
( Entropy (U2 ) + Entropy (U4 ))
8
8
Xt thuc tnh Size
Gain(U, Size) = Entropy (U)
5
3
( Entropy (Ucompact ) + Entropy (Usub ))
8
8
Xt thuc tnh Cylinder
Gain(U, Cylinder ) = Entropy (U)
4
4
( Entropy (U4 ) + Entropy (U6 ))
8
8
Hong Th Lan Giao
H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Xt li V d v cc bnh nhn
Entropy (U) = ( 12 log2

1
2

+ 21 log2 12 ) = 1.

Gain(U, au u) = 1 21 ( 13 log2 13 + 32 log2 23 ) 12 ( 13 log2

2
2
2
3 log2 3 ) = log2 3 3 ' 1 0.918

1
3

Gain(U, au c) = 1 56 ( 35 log2 53 + 52 log2 52 ) 61 log2 1 ' 1 0.809

Gain(U, Thn nhit) =
1 31 (log2 1) 13 ( 21 log2

Hong Th Lan Giao

H h tr quyt nh

1
2

+ 21 log2 12 ) 13 (log2 1) =

1
3

' 1 0.667

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Chn thuc tnh phn nhnh

Thuc tnh "Thn nhit" c chn u tin. Vi thuc tnh ny
c 3 gi tr: "Bnh thng", "Cao", "Rt cao". Do , c 3 nhnh
tng ng vi 3 nt trong: U1 = {u1 , u4 } (c cng gi tr quyt
nh l "Khng"); U2 = {u2 , u5 } v U3 = {u3 , u6 } (c cng gi tr
quyt nh "C").
Lt phn nhnh tip theo chng ta xt nt U2
Entropy (U2 ) = ( 12 (log2 ) + 21 (log2 ) = 1.
Gain(U2 , au u) = 12 (log2 1 + log2 1) = 0.
Gain(U2 , au c) = 21 (log2 1 + log2 1) = 0.
Chn thuc tnh "au u" hay "au c" nh nhau.
Hong Th Lan Giao
H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Cc lut quyt nh
(Thn nhit, Bnh thng) (Cm cm, Khng)
(Thn nhit, Rt cao) (Cm cm, C)
(Thn nhit, Cao)

(au u, C) (Cm cm, C)

(Thn nhit, Cao)

(au u, Khng) (Cm cm, Khng)

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT

Cc khi nim c bn

Cy quyt nh

Xy dng cy quyt nh da vo Entropy

Information gain

Cu hi
1. Min gi tr v ngha ca Entropy , Gain.
2. H thng c xem l n nh vi gi tr no ca Entropy ?
3. u v nhc im khi xy dng cy quyt nh da vo
Entropy ?
4. Cc lut quyt nh sinh ra t cy quyt nh ?

Hong Th Lan Giao

H h tr quyt nh

Khoa CNTT