Perceptron nhiu lp Chng 4

4.3. Hc tng cng v hc trc tuyn

Xt 1 perceptron nhiu lp vi 1 lp u vo ca cc nt ngun, 1 hoc nhiu lp n, v 1 lp u ra gm 1 hoc nhiu n-ron; nh c minh ha trong Hnh 4.1. t T = * ( ) ( )+ (4.1)

biu th mu hun luyn c dng hun luyn mng li trong 1 dng c gim st. t ( ) biu th hm tn hiu c sinh ra u ra ca n-ron j trong lp u ra bi kch thch ( ) t vo lp u vo. Do , tn hiu li c sinh ra u ra ca n-ron j c xc nh bi ( ) ( ) ( ) (4.2)

trong ( ) l phn t th i ca vec-t p ng mong mun ( ). Theo cch k hiu ca thut ton LMS c hc Chng 3, nng lng li tc thi ca n-ron j c xc nh bi Ej(n) = ( ) (4.3)

Cng cc thnh phn nng lng li ca tt c cc n-ron trong lp u ra, chng ta biu din tng nng lng li tc thi ca ton b mng li bi E( ) ( ) ( ) (4.4)

trong tp C bao gm tt c cc n-ron trong lp u ra. Vi mu hun luyn bao gm N v d, nng lng li trung bnh trn mu hun luyn, hay ri ro theo kinh nghim (empirical risk), c xc nh bng Eav( ) ( ) ( ) (4.5)

Hin nhin, nng lng li tc thi, v do nng lng li trung bnh, l 2 hm ca tt c cc khi lng khp thn kinh c th iu chnh c (adjustable synaptic weight) (v d, cc thng s t do) ca perceptron nhiu lp. S ph thuc hm khng bao gm trong cc cng thc ca E( ) v Eav( ), n thun n gin ha cch k hiu. Ph thuc vo cch hc c gim st ca perceptron nhiu lp c biu din thc t nh th no, chng ta c th xc nh 2 phng php khc nhau l hc tng cng v hc trc tuyn, c tho lun tip theo trong bi cnh ca gradient descent xung dc.

Perceptron nhiu lp Chng 4

Hc tng cng
Trong phng php tng cng ca hc c gim st, cc s iu chnh ti cc khi lng khp thn kinh (synaptic weight) ca perceptron nhiu lp c biu din sau s trnh din ca tt c N v d trong mu hun luyn T (,mu hun luyn) cu thnh thi i (epoch) ca hun luyn. Ni cch khc, hm gi tr ca hc tng cng c xc nh bi nng lng li trung bnh Eav. Cc s iu chnh ti cc khi lng khp thn kinh (synaptic weight) ca perceptron nhiu lp c thc hin trn 1 c s tng thi i (epoch-by-epoch basis). Do , 1 s nhn thc r ca ng cong hc tp c thu c bng cch v th Eav chng li s lng cc thi i (epoch), trong , vi mi thi i (epoch) ca hun luyn, cc mu trong mu hun luyn T c sp xp ngu nhin. ng cong hc tp sau c tnh ton bi trung bnh ton b mt s lng ln va ca cc s nhn thc nh vy, trong mi s nhn thc c biu din cho 1 tp khc nhau ca cc iu kin ban u c chn ngu nhin. Vi phng php gradient descent c dng biu din hun luyn, cc thun li ca hc tng cng c lit k di y: s nh gi chnh xc ca vec-t gradient (v d, o hm ca hm gi tr Eav vi vec-t khi lng w), bng cch m bo, di nhng iu kin c bn, s hi t ca phng php steepest descent i vi 1 mc ti thiu cc b. s song song ca qu trnh hc.

Tuy nhin, theo nh gi thc t, hc tng cng yu cu phn no trong iu kin yu cu lu tr. Trong vn cnh thng k, hc tng cng c th c xem nh 1 dng ca hm thng k. N v th ph hp cho gii quyt cc vn hi quy khng tuyn tnh.

Hc trc tuyn
Trong phng php trc tuyn ca hc c gim st, cc s iu chnh ti cc khi lng khp thn kinh (synaptic weight) ca perceptron nhiu lp c biu din trn 1 c s tng v d (example-by-example basis). Hm gi tr c ti thiu do l tng nng lng li tc thi E( ). Xt 1 thi i (epoch) ca N v d hun luyn c sp xp theo th t {x(1), d(1)}, {x(2), d(2)}, , {x(N), d(N)}. Cp v d u tin {x(1), d(1)} trong thi i (epoch) c a vo mng li, v cc s iu chnh khi lng c biu di s dng phng php gradient descent. Sau cp v d th hai {x(2), d(2)} trong thi i (epoch) c a vo mng li, iu ny dn ti cc s iu chnh hn na ti cc khi lng trong mng li. Th tc ny c tip tc cho n v d cui cng {x(N), d(N)}. Khng may, th tc nh vy i ngc vi s song song ca hc trc tuyn.

Perceptron nhiu lp Chng 4 i vi 1 tp cc iu kin lc u cho trc, 1 s nhn thc n ca ng cong hc tp c thu c bng cch v th gi tr cui E( ) s lng ca cc thi i (epoch) c s dng trong phin hun luyn, ni m trc cc v d hun luyn c sp xp ngu nhin sau mi thi i (epoch). Cng nh hc tng cng, ng cong hc tp ca hc trc tuyn c tnh ton bi trung bnh ton b cc nhn thc nh vy trn 1 s lng ln ca cc iu kin lc u c chn ngu nhin. Mt cch t nhin, vi 1 mng li cu trc cho trc, ng cong hc tp t c bng vic hc trc tuyn s kh khc so vi bng vic hc tng cng. Cho rng cc v d hun luyn c a vo mng li trong 1 trng thi ngu nhin, cch s dng ca hc trc tuyn to tm kim trong khng gian khi lng a chiu ngu nhin trong t nhin; bi l do ny m phng php hc trc tuyn i khi c cp nh l phng php ngu nhin. Tnh ngu nhin ny c nh hng ng mong c trong vic to n t ging vi qu trnh hc b gi li bi 1 mc ti thiu cc b, iu m l 1 thun li v cng ca hc trc tuyn so vi hc tng cng. Mt u im khc ca hc trc tuyn l thc t n i hi t s lu tr hn hc tng cng. Hn na, khi d liu hun luyn d tha (v d mu hun luyn T cha vi sao chp ca cng mt v d), chng ta thy rng, khng nh hc tng cng, hc trc tuyn c th tn dng s d tha ny bi v cc v d c a vo 1 ln cng 1 thi im. Mt thuc tnh hu dng khc ca hc trc tuyn l kh nng theo di nhng thay i nh ca n trong d liu hun luyn, c bit khi mi trng l nguyn nhn sinh ra d liu l khng tnh. Tng kt li, mc d c nhng nhc im ca hc trc tuyn, n c a chung cao cho gii quyt cc vn phn loi mu bi 2 l do quan trng trong thc t: Hc trc tuyn l n gin thc hin. N cung cp cc gii php hiu qu cc vn trn quy m ln v phn loi mu kh.

Bi 2 l do ny m nhiu ti liu c gii thiu trong chng ny c dnh cho hc trc tuyn.

4.4. Thut ton lan truyn ngc

Tnh ph bin ca hc trc tuyn i vi hun luyn c gim st ca perceptron nhiu lp c lm ni bt hn bi s pht trin ca thut ton lan truyn ngc. miu t thut ton ny, xt Hnh 4.3, m t n-ron j ang c cung cp (feed) bi 1 tp hp cc hm tn hiu c sinh ra bi 1 lp cc n-ron bn tri n. Trng cc b c gy ra (induced local field) ( ) c sinh ra ti u vo ca hm kch thch kt hp vi n-ron j do l

Perceptron nhiu lp Chng 4 ( ) ( ) ( ) (4.6)

trong m l tng s cc u vo (bao gm c ng cho (bias)) a vo n-ron j. Khi lng khp thn kinh (synaptic weight) (tng ng vi u vo c nh ) bng ng cho (bias) a vo n-ron j. Do , hm tn hiu ( ) xut hin u ra ca n-ron j ti ln lp n l ( ) ( ( )) (4.7)

Trong 1 trng thi tng t vi thut ton LMS c hc Chng 3, thut ton lan truyn ngc a vo 1 sa li ( ) ti khi lng khp thn kinh (synaptic weight) ( ), m tng ng vi o hm ring biu din gradient ny nh
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

. Theo nguyn tc chui ca gii tch, chng ta c th

(4.8)

o hm ring

( ) ( )

m t 1 h s nhy, xc nh hng tm kim trong khng gian khi lng . ( ), chng ta thu c (4.9)

i vi khi lng khp thn kinh (synaptic weight) Ly vi phn 2 v ca phng trnh (4.4) vi
( ) ( )

( ) ( ), chng ta thu c

Ly vi phn 2 v ca phng trnh (4.2) vi

( ) ( )

(4.10) ( ), chng ta c ( ( )) (4.11)

Tip , ly vi phn phng trnh (4.7) vi

( ) ( )

trong s dng ca du phy (bn v phi) biu th s ly vi phn vi argument. Cui cng, ly vi phn phng trnh (4.6) vi ( ) thu c
( ) ( )

( )

(4.12)

Thay t phng trnh (4.9) ti (4.12) vo phng trnh (4.8) thu c

( ) ( )

( )

( ( )) ( )

(4.13)

Perceptron nhiu lp Chng 4 Sa sai ( ) a vo ( ) c nh ngha bi nguyn tc delta, hay ( )

( ) ( )

(4.14)

trong l thng s t l hc ca thut ton lan truyn ngc. Du trong phng trnh (4.14) gii thch cho gradient descent trong khng gian khi lng (v d, tm 1 hng cho s thay i khi lng m lm gim gi tr ca E( )). Do , thay phng trnh (4.13) vo phng trnh (4.14) thu c ( ) trong gradient cc b ( ) c nh ngha bi ( )
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

(4.15)

( )

( ( ))

(4.16)

Perceptron nhiu lp Chng 4 Trng hp 1: N-ron j l 1 nt u ra Khi n-ron j c t trong 1 lp u ra ca mng li, n c cung cp vi 1 phn hi mong mun ca chnh n. Chng ta c th s dng Phng trnh (4.2) tnh ton tn hiu li ( ) c lin kt vi n-ron; xem Hnh 4.3. C ( ) xc nh, chng ta n gin vn tnh ton qu tch gradient ( ) bng cch s dng phng trnh (4.16). Trng hp 2: N-ron l 1 nt n Khi n-ron j c t trong 1 lp n ca mng li, khng c phn hi mong mun xc nh cho n-ron ny. Do , tn hiu li vi 1 n-ron n phi c xc nh quy v hot ng ngc trong iu kin cc tn hiu li ca tt c cc n-ron m n-ron n c kt ni trc tip; iu ny ti s khai trin ca thut ton phn hi ngc tr ln phc tp. Xt 1 trng hp Hnh 4.4, trong miu t n-ron j nh l 1 nt n ca mng li. Theo phng trnh (4.16), chng ta c th nh ngha li qu tch gradient ( ) i vi n-ron n j nh l ( ) =
( ) ( ) ( ) ( ) ( ) ( )

( ( )) n-ron j c n

(4.17)

trong dng th 2 chng ta s dng phng trnh (4.11). tnh o hm tng phn
( ) ( )

, chng ta c th tin hnh nh sau: T Hnh 4.4, chng ta thy rng E( ) = ( ) n-ron k l 1 nt u ra (4.18)

l phng trnh (4.4) vi ch s k thay cho ch s j. Chng ta thc hin s thay th ny trnh ln ln vi vic dng ch s j c nhc n nh l 1 n-ron n trong trng hp 2. Ly vi phn phng trnh (4.18) vi hm tn hiu ( ), chng ta thu c
( ) ( )

( ) ( ) ( ) ( )

(4.19) v vit li phng trnh

Tip theo chng ta s dng quy tc chui cho o hm tng phn (4.19) di dng tng ng
( ) ( )

( ) ( )

( ) ( )

(4.20)

Tuy nhin, t Hnh 4.4, chng ta ch rng ( ) ( ) ( )

Perceptron nhiu lp Chng 4 = V vy,

( ) ( )

( )

( ))

n-ron k l 1 nt u ra

(4.21)

( ))

(4.22)

Chng ta cng ch t Hnh 4.4 rng vi n-ron k, trng cc b c gy ra (induced local field) l ( ) ( ) ( ) (4.23)

trong m l tng s u vo (bao gm c ng cho (bias)) a vo n-ron k. Ti y ln na, khi lng khp thn kinh (synaptic weight) ( ) bng ng cho (bias) ( ) a vo nron k, v u vo tng ng c c nh gi tr +1. Ly vi phn phng trnh (4.23) vi ( )
( ) ( )

( )

(4.24)

Bng cch s dng phng trnh (4.22) v (4.24) vo phng trnh (4.20), chng ta thu c o hm tng phn mong mun
( ) ( )

( ) ( )

( )) ( )

( )

(4.25)
( ) c cho

trong , dng th 2, chng ta s dng nh ngha ca qu tch gradient phng trnh (4.16), vi ch s k thay cho ch s j.

Cui cng, s dng phng trnh (4.25) vo phng trnh (4.27), chng ta thu c cng thc phn hi ngc i vi qu tch gradient ( ), c m t bng ( ) ( ( )) ( ) ( ) n-ron j c n (4.26)

Hnh 4.5 ch ra s miu t biu lung tn hiu ca phng trnh (4.26), gi s rng lp u ra bao gm n-ron. Tha s bn ngoi ( ( )) lin quan n s tnh ton ca qu tch gradient ( ) trong phng trnh (4.26) ph thuc duy nht vo hm kch hot c lin kt vi n-ron n j. Tha s cn li lin quan n s tnh ton ny gi l, tng i vi k ph thuc vo 2 tp s hng. Tp s hng th nht, ( ), yu cu hiu bit ca cc tn hiu li ( ) i vi tt c cc n-ron nm trong lp ti ngay lp tc ca n-ron n j v c kt ni trc tip ti n-ron j; xem Hnh 4.4. Tp s hng th 2, ( ), bao gm cc khi lng khp thn kinh (synaptic weight) c lin kt vi cc kt ni ny.

Perceptron nhiu lp Chng 4 Chng ta by gi tng hp cc mi lin h m chng ta thu c t thut ton phn hi ngc. u tin, sa li ( ) a vo khi lng khp thn kinh (synaptic weight) kt ni n-ron i ti n-ron j c xc nh bi quy tc delta: ( ( ) Th 2, qu tch gradient ) ( ) ( ) ( )

( )

( )

(4.27)

( ) ph thuc vo vic n-ron j l 1 nt u ra hay l 1 nt n:

1. Nu n-ron j l 1 nt u ra, ( ) bng tch ca o hm ( ( )) v tn hiu li ( ), c 2 u c lin kt vi n-ron j; xem phng trnh (4.16). 2. Nu n-ron j l 1 nt n, ( ) bng tch ca o hm c lin kt ( ( )) v tng khi lng ca cc c tnh ton vi cc n-ron lp n hoc u ra tip theo c kt ni vi n-ron j; xem phng trnh (4.26).

Perceptron nhiu lp Chng 4

Hm kch hot
S tnh ton ca cho mi n-ron ca perceptron nhiu lp cn hiu bit ca o hm ca ( ) c lin kt vi n-ron . o hm ny tn ti, chng ta cn hm ( ) lin tc. Trong iu kin c bn, ly vi phn l yu cu duy nht m 1 hm kch hot phi tha mn. Mt v d ca 1 hm kch hot kh vi phn lin tc khng tuyn tnh thng thng c s dng trong cc perceptron nhiu lp l xch-ma phi tuyn, vi 2 dng c m t di y: 1. Hm logic. Dng ny ca xch-ma phi tuyn, trong dng c bn ca n, c nh ngha bi ( ( ))
( ( ))

(4.32)

trong ( ) l trng cc b c gy ra (induced local field) ca n-ron j v l 1 thng s dng c th iu chnh c. Theo tnh phi tuyn ny, bin ca u ra nm trong on . Ly vi phn phng trnh (4.32) vi ( ), chng ta thu c ( ( )) Vi ( )
( [ ( ( )) ( )) ]

(4.33) ( ( )) t phng

( ( )), chng ta c th loi b s hng ly tha ( ( )) nh l ( )] ( )

trnh (4.33) v kt qu biu din o hm ( ( ))

( )[

(4.34)

Vi 1 n-ron j c t trong lp u ra, tch gradient cho n-ron j nh l ( ) ( ) = [ ( ) ( ( )) ( )] ( )[

( ). Do , chng ta c th biu din qu

( )]

n-ron j l 1 nt u ra

(4.35)

trong ( ) l hm tn hiu ti u ra ca n-ron j, v ( ) l phn hi mong mun ca n. Ni cch khc, vi 1 n-ron n ty j, chng ta c th biu din qu tch gradient nh l ( ) = ( ( )) ( )[ ( ) ( )] ( ) ( ) ( ) n-ron j c n (4.36) ( )

Ch t phng trnh (4.34) rng o hm

( ( )) t gi tr ln nht ca n ti

v gi tr nh nht ca n (bng 0) ti ( ) hoc ( ) . Bi v s lng ca s thay i trong 1 khi lng khp thn kinh (synaptic weight) ca mng li t l vi o hm

Perceptron nhiu lp Chng 4 ( ( )), suy ra rng vi 1 hm kch hot xch-ma, cc khi lng khp thn kinh (synaptic weight) b thay i nhiu nht vi cc n-ron trong mng li m ti cc hm tn hiu trong tm trung ca chng. Theo Rumelhart et al. (1986a), c im ny ca hc phn hi ngc ng gp cho tnh n nh ca n nh l 1 thut ton hc. 2. Hm tip tuyn hypebol. Mt dng khc thng c s dng ca xch-ma phi tuyn l hm tip tuyn hypebol m dng ph bin nht ca n c nh ngha bi ( ( )) ( ( )) (4.37)

trong v l cc hng s dng. Trong thc t, hm tip tuyn hypebol ch l hm logic c thay i t l v sai lch. o hm ca n vi ( ) c cho bi ( ( )) = ( [ ( ( )) ( ( )][ ( ))) ( )] (4.38)

Vi n-ron j c t trong lp u ra, qu tch gradient l ( ) ( ) [ ( ) Vi n-ron j trong 1 lp n, chng ta c ( ) [ ( ( )) ( )][ ( ) ( ) (4.40) ( )] ( ) ( ) n-ron j c n ( ( )) ( )][ ( )][ ( )] (4.39)

Bng cch s dung phng trnh (4.35) v (4.36) cho hm logic v phng trnh (4.39) v (4.40) cho hm tip tuyn hypebol, chng ta c th tnh ton qu tch gradient ( ) khng cn hiu bit r rng v hm kch hot.

T l hc
Thut ton lan truyn ngc cung cp 1 php tnh xp x ng i cong trong khng gian khi lng c tnh ton bi phng php steepest descent. Chng ta ly thng s t l hc cng nh, s thay i ti cc khi lng khp thn kinh (synaptic weight) trong mng li s cng nh t 1 ln lp ti ln k tip, v s cng phng hn ng i cong trong khng gian khi lng. S ci thin ny, tuy nhin, c t c vi s mt mt ca 1 t l hc chm hn. Ni

10

Perceptron nhiu lp Chng 4 cch khc, nu chng ta ly thng s t l hc qu ln tng tc t l hc, do cc s thay i ln trong khi lng khp thn kinh (synaptic weight) gi thit 1 hnh thc nh vy l mng li c th tr ln khng c nh (v d, dao ng). Mt phng php n gin tng t l hc trong khi trnh s nguy him ca tnh khng n nh l thay i quy tc delta ca phng trnh (4.15) bng cch thm vo 1 s hng momen ng lng nh di y ( ) ( ) ( ) ( ) (4.41)

trong thng l 1 s dng gi l hng s momen. N iu khin vng lp phn hi hot ( ), nh c m t trong Hnh 4.6, trong ng xung quanh l ton t n v thi gian tr. Phng trnh (4.41) c gi l quy tc delta tng qut; n bao hm quy tc delta ca phng trnh (4.15) nh 1 trng hp c bit ( ). thy hiu qu ca chui ca cc mu gii thiu trn cc khi lng khp thn kinh (synaptic weight) bi hng s momen , chng ta vit phng trnh (4.41) nh l 1 chui thi gian vi ch s t. Ch s t i t thi im ban u 0 ti thi im hin ti n. Phng trnh (4.41) ( ). Gii c th c xem l 1 phng trnh sai phn bc nht trong khi lng sa sai ( ), chng ta c phng trnh ny vi ( )
( )

( ) ( )

(4.42)

miu t 1 chui thi gian di n + 1. T phng trnh (4.13) v (4.16), chng ta ch rng tch ( ) ( ) bng vi tng ng ( )
( ) ( ) ( )

. Do chng ta c th vit li phng trnh (4.42) di dng

(4.43)

Da vo mi lin h ny, chng ta c th dn n nhng nhn xt chnh xc sau: ( ) miu t tng ca 1 chui thi gian c iu chnh theo hm 1. iu chnh hin ti m. Vi 1 chui thi gian hi t, hng s momen phi c gii hn trong phm vi | | . Khi , thut ton lan truyn ngc hot ng khng momen. Hng s momen cng c th dng hoc m, mc d khng chc xy ra rng 1 s m s c dng trong thc t. 2. Khi o hm ring
( ) ( )

c cng du i s trn nhng ln lp lin tip, tng c iu

( ) ln dn v ln, v do khi lng ( ) c iu chnh theo hm m chnh bi 1 khi lng ln. S bao gm momen trong thut ton lan truyn ngc c xu hng tng s xung dc theo cc hng vng chc.

11

Perceptron nhiu lp Chng 4 3. Khi o hm ring

( ) ( )

tri du trn nhng ln lp lin tip, tng c iu chnh theo

( ) b li v ln, v do khi lng ( ) c iu chnh bi 1 khi hm m lng ln. S bao gm momen trong thut ton lan truyn ngc c 1 hiu qu n nh trong cc hng m dao ng biu hin. S hp nht ca momen trong thut ton lan truyn ngc din t 1 s sa i nh ti s cp nht khi lng; tuy nhin, n c th c mt vi nh hng c ch trong hnh vi hc ca thut ton. S hn momen cng c li ch ngn qu trnh hc chm dt trong 1 ti thiu nng cc b trn b mt li. Xut pht t thut ton lan truyn ngc, gi s rng thng s t l hc l hng s biu din bi . Tuy nhin, trong thc t, n nn c nh ngha bi ji; ngha l, thng s t l hc nn kt ni ph thuc. Su hn, nhiu iu th v c th c thc hin bng cch lm cho thng s t l hc khc i so vi cc phn khc ca mng li. Chng ti cung cp chi tit hn vn ny trong cc mc tip theo. Cng ng ch rng trong ng dng ca thut ton lan truyn ngc, chng ta c th chn tt c cc khi lng khp thn kinh (synaptic weight) trong mng li l c th iu chnh c, hoc chng ta c th buc bt k s no ca khi lng trong mng li tn ti c nh trong sut qu trnh a vo. Trong trng hp th 2, cc tn hiu li c lan truyn ngc qua mng li trong trng thi thng thng; tuy nhin, cc khi lng khp thn kinh (synaptic weight) c nh cn li khng thay i g. iu ny c th c thc hin n gin bng cch lm cho cc thng s t l hc ji ca khi lng khp thn kinh (synaptic weight)

Cc tiu chun dng

Thng thng, thut ton lan truyn ngc khng th c ch ra hi t, v khng c cc tiu chun xc nh r rng cho s dng hot ng ca n. D nhin, c vi tiu chun hp l, mi ci vi tnh cht thit thc ca n, c dng kt thc cc s iu chnh khi lng. to thnh 1 tiu chun nh vy, l hp l ngh trong iu kin ca cc thuc tnh duy nht ca 1 ti thiu cc b hoc ton b ca b mt li. t vec-t khi lng w* biu din 1 ti thiu, l cc b hoc ton b. Mt iu kin cn ca w* l 1 ti thiu l vec-t gradient g(w) (v d o hm tng phn bc nht) ca b mt li vi khi lng vec-t w phi bng 0 ti w = w*. Do , chng ta c th to thnh 1 tiu chun hi t d nhn bit cho hc lan truyn ngc nh di y (Kramer v Sangiovanni-Vicentelli, 1989): Thut ton lan truyn ngc c xem nh c hi t khi quy tc Euclide ca vec-t gradient t ti 1 ngng gradient nh thch hp. Mt hn ch ca tiu chun hi t ny l th nghim thnh cng, thi gian hc c th di. N cng cn s tnh ton ca vec-t gradient g(w).

12

Perceptron nhiu lp Chng 4 Thuc tnh duy nht khc ca 1 ti thiu m chng ta c th s dng l thc t rng hm gi tr Eav(w) khng thay i ti im w = w*. Do chng ta c th xut 1 tiu chun khc ca hi t: Thut ton lan truyn ngc c xem nh c hi t khi t l tuyt i ca s thay i trong trung bnh bnh phng li trn mi thi i (epoch) l nh thch hp. T l ca thay i trong trung bnh bnh phng li c trng c xem nh nh nu n nm trong khong 0,1 n 1% trn mi thi i (epoch). i khi 1 gi tr nh bng 0,01% trn mi thi i (epoch) c s dng. Khng may, tiu chun ny c th dn n 1 s chm dt sm hn thng thng ca qu trnh hc. C 1 tiu chun hi t hu dng, c xc nhn v l thuyt khc: Sau mi ln hc lp li, mng li c kim tra s thc thi tng qut ca n. Qu trnh hc c dng khi s thc thi tng qut t mc cao nht; xem mc 4.13 bit thm chi tit.

Tng kt thut ton lan truyn ngc

Hnh 4.1 m t bn v cu trc ca 1 perceptron nhiu lp. th lung tn hiu tng ng ca hc lan truyn ngc, kt hp c cc s tnh ton tin ln v tht li trong qu trnh hc, c m t Hnh 4.7 trong trng hp L = 2 v . Phn nh ca th lung tn hiu gii thch cho s vt qua v pha trc. Phn thp hn ca th lung tn hiu gii thch cho s vt qua v pha sau, m c nhc n nh l 1 th nhy cho tnh ton cc gradient cc b trong thut ton lan truyn ngc (Narendra v Parthasarathy, 1990). Trc , chng ta cp rng cp nht lin tc ca cc khi lng l phng php a thch hn dnh cho s thc hin trc tuyn ca thut ton lan truyn ngc. i vi ch thi hnh, thut ton xoay quanh mu hun luyn *( ( ) ( ))+ nh di y: 1. Khi to. Gi s rng khng thng tin trc no c sn, la chn cc khi lng khp thn kinh (synaptic weight) v im bt u t 1 s phn b u m trung bnh ca chng bng 0 v s khc bit c chn lm tiu chun lch ca trng cc b thu c (induced local field) ca cc n-ron nm thi k chuyn tip gia cc phn tuyn tnh v tiu chun ca hm kch hot xch-ma (sigmoid). 2. Cc s a vo cc v d hun luyn. Trnh by mng li trong 1 thi i (epoch) ca cc v d hun luyn. Vi mi v d trong mu, c sp t theo mt vi kiu, trnh by chui cc s tnh ton tin ln v tht li c m t theo th t di im 3 v 4. 3. S tnh ton tin ln. Cho 1 v d hun luyn trong thi i (epoch) c biu din bi ( ( ) ( )), vi vec-t u x(n) vo c a vo lp u vo ca cc giao im giao cm v vec-t phn hi mong mun d(n) c a vo lp u ra ca cc giao im tnh ton. Tnh ton cc trng cc b thu c (induced local field) v cc tn hiu hm ca mng li bng cch

13

Perceptron nhiu lp Chng 4 tip tc tin ln qua mng li, lp qua lp. Trng cc b thu c (induced local field) ca n-ron j trong lp l l
() ()

( )

()

( )

( )

(4.44)

trong v
()

( ) l tn hiu (hm) u ra ca n-ron i trong lp trc l 1 ln lp th n,

( )

( ) l khi lng khp thn kinh (synaptic weight) ca n-ron j trong lp l m c cung , chng ta c ( ) , v
()

cp t n-ron i trong lp l - 1. Vi

( )

()

( ) l

ng cho (bias) a vo n-ron j trong lp l. Gi thit s dng 1 hm xch-ma, tn hiu u ra ca n-ron trong lp l
()

( ( )) ), t ( ) trong lp u ra (v d

Nu n-ron trong lp n u tin (v d

( )

( )

trong ( ) l phn t th ca vec-t u vo ( ). Nu n-ron , trong c cp nh l su ca mng li), t

( )

( )

( )

Tnh ton tn hiu li ( ) trong ( ) ( ) (4.45)

( ) l phn t th ca vec-t phn hi mong mun d( ). (v d, cc gradient cc b) ca mng li, c xc

4. S tnh ton tht li. Tnh ton nh bi

( ) ()

( )

( )
( )

( )

( ))
( )

( ))

( )

( )

trong du phy trong ( ) biu din php ly vi phn vi argument. Thay i cc khi lng khp thn kinh (synaptic weight) ca mng li trong lp dn ti quy tc delta c tng qut ha
()

()

( )

()

)+

()

( )

( )

(4.47)

14

Perceptron nhiu lp Chng 4 trong l thng s t l hc v l hng s moment.

5. Ln lp. Cc tnh ton ln lp tin ln v tht li di mc 3 v 4 a vo cc thi i (epoch) ca cc v d hun luyn ti mng li cho ti khi gp cc tiu chun dng c chn. Ch : Th t cc s trnh by ca cc v d hun luyn nn c ngu nhin t thi i (epoch) ny ti thi i (epoch) khc. Momen v thng s t l hc c iu chnh c trng (v thng gim) trong khi s ln lp hun luyn tng. Chng minh nhng iu ny s c trnh by sau.

4.5. Vn XOR

15