1

Gii thiu v Mng

Nron
Trnh by:
Phm Nguyn Khang
2
Ni dung trnh by
S lc v mng n-ron
Perceptron
Mng nron nhiu tng (MLP)
Gii thut lan truyn ngc (back propagation)
Mt s vn ca mng n-ron
Hng pht trin
Tho lun
3
N-ron sinh hc
4
N-ron sinh hc
5
S lc v mng n-ron
Mng n-ron l 1 h cc qu trnh x l thng tin da trn m hnh cc
n-ron thn kinh ca con ngi.
Kt hp 1 s lng ln cc thnh phn n gin (n-ron) cu trc
phc tp nhm gii quyt 1 vn c th no
Ging nh con ngi, mng n-ron hc bng cc v d (mu)
Lnh vc ng dng:
Pht hin bnh
D bo th trng chng khon
Nghin cu khch hng
Kim tra tin cy ca my mc
Qun l ri ro

6
N-ron nhn to (artificial
neuron)
M phng cc n-ron sinh hc
Nhn cc tn hiu t cc n-ron khc (hay t u vo). Nu gi tr (tng
cc tn hiu c nhn h s) nhn c vt qu mt ngng no ,
n-ron ny s kch hot (n s gi tn hiu n cc n-ron khc na)
7
Perceptron
Perceptron l m hnh n gin nht ca mng n-ron (ch c 1 tng).
Perceptron = 1 n-ron nhn to
M hnh ton ca perceptron:
f c gi l hm kch hot (activation action) hay hm truyn c th l:
Tuyn tnh
Ngng (Heaviside step)
Logistic sigmoid
Gauss

=

=
d
i
i i
w v f output
1
x
e
x g

+
=
1
1
) (
8
Hun luyn Perceptron (1)
Xt trng hp,
perceptron x dng hm
kch hot ngng.
Nu cho w
0
= -t, v v
0
=
1, ta c th vit li:

> +
=

=
=
d
i
i i
d
i
i i
t w v
t w v
output
1
1
, 1
, 1

=

> +
=

=
=
=
d
i
i i d
i
i i
d
i
i i
w v sign
w v
w v
output
0
0
0
0 , 1
0 , 1
9
Hun luyn Perceptron (2)
Khi to:
Khi to w (0 hoc ngu nhin)
Chn tc hc
Hc:
Vi mi mu (x, t) trong tp hc
Tnh y = f(x,w)
Nu (y != t) thay i vect trng s w
w(mi) = w(c) + (t-y)x
Lp li qu trnh hc cho n khi iu kin
dng tha mn
10
iu kin dng
Qu trnh hc, thc cht l qu trnh i tm cc
trng s w sao cho li xy ra l nh nht
phng php gii bi ton ti u
iu kin dng trong qu trnh hc c th l 1
trong cc tiu ch, hay kt hp cc tiu ch:
Li nh n mc chp nhn c
Sau mt s bc lp ln

11
Kh nng ca Perceptron
Phng trnh v.w = 0 chnh l mt siu phng trong
khng gian d-chiu.
Perceptron c kh nng phn lp tuyn tnh. C th
dng gii bi ton hi quy tuyn tnh
Hn ch ca Perceptron:
Khng th phn lp phi tuyn
V d: bi ton XOR
Gii php:
S dng mng n-ron nhiu tng MLP
12
Mng n-ron nhiu tng (MLP)
13
Mng n-ron nhiu tng (MLP)
14
Mng n-ron nhiu tng (MLP)
input, output v cc tng n (hiden
layers).
S lng cc n-ron trong tng input =
s chiu
S lng cc n-ron trong tng output
thng = s cc lp
Thng thng cc mng n-ron dng 1
tng n. S lng cc n-ron trong tng
n thng c chn = (s n-ron
tng input + s n-ron trong tng
output)
Ch : cc n-ron trong tng input thc
s khng phi l cc n-ron. N ch l
ch a d liu ca cc mu vo
15
Hun luyn MLP
L qu trnh thay i gi tr ca w mng biu din c tp d liu
hc.
S khc nhau gia gi tr tht ca mu v kt qu d on ca mng
gi l li (hc c gim st). Hm li thng dng l sum squared error
Hm li s l mt hm (n+1) bin. Nu v trong khng gian n+1 chiu
ta s c mt mt li (error surface)
Qu trnh hun luyn s tm cc w
i
* li nh nht.
Trong trng hp tuyn tnh, vi hm li l sum squared error th mt
li c dng parapola c th tm c w* li = min tng i d
dng
Trong trng hp tng qut, hm li phc tp (v d: khng phi hm
li) vn cc tiu cc b (local minimum) gii php ???
16
Mt s gii thut hun luyn MLP
Back propagation (kh ni ting, d hiu)
Conjugate gradient descent & Levenber-
Marquardt (Bishop 1995, Shepherd 1997)
Quick propagation (Fahlman, 1988)
Delta-Bar-Delta(Jacob 1988)
17
Gii thut lan truyn ngc
Xt mt mng n-rn 3 tng: input, hiden, output
Hm kch hot ca cc n-ron: logistic sigmoid
Gii thut lan truyn ngc gm 2 giai on:
Lan truyn tin (tnh output ca cc n-ron)
Lan truyn ngc (thay i trng s ca cc cung, dng thng tin
gradient ca hm li)
18
Gradient ca hm li (1)
N-ron tng output
Hm li:
o hm ring ca hm
li theo cc trng s b:
x
k
i
i i
e
x g
b y b v
v g z

=
+
=
+ =
=

1
1
) (
) (
1
0
b
v
v
z
z
E
b
E

b
1
b
2
b
k
b
0
y
1
y
2
y
k
1
( )
2
2
1
) ( t z z E =
Gi tr tht ca
mu hun luyn
19
Gradient ca hm li (2)
Ta c: t:

>
=
=

0 ,
0 , 1
) 1 (
) (
i y
i
b
v
z z
v
g
v
z
t z
z
E
i i
) 1 ( ) ( z z t z
v
z
z
E
p =

=
Ta c:

>
=
=

0
0
i py
i p
b
E
i
i
20
Gradient ca hm li (3)
N-ron tng n
o hm ring ca hm
li theo cc trng s a:

=
+ =
=
d
i
i i
a x a u
u g y
1
0
) (
a
u
u
y
y
E
a
E

a
1
a
2
a
d
a
0
x
1
x
2
x
d
1
21
Gradient ca hm li (2)
t: Ta c:

>
=
=

= =

=
0 ,
0 , 1
) 1 (
) 1 ( ) (
1
j x
j
a
u
y y
u
y
b
y
v
p z z t z
v
z
z
E
y
v
v
z
z
E
y
E
j
j
i
i
i i i i i
i
i
i
K
i
i
i
i
i
) 1 (
1
y y b p
u
y
y
E
q
K
i
i i

=

=
Ta c:

>
=
=

0
0
i qx
i q
a
E
i i
22
iu chnh trng s
Sau khi tnh c o hm ring ca hm li theo
tng trng s. Trng s s c iu chnh bng
cch tr bt 1 lng bng tch ca o hm ring
v tc hc:
i
i i
w
E
w w

=
23
Mt s vn vi mng n-ron
Vn kin trc mng: n-ron no nn ni vi n-ron
no ?
Trng s ca cc cung ni v ngng thay i thng
xuyn. c nhiu nghin cu v vn ny v cng
c mt s kt qu:
Nu mng gy ra li, th c th xc nh chnh xc n-ron no
gy ra li iu chnh n-ron ny
Vi cch tip cn ny, mng phi bit rng n gy ra li.
Trong thc t, li ch c bit sau mt thi gian di.
Chc nng ca mt n-ron khng qu n gin nh m
hnh.
Mng no-ron hot ng nh 1 hp en
24
Mt s hng dn khi s dng
mng n-ron
Xy dng mng khi to (dng 1 tng n c s n-ron =
tng s n-ron ca tng input & output)
Hun luyn mng dng cc gii thut hun luyn. Nn
thc hin trn nhiu mng khc nhau trnh trng hp
cc tiu cc b
Nu my khng thuc bi (under-learning) thm 1
vi n-ron cho tng n
Ngc li nu my hc vt (over-learning) bt 1
vi n-ron ra khi tng n
Khi tm c mt kin trc mng tng i tt .
Ly mu li tp d liu v hun luyn li tm ra cc
mng mi
25
Hng pht trin
Tch hp logic m vo mng n-ron
Mng n-ron xung (pulse neural networks): cc
n-ron giao tip vi nhau thng qua cc xung
S dng phn cng cho mng