Tr Tu Nhn To

Nguyn Nht Quang

quangnn-fit@mail.hut.edu.vn
Vin Cng ngh Thng tin v Truyn thng
Trng i hc Bch Khoa H Ni
Nm hc 2009-2010

Ni dung mn hc:

Gii thiu v Tr tu nhn to

Tc t

nh ngha

Mi trng cng vic

Cc kiu mi trng
g

Cc kiu tc t

Gii quyt vn : Tm kim, Tha mn rng buc

Logic v suy din

Biu din tri thc

Suy din
vi tri thc khng chc
chn

Hc my

Lp k hoch
Tr tu nhn to

Tc t - nh

ngha
g

Tc t l bt c ci g (con ngi, ngi my, software

robots,, cc b
n nhit,)
, ) c kh nng
g cm nhn
((nhn
bit))
mi trng xung quanh n thng qua cc b phn cm bin
(sensors) v hnh ng ph hp theo mi trng thng
qua cc b phn hot ng (actuators)

Tc t con ngi

Cc b phn cm bin: mt, tai, v mt s b phn c th khc

Cc b phn hot ng: tay, chn, ming, v mt s b phn c
th khc

Tc t ngi my

Cc b phn cm bin: cc my quay (cameras), cc b truy tm

tn hiu hng ngoi
Cc b phn hot ng:
ng cc loi ng c (motors)
Tr tu nhn to

Tc t v Mi trng
g

Hm tc t: l hm nh x t lch s nhn thc ti cc

hnh ng:

f: P* A

Chng trnh tc t: hot ng (chy) da trn kin trc

thc t ca hm f

Tc t = Kin
trc + Chng trnh
Tr tu nhn to

V d: Th gii ca my ht bi

Cc nhn thc

V tr v mc sch s
V d: [A, Bn], [B, Sch]

Cc hnh ng

Di chuyn (my ht bi) sang tri, sang phi, ht bi, hoc khng
lm g c
Tr tu nhn to

Tc t my
y ht bi

Bng hnh ng ca tc t my ht bi
Chui cc nhn
thc

Hnh ng
g

[A, Sch]

Di chuyn sang phi

[A, Bn]

Ht bi

[[B,, Sch]
]

Di chuyn
y sang
g tri

[B, Bn]

Ht bi

[A, Sch], [A, Sch]

Di chuyn sang phi

[A, Sch], [A, Bn]

Ht bi

...

function Reflex-Vacuum-Agent(
g
( [[location,status])
,
]) returns an action
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left
Tr tu nhn to

Tc t hp
p l
(1)
( )

Tc t cn phn u lm ng vic cn lm, da

trn nhng g n nhn thc (nhn bit) c v da
trn cc hnh ng m n c th thc hin

Mt
hnh ng
g ng
g (hp
( p l)
) l hnh ng
gg
gip
p cho tc
t t c thnh cng cao nht i vi mc tiu t ra

nh gi hiu qu hot ng: l tiu chun nh gi

mc
th
thnh
h cng
ttrong h
hott ng
ca
mt
t tc
t t

V d: Tiu ch nh gi hiu qu hot ng ca mt tc t my

ht bi c th l: mc lm sch, thi gian ht bi, mc in
nng tiu tn,
mc ting

n gy ra,

Tr tu nhn to

Tc t hp l (2)

Tc t hp
p l

Vi mi chui nhn thc c c,

Mt tc t hp l cn phi la chn mt hnh ng
gip cc i ha tiu ch nh gi hiu qu hot ng
ca tc t ,
Da trn cc thng tin c cung cp bi chui nhn
thc v cc tri thc c s hu bi tc t

Tr tu nhn to

Tc t hp l (3)

S hp l S thng sut mi th

S thng sut mi th = Bit tt c mi th

th, vi tri thc v hn
V cc nhn thc c th khng cung cp tt c cc thng tin lin
quan

Cc tc t c th
thc hin cc hnh ng nhm thay i

cc nhn thc trong tng lai, vi mc ch thu c

cc thng
g tin hu ch ((v d:
thu thp
p thng
g tin, khm p
ph
tri thc)

Tc t t tr (autonomous agent) l mt tc t m cc
h h ng
hnh
ca
n
c

quyt
t nh
h bi chnh
h h ki
kinh
h
nghim ca tc t (cng vi kh nng hc v thch
nghi)
Tr tu nhn to

Mi trng cng vic PEAS (1)

PEAS

Performance measure: Tiu ch nh gi hiu qu hot ng

Environment: Mi trng xung quanh

Actuators: Cc b phn hnh ng

Sensors: Cc b phn cm bin

thit k mt
t tc
t t thng
th minh
i h (h
(hp l),
l) ttrc
ti
tin cn

phi xc nh (thit lp) cc gi tr ca cc thnh phn

ca PEAS

Tr tu nhn to

10

Mi trng cng vic PEAS (2)

V d: Thit k mt tc t li xe taxi t ng

nh

h gi
i hi
hiu qu
h
hott ng
(P):
(P) an ton,
t nhanh,
h h
ng lut giao thng, mc hi lng ca khch
hng, ti u li nhun,
Mi trng xung quanh (E): cc con ng (ph),
cc phng tin khc cng tham gia giao thng,
g ngi
g
i b, cc khch hng,
g
nhng
Cc b phn hnh ng (A): bnh li, chn ga,
phanh, n tn hiu, ci xe,
Cc b phn cm bin (S): my quay (cameras),
ng h tc , GPS, ng h o khong cch qung
ng, cc b cm bin ng c,
Tr tu nhn to

11

Mi trng cng vic PEAS (3)

V d: Thit k mt tc t chun on y t
nh gi hiu qu hot ng (P): mc sc khe
ca bnh nhn, cc tiu ha cc chi ph, cc vic kin
co,
Mi trng xung quanh (E): bnh nhn, bnh vin,
nhn vin y t,
Cc b phn hnh ng (A): hin th trn mn hnh
cc cu hi, cc xt nghim, cc chun on, cc
iu tr, cc ch dn,
Cc b phn cm bin (S): bn phm nhp vo
cc thng tin v triu chng, cc tr li ca bnh nhn
i vi cc cu hi
hi,
Tr tu nhn to

12

Mi trng cng vic PEAS (4)

V d: Thit k mt tc t nht vt

nh gi hiu qu hot ng (P): t l (bao nhiu

phn trm) cc vt c t vo ng cc thng
Mi trng xung quanh (E): dy chuyn chuyn
ng trn c cc vt, cc thng ng
Cc b phn hnh ng (A): cnh tay v bn tay
c kt ni
Cc b phn cm bin (S): my quay (camera), cc
b cm bin cc gc (cc hng)
Tr tu nhn to

13

Mi trng cng vic PEAS (5)

V d: Thit k mt tc t dy ting Anh tng tc

nh gi hiu qu hot ng (P): cc i ha im

thi ting Anh ca hc vin
Mi trng xung quanh (E): mt nhm hc vin
Cc b phn hnh ng (A): hin th mn hnh cc
bi tp, cc gi , sa (cha) bi tp
Cc b phn cm bin (S): bn phm

Tr tu nhn to

14

Mi trng cng vic PEAS (6)

V d: Thit k mt tc t lc th rc (spam emails

filt i )
filtering)

nh gi hiu qu hot ng (P): kh nng lc th

rc
((amountt off error: false
f l positives,
iti
f l negatives)
false
ti
)
Mi trng xung quanh (E): email server and clients
Cc b phn hnh ng (A): nh du th rc, gi
thng bo
Cc b phn cm bin (S): nhn v phn tch ni
dung cc emails
Tr tu nhn to

15

Cc kiu mi trng
g ((1))

C th quan st c hon ton (hay c th quan st c

mt phn)?

Xc nh

((hay
y ngu
g nhin)?
)

Cc b cm bin
ca mt tc t cho php n truy cp ti trng thi y

ca mi trng ti mi thi im

Trng thi tip theo ca mi trng c xc nh hon ton da trn

trng thi hin ti v hnh ng ca tc t (ti trng thi hin ti ny)
Nu mt mi trng
g l xc nh, ngoi
g
tr i vi cc hnh ng
g ca cc
tc t khc, th gi l mi trng chin lc

Phn on (hay lin tip)?

Kinh nghim ca tc t c chia thnh cc giai on (chng/hi)

Mi giai on bao gm vic nhn thc ca tc t v hnh ng m n
thc hin
mi giai on,
on vic la chn hnh ng thc hin ch ph thuc vo
giai on (khng ph thuc vo cc giai on khc)
Tr tu nhn to

16

Cc kiu mi trng (2)

Tnh (hay ng)?

Mi trng khng thay i trong khi tc t cn nhc (xem nn

a ra hnh ng no)
Mi trng bn ng (semi-dynamic) l mi trng m khi thi
gian tri qua th n (mi trng) khng thay i,
i nhng hiu qu
hot ng ca tc t th thay i

Ri rc (hay
(h lin
li tc)?
)?

V d: cc chng trnh tr chi c tnh gi

Mt tp hu hn cc nhn thc v cc hnh ng, c nh

ngha phn bit r rng

Tc t n l (hay a tc t)?

Mt tc t hot ng c lp (khng ph thuc / lin h vi cc

tc t khc) trong mt mi trng
Tr tu nhn to

17

Cc kiu mi trng V d
Quan st y ?
Xc nh?
Phn on?
Tnh?
Ri rc?
T t n?
Tc
?

Chi c
tnh g
gi
c
chin lc
khng
bn ng
c
kh
khng

Chi c khng
tnh g
gi
c
chin lc
khng
c
c
kh
khng

Li xe taxi
khng
khng
khng
khng
khng
kh
khng

Kiu ca mi trng c nh hng quyt nh i vi vic thit k

tc t

Mi trng trong thc t thng c cc c im: ch c th quan

st c mt phn, ngu nhiu, lin tip, thay i (ng), lin tc, a
tc t
Tr tu nhn to

18

Cc kiu tc t

4 kiu tc t c bn

Tc t phn x n gin (simple reflex agents)

Tc t phn
T
h x da
d
trn
t m
h
hnh
h ((model-based
d lb
d reflex
fl
agents)

Tc t da trn mc tiu (goal-based agents)

Tc t da trn li ch ((utility-based
y
agents)
g
)

Tr tu nhn to

19

Tc t p
phn x
n gin
g
(1)
( )

Tr tu nhn to

20

Tc t phn x n gin (2)

Tc t phn x n gin:
Hnh
H h ng
theo
th mt
t quy tc
t (lut)
(l t) c
iu
i kin
ki ph
h hp
h
vi trng thi hin thi (ca mi trng)
function SIMPLE-REFLEX-AGENT(percept)
static: rules (tp cc lut c dng: iu kin-hnh ng)
state INTERPRET-INPUT(percept)
rule RULE-MATCH(state, rules)
action RULE-ACTION[rule]
return action

Tr tu nhn to

21

Tc t phn x da trn m hnh (1)

Tr tu nhn to

22

Tc t phn x da trn m hnh (2)

Tc t phn x da trn m hnh:

S dng mt m hnh ni b gim st trng thi hin ti ca

mi trng
La chn hnh ng: ging nh i vi tc t phn x n gin

function REFLEX-AGENT-WITH-STATE(percept)
static: state (m t trng thi hin ti ca mi trng)
rules (tp cc lut c dng: iu kin-hnh
kin hnh ng)
action (hnh ng gn nht)
state UPDATE-STATE(state,
UPDATE-STATE(state action,
action percept)
rule RULE-MATCH(state, rules)
action RULE-ACTION[rule]
return action
Tr tu nhn to

23

Tc t da trn mc tiu (1)

Bit v trng thi hin ti ca mi trng: cha

Cn bit thm thng tin v mc tiu

Trng thi hin ti ca mi trng: mt ng t, xe taxi c th r

tri, r phi, hoc i thng
Thng tin v mc tiu: xe taxi cn i ti ch n ca hnh khch

Tc t da trn mc tiu

Theo di trng thi hin ti ca mi trng

Lu
u g
gi
mt
t tp cc mc
c tiu
t u (cn
(c t c)

Chn hnh ng cho php (rt cuc) s t n cc mc tiu

Tr tu nhn to

24

Tc t da
trn mc
tiu (2)
( )

Tr tu nhn to

25

Tc t da trn li ch (1)

Trong nhiu mi trng, thng tin v cc mc tiu khng

nh gi hiu qu ca cc hnh ng

C rt nhiu chui cc hnh ng cho php taxi i n ch (tc

l t n mc tiu)
Nhng: chui hnh ng no nhanh hn, an ton hn, ng tin
cy hn, chi ph thp hn?

Cn s nh gi li ch i vi tc t

Hm li ch (utility function)

nh x t chui cc trng thi ca mi trng ti mt gi tr s

thc (th hin mc li ch i vi tc t)

Tr tu nhn to

26

Tc t da
trn li
ch ((2))

Tr tu nhn to

27

Tc t c kh nng hc (1)

Kh nng hc cho php tc t ci thin hiu qu hot

ng ca n

4 thnh phn to nn mt tc t c kh nng hc

Thnh phn hnh ng: m nhim vic la chn cc hnh ng

Thnh phn nh gi (bnh lun): nh gi hiu qu hot ng

Thnh phn
hc: gip ci thin hiu qu hot ng - da trn cc
nh gi, thay i (ci thin) thnh phn hnh ng
Thnh
p
phn
s
sn sinh
s
kinh nghim:
g c nhim
v
xut
ut cc hnh

ng gip sn sinh ra (dn n) cc kinh nghim mi

Tr tu nhn to

28

Tc t c kh nng
g hc
((2))

Tr tu nhn to

29

C s tri thc ca tc t

Mt c s tri thc (a knowledge base) l mt tp cc mnh

(pht biu) c biu din trong mt ngn ng hnh thc,
cung cp
tri thc (hiu
bit)
cho mt tc t
Tc t khai thc c s tri thc (m n s hu) trong qu trnh
a ra cc hnh ng
Cc tc t c th c xem xt mc

Tri thc: Tc t bit nhng g? Cc mc tiu ca tc t l g?

Ci t h thng: Cc cu trc d liu trong c s tri thc? Cc gii thut
s dng cc tri thc ny?

Tc t cn c kh nng

Thu th
Th
thp (cp
( nht)
ht) cc
ttrii th
thc mi
i
Cp nht vic biu din (bn trong tc t) i vi mi trng xung quanh
Suy din ra cc thuc tnh n ca mi trng xung quanh
S lun
Suy
l a
ra cc
h
hnh
h ng
hp
h l
Tr tu nhn to

30

a tc t (1)

Mi trng hot ng: Cng tc (hp tc) hay l Cnh

tranh (i khng)?

Trong nhiu bi ton thc t, mi trng hot ng lun

thay i (bin ng) tc t cn cp nht

Cn mt m hnh biu din k hoch ca cc tc t khc

Cc tc t cng tc

Cng chia s cc mc tiu hoc cc k hoch

V d: Lp k hoch (cho hot ng nhm) trong tr chi tennis
nh i
Cc c ch cng tc: Phn tch v phn phi cc nhim v cho
mi tc t
Tr tu nhn to

31

a tc t (2)

Cc tc t cnh tranh

V d: chi c
Mi tc t phi nhn bit c s tn ti (v hot ng) ca cc
tc t khc
Mi t
tc t t
tnh
h ton
t (d
(d
on)
)
c cc
k h
hoch
h ca
((mt
t s)
)
cc tc t khc
Mi tc t tnh ton (d on) c nh hng ca cc k hoch
ca
cc
t
tc t kh
khc i vi
i k h
hoch
h ca
b
bn thn
th n

Mi tc t quyt nh hnh ng ti u i vi d on nh
hng ny

Tr tu nhn to

32

Tc t - Tng
g kt

Tc t tng tc vi mi trng thng qua cc b phn cm

bin v cc b
p
phn
hnh ng
g

Hm tc t m t cc hnh ng m tc t thc hin trong

cc tnh hung

Mt tc t hp l s cc i ha hiu qu hot ng ca n

Cc chng trnh tc t ci t (thc hin) cc hm tc t

C m
Cc
t PEAS xc
nh
h mi
i ttrng
cng
vic
i

Cc mi trng c phn loi da theo cc tiu ch: C th

quan st c?
q
Xc nh?

Phn on?
tnh? Ri rc?
Tc t
n l?

Cc kin trc tc t c bn: Phn x, Phn x theo trng

thi Da trn mc tiu,
thi,
tiu Da trn li ch
Tr tu nhn to

33