Tr Tu Nhn To

Nguyn Nht Quang

quangnn-fit@mail.hut.edu.vn Vin Cng ngh Thng tin v Truyn thng Trng i hc Bch Khoa H Ni
Nm hc 2010-2011

Ni dung mn hc:
Gii thiu v Tr tu nhn to Tc t
nh ngha Mi trng cng vic Cc kiu mi trng g Cc kiu tc t

Gii quyt vn : Tm kim, Tha mn rng buc Logic v suy din Biu din tri thc Biu din tri thc khng chc chn Hc my Lp k hoch
Tr tu nhn to 2

Tc t - nh ngha g
Tc t l bt c ci g (con ngi, ngi my, software robots, cc b n nhit,) c kh nng cm nhn (nhn bit) , , ) g ( ) mi trng xung quanh n thng qua cc b phn cm bin (sensors) v hnh ng ph hp theo mi trng thng qua cc b phn hot ng (actuators) Tc t con ngi
Cc b phn cm bin: mt, tai, v mt s b phn c th khc Cc b phn hot ng: tay, chn, ming, v mt s b phn c th khc

Tc t ngi my
Cc b phn cm bin: cc my quay (cameras), cc b truy tm tn hiu hng ngoi Cc b phn hot ng cc loi ng c (motors) ng:
Tr tu nhn to 3

Tc t v Mi trng g

Hm tc t: l hm nh x t lch s nhn thc ti cc hnh ng:

f: P* A
Chng trnh tc t: hot ng (chy) da trn kin trc thc t ca hm f Tc t = Kin trc + Chng trnh
Tr tu nhn to 4

V d: Th gii ca my ht bi

Cc nhn thc
V tr v mc sch s V d: [A, Bn], [B, Sch]

Cc hnh ng
Di chuyn (my ht bi) sang tri, sang phi, ht bi, hoc khng lm g c
Tr tu nhn to 5

Tc t my ht bi y
Bng hnh ng ca tc t my ht bi
Chui cc nhn thc [A, Sch] [A, Bn] [ , ] [B, Sch] [B, Bn] [A, Sch], [A, Sch] [A, Sch], [A, Bn] ... Ht bi Di chuyn sang tri y g Ht bi Di chuyn sang phi Ht bi Hnh ng g Di chuyn sang phi

function Reflex-Vacuum-Agent( [ g ( [location,status]) returns an action , ]) if status = Dirty then return Suck else if location = A then return Right else if location = B then return Left
Tr tu nhn to 6

Tc t hp l (1) p ( )
Tc t cn phn u lm ng vic cn lm, da trn nhng g n nhn thc (nhn bit) c v da trn cc hnh ng m n c th thc hin Mt hnh ng ng (hp l) l hnh ng g p cho tc g g ( p ) g gip t t c thnh cng cao nht i vi mc tiu t ra nh gi hiu qu hot ng: l tiu chun nh gi mc th h cng t thnh trong h t ng ca mt tc t hot t t
V d: Tiu ch nh gi hiu qu hot ng ca mt tc t my ht bi c th l: mc lm sch, thi gian ht bi, mc in nng tiu tn, mc ting gy ra, n

Tr tu nhn to

Tc t hp l (2)
Tc t hp l p Vi mi chui nhn thc c c, Mt tc t hp l cn phi la chn mt hnh ng gip cc i ha tiu ch nh gi hiu qu hot ng ca tc t , Da trn cc thng tin c cung cp bi chui nhn thc v cc tri thc c s hu bi tc t

Tr tu nhn to

Tc t hp l (3)
S hp l S thng sut mi th
S thng sut mi th = Bit tt c mi th vi tri thc v hn th, V cc nhn thc c th khng cung cp tt c cc thng tin lin quan

Cc tc t c th thc hin cc hnh ng nhm thay i cc nhn thc trong tng lai, vi mc ch thu c cc thng tin hu ch (v d: thu thp thng tin, khm p g ( p g ph tri thc) Tc t t tr (autonomous agent) l mt tc t m cc hnh ng h h ca n c quyt h bi chnh ki h t nh h h kinh nghim ca tc t (cng vi kh nng hc v thch nghi)
Tr tu nhn to 9

Mi trng cng vic PEAS (1)

PEAS
Performance measure: Tiu ch nh gi hiu qu hot ng Environment: Mi trng xung quanh Actuators: Cc b phn hnh ng Sensors: Cc b phn cm bin

thit k mt tc t th minh (h l) t ti cn t t thng i h (hp l), trc tin phi xc nh (thit lp) cc gi tr ca cc thnh phn ca PEAS

Tr tu nhn to

10

Mi trng cng vic PEAS (2)

V d: Thit k mt tc t li xe taxi t ng nh i hiu hot ng (P): h gi hi qu h t (P) an t nhanh, ton, h h ng lut giao thng, mc hi lng ca khch hng, ti u li nhun, Mi trng xung quanh (E): cc con ng (ph), cc phng tin khc cng tham gia giao thng, g g g nhng ngi i b, cc khch hng, Cc b phn hnh ng (A): bnh li, chn ga, phanh, n tn hiu, ci xe, Cc b phn cm bin (S): my quay (cameras), ng h tc , GPS, ng h o khong cch qung ng, cc b cm bin ng c,
Tr tu nhn to 11

Mi trng cng vic PEAS (3)

V d: Thit k mt tc t chun on y t nh gi hiu qu hot ng (P): mc sc khe ca bnh nhn, cc tiu ha cc chi ph, cc vic kin co, Mi trng xung quanh (E): bnh nhn, bnh vin, nhn vin y t, Cc b phn hnh ng (A): hin th trn mn hnh cc cu hi, cc xt nghim, cc chun on, cc iu tr, cc ch dn, Cc b phn cm bin (S): bn phm nhp vo cc thng tin v triu chng, cc tr li ca bnh nhn i vi cc cu hi hi,
Tr tu nhn to 12

Mi trng cng vic PEAS (4)

V d: Thit k mt tc t nht vt nh gi hiu qu hot ng (P): t l (bao nhiu phn trm) cc vt c t vo ng cc thng Mi trng xung quanh (E): dy chuyn chuyn ng trn c cc vt, cc thng ng Cc b phn hnh ng (A): cnh tay v bn tay c kt ni Cc b phn cm bin (S): my quay (camera), cc b cm bin cc gc (cc hng)
Tr tu nhn to

13

Mi trng cng vic PEAS (5)

V d: Thit k mt tc t dy ting Anh tng tc nh gi hiu qu hot ng (P): cc i ha im thi ting Anh ca hc vin Mi trng xung quanh (E): mt nhm hc vin Cc b phn hnh ng (A): hin th mn hnh cc bi tp, cc gi , sa (cha) bi tp Cc b phn cm bin (S): bn phm

Tr tu nhn to

14

Mi trng cng vic PEAS (6)

V d: Thit k mt tc t lc th rc (spam emails filtering) filt i ) nh gi hiu qu hot ng (P): kh nng lc th rc ( (amount of error: false positives, f l negatives) t f f l iti false ti ) Mi trng xung quanh (E): email server and clients Cc b phn hnh ng (A): nh du th rc, gi thng bo Cc b phn cm bin (S): nhn v phn tch ni dung cc emails
Tr tu nhn to

15

Cc kiu mi trng ( ) g (1)

C th quan st c hon ton (hay c th quan st c mt phn)?
Cc b cm bin ca mt tc t cho php n truy cp ti trng thi y ca mi trng ti mi thi im

Xc nh (hay ngu nhin)? ( y g )

Trng thi tip theo ca mi trng c xc nh hon ton da trn trng thi hin ti v hnh ng ca tc t (ti trng thi hin ti ny) Nu mt mi trng l xc nh, ngoi tr i vi cc hnh ng ca cc g g g tc t khc, th gi l mi trng chin lc

Phn on (hay lin tip)?

Kinh nghim ca tc t c chia thnh cc giai on (chng/hi) Mi giai on bao gm vic nhn thc ca tc t v hnh ng m n thc hin mi giai on vic la chn hnh ng thc hin ch ph thuc vo on, giai on (khng ph thuc vo cc giai on khc)
Tr tu nhn to 16

Cc kiu mi trng (2)

Tnh (hay ng)?
Mi trng khng thay i trong khi tc t cn nhc (xem nn a ra hnh ng no) Mi trng bn ng (semi-dynamic) l mi trng m khi thi gian tri qua th n (mi trng) khng thay i nhng hiu qu i, hot ng ca tc t th thay i
V d: cc chng trnh tr chi c tnh gi

Ri rc (h li tc)? (hay lin )?

Mt tp hu hn cc nhn thc v cc hnh ng, c nh ngha phn bit r rng

Tc t n l (hay a tc t)?
Mt tc t hot ng c lp (khng ph thuc / lin h vi cc tc t khc) trong mt mi trng
Tr tu nhn to 17

Cc kiu mi trng V d
Quan st y ? Xc nh? Phn on? Tnh? Ri rc? Tc n? T t ? Chi c tnh gi g c chin lc khng bn ng c khng kh Chi c khng tnh gi g c chin lc khng c c khng kh Li xe taxi khng khng khng khng khng khng kh

Kiu ca mi trng c nh hng quyt nh i vi vic thit k tc t Mi trng trong thc t thng c cc c im: ch c th quan st c mt phn, ngu nhiu, lin tip, thay i (ng), lin tc, a tc t
Tr tu nhn to 18

Cc kiu tc t
4 kiu tc t c bn Tc t phn x n gin (simple reflex agents) Tc T t phn x da trn m h h ( d l b h d t hnh (model-based reflex d fl agents) Tc t da trn mc tiu (goal-based agents) Tc t da trn li ch (utility-based agents) ( y g )

Tr tu nhn to

19

Tc t phn x n gin (1)

Tc t phn x n gin: H h th mt quy t (l t) c i ki ph h Hnh ng theo t tc (lut) iu kin h hp vi trng thi hin thi (ca mi trng)
function SIMPLE-REFLEX-AGENT(percept) static: rules (tp cc lut c dng: iu kin-hnh ng) state INTERPRET-INPUT(percept) rule RULE-MATCH(state, rules) action RULE-ACTION[rule] return action

Tr tu nhn to

20

Tc t phn x n gin (2) p g ( )

Tr tu nhn to

21

Tc t phn x da trn m hnh (1)

Tc t phn x da trn m hnh:
S dng mt m hnh ni b gim st trng thi hin ti ca mi trng La chn hnh ng: ging nh i vi tc t phn x n gin function REFLEX-AGENT-WITH-STATE(percept) static: state (m t trng thi hin ti ca mi trng) rules (tp cc lut c dng: iu kin hnh ng) kin-hnh action (hnh ng gn nht) state UPDATE-STATE(state action percept) UPDATE-STATE(state, action, rule RULE-MATCH(state, rules) action RULE-ACTION[rule] return action
Tr tu nhn to 22

Tc t phn x da trn m hnh (2)

Tr tu nhn to

23

Tc t da trn mc tiu (1)

Bit v trng thi hin ti ca mi trng: cha Cn bit thm thng tin v mc tiu
Trng thi hin ti ca mi trng: mt ng t, xe taxi c th r tri, r phi, hoc i thng Thng tin v mc tiu: xe taxi cn i ti ch n ca hnh khch

Tc t da trn mc tiu
Theo di trng thi hin ti ca mi trng Lu g mt tp cc mc tiu (cn t c) u gi t c t u (c Chn hnh ng cho php (rt cuc) s t n cc mc tiu

Tr tu nhn to

24

Tc t da trn mc tiu (2) ( )

Tr tu nhn to

25

Tc t da trn li ch (1)
Trong nhiu mi trng, thng tin v cc mc tiu khng nh gi hiu qu ca cc hnh ng
C rt nhiu chui cc hnh ng cho php taxi i n ch (tc l t n mc tiu) Nhng: chui hnh ng no nhanh hn, an ton hn, ng tin cy hn, chi ph thp hn?

Cn s nh gi li ch i vi tc t Hm li ch (utility function)
nh x t chui cc trng thi ca mi trng ti mt gi tr s thc (th hin mc li ch i vi tc t)

Tr tu nhn to

26

Tc t da trn li ch ( ) (2)

Tr tu nhn to

27

Tc t c kh nng hc (1)
Kh nng hc cho php tc t ci thin hiu qu hot ng ca n 4 thnh phn to nn mt tc t c kh nng hc
Thnh phn hnh ng: m nhim vic la chn cc hnh ng Thnh phn nh gi (bnh lun): nh gi hiu qu hot ng Thnh phn hc: gip ci thin hiu qu hot ng - da trn cc nh gi, thay i (ci thin) thnh phn hnh ng Thnh p sn sinh kinh nghim: c nhim v xut cc hnh phn s s g ut ng gip sn sinh ra (dn n) cc kinh nghim mi

Tr tu nhn to

28

Tc t c kh nng hc (2) g ( )

Tr tu nhn to

29

C s tri thc ca tc t
Mt c s tri thc (a knowledge base) l mt tp cc mnh (pht biu) c biu din trong mt ngn ng hnh thc, cung cp tri thc (hiu bit) cho mt tc t Tc t khai thc c s tri thc (m n s hu) trong qu trnh a ra cc hnh ng Cc tc t c th c xem xt mc
Tri thc: Tc t bit nhng g? Cc mc tiu ca tc t l g? Ci t h thng: Cc cu trc d liu trong c s tri thc? Cc gii thut s dng cc tri thc ny?

Tc t cn c kh nng
Thu thp (cp ht) tri thc i Th th ( nht) cc t i th mi Cp nht vic biu din (bn trong tc t) i vi mi trng xung quanh Suy din ra cc thuc tnh n ca mi trng xung quanh Suy l S lun a ra cc h h ng hp l hnh h
Tr tu nhn to 30

a tc t (1)
Mi trng hot ng: Cng tc (hp tc) hay l Cnh tranh (i khng)? Trong nhiu bi ton thc t, mi trng hot ng lun thay i (bin ng) tc t cn cp nht Cn mt m hnh biu din k hoch ca cc tc t khc Cc tc t cng tc
Cng chia s cc mc tiu hoc cc k hoch V d: Lp k hoch (cho hot ng nhm) trong tr chi tennis nh i Cc c ch cng tc: Phn tch v phn phi cc nhim v cho mi tc t
Tr tu nhn to 31

a tc t (2)
Cc tc t cnh tranh
V d: chi c Mi tc t phi nhn bit c s tn ti (v hot ng) ca cc tc t khc Mi t t t h ton (d ) tc tnh t (d on) c cc k h h ca ( t s) hoch (mt ) cc tc t khc Mi tc t tnh ton (d on) c nh hng ca cc k hoch ca cc t t kh i vi k h h ca b thn n tc khc i hoch bn th Mi tc t quyt nh hnh ng ti u i vi d on nh hng ny

Tr tu nhn to

32

Tc t - Tng kt g
Tc t tng tc vi mi trng thng qua cc b phn cm bin v cc b p hnh ng phn g Mt tc t hp l s cc i ha hiu qu hot ng ca n Hm tc t m t cc hnh ng m tc t thc hin trong cc tnh hung Cc chng trnh tc t ci t (thc hin) cc hm tc t Cc C m t PEAS xc nh mi t cng vic h i trng i Cc mi trng c phn loi da theo cc tiu ch: C th q quan st c? Xc nh? Phn on? Tnh? Ri rc? Tc t n l? Cc kin trc tc t c bn: Phn x n gin, Da trn m hnh, hnh Da trn mc tiu, Da trn li ch tiu
Tr tu nhn to 33