Trng i hc Cng Ngh Thng Tin

BI GING MN HC L THUYT THNG TIN

NI DUNG MN HC

Bi 1 Bi 2 Bi 3 Bi 4 Bi 5 Bi 6 Bi 7 Bi 8 Bi 9

Gii thiu Mt s khi nim c bn Chun b ton hc Lng tin Entropy M hiu M ha ti u ngun ri rc khng nh M ha ngun ph qut Knh ri rc khng nh, lng tin tng h
Trang 2

NI DUNG MN HC (tt)

Bi 10 Bi 11 Bi 12 Bi 13 Bi 14 Bi 15

M ha chng nhiu, nh l knh M khi tuyn tnh C s ton hc ca m ha chng nhiu M vng Gii thiu v mt m ha Mt s vn nng cao

Trang 3

TI LIU THAM KHO

1. Information Theory - Robert B.Ash, Nh xut bn Dover, Inc, 1990. 2. Introduction to Information Theory - Masud Mansuripur, Nh xut bn PrenticeHall, Inc, 1987. 3. A Mathematical Theory of Communication - C. E. Shannon, Tp ch Bell System Technical, s 27, trang 379423 v 623 656, thng 7 v thng 10, 1948. 4. C s L thuyt truyn tin (tp mt v hai) - ng Vn Chuyt, Nguyn Tun Anh, Nh xut bn Gio dc, 1998.

Trang 4

HNH THC NH GI

S c thng bo c th cho tng kha hc. Tuy nhin, thng l c hnh thc nh bn di. Thi (80%)

Gia k: thi vit (30%) Cui k: thi trc nghim 50 cu / 90 pht (50%) Np bi tp ln v bo co vo cui hc k

Lm bi tp ln (20%)

Trang 5

CC MN LIN QUAN

L thuyt xc sut K thut truyn s liu X l tn hiu s

Trang 6

Bi 1 Gii thiu
1.1 Thng tin l g? 1.2 Vai tr ca thng tin 1.3 L thuyt thng tin nghin cu nhng g? 1.4 Nhng ng dng ca l thuyt thng tin 1.5 L thuyt thng tin Lch s hnh thnh v quan im khoa hc hin i

Trang 7

Thng tin l g?

Mt vi v d

Hai ngi ni chuyn vi nhau. Ci m trao i gia h gi l thng tin. Mt ngi ang xem tivi/nghe i/c bo, ngi ang nhn thng tin t i pht/bo. Qu trnh ging dy trong lp. Cc my tnh ni mng v trao i d liu vi nhau. My tnh np chng trnh, d liu t a cng vo RAM thc thi.

Trang 8

Thng tin l g? (tt)

Nhn xt

Thng tin l ci c truyn t i tng ny n i tng khc bo mt iu g . Thng tin ch c ngha khi iu bn nhn cha bit. Thng tin xut hin di nhiu dng m thanh, hnh nh, ... Nhng dng ny ch l v bc vt cht cha thng tin. V bc l phn xc, thng tin l phn hn. Ng ngha ca thng tin ch c th hiu c khi bn nhn hiu c cch biu din ng ngha ca bn pht. Mt trong nhng phng tin din t thng tin l ngn ng. C hai trng thi ca thng tin: truyn v lu tr. Mi trng truyn/lu tr c gi chung l mi trng cha tin hay knh tin.
Trang 9

Vai tr ca thng tin

Cc i tng sng lun lun c nhu cu hiu v th gii xung quanh, thch nghi v tn ti. y l mt qu trnh quan st, tip nhn, trao i v x l thng tin t mi trng xung quanh. Thng tin tr thnh mt nhu cu c bn, mt iu kin cn cho s tn ti v pht trin. Khi KHKT, XH ngy cng pht trin, thng tin cng th hin c vai tr quan trng ca n i vi chng ta. V d, hnh ng xut pht t suy ngh, nu suy ngh ng, th hnh ng mi ng. Suy ngh li chu nh hng t cc ngun thng tin c tip nhn. V vy thng tin c th chi phi n suy ngh v kt qu l hnh ng ca con ngi.

Trang 10

LTTT nghin cu nhng vn g?

gc khoa hc k thut, LTTT nghin cu nhm to ra mt c s h tng tt cho vic truyn thng tin chnh xc, nhanh chng v an ton; lu tr thng tin mt cch hiu qu. cc gc nghin cu khc LTTT nghin cu cc vn v cch t chc, biu din v truyn t thng tin, v tng qut l cc vn v x l thng tin. M ho chng nhiu M ho ti u (hay nn d liu) Mt m ho

Ba lnh vc nghin cu c bn ca mn hc

Trang 11

Nhng ng dng ca LT thng tin

Cuc cch mng thng tin ang xy ra, s pht trin mnh m ca cc phng tin mi v truyn thng, lu tr thng tin lm thay i ngy cng su sc x hi chng ta. LTTT ng mt vai tr quyt nh trong s pht trin ny bng cch cung cp c s l thuyt v mt ci nhn trit hc su sc i vi nhng bi ton mi v thch thc m chng ta chm trn hm nay v mai sau. Nhng ng dng ph bin ca LTTT l truyn thng v x l thng tin bao gm: truyn thng, nn, bo mt, lu tr, ... Cc tng ca LTTT c p dng trong nhiu lnh vc nh vt l, ngn ng hc, sinh vt hc, khoa hc my tnh, tm l hc, ha hc
Trang 12

Nhng ng dng ca LT thng tin (tt)

Mi quan h gia LTTT v thng k c tm thy, cc phng php mi v phn tch thng k da trn LTTT c ngh. ng dng vo qun l kinh t. V d, l thuyt u t ti u xut hin ng thi vi l thuyt m ha ngun ti u. ng dng vo ngn ng hc. ng dng n tm l thc nghim v c bit l lnh vc dy v hc.

Trang 13

Lch s hnh thnh

Cuc cch mng ln nht v cch nhn th gii khoa hc l chuyn hng t thuyt quyt nh Laplacian n bc tranh xc sut ca t nhin. Th gii chng ta ang sng trong ch yu l xc sut. Kin thc ca chng ta cng l mt dng xc sut. LTTT ni ln sau khi c hc thng k v lng t pht trin, v n chia x vi vt l thng k cc khi nim c bn v entropy. Theo lch s, cc khi nim c bn ca LTTT nh entropy, thng tin tng h c hnh thnh t vic nghin cu cc h thng mt m hn l t vic nghin cu cc knh truyn thng. V mt ton hc, LTTT l mt nhnh ca l thuyt xc sut v cc qu trnh ngu nhin (stochastical process).
Trang 14

Lch s hnh thnh (tt)

Quan trng v c ngha nht l quan h lin kt gia LTTT v vt l thng k. Trong mt thi gian di trc khi LTTT c hnh thnh, L. Boltzman v sau l L.Szilard nh ng ngha ca thng tin vi khi nim nhit ng hc ca entropy. Mt mt khc, D. Gabor ch ra rng l thuyt truyn thng phi c xem nh mt nhnh ca vt l. C. E. Shannon l cha ca LTTT.

Trang 15

Bi 2 Mt s khi nim c bn
2.1 Thng tin (Information) 2.2 M hnh ca cc qu trnh truyn tin 2.3 Cc loi h thng truyn tin Lin tc v ri rc 2.4 Ri rc ho

Trang 16

Thng tin

Thng tin l mt khi nim tru tng, phi vt cht v rt kh c nh ngha chnh xc. Hai nh ngha v thng tin. Thng tin l s cm hiu ca con ngi v th gii xung quanh thng qua s tip xc vi n. Thng tin l mt h thng nhng tin bo v mnh lnh gip loi tr s khng chc chn (uncertainty) trong trng thi ca ni nhn tin. Ni ngn gn, thng tin l ci m loi tr s khng chc chn. nh ngha u cha ni ln c bn cht ca thng tin. nh ngha th hai ni r hn v bn cht ca thng tin v c dng nh lng thng tin trong k thut.

Trang 17

Thng tin (tt)

Thng tin l mt hin tng vt l, n thng tn ti v c truyn i di mt dng vt cht no . Nhng dng vt cht dng mang thng tin c gi l tn hiu. L thuyt tn hiu nghin cu cc dng tn hiu v cch truyn thng tin i xa vi chi ph thp, mt ngnh m c quan h gn gi vi LTTT. Thng tin l mt qu trnh ngu nhin. Tn hiu mang tin tc cng l tn hiu ngu nhin v m hnh ton hc ca n l cc qu trnh ngu nhin thc hay phc. V LTTT l l thuyt ngu nhin ca tin tc, c ngha l n xt n tnh bt ng ca tin tc i vi ni nhn tin.
Trang 18

M hnh ca cc qu trnh truyn tin

Khi nim thng tin thng i km vi mt h thng truyn tin. Nhiu Ngun pht Knh truyn Ngun nhn

S truyn tin (transmission)

L s dch chuyn thng tin t im ny n im khc trong mt mi trng xc nh. L mt tp hp cc tin m h thng truyn tin dng lp cc bng tin hay thng bo (message) truyn tin. Bng tin chnh l dy tin c bn pht truyn i. Thng tin c th thuc nhiu loi nh
(1) mt dy k t nh trong in tn (telegraph) ca cc h thng gi in tn (teletype system);
Trang 19

Ngun tin (information source)

M hnh ca cc qu trnh truyn tin (tt)

(2) mt hm theo ch mt bin thi gian f(t) nh trong radio v in thoi; (3) mt hm ca thi gian v cc bin khc nh trong tivi trng en y thng tin c th c ngh nh l mt hm f(x, y, t) ca to hai chiu v thi gian biu din cng nh sng ti im (x, y) trn mn hnh v thi gian t; (4) mt vi hm ca mt vi bin nh trong trng hp tivi mu y thng tin bao gm ba hm f(x, y, t), g(x, y, t), h(x, y, t) biu din cng nh sng ca cc ba thnh phn mu c bn (xanh l cy, , xanh dng)

Thng tin trc khi c truyn i, tu theo yu cu c th c m ho nn, chng nhiu, bo mt, ... L ni hnh thnh v truyn (hoc lu tr) tn hiu mang tin ng thi y xy ra cc tp nhiu (noise) ph hy tin tc. Trong LTTT knh l mt khi nim tru tng i biu cho hn hp tn hiu v tp nhiu.
Trang 20

Knh tin (channel)

Mt s khi nim (tt)

Mi trng truyn tin thng rt a dng

mi trng khng kh, tin c truyn di dng m thanh v ting ni, ngoi ra cng c th bng la hay bng nh sng; mi trng tng in ly trong kh quyn ni m thng xuyn xy ra s truyn tin gia cc v tinh nhn to vi cc trm rada di mt t; ng truyn in thoi ni xy ra s truyn tn hiu mang tin l dng in hay ng truyn cp quang qua bin trong tn hiu mang tin l sng nh sng v.v

Nhiu (noise)

Cho d mi trng no cng c nhiu. Nhiu rt phong ph v a dng v thng i km vi mi trng truyn tin tng ng.

Chng hn nu truyn di dng sng in t m c i qua cc vng ca tri t c t trng mnh th tn hiu mang tin thng b nh hng t nhiu bi t trng ny. Nn c th coi t trng ny l mt loi nhiu. Nu truyn di dng m thanh trong khng kh th ting n xung quanh c th coi l mt loi nhiu.
Trang 21

Mt s khi nim (tt)

Nhiu c nhiu loi chng hn nhiu cng, nhiu nhn. Nhiu cng l loi nhiu m tn hiu mang tin b tn hiu nhiu cng thm vo. Nhiu nhn l loi nhiu m tn hiu mang tin b tn hiu nhiu nhn ln. L ni tip nhn thng tin t knh truyn v c gng khi phc li thnh thng tin ban u nh bn pht pht i. Tin n c ni nhn thng khng ging nh tin ban u v c s tc ng ca nhiu. V vy ni nhn phi thc hin vic pht hin sai v sa sai. Ni nhn cn c th phi thc hin vic gii nn hay gii m thng tin c m ho bo mt nu nh bn pht thc hin vic nn hay bo mt thng tin trc khi truyn
Trang 22

Ni nhn tin (sink)

Cc loi h thng truyn tin

Cc ngun tin thng thy trong t nhin c gi l cc ngun tin nguyn thu. y l cc ngun tin cha qua bt k mt php bin i nhn to no. Cc tn hiu m thanh, hnh nh c pht ra t cc ngun tin nguyn thu ny thng l cc hm lin tc theo thi gian v theo mc, ngha l c th biu din mt thng tin no di dng mt hm s(t) tn ti trong mt qung thi gian T v ly cc tr bt k trong mt phm vi (smin, smax) no . s(t) smax

smin
Trang 23

Cc loi h thng truyn tin (tt)

Cc ngun nh vy c gi l cc ngun lin tc (continuous source), cc tin c gi l tin lin tc (continuous information) v knh tin c gi l knh lin tc (continuous channel). Tuy nhin vn c nhng ngun nguyn thu l ri rc

Bng ch ci ca mt ngn ng. Cc tin trong h thng in tn, cc lnh iu khin trong mt h thng iu khin, ...

Trong trng hp ny cc ngun c gi l ngun ri rc (discrete source), cc tin c gi l tin ri rc (discrete information) v knh tin c gi l knh ri rc (discrete channel). S phn bit v bn cht ca tnh ri rc v tnh lin tc l s lng tin ca ngun trong trng hp ri rc l hu hn cn trong trng hp lin tc l khng m c.

Trang 24

Ri rc ha

Cc h thng lin tc c nhiu nhc im ca nh cng knh, khng hiu qu, v chi ph cao. Cc h thng truyn tin ri rc c nhiu u im hn, khc phc c nhng nhc im trn ca cc h thng lin tc v c bit ang ngy cng c pht trin v hon thin dn nhng sc mnh v u im ca n. Ri rc ho thng bao gm hai loi: Ri rc ho theo trc thi gian, cn c gi l ly mu (sampling) v ri rc ho theo bin , cn c gi l lng t ho (quantize). Ly mu mt hm l trch ra t hm ban u cc mu c ly ti nhng thi im xc nh. Vn l lm th no s thay th hm ban u bng cc mu ny l mt s thay th tng ng, iu ny c gii quyt bng nh l ly mu ni ting ca Shannon.
Trang 25

Ly mu (Sampling)

Ri rc ha (tt)

nh l ly mu ca Shannon

Mt hm s(t) c ph hu hn, khng c thnh phn tn s ln hn max (= 2fmax) c th c thay th bng cc mu ca n c ly ti nhng thi im cch nhau mt khong t /max, hay ni cch khc tn s ly mu F 2fmax. s(t) smax

smin

Trang 26

Ri rc ha (tt)

Lng t ho (Quantize)

Bin ca cc tn hiu thng l mt min lin tc (smin, smax). Lng t ho l phn chia min ny thnh mt s mc nht nh, chng hn l smin = s0, s1, ..., sn = smax v qui cc gi tr bin khng trng vi cc mc ny v mc gn vi n nht. Vic lng t ho s bin i hm s(t) ban u thnh mt hm s(t) c dng hnh bc thang. S khc nhau gia s(t) v s(t) c gi l sai s lng t. Sai s lng t cng nh th s(t) biu din cng chnh xc s(t). s(t) smax

smin
Trang 27

Ngun ri rc

Ngun tin lin tc sau khi c ly mu v lng t ho s tr thnh ngun ri rc. Chng ta hc ch yu cc ngun ri rc. Mt ngun ri rc l mt bng ch ci A gm m k hiu, A = {a1, a2, ..., am}, vi nhng xc sut xut hin p(ai), i = 1, .., m. nh ngha khng din t mi quan h gia tin trc v sau trong mt bn tin, nn y c gi l mt ngun ri rc khng nh (discrete memoryless source). L mt dy (c th v hn) cc k hiu lin tip t bng ch ci ca ngun tin, x = (... a2a1a0a1a2...) Trong thc t bng tin c bt u v kt thc cho nn bng tin l mt dy hu hn cc k hiu, x* = (a1a2 an)
Trang 28

Ngun ri rc

Bng tin ca mt ngun tin ri rc khng nh

Bi 3 Chun b ton hc
3.1 Xc sut (Probability) 3.2 Bt ng thc Chebyshev v lut yu ca s ln 3.3 Tp li (Convex sets) v hm li (convex functions), bt ng thc Jensen

Trang 29

Xc sut

Khng gian mu (Sample space)

S kin (Event), s kin c bn (elementary event)

L tp (hay khng gian) tt c cc kt qu c th c ca mt th nghim. Thng c k hiu l E hay S. Nu khng gian mu l ri rc th E c th c biu din bng E = {e1, e2, ..., en} Mi tp con ca E (khng gian mu) c gi l mt s kin, c bit mi phn t ca E c gi l mt s kin c bn. Trong mt th nghim tung ng xu th E = {U (p), N (nga)}. Nu ng tin l ng nht th xc sut P(U) = P(N) = 1/2. Trong mt th nghim tung con xc xc th E = {1, 2, 3, 4, 5, 6}. Nu con xc xc l ng nht th xc sut P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6, P(2, 5) = 1/3, P(1, 3, 5) = 1/2.
Trang 30

V d

Xc sut (tt)

Ly mt vn bn ting Anh in hnh v nht mt k t bt k th E = {a, b, c, ..., x, y, z} v xc sut ca cc k t c phn b nh sau P(a) = 0,0642 , ..., P(e) = 0,103 , ..., P(z) = 0,0005. Mt bin ngu nhin ri rc x c nh ngha bng cch gn mt s thc xi ti mi s kin c bn ei ca khng gian mu ri rc E. Xc sut ca xi c nh ngha l xc sut ca s kin c bn tng ng v c k hiu l p(xi). Tr trung bnh v phng sai ca bin ngu nhin ri rc x ln lt c k hiu v nh ngha nh sau E(x) = x = x i p (x i )
i
Trang 31

Bin ngu nhin ri rc (Discrete random variable)

Tr trung bnh (k vng) (average, expected value), phng sai (variance)

Var(x)

Xc sut (tt) 2 2 = E ((x x ) ) = (x i x ) p(x i )

= E x x trong E(x2) l tr k vng ca x2. Tng qut, tr k vng ca mt hm ca x, chng hn f(x), c nh ngha bng
2

( )

E ( f (x )) = f (x i ) p(x i )
i

Xc sut ng thi (joint probability), xc sut c iu kin (conditional probability)

Mt cp bin ngu nhin (x, y) lin kt vi mt th nghim to thnh mt bin ngu nhin ni (joint random variable). Nu x, y l ri rc, s phn b xc sut ni hay xc sut ng thi c nh ngha l pij = P(x = xi, y = yj)
Trang 32

Xc sut (tt)

Xc sut ca y trong iu kin bit x c gi l xc sut c iu kin v c nh ngha l

p ( y j xi ) =

p (xi , y j ) p( xi )

trong xc sut l (marginal probability) p(xi) c gi thit l khc khng. Cc xc sut l c nh ngha nh sau: p(xi) = p xi , y j

p(yj) =

p(x , y )
i j i

Trang 33

V d

Xc xc Th nghim tung ng thi 6 mt ng xu v con xc xc. 5 T kt qu trn ta thy 4 P(U, 5) = 1/18 3 P(ng xu = U) = 5/9 P(ng xu = N) = 4/9 2 P(Xc xc = 5) = 7/72 1 P(Xc xc = 5 bit ng xu = U) =(1/18)/(5/9)=1/10

1/12 1/18 1/9 1/9 1/9 1/12 U

1/12 1/24 1/24 1/6 1/18 1/18 N ng xu

Trang 34

Xc sut (tt)

S c lp (Independence)

Hai bin ngu nhin x v y c gi l c lp nu p(xi, yj) = p(xi)p(yj) i, j. Chng ta thy nu hai bin x v y c lp th p (xi , y j ) p ( xi ) p ( y j ) p y j xi = = = p(y j ) p ( xi ) p ( xi ) c ngha l xc sut yj trong iu kin c xi xy ra hay khng xy ra u nh nhau, khng thay i, v ngc li. Cng t s c lp chng ta suy ra mt kt qu m hay c s dng sau ny E(xy) = E(x) E(y) = x y

Trang 35

Xc sut (tt)

S tng quan (correlation)

S tng quan C gia hai bin x v y c nh ngha l tr k vng ca (x x )(y y): C(x, y) = E((x x )(y y )) = = E(xy) x y Trong trng hp x v y l c lp chng ta suy ra C(x, y) = 0. Tuy nhin iu ngc li th khng ng.

Trang 36

Q1

Bt ng thc Chebyshev v lut yu ca s ln

Bt ng thc Chebyshev

Cho mt bin ngu nhin x c tr trung bnh l x v phng sai l x2, bt ng thc Chebyshev i vi mt s dng tu l

Chng minh

x2 P(|x x | ) 2

1,|x - x| nh ngha mt hm f(x) nh sau f (x ) = 0 ,|x - x| < Th

P(|x x| ) = f(xi)p(xi)

Trang 37

Slide 37 Q1 Bt ng thc Chebyshev ch ra cn trn ca xc sut mt i lng ngu nhin lch khi k vng ton hc ca n: gi s X l i lng ngu nhin c k vng ton hc l EX=a v phng sai DX=d2. Bt ng thc Chebyshev ch r rng vi e>0 cho trc, xc sut ca bin c |X-a|>=e khng vt qu d2/e2. Bt ng thc ny c dng chng minh lut s ln.
Quang, 3/12/2008

Bt ng thc Chebyshev (tt)

xx
1

x +

Da trn hnh chng ta c V vy,

P xx i

xx f(x) 2 2 xx p (x i ) = x2

Trang 38

Lut yu ca s ln (tt)

Xt mt th nghim nh phn trong cc kt qu ca th nghim l 0 v 1 vi cc xc sut tng ng l p0 v 1 p0. Th nghim ny c lp li N ln mt cch c lp, v kt qu trung bnh c nh ngha l yN; tc l, yN bng tng s cc s 1 trong N ln th nghim chia cho N. R rng, yN l mt bin ngu nhin c khng gian mu l {0, 1/N, 2/N, ..., 1}. nh ngha x(n) l bin ngu nhin tng ng vi kt qu ca ln th nghim th n, chng ta c

1 yN = N

(n ) x
n =1

Trang 39

Lut yu ca s ln (tt)
1 yN = N

E (x )
N

(n )

n =1

1 = N
(n )

x =x
n =1
2

= E yN yN
2 y

((

))
2

1 = E N

x
n =1

2 2 N 1 N 1 (n ) ( n) E x x = =E x N x 2 N N n =1 = 1 n

1 = 2 N

(n ) x x E
n =1

((

))
2

2 1 = 2 N x2 = x N N

Trang 40

Lut yu ca s ln (tt)

i vi mt s nguyn dng tu , theo bt ng thc Chebyshev chng ta c

t y chng ta dn ra c lut yu ca s ln

y2 P | y N y N | 2

1 P N

2 (n ) x x x 2 N n =1 N

Ch rng v phi tin ti 0 khi N tin ra v cng. Lut yu ca s ln v vy khng inh rng tr trung bnh mu ca x tip cn tr trung bnh thng k vi xc sut cao khi N .
Trang 41

Tp li

Trong khng gian clit, mt tp S c gi l li () nu i vi mt cp im P1, P2 thuc S th mi im thuc on P1P2 cng thuc S. P1 P1 P2 P2

(b) (a) Nu P1 = (x1, x2, ..., xn) v P2 = (y1, y2, ..., yn) l cc im trong khng gian clit n chiu, th on thng ni chng c biu din bng tp cc im P, trong P = P1 + (1)P2 = (x1 + (1)y1, x2 + (1)y2, ..., xn + (1)yn) v [0, 1].
Trang 42

Hm li

Mt v d quan trng ca tp li l tp tt c cc im (p1, p2, ..., pn) trong (p1, p2, ..., pn) l mt s phn b xc sut (tc l cc pi [0, 1] v pi = 1). Mt hm thc f(P), c nh ngha trn tp li S, c gi l li nu cp im P1, P2 S, v [0, 1] bt ng thc sau y ng: f(P1 + (1)P2) f(P1) + (1)f(P2) f((x1 + (1-)x2) f(x) f(x2) f(x1) f(x1) + (1-)f(x2) x2 x
Trang 43

x1 (x1 + (1-)x2

Q2

nh l, bt ng thc Jensen

Nu 1, ..., N l cc s khng m c tng bng 1 th i vi mi tp im P1, ..., PN trong min xc nh ca hm li f(P) bt ng thc sau y ng

N N f n Pn n f (Pn ) n =1 n =1

Cho bin ngu nhin x ly cc gi tr x1, ..., xn vi cc xc sut p1, ..., pn. Cho f(x) l mt hm li c min xc nh cha x1, ..., xn. Chng ta c E(x) = pi xi v E(f(x)) = pi f ( xi ) . i i p dng nh l trn chng ta c f(E(x)) E(f(x)) y c gi l bt ng thc Jensen.
Trang 44

Slide 44 Q2 f(x1)+f(x2)+...+f(xn) >= nf((x1+x2+...+xn)/n)

Quang, 3/13/2008

Bi 4 Lng tin
4.1 Lng tin 4.2 Lng tin trung bnh
Vn c bn ca truyn thng l vic ti sinh ti mt im hoc chnh xc hoc gn ng mt thng bo c chn ti mt im khc. (Claude Shannon 1948)

Trang 45

Lng tin

Lng tin (measure of information) dng so snh nh lng cc tin tc vi nhau. Mt tin i vi ngi nhn u mang hai ni dung, mt l bt ng ca tin, hai l ngha ca tin. Kha cnh ng ngha ch c ngha i vi con ngi. Kha cnh quan trng nm ch tin tht s l mt ci c chn t mt tp cc tin (tp cc kh nng) c th. Nu s tin trong tp tin cng nhiu th s mang li mt lng tin cng ln khi nhn c mt tin (gi s cc tin l bnh ng nh nhau v kh nng xut hin). s truyn tin t hiu qu cao chng ta khng th i i cc tin nh nhau nu chng xut hin t nhiu khc nhau.
Trang 46

V d
n=16 -> log2n cu hi yes=0 -> lng tin log2n bit yes=0 yes=0 no=1 no=1 no=1 Kt qu: 1010 -> s 11 Qu trnh thc hin 4 cu hi no=1 yes=0

Lng tin

Xt mt tin x c xc sut xut hin l p(x), th chng ta c th xem tin ny nh l mt tin trong mt tp c 1/p(x) tin vi cc tin c xc sut xut hin nh nhau. Nu p(x) cng nh th 1/p(x) cng ln v v vy lng tin khi nhn c tin ny cng s cng ln. Vy lng tin ca mt tin t l thun vi s kh nng ca mt tin v t l nghch vi xc sut xut hin ca tin . Xc sut xut hin ca mt tin t l nghch vi bt ng khi nhn c mt tin. lng tin s kh nng bt ng xc sut Mt tin c xc sut xut hin cng nh th c bt ng cng ln v v vy c lng tin cng ln.
Trang 48

Lng tin (tt)

Xt mt ngun A = {a1, a2,, am} vi cc xc sut xut hin l p(ai) i = 1, ..., m. K hiu lng tin trong mi tin ai l I(ai). Vy hm f dng biu th lng tin phi tho mn nhng iu kin g? Phn nh c cc tnh cht thng k ca tin tc.

V d c hai ngun K, L vi s tin tng ng l k, l (gi thuyt u l ng xc sut). Nu k > l, th bt ng khi nhn mt tin bt k ca ngun K phi ln hn bt ng khi nhn mt tin bt k ca ngun L, vy f(k) > f(l) Gi thit hai ngun c lp K v L vi s tin tng ng l k v l. Cho vic nhn mt cp ki v lj bt k ng thi l mt tin ca ngun hn hp KL. S cp kilj m ngun ny c l k*l.
Trang 49

Hp l trong tnh ton.

Lng tin (tt)

bt ng khi nhn c mt cp nh vy phi bng tng lng tin ca khi nhn c ki v lj. V vy chng ta phi c: f(kl) = f(k) + f(l)

Khi ngun ch c mt tin, lng tin cha trong tin duy nht phi bng khng. f(1) = 0 Lng o thng tin ca mt tin c o bng logarit ca bt ng ca tin hay nghch o xc sut xut hin ca tin . 1 I ( x ) = log = log p( x) p ( x)

nh ngha

Trang 50

Lng tin (tt)

Lng tin cha trong mt dy x = a1a2 an vi ai A l

n 1 I ( x ) = log = log p(ai ) p( x) i =1 Trong trng hp m k hiu ca ngun ng xc sut vi nhau tc p(ai) = 1/m th 1 I (ai ) = log = log m p(ai )

Nu x = a1a2 an vi ai A I(x) = n logm

Trang 51

Lng tin trung bnh

n v ca lng tin

Nu c s l 2 th n v l bits (cho cc k s nh phn); nu c s l e th n v l nats (cho n v t nhin), nu c s l 10 th n v l Hartley. Lng tin trung bnh ca mt ngun tin A l lng tin trung bnh cha trong mt k hiu bt k ca ngun tin. N thng c k hiu l I(A) v c tnh bng cng thc sau

nh ngha

I ( A) =

ai A

p(ai ) I (ai ) = p(ai ) log p(ai )

ai A
Trang 52

V d
S cu hi nh phn? 1/2 1/4 1/8 1/8

(1/2 x 1) + (1/4 x 2) + (1/8 x 3) + (1/8 x 3) = 1.75 cu hi (trung bnh) <> 2 cu hi

V d

Cho mt ngun tin U bao gm 8 tin U = {u0, u1, u2, u3, u4, u5, u6, u7}, vi cc xc sut xut hin nh sau: p(u0) p(u1) p(u2) p(u3) p(u4) p(u5) p(u6) p(u7) 1/4 1/4 1/8 1/8 1/16 1/16 1/16 1/16 Hy cho bit lng tin ring ca mi tin v lng tin trung bnh ca ngun ny trong n v bits.

Gii

Lng tin ring ca mi tin l I(u0) I(u1) I(u2) I(u3) I(u4) I(u5) I(u6) I(u7) 2 2 3 3 4 4 4 4
Trang 54

V d (tt)

Lng tin trung bnh ca ngun l I(U) = (1/4) 2 + (1/4) 2 + (1/8) 3 + (1/8) 3 + (1/16) 4 + (1/16) 4 + (1/16) 4 + (1/16) 4 = 2,75 bits. iu ny ni ln mt ngha quan trng rng, chng ta c th biu din mi tin trong ngun U bng mt chui c chiu di trung bnh l 2,75 bits. N s tt hn so vi trong trng hp chng ta khng ch n cu trc thng k ca ngun. Lc chng ta s biu din mi tin trong 8 tin ca ngun bng cc chui c chiu di l 3 bits.

Trang 55

Bi 5 Entropy
5.1 Entropy ca mt bin ngu nhin ri rc 5.2 Cc c tnh ca entropy 5.3 Entropy v cc dy ca mt bin ngu nhin

Trang 56

Entropy ca mt bin ngu nhin ri rc

nh ngha

Cho x l mt bin ngu nhin vi khng gian mu X = {x1, ... , xN} v o xc sut P(xn) = pn. Entropy ca x c nh ngha l: N H (x ) = pn log( pn )
n =1

p ln(p) e-1

e-1 = 0,37
Trang 57

Entropy ca mt bin ngu nhin ri rc (tt)

V d

Cho X = {0, 1}, P(0) = p, cn P(1) = 1p. Th H(x) = plog(p) (1 p) log(1 p) H(x) 1

0,5
Trang 58

Cc c tnh ca entropy
1. Entropy l mt i lng lun lun dng hoc bng khng.

H(x) = 0 c mt xc sut pi = 1, cn tt c cc xc sut cn li bng 0. iu ny ni ln rng bt ng v mt th nghim ch c mt kt qu duy nht l bng 0.

2. H(x) log N v du bng xy ra p1 = p2 = ... = pN = 1/N. Hay ni cch khc entropy t cc i khi xc sut xut hin ca cc k hiu bng nhau.

Chng minh

1 H ( x ) ln( N ) = pn ln ( pn ) pn ln ( N ) = pn ln Np n =1 n =1 n =1 n N 1 N 1 N pn pn = 1 1 = 0 Np 1 = n =1 n n =1 N n =1
N N N
Trang 59

Cc c tnh ca entropy (tt)

3. Cho bin ngu nhin x c khng gian mu X = {x1, ..., xN} v bin ngu nhin y c khng gian mu Y = {y1, ..., yM}. Th bin ngu nhin ni z = (x, y) c khng gian mu Z = {(x1, y1), ..., (x1, yM), (x2, y1), ..., (x2, yM), ..., (xN, y1), ..., (xN, yM)} gm NM phn t. Nu x, y c lp nhau th H(z) = H(x) + H(y).

Chng minh
N M n=1 m=1 N
n=1

H(z) = P(xn , ym ) logP(xn , ym ) = P(xn )P( ym )[logP(xn ) + logP( ym )]

n=1 m=1

N M

= P(xn ) logP(xn )P( ym ) P(xm ) logP(xm )P( yn )

m=1 m=1 n=1

= H (x) + H (y)
Trang 60

Cc c tnh ca entropy (tt)

4. Xt mt bin ngu nhin x c khng gian mu X = {x1, ..., xn, xn+1, ..., xN} v cc xc xut p(xi) = pi. Chng ta phn X thnh hai khng gian con, Y = {x1, ..., xn} v Z = {xn+1, ..., xN}. Cc n xc sut lin kt vi Y v Z c cho bi P(Y) = i =1 pi N v P(Z) = i = n +1 pi . Hn na, chng ta nh ngha cc bin ngu nhin y v z bng P(yi) = P(xi)/P(Y), i = 1, 2, ..., n v P(zi) = P(xi)/P(Z), i = n+1, n+2, ..., N. H(x) by gi c th c vit thnh

H ( x ) = pi log pi = pi log pi
i =1 i =1

i = n +1

p log p
i

= P(Y )P( yi )(logP( yi ) + logP(Y )) P(Z ) P(zi )(logP(zi ) + logP(Z ))

= [P(Y )log P(Y ) + P(Z )log P(Z )] + [P(Y )H ( y) + P(Z )H ( z)]

Trang 61

i=1

i =n+1

Cc c tnh ca entropy (tt)

Trong biu thc cui cp ngoc vung u biu din bt ng lin kt vi th nghim th nht (l chn mt trong hai khng gian mu Y v Z) cn cp ngoc vung th hai biu din bt ng trung bnh lin kt vi th nghim th hai (sau khi chn mt trong hai khng gian mu, s chn tip s kin c bn no). Cng thc ny din t mt tnh cht ca entropy l tnh cht nhm.

Ngi ta chng minh c rng cng thc nh ngha ca H(x) l cng thc duy nht ph hp o v bt ng, ci m phi tho mn cc tnh cht 2,3, 4 v cng thm tnh lin tc. Mc du hai khi nim lng tin trung bnh v entropy xut hin mt cch c lp v trong nhng lnh vc khc nhau (entropy vn xut pht t vic nghin cu cc qu trnh nhit ng) nhng chng c cng cng thc ging nhau. V vy chng ta c th xem lng tin trung bnh ca mt ngun chnh l entropy ca ngun .
Trang 62

Entropy v cc dy ca mt bin ngu nhin

V d

Xt mt bin ngu nhin x c khng gian mu X = {x1, x2}, P(x1) = p1 = 1/3, P(x2) = 2/3. Th entropy ca x l H(x) = (1/3) log(1/3) (2/3) log(2/3) = 0.918295834 bits Chng ta hy lp li th nghim ny N ln nhn mt dy N phn t. Tng qut c n 2N dy c th. Nu trong dy c n phn t x1 th xc sut xut hin ca dy l p1n(1p1)Nn N! dy nh vy, nn tng xc sut ca chng N C (n )= n!( N n)! bng n N N-n

( n ) p1 (1-p1 )

Bng bn di trnh by xc sut ca cc dy khc nhau i vi N = 15

Trang 63

Entropy v cc dy ca mt bin ngu nhin (tt)

S dy P mi dy S dy P mi dy P t ng c ng P tng cng n N n N N p n(1p )Nn n(1p )Nn N p n(1p )Nn (n ) p1n(1p1)Nn (n p ( ) ) 1 (n ) 1 n 1 1 1 1 0 1 215x0.584962501 0.002284 8 6435 215x1.118295834 0.057404 1 15 215x0.651629167 0.017127 9 5005 215x1.184962501 0.022324 0.059946 10 3003 215x1.251629167 0.006697 2 105 215x0.718295834 3 455 215x0.784962501 0.129883 11 1365 215x1.318295834 0.001522 0.194825 12 455 215x1.384962501 0.000254 4 1365 215x0.851629167 5 3003 215x0.918295834 0.214307 13 105 215x1.451629167 0.000029 0.178589 14 15 215x1.518295834 0.000002 6 5005 215x0.984962501 7 6435 215x1.051629167 0.114807 15 1 215x1.584962501 0.000000

Trang 64

Nhn xt

Nhng dy c xc sut ln (dy c kh nng) l nhng dy m c n gn vi gi tr Np1 = 5, c th l 2 n 8. Ni cch khc, Xc sut xut hin ca mt dy m c n nm xa gi tr Np1 l rt nh. Xsut ring ca nhng dy c kh nng nm gia 2150.718295834 v 215 1.118295834, ci m gn st vi 2NH(x) = 2150.918295834. Ni cch khc, Tt c nhng dy c kh nng l nhiu hay t ng xc sut vi xc sut 2NH(x). S lng tng cng cc dy kh nng (2 n 8) l 22803 = 215 0.965129067 ci m khng xa so vi 2NH(x). Ni cch khc, S lng cc dy c kh nng l khong 2NH(x).
Trang 65

nh l

nh l 5.1

Cho cc s > 0 v > 0 nh tu , mt s nguyn dng N0 sao cho mt dy c chiu di bt k N N0 s ri vo mt trong hai lp sau y: (1) Mt tp cc dy m c tng xc sut ca chng nh hn hoc bng . (2) Tp cn li bao gm cc dy c xc sut tho mn bt ng thc NH A N NH + A N

< p<2

vi A l mt s dng no . Hay ni cch khc, log p 1 H N

Trang 66

Chng minh nh l

Chng minh cho ngun ri rc khng nh A = {a1, a2, ..., aK}. Gi x l bin ngu nhin gn vi ngun A. Ta c

H ( x ) = p (ak ) log p (ak )

k =1

Gi y l bin ngu nhin bng cch nh x mi ai ti log p(ai).

y = p (ai ) log p (ai ) = H ( x )

Xt cc dy c chiu di N. C tt c KN dy nh vy. Ta k hiu cc dy ny bng cc Si v xc sut ca dy l P(Si). Ta c

i =1

P( Si ) = p(a ( j ) )
j =1

trong a(j) l k hiu th j ca dy.

Trang 67

Chng minh nh l

Gi z l bin ngu nhin bng cch nh x mi Si ti -log P(Si). N Ch log P ( S ) = log p (a ( j ) )

i

j =1

V vy z l tng ca N bin ngu nhin y c lp. p dng lut yu ca s ln cho hai s > 0 v > 0 nh tu , tn ti N0 sao cho vi mi N N0 N 1 ( j) y P y N j =1 hay

1 N ( ) j P log p(a ) H (x) N j =1

Trang 68

Chng minh nh l (tt)

Hay

V vy chng ta c th kt lun rng vi xc sut ln hn 1 1 log P ( S i ) H ( x ) N i vi mi N N0. T y ta suy ra rng cc dy c chia thnh hai nhm, mt nhm c tng xc xut nh hn hoc bng v nhm th hai bao gm cc dy tho iu kin . 1 1 H (x) log N P( Si ) V vy nh l c chng minh.
Trang 69

1 P N log P(S i ) H ( x )

Bi 6 M hiu
6.1 Gii thiu 6.2 M hiu v cc thng s c bn ca m hiu 6.3 Mt s phng php biu din m 6.4 iu kin phn tch m

Trang 70

Gii thiu

Trong cc h thng truyn tin, bn nhn thng bit tp hp cc tin m bn pht dng lp nn cc bn tin. Cc tin thng s c nh x (m ha) thnh mt dng biu din khc thun tin hn pht i. Xt mt ngun tin A = {a, b, c, d}. Chng ta c th thit lp mt song nh nh sau t A vo tp cc chui trn bng ch ci {0, 1} a 00 c 10 b 01 d 11 Vy pht i bn tin baba chng ta pht i chui 01000100. Khi bn nhn nhn c chui ny th xc nh c bn tin bn pht pht i l baba.
Trang 71

V d

M hiu v nhng thng s c bn

M hiu (Code), c s m

M hiu l mt tp hu hn cc k hiu v php nh x cc tin/bn tin ca ngun tin thnh cc dy k hiu tng ng. Tp cc k hiu v php nh x ny thng s phi p ng cc yu cu ty theo h thng truyn tin t ra. Tp cc k hiu m dng biu din c gi l bng k hiu m, cn s cc k hiu th c gi l c s m, v thng k hiu l m. Nu m c c s hai th gi l m nh phn, cn nu m c c s ba th gi l m tam phn ... M ho l qu trnh dng cc k hiu m biu din cc tin ca ngun.
Trang 72

M ho (Encoding), gii m (decoding)

M hiu v nhng thng s c bn (tt)

Ni cch khc m ho l mt php bin i t ngun tin thnh m hiu, hay m ho l php bin i t mt tp tin ny thnh mt tp tin khc c c tnh thng k yu cu. Qu trnh ngc li ca qu trnh m ho c gi l gii m. T m l chui k hiu m biu din cho tin ca ngun. Tp tt c cc t m tng ng vi cc tin ca ngun c gi l b m. V vy c th ni m ho l mt php bin i mtmt gia mt tin ca ngun v mt t m ca b m. Trong mt s trng hp ngi ta khng m ho mi tin ca ngun m m ho mt bn tin hay khi tin. Lc ny chng ta c khi nim m khi.
Trang 73

T m (Code word), b m

M hiu v nhng thng s c bn (tt)

Cc t m thng c k hiu l u, v, w. Chiu di t m l s k hiu c trong t m thng c k hiu l l. Chiu di trung bnh ca b m thng c k hiu l l v c cho bng cng thc n l = p ( xi )li i =1 trong n l s tin ca ngun cn li l chiu di t m tng ng vi tin xi ca ngun. Mt b m c gi l m u nu cc t m ca b m c chiu di bng nhau.
Trang 74

Chiu di t m, chiu di trung bnh

Phn loi m: m u, m y, m vi

M hiu v nhng thng s c bn (tt)

Mt b m u c c s m l m, chiu di t m l l v s lng t m n bng vi ml th c gi l m y, ngc li th c gi l m vi. Ngoi ra khi nim m y cn c dng theo ngha rng hn nh sau: mt b m c gi l y theo mt tnh cht no (chng hn tnh u hay tnh prefix nh sau ny cc bn s thy) nu khng th thm mt t m no vo m vn gi c tnh cht . Cho bng k hiu m A = {0, 1}. Th b m X1 = {0, 10, 11} l m khng u, b m X2 = {00, 10, 11} l m u nhng vi cn b m X3 = {00, 01, 10, 11} l m u v y.

V d

Trang 75

Mt s phng php biu din m

Bng i chiu m

L cch lit k cc tin ca ngun v t m tng ng trong mt bng. a1 a2 a3 a4 a5 a6 T m 00 010 011 10 110 111 Tin

Mt to m

L cch biu din mi t m w = a0a1al-1 bng mt im (l, b) trong mt phng to hai chiu, trong l l chiu di t m cn b l trng s ca t m c tnh nh sau vi m l c s l 1 m

b=

i=0

i a m i

Trang 76

Mt s phng php biu din m (tt)

V d
Tin a1 a2 a3 a4 a5 a6 T m 00 010 011 10 110 111 Tin T m Chiu di l Trng s b a1 a2 a3 00 010 011 2 3 3 0 2 6 a4 a5 a6 10 110 111 2 3 3 1 3 7

b 7 6 5 4 3 2 1 0

a6 a3

a5 a2 a4 a1

4 l

Trang 77

Mt s phng php biu din m (tt)

Cy m

L cch biu din cc t m bng cc nt l ca mt cy. Mi nt l biu din cho t m trng vi nhn ca con ng i t nt gc n nt l ny. 0 1 0 00 1 0 1 10 0 1 0 1

110 111 010 011 M c c s m th cy m tng ng s l cy m phn. Phng php cy m ch cho php biu din nhng m prefix, tc l khng c t m no trng vi phn i u ca mt t m khc.
Trang 78

Mt s phng php biu din m (tt)

hnh kt cu m

L mt dng c bit ca cy m, trong cc nt l trng vi nt gc v ngoi ra mi cnh ca hnh kt cu m u l cnh c hng. V vy mt t m c biu din bng mt chu trnh xut pht t nt gc v quay tr v li nt gc. 0 0 0,1 0,1 1 0

Hm cu trc m

L cch biu din s phn b cc t m theo di ca chng. Phng php ny biu din bng mt hm G(li) cho bit c bao nhiu t m c chiu di li.
Trang 79

Mt s phng php biu din m (tt)

V d

B m trong cc v d trn c biu din bng hm cu trc m sau y G(li) = 2, khi li = 2 4, khi li = 3

Trang 80

iu kin phn tch m

V d

Xt b m X1 = {0, 10, 11} m ho cho ngun A = {a, b, c}. Gi s bn pht pht i bng tin x = abaac, lc chui t m tng ng c pht i l y = 0100011. Vn l bn nhn sau khi nhn c chui t m y lm sao c th nhn bit c bng tin tng ng m bn pht pht. lm c iu ny, bn nhn phi thc hin mt qu trnh c gi l tch m. Chng hn vi chui k hiu m nhn c nh trn th bn nhn ch c mt kh nng tch m hp l l 0 | 10 | 0 | 0 | 11 v xc nh c bng tin c gi i l abaac.

Trang 81

iu kin phn tch m (tt)

Xt mt b m khc X2 = {0, 10, 01} m ho cho ngun A trn. Gi s bn nhn nhn c chui k hiu l y = 01010 v thc hin qu trnh tch m. y ta thy bn nhn c th thc hin c ba kh nng tch m hp l sau 0 | 10 | 10, 01 | 0 | 10 v 01 | 01 | 0. V v vy bn nhn s khng bit c chnh xc bn pht pht i bng tin no trong ba bng tin sau abb hay cab hay cca. Mt m nh vy th khng ph hp cho vic tch m v c gi l m khng phn tch c (uniquely undecodable code). V vy iu kin mt b m l phn tch c (uniquely decodable code) l khng tn ti dy t m ny trng vi dy t m khc ca cng b m.

Trang 82

iu kin phn tch m (tt)

Xt mt b m khc X3 = {010, 0101, 10100} m ho cho ngun A trn. Gi s bn nhn nhn c chui k hiu l 01010100101 v thc hin qu trnh tch m. y ta thy ch c mt cch tch m duy nht l 0101 | 010 | 0101 nhng vic tch m tr nn kh khn hn so vi b m X1. Chng hn lc chng ta gp chui 010 chng ta cha dm chc l mt t m v n c th l phn i u ca t m 0101, iu ny ph thuc vo k hiu i ngay sau chui 010. Nu k hiu i ngay sau l 0 th chng ta khng nh c 010 l t m v 0 l phn i u ca mt t m khc sau . Cn nu k hiu i ngay sau l 1 th chng ta khng khng nh c, v c hai kh nng hoc 010 l mt t m v 1 l phn i u ca mt t m khc sau , hoc 0101 l mt t m.
Trang 83

iu kin phn tch m (tt)

Nguyn nhn ca iu ny l do trong b m c mt t m ny l tip u ng ca mt t m khc. V cng chnh l nguyn nhn v bn cht ca vic mt dy k hiu c th tch thnh hai dy t m khc nhau. Tht vy, nu khng c t m no l tip u ng ca t m khc (hay m l prefix) th vi mi dy t m ch c duy nht mt cch tch thnh cc t m thnh phn. V vy nh sau ny chng ta s thy cc m thng c s dng l cc m prefix. Da vo tnh tip u ng trn, nhn bit mt b m (d nhin khng phi l m prefix) c phn tch c hay khng ngi ta thng dng mt cng c c gi l bng th m.

Trang 84

Bng th m

Bn cht ca bng th m l phn tch nhng t m di thnh nhng t m ngn i u. Chng hn t m di u1 c th c phn tch thnh v11v12...v1kw11 trong v11, .., v1k l cc t m ngn cn w11 l phn cn li ca u1. Nu w11 cng l mt t m th b m ny l khng phn tch c v chui v11v12...v1kw11 c t nht hai cch phn tch thnh cc t m, l u1 v v11, v12, ..., v1k, w11. Cn nu ngc li w11 khng l t m th chng ta dng n xt tip. Trong ln xt tip theo chng ta xt xem mi w11 ny c l tip u ng ca cc t m hay khng, nu ng vi mt t m no , gi s l u2, th t m ny s c dng v21...v2lw22 trong v21, ..., v2l l cc t m ngn (l c th bng 0) cn w22 l tip v ng cn li.
Trang 85

Bng th m (tt)

Tng t nu w22 cng l mt t m th b m l khng phn tch c v chui v11v12...v1kw11v21...v2lw22 c t nht hai cch phn tch thnh cc t m, l v11v12...v1kw11 | v21 | ... | v2l | w22, v v11 | v12 | ... | v1k | w11v21...v2lw22. Nu ngc li w22 khng l t m th chng ta dng n xt tip theo khun mu tng t nh trn. V vy chng ta kt lun rng Nu trong mt ln phn tch no , c mt t m di, chng hn u, c phn tch thnh dy wiiv(i+1)1...v(i+1)n trong wii l tip v ng ca mt t m no trong ln phn tch ngay trc , cn v(i+1)1, ..., v(i+1)n l cc t m ngn th b m l khng phn tch c.

Trang 86

Bng th m (tt)

Tht vy, lc s tn ti mt dy k hiu sau v11v12...v1kw11v21...v2lw22 . . .w(i1)(i1)vi1...vimwiiv(i+1)1...v(i+1)n ci m c th phn tch thnh hai dy t m khc nhau. Cch 1 l v11 | v12 | ... | v1k | w11v21...v2lw22 | . . . | w(i1)(i1)vi1...vimwii | v(i+1)1 | ... | v(i+1)n Cch 2 l v11v12...v1kw11 | v21 | ... | v2l | w22 ...w(i1)(i1) | vi1 | . . . | vim | wiiv(i+1)1...v(i+1)n

Trang 87

Cch xy dng bng th m

(1) em cc t m xp thnh mt ct, theo th t chiu di ca t m t nh n ln, nh du l ct 1. (2) Trong ct ny, i chiu cc t m ngn vi cc t m di hn, nu t m ngn l tip u ng ca t m di th ghi tip v ng vo ct tip theo v nh du l ct 2. (3) Tip tc, i chiu cc chui trong ct 1 v ct 2 vi nhau, nu c chui no trong ct ny l tip u ng ca chui trong ct kia th tip v ng s c ghi vo ct tip theo l ct 3. (4) Tip tc theo khun mu ny nu ang xt ct th j th i chiu cc chui trong ct ny vi ct 1. Nu c chui no trong ct ny l tip u ng ca chui trong ct kia th tip v ng s c ghi vo ct j + 1. Thc hin cho n khi khng th in thm c na hoc ct mi thm vo trng vi mt ct trc hoc c mt chui trong ct mi trng vi mt t m.
Trang 88

Bng th m (tt)

V d

Lp bng th m cho b m nh ni trn A = {00, 01, 011, 1100, 00010} 2 3 010 0 1 4 0 5 0 1 M l khng phn tch c trn chui 000101100 v c hai cch phn tch khc nhau 00 | 01 | 011 | 00 00010 | 1100

1 00 01 011 1100 00010

100 1 11 11 0010 0010 100 00 10

So snh
Trang 89

Bng th m (tt)

V d: Xt b m {010, 0101, 10100} c bng th m nh sau:

1 010 0101 10100 1 2 3 0100 0 4 5 10 101 6 100 00 M l phn tch c

Bng th m (tt)

iu kin cn v mt b m phn tch c l khng c phn t no trong cc ct t j 2 trng vi mt phn t trong ct 1. chm gii m, thng k hiu l Tch, l s k hiu cn phi nhn c c th phn tch (nhn dng) c t m. Trong trng hp khng c chui no trong cc ct j 2 trng vi t m nhng c hai ct k, l no (k l, k, l 2 ) trng nhau th m l phn tch c nhng c chm gii m v hn.

chm gii m

Trang 91

Bng th m (tt)

Xt b m {01, 10, 011, 100} c bng th m nh sau: 1 01 10 011 100 1 0 2 3 0 00 1 11 1 11 0 00 4

Bng th m ny c cc ct 3 v 4 trng nhau v cc chui nn b m c chm gii m trong trng hp xu nht l v hn. Chng hn vi chui c dng sau y th trong qu trnh nhn cha ht chui chng ta khng th thc hin c vic tch m: 0110101010...
Trang 92

Bi tp

Hy lp bng th m cho nhng b m sau. Cho bit m c phn tch c khng, nu c th chm gii m (trong trng hp xu nht) l bao nhiu. X1 = {00, 01, 100, 1010, 1011} X2 = {00, 01, 101, 1010} X3 = {00, 01, 110, 111, 1100} X4 = {00, 01, 110, 111, 1110} X5 = {00, 01, 110, 111, 0111} X6 = {00, 01, 110, 111, 1011, 1101}

Trang 93

Bt ng thc Kraft

nh l 6.1

Cho l1, l2, ..., lK l cc chiu di ca mt b m prefix c bng k hiu m kch thc m (tc gm m k hiu m). Th
i =1 li m 1 K

Ngc li, nu cc s nguyn l1, l2, ..., lK tho bt ng thc trn th tn ti mt b m prefix vi cc t m c chiu di l l1, l2, ..., lK.

Chng minh
Chiu thun Gi T l cy m tng ng vi b m trn
Trang 94

Bt ng thc Kraft
M c 0 M c 1 M c 2 M c 3 G c

m-2 m-2 m-2 m-2 m-2 m-2 m-3 m-3 m-3 m-3 m-3 Nt l mc li s c gn trng s l m-li. Trng s ca mi nt cha c tnh bng tng trng s ca cc nt con. Vi cch gn ny, chng ta suy ra trng s ca nt cha mc h l m-h. iu ny ng l v mi nt cha mc h c ti a m nt con mc h + 1.
Trang 95

Bt ng thc Kraft (tt)

T y suy ra, trng s ca nt gc l 1. M trng s ca nt gc chnh l tng trng s ca cc nt l. Vy suy ra iu cn chng minh. Chiu o Chng ta chng minh bng cch xy dng mt cy m cho n. iu ny l thc hin c theo nh chng minh ca chiu thun.

V d

Tm b m prefix cho cc b m nh phn c cc chiu di t m tng ng nh sau. {2, 2, 3, 4, 4}, {2, 2, 3, 3, 3, 4, 4}, {2, 2, 3, 4, 4, 4, 5, 5}
Trang 96

nh l

nh l 6.2

Mt m phn tch c th c cc chiu di t m tho mn bt K ng thc Kraft. li m 1

i =1

Chng minh

Gi l1 l2 ... lK l cc chiu di t m vi c s l m. Vi s nguyn N bt k ta c th vit

li m i =1
K

L m
i1 =1 i N =1
Trang 97

l i1 + L + l i N

nh l 6.2 (tt)

Ch l i + L + l i l chiu di ca mt dy N t m v c th 1 N nhn gi tr bt k gia Nl1 v NlK. Gi Aj l s dy N t m m c tng chiu di l j. Th

li m = i =1
K

j = Nl1

j A m j

Nl K

V b m l phn tch c, nn cc dy N t m m c tng chiu di l j phi khc nhau. S cc dy c chiu di j ti a l mj. V vy Aj mj v

li m i =1
K

j = Nl1

j j m m = N (lK l1 ) + 1
Trang 98

Nl K

Chng minh nh l (tt)

N u

li m >1 i =1

Th vi N ln

m i =1
K li
K

s ln hn

N (l K l1 ) + 1

V vy chng ta c c iu cn chng minh.

li m 1 i =1

Kt hp hai nh l trn chng ta rt ra mt nhn xt sau. Nu mt m phn tch c th tn ti mt b m tng ng v chiu di cc t m m c tnh prefix.
Trang 99

Bi 7 M ha ti u ngun ri rc khng nh
7.1 Cc nh l v gii hn trn v di ca chiu di trung bnh 7.2 M ho theo Shannon v Fano 7.3 Phng php m ho ti u theo Huffman

Trang 100

Cc nh l v gii hn trn v di ca chiu di trung bnh

nh l 7.1

Chng minh

Cho ngun tin X = {a1, ..., aK} vi cc xc sut tng ng p1, ..., pK. Mt b m phn tch c bt k cho ngun ny vi c s m m, chiu di trung bnh t m s tho (trong H(X) l entropy ca ngun vi c s ca logarit l m). H (X ) l log m

m li H ( X ) l ln m = pi ln pi pi li ln m = pi ln pi i =1 i =1 i =1
K K K

m li K li pi p 1 = m 1 11 = 0 i =1 i i =1
K
Trang 101

Cc nh l v gii hn trn v di ca chiu di trung bnh (tt)

nh l 7.2

m li = 1 , tc l pi = m li Ch du = xy ra khi v ch khi pi Cho ngun tin X = {a1, ..., aK} vi cc xc sut tng ng p1, ..., pK, c th xy dng mt m prefix vi c s m sao cho H (X ) l< +1 log m Chn chiu di li ca t m cho tin ai theo qui tc li = log mi Chng ta c
p
li pi pi li = log m l log m pi i m

Chng minh

m
i =1
Trang 102

li

pi = 1
i =1

Chng minh nh l (tt)

V cc chiu di c chn ny tho bt ng thc Kraft nn tn ti mt m prefix tng ng c cc chiu di ny. Tip tc chng ta c
pi pi li = log m l < log i m +1

pl
i =1

i i

< pi log + pi

K pi log pi H (X ) = log m + 1 = log m + 1 i =1

i =1

pi m

i =1

iu ny hon tt chng minh ca chng ta.

Trang 103

H qu

C th m ho mt ngun m c chiu di trung bnh tip cn H (X ) n log m vi sai s nh tu . Chng ta thc hin iu ny bng cch m ho cc dy N tin ca ngun X = {a1, ..., aK} theo nh l 7.2. Lc ny chng ta c ngun mi vi kch thc l KN, mi phn t l mt dy ca N tin c ly c lp t ngun X. Entropy ca ngun mi ny l NH(X) v chiu di trung bnh cc t m ca n theo nh ngha s l N ln chiu di trung bnh cc t m ca ngun ban u, l . p dng nh l 7.1 v nh l 7.2 i vi ngun mi chng ta c
Trang 104

H qu (tt)

NH (X ) NH (X ) H (X ) H (X ) 1 Nl < +1 l < + log m log m log m log m N V N c th ln tu , nn l tip cn n H(X) / log m vi tc

p dng nh l 7.1 v nh l 7.2 i vi ngun mi ta c

tng ng vi 1/N tin n 0 khi N tin ra v cng. nh gi mt phng php m ho no l tt hay khng ngi ta a ra khi nim hiu sut lp m. Hiu sut lp m h c nh ngha bng t s ca entropy ca ngun vi chiu di trung bnh ca b m c lp

Hiu sut lp m

H (X ) h= l
Trang 105

M ha ti u

L php m ha m kt qu l mt b m c chiu di trung bnh l nh nht trong tt c cc php m ha c th c cho ngun. B m ca php m ha ti u cho ngun c gi l b m ti u. Ba php m ha: Shannon, Fano, Huffman. Trong mi php m ha chng ta s m ha vi c s m m = 2 trc (m ha nh phn), sau s m rng cho trng hp m > 2.

Trang 106

Phng php m ho Shannon

B1. Sp xp cc xc sut theo th t gim dn. Khng mt tng qut gi s p1 ... pK. B2. nh ngha q1 = 0, qi =

p
j =1

i 1

, i = 1, 2, ..., K.

B3. i qi sang c s 2, (biu din qi trong c s 2) s c mt chui nh phn B4. T m c gn cho ai l li k hiu ly t v tr sau du phy ca chui nh phn tng ng vi qi, trong li = log pi 2
Trang 107

V d

Hy m ho ngun S = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05.
i 1

Tin Xc sut q = q Biu din i j ai pi nh phn j =1 a1 a2 a3 a4 a5 a6

li = log2 pi
2 2 3 4 4 5
do di cua so chu so sau dau phay

T m wi 00 01 100 1100 1101 11110

0,3 0,25 0,2 0,12 0,08 0,05

0 0,3 0,55 0,75 0,87 0,95

0,00 0,01001... 0,10001... 0,11000... 0,11011... 0,111100...

H = 2.36, l = 2,75, h = 2,36/2,75 = 85,82%

Trang 108

Nhn xt - Bi tp

Phng php Shannon cho kt qu l mt m prefix. Phng php Shannon c th m rng cho trng hp m > 2 Hy m ho cc ngun sau bng phng php Shannon. Tnh entropy ca ngun, chiu di trung bnh v hiu sut ca php m ha. S1 = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,25; 0,21; 0,19; 0,16; 0,14; 0,05. S2 = {a1, a2, a3, a4, a5, a6 , a7, a8} vi cc xc sut ln lt l 0,21; 0,18; 0,15; 0,14; 0,12; 0,01; 0,06 ; 0,04. S3 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt l 0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.
Trang 109

Bi tp

Phng php m ho Fano

B1. Sp xp cc xc sut theo th t gim dn. Khng mt tng qut gi s p1 ... pK. B2. Phn cc xc sut thnh 2 nhm c tng xc sut gn bng nhau nht. B3. Gn cho hai nhm ln lt cc k hiu 0 v 1 (hoc ngc li). B4. Lp li bc 2 cho cc nhm con cho n khi khng th tip tc c na. B5. T m ng vi mi tin l chui bao gm cc k hiu theo th t ln lt c gn cho cc nhm c cha xc sut tng ng ca tin.

Trang 110

V d

Hy m ho ngun S = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05. Phn nhm ln Tin Xc sut 1 2 3 4 0 0 a1 0,3 0 1 a2 0,25 1 0 a3 0,2 1 1 0 a4 0,12 1 1 1 0 a5 0,08 1 1 1 1 a6 0,05 T m 00 01 10 110 1110 1111

H = 2.36, l = 2,38, h = 2,36/2,38 = 99,17%

Trang 111

Ch

Ch , trong nhiu trng hp c nhiu hn mt cch chia thnh cc nhm c tng xc sut gn bng nhau, ng vi mi cch chia c th s cho ra cc b m c chiu di trung bnh khc nhau. Hy m ho ngun S = {a1, a2, a3, a4, a5, a6, a7, a8} vi cc xc sut ln lt l 0,23; 0,2; 0,14; 0,12; 0,1; 0,09; 0,06; 0,06.

V d

Trang 112

V d
ai a1 a2 a3 a4 a5 a6 a7 a8

pi 0,23 0,2 0,14 0,12 0,1 0,09 0,06 0,06

1 0 0 1 1 1 1 1 1

2 0 1 0 0 1 1 1 1

3 4

0 1 0 0 1 1

0 1 0 1

wi 00 01 100 101 1100 1101 1110 1111

ai a1 a2 a3 a4 a5 a6 a7 a8

pi 0,23 0,2 0,14 0,12 0,1 0,09 0,06 0,06

1 0 0 0 1 1 1 1 1

2 0 1 1 0 0 1 1 1

wi 00 0 010 1 011 0 100 1 101 0 110 1 0 1110 1 1 1111

3 4

l 1 = 2,88,
Trang 113

l 2 = 2,89

Nhn xt - Bi tp

Nhn xt

Phng php Fano thng cho kt qu tt hn phng php Shannon. Hy m ho cc ngun sau bng phng php Fano. Tnh hiu sut ca php m ha. S1 = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,25; 0,21; 0,19; 0,16; 0,14; 0,05. S2 = {a1, a2, a3, a4, a5, a6 , a7, a8} vi cc xc sut ln lt l 0,21; 0,18; 0,15; 0,14; 0,12; 0,1; 0,06 ; 0,04. S3 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt l 0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.
Trang 114

Bi tp

Phng php m ho ti u Huffman

Trc ht xt c s m m = 2. Trng hp m > 2, chng ta s c mt s ch v s khc bit so vi trng hp m = 2. Cho ngun S = {a1, ..., aK} c cc xc sut ln lt l p1, ..., pK. Gi l1, ..., lK l chiu di cc t m tng ng vi b m ti u cho S. Nu pi > pj th li lj. Vi pi > pj, gi s li > lj. Xt b m mi bng cch hon i hai t m c chiu di li v lj cho nhau. Xt hiu chiu di trung bnh ca b m mi so vi b m c l = pilj + pjli pili pjlj = (pj pi)(li lj) < 0 iu ny mu thun vi nh ngha ca b m ti u.
Trang 115

Chng minh

Hai nh l ca Huffman

B ny tht s pht biu mt iu rng, m ho ti u cho mt ngun tin th tin c xc su cng ln phi c m ho thnh t m c chiu di cng nh. Trong b m ti u (m = 2) cho mt ngun tin, th hai t m tng ng vi hai tin c xc sut nh nht phi c chiu di bng nhau (lK1 = lK) v c th lm cho chng ch khc nhau duy nht bit cui (bit tn cng bn phi). Nu lK1 < lK th loi b bit cui cng ca t m wK chng ta c mt b m mi vn c tnh prefix nhng c chiu di trung bnh nh hn b m c.
Trang 116

nh l 7.3 (nh l s 1 ca Huffman)

Chng minh

Hai nh l ca Huffman (tt)

Gi s wK1 v wK khng tho iu kin l khc nhau ch bit cui. Nu c mt t m wi khc c chiu di bng lK ng thi khc t m wK ch bit cui th chng ta c th hon i wK1 v wi cho nhau, v vy nh l cng c chng minh. Nu khng tn ti mt t m wi nh vy th chng ta c th to ra mt b m mi bng cch b i bit cui ca t m wK. B m mi ny khng vi phm iu kin prefix v c chiu di trung bnh nh hn b m c. V vy nh l c chng minh.

Trang 117

Hai nh l ca Huffman (tt)

nh l 7.4 (nh l s 2 ca Huffman)

Xt mt ngun mi S = {a1, ..., aK1} vi s phn b xc sut l p1, ... , pK1 trong pi = pi vi 1 i K 2 cn pK1 = pK1 + pK. Nu {w1, ..., wK1} lm mt m ti u cho S th m nhn c theo qui tc sau l m ti u cho S. 1iK2 wi = wi, wK1 = wK10 wK = wK11 V lK = lK1 = 1 + lK1, nn l = p1l1 + ... + pKlK = p1l1 + ... + (pK1 + pK)(1 + lK1) = l ' + (pK1 + pK)
Trang 118

Chng minh

Hai nh l ca Huffman (tt)

S khc bit gia l v l ' l mt hng s. Nn nu m ti u cho ngun S l tt hn m theo qui tc pht biu th m c dn xut t m ti u ny bng cch b i hai t m wK v wK1 v thay vo t m m b i bit cui ca wK th s c mt m ti u tt hn cho ngun S, iu ny mu thun. Vy m nhn c cho S theo qui tc trn l ti u. nh l nh l 7.3 v 7.4 cho php qui bi ton tm m ti u cho ngun c K tin v bi ton tm m ti u cho ngun c K1 tin. V qu trnh ny c th c lp li cho n khi ch cn hai tin. Lc th m ti u l d thy.

Trang 119

Gii thut m ha Huffman

B1. Sp xp cc xc sut theo th t gim dn chng hn p1 ... pK B2. Gn 0 ti bit cui ca wK1 v 1 n bit cui ca wK hoc ngc li. Tuy nhin chng ta s qui c thc hin theo chiu th nht. B3. Kt hp pK v pK1 to thnh mt tp xc sut mi p1, ... , pK2, pK1 + pK B4. Lp li cc bc trn cho tp mi ny.

V d

Hy m ho ngun S = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05.
Trang 120

V d
ai a1 a2 a3 a4 a5 a6

0,55 0 00 0,3 0 0,45 1 01 0,2 0,2 0,25 0 0,25 1 11 0,12 0,13 0 0,2 1 101 0,08 0 0,12 1 1000 0,05 1 1001 0,3 0,25 0,3 0,25 0,45

pi

Ln 1 0,3 0,25

Ln 2

Ln 3

Ln 4

wi

H = 2.36, l = 2,38, h = 2,36/2,38 = 99,17%

Trang 121

Nhn xt

Nhn xt

So snh vi phng php Fano ta thy trong trng hp trn th c hai phng php cho hiu sut bng nhau. Tuy nhin trong trng hp tng qut phng php Fano khng phi l phng php m ha ti u. Trong trng hp nu xc sut pK1 + pK bng vi mt xc sut pi no th chng ta c th t pK1 + pK nm di hoc nm trn xc sut pi th kt qu chiu di t m trung bnh vn khng thay i cho d cc t m kt qu c th khc nhau.

Ch

Trang 122

M rng cho c s m > 2

Nu K m th vic m ho ti u l qu tm thng Gi s K > m, tn ti n sao cho: m + (n 1)(m 1) < K m + n(m 1). Chng ta s b sung vo mt s tin ph c xc sut bng 0 sao cho tng s tin ca ngun bng vi m + n(m 1). Sau th tc m ho trn c iu chnh nh sau B1. Sp xp cc xc sut theo th t gim dn chng hn p1 ... pK B2. Gn ln lt cc k hiu 0, 1, ..., m 1 ti cc bit cui ca m t m c xc sut nh nht B3. Kt hp m xc sut nh nht li thnh mt v to vi K m xc sut cn li thnh mt tp mi. B4. Lp li cc bc trn cho tp mi ny.

Trang 123

V d

Hy m ho ngun S = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05 vi m = 3. ai pi Ln 1 Ln 2 wi a1 0,3 0,3 0,45 0 1 0,25 0,3 1 2 a2 0,25 0,2 0 0,25 2 00 a 0,2
3

a4 a5 a6 a7

0,13 1 0,08 0 0,12 2 0,05 1 0,0 2 0,12

02 010 011

H = 1.49, l = 1,58, h = 1,49/1,58 = 94,24%

Trang 124

Bi tp

Hy m ho cc ngun sau bng phng php Huffman theo cc c s m = 2 v m = 3. Tnh hiu sut ca php m ha trong mi trng hp. S1 = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,25; 0,21; 0,19; 0,16; 0,14; 0,05. S2 = {a1, a2, a3, a4, a5, a6 , a7, a8} vi cc xc sut ln lt l 0,23; 0,2; 0,14; 0,12; 0,1; 0,09; 0,06 ; 0,06. S3 = {a1, a2, a3, a4, a5, a6 , a7, a8} vi cc xc sut ln lt l 0,21; 0,18; 0,15; 0,14; 0,12; 0,01; 0,06 ; 0,04. S4 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt l 0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.

Trang 125

Nhn xt

Xt ngun S = {a1, a2, a3, a4} c s phn b xc sut l {0,4; 0,25; 0,2; 0,15}. Xt ngun mi S2 = {aiaj, 1 i, j 4} c tp phn b xc sut l {0,16; 0,1; 0,08; 0,06; 0,1; 0,0625; 0,05; 0,0375; 0,08; 0,05; 0,04; 0,03; 0,06; 0,0375; 0,03; 0,0225}. H(S) = 1,9 v H(S2) = 2H(S) = 3,8. Hai bng sau y trnh by kt qu vic m ho ti u cho S v S2 theo Huffman. Vic m ho cho mt dy tin (hay khi tin) th cho hiu sut cao hn so vi vic m ho cho tng tin.

Nhn xt

Trang 126

Nhn xt (tt)
Tin a1 a2 a3 a4

pi 0,4 0,25 0,2 0,15

T m 1 01 000 001

l S = 1,95
h1 = 97,63% l S 2 = 3,8375 h2 = 99,26%

Tin a1a1 a1a2 a2a1 a1a3 a3a1 a2a2 a1a4 a4a1

pij 0,16 0,1 0,1 0,08 0,08 0,0625 0,06 0,06

T m 000 101 110 0010 0011 0110 0111 1000

Tin a2a3 a3a2 a3a3 a2a4 a4a2 a3a4 a4a3 a4a4

pij 0,05 0,05 0,04 0,0375 0,0375 0,03 0,03 0,0225

T m 1110 1111 01000 01001 01010 01011 10010 10011

Trang 127

Bi 8 M ha ngun ph qut
8.1 Ngun ri rc khng nh vi thng k khng bit trc 8.2 Cc vect tn xut v taentropy (quasientropy) 8.3 Mt s m ho ph qut cho ngun ri rc khng nh

Trang 128

Gii thiu

Vn ny khng c khi xng bi Shannon m bi B. M. Fitingof. L thuyt ca Shannon da trn kin thc v cc hm phn b xc sut v chng minh tn ti php m ho ti u. M ho ngun ph qut tip cn theo cch khc bng vic li dng cu trc ca cc dy v cng i n c cng kt qu ti u. Trong trng hp m cc hm phn b xc sut l khng c sn hoc s thng k v ngun l thay i theo thi gian, nhng iu m thng xy ra trong thc t, th k thut m ho ngun ph qut l mt la chn thch hp hn l dng k thut ca Shannon.
Trang 129

Ngun ri rc khng nh vi thng k khng bit trc

Xt ngun A = {a1, ..., aK} c s phn b xc sut l {p1, ..., pK} sinh ra mt dy cc k hiu c lp c phn b ng nht. Chng ta gi thit rng s phn b xc sut {p1, ..., pK} l c nh nhng khng c bit trc bi b m ho (encoder). Thc t s phn b xc sut thng l khng c bit trc hoc ch c bit mc gn ng, hoc s phn b ny thay i chm theo thi gian. V vy mt s m ho da trn xc sut c th hiu qu khung thi gian ny nhng s khng hiu qu khung thi gian khc.

Trang 130

Ngun ri rc khng nh vi thng k khng bit trc (tt)

nh gi nh hng ca s bit khng chnh xc v thng k ca ngun n hiu qu ca vic m ho Xt ngun ri rc khng nh nh phn vi s phn b xc sut l P(0) = p, P(1) = 1 p. Nu b m ho c cung cp xc sut gn ng vi p l p0 th theo phng php ca Shannon k hiu 0 s c gn vi t m c chiu di l log p0 cn 1 c gn vi t m c chiu di log (1 p0). Chiu di trung bnh ca cc t m l l = p log p0 (1p) log(1p0) Chiu di trung bnh t m ti u l l opt = p log p (1p) log(1p)
Trang 131

Ngun ri rc khng nh vi thng k khng bit trc (tt)

l

l opt

Ch rng l l mt tip tuyn ca l opt ti p = p0, nhng khi p lch ra xa p0 th khong cch gia hai th gia tng kh nhanh.

Trang 132

Ngun ri rc khng nh vi thng k khng bit trc (tt)

Trong bi ny chng ta pht trin cc tng c bn v m ho ph qut, mt s m ho khng da trn xc xut ca cc dy m li da vo cu trc ca chng. Chng ta s chng minh rng nguyn dng nh tu c kh nng m ho mt ngun sao cho l H(x) + i s phn b xc sut {p1, ..., pK} ca ngun. c th c lm nh Cn trn ca l tu bng cch chn chiu di khi tin cn m ho ln. l opt 0
Trang 133

Cc vect tn sut v ta-entropy

Xt cc dy ngun Si c chiu di N. C KN dy v ta gi tp KN dy ny l khng gian mu S. Chng ta k hiu Nki l s k hiu ak c trong dy Si v qki l tn sut ca ak trong Si qki = Nki / N Vect (q1i, ..., qKi) (k hiu l Q(Si) hay gn hn l Qi) c gi l vect tn sut ng vi chui Si. Gi cc qk (k = 1, ..., K) l cc bin ngu nhin trn S bng cch gn mi Si vi qki. Chng ta c b sau. Gi tr trung bnh ca qk chnh l xc sut pk ca ak.

E (q k ) = P(S i )qki = pk
i =1
Trang 134

KN

Cc vect tn sut v ta-entropy (tt)

Chng minh

nh ngha bin ngu nhin xk(n) bng 1/N nu ngun sinh ra k hiu ak ti v tr th n ca dy v bng 0 nu ngc li. V ngun l khng nh, dy xk(1), ..., xk(N) l c lp v c phn b ng nht. N (n) Gi tr trung bnh ca xk(n) bng pk/N n. M q k = x k N n =1 V vy (n)

E (q k ) = E (x k ) = pk
n =1

Mi dy Si c tng ng mt vect tn sut Qi, nhng ngc li vi mt vect Q = (q1, ..., qK) c th tng ng vi nhiu dy Si.
Trang 135

Cc vect tn sut v ta-entropy (tt)

Gi (Q) l s cc dy Si m c cng vect tn sut Q (tc l nhng dy m c s ln xut hin ca mi ak trong dy bng nhau v bng Nk = Nqk k = 1, ..., K).

(Q) =

N!

K k =1

Nk !

Gi (K, N) l s vect biu din cho cc dy ngun c chiu di N. Con s ny c th din t thnh mt bi ton tp hp tng ng kh quen thuc l: C bao nhiu b gm K s nguyn khng m m c tng bng N.

N + K 1 ( K , N ) = N
Trang 136

Cc vect tn sut v ta-entropy (tt)

Chng minh

Xt mt hng gm N + K 1 khong trng. Dng N i tng ging nhau lp vo N khong trng bt k. K 1 khong trng cn li s chia N i tng ny thnh K nhm. Do ng vi mi cch lp N i tng vo N + K 1 v tr chng ta c mt tng tng ng. V vy s lng tng ny bng N + K 1 N Vi mi dy Si chng ta c tng ng mt vect Qi = (q1i, ..., qKi). Chng ta nh ngha mt bin ngu nhin (Q) gn mi dy Si vi gi tr (k hiu l (Si) hoc (Qi))

k =1

qki log qki

Trang 137

Cc vect tn sut v ta-entropy (tt)

Ch Qi l mt vect xc sut v (Qi) c cng thc ging nh ca entropy nn chng ta gi (Qi) l taentropy. D nhin (Qi) c tt c cc tnh cht ca hm entropy H(Qi) ci m ch ph thuc duy nht vo Qi. Chng ta c nh l sau thit lp mi quan h gia (Q) (hay vit r ra l (q1, ..., qK)) vi entropy ca ngun H(p1, ..., pK). E((Q)) H(p1, ..., pK)

nh l 8.1 Chng minh

K K E ((Q)) = E q k log q k = E ( q k log q k ) k =1 k =1
Trang 138

Cc vect tn sut v ta-entropy (tt)

M hm x log x l hm li, v vy theo bt ng thc Jensen chng ta c E(qk logqk) E(qk) log E(qk) Theo mt b trc y chng ta c E(qk) = pk. V vy

E ((Q )) = pk log pk = H ( p1 ,..., p K )

k =1

Trang 139

Mt s m ho ph qut cho ngun ri rc khng nh

Mt t m cho mt dy Si gm hai phn: phn u l chui m ho cho vect tun sut Qi tng ng ca dy Si, phn th hai l chui m ho cho dy Si trong s cc dy c cng vect Qi. V tng cc vect tn sut khc nhau l (K, N), nn s bit dng biu din cho phn u l Tng t s bit biu din cho phn th hai l

log ( K , N )

log (Qi )

V vy t m biu din cho dy Si c chiu di l l(Si) = log ( K , N ) + log (Qi ) < log (K, N) + log (Qi) + 2.
Trang 140

Mt s m ho ph qut cho ngun ri rc khng nh (tt)

Chng ta chng minh c gi tr trung bnh ca l(Si) tho N K 1 ) + ( K 1) log(1 + ) E (l ( S i )) < NH ( p1 ,..., p K ) + N log(1 + N K 1 Suy ra chiu di trung bnh trn mt k t ngun tho

E (l ( S i )) K 1 K 1 N < H ( p1 ,..., p K ) + log(1 + )+ log(1 + ) l= N N N K 1 Ch thnh phn nm trong du mc vung tin n 0 khi N vi tc bng vi tc ca log N 0 N iu ny ni ln rng phng php ny tip cn n entropy ca ngun chm hn so vi cc phng php m bit trc xc sut. iu ny cng d hiu v cng l ci gi phi tr nu chng ta khng bit trc xc sut.
Trang 141

V d

Bng sau y m t vic m ho ph qut cho mt ngun nh phn cho tng khi c chiu di 7. C (2, 7) = 8 vect tn sut v v vy cn dng 3 bit m ho 8 vect ny; 3 bit ny s l 3 bit u ca mi t m. Cc bit cn li dng nhn bit mi dy trong lp cho (l lp cc dy c cng vect tn sut). Qi (0/7,7/7) (1/7,6/7)

(Qi) 1
7

Si 1111111 0111111 1011111 ... 1111110

Trang 142

(Si)
0 0,592

wi 000 001 000 001 001 001 110

V d (tt)
Qi (2/7,5/7)

(Qi)
21

Si 0011111 0101111 ... 1111100 0001111 0010111 ... 1111000 0000111 0001011 ... 1110000
Trang 143

(Si)
0,863

wi 010 00000 010 00001 ... 010 10100 011 000000 011 000001 011 100010

(3/7,4/7)

35

0,985

(4/7,3/7)

35

0,985

100 000000 100 000001 100 100010

V d (tt)
Qi (5/7,2/7)

(Qi)
21

Si 0000011 0000101 ... 1100000 0000001 0000010 ... 1000000 0000000

(Si)
0,863

wi 101 00000 101 00001 ... 101 10100 110 000 110 001 110 110

(6/7,1/7) (7/7,0/7)

7 1

0,592 0

111

Trang 144

Bi 9 Knh ri rc khng nh Lng tin tng h

9.1 Knh ri rc khng nh v ma trn knh 9.2 Entropy iu kin v lng tin tng h 9.3 Mt s loi knh 9.4 S nhp nhng (equivocation) v tc truyn tin 9.5 Dung lng knh

Trang 145

Knh ri rc khng nh v ma trn knh

nh ngha Mt knh ri rc khng nh (DMC) c nh ngha bng mt bng k hiu u vo (ngun pht) X = {x1, ..., xK}, mt bng k hiu u ra (ngun nhn) Y = {y1, ..., yJ}, v mt s phn b xc sut c iu kin p(yj | xk), vi 1 k K, 1 j J.
X

p(yj | xk)

xk

yj

Bng k hiu u ra khng nht thit ging bng k hiu u vo. iu ny c ngha l bn nhn c th nhn nhng k hiu m khng ging vi nhng k hiu m bn pht pht i.
Trang 146

Nhn xt

Thut ng khng nh (memoryless) suy ra rng

vi N bt k. n =1 Mt knh ri rc khng nh thng c biu din di dng mt ma trn knh [p(yj | xk)] c kch thc K J. y1 y2 x1 p(y1 | x1) p(y2 | x1) x2 p(y1 | x2) p(y2 | x2) ... ... ... xK p(y1 | xK) p(y2 | xK)
Trang 147

p{ y j1 K y jN | x k1 L x kN } =

p( y jn | xkn )
yJ p(yJ | x1) p(yJ | x2) ... p(yJ | xK)

... ... ... ...

Nhn xt (tt)

Chng ta thy, ma trn knh chnh l ci m biu din tnh cht tp nhiu ca knh truyn. Ch , nu chng ta bit s phn b xc sut trn X th s phn b xc sut ca Y s c xc nh nh sau

p( y j ) =

k =1

p( xk ) p( y j | xk )

Trang 148

Entropy iu kin v lng tin tng h

Xt bi ton truyn tin sau Cho bit cu trc thng k ca ngun X v ma trn knh. Hy xc nh k hiu xk no c pht pht i khi nhn c u nhn mt k hiu yj no ? Cho ngun X = {x1, x2} vi cc xc sut ln lt l p(x1) = 1/4, p(x2) = 3/4, ngun Y = {y1, y2} v ma trn knh l y1 y2 x1 4/5 1/5 x2 2/5 3/5

V d

Nu nhn c y1 th xk no c kh nng c pht i?

Trang 149

V d
p( xk | y j ) = p( xk , y j ) p( y j ) = p( xk ) p( y j | xk )
i =1

p ( xi , y j )

p( xk ) p( y j | xk )
i =1

p ( xi ) p ( y j | xi )
3 p ( x 2 | y1 ) = 5

p ( x1 ) p ( y1 | x1 ) p ( x1 | y1 ) = p ( x1 ) p( y1 | x1 ) + p ( x 2 ) p ( y1 | x 2 ) (1 / 4) (4 / 5) 2 = = (1 / 4) (4 / 5) + (3 / 4) (2 / 5) 5

p(x1 | y1) < p(x2 | y1), nh vy chng ta c th khng nh c k hiu x2 c kh nng c pht i hn x1?
Trang 150

V d (tt)

, trong cng thc ca p(xi | yj) c cha tha s p(xi), nn p(xi | yj) b nh hng bi xc sut l p(xi). V vy cng bng trong vic so snh chng ta phi da trn t s p(xi | yj)/p(xi) ci m khng b nh hng nhiu bi p(xi). p ( x1 | y1 ) 2 / 5 8 = = p ( x1 ) 1/ 4 5

p ( x 2 | y1 ) 3 / 5 4 = = p( x2 ) 3/ 4 5

Nh vy thc s k hiu x1 mi c kh nng c pht i hn k hiu x2. T xc sut iu kin chng ta gii thiu khi nim lng tin c iu kin.
Trang 151

Lng tin c iu kin I(xk | yj)

nh ngha
I(yj | xk) = log p(yj | xk) I(xk | yj) = log p(xk | yj) p(yj | xk) 1 th I(yj | xk) 0 v ngc li. Nu khi pht i xk v bit chc yj s nhn c th pha nhn chng ta khng cn tn thm thng tin g gii thch. Nu p(yj | xk) = 1/2 (I(yj | xk) = 1 bit) th khi pht i xk bn nhn s c hai kh nng v yj ch l mt trong hai kh nng , c ngha l bn nhn cn thm thng tin (cn thm 1 bit) bit chnh xc l kh nng no. Xc sut p(yj | xk) = 1/2 ch xy ra khi knh truyn c nhiu.
Trang 152

Lng tin c iu kin I(xk | yj)

V vy lng tin c iu kin cn c gi l lng tin b mt i do nhiu. Khi pht i xk bn nhn s c mt tp cc yj c kh nng c nhn. Ngc li khi nhn c yj bn pht s c mt tp cc xk c kh nng c pht. o mc quan h gia xk vi yj chng ta gii thiu khi nim lng tin tng h.

Trang 153

Lng tin tng h

nh ngha

Lng tin tng h gia hai tin l lng tin ca ca tin ny c cha trong tin kia v ngc li. Bng cng thc Lng tin tng h = Lng tin ring Lng tin b mt i I(xk, yj) = I(xk) I(xk | yj) = I(yj) I(xk | yj) p( x k |y j ) p( y j|x k ) = log = log p( x k ) p( y j ) Nu p(xk | yj) = 1 c ngha l nu yj nhn c th chc chn xk c pht i, iu ny c ngha l lng tin ca xk c truyn nguyn vn thng qua knh, do I(xk, yj) = I(xk).

Trang 154

Lng tin c iu kin trung bnh

I(X | y j ) =
I (Y | xk ) = I(X | Y) =
k =1
J

p(xk | y j )I (xk | y j ) = p(xk | y j ) log p(xk | y j )

k =1
J j =1 J j =1

p( y j | xk )I ( y j | xk ) = p( y j | xk ) log p( y j | xk ) p( y j )I ( X | y j ) = p( y j ) p(xk | y j ) log p(xk | y j )

j =1 K j =1 k =1 J K

= p( xk , y j ) log p( xk | y j )
I (Y | X ) = p( xk , y j ) log p( y j | xk )
k =1 j =1
Trang 155

k = 1 j =1 K J

Entropy iu kin

nh ngha

Xt hai bin ngu nhin x v y vi phn b xc sut ng thi p(xk, yj), k = 1, ..., K , j = 1, ..., J. Entropy iu kin ca x cho y c nh ngha l
H (x | y ) = p( xk , y j ) log p( xk | y j )
k =1 j =1 K J

H(x | y) c th c din dch nh l bt ng trung bnh v x sau khi bit y. H(x | y) H(x), du = xy ra x v y l c lp.

nh l 9.1

Trang 156

Chng minh
H (x | y) H (x) = p(xk , y j ) ln p(xk | y j ) + p(xk ) ln p(xk )
K J K k =1 j =1 K J k =1

Ly tng trn nhng cp (k, j) m p(xk, yj) 0. V vy p( xk ) 1 H (x | y) H (x) = p( xk , y j ) k j p( xk | y j )

p(xk ) = p(xk , y j ) ln p(xk | y j ) k =1 j =1

p( xk ) p( y j ) p( xk , y j )
k j

= p( xk ) p( y j ) 1 0
k j
Trang 157

Chng minh (tt)

Du = xy ra p(xk) = p(xk | yj) i vi tt c cc cp (k, j) m p(xk, yj) 0 ng thi tng p(xk)p(yj) trn tt c nhng cp ny bng 1. iu kin th hai tng ng vi iu kin p(xk)p(yj) = 0 bt k khi no m p(xk, yj) = 0. C hai iu kin ny cui cng tng ng vi iu kin l x v y c lp. H(x, y) = H(x) + H(y | x) = H(y) + H(x | y)

nh l 9.2

Trang 158

Chng minh
H (x, y) = p( xk , y j ) log p( xk , y j )
= p( xk , y j ) log p( xk ) + log p( y j | xk )
k j

= p( xk )[log p( xk )] p( y j , y k ) log p( y j | xk )
k k j

= H ( x) + H ( y | x)

Phn th hai chng minh hon ton tng t. Kt hp hai nh l trn chng ta suy ra rng H(x, y) H(x) + H(y) du = xy ra x, y l c lp.
Trang 159

Lng tin tng h trung bnh

I ( X , Y ) = p( xk , y j ) I ( xk , y j )
p( xk | y j ) = p ( x k , y j ) log p( xk ) k j
k j

p( y j | xk ) = p ( xk , y j ) log p( y j ) k j p( xk , y j ) = p ( xk , y j ) log p( xk ) p( y j ) k j

Nu biu din theo entropy th chng ta c I(x, y) = H(x) H(x | y) = H(y) H(y | x)

Trang 160

Mt s loi knh ri rc khng nh

Knh i xng (Symmetric channel)

L knh m mi dng ca ma trn knh cha cng tp cc s p1, ..., pJ v mi ct cha cng tp cc s q1, ..., qK.
j=1 [p(yj | xk)] = 0,2 0,3 0,2 [p(yj | xk)] = 0,3 0,5 [p(yj | xk)] =
Trang 161

V d

2 0,2 0,3 0,3 0,5 0,2 1

3 0,3 0,2 0,5 0,2 0,3

4 0,3 k = 1 0,2 k = 2

Cc ma trn biu din cc knh i xng Knh i xng nh phn (binary symmetric channel BSC).

01

Nhn xt

Knh i xng th H(y | x) c lp vi s phn b xc sut ca ngun pht v c xc nh duy nht bng ma trn knh.

Chng minh

H (y | x) = p ( x k , y j ) log p ( y j | x k ) = p ( x k ) p ( y j | x k ) log p ( y j | x k )
k =1 j =1 K k =1 K J

= p ( x k ) p 'j log p 'j = p 'j log p 'j

j =1
Trang 162

j =1 J

k =1 J

j =1

Knh khng mt (Lossless channel)

Cnh ni gia xk v yj ngha l p(yj | xk) 0. Trong knh khng mt u ra xc nh duy nht u vo, v vy H(x | y) = 0. x1 xK x1 y1 y2 ym xm ym+1 xm+1 yJ xK

Knh n nh (Deterministic channel)

x1 x2

y1 y2 yJ Trong knh ny u vo xc nh duy nht u ra, v vy H(y | x) = 0

Trang 163

Knh v dng (Useless channel)

Mt knh l v dng nu v ch nu x v y l c lp vi mi s phn b xc sut ca u vo (ngun pht). i vi mt knh v dng th H(x | y) = H(x), tc l kin thc v u ra khng lm gim bt ng v u vo. V vy, i vi mc ch xc nh n nh u vo, chng ta c th pht l u ra hon ton. By gi chng ta s chng minh rng. Mt knh ri rc khng nh l v dng nu v ch nu ma trn knh ca n c cc dng ging nhau. iu kin Gi s ma trn c cc dng ging nhau p1, ..., pJ. Th i vi mi u ra yj
Trang 164

Chng minh

Knh v dng (tt)

p( y j ) =
k =1

p( xk , y j ) =

k =1

' p ( x ) p ( y | x ) = p k j k j

k =1

' p ( x ) = p k j

i vi mi cp u vo ra (xk, yj), chng ta c p(xk, yj) = p(xk) p(yj | xk) = p(xk) pj = p(xk) p(yj) V vy u vo v u ra c lp nhau bt chp s phn b xc sut ca u vo. iu kin cn Gi s cc dng ca ma trn khng ging nhau mt ct chng hn j0 m c cc phn t khng ging nhau. Gi s p(yj0 | xk0) l phn t ln nht trong ct ny. Xt s phn b ng nht (ng xc sut) u vo (u pht), chng ta c
Trang 165

Knh v dng (tt)

1 K p( y j 0 ) = p( xk ) p( y j 0 | xk ) = p( y j 0 | xk ) < p( y j 0 | xk 0 ) K k =1 k =1

Tc l p(yj0) p(yj0 | xk0). V vy p(xk0, yj0) = p(xk0) p(yj0 | xk0) p(xk0) p(yj0). Mu thun vi gi thit l x, y c lp vi mi s phn b xc sut ca u vo.

Trang 166

S nhp nhng (equivocation) v tc truyn tin

Xt mt knh nh phn i xng vi xc sut cho . Gi s rng ti u vo P(0) = P(1) = 1/2, tc sinh thng tin u pht l H(x) = 1 bit/k hiu. Mt thit b c gi l b quan st, nhn mi cp k hiu vo/ra (x, y) v sinh ra mt k hiu z z = 0 nu x = y, z = 1 nu x y B quan st Knh z(2)z(1)

x(2)x(1)

y(2)y(1)

Trang 167

S nhp nhng (equivocation) v tc truyn tin (tt)

S phn b ca z c tm thy nh sau: P(z = 1) = P(x = 0) P(y = 1 | x = 0) + P(x = 1) P(y = 0 | x = 1) = /2 + /2 = P(z = 0) = 1 P(z = 0) = 1 Tc sinh thng tin bi b quan st v vy bng H(z) = log (1 ) log(1 ) bits/k hiu i vi mt dy u ra cho y(1)y(2)..., ni nhn (receiver) c th xy dng li chnh xc dy u vo x(1)x(2)... ch khi u ra ca b quan st z(1)z(2)... c to sn. Tc truyn thng tin trn knh, thng k hiu l R, l bng tc sinh thng tin H(x) tr tc sinh thng tin b sung H(z). R = H(x) H(z)
Trang 168

V d

Chng hn, nu d liu u vo c sinh tc 1000 k hiu/giy v = 0,01, chng ta c H(x) = 1 tc d liu u vo = 1000 bits/giy H(z) = 0,081 tc d liu b sung = 81 bits/giy R = 0,919 tc truyn thng tin = 919 bits/giy Mt ngi c th l lun rng trong mt dy di, v = 0,01, ngha l ch 1% s bit c truyn b li, v v vy tc truyn thng tin phi l 990 bits/giy. Cu tr li l rng kin thc v s bit b li khng xy dng li d liu, m chng ta cn phi bit thm v v tr li na, v v l do ny nn tc truyn thng tin l thc s bng mt gi tr thp hn l 919 bits/giy.

Trang 169

Nhn xt

Trong trng hp tt nht = 0, chng ta c H(z) = 0 v v vy R = 1000 bits/giy. Trong mt trng hp khc nu = 1/2, th H(z) = 1, kt qu l R = 0 bits/giy. C hai kt lun l nht qun vi s mong i ca chng ta. i vi knh nh phn i xng vi u vo ng xc sut, chng ta chng minh c rng H(z) = H(x | y). Tng qut chng ta chng minh c rng S ti xy dng chnh xc dy u vo t dy u ra l c th nu b quan st c th sinh ra thng tin b sung tc ln hn hay bng H(x | y).

Trang 170

Nhn xt (tt)

thy iu ny mt cch trc quan, quan st rng i vi cc dy di c chiu di N c khong 2NH(x | y) dy u vo c th sinh ra mt dy u ra c th. Ch khi thng tin b sung c sinh ra ti tc H(x | y) hay nhanh hn mi cho php phn bit gia cc kh nng ny. i vi l do ny, H(x | y) thng c coi nh l s nhp nhng (equivocation) ca knh. V chng ta nh ngha li tc truyn thng tin trn knh l R = H(x) H(x | y) = I(x, y)

Trang 171

Dung lng knh

Theo phn trn tc truyn tin trn knh c nh ngha l R = H(x) H(x | y) = I(x, y) I(x, y) tng qut l mt hm ca s phn b xc sut u vo {p1, , pK}. V vy, c th tm thy mt s phn b m cc i I(x, y). Gi tr cc i ca I(x, y) c nh ngha l dung lng knh C v l mt hm ca ma trn knh. C = Cc i (trn cc s phn b xc sut u vo) ca I(x, y). Tng qut, vic tnh dung lng knh l mt bi ton kh v l mt bi ton cha c gii mt cch trit . Tuy nhin i vi cc knh c gii thiu trn C c th tnh ton c nh phn sau y trnh by.

Trang 172

Knh i xng
C = log J +
' ' p log p j j j =1 J

trong p1, , pJ l cc phn t ca cc hng ca ma trn. Trong trng hp knh nh phn i xng vi xc sut cho l p chng ta c C = 1 H(p) vi H(p) = plogp (1p)log(1p) H(x | y) = 0, v vy C = Max {H(x) H(x | y)} = Max{H(x)} = log K trong K l kch thc ca bng k hiu u vo. Dung lng t c trong trng hp u vo c s phn b ng xc sut.
Trang 173

Knh khng mt

Knh n nh

y H(y | x) = 0, v vy C = Max {H(y) H(y | x)} = Max{H(y)} = log J trong J l kch thc ca bng k hiu u ra. y H(x | y) = H(x), v vy C = Max {H(x) H(x | y)} = Max{H(x) H(x)} = 0 Mt knh v dng th c dung lng knh bng 0.

Knh v dng

Trang 174

Bi 10 M ha chng nhiu, nh l knh

0.1 Gii thiu bi ton chng nhiu 1 10.2 nh l knh c nhiu cho knh nh phn i xng ri rc (BSC) 10.3 nh l ngc ca knh truyn c nhiu

Trang 175

Gii thiu bi ton chng nhiu

Mc tiu chng nhiu l bn nhn c th on (gii m) c cng chnh xc cng tt dy k hiu c pht. Chng hn xt ngun nh phn i xng vi xc sut cho , ng thi gi s ngun pht ng xc sut, tc P(0) = P(1) = 1/2. Vi < 1/2, xt c ch gii m bn nhn nh sau: Nu y = 0 th on x = 0 v nu y = 1 th on x = 1. Xc sut gii m b li ca c ch ny l P(li) = P(y = 0) P(x = 1 | y = 0) + P(y = 1) P(x = 0 | y = 1) = /2 + /2 = . Ch trong trng hp y chng ta tnh c P(y = 0) = P(y = 1) = 1/2 v P(x y | y) = . Vn quan trng l c th gim c xc sut gii m b li hay khng?
Trang 176

Gii thiu bi ton chng nhiu (tt)

Mt hng gii quyt nh sau: gi 0 chng ta gi chui 3 k hiu 0 v tng t gi 1 chng ta gi 3 k hiu 1. C ch gii m ca bn nhn nh sau: Nu chui nhn c nhiu k hiu 0 hn 1 th gii m thnh 0 v ngc li. Chng hn bn nhn nu nhn c 010 th gii m thnh 0 cn nu nhn c 110 th gii m thnh 1. C ch ny c xc sut gii m b li l P(li) = 3(1 )2 + 3 < Xc sut ny nh hn . Tuy nhin hiu sut truyn thng tin b gim xung 3 ln. Tng t nu mun xc sut gii m tin n 0 chng ta s m ho 0 thnh dy 2n + 1 k hiu 0 v m ho 1 thnh 2n + 1 k hiu 1, nhng tng ng lc ny hiu sut truyn thng tin gim xung 2n + 1 ln so vi ban u.
Trang 177

Gii thiu bi ton chng nhiu (tt)

C mt cch c th gim xc sut gii m li xung gn bng 0 nhng khng gim hiu sut truyn thng tin xung gn bng 0 m ch cn nh hn mt ngng no l . Ngng chnh l dung lng knh. Cch ny cng khai thc tng trn ch thay v gi i 0 v 1, ci m c khong cch Hamming gia chng l 1 th chng ta s m ho ln lt thnh 000 v 111, ci m c khong cch Hamming gia chng l 3 v v vy gim xc sut gii m b li.

Trang 178

nh l knh c nhiu cho knh nh phn i xng ri rc (BSC)

Xt knh nh phn i xng vi xc sut cho p. Dung lng knh trong n v bits/k hiu l C = 1 H(p) vi H(p) = plogp (1p)log(1p) Gi s thi gian truyn 1 k hiu l T, s k hiu c truyn trong 1 giy l 1/T, th dung lng theo n v bits/giy l C = [1 H(p)]/T Xt ngun X c entropy H(X) bits/k hiu, tc l ngun ny to ra thng tin tc theo n v bits/giy. R = H(X)/T Chng no m R (bits/giy) cn nh hn C (bits/giy), th vic truyn trn knh vi t l li nh tu l c th thc hin c. chng minh nh l ny cn mt s khi nim sau.
Trang 179

nh l 10.1.

Cc khi nim

Trng s Hamming

Trng s Hamming ca mt dy k hiu v = a1a2...an , trong mi ai {0, 1, ..., m1}, l s k hiu khc 0 ca dy, v thng c k hiu l w(v). Khong cch Hamming ca hai dy k hiu v1, v2 vi chiu di bng nhau l s v tr khc nhau ca hai dy, v thng c k hiu l d(v1, v2). Xt a, b {0, 1, ..., m1} th a b = (a + b) mod m. Nu v1 = a1a2...an, v2 = b1b2...bn th v1 v2 = c1c2...cn trong ci = ai bi vi i = 1, 2, ..., n.

Khong cch Hamming

Php cng c s m,

Trang 180

Cc khi nim (tt)

V d w(10100) = 2, w(01120) = 3.

d(10100, 10001) = 2, d(011010, 101101) = 5. Vi m = 2 th 1011 1101 = 0110. Vi m = 3 th 1021 2120 = 0111. d(v1, v2) = w(v1 v2 ) d(v1, v2) + d(v2, v3) d(v1, v3)

B Nhn xt

Bt ng thc th hai c dng ca bt ng thc tam gic: tng hai cnh ca mt tam gic ln hn hoc bng cnh cn li. nh l 10.1 ng cho knh ri rc khng nh bt k. Tuy nhin y chng ta ch chng minh cho knh nh phn i xng ri rc.
Trang 181

Chng minh nh l

tng chng minh l m ho cc dy d liu thnh cc t m, trong cc k hiu m ly t bng k hiu u vo ca knh v x l cc t m ny nh cc u vo c bn ca knh. Xc sut li nh tu c th t c da trn s m ho nh sau: (1) chn chiu di N ca dy d liu di (2) m ho cc dy ny thnh cc t m c khong cch Hamming xa nhau. Nguyn tc gii m u ra c thit k nh sau: dy k hiu nhn c u ra vj s c gii m thnh t m wi m c khong cch Hamming nh nht i vi vj. Vi cch chn ny xc sut gii m li l nh nht. Tht vy p(wi | vj) = p(wi)p(vj | wi)/p(vj)

Trang 182

Chng minh nh l (tt)

Do khi chng ta khng r v p(wi) v d nhin s ko theo p(vj) th p(wi | vj) ln nht khi p(vj | wi) l ln nht. M p(vj | wi) = pD(1p)ND trong D l khong cch Hamming gia vj v wi, N l chiu di ca chng, p l xc sut cho. Nu xc sut cho p < 0,5 th p(vj | wi) s ln nht khi D l nh nht. Chng minh rng > 0 nh tu , vi N ln tn ti cch m ho cc dy d liu thnh cc t m sao cho vi nguyn tc gii m trn c xc sut gii m li l nh hn . Tht vy s dy d liu c chiu di N l vo khong M = 2NH(X) = 2NRT trong khi tng s dy c chiu di N l 2N.

Trang 183

Chng minh nh l (tt)

i {w1, w2, , wM} l mt tp t m bt k, Pe l xc sut gii G m li i vi tp ny. Nu chng ta chng minh c rng > 0 nh tu , vi N ln gi tr trung bnh ca Pe, Pe , nh hn th s tn ti mt tp t m m c xc sut gii m li Pe nh hn . Vi xc sut li trn ng truyn l p, mt dy c chiu di N s c trung bnh Np v tr li. Vi hai s dng , nh tu , theo lut yu ca s ln vi N ln th xc sut s v tr ca chui nhn vj khc vi chui pht wi ln hn N(p + ) l nh hn . Hay ni theo ng cnh ca khong cch Hamming l P[d(wi, vj) > N(p + )] < V vy b m m chng ta mong mun s nh sau: Khong cch Hamming gia hai t m bt k l 2N(p + ) + 1
Trang 184

Chng minh nh l (tt)

Nh vy vi mi vj nhn c theo bt ng thc tam gic tn ti mt t m wi m c d(wi, vj) N(p + ) cn cc t m wk khc c d(wk, vj) N(p + ) + 1 V vy chng ta s gii m c duy nht vj thnh wi. Vi tng ny, chng ta s a ra c ch gii m lng hn cho mt tp t m bt k {w1, w2, , wM}, nhng cng s m bo xc sut gii m li l nh hn . Vi mi dy vj nhn c, nh ngha mt tp kim tra Aj bao gm tt c nhng dy c chiu di N v c khong cch Hamming so vi vj nh hn hay bng N(p + ). Nu t m c truyn wi l t m duy nht thuc tp Aj th gii m vj thnh wi. Ngc li thng bo mt li xy ra.
Trang 185

Chng minh nh l (tt)

Mt li xy ra thuc vo mt trong hai trng hp sau y (1) T m c truyn wi khng thuc Aj, tc l d(wi, vj) > N(p + ) Li ny xy ra vi xc sut nh hn . (2) Tn ti mt t m wk khc cng thuc Aj. Lc ny chng ta khng bit nn gii m vj thnh wi hay wk. Chng ta chng minh rng theo cch ny xc sut gii m li trung bnh s nh hn vi nh tu cho trc. Chng ta c

Pe +

i =1 i j

P(wi A j )

Trang 186

Chng minh nh l (tt)

tnh Pe chng ta s tnh gi tr trung bnh ca P(wi Aj). Gi tr trung bnh ny s bng s dy thuc tp Aj chia cho tng s dy
N ( p + )

Suy ra

P (Wi A j ) =

N k k =0 2N

N k k =0 Pe < + ( M 1) N 2
Trang 187

N ( p + )

Chng minh nh l (tt)

k =0 vi H() = log (1)log(1). p dng vo bt ng thc trn chng ta c Pe + M2N [1 H(p + )] = + 2NRT 2N [1 H(p + )] = + 2N [1 H(p + ) RT] V v c th nh tu , nn chng no R < [1 H(p)]/T = C (bits/giy) th c th lm cho Penh tu bng cch tng N. Chng minh c hon tt.

M chng ta c mt bt ng thc ni ting sau N N NH ( )

k2

Trang 188

V d

Xt v d trc y, mt knh i xng nh phn c xc sut cho = 0,01. Tc truyn k hiu f = 1000 k hiu/giy (tc T = 0,001 giy). Chng ta c C = 0,919 bits/k hiu hay C = 919 bits/giy. nh l knh cho php chng ta kt lun, vi xc sut ng tin ti 1, rng vi N kh ln chng hn N = 1000, th trong 21000 dy c chiu di 1000 chng ta c th chn c 2K dy vi K < 919 sao cho khong cch Hamming gia cc dy l 2N + 1 = 21. Khong cch Hamming ca mt b m A, vi iu kin A l m u, k hiu l d(A), l khong cch Hamming nh nht trong tt c cc khong cch gia hai t m bt k ca A.
Trang 189

Khong cch Hamming ca b m

nh l

nh l 10.2

Mt b m nh phn c khong cch Hamming d th c th Pht hin sai c t bit nu d t + 1. Sa sai c t bit nu d 2t + 1. Gi wi l t m pht, vi l dy nhn tng ng. Nu sai ti a t > 0 bit th d(wi, vi) t. Do t hp nhn s khng th trng vi bt k t m no v khong cch Hamming gia hai t m bt k l t + 1. V vy bn nhn c th pht hin c sai. Tng t nu d(wi, wj) 2t + 1, theo bt ng thc tam gic d(wj, vi) t + 1 t m wj wi. V vy bn nhn c th gii m ng vi thnh wi da trn s khc bit ny.
Trang 190

Chng minh

nh l ngc ca knh truyn c nhiu

nh l 10.2

Nu tc truyn tin R (bits/giy) ln hn dung lng knh C (bits/giy), th s truyn thng trn knh vi t l li nh tu l khng th thc hin c. Hay ni cch khc xc sut gii m li tin n 1 khi chiu di ca dy cn truyn gia tng. nh l ny ni cch khc nu tc truyn tin ln hn dung lng knh th vic truyn khng c m bo c ngha l chng ta khng th gii m ng c.

Trang 191

Bi 11 C s ton hc ca m chng nhiu

Bi ny trnh by cc c s ton hc ca m khi tuyn tnh. Cc kin thc ny l rt quan trng hiu c cch xy dng cc loi m khi tuyn tnh. Cc khi nim c trnh by bao gm cc cu trc i s nh nhm, trng v c bit l cc trng GF(2) v GF(2m), y l cc trng c ng dng c bit vo trong vic xy dng cc m khi tuyn tnh chng nhiu.

Trang 192

Bi 11 C s ton hc ca m chng nhiu

11.1 Mt s khi nim c bn 11.2 Trng GF(2) v cc a thc trn trng GF(2) 11.3 Trng GF(2m)

Trang 193

Mt s khi nim c bn

Php ton ng

Cho G l mt tp hp, mt php ton hai ngi f c gi l ng trn G nu f c dng f:GGG tc l nu a, b G th f(a, b) G. f(a, b) c mt cch vit tng ng l afb v ngc li f(b, a) cn c vit l bfa. Chng hn nu f l php cng th thay v vit +(a, b) chng ta thng vit l a + b. K t y tr v sau khi ni n mt php ton nu chng ta khng ni g thm th c ngha l php ton ny c tnh ng.

Ch

Trang 194

Mt s khi nim c bn (tt)

Tnh kt hp

Mt php ton hai ngi f trn G c gi l c tnh kt hp nu a, b, c G th (afb)fc = af(bfc) Mt php ton hai ngi f trn G c gi l c tnh giao hon nu a, b G th afb = bfa Trn tp s thc khc 0, php cng v php nhn c tnh kt hp v giao hon nhng php tr v php chia khng c tnh kt hp v giao hon.
Trang 195

Tnh giao hon

V d

Nhm

Tnh phn phi

Php ton f1 c gi l c tnh phn phi i vi php ton f2 nu a, b, c G th af1(bf2c) = (af1b)f2(af1c) Chng hn trn tp s thc, php nhn c tnh phn phi i vi php cng v a, b, c R a(b+c) = (ab)+(ac) Mt tp G , vi mt php ton hai ngi f c gi l mt nhm nu tho 3 iu kin sau: (1) f c tnh kt hp.

Nhm

Trang 196

Nhm (tt)
(2) G cha phn t e, sao cho a G th afe = efa = a. e c gi l phn t trung ho (i vi mt s php ton e cn c gi l phn t n v) (3) Mi phn t u c phn t i xng, tc l a G, tn ti phn t b G sao cho afb = bfa = e Chng hn, trn tp R nu f l php cng th phn t trung ho l s 0, cn trn tp s thc khc 0 nu f l php nhn th phn t trung ho l 1 v cn c gi l phn t n v. Mt nhm m php ton f c tnh giao hon th c gi l nhm giao hon.
Trang 197

Nhm giao hon

Nhm (tt)

Nhm hu hn, nhm v hn

Mt nhm c s phn t hu hn c gi l nhm hu hn, mt nhm c s phn t v hn c gi l nhm v hn. Cho G l mt nhm. Mt tp H con ca G c gi l mt nhm con nu H ng vi php ton hai ngi ca G v tho iu kin ca mt nhm. Tp cc s chn 0 l mt nhm con ca tp s t nhin vi php cng thng thng.

Nhm con

Trang 198

Php cng v nhn modulo

Php cng modulo v php nhn modulo

Cho mt s nguyn dng m xc nh. Xy dng mt tp s nguyn sau G = {0, 1, , m 1}. Vi + l php cng thng thng. nh ngha php ton mi nh sau v gi l php cng modulo a, b G th a b = (a + b) mod m Tng t vi l php nhn thng thng. nh ngha php ton mi nh sau v gi l php nhn modulo a, b G th a b = (a b) mod m

Trang 199

V d

Tp R l mt nhm giao hon i vi php cng v l mt nhm v hn. Tp R {0} l mt nhm giao hon i vi php nhn v l mt nhm v hn. Vi m l mt s nguyn dng xc nh, tp G = {0, 1, , m 1} vi php cng modulo l mt nhm giao hon v l mt nhm hu hn. Hai bng sau y trnh by ln lt trng hp m = 5 v m = 6.

Trang 200

V d (tt)
m=5 m=6 3 3 4 0 1 2 4 4 0 1 2 3

0 1 2 3 4 5 0 0 1 2 3 4 5 1 1 2 3 4 5 0 2 2 3 4 5 0 1 3 3 4 5 0 1 2 4 4 5 0 1 2 3 5 5 0 1 2 3 4 Tng t tp G = {1, , m 1} vi php nhn modulo v m nguyn t l mt nhm giao hon hu hn.
0 0 1 2 3 4 1 1 2 3 4 0 2 2 3 4 0 1

0 1 2 3 4

Trang 201

B 11.1

Nu m l mt s nguyn t th G = {0, 1, , m 1} l mt nhm giao hon vi php nhn modulo . Ngc li nu m khng nguyn t th G khng l mt nhm.

1 2 3 4

m=5 1 2 3 1 2 3 2 4 1 3 1 4 4 3 2

4 4 3 2 1

1 2 3 4 5
Trang 202

m=6 1 2 3 4 1 2 3 4 2 4 0 2 3 0 3 0 4 2 0 4 5 4 3 2

5 5 4 3 2 1

Trng

Trng

Mt tp G vi hai php ton ng hai ngi bt k, chng hn k hiu l + v *, c gi l mt trng nu tho 3 iu kin sau (1) G l nhm giao hon i vi php +. Phn t trung ho trong php + thng c k hiu l 0. (2) Tp cc phn t khc 0 l mt nhm i vi php *. Phn t trung ho trong php * thng c gi l phn t n v v k hiu l 1. (3) Php * c tnh phn phi i vi php +. Php + v php * trn khng nht thit l php cng v php nhn thng thng m chng c th l bt k php no. Chng ta k hiu nh vy d trnh by.
Trang 203

Ch

Trng (tt)

Cc phn t ca mt trng khng nht thit l cc s nguyn hay thc m c th l bt k ci g, chng hn c th l cc s phc, vect, ma trn hay a thc ... T nh ngha ca trng chng ta suy ra mt trng bao gm ti thiu hai phn t: phn t trung ho ca php + (k hiu l 0) v phn t trung ho ca php * (k hiu l 1). Cc phn t 0 v 1 khng nht thit l s 0 v s 1 theo ngha thng thng m c th l bt k ci g chng hn l ma trn 0 v ma trn n v, ... Mt trng m php * c tnh giao hon th c gi l trng giao hon.

Trng giao hon

Trang 204

Trng (tt)

Chng hn trong v d slide 198 vi m = 5 chng ta thy G l mt trng giao hon. Tng qut chng ta c b sau v dnh vic chng minh cho cc bn sinh vin. Cho p l mt s nguyn t bt k, G = {0, 1, ..., p 1} th G l mt trng giao hon i vi php cng modulo v php nhn modulo . Sau y l mt s tnh cht ca trng Mi phn t a ca trng u tho a * 0 = 0.

B 11.2

Tnh cht 1

Trang 205

Trng Galois

Tnh cht 2

Nu a, b l hai phn t khc 0 ca trng th a * b 0. Nu a 0 v a * b = a * c th b = c. Hay ni cch khc nu a 0 v b c th a * b a * c. S phn t ca mt trng c gi l bc ca mt trng. Mt trng c s phn t hu hn c gi l trng hu hn, mt trng c s phn t v hn c gi l trng v hn. Mt trng c s phn t hu hn c gi l trng Galois. Nu bc ca trng Galois l q th trng c k hiu l GF(q).
Trang 206

Tnh cht 3

Bc ca mt trng, trng hu hn, trng v hn.

Trng GF(q)

Trng Galois

i vi cc trng hu hn tc l trng Galois chng ta c nh l sau. Mt trng hu hn th s phn t ca n phi c dng pm trong p l mt s nguyn t cn m l mt s nguyn dng. Hay ni cch khc cc trng Galois u c dng GF(pm) trong p l mt s nguyn t cn m l mt s nguyn dng. i vi cc trng GF(p) vi p nguyn t th chnh l tp {0, 1, 2, ..., p 1} vi hai php ton cng modulo v nhn modulo nh bit. i vi cc trng GF(pm), v tnh phc tp ca chng, chng ta s gii thiu sau. Ch lc ny cc phn t ca trng GF(pm) khng n thun l cc s m s c dng kh c bit.
Trang 207

nh l 11.1

Trng Galois (tt)

K hiu cc phn t i xng

Phn t i xng ca a trong php + c k hiu l a, phn t i xng ca a trong php * c k hiu l a1. i vi mt trng giao hon, t hai php + v php * chng ta nh ngha thm hai php v php / nh sau (khng nht thit l php tr v php chia bnh thng) a b = a + (b) a / b = a * b1 trong b l phn t i xng ca b qua php +, cn b1 l phn t i xng ca b qua php *. Vy mt trng giao hon G c bn php ton +, , *, /. Php + v ng trn G, php * v / ng trn G {0}.
Trang 208

Php v php /

Tr ring ca mt trng

Xt mt trng GF(q). Xt cc dy tng ca cc phn t n v

i =1

1 = 1 + 1 + L + 1

(k ln, vi k = 1, 2, 3, )

V trng ng vi php cng nn kt qu ca nhng tng ny cng l cc phn t ca trng. V k c th nhn v hn gi tr m trng ch c q phn t nn tn ti hai gi tr k1 v k2 khc nhau (gi s k1 > k2 ) sao cho T y suy ra
i =1

1 = 1
i =1
k1 k 2 i =1

k1

k2

1 = 0

Trang 209

Tr ring ca mt trng

Tr ring ca mt trng k hiu l s nguyn dng nh nht sao cho D thy i vi cc trng GF(p) = {0, 1, 2, ..., p 1} vi p l mt s nguyn t th tr ring = p. Tng qut chng ta c nh l sau. Tr ring ca mt trng GF(q) l mt s nguyn t. Gi s khng nguyn t = k l (k, l nguyn > 1). T qui tc phn phi ca php nhn i vi php cng suy ra
i =1

1 = 0

nh l 11.2

Chng minh

Trang 210

Tr ring ca mt trng (tt)

i =1

1 1 = 1 = 1 = 0
i =1 i =1
l

k l

i =1

Suy ra

M k, l < , iu ny mu thun vi nh ngha ca .

i =1

1 = 0

i =1

1 = 0

Chu k ca mt phn t

Xt mt phn t a bt k khc 0 ca trng GF(q). Xt cc lu tha ak ca a vi k = 1, 2, 3, V trng ng vi php nhn nn cc ak cng l cc phn t ca trng. V k c th nhn v hn gi tr m trng ch c q phn t nn tn ti hai gi tr k1 v k2 khc nhau (gi s k1 > k2 ) sao cho

a k1 = a k 2 a k1 k 2 = 1
Trang 211

Chu k ca mt phn t

Chu k ca mt phn t a ca mt trng GF(q) l s nguyn dng nh nht n sao cho an = 1. Xt trng GF(7) = {0, 1, 2, 3, 4, 5, 6} vi hai php v . Tr ring ca trng ny l 7. Cn chu k ca cc phn t khc 0 ca trng c trnh by trong bng sau Phn t Chu k 1 1 2 3 3 6 4 3 5 6 6 2

V d

T nh ngha trn chng ta thy dy cc lu tha ca a a1, a2, ..., ak, ..., an = 1, an+1 = a, ... s lp li sau n phn t.
Trang 212

Nhm tun hon

B 11.3

Dy a1, a2, ..., ak, ..., an = 1 to nn mt nhm con ng vi php nhn trn trng GF(q). Mt nhm (khng cha phn t 0) vi php nhn * c gi l tun hon nu tn ti mt phn t trong nhm m cc lu tha ca n to nn mi phn t trong nhm. T nh ngha ny suy ra mt nhm hu hn c gi l tun hon nu tn ti mt phn t trong nhm c chu k ng bng s phn t ca nhm. Nu a l mt phn t khc 0 ca mt trng GF(q) th aq1 = 1
Trang 213

Nhm tun hon

nh l 11.3

Nhm tun hon (tt)

Chng minh

Gi b1, b2, ..., bq-1 l q 1 phn t khc nhau v khc 0 ca trng. Theo tnh cht 3 v tnh cht 2 ca trng chng ta c a*b1, a*b2, ..., a*bq-1 cng l q 1 phn t khc nhau v khc 0 ca trng. V vy chng ta c a*b1*a*b2* ... *a*bq-1 = b1*b2* ... *bq-1 T y suy ra aq1 = 1. Hon tt chng minh. Chu k ca mt phn t bt k khc 0 ca mt trng GF(q) l c s ca q 1.

nh l 11.4

Trang 214

Phn t c s

Chng minh

Gi n l chu k ca phn t a khc 0 ca trng GF(q). Gi s q 1 khng chia ht cho n. Do q 1 = kn + r, trong r l s d ca php chia q 1 cho n, 0 < r < n. Chng ta c aq-1 = akn+r = (an)k*ar Do aq-1 = 1 v an = 1 suy ra ar = 1. M 0 < r < n iu ny mu thun vi nh ngha chu k ca a. Vy q 1 chia ht cho n. Mt phn t a khc 0 ca mt trng GF(q) c gi l phn t c s nu chu k ca a bng q 1. T nh ngha ny nu a l mt phn t c s th cc lu tha ca a gm a0 = 1, a1 = a, a2, , aq 2 hnh thnh nn q 1 phn t khc 0 ca trng.
Trang 215

Phn t c s

V d

Xt trng GF(7) nh trong v d slide 209. Chu k ca cc phn t khc 0 ca trng u l c s ca 6. c bit cc phn t 3 v 5 c chu k bng 6 nn chng l cc phn t c s ca trng GF(7). 31 = 3 32 = 2 33 = 6 34 = 4 35 = 5 36 = 1 51 = 5 52 = 4 53 = 6 54 = 2 55 = 3 56 = 1

Trong cc trng Galois th trng GF(2) v trng GF(2m) l nhng trng c nhiu ng dng c bit trong l thuyt m, nn chng ta s ch trnh by hai trng ny.

Trang 216

Trng GF(2)

Trng GF(2)

Trng GF(2) bao gm hai phn t {0, 1} vi hai php cng + v nhn * nh sau + 0 1 0 0 1 1 1 0 * 0 1 0 0 0 1 0 1

Phn t i xng ca 0 v 1 qua php cng cng chnh l 0 v 1. Phn t i xng ca 1 qua php nhn cng chnh l 1. Trong trng GF(2) th php tr ging vi php cng, php chia cho mt s khc 0 cng ging vi php nhn.
Trang 217

Cc a thc trn trng GF(2)

a thc trn trng GF(2)

Mt a thc trn trng GF(2), chng hn k hiu l f(x), l a thc c dng f(x) = a0 + a1x + a2x2 + + anxn trong cc h s ai GF(2). L bc ln nht ca a thc. a thc f(x) = 1 + x + x3 c bc 3 a thc g(x) = x + x2 + x5 c bc 5.
Trang 218

Bc ca a thc

V d

Cc a thc trn trng GF(2) (tt)

Php cng a thc v nhn a thc

Vi f(x) = a0 + a1x + a2x2 + + anxn, g(x) = b0 + b1x + b2x2 + + bnxn vi cc h s ai v bj thuc trng GF(2) chng ta nh ngha cc php cng a thc v nhn a thc nh sau

n i + ( a b ) x f(x) + g(x) = i i i=0 n f(x) * g(x) = (ai *bj )xi + j , j = trong ai + bi v ai * bj cith c0 hin trn trng GF(2).

Trang 219

Cc a thc trn trng GF(2) (tt)

V d

Cho f(x) = 1 + x + x3, g(x) = x + x2 th f(x) + g(x) = (1 + x + x3) + (x + x2) = 1 + x2 + x3 f(x) * g(x) = (1 + x + x3) * (x + x2) = x + x3 + x4 + x5 Nu g(x) c bc khc 0 th chng ta c th chia f(x) cho g(x) v c th vit nh sau f(x) = q(x) * g(x) + r(x) trong q(x) l a thc thng cn r(x) l a thc d c bc nh hn a thc chia g(x). f(x) = 1 + x + x4 + x5 + x6 chia cho g(x) = 1 + x + x3
Trang 220

V d

Cc a thc trn trng GF(2) (tt)

1 + x + x4 + x5 + x6 = (x2 + x3) * (1 + x + x3) + (1 + x + x2) phn tch mt a thc ra thnh cc tha s trong i s Euclid chng ta c Nu f(a) = 0 th f(x) chia ht cho (x - a). iu ny cng ng trn trng GF(2). f(x) = 1 + x + x3 + x5 c f(1) = 0, nn f(x) chia ht cho (x - 1) m trong trng GF(2), php tr cng chnh l php cng tc l f(x) chia ht cho (x + 1). 1 + x + x3 + x5 = (1 + x)(1 + x3 + x4)

V d

Trang 221

a thc ti gin

Mt a thc trn GF(2) c gi l ti gin nu n khng phn tch c thnh tch ca hai a thc c bc nh hn. 1, 2, 3, 4 x 1+x 1 + x + x2 1 + x + x3 1 + x2 + x3 1 + x + x4 1 + x3 + x4 1 + x + x2 + x3 + x4 5 1 + x2 + x5 1 + x3 + x5 1 + x + x2 + x3 + x5 1 + x + x2 + x4 + x5 1 + x + x3 + x4 + x5 1 + x2 + x3 + x4 + x5 6 1 + x + x6 1 + x3 + x6 1 + x + x2 + x4 + x6 1 + x + x3 + x4 + x6 1 + x5 + x6 1 + x + x2 + x5 + x6 1 + x2 + x3 + x5 + x6 1 + x + x4 + x5 + x6 1 + x2 + x4 + x5 + x6

Trang 222

Cho f(x) l mt a thc trn trng GF(2), th

f (x)

2n

= f (x

2n

Chng minh

t f(x) = a0 + a1x + + anxn. [f(x)]2 = (a0 + a1x + + anxn)2 = a02 + a0*(a1x + + anxn) + a0*(a1x + + anxn) + (a1x + + anxn)2 = a02 + (a1x + + anxn)2 = a02 + (a1x)2 + + (anxn)2 = f(x2) (v trong GF(2) ai2 = ai ) iu ny cng gip chng ta suy ra iu phi chng minh.
Trang 223

Trng GF(2m)

Trc ht chng ta k hiu trng GF(2m) nh sau GF(2m) = {0, 1, a1, a2, ..., a m } 2 2 trong 0 v 1 GF(2). Trng GF(2) l mt trng con ca GF(2m) v c gi l trng c s ca GF(2m). Nu a l mt phn t GF(2m), f(x) l mt a thc trn trng GF(2), th f(a) cng l mt phn t ca GF(2m). C v hn a thc f(x) trn trng GF(2) m ch c hu hn (2m) phn t GF(2m), nn a 0 ca GF(2m) tn ti hai a thc f1(x) v f2(x) khc nhau sao cho f1(a) = f2(a). T y nu t f(x) = f1(x) f2(x) (ch trong trng GF(2) th php ging vi php cng +) th f(a) = 0.
Trang 224

Ch

a thc ti thiu

a thc ti thiu (minimal polinomial)

Cho a l mt phn t khc 0 ca trng GF(2m). a thc ti thiu ca a l a thc f(x) khc 0 trn trng GF(2) v c bc nh nht sao cho f(a) = 0. Mt ln na ta phi ch rng khi chng ta vit f() = 0 hoc f() = 1 th cc k hiu 0 v 1 khng nht thit l cc s 0 v 1, m s c hiu tu theo ng cnh. Chng hn nu phn t l mt ma trn th 0 chnh l ma trn 0 cn 1 chnh l ma trn n v.

Trang 225

a thc ti thiu (tt)

0 V d 0 T4 4 = Chng hn nu a l ma trn 4 4 bn 0 1

1 0 0 0 1 0 0 0 1 1 0 0 trong cc php cng v nhn trn ma trn vn thc hin nh bnh thng vi ch rng cc php cng v nhn cc phn t ca ma trn c thc hin trn trng GF(2). Chng ta c th kim tra rng 1 + T + T4 = 0 vi ch rng 1 l ma trn n v, cn 0 l ma trn 0. V f(x) = 1 + x + x4 l a thc ti thiu ca a
Trang 226

nh l

Hn na chng ta cng c T15 = 1 v chng ta c th kim tra rng 15 chnh l chu k ca a.

nh l 11.5

Cho a l mt phn t khc 0 ca trng GF(2m) c bc ca a thc ti thiu ca a l k. Gi Z l tp tt c cc phn t c dng b0 + b1a + + bk-1ak-1 trong bi GF(2). Th Z l mt tp con ca GF(2m) v hnh thnh nn mt trng c 2k phn t.

Trang 227

Chng minh

u tin chng ta chng minh cc phn t c hnh thnh t b0 + b1a + + bk-1ak-1 l khc nhau bng cch chng minh cc phn t 1, a, a2, , ak-1 l c lp tuyn tnh. k 1 k 1 Tht vy nu i i b a = c a i i th
i=0 i=0

p(a) =

k 1 i=0

i ( b c ) a i i =0

Vy chng ta c a thc p(x) c bc nh hn k tho p(a) = 0. M bc ca a thc ti thiu ca a bng k. Vy p(x) = 0, suy ra bi = ci i = 0, 1, ..., k 1.
Trang 228

Chng minh (tt)

Th hai, r rng Z l mt nhm giao hon i vi php +. Tht vy nu

k 1 i=0 i b a i Z,

k 1 i=0

i c a i Z

k 1 i=0

(bi + ci )a i = (ci + bi )a i Z
i=0

k 1

chng minh tp Z0 = Z {0} l mt nhm i vi php nhn * chng ta chng minh nu

k 1 i=0

i i c a , b a Z i 0 i Z0

k 1 i=0

k 1 i=0

bi a i * ci a i Z 0
i=0

k 1

Gi f ( x) =

i=0

i d x i l a thc ti thiu ca a, trong h s

cao nht dk = 1.
Trang 229

Chng minh (tt)

T y suy ra

xk =

k 1 i=0

i d x i

Do mi an vi n k u c th biu din thng qua mt a thc g(a) no c bc k 1. V vy k 1 i k 1 i bi a * ci a cng vy. Suy ra V r rng nu
k 1 i=0 i
k 1 i=0 i i b a * c a i i Z i=0
k 1 i=0

k 1

i=0

i=0

i c a bi a 0, i 0 i=0

k 1

i i b a * c a i i 0 i=0

k 1

Tnh cht ny c k tha t trng GF(2m).

Trang 230

H qu

Cui cng tnh phn phi ca php nhn * i vi php cng + chng ta cng k tha t trng GF(2m). Chng minh hon tt. Nu a thc ti thiu ca phn t a GF(2m) c bc bng m th trng Z trng vi trng GF(2m) v mi phn t ca trng c th c biu din nh mt vect m thnh phn (b0b1bm-1) Gi f(x) l a thc ti thiu ca phn t a 0 ca trng GF(2m) th f(x) l a thc ti gin trn trng GF(2).

H qu 11.1

nh l 11.6

Trang 231

Chng minh

Gi s f(x) = g(x) * h(x) trong g(x) v h(x) c bc ln hn 0 v nh hn bc ca f(x). Chng ta c f(a) = g(a) * h(a) = 0, suy ra g(a) = 0 hoc h(a) = 0. iu ny mu thun vi nh ngha v a thc ti thiu ca a. Cho f(x) l a thc ti thiu ca phn t a 0 ca trng GF(2m) v p(x) l a thc bt k trn trng GF(2) sao cho p(a) = 0. Th p(x) chia ht cho f(x). Chia p(x) cho f(x) chng ta c p(x) = g(x) * f(x) + r(x) trong bc ca r(x) nh hn bc ca f(x). Thay x = a r(a) = 0, nn t nh ngha ca a thc ti thiu r(x) = 0 p(x) chia ht cho f(x).
Trang 232

B 11.5

Chng minh

nh l

nh l 11.7

Cho f(x) l a thc ti thiu ca phn t a 0 ca trng GF(2m) v p(x) l a thc ti gin trn trng GF(2) sao cho p(a) = 0. Th f(x) = p(x).

Chng minh

Theo B 11.5 trn chng ta c p(x) chia ht cho f(x) tc l chng ta c th vit p(x) = g(x) * f(x) Do p(x) l a thc ti gin nn f(x) = 1 hoc f(x) = p(x). Tuy nhin f(x) khng th bng 1 nn suy ra f(x) = p(x). 2m 1 phn t khc 0 ca trng GF(2m) u l nghim ca phng trnh 2 m 1 x +1= 0
Trang 233

H qu 11.2

H qu

H qu 11.3 m

x 2 1 + 1 chia ht cho cc a thc ti thiu ca cc phn t khc 0 ca trng GF(2m). Chng ta s dn ra mt h qu mnh hn nh sau. Trc ht chng ta phn tch 2 m 1 x +1 thnh tch ca cc am thc ti gin trn trng GF(2) 2 1 x + 1 = p1(x) * p2(x) * * pl(x) m V x 2 1 + 1 c cc nghim l cc phn t ca trng GF(2m) nn cc phn t ca trng GF(2m) s l nghim ca mt pi(x) no v ngc li mt pi(x) bt k s c cc nghim l cc phn t ca trng GF(2m). Hn na nu pi(x) c bc t th s c t nghim trong trng GF(2m).
Trang 234

H qu

H qu 11.4

+ 1 thnh tch cc a thc ti gin, Trong vic trin khai x th mi a thc ti gin s l a thc ti thiu ca mt phn t khc 0 no ca trng GF(2m).

2 m 1

nh l 11.8

Cho a l mt phn t khc 0 ca trng GF(2m) v f(x) l mt a thc trn trng GF(2). Nu f(a) = 0 th

f (a

2l

) = 0 l = 0, 1, 2, ...

H qu 11.5

Nu f(x) l a thc ti thiu ca phn t a 0 ca trng GF(2m) th f(x) cng l a thc ti thiu ca cc phn t

2l

vi l = 0, 1, 2, ... ca trng GF(2m).

Trang 235

H qu (tt)

Hay ni cch khc cc phn t a vi l = 0, 1, 2, ... l cc nghim ca a thc ti thiu f(x) ca phn t a. Hn na chng ta s chng minh rng ngoi chng ra f(x) khng cn nghim no khc thuc trng GF(2m). V vy nu c bao nhiu phn t a khc nhau th f(x) c bc by nhiu. lm r iu ny gi k l s nguyn dng nh nht sao cho S k chc chn tn ti v chng ta c V s k biu din chu k ca dy
2l

2l

2k

=a

2 m 1

= 1 hay a

2m

=a

2 l vi l = 0, 1, 2, ...
Trang 236

B d 11.6

Cho a l mt phn t khc 0 ca trng GF(2m) v k l s nguyn dng nh nht sao cho th k l mt c s ca m.

2k

=a

Chng minh

2l

Chia m cho k, m = n k + r, trong r l s d v r < k

2k

2k = a a

2k

=a

2k

= a hay a
2 nk

2 2k

=a

Tip tc theo kiu ny chng ta c a ta c

= a . Mt khc chng
2r

a=a

2m

=a

2 n k + r

=a

2 n k 2 r

2 n k = a

= (a )

2r

Trang 237

B (tt)
Do nh ngha ca k r = 0. Hon tt chng minh.

Phn t lin hp

Cho a l mt phn t khc 0 ca trng GF(2m) v k l s nguyn dng nh nht sao cho th cc phn t a vi l = 0, 1, 2, ..., k - 1 c gi l cc phn t lin hp ca a v k c gi l s phn t lin hp ca a. T nh ngha chng ta thy tp cc phn t lin hp ca a l tp cc phn t khc nhau c sinh ra bi
2l

2k

=a

a vi l = 0, 1, 2, ...

2l

B d 11.7

Nu a1 v a2 l cc phn t lin hp bt k ca a th a1 l phn t lin hp ca a2 v ngc li a2 l phn t lin hp ca a1.

Trang 238

B (tt)

Tht vy gi s (vi k l s phn t lin hp ca a) Th

a1
2 l 2 l1

a1 = a
= (a

2 l1

, a2 = a

2l2

,0 l1 < l 2 < k
=a
2l2

2 l1 2 l 2 l1

=a

2 l1 2 l 2 l1

= a2

a2

2 k + l1 l 2

= (a =a

2 l 2 2 k + l1 l 2

=a

2 l 2 2 k + l1 l 2 2 l1

2 k + l1

= (a

2 k 2 l1

=a

= a1

Hon tt chng minh. Ch b ny ni ln rng cc phn t lin hp ca a l lin hp vi nhau.

Trang 239

B (tt)

V cc phn t a thc ti thiu f(x) ca a, nn ta s chng minh f(x) c dng

2 lvi l = 0, 1, 2, ..., k - 1 l cc nghim ca a

f (x) = (x + a ) * ( x + a ) * K * (x + a

2 k 1

)=

k 1 i=0

(x + a

2i

chng minh iu ny chng ta s chng minh

p (x) =

k 1 i=0

(x + a

2i

l mt a thc ti gin v do p(a) = 0 nn theo nh l 11.7 chng ta suy ra f(x) = p(x).

Trang 240

B (tt)

B d 11.7

Cho a l mt phn t khc 0 ca trng GF(2m) v k l s nguyn dng nh nht sao cho

a
th

2k

p (x) =

k 1 i=0

=a
2i

(x + a

l mt a thc ti gin trn trng GF(2).

Chng minh

[ p (x)]

= (x + a i = 0
Trang 241

k 1

2i

) = (x + a i=0

k 1

2i 2

Chng minh (tt)

(x + a
2i 2
2

) = x + (a
k 1 i=0 k 1

2i

+a

2i

) x + (a
k

2i 2
2

) =x +a
2i

2 i +1

[ p ( x)] = = =

(x

+a +a +a

2 i +1

) = (x + a
i =1 2 2k

i =1 k 1 i =1

(x (x

2i

)(x + a
2

)
k 1 i=0

2i

)(x + a ) =

(x

+a

2i

= p(x 2 )
Trang 242

Chng minh (tt)

Mt khc p(x) l mt a thc ca x v c th biu din p(x) = b0 + b1x + + bkxk trong cc bi vi i = 0, 1, 2, , k l cc a thc trn trng GF(2) ca a. V vy cc bi l cc phn t ca trng GF(2m). [ p(x)]2 = (b0 + b1x + Kbk x k ) 2
=
i=0

bi 2 x 2i + (1 + 1) bi b j x i + j = bi 2 x 2i
i=0 j=0 i j i=0

Do [p(x)]2 = p(x2) suy ra bi = bi2 i = 0, 1, 2, , k iu ny ch ng nu cc bi bng phn t 0 hoc phn t 1 tc l cc bi GF(2) hay p(x) l mt a thc trn trng GF(2).
Trang 243

Chng minh (tt)

Nu p(x) khng ti gin tc p(x) c th phn tch thnh p(x) = q(x) * h(x) trong bc ca q(x) v h(x) nh hn bc ca p(x) l k. Nhng do p(a) = 0 q(a) = 0 hoc h(a) = 0. 2l Gi s q(a) = 0, theo nh l 12.8 q(x) c cc nghim l a vi l = 0, 1, 2, ..., k 1, q(x) c bc ti thiu l k, mu thun. T y suy ra iu phi chng minh.

nh l 11.9

Cho a l mt phn t khc 0 ca trng GF(2m) v k l s nguyn dng nh nht sao cho
a
2k

=a

th a thc ti thiu ca a l f (x) =

Trang 244

k 1 i=0

(x + a

2i

) v c bc = k.

H qu

H qu 11.6

Bc ca mt a thc ti thiu ca mt phn t a khc 0 ca trng GF(2m) l mt c s ca m. Cho a l mt phn t khc 0 ca trng GF(2m) c chu k ba ng n th cc phn t lin hp ca a cng c chu k bng n. Gi k l s thnh phn lin hp ca a. i = 0, 1, ..., k Chng ta chng minh rng khng s nguyn dng l < n
n
2l

nh l 11.10

Chng minh

2i

2i n n 2i = (a ) = 1 =a a
l i 2

=1

Trang 245

Chng minh
Tht vy gi s tn ti, suy ra a =1 Chia 2i l cho n 2i l = h n + r trong 0 r < n,
2i l

1= a

2i l

= a h n + r = (a n ) h a r = a r

T nh ngha khi nim chu k, r = 0. T nh l 11.4 n l mt c s ca 2m 1, n l. Kt hp vi 2i l = h n n l mt c s ca l, n l v l.

nh l 11.11

m 1 u tn ti mt a thc ti gin bc m trn trng GF(2).

Trang 246

nh l

nh l 11.12

Vi mt a thc ti gin p(x) bt k c bc m, p(x) = b0 + b1x + + bmxm trong bm = 1, chng ta lun xy dng c mt trng GF(2m) trong p(x) l a thc ti thiu ca mt phn t ca trng.
Tm m = 0 1 0 0 L L L L L 0 0 0 M 0 b0 0 0 M 0 1 0 M 0 0 1 M 0 0 0 M 0

xy dng trng GF(2m) chng ta cho phn t a l mt ma trn mm nh bn

b1 b2

b3 L bm 2

0 0 0 M 1 bm 1

Trang 247

nh l (tt)

Trn ma trn nh ngha php cng v nhn ma trn nh bnh thng, vi ch rng vic cng hoc nhn hai phn t trong 2 ca hai ma trn c thc hin nh trn trng GF(2). Chng ta cng nhn rng ma trn ny c a thc ti thiu l p(x). T y chng ta c th dn ra c cc phn t cn li ca trng GF(2m) nh nh l 11.5. Ch , phn t 0 chnh l ma trn 0 v phn t 1 chnh l ma trn n v.
m 2, cc a thc ti gin bc m trn trng GF(2) u l c s ca 2 m 1 x +1 Chng ta c th quay tr v bng lit k cc a thc ti gin kim tra rng cc a thc ti gin bc 3 l c s ca x7 1, cc a thc ti gin bc 4 l c s ca x15 1,
Trang 248

nh l 11.13

nh l (tt)

a thc cn bn

Mt a thc cn bn l mt a thc ti gin, ng thi khng tn ti s nguyn dng n < 2m 1 sao cho xn + 1 chia ht cho n. V d, khng tn ti s nguyn dng n < 15 sao cho xn + 1 chia ht cho 1 + x + x4 nn 1 + x + x4 l a thc cn bn. Cn a thc 1 + x + x2 + x3 + x4 l ti gin nhng khng cn bn v x5 + 1 chia ht cho n. 1, 2, 3 4, 5 6, 7, 8 1+x 1 + x + x4 1 + x + x6 1 + x + x2 1 + x3 + x4 1 + x3 + x7 1 + x + x3 1 + x2 + x5 1 + x2 + x3 + x4 + x8 1 + x2 + x3 1 + x3 + x5
Trang 249

nh l (tt)

nh l 11.14

Chng minh

Cho a l mt phn t khc 0 ca trng GF(2m) c a thc ti thiu l f(x). Nu f(x) l mt a thc cn bn trn trng GF(2) v c bc bng m th a c chu k l 2m 1 v a l mt phn t l c s ca GF(2m). a2

Gi n l chu k ca a. t p(x) = xn + 1, th p(a) = 0. B 11.5 p(x) chia ht cho f(x). Kt hp iu ny vi nh ngha ca khi nim a thc cn bn, n = 2m 1. nh l ny gi cho chng ta cch xy dng trng GF(2m) da trn mt phn t c s c a thc ti thiu l mt a thc cn bn bc m.

Trang 250

Tm tt

Tm tt

=1 a l mt phn t ca trng th a Chu k ca mt phn t l mt c s ca 2m 1. Cc a thc ti thiu ca trng GF(2m) l cc a thc ti gin v l c s ca m
GF(2m)

2 m 1

x 2 1 + 1

Hn na bc ca chng l c ca m. S phn t lin hp khc nhau ca a, k c a, l c s ca m. Cc phn t lin hp ca nhau c cng a thc ti thiu, hn na chng l cc nghim ca a thc ti thiu ny v bc ca a thc ti thiu ny bng s cc phn t lin hp khc nhau. Cc phn t lin hp th c cng chu k. k 2 1 +1 Cc a thc ti gin bc k l c ca x
Trang 251

Tm tt (tt)

Mt phn t a c a thc ti thiu bc m th cc t hp tuyn tnh (vi bi GF(2)) b01 + b1a + + bk - 1ak - 1 s sinh ra ton b cc phn t ca trng GF(2m) Mt phn t a c a thc ti thiu bc m v cng l a thc cn bn th cc ly tha ca n s sinh ra ton b cc phn t ca trng GF(2m).

Trang 252

V d

Xy dng trng GF(2m) vi m = 4. Chng ta k hiu 0 l ma trn 0, k hiu 1 l ma trn n v (c kch thc l 44). Ly phn t a l ma trn sau 0 1 0 0 0 0 1 0 T4 4 = 0 0 0 1 1 1 0 0 Chng ta c a thc ti thiu ca a l f(x) = 1 + x + x4 y l mt a thc cn bn bc 4. V vy theo nh l 11.14, 15 phn t ca GF(24) khng tnh phn t 0 s c dng ai, i = 0, 1, , 14 vi ch a0 = 1. Cn theo nh l 12.12 chng s c dng b0 + b1a + b2a2 + b3a3 trong cc bi = 0 hoc 1.
Trang 253

V d (tt)

Vy c hai cch xy dng trng GF(24) nh trn. Cc bng sau y biu din cc phn t khc 0 v khc 1 ca trng GF(24) theo cc dng: ly tha ca a (ai), a thc ca a, vect, dng ma trn.
a2 a2 0010 a3 a3 0001 a4 1+a 1100 a5 a + a2 0110

a a 0100

0 0 0 1

1 0 0 0 1 0 0 0 1 1 0 0

0 0 1 0

0 1 0 0 0 1 1 0 0 1 1 0

0 1 0 0

0 0 1 1 0 0 1 1 0 0 1 1
Trang 254

1 0 0 1

1 0 0 1 1 0 0 1 1 1 0 1

0 0 1 1

1 1 0 0 1 1 1 0 1 0 1 0

V d (tt)

a6 a2 + a3 0011

a7 1 + a + a3 1101

a8 1 + a2 1010

a9 a + a3 0101

a10 1 + a + a2 1110

0 1 1 0

0 1 0 1

1 0 1 0

1 1 0 1

1 1 0 1

1 0 1 1

0 1 0 1

1 0 1 0

1 0 1 1

1 1 1 0

1 1 1 1

0 1 1 1

1 0 1 0

0 1 1 1

1 0 1 1

0 1 0 1

0 1 0 1

1 1 1 1

0 1 1 1

1 0 1 1

Trang 255

V d (tt)
a11 a + a2 + a3 0111 0 1 1 1 1 1 1 1 1 0 1 1 1 0 0 1

a12 1 + a + a2 + a3 1111 1 1 1 1 1 0 1 1 1 0 0 1 1 0 0 0

a13 1 + a2 + a3 1011 1 0 1 1 1 0 0 1 1 0 0 0 0 1 0 0

1 1 0 0

a14 1 + a3 1001 0 0 1 0 0 0 1 0 0 0 1 0

Chu k, a thc ti thiu ca cc phn t lin hp

a, a2, a4, a8 a3, a6, a12, a9 a5, a10 a7, a14, a13, a11 15 5 3 15 x 1 + x 1 + x + x4 1 + x + x2 + x3 + x4 1 + x + x2 1 + x3 + x4
Trang 256

Bi 12 M khi tuyn tnh

12.1 Gii thiu 12.2 Cc khi nim v nguyn l hot ng 12.3 Vn pht hin sai v sa sai 12.4 Mt s gii hn

Trang 257

Gii thiu

M khi tuyn tnh c xy dng da trn cc kt qu ca i s tuyn tnh l mt lp m c dng rt ph bin trong vic chng nhiu. Mt m khi c chiu di n gm 2k t m c gi l m tuyn tnh C(n, k) nu v ch nu 2k t m hnh thnh mt khng gian vect con k chiu ca khng gian vect n chiu gm tt c cc vect n thnh phn trn trng GF(2). M tuyn tnh C(n, k) c mc ch m ho nhng khi tin (hay thng bo) k bit thnh nhng t m n bit. Hay ni cch khc trong n bit ca t m c cha k bit thng tin. Qui c vit du + thay cho du v du + s c hiu theo ng cnh.
Trang 258

nh ngha

Cch biu din m Ma trn sinh

M tuyn tnh C(n, k) l mt khng gian con k chiu ca khng gian vect n thnh phn, k t m c lp tuyn tnh, chng hn (g0, g1, ..., gk1) sao cho mi t m trong C l mt t hp tuyn tnh ca k t m ny (vi ai {0, 1} i = 0, 1, ..., k1) w = a0g0 + a1g1 + ... + ak1gk1 k t m ny to thnh mt ma trn cp k n nh sau g 01 g 0( n 1) L g 0 g 00 g g g g L 10 11 1( n 1) 1 Gk n = = M M M M g g g L ( k 1)1 ( k 1)( n 1) g k 1 ( k 1)0

Vi gi = (gi0, gi1, , gi(n1)), vi i = 0, 1, , k1.

Trang 259

Cch m ha

Nu u = (a0, a1, , ak1) l thng tin cn c m ho th t m w tng ng vi u c ta bng cch ly u nhn vi G w = u G = (a0, a1, , ak1) hay w = a0g0 + a1g1 + + ak1gk1 V cc t m tng ng vi cc thng bo c sinh ra bi G theo cch trn nn G c gi l ma trn sinh ca b m.

Trang 260

V d

Cho ma trn sinh ca mt m tuyn tnh C(7, 4) sau

g 0 1 1 g 1 0 1 G4 7 = = g 2 0 1 g 3 1 0 0 1 0 0 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 1

Nu u = (1101) l thng tin cn m ho th t m tng ng l w = 1.g0 + 1.g1 + 0.g2 + 1.g3 = (1100101) Bt k k t m c lp tuyn tnh no cng c th c dng lm ma trn sinh cho b m. Mt b m tuyn tnh (hay cn gi l khng gian m) c th c nhiu ma trn sinh khc nhau cng biu din. Mi ma trn sinh tng ng vi mt cch m ha khc nhau.
Trang 261

Cch gii m
Ly ma trn sinh nh trong v d trn. u = (a0, a1, a2, a3) l thng bo, w = (b0, b1, b2, b3, b4, b5, b6) l t m tng ng. Chng ta c h phng trnh sau lin h gia u v w. w=uG b0 = a0 + a1 + a3 (1) b1 = a0 + a2 (2) g 0 1 1 0 1 0 0 0 b2 = a1 + a3 (3) g 1 0 1 1 1 0 0 1 b3 = a0 + a1 (4) G4 7 = = g 2 0 1 0 0 0 1 1 b4 = a1 (5) g 3 1 0 1 0 0 0 1 b5 = a2 (6) b6 = a2 + a3 (7)

Trang 262

Cch gii m (tt)

Chn bn phng trnh n gin nht gii cc ai theo cc bj. Chng hn cc phng trnh (4), (5), (6), (7) chng ta gii c a0 = b3 + b4 g 0 1 1 0 1 0 0 0 g 1 0 1 1 1 0 0 a1 = b4 1 G4 7 = = g 2 0 1 0 0 0 1 1 a2 = b5 g 1 0 1 0 0 0 1 a3 = b5 + b6 3 H phng trnh trn c gi l h phng trnh gii m. C th c nhiu h phng trnh gii m khc nhau nhng s cho kt qu nh nhau. w = 1001011 u=? w = 0101110 u=?

Trang 263

M tuyn tnh h thng

Mt m tuyn tnh C(n, k) c gi l m tuyn tnh h thng nu mi t m c mt trong hai dng sau Dng 1: T m bao gm phn thng tin k bit i trc v phn cn li (gm n k bit) i sau (phn ny cn c gi l phn d tha hay phn kim tra).
k bit thng tin n k bit kim tra

Dng 2: Ngc ca dng 1, t m bao gm phn kim tra i trc v phn thng tin i sau.
n k bit kim tra k bit thng tin

Trang 264

Ma trn sinh h thng

1 0 L 0 P L P P 00 01 0(n k 1) L P P 0 1 L 0 P 10 11 1(n k 1) Gk n = I kk | Pk (n k ) = M M M M M M 0 0 L 1P(k 1)0 P(k 1)1 L P(k 1)(n k 1) 4 43 4 1444444 4 24444444 3 142 k k k (n k )

V d

Ght ( 47 )

1 0 = 0 0

0 0 0 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 0 0 1 1 0 1

M ha u = 1101 w = u Ght = 1101000 Gii m w = 0110100 u = 0110

Trang 265

V d

Dng cc php bin i s cp bin i cc ma trn sinh sau thnh ma trn sinh h thng.
1 0 = 0 1 1 0 1 0 0 1 1 1 0 1 0 1 0 1 1 0 0 1 0 0 1 0 1 0 1 1 = 1 1 1 0 1 0 0 1 0 1 0 1 0 1 0 0 1 1 0 0 0 0 1 0 1 1

G47

G 4 7

Khng phi mi ma trn sinh u c th bin i thnh ma trn sinh h thng.

Trang 266

V d (tt)

Hy thc hin php m ha v gii m trn ma trn sinh sau.

1 1 G4 7 = 0 1
u = a1 a2 a3 a4

1 0 0 1 0 0 0 1 0 1 0 1 0 0 1 1 0 1 0 0 0 0 1 1

th w = b1 b2 b3 b4 b5 b6 b7

M ha Gii m

u = (1101) w = (1110010) w = (1011000) u = (0110)

Trang 267

Pht hin sai v sa sai

Nguyn l pht hin sai: Kim tra xem t hp nhn c phi l t m hay khng, nu khng th t hp nhn l sai. Nguyn l sa sai: Kim tra xem t hp nhn c khong cch Hamming gn vi t m no nht, th chnh l t m ng c pht i. Nguyn l ny c gi l nguyn l khong cch Hamming ti thiu. Cho S l mt khng gian con k chiu ca khng gian V n chiu. Gi Sd l tp tt c cc vect v trong V sao cho u S, u v = 0 (php nhn v hng ca hai vect). Sd c chng minh l mt khng gian con ca V v c s chiu l n k. Sd c gi l khng gian b trc giao ca S v ngc li.
Trang 268

Khng gian b trc giao

Cch pht hin sai

H qu

Mi ma trn G bt k kch thc k n vi k hng c lp tuyn tnh lun tn ti ma trn H kch thc (n k) n vi (n k) hng c lp tuyn tnh sao cho G HT = 0, trong HT l ma trn chuyn v ca ma trn H. Ni cch khc cc vect hng ca H u trc giao vi cc vect hng ca G. Nu v l mt t m c sinh ra t ma trn sinh G c ma trn trc giao tng ng l H th v HT = 0 Ngc li nu v HT = 0 th v l mt t m.
Trang 269

Cch pht hin sai

Ma trn kim tra

Ma trn kim tra

Ma trn kim tra ca mt b m c ma trn sinh Gkn l ma trn H c kch thc (n k) n sao cho G HT = 0
v HT c gi l syndrome hay vect sa sai ca v v c k hiu l s(v). v l t m khi v ch khi s(v) = 0.

Syndrome vect sa sai (corrector)

V d

Tm ma trn kim tra ng vi ma trn sinh sau. 1 1 0 1 0 0 0 1 0 1 1 1 0 0 G 4 7 = 0 1 0 0 0 1 1 1 0 1 0 0 0 1

Trang 270

Ma trn kim tra (tt)

H c kch thc 3 7. Gi h = (a0, a1, a2, a3, a4, a5, a6) l mt hng bt k ca H. h trc giao vi mi hng ca G nn chng ta c h bn phng trnh sau 1 1 0 1 0 0 0 a0 + a1 + a3 =0 1 0 1 1 1 0 0 a0 + a2 + a3 + a4 = 0 G4 7 = 0 1 0 0 0 1 1 a1 + a5 + a6 =0 0 1 1 0 0 1 0 a0 + a2 + a6 =0 Vn l tm c 3 vect h c lp tuyn tnh l nghim ca h phng trnh trn. Ch , h phng trnh trn c th cho php chng ta gii bn bin theo ba bin cn li. Chng hn chng ta gii a3, a4, a5, a6 theo a0, a1, a2 nh sau.
Trang 271

Ma trn kim tra (tt)

a3 = a0 + a1 a4 = a1 + a2 a5 = a0 + a1 + a2 a6 = a0 + a2 Cho (a0, a1, a2) ln lt cc gi tr (1, 0, 0), (0, 1, 0), (0, 0, 1) (c lp tuyn tnh vi nhau), ta xc nh c (a3, a4, a5, a6) ln lt nh sau (1, 0, 1, 1), (1, 1, 1, 0), (0, 1, 1, 1). Vy H l

Ch

1 0 0 1 0 1 1 H 3 7 = 0 1 0 1 1 1 0 0 0 1 0 1 1 1

C th tn ti nhiu ma trn kim tra khc nhau ca cng mt b m v chng u c kh nng kim tra nh nhau.
Trang 272

Ma trn kim tra (tt)

B 12.1

Nu ma trn sinh h thng ca mt m tuyn tnh h thng c dng Gkn = [Ikk | Pk(nk)] th H(nk)n = [Pk(nk)T | I(nk)(nk)] l mt ma trn kim tra ca m. Tng t nu ma trn sinh c dng Gkn = [Pk(nk) | Ikk] th ma trn kim tra c dng H(nk)n = [I(nk)(nk) | Pk(nk)T] trong I(nk)(nk) l ma trn n v kch thc (nk)(nk), cn Pk(nk)T l ma trn chuyn v ca ma trn Pk(nk).

Trang 273

Chng minh
1 0 L 0 P P P L 00 01 0( n k 1) P P L 0 1 L 0 P 10 11 1( n k 1) Gk n = M M M M M M 0 0 L 1 P( k 1)0 P( k 1)1 L P( k 1)( n k 1) 4 43 4 1444444 4 24444444 3 142 k k k (n k ) P P P L L 1 0 0 00 10 (k 1)0 P P(k 1)1 0 1 L 0 L P01 11 H ( n k ) n = M M M M M M P0(n k 1) P 0 L 1 1(n k 1) L P (k 1)(n k 1) 0 4 43 4 4 244444444 3142 14444444 ( n k ) ( n k ) ( n k ) k
Trang 274

Chng minh (tt)

Ta chng minh

G HT = 0 Chng minh iu ny vic chng minh gi hj = 0 i = 0, , k1, j = 0, , nk1 trong gi = (gi0, , gi(n1)) l hng i ca G cn hj = (hj0, , hj(n1)) l hng j ca ma trn H. Tht vy
gi h j =
n 1 s=0

g is h js = g is h js + g is h js
s=0 s=k

k 1

n k 1

= h ji + g i ( k + j ) = Pij + Pij = 0

Trang 275

V d

Tm ma trn H cho cc ma trn sinh sau

1 0 = 0 0 1 1 = 0 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 0 0 1 1 0 1 0 0 0 1 1 0 1 0 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 1 = 0 1 1 0 0 1 0 0 0 1 0 1 0 1 0 0 1 1 0 1 0 0 0 0 1 1

Ght ( 47 )

G47

G47

Trang 276

Kh nng chng nhiu tng ng

Hai m tuyn tnh C(n, k) c gi l c kh nng chng nhiu tng ng nu chng c cng khong cch Hamming. Nu hon v hai ct ca mt ma trn sinh s to ra mt b m mi c kh nng chng nhiu tng ng vi b m c. Ni cch khc vic hon v hai ct ca ma trn sinh khng lm thay i kh nng chng nhiu. Khong cch Hamming ca mt m tuyn tnh bng trng s nh nht khc 0 ca b m.

B 12.2

B 12.3

Trang 277

B 12.4

Gi H l ma trn kim tra ca mt m tuyn tnh, nu mt t m c trng s d th tn ti d ct ca H c tng bng 0. Nu trong ma trn kim tra H ca mt m tuyn tnh s ct ph thuc tuyn tnh nh nht l d th khong cch Hamming ca b m bng d.

H qu

V d 12.5

1 0 0 1 0 1 1 H 3 7 = 0 1 0 1 1 1 0 0 0 1 0 1 1 1
Trang 278

d = 3 (3, 4, 6)

Cch sa sai

Vect li

L vect biu din cc v tr li gia t m truyn v t hp nhn, mi v tr li c biu din bng bit 1, cn li l 0. Nu t m truyn l w, vect li l e v vect nhn l v th v=w+e e=v+w w=e+v
w = 1011011, e = 0010100 v = w + e = 1001111. w = 0110010, v = 0010011 e = w + v = 0100001. v = 1011001, e = 0010010 w = v + e = 1001011.
Trang 279

V d

Tp gii m - coset

Cho S l mt khng gian con cc t m ca khng gian V, coset ca mt phn t z V i vi S c k hiu l z + S v c nh ngha nh sau z + S = {z + w: w S}

B 12.5
Tp coset z + S c cc tnh cht sau. (1) z z + S. (2) Nu z S th z + S = S. (3) Nu v z + S th v + S = z + S. (4) Nu v z + S th v + S v z + S ri nhau.

Trang 280

S gii m

Vi mi vect nhn v chng ta s c mt tp coset tng ng l v + S. Trong tp ny chn phn t c trng s nh nht, chng hn l z. Phn t ny thng c gi l coset leader. Thng bo t m c truyn chnh l w = v + z. Cc phn t ca mt tp coset c cng mt syndrome nh nhau. Cc tp coset khc nhau c cc syndrome khc nhau. e = (a1, a2, ..., an), cc ct ca H ln lt bng h1, h2, ..., hn th

B 12.6

s (e) = e H = ai hi = ai hi
T i =1 ai 0
Trang 281

S gii m (tt)

Ngha l s(e) bng tng nhng ct nhng v tr tng ng vi nhng v tr bng 1 ca e. Nu v tr li sai l 3 th syndrome ca vect nhn s l ct s 3 ca H.
1 1 G4 7 = 0 1 1 0 1 0 0 1 0 1 1 1 0 0 0 1 0 0 0 0 1 0 0 0 1 1

1 0 0 1 0 1 1 0 1 0 1 1 1 0 H 3 7 = 0 0 1 0 1 1 1

Tm v tr li sai ca cc vect nhn sau y v = 0010011 s(v) = ? e=? w=? v = 0101101 s(v) = ? e=? w=?
Trang 282

M tuyn tnh Hamming

M tuyn tnh Hamming l m c ma trn H c tnh cht gi tr ca ct hi bng i (i = 1, 2, ...)

0 0 0 1 1 1 1 0 1 1 0 0 1 1 H 3 7 = 1 0 1 0 1 0 1

B 12.7

Cc m tuyn tnh Hamming u c khong cch Hamming d = 3. V vy c th pht hin sai 2 bit v sa sai 1 bit.

Trang 283

Ma trn sinh ca m tuyn tnh Hamming

Xt m tuyn tnh Hamming C(7, 4) c cc bit thng tin nm cc v tr 3, 5, 6, 7. Hy xc nh ma trn sinh G ca b m. Gi w = (a1, a2, a3, a4, a5, a6, a7) l mt t m. Chng ta c h phng trnh sau c dn ra t cng thc w HT = 0. a4 + a5 + a6 + a7 = 0 a2 + a3 + a6 + a7 = 0 a1 + a3 + a5 + a7 = 0 T y suy ra cng thc tnh cc bit kim tra a1, a2, a4 theo cc bit thng bo a3, a5, a6, a7 nh sau a1 = a3 + a5 + a7 a2 = a3 + a6 + a7 a4 = a5 + a6 + a7
Trang 284

Ma trn sinh ca m tuyn tnh Hamming

b1 b2 b3 b4 u= 1 0 1 0 a1 a2 a3 a4 a5 a6 a7 1 0 1 1 0 1 0

th w =

V d

1 1 G4 7 = 0 1

1 1 0 0 0 0 0 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 0 1

Xt m tuyn tnh Hamming C(7, 4) c cc bit thng tin nm cc v tr 1, 2, 3, 4. Hy xc nh ma trn sinh G ca b m.

Trang 285

Bi 13 M vng
13.1 Gii thiu 13.2 Cc tnh cht ca m vng 13.3 Ma trn sinh v ma trn kim tra ca m 13.4 M BCH

Trang 286

Gii thiu

nh ngha

Mt m tuyn tnh C(n, k) c gi l m vng nu w = a0a1an2an1 l mt t m th v = an1a0a1an2 cng l mt t m. Ngha l dch vng (sang tri hay phi) mt t m th kt qu cng l mt t m. y qui c dch phi. Nu w = a0a1an2an1 l mt t m th w(x) = a0 + a1x + + an2xn - 2 + an1xn - 1 l a thc m tng ng vi t m w. Bng sau y trnh by mt m vng C(7, 4).
Trang 287

a thc m

V d

V d
m w w(x) m w w(x)

0000 1000 0100 1100 0010 1010

0000000 1101000 0110100 1011100 0011010 1110010

0 1 + x + x3 x + x2 + x4 1 + x2 + x3 + x4 x2 + x3 + x5 1 + x + x2 + x5

0001 1001 0101 1101 0011 1011

0110 0101110 x + x3 + x4 + x5 1110 1000110 1 + x4 + x5

x3 + x4 + x6 1 + x + x4 + x6 x + x2 + x3 + x6 1 + x2 + x6 x2 + x4 + x5 + x6 1 + x + x2 + x3 + x4 + x5 + x6 0111 0100011 x + x5 + x6 1111 1001011 1 + x3 + x5 + x6

0001101 1100101 0111001 1010001 0010111 1111111

Trang 288

Gii thiu (tt)

w(i), w(i)(x)

w(i) l t m do dch t m w i bit, v w(i)(x) l a thc m tng ng ca w(i). w(0) chnh l w. i w( i ) w(i)(x) 0 1101000 1 + x + x3 1 0110100 x + x2 + x4 = x * (1 + x + x3) = x * w(x) 2 0011010 x2 + x3 + x5 = x2 (1 + x + x3) = x2 * w(x) 3 0001101 x3 + x4 + x6 = x3 (1 + x + x3) = x3 * w(x) 4 1000110 1 + x4 + x5 = x4 + x5 + x7 mod 7 5 0100011 x + x5 + x6 = x5 + x6 + x8 mod 7 6 1010001 1 + x2 + x6 = x6 + x7 mod 7 + x9 mod 7
Trang 289

Gii thiu (tt)

w(i)(x) = xi * w(x) tuy nhin nu w(i)(x) c xp vi p n th xp c thay bng xp mod n. Mc khc trn trng GF(2) chng ta c xn + j = xj * (xn + 1) + xj hay xn + j mod (xn + 1) = xj w(i)(x) = [xi * w(x)] mod (xn + 1)

B 13.1

Trang 290

Cc tnh cht ca m vng

nh l 13.1

a thc m khc 0 c bc nh nht l duy nht. Hay ni cch khc khng tn ti hai a thc m khc 0, khc nhau v cng c bc nh nht. Gi s hai a thc m khc nhau, cng c bc nh nht l r, 0 < r < n. g(x) = g0 + g1x + + gr1xr - 1 + xr f(x) = f0 + f1x + + fr1xr - 1 + xr T y suy ra a thc m g(x) + f(x) c bc nh hn r, mu thun. Chng minh hon tt.
Trang 291

Chng minh

Cc tnh cht ca m vng (tt)

K hiu a thc m c bc nh nht l g(x) g(x) = g0 + g1x + + gr1xr - 1 + xr H s t do g0 ca g(x) phi bng 1. Gi s g0 = 0. Suy ra g(x) = x * (g1 + + gr1xr - 2 + xr - 1) t f(x) = (g1 + + gr1xr - 2 + xr - 1), suy ra f(x) cng l mt a thc m. V f(x) tng ng vi t m c dch tri 1 bit hay dch phi (n 1) bit t t m ng vi g(x). M bc ca f(x) bng r 1 < r mu thun vi nh ngha ca g(x).
Trang 292

nh l 13.2

Chng minh

Cc tnh cht ca m vng (tt)

nh l 13.3

Mt a thc v(x) trn trng GF(2) c bc n 1 l a thc m nu v ch nu n l mt bi s ca g(x). Tc l n c th vit v(x) = q(x) * g(x).
Chiu thun Nu v(x) = q(x) * g(x) v c bc n 1 th v(x) l a thc m. p p v ( x ) = q ( x ) * g ( x ) = qi x i * g ( x ) = qi x i * g ( x ) i=0 i=0 vi p l bc ca q(x) v p + r n 1. Do xi * g(x) vi 0 i p l a thc m, nn v(x) l a thc m v n l mt t hp tuyn tnh ca cc a thc m.

Chng minh

Trang 293

Cc tnh cht ca m vng (tt)

Chiu ngc Nu v(x) l a thc m th chia v(x) cho g(x) v(x) = q(x) * g(x) + r(x) trong r(x) l a thc d v c bc nh hn bc ca g(x). i vi cc a thc trn trng GF(2) chng ta c th suy ra r(x) = q(x) * g(x) + v(x) Nn r(x) l mt a thc m. Theo nh ngha ca g(x) suy ra r(x) = 0. Chng minh hon tt. T nh l ny chng ta gi g(x) l a thc sinh, v t g(x) c th sinh ra tt c cc a thc m khc.

Trang 294

Cc tnh cht ca m vng (tt)

nh l 13.4

a thc sinh ca mt m vng C(n, k) c bc r = n k. Mi a thc m w(x) l mt bi s ca g(x) w(x) = q(x) * g(x) C 2k t m nn c 2k a thc q(x). Suy ra bc ca q(x) l k 1. Suy ra bc ca g(x) l n k. T nh l ny a thc sinh g(x) c th c biu din nh sau g(x) = g0 + g1x + + gn kxn k trong g0 = gn k = 1.
Trang 295

Chng minh

Cc tnh cht ca m vng (tt)

nh l 13.5

a thc sinh ca m vng C(n, k) l mt c s ca xn + 1. B 13.1 suy ra g(i)(x) = [xi * g(x)] mod (xn + 1) xi * g(x) = q(x) * (xn + 1) + g(i)(x) Chn i = k q(x) = 1 tc xk * g(x) = (xn + 1) + g(i)(x) xn + 1 = xk * g(x) + g(i)(x) Do g(i)(x) l mt a thc m nn g(i)(x) l mt bi ca g(x), xn + 1 l mt bi ca g(x). Chng minh hon tt.
Trang 296

Chng minh

Cc tnh cht ca m vng (tt)

nh l 13.6

Nu g(x) l mt a thc c bc (n k) v l c s ca (xn + 1) th g(x) sinh ra m vng C(n, k), hay ni cch khc g(x) l a thc sinh ca mt m vng C(n, k) no . Xt k a thc g(x), x * g(x), , xk 1 * g(x). Cc a thc ny u c bc n 1. Gi v(x) l mt t hp tuyn tnh ca k a thc ny vi cc h s ai GF(2). v(x) = a0g(x) + a1x * g(x) + + ak 1xk 1 * g(x) v(x) l mt a thc c bc n 1 v l bi s ca g(x).
Trang 297

Chng minh

Cc tnh cht ca m vng (tt)

C tt c 2k t hp tuyn tnh v(x) khc nhau v to nn mt khng gian tuyn tnh ca cc a thc m vi g(x), x * g(x), , xk 1 * g(x) l cc a thc lm c s. Chng ta chng minh rng b m tng ng vi khng gian ny l m vng. Gi w(x) = b0 + b1x + + bn 1xn 1 l mt a thc ca khng gian. Chng ta chng minh w(1)(x) = bn 1 + b0x + b1x2 + + bn 2xn 1 cng l mt a thc ca khng gian.
Trang 298

Cc tnh cht ca m vng (tt)

Theo B 13.1 chng ta c w(1)(x) = [x * w(x)] mod (xn + 1) Da vo biu din ca v(x) v w(1)(x) chng ta suy ra x * w(x) = bn 1(xn + 1) + w(1)(x) Do v(x) v (xn + 1) u l bi ca g(x) nn w(1)(x) cng l bi ca g(x). Suy ra w(1)(x) cng l a thc m. Hon tt chng minh.

Trang 299

Ma trn sinh
1 444 k7 n4 k4 7 +1 4 6444 444 8 6444 4 8 g L 0 0 0 L g g g 1 2 nk 0 0 L 0 0 g 0 g1 L g n k 1 g n k = L 0 0 0 L g g g g 0 nk 2 n k 1 nk M M M M M M M M M 0 L 0 g1 g 2 L g nk g0 0

G k n

V d

Tm mt m vng C(7, 4). Theo cc tnh cht ca m vng suy ra a thc sinh ca m c bc bng 3 v l mt c s ca x7 + 1. Phn tch a thc ny ra tha s chng ta c
Trang 300

V d

x7 + 1 = (1 + x)(1 + x + x3)(1 + x2 + x3) Chn chng hn g(x) = (1 + x + x3)

G47

1 0 = 0 0

1 0 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1

Trang 301

M vng dng h thng

T dng h thng loi 1 chng ta c th dch vng k bit bin i sang dng h thng loi 2 v ngc li.

G47

1 0 = 0 0

1 0 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1

Ght ( 47 )

1 0 = 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 1 1

1 1 1 0

0 1 1 1

M ha thnh t m h thng

u(x) l thng bo, w(x) l t m h thng loi 2 tng ng. xnk * u(x) = q(x) * g(x) + a(x) w(x) = xnk * u(x) + a(x) = q(x) * g(x)
Trang 302

V d

Cho m vng C(7, 4) c ma trn sinh l g(x) = (1 + x + x3). Hy m ho thng bo u = 1010 thnh t m h thng dng 2. u(x) = 1 + x2. Nhn u(x) vi xnk = x3 ri chia cho g(x) chng ta c. x3 * (1 + x2) = x3 + x5 = x2 * (1 + x + x3) + x2 T y suy ra w(x) = x2 + x3 + x5 w = 0011010 l t m h thng dng 2 tng ng vi u.

Trang 303

Ma trn kim tra ca m vng

C mt cch khc tm ma trn kim tra ca m vng xn + 1 = g(x) * h(x) h(x) c gi l a thc i ngu ca g(x). h(x) c bc k h(x) = h0 + h1x + + hkxk Ma trn sau l mt ma trn kim tra ca m vng

H ( n k )n

k7 n k4 +1 7 1 6444 4 44 8 4 4444 8 644 h h L L 0 0 0 h h k 1 k 2 0 k 0 L 0 0 hk hk 1 L h1 h0 = L L 0 0 0 h h h h k 2 1 0 M M M M M M M M M L 0 hk hk 1 hk 2 L h0 0 0 Trang 304

V d

Cho m vng C(7, 4) c ma trn sinh l g(x) = (1 + x + x3). T y suy ra h(x) = (1 + x + x2 + x4) Ma trn kim tra ca b m l

H 37

1 0 1 1 1 0 0 0 1 0 1 1 1 0 = 0 0 1 0 1 1 1

Trang 305

ng dng trng GF(2m) xy dng m vng

nh l 13.7

Cho a l mt phn t khc 0 ca trng GF(2m) c chu k l n, a thc ti thiu f(x) ca a c bc l m. Th m c ma trn sau lm ma trn kim tra l mt m vng C(n, n m), trong mi phn t trong ma trn bn di c thay th bng vect m thnh phn tng ng ca n. Hmn = [1 a a2 an 2 an1] Hn na m vng ny c a thc sinh chnh l f(x). Xt trng GF(24) v a c a thc ti thiu l f(x) = 1 + x + x4
Trang 306

V d

ng dng trng GF(2m) xy dng m vng (tt)

T y suy ra ma trn kim tra ca m vng (15, 11).

H 415

1 0 = 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 1 0 0

0 1 1 0

0 0 1 1

1 1 0 1

1 0 1 0

0 1 0 1

1 1 1 0

0 1 1 1

1 1 1 1

1 0 1 1

1 0 0 1

Nu a thc ti thiu ca a l f(x) = 1 + x + x2 + x3 + x4 th a c chu k l 5 v cc phn t 1, a, ..., a4 c biu din nh sau. a3 = (0001) 1 = (1000) a = (0100) a4 = (1111) a2 = (0010)
Trang 307

ng dng trng GF(2m) xy dng m vng (tt)

T y suy ra ma trn kim tra ca m vng (5, 1)

H 45

1 0 = 0 0

0 0 0 1 1 0 0 1 0 1 0 1 0 0 1 1

Trang 308

M BCH nh phn

Do Bose, Chaudhuri v Hocquenghem sng lp ra. L m vng c kh nng sa c nhiu li. i vi cc s nguyn dng m v t bt k chng ta s xy dng mt m BCH nh phn c cc thng s sau: n = 2m 1 di t m: n k mt S bit kim tra: Khong cch Hamming: dmin 2t + 1

Trang 309

nh l

nh l 13.8

Cho a l mt phn t ca trng GF(2m) c a thc ti thiu l mt a thc cn bn bc m. Th m c ma trn sau lm ma trn kim tra l mt m vng c khong cch Hamming 2t + 1, trong mi phn t trong ma trn bn di c thay th bng vect m thnh phn tng ng ca n.

1 a 3 1 a H = 1 a 5 M M 1 a 2t 1

a 2 ( 2t 1)

a2 a6 a 10 M

L a n2 L a 3( n 2 ) L a 5( n 2 ) M M L a ( 2t 1)(( n 2)
Trang 310

a ( 2t 1)(( n 1) a n 1 a 3( n 1) a 5( n 1) M

nh l (tt)

Hn na a thc sinh g(x) ca b m l a thc bi s chung nh nht ca cc a thc ti thiu ca cc phn t a, a3, a5, , a2t1. Ma trn A sau c nh thc bng ( y i y j ) i> j vi i, j {1, 2, , r}. nh thc ny c gi l nh thc 1 1 L Vandermonde. 1

B 13.2

y 12 A = y1 M r 1 y 1

y2 y2 M y2
2

r 1

yr 2 L yr M M r 1 L yr L

Trang 311

V d

1 a a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a12 a14 H = 3 6 9 12 15 18 21 24 27 30 33 36 39 42 1 a a a a a a a a a a a a a a

Cho m = 4, t = 2 chng ta s xy dng mt m vng c chiu di t m l 24 1 = 15 v c khong cch Hamming d 5. Vic xy dng s da vo trng GF(24). Gi a l phn t c a thc ti thiu l a thc cn bn bc 4 sau f1(x) = 1 + x + x4 y chnh l trng GF(24) trong v d slide 250. a c chu k n = 2m 1 = 15. Chng ta c ma trn kim tra ca b m nh sau.

Thay mi phn t ai bng vect 4 thnh phn tng ng

Trang 312

V d (tt)
1 0 0 0 H= 1 0 0 0 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 0 0 1 1 0 1 0 1 1 1 1 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1

Trang 313

V d (tt)

a thc sinh g(x) l bi s ca hai a thc ti thiu tng ng vi phn t a v a3. Theo v d slide 250, a thc ti thiu ca a3 l f3(x) = 1 + x + x2 + x3 + x4. T y suy ra g(x) = f1(x) * f3(x) = (1 + x + x4) * (1 + x + x2 + x3 + x4) = 1 + x4 + x6 + x7 + x8 Trong trng hp a thc ti thiu ca a khng phi l a thc cn bn, chng ta s tm c m vng c chiu di n 2m + 1, vi n l chu k ca a.
Trang 314

Ch