LTTT Slide v1 PDF

Trng i hc Bch khoa
Khoa Cng Ngh Thng Tin
BI GING MN HC
L THUYT THNG TIN
Ging Vin: H Vn Qun
E-mail: hcquan@dit.hcmut.edu.vn
Web site: http://www.dit.hcmut.edu.vn/~hcquan/student.htm
NI DUNG MN HC
Bi 1
Bi 2
Bi 3
Bi 4
Bi 5
Bi 6
Bi 7
Bi 8
Bi 9
Gii thiu
Mt s khi nim c bn
Chun b ton hc
Lng tin
Entropy
M hiu
M ha ti u ngun ri rc khng nh
M ha ngun ph qut
Knh ri rc khng nh, lng tin tng h
Trang 2
L thuyt Thng tin - Khoa Cng Ngh Thng Tin
NI DUNG MN HC (tt)
Bi 10
Bi 11
Bi 12
Bi 13
Bi 14
Bi 15
M ha chng nhiu, nh l knh

M khi tuyn tnh
C s ton hc ca m ha chng nhiu
M vng
Gii thiu v mt m ha
Mt s vn nng cao
Trang 3
TI LIU THAM KHO

1. Information Theory - Robert B.Ash, Nh xut bn Dover, Inc,
1990.
2. Introduction to Information Theory - Masud Mansuripur, Nh
xut bn PrenticeHall, Inc, 1987.
3. A Mathematical Theory of Communication - C. E. Shannon,
Tp ch Bell System Technical, s 27, trang 379423 v 623
656, thng 7 v thng 10, 1948.
4. C s L thuyt truyn tin (tp mt v hai) - ng Vn
Chuyt, Nguyn Tun Anh, Nh xut bn Gio dc, 1998.
Trang 4
HNH THC NH GI
S c thng bo c th cho tng kha hc. Tuy nhin,

thng l c hnh thc nh bn di.
Thi trc nghim
Gia k: 30 cu / 45 pht
Cui k: 45 cu / 90 pht
c php xem ti liu trong 2 t giy A4
Lm bi tp ln cng im (khng bt buc)
Np bi tp ln v bo co vo cui hc k
Cng ti a 2 im
Trang 5
CC MN LIN QUAN
L thuyt xc sut
K thut truyn s liu
X l tn hiu s
Trang 6
Bi 1 Gii thiu
1.1 Thng tin l g?
1.2 Vai tr ca thng tin
1.3 L thuyt thng tin nghin cu nhng g?
1.4 Nhng ng dng ca l thuyt thng tin
1.5 L thuyt thng tin Lch s hnh thnh v quan im
khoa hc hin i
Trang 7
Thng tin l g?
Mt vi v d
Hai ngi ni chuyn vi nhau. Ci m trao i gia h gi l

thng tin.
Mt ngi ang xem tivi/nghe i/c bo, ngi ang nhn
thng tin t i pht/bo.
Qu trnh ging dy trong lp.
Cc my tnh ni mng v trao i d liu vi nhau.
My tnh np chng trnh, d liu t a cng vo RAM
thc thi.
Trang 8
Thng tin l g? (tt)
Nhn xt
Thng tin l ci c truyn t i tng ny n i tng

khc bo mt iu g . Thng tin ch c ngha khi
iu bn nhn cha bit.
Thng tin xut hin di nhiu dng m thanh, hnh nh, ...
Nhng dng ny ch l v bc vt cht cha thng tin. V
bc l phn xc, thng tin l phn hn.
Ng ngha ca thng tin ch c th hiu c khi bn nhn hiu
c cch biu din ng ngha ca bn pht.
Mt trong nhng phng tin din t thng tin l ngn ng.
C hai trng thi ca thng tin: truyn v lu tr. Mi trng
truyn/lu tr c gi chung l mi trng cha tin hay knh
tin.
Trang 9
Vai tr ca thng tin
Cc i tng sng lun lun c nhu cu hiu v th gii xung

quanh, thch nghi v tn ti. y l mt qu trnh quan st,
tip nhn, trao i v x l thng tin t mi trng xung quanh.
Thng tin tr thnh mt nhu cu c bn, mt iu kin cn cho
s tn ti v pht trin.
Khi KHKT, XH ngy cng pht trin, thng tin cng th hin
c vai tr quan trng ca n i vi chng ta.
V d, hnh ng xut pht t suy ngh, nu suy ngh ng, th
hnh ng mi ng. Suy ngh li chu nh hng t cc ngun
thng tin c tip nhn. V vy thng tin c th chi phi n
suy ngh v kt qu l hnh ng ca con ngi.
Trang 10
LTTT nghin cu nhng vn g?
gc khoa hc k thut, LTTT nghin cu nhm to ra mt

c s h tng tt cho vic truyn thng tin chnh xc, nhanh
chng v an ton; lu tr thng tin mt cch hiu qu.
cc gc nghin cu khc LTTT nghin cu cc vn v
cch t chc, biu din v truyn t thng tin, v tng qut l
cc vn v x l thng tin.
Ba lnh vc nghin cu c bn ca mn hc
M ho chng nhiu
M ho ti u (hay nn d liu)
Mt m ho
Trang 11
Nhng ng dng ca LT thng tin
Cuc cch mng thng tin ang xy ra, s pht trin mnh m
ca cc phng tin mi v truyn thng, lu tr thng tin lm
thay i ngy cng su sc x hi chng ta.
LTTT ng mt vai tr quyt nh trong s pht trin ny bng
cch cung cp c s l thuyt v mt ci nhn trit hc su sc
i vi nhng bi ton mi v thch thc m chng ta chm
trn hm nay v mai sau.
Nhng ng dng ph bin ca LTTT l truyn thng v x l
thng tin bao gm: truyn thng, nn, bo mt, lu tr, ...
Cc tng ca LTTT c p dng trong nhiu lnh vc
nh vt l, ngn ng hc, sinh vt hc, khoa hc my tnh, tm
l hc, ha hc
Trang 12
Nhng ng dng ca LT thng tin (tt)
Mi quan h gia LTTT v thng k c tm thy, cc

phng php mi v phn tch thng k da trn LTTT c
ngh.
ng dng vo qun l kinh t. V d, l thuyt u t ti u
xut hin ng thi vi l thuyt m ha ngun ti u.
ng dng vo ngn ng hc.
ng dng n tm l thc nghim v c bit l lnh vc dy v
hc.
Trang 13
Lch s hnh thnh
Cuc cch mng ln nht v cch nhn th gii khoa hc l

chuyn hng t thuyt quyt nh Laplacian n bc tranh
xc sut ca t nhin.
Th gii chng ta ang sng trong ch yu l xc sut. Kin
thc ca chng ta cng l mt dng xc sut.
LTTT ni ln sau khi c hc thng k v lng t pht trin,
v n chia x vi vt l thng k cc khi nim c bn v
entropy.
Theo lch s, cc khi nim c bn ca LTTT nh entropy,
thng tin tng h c hnh thnh t vic nghin cu cc h
thng mt m hn l t vic nghin cu cc knh truyn thng.
V mt ton hc, LTTT l mt nhnh ca l thuyt xc sut v
cc qu trnh ngu nhin (stochastical process).
Trang 14
Lch s hnh thnh (tt)
Quan trng v c ngha nht l quan h lin kt gia LTTT v

vt l thng k.
Trong mt thi gian di trc khi LTTT c hnh thnh, L.
Boltzman v sau l L.Szilard nh ng ngha ca
thng tin vi khi nim nhit ng hc ca entropy. Mt mt
khc, D. Gabor ch ra rng l thuyt truyn thng phi c
xem nh mt nhnh ca vt l.
C. E. Shannon l cha ca LTTT.
Trang 15
Bi 2 Mt s khi nim c bn
2.1 Thng tin (Information)
2.2 M hnh ca cc qu trnh truyn tin
2.3 Cc loi h thng truyn tin Lin tc v ri rc
2.4 Ri rc ho
Trang 16
Thng tin
Thng tin l mt khi nim tru tng, phi vt cht v rt kh

c nh ngha chnh xc. Hai nh ngha v thng tin.
Thng tin l s cm hiu ca con ngi v th gii xung quanh
thng qua s tip xc vi n.
Thng tin l mt h thng nhng tin bo v mnh lnh gip loi
tr s khng chc chn (uncertainty) trong trng thi ca ni
nhn tin. Ni ngn gn, thng tin l ci m loi tr s khng
chc chn.
nh ngha u cha ni ln c bn cht ca thng tin. nh
ngha th hai ni r hn v bn cht ca thng tin v c dng
nh lng thng tin trong k thut.
Trang 17
Thng tin (tt)
Thng tin l mt hin tng vt l, n thng tn ti v c

truyn i di mt dng vt cht no .
Nhng dng vt cht dng mang thng tin c gi l tn
hiu.
L thuyt tn hiu nghin cu cc dng tn hiu v cch truyn
thng tin i xa vi chi ph thp, mt ngnh m c quan h gn
gi vi LTTT.
Thng tin l mt qu trnh ngu nhin.
Tn hiu mang tin tc cng l tn hiu ngu nhin v m hnh
ton hc ca n l cc qu trnh ngu nhin thc hay phc.
V LTTT l l thuyt ngu nhin ca tin tc, c ngha l n xt
n tnh bt ng ca tin tc i vi ni nhn tin.
Trang 18
M hnh ca cc qu trnh truyn tin
Khi nim thng tin thng i km vi mt h thng truyn tin.

Nhiu
Ngun pht
Ngun nhn
S truyn tin (transmission)
Knh truyn
L s dch chuyn thng tin t im ny n im khc trong

mt mi trng xc nh.
Ngun tin (information source)
L mt tp hp cc tin m h thng truyn tin dng lp cc

bng tin hay thng bo (message) truyn tin.
Bng tin chnh l dy tin c bn pht truyn i.
Thng tin c th thuc nhiu loi nh
(1) mt dy k t nh trong in tn (telegraph) ca cc h thng gi in
tn (teletype system);
Trang 19
M hnh ca cc qu trnh truyn tin (tt)

(2) mt hm theo ch mt bin thi gian f(t) nh trong radio v in thoi;
(3) mt hm ca thi gian v cc bin khc nh trong tivi trng en
y thng tin c th c ngh nh l mt hm f(x, y, t) ca to hai
chiu v thi gian biu din cng nh sng ti im (x, y) trn mn
hnh v thi gian t;
(4) mt vi hm ca mt vi bin nh trong trng hp tivi mu y
thng tin bao gm ba hm f(x, y, t), g(x, y, t), h(x, y, t) biu din cng
nh sng ca cc ba thnh phn mu c bn (xanh l cy, , xanh
dng)
Thng tin trc khi c truyn i, tu theo yu cu c th

c m ho nn, chng nhiu, bo mt, ...
Knh tin (channel)
L ni hnh thnh v truyn (hoc lu tr) tn hiu mang tin

ng thi y xy ra cc tp nhiu (noise) ph hy tin tc.
Trong LTTT knh l mt khi nim tru tng i biu cho
hn hp tn hiu v tp nhiu.
Trang 20
Mt s khi nim (tt)
Mi trng truyn tin thng rt a dng
mi trng khng kh, tin c truyn di dng m thanh v ting ni,

ngoi ra cng c th bng la hay bng nh sng;
mi trng tng in ly trong kh quyn ni m thng xuyn xy ra s
truyn tin gia cc v tinh nhn to vi cc trm rada di mt t;
ng truyn in thoi ni xy ra s truyn tn hiu mang tin l dng
in hay ng truyn cp quang qua bin trong tn hiu mang tin l
sng nh sng v.v
Nhiu (noise)
Cho d mi trng no cng c nhiu. Nhiu rt phong ph v

a dng v thng i km vi mi trng truyn tin tng ng.
Chng hn nu truyn di dng sng in t m c i qua cc vng ca

tri t c t trng mnh th tn hiu mang tin thng b nh hng t
nhiu bi t trng ny. Nn c th coi t trng ny l mt loi nhiu.
Nu truyn di dng m thanh trong khng kh th ting n xung quanh
c th coi l mt loi nhiu.
Trang 21
Mt s khi nim (tt)
Nhiu c nhiu loi chng hn nhiu cng, nhiu nhn.

Nhiu cng l loi nhiu m tn hiu mang tin b tn hiu nhiu
cng thm vo.
Nhiu nhn l loi nhiu m tn hiu mang tin b tn hiu nhiu
nhn ln.
Ni nhn tin (sink)
L ni tip nhn thng tin t knh truyn v c gng khi phc

li thnh thng tin ban u nh bn pht pht i.
Tin n c ni nhn thng khng ging nh tin ban u v
c s tc ng ca nhiu. V vy ni nhn phi thc hin vic
pht hin sai v sa sai.
Ni nhn cn c th phi thc hin vic gii nn hay gii m
thng tin c m ho bo mt nu nh bn pht thc
hin vic nn hay bo mt thng tin trc khi truyn
Trang 22
Cc loi h thng truyn tin
Cc ngun tin thng thy trong t nhin c gi l cc ngun

tin nguyn thu. y l cc ngun tin cha qua bt k mt php
bin i nhn to no.
Cc tn hiu m thanh, hnh nh c pht ra t cc ngun tin
nguyn thu ny thng l cc hm lin tc theo thi gian v
theo mc, ngha l c th biu din mt thng tin no di
dng mt hm s(t) tn ti trong mt qung thi gian T v ly
cc tr bt k trong mt phm vi (smin, smax) no .
s(t)
smax
smin
t
Trang 23
Cc loi h thng truyn tin (tt)
Cc ngun nh vy c gi l cc ngun lin tc (continuous

source), cc tin c gi l tin lin tc (continuous information)
v knh tin c gi l knh lin tc (continuous channel).
Tuy nhin vn c nhng ngun nguyn thu l ri rc
Bng ch ci ca mt ngn ng.

Cc tin trong h thng in tn, cc lnh iu khin trong mt h thng
iu khin, ...
Trong trng hp ny cc ngun c gi l ngun ri rc

(discrete source), cc tin c gi l tin ri rc (discrete
information) v knh tin c gi l knh ri rc (discrete
channel).
S phn bit v bn cht ca tnh ri rc v tnh lin tc l s
lng tin ca ngun trong trng hp ri rc l hu hn cn
trong trng hp lin tc l khng m c.
Trang 24
Ri rc ha
Cc h thng lin tc c nhiu nhc im ca nh cng knh,

khng hiu qu, v chi ph cao.
Cc h thng truyn tin ri rc c nhiu u im hn, khc
phc c nhng nhc im trn ca cc h thng lin tc v
c bit ang ngy cng c pht trin v hon thin dn
nhng sc mnh v u im ca n.
Ri rc ho thng bao gm hai loi: Ri rc ho theo trc thi
gian, cn c gi l ly mu (sampling) v ri rc ho theo
bin , cn c gi l lng t ho (quantize).
Ly mu (Sampling)
Ly mu mt hm l trch ra t hm ban u cc mu c ly
ti nhng thi im xc nh.
Vn l lm th no s thay th hm ban u bng cc mu
ny l mt s thay th tng ng, iu ny c gii
quyt bng nh l ly mu ni ting ca Shannon.
Trang 25
Ri rc ha (tt)
nh l ly mu ca Shannon
Mt hm s(t) c ph hu hn, khng c thnh phn tn s ln

hn max (= 2fmax) c th c thay th bng cc mu ca n
c ly ti nhng thi im cch nhau mt khong t
/max, hay ni cch khc tn s ly mu F 2fmax.
s(t)
smax
smin
t
Trang 26
Ri rc ha (tt)
Lng t ho (Quantize)
Bin ca cc tn hiu thng l mt min lin tc (smin, smax).

Lng t ho l phn chia min ny thnh mt s mc nht
nh, chng hn l smin = s0, s1, ..., sn = smax v qui cc gi tr
bin khng trng vi cc mc ny v mc gn vi n nht.
Vic lng t ho s bin i hm s(t) ban u thnh mt hm
s(t) c dng hnh bc thang. S khc nhau gia s(t) v s(t)
c gi l sai s lng t. Sai s lng t cng nh th s(t)
biu din cng chnh xc s(t).
s(t)
smax
smin
Trang 27
Ngun ri rc
Ngun ri rc
Ngun tin lin tc sau khi c ly mu v lng t ho s tr

thnh ngun ri rc.
Chng ta hc ch yu cc ngun ri rc.
Mt ngun ri rc l mt bng ch ci A gm m k hiu, A =
{a1, a2, ..., am}, vi nhng xc sut xut hin p(ai), i = 1, .., m.
nh ngha khng din t mi quan h gia tin trc v sau
trong mt bn tin, nn y c gi l mt ngun ri rc khng
nh (discrete memoryless source).
Bng tin ca mt ngun tin ri rc khng nh
L mt dy (c th v hn) cc k hiu lin tip t bng ch ci

ca ngun tin, x = (... a2a1a0a1a2...)
Trong thc t bng tin c bt u v kt thc cho nn bng tin
l mt dy hu hn cc k hiu, x* = (a1a2 an)
Trang 28
Bi 3 Chun b ton hc
3.1 Xc sut (Probability)
3.2 Bt ng thc Chebyshev v lut yu ca s ln
3.3 Tp li (Convex sets) v hm li (convex functions), bt
ng thc Jensen
3.4 Cng thc Stirling
Trang 29
Xc sut
Khng gian mu (Sample space)
S kin (Event), s kin c bn (elementary event)
L tp (hay khng gian) tt c cc kt qu c th c ca mt th

nghim. Thng c k hiu l E hay S. Nu khng gian mu
l ri rc th E c th c biu din bng E = {e1, e2, ..., en}
Mi tp con ca E (khng gian mu) c gi l mt s kin,
c bit mi phn t ca E c gi l mt s kin c bn.
V d
Trong mt th nghim tung ng xu th E = {U (p), N (nga)}.

Nu ng tin l ng nht th xc sut P(U) = P(N) = 1/2.
Trong mt th nghim tung con xc xc th E = {1, 2, 3, 4, 5,
6}. Nu con xc xc l ng nht th xc sut P(1) = P(2) =
P(3) = P(4) = P(5) = P(6) = 1/6, P(2, 5) = 1/3, P(1, 3, 5) = 1/2.
Trang 30
Xc sut (tt)
Bin ngu nhin ri rc (Discrete random variable)
Ly mt vn bn ting Anh in hnh v nht mt k t bt k

th E = {a, b, c, ..., x, y, z} v xc sut ca cc k t c phn
b nh sau P(a) = 0,0642 , ..., P(e) = 0,103 , ..., P(z) = 0,0005.
Mt bin ngu nhin ri rc x c nh ngha bng cch gn
mt s thc xi ti mi s kin c bn ei ca khng gian mu ri
rc E. Xc sut ca xi c nh ngha l xc sut ca s kin
c bn tng ng v c k hiu l p(xi).
Tr trung bnh (k vng) (average, expected value),

phng sai (variance)
Tr trung bnh v phng sai ca bin ngu nhin ri rc x ln

lt c k hiu v nh ngha nh sau
E(x) = x = x i p (x i )
i
Trang 31
Var(x)
Xc sut (tt)
2
2
= E ((x x ) ) = (x i x ) p(x i )
( )
= E x x
trong E(x2) l tr k vng ca x2.
Tng qut, tr k vng ca mt hm ca x, chng hn f(x), c
nh ngha bng
2
E ( f (x )) = f (x i ) p(x i )
i
Xc sut ng thi (joint probability), xc sut c iu

kin (conditional probability)
Mt cp bin ngu nhin (x, y) lin kt vi mt th nghim to

thnh mt bin ngu nhin ni (joint random variable). Nu x, y
l ri rc, s phn b xc sut ni hay xc sut ng thi c
nh ngha l
pij = P(x = xi, y = yj)
Trang 32
Xc sut (tt)
Xc sut ca y trong iu kin bit x c gi l xc sut c

iu kin v c nh ngha l
p ( y j xi ) =
p (xi , y j )
p( xi )
trong xc sut l (marginal probability) p(xi) c gi thit

l khc khng.
Cc xc sut l c nh ngha nh sau:
p(xi) = p xi , y j
p(yj) =
p(x , y )
i
Trang 33
V d
Xc xc
Th nghim tung ng thi
6
mt ng xu v con xc xc.
5
T kt qu trn ta thy
4
P(U, 5) = 1/18
3
P(ng xu = U) = 5/9
P(ng xu = N) = 4/9
2
P(Xc xc = 5) = 7/72
1
P(Xc xc = 5 bit ng xu = U)
Trang 34
1/12
1/12
1/18
1/24
1/9
1/24
1/9
1/6
1/9
1/12
1/18
1/18
N ng xu
Xc sut (tt)
S c lp (Independence)
Hai bin ngu nhin x v y c gi l c lp nu

p(xi, yj) = p(xi)p(yj) i, j.
Chng ta thy nu hai bin x v y c lp th
p (xi , y j ) p ( xi ) p ( y j )
p y j xi =
=
= p(y j )
p ( xi )
p ( xi )
c ngha l xc sut yj trong iu kin c xi xy ra hay khng
xy ra u nh nhau, khng thay i, v ngc li.
Cng t s c lp chng ta suy ra mt kt qu m hay c s
dng sau ny
E(xy) = E(x) E(y) = x y
Trang 35
Xc sut (tt)
S tng quan (correlation)
S tng quan C gia hai bin x v y c nh ngha l tr k

vng ca (x x )(y y):
C(x, y) = E((x x )(y y )) =
= E(xy) x y
Trong trng hp x v y l c lp chng ta suy ra C(x, y) = 0.
Tuy nhin iu ngc li th khng ng.
Trang 36
Bt ng thc Chebyshev
v lut yu ca s ln
Bt ng thc Chebyshev
Cho mt bin ngu nhin x c tr trung bnh l x v phng sai

l x2, bt ng thc Chebyshev i vi mt s dng tu l
Chng minh
x2
P(|x x | ) 2
1,|x - x|
nh ngha mt hm f(x) nh sau f (x ) =
0 ,|x - x| <
Th
P(|x x| ) = f(xi)p(xi)
Trang 37
Bt ng thc Chebyshev (tt)

xx
Da trn hnh chng ta c
V vy,
x +
xx
f(x)

2
2
xx
p (x i ) = x2

P x x
i Trang
38
Lut yu ca s ln (tt)
Xt mt th nghim nh phn trong cc kt qu ca th

nghim l 0 v 1 vi cc xc sut tng ng l p0 v 1 p0.
Th nghim ny c lp li N ln mt cch c lp, v kt qu
trung bnh c nh ngha l yN; tc l, yN bng tng s cc s
1 trong N ln th nghim chia cho N.
R rng, yN l mt bin ngu nhin c khng gian mu l {0,
1/N, 2/N, ..., 1}.
nh ngha x(n) l bin ngu nhin tng ng vi kt qu ca
ln th nghim th n, chng ta c
1
yN =
N
(n )
x
n =1
Trang 39
Lut yu ca s ln (tt)
1
yN =
N
((
= E yN yN
2
y
))
2
E (x )
N
(n )
n =1
1
= E
N
1
=
N
x
n =1
(n )
x =x
n =1
2
2
N
1 N
1

(n )
(
n)
= E x N x = 2 E x x
N n =1
N n =1
1
= 2
N
((
(n )
x
x
E
n =1
))
2
2
1
= 2 N x2 = x
N
N
Trang 40
Lut yu ca s ln (tt)
i vi mt s nguyn dng tu , theo bt ng thc

Chebyshev chng ta c
y2
P | y N y N | 2
t y chng ta dn ra c lut yu ca s ln
1
P
N
(n )
x
n =1
Ch rng v phi tin ti 0 khi N tin ra v cng.

Lut yu ca s ln v vy khng inh rng tr trung bnh mu
ca x tip cn tr trung bnh thng k vi xc sut cao khi N
.
Trang 41
Tp li
Trong khng gian clit, mt tp S c gi l li (convex cap

()) nu i vi mt cp im P1, P2 thuc S th mi im
thuc on P1P2 cng thuc S.
P1
P1
P2
P2
(b)
(a)
Nu P1 = (x1, x2, ..., xn) v P2 = (y1, y2, ..., yn) l cc im trong
khng gian clit n chiu, th on thng ni chng c biu
din bng tp cc im P, trong
P = P1 + (1)P2
= (x1 + (1)y1, x2 + (1)y2, ..., xn + (1)yn) v [0, 1].
Trang 42
Hm li
Mt v d quan trng ca tp li l tp tt c cc im (p1, p2,

..., pn) trong (p1, p2, ..., pn) l mt s phn b xc sut (tc l
cc pi [0, 1] v pi = 1).
Mt hm thc f(P), c nh ngha trn tp li S, c gi l
li nu cp im P1, P2 S, v [0, 1] bt ng thc sau
y ng:
f(P1 + (1)P2) f(P1) + (1)f(P2)
f((x1 + (1-)x2)
f(x)
f(x2)
f(x1)
f(x1) + (1-)f(x2)
x1 (x1 + (1-)x2
x2
Trang 43
nh l, bt ng thc Jensen
Nu 1, ..., N l cc s khng m c tng bng 1 th i vi

mi tp im P1, ..., PN trong min xc nh ca hm li f(P)
bt ng thc sau y ng
N
N
f n Pn n f (Pn )
n =1
n =1
Cho bin ngu nhin x ly cc gi tr x1, ..., xn vi cc xc sut

p1, ..., pn. Cho f(x) l mt hm li c min xc nh cha x1, ...,
xn. Chng ta c E(x) = pi xi v E(f(x)) = pi f ( xi ) .
i
i
p dng nh l trn chng
ta c
f(E(x)) E(f(x))
y c gi l bt ng thc Jensen.
Trang 44
Bi 4 Lng tin
4.1 Lng tin
4.2 Lng tin trung bnh
Vn c bn ca truyn thng l vic ti sinh ti mt im hoc
chnh xc hoc gn ng mt thng bo c chn ti mt im
khc.
(Claude Shannon 1948)
Trang 45
Lng tin
Lng tin (measure of information) dng so snh nh lng

cc tin tc vi nhau.
Mt tin i vi ngi nhn u mang hai ni dung, mt l
bt ng ca tin, hai l ngha ca tin.
Kha cnh ng ngha ch c ngha i vi con ngi.
Kha cnh quan trng nm ch tin tht s l mt ci c
chn t mt tp cc tin (tp cc kh nng) c th.
Nu s tin trong tp tin cng nhiu th s mang li mt lng
tin cng ln khi nhn c mt tin (gi s cc tin l bnh ng
nh nhau v kh nng xut hin).
s truyn tin t hiu qu cao chng ta khng th i i
cc tin nh nhau nu chng xut hin t nhiu khc nhau.
Trang 46
Lng tin
Xt mt tin x c xc sut xut hin l p(x), th chng ta c th

xem tin ny nh l mt tin trong mt tp c 1/p(x) tin vi cc
tin c xc sut xut hin nh nhau.
Nu p(x) cng nh th 1/p(x) cng ln v v vy lng tin khi
nhn c tin ny cng s cng ln.
Vy lng tin ca mt tin t l thun vi s kh nng ca mt
tin v t l nghch vi xc sut xut hin ca tin .
Xc sut xut hin ca mt tin t l nghch vi bt ng khi
nhn c mt tin.
lng tin s kh nng bt ng xc sut
Mt tin c xc sut xut hin cng nh th c bt ng cng
ln v v vy c lng tin cng ln.
Trang 47
Lng tin (tt)
Xt mt ngun A = {a1, a2,, am} vi cc xc sut xut hin l

p(ai) i = 1, ..., m.
K hiu lng tin trong mi tin ai l I(ai). Vy hm f dng
biu th lng tin phi tho mn nhng iu kin g?
Phn nh c cc tnh cht thng k ca tin tc.
V d c hai ngun K, L vi s tin tng ng l k, l (gi thuyt u l

ng xc sut). Nu k > l, th bt ng khi nhn mt tin bt k ca
ngun K phi ln hn bt ng khi nhn mt tin bt k ca ngun L,
vy
f(k) > f(l)
Hp l trong tnh ton.
Gi thit hai ngun c lp K v L vi s tin tng ng l k v l. Cho

vic nhn mt cp ki v lj bt k ng thi l mt tin ca ngun hn hp
KL. S cp kilj m ngun ny c l k*l.
Trang 48
Lng tin (tt)
bt ng khi nhn c mt cp nh vy phi bng tng lng tin ca

khi nhn c ki v lj. V vy chng ta phi c:
f(kl) = f(k) + f(l)
Khi ngun ch c mt tin, lng tin cha trong tin duy nht
phi bng khng.
f(1) = 0
nh ngha
Lng o thng tin ca mt tin c o bng logarit ca bt

ng ca tin hay nghch o xc sut xut hin ca tin .
1
I ( x ) = log
= log p( x)
p ( x)
Trang 49
Lng tin (tt)
Lng tin cha trong mt dy x = a1a2 an vi ai A l

n
1
I ( x ) = log
= log p(ai )
p( x)
i =1
Trong trng hp m k hiu ca ngun ng xc sut vi nhau
tc p(ai) = 1/m th
1
I (ai ) = log
= log m
p(ai )
Nu x = a1a2 an vi ai A
I(x) = n logm
Trang 50
Lng tin trung bnh
n v ca lng tin
Nu c s l 2 th n v l bits (cho cc k s nh phn); nu c

s l e th n v l nats (cho n v t nhin), nu c s l 10
th n v l Hartley.
nh ngha
Lng tin trung bnh ca mt ngun tin A l lng tin trung

bnh cha trong mt k hiu bt k ca ngun tin. N thng
c k hiu l I(A) v c tnh bng cng thc sau
I ( A) =
p(ai ) I (ai ) = p(ai ) log p(ai )
ai A
ai A
Trang 51
V d
Cho mt ngun tin U bao gm 8 tin U = {u0, u1, u2, u3, u4, u5,
u6, u7}, vi cc xc sut xut hin nh sau:
p(u0) p(u1) p(u2) p(u3) p(u4) p(u5) p(u6) p(u7)
1/4 1/4 1/8 1/8 1/16 1/16 1/16 1/16
Hy cho bit lng tin ring ca mi tin v lng tin trung bnh
ca ngun ny trong n v bits.
Gii
Lng tin ring ca mi tin l

I(u0) I(u1) I(u2) I(u3) I(u4) I(u5) I(u6) I(u7)
2
2
3
3
4
4
4
4
Trang 52
V d (tt)
Lng tin trung bnh ca ngun l

I(U) = (1/4) 2 + (1/4) 2 + (1/8) 3 + (1/8) 3 + (1/16) 4
+ (1/16) 4 + (1/16) 4 + (1/16) 4 = 2,75 bits.
iu ny ni ln mt ngha quan trng rng, chng ta c th
biu din mi tin trong ngun U bng mt chui c chiu di
trung bnh l 2,75 bits. N s tt hn so vi trong trng hp
chng ta khng ch n cu trc thng k ca ngun. Lc
chng ta s biu din mi tin trong 8 tin ca ngun bng cc
chui c chiu di l 3 bits.
Trang 53
Bi 5 Entropy
5.1 Entropy ca mt bin ngu nhin ri rc
5.2 Cc c tnh ca entropy
5.3 Entropy v cc dy ca mt bin ngu nhin
Trang 54
Entropy ca mt bin ngu nhin ri rc
nh ngha
Cho x l mt bin ngu nhin vi khng gian mu X = {x1, ... ,

xN} v o xc sut P(xn) = pn. Entropy ca x c nh ngha
l:
N
H (x ) = pn log( pn )
n =1
p ln(p)
e-1
e-1 = 0,37
Trang 55
Entropy ca mt bin ngu nhin ri rc (tt)
V d
Cho X = {0, 1}, P(0) = p, cn P(1) = 1p. Th

H(x) = plog(p) (1 p) log(1 p)
H(x)
1
0,5
Trang 56
Cc c tnh ca entropy
1. Entropy l mt i lng lun lun dng hoc bng khng.
H(x) = 0 c mt xc sut pi = 1, cn tt c cc xc sut cn li bng 0.

iu ny ni ln rng bt ng v mt th nghim ch c mt kt qu
duy nht l bng 0.
2. H(x) log N v du bng xy ra p1 = p2 = ... = pN = 1/N.

Hay ni cch khc entropy t cc i khi xc sut xut hin
ca cc k hiu bng nhau.
Chng minh
H ( x ) ln( N ) = pn ln ( pn ) pn ln ( N ) = pn ln
n =1
n =1
n =1
Npn
N
1
N 1 N
pn
1 = pn = 1 1 = 0
n =1
Npn n =1 N n =1
N
Trang 57
Cc c tnh ca entropy (tt)

3. Cho bin ngu nhin x c khng gian mu X = {x1, ..., xN} v
bin ngu nhin y c khng gian mu Y = {y1, ..., yM}. Th bin
ngu nhin ni z = (x, y) c khng gian mu Z = {(x1, y1), ...,
(x1, yM), (x2, y1), ..., (x2, yM), ..., (xN, y1), ..., (xN, yM)} gm NM
phn t. Nu x, y c lp nhau th H(z) = H(x) + H(y).
Chng minh
N M
N M
n=1 m=1
N
n=1 m=1
H(z) = P(xn , ym ) logP(xn , ym ) = P(xn )P( ym )[logP(xn ) + logP( ym )]

M
m=1
m=1
n=1
= P(xn ) logP(xn )P( ym ) P(xm ) logP(xm )P( yn )

n=1
= H (x) + H (y)
Trang 58

4. Xt mt bin ngu nhin x c khng gian mu X = {x1, ..., xn,
xn+1, ..., xN} v cc xc xut p(xi) = pi. Chng ta phn X thnh
hai khng gian con, Y = {x1, ..., xn} v Z = {xn+1, ..., xN}. Cc
n
xc sut lin kt vi Y v Z c cho bi P(Y) = i =1 pi
N
v P(Z) = i = n +1 pi . Hn na, chng ta nh ngha cc bin
ngu nhin y v z bng P(yi) = P(xi)/P(Y), i = 1, 2, ..., n v P(zi)
= P(xi)/P(Z), i = n+1, n+2, ..., N. H(x) by gi c th c vit
thnh
N
H ( x ) = pi log pi = pi log pi
i =1
i =1
p log p
i = n +1
i=1
i =n+1
= P(Y )P( yi )(logP( yi ) + logP(Y )) P(Z ) P(zi )(logP(zi ) + logP(Z ))
= [P(Y )log P(Y ) + P(Z )log P(Z )] + [P(Y )H ( y) + P(Z )H ( z)]

Trang 59
Trong biu thc cui cp ngoc vung u biu din bt ng lin kt

vi th nghim th nht (l chn mt trong hai khng gian mu Y v Z)
cn cp ngoc vung th hai biu din bt ng trung bnh lin kt vi
th nghim th hai (sau khi chn mt trong hai khng gian mu, s
chn tip s kin c bn no). Cng thc ny din t mt tnh cht ca
entropy l tnh cht nhm.
Ngi ta chng minh c rng cng thc nh ngha ca

H(x) l cng thc duy nht ph hp o v bt ng, ci m
phi tho mn cc tnh cht 2,3, 4 v cng thm tnh lin tc.
Mc du hai khi nim lng tin trung bnh v entropy xut
hin mt cch c lp v trong nhng lnh vc khc nhau
(entropy vn xut pht t vic nghin cu cc qu trnh nhit
ng) nhng chng c cng cng thc ging nhau. V vy
chng ta c th xem lng tin trung bnh ca mt ngun chnh
l entropy ca ngun .
Trang 60
Entropy v cc dy ca
mt bin ngu nhin
V d
Xt mt bin ngu nhin x c khng gian mu X = {x1, x2},

P(x1) = p1 = 1/3, P(x2) = 2/3. Th entropy ca x l
H(x) = (1/3) log(1/3) (2/3) log(2/3) = 0.918295834 bits
Chng ta hy lp li th nghim ny N ln nhn mt dy N
phn t. Tng qut c n 2N dy c th. Nu trong dy c n
phn t x1 th xc sut xut hin ca dy l p1n(1p1)Nn
C (nN) = N! dy nh vy, nn tng xc sut ca chng
n!( N n)!
bng
n
N
N-n
( n ) p1 (1-p1 )
Bng bn di trnh by xc sut ca cc dy khc nhau i vi

N = 15
Trang 61
Entropy v cc dy ca
mt bin ngu nhin (tt)
S dy P mi dy
S dy P mi dy
P
tng
cng
P tng cng
n N
n
(n ) p1n(1p1)Nn (nN ) p1n(1p1)Nn
(nN ) p1n(1p1)Nn (nN ) p1n(1p1)Nn
0
1 215x0.584962501
0.002284
8 6435 215x1.118295834
0.057404
1
15 215x0.651629167
0.017127
9 5005 215x1.184962501
0.022324
2
105 215x0.718295834
0.059946
10 3003 215x1.251629167
0.006697
3
455 215x0.784962501
0.129883
11 1365 215x1.318295834
0.001522
4 1365 215x0.851629167
0.194825
12
455 215x1.384962501
0.000254
5 3003 215x0.918295834
0.214307
13
105 215x1.451629167
0.000029
6 5005 215x0.984962501
0.178589
14
15 215x1.518295834
0.000002
7 6435 215x1.051629167
0.114807
15
1 215x1.584962501
0.000000
Trang 62
Nhn xt
Nhng dy c xc sut ln (dy c kh nng) l nhng dy m

c n gn vi gi tr Np1 = 5, c th l 2 n 8. Ni cch khc,
Xc sut xut hin ca mt dy m c n nm xa gi tr Np1 l
rt nh.
Xsut ring ca nhng dy c kh nng nm gia 2150.718295834
v 215 1.118295834, ci m gn st vi 2NH(x) = 2150.918295834.
Ni cch khc,
Tt c nhng dy c kh nng l nhiu hay t ng xc sut vi
xc sut 2NH(x).
S lng tng cng cc dy kh nng (2 n 8) l 22803 =
215 0.965129067 ci m khng xa so vi 2NH(x). Ni cch khc,
S lng cc dy c kh nng l khong 2NH(x).
Trang 63
nh l
nh l 5.1
Cho cc s > 0 v > 0 nh tu , mt s nguyn dng N0

sao cho mt dy c chiu di bt k N N0 s ri vo mt trong
hai lp sau y:
(1) Mt tp cc dy m c tng xc sut ca chng nh hn
hoc bng .
(2) Tp cn li bao gm cc dy c xc sut tho mn bt ng
thc
NH A N
NH + A N
< p<2
vi A l mt s dng no . Hay ni cch khc,

log p 1
H
N
Trang 64
Chng minh nh l
Chng minh cho ngun ri rc khng nh A = {a1, a2, ..., aK}.

Gi x l bin ngu nhin gn vi ngun A. Ta c
K
H ( x ) = p (ak ) log p (ak )

k =1
Gi y l bin ngu nhin bng cch nh x mi ai ti log p(ai).

K
y = p (ai ) log p (ai ) = H ( x )

i =1
Xt cc dy c chiu di N. C tt c KN dy nh vy. Ta k
hiu cc dy ny bng cc Si v xc sut ca dy l P(Si). Ta c
N
P( Si ) = p(a ( j ) )
j =1
trong a(j) l k hiu th j ca dy.

Trang 65
Chng minh nh l
Gi z l bin ngu nhin bng cch nh x mi Si ti -log P(Si).

N
Ch
log P ( S ) = log p (a ( j ) )
i
j =1
V vy z l tng ca N bin ngu nhin y c lp.

p dng lut yu ca s ln cho hai s > 0 v > 0 nh tu ,
tn ti N0 sao cho vi mi N N0
(
j)
y
P
y
N j = 1
hay
1 N
(
)
j
P log p(a ) H (x)
N j =1
Trang 66
Chng minh nh l (tt)
Hay
P log P(S i ) H ( x )
N
V vy chng ta c th kt lun rng vi xc sut ln hn 1

1
log P ( S i ) H ( x )
N
i vi mi N N0.
T y ta suy ra rng cc dy c chia thnh hai nhm, mt
nhm c tng xc xut nh hn hoc bng v nhm th hai
bao gm cc dy tho iu kin .
1
1
H (x)
log
N
P( Si )
V vy nh l c chng minh.
Trang 67
Bi 6 M hiu
6.1 Gii thiu
6.2 M hiu v cc thng s c bn ca m hiu
6.3 Mt s phng php biu din m
6.4 iu kin phn tch m
Trang 68
Gii thiu
Trong cc h thng truyn tin, bn nhn thng bit tp hp cc

tin m bn pht dng lp nn cc bn tin.
Cc tin thng s c nh x (m ha) thnh mt dng biu
din khc thun tin hn pht i.
V d
Xt mt ngun tin A = {a, b, c, d}. Chng ta c th thit lp

mt song nh nh sau t A vo tp cc chui trn bng ch ci
{0, 1}
a 00
c 10
b 01
d 11
Vy pht i bn tin baba chng ta pht i chui 01000100.
Khi bn nhn nhn c chui ny th xc nh c bn tin
bn pht pht i l baba.
Trang 69
M hiu v nhng thng s c bn
M hiu (Code), c s m
M hiu l mt tp hu hn cc k hiu v php nh x cc

tin/bn tin ca ngun tin thnh cc dy k hiu tng ng. Tp
cc k hiu v php nh x ny thng s phi p ng cc yu
cu ty theo h thng truyn tin t ra.
Tp cc k hiu m dng biu din c gi l bng k hiu
m, cn s cc k hiu th c gi l c s m, v thng k
hiu l m. Nu m c c s hai th gi l m nh phn, cn nu
m c c s ba th gi l m tam phn ...
M ho (Encoding), gii m (decoding)
M ho l qu trnh dng cc k hiu m biu din cc tin

ca ngun.
Trang 70
M hiu v nhng thng s c bn (tt)
Ni cch khc m ho l mt php bin i t ngun tin thnh

m hiu, hay m ho l php bin i t mt tp tin ny thnh
mt tp tin khc c c tnh thng k yu cu.
Qu trnh ngc li ca qu trnh m ho c gi l gii m.
T m (Code word), b m
T m l chui k hiu m biu din cho tin ca ngun. Tp tt

c cc t m tng ng vi cc tin ca ngun c gi l b
m.
V vy c th ni m ho l mt php bin i mtmt gia
mt tin ca ngun v mt t m ca b m.
Trong mt s trng hp ngi ta khng m ho mi tin ca
ngun m m ho mt bn tin hay khi tin. Lc ny chng ta c
khi nim m khi.
Trang 71
Chiu di t m, chiu di trung bnh
Cc t m thng c k hiu l u, v, w.
Chiu di t m l s k hiu c trong t m thng c k
hiu l l. Chiu di trung bnh ca b m thng c k hiu l
l v c cho bng cng thc
n
l = p ( xi )li
i =1
trong n l s tin ca ngun cn li l chiu di t m tng
ng vi tin xi ca ngun.
Phn loi m: m u, m y, m vi
Mt b m c gi l m u nu cc t m ca b m c
chiu di bng nhau.
Trang 72
Mt b m u c c s m l m, chiu di t m l l v s
lng t m n bng vi ml th c gi l m y, ngc li th
c gi l m vi.
Ngoi ra khi nim m y cn c dng theo ngha rng hn
nh sau: mt b m c gi l y theo mt tnh cht no
(chng hn tnh u hay tnh prefix nh sau ny cc bn s
thy) nu khng th thm mt t m no vo m vn gi c
tnh cht .
V d
Cho bng k hiu m A = {0, 1}. Th b m X1 = {0, 10, 11} l

m khng u, b m X2 = {00, 10, 11} l m u nhng vi
cn b m X3 = {00, 01, 10, 11} l m u v y.
Trang 73
Mt s phng php biu din m
Bng i chiu m
L cch lit k cc tin ca ngun v t m tng ng trong mt

bng.
Tin
a1 a2 a3 a4 a5 a6
T m 00 010 011 10 110 111
Mt to m
L cch biu din mi t m w = a0a1al-1 bng mt im (l,

b) trong mt phng to hai chiu, trong l l chiu di t
m cn b l trng s ca t m c tnh nh sau vi m l c s
l 1
m
b=
i
a
m
i
i=0
Trang 74
Mt s phng php biu din m (tt)
V d
Tin
a1 a2 a3 a4 a5 a6
T m 00 010 011 10 110 111
Tin
T m
Chiu di l
Trng s b
a1 a2 a3
00 010 011
2 3
3
0 2
6
a4 a5 a6
10 110 111
2 3
3
1 3
7
b
7
6
5
4
3
2
1
0
Trang 75
a6
a3
a5
a2
a4
a1
4 l
Cy m
L cch biu din cc t m bng cc nt l ca mt cy. Mi

nt l biu din cho t m trng vi nhn ca con ng i t
nt gc n nt l ny.
0
1
0
00
1
0
0
1 10
1
0
110 111
010 011
M c c s m th cy m tng ng s l cy m phn.
Phng php cy m ch cho php biu din nhng m prefix,
tc l khng c t m no trng vi phn i u ca mt t m
khc.
Trang 76
hnh kt cu m
L mt dng c bit ca cy m, trong cc nt l trng vi

nt gc v ngoi ra mi cnh ca hnh kt cu m u l
cnh c hng. V vy mt t m c biu din bng mt chu
trnh xut pht t nt gc v quay tr v li nt gc.
0
Hm cu trc m
1
0,1 0,1
L cch biu din s phn b cc t m theo di ca chng.

Phng php ny biu din bng mt hm G(li) cho bit c bao
nhiu t m c chiu di li.
Trang 77
V d
B m trong cc v d trn c biu din bng hm cu trc

m sau y
G(li) = 2, khi li = 2
4, khi li = 3
Trang 78
iu kin phn tch m
V d
Xt b m X1 = {0, 10, 11} m ho cho ngun A = {a, b, c}.

Gi s bn pht pht i bng tin x = abaac, lc chui t m
tng ng c pht i l y = 0100011.
Vn l bn nhn sau khi nhn c chui t m y lm sao c
th nhn bit c bng tin tng ng m bn pht pht.
lm c iu ny, bn nhn phi thc hin mt qu trnh
c gi l tch m. Chng hn vi chui k hiu m nhn c
nh trn th bn nhn ch c mt kh nng tch m hp l l
0 | 10 | 0 | 0 | 11 v xc nh c bng tin c gi i l
abaac.
Trang 79
iu kin phn tch m (tt)
Xt mt b m khc X2 = {0, 10, 01} m ho cho ngun A trn.

Gi s bn nhn nhn c chui k hiu l y = 01010 v thc
hin qu trnh tch m. y ta thy bn nhn c th thc hin
c ba kh nng tch m hp l sau 0 | 10 | 10, 01 | 0 | 10 v
01 | 01 | 0. V v vy bn nhn s khng bit c chnh xc
bn pht pht i bng tin no trong ba bng tin sau abb hay
cab hay cca.
Mt m nh vy th khng ph hp cho vic tch m v c
gi l m khng phn tch c (uniquely undecodable code).
V vy iu kin mt b m l phn tch c (uniquely
decodable code) l khng tn ti dy t m ny trng vi dy t
m khc ca cng b m.
Trang 80
Xt mt b m khc X3 = {010, 0101, 10100} m ho cho

ngun A trn. Gi s bn nhn nhn c chui k hiu l
01010100101 v thc hin qu trnh tch m. y ta thy ch
c mt cch tch m duy nht l 0101 | 010 | 0101 nhng vic
tch m tr nn kh khn hn so vi b m X1.
Chng hn lc chng ta gp chui 010 chng ta cha dm chc
l mt t m v n c th l phn i u ca t m 0101,
iu ny ph thuc vo k hiu i ngay sau chui 010.
Nu k hiu i ngay sau l 0 th chng ta khng nh c 010
l t m v 0 l phn i u ca mt t m khc sau . Cn
nu k hiu i ngay sau l 1 th chng ta khng khng nh
c, v c hai kh nng hoc 010 l mt t m v 1 l phn i
u ca mt t m khc sau , hoc 0101 l mt t m.
Trang 81
Nguyn nhn ca iu ny l do trong b m c mt t m ny

l tip u ng ca mt t m khc.
V cng chnh l nguyn nhn v bn cht ca vic mt dy
k hiu c th tch thnh hai dy t m khc nhau.
Tht vy, nu khng c t m no l tip u ng ca t m
khc (hay m l prefix) th vi mi dy t m ch c duy nht
mt cch tch thnh cc t m thnh phn. V vy nh sau ny
chng ta s thy cc m thng c s dng l cc m prefix.
Da vo tnh tip u ng trn, nhn bit mt b m (d
nhin khng phi l m prefix) c phn tch c hay khng
ngi ta thng dng mt cng c c gi l bng th m.
Trang 82
Bng th m
Bn cht ca bng th m l phn tch nhng t m di thnh

nhng t m ngn i u.
Chng hn t m di u1 c th c phn tch thnh
v11v12...v1kw11 trong v11, .., v1k l cc t m ngn cn w11 l
phn cn li ca u1.
Nu w11 cng l mt t m th b m ny l khng phn tch
c v chui v11v12...v1kw11 c t nht hai cch phn tch thnh
cc t m, l u1 v v11, v12, ..., v1k, w11.
Cn nu ngc li w11 khng l t m th chng ta dng n
xt tip. Trong ln xt tip theo chng ta xt xem mi w11 ny
c l tip u ng ca cc t m hay khng, nu ng vi mt
t m no , gi s l u2, th t m ny s c dng
w11v21...v2lw22 trong v21, ..., v2l l cc t m ngn (l c th
bng 0) cn w22 l tip v ng cn li.
Trang 83
Bng th m (tt)
Tng t nu w22 cng l mt t m th b m l khng phn

tch c v chui v11v12...v1kw11v21...v2lw22 c t nht hai cch
phn tch thnh cc t m, l v11v12...v1kw11 | v21 | ... | v2l |
w22, v v11 | v12 | ... | v1k | w11v21...v2lw22.
Nu ngc li w22 khng l t m th chng ta dng n xt
tip theo khun mu tng t nh trn. V vy chng ta kt
lun rng
Nu trong mt ln phn tch no , c mt t m di, chng
hn u, c phn tch thnh dy wiiv(i+1)1...v(i+1)n trong wii l
tip v ng ca mt t m no trong ln phn tch ngay trc
, cn v(i+1)1, ..., v(i+1)n l cc t m ngn th b m l khng
phn tch c.
Trang 84
Bng th m (tt)
Tht vy, lc s tn ti mt dy k hiu sau

v11v12...v1kw11v21...v2lw22 . . .w(i1)(i1)vi1...vimwiiv(i+1)1...v(i+1)n
ci m c th phn tch thnh hai dy t m khc nhau.
Cch 1 l
v11 | v12 | ... | v1k | w11v21...v2lw22 | . . . | w(i1)(i1)vi1...vimwii | v(i+1)1 |
... | v(i+1)n
Cch 2 l
v11v12...v1kw11 | v21 | ... | v2l | w22 ...w(i1)(i1) | vi1 | . . . | vim |
wiiv(i+1)1...v(i+1)n
Trang 85
Cch xy dng bng th m

(1) em cc t m xp thnh mt ct, theo th t chiu di ca t
m t nh n ln, nh du l ct 1.
(2) Trong ct ny, i chiu cc t m ngn vi cc t m di
hn, nu t m ngn l tip u ng ca t m di th ghi tip
v ng vo ct tip theo v nh du l ct 2.
(3) Tip tc, i chiu cc chui trong ct 1 v ct 2 vi nhau,
nu c chui no trong ct ny l tip u ng ca chui trong
ct kia th tip v ng s c ghi vo ct tip theo l ct 3.
(4) Tip tc theo khun mu ny nu ang xt ct th j th i
chiu cc chui trong ct ny vi ct 1. Nu c chui no
trong ct ny l tip u ng ca chui trong ct kia th tip v
ng s c ghi vo ct j + 1. Thc hin cho n khi khng
th in thm c na hoc ct mi thm vo trng vi mt
ct trc hoc c mt chui trong ct mi trng vi mt t
Trang 86
m.
Bng th m (tt)
V d
Lp bng th m cho b m nh ni trn A = {00, 01, 011,

1100, 00010}
1
2
3
010 0
00
01
1
100
011
1100
00010
4
0
1
5
0
1
11
11
0010 0010
100
00
M l khng phn tch c

trn chui 000101100 v c hai
cch phn tch khc nhau
00 | 01 | 011 | 00
00010 | 1100
10
Trang 87
Bng th m (tt)
iu kin cn v mt b m phn tch c l khng c

phn t no trong cc ct t j 2 trng vi mt phn t trong
ct 1.
chm gii m
chm gii m, thng k hiu l Tch, l s k hiu cn phi

nhn c c th phn tch (nhn dng) c t m.
Trong trng hp khng c chui no trong cc ct j 2 trng
vi t m nhng c hai ct k, l no (k l, k, l 2 ) trng
nhau th m l phn tch c nhng c chm gii m v
hn.
Trang 88
Bng th m (tt)
Xt b m {01, 10, 011, 100} c bng th m nh sau:

1
2 3
01 1 0
10 0 00
011
1
100
11
4
1
11
0
00
Bng th m ny c cc ct 3 v 4 trng nhau v cc chui nn

b m c chm gii m trong trng hp xu nht l v hn.
Chng hn vi chui c dng sau y th trong qu trnh nhn
cha ht chui chng ta khng th thc hin c vic tch m:
0110101010...
Trang 89
Bi tp
Hy lp bng th m cho nhng b m sau. Cho bit m c

phn tch c khng, nu c th chm gii m (trong
trng hp xu nht) l bao nhiu.
X1 = {00, 01, 100, 1010, 1011}
X2 = {00, 01, 101, 1010}
X3 = {00, 01, 110, 111, 1100}
X4 = {00, 01, 110, 111, 1110}
X5 = {00, 01, 110, 111, 0111}
X6 = {00, 01, 110, 111, 1011, 1101}
Trang 90
Bt ng thc Kraft
nh l 6.1
Cho l1, l2, ..., lK l cc chiu di ca mt b m prefix c bng

k hiu m kch thc m (tc gm m k hiu m). Th
K
li
m
1
i =1
Ngc li, nu cc s nguyn l1, l2, ..., lK tho bt ng thc

trn th tn ti mt b m prefix vi cc t m c chiu di l l1,
l2, ..., lK.
Chng minh
Chiu thun
Gi T l cy m tng ng vi b m trn
Trang 91
Bt ng thc Kraft
Mc 0
Mc 1
Gc
Mc 2
Mc 3
m-2 m-2
m-2
m-2 m-2 m-2
m-3
m-3 m-3 m-3 m-3
Nt l mc li s c gn trng s l m-li.
Trng s ca mi nt cha c tnh bng tng trng s ca cc
nt con.
Vi cch gn ny, chng ta suy ra trng s ca nt cha mc h
l m-h.
iu ny ng l v mi nt cha mc h c ti a m nt con mc
h + 1.
Trang 92
Bt ng thc Kraft (tt)

T y suy ra, trng s ca nt gc l 1.
M trng s ca nt gc chnh l tng trng s ca cc nt l.
Vy suy ra iu cn chng minh.
Chiu o
Chng ta chng minh bng cch xy dng mt cy m cho n.
iu ny l thc hin c theo nh chng minh ca chiu
thun.
V d
Tm b m prefix cho cc b m nh phn c cc chiu di t

m tng ng nh sau.
{2, 2, 3, 4, 4}, {2, 2, 3, 3, 3, 4, 4}, {2, 2, 3, 4, 4, 4, 5, 5}
Trang 93
nh l
nh l 6.2
Mt m phn tch c th c cc chiu di t m tho mn bt

K
ng thc Kraft.
li
m
1
i =1
Chng minh
Gi l1 l2 ... lK l cc chiu di t m vi c s l m.
Vi s nguyn N bt k ta c th vit
li
m
i =1
i1 =1
i N =1
L m
l i1 + L + l i N
Trang 94
nh l 6.2 (tt)
Ch l i + L + l i l chiu di ca mt dy N t m v c th
1
N
nhn gi tr bt k gia Nl1 v NlK. Gi Aj l s dy N t m m
c tng chiu di l j. Th
N
li
m =
i =1
K
Nl K
j
A
m
j
j = Nl1
V b m l phn tch c, nn cc dy N t m m c tng

chiu di l j phi khc nhau.
S cc dy c chiu di j ti a l mj. V vy Aj mj v
N
li
m
i =1
Nl K
m m
j
j = Nl1
= N (lK l1 ) + 1
Trang 95
Chng minh nh l (tt)
li
m
>1
Nu
i =1
i =1
li
N (l K l1 ) + 1
Th vi N ln
V vy chng ta c c iu cn chng minh.
s ln hn
li
m
1
i =1
Kt hp hai nh l trn chng ta rt ra mt nhn xt sau.

Nu mt m phn tch c th tn ti mt b m tng ng
v chiu di cc t m m c tnh prefix.
Trang 96
Bi 7 M ha ti u
ngun ri rc khng nh
7.1 Cc nh l v gii hn trn v di ca chiu di trung
bnh
7.2 M ho theo Shannon v Fano
7.3 Phng php m ho ti u theo Huffman
Trang 97
Cc nh l v gii hn trn v di ca
chiu di trung bnh
nh l 7.1
Cho ngun tin X = {a1, ..., aK} vi cc xc sut tng ng p1,

..., pK. Mt b m phn tch c bt k cho ngun ny vi c
s m m, chiu di trung bnh t m s tha (trong H(X) l
entropy ca ngun vi c s ca logarit l m).
H (X )
l
log m
Chng minh
m li
H ( X ) l ln m = pi ln pi pi li ln m = pi ln
pi
i =1
i =1
i =1
K
m li
K li
pi
1 = m 1 1 1 = 0
i =1
pi
i =1
K
Trang 98
Cc nh l v gii hn trn v di ca
chiu di trung bnh (tt)
nh l 7.2
m li
= 1 , tc l pi = m li
Ch du = xy ra khi v ch khi
pi
Cho ngun tin X = {a1, ..., aK} vi cc xc sut tng ng p1,
..., pK, c th xy dng mt m prefix vi c s m sao cho
H (X )
l<
+1
log m
Chng minh
Chn chiu di li ca t m cho tin ai theo qui tc li = log mi

Chng ta c
li = log mpi li log mpi m li pi

K
i =1
i =1
m li pi = 1
Trang 99
Chng minh nh l (tt)
V cc chiu di c chn ny tho bt ng thc Kraft nn

tn ti mt m prefix tng ng c cc chiu di ny.
Tip tc chng ta c
li = log mpi li < log mpi + 1

K
i =1
i =1
i =1
pi
p
l
<
p
log
i i i m + pi
K pi log pi
H (X )
+ 1 =
=
+1
log m
i =1 log m
iu ny hon tt chng minh ca chng ta.

Trang 100
H qu
C th m ho mt ngun m c chiu di trung bnh tip cn

H (X )
n
log m
vi sai s nh tu .
Chng ta thc hin iu ny bng cch m ho cc dy N tin
ca ngun X = {a1, ..., aK} theo nh l 7.2.
Lc ny chng ta c ngun mi vi kch thc l KN, mi phn
t l mt dy ca N tin c ly c lp t ngun X.
Entropy ca ngun mi ny l NH(X) v chiu di trung bnh
cc t m ca n theo nh ngha s l N ln chiu di trung
bnh cc t m ca ngun ban u, l .
p dng nh l 7.1 v nh l 7.2 i vi ngun mi chng ta
c
Trang 101
H qu (tt)
p dng nh l 7.1 v nh l 7.2 i vi ngun mi ta c
NH (X )
NH (X )
H (X )
H (X ) 1
Nl <
+1
l <
+
log m
log m
log m
log m N
V N c th ln tu , nn l tip cn n H(X) / log m vi tc
tng ng vi 1/N tin n 0 khi N tin ra v cng.

nh gi mt phng php m ho no l tt hay khng
ngi ta a ra khi nim hiu sut lp m.
Hiu sut lp m
Hiu sut lp m h c nh ngha bng t s ca entropy ca

ngun vi chiu di trung bnh ca b m c lp
H (X )
h=
l
Trang 102
M ha ti u
L php m ha m kt qu l mt b m c chiu di trung

bnh l nh nht trong tt c cc php m ha c th c cho
ngun.
B m ca php m ha ti u cho ngun c gi l b m ti
u.
Ba php m ha: Shannon, Fano, Huffman.
Trong mi php m ha chng ta s m ha vi c s m m = 2
trc (m ha nh phn), sau s m rng cho trng hp m
> 2.
Trang 103
Phng php m ho Shannon

B1. Sp xp cc xc sut theo th t gim dn. Khng mt tng
qut gi s p1 ... pK.
i 1
B2. nh ngha q1 = 0, qi =
p
j =1
, i = 1, 2, ..., K.
B3. i qi sang c s 2, (biu din qi trong c s 2) s c mt

chui nh phn
B4. T m c gn cho ai l li k hiu ly t v tr sau du phy
ca chui nh phn tng ng vi qi, trong li = log pi
2
Trang 104
V d
Hy m ho ngun S = {a1, a2, a3, a4, a5, a6} vi cc xc sut

ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05.
i 1
Tin Xc sut q = q Biu din
i
j
ai
pi
nh phn
j =1
li = log2 pi
T m
wi
a1
a2
a3
a4
a5
a6
2
2
3
4
4
5
00
01
100
1100
1101
11110
0,3
0,25
0,2
0,12
0,08
0,05
0
0,3
0,55
0,75
0,87
0,95
0,00
0,01001...
0,10001...
0,11000...
0,11011...
0,111100...
H = 2.36, l = 2,75, h = 2,36/2,75 = 85,82%

Trang 105
Nhn xt - Bi tp
Phng php Shannon cho kt qu l mt m prefix.

Phng php Shannon c th m rng cho trng hp m > 2
Bi tp
Hy m ho cc ngun sau bng phng php Shannon. Tnh

entropy ca ngun, chiu di trung bnh v hiu sut ca php
m ha.
S1 = {a1, a2, a3, a4, a5, a6} vi cc xc sut ln lt l 0,25;
0,21; 0,19; 0,16; 0,14; 0,05.
S2 = {a1, a2, a3, a4, a5, a6 , a7, a8} vi cc xc sut ln lt l
0,21; 0,18; 0,15; 0,14; 0,12; 0,01; 0,06 ; 0,04.
S3 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt
l 0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.
Trang 106
Phng php m ho Fano

B1. Sp xp cc xc sut theo th t gim dn. Khng mt tng
qut gi s p1 ... pK.
B2. Phn cc xc sut thnh 2 nhm c tng xc sut gn bng
nhau nht.
B3. Gn cho hai nhm ln lt cc k hiu 0 v 1 (hoc ngc
li).
B4. Lp li bc 2 cho cc nhm con cho n khi khng th tip
tc c na.
B5. T m ng vi mi tin l chui bao gm cc k hiu theo th
t ln lt c gn cho cc nhm c cha xc sut tng
ng ca tin.
Trang 107
V d

ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05.
Phn nhm ln
Tin Xc sut
1 2 3 4
a1 0,3
0 0
a2 0,25
0 1
a3 0,2
1 0
a4 0,12
1 1 0
a5 0,08
1 1 1 0
a6 0,05
1 1 1 1
T m
00
01
10
110
1110
1111
H = 2.36, l = 2,38, h = 2,36/2,38 = 99,17%

Trang 108
Ch
Ch , trong nhiu trng hp c nhiu hn mt cch chia

thnh cc nhm c tng xc sut gn bng nhau, ng vi mi
cch chia c th s cho ra cc b m c chiu di trung bnh
khc nhau.
V d
Hy m ho ngun S = {a1, a2, a3, a4, a5, a6, a7, a8} vi cc xc

sut ln lt l 0,23; 0,2; 0,14; 0,12; 0,1; 0,09; 0,06; 0,06.
Trang 109
V d
ai
a1
a2
a3
a4
a5
a6
a7
a8
pi
0,23
0,2
0,14
0,12
0,1
0,09
0,06
0,06
1
0
0
1
1
1
1
1
1
l 1 = 2,88,
2
0
1
0
0
1
1
1
1
3 4
0
1
0
0
1
1
0
1
0
1
wi
00
01
100
101
1100
1101
1110
1111
ai
a1
a2
a3
a4
a5
a6
a7
a8
pi
0,23
0,2
0,14
0,12
0,1
0,09
0,06
0,06
1
0
0
0
1
1
1
1
1
l 2 = 2,89
Trang 110
2
0
1
1
0
0
1
1
1
3 4
wi
00
0
010
1
011
0
100
1
101
0
110
1 0 1110
1 1 1111
Nhn xt - Bi tp
Nhn xt
Phng php Fano thng cho kt qu tt hn phng php

Shannon.
Bi tp
Hy m ho cc ngun sau bng phng php Fano. Tnh hiu

sut ca php m ha.
0,21; 0,19; 0,16; 0,14; 0,05.
0,21; 0,18; 0,15; 0,14; 0,12; 0,1; 0,06 ; 0,04.
S3 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt
l 0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.
Trang 111
Phng php m ho ti u Huffman
Trc ht xt c s m m = 2. Trng hp m > 2, chng ta s

c mt s ch v s khc bit so vi trng hp m = 2.
Cho ngun S = {a1, ..., aK} c cc xc sut ln lt l p1, ..., pK.
Gi l1, ..., lK l chiu di cc t m tng ng vi b m ti u
cho S. Nu pi > pj th li lj.
Chng minh
Vi pi > pj, gi s li > lj. Xt b m mi bng cch hon i hai

t m c chiu di li v lj cho nhau. Xt hiu chiu di trung
bnh ca b m mi so vi b m c
l = pilj + pjli pili pjlj = (pj pi)(li lj) < 0
iu ny mu thun vi nh ngha ca b m ti u.
Trang 112
Hai nh l ca Huffman
nh l 7.3 (nh l s 1 ca Huffman)
B ny tht s pht biu mt iu rng, m ho ti u cho

mt ngun tin th tin c xc su cng ln phi c m ho
thnh t m c chiu di cng nh.
Trong b m ti u (m = 2) cho mt ngun tin, th hai t m
tng ng vi hai tin c xc sut nh nht phi c chiu di
bng nhau (lK1 = lK) v c th lm cho chng ch khc nhau
duy nht bit cui (bit tn cng bn phi).
Chng minh
Nu lK1 < lK th loi b bit cui cng ca t m wK chng ta

c mt b m mi vn c tnh prefix nhng c chiu di
trung bnh nh hn b m c.
Trang 113
Hai nh l ca Huffman (tt)
Gi s wK1 v wK khng tha iu kin l khc nhau ch bit

cui.
Nu c mt t m wi khc c chiu di bng lK ng thi khc
t m wK ch bit cui th chng ta c th hon i wK1 v wi
cho nhau, v vy nh l cng c chng minh.
Nu khng tn ti mt t m wi nh vy th chng ta c th to
ra mt b m mi bng cch b i bit cui ca t m wK. B m
mi ny khng vi phm iu kin prefix v c chiu di trung
bnh nh hn b m c. V vy nh l c chng minh.
Trang 114
nh l 7.4 (nh l s 2 ca Huffman)
Xt mt ngun mi S = {a1, ..., aK1} vi s phn b xc sut

l p1, ... , pK1 trong pi = pi vi 1 i K 2 cn pK1 =
pK1 + pK. Nu {w1, ..., wK1} lm mt m ti u cho S th m
nhn c theo qui tc sau l m ti u cho S.
1iK2
wi = wi,
wK1 = wK10
wK = wK11
Chng minh
V lK = lK1 = 1 + lK1, nn
l = p1l1 + ... + pKlK = p1l1 + ... + (pK1 + pK)(1 + lK1)
= l ' + (pK1 + pK)
Trang 115
S khc bit gia l v l ' l mt hng s.

Nn nu m ti u cho ngun S l tt hn m theo qui tc
pht biu th m c dn xut t m ti u ny bng cch b
i hai t m wK v wK1 v thay vo t m m b i bit cui ca
wK th s c mt m ti u tt hn cho ngun S, iu ny
mu thun.
Vy m nhn c cho S theo qui tc trn l ti u.
nh l nh l 7.3 v 7.4 cho php qui bi ton tm m ti u
cho ngun c K tin v bi ton tm m ti u cho ngun c K1
tin. V qu trnh ny c th c lp li cho n khi ch cn hai
tin. Lc th m ti u l d thy.
Trang 116
Gii thut m ha Huffman

B1. Sp xp cc xc sut theo th t gim dn chng hn p1 ...
pK
B2. Gn 0 ti bit cui ca wK1 v 1 n bit cui ca wK hoc
ngc li. Tuy nhin chng ta s qui c thc hin theo chiu
th nht.
B3. Kt hp pK v pK1 to thnh mt tp xc sut mi p1, ... ,
pK2, pK1 + pK
B4. Lp li cc bc trn cho tp mi ny.
V d

ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05.
Trang 117
V d
ai
a1
a2
a3
a4
a5
a6
pi
Ln 1
0,3
0,25
Ln 2
Ln 3
Ln 4
wi
0,55 0 00
0,3 0 0,45 1 01
0,2
0,2
0,25 0 0,25 1
11
0,12
0,13 0 0,2 1
101
0,08 0 0,12 1
1000
0,05 1
1001
0,3
0,25
0,3
0,25
0,45
H = 2.36, l = 2,38, h = 2,36/2,38 = 99,17%
Trang 118
Nhn xt
Nhn xt
So snh vi phng php Fano ta thy trong trng hp trn th

c hai phng php cho hiu sut bng nhau.
Tuy nhin trong trng hp tng qut phng php Fano
khng phi l phng php m ha ti u.
Ch
Trong trng hp nu xc sut pK1 + pK bng vi mt xc sut

pi no th chng ta c th t pK1 + pK nm di hoc nm
trn xc sut pi th kt qu chiu di t m trung bnh vn
khng thay i cho d cc t m kt qu c th khc nhau.
Trang 119
M rng cho c s m > 2

Nu K m th vic m ho ti u l qu tm thng
Gi s K > m, tn ti n sao cho: m + (n 1)(m 1) < K m +

n(m 1). Chng ta s b sung vo mt s tin ph c xc sut
bng 0 sao cho tng s tin ca ngun bng vi m + n(m 1).
Sau th tc m ho trn c iu chnh nh sau
B1. Sp xp cc xc sut theo th t gim dn chng hn p1 ...
pK
B2. Gn ln lt cc k hiu 0, 1, ..., m 1 ti cc bit cui ca m
t m c xc sut nh nht
B3. Kt hp m xc sut nh nht li thnh mt v to vi K m
xc sut cn li thnh mt tp mi.
B4. Lp li cc bc trn cho tp mi ny.
Trang 120
V d

ln lt l 0,3; 0,25; 0,2; 0,12; 0,08; 0,05 vi m = 3.
ai
pi
Ln 1 Ln 2
wi
a1 0,3
0,3
0,45 0 1
0,25
0,3 1 2
a2 0,25
0,2 0 0,25 2 00
a 0,2
3
a4
a5
a6
a7
0,13 1
0,08 0 0,12 2
0,05 1
0,0 2
0,12
02
010
011
H = 1.49, l = 1,58, h = 1,49/1,58 = 94,24%

Trang 121
Bi tp
Hy m ho cc ngun sau bng phng php Huffman theo

cc c s m = 2 v m = 3. Tnh hiu sut ca php m ha trong
mi trng hp.
0,21; 0,19; 0,16; 0,14; 0,05.
0,23; 0,2; 0,14; 0,12; 0,1; 0,09; 0,06 ; 0,06.
0,21; 0,18; 0,15; 0,14; 0,12; 0,01; 0,06 ; 0,04.
S4 = {a1, a2, a3, a4, a5, a6 , a7, a8 , a9} vi cc xc sut ln lt l
0,25; 0,19; 0,15; 0,11; 0,09; 0,07; 0,06; 0,04; 0,04.
Trang 122
Nhn xt
Xt ngun S = {a1, a2, a3, a4} c s phn b xc sut l {0,4;

0,25; 0,2; 0,15}. Xt ngun mi S2 = {aiaj, 1 i, j 4} c tp
phn b xc sut l {0,16; 0,1; 0,08; 0,06; 0,1; 0,0625; 0,05;
0,0375; 0,08; 0,05; 0,04; 0,03; 0,06; 0,0375; 0,03; 0,0225}.
H(S) = 1,9 v H(S2) = 2H(S) = 3,8.
Hai bng sau y trnh by kt qu vic m ho ti u cho S v
S2 theo Huffman.
Nhn xt
Vic m ho cho mt dy tin (hay khi tin) th cho hiu sut

cao hn so vi vic m ho cho tng tin.
Trang 123
Nhn xt (tt)
Tin
a1
a2
a3
a4
pi
0,4
0,25
0,2
0,15
T m
1
01
000
001
l S = 1,95
h1 = 97,63%
l S 2 = 3,8375
h2 = 99,26%
Tin
a1a1
a1a2
a2a1
a1a3
a3a1
a2a2
a1a4
a4a1
pij
0,16
0,1
0,1
0,08
0,08
0,0625
0,06
0,06
T m
000
101
110
0010
0011
0110
0111
1000
Tin
a2a3
a3a2
a3a3
a2a4
a4a2
a3a4
a4a3
a4a4
Trang 124
pij
0,05
0,05
0,04
0,0375
0,0375
0,03
0,03
0,0225
T m
1110
1111
01000
01001
01010
01011
10010
10011
Bi 8 M ha ngun ph qut
8.1 Ngun ri rc khng nh vi thng k khng bit trc
8.2 Cc vect tn xut v taentropy (quasientropy)
8.3 Mt s m ho ph qut cho ngun ri rc khng
nh
Trang 125
Gii thiu
Vn ny khng c khi xng bi Shannon m bi B. M.

Fitingof.
L thuyt ca Shannon da trn kin thc v cc hm phn b
xc sut v chng minh tn ti php m ho ti u.
M ho ngun ph qut tip cn theo cch khc bng vic li
dng cu trc ca cc dy v cng i n c cng kt qu ti
u.
Trong trng hp m cc hm phn b xc sut l khng c sn
hoc s thng k v ngun l thay i theo thi gian, nhng
iu m thng xy ra trong thc t, th k thut m ho ngun
ph qut l mt la chn thch hp hn l dng k thut ca
Shannon.
Trang 126
Ngun ri rc khng nh vi
thng k khng bit trc
Xt ngun A = {a1, ..., aK} c s phn b xc sut l {p1, ..., pK}

sinh ra mt dy cc k hiu c lp c phn b ng nht.
Chng ta gi thit rng s phn b xc sut {p1, ..., pK} l c
nh nhng khng c bit trc bi b m ho (encoder).
Thc t s phn b xc sut thng l khng c bit trc
hoc ch c bit mc gn ng, hoc s phn b ny
thay i chm theo thi gian.
V vy mt s m ho da trn xc sut c th hiu qu
khung thi gian ny nhng s khng hiu qu khung thi gian
khc.
Trang 127
thng k khng bit trc (tt)
nh gi nh hng ca s bit khng chnh xc v thng k

ca ngun n hiu qu ca vic m ho
Xt ngun ri rc khng nh nh phn vi s phn b xc sut
l P(0) = p, P(1) = 1 p.
Nu b m ho c cung cp xc sut gn ng vi p l p0 th
theo phng php ca Shannon k hiu 0 s c gn vi t m
c chiu di l log p0 cn 1 c gn vi t m c chiu di
log (1 p0).
Chiu di trung bnh ca cc t m l
l = p log p0 (1p) log(1p0)
Chiu di trung bnh t m ti u l
l opt = p log p (1p) log(1p)
Trang 128
l
l opt
Ch rng l l mt tip tuyn ca l opt ti p = p0, nhng khi p

lch ra xa p0 th khong cch gia hai th gia tng kh nhanh.
Trang 129
Trong bi ny chng ta pht trin cc tng c bn v m ho

ph qut, mt s m ho khng da trn xc xut ca cc
dy m li da vo cu trc ca chng.
Chng ta s chng minh rng nguyn dng nh tu c
kh nng m ho mt ngun sao cho l H(x) + i s
phn b xc sut {p1, ..., pK} ca ngun.
c th c lm nh
Cn trn ca l
tu bng cch chn
chiu di khi tin cn
m ho ln.
l opt
0
Trang 130
Cc vect tn sut v ta-entropy
Xt cc dy ngun Si c chiu di N.
C KN dy v ta gi tp KN dy ny l khng gian mu S.
Chng ta k hiu Nki l s k hiu ak c trong dy Si v qki l tn
sut ca ak trong Si
qki = Nki / N
Vect (q1i, ..., qKi) (k hiu l Q(Si) hay gn hn l Qi) c gi
l vect tn sut ng vi chui Si.
Gi cc qk (k = 1, ..., K) l cc bin ngu nhin trn S bng cch
gn mi Si vi qki. Chng ta c b sau.
Gi tr trung bnh ca qk chnh l xc sut pk ca ak.
KN
E (q k ) = P(S i )qki = pk
i =1 Trang 131
Cc vect tn sut v ta-entropy (tt)
Chng minh
nh ngha bin ngu nhin xk(n) bng 1/N nu ngun sinh ra k

hiu ak ti v tr th n ca dy v bng 0 nu ngc li.
V ngun l khng nh, dy xk(1), ..., xk(N) l c lp v c phn
b ng nht.
N
(n)
Gi tr trung bnh ca xk(n) bng pk/N n. M q k = x k
N
n =1
V vy
(n)
E (q k ) = E (x k ) = pk
n =1
Mi dy Si c tng ng mt vect tn sut Qi, nhng ngc

li vi mt vect Q = (q1, ..., qK) c th tng ng vi nhiu
dy Si.
Trang 132
Gi (Q) l s cc dy Si m c cng vect tn sut Q (tc l

nhng dy m c s ln xut hin ca mi ak trong dy bng
nhau v bng Nk = Nqk k = 1, ..., K).
(Q) =
N!
K
k =1
Nk !
Gi (K, N) l s vect biu din cho cc dy ngun c chiu

di N.
Con s ny c th din t thnh mt bi ton tp hp tng
ng kh quen thuc l: C bao nhiu b gm K s nguyn
khng m m c tng bng N.
N + K 1
( K , N ) =
N
Trang 133
Chng minh
Xt mt hng gm N + K 1 khong trng. Dng N i tng

ging nhau lp vo N khong trng bt k. K 1 khong trng
cn li s chia N i tng ny thnh K nhm. Do ng vi
mi cch lp N i tng vo N + K 1 v tr chng ta c mt
tng tng ng. V vy s lng tng ny bng
N + K 1
Vi mi dy Si chng ta c tng ng mt vect Qi = (q1i, ...,

qKi). Chng ta nh ngha mt bin ngu nhin (Q) gn mi
dy Si vi gi tr (k hiu l (Si) hoc (Qi))
qki log qki
k =1
Trang 134
Ch Qi l mt vect xc sut v (Qi) c cng thc ging

nh ca entropy nn chng ta gi (Qi) l taentropy.
D nhin (Qi) c tt c cc tnh cht ca hm entropy H(Qi)
ci m ch ph thuc duy nht vo Qi.
Chng ta c nh l sau thit lp mi quan h gia (Q) (hay
vit r ra l (q1, ..., qK)) vi entropy ca ngun H(p1, ..., pK).
nh l 8.1
E((Q)) H(p1, ..., pK)
Chng minh
K
K
E ((Q)) = E q k log q k = E ( q k log q k )
k =1
k =1
Trang 135
M hm x log x l hm li, v vy theo bt ng thc

Jensen chng ta c
E(qk logqk) E(qk) log E(qk)
Theo mt b trc y chng ta c E(qk) = pk. V vy
K
E ((Q )) = pk log pk = H ( p1 ,..., p K )

k =1
Trang 136
Mt s m ho ph qut cho
ngun ri rc khng nh
Mt t m cho mt dy Si gm hai phn: phn u l chui m

ho cho vect tun sut Qi tng ng ca dy Si, phn th hai
l chui m ho cho dy Si trong s cc dy c cng vect Qi.
V tng cc vect tn sut khc nhau l (K, N), nn s bit
dng biu din cho phn u l
log ( K , N )
Tng t s bit biu din cho phn th hai l
log (Qi )
V vy t m biu din cho dy Si c chiu di l

l(Si) = log ( K , N ) + log (Qi )
< log (K, N) + log (Qi) + 2.
Trang 137
Mt s m ho ph qut cho
ngun ri rc khng nh (tt)
Chng ta chng minh c gi tr trung bnh ca l(Si) tho
N
K 1
) + ( K 1) log(1 +
)
E (l ( S i )) < NH ( p1 ,..., p K ) + N log(1 +
N
K 1
Suy ra chiu di trung bnh trn mt k t ngun tho
E (l ( S i ))
K 1 K 1
N
< H ( p1 ,..., p K ) + log(1 +

)+
log(1 +
)
l=
N
N
N
K 1
Ch thnh phn nm trong du mc vung tin n 0 khi N

vi tc bng vi tc ca log N
0
N
iu ny ni ln rng phng php ny tip cn n entropy
ca ngun chm hn so vi cc phng php m bit trc
xc sut. iu ny cng d hiu v cng l ci gi phi tr nu
chng ta khng bit trc xc sut.
Trang 138
V d
Bng sau y m t vic m ho ph qut cho mt ngun nh

phn cho tng khi c chiu di 7.
C (2, 7) = 8 vect tn sut v v vy cn dng 3 bit m
ho 8 vect ny; 3 bit ny s l 3 bit u ca mi t m. Cc
bit cn li dng nhn bit mi dy trong lp cho (l lp
cc dy c cng vect tn sut).
Qi
(0/7,7/7)
(Qi)
1
(1/7,6/7)
Si
1111111
0111111
1011111
...
1111110
(Si)
0
0,592
Trang 139
wi
000
001 000
001 001
001 110
V d (tt)
Qi
(Qi)
(2/7,5/7)
21
(3/7,4/7)
(4/7,3/7)
35
35
Si
0011111
0101111
...
1111100
0001111
0010111
...
1111000
0000111
0001011
...
1110000
(Si)
0,863
0,985
wi
010 00000
010 00001
...
010 10100
011 000000
011 000001
011 100010
0,985
Trang 140
100 000000
100 000001
100 100010
V d (tt)
Qi
(Qi)
(5/7,2/7)
21
(6/7,1/7)
(7/7,0/7)
7
1
Si
0000011
0000101
...
1100000
(Si)
0,863
0000001
0000010
...
1000000
0,592
0000000
wi
101 00000
101 00001
...
101 10100
110 000
110 001
110 110
Trang 141
111
Bi 9 Knh ri rc khng nh
Lng tin tng h
9.1 Knh ri rc khng nh v ma trn knh
9.2 Entropy iu kin v lng tin tng h
9.3 Mt s loi knh
9.4 S nhp nhng (equivocation) v tc truyn tin
9.5 Dung lng knh
Trang 142
Knh ri rc khng nh v ma trn knh
nh ngha
Mt knh ri rc khng nh (DMC) c nh ngha bng mt
bng k hiu u vo (ngun pht) X = {x1, ..., xK}, mt bng k
hiu u ra (ngun nhn) Y = {y1, ..., yJ}, v mt s phn b xc
sut c iu kin p(yj | xk), vi 1 k K, 1 j J.
X
xk
p(yj | xk)
yj
Bng k hiu u ra khng nht thit ging bng k hiu u

vo. iu ny c ngha l bn nhn c th nhn nhng k hiu
m khng ging vi nhng k hiu m bn pht pht i.
Trang 143
Nhn xt
Thut ng khng nh (memoryless) suy ra rng
p{ y j1 K y jN | x k1 L x kN } =
p( y jn | xkn )
vi N bt k.
n =1
Mt knh ri rc khng nh thng c biu din di dng
mt ma trn knh [p(yj | xk)] c kch thc K J.
y1
y2
x1 p(y1 | x1) p(y2 | x1)
x2 p(y1 | x2) p(y2 | x2)
...
...
...
xK p(y1 | xK) p(y2 | xK)
...
...
...
...
yJ
p(yJ | x1)
p(yJ | x2)
...
p(yJ | xK)
Trang 144
Nhn xt (tt)
Chng ta thy, ma trn knh chnh l ci m biu din tnh cht

tp nhiu ca knh truyn.
Ch , nu chng ta bit s phn b xc sut trn X th s phn
b xc sut ca Y s c xc nh nh sau
p( y j ) =
p( xk ) p( y j | xk )
k =1
Trang 145
Entropy iu kin v lng tin tng h
Xt bi ton truyn tin sau

Cho bit cu trc thng k ca ngun X v ma trn knh. Hy
xc nh k hiu xk no c pht pht i khi nhn c
u nhn mt k hiu yj no ?
V d
Cho ngun X = {x1, x2} vi cc xc sut ln lt l p(x1) = 1/4,

p(x2) = 3/4, ngun Y = {y1, y2} v ma trn knh l
y1 y2
x1 4/5 1/5
x2 2/5 3/5
Nu nhn c y1 th xk no c kh nng c pht i?

Trang 146
V d
p( xk | y j ) =
p( xk , y j )
p( y j )
p( xk ) p( y j | xk )
K
p ( xi , y j )
i =1
p ( x1 ) p ( y1 | x1 )
p ( x1 | y1 ) =
p ( x1 ) p( y1 | x1 ) + p ( x 2 ) p ( y1 | x 2 )
p( xk ) p( y j | xk )
K
p ( xi ) p ( y j | xi )
i =1
3
p ( x 2 | y1 ) =
5
(1 / 4) (4 / 5)
2
=
=
(1 / 4) (4 / 5) + (3 / 4) (2 / 5) 5
p(x1 | y1) < p(x2 | y1), nh vy chng ta c th khng nh c

k hiu x2 c kh nng c pht i hn x1?
Trang 147
V d (tt)
, trong cng thc ca p(xi | yj) c cha tha s p(xi), nn

p(xi | yj) b nh hng bi xc sut l p(xi).
V vy cng bng trong vic so snh chng ta phi da trn
t s p(xi | yj)/p(xi) ci m khng b nh hng nhiu bi p(xi).
p ( x1 | y1 ) 2 / 5 8
=
=
p ( x1 )
1/ 4 5
p ( x 2 | y1 ) 3 / 5 4
=
=
p( x2 )
3/ 4 5
Nh vy thc s k hiu x1 mi c kh nng c pht i hn k

hiu x2.
T xc sut iu kin chng ta gii thiu khi nim lng tin c
iu kin.
Trang 148
Lng tin c iu kin I(xk | yj)
nh ngha
I(yj | xk) = log p(yj | xk)

I(xk | yj) = log p(xk | yj)
p(yj | xk) 1 th I(yj | xk) 0 v ngc li.
Nu khi pht i xk v bit chc yj s nhn c th pha nhn
chng ta khng cn tn thm thng tin g gii thch.
Nu p(yj | xk) = 1/2 (I(yj | xk) = 1 bit) th khi pht i xk bn nhn
s c hai kh nng v yj ch l mt trong hai kh nng , c
ngha l bn nhn cn thm thng tin (cn thm 1 bit) bit
chnh xc l kh nng no.
Xc sut p(yj | xk) = 1/2 ch xy ra khi knh truyn c nhiu.
Trang 149
Lng tin c iu kin I(xk | yj)
V vy lng tin c iu kin cn c gi l lng tin b mt

i do nhiu.
Khi pht i xk bn nhn s c mt tp cc yj c kh nng c
nhn.
Ngc li khi nhn c yj bn pht s c mt tp cc xk c kh
nng c pht.
o mc quan h gia xk vi yj chng ta gii thiu khi
nim lng tin tng h.
Trang 150
Lng tin tng h
nh ngha
Lng tin tng h gia hai tin l lng tin ca ca tin ny

c cha trong tin kia v ngc li. Bng cng thc
Lng tin tng h = Lng tin ring Lng tin b mt i
I(xk, yj) = I(xk) I(xk | yj) = I(yj) I(xk | yj)
p( x k |y j )
p( y j|x k )
= log
= log
p( x k )
p( y j )
Nu p(xk | yj) = 1 c ngha l nu yj nhn c th chc chn
xk c pht i, iu ny c ngha l lng tin ca xk
c truyn nguyn vn thng qua knh, do I(xk, yj) = I(xk).
Trang 151
Lng tin c iu kin trung bnh

I(X | y j ) =
I (Y | xk ) =
I(X | Y) =
k =1
k =1
p(xk | y j )I (xk | y j ) = p(xk | y j ) log p(xk | y j )

J
j =1
j =1
p( y j | xk )I ( y j | xk ) = p( y j | xk ) log p( y j | xk )
J
j =1
k =1
p( y j )I ( X | y j ) = p( y j ) p(xk | y j ) log p(xk | y j )

j =1
K
= p( xk , y j ) log p( xk | y j )
k = 1 j =1
K J
I (Y | X ) = p( xk , y j ) log p( y j | xk )
k =1 j =1
Trang 152
Entropy iu kin
nh ngha
Xt hai bin ngu nhin x v y vi phn b xc sut ng thi

p(xk, yj), k = 1, ..., K , j = 1, ..., J. Entropy iu kin ca x
cho y c nh ngha l
K
H (x | y ) = p( xk , y j ) log p( xk | y j )
k =1 j =1
H(x | y) c th c din dch nh l bt ng trung bnh v x

sau khi bit y.
nh l 9.1
H(x | y) H(x), du = xy ra x v y l c lp.

Trang 153
Chng minh
K J
k =1 j =1
K J
k =1
H (x | y) H (x) = p(xk , y j ) ln p(xk | y j ) + p(xk ) ln p(xk )
p(xk )
= p(xk , y j ) ln
p(xk | y j )
k =1 j =1
Ly tng trn nhng cp (k, j) m p(xk, yj) 0. V vy

p( xk )
1
H (x | y) H (x) = p( xk , y j )
p( xk | y j )
k j
p( xk ) p( y j ) p( xk , y j )
k
= p( xk ) p( y j ) 1 0
Trang 154
Chng minh (tt)
Du = xy ra p(xk) = p(xk | yj) i vi tt c cc cp (k, j)

m p(xk, yj) 0 ng thi tng p(xk)p(yj) trn tt c nhng cp
ny bng 1.
iu kin th hai tng ng vi iu kin p(xk)p(yj) = 0 bt
k khi no m p(xk, yj) = 0.
C hai iu kin ny cui cng tng ng vi iu kin l x
v y c lp.
nh l 9.2
H(x, y) = H(x) + H(y | x) = H(y) + H(x | y)
Trang 155
Chng minh
H (x, y) = p( xk , y j ) log p( xk , y j )
k
= p( xk , y j ) log p( xk ) + log p( y j | xk )
= p( xk )[log p( xk )] p( y j , y k ) log p( y j | xk )
k
= H ( x) + H ( y | x)
Phn th hai chng minh hon ton tng t.

Kt hp hai nh l trn chng ta suy ra rng
H(x, y) H(x) + H(y)
du = xy ra x, y l c lp.
Trang 156
Lng tin tng h trung bnh

I ( X , Y ) = p( xk , y j ) I ( xk , y j )
k
p( xk | y j )
= p ( x k , y j ) log
p( xk )
k j
p( y j | xk )
= p ( xk , y j ) log
p( y j )
k j
p( xk , y j )
= p ( xk , y j ) log
p( xk ) p( y j )
k j
Nu biu din theo entropy th chng ta c

I(x, y) = H(x) H(x | y) = H(y) H(y | x)
Trang 157
Mt s loi knh ri rc khng nh
Knh i xng (Symmetric channel)
L knh m mi dng ca ma trn knh cha cng tp cc s

p1, ..., pJ v mi ct cha cng tp cc s q1, ..., qK.
V d
[p(yj | xk)] =
Cc ma trn biu
din
cc knh i xng
[p(yj | xk)] =
j=1
0,2
0,2
0,3
0,3 k = 1
0,3
0,3
0,2
0,2 k = 2
0,2
0,3
0,5
0,3
0,5
0,2
0,5
0,2
0,3
Knh i xng nh
1
[p(yj | xk)] =
01
phn (binary
1
symmetric channel
Trang 158
BSC).
Nhn xt
Knh i xng th H(y | x) c lp vi s phn b xc sut ca

ngun pht v c xc nh duy nht bng ma trn knh.
Chng minh
H (y | x) = p ( x k , y j ) log p ( y j | x k )
k =1 j =1
K
= p ( x k ) p ( y j | x k ) log p ( y j | x k )
k =1
K
j =1
J
k =1
J
j =1
= p ( x k ) p 'j log p 'j

= p 'j log p 'j
j =1
Trang 159
Knh khng mt (Lossless channel)
Cnh ni gia xk v yj ngha l p(yj | xk) 0. Trong knh khng

mt u ra xc nh duy nht u vo, v vy H(x | y) = 0.
x1
xK
x1
y1 y2
ym
ym+1
yJ
Knh n nh (Deterministic channel)

x1 x2
xm
xm+1
xK
y1
y2
yJ
Trong knh ny u vo xc nh duy nht u ra, v vy
Trang 160
H(y | x) = 0
Knh v dng (Useless channel)
Mt knh l v dng nu v ch nu x v y l c lp vi mi
s phn b xc sut ca u vo (ngun pht).
i vi mt knh v dng th H(x | y) = H(x), tc l kin thc
v u ra khng lm gim bt ng v u vo. V vy, i
vi mc ch xc nh n nh u vo, chng ta c th pht
l u ra hon ton. By gi chng ta s chng minh rng.
Mt knh ri rc khng nh l v dng nu v ch nu ma trn
knh ca n c cc dng ging nhau.
Chng minh
iu kin
Gi s ma trn c cc dng ging nhau p1, ..., pJ. Th i vi
mi u ra yj
Trang 161
Knh v dng (tt)

p( y j ) =
p( xk , y j ) =
k =1
'
p
(
x
)
p
(
y
|
x
)
=
p
k
j
k
j
k =1
'
p
(
x
)
=
p
k
j
k =1
i vi mi cp u vo ra (xk, yj), chng ta c

p(xk, yj) = p(xk) p(yj | xk) = p(xk) pj = p(xk) p(yj)
V vy u vo v u ra c lp nhau bt chp s phn b xc
sut ca u vo.
iu kin cn
Gi s cc dng ca ma trn khng ging nhau mt ct
chng hn j0 m c cc phn t khng ging nhau.
Gi s p(yj0 | xk0) l phn t ln nht trong ct ny. Xt s phn
b ng nht (ng xc sut) u vo (u pht), chng ta c
Trang 162
Knh v dng (tt)

K
1 K
p( y j 0 ) = p( xk ) p( y j 0 | xk ) =
p( y j 0 | xk ) < p( y j 0 | xk 0 )
K k =1
k =1
Tc l p(yj0) p(yj0 | xk0). V vy p(xk0, yj0) = p(xk0) p(yj0 | xk0)

p(xk0) p(yj0). Mu thun vi gi thit l x, y c lp vi mi s
phn b xc sut ca u vo.
Trang 163
S nhp nhng (equivocation)

v tc truyn tin
Xt mt knh nh phn i xng vi xc sut cho . Gi s

rng ti u vo P(0) = P(1) = 1/2, tc sinh thng tin u
pht l H(x) = 1 bit/k hiu.
Mt thit b c gi l b quan st, nhn mi cp k hiu
vo/ra (x, y) v sinh ra mt k hiu z
z = 0 nu x = y, z = 1 nu x y
B quan st
x(2)x(1)
Knh
z(2)z(1)
y(2)y(1)
Trang 164
S nhp nhng (equivocation)

v tc truyn tin (tt)
S phn b ca z c tm thy nh sau:

P(z = 1) = P(x = 0) P(y = 1 | x = 0) + P(x = 1) P(y = 0 | x = 1)
= /2 + /2 =
P(z = 0) = 1 P(z = 0) = 1
Tc sinh thng tin bi b quan st v vy bng
H(z) = log (1 ) log(1 ) bits/k hiu
i vi mt dy u ra cho y(1)y(2)..., ni nhn (receiver) c
th xy dng li chnh xc dy u vo x(1)x(2)... ch khi u ra
ca b quan st z(1)z(2)... c to sn.
Tc truyn thng tin trn knh, thng k hiu l R, l bng
tc sinh thng tin H(x) tr tc sinh thng tin b sung
H(z).
R = H(x) H(z)
Trang 165
V d
Chng hn, nu d liu u vo c sinh tc 1000 k

hiu/giy v = 0,01, chng ta c
H(x) = 1
tc d liu u vo = 1000 bits/giy
H(z) = 0,081 tc d liu b sung = 81 bits/giy
R
= 0,919 tc truyn thng tin = 919 bits/giy
Mt ngi c th l lun rng trong mt dy di, v = 0,01,
ngha l ch 1% s bit c truyn b li, v v vy tc
truyn thng tin phi l 990 bits/giy.
Cu tr li l rng kin thc v s bit b li khng xy
dng li d liu, m chng ta cn phi bit thm v v tr li
na, v v l do ny nn tc truyn thng tin l thc s bng
mt gi tr thp hn l 919 bits/giy.
Trang 166
Nhn xt
Trong trng hp tt nht = 0, chng ta c H(z) = 0 v v vy

R = 1000 bits/giy.
Trong mt trng hp khc nu = 1/2, th H(z) = 1, kt qu l
R = 0 bits/giy.
C hai kt lun l nht qun vi s mong i ca chng ta.
i vi knh nh phn i xng vi u vo ng xc sut,
chng ta chng minh c rng H(z) = H(x | y).
Tng qut chng ta chng minh c rng
S ti xy dng chnh xc dy u vo t dy u ra l c th
nu b quan st c th sinh ra thng tin b sung tc ln
hn hay bng H(x | y).
Trang 167
Nhn xt (tt)
thy iu ny mt cch trc quan, quan st rng i vi cc

dy di c chiu di N c khong 2NH(x | y) dy u vo c th
sinh ra mt dy u ra c th.
Ch khi thng tin b sung c sinh ra ti tc H(x | y) hay
nhanh hn mi cho php phn bit gia cc kh nng ny.
i vi l do ny, H(x | y) thng c coi nh l s nhp
nhng (equivocation) ca knh. V chng ta nh ngha li tc
truyn thng tin trn knh l
R = H(x) H(x | y) = I(x, y)
Trang 168
Dung lng knh
Theo phn trn tc truyn tin trn knh c nh ngha l

R = H(x) H(x | y) = I(x, y)
I(x, y) tng qut l mt hm ca s phn b xc sut u vo
{p1, , pK}. V vy, c th tm thy mt s phn b m cc i
I(x, y). Gi tr cc i ca I(x, y) c nh ngha l dung
lng knh C v l mt hm ca ma trn knh.
C = Cc i (trn cc s phn b xc sut u vo) ca I(x, y).
Tng qut, vic tnh dung lng knh l mt bi ton kh v l
mt bi ton cha c gii mt cch trit .
Tuy nhin i vi cc knh c gii thiu trn C c th
tnh ton c nh phn sau y trnh by.
Trang 169
Knh i xng
C = log J +
'
'
p
log
p
j
j
j =1
trong p1, , pJ l cc phn t ca cc hng ca ma trn.

Trong trng hp knh nh phn i xng vi xc sut cho l
p chng ta c
C = 1 H(p) vi H(p) = plogp (1p)log(1p)
Knh khng mt
H(x | y) = 0, v vy
C = Max {H(x) H(x | y)} = Max{H(x)} = log K
trong K l kch thc ca bng k hiu u vo. Dung lng
t c trong trng hp u vo c s phn b ng xc
sut.
Trang 170
Knh n nh
y H(y | x) = 0, v vy
C = Max {H(y) H(y | x)} = Max{H(y)} = log J
trong J l kch thc ca bng k hiu u ra.
Knh v dng
y H(x | y) = H(x), v vy
C = Max {H(x) H(x | y)} = Max{H(x) H(x)} = 0
Mt knh v dng th c dung lng knh bng 0.
Trang 171
Bi 10 M ha chng nhiu,
nh l knh
10.1 Gii thiu bi ton chng nhiu
10.2 nh l knh c nhiu cho knh nh phn i xng ri
rc (BSC)
10.3 nh l ngc ca knh truyn c nhiu
Trang 172
Gii thiu bi ton chng nhiu
Mc tiu chng nhiu l bn nhn c th on (gii m) c

cng chnh xc cng tt dy k hiu c pht.
Chng hn xt ngun nh phn i xng vi xc sut cho ,
ng thi gi s ngun pht ng xc sut, tc P(0) = P(1) =
1/2.
Vi < 1/2, xt c ch gii m bn nhn nh sau: Nu y = 0
th on x = 0 v nu y = 1 th on x = 1.
Xc sut gii m b li ca c ch ny l
P(li) = P(y = 0) P(x = 1 | y = 0) + P(y = 1) P(x = 0 | y = 1) =
/2 + /2 = .
Ch trong trng hp y chng ta tnh c
P(y = 0) = P(y = 1) = 1/2 v P(x y | y) = .
Vn quan trng l c th gim c xc sut gii m b li
hay khng?
Trang 173
Gii thiu bi ton chng nhiu (tt)
Mt hng gii quyt nh sau: gi 0 chng ta gi chui 3 k

hiu 0 v tng t gi 1 chng ta gi 3 k hiu 1.
C ch gii m ca bn nhn nh sau: Nu chui nhn c nhiu
k hiu 0 hn 1 th gii m thnh 0 v ngc li.
Chng hn bn nhn nu nhn c 010 th gii m thnh 0 cn
nu nhn c 110 th gii m thnh 1.
C ch ny c xc sut gii m b li l
P(li) = 3(1 )2 + 3 <
Xc sut ny nh hn . Tuy nhin hiu sut truyn thng tin b
gim xung 3 ln.
Tng t nu mun xc sut gii m tin n 0 chng ta s m
ho 0 thnh dy 2n + 1 k hiu 0 v m ho 1 thnh 2n + 1 k
hiu 1, nhng tng ng lc ny hiu sut truyn thng tin
gim xung 2n + 1 ln so vi ban u.
Trang 174
Gii thiu bi ton chng nhiu (tt)
C mt cch c th gim xc sut gii m li xung gn bng 0

nhng khng gim hiu sut truyn thng tin xung gn bng 0
m ch cn nh hn mt ngng no l .
Ngng chnh l dung lng knh.
Cch ny cng khai thc tng trn ch thay v gi i 0
v 1, ci m c khong cch Hamming gia chng l 1 th
chng ta s m ho ln lt thnh 000 v 111, ci m c
khong cch Hamming gia chng l 3 v v vy gim xc
sut gii m b li.
Trang 175
nh l knh c nhiu cho knh

nh phn i xng ri rc (BSC)
Xt knh nh phn i xng vi xc sut cho p.

Dung lng knh trong n v bits/k hiu l
C = 1 H(p) vi H(p) = plogp (1p)log(1p)
Gi s thi gian truyn 1 k hiu l T, s k hiu c truyn
trong 1 giy l 1/T, th dung lng theo n v bits/giy l
C = [1 H(p)]/T
Xt ngun X c entropy H(X) bits/k hiu, tc l ngun ny to
ra thng tin tc theo n v bits/giy.
R = H(X)/T
nh l 10.1.
Chng no m R (bits/giy) cn nh hn C (bits/giy), th vic

truyn trn knh vi t l li nh tu l c th thc hin c.
chng minh nh l ny cn mt s khi nim sau.
Trang 176
Cc khi nim
Trng s Hamming
Khong cch Hamming
Trng s Hamming ca mt dy k hiu v = a1a2...an , trong

mi ai {0, 1, ..., m1}, l s k hiu khc 0 ca dy, v
thng c k hiu l w(v).
Khong cch Hamming ca hai dy k hiu v1, v2 vi chiu di
bng nhau l s v tr khc nhau ca hai dy, v thng c k
hiu l d(v1, v2).
Php cng c s m,
Xt a, b {0, 1, ..., m1} th a b = (a + b) mod m.

Nu v1 = a1a2...an, v2 = b1b2...bn th v1 v2 = c1c2...cn trong ci
= ai bi vi i = 1, 2, ..., n.
Trang 177
Cc khi nim (tt)
V d
w(10100) = 2, w(01120) = 3.
d(10100, 10001) = 2, d(011010, 101101) = 5.

Vi m = 2 th 1011 1101 = 0110. Vi m = 3 th 1021 2120
= 0111.
B
Nhn xt
d(v1, v2) = w(v1 v2 )

d(v1, v2) + d(v2, v3) d(v1, v3)
Bt ng thc th hai c dng ca bt ng thc tam gic: tng

hai cnh ca mt tam gic ln hn hoc bng cnh cn li.
nh l 10.1 ng cho knh ri rc khng nh bt k. Tuy
nhin y chng ta ch chng minh cho knh nh phn i
xng ri rc.
Trang 178
Chng minh nh l
tng chng minh l m ho cc dy d liu thnh cc t m,

trong cc k hiu m ly t bng k hiu u vo ca knh v
x l cc t m ny nh cc u vo c bn ca knh.
Xc sut li nh tu c th t c da trn s m ho nh
sau:
(1) chn chiu di N ca dy d liu di
(2) m ho cc dy ny thnh cc t m c khong cch
Hamming xa nhau.
Nguyn tc gii m u ra c thit k nh sau: dy k hiu
nhn c u ra vj s c gii m thnh t m wi m c
khong cch Hamming nh nht i vi vj.
Vi cch chn ny xc sut gii m li l nh nht. Tht vy
p(wi | vj) = p(wi)p(vj | wi)/p(vj)
Trang 179
Chng minh nh l (tt)
Do khi chng ta khng r v p(wi) v d nhin s ko theo

p(vj) th p(wi | vj) ln nht khi p(vj | wi) l ln nht. M
p(vj | wi) = pD(1p)ND
trong D l khong cch Hamming gia vj v wi, N l chiu di
ca chng, p l xc sut cho.
Nu xc sut cho p < 0,5 th p(vj | wi) s ln nht khi D l nh
nht.
Chng minh rng > 0 nh tu , vi N ln tn ti cch
m ho cc dy d liu thnh cc t m sao cho vi nguyn tc
gii m trn c xc sut gii m li l nh hn .
Tht vy s dy d liu c chiu di N l vo khong
M = 2NH(X) = 2NRT
trong khi tng s dy c chiu di N l 2N.
Trang 180
Chng minh nh l (tt)
Gi {w1, w2, , wM} l mt tp t m bt k, Pe l xc sut gii

m li i vi tp ny.
Nu chng ta chng minh c rng > 0 nh tu , vi N
ln gi tr trung bnh ca Pe, Pe , nh hn th s tn ti mt
tp t m m c xc sut gii m li Pe nh hn .
Vi xc sut li trn ng truyn l p, mt dy c chiu di N
s c trung bnh Np v tr li.
Vi hai s dng , nh tu , theo lut yu ca s ln vi N
ln th xc sut s v tr ca chui nhn vj khc vi chui
pht wi ln hn N(p + ) l nh hn . Hay ni theo ng cnh
ca khong cch Hamming l
P[d(wi, vj) > N(p + )] <
V vy b m m chng ta mong mun s nh sau: Khong
cch Hamming gia hai t m bt k l 2N(p + ) + 1
Trang 181
Chng minh nh l (tt)
Nh vy vi mi vj nhn c theo bt ng thc tam gic tn

ti mt t m wi m c
d(wi, vj) N(p + )
cn cc t m wk khc c
d(wk, vj) N(p + ) + 1
V vy chng ta s gii m c duy nht vj thnh wi.
Vi tng ny, chng ta s a ra c ch gii m lng hn
cho mt tp t m bt k {w1, w2, , wM}, nhng cng s m
bo xc sut gii m li l nh hn .
Vi mi dy vj nhn c, nh ngha mt tp kim tra Aj bao
gm tt c nhng dy c chiu di N v c khong cch
Hamming so vi vj nh hn hay bng N(p + ).
Nu t m c truyn wi l t m duy nht thuc tp Aj th gii
m vj thnh wi. Ngc li thng bo mt li xy ra.
Trang 182
Chng minh nh l (tt)
Mt li xy ra thuc vo mt trong hai trng hp sau y

(1) T m c truyn wi khng thuc Aj, tc l
d(wi, vj) > N(p + )
Li ny xy ra vi xc sut nh hn .
(2) Tn ti mt t m wk khc cng thuc Aj. Lc ny chng ta
khng bit nn gii m vj thnh wi hay wk.
Chng ta chng minh rng theo cch ny xc sut gii m li
trung bnh s nh hn vi nh tu cho trc.
Chng ta c
Pe +
P(wi A j )
i =1
i j
Trang 183
Chng minh nh l (tt)
tnh Pe chng ta s tnh gi tr trung bnh ca P(wi Aj).

Gi tr trung bnh ny s bng s dy thuc tp Aj chia cho tng
s dy
N ( p + )
Suy ra
P (Wi A j ) =
N
k
k =0
2N
N ( p + )
N
k
k =0
Pe < + ( M 1)
2N
Trang 184
Chng minh nh l (tt)
M chng ta c mt bt ng thc ni ting sau

N N

NH ( )

k2
k =0
vi H() = log (1)log(1).
p dng vo bt ng thc trn chng ta c
Pe + M2N [1 H(p + )] = + 2NRT 2N [1 H(p + )]
= + 2N [1 H(p + ) RT]
V v c th nh tu , nn chng no R < [1 H(p)]/T = C
(bits/giy) th c th lm cho Penh tu bng cch tng N.
Chng minh c hon tt.
Trang 185
V d
Xt v d trc y, mt knh i xng nh phn c xc sut

cho = 0,01. Tc truyn k hiu f = 1000 k hiu/giy (tc
T = 0,001 giy). Chng ta c C = 0,919 bits/k hiu hay C = 919
bits/giy.
nh l knh cho php chng ta kt lun, vi xc sut ng tin
ti 1, rng vi N kh ln chng hn N = 1000, th trong 21000
dy c chiu di 1000 chng ta c th chn c 2K dy vi K
< 919 sao cho khong cch Hamming gia cc dy l 2N + 1
= 21.
Khong cch Hamming ca b m
Khong cch Hamming ca mt b m A, vi iu kin A l m

u, k hiu l d(A), l khong cch Hamming nh nht trong
tt c cc khong cch gia hai t m bt k ca A.
Trang 186
nh l
nh l 10.2
Mt b m nh phn c khong cch Hamming d th c th

Pht hin sai c t bit nu d t + 1.
Sa sai c t bit nu d 2t + 1.
Chng minh
Gi wi l t m pht, vi l dy nhn tng ng. Nu sai ti a t

> 0 bit th d(wi, vi) t. Do t hp nhn s khng th trng
vi bt k t m no v khong cch Hamming gia hai t m
bt k l t + 1. V vy bn nhn c th pht hin c sai.
Tng t nu d(wi, wj) 2t + 1, theo bt ng thc tam gic
d(wj, vi) t + 1 t m wj wi. V vy bn nhn c th gii m
ng vi thnh wi da trn s khc bit ny.
Trang 187
nh l ngc ca knh truyn c nhiu
nh l 10.2
Nu tc truyn tin R (bits/giy) ln hn dung lng knh C

(bits/giy), th s truyn thng trn knh vi t l li nh tu
l khng th thc hin c. Hay ni cch khc xc sut gii
m li tin n 1 khi chiu di ca dy cn truyn gia tng.
nh l ny ni cch khc nu tc truyn tin ln hn dung
lng knh th vic truyn khng c m bo c ngha l
chng ta khng th gii m ng c.
Trang 188
Bi 11 C s ton hc ca m chng nhiu
Bi ny trnh by cc c s ton hc ca m khi tuyn tnh.

Cc kin thc ny l rt quan trng hiu c cch xy dng
cc loi m khi tuyn tnh.
Cc khi nim c trnh by bao gm cc cu trc i s nh
nhm, trng v c bit l cc trng GF(2) v GF(2m), y l
cc trng c ng dng c bit vo trong vic xy dng cc
m khi tuyn tnh chng nhiu.
Trang 189
Bi 11 C s ton hc ca m chng nhiu

11.1 Mt s khi nim c bn
11.2 Trng GF(2) v cc a thc trn trng GF(2)
11.3 Trng GF(2m)
Trang 190
Mt s khi nim c bn
Php ton ng
Cho G l mt tp hp, mt php ton hai ngi f c gi l

ng trn G nu f c dng
f:GGG
tc l nu a, b G th f(a, b) G.
Ch
f(a, b) c mt cch vit tng ng l afb v ngc li f(b, a)

cn c vit l bfa. Chng hn nu f l php cng th thay v
vit +(a, b) chng ta thng vit l a + b.
K t y tr v sau khi ni n mt php ton nu chng ta
khng ni g thm th c ngha l php ton ny c tnh ng.
Trang 191
Mt s khi nim c bn (tt)
Tnh kt hp
Tnh giao hon
Mt php ton hai ngi f trn G c gi l c tnh kt hp nu

a, b, c G th
(afb)fc = af(bfc)
Mt php ton hai ngi f trn G c gi l c tnh giao hon
nu a, b G th
afb = bfa
V d
Trn tp s thc khc 0, php cng v php nhn c tnh kt

hp v giao hon nhng php tr v php chia khng c tnh
kt hp v giao hon.
Trang 192
Nhm
Tnh phn phi
Php ton f1 c gi l c tnh phn phi i vi php ton f2

nu a, b, c G th
af1(bf2c) = (af1b)f2(af1c)
Chng hn trn tp s thc, php nhn c tnh phn phi i vi
php cng v a, b, c R
a(b+c) = (ab)+(ac)
Nhm
Mt tp G , vi mt php ton hai ngi f c gi l mt

nhm nu tho 3 iu kin sau:
(1) f c tnh kt hp.
Trang 193
Nhm (tt)
(2) G cha phn t e, sao cho a G th afe = efa = a. e c

gi l phn t trung ho (i vi mt s php ton e cn
c gi l phn t n v)
(3) Mi phn t u c phn t i xng, tc l a G, tn
ti phn t b G sao cho
afb = bfa = e
Chng hn, trn tp R nu f l php cng th phn t trung ho
l s 0, cn trn tp s thc khc 0 nu f l php nhn th phn
t trung ho l 1 v cn c gi l phn t n v.
Nhm giao hon
Mt nhm m php ton f c tnh giao hon th c gi l

nhm giao hon.
Trang 194
Nhm (tt)
Nhm hu hn, nhm v hn
Mt nhm c s phn t hu hn c gi l nhm hu hn,

mt nhm c s phn t v hn c gi l nhm v hn.
Nhm con
Cho G l mt nhm. Mt tp H con ca G c gi l mt

nhm con nu H ng vi php ton hai ngi ca G v tho
iu kin ca mt nhm.
Tp cc s chn 0 l mt nhm con ca tp s t nhin vi
php cng thng thng.
Trang 195
Php cng v nhn modulo
Php cng modulo v php nhn modulo
Cho mt s nguyn dng m xc nh. Xy dng mt tp s

nguyn sau G = {0, 1, , m 1}. Vi + l php cng thng
thng. nh ngha php ton mi nh sau v gi l php
cng modulo
a, b G th a b = (a + b) mod m
Tng t vi l php nhn thng thng. nh ngha php
ton mi nh sau v gi l php nhn modulo
a, b G th a b = (a b) mod m
Trang 196
V d
Tp R l mt nhm giao hon i vi php cng v l mt

nhm v hn.
Tp R {0} l mt nhm giao hon i vi php nhn v l
mt nhm v hn.
Vi m l mt s nguyn dng xc nh, tp G = {0, 1, , m
1} vi php cng modulo l mt nhm giao hon v l mt
nhm hu hn.
Hai bng sau y trnh by ln lt trng hp m = 5 v m = 6.
Trang 197
V d (tt)
m=5
0
1
2
3
4
0
0
1
2
3
4
1
1
2
3
4
0
2
2
3
4
0
1
m=6
3
3
4
0
1
2
4
4
0
1
2
3
0
1
2
3
4
5
0
0
1
2
3
4
5
1
1
2
3
4
5
0
2
2
3
4
5
0
1
3
3
4
5
0
1
2
4
4
5
0
1
2
3
5
5
0
1
2
3
4
Tng t tp G = {1, , m 1} vi php nhn modulo v m

nguyn t l mt nhm giao hon hu hn.
Trang 198
B 11.1
Nu m l mt s nguyn t th G = {0, 1, , m 1} l mt
nhm giao hon vi php nhn modulo . Ngc li nu m
khng nguyn t th G khng l mt nhm.
1
2
3
4
m=5
1 2 3
1 2 3
2 4 1
3 1 4
4 3 2
4
4
3
2
1
1
2
3
4
5
m=6
1 2 3 4
1 2 3 4
2 4 0 2
3 0 3 0
4 2 0 4
5 4 3 2
Trang 199
5
5
4
3
2
1
Trng
Trng
Mt tp G vi hai php ton ng hai ngi bt k, chng hn k

hiu l + v *, c gi l mt trng nu tho 3 iu kin sau
(1) G l nhm giao hon i vi php +. Phn t trung ho
trong php + thng c k hiu l 0.
(2) Tp cc phn t khc 0 l mt nhm i vi php *. Phn t
trung ho trong php * thng c gi l phn t n v
v k hiu l 1.
(3) Php * c tnh phn phi i vi php +.
Ch
Php + v php * trn khng nht thit l php cng v php

nhn thng thng m chng c th l bt k php no. Chng
ta k hiu nh vy d trnh by.
Trang 200
Trng (tt)
Cc phn t ca mt trng khng nht thit l cc s nguyn

hay thc m c th l bt k ci g, chng hn c th l cc s
phc, vect, ma trn hay a thc ...
T nh ngha ca trng chng ta suy ra mt trng bao gm
ti thiu hai phn t: phn t trung ho ca php + (k hiu l 0)
v phn t trung ho ca php * (k hiu l 1). Cc phn t 0 v
1 khng nht thit l s 0 v s 1 theo ngha thng thng m
c th l bt k ci g chng hn l ma trn 0 v ma trn n v,
...
Trng giao hon
Mt trng m php * c tnh giao hon th c gi l trng

giao hon.
Trang 201
Trng (tt)
B 11.2
Chng hn trong v d slide 198 vi m = 5 chng ta thy G l

mt trng giao hon.
Tng qut chng ta c b sau v dnh vic chng minh
cho cc bn sinh vin.
Cho p l mt s nguyn t bt k, G = {0, 1, ..., p 1} th G l
mt trng giao hon i vi php cng modulo v php
nhn modulo .
Sau y l mt s tnh cht ca trng
Tnh cht 1
Mi phn t a ca trng u tho a * 0 = 0.

Trang 202
Trng Galois
Tnh cht 2
Tnh cht 3
Nu a 0 v a * b = a * c th b = c. Hay ni cch khc nu a

0 v b c th a * b a * c.
Bc ca mt trng, trng hu hn, trng v hn.
Nu a, b l hai phn t khc 0 ca trng th a * b 0.
S phn t ca mt trng c gi l bc ca mt trng. Mt

trng c s phn t hu hn c gi l trng hu hn, mt
trng c s phn t v hn c gi l trng v hn.
Trng GF(q)
Mt trng c s phn t hu hn c gi l trng Galois.

Nu bc ca trng Galois l q th trng c k hiu l
GF(q).
Trang 203
Trng Galois
i vi cc trng hu hn tc l trng Galois chng ta c

nh l sau.
nh l 11.1
Mt trng hu hn th s phn t ca n phi c dng pm trong

p l mt s nguyn t cn m l mt s nguyn dng. Hay
ni cch khc cc trng Galois u c dng GF(pm) trong p
l mt s nguyn t cn m l mt s nguyn dng.
i vi cc trng GF(p) vi p nguyn t th chnh l tp
{0, 1, 2, ..., p 1} vi hai php ton cng modulo v nhn
modulo nh bit.
i vi cc trng GF(pm), v tnh phc tp ca chng, chng
ta s gii thiu sau. Ch lc ny cc phn t ca trng
GF(pm) khng n thun l cc s m s c dng kh c bit.
Trang 204
Trng Galois (tt)
K hiu cc phn t i xng
Phn t i xng ca a trong php + c k hiu l a, phn t

i xng ca a trong php * c k hiu l a1.
Php v php /
i vi mt trng giao hon, t hai php + v php * chng ta

nh ngha thm hai php v php / nh sau (khng nht thit
l php tr v php chia bnh thng)
a b = a + (b)
a / b = a * b1
trong b l phn t i xng ca b qua php +, cn b1 l
phn t i xng ca b qua php *.
Vy mt trng giao hon G c bn php ton +, , *, /. Php +
v ng trn G, php * v / ng trn G {0}.
Trang 205
Tr ring ca mt trng
Xt mt trng GF(q). Xt cc dy tng ca cc phn t n v

k
1 = 1 + 1 + L + 1
(k ln, vi k = 1, 2, 3, )
i =1
V trng ng vi php cng nn kt qu ca nhng tng ny

cng l cc phn t ca trng. V k c th nhn v hn gi tr
m trng ch c q phn t nn tn ti hai gi tr k1 v k2 khc
nhau (gi s k1 > k2 ) sao cho
T y suy ra
k1
k2
i =1
i =1
1 = 1
k1 k 2
1 = 0
i =1
Trang 206
Tr ring ca mt trng
Tr ring ca mt trng k hiu l s nguyn dng nh nht
sao cho
1 = 0
i =1
nh l 11.2
D thy i vi cc trng GF(p) = {0, 1, 2, ..., p 1} vi p l

mt s nguyn t th tr ring = p. Tng qut chng ta c nh
l sau.
Tr ring ca mt trng GF(q) l mt s nguyn t.
Chng minh
Gi s khng nguyn t = k l (k, l nguyn > 1). T qui

tc phn phi ca php nhn i vi php cng suy ra
Trang 207
Tr ring ca mt trng (tt)

k
k l
i =1
i =1
i =1
i =1
1 1 = 1 = 1 = 0
Suy ra
1 = 0
i =1
1 = 0
i =1
M k, l < , iu ny mu thun vi nh ngha ca .
Chu k ca mt phn t
Xt mt phn t a bt k khc 0 ca trng GF(q). Xt cc lu

tha ak ca a vi k = 1, 2, 3, V trng ng vi php nhn
nn cc ak cng l cc phn t ca trng. V k c th nhn v
hn gi tr m trng ch c q phn t nn tn ti hai gi tr k1
v k2 khc nhau (gi s k1 > k2 ) sao cho
a k1 = a k 2 a k1 k 2 = 1
Trang 208
Chu k ca mt phn t
Chu k ca mt phn t a ca mt trng GF(q) l s nguyn

dng nh nht n sao cho an = 1.
V d
Xt trng GF(7) = {0, 1, 2, 3, 4, 5, 6} vi hai php v .

Tr ring ca trng ny l 7. Cn chu k ca cc phn t khc
0 ca trng c trnh by trong bng sau
Phn t
Chu k
1
1
2
3
3
6
4
3
5
6
6
2
T nh ngha trn chng ta thy dy cc lu tha ca a

a1, a2, ..., ak, ..., an = 1, an+1 = a, ...
s lp li sau n phn t.
Trang 209
Nhm tun hon
B 11.3
Nhm tun hon
Dy a1, a2, ..., ak, ..., an = 1 to nn mt nhm con ng vi

php nhn trn trng GF(q).
Mt nhm (khng cha phn t 0) vi php nhn * c gi l
tun hon nu tn ti mt phn t trong nhm m cc lu tha
ca n to nn mi phn t trong nhm.
T nh ngha ny suy ra mt nhm hu hn c gi l tun
hon nu tn ti mt phn t trong nhm c chu k ng bng
s phn t ca nhm.
nh l 11.3
Nu a l mt phn t khc 0 ca mt trng GF(q) th

q1 = 1
a
Trang 210
Nhm tun hon (tt)
Chng minh
Gi b1, b2, ..., bq-1 l q 1 phn t khc nhau v khc 0 ca

trng. Theo tnh cht 3 v tnh cht 2 ca trng chng ta c
a*b1, a*b2, ..., a*bq-1 cng l q 1 phn t khc nhau v khc 0
ca trng. V vy chng ta c
a*b1*a*b2* ... *a*bq-1 = b1*b2* ... *bq-1
T y suy ra aq1 = 1. Hon tt chng minh.
nh l 11.4
Chu k ca mt phn t bt k khc 0 ca mt trng GF(q) l

c s ca q 1.
Trang 211
Phn t c s
Chng minh
Gi n l chu k ca phn t a khc 0 ca trng GF(q). Gi s

q 1 khng chia ht cho n. Do q 1 = kn + r, trong r l
s d ca php chia q 1 cho n, 0 < r < n. Chng ta c
aq-1 = akn+r = (an)k*ar
Do aq-1 = 1 v an = 1 suy ra ar = 1. M 0 < r < n iu ny mu
thun vi nh ngha chu k ca a. Vy q 1 chia ht cho n.
Phn t c s
Mt phn t a khc 0 ca mt trng GF(q) c gi l phn

t c s nu chu k ca a bng q 1.
T nh ngha ny nu a l mt phn t c s th cc lu
tha ca a gm a0 = 1, a1 = a, a2, , aq 2 hnh thnh nn q 1
phn t khc 0 ca trng.
Trang 212
V d
Xt trng GF(7) nh trong v d slide 209. Chu k ca cc

phn t khc 0 ca trng u l c s ca 6. c bit cc
phn t 3 v 5 c chu k bng 6 nn chng l cc phn t c s
ca trng GF(7).
31 = 3 32 = 2 33 = 6 34 = 4 35 = 5 36 = 1
51 = 5 52 = 4 53 = 6 54 = 2 55 = 3 56 = 1
Trong cc trng Galois th trng GF(2) v trng GF(2m) l

nhng trng c nhiu ng dng c bit trong l thuyt m,
nn chng ta s ch trnh by hai trng ny.
Trang 213
Trng GF(2)
Trng GF(2)
Trng GF(2) bao gm hai phn t {0, 1} vi hai php cng +

v nhn * nh sau
+
0
1
0
0
1
1
1
0
*
0
1
0
0
0
1
0
1
Phn t i xng ca 0 v 1 qua php cng cng chnh l 0 v

1. Phn t i xng ca 1 qua php nhn cng chnh l 1.
Trong trng GF(2) th php tr ging vi php cng, php
chia cho mt s khc 0 cng ging vi php nhn.
Trang 214
Cc a thc trn trng GF(2)
a thc trn trng GF(2)
Bc ca a thc
Mt a thc trn trng GF(2), chng hn k hiu l f(x), l a

thc c dng
f(x) = a0 + a1x + a2x2 + + anxn
trong cc h s ai GF(2).
L bc ln nht ca a thc.
V d
a thc f(x) = 1 + x + x3 c bc 3 a thc g(x) = x + x2 + x5 c

bc 5.
Trang 215
Cc a thc trn trng GF(2) (tt)
Php cng a thc v nhn a thc
Vi f(x) = a0 + a1x + a2x2 + + anxn, g(x) = b0 + b1x + b2x2 +

+ bnxn vi cc h s ai v bj thuc trng GF(2) chng ta
nh ngha cc php cng a thc v nhn a thc nh sau
n
i
+
(
a
b
)
x
f(x) + g(x) =
i i
i=0
n
f(x) * g(x) = (ai *bj )xi + j
, j = 0hin trn trng GF(2).
trong ai + bi v ai * bj cithc
Trang 216
V d
Cho f(x) = 1 + x + x3, g(x) = x + x2 th

f(x) + g(x) = (1 + x + x3) + (x + x2) = 1 + x2 + x3
f(x) * g(x) = (1 + x + x3) * (x + x2) = x + x3 + x4 + x5
Nu g(x) c bc khc 0 th chng ta c th chia f(x) cho g(x) v
c th vit nh sau
f(x) = q(x) * g(x) + r(x)
trong q(x) l a thc thng cn r(x) l a thc d c bc
nh hn a thc chia g(x).
V d
f(x) = 1 + x + x4 + x5 + x6 chia cho g(x) = 1 + x + x3

Trang 217
1 + x + x4 + x5 + x6 = (x2 + x3) * (1 + x + x3) + (1 + x + x2)

phn tch mt a thc ra thnh cc tha s trong i s
Euclid chng ta c
Nu f(a) = 0 th f(x) chia ht cho (x - a).
iu ny cng ng trn trng GF(2).
V d
f(x) = 1 + x + x3 + x5 c f(1) = 0, nn f(x) chia ht cho (x - 1)

m trong trng GF(2), php tr cng chnh l php cng tc l
f(x) chia ht cho (x + 1).
1 + x + x3 + x5 = (1 + x)(1 + x3 + x4)
Trang 218
a thc ti gin
Mt a thc trn GF(2) c gi l ti gin nu n khng phn

tch c thnh tch ca hai a thc c bc nh hn.
1, 2, 3, 4
x
1+x
1 + x + x2
1 + x + x3
1 + x2 + x3
1 + x + x4
1 + x3 + x4
1 + x + x2 + x3 + x4
5
1 + x2 + x5
1 + x3 + x5
1 + x + x2 + x3 + x5
1 + x + x2 + x4 + x5
1 + x + x3 + x4 + x5
1 + x2 + x3 + x4 + x5
Trang 219
6
1 + x + x6
1 + x3 + x6
1 + x + x2 + x4 + x6
1 + x + x3 + x4 + x6
1 + x5 + x6
1 + x + x2 + x5 + x6
1 + x2 + x3 + x5 + x6
1 + x + x4 + x5 + x6
1 + x2 + x4 + x5 + x6
Cho f(x) l mt a thc trn trng GF(2), th
f (x)
2n
= f (x
2n
Chng minh
t f(x) = a0 + a1x + + anxn.

[f(x)]2 = (a0 + a1x + + anxn)2
= a02 + a0*(a1x + + anxn) + a0*(a1x + + anxn) +
(a1x + + anxn)2
= a02 + (a1x + + anxn)2
= a02 + (a1x)2 + + (anxn)2
= f(x2) (v trong GF(2) ai2 = ai )
iu ny cng gip chng ta suy ra iu phi chng minh.
Trang 220
Trng GF(2m)
Trc ht chng ta k hiu trng GF(2m) nh sau

GF(2m) = {0, 1, a1, a2, ..., a m }
2 2
trong 0 v 1 GF(2). Trng GF(2) l mt trng con ca
GF(2m) v c gi l trng c s ca GF(2m).
Ch
Nu a l mt phn t GF(2m), f(x) l mt a thc trn trng

GF(2), th f(a) cng l mt phn t ca GF(2m).
C v hn a thc f(x) trn trng GF(2) m ch c hu hn
(2m) phn t GF(2m), nn a 0 ca GF(2m) tn ti hai a
thc f1(x) v f2(x) khc nhau sao cho f1(a) = f2(a). T y nu
t f(x) = f1(x) f2(x) (ch trong trng GF(2) th php
ging vi php cng +) th f(a) = 0.
Trang 221
a thc ti thiu
a thc ti thiu (minimal polinomial)
Cho a l mt phn t khc 0 ca trng GF(2m). a thc ti

thiu ca a l a thc f(x) khc 0 trn trng GF(2) v c bc
nh nht sao cho f(a) = 0.
Mt ln na ta phi ch rng khi chng ta vit f() = 0 hoc
f() = 1 th cc k hiu 0 v 1 khng nht thit l cc s 0 v 1,
m s c hiu tu theo ng cnh.
Chng hn nu phn t l mt ma trn th 0 chnh l ma trn
0 cn 1 chnh l ma trn n v.
Trang 222
a thc ti thiu (tt)
1 0 0
0 1 0
0 0 1
1 0 0
trong cc php cng v nhn trn ma trn vn thc hin nh
bnh thng vi ch rng cc php cng v nhn cc phn t
ca ma trn c thc hin trn trng GF(2).
Chng ta c th kim tra rng
1 + T + T4 = 0
vi ch rng 1 l ma trn n v, cn 0 l ma trn 0.
V f(x) = 1 + x + x4 l a thc ti thiu ca a
0
V d
0
T4 4 =
Chng hn nu a l ma trn 4 4 bn
0
Trang 223
nh l
Hn na chng ta cng c
T15 = 1
v chng ta c th kim tra rng 15 chnh l chu k ca a.
nh l 11.5
Cho a l mt phn t khc 0 ca trng GF(2m) c bc ca a

thc ti thiu ca a l k. Gi Z l tp tt c cc phn t c dng
b0 + b1a + + bk-1ak-1
trong bi GF(2). Th Z l mt tp con ca GF(2m) v hnh
thnh nn mt trng c 2k phn t.
Trang 224
Chng minh
u tin chng ta chng minh cc phn t c hnh thnh t

b0 + b1a + + bk-1ak-1 l khc nhau bng cch chng minh cc
phn t 1, a, a2, , ak-1 l c lp tuyn tnh.
k 1
k 1
Tht vy nu
i
i
b
a
=
c
a
i
i
th
i=0
p(a) =
i=0
k 1
i
(
b
c
)
a
i i =0
i=0
Vy chng ta c a thc p(x) c bc nh hn k tho p(a) = 0.

M bc ca a thc ti thiu ca a bng k. Vy p(x) = 0, suy ra
bi = ci i = 0, 1, ..., k 1.
Trang 225
Chng minh (tt)
Th hai, r rng Z l mt nhm giao hon i vi php +.

Tht vy nu
k 1
k 1
k 1
k 1
i=0
i=0
i=0
i=0
i
b
a
i Z,
i
c
a
i Z
i
(
b
+
c
)
a
i i =
i
(
c
+
b
)
a
i i Z
chng minh tp Z0 = Z {0} l mt nhm i vi php nhn

* chng ta chng minh nu
k 1
k 1
k 1
k 1
i=0
i=0
i=0
i=0
i
i
c
a
,
b
a
Z
i
0 i Z0
Gi f ( x) =
i
i
b
a
*
c
a
i i Z0
i
d
x
i l a thc ti thiu ca a, trong h s
i=0
cao nht dk = 1.
Trang 226
Chng minh (tt)

T y suy ra
xk =
k 1
i
d
x
i
i=0
Do mi an vi n k u c th biu din thng qua mt a

thc g(a) no c bc k 1. V vy k 1 i k 1 i
bi a * ci a
cng vy. Suy ra
V r rng nu
k 1
k 1
k 1
i=0
i=0
i=0
i=0
i
i
b
a
*
c
a
i i Z
k 1
bi a 0, ci a 0
i
i=0
i=0
k 1
k 1
i
b
a
*
c
a
i i 0
i=0
i=0
Tnh cht ny c k tha t trng GF(2m).

Trang 227
H qu
H qu 11.1
Cui cng tnh phn phi ca php nhn * i vi php cng +

chng ta cng k tha t trng GF(2m). Chng minh hon tt.
Nu a thc ti thiu ca phn t a GF(2m) c bc bng m th
trng Z trng vi trng GF(2m) v mi phn t ca trng c
th c biu din nh mt vect m thnh phn
(b0b1bm-1)
nh l 11.6
Gi f(x) l a thc ti thiu ca phn t a 0 ca trng

GF(2m) th f(x) l a thc ti gin trn trng GF(2).
Trang 228
Chng minh
B 11.5
Gi s f(x) = g(x) * h(x) trong g(x) v h(x) c bc ln hn 0

v nh hn bc ca f(x). Chng ta c f(a) = g(a) * h(a) = 0, suy
ra g(a) = 0 hoc h(a) = 0. iu ny mu thun vi nh ngha
v a thc ti thiu ca a.
Cho f(x) l a thc ti thiu ca phn t a 0 ca trng
GF(2m) v p(x) l a thc bt k trn trng GF(2) sao cho p(a)
= 0. Th p(x) chia ht cho f(x).
Chng minh
Chia p(x) cho f(x) chng ta c

p(x) = g(x) * f(x) + r(x)
trong bc ca r(x) nh hn bc ca f(x).
Thay x = a r(a) = 0, nn t nh ngha ca a thc ti thiu
r(x) = 0 p(x) chia ht cho f(x).
Trang 229
nh l
nh l 11.7
Chng minh
Cho f(x) l a thc ti thiu ca phn t a 0 ca trng

GF(2m) v p(x) l a thc ti gin trn trng GF(2) sao cho
p(a) = 0. Th f(x) = p(x).
Theo B 11.5 trn chng ta c p(x) chia ht cho f(x) tc l

chng ta c th vit
p(x) = g(x) * f(x)
Do p(x) l a thc ti gin nn f(x) = 1 hoc f(x) = p(x). Tuy
nhin f(x) khng th bng 1 nn suy ra f(x) = p(x).
H qu 11.2
2m 1 phn t khc 0 ca trng GF(2m) u l nghim ca

phng trnh
2 m 1
x
+1= 0
Trang 230
H qu
H qu
11.3
m
x 2 1 + 1 chia ht cho cc a thc ti thiu ca cc phn t

khc 0 ca trng GF(2m).
Chng ta s dn ra mt h qu mnh hn nh sau. Trc ht
chng ta phn tch
2 m 1
x
+1
thnh tch ca cc amthc ti gin trn trng GF(2)
2 1
x
+ 1 = p1(x) * p2(x) * * pl(x)
m
V x 2 1 + 1 c cc nghim l cc phn t ca trng GF(2m)
nn cc phn t ca trng GF(2m) s l nghim ca mt pi(x)
no v ngc li mt pi(x) bt k s c cc nghim l cc
phn t ca trng GF(2m).
Hn na nu pi(x) c bc t th s c t nghim trong trng
GF(2m).
Trang 231
H qu
H qu 11.4
2 m 1
+ 1 thnh tch cc a thc ti gin,

Trong vic trin khai x
th mi a thc ti gin s l a thc ti thiu ca mt phn t
khc 0 no ca trng GF(2m).
nh l 11.8
Cho a l mt phn t khc 0 ca trng GF(2m) v f(x) l mt

a thc trn trng GF(2). Nu f(a) = 0 th
f (a
2l
) = 0 l = 0, 1, 2, ...
H qu 11.5
Nu f(x) l a thc ti thiu ca phn t a 0 ca trng

GF(2m) th f(x) cng l a thc ti thiu ca cc phn t
2l
vi l = 0, 1, 2, ... ca trng GF(2m).
Trang 232
H qu (tt)
2l
Hay ni cch khc cc phn t a vi l = 0, 1, 2, ... l cc

nghim ca a thc ti thiu f(x) ca phn t a.
Hn na chng ta s chng minh rng ngoi chng ra f(x)
khng cn nghim no khc thuc trng GF(2m).
2l
V vy nu c bao nhiu phn t a khc nhau th f(x) c bc

by nhiu.
lm r iu ny gi k l s nguyn dng nh nht sao cho
2k
=a
S k chc chn tn ti v chng ta c
2 m 1
= 1 hay a
2m
V s k biu din chu k ca dy
=a
2 l vi l = 0, 1, 2, ...
Trang 233
B d 11.6
Cho a l mt phn t khc 0 ca trng GF(2m) v k l s

nguyn dng nh nht sao cho
2k
th k l mt c s ca m.
=a
Chng minh
2l
Chia m cho k, m = n k + r, trong r l s d v r < k
2k
2k
= a a
2k
=a
2k
Tip tc theo kiu ny chng ta c a

ta c
a=a
2m
=a
2 n k + r
=a
2 n k 2 r
= a hay a
2 nk
2 2k
=a
= a . Mt khc chng
2 n k
= a
Trang 234
2r
= (a )
2r
B (tt)
Do nh ngha ca k r = 0. Hon tt chng minh.
Phn t lin hp

2l
2k
=a
th cc phn t a vi l = 0, 1, 2, ..., k - 1 c gi l cc phn

t lin hp ca a v k c gi l s phn t lin hp ca a.
T nh ngha chng ta thy tp cc phn t lin hp ca a l
tp cc phn t khc nhau c sinh ra bi
2l
a vi l = 0, 1, 2, ...
B d 11.7
Nu a1 v a2 l cc phn t lin hp bt k ca a th a1 l phn

t lin hp ca a2 v ngc li a2 l phn t lin hp ca a1.
Trang 235
B (tt)
Tht vy gi s (vi k l s phn t lin hp ca a)
a1 = a
Th
a1
2 l 2 l1
a2
= (a
2 k + l1 l 2
2 l1
2 l1 2 l 2 l1
= (a
=a
, a2 = a
2l2
=a
,0 l1 < l 2 < k
2 l1 2 l 2 l1
2 l 2 2 k + l1 l 2
2 k + l1
= (a
=a
2 k 2 l1
=a
2l2
= a2
2 l 2 2 k + l1 l 2
=a
2 l1
= a1
Hon tt chng minh.

Ch b ny ni ln rng cc phn t lin hp ca a l lin
hp vi nhau.
Trang 236
B (tt)
2 lvi l = 0, 1, 2, ..., k - 1 l cc nghim ca a
V cc phn t a
thc ti thiu f(x) ca a, nn ta s chng minh f(x) c dng
2
f (x) = (x + a ) * ( x + a ) * K * (x + a
2 k 1
)=
k 1
(x + a
2i
i=0
chng minh iu ny chng ta s chng minh
p (x) =
k 1
(x + a
2i
i=0
l mt a thc ti gin v do p(a) = 0 nn theo nh l 11.7

chng ta suy ra f(x) = p(x).
Trang 237
B (tt)
B d 11.7

a
th
p (x) =
2k
k 1
=a
(x + a
2i
i=0
l mt a thc ti gin trn trng GF(2).
Chng minh
[ p (x)]
k 1
= (x + a
i = 0
2i
k 1
) = (x + a
i=0
Trang 238
2i 2
Chng minh (tt)

(x + a
2i 2
) = x + (a
[ p ( x)] =
2
=
=
k 1
(x
(x
i=0
k 1
i =1
k 1
(x
2i
+a
+a
2i
2 i +1
) x + (a
2i 2
) =x +a
) = (x + a
2i
2 i +1
i =1
i =1
+a
+a
2i
2i
)(x + a
2
2k
)(x + a ) =
)
k 1
(x
i=0
= p(x 2 )
Trang 239
+a
2i
Chng minh (tt)

Mt khc p(x) l mt a thc ca x v c th biu din
p(x) = b0 + b1x + + bkxk
trong cc bi vi i = 0, 1, 2, , k l cc a thc trn trng
GF(2) ca a. V vy cc bi l cc phn t ca trng GF(2m).
[ p(x)]2 = (b0 + b1x + Kbk x k ) 2
=
2 2i
b
i x + (1 + 1)
i=0
i+ j
b
b
x
=
i j
i=0 j=0
i j
2 2i
b
ix
i=0
Do [p(x)]2 = p(x2) suy ra

bi = bi2 i = 0, 1, 2, , k
iu ny ch ng nu cc bi bng phn t 0 hoc phn t 1 tc
l cc bi GF(2) hay p(x) l mt a thc trn trng GF(2).
Trang 240
Chng minh (tt)

Nu p(x) khng ti gin tc p(x) c th phn tch thnh
p(x) = q(x) * h(x)
trong bc ca q(x) v h(x) nh hn bc ca p(x) l k. Nhng
do p(a) = 0 q(a) = 0 hoc h(a) = 0.
2l
Gi s q(a) = 0, theo nh l 12.8 q(x) c cc nghim l a
vi l = 0, 1, 2, ..., k 1, q(x) c bc ti thiu l k, mu thun.
T y suy ra iu phi chng minh.
nh l 11.9

a
2k
=a
th a thc ti thiu ca a l f (x) =
k 1
(x + a
i=0
Trang 241
2i
) v c bc = k.
H qu
H qu 11.6
nh l 11.10
Bc ca mt a thc ti thiu ca mt phn t a khc 0 ca

trng GF(2m) l mt c s ca m.
2l
a
Cho a l mt phn t khc 0 ca trng GF(2m) c chu k bng
n th cc phn t lin hp ca a cng c chu k bng n.
Chng minh
Gi k l s thnh phn lin hp ca a. i = 0, 1, ..., k

n
2i
2i n
n 2i
= (a ) = 1
a = a
Chng ta chng minh rng khng s nguyn dng l < n

l
2i
a =1
Trang 242
Chng minh
2 i l
Tht vy gi s tn ti, suy ra a

=1
Chia 2i l cho n
2i l = h n + r
trong 0 r < n,
1= a
2i l
= a h n + r = (a n ) h a r = a r
T nh ngha khi nim chu k, r = 0.

T nh l 11.4 n l mt c s ca 2m 1, n l. Kt hp
vi 2i l = h n n l mt c s ca l, n l v l.
nh l 11.11
m 1 u tn ti mt a thc ti gin bc m trn trng

GF(2).
Trang 243
nh l
nh l 11.12
Vi mt a thc ti gin p(x) bt k c bc m,

p(x) = b0 + b1x + + bmxm
trong bm = 1, chng ta lun xy dng c mt trng
GF(2m) trong p(x) l a thc ti thiu ca mt phn t ca
trng.
xy dng trng
GF(2m) chng ta cho
phn t a l mt ma
trn mm nh bn
Tm m =
0
0
0
0
1
0
0
1
L
L
0
0
M
0
M
0
M
0
M
0
L
L
M
0
b0
b1 b2
b3 L bm 2
Trang 244
0
0
0
M
1
bm 1
nh l (tt)
Trn ma trn nh ngha php cng v nhn ma trn nh bnh

thng, vi ch rng vic cng hoc nhn hai phn t trong 2
ca hai ma trn c thc hin nh trn trng GF(2).
Chng ta cng nhn rng ma trn ny c a thc ti thiu l
p(x). T y chng ta c th dn ra c cc phn t cn li
ca trng GF(2m) nh nh l 11.5.
Ch , phn t 0 chnh l ma trn 0 v phn t 1 chnh l ma
trn n v.
nh l 11.13
m 2, cc a thc ti gin bc m trn trng GF(2) u l

c s ca
2 m 1
x
+1
Chng ta c th quay tr v bng lit k cc a thc ti gin
kim tra rng cc a thc ti gin bc 3 l c s ca x7 1,
cc a thc ti gin bc 4 l c s ca x15 1,
Trang 245
nh l (tt)
a thc cn bn
Mt a thc cn bn l mt a thc ti gin, ng thi khng

tn ti s nguyn dng n < 2m 1 sao cho xn + 1 chia ht cho
n.
V d, khng tn ti s nguyn dng n < 15 sao cho xn + 1
chia ht cho 1 + x + x4 nn 1 + x + x4 l a thc cn bn.
Cn a thc 1 + x + x2 + x3 + x4 l ti gin nhng khng cn
bn v x5 + 1 chia ht cho n.
1, 2, 3
4, 5
6, 7, 8
1+x
1 + x + x4 1 + x + x6
1 + x + x2 1 + x3 + x4 1 + x3 + x7
1 + x + x3 1 + x2 + x5 1 + x2 + x3 + x4 + x8
1 + x2 + x3 1 + x3 + x5
Trang 246
nh l (tt)
nh l 11.14
Cho a l mt phn t khc 0 ca trng GF(2m) c a thc ti

thiu l f(x). Nu f(x) l mt a thc cn bn trn trng GF(2)
v c bc bng m th a c chu k l 2m 1 v a l mt phn t l
c s ca GF(2m).
a2
Chng minh
Gi n l chu k ca a. t p(x) = xn + 1, th p(a) = 0.

B 11.5 p(x) chia ht cho f(x). Kt hp iu ny vi nh
ngha ca khi nim a thc cn bn, n = 2m 1.
nh l ny gi cho chng ta cch xy dng trng GF(2m)
da trn mt phn t c s c a thc ti thiu l mt a thc
cn bn bc m.
Trang 247
Tm tt
Tm tt
2 m 1
=1
a l mt phn t ca trng GF(2m) th a
Chu k ca mt phn t l mt c s ca 2m 1.
Cc a thc ti thiu ca trng GF(2m) l cc a thc ti gin
v l c s ca
m
x 2 1 + 1
Hn na bc ca chng l c ca m.
S phn t lin hp khc nhau ca a, k c a, l c s ca m.
Cc phn t lin hp ca nhau c cng a thc ti thiu, hn
na chng l cc nghim ca a thc ti thiu ny v bc ca
a thc ti thiu ny bng s cc phn t lin hp khc nhau.
Cc phn t lin hp th c cng chu k. k
2 1
+1
Cc a thc ti gin bc k l c ca x
Trang 248
Tm tt (tt)
Mt phn t a c a thc ti thiu bc m th cc t hp tuyn

tnh (vi bi GF(2))
b01 + b1a + + bk - 1ak - 1
s sinh ra ton b cc phn t ca trng GF(2m)
Mt phn t a c a thc ti thiu bc m v cng l a thc
cn bn th cc ly tha ca n s sinh ra ton b cc phn t
ca trng GF(2m).
Trang 249
V d
Xy dng trng GF(2m) vi m = 4.

Chng ta k hiu 0 l ma trn 0, k hiu 1 l ma trn n v (c
kch thc l 44). Ly phn t a l ma trn sau
0 1 0 0
0 0 1 0
T4 4 =
0 0 0 1
0
0
1
1
Chng ta c a thc ti thiu ca a l f(x) = 1 + x + x4

y l mt a thc cn bn bc 4. V vy theo nh l 11.14,
15 phn t ca GF(24) khng tnh phn t 0 s c dng ai, i = 0,
1, , 14 vi ch a0 = 1.
Cn theo nh l 12.12 chng s c dng b0 + b1a + b2a2 + b3a3
trong cc bi = 0 hoc 1.
Trang 250
V d (tt)
Vy c hai cch xy dng trng GF(24) nh trn.

Cc bng sau y biu din cc phn t khc 0 v khc 1 ca
trng GF(24) theo cc dng: ly tha ca a (ai), a thc ca a,
vect, dng ma trn.
a
a
0100
0
0
1 0 0
0 1 0
0 0 1
1 0 0
a2
a2
0010
0
0
0 1 0
0 0 1
1 0 0
1 1 0
a3
a3
0001
0
1
0 0 1
1 0 0
1 1 0
0 1 1
a5
a + a2
0110
a4
1+a
1100
1
0
1 0 0
1 1 0
0 1 1
1 0 1
Trang 251
0
0
1 1 0
0 1 1
1 0 1
0 1 0
V d (tt)
a6
a2 + a3
0011
0
1
0
1
0
1
1
0
1
0
a7
1 + a + a3
1101
1
1
0
1
1
1
0
1
1
0
1
0
1
1
0
1
a8
1 + a2
1010
1
0
1
1
1
0
1
1
1
1
a9
a + a3
0101
0
1
1
1
0
0
1
1
1
Trang 252
1
0
1
1
a10
1 + a + a2
1110
0
1
0
0
1
1
1
1
1
0
1
1
1
1
0
1
V d (tt)
a11
a + a2 + a3
0111
0 1 1 1
1 1 1 1
1 0 1 1
1 0 0 1
a12
1 + a + a2 + a3
1111
1 1 1 1
1 0 1 1
1 0 0 1
1 0 0 0
a13
1 + a2 + a3
1011
1 0 1 1
1 0 0 1
1 0 0 0
0 1 0 0
1
1
a14
1 + a3
1001
0 0 1
0 0 0
1 0 0
0 1 0
Chu k, a thc ti thiu ca cc phn t lin hp
a, a2, a4, a8
a3, a6, a12, a9
a5, a10 a7, a14, a13, a11
15
5
3
15
x 1 + x 1 + x + x4 1 + x + x2 + x3 + x4 1 + x + x2 1 + x3 + x4
Trang 253
Bi 12 M khi tuyn tnh

12.1 Gii thiu
12.2 Cc khi nim v nguyn l hot ng
12.3 Vn pht hin sai v sa sai
12.4 Mt s gii hn
Trang 254
Gii thiu
M khi tuyn tnh c xy dng da trn cc kt qu ca i

s tuyn tnh l mt lp m c dng rt ph bin trong vic
chng nhiu.
nh ngha
Mt m khi c chiu di n gm 2k t m c gi l m tuyn

tnh C(n, k) nu v ch nu 2k t m hnh thnh mt khng gian
vect con k chiu ca khng gian vect n chiu gm tt c cc
vect n thnh phn trn trng GF(2).
M tuyn tnh C(n, k) c mc ch m ho nhng khi tin (hay
thng bo) k bit thnh nhng t m n bit. Hay ni cch khc
trong n bit ca t m c cha k bit thng tin.
Qui c vit du + thay cho du v du + s c hiu theo
ng cnh.
Trang 255
Cch biu din m Ma trn sinh

M tuyn tnh C(n, k) l mt khng gian con k chiu ca khng
gian vect n thnh phn, k t m c lp tuyn tnh, chng
hn (g0, g1, ..., gk1) sao cho mi t m trong C l mt t hp
tuyn tnh ca k t m ny (vi ai {0, 1} i = 0, 1, ..., k1)
w = a0g0 + a1g1 + ... + ak1gk1
k t m ny to thnh mt ma trn cp k n nh sau
g 01
g 0( n 1)
L
g 0 g 00
g g
g
g
L
10
11
1( n 1)
1
Gk n =
=
M
M
M M
g
g
g
L
( k 1)1
( k 1)( n 1)
g k 1 ( k 1)0
Vi gi = (gi0, gi1, , gi(n1)), vi i = 0, 1, , k1.

Trang 256
Cch m ha
Nu u = (a0, a1, , ak1) l thng tin cn c m ho th t m

w tng ng vi u c ta bng cch ly u nhn vi G
w = u G = (a0, a1, , ak1)
hay
w = a0g0 + a1g1 + + ak1gk1
V cc t m tng ng vi cc thng bo c sinh ra bi G
theo cch trn nn G c gi l ma trn sinh ca b m.
Trang 257
V d
Cho ma trn sinh ca mt m tuyn tnh C(7, 4) sau

g 0 1
g 1
1
G4 7 = =
g 2 0

g 3 1
1 0 1 0 0 0
0 1 1 1 0 0
1 0 0 0 1 1
0 1 0 0 0 1
Nu u = (1101) l thng tin cn m ho th t m tng ng l

w = 1.g0 + 1.g1 + 0.g2 + 1.g3 = (1100101)
Bt k k t m c lp tuyn tnh no cng c th c dng
lm ma trn sinh cho b m.
Mt b m tuyn tnh (hay cn gi l khng gian m) c th c
nhiu ma trn sinh khc nhau cng biu din.
Mi ma trn sinh tng ng vi mt cch m ha khc nhau.
Trang 258
Cch gii m
Ly ma trn sinh nh trong v d trn.
u = (a0, a1, a2, a3) l thng bo, w = (b0, b1, b2, b3, b4, b5, b6) l
t m tng ng.
Chng ta c h phng trnh sau lin h gia u v w.
w=uG
b0 = a0 + a1 + a3
(1)
b1 = a0 + a2
(2)
g 0 1 1 0 1 0 0 0
b2 = a1 + a3
(3)
g 1 0 1 1 1 0 0
1
b3 = a0 + a1
(4)
G4 7 = =
g 2 0 1 0 0 0 1 1
b4 = a1
(5)

g 3 1 0 1 0 0 0 1
b5 = a2
(6)
b6 = a2 + a3
(7)
Trang 259
Cch gii m (tt)

Chn bn phng trnh n gin nht gii cc ai theo cc bj.
Chng hn cc phng trnh (4), (5), (6), (7) chng ta gii c
a0 = b3 + b4
g 0 1 1 0 1 0 0 0
g 1 0 1 1 1 0 0
a1 = b4
1
G4 7 =
=
g 2 0 1 0 0 0 1 1
a2 = b5

g
1
0
1
0
0
0
1
a3 = b5 + b6
3
H phng trnh trn c gi l h phng trnh gii m.
C th c nhiu h phng trnh gii m khc nhau nhng s
cho kt qu nh nhau.
w = 1001011
u=?
w = 0101110
u=?
Trang 260
M tuyn tnh h thng
Mt m tuyn tnh C(n, k) c gi l m tuyn tnh h thng

nu mi t m c mt trong hai dng sau
Dng 1: T m bao gm phn thng tin k bit i trc v phn
cn li (gm n k bit) i sau (phn ny cn c gi l phn d
tha hay phn kim tra).
k bit thng tin n k bit kim tra
Dng 2: Ngc ca dng 1, t m bao gm phn kim tra i

trc v phn thng tin i sau.
n k bit kim tra k bit thng tin
Trang 261
Ma trn sinh h thng
1 0 L 0 P
L
P
P
00
01
0(n k 1)
P11 L P1(n k 1)
0 1 L 0 P10
Gk n = I kk | Pk (n k ) =
M
M
M
M
M M
0 0 L 1P(k 1)0 P(k 1)1 L P(k 1)(n k 1)

4 43
4 1444444
424444444
3
142
k k
k (n k )
V d
Ght ( 47 )
1
0
=
0
0 0 0 1 1 0
M ha
1 0 0 0 1 1 u = 1101 w = u Ght = 1101000
Gii m
0 1 0 1 1 1
w = 0110100 u = 0110
0 0 1 1 0 1
Trang 262
V d
Dng cc php bin i s cp bin i cc ma trn sinh sau

thnh ma trn sinh h thng.
G47
1
0
=
0
1 0 1 0 0 1
1 1 0 1 0 1
0 1 1 0 0 1
0 0 1 0 1 0
G 4 7
1
1
=
1
1 0 1 0 0 1
0 1 0 1 0 1
0 0 1 1 0 0
0 0 1 0 1 1
Khng phi mi ma trn sinh u c th bin i thnh ma trn

sinh h thng.
Trang 263
V d (tt)
Hy thc hin php m ha v gii m trn ma trn sinh sau.
1
1
G4 7 =
0
1
u = a1 a2 a3 a4
M ha
Gii m
1 0 0 1 0 0
0 1 0 1 0 1
0 0 1 1 0 1
0 0 0 0 1 1
th w = b1 b2 b3 b4 b5 b6 b7
u = (1101) w = (1110010)
w = (1011000) u = (0110)
Trang 264
Pht hin sai v sa sai
Nguyn l pht hin sai: Kim tra xem t hp nhn c phi l

t m hay khng, nu khng th t hp nhn l sai.
Nguyn l sa sai: Kim tra xem t hp nhn c khong cch
Hamming gn vi t m no nht, th chnh l t m ng
c pht i.
Nguyn l ny c gi l nguyn l khong cch Hamming
ti thiu.
Khng gian b trc giao
Cho S l mt khng gian con k chiu ca khng gian V n chiu.

Gi Sd l tp tt c cc vect v trong V sao cho u S, u v =
0 (php nhn v hng ca hai vect). Sd c chng minh l
mt khng gian con ca V v c s chiu l n k. Sd c gi l
khng gian b trc giao ca S v ngc li.
Trang 265
Cch pht hin sai
H qu
Mi ma trn G bt k kch thc k n vi k hng c lp tuyn

tnh lun tn ti ma trn H kch thc (n k) n vi (n k)
hng c lp tuyn tnh sao cho G HT = 0, trong HT l ma
trn chuyn v ca ma trn H.
Ni cch khc cc vect hng ca H u trc giao vi cc vect
hng ca G.
Cch pht hin sai
Nu v l mt t m c sinh ra t ma trn sinh G c ma trn

trc giao tng ng l H th
v HT = 0
Ngc li nu
v HT = 0
th v l mt t m.
Trang 266
Ma trn kim tra
Ma trn kim tra
Syndrome vect sa sai (corrector)
Ma trn kim tra ca mt b m c ma trn sinh Gkn l ma trn

H c kch thc (n k) n sao cho
G HT = 0
v HT c gi l syndrome hay vect sa sai ca v v c k
hiu l s(v). v l t m khi v ch khi s(v) = 0.
V d
Tm ma trn kim tra ng vi ma trn sinh sau.

1 1 0 1 0 0 0
1 0 1 1 1 0 0
G 4 7 =
0 1 0 0 0 1 1
1
0
0
0
1
0
1
Trang 267
Ma trn kim tra (tt)
H c kch thc 3 7.
Gi h = (a0, a1, a2, a3, a4, a5, a6) l mt hng bt k ca H. h
trc giao vi mi hng ca G nn chng ta c h bn phng
trnh sau
1 1 0 1 0 0 0
a0 + a1 + a3
=0
1 0 1 1 1 0 0
a0 + a2 + a3 + a4 = 0
G4 7 =
0 1 0 0 0 1 1
a1 + a5 + a6
=0
a0 + a2 + a6
=0
1 0 1 0 0 0 1
Vn l tm c 3 vect h c lp tuyn tnh l nghim ca
h phng trnh trn.
Ch , h phng trnh trn c th cho php chng ta gii bn
bin theo ba bin cn li. Chng hn chng ta gii a3, a4, a5, a6
theo a0, a1, a2 nh sau.
Trang 268
Ma trn kim tra (tt)
a3 = a0 + a1
a4 = a1 + a2
a5 = a0 + a1 + a2
a6 = a0 + a2
Cho (a0, a1, a2) ln lt cc gi tr (1, 0, 0), (0, 1, 0), (0, 0, 1)
(c lp tuyn tnh vi nhau), ta xc nh c (a3, a4, a5, a6)
ln lt nh sau (1, 0, 1, 1), (1, 1, 1, 0), (0, 1, 1, 1).
Vy H l
Ch
1 0 0 1 0 1 1
H 3 7 = 0 1 0 1 1 1 0
0 0 1 0 1 1 1
C th tn ti nhiu ma trn kim tra khc nhau ca cng mt

b m v chng u c kh nng kim tra nh nhau.
Trang 269
Ma trn kim tra (tt)
B 12.1
Nu ma trn sinh h thng ca mt m tuyn tnh h thng c

dng
Gkn = [Ikk | Pk(nk)]
th
H(nk)n = [Pk(nk)T | I(nk)(nk)]
l mt ma trn kim tra ca m.
Tng t nu ma trn sinh c dng
Gkn = [Pk(nk) | Ikk]
th ma trn kim tra c dng
H(nk)n = [I(nk)(nk) | Pk(nk)T]
trong I(nk)(nk) l ma trn n v kch thc (nk)(nk), cn
Pk(nk)T l ma trn chuyn v ca ma trn Pk(nk).
Trang 270
Chng minh
1 0 L 0 P
P
P
L
00
01
0( n k 1)
P11
P1( n k 1)
L
0 1 L 0 P10
Gk n =
M
M
M M
M M
0 0 L 1 P( k 1)0 P( k 1)1 L P( k 1)( n k 1)

4 43
4 1444444
424444444
3
142
k k
k (n k )
P
P10
P(k 1)0 1 0 L 0
L
00
P11
P(k 1)1 0 1 L 0
L
P01
H ( n k ) n =
M
M
M
M
M
M
P0(n k 1) P1(n k 1) L P(k 1)(n k 1) 0 0 L 1

4 43
4
3142
144444444244444444
(n k )(n k )
(n k ) k Trang 271
Chng minh (tt)
Ta chng minh
G HT = 0
Chng minh iu ny vic chng minh
gi hj = 0 i = 0, , k1, j = 0, , nk1
trong
gi = (gi0, , gi(n1)) l hng i ca G cn
hj = (hj0, , hj(n1)) l hng j ca ma trn H.
Tht vy
gi h j =
n 1
k 1
n k 1
s=0
s=0
s=k
g is h js = g is h js + g is h js
= h ji + g i ( k + j ) = Pij + Pij = 0
Trang 272
V d
Tm ma trn H cho cc ma trn sinh sau
Ght ( 47 )
1
0
=
0
0 0 0 1 1 0
1 0 0 0 1 1
0 1 0 1 1 1
0 0 1 1 0 1
G47
1
1
=
0
0 0 0 1 1 0
1 0 0 1 0 1
0 0 1 1 0 1
0 1 0 0 0 1
G47
1
1
=
0
1 0 0 1 0 0
0 1 0 1 0 1
0 0 1 1 0 1
0 0 0 0 1 1
Trang 273
Kh nng chng nhiu tng ng
B 12.2
Hai m tuyn tnh C(n, k) c gi l c kh nng chng nhiu

tng ng nu chng c cng khong cch Hamming.
Nu hon v hai ct ca mt ma trn sinh s to ra mt b m
mi c kh nng chng nhiu tng ng vi b m c. Ni
cch khc vic hon v hai ct ca ma trn sinh khng lm thay
i kh nng chng nhiu.
B 12.3
Khong cch Hamming ca mt m tuyn tnh bng trng s

nh nht khc 0 ca b m.
Trang 274
B 12.4
H qu
Gi H l ma trn kim tra ca mt m tuyn tnh, nu mt t

m c trng s d th tn ti d ct ca H c tng bng 0.
Nu trong ma trn kim tra H ca mt m tuyn tnh s ct ph
thuc tuyn tnh nh nht l d th khong cch Hamming ca b
m bng d.
V d 12.5
1 0 0 1 0 1 1
H 3 7 = 0 1 0 1 1 1 0
0 0 1 0 1 1 1
Trang 275
d = 3 (3, 4, 6)
Cch sa sai
Vect li
L vect biu din cc v tr li gia t m truyn v t hp

nhn, mi v tr li c biu din bng bit 1, cn li l 0.
Nu t m truyn l w, vect li l e v vect nhn l v th
v=w+e
e=v+w
w=e+v
V d
w = 1011011, e = 0010100 v = w + e = 1001111.

w = 0110010, v = 0010011 e = w + v = 0100001.
v = 1011001, e = 0010010 w = v + e = 1001011.
Trang 276
Tp gii m - coset
Cho S l mt khng gian con cc t m ca khng gian V, coset

ca mt phn t z V i vi S c k hiu l z + S v c
nh ngha nh sau
z + S = {z + w: w S}
B 12.5
Tp coset z + S c cc tnh cht sau.
(1) z z + S.
(2) Nu z S th z + S = S.
(3) Nu v z + S th v + S = z + S.
(4) Nu v z + S th v + S v z + S ri nhau.
Trang 277
S gii m
Vi mi vect nhn v chng ta s c mt tp coset tng ng l

v + S.
Trong tp ny chn phn t c trng s nh nht, chng hn l
z. Phn t ny thng c gi l coset leader.
Thng bo t m c truyn chnh l w = v + z.
B 12.6
Cc phn t ca mt tp coset c cng mt syndrome nh nhau.

Cc tp coset khc nhau c cc syndrome khc nhau.
e = (a1, a2, ..., an), cc ct ca H ln lt bng h1, h2, ..., hn th
n
s (e) = e H = ai hi = ai hi
T
i =1
ai 0
Trang 278
S gii m (tt)
Ngha l s(e) bng tng nhng ct nhng v tr tng ng vi

nhng v tr bng 1 ca e.
Nu v tr li sai l 3 th syndrome ca vect nhn s l ct s 3
ca H.
1
1
G4 7 =
0
1
0
1
0
0
1
0
1
1
1
0
0
0
1
0
0
0
0
1
0
0
0
1
1 0 0 1 0 1 1
H 3 7 = 0 1 0 1 1 1 0
0 0 1 0 1 1 1
Tm v tr li sai ca cc vect nhn sau y

v = 0010011 s(v) = ?
e=?
w=?
v = 0101101 s(v) = ?
e=?
w=?
Trang 279
M tuyn tnh Hamming
M tuyn tnh Hamming l m c ma trn H c tnh cht gi tr

ca ct hi bng i (i = 1, 2, ...)
0 0 0 1 1 1 1
H 3 7 = 0 1 1 0 0 1 1
1 0 1 0 1 0 1
B 12.7
Cc m tuyn tnh Hamming u c khong cch Hamming d =

3. V vy c th pht hin sai 2 bit v sa sai 1 bit.
Trang 280
Ma trn sinh ca m tuyn tnh Hamming
Xt m tuyn tnh Hamming C(7, 4) c cc bit thng tin nm

cc v tr 3, 5, 6, 7. Hy xc nh ma trn sinh G ca b m.
Gi w = (a1, a2, a3, a4, a5, a6, a7) l mt t m. Chng ta c h
phng trnh sau c dn ra t cng thc w HT = 0.
a4 + a5 + a6 + a7 = 0
a2 + a3 + a6 + a7 = 0
a1 + a3 + a5 + a7 = 0
T y suy ra cng thc tnh cc bit kim tra a1, a2, a4 theo cc
bit thng bo a3, a5, a6, a7 nh sau
a1 = a3 + a5 + a7
a2 = a3 + a6 + a7
a4 = a5 + a6 + a7
Trang 281
Ma trn sinh ca m tuyn tnh Hamming

b1 b2 b3 b4
u= 1 0 1 0
V d
th w =
1
1
G4 7 =
0
a1 a2 a3 a4 a5 a6 a7
1 0 1 1 0 1 0
1 1 0 0 0 0
0 0 1 1 0 0
1 0 1 0 1 0
1 0 1 0 0 1
Xt m tuyn tnh Hamming C(7, 4) c cc bit thng tin nm

cc v tr 1, 2, 3, 4. Hy xc nh ma trn sinh G ca b m.
Trang 282
Bi 13 M vng
13.1 Gii thiu
13.2 Cc tnh cht ca m vng
13.3 Ma trn sinh v ma trn kim tra ca m
13.4 M BCH
Trang 283
Gii thiu
nh ngha
a thc m
Mt m tuyn tnh C(n, k) c gi l m vng nu w =

a0a1an2an1 l mt t m th v = an1a0a1an2 cng l mt
t m.
Ngha l dch vng (sang tri hay phi) mt t m th kt qu
cng l mt t m. y qui c dch phi.
Nu w = a0a1an2an1 l mt t m th w(x) = a0 + a1x + +
an2xn - 2 + an1xn - 1 l a thc m tng ng vi t m w.
V d
Bng sau y trnh by mt m vng C(7, 4).

Trang 284
V d
m
w(x)
0000
1000
0100
1100
0010
1010
0000000
1101000
0110100
1011100
0011010
1110010
0
1 + x + x3
x + x2 + x4
1 + x2 + x3 + x4
x2 + x3 + x5
1 + x + x2 + x5
0001
1001
0101
1101
0011
1011
0110 0101110 x + x3 + x4 + x5
1110 1000110 1 + x4 + x5
w(x)
x3 + x4 + x6
1 + x + x4 + x6
x + x2 + x3 + x6
1 + x2 + x6
x2 + x4 + x5 + x6
1 + x + x2 + x3 +
x4 + x5 + x6
0111 0100011 x + x5 + x6
1111 1001011 1 + x3 + x5 + x6
0001101
1100101
0111001
1010001
0010111
1111111
Trang 285
Gii thiu (tt)
w(i), w(i)(x)
w(i) l t m do dch t m w i bit, v w(i)(x) l a thc m

tng ng ca w(i). w(0) chnh l w.
i
w(i)
w(i)(x)
0 1101000 1 + x + x3
1 0110100 x + x2 + x4 = x * (1 + x + x3) = x * w(x)
2 0011010 x2 + x3 + x5 = x2 (1 + x + x3) = x2 * w(x)
3 0001101 x3 + x4 + x6 = x3 (1 + x + x3) = x3 * w(x)
4 1000110 1 + x4 + x5 = x4 + x5 + x7 mod 7
5 0100011 x + x5 + x6 = x5 + x6 + x8 mod 7
6 1010001 1 + x2 + x6 = x6 + x7 mod 7 + x9 mod 7
Trang 286
Gii thiu (tt)
w(i)(x) = xi * w(x) tuy nhin nu w(i)(x) c xp vi p n th xp

c thay bng xp mod n.
Mc khc trn trng GF(2) chng ta c
xn + j = xj * (xn + 1) + xj hay xn + j mod (xn + 1) = xj
B 13.1
w(i)(x) = [xi * w(x)] mod (xn + 1)
Trang 287
Cc tnh cht ca m vng
nh l 13.1
a thc m khc 0 c bc nh nht l duy nht. Hay ni cch

khc khng tn ti hai a thc m khc 0, khc nhau v cng c
bc nh nht.
Chng minh
Gi s hai a thc m khc nhau, cng c bc nh nht l r, 0

< r < n.
g(x) = g0 + g1x + + gr1xr - 1 + xr
f(x) = f0 + f1x + + fr1xr - 1 + xr
T y suy ra a thc m g(x) + f(x) c bc nh hn r, mu
thun. Chng minh hon tt.
Trang 288
Cc tnh cht ca m vng (tt)
nh l 13.2
K hiu a thc m c bc nh nht l g(x)

g(x) = g0 + g1x + + gr1xr - 1 + xr
H s t do g0 ca g(x) phi bng 1.
Chng minh
Gi s g0 = 0. Suy ra
g(x) = x * (g1 + + gr1xr - 2 + xr - 1)
t f(x) = (g1 + + gr1xr - 2 + xr - 1), suy ra f(x) cng l mt a
thc m. V f(x) tng ng vi t m c dch tri 1 bit hay
dch phi (n 1) bit t t m ng vi g(x).
M bc ca f(x) bng r 1 < r mu thun vi nh ngha ca
g(x).
Trang 289
nh l 13.3
Mt a thc v(x) trn trng GF(2) c bc n 1 l a thc

m nu v ch nu n l mt bi s ca g(x). Tc l n c th
vit v(x) = q(x) * g(x).
Chng minh
Chiu thun
Nu v(x) = q(x) * g(x) v c bc n 1 th v(x) l a thc m.
p
p
v ( x ) = q ( x ) * g ( x ) = qi x i * g ( x ) = qi x i * g ( x )
i
i=0
=
0
vi p l bc ca q(x) v p + r n 1. Do xi * g(x) vi 0 i p
l a thc m, nn v(x) l a thc m v n l mt t hp tuyn
tnh ca cc a thc m. Trang 290
Chiu ngc
Nu v(x) l a thc m th chia v(x) cho g(x)
v(x) = q(x) * g(x) + r(x)
trong r(x) l a thc d v c bc nh hn bc ca g(x).
i vi cc a thc trn trng GF(2) chng ta c th suy ra
r(x) = q(x) * g(x) + v(x)
Nn r(x) l mt a thc m. Theo nh ngha ca g(x) suy ra
r(x) = 0. Chng minh hon tt.
T nh l ny chng ta gi g(x) l a thc sinh, v t g(x) c
th sinh ra tt c cc a thc m khc.
Trang 291
nh l 13.4
a thc sinh ca mt m vng C(n, k) c bc r = n k.
Chng minh
Mi a thc m w(x) l mt bi s ca g(x)

w(x) = q(x) * g(x)
C 2k t m nn c 2k a thc q(x). Suy ra bc ca q(x) l k
1. Suy ra bc ca g(x) l n k.
T nh l ny a thc sinh g(x) c th c biu din nh sau
g(x) = g0 + g1x + + gn kxn k
trong g0 = gn k = 1.
Trang 292
nh l 13.5
a thc sinh ca m vng C(n, k) l mt c s ca xn + 1.
Chng minh
B 13.1 suy ra
g(i)(x) = [xi * g(x)] mod (xn + 1)
xi * g(x) = q(x) * (xn + 1) + g(i)(x)
Chn i = k q(x) = 1 tc
xk * g(x) = (xn + 1) + g(i)(x)
xn + 1 = xk * g(x) + g(i)(x)
Do g(i)(x) l mt a thc m nn g(i)(x) l mt bi ca g(x),
xn + 1 l mt bi ca g(x). Chng minh hon tt.
Trang 293
nh l 13.6
Nu g(x) l mt a thc c bc (n k) v l c s ca (xn + 1)

th g(x) sinh ra m vng C(n, k), hay ni cch khc g(x) l a
thc sinh ca mt m vng C(n, k) no .
Chng minh
Xt k a thc g(x), x * g(x), , xk 1 * g(x).

Cc a thc ny u c bc n 1.
Gi v(x) l mt t hp tuyn tnh ca k a thc ny vi cc h
s ai GF(2).
v(x) = a0g(x) + a1x * g(x) + + ak 1xk 1 * g(x)
v(x) l mt a thc c bc n 1 v l bi s ca g(x).
Trang 294
C tt c 2k t hp tuyn tnh v(x) khc nhau v to nn mt

khng gian tuyn tnh ca cc a thc m vi g(x), x * g(x), ,
xk 1 * g(x) l cc a thc lm c s.
Chng ta chng minh rng b m tng ng vi khng gian
ny l m vng.
Gi
w(x) = b0 + b1x + + bn 1xn 1
l mt a thc ca khng gian.
Chng ta chng minh
w(1)(x) = bn 1 + b0x + b1x2 + + bn 2xn 1
cng l mt a thc ca khng gian.
Trang 295
Theo B 13.1 chng ta c

w(1)(x) = [x * w(x)] mod (xn + 1)
Da vo biu din ca v(x) v w(1)(x) chng ta suy ra
x * w(x) = bn 1(xn + 1) + w(1)(x)
Do v(x) v (xn + 1) u l bi ca g(x) nn w(1)(x) cng l bi
ca g(x). Suy ra w(1)(x) cng l a thc m. Hon tt chng
minh.
Trang 296
Ma trn sinh
G k n
1 444
k7
k4
7
+ 14448 64444
4
6444n4
8
g
0
0 L
0
g1 g 2 L g n k
0
0 L
0
0 g 0 g1 L g n k 1 g n k
= 0
0
0 g 0 L g n k 2 g n k 1 g n k L
M
M
M
M
M
M
M
M
M
0 L 0
g1
g 2 L g nk
g0
0
V d
Tm mt m vng C(7, 4).

Theo cc tnh cht ca m vng suy ra a thc sinh ca m c
bc bng 3 v l mt c s ca x7 + 1. Phn tch a thc ny ra
tha s chng ta c Trang 297
V d
x7 + 1 = (1 + x)(1 + x + x3)(1 + x2 + x3)

Chn chng hn
g(x) = (1 + x + x3)
G47
1
0
=
0
1 0 1 0 0 0
1 1 0 1 0 0
0 1 1 0 1 0
0 0 1 1 0 1
Trang 298
M vng dng h thng
G47
T dng h thng loi 1 chng ta c th dch vng k bit bin

i sang dng h thng loi 2 v ngc li.
1
0
=
0
1 0 1 0 0 0
1 1 0 1 0 0
0 1 1 0 1 0
0 0 1 1 0 1
Ght ( 47 )
1
0
=
0
0
1
0
0
0
0
1
0
0
0
0
1
1
0
1
1
M ha thnh t m h thng
u(x) l thng bo, w(x) l t m h thng loi 2 tng ng.

xnk * u(x) = q(x) * g(x) + a(x)
w(x) = xnk * u(x) + a(x) = q(x) * g(x)
Trang 299
1
1
1
0
0
1
1
V d
Cho m vng C(7, 4) c ma trn sinh l g(x) = (1 + x + x3). Hy

m ho thng bo u = 1010 thnh t m h thng dng 2.
u(x) = 1 + x2.
Nhn u(x) vi xnk = x3 ri chia cho g(x) chng ta c.
x3 * (1 + x2) = x3 + x5 = x2 * (1 + x + x3) + x2
T y suy ra
w(x) = x2 + x3 + x5
w = 0011010
l t m h thng dng 2 tng ng vi u.
Trang 300
Ma trn kim tra ca m vng
C mt cch khc tm ma trn kim tra ca m vng

xn + 1 = g(x) * h(x)
h(x) c gi l a thc i ngu ca g(x). h(x) c bc k
h(x) = h0 + h1x + + hkxk
Ma trn sau l mt ma trn kim tra ca m vng
H ( n k )n
1 444
k7
n4
k4
+4
7
144
64444
8
8 644
h h
L
L
0
0
0
h
h
k 1
k 2
0
k
0 L 0
0 hk hk 1 L h1 h0
= 0
0
h0 L 0
hk L h2 h1
M
M
M
M M
M
M
M
M
L 0 hk hk 1 hk 2 L h0
0
0
Trang 301
V d
Cho m vng C(7, 4) c ma trn sinh l g(x) = (1 + x + x3).

T y suy ra
h(x) = (1 + x + x2 + x4)
Ma trn kim tra ca b m l
H 37
1 0 1 1 1 0 0
= 0 1 0 1 1 1 0
0 0 1 0 1 1 1
Trang 302
ng dng trng GF(2m)

xy dng m vng
nh l 13.7
Cho a l mt phn t khc 0 ca trng GF(2m) c chu k l n,

a thc ti thiu f(x) ca a c bc l m. Th m c ma trn sau
lm ma trn kim tra l mt m vng C(n, n m), trong mi
phn t trong ma trn bn di c thay th bng vect m
thnh phn tng ng ca n.
Hmn = [1 a a2 an 2 an1]
Hn na m vng ny c a thc sinh chnh l f(x).
V d
Xt trng GF(24) v a c a thc ti thiu l

f(x) = 1 + x + x4
Trang 303
ng dng trng GF(2m)

xy dng m vng (tt)
T y suy ra ma trn kim tra ca m vng (15, 11).
H 415
1
0
=
0
0
1
0
0
0
0
1
0
0
0
0
1
1
1
0
0
0
1
1
0
0
0
1
1
1
1
0
1
1
0
1
0
0
1
0
1
1
1
1
0
0
1
1
1
1
1
1
1
1
0
1
1
1
0
0
Nu a thc ti thiu ca a l f(x) = 1 + x + x2 + x3 + x4 th a c

chu k l 5 v cc phn t 1, a, ..., a4 c biu din nh sau.
a3 = (0001)
1 = (1000)
a = (0100)
a4 = (1111)
a2 = (0010)
Trang 304
ng dng trng GF(2m)

xy dng m vng (tt)
T y suy ra ma trn kim tra ca m vng (5, 1)
H 45
1
0
=
0
0 0 0 1
1 0 0 1
0 1 0 1
0 0 1 1
Trang 305
M BCH nh phn
Do Bose, Chaudhuri v Hocquenghem sng lp ra.

L m vng c kh nng sa c nhiu li.
i vi cc s nguyn dng m v t bt k chng ta s xy
dng mt m BCH nh phn c cc thng s sau:
n = 2m 1
di t m:
n k mt
S bit kim tra:
Khong cch Hamming: dmin 2t + 1
Trang 306
nh l
nh l 13.8
Cho a l mt phn t ca trng GF(2m) c a thc ti thiu l

mt a thc cn bn bc m. Th m c ma trn sau lm ma trn
kim tra l mt m vng c khong cch Hamming 2t + 1,
trong mi phn t trong ma trn bn di c thay th
bng vect m thnh phn tng ng ca n.
1
a
3
1
a
H = 1 a 5
M
M
1 a 2t 1
a2
a6
a 10
M
a 2 ( 2t 1)
L
a n2
L
a 3( n 2 )
L
a 5( n 2 )
M
M
L a ( 2t 1)(( n 2)
Trang 307
a ( 2t 1)(( n 1)
a n 1
a 3( n 1)
a 5( n 1)
M
nh l (tt)
Hn na a thc sinh g(x) ca b m l a thc bi s chung

nh nht ca cc a thc ti thiu ca cc phn t a, a3, a5, ,
a2t1.
B 13.2
Ma trn A sau c nh thc bng ( y i y j )

i> j
vi i, j {1, 2, , r}. nh thc ny c gi l nh thc
1
1
L
Vandermonde.
1
y
12
A = y1
M
y1 r 1
y2
y2
M
y2
r 1
y r
2
L yr
M
M
r 1
L y r
L
Trang 308
V d
Cho m = 4, t = 2 chng ta s xy dng mt m vng c chiu

di t m l 24 1 = 15 v c khong cch Hamming d 5.
Vic xy dng s da vo trng GF(24).
Gi a l phn t c a thc ti thiu l a thc cn bn bc 4
sau
f1(x) = 1 + x + x4
y chnh l trng GF(24) trong v d slide 250.
a c chu k n = 2m 1 = 15. Chng ta c ma trn kim tra ca
b m nh sau.
1 a a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a12 a14

H = 3 6 9 12 15 18 21 24 27 30 33 36 39 42
1 a a a a a a a a a a a a a a
Thay mi phn t ai bng vect 4 thnh phn tng ng

Trang 309
V d (tt)
1
0
H=
1
0
0
0 0 0 1 0 0 1 1 0 1 0 1 1 1
1 0 0 1 1 0 1 0 1 1 1 1 0 0
0 1 0 0 1 1 0 1 0 1 1 1 1 0
0 0 1 0 0 1 1 0 1 0 1 1 1 1
0 0 0 1 1 0 0 0 1 1 0 0 0 1
0 0 1 1 0 0 0 1 1 0 0 0 1 1
0 1 0 1 0 0 1 0 1 0 0 1 0 1
1 1 1 1 0 1 1 1 1 0 1 1 1 1
Trang 310
V d (tt)
a thc sinh g(x) l bi s ca hai a thc ti thiu tng ng

vi phn t a v a3.
Theo v d slide 250, a thc ti thiu ca a3 l
f3(x) = 1 + x + x2 + x3 + x4.
T y suy ra
g(x) = f1(x) * f3(x)
= (1 + x + x4) * (1 + x + x2 + x3 + x4)
= 1 + x4 + x6 + x7 + x8
Ch
Trong trng hp a thc ti thiu ca a khng phi l a thc

cn bn, chng ta s tm c m vng c chiu di n 2m + 1,
vi n l chu k ca a. Trang 311

LTTT Slide v1 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

LTTT Slide v1 PDF

Uploaded by

Copyright:

Available Formats

Trng i hc Bch khoa

Khoa Cng Ngh Thng Tin

M ha chng nhiu, nh l knh

TI LIU THAM KHO

S c thng bo c th cho tng kha hc. Tuy nhin,

Lm bi tp ln cng im (khng bt buc)

Hai ngi ni chuyn vi nhau. Ci m trao i gia h gi l

Thng tin l g? (tt)

Thng tin l ci c truyn t i tng ny n i tng

Vai tr ca thng tin

Cc i tng sng lun lun c nhu cu hiu v th gii xung

LTTT nghin cu nhng vn g?

gc khoa hc k thut, LTTT nghin cu nhm to ra mt

Nhng ng dng ca LT thng tin

Nhng ng dng ca LT thng tin (tt)

Mi quan h gia LTTT v thng k c tm thy, cc

Lch s hnh thnh

Cuc cch mng ln nht v cch nhn th gii khoa hc l

Lch s hnh thnh (tt)

Quan trng v c ngha nht l quan h lin kt gia LTTT v

Thng tin l mt khi nim tru tng, phi vt cht v rt kh

Thng tin (tt)

Thng tin l mt hin tng vt l, n thng tn ti v c

M hnh ca cc qu trnh truyn tin

Khi nim thng tin thng i km vi mt h thng truyn tin.

S truyn tin (transmission)

L s dch chuyn thng tin t im ny n im khc trong

Ngun tin (information source)

L mt tp hp cc tin m h thng truyn tin dng lp cc

M hnh ca cc qu trnh truyn tin (tt)

Thng tin trc khi c truyn i, tu theo yu cu c th

Knh tin (channel)

L ni hnh thnh v truyn (hoc lu tr) tn hiu mang tin

Mt s khi nim (tt)

Mi trng truyn tin thng rt a dng

mi trng khng kh, tin c truyn di dng m thanh v ting ni,

Cho d mi trng no cng c nhiu. Nhiu rt phong ph v

Chng hn nu truyn di dng sng in t m c i qua cc vng ca

Mt s khi nim (tt)

Nhiu c nhiu loi chng hn nhiu cng, nhiu nhn.

Ni nhn tin (sink)

L ni tip nhn thng tin t knh truyn v c gng khi phc

Cc loi h thng truyn tin

Cc ngun tin thng thy trong t nhin c gi l cc ngun

Cc loi h thng truyn tin (tt)

Cc ngun nh vy c gi l cc ngun lin tc (continuous

Bng ch ci ca mt ngn ng.

Trong trng hp ny cc ngun c gi l ngun ri rc

Cc h thng lin tc c nhiu nhc im ca nh cng knh,

Mt hm s(t) c ph hu hn, khng c thnh phn tn s ln

Bin ca cc tn hiu thng l mt min lin tc (smin, smax).

Ngun tin lin tc sau khi c ly mu v lng t ho s tr

Bng tin ca mt ngun tin ri rc khng nh

L mt dy (c th v hn) cc k hiu lin tip t bng ch ci

Khng gian mu (Sample space)

S kin (Event), s kin c bn (elementary event)

L tp (hay khng gian) tt c cc kt qu c th c ca mt th

Trong mt th nghim tung ng xu th E = {U (p), N (nga)}.

Bin ngu nhin ri rc (Discrete random variable)

Ly mt vn bn ting Anh in hnh v nht mt k t bt k

Tr trung bnh (k vng) (average, expected value),

Tr trung bnh v phng sai ca bin ngu nhin ri rc x ln

Xc sut ng thi (joint probability), xc sut c iu