Professional Documents
Culture Documents
On Tap Ly Thuyet Thong Tin
On Tap Ly Thuyet Thong Tin
Ngnh o to: in t vin thng H o to : i hc Mn hc : L thuyt thng tin Phn 1: l thuyt thng tin
S VHT: 4
Cu 1: (1 im): nh ngha lng thng tin ring ( bt nh) ca mt bin ngu nhin. Xc nh cc n v o - nh ngha lng thng tin ring ( bt nh) Lng thng tin ring l bt nh tim nng cha trong mt bin c ngu nhin xk . K hiu I ( xk )
I ( xk ) = k ln p ( xk ) - Cc n v o k = 1
k=
I ( xk ) = ln p ( xk ) (nat)
H1 ( A ) = M I ( a i )
a2 ... as a1 A= p ( a1 ) p ( a 2 ) ... p ( a s ) 0 p (ai ) 1
s'
p(a ) = 1
i =1 i
H1 ( A ) = p ( a i ) log p ( a i ) (bt)
i =1
http://www.ebook.edu.vn
- Khi p ( a k ) = 1 , p ( a i ) = 0 vi i k th H1 ( A ) = H1 ( A )min = 0
H1 ( A )max = logs
- Entropy ca ngun ri rc l mt i lng gii ni 0 H1 ( a ) logs
Cu 4: (1 im) nh ngha kh nng thng qua knh ri rc, nu cc tnh cht? - nh ngha: Kh nng thng qua ca knh ri rc l gi tr cc i ca lng thng tin cho trung bnh truyn qua knh trong mt n v thi gian ly theo mi kh nng c th c ca ngun tin A.
- Cc tnh cht:
+ C' 0
H1 ( A ) = H1 ( A )max = 1bit
1 2
- ngha: 1 bt l lng thng tin ring trung bnh cha trong mt bin c ca mt ngun ri rc 2 phn ng xc sut.
http://www.ebook.edu.vn
Cu 6: (2 im) Xc nh hai trng thi cc oan ca knh ri rc. - Knh b t: Cc ngun tin A v B hai u thu v pht l c lp.
p (ai b j ) = p(ai ) p ( b j ai ) = p (ai ) p (ai b j ) = p (ai ) p ( b j )
Ta c:
H(A bj ) = H(A) H ( A B) = H ( A )
Nhn xt: Lng thng tin tn hao trung bnh ng bng entropy ca ngun. Knh khng th truyn tin c - Knh khng nhiu: AB p ( a k bk ) = 1
H ( A bk ) = 0 H ( A B) = 0
Nhn xt: Lng thng tin tn hao trn knh bng 0 Cu 7: (2 im) Entropy c iu kin H ( A B ) : nh ngha v nu cc tnh cht
- nh ngha: Entropy c iu kin v 1 trng tin A khi r trng tin B c xc nh theo cng thc sau:
H ( A B ) = p ( a i b j ) log p ( a i b j )
i =1 j=1 s t
- Cc tnh cht: + H ( AB ) = H ( A ) + H ( B A ) = H ( B ) + H ( A B )
+ 0 H ( A B) H ( A ) + H ( AB ) = H ( A ) + H ( B ) Cu 8: (2 im) Lng thng tin cho trung bnh truyn qua knh ri rc: nh ngha v tnh cht - nh ngha:
I ( A, B ) = M I ( a i ,b j )
vi
I ( a i ,b j ) = log
p (ai b j ) p (ai )
http://www.ebook.edu.vn
I ( A,B ) = p ( a i b j ) log
i =1 j=1
p (ai b j ) p (ai )
p ( a2 ) = 1 p p ( b1 a2 ) = p ( b2 a1 ) ps = 1 pd
Cho tc truyn tin ca knh vk = 1 T
a1
p ( b1 a2 ) = pd
B
b1
ps
ps
a2 Tnh kh nng thng qua C ' ca knh ny. Gii: 1 1 H ( B) H ( B A ) Ta c C' = max I ( A,B ) = max T A T A Trong :
p ( b2 a1 ) = pd
b2
H ( B A ) = p ( a i ) p ( b j a i ) log p ( b j a i )
i =1 j=1
= p ( a1 ) p ( b1 a1 ) log p ( b1 a1 ) + p ( b 2 a1 ) log p ( b 2 a1 ) p(a2 ) p ( b1 a 2 ) log p ( b1 a 2 ) + p ( b 2 a 2 ) log p ( b 2 a 2 ) = p (1 ps ) log (1 ps ) + ps log ps (1 p ) ps log ps + (1 ps ) log (1 ps ) = ps log ps + (1 ps ) log (1 ps )
Ta thy H ( B A ) khng ph thuc vo xc sut tin nghim ca cc tin thuc ngun A. Do :
1 1 max H ( B ) H ( B A ) T A T Ta c max H ( B ) = H ( B )max = log 2 = 1bit C' =
A
C' C'max
C'max =
0,5
ps
http://www.ebook.edu.vn
Cu 10: (3 im) A chn ngu nhin mt trong cc s t 0 n 7. Tnh s cu hi trung bnh ti thiu m B cn t cho A xc nh c s m A chn. Nu thut ton hi? Gi s A chn s 3, hy t cc cu hi cn thit? bt nh ca s c chn ngu nhin: 1 I ( a i ) = log p ( a i ) = log = 3bit 8 tm c s chn ca A, B phi t cc cu hi cho A thu c mt lng thng tin cn thit l 3 bt. Mi cu hi ca B (dng la chn) c th xem l ngun ri rc nh phn, bi vy lng thng tin nhn c sau mi cu tr li tng ng l: H ( B ) = plog p (1 p ) log (1 p ) Vi p : xc sut nhn cu tr li ng 1 p : xc sut nhn cu tr li sai
Vy s cu hi cn thit n l : n =
H ( B )max khi p = 1 p =
n min =
1 2
3bit = 3 ln hi 1bit
Gi s A chn s B. Cc cu hi b c th t cho A l: - Cu 1 - S A chn ln hn 3? Tr li: Sai - Cu 2 - S A chn ln hn 1? Tr li: ng - Cu 3 - S A chn ln hn 2? Tr li: Sai Vy s A chn l 3 Cu 11: (4 im) Mt thit b v tuyn in gm 16 khi c tin cy nh nhau v c mc ni tip. Ta s dng mt thit b o o tn hiu ra ca cc khi. Gi s c mt khi no b hng. Hy tnh s ln o trung bnh ti thiu tm c khi b hng. Nu thut ton o? Gi s khi hng l khi th 6, tm v tr cc im o cn thit? Theo gi thit bt nh ca khi hng l: 1 I ( a i ) = log p ( a i ) = log = 4bit 16 Lng thng tin nhn c sau mi php o: H ( B ) = plog p (1 p ) log (1 p )
http://www.ebook.edu.vn 5
Vi
H ( B ) max khi p = 1 p =
H ( B )max
n min =
1 2 = 1bit
4bit = 4 ln o 1bit
1 . 2
Gi s khi hng l khi 6. Cc php o cn thit l: - Ln 1: o u ra khi 8: Khng c tn hiu, khi hng nm trong cc khi t 1 8. - Ln 2: o u ra khi 4: Khng c tn hiu, khi hng nm trong cc khi t 5 8. - Ln 3: o u ra khi 6: Khng c tn hiu, khi hng nm trong khi 5 hoc 6. - Ln 4: o u ra khi 5: C tn hiu. Vy khi hng l khi 6 Cu 12: (4 im) Trong b t l kh 52 qun (khng k fng teo), A rt ra mt qun bi bt k. Tnh s cu hi trung bnh ti tiu m B cn t cho A xc nh c qun bi m A rt. Nu thut ton hi? Gi s A rt ra 7 qun r, hy nu cc cu hi cn thit? bt nh v qun bi m A rt: 1 I ( a i ) = log p ( a i ) = log < 6bit 52 Gi s B t cho A cu hi dng la chn, khi lng thng tin nhn c sau mi cu tr li ca A l: H ( B ) = plog p (1 p ) log (1 p ) Vi p : xc sut nhn cu tr li l ng 1 p : xc sut nhn cu tr li l sai
S cu hi cn thit xc nh c qun bi A rt l: n =
http://www.ebook.edu.vn 6
I(ai ) H ( B)
1 . 2 Gi s A rt ra 7 r. Cc cu hi cn thit c th nh sau: - Cu 1: Qun A rt l qun ? ng - Cu 2: Qun A rt l qun c? Sai - Cu 3: Qun A rt c gi tr 7? ng (gi s J = 11, Q = 12, K = 13, At=1) - Cu 4: Qun A rt c gi tr 3? Sai - Cu 5: Qun A rt c gi tr 5? Sai - Cu 6: Qun A rt l 6 r? Sai Vy qun A rt l 7 r
Cu 13 :(4 im) Trong 27 ng xu c 1 ng xu gi nh hn. tm c ng xu gi ngi ta s dng mt cn a thng bng. Hy tnh s ln cn trung bnh ti thiu xc nh c ng xu gi. Nu thut ton cn ? Theo gi thit bt nh cha trong s kin ng xu gi l : I(xi) = - log p(xi) = - log 1/27 = log 27 bit Khi s dng cn a thng bng, sau mi ln cn cc s kin c th c l : - Cn thng bng vi xc sut p - Cn lch tri vi xc sut q - Cn lch phi vi xc sut 1-p-q Lng thng tin nhn c sau mi ln cn : H(B) = -plog p qlog q (1-p-q)log (1-p-q) xc nh c ng xu gi tng lng thng tin nhn c sau cc ln cn phi khng nh hn bt nh ca ng xu gi. Nh vy s ln cn cn thit l : n = I(xi)/H(B) n c gi tr nh nht th H(B) phi t gi tr cc i. Ta c H(B) = H(B)max= log 3 khi p = q = 1-p-q = 1/3. Khi nmin= I(xi)/H(B)max = log27/log 3 = 3 ln cn. Thut ton cn nh sau( m bo p = q = 1-p-q ) - Ln 1 : Chia 27 ng xu thnh 3 phn, mi phn c 9 ng xu. Ly 2 phn bt k t ln mi bn cn 1 phn . Nu cn thng bng th ng xu gi
http://www.ebook.edu.vn 7
nm trong 9 ng xu cha cn. Ngc li, tu theo cn lch tri hay lch phi ta cng xc nh c phn c cha ng xu gi. - Ln 2 : Chia 9 ng c cha ng xu gi thnh 3 phn nh nhau, mi phn c 3 ng xu. t 2 phn bt k ln 2 bn cn. Kt qu ca php cn s gip ta xc nh c 3 ng xu c cha ng xu gi. - Ln 3 : Ly 2 ng xu bt k trong 3 ng xu c cha ng xu gi t ln 2 a cn. Sau ln cn ny ta s xc nh c ng xu gi. Cu 14 : (2 im) Tnh entropie ca trng s kin ng thi ? Xt 2 trng s kin A v B sau : ai bj A= B= i = 1,0 j = 1, t p ( a i ) p ( b j ) Khi , trng s kin ng thi C = A.B l : aib j C= i, j, i = 1,0, j = 1, t p a p b ( ) ( ) i j Theo nh ngha, entropie ca trng s kin ng thi c xc nh nh sau :
H ( C ) = p ( a i b j ) log p ( a i b j )
i =1 j=1
Cu 15: (2 im) Cho knh nh phn i xng khng nh sau (hnh v). Hy tnh phn b xc sut ca cc tin u ra? Bit rng p(a1)=p ; p(a2)=1-p. A p ( b1 a2 ) = pd Theo cng thc xc sut y ta c: p ( b1 ) = p ( a1 ) .p ( b1 a1 ) + p ( a 2 ) .p ( b1 a 2 )
a1
B
b1
p ( b 2 ) = p ( a1 ) .p ( b 2 a1 ) + p ( a 2 ) .p ( b 2 a 2 ) = 1 p ( b1 )
a2
pS
pS
p ( b2 a1 ) = pd
b2
Phn 2: l thuyt m ho Cu 1: (1 im): nh ngha di trung bnh ca t m ? Pht biu nh l m ho th nht ca Shannon? Xt php m ho cc tin ri rc sau: a i ini
http://www.ebook.edu.vn 8
ini ai A= V= p(a ) i p (ai ) - nh ngha: di trung bnh ca t m l k vng ca i lng ngu nhin n i
n = p ( ai ) ni
i =1 s
- nh l m ho th nht ca Shannon (i vi m nh phn) Lun lun c th xy dng c mt php m ho cc tin ri rc c hiu qu m di trung bnh ca t m c th nh tu , nhng khng nh hn entropie xc nh bi cc c tnh thng k ca ngun
n H1 ( A )
Cu 2: (1 im) Nu nguyn tc lp m tit kim? T nh l m ho th 1 ca Shannon:
n = p ( a i ) n i H1 ( A ) = p ( a i ) log p ( a i )
i =1 i =1
ta c: n i log
1 p (ai )
Nguyn tc: Cc tin c xc sut xut hin ln c m ho bng cc t m c di nh v ngc li cc tin c xc sut xut hin nh c m ho bng cc t m c di ln. Cu3: (1 im) Trng s ca t m: nh ngha v tnh cht?
( )
( )
) (
- nh l v kh nng pht hin sai: M u nh phn c tha vi khong cch Hamming d 0 > 1 c kh nng
pht hin t sai tho mn iu kin t d 0 1 .
- nh l v kh nng sa sai:
http://www.ebook.edu.vn 9
i7 = 0 1 1 0 1 0 0 7 j =1 0 1 0 0 1 1 * * * * * d i7 , 7 j =5
- Tnh cht:
n i n i
+ d in , n j 0
n j n j
( ) d ( , ) = 0 d ( , ) n
khi in n j
) (
) (
nh ngha khong cch m ti thiu: d0= min d( ain, ajn) vi mi i,j Cu 6: (2 im) Cho m xyclic V - ( n - k ) c a thc sinh g ( x ) = 1+ x + x 3
x7 + 1 Ta c : h ( x ) = = x4 + x2 + x + 1 g(x)
http://www.ebook.edu.vn
10
( )
Ta c G.H T = 0
Cu 7: (2 im) hy thit lp t m h thng ca b m xyclic (7,4) c a thc
x6 + x 4 x 3 + x + 1 x6 + x 4 + x3 x3 + 1 x3 x3 + x + 1 r(x) = x +1
- Bc 4: Thit lp t m f(x)
f ( x ) = a ( x ) x n k + r ( x ) f (x) = x6 + x4 + x + 1 1 1 0 0 1 0 1 x6 . . . . . . x6
http://www.ebook.edu.vn 11
)(
)
d0
M (n, k)
1 + x + x2 + x4
1 + x2 + x3 + x 4
x
i=0
2 3 3 4 4
7 1
p (ai )
14 0 0 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1
12
T m ini
ni 2 2
a2 a3
a4
14 18
18
0 1 0 1 1 1 1 1 0 0 1 1 1 0 1 0 1 1 0 1
3 3 3 5 5 5 6 6
a5 a6
a7 a8
18 1 32
1 32 1 32
a9
a10
1 64
1 64
http://www.ebook.edu.vn
- nh gi hiu qu:
10 1 1 1 1 n = p ( a i ) n i = 2.2. + 3.3. + 3.5. + 2.6. 4 8 32 64 i =1 9 15 6 25 =1+ + + = 2. du 8 32 32 32 10 1 1 1 1 1 = 2. .log 4 + 3. log8 + 3. log 32 + 2. log 64 H1 ( A ) = p ( a i ) log p (ai ) 4 8 32 64 i =1
= 2.
25 bit 32
- Bc 3: (ln 2): x 2 .v ( x ) = x 6 + x 5 + x 4 + x + 1
x6 + x 5 + x 4 + x + 1 x 3 + x + 1 x6 + x 4 + x3 + x 2 x2 + x x5 + x3 + x 2 + x + 1 x5 + x3 + x 2 + x r2 ( x ) = 1 W ( r2 ( x ) ) = 1 = t - Bc 4:
x 2 v ( x ) + r2 ( x ) x 6 + x 5 + x 4 + x f (x) = = 2 x x2 ^ f ( x ) = x 6 + x 4 + x 3 + x 2 0011101
^ *
Sai x 5 c sa
thc sinh g ( x ) = 1+ x 2 + x 3 + x 4 .
Cho m xyclic (7, 3) c g ( x ) = 1 + x 2 + x 3 + x 4 ma trn sinh:
xg ( x ) = x + x 3 + x 4 + x 5 0 1 0 1 1 1 0 x 2g ( x ) = x 2 + x 4 + x 5 + x 6 0 0 1 0 1 1 1
(x + x )g(x) = x + x + x + x (1 + x + x ) g ( x ) = 1 + x + x + x
2 2 3 2 4 6
(1 + x ) g ( x ) = 1 + x + x 2 + x 5 (1 + x 2 ) g ( x ) = 1 + x3 + x 5 + x 6
6
111 0 0 1 0 1 0 0 1 0 11 0 111 0 0 1 11 0 0 1 0 1
http://www.ebook.edu.vn
14
Cu 12: (3 im) Pht biu v chng minh gii hn Hamming? nh ngha m hon thin? - Gii hn Hamming. Vi m tuyn tnh (n, k), iu kin cn s c t sai l:
C
i =0
i n
2n k
S cc trng thi khc nhau ca cc du kim tra l: 2r = 2n k sa c sai, mi trng thi ca cc du kim tra ch c gn ti a cho 1 kiu sai. Vy sa c tt c cc kiu sai c trng s t ta c:
C
i =0
i n
2n k
Cu 13: (4 im) Cho phn tch ca x15+1 nh sau : X15+1=(1+x)(1+x+x2)(1+x+x4)(1+x3+x4)(1+x+x2+x3+x4) Hy nu tt c cc m xyclic c di 15 trn vnh Z2[x]/x15+1?
t
g1 ( X ) = 1 + X g3 ( X ) = 1 + X + X 4 g2 ( X ) = 1 + X + X2 g 4 ( X ) = 1 + X3 + X 4
http://www.ebook.edu.vn
15
TT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
a thc sinh g1 ( X ) g2 ( X ) g3 ( X ) g4 ( X ) g5 ( X )
M (n,k) (15,14) (15,13) (15,11) (15,11) (15,11) (15,12) (15,10) (15,10) (15,10) (15,9) (15,9) (15,9) (15,7) (15,7) (15,7) (15,8) (15,8) (15,8) (15,5) (15,5) (15,6) (15,6) (15,6) (15,5) (15,3) (15,4) (15,4) (15,4) (15,1) (15,2) (15,15)
16
g1.g 2 g1.g3
g1.g 4
g1.g5 g 2 .g 3
g 2 .( g 4 )
g 2 .g 5
g 3 .g 4
g 3 .g 5
g 4 .g 5
g1.g 2 .g 3
g1.g 2 .g 4 g1.g 2 .g 5
g 2 .g 3.g 4
g 2 .g 3 .g 5 g1.g 3 .g 4
g1.g 3.g 5
g1.g 4 .g 5 g 2 .g 4 .g 5
g 3 .g 4 .g 5
g1.g 2 .g 3 .g 4
g1.g 2 .g 3.g 5
g1.g 2 .g 4 .g 5 g 2 .g 3 .g 4 .g 5
g1.g 3 .g 4 .g 5
1
http://www.ebook.edu.vn
Cu 14:(3 im) M t s chc nng thit b m ho theo phng php chia cho m xyclic h thng (7,4) c a thc sinh g(x)=1+x+x3. Tm t m u ra ca thit b ny khi a thc thng tin u vo a(x)=1+x3.
M (7, 4) c g ( X ) = 1 + X + X 3
V1
1 4
Vo
V2
Vo 1 0 0 1 0 0 0
Ra 1 0 0 1 1 1 0
T m ra 0 1 1 1 0 0 1 X + X 2 + X3 + X 6
http://www.ebook.edu.vn
17
Cu 15: (2 im) M t vnh a thc vi 2 php ton cng v nhn cc a thc theo modulo xn+1
Vnh a thc:
Z2 [ x ]
n + 1
n 1
f ( X ) = fi xi ; deg f ( X ) n 1 ; fi GF ( 2 )
Php cng: a ( X ) = a i xi ; b ( X ) = bi xi
i =0
i =0 n 1
n 1
c ( X ) = a ( X ) + b ( X ) = ci xi trong ci = a i + bi mod 2
i =0
i =0 n 1
http://www.ebook.edu.vn
18