Professional Documents
Culture Documents
Ôn tập LTTT
Ôn tập LTTT
S VHT: 4
I ( xk ) = k ln p ( xk )
- Cc n v o
k = 1
I ( xk ) = ln p ( xk ) (nat)
1
I ( xk ) = log 2 p ( xk ) (bt)
ln 2
1
I ( xk ) = lg p ( xk ) (hart)
k=
ln10
1 nat = 1,443 bt
1 hart = 3,322 bt
k=
H1 ( A ) = M I ( a i )
a2
...
as
a1
A=
p ( a1 ) p ( a 2 ) ... p ( a s )
0 p (ai ) 1
p(a ) = 1
i =1
s'
H1 ( A ) = p ( a i ) log p ( a i ) (bt)
i =1
http://www.ebook.edu.vn
- Khi p ( a k ) = 1 , p ( a i ) = 0 vi i k th H1 ( A ) = H1 ( A )min = 0
H1 ( A )max = logs
- Entropy ca ngun ri rc l mt i lng gii ni
0 H1 ( a ) logs
Cu 4: (1 im) nh ngha kh nng thng qua knh ri rc, nu cc tnh cht?
- nh ngha: Kh nng thng qua ca knh ri rc l gi tr cc i ca lng
thng tin cho trung bnh truyn qua knh trong mt n v thi gian ly theo
mi kh nng c th c ca ngun tin A.
- Cc tnh cht:
+ C' 0
(bits)
H1 ( A )
p 1 p
H1 ( A )max = plog p (1 p ) log (1 p )
1
Khi p = 1 p =
0,5
1
2
H1 ( A ) = H1 ( A )max = 1bit
- ngha: 1 bt l lng thng tin ring trung bnh cha trong mt bin c ca
mt ngun ri rc 2 phn ng xc sut.
http://www.ebook.edu.vn
p ( b j ai ) = p (ai )
p (ai b j ) = p (ai ) p ( b j )
H(A bj ) = H(A)
Ta c:
H ( A B) = H ( A )
Nhn xt: Lng thng tin tn hao trung bnh ng bng entropy ca
ngun. Knh khng th truyn tin c
- Knh khng nhiu:
AB
p ( a k bk ) = 1
H ( A bk ) = 0
H ( A B) = 0
Nhn xt: Lng thng tin tn hao trn knh bng 0
Cu 7: (2 im) Entropy c iu kin H ( A B ) : nh ngha v nu cc tnh cht
H ( A B ) = p ( a i b j ) log p ( a i b j )
i =1 j=1
- Cc tnh cht:
+ H ( AB ) = H ( A ) + H ( B A ) = H ( B ) + H ( A B )
+ 0 H ( A B) H ( A )
+ H ( AB ) = H ( A ) + H ( B )
Cu 8: (2 im) Lng thng tin cho trung bnh truyn qua knh ri rc: nh
ngha v tnh cht
- nh ngha:
I ( A, B ) = M I ( a i ,b j )
vi
I ( a i ,b j ) = log
http://www.ebook.edu.vn
p (ai b j )
p (ai )
I ( A,B ) = p ( a i b j ) log
i =1 j=1
p (ai b j )
p (ai )
I ( A, B ) = H ( A ) H ( A B ) = H ( B ) H ( B A )
- Tnh cht:
+ I ( A,B ) 0
+ I ( A, B ) H ( A )
Cu 9: (3 im) Cho knh i xng nh phn sau
A
p ( a1 ) = p
p ( a2 ) = 1 p
p ( b1 a2 ) = pd
a1
b1
p ( b1 a2 ) = p ( b2 a1 ) ps = 1 pd
Cho tc truyn tin ca knh vk = 1 T
a2
Tnh kh nng thng qua C ' ca knh ny.
Gii:
1
1
Ta c C' = max I ( A,B ) = max H ( B ) H ( B A )
T A
T A
Trong :
2
ps
ps
b2
p ( b2 a1 ) = pd
H ( B A ) = p ( a i ) p ( b j a i ) log p ( b j a i )
i =1 j=1
= p ( a1 ) p ( b1 a1 ) log p ( b1 a1 ) + p ( b 2 a1 ) log p ( b 2 a1 )
p ( a 2 ) p ( b1 a 2 ) log p ( b1 a 2 ) + p ( b 2 a 2 ) log p ( b 2 a 2 )
= p (1 ps ) log (1 ps ) + ps log ps (1 p ) ps log ps + (1 ps ) log (1 ps )
= ps log ps + (1 ps ) log (1 ps )
Ta thy H ( B A ) khng ph thuc vo xc sut tin nghim ca cc tin thuc
ngun A. Do :
1
1
max H ( B ) H ( B A )
T A
T
Ta c max H ( B ) = H ( B )max = log 2 = 1bit
C' =
C'max =
1
khi ps = 0 (knh khng nhiu)
T
C'
= 1 + ps log ps + (1 ps ) log (1 ps )
C'max
http://www.ebook.edu.vn
C' C'max
0,5
ps
p : xc sut nhn cu tr li ng
1 p : xc sut nhn cu tr li sai
Vy s cu hi cn thit n l : n =
I(ai )
H ( B)
H ( B )max khi p = 1 p =
n min =
I (ai )
H ( B )max
1
2
3bit
= 3 ln hi
1bit
Gi s A chn s B. Cc cu hi b c th t cho A l:
- Cu 1 - S A chn ln hn 3? Tr li: Sai
- Cu 2 - S A chn ln hn 1? Tr li: ng
- Cu 3 - S A chn ln hn 2? Tr li: Sai
Vy s A chn l 3
Cu 11: (4 im) Mt thit b v tuyn in gm 16 khi c tin cy nh nhau
v c mc ni tip. Ta s dng mt thit b o o tn hiu ra ca cc khi.
Gi s c mt khi no b hng. Hy tnh s ln o trung bnh ti thiu tm
c khi b hng. Nu thut ton o? Gi s khi hng l khi th 6, tm v tr
cc im o cn thit?
Theo gi thit bt nh ca khi hng l:
1
I ( a i ) = log p ( a i ) = log = 4bit
16
Lng thng tin nhn c sau mi php o:
H ( B ) = plog p (1 p ) log (1 p )
http://www.ebook.edu.vn
Vi
p : xc sut c tn hiu
1 p : xc sut khng c tn hiu
1
2
= 1bit
H ( B ) max khi p = 1 p =
H ( B )max
n min =
4bit
= 4 ln o
1bit
1
.
2
p : xc sut nhn cu tr li l ng
1 p : xc sut nhn cu tr li l sai
S cu hi cn thit xc nh c qun bi A rt l: n =
http://www.ebook.edu.vn
I(ai )
H ( B)
1
2
log52 bit
< 6 ln
1bit
1
.
2
Gi s A rt ra 7 r. Cc cu hi cn thit c th nh sau:
- Cu 1: Qun A rt l qun ? ng
- Cu 2: Qun A rt l qun c? Sai
- Cu 3: Qun A rt c gi tr 7? ng (gi s J = 11, Q = 12, K = 13,
At=1)
- Cu 4: Qun A rt c gi tr 3? Sai
- Cu 5: Qun A rt c gi tr 5? Sai
- Cu 6: Qun A rt l 6 r? Sai
Vy qun A rt l 7 r
nm trong 9 ng xu cha cn. Ngc li, tu theo cn lch tri hay lch
phi ta cng xc nh c phn c cha ng xu gi.
- Ln 2 : Chia 9 ng c cha ng xu gi thnh 3 phn nh nhau, mi phn
c 3 ng xu. t 2 phn bt k ln 2 bn cn. Kt qu ca php cn s
gip ta xc nh c 3 ng xu c cha ng xu gi.
- Ln 3 : Ly 2 ng xu bt k trong 3 ng xu c cha ng xu gi t ln
2 a cn. Sau ln cn ny ta s xc nh c ng xu gi.
Cu 14 : (2 im) Tnh entropie ca trng s kin ng thi ?
Xt 2 trng s kin A v B sau :
b j
ai
A=
B=
i = 1,0
j = 1, t
p ( a i )
p ( b j )
Khi , trng s kin ng thi C = A.B l :
a i b j
C=
i, j, i = 1,0, j = 1, t
p ( a i ) p ( b j )
Theo nh ngha, entropie ca trng s kin ng thi c xc nh nh
sau :
s
H ( C ) = p ( a i b j ) log p ( a i b j )
i =1 j=1
Cu 15: (2 im) Cho knh nh phn i xng khng nh sau (hnh v).
Hy tnh phn b xc sut ca cc tin u ra?
Bit rng p(a1)=p ; p(a2)=1-p.
A
p ( b1 a2 ) = pd
Theo cng thc xc sut y ta c:
p ( b1 ) = p ( a1 ) .p ( b1 a1 ) + p ( a 2 ) .p ( b1 a 2 )
a1
b1
pS
p ( b 2 ) = p ( a1 ) .p ( b 2 a1 ) + p ( a 2 ) .p ( b 2 a 2 )
= 1 p ( b1 )
a2
pS
p ( b2 a1 ) = pd
b2
Phn 2: l thuyt m ho
Cu 1: (1 im): nh ngha di trung bnh ca t m ? Pht biu nh l m
ho th nht ca Shannon?
Xt php m ho cc tin ri rc sau: a i ini
http://www.ebook.edu.vn
ini
ai
A=
V =
p
a
(
)
i
p (ai )
n = p ( ai ) ni
i =1
i =1
i =1
n = p ( a i ) n i H1 ( A ) = p ( a i ) log p ( a i )
ta c: n i log
1
p (ai )
( )
- Tnh cht:
( )
+ 0 W ini 1
) (
+ W in + nj = d in , nj
- nh l v kh nng sa sai:
http://www.ebook.edu.vn
d 1
sa c t sai tho mn iu kin: t 0 . Trong [ x ] k hiu phn
2
nguyn ca s x.
Cu 5: (2 im) Khong cch m: nh ngha, tnh cht? nh ngha khong
cch m ti thiu?
- nh ngha:
Khong cch gia hai t m in v nj l s cc du m khc nhau v gi
tr tnh theo cng mt th t gia 2 t m ny.
i7 = 0 1 1 0 1 0 0
V d:
7j = 1 0 1 0 0 1 1
* *
* * *
d i7 , 7j = 5
- Tnh cht:
(
)
d ( , ) = 0
d ( , ) n
+ d in , nj 0
n
i
n
j
n
i
n
j
khi in nj
) (
) (
G= 2
x
3
x
Ma trn kim tra H :
+x
+x2
+x3
+x4
+x3 1 1 0 1 0 0 0
+x4 0 1 1 0 1 0 0
=
+ x5 0 0 1 1 0 1 0
+ x 6 0 0 0 1 1 0 1
x7 + 1
Ta c : h ( x ) =
= x4 + x2 + x + 1
g(x)
http://www.ebook.edu.vn
10
x7 + 1
x3 + x + 1
x7 + x5 + x 4 x 4 + x 2 + x + 1
x5 + x 4 + 1
x5 + x3 + x2
x 4 + x3 + x 2 + 1
x4 + x2 + x
x3 + x + 1
x3 + x + 1
0
( )
h * ( X ) = X ( ) h X 1 = x 4 + x 3 + x 2 + 1
h* ( x )
*
x.h
x
(
)
H=
........
r 1 *
x
.h
x
(
)
2
3
1 + x + x + x 4 1 0 1 110 0
H = x + x 3 + x 4 + x 5 = 0 1 0 111 0
x 2 + x 4 + x 5 + x 6 0 0 1 011 1
deg h x
Ta c G.H T = 0
Cu 7: (2 im) hy thit lp t m h thng ca b m xyclic (7,4) c a thc
- Bc 2: Nng bc a ( x ) x n-k = x 3 + x x 74 = x 6 + x 4
- Bc 3: Chia tnh d:
x6 + x 4 x 3 + x + 1
x6 + x 4 + x3 x3 + 1
x3
x3 + x + 1
r(x) = x +1
- Bc 4: Thit lp t m f(x)
f ( x ) = a ( x ) x n k + r ( x )
f (x) = x6 + x4 + x + 1 1 1 0 0 1 0 1
x6 . . . . . . x6
http://www.ebook.edu.vn
11
)(
x 7 +1= ( 1+ x ) 1 + x + x 3 1 + x 2 + x 3
1+ x
1 + x + x3
1 + x 2 + x3
1 + x + x2 + x4
1 + x2 + x3 + x 4
i=0
M (n, k)
d0
( 7,6 )
( 7,4 )
( 7,4 )
( 7,3)
( 7,3)
( 7,1)
( 7,7 )
3
3
4
4
7
a1 a2 a3 a4 a5 a6 a7 a8 a9 a10
A= 1 1 1 1 1 1
1
1
1
1
4 4 8 8 8 32 32 32 64 64
nh gi hiu qu ca php m ho ny?
Hy gii m cho dy bt nhn c c dng: 101100111010101
TT
ai
p (ai )
a1
14
a2
14
a3
18
a4
18
a5
18
a6
1 32
a7
1 32
a8
1 32
a9
1 64
10
a10
1 64
http://www.ebook.edu.vn
T m ini
12
ni
- nh gi hiu qu:
10
1
1
1
1
n = p ( a i ) n i = 2.2. + 3.3. + 3.5. + 2.6.
4
8
32
64
i =1
9 15 6
25
=1+ +
+
= 2.
du
8 32 32
32
10
1
1
1
1
1
= 2. .log 4 + 3. log8 + 3. log 32 + 2. log 64
H1 ( A ) = p ( a i ) log
p (ai )
4
8
32
64
i =1
= 2.
25
bit
32
n = H1 ( A ) php m ho l ti u
- Gii m cho dy bt nhn sau:
1 0 1 1 0 0 1 1 1 0 1 0 1 0 1 ...
a4
a3
a8
a4 a2
x6 + x 5 + x 4 + x 3 + x 2 x 3 + x + 1
x6 + x 4 + x3 + x 2
x2 + x
x5
x5 + x3 + x 2 + x
r0 ( x ) = x 3 + x 2 + x
d 1 4 1
- Bc 2: W ( r0 ( x ) ) = 3 > t =
=
= 1,5
2 2
- Bc 3: (ln 1) x.v ( x ) = x 6 + x 5 + x 4 + x 3 + 1
x6 + x 5 + x 4 + x 3 + 1 x 3 + x + 1
x6 + x 4 + x3 + x 2
x2 + x
x5 + x 2 + 1
x5 + x3 + x 2 + x
r1 ( x ) = x 3 + x + 1
W ( r1 ( x ) ) = 3 > t = 1 Bc 3
http://www.ebook.edu.vn
13
- Bc 3: (ln 2): x 2 .v ( x ) = x 6 + x 5 + x 4 + x + 1
x6 + x 5 + x 4 + x + 1 x 3 + x + 1
x6 + x 4 + x3 + x 2
x2 + x
x5 + x3 + x 2 + x + 1
x5 + x3 + x 2 + x
r2 ( x ) = 1
W ( r2 ( x ) ) = 1 = t
- Bc 4:
x 2 v ( x ) + r2 ( x ) x 6 + x 5 + x 4 + x
f (x) =
=
2
x
x2
^
f ( x ) = x 6 + x 4 + x 3 + x 2 0011101
^
Sai x 5 c sa
thc sinh g ( x ) = 1+ x 2 + x 3 + x 4 .
Cho m xyclic (7, 3) c g ( x ) = 1 + x 2 + x 3 + x 4 ma trn sinh:
1 + x 2 + x 3 + x 4 1 0 1 110 0
G = x + x 3 + x 4 + x 5 = 0 1 0 111 0
x 2 + x 4 + x 5 + x 6 0 0 1 011 1
xg ( x ) = x + x 3 + x 4 + x 5 0 1 0 1 1 1 0
x 2g ( x ) = x 2 + x 4 + x 5 + x 6 0 0 1 0 1 1 1
(1 + x ) g ( x ) = 1 + x + x 2 + x 5
(1 + x 2 ) g ( x ) = 1 + x3 + x 5 + x 6
(x + x )g(x) = x + x + x + x
(1 + x + x ) g ( x ) = 1 + x + x + x
2
http://www.ebook.edu.vn
14
111 0 0 1 0
1 0 0 1 0 11
0 111 0 0 1
11 0 0 1 0 1
C
i =0
i
n
2n k
C
i =0
t
g1 ( X ) = 1 + X
g2 ( X ) = 1 + X + X2
g3 ( X ) = 1 + X + X 4
g 4 ( X ) = 1 + X3 + X 4
http://www.ebook.edu.vn
15
i
n
2n k
TT
1
a thc sinh
g1 ( X )
M (n,k)
(15,14)
g2 ( X )
(15,13)
g3 ( X )
(15,11)
g4 ( X )
(15,11)
g5 ( X )
(15,11)
g1.g 2
(15,12)
g1.g3
(15,10)
g1.g 4
(15,10)
g1.g5
(15,10)
10
g 2 .g 3
(15,9)
11
g 2 .( g 4 )
(15,9)
12
g 2 .g 5
(15,9)
13
g 3 .g 4
(15,7)
14
g 3 .g 5
(15,7)
15
g 4 .g 5
(15,7)
16
g1.g 2 .g 3
(15,8)
17
g1.g 2 .g 4
(15,8)
18
g1.g 2 .g 5
(15,8)
19
g 2 .g 3.g 4
(15,5)
20
g 2 .g 3 .g 5
(15,5)
21
g1.g 3 .g 4
(15,6)
22
g1.g 3.g 5
(15,6)
23
g1.g 4 .g 5
(15,6)
24
g 2 .g 4 .g 5
(15,5)
25
g 3 .g 4 .g 5
(15,3)
26
g1.g 2 .g 3 .g 4
(15,4)
27
g1.g 2 .g 3.g 5
(15,4)
28
g1.g 2 .g 4 .g 5
(15,4)
29
g 2 .g 3 .g 4 .g 5
(15,1)
30
31
g1.g 3 .g 4 .g 5
(15,2)
(15,15)
http://www.ebook.edu.vn
16
M (7, 4) c g ( X ) = 1 + X + X 3
1 4
V1
Vo
RA
1 7
57
V2
a ( X ) = 1 + X3
a ( X ) .x n k = X 6 + X3
Xung
nhp
Vo
1
2
3
4
5
6
7
1
0
0
1
0
0
0
Trng thi cc nh
1
2
1
1
1
0
0
1
1
1
1
1
0
1
1
0
0
1
0
0
0
0
0
0
T m ra 0 1 1 1 0 0 1 X + X 2 + X3 + X 6
http://www.ebook.edu.vn
17
Ra
1
0
0
1
1
1
0
Vnh a thc:
Z2 [ x ]
n + 1
n 1
f ( X ) = fi xi ; deg f ( X ) n 1 ; fi GF ( 2 )
i =0
n 1
n 1
Php cng: a ( X ) = a i xi ; b ( X ) = bi xi
i =0
i =0
n 1
c ( X ) = a ( X ) + b ( X ) = ci xi trong ci = a i + bi mod 2
i =0
http://www.ebook.edu.vn
18