You are on page 1of 73

# Xn Yn

W W

()

W
{1,2,,k}.
Xn(W).
,
Yn, p(yn|xn).
W,
W=g(Yn).
, W W.
g,
W W ?

1. (RX, , RY)
RX, e
RY =[p(y|x)], p(y|x)
y,
x. ,

p(y|x) 0, x;

p( y | x) 1.
yRY

, X , Y
.

, ,

, .

3. (k, n) (RX, , RY)
:
1. {1,2,,k}.
2. X : {1,2,,k} RXn,
X(1), X(2),..., X(k).
3.
g : RYn {1,2,,k},

.

4. ( )

i P g (Y) i | X i
,
i.

5. (n)
(k,n) :

( n ) max i .
i{1,2,..., k }

7.
- (k,n) :

log k
R .
n
BSC
(BSC), RX =
RY = {0,1}.
:
1
0 0
1 1
, 0 .
1 2
1 1 1

## 0 < <1/2, CBSC > 0.

BSC
M
.

M {0,1}n e , ..
x = (a1,a2,,an) M,
P(x) = 1/k,
k = |M|, .. k
( ).
,
max P(y | x) P(y | x),
xM

g(y) = x, y {0,1}n.

BSC, ..
.

BSC, {0,1}n,
:
d : {0,1}n {0,1}n R,
: x {0,1}n y {0,1}n
d(x,y)
x y , ..
x = (a1,a2,,an) y = (b1,b2,,bn),
n
d (x, y ) ai bi .
i 1

## , n = 5, x=00110 y=10101, d(x,y) = 3,

x y ,
.

, .. x, y, z {0,1}n
:
i. d(x,y) 0;
ii. d(x,y) = 0 x = y;
iii. d(x,y) = d(y,x);
iv. d(x,z) d(x,y) + d(y,z).
i.-iii. .
iv, z = (c1,c2,,cn).
n n n
d (x, y ) d (y, z) ai bi bi ci ai bi bi ci
i 1 i 1 i 1
n n
ai bi bi ci ai ci d (x, z).
i 1 i 1

d S = {0,1}n
.

x y.
x y S, BSC,
{0,1}n,
y x.

1. BSC
1 1
, 0 .
1 2

P(y|x1) > P(y|x2) d(x1,y) < d(x2,y),
y S x1, x2 .

: d(x,y) = m (0 m n). ,
P(y|x) = P((b1,, bn)| (a1,, an)) = P(b1| a1) P(bn| an),
aj {0,1} bj {0,1}, j=1,2,,n, :
P(y|x) = m (1 )n m, 0 m n.

, m1 = d(x1,y) m2 = d(x2,y), :
P(y | x1 ) m1 (1 )nm1 ,
P(y | x2 ) m2 (1 )nm2 .
,
m2 m1
P(y | x1 ) (1 )
m1
1
n m1
m2 ,
P(y | x2 ) (1 )nm2

0 < <1/2,
1/2 <1 < 1. , 1 < ,
1
1.

P(y|x1) > P(y|x2).
m2 m1
P(y | x1 ) 1
1 .
P( y | x 2 )
(1 )/ > 1, m1 m2 >0, .. m1 < m2.

m1 < m2.
m2 m1
1 P(y | x1 )
1 ,
P(y | x 2 )

## .. P(y|x1) > P(y|x2).

?

,

M {0,1}n ( )
BSC.
, y S,
x M, d(x,y) e
.
,
.
, y S, g(y) = x, x
:
min d (x, y) d (x, y).
xM

BSC.

,
BSC,
k ( )
{0,1}n
.
, ,
,
.
, ,
BSC,
M {0,1}n

.

n-

r (0 r n) ( 1 0 0 1).
, ,
k n
( ),

.

: M = {x1, x2,,xk}
n
d(xi, xj) 2r + 1, i j, i, j {1,,k},
.. M
2r + 1 .
,
, r
n- ,
.

, xi M, Si
y {0,1}n, d(xi, y) r, ..
Si = {y {0,1}n | d(xi, y) r}, i = 1,2,,k.
,
g(y) = xi, y Si, i = 1,2,,k.
, d(xi, xj) 2r + 1, Si Sj = , i j.
, xi M, r
, y
Si
xi, .
, r
xi,
Si,
.
Si
Si
r xi S =
{0,1}n n-
.

xi xj

,
r = 4, d(xi, y) = 3, d(xi, xj) = 9
()
2. ( )
k
BSC
r
, :

2n
k.
r
n

i 0 i
2
: ,
Si.
xi Si, xi Si.
1 xi n :
n- xi
.
n
2 xi 2 :

n- xi
.
, Si :

n n r n
1 n .
2 r i 0 i
2:
2,
Si Sj = , i j; i, j {1,2,,k}.
Si, i = 1,2,,k,
.
, k ,

r
n
k
i 0 i
n- .
, n- 2n,
:
r
n
k 2n.
i 0 i

r=2 ,
r
n
n
2 n

i 0 i
k = 10, 2
,
5 2
n = 9.
6 32/11 2.9
, 29 = 512
7 128/29 4.4
n- ,
8 256/37 6.9

9 256/23 11.1
10 128/7 18.3 11.

,
10 , n = 4,
24 = 16 .
-
.
M0 = {(0,0), (0,1), (1,0), (1,1)} e
BSC.

, ,
.
1 ,
.
M BSC,
M = {(0,0,0), (0,1,1), (1,0,1), (1,1,0)}.
23 = 8
,
,
(1 3) .
-
, -
, 1 3
3- .
2 , .

.

2:
+ 0 1 0 1
0 0 1 0 0 0
1 1 0 1 0 1

{0,1}

2 GF(2).
, x = (a1, a2, a3) M,
M ={(0,0,0), (0,1,1), (1,0,1), (1,1,0)}, :
a1 + a2 + a3 = 0,

GF(2).

.
F = [fij]mn, n m, e 0
1,
GF(2):
f11a1 f12 a2 f1n an 0
f a f a f 2 n an 0
21 1 22 2
.

f m1a1 f m 2 a2 f mn an 0

GF(2)
n- , M

{0,1}n.

, M
,
:
f11a1 f12 a2 f1n an 0
f a f a f 2 n an 0
21 1 22 2
.

f m1a1 f m 2 a2 f mn an 0

F
.

F t, ..
rang(F) = t, 1 t m,
t
F, n t = s (0 s n 1)

s
t = n s .
, 2s n-
,
BSC.
1
:
1 0 0 1 1 0
F 0 1 1 1 0 1 , r (F) 3.
1 0 1 1 0 1

, ,
.
, s = n t = 6 3 = 3, 23 = 8
.
1:

:
1 2 3 4 5 6

6-
x1 0 0 0 0 0 0

x2 0 0 1 0 0 1 3
x3 1 1 1 0 1 0 ,
x4 1 1 0 0 1 1 3
x5 1 1 0 1 0 0

F x = 0,
x6 1 1 1 1 0 1 .
x7 0 0 1 1 1 0
x8 0 0 0 1 1 1

F x = 0, x -
-, 0 :
a1 0
x , 0 0 .
a6 0
,
f11a1 f12 a2 f1n an 0
f a f a f 2 n an 0
21 1 22 2
.

f m1a1 f m 2 a2 f mn an 0

F x = 0, 0 m-
, x n- -
a1, a2,..., an {0,1}.
-
- (k, n), k = 2s.
-
F.

s
R
n

(
) () n-
.
2
:
F [1 1 1].
n
t = 1 s = n 1,

n 1 1
R 1 .
n n
F x = 0 :
a1 + a2 ++ an = 0,
k = 2n1
,
.
n,
1.

,
,
2.
3

n- .
2 ,
, ,
x1 = (0,0,,0) x2 = (1,1,,1).

F x = 0, :
1 1 0 0 0
1
0 1 0 0
1 0 0 1 0
F n 1

1 0 0 0 1
n
3:

1 1 0 0 0
1
0 1 0 0
1 0 0 1 0
F n 1

1 0 0 0 1
n

t = r(F) = n 1 s = n t = 1.
R = s/n = 1/n,
, n,
.
2 3
( 2)
(
3) .
,

,
.

S F,
:
1. x + (y + z) = (x + y) + z, x, y, z S;
2. x + y = y + x, x, y S;
3. o S, o + x = x + o = x , j x S;
3. j x S, x S,
x + (x) = x + x = o.
B1. (x + y) = x + y, x, y S F;
B2. ( + )x = x + x , j x S , F;
B3. (x) = ()x, j x S , F;
B4. 1x = x, j x S, 1 F.

M
S(F), x, y M , F, :
x + y M.
S = {0,1}n e

S = {0,1}n
GF(2),

n- 2:
(a1, a2,, an) + (b1, b2,, bn) = (a1 b1, a2 b2,, an bn).
, ,
:
(a1, a2,, an) = (a1, a2,, an),
{0,1}.
x S, x + x = o, .. x = x,
S, .
M {0,1}n
M
Fx = 0, S.
, M = {x S | Fx = o}, x, y M,
x + y M.
:
F( x + y) = Fx + Fy = o + o = o.
, x + y M, M S,
- (2s, n)
S.
T 3
, .
3. M S = {0,1}n
A n , x M,
Ax = 0. Ax = 0 e e
M.

: ,
s n, = 2s
M.
3:
, B
n, M.
B s n.
x M
s e1, e2,, es M,
:
x = 1e1 + 2e2 ++ ses , i {0,1}, i = 1,,s.
1, 2,, s 2s

x M. M 2s
.
,
x M, Ax = 0.
3:
r(B) = s, B s
. ,
s, .. (n s +1)-, (n s +2)-,..., (n 1)-
n- .
n s B

s .
, j- B :
b1 j b1,n s 1 b1,n s 2 b1,n 1 b1n
b b b b b
2 j 2,n s 1 2,n s 2 j , s 1
2, n 1
js ,
2n

j1
j2

j
b ,n s 1
b ,n s 2
b ,n 1
b bn
j 1, 2, ,n s
3:
j = 1, 2,..., ns,
:
bij j1bi ,n s 1 j 2bi ,n s 2 j , s 1bi ,n 1 js bin , i 1, , .
,
s
ji (j = 1, 2,..., ns, i = 1, 2,..., s).

A = [Ins, ],
Ins ns.
xi = (bi1, bi2,, bin) M, :
3:
1 0 0 11 12 1s bi1
0 1 0
2s
bi 2
Axi
21 22

0 0 1 n s ,1 n s ,2 ns ,s bin
bi1 11bi ,n s 1 12bi ,n s 2 1s bin 0
b b b b
21 i , n s 1 22 i , n s 2 0

i2 2 s in
.

i ,n s
b b
n s ,1 i , n s 1 b
n s ,2 i , n s 2 n s , s in
b 0
, ji
:
bij j1bi ,n s 1 j 2bi ,n s 2 j , s 1bi ,n 1 js bin , i 1, , .
3:
,
Ax = 0 M.
A = [Ins, ] r(A) = n s,
Ax = 0 s
2
: 0 1.
, 2s Ax = 0,
M 2s
.
4
n = 6 M {0,1}6
:
0 0 0 0 0 0
b0 = (0,0,0,0,0,0) 1 0 1 0 0 1
b1 = (1,0,1,0,0,1)
1 1 0 0 1 0
b2 = (1,1,0,0,1,0)
0 1 1 0 1 1
b3 = (0,1,1,0,1,1) B
1 1 1 1 0 0
b4 = (1,1,1,1,0,0)
b5 = (0,1,0,1,0,1) 0 1 0 1 0 1
0 0 1 1 1 0
b6 = (0,0,1,1,1,0)
b7 = (1,0,0,1,1,1)
1 0 0 1 1 1
r(B) = 3, , b1, b2 b4
.
4:
:
b1 1 0 1 0 0 1
b 1 1 0 0 1 0
2
b4 1 1 1 1 0 0

.

GF(2).

,
: 1 0 0 1
1 0 1 0 1,
11 12 13 11 12 13

1 1 0 0
4:
0 0 0 1
1 0 1 0 1, 1, 0,
21 22 23 21 22 23

1 1 0 0
1 0 0 1
0 0 1 0 1, 0, 1.
31 32 33 31 32 33

1 1 0 0

A 36 :

1 0 0 1 1 1
A 0 1 0 1 1 0 ,
0 0 1 1 0 1
F = A (k = 8, n = 6)
b0, b1,..., b7.
-

F m n (m n, r(F) = m)
-
M.
y S = {0,1}n, ,
F y = c, c {0,1}m -.
c ( )
y.
, c {0,1}m,

Sc = {y S | F y = c}.
, Sc y S
.
-

, y0 Sc

y0 + M ={y S | y = y0 + x, x M} S,
.
Sc = y0 + M.
, y Sc, F y0 = F y = c. ,
F(y y0) = Fy Fy0 = 0,
x M, x = y y0, . y = y0 + x.
y y0 + M, .. Sc y0 + M.
, y y0 + M, y = y0 + x, x M.
,
Fy = F(y0 + x) = Fy0 + Fx = c + 0 = c,
.. y Sc, y0 + M Sc.
, Sc = y0 + M

-, y = (b1, b2, , bn) S,
:
n
T (y ) bi ( ),
i 1

y
.
,
:
n

b d (0, y ) T (y).
i 1
i

T(y)
y
.

:
n
d (x, y ) ai bi ,
i 1
x = (a1,a2,,an) y = (b1,b2,,bn),
:
d(x + z, y + z) = d(x, y), x, y, z S.
, z = (c1,c2,,cn),
x + z = (a1 + c1,,an + cn), a y + z = (b1 + c1,,bn + cn).
, n
d (x z, y z ) (ai ci ) (bi ci )
i 1
n
ai bi
i 1

d (x, y ).

:
i. (y) 0, y S;
ii. (y) = 0 y = 0 S;
iii. (x + y) (x) + (y), x, y S.
i. ii. ,
iii.
(x + y) = d(0, x + y) d(0, x) + d(x, x + y)
= d(0, x) + d(0, y)
= (x) + (y)
-

:
d(x + z, y + z) = d(x, y), x, y, z S
:
d(y y0, y) = d(y y0 y, y y) = d( y0, 0) = d(0, y0) = T(y0).
, y, y0 S

## d(y y0, y) = T(y0).

y1 S, T(y1) T(y0),
:
d(y y0, y) d(y y1, y).
-

y S,
?
,
- F
M ,
y S x0 = g(y) M,
:
, Sc = {y1 S | Fy1 = Fy = c};
, y0 Sc ,
.. T(y0) T(y1), y1 Sc.
x0 = y y0.
d(y y0, y) d(y y1, y), :
d(x0, y) d(x, y), x = y y1 M.
x M,
Sc = y0 + M, y1
Sc, y y1
M.
5
:

1 0 0 0 1 1
0 1 0 0 1 0
F .
0 0 1 0 1 1

0 0 0 1 0 1
r(F) = 4 = t, :
s = n t = 6 4 = 2
4
Fx = 0 2s = 22 = 4 :
x1 = (0,0,0,0,0,0), x2 = (1,1,1,0,1,0),
x3 = (1,0,1,1,0,1), x4 = (0,1,0,1,1,1).
5
m = 4, 24 = 16 ,
c {0,1}4 Sc
, Sc = {y {0,1}6 | F y = c}.

Sc = y0 + M 4 .
, c
, , :
Sc = y0 + M
y0 T(y0) = 0;

T(y0) = 1;
T(y0) = 2
, T(y0) = 3.
c Sc = y + M T(y0)

K0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 0 1 0 1 1 0 1 0 1 0 1 1 1 0
1 0 0 0 1 0 0 0 0 0 0 1 1 0 1 0 0 0 1 1 0 1 1 1 0 1 1 1 1
0 1 0 0 0 1 0 0 0 0 1 0 1 0 1 0 1 1 1 1 0 1 0 0 0 1 1 1 1
K1 0 0 1 0 0 0 1 0 0 0 1 1 0 0 1 0 1 0 0 1 0 1 0 1 1 1 1 1 1
0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 0 1 0 1 0 0 1 0 1 0 0 1 1 1
1 1 1 0 0 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 1 1 0 1 0 1 0 1 1
1 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 0 0 1 0 1 1 0 1
1 1 0 0 1 1 0 0 0 0 0 0 1 0 1 0 0 1 1 1 0 1 1 0 0 1 1 1 2
1 0 1 0 1 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 1 1 1 1 1 1 1 2
1 0 0 1 1 0 0 1 0 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 1 2
K2 0 1 1 0 1 0 0 0 1 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 2
0 0 1 1 1 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 0 0 1 1 0 1 1 0 2
0 1 0 1 0 1 0 1 0 0 1 0 1 1 1 0 1 1 1 0 0 1 0 0 0 0 1 1 2
1 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 1 1 1 0 0 0 0 0 1 1 0 2
1 1 0 1 1 1 0 1 0 0 0 0 1 1 1 0 0 1 1 0 0 1 1 0 0 0 1 1 3
K3 0 1 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 0 3
5:
K0 ,
()
, y0 = (0,0,0,0,0,0) e
T(y0) = 0.
K1 6
1.
K2 7
2, .
K3 2
3.
5:

.
, ,
y .

,
,
.
, n = 6 t = 4 (s = 2),
, .

t = m , s = n m
r
, ,
, : k = 2s = 2nm
( k ).
, : 2n
k.
r
n

i 0 i
r
n
2 2
n nm

i 0 i

n
r
2 .
m

i 0 i

F -
,
.
, r
,
?

0
f1 j
f f1 j
2j 0
f2 j
Fe j F 1 .
f mj
0
e j- f mj

F, a 0

1 0 0 , ej,
0 1 0
e1 , e2 , , en . n-

0 0 1 j-
F.

, F
, (
n )
.

n- .
m F
n = 2m 1
,
F -
m
s = n m = 2m m 1 ;
M
k 2s 22 m1 .
m

y
.
1. Fy.
2. Fy = 0, y
.
3. Fy 0, - Fy
F.
3.1. , j- ,

j- , x = y + ej,
ej .
3.1. ,
.
.
6
F :
0 0 0 1 1 1 1
F 0 1 1 0 0 1 1 .
1 0 1 0 1 0 1
, :
y = 1101011,
:
y + (0,0,0,0,0,1,0) = (1,1,0,1,0,0,1).
1
1

0 0 0 1 1 1 1 0 1 Fy 6-

Fy 0 1 1 0 0 1 1 1 1 . F,
1 0 1 0 1 0 1 0 0 6-
.
1
1

,
,
n- .
R :

s 2m m 1 m
R 1 m .
n 2 1
m
2 1

m, n, s, k R
:

m n s k R
1 1 0 1 0
2 3 1 2 1/3 0.33
3 7 4 16 4/7 0.57
4 15 11 2 048 11/15 0.73
5 31 26 49 108 864 26/31 0.84

F 4 15,
2000 .
5 31,
50 .

,
n- .
.
4.
r n-
BSC,
F -
:
2r F
{0,1}m
GF(2).