 p11 , p12 ,.......

, p1s  Entropía apriori de A Propiedades de la información mutua

P   AB     r
1
H (A | b j ck )   A P(a | b j c k ) log
1
 H ( A)   p(a i ) log
i 1 p(a i ) P(a | b j ck )
 pr1 , p r 2 ,......., prs 
Entropía a posteriori de A dado bj
 A   p(a1 ), p(a 2 ),......., p(a r ) s
1 H (A | B)   B P(b j )H (A | b j )
H ( A | b j )   p(a | b j ) log
 A   AB   B   p(b1 ), p(b2 ),......., p(br )
H (A | B,C)   B,C P(b,c)H (A | b,c)
j 1 p(a | b j )
Equivocación de A respecto a B
Extensión de orden n de un canal
H ( A | B)  b p(b j ) H ( A | b j ) I (A;B,C)  H(A)  H(A | B,C)
An  1 ,  2 ,...,  r n  j
I (A; B, C,..., D) 
Generalización 1er teorema de Shannon

B  1 ,  2 ,...,  sn
n
 H ( A | b j )   p(ai |b j )lij  L j
I(A; B)  I(A; B, C),...,  I(A; D | B, C,...)
A
P(a | b, c)
 11 12 ... 1 s n  L I (A; B, C)   A, B ,C P(a, b, c) log
 : : ... :  lim n  H ( A | B) P(a)

x  n
 : :  P(a, b, c)

: ...
 I (A; B, C)   A, B ,C P(a, b, c) log
 r n 1  r n 2 ...  r n sn  Información mutua entre A y B P(a) P(b, c)
I(A;B)  H ( A)  H (A | B)
 ij  P( p j |  i )
H (A)  H(A | B)  H(A | B,C)
Entropia Afin o conjunta entre A y B
Cuadrado de Kronecker (Tensorial) en BSC H (A, B)  H ( A)  H (B)  I (A;B)
con error p I (A; B)  H(A)  H(A | B)
Canales en serie I(A;C | B)  H(A | B)  H(A | B, C)
 p p p p  AB * BC    AC 
p  p 
  p p   p p   I (A, B)  I(A,C) C  max I ( A; B)
 A
  Solo para canales en serie:
 p p p p
P(a i | b j ,ck )  P(a i | b j )
p  p 
  p p   p p   P(a | b, c)
 BC
P(b, c) A P(a | b, c) log
P(a | c)
0
Bayes
H(A | C)  H (A | B)
p(b j |ai ) p(a i )
p(ai |b j ) 
p(b j )