I Definitions

Enloding

1 Sournecoding naturalredundancy
removing

2 Channelcoding introducing
controlled
redundancy for error connealment

and correction

Modulation transmission
of 2k bits over the channel faitoring

1 powerefficiency

2 frequency affiliency
I Information of discrete sourie

a Information Pi ofsymbol Xi
Probability

Ii ed Pi
b Entropic Average information that is iontained in source

bit symbol
H x E Ii
If Pild Pi H

c Decision content I maximum entropy

4 Requirements N equipropable symbol

number
y ofsymbols
Held n Ho

d Source redundancy

Ra Ho H IR bit symbol

rel Sourie redundancy

ra Ho H 100
Ho

TH Channel with memory

a
In Pi A

b Plxchlexj.xlle.nl Xi P Xj xi

If In P xj Xi
o P xi P Xj.xi P Xi
o Cond Probability P x at
exji
x hn exit P Xj til

If Plxjixil A

d Bayes Theory

Pli Pljtil Pljl Pliljl

Pli.gl
e stochastically incoherent

Pli.gl Phil Pij

Plilj Plj
o

f
Cond entropy

H Xi
Il P xylxi ld P Xj xi H bit symbol

I Capicity of a disirate source

Input Xi Zi
a Transmission
probability Output

fi Ie
o
Play Xi 1 P xi A
Ej

b Probability
of symbolerror Xi x Ej
Pe P Xi
InIggy
PC Xi
Ej

tatsachlich
ribertragener Informationsgehalt
c Transinformation TIX 21 actual transmitted information

I undismpted transmission TXZ Hx 4121

Receiver ride

T x 2 H z H Z x
IT bitsymbol

Effy Phil
H 21 1 P agita ed
with
Plajixil
Sender side

bit
T IX 2 H x H x 2 IT a symbol

with H x 121
gg gPl2j1 PCxil2j1ld Plxil2jl

Trans
information flow

T
It It bits
d Channel
capacity max possible transinformation
flow through th v rit
all possible souries
o C may T H x H x z To bits
my

tilt H Iz x
my

Requirement for lois free

transmission H'IT