DIGTAL MODOLATION (OprNG hecy

ngormchen SeuCe
Censidder
eute eTeA talk, , dchionay et
Thene ese
ang uogg elt
Caled apha e

Gital Cemuni Cahon

Ci) Binary M 2
aqkeisalpabe 1 (o er 1)
Ch) ucter nonu M4 2 Coo, ol, to, )

exa M 16 (o000

Mahenmadutal uodel Semte

S CSe S S-

S Samp SRace
éipaiment7 Rmission f Sqmbtls

S Kanden
OutCeme|event7 Vaiable
Aho bobikty he Conesondug event

Y P O Swe OY no q No infomation

Tnws
When Te Out Cenme lem SSion Sonce independet
durina SuceysiVe eYves Memonyloss Scne
dis tretw ymbok hen DCree Memonloss
1 lne

Soue
inNee y opotovas
e Rrorooltu C Cceuande
LMpathon
Set nhovmatio

CS) log f )

Fvofei e o S Inpiwmatien
CSv) o. Ao
p oR41
(3) 1 ) 2 1G) bov P 7
Rk
ICS Si log R) oY DuCelo Menen los Seurce

aAmalaen gaimed Aro tembnod ocCusante

S
ualu egri th m bak 2

Enory
Mean vaue 1 CSv) oves ttue Sounce apha bet S
ONeua Lutp Ceulet |mmhet pi ke heunte

E 1 Cs) J
Fov a Di nary Meovy los cuce ouLtn

Yo, henectively Enhopy ven y

Po eg, Po- P, Jog, P,

- Ylea --Po) Loa, C-1)

Fropevtes O Enropy
0 HS) log, kK ' aphabe me Boune

H CS) l , t K) P Ykov au
mbos
S3 o ( )O ev
'
 Swwem Cal about
aumens
Cie)
SA
So
Sc
SP,y
ten HCSA) HCSp)h (S)
4 CS)
Ro, q' hhmbols

HCS) Plog : dea

P1 B axiem

Ceido
oq - H CS). - l s Ck)
Po3, P )

Los,
N
n (2) 2 - hopey -7

uAun (R) z 1-
Pummu ve
uuhpjwg

7 lo,e [1-4.0%)

o,
o

heck
P6.)
ate whumehen R H Y7
MEsage pmbe/T
uut 7
sm (ec
H7 uluepy

R sc

Sence eShi caency

s HCS)
HCS)ma
Rodndancy Kpg - s

Numen (al C
S.CSoSi) P toR

S4
find Ci 1S) ti) As) ii) R wteu tmbas
o ms h 5 ms
Cuco eNes
3. 32 x log C54)= 5.T by
1 () o, C54) 10

3,32x kog 5A/53) = o.621 b i

HCS) 332 log4)

53 x3 32 o
On 133 bit messae mba
n aeaR 5.
eauuae
S3 53
54x 10)+ (54x5) (xio)+ 5 25 m Aec 246

ppn bet Yate 54 65 5 tm (ec S4m= 66 6l6

825 Xlo5

Jno a e = H 65 45x o133 8.4 bits|sc

CS Lo,t K) 46 Yk
 Memory l e s Bourde
DBCete
ERtenko S k sqmkos

bowile emt Aymbe

n bi
nh ex\ENM on Greupung
om bena
ke O Cad OY Heaa uslm. du ved

Let tu
PYobaihes be duned by

P RPa
e

2 P
o d

E en ded Souy Ce

bouse eurce
no mbots

ne m bols uped togelha

ie ta
Ci.e) S so So, Sosi, SSSo, S1S
Aus ete emOry uss Une Probabu hes
Sour Ce
5in ce
ndepeudeat
Bet S
umbot
OCCeuRane ven by
contou
n m bos 002

PCo R
2
and eah Anu Si can be So a 2
o2
Ps), b2
P CoR) PSu)

Lheckk

P6)
EW W SW
 umenk

about

P.

Coide Bouute
S So So, SosS Sos2, S,so, 9,9,S S2S, S2S1, S22f

3 u 0s

H (S) H6) P)

3bi
*4CS)= n .tCS)
3 2x5)
Tws pi Veifud

SovRCE CoDIN 6 THEOREM

b Bwon n l n (e

Sen SEntodu
uCunkscd PMoP
Qplitude
wwCh Pwduces lale wOvds
encoden k

pY m @wiqueay deco dab

inbin
'
www Po OcCun
K t bois

P Po
wtn P(S)- P
Poani oulc Mmot Sk
Coider Cade wia le
a2

ieuna qmeot
P n Me Valne u m n
 Codnn thun

mum

euwdn u fhaent
ben u

Scnte Codn hewreM

heit% o det Bun nat
mun
dus cnut
Mewonlus n Le
ONealoR Enrop HCS)
Code word
ung L Y
am emce ucodua
Semce ou
codun
auertak code woro
L Hc) medeas close seuqun can be
HCs) P i le
Ceakmg ne exen Mon
Codin egi cvu
2 min

7n i en ncoder
Code woa Provide a Code wbd
A
wb
ual Mae omce mne
Pre ix Cedune
Let S
s, S S- k
K smbot
entoded ofp
R
Cawy0 2oded otp)) b
uphputed as
bRXo Ym- 1 7 Code oond
kagu m
(usy
ukiicodo
04a Seunce Code
Ucue decodable
Aveiagg louatn E Ce. Can e
ashaily mdo equal b in
NO Cod werd
End Co4 Cbde w0Td k
a 0e tode
uod
ehdnS., b, nepviiaole
eniible nbtautanews
tstauua to de
as beAce
BeACe ep,
ucodes efp aud tode udds
wnd
oVANAN. t o Ii wwe Jm
Caued kue Aekia (11X,X =bu le4
DA tode ookd
XnC Me Mla
nequi i
Duaete Meman es Channel
PSPoP
Encoded fr has b
ouud tode by 2 bo, br bh-
an m-1
Code uoord
.)enaun
h tode
bx a fiukix (ode

Jne
KI Mc miUan 1 n2quau h
2 Celeod not

Cot uend Jenqiu

Con vege DMomle. and uu
Lf S
milan tnegualhy he.
Me
us &oun ce ouus akr

Can be Conshu eled

PRehix Cde uoOTd lengt
Code
Wesage
he enoopy S e
I HCS) H4 L4 H)+
i e k i x tede bounde
uen

k:o
Mc illau nequalun

K-1 a s ed
2 7** &2 inequality u
k-O
Mi code Cam cesmcee

k-
Code uevd ngn L l k Pe

und k-o
t geun ce
nket K
uCS)
KO
Ak 2
Ko
L HS)

n orden exlenion Ceended tode)

Lee e enleihu Cocde
loCKs ok nkmbels.
HCSH ns)
bui n d e enendad
tode oord engtn Code

then
hen code Pitix tode,

HCS Ln < 1+ HCSh)

nH Ln n HS) +1
 HCS)
07
Lods WCTd tee

bu no

Price
doodung tn pl rihy
The
Jestes non HCS)

escen y
3
nHS) - H)
Loln
Profoticmal

esngn
Put Codd
The
bas o c c u o n ce

Prorao
C. 1 instad ik
do Jt
undtr kan drvg Simusy
x,Y,z
Scun ouphae n Englush S SA,9,
Ona oded os Potnen n alphaket biny om
eus der
iO0b, lidol,
wOTd A BC b 1, 1o, 11, 1Olo
wend enah Ln) 3 5,5, 5
Ccd 1,22
Code wCTd 25
o do
wevd lerth
10)+1(2)+ |(2)
(Ho)+ 1.L)+1Ck2)
L
Aveaag Code w o d

nx K- bts |Beua tmbR

k
ood of type
Codo
k cuns
yp
noime e Cods uwcr4 Code
e
he Cede oo1doh tupeK
wOido type k Can Ottn
PoSble no. hmes ) 1) wovd lena
a woTd
n Kmioe
u ine
p. whar u toua nftunte
Simllaby do to ABB, AAB and
 m Cas about ens

umnCn proe eg tada

Codin bes
7Suoe Cplo alo called as Cempact Codl s SSs Ss
oSS i s o.S O-
Prbvides
OPhimum
unimum Aedumdancy
malle avesag tode
uwwrd Lnah foY a ven keure
hinuPle
ce
Teplace u bouce
m mbok loio111
wt Pob. eoctusante bots wit Simplu Cneto
Cede I90
Agorthm o 6ojlolol lo1
ATYa souace 6qmbots decnoasunq order Prbabiity
AUOSmbels lewest Pvebabiky as osi Sp'Hi
ah
Combine uoo eme imie w t shcs
mbots iuto achuel
kne
he ur
Suun Prohoblties .aund uplace
Ogmal
isr
Reduncuen n Sito y1
Pthobiha
bue tu a
KCfeat PYocedune tunal
kwO wwith

ne code 2aCh sounce mbot mav

obam
uwtttn
backwand dme ctien
1 he
Amediale ta4e3
w u
Stepi s tep2 Sep3
Eample
Sy

2
O.2

A
nd
Cininal S

,So
So
O2
O21
o-1 sSa
S O.2

O:2
O.T
N N
nn Phn Cal about aNumen

Vucete
Momory les hannes.
e taiked ab Leu ce han nel

SouCe
Sene vati on An bmahon

Huffmamcode lutte Adun dan ty

Chann el
hansml 6\on a Infor mahn
Ducvete Mwony Les hannel
stathhcal Model

Channal>%
seae AIP to Channel Hco RMmota
Tzea
j e Søun
T z e d by
mbot
seunte

7 Enhppy of 9mbd Roed

Randem Vaiable
by e TedeLVez
Channel HCY) avro
P
Random Vavioble
Yoisj versuEn
Channel duxke aYe ile ip

w 2
y ae
dones fondmg to X
alphahe8
dis Cole
Cnan AA
i te kie ,V+ alpihabet s.
wmch Can be
SmboS
o X
esent outpu mbat
Membr e s

Chaunel
e
dupeuds

Mvdel
Malhe ruahcal
o 1, i-f
-
alphabe
X tm
alphab} 3
V-eutp
- P(&/x-7j)
yebabiuky.
?( *k xj)
Tamsitienm

PCk/x j) 2 .
 Can x intachagably ukkA
Note X, x y
2epAenntalen
han nee
Mate mahcal
Chahne Wthi 1
R7o)
PCo)
Pol
PC P-

Conec Necephon
when PC)P(/4)
yecepho
P/) e YOT
Stk
AsD
Manqunal P
k-
K-0

P CuoMo) P 10

Pol) (-hj
Coyditenal Prep .o

JK. =
PC /1) P/1),Coweck ecephen
Oend Peb.
emor in eapRim
34k PC9k/xj) 9,
Ros Channel P ixed

Coloum namel o{p hred

Te Masanal Pvobeuheihy
K-

Cie)j- b to j-I.
Pyopeun e nanner
Thu ue kundament

he PYrolooublih eon we Can wTI

New
pA) nd P4)- P,4)
R Dp P9/x)
PC Py)
P 97 PC).R)
P
PCl
 about

Maaq al du buluen
PC

P()-PCA) fo ko, \,k-i

P KS
T-k
no. a Ap - ho: of oP Ci..
fyobabiuy a t e

C.e PC
Re

KAJ

Co vect Cecep tion e-e

Pvohobilry
disri lovien
Mag ma p

PC)
&mbols,
Caea ous APriovpbobithes et op

HANNEL BS)
BINARY yMmETKiC
D u t e MenapryK annel
SRaciol c a o

K2
Trauwitien Protay dioaRam

10

-P

Cod a bence p alphae

AVn
1Cs):oo -,lo
HC

Sila ev Qu elp alphabet

HCa Log,
o
(h)
New Lond iona eukepy. ti.e) te avenage , Ruho anjallade
roonn eacn o mbot e abou
put
PC7 tog, for a k= 0,
HC
x k EC)
QNeLageNalue Hly)
Pi/yRY)

i (w/y-
P(9)
j0
- HC4)
-
 aA .ins Ains

$imlla PCI)
KO J:o
LPC
Ci Equvocathem
abo ut he Rx
ti)
MROwMe In The Channel
used to
Eonow
Calculate Pe
Ci ohen Pro hat t su be Rxes as
o

Mutua Lnkoi mahien

H CX)
No f n Can
Antoyatien Pvoviekod
aNeao eNe tmbet(Sounce Pr Uieu)
toY) No O bre m an aVeIOR
ne
P mbe (creodas Pr
aviu0)
HCX/) NO'. o n kornvntion Providoda Obeut

sea each ue
mbel Sounceby o
Ob heaay e mb 9) oNeaage
Af ecerve Tue Canned
Cmount

Lost annol
meunt hdmahen
ad A ALOPavn
age s H /y)
o a rmbet
esRecieved q edung mds Tj end Rxy
nt inho
The
we Mulual kp
aed
1[xy HCX)- H(y)

Cie)
TCxyJ ILY: x]
Ciy S y m m e n e

H H )
J Y) =
O
J0

P
 0A
(6(d
( dZ
6xad
(
T
Q- O=|
[hd hot (6'1)d
T(hxs
Hrdo - 0
6 )d
O-
T6z]d
ah/z)d
Cd7 bo (xd
6K)d
 k-

P)e
Let P

un ) 21-1
dnP(H)". P 7 ) -P(xj) ply)
POPY) PCa PCyv)
9)
/RCy) u P
PORPCy

e
K-
PX )PC

1(xY] leqe
PCj).Py)

PCjPJE)
P(x,)

PCxy)

- Pcy) -I
JFo Ko

Px)2 f9)-

(9/)
HC)-H
10y) (K/y)
HOX)- H
- H CY/)
I CxY) (V) HCY)
H
1 ; C x , Y )
1(y:)
= HY)-
yPvpeatby
 N

F w
 2

W w
1W W
Rate Inovmatien nanne capacuhy
Centams H CK) bi
Sownce Biumbot

when channe aCcepts s Mescae mbets e

Y HC) bit ees hansmiHe through t

Channe

Rate e yansMsian YHC)

1

Chanel SDime ino LoSt

ne
amt hanne CaRaied tne

notng mulial Ano Tnmaten

unsws ion nbo

Qnbo
A c a Rate b
Chann
el

sita drans m s ion Posi ble

The maiMum ate

a Channel thannel ApaCaty

Mar Re
max 1 o).s i /sec
ShtsMnt
Lodug
pacy Theorem
Chann Can ba wsed
CAanw
ASSUm4
R C
Tc sednds amsm I 1 Cn
uko
Yate
ohen The a Ceduua
fhrne
exisls
Sate ment
hen
aile pide) ARaunamWa "b
Cmax
wCh enables
even n e
echnique as Pasible
ao LSAOA
usut po
Bho w he Channd
nee
uen e
enevs annet Con hoeled
C en h
2 Re lnOu
Nluable TxLan
by y Codu Technuque Omy enoY k not Pon LAa
 .ebnndon
trockie
KE 1Do
e

Wa1-w,
WPY,)=
Sromec Chonn 1)PG,)-
Cox-

Two Olr mels C 0,1 )

ole
nme me
PC4l0)- P(o11)P

TyOitien dupan

-P

Channe Makn 1-P

P%/ka) 1-P
fCYA)4C%/2)
n h s o p y E q u v o c a t i c n

Chec

I Cx)

1
Prebabi
ihy
TO t a l
af = Fw+Pu
Theorern

P C % / z , ) . P C Z )

Pw+Pu
HCYh Fo)
Pu+P
+ Pa)=
9o/%o). PC41/z)
PC P C / z ) PCaD
PCY)- PCTo)+
PC9/1).

Py)
P t ) Jn, Furta
(
HCY