Unit-

alAsSMAte
Date
Page
205a3
OMMUNICATION CANNEL
sepreed ran
Li4h minis
no-

SoUsce soue coding G hanne

Coding
dd ing
chenne
Teddndany fro eTros cotlso |

Sin
Soo e oclng Jchannel decs oing

chan nel: Cdiscvele)

ayheve y is the se a
(9lx) 92 xS +he inpt
oufbals

NoSe

P(4a) Plsalx) --

'
Plulx)= 29/) Pl9a}) -

A
thanna

iserele
roa

i
communicadon
rllk1) r[vz|as) - . P(yx

P(i ai)

channel can be ebresend ed tn the oym o/ a mat

tnown as Channel matix. a denod ed
Px).
by,
Ctonsi der iscrele ch apnel oith ossible inus and fossnble opus
Hhe_earve sboncin2 channel matix can be represededd as olioLss
9, ya yom

PLY/x) PlUaja,) Piss]a)-. Plumli)

PLui) aPS*) rl9a|x») r4sa). Plsm|aa)

aPL9an) P[yalxn) P[u3lan) flym]xm

4s that 2P|x) =
cle
ogi

Channel Diagram 1_a_piefoical epreseniafion discrele commcunicaton

ch annel
 alasSMate
Date

Page
-
P(9lx)

9a

Pgm)xo)
m
Sam au oulgoing
ransuben robabiübco -o a qiven inud node in the channel

diagram ic ales ni

lote a channel iagram -07 the

95
qiven channel mad x
x)
3
Peul
o6

o's]

3
'.
93
-
rdehe channed malx fen the otlesir Channe!.

93
o8
 clAsSMAte
Date

Page

The misingbrobab:theo can beounce he

the daed
bq o
all 4he outgoi brob abibe
92
rom he node
93
ene s thed Stum

0-3

P(9/x)= 24 -1 0-3

3
ointProbabtity madix
P( )= e9f2:) Pz)=Pxu/)rtsj)

The jint obabit

maix e/ achannel gves he morrmadion of the input and
the ubd fhe channel in a_coliechve ashion deroied by P 9)
The oint probaoiukp mafix can be obiained
roro the channel matn cusing

he-ollooIng velakor

Plxy)e Plylx)PCX).
The muthbustation isht madrix muikpucation ba os wise

|
Note
malhbucahon

3 ile the channel matrix and oint obaniu matixn e Jsli oing channes,

3 93

ASS ume the mpct_probabiiteo 0:8,o4,05

D 92 y3.
chonnel madi,
P(9Ix). os oS
6.4
 alasSmate
Date

Page

)Joini babky matwx, Px,9) = P(9/x).P(x)_

o Px/9): PlY)

Xo3

61 o-4 3
a
15

o-03
1s

2
oNode
o.o8

i)me sun
i) Sum q
ending

ef at
elemends
pb

demenls
i)
abi,
vow

colomn
4 P(x«9) gvec

o
=Pta)
Pcny)give the
the c

joint bobo ma
a* give
vepechve ole þrobabiy- for

P(W,9) =Pls)
ti) Soum al elerneds a Joint probabikhy
nahia i
alagr on

Z P(yj)=
Considela_channel th he olloninq Joint Probabiuky
O-OS

PCX) 2o-o5
3 o-0S

n
) Enfut_robabitihe Po).

6) o-stped probabriho Pt).

channel motw PCZlx).

4) channd diagom.

Ptxl

PCR) Z PU,Yi)

P a4) ot+0to.os as
Pa1) oo50o0:1+. o5
[o+o1+o1 +o-0s os
 clAsSMAtee
Date
Page-

Pla)=[o-101+0+o05o8s
PC-To-2s o25 o-a5 O-2s

Qus dprobab iibes

P()=

P(y3)
P92) (
Py) = (o1+o-oso+o1): o95
o1to1+ 01) o3
Co1+ o1) =O
PCI) = (ooS +1+ o-os+ oo): O25.
PCD=[o2s o3 o2 os3

channe ma ix P(i)
PLY/x)= Ptz,
Ptx)
)
oOS o-a5
o-0S 0-2s o s5
o. O-oS o-aS

o-os oas Is

ch annel malrix P(xIy)

P(x1)= Plx.)
PY)

O-OS
1s
o-o s 2 2ls

oros 3 2 S
O-oS
 alasSMAte

(P Date
Page
2:

Binarq Sqmmedic channel A ch annal is aaied

to be bino f ithaas

exadHawo inbds and wo oudbuds.

J
Abinar_channe s Scuic do be 6 noy Symmedric ehannel cesc) the

1ranscibon obobiuty om Same

Sten as that fo a to y
#had in P
is bs,
The Chaonel diaqram or a binoy Sqmmedaic channel given

-P

9
The channel madi

P(1X)=P 92.
(-P)

5 aide channe madier he chan ne given, consider the channd t beB

ihe channe maiix 9

Pyix)= 0-7 o3

wrte the channel mai o7 channel qiven

 alasSMAte
Date
Page

0.9

O2 V2

O-S

73 Y3

The ConTesbondi ng channe0_malix

P(Y/x)= o2 o-7

3 O-5

Comlle he ch annel madix and :0Tas channel diogram

9
o-8
channel mafrix PL/x) = 3a(o2) 6o3

o.2

0-2
93

Combtete he channe) malix ancd dhraw the channe0 diagem

9
Channne maix P(x) =*O 0-2 06
2 O-4
lo3
o5 o
4 0-4
13

.
 elassmate
Date

Page

Chona Eniropies

Tnput &yslem otpal: Y

H)(opat entnfy|
cchonnel) 4cy) ouput eopy

by H): 1 the average amounf oun cetai nity f the inb

Lnou

Side
Endro

o he ahannel denotebp Hcx) .1 can be calulade. as folows

HCx) Pli) Jo
Pi)

outpu entzopy RY}_ Ourput etoby hev s the measune of uncertaunity

af the eutpet side e #he channel I* epreecl asfollos

HLy)
J7
P) Aqa b'isabo

Joint Ento_H(x) It* dejinecd as the averoe am oun oohrse uhen

One then- rom the set

Spmbsls is sed ido the channet ond one e
channe. tt Can be
mSymbols rom he set y is ecieved hom
the
he
calcad ed as olto

HCXY)E P,) Aoqa

P(ai, 95)
bi symb

icordih on_nloby H(y/x) Defined as average amount an cerfasni a

fhe oufps Side he channelgen he infomahon abaus the inbed to
the channd

HOY/x)
PiIx)

condihonal Evi2ok RiNIy This 1s the mea sne on cerfainiq at the ip

sde l the ehannel þroided the ausput q he chane0is knouun. t9
PI, 9i) dop br symb

consideL the fotonoinq channel

 elASSMAte
Date

Page

P(x9) O-0s O8 O-OS

o1o
oS

)
Ho.
Ax, .
The inau_þrobabiuti eocan be abtai ned by addinq al emens O re in he
joint_obabiüky_ max given

Pu)= 0-65+0t 02+ O.oS d:3

PCaa) o+ot01+0
O

PCZn) Oos+o-6S+0 o2
PCx)= 0-2 03 o-a].

Hence

Rex) .:3 seqa lo-3o o Vo-2t o-3 doa o3 0 ceYoa.

9H bi kleymbol

Adld. etermends eolum

P oise to get outpd_oba6iuie
P(y) O PL4)= o1s P93) o's
PA) o:2s., Plv)=[oi ois o5a.
Hentt,

743 btisSqmb
.'
 Dafe
Page

e tnoc fha

PCRi,)

O-20

ooSS OS

3-1219 bila sypmb

B122 biis Symb

HYI
e knos thod

HUIx) ZI P2i,i)Joq
PlY/x)

P(YIx)= Plx.)
Plx)

PCx) o-9 0-2 Os

PLY/x) =O-0Ss O20 0:3

0-10

O-20 O-10
o3
O-0S O-oS o-2

6 al3

Pt9

2
HY/x) o-65 doga
o-a0Dga
 elasSMAte
Date

Page. -C)
H (Y/x) ISIbiks (sqmb

Pli,9i) Loga
P(19)
PCxIY) Plx,v)
PY).

PL9) Co. O1S os -2SJ

.
o-oS
PXIV)= o2 O-oS

O ala.

o.2 als
O.oS
12
IS o2

H(XIY)

-379 bits/symb

NOTE Hx Y)= H(XIY) +HY)

H(Yx) + H(x)

H VIx) x,9) Hl).

HIy) Hx¥)- HOY)

LO.
O-OS
o-LS

01 0-os

PCx) a 0-35 o:1S

9556itdymb
.
,
.
 clasSMAte
Date
Page

bessym Plu)0-2s.
o-3 0-95
o
98S4 besmb

HCx. y) rCai,9) Aoga

Plai

3-0464H bikslsymb

isynl
Pl)
3-06h- 9Sh

MNY) 22i,91) Joqs /P(n/y) bts y

0-3
PCX)=
O-IO

o-OS

O-0S

PLx) [oas o o1S -LS

o6G 6ise symb

PCY) -25 0-35s o-2

 alaSSMAte
Date

Page

P(i,91) sO2
Pi,9D
666 bIfsymb

66-06 6
0.6biissymb

66- 8sT
o-809
bits |symb

P(yIX)= a 3
0-3
OS

PCxIy) PCY/x). Ptx)

POw)= a. 4J.

Pxy) S
0.2 0.5 |xO.25 0-01S O-os a/25
O.4 o.s X02S O-02S 12S

Py)- [o3 3 0H

Hc*- PCai) L09 Ptai) 5bit cmbus

)Hey= Z
Pl«) to 1ptu:)E $ bit/smbsu

TAT91)
997 biis/symb

Plai,i)oga
 Dote

wnemasir

P(oisi)

B.
o6 o-2 O-3
O-6

o-a -G

P(X )=0a
0-2
o-6
0-3
02 | b
da -3
hs
ls

2
PI=|13 1

S3496 bils/symb

Hc) Zru) Loqa'pezi)= 3849 bHs/synb

Pai,9i) soa a95591 bis/symo

PCal,gi)

1-33 biis lyns

 alasSMAte
Date

Page

o-S

33 S 93
pw)0-,0:3,

OS
PVIx) oS oS
OS

OS
Xo 3 O.3

P(x) o-S Xo3 O-35

O.5Xo DS O

Pc)= 3s os
R) 2 r(ai) toq Jeca*) 295 S bts lymb

H= PH)109 P4)=5n biissqs

Py)

PVIx)_
vi j
P(is ) Ao92 pi, )=
0-495bis|sqb

snb
biis

e(xo.a 0:3 0a
o13

.
 elAsSMAte
Date

Page

PCXy)=_|' O
O
314 xO.3 0-04S

XO-2 e-e'lis Is

xo-2 O.2

rY=31s |30
HC) Z ¢ai) ADq rta) 24 biis synb

) HV) 2yi)LDq(Pcyj) = '8a13h biis lsywb

3165 bids synb

Paij})

0.q43 bits |synub

P(lyi)

Mudaal Tnformahg T, v)
mdal omab0n
ihe the channel isgiven by

Jx.) Ha)-Ht/) bil/eyn

HY) H(Y/x)bitsspuooir
cwe can no desne fhe averape Tate o informabon raunsimission oe n
Channel as

bps.
Cuhere 36 thebuad rade o Smbol ate.
capacrk
Righes the Re beHer the mtcucel a6 channe

Proberheo nudeal Infoomakon

The mautaal ntormakon 4 chanhel is Spnmesc oith. especd ifs.

argaements
 elasSMAte
Date

Page

Proo

TOR )=HX)-H (x/y)_

PCal)
lnalb the is derm by P(yi/i) oe oiged

ai) )
PCA

PRi, ) 1o
PCais9)

Pi,2 41) seqa Tptz)

pai9)
Z Pii) Loga PCai/ 9i)
Pla)

loe knao 4has P(il4i)= P(ais i)

P)

TxY)=

8y inlerehanging the ovdets. and a7quments

I(x. v)

Mutrual infomahon is aloays non-negakve

ie
CDe krocs that
Pa:).Pr
 clASSMAte
Date

Page

Aoge

ce Joge

AIso we knouw thal

PCRi

nla
) AoePai

I-2.
9)_|

Let
2=PCai) PCY)_
PCi 9)

Hen nPcai)_ ]2
Leo) pr9)J
_ PCai) PU)_

PL%, 9i)

Hena,

Pai)Ptup PCi 9)J

mauhbring both sides y ogae

Ttx.) 7
TO.7)eqef-x
Ttx Y)20

The matualintormaion a channel may be enpo essed in 1ems ef

he enf the channel outped as otl

ICx Y)=
H(Y)-HcY/»)

Pehe mutua informain achannel is veladed to the joint entro

and input and autput entopres b theo ouwing relaho.
 elasSMAte
Date
Page.

xV) tlx/) +tC)

HCx.V)-Ht)
I CX.v) HCx)> HY)- H,Y)
RCX

HC

Hx Y)
Find the mudual informahon o the olioing binory channel

92

Let the inpud robabiuh oo be

PlvIx) o-8
o3 o-1

I%y) H(X)- H(xIy)_

= Htv)- HLYlx) PY), Plxiy) , RYl

Pl )
T(xY)= H(Y)- HtY/x)

P(Y/x) P(x

0.8 x0-6 0-12

3 .ao

P
D:3 012 o
28

Ply)-o

1
 alASSMAte
Date

Page

Jy)E_o:18$ 6ik Sqmb

16 Find the averagerade inormakan fran pm i o the foloing

chaanel
sh

l
PCa)0-s PC)= ) 73 1000
13

sb|se

PCYIX
12

PlX Y)

312
o-aS o.as

PCNV) /s

PCY) )s

HY) 5306 bi4s y

HVlx)= 090s brth /»n

(xy)O.66564 bis
R47 6ss64 byg
 elassmate

Date
Page

Fnd the
aveae 7afe in/omahon transmissiOn ave he fsl oDing bihor

erasene chann

Px) Px)=[a aand 7s oo S mb

Jorn vob ubicy mix,

13
P(x.V)z
12

4 sl12

HCY s54s 6iissymbols

HUY/x)= 0-184l 6r1s|Symb

Txv) o.1fo4 biLssynb

Rtr .1104xiooo 11o.4 6s

16.
Find he avevage ale
0-2
informah on dransmi sei on
0-
r the tolloping chanoe

PxY)| o3

Rs O6 sym sea

we thou Tlx.v) = HV) H(/x)

Ply)=0-h oh 2 PCx O4 o o

PYx) P(xY)
Pex)
 alassmAte
Date

Page

Lo-
02

oS O-25

PlyIx) -aS
o-s

HC)= :3/72 biissqs

Tx Y)= 0.397h 6rds/sq eb

REy 91hi bps

o-os o

POY) s 100 b kymo

-2
a6S O-OS

ICKy)HY)-HlYI)

oS
o. P(x) a 6-3 0-
213
HUY=(6644biis|sqnb

RCYN)= lis
13

R(YI)= 1303 6ih {n

 clasSMAte
Date

Page

P
Fiod
eR
7S Soo sqmo 5e
0.2

o 93

XOH
Y

o3
3.
o.3

PCY I =
o 4 o

o-6

3lx Y)
H(v)-HYlx)

P(xY)=E O-oS o 034S O.o 315

ooI25 O-0875 .o35
IS S O-HS

P)= aa]s0

HYD HIaS bits sy mb

Hx)= 14596brbsymb

con Gidey oltouwing Hwo channels conneded in om o cascade connechon.

o-3

POk) og 0-2 PC/Y)= os oS

os o9
Consider he input pobabibutico as o 2
Find ndual informah on _fs channel ix Y).

8) 4he chahnel OuPomocastade channes

Draw

Find the mnutual inlormahon cascace ahannel

7 T v)#)-HYlx),
 elasSMAte
Date

Page

PCxY PC):P0)
9 9293
o-3

o-8 xOS

oa
O-Ho Olo
O-lo

PY) o 61 0a

H) 35h6biis Ssq mb

0:8a53 6i
/s9ab_

TCX,Y) =O5aq2 bits|symbos

O 2
3 23.
o3

pl2/x) : (A,79
channeli

*9 * Ci y3 2)
erzsla) 3) +a9s za).

O.81t
 alAsSMAte
Date
H
Page

oS6 to- O-66y

o-3Htlz o3

P(Z41): o-3.

72
os&
o-6G
Pz/x) =

rx,z)= Hlz)- Hlz/x).

PCXZ) PZ/x) pPx

o52 o.t8xo3 O LS6

o-6G
-344OS o33
o3 1 o.2

o 954
Pte)-s*C

0.93S3 bits/9

I CX2) H(2) - Hz/x) o9938-o.433 2o-os SH 6tfsSqmb

Considea a cascade Cbnnecfen ohannels cwhore Chahnel maurices ore aas

low
 clASSMALte
Date

Page

Plxlo: o-q Palv) 1

o9

to ocCu Find
A5some that the inpeulo ave ealy ke
I CY) and rix,z).

we Know fhat t(x,y)= H)-HCY/x).

PO PClx) PCx)

os
-1

HS O-oS
PC4)
O-ts

PY)=os o-s]

Vivj
Gh6 899 bik syms
 alassMAtee
Date
Page.

P2/x )(291-Z)+Cai-9>z) =0.1+ 0-o o82

PLZ (
PZ|aI)= (Ai>92+z + (aI
): -9I+Z)+ (*1>yiz) =o-Ixo4): °18
Plzal)= (23 92 za) + (32> yIz)=o9
I+2

xo-9+o1xoI
= o-txo.4+ o.9xo o:18

0.82

P(z/x) =-82 013

I Cx z)
Hence Hz)-Hz/x)

Plz)= PLZ/x). Plx)

= 2
8 oO8 Xo.s
O-8a]xos o.H

biis /ymo

0. 800 bils/semb

:
I(Yz)= Ha)- H(zlx)
1- 0.6 8oo

. o-31a22 biis synt

 alAsSMAte
Date

Page

Sheaal channels

Smmetric channels A channel is saicl to be mmetrcaS He00

the chanael maliane_fevmutaions o each other and So 1be colomng

mmeno ee TO o he channel matix

edahen o 0fhea ro and olumn Som eo fo all eolt

oelemini hc changel: A_chanoel is i to be dede vninishe P)Ps

_entnes eifhe or for a iand j
The oupd can bedetemi oed

9
3 93

ossless channels A channe! is Saiel to be cosrlessf the nb an ée

delermined the outbut

23

Noise less

dosstess.
channels A channel is noiselec 4 s both lelermini shcs an
 elasSMAte
Date

Page

91

9
93

Channe cabacil ahas o a ahannel is delned as the maximom possble

vade o iform aion Tansmission Oveg the ehannel, e,.
Ca max t Max Ty) 3
asSuning 7s being
Hen

Symmelric channel

cens:der an '7 inþus s outbut channet cshose channel matix can be vepreserled
las follouws,

PS

PYIx) Ph P

Ps P P
Pa

we know thcd Yi/xi)= 1_

AS
ad_robabiüho ae indebondent e o, P(Y/*i)can be Simby eated ac

|we

AlSo
Kno fhc?

HCYlx)=
T (x,Y)=

ZP*3D
H(1)
- HCY/x)

sega(_"Zevuja
 elassmate
Date.

Page.

H(Wx)=ZALqa

ohnahon the abau Channal is 9ven b

C Max TCA]="RHC)- HVA)
max HCy)-h)_

C Max HO-h-
Cz{ls-h) biis /sgeb
7s is_given

lc:(cgas-h)7s bps-|

he abal binoy Smmeric channe is,

c (3 h) -
C -h)bitssyn
c- CI-h)7s biislsynb

Find

Thesee
the channel

Plx)=

channe
| cabaity for the

o6
follooing channel,

epuk sgien b,

utpad ines s3.

o6doqo Yos+0gtwgn lo-3+0Ido9 "l0*)

 elasSMAte
Date

Page-

0g, 3-'895

.
PLYIA)=
O-a

2
81

O.
bits/Sgn

o-1 TS1o0o sgmb|sec.

No colemns St
Channe cabau c:to,S- 5)x

tca-h) x (ooc

h Bq
o.1 s07l67+ o. L0qs lo o-2cg o-2

1SC btks/sqmb

ro - i1S6)xio0o

8h35 bisStb

Find the Channe Capat

TS LOOe ymb |Sec.
2

13 93
.
 elasSMAte
Date

Page-

C(soqat-h)z

h bite|sy6

S. Pnd or in a iymmeic
he ehannel copaceih choanne

S 100o Synbsec

Channe C a-h)*s
C1-h)7s

o:3
PCYI)

h', 1 2oq lo-1 + o-3 xoq 103

z 0.4 8129 bis/sqb

C -h )»s

-o §sI2)1ooo

7-4o9 pS

Zhout oa bing Erasore channe CBeC)

 elassmate
Date

same s af probabiüh wn ne oIumn) Page

gymmetic Som shouwd be om Cumni ea.CH 9mvmefrc

The channe matix

PCYIx) ==

Les asSumehe inpas probabitho be and -4).

we krn ocw thad

C Max

ICx.Y) = HCV)- H(x)

HCX) H(x/).

Cwe can vse the exbresSi on Sx;) HY) Hly/x) ao

Hx)has informch on

aboud the symb as coet thogh not conSidesed ad the eoeve

+he mut uad nformahon,
here ove the olloing exession used t inc
I(x,y) = HCx) Ha/y)
we knouw thad

TP)
Also

POx.v) P(Y/x). P¢s)

PA

Pl

PCxY) P(x/Y) P)
PLxY) PxY)
PCY)
 Date.

Page

14
Pxy)- P4

2t-d) PCAD
P

PC)

14 pk+ 1-) saq [o-

H/Y) 11 z+ l-4)i

TX Y)= htx)-Htxl v)

Ht)- Htx)
HD1-9)

Ihe channel capoc

c Max r(x,

sC P
max

Prs bps
Hx)}

.Fnd the channel capauty_jo the oning bin

oy eras ore chonhe,

Ao gven thad Y= 5oo Smb[sec.

6
 alassMAate
Date

Page

Cr p: rs_
o6x5oo
300bps

Channel ai osing MoTogal melhod:

cons)der the olio oi ng channel

P 9
PI2

P21

Paz
Steb i uide the channel mahix

P/x) P Piz

Pa P

tep Formalafe thefotiaing qahons

2
PIa

Paz Pai
aqa Fa
+ Paa Lqa Paa

siep6* solve o
6ep channel apai

Fnd the capacity the jollowing channel using urgos method

-3
O-6
PLYI
0-3

ote channel madix is 9tven

 elassmAte
Date
Page

Gkeba Foimuleute epuobans

O-6

o3 oILoqa O-+ 0:6 Ao 6+ °3eDgao 3

o-3 L 3 aoga o-3 +-i Loga
12
*l +06 Aogao

o6 +6-3 2+ 0-1 3-12954

s4ep: chanast cap

ca[a'+ a*+]
0-a9956 boissnas

&.

0-3

o
8 o-6 a -6+01 Aogao- + 1 doqa oi

3
1 o aJ
2 toq20 2 -jLo®1ro) ioqa0:1

o-8 + o143+O193 = 02192

o.2

o-1
+o1914 o-1Q37 I56T
+O3 2+0-6Q3' -29s :
aa -1-aIsoH6

C o.hs2cbits[sb
 ( classmate
Date

Page

9.

9
93

channee maizx

PCYIx)

a9 O2
O-S
og>0S
o2 o-cLa3 o-a
sgao-a o.2 A0Rao2 o-6 s.
Q10'S .t 09Q1to-H Q32 - I4a3IS

-136076

O2 +02 aa +o6 93 --3 os

Shanmas Harhey las hannon Rottey iaw Nader thad the apaiy As4
channelS ien

Cuhere 8:s the channel bando:dh

SSiqnal pouel.
N NOi poae
 alasSMAte
Date
Page.

Kepesentimq N in tems Noise_power chaedval densit

N- No6 is given b
C

NoO No

No

N 1

Leks consi dey the case ohere bandwidh is ten oing to nhnity
NoB/s.
NOB

S_
No
oqatS No
No8/s

Lef
S
No G

As 8 o 2
Co im cg (l+z)

m Cl- 2) e.

Co S/No Log

Coo H