Lecture 1

Digital Notes Allowed

Quiz 1->
Compulsory
Quiz 293 >
- Best 1 of 2

Assignment/project >
-
15 %
Discrete MemoryLess Source
Lecture 2
Lecture 4
① P(2/x x) ,
=
p(2/X)
② 9(x 2/x)
,
=
+ (x2)p(2/X)
↑ (X 2) =
, p(u) p(z)
2- Y-
> X is also

Markon Chain

If X-Y >Z
- E

I (XiY) > I (X 2)
;

Data
grocessing Inequality
[ (xiY, 2) =
I (Xi y) +
I(X(x)
↳
= I(X ; 2) + [ (X; Y/z)
2
independent
= (x; y) =
[(x; 2) + [(x ; Y(z)
-
>O 70 38

ExiY)
> =(xiz)

equality
 Aperiodic &(xi/X)) 0
.
Vi

irf
S

O
> L
I-H
I
M

I
Initial probabilities & (X =

&
o =
1/3
1)
9(V :
E3
=

time
C. 1- independent
> of

Aperiodic 2) ,
10
Initial Entropy
H(x1)
>10925-2109 dis
=
=
-
0 . 9
Symbols

Transition
probability matrix

C
Y
=
[pij] &
j
=
P(X ;/Xi)

p
= 1 -
2x
# 11

[p(0) +(11] & = [P(o) 9(11]

LV

steady State Only for

Steady State
Prob Markov
ability Chain
 =
Hp(X) e Like equation
of entropy with
base D .

Conclusion :

logp I
may not be

integer

When-logpp ; are all

integers
for F ; then the prefix code
is optimal :

li
+ =
-

logp9
*
Sli P Hill)
*
L = =

Pi = D-li +
D-adic
distribution

if The 7 smallest integer e

li =
T 109
-

p 97
Stlogy)
S Dei
·
=
Spflospei) i

I
Sei
Spflogp7
= 2
 lim (n - H(x)
n+ c

② X: are independent and identically

distributed

H(x) 1 (n < H(x) +1

③ If &XnY is a marnou chain

CSteady State
Stationary)

Hcs(Y2/X1) = La =
Hss(X2/X1) +

1
-

Shannon Code
suboptimal
di =
logp97
↓ sn < H(x) + 1

Lsh ,
non no of
signals in block

Lsh , n
>
-
H(k)
n 0
=
 Ex a 10 .
3
D =2
D = > 0. 5
21 .
2

d =

C-1092057 91 747 =
.
= 2

12 = 1

13 3
=

O /
od Y
&

S&
I ·

,
2
10
: 0

C(a)
&(b)
=
=
10
0
L :

Sili
c(c) =
110 = 1 7
.

bits/symbol
H(x) =
1 48.
bits/symbol
d Construct
Block
a Shannon
length -
2
code of

aa I

ab.
&

a
↑

ac
DC
 d
↑ Ly-e) =

H(Y) =
H(t(
-
1) ,
21-e) (l)
,

=
H (l 1 -l) ,
+ 10g224 - l)
= H(l ,
1 - 1) + 4 d)
-

Dr)
Channel Coding
Error Control Coding
Redundancy or extra bits

K bits > n
-
bits
Channel
> k
Coding n

& 4)
redundant
-
a

R
In
=
Code rate :
The Sardinas Patterson
Test For UDC

A decodable
code is
uniquely↑

if 'S' contains
and
only if
no codewords.
~

1) Check
is
the
whether
instantaneous
given codeword

If then UDC
yes
.

2) If not for
Suffix
Y
then
and
check
make different dangling
Sets for these
Leg S2 , 53)

3) Now check whether these Sass are

present in the
original codeword

20 ,
10 , 11 Y -UDC

20,02 124 ,
:
Si

Se : 21Y
Not present hene UDC .
 20 109
2) ,
01 ,

S2 : 22Y Not UDL

-

SS : Goy

2110 104
d) ,
11 ,

S2 : GoY
UDC
S3to

Channel Coding
[Error control
coding)
lebits >
-
a bits
(n> 4)
(n-h) redundant
bits

↑
(rDi Code rate)
K bits data > 24 -
codewords

24 = m

:
10g2m
 Radius In t

Sphere Packing

Un dur

v & (no" = (n(2)u1

%2
~ (m+ 65)" (n(P + 4)
=

Mmax (n6fjal = (ncp + 87)n/2

M2
Mmax =
(1 +

logeMmax :

1092 (I +)

#max = 10
(Rmax) C

H
Maximum Rate (Capacity
 Linear Block Codes : -

1) The sum of two codewords

belonging
to the also
code is codeword
belonging
to the code .

ii) The all-zero codeword , is always a

codeword ,

iii)
The Minimum Hamming
im the
Distance
linear
bete
a

block
code is equal O
+
the minimum
*
d 7
weight
=
=
il W .

Ex
C: 20000 ,
1010 , 0101 ,
11117 (4 2) ,

Block Code
0000 + 0000 : 0006
0000 +
1010 :
1010
:
n

*
d = 2
WA = 2

Check Codes
Parity : -
Given K

information bits add one bit so that

 in) a 0 = a
a . 1 = a
&

- Y
elements
Identity

- a - a =0
,
aa = 1

Finite Fields > -

Galois Field

(GF(q))

Ring : -
Addition , Subtraction
Multiplication
,

only
-

Inverses don't exist

Abelian
Group :- Addition & Subtraction
 Scalar -

Standard form of a
or
Systematic

:
6 [[1P] E =
Kyk identity
matril

2 Row echleon
form P =
kX( R) -

< matrix)
parity

Ex G= 0 12 I
10 I O

1221
I

(4, 3) over GF13)

G = [[19] ritrow ;
34343x C
j s column
;
3 - V3 -r
,
-
ra

6= 012 /
I ⑧ I O
- 01 20 3

>
vz
-
v
v
,
-

,
 of
Systematic form
Generator Matrix

G : [I10]

Parity Check

+
CH =
0

C =
:G

H" (I1P] =
0

H = c -

0 II]

* For check distance &*

a
binary minimum

of linear block is
given by the
a

minimum number of rows HT /or columns)

of H

whose Sum is equal to zero vector

.

* For a non binary case , the rows of HT

weighted the codewords has to be
by
considered
.
 GF (5) :
20 ,
1
,
2
, 3 ,
44
God 5

X = 2 ,
3
-
-

GF (4) - 20 , 1 , 4 , % + 1 %

↓ I I+
>
- -
1
O
O I U x +1

I 10 If R

N na + 1 O L

n+
1
x+ 1 u I O

O I
+
-

O O O O O

I c
I O u + 1

N O R n+1 2
2

x+ 1 O n +1 I N