Panipat Institute of Engineering & Technology

Samalkha
Computer Science & Engineering Department

Practical File of Information Theory and Coding Lab

Code - PC-CS-CYS-312LA

Submitted to: Mr. Vishal Jain Submitted by: Name: Asmit

Assistant Professor Roll no: 2820425
CSE-ET 3rd Year
Cyber Security
Section -F

Affiliated to
 INDEX
Sr. PRACTICAL STATEMENT DATE SIGNATURE
1. Introduction of MATLAB.

2. Implementation of Self Information.

3. Determination of entropy of a given source as well as

source rate.
4. Determination of Entropy of binary source.

5. Determination of Marginal and Joint entropies.

6. Determination of Conditional entropies and Trans

information.
7. Determination of various entropies and mutual
information of a given channel (Binary symmetric
channel).

8. Generation and evaluation of variable length source

coding using MATLAB (Shannon coding and
decoding).
9. Generation and evaluation of variable length source
coding using MATLAB (Huffman coding and
decoding).
 Practical no – 2
Aim :- Implementation of Self Information.

Software Required :- MATLAB 2015 a

Self Information

p=[0.1 0.15 0.2 0.25 0.3];

m=[];
for i=1:5
ix=-log2(p(i));
m(i)=ix;
end
m
plot(p,m)

Result:
 Practical no – 3
Aim :- Determination of entropy of a given source as well as source rate.
Software Required :- MATLAB 2015 a

Source Entropy and Rate

p=[0.2 0.4 0.1 0.2 0.1];
hx=0;
for i=1:5
hx=hx+(-p(i)*log2(p(i)));
end
hx

Result:
hx = 2.1219

entropy rate
p=[0.2 0.4 0.1 0.2 0.1];
t=[0.0001 0.0002 0.0003 0.0004 0.0005];
hx=0;
tx=0;
for i=1:5
hx=hx+(-p(i)*log2(p(i)));
tx=tx+p(i)*t(i);
rx=hx/tx;
end
hx
tx
rx

Result:
hx = 2.1219
tx = 2.6000e-04
rx = 8.1613e+03
 Practical no – 4

Aim :- Determination of Entropy of binary source

Software Required :- MATLAB 2015 a

Entropy of binary source

p=0:0.1:1;
p1=1-p;
m=[];
for i=1:11
hx=-(p(i)*log2(p(i))+p1(i)*log2(p1(i)));
m(i)=hx
end
m
plot(p,m)
title('Entropy Of Binary Source')
xlabel(' Probabilty ')
ylabel(' Entropy ')
Result:
 Practical no – 5

Aim : - Determination of Marginal and Joint entropies

Software Required :- MATLAB 2015 a

Marginal and Joint entropies

pxy=[0.1 0.15 0.05 ;0.2 0.07 0.08 ;0.01 0.04 0.3];

py=sum(pxy);
hy=0;
for i=1:3
hy=hy+(-py(i)*log2(py(i)));
end
hy
px=sum(pxy,2);
hx=0;
for i=1:3
hx=hx+(-px(i)*log2(px(i)));
end
hx
hxy=0;
for i=1:3
for j=1:3
hxy=hxy+(-pxy(i,j)*log2(pxy(i,j)));
end
end
hxy

Result:
hy = 1.5526
hx = 1.5813
hxy = 2.7566
 Practical no – 6

Aim :- Determination of Conditional entropies and Trans information.

Software Required :- MATLAB 2015 a

Conditional entropies and Trans information

pxy=[0.1 0.15 0.05 ;0.2 0.07 0.08 ;0.01 0.04 0.3];

py=sum(pxy);
hy=0;
for i=1:3
hy=hy+(-py(i)*log2(py(i)));
end
hy
px=sum(pxy,2);
hx=0;
for i=1:3
hx=hx+(-px(i)*log2(px(i)));
end
hx
hxy=0;
for i=1:3
for j=1:3
hxy=hxy+(-pxy(i,j)*log2(pxy(i,j)));
end
end
hxy
h1=hxy-hx
h2=hxy-hy
I=hx-h2 OR I=hy-h1
h1
h2
I
Result:
hy = 1.5526
hx = 1.5813
hxy = 2.7566
h1 = 1.1753
h2 = 1.2039
I = 0.3774
 Practical no - 7

Aim :- Determination of various entropies and mutual information of a given

channel (Binary symmetric channel).

Software Required :- MATLAB 2022 a

Binary symmetric channel

a=[0.9 0.1 ;0.1 0.9];
y=[0.7 0.3];
m=2;
k=0;
hy=0;
for i=1:2
k=k+(a(i)*log2(a(1,i)));
hy=hy+(-y(i)*log2(y(i)));
end
k
hy
c=log2(m)+k
I= hy +k
E=(I/c)*100
R=100-E

Result:
k = -0.4690
hy =0.8813
c = 0.5310
I = 0.4123
E = 77.6444
R = 22.3556
 Practical no - 8

Aim :- Generation and evaluation of variable length source coding using

MATLAB (Shannon coding and decoding).

Software Required :- MATLAB 2022 a

Shannon code

x=[0.25 0.12 0.13 0.1 0.08 0.07 0.06 0.05 0.04 0.035
0.025 0.02 0.01 0.01];
m=[];
for i=1:14
b=-log2(x(i));
l=fix(b);
if l==b
m(i)=l;
else
l=l+1;
m(i)=l;
end
end
lc=0;
for i=1:14
lc=lc+m(i)*x(i);
end
lc
hx=0;
for i=1:14
hx=hx+(-x(i)*log2(x(i)));
end
hx
eff=hx/lc

Result:
lc= 3.7050
hx = 3.3354
eff = 0.9003
 Practical no – 9

Aim :- Generation and evaluation of variable length source coding using

MATLAB (Huffman coding and decoding).

Software Required :- MATLAB 2022 a

Huffman Code:

x=[1 2 3 4 5];
p=[0.1 0.2 0.4 0.1 0.2];
dict=huffmandict(x,p)
dict(:,:);
hcode=huffmanenco(x,dict);
for i=1:5
samplecode=dict(i,2);
end

Result:
dict=

[1][1x4 double]

[2] [1x3 double]

[3] [1]

[4] [1x4 double]

[5] [1x2 double]