ECE Dept.

/SOE
Central University of Karnataka Internal Evaluation
(Established by an Act of the Parliament in 2009)
Assignments

School of Engineering 2020-21(Even Sem)

FIRST ASSIGNMENT
Sem :IV Course Name: Information Theory Course Code: EE2340
Date:05.04.2021 Due Date:19.04.2021 Max. Marks: 20*

Q. No Description of Question Marks

a Derive an expression for the entropy of nth extension of a zero memory source 2
A source emits one of the four messages m1, m2, m3, and m with equal
1 probabilities. Find the entropy of the source using conventional method as well
b 2
as using extremal property.
A binary source is emitting an independent sequence of 0s and 1s with
probabilities ‘p’ and ‘q’ so that the total probability is unity. Plot the entropy of
a 2
2 the source vs. probability. Show that entropy reaches maximum when all
symbols are equi-probable.
b Define source coding. Give detailed classification of source codes 2
State Kraft-McMillan Inequality. Test the KMI for S1=0, S2=10, S3=110, and
a S4=1110. 2
3
b What is Markoff source? Explain it with typical state diagram 2
a Explain the tree digram representation from the state diagram with example. 2
b For the First order Markov source shown below i) Find the stationary distribution
ii) Find the entropy of each state and iii) Entropy of the complete source.

4
2

* Marks will be scaled down to actual weightage as per course credit(s).

Course Coordinator

________________________________________________________________________________________________________
Kadaganchi, Aland Road, Kalaburagi 585 367, Karnataka, India.
Website: www.cuk.ac.in