Tutorial Sheet

Subjec- ITC Chapter- Elements of Information Theory

Q1. The probabilities of the five possible outcomes of an experiment are given as

1 1 1 1
P ( x 1) = , P ( x 2 ) = , P ( x 3 )= & P ( x 4 )=P ( x 5 ) =
2 4 8 16

Q2. Determine the Huffman code for the following messages with their probabilities given

x 1=0.05, x 2=0.15, x 3=0.2, x 4 =0.05, x 5=0.15 , x 6=0.3 , x 7=0.1

1
Q3. Calculate the amount of information if it given that P ( x i) =
4

Q4. Given a binary symmetric channel (Figure 1) with P ( x 1) =β

Show that the mutual information is given by

I (X ; Y )=H (Y )+ p log 2 p+( 1− p)log 2(1− p)

Q5. Given a binary symmetric channel (Figure 1) with P ( x 1) =β

Calculate I (X ; Y ) for β = 0.5 and p = 0.1.

Figure 1: BSC