You are on page 1of 2

Department of Electronics and Communication Engineering

National Institute of Technology Trichy


Problem set-1 (Information measure)
Subject: Statistical Theory of Communication
1. The inhabitants of a certain village are divided into two groups A and B.Half the people in group
A always tell the truth, 3/10 always lie,and 2/10 always refuse to answer. In group B,3/10 of the
people are truthfulm ,half are liars and 2/10 always refuse to answer.Let p be the probability that
a person selected at random will belong to group A.Let I=I(p) be the information conveyed about
a persons truth telling status by specifying his group membership.Find the maximum possible
value of I and the percentage of people in group A for which maximum occurs.
2. A single unbiased die is tossed once.If the face of the die is 1,2,3 or 4 an unbiased coin is tossed
once.If the face of the die is 5 or 6,the coin is tossed twice.Find the information conveyed about
the face of the die by the number of heads obtained.
3. Suppose that in a certain city,3/4 of the high-school students pass and 1/4 fail.Of those who pass,10
percent own cars,while 50 percent of the failing students own cars.All of the car-owning students
belong to group A,while 40 percent of those who do not own cars but pass,as well as 40 percent of
those who do not own cars but fail,belong to group A.
How much information is conveyed about a students academic standing by specifying whether
or not he owns a car?
How much information is conveyed about a studentss academic standing by specifying whether
or not he belongs to a group
If a students academic standing,car-owning status,and group A status are trasmitted by three
successive binary digits,how much information is conveyed by each digit?
4. Establish the following
H(Y, Z/X) = H(Y /X) + H(Z/X)
H(Y, Z/X) = H(Y /X) + H(Z/X, Y )
H(Z/X, Y ) H(Z/X)
5. prove that H(X, Y ) = H(X) + H(Y /X) = H(Y ) + H(X/Y )
6. Prove that H(Y /X) H(Y ) with equality if and only if X and Y are independent Given a discrete
random variable X with values x1 ,x2 ,...xM ,define the random variable Y by Y=g(X),where g is an

Figure 1:

Figure 2:

abitrary function.Show that H(Y ) H(X).Under what conditions on the function will there be
equality? [HINT:H(Y/X)=0 (why?)]
7. Apply Huffmans encoding procedure and shannon-fano encoding procedure to the following message ensemble and determine the average length of the encoded message.

P (X) = 0.18, 0.17, 0.16, 0.15, 0.10, 0.08, 0.05, 0.05, 0.04, 0.02
8. Determine the rate of transmission of the channel mentioned in figure 1.
9. The random variable as the pdf as shown in the figure 2. Find the corresponding entropy function.
10. Find the entropy of a continous random variable with the density function f (x) = bx2 , 0 x a
and 0 , otherwise.