Problern Set # I

1.

A single unbiased die is tossed once. If the face of die is l, 2, 3, or 4 then an unbiased coin is tossed twice otherwise it is tossed only once. Find the amount of information conveyed about the face of the die by the number of heads
observed.

2. 3.

Let X be discrete random variable taking values Xr: .--.....xn- Let Y: g(x), when g is an arbitrary function. slrow that H(Y) < F(X). Under what
conditions (on g) does the equality hold?

L"t X

and

Y bc rundo* variables taking values in {xs,......-xn} and {y,,.'..'Yn}

respectively.

.Let Z= X + Y

'

( xr, Yi are real numbers)

(a) (b) 4. 5.

H (YfX). Show that H Hence if X and Y are independent,H(Zlx): H (Y) and Thus H (Y)< H (Z) and similarly H (X) <H (Z).

tiXl:

I (Z). Give: an example in which H (X) , H (Y) > H

Let Xr and X2 take values in t0, l), i.e. two possible values and let Y=Xt.Xz, Z:X1X2. Computethe maximum amount of information supplied by Y about Z andthe conditions un{pr which this happens.
Consider a coin with labels 0 and 1. Let P,(Xr:l) = p. where Xq is what you Xu = l} whcre Xr, Xz observe alter flipping the coin once. Let N = min {n tossing the coin nd-infinitunt. by obtainctl of observations ..... is thb sequence Let Yn (Xr..........Xn),'

2,,

X,, un = pgl(X,) =| l39r i=r

,t

,v,, =

6in(X,) lsigt
Y)

(a) (b) (c)

(d)

Compute the entropy ofN, Yn, Zn, un, v1 Compute I (un, vn) and its limit as n -+ oo Compute or get suitable lower/upper bounds on I (N, and I (un, Zo) /(N, v, ) I (N ,u,,), /(lr, v, ), Compute

zurd

I (N, Zn)

lJ*

]!1