Information Theory and Coding (EC60332)

Indian Institute of Technology Kharagpur

Department of Electronics and Electrical Communication Engineering
Instructor: Jithin Ravi

Problems Set 3 (Problems on Limits, WLLN, AEP) January 2023

1. Let X1 , X2 , . . . , Xn be n IID random variables, each with distribution function FX (x). Let
us define Y = max(X1 , X2 , . . . , Xn ), i.e., Y denotes the maximum of n IID random variables.
Find the distribution function of Y .

2. Let X be a continuous random variable with cdf FX (x). Find the distribution function of the
random variable Y defined as Y , FX (X).

3. Let X be a random variable defined on a probability space (Ω, F, P ). For any A ∈ F, let
f (X, A) be a non-negative valued function of X and A. It is known that E(f (X, A)) = Pr(A).
Identify the function f (x, A).

4. Prove that for any sequence an , lim an = 0 if and only if lim |an | = 0.
n→∞ n→∞

5. Suppose sequence an is such that an ≤ an+1 for all n. Comment on the limit of an , i.e.,
lim an .
n→∞

6. Let {Xn ; n ≥ 1} be a sequence of independent but not identically distributed random vari-
ables. Let us define Sn , X1 + X2 + · · · + Xn . Suppose there exists a constant C such that
VAR(Xn ) ≤ C for all n. Then show that, for all  > 0,
  
 Sn E(Sn ) 
lim Pr   −  ≥  = 0.
n→∞ n n 

7. Problem 3.4 of Cover and Thomas (2nd edition).

8. Problem 3.6 of Cover and Thomas (2nd edition).

9. Problem 3.7 of Cover and Thomas (2nd edition).

10. Problem 3.8 of Cover and Thomas (2nd edition).