Assignment 3- Joint Entropy, Mutual Information

Date: 17/09/2020
Last date of submission: 21/09/2020

𝒓𝟎 𝒓𝟏 𝒓𝟐
𝒔𝟎 ∝ 0 ∝
𝒔𝟏 0 ∝ ∝
𝒔𝟐 ∝ ∝ 0

Consider the joint probabilities shown in the table corresponding to 𝑃(𝑠𝑖 , 𝑟𝑗 ). Answer
the following questions.

1) What is the value of 𝛼? c) 𝑙𝑜𝑔2 4

a) 1/2 d) 𝑙𝑜𝑔2 2
b) 1/3
c) 1/4 4) What is 𝐻(𝑆/𝑅)?
d) 1/6 a) 𝑙𝑜𝑔2 4

b) 𝑙𝑜𝑔2 3
2) What is 𝐻(𝑆)? c) 𝑙𝑜𝑔2 2
a) 𝑙𝑜𝑔2 6 d) 𝑙𝑜𝑔2 1
b) 𝑙𝑜𝑔2 3
c) 𝑙𝑜𝑔2 2
d) 𝑙𝑜𝑔2 1.5 5) What is 𝐼(𝑆; 𝑅)?

a) 𝑙𝑜𝑔2 6
b) 𝑙𝑜𝑔2 3
3) What is 𝐻(𝑆, 𝑅)? c) 𝑙𝑜𝑔2 2
a) 𝑙𝑜𝑔2 9 d) 𝑙𝑜𝑔2 1.5
b) 𝑙𝑜𝑔2 6