Scribd
Upload
7 views13 pages

ITC Tutorial 4

This document contains information about an upcoming tutorial on information theory and coding to be given by Dr. Atri Mukhopadhyay and Dr. Sumit K. Chattopadhyay, assistant professors in the EEE Department at BITS Pilani Hyderabad Campus. The tutorial will cover topics such as entropy, mutual information, and channel capacity calculations. It provides examples of calculating entropy for single and joint random variables, deriving the mutual information between two random variables, and finding the maximum channel capacity.

Uploaded by

Anon Poet
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
7 views13 pages

ITC Tutorial 4

This document contains information about an upcoming tutorial on information theory and coding to be given by Dr. Atri Mukhopadhyay and Dr. Sumit K. Chattopadhyay, assistant professors in the EEE Department at BITS Pilani Hyderabad Campus. The tutorial will cover topics such as entropy, mutual information, and channel capacity calculations. It provides examples of calculating entropy for single and joint random variables, deriving the mutual information between two random variables, and finding the maximum channel capacity.

Uploaded by

Anon Poet
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd

Information Theory and

Coding - Tutorial

Dr. Atri Mukhopadhyay, Dr. Sumit K. Chattopadhyay


Assistant Professor
EEE Department
BITS Pilani Hyderabad Campus

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Q1

06/29/2023 2

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Solution

06/29/2023 3

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Solution

06/29/2023 4

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Solution

06/29/2023 5

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Q2

Y
X

06/29/2023 6

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


1
𝑝1 𝑝 1+𝑝 (1 − 𝑝 1)

Input Probabilities Output Probabilities

1− 𝑝 1 (1 − 𝑝)(1 − 𝑝 1)

So, we have P(X), P(Y|X) and P(Y)


Use

06/29/2023 7

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


𝑑 ( 𝐼 ( 𝑋 ; 𝑌 )) 1 − 𝑝1
=( 1− 𝑝 ) 𝑙𝑜𝑔 − 𝑝𝑙𝑜𝑔𝑝
𝑑 𝑝1 𝑝1 +𝑝 (1 −𝑝 1)

Solve for

06/29/2023 8

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


06/29/2023 9

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


06/29/2023 10

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


Q3

06/29/2023 11

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


𝐻 ( 1 1
3 3
1 1 1
3 3 3 ) ( )
2 1
+ ( 𝑝 1 − 𝑝3 ) , , − ( 𝑝 1 − 𝑝 3 ) − ( 𝑝 1 +𝑝 3 ) 𝐻 , − ( 1 −𝑝 1 − 𝑝 3 ) 𝑙𝑜𝑔 3
3 3

06/29/2023 12

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


06/29/2023 13

ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

You might also like