EEE 546: Information Theory and Security

Description:
• Review of probability theory: Sample space, Conditional probability, probability distributions,
probability distribution functions, stochastic processes, statistical averages. Random variable and
random process-covariance, Ergodic process, Markov process
• Source Coding: Mathematical model for information sources, Average mutual Information,
Entropy in discrete and continuous cases, definition and properties of entropy, Joint and
conditional entropy. The problem of unique decipherable code, instantaneous code, Kraft
McMillan inequality
• Discrete and Analogue Coding: Hufmanns Coding Algorithm, Discrete stationery sources, The
Lempel Ziv algorithm, Optimum Quantization Rate, distortion function for memoryless Gaussian
source. Upper bound of scalar quantization, vector quantisation, Linear predictive coding
• Channel Coding: Channel models, Binary symmetric channel, Discrete memoryless channel,
Discrete input contentious output channel, Channel capacity, Shannon fundamental theorem,
Shannon Hartley theorem and its implications. Achieving channel capacity with orthogonal
signals, channel reliability functions
• Block and Convolution Codes: Linear block codes, Hadamard code, Golay code, Cyclic code,
encoders, BCH code. Error detection and correction capability, convolutional codes, convolutional
encoder, Viterbi algorithm.