MEKELLE UNIVERSITY

ETHIOPIAN INSTITUTE OF TECHNOLOGY- MEKELLE

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING
Course Title Information Theory and Coding Techniques
Course Code ECEG 6203
Credit Hours 3 (2Hrs Lecture, 3Hrs Tutorial)
Prerequisite(s) Stochastic Processes
This course exposes students with the mathematical and other tools
Course Rationale (algorithms) required in the modelling, analysis, design and performance
evaluation of modern compression techniques and error correcting codes
 Understand the foundations of information Theory and the
information theoretic approach in the characterization, modelling
and processing of information.
 Enable students to the effect of noise on transmission of digital data.
 Provide the essential tools necessary in the design and performance
Course Objectives
evaluation of different codes used in source coding and error
correction
 To provide the mathematical and other tools (algorithms) required in
the modelling, analysis, design and performance evaluation of
modern compression techniques and error correcting codes.
Entropy, Data Compression, Channel capacity, Linear Block codes and
Course Description
Trellis and Graph Based Codes
1. Vector algebra: Groups, fields and Vector spaces – Elementary
Properties of Galois fields – Primitive polynomials and Galois fields
of Order pm - Zech’s algorithms. Euclidean domains and Euclid’s
Course Contents algorithm – Minimal polynomials and Conjugate elements –
Factoring
2. Entropy: Entropy, relative entropy; Mutual information, Chain
rules; Data processing inequality; The Asymptotic equipartition
 property, Entropy rates for stochastic Processes
3. Data Compression: The source coding theorem, Kraft Inequality,
Shannon-Fano Codes, Huffman codes and Universal source codes,
Lossy Compression: Differential Entropy, Rate distortion theory.
4. Channel Capacity: Discrete Channels, Random coding Bound and
Converse, Gaussian channels, Parallel Gaussian channels and water
pouring, band limited channels
5. Linear Block Codes: Properties of Linear Block codes; Specific
Linear block Codes; Optimum decision decoding; Hard Decision
decoding; Cyclic codes, BCH codes, Reed-Solomon codes.
6. Trellis and Graph Based Codes: Structure of convolution code;
decoding of convolutional codes; Punctured convolution codes;
Tubro codes and Iterative decoding; LDPC codes and TCM
Tutorials/Labs
Tutorials/Labs, Lectures using slides, Tutorials using white Board and Markers;
Delivery Methods
labs using trainer kits
Tests(10%) + Projects/Term paper(40%) + Assignments(10) = 60%,
Mode of Assessment
Final exam = 40%
1. T.M. Cover and J.A. Thomas, Elements of Information Theory,
2nd Ed., John Wiley and Sons, 2006.
2. S.B. Wicker, Error Control Systems for Digital Communication
References
and Storage, Prentice-Hall, 1995.
3. R.G. Gallager, Information Theory and reliable Communication,
John Wiley and Sons, 1968
Instructor Tekle Birhane