Mallabhum Institute of Technology

ELECTRONICS AND COMMUNICATION ENGINEERING

COURSE DESCRIPTION FORM
Course Title INFORMATION THEORY & CODING

Course Code EC 604B

Regulation

Course Structure Lectures Tutorials Practicals Credits

3 0 0

Course Coordinator Ms. Mayumi Mukherjee

I. COURSE OVERVIEW:

The aim of this course is to provide students with a thorough understanding of information theory is
considered a branch of communication theory, it actually spans a wide number of disciplines
including computer science, probability, statistics, economics, etc. The most basic questions treated
by information theory are: how can ‘information’ measured? How can ‘information’ be transmitted?
From a communication theory perspective it is reasonable to assume that the information is carried
out either by signals or by symbols. Shannon’s sampling theory tells us that if the channel is band
limited, in place of the signal we can consider its samples without any loss. Therefore, it makes
sense to confine the information carriers to discrete sequences of symbols, unless differently stated.
we will see how we can further reduce the error probability using error control coding. This is done
by encoding our data in the transmitter before the digital modulation, and by decoding the received
data in the receiver after the digital demodulation. There are error control codes over alphabets of
any size. We will only consider binary error control codes, i.e. the alphabet consists of two symbols,
normally denoted 0 and 1. The general idea of error control codes is to let the encoder calculate
extra control bits from the information that we wish to transmit, and to transmit those control bits
together with the information. If that is done in a clever way, then the decoder can detect or correct
the most probable error patterns. Thus, both the encoding and the decoding of the data are done by
clever mapping of sequences of bits on sequences of bits. The available signal energy per
information bit is always limited. Transmitting control bits together with the information demands
extra energy. It is then natural to ask if that energy could be better used by simply amplifying the
signals instead. In most reasonable situations, however, it is possible to show that using error control
codes is a better way to utilize that energy. This is what we’ll do throughout this lectures.
II. PREREQUISITE(S):

Level Credits Periods / Week Prerequisites

Knowledge of
UG 3 4
1) Signals &Systems 2) Analog Communications n

III. COURSE ASSESSMENT METHODS:

a) Marks Distributions (Traditional Evaluation methods)

University
Session Marks End Exam Total Marks
Marks
There shall be two class tests. Each class test consists of
subjective type. The subjective test is for 2 5 marks, with
duration of 60 minutes. Subjective test of each semester shall
contain three types of questions; (i) Very short answer type (ii) External
(70)
short answer type & (iii) long answer type question. The student
has to answer all of them.

There will be four continuous assessment names as CA1, CA2, 100

CA3, CA4. Each carrying 25 marks which includes the
average of two internal tests, two assignments and marks for
attendance. Internal
(30)
The average of CA marks are considered to be the final internal
marks which is 30.

b) Expected Learning Outcomes and Methods for Assessing

Sl. Assessment
Expected Learning Outcomes
No. methods

read assigned material prior to class, bring text and supplement to

a class, and take part in class discussions. (Supplement not Review
required)
b successfully complete any outside assignments given during the Assigning exercises
semester
c discuss end of chapter question problems and short cases Discussions/ review
assigned
 d Integrate management principles into management practices. Group Activity

Assess managerial practices and choices relative to ethical

e Model Design
principles and standards
Specify how the managerial tasks of planning, organizing, and
f Group assignments
controlling can be executed in a variety of circumstances.
g Determine the most effective action to take in specific situations. Assigning exercises

III. EVALUATION SCHEME:

S. No Component Duration Marks

1 I Class Test 45 minutes 25

2 II Class Test 45 minutes 25

3 External Examination 180 minutes 70

IV. COURSE OBJECTIVES:

1. To introduce information theory, the fundamentals of error control coding techniques and
their applications .
2. To provide a complementary U/G physical layer communications course to ECE4601 and
ECE4606.
3. This class will first introduce the basic concepts of information theory, leading to the
channel capacity theorem.
4. Afterwards, the course will consider error control coding techniques and applications.

V. COURSE OUTCOMES:

This course covers the fundamental concepts of information theory and error control coding.
At the conclusion of the course, several objectives will be achieved:
1. Students will be introduced to the basic notions of information and channel capacity.
2. Design the channel performance using Information theory.
3. Comprehend various error control code properties
4. Apply linear block codes for error detection and correction
5. Students will be introduced to convolutional and block codes, decoding techniques, and
automatic repeat request (ARQ) schemes.
6. Apply convolution codes for performance analysis & cyclic codes for error detection and
correction.
7. Design BCH & RS codes for Channel performance improvement against burst errors.
VI. HOW COURSE OUTCOMES ARE ASSESSED:

Program Outcomes Level Proficiency accessed by

a) An ability to apply the knowledge of mathematics, science, H Assignments
and engineering to solve Electronics and communications and
engineering problems Exercises
An ability to design and conduct engineering experiments, as Assignments
well as to analyze and interpret data. and
b) H
Exercises

An ability to design and construct a hardware and software Hands on

c) system, component, or process to meet desired needs, H Practice Sessions
within realistic constraints.
An ability to function on multi – disciplinary teams. Group Discussions
d) S
An ability to identify, formulate and solve engineering Exercises
e) problems. N and
assignments
An understanding of professional, social and ethical Work shops
f) S
responsibility.
An ability to communicate effectively. Oral
g) N
Discussions
The broad education necessary to understand the impact of
h) engineering solutions in a global, economic, environmental, H --
and societal context.
An ability to recognize the need for and an ability to engage in
i) life –long learning. N --

An ability to gain knowledge of contemporary issues.

j) S Team participation

An ability to use the techniques, skills, and modern

k) engineering tools necessary for engineering practice. H Exams and seminar

Graduates are able to participate and succeed in

competitive examination like GRE, GATE, TOEFL, GMAT
l) H Exercises
etc.

The use of current application software’s; the design and

use of circuits; and the analysis, design, testing, and
m) S Assignments
documentation of Analog and Digital circuits for the use in
Communications engineering.
The basic knowledge of electronics, electrical components,
n) computer architecture and applications of microcomputer H Exercises
systems, communications needed in data transport
N=None S=Supportive H=Highly related
VII. SYLLABUS:

UNIT I
Source Coding
Uncertainty and information, average mutual information and entropy, information measures for
continuous random variables, source coding theorem, Huffman codes.

UNIT II
Channel Capacity And Coding
Channel models, channel capacity, channel coding, information capacity theorem, The Shannon limit.

UNIT III
Linear And Block Codes For Error Correction
Matrix description of linear block codes, equivalent codes, parity check matrix, decoding of a linear block
code, perfect codes, Hamming codes

UNIT IV
Cyclic Codes
Polynomials, division algorithm for polynomials, a method for generating cyclic codes, matrix description
of cyclic codes, Golay codes.

UNIT V
BCH Codes
Primitive elements, minimal polynomials, generator polynomials in terms of minimal polynomials,
examples of BCH codes.

UNIT VI
CONVOLUTIONAL CODES
Tree codes, trellis codes, polynomial description of convolutional codes, distance notions for
convolutional codes, the generating function, matrix representation of convolutional codes, decoding of
convolutional codes, distance and performance bounds for convolutional codes, examples of
convolutional codes, Turbo codes, Turbo decoding.

Text Books:
1. Information theory, coding and cryptography - Ranjan Bose; TMH.

2. Error Control Coding - Shu Lin and D J Costello Jr; Prentice Hall.

Reference Books:
1. Information and Coding - N Abramson; McGraw Hill.

2. Introduction to Information Theory - M Mansurpur; McGraw Hill.

3. Information Theory - R B Ash; Prentice Hall.

VIII. Course Plane:

Lecture
Reference
Unit No. Course Learning Objective Topics to be covered

DAY 1 Overview of the Syllabus

Coding & reliable digital

1) Coding for reliable transmission & storage,
DAY 2 transmission simplified model of a coded T2
2) Types of codes system, Different type of code
name.

Different type of error like

1) Types of error memory less channel & channel
DAY 3 2) Error control strategies with memory, BSE Channel, T2
3) Automatic repeat Error control strategies like stop
III request scheme &wait ARQ and Continuous ARQ

Introduction to block codes ,

1) Introduction to block example like (5,4) , (7,4) , (6,3)
DAY 4-5
codes Block codes, Single parity check T2

2) Single parity check code code Example (8,7) block codes,

problem.

1) Hamming codes
Hamming codes, Error
2) Error syndrome
DAY 6-8 syndrome, Minimum distance of T2
3) Minimum distance of
block code, problem
block code

1) Linear Block Code Introduction to Linear Block

DAY 9-11 2) Generator matrix Code, Generator matrix, T2
3) Standard array Standard array, problem

1) Parity-check matrices
Parity-check matrices, Error
DAY 12- 2) Error Syndrome
Syndrome, Error detection & T2
16 3) Error detection &
Correction, problem
Correction

1) Introduction to Cyclic Introduction to Cyclic codes,

DAY 17-
codes Generator polynomials, Parity T2
19
2) Generator polynomials check matrix, problem
 3) Parity check matrix

1) Symmetric & Non Symmetric & Non symmetric

symmetric matrices with matrices with example ,
DAY 20- example Symmetric & Non symmetric
T2
IV 24 2) Encoding of a cyclic matrices , Generator
code polynomial, Minimal Polynomial
3) Generator polynomial with example
4) Minimal Polynomial

DAY 25- 1) Decoding of cyclic code Decoding of cyclic code, Problem

T2
26 solving
2) Problem solving
DAY 27 Golay codes Golay codes with example T2

1) Definition of Definition of information,

DAY 28- information Information Sources
T1
29 2)Information Sources; Classification, Related Example
Classification
3) Related Example
1) Entropy; Entropy of Entropy; Entropy of binary
DAY 30-
binary source source T2
31
I 2) Information Rate

Types of Channel with

DAY 32 Types of Channel with Example
Example T1

1) Condition of Joint Condition of Joint probability &

probability & Joint Joint Entropy, Mutual
Day 33- Entropy information, Uncertainty of T1
34
2) Mutual information communication channel.
3) Uncertainty of
communication channel
1) Source Coding theorem Source Coding theorem,
DAY 35-
2) Huffman theorem Huffman theorem, Problem
36 T1
3) Problem solving solving
 1) Channel capacity, Channel capacity, Channel T1
2) Channel coding, coding, information capacity
DAY 37- 3) information capacity
38 theorem,
II theorem,

1) The Shannon limit The Shannon limit, Proof of

DAY 39 2)Proof of Harley Shannon T1
Harley Shannon Law
Law

1) Introduction to BCH
code Introduction to BCH code,
DAY 40- 2) Primitive elements, Primitive elements, Minimal
T2
41 3) Minimal polynomials, polynomials, Contribution of
4) Contribution of BCH BCH codes
codes

V 1) Generator polynomials
Generator polynomials in terms
DAY 42- in terms of
of Minimal polynomials, T2
44 2) Minimal polynomials,
Examples of BCH codes
3) Examples of BCH codes

DAY 45- 1) Error Syndrome Error Syndrome, Decoding of

T2
46
2) Decoding of BCH codes BCH codes with example

1) Introduction to
DAY 46-
convolution codes
47
2) Convolution encoder
Introduction to convolution
T2
codes, Convolution encoder

1) polynomial description
polynomial description, Distance
DAY 48 2) Distance notions for T2
notions for convolution codes
convolution codes
VI
1) Generating function, Generating function, Matrix
2) Matrix representation representation , Decoding of
DAY 49- 3) Decoding of convolution codes, Examples of
T2
51 convolution codes convolution codes
4) Examples of
convolution codes

Tree codes, trellis codes, Tree codes, trellis codes T2

DAY 52
 Turbo codes, Turbo
DAY 53 Turbo codes, Turbo decoding. T2
decoding.

IX. MAPPING COURSE OUTCOMES LEADING TO THE CHIEVEMENT OF PROGRAM

OUTCOMES:

Subject a b c d e f g h i j
Code
Information theory & coding
1. an ability to apply the
√ √
knowledge of Mathematics,
science and engineering in
Electronics and
communications

2. an ability to Design &

Conduct Experiments, as well
√ √ √ √
as analyze & Interpret Data

3. an ability to design a system,

EC 604B
component, or process to meet
desired needs with in realistic
constraints such as economic,
environmental, social, political, √ √
ethical, health and safety,
manufacturability, and
sustainability

4. an ability to function on
multidisciplinary teams √ √

5. an ability to use the

techniques, skills, and modern
engineering tools necessary for √ √ √
engineering practice
X. MAPPING COURSE OBJECTIVES LEADING TO THE ACHIEVEMENT OF PROGRAM
OUTCOMES:

Code Course
objectives a) b) c) d) e) f) g) h) i) j)

I. √ √
II. √ √ √
EC 701
III. √ √
IV. √

Prepared by:

Mrs. Mayumi Mukherjee

Assistant Professor
Dept. of ECE