ECE458

INFORMATION THEORY AND CODING

LECTURE 8
Associate Prof. Fatma Newagy
Fatma_newagy@eng.asu.edu.eg
LAST LECTURES TOPICS
 Part 1: Information Theory

fatma_newagy@eng.asu.edu.eg
2
TODAY’S TOPICS
 Part 2: Channel coding
 Linear Systematic Block Codes

fatma_newagy@eng.asu.edu.eg
 Generator Matrix and Encoding Process
 Parity Check Matrix
 Encoding Circuit

3
Channel Coding in Our Everyday Lives: Examples

fatma_newagy@eng.asu.edu.eg
4

4
Channel Coding in Our Everyday Lives: Examples

fatma_newagy@eng.asu.edu.eg
5
WHAT IS CHANNEL CODING?
 Channel Coding means :
Introducing redundancy (i.e., adding extra bits) to
information messages to protect against channel

fatma_newagy@eng.asu.edu.eg
errors.

 Channel coding, also known as forward error

control coding (FECC), is a process of detecting and
correcting bit errors in digital communication systems.

6
Channel coding
Source of User of
Information Information

Source Source
Encoder Decoder

fatma_newagy@eng.asu.edu.eg
B1B2.. Bk B*1B*2.. B*k
Code Rate
Channel Channel
Encoder r=k/n Decoder
r<1
W1W2.. Wn W*1W*2.. W*n

Modulator De-Modulator

Channel

 Linear block code

 Convolutional code 7
 Cyclic code 7
 WHAT ARE LINEAR BLOCK CODES?

Linear Block Codes

 Information sequence is segmented into message blocks of
fixed length. Why we need to segmentize the a

fatma_newagy@eng.asu.edu.eg
stream of iformation bits ?

 Each k-bit information message is encoded into an n-bit

codeword (n>k)

2k Binary Block 2k
k-bit Messages Encoder n-bit DISTINCT
codewords

8
 WHAT ARE LINEAR BLOCK CODES?

Linear Block Codes

 Modulo-2 sum of any two codewords is also a codeword

 Each codeword v that belongs to a block code C is a linear

combination of k linearly independent codewords in C, i.e.,

v  u 0g0  u 1g1  ...  u k 1gk-1 ,

gi   g i 0 g i 1 ... g i ,n 1 
9
Linearly independent
codewords
 LINEAR INDEPENDENCE
 A set of vectors g0, g1,…, gk-1 are linearly independent if
there exists no scalars u0, u1,…, uk-1 that satisfy

v  u 0g0  u1g1  ...  u k 1gk-1  0

Unless u0=u1=…= uk-1=0
If U != 0
for g to be independent V != 0
see slide 13

 Examples
 [0 1 0 ], [1 0 1], [1 1 1] are ………
 Linearly Dependent
 [0 1 0 ], [1 0 1], [0 0 1] are ………
 Linearly Independent
10
 WHY LINEAR?
 Encoding Process
 Store and Index 2k codewords of length n
 Complexity
 Huge storage requirements for large k
 Extensive search processing for large k

fatma_newagy@eng.asu.edu.eg
 Linear Block Codes
 Stores k linearly independent codewords
 Encoding process through linear combination of codewords g0,
g1,…, gk-1 based on input message u=[u0, u1,…, uk-1]

 g0   g 00 g 01 ... g 0 ,n 1 

v=u.G g   g
 1
 
 .   .
10 g11 ... g 1,n 1 
. . 

G=   . . .

 .   
 .   . . . 
   
gk 1   g k 1, 0 g k 1,1 ... g k 1,n 1  11

Generator Matrix
EXAMPLE Message Codeword
0000 0000000
 g0  1 1 0 1 0 0 0 0001 1010001 g3
 g  0 0
1 1 0 1 0 g2
G=  1   
0010 1110010

g2  1 1 1 0 0 1 0 0011 0100011
    g1
 g3  1

fatma_newagy@eng.asu.edu.eg
0 1 0 0 0 1 0100 0110100
0101 1100101
0110 1000110
u= [0 1 1 0]
0111 0010111
1000 1101000 g0
Linear Block 1001 0111001

Encoder (v=u.G) 1010 0011010

1011 1001011
1100 1011100
v= g1+g2 1101 0001101
v= [1 0 0 0 1 1 0] 1110 0101110
12
1111 1111111
EXAMPLE
 g0  1 1 0 1 0 0 0
 g  0 0 1 0 1 1 1 Linearly
G=  1     Dependent
g2  1 1 1 1 1 1 1
   

fatma_newagy@eng.asu.edu.eg
 g3  1 0 1 0 0 0 1

u= [0 1 1 1] u= [1 0 0 1]

Block Encoder Block Encoder

(v=u.G) (v=u.G)

v= g1+g2+g3 NOT DISTINCT v= g0+g3

v= [0 1 1 1 0 0 1] v= [0 1 1 1 0 0 1] 13
 LINEAR SYSTEMATIC BLOCK CODES
n-k bits k bits

Redundant Checking Message

Part Part

fatma_newagy@eng.asu.edu.eg
p-matrix kxk- identity matrix
 p 00 p 01 ... p 0 ,n k 1 1 0 ... 0 
 p p11 ... p1,n k 1 0 1 ... 0 
 10
 . . . . . .
G=P Ik    
 . . . . . .
 . . . . . .
 
 p k 1,0 p k 1,1 ... p k 1,n k 1 0 0 ... 1 
14
 THE PARITY CHECK MATRIX
 For any k x n matrix G with k linearly independent rows,
there exists an (n-k) x n matrix H (Parity Check Matrix), such
that
 G.HT=0

fatma_newagy@eng.asu.edu.eg
 1 0 ... 0 p 00 p 01 ... p k 1,0 
 0 1 ... 0 p 01 p11 ... p k 1,1 
 
 . . . . . . 
H=Ik P   
T

 . . . . . . 
 . . . . . . 
 
 0 0 ... 1 p 0 ,n k 1 p1,n k 1 ... p k 1,n k 1 
15
EXAMPLE

1 1 0 1 0 0 0
0 1 1 0 1 0 0
G 

fatma_newagy@eng.asu.edu.eg
1 1 1 0 0 1 0
 
1 0 1 0 0 0 1

1 0 0 1 0 1 1 
H  0 1 0 1 1 1 0 
 
0 0 1 0 1 1 1  16
 1 1 0 1 0 0 0
0 1 1 0 1 0 0
G 
ENCODING CIRCUIT 1 1 1 0 0 1 0
 
1 0 1 0 0 0 1

Input u Message Register

u0 u1 u2 u3
To channel

fatma_newagy@eng.asu.edu.eg
Output v

+ + +

v0 v1 v2

Parity Register

[u0 u1 u2 u3] Encoder Circuit [v0 v1 v2 u0 u1 u2 u3]

17