Information Theory

Linear Block Codes

• Linear Code and Its properties
• Generator Matrix to generate codeword in
linear Block code
• Introduction to error correcting codes
• Coding and decoding of Linear Block code
• Minimum distance consideration
• Conversion of non-systematic form of matrices
Into systematic form.
 Linear Block codes and Its properties
• Definition-A block code is said to be linear block code if the
sum of any two code word gives an another codeward
• Ie=cn=ci+cj
• Eg 1000,0101,1111 0101+1010=1111
• In linear block code each block containing k message bits is
encoded into a block of n-bits by adding (n-k) parity check bits
• K bits(Message block)channel Encoder(adding n-k parity
check bits)n-bits(Linear Block code)
• Code word:-
– Kbits|(n-k bits)
– N-bits
 1Introduction
Some basic definitions:-
i) Weight of a codeword(w)
It is defined as the numbers of non zero bits in C(codeword)
c=010101
Total 3 non zero bits are present in C
:: w©=3
Ii Minimum Distance:
The minimum distance of a block code is the smallest distance between any
pair of code words in a code.
Eg:-
C1 and C pair of codeword.
C1=010101
C2=011011
Dmin=3
 Linear block code
– A linear block code is a linear combination of
Parity bits and message bits.
(K+n-k=n)
– {0000,0101(weight w=2),1010(w=2),1111(w=4)}
– Distance between 0000 and 0101 is d=2 as they
differ in their position at 2 position only
– 0101 and 1010 distance is d=4 as they differ by 4
– 1010 and 1111 distance position difference is 2
– Dmin=2
– Wmin=2
Properties:-
i- All the zero codeword is a valid codeword (0000) valid
Given any 3 codeword ci, cj and ck such that ck=ci+cj
Then d(Ci, Cj)=w(Cn)
Ci=0101
Cj=1010
--------------
1111
---------------
D(ci, cj)=4
W(ck)=2+2=4
Ii. Minimum distance of code(dmin) is equal to min w(c) exce ot
akk zero codeward
Dmin=min w(c) except all zero codeword
 Generator Matrix to generate codeword in
Linear Block code
• I Encoding Operation using matrix
Encoding means forming codeword using the concept of matrix
Message block on a matrix is represented as
D=[d1,d2,d3………dn]
So,2^k combinations of codeword are possible
Each message block is encoded in n bit codeword©
The n bit codeword in a matrix is represented as
C=[D][G] where[G] is a generator matrix
[C1,C2,C3,C4,……CN]=[d1,d2,d3……..dn][1 0 0 p11 p12 …..p1(n-k)]
[0 1 0 p21 p22……p2(n-k)]
..
..
[0 0 01 pk1………….pk(n-k)]
 II Generator matrix
• i– A generator matrix of (n,k) linear code has k
rows and n colmns
• It is given as [G]=[Ik:Pkxn]
• [G]=[100 p11p12p13…p1(n-k)]
[010 p21 p22p…..p2(n-k)]
Identity MatrixParity Matrix(jx (n-k))
A generator matrix is a combination of Identity matrix and parity
matrix
 Problems
• Ques: The generator matrix of (6,3)block code is shown below
• G=[1 0 0 :0 1 1]
[0 1 0: 1 0 1]
[0 0 1:1 1 0]
Obtain all code vectors of this code
Sol. Here (n,k) =(6,3)
Length of code word (n)=6
No. of message bits(k)=3
G=[Ik:P]
kxk(3x3) :kx(n-k)(3x(6-3))
3x3
In=[100] and P=[0 1 1]
[010] [ 1 0 1]
[001] [ 1 1 0] 3X3
Approach I
• Since No. of message bits(k)=3
• Therefore Possible combination of codeword =2^k=2^3=8
• So those code words are
• D=000 C=[D][G]
001 (C1,C2,C3,C4,C5,C6,C7,C8)
010 C1=[000][100:011]
011 [010:101]
100 [001:110]
101 C1=0(+)0(+)00
110 c1=[000000]
111
C2=[010101]
C3=[011011]
C4=[100011]
 Code word code vector
• Message parity
• 000 000
• 001 110
• 010 101
• 011 011
• 100 011
• 101 101
• 110 110
• 111 000
 Approach II
• To check bits matrix can also be given as
• C=MP
• So no of parity check bits are n-k=>6-3=3
• No of Parity check bits C1,c2,c3
• And no. of message bits vector=k=3 ie M1,M2,M3
• Thus matrix representation0 will
1 1
be
• [c1,c2,c3]=[m1,m2,m3] 1 0 1
C1=m2(+)m3
1 1 0
C2=m1(+)m3
C3=m2(+)m1
K=3
2^k=8 combination is possible
 Final Vector codes
M1 M2 M3 C1 C2 c3
0 0 0 0 0 0
0 0 1 1 1 0
0 1 0 1 0 1
0 1 1 0 1 1
1 0 0 0 1 1
1 0 1 1 0 1
1 1 0 1 1 0
1 1 1 0 0 0
Some questions
Parity check Matrix
 Error Coding Block codes Hamming Codes

• Definition:
• They are the first class of Linear codes devised
for error control
• Hamming Codes are (n,k) codes with following
Properties
i. 2^q-1,q=no. of Parity Bits(n-k)
ii. K=2^q-q-1, k= no. of information bits
iii. Q>=3 i.e minimum no. of Parity bits=3
 (7,4) Hamming Code
4 bit information word is encoded into 7 bit Codeword. Three parity-check
bits are required

• Parity bits:- c=(i1,i2,i3,i4,p1,p2,p4)

i. p1=i1+i2+i3 eg codeword ->i=0101
ii. P2=i2+i3+i4 i1=0,i2=1,i3=0,i4=1
iii. P3=i1+i2+i4 p1=0+1+0=1
p2=1+0+1=0
p3=0+1+1=0
 Block code-error coding hamming codes

• Received word V=(v1,v2,v3,v4,v5,v6,v7)

• For error checking,3 parity check sum is used.

• I s1=(v1+v2+v3+v5 -> 0
• Ii. S2=(v2+v3+v4+v6 0
• S3=(v1+v2+v41
• Error syndrome S=(s1,s2,s3)
• S=001
 Syndrome table for (7,4) hamming code

• Error Pattern Error Syndrome(s)

c1 C2 c3 c4 c5 c6 c7 s1 s2 s3 valu
e
0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0 0 1 1
0 0 0 0 0 1 0 0 1 0 2
0 0 0 0 1 0 0 1 0 0 4
0 0 0 1 0 0 0 0 1 1 3
0 0 1 0 0 0 0 1 1 0 6
0 1 0 0 0 0 0 1 1 1 7
1 0 0 0 0 0 0 1 0 1 5
 Questions examples
• Q1-Let c7=0111010 be the codeword sent
over a channel if no error occurred than
receive word is V=c7=0111010
• V=0111010, calculate parity check sums.
• S=(v1+v2+v3)+v5=0+1+1+0=0
• S2=(v2+v3+v4)+v6=1+1+1+1=0
• S3=v1+v2+v4+v7=0+1+1+0=0
• S=0,0,0 No error occured
• Q2: Codeword C of(7,4) hamming code incurs a
single Error, so giving v,(1011001) find c
• S1=(v1+v2+v3+v5)=1+0+1+0=0
• S2=(v2+v3+v4+v6=0+1+1+0=0
• S3=(v1+v2+v4)+v7=1+0+1+1=1
• S=001
• E=0000001
• C=v+e
• 1 0 1 1 0 0 1
0 0 0 0 0 01
--------------------
1011000 ans