PRINCIPLES OF COMMUNICATIONS

LINEAR BLOCK CODES

(MÃ KHỐI TUYẾN TÍNH)
Instructor:
Name: Đoàn Bảo Sơn
Office: Faculty of Electrical - Electronics Engineering
Phone: 0913 706061
Email: sondb@vaa.edu.vn

1
 1. Introduction

 linear block codes:

• q-ary block code 𝓒(N, K): qk codewords (Ex: q = 2 → binary)
• q-ary codeword: N symbols
• each q-ary information word: K symbols
• linearity: linear combination of the K symbols of the information
word → N symbols of the codeword
 convolutional codes:
• rate: k/n
• codeword: n symbols (function of the current information word
+ given number of previous information words)
• current information word: k symbols
• length of the input and output sequences: infinite
 concatenated codes: concatenation of linear block codes or
convolutional codes.
2
 2. Finite fields
2.1. Fields
 Field F:
 non-empty set
 two operations: addition and multiplication
 satisfying the following axioms (tiên đề):
• F is an Abelian group under addition. It has the properties of
associativity, identity element written 0, symmetry and
commutativity;
• associative (kết hợp): if a,b,c ∊ F, then a(bc) = (ab)c;
• commutative (giao hoán): if a, b ∊ F, then ab = ba;
• right distributive and left distributive with respect to addition:
if a,b,c ∊ F, then a(b + c) = ab + ac and (a + b)c = ac + bc;
• the field has an identity element denoted as 1 for the
multiplication;
• each non-zero element of F is invertible; if a ∊ F (a ≠ 0), a - 1 is
its inverse with aa - 1 = 1. 3
 2. Finite fields
2.2. Finite fields
 Finite field (Galois Field)
 q elements: Fq or GF(q)
 q: prime number (số nguyên tố) or q = pm (p: prime number)
 q is a prime number ⇒ the addition and multiplication in the finite
field is the addition and multiplication modulo q
 elements: 0 and 1 ⇒ F2: the simplest finite field
 For digital communications: F2 and Fq (q = 2m)
 Example 1 – Addition and multiplication in F2

 Example 2 – Addition and multiplication in F5

4
 2. Finite fields
2.3. Irreducible and primitive polynomials
 f(p): polynomial the coefficients of which are elements of F2
f(p) = f0 + f1p + f2p2 + … + fmpm with fi ∊ F2
degree of the polynomial = highest non-zero power of p
m ≠ 0 ⇒ polynomial is of degree = m
Definition 3.1.
A polynomial is irreducible (tối giản) in if it cannot be written
as a product of polynomials b(p)c(p) with b(p) and c(p) of degree
higher or equal to 1.
Theorem 3.1.
All irreducible polynomials in F2 of degree m divide (p2m -1 −1).

5
 2. Finite fields
2.3. Irreducible and primitive polynomials
Definition 3.2: An irreducible polynomial f(p) of degree m is a
primitive (nguyên thủy) polynomial if the smallest positive integer n
for which f(p) divide pn+ 1 is n = 2m − 1
Example: List of primitive polynomials for m ≤ 16

List of the elements of the finite field F4:

6
 2. Finite fields
2.3. Finite field with 2m elements
 Finite field: Fq (q = 2m)
 Basis field: F2
 Irreducible polynomial in F2: f(p) (f(α) = 0)
 Successive powers of α generate the 2m − 1 non-zero elements of
the finite field Fq
 Example 1– Addition and multiplication in F4
• F4 (m = 2)
• primitive polynomial: f(p)= 1 + p + p2
• f(α) = 1 + α + α2 = 0
• successive powers of α → 22 − 1 non-zero elements of F4 (see
Ex. in 3.2.2.)

7
 2. Finite fields
2.3. Finite field with 2m elements
Addition and multiplication in F4

α2 = 1 + α, α3 = α + α2 = 1 and α4 = α
⇒ F4: finite field
 Example 2:
• F16 ( m = 4)
• irreducible polynomial: f(p) = 1 + p+ p4
• f(α) =1 + α + α4 = 0
• successive powers of α generate the 24 − 1 = 15 non-zero
elements of F16

8
 2. Finite fields
2.3. Finite field with 2m elements
List of elements of finite field F16 built from the irreducible polynomial
1 + p + p4

9
 2. Finite fields
2.3. Finite field with 2m elements
 At each non-zero element of the finite field with q = 2m, we
associate a minimal polynomial.
Definition 3.3. The minimal polynomial mi(p) is the polynomial with
the lowest degree for which αi is a root.
For the previous example, the 15 minimal polynomials are the
following:

Theorem 3.2. Each minimal polynomial mi(p) is irreducible.

10
 3. Linear block codes
3.1. Introduction
 q-ary linear block code C (N, K): qK codewords
 associate:
q-ary codeword composed of N symbols ⇒ q-ary information word
composed of K symbols.
 linearity: linear combination of the K symbols of the information
word ⇒ N symbols of the codeword
 coding operation in a matrix form.
 binary linear block codes: q = 2
 information word and codeword using vector
• information word: K elements of information
u = [u0, u1, …, uK-1]
uk ∊ Fq
• codeword: N elements
c = [c0, c1, …, cN-1]
cn ∊ Fq 11
 3. Linear block codes
3.1. Introduction
 Matrix: c = uG
c: associated codeword
u: information word
G: generator matrix (KxN)

IK: identity matrix (KxK); c = [u0, u1, …, uK-1, cK, cK+1,…, cN-1] 12
 3. Linear block codes
3.1. Introduction
 Example 1:
Repetition code C1 (3, 1) in F2
G = (1 1 1)
The information bit is repeated three times:

 Example 2:
Parity check code C2 (3, 2) in F2

c2: parity bit

c2 = u0 + u1
codeword: even number of 1 ⇒ C2: 000, 011, 110 and 101
13
 3. Linear block codes
3.1. Introduction
 Example 3:
Hamming code C3 (7, 4) in F2
• generator matrix:

• systematic generator matrix:

add lines 1, 2 and 3 → line 1 1000110
add lines 2, 3 and 4 → line 2 0100011
add line 3 and 4 → line 3. 𝐺=
0010111
last line → unchanged 000110 1
• three parity or redundancy bits:
c = u.G
u = [u0, u1, u2, u3]
14
 3. Linear block codes
3.1. Introduction
 Example 3 (cont.):
Hamming code C3 (7, 4) in F2
24 codewords → List of codewords for Hamming code C3 (7, 4)

15
 3. Linear block codes
3.1. Introduction
 Definition 3.4. The rate R of a block code (N, K) is equal to:
𝑵
𝑹=
𝑲

 Definition 3.5. Let c1 and c2 be two codewords of code C, α1 and α2

be two elements of the finite field. The linearity implies that α1c1 +
α2c2 is also a codeword of C. Consequently, the word c0 = *00 … 0+ is
always a codeword of any linear code. This codeword is called the
null codeword.

16
 3. Linear block codes
3.2. Minimum distance of a code
 Definition 3.6. Let c1 and c2 be two codewords of length N of the
code C, the Hamming distance dH(c1, c2) is equal of the number of
elements which are different.

 Definition 3.7. The Hamming weight w(c) of a binary codeword c is

equal to the number of non-zero elements of this codeword.

 Definition 3.8. The minimum distance dmin of the code C is the

Hamming distance between the pair of codewords with the
smallest Hamming distance:
𝒅𝒎𝒊𝒏 = min 𝒅𝑯 𝒄𝒊 , 𝒄𝒋
𝒊,𝒋,𝒊≠𝒋

17
 3. Linear block codes
3.2. Minimum distance of a code
 Definition 3.8. The minimum distance dmin of the code C is the
Hamming distance between the pair of codewords with the
smallest Hamming distance:
𝒅𝒎𝒊𝒏 = min 𝒅𝑯 𝒄𝒊 , 𝒄𝒋
𝒊,𝒋,𝒊≠𝒋
When the code is linear:
Minimum distance dmin is equal to the minimum Hamming weight of
the code C (by excluding the null codeword c0):
𝒅𝒎𝒊𝒏 = min 𝒘 𝒄𝒊
𝒊,𝒊≠𝟎
 Note: The minimum distance was the unique criterion to evaluate
the performance of error correcting codes

18
 3. Linear block codes
3.3. Parity check matrix
 Block linear code (N, N − K) associated with each block linear code
C (N, K).
 H: generator matrix of this dual code
 Each of the codewords c of the code C is orthogonal to all the
codewords of the dual code
𝐜. 𝐇𝐓 = 𝟎
 Relation between the generator matrix G of the code C and H:
𝐆. 𝐇𝐓 = 𝟎
 G: systematic ⇒ H: parity check matrix of the code C.

19
 3. Linear block codes
3.3. Parity check matrix (cont.)
 Example: The parity check matrix of the Hamming code C3 (7, 4)

Each of the three lines of the parity check matrix correspond to a

parity check equation (addition modulo 2 in the case of binary code)
involving the different bits of the codewords c = (c0 c1 c2 c3 c4 c5 c6) =
(u0 u1 u2 u3 c4 c5 c6)

20
 3. Linear block codes
3.4. Hamming codes
 Hamming codes: perfect binary linear block codes (N, K)
 N = 2J − 1; K = 2J− 1 − J
 Parity check matrix H: dimension J × N (J = N − K)
 Columns of H: N binary non-null vectors containing J elements
 For example, for J = 3:
• Hamming code: (7, 4) code
• parity check matrix

21
 3. Linear block codes
3.5. Decoding of binary linear block codes
 r=c+e
• r: received word
• c: transmitted codeword
• e: error vector
 Straightforward approach: compare the received word r with all
the 2K codewords of the code C
 Error syndrome s: dimension 1 × (N − K)
s = rHT = cHT + eHT = eHT (since: cHT = 0)
 If there is no transmission error, the error syndrome s is the
null vector
 Decoding methods:
• the standard array,
• the syndrome decoding
• and the Viterbi algorithm.
22
 3. Linear block codes
3.5. Decoding of binary linear block codes
 Syndrome decoding
 error syndrome: 2N-K different values
 s = rHT
 syndrome → vector ê.
 This method only requires a look-up table that associates the error
syndromes with the error vectors.
Syndrome table
 Estimated codeword:
𝐜=𝐫+𝐞
⇒ Having to perform the product rHT
and memorize 2N-K error vectors

23
 3. Linear block codes
3.5. Decoding of binary linear block codes
 Syndrome decoding (cont.)
 Example:
 code C4 (5, 2)
 generator matrix:

 associated parity check matrix:

24
 3. Linear block codes
3.5. Decoding of binary linear block codes
 Syndrome decoding (cont.)
 Example (cont.):
 code C4 (5, 2)
 information word u = [1 1]
 associated codeword: c = [1 1 1 1 0]
 received vector: r = [1 1 1 0 0]
 calculate error syndrome → Syndrome decoding
table for the (5, 2) code
Syndrome associated with the received word r:
s = rHT = [0 1 0]
Syndrome decoding table ⇒ ê = [0 0 0 1 0]
𝐜 = 𝐫 + 𝐞 ⇒ ĉ = [1 1 1 1 0]

25
 4. Cyclic codes
4.1. Definition and properties
Note. – In this section, the most significant bit (MSB) of the vectors is
on the right.
 Cyclic codes: subset of the linear block codes
 For linear block codes: K codewords are required to determine the
set of 2K codewords
 For cyclic codes, only one codeword is enough.
 Cyclic codes: finite field F2 (Fq)
 Main property of the cyclic codes:
• codeword: c = [c0 c1 … cN - 2 cN – 1]
• right cyclic shift of one position: c’ = *cN - 1 c0 … cN - 3 cN - 2]
c’: is also a codeword
• cyclic code (N, K)
• associate a polynomial c(p)

26
 4. Cyclic codes
4.1. Definition and properties (cont.)
 Properties of the cyclic codes can be obtained easily using the
algebra of the polynomials modulo pN − 1.

 adding cN - 1 and substracting cN – 1:

 polynomial c′(p) associated with the codeword c’:

 pc(p) modulo pN − 1:

 right cyclic shift of 1 position ⇔ multiplication by p modulo pN − 1

 right cyclic shift of i positions ⇔ multiplication by pi modulo pN −27 1
 4. Cyclic codes
4.2. Properties of the cyclic codes
 If c(p) is a polynomial of degree N − 1 associated with a codeword
of a cyclic code (N, K), then:

is also a polynomial associated with a codeword

 It is possible to build a cyclic code (N, K) from the generator
polynomial denoted as g(p) of degree N − K:

g(p) is the polynomial with minimum degree among the 2K codewords

of the cyclic code
 The polynomial g(p) of degree N − K should be a factor of pN−1.

28
 4. Cyclic codes
4.2. Properties of the cyclic codes (cont.)
 The set of the 2K polynomials associated with the 2K codewords of
a cyclic code (N, K) can be obtained by performing the
multiplication of g(p) by the 2K polynomials of degree lower or
equal to K − 1.
u(p): associated with the information word u
u = [u0 u1 … uK-2 uK-1]

 Each polynomial factor of pN − 1 can generate a cyclic code

Example: p7 − 1 can be decomposed into the product of three
irreducible polynomials ⇒ 3 cyclic codes for N = 7
– a code (7, 6) with g(p) = 1 + p;
– a code (7, 4) with g(p) = 1 + p + p3;
2 3 Hamming codes
– a code (7, 4) with g(p) = 1 + p + p . 29
 4. Cyclic codes
4.2. Properties of the cyclic codes (cont.)
 Generator matrix: G(N x K)
• linear block code: obtained from K independent codewords
• cyclic codes: choose the codewords associated with the
following polynomials:
g(p); g(p).p; g(p).p2; …; g(p).pN-K
• Example: Hamming code (7, 4) with g(p) = 1 + p + p3
⇒ generator matrix: G(7 x 4)

⇒ generator matrix is not systematic

30
 4. Cyclic codes
4.2. Properties of the cyclic codes (cont.)
 Systematic code:
• information word: u = [u0 u1 … uK-2 uK-1]
• associated polynomial: u(p) = u0 + u1p + … + uK-2pK-2 + uK-1pK-1

To obtain a systematic codeword c(p) = q(p)g(p) we have to:

– multiply the polynomial u(p) by pN-K;
– perform the division of pN-Ku(p) by g(p) to obtain the remainder t(p);
– add the remainder t(p) to pN-Ku(p):

31
 4. Cyclic codes
4.2. Properties of the cyclic codes (cont.)
 Example: Hamming code C (7,4)
• generator polynomial: g(p) = 1 + p + p3
• information word: u = [1 1 1 0]
• polynomial associated with u:
u(p) = 1 + 1p + 1p2 + 0p3
pN-Ku(p) = p3 + p4 + p5
t(p) = p

⇒ associated codeword:

32
 4. Cyclic codes
4.3. Encoding of cyclic codes
 Cyclic encoder: modulo 2 adders and shift registers
 Implementation of a division of polynomials
a(p) = g(p)q(p) + t(p)
a(p): dividend polynomial
g(p): divisor polynomial
q(p): quotient
t(p): remainder
Ex: Division of
1 + p2 + p6 by 1 + p + p2

33
 4. Cyclic codes
4.3. Encoding of cyclic codes
 Hardware structure of the division by 1 +p + p2 :

 Hardware scheme composed of shift registers

 Bits enter in the structure starting by the MSB

34
 4. Cyclic codes
4.3. Encoding of cyclic codes
 Sequencing of the division

35
 4. Cyclic codes
4.4. Decoding of cyclic codes
The decoding of cyclic codes is composed of two phases:
– syndrome calculation;
– error localization.
 Syndrome calculation: dividing the polynomial associated with the
received word r(p) by g(p)
r(p) = q(p)g(p) + s(p)
The syndrome is the remainder of this division:
– if s(p) = 0, then the received word r(p) is a codeword;
– the syndrome s(p) is a polynomial of degree less or equal to N − K − 1.
From the value of s(p) it is possible to estimate the error vector ê(p).
𝑐 𝑝 = 𝑟 𝑝 + 𝑒(𝑝)
𝑐 𝑝 : error vector
r(p): received vector
𝑒 𝑝 : estimated error vector
36
 4. Cyclic codes
4.4. Decoding of cyclic codes
 Correction of a single error
 Single error: e(p) = pj
 Syndrome:
s(p) = pj mod g(p)
 Hardware structure of the decoder for the Hamming code (7, 4):

37
 4. Cyclic codes
4.4. Decoding of cyclic codes
 Correction of a single error
 Generator polynomial: g(p) = 1 + p + p3
 The result of the multiplication of the Syndrome table
syndrome s(p) by pN-j is equal to 1:
s(p)pN-j = pjpN-j = pN = 1
⇒ To determine the position of the error,
we just have to successively multiply s(p)
by p until the result is equal to 1.

38
 4. Cyclic codes
4.4. Decoding of cyclic codes
 Correction of a single error (cont.)
 Example:
• Codeword: c = [1 1 1 1 1 1 1]
• Error vector e = [0 0 0 0 1 0 0]
• e(p) = p4 (a single error occurs on the fifth bit from the LSB)

39