Professional Documents
Culture Documents
3F4 Error Control Coding: Dr. I. J. Wassell
3F4 Error Control Coding: Dr. I. J. Wassell
Dr. I. J. Wassell
Introduction
• Error Control Coding (ECC)
– Extra bits are added to the data at the
transmitter (redundancy) to permit error
detection or correction at the receiver
– Done to prevent the output of erroneous bits
despite noise and other imperfections in the
channel
– The positions of the error control coding and
decoding are shown in the transmission model
Transmission Model
Error Modulator
Digital Source
Control
Line X(w)
(Transmit
Source Encoder Coding
Coding Filter, etc)
Hc(w) Channel
Transmitter
N(w) Noise
+
Error Demod
Digital Source Line
Control (Receive
Sink Decoder
Decoding
Decoding
Filter, etc)
Y(w)
Receiver
Error Models
• Binary Symmetric Memoryless Channel
– Assumes transmitted symbols are binary
– Errors affect ‘0’s and ‘1’s with equal
probability (i.e., symmetric)
– Errors occur randomly and are independent
from bit to bit (memoryless)
1-p
0 0 p is the probability of
p
bit error or the Bit
IN OUT Error Rate (BER) of
p
1 1 the channel
1-p
Error Models
• Many other types
• Burst errors, i.e., contiguous bursts of bit
errors
– output from DFE (error propagation)
– common in radio channels
– Insertion, deletion and transposition errors
• We will consider mainly random errors
Error Control Techniques
• Error detection in a block of data
– Can then request a retransmission, known as
automatic repeat request (ARQ) for sensitive
data
– Appropriate for
• Low delay channels
• Channels with a return path
– Not appropriate for delay sensitive data, e.g.,
real time speech and data
Error Control Techniques
• Forward Error Correction (FEC)
– Coding designed so that errors can be corrected
at the receiver
– Appropriate for delay sensitive and one-way
transmission (e.g., broadcast TV) of data
– Two main types, namely block codes and
convolutional codes. We will only look at block
codes
Block Codes
• We will consider only binary data
• Data is grouped into blocks of length k bits
(dataword)
• Each dataword is coded into blocks of
length n bits (codeword), where in general
n>k
• This is known as an (n,k) block code
Block Codes
• A vector notation is used for the datawords
and codewords,
– Dataword d = (d1 d2….dk)
– Codeword c = (c1 c2……..cn)
• The redundancy introduced by the code is
quantified by the code rate,
– Code rate = k/n
– i.e., the higher the redundancy, the lower the
code rate
Block Code - Example
• Dataword length k = 4
• Codeword length n = 7
• This is a (7,4) block code with code rate = 4/7
• For example, d = (1101), c = (1101001)
Error Control Process
Source code Codeword
data chopped (n bits)
101101 1000 into blocks
Channel
1000
Dataword coder
(k bits)
Codeword +
Dataword possible errors
(k bits) (n bits)
Channel Channel
decoder
Error flags
Error Control Process
• Decoder gives corrected data
• May also give error flags to
– Indicate reliability of decoded data
– Helps with schemes employing multiple layers
of error correction
Parity Codes
• Example of a simple block code – Single
Parity Check Code
– In this case, n = k+1, i.e., the codeword is the
dataword with one additional bit
– For ‘even’ parity the additional bit is,
q i 1 di (mod 2)
k
Dataword Codeword
0 0 0 0 0 0 0
0 0 1 0 0 1 1
0 1 0 0 1 0 1
0 1 1 0 1 1 0
1 0 0 1 0 0 1
1 0 1 1 0 1 0
1 1 0 1 1 0 0
1 1 1 1 1 1 1
Parity Codes
• To decode
– Calculate sum of received bits in block (mod 2)
– If sum is 0 (1) for even (odd) parity then the dataword is the
first k bits of the received codeword
– Otherwise error
• Code can detect single errors
• But cannot correct error since the error could be in
any bit
• For example, if the received dataword is (100000) the
transmitted dataword could have been (000000) or
(110000) with the error being in the first or second
place respectively
• Note error could also lie in other positions including
the parity bit
Parity Codes
• Known as a single error detecting code
(SED). Only useful if probability of getting
2 errors is small since parity will become
correct again
• Used in serial communications
• Low overhead but not very powerful
• Decoder can be implemented efficiently
using a tree of XOR gates
Hamming Distance
• Error control capability is determined by the
Hamming distance
• The Hamming distance between two
codewords is equal to the number of
differences between them, e.g.,
10011011
11010010 have a Hamming distance = 3
• Alternatively, can compute by adding
codewords (mod 2)
=01001001 (now count up the ones)
Hamming Distance
• The Hamming distance of a code is equal to
the minimum Hamming distance between
two codewords
• If Hamming distance is:
1 – no error control capability; i.e., a single error
in a received codeword yields another valid
codeword
XXXXXXX X is a valid codeword
Note that this representation is diagrammatic
only.
In reality each codeword is surrounded by n
codewords. That is, one for every bit that
could be changed
Hamming Distance
• If Hamming distance is:
2 – can detect single errors (SED); i.e., a single
error will yield an invalid codeword
XOXOXO X is a valid codeword
O in not a valid codeword
See that 2 errors will yield a valid (but
incorrect) codeword
Hamming Distance
• If Hamming distance is:
3 – can correct single errors (SEC) or can detect
double errors (DED)
XOOXOOX X is a valid codeword
O in not a valid codeword
See that 3 errors will yield a valid but
incorrect codeword
Hamming Distance - Example
• Hamming distance 3 code, i.e., SEC/DED
– Or can perform single error correction (SEC)
X is a valid codeword
O is an invalid codeword
Hamming Distance
• The maximum number of detectable errors is
dmin 1
• That is the maximum number of correctable errors
is given by,
d min 1
t
2
where dmin is the minimum Hamming distance
between 2 codewords and . means the smallest
integer
Linear Block Codes
• As seen from the second Parity Code
example, it is possible to use a table to hold
all the codewords for a code and to look-up
the appropriate codeword based on the
supplied dataword
• Alternatively, it is possible to create
codewords by addition of other codewords.
This has the advantage that there is now no
longer the need to held every possible
codeword in the table.
Linear Block Codes
• If there are k data bits, all that is required is to
hold k linearly independent codewords, i.e., a set
of k codewords none of which can be produced by
linear combinations of 2 or more codewords in the
set.
• The easiest way to find k linearly independent
codewords is to choose those which have ‘1’ in
just one of the first k positions and ‘0’ in the other
k-1 of the first k positions.
Linear Block Codes
• For example for a (7,4) code, only four
codewords are required, e.g.,
1 0 0 0 1 1 0
0 1 0 0 1 0 1
0 0 1 0 0 1 1
0 0 0 1 1 1 1
c3 c1 c2
Linear Block Codes
• 0 is always a codeword, i.e.,
Since all zeros is a dataword then,
k
c 0 ai 0
i 1
Error Correcting Power of LBC
• The Hamming distance of a linear block code (LBC)
is simply the minimum Hamming weight (number of
1’s or equivalently the distance from the all 0
codeword) of the non-zero codewords
• Note d(c1,c2) = w(c1+ c2) as shown previously
• For an LBC, c1+ c2=c3
• So min (d(c1,c2)) = min (w(c1+ c2)) = min (w(c3))
• Therefore to find min Hamming distance just need to
search among the 2k codewords to find the min
Hamming weight – far simpler than doing a pair
wise check for all possible codewords.
Linear Block Codes – example 1
• For example a (4,2) code, suppose;
1 0 1 1 a1 = [1011]
G
0 1 0 1 a2 = [0101]
• For d = [1 1], then;
1 0 1 1
0 1 0 1
c
_ _ _ _
1 1 1 0
Linear Block Codes – example 2
• A (6,5) code with
1 0 0 0 0 1
0 1 0 0 0 1
G 0 0 1 0 0 1
0 0 0 1 0 1
0 0 0 0 1 1
• Is an even single parity code
Systematic Codes
• For a systematic block code the dataword
appears unaltered in the codeword – usually
at the start
• The generator matrix has the structure,
k R R=n-k
1 0 .. 0 p11 p12 .. p1R
0 1 .. 0 p21 p22 .. p2 R
G I | P
.. .. .. .. .. .. .. ..
0 0 .. 1 pk1 pk 2 .. pkR
and so,
k k
b j .c b j . d i a i d i (a i .b j ) 0
i 1 i 1
c3
b1
Error Syndrome
• For error correcting codes we need a method to
compute the required correction
• To do this we use the Error Syndrome, s of a
received codeword, cr
s = crHT
• If cr is corrupted by the addition of an error vector,
e, then
cr = c + e
and
s = (c + e) HT = cHT + eHT
s = 0 + eHT
Syndrome depends only on the error
Error Syndrome
• That is, we can add the same error pattern to
different codewords and get the same syndrome.
– There are 2(n - k) syndromes but 2n error patterns
– For example for a (3,2) code there are 2 syndromes and
8 error patterns
– Clearly no error correction possible in this case
– Another example. A (7,4) code has 8 syndromes and
128 error patterns.
– With 8 syndromes we can provide a different value to
indicate single errors in any of the 7 bit positions as
well as the zero value to indicate no errors
• Now need to determine which error pattern caused
the syndrome
Error Syndrome
• For systematic linear block codes, H is
constructed as follows,
G = [ I | P] and so H = [-PT | I]
where I is the k*k identity for G and the
R*R identity for H
• Example, (7,4) code, dmin= 3
1 0 0 0 0 1 1
0 0 1 1 1 1 0 0
1
G I | P
0
1
0
0
1
0
0
1
1
0
1 0
H - P T | I 1 0 1 1 0 1 0
1 1 0 1 0 0 1
0 0 0 1 1 1 1
Error Syndrome - Example
• For a correct received codeword cr = [1101001]
In this case,
0 1 1
1 0 1
1 1 0
s c r H T 1 1 0 1 0 0 11 1 1 0 0 0
1 0 0
0 1 0
0 1
0
Error Syndrome - Example
• For the same codeword, this time with an
error in the first bit position, i.e.,
cr = [1101000]
0 1 1
1 0 1
1 1 0
s c r H T 1 1 0 1 0 0 01 1 1 0 0 1
1 0 0
0 1 0
0 1
0
c1 (all zero) c2 …… cM s0
e1 c2+e1 …… cM+e1 s1 All patterns in
e2 c2+e2 …… cM+e2 s2 row have same
e3 c2+e3 …… cM+e3 s3 syndrome
… …… …… …… … Different rows
eN c2+eN …… cM+eN sN have distinct
syndromes
cr c
+
Standard Array
• For the same received codeword c2 + e3, note that the
unique syndrome is s3
• This syndrome identifies e3 as the corresponding error
pattern
• So if we calculate the syndrome as described
previously, i.e., s = crHT
• All we need to do now is to have a relatively small
table which associates s with their respective error
patterns. In the example s3 will yield e3
• Finally we subtract (or equivalently add in modulo 2
arithmetic) e3 from the received codeword (c2 + e3) to
yield the most likely codeword, c2
Hamming Codes
• We will consider a special class of SEC codes
(i.e., Hamming distance = 3) where,
– Number of parity bits R = n – k and n = 2R – 1
– Syndrome has R bits
– 0 value implies zero errors
– 2R – 1 other syndrome values, i.e., one for each
bit that might need to be corrected
– This is achieved if each column of H is a different
binary word – remember s = eHT
Hamming Codes
• Systematic form of (7,4) Hamming code is,
1 0 0 0 0 1 1
0 0 1 1 1 1 0 0
1
G I | P
0
1
0
0
1
0
0
1
1
0
1 0
H - P T | I 1 0 1 1 0 1 0
1 1 0 1 0 0 1
0 0 0 1 1 1 1
e = 0010000
cr= 0100011