Are you sure?
This action might not be possible to undo. Are you sure you want to continue?
HAMMING CODES
Hamming code with matrix manipulation
Consider the four unencoded information bits
3 2 1
, , u u u
and
4
u . This 1 x 4 matrix can be represented as;
   
4 3 2 1
u u u u u =
The sevenbit encoded data is represented as;
   
7 6 5 4 3 2 1
v v v v v v v v =
The encoding process of the Hamming code can be
represented as modulo2 multiplication;
   
7 6 5 4 3 2 1 4 3 2 1
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
v v v v v v v u u u u =
(
(
(
(
¸
(
¸

where the 4 x 7 matrix is known as a generator matrix
represented by   G .
Note:
If we examine carefully this multiplication operation it is
consistent with Eq (1) above.
EEE332 – G: Hamming code Part 2 2/17 October 2008
We can represent the encoding process by multiplying our
information vector   u by a generator matrix   G to obtain
the encoded bit sequence   v .
     G u v = (2)
EXAMPLE 3
Encoding by matrix multiplication
A source outputs the four information bits 1100. These
four bits are to be encoded using the Hamming code just
described. Using the generator matrix in Eq (2), determine
the encoded sevenbit pattern to be transmitted.
Solution
EEE332 – G: Hamming code Part 2 3/17 October 2008
Let’s take a closer look on generator matrix   G for our
Hamming code. From the above
 
(
(
(
(
¸
(
¸
=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
G
We can see that the generator matrix has the following
properties;
1. The matrix is k rows by n columns: where k=4, the
number of information bits from the source; and n=7,
the total number of encoced bits.
2. The last four columns of   G form an identity matrix.
The identity matrix maps the four information bits into
the last four bits of the encoded sequence (sevenbit).
3. The first three columns of   G shows the relationship
between the parity (check) bits with data bits.
4. The generator matrix is not unique. For example, we
can develop an equally valid Hamming code by
switching the first and second columns in  G .
EEE332 – G: Hamming code Part 2 4/17 October 2008
Now let’s deconstruct the generator matrix into two
submatrices as i.e. a 4 x 4 identity submatrix   I and 4 x 3
parity submatrix   P (which produce the parity bits).
 
(
(
(
(
(
¸
(
¸
=
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 1
1 1 1
1 1 0
0 1 1
G
       
4 4x
I P G =
Figure 1: Deconstruction of generator matrix   G
Terminology
 Systematic code – A code in which the information
bits are always repeated in a portion of the encoded
sequence (the input data are embedded in the
encoded output). A nonsystematic code is the one in
which the output does not contain the input bits.
4 x 3 parity submatrix  P
4 x 4 identity submatrix   I
EEE332 – G: Hamming code Part 2 5/17 October 2008
The systematic code is very much desirable because
they make it easy to extract the original data from the
encoded sequence.
 Block code – divides the bit stream from the source
encoder into kbit blocks. Each kbit blocks is then
encoded into an nbit block prior to transmission.
The Hamming code that we’ve just developed is a block
code which takes data stream from source encoder,
divides it into fourbit block, and then encodes each
fourbit block into a sevenbit block prior to
transmission.
Block codes are described in term of n and k, stated as
( ) k n, . The code we’ve developed is called a ( ) 4 , 7
Hamming code.
The code rate given as ( ) r ;
n
k
r =
Sometimes the Hamming code above is also called
“rate foursevenths” code
EEE332 – G: Hamming code Part 2 6/17 October 2008
Hamming code – (15, 11)
Suppose we want to develop a Hamming code that
provides singleerror detection and correction capabilities
for a larger block of data.
Suppose we want to use four parity bits (instead of three)
– we can produce eleven (11) different combinations of
those four bits with two or more “1”s. Therefore, we can
use four parity bits to provide singleerror detection and
correction capabilities for a block of 11 information bits.
(Why 11 information data bits? – check previous matrix
multiplication).
Thus the generator matrix for the (15, 11) Hamming code
is given as follows;
 
(
(
(
(
(
(
(
(
(
(
(
(
(
(
(
¸
(
¸
=
1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
0 1 0 0 0 0 0 0 0 0 0 1 1 1 0
0 0 1 0 0 0 0 0 0 0 0 1 1 0 1
0 0 0 1 0 0 0 0 0 0 0 1 0 1 1
0 0 0 0 1 0 0 0 0 0 0 0 1 1 1
0 0 0 0 0 1 0 0 0 0 0 1 1 0 0
0 0 0 0 0 0 1 0 0 0 0 1 0 1 0
0 0 0 0 0 0 0 1 0 0 0 0 1 1 0
0 0 0 0 0 0 0 0 1 0 0 1 0 0 1
0 0 0 0 0 0 0 0 0 1 0 0 1 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 1 1
G
EEE332 – G: Hamming code Part 2 7/17 October 2008
This new (15, 11) code is not as powerful as our (7, 4)
code for protecting information since we can only protect
the information from only singlebit error in 11
information bits.
If we break the 11 bit information data into three fourbit
blocks, and use the (7, 4) code in order to get better
protection.
However, the new code provides less overhead i.e. only
four parity bits required for 11 information data bits.
If we break up into three fourbit data will end up with 3 x
3 = 9 parity bits.
EEE332 – G: Hamming code Part 2 8/17 October 2008
Decoding the Hamming code
Use the (7, 4) code for decoding, error detection and
correction
As discussed the matrix   G can be decomposed into a 4 x
3 parity matrix   P and 4 x 4 identity matrix   I
       
4 4×
= I P G
Let’s consider forming a new 3 x 7 matrix   H
       
T
P I H
3 3×
=
Note that  
T
P becomes a matrix of dimension 3 x 4.
Transpose is to turn the rows into column or columns into
rows.
The matrix is   H is thus
 
(
(
(
¸
(
¸
=
1 1 1 0
0 1 1 1
1 1 0 1
1 0 0
0 1 0
0 0 1
H
Figure 2: deconstruction of matrix   H
3x3 identity submatrix   I 3x4 transpose of parity matrix   P
EEE332 – G: Hamming code Part 2 9/17 October 2008
Now let   r represents the received codeword
   
7 6 5 4 3 2 1
v v v v v v v r =
Suppose we post multiply the received codeword by the
transpose of   H
    
 
 
ex ex ex
T
v v v v v v
v v v v v v v v v v v v
v v v v v v v H r
3 3 2 2 1 1
7 6 5 3 6 5 4 2 7 6 4 1
7 6 5 4 3 2 1
1 0 1
1 1 1
1 1 0
0 1 1
1 0 0
0 1 0
0 0 1
© © ©
= © © © © © © © © ©
=
(
(
(
(
(
(
(
(
(
¸
(
¸
 =
The resulting 1 x 3 matrix is called the syndrome, tells us
which expected check bits agree with their received
counterpart and which check bit do not.
ex ex
v v if v v
1 1 1 1
; 0 = = ©
ex ex
v v if v v
1 1 1 1
; 1 = = ©
Similar results are obtained for
ex
v v
2 2
© and
ex
v v
3 3
©
EEE332 – G: Hamming code Part 2 10/17 October 2008
As we’ve already seen, this agreement or disagreement is
exactly what we need for singlebit error detection and
correction in the received sevenbit sequence.
Let’s use   s to represent the threebit syndrome. Thus,
we can crate a table similar to the Table 2 (Note F).
  s Bit received in Error
  0 0 0 None
  0 0 1
1
v
  0 1 0
2
v
  1 0 0
3
v
  0 1 1
4
v
  1 1 0
5
v
  1 1 1
6
v
  1 0 1
7
v
EEE332 – G: Hamming code Part 2 11/17 October 2008
What is  
T
H ? Let’s look at it deeper. Figure 3 below
shows the deconstruction of the matrix  
T
H
 
(
(
(
(
(
(
(
(
(
(
¸
(
¸
=
1 0 1
1 1 1
1 1 0
0 1 1
1 0 0
0 1 0
0 0 1
T
H
Figure 3: Deconstruction of decoding matrix  
T
H
Deconstruction of  
T
H into a 3 x 3 identity submatrix   I
and 4 x 3 parity submatrix   P .
The parity submatrix produces the calculated check bits
ex ex
v v
2 1
and
ex
v
3
when modulo2 multiplication involves
the first, second, and third column of  
T
H , respectively.
The identity submatrix causes the exclusiveORing of the
first, second, and third received bit with the calculated
check bit.
3x3 identity submatrix   I
4x3 parity submatrix   P
EEE332 – G: Hamming code Part 2 12/17 October 2008
The exclusiveORing produces 0 if the received and
calculated check bits agree or 1 if they disagree, since
¹
´
¦
=
=
= ©
B A if
B A if
B A
1
0
Multiplication using  
T
H followed by syndrome decoding
using Table 1 above is therefore equivalent to the
decoding process we originally described in Example 1
(Note F). Thus  
T
H is called the decoding matrix.
EXAMPLE 1
Using matrix operations to detect and correct an error –
part 1.
As in Example 1 (Note F) a communication system
employs the (7, 4) Hamming code described in Eq. 1
(Note F), is received.
Use matrix operations to determine if an error occurred
and then determine the original four bits from the source.
Solution
EEE332 – G: Hamming code Part 2 13/17 October 2008
Geometric Interpretation of Error Control Coding
Let’s begin by considering an encoder that uses majority
voting, like in the previous example.
Let’s use onebit blocks of information. We can represent
the information sequence (one bit) as
   
1
u u =
and the encoded sequence (three bits) as
   
3 2 1
v v v v =
where
1 3
1 2
1 1
u v
u v
u v
=
=
=
Figure below shows the encoder;
Unencoded onebit sequence
u
1
Channel
encoder
Figure 3: Threebit channel encoder
Encoded threebit sequence
v
1
v
2
v
3
EEE332 – G: Hamming code Part 2 14/17 October 2008
Let’s consider the information bit u
1
to be a point in a
onedimensional vector space as shown in Figure 4.
Now let’s represent the encoded threebit sequence v
1
v
2
v
3
as a point in three dimensional space that represent
possible values for the encoded threebit
sequence i.e. (0,
0, 0) and (1, 1, 1) as shown in Figure 5.
Note:
The encoding process maps a point from onedimension
vector space ( )
1
u into a point in threedimensional vector
space ( )
3 2 1
, , v v v .
1
v
3
v
2
v
(0,0,0)
(1,1,1)
Figure 5: Vector space representation of the output
0 1
u
1
Figure 4: vector space representation of a single input
EEE332 – G: Hamming code Part 2 15/17 October 2008
Output sequence – threebit received sequence as a point
in threedimensional vector space ( )
3 2 1
, , v v v .
Figure 6 shows the received 3bit sequence with eight
possible received bits sequences.
Two of these eight points (black) represent valid
codeword (signify no error has occurred in transmission).
If we receive (1,1,1) we assume the information bit from
the source is 1, and if we receive (0,0,0), we assume that
the information bit from the source is 0.
The other six points represent errors (invalid codewords)
in the received bit sequence i.e. (0,0,1), (0,1,0), (0,01,1),
(1,0,0), (1,0,1), and (1,1,0).
Figure 6: vector space representation of the
received bit sequence
EEE332 – G: Hamming code Part 2 16/17 October 2008
If two bits are “1”s  assume the sequence (1, 1, 1) was
transmitted and that one bit error occurred
If two bits are “0”s  assume the sequence (0, 0, 0) was
transmitted and that one bit error occurred
As shown in Figure 6 this process of finding the correct
bits transmitted as finding the black dot (valid codewords)
closest to the gray dot (invalid codewords) i.e.
geometrically closest to it.
Finding the closest point in space is equivalent to
assuming to the least possible number of errors has
occurred in the received codeword.
Observed the following;
1. All onebit errors result in points corresponding to
invalid codewords (gray dots), so all singlebit errors
can be detected. In Figure 6, any onebit error is
closer to the correct valid codeword than it is to any
incorrect valid codeword – for example, the point (1,
1, 0) is geometrically closer to (1, 1, 1) than it is to
(0, 0, 0) – so all onebit error can be corrected.
2. All twobit errors also correspond to invalid
codewords (gray dots), so all can be detected.
However, the point is closer to the incorrect valid
codeword to the correct valid codeword. For
example, if (1, 1, 1) is transmitted and the first two
bits are received in error, then (0, 0, 1) is received.
EEE332 – G: Hamming code Part 2 17/17 October 2008
Since (0, 0, 1) is closer to (0, 0, 0) than it is to (1, 1,
1). If we try to correct the error will end up with (0,
0, 0) which gives incorrect value.
3. All threebit errors result in received sequences
representing valid codewords, so threebit errors
cannot even be detected.
Using the generator matrix in Eq (2).We can represent the encoding process by multiplying our information vector u by a generator matrix G to obtain the encoded bit sequence v . Solution EEE332 – G: Hamming code Part 2 2/17 October 2008 . v u G EXAMPLE 3 Encoding by matrix multiplication (2) A source outputs the four information bits 1100. determine the encoded sevenbit pattern to be transmitted. These four bits are to be encoded using the Hamming code just described.
4. 3. The matrix is k rows by n columns: where k=4. The identity matrix maps the four information bits into the last four bits of the encoded sequence (sevenbit). EEE332 – G: Hamming code Part 2 3/17 October 2008 . The first three columns of G shows the relationship between the parity (check) bits with data bits. 1. the number of information bits from the source. 2. and n=7. we can develop an equally valid Hamming code by switching the first and second columns in G . For example. the total number of encoced bits. The last four columns of G form an identity matrix. From the above 1 0 G 1 1 1 0 1 0 0 0 1 1 0 1 0 0 1 1 0 0 1 0 0 1 0 0 0 1 We can see that the generator matrix has the following properties.Let’s take a closer look on generator matrix G for our Hamming code. The generator matrix is not unique.
EEE332 – G: Hamming code Part 2 4/17 October 2008 . 1 0 G 1 1 1 0 1 0 0 0 1 1 0 1 0 0 1 1 0 0 1 0 0 1 0 0 0 1 4x4 G P I 4 x 3 parity submatrix P 4 x 4 identity submatrix I Figure 1: Deconstruction of generator matrix G Terminology Systematic code – A code in which the information bits are always repeated in a portion of the encoded sequence (the input data are embedded in the encoded output). A nonsystematic code is the one in which the output does not contain the input bits. a 4 x 4 identity submatrix I and 4 x 3 parity submatrix P (which produce the parity bits).Now let’s deconstruct the generator matrix into two submatrices as i.e.
stated as n. The code rate given as r . 4 Hamming code.The systematic code is very much desirable because they make it easy to extract the original data from the encoded sequence. r k n Sometimes the Hamming code above is also called “rate foursevenths” code EEE332 – G: Hamming code Part 2 5/17 October 2008 . Each kbit blocks is then encoded into an nbit block prior to transmission. k . Block code – divides the bit stream from the source encoder into kbit blocks. The Hamming code that we’ve just developed is a block code which takes data stream from source encoder. divides it into fourbit block. The code we’ve developed is called a 7 . Block codes are described in term of n and k. and then encodes each fourbit block into a sevenbit block prior to transmission.
Hamming code – (15. we can use four parity bits to provide singleerror detection and correction capabilities for a block of 11 information bits. Thus the generator matrix for the (15. 11) Hamming code is given as follows. Suppose we want to use four parity bits (instead of three) – we can produce eleven (11) different combinations of those four bits with two or more “1”s. Therefore. 11) Suppose we want to develop a Hamming code that provides singleerror detection and correction capabilities for a larger block of data. (Why 11 information data bits? – check previous matrix multiplication). 1 1 1 0 0 G 0 1 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 0 0 0 0 1 6/17 October 2008 EEE332 – G: Hamming code Part 2 .
This new (15. 4) code for protecting information since we can only protect the information from only singlebit error in 11 information bits. If we break the 11 bit information data into three fourbit blocks. EEE332 – G: Hamming code Part 2 7/17 October 2008 . However.e. If we break up into three fourbit data will end up with 3 x 3 = 9 parity bits. only four parity bits required for 11 information data bits. 4) code in order to get better protection. the new code provides less overhead i. 11) code is not as powerful as our (7. and use the (7.
4) code for decoding. Transpose is to turn the rows into column or columns into rows. The matrix is H is thus 1 0 0 H 0 1 0 0 0 1 3x3 identity submatrix I 1 0 1 1 1 1 1 0 0 1 1 1 3x4 transpose of parity matrix P Figure 2: deconstruction of matrix EEE332 – G: Hamming code Part 2 8/17 H October 2008 . error detection and correction As discussed the matrix G can be decomposed into a 4 x 3 parity matrix Pand 4 x 4 identity matrix I G P I 44 Let’s consider forming a new 3 x 7 matrix H H I T 3 3 P T Note that P becomes a matrix of dimension 3 x 4.Decoding the Hamming code Use the (7.
tells us which expected check bits agree with their received counterpart and which check bit do not. v1 v1ex 0. if v1 v1ex Similar results are obtained for v2 v2 ex and v3 v3ex EEE332 – G: Hamming code Part 2 9/17 October 2008 . if v1 v1ex v1 v1ex 1.Now let r represents the received codeword r v 1 v 2 v3 v 4 v5 v 6 v 7 Suppose we post multiply the received codeword by the transpose of H 1 0 0 0 1 0 0 0 1 r H T v1 v2 v3 v4 v5 v6 v7 1 1 0 0 1 1 1 1 1 1 0 1 v1 v4 v6 v7 v2 v4 v5 v6 v3 v5 v6 v7 v 1 v1ex v2 v2 ex v3 v3ex The resulting 1 x 3 matrix is called the syndrome.
As we’ve already seen. Let’s use s to represent the threebit syndrome. we can crate a table similar to the Table 2 (Note F). Thus. this agreement or disagreement is exactly what we need for singlebit error detection and correction in the received sevenbit sequence. 0 0 0 1 0 0 0 1 0 0 0 1 1 1 0 0 1 1 1 1 1 1 0 1 s Bit received in Error None v1 v2 v3 v4 v5 v6 v7 EEE332 – G: Hamming code Part 2 10/17 October 2008 .
second. EEE332 – G: Hamming code Part 2 11/17 October 2008 .What is H ? Let’s look at it deeper. The parity submatrix produces the calculated check bits v1ex v2 ex and v3ex when modulo2 multiplication involves the first. respectively. and third column of H T . second. and third received bit with the calculated check bit. Figure 3 below T shows the deconstruction of the matrix H T H T 1 0 0 0 1 0 0 0 1 1 1 0 0 1 1 1 1 1 1 0 1 3x3 identity submatrix I 4x3 parity submatrix P Figure 3: Deconstruction of decoding matrix H T T Deconstruction of H into a 3 x 3 identity submatrix I and 4 x 3 parity submatrix P. The identity submatrix causes the exclusiveORing of the first.
since 0 if A B A B 1 if A B Multiplication using H followed by syndrome decoding using Table 1 above is therefore equivalent to the decoding process we originally described in Example 1 T (Note F).The exclusiveORing produces 0 if the received and calculated check bits agree or 1 if they disagree. 1 (Note F). As in Example 1 (Note F) a communication system employs the (7. Use matrix operations to determine if an error occurred and then determine the original four bits from the source. Solution EEE332 – G: Hamming code Part 2 12/17 October 2008 . T EXAMPLE 1 Using matrix operations to detect and correct an error – part 1. 4) Hamming code described in Eq. is received. Thus H is called the decoding matrix.
Unencoded onebit sequence u1 Channel encoder Encoded threebit sequence v1 v2 v3 Figure 3: Threebit channel encoder EEE332 – G: Hamming code Part 2 13/17 October 2008 . We can represent the information sequence (one bit) as u u 1 and the encoded sequence (three bits) as v v where v1 u1 v 2 u1 v 3 u1 1 v2 v3 Figure below shows the encoder. like in the previous example. Let’s use onebit blocks of information.Geometric Interpretation of Error Control Coding Let’s begin by considering an encoder that uses majority voting.
0) and (1.0) v1 v2 Figure 5: Vector space representation of the output Note: The encoding process maps a point from onedimension vector space u1 into a point in threedimensional vector space v1. v3 (1. 1) as shown in Figure 5. EEE332 – G: Hamming code Part 2 14/17 October 2008 . 0 1 u1 Figure 4: vector space representation of a single input Now let’s represent the encoded threebit sequence v1 v2 v3 as a point in three dimensional space that represent possible values for the encoded threebit sequence i.1) (0. (0. v2 .Let’s consider the information bit u1 to be a point in a onedimensional vector space as shown in Figure 4. 1.1. 0.e. v3 .0.
(0.0.Output sequence – threebit received sequence as a point in threedimensional vector space v1. (0. and (1. (1.e.0.0).01. The other six points represent errors (invalid codewords) in the received bit sequence i.1. Figure 6 shows the received 3bit sequence with eight possible received bits sequences. (0.1).0).0.0.1. If we receive (1.1.1). we assume that the information bit from the source is 0.1) we assume the information bit from the source is 1.0).1). (1. v2 . and if we receive (0. Figure 6: vector space representation of the received bit sequence Two of these eight points (black) represent valid codeword (signify no error has occurred in transmission).0). EEE332 – G: Hamming code Part 2 15/17 October 2008 . v3 .
assume the sequence (0. In Figure 6. 0. 1. 2. Observed the following. then (0.If two bits are “1”s . 0. so all singlebit errors can be detected. 0. 1. However. 1) than it is to (0. any onebit error is closer to the correct valid codeword than it is to any incorrect valid codeword – for example. 1) is received. For example. All onebit errors result in points corresponding to invalid codewords (gray dots). so all can be detected. All twobit errors also correspond to invalid codewords (gray dots). the point (1. 0) was transmitted and that one bit error occurred As shown in Figure 6 this process of finding the correct bits transmitted as finding the black dot (valid codewords) closest to the gray dot (invalid codewords) i. the point is closer to the incorrect valid codeword to the correct valid codeword.e. geometrically closest to it. 0) – so all onebit error can be corrected. if (1. 1. 1. 1) is transmitted and the first two bits are received in error. 1) was transmitted and that one bit error occurred If two bits are “0”s . Finding the closest point in space is equivalent to assuming to the least possible number of errors has occurred in the received codeword.assume the sequence (1. 0) is geometrically closer to (1. 1. EEE332 – G: Hamming code Part 2 16/17 October 2008 .
0. 0) which gives incorrect value. EEE332 – G: Hamming code Part 2 17/17 October 2008 . 1. 0. If we try to correct the error will end up with (0. All threebit errors result in received sequences representing valid codewords. 0) than it is to (1. 1) is closer to (0. 0. so threebit errors cannot even be detected. 1). 3.Since (0.
This action might not be possible to undo. Are you sure you want to continue?