EEE332 – G: Hamming code Part 2 1/17 October 2008

HAMMING CODES

Hamming code with matrix manipulation

Consider the four un-encoded information bits
3 2 1
, , u u u
and
4
u . This 1 x 4 matrix can be represented as;

| | | |
4 3 2 1
u u u u u =

The seven-bit encoded data is represented as;

| | | |
7 6 5 4 3 2 1
v v v v v v v v =

The encoding process of the Hamming code can be
represented as modulo-2 multiplication;

| | | |
7 6 5 4 3 2 1 4 3 2 1
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
v v v v v v v u u u u =
(
(
(
(
¸
(

¸

-

where the 4 x 7 matrix is known as a generator matrix
represented by | | G .

Note:
If we examine carefully this multiplication operation it is
consistent with Eq (1) above.

EEE332 – G: Hamming code Part 2 2/17 October 2008

We can represent the encoding process by multiplying our
information vector | | u by a generator matrix | | G to obtain
the encoded bit sequence | | v .

| | | || | G u v = (2)


EXAMPLE 3
Encoding by matrix multiplication

A source outputs the four information bits 1100. These
four bits are to be encoded using the Hamming code just
described. Using the generator matrix in Eq (2), determine
the encoded seven-bit pattern to be transmitted.

Solution













EEE332 – G: Hamming code Part 2 3/17 October 2008


Let’s take a closer look on generator matrix | | G for our
Hamming code. From the above

| |
(
(
(
(
¸
(

¸

=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
G

We can see that the generator matrix has the following
properties;

1. The matrix is k rows by n columns: where k=4, the
number of information bits from the source; and n=7,
the total number of encoced bits.

2. The last four columns of | | G form an identity matrix.
The identity matrix maps the four information bits into
the last four bits of the encoded sequence (seven-bit).

3. The first three columns of | | G shows the relationship
between the parity (check) bits with data bits.

4. The generator matrix is not unique. For example, we
can develop an equally valid Hamming code by
switching the first and second columns in| | G .



EEE332 – G: Hamming code Part 2 4/17 October 2008


Now let’s de-construct the generator matrix into two
submatrices as i.e. a 4 x 4 identity submatrix | | I and 4 x 3
parity submatrix | | P (which produce the parity bits).


| |
(
(
(
(
(
¸
(

¸

=
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 1
1 1 1
1 1 0
0 1 1
G

| | | | | | | |
4 4x
I P G =





Figure 1: Deconstruction of generator matrix | | G


Terminology

- Systematic code – A code in which the information
bits are always repeated in a portion of the encoded
sequence (the input data are embedded in the
encoded output). A non-systematic code is the one in
which the output does not contain the input bits.

4 x 3 parity submatrix| | P
4 x 4 identity submatrix | | I
EEE332 – G: Hamming code Part 2 5/17 October 2008

The systematic code is very much desirable because
they make it easy to extract the original data from the
encoded sequence.
- Block code – divides the bit stream from the source
encoder into k-bit blocks. Each k-bit blocks is then
encoded into an n-bit block prior to transmission.

The Hamming code that we’ve just developed is a block
code which takes data stream from source encoder,
divides it into four-bit block, and then encodes each
four-bit block into a seven-bit block prior to
transmission.

Block codes are described in term of n and k, stated as
( ) k n, . The code we’ve developed is called a ( ) 4 , 7
Hamming code.

The code rate given as ( ) r ;

n
k
r =

Sometimes the Hamming code above is also called
“rate four-sevenths” code






EEE332 – G: Hamming code Part 2 6/17 October 2008

Hamming code – (15, 11)
Suppose we want to develop a Hamming code that
provides single-error detection and correction capabilities
for a larger block of data.

Suppose we want to use four parity bits (instead of three)
– we can produce eleven (11) different combinations of
those four bits with two or more “1”s. Therefore, we can
use four parity bits to provide single-error detection and
correction capabilities for a block of 11 information bits.
(Why 11 information data bits? – check previous matrix
multiplication).

Thus the generator matrix for the (15, 11) Hamming code
is given as follows;

| |
(
(
(
(
(
(
(
(
(
(
(
(
(
(
(
¸
(

¸

=
1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
0 1 0 0 0 0 0 0 0 0 0 1 1 1 0
0 0 1 0 0 0 0 0 0 0 0 1 1 0 1
0 0 0 1 0 0 0 0 0 0 0 1 0 1 1
0 0 0 0 1 0 0 0 0 0 0 0 1 1 1
0 0 0 0 0 1 0 0 0 0 0 1 1 0 0
0 0 0 0 0 0 1 0 0 0 0 1 0 1 0
0 0 0 0 0 0 0 1 0 0 0 0 1 1 0
0 0 0 0 0 0 0 0 1 0 0 1 0 0 1
0 0 0 0 0 0 0 0 0 1 0 0 1 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 1 1
G

EEE332 – G: Hamming code Part 2 7/17 October 2008

This new (15, 11) code is not as powerful as our (7, 4)
code for protecting information since we can only protect
the information from only single-bit error in 11
information bits.

If we break the 11 bit information data into three four-bit
blocks, and use the (7, 4) code in order to get better
protection.

However, the new code provides less overhead i.e. only
four parity bits required for 11 information data bits.

If we break up into three four-bit data will end up with 3 x
3 = 9 parity bits.
















EEE332 – G: Hamming code Part 2 8/17 October 2008

Decoding the Hamming code

Use the (7, 4) code for decoding, error detection and
correction

As discussed the matrix | | G can be decomposed into a 4 x
3 parity matrix | | P and 4 x 4 identity matrix | | I

| | | | | | | |
4 4×
= I P G 

Let’s consider forming a new 3 x 7 matrix | | H

| | | | | | | |
T
P I H 
3 3×
=

Note that | |
T
P becomes a matrix of dimension 3 x 4.
Transpose is to turn the rows into column or columns into
rows.

The matrix is | | H is thus

| |
(
(
(
¸
(

¸

=
1 1 1 0
0 1 1 1
1 1 0 1
1 0 0
0 1 0
0 0 1
H




Figure 2: deconstruction of matrix | | H
3x3 identity submatrix | | I 3x4 transpose of parity matrix | | P
EEE332 – G: Hamming code Part 2 9/17 October 2008

Now let | | r represents the received codeword

| | | |
7 6 5 4 3 2 1
v v v v v v v r =

Suppose we post multiply the received codeword by the
transpose of | | H

| || | | |
| |
| |
ex ex ex
T
v v v v v v
v v v v v v v v v v v v
v v v v v v v H r
3 3 2 2 1 1
7 6 5 3 6 5 4 2 7 6 4 1
7 6 5 4 3 2 1
1 0 1
1 1 1
1 1 0
0 1 1
1 0 0
0 1 0
0 0 1
© © ©
= © © © © © © © © ©
=
(
(
(
(
(
(
(
(
(
¸
(

¸

- =


The resulting 1 x 3 matrix is called the syndrome, tells us
which expected check bits agree with their received
counterpart and which check bit do not.

ex ex
v v if v v
1 1 1 1
; 0 = = ©
ex ex
v v if v v
1 1 1 1
; 1 = = ©

Similar results are obtained for
ex
v v
2 2
© and
ex
v v
3 3
©

EEE332 – G: Hamming code Part 2 10/17 October 2008

As we’ve already seen, this agreement or disagreement is
exactly what we need for single-bit error detection and
correction in the received seven-bit sequence.

Let’s use | | s to represent the three-bit syndrome. Thus,
we can crate a table similar to the Table 2 (Note F).

| | s Bit received in Error
| | 0 0 0 None
| | 0 0 1
1
v
| | 0 1 0
2
v
| | 1 0 0
3
v
| | 0 1 1
4
v
| | 1 1 0
5
v
| | 1 1 1
6
v
| | 1 0 1
7
v











EEE332 – G: Hamming code Part 2 11/17 October 2008

What is | |
T
H ? Let’s look at it deeper. Figure 3 below
shows the deconstruction of the matrix | |
T
H

| |
(
(
(
(
(
(
(
(
(
(
¸
(

¸

=
1 0 1
1 1 1
1 1 0
0 1 1
1 0 0
0 1 0
0 0 1
T
H

Figure 3: Deconstruction of decoding matrix | |
T
H

Deconstruction of | |
T
H into a 3 x 3 identity submatrix | | I
and 4 x 3 parity submatrix | | P .

The parity submatrix produces the calculated check bits
ex ex
v v
2 1
and
ex
v
3
when modulo-2 multiplication involves
the first, second, and third column of | |
T
H , respectively.

The identity submatrix causes the exclusive-ORing of the
first, second, and third received bit with the calculated
check bit.


3x3 identity submatrix | | I
4x3 parity submatrix | | P
EEE332 – G: Hamming code Part 2 12/17 October 2008

The exclusive-ORing produces 0 if the received and
calculated check bits agree or 1 if they disagree, since

¹
´
¦
=
=
= ©
B A if
B A if
B A
1
0


Multiplication using | |
T
H followed by syndrome decoding
using Table 1 above is therefore equivalent to the
decoding process we originally described in Example 1
(Note F). Thus | |
T
H is called the decoding matrix.


EXAMPLE 1
Using matrix operations to detect and correct an error –
part 1.

As in Example 1 (Note F) a communication system
employs the (7, 4) Hamming code described in Eq. 1
(Note F), is received.

Use matrix operations to determine if an error occurred
and then determine the original four bits from the source.

Solution





EEE332 – G: Hamming code Part 2 13/17 October 2008

Geometric Interpretation of Error Control Coding

Let’s begin by considering an encoder that uses majority
voting, like in the previous example.

Let’s use one-bit blocks of information. We can represent
the information sequence (one bit) as

| | | |
1
u u =

and the encoded sequence (three bits) as

| | | |
3 2 1
v v v v =

where
1 3
1 2
1 1
u v
u v
u v
=
=
=


Figure below shows the encoder;


Unencoded one-bit sequence
u
1

Channel
encoder
Figure 3: Three-bit channel encoder
Encoded three-bit sequence
v
1
v
2
v
3

EEE332 – G: Hamming code Part 2 14/17 October 2008

Let’s consider the information bit u
1
to be a point in a
one-dimensional vector space as shown in Figure 4.


Now let’s represent the encoded three-bit sequence v
1
v
2

v
3
as a point in three dimensional space that represent
possible values for the encoded three-bit

sequence i.e. (0,
0, 0) and (1, 1, 1) as shown in Figure 5.


Note:
The encoding process maps a point from one-dimension
vector space ( )
1
u into a point in three-dimensional vector
space ( )
3 2 1
, , v v v .

1
v

3
v
2
v
(0,0,0)
(1,1,1)
Figure 5: Vector space representation of the output
0 1
u
1

Figure 4: vector space representation of a single input
EEE332 – G: Hamming code Part 2 15/17 October 2008

Output sequence – three-bit received sequence as a point
in three-dimensional vector space ( )
3 2 1
, , v v v .

Figure 6 shows the received 3-bit sequence with eight
possible received bits sequences.



Two of these eight points (black) represent valid
codeword (signify no error has occurred in transmission).
If we receive (1,1,1) we assume the information bit from
the source is 1, and if we receive (0,0,0), we assume that
the information bit from the source is 0.

The other six points represent errors (invalid codewords)
in the received bit sequence i.e. (0,0,1), (0,1,0), (0,01,1),
(1,0,0), (1,0,1), and (1,1,0).

Figure 6: vector space representation of the
received bit sequence
EEE332 – G: Hamming code Part 2 16/17 October 2008

If two bits are “1”s - assume the sequence (1, 1, 1) was
transmitted and that one bit error occurred

If two bits are “0”s - assume the sequence (0, 0, 0) was
transmitted and that one bit error occurred

As shown in Figure 6 this process of finding the correct
bits transmitted as finding the black dot (valid codewords)
closest to the gray dot (invalid codewords) i.e.
geometrically closest to it.

Finding the closest point in space is equivalent to
assuming to the least possible number of errors has
occurred in the received codeword.

Observed the following;
1. All one-bit errors result in points corresponding to
invalid codewords (gray dots), so all single-bit errors
can be detected. In Figure 6, any one-bit error is
closer to the correct valid codeword than it is to any
incorrect valid codeword – for example, the point (1,
1, 0) is geometrically closer to (1, 1, 1) than it is to
(0, 0, 0) – so all one-bit error can be corrected.

2. All two-bit errors also correspond to invalid
codewords (gray dots), so all can be detected.
However, the point is closer to the incorrect valid
codeword to the correct valid codeword. For
example, if (1, 1, 1) is transmitted and the first two
bits are received in error, then (0, 0, 1) is received.
EEE332 – G: Hamming code Part 2 17/17 October 2008

Since (0, 0, 1) is closer to (0, 0, 0) than it is to (1, 1,
1). If we try to correct the error will end up with (0,
0, 0) which gives incorrect value.

3. All three-bit errors result in received sequences
representing valid codewords, so three-bit errors
cannot even be detected.

Using the generator matrix in Eq (2).We can represent the encoding process by multiplying our information vector u  by a generator matrix G  to obtain the encoded bit sequence v . Solution EEE332 – G: Hamming code Part 2 2/17 October 2008 . v  u G  EXAMPLE 3 Encoding by matrix multiplication (2) A source outputs the four information bits 1100. determine the encoded seven-bit pattern to be transmitted. These four bits are to be encoded using the Hamming code just described.

4. 3. The matrix is k rows by n columns: where k=4. The identity matrix maps the four information bits into the last four bits of the encoded sequence (seven-bit). EEE332 – G: Hamming code Part 2 3/17 October 2008 . The first three columns of G  shows the relationship between the parity (check) bits with data bits. 1. the number of information bits from the source. 2. and n=7. we can develop an equally valid Hamming code by switching the first and second columns in G . For example. the total number of encoced bits. The last four columns of G  form an identity matrix. From the above 1 0 G    1 1  1 0 1 0 0 0 1 1 0 1 0 0  1 1 0 0 1 0 0 1 0 0 0 1  We can see that the generator matrix has the following properties.Let’s take a closer look on generator matrix G  for our Hamming code. The generator matrix is not unique.

EEE332 – G: Hamming code Part 2 4/17 October 2008 . 1  0 G    1  1  1 0 1 0 0 0  1 1 0 1 0 0 1 1 0 0 1 0  0 1 0 0 0 1  4x4 G   P I   4 x 3 parity submatrix P 4 x 4 identity submatrix I  Figure 1: Deconstruction of generator matrix G  Terminology  Systematic code – A code in which the information bits are always repeated in a portion of the encoded sequence (the input data are embedded in the encoded output). A non-systematic code is the one in which the output does not contain the input bits. a 4 x 4 identity submatrix I  and 4 x 3 parity submatrix P (which produce the parity bits).Now let’s de-construct the generator matrix into two submatrices as i.e.

stated as n. The code rate given as r . 4 Hamming code.The systematic code is very much desirable because they make it easy to extract the original data from the encoded sequence. r k n Sometimes the Hamming code above is also called “rate four-sevenths” code EEE332 – G: Hamming code Part 2 5/17 October 2008 . Each k-bit blocks is then encoded into an n-bit block prior to transmission. k .  Block code – divides the bit stream from the source encoder into k-bit blocks. The Hamming code that we’ve just developed is a block code which takes data stream from source encoder. divides it into four-bit block. The code we’ve developed is called a 7 . Block codes are described in term of n and k. and then encodes each four-bit block into a seven-bit block prior to transmission.

Hamming code – (15. we can use four parity bits to provide single-error detection and correction capabilities for a block of 11 information bits. Thus the generator matrix for the (15. 11) Hamming code is given as follows. Suppose we want to use four parity bits (instead of three) – we can produce eleven (11) different combinations of those four bits with two or more “1”s. Therefore. 11) Suppose we want to develop a Hamming code that provides single-error detection and correction capabilities for a larger block of data. (Why 11 information data bits? – check previous matrix multiplication). 1 1  1 0  0 G   0  1  1 1  0 1  1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0  0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0  1 0 1 0 0 0 0 1 0 0 0 0 0 0  0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0  1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0  1 1 1 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 0 0 0 0 1  6/17 October 2008 EEE332 – G: Hamming code Part 2 .

This new (15. 4) code for protecting information since we can only protect the information from only single-bit error in 11 information bits. If we break the 11 bit information data into three four-bit blocks. EEE332 – G: Hamming code Part 2 7/17 October 2008 . However.e. If we break up into three four-bit data will end up with 3 x 3 = 9 parity bits. only four parity bits required for 11 information data bits. 4) code in order to get better protection. the new code provides less overhead i. 11) code is not as powerful as our (7. and use the (7.

4) code for decoding. Transpose is to turn the rows into column or columns into rows. The matrix is H  is thus 1 0 0 H   0 1 0  0 0 1  3x3 identity submatrix I  1 0 1 1 1 1 1 0  0 1 1 1  3x4 transpose of parity matrix P  Figure 2: deconstruction of matrix EEE332 – G: Hamming code Part 2 8/17 H  October 2008 . error detection and correction As discussed the matrix G  can be decomposed into a 4 x 3 parity matrix Pand 4 x 4 identity matrix I  G  P  I 44  Let’s consider forming a new 3 x 7 matrix H  H   I  T 3 3  P T  Note that P  becomes a matrix of dimension 3 x 4.Decoding the Hamming code Use the (7.

tells us which expected check bits agree with their received counterpart and which check bit do not. v1  v1ex  0. if v1  v1ex Similar results are obtained for v2  v2 ex and v3  v3ex EEE332 – G: Hamming code Part 2 9/17 October 2008 . if v1  v1ex v1  v1ex  1.Now let r  represents the received codeword r   v 1 v 2 v3 v 4 v5 v 6 v 7  Suppose we post multiply the received codeword by the transpose of H  1 0 0  0 1 0    0 0 1  r H T  v1 v2 v3 v4 v5 v6 v7  1 1 0    0 1 1    1 1 1  1 0 1    v1  v4  v6  v7 v2  v4  v5  v6 v3  v5  v6  v7  v 1  v1ex v2  v2 ex v3  v3ex  The resulting 1 x 3 matrix is called the syndrome.

As we’ve already seen. Let’s use s  to represent the three-bit syndrome. we can crate a table similar to the Table 2 (Note F). Thus. this agreement or disagreement is exactly what we need for single-bit error detection and correction in the received seven-bit sequence. 0 0 0 1 0 0 0 1 0 0 0 1 1 1 0 0 1 1 1 1 1 1 0 1 s  Bit received in Error None v1 v2 v3 v4 v5 v6 v7 EEE332 – G: Hamming code Part 2 10/17 October 2008 .

second. EEE332 – G: Hamming code Part 2 11/17 October 2008 .What is H  ? Let’s look at it deeper. The parity submatrix produces the calculated check bits v1ex v2 ex and v3ex when modulo-2 multiplication involves the first. respectively. and third column of H T . second. and third received bit with the calculated check bit. Figure 3 below T shows the deconstruction of the matrix H  T H  T 1 0 0  0 1 0    0 0 1     1 1 0   0 1 1  1 1 1    1 0 1  3x3 identity submatrix I  4x3 parity submatrix P Figure 3: Deconstruction of decoding matrix H  T T Deconstruction of H  into a 3 x 3 identity submatrix I  and 4 x 3 parity submatrix P. The identity submatrix causes the exclusive-ORing of the first.

since 0 if A  B A B 1 if A  B Multiplication using H  followed by syndrome decoding using Table 1 above is therefore equivalent to the decoding process we originally described in Example 1 T (Note F).The exclusive-ORing produces 0 if the received and calculated check bits agree or 1 if they disagree. 1 (Note F). As in Example 1 (Note F) a communication system employs the (7. Use matrix operations to determine if an error occurred and then determine the original four bits from the source. Solution EEE332 – G: Hamming code Part 2 12/17 October 2008 . T EXAMPLE 1 Using matrix operations to detect and correct an error – part 1. 4) Hamming code described in Eq. is received. Thus H  is called the decoding matrix.

Unencoded one-bit sequence u1 Channel encoder Encoded three-bit sequence v1 v2 v3 Figure 3: Three-bit channel encoder EEE332 – G: Hamming code Part 2 13/17 October 2008 . We can represent the information sequence (one bit) as u   u  1 and the encoded sequence (three bits) as v v where v1  u1 v 2  u1 v 3  u1 1 v2 v3  Figure below shows the encoder. like in the previous example. Let’s use one-bit blocks of information.Geometric Interpretation of Error Control Coding Let’s begin by considering an encoder that uses majority voting.

0) and (1.0) v1 v2 Figure 5: Vector space representation of the output Note: The encoding process maps a point from one-dimension vector space u1  into a point in three-dimensional vector space v1. v3 (1. 1) as shown in Figure 5. EEE332 – G: Hamming code Part 2 14/17 October 2008 . 0 1 u1 Figure 4: vector space representation of a single input Now let’s represent the encoded three-bit sequence v1 v2 v3 as a point in three dimensional space that represent possible values for the encoded three-bit sequence i.1) (0. (0. v2 .Let’s consider the information bit u1 to be a point in a one-dimensional vector space as shown in Figure 4. 1.1. 0.e. v3 .0.

(0.0.Output sequence – three-bit received sequence as a point in three-dimensional vector space v1. (0. and (1. (1.e.0.0).01. The other six points represent errors (invalid codewords) in the received bit sequence i.1. Figure 6 shows the received 3-bit sequence with eight possible received bits sequences. (0.1).0).0.0.1. If we receive (1.1.1). we assume that the information bit from the source is 0.1) we assume the information bit from the source is 1.0).1). (1. v2 . and if we receive (0. Figure 6: vector space representation of the received bit sequence Two of these eight points (black) represent valid codeword (signify no error has occurred in transmission).0). EEE332 – G: Hamming code Part 2 15/17 October 2008 . v3 .

assume the sequence (0. In Figure 6. 0. 1. 2. Observed the following. then (0.If two bits are “1”s . 0. so all single-bit errors can be detected. 0. 1. However. 1) than it is to (0. any one-bit error is closer to the correct valid codeword than it is to any incorrect valid codeword – for example. 1) is received. For example. All one-bit errors result in points corresponding to invalid codewords (gray dots). so all can be detected. All two-bit errors also correspond to invalid codewords (gray dots). the point (1. 0) was transmitted and that one bit error occurred As shown in Figure 6 this process of finding the correct bits transmitted as finding the black dot (valid codewords) closest to the gray dot (invalid codewords) i. the point is closer to the incorrect valid codeword to the correct valid codeword.e. geometrically closest to it. 0) – so all one-bit error can be corrected. if (1. 1. 1. 1) is transmitted and the first two bits are received in error. 1) was transmitted and that one bit error occurred If two bits are “0”s . Finding the closest point in space is equivalent to assuming to the least possible number of errors has occurred in the received codeword.assume the sequence (1. 0) is geometrically closer to (1. 1. EEE332 – G: Hamming code Part 2 16/17 October 2008 .

0. 0) which gives incorrect value. EEE332 – G: Hamming code Part 2 17/17 October 2008 . 1. 0. If we try to correct the error will end up with (0. All three-bit errors result in received sequences representing valid codewords. 0) than it is to (1. 1) is closer to (0. 0. so three-bit errors cannot even be detected. 1). 3.Since (0.