EECS 554

Solutions to Homework 6
Due: 11/4

1 Linear Block Codes

1.1 Consider the linear code over Z/7 with check matrix
[ ]
1 2 3 4 5 6
1 4 2 2 4 1

Find the minimum distance of this code.

Solution
All columns are non-zero, and no column is a multiple of any other column (we need to
perform 5 + 4 + 3 + 2 + 1 = 15 checks for this), so dmin > 2. However, adding the first,
second and fourth columns (mod 7) gives zero, so dmin = 3.

End Solution

1
 1.2 Find the best (i.e., highest rank) binary linear single-error-correcting code of length
n = 6.

Solution
We want d ≥ 3 and thus from Singleton (6) we get k ≤ n − d + 1 ≤ 4. However from
(6)bound
Hamming bound we have k ≤ n − log2 [ 0 + 1 ] = 6 − log2 7 ≈ 3.2, so k ≤ 3.
So the best we can hope for is a (6, 3) code. Let’s try to design it through its parity-
check matrix in systematic form, which is a 3 × 6 matrix with I 3 as its submatrix. We need
to find the remaining 3 columns such that every group of 0, 1, and 2 columns are linearly
independent. If these columns have weight 0 or 1, this is impossible. So the only options are
columns with weight 2 and 3, i.e., the columns (0, 1, 1)T , (1, 0, 1)T , (1, 1, 0)T , and (1, 1, 1)T .
Any choice of three different columns from this set will do.
It is interesting that such code can be constructed from the (7, 4) Hamming code as well.

End Solution

2
1.3 What is the best (i.e., highest minimum-distance) binary linear (9, 4) code?

Solution
For t = 1 we have ( ) ( )
9 9
+ = 1 + 9 = 10 ≤ 29−4 = 32 (1)
0 1
and for t = 2 we have
( ) ( ) ( )
9 9 9
+ + = 1 + 9 + 36 = 46 > 29−4 = 32 (2)
0 1 2

so it is possible that the best (9, 4) code has single error correction capability. Since t ≤ 1,
we can have dmin ≤ 4.
To find the highest dmin we will now try to explicitly construct the corresponding parity
check matrix. The requirement is that all groups of 3 columns are linearly independent.
Let’s see if we can construct such an H in its systematic form. We need to select 4 columns.
Clearly we cannot use columns of weight 1 or 2. But if we use 4 columns of weight 4 (there
are 5 of them) then all groups of three columns are independent. Indeed if the group contains
1 weight-4 column then there is no way to cancel the 4 1s by adding 2 weight-1 columns.
Also, if the group contains 2 weight-4 columns then their sum is a weight-2 column and
there is no way to cancel the 2 1s by adding 1 more weight-1 column. Finally, if the group
contains 3 weight-4 columns then the sum of the first 2 is a weight-2 column and there is no
way to cancel the 2 1s by adding 1 more weight-4 column. Thus it is possible to construct
a (9, 4) code with dmin = 4.

End Solution

3
1.4 Consider the (5, 2) binary linear code given by the generator matrix
( )
1 0 1 0 1
G= .
0 1 0 1 1

1. Find the syndrome table for this code.

2. Evaluate the probability of decoding error for a BSC with crossover probability p.

Solution

1. Since G is in systematic form, the parity check matrix is

 
1 0 1 0 0
H =  0 1 0 1 0 .
1 1 0 0 1

The code has 25−2 = 8 cosets. Let’s evaluate the syndromes for each of the 5 weight-
1 error patterns. These will be of the form s = eH T which will correspond to the
columns of H. There are 8 − 5 − 1 = 2 additional nonzero error patterns that are
correctable. Let’s try (11000) and (10010), which give the following syndromes (110)
and (111) which do not correspond to any of the already found syndromes. So the
overall syndrome table is
error syndrome
00000 000
10000 101
01000 011
00100 100
00010 010
00001 001
11000 110
10010 111

2. The probability of decoding error is the probability of the above error patterns not
occurring, which is

P e = 1 − [(1 − p)5 + 5p(1 − p)4 + 2p2 (1 − p)3 ]

End Solution

4
 1.5 Consider mixed error correction/error detection of an (n, k) linear code. In particular
assume that the code has guaranteed error correction capability of t errors. We set up the
receiver so that it corrects all errors up to weight t and in addition it detects the rest of
detectable error patterns.
Conditioned that a codeword c is transmitted over a BSC with crossover probability p,
define and evaluate the following performance measures for this decoder.

1. Probability of correct decoding

2. Probability of error detection

3. Probability of incorrect decoding

Solution

1. Probability of correct decoding. This is the probability that an error pattern of weight
≤ t occurred in the channel and thus it will be correctly identified and corrected. The
corresponding probability is

∑t ( )
n i
PCD = p (1 − p)n−i
i=0
i

2. Probability of error detection. This is the probability that an error occurred in the
channel that results in a received vector outside all spheres of radius t centered at
codewords. These error pattern can be identified in the standard array as all cosets
(not only coset leader!) with coset leaders having weight > t. Alternatively, these error
patterns can be thought off as ei,j = ei + cj where ei are coset leaders with weight > t
and cj ∈ C. So, ∑
PED = pw(e) (1 − p)n−w(e) .
e:e=e′ +c,w(e′ )>t,c∈C

3. Probability of incorrect decoding. Is the probability that an error occurs that will result
in an observation lying inside a sphere of a codeword that was not the one transmitted.
In fact it is straightforward to see that

PID = 1 − PCD − PED

End Solution

5
 1.6 Consider a linear (n, k) block code C.
The dual code C d of C is the maximal linear code with the property that every codeword
def ∑
x ∈ C and every codeword y ∈ C d we have xT y = ni=1 xi yi = 0 (it is tempting to say that
x and y are orthogonal, but we have not defined an inner product in this space).
Further, consider a set of coordinates U ⊂ {1, 2, . . . , n}.
A punctured code PU (C) is a code generated by C by eliminating all coordinates i ∈ U
from all codewords in C.
A shortened code SU (C) is a code generated by C by considering only those codewords in
C that have 0 in all coordinates i ∈ U , and then eliminating all coordinates i ∈ U .
Show that SU (C d ) = PU (C)d .

Solution
We will show that ∀x ∈ SU (C d ) and ∀y ∈ PU (C), we have xT y = 0.
Indeed, for any x ∈ SU (C d ), there exists an x′ ∈ C d , such that
{
′ xi , i ∈
/U
xi = (1)
0 ,i ∈ U

Also, for any y ∈ PU (C), there exists an y ′ ∈ C, such that

yi′ = yi ,i ∈
/U (2)

Now we have
∑ ∑ ∑ ∑
xT y = xj yj = x′i yi′ = x′i yi′ + 0yi′ (3)
j i∈U
/ i∈U
/ i∈U
∑ ∑ ∑
= x′i yi′ + x′i yi′ = x′i yi′ = x′T y ′ = 0, (4)
i∈U
/ i∈U i

since x′ ∈ C d and y ′ ∈ C.
Next we need to show that SU (C d ) is the maximal subspace with this property. To do this
consider A, the maximal subspace that is “orthogonal” to PU (C) (this can be constructed
by collecting all vectors that are “orthogonal” to PU (C)). Clearly SU (C d ) ⊂ A. Now let’s
suppose that there exists an x ∈ A with x ∈ / SU (C d ). We want to show that this leads to a
contradiction, thus A = SU (C d ).
We can write x = x1 + x2 , with x1 ∈ SU (C d ). Since x ∈ A and x1 ∈ SU (C d ), we have
that x2 is “orthogonal” to PU (C). Consider the vector x′ ∈ {0, 1}n with
{
′ x1i + x2i , i ∈ /U
xi = , (5)
0 ,i ∈ U

Clearly, x′ is “orthogonal” to C (why?), which implies that x′ ∈ C d . This further implies

that x ∈ SU (C d ), which is a contradiction.

End Solution

6
1.7 Show that any linear code can correct up to dmin − 1 number of erasures.

Solution
For any linear block code with parity check matrix H, erasure decoding can be performed
as follows. Let e be the set of erased positions and let y be the received vector (output of the
channel). Also denote y ē the length n − |e| vector resulting from keeping only the non-erased
positions of y. Similarly, denote H ē the n − k × n − |e| matrix containing only the columns
of H that correspond to non-erased positions of y, and similarly for H e . Since no errors
occurred in the channel, there is a codeword that agrees with y in the non-erased positions.
We only need to find the components of this codeword for the erased positions. Since
Hy T = 0 we have that H e y Te = −H ē y Tē = dT . The right-hand side of this equation can
be evaluated, since the quantities involved are available at the receiver. This is a system of
n−k equations with |e| unknowns (the components of y e ). We already know that the system
has a solution (the one corresponding to the transmitted codeword). Thus the question is if
this solution is unique. For this to be true, H e has to be full rank. Since any set of dmin − 1
columns of H are linearly independent, then, H e is full rank as long as |e| ≤ dmin − 1.
(Observe that the Singleton bound guarantees that dmin − 1 ≤ n − k).

End Solution