1100
3, MAY 1993
r 0.9999;96715 0 0
0.9999589994 0 0
p= 0
0
0
0
0.9892556344
0
0.4434131733
0.0725537809 0.0333957486 0.1377107457 0.3283679031
VI. CONCLUSION
We have investigated super channels consisting of renewal inner
channels and constrained codes. Three sets of recursions representing
six widely used modulation codes have been developed to obtain
the error-free runs of such super channels. It should be noted that
. these recursions representing the error-free runs of the super channels
are fundamentally exact expressions, irrespective of the renewal/non­
renewal nature of the super channel.
We have furthermore experimentally investigated the hypothesis
that Fritchman partitioned Markov chains can also be used to model
the super channels, thus implicitly assuming that the latter can be
modeled with adequate numerical precision by renewal processes.
While we are unable to analytically prove this hypothesis, in all
the experimental investigations, we found that Fritchman partitioned
Markov chains represented the super channels with excellent numeri­
cal precision-higher order statistical distributions deviated typically
unly in the third significant digit. It is interesting to note that in all
such investigations undertaken, the super channel and inner channel
Markov chains were found tu have the same number of statcs.
The procedure that was presented here can also be used to develop
recursions representing other constrained codes. However, it should
be noted that for constrained cudes which exhibit error propagation, it
is more difficult to derivc recursions representing the error-free runs.
Also, such super channels will not be renewal processes.
Finally, by using the analytically derived error-free runs (thus not
necessarily referring to the partitioned Markov chain representations
of the super channels) a proposed outer error control code in a
concatenated scheme can he evaluated using previously published
techniques-see e.g., [1] and [3].
REFbRENCES
[1] L. N. Kanal and A. R. K. Sastry, "Models for channels with memory
and their applications to error control," Proc. IEEE, vol. 66, no. 7, pp.
724--744, July 1978.
[2] W. Turin, Performance Analysis of Digital Transmission Systems. New
York: W. H. Freeman, 1990.
[3] B. D. Fritchman, "A binary channel characterization using partitioned
Markov chains," IF:F:E Trans. Inform. Theory, vol. IT-13, pp. 221-227,
Apr. 1967.
[4] S. Tsai, "Markov characterization of the H. F. channel," IEEE Trans.
Commun. Technol., vol. COM-l7, no. 2, pp. 24--32, Feb. 1969.
[5] K. Brayer, "Characterization of the digital high speed AUTOVON
channel, " Final Rep ., The MITRE Corporation, MTR-2968, Feb. 1976.
[6] A. I. Drukarev and K. P. Yiu, "Perfor mance of error-correcting codes
on channels with memory," IEEE Trans. Commun., . vol. COM-34, no.
6, pp. 513-521, June 1986.
[7] TEMC 32-2, "Fail-safe telemetry system technical manu al, " M.L. Engi­
neering Ltd., Plymouth, UK, i s s . 4, pp. 2-8, Jan. 1987.
[8] CCIR: I nternation al Radio Consultive Committee, "Recommendations
and reports of the CCIR, 1986," Vol. VIII-L, Land Mobile Service,
Amateur service, Amateur Satellite Service, Duvrovnik, 1986, report
903.
[91 K. A. S. Imrnink, Coding Techniques for the Optical and Magnetic
Recording Channel. New York, Prentice-Hall, 1991.
0018-9448/93$03.00 © 1993 IEEE
IEEE TRANSACTIONS ON INFORMATION TIIEORY, VOL. 39, NO.
0 0.0000003285 1
0.0000410006
0
0.0107443656
0.5565868267
j. (34)
0.4279718217.
[10] M. Y. Rhee, Ermr-Correcting Coding Theory. New York: McGraw­
Hill, 1989.
[11] T. Fergus on, "Channel modeling and code performance evaluation,"
NSA Tech. J. (Special Mathematics and Engineering Issue), pp. 67-91,
1981-1983.
[12] A. H. S. Ang and W. H. Tang , Probability Concepts in Enginee rin g
Planning and Design, Volume II: Decision, Risk and Reliability. New
York: John Wiley , 1984.
[13] R. Goodman , Introduction to Stochastic Models. Menlo Park, CA:
Benjamin/Cummings, 1988.
[14] J. L. LoCicero, D. J. Costello, and L. C. Peach, "Characteristics of the
Hedeman H - 1, H - 2, and H - 3 codes," IEEE Trans. Commun.,
vol. COM-27, no. 6, pp. 901-908, June 1981.
[15] F. Swarts, D. R. Oosthuizen, and H. C. Ferreira, "On the real time
measurement and theoretical modelling of error distributions on digital
communication channels," in Proc. IEEE Comsig '90 Coni, Johannes­
burg, South Africa, June 29, 1990, pp. 78-92.
Using Combinatorial Optimization to Design
Good Unit-Memory Convolutional Codes
Amir Said, Student Member, IEEE, and
Reginaldu Palazzo, Jr., Member, IEEE
Abstract-A new method to design good uuit-memory convolutional
codes is presented. It is based on the decomposition of the original prob­
lem into two easier subproblems, which can be formulated as optimization
problems and solved by new efficient heuristic search algorithms. The
efficacy of the new method is demonstrated by a table containing 33 new
nnit-memory convolutional codes (n, k) with 5 :::; k :::; 8, rates R kin=
between 1/4 and 7/9, and complete memory (1\1 =k), as well as 12
new linear block codes. Most of the new codes found have maximum free
distance.
I"dex Terms-Combinatorial optimization, unit-memory codes, column
selection problem, column matching problem.
I. INTRODUCTION
It is known that unit-memory convolutional codes (UMC) may
have a better error correcting capability than the usual multimemory
convolutional codes with the same rate and number of elements
[1]-[3]. However, each UMC is defined from a larger number
of elements than the corresponding multimemory code; and even
for a small number of memory elements, finding these codes by
exhaustive search may be practically impossible. As a consequence,
Manuscript received May 3, 1990; revised March 12, 1992. This work
was supported by COllselho Nacional de Desenvolvimento Cientifico e Tec­
noI6gico-CNPq, Brazil, under Grant 301416/85-0. This work was presented
in part at the 4th Joint Swedish -Soviet International Workshop on Information
Theory, Gotland, Sweden, August 27-September 1, 1989.
The authors are with the Faculty of Electrical Engineering, State University
of Campinas-UNICAMP, l3081-970, Campinas, SP, Brazil.
IEEE Log Number 9207884.
IEEE TRANSACfIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993 1101

few complete memory UMC's with maximum free distance are
known.
This correspondence introduces a UMC design method based on
mathematical optimization models with the purpose of providing
a systematic way of finding these codes. To allow the use of
standard optimization techniques, we looked for equivalent models
with desirable properties, such as convexity and linearity in the
real field. It is shown that the problem can be decomposed into
two subproblems which can be formulated as standard optimization
problems with integer variables. However, even in standard form, the
known upper hound, we also use dr,ee to indicate the estimate of the
maximum free distance used by the search algorithms.
The distance profile of the UMC is the sequence (P2,P3,'" ,Pm,
...) with the minimum weight of all codewords of length m (called
extended row distance profile, and denoted by J;;, in [2]). Thommesen
and Justesen [2] show that the set of error patterns which can be
always corrected with Viterbi decoding depends on the distance
profile. Let us also define h (i) as the k-dimensional column vector
with elements equal to the binary representation of i. For example,
u4(13) [HOI]!. These vectors, with elements from GF(2), will be
=

complexity in solving these optimization problems is large because used as columns in the generator matrices, Go and G1, and called
their variables are constrained to integer values (NP-type problems generator matrices column types.
[7]). We define, from higher to lower priority, the goals to be achieved
To reduce the search effort, we propose two local search algo­ in the search for good codes as:
rithms, also called heuristic algorithms, that are very efficient in catastrophic codes elimination;
•

finding good codes. The first algorithm employed in the search for free distance maximization;
•

good codes is concerned with the column types selection problem minimization of the number of trellis paths corresponding to
•

(Section Ill), All linear block codes (Be) used in the generation of dfyee;
the (n, k) UMC's came from this algorithm.To show the importance maximization of distance profile.
•

of this algorithm, we tabulate 6 new linear block codes: (32,8, 13),
(60,8,27), (73,8,33), (76,8,35), (43,9,17), and (46,9,19). Since
these codes have odd Hamming distances, the corresponding extended
codes (33,8,14), (61,8,28), (74,8,34), (77,8,36), (44,9,18), and
(47,9,20) are easily determined by the extension method [12, Ch. 1,
pp.27-28].
The second algorithm is concerned with the column types matching
problem (Section IV). This is essentially an inhcrent problem of
linear trellis codes, which in particular has to do with unit-memory
codes. Computational results show that these algorithms provide
These goals are based on the assumption that maximum likelihood
decoding (Viterbi algorithm, [9]) will be used (sequential decoding
may require different priorities).The last goal needs a more rigorous
definition because the distance profile is a sequence of numbers, and
it may not be possible to maximize the value of all elements of the
sequence. This definition is presented later where it can be more
clearly understood.
B. Decomposition Strategy

(n, k) UMC's with maximum free distance for a fairly wide range A necessary condition for a UMC to have dfree = diree is given
of rates and code dimensions. These successful results indicate that by the following lemma.
this modeling approach may be a powerful tool to code design. Lemma 1: A UMC with generator matrix G(D) Go + DG1 =

achieves dfree drrec only if the minimum distance of the block

=

code with generator matrix [Go G 1] is greater than or equal to drree.

II. PROBLEM DECOMPOSITION

Proof: From (1) it can be seen that the codewords of length 2,

A. Definitions formed by the inputs {x: x(O) i 0 and x(t) 0 for t > O} (leaving
=

A time-invariant (n,k) UMC, with input xU) = (X1,X2,···,Xk) the zero state in one transition and returning to the zero state in the
and output V(!) = (V1, V2,"', Vn) at discrete time t, is represented next transition), are equal to the codewords of a BC with generator
by the encoding rule matrix [Go Gd. Hence, the free distance of the UMC cannot be
greater than the minimum distance of this block code. 0
with xU) = 0 for t < O. (1)
An immediate consequence of this lemma is that drree is upper
The code is completely defined by the k x n matrices Go and G1• bounded by the maximum minimum-distance of all linear (2n, k)
Applying the polynomial transform to (1), we obtain Be's, as tabulated by Verhoeff [10]. In fact, diree is quite frequently
equal to this upper bound. This lemma also indicates a relationship
V(D) = x(D)G(D), (2) between the (2n,k) BC's and the (n,k) UMC's: we can constrain
where the search for optimal generator matrices Go and G1 by using the
generator matrices of BC's with minimum distance equal to or greater
G(D) = Go + DG1• (3) than di,ee' as shown below.
Note that while the minimum distance of a BC does not depend
on how the columns are allocated in the generator matrix, the
We call.-v(!) and y(t) the input and output bytes, respectively. For
distance properties of a convolutional code do depend on the pairwise
the purpose of code design, a codeword of length m is defined as
combination of the columns of Go and G1, which in turn defines the
a sequence of m output bytes or, equivalently, a path in the trellis
columns of G(D). For example, if the generator matrix of the (10, .
generated by the encoder, starting in the zero state and returning to
4) BC is formed with the column types {ti} shown below, then its
the zero state for the first time after m input bytes.If a given path
minimum distance is 4:
in the trellis returns to the zero state more than once, we assume
that at least two codewords are generated. For the sake of simplicity, t1 = [1000]!, t2 = [0100]', t3 = [HOO]', t4 = [1010]',
we consider binary encoders with M memory elements, such that ts [OHO]', t6 [1110]', t7 [1001]', ts = [1101]\
M = k, where k is the number of input bits. The encoder states
= = =

are numbered from 0 to N = 2k - 1. We use dIree to designate the tg = [1011]', t10 = [1111]!.
free distance of a specific (n, k) UMC, while drree is the maximum If these ten column types are matched in five ordered pairs, such as
free distance of all the (n, k) UMC's. Of course, drree is not known
in advance; but since we found that in many cases it is equal to a (h, tg), (t2 , t4), (ts, t6), (h, t3), (tg, tID)'
1102 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 3, MAY 1993

M is column type j, i.e., h(j). k

[� l
then these pairs may form the following generator matrix of a (5,4) of For example, if = 3, we have
UMC with drcee dicee = 4:
M= [001 001 01 00 0101
1 111]1 .
=

1
1+D D D 1 +D l�D

G(D) =
1+D D
D 1+D 0 1+D
o 0 1 l+D
We can explore these facts and decompose the UMC design
problem into two subproblems:
1. find the sets of generator matrix columns that form distinct
(2n, k) BC's with minimum distance greater than or equal to
dicee (column types selection problem);
2. use these sets of columns to find the pairwise combination of
the columns in Go and G1 that forms a UMC with
(column types matching problem).
If we aim to find the optimum UMC, then we have to search for
all BC's in the column types selection problem, which still may be
impractical. On the other hand, if we aim at finding good UMC codes
then we may search only for a few BC generator matrices. In fact, all
our computational results were obtained using at most six different
block codes for each UMC. The advantage is that already known
BC's can be used or, when searching for new BC's, the computational
effort to find the minimum distance of a (2n, k) BC is much smaller
than the effort to find the free distance of the UMC.
When the minimum distance of the block code is equal to
that is, P2 = aicee and pm > aicee' Tn :::: 3, rn an integer, the BC and
the corresponding UMC have the same number of minimum weight
codewords. Hence, in some cases, we can minimize the number of
minimum weight UMC trellis paths simply by choosing a BC with
the smallest number of minimum-weight codewords.
There are three main reasons to look for a new method to design
block codes: the known code may not have the minimum number
of codewords with minimum weight; only one good code is known
while we should try several codes to construct the UMC; we wanted
to test the efficiency of combinatorial optimization to solve this kind
of problem. Actually, the proposed algorithm provided new BC's
that improved the lower bounds on Verhoeff's minimum distance
table [10] and was efficient in finding more than one code with the
desired minimum distance. It turned out to be a simple, versatile, and
practical tool.
.
1
For this case, the vector c = (Cl. C2. C3, C4, C5, CG, C7) is such that
Ci = 1 for 1 :::: i :::: 7 and that }II is the parity-check matrix of the
(7,4) Hamming code. Note that matrix Mcontains every one of the
column types, where as its transpose, M', contains all the nonzero
input bytes. Also note that the elements of the product M,21 Mare
the single digit result of all the binary scalar products that form the
codewords. The N x N matrix H defined by H= l"vlt 2111;[ can be
used to calculate the vector of Hamming weights w, as stated by the
dr,"" = diccc following theorem [5], [6].
Theorem 1 (MacDonald): A list of the Hamming weights of all
nonzero codewords of a block code represented by the vector c is
equal to the components of the N-dimensional vector w, which can
be calculated using real numbers arithmetic from
w=Hc. (4)
Proof' The element Hij is the result of the scalar product over
GF(2) between the ith input byte h(i) with the jth column type.
aicee, Hence, if the generator matrix has c] columns of type j, then the
Hijej is the contribution of these columns to the ith
real product
codeword weight. Summing all these terms up results in (4). 0
We can use this result to search for codes which have no codeword
with weight less than d;,ee' Formally, c should satisfy the constraints
(5)
where 1 is the N-dimensional vector of all ones and the vector
inequality represents term-by-term inequalities.
This linear set of constraints allows the formulation of the design
problem as a standard optimization problem. The objective function to
be minimized should add a penalty for each occurrence of a codeword
with weight violating constraints (5), and when all those constraints
are satisfied, it should count the number of codewords with minimum
weight. If we set drcee as a parameter, then one function that can be
used in the objective function is

III. THE COLUMN TYPES SELECTION PROBLEM

if x :::: dr,"e (6)
otherwise.
A. Problem Formulation
In the column types selection problem, it is necessary to calculate
The exponent fJ > 1 is usually 2 or 3, but a greater value may be
required to enusre that constraints (5) are satisfied.
the Hamming weight of all codewords, which are formed by scalar
From the definitions above, we can formulate the column types
products over GF(2) (binary operations). However, the binary opera­
selection problem as the following nonlinear integer optimization
tions can hinder some interesting properties, like convexity. To avoid
problem.
including these operations explicitly in the mathematical formulation,
we employed the modular representation of the code described below Problem (PI):
(see [5, pp. 59-62] for a more detailed description).
N
Since the ordering of the columns in a BC is irrelevant, a block
code can be described by a list with the number of columns of
Min. 7r(c) = L V(Wi) (7)
each type, and its elements can be used as variables in the design
s.t.Hc=w (8)
problem. Formally, let c = (Cl,C2,"',CN) be the N-dimensional
vector (N = 2k - 1) such that Ci = P if the code generator matrix ltc = 2n (9)
contains p columns of type i, bk(i). Note that if p = 1 for each CiE{O, 1,2"", 2n}, for i = L2.···,N. (10)
column type, then the vector c has all its components Ci, 1 :::: i :::: N.
On the other hand, if P 'I 1 for at least one column type, then the Equation (8) represents a change of variables and constraint (9)
vector c has at most (N -1) distinct components with dimensionality ensures that exactly 2n column types are selected. Note that excluding
N. Let the k x N matrix 1'vl be defined as being formed by all k­ constraints (10), this optimization problem has a convex feasible
dimensional binary column types but the all zero one. The jth column solution space and its objective function is convex U.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993 1103

TABLE I
SOME PARAMETERS OF THE NEW UNIT-MEMORY CODES

k n kin UB drree dm nm Wo GB --y(dB)

5 7 0.71 6 6 77 0.59 0.35 3.31

5 8 0.63 8 7 21 0.87 0.5 8 3.40
5 9 0.56 8 8 14 1.00 0.83 3.47
5 11 0.45 10 10 6 1.89 1. 38 3.57
5 12 0. 42 12 12 39 2.21 1.68 3.98
5 20 0.25 20 20 18 26 5.33 4.29 3.98
6 9 0.67 8 8 7 307 0.85 U.55 4.26
6 10 0.60 8 8 19 1.05 0.79 3.80
6 11 0.55 9 9 14 1.51 1 . 05 3.90
6 13 0.46 12 12 74 2.00 1.60 4.42
6 17 0.35 16 16 45 3.33 2.81 4.51
6 18 0.33 16 16 15 2 3.84 3.13 4.26
6 24 0.25 24 24 20 60 6.00 5.15 4.77
7 9 0.78 7 6 43 0.43 0.32 3.68
7 10 0.70 8 8 337 0.75 0.53 4.47
7 11 0.64 8 8 18 1.04 0.76 4.06
7 12 0.58 10 10 1 31 1.33 1.01 4.65
7 13 0.54 11 11 66 1.52 1.27 4.72
7 14 0.50 12 12 10 53 2.00 1.54 4.77
7 15 0.47 12 12 4 2.25 1.82 4.47
7 16 0.44 14 14 69 2.51 2.11 4.86
7 21 0.33 19 19 16 35 4.16 3.65 5.01
7 28 0.25 26 25 22 52 6.74 6.01 5.12
8 11 0.73 8 8 325 0.67 0.52 4.64
8 12 0.67 9 8 8 21 0.93 0.74 4.26
8 13 0.62 10 10 172 1.12 0.98 4.88
8 14 0.57 11 11 67 1.53 1.23 4.97
8 15 0.53 12 12 119 1.73 1.49 5.05
8 16 0.50 13 13 12 54 2.16 1.76 5.12
8 18 0.44 16 16 237 2.77 2.3 3 5.51
8 20 0.40 17 16 10 3.47 2.92 5.05
8 22 0.36 19 19 55 3.96 3.54 5.38
8 24 0.33 22 22 18 146 4.69 4.17 5.64

B. Local Search Algorithm Algorithm I-Column Types Selection Problem: 1) choose an ini­

Let the vector function 6(i,j) = (81(i,j),82(i,j),···,8N(i,j») tial solution c, satisfying constraints (9) and (10). A random selection
can provide fairly good initial solutions;
be defined for i f. j by
2) find i (/. I such that Ci ;::: 1 and j (/. 0 that minimize the
objective function of Problem (PI): 71"(c+ 6(i,j)); ,
8p(i,j) =
{-I� ifp =i
ifp = j (11)
3) update the solution vector
and 0; save c
C -!- C + 8(i,j) and the tabu lists I
if it is better than the best solution already found; and
otherwise.
4) if a "good solution" c is found, satisfying at least constraint (5),
then stop Algorithm I; otherwise return to step 2.
If a solution vector C is feasible to Problem (PI) [it satisfies In the last step of Algorithm I, we have used the rather vague term
constraints (9) and (10)], and if Ci > 0, then each vector �(i,j) "good solution" to allow considering the number of codewords with
represents the least change that maintains the feasibility of c+ 8(i j).
, minimum weight. Algorithm I can be interpreted as a progressive
It may be interpreted -as a column type substitution in the solution enhancement of a BC via column types substitutions. It is expected
set (column type i in and j out). These vectors can be used in a to give better results than, for example, substitution of rows or single
local search to detect the steepest descent feasible direction in the elements of the generator matrix because it is based on the properties
neighborhood of a given solution, as detailed in Algorithm I. of Problem (Pi).
Due to constraints (10), Problem (PI) is not convex. So, a steepest
descent search algorithm can be trapped by the local minima in IV. THE COLUMN 'TYPEs MATCHING PROBLEM
the objective function, without ever reaching the global minimum.
But there are methods that can be used in the search algorithm
A. Problem Formulation
to overcome this problem. We have chosen "tabu lists" [8]. The
From the solution provided by Algorithm I, we can construct the
objective of these lists is to avoid cycling in a nonmonotonic search.
1\vo "tabu" lists I and 0 are defined, that is, I and 0 contain the last
set {tt, t2 ,···, hn}, where each ti corresponds to a selected column
type. The variables of the matching problem can be represented by
L column types taken "in" and "out" of the set of selectcd column
types, respectively. (The length of the lists, L, was chosen between
a 2n x 2n matrix A such that

3 and 6.) Algorithm I describes the usage of those lists. Cycling is

avoided because after a column type got "in", it cannot get "out" in Apq =
{ 1 if column
and tq
�pes tp
m
and tq f?rm a pair, with tp in
G1(tp + Dtq IS a column of G(D»)
Go

the next L iterations, and vice versa. o otherwise.

1104 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993

TABLE II
GENERATOR MATRIX Rows (IN HEXADECIMAL NOTATION)

Go
k n Gl

5 7 43 21 13 09 06
25 5C 51 33 3A
5 8 87 46 27 15 OB
4C AD 31 A6 FF
-5 9 107 08B 045 02F OlE
IDC 039 077 143 OF2
5 11 43C 21B U9 OB7 06A
6B2 lF6 56B 32A 317
5 12 82B 46E 279 lOE 0B4
063 B2C CB8 E9B 050
5 20 8780B 441FO 262B6 14FIE OD7AB
5047E EC781 472E7 24F6C 9614B
6 9 107 085 045 026 017 OOB
040 15A OAE IE8 194 18B
6 10 200 lOC 08B 046 02F 015
226 OED 279 IFI lOB IDE
6 II 40E 215 100 096 059 03C
6E5 3FC 273 503 46E 169
6 13 107F 0878 0427 0253 014B 0090
14CF ID25 0756 13A6 06F8 OBOF
6 17 10713 08540 046F2 0279C 016AS OOA6B
lC768 1EOBB 0958E 13F02 1280E 1333A
6 18 20BB5 100C5 086EC 04E96 02ACB 01B5C
32574 2781F 3B1CE 29BB1 176A;l 0FE48
6 24 839E7E 434AEO 233045 llOOBB OB0731 05BBA6
239287 E82C90 2F6FOA B67802 0091FC 5 85A3B

Assuming that we have a feasible solUtion A, the next step. is to

employ the Viterbi algorithm [9] to calculate the minimum distances
in the trellis over S stages. The result is the truncated distance profile
{p2(A),P3(A),··· ,p5(A)}. It is also possible to find the number of
codewords of length m with weight equal to Pm(A). These numbers
define the sequence {1)2(A), 1!3(A), . .. , 1/5 (.4)}. In order to find
the best solution possible, a "reference" distance profile defined by
{ T2 , T3, ..., T5 }, with the expected optimum values of pm (A) is
�I
also needed. ao =10 a.=2
The objective function should be a type of distance profile eval­ Po = 5 P, = 6

uation function that adds a penalty whenever Pm (A) < ,.m. A '1.=2 '13'3
generalization of function (6) can be defined as Fig 1. Calculation of Problem (P2) objective function.

rex , y) = {�+ 1 - xlB if x � y

otherwise.
(12) column type. The parameters IXm in the objective function are used
primarily as pepalty "weights" for each component of the distance
profile with the purpose of enforcing the maximization of the free
The inclusion of this penalty function into the formulation of the distance and, secondly, to maximize the �istance profile growth rate.
optimization problem leads to the following. We could have considered only the minimum distance in the
objective function of Problem (P2). However we found that including
Problem (P2):
the distance profile and number of paths in the formulation would not
compromise the efficiency. On the other hand, the algorithm would
5 be much more "sensitive" to small improvements in the code and
Min. �(A) = L IXml/m(A)r(Pm(A),Tm) (13) find a UMC with dfree = dEree with less effort. So this formulation
m=2 is a consequence of the choice of a heuristic search algorithm. As
2n 2n
a result, the algorithm, while searching for a UMC with maximum
for i 1,2,... ,2n (14) dlee., simultaneously improves the distance profile and reduces the
=

j=l j=l
number of paths with minimum weight as a byproduct. Equation
for i,j 1,2,···,2n. (15)
(13) gives ttie exact definition to the expression "maximization of the
=

distance profile."
As an example of the application of this objective function, let us
Constraints (14) state that each collimn type cannot be simultane­ consider Fig. 1, with the result of the Viterbi algorithm applied to the
ously in matrices Go and G1, and should be matched to only one code generated by a particular matching. The dark lines represent
IEEE TRANSACI'IONS ON INFORMATION THEORY, VOL 39, NO.3, MAY 1993 1105

TABLE III
GENERATOR MATRIX Rows (IN HEXADECIMAL NOTATION)

GO
k n Gl

7 9 101 081 043 023 011 OOB 006

lAD 117 1F8 067 079 12A 1B6
7 10 202 106 084 043 025 016 OOF
3A9 233 OE7 lIB 383 050 314
7 11 40E 207 lOA 08B 045 020 01F
63A 709 587 314 2F6 SOB 047·
7 12 813 40E 216 llA 080 047 03F
4F2 C71 95F C9C 016 ADS 22F
7 13 103E 082C 0435 0213 0139 0090 004F
OE32 099B 0751 1809 lBAO 1580 02D7
7 14 207C 1060 0859 O41B 027A 012B 009E
OE9C 1C65 3B8A 2E7F 0347 37A1 38FO
7 15 4087 2090 lOSE 08AB 0440 0278 0173
5687 1C3B 63CF 6B29 64F8 45B3 34AD
7 16 816F 41A5 21AA 100C 08E9 05FO 02B6
11B1 F213 05CO 35C3 6805 7CA8 3AFF
7 21 10282F 082C9C 04238F 021F7E 0105E6 009AC3 00523B
1AAF22 OEE140 157446 078842 OCBE1C 145FBO 05F3C9
7 28 8030027 41C217F 21A9753 llB14BC 0955FOO 044BB96 028E7F8
6483807 6000F62 9B4B045 C72A9EB 029E84C B54EC2A AE67AF7
8 11 407 201 105 083 044 026 016 OOA
413 3AC 658 57F 077 3C2 100 6B8
8 12 80B 405 203 lOE 08C 04F 027 016
54E C65 FA1 417 098 ACO 4BC 66B
8 13 lOlA 0813 0416 0200 OllC 008B 0055 002E
08FE 1404 1027 1FFE 0393 0731 026E llAC
8 14 203F 1026 080E 0439 0215 011A 0087 006B
ISDO 2AC7 0970 20F7 1D1E 2304 32BC 3028
8 15 4032 207F 1057 085C 0471 0278 013E 009B
IC9E 4414 300B 60CO 497B 3519 620B 27F7
8 16 8033 40E1 20EC 1030 087E 04BA ,02CB 0155
0058 A590 007E 3994 A623 C087 46A9 F7C1
8 18 2022C 1004B 083C5 043FE 02259 D1308 00893 004E3
OB99F 1755B 0363E 22703 25AB3 lEC62 107A6 32A67
8 20 8052F 4009A 2OB33 100El 08AF8 047CC 02E4B 0129F
AC361 C1COB 06905 2F852 23200 70FA3 740EC 98019
8 22 203300- 103402 083C75 0412FD 022AC3 01009A 008DE1 004B04
346639 30EC9B 21998E 281061 lE29AE 2EF960 OAA7FO 13E07B
8 24 8001B8 4086E4 2002CA lOF800 08E056 049893 020071 017FOO
88C3FB 09B58B FC4C96 CA3F42 594B8C 55FA3A B6E508 FFFFFF

paths in the trellis corresponding to the codewords of minimum B. Local Search Algorithm
weight. Observe that there are two paths with length 2 and weight 5, A feasible solution from Problem (P1) makes it possible to obtain
and three paths with length 3 and weight 6 (P2(A) = 5, parA) = 6, matrices Go and G1. The minimum change that keeps the feasibility
1/2(A) = 2, 1/3(A) = 3). Using T2 = 6, Ta = 7, a2 = 10, aa = 2, is interchanging the position of two columns in these matrices. This
(3 2, and S 3, we have
= =
process can be used as the elementary operation required to detect
the steepest descent direction in Problem (P2) as in the following
erA) = 10 x 2 x (7 - 5)2 + 2 x 3 x (8 - 6)2 = 104.
local search algorithm.

The reference distance profile sequence {r m} can be defined to Algorithm II-Column Types Matching Problem: 1) build up ma­
impose the colldition that each BC obtained by truncating the UMC trices Go and Gl with the column types provided by Algorithm I
should satisfy the Gilbert-Varshamov lower bound [3]. Defining from a random matching:
wo as the mean grQwt\1 of the distance profile, that is, Wo = 2) for each column i = 1,2", . ,2n, find the column j -:f i, that,
[ps(A) - P2(A)]/(S - 2), we want if iuterchanged with column i, will minimize the objective function
(14), that is, e(A);
Wo > n1-{-l(l - kin) = GB (16) 3) if the objective function decreases, then interchange columns i
and j and go back to step 2, otherwise go to step 4;
where 1-{-1 is the inverse binary entropy function. 4) if a "good" solution is found (at least dfre. = dIree) then stop,
The reference dist!lnce profile sequence {Tm} can be initially else go to step 1 to restart the search from another initial solution.
chosen as Tm = drree + (m - 2) . GB. The set of weights {am} Note that to reduce the search effort, only a subset pf all possible
should form a decreasing sequence, e.g., am = 2-m. The best values column interchanges is tried in each iteration, and the method to
depend on the specific code and usually can be found after a few trials. avoid local minima is to restart with a new initial solution. If the
This choice is not qiti"al: smaIl variations in the distance profile will reference distance profile i� properly chosen and a number S is large
not degrade the code performance. enough, this search algOrithm will naturally eliminate the catastrophic
1106 IEEE TRANSACTIONS ON INFORMAI10N THEORY, VOL. 39, NO.3, MAY 1993

TABLE IV
GENERATOR MATRIX COLUMNS OF THE NEW LINEAR BLOCK CODES

Generator Matrix Columns

k n d dB (decimal representation)

8 32 13 12-13 1 2 4 8 16 32 64 128 28 31
41 49 59 62 67 77 90 102 112 147
149 167 178 180 185 198 209 219 220 233
236 254
8 60 27 26-28 1 2 4 8 16 32 64 128 13 23
25 26 35 37 49 59 61 69 71 73
77 79 81 90 91 92 99 100 111 112
115 117 119 123 133 143 1 47 152 155 173
174 177 191 195 197 203 204 206 208 209
215 226 229 231 233 236 243 249 250 253
8 73 33 32-34 1 2 4 8 16 32 64 128 11 13
21 22 26 27 37 40 47 50 55 60
71 75 76 80 82 85 88 95 96 102
105 106 110 113 115 11� 121 126 127 129
132 133 135 139 147 148 150 152 156 163
167 169 173 174 176 181 186 191 193 194
200 207 210 219 221 222 227 228 230 236
240 249 255
8 76 35 34-36 1 2 4 8 16 32 64 128 13 17
23 26 29 30 38 44 45 47 50 51
52 53 57 59 63 70 82 83 85 91
93 94 99 100 101 106 111 112 118 119
120 124 133 134 138 141 142 145 150 158
159 160 164 167 171 174 182 184 188 194
199 200 201 203 209 212 218 221 222 223
224 227 236 244 247 255
9 43 17 16-18 1 2 4 8 16 32 64 128 256 21
30 37 46 66 95 107 114 124 133 147
156 174 184 200 203 205 244 278 311 327
334 371 372 376 385 414 423 424 429 436
464 471 481
9 46 19 18-20 1 2 4 8 16 32 64 12� 256 14
21 56 62 1 04 115 122 131 167 185 190
220 235 246 267 284 303 311 322 328 345
350 357 361 364 388 402 409 416 429 435
447 451 461 494 500 504

d is the minimum Hamming distance achieved by the (n, k) block code.

dB is the upper and lower-bound on the Hamming distance from Verhoeff table [1O).

codes, due to their poor distance profile. Now the term "good" is used Tables II and III present for each new (n, k) UMC the rows in
to evaluate the number of minimum weight paths and the distance hexadecimal notation of the corresponding generator matrices. All
profile growth. BC's used were obtained from Algorithm T. In most of the cases,
the maximum minimum-distance is achieved, and Algorithm II was
able to search for a UMC with dfree equal to the maximum minimum-
V. COMPUTATIONAL RESULTS
distance of the block code (2n, k). The only exceptions are the (8,5),
(9,7), (12, 8), and (20, 8) codes. The last two codes,(12,8) and (20, 8),
Algorithms I and II were programmed and run on a PC AT
may achieve the maximum dfree since no (24,8) and (40,8) BC's
microcomputer. To show the efficacy of the new design method, we
with minimum distance equal to the corresponding upper bounds are
have tabulated 33 new codes. We believe this sample of codes gives a
known. All (8,5) UMC's found with dfree = 8 were catastrophic;
good indication of the potentialities of the new method. Some codes
and the code (9,7) may be considered a special case due to its high
which have free distance equal to previously published codes (UMC
rate. The (32,8) BC improves the lower bound on Verhoeff's table
or multimemory) are not listed, but there are some exceptions which
are included, and the reason for that are discussed ahead. Table I (updated version [11j), and the corresponding UMC has higher dfree
contains some parameters of the new codes, viz. than its quasi-cyclic version [3]. Note that all new codes satisfy
inequality (16).
k, n number of input and output bits, respectively
Some codes that duplicate previous results are as follows.
UB upper bound on dfree (from [11])
dfrcc free distance of the new UMC • The code (14,7) is very similar to the code found by Justesen
d= free distance of the optimum multimemory code e/ al. [3]; it is included to show that good distance profiles can
'Tlm number of minimum weight paths be found with Algorithm II. The Justesen et al. code distance
lUa mean growth of the distance profile profile is 12, 12, 13, 15, 18, 19, 22, ..., and the code found by
GB Gilbert- Varshamov lower bound on acceptable lUa Algorithm II has distance profile 12, 12, 13, 15, 17, 19, 21.
1= Rdfree/2 asymptotic coding gain (AWGN channel and hard • Codes (20,5), (18,6), and (24,6) have the same free distance
decoding). as the unit-memory codes found by Lee [1]. The differences are
IEEE TRANSACfIONS ON INFORMATION THEORY, VOL, 39, NO.3, MAY 1993 1107

TABLE V
MATRIX P COLUMNS OF Ho,ig OF THE NEW LINEAR BLOCK CODES

Matrix P Columns
n n d dB (4 bit byte decimal representation)

8 33 14 13-14 7 4 12 14 13 3 0 0
11 14 15 13 12 12 3 0
1 13 4 8 11 6 14 1
12 6 13 0 11 7 0 15
11 5 4 9 11 8 7 15
4 1 11 15 9 7 15 15
8 61 28 27-28 14 5 12 11 7 0 0 0
15 9 4 1 3 15 0 0
15 2 14 9 8 8 7 0
14 5 12 13 3 0 15 0
10 10 5 12 12 3 15 0
11 10 9 8 7 15 15 0
15 13 11 5 12 12 12 3
11 10 1 7 14 1 0 15
7 11 10 10 6 14 1 15
12 5 11 7 0 0 15 15
6 3 2 0 14 1 15 15
14 4 13 3 0 15 15 15
13 10 1 7 15 15 15 15
8 74 34 33-34 14 9 7 12 3 0 0 0
6 12 2 13 12 3 0 0
10 14 ]] 9 7 15 0 0
12 12 10 6 1 0 15 0
5 9 5 3 15 0 15 0
2 5 4 3 0 15 15 0
6 11 9 8 7 15 15 0
11 6 6 14 14 14 14 1
7 3 14 1 0 0 0 15
8 10 6 1 15 0 0 15
7 6 10 9 8 7 0 15
9 4 13 12 3 15 0 15
6 13 4 12 12 12 3 15
5 7 4 13 3 0 15 15
10 6 13 12 12 3 15 15
1 8 12 5 3 15 15 15
8 8 8 8 8 8 8 8

that while Lee's (18,6) code has 45 minimum-weight paths, the
new code has only 2, and the distance profile growth rate of
Lee's (20,5) and (24,6) codes do not satisfy (16).
To illustrate the computational effort required by the algorithms, a
(24,8) UMC, with 384 0-1 design variables, was found after roughly
five hours of CPU time, which can be considered an upper bound
on the allowed search time. However, the computational effort was
highly dependent on the code and optimization objectives. In some
cases, the time required to enhance the code's distance profile or
For the sake of understanding the notation, let us consider the
first column of matrix P of the linear block code (74,8,34), that
is, (14,6,10,12,5,2,6,11,7,8,7,9,6,5,10,1,8)t. This represents the
binary column (1110 0110 1010 1100 0101 0010 0110 1011 0111
1000 0111 1001 0110 0101 1010 0001 1000)'. Note that the rightmost
3 bits "0" must be disconsidered in order to match the right amount
of row of Horig.
VI. CONCLUSION

number of codewords with minimum distance was much higher than
the time required to find a code with maximum dfree• The time
required to solve the column types selection problem is usually
smaller than the time to solve the matching problem, but for some
low rate codes it was greater.
Tables IV and V present the new linear block codes found by
Algorithm 1. Particularly, Table IV presents the generator matrix
columns in decimal representation, whereas Table V presents matrix
P columns, in decimal representation for each 4 bit byte, of
Horig = [Pll]
where Horig is the parity-check matrix of the original code, and I is
the identity matrix when the extension method [12, Ch.1, pp. 27-28]
is employed'.
We have presented a new method to design good unit-memory
convolutional codes. We have shown how the original problem can
be decomposed in the column types selection and matching problems.
From this decomposition, efficient algorithms were devised to solve
the corresponding problems. As a result, we were ahle to find new
UMC's, most of them with maximum df,ee.
From the computational results, we believe that, with further
research, this methodology can be enhanced with the inclusion of
algebraic properties of the UMC and with the adaptation of more
powerful combinatorial optimization methods to the particularities of
the problems, leading to more efficient methods to design optimal
UMC's. The techniques used to obtain the optimization problems
are not very restrictive, hence there is the interesting possibility of
adapting the method to the design of linear unit-memory codes over
q-ary fields or rings, and using the Hamming, Lee, or Euclidean
distance to measure its performance.
1108 IEEE TRANSACfIONS ON INFORMATION THEORY, VOL.39, NO. 3, MAY 1993

TABLE V
(CONTINUED)

Matrix P Columns
n n d dH (4 bit byte decimal representation)

8 77 36 35 - 36 14 3 10 <} 7 U U 0
8 6 15 13 12 3 0 0
13 7 12 12 3 15 0 0
7 1 12 3 15 15 0 0
9 15 12 8 11 8 7 0
14 5 11 7 15 0 15 0
10 9 6 1 0 15 15 0
9 11 11 8 7 15 15 0
2 1 7 12 12 12 12 3
5 10 6 14 1 0 0 15
2 14 14 6 14 1 0 15
6 7 13 3 0 15 0 15
0 9 10 6 14 14 1 15
11 9 8 7 0 0 15 15
9 2 5 3 15 0 15 15
5 13 12 12 12 3 15 15
3 3 15 9 7 15 15 15
9 44 18 1 7 - 18 10 5 15 15 12 3 0 0 0
6 15 4 6 5 3 15 0 0
6 2 13 9 11 8 8 7 0
1 9 8 15 4 12 3 15 0
9 3 15 8 7 5 12 12 3
10 14 13 4 3 3 15 a 15
5 3 3 10 10 9 8 7 15
4 0 6 12 3 14 1 15 15
12 8 8 U 8 4 12 12 12
9 47 20 19 - 20 4 9 13 11 7 3 0 0 0
5 7 0 10 6 14 14 0
12 10 11 7 7 14 1 15 0
10 14 5 11 5 12 12 12 3
12 14 12 9 4 12 3 0 15
11 4 6 13 12 3 15 0 15
1 2 12 9 3 8 8 7 IS
7 3 5 5 3 15 0 15 15
12 10 7 6 1 3 15 15 15
U U 0 8 8 8 8 8 8

ACKNOWLEDGMENT [10] T. Verhoeff, "An upd ated table of minimum-distance bounds for binary
linear codes," IEEE Trans. Inform. Theory, v ol . IT-33, p. 665-680,
The authors would like to thank the referees for their detailed Sepl. 1987.
[11] __ , "An updated table of minimum-distance bounds for binary linear
comments and valuable suggestions which improved the presentation
codes," Dept. M ath. Comput . Sci., Eindhoven Univ. Techno!., Jan 1989.
of this work. [12] F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting
Codes. Amsterd am, The Netherlands: North-Holland, 1977.

REFERENCES

[1] L. N. Lee, "Short unit-memory byte oriented bi n ary convolutional codes

having maximal free distance," IEEE Trans. Inform. Theory, vol . IT-22,
pp. 349-352, May 1976.
[2] C. T hommesen and J. Justesen, "Bounds on distance and error expo­
nents of unit-memory codes," IEEE Trans. Infrom. Theory, vol. IT-29,
pp. 637 -649, Sept . 1983.
[3] J. Ju stesen, E. Paaske, and M. Balas, "Quasi-cyclic unit-memory codes,"
IEEE Trans. Inform. Theory, vol. IT-36, pp. 540-547, May 1990.
[4] C. Chang and K. Yao, "Systolic array processing of the Viterbi algo­
rithm," IEEE Trans. Inform. Theory, vol. IT-35, pp. 76-86, Jan. 1 989.
[5] W. W. Peterson and E.1. Weldon, Error Currecting Cude,� 2nd ed.
Cambridge, MA, MIT Press, 1 972.
[6] J. E. MacDonald, "Design method for maximum minimum-distances
error-correcting codes," IBM J. Res. Develop., vol. 4, pp. 43-57, Jan.
1960.
[7] G. L. Nemhauser and L. A. Wolsey , Integer and Combinatorial
Optimization. New York: Wiley, 1986.
[8] F. Gl over, "Tabu search," CAAI Rep. 88-3, Univ. Colorado, Boulder,
Nov. 198ti.
[9] A. J. Viterbi and J. K. Omura, Principles of Digital Communication and
Coding. New York: McGraw-Hill, 1979.