Some Notes On Fix-Free Codes

Ali Kakhbod, Ali Nazari

Department of Electrical Engineering and Computer Science
University of Michigan, Ann Arbor, MI, USA
Email: akakhbod, anazari@umich.edu
Morteza Zadimoghaddam
Department of Computer Engineering
Sharif University of Technology, Iran
Email: zadimoghaddam@ce.sharif.edu
AbstractA variable-length code is called a x-free code if
it is both prex-free and sufx-free. In this paper, we consider
some basic properties of x-free codes. We obtain one lower and
one upper bound on the redundancy of the optimal x-free code.
Comparing these bounds, we derive an upper bound on the
length of the most likely source symbol in terms of its probability.
Index Terms Fix-free code, Prex-free, Sufx-free, Redundancy.
I. INTRODUCTION
A set of codewords is called x-free if it is both prex-free
and sufx-free, i.e. , any codeword of a x-free code is
neither a prex nor a sufx of any other codeword. Fix-free
codes are also known as Reversible Variable Length Codes
(RVLCs).
In x-free codes, the decoding of any nite set of codewords
in both directions is possible, which can be used by a decoder
to enhance robustness in presence of channel noise, and also
speed up the decoding process by factor 2. Fix-free codes
have been used in the development of international video
compression standards such as H.263+ and MPEG-4.
Consider a source with m symbols and the probability
distribution of T = (p
1
, p
2
, , p
m
) with p
1
p
2
. . .
p
m
, let o = s
1
, s
2
, ..., s
m
be a binary x-free code over
distribution T with codeword lengths L = (l
1
, l
2
, , l
m
).
The Kraft sum of a code o is dened as:
/(L) =
m

i=1
2
li
(1)
The necessary and sufcient condition for the existence
of prex-free codes with codeword lengths L, is /(L) 1,
however in regard to the existence of x-free codes, there is a
remarkable conjecture. Ahlswede, Balkenhol, and Khachatrian
propose in [1] the conjecture that /(L)
3
4
implies the
existence of a binary x-free code with codeword lengths L.
Later Harada and Kobayashi generalized the conjecture for
q-array alphabets [2]. Over the last few years, many attempts
have been made to prove the 3/4-conjecture either for the
general case [1], [6], or for the special kind of length [1],
[2], [3], [5], [7]. Despite this, the main conjecture remains
unsolved. For some special cases, the conjecture was proven
by exhaustive computer searches [4], [5]. A complete
history about what have been done on 3/4-conjecture can be
found in [7] and [8]. The results of some of these works are
illustrated below:
(a): If /(L)
5
8
, there exists a x-free code with length
vector L. [5]
(b): If l
1
= 1 and /(L)
3
4
, there exists a x-free code
with length vector L. [6]
The performance of a code is usually measured by its
redundancy, which is dened as:
= L
ave
H(T) (2)
where the rst part denotes the average length of a code,
L
ave
, and the second part refers to the entropy of the source.
The code with minimum redundancy over T is called the
optimal code.
After Ahlswede et. al. [1] proved that for x-free codes,
0 < 2, several works have been congured based
on the redundancy of such codes. Recently, Yekhanin[6]
improved the upper bound on , the redundancy of a
x free code, from 2 to 4 log
2
5. He showed that if
the 3/4-conjecture is proven, the upper bound on will
be reduced to 3log
2
3 which is approximately equal to 1.415.
Ye and Yeung [3], [4] also derived several upper bounds
on in terms of p
1
(the probability of the most probable
symbol). Their results can be summarized in the following
inequality
< h(p
1
) = min
_
4 3p
1
H
b
(p
1
), 2 H
b
(p
1
)
(1 p
1
) log
_
1 2
log p1
_
p
1
(1 log p
1
|)
_
(3)
Where H
b
(p
1
) is the entropy of binary distribution
(p
1
, 1 p
1
).
In this paper, rst we derive a lower bound on the redun-
dancy of x-free codes according to some knowledge about the
source and then introduce a new upper bound on the length of
most probable symbol in the optimal x-free code. Throughout
this paper, all exponents, logarithms are of base 2.
1015
978-1-4244-2247-0/08/$25.00 2008 IEEE.
II. SIMPLE LOWER BOUND ON THE
REDUNDANCY OF OPTIMAL FIX-FREE CODE
Assume that a discrete memoryless source, o, is given.
Let T = (p
1
, p
2
, , p
m
) be the probability distribution
of the source symbols. Without loss of generality, suppose
p
1
p
2
p
m
. With this assumption, it is clear that
l
1
l
2
l
m
. Moreover, assume o
f
= s
1
, s
2
, , s
m

is an optimal binary x-free code of the given source. In this

section, we will nd a new lower bound on the redundancy
of the optimal x-free code in terms of p
1
, the probability of
the most likely symbol, and l
1
, the length of this symbol.
Denition 1: For any F
l1
2
, we dene
1

= k : is a
prex of s
k
, and
2

= k : is a sufx of s
k
. Moreover,
we dene P
1
() =

i
1

p
i
, and P
2
() =

i
2

p
i
.
Theorem 1: Let /, the set of codewords satised in the
x-free condition, and p
1
and l
1
, the probability of the most
probable codeword and its length, be known. The redundancy
of the x-free source satises the following inequality:
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
) (4)
Proof:
=

i
p
i
_
[s
i
[ + log(p
i
)
_
=
1
2

F
l
1
2
_

k
1

p
k
([s
k
[ + log(p
k
))
+

k
2

p
k
([s
k
[ + log(p
k
))
_
=
1
2

F
l
1
2

k
1

p
k
P
1
()
P
1
()
_
[s

k
[ +[[
+log
_
p
k
P
1
()
P
1
()
_
_
+
1
2

F
l
1
2

k
2

p
k
P
2
()
P
2
()
_
[s

k
[ +[[
+log
_
p
k
P
2
()
P
2
()
_
_
where [s

k
[ = [s
k
[ [[ = [s
k
[ l
1
. Therefore, the redundancy
can be written as
=
1
2

F
l
1
2
_
P
1
()

k
1

p
k
P
1
()
_
[s

k
[ + log
p
k
P
1
()
_
+P
1
()
_
[[ + log P
1
()
_
_
+
1
2

F
l
1
2
_
P
2
()

k
2

p
k
P
2
()
_
[s

k
[ + log
p
k
P
2
()
_
+P
2
()
_
[[ + log P
2
()
_
_
For any ,

k
1

p
k
P1()
_
[s

k
[ +log
p
k
P1()
_
can be considered
as the redundancy of a source with codewords s
k
: k

, and probability distribution

p
k
P1()
: k
1

. Here,
by s
k
we mean s
k
without its rst [[ letters. Similarly,

k
2

p
k
P2()
_
[s

k
[ + log
p
k
P2()
_
can be considered as the
redundancy of a source with codewords s
k
/ : k
2

,
and probability distribution
p
k
P2()
: k
2

. Here, by s
k
/
we mean s
k
without its last [[ letters. Therefore, these two
terms have positive values.

1
2

F
l
1
2
P
1
()
_
[[ + log P
1
()
_
+
1
2

F
l
1
2
P
2
()
_
[[ + log P
2
()
_

1
2
min
P1()
_

F
l
1
2
P
1
()
_
[[ + log P
1
()
_
_
+
1
2
min
P2()
_

F
l
1
2
P
2
()
_
[[ + log P
2
()
_
_
To nd the minimum value of Q(P
1
)

F
l
1
2
P
1
()
_
[[ +
log P
1
()
_
, we use Lagrange multipliers. Here, the only
constraint is g(P
1
)

=s1
P
1
() (1 p
1
) = 0. The
partial derivatives of the function f(P
1
) Q(P
1
) + g(P
1
)
are set to zero. Therefore, for any ,= s
1
,
f
P
1
()
= [[ + log P
1
() +
1
ln2
+ = 0.
Solving the above equation, for any ,= s
1
, we get P
1
() =
2
(+l1+
1
ln 2
)
. To determine , we use the fact that g(P
1
) = 0.
Finally, we conclude that the optimal distribution is P

() =
1p1
2
l
11
. Moreover, the same function achieves the minimum in
the second minimization problem. Therefore,

F
l
1
2
P

()[[ +

F
l
1
2
P

() log P

()
= l
1
(1 p
1
) log
_
2
l1
1
_
H
b
(p
1
)
III. NEW UPPER BOUND ON THE LENGTH OF THE
MOST PROBABLE SOURCE SYMBOL
Ye and Yeung [4] derived an upper bound on the redundancy
of x-free codes in an elegant way in regard to the partial
information about the probability of the most likely symbol.
In the next theorem, using the new results of [5], [6], we
improve the corresponding bound.
Theorem 2: Let p
1
be the probability of the most likely
symbol of an optimal code. The redundancy of the code
1016
satises in the following inequality
min
_
2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
_
1 log p
1
|
_
, 3 2p
1
H
b
(p
1
)
_
(5)
Proof: Consider a code o
1
with codeword lengths, L
1
as follows:
l
1
i
=
_
log p
1
| + 1 i = 1
_
log
_
p
i
1.252
log p
1

1p1
__
+ 1 i ,= 1
_
The validity of codeword lengths is guaranteed since p
1
,
p
i
1.252
log p
1

1p1
are less than 1. The Kraft sum associated
with the code can be calculated as follows:
/(L
1
) =
n

i=1
2
l
1
i
= 2
l
1
1
+
n

i=2
2
l
1
i
=
1
2
2
log p1
+
1
2
n

i=2
2

_
log
_
pi
1.252
log p
1

1p
1
_
_

1
2
2
log p1
+
1
2
_
1.25 2
log p1
_
=
5
8
According to (a), there exists a x-free code with L
1
codeword lengths. The redundancy of this code is:

1
= L
1
ave
H(T) = p
1
(l
1
1
+ log p
1
) +
n

i=2
p
i
_
l
1
i
+ log p
i
_
< p
1
_
log p
1
| + 1 + log p
1
_
+ 2
n

i=2
p
i

i=2
_
p
i
log
_
1.25 2
log p1
1 p
1
_
_
= 2 p
1
_
1 log p
1
|
_
+ p
1
log p
1
(1 p
1
) log
_
1.25 2
log p1
1 p
1
_
= 2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
(1 log p
1
|)
f(p
1
) (6)
Dene f(p
1
) as the above function. Moreover, consider
another code o
2
with codeword lengths, L
2
as follows:
l
2
i
=
_
1 i = 1
_
log
pi
1p1
_
+ 2 i ,= 1
_
The validity of codeword lengths is guaranteed since for
any codeword
pi
1p1
is less than 1. The Kraft sum of the code
can be calculated as follows:
/(L
2
) =
n

i=1
2
l
2
i
= 2
l
2
1
+
n

i=2
2
l
2
i
2
1
+ 2
2
n

i=2
_
p
i
1 p
1
_
=
3
4
(7)
Since l
2
1
= 1, and /(L
2
)
3
4
, according to (b), there exists
a x-free code with L
2
codeword lengths. The redundancy of
o
2
can be calculated as follows:

2
= L
2
ave
H(T)
= p
1
(l
2
1
+ log p
1
) +
n

i=2
p
i
_
l
2
i
+ log p
i
_
< p
1
_
1 + log p
1
_
+
n

i=2
p
i
_
log
1
1 p
1
+ 3
_
= 3 2p
1
H
b
(p
1
)
g(p
1
) (8)
Dene g(p
1
) as the above function. f(p
1
) and g(p
1
) as
functions of p
1
are shown in Fig. 1. As it can be easily
observed, f is less than g for p
1
< 0.33, In Fig. 1. we also
compared the min(f(p
1
), g(p
1
)) with h(p
1
) derived by
Ye and Yeung[3], [4].
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.5
1
1.5
2
2.5
3


f(p1)
g(p1)
min(f(p1),g(p1))
h(p1)
Fig. 1. Upper bounds on the redundancy of optimal x-free codes
o
1
and o
2
are predened x-free codes, therefore they are
not necessary optimal. Due to the fact that, the optimal x-
free code is a code with minimum redundancy among all x-
free codes, the redundancy of optimal x-free code is upper
bounded by the redundancy of the mentioned codes, o
1
and
o
2
, therefore, min(f(p
1
), g(p
1
)).
1017
According to lower bound, derived in the previous section,
and upper bound, introduced in recent theorem, we can
originate the following Corollary.
Corollary 1: The maximum length of the most probable
codeword, l
1
, in the optimal x free code with source dis-
tribution vector, T, is bounded by:
(l
1
)
max
=
_
T(p
1
) p
1
< 0.33
((p
1
) p
1
0.33
where
T(p
1
) =
_
_
2 (1 p
1
) log
_
1.25 2
log p1
_
_
p
1
(1 log p
1
|)
_
((p
1
) =
_
3 2p
1
p
1
_
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
2
4
6
8
10
12
14


X: 0.33
Y: 9
F(p1)
G(p1)
Fig. 2. Upper bounds on the length of most probable codeword in
the optimal code
Proof: According to the Theorem 1 and Theorem 2, in
case p
1
< 0.33 we have:
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
)
< 2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
(1 log p
1
|) (9)
Therefore,
l
1
<
2 (1 p
1
) log
_
1.25 2
log p1
_
p
1

_
1 log p
1
|
_
and for p
1
0.33 :
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
) < 3 2p
1
H
b
(p
1
)
thus,
l
1
<
3 2p
1
p
1
(10)
Corollary 2: If p
1
> 0.75, the length of the most probable
code in the optimal x-free code is equal to 1.
Proof: Based on previous corollary 1, if
32p1
p1
< 2, the
length of the most probable codeword, l
1
, is at most equal to
1.
IV. CONCLUSION
In this paper we consider some basic properties of x-free
codes. The lower bound on the redundancy of the optimal
x-free code is obtained by some partial information about
the source. Moreover, by using a set of pre-dened codes we
nd an upper bound on the redundancy. Using our results, we
establish a new upper bound on l
1
in terms p
1
. All of the
results can be extended similalrly for D array codes.
V. REFERENCES
[1] R. Ahlswede, B. Balkenhol, and L. Khachatrian, Some
properties of x-free codes, in Proc. 1st Int Seminar on
Coding Theory and Combinatorics, Thahkadzor, Armenia,
1996, pp. 20-33.
[2] K. Harada, and K. Kobayashi, a note on the x-free
property, IEICE Trans. Fundam., vol. E82A, no. 10,pp. 2121-
2128, Oct. 1999.
[3] C. Ye, and R. W. Yeung, On the x-free codes, Proc. Int.
Symp. On Information Theory, Sorrento, Italy, p. 426, June
2000.
[4] C. Ye, and R. W. Yeung, Some basic properties of x-free
codes, IEEE Trans. Inform. Theory, vol. 47, no. 1, pp. 72-87,
Jan. 2001.
[5] S. Yekhanin, Sufcient conditions of existence of x-free
codes, Proc. Int. Symp. On Information Theory, Washington
D.C., pp. 284, June 2001.
[6] S. Yekhanin, Improved upper bound for the redundancy
of x-free codes, IEEE Trans. Inform. Theory, vol. 50, no.
11, pp. 2815-2818, Nov. 2004.
[7] Z. Kukorelly, and K. Zeger, Sufcient Condition for
Existence of Binary Fix-Free Codes, IEEE Trans. Inform.
Theory, vol. 51, no. 10, pp. 3433-3444, Oct. 2005.
[8] H. Schnettler, On the
3
4
Conjecture for Fix-Free Codes,
arXiv:0709.2598v1, Sep. 2007.
1018