i=1
2
li
(1)
The necessary and sufcient condition for the existence
of prexfree codes with codeword lengths L, is /(L) 1,
however in regard to the existence of xfree codes, there is a
remarkable conjecture. Ahlswede, Balkenhol, and Khachatrian
propose in [1] the conjecture that /(L)
3
4
implies the
existence of a binary xfree code with codeword lengths L.
Later Harada and Kobayashi generalized the conjecture for
qarray alphabets [2]. Over the last few years, many attempts
have been made to prove the 3/4conjecture either for the
general case [1], [6], or for the special kind of length [1],
[2], [3], [5], [7]. Despite this, the main conjecture remains
unsolved. For some special cases, the conjecture was proven
by exhaustive computer searches [4], [5]. A complete
history about what have been done on 3/4conjecture can be
found in [7] and [8]. The results of some of these works are
illustrated below:
(a): If /(L)
5
8
, there exists a xfree code with length
vector L. [5]
(b): If l
1
= 1 and /(L)
3
4
, there exists a xfree code
with length vector L. [6]
The performance of a code is usually measured by its
redundancy, which is dened as:
= L
ave
H(T) (2)
where the rst part denotes the average length of a code,
L
ave
, and the second part refers to the entropy of the source.
The code with minimum redundancy over T is called the
optimal code.
After Ahlswede et. al. [1] proved that for xfree codes,
0 < 2, several works have been congured based
on the redundancy of such codes. Recently, Yekhanin[6]
improved the upper bound on , the redundancy of a
x free code, from 2 to 4 log
2
5. He showed that if
the 3/4conjecture is proven, the upper bound on will
be reduced to 3log
2
3 which is approximately equal to 1.415.
Ye and Yeung [3], [4] also derived several upper bounds
on in terms of p
1
(the probability of the most probable
symbol). Their results can be summarized in the following
inequality
< h(p
1
) = min
_
4 3p
1
H
b
(p
1
), 2 H
b
(p
1
)
(1 p
1
) log
_
1 2
log p1
_
p
1
(1 log p
1
)
_
(3)
Where H
b
(p
1
) is the entropy of binary distribution
(p
1
, 1 p
1
).
In this paper, rst we derive a lower bound on the redun
dancy of xfree codes according to some knowledge about the
source and then introduce a new upper bound on the length of
most probable symbol in the optimal xfree code. Throughout
this paper, all exponents, logarithms are of base 2.
1015
9781424422470/08/$25.00 2008 IEEE.
II. SIMPLE LOWER BOUND ON THE
REDUNDANCY OF OPTIMAL FIXFREE CODE
Assume that a discrete memoryless source, o, is given.
Let T = (p
1
, p
2
, , p
m
) be the probability distribution
of the source symbols. Without loss of generality, suppose
p
1
p
2
p
m
. With this assumption, it is clear that
l
1
l
2
l
m
. Moreover, assume o
f
= s
1
, s
2
, , s
m
= k : is a
prex of s
k
, and
2
= k : is a sufx of s
k
. Moreover,
we dene P
1
() =
i
1
p
i
, and P
2
() =
i
2
p
i
.
Theorem 1: Let /, the set of codewords satised in the
xfree condition, and p
1
and l
1
, the probability of the most
probable codeword and its length, be known. The redundancy
of the xfree source satises the following inequality:
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
) (4)
Proof:
=
i
p
i
_
[s
i
[ + log(p
i
)
_
=
1
2
F
l
1
2
_
k
1
p
k
([s
k
[ + log(p
k
))
+
k
2
p
k
([s
k
[ + log(p
k
))
_
=
1
2
F
l
1
2
k
1
p
k
P
1
()
P
1
()
_
[s
k
[ +[[
+log
_
p
k
P
1
()
P
1
()
_
_
+
1
2
F
l
1
2
k
2
p
k
P
2
()
P
2
()
_
[s
k
[ +[[
+log
_
p
k
P
2
()
P
2
()
_
_
where [s
k
[ = [s
k
[ [[ = [s
k
[ l
1
. Therefore, the redundancy
can be written as
=
1
2
F
l
1
2
_
P
1
()
k
1
p
k
P
1
()
_
[s
k
[ + log
p
k
P
1
()
_
+P
1
()
_
[[ + log P
1
()
_
_
+
1
2
F
l
1
2
_
P
2
()
k
2
p
k
P
2
()
_
[s
k
[ + log
p
k
P
2
()
_
+P
2
()
_
[[ + log P
2
()
_
_
For any ,
k
1
p
k
P1()
_
[s
k
[ +log
p
k
P1()
_
can be considered
as the redundancy of a source with codewords s
k
: k
. Here,
by s
k
we mean s
k
without its rst [[ letters. Similarly,
k
2
p
k
P2()
_
[s
k
[ + log
p
k
P2()
_
can be considered as the
redundancy of a source with codewords s
k
/ : k
2
,
and probability distribution
p
k
P2()
: k
2
. Here, by s
k
/
we mean s
k
without its last [[ letters. Therefore, these two
terms have positive values.
1
2
F
l
1
2
P
1
()
_
[[ + log P
1
()
_
+
1
2
F
l
1
2
P
2
()
_
[[ + log P
2
()
_
1
2
min
P1()
_
F
l
1
2
P
1
()
_
[[ + log P
1
()
_
_
+
1
2
min
P2()
_
F
l
1
2
P
2
()
_
[[ + log P
2
()
_
_
To nd the minimum value of Q(P
1
)
F
l
1
2
P
1
()
_
[[ +
log P
1
()
_
, we use Lagrange multipliers. Here, the only
constraint is g(P
1
)
=s1
P
1
() (1 p
1
) = 0. The
partial derivatives of the function f(P
1
) Q(P
1
) + g(P
1
)
are set to zero. Therefore, for any ,= s
1
,
f
P
1
()
= [[ + log P
1
() +
1
ln2
+ = 0.
Solving the above equation, for any ,= s
1
, we get P
1
() =
2
(+l1+
1
ln 2
)
. To determine , we use the fact that g(P
1
) = 0.
Finally, we conclude that the optimal distribution is P
() =
1p1
2
l
11
. Moreover, the same function achieves the minimum in
the second minimization problem. Therefore,
F
l
1
2
P
()[[ +
F
l
1
2
P
() log P
()
= l
1
(1 p
1
) log
_
2
l1
1
_
H
b
(p
1
)
III. NEW UPPER BOUND ON THE LENGTH OF THE
MOST PROBABLE SOURCE SYMBOL
Ye and Yeung [4] derived an upper bound on the redundancy
of xfree codes in an elegant way in regard to the partial
information about the probability of the most likely symbol.
In the next theorem, using the new results of [5], [6], we
improve the corresponding bound.
Theorem 2: Let p
1
be the probability of the most likely
symbol of an optimal code. The redundancy of the code
1016
satises in the following inequality
min
_
2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
_
1 log p
1

_
, 3 2p
1
H
b
(p
1
)
_
(5)
Proof: Consider a code o
1
with codeword lengths, L
1
as follows:
l
1
i
=
_
log p
1
 + 1 i = 1
_
log
_
p
i
1.252
log p
1
1p1
__
+ 1 i ,= 1
_
The validity of codeword lengths is guaranteed since p
1
,
p
i
1.252
log p
1
1p1
are less than 1. The Kraft sum associated
with the code can be calculated as follows:
/(L
1
) =
n
i=1
2
l
1
i
= 2
l
1
1
+
n
i=2
2
l
1
i
=
1
2
2
log p1
+
1
2
n
i=2
2
_
log
_
pi
1.252
log p
1
1p
1
_
_
1
2
2
log p1
+
1
2
_
1.25 2
log p1
_
=
5
8
According to (a), there exists a xfree code with L
1
codeword lengths. The redundancy of this code is:
1
= L
1
ave
H(T) = p
1
(l
1
1
+ log p
1
) +
n
i=2
p
i
_
l
1
i
+ log p
i
_
< p
1
_
log p
1
 + 1 + log p
1
_
+ 2
n
i=2
p
i
i=2
_
p
i
log
_
1.25 2
log p1
1 p
1
_
_
= 2 p
1
_
1 log p
1

_
+ p
1
log p
1
(1 p
1
) log
_
1.25 2
log p1
1 p
1
_
= 2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
(1 log p
1
)
f(p
1
) (6)
Dene f(p
1
) as the above function. Moreover, consider
another code o
2
with codeword lengths, L
2
as follows:
l
2
i
=
_
1 i = 1
_
log
pi
1p1
_
+ 2 i ,= 1
_
The validity of codeword lengths is guaranteed since for
any codeword
pi
1p1
is less than 1. The Kraft sum of the code
can be calculated as follows:
/(L
2
) =
n
i=1
2
l
2
i
= 2
l
2
1
+
n
i=2
2
l
2
i
2
1
+ 2
2
n
i=2
_
p
i
1 p
1
_
=
3
4
(7)
Since l
2
1
= 1, and /(L
2
)
3
4
, according to (b), there exists
a xfree code with L
2
codeword lengths. The redundancy of
o
2
can be calculated as follows:
2
= L
2
ave
H(T)
= p
1
(l
2
1
+ log p
1
) +
n
i=2
p
i
_
l
2
i
+ log p
i
_
< p
1
_
1 + log p
1
_
+
n
i=2
p
i
_
log
1
1 p
1
+ 3
_
= 3 2p
1
H
b
(p
1
)
g(p
1
) (8)
Dene g(p
1
) as the above function. f(p
1
) and g(p
1
) as
functions of p
1
are shown in Fig. 1. As it can be easily
observed, f is less than g for p
1
< 0.33, In Fig. 1. we also
compared the min(f(p
1
), g(p
1
)) with h(p
1
) derived by
Ye and Yeung[3], [4].
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.5
1
1.5
2
2.5
3
f(p1)
g(p1)
min(f(p1),g(p1))
h(p1)
Fig. 1. Upper bounds on the redundancy of optimal xfree codes
o
1
and o
2
are predened xfree codes, therefore they are
not necessary optimal. Due to the fact that, the optimal x
free code is a code with minimum redundancy among all x
free codes, the redundancy of optimal xfree code is upper
bounded by the redundancy of the mentioned codes, o
1
and
o
2
, therefore, min(f(p
1
), g(p
1
)).
1017
According to lower bound, derived in the previous section,
and upper bound, introduced in recent theorem, we can
originate the following Corollary.
Corollary 1: The maximum length of the most probable
codeword, l
1
, in the optimal x free code with source dis
tribution vector, T, is bounded by:
(l
1
)
max
=
_
T(p
1
) p
1
< 0.33
((p
1
) p
1
0.33
where
T(p
1
) =
_
_
2 (1 p
1
) log
_
1.25 2
log p1
_
_
p
1
(1 log p
1
)
_
((p
1
) =
_
3 2p
1
p
1
_
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
2
4
6
8
10
12
14
X: 0.33
Y: 9
F(p1)
G(p1)
Fig. 2. Upper bounds on the length of most probable codeword in
the optimal code
Proof: According to the Theorem 1 and Theorem 2, in
case p
1
< 0.33 we have:
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
)
< 2 H
b
(p
1
) (1 p
1
) log
_
1.25 2
log p1
_
p
1
(1 log p
1
) (9)
Therefore,
l
1
<
2 (1 p
1
) log
_
1.25 2
log p1
_
p
1
_
1 log p
1

_
and for p
1
0.33 :
l
1
(1 p
1
) log(2
l1
1) H
b
(p
1
) < 3 2p
1
H
b
(p
1
)
thus,
l
1
<
3 2p
1
p
1
(10)
Corollary 2: If p
1
> 0.75, the length of the most probable
code in the optimal xfree code is equal to 1.
Proof: Based on previous corollary 1, if
32p1
p1
< 2, the
length of the most probable codeword, l
1
, is at most equal to
1.
IV. CONCLUSION
In this paper we consider some basic properties of xfree
codes. The lower bound on the redundancy of the optimal
xfree code is obtained by some partial information about
the source. Moreover, by using a set of predened codes we
nd an upper bound on the redundancy. Using our results, we
establish a new upper bound on l
1
in terms p
1
. All of the
results can be extended similalrly for D array codes.
V. REFERENCES
[1] R. Ahlswede, B. Balkenhol, and L. Khachatrian, Some
properties of xfree codes, in Proc. 1st Int Seminar on
Coding Theory and Combinatorics, Thahkadzor, Armenia,
1996, pp. 2033.
[2] K. Harada, and K. Kobayashi, a note on the xfree
property, IEICE Trans. Fundam., vol. E82A, no. 10,pp. 2121
2128, Oct. 1999.
[3] C. Ye, and R. W. Yeung, On the xfree codes, Proc. Int.
Symp. On Information Theory, Sorrento, Italy, p. 426, June
2000.
[4] C. Ye, and R. W. Yeung, Some basic properties of xfree
codes, IEEE Trans. Inform. Theory, vol. 47, no. 1, pp. 7287,
Jan. 2001.
[5] S. Yekhanin, Sufcient conditions of existence of xfree
codes, Proc. Int. Symp. On Information Theory, Washington
D.C., pp. 284, June 2001.
[6] S. Yekhanin, Improved upper bound for the redundancy
of xfree codes, IEEE Trans. Inform. Theory, vol. 50, no.
11, pp. 28152818, Nov. 2004.
[7] Z. Kukorelly, and K. Zeger, Sufcient Condition for
Existence of Binary FixFree Codes, IEEE Trans. Inform.
Theory, vol. 51, no. 10, pp. 34333444, Oct. 2005.
[8] H. Schnettler, On the
3
4
Conjecture for FixFree Codes,
arXiv:0709.2598v1, Sep. 2007.
1018