Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Save to My Library
Look up keyword
Like this
14Activity
0 of .
Results for:
No results containing your search query
P. 1
Sorting

Sorting

Ratings: (0)|Views: 314 |Likes:
Published by ainugiri

More info:

Published by: ainugiri on Aug 19, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

09/04/2010

pdf

text

original

 
BOLYAI SOCIETY
Entropy, Search, Complexity,
pp. 209–232.
MATHEMATICAL STUDIES, 16
Analysis of Sorting Algorithms byKolmogorov Complexity(A Survey)
PAUL VIT´ANYI
Recently, many results on the computational complexity of sorting algorithmswere obtained using Kolmogorov complexity (the incompressibility method). Es-pecially, the usually hard average-case analysis is ammenable to this method.Here we survey such results about Bubblesort, Heapsort, Shellsort, Dobosiewicz-sort, Shakersort, and sorting with stacks and queues in sequential or parallelmode. Especially in the case of Shellsort the uses of Kolmogorov complexity sur-prisingly easily resolved problems that had stayed open for a long time despitestrenuous attacks.
1. Introduction
We survey recent results in the analysis of sorting algorithms using a newtechnical tool: the incompressibility method based on Kolmogorov complex-ity. Complementing approaches such as the counting method and the prob-abilistic method, the new method is especially suited for the average-caseanalysis of algorithms and machine models, whereas average-case analysis isusually more difficult than worst-case analysis using the traditional meth-ods. Obviously, the results described can be obtained using other proof methods – all true provable statements must be provable from the axiomsof mathematics by the inference methods of mathematics. The question iswhether a particular proof method facilitates and guides the proving effort.The following examples make clear that thinking in terms of coding and the
Supported in part via NeuroCOLT II ESPRIT Working Group.
 
210
P. Vit´anyi
incompressibility method suggests simple proofs that resolve long-standingopen problems. A survey of the use of the incompressibility method in com-binatorics, computational complexity, and the analysis of algorithms is [16]Chapter 6, and other recent work is [2, 15].We give some definitions to establish notation. For introduction, details,and proofs, see [16]. We write
string 
to mean a finite binary string. Otherfinite objects can be encoded into strings in natural ways. The set of stringsis denoted by
{
0
,
1
}
. Let
x,y,z
∈ N 
, where
denotes the set of naturalnumbers. Identify
and
{
0
,
1
}
according to the correspondence(0
,ε
)
,
(1
,
0)
,
(2
,
1)
,
(3
,
00)
,
(4
,
01)
, ... .
Here
ε
denotes the
empty wor
with no letters. The
length 
of 
x
is thenumber of bits in the binary string
x
and is denoted by
l
(
x
). For example,
l
(010) = 3 and
l
(
ε
) = 0. The emphasis is on binary sequences only forconvenience; observations in every alphabet can be so encoded in a waythat is ‘theory neutral.’
Self-delimiting Codes:
A binary string
y
is a
proper prefix 
of a binarystring
x
if we can write
x
=
yz
for
z
=
ε
. A set
{
x,y,...
} ⊆ {
0
,
1
}
is
prefix- free
if for every pair of distinct elements in the set neither is a proper prefixof the other. A prefix-free set is also called a
prefix code
. Each binary string
x
=
x
1
x
2
...x
n
has a special type of prefix code, called a
self-delimiting code
,¯
x
= 1
n
0
x
1
x
2
...x
n
.
This code is self-delimiting because we can effectively determine where thecode word ¯
x
ends by reading it from left to right without backing up. Usingthis code we define the standard self-delimiting code for
x
to be
x
=
l
(
x
)
x
.It is easy to check that
l
(¯
x
) = 2
n
+ 1 and
l
(
x
) =
n
+ 2log
n
+ 1.Let
·
,
·
be a standard one-one mapping from
N ×
to
, for technicalreasons chosen such that
l
x,y
=
l
(
y
)+
l
(
x
)+2
l
l
(
x
)
+1, for example
x,y
=
x
y
= 1
l
(
l
(
x
)
)
0
l
(
x
)
xy
.
Kolmogorov Complexity:
Informally, the Kolmogorov complexity, or al-gorithmic entropy,
(
x
) of a string
x
is the length (number of bits) of ashortest binary program (string) to compute
x
on a fixed reference univer-sal computer (such as a particular universal Turing machine). Intuitively,
(
x
) represents the minimal amount of information required to generate
x
by any effective process, [10]. The conditional Kolmogorov complexity
(
x
|
y
) of 
x
relative to
y
is defined similarly as the length of a shortest
 
Analysis of Sorting Algorithms by Kolmogorov Complexity 
211
program to compute
x
, if 
y
is furnished as an auxiliary input to the com-putation. The functions
(
·
) and
(
· | ·
), though defined in terms of aparticular machine model, are machine-independent up to an additive con-stant (depending on the particular enumeration of Turing machines andthe particular reference universal Turing machine selected). They acquirean asymptotically universal and absolute character through Church’s the-sis, and from the ability of universal machines to simulate one another andexecute any effective process, see for example [16]. Formally:
Definition 1.
Let
0
,
1
,...
be a standard enumeration of all Turing ma-chines. Choose a universal Turing machine
that expresses its universalityin the following manner:
(
i,p
,y
)
=
i
 p,y
for all
i
and
 p,y
, where
p
denotes a Turing program for
i
and
y
anauxiliary input. We fix
as our
reference universal computer 
and definethe
conditional Kolmogorov complexity 
of 
x
given
y
by
(
x
|
y
) = min
q
∈{
0
,
1
}
{
l
(
q
) :
q,y
=
x
}
,
for every
q
(for example
q
=
i,p
above) and auxiliary input
y
. The
unconditional Kolmogorov complexity 
of 
x
is defined by
(
x
) =
(
x
|
ε
).For convenience we write
(
x,y
) for
x,y
, and
(
x
|
y,z
) for
x
|
y,z
.
Incompressibility:
First we show that the Kolmogorov complexity of a string cannot be significantly more than its length. Since there is aTuring machine, say
i
, that computes the identity function
i
(
x
)
x
,and by definition of universality of 
we have
i,p
=
i
(
 p
). Hence,
(
x
)
l
(
x
) +
c
for fixed
c
2log
i
+ 1 and all
x
.
1 2
It is easy to see that there are also strings that can be described byprograms much shorter than themselves. For instance, the function definedby
(1) = 2 and
(
i
) = 2
(
i
1)
for
i >
1 grows very fast,
(
k
) is a “stack”of 
k
twos. It is clear that for every
k
it is the case that
(
k
) has complexityat most
(
k
) +
O
(1).
1
“2log
i
” and not “log
i
” since we need to encode
i
in such a way that
can determinethe end of the encoding. One way to do that is to use the code 1
l
(
l
(
i
))
0
l
(
i
)
i
which haslength 2
l
(
l
(
i
)
)
+
l
(
i
) + 1
<
2log
i
bits.
2
In what follows, “log” denotes the binary logarithm.
r
” is the greatest integer
q
such that
q
r
.

Activity (14)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
puneet04 liked this
manuelq9 liked this
ainugiri liked this
pandia_mca liked this
Tanya Moldovan liked this
ishansaxena1988 liked this

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->