Professional Documents
Culture Documents
Linear-Time Sorting
Lower Bounds
Claim
searching among n preprocessed items requires (lg n) time
= binary search, AVL tree search optimal
theorem
proof
counterexample
Decision Tree
Any comparison algorithm can be viewed/specified as a tree of all possible comparison
outcomes & resulting output, for a particular n:
example, binary search for n = 3:
Lecture 7
Linear-Time Sorting
A[1] < x?
NO
NO
x A[0]
YES
A[2] < x?
A[0] < x?
YES
NO
YES
A[2] < x
Lecture 7
Linear-Time Sorting
# leaves n!
= height lg n!
= lg(1 2 (n 1) n)
= lg 1 + lg 2 + + lg(n 1) + lg n
n
X
=
lg i
i=1
n
X
i=n/2
n
X
i=n/2
lg i
n
lg
2
|{z}
=lg n1
n
n
lg n = (n lg n)
2
2
2n
n n
e
Linear-time Sorting
If n keys are integers (fitting in a word) 0, 1, . . . , k 1, can do more than compare them
= lower bounds dont apply
if k = nO(1) , can sort in O(n) time
OPEN: O(n) time possible for all k?
Counting Sort
L = array of k empty lists
linked or Python lists
for j in range n:
L[key(A[j])].append(A[j])
O(1)
| {z }
random access using integer key
O(n)
output = [ ]
for i in range k:
output.extend(L[i])
O( i (1 + |L[i]|)) = O(k + n)
O(k)
Lecture 7
Linear-Time Sorting
Time: (n + k)
also (n + k) space
Intuition: Count key occurrences using RAM output <count> copies of each key in order
. . . but item is more than just a key
CLRS has cooler implementation of counting sort with counters, no lists but time bound
is the same
Radix Sort
imagine each integer in base b
= d = logb k digits {0, 1, . . . , b 1}
sort (all n items) by least significant digit can extract in O(1) time
sort by most significant digit can extract in O(1) time
sort must be stable: preserve relative order of items with the same key
= dont mess up previous sorting
For example:
3
4
6
8
4
7
3
2
5
5
3
3
2
5
9
7
7
9
6
0
5
7
3
4
4
6
3
8
sort
2
5
3
5
5
2
3
0
5
6
7
7
9
9
7
3
4
8
3
4
6
sorted
2
2
3
3
5
5
5
0
9
6
9
5
7
7
sorted
3
3
4
4
6
7
8
2
5
3
5
5
2
3
9
5
6
7
7
0
9
sorted
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.