Lec4 PDF

Introduction to Algorithms
6.046J/18.401J
LECTURE 4
Quicksort
Divide and conquer
Partitioning
Worst-case analysis
Intuition
Randomized quicksort
Analysis
Prof. Charles E. Leiserson
September 21, 2005
Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson
L4.1
Quicksort
Proposed by C.A.R. Hoare in 1962.
Divide-and-conquer algorithm.
Sorts in place (like insertion sort, but not
like merge sort).
Very practical (with tuning).
September 21, 2005
L4.2
Divide and conquer

Quicksort an n-element array:
1. Divide: Partition the array into two subarrays
around a pivot x such that elements in lower
subarray x elements in upper subarray.
xx
xx
xx
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
Key: Linear-time partitioning subroutine.
September 21, 2005
L4.3
Partitioning subroutine
PARTITION(A, p, q) A[ p . . q]
x A[ p]
pivot = A[ p]
Running
Running time
time
ip
== O(n)
O(n) for
for nn
for j p + 1 to q
elements.
elements.
do if A[ j] x
then i i + 1
exchange A[i] A[ j]
exchange A[ p] A[i]
return i
Invariant: xx
p
September 21, 2005
xx
xx
i
??
j
q
L4.4
Example of partitioning
66 10
10 13
13 55
i
j
September 21, 2005
88
33
22 11
11
L4.5
66 10
10 13
13 55
i
j
September 21, 2005
88
33
22 11
11
L4.6
66 10
10 13
13 55
i
j
September 21, 2005
88
33
22 11
11
L4.7
66 10
10 13
13 55
66
September 21, 2005
88
33
22 11
11
55 13
13 10
10 88
i
j
33
22 11
11
L4.8
66 10
10 13
13 55
66
September 21, 2005
88
33
22 11
11
55 13
13 10
10 88
i
j
33
22 11
11
L4.9
66 10
10 13
13 55
66
September 21, 2005
88
33
22 11
11
55 13
13 10
10 88
i
33
j
22 11
11
L4.10
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
September 21, 2005
33 10
10 88 13
13 22 11
11
i
j
L4.11
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
September 21, 2005
33 10
10 88 13
13 22 11
11
i
j
L4.12
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
33 10
10 88 13
13 22 11
11
66
55
33
September 21, 2005
22
i
88 13
13 10
10 11
11
j
L4.13
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
33 10
10 88 13
13 22 11
11
66
55
33
September 21, 2005
22
i
88 13
13 10
10 11
11
j
L4.14
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
33 10
10 88 13
13 22 11
11
66
55
33
September 21, 2005
22
i
88 13
13 10
10 11
11
L4.15
66 10
10 13
13 55
88
33
22 11
11
66
55 13
13 10
10 88
33
22 11
11
66
55
33 10
10 88 13
13 22 11
11
66
55
33
22
88 13
13 10
10 11
11
22
55
33
66
i
88 13
13 10
10 11
11
September 21, 2005
L4.16
Pseudocode for quicksort

QUICKSORT(A, p, r)
if p < r
then q PARTITION(A, p, r)
QUICKSORT(A, p, q1)
QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)
September 21, 2005
L4.17
Analysis of quicksort
Assume all input elements are distinct.
In practice, there are better partitioning
algorithms for when duplicate input
elements may exist.
Let T(n) = worst-case running time on
an array of n elements.
September 21, 2005
L4.18
Worst-case of quicksort
Input sorted or reverse sorted.
Partition around min or max element.
One side of partition always has no elements.
T (n) = T (0) + T (n 1) + (n)

= (1) + T (n 1) + (n)
= T (n 1) + (n)
= ( n 2 )
September 21, 2005
(arithmetic series)
L4.19
Worst-case recursion tree

T(n) = T(0) + T(n1) + cn
September 21, 2005
L4.20

T(n) = T(0) + T(n1) + cn
T(n)
September 21, 2005
L4.21

T(n) = T(0) + T(n1) + cn
cn
T(0) T(n1)
September 21, 2005
L4.22

T(n) = T(0) + T(n1) + cn
cn
T(0) c(n1)
T(0) T(n2)
September 21, 2005
L4.23

T(n) = T(0) + T(n1) + cn
cn
T(0) c(n1)
T(0) c(n2)
T(0)
O
(1)
September 21, 2005
L4.24

T(n) = T(0) + T(n1) + cn
cn
T(0) c(n1)
T(0) c(n2)
T(0)
n
k = (n 2 )
k =1
O
(1)
September 21, 2005
L4.25

T(n) = T(0) + T(n1) + cn
cn
(1) c(n1)
h=n
(1) c(n2)
(1)
n
k = (n 2 )
k =1
O
T(n) = (n) + (n2)

= (n2)
(1)
September 21, 2005
L4.26
Best-case analysis
(For intuition only!)
If were lucky, PARTITION splits the array evenly:

T(n) = 2T(n/2) + (n)
(same as merge sort)
= (n lg n)
What if the split is
1 9
always 10 : 10 ?
T (n) = T (101 n ) + T (109 n ) + (n)

What is the solution to this recurrence?
September 21, 2005
L4.27
Analysis of almost-best case

T (n)
September 21, 2005
L4.28

cn
T (101 n )
September 21, 2005
T (109 n )
L4.29

cn
1
10
cn
9
10
cn
9
9
81
1
T (100
n ) T (100
n ) T (100
n )T (100
n)
September 21, 2005
L4.30

1
10
cn
9
100
9
10
cn
O(n)
O(n) leaves
leaves
(1)
cn
log10/9n
9
81
cn
cn
100
100
1
100
cn
cn
cn
cn
cn
(1)
September 21, 2005
L4.31

1
10
9
100
9
10
cn
(n lg n)
Lucky!
September 21, 2005
O(n)
O(n) leaves
leaves
(1)
cn
log10/9n
9
81
cn
cn
100
100
log10n
1
cn
100
cn
cn
cn
cn
cn
(1)
cn log10n T(n) cn log10/9n + (n)
L4.32
More intuition
Suppose we alternate lucky, unlucky,
lucky, unlucky, lucky, .
L(n) = 2U(n/2) + (n) lucky
U(n) = L(n 1) + (n) unlucky
Solving:
L(n) = 2(L(n/2 1) + (n/2)) + (n)
= 2L(n/2 1) + (n)
= (n lg n) Lucky!
How can we make sure we are usually lucky?
September 21, 2005
L4.33
IDEA: Partition around a random element.
Running time is independent of the input
order.
No assumptions need to be made about
the input distribution.
No specific input elicits the worst-case
behavior.
The worst case is determined only by the
output of a random-number generator.
September 21, 2005
L4.34
analysis
Let T(n) = the random variable for the running
time of randomized quicksort on an input of size
n, assuming random numbers are independent.
For k = 0, 1, , n1, define the indicator
random variable
Xk =
1 if PARTITION generates a k : nk1 split,

0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are

equally likely, assuming elements are distinct.
September 21, 2005
L4.35
Analysis (continued)
T(n) =
T(0) + T(n1) + (n) if 0 : n1 split,

T(1) + T(n2) + (n) if 1 : n2 split,
M
T(n1) + T(0) + (n) if n1 : 0 split,
n 1
= X k (T (k ) + T (n k 1) + (n) )
k =0
September 21, 2005
L4.36
Calculating expectation
n 1
E[T (n)] = E X k (T (k ) + T (n k 1) + (n) )

k =0
Take expectations of both sides.
September 21, 2005
L4.37
n 1
E[T (n)] = E X k (T (k ) + T (n k 1) + (n) )

k =0
n 1
E[ X k (T (k ) + T (n k 1) + (n) )]
k =0
Linearity of expectation.
September 21, 2005
L4.38
n 1
E[T (n)] = E X k (T (k ) + T (n k 1) + (n) )

k =0
=
=
n 1
E[ X k (T (k ) + T (n k 1) + (n) )]
k =0
n 1
E[ X k ] E[T (k ) + T (n k 1) + (n)]
k =0
Independence of Xk from other random

choices.
September 21, 2005
L4.39
n 1
E[T (n)] = E X k (T (k ) + T (n k 1) + (n) )

k =0
=
=
n 1
E[ X k (T (k ) + T (n k 1) + (n) )]
k =0
n 1
E[ X k ] E[T (k ) + T (n k 1) + (n)]
k =0
n 1
n 1
n 1
= 1 E [T (k )] + 1 E [T (n k 1)] + 1 (n)
n k =0
n k =0
n k =0
Linearity of expectation; E[Xk] = 1/n .

September 21, 2005
L4.40
n 1
E[T (n)] = E X k (T (k ) + T ( n k 1) + (n) )
k =0
=
=
n 1
E[ X k (T (k ) + T (n k 1) + (n) )]
k =0
n 1
E[ X k ] E[T (k ) + T (n k 1) + (n)]
k =0
n 1
n 1
n 1
= 1 E [T (k )] + 1 E [T (n k 1)] + 1 (n)
n k =0
n k =0
n k =0
n 1
= 2 E [T (k )] + (n)
n k =1
September 21, 2005
Summations have
identical terms.
L4.41
Hairy recurrence
n 1
E[T (n)] = 2 E [T (k )] + (n)

n k =2
(The k = 0, 1 terms can be absorbed in the (n).)
Prove: E[T(n)] an lg n for constant a > 0 .
Choose a large enough so that an lg n
dominates E[T(n)] for sufficiently small n 2.
n 1
Use fact:
1 n 2 lg n 1n 2
k
lg
k
(exercise).
2
8
k =2
September 21, 2005
L4.42
Substitution method
n 1
E [T (n)] 2 ak lg k + (n)
n k =2
Substitute inductive hypothesis.
September 21, 2005
L4.43
Substitution method
n 1
E [T (n)] 2 ak lg k + (n)
n k =2
2a 1 n 2 lg n 1 n 2 + (n)
n 2
8
Use fact.
September 21, 2005
L4.44
Substitution method
n 1
E [T (n)] 2 ak lg k + (n)
n k =2
2a 1 n 2 lg n 1 n 2 + (n)
n 2
8
= an lg n an (n)
4
Express as desired residual.
September 21, 2005
L4.45
Substitution method
n 1
E [T (n)] 2 ak lg k + (n)
n k =2
= 2a 1 n 2 lg n 1 n 2 + (n)
n 2
8
= an lg n an (n)
4
an lg n ,
if a is chosen large enough so that
an/4 dominates the (n).
September 21, 2005
L4.46
Quicksort in practice
Quicksort is a great general-purpose
sorting algorithm.
Quicksort is typically over twice as fast
as merge sort.
Quicksort can benefit substantially from
code tuning.
Quicksort behaves well even with
caching and virtual memory.
September 21, 2005
L4.47

Lec4 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec4 PDF

Uploaded by

Copyright:

Available Formats

Introduction to Algorithms

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Divide and conquer

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Pseudocode for quicksort

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

T (n) = T (0) + T (n 1) + (n)

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

Worst-case recursion tree

T(n) = (n) + (n2)

September 21, 2005

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson

If were lucky, PARTITION splits the array evenly:

T (n) = T (101 n ) + T (109 n ) + (n)

Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson