2.

3 Q UICKSORT
‣ quicksort
‣ selection
‣ duplicate keys
Algorithms
‣ system sorts

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Quicksort overview

Step 1. Shuffle the array.

Step 2. Partition the array so that, for some j

・Entry a[j] is in place.

・No larger entry to the left of j.
・No smaller entry to the right of j.
Step 3. Sort each subarray recursively.

input Q U I C K S O R T E X A M P L E
shuffle K R A T E L E P U I M Q C X O S
partitioning item
partition E C A I E K L P U T M Q R X O S
not greater not less
sort left A C E E I K L P U T M Q R X O S
sort right A C E E I K L M O P Q R S T U X
result A C E E I K L M O P Q R S T U X
Quicksort overview
6
 Tony Hoare
n u m b e r ) . 9.9 X 10 45 is u s e d to r e p r e s e n t i n f i n i t y . I m a g i n a r y
v a l u e s of x m a y n o t be n e g a t i v e a n d reM v a l u e s of x m a y n o t be
s m a l l e r t h a n 1.
V a l u e s of Qd~'(x) m a y be c a l c u l a t e d e a s i l y by h y p e r g e o m e t r i c
series if x is n o t t o o s m a l l n o r (n - m ) t o o large. Q~m(x) c a n be
c o m p u t e d f r o m a n a p p r o p r i a t e set of v a l u e s of Pnm(X) if X is n e a r
conunent I a n d J are o u t p u t v a r i a b l e s , a n d A is t h e a r r a y ( w i t h
s u b s c r i p t b o u n d s M : N ) w h i c h is o p e r a t e d u p o n b y t h i s p r o c e d u r e .
P a r t i t i o n t a k e s t h e v a l u e X of a r a n d o m e l e m e n t of t h e a r r a y A,
a n d r e a r r a n g e s t h e v a l u e s of t h e e l e m e n t s of t h e a r r a y in s u c h a
w a y t h a t t h e r e e x i s t i n t e g e r s I a n d J w i t h t h e following p r o p e r t i e s :
M _-< J < I =< N p r o v i d e d M < N

・Invented quicksort to translate Russian into English.

1.0 or ix is n e a r 0. L o s s of s i g n i f i c a n t d i g i t s o c c u r s for x as s m a l l as A[R] =< X f o r M =< R _-< J
1.1 if n is l a r g e r t h a n 10. L o s s of s i g n i f i c a n t d i g i t s is a m a j o r diffi- A[R] = X f o r J < R < I
c u l t y in u s i n g finite p o l y n o m i M r e p r e s e n t a t i o n s also if n is l a r g e r A[R] ~ X f o r I =< R ~ N

・[ but couldn't explain his algorithm or implement it! ]

t h a n m . H o w e v e r , Q L E G h a s b e e n t e s t e d in r e g i o n s of x a n d n The procedure uses an integer procedure random (M,N) which
b o t h large a n d small; chooses equiprobably a random integer F between M and N, and
procedure Q L E G ( m , n m a x , x, ri, R, Q); v a l u e In, n m a x , x, ri; also a p r o c e d u r e e x c h a n g e , w h i c h e x c h a n g e s t h e v a l u e s of its t w o
r e a l In, m n a x , x, ri; r e a l a r r a y R , Q; parameters ;

・Learned Algol 60 (and recursion).

begin r e a l t, i, n, q0, s;
n : = 20;
i f n m a x > 13 t h e n
begin real X; integer F;
F : = r a n d o m ( M , N ) ; X : = A[F];
I:=M; J:=N;

・Implemented quicksort.
n := nmax +7; up: for I : = I s t e p 1 u n t i l N d o
i f ri = 0 t h e n i f X < A [I] t h e n g o to d o w n ;
begin ifm = 0then I:=N;
Q[0] : = 0.5 X 10g((x + 1 ) / ( x - 1)) down: f o r J : = J s t e p --1 u n t i l M d o
else if A[J]<X then go to change;
begin t : = - - 1 . 0 / s q r t ( x X x - - 1); J:=M;
q0 : = 0; c h a n g e : i f I < J t h e n b e g i n e x c h a n g e (A[IL A[J]); Tony Hoare

Q[O] : = t ; I := I+ 1;J:= J - 1;
for i : = 1 step 1 until m do g o to u p 1980 Turing Award
begin s := (x+x)X(i-1)Xt end 2: b e g i n
4
×Q[0]+ (3i-i×i-2)×q0; else i f [ < F t h e n b e g i n e x c h a n g e (A[IL A[F]) i i f c >= 0 t h e n
q0 : = Q[0]; I:=I+l 3:begin
Q[0] : = s e n d e n d ; end e:= aXe;f := bXd;goto8
if x = 1 then else i f F < J t l l e n b e g i n e x c h a n g e (A[F], A[J]) ; end 3 ;
Q[0] : = 9.9 I" 45; J:=J-1 e:=bXc;
R [ n + 1] : = x - s q r t ( x X x - 1); end ; ifd ~ 0then
for i : = n s t e p --1 u n t i l 1 d o end partition 4: begin
R[i] : = (i + m ) / ( ( i + i + 1) X x f:=bXd; goto8
+(m-i- 1) X R [ i + l ] ) ; e n d 4;
ALGORITHM 61
g o to t h e e n d ; f:=aXd; goto8
if m = 0 then ALGORITHM
PROCEDURES 64F O R R A N G E ARITHMETIC 5: e n d 2;
b e g i n i f x < 0.5 t b e n QUICKSORT
ALLAN GIBB* ifb > 0 then
Q[0] : = a r c t a n ( x ) - 1.5707963 e l s e U n iA.
C. i t y HOARE
v e r sR. of A l b e r t a , C a l g a r y , A l b e r t a , C a n a d a 6: b e g i n
Q[0] : = - a r e t a n ( 1 / x ) e n d e l s e if d > 0 then
Elliott
b e g i n Brothers Ltd., Borehamwood, Hertfordshire, Eng.
begin t : = 1 / s q r t ( x X x + 1); begin
p r o c e d u r e R A N G E S U M (a, b, c, d, e, f);
q0 : = 0; procedure quicksort (A,M,N); value M,N; e : = M I N ( a X d, b X c);
real a , b , c , d , e , f ;
q[0] := t; a r r a y A; i n t e g e r M , N ; f : = M A X ( a X c , b X d); go t o 8
c o m m e n t The term "range n u m b e r " was used by P. S. Dwyer,
for i : = 2 step 1 until m do comment Q u i c k s o r t is a v e r y f a s t a n d c o n v e n i e n t m e t h o d of e n d 6;
b e g i n s : = (x + x) X (i -- 1) X t X Q[0I
Linear Computations (Wiley, 1951). Machine procedures for
s o r t i n g a n a r r a y in t h e r a n d o m - a c c e s s s t o r e of a c o m p u t e r . T h e e : = b X c; f : = a X c; go t o 8
range arithmetic were developed about 1958 by Ramon Moore,
+(3i+iX i -- 2) × q0; e n t i r e c o n t e n t s of t h e s t o r e m a y be s o r t e d , since no e x t r a s p a c e is e n d 5;
"Automatic Error Analysis in Digital C o m p u t a t i o n , " LMSD
qO : = Q[0]; r e q u i r e d . T h e a v e r a g e n u m b e r of c o m p a r i s o n s m a d e is 2 ( M - - N ) In f:=aXc;
Report 48421, 28 Jan. 1959, Lockheed Missiles and Space Divi-
Q[0] := s e n d e n d ; ( N - - M ) , a n d t h e a v e r a g e n m n b e r of e x c h a n g e s is one s i x t h t h i s i f d _-<O t h e n
sion, Palo Alto, California, 59 pp. If a _< x -< b and c ~ y ~ d,
R[n + 1] : = x - s q r t ( x × x + 1); a m o u n t . S u i t a b l e r e f i n e m e n t s of t h i s m e t h o d will be d e s i r a b l e for 7: b e g i n
then R A N G E S U M yields an interval [e, f] such t h a t e =< (x + y)
for i := n step - 1 until 1 do its i m p l e m e n t a t i o n on a n y a c t u a l c o m p u t e r ; e:=bXd; goto8
f. Because of machine operation (truncation or rounding) the
R[i] : = (i + m ) / ( ( i -- m + 1) × R[i + 1] begin i n t e g e r 1,J ; end 7 ;
machine sums a -4- c and b -4- d may not provide safe end-points
--(i+i+ 1) X x); if M < N then begin partition (A,M,N,I,J); e:=aXd;
of the output interval. Thus R A N G E S U M requires a non-local
for i : = 1 step 2 until nmax do quicksort (A,M,J) ; 8: e : = A D J U S T P R O D (e, - 1 ) ;
real procedure A D J U S T S U M which will compensate for the
Rill : = -- Rill; q u i c k s o r t (A, I, N ) f := A D J U S T P R O D (f, 1)
machine arithmetic. The body of A D J U S T S U M will be de-
the: for i : = 1 step 1 until nnmx do end end RANGEMPY;
pendent upon the type of machine for which it is written and so
Q[i] : = Q[i - 1] X R[i] end quieksort p r o c e d u r e R A N G E D V D (a, b, c, d, e, f) ;
is not given here. (An example, however, appears below.) I t
end QLEG; real a, b, c, d, e, f;
is assumed t h a t A D J U S T S U M has as parameters real v and w,
c o m m e n t If the range divisor includes zero the program
* T h i s p r o c e d u r e w a s d e v e l o p e d in p a r t u n d e r t h e s p o n s o r s h i p and integer i, and is accompanied by a non-local real procedure
exists to a non-local label " z e r o d v s r " . R A N G E D V D assumes a
of t h e A i r F o r c e C a m b r i d g e R e s e a r c h C e n t e r .
Communications of the ACM (July 1961)
C O R R E C T I O N 65
ALGORITHM which gives an upper bound to the magnitude
of the error involved in the machine representation of a number.
non-local real procedure A D J U S T Q U O T which is analogous
FIND (possibly identical) to A D J U S T P R O D ;
The output A D J U S T S U M provides the left end-point of the 7
C.output A. R.intervalHOARE begin
of R A N G E S U M when A D J U S T S U M is called
Tony Hoare

・Invented quicksort to translate Russian into English.

・[ but couldn't explain his algorithm or implement it! ]
・Learned Algol 60 (and recursion).
・Implemented quicksort.
Tony Hoare

1980 Turing Award
“ There are two ways of constructing a software design: One way is
to make it so simple that there are obviously no deficiencies, and
the other way is to make it so complicated that there are no obvious
deficiencies. The first method is far more difficult. ”

“ I call it my billion-dollar mistake. It was the invention of the null

reference in 1965… This has led to innumerable errors,
vulnerabilities, and system crashes, which have probably caused
a billion dollars of pain and damage in the last forty years. ”

8
Bob Sedgewick

・Refined and popularized quicksort.

・Analyzed many versions of quicksort.

rithms, and we can learn from that experience to separate

good algorithms from bad ones. Third, if the tile fits into
Bob Sedgewick
the memory of the computer, there is one algorithm,
called Quicksort, which has been shown to perform well
Programming S. L. Graham, R. L. Rivest in a variety of situations. Not only is this algorithm
Techniques Editors simpler than many other sorting algorithms, but empir-

Implementing ical [2, ll, 13, 21] and analytic [9] studies show that
Quicksort can be expected to be up to twice as fast as its

Quicksort Programs nearest competitors. The method is simple enough to be

learned by programmersActa Informatica
9 by
who have no
Springer-Verlag
7, 327--355 (1977)
previous experi-
1977
ence with sorting, and those who do know other sorting
Robert Sedgewick methods should also find it profitable to learn about
Brown University Quicksort.
Because of its prominence, it is appropriate to study
The Analysis of Quicksort Programs*
how Quicksort might be improved. This subject has
This paper is a practical study of how to implement received considerable attention (see, for example, [1, 4,
the Quicksort sorting algorithm and its best variants on 11, 13, 14, 18, 20]), but few real improvements have been Robert Sedgewick
real computers, including how to apply various code suggested beyond those described by C.A.R. Hoare, the
optimization techniques. A detailed implementation inventor of Quicksort, in his original papers [5, 6].Received Hoare January 19, t976
combining the most effective improvements to also showed how to analyze Quicksort and predict its
running time. The analysis Summary. The been
has since Quicksort
extendedsorting
to algorithm and its best variants are presented
Quicksort is given, along with a discussion of how to and analyzed. Results are derived which make it possible to obtain exact formulas de-
implement it in assembly language. Analytic results the improvements that he suggested, and used to indicate
scribing the total expected running time of particular implementations on real com-
describing the performance of the programs are how they may best be implemented
puters of Quick,sort [9,and15,an17]. The
improvement called the median-of-three modification.
summarized. A variety of special situations are subject of the carefulDetailed analysis of the effect of has
implementation of Quicksort an implementation technique called loop unwrapping
considered from a practical standpoint to illustrate not been studied as widely as global
is presented. The paper improvements
is intended tonot only to present results of direct practical utility,
Quicksort's wide applicability as an internal sorting the algorithm, butbut thealso to illustrate
savings to be the intriguing
realized are asmathematics which arises in the complete analysis
of thisofimportant
significant. The history Quicksortalgorithm.
is quite complex,
method which requires negligible extra storage.
Key Words and Phrases: Quicksort, analysis of and [15] contains a full survey of the many variants
algorithms, code optimization, sorting which, have been proposed. 1. Introduction
CR Categories: 4.0, 4.6, 5.25, 5.31, 5.5 The purpose of this paper is to describe in detail how
Quicksort can best beInimplemented
t96t-62 C.A.R. Hoareactual
to handle presented a new algorithm called Quicksort [7, 8]
applications on real computers. A general description ofinto order by computer. This method combines
which is suitable for putting files
elegancebyand
the algorithm is followed efficiency,of and
descriptions it remains today the most useful general-purpose
the most
effective improvementssortingthat method
have for beencomputers.
proposed The (as practical utility of the algorithm has meant
demonstrated in [15]). Next, an implementation of to countless modifications (though few real
not only that it has been sfibjected 9
Quicksort partitioning demo

Repeat until i and j pointers cross.

・Scan i from left to right so long as (a[i] < a[lo]).

・Scan j from right to left so long as (a[j] > a[lo]).

・Exchange a[i] with a[j].

K R A T E L E P U I M Q C X O S

lo i j

10
Quicksort partitioning demo

Repeat until i and j pointers cross.

・Scan i from left to right so long as (a[i] < a[lo]).

・Scan j from right to left so long as (a[j] > a[lo]).

・Exchange a[i] with a[j].

When pointers cross.

・Exchange a[lo] with a[j].

E C A I E K L P U T M Q R X O S

lo j hi

partitioned!
11
The music of quicksort partitioning (by Brad Lyon)

https://googledrive.com/host/0B2GQktu-wcTicjRaRjV1NmRFN1U/index.html

12
 Quicksort: Java code for partitioning

private static int partition(Comparable[] a, int lo, int hi)

{
int i = lo, j = hi+1;
while (true)
{
while (less(a[++i], a[lo])) find item on left to swap
if (i == hi) break;

while (less(a[lo], a[--j])) find item on right to swap

if (j == lo) break;

if (i >= j) break; check if pointers cross

exch(a, i, j); swap
}

exch(a, lo, j); swap with partitioning item

before v
return j; return index of item now known to be in place
} lo hi

before v during v "v !v

lo hi i j

during after "v v !v

before v v "v !v

lo hi i j lo j hi

during v "v !v after "v v !v Quicksort partitioning overview

13
Quicksort: Java implementation

public class Quick

{
private static int partition(Comparable[] a, int lo, int hi)
{ /* see previous slide */ }

public static void sort(Comparable[] a)

{
shuffle needed for
StdRandom.shuffle(a);
performance guarantee
sort(a, 0, a.length - 1);
(stay tuned)
}

private static void sort(Comparable[] a, int lo, int hi)

{
if (hi <= lo) return;
int j = partition(a, lo, hi);
sort(a, lo, j-1);
sort(a, j+1, hi);
}
}

15
Quicksort trace

lo j hi 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
initial values Q U I C K S O R T E X A M P L E
random shuffle K R A T E L E P U I M Q C X O S
0 5 15 E C A I E K L P U T M Q R X O S
0 3 4 E C A E I K L P U T M Q R X O S
0 2 2 A C E E I K L P U T M Q R X O S
0 0 1 A C E E I K L P U T M Q R X O S
1 1 A C E E I K L P U T M Q R X O S
4 4 A C E E I K L P U T M Q R X O S
6 6 15 A C E E I K L P U T M Q R X O S
no partition 7 9 15 A C E E I K L M O P T Q R X U S
for subarrays
of size 1 7 7 8 A C E E I K L M O P T Q R X U S
8 8 A C E E I K L M O P T Q R X U S
10 13 15 A C E E I K L M O P S Q R T U X
10 12 12 A C E E I K L M O P R Q S T U X
10 11 11 A C E E I K L M O P Q R S T U X
10 10 A C E E I K L M O P Q R S T U X
14 14 15 A C E E I K L M O P Q R S T U X
15 15 A C E E I K L M O P Q R S T U X

result A C E E I K L M O P Q R S T U X

Quicksort trace (array contents after each partition)

16
Quicksort: implementation details

Partitioning in-place. Using an extra array makes partitioning easier


(and stable), but is not worth the cost.


Terminating the loop. Testing whether the pointers cross is trickier

than it might seem.


Equal keys. When duplicates are present, it is (counter-intuitively)

better to stop scans on keys equal to the partitioning item's key. stay tuned



Preserving randomness. Shuffling is needed for performance guarantee.
Equivalent alternative. Pick a random partitioning item in each subarray.

18
Quicksort: best-case analysis

Best case. Number of compares is ~ n lg n.

initial values

random shuffle

21
Quicksort: worst-case analysis

Worst case. Number of compares is ~ ½ n 2 .

initial values

random shuffle

22