Design and Analysis of Algorithms

CSE 5311
Lecture 7 Quick Sort

Junzhou Huang, Ph.D.

Department of Computer Science and Engineering

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 1

 Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 2

 Divide and Conquer

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 3

 Partitioning Subroutine

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 4

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 5

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 6

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 7

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 8

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 9

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 10

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 11

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 12

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 13

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 14

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 15

 Example of Partitioning

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 16

 Running Time for PARTITION

The running time of PARTITION on the subarray A[p...r]

is Θ(n) where n=r-p+1

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 17

 Pseudo code for Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 18

 Analysis of Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 19

 Worst-case of Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 20

 Worst-case Decision Tree

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 21

 Worst-case Decision Tree

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 22

 Worst-case Analysis

For the worst case, we can write the recurrence equation as

T ( n ) = max {T ( q ) + T ( n − q − 1)} + Θ( n )
0≤ q ≤ n −1

We guess that T ( n ) ≤ cn 2

T ( n ) ≤ max {cq 2 + c( n − q − 1) 2 } + Θ( n )
0≤ q ≤ n −1

= c ⋅ max {q 2 + ( n − q − 1) 2 } + Θ( n )
0≤ q ≤ n −1

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 23

 Worst-case Analysis

This expression achieves a maximum at either end points:

q=0 or q=n-1.

Using the maximum of T(n) we have

T ( n ) ≤ max {cq 2 + c( n − q − 1) 2 } + Θ( n )
0≤ q ≤ n −1

T ( n ) ≤ cn 2 − c( 2n − 1) + Θ( n )
≤ cn 2
Thus
T ( n ) = O ( n 2 ).

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 24

 Worst-case Analysis

We can also show that the recurrence equation as

T ( n ) = max {T ( q ) + T ( n − q − 1)} + Θ( n )
0≤ q ≤ n −1

Has a solution of T ( n ) = Ω( n 2 )

We guess that T ( n ) ≥ cn 2

T ( n ) ≥ max {cq 2 + c( n − q − 1) 2 } + Θ( n )
0≤ q ≤ n −1

= c ⋅ max {q 2 + ( n − q − 1) 2 } + Θ( n )
0≤ q ≤ n −1

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 25

 Worst-case Analysis
Using the maximum of T(n) we have

T ( n ) ≥ cn 2 − c( 2n − 1) + Θ( n )
= cn 2 − c ( 2n − 1) + c1n
= cn 2 + ( c1 − 2c )n + c
≥ cn 2 + ( c1 − 2c )n

We can pick the constant c1 large enough so that c1 − 2c ≥ 0

and T ( n ) ≥ cn 2 = Ω( n 2 )

Thus the worst case running time of quicksort is Θ( n 2 )

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 26

 Best-case Analysis

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 27

 Analysis of Almost Best-case

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 28

 Analysis of Almost Best-case

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 29

 Analysis of Almost Best-case

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 30

 Analysis of Almost Best-case

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 31

 Analysis of Almost Best-case

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 32

 Best-case Analysis

For the best case, we can write the recurrence equation as

T ( n ) = min {T ( q ) + T ( n − q − 1)} + Θ( n )
0≤ q ≤ n −1

We guess that
T ( n ) ≥ cn log n = Ω( n log n )

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 33

 Best-case Analysis

T ( n ) ≥ min { cq log q + c ( n − q − 1 ) log( n − q − 1 )} + Θ ( n )

0 ≤ q ≤ n −1

= c ⋅ min { q log q + ( n − q − 1 ) log( n − q − 1 )} + Θ ( n )

0 ≤ q ≤ n −1

Let Q = q log q + ( n − q − 1 ) log( n − q − 1 )

q ( n − q − 1)
dQ / dq = c { + log q − log( n − q − 1 ) − }= 0
q ( n − q − 1)
n −1
⇒ q =
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 34

 Best-case Analysis
This expression achieves a minimum at
n −1
q =
2
Using the minimum of T(n) we have

n −1 n −1 n −1 n −1
T ( n ) ≥ c{ log + log } + Θ( n )
2 2 2 2
= cn log(n − 1) + c( n − 1) + Θ( n )
= cn log(n − 1) + Θ( n )
≥ cn log n
= Ω( n log n )

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 35

 More Intuition

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 36

 Randomized Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 37

 Randomized Quicksort
Standard Problematic Algorithm :

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 38

 Randomized Quicksort

RANDOMIZED-PARTITION (A, p, r)
1 i←RANDOM(p, r)
2 exchange A[r]↔A[i]
3 return PARTITION(A, p, r)

RANDOMIZED-QUICKSORT (A,p,r)
1 if p<r
2 then q←RANDOMIZED-PARTITION (A, p, r)
RANDOMIZED-QUICKSORT (A, p, q-1)
RANDOMIZED-QUICKSORT (A, q+1, r)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 39

 Randomized Quicksort

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 40

 Randomized Quicksort Analysis

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 41

 Calculating Expectation

Take expectation
in both side

Independence of Xk
from other random
choice

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 42

 Calculating Expectation

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 43

 Calculating Expectation

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 44

 Calculating Expectation
n −1 n / 2 −1 n −1
∑ k lg k = ∑ k lg k + ∑ k lg k
k =1 k =1 k = n / 2 

The lg k in the first summation is bounded above by lg (n/ 2) = lg n − 1

The lg k in the second summation is bounded above by lg n
n / 2 −1 n −1
≤ (lg n − 1) ∑ k + lg n ∑ k
k =1 k = n / 2 
n −1 n / 2 −1
= lg n ∑ k − ∑k
k =1 k =1
1 1 n n
≤ n(n − 1) lg n − ( − 1)
2 2 2 2
1 1
≤ n 2 lg n − n 2
2 8
For n ≥ 2 this is the bound.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 45
 Substitution Method

Desired - Residual

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 46

 Quicksort: Summary

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 47