Lecture –07

02 Oct 2019

Advance Algorithm Analysis

 Recap
• Lecture 6
• Binary Search
• Assignment-2
• Substitution method (backward) to know the run
time of recurrence

2
 In Today’s Lecture
• Quick Sort (last lecture w.r.t sorting)
• Recurrence
• Substitution Method
• Tree Method
• Master Method

• Reading
• Chapter-7 (7.1, 7.2) PP170-179

3
 Algorithm : Invariant Properties
• Initialization
• E.g (It is true prior to the first iteration of loop, insertion sort)
• Maintenance
• If it is true before an iteration, then it is true after next iteration
• Termination
• When the loop terminates, the invariants gives useful property that
helps to show that algorithm is correct

4
 Quick Sort (last sorting algorithm)

• Quick sort invented in 1959 by Tony Hoare

• It is
• Based on divide and conquer strategy
• A comparison sort
• Requires small memory to perform sorting (in place sorting)
• Its divide and conquer strategy is similar but differs in volume
of work
• Merge sort (low work on division, more work in combining)
• Quick sort (more work on division, little work in combining)

5
 Quick Sorts : Basics
• It selects a data element from an unsorted list
• Place that element in its correct position (with repositioning of
few elements)
• After its correct positioning, it is guaranteed that all elements
that are smaller to it are on its left side (unsorted) and larger
element on the right side

6
 Description of Quick Sort
• Divide
• Partition the array A[p..r] into two sub arrays
• A[p..q-1] and
• A[q+1…r]

Where is q?

7
 Description of Quick Sort
• Divide
• Partition the array A[p..r] into two sub arrays
• A[p..q-1] left half and
• A[q+1…r] right half
• Each element in left half is equal or less than A[q]
• Each element in right half is greater than A[q]

8
 Description of Quick Sort
• Divide
• Partition the array A[p..r] into two sub arrays
• A[p..q-1] left half and
• A[q+1…r] right half
• Each element in left half is equal or less than A[q]
• Each element in right half is greater than A[q]
• Conquer
• Sort the two subarrays A[p..q-1] left half and A[q+1…r] right half

9
 Description of Quick Sort
• Divide
• Partition the array A[p..r] into two sub arrays
• A[p..q-1] left half and
• A[q+1…r] right half
• Each element in left half is equal or less than A[q]
• Each element in right half is greater than A[q]
• Conquer
• Sort the two subarrays A[p..q-1] left half and A[q+1…r] right half
• Combine
• No work required (array already sorted)

10
Quick Sort : Outline

11
 Quick Sorts : Basics
1. Select the pivot element
2. Find the partition position
3. Repeat step 1 and 2

12
Quick Sort : Working

13
 Quick Sort
• A[r] is called the pivot
• Partitions the elements A[p…r-1] in to two sets
• those ≤ pivot and
• those > pivot
• Operates in place
• Final result:
p pivot r

≤ pivot > pivot

14
 Quick Sort : Algorithm Run

…5 7 1 2 8 4 3 6…
p r

15
 Quick Sort : Algorithm Run
i

…5 7 1 2 8 4 3 6…
p r

16
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

17
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

18
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

19
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

20
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

21
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

22
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

23
 Quick Sort : Algorithm Run
i j

…5 7 1 2 8 4 3 6…
p r

24
 Quick Sort : Algorithm Run
i j

…5 1 7 2 8 4 3 6…
p r

25
 Quick Sort : Algorithm Run
i j

…5 1 7 2 8 4 3 6…
p r

26
 Quick Sort : Algorithm Run
i j

…5 1 7 2 8 4 3 6…
p r

27
 Quick Sort : Algorithm Run
i j

…5 1 2 7 8 4 3 6…
p r

28
 Quick Sort : Algorithm Run
i j

…5 1 2 7 8 4 3 6…
p r

29
 Quick Sort : Algorithm Run
i j

…5 1 2 7 8 4 3 6…
p r

30
What is happening
BTW?

31
 Quick Sort : Algorithm Run
i j

…5 1 2 7 8 4 3 6…
p r

32
 Quick Sort : Algorithm Run
i j

…5 1 2 7 8 4 3 6…
p r

33
 Quick Sort : Algorithm Run
i j

…5 1 2 4 8 7 3 6…
p r

34
 Quick Sort : Algorithm Run
i j

…5 1 2 4 8 7 3 6…
p r

35
 Quick Sort : Algorithm Run
i j

…5 1 2 4 8 7 3 6…
p r

36
 Quick Sort : Algorithm Run
i j

…5 1 2 4 3 7 8 6…
p r

37
 Quick Sort : Algorithm Run
i j

…5 1 2 4 3 7 8 6…
p r

38
 Quick Sort : Algorithm Run
i j

…5 1 2 4 3 7 8 6…
p r

39
 Quick Sort : Algorithm Run
i j

…5 1 2 4 3 7 8 6…
p r

i j

…5 1 2 4 3 6 8 7…

p r

40
 Quick Sort : Algorithm Run
i j

…5 1 2 4 3 7 8 6…
p r

i j

…5 1 2 4 3 6 8 7…

p r

41
 Quick Sort : Running Time
• Θ (n)

42
 Quick Sort : Running Time
T(n)
• Θ (n)
O(n)
T(n/2)
T(n/2)

43
 Complete Run

8 5 1 3 6 2 7 4

44
 Complete Run

8 5 1 3 6 2 7 4

45
 Complete Run

1 3 2 4 6 8 7 5

46
 Complete Run

1 3 2 4 6 8 7 5

47
 Complete Run

1 3 2 4 6 8 7 5

48
 Complete Run

1 2 3 4 6 8 7 5

49
 Complete Run

1 2 3 4 6 8 7 5

50
 Complete Run

1 2 3 4 6 8 7 5

51
 Complete Run

1 2 3 4 6 8 7 5

52
 Complete Run

1 2 3 4 6 8 7 5

53
 Complete Run

1 2 3 4 5 8 7 6

What happens here?

Hint: Split situation?

54
 Complete Run

1 2 3 4 5 8 7 6

55
 Complete Run

1 2 3 4 5 8 7 6

56
 Complete Run

1 2 3 4 5 6 7 8

57
 Complete Run

1 2 3 4 5 6 7 8

58
 Some observations
• Divide and conquer: different than MergeSort – do the work
before recursing
• How many times is/can an element selected for as a pivot?
• What happens after an element is selected as a pivot?

1 3 2 4 6 8 7 5

59
 Is Quicksort correct?
• Assuming Partition is correct
• Proof by induction
• Base case: Quicksort works on a list of 1 element
• Inductive case:
• Assume Quicksort sorts arrays for arrays of smaller < n elements, show
that it works to sort n elements
• If partition works correctly then we have:
• and, by our inductive assumption, we have:
pivot

A sorted sorted

≤ pivot > pivot

60
Quick Sort : Invariants

61
Quick Sort : Invariants

62
Quick Sort : Invariants

63
 Running time of Quicksort- Worst Case
• Worst Case?
• Each call to Partition splits the array into
• an empty array and n-1 array

64
 Running Time : Worst Case
• Each partition splits the array into an empty array and n-1
array

n-1 {5,3,7,2,11} 0

T(n)=T(n-1) + T(0)+Θ(n)
T(n)=T(n-1) +Θ(n)
T(n)=Θ(n2)
65
 Running time of Quicksort?

Which is? Θ(n2)

• When does this happen?

• sorted
• reverse sorted
• near sorted/reverse sorted

66
 Running time of Quicksort- Best Case

• Each call to Partition splits the array into two

equal parts

• O(n log n)
• When does this happen?
• random data

67
 Average Case
• How close to “even” splits do they need to be to maintain an
O(n log n) running time?
• Say the Partition procedure always splits the array into some
constant ratio b-to-a, e.g. 9-to-1
• What is the recurrence?

68
 Average Case : Balanced Partition
• The average case is close to best case

69
Average Case : Balanced Partition

70
 Quick Vs Merge Vs Insertion
• Question: If QuickSort has worst case running time of
O(n2) and MergeSort has worst case (and best case!)
running time of Θ(n*log(n)), why don’t we just use
MergeSort?

71
 Quick Vs Merge Vs Insertion
• Question: If QuickSort has worst case running time of
O(n2) and MergeSort has worst case (and best case!)
running time of Θ(n*log(n)), why don’t we just use
MergeSort?
• MergeSort has a lot of copying overhead specially in the
Combine phase.
• QuickSort sorts the list in place. Some swapping is required,
but usually not all members have to be swapped.
• What about insertion sort with worst case time of O(n2) ?

72
RECURRENCE

73
 Three Approaches for Recurrence
• Substitution method: when we have a good guess of the
solution, we prove that it’s correct
• Recursion-tree method: If we don’t have a good guess of
the solution, the recursion tree can help. Then solve with
substitution method.
• Master method: Provides solutions for recurrences of the
form:

74