Searching and Sorting

Unit 8: Searching and Sorting
UNIT 8
SEARCHING, SORTING TECHNIQUES (8 HOURS)
SYLLABUS CONTENT
 Searching:
 Linear Search
 Binary Search
 Sorting:
 Bubble Sort
 Selection Sort
 Heap Sort
 Insertion Sort
 Merge Sort
 Quick Sort
 Radix Sort
 Analysis and comparison of Searching and Sorting Techniques
DR. NEEPA SHAH 2
Dr. Neepa Shah 1

LINEAR SEARCH
 Unordered List
 One occurrence
 All occurrences
DR. NEEPA SHAH 3
BINARY SEARCH
 Given an integer X and integers A0 , A1 , …, AN-1, which are

presorted, find i such that Ai = X, or return i = -1 if X is not in
the input.
DR. NEEPA SHAH 4
Dr. Neepa Shah 2

BINARY SEARCH: EXAMPLE
DR. NEEPA SHAH 5
BINARY SEARCH: EXAMPLE CONTD.
DR. NEEPA SHAH 6
Dr. Neepa Shah 3

BINARY SEARCH
int binarySearch(int []a, int x, int N)
{
int low = 0, high = N-1;
while (low <= high)
{
int mid = (low + high) / 2; // middle subscript
if (a[mid] < x)
low = mid + 1;
else if (x < a[mid])
high = mid – 1;
else
return mid; // found
}
return –1; // not found
DR. NEEPA SHAH } 7
WHAT IS SORTING?
 Examples:
 Sorting Books in Library
 Sorting Individuals by Height (Feet and Inches)
 Sorting Movies in Blockbuster (Alphabetical)
 Sorting Numbers (Sequential)
DR. NEEPA SHAH 8
Dr. Neepa Shah 4

CLASSIFICATION
 Sorting algorithms are often classified by:

 Computational Complexity
 Computational Complexity of Swaps
 Memory usage
 Recursion
 Stability
DR. NEEPA SHAH 9
SORTING ALGORITHMS
 Bubble Sort
 Selection Sort
 Insertion Sort
 Shell Sort
 Heap Sort
 Quick Sort
 Bucket Sort
 Merge Sort
 Radix Sort
DR. NEEPA SHAH 10
Dr. Neepa Shah 5

BUBBLE SORT
 One of the simplest sorting algorithm
 Also known as Exchange sort
DR. NEEPA SHAH 11
9, 6, 2, 12, 11, 9, 3, 7
6, 9, 2, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 11, 12, 9, 3, 7
6, 2,
BUBBLE SORT EXAMPLE
9, 11, 9, 12, 3, 7
6, 2,
DR. NEEPA SHAH
9, 11, 9, 3, 12, 7 12
6, 2, 9, 11, 9, 3, 7, 12
Dr. Neepa Shah 6

BUBBLE SORT EXAMPLE CONTD.

First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
6, 6,
2, 2, 9, 9,
11,3,
11,
9,7,
11,
3,11,
7, 12
DR. NEEPA SHAH 13

First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
Third Pass
2, 6, 9, 9, 3, 7, 11, 12
2, 6, 9, 3,
9, 7,
3, 9,
9, 7, 11, 12
DR. NEEPA SHAH 14
Dr. Neepa Shah 7


First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
Third Pass
2, 6, 9, 9, 3, 7, 11, 12
Fourth Pass
2, 6, 9, 3, 7, 9, 11, 12
2, 6, 9,
3, 3, 9,
9,
7, 7, 9, 11, 12
DR. NEEPA SHAH 15

First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
Third Pass
2, 6, 9, 9, 3, 7, 11, 12
Fourth Pass
2, 6, 9, 3, 7, 9, 11, 12
Fifth Pass
2, 6, 3, 7, 9, 9, 11, 12
2, 6,
3, 3,
6, 7, 9, 9, 11, 12
DR. NEEPA SHAH 16
Dr. Neepa Shah 8


First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
Third Pass
2, 6, 9, 9, 3, 7, 11, 12
Fourth Pass
2, 6, 9, 3, 7, 9, 11, 12
Fifth Pass
2, 6, 3, 7, 9, 9, 11, 12
Sixth Pass
2, 3, 6, 7, 9, 9, 11, 12
DR. NEEPA SHAH
2, 3, 6, 7, 9, 9, 11, 12 17
BUBBLE SORT PROCEDURE
1. Do steps 2-3 for i= n-1 down to 1

2. Do step 3 for j=0 upto i-1
3. If the 2 consecutive elements sj and sj+1, are out of order swap them
DR. NEEPA SHAH 18
Dr. Neepa Shah 9

BUBBLE SORT ALGORITHM
void sort(int a[]) {

for(i = SIZE-1; i > 0; i--) {
for(j = 0; j < i; j++) {
if(a[j] > a[j+1]) {
swap(a, j, j+1);
}
}
}
DR. NEEPA SHAH 19
BUBBLE SORT
Analysis: Worst Case Complexity O(n2)
Pros: Simplicity and ease of implementation
Cons: Horribly Inefficient
DR. NEEPA SHAH 20
Dr. Neepa Shah 10

ALREADY SORTED COLLECTIONS?
 What if the collection was already sorted?

 What if only a few elements were out of place and after a couple of “bubble ups,” the collection was sorted?
 We want to be able to detect this and “stop early”!
1 2 3 4 5 6
5 12 35 42 77 101
DR. NEEPA SHAH 21
USING A BOOLEAN “FLAG”
 We can use a Boolean variable to determine if any swapping occurred during the “bubble up.”
 If no swapping occurred, then we know that the collection is already sorted!
 This Boolean “flag” needs to be reset after each “bubble up.”
DR. NEEPA SHAH 22
Dr. Neepa Shah 11

AN EXAMPLE
N 8 did_swap true
to_do 7
index
98 23 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 23
AN EXAMPLE
N 8 did_swap false
to_do 7
index 1
98 23 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 24
Dr. Neepa Shah 12

AN EXAMPLE
N 8 did_swap false
to_do 7
index 1
Swap
98 23 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 25
AN EXAMPLE
N 8 did_swap true
to_do 7
index 1
Swap
23 98 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 26
Dr. Neepa Shah 13

AN EXAMPLE
N 8 did_swap true
to_do 7
index 2
23 98 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 27
AN EXAMPLE
N 8 did_swap true
to_do 7
index 2
Swap
23 98 45 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 28
Dr. Neepa Shah 14

AN EXAMPLE
N 8 did_swap true
to_do 7
index 2
Swap
23 45 98 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 29
AN EXAMPLE
N 8 did_swap true
to_do 7
index 3
23 45 98 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 30
Dr. Neepa Shah 15

AN EXAMPLE
N 8 did_swap true
to_do 7
index 3
Swap
23 45 98 14 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 31
AN EXAMPLE
N 8 did_swap true
to_do 7
index 3
Swap
23 45 14 98 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 32
Dr. Neepa Shah 16

AN EXAMPLE
N 8 did_swap true
to_do 7
index 4
23 45 14 98 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 33
AN EXAMPLE
N 8 did_swap true
to_do 7
index 4
Swap
23 45 14 98 6 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 34
Dr. Neepa Shah 17

AN EXAMPLE
N 8 did_swap true
to_do 7
index 4
Swap
23 45 14 6 98 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 35
AN EXAMPLE
N 8 did_swap true
to_do 7
index 5
23 45 14 6 98 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 36
Dr. Neepa Shah 18

AN EXAMPLE
N 8 did_swap true
to_do 7
index 5
Swap
23 45 14 6 98 67 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 37
AN EXAMPLE
N 8 did_swap true
to_do 7
index 5
Swap
23 45 14 6 67 98 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 38
Dr. Neepa Shah 19

AN EXAMPLE
N 8 did_swap true
to_do 7
index 6
23 45 14 6 67 98 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 39
AN EXAMPLE
N 8 did_swap true
to_do 7
index 6
Swap
23 45 14 6 67 98 33 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 40
Dr. Neepa Shah 20

AN EXAMPLE
N 8 did_swap true
to_do 7
index 6
Swap
23 45 14 6 67 33 98 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 41
AN EXAMPLE
N 8 did_swap true
to_do 7
index 7
23 45 14 6 67 33 98 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 42
Dr. Neepa Shah 21

AN EXAMPLE
N 8 did_swap true
to_do 7
index 7
Swap
23 45 14 6 67 33 98 42
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 43
AN EXAMPLE
N 8 did_swap true
to_do 7
index 7
Swap
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 44
Dr. Neepa Shah 22

AFTER FIRST PASS OF OUTER LOOP

N 8 did_swap true
to_do 7
index 8 Finished first “Bubble Up”
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 45
THE SECOND “BUBBLE UP”

N 8 did_swap false
to_do 6
index 1
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 46
Dr. Neepa Shah 23


N 8 did_swap false
to_do 6
index 1
No Swap
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 47

N 8 did_swap false
to_do 6
index 2
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 48
Dr. Neepa Shah 24


N 8 did_swap false
to_do 6
index 2
Swap
23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 49

N 8 did_swap true
to_do 6
index 2
Swap
23 14 45 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 50
Dr. Neepa Shah 25


N 8 did_swap true
to_do 6
index 3
23 14 45 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 51

N 8 did_swap true
to_do 6
index 3
Swap
23 14 45 6 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 52
Dr. Neepa Shah 26


N 8 did_swap true
to_do 6
index 3
Swap
23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 53

N 8 did_swap true
to_do 6
index 4
23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 54
Dr. Neepa Shah 27


N 8 did_swap true
to_do 6
index 4
No Swap
23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 55

N 8 did_swap true
to_do 6
index 5
23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 56
Dr. Neepa Shah 28


N 8 did_swap true
to_do 6
index 5
Swap
23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 57

N 8 did_swap true
to_do 6
index 5
Swap
23 14 6 45 33 67 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 58
Dr. Neepa Shah 29


N 8 did_swap true
to_do 6
index 6
23 14 6 45 33 67 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 59

N 8 did_swap true
to_do 6
index 6
Swap
23 14 6 45 33 67 42 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 60
Dr. Neepa Shah 30


N 8 did_swap true
to_do 6
index 6
Swap
23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 61
AFTER SECOND PASS OF OUTER LOOP

N 8 did_swap true
to_do 6
index 7 Finished second “Bubble Up”
23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 62
Dr. Neepa Shah 31

THE THIRD “BUBBLE UP”

N 8 did_swap false
to_do 5
index 1
23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 63

N 8 did_swap false
to_do 5
index 1
Swap
23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 64
Dr. Neepa Shah 32


N 8 did_swap true
to_do 5
index 1
Swap
14 23 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 65

N 8 did_swap true
to_do 5
index 2
14 23 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 66
Dr. Neepa Shah 33


N 8 did_swap true
to_do 5
index 2
Swap
14 23 6 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 67

N 8 did_swap true
to_do 5
index 2
Swap
14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 68
Dr. Neepa Shah 34


N 8 did_swap true
to_do 5
index 3
14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 69

N 8 did_swap true
to_do 5
index 3
No Swap
14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 70
Dr. Neepa Shah 35


N 8 did_swap true
to_do 5
index 4
14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 71

N 8 did_swap true
to_do 5
index 4
Swap
14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 72
Dr. Neepa Shah 36


N 8 did_swap true
to_do 5
index 4
Swap
14 6 23 33 45 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 73

N 8 did_swap true
to_do 5
index 5
14 6 23 33 45 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 74
Dr. Neepa Shah 37


N 8 did_swap true
to_do 5
index 5
Swap
14 6 23 33 45 42 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 75

N 8 did_swap true
to_do 5
index 5
Swap
14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 76
Dr. Neepa Shah 38

AFTER THIRD PASS OF OUTER LOOP

N 8 did_swap true
to_do 5
index 6 Finished third “Bubble Up”
14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 77
THE FOURTH “BUBBLE UP”

N 8 did_swap false
to_do 4
index 1
14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 78
Dr. Neepa Shah 39


N 8 did_swap false
to_do 4
index 1
Swap
14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 79

N 8 did_swap true
to_do 4
index 1
Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 80
Dr. Neepa Shah 40


N 8 did_swap true
to_do 4
index 2
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 81

N 8 did_swap true
to_do 4
index 2
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 82
Dr. Neepa Shah 41


N 8 did_swap true
to_do 4
index 3
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 83

N 8 did_swap true
to_do 4
index 3
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 84
Dr. Neepa Shah 42


N 8 did_swap true
to_do 4
index 4
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 85

N 8 did_swap true
to_do 4
index 4
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 86
Dr. Neepa Shah 43

AFTER FOURTH PASS OF OUTER LOOP

N 8 did_swap true
to_do 4
index 5 Finished fourth “Bubble Up”
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 87
THE FIFTH “BUBBLE UP”

N 8 did_swap false
to_do 3
index 1
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 88
Dr. Neepa Shah 44


N 8 did_swap false
to_do 3
index 1
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 89

N 8 did_swap false
to_do 3
index 2
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 90
Dr. Neepa Shah 45


N 8 did_swap false
to_do 3
index 2
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 91

N 8 did_swap false
to_do 3
index 3
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 92
Dr. Neepa Shah 46


N 8 did_swap false
to_do 3
index 3
No Swap
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 93
AFTER FIFTH PASS OF OUTER LOOP

N 8 did_swap false
to_do 3
index 4 Finished fifth “Bubble Up”
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 94
Dr. Neepa Shah 47

FINISHED “EARLY”
N 8 did_swap false
to_do 3
We didn’t do any swapping,
index 4 so all of the other elements
must be correctly placed.
We can “skip” the last two

passes of the outer loop.
6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8
DR. NEEPA SHAH 95
SUMMARY
 “Bubble Up” algorithm will move largest value to its correct

location (to the right)
 Repeat “Bubble Up” until all elements are correctly placed:
 Maximum of N-1 times
 Can finish early if no swapping occurs
 We reduce the number of elements we compare each time one
is correctly placed
DR. NEEPA SHAH 96
Dr. Neepa Shah 48

SELECTION SORT
 The idea of selection sort is rather simple.
 We repeatedly find the next largest (or smallest) element in the array and move it to
its final position in the sorted array.
 We begin by selecting the largest element and moving it to the highest index
position. We can do this by swapping the element at the highest index and the largest
element.The process stops when the effective size of the array becomes 1
DR. NEEPA SHAH 97
SELECTION SORT - EXAMPLE
32 91 12 55 74 73 18
DR. NEEPA SHAH 98
Dr. Neepa Shah 49

32 91 12 55 74 73 18
DR. NEEPA SHAH 99

SWAP !
32 91 12 55 74 73 18
DR. NEEPA SHAH 100
Dr. Neepa Shah 50

12 91 32 55 74 73 18
DR. NEEPA SHAH 101
12 91 32 55 74 73 18
DR. NEEPA SHAH 102
Dr. Neepa Shah 51


SWAP !
12 91 32 55 74 73 18
DR. NEEPA SHAH 103
12 18 32 55 74 73 91
DR. NEEPA SHAH 104
Dr. Neepa Shah 52


NO SWAPPING !
12 18 32 55 74 73 91
DR. NEEPA SHAH 105
12 18 32 55 74 73 91
DR. NEEPA SHAH 106
Dr. Neepa Shah 53

SELECTION SORT - EXAMPLE SWAP !
12 18 32 55 74 73 91
DR. NEEPA SHAH 107
12 18 32 55 73 74 91
DR. NEEPA SHAH 108
Dr. Neepa Shah 54

12 18 32 55 73 74 91
DR. NEEPA SHAH 109
NO SWAPPING !
12 18 32 55 73 74 91
DR. NEEPA SHAH 110
Dr. Neepa Shah 55

12 18 32 55 73 74 91
DR. NEEPA SHAH 111
SELECTION SORT PROCEDURE
1. Do steps 2-3 for i= n-1 down to 1

2. Locate the index m of the largest / smallest element among
{s0…sj}
3. Swap si and sm
DR. NEEPA SHAH 112
Dr. Neepa Shah 56

SELECTION SORT ALGORITHM

void sort(int a[])
{
for(i = SIZE-1; i > 0 ; i--)
{
m = 0;
for(j = 1; j <= i; j++)
{
if(a[j] > a[m])
m = j;
}
swap(a, i, m);
DR. NEEPA SHAH
} 113
COMPARISON
 Bubble Sort
 Selection Sort
DR. NEEPA SHAH 114
Dr. Neepa Shah 57

INSERTION SORT
 Insertion sort keeps making the left side of the array sorted until the
whole array is sorted. It sorts the values seen so far and repeatedly
inserts unseen values in the array into the left sorted array.
 The insertion sort algorithm is the sort unknowingly used by most card
players
DR. NEEPA SHAH 115
INSERTION SORT
 while some elements unsorted:
 Using linear search, find the location in the sorted portion where
the 1st element of the unsorted portion should be inserted
 Move all the elements after the insertion location up one
position to make space for the new element
69
38 45
60 60
66 45
66 79 47 13 74 36 21 94 22 57 16 29 81
the fourth iteration of this loop is shown here
DR. NEEPA SHAH 116
Dr. Neepa Shah 58

An insertion sort partitions the array into two regions
DR. NEEPA SHAH 117
EXAMPLE OF INSERTION SORT

8 2 4 9 3 6
DR. NEEPA SHAH 118
Dr. Neepa Shah 59


8 2 4 9 3 6
DR. NEEPA SHAH 119

8 2 4 9 3 6
2 8 4 9 3 6
DR. NEEPA SHAH 120
Dr. Neepa Shah 60


8 2 4 9 3 6
2 8 4 9 3 6
DR. NEEPA SHAH 121

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
DR. NEEPA SHAH 122
Dr. Neepa Shah 61


8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
DR. NEEPA SHAH 123

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
DR. NEEPA SHAH 124
Dr. Neepa Shah 62


8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
DR. NEEPA SHAH 125

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
DR. NEEPA SHAH 126
Dr. Neepa Shah 63


8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
DR. NEEPA SHAH 127

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
DR. NEEPA SHAH 128
Dr. Neepa Shah 64

INSERTION SORT EXAMPLE
 5 7 0 3 4 2 6 1 (0)
 5 7 0 3 4 2 6 1 (0)
 0 5 7 3 4 2 6 1 (2)
 0 3 5 7 4 2 6 1 (2)
 0 3 4 5 7 2 6 1 (2)
 0 2 3 4 5 7 6 1 (4)
 0 2 3 4 5 6 7 1 (1)
 0 1 2 3 4 5 6 7 (6)
DR. NEEPA SHAH 129
An insertion sort of an array of five integers
DR. NEEPA SHAH 130
Dr. Neepa Shah 65

INSERTION SORT PROCEDURE
1. Do steps 2-4 for i= 1 upto n-1

2. Hold the element s
3. Locate the least index j for which sj >= si
4. Shift the subsequence {sj … si-1} up one position to {sj+1 … si}
5. Copy the held value of si into sj
DR. NEEPA SHAH 131
INSERTION SORT ALGORITHM

void insertionSort(int[] a) {
for (int i = 1; i < SIZE; i++) {
int ai = a[i], j;
// Shuffle up all sorted items > a[i]

for (j = i; j > 0 && a[j-1] > ai; j--) {
a[j] = a[j–1];
}
// Insert the current item
a[j] = ai;
}
}
DR. NEEPA SHAH 132
Dr. Neepa Shah 66

REVIEW: ONE STEP OF INSERTION SORT

sorted next to be inserted
3 4 7 12 14 14 20 21 33 38 10 55 9 23 28 16
temp
less than 10
10
3 4 7 10 12 14 14 20 21 33 38 55 9 23 28 16
sorted
DR. NEEPA SHAH 133
INSERTION SORT: NUMBER OF COMPARISONS

# of Sorted Best case Worst case
Elements
0 0 0
1 1 1
2 1 2
… … …
n-1 1 n-1
n-1 n(n-1)/2
DR. NEEPA SHAH 134
If the sequence is nearly sorted, then insertion sort
will run nearly in O(n) time.
Dr. Neepa Shah 67

DIVIDE AND CONQUER STRATEGY
DIVIDE-AND-CONQUER
 Divide and Conquer is a method of algorithm design.
 This method has three distinct steps:

 Divide: If the input size is too large to deal with in a straightforward manner,
divide the data into two or more disjoint subsets.
 Recur: Use divide and conquer to solve the sub-problems associated with the
data subsets.
 Conquer: Take the solutions to the sub-problems and “merge” these solutions
into a solution for the original problem.
DR. NEEPA SHAH 136
Dr. Neepa Shah 68

MERGE-SORT
 Algorithm:
 Divide
 Recur: Recursive sort sequences S1 and S2.
 Conquer: Put back the elements into S by merging the sorted
sequences S1 and S2 into a unique sorted sequence.
DR. NEEPA SHAH 137
MERGE-SORT EXAMPLE
DR. NEEPA SHAH 138
Dr. Neepa Shah 69

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
DR. NEEPA SHAH 139
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
DR. NEEPA SHAH 140
Dr. Neepa Shah 70

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23
DR. NEEPA SHAH 141
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23
Merge
DR. NEEPA SHAH 142
Dr. Neepa Shah 71

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23
23
Merge
DR. NEEPA SHAH 143
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23
23 98
Merge
DR. NEEPA SHAH 144
Dr. Neepa Shah 72

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98
DR. NEEPA SHAH 145
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98
Merge
DR. NEEPA SHAH 146
Dr. Neepa Shah 73

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14
Merge
DR. NEEPA SHAH 147
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
Merge
DR. NEEPA SHAH 148
Dr. Neepa Shah 74

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
Merge
DR. NEEPA SHAH 149
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
14
Merge
DR. NEEPA SHAH 150
Dr. Neepa Shah 75

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
14 23
Merge
DR. NEEPA SHAH 151
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
14 23 45
Merge
DR. NEEPA SHAH 152
Dr. Neepa Shah 76

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
98 23 45 14
23 98 14 45
14 23 45 98
Merge
DR. NEEPA SHAH 153
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14
23 98 14 45
14 23 45 98
DR. NEEPA SHAH 154
Dr. Neepa Shah 77

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67
23 98 14 45
14 23 45 98
DR. NEEPA SHAH 155
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67
23 98 14 45
14 23 45 98 Merge
DR. NEEPA SHAH 156
Dr. Neepa Shah 78

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67
23 98 14 45 6
14 23 45 98 Merge
DR. NEEPA SHAH 157
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67
23 98 14 45 6 67
14 23 45 98 Merge
DR. NEEPA SHAH 158
Dr. Neepa Shah 79

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67
14 23 45 98
DR. NEEPA SHAH 159
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67
14 23 45 98 Merge
DR. NEEPA SHAH 160
Dr. Neepa Shah 80

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33
14 23 45 98 Merge
DR. NEEPA SHAH 161
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 Merge
DR. NEEPA SHAH 162
Dr. Neepa Shah 81

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98
Merge
DR. NEEPA SHAH 163
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6
Merge
DR. NEEPA SHAH 164
Dr. Neepa Shah 82

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33
Merge
DR. NEEPA SHAH 165
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42
Merge
DR. NEEPA SHAH 166
Dr. Neepa Shah 83

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
Merge
DR. NEEPA SHAH 167
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
DR. NEEPA SHAH Merge 168
Dr. Neepa Shah 84

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14
Dr. Neepa Shah 85

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33
Dr. Neepa Shah 86

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33 42 45
Dr. Neepa Shah 87

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
Dr. Neepa Shah 88

98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
98 23 45 14 6 67 33 42
23 98 14 45 6 67 33 42
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
DR. NEEPA SHAH 177
98 23 45 14 6 67 33 42
6 14 23 33 42 45 67 98
DR. NEEPA SHAH 178
Dr. Neepa Shah 89

MERGE SORT ALGORITHM
Overview:
 Split array into two halves
 Sort the left half (recursively)
 Sort the right half (recursively)
 Merge the two sorted halves

DR. NEEPA SHAH 179
MERGE SORT PROCEDURE
1. If q-p > 1 do steps 2-5

2. Split s into 2 subsequences, a = {sp … sm-1} and b = {sm … sq-1}, where m = (q-p)/2.
3. Sort a
4. Sort b
5. Merge a and b back into s, preserving order
DR. NEEPA SHAH 180
Dr. Neepa Shah 90

MERGE SORT ALGORITHM
void sort(int a[],int p, int q){
if (q-p < 2)
{
return;
}
int m = (p + q) / 2;
sort(a, p, m);
sort(a, m+1, q);
merge (a, p, m , q);
}
DR. NEEPA SHAH 181
RUNNING TIME OF MERGE-SORT
 At each level in the binary tree created for Merge Sort, there are n elements, with O(1) time spent at each
element
 → O(n) running time for processing one level
 The height of the tree is O(log n)
 Therefore, the time complexity is O(nlog n)
DR. NEEPA SHAH 182
Dr. Neepa Shah 91

MERGE SORT - ANALYSIS
Algorithmic Complexity : O (n log n)
Pros
 Marginally faster than heap sort for larger data sets
Cons
 Atleast twice the memory requirements to other sorts
DR. NEEPA SHAH 184
QUICK-SORT
1) Divide : If the sequence S has 2 or more elements, select an element x from S to be your pivot. Any arbitrary
element, like the last, will do. Remove all the elements of S and divide them into 3 sequences:
L, holds S’s elements less than x
E, holds S’s elements equal to x
G, holds S’s elements greater than x
2) Recurse: Recursively sort L and G
3) Conquer: Finally, to put elements back into S in order, first inserts the elements of L, then those of E, and those of
G.
DR. NEEPA SHAH 185
Dr. Neepa Shah 92

IDEA OF QUICK SORT
1) Select: pick an element
2) Divide: rearrange elements so that x goes to its final

position E
3) Recurse and Conquer: recursively sort
DR. NEEPA SHAH 186
QUICK-SORT TREE
DR. NEEPA SHAH 187
Dr. Neepa Shah 93

IN-PLACE QUICK-SORT
 Divide step: l scans the sequence

from the left, and r from the right.
 A swap is performed when l is at
an element larger than the pivot
and r is at one smaller than the
pivot.
DR. NEEPA SHAH 188
IN PLACE QUICK SORT (CONT’D)
A final swap with the pivot completes the divide step
DR. NEEPA SHAH

189
Dr. Neepa Shah 94

QUICK SORT PROCEDURE / ALGORITHM
1. If q-p > 1 do steps 2-5

2. Apply partition algorithm and
3. obtain pivot index m
4. Apply quick sort to {s0 … sm-1}
5. Apply quick sort to {sm+1 … sn-1}
DR. NEEPA SHAH 190
QUICK SORT PROCEDURE / ALGORITHM
void sort(int a[],int p, int q){

if (q-p < 2)
{
return;
}
int m = partition(a, p, q) ;
sort(a, p, m);
sort(a, m+1, q);
}
DR. NEEPA SHAH 191
Dr. Neepa Shah 95

PARTITION ALGORITHM
int partition(int a[], int p, int q)
{
int pivot = a[p], i = p, j = q;

while (i < j)
{
while (i < j && a[--j] >= pivot) ;
if(i < j)
a[i] = a[j];
while (i < j && a[++i] <= pivot) ;

if(i < j)
a[j] = a[i];
}
a[j] = pivot;
DR. NEEPA SHAH 192
return j;
}
QUICK SORT RUNNING TIME
 Worst case: when the pivot does not divide the sequence in two
 At each step, the length of the sequence is only reduced by 1
 Total running time
1
 length( S ) = O(n
i =n
i
2
)
 General case:
 Time spent at level i in the tree is O(n)
 Running time: O(n) * O(height)
 O(n log n)
DR. NEEPA SHAH 193
Dr. Neepa Shah 96

HEAPSORT
WHY STUDY HEAPSORT?
 Heapsort is always O(n log n)

 Quicksort is usually O(n log n) but in the worst case slows to O(n2)
DR. NEEPA SHAH 195
Dr. Neepa Shah 97

WHAT IS A “HEAP”?
A balanced, left-justified binary tree in which no node has a value

greater than the value in its parent
DR. NEEPA SHAH 196
BALANCED BINARY TREES
 The depth of a node is its distance from the root

 The depth of a tree is the depth of the deepest node
 A binary tree of depth n is balanced if all the nodes at depths 0 through n-2 have two children
n-2
n-1
n
Balanced Balanced Not balanced
DR. NEEPA SHAH 197
Dr. Neepa Shah 98

LEFT-JUSTIFIED BINARY TREES
 A balanced binary tree of depth n is left-justified if:

 it has 2n nodes at depth n (the tree is “full”)
or
 it has 2k nodes at depth k, for all k < n, and all the leaves at depth n are as far left as possible
Left-justified Not left-justified

DR. NEEPA SHAH 198
APPROACH: HEAP SORT
 Top down
 Bottom up
 First, we will learn how to turn a binary tree into a heap

 Next, we will learn how to turn a binary tree back into a heap after it has been changed in a certain way
 Finally, we will see how to use these ideas to sort an array
DR. NEEPA SHAH 199
Dr. Neepa Shah 99

THE HEAP PROPERTY
12 12 12
8 3 8 12 8 14
Blue node has heap Blue node has heap Blue node does not have
property property heap property
 A node has the heap property if the value in the node is larger than the values in its children
 All leaf nodes automatically have the heap property
 A binary tree is a heap if all nodes in it have the heap property
DR. NEEPA SHAH 200
SIFTUP
12 14
8 14 8 12
Blue node does not have Blue node has heap
heap property property
 Given a node that does not have the heap property, you can give it the heap property
by exchanging its value with the value of the larger child
 This is sometimes called sifting up
DR. NEEPA SHAH 201
Dr. Neepa Shah 100

CONSTRUCTING A HEAP I
 A tree consisting of a single node is automatically a heap

 We construct a heap by adding nodes one at a time:
 Add the node just to the right of the rightmost node in the deepest level
 If the deepest level is full, start a new level
 Examples:
Add a new Add a new
node here node here
DR. NEEPA SHAH 202
CONSTRUCTING A HEAP II
 Each time we add a node, we may destroy the heap property of its parent node
 To fix this, we sift up
 But each time we sift up, the value of the topmost node in the sift may increase, and this may destroy the heap
property of its parent node
 We repeat the sifting up process, moving up in the tree, until either
 We reach nodes whose values don’t need to be swapped (because the parent is still larger than both children), or
 We reach the root
DR. NEEPA SHAH 203
Dr. Neepa Shah 101

CONSTRUCTING A HEAP III

8 8 10 10
10 8 8 5
1 2 3
10 10 12
8 5 12 5 10 5
12 8 8
4
DR. NEEPA SHAH 204
OTHER CHILDREN ARE NOT AFFECTED

12 12 14
10 5 14 5 12 5
8 14 8 10 8 10
 The node containing 8 is not affected because its parent gets larger, not smaller
 The node containing 5 is not affected because its parent gets larger, not smaller
 The node containing 8 is still not affected because, although its parent got smaller, its parent is still
greater than it was originally
DR. NEEPA SHAH 205
Dr. Neepa Shah 102

A SAMPLE HEAP
 Here’s a sample binary tree 25
after it has been heapified
22 17
 Notice that heapified does not
mean sorted
 Heapifying does not change the 19 22 14 15
shape of the binary tree; this
18 14 21 3 9 11
binary tree is balanced and left-
justified
DR. NEEPA SHAH 206
REMOVING THE ROOT
 Notice that the largest 11

number is now in the root
 Suppose we discard the 22 17
root:
 How can we fix the binary
tree so it is once again 19 22 14 15
balanced and left-justified?
 Solution: remove the 18 14 21 3 9 11
rightmost leaf at the
deepest level and use it for
the new root
DR. NEEPA SHAH 207
Dr. Neepa Shah 103

THE REHEAP METHOD
 Our tree is balanced and

11
left-justified, but no longer a
heap 22 17
 However, only the root lacks
the heap property
19 22 14 15
 We can siftUp() the root
 After doing this, one and 18 14 21 3 9
only one of its children may
have lost the heap property
DR. NEEPA SHAH 208
THE REHEAP METHOD CONTD
22
 Now the left child of the
11 17
root (still the number 11)
lacks the heap property
 We can siftUp() this node 19 22 14 15
 After doing this, one and 18 14 21 3 9

only one of its children may
have lost the heap property
DR. NEEPA SHAH 209
Dr. Neepa Shah 104

THE REHEAP METHOD CONTD.
 Now the right child of the left 22

child of the root (still the
number 11) lacks the heap 22 17
property:
 We can siftUp() this node
19 11 14 15
 After doing this, one and only
one of its children may have 18 14 21 3 9
lost the heap property —but it
doesn’t, because it’s a leaf
DR. NEEPA SHAH 210
THE REHEAP METHOD CONTD.
 Our tree is once again a heap, 22

because every node in it has the
heap property 22 17
 Once again, the largest value is in

the root 19 21 14 15
 We can repeat this process until
the tree becomes empty 18 14 11 3 9
 This produces a sequence of

values in order largest to smallest
DR. NEEPA SHAH 211
Dr. Neepa Shah 105

SORTING
 What do heaps have to do with sorting an array?

 Because the binary tree is balanced and left justified, it can be represented as an array
 All our operations on binary trees can be represented as operations on arrays
 To sort:
heapify the array;
while the array isn’t empty {
remove and replace the root;
reheap the new root node;
}
DR. NEEPA SHAH 212
MAPPING INTO AN ARRAY

25
22 17
19 22 14 15
18 14 21 3 9 11
0 1 2 3 4 5 6 7 8 9 10 11 12
25 22 17 19 22 14 15 18 14 21 3 9 11
 The left child of index i is at index 2*i+1
DR. NEEPA SHAH  The right child of index i is at index 2*i+2 213
 Example: the children of node 3 (19) are 7 (18) and 8 (14)
Dr. Neepa Shah 106

REMOVING AND REPLACING THE ROOT

0 1 2 3 4 5 6 7 8 9 10 11 12
25 22 17 19 22 14 15 18 14 21 3 9 11
0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25
 The “root” is the first element in the array

 The “rightmost node at the deepest level” is the last element
 Swap them and pretend that the last element in the array no longer exists—that is,
the “last index” is 11 (containing the value 9)
DR. NEEPA SHAH 214
REHEAP AND REPEAT

0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25
0 1 2 3 4 5 6 7 8 9 10 11 12
22 22 17 19 21 14 15 18 14 11 3 9 25
0 1 2 3 4 5 6 7 8 9 10 11 12
9 22 17 19 22 14 15 18 14 21 3 22 25
 Reheap the root node (index 0, containing 11) And again, remove and replace the root node
 Remember, though, that the “last” array index is changed
DR. NEEPA SHAH 215
 Repeat until the last becomes first, and the array is sorted!
Dr. Neepa Shah 107

ANALYSIS I
 Here’s how the algorithm starts:

heapify the array;
 Heapifying the array: we add each of n nodes

 Each node has to be sifted up, possibly as far as the root
 Since the binary tree is perfectly balanced, sifting up a single node takes O(log n) time
 Since we do this n times, heapifying takes n*O(log n) time, that is, O(n log n) time
DR. NEEPA SHAH 216
ANALYSIS II
 Rest of the algorithm:

}
 We do the while loop n times (actually, n-1 times), because we remove one of the n nodes each time
 Removing and replacing the root takes O(1) time
 Therefore, the total time is n times however long it takes the reheap method
DR. NEEPA SHAH 217
Dr. Neepa Shah 108

ANALYSIS III
 To reheap the root node, we have to follow one path from the root to a leaf node (and we might stop before we
reach a leaf)
 The binary tree is perfectly balanced
 Therefore, this path is O(log n) long
 And we only do O(1) operations at each node
 Therefore, reheaping takes O(log n) times
 Since we reheap inside a while loop that we do n times, the total time for the while loop is n*O(log n), or O(n
log n)
DR. NEEPA SHAH 218
ANALYSIS IV
 Here’s the algorithm again:

heapify the array;
}
 We have seen that heapifying takes O(n log n) time

 The while loop takes O(n log n) time
 The total time is therefore O(n log n) + O(n log n)
 This is the same as O(n log n) time
DR. NEEPA SHAH 219
Dr. Neepa Shah 109

CLASSIFICATION OF SORTING ALGORITHMS

 Comparison sorts
 Bubble sort
 Insertion sort
 Selection sort
 Shell sort
 Merge sort
 Quick sort
 Heap sort
 Non-Comparison sorts
 Radix sort
DR. NEEPA SHAH 220
RADIX SORT
 Radix is the base of a number system
 Radix sort is a multiple pass distribution sort.
 Radix sort uses bucket sort as the stable sorting algorithm, where the initial relative order of
equal keys is unchanged.
DR. NEEPA SHAH 221
Dr. Neepa Shah 110

RADIX SORT
 ASCII (r = 26)
 Decimal (r = 10)
 Bit String (r = 2)
 Octal (r = 8)
 Hexadecimal (r = 16)
DR. NEEPA SHAH 222
RADIX SORT EXAMPLE

Input data:
a b a b a c c a a a c b b a b c c a a a c b b a
DR. NEEPA SHAH 223
Dr. Neepa Shah 111

RADIX SORT EXAMPLE

Pass 1: Looking at rightmost position.
Place into appropriate pile.
a b c
DR. NEEPA SHAH 224
RADIX SORT EXAMPLE

Pass 1: Looking at rightmost position.
b b a Join piles.
c c a
c a a b a b a a c
a b a a c b b a c
a b c
DR. NEEPA SHAH 225
Dr. Neepa Shah 112

RADIX SORT EXAMPLE

Pass 2: Looking at next position.
a b a c a a c c a b b a a c b b a b b a c a a c
a b c
DR. NEEPA SHAH 226
RADIX SORT EXAMPLE

Pass 2: Looking at next position.
a b a c a a c c a b b a a c b b a b b a c a a c
a a c Join piles.
b a c
b a b b b a a c b
c a a a b a c c a
a b c
DR. NEEPA SHAH 227
Dr. Neepa Shah 113

RADIX SORT EXAMPLE

Pass 3: Looking at last position.
c a a b a b b a c a a c a b a b b a c c a a c b
a b c
DR. NEEPA SHAH 228
RADIX SORT EXAMPLE

Pass 3: Looking at last position.
c a a b a b b a c a a c a b a b b a c c a a c b
Join piles.
a c b b b a
a b a b a c c c a
a a c b a b c a a
a b c
DR. NEEPA SHAH 229
Dr. Neepa Shah 114

RADIX SORT EXAMPLE

Result is sorted.
a a c a b a a c b b a b b a c b b a c a a c c a
DR. NEEPA SHAH 230
CLASSIFICATION OF RADIX SORT
Radix sort is classified based on how it works internally:
 least significant digit (LSD) radix sort

 most significant digit (MSD) radix sort
DR. NEEPA SHAH 231
Dr. Neepa Shah 115

RADIX SORT EXAMPLE
 Least-significant-digit-first
Example: 275, 087, 426, 061, 509, 170, 677, 503
DR. NEEPA SHAH 232
RADIX SORT CONTD.
DR. NEEPA SHAH 233
Dr. Neepa Shah 116

Example for LSD Radix Sort

d (digit) = 3, r (radix) = 10 ascending order
179, 208, 306, 93, 859, 984, 55, 9, 271, 33
front[0] NULL rear[0]
front[1] 271 NULL rear[1]
front[3] 93 33 NULL rear[3]
front[9] 179 859 9 NULL rear[9]
DR. NEEPA SHAH
271, 93, 33, 984, 55, 306, 208, 179, 859, 9 After the first pass 234
front[0] 306 208 9 null rear[0]

front[1] null rear[1]
front[3] 33 null rear[3]
front[5] 55 859 null rear[5]

DR. NEEPA SHAH
front[9] 93 null 235
306, 208, 9, 33, 55, 859, 271, 179, 984, 93 (second pass) rear[9]
Dr. Neepa Shah 117

front[0] 9 33 55 93 null rear[0]


DR. NEEPA SHAH
front[9] 984 null 236
9, 33, 55, 93, 179, 208, 271, 306, 859, 984 (third pass) rear[9]
RADIX SORT
 Every integer can be represented by at most k digits

 d1d2…dk where di are digits in base r
 d1: most significant digit
 dk: least significant digit
DR. NEEPA SHAH 237
Dr. Neepa Shah 118

RADIX SORT CONTD.
 Algorithm
 sort by the least significant digit first
=> Numbers with the same digit go to same bin
 reorder all the numbers: the numbers in bin 0 precede the numbers in bin 1, which precede the
numbers in bin 2, and so on
 sort by the next least significant digit
 continue this process until the numbers have been sorted on all k digits
DR. NEEPA SHAH 238
STEPS / PROCEDURE
1. For k=1 to number of digits do steps 2 and 3

2. For i=0 to n-1 do
Temp=x[i]
d = kth digit of temp
Place temp at rear of Q[d]
3. For j=0 to 9 do
delete elements of Q[j] and place in next sequential position of queue x
DR. NEEPA SHAH 239
Dr. Neepa Shah 119

ALGORITHM
// base 10
// FIFO
// d times of counting sort
// scan A[i], put into correct slot
// re-order back to original array
DR. NEEPA SHAH 240
RADIX SORT ANALYSIS
 Running time
 k passes over the numbers (i.e. k counting sorts, with range being 0..r)
 each pass takes O(N+r)
 total: O(Nk+rk)
 r and k are constants: O(N)
NEEPA SHAH 241
Dr. Neepa Shah 120

COMPARISON OF SORTING ALGORITHMS

Algorithm Average Worst case Space usage
Selection n2 n2 In place
Bubble n2 n2 In place
Insertion n2 n2 In place
Merge O(nlog2n) O(nlog2n) Extra n entries
Quick O(nlog2n) O(n2)
Heap O(nlog2n) O(nlog2n) In place
Radix O(m + n) O(m + n) Extra space for links
NEEPA SHAH 242
Dr. Neepa Shah 121

Searching and Sorting

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Searching and Sorting

Uploaded by

Copyright:

Available Formats

Unit 8: Searching and Sorting

DR. NEEPA SHAH 2

Dr. Neepa Shah 1

DR. NEEPA SHAH 3

 Given an integer X and integers A0 , A1 , …, AN-1, which are

DR. NEEPA SHAH 4

Dr. Neepa Shah 2

BINARY SEARCH: EXAMPLE

DR. NEEPA SHAH 5

BINARY SEARCH: EXAMPLE CONTD.

DR. NEEPA SHAH 6

Dr. Neepa Shah 3

DR. NEEPA SHAH 8

Dr. Neepa Shah 4

 Sorting algorithms are often classified by:

DR. NEEPA SHAH 9

Dr. Neepa Shah 5

 One of the simplest sorting algorithm

 Also known as Exchange sort

DR. NEEPA SHAH 11

Dr. Neepa Shah 6

BUBBLE SORT EXAMPLE CONTD.

DR. NEEPA SHAH 13

BUBBLE SORT EXAMPLE CONTD.

DR. NEEPA SHAH 14

Dr. Neepa Shah 7

BUBBLE SORT EXAMPLE CONTD.

DR. NEEPA SHAH 15

BUBBLE SORT EXAMPLE CONTD.

Dr. Neepa Shah 8

BUBBLE SORT EXAMPLE CONTD.

BUBBLE SORT PROCEDURE

1. Do steps 2-3 for i= n-1 down to 1

DR. NEEPA SHAH 18

Dr. Neepa Shah 9

BUBBLE SORT ALGORITHM

void sort(int a[]) {

DR. NEEPA SHAH 19

Analysis: Worst Case Complexity O(n2)

Pros: Simplicity and ease of implementation

Cons: Horribly Inefficient

DR. NEEPA SHAH 20

Dr. Neepa Shah 10

ALREADY SORTED COLLECTIONS?

 What if the collection was already sorted?

 We want to be able to detect this and “stop early”!

DR. NEEPA SHAH 21

USING A BOOLEAN “FLAG”

 If no swapping occurred, then we know that the collection is already sorted!

 This Boolean “flag” needs to be reset after each “bubble up.”

DR. NEEPA SHAH 22

Dr. Neepa Shah 11

DR. NEEPA SHAH 23

DR. NEEPA SHAH 24

Dr. Neepa Shah 12

DR. NEEPA SHAH 25

DR. NEEPA SHAH 26

Dr. Neepa Shah 13

DR. NEEPA SHAH 27

DR. NEEPA SHAH 28

Dr. Neepa Shah 14

DR. NEEPA SHAH 29

DR. NEEPA SHAH 30