Sorting

CS-240 & CS-341

Dick Steflik
Exchange Sorting

• Method : make n-1 passes across the data,

on each pass compare adjacent items,
swapping as necessary (n-1 compares)
• O(n2)
 Exchange Sorting
Basic Algorithm ( no improvements)

int ExchangeSort ( int & num[ ] , const int & numel )

{
int i,j,k, moves = 0;
for ( i = 0 ; i < (numel - 1) ; i++ )
for ( j = 0 ; j < (numel - 1) ; j++ )
if ( num [ j ] < num [ j-1 ])
{
temp = num [ j ];
num [ j ] = num [ j - 1 ];
num [ j - 1] = temp;
moves++;
}
return moves;
}
Exchange Sort
Improvement #1 : don’t revist places that are already in the correct place

int ExchangeSort ( int & num[ ] , const int & numel )

{
int i,j,k, moves = 0;
for ( i = 0 ; i < (numel - 1) ; i++ )
for ( j = i ; j < numel ; j++ )
if ( num [ j ] < num [ j-1 ])
{
temp = num [ j ];
num [ j ] = num [ j - 1 ];
num [ j - 1] = temp;
moves++;
}
return moves;
}
Exchange Sort
int ExchangeSort ( int & num[ ] , const int & numel )
{ int i,j,k, swap,moves = 0;
swap = 1;
for ( i = 0 ; i < (numel - 1) ; i++ )
if ( swap != 0 )
// Improvement # 2:
{ swap = 0;
// if a pass through the inner loop occurs
for ( j = 0 ; j < numel ; j++ )
// that doesn’t product a swap, you’re done.
if ( num [ j ] < num [ j-1 ])
{
// both of the improvements improve the
temp = num [ j ];
num [ j ] = num [ j - 1 ]; // overall performance but to not improve
// O(n2)
num [ j - 1] = temp;
moves++;
swap = 1;
}
}
return moves;
Exchange Sort

3 7 5 2 4 compare 3 and 7 ; 7 is > 3 so advance

3 5 7 2 4 compare 7 and 5, 7 > 5 so swap them

3 5 2 7 4 compare 7 and 2, 7 >4 so swap them

3 5 2 4 7 compare 7 and 4, 7 >4 so swap them

End of pass 1; notice that 7 is in the right place

Exchange Sort - pass 2
3 5 2 4 7

3 5 2 4 7 compare 3 and 5 ; 3 is >5 so advance

3 2 5 4 7 compare 5 and 2, 5 < 2 so swap them

3 2 4 5 7 compare 5 and 4, 5 > 4 so swap them

3 2 4 5 7 compare 5 and 7, 5 < 7 so pass 2 done

End of pass 2; notice that 5 and 7 are in the right places

Exchange Sort - pass 3
3 2 4 5 7

2 3 4 5 7 compare 3 and 2 ; 3 is >2 so swap them

2 3 4 5 7 compare 3 and 4, 3 <42 so advance

2 3 4 5 7 compare 4 and 5, 4 < 5 so advance

2 3 4 5 7 compare 5 and 7, 5 < 7 so pass 3 done

End of pass 3; notice that 4, 5 and 7 are in the right places

 Exchange Sort - pass n-1 (4)
2 3 4 5 7

2 3 4 5 7 compare 2 and 3 ; 2 is < 3 so advance

2 3 4 5 7 compare 3 and 4, 3 <4 so advance

2 3 4 5 7 compare 4 and 5, 4 < 5 so advance

2 3 4 5 7 compare 5 and 7, 5 < 7 so pass n-1 done

End of pass 4; notice that everything is where it should be.

Exchange Sort Improvements

• on each successive pass, do one less

compare, because the last item from that
pass is in place
• if you ever make a pass in which no swap
occurs, the sort is complete
• These will both improve performance but
Big O will remain O(n2)
Insertion Sort

• Strategy: divide the collection into two lists,

one listed with one element (sorted) and the
other with the remaining elements.
• On successive passes take an item from the
unsorted list and insert it into the sorted list
so the the sorted list is always sorted
• Do this until the unsorted list is empty
Insertion Sort
sorted unsorted

3 7 5 2 4
take an item from the unsorted list (7) and
insert into the sorted list
sorted unsorted

3 7 5 2 4
take next item from the unsorted list (5) and
sorted unsorted insert into the sorted list

3 5 7 2 4
take next item from the unsorted list (2)
sorted unsorted and insert into the sorted list

2 3 5 7 4
take next item from the unsorted list (4)
sorted unsorted and insert into the sorted list

2 3 4 5 7
Insertion Sort
Void InsertionSort ( int A[ ] , int n )
{
int i , j;
int temp;
for ( i = i < n , i++ )
{ // scan down list looking for correct place to put new element
j=i ;
temp = A[ i ];
while ( j < 0 && temp < A[ j-1 ])
{ // shift the list 1 element to the right to make room for new element
A[ j ] = A[ j+1 ];
j--;
}
A [ j ] = temp;
}
}
Insertion Sort

• Note that each insertion could be O(n-1)

and there are n-1 insertions being done
therefore Big O is O(n2)
• This is very much like building an ordered
linked list except there is more data
movement
Selection Sort

• Strategy: make a pass across the data

looking for the largest item, swap the
largest with the last item in the array.
• On successive passes (n-1) assume the array
is one smaller (the last item is in the correct
place) and repeat previous step
Selection Sort
int SelectionSort ( int num [ ] , int numelem )
{ int i , j , min minidx , temp , moves = 0;
for ( i = 0 ; i < numelem ; i++)
{ min = num [ i ] ;
minidx = i ;
for ( j = i + 1 ; j < numelem ; j++ )
{ min = num[ j ] ; minidx = j ; }
if ( min < num[ i ] )
{ temp = num[ i ] ;
num[ i ] = min ;
num[ minidx ];
moves++;
}
}
Selection Sort
biggest last

3 7 5 2 4

3 4 5 2 7
biggest last

3 4 5 2 7

3 4 2 5 7
biggest last

3 4 2 5 7

3 2 4 5 7

3 2 4 5 7

2 3 4 5 7
Selection Sort
• Notice that in selection sort, there is the least
possible data movement
• There are still n-1 compares on sublists that
become one item smaller on each pass so,
Big O is still O(n2)
• This method has the best overall performance
of the O(n2) algorithms because of the limited
amount of data movement
Heap Sort
• A heap is a tree structure in which the key at
the root is max(keys) and the key of every
parent is greater than the key of either of its
children
• Visualize your array as a complete binary
tree
• Arrange the tree into a heap by recursively
reapplying the heap definition
Heap Sort

• repeat the following n-1 times

– swap the root with the last element in the tree
– starting at the root reinforce the heap definition
– decrement the index of the last element
A Sample Heap

8 Notice: largest key is at root

5 7

Notice: Keys of children are < key of parent

3 2
Heap Sort

void HeapSort( int A[ ] , int n )

{ // this constructor turns the array into a max heap
Heap H(A,n);
int elt;
for (int i = n-1 ; i >= 1 ; i--)
{
// delete smallest element from heap and place it in A[ i ]
elt = H.Hdelete ( ); // Hdelete deletes the largest element and reenforces the heap property
A[ I ] = elt ;
}
}
Visualize array to sort as tree
0 1 2 3 4
0
3 7 5 2 4 3

1 2
7 5

3 4
2 4
 Make into a heap
0 0
3 0
7 7

7 1 5 1
3 5 4 1 5

3 4 3 4 3 4
2 4 2 4 2 3

STEP 1 STEP 2 STEP 3

Original Array Start at root, compare root to the Repeat on preorder traversal
children, swap root with largest path. At this point we have a
child heap

Note: Larger arrays will take more steps

 Now the sort starts
0 0
7 0
3 5
2 2
3 2

4 1 5 1
4 5 4 1 3

3 4 3 4 3 4
2 3 2 7 2 7

Swap the root and the last node Re heapize it by swaping the root
with its largets child
Notice what is left is no longer a
heap
 Sorting...
0 0
2 4

2 2

4 1 3 2 1 3

3 4 3 4
5 7 5 7

Reenforce the heap

Swap the root and the last node property on the remaining
Notice what is left is no longer a tree
heap
 Still Sorting...
0 0 0
3 3 2
2 2 2

2 1 4 1 1
2 4 3 4

3 4 3 4 3 4
5 7 5 7 5 7

Swap the root with the last Reenforce the heap Swap the root with the last
element property if necessary element
And what we have left is...
0
2

3 1 4

3 4
5 7

0 1 2 3 4

2 3 4 5 7
The Merge Principle
• Assume there exist two sorted lists (with queue behavior)

7 5 3 1

8 6 4 2

Compare the items at the front of the list and move the smaller to the back of a third list

7 5 3
1
8 6 4 2

Do it again

7 5 3
2 1
8 6 4
The Merge Principle
And again

7 5
3 2 1
8 6 4

And again, and again, and again…until

8 7 6 5 4 3 2 1

We have a list whose length is the sum of the lengths and contains all of the elements of
both lists, and this list is also ordered
Merge Sort

void MergeSort ( int data[ ] , int n )

{
int n1 , n2 ; // size of first subarray and second subarrays respectively
if (n > 1 )
{
n1 = n / 2 ;
n2 = n - n1;
MergeSort ( data , n1); // sort from data[0] to data[n1-1]
MergeSort ((data + n1) , n2); // sort from data[n1] to end
// merge the two sorted halves
Merge (data , n1 , n2 );
}
}
Merge Sort
75 17 5 12 19 24 4 43 34 11 3 33 14 23 8 27

Picture the given list as a collection of n, 1 element sorted lists (i.e. 75 is a sorted 1 element
list, as is 17. Now merge the adjacent lists of length 1 into lists of length 2….

17 75 5 12 19 24 4 43 11 34 3 33 14 23 8 27

...now merge the lists of length 2 into lists of length 4

5 17 12 75 4 19 24 43 3 11 33 34 8 14 23 27

...now merge the lists of length 4 into lists of length 8

4 5 12 17 19 24 43 75 3 8 11 14 23 27 33 34

...now merge the lists of length 8 into lists of length 16

3 4 5 8 11 12 14 17 19 23 23 27 33 34 43 75

…and there you have it merge sort

Quick Sort
• This sorting method by far outshines all of the others for flat out speed
• Big O is log2n
• there are problems, worst case performance is when data is already in
sorted order or is almost in sorted order (we’ll analyze this separately)
• and there are solutions to the problems
• and there is an improvement to make it faster still
 Quick Sort
Pick the leftmost element as the pivot (23). Now , start two cursors (one at either end) going towards the
middle and swap values that are > pivot (found with left cursor) with values < pivot (found with right cursor)

23 17 5 12 19 24 4 43 34 11 3 33 14 26 8 27

swap

23 17 5 12 19 8 4 43 34 11 3 33 14 26 24 27

swap

23 17 5 12 19 8 4 14 34 11 3 33 43 26 24 27

swap

23 17 5 12 19 8 4 14 3 11 34 33 43 26 24 27
swap

Finally, swap the pivot and the value where the cursors passed each other

11 17 5 12 19 8 4 14 3 23 34 33 43 26 24 27

Note : 23 is now in the right place and everything to its left is < 23
and everything to its left is > 23
Quick Sort
Now, repeat the process for the right partition

11 17 5 12 19 8 4 14 3 23 34 33 43 26 24 27

swap

11 17 5 12 19 8 4 14 3

swap
11 3 5 4 19 8 12 14 17

swap

11 3 5 4 8 19 12 14 17

swap

8 3 5 4 11 19 12 14 17

Note: the 11 is now in the right place, and the left partition is all < pivot and the right partition is all > pivot
Quick Sort
Now, repeat the process with the right partition

8 3 5 4 11 19 12 14 17

8 3 5 4

Notice that there is nothing to swap , so swap the pivot and the 4, now the 8 is on the right place

4 3 5 8

Repeat the process on the leftmost partition again

4 3 5

The 4 is now in the right place and the left and right partitions
3 4 5 are both of size one so they must also be in the right place
Quick Sort
Now that we’ve exhausted the left partitions, back up and do the right partition

3 4 5 8 11 19 12 14 17 23

17 12 14 19

swap

14 12 17

swap

Since the left partition is just two items, just

compare and swap if necessary

12 14

3 4 5 8 11 12 14 17 19 23

Now back up and do the remaining right partition

Quick Sort

3 4 5 8 11 12 14 17 19 23 34 33 43 26 24 27

swap
34 33 27 26 24 43

swap
24 33 27 26 34 43

swap
24 26 27 33

24 26 27 33

26 27 33

3 4 5 8 11 12 14 17 19 23 24 26 27 33 34 43

All done
Quick Sort (worst case)
• If the data is already sorted watch what happens to the partitions

3 4 5 8 11 12 14 17 19 23 24 26 27 33 34 43

There is nothing to swap

4 5 8 11 12 14 17 19 23 24 26 27 33 34 43

Again, nothing to swap…..

The partitions are always the maximum size and the performance
degrades to O(n2)
 Quick Sort (worst case solution)
• The problem comes from the way we picked the pivot
• Pick the pivot a different way
– median-of-three
• instead of always picking the leftmost value of the partition use the median of
the first, the middle and the last value in the partition.
Quick Sort Improvement
• Since Quick sort is a recursive algorithm, a lot of time gets eaten up
in the recursion involved with sorting the small partitions
• Solution - use a different algorithm for the small partitions
– remember for small collections of data the n2 algorithms perform better
than the log2n algorithms
– this is one place where exchange sort excels
• if the partition size is 15 or less use exchange (or selection sort)
Radix Sort