Sorting

Jan 8, 2023 - Data Structure and Algorithm
Sorting
Sorting Techniques
Sorting is nothing but arranging the data in ascending or descending order. The term
sorting
came into picture, as humans realised the importance of searching quickly.
There are so many things in our real life that we need to search for, like a particular
record in database, roll numbers in merit list, a particular telephone number in
telephone directory, a particular page in a book etc. All this would have been a mess if
the data was kept unordered and unsorted, but fortunately the concept of sorting
came into existence, making it easier for everyone to arrange data in an order, hence
making it easier to search.
Sorting arranges data in a sequence which makes searching easier.
Sorting Efficiency
If you ask me, how will I arrange a deck of shuffled cards in order, I would say, I will
start by checking every card, and making the deck as I move on.
It can take me hours to arrange the deck in order, but that's how I will do it.
Since the beginning of the programming age, computer scientists have been working
on solving the problem of sorting by coming up with various different algorithms to
sort data.
The two main criterias to judge which algorithm is better than the other have been:
1. Time taken to sort the given data.

2. Memory Space required to do so.
Different Sorting Algorithms
There are many different techniques available for sorting, differentiated by their
efficiency and space requirements. Following are some sorting techniques which we
will be covering in next few tutorials.
1. Bubble Sort
2. Insertion Sort
3. Selection Sort
4. Quick Sort
5. Merge Sort
6. Heap Sort
7. Counting Sort
8. Shell Sort
Although it's easier to understand these sorting techniques, but still we suggest you to
first learn about Space complexity, Time complexity and the searching algorithms, to
warm up your brain for sorting algorithms.
Bubble Sort
Bubble Sort is a simple algorithm which is used to sort a given set of n elements
provided in form of an array with n number of elements. Bubble Sort compares all the
element one by one and sort them based on their values.
If the given array has to be sorted in ascending order, then bubble sort will start by
comparing the first element of the array with the second element, if the first element
is greater than the second element, it will swap both the elements, and then move on
to compare the second and the third element, and so on.
If we have total n elements, then we need to repeat this process for n-1 times.
It is known as bubble sort, because with every complete iteration the largest element
in the given array, bubbles up towards the last place or the highest index, just like a
water bubble rises up to the water surface.
Sorting takes place by stepping through all the elements one-by-one and comparing it
with the adjacent element and swapping them if required.
Implementing Bubble Sort Algorithm
Following are the steps involved in bubble sort(for sorting a given array in ascending
order):
1. Starting with the first element(index = 0), compare the current element with
the next element of the array.
2. If the current element is greater than the next element of the array, swap them.
Copyright © 2023 Powered by www.benchpartner.com
3. If the current element is less than the next element, move to the next element.
Repeat Step 1.
Let's consider an array with values {5, 1, 6, 2, 4, 3}
Below, we have a pictorial representation of how bubble sort will sort the given array.

Image not found or type unknown
So as we can see in the representation above, after the first iteration, 6 is placed at
the last index, which is the correct position for it.
Similarly after the second iteration, 5 will be at the second last index, and so on.

Another Example of Bubble Sort
We take an unsorted array for our example. Bubble sort takes Ο(n2) time so we're
keeping it short and precise.

Bubble sort starts with very first two elements, comparing them to check which one is
greater.

In this case, value 33 is greater than 14, so it is already in sorted locations. Next, we
compare 33 with 27.

We find that 27 is smaller than 33 and these two values must be swapped.

The new array should look like this −



Next we compare 33 and 35. We find that both are in already sorted positions.

Then we move to the next two values, 35 and 10.


We know then that 10 is smaller 35. Hence they are not sorted.

We swap these values. We find that we have reached the end of the array. After one
iteration, the array should look like this −

To be precise, we are now showing how an array should look like after each iteration.
After the second iteration, it should look like this

Notice that after each iteration, at least one value moves at the end.

And when there's no swap required, bubble sorts learns that an array is completely
sorted.

Now we should look into some practical aspects of bubble sort.
Insertion Sort
This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained

which is always sorted. For example, the lower part of an array is maintained to be
sorted. An element which is to be 'insert'ed in this sorted sub-list, has to find its
appropriate place and then it has to be inserted there. Hence the name, insertion
sort.
The array is searched sequentially and unsorted items are moved and inserted into the
sorted sub- list (in the same array). This algorithm is not suitable for large data sets as
its average and worst case complexity are of Ο(n2), where n is the number of items.
Characteristics of Insertion Sort:
1. It is efficient for smaller data sets, but very inefficient for larger lists.
2. Insertion Sort is adaptive, that means it reduces its total number of steps if a
partially sorted array is provided as input, making it efficient.
3. It is better than Selection Sort and Bubble Sort algorithms.
4. Its space complexity is Like bubble Sort, insertion sort also requires a single
additional memory space.
5. It is a stable sorting technique, as it does not change the relative order of
elements which are equal.
Below, we have a pictorial representation of how bubble sort will sort the given array.

As you can see in the diagram above, after picking a key, we start iterating over the
elements to the left of the key.
How Insertion Sort Works?

We take an unsorted array for our example

Insertion sort compares the first two elements.

It finds that both 14 and 33 are already in ascending order. For now, 14 is in sorted
sub-list.
Insertion sort moves ahead and compares 33 with 27.
And finds that 33 is not in the correct position.
It swaps 33 with 27. It also checks with all the elements of sorted sub-list. Here we see
that the sorted sub-list has only one element 14, and 27 is greater than 14. Hence,
the sorted sub-list remains sorted after swapping.
By now we have 14 and 27 in the sorted sub-list. Next, it compares 33 with 10.
These values are not in a sorted order.

So we swap them.
However, swapping makes 27 and 10 unsorted.
Hence, we swap them too.
Again we find 14 and 10 in an unsorted order.
We swap them again. By the end of third iteration, we have a sorted sub-list of 4
items.
This process goes on until all the unsorted values are covered in a sorted sub-list. Now
we shall see some programming aspects of insertion sort.

Selection Sort
Selection sort is a simple sorting algorithm. This sorting algorithm is an in-place

comparison- based algorithm in which the list is divided into two parts, the sorted part
at the left end and the unsorted part at the right end. Initially, the sorted part is empty
and the unsorted part is the entire list.
The smallest element is selected from the unsorted array and swapped with the
leftmost element, and that element becomes a part of the sorted array. This process
continues moving unsorted array boundary by one element to the right.
This algorithm is not suitable for large data sets as its average and worst case
complexities are of
Ο (n2), where n is the number of items.
How Selection Sort Works?
Following are the steps involved in selection sort (for sorting a given array in
ascending order):
1. Starting from the first element, we search the smallest element in the array, and
replace it with the element in the first position.
2. We then move on to the second position, and look for smallest element present in
the subarray, starting from index 1, till the last index.
3. We replace the element at the second position in the original array, or we can
say at the first position in the subarray, with the second smallest element.
4. This is repeated, until the array is completely sorted.
Let's consider an array with values {14, 33, 27, 10, 35, 19, 24, 44}
For the first position in the sorted list, the whole list is scanned sequentially. The first
position where 14 is stored presently, we search the whole list and find that 10 is the
lowest value.

So we replace 14 with 10. After one iteration 10, which happens to be the minimum
value in the list, appears in the first position of the sorted list.
For the second position, where 33 is residing, we start scanning the rest of the list in a
linear manner.
We find that 14 is the second lowest value in the list and it should appear at the
second place. We swap these values.
After two iterations, two least values are positioned at the beginning in a sorted
manner.
The same process is applied to the rest of the items in the array. Following is a
pictorial depiction of the entire sorting process

Now, let us learn some programming aspects of selection sort.
Quick Sort
Quick Sort is also based on the concept of Divide and Conquer, just like merge sort.
But in quick sort all the heavy lifting(major work) is done while dividing the array into
subarrays, while in case of merge sort, all the real work happens during merging the
subarrays. In case of quick sort, the combine step does absolutely nothing.
It is also called partition-exchange sort. This algorithm divides the list into three
main parts:
1. Elements less than the Pivot element

2. Pivot element(Central element)
3. Elements greater than the pivot element
Pivot element can be any element from the array, it can be the first element, the last
element or any random element. In this tutorial, we will take the rightmost element or
the last element as pivot.
For example: In the array {52, 37, 63, 14, 17, 8, 6, 25}, we take 25 as pivot. So
after the first pass, the list will be changed like this.
{6 8 17 14 25 63 37 52}
Hence after the first pass, pivot will be set at its position, with all the elements
smaller to it on its left and all the elements larger than to its right. Now 6 8 17 14
and 63 37 52 are considered as two separate sunarrays, and same recursive logic will
be applied on them, and we will keep doing this until the complete array is sorted.
How Quick Sorting Works?
Following are the steps involved in quick sort algorithm:
1. After selecting an element as pivot, which is the last index of the array in our
case, we divide the array for the first time.
2. In quick sort, we call this partitioning. It is not simple breaking down of array
into 2 subarrays, but in case of partitioning, the array elements are so positioned
that all the elements smaller than the pivot will be on the left side of the pivot
and all the elements greater than the pivot will be on the right side of it.
3. And the pivot element will be at its final sorted position.
4. The elements to the left and right, may not be sorted.
5. Then we pick subarrays, elements on the left of pivot and elements on the right
of pivot, and we perform partitioning on them by choosing a pivot in the
subarray.
Let's consider an array with values {9, 7, 5, 11, 12, 2, 14, 3, 10, 6}
Below, we have a pictorial representation of how quick sort will sort the given array.

In step 1, we select the last element as the pivot, which is 6 in this case, and call for
partitioning, hence re-arranging the array in such a way that 6 will be placed in its
final position and to its left will be all the elements less than it and to its right, we will
have all the elements greater than it.
Then we pick the subarray on the left and the subarray on the right and select a pivot
for them, in the above diagram, we chose 3 as pivot for the left subarray and 11 as
pivot for the right subarray.
And we again call for partitioning.
Merge Sort
Merge Sort follows the rule of Divide and Conquer to sort a given set of
numbers/elements, recursively, hence consuming less time.
In the last two tutorials, we learned about Selection Sort and Insertion Sort, both of
which have a worst-case running time of O(n2). As the size of input grows, insertion
and selection sort can take a long time to run.
Merge sort, on the other hand, runs in O(n*log n) time in all the cases.
Before jumping on to, how merge sort works and its implementation, first lets
understand what is the rule of Divide and Conquer?
The concept of Divide and Conquer involves three steps:
1. Divide the problem into multiple small problems.

2. Conquer the sub problems by solving them. The idea is to break down the
problem into atomic sub problems, where they are actually solved.
3. Combine the solutions of the sub problems to find the solution of the actual
problem.


How Merge Sort Works?
To understand merge sort, we take an unsorted array as the following
We know that merge sort first divides the whole array iteratively into equal halves
unless the atomic values are achieved. We see here that an array of 8 items is divided
into two arrays of size 4.
This does not change the sequence of appearance of items in the original. Now we
divide these two arrays into halves.
We further divide these arrays and we achieve atomic value which can no more be
divided.
Now, we combine them in exactly the same manner as they were broken down. Please
note the color codes given to these lists.
We first compare the element for each list and then combine them into another list in
a sorted manner. We see that 14 and 33 are in sorted positions. We compare 27 and
10 and in the target list of 2 values we put 10 first, followed by 27. We change the
order of 19 and 35 whereas 42 and 44 are placed sequentially.
In the next iteration of the combining phase, we compare lists of two data values, and
merge them into a list of found data values placing all in a sorted order.

After the final merging, the list should look like this
Now we should learn some programming aspects of merge sorting.
Below, we have a pictorial representation of how merge sort will sort the given array.

Heap Sort
Heap Sort is one of the best sorting methods being in-place and with no quadratic
worst-case running time. Heap sort involves building a Heap data structure from the
given array and then utilizing the Heap to sort the array.
You must be wondering, how converting an array of numbers into a heap data
structure will help in sorting the array. To understand this, let's start by understanding
what a Heap is.
What is a Heap?
Heap is a special tree-based data structure that satisfies the following special heap
properties:
1. Shape Property:
Heap data structure is always a Complete Binary Tree, which means all levels of the
tree are fully filled.
2. Heap Property:
All nodes are either greater than or equal to or less than or equal to each of its
children. If the parent nodes are greater than their child nodes, heap is called a Max-
Heap, and if the parent nodes are smaller than their child nodes, heap is called Min-
Heap.

How Heap Sort Works?
Heap sort algorithm is divided into two basic parts:
Creating a Heap of the unsorted list/array.

Then a sorted array is created by repeatedly removing the largest/smallest
element from the heap, and inserting it into the array. The heap is reconstructed
after each removal.
Initially on receiving an unsorted list, the first step in heap sort is to create a Heap
data structure (Max-Heap or Min-Heap).
Once heap is built, the first element of the Heap is either largest or smallest
(depending upon Max-Heap or Min-Heap), so we put the first element of the heap in
our array.
Then we again make heap using the remaining elements, to again pick the first
element of the heap and put it into the array. We keep on doing the same repeatedly
untill we have the complete sorted list in our array.
In the below algorithm, initially heapsort() function is called, which calls heapify() to
build the heap.
Counting Sort
Counting Sort Algorithm is an efficient sorting algorithm that can be used for sorting
elements within a specific range. This sorting technique is based on the
frequency/count of each element to be sorted and works using the following algorithm-
Input: Unsorted array A[] of n elements

Output: Sorted arrayB[]
Step 1: Consider an input array A having n elements in the range of 0 to k, where n

and k are positive integer numbers. These n elements have to be sorted in ascending
order using the counting sort technique. Also note that A[] can have distinct or
duplicate elements
Step 2: The count/frequency of each distinct element in A is computed and stored in

another array, say count, of size k+1. Let u be an element in A such that its frequency
is stored at count[u].
Step 3: Update the count array so that element at each index, say i, is equal to -
Step 4: The updated count array gives the index of each element of array A in the
sorted sequence. Assume that the sorted sequence is stored in an output array, say B,
of size n.
Step 5: Add each element from input array A to B as follows:
1. Set i=0 and t = A[i]

2. Add t to B[v] such that v = (count[t]-1).
3. Decrement count[t] by 1
4. Increment i by 1
Repeat steps (a) to (d) till i = n-1
Step 6: Display B since this is the sorted array
Example of Counting Sort
Let us trace the above algorithm using an example:
Consider the following input array A to be sorted. All the elements are in range 0 to 9
A[]= {1, 3, 2, 8, 5, 1, 5, 1, 2, 7}
Step 1: Initialize an auxiliary array, say count and store the frequency of every
distinct element. Size of count is 10 (k+1, such that range of elements in A is 0
to k)

Figure 1: count array
Step 2: Using the formula, updated count array is -


Figure 2: Formula for updating count array


Figure 3 : Updated count array
Step 3: Add elements of array A to resultant array B using the following steps:
For, i=0, t=1, count[1]=3, v=2. After adding 1 to B[2], count[1]=2 and i=1

Figure 4: For i=0
For i=1, t=3, count[3]=6, v=5. After adding 3 to B[5], count[3]=5 and i=2

Figure 5: For i=1
For i=2, t=2, count[2]=

Image 5, unknown
not found or type v=4. After adding 2 to B[4], count[2]=4 and i=3

Figure 6: For i=2
For i=3, t=8, count[8]= 10, v=9. After adding 8 to B[9], count[8]=9 and i=4

Figure 7: For i=3
On similar lines, we have the following:



Image
not
found
or
type
unknown
Figure 8: For i=4

Image
not
found
or type
unknown
Figure 9: For i=5

Figure 10: For i=6

Figure 11: For i=7

Figure 12: For i=8

Figure 13: For i=9
Thus, array B has the sorted list of elements.
Shell Sort
Shell sort is a highly efficient sorting algorithm and is based on insertion sort
algorithm. This algorithm avoids large shifts as in case of insertion sort, if the smaller
value is to the far right and has to be moved to the far left.
This algorithm uses insertion sort on a widely spread elements, first to sort them and
then sorts the less widely spaced elements. This spacing is termed as interval. This
interval is calculated based on Knuth's formula as
Knuth's Formula
h = h * 3 + 1 where
h is interval with initial value 1
This algorithm is quite efficient for medium-sized data sets as its average and worst-
case complexity of this algorithm depends on the gap sequence the best known is
Ο(n), where n is the number of items. And the worst case space complexity is O(n).
How Shell Sort Works?
Let us consider the following example to have an idea of how shell sort works. We take
the same array we have used in our previous examples. For our example and ease of
understanding, we take the interval of 4. Make a virtual sub-list of all values located at
the interval of 4 positions.
Here these values are {35, 14}, {33, 19}, {42, 27} and {10, 44}
We compare values in each sub-list and swap them (if necessary) in the original array.
After this step, the new array should look like this

Then, we take interval of 1 and this gap generates two sub-lists - {14, 27, 35, 42},
{19, 10, 33, 44}

We compare and swap the values, if required, in the original array. After this step, the
array should look like this −


Image
Jannot found- or
8, 2023 type
Data unknown
Structure and Algorithm
Finally, we sort the rest of the array using interval of value 1. Shell sort uses insertion
sort to sort the array.
Following is the step-by-step depiction

We see that it required only four swaps to sort the rest of the array.

Sorting

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sorting

Uploaded by

Copyright:

Available Formats

Jan 8, 2023 - Data Structure and Algorithm

came into picture, as humans realised the importance of searching quickly.

Sorting arranges data in a sequence which makes searching easier.

1. Time taken to sort the given data.

Different Sorting Algorithms

Implementing Bubble Sort Algorithm

Let's consider an array with values {5, 1, 6, 2, 4, 3}

Copyright © 2023 Powered by www.benchpartner.com

Another Example of Bubble Sort

The new array should look like this −

Copyright © 2023 Powered by www.benchpartner.com

Then we move to the next two values, 35 and 10.

Copyright © 2023 Powered by www.benchpartner.com

Image not found or type unknown

Now we should look into some practical aspects of bubble sort.

This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained

Characteristics of Insertion Sort:

How Insertion Sort Works?

We take an unsorted array for our example

Insertion sort compares the first two elements.

Image not found or type unknown

Insertion sort moves ahead and compares 33 with 27.

Image not found or type unknown

And finds that 33 is not in the correct position.

Image not found or type unknown

Image not found or type unknown

Image not found or type unknown

These values are not in a sorted order.

Copyright © 2023 Powered by www.benchpartner.com

Image not found or type unknown

Image not found or type unknown

However, swapping makes 27 and 10 unsorted.

Image not found or type unknown

Hence, we swap them too.

Image not found or type unknown

Again we find 14 and 10 in an unsorted order.

Image not found or type unknown

Image not found or type unknown

Image not found or type unknown

Selection sort is a simple sorting algorithm. This sorting algorithm is an in-place

Ο (n2), where n is the number of items.

How Selection Sort Works?

Image not found or type unknown

Copyright © 2023 Powered by www.benchpartner.com

Image not found or type unknown

Image not found or type unknown

Image not found or type unknown

Image not found or type unknown

Image not found or type unknown

Copyright © 2023 Powered by www.benchpartner.com

Image not found or type unknown

Now, let us learn some programming aspects of selection sort.

1. Elements less than the Pivot element

How Quick Sorting Works?

Following are the steps involved in quick sort algorithm:

pivot for the right subarray.

And we again call for partitioning.

The concept of Divide and Conquer involves three steps:

1. Divide the problem into multiple small problems.

Copyright © 2023 Powered by www.benchpartner.com

How Merge Sort Works?

To understand merge sort, we take an unsorted array as the following

Image not found or type unknown

Image not found or type unknown