Subject: Data Structure

Using C++ (CSET202)

Week 5 Lecture

Merge Sort and Count Sort

Algorithms
What, Why and How?

by

Dr Gaurav Kumar
Asst. Prof, Bennett University
Quick Recap of Last Week’s Learnings

• Analyzed the Operations of Bubble and Selection Sorting

Algorithms

• Optimized the performance of Bubble Sorting Algorithm

• Understood the Philosophy, Concept, Visualization, and

Time Complexity of Insertion and Shell Sort Algorithms
Quick Recap of Last Week’s Learnings

Understanding Bubble Sort void bubbleSort (int array[], int size)

{
N=6 flag=1;
for (int pass = 1; pass < size; ++pass)
1 2 3 4 5 6 {
for (int i = 0; i <= size - pass; ++i)
{
Pass #Comp #Swap if (array[i] > array[i + 1])
{
1 5 0
int temp = array[i];
array[i] = array[i + 1];
For n values array[i + 1] = temp;
Pass #Comp #Swap flag=0;
}
n-1 0 }
1
if (flag==1) break;
}
Total Time= Total Comparisons + Total Swapping }
= n-1 + 0 = n-1
Best Case Analysis TC = O(n)
Quick Recap of Last Week’s Learnings

TC of Bubble Sort

Worst Case Analysis

Average Case Analysis TC = O(n2)

Best Case Analysis

Optimized Best Case Analysis TC = O(n)

Quick Recap of Last Week’s Learnings

Understanding Selection Sort

Selects the smallest element from an

unsorted list in each iteration and places
that element at the beginning of the
unsorted list.
Quick Recap of Last Week’s Learnings

TC of Selection Sort
Total Time= Total Comparisons + Total Swapping
Worst Case Analysis

Average Case Analysis TC = O(n2)

Best Case Analysis

Can TC be further improved ?
Quick Recap of Last Week’s Learnings

Best Case Analysis of Selection Sort Pass #Comp #Swap

1 5 0
1 2 3 4 5 6
2 4 0
(List is already sorted)
3 3 0
TC = O(n2) 4 2 0

5 1 0
Can TC further improve? For Visualization
Quick Recap of Last Week’s Learnings

Best Case Analysis of Selection Sort

Pass #Comp #Swap 1 2 3 4 5 6

1 5 0

2 4 0 Swap = 0 will be responsible

3 3 0 for Sorted List?
4 2 0

5 1 0
Quick Recap of Last Week’s Learnings

Pass #Comp #Swap 1 2 5 6 4 3

1 5 0 1 2 5 6 4 3
2 4 0 1 2 5 6 4 3
3 3 1 1 2 3 6 4 5
4 2 1 1 2 3 4 6 5
5 1 1
1 2 3 4 5 6
Looking at No Swap Execution (Swap=0) in the first pass, we can not say that list is completely sorted.

Hence, by introducing flag concept (as similar to bubble sort) in the selection sorting algorithm, we can

not improve the time complexity.

Note: Kindly make a note while reading the Week -3 Lecture Slides (old)
Some Important Points

Stable Sorting Algorithm

A sorting algorithm is said to be stable if it maintains the relative order of
records in the case of equality of keys.
Some Important Points

In-place Sorting Algorithm

An in-place algorithm is an algorithm that does not need an extra space and produces an
output in the same memory that contains the data by transforming the input ‘in-place’.
However, a small constant extra space used for variables is allowed.
General Assessment Time
10
9
8
7
6
5
4
3
2
1
What is the Best Case Time Complexity of
Insertion Sort algorithm?

a) f(n)= O(nlogn)

b) f(n)= O(n2)

Which one is the c) f(n)= O(n)

correct answer? d) I am confused

Correct Answer is c
General Assessment Time
10
9
8
7
6
5
4
3
2
1
What is the Time Complexity of this
algorithm?
a) f(n)= O(n)
Algorithm print(n)
{ b) f(n)= O(n2)
for ( i=1; i>0; i=i*2)
c) f(n)= O(logn)
{
Which one is the printf(“This is number: %d”, i); d) I am confused
correct answer? }
}

Correct Answer is c
General Assessment Time
10
9
8
7
6
5
4
3
2
1
What is the Worst Case Time Complexity
of Shell Sort Algorithm (Gap Interval N/2…)?

a) f(n)= O(nlogn)

b) f(n)= O(n2)

Which one is the c) f(n)= O(n(logn)2)

d) I am confused
correct answer?

Correct Answer is c
Analyzing Time Complexity of Shell Sort

Worst Case Analysis

TC = O(n (logn)2)
Average Case Analysis

Best Case Analysis TC = O(nlogn)

Can TC further improve ?

Scope of Improvement

5 4 3 2 1

We have performed gap comparisons and shifting,

can comparison and shifting further minimize?
Visualization of Merge Sort Algorithm
Concept of Merge Sort Algorithm

• Unsorted array with n elements is divided into n subarrays, each

having one element, because a single element is always sorted in itself.

• Then, it repeatedly merges these subarrays, to produce new sorted

subarrays, and in the end, one complete sorted array is produced.

• It is based on Divide and Conquer Strategy

1. Divide the problem into multiple small problems.

2.Conquer the sub-problems by solving them. The idea

is to break down the problem into atomic sub-
problems, where they are actually solved.
3.Combine the solutions of the sub-problems to
find the solution of the actual problem.
Understanding Merge Sort Algorithm
p q r

Step 1
Divide the unsorted list into n sub-
lists, each comprising 1 element
(a list of 1 element is sorted).
Understanding Merge Sort Algorithm p q r

Step 2
Repeatedly merge sub-lists to produce
newly sorted sub-lists until there is only 1
sub-list remaining. This will be the sorted
list.
Understanding Merge Sort Algorithm p q r

2
In merge sort we follow the following steps

1. We take a variable p and point to the starting index of an array

and we take another variable r and point it to the last index of

an array.

2. Then we find the middle of the array using the formula (p +

r)/2 and mark the middle index as q, and break the array into
two subarrays, from p to q and from q + 1 to r index.

3. Then we divide these 2 subarrays again, just like we divided our

main array, and this process will continue until one element left
in the list.

4. Once we have divided the main array into subarrays with single
element, then we start merging the subarrays.
Understanding Merge Sort Algorithm p q r

In merge sort we follow the following steps 2

1. We take a variable p and point to the starting index of an array and we take
void mergeSort(int a[], int p, int r)
another variable r and point it to the last index of an array.
{
2. Then we find the middle of the array using the formula (p + r)/2 and mark the if(p < r)
middle index as q, and break the array into two subarrays, from p to q and {
from q + 1 to r index. q = (p + r) / 2; //division
mergeSort(a, p, q);
3. Then we divide these 2 subarrays again, just like we divided our main array and
mergeSort(a, q+1, r);
this process will continue.
merge(a, p, q, r);
4. Once we have divided the main array into subarrays with single element, then
}
we start merging the subarrays.
}
Understanding Merge Sort Algorithms

void mergeSort(int a[], int p, int r)

mergeSort(a, 0, 7) 2
{
if(p < r)
{ mergeSort(a, 0, 3)

q = (p + r) / 2; //division
mergeSort(a, p, q);
mergeSort(a, q+1, r); mergeSort(a, 0, 1)
mergeSort(a, 2, 3)
merge(a, p, q, r);
} merge(a, 2, 2,3)
merge(a, 0, 0,1)
} mergeSort(a, 0, 0) mergeSort(a, 1, 1)
(will not execute) (will not execute)
mergeSort(a, 2, 2) mergeSort(a, 3, 3)
(will not execute) (will not execute) ….. ….
Understanding Merge Concept in Merge Sort Algorithm

void merge(array, low, mid, high) // Copy the remaining elements of the list if there are any
{ while (j <= high)
i mid j high
i=low { mid j high
i
j= mid+1 temp[k] = a[j];
k= high j++;
1 2 3 4 5 6
while (i < =mid && j <= high) k++;
k
{ } k
if (a[i]<=a[j]) while (i <= mid)
{ {
temp[k] = a [i]; temp[k] = a[i];
mid i j high
i++, k++; i++;
} k++;
else { } k 4 5 6
temp[k] = a[j];
j++, k++; for(k=low; k<high; k++)
1 2 3
} {
} a[k] = temp[k];
}
}
Understanding Merge Sort Algorithm

void mergeSort(int a[], int p, int r) T(n)

{
if(p < r)
1
{
1
q = (p + r) / 2;
mergeSort(a, p, q); T(n/2)

mergeSort(a, q+1, r); T(n/2)

merge(a, p, q, r); n

}
} T(n)= 2T(n/2) + n +2

T(n)= 2T(n/2) + n
Understanding TC of Merge Sort Algorithms

Recurrence Relation
T(n)= 2T(n/2) + n for n>1
=1 for n=1

(Apply Master Theorem to solve this recurrence relations)

Time complexity of Merge Sort is O(nLogn) in all 3 cases (worst, average and best).
Understanding TC of Merge Sort Algorithms

void mergeSort(int a[], int p, int r)

{
if(p < r)
Number of operations
{ for merging 8 elements
q = (p + r) / 2; merge()

mergeSort(a, p, q); 3 times

merge
merge()
mergeSort(a, q+1, r); operation is
called to
merge(a, p, q, r); sort
merge() 8 elements
}
}

*Merge operation takes n times to sort and merge the n Total Time = merge() + merge() + merge() (3 Times)
element Total Time = 8+8+8
Understanding TC of Merge Sort Algorithms

void mergeSort(int a[], int p, int r)

{
Generalize it for n elements

if(p < r)
{ Number of operations
for merging n elements
q = (p + r) / 2; merge() n

mergeSort(a, p, q); logn times for

merge() n elements
mergeSort(a, q+1, r); . n
.
merge(a, p, q, r);
.
merge()
n
}
}

Total Time = merge() + merge() + merge() Total Time = n + n + n +…..+n (logn times)
+…..+merge() (logn times) TC=nlogn
Time and Space Complexity of Merge Sort

Worst Case Analysis

Average Case Analysis TC = O(n logn)

Best Case Analysis SC = O(n) (due to extra arrays)

Can TC further improve ?

Sorting In Place: No
Stable: Yes
*This is auxiliary Space Complexity
Non-Comparison based
Count Sort
Sorting Algorithm
Count Sort (Liner Sorting Algorithm)

Counting Sort Algorithm is an efficient sorting

algorithm that is based on non-comparison-
based strategy and can be used for sorting
elements within a specific range.
Count Sort (Liner Sorting Algorithm)

Step -1- Find out the maximum element from the given array.
Count Sort (Liner Sorting Algorithm)

Step -2- Create an Auxiliary or Temp or Count Array of length max+1 and initialize all
elements with 0. This array is used for storing the count of the elements in the array.

Count array
Count Sort (Liner Sorting Algorithm)

Step -3- Store the count of each element at their respective index in count array
For example: if the count of element 3 is 2 then, 2 is stored in the 3rd position of count array. If element "5" is not present in the

array, then 0 is stored in 5th position.

Original Array

Count array
Count Sort (Liner Sorting Algorithm)

Step -4 Store cumulative sum of the elements of the count array. It helps in placing
the elements into the correct index of the sorted array.
(Cumulative Sum= Count[i] = Count [i] + Count [i-1])

Count Array

0+1 1+2 2+3

Modified Count Array

Cumulative Sum
Count Sort (Liner Sorting Algorithm)

Step -5 Find the index of each element of the original array in the count array. This gives the
cumulative count. (Cumulative count helps placing the elements into the correct index of the
sorted array) Place the element at the correct index in output array.

Output Array Index start with 0

Output Array Index start with 1
Count Sort (Liner Sorting Algorithm)
Counting Sort Algorithm
countingSort(array, size)
{
max <- find largest element in array
Count
initialize count array with all zeros
array
for j <- 0 to size
find the total count of each unique element
and store the count at jth index in count array
for i <- 1 to max
find the cumulative sum and store it in count array itself
for j <- size down to 1
restore the elements to array
decrease count of each element restored by 1
}
Count Sort (Liner Sorting Algorithm)

countingSort(array, size) void countSort(int array[], int size)

{
int output[10], count[10];
{ Int max = array[0];
for (int i = 1; i < size; i++)
max <- find largest element in array {
if (array[i] > max)
initialize count array with all zeros max = array[i];
}
for (int i = 0; i <= max; ++i)
for j <- 0 to size {
count[i] = 0;
find the total count of each unique element }
for (int j = 0; j < size; j++)
and store the count at jth index in count array {
count[array[j]]++; // Store the count of each element
for i <- 1 to max }
for (int i = 1; i <= max; i++)
{
find the cumulative sum and store it in count array count[i] += count[i - 1]; // Store the cumulative count of each array
}
itself for (int i = size - 1; i >= 0; i--)
{
for j <- size down to 1 output[count[array[i]] - 1] = array[i]; // placing the original array element into the sorted output array

restore the elements to array count[array[i]]--;

}
for (int i = 0; i < size; i++)
decrease count of each element restored by 1 {
array[i] = output[i]; // Copy the sorted elements into original array
} }
}
Time Complexity of Count Sort Algorithm

countingSort(array, size)
There are mainly four main loops.
{
max <- find largest element in array
for-loop time of counting
initialize count array with all zeros
1st O(max)
for j <- 1 to size
find the total count of each unique element 2nd O(size)
and store the count at jth index in count array
3rd O(max)
for i <- 1 to max
4th O(size)
find the cumulative sum and store it in count array itself
for j <- size down to 1
restore the elements to array Overall complexity = O(max)+O(size)+O(max)+O(size)
decrease count of each element restored by 1 = O(max+size)
}
•Worst Case Complexity: O(n+k)
•Best Case Complexity: O(n+k)
•Average Case Complexity: O(n+k)

Note: Counting sort is a stable algorithm because the relative order of similar elements are not changed after sorting.
Any Queries?
Office MCub311
Discussion Time: 3-5 PM
Mob: +91-8586968801