INTERNATIONAL ISLAMIC UNIVERSITY, ISLAMABAD

FACULTY OF BASIC AND APPLIED SCIENCES

BS in COMPUTER SCIENCE

Subject: Design & Analysis of Algorithms

Assignment: 01

Submitted To: Ma’am Sadia Saleem

Due Date: 9th Oct 2022

Group Members

Names Registration no.

Ramsha Nadeem 4223-FBAS/BSCS/F20

Hafsa Zubair 4224-FBAS/BSCS/F20

Zainab Abbasi 4226-FBAS/BSCS/F20

 1. BUBBLE SORT
 COST & COMPLEXITY:
 Example dry run:

 Pseudo code:
Input: Unsorted Values
Output: Sorted Values
for i  1 to length[A]
do for j  1 to length[A]-i
do if A[j] > A[j +1]
then exchange A[j]  A[j+1]
 Complexity:

COST NO. OF TIMES IT RUNS

C1 n-1
C2 Σn i = 1(n - ti)

C3 Σn i = 1(n - ti)

C4 Σn i = 1(n - ti)
Total Running Time of Bubble sort in all 3 sorts can be calculated by this general formula :
(T(n)) = C1 * (n – 1) + C2 * Σ n i = 1(n - ti) + C3 * Σ n i = 1(n - ti)+ C4 * Σ n i = 1(n - ti)
= n + (C2 + C3 + C4) Σ n i = 1(n - t i)
Where, Σ n i = 1(n - ti) = Σ n i = 1 (n) - Σn i = 1 (ti)

 Worst Case:
In the worst case, the array is sorted in reverse.. So we need to do (n-1) comparisons in the
first iteration, (n-2) in the second iteration, and so on till n-1 iterations. Hence, for the time
complexity of the bubble sort in the worst case ; ti = i
(T(n)) = n + (C2 + C3 + C4) (Σ n i=1 n) - (C2 + C3 + C4) (Σ n i = 1 ti)

(T(n)) = n + n2 – (( n + 1 ) ( n ))/ 2
(T(n)) = n + n2 – (n2 + n ) / 2
(T(n)) = O(n2)

 Best Case:
The best case is when the given array is already arranged or sorted. In this case, we check n-
1 elements to see if there is any need for swaps. Even when no swapping occurred we still
continue and complete n-1 iterations. Therefore, in the best scenario, the time complexity of
the standard bubble sort would be ,by taking ; ti = 1
(T(n)) = n + (C2 + C3 + C4) (Σ n i=1 n) - (C2 + C3 + C4) (Σ n i = 1 ti)

(T(n)) = n + n2 – n
(T(n)) = O(n2)

 Average Case:
The average case would be when the input array is partially sorted. In this case too, we
check n-1 elements to see if there is any need for swaps. Even when no swapping occurred
we still continue and complete n-1 iterations.
Therefore, in the average scenario, the time complexity of the standard bubble sort would
also be ,by letting ; ti = (i+1)/2
(T(n)) = n + (C2 + C3 + C4) (Σ n i=1 n) - (C2 + C3 + C4) (Σ n i = 1 ti)
 (T(n)) = n + n2 – (( n + 1 ) ( n )) / 4 + n
(T(n)) = 2n + n2 – (n2 + n ) / 4
(T(n)) = O(n2)

 OPTIMIZATION:
Bubble Sort is algorithm which compares two adjacent elements and swap them if they're not
within the right order. To sort the complete array, the array is traversed n-1 time for an array
having n elements. Within the first pass the biggest element moves to the last position (if the
sorting is in ascending order) and within the 2nd pass, we'll consider (n-1) elements
only because last position already has largest element. Like that, in the second pass, the 2nd
largest number is moved to n-1 position and so on. After (n-1) passes, (n-1) elements are moved
to their proper positions. The last element should be the smallest.

 Original Algorithm:

Alg: BubbleSort(array)

for i=0 to n-1

for j=0 to n-1-i

if array[j] > array[j+1] then

swap array[j] and array[j+1]

 Problem with this Sort:

Now the matter with this kind is that if there's an array, let’s say there are 5 elements within
the array, so in step with this sorting technique, there'll be 4 passes, but if that's an array that
may be sorted just after 2 passes so there'll be unnecessary 2 passes.
Just like that if there's an array that's already sorted then there'll be no swapping because
adjacent elements are always in order, but still we'll continue with the passes and there'll still be
(n-1) passes.
 How to Optimize this Sort?

For an array let’s say that it's already sorted, there shouldn't be any passes. For this we are able
to have a flag variable which made true before each pass and is made false when a swapping is
performed. If there's no swapping it'll break the loop and save time.

 Optimized Algorithm:

Alg: BubbleSort(array)

flag = false

for i=0 to n-1

for j=0 to n-1-i

if array[j] > array[j+1] then

flag = true

swap array[j] and array[j+1]

if !flag then

break

 Time Complexity:

Now this optimized sort will be O(n) for best case (array which is already sorted) but for the
worst case (array which is not already sorted) it will still be O(n2).

 APPLICATIONS:

 When you must write some sort of code yourself it is easier to write a code for bubble sort
algorithm as compared to other sorting algorithms. Because it is easier and simpler and
does not involve any complexity.
 Another example of bubble sort algorithm would be a small size embedded device where
you have to keep the code size as small as possible.

 For detecting very small errors.

 Real life example: Before the teacher, who is tasked with lining you all up in increasing
order of height, you are arranged in a random order. Here, the bubble sort method is
helpful. The height of each person is a component of the list in this instance. The pupils
gradually begin standing more formally with each pass the teacher gives them until they
are all standing according to height.
 Bubble sort is not really recommended to be used as it applies for niche and
specific conditions.

2. SELECTION SORT
 COST & COMPLEXITY:
 Example dry run:
 Pseudo code:
Input: Unsorted Values
Output: Sorted Values

(n-tj)

(n-tj)

(n-tj)

 Complexity:
Total Running Time of Selection sort in all 3 sorts can be calculated by this general formula :
(T(n)) = C1 * 1 + C2 * (n – 1) + C3 * ( n - 1 ) + (C4 + C5 + C6 )* Σ n - 1j = 1( n – tj ) + C7 * ( n -
1)
(T(n)) = C1*1 + (C2 + C3 + C7)*( n - 1 )+(C4 + C5 + C6 )* (Σ n - 1j = 1 n - Σ n - 1j = 1 tj )

 Worst Case:
When values are in reverse order, worst case occurs. Here we have to compare every element
(n-1 values) with every other element in the array ((n-j) values). So, the time complexity for
worst case is: tj = j
(T(n)) = C1*1 + (C2 + C3 + C7)*( n - 1 )+(C4 + C5 + C6 )* (n2 - [( n - 1 )( n )] / 2)
(T(n)) = n + n2 – (n2 – n)/2
(T(n)) = O( n2 )

 Best Case:
Best case occurs when elements are already sorted as input. But still we have to do
comparison of n-1 number of elements (it excludes the last one as it will be sorted
automatically) with all other values until the array ends. So, the time complexity for best
case is: tj = 1
(T(n)) = C1*1 + (C2 + C3 + C7)*( n - 1 )+(C4 + C5 + C6 )* (n2 – n)
(T(n)) = n + n2 – n
(T(n)) = O( n2 )
 Average Case:
This case occurs when input values are partially sorted. Till the array stops, comparison for
finding minimum value from all unsorted elements (n-j) continues for every element of array
(n-1 values), then tj = (j-1)/2
(T(n)) = C1*1+(C2 + C3 + C7)*( n - 1 )+(C4 + C5 + C6 )*(n2 – (1/2 *[(n - 1)(n)]/2)
(T(n)) = n + n2 – (n2 – n)/4
(T(n)) = O( n2 )

 OPTIMIZATION:
Selection sort is a simple algorithm. This algorithm split the list into two parts, the sorted part at
the left end and the unsorted part at the right end. At the beginning there is no sorted part
because the whole list is unsorted. In the first pass the smallest element is chosen from the
unsorted list and swapped with the leftmost element or the element at first index, and
that element becomes part of the sorted array. This process continues moving unsorted list
boundary by one element to the right.

 Original Algorithm:

Alg: SelectionSort(array)

for j=1 to n-1

minimum = j

for i=j+1 to n

if array[i] < array[minimum] then

minimum = i

swap array[j] and array[minimum]

 Problem with this Sort:

Now the matter with this algorithm is that let’s assume there it performs same number of
comparisons regardless of whether the array is already sorted or not. So it has way of finding
whether the list is in order before the whole algorithm is executed.
 How to Optimize this Sort?

The way this algorithm works is that both the biggest and smallest element of an array is found
in a very single iteration and place them in their respective positions, at that time the second
largest and smallest element of an array is found and are placed in their correct locations and
then on. This sorting process executes from both ends of an array thus saving half the iterations
compared to the first selection sort algorithm.

 Optimized Algorithm:

Alg: SelectionSort(array)

k=0

for i=n-1 to k

indexofLarge = indexofSmall = k

for j=k+1 to i

if array[j] > array[indexofLarge] then

indexofLarge = j

if array[j] < array[indexofSmall] then

indexofSmall = j

Large = array[indexofLarge]

Small = array[indexofSmall]

array[indexofLarge] = array[i]

array[indexofSmall] = array[k]

if indexofLarge == k then

temp = array[i]

array[i] = Large

array[indexofSmall] = temp
 else

array[i] = Large

temp = array[k]

array[k] = Small

array[indexofLarge] = temp

else

array[k] = Small

k = k+1

 Time Complexity:

Now this algorithm is still in order O(n2) but it performs less number of iterations and saves
half the time of execution (down to almost 50%)

 APPLICATIONS:
 When memory write could be a costly operation in terms of your time or hardware
durability, selection sort is employed.
 This is important if writes are significantly costlier than reads, like with EEPROM or non-
volatile storage, where every write lessens the lifespan of the memory.

 Real life example: Consider you have 8 shoes pairs with sizes 6 to 13. You find them not
in order. Their arrangement from smallest to largest will be an example of selection sort.
 3. INSERTION SORT
 COST & COMPLEXITY:
 Example dry run:

 Pseudo code:
Input: Unsorted Array
Output: Sorted Array
Loop i ← 1 to n //value at 0 index is already sorted
key ← A[i] //assigning value of A[i] to the variable ”key”
j←i-1
while j > 0 and A[j] > key //j>0 is stopping queue and A[j] > key is for comparing
values
A[j + 1] ← A[j]
j←j-1
end while
A[j + 1] = key //placing smallest value of unsorted array into ordered part of array
end for
 Complexity:

COST NO. OF TIMES IT RUNS

C1 n

C2 n-1

C3 n-1

C4 Σ n - 1j = 1(tj)

C5 Σ n - 1j = 1 (tj - 1)

C6 Σ n - 1j = 1(tj - 1)

C8 n-1

Then Total Running Time of Insertion sort in all 3 sorts can be calculated by this general
formula :
(T(n)) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4 * Σ n - 1j = 1( t j ) + ( C5 + C6 ) * Σ n - 1j = 1 ( t j - 1)
+ C8 * ( n - 1 )

 Worst Case:
When the array is sorted in descending order, then tj = j
Therefore,
T( n ) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4 * ( n - 1 ) ( n ) / 2 + (C5 + C6 ) * ( ( n - 1 ) (n ) / 2
– (n -1)) + C8 * ( n - 1 )
T( n ) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4 * ( n - 1 ) ( n ) / 2 + (C5 + C6 ) * ( ( n2 - 3n + 2 ) /
2) + C8 * ( n - 1 )
T(n) = C * ( n 2) or O( n2 )

 Best Case:
When the array is already sorted, then tj = 1
Therefore,
T( n ) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4 * n + C8 * ( n - 1 )
T(n) = C * ( n ) or O( n )
 Average Case:
Let's consider that for calculating average case :tj = (j-1)/2
Therefore,
T( n ) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4/2 * ([( n - 1 ) ( n )] / 2– (n -1)) + (C5 + C6 )/2 *
( ( n - 1 ) (n - 2 ) / 2 – (n -1) ) + C8 * ( n - 1 )

T( n ) = C1 * n + ( C2 + C3 ) * ( n - 1 ) + C4/2 *( ( n2 - 3n + 2 ) / 2)) + (C5 + C6 )/2 * ( ( n2 –

7n + 6 ) / 2 ) + C8 * ( n - 1 )

T(n) = C * ( n 2) or O( n2 )

 OPTIMIZATION:
Insertion sort is an algorithm that places an unsorted element in it’s proper position in each
iteration. during this we assume that the primary element is already sorted so we take the
second element and store it in key element and compare it with first element, next we take third
element and compare it with left elements, if there's an element smaller than it then place it
before that element otherwise place it at the start. This process is repeated until it reach the tip
of the array

 Original Algorithm:

Alg: InsertionSort(array)

for j=2 to n

key = array[j]

i = j-1

while i>0 and array[i]>key

array[i+1] = array[i]

i = i-1

array[i+1] = key
 Problem with this Sort:

Now the matter with this algorithm is that suppose we've the array that's in decreasing order so
to insert the last element there would be n-1 comparisons and should be n-1 swaps and to insert
second last element there would be n-2 comparisons and almost n-2 swaps and then on. So this
could be quite inefficient.

 How to Optimize this Sort?

To reduce the amount of comparisons, there's a method called binary insertion sort which
divides the array into two subarrays, sorted and unsorted. the primary element of the array
is within the sorted subarray, and also the rest are within the unsorted array.
We then iterate from the second element to the last element. for every iteration, we
make this element our “key” which we'll must add in sorted array.
To do this, we first use binary search on the sorted subarray to search out the position of the
element that's just greater than the key. Then right shift all elements from that position to i-
1 then make that position the key.

 Optimized Algorithm:

Alg: BinarySearch(array,n, key)

Left = 0

Right = n

while Left < Right:

mid = (Left + Right)/2

if array[mid] <= key then

Left = mid + 1

else

Right = mid

The above function will return index of element just greater than key in array from 0 to n
BinaryInsertionSort(array)

for i = 1 to n

key = array[i]

pos = BinarySearch(array, i, key)

j=i

while j > pos

array[j] = array[j-1]

j = j-1

array[pos] = key

 Time Complexity:

Although in worst and average case its complexity will be O(n2) but still it will run faster than
original sort as it makes less comparisons. In best case however, it would be O(n Log n). So
this sort will work a lot better for array of small size.

 APPLICATIONS:
 Insertion sort can be used for game programming for high score list sorting. The score list
starts from no score or having already sorted list of previous tries, after that every time we
play, we have one unsorted number in list. That one number is quickly sorted into list by
insertion sort.

 The results of students from highest to lowest marks are stored in school’s computer record
by insertion sort. Every time the record of a student is entered, his/her marks are compared
with the sorted ones to place it in correct position.

 Employees data in company's record arranged according to their qualification. New

employee’s data is placed with others record by using insertion sort.
 Real life example: Tailors arrange shirts in a cupboard, they constantly keep them
in sorted order of size and as a result insert new shirts at the proper position in no
time by moving different shirts forward to keep the right location for a new shirt.

 COMPARISON BETWEEN ALL SORTS

1) Selection sort
Selection sort is a sorting method that uses comparisons. The input array is split into two
subarrays:
1. sorted subarray that starts out empty
2. Subarray that is not sorted and initially contains all the elements

 Time Complexity:

 Best-case time complexity of selection sort is 0(N^2) because to find the minimum value
at each iteration we’ll have to transverse the unsorted array.
 Worst case time complexity is 0(N^2) because at each iteration the unsorted array will
have to be transverse.
 Average case time complexity is 0(N^2)

 Space complexity: space complexity is 0(1), since no extra space required.

 Stability: selection sort is not a stable algorithm, if made stable then space complexity
increases to 0(N).
 Selection sort works similarly every kind of arrays i.e. data sets.
 Better applied on linked lists
 Performs well on small data sets
  It works inefficiently when dealing with huge number of lists

2) Bubble sort
Another comparison-based sorting algorithm is bubble sort. It makes numerous runs across the
input array, swapping neighboring elements that are out of order in each iteration (ascending or
descending). The process doesn't stop until the entire array is sorted. It is named bubble sort
because data bubbles to the top of data set.

 Time complexity:

 best case time complexity is 0(N) , this will be the case when we’ll have an already sorted
array.
 Worst case time complexity is 0(N^2) when the array will be in reverse order.
 Average case time complexity is 0(N^2)

 Space complexity: space complexity is 0(1), since no extra space required.

 Stability: Bubble sort is a stable algorithm
 Works well when given partially sorted array.
 Used for detecting errors in partially sorted arrays
 Bubble sort is simple and quite easy to implement. It is simple to describe to computer.
 Does not deal with huge number of items in list
 Comparatively slow algorithm

3) Insertion sort
Another comparison-based sorting technique is insertion sort. Additionally, it keeps track of
two subarrays: an unsorted subarray that holds the other elements of the array and a sorted
subarray that initially only contains the first element.
With this approach, each element is individually selected from the unsorted section and placed
in the sorted part at the appropriate location.

 Time complexity:

 Best case time complexity is 0(N) . this is the case when the array is already sorted.
 Worst case time complexity is 0(N^2), this occurs when we have reverse sorted
array.
  Average case time complexity is 0(N^2)

 Space complexity: space complexity is 0(1), since no extra space required.

 Stability: insertion sort is stable algorithm
 Works well when given partially sorted array.
 Particularly useful for small number of items in a list
 Easily computed.
 Inefficient for larger value of N.
 It has the fastest best case running time .
 The benefit of insertion sort is that it only scans the necessary number of items to place
the k+1st element.
Insertion sort is the most efficient sorting algorithm among bubble sort and selection sort.

REFERENCES
 https://www.baeldung.com/cs/bubble-sort-time-complexity
 https://en.wikipedia.org/wiki/Selection_sort
 https://iq.opengenus.org/insertion-sort-analysis/
 https://www.crio.do/blog/insertion-sort/
 https://www.quora.com/Where-is-the-insertion-sort-algorithm-used
 https://www.quora.com/What-are-some-applications-of-the-bubble-sort-algorithm
 https://www.quora.com/What-are-the-applications-of-bubble-sorting-in-computer-graphics
 https://www.geeksforgeeks.org/comparison-among-bubble-sort-selection-sort-and-
insertion-sort/
 https://www.geeksforgeeks.org/sorting-algorithm-slightly-improves-selection-sort/
 https://stackoverflow.com/questions/18293913/how-to-improve-selection-sort
 http://ijcset.com/docs/IJCSET13-04-05-068.pdf
 https://faculty.cs.niu.edu/~mcmahon/CS241/Notes/Sorting_Algorithms/insertion_sort.
html
 https://www.geeksforgeeks.org/binary-insertion-sort/
 https://www.interviewkickstart.com/learn/binary-insertion-sort
 https://www.programiz.com/dsa/bubble-sort
 https://www.geeksforgeeks.org/bubble-sort/
 https://www.includehelp.com/c-programs/implement-optimized-bubble-sort.aspx