Complexity of Search Algorithms

Search Algorithms
 Two Approaches:
 Sequential Search
 Sequential Search on Unsorted Data
 Sequential Search on Sorted Data
 Binary Search
Sequential Search
 Every element in the array will be examined
sequentially, starting from the first element.
 The process will be repeated until the last element
of the array or until the searched data is found.
Sequential Search Implementation
int SequenceSearch(int search_key,int array [ ],int array_size )
{
int index =-1; //-1 means record is not found
operation 1
for (int p = 1 0; p < n+1
operation array_size;
times p++ )
n times
{
if ( search_key == array[p] ){ n times
index = p;//assign current array index
2 times
break;
}//end if
} //end for
return index;
} //end function
1+1+n+1 +n+n+2
=3n+5
Sequential Search Analysis
 Searching time for sequential search is O(n).
 If the searched key is located at the end of the list or
the key is not found, then the loop will be repeated
based on the number of element in the list, O(n).
 If the list can be found at index 0, then searching time
is, O(1).
Improvement of Basic Sequential
Search
 The efficiency of basic search technique can be
improved by searching on a sorted list.
 For searching on ascending list, the search key will be
compared one by one until :
1.the searched key is found.
2.Or until the searched key value is smaller than the
item compared in the list.
Sequential Search on Sorted Data
Implementation
int SortedSeqSearch (int search_key,int array[ ],int array_size)
{
int index = -1; //-1 means record not found
for (int p = 0; p < array_size; p++ )
{
if (search_key < array [p])
break;
else if (search_key == array[p])
{
index = p;
break;
}
}
return index;
}
Sorted Sequential Search Analysis
 Searching time for sorted sequential search is O(n)
only if the searched key is located at the end of the
list or the key is greater than (or less than for
descending arrangement) all the elements in the
list.
Binary Search
 Consider a list in ascending sorted order.
 Search process is started at the middle of the list, then
the algorithm determine which half the item is in
(because the list is sorted).
 It divides the working range in half with a single test.
 By repeating the procedure, the result is an efficient
search algorithm  O(log2 n).
Binary Search
 Starts by comparing the search key with the element at the
middle
 If the value matches, it will be return to the calling
function (index = MIDDLE)
 If the search key < the middle element, search will be
focused on the elements from the first element to the
element before the middle element (MIDDLE -1)
 If the search key > the middle element, search will only be
focused on the elements from after the middle element
(MIDDLE +1) to the last element.
 Search is repeated until the searched key is found or the
last element in the subset is traversed (LEFT>RIGHT ).
Binary Search Implementation
int search(int a[], int beg, int end, int item)
{
if (beg>end)
return -1; Only 1 comparison is
int mid = (beg + end)/2; required
if(a[mid]==item)
return mid;
else if(item < a[mid])
return search(a,beg,mid-1,item); 1 comparison + T(n/2)
else
return search(a,mid+1,end,item);

}
 If array is of size 8 then we can divide it into 3 equal
halves as
23=8
If array is of size n then we can divide it into k equal
halves as
2k=n (1)
T(1)=1 means constant time (2)
only one comparison is required
T(n)=2+T(n/2)
=2+[2+T(n/4)]
=4+[T(n/4)]
=4+[2+T(n/8)]
=6+[T(n/8)]
= 

=2*k +[T(n/2k) (3)

Applying log on both sides of equation (1)
log(2k)=log(n)
k log(2)=log(n)
k=log(n)/log(2) (4)
in logarithm log(n)/log(2)=log2n
Therefore (3) becomes
k= log2n (5)
Applying (1) and (5) , equation(3) becomes
2*log2n +[T(n/n)]
2*log2n +[T(1)]
2*log2n +1
Ignoring the constants the order of binary search becomes

O(log2n)
Bubble Sort
 Go through multiple passes over the array.
 In every pass:
 Compare adjacent elements in the list.
 Exchange the elements if they are out of order.
 Each pass moves the largest (or smallest) elements to
the end of the array
 Repeating this process in several passes eventually
sorts the array into ascending (or descending) order.
 Bubble sort complexity is O(n2) and only suitable to
sort array with small size of data.
Bubble Sort Implementation
void BubbleSort(int data[], int listSize)
{
int pass, tempValue;
for ( pass = 1;pass < listSize; pass++ )
{
for (int x = 0; x < listSize - pass; x++)
if ( data[x]>data[x+1] )
{
tempValue = data[x];
data[x] = data[x+1];
data[x+1] = tempValue;
}
}
}
Bubble Sort Analysis
 The number of comparison between elements and the number of
exchange between elements determine the efficiency of Bubble
Sort algorithm.
 Generally, the number of comparisons between elements in
Bubble Sort can be stated as follows:
(n-1)+(n-2)+….+2+1=n(n-1)/2  O(n2)
 In any cases, (worst case, best case or average case) the number of
comparisons between elements is the same.
 The difference can be seen in the number of swapping elements.
Worst case has maximum number of swapping, while best case
has no swapping since all data is already in the right position.
 If the list is already sorted, the next pass shouldn't be continued
and the sorting process should stop.
Improved Bubble Sort
void BubbleSort(int data[], int listSize)
{
int pass, tempValue;
int sorted = 0;
int pass=1;
while(pass<listSize && !sorted)
{
sorted=1;
for (int x = 0; x < listSize - pass; x++)
if ( data[x]>data[x+1] )
{
tempValue = data[x];
data[x] = data[x+1];
data[x+1] = tempValue;
sorted =0;
}
}pass++;
}
Improved Bubble Sort Analysis
 Complexity is measured based on time consuming
operations to compare and swap elements.
 Number of comparisons
 a for loop embedded inside a while loop
 Worst Case (n‐1)+(n‐2)+(n‐3) …+1 , or O(n2)
 Best Case – (n‐1) or O(n)
 Number of Swaps
 inside a conditional ‐> #swaps data dependent !!
 Best Case 0, or O(1)
 Worst Case (n‐1)+(n‐2)+(n‐3) …+1 , or O(n2)
SELECTION SORT
The Selectionsort Algorithm
 Start by
finding the 70
smallest
60
entry.
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
 Start by
finding the 70
smallest
60
entry.
50
 Swap the
40
smallest
30
entry with
the first 20

entry. 10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
 Start by
finding the 70
smallest
60
entry.
50
 Swap the
40
smallest
30
entry with
the first 20

entry. 10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
 Part of the 60
array is now
50
sorted.
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
 Find the 60
smallest
50
element in
40
the unsorted
side. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
 Find the 60
smallest
50
element in
40
the unsorted
side. 30
20
 Swap with
the front of 10

the unsorted 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
side.
The Selectionsort Algorithm
Sorted side Unsorted side

70
 We have 60
increased the
50
size of the
sorted side by 40
one element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
 The process 60
continues... Smallest
50 from
40 unsorted

30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
 The process 60
continues...
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side
is bigger Sorted side Unsorted side

70
 The process 60
continues...
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The Selectionsort Algorithm
 The process Sorted side Unsorted side

keeps adding one 70

more number to
60
the sorted side.
50
 The sorted side
40
has the smallest
30
numbers,
arranged from 20

small to large. 10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The Selectionsort Algorithm
 We can stop Sorted side Unsorted side

when the 70
unsorted side
60
has just one
50
number, since
40
that number
must be the 30

largest number. 20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The Selectionsort Algorithm
 The array is now
sorted. 70
 We repeatedly 60
selected the 50
smallest
40
element, and
30
moved this
element to the 20

front of the 10

unsorted side. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
Selection Sort
 Strategy
 Choose the largest/smallest item in the array and place
the item in its correct place
 Choose the next largest/next smallest item in the array
and place the item in its correct place.
 Repeat the process until all items are sorted.
 Does not depend on the initial arrangement of the
data
 Only appropriate for small n ‐ O(n2) algorithm
Selection Sort Implementation
void selectionSort(int Data[], int size)
{
for(i=0;i<size-1;i++)
{
int minIndex = i;
int minval=Data[i];
for (int p=i+1;p <size; p++)
{ if (Data[p]< Data[minIndex])
{minval = array[p];
minIndex = p;}
}
Data[minIndex]=Data[i];
Data[i]=minval;}
}
Let size of array is 5
Value of i No of No of swaps
comparisons
0 4 1
1 3 1
2 2 1
3 1 1

# of comparisons= (n-1)+(n-2)+(n-3)+….+1
=n(n-1)/2
= n2
# of swaps= n-1
Selection Sort Analysis
 For an array with size n, the external loop will iterate from 1 to n-1.
 n-1  O(n)
 For each iteration, to find the minimum number in subarray,
 Therefore the total comparison for Selection Sort for all iterations is
(n‐1) + (n‐2) + ….. 2 + 1.
 Generally, the number of comparisons between elements in Selection
Sort can be stated as:
 (n-1)+(n-2)+….+2+1=n(n-1)/2O(n2)
 In any case of Selection Sort (worst case, best case or average case) the
number of comparisons between elements is the same.
INSERTION SORT
Insertion Sort
 Strategy
 Take multiple passes over the array
 Partition the array into two regions: sorted and unsorted
 Take each item from the unsorted region and insert it into its
correct order in the sorted region
 Find the next unsorted element and insert it in correct place,
relative to the ones already sorted.
 Appropriate for small arrays
The Insertionsort Algorithm
Sorted side Unsorted side

 The sorted 70
side starts 60
with just the 50
first element, 40
which is not 30
necessarily
20
the smallest
10
element.
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

 The sorted 70
side grows by 60
taking the 50
front element 40
from the 30
unsorted
20
side...
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

 ...and 70
inserting it in 60
the place that 50
keeps the 40
sorted side 30
arranged
20
from small to
10
large.
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

 In this 70
example, the 60
new element 50
goes in front 40
of the 30
element that
20
was already
10
in the sorted
0
side. [1] [2] [3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]
The Insertionsort Algorithm
Sorted side Unsorted side

 Sometimes 70
we are lucky 60
and the new 50
inserted item 40
doesn't need 30
to move at
20
all.
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

 Sometimes 70
we are lucky 60
twice in a 50
row. 40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How to Insert One Element
Copy the new Sorted side Unsorted side
element to a
70
separate
location. 60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Shift
elements in
70
the sorted
side, creating 60

an open 50
space for the 40
new element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Shift
elements in
70
the sorted
side, creating 60

an open 50
space for the 40
new element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Continue
shifting
70
elements...
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Continue
shifting
70
elements...
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
...until you
reach the
70
location for
the new 60

element. 50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Copy the new Sorted side Unsorted side
element back
70
into the array,
at the correct 60

location. 50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 The last Sorted side Unsorted side

element 70
must also be
60
inserted.
50
Start by
40
copying it...
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How many
shifts will occur 70
before we copy
60
this element
50
back into the
array? 40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
  Four items
are shifted.
70
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
  Four items
are shifted.
70
And then 60
the element is
50
copied back
40
into the array.
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
Insertion Sort Implementation
void insertionSort(int data[],int n)
{
int item;
int pass, insertIndex;
for(pass=1;pass<n;pass++)
{ item = data[pass];
insertIndex = pass;
while((insertIndex >0)&&(data[insertIndex-1]>item))
{
data[insertIndex]= data[insertIndex -1];
insertIndex --;
}
data[insertIndex] = item;
}
Value of pass No of No of Total
comparisons movements
1 1 1 2
2 2 2 4
3 3 3 6
4 4 4 8
n-1 n-1 n-1 2(n-1)

=2+4+6+8….+2(n-1)
=2(1+2+3+4+…n-1)
=2[n(n-1)/2]
=n(n-1)
=O(n2)
 Insertion Sort Analysis
 For best case:
 The number of comparison is n‐1 which gives linear time
complexity  O(n)
 The number of swaps are zero, which gives constant complexity
 O(1)
 Worst case for insertion sort is when we have totally unsorted
data. In each pass, the number of iteration for while loop is
maximum.
 The number of comparisons are: (n-1)+(n-2) + …. + 2 + 1 = n(n-
1)/2  O(n2)
 The number of swaps are: (n-1)+(n-2) + …. + 2 + 1 = n(n-1)/2 
O(n2)