Advanced Algorithms Analysis

Lecture-3
Sorting Algorithms

By

Yohannes Biadgligne

Yohannes Biadgligne Advanced Algorithms

 Objective of this Lecture
At the end of this chapter, you should be able to:
• Explain insertion sort and its analysis
• Explain bubble sort and its analysis
• Explain shell-sort and its analysis
• Explain radix sort and its analysis
• Trace the heapsort and Fix-Heap algorithms
• Explain the analysis of heapsort
• Explain merge sort and its analysis
• Explain quicksort and its analysis
• Explain external poly-phase merge sort and its analysis)
(presented

Yohannes Biadgligne Advanced Algorithms

 Assumptions
• In discussing sorting, we assume that
– All of our sort algorithms will sort the list into increasing
order based on the key value.
– Each of the values in the list is distinct, because the
presence of duplicates will not significantly change any of
the analyses that we do.

Yohannes Biadgligne Advanced Algorithms

 Introduction
• Because of the significant time savings of binary search
over sequential search,
– software designers will frequently choose to keep information
sorted so that searches can be done by binary or other non-
sequential methods.
• Sorting: Putting objects(lists) in order
• Sorting Algorithm: is an algorithm that accepts a list [x]
and tries to put the list in order
• The eight sorting algorithms discussed in this chapter
are only a sampling of the possible sorts, but they
exhibit a wide range of behaviors.

Yohannes Biadgligne Advanced Algorithms

 Introduction
• Insertion Sort: accomplishes sorting by inserting new elements into the
correct place in a list that is already sorted.
• Bubble sort: compares pairs of elements,
swapping those that are out of order, until the list is sorted.
• Shell sort: is a multi-pass sort that breaks the list into sub-lists that are
sorted, and on each successive pass the number of sub-lists is decreased
as their size is increased.
• Radix sort: is a multi-pass sort that separates the list into buckets, each
pass using a different part of the key.

Yohannes Biadgligne Advanced Algorithms

 Introduction
• Heapsort: builds a binary tree with list elements so
that each node has a value larger than its children.
– This places the largest element at the root so
that when it is removed and the heap is fixed, the next
largest element moves to the root. This is repeated until
all of the elements are back in the now-sorted list
• Merge sort: starts with two sorted lists and creates
one sorted list by merging these two.
• Quicksort: is a recursive sort that picks a pivot
element from the list and then subdivides the list
into two parts that contain the elements that are
smaller and larger than the pivot.
• External sort: works with lists that are so large
that they cannot be reasonably held in a computer’s
memory all at once.
Yohannes Biadgligne Advanced Algorithms
 Insertion Sort

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• The basic idea of insertion sort is that if you have a list that is sorted and
need to add a new element,
– the most efficient process is to put that new element
into the correct position instead of adding it anywhere and then resorting the entire
list.
• Accomplishes its task by considering that the first element of any list is
always a sorted list of size 1.
– We can create a two-element sorted list by
correctly inserting the second element of the list into the one element list containing
the first element.
– We can now insert the third element of the list into the
two-element sorted list.
– This process is repeated until all of the elements
have been put into the expanding sorted portion of the list.

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• Example

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• Idea: like sorting a hand of playing cards
– Start with an empty left hand and the cards facing down
on the table.
– Remove one card at a time from the table, and insert it
into the correct position in the left hand
• compare it with each of the cards already in the hand,
from right to left
– The cards held in the left hand are sorted
• these cards were originally the top cards of the pile on
the table

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• To insert 12, we need to make room for it by
moving first 36 and then 24.

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• The following algorithm carries out this process:

Yohannes Biadgligne Advanced Algorithms

 Insertion Sort
• This algorithm copies the next value to be inserted into new Element.
• It then makes space for this new element by moving all elements that are
larger one position
over in the array.
– This is done by the while loop.
– The last copy of the loop moved the element in position location + 1 to position
location + 2.
– This means that position location + 1 is available for the “new” element.

Yohannes Biadgligne Advanced Algorithms

 Worst–Case Analysis
• When we look at the inner while loop, we see that
the most work this loop will do is if the new element
to be added is smaller than all of the elements
already in the sorted part of the list.
– In this situation, the loop will stop when location
becomes 0.
– So, the most work the entire algorithm will do is in the
case where every new element is added to the front of
the list.
– For this to happen, the list must be in decreasing order
when we start.

Yohannes Biadgligne Advanced Algorithms

 Worst–Case Analysis
• Let’s look at how this input set will be handled.
• The first element to be inserted is the one in the second location of the list.
– This is compared to one other element at most.
– The second element inserted (that’s at location 3) will be compared to the two previous
elements, and
– The third element inserted will be compared to the three previous elements.
– In general, the ith element inserted will be compared to I previous elements.
– This process is repeated for N-1 elements.
– This means that the worst-case complexity for insertion sort is given by

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• Average-case analysis will be a two-step process.
– We first need to figure out the average number of
comparisons needed to move one element into place.
– Then determine the overall average number of operations
by using the first step result for all of the other elements.
• How many comparisons it takes to move the ith
element into position.
– Adding the ith element to the sorted part of the list does at
most i comparisons.
– How many positions is it possible to move the ith element
into?
• There are two possible positions for the first element to be added
—either in location 1 or location 2
• There are three possible positions for the second element to be
added—either in locations 1, 2, or 3
• It appears that there are i + 1 possible positions for the ith
element. We consider each of these equally likely.

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• How many comparisons does it take to get to each of these i + 1 possible positions?
– If we are adding the fourth element, and it goes into location 5, the first comparison will fail.
– If it goes into location 4, the first comparison will succeed, but the second will fail.
– If it goes into location 3, the first and second comparisons will succeed, but the third will fail.
– If it goes into location 2, the first, second, and
third comparisons will succeed, but the fourth will fail.
– If it goes into location 1, the first, second, third, and fourth comparisons will succeed, and there will
be no further comparisons because location will have become zero.
– This implies that for the ith element, it will do 1, 2, 3, . . ., i comparisons for locations i+1, i, i-1, . . ., 2, and it will do i comparisons for location
1.

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• The average number of comparisons to insert the ith element is given by the formula

• This is just the average amount of work to insert the ith element.

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• This now needs to be summed up for each of the 1 through N-1 elements that gets “added” to the list.
• The final average case result is given by the formula

• By application of Equation 1.11

• We get

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• By using Equations 1.9 and 1.10, we have

• Equations 1.15, 1.14, 1.20, and the last formula, we get

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• By applications of Equations 1.15, 1.14, 1.20, and the last formula, we get

• By using Equations 1.9 and 1.10, we have

Yohannes Biadgligne Advanced Algorithms

 End of insertion sort.

Yohannes Biadgligne Advanced Algorithms

 Bubble sort

Yohannes Biadgligne Advanced Algorithms

 Bubble sort
• The general idea is to allow smaller values to move toward the top of the list while larger values move to the bottom.
• There are a number of varieties of bubble sort. This section will deal with one of these, and the others will be left as exercises.
• It makes a number of passes through the list of elements.
• On each pass it compares adjacent element values. If they are out of order, they are swapped.
• It compares the elements in locations 1 and 2, then the elements in locations 2 and 3, then 3 and 4,
and so on, swapping those that are out of order.

Yohannes Biadgligne Advanced Algorithms

 Bubble sort
• On the first pass the largest element will be swapped with all of the remaining elements, moving to
the end of the list.
• The second pass, therefore, no longer needs to look at the last element in the list.
– It will move the second largest element down the list.
• The process continues with each additional pass moving one more of the larger values down in the
list.
• During all of this, the smaller values are also moving toward the front of the list.
• If on any pass there are no swaps, all of the elements are now in order and the algorithm can stop.
• It should be noted that each pass has the potential of moving a number of the elements closer to
their final position, even though only the largest element on that
pass is guaranteed to wind up in its final location.

Yohannes Biadgligne Advanced Algorithms

 Bubble sort
• Example:

Yohannes Biadgligne Advanced Algorithms

 Bubble sort
• The following algorithm carries out this bubble sort version:

Yohannes Biadgligne Advanced Algorithms

 Worst-Case Analysis
• Having the input data in reverse order will lead us to the worst case.
• If the largest value is first, it will be swapped with every other element down the list until it is in
the last position.
– At the start of the first pass, the second largest element was in the second position but, the first comparison
on the first pass swapped it into the first position of the list.
– At the start of the second pass, the second largest
element is now in the first position, and it will be swapped
with every other element in the list until it is in the second to last position.
– This gets repeated for every other element, and so the algorithm has to execute the for loop N-1 times.

Yohannes Biadgligne Advanced Algorithms

 Worst-Case Analysis
• How many comparisons are done in this worst case?
– We have said that the first pass will do N-1 comparisons
of adjacent values, and the second pass will do N-2 comparisons.
– Further examination will show that each successive pass reduces the number of comparisons by 1.
• This means that the worst case is given by the formula:

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• We have already said that in the worst case, there would be N-1 repetitions of the inner for loop.
• For the average case, we will assume that it is equally likely that on any of these passes there
will be no swaps done.
• We need to know how many comparisons are done in each of these possibilities.
– If we stop after one pass, we have done N-1 comparisons.
– If we stop after two passes, we have done N-1 + N-2
comparisons.
– For now, let’s say that C(i) will calculate how many
comparisons are done on the first i passes.
– Because the algorithm stops when there are no swaps done, the average case is found by looking at all of
the places bubble sort can stop.

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• This gives the equation:

• Where, C(i) is the number of comparisons done in the first i passes of the for loop before we stop.
• So, how many comparisons is this?
• C(i) is given by the equation

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• Substituting this back into the first equation gives the following:

Because N is a constant relative to i, by using Equations 1.11 and 1.14 we can get

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis
• Again using Equation 1.11 and some basic algebra, we get

• Now, we apply Equations 1.15 and 1.16 to get

Yohannes Biadgligne Advanced Algorithms

 Average-Case Analysis

Yohannes Biadgligne Advanced Algorithms

 End of Bubble Sort

Yohannes Biadgligne Advanced Algorithms

 Assignments

Yohannes Biadgligne Advanced Algorithms

 Presentation Assignment I (5%)
Sorting Presented by Date
Algorithm
Shell Sort Gezahgn & Esmael 9/03/2012

Radix Sort Amsalu & Getasew 9/03/2012

Heap Sort Demamu & 9/03/2012

Bethelehem
Merge Sort Yibeltal & Dires 9/03/2012
Quick Sort Bihonegn & Mebratu 16/03/2012
External poly Emebet & Amare 16/03/2012
phase merge
sort
Selection sort Abdirahman 16/03/2012
&Teklhiwot

Selection Samuel & Minasie 16/03/2012

(searching)
Yohannes Biadgligne Advanced Algorithms
 THE END.

Yohannes Biadgligne Advanced Algorithms