COMP2230 Introduction to

Algorithmics

Lecture 6

A/Prof Ljiljana Brankovic
Lecture Overview
Revision from Lecture 3:
Analysis of Recursive Algorithms

Divide and Conquer
Text, Chapter 5, Sections 5.1 and 5.2

Sorting Algorithms
Text, Chapter 6

Next week: Sorting Algorithms, Text, Chapter 6;
Greedy Algorithms, Text, Chapter 7

Revision from Lecture 3:
Analysis of Recursive Algorithms
We shall not try to calculate the exact time needed to
execute an algorithm; this would be very difficult to
do, and it would depend on the implementation,
platform, etc.

We shall only be estimating the time needed for
algorithm execution, as a function of the size of the
input.

We shall estimate the running time by counting some
dominant instructions.

A barometer instruction is an instruction that is
executed at least as often as any other instruction in
the algorithm.

Example 2.4.3
example(n) {
if (n == 1)
return
for i = 1 to n
x = x + 1
example(n/2)
}


Recurrence relation:
C
1
= 0,
C
n
= n + C
n/2
, n>1
Example: Towers of Hanoi
After creating the world, God set on Earth 3 diamond rods and 64
golden rings, all of different size. All the rings were initially on
the first rod, in order of size, the smallest at the top. God also
created a monastery nearby where monks task in life is to
transfer all the rings onto the second rod; they are only allowed
to move a single ring from one rod to another at the time, and a
ring can never be placed on top of another smaller ring.
According to the legend, when monks have finished their task,
the world will come to an end.

If monks move one ring per second and never stop, it will take
them more that 500,000 million years to finish the job (more
than 25 times the estimated age of the universe!).

Example: Towers of Hanoi
The following is a way to move 3 rings from rod
1 to rod 2.



Example: Towers of Hanoi
The following is an algorithm that moves m
rings from rod i to rod j.

Hanoi (m,i,j){
\\Moves the m smallest rings from rod i to rod j
if m > 0 then {
Hanoi(m-1,i,6-i-j)
write i j
Hanoi(m-1,6-i-j,j)
}
}

Example: Towers of Hanoi
Recurrence relation:



Or, equivalently,
t
0
= 0
t
n
= 2t
n-1
+ 1, n > 0

=
+
=
0 if , 0
otherwise , 1 ) 1 ( 2
) (
m
m t
m t
Example: Fibonacci sequence
Suppose that at the beginning of the year there is one pair of
rabbits and that every month each pair produces a new pair
that becomes productive after one month. Suppose further
that no deaths occur. Let a
n
denote the number of rabbits
at the end of nth month. Show that a
n
= f
n+1
, n > 1.

Months Pairs Comment
1 1 The pair did not multiply because it is not productive
yet.
2 2 The first pair became productive and multiplied
3 3 Only first pair multiplied

n-2 a
n-2

n-1 a
n-1

n a
n-1
+
a
n-2

Each pair that was alive 2 months ago reproduced
Example: Fibonacci sequence
Fibonacci(n){
\\Calculates the n
th
Fibonacci number
if n<3 return 1
else return Fibonacci(n-1)+Fibonacci(n-2)
}

Write a recurrence relation!
t
0
= 0
t
n
= t
n-1
+ t
n-2
+ 1, n > 0
More Examples
Factorial(n){
if n=0 return 1
else return Factorial(n-1)n
}

Write a recurrence relation!

t
0
= 0
t
n
= t
n-1
+ 1, n > 0


More Examples

NumberOfBinaryDigits(n){
if n=1 return 1
else return NumberOfBinaryDigits(n/2)+1
}

Write a recurrence relation!

t
0
= 0
t
n
= t
n/2
+ 1, n > 0
Divide and Conquer
Divide and Conquer is a basic algorithmic design
technique that uses recursion to split large problems
into smaller sub-problems that can be solved easily.

Works with two basic steps:
If the problem is small enough, solve it.
Otherwise split the problem into smaller instances of
the same problem and recurse.

Not every problem can be solved with this
technique.
Divide and Conquer


Subproblem1
of size n/2
Merge the two solutions into
a solution of the original problem
Find a solution to
subproblem1
Find a solution to
subproblem2
Subproblem2
of size n/2
Problem
of size n
Divide and Conquer
In general, the problem will be divided into b
subproblems of size n/b and a of these
problems will need to be solved; f(n) is the
time required to merge the solutions to
subproblems into a solution of the original
problem.

T(n) = aT(n/b) + f(n)


Divide and Conquer




T(n) = aT(n/b) + f(n)
To solve this recurrence relation, we use Master
Theorem:

If T(n) = aT(n/b) + f(n) and f(n) = O(n
k
) then

O (n
k
) if a < b
k

T(n) = O(n
k
log n) if a = b
k

O(n
log
b
a
) if a > b
k

Tiling a Deficient Plane

Imagine we have a set of L shaped tiles and a
plane with one square missing.
How might we use a D & C approach to tile the
plane?
Tiling a Deficient Plane
Algorithm 5.1.4 Tiling a Deficient Board with
Trominoes
This algorithm constructs a tiling by trominoes of a deficient nn board where
n is a power of 2.
Input Parameters: n, a power of 2 (the board size);
the location L of the missing square
Output Parameters: None
tile(n,L) {
if (n == 2) {
// the board is a right tromino T
tile with T
return
}
divide the board into four n/2 n/2 subboards
place one tromino as in Figure 5.1.4(b)
// each of the 1 1 squares in this tromino
// is considered as missing
let m
1
,m
2
,m
3
,m
4
be the locations of the missing squares
tile(n/2,m
1
)
tile(n/2,m
2
)
tile(n/2,m
3
)
tile(n/2,m
4
)
}
Write the corresponding recurrence relation!
MergeSort
10 36 40 12 1 7 30 9 44 15
Split
10 36 40 12 1
7 30 9 44 15
10 36
40 12 1
7 30 9 44 15
10 36
40 12 1
1 12
7 30 9 44 15
44
15
MergeSort
10 36 40 12 1 7 30 9 44 15
Split
10 36 40 12 1
7 30 9 44 15
10 36
40 12 1
7 30 9 44 15
10 36
40 12 1
1 12
7 30 9 44 15
44
15
1 12
15 44
10 36
1 12 40
7 30 9 15 44
1 10 12 36 40
7 9 15 30 44
1 7 9 10 12 15 30 36 40 44
How do we merge?
1 10 12 36 40
7 9 15 30 44
1
1 10 12 36 40
7 9 15 30 44
1
7
1 10 12 36 40
7 9 15 30 44
1 7 9
1 10 12 36 40
7 9 15 30 44
1 9 10
7
1 10 12 36 40
7 9 15 30 44
1 9 10 12
7
1 10 12 36 40
7 9 15 30 44
1 7 9 10 12 15
1 10 12 36 40
7 9 15 30 44
1 9 10 12 15 30
7
1 10 12 36 40
7 9 15 30 44
1 9 10 12 15 30 36
7
1 10 12 36 40
7 9 15 30 44
1 7 9 10 12 15 30 36 40
1 10 12 36 40
7 9 15 30 44
1 9 10 12 15 30 36 40 44
7
1 10 12 36 40
7 9 15 30 44
1 9 10 12 15 30 36 40 44
7
Algorithm 5.2.2 Merge
This algorithm receives as input indexes i, m, and j, and an array a, where
a[i], ... , a[m] and a[m +1], ... , a[j] are each sorted in nondecreasing order.
These two nondecreasing subarrays are merged into a single nondecreasing
array.
Input Parameters: a,i,m,j
Output Parameter: a
merge(a,i,m,j) {
p = i // index in a[i], ... , a[m]
q = m + 1 // index in a[m + 1], ... , a[j]
r = i // index in a local array c
while (p m && q j) {
// copy smaller value to c
if (a[p] a[q]) {
c[r] = a[p]
p = p + 1
}
else {
c[r] = a[q]
q = q + 1
}
r = r + 1
}
...
...
// copy remainder, if any, of first subarray to c
while (p m) {
c[r] = a[p]
p = p + 1
r = r + 1
}
// copy remainder, if any, of second subarray to c
while (q j) {
c[r] = a[q]
q = q + 1
r = r + 1
}
// copy c back to a
for r = i to j
a[r] = c[r]
}
Algorithm 5.2.3 Mergesort
This algorithm sorts the array a[i], ... , a[j] in nondecreasing order. It uses
the merge algorithm (Algorithm 5.2.2).
Input Parameters: a,i,j
Output Parameter: a
mergesort(a,i,j) {
// if only one element, just return
if (i == j)
return
// divide a into two nearly equal parts
m = (i + j)/2
// sort each half
mergesort(a,i,m)
mergesort(a,m + 1,j)
// merge the two sorted halves
merge(a,i,m,j)
}
Stable sort algorithms

A sorting algorithm is stable if it preserves
the original ordering of equal elements.

Mergesort is stable.
Bubble Sort
Very, very simple sorting algorithm.

Works by swapping adjacent elements if
theyre out of order.

Starts at the end, bubbling the lowest
element down, incrementally working on
smaller sections of the array.

Can work the other way of course, taking the
highest to the end.
Bubble Sort
bubbleSort(int[] A){
for (int i = 0; i < A.length; i++){
for (int j = A.length-1; j > i; j--){
if (A[j] < A[j-1]){
int temp = A[j];
A[j] = A[j-1];
A[j-1] = temp;
}
}
}
}
Bubble Sort
4 10 3 1 5
i
j
4 10 3 1 5
i j
4 10 1 3 5
i
j
4 1 10 3 5
i
j
1 4 10 3 5
i
j
1 4 10 3 5
i
j
1 4 10 3 5
i
j
1 4 3 10 5
i j
1 3 4 10 5
i j
1 3 4 10 5
1 3 4 5 10
i
j
1 3 4 5 10
i j
i j
Bubble Sort
Worst case time T(n
2
).

Easy to see from two loops.

What is the worst possible input?

Bubble sort is not commonly used in practice,
though it can be sped up through various
methods, but this does not change the
asymptotic time.
Insertion Sort


Works by making a sorted area at the front
of the array (just by first partitioning the first
element, which is a sorted one element array),
into which it inserts subsequent elements by
shuffling elements up until it finds the correct
place.
Algorithm 6.1.2 Insertion Sort
This algorithm sorts the array a by first inserting a[2] into the sorted array a[1]; next
inserting a[3] into the sorted array a[1], a[2]; and so on; and finally inserting a[n] into the
sorted array a[1], ... , a[n - 1].
Input Parameters: a
Output Parameters: None
insertion_sort(a) {
n = a.last
for i = 2 to n {
val = a[i] // save a[i] so it can be inserted
j = i 1 // into the correct place
// if val < a[j],move a[j] right to make room for a[i]
while (j 1 && val < a[j]) {
a[j + 1] = a[j]
j = j - 1
}
a[j + 1] = val // insert val
}
}
Insertion Sort Example
4 10 3 1 5
Sorted
area
val=10
4 10 3 1 5 val=3
4 10 10 1 5 val=3
3 4 10 1 5 val=3
3 4 10 1 5 val=1
3 4 10 10 5 val=1
3 4 4 10 5 val=1
3 3 4 10 5 val=1
Insertion Sort Example
1 3 4 10 5
Sorted
area
val=1
1 3 4 10 5 val=5
1 3 4 10 10 val=5
1 3 4 5 10 val=5
Insertion Sort
Again, in the worst case, running time is T(n
2
).

The worst case is where val has to be inserted in the
first position, giving the running time of:





Prove this! That is, prove both O and O.

i=1
n-1
i = n(n-1)/2 = O(n
2
)