Algorithm Fundamental Techniques

MEETING 7
Algorithm Fundamental Techniques
• The Greedy Method
• The Fractional Knapsack Problem
• Task Scheduling
• Text Compression and Huffman Coding (Self Study)
• Divide-and-Conquer
• Recurrences and the Master Theorem
• Merge-Sort Algorithm
• The Maxima-Set Problem
• Dynamic Programming
Text Compression and Huffman Coding

• Discuss and illustrate by students (Self Study).

Application: Maxima Sets
• We can visualize the various trade-offs for optimizing two
dimensional data, such as points representing hotels
according to their pool size and restaurant quality, by
plotting each as a two dimensional point, (x, y), where x is
the pool size and y is the restaurant quality score.
• We say that such a point is a maximum point in a set if
there is no other point, (x′, y′), in that set such that x ≤ x′ and
y ≤ y′.
Application: Maxima Sets
• The maximum points are the best potential choices based
on these two dimensions and finding all of them is the
maxima set problem.
• We can efficiently find
all the maxima points
by divide-and-conquer.
• Here the set is
{A,H,I,G,D}.
 Recurrences – Divide-and-Conquer
• Divide-and conquer is a general algorithm design paradigm:
• Divide: divide the input data S in two or more disjoint subsets S1, S2, …
• Conquer: solve the subproblems recursively
• Combine: combine the solutions for S1, S2, …, into a solution for S

• The base case for the recursion

are subproblems of constant size.
• Analysis can be done using
recurrence equations.
Example – The classic Merge-Sort algorithm
• Merge-sort applies the divide-
and-conquer technique to the
sorting problem.
Example – The classic Merge-Sort algorithm
• The three divide-and-conquer steps for the Merge-sort are:
• Divide: If S has zero or one element, return S immediately; it is
already sorted. Otherwise (S has at least two elements), put the
elements of S into two sequences, S1 and S2, each containing
about half of the elements of S; that is, S1 contains the first n/2
elements of S, and S2 contains the remaining n/2 elements.
• Recur: Recursively sort the sequences S1 and S2.
• Conquer: Put back the elements into S by merging the sorted
sequences S1 and S2 into a sorted sequence.
Example – The classic Merge-Sort algorithm
• We can visualize an execution of the merge-sort algorithm
using a binary tree T, called the merge-sort tree.

Input sequences processed at Output sequences generated at

each node of T each node of T
The classic Merge-Sort algorithm
Recurrence Equation Analysis
• To analyze the running time of a divide-and-conquer
algorithm, we often use a recurrence equation.
• For instance, The conquer step of merge-sort consists of
merging two sorted sequences, each with n/2 elements and
implemented by means of a doubly linked list, takes at
most bn steps, for some constant b.
• Likewise, the basis case (n < 2) will take at b most steps.
Recurrence Equation Analysis
• Therefore, if we let T(n) denote the running time of merge-
sort:

• We can therefore analyze the running time of merge-sort

by finding a closed form solution to the above equation.
• That is, a solution that has T(n) only on the left-hand side.
 Recurrence Equation Analysis –
The Iterative Substitution Method
• One way to solve a divide-and-
conquer recurrence equation is to use
the iterative substitution method,
which is more colloquially known as
the “plug-and-chug” method.
• In the iterative substitution, or “plug-
and-chug,” technique, we iteratively
apply the recurrence equation to itself
and see if we can find a pattern:
Recurrence Equation Analysis –
The Iterative Substitution Method
• Note that base, 𝑇(𝑛) = 𝑏, case occurs when 2𝑖 = 𝑛. That is, 𝑖 = log 𝑛.

• So, 𝑇(𝑛) = 𝑏𝑛 + 𝑏𝑛 𝑙𝑜𝑔𝑛

• Thus, 𝑇(𝑛) is 𝑂(𝑛 log 𝑛).

The Maxima-Set Problem
The Maxima-Set Problem
Analysis of the Divide-and-Conquer
Maxima-Set Algorithm

The running time for the divide-and-conquer maxima-set

algorithm is 𝑂(𝑛 log 𝑛).