Analysis and Design of Algorithms Unit 8

Unit 8 Transform and Conquer

Structure:
8.1 Introduction
Objectives
8.2 Presorting
8.3 Gaussian Elimination
Basics of Gaussian elimination
8.4 Balanced Search Trees
Fundamentals of balanced search trees
AVL trees
2-3 trees
8.5 Heaps and Heapsort
Max and min heaps
Architectural approach of heaps and algorithms
Heapsort
8.6 Problem Reduction
Computing the least common multiple
Counting paths in graphs
8.7 Summary
8.8 Glossary
8.9 Terminal Questions
8.10 Answers

8.1 Introduction
By now you must be familiar with the concepts of decrease-and-conquer
technique. In this unit we deal with a group of design methods that are
based on the idea of transformation. We call this general technique
transform-and-conquer. It is called so because these methods involve a two-
stage procedure.
A. Transform – Here the problem’s instance is changed to a level where
the solution to the problem can be obtained easily.
B. Conquer – In this stage the problem is solved.

Sikkim Manipal University B1480 Page No. 155

Analysis and Design of Algorithms Unit 8

The three variations of the transform and conquer approach which is

illustrated in figure 8.1 are:
1) Instance simplification: It is the transformation to a simpler and more
suitable instance of the same problem, such as
 presorting
 Gaussian elimination
2) Representation change: It is the transformation to a different
representation of the same instance, such as
 balanced search trees
 Horner’s rule fast exponentiation
 heaps and heapsort
3) Problem reduction: It is the transformation to an instance of a different
problem for which an algorithm is already available, such as
 computing least common multiple (lcm)
 counting paths in graphs

Simpler instance
Problem’s or
instance Another representation Solution
or
Another problem’s instance

Figure 8.1: Transform and Conquer Strategy

Objectives:
After studying this unit you should be able to:
 define the technique of transform and conquer
 describe the method of presorting
 explain Gaussian elimination technique
 explain the approaches of AVL and 2-3 trees in balanced search trees
 define heap and apply it to heapsort
 apply the problem reduction strategy

Sikkim Manipal University B1480 Page No. 156

Analysis and Design of Algorithms Unit 8

8.2 Presorting
Presorting is sorting before performing any operation. The time efficiency of
algorithms that involve sorting depends on the efficiency of the sorting
algorithm being used. The time spent in presorting should be offset by the
advantages of presorting.
Let us first define presorting.
Presorting is a process of preprocessing data in order to improve it so that it
can later be refined in a better way by any sorting algorithm.
Let us now consider the application of presorting to compute the mode.
Example: Computing a mode
A mode is a value that has the most number of occurrences in a given list of
numbers. For example, for the following list of numbers - 9, 3, 5, 9, 4, 9, 7,
8, 9, the mode is 9. (If you find that several different values occur most
often, then you can consider any of them as the mode.)
In the brute force approach to computing a mode, you would scan the list
and compute the frequencies of all its distinct values, then find the value
with the largest frequency. Whereas in order to implement this idea here,
you will have to store the values already encountered, along with their
frequencies, in a separate list. In all iterations, compare the ith element of the
original list with the values already encountered by traversing the separate
list. If a matching value is found, increment its frequency; otherwise, add the
current element to the list of distinct values seen so far with the frequency of
1. Hence considering the previous list of numbers – 9, 3, 5, 9, 4, 9, 7, 8, 9,
let us sort the list first. We get the sorted list as – 3, 4, 5, 7, 8, 9, 9, 9. Here
we can clearly see that 9 has the highest frequency, and therefore it is the
mode.
The worst case scenario that you are most likely to encounter here is to
encounter a list with no equal elements. For such a list, its ith element is
compared with i−1 elements of the other list which stores the frequency of
distinct values seen so far before being added to the list with a frequency of
1. The worst-case number of comparisons made by this algorithm in
creating the frequency list is given in equation Eq: 8.1.
n
(n  1)n
C (n)   (i  1)  0  1   (n  1)  (n2 )
i 1 2 Eq: 8.1

Sikkim Manipal University B1480 Page No. 157

Analysis and Design of Algorithms Unit 8

The additional n−1 comparisons needed to find the largest frequency in the
auxiliary list do not change the quadratic worst-case efficiency class of the
algorithm. Hence we first sort the input. Then all equal values will be
adjacent to each other. To compute the mode, all we need to do is to find
the longest run of adjacent equal values in the sorted array.
Let us now discuss the algorithm Presort Computing Mode which presorts
an array of numbers and then computes the mode.
Algorithm: Presort Computing Mode (A [0...n − 1])
//Computes the mode of an array by sorting it first
//Input: An array A [0...n − 1] of orderable elements
//Output: The array’s mode
Sort the array A
i←0 //current run begins at position i
modefrequency ← 0 //highest frequency seen so far
while i ≤ n − 1 do
runlength ← 1; runvalue←A[i]
while i + runlength ≤ n − 1 and A [i + runlength] = runvalue
runlength←runlength+1
if runlength> modefrequency
modefrequency ← runlength; modevalue ← runvalue
i ← i + runlength
return modevalue

Let us now trace the algorithm to compute the mode using presort.
Algorithm Tracing for Computing Mode Using Presort
A[]=[1,3,3];
n=2;
while 0<= 2-1 do
runlength=1;
runvalue=A[0];
while 0+1<=2-1 and A[0+1]=A[0]
runlength=1+1
if 2>0
modefrequency=2; //modefrequency value changes from 0 to 2
modevalue=2; //modevalue variable is assigned the value 2
i=0+2 //value of i changes to 2
return 2 //modevalue has been assigned the value 2

Sikkim Manipal University B1480 Page No. 158

Analysis and Design of Algorithms Unit 8

Self Assessment Questions

1. _______, ________ & ________ are three instances of transform and
conquer technique.
2. The worst case efficiency of brute force algorithm is _______.
3. The searching requires _______ comparisons to in the worst case,
when the array is sorted first.

8.3 Gaussian Elimination

In the previous section we learnt about presorting. In this section we will
learn about Gaussian elimination which is another example of instance
simplification.
The algorithm of Gaussian elimination is used for solving systems of linear
equations, calculating the inverse of an invertible square matrix, and finding
the rank of a matrix.
8.3.1 Basics of Gaussian elimination
For many applications, we need to solve a system of n equations in n
unknowns as shown in the set of equations in Eq: 8.2
a11x1 + a12x2 + · · · + a1nxn = b1
a21x1 + a22x2 + · · · + a2nxn = b2
.
.
.
an1x1 + an2x2 + · · · + anxn = bn Eq: 8.2
Here n is a large number; hence to solve such systems of linear equations
we use the Gaussian elimination technique. The idea of Gaussian
elimination is to transform a system of n linear equations with n unknowns to
an equivalent system (i.e., a system with the same solution as the original
one) with an upper triangular coefficient matrix, a matrix with all zeros below
its main diagonal.

Sikkim Manipal University B1480 Page No. 159

Analysis and Design of Algorithms Unit 8

The system can be written as a matrix notation as shown in Matrix: 8.1.

 a11 a12 a1n   x1   b1 

a a22 a2 n   x2  b2 
 21 
    
    
 an1 an 2 ann   xn  bn 
Matrix: 8.1
To perform Gaussian elimination on a system you will first have to augment
the matrix. Your augmented matrix should look like Matrix: 8.2.

 a11 a12 a1n b1   x1 

 
 a21 a22 a2 n b2   x2 
  
  
 an1 an 2 ann bn   xn 
Matrix: 8.2
Now perform row operations so as to make the elements below the main
diagonal of the matrix to 0. This results in Matrix 8.3.
 a '11 a '12 a '1( n 1) a '1n b '1 
 
 0 a ' 22 a ' 2( n 1) a '2 n b'2 
 
 
 0 0 a '( n 1)( n 1) a '( n 1) n 
 0 0 a ' nn b ' n 
 0
Matrix: 8.3
th
Solve the equation of the n row for xn, then substitute back into the
equation of the (n-1)st row to obtain a solution for xn-1 and so on.
Let us now discuss the algorithm for Gaussian elimination.
Algorithm: Gauss elimination (A [1...n, 1...n], b [1...n])
//Applies Gaussian elimination to matrix A of a system’s coefficients,
//augmented with vector b of the system’s right-hand side values.
//Input: Matrix A[1..n, 1,..n] and column-vector b[1..n]
//Output: An equivalent upper-triangular matrix in place of A with the
//corresponding right-hand side values in the (n + 1)st column.
for i ←1 to n do A[i, n + 1]←b[i] //augments the matrix
for i ←1 to n − 1 do
for j ←i + 1 to n do
for k←i to n + 1 do
A[j , k] ←A[j , k]− A[i, k] ∗ temp

Sikkim Manipal University B1480 Page No. 160

Analysis and Design of Algorithms Unit 8

Let us now trace the algorithm for Gauss elimination.

Algorithm Tracing for Gauss elimination n=2
//Applies Gaussian elimination to matrix A of a system’s coefficients,
//augmented with vector b of the system’s right-hand side values.
//Input: Matrix A[1..2, 1,..2] and column-vector b[1..2]
//Output: An equivalent upper-triangular matrix in place of A with the
//corresponding right-hand side values in the (2 + 1)st column.
for i ←1 to 2 do A[1, 3]←b[1] //augments the matrix
for i ←1 to 1 do
for j ←2 to 2 do
for k←1 to 2 do
A[2, 1] ←A[2 , 1]− A[1, 1] ∗ temp

Activity 1
2 4  2
 
A 4 9  3
 
 2  3 7 
Use Gaussian elimination to compute the inverse of the above matrix.

Self Assessment Questions

4. Gaussian elimination is an algorithm for solving systems of
__________ equations.
5. In Gaussian elimination we make the ________ coefficient matrix zero.
6. Gaussian elimination can be used to find the _________of a matrix.

8.4 Balanced Search Trees

In the previous section we discussed Gaussian elimination. In this section
we will discuss balanced search trees, an instance of representation change
variety of transform and conquer.
Many data structures use binary search trees. Operations on these search
trees are often proportional to the height of the tree. We can guarantee the
efficiency of such operations by ensuring that the height of the tree is
logarithmic in the number of nodes.

Sikkim Manipal University B1480 Page No. 161

Analysis and Design of Algorithms Unit 8

8.4.1 Fundamentals of balanced search tree

The binary search trees are one of the principle data structures for
implementing dictionaries. A balanced search tree is a binary tree whose
nodes contain elements of a set of orderable items, one element per node.
Hence all elements in the left sub-tree are smaller than the element in the
sub-tree’s root and all the elements in the right sub-tree are greater than it.
Thus there is a gain in the time efficiency of searching, insertion, and
deletion, which are all in ө(log n), but only in the average case. In the worst
case, these operations are in ө(n) because the tree can degenerate into a
severely unbalanced one with height equal to n − 1.
There are two approaches to balanced search trees. The first approach is of
the instance simplification variety, where an unbalanced binary search tree
is transformed to a balanced one. Examples of these are AVL trees and red-
black trees. The second approach is of the representation change variety.
Specific cases of such trees are 2-3 trees, 2-3-4 trees and B-trees. In this
unit, we discuss only AVL and 2-3 trees.
8.4.2 AVL trees
An AVL tree is a binary search tree having a balance factor of every node,
which is defined as the difference between the heights of the node’s left and
right sub-trees, is either 0 or +1 or −1. (The height of the empty tree is
defined as −1.)
For example, the binary search tree in figure 8.2 (a) is an AVL tree but that
in 8.2 (b) is not. The numbers above the nodes indicate the nodes’ balance
factors.

Figure 8.2: (a) AVL Tree. (b) Binary Search Tree that is not an AVL Tree

Sikkim Manipal University B1480 Page No. 162

Analysis and Design of Algorithms Unit 8

If an insertion of a new node makes an AVL tree unbalanced, we transform

the tree by rotation. A rotation in an AVL tree is a local transformation of its
sub-tree rooted at a node whose balance has become either +2 or −2; if
there are several such nodes, we rotate the tree rooted at the unbalanced
node that is the closest to the newly inserted leaf. The rotations can be
classified as single rotations and double rotations.
Single rotations
Single rotations are the rotations performed to balance the AVL tree when a
new node is added. Here a node is rotated only once so as to balance the
tree. They are of two types.
R-rotation – The first rotation type is called the single right rotation or
R-rotation. Figure 8.3 presents the single R-rotation in its most general form.
Note that this rotation is performed after a new key is inserted into the left
sub-tree of the left child of a tree whose root had the balance of +1 before
the insertion.

2 0
3
1 2

2 0 0
0
1 3
1

Figure 8.3: R-rotation

L-rotation – The symmetric single left rotation or L-rotation is another type

of single rotation. It is the mirror image of the single R-rotation (Refer figure
8.4). It is performed after a new key is inserted into the right sub-tree of the
right child of a tree whose root had the balance of −1 before the insertion.
-2 0

1 2
-1
0 0
2
0 1 3

3
3
Figure 8.4: L-rotation

Sikkim Manipal University B1480 Page No. 163

Analysis and Design of Algorithms Unit 8

Double rotations
These rotations comprise of single rotation performed twice. They can be
classified as LR rotation and RL rotation.
LR-rotation – The double left-right rotation (LR-rotation) is a combination of
two rotations: we perform, the L-rotation of the left sub-tree of root followed
by the R-rotation of the new tree rooted at (Refer figure 8.5). It is performed
after a new key is inserted into the right sub-tree of the left child of a tree
whose root had the balance of +1 before the insertion.

3 0

-1 2

1 0 0

0 1 3

Figure 8.5: L-R-rotation

RL-rotation – This is the mirror image of the double LR-rotation. It is

performed after a new key is inserted into the left sub-tree of the right child
of a tree whose root had the balance of -1 before the insertion (Refer figure
8.6).
-2
0
1
1 2

3 0 0

0 1 3

Figure 8.6: R-L-rotation

The drawbacks of AVL trees are the need to maintain balances for the tree’s
nodes, frequent rotations, and overall complexity, especially of the deletion
operation.

Sikkim Manipal University B1480 Page No. 164

Analysis and Design of Algorithms Unit 8

Let us next discuss the algorithm to balance AVL trees.

Algorithm: AVL balance(x)
// Input: Node x
// Output: x and nodes above it are balanced
while x ≠null do
if x.left.height > x.right.height + 1 then
if x.left.left.height < x.left.right.height
then LR-Rotation(x)
else R-Rotation(x)
else if x.left.height + 1 < x.right.height then
if x.right.left.height > x.right.right.height
then RL-Rotation(x)
else L-Rotation(x)
x ← x.parent

Let us now trace the algorithm to balance an AVL.

Algorithm Tracing of the Naive Algorithm to Balance an AVL
// Input: Node 2
// Output: 2 and nodes above it are balanced
while 2 ≠null do
if 2.left.height > 2.right.height + 1 then
if 2.left.left.height < 2.left.right.height
then LR-Rotation(2)
else R-Rotation(2)
else if 2.left.height + 1 < 2.right.height then
if 2.right.left.height > 2.right.right.height
then RL-Rotation(2)
else L-Rotation(2)
2 ← 2.parent

8.4.3 2-3 trees

A 2-3 tree is a data structure, which was introduced because the binary
search tree had no guarantee of being balanced when some random
insertion or deletion is made. A 2-3 tree has 2 types of nodes:
o 2-node
o 3-node

Sikkim Manipal University B1480 Page No. 165

Analysis and Design of Algorithms Unit 8

2-node
X

P Q

Figure 8.7: 2-Node

The figure 8.7 shows a 2-node structure. It has one data element and two
children. Every 2-node must have the following properties
1. Every value appearing in the child P must be ≤X
2. Every value appearing in the child Q must be ≥X
3. The length of the path from the root of a 2-node to every leaf in its child
must be the same.
3-node
X|Y

P Q R

Figure 8.8: 3-Node

The figure 8.8 shows a 3-node structure. It has 2 data elements and 3
children. Every 3-node must have the following properties:
1. Every value appearing in child P must be ≤ X
2. Every value appearing in child Q must be in between X and Y
3. Every value appearing in child R must be ≥ Y
4. The length of the path from the root of a 3-node to every leaf in its child
must be the same
Properties of 2-3 trees
We will discuss the operations done in a 2-3 tree using figure 8.9 which is a
2-3 tree with numerical keys. As explained earlier, if the key of the child is
smaller than the smallest key of its parent then the child is a left child i.e. it is
placed in the left branch in the sub-tree.

Sikkim Manipal University B1480 Page No. 166

Analysis and Design of Algorithms Unit 8

Figure 8.9: 2-3 Tree

Similarly if a child is larger than the largest key of its parent then it is the
right child, and if it is in between it takes the middle sub-tree. Let us now try
to insert a key ‘28’ into the tree in the figure 8.9.

Figure 8.10: Inserting 28 into the Tree

Here in the figure 8.10 we see that the node containing 27 and 29 have
been split open to accommodate 28 which is now in a temporary node.
It should be remembered that it is always the middle value of the tree that is
pushed upwards. The insertion stops as soon as a node with only one key is
reached.

Figure 8.11: Further Splitting the Node Containing 25 and 31

Sikkim Manipal University B1480 Page No. 167

Analysis and Design of Algorithms Unit 8

Proceeding with the insertion, in the figure 8.11, we can see that the node
containing 25 and 31 have been split to accommodate 28. On doing this, the
node containing 21 and 23 becomes the left child of 25 and the node with 27
becomes the right child. Similarly 29 becomes the left child of 31 and the
node with 33 and 35 become the right child.

Figure 8.12: Balanced Tree

At this point (Refer figure 8.12) we see that the node with 9 and 19 has been
split. As 28 is greater than 19, it becomes the right child and 9 being smaller
becomes the left child. Here we can see that the tree has four levels, but is
balanced after insertion.
Self Assessment Questions
7. An AVL tree is a _________ tree.
8. The _________________ is the mirror image of the RL-rotation.
9. The two nodes of 2-3 tree are ___________ and ____________.

8.5 Heaps and Heapsort

In the previous section we learnt about balanced search trees. In this
section we will see another instance of representation change i.e. heaps
and heap sorts.
Heaps are data structures that are especially suitable for implementing
priority queues. A priority queue is a set of items with an orderable
characteristic called an item’s priority, with the following operations:
 finding an item with the highest (i.e., largest) priority
 deleting an item with the highest priority
 adding a new item to the set

Sikkim Manipal University B1480 Page No. 168

Analysis and Design of Algorithms Unit 8

The heap is the data structure that serves as a foundation of a theoretically

important sorting algorithm called heapsort which we will discuss after we
define the heap.
8.5.1 Max and min heaps
A heap can be defined as a binary tree with keys assigned to its nodes (one
key per node) provided the following two conditions are met:
The tree’s shape requirement: In a binary tree, all its levels must be full
except possibly the last level, where only some rightmost leaves may be
missing. This requirement is valid for both max and min heaps.

Figure 8.13: Max Heap

The parental dominance requirement for max heap: The key at each
node has to be greater than or equal to the keys at its children i.e.
key(parent) ≥ key(child). The figure 8.13 represents a max heap. All the
concepts in this unit utilizes the approach of max heaps.

Figure 8.14: Min Heap

The parental dominance requirement for min heap: The key at each
node has to be lesser than or equal to the keys at its children
i.e. key(parent) ≤ key(child). The figure 8.14 represents a min heap.

(a) (b)
Figure 8.15: Illustration of Heap

Sikkim Manipal University B1480 Page No. 169

Analysis and Design of Algorithms Unit 8

In figure 8.15, the first tree i.e. figure 8.15(a) is a heap, but the second tree
i.e. figure 8.15(b) is not, as the tree’s shape requirement is violated.
8.5.2 Architectural approach of heaps and algorithms
The two principal ways to construct a heap are:
1. Bottom-up heap construction algorithm
2. Top-down heap construction algorithm
Let us now discuss the bottom-up heap construction.
Bottom-up heap construction
It initializes the essentially complete binary tree with n nodes by placing
keys in the order given and then “heapifies” the tree as follows. Starting with
the last parental node and ending with the root, the algorithm checks
whether the parental dominance holds for the key at this node.
2 2

9 7 9 8

6 5 8 6 5 7

Figure 8.16: Checking for Parental Dominance

If it does not, the algorithm exchanges the node’s key K with the larger key
of its children and checks whether the parental dominance holds for K in its
new position (Refer to figures 8.16 and 8.17).

2 2 9

9 8 9 8 2 8

6 5 7 6 5 7 6 5 7

Figure 8.17: Continue Checking for Parental Dominance

This process continues until the parental dominance requirement for K is

satisfied. After completing the “heapification” of the sub-tree rooted at the
current parental node, the algorithm proceeds to do the same for the node’s
immediate predecessor. The algorithm stops after this is done for the tree’s
root to give the final heap in figure 8.18(a). The numbers above the nodes in
the tree indicate their position in the array which is shown by the
figure 8.18(b).

Sikkim Manipal University B1480 Page No. 170

Analysis and Design of Algorithms Unit 8
0

1 9 2
[0] 9
3 6 4 5 8
[1] 6
5 7
2 [2] 8
[3] 2
[4] 5

[5] 7

(a) (b)
Figure 8.18: Final Heap and Array Representation

Since the value of a node’s key does not change during the process of
shifting it down the tree, it need not be involved in intermediate swaps. The
empty nodes are swapped with larger keys in its children until a final
position is reached where it accepts the “erased” value again.
Let us now study the algorithm for bottom-up heap construction.
Algorithm: Heap Bottom-up (H [1...n])
//Constructs a heap from the elements of a given array
// by the bottom-up algorithm
//Input: An array H[1..n] of orderable items
//Output: A heap H[1..n]
for i ←n/2 down to 1 do
k←i; v←H[k]
heap←false
while not heap and 2 * k ≤ n do
j ←2 * k
if j <n //there are two children
if H[ j ]<H[ j + 1] j ←j + 1
if v ≥ H[j ]
heap←true
else H[k]←H[j ]; k←j
H[k]←v

Sikkim Manipal University B1480 Page No. 171

Analysis and Design of Algorithms Unit 8

Let us now trace the bottom-up heap construction algorithm.

Algorithm Tracing of Bottom-up Heap Construction
n=2
Algorithm: Heap bottom-up (H [1...2])

//Constructs a heap from the elements of a given array

// by the bottom-up algorithm
//Input: An array H[1..2] of orderable items
//Output: A heap H[1..2]
for i ←2/2=1 down to 1 do
k←1; v←H[1]
heap←false
while not heap and 2 * 1 ≤ 2 do
j ←2 * 1
if j <n //there are two children
if H[ 2 ]<H[ 2 + 1] 2 ←2 + 1
if v ≥ H[2 ]
heap←true
else H[1]←H[2 ]; 1←2
H[1]←v
To find the efficiency in the worst case, assume n = 2k −1 so that a heap’s
tree is full, i.e., the largest number of nodes occur on each level. Let h be
the height of the tree; according to the first property of heaps in the list at the
beginning of the section, h = log2 n (or just log2 (n + 1) − 1= k − 1 for the
specific values of n we are considering). In the worst case of the heap
construction algorithm each key on level i of the tree will travel to the leaf
level h.
Since moving to the next level down requires two comparison, one to find
the larger child and the other to determine whether the exchange is
required, the total number of key comparisons involving a key on level i will
be 2(h − i). Equation Eq: 8.3 gives the total number of key comparisons in
the worst case.
h 1 h 1
Cworst (n)    2(h  i)   2(h  i )2i  2(n  log 2 (n  1))
i  0 level i keys i 0
Eq: 8.3
Here the validity of the last equality can be proved either by using the


h
closed-form formula for the sum i 1
i 2i h or by mathematical induction in h.

Sikkim Manipal University B1480 Page No. 172

Analysis and Design of Algorithms Unit 8

Thus, with this bottom-up algorithm, a heap of size n can be constructed

with fewer than 2n comparisons.
Top-down heap construction algorithm
The top-down heap construction algorithm (which is less efficient) constructs
a heap by successive insertions of a new key into a previously constructed
heap shown in figure 8.19.
8

5 7

2 4 6

Figure 8.19: Pre-constructed Heap

To insert a new key into a heap, first attach a new node with key K in it after
the last leaf of the existing heap. Then shift K up to its appropriate place in
the new heap as in figure 8.20.
8

5 7

2 4 6 9

Figure 8.20: Inserting 9 into the Heap

Compare K with its parent’s key. Stop if the parent key is greater than or
equal to K (the structure is a heap), else, swap these two keys and compare
K with its new parent (Refer to figure 8.21). This swapping continues until K
is not greater than its last parent or it reaches the root. In this algorithm, too,
we can shift up an empty node until it reaches its proper position, where it
will get K’s value.

8 9

5 9 5 8

2 4 6 7 2 4 6 7

Figure 8.21: Check for Parental Dominance until Tree is Balanced

Sikkim Manipal University B1480 Page No. 173

Analysis and Design of Algorithms Unit 8

This insertion operation doesn’t require more key comparisons than the
heap’s height. Since the height of a heap with n nodes is about log2 n, the
time efficiency of insertion is in O(log n).
Deleting the root key from a heap
The following steps indicate the procedure to delete the root key from a
heap in the figure 8.22.
9

8 6

2 5 1

Figure 8.22: Sample Heap

Step 1: Exchange the root’s key with the last key K of the heaps in the
figure 8.23.
1

8 6

2 5 9
Figure 8.23: Exchanging the Root Key with the Smallest Key

Step 2: Decrease the heap’s size by 1 (Refer figure 8.24).

1

8 6

2 5
Figure 8.24: Delete the Key Having the Original Root Key

Step 3: “Heapify” the smaller tree by shifting K down the tree exactly in the
same way we did it in the bottom-up heap construction algorithm. That is,
verify the parental dominance for K: if it holds, we are done (Refer figure
8.25); if not, swap K with the larger of its children and repeat this operation
until the parental dominance condition holds for K in its new position.
8

5 6

2 1

Figure 8.25: Heapified Tree

Sikkim Manipal University B1480 Page No. 174

Analysis and Design of Algorithms Unit 8

We can determine the efficiency of deletion by the number of key

comparisons needed to “heapify” the tree after the swap has been made
and the size of the tree is decreased by 1. Since it cannot require more key
comparisons than twice the heap’s height, the time efficiency of deletion is
in O(log n) as well.
8.5.3 Heapsort
A heap is used to implement heapsort, which is an optimal sorting algorithm.
This can be done simply by inserting keys into the heap, then removing
them one by one.
Basics of heapsort
Heapsort is a comparison-based sorting algorithm, and belongs to the
selection sort family. Although it is slower in practice on most machines than
quick sort, it has a more favorable worst-case O(n log n) runtime.This is a
two-stage algorithm that works as follows.
Stage 1: (heap construction): Construct a heap for a given array.
Stage 2: (maximum deletions): Apply the root-deletion operation n−1
times to the remaining heap.
We will see the implementation of heapsort using the following example.
Consider the following numbers 5,3,8,2,4,7,9. The tree representation of the
set of numbers can be seen in the figure 8.26.

3 8

2 4 7 9

Figure 8.26: Tree for the List

Let us now perform the first stage of heapification on the tree to make it
balanced as shown in the figure 8.27.

5 5 9

4 8 4 9 4 5

2 3 7 9 2 3 7 8 2 3 7 8

Figure 8.27: Process of Heapification

Sikkim Manipal University B1480 Page No. 175
Analysis and Design of Algorithms Unit 8

Now that we have got the heapified tree (Refer figure 8.28) we will perform
stage two which includes the deletion of the nodes

4 8

2 3 7 5

Figure 8.28: Heapified Tree

To perform the node deletion, now that we have the largest value on the top
of the heap we can just push it into an array and replace the value by the left
most element in the tree, which is deleted (Refer figure 8.29)

9
4 8

3 7 5

Figure 8.29: Node Deletion

We can now see in that in the figure 8.29 that the top most element which is
also the largest element is pushed into the array. So now we are left with a
tree which is not balanced. Therefore we will repeat the process of
hepification at the end of which we will have the largest element in the top
node of the tree. This element is now pushed again in the array, and
replaced by the bottom leftmost element. Hence we repeat this process
again and again which will finally give us the sorted array as in figure 8.30.

2 3 4 5 7 8 9

Figure 8.30: Sorted Array

Since we already know that the heap construction stage of the algorithm is
in O(n), we need to investigate just the time efficiency of the second stage.
For the number of key comparisons, C(n), needed for eliminating the root
keys from the heaps of diminishing sizes from n to 2, we get the inequality
shown in equation Eq: 8.4.

Sikkim Manipal University B1480 Page No. 176

Analysis and Design of Algorithms Unit 8

n 1
C (n)  2[log 2 (n  1)]  2[log 2 (n  2)    2[log 2 1]  2 log 2 i
i 1

n 1
  log 2 (n  1)  2(n  1) log 2 (n  1)  2n log 2 n
i 1 Eq: 8.4
This means that C(n) є O(n log n) for the second stage of heapsort. For both
stages, we get O(n) + O(n log n) = O(n log n). A more detailed analysis
shows that, in fact, the time efficiency of heapsort is in O(n log n) in both the
worst and average cases.

Activity 2
Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by the bottom-up algorithm.

Self Assessment Questions

10. _________ heap construction algorithm is less efficient than its
counterpart.
11. A heap can be defined as ________ with keys assigned to its nodes.
12. The time efficiency of heapsort is ________ in both worst and average
cases.

8.6 Problem Reduction

The previous section explained heap sort. This section explains the
technique of problem reduction.
“If you need to solve a problem, reduce it to another problem that you know
how to solve. “
This strategy can be used for actual problem solving, too. But it is difficult to
find a problem to which the problem at hand should be reduced to. We
should also ensure that our reduction-based algorithm is more efficient than
solving the original problem directly. This strategy can simply be shown by
the figure 8.31.

Problem 1 reduction Problem 2 Alg. A Solution to

(to be solved) (can be solved using Problem 2
algorithm A)

Figure 8.31: Problem Reduction Strategy

Sikkim Manipal University B1480 Page No. 177
Analysis and Design of Algorithms Unit 8

8.6.1 Computing the least common multiple

You know that the least common multiple of two positive integers, denoted
as lcm (m, n), is defined as the smallest integer that is divisible by both m
and n. For example, lcm(24, 60) = 120, and lcm(11, 5) = 55. Given the prime
factors of m and n, lcm(m, n) can be computed as the product of all the
common prime factors of m and n times the product of m’s prime factors that
are not in n times n’s prime factors that are not in m.
For example consider equation Eq: 8.5
30 = 2 · 3 · 5
60 = 2 · 2 · 3 · 5
lcm(30, 60) = 2 · 2 · 3 · 5 = 60 Eq: 8.5
This algorithm for computing the least common multiple is inefficient and
requires a list of consecutive primes.
A much more efficient algorithm for computing the least common multiple
can be designed by using problem reduction. The Euclid’s algorithm helps to
find the greatest common divisor (gcd), which is a product of all the common
prime factors of m and n. It is not difficult to see that the product of lcm(m, n)
and gcd(m, n) includes every factor of m and n exactly once and hence is
simply equal to the product of m and n. This observation leads to the
formula in equation Eq: 8.6

mn
lcm(m, n) 
gcd(m, n) Eq: 8.6
Considering the equation Eq: 8.6, let us now solve the previous example in
Eq:8.5 to find the lcm(24,60). Here we can see that we get the same result
(Refer Eq: 8.7) as in Eq: 8.5.

30 x 60
lcm(30,60)   60
2 x 3x 5 Eq: 8.7
8.6.2 Counting paths in graphs
Let us now consider the problem of counting paths between two vertices in
a graph. It is not difficult to prove by mathematical induction that the number
of different paths of length k > 0 from the ith vertex to the jth vertex of a graph
(undirected or directed) equals the (i, j)th element of Ak where A is the
adjacency matrix of the graph. Thus, we can solve the problem of counting a

Sikkim Manipal University B1480 Page No. 178

Analysis and Design of Algorithms Unit 8

graph’s paths by using an algorithm for computing an appropriate power of

its adjacency matrix.
You can order the vertices of an adjacency matrix however you wish. This
means that you can have n! different orderings for a single graph. Usually,
to make things easier, we choose to order them alphabetically (a, b, c, d …).
You have a matrix which each column and row is labeled with the vertices.
Then you say how many edges connect i with j, and then you put the
answer in the (i,j)th position of the matrix. For example consider the graph in
the figure 8.32 its adjacency matrix can be written as matrix A.
a b a b c d

a 0 1 1 0
b 1 0 0 1
A  
c 1 0 0 1
d 0 1 1 0
 
d c

Figure 8.32: Counting Paths in a Graph

Hence if we want to calculate the no of paths of length n we can find it by
calculating An where A is the adjacency matrix. For example if we want to
find the no of paths of length 4 from a to d in the graph from the figure 8.30,
we have to find the value at the position (1,4) in the matrix A4.

a b c d

a 8 0 0 8
b 0 8 8 0
A4   
c 0 8 8 0
d 8 0 0 8
  Matrix: 8.4
Hence we find that in the matrix 8.4 in the position of the first row and fourth
column i.e. (1, 4) the value is 8. Therefore the no of paths of length 4 from a
to d is 8.
Activitiy 3
m.n
Prove the equality lcm(m, n) =
gcd(m,n)

Sikkim Manipal University B1480 Page No. 179

Analysis and Design of Algorithms Unit 8

Self Assessment Questions

13. Greatest common divisor can be computed easily by ________.
14. The problem of counting a graph’s paths can be solved with an
algorithm for an appropriate power of its _______________.
15. What is the formula obtained to find the lcm in the problem reduction
approach to find the lcm?

8.7 Summary
In this unit we have learnt that the transform and conquer technique works
as two stage method i.e. “Transform” where the problem is modified and
“Conquer” where the problem is solved. We now know that there are three
principal varieties of the transform-and-conquer strategy: instance
simplification, representation change, and problem reduction.
We have learnt that presorting, balanced tree search, Gaussian elimination,
heaps, counting paths in a graph etc. are all strategies illustrating the 3
different varieties involved in the implementation of this technique. These
strategies help us solve complex problems encountered such as sorting
elements, and counting paths in graphs.

8.8 Glossary
Term Description

Adjacency matrix Representation of the vertices of a graph adjacent to the

other vertices.

Rank of a matrix The maximal number of linearly independent columns of A

is the column rank of a matrix A. Similarly, the maximal
number of linearly independent rows of A is the row rank of
the matrix.
8.9 Terminal Questions
1. Explain the algorithm to compute mode via presorting.
2. Briefly explain the four rotations in an AVL tree.
3. Differentiate between bottom-up and top-down heap construction.
4. What is heap sort?
5. Describe 2-3 trees.

Sikkim Manipal University B1480 Page No. 180

Analysis and Design of Algorithms Unit 8

8.10 Answers
Self Assessment Questions
1. Instance simplification, problem reduction , representation change
2. ө(n2)
3. [log2 n] +1
4. Linear
5. Lower triangular
6. Rank
7. Binary search
8. LR-rotation
9. 2-node, 3-node
10. Top-down
11. Binary trees
12. O(n log n)
13. Euclid’s algorithm
14. Adjacent matrix
mn
15. lcm (m, n)  gcd( m, n)

Terminal Questions
1. Refer to 8.2 – Presorting
2. Refer to 8.4.2 – AVL trees
3. Refer to 8.5.2 – Architectural approach of heaps and algorithms
4. Refer to 8.5.3 – Heapsort
5. Refer to 8.4.3 – 2-3 trees

References
 Anany V. Levetin. (2002). Introduction to the analysis and design of
algorithms. Addison-Wesley Longman Publishing Co.
 Thomas H. Cormen, Charles E. Leiserson, Ronald L Rivest, Clifford
Stein. (2006). Introduction to algorithms,2nd Edition, PHI

E-References
 http://lcm.csa.iisc.ernet.in/dsa/node118.html
 http://mathworld.wolfram.com/GaussianElimination.html

Sikkim Manipal University B1480 Page No. 181