Lec4-Heaps and Search Trees

Heaps and Search trees
Lecture #4 of Algorithms, Data structures and Complexity
July 9, 2015
⃝
c GfH&JPK
#4: Heaps + Search trees
Plan for today

It is about trees and ordering
First heaps
• what is a heap?
• how to build a heap?
• use of heaps for priority queues and sorting
Then binary search trees
• what is a binary search tree?

• how to find, insert and delete elements of a binary search tree?
• rotations for balancing trees
⃝
c GfH&JPK 1
Trees in arrays
16
16 14 10 8 7 9 3 2 4 1
1 2 3 4 5 6 7 8 9 10
14 10
8 7 9 3
2 4 1
The elements in an array E can be seen as the nodes of a binary tree.
• E[1] is the root of the tree
• E[2i] is the left- and E[2i+1] is the right-child of E[i]
⃝
c GfH&JPK 2
Heaps
A binary tree with a representation in an array has special properties.
• all leaves are on at most two adjacent levels

• all levels – except possibly the lowest – are completely filled
• all leaves on the lowest level occur “to the left”
Such a special binary tree is a heap if for the keys in the nodes:
• each node key exceeds that of all its children, or this is a maxheap
• the keys of all its children exceed the key of the node. this is a minheap
Previous slide shows a maxheap.
⃝
c GfH&JPK 3
Inserting an element in a maxheap
To add an element x to a maxheap H with n elements
• put x into the first (left-most) open position

• if x is greater than its parent, swap them, and repeat at a higher level
Doing such an insertion requires O(log n) comparisons
• the level k of a heap of n elements is bounded: n 6 2k+1 − 1

⇒ k = ⌈log(n + 1)⌉ − 1
To re-order an array into a maxheap do not use repeated insertions, but

heapify (or fixheap) instead [Floyd 1964]
⃝
c GfH&JPK 4
Heapify – Strategy
Consider E[i] and assume its left and right subtrees are heaps
• but E[i] may be smaller than its children
Construct a heap from the left- and right subtrees and E[i], let the value
of E[i] “float down” in the structure such that the structure rooted at E[i]
is a heap
• determine the maximum of the values E[i] and of its children

• if E[i] is the largest, then the subtree rooted at it is a heap. Done.
• otherwise, swap E[i] with the largest and heapify the corresponding subtree
⃝
c GfH&JPK 5
Heapify – Algorithm
void Heapify(int E[ ], int i) // E[i] may be smaller than its children

{ int max, left = 2·i, right = 2·i +1;
if ((left 6 E.heapsize) && (E[left] > E[i])) max = left;
else max = i;
if ((right 6 E.heapsize) && (E[right] > E[max])) max = right;
// E[max ] = largest element of E[i], E[left ] and E[right ]
if (max ! = i) // if max = i then heap is ready
{ swap(E[i], E[max]);
Heapify(E, max); // heapify the subtree rooted at max
}
}
⃝
c GfH&JPK 6
Heapify – Example
1 4 1
Heapify(E,1) 16 Heapify(E,2)
2 3
Heap 2
16 10
3 Heap 4 10
4 5 6 7 4 5 6 7
7 9 3 14 7 9 3
14
9 9 10
8 10 2 1
2 8 1 8
Heap
16 Heapify(E,4) 16
2 3 2 3
14 10 14 10
4 5 6 7 4 5 6 7
4 7 9 3 8 7 9 3
9 10 9 10
2 8 1 2 4 1
⃝
c GfH&JPK 7
Constructing a maxheap – Algorithm

void buildHeap(int E[ ])
{ int i;
E.heapsize = E.length;
for (i = ⌊E.length/2⌋; i > 1; i −−)
Heapify(E,i);
}
at each call of Heapify(E, i) nodes i+1, . . . , E.length are all roots of maxheaps
This algorithm can also be written in a recursive way (easier to analyse!):

void constructHeap(int E[ ], int i)
{ if (2· i 6 E.length) constructHeap(E, 2· i);
if (2· i +1 6 E.length) constructHeap(E, 2· i +1);
Heapify(E,i);
}
⃝
c GfH&JPK 8
Constructing a heap – Example 1
1 4 1 4 Heapify(E,4)
Heapify(E,5)
2 3 3 2
1 1 3 3
4 5 16 6 7 4 5 16 6 7
2 9 10 2 9 10
8 9 10 8 9 10
14 8 7 14 8 7
4 1 3 2 16 9 10 14 8 7
10 elements
⃝
c GfH&JPK 9
Constructing a heap – Example 2

1 4 Heapify(E,3) 1 4 Heapify(E,2)
2 2 3
1 3 3 1 10
4 5 16 6 7 4 5 16 6 7
14 9 10 14 9 3
8 9 10 8 9 10
2 8 7 2 8 7
1 4 1 4 Heapify(E,1)
recursion
Heapify(E,5)
2 16 3 2 16 3
10 10
4 5 6 7 4 5 6 7
14 1 9 3 14 7 9 3
8 9 10 8 9 10
2 8 7 2 8 1
⃝
c GfH&JPK 10
Constructing a maxheap – Complexity intuition
constructHeap splits the problem of creating a heap of n elements

into two subproblems of creating heaps each with half the number of
elements, and applies Heapify to merge the two into a single maxheap
in log n steps.
W (n) = 2 · W (n/2) + log n
Apply the Master theorem!
⃝
c GfH&JPK 11
Constructing a maxheap – Complexity intuition
constructHeap splits the problem of creating a heap of n elements

into two subproblems of creating heaps each with half the number of
elements, and applies Heapify to merge the two into a single maxheap
in log n steps.
W (n) = 2 · W (n/2) + log n
Apply the Master theorem!
b = c = 2, so E = 1; and f (n) = log n, so f (n) ∈ O(nE−0.5). The

theorem says:
W (n) ∈ Θ(n)
⃝
c GfH&JPK 12
Heapsort – Algorithm
Idea: you know where the maximum element of a (max)heap is.
void heapSort(int E[ ])
{ int i;
buildHeap(E);
for (i = E.length; i > 2; i −−)
{ swap(E[i], E[1]);
E.heapsize = E.heapsize−1;
Heapify(E,1);
}
}
This algorithm sorts array E in nondecreasing order.
⃝
c GfH&JPK 13
Heapsort – Complexity analysis
Worst case complexity of Heapify is at most ⌊2· log n⌋ for n nodes
The worst case complexity of buildHeap is Θ(n)
This yields for heapsort:
∑
n−1 ∫ n
W (n) = 2·⌊log i⌋ 6 2 · (log e) ln x dx = 2·n· log n + c1·n + c2
i=1 1
No extra space is needed (tail recursion in Heapify can be removed):
heapsort sorts in Θ(n· log n) and sorts in-place
⃝
c GfH&JPK 14
Complexity of sorting algorithms
Algorithm Worst case Average Space usage
Insertion sort Θ(n2) Θ(n2) in place

Quicksort Θ(n2) Θ(n· log n) Θ(log n) extra
Mergesort Θ(n· log n) Θ(n· log n) Θ(n) extra
Heapsort Θ(n· log n) Θ(n· log n) in place
⃝
c GfH&JPK 15
Binary search tree

A binary search tree (BST) is
• a binary tree with keys in nodes, i.e. such that:
• the key at a node is greater than all keys in its left subtree
• the key at a node is less or equal than all keys in its right subtree
An inorder traversal of a binary search tree yields a sorted list of keys

2
6
3
3 7
7
2 5 9
6 9
two binary search trees

containing 2, 3, 5, 6, 7, 9 5
⃝
c GfH&JPK 16
Binary search tree

A node in a binary search tree has (at least) four features:
• a key – the “value” of the node

• a (pointer to a) left and right subtree (possibly empty)
• a (pointer to its) parent (which is empty for the root)
nil
12 parent
key A of A and B
left right
left parent right
child of A child of A
6 225
B C
⃝
c GfH&JPK 17
Searching for key k in a BST – Strategy
Exploit the binary search tree property:
• the key at a node is greater than all keys in its left subtree
• the key at a node is less or equal than all keys in its right subtree
Thus,
• if key of current node equals k, we are done

• If k < key of current node, we recursively search the left subtree
• If k > key of current node, we recursively search the right subtree
⃝
c GfH&JPK 18
Searching for key k in a BST – Algorithm
node bstSearch (node root, int k )

{ if (root == null || root.key == k )
return root;
else
if (k < root.key ) return bstSearch(root.left, k );
else return bstSearch(root.right, k );
}
The worst-case complexity is linear in the level d of the tree: Θ(d)

– for a chain-like tree structure with n nodes this amounts to Θ(n)
– if the BST is as balanced as possible this amounts to Θ(log n)
Note that such search procedure does not work for heaps; Why?
⃝
c GfH&JPK 19
15 15
10 < 15 10 > 5
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 14 17 31
6 lookup key 10 6
15 15
10 < 12
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 14 17 31
6 6
⃝
c GfH&JPK 20
Insertion of key k into a BST – Strategy
Search for key k in the tree T , exactly like we did before.
However, we do not stop if we find a node that has key k; in such a case
we try to insert k in the right subtree of that node.
So the search will end at a leaf.
Now replace the empty tree enccountered by a node with key k
⃝
c GfH&JPK 21
Insertion of node with key k into a BST – Algorithm
void bstIns (bsttree T, node z) // node z has key k

{ node x = T.root, y = null ;
while (x ! = null ) // search leaf position to insert z
{ y = x; // keep track of parent of x
if (z.key < x.key ) x = x.left; // go left
else x = x.right; // go right
}
z.parent = y ; // start inserting z
if (y == null ) T.root = z; // z is new root
else if (z.key < y.key ) y.left = z; // put z in right place
else y.right = z;
}
⃝
c GfH&JPK 22
Insertion of key k into a BST – Example
bstIns(T, 17)
30 30
35 13 35
13
38 7 23 38
7 23
18 28 18 28
17
⃝
c GfH&JPK 23
Deletion: the two simple cases

15 15
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 17 31
6 6
15 15
5 16 5
3 12 20 3 12 20
10 14 17 31 10 14 17 31
6 6
⃝
c GfH&JPK 24
Deletion: the more involved case
15 6 15
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 14 17 31
15
6 16
3 12 20
10 14 17 31
⃝
c GfH&JPK 25
Deletion from a BST – Strategy
Deleting node z from a BST is performed as follows:
• If z has no children, we modify the parent of z by replacing z by null
• If z has one child, we “splice out” z by linking its parent and its child
• If z has two children then:

– we determine its successor
– eliminate that
– replace z ’s key with that of its successor
First consider how to determine the successor of a node in a BST
⃝
c GfH&JPK 26
Finding the successor of a key – Algorithm

node bstSucc (node x)
{ if (x.right ! = null )
{ x = x.right; // go right
while (x.left ! = null ) x = x.left; // go as left as possible
return x; // return leftmost node in right subtree
}
else
{ node y = x.parent; // go up the tree from x
while (y ! = null && y.right == x) // until left child found
{ x = y; // go up the tree again
y = y.parent;
}
return y ;
} // this case is never used for deletion!
}
⃝
c GfH&JPK 27
Finding the successor of a key – Case 1

15 15
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 14 17 31
6 successor of 5 6
15 15
5 16 5 16 go right
and then
3 12 20 3 12 20
go left
10 14 17 31 10 14 17 31 as much
as possible
6 6
⃝
c GfH&JPK 28
Finding the successor of a key – Case 2

15 15
5 16 5 16
3 12 20 3 12 20
10 14 17 31 10 14 17 31
6 successor of 14 6
15 15
5 16 5 16
go up left
as far as
12 20
3 12 20 3 possible
10 14 17 31 10 14 17 31 until you
have to
6 6 go up right
⃝
c GfH&JPK 29
Deletion from a BST – Algorithm

node bstDel (bsttree T, node z) // z is node in T
{ node x, y ;
if (z.left == null || z.right == null ) y = z;
else y = bstSucc(z); // y is node to be eliminated
if (y.left ! = null ) x = y.left;
else x = y.right; // x is non-nil child of y (if any)
if (x ! = null ) x.parent = y.parent; // structurally delete node y
if (y.parent == null ) T.root = x;
else
if (y == (y.parent).left) // replace y by x
(y.parent).left = x;
else (y.parent).right = x;
if (y ! = z) z.key = y.key; // replace z ’s key by y ’s key
return y ; // i.e., logically delete node z
}
⃝
c GfH&JPK 30
Complexity of BST operations
Operation Time
bstSearch Θ(d)
bstSucc Θ(d)
bstMax Θ(d)
bstIns Θ(d)
bstDel Θ(d)
all operations are linear in the depth d of the bs-tree
this is log n if the tree is not too much “unbalanced”
balancing of binary trees amounts to using rotations
⃝
c GfH&JPK 31
Rotations - Concept
leftRotate(1) 2
1
2 1
rightRotate(2)
A C
B C A B
if original tree is a bst, then the rotated tree is a bst

moreover, the inorder traversal of original and rotated tree are identical!
⃝
c GfH&JPK 32
Rotate-left – Example
15
15
5 leftRotate(5) 12 20
3 12 20 5 14 17 31
10 14 17 31
3 10 13
13
⃝
c GfH&JPK 33
Rotate-left – Algorithm
void leftRotate (bsttree T, node x)
{ node y = x.right;
x.right = y.left;
(y.left).parent = x;
y.parent = x.parent;
if (x.parent == null )
T.root = y ;
else
if (x == (x.parent).left))
(x.parent).left = y ;
else (x.parent).right = y ;
y.left = x;
x.parent = y ;
}
this algorithm takes Θ(1) time in worst case
rightRotate is similar (symmetric) and is omitted here
⃝
c GfH&JPK 34

Lec4-Heaps and Search Trees

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec4-Heaps and Search Trees

Uploaded by

Copyright:

Available Formats

Heaps and Search trees

Lecture #4 of Algorithms, Data structures and Complexity

Plan for today

Then binary search trees

• what is a binary search tree?

The elements in an array E can be seen as the nodes of a binary tree.

• E[1] is the root of the tree

• E[2i] is the left- and E[2i+1] is the right-child of E[i]

A binary tree with a representation in an array has special properties.

• all leaves are on at most two adjacent levels

Previous slide shows a maxheap.

Inserting an element in a maxheap

To add an element x to a maxheap H with n elements

• put x into the first (left-most) open position

Doing such an insertion requires O(log n) comparisons

• the level k of a heap of n elements is bounded: n 6 2k+1 − 1

To re-order an array into a maxheap do not use repeated insertions, but

• but E[i] may be smaller than its children

• determine the maximum of the values E[i] and of its children

void Heapify(int E[ ], int i) // E[i] may be smaller than its children

Constructing a maxheap – Algorithm

This algorithm can also be written in a recursive way (easier to analyse!):

Constructing a heap – Example 1

Constructing a heap – Example 2

Constructing a maxheap – Complexity intuition

constructHeap splits the problem of creating a heap of n elements

W (n) = 2 · W (n/2) + log n

Apply the Master theorem!

Constructing a maxheap – Complexity intuition

constructHeap splits the problem of creating a heap of n elements

W (n) = 2 · W (n/2) + log n

Apply the Master theorem!

b = c = 2, so E = 1; and f (n) = log n, so f (n) ∈ O(nE−0.5). The

Idea: you know where the maximum element of a (max)heap is.

This algorithm sorts array E in nondecreasing order.

Heapsort – Complexity analysis

Worst case complexity of Heapify is at most ⌊2· log n⌋ for n nodes

The worst case complexity of buildHeap is Θ(n)

This yields for heapsort:

No extra space is needed (tail recursion in Heapify can be removed):

heapsort sorts in Θ(n· log n) and sorts in-place

Complexity of sorting algorithms

Algorithm Worst case Average Space usage

Insertion sort Θ(n2) Θ(n2) in place

Binary search tree

An inorder traversal of a binary search tree yields a sorted list of keys

two binary search trees

Binary search tree

• a key – the “value” of the node

Searching for key k in a BST – Strategy

Exploit the binary search tree property:

• if key of current node equals k, we are done

Searching for key k in a BST – Algorithm

node bstSearch (node root, int k )

The worst-case complexity is linear in the level d of the tree: Θ(d)

Insertion of key k into a BST – Strategy

Search for key k in the tree T , exactly like we did before.

So the search will end at a leaf.

Now replace the empty tree enccountered by a node with key k

Insertion of node with key k into a BST – Algorithm

void bstIns (bsttree T, node z) // node z has key k

Insertion of key k into a BST – Example

Deletion: the two simple cases