2 AdvancedDataStructures

Advanced Data Structures
CHAPTER 2
Outlines
• Binary search trees
• B-trees
• Heaps and priority queues
• Skip list
Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 2

Binary Search Trees
Binary Search Trees: Nodes
left key parent right
internal node
left key parent right left key parent right
root node leaf node

Property of Binary Search Trees
For any node n in a binary search tree T,
• if p is a node in the left subtree of n, then p.key
≤ n.key;
• if q is a node in the right subtree of n, then
n.key ≤ q.key.
T
n
p q

Example of Binary Search Trees
12
10
8 17
6 19
4 9
4 9 12 20
5

Inorder Traversal of Binary Search Trees
12
inorder(T: node)
if T is not null, 8 17
then inorder(T.left)
print(T.key) 4 9
inorder(T.right)
return.
5
} 10
6 19
4 9 12 20

Search in Binary Search Trees
12
search(n: node, k: key)
if n is null or n.key = k, 8 17
then return(n)
if k < n.key, 4 9
then search(n.left, k)
else search(n.right, k) 5
return. 10
6 19
4 9 12 20

Insertion in Binary Search Trees
insert(T: node, z: node) insert(T: node, z: node)
y = null if T is null
x=T The tree T is empty.
then T.root = z
while x is not null return
y=x if key[z] < key[T]
if z.key < x.key then if T.left is null
then x = x.left Found the place, and then T.left = z
else x = x.right insert as the right
left child.
child. z.p = T
z.parent= y else insert(x.left, z)
if y = NIL else if T.right is null
then T.root= z then x.right = z
else if z.key < y.key z.p = T
then y.left= z else insert(x.right, z)
else y.right = z return
Deletion in Binary Search Trees
• See textbook

B-trees
B-trees
• Index structures for large amount of data.
• All data cannot be resided in the main
memory.
• Data is stored in a disk.

Disk Operations
• In a disk, data is
organized in disk
pages.
• To read data in a
disk,
• The disk rotates, and
the R/W head moves
to the page
containing the data.
• Then, the whole disk
page is read.
From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.
B-trees: Nodes
Internal node
number false
of keys key1 key2 key3 keyn
leaf
Pointers to other nodes
leaf node
number true
of keys key1 key2 key3 keyn
leaf
Pointers to data pages

Properties of B-trees
For any node n in a B-tree T
n p key1 … keyi keyi+1 … false
c q key1 … keyi keyi+1 … false
• n.keyi ≤ c.key1 ≤ c.key2 ≤ … ≤ c.keyq ≤ n.keyi+1

• If n is a leaf node, the depth of n is h, where h
is the tree’s height.
Properties of B-trees
• The minimum degree, t, of the B-tree is the lower bound o
n the number of keys a node can contain. (t ≥ 2)
• Every node other than the root must have at least t − 1 keys
.
• Every internal node other than the root thus has at least t ch
ildren.
• If the tree is nonempty, the root must have at least one key.
• Every node can contain at most 2t − 1 keys. Therefore, an i
nternal node can have at most 2t children.
• We say that a node is full if it contains exactly 2t − 1 keys.

Search in B-trees
65
11 17 68 71
6 8
12 16
75 88
26 60 69 70
66 67
Search in B-trees
B-TREE-SEARCH(x, k)
i=1
► find a proper child
while i ≤ x.n and k > x.keyi do i =i + 1
if i ≤ x.n and k = x.keyi
then return (x, i )
if x.leaf
then return NIL
► recursively search in the proper subtree
else DISK-READ (x.ci)
return B-TREE-SEARCH (x.ci, k)

Creating B-trees
B-TREE-CREATE(T )
x = ALLOCATE-NODE()
x.leaf = TRUE
x.n = 0
DISK-WRITE(x)
T.root = x

Insertion in B-trees
17 11
8
17 26 65
26 65
11

6
17
11
8 11 12 16
12 16
26 65

60
11 17 65
6 8
12 16
26 65 65 71
71
66
11 17 65 71
6 8
12 16
75 88
26 60
67 71 75 88
70
11 17 65 71
6 8
12 16
75 88
26 60 69
66 67 68 69
65 70
11 17 11 17 65 7171 71
68
6 8
12 16
75 88
26 60 69
66 67
65
11 17 68 71
6 8
12 16
75 88
26 60 69 70
66 67
B-TREE-INSERT(T, k)
r = T.root
if r.n = 2t − 1
► The root node r is full, then create a new root node
then s = ALLOCATE-NODE()
T.root = s; s.leaf = FALSE
s.n = 0; s.c1 = r
► Split old root into two, and put under the new root
B-TREE-SPLIT-CHILD(s, 1, r)
B-TREE-INSERT-NONFULL(s, k)
else B-TREE-INSERT-NONFULL(r, k)
Splitting Nodes in B-trees
B-TREE-SPLIT-CHILD(x, i, y) ►Move up other 1/2 of old node
►Create a new node for j = x.n + 1 downto i + 1
z = ALLOCATE-NODE() do x.cj+1 = x.cj
z.leaf = y.leaf x.ci+1 = z
z.n = t − 1 for j = x.n downto i
►Move 1/2 of old node to new
for j = 1 to t − 1 do x.keyj+1 = x.keyj
do z.keyj = y.keyj+t x.keyi = y.keyt
if not y.leaf x.n = x.n + 1
then for j = 1 to t DISK-WRITE(y)
do z.cj = y.cj+t DISK-WRITE(z)
y.n = t − 1 DISK-WRITE(x)
Binary Heaps
Binary Heaps
• Binary heap is an array
• can be viewed as a nearly
complete binary tree
1
• each node of the tree
corresponds to an element left 2 right 3
of the array that stores the
value in the node. 4 5 6 7
• The tree is completely 8 9 10 11 12 13
filled on all levels except

possibly the lowest, which
is filled from the left up to
a point. 1 2 3 4 5 6 7 8 9 10 11 12 13
Representation of Biinary Heaps
• An array A that represents a heap is an object
with two attributes:
• length[A], which is the number of elements in
the array
• heap-size[A], the number of elements in the
heap stored within array A.
heap-size length
1 2 3 … … n x … x

Properties of Binary Heaps
• If a heap contains n elements, its height is lg2
n.
• In a max-heaps
For every non-root node i, A[PARENT(i)] ≥ A[i]
• In a min-heaps
For every non-root node i, A[PARENT(i)] ≤ A[i]

Examples
15
Max heap
12 8
9 4 6 5 5
3 6 2 1
9 7
Min heap 11 19 20 18
15 12
Heapify
12 8
9 4 6 5
3 6 2 1

Heapify
12
7
12
7 8
9 4 6 5
3 6 2 1

Heapify
12
7
9 8
7
9 4 6 5
3 6 2 1

Heapify
MAX-HEAPIFY(A, i )
l = LEFT(i )
r = RIGHT(i )
► Find node containing the largest value among i and its valid children.
if l ≤ heap-size[A] and A[l] > A[i ]
then largest = l
else largest = i
if r ≤ heap-size[A] and A[r] > A[largest]
then largest = r
► Swap the largest node and node i if the node i is not the largest
if largest  i
then exchange A[i ] and A[largest]
MAX-HEAPIFY(A, largest)
Time Complexity for Heapify
• worst-case
• The left subtree is one-level taller than the right
subtree, and both are complete binary trees
• The left subtree contains (2n-1)/3 nodes and the right
subtree contains (n-2)/3.
Let T(n) be time spent to heapify an n-node heap.
T(n) = T((2n-1)/3) + k
T(n)  Ο(lg n)
T(n)  Ο(h)
• since the height h of a heap is in Ο(lg n)
Building Max-heaps: BUILD-MAX-HEAP(A)
2
11 3
7
4
9 11
10
52 6 7
3
8 9
4 10
2 11
5

1
11
11
10
1 7
9 10
5
1 6 3
8 4 2 5
1

heap-size[A] = length[A]
► Start from the rightmost node in the level above leaves
for i = length[A]/2 downto 1
do MAX-HEAPIFY(A, i )
1
2 3
4 5 6 7
8 9 10 11
Time to Build Max-heaps
Let n be the heap size,
T(n) be the time spent in BUILD-MAX-HEAP for heap of size n,
t(h) be the time spent in MAX-HEAPIFY for the heap of height h,
k(h, n) be the number of subtrees of height h in the heap of size n.
lg n lg n lg n
T(n)   t(h) k(h, n) =  c h n/2h+1 = c n  (h/2h)
h=0 h=0 2 h=0
lg n
T(n) = O(n  (h/2h))
h=0

Since  (h/2h)) = 2, T(n) = O(n).
h=0

Heapsort
HEAPSORT(A)
BUILD-MAX-HEAP(A)
for i = length[A] downto 2
► Swap the max node with last in heap, and
► heapify heap, excluding the last (old max)
do exchange A[1] and A[i ]
heap-size[A] = heap-size[A] − 1
MAX-HEAPIFY(A, 1)

Time for Heapsort
• The call to BUILD-MAX-HEAP takes O(n)
• Each of the n − 1 calls to MAX-HEAPIFY
takes O(lg n)
• HEAPSORT takes O(n lg n).

Priority Queues
Priority Queues
• A priority queue is a max-heap with
operations for queues.
• Insert
• Extract-max
• Increase-key
HEAP-MAXIMUM(A)
return A[1]
Extract-Max In Priority Queues
HEAP-EXTRACT-MAX(A)
if heap-size[A] < 1
then error “heap underflow”
► Take the maximum element from the root
max = A[1]
► Move the last element in the heap to the root
A[1] = A[heap-size[A]]
heap-size[A] = heap-size[A] − 1
► Heapify
MAX-HEAPIFY(A, 1)
return max
Increase Key
HEAP-INCREASE-KEY(A, i, key)
if key < A[i ]
then error “new key < current key”
A[i ] = key
► Swap the increased element with its parent up toward
► the root until it is not greater than its parent
while i > 1 and A[PARENT(i)] < A[i ]
do exchange A[i] and A[PARENT(i)]
i = PARENT(i)
Insert
MAX-HEAP-INSERT(A, key)
► Insert the minimum element
heap-size[A] = heap-size[A] + 1
A[heap-size[A]]=−∞
► Increase the minimum element
HEAP-INCREASE-KEY(A, heap-size[A], key)

Binomial Heaps
Why Binomial Heaps
• Union two binary heaps takes (n).
• Can it be reduced?
• Use binomial heaps
• A binomial heap is a binomial tree that has
the property of heaps.

Binomial Trees
• The binomial tree Bk is an ordered tree
which consists of two binomial trees Bk−1
that are linked together.
• the root of one is the leftmost child of the root
of the other.
• The binomial tree B0 consists of a single
node.

Jaruloj Chongstitvatana
Examples
Chapter 2: Advanced Data Structures
From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein,

53
Introduction to Algorithms, MIT Press, 2001.

Examples
level

Properties of Binomial Trees
For the binomial tree Bk,
• the number of nodes = 2k
• the height of the tree = k
• the number of nodes at depth i = kCi
• the degree of root = k (maximum degree)
• if the children of the root are numbered from
left to right by k − 1, k − 2, . . . , 0, child i is
the root of a subtree Bi.
See the proof in the textbook.
Binomial Heaps
• A binomial heap is a set of binomial trees
with heap properties.
head[H] 10 1 6
B0 12 25 8 14 29
18 11 17 38
B2 27
B3
Nodes in Binomial Heaps
parent
key
degree
child sibling

Examples of Nodes in Binomial Heaps
\
10 1 6 6
3
8 14 29 \
12 25
18 11 17 38 8 14
2 1
27
11 17 38
1 0 0
\ \ \ \

Find Minimum in Binomial Heaps
BINOMIAL-HEAP-MINIMUM(H)
y = NIL
head
x = H.head 10 1 6
min = ∞
while x  NIL 12 25 8 14 29
do if x.key < min
then min = x.key 18 11 17 38
y=x
x = x.sibling 27
return y
Maximum number of trees = lg n + 1,
the running time of BINOMIAL-HEAP-MINIMUM = O(lg n)
Link Binomial Heaps
• Make node y the new B2 8 B2 6
head of the linked list of
11 17 14 29
node z’s children in O(1)
time.
BINOMIAL-LINK(y, z) 27 y 38 z
y.p = z
y.sibling = z.child B3
z.child = y
z.degree = z.degree + 1

Union Binomial Heaps
• Link the roots of all trees in both heaps, in a
non-decreasing order of the degree.
• Go through each tree in the heap, and

combine two trees of equal degree in the
heaps, according to one of the following four
cases.

Union Binomial Heaps: case 1
• x.degree < next-x.degree
• move up to the next tree.
• x.degree = next-x.degree = next-x.sibling.degree
• move up to the next tree (and they will be linked in
next step)
R. Rivest, and C. Stein, Introduction

From: Cormen, T., C. Leiserson,
to Algorithms, MIT Press, 2001.

• x.degree = next-x.degree
• Link x and next-x.
• Use the node with smaller key as the root.
x.key  next-x.key
x.key<next-x.key
Binomial-Heap-Union
BINOMIAL-HEAP-UNION(H1, H2) ► Cases 1 and 2
H = MAKE-BINOMIAL-HEAP() then prev-x = x
H.head = x = next-x
BINOMIAL-HEAP-MERGE(H1, H2) else if x.key ≤ next-x.key
► Case 3
Free the objects H1 and H2 but not then x.sibling = next-x.sibling
the lists they point to BINOMIAL-LINK(next-x, x)
► Case 4
if H.head = NIL else if prev-x = NIL
then return H then H.head = next-x
prev-x = NIL else prev-x.sibling = next-x
x = H.head BINOMIAL-LINK(x,next-x)
next-x = x.sibling x = next-x
while next-x  NIL next-x = sibling[x]
do if (x.degree  next-x.degree) or return H
(next-x.sibling  NIL and
next-x.sibling.degree=x.degree)

Binomial Heaps: Insert a Node
BINOMIAL-HEAP-INSERT(H, x)
H’ = MAKE-BINOMIAL-HEAP()
x.p = NIL
x.child = NIL
x.sibling = NIL
x.degree = 0
H’.head = x
H = BINOMIAL-HEAP-UNION(H, H’ )

Fibonacci Heaps
Why Fibonacci Heaps
• Every insertion in a binomial heap requires
the structure adjustment.
• Take long time (O(lg n)) for that.
• Do we need to adjust it that often?
• No, if you use Fibonacci heaps instead!

Fibonacci Heaps
• A Fibonacci heap is a collection of min-
heap-ordered trees.
• The trees in a Fibonacci heap are not
constrained to be binomial trees.
• Trees within Fibonacci heaps are rooted but
unordered. (unlike trees within binomial
heaps, which are ordered)
• Circular, doubly linked lists are used in
Fibonacci heaps.
Fibonacci Heaps:Example

Properties of unordered binomial tree Uk
• Number of nodes = 2k
• Height of the tree = k
• Number of nodes at depth i = kCi
• The degree of root = k, which is greater than
that of any other node
• The children of the root are roots of subtrees
U0,U1, . . . ,Uk−1 in some order.

Nodes in Fibonacci Heaps
parent
key
degree
mark
left right
child

Links in Fibonacci Heaps

Fibonacci Heaps: Insert a node
• Link the node in the root list of H
• Update min[H] if necessary

Consolidate Heaps
• Find two roots x and y in the root list with th
e same degree, where key[x] ≤ key[y].
• Link y to x, using FIB-HEAP-LINK, i.e. rem
ove y from the root list, and make y a child o
f x.
• The field degree[x] is incremented, and the
mark on y, if any, is cleared.

Time Complexity
Procedure Binary Heap Binomial Heap Fibonacci Heap
(worst-case) (worst-case) (amortized)
Make-heap (1) (1) (1)
Insert (lg n) Ο(lg n) (1)
Minimum (1) Ο(lg n) (1)
Extract-min (lg n) (lg n) Ο(lg n)
Union (n) Ο(lg n) (1)
Decrease-key (lg n) (lg n) (1)
delete (lg n) (lg n) Ο(lg n)

Skip Lists
Skip Lists
• Alternative to binary search trees
• Probabilistic data structure
• Easy to implement
• Good performance in average-case

Structure of Skip Lists
• Max. level H of the skip list
• An array H of header pointers
• A set of nodes such that a node N contains
• a key k and a set of values v,
• the height of h node N, where h<=H, and
H • an array of h pointers.
4 7 nil
2 6 11
9 12
Skip List v.s. Binary search tree
4 7 nil
2 6 11
9 12
4 11
2 6 9 12
Search in a Skip List
Search(list, searchkey)
cn = list.head ►Start at the list head
►Start skip from top level to bottom
for i=list.level downto 1
do ► Forward until key of node exceeds search key
while ((cn.fwd[i]).key < searchkey)
do cn = cn.fwd[i]
cn = cn.fwd[1]
if cn.key = searchkey
then return(cn)
else return(null)
H
4 7 nil
2 6 11
9 12
Search for 11
4 11
2 6 9 12
H
4 7 nil
2 6 11
9 12
Search for 12
7
4 11
2 6 9 12
Insert in a Skip List
Insert 5
update[3]
cn update[1]
update[2]
cn
4 7 nil
2 6 11
9 12

Insert in a Skip List
cn = list.head update[i].fwd[i] = node
►Search for a place to insert ► Insert the node from level 2 up
for i=list.maxlevel downto 1 ► Toss a coin
do while ((cn.fwd[i]).key < while (rand() >.5)
node.key) do addLevel(node)
do cn = cn.fwd[i] i++
update[i]=cn ► Update link in level i
► The key is already in the list if (i <= list.level)
if cn.fwd[1].key = node.key then node.fwd[i] =
then return(false) update[i].fwd[i]
► Insert the node in level 1 update[i].fwd[i] = node
i=1 else addLevel(list.head)
node.fwd[i] = update[i].fwd[i] (list.head).fwd[i] = node

Delete a node
• Only remove the node, and connect the lost
links.
• The list can become uneven.
• But it is more efficient than deletion in
binary search trees.

2 AdvancedDataStructures

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2 AdvancedDataStructures

Uploaded by

Copyright:

Available Formats

Advanced Data Structures

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 2

left key parent right

left key parent right left key parent right

root node leaf node

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 4

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 5

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 6

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 7

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 8

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 10

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 12

Pointers to data pages

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 14

c q key1 … keyi keyi+1 … false

• n.keyi ≤ c.key1 ≤ c.key2 ≤ … ≤ c.keyq ≤ n.keyi+1

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 16

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 18

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 19

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 20

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 21

• The tree is completely 8 9 10 11 12 13

filled on all levels except

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 31

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 32

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 34

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 35

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 36

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 39

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 40

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 42

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 43

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 44

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 49

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 51

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 52

Chapter 2: Advanced Data Structures

From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein,

Introduction to Algorithms, MIT Press, 2001.

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 54

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 57

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 58

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 60

• Go through each tree in the heap, and

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 61

R. Rivest, and C. Stein, Introduction

to Algorithms, MIT Press, 2001.

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 65

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 66

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 68

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 70

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 71

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 72

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 73

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 74

Jaruloj Chongstitvatana Chapter 2: Advanced Data Structures 75

Insert (lg n) Ο(lg n) (1)

Minimum (1) Ο(lg n) (1)

Extract-min (lg n) (lg n) Ο(lg n)

Union (n) Ο(lg n) (1)

Decrease-key (lg n) (lg n) (1)

delete (lg n) (lg n) Ο(lg n)