Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

3.2 B INARY S EARCH T REES

‣ BSTs
‣ ordered operations
‣ deletion
Algorithms F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Binary search trees

Definition. A BST is a binary tree in symmetric order.

root
a left link
a subtree
A binary tree is either:
・Empty. right child
・Two disjoint binary trees (left and right). of root

null links

Anatomy of a binary tree

parent of A and R
Symmetric order. Each node has a key, S
key
left link
and every node’s key is: of E E X
A R
・
9
Larger than all keys in its left subtree. value
C H associated
・Smaller than all keys in its right subtree. with R

keys smaller than E keys larger than E

Anatomy of a binary search tree

3
BST representation in Java

Java definition. A BST is a reference to a root Node.

A Node is comprised of four fields:

・A Key and a Value.
・A reference to the left and right subtree.
smaller keys larger keys

private class Node

{
private Key key; BST
private Value val;
Node key val
private Node left, right;
public Node(Key key, Value val)
{
left right
this.key = key;
this.val = val;
} BST with smaller keys BST with larger keys
}
Binary search tree
Key and Value are generic types; Key is Comparable
4
BST implementation (skeleton)

public class BST<Key extends Comparable<Key>, Value>

{
private Node root; root of BST

private class Node

{ /* see previous slide */ }

public void put(Key key, Value val)

{ /* see next slides */ }

public Value get(Key key)

{ /* see next slides */ }

public void delete(Key key)

{ /* see next slides */ }

public Iterable<Key> iterator()

{ /* see next slides */ }

5
Binary search tree demo

Search. If less, go left; if greater, go right; if equal, search hit.

successful search for H

E X

A R

C H

6
Binary search tree demo

Insert. If less, go left; if greater, go right; if null, insert.

insert G

E X

A R

C H

G M

7
BST search: Java implementation

Get. Return value corresponding to given key, or null if no such key.

public Value get(Key key)

{
Node x = root;
while (x != null)
{
int cmp = key.compareTo(x.key);
if (cmp < 0) x = x.left;
else if (cmp > 0) x = x.right;
else if (cmp == 0) return x.val;
}
return null;
}

Cost. Number of compares is equal to 1 + depth of node.

8
BST insert

Put. Associate value with key.

inserting L
S
E X
A R
Search for key, then two cases: C H
M
・ Key in tree ⇒ reset value.
search for L ends P

・Key not in tree ⇒ add new node. at this null link

S
E X
A R
C H
M
create new node L P

S
E X
A R
C H
M
reset links L P
on the way up

Insertion into a BST

9
BST insert: Java implementation

Put. Associate value with key.

concise, but tricky,
public void put(Key key, Value val) recursive code;
read carefully!
{ root = put(root, key, val); }

private Node put(Node x, Key key, Value val)

{
if (x == null) return new Node(key, val);
int cmp = key.compareTo(x.key);
if (cmp < 0)
x.left = put(x.left, key, val);
else if (cmp > 0)
x.right = put(x.right, key, val);
else if (cmp == 0)
x.val = val;
return x;
}

Cost. Number of compares is equal to 1 + depth of node.

10
 best case
H
C S
Tree shape A E R X

・Many BSTs correspond to same set of keys. typical case S

・ Number of compares for best case
search/insert
H
is equal to 1 + depth
E
of node.
X
C S A R
A E R X C H

best case typical case A worst case

H S
C
C S E X E
A E R X A R H
C H R
S
typical case
X
S A worst case
E X C BST possibilities
A R E
C H H
R
S
A Tree shape X
Remark. worst casedepends on order of insertion.
C
E BST possibilities
H 11
R
BST insertion: random order visualization

Ex. Insert keys in random order.

12
Correspondence between BSTs and quicksort partitioning

H T

D O S U

A E I Y

C M

Remark. Correspondence is 1-1 if array has no duplicate keys.

13
BSTs: mathematical analysis

Proposition. If N distinct keys are inserted into a BST in random order,

the expected number of compares for a search/insert is ~ 2 ln N.
Pf. 1-1 correspondence with quicksort partitioning.

Proposition. [Reed, 2003] If N distinct keys are inserted in random order,

expected height of tree is ~ 4.311 ln N.
How Tall is a Tree?
How Tall is a Tree?
Bruce Reed
Bruce Reed CNRS, Paris, France
CNRS, Paris, France reed@moka.ccr.jussieu.fr
reed@moka.ccr.jussieu.fr

ABSTRACT purpose of this note to

ABSTRACT Let H~ be the height of a random binaryofsearch
purpose tree toonprove
this note n have:
that for /3 -- ½ + ~ 3,
nodes. We
Let H~ be the height of a random binary search tree on n show that there exists constants
have: a = 4.31107...
THEOREM 1. E(H~)
nodes. We show that there existsa constants n d / 3 = 1.95... such that E(H~) = c~logn - / 3 1 o g l o g n +
a = 4.31107...
O(1), We also show that Var(H~) =THEOREM
a n d / 3 = 1.95... such that E(H~) = c~logn - / 3 1 o g l o g n +
O(1). 1. E(H~) = ~ l o g nVar(Hn) - / 3 1 o g l=o gO(1) .
n + O(1) a
O(1), We also show that Var(H~) = O(1). Var(Hn) = O(1) .
R e m a r k By the defini
Categories and Subject Descriptors nition given
R e m a r k By the definition of a, /3 = 7"g~" 3~ is more sug
The first d
But… Worst-case height is N. Categories and Subject Descriptors
E.2 [ D a t a S t r u c t u r e s ] : Trees
E.2 [ D a t a S t r u c t u r e s ] : Trees nition given is more suggestive ofaswhy
as we will see.
we will see.
this value is corre
For more information o
may consult [6],[7], [1],o
1. THE RESULTS For more information on random binary search trees,
(exponentially small chance when keys are inserted in random order)
1. THE RESULTS A binary search tree is a binary may tree consult
R
to each node
e m aaxe
r k drawn
[6],[7],of[1],
After I from
announced
R e m a r
[2], [9], [4], and [8].
which
these developed
k After I announ
an alternativ
results, Drmota(unpubl
we have associated
A binary search tree is a binary tree to each node of which a key; these keys some O(1) using completely
totally developed
v cannotanbealternative proof of the fact that Var(Hn)
we have associated a key; these keys axeordered
drawn set fromand some the key at
O(1) using
larger than
completely different proofs
techniques.illuminate 14differ
As our t
the key at
totally ordered set and the key at v cannot be larger thanits right child nor smaller than the key at its left decided to submit theha
j
child. proofs illuminate different aspects of the problem, we
the key at its right child nor smaller thanGiven a binary
the key at its leftsearch tree T and a new key k, we
decided to submit the journal and asked
versions to thatsame
the theyjourbe
child. Given a binary search treeinsert T and k into
a newT by keytraversing
k, we the tree starting at the root
ST implementations: summary

guarantee average case

ordered operations
implementation
ops? on keys
search insert search hit insert

sequential search
N N N/2 N no equals()
(unordered list)

binary search
lg N N lg N N/2 yes compareTo()
(ordered array)

BST N N 1.39 lg N 1.39 lg N next compareTo()

15
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Minimum and maximum

Minimum. Smallest key in table.

Maximum. Largest key in table.

S max()max
min()
min E X
A R
C H
M

Examples of BST order queries

Q. How to find the min / max?

18
Floor and ceiling

Floor. Largest key ≤ a given key.

Ceiling. Smallest key ≥ a given key.

floor(G)
S max()
min()
E X
A R
C H ceiling(Q)

M
floor(D)
Examples of BST order queries

Q. How to find the floor / ceiling?

19
Computing the floor

Case 1. [k equals the key at root] finding floor(G)

S
The floor of k is k. E X
A R
C H
G is less than S so
Case 2. [k is less than the key at root] M floor(G) must be
on the left
The floor of k is in the left subtree. S
E X
A R
Case 3. [k is greater than the key at root] C H
The floor of k is in the right subtree G is greater than E so
M
floor(G) could be
(if there is any key ≤ k in right subtree); on the right S
otherwise it is the key in the root. E X
A R
C H
M
floor(G)in left
subtree is null
S
E X
A result R
C H
M
20
Computing the floor function
Computing the floor

finding floor(G)
S
public Key floor(Key key) E X
{ A R
Node x = floor(root, key); C H
G is less than S so
if (x == null) return null; M floor(G) must be
return x.key; on the left
} S
private Node floor(Node x, Key key) E X
{ A R
if (x == null) return null; C H
int cmp = key.compareTo(x.key); M
G is greater than E so
floor(G) could be
if (cmp == 0) return x; on the right S
E X
if (cmp < 0) return floor(x.left, key); A R
C H
Node t = floor(x.right, key);
M
if (t != null) return t;
floor(G)in left
else return x; subtree is null
S
}
E X
A result R
C H
M
21
Computing the floor function
Subtree counts

In each node, we store the number of nodes in the subtree rooted at that
node; to implement size(), return the count at the root.

node count 8

6 S 1
2 E 3 X
A 1 2 R
C H 1
M

Remark. This facilitates efficient implementation of rank() and select().

22
BST implementation: subtree counts

private class Node public int size()

{ { return size(root); }
private Key key;
private Value val; private int size(Node x)
private Node left; {
private Node right; if (x == null) return 0;
private int count; return x.count; ok to call
when x is null
} }

number of nodes in subtree

private Node put(Node x, Key key, Value val)

{
if (x == null) return new Node(key, val, 1);
int cmp = key.compareTo(x.key);
if (cmp < 0) x.left = put(x.left, key, val);
else if (cmp > 0) x.right = put(x.right, key, val);
else if (cmp == 0) x.val = val;
x.count = 1 + size(x.left) + size(x.right);
return x;
}
23
Rank

Rank. How many keys < k ?

node count 8

6 S 1
Easy recursive algorithm (3 cases!) 2 E 3 X
A 1 2 R
C H 1
M

public int rank(Key key)

{ return rank(key, root); }

private int rank(Key key, Node x)

{
if (x == null) return 0;
int cmp = key.compareTo(x.key);
if (cmp < 0) return rank(key, x.left);
else if (cmp > 0) return 1 + size(x.left) + rank(key, x.right);
else if (cmp == 0) return size(x.left);
}

24
Inorder traversal

・Traverse left subtree.

・Enqueue key.
・Traverse right subtree.
public Iterable<Key> keys()
{ BST
Queue<Key> q = new Queue<Key>();
key val
inorder(root, q);
return q;
} left right

private void inorder(Node x, Queue<Key> q)

BST with smaller keys BST with larger keys
{
if (x == null) return;
smaller keys, in order key larger keys, in order
inorder(x.left, q);
q.enqueue(x.key); all keys, in order
inorder(x.right, q);
}

Property. Inorder traversal of a BST yields keys in ascending order.

25
BST: ordered symbol table operations summary

sequential binary
BST
search search

search N lg N h

insert N N h
h = height of BST
(proportional to log N
min / max N 1 h
if keys inserted in random order)

floor / ceiling N lg N h

rank N lg N h

select N 1 h

ordered iteration N log N N N

order of growth of running time of ordered symbol table operations

26
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 ST implementations: summary

guarantee average case

ordered operations
implementation
iteration? on keys
search insert delete search hit insert delete

sequential search
N N N N/2 N N/2 no equals()
(linked list)

binary search
lg N N N lg N N/2 N/2 yes compareTo()
(ordered array)

BST N N N 1.39 lg N 1.39 lg N ??? yes compareTo()

Next. Deletion in BSTs.

29
BST deletion: lazy approach

To remove a node with a given key:

・Set its value to null.
・Leave key in tree to guide search (but don't consider it equal in search).
E E

A S A S
delete I
C I C ☠ tombstone

H R H R

N N

Cost. ~ 2 ln N' per insert, search, and delete (if keys in random order),
where N' is the number of key-value pairs ever inserted in the BST.

Unsatisfactory solution. Tombstone (memory) overload.

30
Deleting the minimum

To delete the minimum key:

・Go left until finding a node with a null left link. go left until
reaching null
S
X
・Replace that node by its right link. left link
A
E
R
・Update subtree counts. C H
M

return that S
node’s right link
E X
A R
C H
public void deleteMin()
available for M
{ root = deleteMin(root); }
garbage collection

private Node deleteMin(Node x) update links and node counts

after recursive calls
{ 7
5 S
if (x.left == null) return x.right;
E X
x.left = deleteMin(x.left);
C R
x.count = 1 + size(x.left) + size(x.right);
H
return x;
M
}
Deleting the minimum in a BST

31
Hibbard deletion

To delete a node with key k: search for node t containing key k.

Case 0. [0 children] Delete t by setting parent link to null.

deleting C
update counts after
recursive calls 7
S 5 S
S
E X 1 E X
E X
A R A R
A R
H H
C H
M M
M replace with
node to delete null link
available for
garbage
collection
C

32
Hibbard deletion

To delete a node with key k: search for node t containing key k.

Case 1. [1 child] Delete t by replacing parent link.

deleting R
update counts after
recursive calls 7

S S 5 S
E X E X E X
A R A H A H
C H C M C M
node to delete
M replace with
child link available for
garbage
collection
R

33
Hibbard deletion
deleting E

node to delete
S
To delete a node with key k: search for node tEcontainingX key k.
A R
C H search for key E
Case 2. [2 children] M

・Find successor x of t. t
S
x has no left child

・Delete the minimum in t's right subtree.

A
E
x R
X don't garbage collect x
but

・Put x in t's spot.

deleting E
C H still a BST
successor
min(t.right)
M
go right, then
node to delete go left until
S reaching null
E X left link S
A R x E X
C H search for key E t.left deleteMin(t.right)
H
M
A R
t C M
S
7
E X
x 5 S
A R
successor H X
C H
min(t.right) A R
M
go right, then C M update links and
go left until node counts after
reaching null recursive calls
left link S
E X Deletion in a BST
x 34
t.left deleteMin(t.right)
H
Hibbard deletion: Java implementation

public void delete(Key key)

{ root = delete(root, key); }

private Node delete(Node x, Key key) {

if (x == null) return null;
int cmp = key.compareTo(x.key);
if (cmp < 0) x.left = delete(x.left, key); search for key
else if (cmp > 0) x.right = delete(x.right, key);
else {
if (x.right == null) return x.left; no right child
if (x.left == null) return x.right; no left child

Node t = x;
x = min(t.right); replace with
successor
x.right = deleteMin(t.right);
x.left = t.left;
}
update subtree
x.count = size(x.left) + size(x.right) + 1;
counts
return x;
}

35
Hibbard deletion: analysis

Unsatisfactory solution. Not symmetric.

Surprising consequence. Trees not random (!) ⇒ sqrt (N) per op.
Longstanding open problem. Simple and efficient delete for BSTs.
36
 ST implementations: summary

guarantee average case

ordered operations
implementation
iteration? on keys
search insert delete search hit insert delete

sequential search
N N N N/2 N N/2 no equals()
(linked list)

binary search
lg N N N lg N N/2 N/2 yes compareTo()
(ordered array)

BST N N N 1.39 lg N 1.39 lg N √N yes compareTo()

other operations also become √N

if deletions allowed

Next lecture. Guarantee logarithmic performance for all operations.

37
 3.2 B INARY S EARCH T REES
‣ BSTs
‣ ordered operations
‣ deletion
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

3.2 B INARY S EARCH T REES

‣ BSTs
‣ ordered operations
‣ deletion
Algorithms F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu