You are on page 1of 180

6.

Trees
Data Structure and Algorithms

Dr. Udaya Raj Dhungana


Assist. Professor
Pokhara University, Nepal
Guest Faculty
Hochschule Darmstadt University of Applied Sciences, Germany
E-mail: udaya@pu.edu.np and udayas.epost@gmail.com

Overview
Trees (7hrs)
1.Concept and definitions
2.Basic operation in binary tree
3.Binary search tree and insertion /deletions
4.Binary tree traversals (pre-order, post-order and in-order )
5.tree height level and depth
6.Balanced trees
7.AVL balanced trees
8.Balancing algorithm
9.The Huffman algorithm
10.Game tree
11.B- Tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 2

Why Tree Data Structure?


• You studied various data structures:
1.Array
• Linear static data structure that can be accessed randomly
2.Linked list
• Linear dynamic data structure that can be accessed
sequentially.
3. Stack and queue
• Linear linear data structure
• Are implemented either by array or linked list. So, these also
have the limitation of array and linked list

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 3

Why Tree Data Structure?


• Suppose you are a given a task to search an element in such linear
data structures:
• Array
• Linked list
• If they contain n items, then time complexity to search an item (any
operation) in worst case is O(n) which is undesirable.
• Solution = ?
• Don’t use the linear data structure
• Store the items in non-linear data structure like Tree
• Solution is therefore using a Tree
• Tree structures enable efficient access and efficient update to
large collections of data (How?). 

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 4

Why Tree Data Structure?

a) List holding 7 items in linear fashion

Check how many computations may


need to search node G in these two
different data structure?
- For list: 7 comparisons [ie O(n)]
- For tree: 3 comparisons [ie O(log n) ] b) Tree holding 7 items in
hierarchy fashion

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 5


Tree Data Structure


• A tree is a non-linear data structure which organizes data in
hierarchical structure in a recursive manner.
• It is a collection of nodes connected by directed (or undirected) edges.
• A tree can be empty with no nodes or a tree is a structure consisting of
• one node called the root and
• zero or one or more subtrees, each of which is also a tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 6


Tree Data Structure


• A tree T is a finite set of one or more nodes such that there is one
designated node R, called the root of T. If the set (T−{R}) is not
empty, these nodes are partitioned into n>0 disjoint sets T0, T1, …,
Tn−1, each of which is a tree, and whose roots R1,R2,...,Rn,
respectively, are children of R.
R

R1 R2
T0 T1

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 7


Tree Data Structure
• A tree has following general properties:
• One node is distinguished as a root;
• Every other node is connected by a (directed) Subtree Subtree
edge from exactly one other node; A direction
is: parent -> children
• The sub-structures are themselves (smaller)
trees,
• A unique path traverses from the root to each
node.
• Path from A to J is A-B-E-J
• Path from C to K is C-G-K

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 8

Recursive Data Structures


• A recursive data structure is a data structure that is partially
composed of smaller or simpler instances of the same data structure.
• For example, linked lists and binary trees can be viewed as
recursive data structures.
• A list is a recursive data structure because a list can be defined as
either
• (1) an empty list or
• (2) a node followed by a list.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 9

Recursive Data Structures


• A binary tree is typically defined as
• (1) an empty tree or
• (2) a node pointing to two binary trees, one its left child and
the other one its right child.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 10


Tree- Terminologies
1.Root:
• In a tree, every individual element is called a node.
• The first node (the top most node in the hierarchy) is called
as Root Node. 
• Root of a tree is a special and first node from which the tree
originates and it therefore doesn’t have parent.
• A tree can have only one root.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 11

Tree- Terminologies
2.Edge:
• A connecting link between any two nodes is called as edge.
• The arrowheads on the edges indicate the direction of the
connection
• In a tree with 'N' number of nodes there will be a maximum of
'N-1' number of edges.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 12

Tree- Terminologies
3.Parent:
• The node which is a predecessor of any node is called as PARENT
NODE.
• Parent node can also be defined as "The node which has child /
children".

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 13

Tree- Terminologies
4.Child:
• The node which is descendant (successor or offspring) of any
node is called as CHILD Node.
• Each node can have arbitrary number of children (or subtrees).
• In a tree, all the nodes except root are child nodes.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 14

Tree- Terminologies
5.Siblings:
• The nodes which belong to same Parent are called as SIBLINGS.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 15

Tree- Terminologies
6.Leaf:
• The nodes with no children are called leaves nodes or external
nodes.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 16

Tree- Terminologies
7.Internal nodes:
• The node which has at least one child is called as INTERNAL
Node

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 17

Tree- Terminologies
8.Degree:
• The total number of children of a node is called as DEGREE of
that Node. 
• The highest degree of a node among all the nodes in a tree is
called as 'Degree of Tree'

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 18

Tree- Terminologies
9.Level:
• In a tree each step from top to bottom is called as a Level and
the Level count starts with '0' and incremented by one at each
level (Step).
• The root node is said to be at Level 0 and the children of root
node are at Level 1 and the children of the nodes which are at
Level 1 will be at Level 2 and so on.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 19

Tree- Terminologies
10.Height:
• The total number of edges from leaf node to a particular node in
the longest path is called as HEIGHT of that Node.
• height of the root node is said to be height of the tree
• height of all leaf nodes is '0'

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 20

Tree- Terminologies
11.Depth:
• The total number of egdes from root node to a particular node is
called as DEPTH of that Node
• the total number of edges from root node to a leaf node in the
longest path is said to be Depth of the tree
• depth of the root node is '0'

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 21

Tree- Terminologies
12.Path:
• The sequence of Nodes and Edges from one node to another
node is called as PATH between those two Nodes
• Length of a Path is total number of nodes in that path
• the path A - B - E - J has length 4.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 22

Tree- Terminologies
13.Sub tree:
• Each child from a node forms a subtree recursively.
• A child of a parent node is a subtree
• Every node has zero or one or more subtree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 23

General Tree
• A general tree is a tree where each node may have zero or more
children
• General trees are used to model applications such as file systems,
web page structure etc.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 24

General Tree
• Application: File System

Fig: Linux File System


Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 25
General Tree
• Application: File System
• One example of a tree structure that you probably use every day is
a file system. In a file system, directories, or folders, are
structured as a tree. In the file system tree, you can follow a path
from the root ( / ) to any directory. That path will uniquely identify
that subdirectory (and all the files in it). Another important
property of trees, derived from their hierarchical nature, is that
you can move entire sections of a tree (called a subtree) to a
different position in the tree without affecting the lower levels of
the hierarchy. For example, we could take the entire subtree
starting with /etc/, detach etc/ from the root and reattach it
under usr/.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 26


General Tree
• Application: Representing an Organization
• Many organizations are hierarchical in nature, such as the military and most
businesses. Consider a company with a president and some number of vice
presidents who report to the president. Each vice president has some number
of direct subordinates, and so on. If we wanted to model this company with a
data structure, it would be natural to think of the president in the root node
of a tree, the vice presidents at level 1, and their subordinates at lower
levels in the tree as we go down the organizational hierarchy.
• Because the number of vice presidents is likely to be more than two, this
company's organization cannot easily be represented by a binary tree. We
need instead to use a tree whose nodes have an arbitrary number of children.
Unfortunately, when we permit trees to have nodes with an arbitrary number
of children, they become much harder to implement than binary trees. To
distinguish them from binary trees, we use the term general tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 27


General Tree
• Application: Representing an Organization
• Administrative Organization: as In Figure 10.1 which illustrates
(part) of the administrative structure governing the University of
Liverpool

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 28


General Tree
• Application: The HTML tags used to create the web page
• The HTML source code and the tree accompanying the source illustrate
another hierarchy. Notice that each level of the tree corresponds to a level
of nesting inside the HTML tags. The first tag in the source is <html> and
the last is </html> All the rest of the tags in the page are inside the pair.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np


29
General Tree
• Application: The HTML tags used to create the web page

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 30


Binary Tree
• A general tree has no restrictions on the number of children each
node can have.
• A binary tree is a specialized case of a general tree.
• Binary tree has a restriction on number of children of its node.
• Every node in a binary tree can have maximum of two children.
• The children are referred to as left child or right child.
• In a binary tree, every node can have either 0 children or 1 child or 2
children but not more than 2 children
• This is one of the most commonly used trees.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 31

Binary Tree
• A binary tree is a finite set of elements that is either empty or is
partitioned into three disjoint subsets (Disjoint means that they have
no nodes in common):
1. The first subset contains a single element called the root of the
tree.
2. The other two subsets are themselves binary trees, called left
subtree and right subtree.
3. A left or right subtree can be empty.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 32

Binary Tree
• A Binary Tree, T is recursively defined as follows:
• T is empty (i.e. contains no nodes) OR
• T consists of a root node which is connected to two distinct
binary trees TLeft and TRight
(`distinct' in the sense that TLeft and TRight have no nodes in common).

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 33


Binary Tree

• These all are binary tree:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 34


Binary Tree
• These are not binary tree:
• In Figure (A) and (B) there are
nodes with more than 2
children;
• Figure (C) is not a tree structure
(in terms of our definition, the
left and right sub-trees of the
root can be viewed as not
containing distinct sets of
nodes). It is a graph.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 35

Binary Tree
• Tree structures enable efficient access and efficient update to large
collections of data. 
• Binary trees in particular are widely used and relatively easy to
implement.
• Binary trees are useful for many things besides searching.
• Just a few examples of applications that trees can speed up include 
• prioritizing jobs, 
• describing mathematical expressions and
• the syntactic elements of computer programs, or
• organizing the information needed to drive data compression
algorithms.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 36

Types of Binary Tree


• Full binary tree
• Perfect binary tree
• Complete binary tree
• Skewed binary tree
• Balanced binary tree
• Extended binary tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 37

Full Binary Tree


• A binary tree in which each node has exactly zero or two children is
called a full binary tree.

• It is also called as strictly binary tree or


Proper Binary Tree or 2-Tree A
• The Hu man coding tree is an example of a full
binary tree. B C
• In a full tree, there are no nodes with exactly
one child.
• In a full binary tree: D E
• Number of Leaf nodes = Number of
Internal nodes + 1
F G

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 38


ff

Full Binary Tree

Strictly binary tree data structure is used to represent mathematical expressions.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 39


Perfect Binary Tree
• A perfect binary tree is a binary tree in which all interior nodes
have two children and all leaves have the same depth or same level.

• Total number of nodes in a Perfect Binary Tree with height h is 2^h — 1.

A
A

A B C
B C

D E F G

In some books, this tree is called as complete binary tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 40


Perfect Binary Tree
Let us consider a perfect binary tree of depth d with n nodes:

A d=0

Depth (d)
B C d=1

d=2
D E F G
No of nodes = 2d + 1 – 1 nodes. Depth d # nodes at depth d # of child nodes
--------------------------------------------------------------
Depth = log(n + 1) – 1 0 1 = 20 2
1 2 = 21 4
Leaf Nodes = 2d 2 4 = 20 8

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 41


Complete Binary Tree


• A complete binary tree is a binary tree in which every level, except
possibly the last, is completely filled, and all nodes are as far left as
possible.
• A complete binary tree is a tree, which is completely
A
filled, with the possible exception of the bottom
level, which is filled from left to right.
• Also called almost complete or nearly complete B C
binary tree
• The heap data structure is an example of a
complete binary tree. D E F
• A complete binary tree of the height h has between
2h and 2(h+1)-1 nodes.

In some books, this tree is called as almost complete binary tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 42

Complete Binary Tree

A A

B C B C

D E F D

It can have between 1 and 2d nodes at the last level d

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 43


Complete Binary Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 44


Complete Binary Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 45


Skewed Binary Tree
• If a tree which is dominated by left child node or right child node, is said
to be a Skewed Binary Tree.
• A skewed binary tree is a pathological/degenerate tree in which the tree
is either dominated by the left nodes or the right nodes. 
• In a skewed binary tree, all nodes except the leaf node have only one
child node.

Skewed Binary Tree Left Skewed Binary Tree Right Skewed Binary Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 46


Extended Binary Tree


• A binary tree can be converted into Full Binary tree by adding
dummy nodes to existing nodes wherever required.
• The full binary tree obtained by adding dummy nodes to a binary
tree is called as Extended Binary Tree.

Dummy Node

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 47


Tree Representation in Memory


• We can represent a binary tree in memory using one of the following
ways:
1. Array Representation of Binary Tree
2. Linked List Representation of Binary Tree

9 5 4 1 3 2 -
0 1 2 3 4 5 6

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 48

Tree: Array Representation


• In array representation of a binary tree, we use one-dimensional
array (1-D Array) to represent a binary tree.
• To represent a binary tree of depth 'n' using array representation, we
need one dimensional array with a maximum size of 2n + 1 – 1.
• If a tree is a full binary tree or at least a complete binary tree, then
this method will be very efficient.
• Consider an array TREE [MaxSize]. Then the nodes of the tree T with
root R are stored as:
• The root R is stored in TREE[0]
• For a node at index i, its left child is stored in TREE[2*i + 1] and
its right child is stored into TREE[2*i + 2]

• The parent of a node at index i is at TREE[⌊ 2 ⌋]


i−1

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 49


Tree: Array Representation


Example: Suppose a tree as: Rules to represent a node in array
• Root is at index 0.
• If a node is at index i :
• Its left child is at 2*i + 1
• Its right child is at 2*i + 2

⌊ 2 ⌋
i−1
A B C D E F G • Its parent is at

0 1 2 3 4 5 6
Array representation of the above tree.
TREE
Array representation of the above tree.
A B C D E F G

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 50

Tree: Array Representation


Other Examples of Tree Representation in Array:

A B C D E - - A B C - - F G
0 1 2 3 4 5 6 0 1 2 3 4 5 6

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 51


Tree: Linked-list Representation
• We can represent the binary tree using doubly linked list data structure.
• A node in linked list represent a node of the binary tree.
• Each list node needs three fields.
• One field to hold data and
• the other two fields to hold the addresses of the left child node and right
child node.

Link to left node Link to right node

• This node can be recursively used to represent a binary tree. If a node doesn’t
have a left child, it will be assigned a null value. If the node doesn’t have a
right child, it will be assigned a null value.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 52

Tree: Linked-list Representation


• Example:

(a) A Binary Tree (b) Linked-list Representation of Binary


Tree of figure (a).

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 53


Tree: Linked-list Representation


• A Tree Node Implementation in C:

struct node
{
int data; Link to left node Link to right node
node *left;
node *right;
};

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 54

Traversal in Binary Tree


• Often we wish to process a binary tree by "visiting" each of its nodes,
each time performing a specific action such as printing the contents
of the node.
• Any process for visiting all of the nodes in some order is called
a traversal. 
• Any traversal that lists every node in the tree exactly once is called
an enumeration of the tree's nodes. 
• Some applications do not require that the nodes be visited in any
particular order as long as each node is visited precisely once.
• For other applications, nodes must be visited in an order that
preserves some relationship.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 55

Traversal in Binary Tree


• There are three ways of traversing a tree :
1. In-order Traversal
2. Pre-order Traversal
3. Post-order Traversal

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 56

Inorder Traversal
• In this traversal method, the left subtree is visited first, then the
root and later the right sub-tree.
• If a binary search tree is traversed in-order, the output will produce
sorted values in an ascending order.

• We start from A, and following in-


order traversal, we move to its
left subtree B. 
• B is also traversed in-order.
• The process goes on until all the
nodes are visited.
• The in-order enumera on for this
tree is D → B → E → A → F → C → G.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 57

ti

Inorder Traversal

Algorithm: Inorder Traversal:


Until all nodes are traversed, repeat the
following steps:
1.Traverse the left subtree in in-order.
2.Traverse root node.
3.Traverse the right subtree in in-order.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 58


Pre-order Traversal
• In this traversal method, the root is visited first, then the left
subtree is visited and the right sub-tree is visited at last.

• We start from A, and following pre-


order traversal, we first visit A itself
and then move to its left subtree B. 
• B is also traversed pre-order. The
process goes on until all the nodes are
visited.
• The pre-order enumera on for this tree
is A → B → D → E → C → F → G.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 59

ti

Pre-order Traversal

Algorithm: Pre-order Traversal:


Until all nodes are traversed, repeat the
following steps:
1.Traverse root node.
2.Traverse the left subtree in pre-order.
3.Traverse the right subtree in pre-order.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 60


Post-order Traversal
• In this traversal method, the left subtree is visited at first, then the
right sub-tree is visited and the root is visited at last.

• We start from A, and following Post-


order traversal, we first visit the left
subtree B. 
• B is also traversed post-order. The
process goes on until all the nodes are
visited.
• The post-order enumera on for this
tree is D → E → B → F → G → C → A

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 61

ti
Post-order Traversal

Algorithm: Post-order Traversal:


Until all nodes are traversed, repeat the
following steps:
1.Traverse the left subtree in post-order.
2.Traverse the right subtree in post-order.
3.Traverse root node.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 62


Tree Traversal Implementation in C++

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 63


Tree Traversal Implementation in C++
1. In-order Traversal :
44->22->55->11->66->33->77
2. Pre-order Traversal
11->22->44->55->33->66->77
3. Post-order Traversal
44->55->22->66->77->33->11

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 64

Tree Traversal Implementation in C++


1. In-order Traversal :
1->3->4->6->7->8->10->13->14
2. Pre-order Traversal
8->3>1->6->4->7-10->14->13
3. Post-order Traversal
1->4->7->6->3->-13->14->10->8

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 65

Binary Search Tree


• A binary search tree (BST) is a binary tree where each node has
comparable value and satisfies:
• The value at a node is greater than or equal to the value of all
nodes in its left subtree and is smaller than the value of all nodes
in its right subtree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 66


Binary Search Tree


• The properties that separate a binary search tree from a regular binary tree are:
• All nodes of left subtree are less than the root node
• All nodes of right subtree are more than the root node
• Both subtrees of each node are also BSTs i.e. they have the above two
properties
• The in-order traversal gives the sorted sequence of nodes in ascending order.
• A “balanced” binary search tree can be searched in O (log n) time, where n
is the number of nodes in the tree
• Some definitions of BST allows the duplicate key values while others not. It
depends upon the nature of application for which we are building BST. If the
application does not allow nodes with equal keys, then duplication won’t be the
case. If duplicate keys are allowed, our convention will be to insert the
duplicate key in the left subtree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 67

Binary Search Tree

Figure: Two Binary Search Trees for a collec on of values. Tree (a) results if values
are inserted in the order 37, 24, 42, 7, 2, 42, 40, 32, 120. Tree (b) results if the same
values are inserted in the order 120, 42, 42, 7, 2, 32, 37, 24, 40.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 68


ti
Operations in Binary Search Tree
• The basic BST operations includes:
• Insertion
• Deletion
• Searching
• Traversing (in-order, pre-order and post-order)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 69

Inserting a node into Binary Search Tree


• Algorithm:
1. Create a node to be inserted (say newNode) and assign the value to
newNode
2. Start from the root node and compare the values of newNode and root node
3. If the newNode is less than the root, go to left child of root,
otherwise go to the right child of the root.
4. Continue the step 2 (each node is a root for its sub tree) until a null pointer
(or leaf node) is found
5. Insert the newNode as a left child if newNode is less than leaf node
6. Insert the newNode as a right child if newNode is greater than the leaf
node.

• Note that a new node is always inserted as a leaf node.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 70

Inserting a node into Binary Search Tree


• A recursive algorithm for inserting a node into a BST is as follows:

• Assume we insert a node N to tree T.


• If the tree is empty, then we return new node N as the tree.
• Otherwise,
• inserting the node N to left sub tree if N<T
• inserting the node N to right sub tree if N>T

Recursive Definition:
Insert(N, T) = N if T is empty
= insert(N, T.left) if N<T
= insert(N, T.right)if N>T

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 71

Inserting a node into Binary Search Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 72


Deleting a node from Binary Search Tree
• Deleting a node could affect all sub trees of that node.  
• For example, deleting node 5 from the tree could result in losing sub
trees that are rooted at 1 and 9.
• Hence we need to be careful about deleting nodes from a tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 73


Deleting a node from Binary Search Tree


• Case 1: Node to be deleted is a leaf node
• Simply remove from the tree. This is a very easy case.
• For example if you want to delete the node 20, then simple assign a NULL
value to the le -child pointer of its parent (30) and delete the node 20.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 74


ft

Deleting a node from Binary Search Tree


• Case 2: Node to be deleted has only one child
• Copy the child to the node and delete the child.
• For example, if you want to delete node 30. First copy its child
key value (i.e. 40) to this node. Assign the null value to the
pointer which points to its child and delete its child.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 75


Deleting a node from Binary Search Tree


• Case 3: Node to be deleted has two children
• This case is little difficult. The node that we want to delete has
left subtree and right subtree.
• Suppose if we want to delete a node, we have to find a node with
either (a) a greatest value (in-order predecessor) from the left
subtree or (b) a lowest value (in-order successor) from the right
subtree.
• If we replace the value of the node to be deleted by its in-order
successor, then we finally delete its in-order successor.
• If we replace the value of the node to be deleted by its in-order
predecessor, then we finally delete its in-order predecessor.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 76

Deleting a node from Binary Search Tree


• Case 3: Node to be deleted has two children
• Suppose we want to delete node 50.

• In-order traversal:
• 30->40->50->60->70->80
• Where 40 = in-order predecessor
• And 60 = in-order successor
• We can replace 50 either by 40 (in-order
predecessor) or by 60 (in-order successor)
• Finally delete in-order successor if you
have replaced 50 by in-order successor.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 77


Deleting a node from Binary Search Tree


• Case 3: Node to be deleted has two children
• Suppose we want to delete node 50. Now we find its in-order
successor which node is 60. Then we replace the value of the
node to be deleted by its in-order successor. Finally delete the
in-order successor.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 78


Deleting a node from Binary Search Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 79


Searching a node in Binary Search Tree
• Searching for a node is similar to inserting a node.
• We start from root, and then go left or right until we find (or not find the node).
• A recursive definition of search is as follows.
• If the node is equal to root, then we return true.
• If the root is null, then we return false.
• Otherwise
• recursively search at T.left if N < T or
• recursively search at T.right if N > T.

Recursive Definition:
Search(N, T) = false if T is empty
= true if T = N
= search(N, T.left) if N < T
= search(N, T.right)if N > T

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 80

Searching a node in Binary Search Tree


• Algorithm(Non-recursive):
1. Read the element to be searched.
2. Compare this element with the value of root node in a tree.
3. If element and root value are matching, display "Node is found" and
terminate the function.
4. If element and root value are not matching, check whether an element
is smaller or larger than a node value.
5. If an element is smaller, continue the search operation in left subtree.
Goto step 2.
6. If an element is larger, continue the search operation in right subtree.
Goto step 2.
7. If we reach to a leaf node and the element is not matched to a leaf
node, display "Element is not found" and terminate the function.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 81

Searching a node in Binary Search Tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 82


Traversing a Binary Search Tree

“The in-order traversal of a BST always


gives the sorted sequence of the data in
ascending order.”

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 83


Balanced Binary Tree


• Balanced Binary Tree is a Binary tree in which height of the left and
the right sub-trees of every node may differ by at most 1.
• AVL Tree and Red-Black Tree are well-known data structure to
generate/maintain Balanced Binary Search Tree.
• Search, insert and delete operations cost O (log n) time in such
balanced tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 84


Balanced Binary Tree


A
A
B
B C C
D
D E F G
E

a) A balanced tree containing b) An unbalanced tree F


7 items Containing same 7 items G

Check how many computations may need to search node G


in these two trees?
- a) 3 comparisons [ie O(log2(n+1)]
- b) 7 comparisons [ie O(n) ]

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 85

Balanced Binary Search Tree


Input order: 1 Input order:
4, 2, 6, 1, 3, 5, 7 4
1, 2, 3, 4, 5, 6, 7
2
2 6 3
4
1 3 5 7
5

a) A balanced tree containing b) An unbalanced tree 6


7 items Containing same 7 items 7

Check how many computations may need to search node G


in these two trees?
- a) 3 comparisons [ie O(log2(n+1)]
- b) 7 comparisons [ie O(n) ]

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 86

Construct a Binary Search Tree


• Example 1: Create the binary search tree using the following data
elements: 43, 10, 79, 90, 12, 54, 11, 9, 50

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 87


Construct a Binary Search Tree
• Example: Create the binary search tree using the following data
elements. 43, 10, 79, 90, 12, 54, 11, 9, 50

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 88


Construct a Binary Search Tree
• Example: Create the binary search tree using the following data
elements. 43, 10, 79, 90, 12, 54, 11, 9, 50

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 89


Student Practice Construct a Binary Search Tree
• Example 2: Construct a Binary Search Tree by inserting the following
sequence of numbers: 10, 12, 5, 4, 20, 8, 7, 15 and 13.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 90


Balanced Binary Search Tree


• Best points of BST:
• In-order traversal of BST gives sorted sequence
• If it is balanced, data can be accessed in O(log n).
• However, building a BST as a balanced tree is not a trivial task.
• For example, if we build a BST from the sorted data, the tree will
grow to the right or to the left.
• A binary search tree with the worst-case structure is no more
efficient than a regular linked list.
• A great care needs to be taken in order to keep the tree as balanced
as possible.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 91


Balanced Binary Search Tree


• A Balanced Binary Tree is a Binary tree in which
• height of the left and the right sub-trees of every node may
differ by at most 1.
• Well-known Balanced Binary Search Tree:
• AVL Tree
• Red-Black Tree
• Complexity of operations in Balanced tree:
• Search, insert and delete operations cost O (log n) time

The heap keeps its balanced shape at the cost of weaker


restrictions on the relative values of a node and its children,
making it a bad search structure.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 92

Class Discussion Binary Heap vs Binary Search Tree


Heap BST

• •

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 93


AVL Trees
• AVL tree is a height-balanced binary search tree.
• The AVL tree is named after its two Soviet inventors, Georgy
Adelson-Velsky and Evgenii Landis
• AVL is a binary search tree with the following additional property:
• For every node x, the heights of its left and right subtrees differ
by at most 1. (It is called balance factor)

Balance Factor (x) = height(left-subtree (x)) − height(right-subtree(x))


= {-1, 0, 1}

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 94


AVL Trees
• In this tree, the balance factor associated with its each node is in
between -1 and +1. therefore, it is an AVL tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 95


AVL Trees
• Example: AVL Tree

• A node X with
• BF(X)<0 is called “left-heavy"
• BF(X)>0 is called "right-heavy",
• BF(X)=0 is simply called "balanced".

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 96

AVL Trees
• Examples:

Q: Is this AVL tree??


If yes, why?
If no, why?

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 97

AVL Trees
• Complexity:
Algorithm Average case Worst case

Space o(n) o(n)

Search o(log n) o(log n)

Insert o(log n) o(log n)

Delete o(log n) o(log n)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 98


Operations on AVL Trees
• Since AVL tree is also a binary search tree therefore, all the
operations are performed in the same way as they are performed in
a binary search tree.
• Searching and traversing do not lead to the violation in property of
AVL tree.
• However, insertion and deletion are the operations which can violate
this property.
• Therefore, after every insertion into AVL tree or deletion from AVL
tree, we need to check the AVL property.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 99

AVL Trees
• An insert(65) operation that violates the AVL tree balance property.

insert(65)

• Prior to the insert operation, all nodes of the tree are balanced
• After inserting the node 65, the node 100 is no longer balanced.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 100


AVL Trees
• An insert(5) operation that violates the AVL tree balance property.

insert(5)

• Prior to the insert operation, all nodes of the tree are balanced
• After inserting the node 5, the node 24 and 7 are no longer balanced.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 101


AVL Trees
• An delete(25) operation that violates the AVL tree balance property.

delete(25)

• Prior to the delete operation, all nodes of the tree were balanced
• After deleting the node 25, the node 50 is no longer balanced.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 102

• An insert(5) operation that violates the AVL tree balance property.

Balancing Algorithm

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np

Balancing Algorithm
• In AVL tree, after performing operations like insertion and deletion
we need to check the balance factor of every node in the tree.
• If every node satisfies the balance factor condition then we conclude
the operation otherwise we must make it balanced.
• Whenever the tree becomes imbalanced due to any operation we
use rotation operations to make the tree again balanced.
• Rotation is the process of moving nodes either to left or to right to
make the tree balanced.
• AVL tree permits difference (balance factor) to be only 1.

Balance Factor = height(left-subtree) − height(right-subtree)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 104


Balancing Algorithm
• If the balance factor is more than 1, then the tree is rebalanced using
some rotation techniques.
• There are four cases due to which AVL tree may become unbalanced:
Case 1:
A node is inserted in the left subtree of left subtree of a node.
Case 2:
A node is inserted in the right subtree of right subtree of a node.
Case 3:
A node is inserted in the right subtree of left subtree of a node.
Case 4:
A node is inserted in the left subtree of right subtree of a node.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 105

Balancing Algorithm
• To deal with the four cases that can violate the AVL property, there
are four different rotations to rebalance the AVL tree.
• AVL Rotations:
1. Left rotation
2. Right rotation
3. Left-Right rotation
4. Right-Left rotation
• The first two rotations are single rotations and the next two
rotations are double rotations.
• To have an unbalanced tree, we at least need a tree of height 2.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 106

Balancing Algorithm
Case Case description that causes Case Name Required
Diagram to violate AVL property Rotation

When a node is inserted in the


left subtree of left subtree of a Left Left (LL) Right Rotation
node

When a node is inserted in the


right subtree of right subtree of Right Right (RR) Left Rotation
a node

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 107


Balancing Algorithm
Case Case description that causes Case Name Required
Diagram to violate AVL property Rotation

When a node is inserted in the LR Rotation


right subtree of left subtree Left Right (LR) = Left rotation +
of C Right rotation

When a node is inserted in the RL Rotation


left subtree of right subtree Right Left (RL) = Right rotation +
of A Left rotation

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 108

Balancing Algorithm

Animation showing the insertion of several elements into an AVL


tree. It includes left, right, left-right and right-left rotations.
Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 109
Balancing Algorithm

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 110


Balancing Algorithm Left Rotation (RR Case)
• If a tree becomes unbalanced, when a node is inserted into the right subtree
of the right subtree of a node, then we perform a single left rotation.
• It is applied on the edge below a node having balance factor -2
-

• In above example, node A has balance factor -2 because a node C is inserted


in the right subtree of the right subtree of A. We perform the left rotation on
the edge below A.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 111


Balancing Algorithm Left Rotation (RR Case)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 112


Balancing Algorithm Right Rotation (LL Case)


• AVL tree may become unbalanced, if a node is inserted in the left subtree of the
left subtree of a node. The tree then needs a right rotation.
• It is applied on the edge below a node having balance factor 2.

• In above example, node C has balance factor 2 because a node A is inserted in the
left subtree of left subtree of C. As depicted, the unbalanced node becomes the
right child of its left child B after performing a right rotation.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 113


Balancing Algorithm Right Rotation (LL Case)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 114


Balancing Algorithm Left-Right Rotation (LR Case)


• Double rotations are slightly complex version of already explained
versions of rotations.
• A left-right rotation is a combination of left rotation followed by
right rotation.

RR Case LL Case

Left-Right Rotation = Left rotation FOLLOWED BY Right rotation

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 115


Balancing Algorithm Left-Right Rotation (LR Case)


• A left-right rotation is a combination of left rotation followed by
right rotation.

Left Right
Rotation Rotation

The Left rotation: The Right rotation:


• makes 2 -the left subtree of 3 • makes 2 -the root
• makes 1 -the left subtree of 2 • makes 3 -the right subtree of 2

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 116

Balancing Algorithm Right-Left Rotation (RL Case)


• The second type of double rotation is Right-Left Rotation. It is a
combination of right rotation followed by left rotation.

Right Left
Rotation Rotation

Right-Left Rotation = Right rotation FOLLOWED BY Left rotation

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 117

Class Discussion Why AVL Tree??


• AVL tree controls the height of the binary search tree by not letting
it to be skewed.
• The time taken for all operations in a binary search tree of height h
can be extended to O(n) if the BST becomes skewed (i.e. worst
case).
• By limiting this height to log n, AVL tree imposes an upper bound on
each operation to be O(log n) where n is the number of nodes.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 118


Constructing an AVL Tree


• Example 1: Construct an AVL tree by inserting the following
elements in the given order. 63, 9, 19, 27, 18, 108, 99, 81

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 119


Constructing an AVL Tree
• Example 1: Construct an AVL tree by inserting the following
elements in the given order. 63, 9, 19, 27, 18, 108, 99, 81

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 120


Constructing an AVL Tree
• Example 1: Construct an AVL tree by inserting the following
elements in the given order. 63, 9, 19, 27, 18, 108, 99, 81

Right
Rotation

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 121

Student Practice Constructing an AVL Tree


• Example 2: Construct an AVL Tree from 1, 2, 3, 4, 5, 6, 7, 8.
• Example 3: Construct AVL tree from the sequence:   50, 25, 10, 5, 7,
3, 30, 20, 8, 15

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 122


Huffman Algorithm
• Fixed-length codes: ASCII coding 
• Standard ASCII coding scheme assigns a unique eight-bit value
to each character.
• Capable to represent 256 unique characters
• ASCII coding takes ⌈log 128⌉ or seven bits to provide the 128
unique codes needed to represent the 128 symbols of the ASCII
character set.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 123


Huffman Algorithm
• Below Table shows the relative frequencies
of the letters of the alphabet. From this
table we can see that the letter 'E' appears
about 60 times more often than the letter
'Z'.
• If some characters are used more frequently
than others, is it possible to take advantage
of this fact and somehow assign them
shorter codes? The price could be that other
characters require longer codes, but this
might be worthwhile if such characters
appear rarely enough.
• variable-length codes (Example Huffman
coding)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 124

Huffman Algorithm
• Huffman coding assigns codes to characters such that the length of the
code depends on the relative frequency or weight of the corresponding
character. Thus, it is a variable-length code.
• The Huffman code for each letter is derived from a full binary tree
called the Huffman coding tree, or simply the Huffman tree.
• Each leaf of the Huffman tree corresponds to a letter, and we define the
weight of the leaf node to be the weight (frequency) of its associated
letter.
• The goal is to build a tree with the minimum external path weight.
• Define the weighted path length of a leaf to be its weight times its
depth.
• A letter with high weight should have low depth, so that it will count the
least against the total path length. As a result, another letter might be
pushed deeper in the tree if it has less weight.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 125

Huffman Algorithm
• Developed by David Hu man in 1951, this technique is the basis for
all data compression and encoding schemes
• It is a famous algorithm used for lossless data encoding
• It follows a Greedy approach, since it deals with generating
minimum length prefix-free binary codes
• It uses variable-length encoding scheme for assigning binary codes
to characters depending on how frequently they occur in the given
text. The character that occurs most frequently is assigned the
smallest code and the one that occurs least frequently gets the
largest code
• Uses a binary tree to determine the encoding of each character, to decode
an encoded le – i.e., to decompress a compressed le, pu ng it back into
ASCII

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 126


fi

ff

fi
tti

Huffman Algorithm
• Example of a Hu man tree (for a text with only six chars):

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 127


ff
Huffman Algorithm
• The major steps involved in Huffman coding:
1. Step I - Building a Huffman tree using the input set of symbols and
weight/ frequency for each symbol
• A Huffman tree needs to be created having n leaf nodes
and n-1 internal nodes
• Priority Queue is used for building the Huffman tree such that
nodes with lowest frequency have the highest priority. A Min
Heap can be used.
• Initially, all nodes are leaf nodes containing the character
itself along with the weight/frequency of that character
• Internal nodes, on the other hand, contain weight and links to
two child nodes

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 128


Huffman Algorithm
• The major steps involved in Huffman coding:
2. Step II - Assigning the binary codes to each symbol by traversing
Huffman tree
• Generally, bit ‘0’ represents the left child and bit ‘1’
represents the right child

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 129

Huffman Algorithm
• Building the Huffman tree for n letter:
1. Create a collection of n initial Huffman trees, each of which is
a single leaf node containing one of the letters.
2. Put the n partial trees onto a priority queue organized by
weight (frequency).
3. Remove the first two trees (the ones with lowest weight) from
the priority queue. Join these two trees together to create a
new tree whose root has the two trees as children, and whose
weight is the sum of the weights of the two trees. Put this new
tree back into the priority queue.
4. This process (step 3) is repeated until all of the partial Huffman
trees have been combined into one.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 130


Huffman Algorithm
A.Crea ng the Hu man Tree
1.Create a leaf node for each character and build a min heap using all the
nodes (The frequency value is used to compare two nodes in min heap)
2.Repeat Steps 3 to 5 while heap has more than one node
3.Extract two nodes, say x and y, with minimum frequency from the heap
4.Create a new internal node z with x as its le child and y as its right child.
Also frequency(z)= frequency(x)+frequency(y)
5.Add z to min heap
6.Last node in the heap is the root of Hu man tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 131


ti
ff

ff
ft

Huffman Algorithm
B.Traversing the Hu man Tree (Encoding processes)
1.Create an auxiliary array
2.Traverse the tree star ng from root node
3.Assign 0 to array while traversing the le child and assign 1 to array while
traversing the right child
4.Print the array elements whenever a leaf node is found

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 132


ff
ti

ft

Huffman Algorithm: Example


• Find the Huffman coding for the following characters along with
their frequencies:

Characters Frequencies
a 10
e 15
i 12
o 3
u 4
s 13
t 1

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 133


Huffman Algorithm: Example
• Build the Huffman Tree:
• Create a leaf node for each character and build a
Characters Frequencies
min heap using all the nodes (The frequency value is
a 10
used to compare two nodes in min heap)
e 15
i 12
o 3
u 4
s 13
t 1

Fig 1: Leaf nodes for each character

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 134


Huffman Algorithm: Example


• Build the Huffman Tree:
• (Extract two nodes, say x and y, with minimum frequency from the heap.
Create a new internal node z with x as its le child and y as its right child.
Also frequency(z) = frequency(x) + frequency(y) and Add z to min heap)
• Extract and Combine node t and o. Add the new internal node to
priority queue:

Fig 2: Combining nodes o and t


Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 135

ft

Huffman Algorithm: Example


• Build the Huffman Tree:
• Extract and Combine node u with an internal node having 4 as the
frequency. Add the new internal node to priority queue:

Fig 3: Combining node u with an internal node having 4 as frequency


Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 136

Huffman Algorithm: Example


• Build the Huffman Tree:
• Extract and Combine node a with an internal node having 8 as the
frequency. Add the new internal node to priority queue:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 137


Huffman Algorithm: Example


• Build the Huffman Tree:
• Extract and Combine nodes i and s. Add the new internal node to
priority queue:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 138


Huffman Algorithm: Example


• Build the Huffman Tree:
• Extract and Combine node e with an internal node having 18 as
the frequency. Add the new internal node to priority queue:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 139


Huffman Algorithm: Example


• Build the Huffman Tree:
• Finally, Extract and Combine internal nodes having 25 and 33 as
the frequency. Add the new internal node to priority queue:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 140


Huffman Algorithm: Example


• Traverse the Huffman Tree:

Fig 8: Assigning binary codes to


Hu man tree (a) Hu man tree a er
assigning 0 to the le child and 1 to the
right child (b) generated variable length
Hu man coding for each character

(a)

(b)
Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 141
ff
ff
ft
ff
ft
Student Practice Huffman Algorithm: Example
• Build a Huffman tree and Huffman coding for the following
characters with given frequencies:

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 142


Game Tree
• A game tree is a type of recursive search function that examines all
possible moves of a strategy game, and their results, in an attempt
to ascertain the optimal move.
• They are very useful for Artificial Intelligence
• The most commonly-cited example is chess
• A game tree is a directed graph whose nodes are positions in a game
and whose edges are moves.
• The complete game tree for a game is the game tree starting at the
initial position and containing all possible moves from each position;
the complete tree is the same tree as that obtained from the
extensive-form game representation.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 143


Game Tree
• Game trees are generally used in board games to determine the best
possible move. For example: Tic-Tac-Toe.
• The idea is to start at the current board position, and check all the
possible moves the computer can make.
• Then, from each of those possible moves, to look at what moves the
opponent may make. Then to look back at the computers.
• Ideally, the computer will flip back and forth, making moves for
itself and its opponent, until the game's completion.
• It will do this for every possible outcome, effectively playing
thousands (often more) of games.
• From the winners and losers of these games, it tries to determine
the outcome that gives it the best chance of success.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 144


Game Tree
• This can be likened to how people think during a board game - for
example "if I make this move, they could make this one, and I could
counter with this" etc.
• Game trees do just that. They examine every possible move and every
opponent move, and then try to see if they are winning after as many
moves as they can think through.
• As one could imagine, there are an extremely large number of possible
games.
• In Tic-Tac-Toe, there are 9 possible first moves (ignoring rotation
optimization). There are 8 on the second turn, then 7, 6, etc. In total,
there are 255,168 possible games.
• The goal of a game tree is to look at them all (on the first move) and
try to choose a move that will, at worst, make losing impossible, and,
at best, win the game.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 145

Game Tree

Figure: Game tree of Tic-Tac-Toe with the possible combinations of the rst two moves  

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 146


fi
Game Tree
• The diagram shows the first two levels, or plies, in the game tree for
tic-tac-toe.
• The rotations and reflections of positions are equivalent, so the first
player has three choices of move: in the center, at the edge, or in
the corner.
• The second player has two choices for the reply if the first player
played in the center, otherwise five choices. And so on.
• The number of leaf nodes in the complete game tree is the number
of possible different ways the game can be played.
• For example, the game tree for tic-tac-toe has 255,168 leaf nodes.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 147

B Tree
• In search trees like binary search tree, AVL Tree, Red-Black tree,
etc., every node contains only one value (key) and a maximum of
two children.
• But there is a special type of search tree called B-Tree in which a
node contains more than one value (key) and more than two
children.
• B-Tree was developed in the year 1972 by Bayer and McCreight with
the name Height Balanced m-way Search Tree. Later it was named
as B-Tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 148

B Tree
• Definition:
• B-Tree is a self-balanced search tree in which every node contains
more than one key and has more than two children.
• It is also known as a height-balanced m-way tree.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 149

Properties of B-tree
• The number of keys in a node and number of children for a node
depends on the order of B-Tree. Every B-Tree has an order.
• B-Tree of Order m has the following properties:
1. All leaf nodes must be at same level.
2. All nodes except root must have at least [m/2]-1 keys and maximum
of m-1 keys.
3. All non leaf nodes except root (i.e. all internal nodes) must have at
least m/2 children (and at most m children).
4. If the root node is a non leaf node, then it must have at least 2 children
and contains a minimum of 1 key..
5. A non leaf node with n-1 keys must have n number of children.
6. All the key values in a node must be in Ascending Order.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 150

B-tree: Example
• B Tree of order 4

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 151


B-tree: Example

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 152


B-tree: Example

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 153


B-tree: Example
• B-Tree of order 5

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 154


Why a B-tree data structure?
• The need for B-tree arose with the rise in the need for lesser time in
accessing the physical storage media like a hard disk. The secondary
storage devices are slower with a larger capacity. There was a need
for such types of data structures that minimize the disk accesses.
• Other data structures such as a binary search tree, AVL tree, red-
black tree, etc can store only one key in one node. If you have to
store a large number of keys, then the height of such trees becomes
very large and the access time increases.
• However, B-tree can store many keys in a single node and can have
multiple child nodes. This decreases the height significantly allowing
faster disk accesses.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 155


Why a B-tree data structure?


• In most of the other self-balancing search trees (like AVL and Red-Black
Trees), it is assumed that everything is in main memory. To understand
the use of B-Trees, we must think of the huge amount of data that cannot
fit in main memory. When the number of keys is high, the data is read
from disk in the form of blocks. Disk access time is very high compared to
the main memory access time. The main idea of using B-Trees is to
reduce the number of disk accesses. Most of the tree operations (search,
insert, delete, max, min, ..etc ) require O(h) disk accesses where h is the
height of the tree. B-tree is a fat tree.
• The height of B-Trees is kept low by putting maximum possible keys in a
B-Tree node.
• Generally, the B-Tree node size is kept equal to the disk block size.
• Since the height of the B-tree is low so total disk accesses for most of the
operations are reduced significantly compared to balanced Binary Search
Trees like AVL Tree, Red-Black Tree, ..etc.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 156

Why a B-tree data structure?


• Time Complexity of B-Tree: 

Sr. No. Algorithm Time Complexity

1 Search O(log n)

2 Insert O(log n)

3 Delete O(log n)

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 157


Operations on a B-Tree
• The following operations are performed on a B-Tree:
• Search
• Insertion
• Deletion

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 158

Search Operations on a B-Tree


• The search operation in B-Tree is similar to the search operation in
Binary Search Tree.
• In a Binary search tree, the search process starts from the root node
and we make a 2-way decision every time (we go to either left
subtree or right subtree).
• In B-Tree also search process starts from the root node but here we
make an n-way decision every time. Where 'n' is the total number of
children the node has.
• In a B-Tree, the search operation is performed with O(log n) time
complexity.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 159


Search Operations on a B-Tree


• Search Algorithm:
1. Read the search element from the user.
2. Compare the search element with first key value of root node in the tree.
3. If both are matched, then display "Given node is found!!!" and terminate the
function
4. If both are not matched, then check whether search element is smaller or larger
than that key value.
5. If search element is smaller, then continue the search process in left subtree.
6. If search element is larger, then compare the search element with next key
value in the same node and repeat steps 3, 4, 5 and 6 until we find the exact
match or until the search element is compared with last key value in the leaf
node.
7. If the last key value in the leaf node is also not matched then display "Element is
not found" and terminate the function.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 160

Search Operations on a B-Tree


• Example: Searching 120 in the given B-Tree. 

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 161


Search Operations on a B-Tree
• Example: Searching 120 in the given B-Tree. 

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 162


Search Operations on a B-Tree
• Example: Searching 120 in the given B-Tree. 

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 163


Search Operations on a B-Tree
• Example: Searching 120 in the given B-Tree. 

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 164


Insertion Operation in B-Tree
• In a B-Tree, a new element must be added only at the leaf node. That
means, the new keyValue is always attached to the leaf node only.
• Insertion Algorithm:
1. Check whether tree is Empty.
2. If tree is Empty, then create a new node with new key value and insert it into
the tree as a root node.
3. If tree is  Not Empty, then nd the suitable leaf node to which the new key
value is added using Binary Search Tree logic.
4. If that leaf node has empty position, add the new key value to that leaf node in
ascending order of key value within the node.
5. If that leaf node is already full, split that leaf node by sending middle value to
its parent node. Repeat the same until the sending value is xed into a node.
6. If the spliting is performed at root node then the middle value becomes new
root node for the tree and the height of the tree is increased by one.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 165


fi

fi

Insertion Operation in B-Tree


Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 166


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 167


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 168


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 169


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 170


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 171


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 172


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 173


Insertion Operation in B-Tree
Example: Construct a B-Tree of Order 3 by inserting numbers from 1 to 10.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 174


Insertion Operation in B-Tree

Example: Construct a B-Tree of Order 3 by


Step 1 Step 2
Step 3 Step 4
inserting 8, 9, 10, 11, 15, 20, 17.

Step 7 Step 6 Step 5

Step 8 Step 9
Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 175
Insertion Operation in B-Tree
Step 8 Step 9

Example: Construct a B-Tree of Order 3 by


inserting 8, 9, 10, 11, 15, 20, 17.

Step 11 Step 10

Step 12 Step 13

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 176


Student Practice Deletion Operation on a B Tree

• Look at https://www.programiz.com/dsa/deletion-from-a-b-tree

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 177


Student Practice B Tree vs B+ Tree


See more about B+ Tree at https://www.programiz.com/dsa/b-plus-tree

B Tree B+ Tree
Data is stored in leaf nodes as well as Data is stored only in leaf nodes.
internal nodes.

Searching is a bit slower as data is stored Searching is faster as the data is stored only
in internal as well as leaf nodes. in the leaf nodes.

No redundant search keys are present. Redundant search keys may be present.

Deletion operation is complex. Deletion operation is easy as data can be


directly deleted from the leaf nodes.

Leaf nodes cannot be linked together. Leaf nodes are linked together to form a
linked list.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 178


Graph is a Tree If …
• An n-vertex graph G(V,E) is a tree if:
• There is a path of edges between any two vertices and there are
exactly n-1 edges.
• There is a path of edges between any two vertices and there are
no cycles in G.
• There is a unique path between any two vertices.

Dr. Udaya R. Dhungana, Pokhara University, Nepal. udaya@pu.edu.np 179

THANK YOU

End of the Chapter

180

You might also like