DSA Merged

Data Structures and Algorithms
Dr. Arup Kumar Pal

Department of Computer Science & Engineering
Indian Institute of Technology (ISM), Dhanbad
Jharkhand-826004
E-mail: arupkrpal@iitism.ac.in
Data Structures and Algorithms Dept. of CSE, IIT(ISM) Dhanbad November 21, 2021 1
Outline
• Course Related Information
• Overview of Course Content
• Basic Concepts in Algorithmic Analysis
• Conclusions
Data Structures and Algorithms Department of CSE, ISM Dhanbad November 21, 2021 2
Lecture Plan
Useful Resources
Text Books:
Horowitz, Sahani, and Rajasekaran, “Fundamentals
of Computer Algorithms”, Universities Press
Lipschutz and Pai, “Data Structures”, TMH
Publishing
Reference Books:
Tenenbaum, “ Data Structures using C/C++”, PHI
Samanta, “Classic Data Structures”, PHI
Evaluation
No Mid Term Examination
Four Quiz Test: 52 Marks (13 Marks Each)
End Semester Examination: 48 Marks
Duration of Each Quiz is 30 minutes
The Questions can be of MCQ type, True/False, fill
in the blanks, short answer questions or numerical
problem.
The schedule for conducting quizzes will be
followed as per the academic calendar.
Introduction
Computer Science is the Study of Algorithms and Data
Algorithms are at the heart of every nontrivial computer application
There is an intimate connection between synthesis of algorithms and
structuring of data
Thus, Data structures and algorithms should be thought of as a single
unit, neither making sense without the other.
Algorithm study comprises four distinct areas:
Machines for executing algorithms- It includes from pocket
calculator to largest general purpose digital computer, and the
goal is to study various machine fabrication and organization so
that the algorithms can be effectively carried out.
Languages for describing algorithms- At one end are the
languages closest to the physical M/Cs and at the other end are
the languages designed for sophisticated problem solving.
Contd…
Foundation of algorithms- It relates basics regarding the
possibility/designing algorithms. For instance, Is a particular
task accomplishable by a computing device? What are
minimum number operations are necessary for an algorithm
to complete function/problem?
Analysis of algorithms- To study/analysis each steps of an
algorithm so that its performance in terms of time
requirements and memory storage as a whole can be
estimated.
Algorithms
Definition of algorithm- An algorithm is a finite
set of instructions, which, if followed,
accomplish a particular task. In addition, every
algorithm must satisfies the following criteria:
Input- There are zero or more quantities which are
externally supplied.
Output- At least one quantity is produced.
Definiteness- Each instruction must be clear and
unambiguous
Contd…
Finiteness- In all cases, the algorithm must terminate after finite
number of steps
Effectiveness- Each instruction must be sufficiently basic and
contribute to problem solution.
Correctness- Algorithm must produce correct output in all
instances.
Difference between algorithm and program- A program
(Operating system) does not satisfy the Finiteness. An
algorithm must terminate.
An algorithm can be described in many ways- English- but
careful that the instructions are Definite, Flowchart- In
graphical form using different symbols, Pseudocode- Almost
code, that means an algorithm should be presented in such a
way that algorithm and code have one-to-one relationship.
Algorithm example
Computation of F0  0, and F1  1
Fibonacci Series Fn  Fn 1  Fn  2 s.t. n  2
Contd…
The quality of an algorithm is decided by two
parameters:
How much extra space does it require other than the
input ?
How much time does it require for execution for a
given input of size n?
Generally, we analyze algorithms in terms of their
time complexities.
We assume in context of time complexity analysis:
All the statements take equal time for execution.
Each statement takes unit time to execute.
Analysis of the algorithms
Execution count for Fn
Total Count: 5n+4
Analysis of Time Complexity
Best case: Inputs are provided in such a way
that the minimum time is required to process
them.
Average case: Average behaviour of the
algorithm is studied for various kinds of inputs.
It needs assumption of statistical distribution of
inputs.
Worst case: Input is given in such a way that
maximum time is required to process them.
Linear Search
An efficient algorithm: Binary Search
Time Complexity
 k, n 1

T n   n
T    k, n  1
 2
n
T n T k
2
 n 
 T  2   2k
2 
 n 
 T  3   3k
2 
 n 
 T  r   rk  Let n  2 
 r

2 
n
T   rk
n
 T 1  rk
 k   log n  k
Thank You… Any
Queries...?
Algorithms
Dr. Arup Kumar Pal

Jharkhand-826004
Outline
• What is an Algorithm?
• Performance Analysis
Data Structures and Algorithms Department of CSE, ISM Dhanbad November 21, 2021 2
Algorithms
Definition of algorithm- An algorithm is a finite set of
instructions, which, if followed, accomplish a particular task. In
addition, every algorithm must satisfies the following criteria:
Input- There are zero or more quantities which are externally supplied.
Output- At least one quantity is produced.
Definiteness- Each instruction must be clear and unambiguous
Finiteness- In all cases, the algorithm must terminate after finite number
of steps
Effectiveness- Each instruction must be sufficiently basic and contribute
to problem solution.
Correctness- Algorithm must produce correct output in all instances
Goals
How to devise algorithms?
Creating an algorithm is an art which may never be fully
automated.
How to validate algorithms?
It is necessary to show that it computes the correct answer
for all possible legal inputs.
How to analyze algorithms?
It refers to the task of determining how much computing
time and storage an algorithm requires.
Algorithm example
Computation of F0  0, and F1  1
Fibonacci Series Fn  Fn 1  Fn  2 s.t. n  2
Contd…
The quality of an algorithm is decided by two
parameters:
How much extra space does it require other than the
input ?
How much time does it require for execution for a
given input of size n?
Generally, we analyze algorithms in terms of their
time complexities.
We assume in context of time complexity analysis:
All the statements take equal time for execution.
Each statement takes unit time to execute.
Analysis of the algorithms
Execution count for Fn
Total Count: 5n+4
Linear Search
An efficient algorithm: Binary Search
A decision tree
The performance of the binary search algorithm can be
described in terms of a decision tree, which is a binary tree that
exhibits the behavior of the algorithm.
Time Complexity
 k, n 1

T n   n
T    k, n  1
 2
n
T n T k
2
 n 
 T  2   2k
2 
 n 
 T  3   3k
2 
 n 
 T  r   rk  Let n  2 
 r

2 
n
T   rk
n
 T 1  rk
 k   log n  k
Analysis of Algorithms
We are interested in the design of "good" data
structures and algorithms.
A data structure is a systematic way of organizing
and accessing data, and an algorithm is a step-by-
step procedure for performing some task in a finite
amount of time.
To classify some data structures and algorithms as
"good", we must have precise ways of analyzing
them.
Analyzing the efficiency of a program involves
characterizing the running time and space usage of
algorithms and data structure operations.
Contd…
In order to study the performance of an algorithm,
we have to estimate the computing time (or time
complexity) and memory requirements.
It is impossible to determine exactly how much
time it takes unless we know the following:
Machine we are executing on
Machine instruction set
Time required by each machine instruction.
Translations a compiler will make from source code to
machine language
Contd…
Generally, frequency count of steps of an
algorithm is estimated and represented in some
asymptotic notations for computing time
complexity of an algorithm.
This lecture introduces the common functions
that are used for analyzing algorithms and some
justification techniques for analyzing
algorithms.
Analysis of Time Complexity
Best case: Inputs are provided in such a way
that the minimum time is required to process
them.
Average case: Average behaviour of the
algorithm is studied for various kinds of inputs.
It needs assumption of statistical distribution of
inputs.
Worst case: Input is given in such a way that
maximum time is required to process them.
Common Functions Used in Analysis
Constant Function f(n) = C assignment statement, invoking
a method or function.
Logarithm Function f(n) = logn binary search
Linear Function f(n) = n print out the elements of an
array of size n.
N-Log-N Function f(n) = nlogn merge sort, which will be
discussed later
Quadratic Function f(n) = n2 Bubble sort
Cubic function f(n) = n3 matrix multiplication
Exponential Function f(n) = bn Towers of the Hanoi
In this function, b is a positive
constant, called the base, and the
argument n is the exponent.
Rate of Growth of Functions
Growth of Functions
We quantify the concept that g grows at least
as fast as f.
We only care about the behaviour for large
problems.
Even bad algorithms can be used to solve small
problems.
Big-oh notation
There are different asymptotic notations, first of all we
will study big-O notation, whose definition is given
below:
f(n) = O(g(n)) [read as f of n equals big-oh of g of n) if and
only if there exist two constants c and n0 such that
f(n) ≤ cg(n) for all n, n ≥ n0, where f(n) is the computing
time of some algorithm.
It means that the computing time of an algorithm is O(g(n))
is that its execution time takes no more than a constant time
g(n), where n is the parameter corresponding to input size, or
input and output size or output size (generally input-size is
considered).
For instance, Fibonacci program, time complexity is O(n).
Thus, O(n) is linear, O(n2) is quadratic, O(n3) is cubic, O(2n)
is exponential.
Contd…
The two constants c and n0 in the definition of
big-O notation are called witness to the
relationship f(n) is O(g(n)) .
Choose n0
Choose c; it may depend on your choice of n0
Once you choose c and n0, you must prove the truth
of the implication (often by induction).
Show that f(x)=x2+2x+1 is O(x2)
Example
Contd…
Contd…
Thus, the big-O notation provides an upper
bound on the running time.
Thus, it gives the worst-case complexity of an
algorithm.
big-omega (Ω) notation
While the O-notation gives an upper bound, the
Ω notation, on the other hand, provides a lower
bound within a constant factor of the running
time.
Thus, it provides the best case complexity of an
algorithm.
f(n) = Ω(g(n)) [read as f of n equals big-omega of g of
n) if and only if there exist two constants c and n0 such
that f(n) ≥ cg(n) for all n, n ≥ n0. where f(n) is the
computing time of some algorithm.
The problem of sorting by comparisons is Ω (n log n), to mean
that no comparison-based sorting algorithm with time complexity
that is asymptotically less than n log n can ever be devised.
Contd…
Theta notation
It represents the upper and the lower bound of
the running time of an algorithm.
It is used for analyzing the average-case
complexity of an algorithm.
f(n) = Θ g(n)) [read as f of n equals theta of g of n)
if and only if there exist positive constants c1, c2 and
n0 such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥
n0.
Contd…
Why is it called Asymptotic Analysis?
In mathematical analysis, asymptotic analysis is a method of
describing limiting behavior.
As an illustration, suppose that we are interested in the
properties of a function f(n) as n becomes very large.
If f(n)=n2 + 3n, then as n becomes very large, the
term 3n becomes insignificant compared to n2.
The function f(n) is said to be "asymptotically equivalent to n2,
as n → ∞". This is often written symbolically as f(n) ~ n2, which
is read as "f(n) is asymptotic to n2".
In Asymptotic Analysis, we evaluate the performance of an
algorithm in terms of input size (we don't measure the actual
running time).
We typically ignore small values of n, since we are usually
interested in estimating how slow the program will be on large
inputs.
Shortcomings of Asymptotic Analysis
Implementation complexity: Algorithms with
better complexity are often (much) more
complicated. This can increase coding time and
the constants.
Asymptotic analysis ignores small input sizes.
At small input sizes, constant factors or low
order terms could dominate running time.
Worst case versus average performance
Guidelines for Asymptotic Analysis
Loops: The running time of a loop is, at most,
the running time of the statements inside the
loop (including tests) multiplied by the number
of iterations.
Contd…
Nested loops: Analyze from the inside out.
Total running time is the product of the sizes of
all the loops.
Contd…
Logarithmic complexity: An algorithm is
O(logn) if it takes a constant time to cut the
problem size by a fraction (usually by 1/2).
Algorithm and Data Structure
An algorithm takes raw data as input and transforms it
into refined data as output. Thus, Computer science is
the study of both algorithm and data.
For processing/transforming data, the requirements are
Machine to hold data
Languages to describe data manipulation
Languages to describe kinds of refined data can be produced
from raw data.
Structure to represent data
There is an intimate connection between structuring data
and synthesis of algorithm, i.e., they worked together as
a unit.

Data Structure, Example
Suppose, there is a list of n pair of (names and phone-
number) and we write a program when given any name,
prints person’s phone-number- it is Searching problem.
Now, question is how the list stored so that search is
efficient? We store arbitrarily, or in any specific manner,
and if it former, search would costly (not efficient),
however, if we store the list with the names in
alphabetical order (sorting problem) search would be
efficient and for program will be different the program
for former.
This is the role of algorithm and data structures to work
together.
Thus, data structures are data representations with
efficient data manipulation.

Thank You… Any
Queries...?
Overview of Data Structures
Dr. Arup Kumar Pal

Jharkhand-826004
Algorithm and Data Structure
An algorithm takes raw data as input and transforms it
into refined data as output. Thus, Computer science is
the study of both algorithm and data.
For processing/transforming data, the requirements are
Machine to hold data
Languages to describe data manipulation
Languages to describe kinds of refined data can be produced
from raw data.
Structure to represent data
There is an intimate connection between structuring data
and synthesis of algorithm, i.e., they worked together as
a unit.

Data Structure, Example
Suppose, there is a list of n pair of (names and phone-
number) and we write a program when given any name,
prints person’s phone-number- it is Searching problem.
Now, question is how the list stored so that search is
efficient? We store arbitrarily, or in any specific manner,
and if it former, search would costly (not efficient),
however, if we store the list with the names in
alphabetical order (sorting problem) search would be
efficient and for program will be different the program
for former.
This is the role of algorithm and data structures to work
together.
Thus, data structures are data representations with
efficient data manipulation.

Contd…
A data structure is a typical way of organizing data
in a computer so that it can be accessed efficiently.
In other words, a data structure is a way of
organizing all data items that considers not only the
elements stored but also their relationship to each
other.
To develop a program of an algorithm, we should
select an appropriate data structure for that
algorithm.
Program=Algorithm + Data Structure
Definition
A data structure is an organization of data values, the
relationships among them, and the functions to answer
particular queries on that data.
Example: Complex Number x+iy
Classification
Contd…
Linear Data Structures
Non-Linear Data Structures
Assignment-1
Problem Statement: Magic Square
A magic square is an n×n matrix of the integers 1 to n2
such that the sum of every row, column and diagonal is
the same.
Contd..
Thank You… Any
Queries...?
Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms
Outline I
1 Introduction to Data Structures
Classification of Data Structure
Data Structure Operations
Introduction to Analysis of Algorithms 2 Introduction to Algorithms
3 Complexity
4 Time Complexity
O (Big O) Notation
Ω (Big Omega) Notation
Θ (Theta) Notation
o (Small / Little o) Notation
ω (Small / Little omega) Notation
Properties of asymptotic notations
5 Space Complexity

Introduction to Data Structures Introduction to Data Structures
1 Introduction to Data Structures Data are simply values or sets of values.

Data Structure Operations A data item refers to a single unit of values.
Data items that are divided into subitems are called group items;
2 Introduction to Algorithms those that are not are called elementary items.
Collections of data are frequently organized into a hierarchy of
3 Complexity
fields, records and files.
4 Time Complexity An entity is something that has certain attributes or properties
O (Big O) Notation which may be assigned values.
Ω (Big Omega) Notation The values themselves may be either numeric or nonnumeric.
Θ (Theta) Notation
o (Small / Little o) Notation Entities with similar attributes form an entity set.
ω (Small / Little omega) Notation Each attribute of an entity set has a range of values, the set of all
Properties of asymptotic notations possible values that could be assigned to the particular attribute.
5 Space Complexity The term information is sometimes used for data with given at-
tributes, or, in other words, meaningful or processed data.
The way that data are organized into the hierarchy of fields, records Each record in a file may contain many field items, but the value
and fiels reflects the relationship between attributes, entities and in a certain field may uniquely determine the record in the file.
entity sets.
Records may also be classified according to length.
I A field is a single elementary unit of information representing I In fixed-length records, all the records contain the same data
an attribute of an entity, items with same amount of spaces assigned to each data
I A record is the collection of field values of a given entity and
item.
I A file is the collection of records of the entities in a given I In variavle-length records, file records may contain different
entity set. lengths.

The organization of data into fields, records and files may not be
complex enough to maintain and efficiently process certain col-
lections of data. For this reason, data are also organized into
more complex types of structures. The study of such data stuc-
tures includes the following three steps:
I Logical or mathemetical description of the structure Data may be organized in many different ways; the logical or
I Implementation of the structure on a computer mathematical model of a particular organization of data is called
I Quantitative analysis of the structure, Which includes deter- a data structure.
mining the amount of memory needed to store the structure
and the time required to process the structure.
Note: The second and third of the steps in the study of data
structures depend on whether the data are stored (a) in the
primary memory of the computer or (b) in a secondary stor-
age unit. The first case deals with the data structure and the
second case, called file managment or data base manage-
ment.
Classification of Data Structure Classification of Data Structure

2 Introduction to Algorithms
Data Structures are normally classified into two broad categories:
3 Complexity
4 Time Complexity 1. Primitive Data Structure

O (Big O) Notation 2. Non-primitive data Structure
Θ (Theta) Notation
5 Space Complexity

Data structures are also classified as:

1. Linear: A data structure is said to be linear if its elements
from a sequence, or, in other words, a linear list, e.g., Arrays,
Linked Lists, etc.
2. Nonlinear: If it is not then it is called nonlinear structures
such as trees and graphs.
Primitive Data Structure I
Data types: A particular kind of data item, as defined by the values it Primitive data structures are basic structures and are directly op-
can take, the programming language used, or the operations that can erated upon by machine instructions.
be performed on it.
Primitive data structures have different representations on differ-
ent computers.
Integers, floats, character and pointers are examples of primitive
data structures.

Primitive Data Structure II Non primitive Data Type I
These data types are available in most programming languages

as built in type.
I Integer: It is a data type which allows all values without These are more sophisticated data structures.
fraction part. We can use it for whole numbers.
I Float: It is a data type which use for storing fractional num- These are derived from primitive data structures.
bers. The non-primitive data structures emphasize on structuring of a
I Character: It is a data type which is used for character val- group of homogeneous or heterogeneous data items.
ues.
I Pointer: A variable that holds memory address of another Examples of Non-primitive data type are Array, List, and File etc.
variable are called pointer.
Non primitive Data Type II Linear data structures I
A Non-primitive data type is further divided into Linear and Non- A data structure is said to be Linear, if its elements are connected
Linear data structure in linear fashion by means of logically or in sequence memory
I Array: An array is a fixed-size sequenced collection of ele- locations.
ments of the same data type.
There are two ways to represent a linear data structure in mem-
I List: An ordered set containing variable number of elements
ory,
is called as Lists.
I File: A file is a collection of logically related information. It I Static memory allocation
can be viewed as a large list of records consisting of various I Dynamic memory allocation
fields.
The possible operations on the linear data structure are: Traver-
sal, Insertion, Deletion, Searching, Sorting and Merging.

Linear data structures II Nonlinear data structures I
Examples of Linear Data Structure are Stack and Queue.

Stack: Stack is a data structure in which insertion and deletion
operations are performed at one end only.
I The insertion operation is referred to as PUSH and deletion
operation is referred to as POP operation. Nonlinear data structures are those data structure in which data
I Stack is also called as Last in First out (LIFO) data structure. items are not arranged in a sequence.
Queue: The data structure which permits the insertion at one Examples of Non-linear Data Structure are Tree and Graph.
end and Deletion at another end, known as Queue.
I End at which deletion is occurs is known as FRONT end and
another end at which insertion occursknown as REAR end.
I Queue is also called as First in First out (FIFO) data struc-
ture.
Nonlinear data structures II Nonlinear data structures III
Graph: Graph is a collection of nodes (Information) and connect-

ing edges (Logical relation) between nodes.
Tree: A tree can be defined as finite set of data items (nodes)
in which data items are arranged in branches and sub branches I A tree can be viewed as restricted graph.
according to requirement. I Graphs have many types:
• Un-directed Graph
I Trees represent the hierarchical relationship between vari- • Directed Graph
ous elements. • Mixed Graph
I Tree consist of nodes connected by edge, the node repre- • Multi Graph
sented by circle and edge lives connecting to circle. • Simple Graph
• Null Graph
• Weighted Graph

Classification of Data Structure Data Structure Operations
Difference between Linear and Non Linear Data 1 Introduction to Data Structures
Structure I Classification of Data Structure
3 Complexity
4 Time Complexity
O (Big O) Notation
Θ (Theta) Notation
5 Space Complexity
Data Structure Operations Data Structure Operations
I Traversing: Accessing each record exactly once so that

The data appearing in our data structures are processed by means certain items in the record may be processed. This access-
of certain operations. The following operations play a major role ing and processing is sometimes called “visiting” the record.
in the data structures: I Searching: Finding the location of the record with a given
key value, or finding the locations of all records which satisfy
I Create: The create operation results in reserving memory
one or more conditions.
for program elements. This can be done by declaration state- I Inserting: Adding a new record to the structure.
ment. Creation of data structure may take place either dur- I Deleting: Removing a record from the structure.
ing compile-time or run-time. malloc() function of C lan- I Sorting: Sorting is a process of arranging all data items in a
guage is used for creation.
data structure in some logical order, say for example, either
I Destroy: Destroy operation destroys memory space allo-
in ascending order or in descending order.
cated for specified data structure. free() function of C lan- I Merging: Combining the records in two different sorted list/files
guage is used to destroy data structure.
into a single sorted list/file.
I Selection: Selection operation deals with accessing a par- I Splitting: Splitting is a process of partitioning single list to
ticular data within a data structure. multiple list.
I Updation: It updates or modifies the data in the data struc- I Other operations, e.g., copying, concatenation, etc.
ture.

Introduction to Algorithms Introduction to Algorithms

Computer Science is the Study of Algorithms and Data
3 Complexity
There is an intimate connection between synthesis of algorithms
and structuring of data
4 Time Complexity
O (Big O) Notation Thus, Data structures and algorithms should be thought of as a
Ω (Big Omega) Notation single unit, neither making sense without the other.
Θ (Theta) Notation
5 Space Complexity
Introduction to Algorithms Introduction to Algorithms
Algorithm study comprises four distinct areas:

I Machines for executing algorithms: It includes from pocket Definition of algorithm: An algorithm is a finite set of instructions,
calculator to largest general purpose digital computer, and which, if followed, accomplish a particular task. In addition, every
the goal is to study various machine fabrication and organi- algorithm must satisfies the following criteria:
zation so that the algorithms can be effectively carried out. I Input: There are zero or more quantities which are exter-
I Languages for describing algorithms: Languages are on nally supplied.
a continuum- At one end are the languages closest to the I Output: At least one quantity is produced.
physical machines and at the other end are the languages I Definiteness: Each instruction must be clear and unam-
designed for sophisticated problem solving. biguous
I Foundation of algorithms: It relates basics regarding the I Finiteness: In all cases, the algorithm must terminate after
possibility/designing algorithms. For instance, Is a particular finite number of steps
task accomplishable by a computing device? What are min- I Effectiveness: Each instruction must be sufficiently basic
imum number operations are necessary for an algorithm to and contribute to problem solution.
complete function/problem? I Correctness: Algorithm must produce correct output in all
I Analysis of algorithms: To study/analysis each steps of an
instances.
algorithm so that its performance in terms of time require-
ments and memory storage as a whole can be estimated.

Introduction to Algorithms Complexity

Difference between algorithm and program-A program (Operat-
ing system) does not satisfy the Finiteness. An algorithm must
terminate. 2 Introduction to Algorithms
An algorithm can be described in many ways- English- but careful 3 Complexity

that the instructions are Definite,
I Flowchart: In graphical form using different symbols, 4 Time Complexity
I Pseudocode: Almost code, that means an algorithm should O (Big O) Notation
be presented in such a way that algorithm and code have Ω (Big Omega) Notation
one-to-one relationship. Θ (Theta) Notation
5 Space Complexity
Complexity Time Complexity

The main objects of study in the field of computational complexity Data Structure Operations
include the time and space needed by an algorithm in order to
deliver its output when presented with legal input. 2 Introduction to Algorithms
The three cases one usually investigates in complexity theory are
as follows: 3 Complexity
I Worst case: The maximum value of f (n) for any possible
4 Time Complexity
input. O (Big O) Notation
I Average case: The expected value of f (n). It need assump-
tion of statistical distribution of inputs. Θ (Theta) Notation
I Best case (Bogus case): The minimum possible value of
f (n). It cheat with a slow algorithm that works fast on some ω (Small / Little omega) Notation
input. Properties of asymptotic notations
5 Space Complexity

Time Complexity Time Complexity
Order of growth I Order of growth II

This is supported by many factors, some of which are the follow-
ing:
I First, when analyzing the running time of an algorithm, we
Obviously, it is meaningless to say that an algorithm A, when pre-
sented with input x, runs in time y seconds. usually compare its behavior with another algorithm that solves
the same problem, or even a different problem. Thus, our
This is because the actual time is not only a function of the al- estimates of times are relative as opposed to absolute.
gorithm used: it is a function of numerous factors, e.g., how and I Second, it is desirable for an algorithm to be not only ma-
on what machine the algorithm is implemented and in what lan- chine independent, but also capable of being expressed in
guage or even what compiler or programmer’s skills, to mention any language, including human languages. Moreover, it should
a few. be technology independent, that is, we want our measure of
the running time of an algorithm to survive technological ad-
Therefore, we should be content with only an approximation of
vances.
the exact time. I Third, our main concern is not in small input sizes; we are
mostly concerned with the behavior of the algorithm under
investigation on large input instances.
Order of growth III Order of growth IV
In fact, counting the number of operations in some “reasonable”

implementation of an algorithm is more than what is needed.
As a consequence of the third factor above, we can go a giant
step further: A precise count of the number of all operations is Once we dispose of lower order terms and leading constants from
very cumbersome, if not impossible, and since we are interested a function that expresses the running time of an algorithm, we
in the running time for large input sizes, we may talk about the say that we are measuring the asymptotic running time of the
rate of growth or the order of growth of the running time. algorithm. Equivalently, in the analysis of algorithms terminology,
we may refer to this asymptotic time using the more technical
I For instance, if we can come up with some constant c > 0 term “time complexity”.
such that the running time of an algorithm A when presented
with an input of size n is at most cn2 , c becomes inconse-
quential as n gets bigger and bigger. Therefore, we may say
about the running times of algorithm A to be “of order” or “in
the order of” n2 .

Order of growth V Order of growth VI

Order of growth VII Order of growth VIII
Definition: We denote by an “elementary operation” any compu-

tational step whose cost is always upperbounded by a constant
amount of time regardless of the input data or the algorithm used.
Let us take, for instance, the operation of adding two integers. If we want to add arbitrarily large numbers, an algorithm whose
For the running time of this operation to be constant, we stipulate running time is proportional to its input size can easily be written
that the size of its operands be fixed no matter what algorithm is in terms of the elementary operation of addition.
used. Likewise, we can choose from a large pool of operations and ap-
Furthermore, as we are now dealing with the asymptotic running ply the fixed-size condition to obtain as many number of elemen-
time, we can freely choose any positive integer k to be the “word tary operations as we wish.
length” of our “model of computation”.
Incidentally, this is but one instance in which the beauty of asymp-
totic notation shows off; the word length can be any fixed positive
integer.

O (Big O) Notation
Order of growth IX 1 Introduction to Data Structures

The following operations on fixed-size operands are examples of
elementary operation.
I Arithmetic operations: addition, subtraction, multiplication
and division. 3 Complexity
I Comparisons and logical operations.
I Assignments, including assignments of pointers when, say, 4 Time Complexity
traversing a list or a tree. O (Big O) Notation
In order to formalize the notions of order of growth and time com- Ω (Big Omega) Notation
plexity, special mathematical notations have been widely used. Θ (Theta) Notation
These notations make it convenient to compare and analyze run- ω (Small / Little omega) Notation
ning times with minimal use of mathematics and cumbersome Properties of asymptotic notations
calculations.
5 Space Complexity
O (Big O) Notation O (Big O) Notation
Definition: Let f (n) and g(n) be two functions from the set of natural
numbers to the set of nonnegative real numbers. f (n) is said to be
O(g(n)) if there exists a natural number n0 and a constant c > 0 such
that Consequently, if limn→∞ f (n)/g(n) exists, then
∀n ≥ n0 , f (n) ≤ cg(n).
lim f (n)/g(n) 6= ∞ implies f (n) = O(g(n)).
n→∞
Informally, this definition says that f grows no faster than some con-
stant times g. The O-notation can also be used in equations as a
simplification tool. This is helpful if we are not interested in the details
of the lower order terms.

Ω (Big Omega) Notation Ω (Big Omega) Notation

Classification of Data Structure Definition: Let f (n) and g(n) be two functions from the set of natural
Data Structure Operations numbers to the set of nonnegative real numbers. f (n) is said to be
Ω(g(n)) if there exists a natural number n0 and a constant c > 0 such
2 Introduction to Algorithms that
∀n ≥ n0 , f (n) ≥ cg(n).
3 Complexity
4 Time Complexity
O (Big O) Notation
Θ (Theta) Notation
5 Space Complexity
Ω (Big Omega) Notation Θ (Theta) Notation

Consequently, if limn→∞ f (n)/g(n) exists, then 2 Introduction to Algorithms
lim f (n)/g(n) 6= ∞ implies f (n) = Ω(g(n)). 3 Complexity

n→∞
Informally, this definition says that f grows at least as fast as some 4 Time Complexity
constant times g. It is clear from the definition that f (n) is Ω(g(n)) if O (Big O) Notation
and only if g(n) is O(f (n)). Ω (Big Omega) Notation
Θ (Theta) Notation
5 Space Complexity

Θ (Theta) Notation Θ (Theta) Notation
Definition: Let f (n) and g(n) be two functions from the set of natural
Θ(g(n)) if there exists a natural number n0 and two positive constants
c1 and c2 such that
Consequently, if limn→∞ f (n)/g(n) exists, then
∀n ≥ n0 , c1 g(n) ≤ f (n) ≤ c2 g(n).
lim f (n)/g(n) 6= c implies f (n) = Θ(g(n)).
n→∞
where c is a constant strictly greater than 0.

An important consequence of the above definition is that f (n) = Θ(g(n))
if and only if f (n) = O(g(n)) and f (n) = Ω(g(n)).
o (Small / Little o) Notation o (Small / Little o) Notation

2 Introduction to Algorithms Definition: Let f (n) and g(n) be two functions from the set of natural
o(g(n)) if for every constant c > 0 there exists a positive integer n0
3 Complexity
such that f (n) < cg(n) for all n ≥ n0 .
4 Time Complexity
O (Big O) Notation lim f (n)/g(n) = 0 implies f (n) = o(g(n)).
Ω (Big Omega) Notation n→∞
Θ (Theta) Notation
5 Space Complexity

o (Small / Little o) Notation ω (Small / Little omega) Notation

Informally, this definition says that f (n) becomes insignificant rel-
ative to g(n) as n approaches infinity. It follows from the def-
inition that f (n) = o(g(n)) if and only if f (n) = O(g(n)), but
g(n) 6= O(f (n)).
For example, n log n is o(n2 ) is equivalent to saying that n log n is 3 Complexity
O(n2 ) but n2 is not O(n log n).
We also write f (n) ≺ g(n) to denote that f (n) is o(g(n)). Using 4 Time Complexity
this notation, we can concisely express the following hierarchy of O (Big O) Notation
complexity classes. Ω (Big Omega) Notation
Θ (Theta) Notation
√ 2 o (Small / Little o) Notation
1 ≺ log log n ≺ log n ≺ n ≺ n3/4 ≺ n ≺ n log n ≺ n2 ≺ 2n ≺ n! ≺ 2n .
5 Space Complexity
ω (Small / Little omega) Notation Properties of asymptotic notations

Definition: Let f (n) and g(n) be two functions from the set of natural Classification of Data Structure
numbers to the set of nonnegative real numbers. f (n) is said to be Data Structure Operations
ω(g(n)) if for every constant c > 0 there exists a positive integer n0
such that f (n) > cg(n) for all n ≥ n0 . 2 Introduction to Algorithms
3 Complexity
lim f (n)/g(n) = ∞ implies f (n) = ω(g(n)).
n→∞
4 Time Complexity
O (Big O) Notation
We use ω-notation to denote a lower bound that is not asymptot- Ω (Big Omega) Notation
ically tight. One way to define it is by f (n) ∈ ω(g(n)) if and only if Θ (Theta) Notation
g(n) ∈ o(f (n)). o (Small / Little o) Notation
For example, n2 /2 = ω(n), but n2 /2 6= ω(n2 ). ω (Small / Little omega) Notation
5 Space Complexity

Properties of asymptotic notations Properties of asymptotic notations
Transitivity: f (n) = Θ(g(n)) and g(n) = Θ(h(n)) =⇒ f (n) =

Θ(h(n)). Valid for O and Ω as well. We generally focus on the upper bound (O) because knowing the
Reflexivity: f (n) = (f (n)). Valid for O and Ω. lower bound (Ω) of an algorithm is of no practical importance, and
we use the Θ notation if the upper bound (O) and lower bound (Ω)
Symmetry: f (n) = Θ(g(n)) if and only if g(n) = Θ(f (n)). are the same.
Transpose symmetry: f (n) = O(g(n)) if and only if g(n) = In every case for a given function f (n) we are trying to find another
Ω(f (n)). function g(n) which approximates f (n) at higher values of n. That
If f (n) is in O(kg(n)) for any constant k > 0, then f (n) is in means g(n) is also a curve which approximates f (n) at higher
O(g(n)). values of n. In mathematics we call such a curve an asymptotic
curve. In other terms, g(n) is the asymptotic curve for f (n). For
If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)), then (f1 + f2 )(n) is this reason, we call algorithm analysis asymptotic analysis.
in O(max(g1 (n)), (g2 (n))).
If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)) then f1 (n)f2 (n) is in
O(g1 (n)g2 (n)).
Space Complexity Space Complexity

Now, suppose that we have two algorithms A1 and A2 of run-
2 Introduction to Algorithms ning times in the order of n log n. Which one should we consider
to be preferable to the other? Technically, since they have the
same time complexity, we say that they have the same running
3 Complexity
time within a multiplicative constant, that is, the ratio between the
two running times is constant. In some cases, the constant may
4 Time Complexity be important and more detailed analysis of the algorithm or con-
O (Big O) Notation ducting some experiments on the behavior of the algorithm may
Ω (Big Omega) Notation be helpful. Also, in this case, it may be necessary to investigate
Θ (Theta) Notation other factors, e.g., space requirements and input distribution. The
o (Small / Little o) Notation latter is helpful in analyzing the behavior of an algorithm on the
ω (Small / Little omega) Notation average.
5 Space Complexity

Space Complexity Space Complexity
We define the space used by an algorithm to be the number of

memory cells (or words) needed to carry out the computational
steps required to solve an instance of the problem excluding the Naturally, in many problems there is a time-space tradeoff: The
space allocated to hold the input. In other words, it is only the more space we allocate for the algorithm the faster it runs, and
work space required by the algorithm. The reason for not includ- vice versa. This, of course, is within limits: in most of the algo-
ing the input size is basically to distinguish between algorithms rithms increasing the amount of space does not result in a notice-
that use “less than” linear space throughout their computation. able speedup in the algorithm running time. However, it is almost
All definitions of order of growth and asymptotic bounds pertain- always the case that decreasing the amount of work space re-
ing to time complexity carry over to space complexity. It is clear quired by an algorithm results in a degradation in the algorithm’s
that the work space cannot exceed the running time of an algo- speed.
rithm, as writing into each memory cell requires at least a con-
stant amount of time. Thus, if we let T (n) and S(n) denote, re-
spectively, the time and space complexities of an algorithm, then
S(n) = O(T (n)).
Data Structure: Arrays
Dr. Arup Kumar Pal

Jharkhand-826004
Data Structures and Algorithms Dept. of CSE, IIT(ISM) Dhanbad August 28, 2021 1
Outline
Introduction
One-Dimensional Array
Memory Allocation for an Array
Operations on Arrays
Multidimensional Arrays
Sparse Matrices
Introduction
An array is a finite ordered and collection of
homogeneous data elements.
Array is finite because it contains only limited
number of element; and ordered, as all the
elements are stored one by one in contiguous
locations of computer memory in a linear ordered
fashion.
All the elements of an array are of the same data
type (say, integer) only and hence it is termed as
collection of homogeneous elements.
Contd…
Array is particularly useful when we are dealing with lot
of variables of the same type.
For example, lets say I need to store the marks in math
subject of 100 students.
To solve this particular problem, either I have to create
the 100 variables of int type or create an array of int
type with the size 100.
Obviously the second option is best, because keeping
track of all the 100 different variables is a tedious task.
On the other hand, dealing with array is simple and
easy, all 100 values can be stored in the same array at
different indexes (0 to 99).
Arrays
An array is a collection of similar data elements. These data
elements have the same data type.
An array is conceptually defined as a collection of <index,
value> pairs.
Each array element is referred to by specifying the array
name followed by one or more subscripts, with each
subscript enclosed in square brackets.
Each subscript must be expressed as a nonnegative integer.
Thus, in the n-element array x, the array elements are x[0],
x[1], x[2], …, x[n-1].
The number of subscripts determines the dimensionality of
the array. (for example x[i] refers to an element in one-
dimensional array x, and y[i][j] to an element in 2-D array y)
Declaration of Arrays
Arrays are declared using the following syntax:
data_type name_of_array[size_of_array]
float arr[10]
int mark[20]
Sample code to read input through array
int j, mark[20];
for(j=0;j<20;j++)
scanf(“%d”,&mark[j]);
Array Memory Representation
The following diagram represents an integer array
that has 12 elements.
The index of the array starts with 0, so the array
having 12 elements has indexes from 0 to 11.
Memory Allocation of an Array
Operations on an Array
Retrieval of an element: Given an array index, retrieve
the corresponding value.
This can be accomplished in O(1) time. This is an important
advantage of the array relative to other structures.
Search: Given an element value, determine whether it is
present in the array.
If the array is unsorted, there is no good alternative to a
linear search that iterates through all of the elements in the
array and stops when the desired element is found.
In the worst case, this requires O(n) time.
If the array is sorted, Binary search can be used. The Binary
search only requires O(log n) time.
Contd…
Insertion at required position: for inserting the
element at required position, all of the existing
elements in the array from the desired location must
be shifted one place to the right.
This requires O(n) time in worst case.
Contd…
Deletion at desired position: leaves a “vacant”
location.
Actually, this location can never be vacant because it refers to
a word in memory which must contain some value.
Thus, if the program accesses a “vacant” location, it doesn’t
have any way to know that the location is vacant.
This requires O(n) time in worst case.
Sorted Arrays
There are several benefits to using sorted arrays,
namely: searching is faster, computing order
statistics (the ith smallest element) is O(1), etc.
The sorting or sorted Arrays may be considered as
of pre-processing steps on data to make
subsequent queries efficient.
The idea is that we are often willing to invest some
time at the beginning in setting up a data structure
so that subsequent operatios on it become faster.
Contd…
Some sorting algorithms such as Heap sort and
Merge sort require O(n log n) time in the worst
case
Whereas other simpler sorting algorithms such
as insertion sort, bubble sort and selection sort
require O(n2) time in the worst case.
Quick sort have a worst case time of O(n2), but
require O(n log n) on the average.
Advantages and disadvantages of Arrays
Advantages
Reading an array element is simple and efficient. The time
complexity is O(1) in both best and worst cases.
This is because any element can be instantly read using indexes
(base address calculation behind the scene) without traversing
the whole array.
Disadvantages
While using array, we must need to make the decision of the
size of the array in the beginning, so if we are not aware how
many elements we are going to store in array, it would make
the task difficult.
The size of the array is fixed so if at later point, if we need to
store more elements in it then it can’t be done. On the other
hand, if we store less number of elements than the declared
size, the remaining allocated memory is wasted.
Row- or Column- Major Representation
Earlier representations of multidimensional arrays
mapped the location of each element of the
multidimensional array into a location of a one-
dimensional array.
Consider a two dimensional array with r rows and c
columns. The number of elements in the array n = rc.
The element in location [i][j], 0 ≤ i < r and 0 ≤ j < c, will be
mapped onto an integer in the range [0, n−1].
If this is done in row-major order — the elements of row 0
are listed in order from left to right followed by the
elements of row 1, then row 2, etc. — the mapping
function is ic + j.
If elements are listed in column-major order, the mapping
function is jr+i.
Contd…
Memory Allocation in 2D-arrays (A mxn)
Logically, a matrix appears as two-dimensional but
physically it is stored in a linear fashion.
In order to map from logical view to physical
structure, we need indexing formula.
Contd…
Sparse Matrices
A matrix is sparse if a large number of its elements are
zeros.
Storing of null elements of a sparse matrix is nothing
but wastage of memory
Rather than store such a matrix as a two-dimensional
array with lots of zeroes, a common strategy is to save
space by explicitly storing only the non-zero elements.
There are several ways to represent this matrix as a
one-dimensional array.
Contd…
Other sparse matrices may have an irregular or
unstructured pattern.
Contd…
Some well known sparse matrices which are symmetric
in form:
Memory Representation
Contd…
Contd…
Contd…
Contd…
Contd…
Contd…
Contd…
Contd…
Three Dimensional (3-D) Array
Reference Book
Classic Data Structures by D. Samanta
Thank You…
Any Queries...?
Arrays Arrays
Outline I
1 Introduction
Arrays
2 Linear Arrays
Representation of Liniear Arrays in Memory
Travarsing Linear Arrays
Inserting and Deleting
Searching
Linear Search
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Introduction Introduction
Classification
1 Introduction
2 Linear Arrays Data structures are classified as either linear or nonlinear

Travarsing Linear Arrays A data structure is said to be linear if its elements form a linear
Inserting and Deleting list
Searching I Linear - Arrays and Linked lists
Linear Search I Nonlinear - Trees and Graphs
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Introduction Linear Arrays
Arrays
1 Introduction
In arrays linear relationship between elements is represented by

means of sequential memory locations. 2 Linear Arrays
The following are the operations on any linear structure: Travarsing Linear Arrays
I Traversal Inserting and Deleting
I Search Searching
I Insertion and Deletion Linear Search
I Sorting Binary Search
I Merging Multidimensional Arrays
Matrices
Sparse Matrices
Arrays Arrays
Linear Arrays Linear Arrays
Linear Arrays I Linear Arrays II
A linear array is a list of a finite number n of homogenous data

elements such that The elements of the array can be denoted by
1. The elements of the array are referenced respectively by an
I Parentheses notation
index set consisting of n consecutive numbers
2. The elements of the array are stored respectively in succes- A(1), A(2), A(3), · · · A(n)
sive memory locations I Bracket notation
A[1], A[2], A[3], · · · A[n]
The number n of elements is called length or size of the array. I Subscript notation
Length can also be calculated by
A1 , A2 , A3 , · · · An
length=upperbound-lowerbound+1
where upperbound is the largest index and lowerbound is the

smallest index of the array
Arrays Arrays
Example
1 Introduction
125 295 295 163 176 264 2 Linear Arrays

A
1 2 3 4 5 6 Representation of Liniear Arrays in Memory
length=6 Inserting and Deleting
A[1]=125, A[2]=295 Searching
Linear Search
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Representation of Liniear Arrays in Memory Travarsing Linear Arrays
The elements of linear array are stored in successive memory 1 Introduction

cells
Accordingly computer does not keep track of address of every
element but tracks only address of first element
2 Linear Arrays
Let A be an array.The address of first element is denoted by Representation of Liniear Arrays in Memory
Base(A) Travarsing Linear Arrays
The computer calculates any address of A by using the formula Inserting and Deleting
Searching
LOC(A[K])=Base(A)+w(K-lower bound) Linear Search
Here w is number of words per memory cell for array A Binary Search
Any element A[K] can be located and processed in a time inde-
Matrices
pendent of K. This is an important property of linear arrays Sparse Matrices
Arrays Arrays
Travarsing Linear Arrays Travarsing Linear Arrays
Array Traversal I Array Traversal II
Let A be an array.Suppose we want to print the elements of A or we

want to count number of elements of A with a given property. This can
be accomplished by traversing This algorithm is an alternative form of the above algorithm which uses
Algorithm 1: Let LA be a linear array with lower bound LB and upper repeat for loop instead of repeat while loop
bound UB Algorithm 2:
1. Set K:=LB [Initialize counter] 1. Repeat for K=LB to UB:
2. Repeat steps 3 and 4 while K≤UB Apply PROCESS to LA[K]
End of loop
3. Apply PROCESS TO A[K] [Visit element]
2. Exit
4. Set K:=K+1 [Increase counter]
End of step 2 loop
5. Exit
Arrays Arrays
Inserting and Deleting Inserting and Deleting
Let A be an array
1 Introduction Insertion refers to the operation of adding another element to A
Deletion refers to removing one of the elements of A
Inserting at the end can be easily done provided memory space
2 Linear Arrays allocated for array is large enough
Travarsing Linear Arrays For insertion at the middle on an average half of the elements
Inserting and Deleting must be moved downward to new locations to accommodate ele-
Searching ment and keep order of other elements
Linear Search
Similarly deletion at end has no difficulty while deletion at the
Binary Search
middle requires each subsequent element be moved one location
upward to fill array
Matrices
Sparse Matrices Here downward refers to location with larger subscripts and up-
ward refers to smaller subscripts
Arrays Arrays
Inserting and Deleting Inserting and Deleting
Insertion Deletion
Insertion Algorithm
Deletion algorithm
Insert element ITEM into Kth position of LA
INSERT(LA, N, K, ITEM) Delete Kth element from LA and assign it to item
1. [Initialize counter.] Set J:=N DELETE(LA,N,K,ITEM)
2. Repeat steps 3 and 4 while J≥ K 1. Set ITEM:=LA[K]
3. [Move Jth element downward.] Set LA[J+1]:= LA[J] 2. Repeat for J=K to N-1 :
[Move J+1st element upward] Set LA[J]:=LA[J+1]
4. [Decrease counter.] Set J:= J-1
End of step 2 loop [End of loop]
5. [Insert element.] Set LA[K]:= ITEM 3. [Reset the number N of elements in LA] Set N:=N-1
6. [Reset N.] Set N:= N+1 4. Exit
7. Exit
Arrays Arrays
Searching Searching
1 Introduction
1. Searching refers to finding location of a particular item in the

given data
2 Linear Arrays 2. Search is said to be successful if the item appears in the data
Representation of Liniear Arrays in Memory and unsuccessful otherwise
Inserting and Deleting 3. There are many different search algorithms. The algorithm one
Searching chooses depends on the way data is organized
Linear Search
4. Two popular searching algorithms are linear and binary search
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Searching Searching
1 Introduction
Here in order to search for a given ITEM in DATA we compare

2 Linear Arrays
ITEM with each element of DATA
Travarsing Linear Arrays This method which traverses DATA sequentially to locate ITEM is
Inserting and Deleting called linear search or sequential search
Searching
Linear Search
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Searching Searching
Algorithm1 A linear array DATA with N elements and a specific ITEM

of information are given. This algorith finds the location LOC of ITEM
in the array DATA or sets LOC=0. Algorithm2 This algorithm finds LOC of ITEM in DATA
1. [Initialize] Set K:=1 and LOC:=0. LINEAR(DATA,N,ITEM,LOC)
2. Repeat Steps 3 and 4 while LOC:=0 and k≤N. 1. [Insert ITEM at end of DATA] Set DATA[N+1]:=ITEM
3. If ITEM=DATA[K], then: Set LOC:=K. 2. [Initialize counter.] Set LOC:=1
4. Set k:=k+1.[Increments counter.] 3. [Search for ITEM.]
[End of Step 2 loop.] Repeat while DATA[LOC]6=ITEM
Set LOC:=LOC+1
5. [Successful?]
If LOC=0, then: [End of loop]
Write: ITEM is not in the array DATA. 4. [Successful?] If LOC=N+1, then: Set LOC:=0
Else: 5. Exit
Write: LOC is the location of ITEM.
[End of If structure.]
6. Exit.
Arrays Arrays
Searching Searching
Complexity
1 Introduction
The complexity is measured by no. of comparisions required to 2 Linear Arrays

find ITEM in DATA. Representation of Liniear Arrays in Memory
Clearly the worst case occurs when one must search through the Inserting and Deleting
entire array. Thus, worst case running time is proportional to n. Searching
What is the running time of the average case? Linear Search
Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Searching Searching
Binary search Binary Search

Algorithm Here DATA is sorted LB is lower bound UB is upper bound.
Algorithm finds LOC of ITEM in DATA or sets LOC=NULL.
BINARY(DATA,LB,UB,ITEM,LOC)
1. [Initialize segment variables.]

Suppose data in an array is sorted in increasing numerical order Set BEG:=LB, END:=UB and MID=INT((BEG+END)/2).
then binary search is very efficient
2. Repeat steps 3 and 4 while DATA[MID]6=ITEM and BEG≤END.
Binary search works as follows: 3. If ITEM<DATA[MID] then:

Set END:=MID-1.
I Assign first=0 and last=N-1 Else:
Set BEG:=MID+1.
first + last
I Calculate Mid= [End of If structure.]
2
I If ITEM is less than A[mid] then make last=mid and repeat 4. Set MID:= INT ((BEG+END)/2).
[End of Step 2 loop.]
I Otherwise make first=mid and repeat
5. If DATA[MID]=ITEM, then:
Set LOC:=MID.
Else:
Set LOC:=NULL.
6. Exit
Arrays Arrays
Searching Searching
Complexity Limitation
Here we can observe that each comparison reduces the sample Binary search algorithm requires two conditions:
size in half,Hence the worst case time complexity will be equal to I The list must be sorted.
log2 n. I One must have direct access to the middle element in any
Hence binary search will be more efficient when the array is sorted. sublist.
Arrays Arrays
Multidimensional Arrays Multidimensional Arrays
Multidimensional Arrays I
1 Introduction
Two-dimensional arrays
2 Linear Arrays A two dimensional array A is a collection of m.n data elements repre-
Representation of Liniear Arrays in Memory sented as A[J,K] such that 1≤J≤m and 1≤K≤n
Travarsing Linear Arrays  
Inserting and Deleting A[1, 1] A[1, 2] A[1, 3] A[1, 4]
Searching A[2, 1] A[2, 2] A[2, 3] A[2, 4]
Linear Search A[3, 1] A[3, 2] A[3, 3] A[3, 4]
Binary Search
Multidimensional Arrays Two dimensional array with 3 rows and 4 columns.
Matrices
Sparse Matrices
Arrays Arrays
Representation of Two-Dimensional Arrayas in Representation of Two-Dimensional Arrayas in

Memory I Memory II
The programming language will store the array A :

I Column-major order: Column by column.
I Row-major order: Row by row.
Arrays Arrays
Representation of Two-Dimensional Arrayas in General Multidimensional arrays I

Memory III
General Multidimensional array is a collection of m1 .m2 . · · · .mn

data elements.
Column-major order:
The element is denoted by A[k1 ][k2 ] · · · [kn ] such that
LOC(A[J, K]) = Base(A) + w[M[K - 1) + (J - 1)]
1 ≤ K1 ≤ m1 , 1 ≤ K2 ≤ m2 , · · · , 1 ≤ Kn ≤ mn .
Row-major order: Column-major order:
LOC(A[J, K]) = Base(A) + w[N[J - 1) + (K - 1)] LOC(A[K1 , K2 , · · · , Kn ]) = Base(A) + w[(((· · · (EN LN−1 + EN−1 )LN−2 ) + · · · + E3 )L2 + E2 )L1 + E1 ]
Row-major order:
LOC(A[K1 , K2 , · · · , Kn ]) = Base(A) + w[(((· · · (E1 L2 + E2 )L3 ) + E3 )L4 + · · · + EN−1 )LN + EN ]
Arrays Arrays
Multidimensional Arrays Matrices
General Multidimensional arrays II

1 Introduction
Where, the length Li of dimension i of C is the number of ele-

ments in the index set, and Li can be4 calculated, as 2 Linear Arrays
Li = upper bound - lower bound + 1 Representation of Liniear Arrays in Memory
and, for a given subscript Ki , the effective index Ei of Li is the
Inserting and Deleting
number of indeces preceding Ki in the index set, and Ei can be
Searching
calculated from Linear Search
Ei = ki - lower bound Binary Search
Matrices
Sparse Matrices
Arrays Arrays
Matrices Matrices
Matrices I Matrices II
Matrix multiplication Algorithm
Here we have A=m.p and B=p.n and C=A.B
MATMUL(A,B,C,M,P,N)
1. Repeat Steps 2 to 4 for I=1 to M:

Algebra of matrices
Suppose A is an m.p matrix and B is an p.n matrix product of A and B 2. Repeat Steps 3 and 4 for J=1 to N:
is m.n matrix whose Cij can be written as 3. Set C(I,J]:=0. [Initialize C(I,J].]
Pp 4. Repeat for K=1 to P:

Cij = Ai1 B1j + Ai2 B2j + · · · + Aip Bpj = k=1 Aik Bkj C(I,J]:=C(I,J]+A[I,K]*B[K,J]
[End of inner loop.]
[End of Step2 middle loop.]
[End of Step1 outer loop.]
5. Exit
Complexity: C = m.n.p
Arrays Arrays
Matrices Matrices
Sparse matrices I
1 Introduction
2 Linear Arrays A general matrix consists of m rows and n columns of numbers.

Representation of Liniear Arrays in Memory We write m × n to designate a matrix with m rows and n columns.
Travarsing Linear Arrays Such a matrix has mn elements. When m = n, we call the matrix
Inserting and Deleting square.
Searching
Linear Search It is very normal to store a matrix in two-dimensional array, say
Binary Search A[1 : m, 1 : n]. Then, we can work with any element by writing
Multidimensional Arrays A(i, j).
Matrices
Sparse Matrices
Arrays Arrays
Matrices Matrices
Sparse matrices II Sparse matrices III
The second matrix of the figure has many zero entries. Such a
matrix is called sparse.
In second matrix, only 8 out of 36 entries are non-zero and that Matrices with a relatively high proportion of zero entries are called
is sparse! sparse matrices for example lower and upper triangular matrices
Here one may save space by storing only those nonzero entries
Arrays Arrays
Matrices Matrices
Representation of Sparse Matrix I Representation of Sparse Matrix II
A specific representation is used for sparse matrix which only

stores the non-zero elements.
Each element of a matrix is uniquely characterized by its row and
column position, say (i, j). Matrix is stored as a list of 3-tuples of
the form (i, j, value).
Sparse matrix A(0 : t, 1 : 3) where t = 8 is the number of non-
zero terms.
The element A(0, 1) and A(0, 2) contain the number of rows and
columns of the matrix. A(0, 3) contains the number of non-zero
terms.
Arrays Arrays
Matrices Matrices
Representation of Sparse Matrix III Transpose of a Sparse Matrix I
Transpose operation can be applied over sparse matrix.

For example we want to represent a lower triangular matrix In transpose operation, we move the elements so that the ele-
ment in the i, j position gets put in the j, i position, i.e., inter-
I Suppose AJK is lower triangular matrix. Then we can store
changing row and column.
the nonzero entries in another array B
I For B[L]=AJK we want L in terms of J and K The elements of diagonal will remain unchanged, since i = j.
I Here elements upto Jth row are
1+2+...(J-1)=J(J-1)/2
I Then Kth element in Jth row is L=J(J-1)/2+K
I Thus L is represented in terms of J and K
Arrays Arrays
Matrices Matrices
Transpose of a Sparse Matrix II Transpose of a Sparse Matrix III
Arrays Arrays
Matrices Matrices
Transpose of a Sparse Matrix IV Product of Sparse Matrices I
For matrices A and B, where A is m×n and B is n×p, the product

matrixPC has dimension m × p. Its (i, j) element is defined as
cij = 1≤k ≤n aik bkj
For 1 ≤ i ≤ m and 1 ≤ j ≤ p, the product of two sparse matrices
may no longer be sparse, for instance,
Linked Lists Linked Lists
Outline I
1 Linked Lists
Linked Lists Representation of Linked Lists in Memory
Traversing a Linked List
Searching a Linked List
Memory Allocation
Insertion into a Linked List
Deletion from a Linked List
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix

Introduction I
A linked list is a linear data structure, in which the elements are

1 Linked Lists
not stored at contiguous memory locations. The elements in a
Representation of Linked Lists in Memory
linked list are linked using pointers(entity that point to the next
elements)
Memory Allocation In simple words, a linked list consists of nodes where each node
Insertion into a Linked List contains a data field and a link field or nextpointer field, contains
Deletion from a Linked List the address of the next node in the list.
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix
Figure: Linked List with 6 Nodes

Introduction II Introduction III
The pointer of the last node contains a special value, called the Advantages of Linked list over Arrays :
null pointer, which is any invalid address.
1. Dynamic size
Null Pointer- It is denoted by in the diagram, signals the end of 2. Ease of insertion/deletion
the list.
Disadvantages of Linked list over Arrays :
The linked list also contains a list pointer variablecalled START
or NAME - which contains the address of the first node in the 1. Random access is not allowed.
list; hence there is an arrow drawn from START to the first node. 2. Extra memory space for pointer is required with each ele-
Clearly, we need only this address in START to trace through the ment of the list.
list.

Representation of Linked Lists in Memory Representation of Linked Lists in Memory
Representation of Linked Lists in Memory I
1 Linked Lists
Traversing a Linked List First of all, LIST requires two linear arrays - INFO and LINK,
Memory Allocation such that INFO[K] and LINK[K] contain, respectively, the informa-
Insertion into a Linked List tion part and the nextpointer field of a node of LIST.
Deletion from a Linked List LIST also requires a variable name - such as START - which con-
Header Linked Lists tains the location of the beginning of the list,
Two-way Lists
Polynomial and a nextpointer sentinel - denoted by NULL - which indicates
Sparse Matrix the end of the list.
Representation of Linked Lists in Memory Representation of Linked Lists in Memory
Representation of Linked Lists in Memory II Representation of Linked Lists in Memory III
Example:
START = 9, so INFO[9] = N is the first character.
LINK[9] = 3, so INFO[3] = O is the second character.
LINK[3] = 6, so INFO[6] = (blank) is the third character.
LINK[6] = 11, so INFO[11] = E is the fourth character.
LINK[11] = 7, so INFO[7] = X is the fifth character.
LINK[7] = 10, so INFO[10] = I is the sixth character.
LINK[10] = 4, so INFO[4] = T is the seventh character.
LINK[4] = 0, the NULL value, so the list has ended.
In other words, NO EXIT is the character string.

Traversing a Linked List Traversing a Linked List
Traversing a Linked List I
1 Linked Lists In traversing a single linked list, we visit every node in the list
Representation of Linked Lists in Memory starting from the first node to the last node.
Searching a Linked List Our traversing algorithm uses a pointer variable PTR which points
Memory Allocation to the node that is currently being processed.
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix
Traversing a Linked List Traversing a Linked List
Traversing a Linked List II Traversing a Linked List III
Algorithm for traversing a Linked List : The procedure below finds the number NUM of elements in a
(Traversing a Linked List) Let LIST be a linked list in memory. This linked list.
algorithm traverses LIST, applying an operation PROCESS to each We traverse the linked list in order to count the number of elements;
element of LIST. The variable PTR points to the node currently being hence the procedure is very similar to the above traversing algorithm.
processed. Procedure: COUNT(INFO, LINK, START, NUM)
1. Set NUM : = 0. [Initializes counter.]
1. Set PTR := START. [Initializes pointer PTR.]
2. Set PTR : = START. [Initializes pointer.]
2. Repeat Steps 3 and 4 while PTR 6= NULL.
3. Repeat Steps 4 and 5 while PTR 6= NULL.
3. Apply PROCESS to INFO[PTR].
4. Set NUM : = NUM + 1. [Increases NUM by 1.]
4. Set PTR := LINK[PTR]. [PTR now points to the next node.]
5. Set PTR : = LINK[PTR]. [Updates pointer.]
5. Exit.
6. Return.

Searching a Linked List Searching a Linked List
Searching a Linked List I
1 Linked Lists
Memory Allocation LIST Is Unsorted: we require two tests. First we have to check
Insertion into a Linked List to see whether we have reached the end of the list; i.e., first we
Deletion from a Linked List check to see whether PTR = NULL If not, then we check to see
Header Linked Lists whether INFO[PTR] = ITEM.
Two-way Lists
Polynomial
Sparse Matrix
Searching a Linked List II Searching a Linked List III

Searching Algorithm : SEARCH(INFO, LINK, START, ITEM,
LOC) LIST is a linked list.
This algorithm finds the location LOC of the node where ITEM
first appears in LIST, or sets LOC = NULL.
1. Set PTR := START. Complexity is same as that of other linear search algorithms.
2. Repeat Step 3 while PTR 6= NULL:
3. If ITEM = INFO[PTR], then: In sorted linear array we can apply a binary search whose running
Set LOC := PTR, and Exit. time is proportional to log2 n.
Else:
Set PTR := LINK[PTR]. [PTR now points to the
next node.]
4. [Search is unsuccessful] Set LOC := NULL.
5. Exit.

Searching a Linked List IV Searching a Linked List V
List is sorted: SRCHSL(INFO, LINK, START, ITEM, LOC). This

algorithm finds the location LOC of the node where ITEM first
appears in LIST, or sets LOC = NULL.
List is sorted: Suppose the data in LIST are sorted. Again we 1. Set PTR := START
search for ITEM in LIST by traversing the list using a pointer vari- 2. Repeat Step 3 while PTR 6=NULL:
3. If ITEM < INFO[PTR], then:
able PTR and comparing ITEM with the contents INFO[PTR] of
Set PTR := LINK[PTR]. [PTR now points to next node.]
each node, one by one, of LIST. Now, however, we can stop once Else if ITEM = INFO[PTR], then
ITEM exceeds INFO[PTR]. The algorithm follows. Set LOC := PTR, and Exit. [Search is successful.]
Else:
Set LOC := NULL, and Exit. [ITEM now exceeds INFO[PTR].]
4. Set LOC := NULL
5. Exit.
Searching a Linked List Memory Allocation
Searching a Linked List VI
The complexity of this algorithm is still the same as that of other

1 Linked Lists
linear search algorithms
The worst-case running time is proportional to the number n of Traversing a Linked List
elements in LIST Searching a Linked List
Memory Allocation
the average-case running time is approximately proportional to Insertion into a Linked List
n/2. Deletion from a Linked List
On the other hand, a binary search algorithm cannot be applied Header Linked Lists
to a sorted linked list, since there is no way of indexing the middle Two-way Lists
element in the list. Polynomial
Sparse Matrix
This property is one of the main drawbacks in using a linked list
as a data structure.

Memory Allocation Memory Allocation
Memory Allocation I Memory Allocation II
We require some mechanism which provides unused memory

space for the new nodes.
Example : Suppose the list of patients is stored in the linear arrays
Analogously, some mechanism is required whereby the memory
BED and LINK (so that the patient in bed K is assigned to BED[K]).
space of deleted nodes becomes available for future use.
Then the available space in the linear array BED may be linked as in
Together with the linked lists in memory, a special list is main- Fig. shown below
tained which consists of unused memory cells.
This list, which has its own pointer, is called the list of available
space or the free-storage list or the free pool.
Memory Allocation III Memory Allocation IV
Observe that BED[10] is the first available bed,

BED[2] is the next available bed and
BED[6] is the last available bed.
Hence BED[6] has the null pointer in its nextpointer field; that is,
LINK[6] = 0.

Garbage Collection Overflow and Underflow
The operating system of a computer may periodically collect all

the deleted space onto the free-storage list. Any technique which
does this collection is called garbage collection.
Sometimes new data are to be inserted into a data structure but
Garbage collection usually takes place in two steps. there is no available space, i.e., the free-storage list is empty.
First the computer runs through all lists, tagging those cells which This situation is usually called overflow.
are currently in use, and then the computer runs through the The term underflow refers to the situation where one wants to
memory, collecting all untagged space onto the free-storage list. delete data from a data structure that is empty.
The garbage collection may take place when there is only some
minimum amount of space or no space at all left in the free-
storage list, or when the CPU is idle and has time to do the col-
lection.
Insertion into a Linked List Insertion into a Linked List
Insertion into a Linked List I
1 Linked Lists
Memory Allocation
Insertion into a Linked List Algorithms which insert nodes into linked lists come up in various
Deletion from a Linked List situations. We discuss three of them here.
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix

Insertion into a Linked List II Insertion into a Linked List III
Inserting at the Beginning of a List

Algorithm : INSFIRST(INFO, LINK, START, AVAIL, ITEM) This
algorithm inserts ITEM as the first node in the list.
1. [OVERFLOW?] If AVAIL = NULL, then: Write: OVERFLOW, and Exit.
2. [Remove first node from AVAIL list.]
Set NEW := AVAIL and AVAIL := LINK[AVAIL].
3. Set INFO[NEW] := ITEM. [Copies new data into new node]
4. Set LINK[NEW] := START. [New node now points to original first node.]
5. Set START := NEW. [Changes START so it points to the new node.]
Figure: Insertion at the beginning of a list
6. Exit.
Insertion into a Linked List IV Insertion into a Linked List V
Algorithm: INSLOC(INFO, LINK, START, AVAIL, LOC, ITEM)

Inserting after a Given Node : 1. [OVERFLOW?] If AVAIL = NULL, then: Write: OVERFLOW, and Exit.
2. [Remove first node from AVAIL list.]
I Suppose we are given the value of LOC where either LOC Set NEW := AVAIL and AVAIL := LINK[AVAIL].
is the location of a node A in a linked LIST or LOC = NULL. 3. Set INFO[NEW] := ITEM. [Copies new data into new node.]
I The following is an algorithm which inserts ITEM into LIST 4. If LOC = NULL, then: [Insert as first node.]
Set LINK[NEW] := START and START := NEW.
so that ITEM follows node A or,when LOC = NULL, so that Else:[Insert after node with location LOC.]
ITEM is the first node. Set LINK [NEW]:=LINK[LOC] and LINK[LOC]:=NEW.
5. Exit.

Insertion into a Linked List VI Insertion into a Linked List VII

Inserting into a Sorted Linked List :
Procedures:
I ITEM must be inserted between nodes A and B so that
INFO(A) < ITEM ≤ INFO(B). I Thus SAVE and PTR are updated as SAVE := PTR and
I Find the location LOC of the last node in LIST whose value PTR := LINK[PTR]
is less than ITEM. I The traversing continues as long as INFO[PTR] > ITEM,
I Traverse the list, using a pointer variable PTR and compar- or in other words, the traversing stops as soon as ITEM ≤
ing ITEM with INFO[PTR] at each node. INFO[PTR]
I While traversing, keep track of the location of the preceding I Then PTR points to node B, so SAVE will contain the loca-
node by using a pointer variable SAVE, as pictured in Fig. tion of the node A.
Insertion into a Linked List Deletion from a Linked List
Insertion into a Linked List VIII
1 Linked Lists
I Algorithm :
1. [Use above Procedure to find the location of the node preced- Searching a Linked List
ing ITEM.] Memory Allocation
SRCHSL(INFO, LINK, START, ITEM, LOC) Insertion into a Linked List
2. [Use previous Algorithm to insert ITEM after the node with Deletion from a Linked List
location LOC.]
Header Linked Lists
Call INSLOC(INFO, LINK, START, AVAIL, LOC, ITEM).
Two-way Lists
3. Exit.
Polynomial
Sparse Matrix

Deletion from a Linked List Deletion from a Linked List
Deletion From a Linked List I Deletion From a Linked List II
Suppose node N is to be deleted from the linked list. The schematic

diagram of such a deletion appears in Fig.
The deletion occurs as soon as the nextpointer field of node A is

changed so that it points to node B.
Deletion From a Linked List III Deletion From a Linked List IV
All of our deletion algorithms will return the memory space of the
deleted node N to the beginning of the AVAIL list.
Observe that three pointer fields are changed as follows:
1. The nextpointer field of node A now points to node B, where
node N previously pointed.
2. The nextpointer field of N now points to the original first node
in the free pool, where AVAIL previously pointed.
3. AVAIL now points to the deleted node N.

Deletion From a Linked List V Deletion From a Linked List VI
There are also two special cases. If the deleted node N is the first All of our deletion algorithms will return the memory space of the
node in the list, then START will point to node B and deleted node N to the beginning of the AVAIL list.
If the deleted node N is the last node in the list, then node A will Accordingly, all of our algorithms will include the following pair of
contain the NULL pointer. assignments, where LOC is the location of the deleted node N:
LINK[LOC] := AVAIL and then AVAIL := LOC
Deletion From a Linked List VII Deleting the Node Following a Given Node I
Suppose we are given the location LOC of a node N in LIST.
Furthermore, suppose we are given the location LOCP of the

node preceding N.
Figure: LINK[LOC] := AVAIL and AVAIL := LOC

Deleting the Node Following a Given Node II Deleting the Node Following a Given Node III
Schematic diagram of the assignment START:=LINK[START] which

effectively deletes the first node from the list.
This covers the case when N is the first node.

Deleting the Node Following a Given Node IV Deleting the Node Following a Given Node V
Algorithm :
Below is the diagram of the assignment LINK[LOCP] := LINK[LOC]
DEL(INFO, LINK, START, AVAIL, LOC, LOCP) This algorithm deletes
which effectively deletes the node N when N is not the first node.
the node N with location LOC. LOCP is the location of the node
which precedes N or, when N is the first node, LOCP = NULL.
1. If LOCP = NULL, then:

Set START := LINK[START]. [Deletes first node.]
Else:
Set LINK[LOCP] := LINK[LOC]. [Deletes node N.]
2. [Return deleted node to the AVAIL list.]
Set LINK[LOC] := AVAIL and AVAIL := LOC.
3. Exit.

Deleting the Node with a Given ITEM of Information I Deleting the Node with a Given ITEM of Information II
Algorithm : DELETE(INFO, LINK, START, AVAIL, ITEM)
This algorithm deletes from a linked list the first node N which contains the given ITEM
of information.
Traverse the list, using a pointer variable PTR and comparing
ITEM with INFO[PTR] at each node. 1. Use above Procedure to find the location of N and its preceding node.
2. If LOC = NULL, then: Write: ITEM not in list, and Exit.
While traversing, keep track of the location of the preceding node
by using a pointer variable SAVE. 3. [Delete node.]
If LOCP = NULL, then:
SAVE and PTR are updated by the assignments SAVE = PTR
Set START := LINK[START]. [Deletes first node.]
and PTR = LINK[PTR].
Else:
The traversing continues as long as INFO[PTR] != ITEM. Set LINK[LOCP] := LINK[LOC].
Then PTR contains the location LOC of node N and SAVE con-
tains the location LOCP of the node preceding N. 4. [Return deleted node to the AVAIL list.]
Set LINK[LOC] := AVAIL and AVAIL := LOC.
5. Exit.
Header Linked Lists Header Linked Lists
HEADER LINKED LISTS I
A header linked list is a linked list which always contains a special

1 Linked Lists
node, called the header node, at the beginning of the list.
Traversing a Linked List Two kinds of widely used header lists:
1. Grounded header list - last node contains the null pointer.
Memory Allocation
2. Circular header list - last node points back to the header
node.
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix

Header Linked Lists Header Linked Lists
HEADER LINKED LISTS II Algorithm : Traversing a Circular Header List

1. Set PTR := LINK[START]. [Initializes the pointer PTR.]
2. Repeat Steps 3 and 4 while PTR 6= START:
4. Set PTR := LINK[PTR]. [PTR now points to the next node.]
The first node in a header list is the node following the header
node, and the location of the first node is LINK[START], not START, [End of Step 2 loop.]
as with ordinary linked lists. 5. Exit.
Above algorithm uses a pointer variable PTR to traverse a circular
header list, is essentially the same as traversing an ordinary linked
list, except that now the algorithm
1. begins with PTR = LINK[START] (not PTR = START)
2. ends when PTR = START (not PTR = NULL).
Header Linked Lists Two-way Lists
There are two other variations of linked lists which sometimes

appear in the literature:
1. A linked list whose last node points back to the first node
instead of containing the null pointer, called a circular list. 1 Linked Lists
2. A linked list which contains both a special header node at Representation of Linked Lists in Memory
the beginning of the list and a special trailer node at the end Traversing a Linked List
of the list. Searching a Linked List
Memory Allocation
Header Linked Lists
Two-way Lists
Polynomial
Sparse Matrix

Two-way Lists Two-way Lists
TWO-WAY LISTS I TWO-WAY LISTS II
A two-way list is a linear collection of data elements, called nodes,

where each node N is divided into three parts:
I An information field INFO which contains the data of N.
I A pointer field FORW which contains the location of the next
node in the list.
I A pointer field BACK which contains the location of the pre-
ceding node in the list.
The list also requires two list pointer variables: FIRST, which
points to the first node in the list, and LAST, which points to the
last node in the list.
Two-Way Header Lists I Operations on Two-Way Lists I
The advantages of a two-way list and a circular header list may be

combined into a two-way circular header list as pictured in Fig.
Traversing- Here it is of no advantage that the data are organized
as a two-way list rather than as a one-way list.
Searching- Here the main advantage is that we can search for
ITEM in the backward direction if we have reason to suspect that
ITEM appears near the end of the list.

Deleting I Deleting II
Deleting- Suppose we want to delete N from the list. Note that

BACK[LOC] and FORW[LOC] are the locations, respectively, of
the nodes which precede and follow node N.
N is deleted from the list by changing the following pair of pointers
in Fig.
FORW [BACK [LOC]] := FORW [LOC] and BACK [FORW [LOC]] := BACK [LOC]
Deleting III Inserting I
Deleting in Two Way List

Algorithm: DELTWL(INFO, FORW, BACK, START, AVAIL, LOC)
Suppose we are given the locations LOCA and LOCB of adjacent
1. [Delete node.] nodes A and B in LIST, and suppose we want to insert a given
Set FORW[BACK[LOC]] := FORW[LOC] and ITEM of information between nodes A and B.
BACK[FORW[LOC]] := BACK[LOC] Node N with contents ITEM is inserted into the list by changing
the following four pointers:
2. [Return node to AVAIL list.]
FORW[LOCA] := NEW, FORW[NEW] := LOCB BACK[LOCB] :=
Set FORW[LOC] := AVAIL and AVAIL := LOC. NEW, BACK[NEW] := LOCA
3. Exit.

Inserting II Inserting III
Algorithm : INSTWL(INFO, FORW, BACK, START, AVAIL, LOCA,

LOCB, ITEM)
1. [OVERFLOW?] If AVAIL = NULL, then: Write: OVERFLOW, and

Exit.
2. [Remove node from AVAIL list and copy new data into node.]
Set NEW := AVAIL, AVAIL := FORW[AVAIL], INFO[NEW] := ITEM.
3. [Insert node into list.]
Set FORW[LOCA] := NEW, FORW[NEW] := LOCB,
BACK[LOCB] := NEW, BACK[NEW] := LOCA.
4. Exit.
Polynomial Polynomial
Represent a Polynomial with a Linked List I
1 Linked Lists In mathematics, a polynomial is an expression that contains

Representation of Linked Lists in Memory a sum of powers in one or more variables multiplied by coef-
Traversing a Linked List ficients.
A polynomial in one variable, x, with constant coefficients is like:
Memory Allocation
a0 + a1 x + a2 x 2 + ... + an−1 x n−1 + an x n .
Deletion from a Linked List We call each item, e.g., an x n , a term.
Header Linked Lists
If two terms have the same power, we call them like terms.
Two-way Lists
Polynomial We can use a linked list to represent a polynomial.
Sparse Matrix
In the linked list, each node has two data fields: coefficient
and power.
Therefore, each node represents a term of a polynomial.

Represent a Polynomial with a Linked List II Represent a Polynomial with a Linked List III
For example, we can represent the polynomial 2 − 4x + 5x 2 with

a linked list:
We can sort a linked list in O(nlogn) time, where n is the total

number of the linked list nodes.
Here we will assume that the linked list is ordered by the power
for the simplicity of algorithms.
Represent a Polynomial with a Linked List IV Represent a Polynomial with a Linked List V
Header linked lists are frequently used for maintaining polynomi-

als in memory.
The header node plays an important part in this representation,
since it is needed to represent the zero polynomial.
This representation of polynomials will be presented in the con-
text of a specific example
Let p(x) denote the following polynomial in one variable (contain-
ing four nonzero terms): p(x) = 2x 8 − 5x 7 − 3x 2 + 4

Add Two Polynomials I Add Two Polynomials II
To add two polynomials, we can add the coefficients of like

terms and generate a new linked list for the resulting poly- When we add them together, we can group the like terms and
nomial. generate the result 3 − 2x 2 + 5x 2 − 3x 3 :
For example, we can use two liked lists to represent polynomials
2 − 4x + 5x 2 and 1 + 2x − 3x 3 :
Since both linked list are ordered by the power, we can use a
two-pointer method to merge the two sorted linked list:
Add Two Polynomials III Add Two Polynomials IV

In this algorithm, we first create two pointers, p1 and p2, to the
head pointers of the two input polynomials.
Then, we generate the new polynomial nodes based on the pow-
ers of these two pointers. There are three cases:
I p1’s power is greater than p2’s power: In this case, we
append a new node with p1’s power and coefficient. Also,
we move p1 to the next node.
I p2’s power is greater than p1’s power: In this case, we
append a new node with p2’s power and coefficient. Also,
we move p2 to the next node.
I p1 and p2 have the same power: In this case, the new
coefficient is the total of p1’s coefficient and p2’s coefficient.
If the new coefficient is not 0, we append a new node with
the same power and the new coefficient. Also, we move both
p1 and p2 to the next nodes.

Add Two Polynomials V Add Two Polynomials VI

After that, we continue to append the remaining nodes from p1 or
p2 until we finish the calculation on all nodes.
The append function can create a new linked list node based on
the input power and coefficient.
Also, it appends the new node to the tail node and returns the This algorithm traverses each linked list node only once.
new tail node:
Therefore, the overall running time is O(n + m), where n and m
are the number of terms for the two input polynomials.
Multiply Two Polynomials I Multiply Two Polynomials II
For example, after we multiply 2 − 4x + 5x 2 and 1 + 2x − 3x 3 , we

can get a linked list:
To multiply two polynomials, we can first multiply each term of

one polynomial to the other polynomial.
Suppose the two polynomials have n and m terms. This process

will create a polynomial with n × m terms.

Multiply Two Polynomials III Multiply Two Polynomials IV
To generate the final linked list, we can first merge sort the
linked list based on each nodes power:
This linked list contains all terms we need to generate the final
result.
However, it is not sorted by the powers.
Also, it contains duplicate nodes with like terms.

Multiply Two Polynomials V Multiply Two Polynomials VI
After the sorting, the like term nodes are grouped together.
Then, we can merge each group of like terms and get the
final multiplication result:
We can describe these steps with an algorithm:

Multiply Two Polynomials VII Multiply Two Polynomials VIII
In this algorithm, we first use a nested while loop to multiply all

term pairs from the two polynomials.
This takes O(nm) time, where n and m are the number of terms
for the two input polynomials.
Also, this process creates a linked list with n × m nodes.
Then, we merge sort the linked list based on each nodes power.
This process takes O(nmlog(nm)) time.
After we sort the linked list, we can use a two-pointer approach
to merge the like terms:
Polynomial Sparse Matrix
Multiply Two Polynomials IX
In this approach, we use one pointer as the start of a like term

1 Linked Lists
group and use another pointer to traverse the like terms in the
same group.
Each time we find a like term, we add its coefficient to the start Searching a Linked List
like term node. Memory Allocation
Once we finish one like term group, we move the start pointer to
the next group and repeat the same process to merge like terms.
Header Linked Lists
The running time of this process is O(nm) as we need to visit Two-way Lists
n × m nodes. Polynomial
Sparse Matrix
Overall, the running time is O(nm) + O(nmlog(nm)) + O(nm) =
O(nmlog(nm)).

Sparse Matrix Sparse Matrix
Sparse Matrix I Sparse Matrix II
In above example matrix, there are only 6 non-zero elements

(those are 9, 8, 4, 2, 5 & 2) and matrix size is 5 X 6.
Here the first row in the right side table is filled with values 5, 6 &
6 which indicates that it is a sparse matrix with 5 rows, 6 columns
& 6 non-zero values.
The second row is filled with 0, 4, & 9 which indicates the non-
zero value 9 is at the 0th-row 4th column in the Sparse matrix.
In the same way, the remaining non-zero values also follow a
similar pattern.
Sparse Matrix III Sparse Matrix IV
In linked representation, we use a linked list data structure to rep-

resent a sparse matrix.
In this linked list, we use two different nodes namely header node
and element node.
Header node consists of three fields and element node consists
of five fields as shown in the image.

Sparse Matrix V Sparse Matrix VI
Consider the above same sparse matrix used in the Triplet repre-
sentation.
This sparse matrix can be represented using linked representa-

tion as shown in the below image.
Sparse Matrix VII Sparse Matrix VIII
In the above representation, H0, H1,..., H5 indicates the header

nodes which are used to represent indexes.
Remaining nodes are used to represent non-zero elements in the
matrix, except the very first node which is used to represent ab-
stract information of the sparse matrix (i.e., It is a matrix of 5 X 6
with 6 non-zero elements).
In this representation, in each row and column, the last node right
field points to its respective header node.
Linked Lists
Linked Lists
Sparse Matrix
Sparse Matrix IX
Linked Lists
Dr. Arup Kumar Pal
Jharkhand-826004
Data Structures and Algorithms Dept. of CSE, IIT(ISM) Dhanbad September 8, 2021 1
Outline
Introduction
Single Linked List
Representation
Operations
Double Linked List
Circular Linked List
Applications of Linked List
Sparse Matrix Manipulation
Polynomial Representation
Generalized Lists (Not in your syllabus)
Introduction
An array stores the data in contiguous memory
locations.
In order to occupy the adjacent space, a block
of memory that is required for the array should
be allocated before hand.
Once the memory is allocated, it cannot be
extended any more.
Deleting an element or inserting an element
may require shifting of elements.
Contd…
The linked list is an alternative to the array when a
collection of objects is to be stored.
The linked list is implemented using pointers.
Thus, an element (or node) of a linked list
contains the actual data to be stored and a
pointer to the next node.
The pointer is simply the address in memory of
the next node.
Thus, a key difference from arrays is that a linked
list does not have to be stored contiguously in
memory.
Contd…
The pointer variable next contains either the

address of the location in memory of the
successor list element or the special value NULL
defined as 0.
NULL is used to denote the end of the list (no successor
element)
Singly Linked List
A Linear Linked List is also called singly linked list or one way
linked list.
In the singly linked list each node is divided into two parts.
Information field – contains the user information.
Next pointer field – contains the address of next node.
Another pointer variable, for example head, is used. That
contains the address of the first node in the list.
The next pointer field of the last node contains NULL value to
indicate the end of the list.
Representation
Example: list of student records
Contd…
Contd…
Contd…
Alternative Way
Instead of statically declaring the structures n1,
n2, n3,
Dynamically allocate space for the nodes
Use malloc() individually for every node allocated
This is the usual way to work with linked lists,
as number of elements in the list is usually not
known in advance (if known, we could have
used arrays)
Contd…
struct node
{
int data;
struct node *next;
};
typedef struct node NODE;
//type definition making it abstract data type
NODE *start;
start=(NODE *)malloc(sizeof(NODE));
//dynamic memory allocation
Contd…
• Now, we can assign values to the respective
fields of NODE
start->data= 10;
start->next= ‘\0’;
• Any number of nodes can be created and linked

to the existing node.
start->next=(NODE *)malloc(sizeof(NODE));
start->next.data= 20;
start->next.next= ‘\0’;
Contd…
• Any number of nodes can be created and linked
to the existing node.
start->next=(NODE *)malloc(sizeof(NODE));
start->next.data= 10;
start->next.next= ‘\0’;
To create and to display
void main()
{ struct node
{ int num;
struct node *ptr;
};
typedef struct node NODE;
NODE *start, *first, *temp = 0;
int count = 0;
int choice = 1;
first = 0;
while (choice)
{ start = (NODE *)malloc(sizeof(NODE));
printf("Enter the data item\n");
scanf("%d", &start-> num);
if (first != 0)
{ temp->ptr = start; temp = start; }
else
{ first = temp = start; }
printf("Do you want to continue(Type 0 or 1)?\n");
scanf("%d", &choice);
}
Contd…
temp->ptr = 0;
/* reset temp to the beginning */
temp = first;
printf("Display of the linked list is\n");
while (temp != 0)
{
printf("%d=>", temp->num);
count++;
temp = temp -> ptr;
}
printf("NULL\n");
printf("No. of nodes in the list = %d\n", count);
}
LIST is Unsorted
Algorithm : SEARCH(INFO, LINK, START, ITEM, LOC)
LIST is a linked list in memory. This algorithm finds the location
LOC of the node where ITEM first appears in LIST, or sets LOC
=NULL.
Begin:
1. Set PTR := START.
2. Repeat Step 3 while PTR ≠ NULL:
3. If ITEM = INFO[PTR], then: Set LOC := PTR, and Exit.
Else: Set PTR := LINK[PTR]. [PTR now points to the next node.]
4. [Search is unsuccessful.] Set LOC := NULL.
5. Exit.
End
Contd…
The complexity of this algorithm is the same as
that of the linear search algorithm with context
of Arrays.
The worst-case running time is proportional to
the number n of elements in LIST.
LIST is Sorted
Algorithm : SEARCH(INFO, LINK, START, ITEM, LOC)
LIST is a sorted list in memory. This algorithm finds the location LOC of
the node where ITEM first appears in LIST, or sets LOC= NULL.
Begin:
1. Set PTR := START.
2. Repeat Step 3 while PTR ≠ NULL:
3. If ITEM < INFO[PTR], then: Set PTR := LINK[PTR]. [PTR now points to next
node.]
Else if ITEM = INFO[PTR], then: Set LOC := PTR, and Exit. [Search is
successful.]
Else:
Set LOC := NULL, and Exit. [ITEM now exceeds INFO[PTR].]
4. Set LOC := NULL.
5. Exit.
End
Contd…
The complexity of this algorithm is still
proportional to the number n of elements in
LIST.
With a sorted linear array we can apply a
binary search algorithm.
A binary search algorithm cannot be applied to
a sorted linked list, since there is no way of
indexing the middle element in the list.
This property is one of the main drawbacks in
using a linked list as a data structure.
Insertion at the Beginning of a List
Contd…
Inserting at the End of the List
Contd…
Inserting after a Given Node
Inserting after a Given Node
Deletion a Node at front in List
Deletion a Node at front in List
Deletion of a node at the end in List
Deletion of a node at the end in List
Deletion of a node at any position
Deletion of a node at any position
A polynomial can be represented as:
f  x   am x  am 1x
m m 1
  a1x  a0 x
1 0
Now:
A  3x  5 x 
12 3
2
Polynomial Addition
Contd…
Algorithm
Contd…
Polynomial Multiplication
Circular Linked List
Circular header lists are frequently used instead of
ordinary linked lists because many operations are much
easier to state and implement using header lists.
In circular lists null pointer is not used, and hence all
pointers contain valid addresses.
Every (ordinary) node has a predecessor, so the first
node may not require a special case.
Deleting First Node from Circular Linked List
Delete last node from Circular Linked List
Double Linked List
Each list discussed before is type of a one-way list,
where we can traverse the list in only one
direction.
A double linked list is a linear collection of data
elements, called nodes, where each node N is
divided into three parts:
An information field INFO which contains the data of N
A pointer field FORW which contains the location of the
next node in the list
A pointer field BACK which contains the location of the
preceding node in the list
Contd…
Sparse Matrix
Contd…
Generalized Lists
Suppose we want to represent the given
polynomial using linear list
Here, we will use the following node
The node structure will be change depending

upon the number of coefficients.
Contd…
Some Examples of Generalized List
Suppose P=3x2y
Contd…
Thank You…
Any Queries...?
Stacks Stacks
Outline I
1 Stacks
Stack Operations
Stacks
2 Array Representation of Stacks
3 Linked Representation of Stacks
4 Applications of Stacks
5 Arithmetic Expressions; Polish Notation
Stacks Stacks
Stacks Stacks
Stacks I
1 Stacks
Stack Operations
A stack is a list of elements in which an element can be inserted

or deleted only at one end.
The end is referred to as the “top of stack”.
So elements are removed from the stack in the reverse order of
that in which they were inserted into the stack.
4 Applications of Stacks This way a stack is a LIFO (Last in First Out) or FILO (First in Last
Out)data structure.

Stacks Stacks
Stacks Stacks
Stack Operations Stack Operations
Stack Operations I
1 Stacks
Stack Operations

Special terminology is used to refer to the two basic operations
associated with stack:
I PUSH: is the term used to insert an element into the stack.
I POP: is the term used to delete an element from the stack.
Stacks Stacks
Stacks Array Representation of Stacks
Stack Operations
Stack Operations: Example I

1 Stacks
Stack Operations
Say following six elements are pushed in order onto an empty
stack: AAA, BBB, CCC, DDD, EEE, FFF
Following figures depict the above operations: 2 Array Representation of Stacks

Stacks Stacks
Array Representation of Stacks Array Representation of Stacks
Array Representation of Stacks I Array Representation of Stacks II
The condition TOP == 0 or TOP == NULL will indicate that the

Can be maintained using arrays or linked list. stack is empty.
We shall discuss discuss array representation representation of

stack.
Array representation requires following:
I A linear array named as STACK.
I A pointer variable TOP which contains the location of the top
element of the stack.
I A variable MAXSTK which gives the maximum number of
elements that can be held by the stack.
Stacks Stacks
Array Representation of Stacks Array Representation of Stacks
Operations on Stack Algorithm for PUSH
Following algorithm pushes (inserts)ITEM into STACK.

PUSH(STACK, TOP, MAXSTK, ITEM)
Operations: PUSH and POP.
This procedure pushes an ITEM onto a stack.
We have already already looked into them. So, its time to discuss
1. [Stack already filled?]
them formally. If TOP = MAXSTK, then: Print: OVERFLOW, and Return.
2. Set TOP := TOP + 1. [Increases TOP by 1.]
3. Set STACK[TOP] := ITEM. [Inserts ITEM in new TOP position.]
4. Return.
Stacks Stacks
Array Representation of Stacks Linked Representation of Stacks
Algorithm for POP

1 Stacks
Stack Operations
Following algorithm pops (deletes) ITEM from the STACK.

POP(STACK, TOP, ITEM)
This procedure deletes the top element of STACK and assigns it
to the variable ITEM. 3 Linked Representation of Stacks
1. [Stack has an item to be removed?]
If TOP = 0, then: Print: UNDERFLOW, and Return.
2. Set ITEM := STACK[TOP]. [Assigns TOP element to ITEM.]
3. Set TOP := TOP - 1. [Decreases TOP by 1.] 4 Applications of Stacks
4. Return.
Stacks Stacks
Linked Representation of Stacks Linked Representation of Stacks
Linked Representation of Stacks I Linked Representation of Stacks II
Can be maintained using one-way list or singly linked list. The condition TOP == NULL will indicate that the stack is empty.
Linked representation requires following:

I The INFO fields of the nodes hold the elements of the stack.
I The LINK fields hold pointers to the neighboring element in
the stack.
I The START pointer of the linked list behaves as the TOP
pointer variable of the stack.
I The null pointer of the last node in the list signals the bottom
of stack.
Stacks Stacks
Linked Representation of Stacks Linked Representation of Stacks
Algorithm for PUSH Algorithm for POP
Following algorithm pops (deletes) ITEM from the STACK.

Following algorithm pushes (inserts)ITEM into STACK.
POP LINKSTACK(INFO, LINK, TOP, AVAIL, ITEM): This proce-
PUSH LINKSTACK(INFO, LINK, TOP, AVAIL, ITEM): This procedure deletes the top element of a linked stack and assigns it to
dure pushes an ITEM into a linked stack the variable ITEM
1. [Available space?] If AVAIL = NULL, then Write OVERFLOW and Exit 1. [Stack has an item to be removed?]
2. [Remove first node from AVAIL list] IF TOP = NULL then Write: UNDERFLOW and Exit.
Set NEW := AVAIL and AVAIL := LINK[AVAIL]. 2. Set ITEM := INFO[TOP] [Copies the top element of stack into ITEM]
3. Set INFO[NEW] := ITEM [Copies ITEM into new node] 3. Set TEMP := TOP and TOP = LINK[TOP]
4. Set LINK[NEW] := TOP [New node points to the original top node in the stack] [Remember the old value of the TOP pointer in TEMP and reset TOP to point to
the next element in the stack ]
5. Set TOP = NEW [Reset TOP to point to the new node at the top of the stack]
4. [Return deleted node to the AVAIL list]
6. Exit. Set LINK[TEMP] = AVAIL and AVAIL = TEMP.
5. Exit.
Stacks Stacks
Applications of Stacks Applications of Stacks
Applications of Stacks
1 Stacks
Stack Operations

Stacks are widely used computer science.
Their specific applications are:
3 Linked Representation of Stacks I Management of Function Calls
I Evaluation of Expressions
I Implementation of certain algorithms (e.g., Quick Sort)

Stacks Stacks
Arithmetic Expressions; Polish Notation Arithmetic Expressions; Polish Notation
Evaluation of Expressions I
1 Stacks
Stack Operations
Operators Precedence:
2 Array Representation of Stacks I In arithmetic expressions operators precedence is observed:
• Highest: Exponentiation (↑)

3 Linked Representation of Stacks • Next highest: Multiplication (∗) and division (/)
• Lowest: Addition (+) and subtraction (−)
An Example:
I Evaluate: 2 ↑ 3 + 5 ∗ 2 ↑ 2 − 12/6
I Answer: 26
Stacks Stacks
Evaluation of Expressions II Notations for Expressions I
An Important Fact:
I Parentheses’ alter the precedence of operators. Infix Notations:
An Example: I Expressions in which operator lies between the operands
are referred to as infix notations.
I (A + B) ∗ C 6= A + (B ∗ C)
I A+B, C-D, P*F, · · · all are infix notations.
I (2 + 3) ∗ 7 = 35 while 2 + (3 ∗ 7) = 23
I A+(B*C) and (A+B)*C are distinguished by parentheses or
How computer evaluates the arithmetic expressions? – is the by applying the operators precedence discussed above.
question we want to seek answer for.
Stacks Stacks
Notations for Expressions II Notations for Expressions III
Prefix or Polish Notations:

I Named in honour of Polish mathematician, Jan Lukasiewiez,
refer to the expressions in which the operator symbol is placed Postfix or Reverse Polish Notations
before its two operands.
I +AB, -CD, *PF, · · · all are examples of prefix or polish ex- I Refer to the expressions in which operator is placed after its
pressions. two operands.
I Simple infix expressions can be converted to polish expres- I AB+, CD-, PF*. . . all are examples of postfix or reverse pol-
sions as follows: ish notations.
• (A + B) ∗ C = [+AB] ∗ C = ∗ + ABC I Like prefix notations, they are also parentheses’ free.
• A + (B ∗ C) = A + [∗BC] = +A ∗ BC
• (A + B)/(C − D) = [+AB]/[−CD]/ + AB − CD
I An important property of these notations is that they are
parentheses free.
Stacks Stacks
How Computer Evaluates Expressions? I How Computer Evaluates Expressions? II
Expressions are represented in infix notations and use of paren-

theses is very common. Following figure depicts the process:
Computer may apply the operators precedence and parentheses’
rules and evaluate the expression.
But, this process is not feasible in terms of computer timing (tim-
ing complexity)as computer takes a lot of time to resolve paren-
theses’.
So, the computer first converts an infix expression into an equiv-
alent postfix expression and then evaluates it.
Stacks Stacks
How Computer Evaluates Expressions? III Algorithm 1 I
Clearly following two procedures (algorithms) are required:

I Algorithm 1: Converting an infix expression to an equivalent
postfix expression. Conversion of an infix expression to an equivalent postfix expres-
I Algorithm 2: Evaluating the postfix expression. sion.
For each algorithm, Stack is the main tool to be utilized.
Stacks Stacks
Algorithm 1 II Algorithm 1-Example

arithmetic infix expression Q : A + (B ∗ C − (D/E ↑ F ) ∗ G) ∗ H
POLISH(Q, P): Suppose Q is an arithmetic expression written in infix notation.
This algorithm finds the equivalent postfix expression P.
1. Push ”(” onto STACK, and add ”)” to the end of Q.
2. Scan Q from left to right and repeat Steps 3 to 6 for each element of Q until the STACK is empty:
3. If an operand is encountered, add it to P.
4. If a left parenthesis is encountered, push it onto STACK.
5. If an operator ⊗ is encountered, then:
(a) Repeatedly pop from STACK and add to P each operator (on the top of STACK) which has
the same precedence as or higher precedence than ⊗.
(b) Add ⊗ to STACK.
6. If a right parenthesis is encountered, then:
(a) Repeatedly pop from STACK and add to P each operator (on the top of STACK) until a left
parenthesis is encountered.
(b) Remove the left parenthesis. [Do not add the left parenthesis to P.]
7. Exit.
Stacks Stacks
Algorithm 2 Algorithm 2-Example
Evaluating the postfix expression.

Arithmetic expression P written in postfix notation: P: 5, 6, 2, +, *,
This algorithm finds the VALUE of an arithmetic expression P written in postfix
12, 4, /, –
notation.
1. Add a right parenthesis ”)” at the end of P. [This acts as a sentinel.]
2. Scan P from left to right and repeat Steps 3 and 4 for each element of P until the sentinel ”)” is encountered.
3. If an operand is encountered, put it on STACK.
4. If an operator ⊗ is encountered, then:

(a) Remove the two top elements of STACK, where A is the top element and B is the next-to-top element.
(b) Evaluate B ⊗ A.
(c) Place the result of (b) back on STACK.
5. Set VALUE equal to the top element on STACK.
6. Exit.
Stacks Stacks
Infix to Prefix I Infix to Prefix II

Algorithm of Infix to Prefix
1. Push “)” onto STACK, and add “(“ to end of the A
2. Scan A from right to left and repeat step 3 to 6 for each element of A until the STACK is empty
3. If an operand is encountered add it to B
4. If a right parenthesis is encountered push it onto STACK
5. If an operator is encountered then:
a. Repeatedly pop from STACK and add to B each operator (on the top of STACK) which has same or
higher precedence than the operator.
b. Add operator to STACK
6. If left parenthesis is encontered then
a. Repeatedly pop from the STACK and add to B (each operator on top of stack until a left parenthesis
is encounterd)
b. Remove the left parenthesis
7. Exit
Stacks Stacks
Example Infix to Prefix I Example Infix to Prefix II

3. Convert expression to postfix form:
Expression: ( A + B ∧ C ) * D + E ∧ 5
1. Reverse the infix expression: 5 ∧ E + D * ) C ∧ B + A (
2. Make every ’(’ as ’)’ and every ’)’ as ’(’: 5 ∧ E + D * ( C ∧ B + A )
Stacks Stacks
Example Infix to Prefix III Evaluation of Prefix Expressions I
Algorithm for evaluating a prefix expression

1. Put a pointer P at the end of the end.
2. If character at P is an operand push it to Stack.

4. Reverse the expression: + * + A ∧ B C D ∧ E 5
3. If the character at P is an operator pop two elements from the Stack. Operate on these elements according
Result: + * + A ∧ B C D ∧ E 5 to the operator, and push the result back to the Stack.
4. Decrement P by 1 and go to Step 2 as long as there are characters left to be scanned in the expression.
5. The Result is stored at the top of the Stack, return it.
6. End.
Stacks
Arithmetic Expressions; Polish Notation
Example- Evaluation of Prefix Expressions I
Expression: + 9 * 2 6
Character Scanned Stack (Front to Back) Explanation
6 6 6 is an operand, push to Stack
2 62 2 is an operand, push to Stack
* 12 (6 * 2) * is an operator, pop 6 and 2,
multiply them and push result to
Stack
9 12 9 9 is an operand, push to Stack
+ 21 (12+9) + is an operator, pop 12 and
9 add them and push result to
Stack
Result: 21
Queues Queues
Outline I
1 Queues
Queues
2 Array Representation of Queues
3 Linked Representation of Queues
4 Deques
5 Priority Queues
Queues Queues
Queues Queues
Queues
1 Queues

1. A queue is a linear list of elements in which deletions can take
place only at one end, called the front, and insertions can take
place only at the other end, called the rear
3 Linked Representation of Queues
2. In other words, the order in which elements enter a queue is the
order in which they leave. Queues are also called first-in first-out
(FIFO) lists. This contrasts with stacks, which are last-in first-out
4 Deques (LIFO) lists.
5 Priority Queues
Queues Queues
Queues Queues
Representation of Queues I Representation of Queues II
This is a schematic diagram of a queue with 4 elements; where

AAA is the front element and DDD is the rear element. Suppose Queues will be maintained in the memory by means of a linear
an element is deleted from the queue,then it must be AAA. Next, array QUEUE and using two pointer variables
suppose EEE is added to the queue and then FFF is added to I FRONT: containing the location of front element of the queue
the queue. Then they must be added at the rear of the queue. and
I REAR: containing the location of the rear element of the
queue
I The condition FRONT = NULL indicates indicates that the
Queue is empty.
I Whenever an element is deleted from the queue, FRONT
increases by 1, i.e., FRONT = FRONT + 1.
I Whenever an element is added to the queue REAR increases
by 1, i.e., REAR = REAR + 1.
Queues Queues
Queues Array Representation of Queues
Representation of Queues III

1 Queues
I Suppose we want to add an element (ITEM)to the queue 2 Array Representation of Queues
when its last part is occupied (i.e., REAR = N) and space is
available in the first part of the queue.
I One way to do this is to simply move the entire queue to the 3 Linked Representation of Queues
beginning of the array, changing FRONT and REAR accord-
ingly, and then inserting ITEM as above.
I This procedure may be very expensive.
4 Deques
5 Priority Queues
Queues Queues
Array Representation of Queues Array Representation of Queues
Array representation of queue I Array representation of queue II
To add more elements we may:

I Either, shift entire queue towards left and changing FRONT
and REAR accordingly
I Or, we may assume that Queue is circular i.e. QUEUE[1]
comes after QUEUE[N].
The first solution is simple but expensive in terms of computations
Queues Queues
Array representation of queue III Array representation of queue IV

So we will always assume the queue as circular. The fig shows how a queue may be maintained by a circular array
QUEUE with N = 5 memory locations
Specifically, instead of increasing REAR to N+1, we reset REAR
= 1 and then assign QUEUE[REAR] = ITEM.
Similarly, if FRONT = N and an element of the QUEUE is deleted,
we reset FRONT = 1 instead of increasing FRONT to N+1.
Suppose our QUEUE has only one element then:
FRONT = REAR6=NULL
Suppose that the only element of the QUEUE is deleted then we

assign :
FRONT = NULL and REAR = NULL
to indicate that the queue is empty

Queues Queues
Array representation of queue V Insertion in Queues I
Let a queue is being maintained in the memory by means of the

linear array QUEUE and pointer variables FRONT and REAR
We now discuss procedures for insertions and deletions in queues.
Let ITEM be a piece of information that is to be added to the
QUEUE.
The procedure QINSERT shown below does the needful
Queues Queues
QINSERT Algorithm Deletion in Queues

QINSERT (QUEUE, N, FRONT, REAR, ITEM)
1. [Queue already filled ? ]
IF FRONT=1 AND REAR=N, or if FRONT = REAR + 1, then:
Write: OVERFLOW, and Return.
Let a queue is being maintained in the memory by means of the
2. [Find new value of REAR.]
linear array QUEUE and pointer variables FRONT and REAR.
If FRONT := NULL, then: [Queue initially empty.]
Set FRONT := 1 and REAR := 1 Let ITEM be a piece of information that is to be deleted from the
QUEUE.
Else if REAR = N, then:
Set REAR := 1 The procedure QDELETE shown below does the needful.
ELSE:
Set REAR := REAR + 1
3. Set QUEUE[REAR] := ITEM. [This inserts new element.]
4. Return.
Queues Queues
Array Representation of Queues Linked Representation of Queues
QDELETE Algorithm
QDELETE (QUEUE, N, FRONT, REAR, ITEM) 1 Queues
1. [Queue already empty?]

If FRONT := NULL, then: Write: UNDERFLOW, and Return.
2. Set ITEM := QUEUE[FRONT].
3. [Find new value of FRONT.]
If FRONT = REAR, then: [Queue has only one element to start.] 3 Linked Representation of Queues
Set FRONT := NULL and REAR := NULL.
Else if FRONT = N, then:
4 Deques
Set FRONT := 1.
Else:
Set FRONT := FRONT + 1
5 Priority Queues
4. Return.
Queues Queues
Linked Representation of Queues Linked Representation of Queues
Linked Representation of Queues I Linked Representation of Queues II
A linked queue is a queue implemented as a linked list with two

pointer variables FRONT and REAR pointing to the nodes which In the case of insertion into a linked queue, a node borrowed from
is in the FRONT and REAR of the queue the AVAIL list and carrying the item to be inserted is added as the
The INFO fields of the list hold the elements of the queue and last node of the linked list representing the queue.
the LINK fields hold pointers to the neighboring elements in the The REAR pointer is updated to point to the last node just added
queue. to the list
In the case of deletion, the first node of the list pointed to by
FRONT is deleted and the FRONT pointer is updated to point to
the next node in the list
Queues Queues
Linked Representation of Queues III Insertion in linked queue I
LINKQ INSERT(INFO, LINK, FRONT, REAR, AVAIL, ITEM)

This procedure inserts an ITEM into a linked queue
1. [Available space?] If AVAIL = NULL, then Write OVERFLOW and Exit
2. [Remove first node from AVAIL list]

Set NEW := AVAIL and AVAIL := LINK[AVAIL]
3. Set INFO[NEW] := ITEM and LINK[NEW]= NULL [Copies ITEM into new node]
4. If (FRONT = NULL) then FRONT = REAR = NEW

[If Q is empty then ITEM is the first element in the queue Q]
else set LINK[REAR] := NEW and REAR = NEW
[REAR points to the new node appended to the end of the list]
5. Exit.
Queues Queues
Deletion in linked queue I Deletion in linked queue II
LINKQ DELETE (INFO, LINK, FRONT, REAR, AVAIL, ITEM)

This procedure deletes the front element of the linked queue and stores
it in ITEM
1. [Linked queue empty?] If (FRONT = NULL) then Write: UNDERFLOW and Exit
2. Set TEMP = FRONT [If linked queue is nonempty, remember FRONT in a tempo-
rary variable TEMP]
3. ITEM = INFO (TEMP)
4. FRONT = LINK (TEMP) [Reset FRONT to point to the next element in the queue]
5. LINK(TEMP) = AVAIL and AVAIL = TEMP [return the deleted node TEMP to the
AVAIL list]
6. Exit.
Queues Queues
Deques Deques
Deques I
1 Queues
A deque (pronounced as “deck” or “dequeue”) is a linear list in

3 Linked Representation of Queues which elements can be added or removed removed at either end,
but not in the middle.
The term deque is a contraction of the term double-ended queue.

4 Deques
5 Priority Queues
Queues Queues
Deques Deques
Deques II Deques III

They are represented by means of a linear array DEQUE and two
pointer variables LEFT and RIGHT as shown
There are two variations of a deque namely, an input-restricted
deque and an output-restricted deque.
I Input-restricted deque is a deque which allows insertions at
only one end of the list but allows deletions at both ends of
the list; and
I Output-restricted deque is a deque which allows deletions at
only one end of the list but allows insertions at both ends of
the list.
Queues Queues
Priority Queues Priority Queues
Priority Queues I
1 Queues
2 Array Representation of Queues A priority queue is a collection of elements such that each el-
ement has been assigned a priority and such that the order in
which elements are deleted and processed comes from the fol-
3 Linked Representation of Queues lowing rules
1. An element of higher priority is processed before any ele-
ment of lower priority.
4 Deques 2. Two elements with the same priority are processed accord-
ing to the order in which they were added to the queue.
5 Priority Queues
Queues Queues
One-Way List Representation of a Priority Queue I One-Way List Representation of a Priority Queue II
One way to maintain a priority queue in memory is by means of a

one-way list, as follows:
(a) Each node in the list will contain three items of information:
i. an information field INFO,

ii. a priority number PRN, and
iii. a link number LINK.
(b) A node X precedes a node Y in the list
i. when X has higher priority than Y or
ii. when both have the same priority but X was added to the list
before Y.
This means that the order in the one-way list corresponds to
the order of the priority queue.
Priority numbers will operate in the usual way, the lower the pri-
ority number, the higher the priority
Queues Queues
One-Way List Representation of a Priority Queue III One-Way List Representation of a Priority Queue IV
This algorithm deletes and processes the first element in a priority

queue which appears in memory as a one-way list.
1. Set ITEM := INFO[START]. [This saves the data in the first
node.]
2. Delete first node from the list.
3. Process ITEM.
4. Exit.
Queues Queues
One-Way List Representation of a Priority Queue V One-Way List Representation of a Priority Queue VI
This algorithm adds an ITEM with priority number N to a priority

queue which is maintained in memory as a one-way list.
(a) Traverse the one-way list until finding a node X whose prior-
ity number exceeds N. Insert ITEM in front of node X.
(b) If no such node is found, insert ITEM as the last element of
the list.
Queues Queues
Array Representation of a Priority Queue I Array Representation of a Priority Queue II
Another way to maintain a priority queue in memory is to use

a separate queue for each level of priority (or for each priority
number).
Each such queue will appear in its own circular array and must
have its own pair of pointers, FRONT and REAR.
In fact, if each queue is allocated the same amount of space, a
two-dimensional array QUEUE can be used instead of the linear
arrays.
Queues Queues
Array Representation of a Priority Queue III Array Representation of a Priority Queue IV
This algorithm deletes and processes the first element in a priority

This algorithm adds an ITEM with priority number M to a priority
queue maintained by a two-dimensional array QUEUE.
queue maintained by a two-dimensional array QUEUE.
1. [Find the first nonempty queue.] Find the smallest K such
1. Insert ITEM as the rear element in row M of QUEUE.
that FRONT[K] 6= NULL.
2. Exit.
2. Delete and process the front element in row K of QUEUE.
3. Exit.
Sorting
Outline
• Basic Terminologies
• Sorting Techniques
• Conclusions
Data Structures and Algorithms Department of CSE, ISM Dhanbad September 25, 2021 2
Introduction
Sorting is one of the fundamental operations in
computer science.
Sorting is one of the most common data-
processing applications.
A sorting algorithm is an algorithm that puts
elements of a list in a certain order, such as
increasing or decreasing, with numerical data, or
alphabetically, with character data.
Efficient sorting is important for optimizing the
use of other algorithms (such as search and
merge algorithms) which require input data to be
in sorted lists.
Sorting
Let A be a list of n elements A1, A2, …, An in
memory.
Sorting A refers to the operation of rearranging
the contents of A so that they are increasing in
order (numerically or lexicographically), that is, so
that A1 ≤ A2 ≤ A3 ≤ … ≤An
Since A has n elements, there are n! ways that the
contents can appear in A.
These ways correspond precisely to the n!
permutations of 1, 2,…, n. Accordingly, each
sorting algorithm must take care of these n!
possibilities.
Properties of Sorting
Stable Sort: A sorting algorithm is stable if the
relative order of elements with the same key
value is preserved by the algorithm.
Non-Stable
Sort
Properties of Sorting
In place Sort: Suppose array data structures is
used to store the data than the sorting method
takes place within the array only, that is,
without using any other extra storage space.
In other words, in place sorting does not
require extra memory space other than the list
itself and hence it is memory efficient method.
Quick Overview
Programmer’s view
1. How to rearrange a given set of data?
2. Which data structures are more suitable to
store data prior to their sorting?
3. How fast can the sorting be achieved?
4. How can sorting be done in memory
constraint situation?
5. How to sort various types of data?
Sorting Techniques
Sorting
Internal Sorting External Sorting

(Entire sorting can be performed (It is performed in external
in a computer’s internal storage) memory)
Sorting Techniques
Internal
Sorting
Sorting by Sorting by
Comparison Distribution
Insertion Selection Exchange Merge
Contd…
Sorting by Comparison: The basic operation involved in
this type of sorting technique is comparison.
A data item is compared with other items in the list of
items in order to find its place in the sorted list.
Comparison is followed by arrangement process like:
Insertion (The item chosen is then inserted into an
appropriate position relative to the previously sorted items)
Selection (Selected item, say smallest/largest item may be
placed in proper place)
Exchange (Here, items are interchanged)
Merge (Two or more lists are merged into an output list)
Contd…
Sorting by Distribution: In this type of sorting,
no key comparison takes place.
Distribution of items is based on the following
choices:
Radix (An item is placed in a space decided by the
bases(radixes) of its components with which it is
composed of.
Counting (Items are sorted based on their relative
counts)
Sorting by Exchange
Here we will discussed sorting algorithms
which are based on the principle of “sorting by
exchange”.
Following are the sorting based on this
principle:
Bubble Sort
Quick Sort
Shell Short (We will not discussed)
Bubble Sort
Bubble sort is the simplest sorting algorithm.
This technique is simple in the sense that it is easy
to understand, easy to implement, and easy to
analyze.
It works by iterating the input array from the first
element to the last, comparing each pair of
elements and swapping them if needed.
Bubble sort continues its iterations until no more
swaps are needed.
The algorithm gets its name from the way
smallest/largest elements “bubble” to the top of
the list.
Bubble Sort
Suppose the list of numbers A[l], A[2], …, A[N] is in memory. The bubble
sort algorithm works as follows:
Steps Remarks
Step 1. Compare A[1] and A[2] and arrange • Step 1 involves n – 1 comparisons.
them in the desired order, so that A[l] < A[2]. • The largest element is “bubbled
Then compare A[2] and A[3] and arrange them up” to the nth position
so that A[2] < A[3]. Continue until we compare • When Step 1 is completed, A[N]
A[N – 1] with A[N] and arrange them so that will contain the largest element.
A[N – 1] < A[N].
Step 2. Repeat Step 1 with one less • Step 2 involves N – 2 comparisons
comparison; that is, now we stop after we • When Step 2 is completed, the
compare and possibly rearrange A[N – 2] and second largest element will occupy
A[N –1]. A[N – 1].
Step 3. Repeat Step 1 with two fewer • Step 3 involves N – 3 comparisons
comparisons; that is, we stop after we • When Step 3 is completed, the
compare and possibly rearrange A[N – 3] and third largest element will occupy
A[N – 2]. A[N – 2].
Contd…
After n – 1 steps, the list will be sorted.
Each of steps is called a pass and the bubble
sort algorithm requires n – 1 passes, where n is
the number of input items.
Example
Suppose the following numbers are stored in an array A: 32, 51, 27, 85, 66, 23,
13, 57
Pass 1:
At the end of this first pass, the largest number, 85, has moved to the last
position.
However, the rest of the numbers are not sorted, even though some of them
have changed their positions.
Contd…
Pass 2
At the end of Pass 2, the second largest number, 66, has moved its way
down to the next-to-last position.
Pass 7. Finally, A1 is compared with A2. Since 13 < 23, no interchange takes
place. Since the list has 8 elements; it is sorted after the seventh pass.
Algorithm
Algorithm 4.4: BUBBLESORT(A, N)
Here A is an array with N elements.
1. Repeat Steps 2 and 3 for K = 1 to N – 1.
2. Set PTR := 1. [Initializes pass pointer PTR.] void bubbleSort(int arr[], int n)
3. Repeat while PTR ≤ N – K: [Executes pass.] {
(a) If A[PTR] < A[PTR + 1], then: int i, j;
Interchange A[PTR] and A[PTR + 1]. for (i = 0; i < n-1; i++)
[End of If structure.] for (j = 0; j < n-i-1; j++)
if (arr[j] > arr[j+1])
(b) Set PTR := PTR + 1.
swap(&arr[j], &arr[j+1]);
[End of inner loop.] }
[End of Step 1 outer loop.]
4. Exit.
Complexity of Bubble Sort Algorithm
Traditionally, the time for a sorting algorithm is measured in
terms of the number of comparisons.
There are n-1 comparisons during the first pass, which places
the largest element in the last position; there are n-2
comparisons in the second step, which places the second
largest element in the next-to-last position; and so on.
Bubble Sort Complexity

Best (Input list is in sorted order) O(n2)
Worst (Input list is sorted in reverse order) O(n2)
Average (Input list is in random order) O(n2)
Contd…
The earlier mentioned function always runs O(n2)
time even if the array is sorted.
To solve this, we can introduce an variable FLAG.
The value of FLAG is set true if there occurs
swapping of elements. Otherwise, it is set false.
After an iteration, if there is no swapping, the
value of swapped will be false.
This means elements are already sorted and there
is no need to perform further iterations.
This will reduce the execution time and helps to
optimize the bubble sort.
Contd...
void bubbleSort(int arr[], int n)
{ int i, j;
bool FLAG;
for (i = 0; i < n-1; i++)
{ FLAG = false;
for (j = 0; j < n-i-1; j++)
{ if (arr[j] > arr[j+1])
{
swap(&arr[j], &arr[j+1]);
FLAG = true;
}
}
// IF no two elements were swapped by inner loop, then break
if (FLAG == false) break;
}
}
Bubble Sort Complexity

Best O(n)
Insertion Sort
Suppose an array A with n elements A[1], A[2], …, A[N] is in memory.
The insertion sort algorithm scans A from A[1] to A[N], inserting each
element A[K] into its proper position in the previously sorted subarray
A[1], A[2], …, A[K – l].
That is:
Pass 1. A[1] by itself is trivially sorted.
Pass 2. A[2] is inserted either before or after A[1] so that: A[1], A[2] is
sorted.
Pass 3. A[3] is inserted into its proper place in A[1], A[2], that is, before
A[1], between A[1] and A[2], or after A[2], so that: A[1], A[2], A[3] is
sorted.
Pass 4. A[4] is inserted into its proper place in A[1], A[2], A[3] so that:
A[1], A[2], A[3], A[4] is sorted.
…
Pass N. A[N] is inserted into its proper place in A[1], A[2], A[N ‒ 1] so that:
A[1], A[2], A[N] is sorted.
Approach
There remains only the problem of deciding how to insert A[K]
in its proper place in the sorted subarray A[1], A[2], …,A[K – 1].
This can be accomplished by comparing A[K] with A[K – l],
comparing A[K] with A[K – 2], comparing A[K] with A[K – 3],
and so on, until first meeting an element A[J] such that A[J] ≤
A[K].
Then each of the elements A[K – l], A[K – 2], …, A[J + 1] is
moved forward one location, and A[K] is then inserted in the J
+ 1 st position in the array.
The algorithm is simplified if there always is an element A[J]
such that A[J] ≤ A[K]; otherwise we must constantly check to
see if we are comparing A[K] with A[1].
This condition can be accomplished by introducing a sentinel
element A[0] = – ∞ (or a very small number).
Example
Suppose an array A contains 8 elements as
follows: 77, 33, 44, 11, 88, 22, 66, 55
Algorithm
Algorithm: INSERTIONSORT(A, N).
This algorithm sorts the array A with N
elements. void insertionSort(int arr[], int n)
1. Set A[0] := – . [Initializes sentinel element.] {
2. Repeat Steps 3 to 5 for K = 2, 3, …, N: int i, key, j;
for (i = 1; i < n; i++) {
3. Set TEMP := A[K] and PTR := K – 1. key = arr[i];
4. Repeat while TEMP < A[PTR]: j = i - 1;
while (j >= 0 && arr[j] > key)
(a) Set A[PTR + 1] := A[PTR]. [Moves
{
element forward.] arr[j + 1] = arr[j];
(b) Set PTR := PTR – 1. j = j - 1;
}
[End of loop.] arr[j + 1] = key;
5. Set A[PTR + 1] := TEMP. [Inserts element in }
proper place.] }

6. Return.
Complexity of Insertion Sort
In the worst case occurs when the array A is in reverse
order and the inner loop must use the maximum
number K – 1 of comparisons.
In the average case, there will be approximately (K –

1)/2 comparisons in the inner loop. Accordingly, for the
average case,
In the best case, insertion sort has a linear running time

(i.e., O(n)).
Selection Sort
Suppose an array A with n elements A[1], A[2],
…, A[N] is in memory.
The selection sort algorithm for sorting A works
as follows.
First find the smallest element in the list and
put it in the first position.
Then find the second smallest element in the
list and put it in the second position. And so
on.
Contd…
Pass 1: Find the location LOC of the smallest in the list of N
elements A[1], A[2], A[N], and then interchange A[LOC] and
A[1]. Then: A[1] is sorted.
Pass 2: Find the location LOC of the smallest in the sublist of N-1
elements A[2], A[3], A[N], and then interchange A[LOC] and
A[2]. Then: A[1], A[2] is sorted, since A[1] ≤ A[2].
Pass 3: Find the location LOC of the smallest in the sublist of N ‒
2 elements A[3], A[4], A[N], and then interchange A[LOC]
and A[3]. Then: A[1], A[2], A[3] is sorted, since A[2] ≤ A[3].
…
Pass N ‒ 1: Find the location LOC of the smaller of the elements
A[N ‒ 1], A[N], and then interchange A[LOC] and A [N ‒
1]. Then: A[1], A[2], ...A[N] is sorted, since A[N ‒ 1] ≤
A[N].
Thus A is sorted after N – 1 passes.
Example
Suppose an array A contains 8 elements as
follows: 77, 33, 44, 11, 88, 22, 66, 55
Algorithm
Algorithm : SELECTIONSORT(A, N)
This algorithm sorts the array A with N elements.
1. Repeat Steps 2 and 3 for K = 1, 2, …, N –1:
2. Call MIN(A, K, N, LOC). void selectionSort(int arr[], int n)
3. [Interchange A[K] and A[LOC].] {
int i, j, min;
Set TEMP := A[K], A[K] := A[LOC] and
A[LOC] := TEMP. // One by one move boundary of unsorted subarray
for (i = 0; i < n-1; i++)
[End of Step 1 loop.] {
// Find the minimum element in unsorted array
4. Exit. min = i;
for (j = i+1; j < n; j++)
________________________________________ if (arr[j] < arr[min])
MIN(A, K, N, LOC) min = j;
// Swap the found minimum element with the first
1. Set MIN := A[K] and LOC := K. element
swap(&arr[min_idx], &arr[i]);
2. Repeat for J = K + 1, K + 2, …, N: }
If MIN > A[J], then: Set MIN := A[J] }
and LOC := A[J] and LOC := J.

[End of loop.]
3. Return.
Complexity of the Selection Sort
MIN(A, K, N, LOC) requires n –K comparisons.
That is, there are n – 1 comparisons during Pass 1 to find
the smallest element, there are n – 2 comparisons during
Pass 2 to find the second smallest element, and so on.
Accordingly, f (n) = (n – 1) + (n – 2) +... + 2 + 1 = O(n2)
Quick Short
In the bubble sort, consecutive items are
compared and possibly exchanged on each
pass through the list, which means that many
exchanges may be needed to move an element
to its correct position.
Quick short is more efficient than the bubble
sort because, here fewer exchanges are
required to correctly position an element.
Approach
Quick sort is an algorithm of the divide-and-conquer
type.
Here, the problem of sorting is reduced to the problem
of sorting two smaller sets.
The reduction step of the Quick sort algorithm finds the
final position of one of the numbers.
Each iteration of the quick sort selects an element,
known as pivot, and divides the list into three groups
A partition of elements whose keys are less than the
pivot’s key, the pivot element that is placed in its
ultimately correct location in the list, and a partition of
elements greater than or equal to the pivot’s key.
Example
Suppose A is the following list of 12 numbers:
Scan from right to left , find

first number less than
We use the first number, 44. This is accomplished as follows. Beginning

with the last number, 66, scan the list from right to left, comparing
each number with 44 and stopping at the first number less than 44.
Here the number is 22. Interchange 44 and 22 to obtain the list.
• Beginning with 22, next scan the list in the opposite direction, from
left to right, comparing each number with 44 and stopping at the first
number greater than 44. The number is 55. Interchange 44 and 55 to
obtain the list.
Contd…
Scan from left to right, find first number

greater than
Scan from right to left , find first

number less than
Contd…
Scan from left to right, find first number

greater than
Contd…
The above reduction step is repeated with each sublist
containing 2 or more elements.
Since we can process only one sublist at a time, we must
be able to keep track of some sublists for future
processing.
This is accomplished by using two stacks, called LOWER
and UPPER, to temporarily “hold” such sublists.
That is, the addresses of the first and last elements of
each sublist, called its boundary values, are pushed onto
the stacks LOWER and UPPER, respectively; and the
reduction step is applied to a sublist only after its
boundary values are removed from the stacks.
Algorithm
Algorithm: QuickSort(A,N)
This algorithm sorts an array A with N elements.
1. [Initialize.] TOP := NULL.
2. [Push boundary values of A onto stacks when A has 2 or more elements.]
If N > 1, then: TOP := TOP + 1, LOWER[1] :=1, UPPER[1] := N.
3. Repeat Steps 4 to 7 while TOP ≠ NULL.
4. [Pop sublist from stacks.]
Set BEG := LOWER[TOP], END :=UPPER[TOP], TOP := TOP – 1.
5. Call QUICK(A, N, BEG, END, LOC)
6. [Push left sublist onto stacks when it has 2 or more elements.]
If BEG < LOC – 1, then:
TOP := TOP + 1, LOWER[TOP] := BEG,
UPPER[TOP] = LOC – 1. [End of If structure.]
7. [Push right sublist onto stacks when it has 2 or more elements.]
If LOC + 1 < END, then:
TOP := TOP + 1, LOWER[TOP] := LOC +1, UPPER[TOP] := END.
[End of If structure.] [End of Step 3 loop.]
8. Exit.
Contd…
Procedure 6.7: QUICK(A, N, BEG, END, LOC) (ii) Set LOC := RIGHT.
Parameters BEG and END contain the boundary (iii) Go to Step 3.
values of the sublist of A to which this procedure
applies. LOC keeps track of the position of the [End of If structure.]
first element A[BEG] of the sublist during the 3. [Scan from left to right.]
procedure. The local variables LEFT and RIGHT
will contain the boundary values of the list of (a) Repeat while A[LEFT] ≤ A[LOC) and
elements that have not been scanned. LEFT ≠ LOC:
1. [Initialize.] Set LEFT := BEG, RIGHT := END LEFT := LEFT + 1.
and LOC := BEG. [End of loop.]
2. [Scan from right to left.] (b) If LOC = LEFT, then: Return.
(a) Repeat while A[LOC) ≤ A[RIGHT] and
(c) If A[LEFT] > A[LOC], then
LOC ≠ RIGHT:
RIGHT := RIGHT – 1. (i) [Interchange A[LEFT] and
[End of loop.] A[LOC].]
(b) If LOC = RIGHT, then: Return. TEMP := A[LOC], A[LOC] :=
(c) If A[LOC] > A[RIGHT], then: A[LEFT],
(i) [Interchange A[LOC) and A[RIGHT].] A[LEFT] := TEMP.
TEMP := A[LOC), A[LOC] := A[RIGHT], (ii) Set LOC := LEFT.
A[RIGHT] := TEMP. (iii) Go to Step 2.
Complexity of Quick Sort Algorithm
In Quick Sort, the list is partitioned into two parts.
So, the complexity will be based on:
Time to partition the given list
Time to sort left sub list
Time to sort right sub list
The best case timing analysis is possible when
the array is always partitioned in half.
n n
Best Case Analysis: T  n   cn  T    T  
2 2
n
 cn  2T   , for some constant c
2
n
Thus, T  n   cn  2T  
2
 n  n   n 
 cn  2  c  2T  2    2cn  22 T  2 
 2  2  2 
 n  n   n
 2cn  22  c 2  2T  3    3cn  23 T  3 
 2  2  2 
 n 
 kcn  2k T  k  [Let 2 k  n  k  log 2 n]
2 
 cn log 2 n  nT 1
 cn log 2 n  n
 O  n log 2 n 
Worst Case Analysis:
T  n   cn  T  0   T  n  1 , for some constant c
Thus, T  n   cn  T  n  1
 cn  c  n  1  T  n  2 
 cn  c  n  1  c  n  2   T  n  3
 cn  c  n  1  c  n  2   c  n  3   c 1  T  0 
 c  n   n  1   n  2    2  1
 n  n  1 
 c 
 2 
 O  n2 
Merging
Suppose A is a sorted list with r elements and B
is a sorted list with s elements.
The operation that combines the elements of A
and B into a single sorted list C with n = r + s
elements is called merging.
One simple way to merge is to place the
elements of B after the elements of A and then
use some sorting algorithm on the entire list.
This method does not take advantage of the
fact that A and B are individually sorted.
Intelligent Approach
Merging Algorithm
Algorithm: MERGING(A, R, B, S, C)
Let A and B be sorted arrays with R and S elements, respectively. This
algorithm merges A and B into an array C with N = R + S elements.
1. [Initialize.] Set NA := 1, NB := 1 and PTR:= 1.
2. [Compare.] Repeat while NA ≤ R and NB≤ S:
If A[NA] < B[NB], then:
(a) Set C[PTR] := A[NA].
(b) Set PTR :=PTR + 1 and NA := NA + 1.
Else:
(a) Set C[PTR] := B[NB].
(b) Set PTR :=PTR + 1 and NB := NB + 1.
[End of loop.]
Contd…
3. [Assign remaining elements to C.]
If NA > R, then:
Repeat for K = 0, 1, 2, …, S – NB:
Set C[PTR + K] := B[NB + K].
[End of loop.]
Else:
Repeat for K = 0, 1, 2, …, R – NA:
Set C[PTR + K] := A[NA + K].
[End of loop.]
4. Exit.
Complexity of the Merging Algorithm
The input consists of the total number n = r + s
of elements in A and B.
Each comparison assigns an element to the
array C, which eventually has n elements.
Accordingly, the number f(n) of comparisons
cannot exceed n: f(n) ≤ n =O(n)
In other words, the merging algorithm can be
run in linear time.
Merge Sort
The merge sort applies the divide-and-conquer
strategy to sort a sequence.
First it subdivides the sequence into subsequences
of singletons.
Then it successively merges the subsequences
pairwise until a single sequence is re-formed.
Each merge preserves order, so each merged
subsequence is sorted.
When the final merge is finished, the complete
sequence is sorted.
.
Example
Algorithm
MergeSort(array a, int left, int right)
{
if ( left < right )
{
mid = (left + right) / 2
MergeSort(a, left, mid)
MergeSort(a, mid+1, right)
merge(a, left, mid, right)
}
}
MergeSort() is a recursive function and low >= high is the
base case, i.e. there is 0 or 1 item
Contd…
Complexity: Merge-Sort Algorithm
Contd…
So, the merge sort works by repeatedly dividing
the array in half until the pieces are singletons,
and then it merges the pieces pairwise until a
single piece remains.
The number of iterations in the first part equals
the number of times n can be halved: that is,
log2n.
The second part of the process reverses the first.
So the second part also has log2n steps.
So the entire algorithm has O(log2n) steps.
Each step compares all n elements.
So the total number of comparisons is O(n lg n).
Limitation
The main drawback of merge-sort is that it
requires an auxiliary array with n elements.
Sorting by Distribution
These sorting algorithms are radically different than the
previous discussed algorithms.
The unique characteristic of a distribution sorting algorithm is
that it does not make use of comparisons to do the sorting.
Instead, distribution sorting algorithms rely on a
priori knowledge about the universal set from which the
elements to be sorted are drawn.
For example, if we know a priori that the size of the universe
is a small, fixed constant, say m, then we can use the counting
sort algorithm.
Similarly, if we have a universe the elements of which can be
represented with a small, finite number of bits (or even digits,
letters, or symbols), then we can use the radix sorting
algorithm.
Counting Sort
The essential requirement in this sorting
algorithm is that the size of the universe from
which the elements to be sorted are drawn is a
small, fixed constant, say m.
For example, suppose that we are sorting
elements drawn from {0,1,…,m-1} i.e., the set
of integers in the interval [0,m-1].
Counting sort uses m counters. The i-th counter
keeps track of the number of occurrences of
the i-th element of the universe.
Example
int count[m]
for ( int k=0; k < n; k++ ) //suppose m<n
count[v[k]]++;
Complexity: O(n)
Radix Sort
The downfall of counting sort is that it may not be too
practical if the range of elements is too large.
For example, if the range of the n elements we need to
sort was from 1 to n3, then simply creating the auxiliary
array will take O(n3) time and counting sort will
asymptotically do worse than insertion sort.
Radix sort helps solve this problem by sorting the
elements digit by digit.
The idea is that we can sort integers by first sorting
them by their least significant digit (i.e. the ones digit),
then sorting the result of that sort by their next
significant digit (i.e. the tens digit), and so on until we
sort the integers by their most significant digit.
Example
Contd…
In case of list of names. Here the radix is 26 (the 26
letters of the alphabet.)
Specifically, the list of names is first sorted according to
the first letter of each name.
That is, the names are arranged in 26 classes, where the
first class consists of those names that begin with “A,”
the second class consists of those names that begin with
“B,” and so on.
During the second pass, each class is alphabetized
according to the second letter of the name. And so on.
If no name contains, for example, more than 12 letters,
the names are alphabetized with at most 12 passes.
Implementation
Consider the following list
Contd…
Contd…
Contd…
Algorithm
radix sort(Array)
{
d= maximum number of digits in the largest element
create d buckets of size 0-9
for i = 0 to d
sort the elements according to ith place digits using
Counting Sort
}
Complexity of Radix Sort
Suppose a list A of n items A1, A2, …, An is given.
Let d denote the radix, and suppose each item Ai is
represented by means of s of the digits: Ai = di1di2 … dis
The radix sort algorithm will require s passes
Hence the number C(n) of comparisons for the
algorithm is bounded as follows: C(n) ≤ d×s×n
Although d is independent of n, the number s does
depend on n.
In the worst case, s = n, so C(n) = O(n2).
In the best case, s = logdn, so C(n) = O(n log n).
In other words, radix sort performs well only when the
number s of digits in the representation of the Ai’s is
small.
Limitation
Another drawback of radix sort is that one may
need d×n memory locations.
Thank You…
Any Queries...?
Sorting Algorithms Sorting Algorithms
Outline I
1 Introduction
Sorting
Classification of Sorting
Applications of Sorting
Sorting Algorithms 2 Bubble Sort
Algorithm
Complexity
Modified version
3 Selection Sort
Algorithm
Complexity
4 Insertion Sort
Algorithm
Complexity
5 Merge Sort
Merging
Introduction
1 Introduction
Outline II Sorting
Complexity Applications of Sorting
2 Bubble Sort
6 Quicksort Sort Algorithm
Algorithm Complexity
Complexity Modified version
Randomized Quick sort 3 Selection Sort
Algorithm
Complexity
7 Count Sort
Algorithm 4 Insertion Sort
Complexity Algorithm
Complexity
5 Merge Sort
8 Radix Sort Merging
Complexity
6 Quicksort Sort
Algorithm
Sorting
Complexity
Randomized Quick sort
What is Sorting? I
Sorting is an algorithm that arranges the elements of a list

in a certain order [either ascending or descending]. The
7 Count Sort
Algorithm output is a permutation or reordering of the input.
Complexity I Let A be a list of n numbers
I Sorting refers to the operation of rearranging elements of A
so that they are in increasing order
I Suppose A is: 8, 4, 19, 2, 7, 13, 5, 16
I The sorted list would be: 2, 4, 5, 7, 8, 13, 16, 19
8 Radix Sort I The above definition of sorting refers to arranging numerical
Algorithm data in increasing order. This restriction is only for notational
Complexity convenience. Sorting also may mean arranging numerical
data in decreasing order

Sorting Classification of Sorting
Why is Sorting Necessary? I Classification of Sorting Algorithms I
Sorting algorithms are generally categorized based on the

following parameters.
I By Number of Comparisons: In this method, sorting al-
Sorting is one of the important categories of algorithms in gorithms are classified based on the number of compar-
computer science and a lot of research has gone into this isons. For comparison based sorting algorithms, best case
category. Sorting can significantly reduce the complexity of behavior is O(n log n) and worst case behavior is O(n2 ).
a problem, and is often used for database algorithms and Comparison-based sorting algorithms evaluate the elements
searches. of the list by key comparison operation and need at least
O(n log n) comparisons for most inputs.
I Non – comparison (linear): sorting algorithms like Count-
ing sort, Bucket sort, Radix sort, etc. Linear Sorting algo-
rithms impose few restrictions on the inputs to improve the
complexity.
Classification of Sorting Classification of Sorting
Classification of Sorting Algorithms II Classification of Sorting Algorithms III
I By Number of Swaps: In this method, sorting algorithms

are categorized by the number of swaps (also called inver- I By Adaptability: With a few sorting algorithms, the com-
sions). plexity changes based on pre-sortedness [quick sort]: pre-
I By Memory Usage: Some sorting algorithms are “in place” sortedness of the input affects the running time. Algorithms
and they need O(1) or O(log n) memory to create auxiliary that take this into account are known to be adaptive.
locations for sorting the data temporarily. I Other Classifications: Another method of classifying sort-
I By Recursion: Sorting algorithms are either recursive [quick ing algorithms is:
sort] or non-recursive [selection sort, and insertion sort], and • Internal Sort: Sort algorithms that use main memory exclu-
there are some algorithms which use both (merge sort). sively during the sort are called internal sorting algorithms.
I By Stability: Sorting algorithm is stable if for all indices i This kind of algorithm assumes high-speed random access
and j such that the key A[i] equals key A[j], if record R[i] to all memory.
• External Sort: Sorting algorithms that use external memory,
precedes record R[j] in the original file, record R[i] precedes
such as tape or disk, during the sort come under this cate-
record R[j] in the sorted list. Few sorting algorithms maintain
gory.
the relative order of elements with equal keys (equivalent
elements retain their relative positions even after sorting).

Introduction Bubble Sort
1 Introduction
Applications of Sorting I Sorting
2 Bubble Sort
Uniqueness testing Algorithm
Complexity
Deleting duplicates
Modified version
Prioritizing events 3 Selection Sort
Frequency counting Algorithm
Complexity
Reconstructing the original order
4 Insertion Sort
Set intersection/union Algorithm
Finding a target pair x, y such that x + y = z Complexity
5 Merge Sort
Efficient searching Merging
Complexity
6 Quicksort Sort
Algorithm
Bubble Sort Bubble Sort
Complexity
Bubble Sort I
7 Count Sort Bubble sort is the simplest sorting algorithm.

Algorithm It works by iterating the input array from the first element
Complexity to the last, comparing each pair of elements and swapping
them if needed.
Bubble sort continues its iterations until no more swaps are
needed.
8 Radix Sort
The algorithm gets its name from the way smaller elements
Algorithm
Complexity “bubble” to the top of the list.

Bubble Sort II Bubble Sort III
Suppose list of A[1], A[2], · · · , A[N] is in memory. Then

Generally, insertion sort has better performance than bub- bubble sort algorithm works as follows
ble sort. 1. Step 1. Compare A[1] and A[2] and arrange them in the de-
sired order, so that A[l] < A[2]. Then compare A[2] and A[3]
Some researchers suggest that we should not teach bubble and arrange them so that A[2] < A[3]. Then compare A[3]
sort because of its simplicity and high time complexity. and A[4] and arrange them so that A[3] < A[4]. Continue
The only significant advantage that bubble sort has over until we compare A[N − 1] with A[N] and arrange them so
that A[N − 1] < A[N].
other implementations is that it can detect whether the input
Observe that Step 1 involves n − 1 comparisons. (During
list is already sorted or not. Step 1, the largest element is “bubbled up” to the nth position
or “sinks” to the nth position.) When Step 1 is completed,
A[N] will contain the largest element.
Bubble Sort IV Bubble Sort V

Suppose we want to sort the numbers 32, 51, 27, 85, 66,
2. Step 2. Repeat Step 1 with one less comparison; that is, 23, 13, 57
now we stop after we compare and possibly rearrange A[N − I Step 1 results as follows
2] and A[N − 1]. (Step 2 involves N − 2 comparisons and,
32, 51, 27, 85, 66, 23, 13, 57
when Step 2 is completed, the second largest element will
32, 27, 51, 85, 66, 23, 13, 57
occupy A[N − 1].)
32, 27, 51, 85, 66, 23, 13, 57
3. Step 3. Repeat Step 1 with two fewer comparisons; that is,
32, 27, 51, 66, 85, 23, 13, 57
we stop after we compare and possibly rearrange A[N − 3]
32, 27, 51, 66, 23, 85, 13, 57
and A[N − 2].
32, 27, 51, 66, 23, 13, 85, 57
···
32, 27, 51, 66, 23, 13, 57, 85
···
I After step 1 the largest number 85 comes to last position
···
I After step 2
4. Step N - 1. Compare A[l] with A[2] and arrange them so that
A[l] < A[2]. 27, 33, 51, 23, 13, 57, 66, 85
After n − 1 steps, the list will be sorted in increasing order. I After Step N-1
13, 23, 27, 33, 51, 57, 66, 85

Algorithm I Complexity I
BUBBLE(DATA,N)
1. Repeat steps 2 and 3 for K=1 to N-1.
2. Set PTR:=1. [Initializes pass pointer PTR.]
Here Step 1 requires n − 1 comparisons Step2 requires n − 2
3. Repeat while PTR≤N-K: [Executes pass.]
comparisons
(a) If DATA[PTR]>DATA[PTR+1], then:
f (n) = (n − 1) + (n − 2) + · · · + 2 + 1 = n(n−1)
2 = O(n2 )
Interchange DATA[PTR] and DATA[PTR+1] 2
[End of If structure] Hence the complexity is of the order of n
(b) Set PTR:=PTR+1
[End of inner loop.]
[End of step 1 outer loop.]
4. Exit.
Complexity Modified version
Complexity II Modified version I
Algorithm takes O(n2 ) (even in best case).

Performance We can improve it by using one extra flag.
I Worst case complexity : O(n2 ) No more swaps indicate the completion of sorting.
I Best case complexity (Improved version) : O(n)
I Average case complexity (Basic version) : O(n2 ) If the list is already sorted, we can use this flag to skip the
I Worst case space complexity : O(1) auxiliary remaining passes.
This modified version improves the best case of bubble sort
to O(n).

Selection Sort Selection Sort
1 Introduction Complexity
Sorting Randomized Quick sort
2 Bubble Sort
Algorithm
Complexity 7 Count Sort
Modified version Algorithm
3 Selection Sort Complexity
Algorithm
Complexity
4 Insertion Sort
Algorithm
8 Radix Sort
Complexity
Algorithm
5 Merge Sort Complexity
Merging
Complexity
6 Quicksort Sort
Algorithm
Selection Sort I Selection Sort II
Selection sort is an in-place sorting algorithm. Advantages

Selection sort works well for small files. I Easy to implement
It is used for sorting the files with very large values and I In-place sort (requires no additional storage space)
small keys. Disadvantages
This is because selection is made based on keys and swaps I Doesn’t scale well: O(n2 )
are made only when required.

Selection Sort III Selection Sort IV
More precisely:
I Pass 1. Find the location LOC of the smallest in the list of
Suppose an array A with n elements A[1], A[2], · · · , A[N] is N elements A[1], A[2], A[N], and then interchange A[LOC]
in memory. The selection sort algorithm for sorting A works and A[1]. Then: A[1] is sorted.
as follows. I Pass 2. Find the location LOC of the smallest in the sublist
First find the smallest element in the list and put it in the first of N − 1 elements A[2], A[3], A[N], and then interchange
position. A[LOC] and A[2]. Then: A[1], A[2] is sorted, since A[1] ≤
A[2].
Then find the second smallest element in the list and put it I Pass 3. Find the location LOC of the smallest in the sublist
in the second position. of N − 2 elements A[3], A[4], A[N], and then interchange
And so on. A[LOC] and A[3]. Then: A[1], A[2], A[3] is sorted, since
A[2] ≤ A[3].
I ···
I ···
Selection Sort V Selection Sort VI
I Pass N - 1. Find the location LOC of the smaller of the

elements A[N − 1], A[N], and then interchange A[LOC] and
A[N–1]. Then: A[1], A[2], · · · , A[N] is sorted, since A[N −
1] ≤ A[N].
Thus, A is sorted after N–1 passes.

Algorithm Algorithm
Algorithm I Algorithm II
Algorithm MIN(A, K, N, LOC): An array A is in memory. This procedure

1. Find the minimum value in the list finds the location LOC of the smallest element among A[K ], A[K +
2. Swap it with the value in the current position 1], · · · , A[N].
1. Set MIN := A[K ] and LOC := K . [Initializes pointers.]
3. Repeat this process for all the elements until the entire array
2. Repeat for J = K + 1, K + 2, · · · , N :
is sorted If MIN > A[J], then: Set MIN := A[J] and LOC := A[J] and LOC := J.
This algorithm is called selection sort since it repeatedly se- [End of loop.]
3. Return.
lects the smallest element.
Algorithm III Complexity I
(Selection Sort) SELECTION(A, N) Observe that MIN(A, K, N, LOC) requires n − K compar-

This algorithm sorts the array A with N elements. isons.
1. Repeat Steps 2 and 3 for K = 1, 2, · · · , N − 1 :
2. Call MIN(A, K , N, LOC).
That is, there are n − 1 comparisons during Pass 1 to find
3. [Interchange A[K ] and A[LOC].] the smallest element, there are n − 2 comparisons during
Set TEMP := A[K ], A[K ] := A[LOC] and A[LOC] := TEMP. Pass 2 to find the second smallest element, and so on.
[End of Step 1 loop.] f (n) = (n − 1) + (n − 2) + · · · + 2 + 1 = n(n−1)
2 = O(n2 )
4. Exit.
Hence the complexity is of the order of n2 .

Selection Sort Insertion Sort
Complexity
1 Introduction
Complexity II Sorting
2 Bubble Sort
Algorithm
Complexity
Performance Modified version
I Worst case complexity : O(n2 ) 3 Selection Sort
I Best case complexity : O(n2 ) Algorithm
I Average case complexity : O(n2 ) Complexity
I Worst case space complexity: O(1) auxiliary 4 Insertion Sort
Algorithm
Complexity
5 Merge Sort
Merging
Complexity
6 Quicksort Sort
Algorithm
Insertion Sort Insertion Sort
Complexity
Insertion sort I
7 Count Sort Insertion sort is a simple and efficient comparison sort.

Algorithm
Complexity In this algorithm, each iteration removes an element from
the input data and inserts it into the correct position in the
list being sorted.
Thechoice of the element being removed from the input is
8 Radix Sort random and this process is repeated until all input elements
Algorithm have gone through.
Complexity

Insertion sort II Insertion sort III
Advantages
I Simple implementation
I Efficient for small data
I Adaptive: If the input list is presorted [may not be com- Suppose an array A with n elements A[1], A[2], · · · , A[N] is
pletely] then insertions sort takes O(n + d), where d is the in memory.
number of inversions Practically more efficient than selec- The insertion sort algorithm scans A from A[1] to A[N], in-
tion and bubble sorts, even though all of them have O(n2 )
serting each element A[K ] into its proper position in the pre-
worst case complexity
I Stable: Maintains relative order of input data if the keys are viously sorted subarray A[1], A[2], · · · , A[K − l].
same
I In-place: It requires only a constant amount O(1) of addi-
tional memory space
I Online: Insertion sort can sort the list as it receives it
Insertion sort IV Insertion sort V
Suppose an array A with n elements is in memory.

That is: The insertion sort algorithm scans A from A[1] to A[N] in-
1. Pass 1. A[1] by itself is trivially sorted. serting each element A[K] into its proper position.
2. Pass 2. A[2] is inserted either before or after A[1] so that:
Consider A with 8 elements: 77, 33, 44, 11, 88, 22, 66, 55
A[1], A[2] is sorted.
3. Pass 3. A[3] is inserted into its proper place in A[1], A[2], 1. In pass 1 A[1] by itself is trivially sorted
that is, before A[1], between A[1] and A[2], or after A[2], so 77, 33, 44, 11, 88, 22, 66, 55
that: A[1], A[2], A[3] is sorted. 2. In pass 2 A[2] is inserted either before or after A[1] so that
4. Pass 4. A[4] is inserted into its proper place in A[1], A[2], A[1], A[2] is sorted
A[3] so that: A[1], A[2], A[3], A[4] is sorted. 33, 77, 44, 11, 88, 22, 66, 55
5. · · · 3. In pass 3 A[3] is inserted so that A[1], A[2], A[3] is sorted

33, 44, 77, 11, 88, 22, 66, 55
6. Pass N. A[N] is inserted into its proper place in A[1], A[2],
· · · , A[N − 1] so that: A[1], A[2], A[N] is sorted. 4. In pass N A[N] is inserted so that A[1], A[2], A[3], · · · , A[N]
is sorted
11, 22, 33, 44, 55, 66, 77, 88

Algorithm
Insertion sort VI Algorithm
(Insertion Sort) INSERTION(A,N) This algorithm sorts the array

A with N elements.
1. Set A[0]:=-∞. [Initializes sentinel element.]
2. Repeat Steps 3 to 5 for K = 2, 3, · · · , N
3. Set TEMP:= A[K] and PTR:=K-1
4. Repeat while TEMP<A[PTR]
(a) Set A[PTR+1]:=A[PTR]. [Moves element forward.]
(b) Set PTR:=PTR-1
[End of loop]
5. Set A[PTR+1]:=TEMP. [Inserts element in proper place.]
[End of Step 2 loop]
6. Return
Complexity Complexity
Complexity I Complexity II
Performance: If every element is greater than or equal to

1. The number of comparisons in the insertion sort can be every element to its left, the running time of insertion sort
computed as is Θ(n). This situation occurs if the array starts out already
n(n−1) sorted, and so an already-sorted array is the best case for
1 + 2 + · · · + (n − 1) = 2 insertion sort.
2. Hence the time complexity is of the order of n2 I Worst case complexity: Θ(n2 )
3. Hence insertion sort is a very slow algorithm when the value I Best case complexity: Θ(n)
of n is very large I Average case complexity: Θ(n2 )
I Worst case space complexity: O(n2 ) total, O(1) auxiliary

Merge Sort Merge Sort
1 Introduction Complexity
Sorting Randomized Quick sort
2 Bubble Sort
Algorithm
Complexity 7 Count Sort
Modified version Algorithm
3 Selection Sort Complexity
Algorithm
Complexity
4 Insertion Sort
Algorithm
8 Radix Sort
Complexity
Algorithm
5 Merge Sort Complexity
Merging
Complexity
6 Quicksort Sort
Algorithm
Merging Merging
Merging I MERGE I
Suppose A is a sorted list with r elements and B is a sorted

list with s elements
The operation that combines the elements of A and B into
a single sorted list with r+s elements is called merging Assume A is sorted with r elements and lower bound LBA. B is
The lists can be merged as follows sorted with s elements and lower bound LBB and C has lower
I At each step starting from the front two arrays elements are bound LBC. Then UBA = LBA + r − 1 and UBB = LBB + s − 1
compared and smaller one is placed in combined list
I The above step is repeated until one of the list becomes
empty
I When one list is empty then the remaining elements of other
list are placed in the combined list
I Thus merging can be done in linear time n that is in n = r +s

Merging Merging
MERGE II MERGE III
MERGE(A,R,LBA,S,LBB,C,LBC)
3. [Assign remaining elements to C.]
1. [Initialize] Set NA := LBA, NB = LBB, PTR := LBC, UBA = If NA > UBA then:
LBA + R − 1, UBB = LBB + S − 1 I Repeat for K=0,1,2, · · · , UBB-NB
2. [Compare.] Repeat while NA ≤ UBA and NB ≤ UBB: Set C[PTR+K]:=B[NB+K]
If A[NA] < B[NB] then [End of loop.]
(a) [Assign element from A to C.] Set C[PTR]:=A[NA] Else:
(b) [Update pointers.] Set PTR:=PTR+1 AND NA:=NA+1 I Repeat for K=0,1,2, · · · , UBA-NA
Else: Set C[PTR+K]:=A[NA+K]
(a) [Assign element from B to C.] Set C[PTR]:=B[NB] [End of loop.]
(b) [Update pointers.] Set PTR:=PTR+1 AND NB:=NB+1 [End of If structure.]
[End of If structure.] 4. Return
[End of loop.]
Merging Merging
Binary Search and Insertion Algorithm I Binary Search and Insertion Algorithm II
The binary search and insertion algorithm does not take into
Suppose the number r of elements in a sorted array A is account the fact that A is sorted. Accordingly, the algorithm
much smaller than the number s of elements in a sorted may be improved in two ways as follows. (Here we assume
array B. that A has 5 elements and B has 100 elements.)
One can merge A with B as follows. For each element A[K] Reducing the target set: Suppose after the first search
of A, use a binary search on B to find the proper location we find that A[1] is to be inserted after B[16]. Then we need
to insert A[K] into B. Each such search requires at most only use a binary search on B[17], ..., B[100] to find the
log s comparisons; hence this binary search and insertion proper location to insert A[2]. And so on.
algorithm to merge A and B requires at most r log s com- Tabbing: The expected location for inserting A[1] in B is
parisons. We emphasize that this algorithm is more efficient near B[20] (that is, B[s/r]), not near B[50]. Hence we first
than the usual merging Algorithm only when r s, that is, use a linear search on B[20], B[40], B[60], B[80] and B[100]
when r is much less than s. to find B[K] such that A[1] ≤ B[K], and then we use a binary
search on B[K - 20], B[K - 19], ..., B[K].

Merging Merging
Merge sort I Merge sort II
Suppose array A with 14 elements:

(66, 33, 40, 22, 55, 88, 60, 11, 80, 20, 50, 44, 77, 30)
1. Merge each pair of elements to obtain the following sorted
pairs:
(33, 66) (22, 40) (55, 88) (11, 60) (20, 80) (44, 50) (30, 77) 5. The original array is now sorted
2. Merge each pair of pairs to obtain the following list of quadru- 6. The algorithm requires atmost log2 n steps to sort an n ele-
plets: ment array
(22, 33, 40, 66) (11, 55, 60, 88) (20, 44, 50, 80) (30, 77)
3. Merge each pair of sorted quadruplets to obtain the following
two sorted subarrays:
(11, 22, 33, 40, 55, 60, 66, 88) (20, 30, 44, 50, 77, 80)
4. Merge the two sorted subarrays to obtain single sorted ar-
ray:
(11, 20, 22, 30, 33, 40, 44, 50, 60, 66, 77, 80, 88)
Merging Merging
Mergesort I Mergesort II
MERGEPASS(A,N,L,B)
N
1. Set Q:=INT( 2∗L ), S:=2*L*Q and R:=N-S
2. Repeat for J = 1, 2, · · · , Q:
(a) Set LB:=1+(2*J-2)*L
The procedure merges pairs of sub arrays of A and assigns (b) Call MERGE(A,L,LB,A,L,LB+L,B,LB)
them to B. [End of loop.]
The N element array A is composed of sorted subarrays 3. [Only one subarray left?]
If R≤L then:
where each subarray has L elements except possibly last
Repeat for J = 1, 2, · · · , R:
subarray which may have fewer than L elements Set B(S+J):=A(S+J).
[End of loop.]
Else:
Call MERGE(A,L,S+1,A,R,L+S+1,B,S+1)
[End of if structure.]
4. Return

Merging Merging
Merge sort I Important Notes I
This algorithm sorts N-element array A using an auxiliary Merge sort is an example of the divide and conquer strat-
array B. egy.
MERGESORT(A, N) Merging is the process of combining two sorted files to make
1. Set L := 1 one bigger sorted file.
2. Repeat steps 3 to 6 while L < N:
Selection is the process of dividing a file into two parts: k
3. Call MERGEPASS(A, N, L, B)
4. Call MERGEPASS(B, N, 2*L, A) smallest elements and n - k largest elements.
5. Set L := 4 * L Selection and merging are opposite operations
[End of Step 2 loop] I selection splits a list into two lists
6. Exit I merging joins two files to make one file
Merging Complexity
Important Notes II Complexity I
Merge sort is Quick sort’s complement

Merge sort accesses the data in a sequential manner
This algorithm is used for sorting a linked list
Merge sort is insensitive to the initial order of its input The algorithm requires atmost log2 n passes
In Quick sort most of the work is done before the recursive Each pass merges n elements that is atmost n comparisons
calls. Quick sort starts with the largest subfile and finishes
Hence the time complexity is atmost n log2 n
with the small ones and as a result it needs stack. More-
over, this algorithm is not stable. Merge sort divides the
list into two parts; then each part is conquered individually.
Merge sort starts with the small subfiles and finishes with
the largest one. As a result it doesn’t need stack. This al-
gorithm is stable.

Merge Sort Quicksort Sort
Complexity
1 Introduction
Complexity II Sorting
2 Bubble Sort
Algorithm
Complexity
Performance Modified version
I Worst case complexity : Θ(n log n) 3 Selection Sort
I Best case complexity : Θ(n log n) Algorithm
I Average case complexity : Θ(n log n) Complexity
I Worst case space complexity: Θ(n) auxiliary 4 Insertion Sort
Algorithm
Complexity
5 Merge Sort
Merging
Complexity
6 Quicksort Sort
Algorithm
Quicksort Sort Quicksort Sort
Complexity
QUICKSORT I
7 Count Sort An Application Of Stacks.

Algorithm Quicksort is an algorithm of the divide-and-conquer type.
Complexity That is, the problem of sorting a set is reduced to the prob-
lem of sorting two smaller sets.
It picks an element as pivot and partitions the given array
around the picked pivot.
8 Radix Sort
The reduction step of the quicksort algorithm finds the final
Algorithm
Complexity position of pivot.

QUICKSORT II QUICKSORT III
Example: Suppose A is the following list of 12 numbers: Beginning with 22, next scan the list in the opposite direc-
tion, from left to right, comparing each number with 44 and
stopping at the first number greater than 44. The number is
The reduction step of the quicksort algorithm finds the final 55. Interchange 44 and 55 to obtain the list.
position of one of the numbers.
we use the first number, 44.
Beginning with the last number, 66, scan the list from right (Observe that the numbers 22, 33 and 11 to the left of 44
to left, comparing each number with 44 and stopping at the are each less than 44.)
first number less than 44. The number is 22. Interchange Beginning this time with 55, now scan the list in the original
44 and 22 to obtain the list direction, from right to left, until meeting the first number
less than 44. It is 40. Interchange 44 and 40 to obtain the
list.
QUICKSORT IV QUICKSORT V
Beginning with 77, scan the list from right to left seeking a
number less than 44. We do not meet such a number before
meeting 44. This means all numbers have been scanned
and compared with 44. Furthermore, all numbers less than
44 now form the sublist of numbers to the left of 44, and all
Beginning with 40, scan the list from left to right. The first numbers greater than 44 now form the sublist of numbers
number greater than 44 is 77. Interchange 44 and 77 to to the right of 44, as shown below:
obtain the list
Thus 44 is correctly placed in its final position, and the task

of sorting the original list A has now been reduced to the
task of sorting each of the above sublists.

Algorithm
QUICKSORT VI Algorithm I
Here A is an array with N elements. Parameters BEG and END

contain the boundary values of the sublist of A to which this pro-
The above reduction step is repeated with each sublist con-
cedure applies. LOC keeps track of the position of the first el-
taining 2 or more elements.
ement A[BEG] of the sublist during the procedure. The local
variables LEFT and RIGHT will contain the boundary values of
the list of elements that have not been scanned.
Algorithm Algorithm
Algorithm II Algorithm III
QUICK(A, N, BEG, END, LOC)

3. [Scan from left to right.]
1. [Initialize.] Set LEFT := BEG, RIGHT := END and LOC := BEG.
(a) Repeat while A[LEFT] ≤ A[LOC) and LEFT 6= LOC:
2. [Scan from right to left.] LEFT := LEFT + 1.
(a) Repeat while A[LOC) ≤ A[RIGHT] and LOC 6= RIGHT: [End of loop.]
RIGHT := RIGHT - 1. (b) If LOC = LEFT, then: Return.
[End of loop.]
(c) If A[LEFT] > A[LOC], then
(b) If LOC = RIGHT, then: Return.
(c) If A[LOC] > A[RIGHT], then: (i) [Interchange A[LEFT] and A[LOC].]
TEMP := A[LOC], A[LOC] := A[LEFT], A[LEFT] := TEMP.
(i) [Interchange A[LOC) and A[RIGHT].]
TEMP := A[LOC), A[LOC] := A[RIGHT), A[RIGHT] := TEMP. (ii) Set LOC := LEFT.
(ii) Set LOC := RIGHT. (iii) Go to Step 2.
(iii) Go to Step 3.

Algorithm IV Complexity
Algorithm: This algorithm sorts an array A with N elements.

1. [Initialize.] TOP := NULL.
2. [Push boundary values of A onto stacks when A has 2 or more elements.]
If N > 1, then: TOP := TOP + 1, LOWER[1] := 1, UPPER[1] := N.
3. Repeat Steps 4 to 7 while TOP 6= NULL.
4. [Pop sublist from stacks.] Quick sort algorithm has a worst-case running time of order
Set BEG := LOWER[TOP], END := UPPER[TOP], TOP := TOP - 1.
5. Call QUICK(A, N, BEG, END, LOC). (n2 / 2),
6. [Push left sublist onto stacks when it has 2 or more elements.]
If BEG < LOC - 1, then: but an average-case running time of order (n log n).
TOP := TOP + 1, LOWER[TOP] := BEG, UPPER[TOP] = LOC - 1.
7. [Push right sublist onto stacks when it has 2 or more elements.]
If LOC + 1 < END, then:
TOP := TOP + 1, LOWER[TOP] := LOC + 1, UPPER[TOP] := END.
8. Exit.
Randomized Quick sort Randomized Quick sort
Randomized Quick sort I Randomized Quick sort II

In normal Quick sort, pivot element was always the leftmost
element in the list to be sorted. Instead of always using
In average-case behavior of Quick sort, we assume that all A[low] as pivot, we will use a randomly chosen element
permutations of the input numbers are equally likely. How- from the subarray A[low · · · high] in the randomized ver-
ever, we cannot always expect it to hold. We can add ran- sion of Quick sort. It is done by exchanging element A[low]
domization to an algorithm in order to reduce the probability with an element chosen at random from A[low · · · high].
of getting worst case in Quick sort. This ensures that the pivot element is equally likely to be
There are two ways of adding randomization in Quick sort: any of the high − low + 1 elements in the subarray.
either by randomly placing the input data in the array or by Since the pivot element is randomly chosen, we can expect
randomly choosing an element in the input data for pivot. the split of the input array to be reasonably well balanced on
The second choice is easier to analyze and implement. The average. This can help in preventing the worst-case behav-
change will only be done at the partition algorithm. ior of quick sort which occurs in unbalanced partitioning.
Even though the randomized version improves the worst
case complexity, its worst case complexity is still O(n2 ).
Quicksort Sort Count Sort
1 Introduction
Randomized Quick sort III Sorting
2 Bubble Sort
Algorithm
Complexity
One way to improve Randomized - Quick sort is to choose Modified version
the pivot for partitioning more carefully than by picking a 3 Selection Sort
random element from the array. One common approach is Algorithm
to choose the pivot as the median of a set of 3 elements Complexity
randomly selected from the array. 4 Insertion Sort
Algorithm
Complexity
5 Merge Sort
Merging
Complexity
6 Quicksort Sort
Algorithm
Count Sort Count Sort
Complexity
Count sort I
Counting sort is not a comparison sort algorithm and gives

7 Count Sort O(n) complexity for sorting.
Algorithm To achieve O(n) complexity, counting sort assumes that
Complexity each of the elements is an integer in the range 1 to K , for
some integer K .
The counting sort runs in O(n) time.
The basic idea of Counting sort is to determine, for each
8 Radix Sort
Algorithm input element X , the number of elements less than X .
Complexity This information can be used to place it directly into its cor-
rect position.

Count sort II Count sort III
Assume, A[1 · · · n] is the input array with length n.

In Counting sort we need two more arrays: array B[1 · · · n]
contains the sorted output and the array C[0 · · · K − 1] pro-
vides temporary storage.
Total Complexity: O(K ) + O(n) + O(K ) + O(n) = O(n) if

K = O(n).
Space Complexity: O(n) if K = O(n).
Note: Counting works well if K = O(n). Otherwise, the
complexity will be greater.

Count Sort Radix Sort
Complexity
1 Introduction
Important Notes I Sorting
Counting sort is efficient if the range of input data is not 2 Bubble Sort
significantly greater than the number of objects to be sorted. Algorithm
Consider the situation where the input sequence is between Complexity
range 1 to 10K and the data is 10, 5, 10K , 5K . Modified version
It is not a comparison based sorting. It running time com- 3 Selection Sort
Algorithm
plexity is O(n) with space proportional to the range of data.
Complexity
It is often used as a sub-routine to another sorting algorithm 4 Insertion Sort
like radix sort. Algorithm
Counting sort uses a partial hashing to count the occur- Complexity
rence of the data object in O(1). 5 Merge Sort
Merging
Counting sort can be extended to work for negative inputs Complexity
also. 6 Quicksort Sort
Algorithm
Radix Sort Radix Sort
Complexity
Radix sort I
Similar to Counting sort and Bucket sort, this sorting algo-

rithm also assumes some kind of information about the in-
7 Count Sort put elements.
Algorithm Suppose that the input values to be sorted are from base d.
Complexity
That means all numbers are d-digit numbers.
In Radix sort, first sort the elements based on the last digit
[the least significant digit].
8 Radix Sort These results are again sorted by second digit [the next to
Algorithm least significant digit].
Complexity Continue this process for all digits until we reach the most
significant digits.
Use some stable sort to sort them by last digit.

Radix sort II Radix sort III
Then stable sort them by the second least significant digit,

then by the third, etc.
If we use Counting sort as the stable sort, the total time is
O(nd) ≈ O(n).
Suppose 9 elements as follows: 348, 143, 361, 423, 538,
128, 321, 543, 366
Radix sort IV Radix sort V

Suppose a list A of n items A1 , A2 , · · · , An is given.

Let d denote the radix (e.g., d = 10 for decimal digits, d =
26 for letters and d = 2 for bits), and suppose each item Ai
1. Take the least significant digit of each element. is represented by means of s of the digits: Ai = di1 di2 · · · dis .
2. Sort the list of elements based on that digit, but keep the The radix sort algorithm will require s passes, the number
order of elements with the same digit (this is the definition of digits in each item.
of a stable sort). Pass K will compare each diK with each of the d digits.
3. Repeat the sort with each more significant digit. Hence the number C(n) of comparisons for the algorithm is
bounded as follows: C(n) ≤ d ∗ s ∗ n.
Although d is independent of n, the number s does depend
on n.
In the worst case, s = n, so C(n) = O(n2 ).
Complexity Complexity
Complexity II Important Notes I
The speed of Radix sort depends on the inner basic opera-

In the best case, s = logd n, so C(n) = O(n log n). tions.
In other words, radix sort performs well only when the num- If the operations are not efficient enough, Radix sort can be
ber s of digits in the representation of the Ai ’s is small. slower than other algorithms such as Quick sort and Merge
Another drawback of radix sort is that one may need d ∗ n sort.
memory locations. These operations include the insert and delete functions of
This comes from the fact that all the items may be “sent to the sub-lists and the process of isolating the digit we want.
the same pocket” during a given pass. If the numbers are not of equal length then a test is needed
This drawback may be minimized by using linked lists rather to check for additional digits that need sorting.
than arrays to store the items during a given pass. This can be one of the slowest parts of Radix sort and also
However, one will still require 2 ∗ n memory locations. one of the hardest to make efficient.
Since Radix sort depends on the digits or letters, it is less
flexible than other sorts.
Sorting Algorithms
Radix Sort
Complexity
Important Notes II
For every different type of data, Radix sort needs to be

rewritten, and if the sorting order changes, the sort needs
to be rewritten again.
In short, Radix sort takes more time to write, and it is very
difficult to write a general purpose Radix sort that can han-
dle all kinds of data.
For many programs that need a fast sort, Radix sort is a
good choice.
Still, there are faster sorts, which is one reason why Radix
sort is not used as much as some other sorts.
Time Complexity: O(nd) ≈ O(n), if d is small.
Trees Trees
Outline I
1 INTRODUCTION
2 BINARY TREES
Terminology
Complete Binary Trees
Trees Strict Binary Trees
Extended Binary Trees: 2-Trees
3 REPRESENTING BINARY TREES IN MEMORY
Linked Representation of Binary Trees
Sequential Representation of Binary Trees
4 PROPERTIES OF BINARY TREES
5 TRAVERSING BINARY TREES
6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
Inorder Traversal
Postorder Traversal
Binary Tree Formation from its Traversals
Trees Trees
Outline II Outline III

7 HEADER NODES; THREADS Deletion
Header Nodes
Threads; Inorder Threading
8 Types of Binary Tree
9 Expression Tree
10 BINARY SEARCH TREES
Searching and Inserting in Binary Search Trees 13 HEAP and HEAPSORT
Complexity of the Searching Algorithm Inserting into a Heap
Application of Binary Search Trees Deleting the Root of a Heap
Deleting in a Binary Search Tree Application to Sorting - Heapsort
11 AVL SEARCH TREES
Insertion in an AVL Search Tree
Deletion in an AVL search tree
12 RED BLACK TREES
Rotations
Insertion
Trees Trees
INTRODUCTION INTRODUCTION
1 INTRODUCTION 8 Types of Binary Tree

2 BINARY TREES 9 Expression Tree
Terminology
Complete Binary Trees 10 BINARY SEARCH TREES
Strict Binary Trees Searching and Inserting in Binary Search Trees
Complexity of the Searching Algorithm
Application of Binary Search Trees
3 REPRESENTING BINARY TREES IN MEMORY Deleting in a Binary Search Tree
Sequential Representation of Binary Trees 11 AVL SEARCH TREES
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Binary Tree Formation from its Traversals 13 HEAP and HEAPSORT
7 HEADER NODES; THREADS Inserting into a Heap
Header Nodes Deleting the Root of a Heap
Threads; Inorder Threading Application to Sorting - Heapsort
Trees Trees
INTRODUCTION BINARY TREES
1 INTRODUCTION
Introduction I 2 BINARY TREES
Terminology
Strict Binary Trees
So far, we have been studying mainly linear types of data struc-
tures: strings, arrays, lists, stacks and queues.
This part defines a nonlinear data structure called a tree. Sequential Representation of Binary Trees
This structure is mainly used to represent data containing a hier-
archical relationship between elements, e.g., records, family trees
and tables of contents. 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
First we investigate a special kind of tree, called a binary tree, Inorder Traversal
which can be easily maintained in the computer. Postorder Traversal
7 HEADER NODES; THREADS
Header Nodes
Trees Trees
BINARY TREES BINARY TREES

9 Expression Tree
Binary Trees I
Searching and Inserting in Binary Search Trees
Complexity of the Searching Algorithm A binary tree T is defined as a finite set of elements, called nodes,
Application of Binary Search Trees such that:
Deleting in a Binary Search Tree I T is empty (called the null tree or empty tree), or
11 AVL SEARCH TREES I T contains a distinguished node R, called the root of T , and
Insertion in an AVL Search Tree the remaining nodes of T form an ordered pair of disjoint
Deletion in an AVL search tree binary trees T1 and T2 .
12 RED BLACK TREES If T does contain a root R, then the two trees T1 and T2 are called,
Rotations respectively, the left and right subtrees of R.
Insertion
Deletion If T1 is nonempty, then its root is called the left successor of R;
similarly, if T2 is nonempty, then its root is called the right succes-
13 HEAP and HEAPSORT sor of R.
Inserting into a Heap
Deleting the Root of a Heap A binary tree T is frequently presented by means of a diagram.
Application to Sorting - Heapsort
Trees Trees
Binary Trees II Binary Trees III
Specifically, the diagram represents a binary tree T as follows.

(i) T consists of 11 nodes, represented by the letters A through
L, excluding I.
(ii) The root of T is the node A at the top of the diagram.
(iii) A left downward slanted line from a node N indicates a left
successor of N, and a right-downward slanted line from N
indicates a right successor of N.
Trees Trees
Binary Trees IV Binary Trees V

Observe that:
(a) B is a left successor and C is a right successor of the node
A. This means, in particular, that every node N of T contains a left
(b) The left subtree of the root A consists of the nodes B, D, E and a right subtree.
and F , and the right subtree of A consists of the nodes C,
Moreover, if N is a terminal node, then both its left and right sub-
G, H, J, K and L.
trees are empty.
Any node N in a binary tree T has either 0, 1 or 2 successors.
Binary trees T and T 0 are said to be similar if they have the same
The nodes A, B, C and H have two successors, the nodes E and structure or, in other words, if they have the same shape.
J have only one successor, and the nodes D, F , G, L and K have
no successors. The trees are said to be copies if they are similar and if they have
the same contents at corresponding nodes.
The nodes with no successors are called terminal nodes.
The above definition of the binary tree T is recursive since T is
defined in terms of the binary subtrees T1 and T2 .
Trees Trees
Terminology
1 INTRODUCTION
Binary Trees VI 2 BINARY TREES
Terminology
Strict Binary Trees
Consider the four binary trees. 4 PROPERTIES OF BINARY TREES
The three trees (a), (c) and (d) are similar. 5 TRAVERSING BINARY TREES
In particular, the trees (a) and (c) are copies since they also have Preorder Traversal
the same data at corresponding nodes. Inorder Traversal
The tree (b) is neither similar nor a copy of the tree (d) because, Postorder Traversal
in a binary tree, we distinguish between a left successor and a Binary Tree Formation from its Traversals
right successor even when there is only one successor. 7 HEADER NODES; THREADS
Header Nodes
Trees Trees
Terminology Terminology

9 Expression Tree
Terminology I
Complexity of the Searching Algorithm Terminology describing family relationships is frequently used to
Application of Binary Search Trees describe relationships between the nodes of a tree T .
Deleting in a Binary Search Tree
Specifically, suppose N is a node in T with left successor S1 and
11 AVL SEARCH TREES
right successor S2 .
Deletion in an AVL search tree Then N is called the parent (or father) of S1 and S2 .
12 RED BLACK TREES Analogously, S1 is called the left child (or son) of N, and S2 is
Rotations called the right child (or son) of N.
Insertion
Deletion Furthermore, S1 and S2 are said to be siblings (or brothers).
13 HEAP and HEAPSORT Every node N in a binary tree T , except the root, has a unique
Inserting into a Heap parent, called the predecessor of N.
Deleting the Root of a Heap
Trees Trees
Terminology II Terminology III
The terms descendant and ancestor have their usual meaning. Terminology from graph theory and horticulture is also used with
a binary tree T .
That is, a node L is called a descendant of a node N (and N is
called an ancestor of L) if there is a succession of children from Specifically, the line drawn from a node N of T to a successor is
N to L. called an edge, and a sequence of consecutive edges is called a
path.
In particular, L is called a left or right descendant of N according
to whether L belongs to the left or right subtree of N. A terminal node is called a leaf, and a path ending in a leaf is
called a branch.
Trees Trees
Terminology IV Terminology V
Each node in a binary tree T is assigned a level number, as fol-

lows.
The root R of the tree T is assigned the level number 0, and every
other node is assigned a level number which is 1 more than the
level number of its parent.
Furthermore, those nodes with the same level number are said
to belong to the same generation.
The depth (or height) of a tree T is the maximum number of nodes
in a branch of T .
This turns out to be 1 more than the largest level number of T . The tree T has depth 5.
Trees Trees
Terminology Complete Binary Trees
1 INTRODUCTION
Terminology VI 2 BINARY TREES
Terminology
Strict Binary Trees
Binary trees T and T 0 are said to be similar if they have the same 5 TRAVERSING BINARY TREES
structure or, in other words, if they have the same shape. 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
The trees are said to be copies if they are similar and if they have Inorder Traversal
the same contents at corresponding nodes. Postorder Traversal
Header Nodes
Trees Trees
Complete Binary Trees Complete Binary Trees

9 Expression Tree
Complete Binary Trees I
Complexity of the Searching Algorithm Consider any binary tree T .
Deleting in a Binary Search Tree Each node of T can have at most two children.
11 AVL SEARCH TREES Accordingly, one can show that level r of T can have at most 2r
Insertion in an AVL Search Tree nodes.
The tree T is said to be complete if all its levels, except possibly
12 RED BLACK TREES the last, have the maximum number of possible nodes, and if all
Rotations the nodes at the last level appear as far left as possible.
Insertion
Deletion Thus there is a unique complete tree Tn with exactly n nodes (we
13 HEAP and HEAPSORT are, of course, ignoring the contents of the nodes).
Inserting into a Heap Full is a special type of complete binary tree.
Trees Trees
Complete Binary Trees Complete Binary Trees
Complete Binary Trees II Complete Binary Trees III
The complete tree T2 6 with 26 nodes

Specifically, the left and right children of the node K are, respec-
tively, 2 ∗ K and 2 ∗ K + 1, and the parent of K is the node bK /2c.
For example, the children of node 9 are the nodes 18 and 19, and
its parent is the node 9/2 = 4.
The depth dn of the complete tree Tn with n nodes is given by
Dn = blog2 n + 1c.
The nodes of the complete binary tree T26 have been purposely This is a relatively small number.
labeled by the integers 1, 2, · · · , 26, from left to right, generation
For example, if the complete tree Tn has n = 1000000 nodes,
by generation.
then its depth Dn = 21.
With this labeling, one can easily determine the children and par-
ent of any node K in any complete tree Tn .
Trees Trees
Complete Binary Trees Strict Binary Trees
1 INTRODUCTION
Complete Binary Trees IV 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Strict Binary Trees Strict Binary Trees

9 Expression Tree
Strict Binary Trees I
Complexity of the Searching Algorithm A strict binary tree is called a binary tree in which each node has
Application of Binary Search Trees exactly zero or two children.
11 AVL SEARCH TREES In a strict binary tree, number of leaf nodes = No. of internal
Insertion in an AVL Search Tree nodes + 1.
12 RED BLACK TREES
Rotations
Insertion
Deletion
13 HEAP and HEAPSORT
Trees Trees
Extended Binary Trees: 2-Trees Extended Binary Trees: 2-Trees
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Extended Binary Trees: 2-Trees I Extended Binary Trees: 2-Trees II
Consider any binary tree T , such as the tree in Fig. (a).
A binary tree tree T is said to be a 2-tree or an extended binary Then T may be “converted” into a 2-tree by replacing each empty
tree if each node N has either 0 or 2 children. subtree by a new node, as pictured in Fig. (b).
In such a case, the nodes with 2 children are called internal

nodes, and the nodes with 0 children are called external nodes.
Sometimes the nodes are distinguished in diagrams by using cir-
cles for internal nodes and squares for external nodes.
The term “extended binary tree” comes from the following opera-
tion.
Trees Trees
Extended Binary Trees: 2-Trees III Extended Binary Trees: 2-Trees IV
Observe that the new tree is, indeed, a 2-tree.

Furthermore, the nodes in the original tree T are now the internal
nodes in the extended tree, and the new nodes are the external
nodes in the extended tree.
An important example of a 2-tree is the tree T corresponding to Consider any algebraic expression E involving only binary opera-
any algebraic expression E which uses only binary operations. tions, such as
E = (a − b)/((c ∗ d) + e)
E can be represented by means of the binary tree T pictured in
Fig.
Trees Trees
BINARY TREES REPRESENTING BINARY TREES IN MEMORY
1 INTRODUCTION
Extended Binary Trees: 2-Trees V 2 BINARY TREES
Terminology
Strict Binary Trees
That is, each variable, or constant in E appears as an “inter-
nal” node in T whose left and right subtrees correspond to the
operands of the operation.
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
REPRESENTING BINARY TREES IN MEMORY REPRESENTING BINARY TREES IN MEMORY

9 Expression Tree
Representing Binary Trees in Memory I
Complexity of the Searching Algorithm Let T be a binary tree.
Deleting in a Binary Search Tree Two ways of representing T in memory as follows.
11 AVL SEARCH TREES 1. The first and usual way is called the link representation of T
Insertion in an AVL Search Tree and is analogous to the way linked lists are represented in
Deletion in an AVL search tree memory.
12 RED BLACK TREES 2. The second way, which uses a single array, called the se-
Rotations quential representation of T .
Insertion
The main requirement of any representation of T is that one
Deletion
should have direct access to the root R of T and, given any node
13 HEAP and HEAPSORT N of T , one should have direct access to the children of N.
Trees Trees
Linked Representation of Binary Trees Linked Representation of Binary Trees
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Linked Representation of Binary Trees I Linked Representation of Binary Trees II
Consider a binary tree T .

Unless otherwise stated or implied, T will be maintained in mem-
ory by means of a linked representation which uses three parallel
arrays, INFO, LEFT and RIGHT, and a pointer variable ROOT as
follows.
First of all, each node N of T will correspond to a location K such
that:
(1) INFO[K ] contains the data at the node N.
(2) LEFT[K ] contains the location of the left child of node N.
(3) RIGHT[K ] contains the location of the right child of node N.
Furthermore, ROOT will contain the location of the root R of T .
If any subtree is empty, then the corresponding pointer will con-
tain the null value; if the tree T itself is empty, then ROOT will
contain the null value.
Trees Trees
Linked Representation of Binary Trees III Linked Representation of Binary Trees IV
Remark 1: Most of our examples will show a single item of infor-

mation at each node N of a binary tree T. In actual practice, an
entire record may be stored at the node N. In other words, INFO
may actually be a linear array of records or a collection of parallel
arrays.
Remark 2: Since nodes may be inserted into and deleted from
our binary trees, we also implicitly assume that the empty lo-
cations in the arrays INFO, LEFT and RIGHT form a linked list
with pointer AVAIL. We will usually let the LEFT array contain the
pointers for the AVAIL list.
Remark 3: Any invalid address may be chosen for the null pointer
denoted by NULL. In actual practice, 0 or a negative number is
used for NULL.
Trees Trees
Linked Representation of Binary Trees V Linked Representation of Binary Trees VI
Suppose we want to draw the tree diagram which corresponds to

the binary tree.
For notational convenience, we label the nodes in the tree dia-
gram only by the key values NAME.
We construct the tree as follows:
(a) The value ROOT = 14 indicates that Harris is the root of the
tree.
(b) LEFT[14] = 9 indicates that Cohen is the left child of Harris,
and RIGHT[14] = 7 indicates that Lewis is the right child of
Harris.
Repeating Step (b) for each new node in the diagram.
Trees Trees
Linked Representation of Binary Trees Sequential Representation of Binary Trees
1 INTRODUCTION
Linked Representation of Binary Trees VII 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Sequential Representation of Binary Trees Sequential Representation of Binary Trees

9 Expression Tree
Sequential Representation of Binary Trees I
Searching and Inserting in Binary Search Trees Suppose T is a binary tree that is complete or nearly complete.
Application of Binary Search Trees Then there is an efficient way of maintaining T in memory called
Deleting in a Binary Search Tree the sequential representation of T .
11 AVL SEARCH TREES This representation uses only a single linear array TREE as fol-
Insertion in an AVL Search Tree lows:
1. The root R of T is stored in TREE[1].
12 RED BLACK TREES 2. If a node N occupies TREE[K ], then its left child is stored in
Rotations TREE[2 ∗ K ] and its right child is stored in TREE[2 ∗ K + 1].
Insertion
Deletion Again, NULL is used to indicate an empty subtree.
13 HEAP and HEAPSORT In particular, TREE[1] = NULL indicates that the tree is empty.
The sequential representation of the binary tree T in Fig. (a)
appears in Fig. (b).
Trees Trees
Sequential Representation of Binary Trees Sequential Representation of Binary Trees
Sequential Representation of Binary Trees II Sequential Representation of Binary Trees III
Observe that we require 14 locations in the array TREE even

though T has only 9 nodes.
In fact, if we included null entries for the successors of the termi-
nal nodes, then we would actually require TREE[29] for the right
successor of TREE[14].
Generally speaking, the sequential representation of a tree with
depth d will require an array with approximately 2d+1 elements.
Accordingly, this sequential representation is usually inefficient
unless, as stated above, the binary tree T is complete or nearly
complete.
For example, a tree T has 11 nodes and depth 5, which means it
would require an array with approximately 26 = 64 elements.
Trees Trees
PROPERTIES OF BINARY TREES PROPERTIES OF BINARY TREES

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Properties of Binary Trees I Properties of Binary Trees II
In any binary tree, the maximum number of nodes on level r is 2r ,

where r ≥ 0. The maximum number of nodes possible in a binary tree of height
h is 2h − 1.
Proof: We prove the result by using induction on r , the number of levels.
Proof: The maximum number of nodes is possible in a binary tree if the maximum
Basis: At r =0, the number of node = 20 = 1 (Since, the root is the only node on
number of nodes are present at each level.
level r =0), hence the property is true for r =0.
Thus, the maximum number of nodes in a binary tree of height h is
Induction Hypothesis:
P max l
For r=k: Suppose the property is true at k th level. So the number of nodes at k th n = li=0 2 (where lmax is the maximum level of the tree)
level is 2k . 2lmax +1 −1
n= 2−1
= 2lmax +1 − 1 = 2h − 1
For r=k+1: Since each node in a binary tree has degree 2, so from each node at
level k , there may be at most 2 nodes at level k + 1. Since From the definition of height, we have h = lmax + 1.
Thus the maximum number of nodes at level k + 1 is 2 × 2k = 2k +1 .
Therefore, the statement is also true for k + 1.
Trees Trees
Properties of Binary Trees III Properties of Binary Trees IV
For any non-empty binary tree, if n is the number of nodes then For any non-empty binary tree, if n0 is the number of leaf nodes
the number of edges is n − 1. and n2 is the number of internal nodes having degree 2, then
Proof: n0 = n2 + 1.
Basis: Let n = 1, it implies that there is only one node in the tree. So the number Proof: Assume n1 is the total number of nodes with one child.
of edge is zero.
Now the total number of nodes n = n0 + n1 + n2
Induction Hypothesis:
And Total number of edges, e = 0 × n0 + 1 × n1 + 2 × n2 = n1 + 2n2 .
For n=k: Suppose the property is true for n = k. So the number of edges is k − 1.
We know that, e = n − 1,
For n=k+1: Now addition of one node includes one extra edge into set. So, now
Hence, n1 + 2n2 = (n0 + n1 + n2 ) − 1, which yields n0 = n2 + 1.
total edges = (k − 1) + 1 = k, Hence proved.
Trees Trees
Properties of Binary Trees V Properties of Binary Trees VI
The height of a complete binary tree with n number of nodes is

dlog2 (n + 1)e.
Proof: Let h be the height of the complete binary tree. Then we can write the 1 2n
following inequality: The total number of binary trees possible with n nodes is n+1 Cn .
n ≤ 20 + 21 + · · · + 2h−1
⇒ n ≤ 2h − 1
⇒ 2h ≥ n + 1
⇒ h = dlog2 (n + 1)e
Trees Trees
TRAVERSING BINARY TREES TRAVERSING BINARY TREES

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Traversing Binary Trees I Traversing Binary Trees II
Preorder
There are three standard ways of traversing a binary tree T with I Process the root R.
root R. These three algorithms, called preorder, inorder and pos- I Traverse the left subtree of R in preorder.
torder, are as follows: I Traverse the right subtree of R in preorder
Trees Trees
Traversing Binary Trees III Traversing Binary Trees IV
Inorder
I Traverse the left subtree of R in inorder.
I Process the root R.
The preorder traversal of T processes A, traverses LT and tra-
I Traverse the right subtree of R in inorder.
verses RT .
Again, the preorder traversal of LT processes the root B and then
D and E.
The preorder traversal of RT processes the root C and then F .
Hence A B D E C F is the preorder traversal of T .
Trees Trees
Traversing Binary Trees V Traversing Binary Trees VI
PostOrder
I Traverse the left subtree of R in postorder
I Traverse the right subtree of R in postorder.
The inorder traversal of T traverses LT , processes A and tra- I Process the root R.
verses RT .
Again, the inorder traversal of LT processes D, B and then E.
The inorder traversal of RT processes C and then F .
Hence D B E A C F is the inorder traversal of T .
Trees Trees
Traversing Binary Trees VII Traversing Binary Trees VIII
The postorder traversal of T traverses LT , traverses RT , and pro-

cesses A.
Preorder traversal: A B D E F C G H J L K
Further, the postorder traversal of LT processes D, E, and then
Inorder traversal: D B F E A G C L J H K
B.
Later, the postorder traversal of RT processes F and then C. Postorder traversal: D F E B G L J K H C A
Hence, D E B F C A is the postorder traversal of T .
Trees Trees
Traversing Binary Trees IX Traversing Binary Trees X
Observe that each algorithm contains the same three steps, and
Let E denote the following algebraic expression:
that the left subtree of R is always traversed before the right sub-
tree.
The difference between the algorithms is the time at which the
root R is processed.
Specifically, in the “pre” algorithm, the root R is processed be-
fore the subtrees are traversed; in the “in” algorithm, the root R
is processed between the traversals of the subtrees; and in the
The preorder and postorder traversals of T are as follows:
“post” algorithm, the root R is processed after the subtrees are
I (Inorder) [a + (b - c)] * [(d - e)/(f + g - h)] traversed.
I (Preorder) * + a b c / d e + f g h The three algorithms are sometimes called, respectively, the node-
I (Postorder) a b c + d e f g + h / * left-right (NLR) traversal, the left-node-right (LNR) traversal and
the left-right-node (LRN) traversal.
Trees Trees
TRAVERSING BINARY TREES TRAVERSAL ALGORITHMS USING STACKS
1 INTRODUCTION
Traversing Binary Trees XI 2 BINARY TREES
Terminology
Strict Binary Trees
Observe that each of the above traversal algorithms is recursively
defined, since the algorithm involves traversing subtrees in the
given order.
Accordingly, we will expect that a stack will be used when the 5 TRAVERSING BINARY TREES
algorithms are implemented on the computer. 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
TRAVERSAL ALGORITHMS USING STACKS TRAVERSAL ALGORITHMS USING STACKS

9 Expression Tree
Traversal Algorithms Using Stacks I
Deleting in a Binary Search Tree Suppose a binary tree T is maintained in memory by some linked
11 AVL SEARCH TREES representation
Insertion in an AVL Search Tree TREE(INFO, LEFT, RIGHT, ROOT)
Here we discuss the implementation of the three standard traver-
12 RED BLACK TREES sals of T using stacks.
Rotations
Insertion We discuss the three traversals separately.
Deletion
Trees Trees
Preorder Traversal Preorder Traversal
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Preorder Traversal I Preorder Traversal II

Algorithm: Initially push NULL onto STACK and then set PTR :=
The preorder traversal algorithm uses a variable PTR (pointer) ROOT . Then repeat the following steps until PTR = NULL or,
which will contain the location of the node N currently being scanned. equivalently, while PTR 6= NULL.
(a) Proceed down the left-most path rooted at PTR, processing

each node N on the path and pushing each right child R(N),
if any, onto STACK . The traversing ends after a node N with
no left child L(N) is processed. (Thus PTR is updated us-
ing the assignment PTR := LEFT [PTR], and the traversing
stops when LEFT [PTR] = NULL.)
(b) [Backtracking.] Pop and assign to PTR the top element on
Here L(N) denotes the left child of node N and R(N) denotes the STACK. If PTR 6= NULL, then return to Step (a); otherwise
right child. Exit.
The algorithm also uses an array STACK, which will hold the ad-
(We note that the initial element NULL on STACK is used as a
dresses of nodes for future processing.
sentinel.)
Trees Trees
Preorder Traversal III Preorder Traversal IV
We simulate the algorithm in the next example. Although the ex-

ample works with the nodes themselves, in actual practice the Consider the binary tree T . We simulate the above algorithm with
locations of the nodes are assigned to PTR and are pushed onto T , showing the contents of STACK at each step.
the STACK.
1. Initially push NULL onto STACK:
STACK: ∅.
Then set PTR := A, the root of T .
2. Proceed down the left-most path rooted at PTR = A as fol-
lows:
(i) Process A and push its right child C onto STACK:
STACK: ∅, C.
(ii) Process B. (There is no right child.)
Trees Trees
Preorder Traversal V Preorder Traversal VI

5. [Backtracking.] Pop K from STACK, and set PTR := K . This
(iii) Process D and push its right child H onto STACK: leaves:
STACK: ∅, C, H. STACK: ∅, C.
(iv) Process G. (There is no right child.) Since PTR 6= NULL, return to Step (a) of the algorithm.
No other node is processed, since G has no left child. 6. Proceed down the left-most path rooted at PTR = K as fol-
3. [Backtracking.] Pop the top element H from STACK, and set lows:
PTR := H. This leaves: (vi) Process K . (There is no right child.)
STACK: ∅, C. No other node is processed, since K has no left child.
Since PTR 6= NULL, return to Step (a) of the algorithm. 7. [Backtracking.] Pop C from STACK, and set PTR := C. This
4. Proceed down the left-most path rooted at PTR = H as fol- leaves:
lows: STACK: ∅.
(v) Process H and push its right child K onto STACK: Since PTR 6= NULL, return to Step (a) of the algorithm.
STACK: ∅, C, K . 8. Proceed down the left most path rooted at PTR = C as
No other node is processed, since H has no left child. follows:
(vii) Process C and push its right child F onto STACK:
STACK: ∅, F .
Trees Trees
Preorder Traversal VII Preorder Traversal VIII
(viiii) Process E. (There is no right child.)

9. [Backtracking.] Pop F from STACK, and set PTR := F . This
leaves:
STACK: ∅. As seen from Steps 2, 4, 6, 8 and 10, the nodes are processed
Since PTR 6= NULL, return to Step (a) of the algorithm. in the order A, B, D, G, H, K, C, E, F.
10. Proceed down the left-most path rooted at PTR = F as fol-
lows: This is the required preorder traversal of T.
(ix) Process F . (There is no right child.) A formal presentation of our preorder traversal algorithm follows:
No other node is processed, since F has no left child.
11. [Backtracking.] Pop the top element NULL from STACK, and
set PTR := NULL. Since PTR = NULL, the algorithm is
completed.
Trees Trees
Preorder Traversal Inorder Traversal
1 INTRODUCTION
Preorder Traversal IX 2 BINARY TREES
Terminology
Algorithm: PREORD(INFO, LEFT, RIGHT, ROOT)
Strict Binary Trees
A binary tree T is in memory. The algorithm does a preorder traversal of T, applying an operation PROCESS
to each of its nodes. An array STACK is used to temporarily hold the addresses of nodes. Extended Binary Trees: 2-Trees
1. [Initially push NULL onto STACK, and initialize PTR.]
Set TOP := 1, STACK [1] := NULL and PTR := ROOT . Linked Representation of Binary Trees
2. Repeat Steps 3 to 5 while PTR 6= NULL :
3. Apply PROCESS to INFO[PTR]. Sequential Representation of Binary Trees
4. [Right child?]
If RIGHT [PTR] 6= NULL, then: [Push on STACK.]
Set TOP := TOP + 1, and STACK [TOP] := RIGHT [PTR].
5. [Left child?] 6 TRAVERSAL ALGORITHMS USING STACKS
If LEFT [PTR] 6= NULL, then:
Set PTR := LEFT [PTR]. Preorder Traversal
Else: [Pop from STACK.]
Set PTR := STACK [TOP] and TOP := TOP − 1. Inorder Traversal
Postorder Traversal
6. Exit.
Header Nodes
Trees Trees
Inorder Traversal Inorder Traversal

9 Expression Tree
Inorder Traversal I
11 AVL SEARCH TREES The inorder traversal algorithm also uses a variable pointer PTR,
Insertion in an AVL Search Tree which will contain the location of the node N currently being scanned,
Deletion in an AVL search tree and an array STACK, which will hold the addresses of nodes for
12 RED BLACK TREES future processing. In fact, with this algorithm, a node is processed
Rotations only when it is popped from STACK.
Insertion
Deletion
Trees Trees
Inorder Traversal II Inorder Traversal III
Algorithm: Initially push NULL onto STACK (for a sentinel) and

then set PTR := ROOT . Then repeat the following steps until
NULL is popped from STACK.
(a) Proceed down the left-most path rooted at PTR, pushing
each node N onto STACK and stopping when a node N with Consider the binary tree T . We simulate the above algorithm with
no left child is pushed onto STACK. T , showing the contents of STACK.
(b) [Backtracking.] Pop and process the nodes on STACK. If
1. Initially push NULL onto STACK:
NULL is popped, then Exit. If a node N with a right child
STACK: ∅.
R(N) is processed, set PTR = R(N) (by assigning PTR :=
Then set PTR := A, the root of T .
RIGHT [PTR]) and return to Step (a).
2. Proceed down the left-most path rooted at PTR = A, push-
We emphasize that a node N is processed only when it is popped ing the nodes A, B, D, G and K onto STACK:
from STACK. STACK: ∅, A, B, D, G, K .
(No other node is pushed onto STACK, since K has no left
child.)
Trees Trees
Inorder Traversal IV Inorder Traversal V
3. [Backtracking.] The nodes K , G and D are popped and pro-

cessed, leaving: 6. Proceed down the left-most path rooted at PTR = M, push-
STACK: ∅, A, B. ing node M onto STACK:
(We stop the processing at D, since D has a right child.) STACK: ∅, A, B, M.
Then set PTR := H, the right child of D. (No other node is pushed onto STACK, since M has no left
4. Proceed down the left-most path rooted at PTR = H, push- child.)
ing the nodes H and L onto STACK: 7. [Backtracking.] The nodes M, B and A are popped and pro-
STACK: ∅, A, B, H, L. cessed, leaving:
(No other node is pushed onto STACK, since L has no left STACK: ∅.
child.) (No other element of STACK is popped, since A does have
5. [Backtracking.] The nodes L and H are popped and pro- a right child.) Set PTR := C, the right child of A.
cessed, leaving: 8. Proceed down the left-most path rooted at PTR = C, push-
STACK: ∅, A, B. ing the nodes C and E onto STACK:
(We stop the processing at H, since H has a right child.) STACK: ∅, C, E.
Then set PTR := M, the right child of H.
Trees Trees
Inorder Traversal VI Inorder Traversal VII

Algorithm: INORD(INFO, LEFT, RIGHT, ROOT)
A binary tree is in memory. This algorithm does an inorder traversal of T, applying an operation PROCESS
to each of its nodes. An array STACK is used to temporarily hold the addresses of nodes.
9. [Backtracking.] Node E is popped and processed. Since E
has no right child, node C is popped and processed. Since 1. [Push NULL onto STACK and initialize PTR.]
Set TOP := 1, STACK [1] := NULL and PTR := ROOT .
C has no right child, the next element, NULL, is popped from 2. Repeat while PTR 6= NULL: [Pushes left-most path onto STACK.]
STACK. (a) Set TOP := TOP + 1 and STACK [TOP] := PTR. [Saves node.]
(b) Set PTR := LEFT [PTR]. [Updates PTR.]
The algorithm is now finished, since NULL is popped from STACK. [End of loop.]
3. Set PTR := STACK [TOP] and TOP := TOP − 1. [Pops node from STACK.]
As seen from Steps 3, 5, 7 and 9, the nodes are processed in the 4. Repeat Steps 5 to 7 while PTR 6= NULL: [Backtracking.]
order K , G, D, L, H, M, B, A, E, C. This is the required inorder 6. [Right child?] If RIGHT [PTR] 6= NULL, then:
traversal of the binary tree T . (a) Set PTR := RIGHT [PTR].
(b) Go to Step 3.
A formal presentation of our inorder traversal algorithm follows: [End of If structure.]
7. Set PTR := STACK [TOP] and TOP := TOP − 1. [Pops node.]
8. Exit.
Trees Trees
Postorder Traversal Postorder Traversal
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Postorder Traversal I Postorder Traversal II
The postorder traversal algorithm is more complicated than the

Algorithm: Initially push NULL onto STACK (as a sentinel) and
preceding two algorithms, because here we may have to save a
then set PTR := ROOT . Then repeat the following steps until
node N in two different situations.
NULL is popped from STACK.
We distinguish between the two cases by pushing either N or its
(a) Proceed down the left-most path rooted at PTR. At each
negative, −N, onto STACK. (In actual practice, the location of N
node N of the path, push N onto STACK and, if N has a
is pushed onto STACK, so −N has the obvious meaning.)
right child R(N), push −R(N) onto STACK.
Again, a variable PTR (pointer) is used which contains the loca- (b) [Backtracking.] Pop and process positive nodes on STACK.
tion of the node N that is currently being scanned. If NULL is popped, then Exit. If a negative node is popped,
that is, if PTR = −N for some node N, set PTR = N (by
assigning PTR := −PTR) and return to Step (a).
We emphasize that a node N is processed only when it is popped
from STACK and it is positive.
Trees Trees
Postorder Traversal III Postorder Traversal IV
2. Proceed down the left-most path rooted at PTR = A, push-

ing the nodes A, B, D, G and K onto STACK. Furthermore,
since A has a right child C, push −C onto STACK after A
but before B, and since D has a right child H, push −H onto
STACK after D but before G. This yields:
STACK: ∅, A, −C, B, D, −H, G, K .
3. [Backtracking.] Pop and process K , and pop and process G.
Consider again the binary tree T . We simulate the above algo- Since −H is negative, only pop −H. This leaves:
rithm with T , showing the contents of STACK. STACK: ∅, A, −C, B, D.
Now PTR = −H. Reset PTR = H and return to Step (a).
1. Initially, push NULL onto STACK and set PTR := A, the root
4. Proceed down the left-most path rooted at PTR = H. First
of T :
push H onto STACK. Since H has a right child M, push −M
STACK: ∅.
onto STACK after H. Last, push L onto STACK. This gives:
STACK: ∅, A, −C, B, D, H, −M, L.
Trees Trees
Postorder Traversal V Postorder Traversal VI

5. [Backtracking.] Pop and process L, but only pop −M. This
leaves:
STACK: ∅, A, −C, B, D, H.
Now PTR = −M. Reset PTR = M and return to Step (a).
6. Proceed down the left-most path rooted at PTR = M. Now, As seen from Steps 3, 5, 7 and 9, the nodes are processed in the
only M is pushed onto STACK. This yields: order K , G, L, M, H, D, B, E, C, A.
STACK: ∅, A, −C, B, D, H, M.
7. [Backtracking.] Pop and process M, H, D and B, but only This is the required postorder traversal of the binary tree T .
pop −C. This leaves:
A formal presentation of our postorder traversal algorithm follows:
STACK: ∅, A.
Now PTR = −C. Reset PTR = C, and return to Step (a).
8. Proceed down the left-most path rooted at PTR = C. First
C is pushed onto STACK and then E, yielding:
STACK: ∅, A, C, E.
9. [Backtracking.] Pop and process E, C and A. When NULL
is popped, STACK is empty and the algorithm is completed.
Trees Trees
Postorder Traversal VII Postorder Traversal VIII
Algorithm: POSTORD(INFO, LEFT, RIGHT, ROOT)

7. Repeat while PTR > 0:
A binary tree T is in memory. This algorithm does a postorder traversal of T , applying an operation PRO-
(a) Apply PROCESS to INFO[PTR].
CESS to each of its nodes. An array STACK is used to temporarily hold the addresses of nodes. (b) Set PTR := STACK [TOP] and TOP := TOP − 1.
1. [Push NULL onto STACK and initialize PTR.] [Pops node from STACK.]
Set TOP := 1, STACK [1] := NULL and PTR := ROOT . [End of loop.]
2. [Push left-most path onto STACK.]
Repeat Steps 3 to 5 while PTR 6= NULL: 8. If PTR < 0, then:
3. Set TOP := TOP + 1 and STACK [TOP] := PTR. (a) Set PTR := −PTR.
[Pushes PTR on STACK.]
4. If RIGHT [PTR] 6= NULL, then: [Push on STACK.] (b) Go to Step 2.
Set TOP := TOP + 1 and STACK [TOP] := −RIGHT [PTR]. [End of If structure.]
5. Set PTR := LEFT [PTR]. [Updates pointer PTR.] 9. Exit.
6. Set PTR := STACK [TOP] and TOP := TOP − 1.
[Pops node from STACK,]
Trees Trees
Binary Tree Formation from its Traversals Binary Tree Formation from its Traversals
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Binary Tree Formation from its Traversals Binary Tree Formation from its Traversals
Binary Tree Formation from its Traversals I Binary Tree Formation from its Traversals II
Trees Trees
HEADER NODES; THREADS HEADER NODES; THREADS

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Header Nodes
1 INTRODUCTION
Header Nodes; Threads I 2 BINARY TREES
Terminology
Strict Binary Trees
Consider a binary tree T .
Variations of the linked representation of T are frequently used Sequential Representation of Binary Trees
because certain operations on T are easier to implement by using 4 PROPERTIES OF BINARY TREES
the modifications. 5 TRAVERSING BINARY TREES
Some of these variations, which are analogous to header and 6 TRAVERSAL ALGORITHMS USING STACKS
circular linked lists, are discussed in this part. Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Header Nodes Header Nodes

9 Expression Tree
Header Nodes I
Deleting in a Binary Search Tree Suppose a binary tree T is maintained in memory by means of a
11 AVL SEARCH TREES linked representation.
Sometimes an extra, special node, called a header node, is added
to the beginning of T .
12 RED BLACK TREES
Rotations When this extra node is used, the tree pointer variable, which we
Insertion will call HEAD (instead of ROOT), will point to the header node,
Deletion and the left pointer of the header node will point to the root of T .
Trees Trees
Header Nodes Header Nodes
Header Nodes II Header Nodes III
Suppose a binary tree T is empty.

Then T will still contain a header node, but the left pointer of
the header node will contain the null value, Thus the condition
LEFT [HEAD] = NULL will indicate an empty tree.
Another variation of the above representation of a binary tree T
is to use the header node as a sentinel.
That is, if a node has an empty subtree, then the pointer field for
the subtree will contain the address of the header node instead
of the null value.
Accordingly, no pointer will ever contain an invalid address, and
the condition LEFT [HEAD] = HEAD will indicate an empty sub-
tree.
Trees Trees
Threads; Inorder Threading Threads; Inorder Threading
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Threads; Inorder Threading I Threads; Inorder Threading II
The threads in a diagram of a threaded tree are usually indicated

Consider again the linked representation of a binary tree T . by dotted lines.
Approximately half of the entries in the pointer fields LEFT and In computer memory, an extra 1-bit TAG field may be used to dis-
RIGHT will contain null elements. tinguish threads from ordinary pointers, or, alternatively, threads
This space may be more efficiently used by replacing the null may be denoted by negative integers when ordinary pointers are
entries by some other type of information. denoted by positive integers.
Specifically, we will replace certain null entries by special pointers There are many ways to thread a binary tree T , but each thread-
which point to nodes higher in the tree. ing will correspond to a particular traversal of T .
These special pointers are called threads, and binary trees with Also, one may choose a one-way threading or a two-way thread-
such pointers are called threaded trees. ing.
The threads in a threaded tree must be distinguished in some Unless otherwise stated, our threading will correspond to the in-
way from ordinary pointers. order traversal of T .
Trees Trees
Threads; Inorder Threading III Threads; Inorder Threading IV

The one-way inorder threading of T appears in Fig.
Accordingly, in the one-way threading of T , a thread will appear
in the right field of a node and will point to the next node in the There is a thread from node E to node A, since A is accessed
inorder traversal of T ; and in the two-way threading of T , a thread after E in the inorder traversal of T .
will also appear in the LEFT field of a node and will point to the
Observe that every null right pointer has been replaced by a
preceding node in the inorder traversal of T .
thread except for the node K , which is the last node in the in-
Furthermore, the left pointer of the first node and the right pointer order traversal of T .
of the last node (in the inorder traversal of T ) will contain the null
value when T does not have a header node, but will point to the
header node when T does have a header node.
There is an analogous one-way threading of a binary tree T which
corresponds to the preorder traversal of T .
On the other hand, there is no threading of T which corresponds
to the postorder traversal of T .
Trees Trees
Threads; Inorder Threading V Threads; Inorder Threading VI

The two-way inorder threading of T appears in Fig.
The two-way inorder threading of T when T has a header node
There is a left thread from node L to node C, since L is accessed appears in Fig.
after C in the inorder traversal of T .
Here the left thread of D and the right thread of K point to the
Observe that every null left pointer has been replaced by a thread header node.
except for node D, which is the first node in the inorder traversal
of T . Otherwise the picture is the same.
All the right threads are the same.
Trees Trees
Threads; Inorder Threading VII Threads; Inorder Threading VIII

Figure shows how T may be maintained in memory by using a
linked representation.
Figure shows how the representation should be modified so that
T is a two-way inorder threaded tree using INFO[20] as a header
node.
Observe that LEFT [12] = −10, which means there is a left thread
from node F to node B.
Analogously, RIGHT [17] = −6 means there is a right thread from
node J to node H.
Last, observe that RIGHT [20] = 20, which means there is an
ordinary right pointer from the header node to itself.
If T were empty, then we would set LEFT [20] = −20, which
would mean there is a left thread from the header node to itself.
Trees Trees
Types of Binary Tree Types of Binary Tree

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Types of Binary Tree Expression Tree
1 INTRODUCTION
Types of Binary Tree I 2 BINARY TREES
Terminology
Strict Binary Trees
Expression Tree 3 REPRESENTING BINARY TREES IN MEMORY
Binary Search Tree Sequential Representation of Binary Trees
Height Balanced Tree (AVL Tree)
Red Black Tree 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
Heap Tree
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Expression Tree Expression Tree

9 Expression Tree
Expression Tree I
Application of Binary Search Trees An expression tree is binary tree which stores an arithmetic ex-
Deleting in a Binary Search Tree pression.
11 AVL SEARCH TREES External nodes contain operands and internal nodes contain op-
Insertion in an AVL Search Tree erators.
12 RED BLACK TREES An expression tree is always a binary tree since an arithmetic
Rotations expression contains either binary operators or unary operators
Insertion (Hence an internal node has at most two children).
Deletion An expression tree may not unique for an expression. But the
13 HEAP and HEAPSORT result they produce must be correct.
Trees Trees
Expression Tree II Construction of an Expression Tree I
Let E denote the following algebraic expression:
Following are the step to construct an expression tree:

I Read one symbol at a time from the postfix expression.
I Check if the symbol is an operand or operator.
I If the symbol is an operand, create a one node tree and push
a pointer onto a stack.
I If the symbol is an operator, pop two pointers from the stack
The preorder and postorder traversals of T are as follows:
namely T1 & T2 and form a new tree with root as the opera-
I (Inorder) [a + (b - c)] * [(d - e)/(f + g - h)] tor, T1 & T2 as a left and right child.
I (Preorder) * + a b c / d e + f g h I A pointer to this new tree is pushed onto the stack.
I (Postorder) a b c + d e f g + h / *
Trees Trees
Construction of an Expression Tree II Evaluation using the Expression Tree I
Trees Trees
BINARY SEARCH TREES BINARY SEARCH TREES

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Binary Search Trees I Binary Search Trees II
One of the most important data structures in computer science, a

binary search tree. Although each node in a binary search tree may contain an entire
record of data, the definition of the binary tree depends on a given
This structure enables one to search for and find an element with field whose values are distinct and may be ordered.
an average running time f (n) = O(log2 n).
Suppose T is a binary tree. Then T is called a binary search
It also enables one to easily insert and delete elements.
tree (or binary sorted tree) if each node N of T has the following
This structure contrasts with the following structures: property: The value at N is greater than every value in the left
subtree of N and is less than every value in the right subtree
(a) Sorted linear array: Here one can search for and find an
of N. (It is not difficult to see that this property guarantees that
element with a running time f (n) = O(log2 n), but it is expen-
the inorder traversal of T will yield a sorted listing of the elements
sive to insert and delete elements.
of T .)
(b) Linked list: Here one can easily insert and delete elements,
but it is expensive to search for and find an element, since
one must use a linear search with running time f (n) = O(n).
Trees Trees
Binary Search Trees III Binary Search Trees IV
The definition of a binary search tree given here assumes that all
the node values are distinct.
There is an analogous definition of a binary search tree which
admits duplicates, that is, in which each node N has the following
property: The value at N is greater than every value in the left
subtree of N and is less than or equal to every value in the
right subtree of N.
Trees Trees
Searching and Inserting in Binary Search Trees Searching and Inserting in Binary Search Trees
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Searching and Inserting in Binary Search Trees I Searching and Inserting in Binary Search Trees II
Suppose an ITEM of information is given. The following algorithm

finds the location of ITEM in the binary search tree T , or inserts
ITEM as a new node in its appropriate place in the tree.
a Compare ITEM with the root node N of the tree.
Suppose T is a binary search tree.
(i) If ITEM < N, proceed to the left child of N.
This part discusses the basic operations of searching and insert- (ii) If ITEM > N, proceed to the right child of N.
ing with respect to T . b Repeat Step (a) until one of the following occurs:
(i) We meet a node N such that ITEM = N. In this case the
In fact, the searching and inserting will be given by a single search search is successful.
and insertion algorithm. (ii) We meet an empty subtree, which indicates that the search
is unsuccessful, and we insert ITEM in place of the empty
subtree.
In other words, proceed from the root R down through the tree T
until finding ITEM in T or inserting ITEM as a terminal node in T .
Trees Trees
Searching and Inserting in Binary Search Trees III Searching and Inserting in Binary Search Trees IV
Consider the binary search tree T . Suppose ITEM = 20 is given.

Simulating the above algorithm, we obtain the following steps:
1. Compare ITEM = 20 with the root, 38, of the tree T . Since
20 < 38, proceed to the left child of 38, which is 14.
2. Compare ITEM = 20 with 14. Since 20 > 14, proceed to
the right child of 14, which is 23.
3. Compare ITEM = 20 with 23. Since 20 < 23, proceed to
the left child of 23, which is 18.
4. Compare ITEM = 20 with 18. Since 20 > 18 and 18 does
not have a right child, insert 20 as the right child of 18.
5. Figure shows the new tree with ITEM = 20 inserted. The
shaded edges indicate the path down through the tree dur-
ing the algorithm.
Trees Trees
Searching and Inserting in Binary Search Trees V Searching and Inserting in Binary Search Trees VI
Suppose the following six numbers are inserted in order into an
empty binary search tree: 40, 60, 50, 33, 55, 11
Figure shows the six stages of the tree. We emphasize that if the
six numbers were given in a different order, then the tree might The formal presentation of our search and insertion algorithm will
be different and we might have a different depth. use the following procedure, which finds the locations of a given
ITEM and its parent.
The procedure traverses down the tree using the pointer PTR and
the pointer SAVE for the parent node.
This procedure will also be used in the next section, on deletion.
Trees Trees
Searching and Inserting in Binary Search Trees VII Searching and Inserting in Binary Search Trees VIII
1. [Tree empty?]
If ROOT = NULL, then: Set LOC := NULL and PAR := NULL, and Return.
2. [ITEM at root?]
If ITEM = INFO[ROOT ], then: Set LOC := ROOT and PAB := NULL, and Return.
FIND(INFO, LEFT, RIGHT, ROOT, ITEM, LOC, PAR) 3. [Initialize pointers PTR and SAVE.]
If ITEM < INFO[ROOT ], then:
Set PTR := LEFT [ROOT ] and SAVE := ROOT .
A binary search tree T is in memory and an ITEM of information is given. This procedure finds the location
Else:
LOC of ITEM in T and also the location PAR of the parent of ITEM. There are three special cases: Set PTR := RIGHT [ROOT ] and SAVE := ROOT .
(i) LOC = NULL and PAR = NULL will indicate that the tree is empty. 4. Repeat Steps 5 and 6 while PTR 6= NULL:
(ii) LOC =
6 NULL and PAR = NULL will indicate that ITEM is the root of T . 5. [ITEM found?]
If ITEM = INFO[PTR], then: Set LOC := PTR and PAR := SAVE, and Return.
(iii) LOC = NULL and PAR 6= NULL will indicate that ITEM is not in T and can be added to T as a 6. If ITEM < INFO[PTR], then:
Set SAVE := PTR and PTR := LEFT [PTR].
child of the node N with location PAR. Else:
Set SAVE := PTR and PTR := RIGHT [PTR].
7. [Search unsuccessful.] Set LOC := NULL and PAR := SAVE.
8. Exit.
Trees Trees
Searching and Inserting in Binary Search Trees IX Searching and Inserting in Binary Search Trees X
INSBST(INFO, LEFT, RIGHT, ROOT, AVAIL, ITEM, LOC)

A binary search tree T is in memory and an ITEM of information is given. This algorithm finds the location
LOC of ITEM in T or adds ITEM as a new node in T at location LOC.
1. Call FIND(INFO, LEFT, RIGHT, ROOT, ITEM,LOC, PAR).

2. If LOC 6= NULL, then Exit.
Observe that, in Step 6, we move to the left child or the right child
3. [Copy ITEM into new node in AVAIL list.]
according to whether ITEM < INFO[PTR] or ITEM > INFO[PTR]. (a) If AVAIL = NULL, then: Write: OVERFLOW , and Exit.
(b) Set NEW := AVAIL, AVAIL := LEFT [AVAIL] and INFO[NEW ] := ITEM.
The formal statement of our search and insertion algorithm fol- (c) Set LOC := NEW , LEFT [NEW ] := NULL and RIGHT [NEW ] := NULL.
lows. 4. [Add ITEM to tree.]
If PAR = NULL, then:
Set ROOT := NEW .
Else if ITEM < INFO[PAR], then:
Set LEFT [PAR] := NEW .
Else:
Set RIGHT [PAR] := NEW .
5. Exit.
Trees Trees
1 INTRODUCTION
Searching and Inserting in Binary Search Trees XI 2 BINARY TREES
Terminology
Strict Binary Trees
Observe that, in Step 4, there are three possibilities: Linked Representation of Binary Trees
1. the tree is empty,
2. ITEM is added as a left child and
3. ITEM is added as a right child. 5 TRAVERSING BINARY TREES
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees

9 Expression Tree
Complexity of the Searching Algorithm I
Searching and Inserting in Binary Search Trees Suppose we are searching for an item of information in a binary
Complexity of the Searching Algorithm search tree T .
Deleting in a Binary Search Tree Observe that the number of comparisons is bounded by the depth
of the tree.
11 AVL SEARCH TREES
Insertion in an AVL Search Tree This comes from the fact that we proceed down a single path of
Deletion in an AVL search tree the tree.
12 RED BLACK TREES Accordingly, the running time of the search will be proportional to
Rotations the depth of the tree.
Insertion
Deletion Suppose we are given n data items, A1 , A2 , · · · , AN , and suppose
the items are inserted in order into a binary search tree T .
Inserting into a Heap Recall that there are n! permutations of the n items.
Each such permutation will give rise to a corresponding tree.
Trees Trees
1 INTRODUCTION
Complexity of the Searching Algorithm II 2 BINARY TREES
Terminology
Strict Binary Trees
It can be shown that the average depth of the n! trees is approxi-
mately c log2 n, where c = 1.4.
Accordingly, the average running time f (n) to search for an item 4 PROPERTIES OF BINARY TREES
in a binary tree T with n elements is proportional to log2 n, that is, 5 TRAVERSING BINARY TREES
f (n) = O(log2 n). 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees

9 Expression Tree
Application of Binary Search Trees I
Complexity of the Searching Algorithm Consider a collection of n data items, A1 , A2 , · · · , AN .
Deleting in a Binary Search Tree Suppose we want to find and delete all duplicates in the collec-
11 AVL SEARCH TREES tion.
Insertion in an AVL Search Tree One straightforward way to do this is as follows:
Algorithm: Scan the elements from A1 to AN (that is, from left to
12 RED BLACK TREES right).
Rotations
Insertion (a) For each element AK compare AK with A1 , A2 , · · · , AK −1 ,
Deletion that is, compare AK with those elements which precede AK .
(b) If AK does occur among A1 , A2 , · · · , AK −1 , then delete AK .
Inserting into a Heap After all elements have been scanned, there will be no duplicates.
Trees Trees
Application of Binary Search Trees II Application of Binary Search Trees III
Using a binary search tree, we can give another algorithm to find

Suppose Algorithm is applied to the following list of 15 numbers: the duplicates in the set A1 , A2 , · · · , AN of n data items.
14, 10, 17, 12, 10, 11, 20, 12, 18, 25, 20, 8, 22, 11, 23 Algorithm: Build a binary search tree T using the elements A1 ,
The output is : A2 , · · · , AN . In building the tree, delete AK from the list whenever
the value of AK already appears in the tree.
14, 10, 17, 12, 11, 20, 18, 25, 8, 22, 23
The main advantage of Algorithm is that each element AK is com-
Algorithm requires :
pared only with the elements in a single branch of the tree.
0 + 1 + 2 + 3 + 2 + 4 + 5 + 4 + 6 + 7 + 6 + 8 + 9 + 5 + 10 = 72
comparisons. Algorithm requires :
0 + 1 + 1 + 2 + 2 + 3 + 2 + 3 + 3 + 3 + 3 + 2 + 4 + 4 + 5 = 38
Trees Trees
Searching and Inserting in Binary Search Trees Deleting in a Binary Search Tree
1 INTRODUCTION
Application of Binary Search Trees IV 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Deleting in a Binary Search Tree Deleting in a Binary Search Tree

9 Expression Tree
Deleting in a Binary Search Tree I
Application of Binary Search Trees Suppose T is a binary search tree, and suppose an ITEM of in-
Deleting in a Binary Search Tree formation is given.
11 AVL SEARCH TREES
This section gives an algorithm which deletes ITEM from the tree
T.
12 RED BLACK TREES The deletion algorithm first uses Procedure to find the location
Rotations of the node N which contains ITEM and also the location of the
Insertion parent node P(N).
Deletion The way N is deleted from the tree depends primarily on the num-
13 HEAP and HEAPSORT ber of children of node N.
Trees Trees
Deleting in a Binary Search Tree II Deleting in a Binary Search Tree III
There are three cases:

1. Case 1: N has no children. Then N is deleted from T by
simply replacing the location of N in the parent node P(N)
by the null pointer. Observe that the third case is much more complicated than the
2. Case 2: N has exactly one child. Then N is deleted from T first two cases.
by simply replacing the location of N in P(N) by the location
In all three cases, the memory space of the deleted node N is
of the only child of N.
returned to the AVAIL list.
3. Case 3: N has two children. Let S(N) denote the inorder
successor of N. (You can verify that S(N) does not have a
left child.) Then N is deleted from T by first deleting S(N)
from T (by using Case 1 or Case 2) and then replacing node
N in T by the node S(N).
Trees Trees
Deleting in a Binary Search Tree IV Deleting in a Binary Search Tree V

Suppose we delete node 44 from the tree T. Note that node 44
Consider the binary search tree in Fig. (a) and suppose T ap- has no children. Fig.(a) pictures the tree after 44 is deleted, and
pears in memory as in Fig. (b). Fig. (b) shows the linked representation in memory. The deletion
is accomplished by simply assigning NULL to the parent node,
33. (The shading indicates the changes.)
Trees Trees
Deleting in a Binary Search Tree VI Deleting in a Binary Search Tree VII
Suppose we delete node 75 from the tree T instead of node 44.

Note that node 75 has only one child. Fig. (a) pictures the tree
after 75 is deleted, and Fig. (b) shows the linked representa-
tion. The deletion is accomplished by changing the right pointer
of the parent node 60, which originally pointed to 75, so that it
now points to node 66, the only child of 75. (The shading indi-
cates the changes.)
Trees Trees
Deleting in a Binary Search Tree VIII Deleting in a Binary Search Tree IX
Suppose we delete node 25 from the tree T instead of node 44 or

node 75. Note that node 25 has two children. Also observe that
node 33 is the inorder successor of node 25. Fig. (a) pictures the
tree after 25 is deleted, and Fig. (b) shows the linked represen-
tation. The deletion is accomplished by first deleting 33 from the
tree and then replacing node 25 by node 33. We emphasize that
the replacement of node 25 by node 33 is executed in memory
only by changing pointers, not by moving the contents of a node
from one location to another. Thus 33 is still the value of INFO[1].
Trees Trees
Deleting in a Binary Search Tree X Deleting in a Binary Search Tree XI

Our deletion algorithm will be stated in terms of Procedures A Procedure A: CASEA(INFO, LEFT, RIGHT, ROOT, LOC, PAR)
and B. This procedure deletes the node N at location LOC, where N does not have two children. The pointer PAR
gives the location of the parent of N, or else PAR = NULL indicates that N is the root node. The pointer
The first procedure refers to Cases 1 and 2, where the deleted CHILD gives the location of the only child of N, or else CHILD = NULL indicates N has no children.
node N does not have two children; and the second procedure 1. [Initializes CHILD.]
If LEFT [LOC] = NULL and RIGHT [LOC] = NULL, then:
refers to Case 3, where N does have two children. Set CHILD := NULL.
Else if LEFT [LOC] 6= NULL, then:
There are many subcases which reflect the fact that N may be a Set CHILD := LEFT [LOC].
Else:
left child, a right child or the root. Set CHILD := RIGHT [LOC].
Also, in Case 2, N may have a left child or a right child. 2. If PAR 6= NULL, then:
If LOC = LEFT [PAR], then:
Set LEFT [PAR] := CHILD.
Procedure B treats the case that the deleted node N has two Else:
Set RIGHT [PAR] := CHILD.
children. [End of If structure.]
Else:
We note that the inorder successor of N can be found by moving Set ROOT := CHILD.
to the right child of N and then moving repeatedly to the left until 3. Return.
meeting a node with an empty left subtree.
Trees Trees
Deleting in a Binary Search Tree XII Deleting in a Binary Search Tree XIII
Procedure B: CASEB(INFO, LEFT, RIGHT, ROOT, LOC, PAR)

3. [Replace node N by its inorder successor.]
This procedure will delete the node N at location LOC, where N has two children. The pointer PAR gives
the location of the parent of N, or else PAR = NULL indicates that N is the root node. The pointer SUC (a) If PAR 6= NULL, then:
gives the location of the inorder successor of N, and PARSUC gives the location of the parent of the inorder If LOC = LEFT [PAR], then:
successor. Set LEFT [PAR] := SUC.
Else:
Set RIGHT [PAR] := SUC.
1. [Find SUC and PARSUC.] [End of If structure.]
(a) Set PTR := RIGHT [LOC] and SAVE := LOC. Else:
(b) Repeat while LEFT [PTR] 6= NULL: Set ROOT := SUC.
Set SAVE := PTR and PTR := LEFT [PTR]. [End of If structure.]
[End of loop.] (b) Set LEFT [SUC] := LEFT [LOC] and RIGHT [SUC] := RIGHT [LOC].
(c) Set SUC := PTR and PARSUC := SAVE.
4. Return.
2. [Delete inorder successor, using Procedure A.]
Call CASEA(INFO, LEFT, RIGHT, ROOT, SUC, PARSUC).
Trees Trees
Deleting in a Binary Search Tree XIV Deleting in a Binary Search Tree XV
Algorithm : DEL(INFO, LEFT, RIGHT, ROOT, AVAIL, ITEM)

A binary search tree T is in memory, and an ITEM of information is given. This algorithm deletes ITEM from
the tree.
1. [Find the locations of ITEM and its parent, using Procedure FIND.]
Call FIND(INFO, LEFT, RIGHT, ROOT, ITEM, LOC, PAR).
We can now formally state our deletion algorithm, using Proce- 2. [ITEM in tree?]
If LOC = NULL, then: Write: ITEM not in tree, and Exit.
dures A and B as building blocks. 3. [Delete node containing ITEM.]
If RIGHT [LOC] 6= NULL and LEFT [LOC] 6= NULL, then:
Call CASEB(INFO, LEFT, RIGHT, ROOT, LOC, PAR).
Else:
Call CASEA(INFO, LEFT, RIGHT, ROOT, LOC, PAR).
4. [Return deleted node to the AVAIL list.]
Set LEFT [LOC] := AVAIL and AVAIL := LOC.
5. Exit.
Trees Trees
AVL SEARCH TREES AVL SEARCH TREES

Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
AVL Search Tree I AVL Search Tree II
Consider the elements A, B, C, D, ..., Z to be inserted into a binary

search tree as shown in Fig. (a).
Observe how the binary search tree turns out to be right skewed.
Again the insertion of elements Z, Y, X, ..., C, B, A in that order in
One of the more popular balanced trees was introduced in 1962
the binary search tree results in a left skewed binary search tree
by AdelsonVelskii and Landis and was known as AVL trees.
(Fig. (b)).
The disadvantage of a skewed binary search tree is that the worst
case time complexity of a search is O(n).
Therefore there arises the need to maintain the binary search tree
to be of balanced height.
By doing so it is possible to obtain for the search operation a time
complexity of O(log n) in the worst case.
Trees Trees
AVL Search Tree III Definition I
An empty binary tree is an AVL tree. A non empty binary tree T is

an AVL tree iff given T L and T R to be the left and right subtrees of
T and h(T L ) and h(T R ) to be the heights of subtrees T L and T R
respectively, T L and T R are AVL trees and |h(T L ) − h(T R )| ≤ 1.
h(T L )−h(T R ) is known as the balance factor (BF ) and for an AVL
tree the balance factor of a node can be either 0, 1, or −1.
An AVL search tree is a binary search tree which is an AVL tree.
Trees Trees
Representation of an AVL Search Tree I Representation of an AVL Search Tree II
AVL search trees like binary search trees are represented using a
linked representation. However, every node registers its balance fac-
tor. Fig. shows the representation of an AVL search tree. The number
against each node represents its balance factor.
Trees Trees
Searching an AVL Search Tree I Balancing Trees I
Whenever we insert a node into a tree or delete a node from a

tree, the resulting tree may be unbalanced.
When we detect that a tree has become unbalanced, we must
Searching an AVL search tree for an element is exactly similar to the
rebalance it.
method used in a binary search tree
AVL trees are balanced by rotating nodes either to the left or to
the right.
We consider four cases that require rebalancing.
All unbalanced trees fall into one of these four cases:
Trees Trees
Balancing Trees II Balancing Trees III
Case 1: Unbalance occurs due to the insertion in the left sub tree of
the left child of the pivot node.
Trees Trees
Balancing Trees IV Balancing Trees V
Case 2: Unbalance occurs due to the insertion in the right sub tree of
the right child of the pivot node.
Trees Trees
Balancing Trees VI Balancing Trees VII
Case 3: Unbalance occurs due to the insertion in the right sub tree of
the left child of the pivot node.
Trees Trees
Balancing Trees VIII Balancing Trees IX
Case 4: Unbalance occurs due to the insertion in the left sub tree of
the right child of the pivot node.
Trees Trees
Insertion in an AVL Search Tree Insertion in an AVL Search Tree
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Insertion in an AVL Search Tree I Rotations I
Inserting an element in its first phase is similar to binary search Based on the position of the inserted node with reference to A, the
tree. rotation are classified as follows:
After insertion the balance factor of node is affected. LL rotation: Inserted node is in the left subtree of left subtree of
Rotations is used to restore the balance of the search tree. node A
To perform rotations: RR rotation: Inserted node is in the right subtree of right subtree
of node A
I It is necessary to identify a specific node A whose BF (A) is
neither 0, 1 or -1. LR rotation: Inserted node is in the right subtree of left subtree of
I Which is the nearest ancestor to the inserted node on the node A
path from the inserted node to the root. RL rotation: Inserted node is in the left subtree of right subtree of
All nodes on the path from the inserted node to A will have their node A
balance factors to be either 0, 1, or -1.
Trees Trees
Rotations II LL Rotation
Inserted node is in the left subtree of left subtree of node A.
Amongst the rotations, LL and RR rotations are called as single ro-

tations and LR and RL are known as double rotations since, LR can
be accomplished by RR followed by LL rotation and RL can be ac-
complished by LL followed by RR rotation. Each of the rotations is
explained with an example.
Trees Trees
LL Rotation Example RR Rotation

Insert 36 into the AVL search tree. Inserted node is in the right subtree of right subtree of node A.
Trees Trees
RR Rotation Example LR Rotation

Insert 65 into the AVL search tree. Inserted node is in the right subtree of left subtree of node A.
Trees Trees
LR Rotation Example RL Rotation

Insert 37 into the AVL search tree. Inserted node is in the left subtree of right subtree of node A.
Trees Trees
RL Rotation Example Insertion Example I

Insert 92 into the AVL search tree.
Construct AVL tree by inserting following elements 64, 1, 14, 26, 13,
110, 98, 85.
Trees Trees
Insertion Example II Insertion Example III
Trees Trees
Insertion in an AVL Search Tree Deletion in an AVL search tree
1 INTRODUCTION
Important Notes 2 BINARY TREES
Terminology
Strict Binary Trees
The time complexity of an insertion operation in an AVL tree is
given by O(height) = O(log n).
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Deletion in an AVL search tree Deletion in an AVL search tree

9 Expression Tree
In case of unbalance due to deletion, one or more rotations need
to be applied to balance the AVL tree.
11 AVL SEARCH TREES
Insertion in an AVL Search Tree On deletion of a node X from the AVL tree, let A be the closest
Deletion in an AVL search tree ancestor node on the path from X to the root node, with a balance
factor of +2 or -2.
12 RED BLACK TREES
Rotations To restore balance the rotation is first classified as L or R depend-
Insertion ing on whether the deletion occurred on the left or right subtree
Deletion of A.
Trees Trees
Rotations R0 Rotation
Depending on the value of BF(B) where B is the root of the left or

right subtree of A, the R or L imbalance is further classied as:
I R0 Rotation
I R1 Rotation
I R-1 Rotation
I L0 Rotation
I L1 Rotation
I L-1 Rotation
The L rotations are mirror images of R rotations.
LL and R0 rotations are identical.
Though LL and R1 are also identical they yield different balance
factors.
LR and R-1 are identical.
Trees Trees
R0 Rotation Example R1 Rotation

Delete 60 from the AVL search tree.
Trees Trees
R1 Rotation Example R-1 Rotation

Delete 39 from the AVL search tree.
Trees Trees
R-1 Rotation Example Deletion Example I

Delete 52 from the AVL search tree. Given the AVL search tree delete the listed elements: 120, 64, 130
Trees Trees
AVL SEARCH TREES RED BLACK TREES
1 INTRODUCTION
Deletion Example II 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
RED BLACK TREES RED BLACK TREES

9 Expression Tree
Red Black Trees I
Deleting in a Binary Search Tree Red-black trees are one of many search-tree schemes that are
“balanced” in order to guarantee that basic dynamic-set opera-
11 AVL SEARCH TREES
tions take O(logn) time in the worst case.
Deletion in an AVL search tree A red-black tree is a binary search tree with one extra bit of stor-
12 RED BLACK TREES age per node: its color, which can be either RED or BLACK.
Rotations
Each node of the tree now contains the attributes color, key, left,
Insertion
right, and p.
Deletion
Trees Trees
Properties of Red Black Trees I Properties of Red Black Trees II
A red-black tree is a binary tree that satisfies the following red-

black properties:
1. Every node is either red or black
2. The root and every leaf (NIL) is black.
3. If a node is red, then both its children are black. In other
hand, if a node is red, then its parent is black.
4. For each node, all simple paths from the node (χ) to de-
scendant leaves contain the same number of black nodes =
black-height(χ). Every node is either red or black.
The root and leaves (NIL’s) are black.
If a node is red, then its parent is black.
Trees Trees
Properties of Red Black Trees III

Lemma: A red-black tree with n internal nodes has height at most
2 log(n + 1)
Proof :
All simple paths from any node χ to a descendant leaf have the
same number of black nodes = black-height(χ).
Trees Trees
Merge red nodes into their black parents.

This process produces a tree in which each node has 2, 3, or 4
children.
The 2-3-4 tree has uniform depth h’ of leaves.
We have h0 ≥ h/2, since at most half the leaves on any path are
red.
Trees Trees
We start by showing that the subtree rooted at any node x contains at

least 2bh(x) - 1 internal nodes.
We prove this claim by induction on the height of x. If the height of x
is 0, then x must be a leaf (T.nil), and the subtree rooted at x indeed
contains at least 2bh(x) − 1 = 20 − 1 = 0 internal nodes.
The number of leaves in each tree is n + 1 For the inductive step, consider a node x that has positive height and
=⇒ n + 1 ≥ 2h0 is an internal node with two children. Each child has a black-height
of either bh(x) or bh(x) − 1, depending on whether its color is red or
=⇒ log(n + 1) ≥ h0 ≥ h/2
black, respectively.
=⇒ h ≤ 2 log(n + 1). Since the height of a child of x is less than the height of x itself, we
can apply the inductive hypothesis to conclude that each child has at
least 2bh(x) − 1 internal nodes. Thus, the subtree rooted at x contains
at least (2bh(x)−1 − 1) + (2bh(x)−1 − 1) + 1 = 2bh(x) − 1 internal nodes,
which proves the claim
To complete the proof of the lemma, let h be the height of the tree. Ac-
cording to property 4, at least half the nodes on any simple path from
the root to a leaf, not including the root, must be black. Consequently,
the black-height of the root must be at least h = 2; thus, n ≥ 2h/2 − 1.
Trees Trees
As an immediate consequence of this lemma, we can implement the

Moving the 1 to the left-hand side and taking logarithms on both sides dynamic-set operations SEARCH, MINIMUM, MAXIMUM, SUCCES-
yields log(n + 1) ≥ h/2, or h ≤ 2 log(n + 1). SOR, and PREDECESSOR in O(log n) time on red-black trees, since
each can run in O(h) time on a binary search tree of height h and any
red-black tree on n nodes is a binary search tree with height O(log n).
Trees Trees
Rotations
1 INTRODUCTION
Modifying operations I 2 BINARY TREES
Terminology
Strict Binary Trees
The operations INSERT and DELETE cause modifications to the Linked Representation of Binary Trees
red-black tree: Sequential Representation of Binary Trees
I the operation itself,
I color changes, 5 TRAVERSING BINARY TREES
I restructuring the links of the tree via “rotations”. 6 TRAVERSAL ALGORITHMS USING STACKS
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Rotations Rotations

9 Expression Tree
Rotations I
Deleting in a Binary Search Tree The search-tree operations TREE-INSERT and TREE-DELETE,
11 AVL SEARCH TREES when run on a red-black tree with n keys, take O(log n) time. Be-
Insertion in an AVL Search Tree cause they modify the tree, the result may violate the red-black
Deletion in an AVL search tree properties. To restore these properties, we must change the col-
12 RED BLACK TREES ors of some of the nodes in the tree and also change the pointer
Rotations structure.
Insertion We change the pointer structure through rotation.
Deletion
Trees Trees
Rotations Rotations
Rotations II Rotations III

Below given figure shows the two kinds of rotations: left rotations
and right rotations
When we do a left rotation on a node x, we assume that its right

child y is not T:nil; x may be any node in the tree whose right child
is not T:nil. The left rotation “pivots” around the link from x to y.
It makes y the new root of the subtree, with x as y’s left child and
y’s left child as x’s right child.
Rotations maintain the inorder ordering of keys:

a ∈ α, b ∈ β, c ∈ γ ⇒ a ≤ A ≤ b ≤ B ≤ c.
A rotation can be performed in O(1) time.
Trees Trees
Rotations Rotations
Rotations IV Rotations V
The pseudocode for LEFT-ROTATE assumes that x:right 6=T:nil

and that the root’s parent is T:nil.
Below figure shows an example of how LEFT-ROTATE modifies

a binary search tree. The code for RIGHT-ROTATE is symmetric.
Both LEFT-ROTATE and RIGHT- ROTATE run in O(1) time. Only
pointers are changed by a rotation; all other attributes in a node
remain the same.
Trees Trees
Rotations Insertion
1 INTRODUCTION
Rotations VI 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Insertion Insertion

9 Expression Tree
Insertion I
10 BINARY SEARCH TREES IDEA:
Insert x in tree.
Application of Binary Search Trees Color x red.
Only red-black property 3 might be violated.
11 AVL SEARCH TREES
Insertion in an AVL Search Tree Move the violation up the tree by recoloring until it can be fixed
Deletion in an AVL search tree with rotations and recoloring.
12 RED BLACK TREES
Rotations
Insertion
Deletion
Trees Trees
Insertion Insertion
Insertion II Insertion III
Insert x =15. Recolor, moving the violation up the tree.
Trees Trees
Insertion Insertion
Insertion IV Insertion V
RIGHT-ROTATE(18). LEFT-ROTATE(7) and recolor.

Trees Trees
Insertion Insertion
Insertion VI Insertion VII
Pseudocode:
Graphical notation:
Trees Trees
Insertion Insertion
Insertion VIII Insertion IX
Case 1:
Case 2:
Trees Trees
Insertion Insertion
Insertion X Insertion XI
Case 3:
Go up the tree performing Case 1, which only recolors nodes.

If Case 2 or Case 3 occurs, perform 1 or 2 rotations, and termi-
nate.
Running time: O(lg n) with O(1) rotations.
Trees Trees
Insertion Insertion
Insertion I Insertion II
We can insert a node into an n-node red-black tree in O(logn) time.

The call RB-INSERT(T,z)inserts node z, whose key is assumed to
have already been filled in, into the red-black tree T.
Trees Trees
Insertion Insertion
Insertion III Insertion IV
The procedures TREE-INSERT and RB-INSERT differ in four ways.

First, all instances of NIL in TREE-INSERT are replaced by T:nil.
Second, we set z.left and z.right to T:nil in lines 1415 of RB-
INSERT, in order to maintain the proper tree structure.
Third, we color z red in line 16.
Fourth, because coloring z red may cause a violation of one of
the red-black properties, we call RB-INSERT-FIXUP.T(T,z) in line
17 of RB-INSERT to restore the red-black properties.
Trees Trees
Insertion Insertion
Insertion V The operation of RB-INSERT-FIXUP.
(a) A node z after insertion. Because both z and its parent z,

p are red, a violation of property 4 occurs. Since zs uncle y is
red, case 1 in the code applies. We recolor nodes and move the
pointer z up the tree, resulting in the tree shown in above figure
(b).
Once again, z and its parent are both red, but zs uncle y is black.
Since is the right child of z, p, case 2 applies. We perform a left
rotation, and the tree that results is shown in (c).
Now, z is the left child of its parent, and case 3 applies. Re-
coloring and right rotation yield the tree in (d), which is a legal
red-black tree.
Trees Trees
Deletion Deletion
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Deletion Deletion
Deletion I Deletion II
Like the other basic operations on an n-node red-black tree, deletion

of a node takes time O(logn). Deleting a node from a red-black tree is
a bit more complicated than inserting a node.
Trees Trees
RED BLACK TREES HEAP and HEAPSORT
Deletion
1 INTRODUCTION
Deletion III 2 BINARY TREES
Terminology
Strict Binary Trees
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
HEAP and HEAPSORT HEAP and HEAPSORT

9 Expression Tree
HEAP and HEAPSORT I
Searching and Inserting in Binary Search Trees Suppose H is a complete binary tree with n elements.
Application of Binary Search Trees Then H is called a heap, or a max-heap, if each node N of H has
Deleting in a Binary Search Tree the following property:
11 AVL SEARCH TREES I The value at N is greater than or equal to the value at each
Insertion in an AVL Search Tree of the children of N.
Deletion in an AVL search tree I Accordingly, the value at N is greater than or equal to the
12 RED BLACK TREES value at any of the descendants of N.
Rotations A minheap is defined analogously: The value at N is less than or
Insertion equal to the value at any of the children of N.
Deletion
13 HEAP and HEAPSORT Unless otherwise stated, we assume that H is maintained in mem-
Inserting into a Heap ory by a linear array TREE using the sequential representation of
Deleting the Root of a Heap H, not a linked representation.
Trees Trees
Inserting into a Heap Inserting into a Heap
Terminology
12 RED BLACK TREES
Rotations
Preorder Traversal
Insertion
Inorder Traversal
Deletion
Postorder Traversal
Trees Trees
Inserting into a Heap Inserting into a Heap - Example I

Suppose we want to add ITEM = 70 to given Heap H.
Suppose H is a heap with N elements, and suppose an ITEM of infor-

mation is given. We insert ITEM into the heap H as follows:
1. First adjoin ITEM at the end of H so that H is still a complete tree,
but not necessarily a heap.
2. Then let ITEM rise to its “appropriate place” in H so that H is
finally a heap.
Trees Trees
Inserting into a Heap - Example II Inserting into a Heap - Example III
Suppose we want to build a heap H from the following list of numbers:

44, 30, 50, 22, 60, 55, 77, 55
Trees Trees
Inserting into a Heap - Example IV Inserting into a Heap - Algorithm I

INSHEAP(TREE, N, ITEM): A heap H with N elements is stored in the array TREE, and an ITEM of information is
given. This procedure inserts ITEM as a new element of H. PTR gives the location of ITEM as it rises in the tree, and
PAR denotes the location of the parent of ITEM.
1. [Add new node to H and initialize PTR.]

Set N := N + 1 and PTR := N.
2. [Find location to insert ITEM.]

Repeat Steps 3 to 6 while PTR < 1.
3. Set PAR := bPTR/2c. [Location of parent node.]
4. If ITEM ≤ TREE[PAR], then:

Set TREE[PTR] := ITEM, and Return.
5. Set TREE[PTR] := TREE[PAR]. [Moves node down.]
6. Set PTR := PAR. [Updates PTR.]

7. [Assign ITEM as the root of H.]

Set TREE[I] := ITEM.
8. Return.
Trees Trees
Inserting into a Heap Deleting the Root of a Heap
1 INTRODUCTION
Inserting into a Heap - Algorithm II 2 BINARY TREES
Terminology
Strict Binary Trees
Suppose an array A with N elements is given. By repeatedly applying Sequential Representation of Binary Trees
Procedure to A, that is, by executing Call INSHEAP(A, J, A[J + 1]) for 4 PROPERTIES OF BINARY TREES
J = 1, 2, ..., N - 1, we can build a heap H out of the array A. 5 TRAVERSING BINARY TREES
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Deleting the Root of a Heap Deleting the Root of a Heap

9 Expression Tree
Application of Binary Search Trees Suppose H is a heap with N elements, and suppose we want to delete
Deleting in a Binary Search Tree the root R of H. This is accomplished as follows:
11 AVL SEARCH TREES
1. Assign the root R to some variable ITEM.
Deletion in an AVL search tree 2. Replace the deleted node R by the last node L of H so that H is
12 RED BLACK TREES still a complete tree, but not necessarily a heap.
Rotations 3. (Reheap) Let L sink to its appropriate place in H so that H is nally
Insertion a heap.
Deletion
Trees Trees
Deleting the Root of a Heap Deleting the Root of a Heap
Deleting the Root of a Heap - Example Deleting the Root of a Heap - Algorithm I
Suppose we want to delete Root R = 95 in a given heap. DELHEAP(TREE, N, ITEM): A heap H with N elements is stored in the array TREE. This procedure assigns the root
TREE[1] of H to the variable ITEM and then reheaps the remaining elements. The variable LAST saves the value
of the original last node of H. The pointers PTR, LEFT and RIGHT give the locations of LAST and its left and right
children as LAST sinks in the tree.
1. Set ITEM := TREE[1]. [Removes root of H.]
2. Set LAST := TREE[N] and N := N − 1. [Removes last node of H.]
3. Set PTR := 1, LEFT := 2 and RIGHT := 3. [Initializes pointers.]
4. Repeat Steps 5 to 7 while RIGHT ≤ N :
5. If LAST ≥ TREE[LEFT ] and LAST ≥ TREE[RIGHT ], then:

Set TREE[PTR] := LAST and Return.
6. IF TREE[RIGHT ] ≤ TREE[LEFT ], then:

Set TREE[PTR] := TREE[LEFT ] and PTR := LEFT .
Else:
Set TREE[PTR] := TREE[RIGHT ] and PTR := RIGHT .
7. Set LEFT := 2 ∗ PTR and RIGHT := LEFT + 1.

Trees Trees
Deleting the Root of a Heap Application to Sorting - Heapsort
1 INTRODUCTION
Deleting the Root of a Heap - Algorithm II 2 BINARY TREES
Terminology
Strict Binary Trees
8. If LEFT = N and if LAST < TREE[LEFT ], then: Set PTR := LEFT .
9. Set TREE[PTR] := LAST .
10. Return. 5 TRAVERSING BINARY TREES
Preorder Traversal
Inorder Traversal
Postorder Traversal
Header Nodes
Trees Trees
Application to Sorting - Heapsort Application to Sorting - Heapsort

9 Expression Tree
Application to Sorting I
Suppose an array A with N elements is given.
11 AVL SEARCH TREES The heapsort algorithm to sort A consists of the two following
Insertion in an AVL Search Tree phases:
1. Phase A: Build a heap H out of the elements of A.
12 RED BLACK TREES 2. Phase B: Repeatedly delete the root element of H.
Rotations
Insertion Since the root of H always contains the largest node in H, Phase
Deletion B deletes the elements of A in decreasing order.
Trees Trees
Heapsort - Algorithm I Complexity of Heapsort I
HEAPSORT(A, N): An array A with N elements is given. This algorithm sorts the elements of A.
1. [Build a heap H, using Procedure.]

Repeat for J = 1 to N − 1 :
Call INSHEAP(A, J, A[J + 1]).
[End of loop.] Suppose the heapsort algorithm is applied to an array A with n
2. [Sort A by repeatedly deleting the root of H, using Procedure.] elements.
Repeat while N > 1:
The algorithm has two phases, and we analyze the complexity of
(a) Call DELHEAP(A, N, ITEM).
each phase separately.
(b) Set A[N + 1] := ITEM.
[End of Loop.]
3. Exit.
Trees Trees
Complexity of Heapsort II Complexity of Heapsort III
Phase B: Suppose H is a complete tree with m elements, and

Phase A: Suppose H is a heap. Observe that the number of suppose the left and right subtrees of H are heaps and L is the
comparisons to find the appropriate place of a new element ITEM root of H. Observe that reheaping uses 4 comparisons to move
in H cannot exceed the depth of H. Since H is a complete tree, its the node L one step down the tree H. Since the depth of H does
depth is bounded by log2 m where m is the number of elements not exceed log2 m, reheaping uses at most 4 log2 m comparisons
in H. Accordingly, the total number g(n) of comparisons to insert to find the appropriate place of L in the tree H. This means that
the n elements of A into H is bounded as follows: g(n) ≤ n log2 n the total number h(n) of comparisons to delete the n elements
of A from H, which requires reheaping n times, is bounded as
Consequently, the running time of Phase A of heapsort is propor- follows: h(n) ≤ 4n log2 n
tional to n log2 n.
Accordingly, the running time of Phase B of heapsort is also pro-
portional to n log2 n.
Trees
HEAP and HEAPSORT
Complexity of Heapsort IV
Since each phase requires time proportional to n log2 n, the run-

ning time to sort the n-element array A using heapsort is propor-
tional to n log2 n, that is, f (n) = O(n log2 n).
Graph Algorithms Graph Algorithms
Outline I
1 Introduction
Types of Graphs
Representation of a Graph
2 Elementary Graph Algorithms
Graph Algorithms TRAVERSING A GRAPH
Breadth-first search
Depth-first search
3 Minimum Spanning Trees
The algorithms of Kruskal
The algorithms of Prim
4 Single-Source Shortest Paths
The Bellman-Ford algorithm
Dijkstra’s algorithm
5 All-Pair Shortest Paths
The Floyd-Warshall algorithm

1 Introduction What is a Graph? I

Types of Graphs

TRAVERSING A GRAPH Nonlinear data structure
Depth-first search A data structure that consists of a set of nodes(vertices) and
a set of edges that relate the node to each other.
The set of edges describes relationships among vertices.
The algorithms of Prim A graph G is an ordered pair defined as follows:
G = (V , E)
4 Single-Source Shortest Paths V (G): A finite, non-empty set of vertices.
E(G): A set of edges (collection of pair of vertices from V).

What is a Graph? II Example of a Graph
V =1,2,3,4,5,6
E={(1,4),(1,6),(2,6),(4,5),(5,6)}
A graph G consists of two things:
1. A set V of elements called nodes (or points or vertices)
2. A set E of edges such that each edge e in E is identified
with a unique (unordered) pair [u, v] of nodes in V, denoted
by e = [u, v]
We indicate the parts of a graph by writing G = (V, E).

Basic terms Order of the Graph

The total no. of vertices in the graph G is called the order of
the graph G.
Node or Vertex: The elements of a graph are connected It is also called the cardinality of the graph G and is repre-
through edges. sented by |V |.
For example the cardinality or order of the graph is 6 be-
Edges: A path or a line between two vertices in a graph.
cause the total no. of vertices in this graph is six.
Adjacent Nodes: Two nodes are called adjacent if they are
connected through an edge.
Suppose e = [u, v]. Then the nodes u and v are called the end-
points of e, and u and v are said to be adjacent nodes or neigh-
bors.
Size of the Graph Self Loop
The total no of edges in the graph G is called the size of the

An edge whose starting and ending point are same are
graph G.
called the self loop or we can say that the edge which is
It is represented by |E|. associated with the same vetex pair say (Vj , Vj ) are called
For example, in the graph there are total of 5 edges so the the self loop.
size of the graph G is 5.

Parallel or Multiple edge The degree of the Vertex

The degree of a node u, written deg(u), is the number of
edges containing u. If deg(u) = 0 - that is, if u does not
The two ore more then two edges are called the parallel
belong to any edge - then u is called an isolated node.
edges if these are associated with thee same vertex pair,
The total number of edges incident on a vertex is called
the degree of that vertex. When we counted the degree of
the vertex then the self-loop is counted two times. In graph
given the degree of node 6 is 3.
Isolated Vertex Pendant Vertex
In a Graph a vertex with degree zero is called the isolated A vertex with degree one is called the pendent vertex of
vertex or in other word we can say that the vertex which is the graph G or in other world if there is only one edge as-
not associated with any edge is called the isolated vertex. sociated with any vertex then the degree of that vertex is
In the graph the vertex 3 is called the isolated vertex. one and vertex is called the pendant vertex.In the graph the
vertex 2 is called the pendant vertex.

Path I Path II
In a Graph G an open walk is called an Path if in a walk no
vertex appears more then once. A path P of length n from a node u to a node v is defined as a
Hence we can say that a path is a finite alternative se- sequence of n + 1 nodes.
quences of vertices and edges in which no of the vertices P = (v0 , v1 , v2 , · · · , vn ) such that u = v0 ; vi−1 is adjacent to vi for
and edges appears more then once. i = 1, 2, · · · , n; and vn = v.
The path P is said to be closed if v0 = vn .
The total number of edges in the path is called the length of
The path P is said to be simple if all the nodes are distinct, with
the path. V1,e1,V2,e4,V4,e5,V5,e6,V3 is an path.
the exception that v0 may equal vn ; that is, P is simple if the
nodes v0 , v1 , · · · , vn−1 are distinct and the nodes v1 , v2 , · · · , vn
are distinct.
A cycle is a closed simple path with length 3 or more. A cycle of
length k is called a k-cycle.
Types of Graphs Types of Graphs
1 Introduction Types of Graphs

Types of Graphs

TRAVERSING A GRAPH Undirected graph
Breadth-first search Directed graph
Depth-first search
Directed Acyclic Graph (DAG)
3 Minimum Spanning Trees Multi graph
The algorithms of Prim Simple graph
Weighted and Unweighted graph
The Bellman-Ford algorithm Complete graph
Dijkstra’s algorithm Connected graph, etc.

Null Graph Regular Graph
A graph without self loops and parallel edges is called the

regular graph if degree of each and every vertex are same
A graph having only vertex set is called the null graph in this
in other world we can say that the simple graph in which the
case we can say that E(G)=0 i.e, empty.
degree of all vertices are same is called the Regular Graph.
Multi graph I Multi graph II

A multigraph is an undirected graph in which multiple edges A graph G is said to be labeled if its edges are assigned
(and sometimes loops) are allowed. data.
Multiple edges are two or more edges that connect the same In particular, G is said to be weighted if each edge e in G
two vertices. is assigned a nonnegative numerical value w(e) called the
A loop is an edge (directed or undirected) that connects a weight or length of e.
vertex to itself; it may be permitted or not. In such a case, each path P in G is assigned a weight or
length which is the sum of the weights of the edges along
the path P.
If we are given no other information about weights, we may
assume any graph G to be weighted by assigning the weight
w(e) = 1 to each edge e in G.
The definition of a graph may be generalized by permitting
the following:

Multi graph III Simple graph

1. Multiple edges: Distinct edges e and e’ are called multiple
edges if they connect the same endpoints, that is, if e = [u,
v] and e’ = [u, v]. A simple graph is an undirected graph in which both multiple
2. Loops: An edge e is called a loop if it has identical end- edges and loops are disallowed as opposed to a multigraph.
points, that is, if e = [u, u].
In a simple graph with n vertices, every vertex degree is at
Such a generalization M is called a multigraph. most n − 1.
In other words, the definition of a graph usually does not
allow either multiple edges or loops.
A multigraph M is said to be finite if it has a finite number of
nodes and a finite number of edges Observe that a graph
G with a finite number of nodes must automatically have
a finite number of edges and so must be finite; but this is
not necessarily true for a multigraph M, since M may have
multiple edges.
Undirected graph Directed graph I
An undirected graph (graph) is a graph in which edges have A Directed graph (digraph) is a graph in which edges have
no orientation. orientations.
The edge (x, y ) is not identical to edge (y, x).
The edge (x, y) is identical to edge (y, x), i.e., they are not
ordered pairs.
The maximum number of edges possible in an undirected
graph without a loop is n(n − 1)/2

Directed graph II Directed graph III

A directed graph G, also called a digraph or graph, is the
same as a multigraph except that each edge e in G is as-
signed a direction, or in other words, each edge e is identi- The outdegree of a node u in G, written outdeg(u), is the
fied with an ordered pair (u, v) of nodes in G rather than an number of edges beginning at u.
unordered pair [u, v]. Similarly, the indegree of u, written indeg(u), is the number
Suppose G is a directed graph with a directed edge e = (u, of edges ending at u.
v). Then e is also called an arc. A node u is called a source if it has a positive outdegree but
Moreover, the following terminology is used: zero indegree.
1. e begins at u and ends at v. Similarly, u is called a sink if it has a zero outdegree but a
2. u is the origin or initial point of e, and v is the destination or
positive indegree.
terminal point of e.
3. u is a predecessor of v, and v is a successor or neighbor of
u.
4. u is adjacent to v, and v is adjacent to u.
Directed graph IV Directed graph V
The notions of path, simple path and cycle carry over from A directed graph G is said to be connected, or strongly con-
undirected graphs to directed graphs except that now the nected, if for each pair u, v of nodes in G there is a path
direction of each edge in a path (cycle) must agree with from u to v and there is also a path from v to u.
the direction of the path (cycle). A node v is said to be On the other hand, G is said to be unilaterally connected if
reachable from a node u if there is a (directed) path from u for any pair u, v of nodes in G there is a path from u to v or
to v. a path from v to u.

Directed graph VI Directed graph VII
Let T be any nonempty tree graph. Suppose we choose any A directed graph G is said to be simple if G has no parallel
node R in T. Then T with this designated node R is called edges. A simple graph G may have loops, but it cannot have
a rooted tree and R is called its root. Recall that there is more than one loop at a given node. A nondirected graph G
a unique simple path from the root R to any other node in may be viewed as a simple directed graph by assuming that
T. This defines a direction to the edges in T, so the rooted each edge [u, v] in G represents two directed edges, (u, v)
tree T may be viewed as a directed graph. Furthermore, and (v, u).
suppose we also order the successors of each node v in Observe that we use the notation [u, v] to denote an un-
T. Then T is called an ordered rooted tree. Ordered rooted ordered pair and the notation (u, v) to denote an ordered
trees are nothing more than the general trees. pair.
Directed Acyclic Graph (DAG) Weighted graph
A weighted graph associates a value (weight) with every

A Directed Acyclic Graph (DAG) is a directed graph that edge in the graph. We can also use words cost or length
contains no cycles. instead of weight.

Unweighted graph Complete graph I

An unweighted graph does not have any value (weight) as-
sociated with every edge in the graph. In other words, an
unweighted graph is a weighted graph with all edge weight A graph G is said to be connected if there is a path between
as 1. any two of its nodes. In other words, a graph G is connected
Unless specified otherwise, all graphs are assumed to be if and only if there is a simple path between any two nodes
unweighted by default. in G.
A complete graph is one in which every two vertices are
adjacent: all edges that could exist are present.
A graph G is said to be complete if every node u in G is
adjacent to every other node v in G. Clearly such a graph is
connected. A complete graph with n nodes will have n(n −
1)/2 edges.
Complete graph II Connected graph I

A Connected graph has a path between every pair of ver-
tices.
In other words, there are no unreachable vertices.
A disconnected graph is a graph that is not connected.

Cyclic graph Tree I

A simple graph of n vertices (n >= 3) and n edges forming
a cycle of length n is called as a cycle graph.
In a cycle graph, all the vertices are of degree 2. A simple graph with no cycle is called as Tree.
A graph containing at least one cycle in it is called as a
A connected graph T without any cycles is called a tree
cyclic graph.
graph or free tree or, simply, a tree.
This means, in particular, that there is a unique simple path
P between any two nodes u and v in T.
Furthermore, if T is a finite tree with m nodes, then T will
have m-1 edges.
Tree II Example I

Example II Example III

Figure (b) is not a graph but a multigraph. The reason is
Figure (a) is a picture of a connected graph with 5 nodes - that it has multiple edges - e4 = [B, C] and e5 = [B, C] - and
A, B, C, D and E - and 7 edges: it has a loop, e6 = [D, D). The definition of a graph usually
[A, B], [B, C], [C, D], [D, E], [A, E], [C, E] [A, C] does not allow either multiple edges or loops.
There are two simple paths of length 2 from B to E: (B, A, Figure (c) is a tree graph with m = 6 nodes and, conse-
E) and (B, C, E). There is only one simple path of length 2 quently, m - 1 = 5 edges. The reader can verify that there
from B to D: (B, C, D). is a unique simple path between any two nodes of the tree
We note that (B, A, D) is not a path, since [A, D) is not an graph.
edge. There are two 4 - cycles in the graph: Figure (d) is the same graph as in Figure (a), except that
[A, B, C, E, A] and [A, C, D, E, A]. now the graph is weighted. Observe that P1 = (B, C, D) and
Note that deg(A) = 3, since A belongs to 3 edges. Similarly, P2 = (B, A, E, D) are both paths from node B to node D.
deg(C) = 4 and deg(D) = 2. Although P2 contains more edges than P1 the weight w(P2 )
= 9 is less than the weight w(P1 ) = 10.
Types of Graphs Representation of a Graph
Example IV 1 Introduction
Types of Graphs

TRAVERSING A GRAPH
Depth-first search

Figure shows a directed graph G with 4 nodes and 7 (directed) edges. The edges e2 and e3 are said to be The algorithms of Kruskal
parallel, since each begins at B and ends at A. The edge e7 is a loop, since it begins and ends at the same The algorithms of Prim
point, B. The sequence P1 = (D, C, B, A) is not a path, since (C, B) is not an edge - that is, the direction of
the edge e5 = (B, C) does not agree with the direction of the path P1 . On the other hand, P2 = (D, B, A) is a
path from D to A, since (D, B) and (B, A) are edges. Thus A is reachable from D. There is no path from C to Dijkstra’s algorithm
any other node, so G is not strongly connected. However, G is unilaterally connected. Note that indeg(D) =
1 and outdeg(D) = 2. Node C is a sink, since indeg(C) = 2 but outdeg(C) = 0. No node in G is a source. 5 All-Pair Shortest Paths

Representation of a Graph Representation of a Graph
Graph Representation Adjacency Matrix Representation I
The graph can be represented in many ways the some of them An adjacency matrix is a square matrix used to represent a
are discussed here. finite graph.
Adjacency Matrix Representation The elements of the matrix indicate whether pairs of ver-
tices are adjacent or not in the graph.
Adjacency List Representation
Space complexity: O(|V 2 |)
Adjacency Matrix Representation II Adjacency Matrix Representation III
The adjacency matrix A of the graph G does depend on the

ordering of the nodes of G; that is, a different ordering of
Definition: For a simple unweighted graph with vertex set V , the nodes may result in a different adjacency matrix. How-
the adjacency matrix is a square |V | × |V | matrix A such that its ever, the matrices resulting from two different orderings are
element: closely related in that one can be obtained from the other
Aij = 1, when there is an edge from vertex i to vertex j, and by simply interchanging rows and columns.
Aij = 0, when there is no edge
Suppose G is an undirected graph. Then the adjacency
Such a matrix A, which contains entries of only 0 and 1, is called
matrix A of G will be a symmetric matrix, i.e., one in which
a bit matrix or a Boolean matrix.
aij = aji for every i and j. This follows from the fact that
each undirected edge [u, v] corresponds to the two directed
edges (u, v) and (v, u).

Adjacency Matrix Representation IV Adjacency Matrix Representation: Example1 I
The above matrix representation of a graph may be ex-

tended to multigraphs. Specifically, if G is a multigraph, then
the adjacency matrix of G is the m × m matrix A = (aij ) de-
fined by setting aij equal to the number of edges from vi to
vj .
Adjacency Matrix Representation: Example1 II Path Matrix I
Consider the powers A, A2 , A3 , · · · of the adjacency matrix A of

a graph G. Let aK (i, j) = the ij entry in the matrix AK . Observe
that a1 (i, j) = aij gives the number of paths of length 1 from node
vi to node vj . One can show that a2 (i, j) gives the number of
paths of length 2 from vi to vj . Then aK (i, j), the ij entry in the
matrix AK , gives the number of paths of length K from vi to vj .
Suppose we now define the matrix Br as follows:
Br = A + A2 + A3 + · · · + Ar
Then the ij entry of the matrix Br gives the number of paths of
length r or less from node vi to vj .

Path Matrix Path Matrix: Example
Consider the graph G in Fig. Suppose the nodes are stored in memory in a linear array DATA as follows: DATA: X, Y,
Z, W
Then we assume that the ordering of the nodes in G is as follows: n1 = X , n2 = Y , n3 = Z and n4 = W . The
adjacency matrix A of G is as follows:
Let G be a simple directed graph with m nodes, v1 , v2 , ..., vm .

The path matrix or reachability matrix of G is the m−square
matrix P = (pij ) defined as follows:
Note thatP = 1,ofif1’sthere
ij number
the is toa the
in A is equal path from
number vini G.
of edges to vj
Pij = 0, Otherwise
Examining the matrix P, we see that the node v2 is not reachable

from any of the other nodes
Path Matrix: Observations I Path Matrix: Observations II
Suppose there is a path from vi to vj . Then there must be a Let A be the adjacency matrix and let P = (pij ) be the path
simple path from vi to vj when vi 6= vj , or there must be a cycle matrix of a digraph G. Then Pij = 1 if and only if there is a
from vi to vj when vi = vj . Since G has only m nodes, such a nonzero number in the ij entry of the matrix
simple path must have length m − 1 or less, or such a cycle must Bm = A + A2 + A3 + · · · + Am
have length m or less.

Path Matrix: Observations III Path Matrix: Observations IV
The transitive closure of a graph G is defined to be the graph

Recall that a directed graph G is said to be strongly connected if, G0 such that G0 has the same nodes as G and there is an edge
for any pair of nodes u and v in G, there are both a path from u (vi , vj ) in G0 whenever there is a path from vi to vj in G. Ac-
to v and a path from v to u. Accordingly, G is strongly connected cordingly, the path matrix P of the graph G is precisely the ad-
if and only if the path matrix P of G has no zero entries. Thus jacency matrix of its transitive closure G0 . Furthermore, a graph
the graph G in Fig. is not strongly connected. G is strongly connected if and only if its transitive closure is a
complete graph.
Linked List Representation Linked List Representation: Example1 I
The linked list representation of the graph is more efficient

then the matrix representation.
In this representation each vertex of the graph G is repre-
sented by a linked list structure.
Space complexity: O(|V | + |E|)

Introduction Elementary Graph Algorithms
Linked List Representation: Example1 II 1 Introduction

Types of Graphs

TRAVERSING A GRAPH
Depth-first search



Elementary Graph Algorithms Elementary Graph Algorithms
Node insertion Edge insertion

Suppose an edge (A, B) is to be inserted in the graph G. The
Suppose a node N is to be inserted in the graph G. Note that procedure first finds the location LOCA of A and the location
N will be assigned to NODE[AVAILN],the first available node. LOCB of B in the node list. Then (A, B) is inserted as an edge
Moreover, since N will be an isolated node, one must also set in G by inserting LOCB in the list of successors of A, which has
ADJ[AVAILN] = NULL the list pointer ADJ[LOCA]. Again, a logical variable FLAG is
used to indicate overflow.
INSNODE(NODE, NEXT, ADJ, START, AVAILN, N, FLAG)
This procedure inserts the node N in the graph G.
1. [OVERFLOW?] If AVAILN = NULL, then: Set FLAG := FALSE, and Return. INSEDGE(NODE, NEXT, ADJ, START, DEST, LINK, AVAILE, A, B, FLAG)
This procedure inserts the edge (A, B) in the graph G.
2. Set ADJ[AVAILN] := NULL.
1. Call FIND(NODE, NEXT, START, A, LOCA).
3. [Removes node from AVAILN list]
Set NEW := AVAILN and AVAILN := NEXT[AVAILN]. 2. Call FIND(NODE, NEXT, START, B, LOCB).
4. [Inserts node N in the NODE list.] 3. [OVERFLOW?] If AVAILE = NULL, then: Set FLAG := FALSE, and Return.
Set NODE[NEW] := N, NEXT[NEW] := START and START := NEW. 4. [Remove node from AVAILE list.] Set NEW := AVAILE and AVAILE := LINK[AVAILE].
5. Set FLAG := TRUE, and Return. 5. [Insert LOCB in list of successors of A.]
Set DEST[NEW] := LOCB, LINK[NEW] := ADJ[LOCA] and ADJ[LOCA] := NEW.
6. Set FLAG := TRUE, and Return.

Edge insertion I Edge insertion II
Procedure for find:

FINDEDGE(NODE, NEXT, ADJ, START, DEST, LINK, A, B, LOC)
This procedure finds the location LOC of an edge (A, B) in the graph G, or sets LOC := NULL.
FIND(INFO, LINK START, ITEM, LOC)
1. Call FIND(NODE, NEXT, START, A, LOCA).
Finds the location LOC of the first node containing ITEM, or sets LOC := NULL. 2. CALL FIND(NODE, NEXT, START, B, LOCB).
1. Set PTR := START. 3. If LOCA = NULL or LOCB = NULL, then: Set LOC := NULL.
Else: Call FIND(DEST, LINK, ADJ[LOCA], LOCB, LOC).
2. Repeat while PTR 6= NULL:
If ITEM = INFO[PTR], then: Set LOC := PTR, and Return. 4. Return.
Else: Set PTR := LINK[PTR].
[End of loop.]
3. Set LOC := NULL, and Return.

Node deletion I Node deletion II
Suppose a node N is to be deleted from the graph G. This

operation is more complicated than the search and insertion
Delete all the edges beginning at N. This is accomplished
operations and the deletion of an edge, because we must
by finding the location BEG of the first successor and the
also delete all the edges that contain N.
location END of the last successor of N, and then adding
Note these edges come in two kinds; those that begin at the successor list of N to the free AVAILE list.
N and those that end at N. Accordingly, our procedure will
Delete N itself from the list NODE.
consist mainly of the following four steps
Find the location LOC of the node N in G.
Delete all edges ending at N; that is, delete LOC from the
list of successors of each node M in G.

Node deletion Algorithm I Node deletion Algorithm II

DELNODE(NODE, NEXT, ADJ, START, AVAILN, DEST, LINK, AVAILE, N, FLAG)
This procedure deletes the node N from the graph G. Procedure for delete:
1. Call FIND(NODE, NEXT, START, N, LOC). [Locates node N.]
2. If LOC = NULL, then: Set FLAG := FALSE, and Return.
3. [Delete edges ending at N.] DELETE(INFO, LINK, START, AVAIL, ITEM, FLAG)
(a) Set PTR := START. Deletes the first node in the list containing ITEM, or sets FLAG := FALSE when ITEM does not appear in the list.
(b) Repeat while PTR 6= NULL: 1. [List empty?] If START = NULL, then: Set FLAG := FALSE, and Return.
(i) Call DELETE(DEST, LINK, ADJ[PTR], AVAILE, LOC, FLAG). 2. [ITEM in first node?] If INFO[START]. = ITEM, then:
(ii) Set PTR := NEXT[PTR]. Set PTR := START, START := LINK[START],
LINK[PTR] := AVAIL, AVAIL := PTR,
[End of loop.] FLAG := TRUE, and Return.
4. [Successor list empty?] If ADJ[LOC] = NULL, then: Go to Step 7.
3. Set PTR := LINK[START] and SAVE := START. [Initializes pointers.]
5. [Find the first and last successor of N.] 4. Repeat Steps 5 and 6 while PTR 6= NULL:
(a) Set BEG := ADJ[LOC], END := ADJ[LOC] and PTR := LINK[END]. 5. If INFO[PTR] = ITEM, then:
(b) Repeat while PTR 6= NULL: Set LINK[SAVE] := LINK[PTR], LINK[PTR] := AVAIL,
Set END := PTR and PTR := LINK[PTR]. AVAIL := PTR, FLAG := TRUE, and Return.
[End of loop.] [End of If structure.]
6. [Add successor list of N to AVAILE list.] 6. Set SAVE := PTR and PTR := LINK[PTR]. [Updates pointers]
Set LINK[END] := AVAILE and AVAILE := BEG. [End of Step 4 loop.]
7. Call DELETE(NODE, NEXT, START, AVAILN, N, FLAG). 7. Set FLAG := FALSE, and Return.
8. Return.
TRAVERSING A GRAPH
Edge deletion 1 Introduction

Types of Graphs
Suppose an edge (A, B) is to be deleted from the graph G. Again 2 Elementary Graph Algorithms
we must first find the location LOCA of A and the location LOCB TRAVERSING A GRAPH
of B in the node list. Then we simply delete LOCB from the list Breadth-first search
of successors of A, which has the list pointer ADJ[LOCA]. A Depth-first search
logical variable FLAG is used to indicate that there is no such
edge in the graph G. The algorithms of Kruskal
DELEDGE(NODE, NEXT, ADJ, START, DEST, LINK, AVAILE, A, B, FLAG)
This procedure deletes the edge (A, B) from the graph G.
1. Call FIND(NODE, NEXT, START, A, LOCA). [Locates node A.] The Bellman-Ford algorithm
2. Call FIND(NODE, NEXT, START, B, LOCB). [Locates node B.] Dijkstra’s algorithm
3. Call DELETE(DEST, LINK, ADJ[LOCA], AVAILE, LOCB, FLAG).
4. Return. 5 All-Pair Shortest Paths

TRAVERSING A GRAPH TRAVERSING A GRAPH
TRAVERSING A GRAPH I TRAVERSING A GRAPH II
During the execution of our algorithms, each node N of G

There are two standard ways that this is done. will be in one of three states, called the status of N, as fol-
1. Breadth-First search lows:
2. Depth-First search I STATUS = 1: (Ready state.) The initial state of the node N.
I STATUS = 2: (Waiting state.) The node N is on the queue or
The breadth-first search will use a queue as an auxiliary stack, waiting to be processed.
structure to hold nodes for future processing, and analo- I STATUS = 3: (Processed state.) The node N has been pro-
gously, the depth-first search will use a stack. cessed.
1 Introduction Breadth-First Search I

Types of Graphs

TRAVERSING A GRAPH
Depth-first search First examine the starting node A.
3 Minimum Spanning Trees Then examine all the neighbors of the neighbors of A. And
The algorithms of Kruskal so on. Naturally, we need to keep track of the neighbors of
The algorithms of Prim a node, and need to guarantee that no node is processed
more than once.


Breadth-First Search II Breadth-First Search III
Algorithm: This algorithm executes a breadth-first search on a graph G beginning at a starting node A.
The above algorithm will process only those nodes which
1. Initialize all nodes to the ready state (STATUS = 1). are reachable from the starting node A.
2. Put the starting node A in QUEUE and change its status to the waiting state (STATUS = 2).
3. Repeat Steps 4 and 5 until QUEUE is empty.
Suppose one wants to examine all the nodes in the graph
4. Remove the front node N of QUEUE. Process N and change the status of N to the processed state G. Then the algorithm must be modified so that it begins
(STATUS = 3).
again with another node (which we will call B) that is still in
5. Add to the rear of QUEUE all the neighbors of N that are in the steady state (STATUS= 1), and change
their status to the waiting state (STATUS = 2). the ready state.
6. Exit. This node B can be obtained by traversing the list of nodes.
BFS-Example-1 I BFS-Example-1 II

BFS-Example-1 III BFS-Example-1 IV

BFS-Example-2 I BFS-Example-2 II
Consider the graph G in Fig.(a). (The adjacency lists of the

nodes appear in Fig.(b).) Suppose G represents the daily flights
between cities of some airline, and suppose we want to fly from
city A to city J with the minimum number of stops. In other words,
we want the minimum path P from A to J (where each edge has
length 1).

BFS-Example-2 III BFS-Example-2 IV
(a) Initially, add A to QUEUE and add NULL to ORIG as follows:

FRONT = 1 QUEUE : A
The minimum path P can be found by using a breadth-first search REAR = 1 ORIG : ∅
beginning at city A and ending when J is encountered. During (b) Remove the front element A from QUEUE by setting FRONT
the execution of the search, we will also keep track of the origin := FRONT + 1, and add to QUEUE the neighbors of A as
of each edge by using an array ORIG together with the array follows:
QUEUE. The steps of our search follow. FRONT = 2 QUEUE : A, F, C, B
REAR = 4 ORIG : ∅, A, A, A
Note that the origin A of each of the three edges is added
to ORIG.
BFS-Example-2 V BFS-Example-2 VI
(e) Remove the front element B from QUEUE, and add to QUEUE
(c) Remove the front element F from QUEUE by setting FRONT the neighbors of B (the ones in the ready state) as follows:
:= FRONT + 1, and add to QUEUE the neighbors of F as fol- FRONT = 5 QUEUE : A, F, C, B, D, G
lows: REAR = 6 ORIG : ∅, A, A, A, F, B
FRONT = 3 QUEUE : A, F, C, B, D Note that only G is added to QUEUE, since the other neigh-
REAR = 5 ORIG : ∅, A, A, A, F bor, C is not in the ready state.
(d) Remove the front element C from QUEUE, and add to QUEUE (f) Remove the front element D from QUEUE, and add to QUEUE
the neighbors of C (which are in the ready state) as follows: the neighbors of D (the ones in the ready state) as follows:
FRONT = 4 QUEUE : A, F, C, B, D FRONT = 6 QUEUE : A, F, C, B, D, G
REAR = 5 ORIG : ∅, A, A, A, F REAR = 6 ORIG : ∅, A, A, A, F, B
Note that the neighbor F of C is not added to QUEUE, since (g) Remove the front element G from QUEUE and add to QUEUE
F is not in the ready state (because F has already been the neighbors of G (the ones in the ready state) as follows:
added to QUEUE). FRONT = 7 QUEUE : A, F, C, B, D, G, E
REAR = 7 ORIG : ∅, A, A, A, F, B, G
BFS-Example-2 VII BFS-Example-3 I
(h) Remove the front element E from QUEUE and add to QUEUE
the neighbors of E (the ones in the ready state) as follows:
FRONT = 8 QUEUE : A, F, C, B, D, G, E, J
REAR = 8 ORIG : ∅, A, A, A, F, B, G, E
We stop as soon as J is added to QUEUE, since J is our
final destination. We now backtrack from J, using the array
ORIG to find the path P. Thus
J←E←G←B←A
is the required path P.
Applications of Breadth First Traversal I Applications of Breadth First Traversal II

Crawlers in Search Engines: Crawlers build index using
Breadth First. The idea is to start from source page and
Shortest Path and Minimum Spanning Tree for unweighted follow all links from source and keep doing same. Depth
graph In an unweighted graph, the shortest path is the path First Traversal can also be used for crawlers, but the advan-
with least number of edges. With Breadth First, we always tage with Breadth First Traversal is, depth or levels of the
reach a vertex from given source using the minimum num- built tree can be limited.
ber of edges. Also, in case of unweighted graphs, any span- Social Networking Websites: In social networks, we can
ning tree is Minimum Spanning Tree and we can use either find people within a given distance ’k’ from a person using
Depth or Breadth first traversal for finding a spanning tree. Breadth First Search till ’k’ levels.
Peer to Peer Networks. In Peer to Peer Networks like Bit- GPS Navigation systems: Breadth First Search is used to
Torrent, Breadth First Search is used to find all neighbor find all neighboring locations.
nodes.
Broadcasting in Network: In networks, a broadcasted packet
follows Breadth First Search to reach all nodes.

Applications of Breadth First Traversal III Applications of Breadth First Traversal IV

In Garbage Collection: Breadth First Search is used in copy-
ing garbage collection using Cheneys algorithm. Refer this Path Finding We can either use Breadth First or Depth First
and for details. Breadth First Search is preferred over Depth Traversal to find if there is a path between two vertices.
First Search because of better locality of reference.
Finding all nodes within one connected component: We can
Cycle detection in undirected graph: In undirected graphs, either use Breadth First or Depth First Traversal to find all
either Breadth First Search or Depth First Search can be nodes reachable from a given node.
used to detect cycle. We can use BFS to detect cycle in a
directed graph also. Many algorithms like Prims Minimum Spanning Tree and
Dijkstras Single Source Shortest Path use structure similar
FordFulkerson algorithm In Ford-Fulkerson algorithm, we
to Breadth First Search.
can either use Breadth First or Depth First Traversal to find
the maximum flow. Breadth First Traversal is preferred as it There can be many more applications as Breadth First Search
reduces worst case time complexity to O(VE 2 ). is one of the core algorithms for Graphs.
To test if a graph is Bipartite We can either use Breadth First
or Depth First Traversal.
BFS-Analysis 1 Introduction
Types of Graphs
In breadth-first search each vertex is enqueued at most once, 2 Elementary Graph Algorithms
and hence dequeued at most once. The operations of enqueu- TRAVERSING A GRAPH
ing and dequeuing take O(1) time, and so the total time devoted
Depth-first search
to queue operations is O(V ). Because the procedure scans the
adjacency list of each vertex only when the vertex is dequeued, 3 Minimum Spanning Trees
it scans each adjacency list at most once. Since the sum of the The algorithms of Kruskal
lengths of all the adjacency lists is Θ(E), the total time spent in The algorithms of Prim
scanning adjacency lists is O(E). The overhead for initialization
is O(V ), and thus the total running time of the BFS procedure is The Bellman-Ford algorithm
O(V + E). Thus, breadth-first search runs in time linear in the Dijkstra’s algorithm
size of the adjacency-list representation of G.

Depth-First Search I Depth-First Search II
The general idea behind a depth-first search beginning at a

starting node A is as follows.
Algorithm: This algorithm executes a depth-first search on a graph G beginning at a starting node A.
I First examine the starting node A.
1. Initialize all nodes to the ready state (STATUS = 1).
I Then examine each node N along a path P which begins 2. Push the starting node A onto STACK and change its status to the waiting state (STATUS = 2).
at A; i.e, we process a neighbor of A, then a neighbor of a 3. Repeat Steps 4 and 5 until STACK is empty.
neighbor of A, and so on. 4. Pop the top node N of STACK. Process N and change its status to the processed state (STATUS = 3).
I After coming to a “dead end”, i.e., to the end of the path P, 5. Push onto STACK all the neighbors of N that are still in the ready state (STATUS = 1), and change their
status to the waiting state (STATUS = 2).
we backtrack on P until we can continue along another, path [End of Step 3 loop.]
P. And so on. 6. Exit.
(This algorithm is similar to the in-order traversal of a binary

tree, and the algorithm is also similar to the way one might
travel through a maze.)
Depth-First Search III DFS-Example I
Again, the above algorithm will process only those nodes which
are reachable from the starting node A. Suppose one wants to
examine all the nodes in G. Then the algorithm must be modified
so that it begins again with another node.

DFS-Example II DFS-Example III

DFS-Example IV DFS-Example-2 I
Consider the graph G in Fig.(a). Suppose we want to find and
print all the nodes reachable from the node J (including J itself).
One way to do this is to use a depth-first search of G starting at
the node J.
The steps of our search follow.

DFS-Example-2 II DFS-Example-2 III
(a) Initially, push J onto the stack as follows: (d) Pop and print the top element G, and then push onto the
STACK: J stack all the neighbors of G (those in the ready state) as
follows:
(b) Pop and print the top element J, and then push onto the
Print G
stack all the neighbors of J (those that are in the ready state)
STACK: D, E, C
as follows:
Note that only C is pushed onto the stack, since the other
Print J
neighbor, E, is not in the ready state (because E has already
STACK: D, K
been pushed onto the stack).
(c) Pop and print the top element K, and then push onto the
(e) Pop and print the top element C, and then push onto the
stack all the neighbors of K (those that are in the ready
stack all the neighbors of C (those in the ready state) as
state) as follows:
follows:
Print K
Print C
STACK: D, E, G
STACK: D, E, F
DFS-Example-2 IV DFS-Example-2 V
(f) Pop and print the top element F, and then push onto the
stack all the neighbors of F (those in the ready state) as
follows: (h) Pop and print the top element D, and push onto the stack all
Print F the neighbors of D (those in the ready state) as follows:
STACK: D, E Print D
Note that the only neighbor D of F is not pushed onto the STACK:
stack, since D is not in the ready state (because D has al- The stack is now empty, so the depth-first search of G start-
ready been pushed onto the stack). ing at J is now complete. Accordingly, the nodes which were
(g) Pop and print the top element E, and push onto the stack printed,
all the neighbors of E (those in the ready state) as follows: J, K, G, C, F, E, D
Print E are precisely the nodes which are reachable from J.
STACK: D
(Note that none of the three neighbors of E is in the ready
state.)

DFS-Example-3 I Applications of Depth First Traversal I
Detecting cycle in a graph: A graph has cycle if and only if

we see a back edge during DFS. So we can run DFS for the
graph and check for back edges.
Path Finding: We can specialize the DFS algorithm to find
a path between two given vertices u and z.
1. Call DFS(G, u) with u as the start vertex.
2. Use a stack S to keep track of the path between the start
vertex and the current vertex.
3. As soon as destination vertex z is encountered, return the
path as the contents of the stack
Applications of Depth First Traversal II Applications of Depth First Traversal III
Topological Sorting: Topological Sorting is mainly used for

scheduling jobs from the given dependencies among jobs.
In computer science, applications of this type arise in in-
Finding Strongly Connected Components of a graph A di-
struction scheduling, ordering of formula cell evaluation when
rected graph is called strongly connected if there is a path
recomputing formula values in spreadsheets, logic synthe-
from each vertex in the graph to every other vertex.
sis, determining the order of compilation tasks to perform in
makefiles, data serialization, and resolving symbol depen- Solving puzzles with only one solution, such as mazes. (DFS
dencies in linkers. can be adapted to find all solutions to a maze by only includ-
To test if a graph is bipartite: We can augment either BFS or ing nodes on the current path in the visited set.)
DFS when we first discover a new vertex, color it opposited
its parents, and for each other edge, check it doesnt link two
vertices of the same color. The first vertex in any connected
component can be red or black.

DFS-Analysis I DFS-Analysis II
The time complexity of DFS actually depends on the data

structure being used to represent the graph. If the graph is represented as an adjacency matrix (a V x V
array):
If the graph is represented as adjacency list: I For each node, we will have to traverse an entire row of
I For each node, we discover all its neighbors by traversing its
length V in the matrix to discover all its outgoing edges.
adjacency list just once in linear time. I Note that each row in an adjacency matrix corresponds to
I For a directed graph, the sum of the sizes of the adjacency
a node in the graph, and that row stores information about
lists of all the nodes is E. So, the time complexity in this case edges emerging from the node. Hence, the time complexity
is O(V ) + O(E) = O(V + E). of DFS in this case is O(V × V ) = O(V 2 ).
I For an undirected graph, each edge appears twice. Once
in the adjacency list of either end of the edge. The time
complexity for this case will be O(V ) + O(2E) ∼ O(V + E).
Elementary Graph Algorithms Minimum Spanning Trees
TRAVERSING A GRAPH
BFS vs DFS I 1 Introduction

Types of Graphs

TRAVERSING A GRAPH
Depth-first search




Minimum Spanning Trees Minimum Spanning Trees
Minimum Spanning Tree I 1 Introduction

Types of Graphs
Find out a tree T in a connected graph that connects all of the
vertices, which we call a spanning tree since it “spans” the 2 Elementary Graph Algorithms
graph G. We call the problem of determining the tree T the TRAVERSING A GRAPH
minimum-spanning-tree problem. Depth-first search



Two minimum spanning tree algorithms are Prim’s and Kruskal’s The Floyd-Warshall algorithm
The algorithms of Kruskal The algorithms of Kruskal
Kruskal’s Algorithm I Kruskal’s Algorithm II
In Kruskal’s algorithm, the set A is a forest whose vertices are Thus, we can determine whether two vertices u and v belong to
all those of the given graph. The safe edge added to A is al- the same tree by testing whether FIND − SET (u) equals FIND −
ways a least-weight edge in the graph that connects two distinct SET (v ). To combine trees, Kruskal’s algorithm calls the UNION
components. procedure.
An edge (u, v ) of least weight. Let C1 and C2 denote the two
trees that are connected by (u, v ). Since (u, v ) must be a light
edge connecting C1 to some other tree. Kruskal’s algorithm
qualifies as a greedy algorithm because at each step it adds
to the forest an edge of least possible weight.
We use a disjoint-set data structure to maintain several disjoint
sets of elements. Each set contains the vertices in one tree
of the current forest. The operation FIND − SET (u) returns a
representative element from the set that contains u.

The algorithms of Kruskal The algorithms of Kruskal
Kruskal’s Algorithm III Kruskal’s Algorithm IV

The algorithms of Kruskal The algorithms of Prim
Kruskal’s algorithm complexity 1 Introduction

Types of Graphs

Worst case time complexity of Kruskals Algorithm is O(ElogV )
TRAVERSING A GRAPH
or O(ElogE). Breadth-first search
The edges are maintained as min heap. Depth-first search
The next edge can be obtained in O(log E) time if graph has 3 Minimum Spanning Trees
E edges. The algorithms of Kruskal
Reconstruction of heap takes O(E) time. The algorithms of Prim
So, Kruskal’s Algorithm takes O(E log E) time. 4 Single-Source Shortest Paths
The value of E can be at most O(V 2 ). The Bellman-Ford algorithm
So, O(log V ) and O(log E) are same.

The algorithms of Prim The algorithms of Prim
Prim’s Algorithm I Prim’s Algorithm II
In order to implement Prim’s algorithm efficiently, we need a fast

way to select a new edge to add to the tree formed by the edges
Prim’s algorithm has the property that the edges in the set A in A. In the pseudocode below, the connected graph G and the
always form a single tree. Each step adds to the tree A a light root r of the minimum spanning tree to be grown are inputs to
edge that connects A to an isolated vertex-one on which no edge the algorithm. During execution of the algorithm, all vertices that
of A is incident. This rule adds only edges that are safe for A; are not in the tree reside in a min-priority queue Q based on a
therefore, when the algorithm terminates, the edges in A form a key attribute. For each vertex v , the attribute v .key is the mini-
minimum spanning tree. This strategy qualifies as greedy since mum weight of any edge connecting v to a vertex in the tree; by
at each step it adds to the tree an edge that contributes the convention, v .key = ∞ if there is no such edge. The attribute
minimum amount possible to the tree’s weight. v .π names the parent of v in the tree. When the algorithm termi-
nates, the min-priority queue Q is empty; the minimum spanning
tree A for G is thus
A = {(v , v .π) : v ∈ V − {r }}
The algorithms of Prim The algorithms of Prim
Prim’s Algorithm III Prim’s Algorithm IV

Minimum Spanning Trees Single-Source Shortest Paths
Prims algorithm complexity 1 Introduction

Types of Graphs
If adjacency list is used to represent the graph, all the ver- 2 Elementary Graph Algorithms
tices can be traversed in O(V + E) time. TRAVERSING A GRAPH
We traverse all the vertices of graph and use a min heap for Breadth-first search
Depth-first search
storing the vertices not yet included in the MST.
To get the minimum weight edge, we use min heap as a 3 Minimum Spanning Trees
priority queue. The algorithms of Kruskal
Min heap operations like extracting minimum element and
decreasing key value takes O(log V ) time. 4 Single-Source Shortest Paths
So, overall time complexity O(E log V ). This time complex- The Bellman-Ford algorithm
ity can be improved and reduced to O(E + V log V ) using
Fibonacci heap. 5 All-Pair Shortest Paths
Single-Source Shortest Paths Single-Source Shortest Paths
Single Source Shortest Path I Single Source Shortest Path II
It is a shortest path problem where the shortest path from a

We shall focus on the single-source shortest-paths problem: given a graph G =
given source vertex to all other remaining vertices is com- (V , E), we want to find a shortest path from a given source vertex s ∈ V to each
puted. vertex v ∈ V . The algorithm for the single-source problem can solve many other
problems, including the following variants.
Dijkstras Algorithm and Bellman Ford Algorithm are the fa-
Single-destination shortest-paths problem: Find a shortest path to a given
mous algorithms used for solving single-source shortest path destination vertex t from each vertex v . By reversing the direction of each edge
problem. in the graph, we can reduce this problem to a single-source problem.
Single-pair shortest-path problem: Find a shortest path from u to for given
vertices u and v . If we solve the single-source problem with source vertex u, we
solve this problem also. Moreover, all known algorithms for this problem have the
same worst-case asymptotic running time as the best single-source algorithms.
All-pairs shortest-paths problem: Find a shortest path from u to for every pair of
vertices u and v . Although we can solve this problem by running a single-source
algorithm once from each vertex, we usually can solve it faster.

Single Source Shortest Path III Single Source Shortest Path IV
In a shortest-paths problem, we are given a weighted, di-

rected graph G = (V , E), with weight function w : E → R
mapping edges to real-valued weights.
The weight w(p) of path p = hv0 , v1 , · · · , vk i is the sum of
the weights
P of its constituent edges:
w(p) = ki=1 w(vi−1 , vi ).
We define the shortest-path weight
A shortest path from vertex u to vertex v is then defined as

any path p with weight w(p) = δ(u, v ).
Single Source Shortest Path V Single Source Shortest Path VI

Properties of shortest paths and relaxation I 1 Introduction

Types of Graphs
Triangle inequality: For any edge (u, v ) ∈ E, we have δ(s, v ) ≤ δ(s, u) +
w(u, v ). 2 Elementary Graph Algorithms
Upper-bound property: We always have v .d ≥ δ(s, v ) for all vertices v ∈ V , TRAVERSING A GRAPH
and once v.d achieves the value δ(s, v ) it never changes. Breadth-first search
No-path property: If there is no path from s to v , then we always have v .d = Depth-first search
δ(s, v ) = ∞.
Convergence property: If s u → v is a shortest path in G for some u, v ∈ V , 3 Minimum Spanning Trees
and if u.d = δ(s, u) at any time prior to relaxing edge (u, v ), then v .d = δ(s, v ) The algorithms of Kruskal
at all times afterward. The algorithms of Prim
Path-relaxation property: If p = hv0 , v1 v , · · · , vk i is a shortest path from s = v0
to vk , and we relax the edges of p in the order (v0 , v1 ), (v1 , v2 ), · · · , (vk−1 , vk ), 4 Single-Source Shortest Paths
then vk .d = δ(s, k). This property holds regardless of any other relaxation steps
that occur, even if they are intermixed with relaxations of the edges of p.
Predecessor-subgraph property: Once v .d = δ(s, v ) for all v ∈ V , the prede-
cessor subgraph is a shortest-paths tree rooted at s. 5 All-Pair Shortest Paths
The Bellman-Ford algorithm The Bellman-Ford algorithm
Bellman Ford I Bellman Ford II
The Bellman-Ford algorithm solves the single-source shortest-

paths problem in the general case in which edge weights may The algorithm relaxes edges, progressively decreasing an es-
be negative. Given a weighted, directed graph G = (V , E) with timate v .d on the weight of a shortest path from the source
source s and weight function w : E → R, the Bellman-Ford algo- s to each vertex v ∈ V until it achieves the actual shortest-
rithm returns a boolean value indicating whether or not there is a path weight δ(s, v ). The algorithm returns TRUE if and only if
negative-weight cycle that is reachable from the source. If there the graph contains no negative-weight cycles that are reachable
is such a cycle, the algorithm indicates that no solution exists. from the source.
If there is no such cycle, the algorithm produces the shortest
paths and their weights.

Bellman Ford III Bellman Ford IV
After initializing the d and π values of all vertices in line 1, the al-
gorithm makes |V |−1 passes over the edges of the graph. Each
pass is one iteration of the for loop of lines 2 − 4 and consists
of relaxing each edge of the graph once. After making |V | − 1
passes, lines 5 − 8 check for a negative-weight cycle and re-
turn the appropriate boolean value. The Bellman-Ford algorithm
runs in time O(VE), since the initialization in line 1 takes θ(V )
time, each of the |V | − 1 passes over the edges in lines 2 − 4
takes θ(E) time, and the for loop of lines 5 − 7 takes O(E) time.
Bellman Ford V Bellman ford algorithm complexity
Time Complexity of Bellman Ford algorithm is relatively high

O(VE), in case E = V 2 , O(V 3 )

Dijkstra’s algorithm Dijkstra’s algorithm
1 Introduction Dijkstra’s algorithm I

Types of Graphs

Dijkstra’s algorithm solves the single-source shortest-paths prob-
TRAVERSING A GRAPH
Breadth-first search lem on a weighted, directed graph G = (V , E) for the case
Depth-first search in which all edge weights are non negative. We assume that
w(u, v ) ≥ 0 for each edge (u, v ) ∈ E. Dijkstra’s algorithm main-
3 Minimum Spanning Trees tains a set S of vertices whose final shortest-path weights from
the source s have already been determined. The algorithm re-
peatedly selects the vertex u ∈ V −S with the minimum shortest-
4 Single-Source Shortest Paths path estimate, adds u to S, and relaxes all edges leaving u. In
The Bellman-Ford algorithm the following implementation, we use a min-priority queue Q of
Dijkstra’s algorithm vertices, keyed by their d values.
Dijkstra’s algorithm II Dijkstra’s algorithm III
Line 1 initializes the d and π values in the usual way, and line
2 initializes the set S to the empty set. The algorithm maintains
the invariant that Q = V − S at the start of each iteration of the
while loop of lines 4 − 8. Line 3 initializes the min-priority queue
Q to contain all the vertices in V ; since S = φ; at that time, the
invariant is true after line 3. Each time through the while loop of
lines 4 − 8, line 5 extracts a vertex u from Q = V − S and line 6
adds it to set S, thereby maintaining the invariant. (The first time
through this loop, u = s.) Vertex u, therefore, has the smallest
shortest-path estimate of any vertex in V − S. Then, lines 7 − 8
relax each edge (u, v ) leaving u, thus updating the estimate v .d
and the predecessor v .π if we can improve the shortest path to
v found so far by going through u.

Dijkstra’s algorithm IV Dijkstra’s algorithm V
Observe that the algorithm never inserts vertices into Q after

line 3 and that each vertex is extracted from Q and added to S
exactly once, so that the while loop of lines 4 − 8 iterates exactly
|V | times.
Because Dijkstra’s algorithm always chooses the “lightest” or
“closest” vertex in V − S to add to set S, we say that it uses
a greedy strategy.
Single-Source Shortest Paths All-Pair Shortest Paths
Dijkstra’s Algorithm complexity 1 Introduction

Types of Graphs

TRAVERSING A GRAPH
Time Complexity of Dijkstra’s Algorithm is O(V 2 ). Breadth-first search
We can improve the algorithm by implementing the min- Depth-first search
priority queue with a binary min-heap. The total running 3 Minimum Spanning Trees
time is therefore O((V + E) log V ), which is O(E log V ) if all The algorithms of Kruskal
vertices are reachable from the source. The algorithms of Prim
We can in fact achieve a running time of O(V log V + E) by
implementing the min-priority queue with a Fibonacci heap The Bellman-Ford algorithm


All-Pair Shortest Paths All-Pair Shortest Paths
All Pairs Shortest Path 1 Introduction

Types of Graphs
It is a shortest path problem where the shortest path be- Representation of a Graph
tween every pair of vertices is computed. 2 Elementary Graph Algorithms
Floyd-Warshall Algorithm and Johnsons Algorithm are the TRAVERSING A GRAPH
famous algorithms used for solving All pairs shortest path Breadth-first search
Depth-first search
problem.


The Floyd-Warshall algorithm The Floyd-Warshall algorithm
Floyd Warshall Algorithm Algorithm I
The key idea of the algorithm is to partition the process of

finding the shortest path between any two vertices to sev-
The Floyd Warshall algorithm provides an alternative way to eral incremental phases.
approach the problem of finding shortest paths.Unlike other Let us number the vertices starting from 1 to n. The matrix
algorithms it finds all shortest paths in a single run. of distances is d[][].
The algorithm maintains a two dimensional array that con- Before the k-th phase k = {1, 2, . . . , n} d[i][j] for any two
tains distances between the nodes. vertices i and j stores the length of the shortest path be-
First distances are calculated only using direct edges between the vertex i and vertex j which contains only the ver-
tween the nodes and after this the algorithm reduces the tices 1, 2, . . . , k − 1 as internal vertices in the path.
distances using intermediate nodes in paths. In other words before k-th phase the value of d[i][j] is equal
to the length of the shortest path from vertex i to the vertex
j, if this path is allowed to enter only the vertex with numbers
smaller than k
Algorithm II Algorithm III
The algorithm works as follows:
For k = 0, we can fill matrix with d[i][j] = Wij if there exists

an edge between i and j with weight wij and d[i][j] = ∞ if
there doesn’t exist an edge.
We can calculate the distance matrix for the k -th phase in
the following way :
dnew [i][j] = min(d[i][j], d[i][k] + d[k ][j])
The time-complexity is O(n3 )

Example-1 I Example-1 II
Consider the weighted graph G in Fig. Assume v1 = R, v2 = S,
v3 = T and v4 = U. Then the weight matrix W of G is as follows:
Applying the algorithm, we obtain the following matrices Q0 , Q1 ,

Q2 , Q3 and Q4 = Q. To the right of each matrix Qk , we show
the matrix of paths which correspond to the lengths in the matrix
Qk .
Example-1 III Example-1 IV
We indicate how the circled entries are obtained:

Ql [4, 2] = MIN(Q0 [4, 2], Q0 [4, 1] + Q0 [1, 2]) = MIN(∞, 4 + 5) = 9
Q2 [1, 3] = MIN(Q1 [1, 3], Q1 [1, 2] + Q1 [2, 3]) = MIN(∞, 5 + ∞) = ∞
Q3 [4, 2] = MIN(Q2 [4, 2], Q2 [4, 3] + Q2 [3, 2]) = MIN(9, 3 + 1) = 4
Q4 [3, 1] = MIN(Q3 [3, 1], Q3 [3, 4] + Q3 [4, 1]) = MIN(10, 5 + 4) = 9
Consider the following graph
Initially, the distance from each node to itself is 0, and the dis-
tance between nodes a and b is x if there is an edge between
nodes a and b with weight x. All other distances are infinite


On the first round, node 1 is the new intermediate node. There
is a new path between nodes 2 and 4 with length 14, because
node 1 connects them. There is also a new path between nodes
2 and 5 with length 6.
On the second round, node 2 is the new intermediate node. This

creates new paths between nodes 1 and 3 and also nodes 3 and
5.
Example-2 V Example-2 VI
On the third round, node 3 is the new intermediate round. There

is a new path between nodes 2 and 4.
The algorithm continues like this, until all nodes have been ap-
pointed inter mediate nodes. After the algorithm has finished,
the array contains the minimum distances between any two nodes:

Graph Algorithms
All-Pair Shortest Paths
Conclusion
The advantage of the FloydWarshall algorithm is that it is

easy to implement.
However, the algorithm can only be used when the graph is
so small that a cubic time complexity is fast enough.
Algorithm Paradigms Algorithm Paradigms
Outline I
1 Introduction
2 Simple recursive algorithms
3 Backtracking algorithms
Algorithm Paradigms
4 Divide and conquer algorithms
5 Dynamic programming algorithms
6 Greedy algorithms
Dynamic programming vs Greedy approach
7 Branch and bound algorithms
8 Brute force algorithms
9 Randomized algorithms
10 Prune and search

1 Introduction Introduction I
An algorithmic paradigm or algorithm design paradigm is a
generic model or framework which underlies the design of
4 Divide and conquer algorithms a class of algorithms.
5 Dynamic programming algorithms An algorithmic paradigm is an abstraction higher than the
notion of an algorithm, just as an algorithm is an abstraction
6 Greedy algorithms higher than a computer program.
Algorithms that use a similar problem-solving approach can
7 Branch and bound algorithms be grouped together.
This classification scheme is neither exhaustive nor disjoint.
The purpose is not to be able to classify an algorithm as one
9 Randomized algorithms type or another, but to highlight the various ways in which a
problem can be attacked.
10 Prune and search
Introduction Simple recursive algorithms
A short list of categories I 1 Introduction
Algorithm types we will consider include:
I Simple recursive algorithms
I Backtracking algorithms 5 Dynamic programming algorithms
I Divide and conquer algorithms
I Dynamic programming algorithms 6 Greedy algorithms
I Greedy algorithms Dynamic programming vs Greedy approach
I Branch and bound algorithms
I Brute force algorithms 7 Branch and bound algorithms
I Randomized algorithms 8 Brute force algorithms
I Prune and search
10 Prune and search

Simple recursive algorithms Simple recursive algorithms
Simple recursive algorithms I Example recursive algorithms I
To count the number of elements in a list:

I If the list is empty, return zero; otherwise,
A simple recursive algorithm:
I Step past the first element, and count the remaining ele-
I Solves the base cases directly.
ments in the list
I Recurs with a simpler subproblem.
I Add one to the result
I Does some extra work to convert the solution to the simpler
subproblem into a solution to the given problem. To test if a value occurs in a list:
I If the list is empty, return false; otherwise,
We call these “simple” because several of the other algo- I If the first thing in the list is the given value, return true; oth-
rithm types are inherently recursive. erwise
I Step past the first element, and test whether the value oc-
curs in the remainder of the list
Backtracking algorithms Backtracking algorithms
1 Introduction Backtracking algorithms I

Backtracking is a technique used to solve problems with a
large search space, by systematically trying and eliminating
4 Divide and conquer algorithms possibilities
Backtracking algorithms are based on a depth-first recur-
sive search
6 Greedy algorithms A backtracking algorithm:
Dynamic programming vs Greedy approach I Tests to see if a solution has been found, and if so, returns
it; otherwise
7 Branch and bound algorithms I For each choice that can be made at this point,
• Make that choice
8 Brute force algorithms • Recur
• If the recursion returns a solution, return it
I If no choices remain, return failure
10 Prune and search Examples: Find path through maze, eight queens problem
Backtracking algorithms Backtracking algorithms
Backtracking algorithms II Backtracking algorithms III
Finding Path through maze: Eight Queens Problem: Find an arrangement of 8 queens
on a single chess board such that no two queens are at-
tacking one another.
1. Place a queen on the first available square in row 1.
2. Move onto the next row, placing a queen on the first available
square there (that doesn’t conflict with the previously placed
queens).
Backtracking algorithms Divide and conquer algorithms
Backtracking algorithms IV 1 Introduction
To color a map with no more than four colors:
color(Country n) 4 Divide and conquer algorithms
I If all countries have been colored (n > number of countries)
return success; otherwise, 5 Dynamic programming algorithms
I For each color c of four colors: If country n is not adjacent
to a country that has been colored c 6 Greedy algorithms
• Color country n with color c
• Recursivly color country n + 1
• If successful, return success
I Return failure (if loop exits) 8 Brute force algorithms
10 Prune and search

Divide and conquer algorithms Divide and conquer algorithms
Divide and conquer algorithms I Divide and conquer algorithms II
Divide and Conquer is an algorithmic pattern. In algorithmic

methods, the design is to take a dispute on a huge input,
break the input into minor pieces, decide the problem on
each of the small pieces, and then merge the piecewise
solutions into a global solution. This mechanism of solving
the problem is called the Divide & Conquer Strategy.
Divide and Conquer algorithm consists of a dispute using
the following three steps.
1. Divide the original problem into a set of subproblems.
2. Conquer: Solve every subproblem individually, recursively.
3. Combine: Put together the solutions of the subproblems to
get the solution to the whole problem.
Generally, we can follow the divide-and-conquer approach
in a three-step process.
Divide and conquer algorithms III Fundamental of Divide & Conquer Strategy I
There are two fundamental of Divide & Conquer Strategy:

Examples: The specific computer algorithms are based on 1. Relational Formula: It is the formula that we generate from
the Divide & Conquer approach: the given technique. After generation of Formula we apply
I Maximum and Minimum Problem D&C Strategy, i.e., we break the problem recursively & solve
I the broken subproblems.
Binary Search
I 2. Stopping Condition: When we break the problem using
Sorting (merge sort, quick sort)
Divide & Conquer Strategy, then we need to know that for
I Tower of Hanoi. how much time, we need to apply divide & Conquer. So the
condition where the need to stop our recursion steps of D&C
is called as Stopping Condition.

Applications of Divide and Conquer Approach I Applications of Divide and Conquer Approach II
2. Quicksort: It is the most efficient sorting algorithm, which is

Following algorithms are based on the concept of the Divide also known as partition-exchange sort. It starts by selecting
and Conquer Technique: a pivot value from an array followed by dividing the rest of the
1. Binary Search: The binary search algorithm is a searching array elements into two sub-arrays. The partition is made by
algorithm, which is also called a half-interval search or log- comparing each of the elements with the pivot value. It com-
arithmic search. It works by comparing the target value with pares whether the element holds a greater value or lesser
the middle element existing in a sorted array. After making value than the pivot and then sort the arrays recursively.
the comparison, if the value differs, then the half that cannot 3. Merge Sort: It is a sorting algorithm that sorts an array by
contain the target will eventually eliminate, followed by con- making comparisons. It starts by dividing an array into sub-
tinuing the search on the other half. We will again consider array and then recursively sorts each of them. After the sort-
the middle element and compare it with the target value. The ing is done, it merges them back.
process keeps on repeating until the target value is met. If 4. Closest Pair of Points: It is a problem of computational
we found the other half to be empty after ending the search, geometry. This algorithm emphasizes finding out the closest
then it can be concluded that the target is not present in the pair of points in a metric space, given n points, such that the
array. distance between the pair of points should be minimal.
Applications of Divide and Conquer Approach III Advantages of Divide and Conquer I
5. Strassen’s Algorithm: It is an algorithm for matrix multipli- Divide and Conquer tend to successfully solve one of the
cation, which is named after Volker Strassen. It has proven biggest problems, such as the Tower of Hanoi, a mathemat-
to be much faster than the traditional algorithm when works ical puzzle. It is challenging to solve complicated problems
on large matrices. for which you have no basic idea, but with the help of the
6. Cooley-Tukey Fast Fourier Transform (FFT) algorithm: divide and conquer approach, it has lessened the effort as
The Fast Fourier Transform algorithm is named after J. W.
it works on dividing the main problem into two halves and
Cooley and John Turkey. It follows the Divide and Conquer
Approach and imposes a complexity of O(nlogn). then solve them recursively. This algorithm is much faster
7. Karatsuba algorithm for fast multiplication: It is one of than other algorithms.
the fastest multiplication algorithms of the traditional time, in- It efficiently uses cache memory without occupying much
vented by Anatoly Karatsuba in late 1960 and got published space because it solves simple subproblems within the cache
in 1962. It multiplies two n-digit numbers in such a way by memory instead of accessing the slower main memory.
reducing it to at most single-digit.
It is more proficient than that of its counterpart Brute Force
technique.

Advantages of Divide and Conquer II Disadvantages of Divide and Conquer I
Since most of its algorithms are designed by incorporating

Since these algorithms inhibit parallelism, it does not involve
recursion, so it necessitates high memory management.
any modification and is handled by systems incorporating
parallel processing. An explicit stack may overuse the space.
It may even crash the system if the recursion is performed
rigorously greater than the stack present in the CPU.
Dynamic programming algorithms Dynamic programming algorithms
1 Introduction Dynamic programming algorithms I

Dynamic programming is a technique that breaks the prob-
lems into sub-problems, and saves the result for future pur-
5 Dynamic programming algorithms poses so that we do not need to compute the result again.
The subproblems are optimized to optimize the overall so-
6 Greedy algorithms lution is known as optimal substructure property. The main
Dynamic programming vs Greedy approach use of dynamic programming is to solve optimization prob-
7 Branch and bound algorithms lems. Here, optimization problems mean that when we are
trying to find out the minimum or the maximum solution of a
8 Brute force algorithms problem. The dynamic programming guarantees to find the
optimal solution of a problem if the solution exists.
10 Prune and search

Dynamic programming algorithms II Dynamic programming algorithms III
Let’s understand this approach through an example.

Consider an example of the Fibonacci series. The following
series is the Fibonacci series:
The definition of dynamic programming says that it is a tech-
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, · · · .
nique for solving a complex problem by first breaking into a
The numbers in the above series are not randomly calcu-
collection of simpler subproblems, solving each subproblem
lated. Mathematically, we could write each of the terms us-
just once, and then storing their solutions to avoid repetitive
ing the below formula:
computations.
F (n) = F (n − 1) + F (n − 2),
With the base values F (0) = 0, and F (1) = 1. To calculate
the other numbers, we follow the above relationship. For
example, F (2) is the sum f (0) and f (1), which is equal to 1.
Dynamic programming algorithms IV Dynamic programming algorithms V

The F (20) term will be calculated using the nth formula
of the Fibonacci series. The below figure shows that how As we can observe in the above figure that F(20) is calcu-
F (20) is calculated. lated as the sum of F(19) and F(18). In the dynamic pro-
gramming approach, we try to divide the problem into the
similar subproblems. We are following this approach in the
above case where F(20) into the similar subproblems, i.e.,
F(19) and F(18). If we recap the definition of dynamic pro-
gramming that it says the similar subproblem should not be
computed more than once. Still, in the above case, the sub-
problem is calculated twice. In the above example, F(18)
is calculated two times; similarly, F(17) is also calculated
twice. However, this technique is quite useful as it solves
the similar subproblems, but we need to be cautious while

Dynamic programming algorithms VI Dynamic programming algorithms VII

storing the results because we are not particular about stor-
ing the result that we have computed once, then it can lead
to a wastage of resources.
In the above example, if we calculate the F(18) in the right
subtree, then it leads to the tremendous usage of resources The value of F(20) can be calculated by adding the values
and decreases the overall performance. of F(19) and F(18), and the values of both F(19) and F(18)
are stored in an array. The final computed value of F(20) is
The solution to the above problem is to save the computed
stored in an array.
results in an array. First, we calculate F(16) and F(17) and
save their values in an array. The F(18) is calculated by
summing the values of F(17) and F(16), which are already
saved in an array. The computed value of F(18) is saved in
an array. The value of F(19) is calculated using the sum of
F(18), and F(17), and their values are already saved in an
array. The computed value of F(19) is stored in an array.
How does the dynamic programming approach work? How does the dynamic programming approach work?
I II
The above five steps are the basic steps for dynamic pro-
The following are the steps that the dynamic programming gramming. The dynamic programming is applicable that are
follows: having properties such as:
I It breaks down the complex problem into simpler subprob- Those problems that are having overlapping subproblems
lems. and optimal substructures. Here, optimal substructure means
I It finds the optimal solution to these sub-problems. that the solution of optimization problems can be obtained
I It stores the results of subproblems (memoization). The pro-
by simply combining the optimal solution of all the subprob-
cess of storing the results of subproblems is known as mem-
lems.
orization.
I It reuses them so that same sub-problem is calculated more In the case of dynamic programming, the space complex-
than once. ity would be increased as we are storing the intermediate
I Finally, calculate the result of the complex problem. results, but the time complexity would be decreased.

Approaches of dynamic programming I Top-down approach I
The top-down approach follows the memorization technique,

while bottom-up approach follows the tabulation method.
Here memorization is equal to the sum of recursion and
caching. Recursion means calling the function itself, while
There are two approaches to dynamic programming: caching means storing the intermediate results.
1. Top-down approach
Advantages
I It is very easy to understand and implement.
2. Bottom-up approach
I It solves the subproblems only when it is required.
I It is easy to debug.
Disadvantages
I It uses the recursion technique that occupies more memory
in the call stack. Sometimes when the recursion is too deep,
the stack overflow condition will occur.
I It occupies more memory that degrades the overall perfor-
mance.
Top-down approach II Top-down approach III
Let’s understand dynamic programming through an exam-

ple. In the above code, we have used the recursive approach
to find out the Fibonacci series. When the value of ’n’ in-
creases, the function calls will also increase, and compu-
tations will also increase. In this case, the time complexity
increases exponentially, and it becomes 2n .
One solution to this problem is to use the dynamic program-
ming approach. Rather than generating the recursive tree
again and again, we can reuse the previously calculated
value. If we use the dynamic programming approach, then
the time complexity would be O(n).

Top-down approach IV Top-down approach V

When we apply the dynamic programming approach in the
implementation of the Fibonacci series, then the code would
look like:
In the above code, we have used the memorization tech-
nique in which we store the results in an array to reuse
the values. This is also known as a top-down approach
in which we move from the top and break the problem into
sub-problems.
Bottom-Up approach I Bottom-Up approach II
The bottom-up is the approach used to avoid the recursion,

The bottom-up approach is also one of the techniques which
thus saving the memory space. The bottom-up is an algo-
can be used to implement the dynamic programming. It
rithm that starts from the beginning, whereas the recursive
uses the tabulation technique to implement the dynamic
algorithm starts from the end and works backward. In the
programming approach. It solves the same kind of prob-
bottom-up approach, we start from the base case to find
lems but it removes the recursion. If we remove the re-
the answer for the end. As we know, the base cases in the
cursion, there is no stack overflow issue and no overhead
Fibonacci series are 0 and 1. Since the bottom approach
of the recursive functions. In this tabulation technique, we
starts from the base cases, so we will start from 0 and 1.
solve the problems and store the results in a matrix.

Bottom-Up approach III Bottom-Up approach IV
Let’s understand through an example.

Suppose we have an array that has 0 and 1 values at a[0]
Key points: and a[1] positions, respectively shown as below:
I We solve all the smaller sub-problems that will be needed to
solve the larger sub-problems then move to the larger prob-
lems using smaller sub-problems.
I We use for loop to iterate over the sub-problems. Since the bottom-up approach starts from the lower values,
I The bottom-up approach is also known as the tabulation or
so the values at a[0] and a[1] are added to find the value of
table filling method.
a[2] shown as below:
Bottom-Up approach V Bottom-Up approach VI
The value of a[3] will be calculated by adding a[1] and a[2],

and it becomes 2 shown as below:
The value of a[5] will be calculated by adding the values of
a[4] and a[3], and it becomes 5 shown as below:
The value of a[4] will be calculated by adding a[2] and a[3],

and it becomes 3 shown as below:

Bottom-Up approach VII Bottom-Up approach VIII

The code for implementing the Fibonacci series using the
bottom-up approach is given below:
Let’s understand through the diagrammatic representation.

Initially, the first two values, i.e., 0 and 1 can be represented
as:
In the above code, base cases are 0 and 1 and then we

have used for loop to find other values of Fibonacci series.
Bottom-Up approach IX Bottom-Up approach X
When i=3 then the values 1and 1 are added shown as be-
When i=2 then the values 0 and 1 are added shown as below:
low:

Bottom-Up approach XI Bottom-Up approach XII

When i=5, then the values 3 and 2 are added shown as
When i=4 then the values 2 and 1 are added shown as be- below:
low:
In the above case, we are starting from the bottom and

reaching to the top.
Dynamic programming algorithms Greedy algorithms
Divide and Conquer Method vs Dynamic Programming 1 Introduction
I 2 Simple recursive algorithms
6 Greedy algorithms
10 Prune and search

Greedy algorithms Greedy algorithms
Greedy algorithms I Greedy algorithms II
Greedy Algorithm solves problems by making the best choice

that seems best at the particular moment. Many optimiza-
tion problems can be determined using a greedy algorithm.
“Greedy Method finds out of many options, but you have to Some issues have no efficient solution, but a greedy al-
choose the best option.” gorithm may provide a solution that is close to optimal. A
greedy algorithm works if a problem exhibits the following
In this method, we have to find out the best method/option
two properties:
out of many present ways.
1. Greedy Choice Property: A globally optimal solution can
In this approach/method we focus on the first stage and de- be reached at by creating a locally optimal solution. In other
cide the output, don’t think about the future. words, an optimal solution can be obtained by creating “greedy”
choices.
This method may or may not give the best output.
2. Optimal substructure: Optimal solutions contain optimal
subsolutions. In other words, answers to subproblems of an
optimal solution are optimal.
Example I Steps for achieving a Greedy Algorithm are I
Machine scheduling Feasible: Here we check whether it satisfies all possible

Fractional Knapsack Problem constraints or not, to obtain at least one solution to our prob-
lems.
Minimum Spanning Tree
Local Optimal Choice: In this, the choice should be the
Huffman Code optimum which is selected from the currently available
Job Sequencing Unalterable: Once the decision is made, at any subse-
Activity Selection Problem quence step that option is not altered.

Dynamic programming vs Greedy approach Dynamic programming vs Greedy approach
1 Introduction Dynamic programming vs Greedy approach I

5 Dynamic programming algorithms Before understanding the differences between the dynamic pro-
6 Greedy algorithms gramming and greedy approach, we should know about the dy-
Dynamic programming vs Greedy approach namic programming and greedy approach separately.
10 Prune and search

What is Greedy method? I What is Greedy method? II

Let’s understand through an example.
Suppose we want to travel from location A to location B. We
can travel from A to B through various ways like walk, bike,
car, bus, train, flight, etc. So, we have various solutions to
achieve the goal. But there is one constraint in this problem
A Greedy method is one of the techniques used to solve
that we have to reach B within 12 hr. We can achieve this
problems. This method is used for solving optimization prob-
goal either by using flight or train. So, we have two solutions
lems. The optimization problems are the problems that re-
in this case, i.e., flight or train. These types of solutions are
quire either maximum or minimum results.
feasible solutions. Feasible solutions are those solutions
that satisfy the constraint given in the problem.
Suppose we want to cover this distance at the minimum
cost, then it is known as a minimization problem. The min-
imization problem is a problem that demands a minimum
cost. In the above problem, if we go by train then it would

What is Greedy method? III What is Greedy method? IV

incur a minimum cost to provide an optimal solution. The Algorithm of Greedy method
optimal solution is the solution that provides the feasible so-
lution and incurs the minimum cost. It means that there can
be only one optimal solution, i.e., multiple solutions are not
possible.
This problem requires a minimum result but some problems
require a maximum result. The problems that require ei-
ther maximum or minimum results are known as optimiza-
tion problems.
A Greedy method says that a problem should be solved in
stages. In each stage, we consider the input from the given
problem and if the input is feasible then we include it. In
this way, we include all the feasible inputs to find an optimal
solution.
What is Greedy method? V What is dynamic programming? I
Dynamic programming is one of the techniques used to

solve the optimization problems. The following is the pro-
cedures that dynamic programming follows to achieve the
In the above greedy algorithm, we have passed two param- optimal solution:
eters, i.e., a and n where ’a’ is the array and n is the size of I It breaks down the complex problem into simpler sub-problems.
the array. In the for loop, we select input values from ’a’ and I We will find the optimal solution to these sub-problems.
then we check whether it is feasible or not. It the selected I Storing the results of these sub-problems.
input is feasible then add that input to the solution. I We will reuse the solution of these sub-problems so that we
do not need to re-calculate them.
I Finally, we will calculate the result of the complex problem.
The above are the five steps used in dynamic programming
to solve the optimization problem. The dynamic program-
ming is applicable to those problems that have certain prop-
erties given as below:

What is dynamic programming? II Differences between Dynamic programming and

Greedy method I
I Overlapping subproblems: The overlapping problems are

those that have common solutions.
I Optimal substructure: The optimal substructure is a sub-
structure that can be obtained by the combination of all the
optimal solution of subproblems.
Greedy algorithms Branch and bound algorithms
Differences between Dynamic programming and 1 Introduction
Greedy method II 2 Simple recursive algorithms
6 Greedy algorithms
10 Prune and search

Branch and bound algorithms Branch and bound algorithms
Branch and bound algorithms I Example branch and bound algorithm I
Branch and bound algorithms are generally used for opti-

mization problems
Travelling salesman problem: A salesman has to visit each
I As the algorithm progresses, a tree of subproblems is formed of n cities (at least) once each, and wants to minimize total
I The original problem is considered the root problem distance travelled
I A method is used to construct an upper and lower bound for
I Consider the root problem to be the problem of finding the
a given problem
I At each node, apply the bounding methods shortest route through a set of cities visiting each city once
I Split the node into two child problems:
• If the bounds match, it is deemed a feasible solution to that
• Shortest route visiting city A first
particular subproblem
• If bounds do not match, partition the problem represented by • Shortest route not visiting city A first
that node, and make the two subproblems into children nodes I Continue subdividing similarly as the tree grows
I Continue, using the best known feasible solution to trim sec-
tions of the tree, until all nodes have been solved or trimmed
Brute force algorithms Brute force algorithms
1 Introduction Brute force algorithms I


Brute force is one of the easiest and straight forward ap-
5 Dynamic programming algorithms proach to solve a problem
6 Greedy algorithms Based on trying all possible solutions
Dynamic programming vs Greedy approach It is useful for solving smallsize instances of a problem
7 Branch and bound algorithms Generally most expensive approach Examples: Sequential
search, factors of number
10 Prune and search

Brute force algorithms Brute force algorithms
Brute force algorithms II Improving brute force algorithms I
A brute force algorithm simply tries all possibilities until a

satisfactory solution is found
Such an algorithm can be: Often, brute force algorithms require exponential time
I Optimizing: Find the best solution. This may require finding Various heuristics and optimizations can be used
all solutions, or if a value for the best solution is known, it I Heuristic: A “rule of thumb” that helps you decide which
may stop when any best solution is found possibilities to look at first
Example: Finding the best path for a travelling salesman I Optimization: In this case, a way to eliminate certain pos-
I Satisficing: Stop as soon as a solution is found that is good sibilites without fully exploring them
enough
Example: Finding a travelling salesman path that is within
10% of optimal
Randomized algorithms Randomized algorithms
1 Introduction Randomized algorithms I


A randomized algorithm uses a random number at least
5 Dynamic programming algorithms once during the computation to make a decision
I Example: In Quicksort, using a random number to choose a
6 Greedy algorithms
Dynamic programming vs Greedy approach pivot
I Example: Trying to factor a large prime by choosing random
7 Branch and bound algorithms numbers as possible divisors
10 Prune and search

Prune and search Prune and search
1 Introduction Prune and search I

Prune and search is a method of solving optimization prob-
lems suggested by Nimrod Megiddo in 1983.
4 Divide and conquer algorithms The basic idea of the method is a recursive procedure in
which at each step the input size is reduced (“pruned”) by a
constant factor 0 < p < 1. As such, it is a form of decrease
6 Greedy algorithms and conquer algorithm, where at each step the decrease is
Dynamic programming vs Greedy approach by a constant factor.
7 Branch and bound algorithms Let n be the input size, T (n) be the time complexity of the
whole prune-and-search algorithm, and S(n) be the time
8 Brute force algorithms complexity of the pruning step. Then T (n) obeys the follow-
ing recurrence relation: T (n) = S(n) + T (n(1 − p)).
This resembles the recurrence for binary search but has a
10 Prune and search larger S(n) term than the constant term of binary search.
Algorithm Paradigms
Prune and search
Prune and search II
In prune and search algorithms S(n) is typically at least lin-

ear (since the whole input must be processed). With this as-
sumption, the recurrence has the solution T (n) = O(S(n)).
This can be seen either by applying the master theorem
for divide-and-conquer recurrences or by observing that the
times for the recursive subproblems decrease in a geomet-
ric series.
In particular, Megiddo himself used this approach in his lin-
ear time algorithm for the linear programming problem when
the dimension is fixed and for the minimal enclosing sphere
problem for a set of points in space.
Hashing Hashing
Outline I
1 Introduction
Hashing
2 Hash functions
3 Collision Resolution
4 Open Addressing: Linear Probing and Modifications
5 Chaining
Hashing Hashing
Hashing
1 Introduction
2 Hash functions The search time of each algorithm discussed so far de-
pends on the number n of elements in the collection S of
data.
3 Collision Resolution A searching technique, called hashing or hash addressing,
which is essentially independent of the number n.
In hashing, the idea is to use a hash function that converts
a given key to a smaller number and uses the small number
as an index in a table called a hash table.
5 Chaining
Hashing Hashing
Hash functions Hash functions
Hash functions I
1 Introduction
The two principal criteria used in selecting a hash function

2 Hash functions are as follows:
1. First of all, the function H should be very easy and quick to
compute.
3 Collision Resolution 2. Second the function H should, as far as possible, uniformly
distribute the hash addresses throughout the set L so that
there are a minimum number of collisions. Naturally, there
is no guarantee that the second condition can be completely
4 Open Addressing: Linear Probing and Modifications fulfilled without actually knowing beforehand the keys and
addresses. However, certain general techniques do help.
One technique is to “chop” a key k into pieces and combine
5 Chaining the pieces in some way to form the hash address H(k ).
Hashing Hashing
Hash functions II Hash functions III
Some popular Hash functions are:

1. Division method
• Choose a number m larger than the number n of keys in K .
2. Midsquare method The key k is squared. Then the hash
(The number m is usually chosen to be a prime number or a
number without small divisors, since this frequently minimizes function H is defined by H(k ) = l where l is obtained by
the number of collisions.) deleting digits from both ends of k 2 . We emphasize that the
• The hash function H is defined by H(k ) = k (mod m) or same positions of k 2 must be used for all of the keys
H(k ) = k (mod m) + 1. Here k (mod m) denotes the remain-
der when k is divided m.
• The second formula is used when we want the hash addresses
to range from 1 to m rather than from 0 to m − 1.
Hashing Hashing
Hash functions IV Example I
3. Folding method
• The key k is partitioned into a number of parts k1 , k2 , . . . ,
Consider a company each of whose 68 employees is assigned
kr where each part, except possibly the last, has the same
number of digits as the required address. a unique 4-digit employee number. Suppose L consists of 100
• Then the parts are added together, ignoring the last carry two-digit addresses: 00, 01, 02, . . . , 99. We apply the above
That is, H(k) = k1 + k2 + · · · + kr where the leading-digit hash functions to each of the following employee numbers:
carries, if any, are ignored. Sometimes, for extra “milling”, the 3205, 7148, 2345
even-numbered parts, k2 , k4 , . . . are each reversed before
the addition.
Hashing Hashing
Division method: Choose a prime number m close to 99, such Midsquare method: The following calculations are performed:
as m = 97. Then H(3205) = 4, H(7148) = 67, H(2345) = 17
That is, dividing 3205 by 97 gives a remainder of 4, dividing
7148 by 97 gives a remainder of 67, and dividing 2345 by 97
gives a remainder of 17. In the case that the memory addresses
begin with 01 rather than 00, we choose that the function H(k ) =
k (mod m) + 1 to obtain:
H(3205) = 4 + 1 = 5,
H(7148) = 67 + 1 = 68, Observe that the fourth and fifth digits, counting from the right,
H(2345) = 17 + 1 = 18 are chosen for the hash address.
Hashing Hashing
Hash functions Collision Resolution
Example IV
1 Introduction
Folding method: Chopping the key k into two parts and adding
2 Hash functions
yields the following hash addresses: H(3205) = 32 + 05 = 37,
H(7148) = 71+48 = 19, H(2345) = 23+45 = 68. Observe that
the leading digit 1 in H(7148) is ignored. Alternatively, one may
want to reverse the second part before adding, thus producing
the following hash addresses:
H(3205) = 32 + 50 = 82,
H(7148) = 71 + 84 + 55,
H(2345) = 23 + 54 = 77
5 Chaining
Hashing Hashing
Collision Resolution Collision Resolution
Collision Resolution I Collision Resolution II
The efficiency of a hash function with a collision resolution

Suppose we want to add a new record R with key k to our
procedure is measured by the average number of probes
file F , but suppose the memory location address H(k ) is
(key comparisons) needed to find the location of the record
already occupied. This situation is called collision.
with a given key k. The efficiency depends mainly on the
The particular procedure that one chooses to resolve colli- load factor λ. Specifically, we are interested in the following
sion depends on many factors. One important factor is the two quantities:
ratio of the number n of keys in K (which is the number of S(λ) = average number of probes for a successful search
records in F ) to the number m of hash addresses in L. This U(λ) = average number of probes for an unsuccessful search
ratio, λ = n/m, is called the load factor.
Hashing Hashing
Open Addressing: Linear Probing and Modifications Open Addressing: Linear Probing and Modifications
Linear Probing I
1 Introduction
2 Hash functions Suppose that a new record R with key k is to be added to

the memory table T , but that the memory location with hash
address H(k ) = h is already filled.
3 Collision Resolution One natural way to resolve the collision is to assign R to the
first available location following T [h].
Accordingly, with such a collision procedure, we will search
4 Open Addressing: Linear Probing and Modifications for the record R in the table T by linearly searching the lo-
cations T [h], T [h+1], T [h+2], . . . until finding R or meeting
an empty location, which indicates an unsuccessful search.
5 Chaining
Hashing Hashing
Linear Probing II Example I
Suppose the table T has 11 memory locations, T [1], T [2], · · · ,

T [11], and suppose the file F consists of 8 records, A, B, C, D,
The above collision resolution is called linear probing. The
E, X , Y and Z , with the following hash addresses:
average numbers of probes for a successful search and for
an unsuccessful search are.
Suppose the 8 records are entered into the table T in the above
Here λ = n/m is the load factor. order. Then the file F will appear in memory as follows:
Hashing Hashing
Although Y is the only record with hash address H(k) = 5, the The average number U of probes for an unsuccessful search
record is not assigned to T [5], since T [5] has already been filled follows:
by E because of a previous collision at T [4]. Similarly, Z does
not appear in T [1].
The average number S of probes for a successful search follows:
The first sum adds the number of probes to find each of the 8
records, and the second sum adds the number of probes to find
an empty location for each of the 11 locations.
Hashing Hashing
Disadvantage Techniques I
One main disadvantage of linear probing is that records

tend to cluster, that is, appear next to one another, when Quadratic Probing: Suppose a record R with key k has
the load factor is greater than 50 percent the hash address H(k) = h. Then, instead of searching the
locations with addresses h, h + 1, h + 2, . . . , we linearly
Such a clustering substantially increases the average search
search the locations with addresses h, h + 1, h + 4, h + 9,
time for a record.
h + 16, . . . , h + i 2 , . . . . If the number m of locations in the
Two techniques to minimize clustering are:
table T is a prime number, then the above sequence will
I Quadratic probing
I Double hashing access half of the locations in T .
Hashing Hashing
Open Addressing: Linear Probing and Modifications Chaining
Techniques II
1 Introduction
2 Hash functions
Double Hashing: Here a second hash function H 0 is used
for resolving a collision, as follows. Suppose a record R with
key k has the hash addresses H(k) = h and H 0 (k) = h0 6=
m. Then we linearly search the locations with addresses h,
h + h0 , h + 2h0 , h + 3h0 , . . . . If m is a prime number, then the
above sequence will access all the locations in the table T .
5 Chaining
Hashing Hashing
Chaining Chaining
Chaining I Chaining II
In chaining first of all there is a table T in memory which

contains the records in F , except that T now has an addi-
tional field LINK which is used so that all records in T with The average number of probes, using chaining, for a suc-
the same hash address h may be linked together to form a cessful search and for an unsuccessful search are
linked list.
Second, there is a hash address table LIST which contains
pointers to the linked lists in T . Here the load factor λ = n/m may be greater than 1, since
Suppose a new record R with key k is added to the file the number m of hash addresses in L (not the number of
F . We place R in the first available location in the table T locations in T ) may be less than the number n of records in
and then add R to the linked list with pointer LIST [H(k)]. If F.
the linked lists of records are not sorted, then R is simply
inserted at the beginning of its linked list. Searching for a
record or deleting a record is nothing more than searching
for a node or deleting a node from a linked list.
Hashing Hashing
Chaining Chaining
Example I Example II
Consider again the data, where the 8 records have the following
hash addresses:
Using chaining, the records will appear in memory as pictured

in Fig. Observe that the location of a record R in table T is not
related to its hash address. A record is simply put in the first
node in the AVAIL list of table T . In fact, table T need not have
the same number of elements as the hash address table.
Hashing
Chaining
Disadvantage
The main disadvantage to chaining is that one needs 3m

memory cells for the data.
Specifically, there are m cells for the information field INFO,
there are m cells for the link field LINK, and there are m
cells for the pointer array LIST
Suppose each record requires only 1 word for its informa-
tion field. Then it may be more useful to use open address-
ing with a table with 3m locations, which has the load factor
λ ≤ 13 , than to use chaining to resolve collisions
String Matching Algorithms
Data Structures and Algorithms Dept. of CSE, IIT(ISM) Dhanbad 17/11/21 1

Outline
Introduction
Naïve or Brute-force search
Automaton search
Rabin-Karp algorithm
Knuth-Morris-Pratt (KMP) Algorithm

Introduction
String-matching is a very important subject in the wider domain
of text processing.
String-matching consists in finding one, or more generally, all the
occurrences of a string (more generally called a pattern) in a text.

Definition
We formalize the string matching problem as follows:
We assume that the text is denoted by T=T[0 … n-1]; its length is
equal to n.
The pattern is denoted by P=P[0 … m-1]; its length is equal
to m≤n.
Both strings are build over a finite set of character called
an alphabet denoted by S..
S. may be character alphabet from letters a to z. For binary
alphabet, S.={0,1} or DNA alphabet, S.={A,T,G,C}
String matching algorithms try to find places where one or several
patterns are found within a searched text.

Contd…
String-matching algorithms, scan the text with the help of a window which
size is generally equal to m.
They first align the left ends of the window and the text, then compare the
characters of the window with the characters of the pattern - this specific work
is called an attempt - and after a whole match of the pattern or after a
mismatch they shift the window to the right.
They repeat the same procedure again until the right end of the window goes
beyond the right end of the text.
This mechanism is usually called the sliding window mechanism.
We associate each attempt with the position j in the text when the window is
positioned on T[j .. j+m-1].
The Brute Force algorithm locates all occurrences of P in T in time O(mn).

Brute Force Algorithm
The brute force algorithm consists in checking, at all positions in
the text between 0 and n-m, whether an occurrence of the pattern
starts there or not.
Then, after each attempt, it shifts the pattern by exactly one
position to the right.
The brute force algorithm requires no preprocessing phase, and a
constant extra space in addition to the pattern and the text.
During the searching phase the text character comparisons can be
done in any order.
The time complexity of this searching phase is O(mn).

Contd…
Algorithm:
for ( j = 0; j <= n - m; j++ )
{
for ( i = 0; i < m && P[i] == T[i + j]; i++ );
if ( i >= m ) return j;
}
Main features:
It is easy (but slow) O(nm) algorithm
No preprocessing phase
Only constant extra space needed
Always shifts the window by exactly 1 position to the right
Comparisons can be done in any order
mn expected text characters comparisons

Example

Automaton Based Search

Building the Minimal DFA

Example
Automaton Search: x = abaa and y = ababbaabaaab

Contd…

Rabin-Karp Algorithm

Rabin-Karp Hashing Details

Rabin-Karp Algorithm

Example
Rabin-Karp with x = abaa and y = ababbaabaaab

Rabin-Karp Algorithm (searching multiple patterns)

Knuth-Morris-Pratt Algorithm

Knuth-Morris-Pratt Window Shift Idea

Knuth-Morris-Pratt Preprocessing

Knuth-Morris-Pratt Main Algorithm

Example
KMP with x = abaa and y = ababbaabaaab

References
http://www-igm.univ-mlv.fr/~lecroq/string/index.html
https://webpages.uncc.edu/ras/courses/StringMatching-2.pdf

Thank
You… Any
Queries...?

DSA Merged

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DSA Merged

Uploaded by

Copyright:

Available Formats

Data Structures and Algorithms

Dr. Arup Kumar Pal

Total Count: 5n+4

Dr. Arup Kumar Pal

Total Count: 5n+4

Data Structures and Algorithms

Data Structures and Algorithms

Dr. Arup Kumar Pal

Data Structures and Algorithms

Data Structures and Algorithms

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures Data are simply values or sets of values.

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures

4 Time Complexity 1. Primitive Data Structure

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Data structures are also classified as:

Primitive Data Structure I

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Primitive Data Structure II Non primitive Data Type I

These data types are available in most programming languages

Non primitive Data Type II Linear data structures I

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Linear data structures II Nonlinear data structures I

Examples of Linear Data Structure are Stack and Queue.

Nonlinear data structures II Nonlinear data structures III

Graph: Graph is a collection of nodes (Information) and connect-

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

I Traversing: Accessing each record exactly once so that

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures

Algorithm study comprises four distinct areas:

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures

An algorithm can be described in many ways- English- but careful 3 Complexity

1 Introduction to Data Structures

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Order of growth I Order of growth II

Order of growth III Order of growth IV

In fact, counting the number of operations in some “reasonable”

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Order of growth V Order of growth VI

Order of growth VII Order of growth VIII

Definition: We denote by an “elementary operation” any compu-

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Order of growth IX 1 Introduction to Data Structures

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures

1 Introduction to Data Structures

Consequently, if limn→∞ f (n)/g(n) exists, then 2 Introduction to Algorithms

lim f (n)/g(n) 6= ∞ implies f (n) = Ω(g(n)). 3 Complexity

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

where c is a constant strictly greater than 0.

1 Introduction to Data Structures

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

1 Introduction to Data Structures

1 Introduction to Data Structures

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

Transitivity: f (n) = Θ(g(n)) and g(n) = Θ(h(n)) =⇒ f (n) =

1 Introduction to Data Structures

Introduction to Analysis of Algorithms Introduction to Analysis of Algorithms

We define the space used by an algorithm to be the number of

Dr. Arup Kumar Pal