Design and Analysis of

Algorithms
Department of Computer Science
Academic Year 2023
Abebezerihun90@gmail.com

Complied by Abebe Z. 1
Chapter One: Introduction and Elementary Data Structures

• Chapter Objectives
• Review of basic Data structures and algorithms
• Understand how to store and organize data using data
structures.
• Analyze running time of algorithms.
• Understand how to compare the time efficiency of
algorithms that solve the same problem?

Complied by Abebe Z. 2
 Overview of Algorithms
• The step by step procedure to solve a problem is known as the
ALGORITHM.
• Algorithm is the procedure a software program uses to
manipulate the data in data structures.
• Algorithms manipulate the data in data structures in various
ways, such as searching for a particular data item and sorting the
data.
• An algorithm is a well-organized, pre-arranged, and defined
computational module that receives some values or set of values
as input and provides a single or set of values as output.

Complied by Abebe Z. 3
• From data structure point of view, following are
some important categories of algorithms;
• Search: Algorithm to search an item in a data
structure.
• Sort: Algorithm to sort an item in a certain order.
• Insert: Algorithm to search an item in a data
structure.
• Update: Algorithm to update/alter an existing item
in a data structure
• Delete: Algorithm to delete an existing item from a
data structure.

Complied by Abebe Z. 4
 Cont…
• Algorithm transforms data structures from one
state to another state in two ways:
• An algorithm may change the value held by
a data structure.
• An algorithm may change the data structure
itself.
• E.g. Statement of the sorting problem
• Input a sentence of a number <a1, a2, …an>
• Out put a permutation (reordering) <a1, a2… an>
such that a1 <a2 <…..<an

Complied by Abebe Z. 5
 Properties of Algorithm
Every algorithm possesses the following algorithm property
a) Input / Out put specified
Every algorithm should have a specified number of input values (or
quantities) which are externally supplied. We must specify:
• The type of data
• The amount of data
Every algorithm should have one or more output procedures.
b) Finiteness (It must terminate)
Every valid algorithm should have complete after a finite number of
steps.

Complied by Abebe Z. 6
 Cont…
c) Definiteness (Absence of Ambiguity)
• Each step must be clearly defined, having one and only one
interpretation.
• At each point in computation, one should be able to tell
exactly what happens next. Definiteness ensures that if the
algorithm is performed at different times or by different
agents (people or computers) using the same data, the output
will be the same.
d) Feasibility:
• It must compute correct answer for all possible legal inputs.
• It should be implementable.
Complied by Abebe Z. 7
Cont..
e) Effectiveness:
-it must be possible to perform each step exactly and in a finite amount
of time.
-It must be possible to perform each instruction.
f) Correctness:
- It must compute correct answer for all possible legal inputs.
-It is most important criteria to evaluate an algorithm.
g) Efficiency
Algorithm must solve the problem with the least amount of
computational resources, Such as time and space.

Complied by Abebe Z. 8
Cont..
h) Sequential
• Each step must have a unique defined preceding and
succeeding step.
• The first step (start step) and last step (halt step) must be
clearly noted.
i) Simplicity
• We would like our algorithms to be easy to understand and
easy to implement.
j) Completeness
• It must solve the problem completely.
k) Language independence:
• It must not depend on any one programming language
Complied by Abebe Z. 9
 How to write an Algorithm?
• There are no well-defined standards for writing
algorithms. It’s problem and computer resource
dependent.
• Algorithm are never written to support a particular
programming code.
• All programming languages share basic code constructs
like loops (do, for, while) , control statements (if-
else),etc.
• These common constructs can be used to write an
algorithm.
• Algorithm is written in step-by-step manner, but it’s not
always the case. Algorithm writing is a process and is
executed after the problem domain is well defined.
Complied by Abebe Z. 10
• That is, we should know the problem domain, for which
we are designing the solution.
• Example: Let’s learn algorithm-writing by using an
example.
• Problem: Design an algorithm to add two numbers and
display the result.

• Writing step number is optional.

• Exercise: Design an algorithm to find a factorial of
positive integer n? Complied by Abebe Z. 11
Maximum contiguous subsequence sum
problem

Given(possibly negative) integers a 1, a 2, . . ., an.

Find (and identify the sequence corresponding to)
the maximum value of
j

A
k i
k

.
The maximum contiguous subsequence sum is zero
if all the integers are negative.

Complied by Abebe Z. 12
Maximum contiguous subsequence sum
problem

What is the sequence corresponding to the maximum

contagious subsequence sum for the following set of
integers?
A:-5 6 -7 8 -1 2

Complied by Abebe Z. 13
 Maximum contiguous subsequence sum problem

-5 -5 6 6 -7 -7 8 8 -1 -1 2 2

-5 6 1 6 -7 -1 -7 8 1 8 -1 7 -1 2 1

-5 6 -7 -6 6 -7 8 7 -7 8 -1 0 8 -1 2 9

-5 6 -7 8 2 6 -7 8 -1 6 -7 8 -1 2 2

-5 6 -7 8 -1 1 6 -7 8 -1 2 8

-5 6 -7 8 -1 2 3

Complied by Abebe Z. 14
Subsequence : Algorithm1
void maxSubsequenceSum( const int a[ ], int n, int & seqStart, int & seqEnd,
int & maxSum){
int maxSum = 0;
for( int i = 0; i < n; i++ )
for( int j = i; j < n; j++ ){
int thisSum = 0;
for( int k= i; k<= j; k++){
thisSum += a[ k ];
}
if( thisSum > maxSum ){
maxSum = thisSum;
seqStart = i;
seqEnd = j ;
}
}
}
}
Complied by Abebe Z. 15
Subsequence : Algorithm2
void maxSubsequenceSum( const int a[ ], int n, int & seqStart,
int & seqEnd, int & maxSum){
int maxSum = 0;
for( int i = 0; i < n; i++ ){
int thisSum = 0;
for( int j = i; j < n; j++ ){
thisSum += a [ j ] ;
if ( thisSum > maxSum ){
maxSum = thisSum;
seqStart = i;
seqEnd = j ;
}
}
}
}
Complied by Abebe Z. 16
Subsequence : Algorithm3
void maxSubsequenceSum( const int a[ ], int n, int & seqStart, int &
seqEnd, int & maxSum){
int thisSum = 0;
int maxSum = 0;
for( int i = 0, j = 0; j < n; j++ ){
thisSum += a [ j ] ;
if( thisSum > maxSum ) {
maxSum = thisSum;
seqstart = i;
seqEnd = j ;
}else if ( thisSum < 0 ){
i=j+1;
thisSum = 0;
}
}
}
Complied by Abebe Z. 17
Algorithm…
The three solution for the subsequence problem all solves the
problem. There may be even other several solutions that
correctly solves the same problem.
Are all the algorithms the same?
Is correctness the only criteria we need to care?
Is there any other rationale in which we can prefer
one solution on the other?
What is a good algorithm?

Complied by Abebe Z. 18
A “Good” Algorithm? Quality of
algorithm

. Efficie
nt
& algo code(data
rithm s
analys tructure
is)

? Reada
b
imple le, ease of
Algorithm

m
maint entation, re
ai u
engin nable (soft sable,
eering ware
)
Complied by Abebe Z. 19
A “Good” Algorithm(Quality)

At the design stage of solving a particular problem, there

are two conflicting goals. These are:
1. To design an algorithm that is easy to understand, code
and design.
 This goal is the concern of software engineers.

2. To design an algorithm that makes efficient use of the

computer resources such as CPU and memory.
 This goal is the concern of data structures and
algorithm analysis.

Complied by Abebe Z. 20
 Algorithm Analysis
• Algorithm analysis: refers to the process of determining the amount
of computing time and storage space required by different
algorithms. It is a study of computer program performance and
resource usage.
• There are two approaches to measure the efficiency of algorithms
(run time of an algorithm).
• A priori Analysis (Theoretical): This is theoretical analysis of an
algorithms. Efficiency of an algorithms is measured by assuming
that all other factors e.g. processor speed, are constant and have no
effect on the implementation.
• A posterior Analysis (Empirical): the selected algorithm is
implemented by a programming language. This is then executed on
a target computer machine. In this analysis, actual statistics like
running time and space required are collected.
Complied by Abebe Z. 21
 Complexity Analysis
• It analyzes the cost of solving interesting problems by
measuring amount of resources needed such as execution time
and space required to store data.
• The complexity of an algorithm is a function describing the
efficiency of an algorithm in terms of the amount of data the
algorithm must process.
• There are two main complexity measure of the efficiency of an
algorithm.
• Time Complexity:-is a function describing the amount of time
an algorithm takes in-terms of the amount of input to the
algorithm.
• Space complexity:-is a function describing the amount of
memory (space) an algorithm takes in terms of the amount of
input to the algorithm. Complied by Abebe Z. 22
• When quantifying the performance of the algorithm in terms of
n T(n), we are mainly interested in how the time and space
requirements change as n grows large. When we try to quantify
the performance, there are three kinds of analysis of program
efficiency:-
• Worst – case ( usually):
• It is the amount of time algorithm takes on the worst possible
set of inputs.
• T(n) = maximum time of algorithm on any input of size n
• It compute the upper bound of T(n) , the longest running time
for any input size n.

Complied by Abebe Z. 23
 • Average Case:
• The amount of time the algorithm takes on an average set of
inputs.
• T(n)= expected time(mean time) of algorithm over all in puts
of size n.
• It compute the optimal (the one which is found most of the
time) bound of T(n), it also need assumption of statistical
distribution of inputs. Gives the average performance of an
algorithm.
• Best Case:-
• It is the amount of time the algorithm takes on the smallest
possible set of inputs. It cheats with a slow algorithm that
works fast on some input. It compute the lower bound of T(n).
Best case is the cheapest among all inputs of size n.
Complied by Abebe Z. 24
Calculate complexity of an algorithm
Rules for computing complexity.
• We assume an arbitrary time unit.
• Execution of one of the following operations takes time 1 .
Assignment statement.
Single read/write statement (I/0).
Single Boolean expression Evaluation.
Single arithmetic operation.
Function return.
• Running time of a selection statement (if, switch) is the time for
the condition evaluation plus the maximum of the running
times for the individual clauses in the selection.

Complied by Abebe Z. 25
Example 1
Sample Code
Count of Basic Operations (Time Units)

int count() • 1 for the assignment statement: int k=0

{
Int k=0; • 1 for the output statement.
cout<< “Enter an integer”; • 1 for the input statement.
cin>>n; • In the for loop:
for (i = 0;i < n;i++) • 1 assignment, n+1tests, and n increments.

k = k+1;
return 0;
•
n loops of 2 units for an assignment, and an addition .
• 1 for the return statement.
}

• T (n) = 1+1+1+(1+n+1+2n)+2n+1 = 5n+6

Complied by Abebe Z. 26
Example 2

Sample Code Count of Basic Operations

(Time Units)
int total(int n) • 1 for the assignment statement: int
sum=0
{
• In the for loop:
Int sum=0; • 1 assignment, n+1tests, and n
for (inti=1;i<=n;i+ increments.
• n loops of 2 units for an
+) assignment, and an addition.
sum=sum+i; • 1 for the return statement.
return sum;
} • T (n) = 1+ (1+n+1+2n)+2n+1 =
5n+4

Complied by Abebe Z. 27
Running Time Example 3
Example: Simple Loop
i = 1;
sum = 0;
while (i <= n)
{
sum += i;
i++;
}

28
Solution: Worst Case Analysis
Times Unit Times
1 assignment i=1 1
1 assignment sum=0 1
1 comparison i<=n n+1
1 Increment of 2 uniti++ n
1 compound assignment of 2 unit n
T(n) = 1 + 1 + (n+1)*1 + n*2 + n*2=5n + 3
Or T(n)=1 + 1 + + =2 + n + 1 + 4n=5n + 3
 The timerequired
 4 for this algorithm is proportional to n
n 1 n
1
i 1 i 1

29
Running Time Example 4(Iterative)
Example: Nested Loop
sum = 0;
for(i=1;i <= n; i++) {
for(j=1;j <= n; j++){
sum = sum + j + i;
}
}

Compiled By Atnafu J. 30
Solution: Worst Case Analysis

Example: Nested Loop

Time units Times sum = 0;
1 1
for(i=1;i <= n; i++) { 1 1
for(j=1;j <= n; j++){ 1 n+1
sum = sum + j; 2 n
} 1 n
} 1 n*(n +1)
2 n*n
2 n*n

T(n) = 1 + 1 + (n+1)*1 + n*2 + 1*n +n*(n+1)*1+n*n*2+n*n*2

= 5n2 + 5n + 3
n 1 n n 1 n n

T(n)=1 + 1 + 1 +  (2  1  1   4) =2 + n + 1 +  (5n 4) = 5n 2 + 5n + 3

i 1 i 1 j 1 j 1 i 1

 The time required for this algorithm is proportional to n2

Compiled By Atnafu J. 31
• It’s difficult to determine the actual running time of
any algorithm because it depends on different factors
such as;
• Programming language.
• Operating system
• The compiler being used.
• Other processes running on the system
• Processor speed and so on.

Complied by Abebe Z. 32
 Asymptotic Analysis
• Asymptotic analysis of an algorithm refers to defining the
mathematical boundation / framing of its run-time
performance.
• Using asymptotic analysis, we can very well conclude the best
case, average case, and worst case scenario of an algorithm.
• Asymptotic analysis is input bound i.e., if there's no input to
the algorithm, it is concluded to work in a constant time.
• Asymptotic analysis refers to computing the running time of
any operation in mathematical units of computation
• For example, the running time of one operation is
computed as f(n) and may be for another operation it is
computed as g(n2)
Complied by Abebe Z. 33
• Asymptotic Notations
• Following are the commonly used asymptotic
notations to calculate the running time complexity of
an algorithm.
• Big-Ο Notation
• Big-Ω Notation
• θ Notation

Complied by Abebe Z. 34
Big-Oh Notation
Definition 1: Let f(n) and g(n) be two functions. We
write:
f(n) = O(g(n)) or f = O(g)
(read "f of n is in big oh of g of n" or "f is in big oh of g")
if there exist two positive integer C and n0 such that f(n)
<= C * g(n) for all positive integers n > n0.

Compiled By Atnafu J. 35
Big-Oh…
It indicates the upper or highest growth rate that
the algorithm can have.
The statement “f(n) is in O(g(n))” means that the
growth rate of f(n) is no more than the growth rate of
g(n).
We can use the big-Oh notation to rank functions
according to their growth rate.

Compiled By Atnafu J. 36
Big O Example 1
Let T(n) = 5n + 3, prove that T(n)=O(n).
To show that T(n) = O(n), we have to show the
existence of a constant c for some sufficiently large
n as given in Definition 1 such that 5n + 3 <= cn
5n + 3 <= 5n + 3n n  1
n  1
5n + 3 <=8n
Therefore, if c = 8, we have shown that T(n) = O(n)
for all n>=1.

Compiled By Atnafu J. 37
Big O Example 2
Let T(n) = 12n7 - 6n5 + 10n2 – 5 prove that T(n)=O(n7).
To show that T(n) = O(n), we have to show the existence of a
constant c for some sufficiently large n as given in Definition
1 such that 12n7 - 6n5 + 10n2 – 5 <= cn
T(n) =12 n7 - 6n5 + 10n2 - 5
T(n) <= 12n7 + 6n7 + 10n7 + 5n7 n  1
T(n) <= 33n7 n  1

Therefore, if c = 33, we have shown that T(n) = O(n) for all

n>=1.
Compiled By Atnafu J. 38
Big-Ω Notation

Complied by Abebe Z. 39
Complied by Abebe Z. 40
θ Notation

Complied by Abebe Z. 41
Growth rates
• The growth rate for an algorithm is the rate at which
the cost of the algorithm grows as the size of its input
grows.

Complied by Abebe Z. 42
Common Asymptotic Analysis
• А list оf sоmе cоmmоn аsymptоtic nоtаtiоns is mеntiоnеd
bеlоw –
Constant O(1)

Logarithm O(Log N)

Linear O(N)

N log n O( N Log N)

Quadratic O(

Cubic O

Polynomial

Exponential

Complied by Abebe Z. 43
 Review of elementary Data Structure

• Data Structure is a way of collecting and organizing data in such a

way that we can perform operations on these data in an effective
way.
• Data structure is the representation of the logical relationship
existing between individual elements of data.
• Data Structures is about rendering data elements in terms of some
relationship, for better organization and storage.
• Data structure specifies;
• Organization of data
• Accessing methods
• Degree of associativity
• Processing alternatives for information
Complied by Abebe Z. 44
 Basic Data Structures

Basic Data Structures

Linear Data Structures Non-Linear Data Structures

Arrays Linked Lists Stacks Queues Trees Graphs Hash Tables

Complied by Abebe Z. 45
 array

Linked list

queue
tree stack

Complied by Abebe Z. 46
 Why Data Structure?
• To solve problem of real world data storage; in sorting, storing,
searching, deleting and so on.
• E.G. – Personnel record, Inventory record, Stack of index
cards.
• Used as a tools to facilitate some other operation by
programmer.
• E.g. stacks, queues and priority queues.
• To directly model a real-world situation. For instance;
• graph, which used to represent routes between locations,
• Telephone cable in city.
• Connections in an electrical circuit or task in project.
• Queues can model customers waiting in line at a bank.
• Allows major improvements in program efficiently.
Complied by Abebe Z. 47
 Abstract Data Types
• Abstract Data Types (ADTs) stores data and allow various
operations on the data to access and change it.
• A mathematical model, together with various operations defined
on the model.
• An ADT is a collection of data and associated operations for
manipulating that data.
• ADTs support abstraction, encapsulation, and
information hiding.
• Abstraction is the structuring of a problem into well-
defined entities by defining their data and operations.
• The principle of hiding the used data structure and to
only provide a well-defined interface is known as
encapsulation.
Complied by Abebe Z. 48
The Core Operations of ADT
• Every Collection ADT should provide a way to:
• add an item
• remove an item
• find, retrieve, or access an item

• Many, many more possibilities

• is the collection empty
• make the collection empty
• give me a sub set of the collection

Complied by Abebe Z. 49
Linked List: Definition
• Linked list is a concrete data structure consisting
of collection of elements called nodes in which
each node is linked to the next node in the list.
• The entire list is referenced by a separate pointer,
called head, that points to the first element of the
list.

Compiled By Abebe Z. 50
Node

A node is a basic unit of a data structure. Nodes

contain data and also may link to other nodes. Links
between nodes are often implemented by pointers
In addition to the data, each node contains a pointer,
which can point to the next node.

Compiled By Abebe Z. 51
How to define a Node
• A Node in linked list can be defined by using Structure.
• Structure is aggregate data types built using elements of
primitive data types.
• It is a user defined data type.
• Structure is defined as:
• struct struct-tag {
• Type1 member variable1;
• type2 member variable2;
• .
• Typen member variablen;
• }variable name1, variable name2;
Compiled By Abebe Z. 52
Example
• Struct Time1 {
• int minutes;
• int seconds;
• }T1, T2…;
• //declare Time1 data type variables T1, T2 etc. at the
time defining.
• //the user defined data type.
• Memory space is reserved for the structure when it is
declared.

Compiled By Abebe Z. 53
Stack
• A stack is an ordered
list in which all
insertions and deletions
are made at one end,
called the top.
• Operations:-
• Push()
• Pop()

Complied by Abebe Z. 54
 Queue
• A queue is an ordered list
in which all insertions take
place at one end, the rear,
while all deletions take
place at the other end, the
front
• Operations:-
• Enqueue()
• Dequeue()

Complied by Abebe Z. 55
 Tree
• A tree is a finite set of one or more nodes such that (i)
there is a specially designated node called the root;
• (ii) the remaining nodes are partitioned into n >=0
disjoint sets Tl, ... , Tn where each of these sets is
a tree. Tl, ... , Tn are called the subtrees of the root.

Complied by Abebe Z. 56
• Types of tree:-
• Full binary tree
• Complete binary
• Balanced binary tree
• Operations
• Traversal
• Search
• Deletion
• Insertion

Complied by Abebe Z. 57
Heaps
• A heap is a complete binary tree with the property that
the value at each node is at least as large as the values
at its children (if they exist).
• Heap:
• Max heap
• Min heap
• Max Heap:- root node value must be greater than or
equal among the keys present at all of its children.
• Min Heap:- root node value must be greater than or
equal among the keys present at all of its children

Complied by Abebe Z. 58
Example

Complied by Abebe Z. 59
 Graph
• A graph is a pictorial representation of a set of objects
where some pairs of objects are connected by links.
• The interconnected objects are represented by points
termed as vertices, and the links that connect the
vertices are called edges.
• It’s is a pair of sets (V, E), where V is the set of vertices
and E is the set of edges, connecting the pairs of
vertices.

Complied by Abebe Z. 60
• In the above graph,
• V = {a, b, c, d, e}
• E = {ab, ac, bd, cd, de}
• Used in Graph Theory
Complied by Abebe Z. 61
 Hashing
• Hashing is a data structure which allows one to easily
determine the presence or absence of an arbitrary element.
• Used for searching and improved searching performance to
O(1) from O(n) linear search and O(logn) binary search.
• Hashing in the data structure is a technique of mapping a
large chunk of data into small tables using a hashing
function
• It uses hash tables to store the data in an array format.
• Each value in the array has assigned a unique index number.

Complied by Abebe Z. 62
• Hash tables use a technique to generate these unique
index numbers for each value stored in an array format.
This technique is called the hash technique.
• Hashing in a data structure is a two-step process.
• The hash function converts the item into a small integer or hash
value. This integer is used as an index to store the original data.
• It stores the data in a hash table. You can use a hash key to
locate data quickly.
• A real life examples of hashing in data structure:
• In schools, the teacher assigns a unique roll number to
each student and uses that roll number to retrieve
information about that student.
• A library has an infinite number of books. The librarian
assigns a unique number to each book.
Complied by Abebe Z. 63
 Hash Function
• H(x) = x % size
• E.g. Array size is 10

Collision

• Solution: -
• Open hashing (chaining)
• Closed hashing (Open addressing)
Complied by Abebe Z. 64
 Disjoint Sets
• It is a data structure that contains a collection of
disjoint or non-overlapping sets.
• Set that is partitioned into the disjoint subsets.
• Operations:-
• Add new sets
• Merge the sets
• Find representative member of the set
• find out whether the two elements are in the same set or not.

Complied by Abebe Z. 65
 Union, Find operation

• S1={1,2,3,4}
• S2={5,6,7,8}
• Intersection of s1 and s2 is empty. //disjoint sets
• Add new edge (4,8) s3={1,2,3,4,5,6,7,8}
• Add new edge (1,5). 1 and 5 vertices in the same set,
which indicates there is a cycle in a graph.
Complied by Abebe Z. 66
• Applications of disjoint sets
• Calculating mutual friends
• Used to detect cycles in a graph
• Used to keep track of connected in an undirected
graph
• Used to calculate the minimum spanning tree in
Kruskal's algorithm
• Representing network connectivity

Complied by Abebe Z. 67