V Sem BCA- ADA Unit-1

DSC13: Design and Analysis of Algorithm

UNIT-1

INTRODUCTION

INTRODUCTION:

The word Algorithm means ” A set of finite rules or instructions to be followed

in calculations or other problem-solving operations ”
Or
” A procedure for solving a mathematical problem in a finite number of steps
that frequently involves recursive operations”.

The word algorithm comes from the name of Persian author:

Abu Ja’far Mohammed Ibu Musa Al Khowarismi [825 AD]

Al Khowarismi means[Algorithm]

Therefore Algorithm refers to a sequence of finite steps to solve a particular

problem.

Mr.Siddarama S-SBMJC-KGF Page 1

 V Sem BCA- ADA Unit-1

Use of the Algorithms:

Algorithms play a crucial role in various fields and have many applications.
Some of the key areas where algorithms are used include:

1. Computer Science: Algorithms form the basis of computer programming

and are used to solve problems ranging from simple sorting and searching
to complex tasks such as artificial intelligence and machine learning.

2. Mathematics: Algorithms are used to solve mathematical problems, such

as finding the optimal solution to a system of linear equations or finding
the shortest path in a graph.

3. Operations Research: Algorithms are used to optimize and make

decisions in fields such as transportation, logistics, and resource
allocation.

4. Artificial Intelligence: Algorithms are the foundation of artificial

intelligence and machine learning, and are used to develop intelligent
systems that can perform tasks such as image recognition, natural
language processing, and decision-making.

5. Data Science: Algorithms are used to analyze, process, and extract

insights from large amounts of data in fields such as marketing, finance,
and healthcare.

What is the need for algorithms?

Mr.Siddarama S-SBMJC-KGF Page 2

 V Sem BCA- ADA Unit-1

1. Algorithms are necessary for solving complex problems efficiently and

effectively.

2. They help to automate processes and make them more reliable, faster, and
easier to perform.

3. Algorithms also enable computers to perform tasks that would be difficult

or impossible for humans to do manually.

4. They are used in various fields such as mathematics, computer science,

engineering, finance, and many others to optimize processes, analyze
data, make predictions, and provide solutions to problems.

What are the Characteristics of an Algorithm?

 Clear and Unambiguous: The algorithm should be unambiguous. Each

of its steps should be clear in all aspects and must lead to only one
meaning.

 Well-Defined Inputs: If an algorithm says to take inputs, it should be

well-defined inputs. It may or may not take input.

 Well-Defined Outputs: The algorithm must clearly define what output

will be yielded and it should be well-defined as well. It should produce at
least 1 output.

 Finite-ness: The algorithm must be finite, i.e. it should terminate after a

finite time.

Mr.Siddarama S-SBMJC-KGF Page 3

 V Sem BCA- ADA Unit-1

 Feasible: The algorithm must be simple, generic, and practical, such that
it can be executed with the available resources. It must not contain some
future technology or anything.

 Language Independent: The Algorithm designed must be language-

independent, i.e. it must be just plain instructions that can be
implemented in any language, and yet the output will be the same, as
expected.

 Input: An algorithm has zero or more inputs. Each that contains a

fundamental operator must accept zero or more inputs.

 Output: An algorithm produces at least one output. Every instruction

that contains a fundamental operator must accept zero or more inputs.

 Definiteness: All instructions in an algorithm must be unambiguous,

precise, and easy to interpret. By referring to any of the instructions in an
algorithm one can clearly understand what is to be done. Every
fundamental operator in instruction must be defined without any
ambiguity.

 Finiteness: An algorithm must terminate after a finite number of steps in

all test cases. Every instruction which contains a fundamental operator
must be terminated within a finite amount of time. Infinite loops or
recursive functions without base conditions do not possess finiteness.

 Effectiveness: An algorithm must be developed by using very basic,

simple, and feasible operations so that one can trace it out by using just
paper and pencil.

Properties of Algorithm:

 It should terminate after a finite time.

 It should produce at least one output.

 It should take zero or more input.

 It should be deterministic means giving the same output for the same
input case.

 Every step in the algorithm must be effective i.e. every step should do
some work.

Mr.Siddarama S-SBMJC-KGF Page 4

 V Sem BCA- ADA Unit-1

Types of Algorithms:

There are several types of algorithms available. Some important

algorithms are:

1. Brute Force Algorithm:

It is the simplest approach to a problem. A brute force algorithm is the

first approach that comes to finding when we see a problem.

2. Recursive Algorithm:

A recursive algorithm is based on recursion. In this case, a problem is

broken into several sub-parts and called the same function again and
again.

3. Backtracking Algorithm:

The backtracking algorithm builds the solution by searching among all

possible solutions. Using this algorithm, we keep on building the solution
following criteria. Whenever a solution fails we trace back to the failure
point build on the next solution and continue this process till we find the
solution or all possible solutions are looked after.

4. Searching Algorithm:

Searching algorithms are the ones that are used for searching elements or
groups of elements from a particular data structure. They can be of
different types based on their approach or the data structure in which the
element should be found.

5. Sorting Algorithm:

Sorting is arranging a group of data in a particular manner according to

the requirement. The algorithms which help in performing this function
are called sorting algorithms. Generally sorting algorithms are used to
sort groups of data in an increasing or decreasing manner.

6. Hashing Algorithm:

Mr.Siddarama S-SBMJC-KGF Page 5

 V Sem BCA- ADA Unit-1

Hashing algorithms work similarly to the searching algorithm. But they

contain an index with a key ID. In hashing, a key is assigned to specific
data.

7. Divide and Conquer Algorithm:

This algorithm breaks a problem into sub-problems, solves a single sub-

problem, and merges the solutions to get the final solution. It consists of
the following three steps:

 Divide
 Solve
 Combine
8. Greedy Algorithm:
In this type of algorithm, the solution is built part by part. The solution
for the next part is built based on the immediate benefit of the next part.
The one solution that gives the most benefit will be chosen as the solution
for the next part.

9. Dynamic Programming Algorithm:

This algorithm uses the concept of using the already found solution to
avoid repetitive calculation of the same part of the problem. It divides the
problem into smaller overlapping subproblems and solves them.

10. Randomized Algorithm:

In the randomized algorithm, we use a random number so it gives

immediate benefit. The random number helps in deciding the expected
outcome.

Advantages of Algorithms:

 It is easy to understand.

 An algorithm is a step-wise representation of a solution to a given

problem.

 In an Algorithm the problem is broken down into smaller pieces or steps

hence, it is easier for the programmer to convert it into an actual program.

Disadvantages of Algorithms:

 Writing an algorithm takes a long time so it is time-consuming.

Mr.Siddarama S-SBMJC-KGF Page 6
 V Sem BCA- ADA Unit-1

 Understanding complex logic through algorithms can be very difficult.

 Branching and Looping statements are difficult to show in

Algorithms(imp)

How to Design an Algorithm?

To write an algorithm, the following things are needed as a pre-requisite:

 The problem that is to be solved by this algorithm i.e. clear problem

definition.

 The constraints of the problem must be considered while solving the

problem.

 The input to be taken to solve the problem.

 The output is to be expected when the problem is solved.

 The solution to this problem is within the given constraints.

Fundaments of Algorithmic problem solving:

Mr.Siddarama S-SBMJC-KGF Page 7

 V Sem BCA- ADA Unit-1

(i) Understanding the Problem

 This is the first step in designing of algorithm.

 Read the problem’s description carefully to understand the problem
statement completely.
 Ask questions for clarifying the doubts about the problem.
 Identify the problem types and use existing algorithm to find solution.
 Input (instance) to the problem and range of the input get fixed.

(ii) Decision making The Decision making is done on the following:

a) Ascertaining the Capabilities of the Computational Device In random-access

machine (RAM), instructions are executed one after another (The central
assumption is that one operation at a time). Accordingly,

algorithms designed to be executed on such machines are called sequential

algorithms.

Mr.Siddarama S-SBMJC-KGF Page 8

 V Sem BCA- ADA Unit-1

 In some newer computers, operations are executed concurrently, i.e., in

parallel. Algorithms that take advantage of this capability are called
parallel algorithms.
 Choice of computational devices like Processor and memory is mainly
based on space and time efficiency

a)Choosing between Exact and Approximate Problem Solving:

 The next principal decision is to choose between solving the problem

exactly or solving it approximately.
 An algorithm used to solve the problem exactly and produce correct
result is called an exact algorithm.
 If the problem is so complex and not able to get exact solution, then we
have to choose an algorithm called an approximation algorithm. i.e.,
produces an Approximate answer. E.g., extracting square roots, solving
nonlinear equations, and evaluating definite integrals.

a) Algorithm Design Techniques

 An algorithm design technique (or “strategy” or “paradigm”) is a general

approach to solving problems algorithmically that is applicable to a
variety of problems from different areas of computing.
 Though Algorithms and Data Structures are independent, but they are
combined together to develop program. Hence the choice of proper data
structure is required before designing the algorithm.
 Implementation of algorithm is possible only with the help of Algorithms
and Data Structures • Algorithmic strategy / technique / paradigm are a
general approach by which many problems can be solved algorithmically.

E.g., Brute Force, Divide and Conquer, Dynamic Programming, Greedy

Technique and soon.

(iii) Methods of Specifying an Algorithm

There are three ways to specify an algorithm. They are:

a. Natural language

b. Pseudocode

Mr.Siddarama S-SBMJC-KGF Page 9

 V Sem BCA- ADA Unit-1

c. Flowchart

a. Natural Language It is very simple and easy to specify an algorithm using

natural language. But many times specification of algorithm by using natural
language is not clear and thereby we get brief specification.

Example: An algorithm to perform addition of two numbers. Such a

specification creates difficulty while actually implementing it. Hence many
programmers prefer to have specification of algorithm by means of Pseudocode.

Example: An algorithm to perform addition of two numbers.

Step 1: Read the first number, say a.

tep 2: Read the first number, say b.

Step 3: Add the above two numbers and store the result in c.

Step 4: Display the result from c.

b) Pseudocode:

• Pseudocode is a mixture of a natural language and programming language

constructs. Pseudocode is usually more precise than natural language.

• For Assignment operation left arrow “←”, for comments two slashes “//”,if
condition, for, while loops are used.

Example:

ALGORITHM Sum(a,b)

//Problem Description: This algorithm performs addition of two numbers

//Input: Two integers a and b

//Output: Addition of two integers

c←a+b

return c

c)Flowchart

Mr.Siddarama S-SBMJC-KGF Page 10

 V Sem BCA- ADA Unit-1

 In the earlier days of computing, the dominant method for specifying

algorithms was a flowchart, this representation technique has proved to
be inconvenient.
 Flowchart is a graphical representation of an algorithm. It is a method of
expressing an algorithm by a collection of connected geometric shapes
containing descriptions of the algorithm’s steps.

(iv) Proving an Algorithm’s Correctness

 Once an algorithm has been specified then its correctness must be proved.
 An algorithm must yield a required result for every legitimate input in a
finite amount of time.
 For Example, the correctness of Euclid’s algorithm for computing the
greatest common

divisor stems from the correctness of the equality gcd(m, n) = gcd(n, m mod n).
A common technique for proving correctness is to use mathematical induction
because an algorithm’s iterations provide a natural sequence of steps needed for
such proofs.

The notion of correctness for approximation algorithms is less straightforward

than it is for exact algorithms. The error produced by the algorithm should not
exceed a predefined limit.

(v) Analyzing an Algorithm

For an algorithm the most important is efficiency. In fact, there are two kinds
of algorithm efficiency. They are:

Mr.Siddarama S-SBMJC-KGF Page 11

 V Sem BCA- ADA Unit-1

 Time efficiency, indicating how fast the algorithm runs, and

 Space efficiency, indicating how much extra memory it uses.
 The efficiency of an algorithm is determined by measuring both time
efficiency and space efficiency.

So factors to analyze an algorithm are:

 Time efficiency of an algorithm

 Space efficiency of an algorithm
 Simplicity of an algorithm
 Generality of an algorithm

(vi) Coding an Algorithm

 The coding / implementation of an algorithm is done by a suitable

programming language like C, C++,JAVA.
 The transition from an algorithm to a program can be done either
incorrectly or very inefficiently. Implementing an algorithm correctly is
necessary. The Algorithm power should not reduce by in efficient
implementation.
 Standard tricks like computing a loop’s invariant (an expression that does
not change its value) outside the loop, collecting common sub
expressions, replacing expensive operations by cheap ones, selection of
programming language and so on should be known to the programmer.
 Typically, such improvements can speed up a program only by a constant
factor, whereas a better algorithm can make a difference in running time
by orders of magnitude. But once an algorithm is selected, a 10–50%
speedup may be worth an effort.
 It is very essential to write an optimized code (efficient code) to reduce
the burden of compiler.

Fundamentals of the Analysis of Algorithm Efficiency:

 The Analysis Framework

The research experience has shown that for most problems, we can achieve
much more spectacular progress in speed than in space.

 Measuring an Input’s Size

Mr.Siddarama S-SBMJC-KGF Page 12

 V Sem BCA- ADA Unit-1

When measuring input size for algorithms solving problems such as checking
primality of a positive integer n. Here, the input is just one number, and it is this
number’s magnitude that determines the input size. In such situations, it is
preferable to measure size by the number b of bits in the n’s binary
representation: b=⌊log2n⌋+1

 Units for Measuring Running Time

Identify the most important operation of the algorithm, called the basic
operation, the operation contributing the most to the total running time, and
compute the number of times the basic operation is executed.

The established framework for the analysis of an algorithm’s time efficiency

suggests measuring it by counting the number of times the algorithm’s basic
operation is executed on inputs of size n.

Orders of Growth

The order of growth of an algorithm is an approximation of the time required to

run a computer program as the input size increases. The order of growth ignores
the constant factor needed for fixed operations and focuses instead on the
operations that increase proportional to input size.

The change in behaviour as the value of ‘n’ increases is called Order of

Growth

Normally the order of growth is determined for larges values of ‘n’ because of
the following two reasons:

i. The behaviour of the algorithm changes as the value of ‘n’ increases.

ii. Large values of ‘n’ are encountered in real time applications.

Worst-Case, Best-Case and Average-Case Efficiencies:

Worst Case Analysis:

Definition: The efficiency of an algorithm for the input of size ‘n’ for which the
algorithm takes longest time to execute among all possible inputs.

Mr.Siddarama S-SBMJC-KGF Page 13

 V Sem BCA- ADA Unit-1

In the worst-case analysis, we calculate the upper limit of the execution time of
an algorithm. It is necessary to know the case which causes the execution of the
maximum number of operations.

For linear search, the worst case occurs when the element to search for is not
present in the array. When x is not present, the search () function compares it
with all the elements of arr [] one by one. Therefore, the temporal complexity of
the worst case of linear search would be Θ (n).

Best Case Analysis:

Definition: The efficiency of an algorithm for the input of size ‘n’ for which the
algorithm takes least time to execute among all possible inputs.

In the best case analysis, we calculate the lower bound of the execution time of
an algorithm. It is necessary to know the case which causes the execution of the
minimum number of operations. In the linear search problem, the best case
occurs when x is present at the first location.

The number of operations in the best case is constant. The best-case time
complexity would therefore be Θ (1) Most of the time, we perform worst-case
analysis to analyze algorithms. In the worst analysis, we guarantee an upper
bound on the execution time of an algorithm which is good information.

Average Case Analysis:

In the average case analysis, we take all possible inputs and calculate the
computation time for all inputs. Add up all the calculated values and divide the
sum by the total number of entries.

We need to predict the distribution of cases. For the linear search problem,
assume that all cases are uniformly distributed. So we add all the cases and
divide the sum by (n + 1).

Algorithm Data Time Time Time Space

Mr.Siddarama S-SBMJC-KGF Page 14

 V Sem BCA- ADA Unit-1

structure complexity:Best complexity:Average complexity:Worst complexity:Worst

Quick sort Array O(n log(n)) O(n log(n)) O(n2) O(n)

Merge
Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)
sort

Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)

Smooth
Array O(n) O(n log(n)) O(n log(n)) O(1)
sort

Bubble
Array O(n) O(n2) O(n2) O(1)
sort

Insertion
Array O(n) O(n2) O(n2) O(1)
sort

Selection
Array O(n2) O(n2) O(n2) O(1)
sort

Bogo sort Array O(n) O(n n!) O(∞) O(1)

Time and Space Complexity (Efficiency):

Space complexity:

Space complexity is a combination of auxiliary space and input space. Where

auxiliary space is the extra space or buffer space that will be used by an
algorithm during execution. Also, we know that space complexity is all about
memory. Below are a few points on how the memory is used during execution.

 Instruction space: the compiled version of instructions are stored in

memory. This memory is called instruction space.
Mr.Siddarama S-SBMJC-KGF Page 15
 V Sem BCA- ADA Unit-1

 Environmental stack: We use environmental stacks in cases where a

function is called inside another function.

For example, a function cat() is called inside the function animals().

Then the variables of function animals() will be stored temporarily in a system

stack when function cat() is being executed inside the function animals().

 Data Space: The variables and constants that we use in the algorithm will
also require some space which is referred to as data space.

In most of the cases, we neglect environmental stack and instruction space.

Whereas, data space is always considered.

How to calculate space complexity?

To calculate the space complexity we need to have an idea about the value of
the memory of each data type. This value will vary from one operating system
to another. However, the method used to calculate the space complexity remains
the same.

Let’s have a look into a few examples to understand how to calculate the space
complexity.

Example 1:

{
int a,b,c,sum;
sum = a + b + c;
return(sum);
}

Here, in this example, we have 4 variables which are a, b, c, and sum. All these
variables are of type int hence, each of them require 4 bytes of memory.

Total requirement of memory would be : ((4*4)+4) => 20 bytes.

Time complexity:

Time complexity is most commonly evaluated by considering the number of

elementary steps required to complete the execution of an algorithm. Also,

Mr.Siddarama S-SBMJC-KGF Page 16

 V Sem BCA- ADA Unit-1

many times we make use of the big O notation which is an asymptotic notation
while representing the time complexity of an algorithm.

Quadratic time complexity:

Example:

for(i=0; i < N; i++)

for(j=0; j < N;j++)

statement;

Here, in this example we have a loop inside another loop. In such a case, the
complexity would be proportional to n square. Hence, it would be of type
quadratic and therefore the complexity is quadratic complexity.

Mr.Siddarama S-SBMJC-KGF Page 17