You are on page 1of 22


Introduction to Algorithms

1.1 Notion of Algorithm
Program: Algorithms + Data Structures
Program Solving the problem

Data Structure Problem‟s instance

Algorithm Problem‟s solution

Can be represented in different ways.What is an Algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem Non-ambigous requirement cannot be relaxed. Range of inputs must be specified. Algorithms for same problem can have dramatically different speeds. Several algorithms for a problem may exist. .

. Practical importance A practitioner‟s toolkit of known algorithms Framework for designing and analyzing algorithms for new problems.Motivation for Algorithm Theoretical importance The study of algorithms represents the core of computer science.

Middle-school algorithm: Find the prime factors of both m and n. Euclid’s algorithm: Apply repeatedly the equality gcd(m.n). identify all the common factors whose product is the gcd.n} divides both m and n: If it does. The last value of t which divides both integers is the gcd. if not.GCD Algorithms Problem: Finding the greatest common divisor of two nonnegative. m and n.n) = gcd(n. t is the answer. not-both-zero integers. denoted gcd(m. Definition-based algorithm: Start by checking whether t = min{m. decrease t by 1 and try again. The last value of m is the gcd.m mod n) until m mod n is 0. .

a mod b) . b) if b = 0 return a else return gcd(b.GCD Algorithms : Euclid’s algorithm function gcd(a. b) while b ≠ 0 t := b b := a mod b a := t return a function gcd(a.

Sieve of Eratosthenes to find prime numbers .

Set k=m and repeat. Mark the numbers 2m.. Find the first number in the list greater than k that has not been identified as composite. . Initially all numbers are unmarked. 3b.. n. We will eliminate composites by marking them. Until k exceeds or equals the square root of n do this: 3a. Call it m. .. as composite. 3.. 2. 2. m is a prime number. . 3c. Set k=1. Write down the numbers 1.Sieve of Eratosthenes to find prime numbers 1. Put it on your list. 3m. Mark the number 1 as special (it is neither prime nor composite). 3.. 4m.

1.2 Fundamentals of Algorithmic Problem Solving Understand the problem Decide on computational means Design an algorithm Prove correctness Analyze the algorithm Code the algorithm .

Analyze the running time of proposed algorithm f. decide between sequential and parallel algorithms 2. decide upon algorithm design technique (to be studied in detail later) c. Understand the problem – input is an instance. decide upon appropriate data structures 4. Code proposed algorithm . decide between exact and approximate algorithms 3. b. Decide on computational means – 1.Fundamentals of Algorithmic Problem Solving a. Prove (mathematically) that the algorithm does what it is supposed to do e. Design the algorithm and decide what way will it be written – flowchart/pseudocode d.

) solves system of equations. Searching – search for a key object in a given set on n objects (may/maynot be sorted).e. search for (all) P in T. number of intersections of lines. solve traveling salesman problem. i. String Processing – given a text (T – very large) and pattern (P – much smaller than P). Combinatorial problems – always counts number of things.3 Important problem types Sorting – given some objects (typically integers). etc. Graph problems – Given a graph. . etc.1. evaluates functions. solve graph coloring problem Numerical problems – (approx. sort them in ascending order. find shortest path.. Geometric problems – computes things that is counted by combinatorial algorithms.

Running time is c(n) × cop Take ratio of this running time to compare speeds of algorithms . i.4 Analysis Framework Input size (written as n or a function of n. i. c(n))– 1.1.e. n is input size where n is number of co-efficients and highest power of x After deciding what n is.. can be number of elements given 2.e.. cop . find out the time taken for performing the basic operation. can be number of bits required to represent given numbers n 0 Eg – In evaluating polynomial an x   a0 x .

Repeat this loop: Return i. then exit the loop. Set i to 1. then exit the loop. Return i. If A[i] = x. then exit the loop. Reverse Linear search . Forward Linear search Set i to i − 1. Set i to i + 1.Set i to n. If i ≤ 0. Repeat this loop: If i > n. then exit the loop.Algorithm for Linear search Have to search for a given element (called „key‟) among n elements in forward or reverse order. If A[i] = x.

Reverse linear search – Best case – Element found at end. Worst and Average case analysis of Linear search Forward linear search – Best case – Element found at beginning. Average case – Element found some where between.Best. Average case – Element found some where between. . Worst case – Element found at end. Worst case – Element found at beginning.

 If we want to say both of the above at the same time – use Theta notation .5 Asymptotic Notations and Basic Efficiency Classes Say we have an algorithm A that runs in time proportional to some complicated function of n If we want to say A takes at most time proportional to another easy function of n – use Big Oh notation . .1.O If we want to say A takes at least time proportional to another easy function of n – use Omega notation .

such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n. 2 Example: 100n+5 = O(n ) g(n) is a upper bound for f(n) .Asymptotic Notations Formal Definition: f(n) = O(g(n)) means there are positive constants c and k.

such that 0 ≤ cg(n) ≤ f(n) for all n ≥ k. 2 Example: n 3 = (n ) g(n) is a lower bound for f(n) .Asymptotic Notations Formal Definition : f(n) = Ω (g(n)) means there are positive constants c and k. The values of c and k must be fixed for the function f and must not depend on n.

The values of c1. such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ k. and k. c2. c2. g(n) is both a upper 1 2 Example: n(n  1) = (n ) and lower bound for 2 f(n) . and k must be fixed for the function f and must not depend on n.Asymptotic Notations Formal Definition : f(n) = Θ (g(n)) means there are positive constants c1.

Use – Say there is an algorithm A with two parts in it.Useful property of Asymptotic Notations Statement – If there are 2 algorithms with running times O(g(n)) and O(h(n)). the running times of both the algorithms combined is the greater of O(g(n)) and O(h(n)). Say B runs in O(g(n)) time and C runs in O(h(n)) time.O(h(n))). The running time of the algorithm A is max(O(g(n)). called B and C. . Take the part of the algorithm that takes maximum time and pass it off as the running time of the algorithm.

n2 + n log(n) = O(n2) Rule 2: If T1(N) = O(f(N)) and T2(N) = O(g(N)) then T1(N) * T2(N) = O(f(N)* g(N)) .Rules to manipulate Big-Oh expressions Rule 1: If T1(N) = O(f(N)) and T2(N) = O(g(N)) then T1(N) + T2(N) = max(O(f (N)). O(g(N))) Examples: 1. n2 + n = O(n2) we disregard any lower-order term 2. nlog(n) = O(n log(n)) 3.

n → ∞ Notation: f(n) = Ω(g(n)) . n → ∞ Notation: f(n) = O(g(n)) f(n) grows faster than g(n) (or g(n) grows slower than f(n)) if lim( f(n) / g(n) ) = ∞. n → ∞ Notation: f(n) = Θ(g(n)) f(n) grows slower than g(n) (or g(n) grows faster than f(n)) if lim( f(n) / g(n) ) = 0. if lim( f(n) / g(n) ) = c. 0 < c < ∞.Using limits to compare growths instead of asymptotic notations f(n) and g(n) have same rate of growth.

Basic efficiency classes Class O(1) log N N N log N N2 Name constant logarithmic linear “n log n” quadratic Example basic operation binary search linear search divide and conquer graph algorithms with adjacency matrix as input 3 embedded loops subsets generation permutations N3 2N N! cubic exponential factorial .