1.

Introduction to Algorithms

1.1 Notion of Algorithm
Program: Algorithms + Data Structures
Program Solving the problem

Data Structure Problem‟s instance

Algorithm Problem‟s solution

Algorithms for same problem can have dramatically different speeds. Range of inputs must be specified. Several algorithms for a problem may exist. Can be represented in different ways. .What is an Algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem Non-ambigous requirement cannot be relaxed.

Practical importance A practitioner‟s toolkit of known algorithms Framework for designing and analyzing algorithms for new problems. .Motivation for Algorithm Theoretical importance The study of algorithms represents the core of computer science.

Middle-school algorithm: Find the prime factors of both m and n.m mod n) until m mod n is 0.n} divides both m and n: If it does. decrease t by 1 and try again. m and n.n). The last value of t which divides both integers is the gcd.n) = gcd(n. denoted gcd(m. .GCD Algorithms Problem: Finding the greatest common divisor of two nonnegative. Definition-based algorithm: Start by checking whether t = min{m. The last value of m is the gcd. not-both-zero integers. Euclid’s algorithm: Apply repeatedly the equality gcd(m. if not. t is the answer. identify all the common factors whose product is the gcd.

GCD Algorithms : Euclid’s algorithm function gcd(a. b) while b ≠ 0 t := b b := a mod b a := t return a function gcd(a. a mod b) . b) if b = 0 return a else return gcd(b.

Sieve of Eratosthenes to find prime numbers .

Call it m. . Put it on your list. Initially all numbers are unmarked.. n. Write down the numbers 1... Set k=m and repeat. Mark the number 1 as special (it is neither prime nor composite). m is a prime number. 3. Set k=1. 3b. Find the first number in the list greater than k that has not been identified as composite. 2. Until k exceeds or equals the square root of n do this: 3a. as composite..Sieve of Eratosthenes to find prime numbers 1. . Mark the numbers 2m. 3m. We will eliminate composites by marking them. . 3. 4m. 3c.. 2.

2 Fundamentals of Algorithmic Problem Solving Understand the problem Decide on computational means Design an algorithm Prove correctness Analyze the algorithm Code the algorithm .1.

b. Design the algorithm and decide what way will it be written – flowchart/pseudocode d. decide between sequential and parallel algorithms 2. Decide on computational means – 1. decide upon appropriate data structures 4. Code proposed algorithm . decide between exact and approximate algorithms 3. Understand the problem – input is an instance. decide upon algorithm design technique (to be studied in detail later) c. Analyze the running time of proposed algorithm f.Fundamentals of Algorithmic Problem Solving a. Prove (mathematically) that the algorithm does what it is supposed to do e.

3 Important problem types Sorting – given some objects (typically integers). Geometric problems – computes things that is counted by combinatorial algorithms. number of intersections of lines. . solve graph coloring problem Numerical problems – (approx. String Processing – given a text (T – very large) and pattern (P – much smaller than P). solve traveling salesman problem. sort them in ascending order.1. Combinatorial problems – always counts number of things. etc. evaluates functions.. i.) solves system of equations. Graph problems – Given a graph. Searching – search for a key object in a given set on n objects (may/maynot be sorted). search for (all) P in T.e. find shortest path. etc.

Running time is c(n) × cop Take ratio of this running time to compare speeds of algorithms . i. can be number of elements given 2.1. cop .. c(n))– 1. i. n is input size where n is number of co-efficients and highest power of x After deciding what n is. can be number of bits required to represent given numbers n 0 Eg – In evaluating polynomial an x   a0 x .e.e.. find out the time taken for performing the basic operation.4 Analysis Framework Input size (written as n or a function of n.

If A[i] = x. Forward Linear search Set i to i − 1. Repeat this loop: Return i. Return i. Set i to i + 1. If i ≤ 0. Repeat this loop: If i > n. then exit the loop. If A[i] = x.Set i to n. then exit the loop.Algorithm for Linear search Have to search for a given element (called „key‟) among n elements in forward or reverse order. then exit the loop. then exit the loop. Set i to 1. Reverse Linear search .

Reverse linear search – Best case – Element found at end. Worst and Average case analysis of Linear search Forward linear search – Best case – Element found at beginning. Average case – Element found some where between. .Best. Worst case – Element found at end. Worst case – Element found at beginning. Average case – Element found some where between.

5 Asymptotic Notations and Basic Efficiency Classes Say we have an algorithm A that runs in time proportional to some complicated function of n If we want to say A takes at most time proportional to another easy function of n – use Big Oh notation .O If we want to say A takes at least time proportional to another easy function of n – use Omega notation .1. . If we want to say both of the above at the same time – use Theta notation .

such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. 2 Example: 100n+5 = O(n ) g(n) is a upper bound for f(n) . The values of c and k must be fixed for the function f and must not depend on n.Asymptotic Notations Formal Definition: f(n) = O(g(n)) means there are positive constants c and k.

such that 0 ≤ cg(n) ≤ f(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.Asymptotic Notations Formal Definition : f(n) = Ω (g(n)) means there are positive constants c and k. 2 Example: n 3 = (n ) g(n) is a lower bound for f(n) .

c2. c2. The values of c1. and k must be fixed for the function f and must not depend on n. such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ k. g(n) is both a upper 1 2 Example: n(n  1) = (n ) and lower bound for 2 f(n) . and k.Asymptotic Notations Formal Definition : f(n) = Θ (g(n)) means there are positive constants c1.

Say B runs in O(g(n)) time and C runs in O(h(n)) time. called B and C. Take the part of the algorithm that takes maximum time and pass it off as the running time of the algorithm.O(h(n))). Use – Say there is an algorithm A with two parts in it. The running time of the algorithm A is max(O(g(n)). the running times of both the algorithms combined is the greater of O(g(n)) and O(h(n)). .Useful property of Asymptotic Notations Statement – If there are 2 algorithms with running times O(g(n)) and O(h(n)).

O(g(N))) Examples: 1. nlog(n) = O(n log(n)) 3. n2 + n log(n) = O(n2) Rule 2: If T1(N) = O(f(N)) and T2(N) = O(g(N)) then T1(N) * T2(N) = O(f(N)* g(N)) .Rules to manipulate Big-Oh expressions Rule 1: If T1(N) = O(f(N)) and T2(N) = O(g(N)) then T1(N) + T2(N) = max(O(f (N)). n2 + n = O(n2) we disregard any lower-order term 2.

n → ∞ Notation: f(n) = O(g(n)) f(n) grows faster than g(n) (or g(n) grows slower than f(n)) if lim( f(n) / g(n) ) = ∞. n → ∞ Notation: f(n) = Θ(g(n)) f(n) grows slower than g(n) (or g(n) grows faster than f(n)) if lim( f(n) / g(n) ) = 0. if lim( f(n) / g(n) ) = c. n → ∞ Notation: f(n) = Ω(g(n)) . 0 < c < ∞.Using limits to compare growths instead of asymptotic notations f(n) and g(n) have same rate of growth.

Basic efficiency classes Class O(1) log N N N log N N2 Name constant logarithmic linear “n log n” quadratic Example basic operation binary search linear search divide and conquer graph algorithms with adjacency matrix as input 3 embedded loops subsets generation permutations N3 2N N! cubic exponential factorial .

Sign up to vote on this title
UsefulNot useful