# GREEDY METHOD General characteristics:1. Problems will have n inputs. 2.

From the input set, find a subset that satisfies some constraints. This subset is called feasible solution. 3. Feasible solution either maximizes or minimizes a give objective function. This feasible solution is called an optimal solution. 4. Greedy algorithm works in stages, by considering one input at a time. 5. Each stage, a decision is made regarding whether the input considered yields an optimal solution. If so, the input is considered otherwise, rejected. 6. Inputs are considered by a selection procedure. Selection procedure is based on some optimization measure. 7. If the inclusion of the next input into the partially constructed optimal solution results in an infeasible solution, then this input is not added to the partial solution, otherwise, it is added. This measure may be the objective function. Examples: 1. Knapsack problem 2. Tree vertex splitting 3. Job sequencing with dead lines 4. MST 5. Single source shortest path 6. Optimal storage on tapes 7. Optimal merge patterns Control Abstraction: Algorithm greedy (a, n) { Solution: = ø For i = 1 to n do { x = select (a); If feasible (solution, x) then solution = union (solution, x) } Return solution; } KNAPSACK PROBLEM Assume there are n objects and a knapsack or bag, objects I has a weight Wi. let m be the capacity of knapsack or bag. Let Pi be the profit knapsack or bag. Let Pi be the profit knapsack or bag. Let Pi be the profit associated with each object. Let Xi be the fraction of each object i. If a fraction Xi of object I is placed in a bag or knapsack, then the profit P iXi is earned. The objective is to fill the knapsack and maximize the profit. Maximize n∑i=1 PiXi ------------------- (1) Subject n∑i=1 Wi Xi <= m ------------ (2) 0<= Xi <=1 Where profits and weights are positive numbers A feasible solution is any set (X1, …….. Xn) satisfying the equations 2.

0. For i=1 to n do { If(w[i]>u) then break. Algorithm greedy knapsack (m. We begin with min weight edge (u. we choose a minimum weight edge (u. This process is repeated until a MST is formed. If weights are assigned. weighted graph.An optimal solution is a feasible solution for which equation 1 is maximized. U= U-W[i]. G/ € G . all cities connected with minimum cost. Electric network 2. Then vertex u is added to T. Prim’s algorithm has the property that the edges in the (T) always form a single tree. } m=capacity of the bag n=number of objects W [1…n] = weights of the objects P [1…n] = profit of the objects Selection procedure: Objects are ordered such that (P[i]/W[i]) >= (P[i+1]/W[i+1]) MINIMUM SPANNING TREE Given a connected. v) connecting a vertex v in the T. In each iteration only safe edges are added (i. } If(i<=n) then X[i] = U/W[i]. A graph can have many different spanning trees. U=m.0. undirected. MST is one with the lowest total cost. n) { For i=1 to n do X[i] =0. APPLICATIONS OF MST 1. Algorithm: Procedure Prim (G: weighted connected graph with n vertices) { T= a minimum weighted edge For i = 1 to n-2 Begin { . Prim’s algorithm finds a minimum spanning tree of G. v) in G = (V. E). PRIM’S ALGORITHM Consider a weighted connected graph G with n vertices. Then in each iteration. a spanning tree of that graph is a sub graph which is a tree and connects all the vertices together.e. Then vertex u outside of T. X[i] = 1. edges which will not form a cycle). V(G/) = V(G) Link/communication problems.

we choose a minimum weight edge (u. T = T with e added } End Return (T) } Explanation: Prim’s Algorithm has the property that the edges in the (T)( always form a single tree. Also. Without the for loop. w) does not create a cycle in T . This process is repeated until a minimum spanning tree is formed. then add (v. If priority queue is implemented as binary heap. Priority queue can be implemented as 1. w from E. Fibonacci heap O(V log V +E) Implementation of priority queue as Binary heap Minimum edge is selected by storing the edges in a priority queue.E = an edge of minimum weight incident to a vertex in T and not forming a cycle in T if added in T. w) from E of lowest cost. We begin with a minimum weight edge (u. Kruskal’s algorithm finds a minimum spanning tree of G. Then in each iteration. Algorithm: { T=ø While t has less than n-1 edges and ( E ≠ 0 ) do { Choose an edge (v. w).E). Cost adjacency matrix O(V2) 3. Then vertex U is added to T. the edge selection and updating MST is implemented as heap which takes O (log V) Therefore. total time for prim’s algorithm using binary heap in O (V log V + E log V). ANALYSIS Minimum edge is selected by storing the edges in a priority queue. Else Discard (v.e. O [E] times. Binary heap 2. w) to T. E=n-1 Therefore.v) in G (V. Total time O (V log V) For loop executes at the max n-2+1 i. } } Explanation: . If (v. Delete v. Construction of heap O (V) Extracting min/max O (log V) Therefore.v) connecting a vertex V in T to the vertex U outside of T. KRUSKAL’S ALOGORITHM Consider a weighted connected graph G with n vertices. in each iteration only safe edges are added that is edges which will not form a cycle.

Inside the while loop. O(E) times. Binary heap: Outside the while loop. the forest is connected and a single MST is formed. it will not form a cycle. In dynamic programming. DYNAMIC PROGRAMMING It is an algorithm approach that can be used when the solution to a problem can be viewed as the results of a sequence of decisions.e. heap would be constructed for n vertices. two computations are carried out : a. One decision sequence will be Many decision sequences will be generated generated. Only safe edges are added i. It is assumed that the weights are positive. Priority queue can be implemented as 1. v) and it is added to T.e. Definition: Principle of optimality: the principle of optimality states that an optimal sequence of decisions has the property that whatever the initial state and decisions are. compared to permutation problems Characteristics: - . 3. This process is repeated until a MST is formed.permutation problems. Algorithm: This algorithm is called as Dijkstra’s algorithm developed by using greedy method. ANALYISIS: Minimum weight edge is selected by storing the edges and vertices in a priority queue. the problem is to determine the shortest paths from V0 to all the remaining vertices of G. while loop performance is stated as O(E log V + E log V). total analysis is therefore O(V + E log V + E log V). Cost adjacency matrix Single source shortest paths: Definition: Given a directed weighed graph G=(V. an optimum sequence of decisions is obtained by using principle of optimality. 2. forest is formed. Retrieval from heap. Fibonacci heap: while loop executes (n-1) times i. Solve problems (selection problems) selection. Sometimes. 2. E). Used to solve selection problems. Update of MST.E) and the weighting function as cost and the source vertex as V0 . T (MST) is not a tree but a forest. Construction of heap would take O(n) time i.e. v) in G = (v. which is O (log n) i. GREEDY METHOD DYNAMIC PROGRAMMING 1. b. the remaining decisions must constitute an optimal decision sequence with regard to the state resulting from the first decision.TSP 3. We begin with minimum weight edge (u.OBST. Therefore. choose a minimum weight edge (u. At the end of the iteration. 3. Edges with minimum cost are considered 2. which is O (log n) i. Optimal solutions to the subproblems are Such problems will be solved easily retained so that recomputation is avoided. O(log V).e. O (log V).e. So. O(v) time. Ex: knapsack DP algorithms often have a polynomial complexity 4.Kruskal’s algorithm has the following properties at each iteration: 1. Then in each iteration.

Examples: 1. If it is not empty. 4. given that stage j+1 has already been solved. Single source shortest path 4. . The left & right sub trees are also BST. The keys in the right sub tree are larger than the key in the root. Multi stage graph 2. 3. Optimal solutions to the sub problems are retained so that recomputation is avoided. 2. TREE 1 takes four comparisons whereas TREE 2 takes 3 comparisons to find an identifier. The keys in the left sub tree are smaller than the key in the root. 5. Every element has the key and no two elements have the same key. Decision at one stage transforms one state into a state in the next stage. It may be empty. 1. All pairs shortest path 3. then it satisfies the following properties. Given the current state. Each stage has number of states associated with it. 2. OBST => (selection problem) 5. the optimal decision for each of the remaining states does not depend on the previous states or decision. TSP => (permutation problem) OBST: Binary search tree: BST is a binary tree. 4. 3. 6. There exists a recursive relationship that identifies the optimal decision for stage j. 1.The problems can be divided into stages with a decision required at each stage. In worst case.

1. j) such that j-i = 1. To compute one C(i. The remaining nodes will be in the right sub tree (r).j) = k ( the value of k which minimizes the equation 1) C(0. Cij > 0 for all i and j and Cij = ∞ if (i and j) ₡ E. So on till j-i = n.a2….j) = p(j) + q(j) +w(i. E0 contains all identifiers x such that x < a1 E1 contains all identifiers x such that x > an. ..j-1) C(i. terminates at an external node at level l. P(i) is the probability to search ai where 1 <= i <= n 3. E) be a directed graph with edge costs Cij . the following points are considered. Let the identifiers are denoted as {a1. 1. TRAVELLING SALESPERSON PROBLEM (TSP) TSP is a permutation problem.….In average case TREE 1 takes 1+2+3+4/5 = 12/5 comparisons whereas TREE 2 takes 1+2+2+3+3/5 = 11/5 comparisons. j-i = 1 is O(n).an 2. r(i. Ek-1 will be in the left sub tree (l).a2…. total complexity is O(n2 * n ) is O(n3). Successful search will terminate in internal nodes and unsuccessful search will terminate in external nodes. Each identifier is searched for equal probability. 5. BST contains n internal nodes and n+ 1 external node. the following assumptions are made. No unsuccessful search is made.E1.e.i) = 0 r(i. Analysis : 1. we can expect unsuccessful searches.j) is O(n). each identifier is searched with different probabilities. The identifiers not in BST can be partitioned into n+1 classes Ei 0 <= i <= n i. j) = mini<k<=j { C(i. C(i.ak-1 and external nodes E0. 6. There are n! permutations for n objects.j) ------. j-i = 2 ….an} such as that a1 < a2….j) & r(i. O(n*n) = O(n2) 2. x is not present in a i < x < ai+1 where 0<= i<= n [x can be less than a1 ] [therefore min value is 0] 4.n) can be solved by first computing all C(i.e. Expected cost of a BST is Min {∑1<=i<=n P (i) + level (ai) + ∑0<=i<=n q(i) + (level (Ei) – 1) } DYNAMIC PROGRAMMING APPROACH Dynamic programming approach to OBST problem is to make a decision as to which of the ai’s should be assigned as the root node of the tree.j) is also recorded w(i. But in general. selection problems have 2n possible selections. j) till j-i=n Therefore. then all internal nodes a1. (n! > 2n) Problem definition: Let G = (V.(1) w(i.j) is also computed.j) } + w(i. then cost is calculated as q(i) + (level (Ei) – 1). 2. Compared to selection problems. We have to compute C(i.k-1) + C(k.i) = q(i) r(i. x = ai ) 7. C(i.e. then cost is calculated as p(i) + level (ai ) (i. Therefore. Also.i) = 0 w(i. To compute w(i. Suppose if we select a k as the root node. Therefore. Permutation problems are harder to solve. j) = O(n) i. For unsuccessful search. 8. j) represents cost of OBST. In this case. If a successful search terminate at an internal node at level l. q(i) is the probability of an unsuccessful search.

the remaining (x i+1 …… X m) can be ignored.s)’s that have to be computed.|s|=2….x2. we have to find a tour with minimum cost.x2.s) be the length of a shortest path starting at vertex i. If the partial solution can no way lead to optimum solution. Problem description: Every tour starts and ends at vertex 1.|s|=n-1 S=V-{1} Analysis Let N be the number of G(i. .s-{j}) } Also. the solution should satisfy a bounding function or criterion function P(x1. Travelling salesperson is to find a tour of minimum cost. These constraints are classified into two categories. G(i.s) = minj € s { Cij + G(j. Problems that can be solved using backtracking approach should satisfy some constraints.x2….ø) = Ci1 The equation is solved by solving |s|=1. 6. 5. 3. The cost of a tour is the sum of the cost of the edges on the tour. all xi’s are calculated and checked whether the solution is optimum or not.Let |V| = n. Let mi be the size of set Si. Implicit constraints are rules that determine which of the tuples in the solution space satisfy the criterion function. Using dynamic programming approach. it is considered else it is rejected. the desired solution is expressed as (x1. In backtracking method.xn).xn) where xi are chosen from Si (solution set). A tour of G is a directed simple cycle that includes every vertex in V. Therefore in Brute Force approach. 2. N= (n-1) 2n-2 Any algorithm can be developed with complexity O(n2 * 2n) Backtracking (General Method) Characteristics 1.then. going through all vertices in s and terminating at vertex 1. we need only less than m trails. Problem which deals with searching for a set of solutions satisfying some constraints can be solved using backtracking. Let G(i. In backtracking method. to check whether solution is an optimum solution or not.x3….x3…. àImplicit àExplicit Explicit constraints are rules that restrict the value of each Xi. InBacktracking method. Each vertex is visited exactly once. say (x1. If optimum. But in Backtracking approach.. 4.. G(i. There are two methods to find the solution àBrute Force Approach àBacktracking Method In Brute Force approach.xi) the partial solution is calculated. we need m trails.

e. àDynamic Trees: trees are dependent of the problem instance. All paths from the root to other nodes define the state space of the problem. 4. A node which has been generated and all of whose children have not yet been generated is called a live node. Let B be the bounding function. 7.X2…. 7.Xi) be a path from the root to a node in a state space tree. Terminologies regarding tree organization of a solution space 1.X[2]…. Some of the problems that can be solved by backtracking approach are à8 Queens àSum of Subsets àHamilton Cycles àGraph Coloring 8. all queens must be on different rows. Algorithm Backtracking(k) foreach X[k] E T (X[1].. Tree organization for a solution space is called as state space tree..X2….X[2]…. Backtracking algorithms determine solutions by searching the solution space. The search is done using a tree organization for the solution space.Ex: For 8 Queens Problem Explicit constraints: Value of Xi will vary from 1 to 8. àStatic Trees: trees are independent of problem instance. There are two search methods used in state space tree: àDepth first àBreadth first Depth Firstnode generation with bounding function is called backtracking. Breadth First node generation is called Branch and Bound technique. A live whose children are currently generated is called E-node. columns.X[k-1]) do { if(Bk(X[1].X[k]!=0) then { . 2. Solution state is a path from root to the node that defines a solution.Xi+1is also a path to a problem state. 6. Implicit constraints: No two Xi’s can be same i. General Method Let (X1.X2…. 3. 5. diagonals. There are two types of state space trees. Answer state is a solution state that satisfies some implicit constraints. A dead nodeis one which is not to be generated further.Xi) be a set of all possible values for Xi+1 such that X1. Each node in the tree defines a problem state. Let T(X1. 8.

àBounding functions are regarded as good if they reduce the number of nodes in the state space tree. a check is made to determine whether a solution has been found then.X3….X[k]) is a path to an answer node) then { Write(X[i:k])..X2. Tree 1 . } if(k < n) then { Backtracking(k+1). Efficiency of backtracking algorithm depends on four factors 1.if(X[1].Xn).X[2]. The number of Xk satisfying Bk. 3. Time for the bounding function Bk.…. Algorithm is recursively invoked.X2. There is a tradeoff that good bounding functions take more time to evaluate.Xn) is treated as a global array. All possible elements for the Kth position of the tuple that satisfy Bk are generated. àThe number of nodes generated can be estimated by an Estimator (algorithm proposed by Monte Carlo). àSearching in a state space tree can be improved by the principle of rearrangement. one by one and adjoined to the current vector (X1.X3…. The time to generate the next Xk. Number of Xk satisfying explicit constraints. 2. Each time Xkis attached. } } } The solution vector(X1. 4.

X2…. These constraints reduce the solution space from 88 to 8! 2 3 4 5 6 7 8 . Implicit Constraint: No 2 Xi’s can be same ie. In tree 2. Eight queen’s problem It is a combinational problem to place eight queens on an 8 x 8 chess board so that no two “attack” i. if a node in level2 is removed. Let us number the rows and columns of the chessboard as 1 through 1 1 2 3 4 5 6 7 8 The queens are numbered 1 to 8.X8) where Xis the column on which queen i is placed.e no two queens are on the same row. No two queens can be on the same rows or columns or diagonal.Tree 2 In tree 1. Explicit Constraint: Value of Xi will vary from 1 to 8. if a node in level2 is removed. then 8 possible solutions are discarded. column. Solution to this problem is represented as (X1. and diagonal. then 12 possible solutions are discarded.

X8) where Xi is column on which queen I placed. But to check for the diagonal position.n).i) returns a Boolean value that is true if the Kthqueen can be placed in column i.i) for j=1 to k-1 do if((x[j]=i) or (abs(x[j]-i)=abs(j-k)) then return false.l).  Diagonal that runs from upper left to lower right has same row-column value.i) then { X[k]=i. if ( k=n ) then Write(x[1:n]) else NQueen(k+1.  Diagonal that runs from upper right to lower left has same row+columnvalue. Then they are on 2 same diagonal the i i only if i-j=k-l ori+j=k+l i-k=j-l k = j l j-l=k-i k = j l Therefore two queens lie on the same diagonal iff |j-l|=|i-k| Algorithm place(k. imagine the chessboard as 2D array.By representing the solution like (X1. return true.X2….j) and (k. it is easy to check that 2 queens are not placed in same row or column. Algorithm NQueen(k.n) for q=1 to n do { if place(k. The total nodes generated in a state space tree for N Queens problem n-2 j 1 + S ( T1(n-i)) j=0 i=0 . } } Algorithm place(k.n) gives a complete solution Algorithm NQueen(k. Suppose two queens are 1 placed at position (I.

The total nodes generated in a 8 queen’s problem state space tree is n-2 j 1 + S ( T1(8-i)) j=0 i=0 1 + 8 + 8*7 + 8*7*6 + 8*7*6*5 +8*7*6*5*4 + 8*7*6*5*4*3 + 8*7*6*5*4*3*2 + 8*7*6*5*4*3*2*1 =1+8+56+336+1680+6720+20160+40320+40320 =109601 nodes .

.