You are on page 1of 16

Amit Shekhar ID: 108266469 NetID: ashekhar CSE 548 – Analysis of Algorithms Solution to Assignment #2 Due Thursday, October 27, 2011

Discussed at Office hours and with Ayon Chakraborty and Amit Arya

we seek multiplication of two n-bit numbers. The recurrence relations for n-bit addition: =⇒ O(log n) Depth: T (n) = T ( n ) + 1 2 n Size: S(n) = 3S( 2 ) + n =⇒ ( 3 )log n ∗ n 2 =⇒ O(nlog 3 ) where n is the task done at Multiplexer and 3 came from the decision to compute more significant half twice.Problem 1 In class we saw how to construct circuits for addition of two n-bit numbers that have depth of O(log n) and O(1). Here is how it is so. Note: You may assume that the size of an addition circuit (both for standard and nonunique representation) is Θ(n). Thereby. Solution: From the class. Hence. This gives the depth of circuit is log n for this part. repeat this process till we get the final value. Once when C n = 1 and another time when C n = 0 2 2 Here. which is definitely an addition of n numbers of 2n bits each. Since we are multiplying n bits. making the length of each number to be added to 2n. Note that we have already established that circuit for Addition of two n-bit numbers has depth of (log n) Method: Now we take each intermediate result in pair and add them. The length of each number becomes 2n because after each bit is multiplied with the multiplicand. the depth of entire circuit becomes O(log2 n) Size Computation: . This is an adaptation of the simple mathematical multiplication. What is the size of this circuit? Explain ıve how to compute the size. This is analogous to MERGE part of a divide and conquer algorithm. we must have shifted one of the intermediate result to n bits to the left. In this problem we look at how to design circuits for multiplying. (a) Design a na¨ circuit that has depth O(log2 n). the resultant (intermediate) number is shifted towards left by one bit. Please note the extra bits can be padded. we establish that circuit for Addition of two n-bit numbers has depth of log n using standard representation.

we seek multiplication of two n-bit numbers. What is the size of this circuit? Explain how to compute the size. Here is how it is so. making the length of each number to be added to 2n. Please note the extra bits can be padded. Solution: From the class. The recurrence relations for n-bit addition: Depth: T (n) = O(1) Size: T (n) = O(n) Here. (b) Design a better circuit that has depth O(log n). Hence. This is an adaptation of the simple mathematical multiplication. Thereby. This is analogous to MERGE part of a divide and conquer algorithm. Each such node requires Θ(n) of circuit size. and repeat this process till we get the final value. The length of each number becomes 2n because after each bit is multiplied with the multiplicand. the resultant (intermediate) number is shifted towards left by one bit. which is definitely an addition of n numbers of 2n bits each. we establish that circuit for Addition of two n-bit numbers has constant depth using Non Unique representation. the depth of entire circuit becomes O(log n) Size Computation: for n nodes once again we get the circuit size as O(n2 ) . Since we are multiplying n bits. we must have shifted one of the intermediate result to n bits to the left. Method: Now we take each intermediate result in pair and add them. we have already established that circuit for Addition of two n-bit numbers has constant depth.If we assume O(nlog 3 ) to be equal to Θ(n) then Size would be O(n2 ) as there can be n nodes for the tree shown. This gives the depth of circuit is log n for this part.

Solution: From Recursion Tree: Work at level 1 ⇒ n n Work at level 2 ⇒ 31 × 21 . .Problem 2 In this problem. . (a) Please describe your construction algorithm and describe the depth of the circuit. . Work at level n ⇒ 3n × 2n n . bd only. Solution: The number can be written as: X = 2m a + b and Y = 2m c + d Multiplication requires: Z = 22m ac + 2m (bc + ad) + bd We see that (bc + ad) can be represented by (a + b)(c + d) − (ac + bd). This reduces the number of multiplication circuit by 1. we continue working on multiplication circuits. As multiplication can be performed by using circuits that can perform multiplication of (a + b)(c + d). . . ac. The objective is to reduce the overall size of the multiplication circuits to something that is o(n2 ). Hint: please use Problem 5 of Problem Set 1 as a guide. The image below shows the abstracted circuit of multiplication in this way: (b) What is the recurrence relation for the size of your circuit? Solution: The recurrence relation: T (n) = 3T ( n ) + O(n) 2 (c) Please solve this recurrence relation.

This also means contribution from leaves will dominate. total work = Σn × ( 3 )i ∀ i= 0 . At the leaf level contribution is T (1) = Θ(1) (by each leaf). . the last term would dominate. log2 n 2 The terms are in Increasing geometric series hence. The number of leaves is 3log2 n ⇒ nlog2 3 Therefore. and f (n) = n logb a ) The solution is: T (n) = O(n = O(nlog2 3 ) . T (n) = Θ(nlog2 3 ) Alternatively. = 1. b = 2. This recurrence relation falls under case 1 of Master Theorem where T (n) = aT ( n ) + f (n) b with a = 3.Depth of the tree = log2 n Therefore. .

. = A[n − 1] = 0. = 1. This algorithm should make at most O(log n) queries. . (2) Describe an algorithm for finding any index m. Here. the index of the first nonzero element. log log n Proof for the number of queries required: i Here. . it’s possible that we may bump into the part where there are only 1s. the first n elements of the array are 0 and all subsequent elements are 1. Show that this algorithm requires O(log log n) array queries.. . In that case. If at all. where A[m] = 1 and m < nc for some constant c > 1. That is. Please note that first occurrence of 1 is at 2( i − 1) < n < 2i and i = log n At max O(log n) queries. Hence. In constant time we can query the array to find the value of an array element A[i].. (3) Describe an algorithm for finding any m. The step can be any suitable 22 based upon infiniteness of the size of array. . . to reach n we step log log n and query at each step whether we have reached at the required place. we step by 22 to reach to the part where there are all 1s. We do not know the value of n.Problem 3 Imagine an infinitely large array A[0 . A[0] = A[1] = . . we should return back to immediately previous point [the index at 2( logn − 1) in the array] and then we should start repeating above thing from that index till we reach the first 1. it approaches to infinity then we can increase the step value to a big number. (1) Describe an algorithm to find the value of n. log n. ∞]. Solution: we solve this problem in the same way as above except that we move forward in the array at a step of 2( 2i ) where ranges from 0 . where A[m] = 1. We can do this in O(log ∗ n) and some constant time where we return back and check for the first 1. . A = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1. we can go forward to raise the step. . How fast can you make this algorithm? Solution: i 22 Once again. Solution: We query elements whose index is given by 2i where i ranges from 0 . . and A[n] = A[n + 1] = A[n + 2] = . In this array. Note that the level of step can be chosen based upon the value of n. .

Solution: Sub problem: For every possible value of target integer K. 0] = 0 Loop Assignment: for all i = 1 . sn } of integers and target integer K. DP technique helps us to compute most of the bigger values using solution to smaller subproblem.Problem 4 Consider the knapsack problem discussed in class: For set S = {s1 . . What part of the algorithm would we modify so that we can compute the set T? What are the modifications? Solution: Pseudocode: Initialization: for all k = 0 . k] B[i − 1. K for all i = 1 . K B[0. N for all k = 0 . k − si ] Goal Cell: Return B[n. . . k] = 0 B[0. we are going to check what subset of S can fit in properly. · · · . 0] = 0 . . k] = B[i − 1. . s2 . . such that ti = K? ti ∈T (1) Recall the algorithm from class. N Knapsack problem for set {s1 . k] (2) The algorithm from (1) only answers the question with a yes or no. . si } Pseudocode: Initialization: for all k = 0 . k] = 0 ¡— table for encoding regions for all i = 0 . . s2 . . then present pseudocode from class. is there a subset T ⊆ S. · · · . . Provide the subproblems. n T AB[i. for all k = 0 . . Though. . . K B[0. In order to check for any intermediate K from 0 . . k] = 0 T AB[0. K B[i. K we try the entire set S. . .

B[0. K = K − si Return B[n. k] T AB[i. . 0] = 0 for all i = 1 . . k] = B[i − 1. for all i = n . k] = 1 else B[i. k] = B[i − 1. . K if(B[i − 1. . k − si ] T AB[i. 1 if(T AB[i. k] = 0 The construction of subset from the Tab Matrix. k]) B[i. . K] == 1) Print i th element from set. k] . . N for all k = 0 . k − si ] > B[i − 1.

. Show that the greedy algorithm always finds the optimal shelf placement. The Recurrence relations is as follows : T(n) = = T (n − 1) + c = (T (n − 2) + c) + c = ((T (n − 3) + c) + c) + c . All the books have the same height h (i. where 1 ≤ i ≤ n. So.Problem 5 Problem from Steve Skiena Consider the problem of storing n books on shelves in a library. The length of each bookshelf at this library is L. Therefore. where 1 ≤ i ≤ n. . The greedy algorithm would fill the first shelf with as many books as we can until we get the smallest i such that bi does not fit. and we can arrange the books such that more number of books fit in a shelf.e. Again because of the condition that height is constant and order of books cannot be changed. we can speak of a book bi . there exists an optimal way other than greedy. so any book fits on any shelf. h = hi = hj for all i. Suppose all the books have the same height h (i. Hence. There are n books to be stored. and then repeat with subsequent shelves. Hence we conclude that greedy method is the same as the alternate optimal one. . the running time = O(n) . The length of each book shelf at the library is L and the shelves are all separated by a distance of greater than h. The order of the books is fixed by the cataloging system and so cannot be rearraged. and then repeat with subsequent shelves. even the alternate solution ends up filling the first shelf with as many books as it can until it get the smallest i such that bi does not fit. j) and the shelves are all separated by a distance of greater than h. = T (n − k) + k ∗ c T (n) is constant for n = 1 after a depth of n − 1. h = hi = hj for all i. that has a thickness ti and height hi . Assume.e. so any book fits on any shelf. and analyze the time complexity. This infers. there should be some space left by the greedy method because height is constant and order of books cannot be changed. j). Book bi has a thickness ti . Solution: Problem : Show that the greedy algorithm always finds the optimal shelf placement Proof by contradiction.

Thus the cost of a particular layout is the sum of the heights of the largest book on each shelf. bi + 1 . . bj ∀i < jandi.5. Solution: Consider books a.6.b. a possible set can start anywhere in the list of books.c. but we have the freedom to adjust the height of each shelf to that of the tallest book on the shelf. (1) Give an example to show that the greedy algorithm of stuffing each shelf as full as possible does not always give the minimum overall height. all the books that can fit in one self. and analyze its time complexity. For every Shelf. which is better than the answer given by Greedy Algorithm. if we order the books as: Shelf 1: a max height = 11 Shelf 2: b. (2) What technique should we use to solve this problem? Solution: Using Technique of dynamic programming. b → max height = 14 Shelf 2: c. Now consider the case where the height of the books is not constant. d → max height = 17 Shelf 3: e → max height = 6 Shelf height will be = 37.3. total height of all shelves = 34.3 and heights 11. However.Problem 6 Problem from Steve Skiena This is a generalization of the previous problem.6. Solution: Pseudo code: Initialize: OPT[][] be the DP memoization matrix initialized with a large value.17. Let the width of the shelf be 13. j ∈ n (4) How many subproblems are there? Solution: Upper limit for the number of subproblems can be nL.6 respectively. (5) Give an algorithm for this problem. we can solve this problem. In such a case. (3) What are the subproblems? Solution: For a height. Thus. c max height = 17 Shelf 2: d. e max height = 6 Hence. the Greedy Algorithm will order the books as: Shelf 1: a.14. Here. Greedy Algorithm does not always give optimal solution for minimizing the overall height of the shelves. . So. a subproblem may contain a list of books bi . Loop and sub problem solving using the previously solved smaller problem . we may have n sub problems.d having widths 3.

. L for all k = 1 . Goal Cell: OP T [1. k + l − 1). k + l] = very large value. OP T [k.for all l = 0 . hk+l } else Current subproblem doesn’t fit in. . n] Time complexity O(nL) . k + l] = max{OP T (k. . . . n − 1 if ((summation of thickness of all books from k . k + l) ≤ Book Shelf length) OP T [k. .

. then we’re done and the 3rd string is a shuffle. j − 1)th node or (i − 1. If we start traversing from (0. but chocochilatspe is not. j)th element it is reachable either from (i. Z is said to be a shuffle of X and Y iff Z can be formed by interleaving the characters from X and Y in a way that maintains the left-to-right ordering of the characters from each string.Problem 7 Suppose you are given three strings of characters: X. Solution: (b) Give an efficient dynamic-programming algorithm that determines whether Z is a shuffle of X and Y. N if (X[i] == Z[i + j]) A[i][j] = A[i][j − 1] i←1+1 else if (Y [j] == Z[i + j]) A[i][j] = A[i − 1][j] j ←j+1 else Return NOT A SHUFFLE! Return A SHUFFLE! Hint: The values the dynamic programming matrix you construct should be Boolean. For any (i. Pseudo code: A is the matrix that stores solution to previously computed smaller subproblems for all i = 0 . n) by above mentioned ways and we actually reach to the end without violating any rule. . j)th node. M and j = 0 . (a) Show that cchocohilaptes is a shuffle of chocolate and chips. . . and |Z| = n+m. . Y. where |X| = n. and Z. we see that each node is being approached from either of the two ways. the 3rd string is not a Shuffle. not numeric. Otherwise. 0) to the end (m. Solution: When we try to make the underlying DAG (Directed acyclic Graph) here. |Y| = m.

this gives us the mid point for any row of any length. This fact gives us a way to solve. Every time. devise a way how a soldier can talk to another soldier that is in the middle of the row i. When we do it recursively to the end when each soldier talks to only its neighbor. challenging problem There are N soldiers lined up. this requires log(n) bits.Problem 8 Fun. independent of N . how to identify mid of the row. In order to enable everyone to signal we must start with the communication with first element on the left and the mid of the row. It is not enough even to count up to N . Unfortunately. The soldiers all want to fire simultaneously. we are in a position to make each soldier talk in Unison. . And we must devise a way such that everyone gets to hear the signal before shooting. by treating the new sub-arrays the same way as we did to the original row we can make communication happen between 1st soldier on the left and the soldier standing in the mid of 1st sub-array. who begins the firing process. faster signal must also be reaching the mid point after traveling to the end. we have two sub-list of equal size – Divided by the mid that we found just now. Now. Firstly. The subsequent level communication requires 2n time 2l Hence the total work required still remains linear i. Now. the soldiers can only talk to their immediate neighbors on the left and right. soldier on the left sends two signals towards right.e. Say. these soldiers are not very intelligent and only have constant memory. O(n) . The algorithm should have running time O(n). about to execute a hapless prisoner. In addition. Solution: Here. is located at one end of the line. Next. Notice that constant memory is very little. The above mechanism can be employed to do this. because unique names also require log(n) bits. So. Devise an algorithm that allows these soldiers to fire in unison. The lieutenant. So. the soldiers can only talk to their immediate neighbors on the left and right. One of the signal travels 3x faster than another.e. Here we can use basic reflection property. . a soldier at the left most position can definitely talk with someone in the mid of the row. when the slower signal reaches the mid. It is not enough to have a name. Now the running cost analysis: The first level communication requires 2n time The next level communication requires 2n time 2 .

You should assume soldiers all operate at the same speed.. i. Hint: Use divide and conquer. they are synchronized.e.Note that they move and interact at exactly the same speed. .

Prove that we √ can achieve O(N 3 /B M ) memory transfers for a multiplication. as per our solution to the recurrence relation. a Block of size B can hold ( B ∗ B) elements. Therefore. B2 √ √ (b) Suppose that we divide the matrix into blocks of size M (i. We’ll use the same DAM model that we described in class and in the last problem set. B by B). the required number of N multiplication will be: ( √B )3 . we get T(n) = O(n3 ) √ √ Now. And the total number of multiplication N will be ( √M )3 √ √ For each ( M ∗ M ) elements there will be M blocks. √ √ (a) Suppose that we divide the matrix into blocks of size B (i. 2 For a matrix of size N ∗ N . a Block of size M can hold ( M ∗ M ) elements. Now we’ll show how to lay out the matrix to optimize for memory transfers (both using blocking and divide and conquer)... there can be ( N ) such blocks as B is the unit of transfer and the B N N counting transfers. Prove that we can achieve O(N 3 /B 3/2 ) memory transfers for a multiplication. Solution: √ √ Here. And a block B can be considered now as a square sub matrix.e. And the memory transfers will be B N O(( √M )3 ∗ M ) B N Upon simplification this becomes: O( B∗√M ) 3 In Big Oh notation it’ll be O( N 3 ) 3 .Problem 9 In class we saw how to analyze divide and conquer matrix multiplication of two N × N matrices. Solution: The recurrence relation for the following method is: T (n) = 8T ( n ) + O(n2 ) 2 Solving the above recurrence relation.e. This also infers that the N ∗ N matrix can now be considered as ( √B ∗ √B ) square matrix. M by M ).

This also infers that the N ∗ N matrix can now be considered as ( √B ∗ √B ) square matrix. A1. (a) What is the recursion relation for the cost of a multiplication? Solution: The recurrence relation for the cost of multiplication: T (n) = 8T ( n ) + O(n2 ) 2 (b) Solve the recursion relation and prove that the divide-and-conquer multiplication achieves √ O(N 3 /B M ) memory transfers. a Block of size B can hold ( B ∗ B) elements. And the total number of multi√ √ N plication will be ( √M )3 For each ( M ∗ M ) elements there will be M blocks. Therefore. the required number of 3 N multiplication will be: ( √B )3 . And a block B can be considered now as a square sub matrix. there can be ( N ) such blocks as B is the unit of transfer and the B N N counting transfers. Solution: Solving the recurrence relation. but good practice for understanding divide and conquer well) Now let’s explore divide and conquer solutions for matrix multiplication (as we discussed in class).Problem 10 (challenging problem. we’ll store the matrix in the same divide and conquer order. A3.g. a Block of size M can hold ( M ∗ M ) elements. A2. we get T(n) = O(n3 ) Proof: √ √ Now. 2 For a matrix of size N ∗ N . And the memory B N transfers will be O(( √M )3 ∗ M ) B N Upon simplification this becomes: O( B∗√M ) 3 . A4. e. We want to multiply A and B (each of size N × N ) and the product will be stored in C: A= A1 A2 A3 A4 B= B1 B2 B3 B4 C= C1 C2 C3 C4 The divide and conquer solution from class is: A1 A2 A3 A4 B1 B2 B3 B4 = A1 B1 + A2 B3 A1 B2 + A2 B4 A3 B1 + A4 B3 A3 B2 + A4 B4 To ensure good memory locality. In Big Oh notation it’ll be O( N 3 ) B2 √ √ Similarly. as per our solution to the recurrence relation.. Thus the matrix is stored in a zig-zag manner in memory.