31 views

Uploaded by Akshay Mehta

vv

- Openvdb Introduction
- Visi Rule
- alg_notes
- Vyom-Freshers-Guide-2007
- Darwin SYS
- Encouraging Viable Client System through Website Structure Change
- Integrating With Oracle HCM Cloud
- S2.DataStructuresandSQL
- A General Algorithm for Computing Distance Transforms in Linear Time
- Markov Models in Medical Decision Making - A Practical Guide -Med Decis Making-1993-Sonnenberg-322-38
- Lsm vs Fractal
- Data Structure
- PS User Security Setup
- L9 - Analysis of Algorithms
- ee220s02lec6 (1)
- Solved Question Papers 2009
- Sorting.pdf
- bst.ppt
- avl1
- CVXV

You are on page 1of 64

Professors Piotr Indyk and Charles E. Leiserson

6.046J/18.410J

Handout 7

Solution:

The following algorithm solves the problem:

1.Sort the elements in S using mergesort.

2.Remove the last element from S. Let y be the value of the removed element.

3.If S is nonempty, look for z = x y in S using binary search.

4.If S contains such an element z, then STOP, since we have found y and z such that x = y + z.

Otherwise, repeat Step 2.

5.If S is empty, then no two elements in S sum to x.

Notice that when we consider an element yi of S during ith iteration, we dont need to look at the

elements that have already been considered in previous iterations. Suppose there exists y j S,

such that x = yi + yj . If j < i, i.e. if yj has been reached prior to yi , then we would have found yi

when we were searching for x yj during jth iteration and the algorithm would have terminated

then.

Step 1 takes (n lg n) time. Step 2 takes O(1) time. Step 3 requires at most lg n time. Steps 24

are repeated at most n times. Thus, the total running time of this algorithm is (n lg n). We can do

a more precise analysis if we notice that Step 3 actually requires (lg(n i)) time at ith iteration.

However, if we evaluate n1

i=1 lg(n i), we get lg(n 1)!, which is (n lg n). So the total running

time is still (n lg n).

Exercise 1-2. Do Exercise 3.1-3 on page 50 in CLRS.

Exercise 1-3. Do Exercise 3.2-6 on page 57 in CLRS.

Exercise 1-4. Do Problem 3-2 on page 58 of CLRS.

Prove or disprove each of the following properties related to asymptotic notation. In each of the

following assume that f , g, and h are asymptotically nonnegative functions.

(a) f (n) = O(g(n)) and g(n) = O(f (n)) implies that f (n) = (g(n)).

Solution:

Since f (n) = O(g(n)), then there exists an n0 and a c such that for all n n0 , f (n)

cg(n). Similarly, since g(n) = O(f (n)), there exists an n0 and a c such that for all

n n0 , g (n) c

f (n). Therefore, for all n max(n0 , n0 ), c1 g(n) f (n) cg(n).

Hence, f (n) = (g(n)).

(b) f (n) + g(n) = (max(f (n), g(n))).

Solution:

For all n 1, f (n) max(f (n), g(n)) and g(n) max(f (n), g(n)). Therefore:

f (n) + g(n) max(f (n), g(n)) + max(f (n), g(n)) 2 max(f (n), g(n))

and so f (n) + g(n) = O(max(f (n), g(n))). Additionally, for each n, either f (n)

max(f (n), g(n)) or else g(n) max(f (n), g(n)). Therefore, for all n 1, f (n) +

g(n) max(f (n), g(n)) and so f (n) + g(n) = (max(f (n), g(n))). Thus, f (n) +

g(n) = (max(f (n), g(n))).

(c) Transitivity: f (n) = O(g(n)) and g(n) = O(h(n)) implies that f (n) = O(h(n)).

Solution:

Since f (n) = O(g(n)), then there exists an n0 and a c such that for all n n0 ,

f (n) cg(n). Similarly, since g(n) = O(h(n)), there exists an n 0 and a c such that

for all n n0 , g (n) c h(n). Therefore, for all n max(n0 , n0 ), f (n) cc

h(n).

Hence, f (n) = O(h(n)).

(d) f (n) = O(g(n)) implies that h(f (n)) = O(h(g(n)).

Solution:

and h(n) = 2n . Then h(f (n)) = 2n and h(g(n)) = 8n . Since 2n is not O(8n ), this

choice of f , g and h is a counter-example which disproves the theorem.

Solution:

Let h(n) = o(f (n)). We prove that f (n) + o(f (n)) = (f (n)). Since for all n 1,

Since h(n) = o(f (n)), then there exists an n0 such that for all n > n0 , h(n) f (n).

Therefore, for all n > n0 , f (n) + h(n) 2f (n) and so f (n) + h(n) = O(f (n)).

(f) f (n) = o(g(n)) and g(n) = o(f (n)) implies f (n) = (g(n)).

Solution:

n) and g(n) = 1 cos( n).

For all even values of n, f (n) = 2 and g(n) = 0, and there does not exist a c 1 for

which f (n) c1 g(n). Thus, f (n) is not o(g(n)), because if there does not exist a c 1

for which f (n) c1 g(n), then it cannot be the case that for any c1 > 0 and sufciently

large n, f (n) < c1 g(n).

For all odd values of n, f (n) = 0 and g(n) = 2, and there does not exist a c for which

g(n) cf (n). By the above reasoning, it follows that g(n) is not o(f (n)). Also, there

cannot exist c2 > 0 for which c2 g(n) f (n), because we could set c = 1/c2 if such

a c2 existed.

We have shown that there do not exist constants c1 > 0 and c2 > 0 such that c2 g(n)

f (n) c1 g(n). Thus, f (n) is not (g(n)).

Problem 1-2. Computing Fibonacci Numbers

The Fibonacci numbers are dened on page 56 of CLRS as

F0 = 0 ,

F1 = 1 ,

Fn = Fn1 + Fn2

for n 2 .

In Exercise 1-3, of this problem set, you showed that the nth Fibonacci number is

Fn =

n

n

,

5

is its conjugate.

A fellow 6.046 student comes to you with the following simple recursive algorithm for computing

the nth Fibonacci number.

F IB(n)

1 if n = 0

2

then return 0

3 elseif n = 1

4

then return 1

5 return F IB(n 1) + F IB(n 2)

This algorithm is correct, since it directly implements the denition of the Fibonacci numbers.

Lets analyze its running time. Let T (n) be the worst-case running time of F IB(n). 1

(a) Give a recurrence for T (n), and use the substitution method to show that T (n) =

O(Fn ).

We use the substitution method, inducting on n. Our Induction Hypothesis is: T (n)

cFn b.

To prove the inductive step:

T (n) cFn1 + cFn2 b b + 1

cFn 2b + 1

Therefore, T (n) cFn b + 1 provided that b 1. We choose b = 2 and c = 10.

For the base case consider n {0, 1} and note the running time is no more than

10 2 = 8.

(b) Similarly, show that T (n) = (Fn ), and hence, that T (n) = (Fn ).

Solution: Again the recurrence is: T (n) = T (n 1) + T (n 2) + 1.

We use the substitution method, inducting on n. Our Induction Hypothesis is: T (n)

Fn .

To prove the inductive step:

T (n) Fn1 + Fn2 + 1

Fn + 1

Therefore, T (n) Fn . For the base case consider n {0, 1} and note the running

time is no less than 1.

1

In this problem, please assume that all operations take unit time. In reality, the time it takes to add two num

bers depends on the number of bits in the numbers being added (more precisely, on the number of memory words).

However, for the purpose of this problem, the approximation of unit time addition will sufce.

Professor Grigori Potemkin has recently published an improved algorithm for computing the nth

Fibonacci number which uses a cleverly constructed loop to get rid of one of the recursive calls.

Professor Potemkin has staked his reputation on this new algorithm, and his tenure committee has

asked you to review his algorithm.

F IB (n)

1 if n = 0

2

then return 0

3 elseif n = 1

4

then return 1

5 sum 1

6 for k 1 to n 2

7

do sum sum + F IB (k)

8 return sum

Since it is not at all clear that this algorithm actually computes the nth Fibonacci number, lets

prove that the algorithm is correct. Well prove this by induction over n, using a loop invariant in

the inductive step of the proof.

(c) State the induction hypothesis and the base case of your correctness proof.

Solution: To prove the algorithm is correct, we are inducting on n. Our induction

hypothesis is that for all n < m, Fib (n) returns Fn , the nth Fibonacci number.

Our base case is m = 2. We observe that the rst four lines of Potemkin guarantee

that Fib (n) returns the correct value when n < 2.

(d) State a loop invariant for the loop in lines 6-7. Prove, using induction over k, that your

Solution: Our loop invariant is that after the k = i iteration of the loop,

sum = Fi+2 .

We prove this induction using induction over k. We assume that after the k = (i 1)

iteration of the loop, sum = Fi+1 . Our base case is i = 1. We observe that after the

rst pass through the loop, sum = 2 which is the 3rd Fibonacci number.

To complete the induction step we observe that if sum = Fi+1 after the k = (i1) and

if the call to F ib (i) on Line 7 correctly returns Fi (by the induction hypothesis of our

correctness proof in the previous part of the problem) then after the k = i iteration of

the loop sum = Fi+2 . This follows immediately form the fact that Fi + Fi+1 = Fi+2 .

(e) Use your loop invariant to complete the inductive step of your correctness proof.

Solution: To complete the inductive step of our correctness proof, we must show that

if F ib (n) returns Fn for all n < m then F ib (m) returns m. From the previous part

we know that if F ib (n) returns Fn for all n < m, then at the end of the k = i iteration

of the loop sum = Fi+2 . We can thus conclude that after the k = m 2 iteration of

the loop, sum = Fm which completes our correctness proof.

(f) What is the asymptotic running time, T (n), of F IB (n)? Would you recommend

Solution: We will argue that T (n) = (Fn ) and thus that Potemkins algorithm,

F ib does not improve upon the assymptotic performance of the simple recurrsive

algorithm, F ib. Therefore we would not recommend tenure for Professor Potemkin.

One way to see that T (n) = (Fn ) is to observe that the only constant in the program

is the 1 (in lines 5 and 4). That is, in order for the program to return F n lines 5 and 4

must be executed a total of Fn times.

Another way to see that T (n) = (Fn ) is to use the substitution method with the

k=1 T (k).

Problem 1-3. Polynomial multiplication

One can represent a polynomial, in a symbolic variable x, with degree-bound n as an array P [0 . . n]

of coefcients. Consider two linear polynomials, A(x) = a 1 x + a0 and B(x) = b1 x + b0 , where

a1 , a0 , b1 , and b0 are numerical coefcients, which can be represented by the arrays [a 0 , a1 ] and

[b0 , b1 ], respectively. We can multiply A and B using the four coefcient multiplications

m1

m2

m3

m4

=

=

=

=

a1 b1 ,

a1 b0 ,

a0 b1 ,

a0 b0 ,

C(x) = m1 x2 + (m2 + m3 )x + m4 ,

which can be represented by the array

[c0 , c1 , c2 ] = [m4 , m3 + m2 , m1 ] .

(a) Give a divide-and-conquer algorithm for multiplying two polynomials of degree-bound n,

represented as coefcient arrays, based on this formula.

Solution:

We can use this idea to recursively multiply polynomials of degree n 1, where n is

a power of 2, as follows:

Let p(x) and q(x) be polynomials of degree n 1, and divide each into the upper n/2

q(x) = c(x)xn/2 + d(x) ,

where a(x), b(x), c(x), and d(x) are polynomials of degree n/2 1. The polynomial

product is then

p(x)q(x) = (a(x)xn/2 + b(x))(c(x)xn/2 + d(x))

= a(x)c(x)xn + (a(x)d(x) + b(x)c(x))xn/2 + b(x)d(x) .

The four polynomial products a(x)c(x), a(x)d(x), b(x)c(x), and b(x)d(x) are com

puted recursively.

(b) Give and solve a recurrence for the worst-case running time of your algorithm.

Solution:

Since we can perform the dividing and combining of polynomials in time (n), re

cursive polynomial multiplication gives us a running time of

T (n) = 4T (n/2) + (n)

= (n2 ) .

(c) Show how to multiply two linear polynomials A(x) = a 1 x + a0 and B(x) = b1 x + b0

Solution:

We can use the following 3 multiplications:

m1 = (a + b)(c + d) = ac + ad + bc + bd ,

m2 = ac ,

m3 = bd ,

(ax + b)(cx + d) = m2 x2 + (m1 m2 m3 )x + m3 .

Solution:

The algorithm is the same as in part (a), except for the fact that we need only compute

three products of polynomials of degree n/2 to get the polynomial product.

(e) Give and solve a recurrence for the worst-case running time of your algorithm.

Solution:

Similar to part (b):

= (nlg 3 )

(n1.585 )

Alternative solution Instead of breaking a polynomial p(x) into two smaller poly

nomials a(x) and b(x) such that p(x) = a(x) + xn/2 b(x), as we did above, we could

do the following:

Collect all the even powers of p(x) and substitute y = x2 to create the polynomial

a(y). Then collect all the odd powers of p(x), factor out x and substitute y = x 2 to

create the second polynomial b(y). Then we can see that

p(x) = a(y) + x b(y)

Both a(y) and b(y) are polynomials of (roughly) half the original size and degree, and

we can proceed with our multiplications in a way analogous to what was done above.

2

Notice that, at each level k, we need to compute yk = yk1

(where y0 = x), which

takes time (1) per level and does not affect the asymptotic running time.

Introduction to Algorithms

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

October 1, 2004

6.046J/18.410J

Handout 9

Reading: Chapters 5-9, excluding 5.4 and 8.4

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudo-code.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

which are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

Exercise 2-2. Do Exercise 8.2-3 on page 170 in CLRS.

In this problem, we consider testing the equivalence of two polynomials in a nite eld.

A eld is a set of elements for which there are addition and multiplication operations that satisfy

commutativity, associativity, and distributivity. Each element in a eld must have an additive and

multiplicative identity, as well as an additive and multiplicative inverse. Examples of elds include

the real and rational numbers.

A nite eld has a nite number of elements. In this problem, we consider the eld of integers

modulo p. That is, we consider two integers a and b to be equal if and only if they have the same

remainder when divided by p, in which case we write a b mod p. This eld, which we denote as

Z/p, has p elements, {0, . . . , p 1}.

2

Consider a polynomial in the eld Z/p:

a(x) =

ai xi mod p

(1)

i=0

A root or zero of a polynomial is a value of x for which a(x) = 0. The following theorem describes

the number of zeros for a polynomial of degree n.

Theorem 1 A polynomial a(x) of degree n has at most n distinct zeros.

Polly the Parrot is a very smart bird that likes to play math games. Today, Polly is thinking of a

polynomial a(x) over the eld Z/p. Though Polly will not tell you the coefcients of a(x), she

will happily evaluate a(x) for any x of your choosing. She challenges you to gure out whether or

not a is equivalent to zero (that is, whether x {0, . . . , p 1} : a(x) 0 mod p).

Throughout this problem, assume that a(x) has degree n, where n < p.

(a) Using a randomized query, describe how much information you can obtain after a

single interaction with Polly. That is, if a is not equivalent to zero, then for a query x

chosen uniformly at random from {0, . . . , p1}, what is the probability that a(x) = 0?

Solution:

If a is not equivalent to zero, then the probability that a(x) = 0 is at most n/p. By

Theorem 1, a has at most n distinct zeros in {0, . . . , p 1}. As we are choosing x

uniformly at random from {0, . . . , p 1}, the probability of choosing any given x in

the eld is 1/p. Thus, the probability that the chosen x is a zero of a is at most n/p.

If a is equivalent to zero, then the probability that a(x) = 0 is 1. By denition, if a is

equivalent to zero, then a(x) = 0 for all x {0, . . . , p 1}.

(b) If n = 10 and p = 101, how many interactions with Polly do you need to be 99%

Solution:

Our strategy is to send k queries to Polly, each time evaluating a(x) for a random

x as described in Part (a). If each query evaluates to zero, then we report that a is

equivalent to zero. Otherwise, we report that a is not equivalent to zero.

In the event that a is equivalent to zero, then our report will always be correct (even

for k = 1). Thus, we focus on the case when a is not equivalent to zero. If we nd

that any query does not evaluate to zero, then our report will also be correct (since a

is not equivalent to zero for a certain x).

The problem thus becomes: if a is not equivalent to zero, choose k such that the

probability that all k queries evaluate to zero is no more than 1%. Let denote the

margin of error in the general case ( = 1% in this part), and let Q i be a random

variable indicating the result of the ith query. The constraint is as follows:

Pr[Q1 = 0 and Q2 = 0 and . . . and Qk = 0]

= Pr[Q1 = 0]Pr[Q2 = 0] . . . Pr[Qk = 0]

(n/p)k

The rst step follows from the fact that all of the queries are independent. The second

step utilizes the bound from Part (a). Solving for k, we have:

(n/p)k

k lg(n/p) lg

k lg / lg(n/p)

The last step above utilizes the assumption that n < p, and thus lg(n/p) < 0.

Substituting = 0.01, n = 10 and p = 101 gives k 1.991. Thus, k = 2 ensures that

we are 99% certain whether or not a is equivalent to zero.

Later, you are given three polynomials: a(x), b(x), and c(x). The degree of a(x) is n, while b(x)

and c(x) are of degree n/2. You are interested in determining whether or not a(x) is equivalent to

b(x) c(x); that is, whether x {0, . . . , p 1} : a(x) b(x) c(x) mod p.

Professor Potemkin recalls that in Problem Set 1, you showed how to multiply polynomials in

(nlg2 (3) ) time. Potemkin suggests using this procedure to directly compare the polynomials.

However, recalling your fun times with Polly, you convince Potemkin that there might be an even

more efcient procedure, if some margin of error is tolerated.

(c) Give a randomized algorithm that decides with probability 1 whether or not a(x)

is equivalent to b(x) c(x). Analyze its running time and compare to Potemkins

proposal.

Solution:

The algorithm checks the equivalence of a(x) and b(x) c(x) by testing if a(x)

b(x) c(x) is equivalent to zero. The testing is done using the randomized procedure

developed in Part (b), choosing k to ensure the desired level of certainty. Pseudocode

for the algorithm is as follows:

Returns whether a(x) b(x) c(x) mod p x {0, . . . , p 1}

Correct with probability at least 1

E QUIV(a[0 . . . n], b[0 . . . n/2], c[0 . . . n/2], p, )

1 k lg / lg(n/p)

2 for i 1 to k

3

do x R ANDOM(0, q 1)

4

a a(x)

5

b b(x)

6

c c(x)

7

if a b c 0 ( mod p)

8

then return false

9 return true

Correctness. For a given value of x, a(x) = b(x) c(x) if and only if a(x) b(x)

c(x) = 0. Thus, a(x) is equivalent to b(x) c(x) if and only if a(x) b(x) c(x) is

equivalent to zero. Our solution to Part (b) shows how to determine with probability

at least 1 whether or not a given polynomial is equivalent to zero. Using this same

procedure, we test whether or not a(x) b(x) c(x) is equivalent to zero, thereby

determining whether or not a(x) is equivalent to b(x) c(x).

Running time. We count the number of arithmetic operations in E QUIV. In this class,

we assume that steps 1 and 7 are (1), as they are arithmetic operations on scalar

values; also, we assume that the call to R ANDOM on line 3 is (1). Each polynomial

evaluation on lines 4-6 is (n), as an n-degree polynomial can be evaluated in (n)

time using Horners rule. The loop runs k = lg / lg(n/p) times, performing (n)

work on each iteration. The total runtime is thus T (n) = (nlg / lg(n/p)).

Consider Potemkins proposal, with a runtime P (n) = (n lg2 (3) ), and let us evalu

ate the conditions under which T (n) = O(P (n)). Note that, in the runtime of our

algorithm, we cannot consider p to be xed as n grows, since we require that n < p.

T (n) = O(P (n))

nlg / lg(n/p) cnlg2 (3)

lg / lg(n/p) cnlg2 (3)1

(2)

That is, the running time of our algorithm is bounded above by the running time of

Potemkins algorithm so long as p and satisfy Equation 2 for sufciently large n.

For example, consider that p = c n for some c > 1 and is a xed constant. Then

lg / lg(n/p) = lg / lg(1/c ) = (1). Thus, the loop executes (1) times and

the algorithm is has a running time of (n), which is asymptotically faster than

Potemkins proposal.

On the other hand, if p = n + 1 while remains xed, then lg / lg(n/p) =

lg / lg(n/(n + 1)) = lg /(lg n lg(n + 1))). Intuitively, one can see that this

d

(lg n) = 1/n, which means that lg(n + 1) lg n 1/n and

is (n) because dn

lg /(lg n lg(n + 1))) lg /(1/n) = (n). Thus, the loop executes (n)

times and the algorithm has a running time of (n2 ), which is asymptotically slower

than Potemkins proposal.

We can prove more rigorously that lg /(lg n lg(n + 1))) = (n) by appealing to

the following identity [CLRS, p.53]:

lim 1 +

n

1

n

=e

Then, using the denition of limit, there exist positive constants c 1 , c2 , and n0 such

that for all n > n0 :

1 n

c1 e 1 +

c2 e

n

1

ln(c1 e) n ln 1 +

ln(c2 e)

n

n+1

ln(c1 e) n ln

ln(c2 e)

n

ln(c1 e) n(ln(n + 1) ln(n)) ln(c2 e)

ln(c1 e)/n ln(n + 1) ln(n) ln(c2 e)/n

n/ ln(c1 e) 1/(ln(n + 1) ln(n)) n/ ln(c2 e)

n/ ln(c1 e) 1/(ln(n) ln(n + 1)) n/ ln(c2 e)

[simplify]

[simplify]

[divide by n]

[take inverse]

[simplify]

Thus, 1/(ln(n) ln(n + 1)) = (n) because it is bounded above and below by

a constant factor times n. By adjusting the constants, this implies that lg /(lg n

lg(n + 1))) = (n).

Finally, we point out the desirable property that the algorithm is logarithmic in for

xed values of p and n. Decreasing the error margin by a given factor results in only

an additive increase in the runtime.

Problem 2-2. Distributed Median

Alice has an array A[1..n], and Bob has an array B[1..n]. All elements in A and B are distinct.

Alice and Bob are interested in nding the median element of their combined arrays. That is, they

want to determine which element m satises the following property:

|{i [1, n] : A[i] m}| + |{i [1, n] : B[i] m}| = n

(3)

This equation says that there are a total of n elements in both A and B that are less than or equal to

m. Note that m might be drawn from either A or B.

Because Alice and Bob live in different cities, they have limited communication bandwidth. They

can send each other one integer at a time, where the value either falls within {0, . . . , n} or is drawn

from the original A or B arrays. Each numeric transmission counts as a communication between

Alice and Bob. One goal of this problem is to minimize the number of communications needed to

compute the median.

(a) Give a deterministic algorithm for computing the combined median of A and B. Your

algorithm should run in O(n log n) time and use O(log n) communications. (Hint:

consider sorting.)

Solution:

The algorithm works as follows. Alice and Bob begin by sorting their arrays using

a deterministic (n log n) algorithm such as HeapSort or MergeSort. Then, Alice

assumes the role of the master and Bob the role of the slave. Alice considers an

element A[i] and sends n i to Bob, who returns two elements: B[n i] and B[n

i + 1]. Because A is sorted, A[i] is the combined median if and only if there are

exactly n i elements in B that are less than A[i]. Because B is sorted, this condition

is reduced to checking whether or not B[n i] < A[i] < B[n i + 1]. Note that

when i = n, Bob needs to return B[0], which is out of range; in this case, he returns

B[0] = to satisfy the above condition.

Alice checks whether B[n i] < A[i] < B[n i + 1] and returns A[i] as the median

if the condition holds. If the condition fails, then she proceeds to do a binary search

within her array. The search is on i, with an initial range of [1, n]. On each step, she

descends into the top half of the range if A[i] is too small (i.e., A[i] < B[n i]); she

descends into the bottom half if A[i] is too big (i.e., A[i] > B[n i + 1]). If the

combined median is stored within A, the search is guaranteed to terminate at the right

spot, with B[n i] < A[i] < B[n i + 1], because A[i] B[n i] is a monotonically

increasing function of i while B[ni+1]A[i] is a monotonically decreasing function

of i.

If the combined median is not held in A, then Alices binary search terminates after

1 + lg n steps and returns a value of NIL. In this case, Alice and Bob swap roles,

with Bob becoming the master and Alice the slave. The procedure is repeated, and this

time the binary search returns the combined median because it must be stored within

B.

For clarity, pseudocode for this algorithm1 is given below.

1

Note that the DONE value used on line 15 of M ASTER and line 3 of S LAVE can be implemented by preceding

each communication with a 0 or 1 to indicate whether or not the following value indicates DONE. This does not impact

the asymptotic running time or communication cost.

A LICE(A[1 . . . n])

1 H EAP S ORT(A)

2 median M ASTER(A)

3 if median = NIL

4

then median S LAVE(A)

5 return median

B OB(B[1 . . . n])

1 H EAP S ORT(B)

2 median S LAVE(B)

3 if median = NIL

4

then median M ASTER(B)

5 return median

M ASTER(M [1 . . . n])

1 lower 1

2 upper n

3 median NIL

4 while lower upper and median = NIL

5

do i lower + (upper lower)/2

6

send n i

7

receive b1

8

receive b2

9

cur M [i]

10

if b1 < cur < b2

11

then median= cur

12

elseif cur < b1

13

then lower i +1

14

else upper i 1

15 send DONE

16 send median

17 return median

S LAVE(S[1 . . . n])

1 while true

2

do receive j

3

if j = DONE

4

then receive median

5

return median

6

if j = 0

7

then send

8

else send S[j]

9

send S[j + 1]

Running Time. Each individual statement is (1) except for the calls to HeapSort

(line 1 of A LICE and B OB), which requires (n lg n) time. The loop in M ASTER

performs a binary search, which (as we saw in lecture) requires (lg n) iterations.

Each iteration does (1) work, so the total running time for the loop is (lg n). The

loop in S LAVE terminates when it receives a DONE value, which happens exactly

when the loop in M ASTER terminates; thus, S LAVE is also (lg n). Alice and Bob

each execute M ASTER, S LAVE and HeapSort; HeapSort dominates, yielding a nal

running time of (n lg n).

Communication cost. Most of the communication is in the loop of M ASTER, in

which three items are relayed between Alice and Bob per each iteration. Since this

loop executes (lg n) times, it contributes (lg n) communications. The items sent

and received at the end of M ASTER contribute (1) communications, leaving the total

at (lg n).

Alternate Solution:

An alternate solution is simpler in some ways, but cannot be adapted to solve Part (b).

As before, Alice and Bob begin by sorting their arrays using a deterministic (n log n)

algorithm such as HeapSort or MergeSort. Then, Alice assumes the role of the master

and Bob the role of the slave. When Alice sends a value A[i] to Bob, Bob returns the

number of elements, count(A[i]), in his array that are less than A[i]. Because A is

sorted, the element A[i] is the combined median if and only if i + count(A[i]) = n.

Alice checks this condition and returns A[i] as the median if the condition holds. If the

condition fails, then she proceeds to do a binary search within her array. The search

is on i, with an initial range of [1, n]. On each step, she descends into the top half of

the range if i + count(A[i]) < n; she descends into the bottom half if i + count > n.

Because the quantity i+count (A[i]) is a monotonic function of i, the search terminates

with A[i] = i + count(A[i]) if the combined median is stored within A.

If the combined median is not held in A, then Alices binary search terminates after

1 + lg n steps and returns a value of NIL. In this case, Alice and Bob swap roles,

with Bob becoming the master and Alice the slave. The procedure is repeated, and this

time the binary search returns the combined median because it must be stored within

B.

For clarity, pseudocode for this algorithm is given below.

A LICE(A[1 . . . n])

1 H EAP S ORT(A)

2 median M ASTER(A)

3 if median = NIL

4

then median S LAVE(A)

5 return median

B OB(B[1 . . . n])

1 H EAP S ORT(B)

2 median S LAVE(B)

3 if median = NIL

4

then median M ASTER(B)

5 return median

M ASTER(M [1 . . . n])

1 lower 1

2 upper n

3 median NIL

4 while lower upper and median = NIL

5

do i lower + (upper lower)/2

6

send A[i]

7

receive count

8

if i + count = n

9

then median= M [i]

10

elseif i + count < n

11

then lower i +1

12

else upper i 1

13 send DONE

14 send median

15 return median

S LAVE(S[1 . . . n])

1 while true

2

do receive val

3

if val = DONE

4

then receive median

5

return median

6

else send |i [1, n] : S[i] val|

Running Time. All but three statements are (1) time. Both Alice and Bob call

HeapSort, which is (n lg n). Line 6 of S LAVE counts how many elements in S are

at most val. This can be implemented in (n) time with a brute-force comparison or,

because the array is sorted, in (lg n) time using a binary search. The last statements

of interest are lines 6-7 of M ASTER, which wait for one iteration of S LAVE. Since the

slave executes (lg n) operations between a receive and send statement, lines 6-7 of

M ASTER are also (lg n).

It remains to account for the loops. The loop in M ASTER is performing a binary

search, which (as we saw in lecture) requires (lg n) iterations. Each iteration does

(lg n) work, so the total running time for the loop is (lg 2 n). The loop in S LAVE

terminates when it receives a DONE value, which happens exactly when the loop

in M ASTER terminates; thus, S LAVE is also (lg 2 n). Alice and Bob each execute

M ASTER, S LAVE and HeapSort; HeapSort dominates, yielding a nal running time of

(n lg n).

Communication cost. Most of the communication is in the loop of M ASTER, in

which two items are relayed between Alice and Bob per each iteration. Since this

loop executes (lg n) times, it contributes (lg n) communications. The items sent

and received at the end of M ASTER contribute (1) communications, leaving the total

at (lg n).

(b) Give a randomized algorithm for computing the combined median of A and B. Your

algorithm should run in expected O(n) time and use expected O(log n) communica

Solution:

The algorithm is almost identical to Part (a). As before, Alice starts as the master

and conducts a binary search through Bobs elements, looking for A[i] such that

B[n i] < A[i] < B[n i + 1]. The only difference is that, instead of nding

the ith largest element by sorting the arrays and referencing A[i] directly, we use

R ANDOMIZED -S ELECT to nd the ith largest element in expected linear time. As

the algorithm is so close to Part (a), we describe only the differences below; in the

pseudocode, the modied lines are denoted by .

A LICE(A[1 . . . n])

1 H EAP S ORT(A)

2 median M ASTER(A)

3 if median = NIL

4

then median S LAVE(A)

5 return median

B OB(B[1 . . . n])

1 H EAP S ORT(B)

2 median S LAVE(B)

3 if median = NIL

4

then median M ASTER(B)

5 return median

10

M ASTER(M [1 . . . n])

1 lower 1

2 upper n

3 median NIL

5

do i lower + (upper lower)/2

6

send n i

7

send n upper

8

send n lower

9

receive b1

10

receive b2

11

cur R ANDOMIZED -S ELECT(M, lower, upper, i lower + 1)

12

if b1 < cur < b2

13

then median= cur

14

elseif cur < b1

15

then lower i +1

16

else upper i 1

17 send DONE

18 send median

19 return median

S LAVE(S[1 . . . n])

1 while true

2

do receive j

3

if j = DONE

4

then receive median

5

return median

6

receive bottom

7

receive top

8

if j = 0

9

then send

10

else send R ANDOMIZED -S ELECT(S, bottom, top, j bot + 1)

11

send R ANDOMIZED -S ELECT(S, bottom, top, j bot + 2)

Correctness. We need to show that all calls to R ANDOMIZED -S ELECT return the cor

responding values from Part (a). On line 11 of M ASTER, we have replaced M [i], the

ith smallest value of the sorted array M , with R ANDOMIZED -S ELECT(M, lower, upper, i

lower+1). To show that these values are equivalent, we need the following loop invari

ant I: before each iteration of the loop, M [lower . . . upper] contains the ith smallest

element in M for all i {lower, . . . , upper}. This invariant is true at the beginning,

since lower = 1, upper = n and all of the elements are within the range.

11

For the inductive step, assume I is true on the current iteration. Then the call to

R ANDOMIZED -S ELECT will partition around the ith smallest element in M , because

1) (by the inductive hypothesis) the smallest element in M [lower . . . upper] is the

lowerth smallest element in M , 2) by our call to R ANDOMIZED -S ELECT, we are

selecting for the i lowerth smallest element in M [lower . . . upper] (we also add 1

to compensate for the 1-based array indexing) and 3) lower + i lower = i. Thus, the

hypothesis will be satised for the range {lower, . . . , i} and {i, . . . , upper} because

the PARTITION subroutine of R ANDOMIZED -S ELECT will place elements on the ap

propriate side of the pivot i. Finally, the inductive hypothesis will hold on the next

iteration because we assign either lower or upper to be adjacent to i (but excluding i

from the next range).

Using the invariant, we conclude that the call to R ANDOMIZED -PARTITION on line

11 of M ASTER returns the ith smallest element in M , which is equivalent to the ex

pression M [i] from Part (a).

It remains to show the equivalent property for lines 10 and 11 of S LAVE. This is done

using the same loop invariant, but translating j = n i, bottom = n upper, and

top = n lower across the call between M ASTER and S LAVE. In this way, we can

show that lines 10 and 11 of S LAVE return the nith and ni+ 1th smallest elements

of S, respectively.

We have shown that our changes from Part (a) preserve the behavior of the algorithm,

and thus the algorithm remains correct.

Running Time. We can write a recurrence to model the running time of the main loop

in M ASTER. Let m = upperlower. On each iteration, m decreases to at most m/2

and R ANDOMIZED -S ELECT runs three times (once in M ASTER, twice in S LAVE)

over a segment of size m, with expected running time (m). Thus E[T (m)] =

E[T (m/2)] + (m), and by Case 3 of the Master Theorem, E[T (m)] = (m).

Finally, noting that m = n at the beginning of the procedure, we have that the ex

pected running time is (n).

Communication cost. The communication cost is identical to Part (a), as the loop

in M ASTER still executes (lg n) iterations and sends (1) items on each iteration.

Thus, the total number of communications is (lg n). (Note that this algorithm gives

a deterministic bound on the number of communications.)

You are consulting for a game show in which n contestants are pitted against n gladiators in order to

see which contestants are the best. The game show aims to rank the contestants in order of strength;

this is done via a series of 1-on-1 matches between contestants and gladiators. If the contestant is

stronger than the gladiator, then the contestant wins the match; otherwise, the gladiator wins the

12

match. If the contestant and gladiator have equal strength, then they are perfect equals and a

tie is declared. We assume that each contestant is the perfect equal of exactly one gladiator, and

each gladiator is the perfect equal of exactly one contestant. However, as the gladiators sometimes

change from one show to another, we do not know the ordering of strength among the gladiators.

The game show currently uses a round-robin format in which (n 2 ) matches are played and con

testants are ranked according to their number of victories. Since few contestants can happily endure

(n) gladiator confrontations, you are called in to optimize the procedure.

(a) Give a randomized algorithm for ranking the contestants. Using your algorithm, the

Solution:

The problem statement does not describe exactly how the contestants and gladiators

are specied, so we rst need to come up with a reasonable representation for the

input. Lets assume the contestants and gladiators are provided to us in two arrays

C[1 . . . n] and G[1 . . . n], where we are allowed to compare elements across, but not

within, these two arrays.

We use a divide-and-conquer algorithm very similar to randomized quicksort. The al

gorithm rst performs a partition operation as follows: pick a random contestant C[i].

Using this contestant, rearrange the array of gladiators into three groups of elements:

rst the gladiators weaker than C[i], then the gladiator that is the perfect equal of C[i],

and nally the gladiators stronger than C[i]. Next, using the gladiator that is the per

fect equal of C[i] we perform a similar partition of the array of contestants. This pair

of partitioning operations can easily be implemented in (n) time, and it leaves the

contestants and gladiators nicely partitioned so that the pivot contestant and glad

iator are aligned with each other and all other contestants and gladiators are on the

correct side of these pivots weaker contestants and gladiators precede the pivots,

and stronger contestants and gladiators follow the pivots. Our algorithm then nishes

by recursively applying itself to the subarrays to the left and right of the pivot position

to sort these remaining contestants and gladiators. We can assume by induction on n

that these recursive calls will properly sort the remaining contestants.

To analyse the running time of our algorithm, we can use the same analysis as that

of randomized quicksort. We are performing a partition operation in (n) time that

splits our problem into two subproblems whose sizes are randomly distributed ex

actly as would be the subproblems resulting from a partition in randomized quicksort.

Therefore, applying the analysis from quicksort, the expected running time of our

algorithm is (n log n).

Interesting side note: Although devising an efcient randomized algorithm for this

problem is not too difcult, it appears to be very difcult to come up with a deter

ministic algorithm with running time better than the trivial bound of O(n 2 ). This

13

remained an open research question until the mid-to-late 90s, when a very compli

cated deterministic algorithm with (n log n) running time was nally discovered.

This problem provides a striking example of how randomization can help simplify the

task of algorithm design.

(b) Prove that any algorithm that solves part (a) must use (n log n) matches in the worst

case. That is, you need to show a lower bound for any deterministic algorithm solving

this problem.

Solution:

Lets use a proof based on decision trees, as we did for comparison-based sorting.

Note that we can model any algorithm for sorting contestants and gladiators as a de

cision tree. The tree will be a ternary tree, since every comparison has three possible

outcomes: weaker, equal, or stronger. The height of such a tree corresponds to the

worst-case number of comparisons made by the algorithm it represents, which in turn

is a lower bound on the running time of that algorithm. We therefore want a lower

bound of (n log n) on the height, H, of any decision tree that solves part (a). To

begin with, note that the number of leaves L in any ternary tree must satisfy

L 3H .

Next, consider the following class of inputs. Let the input array of gladiators G be

xed and consist of n gladiators sorted in order of increasing strength, and consider

one potential input for every permutation of the contestants. Our algorithm must in

this case essentially sort the array of contestants. In our decision tree, if two different

inputs of this type were mapped to the same leaf node, our algorithm would attempt

to apply to both of these the same permutation of contestants, and it follows that the

algorithm could not compute a ranking correctly for both of these inputs. Therefore,

we must map every one of these n! different inputs to a distinct leaf node, so

L

3H

H

H =

n!

n!

log3 n!

(n log n) [Using Stirlings approximation]

Introduction to Algorithms

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

6.046J/18.410J

Handout 9

Reading: Chapters 12.1-12.4, 13, 18.1-18.3

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

Three-hole punch your paper on submissions.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudo-code.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

which are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

Exercise 3-2. Do Exercise 12.2-1 on page 259 in CLRS.

Exercise 3-3. Do Exercise 12.3-3 on page 264 in CLRS.

Exercise 3-4. Do Exercise 13.2-1 on page 278 in CLRS.

The computer science department makes a move to a new building offering the faculty and graduate

students boxes, crates and other containers. Prof. Potemkin, afraid of his questionable tenure case,

spends all of his time doing research and absentmindedly forgets about the move until the last

minute. His secretary advises him to use the only remaining boxes, which have capacity exactly

1 kg. His belongings consists of n books that weigh between 0 and 1 kilograms. He wants to

minimize the total number of used boxes.

Prof. Potemkin realizes that this packing problem is NP-hard, which means that the research

community has not yet found a polynomial time algorithm 1 that solves this problem exactly.

He thinks of the heuristic approach called BEST-PACK:

1.Take the books in the order in which they appear on his shelves.

2.For each book, scan the boxes in increasing order of the remaining capacity and place the

book in the rst box in which it ts.

(a) Describe a data structure that supports efcient implementation of BEST-PACK. Show

Solution: BEST-PACK can be implemented using any data structure that supports the

following three operations:

1. Insert(x), where x is an element and key[x] is a number

2. Delete(x)

3. Successor(x), which reports the smallest x such that key[x] k

There are several ways to obtain such a data structure. For example, one can use redblack trees or 2 3 trees. Because they are balanced, they support Insert, Delete and

Successor operations in O(log n) time. Even though the Successor operation was not

explained for 2 3 trees, they can be implemented by modifying search.

Our implementation is as follows: We use the remaining capacity of the boxes as the

key in the binary tree. Suppose that the elements weigh w1 , . . . , wn . Then, for a given

book with weight wi , if there are no boxes that are already used and whose remaining

capacity is greater than wi (i.e., the successor of wi ), then we assign wi to a new box.

(b) Analyze the running time of your implementation.

Solution: The BEST-PACK implementation performs O(n) operations on the data

structure which implies that the total running time is O(n log n)

That is, an algorithm with running time O(nk ) for some xed k.

Soon, Prof. Potemkin comes up with another heuristic WORST-PACK, which is as follows:

1.Take the books in the order in which they appear on his shelves.

2.For each book, nd a partially used box which has the maximum remaining capacity. If

possible, place the book in that box. Otherwise, put the book into a new box.

(c) Describe a data structure that supports an efcient implementation of WORST-PACK.

Solution: WORST-PACK can be implemented using any priority queue data structure.

We learned in recitation that a heap implements this data structure in O(log n) time.

You can also use a balanced search tree to implement these operations.

Our implementation is as follows: Pick a book. Delete the maximum from the priority

queue. If the capacity is greater than the weight of the book, insert the book and reduce

the capacity of the box. Reinsert the box in the priority queue. Otherwise pick a new

box and insert the book.

(d) Analyze the running time of your implementation.

Solution: Our implementation performs O(n) operations. This means that the total

running time is O(n log n).

4

Problem 3-2. AVL Trees

An AVL tree is a binary search tree with one additional structural constraint: For any of its internal

nodes, the height difference between its left and right subtree is at most one. We call this property

balance. Remember that the height is the maximum length of a path to the root.

For example, the following binary search tree is an AVL tree:

In this case, we can rebalance the tree by doing a simple operation, called a rotation, as follows:

=

Rotation

1

Unbalanced

Balanced

(a) If we insert a new element into an AVL tree of height 4, is one rotation sufcient to

re-establish balance? Justify your answer.

Solution: No, one rotation is not always sufcient to re-establish balance. For exam

ple, consider the insertion of the shaded node in the following AVL tree:

Though the original tree was balanced, more than one rotation is needed to restore

balance following the insertion. This can be seen by an exhaustive enumeration of the

rotation possibilities.

The problem asks for a tree of height 4, so we can extend the above example into a

larger tree:

(b) Denote the minimum number of nodes of an AVL tree of height h by M (h). A tree

of height 0 has one node, so M (0) = 1. What is M (1)? Give a recurrence for M (h).

Show that M (h) is at least Fh , where Fh is the hth Fibonacci number.

Solution: M (1) = 2. For h 2, the tree will consist of a root plus two subtrees.

Since the tree is of height h, one of the subtrees must be of height h 1. The minimum

number of nodes in this subtree is M (h1). Since the height of the subtrees can differ

by at most 1, the minimum number of nodes in the other subtree is M (h 2). Thus

the total number of nodes is M (h) = M (h 1) + M (h 2) + 1.

Note that M (h) is remarkably similar to the Fibonacci numbers and that the recursion

holds for the worse case AVL trees, which are called Fibonacci trees. It is easy to

F (h + 3) 1. Note that, as shown in Problem Set 1,

h+3

h

1+ 5

(c) Denote by n the number of nodes in an AVL tree. Note that n M (h). Give an upper

that h is O(lg n).

h+3

Introduction to Algorithms

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

6.046J/18.410J

Handout 18

Reading: Chapters 17, 21.121.3

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

Three-hole punch your paper on submissions.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudo-code.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

which are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

A father decides to start taking his young daughter to go skiing once a week. The daughter may

lose interest in the enterprise of skiing at any moment, so the kth week of skiing may be the last,

for any k. Note that k is unknown.

The father now has to decide how to procure skis for his daughter for every weekly session (until

she quits). One can buy skis at a one-time cost of B dollars, or rent skis at a weekly cost of R

dollars. (Note that one can buy skis at any timee.g., rent for two weeks, then buy.)

Give a 2-competitive algorithm for this problemthat is, give an online algorithm that incurs a

total cost of at most twice the ofine optimal (i.e., the optimal scheme if k is known).

Suppose we had code lying around that implemented a stack, and we now wanted to implement a

queue. One way to do this is to use two stacks S1 and S2 . To insert into our queue, we push into

stack S1 . To remove from our queue we rst check if S2 is empty, and if so, we dump S1 into S2

(that is, we pop each element from S1 and push it immediately onto S2 ). Then we pop from S2 .

For instance, if we execute I NSERT(a), I NSERT(b), D ELETE(), the results are:

S1

I NSERT(a) S1

I NSERT(b) S1

D ELETE() S1

S1

=[]

S2 =[]

=[a]

S2 =[]

=[b a] S2 =[]

=[]

S2 =[a b] dump

=[]

S2 =[b]

pop (returns a)

Suppose each push and pop costs 1 unit of work, so that performing a dump when S 1 has n elements

costs 2n units (since we do n pushes and n pops).

(a) Suppose that (starting from an empty queue) we do 3 insertions, then 2 removals,

then 3 more insertions, and then 2 more removals. What is the total cost of these 10

operations, and how many elements are in each stack at the end?

elements, and S2 has 2.

(b) If a total of n insertions and n removals are done in some order, how large might the

running time of one of the operations be (give an exact, non-asymptotic answer)? Give

a sequence of operations that induces this behavior, and indicate which operation has

Solution: An insertion always takes 1 unit, so our worst-case cost must be caused by

a removal. No more that n elements can ever be in S1 , and no fewer than 0 elements

can be in S2 . Therefore the worst-case cost is 2n + 1: 2n units to dump, and one extra

to pop from S2 . This bound is tight, as seen by the following sequence: perform n

insertions, then n removals. The rst removal will cause a dump of n elements plus a

pop, for 2n + 1 work.

(c) Suppose we perform an arbitrary sequence of insertions and removals, starting from

an empty queue. What is the amortized cost of each operation? Give as tight (i.e.,

non-asymptotic) of an upper bound as you can. Use the accounting method to prove

your answer. That is, charge $x for insertion and $y for deletion. What are x and y?

Solution: The tightest amortized upper bounds are 3 units per insertion, and 1 unit per

removal. We will prove this 2 ways (using the accounting and potential methods; the

aggregate method seems too weak to employ elegantly in this case). (We would also

accept valid proofs of 4 units per insertion and 0 per removal, although this answer is

looser than the one we give here.)

Here is an analysis using the accounting method: with every insertion we pay $3: $1

is used to push onto S1 , and the remaining $2 remain attached to the element just

inserted. Therefore every element in S1 has $2 attached to it. With every removal we

pay $1, which will (eventually) be used to pop the desired element off of S 2 . Before

that, however, we may need to dump S1 into S2 ; this involves popping each element

off of S1 and pushing it onto S2 . We can pay for these pairs of operations with the $2

attached to each element in S1 .

(d) Now well analyze the structure using the potential method. For a queue Q imple

Use this potential function to analyze the amortized cost of insert and delete opera

tions.

Solution: Let |S1i | denote the number of elements in S1 after the ith operation. Then

the potential function on our structure Qi (the state of the queue after the ith oper

ation) is dened to be (Qi ) = 2|S1

i |. Note that |S1i | 0 at all times, so (Qi ) 0.

Also, |S10 | = 0 initially, so (Q0 ) = 0 as desired.

Now we compute the amortized costs: for an insertion, we have S 1i+1 = S1i + 1, and

the actual cost ci = 1, so

ci = ci + (Qi+1 ) (Qi ) = 1 + 2(S1i + 1) 2(S1i ) = 3.

For a removal, we have two cases. First, when there is no dump from S 1 to S2 , the

actual cost is 1, and S1

i+1 = S1i . Therefore ci = 1. When there is a dump, the actual

cost is 2|S1 i | + 1, and we have S1i+1 = 0. Therefore we get

ci = (2|S1i | + 1) + 0 2|S1i | = 1

as desired.

Problem 4-2. David Digs Donuts

Your TA David has two loves in life: (1) roaming around Massachusetts on his forest-green Can

nondale R300 road bike, and (2) eating Boston Kreme donuts. One Sunday afternoon, he is biking

along Main Street in Acton, and suddenly turns the corner onto Mass Ave. (Yes, that Mass Ave.)

His growling stomach announces that it is time for a donut. Because Mass Ave has so many donut

shops along it, David decides to nd a shop somewhere along that street. He faces two obstacles in

his quest to satisfy his hunger: rst, he does not know whether the nearest donut shop is to his left

or to his right (or how far away the nearest shop is); and second, when he goes riding his contact

lenses dry out dramatically, blurring his vision, and he cant see a donut shop until he is directly in

front of it.

You may assume that all donut shops are at an integral distance (in feet) from the starting location.

(a) Give an efcient (deterministic) algorithm for David to locate a donut shop on Mass

Ave as quickly as possible. Your algorithm will be online in the sense that the location

of the nearest donut shop is unknown until you actually nd the shop. The algorithm

should be O(1)-competitive: if the nearest donut shop is distance d away from Davids

starting point, the total distance that David has to bike before he gets his donut should

be O(d). (The optimal ofine algorithm would require David to bike only distance d.)

Solution: WLOG, lets call the two directions of Mass Ave east and west.

1.

2.

3.

4.

i := 0.

direction := east;

Repeat the following until a donut is found:

(a) Bike 2i units in direction direction. If you pass a donut shop, stop and eat.

(b) Bike 2i units back to the origin.

(c) i := i + 1.

(d) direction := direction.

Notice that you are back at the origin after every iteration of the loop.

Suppose that the nearest donut shop is d feet away from the origin. Let k be such that

2k < d 2k+1 . Observe that in the ith iteration of the loop, we explore the stretch of

road between the origin and 2i , so after k + 2 iterations we are guaranteed to have

The total distance that we travel in the ith iteration is 22i , so the total distance traveled

i+1

in these k + 2 iterations is k+2

2 2k+3 = 16 2k < 16d. Thus the algorithm

i=0 2

is 16-competitive.

(b) Optimize the competitive ratio for your algorithmthat is, minimize the constant hid

Solution: The only tweak to the above is to more tightly analyze the last iteration.

In the (k + 2)nd iteration, we are done after we travel distance d, since we were

at the origin and had to travel only distance d. Thus the total distance we travel is

k+1 i+1

+ d 8 2k + d 9d. (One can also try to optimize the base of the

i=0 2

exponential search, but it turns out that two is optimal.)

(c) Suppose you ip a coin to decide whether to start moving to the left or to the right ini

tially. Show that incorporating this step into your algorithm results in an improvement

Solution: Then, using the above notation, with probability 1/2 we will nd the shop

in the (k + 1)st iteration, and with probability 1/2 we will nd it in the (k + 2)nd

(

k+1

i=0

k+2

2

k + d)/2 + (

i=0

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

6.046J/18.410J

Handout 21

Reading: Chapters 15, 16

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

Three-hole punch your paper on submissions.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudo-code.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

which are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

Exercise 4-3. Do exercise 15.4-4 on page 356 in CLRS and show how to reconstruct the actual

In this problem you will write a program (real code that runs!!!) to solve the following typesetting

problem. Because of the trouble you may encounter while programming, we advise you to

START THIS PROBLEM AS SOON AS POSSIBLE.

You have an input text consisting of a sequence of n words of lengths 1 , 2 , . . . , n , where the

length of a word is the number of characters it contains. Your printer can only print with its built-in

Courier 10-point xed-width font set that allows a maximum of M characters per line. (Assume

that i M for all i = 1, . . . , n.) When printing words i and i + 1 on the same line, one space

character (blank) must be printed between the two words. Thus, if words i through j are printed

on a line, the number of extra space characters at the end of the linethat is, after word jis

M j + i jk=i k .

To produce nice-looking output, the heuristic of setting the cost to the square of the number of

extra space characters at the end of the line has empirically shown itself to be effective. To avoid

the unnecessary penalty for extra spaces on the last line, however, the cost of the last line is 0. In

other words, the cost linecost(i, j) for printing words i through j on a line is given by

linecost(i, j) =

M j + i j

k

k=i

if j = n (i.e. last line),

otherwise.

The total cost for typesetting a paragraph is the sum over all lines in the paragraph of the cost of

each line. An optimal solution is an arrangement of the n words into lines in such a way that the

total cost is minimized.

(a) Argue that this problem exhibits optimal substructure.

Solution: First, notice that linecost(i, j) is dened to be if the words i through j

do not t on a line to guarantee that no lines in the optimal solution overow. (This

relies on the assumption that the length of each word is not more than M .) Second,

notice that linecost(i, j) is dened to be 0 when j = n, where n is the total number

of words; only the actual last line has zero cost, not the recursive last lines of subprob

lems, which, since they are not the last line overall, have the same cost formula as any

other line.

Consider an optimal solution of printing words 1 through n. Let i be the index of the

rst word printed on the last line of this solution. Then typesetting of words 1, . . . , i1

must be optimal. Otherwise, we could paste in an optimal typesetting of these words

and improve the total cost of solution, a contradiction. Please notice that the same

cut-and-paste argument can be applied if we take i to be the index of the rst word

printed on the kth line, where 2 k n. Therefore this problem displays optimal

substructure.

(b) Dene recursively the value of an optimal solution.

Solution: Let c(j) be the optimal cost of printing words 1 through j. From part (a),

we see that given the optimal i (i.e., the index of the rst word printed on the last line

of an optimal solution), we have c(j) = c(i 1) + linecost(i, j). But since we do

not know what i is optimal, we need to consider every possible i, so our recursive

denition of the optimal cost is

c(j) = min {c(i 1) + linecost(i, j)} .

1ij

(c) Describe an efcient algorithm to compute the cost of an optimal solution.

Solution: We calculate the values of an array for c from index 1 to n, which can be

done efciently since each c(k) for 1 k < j will be available by the time c(j) is

computed. To keep track of the actual optimal arrangement of the words, we record an

array p, where p(k) is the i (in the recursive denition of c) which led to the optimal

c(k). Then, after the arrays for c and p are computed, the optimal cost is c(n) and the

optimal solution can be found by printing words p(n) through n on the last line, words

p(p(n) 1) through p(n) 1 on the next to last line, and so on.

Notice that computing linecost(i, j) takes in general O(j i + 1) time because of

summation in the formula. However, it is possible to do this computation in O(1) time

with some additional pre-processing. We create an auxillary array L[0 . . . n], where

L[i] is a cumulative sum of lengths of words 1 thru i.

L[0] = 0

L[i] = L[i 1] + i =

k=1

Filling in this array takes O(n) time using recursion. Using the auxillary array we

linecost(i, j) = 0

if j = n (i.e. last line),

otherwise.

This algorithm uses (n) space for the arrays and runs in O(n2 ) time, since each value

(M + 1)/2 words can t on a single line, we can reduce running time to O(nM )a

(d) Write code (in any language you wisheven Visual Java ++ :-) 1 ) to print an optimal

arrangement of the words into lines. For simplicity, assume that a word is any se

quence of characters not including blanksso a word is everything included between

two space characters (blanks).

1

(d) requires 5 parts: you should turn in the code you have written, and the output of your program

on the two input samples using two values of M (the maximum number of characters per line),

namely M = 72 and M = 40, on each input sample.

Sample 1 is from A Capsule History of Typesetting by Brown, R.J. Sample 2 is from Out of Their

Minds, by Shasha, Lazere. Remember that collaboration, as usual, is allowed to solve problems,

but you must write your program by yourself.

/* NOTE: This is an implementation of the O(nM) algorithm. */

#include <stdio.h>

#include <limits.h>

/* macros */

int auxL[WORD_NUM+1];

/* auxillary array for computing lengths

of lines - MM*/

/* function prototypes */

/* verify arguments */

exit(1);

exit(2);

exit(3);

n = 1;

while(!feof(ifile)) {

strcpy(words[n++], read_word);

if(n == WORD_NUM)

n--;

auxL[0] = 0;

l = 0;

do {

l++;

strcat(lines[l], words[i]);

int i, j;

long c[WORD_NUM+1];

c[j] = LONG_MAX;

int k;

if(extras < 0)

return 0;

else

Solutions:

sample1 72

COST = 160

1:[67] The first practical mechanized type casting machine was invented in

2:[69] 1884 by Ottmar Mergenthaler. His invention was called the "Linotype".

3:[72] It produced solid lines of text cast from rows of matrices. Each matrice

5:[69] letter had been engraved or stamped. The line-composing operation was

11:[70] the punched code into electrical signals that could be sent by wire to

12:[71] tape-punching units in many cities simultaneously. The first major news

sample1 40

COST = 360

sample2 72

COST = 229

1:[65] Throughout his life, Knuth had been intrigued by the mechanics of

2:[70] printing and graphics. As a boy at Wisconsin summer camp in the 1940s,

3:[71] he wrote a guide to plants and illustrated the flowers with a stylus on

4:[69] the blue ditto paper that was commonly used in printing at that time.

5:[71] In college, he recalls admiring the typeface used in his math texbooks.

6:[71] But he was content to leave the mechanics of designing and setting type

7:[72] to the experts. "I never thought I would have any control over printing.

8:[71] Printing was done by typographers, hot lead, scary stuff. Then in 1977,

9:[71] I learned about new printing machines that print characters made out of

10:[69] zeros and ones, just bits, no lead. Suddenly, printing was a computer

12:[66] tools using the new technology with which to write my next books."

13:[67] Knuth designed and implemented TeX, a computer language for digital

15:[68] thoroughness. For example, he wrote a paper called "The letter S" in

16:[67] which he dissects the mathematical shape of that letter through the

17:[67] ages, and explains his several day effort to find the equation that

sample2 40

COST = 413

33:[8] outline

Here is what Sample 1 should look like when typeset with M = 50. Feel free to use this output

to debug your code.

The first practical mechanized type casting

(e) Suppose now that the cost of a line is dened as the number of extra spaces. That is,

when words i through j are put into a line, the cost of that line is

0

if j = n (i.e. last line),

linecost(i, j) =

j

M j + i k=i k otherwise;

and that the total cost is still the sum over all lines in the paragraph of the cost of each

line. Describe an efcient algorithm that nds an optimal solution in this case.

Solution: We use a straightforward greedy algorithm, which puts as many words as

possible on each line before going to the next line. Such an algorithm runs in linear

time.

Now we show that any optimal solution has the same cost as the solution obtained by

this greedy algorithm. Consider some optimal solution. If this solution is the same as

the greedy solution, then we are done. If it is different, then there is some line i which

has enough space left over for the rst word of the next line. In this case, we move

the rst word of line i + 1 to the end of line i. This does not change the total cost,

since if the length of the word moved is l, then the reduction to the cost of line i will

10

be l + 1, for the word and the space before it, and the increase of the cost of line i + 1

will also be l + 1, for the word and the space after it. (If the moved word was the only

word on line i + 1, then by moving it to the previous line the total cost is reduced, a

contradiction to the supposition that we have an optimal solution.) As long as there

are lines with enough extra space, we can keep moving the rst words of the next lines

back without changing the total cost. When there are no longer any such lines, we will

have changed our optimal solution into the greedy solution without affecting the total

cost. Therefore, the greedy solution is an optimal solution.

Problem 4-2. Manhattan Channel Routing

A problem that arises during the design of integrated-circuit chips is to hook components together

with wires. In this problem, well investigate a simple such problem.

In Manhattan routing, wires run on one of two layers of an integrated circuit: vertical wires run

on layer 1, and horizontal wires run on layer 2. The height h is the number of horizontal tracks

used. Wherever a horizontal wire needs to be connected to a vertical wire, a via connects them.

Figure 1 illustrates several pins (electrical terminals) that are connected in this fashion. As can be

seen in the gure, all wires run on an underlying grid, and all the pins are collinear.

In our problem, the goal is to connect up a given set of pairs of pins using the minimum number

of horizontal tracks. For example, the number of horizontal tracks used in the routing channel of

Figure 1 is 3 but fewer might be sufcient.

Let L = {(p1 , q1 ), (p2 , q2 ), . . . , (pn , qn )} be a list of pairs of pins, where no pin appears more than

once. The problem is to nd the fewest number of horizontal tracks to connect each pair. For exam

ple, the routing problem corresponding to Figure 1 can be specied as the set {(1, 3), (2, 5), (4, 6), (8, 9)}.

(a) What is the minimum number of horizontal tracks needed to solve the routing problem

in Figure 1?

Solution: You can verify that the wire connecting pins 4 and 6 could be at the same

height as the wire connecting pins 1 and 3, making the number of horizontal track

needed 2. Note that this is the minimum possible. Otherwise the wire connecting pins

2 and 3 and the wire connecting 1 and 3 would be on the same track, violating the

problem specications.

(b) Give an efcient algorithm to solve a given routing problem having n pairs of pins us

ing the minimum possible number of horizontal tracks. As always, argue correctness

(your algorithm indeed minimizes the number of horizontal tracks), and analyze the

running time.

Algorithm description

The following algorithm routes pin pairs greedily into available horizontal tracks in

order of the smaller pin of a pair.

11

h=3

Figure 1: Pins are shown as circles. Vertical wires are shown as solid. Horizontal wires are dashed.

Vias are shown as squares.

1. Go through L and if qi > pi swap them. For each pair, we call the smaller pin,

start, and the larger, end. Sort the start and end pins of the pairs in an increasing

order. The resulting list contains 2n values.

2. Place all available horizontal tracks in a stack S.

3. Go through the list in sorted order.

If the current pin is a start pin, pop the rst available horizontal track from S

and route it in that horizontal track. If S is empty, then it is not possible to

route all of the pin pairs using the given number of horizontal tracks. Report

an error in this case.

If the current pin is an end pin, look up in which horizontal track it has been

routed and push that horizontal track back onto S.

Correctness

Suppose that the algorithm terminates with a routing requiring m horizontal tracks.

Let k denote the rst pair of pins routed in the mth horizontal track. Let s k denote

the start pin and fk denote the end pin of this pair. Let fl be the earliest nish pin

appearing in the sorted list after sk . Necessarily, fl > sk . The closest routing in each

of the m 1 horizontal tracks already in use starts before sk . Each routing terminates

after fl . Thus there are m routings in between [sk , fl ], i.e., any routing must use at

least m vertical tracks. Thus the routing returned by the algorithm is optimal.

The above argument shows that given an innite supply of horizontal tracks, our algo

rithm will always produce a routing that uses the fewest number of horizontal tracks.

Thus, if the algorithm terminates with an error, it means that a given number of hor

izontal tracks is less than the number of horizontal tracks in an optimal routing, and

12

hence it is impossible to route all the pin pairs.

Analysis

This algorithm runs in O(n lg n) because it is necessary to sort 2n items (which can

be accomplished using heapsort or mergesort). Notice that scanning through the list

and assigning horizontal tracks takes O(1) time per connection, for a total of O(2n) =

O(n) time.

This is also known as the interval-graph coloring problem. We can create an interval

graph whose vertices are the given pairs of pins and whose edges connect incompatible

pairs of pins. The smallest number of colors required to color every vertex so that

no two adjacent vertices are given the same color corresponds to nding the fewest

number of horizontal tracks needed to connect all of the pairs of pins.)

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

6.046J/18.410J

Handout 26

Reading: Chapters 22, 24, and 25.

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudo-code.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

which are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

Exercise 6-2. Do Exercise 22.4-3 on page 552 in CLRS.

Exercise 6-3. Do Exercise 22.5-7 on page 557 in CLRS.

Exercise 6-4. Do Exercise 24.1-3 on page 591 in CLRS.

Exercise 6-5. Do Exercise 24.3-2 on page 600 in CLRS.

Exercise 6-6. Do Exercise 24.4-8 on page 606 in CLRS.

Exercise 6-7. Do Exercise 25.2-6 on page 635 in CLRS.

Exercise 6-8. Do Exercise 25.3-5 on page 640 in CLRS.

2

Problem 6-1. Truckin

Professor Almanac is consulting for a trucking company. Highways are modeled as a directed

graph G = (V, E) in which vertices represent cities and edges represent roads. The company is

planning new routes from San Diego (vertex s) to Toledo (vertex t).

(a) It is very costly when a shipment is delayed en route. The company has calculated the

probability p(e) [0, 1] that a given road e E will close without warning. Give an

efcient algorithm for nding a route with the minimum probability of encountering

a closed road. You should assume that all road closings are independent.

Solution:

To simplify the solution, we use the probability q(e) = 1 p(e) that a road will be

open. Further, we remove from the graph roads with p(e) = 1, as they are guaranteed

to be closed and will never be included in a meaningful solution. (Following this

transformation, we can use depth rst search to ensure that some path from s to t

has a positive probability of being open.) By eliminating p(e) = 1, we now have

0 < q(e) 1 for all edges e E. It is important to have eliminated the possibility of

q(e) = 0, because we will be taking the logarithm of this quantity later.

Because the road closings are independent, the probability that a given path will be

open is the product of the probabilities of the edges being open. That is, for each path

r = e1 , e2 , . . . , en , the probability Q(r) of the path being open is:

Q(r) =

q(ei )

i=1

Our goal is to nd the path r, beginning at s and ending at t, that maximizes Q(r).

Taking the negative logarithm of both sides yields:

lg Q(r) = lg

q(ei )

i=1

lg q(ei )

i=1

lg Q(r) =

w(ei )

i=1

The right hand side is a sum of edge weights w(e) along the path from s to t. We

can minimize this quantity using Dijkstras algorithm for single-source shortest paths.

Doing so will yield the path r that minimizes lg Q(r), thereby maximizing Q(r).

This path will have the maximum probability of being open, and thus the minimum

probability of being closed.

The running time is O(E+V lg V ). We only spend (E) to remove edges with p(e) =

1 and to update the edge weights; Dijkstras algorithm dominates with a runtime of

O(E + V lg V ).

Alternate Solution:

It is also possible to modify Dijkstras algorithm to directly compute the path with the

highest probability of being open. As above, let q(e) = 1p(e) denote the probability

that a given road e E will be open, and let Q(r) denote the probability that a given

path r will be open. For a given vertex v, let o[v] denote the maximum value of Q(r)

over all paths r from s to v. Then, make the following modications to Dijkstras

algorithm:

1. Change I NITIALIZE -S INGLE -S OURCE to assign o[s] = 1 for the source vertex

and o[v] = 0 for all other vertices:

I NITIALIZE -S INGLE -S OURCE (G, s)

1 for each vertex v V [G]

2

do o[v] 0

3

[v] NIL

4 o[s] 1

That is, we can reach the source vertex with probability 1, and the probability of

reaching all other vertices will increase monotonically from 0 using R ELAX.

2. Instead of E XTRACT-M IN, use E XTRACT-M AX (and a supporting data structure)

to see which vertex to visit. That is, rst explore paths with the highest probability

of being open.

3. Rewrite the R ELAX step as follows:

R ELAX (u, v, q)

1 if o[v] < o[u] q(e) where e = (u, v)

2

then o[v] o[u] q(e)

3

[v] u

That is, if a vertex v can be reached with a higher probability than before along

the edge under consideration, then increase the probability o[v]. Because the

probabilities of roads being open are independent, the probability of a path being

open is the product of the probabilities of each edge being open.

The argument for correctness parallels that of Dijkstras algorithm, as presented in

lecture. The proof relies on the following properties:

1. Optimal substructure. A sub-path of a path with the highest probability of being

open must also be a path with the highest probability of being open. Otherwise we

could increase the overall probability of being open by increasing the probability

along this sub-path (cut-and-paste).

2. Triangle inequality. Let (u, v) denote the highest probability of a path from u

to v being open. Then for all u, v, x V , we have (u, v) (u, x) (x, v).

Otherwise (u, v) could be increased if we chose the path through x.

3. Well-denedness of shortest paths. Since q(e) [0, 1], the probability of a path

being open can only decrease as extra edges are added to a path. Since we are

exploring the path with the highest probability of being open, this ensures that

there are no analogues of negative-weight cycles.

Properties (1) and (2) are true whenever an associative operator is used to combine

edge weights into a path weight. In Dijkstras shortest path algorithm, the operator is

addition; here it is multiplication.

The running time is O(E + V lg V ). We spend (E) to calculate q(e) = 1 p(e),

and Dijkstras algorithm runs in O(E + V lg V ).

(b) Many highways are off-limits for trucks that weigh more than a given threshold. For a

given highway e E, let w(e) + denote the weight limit and let l(e) + denote

the highways length. Give an efcient algorithm that calculates: 1) the heaviest truck

that can be sent from s to t, and 2) the shortest path this truck can take.

Solution:

First, we modify Dijkstras algorithm to nd the heaviest truck that can be sent from

s to t. The weight limit w(e) is used as the edge weight for e. There are three modi

cations to the algorithm:

1. In I NITIALIZE -S INGLE -S OURCE, assign a value of to the source vertex and a

value of 0 to all other vertices.

2. Instead of E XTRACT-M IN, use E XTRACT-M AX (and a supporting data structure)

to see which vertex to visit. That is, rst explore those paths which support the

heaviest trucks.

3. In the R ELAX step, use min in place of addition. That is, maintain the minimum

weight limit encountered on a given path instead of the total path length from the

source.

As in Part (a), the proof of correctness follows that of Dijkstras algorithm. Since the

min operator is associative, the optimal paths exhibit optimal substructure and support

the triangle inequality. There are no analogues of negative-weight cycles because the

weight supported by a path can only decrease as the path becomes longer (and we are

searching for the heaviest weight possible).

Given the weight of the heaviest truck that can pass from s to t, we can nd the shortest

path as follows. Simply remove all edges from the graph that are less than the weight

of the heaviest truck. Then, run Dijkstras algorithm (unmodied) to nd the shortest

path.

The overall runtime of our algorithms is that of Dijkstras algorithm: O(E + V lg V ).

(c) Consider a variant of (b) in which trucks must make strictly eastward progress with

each city they visit. Adjust your algorithm to exploit this property and analyze the

runtime.

Solution:

Remove from the graph any edges that do not make eastward progress. Because

we always go eastward, there are no cycles in this graph. Thus, we can use DAG S HORTEST-PATHS to solve the problem in (V + E) time. We need only modify the

I NITIALIZE -S INGLE -S OURCE and R ELAX procedures as in (b).

Problem 6-2. Constructing Construction Schedules

Consider a set of n jobs to be completed during the construction of a new ofce building. For

each i {1, 2, . . . , n}, a schedule assigns a time xi 0 for job i to be started. There are some

constraints on the schedule:

1. For each i, j {1, 2, . . . , n}, we denote by A[i, j] the minimum latency from the start

of job i to the start of job j. For example, since it takes a day for concrete to dry, construction

of the walls must begin at least one day after pouring the foundation. The constraint on the

schedule is:

i, j {1, 2, . . . , n} : xi + A[i, j] xj

(1)

2. For each i, j {1, 2, . . . , n}, we denote by B[i, j] the maximum latency from the start

of job i to the start of job j. For example, weatherproong must be added no later than one

week after an exterior wall is erected. The constraint on the schedule is:

i, j {1, 2, . . . , n} : xi + B[i, j] xj

(2)

(a) Show how to model the latency constraints as a set of linear difference equations. That

is, given A[1 . . n, 1 . . n] and B[1 . . n, 1 . . n], construct a matrix C[1 . . n, 1 . . n] such

that the following constraints are equivalent to Equations (1) and (2):

i, j {1, 2, . . . n} : xi xj C[i, j]

(3)

Solution:

Re-arranging Equation (1) yields:

i, j {1, 2, . . . , n} : xi xj A[i, j]

(4)

6

Re-arranging Equation (2) yields:

i, j {1, 2, . . . , n} : xi xj B[i, j]

i, j {1, 2, . . . , n} : xj xi B[i, j]

i, j {1, 2, . . . , n} : xi xj B[j, i]

(5)

i, j {1, 2, . . . , n} : C[i, j] = min(A[i, j], B[j, i])

(b) Show that the Bellman-Ford algorithm, when run on the constraint graph correspond

ing to Equation (3), minimizes the quantity (max{xi } min{xi }) subject to Equation

Solution:

Recall that the Bellman-Ford algorithm operates on a graph in which each constraint

xj xi C[i, j] is translated to an edge from vertex vi to vertex vj with weight

wij = C[i, j]. Also, an extra vertex s is added with a 0-weight edge from s to each

vertex v V . If Bellman-Ford detects a negative-weight cycle in this graph, then

the constraints are unsatisable. We thus focus on the case in which there are no

negative-weight cycles. The proof takes the form of a lemma and a theorem.

Lemma 1 When Bellman-Ford is run on the constraint graph, max{xi } = 0.

Proof. Let p = s, v1 , . . . , vk be the shortest path from vertex s to vertex vk as re

ported by Bellman-Ford when run over the constraint graph. By the optimal substruc

ture of shortest paths, s, v1 must be the shortest path from s to v1 . By construction,

the edge from s to v1 has a weight of 0. Thus x1 = (s, v1 ) = w(s, v1) = 0. In

combination with the constraint xi 0 for all xi , this implies that maxi xi = 0.

Theorem 2 When Bellman-Ford is run on the constraint graph, it minimizes the quan

tity (max{xi } min{xi })

Proof. Since max{xi } = 0 (by Lemma 1), it sufces to show that Bellman-Ford

maximizes min{xi }. Let xk = min{xi } in the solution produced by Bellman-Ford,

and consider the shortest path p = v0 , v1 , . . . , vk from s = v0 to vk . The weight of

k1

k1

k1

C[i, i + 1] = i=1

C[i, i + 1].

wi(i+1) = w(v0, v1 ) + i=1

this path is w(p) = i=0

The path corresponds to the following set of constraints:

x1 x0

x2 x1

x3 x2

. . .

xk xk1

0

C[1, 2]

C[2, 3]

C[k 1, k]

xk

k1

C[i, i + 1]

i=1

= w(p)

That is, in any solution that satises the constraints, xk cannot be greater than w(p),

the weight of the shortest path from s to vk . As Bellman-Ford sets xk to the shortest

path value, this implies that xk is as large as possible.

(c) Give an efcient algorithm for minimizing the overall duration of the construction

schedule. That is, given A[1 . . n, 1 . . n] and B[1 . . n, 1 . . n], choose {x1 , x2 , . . . , xn }

for all xi . Assume that an unlimited number of jobs can be performed in parallel.

Solution:

The algorithm is as follows:

1. Construct a constraint graph from the constraints in Equation (3). However, if

C[i, j] = , then do not add the edge (vj , vi ) to the graph.

2. Run Bellman-Ford on the constraint graph to obtain a solution {x1 , x2 , . . . , xn }.

3. Set y = min{xi } and calculate a new solution xi = xi y for all xi . Output

{x1 , x2 , . . . , xn }.

This algorithm differs from Part (b) in two ways. First, to incorporate the constraint

xi 0, we simply shift the solution xi (in Step 3) so that all values are non-negative.

This linear shift will not affect the feasibility of the difference constraints, as the

difference between each pair of variables remains unchanged. Also, min{xi } =

min{xi } y = 0, which (in combination with Theorem 2) implies that the algorithm

minimizes max{xi } subject to the constraints.

The second difference between the algorithm and Part (b) is in the construction of the

constraint graph. In order to improve the runtime, we omit edges from the constraint

graph that correspond to an innite weight C[i, j]. Because these edge weights are

innite, they will not impact the shortest path found by Bellman-Ford.

The running time is (V 2 ) for Step 1, (V E) for Step 2, and (V ) for Step 3; the

overall runtime is (V 2 + V E). In terms of the scheduling problem, the running time

is (n2 + nk), where k represents the number of (non-innite) constraints between

jobs. If each job has at least one constraint, then the runtime is (nk). Though

k = (n2 ) in the worst case, we would expect the number of constraints to be sparse

in practice. Thus, the algorithm above is a signicant improvement over the (n3 )

algorithm that would result from a naive construction of the constraint graph.

(d) If the constraints are infeasible, wed like to supply the user with information to help in

diagnosing the problem. Extend your algorithm from (c) so that, if the constraints are

infeasible, your algorithm prints out a set S of conicting constraints that is minimal

that is, if any constraint is dropped from S, the remaining constraints in S would be

feasible.

Solution:

A simple algorithm is to run depth-rst search on the constraint graph, starting from

vertex s. Upon encountering a back edge, construct a cycle in the graph by tracing

back through the predecessor matrix to the target of the back edge. Then print out the

constraint corresponding to each edge in the cycle; these constraints form a set S as

specied.

A set of linear difference constraints are conicting if and only if they form a cycle

in the constraint graph. Thus, identifying a single cycle in the constraint graph will

indicate a set of conicting constraints S. This set is minimal in that, with the removal

of any edge, the cycle is broken and the remaining constraints in S are feasible.

The running time is (n + k), where k is the number of (non-innite) constraints

between jobs. This runtime follows directly from the (V + E) runtime of depth-rst

search on the constraint graph.

Alternate Solution:

It is also possible to utilize the predecessor matrix that is constructed as part of the

Bellman-Ford algorithm. If the last pass of Bellman-Ford is modied to update the

predecessor matrix for the vertices under consideration, then the infeasible constraints

will be manifested as cycles in the predecessor graph. A cycle can be detected in the

predecessor graph using depth rst search. In this case, the runtime for the search is

(V ) = (n), since each vertex has at most one incoming edge (E = (V )).

While this is arguably a more elegant solution, the asymptotic runtime is no better than

the rst solution when considered as an extension of Bellman-Ford. Bellman-Ford is

(nk), which dominates the overall runtime.

Note:

It is not a correct solution to simply trace back in the predecessor graph from the rst

edge that fails the test in Bellman-Ford. If there are multiple cycles in the graph, the

rst edge found might not be part of a cycle in the predecessor graph.

Problem 6-3. Honeymoon Hiking

Alice and Bob (after years of communicating in private) decide to get married and go hiking

for their honeymoon. They obtain a map, which they naturally regard as an undirected graph

G = (V, E); vertices represent locations and edges represent trails. Some of the locations are bus

stops, denoted by S V .

1. It begins and ends at different bus stops.

2. All of the uphill segments come at the beginning (and all of the downhill segments come at

the end).1

Let w(e) + denote the length of a trail e E, and let h(v) denote the elevation (height)

of a location v V . You may assume that no two locations have exactly the same elevation.

(a) Give an efcient algorithm to nd the shortest romantic hike for Alice and Bob (if any

Solution:

The algorithm is as follows:

1. Construct a directed acyclic graph GU = (VU , EU ) that represents the uphill trail

segments. The vertices in this graph are the same as the original: VU = V . For

each undirected edge in G, there is an edge in GU that is directed uphill:

EU = {(u, v) : (u, v) E and h(u) < h(v)}

Because the elevation of each vertex is distinct, one direction of each trail segment

must be strictly uphill. This implies that GU is acyclic, as it is impossible to form

a loop while hiking strictly uphill.

2. For each bus stop s S, compute the shortest uphill path to all other vertices

v V . As GU is acyclic, this can be done by running DAG -S HORTEST-PATHS

from each s S over the graph GU .

3. For each vertex v VU , compute the two closest bus stops s1 [v] and s2 [v] from

which one can reach v by hiking strictly uphill. This can be done by taking the

two minima, over all vertices s S, of the shortest uphill path from s to v (as

computed in (2)). (Note that if v is a bus stop, then one of these paths has zero

lengththat is, if v S then s1 [v] = v or s2 [v] = v.)

4. Observe that for every vertex v, there is a romantic hike s1 [v] v s2 [v].

Further, this is the shortest romantic hike that has v as its high point (i.e., where

the hike transitions from uphill to downhill). Thus, the shortest romantic hike

overall can be found by taking the minimum over all high points v V . Using

l[v] to denote the length of the path s1 [v] v s2 [v] in G, the shortest hike is:

Shortest = min l[v]

vV

After all, what is more frustrating than going uphill after you have started going downhill?

10

The actual path of the shortest romantic hike can be recovered by keeping track of the

vertex v that is selected as the minimum.

Correctness relies on the observation that every romantic hike will have a high point,

where the hike turns from uphill to downhill (we assume that all elevations are distinct,

so every trail is either strictly uphill or strictly downhill). The shortest romantic hike

for a given high point will correspond to the two closest bus stops for which there

are uphill paths from the bus stops to the high point. The hike itself will start at one

of these stops, go uphill to the high point, and then downhill to the second stop. We

compute the shortest overall romantic hike as the minimum over all possible high

points.

Step 1 runs in (V + E) time, as all edges and vertices must be copied over into

the new graph. Step 2 runs in (S(V + E)) time, as DAG -S HORTEST-PATHS takes

(V + E) time to run on each vertex s S. Step 3 runs in (SV ) time, as it selects

the smallest and second-smallest distances out of S bus stops, for each of V vertices.

Step 4 runs in (V ) time, as it selects the minimum length over V points.

The overall runtime is thus (SV + SE), as Step 2 dominates.

(b) Give an efcient algorithm to nd the longest romantic hike for Alice and Bob (if any

Solution:

The same algorithm as in Part (a) applies, with minor modications to compute the

longest path instead of the shortest:

1. Each min operation from Part (a) is replaced with max.

2. DAG -S HORTEST-PATHS is modied to compute longest paths instead of shortest

paths. This involves modifying I NITIALIZE -S INGLE -S OURCE to invert the roles

of 0 and and modifying R ELAX to use max instead of min.

Correctness follows the same argument as in Part (a). The running time is identical.

Note that while this problem is very similar to (a), it does introduce the possibility of

traversing an edge twice in the undirected graph. Some other approaches to (a) break

down given this possibility.

Alternate Solution:

Use the same algorithm as in Part (a), but invert all of the edge weights on the original

graph. That is, use edge weights w (e) = w(e) for all edges e E.

As the algorithm in Part (a) breaks the graph into DAGs, there cannot be any negativeweight cycles (there are no cycles at all). Thus, DAG -S HORTEST-PATHS still gives the

correct answer. Steps 3 and 4 remain unchanged in selecting the shortest composition

of paths from GD and GU .

The runtime is identical to Part (a).

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

December 1, 2004

6.046J/18.410J

Handout 31

Problem 8-1. Inspirational res

To foster a spirit of community and cut down on the cliqueishness of various houses, MIT has

decided to sponsor community-building activities to bring together residents of different living

groups. Specically, they have started to sponsor ofcial gatherings in which they will light copies

of CLRS on re.

Let G be the set of living groups at MIT, and for each g G, let residents(g) denote the number of

residents of living group g. President Hockeld has asked you to help her out with the beginning

of her administration. She gives you a list of book-burning parties P that are scheduled for Friday

night. For each party p P , you are given the number size(p) of people who can t into the site

of party p.

The administrations goal is to issue party invitations to students so that no two students from the

same living group receive invitations to the same book-burning party. Formally, they want to send

invitations to as many students as possible while satisfying the following constraints:

for all g G, no two residents of g are invited to the same party;

(a) Formulate this problem as a linear-programming problem, much as we did for shortest

paths. Any legal set of invitations should correspond to a feasible setting of the vari

ables for your LP, and any feasible integer setting of the variables in your LP should

correspond to a legal set of invitations. What objective function maximizes the number

of students invited?

sent to residents of group g G.

max

s.t.

pP,gG

xp,g

p P

x

gG p,g

g G

pP xp,g

g G, p P

xp,g

g G, p P

xp,g

size(p)

residents(g)

1

0

If we have a legal set of invitations, then it is easy to see that all of these constraints are

satised, since the set of invitations must satisfy the stated conditions. Similarly, any

feasible integral setting of the variables yields xp,g {0, 1}, and it is straightforward

to verify that sending an invitation to party P to some resident of group g if and only

if xp,g = 1 will satisfy the requirements. (We do not permit the administration to send

more than one invitation to a party p to the same student; thus we can send only one

invitation to party p to group g.) Because the objective function p,g xp,g measures

the number of invitations sent (and thus the number of students invited), an optimal

setting of the variables for the LP therefore corresponds to a maximum number of

invited students.

(b) Show how this problem can be solved using a maximum-ow algorithm. Your algo

rithm should return a set of legal invitations, if one exists, and return FAIL if none

exists.

Solution: Dene the graph G = (V, E), and capacities c on the edges, where

V = {s, t} G P , where s and t are brand new source and sink nodes, respec

tively.

(g, p), (s, g), (p, t) E for every g G, p P .

c(g, p) = 1, c(s, g) = residents(g), and c(p, t) = size(p).

the graph. (There was a slight ambiguity in the phrasing of the question here: if you

interpret if one exists to mean if there exists any legal set of invitations, then you

should never fail; if you interpret it to mean if there exists a legal set of invitations

such that every student gets an invitation, then you should return fail if the value

of f is less than gG residents(g).) For every party p, send an invitation to party p

to an uninvited member of group g if and only if f (g, p) = 1.

The constraints on the graph guarantee that no two residents of the same house can get

an invitation to the same party and that no party has more invitations sent to it than its

size. Thus if the algorithm returns a set of invitations, then they are valid. Conversely,

any legal set of i invitations can be expressed as a ow of value i.

(c) (Optional.) Can this problem can be solved more efciently than with a maximum-

ow algorithm?

Solution: It seems like some version of a greedy algorithm can be used to solve

this problem more efciently, but after an hour-long course-staff meeting, we werent

convinced either way.

Problem 8-2. Zippity-doo-dah day

On Interstate 93 south of Boston, an ingenious device for controlling trafc has been installed.

A lane of trafc can be switched so that during morning rush hour, trafc ows northward to

Boston, and during evening rush hour, if ows southward away from Boston. The clever engi

neering behind this design is that the reversible lane is surrounded by movable barriers that can be

For some reason, gazillions of people have decided to drive from Gillette Stadium in Foxboro,

MA to Fenway Park. (They seem to be cursing a lot, or, at the very least, you hear them shouting

the word curse over and over.) Governor Mitt asks you for assistance in making use of the

zipper-lane technology to increase the ow of trafc from Foxboro to Fenway.

We can model this road network as directed graph G = (V, E) with source s (Foxboro), sink t

(Fenway), and integer capacities c : E + on the edges. You are given a maximum ow f in

the graph G representing the rate at which trafc can move between these two locations. In this

question, you will explore how to increase the maximum ow using zippered edges in the graph.

Let (u, v) E be a particular edge in G such that f (u, v) > 0 and c(v, u) 1. That is, there is

positive ow on this edge already, and there is positive capacity in the reverse direction. Suppose

that zipper technology increases the capacity of the edge (u, v) by 1 while decreasing the capacity

of its transpose edge (v, u) by 1. That is, the zipper moves 1 unit of capacity from (v, u) to (u, v).

(a) Give an O(V + E)-time algorithm to update the maximum ow in the modied graph.

Solution: The algorithm is simple: increment the capacity of (u, v), and search for a

single augmenting path in the residual graph in O(E + V ) time. Augment along that

path if one is found. Decrement the capacity of (v, u), and return the resulting ow.

If there is a minimum cut in G that (u, v) does not cross, then the same ow f is a

maximum ow in the modied graph: it is feasible because no capacities have de

creased, and it is maximum because the same unmodied minimum cut still gives an

upper bound on the size of the ow. If (u, v) does cross all minimum cuts, then the

maximum ow will increase by 1. Thus, the new maximum ow has value either |f |

or |f |+1, and a single augmenting path will sufce to nd the updated maximum ow

(since the residual capacities are all integral).

Finally, we observe that decrementing the capacity of (v, u) does not affect the max

imum ow: because there is positive ow on (u, v) in f , there is net negative ow

on (v, u), and the soon-to-be-deleted unit of capacity on (v, u) is unused. Eliminating

any capacity unused by the max ow cannot make the ow infeasible, and it therefore

must remain a maximum ow in the modied graph.

Zap 86 years into the future! Zipper lanes are commonplace on many more roads in the Boston

area, allowing one lane of trafc to be moved from one direction to the other. You are once again

given the directed graph G = (V, E) and integer capacities c : E + on the edges. You

also have a zipper function z : E {0, 1} that tells whether an additional unit of capacity can be

moved from (v, u) to (u, v). For each (u, v) E, if z(u, v) = 1, then you may now choose to move

1 unit of capacity from the transpose edge (v, u) to (u, v). (You may assume that if z(u, v) = 1,

then the edge (v, u) exists and has capacity c(v, u) 1. Again, you are given a source node s V ,

a sink node t V , and a maximum ow f . Governor Mitt IV asks you to congure all the zippered

lanes so that the maximum ow from s to t in the congured graph is maximized.

(b) Describe an algorithm that employs a maximum-ow computation to determine the

following:

1. the maximum amount that the ow can be increased in this graph after your cho

sen zippered lanes are opened; and

2. a conguration of zippered lanes that allows this ow to be achieved.

Solution: One can solve this problem by extending the denition of an augmenting

path to include the possibility of zipping an edge to increase the capacity of a min cut,

but theres an easier way.

Note that in a net ow, there is never positive ow in both directions (u, v) and (v, u).

The idea of the algorithm is this: since we can compute a maximum ow that only

uses an edge in one direction, well add the zippered capacity in both directions, and

compute max ow. The above note guarantees that the resulting ow satises the

zippered capacity constraintsi.e., does not use the extra zipped capacity along both

(u, v) and (v, u).

Given G = (V, E), nodes s, t, capacity function c, and zipper function z:

1. For every pair {u, v} E, set c (u, v) := z(u, v) + c(u, v).

2. Run the Edmonds/Karp max ow algorithm on G with capacities c to get a max

ow f .

3. For every pair (u, v) with z(u, v) = 1, if f (u, v) > 0, then set the direction of the

zippered lane on (u, v) so that the unit of capacity is moved from the transpose

edge (v, u) to the edge (u, v). If f (u, v) = f (v, u) = 0, set the direction of the

zipper lane for (u, v) arbitrarily.

4. Return f and the direction settings above.

The running time of this algorithm is then O(V E 2 ).

To prove correctness, well show that (i) the value of f is at least the max ow f for

the best zippered setting, and (ii) the value of f is at most f . For condition (i), observe

the following: the capacity of each edge in the best zippered setting is upper bounded

by the capacities c , which immediately implies that the value of f is at most f . For

condition (ii), note that f is a feasible ow in some zippered graph (in particular, in

the graph with edges oriented as per the returned directions).

Because the graph G is actually a network of roads, it is nearly planar, and thus |E| = O(V ).

(c) Give an algorithm that runs in time O(V 2 ) to solve the graph conguration problem

under the assumption that |E| = O(V ). You should assume that the original ow f

has already been computed and you are simply determining how best to increase the

ow.

Solution: Run the algorithm from part (a) on each edge in E, in an arbitrary order.

This takes O(E 2 ) = O(V 2 ) time under the stated assumptions.

Massachusetts Institute of Technology

Professors Piotr Indyk and Charles E. Leiserson

December 8, 2004

6.046J/18.410J

Handout 34

Reading: Chapters 32.132.2, 30.130.2, 34.134.2, 35.1

Both exercises and problems should be solved, but only the problems should be turned in.

Exercises are intended to help you master the course material. Even though you should not turn in

the exercise solutions, you are responsible for material covered in the exercises.

Mark the top of each sheet with your name, the course number, the problem number, your

recitation section, the date and the names of any students with whom you collaborated.

Three-hole punch your paper on submissions.

You will often be called upon to give an algorithm to solve a certain problem. Your write-up

should take the form of a short essay. A topic paragraph should summarize the problem you are

solving and what your results are. The body of the essay should provide the following:

1. A description of the algorithm in English and, if helpful, pseudocode.

2. At least one worked example or diagram to show more precisely how your algorithm works.

3. A proof (or indication) of the correctness of the algorithm.

4. An analysis of the running time of the algorithm.

Remember, your goal is to communicate. Full credit will be given only to correct algorithms

that are which are described clearly. Convoluted and obtuse descriptions will receive low marks.

Exercise 9-1. On-line String Matching

Recall that in an on-line algorithm, the input is generated as the algorithm is running. The idea

is to solve the problem efciently before seeing all the input. You cant scan forward to look at

future input, but you can store all input seen so far, or some computation on it.

(a) In this setting, the text T [1 . . . n] is being broadcast on the network, one letter at a

time, in the order T [1], T [2], . . .. You are interested in checking if the text seen so far

contains a pattern P , where P has length m. Every time you see the next letter of the

Design an algorithm that solves this problem efciently. Your algorithm should use

no more than (m) time on preprocessing P . In addition it should do only constant

amount of work per letter received. Your algorithm can be randomized, with constant

probability of correctness.

Solution:

For both parts we are going to use Karp-Rabin (KR) algorithm seen in the class (the

nite state machine / Knuth-Morris-Pratt algorithms could be used as well). As in the

class, we will assume that the alphabet is {0, 1} (the algorithm can be easily modied

to handle arbitrary alphabet). Recall that KR used log O(1) n time to nd a random

prime q, and O(m) time to hash the pattern P ; we will refer to these two steps as

preprocessing.

After the preprocessing, the KR algorithm computes the hash values for the m-length

substrings of T in an incremental way. Whenever a new symbol T [i] is given, KR

computes the hash value of the substring T [i m + 1 . . . i] in constant time. Thus, KR

works in our setting without any modications.

(b) Now say that you have the same pattern P , but the text T [1 . . . n] is being broadcast in

reverse. That is, in the order T [n], T [n 1], . . . Modify your algorithm so that it still

Solution:

For an array A[1 . . . n], let AR be the reverse of A, i.e., AR [i] = A[n i + 1].

Searching for an occurrence of P in T is equivalent to searching for an occurrence of

P R in T R ; thus, we can focus on the latter task. Since the symbols of T are given

in the reverse order, it means that the symbols of T R are given in the proper order

(i.e., T R [1], T R [2] . . .). Thus, we can nd the occurrences of P R in T R by using the

algorithm from the part (a).

Exercise 9-2. Some Summations

Assume you are given two sets A, B {0 . . . m}. Your goal is to compute the set C = {x + y :

x A, y B}. Note that the set of values in C could be in the range 0 . . . 2m.

Your solution should run in time O(m log m) (the sizes of |A| and |B| do not matter).

Example:

A = {1, 4}

B = {1, 3}

C = {2, 4, 5, 7}

Solution:

The key realization is that when two polynomials multiply, their exponents are added in every

possible pairing. So we take our set A, and do the following: Turn the set into a polynomial

of maximum degree m. The coefcient of x0 is the number of times 0 appears in the set. The

coefcient of x1 is the number of times 1 appears in the set, and on. Do the same with B. The

Example:

A = x 4 + x1

B = x 3 + x1

C = x 7 + x 5 + x 4 + x2

C = A B. Recall that the coefcient of xi in C is given by:

ci =

aj bij

j =0

Note that in this case, all terms of the sum are non-negative. We can compute C in O(m log m)

For a simple argument of correctness, we observe that every coefcient in the target polynomial

(ci xi ) represents the number of ways we can add a number from A and from B to get to i. First, we

note that all coefcients in A and B are either zero or one (no multisets were indicated). Clearly

if there is no a A, b B that sum to i, then for all j {0..i}, at least one of a j , bij is zero.

Similarly, if ci = z, then there are z pairs aj , bij since each such pair will multiply out to xi , and

then the sum of all of those pairs will yield z xi .

The running time of this algorithm is the time to convert A and B into polynomials. If A is a set (not

a multiset) {0..m} then this can be done in O(m). Same is true for B. To multiply them takes

O(m log m). Finally, to take the resulting polynomial and output the set C takes O(2m) = O(m).

As an aside, we also note that if we allowed multisets, everything still works. If there are two ones

in one set, and two threes in the other, there are indeed four different ways to get a target value of

3.

Exercise 9-3. Do Problem 35-5, on page 1051 of CLRS.

Solution:

(a) In the best case, each job has its own machine. Since a job cannot be broken down into smaller

pieces, the makespan cannot be shorter than the longest job. That is, C max

max pk .

1kn

(b) In the best case, the jobs are evenly distributed over the machines, such that each machine n

ishes at exactly the same time, Ceven . (Starting from such a distribution, any re-allocation

would clearly increase the makespan, as the maximum nish time would increase.) The

makespan of an even distribution

is the sum of the job lengths divided by the number of pro

pk . Since Cmax

Ceven , the claim follows.

1kn

(c) For each machine Mi , we maintain a set ai of the jobs that are allocated to Mi , as well as a

eld fi indicating the time at which Mi will become idle (under the current allocation of jobs).

The G REEDY-S CHEDULE algorithm assigns each job to the machine that will become idle rst.

The machines are organized in a priority queue (implemented as a M IN -H EAP) using the next

nish time fi as the key for machine Mi .

G REEDY-S CHEDULE(J1 . . . Jn , p1 . . . pn , M1 . . . Mm )

1 Q empty M IN -H EAP

2 for i 1 to m

3

fi 0

// initialize nish time

4

ai {} // initialize job allocation

5

I NSERT(Q, Mi , ai , fi ), using fi as key

6 for j 1 to n

7

do Mk , ak , fk E XTRACT-M IN(Q)

8

ak ak Jj

9

fk fk + pj

10

I NSERT(Q, Mk , ak , fk )

11 return {a1 , a2 , . . . , am }

This algorithm directly implements the greedy strategy. It iterates over the jobs, assigning

each job to the machine that will be idle next. Each call to I NSERT and E XTRACT-M IN takes

O(lg m) time, as there are m machines in the heap. These calls are enclosed in two loops; the

rst executes m times and the second executes n times. Assuming n m (as otherwise the

problem is trivial), the overall runtime is O(n lg m).

(d) Let Jj denote the job that has the largest completion time as scheduled by the greedy algorithm

denote the value of fk on iteration j of the job allocation loop

(line 6 of the algorithm). Then Cj can be written as follows:

Cj = Cmax =

min fjk

+ pj

k[1,m]

min fmk

+ pj

k[1,m]

1/m

pk + p j

1kn

pk + max pk

1kn

1kn

Cmax

+ Cmax

2 Cmax

1/m

The rst line is a direct consequence of the greedy algorithm: on iteration j, the algorithm

selects the minimum nish time and schedules job Jj to execute next on the corresponding

machine. The second line follows because the nish times are monotonically increasing with

successive iterations of the j loop. By the same argument as in part (b), the minimum nish

time is maximized when all nish times are the same; thus, the average load is used as an

upper bound in the third line. The fourth line recognizes that p j is at most max pk . Finally,

1kk

we substitute the results from (a) and (b) to conclude that C j 2Cmax

, thereby showing that

the greedy algorithm is a 2-approximation algorithm.

- Openvdb IntroductionUploaded byRafael Campos
- Visi RuleUploaded byBruce Herrera
- alg_notesUploaded byAnonymous GNWnQac
- Vyom-Freshers-Guide-2007Uploaded byapi-3777212
- Darwin SYSUploaded byMohit Bagadia
- Encouraging Viable Client System through Website Structure ChangeUploaded byIJCERT PUBLICATIONS
- Integrating With Oracle HCM CloudUploaded bysjawad
- S2.DataStructuresandSQLUploaded bysargentshriver
- A General Algorithm for Computing Distance Transforms in Linear TimeUploaded byNoisyperson
- Markov Models in Medical Decision Making - A Practical Guide -Med Decis Making-1993-Sonnenberg-322-38Uploaded byAmgad Alsisi
- Lsm vs FractalUploaded bysanthuj
- Data StructureUploaded bySabrina Zotti
- PS User Security SetupUploaded byabhi10aug
- L9 - Analysis of AlgorithmsUploaded bymurad
- ee220s02lec6 (1)Uploaded byanony_1
- Solved Question Papers 2009Uploaded byGovind Rao
- Sorting.pdfUploaded byAshutosh Kumar
- bst.pptUploaded bySheikh Adnan Ahmed Usmani
- avl1Uploaded byLalit Jain
- CVXVUploaded bypakizeatam
- Height, Depth and Level of a TreeUploaded byTamilselvi
- data structuresUploaded byMohammed Jeelan
- Scimakelatex.14826.PopeyUploaded bymaxxflyy
- Tree TraversalUploaded byGunner_MH1
- cachemisaofaUploaded by蘇意喬
- CSG(week 8)Uploaded byDineshNewalkar
- AnalysisUploaded byNagarajuRaginaboyina
- NSF Unified Tree-Of-Life ProposalUploaded byAlex
- Algo Balanced TreesUploaded bykalyan.b.aninda5312
- Project_B_The Iterative Closest Point AlgorithmUploaded byTnQuoc Bao

- OTT-CP-27032015Uploaded bySaurabh Ghosh
- Optimal Currency AreaUploaded byAkshay Mehta
- Chottu KoolUploaded byAkshay Mehta
- MTexternal.pdfUploaded byRashmi Jamadagni
- 6 Triangle RasterizationUploaded byAkshay Mehta
- ch11Uploaded byfoxhoundzz13
- Opportunities for Export of Fruits and Vegetable Based Products (Summary)Uploaded byAkshay Mehta
- Processed Fruits and VegetablesUploaded byAnkit Singh
- NCMAT SampleUploaded byNikesh Mirchandani
- CookiesUploaded byrenkinjutsushi1
- CMDM-PractQns_EUploaded byashwani1883
- 5 3d TransformationsUploaded byAkshay Mehta
- 2 Graphics DevicesUploaded byAkshay Mehta
- 9 Camera ModelsUploaded byAkshay Mehta
- 7 ClippingUploaded byAkshay Mehta
- 4 OpenGLUploaded byAkshay Mehta
- CS580-02AnewUploaded byAkshay Mehta
- Applebaum, 2004Uploaded byAkshay Mehta
- A Practical Approach to Solve SMC ProblemUploaded byAkshay Mehta
- Virtual mUploaded byAkshay Mehta
- ch10Uploaded byAkshay Mehta
- 20150307 Reeby Sports Working v003Uploaded byAmit Srivastava
- PGP1 Grp11 CA TechnologiesUploaded byAkshay Mehta
- Knapsack ProblemUploaded byAkshay Mehta
- Horner SchemeUploaded byAkshay Mehta
- CLR ExplainedUploaded byAkshay Mehta
- m01L01Uploaded byKushal Verma
- KM SharingUploaded byAkshay Mehta

- workflowsim(1)Uploaded bySanthosh B Acharya
- Automatic SQL Tuning in Oracle Database 10g and 11gUploaded byjiyangwu
- Threads TheoryUploaded bySudheera Palihakkara
- Parallel Computing With ElmerUploaded bypilafa
- Ashish SoftwareUploaded byKumar Rajput
- oos qbankUploaded byVinod Nayyar
- Introducing_Blue_Prism.pdfUploaded byRon
- OpenGL Basic FunctionsUploaded byAishwaryaNawkar
- SynapseIndia Feedback on Programming and Software DevelopmentUploaded bysynapseindia
- Oop Lab ManualUploaded bygopitheprince
- ABAPPROXY2Uploaded byRenato Vicentin Ferreira
- What is New in ABAP Testing and Troubleshooting With SAP NetWeaver 7Uploaded byashlafir
- Log4j_PPTUploaded byMukesh Kumar
- vbfilepart1Uploaded bydangwalpr
- c Make ListsUploaded byfghjk
- chapter...1....2Uploaded bytbijle
- Puzzles and Intel QuestionsUploaded byapi-3713843
- AdvancedOOPUploaded byDimitris Vafiadis
- Speed Mark Advanced Scripting Reference en 3 4Uploaded bytheflamebearer
- django-1.4-cheatsheet.pdfUploaded byOscar Martínez López
- Computing Revision ChecklistUploaded byRottenMess
- Simple FAT y SD Tutorial Parte 1-InglesUploaded byAndyGarcia
- Matrix LogUploaded bym
- 7.docxUploaded byihavedreams2017
- Array in MacrosUploaded byGerman Galdamez
- Column GenerationUploaded bybscjjw
- TMS320C64XXUploaded byaldo_suvi
- Selenium Framework Usage GuidelinesUploaded byraj_esh_0201
- Computer Science Viva-Voce QuestionUploaded bymubark_ali
- MATLAB Programming & Its Applications for Electrical EngineersUploaded byRohan Sharma