nalysis of

lgorithms

Tomado del libro de Cormen

Depto Sistemas Universidad Nacional de Colombia
Sede Bogotá
 D.P. Characteristics

• D. P. like the divide and conquer method, solves

problems by combining the solutions to
subproblems.
• D. P. is applicable when the subproblems are not
independent, that is, when subproblems share
subsubproblems.
• A dynamic-programming algorithm solves every
subsubproblem just once and then saves answer in a
table, thereby avoiding the work of recomputing the
answer every time the subproblem is encountered.
• D. P. is typically applied to optimization problems.
 Dynamic-programming Algorithm

Steps:
1. Characterize the structure of an
optimal solution.
2. Recursively define the value of an
optimal solution.
3. Compute the value of an optimal
solution in a bottom-up fashion.
4. Construct an optimal solution from
computed information.
 Assembly-line Sheduling

A company produce automobiles in a factory that

has two assembly lines. Each assembly line has n
stations, numbered j = 1,2,...n. We denote the jth
station on line i = 1 or 2 by Si,j. The jth station on
line 1 performs the same function as the jth station
on line 2.
We denote the assembly time required at station Si,j
by ai,j.
The time to transfer a chassis away from assembly
line i after having gone through station Si,j is ti,j.
There is also an entry time ei for the chassis to
enter assembly line i and an exit time xi for the
completed auto to exit assembly line i.
 A manufacturing problem to find
the fastest way through a factory.
 What is the matter?

The problem is to determine which

stations to choose from line 1 and
which to choose from line 2 in order
to minimize the total time through the
factory for one auto.
 Solution...?

The obvious, “brute force” way of minimizing the

time through the factory is infeasible when there
are many stations.
If we know the solutions for both lines this time is
easy to compute in (n) time.
Unfortunately, there are 2n possible ways to
choose stations, therefore, for enumerating all
possible ways and computing how long each
takes would require (2n) time.
The structure of the fastest way through the factory

IF (Si,j fastest way through the factory) AND

(fastest way to Si,j is through Si-1,j-1)
THEN Si-1,j-1 fastest way through the factory

Therefore, we can say that for assembly-line sheduling,

an optimal solution to a problem contains within it an
optimal solution to subproblems. We refer to this
property as optimal substructure.
To solve the problem of finding the fastest way through
the station j of either line, we solve the subproblems of
finding the fastest ways through station j-1 on both
lines.
 En resumen
 f1[ j] = min( f1 [ j–1] + a1,j , f2[ j–1] + t2,j-1 +
a1,j)
 f2[ j] = min( f2 [ j–1] + a2,j , f1 [ j–1] + t1,j-1 +
a2,j)

 f1[1] = e1 + a1,1
 f1[ j] = min( f1 [ j–1] + a1,j , f2[ j–1] + t2,j-1 +
a1,j)

 f2 [1] = e2 + a2,1
 f2[ j] = min( f2 [ j–1] + a2,j , f1 [ j–1] + t1,j-1 +
a2,j)
Then: f* = min ( f1[n]+x1, f2[n]+x2 ).
 A recursive solution II.

The fi[j] values give the values of optimal

solutions to subproblems. To help us keep
track of how to construct an optimal
solution, let us define li[j] to be the line
number, 1 or 2, whose station j-1 is used in a
fastest way through station Si,j.

We also define l* to be the line whose station

n is used in a fastest way through the entire
factory.
 Computing the fastest time.

At this point, it would be a simple matter to write

a recursive algorithm to compute the fastest way
through the factory. There is a problem with
such a recursive algorithm: It is running time is
exponential in n, it is to say the total number of
references to all fi [ j ] values is (2n) .

To see why, let ri(j) be the number of references

made to fi [j] in a recursive algorithm. We have:

r1(n) = r2(n) = 1 and

r1( j ) = r2( j ) = r1( j+1 ) + r2( j+1 )
However, if we compute and save the fi[j] values,
we can obtain an algorithm that to run in (n).
Constructing the fastest way through the factory

Having computed the values of fi [ j ], f*, li [ j ] and l*,

we need construct the sequence of stations used in
the fastest way through the factory, we can use
below algorithm.
In DP we build an optimal solution to the problem from optimal solutions to subproblems.

Is some similar to the paradigm for Sorting Algorithms, to do problems m subproblems

that are includes in the original problem and are whole it.

The idea is to get sub problems that express the main problem space simple as possible
but without left some details.

When this process has end we must solve them for build the solution.
Optimal substructure varies across problem domains in two ways:
• how many subproblems are used in an optimal solution to the original problem
• how many choices we have in determining which subproblem(s) to use in an optimal
solution
The running time of a dynamic programming algorithm depends on the product of two
factors:

• The number of subproblems overall

• The choices we look at for each subproblem

DP uses optimal substructure in a bottom up fashion.

We first find optimal solutions to subproblems and, having solved these we find an
optimal solution to the problem.

That requires make a choice among subproblems as to which we’ll use in solving the
problem.

Its cost is usually the subproblem cost plus a cost that is directly attributable to the choice
itself.
 Overlapping sub problems

The second ingredient that an optimization

problem must have for dynamic programming to
be applicable is that the space of subproblems
must be small.

A recursive algorithm solves the same sub

problems over and over, rather than always
generating new subproblems.
When a recursive algorithm revisits the same
problem has overlapping subproblems.

DP algorithms take advantage of this by solving

each sub problem once and then storing the solution
in a table where it can be looked up when needed,
using constant time per lookup.

• Two subproblems of the same problem are

independent if they do not share resources
• Two sub problems are overlapping if they are
really the same sub problem that occurs as a
subproblem of different problems
 Figure 1

Recursive Matrix Chain Matrix Chain Order Memoized

Now Matrix Chain Order repeatedly looks up the
solution to subproblems in lower rows when solving
subproblems in higher rows.

In the figure 2 we can see the recursion tree for

Recursive Matrix Chain (p, 1, 4).

In fact, the time to compute it by this recursive

procedure is at least exponential in n.

Recursive Matrix Chain Figure 2

 Figure 1 Figure 2

Recursive Matrix Chain Matrix Chain Order Memoized

Matrix Chain to compute an optimal parenthesization of a chain of n matrices.
If we assume that the execution of lines 1-2 and 6-7 takes at least unit time,
then yields the recurrence:

T (1)  1
n 1
for n > 1
T ( n)  1   (T ( k )  T ( m  k )  1)
i 1
n 1
T ( n)  2 T (i )  n
i 1

As T ( n)  ( 2 ) using the substitution method.

n

We have as a result:

n 1 n 1
T ( n )  2 2 i 1
 n  2 2i  n  2( 2 n 1  1)  n  2 n 1
i 1 i 1
The work is at least and exponential of n.

Compare this top down with the bottom up, DP. The latter is more efficient
because it takes advantage of the overlapping-subproblems property.

There are only (n2) different subproblems, and DP solves each exactly once.
Reconstructing an optimal solution

As a practical manner, we often store which

choice we made in each subproblem in a table so
that we do not have to reconstruct this
information from the costs that we stored.

From the results we always can rebuild a solution

for a main problem.
 Longest common
Subsequence LCS
In Biological applications, we often want to
compare the DNA of two (or more)
different organisms.

A strand of DNA consists of a string of

molecules called bases , where the
possible bases are adenine, guanine,
cytosine, and thymine.

A strand of DNA can be expressed as a

string over the finite set {A,C,G,T}.
For Example:

The DNA of one organism may be,

S1= ACCGGTCGAGTGCGCGGAAGCCGGCCGAA

While the DNA of another organism may be,

S2= GTCGTTCGGAATGCCGTTGCTCTGTAAA

One goal of comparing two strands of DNA is

to determinate how “similar” the two strands
are, as some measure of how closely related
the two organisms are.
Similarity can be and is defined in many
different ways.

For example, we can say that two DNA strands

are similar if one is a substring of the other.

In our example, neither S1 nor S2 is a

substring of the other.

Alternatively, we could say that two strands

are similar if the number of changes needed
to turn one into the other is small.
 We formalize this last notion of
similarity as the longest-common-
subsequence problem.

A subsequence of a given
sequence is just the given
sequence with zero or more
elements left out.
 FORMALLY

Given a sequence X = [x1, x2,……..xm]

Another sequence Z =[z1, z2,……..zk] is a subsequence

of X if there exists a strictly increasing sequence

[i1, i2,……..ik] of indices of X such that for all

j = 1,2,…….k, we have xij = zj.

For example: Z = [B, C, D, B] is a

Subsequence of X =[A, B, C, B, D, A, B]
with corresponding index sequence
[2,3,5,7].
In the longest-common-subsequence problem,
we are given two sequences:
X= [x1,x2,……..xm] and
Y= [y1,y2,……..yn] and wish to find a
maximum-length common subsequence
of X and Y.

Shows that the LCS problem can be solved

efficiently using dynamic programming.
 Step 1: Characterizing a longest
common subsequence

A brute-force approach to solving the LCS problem is

to enumerate all subsequences of X and check each
subsequences to see if it is also a subsequence of Y,
keeping track of the longest subsequence found. Each
subsequence of X corresponds to a subset of the
indices [1, 2, …,m] of X. There are 2m subsequences of
X, so this approach requires exponential time, making
it impractical for long sequences.
 The LCS problem has an optimal-substructure
property, however, as the following theorem shows.

As we shall see, the natural classes of sub problems

correspond to pairs of “prefixes” of the two input
sequences.

To be precise, given a sequence X=[x1, x2,…..xm],

We define the ith prefix of X, for i = 0,1,…..m, as
Xi = [x1,x2,….xi].

IfX = [A, B, C, B, D, A, B], then

X4= [A, B, C, B] and X0 is the empty Sequence
 Theorem: Optimal substructure
Of an LCS

Let X = [x1, x2,….,xm] and Y = [y1,y2,….yn] be

Sequences, and let Z = [z1,z2,….zk] be any LCS
of X and Y.

i. If xm = yn, then zk = xm= yn and zk-1 is an LCS of

xm-1 and yn-1
ii. If xm ≠ yn, then zk ≠ xm implies that Z is an
LCS of xm-1 and Y
iii. If xm ≠ yn, then zk ≠ yn implies that Z is an LCS
of X and yn-1
 Step 2: A Recursive Solution

The theorem implies that there are either one or two

sub problems to examine when finding an LCS of
X = [x1,x2,……..xm] and Y = [y1,y2,……..yn] If
xm = yn, we must find an LCS of xm-1 and yn-1
Appending xm = yn To this LCS yields an LCS of X
and Y.
If xm ≠ yn, then we must solve two subproblems :
Finding an LCS of xm-1 and Y and finding an LCS of X
and yn-1 Whichever of these two LCS’s is longer is
an LCS of X and Y.

We can readily see the overlapping-subproblems

property in the LCS problem.
To find an LCS of X and Y, we may need to find the
LCS’s of X and yn-1 and of xm-1 and Y. But each of
These sub problems has the subsubproblem of finding
The LCS of yn-1 and xm-1. Many other sub problems
Share subsubproblems.
Let us define c[i,j] to be the length of an LCS of the
sequences Xi and Yj . If either i = 0 or j= 0, one of the
sequences has length 0, so the LCS has length 0.
The optimal substructure of the LCS problem gives the
recursive formula:
0 if i = 0 or j = 0
C[i, j] = C[i-1,j-1]+1 if i,j > 0 and Xi = Yj
(EQ.1) max(C[i,j-1],C[i-1,j]) if i,j > 0 and Xi ≠ Yj

Observe that in this recursive formulation, a condition

In the problem restricts which subproblems we may
Consider. When Xi = Yj , we should consider the
Subproblem of finding the LCS of yn-1 and xm-1.
 Step 3: Computing the Length of an LCS

Based on equation (EQ.1), we could easily write an

exponential-time recursive algorithm to
compute the length of an LCS of two sequences.
Since there are only (mn) distinct
subproblems, however, we can use dynamic
programming to compute the solutions bottom
up.
Procedure LCS-LENGTH takes two sequences
X = [x1,x2,……..xm] and Y = [y1,y2,……..yn] as
inputs.
Procedure LCS-LENGTH (X,Y)
1 m  length [X]
2 n  length [Y]
3 for i  1 to m
4 do c[i, 0]  0
5 for j  0 to n
6 do c[0, j]  0
7 for i  1 to m
8 do for j  1 to n
9 do if xi = yj
10 then c[i, j]  c[i-1, j-1]+1
11 b[i, j]  “ ”
12 else if c[i-1, j] ≥ c[i, j-1]
13 then c[i, j]  c[i-1, j]
14 b[i, j]  “ ”
15 else c[i, j]  c[i, j-1]
16 b[i, j]  “ ”
17 Return c and b
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi
1 A
2 B
3 C

4 B
5 D
6 A
7 B
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0
2 B 0
3 C 0

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0
2 B 0
3 C 0

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0
2 B 0
3 C 0

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0
3 C 0

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2

4 B 0
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2

4 B 0 1 1 2 2 3 3
5 D 0
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2

4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2

4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0
 j 0 1 2 3 4 5 6
i Yj B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2

4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
The table is produced by LCS-LENGTH on sequences
X = [A, B, C, B, D, A, B] and
Y = [B, D, C, A, B, A].

The running time of procedure is O(mn), since each

table entry takes O(1) time to compute.
PRINT-LCS(b, X, i, j)
1. If i = 0 or j = 0
2. then return
3. If b[i, j] = “ ”
4. then PRINT-LCS(b, X, i-1,j-1)
5. print xi
6. Else if b[i, j] = “ ”
7. then PRINT-LCS(b, X, i-1, j)
8. Else Print-LCS(b, X, I, j, -1)

This procedure prints “BCBA”

The procedure takes time O(m+n).