l1 Ps

W4231:Analysis of Algorithms Textbook
Le ture 1: Introdu tion [CLR℄ Thomas H. Cormen, Charlie E. Leiserson and Ronald L. Rivest.
Introdu tion to Algorithms. MIT Press, 1990.
| 4231: Analysis of Algorithms | 1 | 4231: Analysis of Algorithms | 2
Topi s Algorithms
Review of basi material. Models of omputation, spa e and time In omputer s ien e we want to solve omputational problems using a
omplexity, lower bounds, re urren es. omputer.
Algorithmi Te hniques. Divide and onquer; dynami programming. An algorithm is an abstra t des ription of a method to do so.
Sorting. And order statisti s. An algorithm better be eÆ ient: be fast and not use too mu h memory.
Sear hing. Data stru tures for representing sets: hash tables, balan ed
sear h trees, union-nd data stru tures.
Algorithms on graphs. In luding onne tivity, strong onne tivity and
shortest paths.
Dealing with hard problems. Intro to NP- ompleteness and approximation
algorithms.
Exponential time is bad Quadrati time is good
Say that you have a problem that, for an input onsisting of n items, an Another algorithm uses 300n2 lo k y les on a 80386, and you use a PC
be solved by going through 2n ases. running at 33MHz. (Su h omputers did exist a few years ago.)
You use Deep Blue, that analyses 200 million ases per se ond. An input with 15 items will take 2 millise onds.
An input with 15 items will take 163 mi rose onds. An input with 30 items will take 8 millise onds.
An input with 30 items will take 5:36 se onds. An input with 50 items will take 22 millise onds.
An input with 50 items will take more than two months. An input with 80 items will take 58 millise onds.
An input with 80 items will take 191 million years.
Role of improved hardware Polynomial time and eÆ ien y
The largest instan e solvable in a day by the 2n algorithm using Deep Blue Whenever an algorithm runs in O(n ) time, where n is the size of the input
has 44 items. Using a omputer 10 times faster we an go to 47 items. (In and is a onstant, we say that the algorithm is \eÆ ient".
general, we go from I to I + 3 or I + 4 items.)
We want to nd polynomial-time algorithms for every interesting problem,
The largest instan e solvable in a day by the 300n2 algorithm on the old PC and with the smallest exponent.
has 97488 items. Using a omputer 10 times faster we an go to 308285
items. (In general, from I to 10I )
p
An algorithm running in O(n log n) time is always preferable to an O(n2)
algorithm, for all but nitely many instan es.
Asymptoti analysis and pra ti e More on asymptoti analysis
An algorithm requiring 10; 000n = O(n) instru tions for an input of size The Alta Vista database ontains 200 Gigabytes of data about a few
n is faster for all but nitely many ases than an algorithm requiring hundred millions web pages.
300n log2 n = O(n log2 n) instru tions.
You don't want to run a quadrati time algorithm on it!
But the latter algorithm is faster when n 233:3 10:8 billions.
Other omputational problems with huge input size: optimizing VLSI design
for hip proje ts; nding patterns in phone alls so as to reate and oer
new alling plans; . . .
Worst ase In Short
We always onsider the running time of an algorithm in the worst ase. For A more eÆ ient algorithm an speed-up a omputation more than a million-
every n, we look at the input of size n that makes the algorithm going most fold in rease in hardware speed.
slowly.
Algorithms are part of omputer te hnology.
Motivations:
With its limitations, worst- ase asymptoti analysis of algorithms is
Often worst- ase running time and \typi al" running time are very lose. an ex eedingly su essful methodology to evaluate the goodness of an
algorithm.
Average- ase analysis requires to know the probability distribution.
Higher reliability.
But sometimes algorithms that are bad in the worst ase are good in
pra ti e.
Analysis of algorithms: Goal How to nd the best possible algorithm for a problem
To nd the best possible way of solving all interesting omputational
problem. 1. Formalize the omputational problem.
(Well, if you have to have a goal it an as well be ambitious.) 2. Design an algorithm that provably works.
3. Analyze the algorithm and prove an upper bound on its running time
(resp. memory use)
4. Prove a lower bound showing that you annot do better.
5. Implement the algorithm and experiment with it to see how does it
ompares to other algorithms.
In this ourse Formalism
Points 1, 5 (formalization and experiments) will not be overed. Point 4 Sin e we want algorithms to be provably good, everything has to be
(lower bounds) is very important but we will over it shortly. The ourse is formalized mathemati ally | otherwise we annot start proving things.
mainly about 2 (design) and 3 (analysis).
The term analysis of algorithm often refers to both proving an upper bound The problem that you want to solve has to be formalized mathemati ally
for a given algorithm and proving a lower bound for every other algorithm. (we will only deal with already-formalized problems).
The omputer you want to use, has to be des ribed as a mathemati al
obje t.
An algorithm is already a mathemati al obje t.
Models of omputation Models of omputation
We want to dene an abstra t model of a omputer that be
RAM. Very lose to a real omputer. Good for upper bounds.
Simple. Turing Ma hine. Very simple. Good for (non-tight) lower bounds.
Quantitative. De ision Tree. Not very general. Good to prove lower bounds for ertain
algorithms.
General.
There is no su h a thing as a \best model of omputation".
DiÆ ulty and importan e of proving lower bounds A simple te hnique to prove a lower bound
A lower bound has to be true with respe t to every possible algorithm. \Information theoreti " bound:
We have to onsider innitely many ases.
If the algorithm learns at most a bit of information on the input at every
In ryptography you want to prove that every attempt to break the ode step;
will fail. Similar diÆ ulties.
And the algorithm an behave in H possible ways depending on the
input.
Then the algorithm must run for at least log2 H steps.
Sorting with omparisons De ision Tree model
You have a sequen e of numbers a1; : : : ; an. The only allowed operations The argument an be made more formal using the de ision tree model.
on su h numbers are: omparing two elements, and swapping the positions
of two elements. The de ision tree for the problem of sorting a sequen e of n numbers has
n! leaves and so the depth has to be at least log n!.
The algorithm learns at most a bit at a time.
The algorithm outputs one out of n! possible permutations of the input.
The algorithm must run for at least log2 n! = (n log n) steps.
Algorithms for Simple(?) Problems Given unsorted sequen e a1; : : : ; an, nd the (n=2)-th element in
as ending order.
Simple algorithm: sort a1; : : : ; an, get s1; : : : ; sn, output sn=2.
O(n log n) time.
Multiplying two long integers (of n digits).
Better algorithm: O(n) time.
Standard method: O(n2) elementary operations.
Better algorithm: O(n1:6).
Even better: O(n log n log log n) with the Fast Fourier Transform.
Multiplying two n n matri es.
Standard method: O(n3) operations.
Better algorithms: O(n2:81), O(n2:37) . . .
Maybe one day O(n2 log2 n).
Re ursive algorithms Master Theorem
Mergesort. Let a; b; 1 onstants. Let = logb a. Let f () be a positive fun tion,
and T () be a fun tion over the integers dened as
We want to sort a1; : : : ; an.
First sort a1; : : : ; an=2 and an=2+1; : : : ; an separately; then merge the two T (1) = and T (n) = aT (n=b) + f (n)
subsequen es.
If T (n) is the number of operations, then then
1. If f (n) = O(n ) with < , then T (n) = (n ).
0
0
T (1) = 1 and T (n) = 2T (n=2) + n
2. If f (n)(n ) then T (n) = (n log n).
where is a onstant.
3. If f (n) =
(n ) with > and (. . . ), then T (n) = (f (n)).
0
Then it must be T (n) = (n log n).
0
Examples Akra & Bazzi
Always assume T (1) = O(1). Akra and Bazzi (1996) have found a more general theorem for solving
re urren es.
1. If T (n) = 7T (n=2) + O(n2) then T (n) = (nlog2 7) = (n2:807:::) We will be happy with the Master Theorem.
(matrix multipli ation).
2. If T (n) = 2T (n=2) + O(n) then T (n) = (n log n) (mergesort).
3. If T (n) = 2T (n=2) + n2 then T (n) = O(n2) (made-up example).

l1 Ps

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

l1 Ps

Uploaded by

Copyright:

Available Formats

W4231:Analysis of Algorithms Textbook

You might also like