Professional Documents
Culture Documents
Algorithms
Srikrishnan Divakaran
DA-IICT
1
Talk Outline
2
Preliminaries and Motivation
3
Quick Sort
4
Worst Case Partitioning of
Quick Sort
5
Best Case Partitioning of Quick
Sort
6
Average Case of Quick Sort
7
Randomized Quick Sort
Randomized-Partition(A, p, r)
1. i Random(p, r)
2. exchange A[r] A[i]
3. return Partition(A, p, r)
Randomized-Quicksort(A, p, r)
1. if p < r
2. then q Randomized-Partition(A, p, r)
3. Randomized-Quicksort(A, p , q-1)
4. Randomized-Quicksort(A, q+1, r)
8
Randomized Quick Sort
9
Deterministic Algorithms
INPUT ALGORITHM
OUTPUT
Goal: Prove for all input instances the algorithm solves the
problem correctly and the number of steps is bounded by a
polynomial in the size of the input.
10
Randomized Algorithms
INPUT ALGORITHM
OUTPUT
RANDOM NUMBERS
11
Las Vegas Randomized
Algorithms
INPUT ALGORITHM
OUTPUT
RANDOM NUMBERS
Goal: Prove that for all input instances the algorithm solves the
problem correctly and the expected number of steps is bounded by
a polynomial in the input size.
Note: The expectation is over the random choices made by the
algorithm.
12
Probabilistic Analysis of
Algorithms
RANDOM ALGORITHM OUTPUT
INPUT DISTRIBUTION
Goal: Show that for all inputs the algorithm works correctly and for
most inputs the number of steps is bounded by a polynomial in the
size of the input.
13
Min-cut for Undirected Graphs
Given an undirected graph, a global min-cut is a cut (S,V-S)
minimizing the number of crossing edges, where a crossing edge is
an edge (u,v) s.t. u∊S and v∊ V-S.
V-S S 14
Graph Contraction
For an undirected graph G, we can construct a new graph G’ by contracting
two vertices u, v in G as follows:
– u and v become one vertex {u,v} and the edge (u,v) is removed;
– the other edges incident to u or v in G are now incident on the new
vertex {u,v} in G’;
Note: There may be multi-edges between two vertices. We just keep them.
b
a b
a
u v
{u,v}
d e
c d e
c
Graph G Graph G’ 15
Karger’s Min-cut Algorithm
C C CD
contract
B B D B
D
A A
A contract
(i) Graph G (ii) Contract nodes C and D (iii) contract nodes A and CD
B
16
(Iv) Cut C={(A,B), (B,C), (B,D)}
Karger’s Min-cut Algorithm
For i = 1 to 100n2
repeat
randomly pick an edge (u,v)
contract u and v
until two vertices are left
ci ← the number of edges between them
Output mini ci
17
Key Idea
• Let C* = {c1*, c2*, …, ck*} be a min-cut in G and Ci be a cut
determined by Karger’s algorithm during some iteration i.
18
Monte Carlo Randomized
Algorithms
INPUT ALGORITHM
OUTPUT
RANDOM NUMBERS
• These expectations are only over the random choices made by the
algorithm independent of the input. Thus independent repetitions of
Monte Carlo algorithms drive down the failure probability
exponentially.
20
Motivation for Randomized
Algorithms
• Simplicity;
• Performance;
21
Analysis of Randomized Quick
Sort
22
Linearity of Expectation
n n
E Xi E[ Xi]
i 1 i 1
23
Notation
z2 z9 z8 z3 z5 z4 z1 z6 z10 z7
2 9 8 3 5 4 1 6 10 7
• Rename the elements of A as z1, z2, . . . , zn, with zi being the ith
smallest element (Rank “i”).
• Define the set Zij = {zi , zi+1, . . . , zj } be the set of elements between
zi and zj, inclusive.
24
Expected Number of Total
Comparisons in PARTITION
indicator
Let Xij = I {zi is compared to zj } random variable
Let X be the total number of comparisons performed by the
algorithm. Then
n 1 n
X X ij
i 1 j i 1
The expected number of comparisons performed by the algorithm is
n 1 n
EX
n 1 n
E[ X ] E X ij ij
i 1 j i 1 i 1 j i 1
by linearity
of expectation
n 1 n
Pr{zi is compared to z j }
i 1 j i 1 25
Comparisons in PARTITION
Observation 1: Each pair of elements is compared at most once
during the entire execution of the algorithm
– Elements are compared only to the pivot point!
– Pivot point is excluded from future calls to PARTITION
Pr{zi is compared to z j }?
n 1 n
E[ X ] Pr{z i is compared to z j }
i 1 j i 1
n 1 n 1 n i n 1 n
n
2 2 2 n 1
E[ X ] O(lg n)
i 1 j i 1 j i 1 i 1 k 1 k 1 i 1 k 1 k i 1
= O(nlgn)
28
Analysis of Karger’s Min-Cut
Algorithm
29
Analysis of Karger’s Algorithm
Let k be the number of edges of min cut (S, V-S). k
30
Analysis of Karger’s Algorithm
• In step 1, Pr [no crossing edge picked] >= 1 – 2/n
• Similarly, in step 2, Pr [no crossing edge picked] ≥ 1-2/(n-1)
• In general, in step j, Pr [no crossing edge picked] ≥ 1-2/(n-j+1)
• Pr {the n-2 contractions never contract a crossing edge}
– = Pr [first step good]
* Pr [second step good after surviving first step]
* Pr [third step good after surviving first two steps]
* …
* Pr [(n-2)-th step good after surviving first n-3 steps]
≥ (1-2/n) (1-2/(n-1)) … (1-2/3)
= [(n-2)/n] [(n-3)(n-1)] … [1/3] = 2/[n(n-1)] = Ω(1/n2)
31
Basic Analytical Tools
32
Tail Bounds
• In the analysis of randomized algorithms, we need to know how
much does an algorithms run-time/cost deviate from its expected
run-time/cost.
• That is we need to find an upper bound on Pr[X deviates from E[X] a
lot]. This we refer to as the tail bound on X.
33
Markov and Chebyshev’s Inequality
34
Yao’s Inequality for
Establishing Lower Bounds
35
Algorithm Analysis in terms of Two
Player Zero Sum Games
COLUMN PLAYER
• the sum of payoffs of two
players is zero in each cell of ALG1 ALG2 … ALGc … ALGn
the table;
Input1
• The row player (one who R
maximizes cost) is the O
adversary responsible for W Input2
designing malicious inputs;
P M (r; c)
L Inputr
• The column player (one who A .
minimizes cost) is the Y .
algorithm designer responsible E
for designing efficient R
algorithms;
Inputm
36
PAYOFF MATRIX M
Pure Strategy
37
Mixed strategies
• For the column player (minimizer)
– Each mixed strategy corresponds to a Las Vegas randomized
algorithm.
• For the row player (maximizer)
– Each mixed strategy corresponds to a probability distribution
over all the input instances.
38
Von Neumann’s
Minimax Theorem
F o r a n y 2 -p er so n 0 -su m g a m e, w e h a v e
T T
m a x m in p M q = m in m ax p M q:
p q q p
T h a t i s, ea ch 2 -p er so n zer o -su m g a m e h a s a sa d d l e
p o i n t w i t h r esp ect t o m i x ed st r a t eg i es.
39
Loomis’ Theorem
m a x m i n p T M ec = m i n m a x eT
r M q;
p c q r
w h er e e i m ea n s r u n n i n g t h e i -t h st r a t eg y w i th p r ob a b i l i ty
1.
40
Yao’s interpretation for Loomis’
Theorem
L et T ( I ; A ) denote the tim e required for algorithm A to run on
input I . T hen by L oom is T heorem , we have
m ax m in E [T ( I p; A ) ] = m in m ax E [T ( I ; A q) ]:
p deterministic algorithm A q input I
T herefore, the follow ing inequality holds for any probability dis-
tribution p and q:
m in E [T ( I p; A ) ] · m ax E [T ( I ; A q) ]:
deterministic algorithm A input I
41
How to use Yao’s Inequality?
• Task 1:
– Design a probability distribution p for the input instance.
• Task 2:
– Obtain a lower bound on the expected running for any
deterministic algorithm running on Ip.
42
Application of Yao’s Inequality
Find bill problem: There are n boxes and exactly one box contains a
dollar bill, and the rest of the boxes are empty. A probe is defined as
opening a box to see if it contains the dollar bill. The objective is to locate
the box containing the dollar bill while minimizing the number of probes
performed.
Randomized Find
1. select x έ {H, T} uniformly at random
2. if x = H then
(a) probe boxes in order from 1 through n and stop if bill is located
3. else
(a) probe boxes in order from n through 1 and stop if bill is located
Proof: We assume that the bill is located in any one of the n boxes
uniformly at random. We only consider deterministic algorithms that does
not probe the same box twice. By symmetry we can assume that the
probe order for the deterministic algorithm is 1 through n.
B Yao’s in-equality, we have
44
References
1. Amihood Amir, Karger’s Min-cut Algorithm, Bar-Ilan University, 2009.
2. George Bebis, Randomizing Quick Sort, Lecture Notes of CS 477/677:
Analysis of Algorithms, University of Nevada.
3. Avrim Blum and Amit Gupta, Lecture Notes on Randomized Algorithms, CMU,
2011.
4. Hsueh-I Lu, Yao’s Theorem, Lecture Notes on Randomized Algorithms,
National Taiwan University, 2010.
5. Rada Mihalcea, Quick Sort, Lecture Notes of CSCE3110: Data Structures,
University of North Texas, http://www.cs.unt.edu/~rada/CSCE3110.
6. Rajeev Motwani and Prabhakar Raghavan, Randomized Algorithms,
Cambridge University Press, 1995.
7. Prabhakar Raghavan, AMS talk on Randomized Algorithms, Stanford
University.
45
1000 Whats, What Nuts
&
Wall (Face Book) Nuts?
46