PwD 2019-20 Winter

Lecture 9
SS+AP
Today’s class
• Randomized algorithms
• Quicksort and its Randomised version

• Probabilistic algorithms/

• Approximation algorithms
• The ‘rho’ ratio
• Constant-factor approx.
• approximation schemata
• APX: polynomial approx.
• Beyond that
Probabilistic algorithms
Main idea
• Build algorithms using a ‘random’ element so as gain improved
performance.
• Randomized: the execution depends on a random choice, but result
(output) does not.
• Probabilistic: both execution and result depend on random choice. The
answer is reliable but only with a certain probability.
• For some cases, improved performance is dramatic.
• Often however, there is some loss in the reliability of results:
• Either no result at all may be produced, or an incorrect result may be returned, or,
for numerical results, an approximate answer may be produced.
• Because of the random element, different runs produce different results
 From Quicksort to randomized Quicksort

10 16 8 12 15 6 3 9 5 INF

right
left

Pivot: 10
1. Increment left pointer until you find an element greater than pivot
2. Decrement right pointer until you find an element smaller than pivot
3. Exchange them
4. Continue..
5. We found the partitioning position
6. Continue recursively
Quicksort, cont’d
.
• Quicksort is a recursive algorithm.
• The set of elements S to be sorted are split into two parts S1, S2 by an
element pivot in S so that S1 contains all elements smaller
than pivot and S2 the rest.
• S1,S2 are then sorted in the same way giving as result the sorted
elements of S1 followed by pivot and then the sorted elements of S2.
• In Quicksort the element pivot is chosen deterministically, for
instance the middle element in the unsorted set.
QS worst-case analysis
• Quicksort has a worst-case time requirement of O(n2).
• The worst case occurs when, at every step, the partition procedure
splits an n-length array into arrays of size 1 and n−1.
• This "unlucky" selection of pivot elements requires O(n) recursive
calls, leading to a O(n2) worst-case.
Randomized Quicksort
• Choosing the pivot randomly or randomly shuffling the array prior to
sorting has the effect of rendering the worst-case very unlikely,
particularly for large arrays.
• Proof that the expected time requirement is O(nlogn) [Wikipedia].
• Using random pivoting we improve the expected or average time
complexity to O(nlogn).
• Worst-Case complexity is still O(n^2).
Probabilistic algorithms
• Algorithms which always return a result, but the result may not always be
correct. We attempt to minimise the probability of an incorrect result, and
using the random element, multiple runs of the algorithm will reduce the
probability of incorrect results.
• These are sometimes called Monte Carlo algorithms.
• Terminology has not been fixed, and varies with author.
• Algorithms that never return an incorrect result, but may not produce
results at all on some runs. Again, we wish to minimise the probability of
no result, and, because of the random element, multiple runs will reduce
the probability of no result.
• These are sometimes called Las Vegas algorithms
Monte Carlo method
• Monte Carlo methods are a broad class of computational algorithms
that rely on repeated random sampling to obtain numerical results.
One of the basic examples of getting started with the Monte Carlo
algorithm is the estimation of Pi.
[https://www.geeksforgeeks.org/estimating-value-pi-using-monte-carlo/]
Monte Carlo method
• Estimating the value of Pi using Monte Carlo
• Pi is the ratio of a circle's circumference to its diameter
• Study the link
• Demo
• Try pwd-ex-mc-pi.py
Approximating intractable
problems
Approximation Algorithms
• Many problems of practical significance are NP-complete problems,
• Yet are too important to be abandoned…
• We still don’t know how to find a solution in polynomial time
• Still there is hope
Approximation Algorithms
• There are three ways to go around NP-Completeness:
1. If the inputs are normally small then an exponential algorithm would do.

2. We may isolate important cases that we can solve in polynomial time

3. Come up with an approach to find a near optimal solution

Near optimal is called approximation..
 Approximation Algorithms
• Suppose we are working on an optimization problem and we want to find a
near optimal solution
• Depending on the problem we may say that we have maximum possible cost
or minimum possible cost
• We say that an algorithm has an approximation ratio of 𝜚(𝑛) if for any input of
size n, the cost 𝐶 of the solution produced by the algorithm is within a factor
𝜚(𝑛) of the cost of the 𝐶 ∗ of an optimal solution:

𝐶 𝐶∗
𝑚𝑎𝑥 ∗ , ≤ 𝜚(𝑛)
𝐶 𝐶
Approximation Algorithms
• If an algorithm achieves an approximation 𝜚(𝑛) is called 𝜚(𝑛) –
approximation algorithm.
• The definition of the approximation ratio and of a 𝜚(𝑛)
approximation algorithm applies for both minimization and
maximization problems.
𝐶 ∗
• For maximization problem 0 < 𝐶 < 𝐶 ∗ with ratio
𝐶
∗ 𝐶
• For minimization problem 0 < 𝐶 < 𝐶 with ratio ∗
𝐶
 Approximation Algorithms
• Some NP-Complete problems allow approximation a. with (small) constant
approximation ratios, e.g., TSP (sometimes 𝜚(𝑛) = 3/2, see next)
• Some NP-Complete problems allow polynomial-time approximation
algorithms that can achieve approximation ratios that are proportional to the
time spent on solving the problem: approximation Schemata
• Most known NP-C problems admit approximation ratios that grow as a
polynomial in the input size n: APX
• Some NP-C problems, e.g., Max Clique, admit none of the above: they are the
hardest problems we face.
Approximating Knapsack
• 0-1 Knapsack is NP-Complete
• If input values are high, then the
solution becomes infeasible and there
is a need of approximate solution.
• Lets solve it!
Input:
W (Capacity of Knapsack)
val[0..n-1] (Values of Items)
wt[0..n-1] (Weights of Items)
Steps
1. Find the maximum valued item, i.e.,
find maximum value in val[]. Let this
maximum value be maxVal.
2. Compute adjustment factor k for all
values
3. Adjust all values, i.e., create a new
array val'[] that values divided by k.
Do following for every value val[i].
4. Run DP based solution for reduced
values, i,e, val'[0..n-1] and all other
parameter same.
Example of Knapsack
Quote of the day