Heuristic and Randomised Algorithms

1 2 3 4
5 6 7 8
9 10 11 12
13 14 15
 Heuristic algorithms
A heuristic is a technique designed for solving a problem
more quickly when classic methods are too slow, or for
finding an approximate solution when classic methods fail to
find any exact solution.
This is achieved by trading optimality, completeness,
accuracy, or precision for speed. In a way, it can be
considered a shortcut. The objective of a heuristic is to
produce a solution in a reasonable time frame that is good
enough for solving the problem at hand. This solution may
not be the best of all the actual solutions to this problem, or it
may simply approximate the exact solution. But it is still
valuable because finding it does not require a prohibitively
long time.
Heuristic algorithms
Heuristics may produce results by themselves, or they may be
used in conjunction with optimization algorithms to improve
their efficiency.
Results about NP-hardness in theoretical computer science
make heuristics the only viable option for a variety of complex
optimization problems that need to be routinely solved in real-
world applications.
Trade-off
The trade-off criteria for deciding whether to use a heuristic
for solving a given problem include the following:
Optimality: When several solutions exist for a given problem,
the heuristic has to guarantee that the near best solution can be
found. In some applications only best solution is acceptable
and a heuristic algorithm may not guarantee that.
Heuristics cont....(Trade off)
Completeness: When several solutions exist for a given
problem(8 queens), can the heuristic find them all if
required? Many heuristics are only meant to find one
solution.
Accuracy and precision: Can the heuristic provide a
confidence interval for the purported solution? Is the error
bar on the solution unreasonably large?
Execution time: There can be many heuristics and some are
faster than others(16-puzzle). Some heuristics are only
marginally quicker than classic methods.
In some cases, it may be difficult to decide whether the
solution found by the heuristic is good enough, because the
theory underlying that heuristic is not very elaborate.
 Heuristics cont....
The heuristic approach to problem solving consists of applying
human intelligence, experience, common sense and certain rules of
thumb (or heuristics) to develop an acceptable, but not necessarily
an optimum, solution to a problem. Of course, determining what
constitutes an acceptable solution is part of the task of deciding
which approach to use.
Often the effort (manpower, computer, and other resources)
required, the time limits on when the solution is needed, and the
cost to compile, process, and analyze all the data required for
deterministic or other complicated procedures preclude their
usefulness or favour the faster, simpler heuristic approach. Thus,
the heuristic approach is generally used when deterministic
techniques or are not available, economical, or practical.
Traveling Salesman problem
Given a complete graph with nonnegative edge costs, find
a minimum cost cycle visiting every vertex exactly once.
Application : Given a number of cities and the costs of
traveling from any city to any other city, what is the
cheapest round-trip route that visits each city exactly once
and then returns to the starting city.
Travelling salesman problem

The Travelling salesman problem (TSP) is known

to be NP-Complete, so an optimal solution for even
moderate size problem is intractable. Instead, the
greedy algorithm can be used to give a good but
not optimal solution (it is an approximation to the
optimal answer) in a reasonably short amount of
time.
The greedy algorithm heuristic is to pick whatever
is currently the best next step regardless of whether
that precludes good, few steps later. It is a heuristic
in that practice says it may be good enough
solution, but usually there are better solutions.
 TSP with the triangle inequality
• The following algorithm computes a near-optimal tour of an
undirected graph G using minimum-spanning-tree algorithm(MST-
PRIM).
• We shall then use the minimum spanning tree to create a tour whose
cost is no more than twice that of the minimum spanning tree’s
weight, as long as the cost function satisfies the triangle inequality.

Approx-
Approx-TSP-Tour(G,c)
TSP-Tour(G,c)
11 select vertexrrv[G]
selectaavertex v[G]to tobe
beaa“root”
“root”vertex;
vertex;
22 call
callMST-Prim(G,c,r)
MST-Prim(G,c,r)to toconstruct
constructMST
MSTwith
withroot
rootr;r;
33 let
letLL==vertices
verticeson
onpreorder
preordertree
treewalk
walkofofMST;
MST;
44 let
letHH==cycle
cyclethat
thatvisits
visitsvertices
verticesin
inthe
theorder
orderL;
L;
return
returnHH
 Cheapest Link Algorithm
The cheapest link algorithm is a greedy heuristic algorithm
for finding candidate solutions to the travelling salesman
problem. The cheapest link algorithm begins by choosing
the route that is the shortest distance between any two
cities. It continues by choosing the next shortest link
between any two cities. After the first two links, you must
make certain that no link closes the loop until all of the
cities are connected and also not more than two edges
emanate from a vertex. This method can at times find the
best overall solution, but it is not guaranteed to.
 Cheapest Link Algorithm

Problem: Find the cheapest Hamiltonian circuit if it exists.

Edge with smallest weight is drawn first. Edge with second smallest
weight is drawn in. Continue unless it closes a smaller circuit or three
edges come out of one vertex. Stop once a complete Hamilton Circuit is
drawn.
 Virus scanning
Many virus scanners use heuristic rules for detecting viruses and
other forms of malware. Heuristic scanning looks for code and/or
behavioural patterns indicative of a class or family of viruses, with
different sets of rules for different viruses. If a file or executing
process is observed to contain matching code patterns and/or to be
performing that set of activities, then the scanner infers that the file
is infected.
The most advanced part of behaviour-based heuristic scanning is
that it can work against highly randomized polymorphic viruses,
which simpler string scanning-only approaches cannot reliably
detect. Heuristic scanning has the potential to detect many future
viruses without requiring the virus to be detected somewhere,
submitted to the virus scanner developer, analyzed, and a
detection update for the scanner provided to the scanner's users.
16-puzzle problem: On a 4*4 square board 15 movable
pieces with unique numbers from 1 to 15 are available
and in the beginning they are not in any order. A program
has to find a series of moves to arrange them in
increasing order of their number.

1 2 3 4
5 6 7 8
9 10 11 12
13 14 15
Randomized algorithms
We define a randomized algorithm as an algorithm
that uses random numbers at least once during the
computation to make a decision. A randomized
algorithm is an algorithm whose working not only
depends on the input but also on certain random
choices made by the algorithm.
Example: In Quicksort, using a random number to choose a
pivot
Example: Real world simulations :Car parking simulation,
scheduling algorithms simulation, flight reservation
simulation, Playing cards simulation
 Quick Sort
Select: pick an arbitrary
element x in S to be the pivot.

Partition: rearrange elements

so that elements with value less
than x go to List L to the left of
x and elements with value
greater than x go to the List R
to the right of x.

Recursion: recursively sort the

lists L and R.

14
 Randomized Algorithms

INPUT ALGORITHM
OUTPUT

RANDOM NUMBERS

 In addition to input, algorithm takes a source of random

numbers and makes random choices during execution,
behavior can vary even on a fixed input;
 Why randomized algorithms? : Avoid worst-case behavior as
randomness can guarantee average case behavior in most
cases & efficient approximate solutions to inflexible problems.
15
Let us play cards
Create a pack of playing cards ( 4 suits (colors) &
each suit will have 2,3,..10, Jack , Queen, King, Ace).
Display the cards. Simulate shuffling of the pack
and display the shuffled cards. Distribute them
among 4 players in a Round robin fashion and
display the cards with each player. Give some value
for each color and card and find the total values of
the cards with each player.
Process scheduling
UNIX operating system uses Round robin Time slicing
with multilevel feed back for process scheduling.
Assume that there are 10 processes waiting in a queue
which is implemented as a linked list of PCB’s
( process control blocks). Assume the PCB have
information about Process ID , CPU burst time
required and amount of memory being used .
Assume the time slice is 2 units. Simulate Round robin
Time slicing until all the jobs complete and find average
waiting time. Modify your program to include random
arrival of jobs with a fixed burst time required and find
the average waiting time of the jobs completed over a
simulation time of 100 units.
Queuing Systems
At a car washing station cars arrive at random. Each car
requires some X amount of time for wash. When a car Y
comes in and if another car is being serviced or one or
more cars are in the waiting queue, then car Y has to
wait at the end of the queue. When the service station is
free the car in the front of the queue moves out to the
service. Simulate the process to find the average waiting
time of cars for a given X.
Note: choose X and frequency of arrival of cars in such a way,
there are cars waiting in the queue most of the time.
 Able-Baker Call Centre Problem
This example illustrates the simulation procedure when
there is more than one service channel. Consider a
computer technical support centre where personnel take
calls and provide service. The time between calls is
random. There are two technical support people: Able
and Baker.
Able is more experienced and can provide service faster
than Baker. Times are usually a continuous measure, but
this time-based example is made discrete for ease of
explanation. The simulation proceeds in a complex
manner because of two servers. When both are free Able
gets the call. Service time distribution is shown in next
slide .
Able-Baker Call Centre Problem
The problem is to find how well the current arrangement is
working. To estimate the system measures of performance, a
simulation of given units of time can be made. The simulation
can find average waiting time & turn-around time.

Service Distribution of Able

T time
Service Probability Cumulative Probability Random Digit

2 0.20 0.20 assignment

01-20
3 0.30 0.50 21-50
4 0.30 0.80 51-80
5 0.20 1.00 81-100

Service Distribution of Baker

Service time Probability Cumulative Probability Random Digit
T
assignment
4 0.20 0.20 01-20
5 0.30 0.50 21-50
6 0.30 0.80 51-80
7 0.20 1.00 81-100
Table 1

89
89
89
 Las Vegas Algorithms
Las Vegas Algorithms: These refer to the randomized
algorithms that always come up with a/the correct
answer. Their expected running time is polynomial in the
size of their input.
 In the worst case, however, a Las Vegas algorithm may
take exponentially long. One example of a Las Vegas
algorithm is Quicksort : it makes some of its decisions
based on random numbers, it always produces the correct
result, and its expected running and worst-case running
times are n log n and n2 , respectively.
Monte Carlo Algorithms
Monte Carlo Algorithms: These refer to the randomized
algorithms that sometimes come up with an incorrect answer.
Like Las Vegas algorithms their expected running time is
polynomial but may be exponential.
By performing independent runs of a Monte Carlo algorithm,
we can decrease the chances of obtaining an incorrect result to
a value as small as we like. For example, if a particular Monte
Carlo algorithm produces an incorrect result with probability
1/4, and we have the ability to detect an incorrect result, then
the probability of not obtaining a correct answer after t
independent runs is (1/4)t.
In some contexts, such as in optimization problems, we do not
have the ability to detect an incorrect answer, but can compare
different answers and pick the best one.