L1

Brute Force Method

Brute Force Method
Brute Force Method
 Analysis of Selection Sort

Time efficiency: Θ(n^2)

 Analysis of Selection Sort - Exercise

• Sort the list E, X, A, M, P, L, E in alphabetical

order by selection sort.
L2
Bubble sort animation
Bubble sort (Sinking sort)

In this technique, the two successive items a [ j ] and a[ j+1] are exchanged
whenever a [ j ]>= a[ j+1]
Pass 1
Eg : 40 50 30 20 10
a [0] 40 40 40 40 40 30
a [1] 50 50 30 30 30 40
a [2] 30 30 50 20 20 20
a[3] 20 20 20 50 10 10
a 4] 10 10 10 10 50 50

Pass 2 Pass 3 Pass 4

30 30 20 20 10
20 20 30 10 20
40 10 10 30 30
10 40 40 40 40
50 50 50 50 50
it is not easy to tell because it If the input is already sorted then no swap will occur.
swaps when two consecutive If the input is sorted in some places then swap will
elements are not in the order occur only in those places.

Comparisons : in outer loop we go from 0 to n-2 comparisons, and in inner loop we

go from j = 0 to n-2-i.
Swap changes depends on the input. How well the list is sorted. Worst case swap will be
high, best case –low and average case it will be medium.

but the number of comparisons are always the same

It is theta n2 because it does not depend on best case , worst case or average case
 Bubble sort - Exercise

• Sort the following list in decreasing order: 89, 56, 12,

99, 45, 92, 17 using bubble sort technique.

• sort the list E, X, A, M,P, L, E in alphabetical order using

bubble sort.

• .
 Pros and cons
• This algorithm is not very popular, however it is still useful.
• The bubble sort compares adjacent items and swaps them if they are out of
order.
• During each pass, until the array is sorted, the algorithm traverses through
the data and compares adjacent items, swapping them if they are not in
ascending order.
• It is simple to understand and implement.
• However, this sort requires several passes over the data, and thus
introduces a major factor of inefficiency.
Bubble sort :Improved version

i=0 , i<n-1 or i<=n-2

j=0, j<n-i-1
L4
 Bubble sort
if bubble sort makes no exchanges on its pass through a list, the list
is sorted and the algorithm can be stopped.

A list is sorted if and only if all its adjacent elements are in a correct
order.
 Bubble sort
Write pseudocode of the method that incorporates this improvement.

Add a boolean flag to register the presence or absence of switches.

Prove that the worst-case efficiency of the improved version is quadratic

Identify worst-case inputs first.

The worst-case inputs will be strictly decreasing arrays.

 Sorting
Stable Selection Sort

• A sorting algorithm is said to be stable if two objects with equal or same

keys appear in the same order in sorted output as they appear in the
input array to be sorted.
• Any comparison based sorting algorithm which is not stable by nature can
be modified to be stable by changing the key comparison operation so
that the comparison of two keys considers position as a factor for objects
with equal key or by tweaking it in a way such that its meaning doesn’t
change and it becomes stable as well.
 Sorting
Selection sort can be made Stable if instead of swapping, the minimum
element is placed in its position without swapping i.e. by placing the
number in its position by pushing every element one step forward.
In simple terms use a technique like insertion sort which means inserting
element in its correct place.
 Stable sorting
stableSelectionSort(int a[], int n)
{
    // Iterate through array elements
    for (int i = 0; i < n - 1; i++) 
    {
  
        // Loop invariant : Elements till a[i - 1]
        // are already sorted.
      // Find minimum element from 
        // arr[i] to arr[n - 1].

        int min = i;
        for (int j = i + 1; j < n; j++)
            if (a[min] > a[j])
                min = j;
  
        // Move minimum element at current i.
        int key = a[min];
        while (min > i) 
        {
            a[min] = a[min - 1];
            min--;
        }
        a[i] = key;
    }
}
 Polynomial

The most straightforward algorithm, which is based on substituting x0

into the formula, is quadratic.

Θ(n2)
Polynomial P(x) = 2x2 +4x +5
p[2] p[1] p[0]
P I (2 power J (1 to i)
to 0)
0 2 1
1*x 1

X*x 2

0+p[2]*x2 3
2x2
1 1
1*x 1
2x2 + 4x 2
0 1 -----
 Polynomial

If the algorithm you designed is in Θ(n2), design a linear algorithm

for this problem.

Analyzing what unnecessary computations the quadratic algorithm does should lead you
to a better (linear) algorithm.
 Polynomial

The above algorithm is very inefficient: we recompute powers of x again and again
as if there were no relationship among them.

Thus, the obvious improvement is based on computing consecutive powers more

efficiently.

If we proceed from the highest term to the lowest, we could compute xi−1 by using xi
but this would require a division and hence a special treatment for x = 0.

Alternatively, we can move from the lowest term to the highest and compute xi by
using xi−1.

Since the second alternative uses multiplications instead of divisions and does not
require any special treatment for x = 0, it is both more efficient and cleaner.

It leads to the following algorithm:

Horner’s method

10+5x+4x2+7x3+8x4

10+5x+4x2+7x3+8x4 = 10+ x (5+ x(4+ x(7+ 8x)))

i.e.: a0x0+ a1x1+ a2x2+ - - - - - - + anxn

= a0+ x (a1+ x (a2+x (a3+ - - - - - - x (an-1+ xan)- - - )))

(((anx+ an-1) x+ an-2) x+an-3) x+ - - - - - - - +a3) x+a2) x+a1) x+a0

 Polynomial

Simple method

O(n)
 Polynomial
Is it possible to design an algorithm with a better than linear efficiency for this
problem?

How many coefficients does a polynomial of degree n have? Can one compute its value
at an arbitrary point without processing all of them?

No, because any algorithm for evaluating an arbitrary polynomial of degree n at an

arbitrary point x must process all its n + 1 coefficients.
(Note that even when x = 1, p(x) = an+an−1 +...+a1 +a0, which needs
at least n additions to be computed correctly for arbitrary an, an−1, ..., a0.)
L5
L6
String match
We may do up to “m” comparisons - m is the length of the pattern
Complexity depends on the length of pattern
L7
 • Consider the problem of counting, in a given text, the number of
substrings that start with an A and end with B. For example, there are four
such substrings in CABAAXBYA.
Design a brute force algorithm for this problem and determine its efficiency
class.

Ans: Note that the number of desired substrings that starts with an A at a
given position i (0 ≤ i < n − 1) in the text is equal to the number of B’s
to the right of that position. This leads to the following simple algorithm:
 Example 1: Traveling Salesman
Problem
• Given n cities with known distances between
each pair, find the shortest tour that passes
through all the cities exactly once before
returning to the starting city
• Alternatively: Find shortest Hamiltonian circuit
2
in a weighted connected graph a b
5 3
• Example: 8 4

c d
How do we represent a solution (Hamiltonian circuit)? 7
a
2
b TSP by Exhaustive Search
5 3
8 4

c d
7
Tour Cost

a→b→c→d→a 2+3+7+5 = 17
a→b→d→c→a 2+4+7+8 = 21
a→c→b→d→a 8+3+4+5 = 20
a→c→d→b→a 8+7+4+2 = 21
a→d→b→c→a 5+4+3+8 = 20
a→d→c→b→a 5+7+3+2 = 17
Travelling Salesman Problem(TSP)

Difference between Hamiltonian Cycle and TSP.

The Hamiltoninan cycle problem is to find if

there exist a tour that visits every city
exactly once.

Here we know that Hamiltonian Tour

exists (because the graph is complete) and
in fact many such tours exist, the problem
is to find a minimum weight Hamiltonian
Cycle.
Travelling Salesman Problem(TSP)
Travelling Salesman Problem(TSP)

A TSP tour in the graph is 1-2-4-3-1

.
The cost of the tour is 10+25+30+15 which is 80.
 TSP – Try it out !!!
• Solve the following instance of Traveling Salesman Problem using
Exhaustive Search Method.

• Outline an exhaustive-search algorithm for the Hamiltonian circuit problem.

 Knapsack problem
• The knapsack problem or rucksack problem is a
problem in combinatorial optimization:
• Given a set of items, each with a weight and a
value, determine the number of each item to
include in a collection so that the total weight is
less than or equal to a given limit and the total
value is as large as possible.
• The problem often arises in resource allocation
where there are financial constraints
 Knapsack problem- Applications
• Knapsack problems appear in real-world decision-making processes in a wide
variety of fields, such as finding the least wasteful way to cut raw materials, [4]
selection of investments and portfolios,[5] selection of assets for
asset-backed securitization,[6] and generating keys for the Merkle–Hellman[7]
and other knapsack cryptosystems.
 Knapsack problem
Given weights and values of n items, put these items in a knapsack of capacity W to
get the maximum total value in the knapsack.

In other words, given two integer arrays val[0..n-1] and wt[0..n-1] which represent
values and weights associated with n items respectively.

Also given an integer W which represents knapsack capacity, find out the maximum
value subset of val[] such that sum of the weights of this subset is smaller than or
equal to W.

You cannot break an item, either pick the complete item, or don’t pick it (0-1
property).

Find the most valuable subset

 Knapsack problem
Item no. Item weight Item value Item Total weight Total value
1 5 $20 {1} 5 20
2 3 $40 {2} 3 40
3 4 $30 {3} 4 30
{1,2,} 5+3=8 20+40=60
Knapsack capacity = 10 {1,3) 5+4=9 20+30=50
{2,3} 3+4=7 40+30=70
{1,2,3} 5+3+4=12 NA
 Knapsack problem
• Brute force is a straightforward approach to solving a problem,
usually directly based on the problem’s statement and
definitions of the concepts involved.

• If there are n items to choose from, then there will be 2n

possible combinations of items for the knapsack.

• An item is either chosen or not chosen. A bit string of 0’s and 1’s
is generated which is of length n. If the ith symbol of a bit string
is 0, then the ith item is not chosen and if it is 1, the ith item is
chosen.
 Knapsack problem
• ALGORITHM BruteForce (Weights [1 … N], Values [1 … N], A[1…N])
• //Finds the best possible combination of items for the KP
• //Input: Array Weights contains the weights of all items
• Array Values contains the values of all items
• Array A initialized with 0s is used to generate the bit strings
• //Output: Best possible combination of items in the knapsack bestChoice [1 .. N]
 Try it out
A thief is robbing a store and can carry a maximal weight of W into his knapsack. There
are n items and weight of ith item is wi and the profit of selecting this item is pi. What
items should the thief take?
 Assignment Problem- Hungarian method
• Assignment
– Something : job, task
– Someone : machine , people

» profit , sales - maximization

» cost, time - minimization

» 3 rows , 3 columns --- ( N * N )

Brute force solution is to consider every possible assignment implies a

complexity of Ω(n!).
 Assignment Problem- Hungarian method

The Hungarian algorithm, utilizes the theorem for polynomial runtime complexity
(worst case O(n3)) and guaranteed optimality:
If a number is added to or subtracted from all of the entries of any one row or
column of a cost matrix, then an optimal assignment for the resulting cost matrix is
also an optimal assignment for the original cost matrix.
Assignment Problem- Hungarian method
Perform row subtraction

Reduced matrix
 Assignment Problem- Hungarian method
• Draw minimum number of lines(vertical / horizontal) which cover all the
lines

• Optimal solution – total number of lines = N ( no.of rows /columns)

 Assignment Problem- Hungarian method

• for allocation of job, search for row / column

with one zero.
• Cross all other zeros in that row or column
Assignment Problem- Hungarian method

The Hungarian algorithm, utilizes the following theorem

for polynomial runtime complexity (worst case O(n3)) and
guaranteed optimality:

If a number is added to or subtracted from all of the entries

of any one row or column of a cost matrix, then an optimal
assignment for the resulting cost matrix is also an optimal
assignment for the original cost matrix.
 Assignment Problem- Try it out
• You work as a manager for a chip manufacturer, and you currently have 3
people on the road meeting clients. Your salespeople are in Jaipur, Pune and
Bangalore, and you want them to fly to three other cities: Delhi, Mumbai
and Kerala. The table below shows the cost of airline tickets in INR between
the cities:

where would you send each of your salespeople in order to minimize fair?
 Assignment Problem- Hungarian method - II
Row subtraction

Column subtraction

No.of lines=3
N= 4

No.of lines != N
Reduced matrix
 Assignment Problem- Hungarian method - II

Find the smallest number from all the uncovered numbers

Perform addition(+) , subtraction(-) .

Numbers at the intersection (+), uncovered numbers (-)

addition(+) First improvement matrix

subtraction(-)
 Assignment Problem- Hungarian method - II

Allocation
 Assignment Problem- Hungarian method – unbalanced matrix
Convert the matrix into n*n matrix

Perform row subtraction and

column subtraction
Assignment Problem- Hungarian method – unbalanced matrix
Assignment Problem- Hungarian method – multiple optimum solution

First improvement matrix

Assignment Problem- Hungarian method – multiple optimum solution

second improvement matrix

Assignment Problem- Hungarian method – multiple optimum solution
L12
 DFS, BFS
• Student presentation

• A- Nishant , Abhishek
• D- Anurag
L13
 Decrease and conquer

Decrease and conquer involves reducing the problem, while Divide and conquer
reduces the problem into several sub problems
 Application

• Insertion sort is best when we want to sort

online data
• Incremental Approach
• Stable sort
 Decrease and conquer

• Insertion Sort
 Improvement
• Use binary search to reduce the number of comparisons in
normal insertion sort.
• Binary Insertion Sort uses binary search to find the proper
location to insert the selected item at each iteration.
• In normal insertion, sorting takes O(i) (at ith iteration) in
worst case.
• We can reduce it to O(log i) by using binary search.
• The algorithm, as a whole, still has a running worst case
running time of O(n2) because of the series of swaps required
for each insertion.
• How to implement Insertion Sort for Linked List?
 Try it out later !!!
• Do an insertion sort on the following example: "CHRIST". Write down the
essence of insertion sort and also a proper pseudocode.
 Topological sorting

• Topological sorting for Directed Acyclic Graph

(DAG) is a linear ordering of vertices such that
for every directed edge uv, vertex u comes
before v in the ordering.

• Topological Sorting for a graph is not possible if

the graph is not a DAG.
 Applications
• A common application of topological sorting is
in scheduling a sequence of jobs. Work flow
management, project management
• Better and faster web application
• Build systems – Libraries
• ordering of formula cell evaluation when we
computing formula values in spreadsheets,
Label every vertex by its indegree
Eliminate 2
Eliminate 5
Eliminate 3
Eliminate 6, 7,8
Try it later !!!
Try it later !!!
Binary search – recursive