Brute Force and Exhaustive Search

Chapter 6
Science is as far removed from brute force as this
sword from a crowbar.
—Edward Lytton (1803–1873), Leila, Book II, Chapter I

Doing a thing well is often a waste of time.

—Robert Byrne, a master pool and billiards player and a writer

Brute Force design technique 2

 Outline

• Brute Force as a Problem Solving Technique

• Different Sorting Algorithm
– Selection Sort;
– Bubble sort;
– Insertion sort;
• Brute-Force String Matching ;
• Closest-Pair Brute-Force Algorithm;
• Exhaustive Search

Brute Force design technique 3

 Table of Contents

• 6.1 Data Sorting

• 6.2 String Matching
• 6.3 Closest-Pair Problem
• 6.4 Exhaustive Search

Brute Force design technique 4

 Introduction

• The Brute Force technique is the simplest technique for designing

algorithms

“A straightforward approach, usually based directly on the problem’s

statement and definitions of the concepts involved”

• The “force” implied by the strategy’s definition is that of a computer and

not that of one’s intellect.

• “Just do it!” would be another way to describe the prescription of the

brute-force approach.

• And often, the brute-force strategy is indeed the one that is easiest to
apply.

Brute Force design technique 5

 Why it is an important technique

• Brute force is applicable to a very wide variety of problems.

• For some important problems—e.g., sorting, searching, matrix multiplication,

string matching—the brute-force approach yields reasonable algorithms of at
least some practical value with no limitation on instance size

• The expense of designing a more efficient algorithm may be unjustifiable if

only a few instances of a problem need to be solved and a brute-force
algorithm can solve those instances with acceptable speed.

• A brute-force algorithm can still be useful for solving small-size instances of a

problem.

• A brute-force algorithm can serve an important theoretical or educational

purpose as a yardstick with which to judge more efficient alternatives for
solving a problem.

Brute Force design technique 6

 Examples of BF

1. Computing an (a > 0, n a nonnegative integer)

2. Computing n!

3. Multiplying two matrices

4. Searching for a key of a given value in a list

Brute Force design technique 7

 6.1 Problem of Sorting

 Selection sort
 Bubble sort
 Insertion sort

Brute Force design technique 8

 Introduction

Fundamental problem in algorithms for applications such as:

 Searching,
 Closest pair problem,
 Element uniqueness,
 Frequency distribution,
 Selection,
 …

Brute Force design technique 9

 Problem Statement

• Denote by a[i], i = 0, 1, ..., n - 1, the elements of a table with n elements

• Ordering is the process of repositioning of the values of the elements in

such a way that they meet one of the following conditions:

 a[0] < a[2] < ... < a[n – 2] < a[n – 1], Ascending order
 a[0] ≤ a[2] ≤ ... ≤ a[n – 2] ≤ a[n – 1], Non-Decreasing order
 a[0] > a[2] > ... > a[n – 2] > a[n – 1], Decreasing order
 a[0] ≥ a[2] ≥ ... ≥ a[n – 2] ≥ a[n – 1], Non Ascending order

Brute Force design technique 10

 Sortable Data Type

• Numbers: the sort is called numerical and the order is based on the
value of the numbers.
• Letters: the sort is called alphabetical and the order is based on the
alphabetical order.
• Letters + Numbers + other sings, the sort is called lexicographic,
where each character is given a numeric value (eg, according to
ASCII code)

Brute Force design technique 11

 Classification of Algorithms

• Structure (sorting tools)

• Stability (maintaining the position of the elements)
• In-place sorting (no need for additional memory)

Brute Force design technique 12

 The structure of sorting algorithms

• Algorithms that rely on comparison sort and have almost linear and
quadratic complexity

• Algorithms that rely on specific data qualities and have linear

complexities.

Brute Force design technique 13

 The Stability of Algorithm

• It makes sense when the data are organized in corresponding tables

• Save the relative position of the elements in the corresponding
tables when there are elements with the same values in the key
table

Corresponding table

Key Table

Brute Force design technique 14

 In-place Sorting

• It does not require extra memory, except, possibly for a few memory
units

Brute Force design technique 15

 6.1.1 Selection Sort Algorithm

• One of the simplest sorting algorithms

• Applies to all data types

Brute Force design technique 16

 Selection Sort Algorithm

• The Idea: Find the smallest!

2, 4, 5,   , 7, 8, 6, 9

a0 ≤ a1≤ ≤ai-1 ∥ ai,  amin, an-1

(i) Elements in the final position (n-i) elements to be sorted

Brute Force design technique 17

 SS Algorithm’s Code

SelectionSort (n, a) {
1. for i ← 0 to n – 2 do
2. { min ← i; // min, the index of the element with the smallest value in step i
3. for j ← i + 1 to n – 1 do
4. if (a[j] ≤ a[min])
5. then min ← j; // update the index
6. swap (a[i], a[min]); // exchange the value
7. }
8. return
}

Brute Force design technique 18

 Dynamics of an example for SS

Table elements
a[0] a[1] a[2] a[3] a[4] a[5] a[6]
Initial data 89 45 68 90 29 34 17
After 1st step (i = 0) 17 45 68 90 29 34 89
After 2nd step (i = 1) 17 29 68 90 45 34 89
After 3rd step (i = 2) 17 29 34 90 45 68 89
After 4th step (i = 3) 17 29 34 45 90 68 89
After 5th step (i = 4) 17 29 34 45 68 90 89
After 6th step (i = 5) 17 29 34 45 68 89 90

Brute Force design technique 19

 Time efficiency of SS algorithm

• Evaluation of the parameter, data size: n

• Most costly operation: Comparison a[j] < a[min]
• Number of comparisons:

Brute Force design technique 20

 Some features of the SS algorithm

• Its execution time depends very little on the degree of sorting of the initial
data

• Number of exchanges (swaping) is part of Θ(n)

• The algorithm does not require additional memory (except for some
variables)

• The algorithm is not stable

Brute Force design technique 21

 6.1.2 Bubble Sort sorting algorithm

• Simple algorithm
• Applies to all data types
• The “worst”

Brute Force design technique 22

 Bubble Sort Sorting Algorithm

• The idea: drag the biggest!

2, 5, 4,   , 6, 7, 8, 9

?
• 𝑎0 , ⋯ , 𝑎𝑗 𝑎𝑗+1 , ⋯ , 𝑎𝑛−𝑖−1 ∥ 𝑎𝑛−𝑖 ≤ ⋯ ≤ 𝑎𝑛−1

(i) Elements in the final position

(n-i) elements to be sorted

Brute Force design technique 23

 BS Algorithm Code

BubbleSort (n, a) {
1. for i ← 0 to n – 2 do
2. for j ← 0 to n – 2 – i do
3. if (a[j] > a[j + 1])
4. then swap(a[j], a[j + 1]); // exchanges values
5. return
}

Brute Force design technique 24

Partial dynamics of a sorting example with BS

?
89 45 ? 68 90 29 34 17
45 89 68 90 29 34 17
? ?
45 68 89 90 29 34 17
?
45 68 89 29 90 34 17
?
45 68 89 29 34 90 17
45 68 89 29 34 17 90
? ? ?
45 68 89 29 34 17 90
?
45 68 29 89 34 ? 17 90
45 68 29 34 89 17 90
45 68 29 34 17 89 90
etj

Brute Force design technique 25

 Time efficiency of BS algorithm

• The evaluation parameter, data size: n

• Most costly operation: Comparison a[j] > a[j + 1]

• Number of comparisons:

Brute Force design technique 26

 Some features of BS algorithm

• The calculation time depends on the organization of the initial data.

• The algorithm does not make exchanges when the data is sorted in
ascending order
• When the data are sorted in descending order the number of
exchanges is n(n-1) / 2
• Despite possible improvements, the algorithm remains in Θ(n2)
class.
• Among the elementary sorting algorithms it is the worst

• The Bubble sort algorithm does not use additional memory

• The algorithm is stable.

Brute Force design technique 27

 6.1.3 Insertion Sort Algorithm

• The most "attractive" algorithm based on the qualities

• Applies to all data types

Brute Force design technique 28

 Insertion Sort Algorithm

• The idea: Put in the right place!

1, 4, 5,   , 2, 8, 6, 9

smaller or equal than the A[i] value greater than the A[i] value

Brute Force design technique 29

 IS algorithm code

InsertionSort(n, a){
for i ← 1 to n – 1 do {
v ← a[i]; // give the next value
j ← i – 1;
while (j ≥ 0 and a[j] > v) do { // Find the first smaller element
a[j + 1] ← a[j];
j ← j – 1;
}
a[j + 1] ← v
}
return
}

Brute Force design technique 30

 Dynamic of a sorting example with IS

Table elements
a[0] a[1] a[2] a[3] a[4] a[5] a[6]
Initial data 89 45 68 90 29 34 17
After 1st step 45 89 68 90 29 34 17
After 2nd step 45 68 89 90 29 34 17
After 3rd step 45 68 89 90 29 34 17
After 4th step 29 45 68 89 90 34 17
After 5th step 29 34 45 68 89 90 17
After 6th step 17 29 34 45 68 89 90

Brute Force design technique 31

 Time efficiency of IS algorithm

• The evaluation parameter, data size: n

• Most costly action: Comparison a[j] > v

• Number of comparisons:

Brute Force design technique 32

 Some features of IS algorithm

• When the data are sorted K(n) = n - 1 ∈ Θ(n)

• The Insertion Sort algorithm does not use additional memory

• The algorithm is stable.

Brute Force design technique 33

6.2 Brute Force String Matching

Brute Force design technique 34

 Introduction

String matching problem can be used :

 in word processing by text Editors (for example, Word),
 in language recognition,
 in bioinformatics (bio-computing),
 in image processing,
 when searching the Internet through search engines (Google, Yahoo,
etc.),
 while searching in an electronic dictionary, etc.

Brute Force design technique 35

 Concepts

• We will use the term pattern to refer to a series of signs ine a

given alphabet, Σ.
• Depending on the problem , a alphabet may consist of :
the 26 lowercase letters of the English alphabet,
128 marks of ASCII standard of signs representation,
Whole number from 0 to 255 (representing all values of possible
combinations in one byte),
etc.

• Usually the required pattern is too short or short (maybe 5-20

characters) while the text where required is much longer
(thousands or more characters).

• Over 80 efficient search algorithms have been developed.

Brute Force design technique 36

 Problem Statement

• The Data :

Given a string formed by n characters called text and a string

formed by m characters (m << n) called a patter or sample.

• The Result :

Find in which position of the text the pattern begins, if any.

Brute Force design technique 37

 The intermediate situation

We denote with :

 {p1,...,pm} pattern and

 {t1,…,tn} text,

The algorithm seek to find the i index, which is the leftmost index of the character of
the first match pattern in the text, such that ti = p1, …, ti + j-1 = pj, …, ti + m - 1 = pm :

Brute Force design technique 38

 String Matching Algorithm

StringMatchingBF (n, t, m, p) {
for i ← 1 to n – m + 1 do {
j ← 1;
while (j ≤ m ) and t[i + j – 1] = p[j]) do
j ← j + 1;
if j > m
then return i;
}
return –1
}

Brute Force design technique 39

 A string matching diagram

Position
Text

Pattern

shifts

Brute Force design technique 40

 Algorithm efficiency

• Worst case when the pattern is not found or is found in the last
search by n - m + 1 attempt for effectiveness in Θ(nm) class.

• In the average case the effectiveness has been linear, that is Θ(n + m)

Brute Force design technique 41

6.3 Closest – Pair Problem

Brute Force design technique 42

 Introduction

• The Closest Pair Problem was first considered in the field of

computational geometry in the early 1970s by M.I. Shamos and D. Hoey.

• This problem has found applications in areas such as:

 Graphics,
 Computer vision,
 Geographic Information System - GIS,
 in aviation and maritime navigation for traffic control, etc.

• We will only consider the plan case, as stated in the problem in 2-D space .
• The problem can be generalized to higher spaces.

Brute Force design technique 43

 Problem Statement

• Find the distance between the two nearest points in a set of points
in the plan.
• Points are specified by their Cartesian coordinates (x, y).
• The distance between two points Pi (xi, yi) and Pj (xj, yj) is the
Euclidean distance calculated according to the formula:

Brute Force design technique 44

 The Algorithm Idea

• The Data :
The amount of points n, and the abscissas x(1:n) and the ordinates
y(1:n)
• Result:
d, the distance of the nearest pair of points

• The algorithm idea

Find the smallest value in a set with n(n – 1)/2 distances

• The square root calculation is a problem, so the distance square is

used in the calculation

Brute Force design technique 45

 Algorithm

closestPair(n, x , y) {
1. dmin ← + ∞ ; // The greater value in computer
2. for i ← 1 to n - 1 do
3. for j ← i + 1 to n do
4. d ←min(d, sqr((x[i] – x[j])2 + (y[i] – y[j])2));

5. return d

Brute Force design technique 46

 Efficiency

• The algorithm have Θ(n2) time efficiency

Brute Force design technique 47

6.4 Exhaustive Search

Brute Force design technique 48

 Introduction

• Problem that search for an element with a special property

• Problem usually among combinatorial objects such as

permutations, combinations, or subsets of a set.

• Most are optimization problems

• Through the exhaustive search are explore all the solution and
between them the optimal is selected.

Brute Force design technique 49

 Exhaustive Search
Method:

– generate a list of all potential solutions to the problem

– evaluate potential solutions one by one, disqualifying

infeasible ones and, for an optimization problem, keeping
track of the best one found so far

– when search ends, announce the solution(s) found

Brute Force design technique 50

 6.4.1 Example 1: Traveling Salesman Problem

• Traveling Salesman Problem – TSP) has a history of over 150 years

• Simple formulation but an expensive solution

• Problem Statement
Given n cities with known distances between each pair, find the shortest
tour (path ) that passes through all the cities exactly once before returning
to the starting city

• Alternatively: Find shortest Hamiltonian circuit in a weighted connected graph

• Hamiltonian circuit: a sequence of n + 1 adjacent vertices where the first

vertex of the sequence is the same as the last one and all the other n – 1
vertices are distinct.

Brute Force design technique 51

 Different formulations of the problem

• The problem can be modeled in different ways:

 As a weighted graph, where the vertices of the graph

represent cities and the weight of the ribs represents
distances.
 As the problem of finding the shortest Hamiltonian circuit
of graphite in electronics

Brute Force design technique 52

 Small Case of the problem
Tour Cost
a→b→c→d→a 2+3+7+5 = 17 a
2
b
a→b→d→c→a 2+4+7+8 = 21 5 3
8 4
a→c→b→d→a 8+3+4+5 = 20
a→c→d→b→a 8+7+4+2 = 21 c
7
d

a→d→b→c→a 5+4+3+8 = 20
a→d→c→b→a 5+7+3+2 = 17

Efficiency: O(n!)
Brute Force design technique 53
 Time Efficiency

• By keeping one city fixed, we produce all the permutations

formed by other n - 1 cities;

• We calculate the length of each path and find the shortest of

them.

• The exhaustive search algorithm is part of class O ((n – 1)!).

Brute Force design technique 54

6.4.2 Knapsack Problem

Brute Force design technique 55

 Problem Statement

• Knapsack problem is a well-known problem in algorithm

• Given n items:
– weights: w1 w2 … wn
– values: v1 v2 … vn
– a knapsack of capacity W
• Find most valuable subset of the items that fit into the
knapsack

• Such problems encountered in practice in the loading of containers,

shipping and cargo transport aircraft

Brute Force design technique 56

 Solution and efficiency

• For each possible subset of n items is calculated the

weight of the combinations that do not exceed the
capacity of the bag and the largest of them is found.

• The number of subsets formed by a set of n elements is

2n - 1,

• Exhaustive research produces an algorithm of class Ω

(2n) and it does not matter at all how efficient the
process of forming possible subsets is.

Brute Force design technique 57

 Simple Case
Initial Data Combinations Weight Value Coments
Articles Weight Value {1} 2 20
1 2 20 {2} 5 30
2 5 30 {3} 10 50
3 10 50 {4} 5 10
4 5 10 {1, 2} 7 50
{1,3} 12 70
{1,4} 7 30
{2,3} 15 80 Maximal value
{2,4} 10 40
{3,4} 15 60
{1,2,3} 17 Not acceptable
{1,2,4} 12 60
{1,3,4} 17 Not acceptable
{2,3,4} 20 Not acceptable
{1,2,3,4} 22 Not acceptable

Brute Force design technique 58

 Discussions
• Exhaustive search results in algorithms that are extremely inefficient for
any initial data.

• These are solid NP problems.

• No polynomial class algorithm for NP problems is known.

• But are there possibilities for such algorithms?

• Methods like backtracking and branch-and-bound allow some but not all
cases of these problems to be solved in less time than exponential.

• Also, for their solution, algorithms that provide approximate solutions can
be used as alternatives.

Brute Force design technique 59

6.4.3 The Assignment Problem

Brute Force design technique 60

 Problem Statement

• A problem that can be solved by exhaustive search is that of assignment

problem.

• This means that each person is assigned one and only one task and each
task is performed by one and only one person.

• Meanwhile, it is known the cost C[i, j] when person i is assigned task j for
each couple
(i, j) = 1, 2,…, n.

• The problem lies in finding such a distribution of persons and tasks so that
the total cost is minimal.

Brute Force design technique 61

 Assignment Problem by Exhaustive Search
• We can describe the possible solutions of the
assignment problem as a series of n-elements (n-tuple)
𝑗1 , ⋯ , 𝑗𝑛 , in which its component, for i = 1,… , n,
indicates the column of the selected element in its row
(i.e., the job number assigned to its person).

• For example, for the cost matrix given above, the

se𝑟𝑖𝑒 2, 3,4, 1 indicate a possible assignment of
person_1 for Job_2, person_2 for Job_3, person_3 for
Job_4, and person_4 for Job_1.

• Proportional solution time with n!

Brute Force design technique 62

Algorithmic Plan: Generate all legitimate assignments, compute
their costs, and select the cheapest one.

Job 0 Job 1 Job 2 Job 3

Person 0
Person 1
9
6
2
4
7
3
8
7
• How many assignments are there?
Person 2 5 8 1 8 • Permutation of n jobs
Person 3 7 6 9 4

9 2 7 8 • Efficiency? O(n!)
6 4 3 7
5 8 1 8
7 6 9 4

Assignment (col.#s) Total Cost

1, 2, 3, 4 9+4+1+4=18
1, 2, 4, 3 9+4+8+9=30
1, 3, 2, 4 9+3+8+4=24
1, 3, 4, 2 9+3+8+6=26
1, 4, 2, 3 9+7+8+9=33
1, 4, 3, 2 9+7+1+6=23
etc.
Brute Force design technique 63
 But the solution was found!

• Exhaustive research is not practical for all dimensions of the

problem.

• An effective algorithm for this problem has been discovered,

called the Hungarian method in honor of the Hungarian
mathematicians König and Egerváry.

• This is a good news: the fact that the problem area is growing
exponentially (or faster) does not necessarily mean that there
will be no effective algorithms to solve it.

• Such cases are rather an exception to the rule.

Brute Force design technique 64

Final Comments on Exhaustive Search
• Exhaustive-search algorithms run in a realistic
amount of time only on very small instances

• In some cases, there are much better

alternatives!
– Euler circuits
– shortest paths
– minimum spanning tree
– assignment problem

• In many cases, exhaustive search or its variation

is the only known way to get exact solution

Brute Force design technique 65

 6.5 Summary
• Brute force is a straightforward approach to solving a problem, usually
directly based on the problem statement and definitions of the concepts
involved.

• The principal strengths of the brute-force approach are wide applicability

and simplicity; its principal weakness is the subpar efficiency of most
brute-force algorithms.

• A first application of the brute-force approach often results in an

algorithm that can be improved with a modest amount of effort.

• Exhaustive search is a brute-force approach to combinatorial problems. It

suggests generating each and every combinatorial object of the problem,
selecting those of them that satisfy all the constraints, and then finding a
desired object.

Brute Force design technique 66

 Summary
• Insertion sort is a direct application of the decrease-(by one)-and-conquer
technique to the sorting problem. It is a (n2) algorithm both in the worst
and average cases, but it is about twice as fast on average than in the
worst case. The algorithm’s notable advantage is a good performance on
almost-sorted arrays.

• The traveling salesman problem, the knapsack problem, and the

assignment problem are typical examples of problems that can be solved,
at least theoretically, by exhaustive-search algorithms.

• Exhaustive search is impractical for all but very small instances of

problems it can be applied to.

Brute Force design technique 67

Thank you
Questions?