Complexity

1 / 18
 Complexity

Introduction

Given a problem P and an instance of P (specific input for a numeric case of

P), we define
Size of problem P: number of bits needed to represent the input data of
an instance of P.
Complexity of an Algorithm for P: measure of the time, as a function of
the instance size, it takes, in the worst case, to solve any instance of P
Computational Complexity of problem P: complexity of the best algorithm
for solving any instance of P in the worst case

2 / 18
 Complexity

Size of a Problem
Size of a Problem P: in most cases, it is equivalent to the number of input
values that define an instance of P.
Example: the Knapsack Problem has input data:
n (number of objects),
W (knapsack capacity),
(pj ) profit vector with n values,
(wj ) weight vector with n values.
The global number of values needed to represent the input data of an
instance of the Knapsack Problem is given by (2n + 2).
To represent a value x, dlog xe bits are needed.
The number of bits needed to represent the input data
Pnof an instance of
the Knapsack Problem is given by dlog ne, dlog W e, j=1 dlog pj e,
Pn
j=1 dlog wj e.
We can assume to use, for all values, the number of bits required by the
largest value: (2n + 2) ∗ dlog (max{n, W , p1 , . . . , pn , w1 , . . . , wn })e
For sufficiently large n, the size of the problem is proportional to n.
3 / 18
 Complexity

Computational Complexity of Feasibility and

Optimization Problems

Feasibility Problem (also called Recognition Problem): determine if at

least a feasible solution exists for the considered problem (the answer is
“yes” or “no”).
Optimization Problem: determine a feasible solution that maximizes (or
minimizes) the objective function of the considered problem.

4 / 18
 Complexity

Computational Complexity

Given a problem P, determine the computing time (number of elementary

operations), expressed as a function of the size of P, to find the solution
of P in the worst case
For example: polynomial function of the input size (“easy” problem) and
exponential function of the input size (“difficult” problem)
For most optimization problems no polynomial-time algorithm is known
The Complexity theory provides a rigorous treatment of these issues

5 / 18
 Complexity

Feasibility versus Optimization Problems

The theory of the Computational Complexity of the problems has been

developed for Feasibility Problems, but it can be applied to Optimization
Problems as well.
Example: the Knapsack Problem
Feasibility version: Given an instance of the Knapsack Problem, determine if
there exists at least one feasible solution whose profit is not smaller than a
given value K .
Optimization version: Given an instance of the Knapsack Problem,
determine the solution with maximum profit.

6 / 18
 Complexity

Feasibility versus Optimization Problems

From the point of view of polynomial-time solvability, the two versions
have the same complexity:
Consider a problem with feasible set F , and a profit function ϕ : F → R.
The optimization version is to find the solution x ∈ F that maximizes ϕ(x),
The feasibility version, given a value K , is to decide whether maxx∈F ϕ(x) is
larger or equal to K (answer “yes” or “no”).
If the profit ϕ(x) of any solution x can be computed in polynomial time, then
a polynomial time algorithm for the optimization version leads to a
polynomial time algorithm for the feasibility version.
It is also true that a polynomial time algorithm for the feasibility version leads
to a polynomial time algorithm for the optimization version:
let U be an upper bound on the optimal profit. We can perform binary search as
follows: solve the feasibility problem with K = 12 U. If the answer “yes”, then next
execution with K = 43 U otherwise with K = 41 U, and so on.
The number of executions is O(log U) that is polynomial if log U is bounded by a
polynomial function of the instance size. If we have a polynomial time feasibility
algorithm, then we obtain a polynomial time algorithm for the optimization
version.
We will focus on the feasibility version.
7 / 18
 Complexity

Polynomial Problems

A problem P is polynomial if there exists at least one algorithm that can

solve P, in the worst case, within a computing time that grows according
to a polynomial function of the size of P.
Examples: given an array (aj ) of n elements (j = 1, . . . , n) (size n):
1 Determine the minimum value of the array: effective algorithm: O(n) time (in
the worst case the computing time is proportional to n: “linear” in n)
2 Sort the n elements according to non-decreasing (or non-increasing) values:
effective algorithm: O(n log n) time.

8 / 18
 Complexity

Classes P and N P

Class P contains all the polynomial problems.

Class N P contains all the problems that can be solved in polynomial time
in the best case (through a “non deterministic Turing Machine”: an ideal
“lucky” algorithm that always makes the correct choice in a decision tree).

9 / 18
 Complexity

Class N P

A problem P belongs to class N P if it can be solved through a Decision

Tree such that:
1. the number of levels
2. the number of descendant nodes of each node
3. the computing time required to consider each node
are polynomial functions of the size of P.
Class P is contained in class N P

10 / 18
 Complexity

Example of a problem in class N P:

Knapsack Problem in Feasibility Version

Knapsack Problem in Feasibility Version: Given an instance of the

Knapsack Problem, determine if there exists at least one feasible solution
whose profit is not smaller than a given value K : KP-01(K )
Binary Decision Tree: at each level j (j = 1, . . . , n) consider object j and
fix the value of the decision variable xj (used to determine if object j is
inserted in the knapsack or not) to 0 or 1:
n levels;
2 descendant nodes for each node;
constant computing time for each node.
KP-01(K ) belongs to class N P

11 / 18
 Complexity

Example of a problem in class N P:

Knapsack Problem in Feasibility Version

In the best case, the algorithm requires a computing time proportional to

the number of levels of the binary decision tree: polynomial time with
respect to the size n of KP-01(K ).
In the worst case, the algorithm requires a computing time proportional to
the global number of nodes of the binary decision tree: exponential time
with respect to the size n of KP-01(K ).

12 / 18
 Complexity

Classification

There are three main classes in computational complexity:

1. Polynomial Problems: Class P;
2. N P-Complete Problems: belong to Class N P, but no polynomial
algorithm has been proposed for their solution in the worst case;
example: KP-01(K );
3. Definitely Difficult Problems: do not belong to Class N P
example: determine all the optimal solutions of KP-01. The number of such
solutions could be exponential with respect to the size n.
example: determine all the permutations of n values. The number of
permutations is n!, more than exponential with respect to the size n.

13 / 18
 Complexity

N P-Complete Problems

A problem P is N P-Complete if both conditions hold:

1. P belongs to class N P
2. There exists an N P-Complete Problem T which is transformable to P
(T ∝ P), i.e.:
for any instance t of T , it is possible to define, in polynomial time in the size
of t, an instance p of P such that, if the solution of p has been determined,
then the solution of t can be obtained in polynomial time in the size of t.

14 / 18
 Complexity

An N P-Complete Problem used for

transformation:
Partition Problem

Given
m positive values (aj ) (j = 1, . . . , m),
a positive value b
determine if there exists a subset of the m values whose sum is equal to
b.
It is a Feasibility Problem with size m
The Partition Problem is known to be N P-Complete

15 / 18
 Complexity

Proof that KP-01 is N P-Complete

KP-01(K ) belongs to class N P

The Partition Problem (PP) is transformable to KP-01(K )
(PP ∝ KP-01(K )) as follows:
Given any instance of PP defined by m, (aj ), b:
1. Define (in time O(m)) an instance (n, (pj ), (wj ), W ) of KP-01(K ):
n=m
W =b
K =b
wj = aj (j = 1, . . . , n),
pj = aj (j = 1, . . . , n).
2. Determine the solution of KP-01(K ) (answer is “yes” or “no”).
3. If the answer is “yes”, then PP has feasible solution, otherwise PP has no
feasible solution.

16 / 18
 Complexity

ILP and LP

ILP (in Feasibility version) is N P-Complete because KP-01(K ) is a

special case of ILP
LP (in Feasibility version): the Simplex Algorithm is polynomial in
“practice” (it rarely needs more than m log n iterations) but in the worst
case it is not: instances have been built for which it takes 2m/2−1
iterations. However, there exist other algorithms (ellipsoid method and
interior point method) that are polynomial.

17 / 18
 Complexity

P versus N P

If one could find a polynomial-time algorithm for a single N P-Complete

problem, then a polynomial-time algorithm would be available for all
problems in class N P, i.e., P = N P.
However, in the last 60 years nobody has been able to find such an
algorithm.
Hence most mathematicians and computer scientists believe that its
existence is unlikely.
The P versus N P question is one of the seven Millennium Problems.

18 / 18