NP-Complete Problems

Khaled W. Mahmoud

1
In This Chapter
 In this chapter we are concerned with problems
◦ whose complexity may be described by exponential function,
◦ problems for which the best known algorithms would require
many years or centuries of computer time for large inputs.

2
Definitions (1)
 Decision problem is one whose output is always yes or no.
 Optimization problem is the problem of finding the best
solution from all feasible solutions
 Example:
 TSP and Hamiltonian Cycle
 Subset and set Partition
 Graph coloring
 Bin Packing
 SAT

3
…
 Traveling salesman Problem, (TSP):
◦ The optimization problem can be stated as follows: Given a
complete graph G and the costs associated with each edge in
G, find a simple circuit C that visits each node in G exactly
once such that the total cost of the edges in C is minimum.
◦ Its decision problem can be stated as follows: Given a
complete graph G and the costs associated with each edge in
G, as well as a target T, does G contain a simple circuit C that
visits each node in G exactly once such that the total cost of the
edges in C is less than or equal to T?
◦ Both cases are exponential problems

4
…
 Hamiltonian Cycle
◦ Decision problem +exponential
◦ Problem of determining whether a Hamiltonian cycle (a cycle  in an
undirected or directed graph that visits each vertex exactly once) exists in a
given graph
◦ The input Graph is not necessary complete as TSP

5
…
 Subset Sum (Simplified version of 01-Knapsack):
◦ The optimization problem can be stated as follows: Given
a set S of integers and an integer k, find a minimum subset of
S whose elements sum up to exactly k.
◦ Its decision problem can be stated as follows: Given a set S
of integers and an integer k, does S contain a subset whose
elements sum up to exactly k? Example:
 S={1,16,64,256,1040,1041,1093,1284,1344} and C=3754
 R={1,16,64,256,1040,1093,1284} is a solution
◦ Both cases are exponential problems

6
…
 Set Partition Problem:
◦ Decision problem +exponential
◦ Given a set of integers (S), determine if S can be partition into
2 subsets with equal sum
◦ Example:
 {2, 3, 4, 7}=> No
 {2, 3, 4, 1}=> Yes

7
…
 Graph coloring:
◦ Its optimization problem can be stated as follows: Given a
graph G, find minimum number of colors such that the
adjacent vertices are not assigned the same color.
◦ Its decision problem can be stated as follows: Given a graph
G and an integer K, can the vertices of G be properly colored
in K or fewer colors such that the adjacent vertices are not
assigned the same color.
 Both cases are exponential problems

8
…
 Bin Packing:
◦ Its optimization problem can be stated as follows: Given an
unlimited number of bins each of capacity one, and n objects
with sizes s1,….,sn where 0<si<=1, find minimum number of
bins used to pack all objects.
◦ its decision problem can be stated as follows: Given an
unlimited number of bins each of capacity one, and n objects
with sizes s1,….,sn where 0<si<=1 and integer k, do the objects
fit in k bins.
◦ Both cases are exponential problems
◦ Optimal Solution: considering all ways to partition S into n or
fewer subsets; there are more than (n/2)n/2 possible partitions

9
…
 Formula satisfiability SAT
◦ Decision problem: is the problem of determining if there exist
a variable  assignments that satisfies a given Boolean formula.
◦ exponential problems
◦ For example:
◦ Given a Boolean formula: x1˄ x2 ˅ x3, What are the values of
x1,x2 and x3 that make the formula true.
◦ We have 8 options = 2n (truth table)

10
…
 A motivation of using decision problems is that if
solving the decision problem can be proven to be
difficult, then solving the corresponding optimization
problem is definitely no easier.

 Can we use TSPd to solve TSPop?

◦ Yes, by assign T with lower bound value. If the output of TSP d
is no, increase the lower bound and try again.
◦ Or start with upper bound and then decrease the bound.

11
Definitions(2)
 A non-deterministic algorithm: has three steps:
◦ “guessing” phase: “proposed solution”
◦ “verifying” phase: a deterministic algorithm takes the input of
the problem and the proposed solution s, and return value true
or false
◦ output step: If the verifying phase returned true, the algorithm
outputs yes. Otherwise, there is no output.

12
…
 Example: Graph Coloring problems (decision version)
◦ Given the following graph of 5 vertices, can it be 4-colored?

S Output Reason
RGRGB false both v2&v5 are green and adjacent.
RGRB false not all vertices are colored
RBYGO false too many colors are used
RGRBY true a valid 4-colring
R%*<G@ false Bad syntax
1

2 4

3 5
13
Definitions (3)
 P Class: decision problems that can be solved in
polynomial time.
◦ All the problems described at the beginning of this chapter are
not in P.
 NP Class: NP stands for non-deterministic polynomial
◦ decision problems
◦ A proposed solution can be checked quickly (in polynomial
time) to see if it really is a solution.

14
…
 All previous problems (decision version) are in NP.
 All previous problems (optimization version) are

not in NP.
 For Example:

◦ TSP (opt): if you suggest a circle that has a minimum cost, no

way to check the correctness of this suggestion but try all
combinations.
◦ TSP(dec): if you suggest a circle that has a cost, then you need
to check that the circle is simple and the summation of all
weights is less than or equal T.

15
Relationship Between NP and P
Theorem: P  NP
 Since P-Problem can be solved in polynomial time then

any proposed solution can be verified in polynomial

time i.e. P  NP
 It is not known whether P=NP or whether P is a proper
subset of NP
o It is believed NP is much larger than P
o But no problem in NP has been proved as not in P: No known
deterministic algorithms that are polynomially bounded for
many problems in NP
o So, “does P = NP?” is still an open question!

16
Definitions (4)
 NP-HARD: A problem Q is said to be NP-hard if for
all problems P in NP, there is a polynomial time
reduction from P to Q (P≤ pQ).
 What is a reduction?
◦ Reduction of a problem P to another problem Q is an algorithm
that transforms any instance I' of P to an instance I of Q, such
that the output of P on I' is yes if and only the output of Q on I
is also yes.
 It should be emphasized that the transformation must
be performed in polynomial time.

17
…
To make this idea clear see the next two examples:
 Example 1:

◦ If we have an algorithm to solve quadratic equation

(ax2+bx+c=0), can we use it to solve linear equation (bx+c=0)?
◦ Linear ≤ p Quadratic

b
a=0
T
b Algorithm for Quadratic Eq.
c
c

18
…
Example 2:
 Let the problem P be : given a sequence of Boolean

values, does at least one of them have the value true?

 Let Q be: given a sequence of integers, is the maximum

of the integers positive?

 Assume that we already have Q, Can we use Q to find a

solution to P.
◦ P≤pQ

19
…
 Let the transformation T be defined as :
 T(x ,x ,….x ) = (y , y ,….y )
1 2 n 1 2 n
 Where yi = 1 if xi = true and yi = 0 if xi = false

Input Input
for p for Q

T Algorithm for Q
Yes or no
answer

20
…
P ≤ pQ
 P is no Hard to solve than Q
 if there is a polynomial time algorithm for Q, then there is

also a polynomial time algorithm for P.

 By contrapositive: if there is no polynomial time algorithm

for P then there is no polynomial time algorithm for Q.

 if there is a polynomial time algorithm for P then we cant

conclude that q has a polynomial time algorithm

 NP-hard does not mean “in NP and hard” but “at least as

hard as any problem in NP”

21
…
 Return back to the definition of NP-HARD:
◦ NP-HARD: A problem Q is said to be NP-hard if for all
problems P in NP, there is a polynomial time reduction from P
to Q (P≤ pQ).
◦ We can conclude:
 If any one NP-HARD problem can be solved in polynomial time
(in P), then every problem in NP can also be solved similarly (This
is why many believe P≠NP)

22
Difinitions (5)
NP-Complete Problems:
 The intersection between NP and NP-HARD is precisely the

set of NP-complete problems.

 The hardest problems in NP
 If any one NP-complete problem can be solved in

polynomial time (in P), then every problem in NP can also

be solved similarly (This is why many believe P≠NP)

23
How to Prove a Problem S is NP-Complete?
 Show S is in NP
 Instead of proving that all NP are reducible to S

◦ Select a known NP-complete problem R.

 SAT was first known NP-complete problem
◦ Since R is NP-complete, all problems in NP are reducible to R
◦ Show how R can be poly. reducible to S
 Then all problems in NP can be poly. reducible to S
(because polynomial reduction is transitive)
 Therefore S is NP-complete

24
…
 Suppose you have a problem to solve
◦ Your colleagues have spent a lot of time to solve it exactly but
in vain
◦ See whether you can prove that it is NP-complete
◦ If yes, then spend your time developing an approximation
(heuristic) algorithm
 Heuristic : The use of fast –polynomial bounded-
algorithms that are not guaranteed to give the best
solution but will give one that is close to the optimal.

25
Reduction Example (1)
 Example 1:Assume that HAM is known to be NPC,
prove that TSPd is NPC.
◦ First: Argue that TSPd is NP
◦ Second : prove that HAM ≤p TSPd
 transforms any instance I' of HAM to an instance I of TSPd, such
that the output of HAM on I' is yes if and only the output of TSP d
on I is also yes.

26
…
 Given Ham-Cycle input graph G=(V,E) „
 Create complete, weighted graph G’=(V,E’) „
◦ Each edge in E is in E’ with weight of 1 „
◦ Each edge not in E is in E’ with weight of 2 „
 Solve TSP with T= |V| „ If such a TSP tour exists in G’, the
same tour is a Hamiltonian Cycle in G

27
…

28
TSP: Heurestic Alg
 The Nearest-Neighbor Strategy
◦ W(n) in O(n2); where n is the number of vertices
◦ The output for the this graph: 1,3,2,4,1 with total of 85 (Not
optimal)
 Shortest-Link Strategy
◦ Considers edges in order of length (cost/weight), from shortest to
longest.
◦ Add vw to C if vw does not make a cycle with edges in C and vw
would not be the third edge in C incident on v or w
◦ When n-1 edges have been added, connect the endpoints of the
path to form a cycle.
◦ Running time is about: O(E log E) for a graph with edges.

29
Reduction Example (2)
 Example 2: Assume that setPartition is known to be
NPC, prove that subsetd is NPC.
◦ First: Argue that subsetd is NP
◦ Second: prove that setPartition ≤p subsetd
 The input for any instance of setPartition is just S
 The input for subsetd is S, and K.
 How we can transform the input of setPartition to input of subset
problem in order to get a solution to setPartition problem.
◦ Solution: let K= summation of S/2.
◦ Example: S={2,3,4,9}, K=9

30
Reduction Example (3)
 Example 3: Assume that subsetd is known to be NPC,
prove that setPartition is NPC.
◦ First: Argue that setPartition is NP
◦ Second: prove that subsetd ≤p setPartition
 The input for subsetd is S, and K.
 The input for any instance of setPartition is just S’
 How we can transform the input of subset to input of setPartition
problem in order to get a solution to subset problem.
◦ Solution: add a new element to S’ = K*2 -sum of S
◦ Example:
 S={2,3,4,8}, K=12
 S’= {2,3,4,8,7}

31
Bin Packing: Heuristic Alg
 First fit decreasing strategy (FFD)
◦ heuristic: Places an object in the first bin in which it fits.
W(n) in Ө(n2)
◦ Example:
 Given objects:0.4, 0.2, 0.4, 0.8, 0.2, 0.2, 0.5, 0.3
 decreasing order: 0.8, 0.5, 0.4, 0.4, 0.3, 0.2, 0.2, 0.2
…
Other heuristic:
1- Simple first-fit strategy:
◦ Take the objects in any order
◦ Place each object in the first bin in which it fits
2- best fit strategy
◦ Instead of placing an object in the first bin in which it fits,
we could put it in the fullest of those bins in which it fits.
◦ Intuitively, this “best-fit” strategy might be expected to
give better solutions, since, it attempts to fill bins as much
as possible. However, in practice, it works only about as
well as the FFD strategy.
Note: Input Size and P
 Sometimes a problem seems to be in P but really isn’t.
 For Example:

◦ 01 knapsack: O(Items No*Cap);

◦ IsPrime(n)? Just loop and do i mod n: O(n).

34
Note: Input Size and P
 Note:
◦ Here “n” or “Cap” is not the count or number of data items.
◦ There’s just one input item. But “n” is a value with a size that
affects the execution time.
◦ The size s of n (or cap) is roughly log10(n) (ex. n= 150
s=2.17=floor(logbn)+1)
◦ If the input size s is log n and the running time of an algorithm is
n, then the running time of the algorithm is an exponential
function of the input size s (n=10s = 10 log n) This is really an
exponential!
 Be careful if “n” is not a count of data items but a
value

35