The Complexity of Integer Programming
Chris Johnson
1
Background
1.1 Integer Programming
Given an objective function of n variables x _{1} ,
f(x _{1} ,
,
x _{n} ) =
, x _{n} of the form
n
i=1
a _{i} x _{i} + c
and a set of constraints that can be written in the form Ax ≤ b (where x is the column
vector of the variables x _{i} ,
of f (x _{1} ,
such as Dantzig’s simplex. However, if we were to also impose that each x _{i} must be an integer value, we stumble into the world of more diﬃcult problems known as integer programs [4].
, x _{n} ), the problem of searching for a minimum (or maximum)
, x _{n} ) is known as a linear program, and can be solved eﬃciently using algorithms
1.2 Computational Complexity
The relative eﬃciency of any algorithm is of general interest to any user of that algorithm; we all want to have some idea how long it will take our programs to run. Computer scientists, computational complexity theorists in particular, have devised formal ways of measuring the amount of time required by an algorithm as a function of the size of the algorithm’s input. For example, searching for a particular value in a set of sorted data can be done in O(log n) time, meaning the number of steps has an upperbound on the order of the logarithm of the number of items in the set. Though the eﬃciency of an algorithm on a “realistic” model of computation is of primary interest to most people, computer scientists are also often concerned with how eﬃciently an algorithm may execute on other types of computers. For example, computers that can per form some constant number of operations at each time step are referred to as deterministic, while those that can perform an abritrary number of operations at each step are referred to as nondeterministic. All real computers are deterministic, though nondeterministic models of computation are of interest, mainly due to the improvement in running time of algorithms with such models. The set of all decision problems (problems with “yes or no” answers) that can be solved in polynomial time on a deterministic machine is referred to as the class P. The class of decision problems that can be solved in polynomial time on a nondeterministic is known as N P. Obviously P ⊆ N P (any problem that can be solved in polynomial time on a
1
deterministic machine can be solved in polynomial time on a nondeterministic machine), though whether P ⊂ N P or P = N P is still an open question. Given that it is not yet clear if P = N P or not, complexity theorists have formulated another class of the “hardest” problems in N P known as N Pcomplete. These are problems in N P that every other problem in N P (and thus every problem in P) can be transformed into in (deterministic) polynomial time [2]. Classic examples of the such problems include the satisﬁability problem, and (the decision problem version of) the travelling salesman problem, while more modern examples include the decision problem versions of games such as minesweeper and Tetris. Of course, there are problems that are even harder than the problems in N P. Games such as chess and go, for instance, are known to reside in EX PT IME, meaning they require exponential time on a deterministic machine. There is, however, a special class of problems that are at least as hard any problem in N P. Suppose we have a problem Π that may or may not be in N P . If every problem in N P can be transformed into Π in polynomial time, then we say that Π is N P hard. It is in this sense we mean that Π is at least as hard as any problem in N P [2].
2 Why is Integer Programming N PHard?
2.1 The Satisﬁability Problem as an Integer Program
The original N P complete problem described by Stephen Cook is known as the satisﬁability problem, which asks us if there is any combination of boolean variables such that a given boolean formula is true [1]. The following conventions allow us to describe the satisﬁability problem as an integer program [4]. The value one will stand for true, and the value zero will stand for false. Binary AND is represented by multiplication, and binary OR is represented by addition. Negation of x, which we will denote x¯, is representated by 1 − x. Our boolean formula will be placed in conjuctive normal form (clauses of ORs that are ANDed together), which means that using our representation, the problem has a solution if there is a set of values such that the formula evaluates to a number that is greater than or equal to one. Furthermore, this implies that each clause must evaluate to a value that is greater than or equal to one, which we can use as our constraints. Consider the following formula from [4].
(x _{1} ∨ x _{2} ∨ x _{3} ) ∧ (x _{1} ∨ x¯ _{2} ) ∧ (x _{2} ∨ x¯ _{3} ) ∧ (x _{3} ∨ x¯ _{1} ) ∧ (x¯ _{1} ∨ x¯ _{2} ∨ x¯ _{3} )
Using the convention described above, this boolean formula becomes the following arith metic formula.
(x _{1} + x _{2} + x _{3} ) · (x _{1} + 1 − x _{2} ) · (x _{2} + 1 − x _{3} ) · (x _{3} + 1 − x _{1} ) · (1 − x _{1} + 1 − x _{2} + 1 − x _{3} )
We may now easily convert this to an integer program.
2
maximize 
y 

x _{1} + x _{2} + x _{3} 
≥ 
y 
x _{1} + 1 − x _{2} 
≥ 
1 
x _{2} + 1 − x _{3} 
≥ 
1 
x _{3} + 1 − x _{1} 
≥ 
1 
1 − x _{1} + 1 − x _{2} + 1 − x _{3} 
≥ 
1 
x _{1} , x _{2} , x _{3} 
≤ 
1 
x _{1} , x _{2} , x _{3} 
≥ 
0 
x _{1} , x _{2} , x _{3} 
∈ Z 
The associated boolean expression has a solution only if after maximization, y ≥ 1. This special case of integer programming, where our variables are limited to the values zero and one, is known as 01 programming. Obviously, as we have shown an N P complete problem can be reduce to a 01 programming problem, we have shown that 01 programming is N Phard.
2.2 Equivalence of General Integer Programming and 01 Pro gramming
We shall now show that the “general” integer programming problem (that is, where our vari ables may be any integer) is polynomially equivalent to 01 programming. By polynomially equivalent, we mean a problem of one type may be transformed into an equivalent problem of the other type in polynomial time [2]. Knowing that these two classes of problems are polynomially equivalent establishes the fact that if one class of problems is N Phard, then so is the other class. The technique we describe appears in [4]. Note that 01 programming is a special case of general integer programming (01 program ming implies integer programming), so we really only need to show that the other direction (that integer programming implies 01 programming). We will show this by describing how to rewrite an integer program as a 01 program. Suppose we have an integer program whose feasible region is bounded by M , and let l = log _{2} M . We can now rewrite each x _{j} in the original problem as a set of variables x _{j}_{,}_{i} where each x _{j}_{,}_{i} ∈ Z _{2} , as each integer has a unique binary representation. Since x _{j} ≤ M for each x _{j} we are only concerned with integers between 0 and M , which will require at most l bits. We then set each x _{j}_{,}_{i} to the ith bit of the binary representation of j, so
x _{j} =
l
i=0
x j,i 2 ^{i} .
Thus every integer program can be written as a 01 program and vice versa. Since the satisﬁability problem can be written as a 01 program, 01 programming is N Phard, but since 01 programming and integer programming are polynomially equivalent, integer programming is also N Phard.
3
3
Algorithms for Integer Programming
Given that integer programming is N P hard, how can we eﬃciently tackle integer program ming problems? There are two approaches we will now consider, both of which approach the problem the same basic way: we will solve the relaxed version of the problem (that is, the linear program with the same constraints, but allowing noninteger solutions), and continue to add constraints until we arrive at an integer solution.
3.1 The Cutting Plane Algorithm
The cutting plane algorithm works by ﬁrst ﬁnding the solution to the relaxed version of the problem. If the solution we found had noninteger solutions, then we will impose a new constraint that eliminates the previously found solution, but does not eliminate any feasible integer solutions, until we arrive at a solution with all integer components [5].
3.2 Branch and Bound
The branch and bound algorithm treats the problem by considering a enumeration tree of
possible integer solutions. Of course, this tree is likely to be too large to the consider each possible enumeration. Thus, we solve the relaxed version of the problem, each time pruning possible solutions that either infeasible or nonoptimal, based on the solution of the relaxed problem [3]. Branch and bound builds this enumeration tree by considering subregions of the feasible region (branching), and then ﬁnding appropriate upper and lower bounds for the optimal solution within a subregion (bounding). One way of doing this would be to ﬁnd the solution to the original relaxed problem, say x ^{0} . If x ^{0} is not an integer, then we break our original problem into two subproblems. In the ﬁrst subproblem we impose the constraints that each x _{i} must be less than or equal
0 i . In the second subproblem, our constraint is that each x _{i} must be greater than or
i 0 +1. We then attempt to solve the relaxed version of these problems, branching
equal to x
to x
again if need be. If while solving any of our subproblems we arrive at an integer solution, however, we know that we will not ﬁnd any better solutions to that subproblem by breaking it up into even more subproblems. We can then kill the nodes of the enumeration tree that are descendents of the current node [6].
4 Conclusion
Despite that linear programming can be solved relatively eﬃciently in many cases, it comes as a bit of a shock that problems can become considerably more diﬃcult by imposing the constraint that we only consider integer solutions. Despite this, there is no avoiding the fact that there are applications where noninteger solutions are undesirable, and so we must consider how to best tackle the problem of integer programming.
4
In this paper we have explained why integer programming is N P hard, and is thus likely to remain intractable for the foreseeable future. We have also brieﬂy discussed two algorithms that we can use for solving integer programming problems by repeatedly solving relaxed versions of our original problem. Of course, using either algorithm, we’re repeatedly solving linear programs, modifying the problem until we arrive at an integer solution. Either algorithm will eventually ﬁnd the solution to the integer program, but there is no guarantee on the amount of time it will take to do so. It is possible that we could essentially build our entire enumeration tree using branch and bound if we do not “luck out” and ﬁnd integer solutions after the ﬁrst few iterations and can kill oﬀ parts of the tree. Depending on the “shape” of our feasibility region, the cutting plane algorithm may require numerous iterations until we’ve imposed enough constraints (cut oﬀ enough of the plane) that we arrive at an integer solution.
References
[1] Stephen A. Cook. The complexity of theoremproving procedures. In Proceedings of the third annual ACM symposium on Theory of Computing, pages 151–158, 1971.
[2] Michael R. Garey and David S. Johnson. Computers and Intractibility: A Guide to the Theory of NPCompleteness, chapter 2. W. H. Freeman and Company, 1979.
[3] George L. Nemhauser and Laurence A. Wolsey. Integer and Combinatorial Optimization, chapter 14. John Wiley and Sons, 1988.
[4] Christos H. Papadimitriou and Kenneth Steiglitz.
Combinatorial Optimization: Algo
rithms and Complexity, chapter 13. Dover Publications, 1998.
[5] Christos H. Papadimitriou and Kenneth Steiglitz.
Combinatorial Optimization: Algo
rithms and Complexity, chapter 14. Dover Publications, 1998.
[6] Christos H. Papadimitriou and Kenneth Steiglitz.
Combinatorial Optimization: Algo
rithms and Complexity, chapter 18. Dover Publications, 1998.
5
Much more than documents.
Discover everything Scribd has to offer, including books and audiobooks from major publishers.
Cancel anytime.