ISYE6189 - Deterministic Optimization and

Stochastic Processes

Topic 5 – Week 5
Integer Programming
 Integer Programming

Learning Outcome
LO2: Apply some methods and the techniques
used to solve linear optimization models using
their mathematical structure
Integer Programming

- Model formulation
Integer Programming
 Introduction to Integer
Programming

• An IP in which all variables are required to be integers

is call a pure integer programming problem.
• An IP in which only some of the variables are required
to be integers is called a mixed integer programming
problem.
• An integer programming problem in which all the
variables must be 0 or 1 is called a 0-1 IP.
• The LP obtained by omitting all integer or 0-1
constraints on variables is called LP relaxation of the
IP.
 Formulating Integer Programming
Problems

• Practical solutions can be formulated as IPs.

• The basics of formulating an IP model
 Example 1: Capital
Budgeting IP

• Stockco is considering four investments. Investment 1

will yield a net present value (NPV) of $16,000;
investment 2, an NPV of $22,000; investment 3, an
NPV of $12,000; and investment 4, an NPV of $8,000.
• Each investment requires a certain cash outflow at
the present time: investment 1, $5,000; investment
2, $7,000; investment 3, $4,000; and investment 4,
$3,000. Currently, $14,000 is available for investment.
 Example 1: Capital
Budgeting IP

• Formulate an IP whose solution will tell

Stockco how to maximize the NPV
obtained from the four investments.
 Example 1: Solution
• Begin by defining a variable for each decision that Stockco
must make.
• The NPV obtained by Stockco is
Total NPV obtained by Stocko = 16x1 + 22x2 + 12x3 + 8x4
• Stockco’s objective function is
max z = 16x1 + 22x2 + 12x3 +8x4
• Stockco faces the constraint that at most $14,000 can be
invested.
• Stockco’s 0-1 IP is
max z = 16x1 + 22x2 + 12x3 +8x4
s.t. 5x1 + 7x2 + 4x3 +3x4 ≤ 14
xj = 0 or 1 (j = 1,2,3,4)
• Fixed-Charge Problems
– Suppose activity i incurs a fixed charge if
undertaken at any positive level. Let

= Level of activity i
= 1 if activity i is undertaken at positive level
= 0 if activity i is not undertaken at positive level
– Then a constraint of the form < must be added to
the formulation. It must be large enough to ensure
that will be less than or equal to .
• In a set-covering problem, each member of a given set must
be “covered” by an acceptable member of some set.
• The objective of a set-covering problem is to minimize the
number of elements in set 3 that are required to cover all the
elements in set 1.
• Given two constraints
f x1 x2 ,..., xn   0

g x1 , x2 ,..., xn   0
•
ensure that at least one is satisfied by adding an either-or-
constraint.
f x1 x 2 ,..., x n   My

g x , x ,..., x   M 1  y 
1 2 n
• M is a number chosen large enough to ensure that both
constraints are satisfied for all values of x1 , x2 ,..., xn that satisfy
the other constraints in the problem.
• Suppose we want to ensure thatf x1 , x2 ,..., xn  > 0 implies
g x1 , x2 ,...,. x n   0 Then we include the following constraint in the

formulation:
 g x1 , x 2 ,..., x n   My
f x , x ,..., x   M 1  y 
1 2 n
y 0 or 1

• Here, M is a large positive number, chosen large enough so

x1 , xof2 ,..., xn
that f < M and – g < M hold for all values
that satisfy the other constraints in the problem.
• This is called an if-then constraint.
• 0-1 variables can be used to model optimization
problems involving piecewise linear functions.
• A piecewise linear function consists of several
straight-line segments.
• The graph of the piecewise linear function is made
of four straight-line segments.
• The points where the slope of the piecewise linear
function changes are called the break points of the
function.
• A piecewise linear function is not a linear function
so linear programming can not be used to solve the
optimization problem.
• By using 0-1 variables, however, a piecewise linear function
can e represented in linear form.
• Suppose the piecewise linear function f (x) has break points
b1 , b.2 ,..., bn
– Step 1 Wherever f (x) occurs in the optimization
problem, replace f (x) by z1 f b1 z 2 f b2  ... . z n f bn 
– Step 2 Add the following constraints to the problem:

z  y , z  y  y , z  y  y ,..., z  y  y , z
1 1 2 1 1 3 2 3 n n 1 n 1 n
 y n 1

y  y  ...  y
1 2 n 1

z  z  ...  z  1
1 2 n

y 0
1
or 1 i  1,2,..., n  1; z0
1
i  1,2,..., n 
• If a piecewise linear function f(x) involved in a
formulation has the property that the slope of the f(x)
becomes less favorable to the decision maker as x
increases, then the tedious IP formulation is
unnecessary.
• LINDO can be used to solve pure and mixed IPs.
• In addition to the optimal solution, the LINDO output
also includes shadow prices and reduced costs.
• LINGO and the Excel Solver can also be used to solve IPs.
Solving Integer Programming
 The Branch-and-Bound Method for Solving
Pure Integer Programming Problems

• In practice, most IPs are solved by some versions

of the branch-and-bound procedure. Branch-and-
bound methods implicitly enumerate all possible
solutions to an IP.
• By solving a single subproblem, many possible
solutions may be eliminated from consideration.  
• Subproblems are generated by branching on an
appropriately chosen fractional-valued variable.
• Suppose that in a given subproblem (call it old
subproblem), assumes a fractional value between the
integers i and i+1. Then the two newly generated
subproblems are
New Subproblem 1

Old subproblem + Constraint
x i
i

New Subproblem 2 Old subproblem + Constraint x  i 1

i

• Key aspects of the branch-and-bound method for

solving pure IPs
– If it is unnecessary to branch on a subproblem, we
say that it is fathomed.
 These three situations (for a max problem) result in a subproblem
being fathomed
– The subproblem is infeasible, thus it cannot yield the optimal solution
to the IP.
– The subproblem yield an optimal solution in which all variables have
integer values. If this optimal solution has a better z-value than any
previously obtained solution that is feasible in the IP, than it becomes a
candidate solution, and its z-value becomes the current lower bound
(LB) on the optimal z-value for the IP.
– The optimal z-value for the subproblem does not exceed (in a max
problem) the current LB, so it may be eliminated from consideration.
– A subproblem may be eliminated from consideration in these
situations
• The subproblem is infeasible.
• The LB is at least as large as the z-value for the subproblem
• Two general approaches are used to determine
which subproblem should be solved next.
– The most widely used is LIFO.
• LIFO leads us down one side of the branch-and-bound tree
and quickly find a candidate solution and then we backtrack
our way up to the top of the other side
• The LIFO approach is often called backtracking.
– The second commonly used approach is jumptracking.
• When branching on a node, the jumptracking method solves
all the problems created by branching.
• When solving IP problems using Solver you
can adjust a Solver tolerance setting.
• The setting is found under the Options.
• For example a tolerance value of .20 causes
the Solver to stop when a feasible solution
is found that has an objective function value
within 20% of the optimal solution.
 The Branch-and-Bound Method for
Solving Mixed Integer Programming
Problems

• In mixed IP, some variables are required to be integers

and others are allowed to be either integer or
nonintegers.
• To solve a mixed IP by the branch-and-bound method,
modify the method by branching only on variables that
are required to be integers.
• For a solution to a subproblem to be a candidate
solution, it need only assign integer values to those
variables that are required to be integers
 Solving Knapsack Problems by
the Branch-and-Bound Method
• A knapsack problem is an IP with a single constraint.
• A knapsack problem in which each variable must be equal to 0
or 1 may be written as
max z = c1x1 + c2x2 + ∙∙∙ + cnxn
s.t. a1x1 + a2x2 + ∙∙∙ + anxn ≤ b
x1 = 0 or 1 (i = 1, 2, …, n)
• When knapsack problems are solved by the branch-and-bound
method, two aspects of the method greatly simplify.
– Due to each variable equaling 0 or 1, branching on xi will
yield in xi =0 and an xi =1 branch.
– The LP relaxation may be solved by inspection.
 Solving Combinatorial Optimization Problems
by the Branch-and-Bound Method

• A combinatorial optimization problem is any

optimization problem that has a finite number of
feasible solutions.
• A branch-and-bound approach is often the most
efficient way to solve them.
• Examples of combinatorial optimization problems
– Ten jobs must be processed on a single machine. It is known
how long it takes to complete each job and the time at
which each job must be completed. What ordering of the
jobs minimizes the total delay of the 10 jobs?
 – A salesperson must visit each of the 10 cities
before returning to his home. What ordering
of the cities minimizes the total distance the
salesperson must travel before returning
home? This problem is called the traveling
sales person problem (TSP).
• In each of these problems, many possible
solutions must be considered.
 Example : Traveling
Salesperson Problem

• Joe State lives in Gary, Indiana and owns insurance agencies

in Gary, Fort Wayne, Evansville, Terre Haute and South
Bend.
• Each December he visits each of his insurance agencies.
• The distance between each agency is known.
• What order of visiting his agencies will minimize the total
distance traveled?
 Example : Solution

• To begin define
1 if Joe leaves city i and travels next to city j
xij  
0 otherwise
Also, for i  j ,
cij  distance between cities i and j
cii  M , where M is a large positive number

• Several branch-and-bound approaches have been developed

for solving TSPs
• The approach we will use is the one in which the subproblems
reduce to assignments problems.
 Ex. – Solution continued

• First solve the assignment problem in subproblem 1.

This solution contains two subtours and cannot be the
optimal solution.
• Now branch on subproblem 1 in a way that will
prevent one of subproblem 1’s subtours from
recurring in solutions to subsequent subproblems.
• Now arbitrarily choose subproblem 2 to solve, applying
the Hungarian method to the cost matrix shown.
• This solution can not be the optimal solution.
 Ex. – Solution continued

• Now branch subproblem 2 in an effort to exclude the

subtour 2-5-2. Thus we add two additional subproblems.
• Following the LIFO approach, next solve subproblem 4 or
subproblem 5. By using the Hungarian method on
subproblem 4, the optimal solution z=668 , x15 = x24 = x31 =
x43 = x52 =1.
• This is a candidate solution and any node that cannot yield
a z-value < 668 may be eliminated from consideration.
• Following the LIFO rule, next solve subproblem 5. z=704
thus this subproblem is eliminated.
 Ex. – Solution continued

• Only sub problem 3 remains. The optimal solution is z =652. It

is possible for this sub problem to yield a solution with no
subtours that beats z=668.
• Next branch on subproblem 3 creating sub problem 6 and 7.
• Both of these sub problems have a z-value that is larger than
668.
• Subproblem 4 thus yields the optimal solution.
• When using branch-and-bound methods to solve TSPs
with many cities, large amounts of computer time is
needed.
• Heuristic methods, or heuristics, can be used to quickly
lead to a good solution.
• Heuristics is a method used to solve a problem by trial
and error when an algorithm approach is impractical.
• Two types of heuristic methods can be used to solve TSP;
nearest neighbor method and cheapest-insertion
method.
• Nearest Neighbor Method
– Begin at any city and then “visit” the nearest city.
– Then go to the unvisited city closest to the city we have most recently
visited.
– Continue in this fashion until a tour is obtained. After applying this
procedure beginning at each city, take the best tour found.
• Cheapest Insertion Method (CIM)
– Begin at any city and find its closest neighbor.
– Then create a subtour joining those two cities.
– Next, replace an arc in the subtour (say, arc (i, j) by the combinations of two
arcs---(i, k) and (k, j), where k is not in the current subtour---that will
increase the length of the subtour by the smallest (or cheapest) amount.
– Continue with this procedure until a tour is obtained. After applying this
procedure beginning with each city, we take the best tour found.
• Three methods to evaluate heuristics
– Performance guarantees
• Gives a worse-case bound on how far away from
optimality a tour constructed by the heuristic can be
– Probabilistic analysis
• A heuristic is evaluated by assuming that the location
of cities follows some known probability distribution
– Empirical analysis
• Heuristics are compared to the optimal solution for a
number of problems for which the optimal tour is
known
 • An IP formulation can be used to solve a TSP but can become
unwieldy and inefficient for large TSPs.
n n

minimize:  cij xij

i 1 j 1
n
s.t.
 xij  1 1 i  n
j 0
n
 xij  1 1 j  n
i 0
ti  t j  nxij  n  1 2  i, j  n
• cij = distance from city i to city j
• xij = 1 if tour visits i then j, and 0 otherwise (binary)
• ti = arbitrary real numbers we need to solve for
• LINGO can be used to solve the IP of a TSP.
 9.7 Implicit Enumeration

• The method of implicit enumeration is often used to solve

0-1 IPs.
• Implicit enumeration uses the fact that each variable must
be equal to 0 or 1 to simplify both the branching and
bounding components of the branch-and-bound process
and to determine efficiently when a node is infeasible.
• The tree used in the implicit enumeration method is
similar to those used to solve 0-1 knapsack problems.
• Some nodes have variable that are specified called fixed
variables.
• All variables whose values are unspecified at a node are called
free variables.
• For any node, a specification of the values of all the free
variables is called a completion of the node.
• Three main ideas used in implicit enumeration
– Suppose we are at ay node with fixed variables, is there an
easy way to find a good completion of the node that is
feasible in the original 0-1 TSP?
– Even is the best completion of a node is not feasible, the
best completion gives us a bound on the best objective
function value that can be obtained via feasible completion
of the node. This bound can be used to eliminate a node
from consideration.
– At any node, is there an easy way to determine if all
completions of the node are infeasible?
• In general, check whether a node has a feasible
completion by looking at each constraint and
assigning each free variable the best value for
satisfying the constraint.
 Cutting Plane Algorithm

• An alternative method to the branch-and-bound method is

the cutting plane algorithm.
• Summary of the cutting plane algorithm
Step 1 Find the optional tableau for the IP’s programming
relaxation. If all variables in the optimal solution assume
integer values, we have found an optimal solution to the IP;
otherwise, proceed to step2.
Step 2 Pick a constraint in the LP relaxation optimal tableau
whose right-hand side has the fractional part closest to 1/2.
This constraint will be used to generate a cut.
Step 2a For the constraint identified in step 2, write its right-
hand side and each variable’s coefficient in the form [x]+ f,
where 0 <= f < 1.
 Step 2b Rewrite the constraint used to generate the cut as
All terms with integer coefficients = all terms with fractional coefficients

Then the cut is

All terms with fractional coefficients <= 0
Step 3 Use the simplex to find the optimal solution to the LP
relaxation, with the cut as an additional constraint.
• If all variables assume integer values in the optimal solution, we
have found an optimal solution to the IP.
• Otherwise, pick the constraint with the most fractional right-hand
side and use it to generate another cut, which is added to the
tableau.
• We continue this process until we obtain a solution in which all
variables are integers. This will be an optimal solution to the IP.
Thank You