Basics on Linear Programming

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodrı́guez-Carbonell

March 24, 2023

Linear Programs (LP’s)
■ A linear program is an optimization problem of the form

min cT x
A1 x ≤ b1
A2 x = b2
A3 x ≥ b3
x ∈ Rn

c ∈ Rn , bi ∈ Rmi , Ai ∈ Rmi ×n , i = 1, 2, 3
■ x is the vector of variables
■ cT x is the cost or objective function
■ A1 x ≤ b1 , A2 x = b2 and A3 x ≥ b3 are the constraints

2 / 22
Linear Programs (LP’s)
■ A linear program is an optimization problem of the form

min cT x
A1 x ≤ b1
A2 x = b2
A3 x ≥ b3
x ∈ Rn

c ∈ Rn , bi ∈ Rmi , Ai ∈ Rmi ×n , i = 1, 2, 3
■ x is the vector of variables
■ cT x is the cost or objective function
■ A1 x ≤ b1 , A2 x = b2 and A3 x ≥ b3 are the constraints
■ Example:
min x + y + z
x+y =3
0≤x≤2
0≤y≤2 2 / 22
Notes on the Definition of LP
■ Solving minimization or maximization is equivalent:

max{ f (x) | x ∈ S } = − min{ −f (x) | x ∈ S }

■ Satisfiability problems are a particular case:

take arbitrary cost function, e.g., c = 0

3 / 22
Equivalent Forms of LP’s
■ This form is not the most convenient for algorithms

■ We will assume problems to be in canonical form:

min cT x
Ax = b
x≥0

c ∈ Rn , b ∈ Rm , A ∈ Rm×n , n ≥ m, rank(A) = m

■ Often variables are identified with columns of the matrix,

and constraints are identified with rows

4 / 22
Methods for Solving LP’s
■ Simplex algorithms
■ Interior-point algorithms

5 / 22
Methods for Solving LP’s
■ Simplex algorithms
■ Interior-point algorithms

5 / 22
Basic Definitions (1)
min cT x
Ax = b
x≥0

■ Any vector x such that Ax = b is called a solution

■ A solution x satisfying x ≥ 0 is called a feasible solution
■ An LP with feasible solutions is called feasible;
otherwise it is called infeasible
■ A feasible solution x∗ is called optimal
if cT x∗ ≤ cT x for all feasible solution x
■ A feasible LP with no optimal solution is unbounded

6 / 22
Basic Definitions (2)
max x + 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ (x, y, s1 , s2 , s3 ) = (−1, −1, 5, 3, 3) is a solution but not feasible

■ (x, y, s1 , s2 , s3 ) = (1, 1, 1, 1, 1) is a feasible solution

7 / 22
Basic Definitions (3)
max x + βy
x + y + s1 = α
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ If α = −1 the LP is not feasible

■ If α = 3, β = 2 then
(x, y, s1 , s2 , s3 ) = (1, 2, 0, 1, 0) is the (only) optimal solution

8 / 22
Basic Definitions (3)
max x + βy
x + y + s1 = α
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 ≥ 0

■ If α = −1 the LP is not feasible

■ If α = 3, β = 2 then
(x, y, s1 , s2 , s3 ) = (1, 2, 0, 1, 0) is the (only) optimal solution
■ There may be more than one optimal solution:
If α = 3 and β = 1 then
{(1, 2, 0, 1, 0), (2, 1, 0, 0, 1), ( 32 , 32 , 0, 21 , 12 )} are optimal

8 / 22
Basic Definitions (4)
y

max x + 2y

y≤2 max x + y

(1, 2) x+y ≤3

(2, 1)
x≥0 x≤2

y≥0 x

9 / 22
Basic Definitions (5)
y

y≤2
min x + 2y x+y ≤3
x+y ≤3
0≤x≤2
y≤2 x≥0 x≤2

x
Unbounded LP
min x + 2y

10 / 22
Basic Definitions (6)
y

max x + 2y

max x + 2y
x+y ≤3 y≤2
0≤x≤2 x+y ≤3
(1, 2)
y≤2

LP is bounded, x≥0 x≤2

but set of feasible
x
solutions is not

11 / 22
Bases (1)
Let us denote by a1 , ..., an the columns of A
Recall that n ≥ m, rank(A) = m.

■ A matrix of m columns (ak1 , ..., akm ) is a basis

if the columns are linearly independent
■ Note that a basis is a square matrix!
■ If (ak1 , ..., akm ) is a basis,
then the variables (xk1 , ..., xkm ) are called basic
■ We usually denote
by B the list of indices (k1 , ..., km ), and
by R the list of indices (1, 2, ..., n) − B; and
by B the matrix (ai | i ∈ B), and
by R the matrix (ai | i ∈ R)
xB the basic variables, xR the non-basic ones

12 / 22
Bases (2)
max x + 2y
x + y + s1 = 3 x y s1 s2 s3
x + s2 = 2
 
1 1 1 0 0
y + s3 = 2 A= 1 0 0 1 0 
x, y, s1 , s2 , s3 ≥ 0 0 1 0 0 1

■ (x, s1 , s2 ) do not form a basis:

 
1 1 0
 1 0 1  does not have linearly independent columns
0 0 0

■ (s1 , s2 , s3 ) form a basis, where xB = (s1 , s2 , s3 ), xR = (x, y)

   
1 0 0 1 1
B= 0 1 0  R= 1 0 
0 0 1 0 1
13 / 22
Bases (3)
■ If B is a basis, then the following holds

BxB + RxR = b

Hence:

xB = B −1 b − B −1 RxR

Non-basic variables determine values of basic ones

■ If non-basic variables are set to 0, we get the solution

xR = 0, xB = B −1 b

Such a solution is called a basic solution

■ If a basic solution satisfies xB ≥ 0 then it is called a

basic feasible solution, and the basis is feasible

14 / 22
Bases (4)
Consider basis (s1 , s2 , s3 )

max x + 2y    
x + y + s1 = 3 1 0 0 1 1
x + s2 = 2 B= 0 1 0  R= 1 0 
y + s3 = 2 0 0 1 0 1
x, y, s1 , s2 , s3 ≥ 0

Equations xB = B −1 b − B −1 RxR are


 s1 = 3 − x − y
s2 = 2 − x
s3 = 2 − y


Basic solution is
 
3  
0
σB =  2  σR =
0
2
So basis (s1 , s2 , s3 ) is feasible 15 / 22
Bases (5)
Basis (x, y, s1 ) is not feasible

max x + 2y    
x + y + s1 = 3 1 1 1 0 0
x + s2 = 2 B= 1 0 0  R= 1 0 
y + s3 = 2 0 1 0 0 1
x, y, s1 , s2 , s3 ≥ 0
  
 x = 2 − s2 2  
0
y = 2 − s3 σB =  2  σR =
0
s1 = −1 + s2 + s3 −1


16 / 22
Bases (6)
A basis is called degenerate
when at least one component of its basic solution xB is null
For example:

max x + 2y    
x + y + s1 = 4 1 1 0 1 0
x + s2 = 2 B= 1 0 1  R= 0 0 
y + s3 = 2 0 1 0 0 1
x, y, s1 , s2 , s3 ≥ 0
  
 x = 2 + s3 − s1 2
y = 2 − s3 σB =  2 
s2 = s1 − s3 0


17 / 22
Geometry of LP’s (1)
■ Set of feasible solutions of an LP is a convex polyhedron

■ Basic feasible solutions are vertices of the convex polyhedron

18 / 22
Geometry of LP’s (2)
max x + 2y
x + y + s1 = 3
■ x + s2 = 2 y
y + s3 = 2 y≤2
x, y, s1 , s2 , s3 ≥ 0 (0, 2) 1 2 (1, 2)
x+y ≤3
■ xB1 = (y, s1 , s2 )
3 (2, 1)
■ xB2 = (x, y, s2 )
x≥0 x≤2
■ xB3 = (x, y, s3 ) 5 4
y≥0 x
■ xB4 = (x, s1 , s3 ) (0, 0) (2, 0)

■ xB5 = (s1 , s2 , s3 )

19 / 22
Geometry of LP’s (3)
■ Theorem (Minkowski-Weyl)
Let P be a feasible LP.
There exist basic feasible solutions v1 , ..., vr ∈ Rn and vectors
r1 , ..., rs ∈ Rn such that a point x is a feasible solution to P iff

r
X s
X
x= λi vi + µ j rj
i=1 j=1
Pr
for certain λi , µj such that i=1 λi = 1 and λi , µj ≥ 0.

20 / 22
Possible Outcomes of an LP (1)
■ Theorem (Fundamental Theorem of Linear Programming)
Let P be an LP.
Then exactly one of the following holds:

1. P is infeasible
2. P is unbounded
3. P has an optimal basic feasible solution

It is sufficient to investigate basic feasible solutions!

21 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value

can be decreased by taking µ∗j larger. Contradiction!

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value

can be decreased by taking µ∗j larger. Contradiction!

■ Otherwise cT rj ≥ 0 for all j. Assume cT x∗ < cT vi for all i.

T ∗
Pr ∗ T
Pr ∗ T ∗ T ∗
Pr ∗ = c T x∗
c x ≥ i=1 λi c vi > i=1 λi c x = c x λ
i=1 i

22 / 22
Possible Outcomes of an LP (2)
Proof: Assume P feasible and with optimal solution x∗ .
Let us see we can find a basic feasible solution as good as x∗ .
By Minkowski-Weyl theorem, we can write
Pr Ps
x∗ = λ ∗v
i=1 i i + µ ∗r
j=1 j j
Pr ∗
where i=1 λi = 1 and λ∗i , µ∗j ≥ 0. Then
Pr Ps
c T x∗ = ∗ T
i=1 λi c vi + ∗ T
j=1 µj c rj

■ If there is j such that cT rj < 0 then objective value

can be decreased by taking µ∗j larger. Contradiction!

■ Otherwise cT rj ≥ 0 for all j. Assume cT x∗ < cT vi for all i.

T ∗
Pr ∗ T
Pr ∗ T ∗ T ∗
Pr ∗ = c T x∗
c x ≥ i=1 λi c vi > i=1 λi c x = c x λ
i=1 i

Contradiction! Thus there is i such that cT x∗ ≥ cT vi ;

in fact, cT x∗ = cT vi by the optimality of x∗ . 22 / 22