Linear Programming: Mohamed Kobeissi

Linear Programming
Mohamed Kobeissi
American University of Beirut
13 février 2024
Mohamed Kobeissi Linear Programming

Linear model
A maximization linear program is of the form :
max z = c1 x1 + c2 x2 + . . . + cn xn / a11 x1 + . . . + a1n xn ≤ b1

a21 x1 + . . . + a2n xn ≤ b2
.. .. ..
. . .
am1 x1 + . . . + amn xn ≤ bm
xi ≥ 0

This program has n variables and m constraints. The function
z is the objective function (or economic function) ; the set of
constraints form the domain, the constraints xi ≥ 0 are the
positivity constraints

Matrix representation of LP
A maximization linear program can be represented in matrix

form :
max z = cX / AX ≤ b
X ≥0
where A is a m × n matrix, X and b are column vectors of size
n
X is the vector of variables, and b is a constant vector

Introductory Example
A manufacturing company makes two types of laptop stands
foldable and compact. It costs $10 per unit to produce a
foldable stand and $7 per unit to produce a compact stand.
Foldable and compact stands are sold at $14 and $10 per unit,
respectively. There are 120 units of raw materials available to
manufacture both stands. While a foldable requires 3 units of
raw materials, a compact stand requires only 1 unit of raw
materials. The company has a total of 80 man hours for
manufacturing operations. A foldable stand and a compact
stand take 2 and 1 man hours, respectively. Moreover, the
company’s storage is limited. There are only 60 units of space
is available. Both laptop stands take unit of storage space.
Taking into account all this information, what is the optimal
production bundle of foldable and compact stands for this
manufacturing facility ?
Scheduling
A post office requires different numbers of full-time employees
on different days of the week. The number of full-time
employees required on each day is given in Table 4. Union rules
state that each full-time employee must work five consecutive
days and then receive two days off. For example, an employee
who works Monday to Friday must be off on Saturday and
Sunday. The post office wants to meet its daily requirements
using only fulltime employees. Formulate an LP that the post
office can use to minimize the number of full-time employees
who must be hired. The number of full-time employees (x)
required on each day is given in the following table
day Mon Tue Wed Thu Fri Sat Sun

x 17 13 15 19 14 16 11

Multiperiod Financial Models
Finco Investment Corporation must determine investment
strategy for the firm during the next three years. Currently
(time 0), $100,000 is available for investment. Investments
A, B, C , D, and E are available. The cash flow associated with
investing $1 in each investment is given in the following table
cash flow (period)

0 1 2 3
A -1 +0.5 +1 0
B 0 -1 +0.5 +1
C -1 +1.2 0 0
D -1 0 0 +1.9
E 0 0 -1 +1.5

For example, $1 invested in investment B requires a $1 cash
outflow at time 1 and returns 50 cent at time 2 and $1 at time
3. To ensure that the company’s portfolio is diversified, Finco
requires that at most $75,000 be placed in any single
investment. In addition to investments AE, Finco can earn
interest at 8% per year by keeping uninvested cash in money
market funds. Returns from investments may be immediately
reinvested. For example, the positive cash flow received from
investment C at time 1 may immediately be reinvested in
investment B. Finco cannot borrow funds, so the cash
available for investment at any time is limited to cash on hand.
Formulate an LP that will maximize cash on hand at time 3

The decision variables :
Let xi , i = 1, 2, 3, 4, 5 be the amount (in dollars)
invested in type A, B, C , D and E respectively
Let Si be the amount (in dollars) invested in money
market funds at time i = 0, 1, 2

The objective function :
Finco wants to maximize cash on hand at time 3. At time 3,
Finco’s cash on hand will be the sum of all cash inflows at
time 3. From the description of investments AE and the fact
that from time 2 to time 3, S2 will increase to 1.08S2 , Time 3
cash on hand
z = x2 + 1.9x4 + 1.5x5 + 1.08S2
Then Finco’s objective function is :
max z = x2 + 1.9x4 + 1.5x5 + 1.08S2

The constraints :
The investments A, C , D and S0 are available at time 0, which
gives the first constraint
x1 + x3 + x4 + S0 = 100000

At time 1, 0.5x1 + 1.2x3 + 1.08S0 is available for investment,
and investments B and S1 are available, thus the second
constraint is
0.5x1 + 1.2x3 + 1.08S0 = x2 + S1

Similarly, x1 + 0.5x2 + 1.08S1 is available for investment at
time 2, and investments E et S2 are available, which gives
x1 + 0.5x2 + 1.08S1 = x5 + S2
Moreover, at most $75,000 can be placed in any of

investments, then xi ≤ 75000 , (i = 1, 2, 3, 4, 5)

Finco’s LP is therefore :
max z = x2 + 1.9x4 +1.5x5 + 1.08S2 /
x1 + x3 + x4 + S0 = 100000
0.5x1 + 1.2x3 + 1.08S0 = x2 + S1
x1 + 0.5x2 + 1.08S1 = x5 + S2
xi ≤ 75000
xi , Si ≥ 0

The transportation problem
A transportation problem is a type of Linear Programming
Problem (LP) in which items are carried from a set of sources
to a set of destinations based on the supply and demand of
the sources and destinations, with the goal of minimizing the
total transportation cost. Assume first that the demand is
equal to the supply. The unit transportation cost from source
Si to destination Dj is cij

Modelling the transportation problem
Let xij be the quantity of items transported from source Si to

destination Dj for i = 1, 2, . . . , m, and n = 1, 2, . . . , n.
The transportation cost is
m X
X m
cij xij
i=1 j=1

The constraints
The supply of source i cannot exceed ai :
xi1 + xi2 + . . . + xin ≤ ai
The demand of destination j must exceed bi :
x1j + x2j + . . . + xmj ≥ bj

The transportation LP is then :
min z = m
P Pm
i=1 j=1 cij xij / x11 + x12 + . . . + x1n ≤ a1
..
.
xm1 + xm2 + . . . + xmn ≤ am
x11 + x21 + . . . + xm1 ≥ b1
..
.
x1n + x2n + . . . + xmn ≥ bn
xi ≥ 0

The Simplex Algorithm - Phase I
Every linear programming problem can be written in the

following standard form.
max z = c T x (1a)
subject to
Ax = b , (1b)
x ≥ 0. (1c)

Here x ∈ Rn is a vector of n unknowns, A ∈ M(m × n) with n
typically much larger than m, c ∈ Rn the coefficient vector of
the objective function, and the expression x ≥ 0 signifies
xi ≥ 0 for i = 1, . . . , n. For simplicity, we assume that
rank A = m, i.e., that the rows of A are linearly independent.

Step 1 : put (P) in standard form
Write (P) in standard form by adding slack variable to

each constraint
max z = c1 x1 + c2 x2 + . . . + cn xn / a11 x1 + . . . + a1n xn + y1 = b1
a21 x1 + . . . + a2n xn + y2 = b2
.. .. ..
. . .
am1 x1 + . . . + amn xn + ym = bm
xi ≥ 0
yi ≥ 0

Step 2 : Write the coefficients of the problem into
a simplex tableau
The coefficients of the linear system are collected in an
augmented matrix as known from Gaussian elimination for
systems of linear equations ; the coefficients of the objective
function are written in a separate bottom row with a zero in
the right hand column.
write the initial tableau simplex (T0 )
x1 x2 ... xn s1 s2 ... sm
s1 a11 a12 ... a1n 1 0 ... 0 b1
s2 a21 a22 ... a2n 0 1 ... 0 b2
.. .. .. .. .. .. .. .. .. ..
. . . . . . . . . .
sm am1 am2 ... amn 0 0 ... 1 bm
z −c1 −c2 ... −cn 0 0 ... 0 0
The variables will be divided into m basic variables and n − m
non-basic variables. We will act on the tableau by the rules of
Gaussian elimination, where the pivots are always chosen from
the columns corresponding to the basic variables. In our
example, x1 and x2 are non-basic variables, and s1 and s2 are
basic variables

Turning a problem into standard form involves the following
steps.
1 Multiply a constraint, where appropriate, by −1.
2 Introduce slack variables to turn inequality constraints
into equality constraints with nonnegative unknowns.
Any inequality of the form a1 x1 + · · · + an xn ≤ c can be
replaced by a1 x1 + · · · + an xn + s = c with s ≥ 0.
3 Replace variables which are not sign-constrained by
differences.
Any real number x can be written as the difference of
nonnegative numbers x = u − v with u, v ≥ 0.

Step 3 : Gaussian elimination
For a given set of basic variables, we use Gaussian

elimination to reduce the corresponding columns to a
permutation of the identity matrix. This amounts to
solving Ax = b in such a way that the values of the
nonbasic variables are zero and the values for the basic
variables are explicitly given by the entries in the right
hand column of the fully reduced matrix. In addition, we
eliminate the coefficients of the objective function below
each pivot.

Step 3 : continue
The solution expressed by the tableau is only admissible if

all basic variables are non-negative, i.e., if the right hand
column of the reduced tableau is free of negative entries.
This is the case in this example. At the initial stage,
however, negative entries may come up ; this indicates
that different initial basic variables should have been
chosen. At later stages in the process, the selection rules
for the basic variables will guarantee that an initially
feasible tableau will remain feasible throughout the
process.

Step 4 : Choose new basic variables
The entering variable shall correspond to the column

which has the most negative entry in the cost function
row. If all cost function coefficients are non-negative, the
cost cannot be lowered and we have reached an optimum.
The algorithm then terminates.
If entering and leaving variable can be found, go to Step 3 and
iterate.

Step 4 : continue
Once the entering variable is determined, the leaving

variable shall be chosen as follows. Compute for each row
the ratio of its right hand coefficient to the corresponding
coefficient in the entering variable column. Select the row
with the smallest finite positive ratio. The leaving variable
is then determined by the column which currently owns
the pivot in this row. If all coefficients in the entering
variable column are non-positive, the cost can be lowered
indefinitely, i.e., the linear programming problem does not
have a finite solution. The algorithm then also terminates.
If entering and leaving variable can be found, go to Step 3 and
iterate.

Two-variables example
The linear program
max z = 3 x1 + 2 x2 (2a)
subject to
x1 + x2 ≤ 1 (2b)
−2 x1 + x2 ≤ 5 (2c)
x1 ≥ 0 (2d)
x2 ≥ 0 (2e)

Graphical resolution

... And by the simplex algorithm
Is written in standard form :
max z = 3 x1 + 2 x2 (3a)
subject to
x1 + x2 + s1 = 1 (3b)
−2 x1 + x2 + s2 = 5 (3c)
x1 , x2 ≥ 0 (3d)
s1 , s2 ≥ 0 (3e)

For our example, the initial tableau reads :
x1 x2 s1 s2
s1 1 1 1 0 1
s2 −2 1 0 1 5
z −3 −2 0 0 0
In this example, x1 and x2 are non-basic variables, and s1 and
s2 are basic variables

.. and another LP
Solve the following LP program
max z = 3x1 + x2 + 2x3 / x1 + 4x3 ≤ 8

−2x1 + x2 − 5x3 ≤ 10
xi ≥ 0

First put P in standard form
max z = 3x1 + x2 + 2x3 / x1 + 4x3 + s1 = 8

−2x1 + x2 − 5x3 + s2 = 10
x1 , x2 ≥ 0
s1 , s 2 ≥ 0

The initial tableau simplex is :
x1 x2 x3 s1 s2
s1 1 0 4 1 0 8
s2 −2 1 −5 0 1 10
z −3 −1 −2 0 0 0
and its corresponding solution is

s1 = 8, s2 = 10, x1 = x2 = x3 = 0, and zmax = 0

In the next simplex iteration, x1 enters the basis, and s1 leaves.
The result of pivoting is :
x1 x2 x3 s1 s2
x1 1 0 4 1 0 8
s2 0 1 3 2 1 26
z 0 −1 10 3 0 24
and its corresponding solution is

x1 = 8, s2 = 26, y1 = x2 = x3 = 0, and zmax = 24

Again, the variable x2 enters the basis, and s2 leaves, which
gives :
x1 x2 x3 s1 s2
x1 1 0 4 1 0 8
x2 0 1 3 2 1 26
z 0 0 13 5 1 50
As all coefficients in z are positives, x1 = 8, x2 = 26, and
x3 = 0 is the optimal solution, and z ∗ = 50

Pivoting and linear systems
let Ax = b be a system of linear equations, with A a m × n

matrix, x ∈ Rn is a vector of n unknowns, and m ≤ n. Assume
that rank(A) = m
to determine all the solutions of this system, one would find m
columns linearly independent that spans the column vectors of
A (and therefore of Rm ), then to express the system in this
basis

Let B be the invertible (why ?) matrix formed by m linearly
independent column vectors of A. After permuting (be
careful !) the columns of A, we can write
A = (N | B)
and
xN
Ax = b ⇔ (N | B) =b
xB

Multiplying the previous system by B −1 yields

−1 xN
B (N | B) = B −1 b ⇔ B −1 NxN + xB = B −1 b
xB
To determine all the solutions of the system, fix xN to obtain

xB
xN = s ∈ Rn−m
xB = B −1 b − B −1 Ns

Example
Find the solutions of the system
Ax = b

0 1 2 2 1
where A = and b =
1 1 2 3 1

The Simplex Algorithm - big-M method
Solve the linear program (P) :
(P) max z = 3x1 − 5x2 − x3

subject to 

 x1 + 2x2 ≤ 6
−2x1 + x2 + 4x3 ≤ −2


 2x1 + x2 − 3x3 ≤ 13
x1 , x2 , x3 ≥ 0


The big-M method
Put (P) in standard form as follows :
(P) max z = 3x1 − 5x2 − x3 − Ma
subject to 

 x1 + 2x2 + s1 = 6
 2x1 − x2 − 4x3 − s2 + a = 2


2x1 + x2 − 3x3 + s3 = 13
x1 , x2 , x3 ≥ 0




s1 , s2 , s3 ≥ 0


To initiate the simplex algorithm, the ”basic” artificial variable
a should be removed from the objective function. This is done
by expressing a in terms of the ”non-basic” variables from the
second constraint. We obtain :
(P) max z = (3 + 2M)x1 − (5 + M)x2 − (1 + 4M)x3 − 2M
subject to 

 x1 + 2x2 + s1 = 6
 2x1 − x2 − 4x3 − s2 + a = 2


2x1 + x2 − 3x3 + s3 = 13
x ,x ,x ≥ 0

 1 2 3



s1 , s2 , s3 ≥ 0

whose corresponding initial tableau simplex is :
x1 x2 x3 s1 s2 s3 a
s1 1 2 0 1 0 0 0 6
a 2 −1 −4 0 −1 0 1 2
s3 2 1 −3 0 0 1 0 13
z −2M − 3 M + 5 4M + 1 0 M 0 0 0
Then proceed the same way as the simplex algorithm

In the first iteration, the variable x1 enters the basis, and a
leaves. After pivoting, the simplex tableau is :
x1 x2 x3 s1 s2 s3 a
s1 0 5/2 2 1 1/2 0 1/2 5
x1 1 −1/2 −2 0 −1/2 0 1/2 1
s3 0 2 1 0 1 1 −1 11
z 0 7/2 −5 0 −3/2 0 1 3
As the artificial variable leaves the basis, its corresponding

column can be removed, and the simplex algorithm continue
as usual ..
Note that M do not appear in the tableau after a leaves !

In the first iteration, the variable x1 enters the basis, and a
leaves. After pivoting, the simplex tableau is :
x1 x2 x3 s 1 s2 s3
s1 0 5/2 2 1 1/2 0 5
x1 1 −1/2 −2 0 −1/2 0 1
s3 0 2 1 0 1 1 11
z 0 7/2 −5 0 −3/2 0 3
As the artificial variable leaves the basis, its corresponding

column can be removed, and the simplex algorithm continue
as usual ..

Example 1
Solve the following LP
max z = 2x1 + 4x4 − 15x5 − 10x6
subject to


 −2x1 + x4 − x5 + x6 ≤ −11
4x1 + 3x3 + x4 − x5 + 2x6 = 40


 4x1 + x2 + 3x4 + x5 − 3x6 = 27
x1 ≥ 0, . . . , x6 ≥ 0


Put the program in standard form an follows :
max z = 2x1 + 4x4 − 15x5 − 10x6 − Ma
subject to


 2x1 − x4 + x5 − x6 − s1 + a = 11
 4x1 + 3x3 + x4 − x5 + 2x6 = 40


4x1 + x2 + 3x4 + x5 − 3x6 = 27
x ≥ 0, . . . , x6 ≥ 0

 1



s1 , a ≥ 0
Initiate the simplex algorithm with the basic variables a, x2 ,

and x3 . The initial tableau simplex can be obtained after
substituting these variables in z from their respective equations

Example 2
Consider the following LP
max z = x1 + 3x2 + 5x3 + x4 + 4x5

subject to


 x1 + x2 − x3 + x4 = 1
3x1 + x2 + 3x3 + 3x4 + x5 = 8


 2x 1 + 7x2 + x4 + 2x5 = 20
x1 ≥ 0, . . . , x5 ≥ 0

give the initial tableau simplex corresponding to the basic

solution x1 = 0, x2 = 2, x3 = 1, x4 = 0, x5 = 3

The linear program can be expressed in matrix form as :
max z = cx
subject to
Ax = b , x ≥ 0
with    
1 1 −1 1 0 1
A= 3 1 3 3 1  , b= 8 
2 7 0 1 2 20

 
1 −1 0
Let B =  1 3 1  be the (invertible !) matrix
7 0 2
corresponding to the basic variables x2 , x3 , and x5
 
6 2 −1
A simple calculation yields B −1 = 5 2 −1 
−21 −7 4

The system Ax = b is now equivalent to B −1 Ax = B −1 b,
where  
10 1 0 11 0
−1
B A=  9 0 1 10 0 
−34 0 0 −38 1
 
2
and B −1 b =  1 
3

And the LP is equivalent to :
max z = 58x1 + 85x4 + 23

subject to 

 10x1 + x2 + 11x4 = 2
9x1 + x3 + 10x4 = 1


 −34x1 − 38x4 + x5 = 3
x1 ≥ 0, . . . , x5 ≥ 0

z is obtained by substituting by
x2 = 2 − 10x1 − 11x4 , x3 = 1 − 9x1 − 10x4 , and
x5 = 3 + 34x1 + 38x4

The initial tableau simplex corresponding to the basic solution
is then
x1 x2 x3 x4 x5
x2 10 1 0 11 0 2
x3 9 0 1 10 0 1
x5 −34 0 0 −38 1 3
z −58 0 0 −85 0 23
which is not optimal ! so the simplex algorithm continue ....

Duality
The concept of duality is best motivated by an example.

Consider the following transportation problem. Some good is
available at location A at no cost and may be transported to
locations B, C , and D according to the following directed
graph :
4 B
1
4
!
A 3
=
D
5
2
*
C

On each of the unidirectional channels, the unit cost of
transportation is cj for j = 1, . . . , 5. At each of the vertices bα
units of the good are sold, where α = B, C , D. How can the
transport be done most efficiently ?

A first, and arguably most obvious way of quantifying
efficiency would be to state the question as a minimization
problem for the total cost of transportation. If xj denotes the
amount of good transported through channel j, we arrive at
the following linear programming program :
minimize c1 x1 + · · · + c5 x5 (4a)
subject to
x1 − x3 − x4 = bB (4b)
x2 + x3 − x5 = bC (4c)
x4 + x5 = bD (4d)
The three equality constraints state that nothing gets lost at

nodes B, C , and D except what is sold

There is, however, a second, seemingly equivalent way of
characterizing efficiency of transportation. Instead of looking
at minimizing the cost of transportation, we seek to maximize
the income from selling the good. Letting yα denote the unit
price of the good at node α = A, . . . , D with yA = 0 by
assumption, the associated linear programming problem is the
following :
maximize yB bB + yC bC + yD bD (5a)
subject to
yB − yA ≤ c1 (5b)
yC − yA ≤ c2 (5c)
yC − yB ≤ c3 (5d)
yD − yB ≤ c4 (5e)
yD − yC ≤ c5 (5f)
The inequality constraints encode that, in a free market, we
can only maintain a price difference that does not exceed the
cost of transportation. If we charged a higher price, then
“some local guy” would immediately be able to undercut our
price by buying from us at one end of the channel, using the
channel at the same fixed channel cost, then selling at a price
lower than ours at the high-price end of the channel

Setting
 
x1
   
yB 1 0 −1 −1 0
x = .
 ..  , y = yC  , and A = 0 1 1 0 −1
x5 yD 0 0 0 1 1

Primal
we can write program (4) as the abstract primal problem
minimize c T x (6a)
subject to Ax = b , x ≥ 0 (6b)

Dual
Likewise, program (5) can be written as the dual problem
maximize y T b (7a)
subject to y A ≤ cT T
(7b)
We shall prove in the following that the minimal cost and the
maximal income coincide, i.e., that the two problems are
equivalent

Linear Programming: Mohamed Kobeissi

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Linear Programming: Mohamed Kobeissi

Uploaded by

Copyright:

Available Formats

Linear Programming

Mohamed Kobeissi Linear Programming

A maximization linear program is of the form :

max z = c1 x1 + c2 x2 + . . . + cn xn / a11 x1 + . . . + a1n xn ≤ b1

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

A maximization linear program can be represented in matrix

Mohamed Kobeissi Linear Programming

day Mon Tue Wed Thu Fri Sat Sun

Mohamed Kobeissi Linear Programming

cash flow (period)

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

z = x2 + 1.9x4 + 1.5x5 + 1.08S2

Then Finco’s objective function is :

max z = x2 + 1.9x4 + 1.5x5 + 1.08S2

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

0.5x1 + 1.2x3 + 1.08S0 = x2 + S1

Mohamed Kobeissi Linear Programming

Moreover, at most $75,000 can be placed in any of

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Let xij be the quantity of items transported from source Si to

Mohamed Kobeissi Linear Programming

The supply of source i cannot exceed ai :

xi1 + xi2 + . . . + xin ≤ ai

The demand of destination j must exceed bi :

x1j + x2j + . . . + xmj ≥ bj

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Every linear programming problem can be written in the

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Write (P) in standard form by adding slack variable to

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

For a given set of basic variables, we use Gaussian

Mohamed Kobeissi Linear Programming

The solution expressed by the tableau is only admissible if

Mohamed Kobeissi Linear Programming

The entering variable shall correspond to the column

Mohamed Kobeissi Linear Programming

Once the entering variable is determined, the leaving

Mohamed Kobeissi Linear Programming

The linear program

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Is written in standard form :

Mohamed Kobeissi Linear Programming

Mohamed Kobeissi Linear Programming

Solve the following LP program

max z = 3x1 + x2 + 2x3 / x1 + 4x3 ≤ 8

Mohamed Kobeissi Linear Programming

max z = 3x1 + x2 + 2x3 / x1 + 4x3 + s1 = 8

Mohamed Kobeissi Linear Programming

and its corresponding solution is

Mohamed Kobeissi Linear Programming

and its corresponding solution is

Mohamed Kobeissi Linear Programming