INDE6372: LECTURE 10

MIDTERM REVIEW
Jiming Peng
Department of Industrial Engineering
University of Houston
Summary from Exam 1
• Max: 29, Mean: 21. STD=5.8.

20
18
16
14
12
10
8 Column1
6
4
2
0
[0,15] (15,21) [21,25) [25,30]
Topics from the syllabus
• Specific topics include
• Basic introduction to linear optimization: modeling, linear algebra,
standard form, solving LO via LO solvers (Solver in Excel, Linprog
in Matlab, GAMS);
• Algorithms: Simplex method, primal and dual simplex method,
interior-point methods;
• Theory: Duality theory, sensitivity Analysis, convex analysis,
complementarity conditions;
• Special Scenarios: network flow models, assignment and
transportation problems
• Extension and Applications: Mixed Integer Programming, resource
allocation, arbitrage detection, compressed sensing, regression
etc.
Linear Programming
• Ozark Farms uses at least 800 lb of special feed daily. The
special feed is a mixture of corn and soybean meal with the
following compositions:
• lb per lb of feedstuff
protein fiber cost
corn 0.09 0.02 0.30
Soybean 0.60 0.06 0.90
meal

• Dietary requirements:
• At least 30% protein
• At most 5% fiber
• Our goal is to determine the daily minimum-cost feed mix.
Formulating the LP
• Decision variables:
• X1=lb of corn in the daily mix
• X2=lb of soybean in the daily mix
• Objective function:
• Minimize z=0.3*x1+0.9*x2
Identifying the Constraints
• 1. Ozark Farms needs at least 800 lb of feed a day:
• x1+x2 ≥ 800(lb)
• 2. Daily feed must include at least 30% protein:
• 0.09*x1+ 0.60*x2 >= 0.3( x1+x2)
• 3. Daily feed may include at most 5% fiber:
• 0.02*x1+ 0.06*x2<= 0.05(x1+x2 )
• 4. Non-negative variables:
• x1,x2>=0
The Scaled Mathematical Model
• Minimize z=0.3x1+0.9x2
• x1 + x2 ≥ 800
• -7x1 +10 x2 ≥ 0 (because -0.21*x1+ 0.30*x2 ≥ 0)
• -3x1+ x2 ≤ 0 ( 0.03*x1+ 0.01*x2≤ 0)
• x1 ,x2 ≥ 0
• Which algorithm would you like to use for the above problem?
• Two phase method
• Dual simplex!
Solving LP by Solver
•
470.5882 4376.471

329.4118

min f=3*x1+9*x2

800 s.t. X1+x2=800

0 7*x1-10x2<=0

-1082.35 3*x1-x2>=0

0 x1, x2>=0

Corn Soybean
Objective function Coefficient/cost 0.3 0.9 437.6471
Total amount 1 1 800 ≥ 800
protein 0.09 0.6 240 ≥ 240
fiber 0.02 0.06 9.411765 ≤ 40
variables
x1 x2
470.5882 329.4118
Output from Solver
Microsoft Excel 14.0 Answer Report
Worksheet: [Diet1.xlsx]Sheet1
Report Created: 10/10/2014 11:15:27 AM
Result: Solver found a solution. All Constraints and optimality conditions are satisfied.
Solver Engine
Engine: Simplex LP
Solution Time: 0.015 Seconds.
Iterations: 2 Subproblems: 0
Solver Options
Max Time Unlimited, Iterations Unlimited, Precision 0.000001, Use Automatic Scaling
Max Subproblems Unlimited, Max Integer Sols Unlimited, Integer Tolerance 1%, Solve Without Integer Constraints, Assume NonNegative

Objective Cell (Min)

Cell Name Original Value Final Value
$F$4 Coefficient/cost 0 437.6470588

Variable Cells
Cell Name Original Value Final Value Integer
$D$10 x1 0 470.5882353 Contin
$E$10 x2 0 329.4117647 Contin

Constraints
Cell Name Cell Value Formula Status Slack
$F$5 Total amount 800 $F$5>=$H$5 Binding 0
$F$6 protein 240 $F$6>=$H$6 Binding 0
$F$7 fiber 9.411764706 $F$7<=$H$7 Not Binding 30.58823529
Sensitivity Report

Microsoft Excel 14.0 Sensitivity Report

Worksheet: [Diet1.xlsx]Sheet1
Report Created: 10/10/2014 11:15:27 AM

Variable Cells
    Final Reduced Objective Allowable Allowable
Cell Name Value Cost Coefficient Increase Decrease
$D$10 x1 470.5882353 0 0.3 0.6 0.93
$E$10 x2 329.4117647 0 0.9 1E+30 0.6

Constraints
    Final Shadow Constraint Allowable Allowable
Cell Name Value Price R.H. Side Increase Decrease
$F$5 Total amount 800 0.547058824 800 1E+30 800
$F$6 protein 240 1.176470588 0 240 168
$F$7 fiber 9.411764706 0 0 1E+30 30.58823529
 Simplex-
11

The Primal Simplex

• S.0 Formulate the initial tableau;
• S.1 Find a variable in the first row with the smallest value. If it
is nonnegative, output the current solution as the optimal
solution.
• S.2 Choose the column picked in S.1, and compute the ratios
between the positive elements in this column and the right-
hand side vector, and choose the minimal ratio and the
corresponding basic variable as the leaving variable;
• S.3 Perform the pivot using the pivot element selected in S.1
and S.2, and update the tableau.
• Note in S.2, if all the elements in the picked row are non-
positive, then the problem is unbounded.
Comments on The Primal Simplex
• When to use it?
• Constraints: Ax ≤ b, b ≥ 0. Ax+s=b, x,s>=0.
• What to do if the problem is not in the standard form?
• Transfer into the standard form first.
• What to do if b has some negative element?
• Use big-M method or solve the phase-1 problem using the dual
simplex
• How to select a variable to enter the basis if there is a tie for
the minimum elements in row 0 of the tableau?
• Select arbitrary or apply Boland’s rule (smallest index first)
• What to do if there is no positive elements in the column
corresponding to the selected entering variable?
• The underlying LO is unbounded, and you can terminate the process.
 Simplex-
13

Wyndor Glass
Z- 3x1 - 5x2 =0 (0)

x1 +s1 =4 (1)
2x2 +s2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)

• Convert to initial tableau

Basic
Z x1 x2 s1 s2 s3 r.h.s.
variable
 Simplex-
14

Wyndor Glass, Iteration 1

Basic
x1 x2 s1 s2 s3 r.h.s.
variable
Z -3 -5 0 0 0 0
s1 1 0 1 0 0 4
s2 0 2 0 1 0 12
s3 3 2 0 0 1 18

• Optimality test
• Entering variable (steepest ascent) – pivot column
• Leaving variable (minimum ratio test) – pivot row
• Gaussian elimination

Basic
x1 x2 s1 s2 s3 r.h.s.
variable
Z -3 -5 0 0 0 0
S1 1 0 1 0 0 4
X2 0 1 0 0.5 0 6
S3 3 2 0 0 1 18
 Simplex-
15

Wyndor Glass, Iteration 2

Basic
x1 x2 s1 s2 s3 r.h.s.
variable
Z -3 0 0 5/2 0 30
s1 1 0 1 0 0 4
x2 0 1 0 1/2 0 6
S3 3 0 0 -1 1 6

• Optimality test
• Entering variable (steepest ascent) – pivot column
• Leaving variable (minimum ratio test) – pivot row
• Gaussian elimination

Basic
x1 x2 s1 s2 s3 r.h.s.
variable
Z -3 0 0 5/2 0 30
s1 1 0 1 0 0 4
x2 0 1 0 1/2 0 6
x1 1 0 0 -1/3 1/3 2
 Simplex-
16

Wyndor Glass, Final Solution

Basic
x1 x2 s1 s2 s3 r.h.s.
variable
Z 0 0 0 3/2 1 36
s1 0 0 1 1/3 -1/3 2
x2 0 1 0 1/2 0 6
x1 1 0 0 -1/3 1/3 2

The Input Tableau:

Z - 3x1 - 5x2 =0
x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18

•  The shadow price, the dual solution is

(y1,y2,y3)=(0,3/2,1)=CBV B-1,
• B=, =,
 INDE6372 Advanced Linear Optimization 17

The Dual Simplex Method

• Prototype: min c’x
• s.t. A’x≤b, x≥0;
• Preconditions: c ≥0.
• S.0 Formulate the initial tableau;
• S.1 Find a leaving variable in the last column with the smallest value. If it is
nonnegative, output the current solution as the optimal solution.
• S.2 Choose the row picked in S.1, and compute the ratios between the
negative elements in this row and the first row, and choose the maximal
ratio and the corresponding basic variable as the entering variable;
• S.3 Perform the pivot using the pivot element selected in S.1 and S.2, and
update the tableau.
• Comments in S.2:
• If all the elements in the picked row are non-negative, then the primal
problem is infeasible.
• You can also multiple the selected row by -1, and then use the minimum
ratio rule.
 INDE6372 Advanced Linear Optimization 18

Primal VS Dual Simplex

• 

Primal Simplex Dual Simplex

Convenient for restarting

Convenient with new constraints
Can help to solve the phase-1
problem in primal simplex
Implemented in most opt. solvers
 Primal-Dual relationship

Primal Problem Dual Problem

opt=max opt=min
Constraint i : Variable i :
<= form yi >= 0
= form yi urs
Variable j: Constraint j:
xj >= 0 >= form
xj urs = form
Example
max Z  5x1  4 x 2 min w  6 y1  5 y2  10 y3
x y

3 x1  8 x 2  6 3 y1  y2  8 y3  5
x1  6 x2   5
8y1  6 y2 4
8 x1  10
y1  0; y2 , y3 urs
x 2  0; x1 urs
Strong Duality Theorem
• Consider the standard LO in the form
• Max cTx Min bTy
• Ax+s=b, x,s≥0. ATy-w=c, y,w≥0
• Theorem: suppose (x,s) and (y,w) are feasible for
the primal and dual problem respectively. Then
we have
• bTy-cTx=xTw+yTs≥0
• The equality holds if and only if xTw=yTs=0. In such a
case, (x,s)/(y,w) is optimal for the primal/dual problem.
• Recall s=b-Ax, thus we have yT(b-Ax)=0, xT(ATy-c)=0.
Example
• Primal Dual
• max 4x1+x2+2x3 Min 2 y1+8y2
• s.t. 8x1+3x2+x3≤2 s.t. 8y1+6y2≥4;
• 6x1+x2+x3≤8 3y1+y2≥1;
• x1, x2, x3 ≥0 y1+y2≥2; y1,y2,y2≥0.
• Suppose the optimal solution of the primal problem is
x*=(0,0,2). What’s the optimal solution to the dual
problem?
• Observation: Since y2(8-6x1-x2-x3)=0, thus we have y2=0.
Since x3(y1+y2-2)=0, thus y1=2.
• Optimal solution: (y1=2,y2=0)!
Example
• Primal Dual
• max 4x1+x2+2x3 Min 12 y1+8y2
• s.t. 8x1+3x2+x3≤12 s.t. 8y1+6y2≥4;
• 6x1+x2+x3≤8 3y1+y2≥1;
• x1, x2, x3 ≥0 y1+y2≥2; y1,y2,y2≥0.
• Suppose the optimal solution of the primal problem is
x*=(0,0,8). What’s the optimal solution to the dual
problem?
• Observation: Since y1(12-8x1-3x2-x3)=0, thus we have
y1=0. Since x3(y1+y2-2)=0, thus y2=2.
• Optimal solution: (y1=0,y2=2)!
Prima-Dual Relationships
•  Primal Dual
Feasible and bounded Feasible and bounded
Optimal Optimal
infeasible unbounded
unbounded infeasible
infeasible infeasible
Sensitivity Analysis
• Sensitivity analysis is concerned with how changes in
an LP’s parameters affect the optimal solution.
• There are six possible changes:
1.Changing the objective function coefficient of a nonbasic variable.
Reduced cost, row 0 in the final tableau
2.Changing the objective function coefficient of a basic variable.
Need to check whether the current basis remains optimal
3.Changing the right-hand side of a constraint.
Shadow price, the solution to the dual problem as listed in row 0 in the final tableau
corresponding to the slack variables

4.Changing the column of a nonbasic variable.

5.Adding a new variable or activity (one more column).
6.Adding a new constraint (one more row). Dual Simplex
Comments on Sensitivity Analysis
• The reduced cost can be viewed as shadow price where
the RHS on non-negativity has been changed
• Estimate the allowable range of the change in some
parameter in the objective function or the RHS of the
constraints;
• Adding new constraints or variables.
 Simplex-
27

Wyndor Glass
Z- 3x1 - 5x2 =0 (0)

x1 +s1 =4 (1)
2x2 +s2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)
Sensitivity Analysis: Wyndor Example
•  The basis matrix and its inverse
• , , cbv=(0,5,3)
• The reduced cost for non-basic variables and shadow
price for rhs of constraints:
• Check the Z-row in the final tableau:
• The allowable range of change:
• For non-basic variable, it is based on the reduced cost;
• For the RHS of constraints, we need to ensure
•
• ≥ 0.
Sensitivity analysis for nonstandard LO
• Consider the following example:
•
Max 3x1 + 4x2+ x3 Max 3x1 + 4x2+ x3 Min 50y1+15y2+10y3
s.t. x1+x2+x3≤50 s.t. x1+x2+x3+s1 =50 s.t. y1+2y2+y3≥3;
2x1-x2+x3≥15 2x1-x2+x3-s2+a2=15 y1-y2+y3≥4;
x1+x2 =10 x1+x2 + a3=10 y1+y2 ≥1;
x1,x2,x3≥0 x1,x2,x3, s1, s2, a2, a3≥0 y1≥0,y2≤0, y3 urs
Z x1 x2 x3 s1 s2 a2 a3 80  
1 0 0 1 M 0 M+3 80
s2 -3 0 0 1 1 -1 -2 15
x3 0 0 1 1 0 0 -1 40
x2 1 1 0 0 0 0 1 10
Other Issues
• How to construct the augmented LO in big-M method or
the two phase method?
• How to deal with the degenerate cases and avoid cycling
in the simplex method?
• Farkas Lemma and separation theorem.