Math 407A: Linear Optimization

2
Lecture 4: LP Standard Form

2 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 3 Math 407A: Linear Optimization 1 / 27
1 LPs in Standard Form

2 Minimization → maximization

3 Linear equations to linear inequalities

4 Lower and upper bounded variables

5 Interval variable bounds

6 Free variable

7 Two Step Process to Standard Form

Lecture 4: LP Standard Form 4 Math 407A: Linear Optimization 2 / 27

LPs in Standard Form

We say that an LP is in standard form if its matrix representation has the form

Lecture 4: LP Standard Form 5 Math 407A: Linear Optimization 3 / 27

LPs in Standard Form

We say that an LP is in standard form if its matrix representation has the form

max cT x

s.t. Ax ≤ b

0≤x

5 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 5 Math 407A: Linear Optimization 3 / 27
LPs in Standard Form

We say that an LP is in standard form if its matrix representation has the form

max cT x It must be a maximization problem.

s.t. Ax ≤ b

0≤x

5 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 5 Math 407A: Linear Optimization 3 / 27
LPs in Standard Form

We say that an LP is in standard form if its matrix representation has the form

max cT x It must be a maximization problem.

s.t. Ax ≤ b Only inequalities of the correct direction.

0≤x

5 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 5 Math 407A: Linear Optimization 3 / 27
LPs in Standard Form

We say that an LP is in standard form if its matrix representation has the form

max cT x It must be a maximization problem.

s.t. Ax ≤ b Only inequalities of the correct direction.

0≤x All variables must be non-negative.

5 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 6 Math 407A: Linear Optimization 3 / 27
Every LP can be Transformed to Standard Form

Lecture 4: LP Standard Form 7 Math 407A: Linear Optimization 4 / 27

Every LP can be Transformed to Standard Form

minimization → maximization

7 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 7 Math 407A: Linear Optimization 4 / 27
Every LP can be Transformed to Standard Form

minimization → maximization
To transform a minimization problem to a maximization problem multiply the
objective function by −1.

7 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 7 Math 407A: Linear Optimization 4 / 27
Every LP can be Transformed to Standard Form

minimization → maximization
To transform a minimization problem to a maximization problem multiply the
objective function by −1.

linear inequalities

7 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 7 Math 407A: Linear Optimization 4 / 27
Every LP can be Transformed to Standard Form

minimization → maximization
To transform a minimization problem to a maximization problem multiply the
objective function by −1.

linear inequalities
If an LP has an inequality constraint of the form

ai1 x1 + ai2 x2 + · · · + ain xn ≥ bi ,

it can be transformed to one in standard form by multiplying the inequality

through by −1 to get

−ai1 x1 − ai2 x2 − · · · − ain xn ≤ −bi .

7 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 8 Math 407A: Linear Optimization 4 / 27
Every LP can be Transformed to Standard Form

linear equations

Lecture 4: LP Standard Form 9 Math 407A: Linear Optimization 5 / 27

Every LP can be Transformed to Standard Form

linear equations
The linear equation
ai1 xi + · · · + ain xn = bi
can be written as two linear inequalities

ai1 x1 + · · · + ain xn ≤ bi

and
ai1 x1 + · · · + ain xn ≥ bi .
or equivalently
ai1 x1 + · · · + ain xn ≤ bi
−ai1 x1 − · · · − ain xn ≤ −bi

9 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 10 Math 407A: Linear Optimization 5 / 27
Every LP can be Transformed to Standard Form

variables with lower bounds

Lecture 4: LP Standard Form 11 Math 407A: Linear Optimization 6 / 27

Every LP can be Transformed to Standard Form

variables with lower bounds

If a variable xi has lower bound li which is not zero (li ≤ xi ) or equivalently,
0 ≤ xi − li , one obtains a non-negative variable wi := xi − li yielding the
substitution
xi = wi + li .
In this case, the bound li ≤ xi is equivalent to the bound 0 ≤ wi .

11 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 11 Math 407A: Linear Optimization 6 / 27
Every LP can be Transformed to Standard Form

variables with lower bounds

If a variable xi has lower bound li which is not zero (li ≤ xi ) or equivalently,
0 ≤ xi − li , one obtains a non-negative variable wi := xi − li yielding the
substitution
xi = wi + li .
In this case, the bound li ≤ xi is equivalent to the bound 0 ≤ wi .
variables with upper bounds

11 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 11 Math 407A: Linear Optimization 6 / 27
Every LP can be Transformed to Standard Form

variables with lower bounds

If a variable xi has lower bound li which is not zero (li ≤ xi ) or equivalently,
0 ≤ xi − li , one obtains a non-negative variable wi := xi − li yielding the
substitution
xi = wi + li .
In this case, the bound li ≤ xi is equivalent to the bound 0 ≤ wi .
variables with upper bounds
If a variable xi has an upper bound ui (xi ≤ ui ), or equivalently, 0 ≤ ui − xi ,
one obtains a non-negative variable wi := ui − xi yielding the substitution

xi = ui − wi .

In this case, the bound xi ≤ ui is equivalent to the bound 0 ≤ wi .

11 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 12 Math 407A: Linear Optimization 6 / 27
Every LP can be Transformed to Standard Form

variables with interval bounds

Lecture 4: LP Standard Form 13 Math 407A: Linear Optimization 7 / 27

Every LP can be Transformed to Standard Form

variables with interval bounds

An interval bound of the form li ≤ xi ≤ ui can be transformed into one
non-negativity constraint and one linear inequality constraint in standard
form by making the substitution

xi = wi + li .

13 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 13 Math 407A: Linear Optimization 7 / 27
Every LP can be Transformed to Standard Form

variables with interval bounds

An interval bound of the form li ≤ xi ≤ ui can be transformed into one
non-negativity constraint and one linear inequality constraint in standard
form by making the substitution

xi = wi + li .

In this case, the bounds li ≤ xi ≤ ui are equivalent to the constraints

0 ≤ wi and wi ≤ ui − li .

13 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 14 Math 407A: Linear Optimization 7 / 27
Every LP can be Transformed to Standard Form

free variables

Lecture 4: LP Standard Form 15 Math 407A: Linear Optimization 8 / 27

Every LP can be Transformed to Standard Form

free variables
Sometimes a variable is given without any bounds. Such variables are called
free variables. To obtain standard form every free variable must be replaced
by the difference of two non-negative variables. That is, if xi is free, then we
get
xi = u i − vi
with 0 ≤ ui and 0 ≤ vi .

15 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 16 Math 407A: Linear Optimization 8 / 27
Transformation to Standard Form

Put the following LP into standard form.

minimize 3x1 − x2
subject to −x1 + 6x2 − x3 + x4 ≥ −3
7x2 + x4 = 5
x3 + x4 ≤ 2

−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 ≤ 2.

Lecture 4: LP Standard Form 17 Math 407A: Linear Optimization 9 / 27

Transformation to Standard Form

The hardest part of the translation to standard form, or at least the part most
susceptible to error, is the replacement of existing variables with non-negative
variables.

Lecture 4: LP Standard Form 18 Math 407A: Linear Optimization 10 / 27

Transformation to Standard Form

The hardest part of the translation to standard form, or at least the part most
susceptible to error, is the replacement of existing variables with non-negative
variables.

Reduce errors by doing the transformation in two steps.

18 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 18 Math 407A: Linear Optimization 10 / 27
Transformation to Standard Form

The hardest part of the translation to standard form, or at least the part most
susceptible to error, is the replacement of existing variables with non-negative
variables.

Reduce errors by doing the transformation in two steps.

Step 1: Make all of the changes that do not involve a variable substitution.

18 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 18 Math 407A: Linear Optimization 10 / 27
Transformation to Standard Form

The hardest part of the translation to standard form, or at least the part most
susceptible to error, is the replacement of existing variables with non-negative
variables.

Reduce errors by doing the transformation in two steps.

Step 1: Make all of the changes that do not involve a variable substitution.

Step 2: Make all of the variable substitutions.

18 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 19 Math 407A: Linear Optimization 10 / 27
Must be a maximization problem

minimize 3x1 − x2
subject to −x1 + 6x2 − x3 + x4 ≥ −3
7x2 + x4 = 5
x3 + x4 ≤ 2
−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 ≤ 2.

Lecture 4: LP Standard Form 20 Math 407A: Linear Optimization 11 / 27

Must be a maximization problem

minimize 3x1 − x2
subject to −x1 + 6x2 − x3 + x4 ≥ −3
7x2 + x4 = 5
x3 + x4 ≤ 2
−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 ≤ 2.

Minimization =⇒ maximization

20 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 20 Math 407A: Linear Optimization 11 / 27
Must be a maximization problem

minimize 3x1 − x2
subject to −x1 + 6x2 − x3 + x4 ≥ −3
7x2 + x4 = 5
x3 + x4 ≤ 2
−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 ≤ 2.

Minimization =⇒ maximization

maximize − 3x1 + x2 .

20 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 21 Math 407A: Linear Optimization 11 / 27
Inequalities must go the right way.

Lecture 4: LP Standard Form 22 Math 407A: Linear Optimization 12 / 27

Inequalities must go the right way.

− x1 + 6x2 − x3 + x4 ≥ −3

22 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 22 Math 407A: Linear Optimization 12 / 27
Inequalities must go the right way.

− x1 + 6x2 − x3 + x4 ≥ −3

becomes

22 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 22 Math 407A: Linear Optimization 12 / 27
Inequalities must go the right way.

− x1 + 6x2 − x3 + x4 ≥ −3

becomes

x1 − 6x2 + x3 − x4 ≤ 3.

22 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 23 Math 407A: Linear Optimization 12 / 27
Equalities are replaced by 2 inequalities.

7x2 + x4 = 5

Lecture 4: LP Standard Form 24 Math 407A: Linear Optimization 13 / 27

Equalities are replaced by 2 inequalities.

7x2 + x4 = 5
becomes

24 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 24 Math 407A: Linear Optimization 13 / 27
Equalities are replaced by 2 inequalities.

7x2 + x4 = 5
becomes

7x2 + x4 ≤ 5

24 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 24 Math 407A: Linear Optimization 13 / 27
Equalities are replaced by 2 inequalities.

7x2 + x4 = 5
becomes

7x2 + x4 ≤ 5

and

24 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 24 Math 407A: Linear Optimization 13 / 27
Equalities are replaced by 2 inequalities.

7x2 + x4 = 5
becomes

7x2 + x4 ≤ 5

and

− 7x2 − x4 ≤ −5

24 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 25 Math 407A: Linear Optimization 13 / 27
Grouping upper bounds with the linear inequalities.

The double bound −2 ≤ x4 ≤ 2 indicates that we should group the upper bound
with the linear inequalities.

Lecture 4: LP Standard Form 26 Math 407A: Linear Optimization 14 / 27

Grouping upper bounds with the linear inequalities.

The double bound −2 ≤ x4 ≤ 2 indicates that we should group the upper bound
with the linear inequalities.

x4 ≤ 2

26 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 27 Math 407A: Linear Optimization 14 / 27
Step 1: Transformation to Standard Form

Combining all of these changes gives the LP

maximize −3x1 + x2
subject to x1 − 6x2 + x3 − x4 ≤ 3
7x2 + x4 ≤ 5
− 7x2 − x4 ≤ −5
x3 + x4 ≤ 2
x4 ≤ 2
−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 .

Lecture 4: LP Standard Form 28 Math 407A: Linear Optimization 15 / 27

Step 2: Variable Replacement
The variable x1 is free, so we replace it by

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27

Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

z2 = x2 + 1 or x2 = z2 − 1 with 0 ≤ z2 .

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

z2 = x2 + 1 or x2 = z2 − 1 with 0 ≤ z2 .

x3 is bounded above (x3 ≤ 5), so we replace it by

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

z2 = x2 + 1 or x2 = z2 − 1 with 0 ≤ z2 .

x3 is bounded above (x3 ≤ 5), so we replace it by

z3 = 5 − x3 or x3 = 5 − z3 with 0 ≤ z3 .

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

z2 = x2 + 1 or x2 = z2 − 1 with 0 ≤ z2 .

x3 is bounded above (x3 ≤ 5), so we replace it by

z3 = 5 − x3 or x3 = 5 − z3 with 0 ≤ z3 .

x4 is bounded below (−2 ≤ x4 ), so we replace it by

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 29 Math 407A: Linear Optimization 16 / 27
Step 2: Variable Replacement
The variable x1 is free, so we replace it by

x1 = z1+ − z1− with 0 ≤ z1+ , 0 ≤ z1− .

x2 has a non-zero lower bound (−1 ≤ x2 ) so we replace it by

z2 = x2 + 1 or x2 = z2 − 1 with 0 ≤ z2 .

x3 is bounded above (x3 ≤ 5), so we replace it by

z3 = 5 − x3 or x3 = 5 − z3 with 0 ≤ z3 .

x4 is bounded below (−2 ≤ x4 ), so we replace it by

z4 = x4 + 2 or x4 = z4 − 2 with 0 ≤ z4 .

29 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 30 Math 407A: Linear Optimization 16 / 27
Step 2: Transformation to Standard Form

Substituting x1 = z1+ − z1− into

maximize −3x1 + x2
subject to x1 − 6x2 + x3 − x4 ≤ 3
7x2 + x4 ≤ 5
− 7x2 − x4 ≤ −5
x3 + x4 ≤ 2
x4 ≤ 2

−1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 .
gives

Lecture 4: LP Standard Form 31 Math 407A: Linear Optimization 17 / 27

Step 2: Transformation to Standard Form

maximize −3z1+ + 3z1− + x2

subject to z1+ − z1− − 6x2 + x3 − x4 ≤ 3
7x2 + x4 ≤ 5
− 7x2 − x4 ≤ −5
x3 + x4 ≤ 2
x4 ≤ 2
0 ≤ z1+ , 0 ≤ z1− , −1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 .

Lecture 4: LP Standard Form 32 Math 407A: Linear Optimization 18 / 27

Step 2: Transformation to Standard Form

Substituting x2 = z2 − 1 into

maximize −3z1+ + 3z1− + x2

subject to z1+ − z1− − 6x2 + x3 − x4 ≤ 3
7x2 + x4 ≤ 5
− 7x2 − x4 ≤ −5
x3 + x4 ≤ 2
x4 ≤ 2

0 ≤ z1+ , 0 ≤ z1− , −1 ≤ x2 , x3 ≤ 5, −2 ≤ x4 .
gives

Lecture 4: LP Standard Form 33 Math 407A: Linear Optimization 19 / 27

Step 2: Transformation to Standard Form

maximize −3z1+ + 3z1− + z2 − 1

subject to z1+ − z1− − 6z2 + x3 − x4 ≤ 3 − 6 = −3
7z2 + x4 ≤ 5 + 7 = 12
− 7z2 − x4 ≤ −5 − 7 = −12
x3 + x4 ≤ 2
x4 ≤ 2

0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , x3 ≤ 5, −2 ≤ x4 .

Lecture 4: LP Standard Form 34 Math 407A: Linear Optimization 20 / 27

Step 2: Transformation to Standard Form

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 + x3 − x4 ≤ −3
7z2 + x4 ≤ 12
− 7z2 − x4 ≤ −12
x3 + x4 ≤ 2
x4 ≤ 2
0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , x3 ≤ 5, −2 ≤ x4 .

Lecture 4: LP Standard Form 35 Math 407A: Linear Optimization 21 / 27

Step 2: Transformation to Standard Form

Substituting x3 = 5 − z3 into

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 + x3 − x4 ≤ −3
7z2 + x4 ≤ 12
− 7z2 − x4 ≤ −12
x3 + x4 ≤ 2
x4 ≤ 2

0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , x3 ≤ 5, −2 ≤ x4 .
gives

Lecture 4: LP Standard Form 36 Math 407A: Linear Optimization 22 / 27

Step 2: Transformation to Standard Form

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 − z3 − x4 ≤ −3 − 5 = −8
7z2 + x4 ≤ 12
− 7z2 − x4 ≤ −12
− z3 + x4 ≤ 2 − 5 = −3
x4 ≤ 2

0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , 0 ≤ z3 , −2 ≤ x4 .

Lecture 4: LP Standard Form 37 Math 407A: Linear Optimization 23 / 27

Step 2: Transformation to Standard Form

Substituting x4 = z4 − 2 into

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 − z3 − x4 ≤ −8
7z2 + x4 ≤ 12
− 7z2 − x4 ≤ −12
− z3 + x4 ≤ −3
x4 ≤ 2

0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , 0 ≤ z3 , −2 ≤ x4 .
gives

Lecture 4: LP Standard Form 38 Math 407A: Linear Optimization 24 / 27

Step 2: Transformation to Standard Form

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 − z3 − z4 ≤ −8 − 2 = −10
7z2 + z4 ≤ 12 + 2 = 14
− 7z2 − z4 ≤ −12 − 2 = −14
− z3 + z4 ≤ −3 + 2 = −1
z4 ≤ 2+2=4

0 ≤ z1+ , 0 ≤ z1− , −0 ≤ z2 , 0 ≤ z3 , 0 ≤ z4 .
which is in standard form.

Lecture 4: LP Standard Form 39 Math 407A: Linear Optimization 25 / 27

Step 2: Transformation to Standard Form

After making these substitutions, we get the following LP in standard form:

Lecture 4: LP Standard Form 40 Math 407A: Linear Optimization 26 / 27

Step 2: Transformation to Standard Form

After making these substitutions, we get the following LP in standard form:

maximize −3z1+ + 3z1− + z2

subject to z1+ − z1− − 6z2 − z3 − z4 ≤ −10
7z2 + z4 ≤ 14
− 7z2 − z4 ≤ −14
− z3 + z4 ≤ −1
z4 ≤ 4

0 ≤ z1+ , z1− , z2 , z3 , z4 .

40 Author: James Burke, University of Washington

Lecture 4: LP Standard Form 41 Math 407A: Linear Optimization 26 / 27
Transformation to Standard Form: Practice

Transform the following LP to an LP in standard form.

minimize x1 − 12x2 + 2x3

subject to −5x1 − x2 + 3x3 = −15

2x1 + x2 − 20x3 ≥ −30
0≤ x2 , 1≤ x3 ≤4

Lecture 4: LP Standard Form 42 Math 407A: Linear Optimization 27 / 27