Advanced Optimization

ctl.mit.edu
Agenda for Lesson

Network Models
Non-Linear Optimization
Tips on Optimization in Practice

2
Networks

3
 Network Representation
Network Terminology
Node or vertices a point (facility, DC, plant, region)
Arc or edge link between two nodes (roads, flows, etc.), may be directional
Network or graph a collection of nodes and arcs

1
6

14
3 7 i j
11

2 8 15
12
4 9 Arcs
16 xij = flow on arc from node i to node j
cij = per unit cost for arc i to j

5
10 13 Nodes:
yj = 1 if node j is used, =0 otherwise
fj = cost of opening node j
hj = unit cost for any flow through node j
4
 Distance/Connectivity Matrix

Washington DC
Morgantown
Indianapolis

Greensboro
Charleston

Harrisburg

Richmond
Columbus

Cincinnati

Lexington
Cleveland
Louisville

Knoxville
Nashville
St. Louis
Chicago

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 Chicago - 300 201 362
2 St. Louis 300 - 245 263 312
3 Indianapolis 201 245 - 114 176 112
4 Louisville 263 114 - 175 86
5 Nashville 312 175 - 180
6 Cleveland 362 - 142 201 251 332
7 Columbus 176 142 - 105
8 Cincinnati 112 105 - 95 204
9 Lexington 86 95 - 170 177
10 Knoxville 180 170 - 299
11 Morgantown 201 - 157 213 209
12 Charleston 251 204 177 157 - 244 318
13 Greensboro 299 244 - 205
14 Harrisburg 332 213 - 120
Washington
15 DC 209 120 - 111
16 Richmond 318 205 111 -

5
Three Network Problems
Shortest Path Problem
Given: One origin, one destination
Find: Shortest path from single origin to single destination

Traveling Salesman Problem

Given: One origin, many destinations, sequential stops, one vehicle
Find: Shortest tour connecting each stop once and only once and
returning to the origin

Transportation & Transshipment Problems

Given: Multiple supply and demand nodes with fixed and/or variable
costs and capacities on nodes and/or arcs
Find: Minimum cost flow of product from origins to destinations

6
Shortest Path Problem

7
Shortest Path Problem (SPP)
What is the objective?
Find the shortest path in a network between two nodes
Why is it important?
Result is used as base for other analysis
Connects physical to operational network
What are the challenges?
What route in practice is used? Shortest? Fastest? Un-restricted?
How frequently should we update the network?
Should we use time or distance?
What is the impact of real-time changes due to congestion or weather?
Speed of calculating versus looking up in a table
What are the primary approaches?
Standard Linear Programming (LP) Formulation
Specialized Algorithms (e.g., Network Simplex, Dijkstras)
8
 LP Formulation for Shortest Path
Find shortest path from 1 to 5 Min z = 3x12 +4x13 +5x14 +7x25 +2x34 +4x35 + x45
2
s.t. x12 + x13 +x14 =1
x12 x25 =0
3
1 5 x13 x34 x35 =0
x14 +x34 x45 = 0
4 x25 +x35 +x45 = 1
x12, x13, x14, x25, x34, x35, x45 0

Minimize: cij xij

i j
Note:
Subject to: Note that integrality is guaranteed
x =1
i ji
" j=s Other specialized algorithms leverage the
network structure to solve much faster.
x - x
i ji i ij
= 0 " j s, j t

x =1
i ij
" j=t 2

xij 0
where: 3
1 5
xij = Number of units flowing from node i to node j
cij = Cost per unit for flow from node i to node j
4
s = Source node where flow starts
t = terminal node where flow ends 9
Traveling Salesman Problem

10
Traveling Salesman Problem (TSP)
What is the objective?
Starting from an origin node, find the minimum distance required to visit
each node once and only once and return to the origin.
Why is it important?
TSP is at core of all vehicle routing problems
Local routing and last mile deliveries are both common and important
What are the challenges?
It is exceptionally hard to solve exactly, due to its size
Possible solutions increase exponentially with number of nodes
What are the primary approaches?
Special algorithms for exact solutions (smaller problems)
Heuristics many available
Nearest Neighbor
Cheapest Insertion

11
Finding TSP using Nearest Neighbor Heuristic
1. Select any node to be the active node
2. Connect the active node to the closest unconnected
node, make that the new active node
3. If there are more unconnected nodes go to step 2,
1 otherwise connect to the origin and end.

14

7
3
11

8
2 15

12
4
9

16

5
13
10

12
Finding TSP using Cheapest Insertion Heuristic
1. Form a sub tour from the convex hull
2. Add the to the tour the unconnected node
that increases the cost the least, continue until
1 all nodes are connected.

14

7
3
11

8
2 15

12
4
9

16

5
13
10

13
Transportation & Transshipment
Problems

14
Transportation & Transshipment Problems
What is the objective?
Given a network of nodes and arcs, find the minimum cost flow of
product from supply nodes that satisfy demand at destination nodes
Why is it important?
Transportation problems are everywhere (see Banner Chemical)
Transshipment problems are at the heart of larger supply chain network
design models
What are the challenges?
Data requirements can be extensive
Where to draw the line on realism versus practicality
Cost structure: variable and/or fixed on arcs and/or nodes? Concave costs?
Single or multiple commodities? Are products fungible?
Is there any variability of demand, supply or flows considered?
Are there any capacity (min or mx) on arcs or transshipment nodes?
What are the primary approaches?
Mixed Integer Linear Programs
Some simulation usually after optimization
15
 R1

Network Flow Models P1 A

R2

R3
R1 B
P1 P2
R2

R3
Transshipment Problem
Min z= cij xij
P2 i j

s.t.
Transportation Problem
Min z= cij xij x S "i S
j ij i
i j

s.t. x D "j D
i ij j

x ij
Si "i S x - x = 0 "j D, S
i ij i ji
j
xij 0 "ij
x i ij
Dj "j D
xij 0 "ij Conservation of Flow Constraints

xij xji
j
The inbound flow must equal the
outbound flow at transshipment points,
cross-docks, mixing centers, etc.
i xij = i xji
16
Non-Linear Programs (NLP)

17
 Non Linear Optimization
Optimal Cut-Out, x* = 0.196 m
0.14 Max Volume (x*) = 0.132 m3

0.12

0.10
Box Volume (cubic meters)

0.08

0.196
0.06
0.608 m

0.04

0.02
1.108 m

-
- 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50
Size of corner cut-out (meters)

18
Example of NLP: Ace Ice Carvers
Ace Ice Carvers creates ice sculptures for parties and
special occasions. The number of sculptures
produced is the product of the two inputs, ice and
labor. Ice can be purchased at $80 per unit and
labor is $20 per hour. A total of $160 is available.

How much labor and how much ice should be

procured in order to maximize the output?

Image CC0 Public Domain from pixabay.com. Problem adapted from Winston (1994) 19
Formulating Ace Ice Carvers 1

Step 1. Determine Decision Variables

x = Number of units of ice to acquire, 0
y = Number of hours of labor to acquire 0
Step 2. Formulate Objective Function
Maximize = z = xy
Step 3. Formulate Constraints
80x + 20y 160
Max z = xi
i

s.t.
Max z = xy cx i i
B
i
80x + 20y 160 xi 0 "i
x0 where:
xi = Number of units of input i to acquire
y0 ci = Cost per unit of input i
B = Budget for input
20
Ace Ice Carvers

Y units of ice
Feasible
Region

X labor hours

21
Ace Ice Carvers

Y units of ice
Feasible
Region

z= xy = 1
X labor hours

22
Ace Ice Carvers

Y units of ice
Feasible
Region

z= xy = 2
z= xy = 1
X labor hours

23
Ace Ice Carvers

Y units of ice
Feasible
Region

z= xy = 3
z= xy = 2
z= xy = 1
X labor hours

24
 Ace Ice Carvers

Y units of ice
Optimal NLP Solution
x = 1 labor hour
Feasible y = 4 units of ice
Max Production = 4 sculptures
Region

Things to Note:
Optimal solution is no
longer in a corner!
NLPs require different
solution techniques and z= xy = 4
tools
Shape of objective function
z= xy = 3
and constraints determine z= xy = 2
whether solution is local or z= xy = 1
global
Convex Min - Global X labor hours
Concave Max - Global

25
Practical Tips for Optimization

26
Optimization Models in Practice (1/2)
What are we here for?
Determining what to solve is rarely readily apparent or agreed upon
by all stakeholders.
Establish and document the over-riding objective of a project early on.
Level of Detail & Scope of the Model
Models cannot fully represent reality (caricature vs. portrait)
Models will NEVER consider all factors
Determine problem boundaries and data aggregation levels
Input Data
Collecting data is hardest, least appreciated, and most time
consuming task in an optimization project.
Data are never complete, clean, or totally correct.
Every extra hour spent on data collection, cleaning, and verification
saves days later on in the project.
27
Optimization Models in Practice (2/2)
Sensitivity and Robustness Analysis
These are all deterministic models data assumed perfect & unchanging!
Optimization models will do anything for a dollar, yuan, peso, euro, etc.
Run multiple what-if scenarios changing uncertain input values and
testing different conditions.
Models versus People (Models dont make decisions, People do!)
Optimization models are good at . . .
Making trade-offs between complicated options and
Uncovering unexpected insights and solutions.
People are good at . . .
Considering intangible and non-quantifiable factors,
Identifying underlying patterns, and
Mining previous experience and insights.
Models should be used for Decision Support not for the Decision itself

28
Key Points from Lesson

29
Key Points from Lesson
Network Optimization
Shortest Path Easy & fast to solve (LP or special algorithms)
Traveling Salesman Problem Hard to solve (heuristics)
Flow Problems (Transportation & Transshipment) Widely used (MILPs)
Non-Linear Programs
Harder to solve than LPs lose corner solutions
Shape of objective function and constraints dictate approach and
difficulty
Practical Tips for Optimization in Practice
Know your problem
Know your team
Know your tool

30
Questions, Comments, Suggestions?
Use the Discussion Forum!

Daisy ready to get optimal

(photos courtesy of Lana Scott)
caplice@mit.edu
ctl.mit.edu