Professional Documents
Culture Documents
L01 Introduction
L01 Introduction
g
a rin
rjy ee
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
N ajib eth
Rajib Kumar Bhattacharjya
.R m
of ion
Pr zat
PT
Professor
i
im
Department of Civil Engineering
pt
O
IIT Guwahati
Email: rkbc@iitg.ernet.in
g
The word “optimization” may be very familiar or may be quite new to
a rin
rjy ee
you.
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
……. but whether you know about optimization or not, you are using
o
N ajib eth
optimization in many occasions of your day to day life …….
.R m
of ion
Pr zat
PT
i
im
pt
O
…………….Examples………………
g
a rin
rjy ee
at ha in
ah tac eng
Newspaper
Cooking
uw at il
G h iv
hawker
IT r B r C
i
, I ma fo
EL Ku ds
o
N ajib eth
.R m
of ion
Pr zat
PT
i
im
pt
O
7/14/2021
Nest
Food
O
pt
im
i
Pr zat
of ion
.R m
N ajib eth
R.K. Bhattacharjya/CE/IITG
PT o
EL Ku ds
, I ma fo
IT r B r C
G h iv
uw at il
ah tac eng
at ha in
i rjy ee
a rin
g
Nest
Food
Example
A farmer has 2400 m of fencing and wants to fence off a rectangular field that borders
a straight river. He needs no fence along the river. What are the dimensions of the
field that has the largest area?
g
a rin
rjy ee
Maximize 𝑓 = 𝑥𝑦
at ha in
1600
ah tac eng
x Subject to 𝑔 𝑥, 𝑦 = 𝑥 + 2𝑦 = 2400
uw at il
G h iv
IT r B r C
i
, I ma fo
400 400
EL Ku ds
1000
o
N ajib eth
y y
.R m
of ion
700 700
Pr zat
PT
2200
i
im
100 pt 100
O
RIVER
R.K. Bhattacharjya/CE/IITG
Example
A manufacturer needs to make a cylindrical can that will hold 1.5 liters of liquid.
Determine the dimensions of the can that will minimize the amount of material used in
its construction.
g
a rin
rjy ee
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
Minimize: A = 2πr2 + 2πrh
o
N ajib eth
.R m
of ion
Pr zat
PT
Constraint: πr2h = 1500
i
im
pt
O
R.K. Bhattacharjya/CE/IITG
Example
Objectives
Topology: Optimal connectivity of the
structure
Minimum cost of material: optimal cross
g
a rin
section of all the members
rjy ee
at ha in
ah tac eng
We will consider the second objective only
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
The design variables are the cross sectional area
N ajib eth
of the members, i.e. 𝐴1 to 𝐴7
.R m
of ion
Pr zat
PT
i
im
pt Using symmetry of the structure 𝐴7 = 𝐴1 , 𝐴6 = 𝐴2 , 𝐴5 = 𝐴3
O
R.K. Bhattacharjya/CE/IITG
Optimization formulation
Objective
g
a rin
rjy ee
at ha in
ah tac eng
uw at il
What are the constraints?
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
One essential constraint is non-negativity of design variables, i.e.
N ajib eth
.R m
𝐴1 , 𝐴2 , 𝐴3 , 𝐴4 ≥ 0
of ion
Pr zat
PT
i
im
pt
O
Is it complete now?
R.K. Bhattacharjya/CE/IITG
Optimization formulation
Member Force Member Force
𝐴𝐵 𝑃 𝐵𝐶 𝑃
− 𝑐𝑠𝑐𝜃 + 𝑐𝑠𝑐𝛼
2 2
𝐴𝐶 𝑃 𝐵𝐷 𝑃
g
a rin
+ 𝑐𝑜𝑡𝜃 − 𝑐𝑜𝑡𝜃 + 𝑐𝑜𝑡𝛼
2 2
rjy ee
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
First set of constraints Another constraint is buckling Another constraint may be the
N ajib eth
.R m
of compression members minimization of deflection at C
of ion
𝑃𝑐𝑠𝑐𝜃
Pr zat
PT
≤ 𝑆𝑦𝑐 𝑃 𝜋𝐸𝐴12
i
im
2𝐴1 ≤ pt 𝑃𝑙 0.566 0.500 2.236 2.700
2𝑠𝑖𝑛𝜃 1.281𝑙 2
O
+ + + ≤ 𝛿𝑚𝑎𝑥
𝑃𝑐𝑜𝑡𝜃 𝐸 𝐴1 𝐴2 𝐴3 𝐴4
≤ 𝑆𝑦𝑡
2𝐴2 𝑃 𝜋𝐸𝐴24
𝑐𝑜𝑡𝜃 + 𝑐𝑜𝑡𝛼 ≤
𝑃𝑐𝑠𝑐𝛼 2 5.76𝑙 2
≤ 𝑆𝑦𝑡
2𝐴3
𝑃
𝑐𝑜𝑡𝜃 + 𝑐𝑜𝑡𝛼 ≤ 𝑆𝑦𝑐 R.K. Bhattacharjya/CE/IITG
2𝐴4
Optimization formulation
𝑃𝑐𝑠𝑐𝜃
𝑆𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑆𝑦𝑐 − ≥0
2𝐴1
𝑃𝑐𝑜𝑡𝜃
𝑆𝑦𝑡 − ≥0
2𝐴2
g
a rin
rjy ee
𝑃𝑐𝑠𝑐𝛼
at ha in
𝑆𝑦𝑡 − ≥0
ah tac eng
2𝐴3
uw at il
𝑃
G h iv
IT r B r C
𝑆𝑦𝑐 − 𝑐𝑜𝑡𝜃 + 𝑐𝑜𝑡𝛼 ≥ 0
i
, I ma fo
2𝐴4
EL Ku ds
o
N ajib eth
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑓 = 1.132𝐴1 𝑙 + 2𝐴2 𝑙 + 1.789𝐴3 𝑙 + 1.2𝐴4 𝑙 𝜋𝐸𝐴12 𝑃
.R m
− ≥0
1.281𝑙 2 2𝑠𝑖𝑛𝜃
of ion
Pr zat
PT
𝜋𝐸𝐴24 𝑃
i
im
pt − 𝑐𝑜𝑡𝜃 + 𝑐𝑜𝑡𝛼 ≥ 0
5.76𝑙 2 2
O
10 × 10−6 ≤ 𝐴1 , 𝐴2 , 𝐴3 , 𝐴4 ≤ 10 × 10−6
R.K. Bhattacharjya/CE/IITG
Traveling salesman problem
E
E A
A
g
a rin
rjy ee
E
at ha in
C
ah tac eng
A C
uw at il
G h iv
IT r B r C
i
, I ma fo
B D
EL Ku ds
C B D
o
N ajib eth
.R m
B
of ion
D Another solution: A B C D E
A possible solution: A B E D C
Pr zat
PT
i
im
pt
O
g
a rin
rjy ee
at ha in
ah tac eng
Traveling
uw at il
G h iv
IT r B r C
i
salesman
, I ma fo
EL Ku ds
o
N ajib eth
problem traveling salesman problem
.R m
of ion
Pr zat
PT
i
im
pt
O
g
a rin
rjy ee
circumstances.
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
• Optimization is the mathematical discipline which is concerned with
EL Ku ds
o
N ajib eth
finding the maxima and minima of functions, possibly subject to
.R m
of ion
constraints.
Pr zat
PT
i
im
pt
O
𝑓 = 𝑥−5 2
25
Minimum Equation of the line
20
g
a rin
rjy ee
10
𝑓′ = 2 × 𝑥 − 5 = 0
at ha in
ah tac eng
5
uw at il
G h iv
𝑥∗ = 5 Optimal solution
IT r B r C
0
i
, I ma fo
0 2 4 6 8 10
EL Ku ds
o
N ajib eth
.R m
of ion
30
Pr zat
𝑓 = 25 − 𝑥 2
PT
Equation of the line
i
im
25
pt
O
20
𝑓 ′ = 2𝑥 = 0
15
10 𝑥∗ = 0 Optimal solution
5
Maximum
0
-5 -3
7/14/2021 -1 1 3 5 14
Introduction to optimization
Equation of the surface
𝑓 𝑥, 𝑦 = − 𝑥 2 + 𝑦 2 + 4
g
a rin
rjy ee
at ha in
In this case, we can obtain the optimal
ah tac eng
solution by taking derivatives with respect
uw at il
G h iv
IT r B r C
i
to variable 𝑥 and 𝑦 and equating them to
, I ma fo
EL Ku ds
zero
o
N ajib eth
.R m
of ion
𝜕𝑓
Pr zat
PT
= −2𝑥 = 0 ⇒ 𝑥∗= 0
i
im
pt
O
𝜕𝑥
7/14/2021 15
7/14/2021
O
pt
im
i
Pr zat
of ion
.R m
N ajib eth
PT o
EL Ku ds
, I ma fo
IT r B r C
G h iv
Objective function is defined as
uw at il
ah tac eng
at ha in
i rjy ee
a rin
g
Minimization/Maximization f(x)
Single variable optimization
16
Single variable optimization
g
Stationary points
a rin
rjy ee
at ha in
For a continuous and differentiable function 𝑓(𝑥), a stationary point 𝑥 ∗ is a
ah tac eng
uw at il
point at which the slope of the function is zero, i.e.𝑓 ′ 𝑥 = 0 at 𝑥 = 𝑥 ∗ ,
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
N ajib eth
.R m
of ion
Pr zat
PT
i
im
pt
O
g
a rin
rjy ee
domain over which 𝑓(𝑥) is defined.
at ha in
ah tac eng
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
N ajib eth
.R m
of ion
A function is said to have a global or absolute
Pr zat
PT
maximum at 𝑥 = 𝑥 ∗ if 𝑓(𝑥 ∗ ) ≥ 𝑓(𝑥) for all 𝑥 in
i
im
pt
O
the domain over which 𝑓(𝑥) is defined.
IT r B r C
G h iv
uw at il
ah tac eng
at ha in
i rjy ee
a rin
g
X
Global optima
Introduction to optimization
Local optima
Local optima
Local optima
Local optima
19
Necessary and sufficient conditions for optimality
Necessary condition
If a function 𝑓(𝑥) is defined in the interval 𝑎 ≤ 𝑥 ≤ 𝑏 and has a relative minimum
at 𝑥 = 𝑥 ∗ , Where 𝑎 ≤ 𝑥 ∗ ≤ 𝑏 and if 𝑓 / 𝑥 exists as a finite number at 𝑥 = 𝑥 ∗ ,
g
a rin
rjy ee
then 𝑓 / 𝑥 ∗ = 0
at ha in
ah tac eng
Proof
uw at il
G h iv
IT r B r C
i
, I ma fo
𝑓 𝑥 ∗ + ℎ − 𝑓(𝑥 ∗ )
EL Ku ds
𝑓/ 𝑥∗ = lim
o
ℎ
N ajib eth
ℎ→0
.R m
Since 𝑥 ∗ is a relative minimum 𝑓(𝑥 ∗ ) ≤ 𝑓 𝑥 ∗ + ℎ
of ion
Pr zat
PT
i
im
For all values of ℎ sufficiently close to zero, hence
pt
O
𝑓 𝑥 ∗ + ℎ − 𝑓(𝑥 ∗ )
≥0 if ℎ ≥ 0
ℎ
𝑓 𝑥 ∗ + ℎ − 𝑓(𝑥 ∗ )
≤0 if ℎ ≤ 0
ℎ
7/14/2021 R.K. Bhattacharjya/CE/IITG 20
Necessary and sufficient conditions for optimality
Thus
𝑓/ 𝑥∗ ≥ 0 If ℎ tends to zero through +ve value
g
𝑓/ 𝑥∗ ≤ 0
a rin
If ℎ tends to zero through -ve value
rjy ee
at ha in
ah tac eng
Thus only way to satisfy both the conditions is to have
uw at il
G h iv
IT r B r C
i
, I ma fo
EL Ku ds
o
N ajib eth
/ ∗
𝒇 𝒙 =𝟎
.R m
of ion
Pr zat
PT
i
im
Note: O
pt
• This theorem can be proved if 𝑥 ∗ is a relative maximum
• Derivative must exist at 𝑥 ∗
• The theorem does not say what happens if a minimum or maximum occurs at an end point of
the interval of the function
• It may be an inflection point also.
R.K. Bhattacharjya/CE/IITG
Sufficient conditions for optimality
Sufficient condition
Suppose at point 𝑥 ∗ , the first derivative is zero and first nonzero higher
g
a rin
rjy ee
derivative is denoted by 𝑛, then
at ha in
ah tac eng
uw at il
1. If 𝑛 is odd, 𝑥 ∗ is an inflection point
G h iv
IT r B r C
i
, I ma fo
𝑓′ 𝑥∗ = 0
EL Ku ds
o
N ajib eth
𝑓 ′′ 𝑥 ∗ = 0
.R m
2. If 𝑛 is even, 𝑥 ∗ is a local optimum
of ion
Pr zat
PT
𝑓3 𝑥∗ = 0
If the derivative is positive, 𝑥∗
is a local minimum
i
im
pt
If the derivative is negative, 𝑥 ∗ is a local maximum
O
𝑓4 𝑥∗ = 0
𝑓𝑛 𝑥∗ ≠ 0
ℎ2 ′′ ∗ ℎ𝑛−1 ℎ 𝑛
𝑓 𝑥 ∗ + ℎ = 𝑓 𝑥 ∗ + ℎ𝑓 ′ 𝑥 ∗ + 𝑓 𝑥 + ⋯+ 𝑓 𝑛−1 𝑥 ∗ + 𝑓 𝑛 𝑥 ∗
g
2! 𝑛−1 ! 𝑛!
a rin
rjy ee
at ha in
Since 𝑓 ′ 𝑥 ∗ = 𝑓 ′′ 𝑥 ∗ = ⋯ = 𝑓 𝑛−1 𝑥 ∗ = 0
ah tac eng
uw at il
G h iv
IT r B r C
ℎ𝑛 𝑛 ∗
i
, I ma fo
𝑓 𝑥∗ + ℎ − 𝑓 𝑥∗ = 𝑓 𝑥
EL Ku ds
𝑛!
o
N ajib eth
ℎ𝑛
.R m
When 𝑛 is even ≥0
of ion
𝑛!
Pr zat
PT
i
im
Thus if 𝑓 𝑛 𝑥 ∗ is positive
pt 𝑓 𝑥 ∗ + ℎ − 𝑓 𝑥 ∗ is positive Hence it is local minimum
O
ℎ𝑛
When 𝑛 is odd, changes sign with the change in the sign of ℎ.
𝑛!
Take an example
𝑓 𝑥 = 𝑥 3 − 10𝑥 − 2𝑥 2 − 10
g
a rin
rjy ee
at ha in
𝑓 ′ 𝑥 = 3𝑥 2 − 10 − 4𝑥 = 0
ah tac eng
Apply necessary condition
uw at il
G h iv
IT r B r C
i
𝑥 ∗ = 2.61 𝑎𝑛𝑑 − 1.28
, I ma fo
Solving for 𝑥 These two points are stationary points
EL Ku ds
o
N ajib eth
.R m
of ion
Apply sufficient condition 𝑓 ′′ 𝑥 = 6𝑥 − 4
Pr zat
PT
i
im
pt
O
𝑓 ′′ 2.61 = 11.66 positive and n is odd 𝑓 ′′ −1.28 = −11.68 negative and n is odd
𝑥1
𝑥2
Minimize 𝑓(𝑋) Where 𝑋 = ⋮
𝑥𝑛
g
a rin
rjy ee
at ha in
ah tac eng
Necessary condition for optimality
uw at il
G h iv
IT r B r C
i
, I ma fo
If 𝑓(𝑋) has an extreme point (maximum or minimum) at 𝑋 = 𝑋 ∗ and if the first partial
EL Ku ds
o
N ajib eth
Derivatives of 𝑓 𝑋 exists at 𝑋 ∗ , then
.R m
of ion
Pr zat
PT
i
𝜕𝑓 𝑋 ∗ 𝜕𝑓 𝑋 ∗ im
𝜕𝑓 𝑋 ∗
pt
O
= =⋯= =0
𝜕𝑥1 𝜕𝑥2 𝜕𝑥𝑛
R.K. Bhattacharjya/CE/IITG
Multivariable optimization without constraints
g
a rin
rjy ee
at ha in
(1) positive definite when 𝑋 ∗ is a relative minimum
ah tac eng
uw at il
(2) negative definite when 𝑋 ∗ is a relative maximum
G h iv
IT r B r C
i
, I ma fo
(3) neither positive nor negative definite when 𝑋 ∗ is neither a minimum nor a maximum
EL Ku ds
o
N ajib eth
Proof Taylor series of two variable function
.R m
of ion
Pr zat
PT
𝜕𝑓 𝜕𝑓 1 𝜕2𝑓 𝜕2𝑓 𝜕2𝑓
i
2 2
im
𝑓 𝑥 + ∆𝑥, 𝑦 + ∆𝑦 = 𝑓 𝑥, 𝑦 + ∆𝑥 + ∆𝑦 + ∆𝑥 + 2∆𝑥∆𝑦 + ∆𝑦 +⋯
pt𝜕𝑥 𝜕𝑦 2! 𝜕𝑥 2 𝜕𝑥𝜕𝑦 𝜕𝑦 2
O
𝜕𝑓 𝜕2𝑓 𝜕2𝑓
𝜕𝑥 1 𝜕𝑥 2 𝜕𝑥𝜕𝑦 ∆𝑥
𝑓 𝑥 + ∆𝑥, 𝑦 + ∆𝑦 = 𝑓 𝑥, 𝑦 + ∆𝑥 ∆𝑦
𝜕𝑓 + 2!
∆𝑥 ∆𝑦 +⋯
𝜕2𝑓 𝜕 2 𝑓 ∆𝑦
𝜕𝑦 𝜕𝑥𝜕𝑦 𝜕𝑦 2
R.K. Bhattacharjya/CE/IITG
Multivariable optimization without constraints
1 𝑇
𝑓 𝑋 ∗ + ℎ = 𝑓 𝑋 ∗ + ℎ𝑇 𝛻𝑓 𝑋 ∗ + ℎ 𝑯ℎ + ⋯
2!
Another test
Since 𝑋 ∗ is a stationary point, the necessary
g
condition gives that 𝛻𝑓 𝑋 ∗ = 0
a rin
𝐴1 = 𝑎11
rjy ee
at ha in
Thus
ah tac eng
∗ ∗
1 𝑇 𝑎11 𝑎12
𝑓 𝑋 +ℎ −𝑓 𝑋 = ℎ 𝑯ℎ + ⋯ 𝐴2 = 𝑎
uw at il
G h iv
2! 21 𝑎22
IT r B r C
i
, I ma fo
𝑎11 𝑎12 𝑎13
EL Ku ds
Now, 𝑋 ∗ will be a minima, if ℎ𝑇 𝑯ℎ is positive
o
N ajib eth
𝐴3 = 𝑎21 𝑎22 𝑎23
.R m
𝑋 ∗ will be a maxima, if ℎ𝑇 𝑯ℎ is negative 𝑎31 𝑎32 𝑎33
of ion
Pr zat
PT
𝑎11 𝑎12 𝑎13 𝑎14 ⋯ 𝑎1𝑛
ℎ𝑇 𝑯ℎ will be positive if 𝑯 is a positive definite matrix
i
im
pt 𝑎21 𝑎22 𝑎23 𝑎24 ⋯ 𝑎2𝑛
𝐴𝑛 = ⋮
O
⋮ ⋮ ⋮ ⋯ ⋮
ℎ𝑇 𝐻ℎ will be negative if 𝑯 is a negative definite matrix 𝑎𝑛1 𝑎𝑛2 𝑎𝑛3 𝑎𝑛4 ⋯ 𝑎𝑛𝑛
30
25
20 Optimal solution 𝑥 ∗ = 0
g
a rin
rjy ee
15
at ha in
Optimal solution 𝑥 ∗ = 0
ah tac eng
10
uw at il
G h iv
5
IT r B r C
i
, I ma fo
0
EL Ku ds
o
-5 -3 -1 1 3 5
N ajib eth
.R m
0
of ion
Pr zat
PT
-5 -3 -1 1 3 5
-5
i
im
pt
O
-10
-25
-30
7/14/2021 R.K. Bhattacharjya/CE/IITG 28
7/14/2021
O
pt
im
i
Pr zat
of ion
.R m
N ajib eth
R.K. Bhattacharjya/CE/IITG
PT o
EL Ku ds
, I ma fo
IT r B r C
G h iv
uw at il
ah tac eng
at ha in
i rjy ee
a rin
g
29