Professional Documents
Culture Documents
Introduction to Optimization
J. McCalley
1
Electricity markets and tools
Day-ahead Real-time
5
Convex functions
The second derivative test is sufficient but not necessary.
www.ebyte.it/library/docs/math09/AConvexInequality.html 6
Convex functions
Definition #2: A function f(x) is convex if a line drawn
between any two points on the function remains on or
above the function in the interval between the two points.
Is a linear function convex?
The set on the left is convex. The set on the right is not.
8
Convex Sets
Definition #3: A set C is convex if a line segment between
any two points in C lies in C.
9
S. Boyd and L. Vandenberghe, “Convex optimization,” Cambridge University Press, 2004.
Global vs. local optima
Example: Solve the following:
Minimize f(x)=x2
Solution: f’(x)=2x=0x*=0.
This solution is a local optimum.
It is also the global optimum.
Example: Solve the following:
Minimize f(x)=x3-17x2+80x-100
Solution: f’(x)=3x2-34x+80=0
Solving the above results in x=3.33 and x=8.
11
Convexity & global vs. local optima
When minimizing a function, if we want to be sure that we
can get a global solution via differentiation, we need to
impose some requirements on our objective function.
We will also need to impose some requirements on the
feasible set S (set of possible values the solution x* may take).
Min f(x) min f ( x)
subject to subject to
h(x)=c Feasible set
xS
g(x)> b
Definition: If f(x) is a convex function, and if S is a convex set, then
the above problem is a convex programming problem.
Definition: If f(x) is not a convex function, or if S is not a convex set,
then the above problem is a non-convex programming problem. 12
Convex vs. nonconvex programming problems
MATHEMATICAL
PROGRAMMING
The desirable quality of a convex
programming problem is that any locally
optimal solution is also a globally optimal Convex
solution. If we have a method of
finding a locally optimal solution, that We address convex
programming
method also finds for us the globally problems in
optimum solution. addressing linear
programming.
The undesirable quality of a non-convex
programming problem is that any
method which finds a locally optimal Non-convex
solution does not necessarily find the We will also, later,
address a special form of
globally optimum solution. non-convex programming
problems called integer
programs. 13
A convex programming problem
We focus on this one, but
Two variables with one min f ( x1 , x2 ) conclusions we derive will also
equality-constraint s.t. h( x1 , x2 ) c apply to the other two. The
benefit of focusing on this one is
that we can visualize it.
14
Contour maps
Definition: A contour map is a. 2-dimensional plane, i.e., a
coordinate system in 2 variables, say, x1, x2, that illustrates
curves (contours) of constant functional value f(x1, x2).
2 2
Example: Draw the contour map for f ( x1 , x2 ) x1 x2
[X,Y] = meshgrid(-
2.0:.2:2.0,-2.0:.2:2.0);
Z = X.^2+Y.^2;
[c,h]=contour(X,Y,Z);
clabel(c,h);
grid;
xlabel('x1');
ylabel('x2');
15
Contour maps and 3-D illustrations
.
2 2
Example: Draw the 3-D surface for f ( x1 , x2 ) x1 x2
[X,Y] = meshgrid(-
2.0:.2:2.0,-2.0:.2:2.0);
Z = X.^2+Y.^2;
surfc(X,Y,Z)
xlabel('x1')
ylabel('x2')
zlabel('f(x1,x2)')
Height is f(x)
Each contour of fixed value f
is the projection onto the x1-
x2 plane of a horizontal slice
made of the 3-D figure at a Contours
value f above the x1-x2 plane.
16
Solving a convex program: graphical analysis
Example: Solve this convex .
program: min f ( x1 , x2 ) x12 x22
s.t. h(x1,x2 ) x1 x2 6
A straight line is a convex set because a line
segment between any two points on it remain on it.
x1 x2 6 x2 x1 6
Superimpose this
relation on top of the
contour plot for f(x1,x2).
“Just touch”:
The two curves are tangent to one another at the solution point.
18
Solving a convex program: graphical analysis
The two curves are tangent to one another at the solution point.
.
The normal (gradient)
vectors of the two curves, at
the solution (tangent) point,
are parallel.
This means the following
two vectors are parallel:
*
2x
{ f ( x1 , x2 )*} x12 x
2 *
2 1
2 x2
*
1
{h( x1 , x2 ) * 6} x1 x2 6 *
1
“Parallel” means that the two vectors have the same direction. We do not know that
they have the same magnitude. To account for this, we equate with a “multiplier” λ:
f ( x , x ) h(x ,x ) c
1 2
*
1 2
*
19
Solving a convex program: graphical analysis
*
f ( x1 , x2 ) h(x1,x2 ) c *
.
Moving everything to the left:
f ( x1 , x2 )* h(x1,x2 )* c 0 Alternately:
f ( x1 , x2 )* c h(x1,x2 )* 0
In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?
20
Solving a convex program: analytical analysis
In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?
*
x f ( x , x ) h(x ,x ) c
1 2 1 2
0
1
f ( x1 , x2 ) h(x1,x2 ) c 0
x2
NO! Because we only have 2 equations, yet 3 unknowns: x1, x2, λ.
So we need another equation. Where do we get that equation?
Recall our equality constraint: h(x1, x2)-c=0 . This must be satisfied!
Therefore:
*
x f ( x1 , x 2 ) h(x ,x
1 2 ) c
1 0 Three equations,
f ( x , x ) h(x ,x ) c 0 three unknowns,
x2 1 2 1 2
we can solve.
0
h( x1 , x2 ) c
21
Solving a convex program: analytical analysis
Observation: The three equations are simply partial derivatives of the
f ( x1 , x2 ) h(x1 ,x2 ) c
function
*
x f ( x1 , x2 ) h(x1,x2 ) c
1 0
f ( x , x ) h(x ,x ) c 0
x2 1 2 1 2
0
h( x1 , x2 ) c
This is obviously true for the first two equations , but it is not so
obviously true for the last one. But to see it, observe
f ( x1 , x2 ) h(x1,x2 ) c 0
h(x1,x2 ) c 0 h(x1,x2 ) c
22
Formal approach to solving our problem
Define the Lagrangian function:
L ( x1 , x2 , ) f ( x1 , x2 ) h(x1 ,x2 ) c
In a convex programming problem, the “first-order conditions” for
finding the solution is given by
L ( x1 , x2 , ) 0
L ( x1, x2 , ) 0
x1
L ( x, ) 0
L ( x1, x2 , ) 0
Or more x
OR x2 compactly
L ( x, ) 0
L ( x1, x2 , ) 0
where we have
used x=(x1, x2)
23
Applying to our example
Define the Lagrangian function:
L ( x1 , x2 , ) f ( x1 , x2 ) h(x1,x2 ) c
2
1
2
2
x x x1 x2 6
L ( x1 , x2 , ) 0
L ( x1, x2 , ) 0 L ( x1 , x2 , ) 2 x1 0
x1 x1
OR L ( x1, x2 , ) 0 L ( x1 , x2 , ) 2 x2 0
x2 x2
L ( x1, x2 , ) 0
L ( x1 , x 2 , ) x1 x2 6 0
A set of 3 linear equations and 3 unknowns;
we can write in the form of Ax=b. 24
Applying to our example
2 0 1 x1 0
0 2 1 x 0
2
1 1 0 6
1
x1 2 0 1 0 1.2247
x2 0 2 1 0 1.2247
1 1 0 6 2.4495
25
Now, let’s go back to our example with a
nonlinear equality constraint.
Example with nonlinear equality
Non-convex because a line connecting two points. in the min f ( x1 , x2 ) x12 x22
set do not remain in the set. (see “notes” of this slide)
s.t. h(x1,x2 ) 2 x1 x2 3
3
2 x1x2 3 x2
2 x1
Superimpose this
relation on top of the
contour plot for f(x1,x2).
“Just touch”:
The two curves are tangent to one another at the solution point.
28
Example with nonlinear equality
The two curves are tangent to one another at the solution point.
.
The normal (gradient)
vectors of the two curves, at
the solution (tangent) point,
are parallel.
This means the following
two vectors are parallel:
*
2x
{ f ( x1 , x2 )*} x12 x 2
2 *
1
2 x2
*
2 x2
{h( x1 , x2 ) * 3} 2 x1 x2 3
*
2 x1
“Parallel” means that the two vectors have the same direction. We do not know that
they have the same magnitude. To account for this, we equate with a “multiplier” λ:
f ( x , x ) h(x ,x ) c
1 2
*
1 2
*
29
Example with nonlinear equality
This gives us the following two equations.
*
x f ( x , x ) h(x ,x ) c
1 2 1 2
0
1
f ( x1 , x2 ) h(x1,x2 ) c 0
x2
L ( x1 , x2 , ) 0
L ( x1, x2 , ) 0 L ( x1 , x2 , ) 2 x1 2x2 0
x1 x1
OR L ( x1, x2 , ) 0 L ( x1 , x2 , ) 2 x2 2x1 0
x2 x2
L ( x1, x2 , ) 0 L ( x1 , x2 , ) 2 x1 x2 3 0
3
x1 x2 1.2247
You can solve this algebraically to obtain 2 and f=3 in
3 both cases
x1 x 2 1.2247
2 31
Example with nonlinear equality
Our approach worked in this case, i.e., we
found a local optimal point that was also a
global optimal point, but because it was
not a convex programming problem, we
had no guarantee that this would happen.
The conditions we established, below, we call first order conditions.
For convex programming problems, they are first order sufficient conditions to provide
the global optimal point.
For nonconvex programming problems, they are first order necessary conditions to
provide the global optimal point.
L ( x, ) 0
x
L ( x, ) 0
Multiple equality constraints
min f ( x) We assume that f and
h are continuously
s.t. h( x) c differentiable.
L ( x, ) f ( x) 1 h1( x ) c1 2 h2( x ) c2
... m hm( x ) cm
First order necessary conditions that (x*, λ*) solves the above:
L(x , ) 0
* *
x
L(x , ) 0
* *
33
Multiple equality & 1 inequality constraint
min f ( x) We assume that f, h, and
s.t. h( x ) c g are continuously
differentiable.
g ( x) b
Solution approach:
•Ignore the inequality constraint and solve the problem.
(this is just a problem with multiple equality constraints).
•If inequality constraint is satisfied, then problem is solved.
•If inequality constraint is violated, then the inequality constraint
must be binding inequality constraint enforced with equality:
g ( x) b
Let’s look at this new problem where the inequality is
binding.
34
Multiple equality & 1 inequality constraint
min f ( x) We assume that f, h, and
s.t. h( x ) c g are continuously
differentiable.
g ( x) b
L ( x, , ) f ( x) 1 h1( x ) c1 2 h2( x ) c2
... m hm( x ) cm g ( x) b
First order necessary conditions that (x*, λ*, μ*) solves the above:
L(x , , *) 0
* *
35
Multiple equality & 1 inequality constraint
L ( x, , ) f ( x) 1 h1( x ) c1 2 h2( x ) c2
... m hm( x ) cm g ( x) b
First order necessary conditions that (x*, λ*, μ*) solves the above:
L (x , , ) 0
* * *
Pi P i Pi P i
where :
Pi P min min generation level
i 1 L C 2 P 2
Subject to 0 2 2 0
n P 2 P 2
P
i 1
i PT L
0 P1 P 2 PT 0
Pi Pi h x c 0
1 P1 P1 0, 1 P1 P1 0
2 P 2 P 2 0, 2 P 2 P 2 0
Pi Pi Pi Pi k 0, k 1,2
k 0, k 1,2
Two unit system, Lagrangian function:
L P1, P 2, , 1, 1, 2, 2 C1 P1 C 2 P 2
P1 P2 PT
1 P1 P1 1 P1 P1
P P P P
2 2 2 2 2 2
Unit 1 Unit 2
Generation
Specifications:
Minimum Generation 200 MW 100 MW
Maximum Generation 380 MW 200 MW
Cost Curve Coefficients:
Quadratic Term 0.016 0.019
Linear Term 2.187 2.407
Constant Term 120.312 74.074
h( P1 , P2 ) P1 P 2 400
200 P1 380 P1 380, P1 200
100 P 2 200 P 2 200, P2 100
LaGrangian function
L P1, P 2, , 1, 1, 2, 2 0.016 P1 2.187 P1 120.312
2
P1 P 2 400
1 P1 200 1 P1 380
2 P 2 100 2 P 2 200
KKT conditions
L
0 0.032 P1 2.187 1 1 0
P1
L
0 0.038 P 2 2.407 2 2 0
P 2
L
0 P1 P 2 400 0
g x c 0
1 P1 200 0, 1 P1 380 0
2 P 2 100 0, 2 P 2 200 0
Assume all inequality constraints are non-binding.
This means that
i 0 and i 0 i 1, n
And KKT conditions become
Solution yields:
P1 220.29
P 2 179.71
9.24
What is = $9.24/MW-hr ???
It is the system “incremental cost.”
C 2 P 2 1120 .25 $ / hr
Total cost/hr are C1+C2
CT C1 P1 C 2 P 2 1378.53 1120.25 2498.78
Now solve EDC for Pd=401 MW to get P1,P2
0.032 0 1 P1 2.187
0 0.038 1 P 2 2.407
1 1 0 401
P1 220.83
P 2 180.17
9.25
Get cost/hr for each unit.
C1 P1 0.016 220.83 2.187 220.83 120.312
2
C 2 P 2 1124 .51 $ / hr
Total cost/hr are C1+C2
CT C1 P1 C 2 P 2 1383.52 1124.51 2508.03