You are on page 1of 13

Chapter 7 Optimization Principles

7.1

Introduction

Optimization is the process of ﬁnding conditions which which give maximum or mini- mum values of a particular function. It is important to understand that optimization is done within the conﬁnes of a concept. We address trade oﬀs or give and take be- haviour in a system to determine the best operating point. In some cases, even simple objectives entail complicated analysis. We will focus on some simple methods which can be applied to a host of engineering problems with limited use of mathematical tools such as software.

7.1.1 The General Optimization Problem

In the mathematical statement of optimization, there is a single function:

(7.1)

which is referred to as the objective function and the x i are the independent variables. It is important that this function be stated as simply as possible. Often, it may require several additional equations to deﬁne variables which appear in the basic statement. As an example, consider the optimization of a heat exchanger. The objective may be to maximize the rate of heat transfer. Thus the objective function will involve an expression which describes the heat transfer rate in a heat exchanger such as an NT U relationship. This in turn requires the overall heat transfer coeﬃcient, which is deﬁned in terms of the ﬂuid stream heat transfer coeﬃcients, and so on. The objective function may also be subject to constraints of the form:

y = f(x 1 , x 2 , x 3 , x 4 , x 4

x n )

(min/max)

 g i (x 1 , x 2 , x 3 , x 4 , x 4 x n ) = 0 i = 1 . n (7.2) h j (x 1 , x 2 , x 3 , x 4 , x 4 x n ) ≤ 0 j = 1 m

where g i represent equality constraints and h j represent inequality constraints. A given problem may be unconstrained or constrained. We will examine both types of

143

144

Mechanical Equipment and Systems

problems and methods to solve them. It is also important to state the constraints in a standard format as given above. This will be more evident when we examine the Lagrange Multiplier method in a later section. If a problem only contains linear equations, then it is often denoted as a linear programming problem (LP), while if one or more equations in the problem statement are non-linear, then it is referred to as a non-linear programming problem (NLP). Unfortunately, many engineering optimization problems fall into the non-linear cate- gory. The ﬁeld of optimization is very broad. We will examine only select methods which utilize calculus (gradient) methods using analytical, approximate analytical, and numerical methods.

7.1.2 Basic Steps in an Optimization Problem

The following steps summarize the process of undertaking an optimization:

Analyze the process or system such that all of the speciﬁc characteristics and variables are deﬁned.

Determine the criterion for optimization and specify the objective function in terms of the above variables.

Develop a mathematical model for the process or system that relates the in- put/output variables using well known physical principles (mass, energy, mo- mentum etc.) and empirical relationships.

If the problem is too large in scope, then break it up into smaller parts, or simplify the objective and the model.

Apply a suitable optimization technique.

Check answers and examine sensitivity to changes in coeﬃcients and assump- tions.

7.2 Unconstrained Optimization Problems and the Gradient Method

Unconstrained optimization problems are the most simply stated. This is not to say that they do not have constraints. It merely implies that the objective function and any explicitly stated equality constraints have been back substituted into the original objective function which leads to a new objective function which is now unconstrained. This approach may not always possible, but attempts at it should always be made ﬁrst. For these other types of problems we require additional methods to be discussed in the next section.

Optimization Principles

145

In an unconstrained problem the principle result is that we must solve:

φ = f(x 1 , x 2 , x 3 ,

x n )

(min/max)

(7.3)

The means to a solution is to consider the gradient of the function deﬁned as:

φ = 0

(7.4)

The gradient of a scalar function is deﬁned as the vector:

φ =

n

i=1

δ

i

∂φ

∂x i

(7.5)

where δ i is a unit vector. If we consider a simple three dimensional function then the gradient is simply:

(7.6)

A necessary condition for a minimum or maximum is that the gradient equal zero, i.e. no slope! So a minimum would be a valley in three dimensional space, and a maximum a hill. Therefore, we must ﬁnd a solution to the following set of equations:

+ j

φ = i

∂φ

∂x

∂φ

∂y

+ k

∂φ

∂z

∂φ

∂x

1

∂φ

∂x 2

.

.

.

∂φ

 

= 0

= 0

= 0

(7.7)

∂x n We will now pursue some fundamental applications. These include but are not limited to: optimization of pumping systems, the design of heat exchangers and other devices, multi-stage compression, design of tanks and structures, etc. We shall also consider other examples in the class notes.

7.3 Constrained Optimization Problems and the Method of Lagrange Multipliers

The general theory for constrained multivariable optimization may be found in Reklaitis et al. (1983), Edgar and Himmelblau (1988), Stoecker (1989) and Winston (1990). The method of Lagrange multipliers may be easily applied to constrained multivari- able applications. The general constrained Non-linear Programming (NLP) problem takes the form:

φ = f(x 1 , x 2 , x 3 ,

x n )

(min/max)

(7.8)

 146 Mechanical Equipment and Systems Subject to

g j (x 1 , x 2 , x 3 , h j (x 1 , x 2 , x 3 ,

, x n ) = 0 , x n ) 0

j

j = m + 1, .

= 1,

, m

.

.

, p

(7.9)

where g j and h j are imposed constraints. It is often more convenient to consider the Langrangian form of the NLP in the following manner. A new objective function is deﬁned as follows:

L(x 1

x n , λ 1

λ p , σ 1

σ pm ) = φ(x i )+

m p

j=1

λ j g j (x i )+

k=m+1

2

λ k (h k (x i )σ k ) (7.10)

where λ j are Lagrange multipliers and σ j are slack variables. The use of slack variables enables the Lagrange multiplier method to be applied to problems with inequality constraints. The problem is now reduced to solving the system of equations deﬁned by

∂L

∂x i

∂L

∂λ

∂L

j

 = 0 i = 1, . . . , n = 0 j = 1, , p = 0 k = 1, , p − m

(7.11)

∂σ j The above system may be solved using numerical methods such as a the mul- tivariable Newton-Raphson method. The constrained formulation for NLP’s with inequality constraints can become quite complex. Given an NLP with n variables and p constraints with p m inequality constraints, optimization of the Lagrangian requires similtaneous solution of a system of n + 2p m equations. In most problems, the number of constraints prescribed should be judiciously chosen. For example, it is not always necessary to prescribe that all x i > 0. In most problems, an optimal solution with x i > 0 may be obtained if a reasonable initial guess is made while leaving the particular x i unconstrained. While in other problems, constraints such as x i < x c may not be necessary if the optimal solution returns x i < x c when x i are unconstrained.

7.4 Newton-Raphson Method in Optimization

The Newton-Raphson method for a single variable was introduced in the last chapter. The method is useful in optimization problems provided the objective function is twice diﬀerentiable. Given an objective function f (x) and solution function:

g(x) =

df dx = 0

(7.12)

Optimization Principles

147

the solution for a variable x is found by making an initial guess x i and solving the following equation:

x i+1 = x i g(x i )

g (x i )

= x i f (x i )

f

(x i )

(7.13)

for the improved solution x i+1 . The procedure is repeated until desired convergence is achieved, which usually occurs in fewer than three iterations. If our objective function is multivariable, the Newton-Raphson method can also

, x n ), we must solve the

be applied. Given an objective function f (x 1 , x 2 , x 3 , x 4 , system:

 g 1 (x 1 , x 2 , x 3 , x n ) = g 2 (x 1 , x 2 , x 3 , x n ) = g 3 (x 1 , x 2 , x 3 , x n ) = . . . g n (x 1 , x 2 , x 3 , x n ) =

df

dx

1

df

dx

2

df

dx 3

df

dx n

=

=

0

0

= 0

= 0

(7.14)

To solve the above system we may write the general non-linear system in a linearized form using a truncated Taylor series expansion of each equation:

∂g 1

∂x

∂g

1

2

∂x

∂g

1

3

∂x

.

.

.

∂g n

1

∂x 1

∂g 1

g 1

 ··· ∂x 2 ∂x n ∂g 2 ∂g 2 ··· ∂x 2 ∂x n ∂g 3 ∂g 3 ··· ∂x 2 ∂x n . . ∂g n ∂g n ··· ∂x 2 ∂x n

x o

x

x

x

.

.

.

1

2

3

x n

=

g

g

g

.

.

.

1

2

3

g

n

x o

(7.15)

148

or in terms of the original objective function:

2 f

∂x 2

2 f

1

∂x 2 ∂x 1

2 f

∂x 3 ∂x 1

.

.

.

2 f

∂x n ∂x 1

2 f

∂x 1 ∂x 2

2 f

∂x 2

2 f

2

∂x 3 ∂x 2

.

2 f

∂x n ∂x 2

2 f

 ··· ∂x 1 ∂x n ∂ 2 f ··· ∂x 2 ∂x n ∂ 2 f ··· ∂x 3 ∂x n . ∂ 2 f ···

∂x

2

n

x o

Mechanical Equipment and Systems

x

x

x

.

.

.

1

2

3

x n

=

∂f

∂x 1

∂f

∂x 2

∂f

∂x 3

.

.

.

∂f

∂x n

x o

(7.16)

We may now solve the above above matrix as before, repeating iterations as required until desired convergence of the solution is achieved. At each stage of iteration the solution is reﬁned according to:

x i,new = x i,old x i

(7.17)

The matrix of second order partial derivatives is also referred to as the Hessian matrix. It will be utilized in the next section to determine the optimality of a solution, i.e. minimum, maximum, or saddle point.

7.5 Tests for Optimality

Up to this point we have not yet determined whether an obtained solution is indeed maximum or minimum. In order to check whether or not a given solution is maximum or minimum, we must return to the calculus of optimization.

7.5.1 Single Variable Functions

Given a single variable function f (x), whose optimal solution x = x opt satisﬁes:

df

dx

x=x opt

= 0

(7.18)

the test for optimality requires that we examine the value of the second derivative of f (x), such that:

d 2 f

dx

2

d 2 f

dx 2

x=x opt

x=x opt

 ≥ 0 min (7.19) ≤ 0 max (7.20)

Optimization Principles

149

Further, if the second derivative is equal to zero:

d 2 f

dx 2

x=x opt

= 0

(7.21)

then we must consider further tests. Given a solution x opt which is referred to as

a stationary point, that is it satisﬁes the condition that f (x opt ) = 0, and the ﬁrst non-zero higher order derivative denoted by n, i.e. f n (x) = d n f/dx n , then:

(i) If n is odd, then x opt is an inﬂection point (neither min nor max) (ii) If n is even, then x opt is a local optimum and if f n (x opt ) > 0 then x opt

is a local minimum or if f n (x opt ) < 0 then x opt is a local maximum.

These are the necessary and suﬃcient conditions for determining the state of the stationary point.

7.5.2 Multi-variable Functions

In the case of a multi-variable function, the procedure is somewhat more complicated. We must determine the eigenvalues of the Hessian matrix which is evaluated at the stationary point (the potential optimal solution). Recall that the Hessian matrix is deﬁned as:

H(x) =

2 f

∂x 2

2 f

1

∂x 2 ∂x 1

2 f

∂x 3 ∂x 1

.

.

.

2 f

∂x n ∂x 1

2 f

∂x 1 ∂x 2

2 f

∂x 2

2 f

2

∂x 3 ∂x 2

.

2 f

∂x n ∂x 2

···

···

···

···

2 f ∂x 1 ∂x n 2 f ∂x 2 ∂x n 2 f ∂x 3 ∂x n

.

2 f ∂x

2

n

(7.22)

If we denote the stationary point of a multi-variable function:

(7.23)

then the eigenvalues of the Hessian matrix are determined by solving the following equation:

x = [x 1 , x 2 , ··· , x n ] opt

(7.24)

where λ are the eigenvalues and I is the identity matrix. The determinant of the n × n matrix yields an n th order polynomial equation in λ which has n real roots or eigenvalues.

det(H(x ) λI) = 0

150

Mechanical Equipment and Systems

Once the eigenvalues have been determined, the following criteria determine whether we have a minimim, maximum, or saddle point (multi-dimensional inﬂection point). If λ i are the eigenvalues of the Hessian matrix the the following apply:

i) H(x ) is positive deﬁnite if all λ i > 0 and the stationary point is a minimum

ii) H(x ) is positive semi-deﬁnite if all λ i 0 with at least one λ i = 0, and

the stationary point is a minimum

iii) H(x ) is negative deﬁnite if all λ i < 0 and the stationary point is a maximum

iv) H(x ) is negative semi-deﬁnite if all λ i 0 with at least one λ i = 0, and

the stationary point is a maximum

v) H(x ) is indeﬁnite if some λ i < 0 and λ i > 0, and

the stationary point is a saddle point

Optimization Principles

PROBLEMS

151

Problem 7.1 Consider the design of a large steel storage tank of diameter D and height H. The goal is to maximize the volume V or capacity of the tank for a ﬁxed amount of material (or surface area, A).

Solution The mathematical statement of the objective function is:

subject to

V

= πD 2 H

4

A

=

πD 2

2

 → maximize (7.25) + πDH (7.26)

It should be stated that this is a relatively trivial example, since we have not really said how the material of area A is being utilized. For example, if the material was a large sheet, how are the ends and sides to be cut from this sheet? We may combine the two equations to solve the problem as a simpler unconstrained problem by writing the height H of the tank as:

H

=

A πD 2

2

πD This gives the following objective function:

(7.27)

πD 3

(7.28)

We can now ﬁnd the optimal diameter by taking the derivative of the objective function, equating to zero, and solving:

V

4 8

dV

dD = A

4

 which gives: D opt and H opt

=

=

3πD 2

8

2A

3π

2A

3π

= 0

(7.29)

(7.30)

(7.31)

The maximum volume of the tank can be found to be:

V max = π

4

2A

3π

3/2

(7.32)

152

Mechanical Equipment and Systems

The aspect ratio of the tank is H/D = 1. Additional constraints requiring the aspect ratio to lie within a particular range could also have been speciﬁed. We have just seen from Example 7.1, how to formulate an optimization problem. In the optimization of engineering systems, many diﬀerent objectives arise. These include: minimize ﬁrst cost of a system, minimize life time costs, maximize proﬁts, minimize or maximize heat transfer, minimize pumping power, minimize weight, min- imize or maximize work transfer, minimize surface area, maximize volume, minimize pressure drop, minimize entropy generation, etc.

Problem 7.2 Re-examine the tank problem in Example 7.1. This time consider the volume to be ﬁxed. We desire to obtain the system which has the least surface area A. This could be for weight purposes or for heat transfer reduction. Formulate the objective function for minimum surface area, minimum weight, and minimum cost.

Problem 7.3

Consider the function given by

f(x) = 5x 6 36x 5 + 165 x 4 60x 3 + 36

2

(7.33)

Find all the stationary points and determine whether they are minimum, maximum,

or inﬂection points.

Problem 7.4

Given the multi-variable function

f (x, y) =

2

xy + x 3 + 4y 2 + 2x 2 xy + 2x 4y

(7.34)

determine the stationary point using Newton-Raphson iteration and determine whether it is a minimum, maximum, or saddle point.

Problem 7.5 - Pumping System

Consider the classic problem of pumping ﬂuid between two points through a circular pipe. We desire to ﬁnd the optimal pipe diameter which minimizes the total cost of the constructing and operating the system, i.e.

C T = C c + C o

(7.35)

Assume that the cost of pump and piping is of the form:

C c = C 0 + C 1 D n L

and the operation cost of the system is of the form:

C o = C 2 m˙ ∆p ρη

(7.36)

(7.37)

Optimization Principles

153

Further, assume that the pressure drop is calculated from:

and that

p = 4fL 2 ρU 2

1

D

(7.38)

(7.39)

The constants C 0 , C 1 , and C 2 are cost indices for the pump, the piping, and the electricity to run the pump. Modify the analysis to consider a pump cost which is proportional to pipe diameter.

Problem 7.6 - Shell and Tube Heat Exchanger

A shell and tube heat exchanger requires a total length of tubing to be equal to 100 m based on heat transfer needs. The costs of the design of the heat exchanger are as follows:

f = 0.046Re

D

1/5

1) tube cost \$900.00 2) shell cost \$1100D 5/2 L 3) cost of space allocated to \$320DL

Here D is the diameter of the shell and L is the length of the shell. The pre-selected tube arrangement allows for up to 200 tubes/m 2 inside the shell cross-section. Find the optimal D and L of the heat exchanger for minimum cost. Solve the problem as an unconstrained problem. Use a conventional calculus approach and the intersection of asymptotes method. Discuss the accuracy of the two solution approaches.

Problem 7.7 - Multi-stage Compression System

Consider a multi-stage compression system containing three compressors and two intercoolers. The ideal work of each compression cycle (see Chapter 4), gives:

W c =

1 p o

kRT i

k

p

i

(k1)/k 1

(7.40)

If p 1 = 100 kP a and p 4 = 1000 kP a, what are the optimal values of p 2 and p 3 such that the total work required is a minimum. Assume k = 1.4 for air and T 1 = 298 K. Verify the solution is indeed a minimum.

Problem 7.8

Solve the following problem using Lagrange multipliers:

subject to:

f = x 2 2y y 2

(7.41)

154

Mechanical Equipment and Systems

Problem 7.9 - Shell and Tube Heat Exchanger Re-solve Example 7.6 using the Lagrange multiplier method.

Optimization Principles

7.6

References

155

Arora, J.S., Introduction to Optimum Design,1989, McGraw-Hill, New York, NY. Bejan, A., G. Tsatsaronis, and Moran, M., Thermal Design and Optimization, 1996, Wiley, New York, NY. Beveridge, G.S.G. and Schechter, R.S., Optimization: Theory and Practice, 1970, McGraw-Hill, New York, NY. Boehm, R., Design Analysis of Thermal Systems, Wiley, 1987. Burmeister, L.C., Elements of Thermal Fluid System Design, 1997, Prentice-Hall, Upper Saddle River, NJ. Edgar, T.F. and Himmelblau, D.M., Optimization of Chemical Processes, 1988, McGraw-Hill, New York, NY. Jaluria, Y., Design and Optimization of Thermal Systems, 1998, McGraw-Hill, New York, NY. Rao,S.S., Engineering Optimization: Theory and Practice, 1996, Wiley, New York, NY. Reklaitis, G.V., Ravindran, A., and Ragsdell, K.M., Engineering Optimiza- tion: Methods and Applications, 1983, Wiley, New York, NY. Stoecker, W.F., Design of Thermal Systems, 1989, McGraw-Hill, New York, NY. Winston, W.L., Operations Research: Applications and Algorithms, PWS-Kent, 1990, Boston.