Professional Documents
Culture Documents
1. Solution of Equations
1.1. Matrix Properties
Mathematical models of chemical engineering
systems can take many forms: they can be sets A matrix is a set of real or complex numbers
of algebraic equations, differential equations, arranged in a rectangular array.
and/or integral equations. Mass and energy bal-
a11 a12 ... a1n
ances of chemical processes typically lead to a21 a22 ... a2n
large sets of algebraic equations:
A=
.. .
.
.
.
.
.
. . . .
a11 x1 +a12 x2 = b1
a21 x1 +a22 x2 = b2 am1 am2 ... amn
Mass balances of stirred tank reactors may lead The numbers aij are the elements of the matrix
to ordinary differential equations: A, or (A)ij = aij . The transpose of A is (AT ) =
aji .
dy
= f [y (t)] The determinant of a square matrix A is
dt
Radiative heat transfer may lead to integral a11 a12 ... a1n
a21 a22 ... a2n
equations:
A= . . . .
1 .. .
.
.
.
.
.
y (x) = g (x) +λ K (x, s) f (s) d s a
n1 an2 ... ann
0
If the i-th row and j-th column are deleted, a
Even when the model is a differential equa- new determinant is formed having n−1 rows and
tion or integral equation, the most basic step in columns. This determinant is called the minor of
the algorithm is the solution of sets of algebraic aij denoted as M ij . The cofactor A′ij of the el-
equations. The solution of sets of algebraic equa- ement aij is the signed minor of aij determined
tions is the focus of Chapter 1. by
A single linear equation is easy to solve for
either x or y: A′ij =(−1)i+j Mij
it may be more difficult to find the x satisfying where the elements aij must be taken from a
this equation. These problems are compounded single row or column of A.
when there are more unknowns, leading to si- If all determinants formed by striking out
multaneous equations. If the unknowns appear whole rows or whole columns of order greater
in a linear fashion, then an important considera- than r are zero, but there is at least one determi-
tion is the structure of the matrix representing the nant of order r which is not zero, the matrix has
equations; special methods are presented here rank r.
6 Mathematics in Chemical Engineering
The value of a determinant is not changed if 0 if j<i−1
the rows and columns are interchanged. If the aij = aij otherwise
0 if j>i+1
elements of one row (or one column) of a de-
terminant are all zero, the value of the deter- Block diagonal and pentadiagonal matrices also
minant is zero. If the elements of one row or arise, especially when solving partial differential
column are multiplied by the same constant, the equations in two- and three-dimensions.
determinant is the previous value times that con-
stant. If two adjacent rows (or columns) are in- QR Factorization of a Matrix. If A is an m
terchanged, the value of the new determinant is × n matrix with m ≥ n, there exists an m × m
the negative of the value of the original determi- unitary matrix Q = [q1 , q2 , . . .qm ] and an m × n
nant. If two rows (or columns) are identical, the right-triangular matrix R such that A=QR. The
determinant is zero. The value of a determinant QR factorization is frequently used in the ac-
is not changed if one row (or column) is multi- tual computations when the other transforma-
plied by a constant and added to another row (or tions are unstable.
column).
A matrix is symmetric if Singular Value Decomposition. If A is an
aij =aji m × n matrix with m ≥ n and rank k ≤ n, con-
sider the two following matrices.
and it is positive definite if
n n AA∗ and A∗ A
T
x Ax= aij xi xj ≥0
i=1j=1
An m × m unitary matrix U is formed from the
eigenvectors ui of the first matrix.
for all x and the equality holds only if x = 0.
If the elements of A are complex numbers, A* U = [u1 ,u2 ,. . .um ]
denotes the complex conjugate in which (A*)ij
An n × n unitary matrix V is formed from the
= a*ij . If A=A* the matrix is Hermitian.
eigenvectors vi of the second matrix.
The inverse of a matrix can also be used to
solve sets of linear equations. The inverse is a V =[v1 ,v2 ,. . .,vn ]
matrix such that when A is multiplied by its in-
verse the result is the identity matrix, a matrix Then the matrix A can be decomposed into
with 1.0 along the main diagonal and zero else- A=U ΣV ∗
where.
where Σ is a k × k diagonal matrix with diagonal
AA−1 =I elements dii =σi >0 for 1 ≤ i ≤ k. The eigenval-
If AT =A−1 the matrix is orthogonal. ues of Σ ∗ Σ are σi2 . The vectors ui for k+1 ≤ i ≤
Matrices are added and subtracted element m and vi for k+1 ≤ i ≤ n are eigenvectors asso-
by element. ciated with the eigenvalue zero; the eigenvalues
for 1 ≤ i ≤ k are σi2 . The values of σi are called
A+B is aij +bij the singular values of the matrix A. If A is real,
Two matrices A and B are equal if aij = bij . then U and V are real, and hence orthogonal ma-
Special relations are trices. The value of the singular value decompo-
sition comes when a process is represented by a
(AB)−1 =B −1 A−1 , (AB)T =B T AT
T −1 linear transformation and the elements of A, aij ,
A−1 = AT , (ABC)−1 =C −1 B −1 A−1
are the contribution to an output i for a partic-
A diagonal matrix is zero except for elements ular variable as input variable j. The input may
along the diagonal. be the size of a disturbance, and the output is
the gain [1]. If the rank is less than n, not all the
aii , i=j
aij = variables are independent and they cannot all be
0, i=j
controlled. Furthermore, if the singular values
A tridiagonal matrix is zero except for elements are widely separated, the process is sensitive to
along the diagonal and one element to the right small changes in the elements of the matrix and
and left of the diagonal. the process will be difficult to control.
Mathematics in Chemical Engineering 7
the condition number can be lengthy so another The LU decomposition algorithm for solving
criterion is also useful. Compute the ratio of the this set is
largest to the smallest pivot and make judgments
b′1 = b1
on the ill-conditioning based on that.
for k = 2, n do
When a matrix is ill-conditioned the LU
a′k = b′ak , b′k = bk − ak
b′k−1
ck−1
decomposition must be performed by using piv- k−1
Then set
A−1 = x(1) |x(2) |x(3) |···|x(n)
matrices, arrow matrices, etc. These structures where A is symmetric and positive definite, is
typically arise from numerical analysis applied to be solved. A preconditioning matrix M is de-
to solve differential equations. Other problems, fined in such a way that the problem
such as modeling chemical processes, lead to Mt = r
sparse matrices but without such a neatly defined
structure — just a lot of zeros in the matrix. For is easy to solve exactly (M might be diagonal,
matrices such as these, special techniques must for example). Then the preconditioned conju-
be employed: efficient codes are available [4]. gate gradient method is
These codes usually employ a symbolic factor- Guess x0
ization, which must be repeated only once for Calculate r0 = f − Ax0
each structure of the matrix. Then an LU factor-
Solve M t0 = r0 , and set p0 = t0
ization is performed, followed by a solution step
using the triangular matrices. The symbolic fac- for k = 1, n (or until convergence)
T
torization step has significant overhead, but this ak =
rk tk
pT Apk
is rendered small and insignificant if matrices k
xk+1 = xk +ak pk
with exactly the same structure are to be used
over and over [5]. rk+1 = rk − ak Apk
The efficiency of a technique for solving sets Solve M tk+1 = rk+1
of linear equations obviously depends greatly on T
rk+1 tk+1
bk = Tt
the arrangement of the equations and unknowns rk k
dient method because it converges faster, pro- where there are n species and m reactions. Then
vided a good “preconditioner” can be found. The if a matrix is formed from the coefficients αij ,
system of n linear equations which is an n×m matrix, and the rank of the ma-
trix is r, there are r independent chemical reac-
Ax = f tions. The other n − r reactions can be deduced
from those r reactions.
10 Mathematics in Chemical Engineering
1.3. Nonlinear Algebraic Equations successive iterates is something like 0.1, 0.01,
0.0001, 10−8 . The error (when it is known) goes
Consider a single nonlinear equation in one un- the same way; the method is said to be quadrati-
known, cally convergent when it converges. If the deriva-
tive is difficult to calculate a numerical approx-
f (x) = 0
imation may be used.
In Microsoft Excel, roots are found by using df f xk +ε −f xk
Goal Seek or Solver. Assign one cell to be x, | k =
dx x ε
put the equation for f (x) in another cell, and let
Goal Seek or Solver find the value of x making In the secant method the same formula is
the equation cell zero. In MATLAB, the process used as for the Newton – Raphson method, ex-
is similar except that a function (m-file) is de- cept that the derivative is approximated by using
fined and the command fzero(‘f’,x0) provides the values from the last two iterates:
the solution x, starting from the initial guess x0 . df f xk −f xk−1
Iterative methods applied to single equations |xk =
dx xk −xk−1
include the successive substitution method
This is equivalent to drawing a straight line
xk+1 = xk +βf xk ≡g xk through the last two iterate values on a plot of f
(x) versus x. The Newton – Raphson method is
and the Newton – Raphson method. equivalent to drawing a straight line tangent to
f xk the curve at the last x. In the method of false po-
xk+1 = xk − sition (or regula falsi), the secant method is used
df /dx xk
to obtain x k +1 , but the previous value is taken
The former method converges if the derivative as either x k −1 or x k . The choice is made so that
of g(x) is bounded [3]. The latter method the function evaluated for that choice has the
opposite sign to f (x k +1 ). This method is slower
dg
dx (x) ≤µ for |x−α| <h
than the secant method, but it is more robust and
keeps the root between two points at all times. In
is based on a Taylor series of the equation about
all these methods, appropriate strategies are re-
the k-th iterate:
quired for bounds on the function or when df /dx
df
= 0. Brent’s method combines bracketing, bisec-
f xk+1 = f xk + |xk xk+1 −xk +
dx tion, and an inverse quadratic interpolation to
d2 f 1 k+1 k 2 provide a method that is fast and guaranteed to
2
|xk x −x +···
dx 2 converge, if the root can be bracketed initially
The second and higher-order terms are neglected [15, p. 251].
and f (x k +1 ) = 0 to obtain the method. In the method of bisection, if a root lies bet-
df
f x0
ween x 1 and x 2 because f (x 1 ) < 0 and f (x 2 ) >
>0, x1 −x0 =
dx 0 0
≤b,
0, then the function is evaluated at the center, x c =
df /dx (x )
x 0.5 (x 1 + x 2 ). If f (x c ) > 0, the root lies between
0
x
2
d f
and 2 ≤c
x 1 and x c . If f (x c ) < 0, the root lies between x c
dx and x 2 . The process is then repeated. If f (x c ) =
Convergence of the Newton–Raphson method 0, the root is x c . If f (x 1 ) > 0 and f (x 2 ) > 0,
depends on the properties of the first and second more than one root may exist between x 1 and x 2
derivative of f (x) [3, 14]. In practice the method (or no roots).
may not converge unless the initial guess is good, For systems of equations the Newton – Raph-
or it may converge for some parameters and not son method is widely used, especially for equa-
others. Unfortunately, when the method is non- tions arising from the solution of differential
convergent the results look as though a mistake equations.
occurred in the computer programming; distin- fi ({xj }) = 0, 1≤i, j≤n,
guishing between these situations is difficult, so
where {xj } = (x1 , x2 ,. . ., xn ) = x
careful programming and testing are required.
If the method converges the difference between Then, an expansion in several variables occurs:
Mathematics in Chemical Engineering 11
n
∂fi
Raphson method to solve for x. Since the ini-
fi xk+1 = fi xk + |xk xk+1
j −xkj +···
j=1
∂xj tial guess is presumably pretty good, this has a
high chance of being successful. That solution
The Jacobian matrix is defined as
is then used as the initial guess for t = 2 ∆t and
k
Jij =
∂fi the process is repeated by moving stepwise to t
k
∂xj x = 1. If the Newton – Raphson method does not
converge, then ∆t must be reduced and a new
and the Newton – Raphson method is solution attempted.
n Another way of using homotopy is to create
k
Jij xk+1 −xk = −fi xk an ordinary differential equation by differentiat-
j=1 ing the homotopy equation along the path (where
h = 0).
For convergence, the norm of the inverse of
the Jacobian must be bounded, the norm of dh [x (t) ,t] ∂h dx ∂h
= + =0
the function evaluated at the initial guess must dt ∂x dt ∂t
be bounded, and the second derivative must be This can be expressed as an ordinary differential
bounded [14, p. 115], [3, p. 12]. equation for x (t):
A review of the usefulness of solution meth-
∂h dx ∂h
ods for nonlinear equations is available [16]. =−
∂x dt ∂t
This review concludes that the Newton – Raph-
son method may not be the most efficient. Broy- If Euler’s method is used to solve this equation,
den’s method approximates the inverse to the Ja- a value x 0 is used, and dx/dt from the above
cobian and is a good all-purpose method, but a equation is solved for. Then
good initial approximation to the Jacobian ma- dx
x1,0 = x0 +∆t
trix is required. Furthermore, the rate of con- dt
vergence deteriorates for large problems, for
is used as the initial guess and the homotopy
which the Newton – Raphson method is better.
equation is solved for x1 .
Brown’s method [16] is very attractive, whereas
Brent’s is not worth the extra storage and compu- ∂h 1,k+1
x −x1,k = −h x1,k ,t
tation. For large systems of equations, efficient ∂x
software is available [11 – 13]. Then t is increased by ∆t and the process is re-
Homotopy methods can be used to ensure peated.
finding the solution when the problem is espe- In arc-length parameterization, both x and t
cially complicated. Suppose an attempt is made are considered parameterized by a parameter s,
to solve f (x) = 0, and it fails; however, g (x) = which is thought of as the arc length along a
0 can be solved easily, where g (x) is some func- curve. Then the homotopy equation is written
tion, perhaps a simplification of f (x). Then, the along with the arc-length equation.
two functions can be embedded in a homotopy
∂h dx
by taking ∂x ds
+ ∂h dt
∂t ds
=0
2
dxT dx
h (x, t) = t f (x) + (1−t) g (x) dt
ds ds
+ ds
=1
In this equation, h can be a n×n matrix for The initial conditions are
problems involving n variables; then x is a vec-
tor of length n. Then h (x, t) = 0 can be solved x (0) = x0
for t = 0 and t gradually changes until at t = 1, h t (0) = 0
(x, 1) = f (x). If the Jacobian of h with respect The advantage of this approach is that it works
to x is nonsingular on the homotopy path (as even when the Jacobian of h becomes singu-
t varies), the method is guaranteed to work. In lar because the full matrix is rarely singular. Il-
classical methods, the interval from t = 0 to t = lustrations applied to chemical engineering are
1 is broken up into N subdivisions. Set ∆t = 1/N available [17]. Software to perform these com-
and solve for t = 0, which is easy by the choice putations is available (called LOCA) [18].
of g (x). Then set t = ∆t and use the Newton –
12 Mathematics in Chemical Engineering
yn+1,i =
R
xn,i +
1
x0,i
The general solution is the one found for the
R+1 R+1 homogeneous equation, and the particular so-
and the equilibrium equation gives lution is any solution to the nonhomogeneous
difference equation. This can be found by meth-
yn,i = Kn,i xn,i ods analogous to those used to solve differential
If these are combined, equations: the method of undetermined coeffi-
cients and the method of variation of parame-
R 1 ters. The last method applies to equations with
Kn+1,i xn+1,i = xn,i + x0,i
R+1 R+1 variable coefficients, too. For a problem such as
is obtained, which is a linear difference equation.
xn+1 −fn xn = 0
This particular problem is quite complicated,
x0 = c
and the interested reader is referred to [19, Chap.
6]. However, the form of the difference equa- the general solution is
tion is clear. Several examples are given here for n
solving difference equations. More complete in-
xn =c fi−1
formation is available in [20]. i=1
An equation in the form This can then be used in the method of variation
xn+1 −xn =fn+1 of parameters to solve the equation
can be solved by xn+1 −fn xn = gn
n
xn = fi
i=1
is attempted; ϕ is raised to the power n. For ex- If this equation is expanded, it can be represented
ample, the difference equation as
cxn−1 +bxn +axn+1 = 0 Pn (λ) = (−λ)n +a1 (−λ)n−1 +a2 (−λ)n−2 +···
coupled with the trial solution would lead to the +an−1 (−λ) +an = 0
equation If the matrix A has real entries then ai are real
aϕ2 +bϕ+c = 0 numbers, and the eigenvalues either are real
Mathematics in Chemical Engineering 13
numbers or occur in pairs as complex numbers 2) Experimental data must be fit with a math-
with their complex conjugates (for definition of ematical model. The data have experimental
complex numbers, see Chap. 3). The Hamilton error, so some uncertainty exists. The param-
– Cayley theorem [19, p. 127] states that the ma- eters in the model as well as the uncertainty in
trix A satisfies its own characteristic equation. the determination of those parameters is de-
sired.
Pn (A) = (−A)n +a1 (−A)n−1 +a2 (−A)n−2 +···
+an−1 (−A) +an I = 0
These problems are addressed in this chapter.
Section 2.2 gives the properties of polynomi-
A laborious way to find the eigenvalues of als defined over the whole domain and Section
a matrix is to solve the n-th order polynomial 2.3 of polynomials defined on segments of the
for the λi — far too time consuming. Instead the domain. In Section 2.4, quadrature methods are
matrix is transformed into another form whose given for evaluating an integral. Least-squares
eigenvalues are easier to find. In the Givens methods for parameter estimation for both linear
method and the Housholder method the matrix and nonlinear models are given in Sections 2.5.
is transformed into the tridiagonal form; then, in Fourier transforms to represent discrete data are
a fixed number of calculations the eigenvalues described in Section 2.7. The chapter closes with
can be found [15]. The Givens method requires extensions to two-dimensional representations.
4 n3 /3 operations to transform a real symmetric
matrix to tridiagonal form, whereas the House-
holder method requires half that number [14]. 2.2. Global Polynomial Approximation
Once the tridiagonal form is found, a Sturm se-
quence is applied to determine the eigenvalues. A global polynomial Pm (x) is defined over the
These methods are especially useful when only entire region of space
a few eigenvalues of the matrix are desired. m
If all the eigenvalues are needed, the Q R al- Pm (x) = c j xj
gorithm is preferred [21]. j=0
The eigenvalues of a certain tridiagonal ma- This polynomial is of degree m (highest power
trix can be found analytically. If A is a tridiago- is x m ) and order m + 1 (m + 1 parameters {cj }).
nal matrix with If a set of m + 1 points is given,
aii = p, ai,i+1 = q,ai+1,i = r, qr>0 y1 = f (x1 ) , y2 = f (x2 ) ,. . ., ym+1 = f (xm+1 )
then the eigenvalues of A are [22] then Lagrange’s formula yields a polynomial of
iπ degree m that goes through the m + 1 points:
λi = p+2(qr)1/2 cos i = 1,2,. . .,n
n+1 (x−x2 )(x−x3 )···(x−xm+1 )
Pm (x) = y +
(x1 −x2 )(x1 −x3 )···(x1 −xm+1 ) 1
This result is useful when finite difference meth-
(x−x1 )(x−x3 )···(x−xm+1 )
ods are applied to the diffusion equation. y +···+
(x2 −x1 )(x2 −x3 )···(x2 −xm+1 ) 2
The evaluation of Pm (x) at a point other than The orthogonality includes a nonnegative
the defining points can be made with Neville’s weight function, W (x) ≥ 0 for all a ≤ x ≤ b.
algorithm [15]. Let P1 be the value at x of the This procedure specifies the set of polynomials
unique function passing through the point (x 1 , to within multiplicative constants, which can be
y1 ); i.e., P1 = y1 . Let P12 be the value at x of the set either by requiring the leading coefficient to
unique polynomial passing through the points x 1 be one or by requiring the norm to be one.
and x 2 . Likewise, Pijk ...r is the unique polyno- b
mial passing through the points x i , x j , x k , . . . , 2
W (x) Pm (x) dx = 1
x r . The following scheme is used: a
tion at the additional points x i+m and x i . Thus, Note that each cj is independent of m, the num-
Pi(i+1)...(i+m) agrees with the function at all the ber of terms retained in the series. The minimum
points {x i , x i+1 , . . . , x i+m }. value of I is
b m
Orthogonal Polynomials. Another form of Imin = W (x) f 2 (x) dx− Wj c2j
the polynomials is obtained by defining them so a j=0
that they are orthogonal. It is required that Pm
(x) be orthogonal to Pk (x) for all k = 0, . . . , Such functions are useful for continuous data,
m − 1. i.e., when f (x) is known for all x.
Typical orthogonal polynomials are given in
b Table 1. Chebyshev polynomials are used in
W (x) Pk (x) Pm (x) dx = 0
a spectral methods (see Chap. 8). The last two
k = 0,1,2,. . ., m−1 rows of Table 1 are widely used in the orthogo-
nal collocation method in chemical engineering.
Table 1. Orthogonal polynomials [15, 23]
a b W (x) Name Recursion
relation
−1 1 1 Legendre (i+1) Pi+1 =(2 i+1)xPi − i Pi−1
√ 1
−1 1 Chebyshev T i+1 =2xT i −T i−1
1−x2
The last entry (the shifted Legendre polynomial 2.3. Piecewise Approximation
as a function of x 2 ) is defined by
1
Piecewise approximations can be developed
W x2 Pk x2 Pm x2 xa−1 dx = 0 from difference formulas [3]. Consider a case
0
k = 0,1,. . ., m−1
in which the data points are equally spaced
xn+1 −xn = ∆x
where a = 1 is for planar, a = 2 for cylindrical,
and a = 3 for spherical geometry. These func- yn = y (xn )
tions are useful if the solution can be proved to
be an even function of x. forward differences are defined by
∆yn = yn+1 −yn
Rational Polynomials. Rational polynomi-
als are ratios of polynomials. A rational poly- ∆2 yn = ∆yn+1 −∆yn = yn+2 −2yn+1 +yn
nomial Ri(i+1)...(i+m) passing through m + 1 Then, a new variable is defined
points
xα −x0
α=
yi = f (xi ) , i = 1,. . .,m+1 ∆x
into elements as shown in Figure 3. Each func- x in e-th element and ci = ceI within the element
tion N i (x) is zero at all nodes except x i ; N i e. Thus, given a set of points (x i , yi ), a finite ele-
(x i ) = 1. Thus, the approximation is ment approximation can be made to go through
NT NT them.
y (x) = ci Ni (x) = y (xi ) Ni (x) Quadratic approximations can also be used
i=1 i=1 within the element (see Fig. 4). Now the trial
where ci = y (x i ). For convenience, the trial functions are
functions are defined within an element by using
new coordinates:
x−xi
u=
∆xi
The ∆x i need not be the same from element
to element. The trial functions are defined as
N i (x) (Fig. 3 A) in the global coordinate sys-
tem and N I (u) (Fig. 3 B) in the local coordinate
system (which also requires specification of the
element). For x i < x < x i+1
NI=1 = 2 (u−1) u− 12
(3)
NI=2 = 4u (1−u)
NI=3 = 2u u− 21
Figure 3. Galerkin finite element method – linear functions The approximation going through an odd num-
A) Global numbering system; B) Local numbering system ber of points (x i , yi ) is then
3
ceI NI (u) x in e−th element
NT y (x) =
y (x) = ci Ni (x) = ci Ni (x) +ci+1 Ni+1 (x) I=1
i=1 with ceI = y (xi ) ,i = (e−1) 2+I
because all the other trial functions are zero in the e−th element
there. Thus Hermite cubic polynomials can also be used;
y (x) = ci NI=1 (u) +ci+1 NI=2 (u) , these are continuous and have continuous first
derivatives at the element boundaries [3].
xi <x<xi+1 , 0<u<1 Splines. Splines are functions that match
Then given values at the points x 1 , . . . , x NT , shown
in Figure 5, and have continuous derivatives up
NI=1 = 1−u, NI=2 = u to some order at the knots, or the points x 2 , . . . ,
and the expansion is rewritten as x NT −1 . Cubic splines are most common. In this
case the function is represented by a cubic poly-
2
y (x) = ceI NI (u) (2)
nomial within each interval and has continuous
I=1
first and second derivatives at the knots.
Mathematics in Chemical Engineering 17
′′ ′′
yi = Ci (xi ) ,yi′ = Ci′ (xi ) ,yi = Ci (xi )
is defined so that
′′
Ci (x) = yi +yi′ (x−xi ) + 21 yi (x−xi )2 +
′′ ′′
1
6∆xi
yi+1 −yi (x−xi )3
dy dy +yN +1 )+O h3
| = |
dx x=x(2)− dx x=x(2)+ The first five terms in Equation 1 are retained
and integrated over two intervals.
Within each element the interpolation is a poly-
nomial of degree NCOL + 1. Overall the func- x0
+2h 2 h
y (x) dx = yα hdα = (y0 +4y1 +y2 )
tion is continuous with continuous first deriva- x0 0
3
Within each pair of intervals the interpolant is When orthogonal polynomials are used, as in
continuous with continuous derivatives, but only Equation 1, the m roots to Pm (x) = 0 are cho-
the function is continuous from one pair to an- sen as quadrature points and called points {x j }.
other. Then the quadrature is Gaussian :
If the finite element representation is used 1 m
(Eq. 2), the integral is y (x) dx = Wj y (xj )
xi+1 0 j=1
1
2
ceI NI (u) (xi+1 −xi ) du
y (x) dx = The quadrature is exact when y is a polynomial
xi 0 I=1
2 1
of degree 2 m − 1 in x. The m weights and m
cI NI (u) du = ∆xi ce1 12 +ce2 12
e
= ∆xi Gauss points result in 2 m parameters, chosen to
I=1 0
∆xi
exactly represent a polynomial of degree 2 m −
= 2
(yi +yi+1 ) 1, which has 2 m parameters. The Gauss points
Since ce1 = yi and ce2 = yi+1 , the result is the and weights are given in Table 2. The weights
same as the trapezoid rule. These formulas can can be defined with W (x) in the integrand as
be added together to give linear elements: well.
b
∆xe e e Table 2. Gaussian quadrature points and weights *
y (x) dx = (y1 +y2 )
e 2 N xi Wi
a 1 0.5000000000 0.6666666667
2 0.2113248654 0.5000000000
If the quadratic expansion is used (Eq. 3), the
0.7886751346 0.5000000000
endpoints of the element are x i and x i+2 , and 3 0.1127016654 0.2777777778
x i+1 is the midpoint, here assumed to be equally 0.5000000000 0.4444444445
spaced between the ends of the element: 0.8872983346 0.2777777778
4 0.0694318442 0.1739274226
xi+2 1
3 0.3300094783 0.3260725774
ceI NI
y (x) dx = (u) (xi+2 −xi ) du 0.6699905218 0.3260725774
xi 0 I=1
0.9305681558 0.1739274226
3 1 5 0.0469100771 0.1184634425
ceI NI (u) du
= ∆xi 0.2307653450 0.2393143353
I=1 0
0.5000000000 0.2844444444
0.7692346551 0.2393143353
= ∆xe ce1 16 +ce2 23 +ce3 16
0.9530899230 0.1184634425
For many elements, with different ∆x e , * For a given N the quadrature points x 2 , x 3 , . . . , x NP−1 are
quadratic elements : given above. x 1 = 0, x NP = 1. For N = 1, W 1 = W 3 = 1/6 and for
N ≥ 2, W 1 = W NP = 0.
b
∆xe e
y (x) = (y1 +4y2e +y3e ) For orthogonal collocation on finite elements
e 6 the quadrature formula is
a
1 NP
If the element sizes are all the same this gives
y (x) dx = ∆xe Wj y (xeJ )
Simpson’s rule. e
0 j=1
For cubic splines the quadrature rule within
one element is Each special polynomial has its own quadra-
ture formula. For example, Gauss – Legendre
xi+1
Ci (x) dx = 1
∆xi (yi +yi+1 )
polynomials give the quadrature formula
2 ∞
xi n
e−x y (x) dx =
′′ ′′
1
− 24 ∆x3i yi +yi+1 Wi y (xi )
0 i=1
For the entire interval the quadrature formula is (points and weights are available in mathe-
xN
T N
T −1 matical tables) [23].
1
y (x) dx = 2
∆xi (yi +yi+1 ) For Gauss – Hermite polynomials the
x1 i=1
N
T −1
quadrature formula is
1 ′′ ′′
− 24 ∆x3i (yi +yi+1 ) ∞
n
i=1 2
e−x y (x) dx = Wi y (xi )
with y′′1 = 0, y′′ NT = 0 for natural cubic splines. −∞ i=1
20 Mathematics in Chemical Engineering
(points and weights are available in mathemati- experimental measurements contain error; the
cal tables) [23]. goal is to find the set of parameters in the model
Romberg’s method uses extrapolation tech- that best represents the experimental data. Ref-
niques to improve the answer [15]. If I 1 is the erence [23] gives a complete treatment relating
value of the integral obtained by using interval the least-squares procedure to maximum likeli-
size h = ∆x, I 2 the value of I obtained by using hood.
interval size h/2, and I 0 the true value of I, then In a least-squares parameter estimation, it is
the error in a method is approximately hm , or desired to find parameters that minimize the sum
of squares of the deviation between the experi-
I1 ≈I0 +chm
mental data and the theoretical equation
m
h N 2
I2 ≈I0 +c
yi −y(xi ;a1 ,a2 ,. . .,aM )
2 χ2 =
i=1
σi
Replacing the ≈ by an equality (an approxima-
tion) and solving for c and I 0 give where yi is the i-th experimental data point for
2m I2 −I1 the value x i , y(xi ;a1 ,a2 ,. . .,aM ) the theoreti-
I0 = cal equation at x i , σi the standard deviation
2m −1
of the i-th measurement, and the parameters
This process can also be used to obtain I 1 , I 2 , . .
{a1 ,a2 ,...,aM } are to be determined to mini-
. , by halving h each time, calculating new esti-
mize χ2 . The simplification is made here that
mates from each pair, and calling them J 1 , J 2 , .
the standard deviations are all the same. Thus,
. . (i.e., in the formula above, I 0 is replaced with
we minimize the variance of the curve fit.
J 1 ). The formulas are reapplied for each pair of J
’s to obtain K 1 , K 2 , . . . . The process continues N
[yi −y (xi ;a1 ,a2 ,. . .,aM )]2
until the required tolerance is obtained. σ2 =
i=1
N
I1 I2 I3 I4
J1 J2 J3
K1 K2
L1 Linear Least Squares. When the model is a
straight line, one is minimizing
Romberg’s method is most useful for a low-
N
order method (small m) because significant im- χ2 = [yi −a−bxi ]2
provement is then possible. i=1
When the integrand has singularities, a va-
riety of techniques can be tried. The integral The linear correlation coefficient r is defined by
may be divided into one part that can be inte- N
(xi −x) (yi −y)
grated analytically near the singularity and an- i=1
r=
other part that is integrated numerically. Some- N
N
2
(yi −y)2
times a change of argument allows analytical (xi −x)
i=1 i=1
integration. Series expansion might be help-
ful, too. When the domain is infinite, Gauss – and
Legendre or Gauss – Hermite quadrature can be N
used. Also a transformation can be made [15]. χ2 = 1−r 2 [yi −y]2
For example, let u = 1/x and then i=1
Polynomial Regression. In polynomial re- (For definition of i, see Chap. 3.) The Nyquist
gression, one expands the function in a poly- critical frequency or critical angular frequency
nomial in x. is
M 1 π
y (x) = aj xj−1 fc = , ωc =
2∆ ∆
j=1
If a function y (t) is bandwidth limited to fre-
The parameters are easily determined using quencies smaller than f c , i.e.,
computer software. In Microsoft Excel, the data
is put into columns A and B and the graph is cre- Y (ω) = 0 for ω>ωc
ated as for a linear curve fit. Then add a trendline
and choose the degree of polynomial desired. then the function is completely determined by
its samples yn . Thus, the entire information con-
Multiple Regression. In multiple regres- tent of a signal can be recorded by sampling at a
sion, any set of functions can be used, not just rate ∆−1 = 2 f c . If the function is not bandwidth
polynomials. limited, then aliasing occurs. Once a sample rate
M
∆ is chosen, information corresponding to fre-
y (x) = aj fj (x) quencies greater than f c is simply aliased into
j=1 that range. The way to detect this in a Fourier
transform is to see if the transform approaches
where the set of functions {fj (x)} is known zero at ± f c ; if not, aliasing has occurred and a
and specified. Note that the unknown param- higher sampling rate is needed.
eters {aj } enter the equation linearly. In this Next, for N samples, where N is even
case, the spreadsheet can be expanded to have
a column for x, and then successive columns yk = y (tk ) , tk = k∆, k = 0, 1, 2,. . .,N −1
for fj (x). In Microsoft Excel, choose Regres-
sion under Tools/Data Analysis, and complete and the sampling rate is ∆; with only N values
the form. In addition to the actual correlation, {yk } the complete Fourier transform Y (ω) can-
one gets the expected variance of the unknowns, not be determined. Calculate the value Y (ω n )
which allows one to assess how accurately they at the discrete points
were determined. ωn = 2πn
,n = −N ,. . ., 0,. . ., N
N∆ 2 2
N
dY 1 2πik 2ikπx/L
= yk e
dx L k=−N L
3. Complex Variables [25 – 31]
Thus, at the grid points
N
3.1. Introduction to the Complex Plane
dY 1 2πik 2ikπxn /L
= yk e
dx n L k=−N L A complex number is an ordered pair of real
numbers, x and y, that is written as
The process works as follows. From the solu-
tion at all grid points the Fourier transform is z = x+iy
obtained by using FFT {yk }. This is multiplied
by 2 π i k/L to obtain the Fourier transform of The variable i is the imaginary unit which has
the derivative: the property
2πik i2 = −1
yk′ = yk
L The real and imaginary parts of a complex num-
The inverse Fourier transform is then taken by ber are often referred to:
using FFT, to give the value of the derivative at
each of the grid points: Re (z) = x, Im (z) = y
Write
Figure 8. Addition in the complex plane
z = x+i y = r (cosθ+i sinθ)
so that
For complex conjugates then
x = rcos θ, y = rsin θ
z ∗ z = |z|2
and
The reciprocal is
y
θ = arctan 1 z∗ 1
1
x = = (cosθ−i sinθ) , arg = −arg z
z |z|2 r z
Since the arctangent repeats itself in multiples
of π rather than 2 π, the argument must be de- Then
fined carefully. For example, the θ given above z1 x1 +i y1 x2 −i y2
= = (x1 +i y1 )
could also be the argument of − (x + i y). The z2 x2 +i y2 x22 +y2
2
Equality :
z1 = z2 if and only if x1 = x2 and y1 = y2
Addition : 3.2. Elementary Functions
z1 +z2 = (x1 +x2 ) +i (y1 +y2 )
Multiplication : Properties of elementary functions of complex
z1 z2 = (x1 x2 −y1 y2 ) +i (x1 y2 +x2 y1 ) variables are discussed here [32]. When the po-
lar form is used, the argument must be specified
The last rule can be remembered by using the because the same physical angle can be achieved
standard rules for multiplication, keeping the with arguments differing by 2 π. A complex
imaginary parts separate, and using i2 = − 1. In number taken to a real power obeys
the complex plane, addition is illustrated in Fig-
ure 8. In polar form, multiplication is u = z n , |z n | = |z|n , arg (z n ) = n arg z (mod2π)
z1 z2 = r1 r2 [cos (θ1 +θ2 ) +i sin (θ1 +θ2 )] u = z n = r n (cos nθ+i sin nθ)
24 Mathematics in Chemical Engineering
Roots of a complex number are complicated by The circular functions with complex argu-
careful accounting of the argument ments are defined
ei z +e−i z ei z −e−i z
z = w1/n with w = R (cos Θ+i sin Θ) , 0≤Θ≤2π cos z = 2
, sin z = 2
,
then sin z
tan z = cos z
and satisfy
Θ
zk = R1/n {cos n
+ (k−1) 2π
n
sin (−z) = −sin z, cos (−z) = cos z
Θ
+i sin n
+ (k−1) 2π
n
}
sin (i z) = i sinh z, cos (i z) = −cosh z
such that
All trigonometric identities for real, circular
(zk )n = w for every k functions with real arguments can be extended
without change to complex functions of complex
z = r (cos θ+i sin θ) arguments. For example,
r n = R, nθ = Θ (mod 2π)
sin2 z+cos2 z = 1,
The exponential function is
sin (z1 +z2 ) = sin z1 cos z2 +cos z1 sin z2
ez = ex (cos y+i sin y)
The same is true of hyperbolic functions. The
Thus, absolute boundaries of sin z and cos z are not
bounded for all z.
z = r (cos θ+i sin θ)
Trigonometric identities can be defined by us-
can be written ing
z = reiθ eiθ = cos θ+i sin θ
For example,
and
ei(α+β) = cos (α+β) +i sin (α+β)
|ez | = ex , arg ez = y (mod 2π)
= eiα eiβ = (cos α+i sin α)
The exponential obeys (cos β+i sin β)
ez =0 for every finite z = cos α cos β−sin α sin β
+i (cos α sin β+cos β sin α)
and is periodic with period 2 π:
Equating real and imaginary parts gives
ez+2π i = ez cos (α+β) = cos α cos β− sin α sin β
Trigonometric functions can be defined by sin (α+β) = cos α sin β+cos β sin α
using
The logarithm is defined as
eiy = cos y+i sin y, and e−iy = cos y−i sin y ln z = ln |z| +i arg z
Thus, and the various determinations differ by multi-
ei y +e−i y
ples of 2 π i. Then,
cos y = 2
= cosh i y
eln z = z
ei y −e−i y
sin y = 2i
= −i sinh i y
ln (ez ) −z≡0 (mod 2 πi)
The second equation follows from the defini-
Also,
tions
ln (z1 z2 ) −ln z1 −ln z2 ≡0 (mod 2 πi)
ez +e−z ez −e−z
cosh z≡ , sinh z≡
2 2 is always true, but
The remaining hyperbolic functions are ln (z1 z2 ) = ln z1 +ln z2
sinh z
tanh z≡ cosh z
1
, coth z≡ tanh z
holds only for some determinations of the loga-
rithms. The principal determination of the argu-
1 1
sech z≡ cosh z
, csch z≡ sinh z ment can be defined as − π < arg ≤ π.
Mathematics in Chemical Engineering 25
3.3. Analytic Functions of a Complex Analyticity implies continuity but the con-
Variable verse is not true: z* = x − i y is continuous but,
because the Cauchy – Riemann conditions are
Let f (z) be a single-valued continuous function not satisfied, it is not analytic. An entire func-
of z in a domain D. The function f (z) is differ- tion is one that is analytic for all finite values of z.
entiable at the point z0 in D if Every polynomial is an entire function. Because
the polynomials are analytic, a ratio of polyno-
f (z0 +h) −f (z0 )
lim mials is analytic except when the denominator
h→0 h
vanishes. The function f (z) = | z2 | is continuous
exists as a finite (complex) number and is in- for all z but satisfies the Cauchy – Riemann con-
dependent of the direction in which h tends to ditions only at z = 0. Hence, f ′ (z) exists only
zero. The limit is called the derivative, f ′ (z0 ). at the origin, and |z |2 is nowhere analytic. The
The derivative now can be calculated with h ap- function f (z) = 1/z is analytic except at z = 0. Its
proaching zero in the complex plane, i.e., any- derivative is − 1/z2 , where z = 0. If ln z = ln |
where in a circular region about z0 . The function z | + i arg z in the cut domain − π < arg z ≤ π,
f (z) is differentiable in D if it is differentiable then f (z) = 1/ln z is analytic in the same cut do-
at all points of D; then f (z) is said to be an an- main, except at z = 1, where log z = 0. Because
alytic function of z in D. Also, f (z) is analytic ez is analytic and ± i z are analytic, e±iz is ana-
at z0 if it is analytic in some ε neighborhood of lytic and linear combinations are analytic. Thus,
z0 . The word analytic is sometimes replaced by the sine and cosine and hyperbolic sine and co-
holomorphic or regular. sine are analytic. The other functions are analytic
The Cauchy – Riemann equations can be except when the denominator vanishes.
used to decide if a function is analytic. Set The derivatives of the elementary functions
f (z) = f (x+i y) = u (x, y) +i v (x, y)
are
d z d n
e = ez , z = n z n−1
Theorem [30, p. 51]. Suppose that f (z) is dz dz
g (w) = g [f (z)] = z
26 Mathematics in Chemical Engineering
Laplace Equation. If f (z) is analytic, where The integral is linear with respect to the inte-
grand:
f (z) = u (x, y) +i v (x, y)
[α1 f1 (z) +α2 f2 (z)] dz
the Cauchy – Riemann equations are satisfied. C
Differentiating the Cauchy – Riemann equa- = α1 f1 (z) dz+α2 f2 (z) dz
C C
tions gives the Laplace equation:
The integral is additive with respect to the path.
∂2u ∂2v ∂2v 2
∂x2
= ∂x∂y
= ∂y∂x
= − ∂∂yu
2 or Let curve C 2 begin where curve C 1 ends and
∂2u 2 C 1 + C 2 be the path of C 1 followed by C 2 . Then,
∂x2
+ ∂∂yu2 =0
f (z) dz = f (z) dz+ f (z) dz
Similarly,
C1 +C2 C1 C2
∂2v ∂2v
+ =0 Reversing the orientation of the path replaces the
∂x2 ∂y 2
integral by its negative:
Thus, general solutions to the Laplace equation
can be obtained from analytic functions [30, p. f (z) dz = − f (z) dz
60]. For example, −C C
f (z0 ) = 1
f (z)
dz where the curve C is a closed, counterclockwise
2πi z−z0
C curve containing z0 and is within the neighbor-
1 f (z)
f ′ (z0 ) = hood where f (z) is analytic. The complex num-
2πi (z−z0 )2
dz
C
ber A−1 is the residue of f (z) at the isolated
Under further restriction on the domain [30, p. singular point z0 ; 2 π i A−1 is the value of the
127], integral in the positive direction around a path
m!
f (z) containing no other singular points.
f (m) (z0 ) = dz If f (z) is defined and analytic in the exterior
2πi (z−z0 )m+1
C | z − z0 | > R of a circle, and if
1 1
Power Series. If f (z) is analytic interior to v (ζ) = f z0 + obtained by ζ =
ζ z−z0
a circle | z − z0 | < r 0 , then at each point inside
the circle the series has a removable singularity at ζ = 0, f (z) is an-
∞
alytic at infinity. It can then be represented by
f (n) (z0 ) a Laurent series with nonpositive powers of z −
f (z) = f (z0 ) + (z−z0 )n
n=1
n! z0 .
If C is a closed curve within which and on
converges to f (z). This result follows from
which f (z) is analytic except for a finite number
Cauchy’s integral. As an example, ez is an en-
of singular points z1 , z2 , . . . , zn interior to the
tire function (analytic everywhere) so that the
region bounded by C, then the residue theorem
MacLaurin series
states
∞
zn
ez = 1+
n! f (z) dz = 2πi(̺1 +̺2 +···̺n )
n=1
C
represents the function for all z.
Another result of Cauchy’s integral formula where ̺n denotes the residue of f (z) at zn .
is that if f (z) is analytic in an annulus R, r 1 < |z The series of negative powers in Equation 5
− z0 | < r 2 , it is represented in R by the Laurent is called the principal part of f (z). If the princi-
series pal part has an infinite number of nonvanishing
∞ terms, the point z0 is an essential singularity.
f (z) = An (z−z0 )n , r1 < |z−z0 | ≤r2 If A−m = 0, A−n = 0 for all m < n, then z0 is
n=−∞ called a pole of order m. It is a simple pole if m
where = 1. In such a case,
1
f (z) ∞
An = dz, A−1
2πi
C
(z−z0 )n+1 f (z) = + An (z−z0 )n
z−z0 n=0
n = 0, ±1, ±2,. . .,
If a function is not analytic at z0 but can be made
and C is a closed curve counterclockwise in R.
so by assigning a suitable value, then z0 is a re-
movable singular point.
Singular Points and Residues [33, p. 159,
When f (z) has a pole of order m at z0 ,
30, p. 180]. If a function in analytic in every
neighborhood of z0 , but not at z0 itself, then z0 is ϕ (z) = (z−z0 )m f (z) , 0< |z−z0 | <r0
called an isolated singular point of the function.
About an isolated singular point, the function has a removable singularity at z0 . If ϕ (z0 ) =
can be represented by a Laurent series. A−m , then ϕ (z) is analytic at z0 . For a simple
A A
pole,
−2 −1
f (z) = ···+ (z−z )2
+ z−z +A0
0 0
(5) A−1 = ϕ (z0 ) = lim (z−z0 ) f (z)
z→z0
+A1 (z−z0 ) +···0< |z−z0 | ≤r0
28 Mathematics in Chemical Engineering
Also | f (z)| → ∞ as z → z0 when z0 is a pole. is the analytic continuation onto the entire z
Let the function p (z) and q (z) be analytic at z0 , plane except for z = 1.
where p (z0 ) = 0. Then An extension of the Cauchy integral formula
p (z)
is useful with Laplace transforms. Let the curve
f (z) = C be a straight line parallel to the imaginary axis
q (z)
and z0 be any point to the right of that (see Fig.
has a simple pole at z0 if, and only if, q (z0 ) = 0 9). A function f (z) is of order zk as |z | → ∞ if
and q′ (z0 ) = 0. The residue of f (z) at the simple positive numbers M and r 0 exist such that
pole is −k
z f (z) <M when |z| >r0, i.e.,
p (z0 )
A−1 = ′
q (z0 ) |f (z)| <M |z|k for |z| sufficiently large
−π<θ<π, r>0
Analytic Continuation [33, p. 165]. If f 1 (z) Theorem [33, p. 167]. Let f (z) be analytic
is analytic in a domain D1 and domain D con- when R (z) ≥ γ and O (z−k ) as | z | → ∞ in that
tains D1 , then an analytic function f (z) may ex- half-plane, where γ and k are real constants and
ist that equals f 1 (z) in D1 . This function is the k > 0. Then for any z0 such that R (z0 ) > γ
analytic continuation of f 1 (z) onto D, and it is
γ+iβ
unique. For example, 1
f (z)
f (z0 ) = − lim dz,
∞ 2 π i β→∞ z−z0
γ−iβ
f1 (z) = z n , |z| <1
n=0 i.e., integration takes place along the line x = γ.
is analytic in the domain D1 : | z | < 1. The se-
ries diverges for other z. Yet the function is the
MacLaurin series in the domain 3.5. Other Results
1
f1 (z) = , |z| <1 Theorem [32, p. 84]. Let P (z) be a polyno-
1−z
mial of degree n having the zeroes z1 , z2 , . . . , zn
Thus, and let π be the least convex polygon containing
1 the zeroes. Then P ′ (z) cannot vanish anywhere
f1 (z) =
1−z in the exterior of π.
Mathematics in Chemical Engineering 29
∞
If a polynomial has real coefficients, the roots a0
f (x) = + (an cos n x+bn sin n x)
are either real or form pairs of complex conju- 2 n=1
gate numbers. where
The radius of convergence R of the Taylor se-
π π
ries of f (z) about z0 is equal to the distance from a0 = 1
f (x) dx, an = 1
f (x) cos n x dx,
π π
z0 to the nearest singularity of f (z). −π −π
1
π
bn = π
f (x) sin n x dx
Conformal Mapping. Let u (x, y) be a har- −π
monic function. Introduce the coordinate trans- The values {an } and {bn } are called the finite
formation cosine and sine transform of f, respectively. Be-
x = x̂ (ξ, η) , y = ŷ (ξ, η)
cause
1
cos n x = einx +e−inx
It is desired that 2
1
and sin n x = einx −e−inx
U (ξ, η) = u [x̂ (ξ, η) , ŷ (ξ, η)] 2i
Thus, cn (t) satisfies has a finite value. Then the improper integral
dcn 2 ∞
= −n2 cn , or cn = cn (0) e−n t
dt f (x) dx
Let cn (0) be the Fourier coefficients of the ini- −∞
T̂ (ω, 0) = fˆ (ω)
Properties of Fourier Transforms [40, p.
324], [15]. The solution is
2
T̂ (ω, t) = fˆ (ω) e−ω t
df
F dx
= −i ω F [f ] = i ω fˆ
The inverse transformation is
F [i x f (x)] = d
dω
F [f ] = d
dω
fˆ
∞
1
1 iωb/a ˆ ω 2
F [f (a x−b)] = |a|
e f a T (x, t) = e−iωx fˆ (ω) e−ω t dω
2π
−∞
F eicx f (x) = fˆ (ω+c)
! "
Because
F [cos ω0 x f (x)] = 1
2
fˆ (ω+ω0 ) +fˆ (ω−ω0 )
2 1 2
e−ω t
=F √ e−x /4t
4πt
F [sin ω0 x f (x)] = 1
2i
fˆ (ω+ω0 ) −fˆ (ω−ω0 )
the convolution integral can be used to write the
F e−iω0 x f (x) = fˆ (ω−ω0 )
! "
solution as
If f (x) is real, then f (− ω) = f̂ * (ω). If f (x) is 1
∞
2
∧ T (x, t) = √ f (y) e−(x−y) /4t
dy
imaginary, then f (− ω) = − f̂ * (ω). If f (x) is 4πt
−∞
∧ ∧
even, then f (ω) is even. If f (x) is odd, then f
32 Mathematics in Chemical Engineering
Finite Fourier Sine and Cosine Transform Table 3. Finite sine transforms [41]
are (−1)n+1 x
n π
2
π
fs (n) = Fsn [f ] = π
f (x) sin n xdx, 1−(−1)n
1
0 n
n = 1, 2,. . ., π
(π−c) x (x≤c)
n2
sin n c (0<c<π)
c (π−x) (x≥c)
2
π
fc (n) = Fcn [f ] = π
f (x) cos n xdx
0 π −x (x<c)
n cos n c (0<c<π) π−x (x)
n = 0, 1, 2,. . .
π 2 (−1)n−1 2[1−(−1)n ]
∞
n − n3
x2
f (x) = fs (n) sin n x,
n=1 2
π(−1)n 6
n3
− πn x3
∞
1
f (x) = 2
fc (0) + fc (n) cos n x
n=1 n
n2 +c2
[1−(−1)n ec π ] ecx
They obey the operational properties n sinh c(π−x)
n2 +c2 sinh c π
d2 f
Fsn dx2
= −n2 Fsn [f ]
n sin k(π−x)
n2 −k2
(|k| =0, 1, 2,. . .) sin k π
n
+ 2πn [f (0) − (−1) f (π)]
π
0 (n=m) ; fs (m) = 2 sin m x (m = 1,2, . . . )
f, f ′ are continuous, f ′′ is piecewise continu-
ous on 0 ≤ x ≤ π.
n
n2 −k2
[1−(−1)n cos k x] cos k x (| k | = 1,2, . . . )
n
1−(−1)n+m ,
n2 −m2 cos m x (m = 1,2, . . . )
(n=m) ; fs (m) = 0
The material is reproduced with permission of McGrawHill, Inc.
Also,
and f 1 is the extension of f, k is a constant
df
Fsn dx
= −nFcn [f ]
df
Fcn dx
= nFsn [f ] − π2 f (0) +(−1)n π2 f (π)
where F 1 and G 1 are odd extensions of F and G, transforms and then using the fast Fourier trans-
respectively, and F 2 and G2 are even extensions form (see Chap. 2).
of F and G, respectively. Finite sine and cosine
transforms are listed in Tables 3 and 4.
4.2. Laplace Transforms
Table 4. Finite cosine transforms [41]
fc (n) = F (x) (0<x<π) Consider a function F (t) defined for t > 0. The
π
F (x) cos n xdx (n = 0,1,. . .) Laplace transform of F (t) is [41]
0
∞
(−1)n f c (n) F (π−x) L [F ] = f (s) = e−st F (t) dt
0
0 when n=1,2, . . . ; f c (0)=π 1
The Laplace transformation is linear, that is,
2 1 (0<x<c)
n sin n c; fc (0) = 2 c−π
−1 (c<x<π)
L [F +G] = L [F ] +L [G]
1−(−1)n π2
− n2
; fc (0) = 2 x Thus, the techniques described herein can be ap-
(−1)n π2 x2
plied only to linear problems. Generally, the as-
n2
; fc (0) = 6 2π
sumptions made below are that F (t) is at least
1
; fc (0) = 0
(π−x)2
−π
piecewise continuous, that it is continuous in
n2 2π 6
each finite interval within 0 < t < ∞, and that
(−1)n ecx −1 1 cx it may take a jump between intervals. It is also
n2 +c2 ce
of exponential order, meaning e−αt | F (t)| is
cosh c(π−x)
1
n2 +c2 c sinh c π bounded for all t > T, for some finite T.
The unit step function is
(−1)n cos k π−1 1
n2 −k2
(|k| =0,1,. . .) k sin k x
0 0≤t<k
(−1)n+m −1
Sk (t) =
; fc (m) = 0 (m = 1,. . .) 1
m sin m x
1 t>k
n2 −m2
1
(|k| =0,1,. . .) − k sin k x
cos k(π−x) and its Laplace transform is
n2 −k2
0 (n=m) ; fc (m) = π 2 (m = 1,2,. . .) cos m x e−ks
L [Sk (t)] =
The material is reproduced with permission of McGrawHill, Inc. s
In particular, if k = 0 then
On the semi-infinite domain, 0 < x < ∞, the
Fourier sine and cosine transforms are 1
L [1] =
∞ s
Fsω [f ] ≡
f (x) sin ω x dx,
0 The Laplace transforms of the first and second
∞
Fcω [f ] ≡
f (x) cos ω x dx and derivatives of F (t) are
0
2 ω′ 2 ω′ dF
f (x) = F
π s
[Fsω [f ]] , f (x) = F
π c
[Fcω [f ]] L dt
= s f (s) −F (0)
The sine transform is an odd function of ω,
d2 F
L dt2
= s2 f (s) −s F (0) − ddtF (0)
whereas the cosine function is an even function
of ω. Also, More generally,
d2 f
Fsω dx2
= f (0) ω−ω 2 Fsω [f ]
dn F
L dtn
= sn f (s) − sn−1 F (0)
d2 f
Fcω dx2
= − ddxf (0) −ω 2 Fcω [f ] dn−1 F
−sn−2 dF
dt
(0) − ··· − dtn−1
(0)
provided f (x) and f ′ (x) → 0 as x → ∞. Thus,
the sine transform is useful when f (0) is known The inverse Laplace transformation is
and the cosine transform is useful when f ′ (0) F (t) = L−1 [f (s)] where f (s) = L [F ]
is known.
Hsu and Dranoff [42] solved a chemical en- The inverse Laplace transformation is not unique
gineering problem by applying finite Fourier because functions that are identical except for
34 Mathematics in Chemical Engineering
Table 5. Laplace transforms (see [23] for a more complete list) More generally, translation gives the following.
L [F] F (t)
1
1
Translation.
s
f (a s−b) = f a s− ab = L a1 ebt/a F t
! "
1 a
,
s2
t
1 tn−1 a>0
sn (n−1)!
1
The step function
√ √1
s πt
1
0 0≤t< h
s−3/2
2 t/π
1 2
S (t) = 1 h
≤t< h
Γ (k)
(k>0) tk−1
sk 2 3
2 ≤t< h
h
1 at
s−a e
has the Laplace transform
1
(s−a)n
(n = 1,2,. . .) 1
(n−1)!
tn−1 eat 1 1
L [S (t)] =
Γ (k) k −1 at s 1−e−hs
(k>0) t e
(s−a)k
The Dirac delta function δ (t − t 0 ) (see Equa-
1
(s−a)(s−b)
1
a−b eat −ebt tion 116 in → Mathematical Modeling) has the
property
s
(s−a)(s−b)
1
a−b aeat −bebt ∞
1 1 δ (t−t0 ) F (t) dt = F (t0 )
s2 +a2 a sin a t
0
s
cos a t
s2 +a2 Its Laplace transform is
1 1
s2 −a2 a sinh a t L [δ (t−t0 )] = e−st0 , t0 ≥0, s>0
s
s2 −a2
cosh a t The square wave function illustrated in Figure
s t 10 has Laplace transform
(s2 +a2 )2 2 a sin a t
1 cs
L [Fc (t)] = tanh
s2 −a2 s 2
t cos a t
(s2 +a2 )2
The triangular wave function illustrated in Fig-
1 1 at
(s−a)2 +b2 be sin b t ure 11 has Laplace transform
s−a
eat cos b t 1 cs
(s−a)2 +b2 L [Tc (t)] = tanh
s2 2
Other Laplace transforms are listed in Table 5.
Mathematics in Chemical Engineering 35
A1
···+ s−a +h (s)
where
(s−a)m p (s)
ϕ (s) ≡ q (s)
1 dm−k ϕ (s)
Am = ϕ (a) , Ak = (m−k)! dsm−k
|a ,
1
a The inverse Laplace transform is then
f (s) = 1−e−as
e−st F (t) dt,
0
F (t) = e2t + [−1−t] et
where F (t) = F (t+a)
36 Mathematics in Chemical Engineering
a s
Quadratic Factors. Let p (s) and q (s) have y (s) = s
+2 y (s) s2 +1
real coefficients, and q (s) have the factor a (s2 +1)
or y (s) = s (s−1)2
(s−a)2 +b2 , b>0
Taking the inverse transformation gives
where a and b are real numbers. Then define ϕ Y (t) = s 1+2tet
! "
(s) and h (s) and real constants A and B such that
Next, let the variable s in the Laplace trans-
f (s) =
p (s)
=
ϕ (s) form be complex. F (t) is still a real-valued func-
q (s) (s−a)2 +b2
tion of the positive real variable t. The properties
A s+B
= (s−a)2 +b2
+h (s) given above are still valid for s complex, but ad-
ditional techniques are available for evaluating
Let ϕ1 and ϕ2 be the real and imaginary parts of the integrals. The real-valued function is O [exp
the complex number ϕ (a + i b). (x 0 t)]:
ϕ (a+i b) ≡ϕ1 +i ϕ2 |F (t)| <M ex0 t , z0 = x0 +i y0
∂t
The solution for T 1 can also be obtained by k ∂x
(0, s) = h t (0, s)
Laplace transforms.
The solution is
t1 = L [T1 ] ∞
Applying this to the differential equation t (x, s) = f (ξ) g (x, ξ, s) dξ
0
∂ 2 t1
st1 −f (x) = α , t1 (0, s) = 0 where [41, p. 227]
∂x2
and solving gives
2 s/α g (x, ξ, s) = exp − |x−ξ| s/α
x √ √ √
1 s−h α/k
′
t1 = √ e− s/α(x −x) f x′ dx′ + √s+h√α/k exp − (x+ξ) s/α
sα
0
One form of the inverse transformation is [41, p.
and the inverse transformation is [40, p. 437], 228]
[44, p. 59] ∞ ∞
2 2
e−β αt cos
T (x, t) = π
[β x−µ (β)] f (ξ)
T1 (x, t) = (7) 0 0
∞
1
2 2
cos [β ξ−µ (β)] dξdβ
√ e−(x−ξ) /4αt −e−(x+ξ) /4αt f (ξ) dξ
4παt µ (β) = arg (β+i h/k)
0
Aij = −Aji
If the two vectors u and v are written in com-
The transpose of A is ponent notation, the scalar product is
u·v = ux vx +uy vy +uz vz
AT
ij = Aji
If u · v = 0 and u and v are not null vectors, then
Any dyadic can be represented as the sum of u and v are perpendicular to each other and θ =
a symmetric portion and an antisymmetric por- π/2.
tion. The single dot product of two dyadics is
Aij = Bij +Cij , Bij ≡ 12 (Aij +Aji ) ,
A·B = ei ej Aik Bkj
i j k
and the unit dyadic is defined as where a is a unit vector in the direction of u×v.
The direction of c is perpendicular to the plane
δ= ei ej δij of u and v such that u, v, and c form a right-
i j
handed system. If u = v, or u is parallel to v,
then θ = 0 and u×v = 0. The following laws are
Operations. The dot or scalar product of valid for cross products.
two vectors is defined as
u×v =−v×u Commutative law fails for
u·v = |u| |v| cos θ, 0≤θ≤π vector product
u×(v×w) = (u×v)×w Associative law fails for
where θ is the angle between u and v. The scalar vector product
product of two vectors is a scalar, not a vector. It u×(v + w) = u×v + u×w Distributive law for vector
product
is the magnitude of u multiplied by the projec- ex ×ex = ey ×ey = ez ×ez = 0
tion of v on u, or vice versa. The scalar product ex ×ey = ez , ey ×ez = ex , ez ×ex =
of u with itself is just the square of the magnitude ey
of u. ex
ey ez
u×v = det ux uy uz
u·u = u2 = u2
vx vy vz
The following laws are valid for scalar products = ex (uy vz −vy uz ) +ey (uz vz −ux vz )
I = trA =
is the velocity. The derivative operation obeys
i Aii
the following laws.
II = trA2 =
i j Aij Aji d
(u+v) = du
+ dv
dt dt dt
III = trA3 =
i j k Aij Ajk Aki d
(u·v) = du
·v+u· dv
dt dt dt
The invariants can also be expressed as d du
dt
(u×v) = dt
×v+u× dv
dt
I1 = I
d dϕ
dt
(ϕu) = dt
u+ϕ du
dt
1
I2 = 2
I2 −II
If the vector u depends on more than one vari-
1 able, such as x, y and z, partial derivatives are
I3 = 6
I3 −3 I·II+2 III = det A
defined in the usual way. For example,
Invariants of two dyadics are available [46]. Be-
cause a second-order dyadic has nine compo- if u (x, y, z) , then
nents, the characteristic equation u(x+∆x, y, z)−u(x, y, z)
∂u
∂x
= lim ∆x
∆t→0
det (λ δ−A) = 0
44 Mathematics in Chemical Engineering
Rules for differentiation of scalar and vector Differential Operators. The vector differ-
products are ential operator (del operator) ∇ is defined in
∂
(u·v) = ∂u
·v+u· ∂v cartesian coordinates by
∂x ∂x ∂x
∂ ∂ ∂
∂
(u×v) = ∂u
×v+u× ∂v ∇ = ex +ey +ez
∂x ∂x ∂x ∂x ∂y ∂z
Differentials of vectors are
The gradient of a scalar function is defined
du = dux ex +duy ey +duz ez
∂ϕ ∂ϕ ∂ϕ
∇ ϕ = ex +ey +ez
d (u·v) = du·v+u·dv ∂x ∂y ∂z
The derivative dr/dt is tangent to the curve in the which is called the circulation of u around path
direction of motion C. This integral depends on the vector and the
dr contour C, in general. If the circulation does not
u = dt depend on the contour C, the vector is said to be
dr
dt
irrotational; if it does, it is rotational. The rela-
Also, tionship with the curl is [48, p. 419]
dr
u=
ds
Mathematics in Chemical Engineering 45
1
∇×u) = lim
n· (∇ u·ds invariant scalar field is invariant; the same is true
∆→0 ∆S
C for the divergence and curl of invariant vectors
Thus, the normal component of the curl equals fields.
the net circulation per unit area enclosed by the The gradient of a vector field is required in
contour C. fluid mechanics because the velocity gradient is
The gradient, divergence, and curl obey a dis- used. It is defined as
tributive law but not a commutative or associa- ∇v =
∂vj
i j ei ej ∂xi and
tive law.
∂vi
∇v)T =
∇ψ (∇ i j ei ej ∂xj
∇ (ϕ+ψ) = ∇ ϕ+∇
∇·v
∇ · (u+v) = ∇ ·u+∇ The divergence of dyadics is defined
∂τji
∇×v
∇ × (u+v) = ∇ ×u+∇ ∇ ·τ = i ei j ∂xj and
∇
∇ ·ϕ=ϕ∇
∂
∇ · (ϕu v) = i ei j ∂xj (ϕuj vi )
∇
∇ ·u=u·∇
where τ is any second-order dyadic.
Useful formulas are [49] Useful relations involving dyadics are
∇·u
∇ · (ϕu) = ∇ ϕ·u+ϕ∇ ∇v) = ϕ (∇
(ϕ δ :∇ ∇·v)
∇×u
∇ × (ϕu) = ∇ ϕ×u+ϕ∇ ∇ · (ϕ δ) = ∇ ϕ
∇×u) −u· (∇
∇ · (u×v) = v· (∇ ∇×v) ∇·τ
∇ · (ϕτ ) = ∇ ϕ·τ +ϕ∇
∇u−v (∇
∇ × (u×v) = v·∇ ∇·u) −u·∇
∇v+u (∇
∇·v) n t:τ = t·τ ·n = τ :n t
∇×u) =
∇ × (∇ ∇·u) −∇2 u
∇ (∇ A surface can be represented in the form
∂2ϕ 2
∂2ϕ f (x, y, z) = c = constant
∇ϕ) = ∇2 ϕ =
∇ · (∇ ∂x2
+ ∂∂yϕ2 + ∂z 2 , where ∇
2
is called the Laplacian operator. ∇ ×(∇∇ϕ) = 0. The normal to the surface is given by
The curl of the gradient of ϕ is zero. n=
∇f
∇ f|
|∇
∇×u) = 0
∇ · (∇
provided the gradient is not zero. Operations can
The divergence of the curl of u is zero. Formulas be performed entirely within the surface. Define
useful in fluid mechanics are
∂
∇,
δII ≡δ− n n, ∇ II ≡δII ·∇ ∂n
∇
≡n·∇
∇v)T = ∇ (∇
∇ ·(∇ ∇·v)
vII ≡δII ·v, vn ≡n·v
∇·τ ) +τ :∇
∇ · (τ ·v) = v· (∇ ∇v
Then a vector and del operator can be decom-
1
∇v =
v·∇ 2
∇×v)
∇ (v·v) −v× (∇ posed into
If a coordinate system is transformed by a rota- ∂
v = vII +nvn , ∇ = ∇ II +n
tion and translation, the coordinates in the new ∂n
system (denoted by primes) are given by The velocity gradient can be decomposed into
x′ l11 l12 l13 ∂vII ∂vn
′ ∇II n) vn +n∇
∇ v = ∇ II vII + (∇ ∇II vn +n +n n
y = l21 l22 l23
∂n ∂n
z′ l31 l32 l33 The surface gradient of the normal is the nega-
tive of the curvature dyadic of the surface.
x a1
y + a2
∇ II n = −B
z a3
The surface divergence is then
Any function that has the same value in all coor-
dinate systems is an invariant. The gradient of an ∇ II ·v = δII : ∇ v = ∇ II ·vII −2 H vn
46 Mathematics in Chemical Engineering
Theorem [43]. If f, g, and h are continuous where n is the normal pointing outward to S. The
functions of x, y, and z, and have continuous normal can be written as
first derivatives in a simply connected domain
n = ex cos (x, n) +ey cos (y, n) +ez cos (z, n)
D, then the line integral
where, for example, cos (x, n) is the cosine of the
(f dx+gdy+hdz) angle between the normal n and the x axis. Then
C the divergence theorem in component form is
is independent of the path if and only if ∂ux
∂uy
∂x
+ ∂y
+ ∂u
∂z
z
dxdydz =
V
∂h ∂g ∂ f ∂h ∂g ∂f
= , = , =
∂y ∂z ∂z ∂x ∂x ∂y [ux cos (x, n) +uy cos (y, n) +uz cos (z, n)] dS
S
Mathematics in Chemical Engineering 47
If the divergence theorem is written for an incre- Stokes Theorem [48, 49]. Stokes theorem
mental volume says that if S is a surface bounded by a closed,
1
nonintersecting curve C, and if u has continuous
∇ ·u = lim un dS derivatives then
∆V →0 ∆V
∆S
u·dr = ∇×u) ·ndS =
(∇ ∇×u) ·dS
(∇
the divergence of a vector can be called the in-
C S S
tegral of that quantity over the area of a closed
volume, divided by the volume. If the vector re- The integral around the curve is followed in the
presents the flow of energy and the divergence is counterclockwise direction. In component nota-
positive at a point P, then either a source of en- tion, this is
ergy is present at P or energy is leaving the region
[ux cos (x, s) +uy cos (y, s) +uz cos (z, s)] ds =
around P so that its temperature is decreasing. C
If the vector represents the flow of mass and the ∂uz ∂u
− ∂zy cos (x, n)
∂y
divergence is positive at a point P, then either a S
source of mass exists at P or the density is de- + ∂u ∂z
x
− ∂u
∂x
z
cos (y, n)
creasing at the point P. For an incompressible
∂uy
fluid the divergence is zero and the rate at which + ∂x
− ∂u
∂y
x
cos (z, n) dS
fluid is introduced into a volume must equal the Applied in two dimensions, this results in
rate at which it is removed. Green’s theorem in the plane:
Various theorems follow from the divergence
∂N ∂M
theorem. (M dx+N dy) = − dxdy
Theorem. If ϕ is a solution to Laplace’s equa- ∂x ∂y
C S
tion
The formula for dyadics is
∇2 ϕ = 0
∇×τ ) dS =
n· (∇ τ T ·dr
in a domain D, and the second partial deriva-
S C
tives of ϕ are continuous in D, then the integral
of the normal derivative of ϕ over any piece-
wise smooth closed orientable surface S in D is Representation. Two theorems give infor-
zero. Suppose u = ϕ ∇ ψ satisfies the conditions mation about how to represent vectors that obey
of the divergence theorem: then Green’s theo- certain properties.
rem results from use of the divergence theorem Theorem [ 48, p. 422]. The necessary and
[49]. sufficient condition that the curl of a vector van-
∂ψ
ish identically is that the vector be the gradient
ϕ∇2 ψ+∇
∇ϕ·∇
∇ψ dV = ϕ dS of some function.
∂n
V S Theorem [ 48, p. 423]. The necessary and
and sufficient condition that the divergence of a vec-
tor vanish identically is that the vector is the curl
∂ψ ∂ϕ
ϕ∇2 ψ−ψ∇2 ϕ dV = ϕ −ψ dS of some other vector.
∂n ∂n
V S
Leibniz Formula. In fluid mechanics and
Also if ϕ satisfies the conditions of the theorem transport phenomena, an important result is the
and is zero on S then ϕ is zero throughout D. If derivative of an integral whose limits of inte-
two functions ϕ and ψ both satisfy the Laplace gration are moving. Suppose the region V (t) is
equation in domain D, and both take the same moving with velocity vs . Then Leibniz’s rule
values on the bounding curve C, then ϕ = ψ; i.e., holds:
the solution to the Laplace equation is unique.
d ∂ϕ
The divergence theorem for dyadics is ϕdV = dV + ϕvs ·ndS
dt ∂t
S
V (t) V (t)
∇ ·τ dV = n·τ dS
V S
48 Mathematics in Chemical Engineering
∂ eθ ∂ ∂
Curvilinear Coordinates. Many of the re- ∇ = er + +ez ,
∂r r ∂θ ∂z
lations given above are proved most easily by ∂ϕ eθ ∂ϕ ∂ϕ
using tensor analysis rather than dyadics. Once ∇ ϕ = er + +ez
∂r r ∂θ ∂z
proven, however, the relations are perfectly gen-
1 ∂2ϕ 2
∂
∇ 2 ϕ = r1 ∂r r ∂ϕ + r 2 ∂θ 2
+ ∂∂zϕ2
eral in any coordinate system. Displayed here are ∂r
+eθ ez r1 +ez er ∂v
∂vz
∂θ
∂z
r
+ez eθ ∂v
∂z
θ
+ez ez ∂v ∂z
z
2 2
∂
∇2 v = er ∂r 1 ∂
r ∂r
(r vr ) + r12 ∂∂θv2r + ∂∂zv2r
∂vθ
− r22 ∂θ
+
∂ 2 vθ 2
+eθ ∂
∂r
1 ∂
r ∂r
(r vθ ) + r12 ∂θ 2
+ ∂∂zv2θ + r22 ∂vr
∂θ
+
∂ 2 vz 2
+ez 1 ∂
r ∂r
r ∂v
∂r
z
+ r12 ∂θ 2
+ ∂∂zv2z
x = rcosθ r = x2 +y y
2
y = rsinθ θ = arctan x
z=z z=z
Differential operators are given by [45] Figure 17. Spherical coordinate system
Mathematics in Chemical Engineering 49
∂vφ
x = r sinθ cosϕ r= x2 +y 2 +z 2 ∇ v = er er ∂v
∂r
r
+er eθ ∂v
∂r
θ
+er eφ ∂r
+
y = r sinθ sinϕ θ = arctan x2 +y 2 /z
y +eθ er 1 ∂vr
r ∂θ
− vrθ +eθ eθ 1 ∂vθ
r ∂θ
+ vrr +
z = r cosθ ϕ = arctan x
∂vφ vφ
1 ∂vr
The unit vectors are related by +eθ eφ r1 ∂θ
+eφ er rsinθ ∂ϕ
− r
+
vφ
er = sin θcos ϕex +sin θsin ϕey +cos θez 1 ∂vθ
+eφ eθ rsinθ ∂ϕ
− r
cotθ +
eθ = cos θcos ϕex +cos θsin ϕey −sin θez
1 ∂vφ
+eφ eφ rsinθ ∂ϕ
+ vrr + vrθ cotθ
eφ = −sin ϕex +cos ϕey
∂ 1 ∂
ex = sin θcos ϕer +cos θcos ϕeθ −sin ϕeφ ∇2 v = er ∂r r 2 ∂r
r 2 vr
ey = sin θsin ϕer +cos θsin ϕeθ +cos ϕeφ
1
+ r2 sin θ
∂
∂θ
sin θ ∂v
∂θ
r
ez = cos θer −sin θeθ
∂ 2 vr ∂vφ
1 2 ∂ 2
Derivatives of the unit vectors are + r2 sin2 θ ∂ϕ2
− r2 sin θ ∂θ
(vθ sin θ) − r2 sin θ ∂ϕ
∂eθ
∂er
∂θ
= eθ , ∂θ
= −er +eθ 1 ∂
r 2 ∂r
r2 ∂v
∂r
θ
+ r12 ∂
∂θ
1 ∂
sin θ ∂θ
(vθ sin θ)
∂ 1 ∂ 1 ∂
∇ = er +eθ +eφ ,
∂r r ∂θ r sin θ ∂ϕ
∂ψ 1 ∂ψ 1 ∂ψ 6. Ordinary Differential Equations
∇ ψ = er +eθ +eφ
∂r r ∂θ r sin θ ∂ϕ as Initial Value Problems
∂
∇ 2 ψ = r12 ∂r r2 ∂ψ
∂r
1
+ r2 sin θ
∂
∂θ
sin θ ∂ψ
∂θ
+ A differential equation for a function that de-
1 ∂ ψ 2 pends on only one variable (often time) is called
+ r2 sin2 θ ∂ϕ2
an ordinary differential equation. The general
∇ ·v = 1 ∂ 1
r2 vr + r sin ∂
(vθ sin θ) + solution to the differential equation includes
r 2 ∂r θ ∂θ
many possibilities; the boundary or initial con-
∂vφ
1
+ r sin θ ∂ϕ
ditions are required to specify which of those
are desired. If all conditions are at one point,
1 ∂ 1 ∂vθ
∇ ×v = er r sin θ ∂θ
vφ sin θ − r sin θ ∂ϕ
+ the problem is an initial value problem and can
be integrated from that point on. If some of the
1 ∂vr
+eθ r sin θ ∂ϕ
− r1 ∂
∂r
r vφ + conditions are available at one point and others
1 ∂ ∂vr
at another point, the ordinary differential equa-
+eφ r ∂r
(r vθ ) − r1 ∂θ tions become two-point boundary value prob-
lems, which are treated in Chapter 7. Initial value
∇·τ = er 1 ∂
r2 τrr problems as ordinary differential equations arise
r 2 ∂r
in control of lumped-parameter models, tran-
∂τφr τθθ +τφφ
1 ∂ 1 sient models of stirred tank reactors, polymer-
+ r sin θ ∂θ
(τθr sin θ) + r sin θ ∂ϕ
− r
+
ization reactions and plug-flow reactors, and
1 ∂ 1 ∂
+eθ r 3 ∂r
r3 τrθ + r sin θ ∂θ
(τθθ sin θ) + generally in models where no spatial gradients
∂τφθ τθr −τrθ −τφφ cotθ
occur in the unknowns.
1
r sin θ ∂ϕ
+ r
+
1 ∂ 1 ∂
+eφ r 3 ∂r
r3 τrφ + r sin θ ∂θ
τθφ sin θ +
∂τφφ τ −τ +τ cotθ
1
r sin θ ∂ϕ
+ φr rφr φθ
50 Mathematics in Chemical Engineering
6.1. Solution by Quadrature that can be used to turn the left-hand side into
an exact differential and can be found by using
When only one equation exists, even if it is non- Fréchet differentials [51]. In this case,
linear, solving it by quadrature may be possible.
Ft
dc F
d
Ft
For exp + c = exp c
V dt V dt V
dy
dt
= f (y) Thus, the differential equation can be written as
y (0) = y0 d Ft Ft F
exp c = exp cin
dt V V V
the problem can be separated
This can be integrated once to give
dy
= dt t
F t′
f (y) Ft F
exp c = c (0) + exp cin t′ dt′
and integrated: V V V
0
y t or
dy ′
= dt = t
f (y ′ ) c (t) = exp − FV t c0
y0 0
t
If the quadrature can be performed analytically F F (t−t′ )
+V exp − V
cin (t′ ) dt′
then the exact solution has been found. 0
For example, consider the kinetics problem If the integral on the right-hand side can be calcu-
with a second-order reaction. lated, the solution can be obtained analytically.
dc If not, the numerical methods described in the
= − kc2 , c (0) = c0 next sections can be used. Laplace transforms
dt
To find the function of the concentration versus can also be attempted. However, an analytic so-
time, the variables can be separated and inte- lution is so useful that quadrature and an inte-
grated. grating factor should be tried before resorting to
numerical solution.
dc
c2
= − kdt,
− 1c = − kt+D
5 3 ′′′
Second-order Adams – Bashforth 12 h y 2 1
251 5 (5)
Fourth-order Adams – Bashforth 720 h y 4 1 0.3
Second-order Runge – Kutta
(midpoint) 2 2 2.0
Runge – Kutta – Gill 4 4 2.8
n+1 n+1
Runge – Kutta – Feldberg y −z 5 6 3.0
∆t n ∆t n
y n+1 = y n +∆t f tn + ,y + f
2 2
k6 = ∆t f tn + ∆t n 8 3544
A popular fourth-order method is the Runge – 2 , y − 27 k1 +2k2 − 2565 k3
Kutta – Gill method with the formulas
+ 1859 11
4104 k4 − 40 k5
k1 = ∆t f (tn , y n )
25 1408 2197
y n+1 = y n + 216 k1 + 2565 k3 + 4104 k4 − 15 k5
k2 = ∆t f tn + ∆t
2
, y n + k21
16
z n+1 = y n + 135 k1 + 126656
825 k3
k3 = ∆t f tn + ∆t
2
, y n +a k1 +b k2
+ 28 561 9 2
56 430 k4 − 50 k5 + 55 k6
k4 = ∆t f (tn +∆t, y n +c k 2 +d k3 )
The value of yn+1 − zn+1 is an estimate of the
y n+1 = y n + 61 (k1 +k4 ) + 13 (b k2 +d k3 ) error in yn+1 and can be used in step-size control
√ √ schemes.
2−1 2− 2
a= 2
,b = 2
, Generally, a high-order method should be
√
2
√
2
used to achieve high accuracy. The Runge –
c= − 2
,d = 1+ 2 Kutta – Gill method is popular because it is high
Another fourth-order Runge – Kutta method is order and does not require a starting method
given by the Runge – Kutta – Feldberg formu- (as does the fourth-order Adams – Bashforth
las [52]; although the method is fourth-order, it method). However, it requires four function eval-
achieves fifth-order accuracy. The popular inte- uations per time step, or four times as many as
gration package RKF 45 is based on this method. the Adams – Bashforth method. For problems
in which the function evaluations are a signif-
k1 = ∆t f (tn , y n ) icant portion of the calculation time this might
be important. Given the speed of computers and
k2 = ∆t f tn + ∆t
4
, y n + k41 the widespread availability of desktop comput-
ers, the efficiency of a method is most important
3
k3 = ∆t f tn + 83 ∆t, y n + 32 9
k1 + 32 k2
only for very large problems that are going to
12
k4 = ∆tf tn + 13 1932
∆t,y n + 2197 k1 − 7200 k + 7296 k be solved many times. For other problems the
2197 2 2197 3
most important criterion for choosing a method
439
k5 = ∆tf tn +∆t, y n + 216 k1 −8 k2 + 3680 k − 845 k
513 3 4104 4 is probably the time the user spends setting up
the problem.
Mathematics in Chemical Engineering 53
6
+ 11 ∆t f y n+1
48
4: y n+1 = 25
y n − 36
25
16 n−2
y n−1 + 25 y 3
− 25 y n−3
12
+ 25 ∆t f y n+1
300
5: y n+1 = 137
y n − 300
137
200 n−2
y n−1 + 137 y Figure 21. Flow through packed bed
75 12
− 137 y n−3 + 137 y n−4
60
The superficial velocity u is given by
+ 137 ∆t f y n+1
Q
The stability properties of these methods are u=
Aϕ
determined in the same way as explicit methods.
They are always expected to be stable, no matter where Q is the volumetric flow rate, A is the
what the value of ∆t is, and this is confirmed in cross-sectional area, and ϕ is the void fraction.
Figure 20. A time constant for flow through the device is
then
L ϕAL
tflow = =
u Q
coefficient inside the catalyst. The time constant The idea of stiffness is best explained by con-
for heat transfer is sidering a system of linear equations:
R2 ̺s Cs R2 dy
tinternal heat transfer = = = Ay
α ke dt
where ̺s is the catalyst density, C s is the cata- Let λi be the eigenvalues of the matrix A. This
lyst heat capacity per unit mass, k e is the effec- system can be converted into a system of n equa-
tive thermal conductivity of the catalyst, and α tions, each of them having only one unknown;
is the thermal diffusivity. The time constants for the eigenvalues of the new system are the same
diffusion of mass and heat through a boundary as the eigenvalues of the original system [3, pp.
layer surrounding the catalyst are 39 – 42], [54, pp. 197 – 199]. Then the stiffness
ratio SR is defined as [53, p. 32]
R
texternal diffusion = kg maxi |Re (λi )|
SR =
texternal heat transfer = ̺s Cs R mini |Re (λi )|
hp
SR = 20 is not stiff, SR = 103 is stiff, and SR =
where k g and hp are the mass-transfer and heat-
106 is very stiff. If the problem is nonlinear, the
transfer coefficients, respectively. The impor-
solution is expanded about the current state:
tance of examining these time constants comes
n
from realization that their orders of magnitude dyi ∂fi
= fi [y (tn )] + [yj −yj (tn )]
differ greatly. For example, in the model of an dt j=1
∂yj
automobile catalytic converter [58] the time con-
stant for internal diffusion was 0.3 s, for internal The question of stiffness then depends on the
heat transfer 21 s, and for flow through the de- eigenvalue of the Jacobian at the current time.
vice 0.003 s. Flow through the device is so fast Consequently, for nonlinear problems the prob-
that it might as well be instantaneous. Thus, the lem can be stiff during one time period and not
time derivatives could be dropped from the mass stiff during another. Packages have been devel-
balance equations for the flow, leading to a set oped for problems such as these. Although the
of differential-algebraic equations (see below). chemical engineer may not actually calculate the
If the original equations had to be solved, the eigenvalues, knowing that they determine the
eigenvalues would be roughly proportional to stability and accuracy of the numerical scheme,
the inverse of the time constants. The time in- as well as the step size employed, is useful.
terval over which to integrate would be a small
number (e.g., five) multiplied by the longest time
constant. Yet the explicit stability limitation ap- 6.5. Differential – Algebraic Systems
plies to all the eigenvalues, and the largest eigen-
value would determine the largest permissible Sometimes models involve ordinary differential
time step. Here, λ = 1/0.003 s−1 . Very small time equations subject to some algebraic constraints.
steps would have to be used, e.g., ∆t ≤ 2 × 0.003 For example, the equations governing one equi-
s, but a long integration would be required to librium stage (as in a distillation column) are
n
reach steady state. Such problems are termed M dx = V n+1 y n+1 − Ln xn − V n y n
dt
stiff, and implicit methods are very useful for
them. In that case the stable time constant is not +Ln−1 xn−1
of any interest, because any time step is stable.
What is of interest is the largest step for which a xn−1 − xn = E n xn−1 − x∗,n
N
solution can be found. If a time step larger than
xi = 1
i=1
the smallest time constant is used, then any phe-
nomena represented by that smallest time con- where x and y are the mole fractions in the liquid
stant will be overlooked— at least transients in it and vapor, respectively; L and V are liquid and
will be smeared over. However, the method will vapor flow rates, respectively; M is the holdup;
still be stable. Thus, if the very rapid transients and the superscript n is the stage number. The
of part of the model are not of interest, they can efficiency is E, and the concentration in equilib-
be ignored and an implicit method used [59]. rium with the vapor is x*. The first equation is
Mathematics in Chemical Engineering 57
an ordinary differential equation for the mass of When A is independent of y, the inverse (from
one component on the stage, whereas the third LU decompositions) need be computed only
equation represents a constraint that the mass once.
fractions add to one. As a second example, the In actuality, higher order backward-differ-
following kinetics problem can be considered: ence Gear methods are used in the computer
dc1 program DASSL [60, 61].
= f (c1 , c2 )
dt Differential-algebraic systems are more com-
dc2
= k1 c1 − k2 c22 plicated than differential systems because the so-
dt
lution may not always be defined. Pontelides et
The first equation could be the equation for a al. [62] introduced the term “index” to identify
stirred tank reactor, for example. Suppose both possible problems. The index is defined as the
k 1 and k 2 are large. The problem is then stiff, minimum number of times the equations must
but the second equation could be taken at equi- be differentiated with respect to time to convert
librium. If the system to a set ofordinary differential equa-
tions. These higher derivatives may not exist,
c1 ⇄2c2
and the process places limits on which variables
The equilibrium condition is then can be given initial values. Sometimes the ini-
tial values must be constrained by the algebraic
c22 k1
= ≡K equations [62]. For a differential-algebraic sys-
c1 k2
tem modeling a distillation tower, the index de-
Under these conditions the problem becomes pends on the specification of pressure for the
dc1
= f (c1 , c2 )
column [62]. Several chemical engineering ex-
dt
amples of differential-algebraic systems and a
0 = kl c1 − k2 c22 solution for one involving two-phase flow are
given in [63].
Thus, a differential-algebraic system of equa-
tions is obtained. In this case, the second equa-
tion can be solved and substituted into the first to 6.6. Computer Software
obtain differential equations, but in the general
case that is not possible. Efficient software packages are widely avail-
Differential-algebraic equations can be writ- able for solving ordinary differential equations
ten in the general notation as initial value problems. In each of the pack-
dy
ages the user specifies the differential equation
F t, y, =0 to be solved and a desired error criterion. The
dt
package then integrates in time and adjusts the
or the variables and equations may be separated step size to achieve the error criterion, within the
according to whether they come primarily from limitations imposed by stability.
differential [y(t)] or algebraic equations [x(t)]: A popular explicit Runge – Kutta package is
dy RKF 45. An estimate of the truncation error at
= f (t, y, x) , g (t, y, x) = 0
dt each step is available. Then the step size can
Another form is not strictly a differential- be reduced until this estimate is below the user-
algebraic set of equations, but the same prin- specified tolerance. The method is thus auto-
ciples apply; this form arises frequently when matic, and the user is assured of the results. Note,
the Galerkin finite element is applied: however, that the tolerance is set on the local
truncation error, namely, from one step to an-
dy
A = f (y) other, whereas the user is generally interested
dt
in the global trunction error, i.e., the error af-
The computer program DASSL [60, 61] can ter several steps. The global error is generally
solve such problems. They can also be solved made smaller by making the tolerance smaller,
by writing the differential equation as but the absolute accuracy is not the same as the
dy tolerance. If the problem is stiff, then very small
=A−1 f (y) step sizes are used and the computation becomes
dt
58 Mathematics in Chemical Engineering
very lengthy. The RKF 45 code discovers this the Newton – Raphson method can be fixed over
and returns control to the user with a message several time steps. Then if the iteration does not
indicating the problem is too hard to solve with converge, the Jacobian can be reevaluated at the
RKF 45. current time step. If the iteration still does not
A popular implicit package is LSODE, a ver- converge, then the step size is reduced and a new
sion of Gear’s method [57] written by Alan Jacobian is evaluated. The successive substitu-
Hindmarsh at Lawrence Livermore Laboratory. tion method can also be used — wh ich is even
In this package, the user specifies the differential faster, except that it may not converge. How-
equation to be solved and the tolerance desired. ever, it too will converge if the time step is small
Now the method is implicit and, therefore, sta- enough.
ble for any step size. The accuracy may not be The Runge – Kutta methods give extremely
acceptable, however, and sets of nonlinear equa- good accuracy, especially when the step size
tions must be solved. Thus, in practice, the step is kept small for stability reasons. If the prob-
size is limited but not nearly so much as in the lem is stiff, though, backward difference implicit
Runge – Kutta methods. In these packages both methods must be used. Many chemical reactor
the step size and the order of the method are ad- problems are stiff, necessitating the use of im-
justed by the package itself. Suppose a k-th or- plicit methods. In the MATLAB suite of ODE
der method is being used. The truncation error solvers, ode45 uses a revision of the RKF45
is determined by the (k + 1)-th order derivative. program, while the ode15s program uses an im-
This is estimated by using difference formulas proved backward difference method. Ref. [64]
and the values of the right-hand sides at previ- gives details of the programs in MATLAB. For-
ous times. An estimate is also made for the k-th tunately, many packages are available. On the
and (k + 2)-th derivative. Then, the errors in a NIST web page, http://gams.nist.gov/ choose
(k − 1)-th order method, a k-th order method, “problem decision tree”, and then “differen-
and a (k + 1)-th order method can be estimated. tial and integral equations” to find packages
Furthermore, the step size required to satisfy the which can be downloaded. On the Netlib web
tolerance with each of these methods can be de- site, http://www.netlib.org/, choose “ode” to find
termined. Then the method and step size for the packages which can be downloaded. Using Mi-
next step that achieves the biggest step can be crosoft Excel to solve ordinary differential equa-
chosen, with appropriate adjustments due to the tions is cumbersome, except for the simplest
different work required for each order. The pack- problems.
age generally starts with a very small step size
and a first-order method— the backward Euler
method. Then it integrates along, adjusting the 6.7. Stability, Bifurcations, Limit Cycles
order up (and later down) depending on the er-
In this section, bifurcation theory is discussed in
ror estimates. The user is thus assured that the
a general way. Some aspects of this subject in-
local truncation error meets the tolerance. A fur-
volve the solution of nonlinear equations; other
ther difficulty arises because the set of nonlinear
aspects involve the integration of ordinary differ-
equations must be solved. Usually a good guess
ential equations; applications include chaos and
of the solution is available, because the solution
fractals as well as unusual operation of some
is evolving in time and past history can be ex-
chemical engineering equipment. An excellent
trapolated. Thus, the Newton – Raphson method
introduction to the subject and details needed to
will usually converge. The package protects it-
apply the methods are given in [65]. For more
self, though, by only doing a few (i.e., three)
details of the algorithms described below and
iterations. If convergence is not reached within
a concise survey with some chemical engineer-
these iterations, the step size is reduced and the
ing examples, see [66] and [67]. Bifurcation re-
calculation is redone for that time step. The
sults are closely connected with stability of the
convergence theorem for the Newton – Raph-
steady states, which is essentially a transient
son method (Chap. 1) indicates that the method
phenomenon.
will converge if the step size is small enough.
Consider the problem
Thus, the method is guaranteed to work. Further ∂u
economies are possible. The Jacobian needed in = F (u, λ)
∂t
Mathematics in Chemical Engineering 59
∂uss ′
The variable u can be a vector, which makes F a ∂t
+ ∂u
∂t
= F (uss +u′ , λ)
vector, too. Here, F represents a set of equations
≈ F (uss , λ) + ∂F | u′ +···
∂u uss
that can be solved for the steady state:
The result is
F (u, λ) = 0
∂u′
If the Newton – Raphson method is applied, = Fuss u′
∂t
Fus δ us = −F (us , λ) A solution of the form
us+1 = us +δus u′ (x, t) = eσt X (x)
is obtained, where gives
∂F
Fus = (us ) σeσt X = Fuss eσt X
∂u
is the Jacobian. Look at some property of the so- The exponential term can be factored out and
lution, perhaps the value at a certain point or the
maximum value or an integral of the solution. (Fuss − σ δ) X = 0
This property is plotted versus the parameter λ;
A solution exists for X if and only if
typical plots are shown in Figure 22. At the point
shown in Figure 22 A, the determinant of the Ja- det |Fuss − σ δ| = 0
cobian is zero:
The σ are the eigenvalues of the Jacobian.
det Fu = 0 Now clearly if Re (σ) > 0 then u′ grows with
For the limit point, time, and the steady solution uss is said to be
unstable to small disturbances. If Im (σ) = 0 it
∂F
=0 is called stationary instability, and the distur-
∂λ
bance would grow monotonically, as indicated
whereas for the bifurcation-limit point in Figure 23 A. If Im (σ) = 0 then the distur-
∂F
=0
bance grows in an oscillatory fashion, as shown
∂λ in Figure 23 B, and is called oscillatory insta-
The stability of the steady solutions is also of bility. The case in which Re (σ) = 0 is the di-
interest. Suppose a steady solution uss ; the func- viding point between stability and instability. If
tion u is written as the sum of the known steady Re (σ) = 0 and Im (σ) = 0 — the point governing
state and a perturbation u′ : the onset of stationary instability — then σ = 0.
However, this means that σ = 0 is an eigenvalue
u = uss +u′ of the Jacobian, and the determinant of the Jaco-
This expression is substituted into the original bian is zero. Thus, the points at which the deter-
equation and linearized about the steady-state minant of the Jacobian is zero (for limit points
value:
60 Mathematics in Chemical Engineering
Exchanging the order of differentiation in the forces are calculated to move from one time to
first term leads to the ordinary differential equa- another time. Rewrite this equation in the form
tion of an acceleration.
d2 ri
d ∂y ∂ f ∂y ∂ f 1
= + = Fi ({r}) ≡ai
dt ∂α ∂y ∂α ∂α dt2 mi
The initial conditions on ∂y/∂α are obtained by In the Verlot method, this equation is written us-
differentiating the initial conditions ing central finite differences (Eq. 12). Note that
∂ ∂y
the accelerations do not depend upon the veloc-
[y (0, α) = y0 ] , or (0) = 0 ities.
∂α ∂α
Next, let ri (t+∆t) =2ri (t) −ri (t−∆t) +ai (t) ∆t2
calculation. Thus, methods have been developed where v is the velocity and η the viscosity. Then
especially for boundary value problems. Exam- the variables can be separated again and the re-
ples include heat and mass transfer in a slab, sult integrated to give
reaction – diffusion problems in a porous cata- ∆p r 2
lyst, reactor with axial dispersion, packed beds, −ηv = − +c1 lnr+c2
L 4
and countercurrent heat transfer.
Now the two unknowns must be specified from
the boundary conditions. This problem is a two-
point boundary value problem because one of
7.1. Solution by Quadrature the conditions is usually specified at r = 0 and
the other at r = R, the tube radius. However, the
When only one equation exists, even if it is non-
technique of separating variables and integrating
linear, it may possibly be solved by quadrature.
works quite well.
For
dy
dt
= f (y)
y (0) = y0
y t
dy ′
= dt = t When the fluid is non-Newtonian, it may not
f (y ′ )
y0 0 be possible to do the second step analytically.
For example, for the Bird – Carreau fluid [74, p.
If the quadrature can be performed analytically,
171], stress and velocity are related by
the exact solution has been found.
η0
As an example, consider the flow of a non- τ =
2 (1−n)/2
Newtonian fluid in a pipe, as illustrated in Figure 1+λ dv
dr
24. The governing differential equation is [73]
1 d ∆p where η 0 is the viscosity at v = 0 and λ the time
(rτ ) = − constant.
r dr L
Putting this value into the equation for stress
where r is the radial position from the center of as a function of r gives
the pipe, τ is the shear stress, ∆ p is the pres-
sure drop along the pipe, and L is the length over η0 ∆p r c1
=− +
2 (1−n)/2 L 2 r
which the pressure drop occurs. The variables dv
1+λ dr
are separated once
∆p This equation cannot be solved analytically for
d (rτ ) = − rdr dv/dr, except for special values of n. For prob-
L
lems such as this, numerical methods must be
and then integrated to give
used.
∆p r 2
rτ = − +c1
L 2
Proceeding further requires choosing a consti- 7.2. Initial Value Methods
tutive relation relating the shear stress and the
velocity gradient as well as a condition specify- An initial value method is one that utilizes the
ing the constant. For a Newtonian fluid techniques for initial value problems but al-
lows for an iterative calculation to satisfy all
dv the boundary conditions. Suppose the nonlinear
τ = −η
dr boundary value problem
Mathematics in Chemical Engineering 63
d2 y
dy Packages to solve boundary value problems
=f x,y,
dx2 dx are available on the internet. On the NIST web
with the boundary conditions page, http://gams.nist.gov/ choose “problem de-
dy cision tree”, and then “differential and integral
a0 y (0) −a1 dx (0) = α, ai ≥0
equations”, then “ordinary differential equa-
dy
b0 y (1) −b1 dx (1) = β, bi ≥0 tions”, “multipoint boundary value problems”.
On the Netlib web site, http://www.netlib.org/,
Convert this second-order equation into two search on “boundary value problem”. Any
first-order equations along with the boundary spreadsheet that has an iteration capability can
conditions written to include a parameter s. be used with the finite difference method. Some
du packages for partial differential equations also
=v
dx have a capability for solving one-dimensional
dv
= f (x,u,v) boundary value problems [e.g., Comsol Multi-
dx
physics (formerly FEMLAB)].
u (0) = a1 s−c1 α
last equation is chosen to ensure the best accu- Four times Equation 8 minus this equation, with
racy. rearrangement, gives
dc −3ci +4ci+1 −ci+2
= +O ∆x2
dx i 2∆x
Thus, for the first derivative at point i = 1
dc −3ci +4c2 −c3
= (15)
Figure 25. Finite difference mesh; ∆x uniform dx i 2∆x
This one-sided difference expression uses only
the points already introduced into the domain.
The finite difference representation of the
The third alternative is to add a false point,
second derivative can be obtained by adding the
outside the domain, as c0 = c (x = − ∆x). Then
two expressions in Equation 8. Rearrangement
the centered first derivative, Equation 11, can be
and division by ∆x 2 give
used:
d2 c ci+1 −2ci +ci−1 d4 c ∆x2
= − +. . . (12) dc c2 −c0
dx2 i ∆x2 dx4 i 4! =
dx 1 2∆x
The truncation error is proportional to ∆x 2 . Because this equation introduces a new variable,
To see how to solve a differential equation, another equation is required. This is obtained by
consider the equation for convection, diffusion, also writing the differential equation (Eq. 13),
and reaction in a tubular reactor: for i = 1.
1 d2 c dc The same approach can be taken at the other
− = Da R (c) end. As a boundary condition, any of three
P e dx2 dx
choices can be used:
To evaluate the differential equation at the i-th
dc cn −cn−1
grid point, the finite difference representations dx n
= ∆x
of the first and second derivatives can be used to
cn−2 −4cn−1 +3cn
dc
give dx n
= 2∆x
d
dc
The truncation error of these approaches is also
dx
D (c)
dx
=0 ∆x 2 [78, Chap. 14], [3, p. 215]. The third ap-
proach employs an “upstream” transport coeffi-
This could be written as cient.
dJ
− =0 (16) D Ci+1/2 ≈D (ci+1 ) , when D (ci+1 ) >D (ci )
dx
in terms of the mass flux J, where the mass flux D ci+1/2 ≈ (ci ) , when D (ci+1 ) <D (ci )
is given by
dc This approach is used when the transport coef-
J = −D (c) ficients vary over several orders of magnitude
dx
and the “upstream” direction is defined as the
Because the coefficient depends on c the equa-
one in which the transport coefficient is larger.
tions are more complicated. A finite difference
The truncation error of this approach is only ∆x
method can be written in terms of the fluxes at
[78, Chap. 14] , [3, p. 253], but this approach is
the midpoints, i + 1/2. Thus,
useful if the numerical solutions show unrealis-
Ji+1/2 −Ji−1/2 tic oscillations [3, 78].
− =0
∆x Rigorous error bounds for linear ordinary dif-
Then the constitutive equation for the mass flux ferential equations solved with the finite dif-
can be written as ference method are dicussed by Isaacson and
ci+1 −ci Keller [79, p. 431].
Ji+1/2 = −D ci+1/2
∆x
If these are combined,
D ci+1/2 (ci+1 −ci ) −D ci−1/2 (ci −ci−1 )
7.4. Orthogonal Collocation
=0
∆x2
The orthogonal collocation method has found
This represents a set of nonlinear algebraic equa- widespread application in chemical engineer-
tions that can be solved with the Newton – Raph- ing, particularly for chemical reaction engineer-
son method. However, in this case a viable iter- ing. In the collocation method [3], the dependent
ative strategy is to evaluate the transport coeffi- variable is expanded in a series.
cients at the last value and then solve
N +2
D cki+1/2 ck+1 k+1
i+1 −ci −D cki−1/2 ck+1
i −ck+1
i−1
y (x) = ai yi (x) (18)
i=1
∆x2
=0
66 Mathematics in Chemical Engineering
N
+2
Suppose the differential equation is dy
(xj ) = Ajk y (xk ) ,
dx
k=1
N [y] = 0 N
+2
Ajk = [yi (xk )]−1 dy
dx
i
(xj )
Then the expansion is put into the differential i=1
equation to form the residual: Similar steps can be applied to the second deriva-
&N +2 ' tive to obtain
Residual = N ai yi (x) N
+2
d2 y
i=1 dx2
(xj ) = Bjk y (xk ) ,
k=1
In the collocation method, the residual is set to N
+2 2
Bjk = [yi (xk )]−1 ddxy2i (xj )
zero at a set of points called collocation points: i=1
&N +2 ' This method is next applied to the differential
N ai yi (xj ) = 0, j = 2,. . .,N +1 equation for reaction in a tubular reactor, after
i=1 the equation has been made nondimensional so
This provides N equations; two more equa- that the dimensionless length is 1.0.
tions come from the boundary conditions, giving 1 d2 c dc
− dx = Da R (c) ,
P e dx2
N + 2 equations for N + 2 unknowns. This pro- (19)
cedure is especially useful when the expansion is dc
− dx (0) = P e [c (0) −cin ] , dc
dx
(1) = 0
in a series of orthogonal polynomials, and when
the collocation points are the roots to an orthog- The differential equation at the collocation
onal polynomial, as first used by Lanczos [80, points is
81]. A major improvement was the proposal by N +2 N +2
1
Villadsen and Stewart [82] that the entire so- Bjk c (xk ) − Ajk c (xk ) = Da R (cj ) (20)
P e k=1 k=1
lution process be done in terms of the solution at
the collocation points rather than the coefficients and the two boundary conditions are
in the expansion. Thus, Equation 18 would be N
+2
evaluated at the collocation points − Alk c (xk ) = P e (c1 −cin ) ,
k=1
(21)
N +2 N
+2
AN +2,k c (xk ) = 0
y (xj ) = ai yi (xj ) , j = 1,. . .,N +2 k=1
i=1
Note that 1 is the first collocation point (x = 0)
and solved for the coefficients in terms of the and N + 2 is the last one (x = 1). To apply the
solution at the collocation points: method, the matrices Aij and Bij must be found
N +2 and the set of algebraic equations solved, per-
ai = [yi (xj )]−1 y (xj ) , i = 1,. . .,N +2 haps with the Newton – Raphson method. If or-
j=1 thogonal polynomials are used and the colloca-
Furthermore, if Equation 18 is differentiated tion points are the roots to one of the orthogonal
once and evaluated at all collocation points, the polynomials, the orthogonal collocation method
first derivative can be written in terms of the val- results.
ues at the collocation points: In the orthogonal collocation method the so-
lution is expanded in a series involving orthog-
N +2
dy dyi onal polynomials, where the polynomials Pi−1
(xj ) = ai (xj ) , j = 1,. . .,N +2
dx dx (x) are defined in Section 2.2.
i=1
N
This can be expressed as
y = a+bx+x (1−x) ai Pi−1 (x)
i=1
dy (22)
N
+2
(xj ) = = bi Pi−1 (x)
dx
i=1
N +2
−1 dyi
[yi (xk )] y (xk ) (xj ) , j = 1,. . .,N +2 which is also
i,k=1
dx
N +2
or shortened to y= di xi−1
i=1
Mathematics in Chemical Engineering 67
dy
d = Q−1 y, dx
= CQ−1 y = Ay,
(23)
d2 y
dx2
= DQ−1 y = By
Thus the derivative at any collocation point can
be determined in terms of the solution at the col-
Figure 26. Orthogonal collocation points location points. The same property is enjoyed by
the finite difference method (and the finite ele-
ment method described below), and this prop-
The collocation points are shown in Figure 26. erty accounts for some of the popularity of the
There are N interior points plus one at each end, orthogonal collocation method. In applying the
and the domain is always transformed to lie on 0 method to Equation 19, the same result is ob-
to 1. To define the matrices Aij and Bij this ex- tained; Equations 20 and 21, with the matrices
pression is evaluated at the collocation points; it defined in Equation 23. To find the solution at
is also differentiated and the result is evaluated a point that is not a collocation point, Equation
at the collocation points. 22 is used; once the solution is known at all col-
N
+2 location points, d can be found; and once d is
y (xj ) = di xi−1
i=1
j known, the solution for any x can be found.
N
+2 To use the orthogonal collocation method, the
dy
dx
(xj ) = di (i−1) xi−2
j matrices are required. They can be calculated as
i=1
N
+2
shown above for small N (N < 8) and by us-
d2 y
dx2
(xj ) = di (i−1) (i−2) xi−3
j ing more rigorous techniques, for higher N (see
i=1
Chap. 2). However, having the matrices listed
These formulas are put in matrix notation, where explicitly for N = 1 and 2 is useful; this is shown
Q, C, and D are N + 2 by N + 2 matrices. in Table 7.
2 For some reaction diffusion problems, the so-
y = Qd, dy
dx
= Cd, ddxy2 = Dd lution can be an even function of x. For example,
for the problem
Qji = xi−1
j , Cji = (i−1) xi−2
j ,
d2 c dc
= kc, (0) = 0, c (1) = 1 (24)
Dji = (i−1) (i−2) xi−3
j
dx2 dx
the solution can be proved to involve only even
In solving the first equation for d, the first and powers of x. In such cases, an orthogonal col-
second derivatives can be written as location method, which takes this feature into
Table 7. Matrices for orthogonal collocation
68 Mathematics in Chemical Engineering
account, is convenient. This can easily be done orthogonal collocation is applied at the interior
by using expansions that only involve even pow- points
ers of x. Thus, the expansion N +1
where the power on x a−1 defines the geometry Note that the effectiveness factor is the average
as planar or Cartesian (a = 1), cylindrical (a = reaction rate divided by the reaction rate eval-
2), and spherical (a = 3). An analogous devel- uated at the external conditions. Error bounds
opment is used to obtain the (N + 1)×(N + 1) have been given for linear problems [83, p. 356].
matrices For planar geometry the error is
N
+1
ϕ2(2N +1)
y (xj ) = di x2i−2
j Error in η =
i=1 (2N +1) ! (2N +2) !
N
+1
dy
dx
(xj ) = di (2i−2) x2i−3
j
This method is very accurate for small N (and
i=1
small ϕ2 ); note that for finite difference methods
2
N+1
2 2i−2
the error goes as 1/N 2 , which does not decrease
∇ y (xi ) = di ∇ x
as rapidly with N. If the solution is desired at the
i=1
xj
y = Qd, dy = Cd,∇2 y = Dd center (a frequent situation because the center
dx
concentration can be the most extreme one), it
Qji = x2i−2
j , Cji = (2i−2) x2i−3
j , is given by
N +1
Dji = ∇2 x2i−2 |xj
Q−1
! "
c (0) = d1 1i
yi
i=1
d = Q−1 y, dy
dx
= CQ−1 y = Ay,
The collocation points are listed in Table 8.
∇2 y = DQ−1 y = By For small N the results are usually more accu-
rate when the weighting function in Equation
In addition, the quadrature formula is
25 is 1 − x 2 . The matrices for N = 1 and N =
W Q = f , W = f Q−1 2 are given in Table 9 for the three geometries.
Computer programs to generate matrices and a
where program to solve reaction diffusion problems,
1 N
+1 OCRXN, are available [3, p. 325, p. 331].
x2i−2 xa−1 dx = Wj x2i−2
j Orthogonal collocation can be applied to dis-
0 j=1
1
tillation problems. Stewart et al. [84, 85] devel-
= ≡fi
2i−2+a oped a method using Hahn polynomials that re-
As an example, for the problem tains the discrete nature of a plate-to-plate distil-
lation column. Other work treats problems with
1 d dc
xa−1 dx
xa−1 dx = ϕ2 R (c) multiple liquid phases [86]. Some of the appli-
dc
cations to chemical engineering can be found in
dx
(0) = 0, c (1) = 1 [87 – 90].
Mathematics in Chemical Engineering 69
Table 8. Collocation points for orthogonal collocation with symmetric polynomials and W =1
Geometry
N Planar Cylindrical Spherical
1 0.5773502692 0.7071067812 0.7745966692
2 0.3399810436 0.4597008434 0.5384693101
0.8611363116 0.8880738340 0.9061793459
3 0.2386191861 0.3357106870 0.4058451514
0.6612093865 0.7071067812 0.7415311856
0.9324695142 0.9419651451 0.9491079123
4 0.1834346425 0.2634992300 0.3242534234
0.5255324099 0.5744645143 0.6133714327
0.7966664774 0.8185294874 0.8360311073
0.9602898565 0.9646596062 0.9681602395
5 0.1488743390 0.2165873427 0.2695431560
0.4333953941 0.4803804169 0.5190961292
0.6794095683 0.7071067812 0.7301520056
0.8650633667 0.8770602346 0.8870625998
0.9739065285 0.9762632447 0.9782286581
7.5. Orthogonal Collocation on Finite in the last element. The orthogonal collocation
Elements method is applied at each interior collocation
point.
In the method of orthogonal collocation on fi- N
P N
P
1 a−1 1
nite elements, the domain is first divided into ∆x2
B I J cJ + x AI J cJ =
k (k) +uI ∆xk ∆xk
J=1 J=1
elements, and then within each element orthog-
= ϕ2 R (cJ ) , I = 2,. . .,N P −1
onal collocation is applied. Figure 27 shows the
domain being divided into NE elements, with The local points i = 2, . . . , NP − 1 represent
NCOL interior collocation points within each el- the interior collocation points. Continuity of the
ement, and NP = NCOL + 2 total points per el- function and the first derivative between ele-
ement, giving NT = NE * (NCOL + 1) + 1 total ments is achieved by taking
number of points. Within each element a local
N
P
coordinate is defined 1
AN P ,J cJ
∆xk−1
J=1
x−x(k) element k−1
u= , ∆xk = x(k+1) −x(k) NP
∆xk 1
= A1,J cJ
∆xk
J=1
element k
The reaction – diffusion equation is written as
1 d
dc
d2 c a−1 dc
at the points between elements. Naturally, the
xa−1 = + = ϕ2 R (c) computer code has only one symbol for the so-
xa−1 dx dx dx2 x dx
lution at a point shared between elements, but
and transformed to give the derivative condition must be imposed. Fi-
1 d2 a−1 1 dc nally, the boundary conditions at x = 0 and x =
+ = ϕ2 R (c) 1 are applied:
∆x2k du2 x(k) +u∆xk ∆xk du
NP
The boundary conditions are typically 1
A1,J cJ = 0,
∆xk J=1
dc dc
(0) = 0, − (1) = Bim [c (1) −cB ]
dx dx in the first element;
where Bim is the Biot number for mass transfer. NP
These become 1
− AN P ,J cJ = Bim [cN P −cB ] ,
∆xN E J=1
1 dc
(u = 0) = 0,
∆x1 du in the last element.
in the first element; These equations can be assembled into an
overall matrix problem
1 dc
− (u = 1) = Bim [c (u = 1) −cB ] ,
∆xN E du A Ac = f
70 Mathematics in Chemical Engineering
Table 9. Matrices for orthogonal collocation with symmetric polynomials and W =1−x 2
The form of these equations is special and is dis- 337]. See also the program COLSYS described
cussed by Finlayson [3, p. 116], who also gives below.
the computer code to solve linear equations aris- The error bounds of DeBoor [91] give the fol-
ing in such problems. Reaction – diffusion prob- lowing results for second-order problems solved
lems are solved by the program OCFERXN [3, p. with cubic trial functions on finite elements with
Mathematics in Chemical Engineering 71
the unknown solution is expanded in a series of Combining this with Equation 26 gives
known functions {bi (x)}, with unknown coef- NT 1 dbj dbi a−1
ficients {ai }. − dx dx
x dxai
i=1 0 & '
N T
NT
−Bim bj (1) ai bi (1) −cB
c (x) = ai bi (x) (28)
&i=1 '
i=1 1 N T
= ϕ2 bj (x) ai bi (x) xa−1 dx
The series (the trial solution) is inserted into the 0 i=1
differential equation to obtain the residual: j = 1,. . .,N T
N
T This equation defines the Galerkin method, and
Residual = 1
ai xa−1 d
xa−1 dbi
i=1
dx dx a solution that satisfies this equation (for all j =
&
N
T
'
1, . . . , ∞) is called a weak solution. For an ap-
−ϕ2 R ai bi (x) proximate solution the equation is written once
i=1
for each member of the trial function, j = 1, . . .
The residual is then made orthogonal to the set , NT. If the boundary condition is
of basis functions.
c (1) = cB
72 Mathematics in Chemical Engineering
Each N i (x) takes the value 1 at the point x i and The matrices for various terms are given in Ta-
zero at all other grid points (Chap. 2). Thus ci ble 10. This equation can also be written in the
are the nodal values, c (x i ) = ci . The first deriva- form
tive must be transformed to the local coordinate
system, u = 0 to 1 when x goes from x i to x i + AAc = f
∆x. where the matrix AA is sparse. If linear elements
dNj 1 dNJ are used the matrix is tridiagonal. If quadratic
= , dx = ∆xe du
dx ∆xe du elements are used the matrix is pentadiagonal.
in the e-th element. Then the Galerkin method Naturally the linear algebra is most efficiently
is carried out if the sparse structure is taken into
account. Once the solution is found the solution
1 NP 1 dNJ dNI at any point can be recovered from
− e ∆x du du
(xe +u∆xe )a−1 duceI
e
I=1 0
& ' ce (u) = ceI=1 (1−u) +ceI=2 u
N
P
ceI NI
−Bim e NJ (1) (1) −c1
I=1 (29) for linear elements
& '
1 N
P
= ϕ2
ceI NI (u) ce (u) = ceI=1 2 (u−1) u− 21
e ∆xe NJ (u) R
0 I=1
a−1
(xe +u∆xe ) du +ceI=2 4u (1−u) +ceI=3 2u u− 12
(xe +uk ∆xe )a−1 These features are built into the code COLSYS
(http://www.netlib.org/ode/).
The error expected from a method one order
higher and one order lower can also be defined.
7.7. Cubic B-Splines Then a decision about whether to increase or
decrease the order of the method can be made
Cubic B-splines have cubic approximations by taking into account the relative work of the
within each element, but first and second deriva- different orders. This provides a method of ad-
tives continuous between elements. The func- justing both the mesh spacing (∆x, sometimes
tions are the same ones discussed in Chapter 2, called h) and the degree of polynomial ( p). Such
and they can be used to solve differential equa- methods are called h – p methods.
tions, too. See Sincovec [92].
7.9. Comparison
7.8. Adaptive Mesh Strategies
What method should be used for any given prob-
In many two-point boundary value problems, lem? Obviously the error decreases with some
the difficulty in the problem is the formation of power of ∆x, and the power is higher for the
a boundary layer region, or a region in which higher order methods, which suggests that the
the solution changes very dramatically. In such error is less. For example, with linear elements
cases small mesh spacing should be used there, the error is
either with the finite difference method or the
finite element method. If the region is known a y (∆x) = yexact +c2 ∆x2
priori, small mesh spacings can be assumed at for small enough (and uniform) ∆x. A computer
the boundary layer. If the region is not known code should be run for varying ∆x to confirm
though, other techniques must be used. These this. For quadratic elements, the error is
techniques are known as adaptive mesh tech-
niques. The general strategy is to estimate the y (∆x) = yexact +c3 ∆x3
error, which depends on the grid size and deriva- If orthogonal collocation on finite elements is
tives of the solution, and refine the mesh where used with cubic polynomials, then
the error is large.
The adaptive mesh strategy was employed by y (∆x) = yexact +c4 ∆x4
Ascher et al. [93] and by Russell and Chris- However, the global methods, not using finite el-
tiansen [94]. For a second-order differential ements, converge even faster [95], for example,
equation and cubic trial functions on finite el- N COL
ements, the error in the i-th element is given by 1
y (N ) = yexact +cN
N COL
Errori = c∆x4i u(4) i
Yet the workload of the methods is also differ-
Because cubic elements do not have a nonzero ent. These considerations are discussed in [3].
fourth derivative, the third derivative in adjacent Here, only sweeping generalizations are given.
elements is used [3, p. 166]: If the problem has a relatively smooth so-
lution, then the orthogonal collocation method
74 Mathematics in Chemical Engineering
is preferred. It gives a very accurate solution, mesh near the singularity, by relying on the bet-
and N can be quite small so the work is small. ter approximation due to a smaller ∆x. Another
If the problem has a steep front in it, the finite approach is to incorporate the singular trial func-
difference method or finite element method is tion into the approximation. Thus, if the solution
indicated, and adaptive mesh techniques should approaches f (x) as x goes to zero, and f (x) be-
probably be employed. Consider the reaction – comes infinite, an approximation may be taken
diffusion problem: as the Thiele modulus ϕ in- as
creases from a small value with no diffusion lim- N
itations to a large value with significant diffusion y (x) = f (x) + ai yi (x)
limitations, the solution changes as shown in i=1
Figure 28. The orthogonal collocation method
This function is substituted into the differential
is initially the method of choice. For interme-
equation, which is solved for ai . Essentially, a
diate values of ϕ, N = 3 – 6 must be used, but
new differential equation is being solved for a
orthogonal collocation still works well (for η
new variable:
down to approximately 0.01). For large ϕ, use of
the finite difference method, the finite element u (x) ≡y (x) −f (x)
method, or an asymptotic expansion for large
ϕ is better. The decision depends entirely on the The differential equation is more complicated
type of solution that is obtained. For steep fronts but has a better solution near the singularity (see
the finite difference method and finite element [97, pp. 189 – 192], [98, p. 611]).
method with adaptive mesh are indicated. Sometimes the domain is infinite. Boundary
layer flow past a flat plate is governed by the Bla-
sius equation for stream function [99, p. 117].
3 2
d d
2 dη 3 +f dη 2 = 0
df
f = dη
= 0 at η = 0
df
dη
= 1 at η→∞
df df dz
Figure 28. Concentration solution for different values of dη
= dz dη
= −z df
dz
Thiele modulus 2
d2 f d2 f d2 z 2
dη 2
= dz 2
dz
dη
+ df
dz dη 2
= z 2 ddzf2 +z df
dz
Another approach is to use a variable mesh, happen in the case of nonlinear problems; an ex-
perhaps with the same transformation. For ex- ample is a compressible flow problem with both
ample, use z = e−η and a constant mesh size in subsonic and supersonic regions. Characteristic
z. Then with 101 points distributed uniformly curves are curves along which a discontinuity
from z = 0 to z = 1, the following are the nodal can propagate. For a given set of equations, it
points: is necessary to determine if characteristics ex-
z = 0.,0.01,0.02,. . .,0.99,1.0 ist or not, because that determines whether the
equations are hyperbolic, elliptic, or parabolic.
η = ∞,4.605,3.912,. . .,0.010,0
Linear Problems For linear problems, the
∆η = ∞,0.639,. . .,0.01 theory summarized by Joseph et al. [105] can
Still another approach is to solve on a finite be used.
mesh in which the last point is far enough away ∂ ∂ ∂
that its location does not influence the solution. , ,. . .,
∂t ∂xi ∂xn
A location that is far enough away must be found
by trial and error. is replaced with the Fourier variables
iξ0 ,iξ1 ,. . .,iξn
real and distinct (excluding zero) then the oper- The equation is thus second order and the type
ator is hyperbolic. is determined by
This formalism is applied to three basic types
−βσ02 +σ12 = 0
of equations. First consider the equation arising
from steady diffusion in two dimensions: The normalization condition
∂2c ∂2c σ02 +σ12 = 1
+ =0
∂x2 ∂y 2
is required. Combining these gives
This gives
1− (1+β) σ02 = 0
−ξ12 −ξ22 = − ξ21 +ξ22 = 0
The roots are real and the equation is hyperbolic.
Thus, When β = 0
σ12 +σ22 = 1 (normalization) ξ12 = 0
The final example is the heat conduction where D is a diffusion coefficient. Special cases
problem written as are the convective diffusive equation
∂T ∂q ∂T ∂c ∂c ∂2c
̺Cp =− , q = −k +u =D 2 (33)
∂t ∂x ∂x ∂t ∂x ∂x
In this formulation the constitutive equation for and Burgers viscosity equation
heat flux is separated out; the resulting set of ∂u ∂u ∂2u
+u =ν 2 (34)
equations is first order and written as ∂t ∂x ∂x
̺Cp ∂T ∂q
+ ∂x =0 where u is the velocity and ν is the kinematic vis-
∂t
cosity. This is a prototype equation for the Navier
k ∂T
∂x
= −q – Stokes equations (→ Fluid Mechanics). For
adsorption phenomena [106, p. 202],
In matrix notation this is
∂c ∂c df ∂c
& '& ∂T
' & '& ∂T
' & ' ϕ +ϕu + (1−ϕ) =0 (35)
̺Cp 0 ∂t
0 1 ∂x
0 ∂t ∂x dc ∂t
+ =
0 0 ∂q
∂t
k 0 ∂q
∂t
−q where ϕ is the void fraction and f (c) gives the
equilibrium relation between the concentrations
This compares with in the fluid and in the solid phase. In these exam-
∂u ∂u ples, if the diffusion coefficient D or the kine-
A0 +A1 =f
∂x0 ∂x1 matic viscosity ν is zero, the equations are hyper-
bolic. If D and ν are small, the phenomenon may
In this case A0 is singular whereas A1 is nonsin-
be essentially hyperbolic even though the equa-
gular. Thus,
tions are parabolic. Thus the numerical methods
det (αA1 −λ0 A0 ) = 0 for hyperbolic equations may be useful even for
parabolic equations.
is considered for any real λ0 . This gives Equations for several methods are given here,
−̺C λ
as taken from [107]. If the convective term is
p 0 α
=0 treated with a centered difference expression the
kα 0
solution exhibits oscillations from node to node,
or and these vanish only if a very fine grid is used.
The simplest way to avoid the oscillations with
α2 k = 0
a hyperbolic equation is to use upstream deriva-
Thus the α is real, but zero, and the equation is tives. If the flow is from left to right, this would
parabolic. give the following for Equations (40):
dci F (ci ) −F (ci−1 ) ci+1 −2ci +ci−1
+ =D
dt ∆x ∆x2
8.2. Hyperbolic Equations for Equation 34:
dui ui −ui−1 ui+1 −2ui +ui−1
The most common situation yielding hyperbolic +ui =ν
dt ∆x ∆x2
equations involves unsteady phenomena with
and for Equation 35:
convection. A prototype equation is
dci ci −ci−1 df dci
∂c ∂F (c) ϕ +ϕui + (1−ϕ) | =0
+ =0 dt ∆x dc i dt
∂t ∂x
If the flow were from right to left, then the for-
Depending on the interpretation of c and F (c), mula would be
this can represent accumulation of mass and con- dci F (ci+1 ) −F (ci ) ci+1 −2ci +ci−1
vection. With F (c) = u c, where u is the veloc- + =D
dt ∆x ∆x2
ity, the equation represents a mass balance on
concentration. If diffusive phenomenon are im- If the flow could be in either direction, a local
portant, the equation is changed to determination must be made at each node i and
the appropriate formula used. The effect of us-
∂c ∂F (c) ∂2c ing upstream derivatives is to add artificial or
+ =D 2 (32)
∂t ∂x ∂x numerical diffusion to the model. This can be
78 Mathematics in Chemical Engineering
ascertained by taking the finite difference form Leaving out the u2 ∆t 2 terms gives the Galerkin
of the convective diffusion equation method. Replacing the left-hand side with
dci ci −ci−1 ci+1 −2ci +ci−1
+u =D cn+1 −cn
i
dt ∆x ∆x2 i
∗n+1
∆tD
+ 2∆x2 ci+1 −2ci∗n+1 +ci−1
∗n+1
u2 ∆t2
u∆t ∆tD Each of these methods tries to avoid oscil-
= − 2∆x cn n
i+1 −ci−1 + ∆x2 + 2∆x2 cn
i+1
lations that would disappear if the mesh were
−2cn n
i +ci−1
Mathematics in Chemical Engineering 79
fine enough. For the steady convective diffusion with boundary and initial conditions
equation these oscillations do not occur provided
c(x,0) = 0
u∆x P e∆
= <=1 (36)
2D 2 c (0,t) = 1, c (L,t) = 0
For large u, ∆x must be small to meet this con- A solution of the form
dition. An alternative is to use a small ∆x in
regions where the solution changes drastically. c (x,t) = T (t) X (x)
Because these regions change in time, the el- is attempted and substituted into the equation,
ements or grid points must move. The crite- with the terms separated to give
ria to move the grid points can be quite com-
1 dT 1 d2 X
plicated, and typical methods are reviewed in =
DT dt X dx2
[107]. The criteria include moving the mesh in
a known way (when the movement is known a One side of this equation is a function of x alone,
priori), moving the mesh to keep some prop- whereas the other side is a function of t alone.
erty (e.g., first- or second-derivative measures) Thus, both sides must be a constant. Otherwise,
uniform over the domain, using a Galerkin or if x is changed one side changes, but the other
weighted residual criterion to move the mesh, cannot because it depends on t. Call the constant
and Euler – Lagrange methods which move part − λ and write the separate equations
of the solution exactly by convection and then dT d2 X
add on some diffusion after that. −λDT , −λX
dt dx2
The final illustration is for adsorption in a The first equation is solved easily
packed bed, or chromatography. Equation 35
can be solved when the adsorption phenomenon T (t) = T (0) e−λDt
is governed by a Langmuir isotherm.
and the second equation is written in the form
αc
f (c) = d2 X
1+Kc +λX = 0
dx2
Similar numerical considerations apply and sim-
ilar methods are available [110 – 112]. Next consider the boundary conditions. If
they are written as
c (L,t) = 1 = T (t) X (L)
8.3. Parabolic Equations in One
c (0,t) = 0 = T (t) X (0)
Dimension
the boundary conditions are difficult to satisfy
In this section several methods are applied to because they are not homogeneous, i.e. with a
parabolic equations in one dimension: separa- zero right-hand side. Thus, the problem must be
tion of variables, combination of variables, fi- transformed to make the boundary conditions
nite difference method, finite element method, homogeneous. The solution is written as the sum
and the orthogonal collocation method. Separa- of two functions, one of which satisfies the non-
tion of variables is successful for linear prob- homogeneous boundary conditions, whereas the
lems, whereas the other methods work for linear other satisfies the homogeneous boundary con-
or nonlinear problems. The finite difference, the ditions.
finite element, and the orthogonal collocation
c (x,t) = f (x) +u (x,t)
methods are numerical, whereas the separation
or combination of variables can lead to analyti- u (0,t) = 0
cal solutions.
u (L,t) = 0
Analytical Solutions. Consider the diffu- Thus, f (0) = 1 and f (L) = 0 are necessary. Now
sion equation the combined function satisfies the boundary
∂c ∂2c conditions. In this case the function f (x) can
=D 2
∂t ∂x be taken as
80 Mathematics in Chemical Engineering
is written in terms of sines and cosines, so that The Galerkin criterion for finding An is the same
the general solution is as the least-squares criterion [3, p. 183]. The so-
√ √ lution is then
X = Bcos λx+Esin λx
∞
x n π x −n2 π2 Dt/L2
The boundary conditions are c (x,t) = 1− + An sin e
√ √ L n=1 L
X (L) = Bcos λL+Esin λL = 0
This is an “exact” solution to the linear prob-
X (0) = B = 0 lem. It can be evaluated to any desired accuracy
by taking more and more terms, but if a finite
If B = 0, then E = 0 is required to have any so-
number of terms are used, some error always oc-
lution at all. Thus, λ must satisfy
curs. For large times a single term is adequate,
√
sin λL = 0 whereas for small times many terms are needed.
For small times the Laplace transform method
This is true for certain values of λ, called eigen- is also useful, because it leads to solutions that
values or characteristic values. Here, they are converge with fewer terms. For small times, the
λn = n2 π 2 /L2 method of combination of variables may be used
Mathematics in Chemical Engineering 81
The other boundary condition is for x = 0 or η = This gives the well-known stability limit
0, D 1
∆t ≤
∆x2 2
f (0) = 1
If other methods are used to integrate in time,
Thus, an ordinary differential equation must be then the stability limit changes according to
solved rather than a partial differential equation. the method. It is interesting to note that the
When the diffusivity is constant the solution is eigenvalues of Equation 37 range from D π 2 /L 2
the well-known complementary error function: (smallest) to 4 D/∆x 2 (largest), depending on
82 Mathematics in Chemical Engineering
the boundary conditions. Thus the problem be- Rearrangement for the case when the velocity
comes stiff as ∆x approaches zero [3, p. 263]. u is the same for all nodes gives
Implicit methods can also be used. Write a
cn+1 −cn
i u(ci+1 −ci−1 ) D
finite difference form for the time derivative and i
+ = (ci+1 −2ci +ci−1 )
∆t 2∆x ∆x2
average the right-hand sides, evaluated at the old
and new times: This is the same equation as obtained using
the finite difference method. This is not always
cn+1
i −cn
i cn n n
i+1 −2ci +ci−1
∆t
= D (1−θ) ∆x2 true, and the finite volume equations are easy to
derive. In two- and three-dimensions, the mesh
cn+1 n+1
i+1 −2ci +cn+1
+Dθ ∆x2
i−1
need not be rectangular, as long as it is possi-
ble to compute the velocity normal to an edge of
Now the equations are of the form the cell. The finite volume method is useful for
applications involving filling, such as injection
− D∆tθ cn+1 + 1+2 D∆tθ
∆x2 i+1 ∆x2
cn+1
i − D∆tθ cn+1
∆x2 i−1
molding, when only part of the cell is filled with
D∆t(1−θ) fluid. Such applications do involve some approx-
= cn
i + ∆x2
cn n n
i+1 −2ci +ci−1
imations, since the interface is not tracked pre-
and require solving a set of simultaneous equa- cisely, but they are useful engineering approxi-
tions, which have a tridiagonal structure. Using mations.
θ = 0 gives the Euler method (as above); θ = 0.5
gives the Crank – Nicolson method; θ = 1 gives
the backward Euler method. The stability limit
is given by
D∆t 0.5
≤
∆x2 1−2θ
whereas the oscillation limit is given by
D∆t 0.25 Figure 30.
≤
∆x2 1−θ
If a time step is chosen between the oscillation The finite element method is handled in a sim-
limit and stability limit, the solution will oscil- ilar fashion, as an extension of two-point bound-
late around the exact solution, but the oscilla- ary value problems by letting the solution at the
tions remain bounded. For further discussion, nodes depend on time. For the diffusion equation
see [3, p. 218]. the finite element method gives
Finite volume methods are utilized exten- dceI
e e
sively in computational fluid dynamics. In this CJI = BJI ceI
e I dt e I
method, a mass balance is made over a cell ac-
counting for the change in what is in the cell and with the mass matrix defined by
the flow in and out. Figure 30 illustrates the ge- 1
ometry of the i-th cell. A mass balance made on e
CJI = ∆xe NJ (u) NI (u) du
this cell (with area A perpendicular to the paper) 0
is
This set of equations can be written in matrix
A∆x(cn+1
i −cn
i ) = ∆tA(Jj−1/2 −Ji+1/2 )
form
where J is the flux due to convection and diffu- dc
CC = AAc
sion, positive in the +x direction. dt
8.4. Elliptic Equations complicated expressions are needed and the fi-
nite element method may be the better method.
Elliptic equations can be solved with both fi- Equation 40 is rewritten in the form
nite difference and finite element methods. One- 2
2
dimensional elliptic problems are two-point 2 1+ ∆x
∆y 2
Ti,j = Ti+1,j +Ti−1,j + ∆x
∆y 2
boundary value problems and are covered in Qi,j
(Ti,j+1 +Ti,j−1 ) −∆x2
Chapter 7. Two-dimensional elliptic problems k
are often solved with direct methods, and it- And this is converted to the Gauss–Seidel itera-
erative methods are usually used for three- tive method.
dimensional problems. Thus, two aspects must 2
s+1 s+1
be considered: how the equations are discretized 2 1+ ∆x
∆y 2
Ti,j s
= Ti+1,j +Ti−1,j
to form sets of algebraic equations and how the 2
s+1
Q
algebraic equations are then solved. + ∆x
∆y 2
s
Ti,j+1 +Ti,j−1 −∆x2 ki,j
The prototype elliptic problem is steady-state
Calculations proceed by setting a low i, com-
heat conduction or diffusion,
puting from low to high j, then increasing i and
∂2T ∂2T repeating the procedure. The relaxation method
k + =Q
∂x2 ∂y 2 uses
possibly with a heat generation term per unit 2
∆x2
s+1
2 1+ ∆x
∆y 2
∗ = Ts
Ti,j i+1,j +Ti−1,j + ∆y 2
volume, Q. The boundary conditions can be
s s+1 Q
Ti,j+1 −Ti,j−1 −∆x2 ki,j
Dirichlet or 1st kind: T = T 1 on boundary S 1
s+1 s ∗ s
Ti,j = Ti,j +β Ti,j −Ti,j
Neumann or 2nd kind: k ∂T
∂n = q2 on boundary S 2
If β = 1, this is the Gauss – Seidel method. If β
Robin, mixed, or 3rd kind: −k ∂T
∂n = h (T −T3 ) on
boundary S 3 > 1, it is overrelaxation; if β < 1, it is underre-
laxation. The value of β may be chosen empiri-
Illustrations are given for constant physical cally, 0 < β < 2, but it can be selected theoreti-
properties k, h, while T 1 , q2 , T 3 are known func- cally for simple problems like this [117, p. 100],
tions on the boundary and Q is a known function [3, p. 282]. In particular, the optimal value of the
of position. For clarity, only a two-dimensional iteration parameter is given by
problem is illustrated. The finite difference for-
mulation is given by using the following nomen- −ln (βopt −1) ≈R
clature and the error (in solving the algebraic equation)
Ti,j = T (i∆x,j∆y)
is decreased by the factor (1 − R)N for every N
iterations. For the heat conduction problem and
The finite difference formulation is then Dirichlet boundary conditions,
Ti+1,j −2Ti,j +Ti−1,j
π2
∆x2 R=
(40) 2n2
Ti,j+1 −2Ti,j +Ti,j−1
+ ∆y 2
= Qi,j /k (when there are n points in both x and y direc-
tions). For Neumann boundary conditions, the
Ti,j = T1 for i,j on boundary S1
value is
k ∂T | = q2 for i,j on boundary S2
∂n i,j π2 1
R=
2n2 1+max [∆x2 /∆y 2 ,∆y 2 /∆x2 ]
−k ∂T | = h (Ti,j −T3 ) for i,j on boundary S3
∂n i,j
Iterative methods can also be based on lines
If the boundary is parallel to a coordinate axis the (for 2D problems) or planes (for 3D problems).
boundary slope is evaluated as in Chapter 7, by Preconditioned conjugate gradient methods
using either a one-sided, centered difference or have been developed (see Chap. 1). In these
a false boundary. If the boundary is more irreg- methods a series of matrix multiplications are
ular and not parallel to a coordinate line, more done iteration by iteration; and the steps lend
Mathematics in Chemical Engineering 85
themselves to the efficiency available in paral- represent a large set of linear equations, which
lel computers. In the multigrid method the prob- are solved using matrix techniques (Chap. 1).
lem is solved on several grids, each more refined If the problem is nonlinear, e.g., with k or Q
than the previous one. In iterating between the a function of temperature, the equations must be
solutions on different grids, one converges to the solved iteratively. The integrals are given for a
solution of the algebraic equations. A chemical triangle with nodes I, J, and K in counterclock-
engineering application is given in [118]. Soft- wise order. Within an element,
ware for a variety of these methods is available,
T = NI (x,y) TI +NJ (x,y) TJ +NK (x,y) TK
as described below.
aI +bI x+cI y
The Galerkin finite element method (FEM) is NI = 2∆
useful for solving elliptic problems and is partic-
aI = xJ yK −xK yJ
ularly effective when the domain or geometry is
irregular [119 – 125]. As an example, cover the bI = yI −yK
domain with triangles and define a trial func-
cI = xK −xJ
tion on each triangle. The trial function takes
the value 1.0 at one corner and 0.0 at the other plus permutation on I,K,J
corners, and is linear in between (see Fig. 31).
1 xI yI
These trial functions on each triangle are pieced
together to give a trial function on the whole 2∆ = det 1 xJ yJ = 2 (area of triangle)
domain. For the heat conduction problem the 1 xK yK
method gives [3] aI +aJ +aK = 1
bI +bJ +bK = 0
cI +cJ +cK = 0
k
AeIJ = − 4∆ (bI bJ +cI cJ )
e = Q QD
FIJ 2
(aI +bI x+cI y) = 3
xI +xJ +xK yI +yJ +yK
x= 3
, y= 3
2
aI +bI x+cI y = 3
∆
or the unsteady diffusion equation This can be done by using the finite differ-
ence [132, p. 162] or the finite element method
∂2T ∂2c
∂c
=D + −R (c)
[133 – 135].
∂t ∂x2 ∂y 2 In fluid flow problems solved with the finite
In the finite difference method an explicit element method, the basis functions for pressure
technique would evaluate the right-hand side at and velocity are often different. This is required
the n-th time level: by the LBB condition (named after Ladyshen-
skaya, Brezzi, and Babuska) [134, 135]. Some-
times a discontinuous basis function is used for
pressure to meet this condition. Other times a
Mathematics in Chemical Engineering 87
penalty term is added, or the quadrature is done Consider the lattice shown in Figure 32. The var-
using a small number of quadrature points. Thus, ious velocities are
one has to be careful how to apply the finite el-
c1 = (1,0), c3 = (0,1), c5 = (−1,0), c7 = (0,−1)
ement method to the Navier – Stokes equations.
Fortunately, software exists that has taken this c2 = (1,1), c4 = (−1,1), c6 = (−1,−1), c8 = (1,−1)
into account.
c0 = (0,0)
Level Set Methods. Multiphase problems
are complicated because the terms in the equa- The density, velocity, and shear stress (for
tions depend on which phase exists at a partic- some k) are given by
ular point, and the phase boundary may move ρ= u=
fi (x,t), ρu ci fi (x,t),
or be unknown. It is desirable to compute on i i
represents the location of the interface. For ex- where the weighting functions are
ample, in flow problems the level set function is
4 1
defined as the solution to w0 = , w1 = w3 = w5 = w7 = ,
9 9
∂φ 1
+u·∇φ = 0 w2 = w 4 = w 6 = w 8 =
∂t 36
The physics governing the velocity of the With these conditions, the solution for veloc-
interface must be defined, and this equation is ity is a solution of the Navier—Stokes equation.
solved along with the other equations repre- These equations lead to a large computational
senting the problem [130 – 137]. problem, but it can be solved by parallel pro-
cessing on multiple computers.
Lattice Boltzmann Methods. Another way
to solve fluid flow problems is based on a mo-
lecular viewpoint and is called the Lattice Boltz-
mann method [138 – 141]. The treatment here
follows [142]. A lattice is defined, and one solves
the following equation for fi (x,t), the probabil-
ity of finding a molecule at the point x with speed
ci .
eq
∂fi fi −fi
+ci ·∇fi = −
∂t τ
The right-hand side represents a single time
relaxation for molecular collisions, and τ is re-
lated to the kinematic viscosity. By means of
a simple stepping algorithm, the computational Figure 32.
algorithm is
∆t eq
x+cci ∆t,t+∆t) = fi (x
fi (x x,t)− (fi −fi )
τ
88 Mathematics in Chemical Engineering
rectly. Four widely used major packages are said to be weakly singular. A Volterra equation
Fluent (http://www.fluent.com/), CFX (now part of the second kind is
of ANSYS), Comsol Multiphysics (formerly t
FEMLAB) (http://www.comsol.com/), and AN- y (t) = g (t) +λ K(t,s)y (s) ds (42)
SYS (http://www.ansys.com/). Of these, Com-
a
sol Multiphysics is particularly useful because
it has a convenient graphical user interface, whereas a Volterra equation of the first kind is
permits easy mesh generation and refinement t
(including adaptive mesh refinement), allows y (t) = λ K (t,s) y (s) ds
the user to add in phenomena and additional a
equations easily, permits solution by continu-
ation methods (thus enhancing convergence), Equations of the first kind are very sensitive to
and has extensive graphical output capabilities. solution errors so that they present severe nu-
Other packages are also available (see http://cfd- merical problems.
online.com/), and these may contain features An example of a problem giving rise to a
and correlations specific to the engineer’s indus- Volterra equation of the second kind is the fol-
try. One important point to note is that for tur- lowing heat conduction problem:
bulent flow, all the programs contain approxima- ̺Cp ∂T
2
= k ∂∂xT2 , 0≤x<∞, t>0
∂t
tions, using the k-epsilon models of turbulence,
or large eddy simulations; the direct numerical ∂T
T (x,0) = 0, ∂x
(0,t) = −g (t) ,
simulation of turbulence is too slow to apply it to
∂T
very big problems, although it does give insight lim T (x,t) = 0, lim =0
x→∞ x→∞ ∂x
(independent of any approximations) that is use-
ful for interpreting turbulent phenomena. Thus, If this is solved by using Fourier transforms the
the method used to include those turbulent cor- solution is
relations is important, and the method also may t
1 1 2
affect convergence or accuracy. T (x,t) = √ g (s) √ e−x /4(t−s) ds
π t−s
0
9.2. Numerical Methods for Volterra 9.3. Numerical Methods for Fredholm,
Equations of the Second Kind Urysohn, and Hammerstein Equations
of the Second Kind
Volterra equations of the second kind are anal-
ogous to initial value problems. An initial value Whereas Volterra equations could be solved
problem can be written as a Volterra equation of from one position to the next, like initial value
the second kind, although not all Volterra equa- differential equations, Fredholm equations must
tions can be written as initial value problems be solved over the entire domain, like bound-
[151, p. 7]. Here the general nonlinear Volterra ary value differential equations. Thus, large sets
equation of the second kind is treated (Eq. 45). of equations will be solved and the notation is
The simplest numerical method involves replac- designed to emphasize that.
ing the integral by a quadrature using the trape- The methods are also based on quadrature
zoid rule. formulas. For the integral
#
1 b
yn ≡y (tn ) = g (tn ) +∆t K (tn ,t0 ,y0 )
2 I (ϕ) = ϕ (y) dy
n−1
1 a
+ K (tn ,ti ,yi ) + K (tn ,tn ,yn )
i=1
2 a quadrature formula is written:
n
This equation is a nonlinear algebraic equation
I (ϕ) = wi ϕ (yi )
for yn . Since y0 is known it can be applied to
i=0
solve for y1 , y2 , . . . in succession. For a single
integral equation, at each step one must solve a Then the integral Fredholm equation can be
single nonlinear algebraic equation for yn . Typ- rewritten as
ically, the error in the solution to the integral n
equation is proportional to ∆t µ , and the power f (x) −λ
i=0
wi K (x,yi ) f (yi ) = g (x) ,
(46)
µ is the same as the power in the quadrature error
a≤x≤b
[151, p. 97].
The stability of the method [151, p. 111] can If this equation is evaluated at the points x = yj ,
be examined by considering the equation n
∂u 2
∂t
= D∇2 u, u = 0 on S, 1+ Sh −x, y≤x
G (x,y,Sh) =
2
with a point source at x0 ,y0 ,z0 1+ Sh −y, x<y
a=2
2
Green’s function is [44, p. 355] Sh
−lnx, y≤x
G (x,y,Sh) = 2
Sh
−lny, x<y
1
u=
8[πD(t−τ )]3/2 a=3
2
−[(x−x0 )2 +(y−y0 )2 +(z−z0 )2 ]/4 D(t−τ ) Sh
+ x1 −1, y≤x
e G (x,y,Sh) = 2
Sh
+ y1 −1, x<y
When the problem is
Green’s functions for the reaction diffusion
∂c
∂t
= D∇2 c problem were used to provide computable error
bounds by Ferguson and Finlayson [163].
c = f (x,y,z) in S at t = 0 If Green’s function has the form
c = ϕ (x,y,z) on S,t>0 u (x) v (y) 0≤y≤x
K (x,y) =
u (y) v (x) x≤y≤1
the solution can be represented as [44, p. 353]
c=
(u)τ =0 f (x,y,z) dxdydz the problem
1
t ∂u
+D ϕ (x,y,z,τ ) ∂n
dSdt f (x) = K (x,y) F [y,f (y)] dy
0
0
When the problem is two dimensional, may be written as
2 2
u= √ 1 e−[(x−x0 ) +(y−y0 ) ]/4D(t−τ ) x
4πD(t−τ ) f (x) − [u (x) v (y) −u (y) v (x)] ·
0
c= (u)τ =0 f (x,y) dxdy
f [y,f (y)] dy = αv (x)
t ∂u
+D ϕ (x,y,τ ) ∂n
dCdt where
0
1
For the following differential equation and α= u (y) F [y,f (y)] dy
boundary conditions 0
1 d
dc
Thus, the problem ends up as one directly for-
xa−1 dx
xa−1 dx = f [x,c (x)] , mulated as a fixed point problem:
dc 2 dc
dx
(0) = 0, Sh dx
(1) +c (1) = g f = Φ (f )
where Sh is the Sherwood number, the problem When the problem is the diffusion – reaction
can be written as a Hammerstein integral equa- one, the form is
tion: x
c (x) = g− [u (x) v (y) −u (y) v (x)]
0
1 f [y,c (y)] y a−1 dy−αv (x)
c (x) = g− G (x,y,Sh) f [y,c (y)] y a−1 dy
0
1
α= u (y) f [y,c (y)] y a−1 dy
0
Green’s function for the differential operators
are [163] Dixit and Taularidis [164] solved problems
involving Fischer – Tropsch synthesis reactions
in a catalyst pellet using a similar method.
94 Mathematics in Chemical Engineering
θ=2π
9.6. Boundary Integral Equations and
∂lnr0
lim ϕ r0 dθ = −ϕ (P ) 2π
Boundary Element Method r0 →0 ∂n
θ=0
The boundary element method utilizes Green’s Thus for an internal point,
theorem and integral equations. Here, the 1
∂lnr ∂ϕ
method is described briefly for the following ϕ (P ) = ϕ −lnr dS (47)
2π ∂n ∂n
boundary value problem in two or three dimen- S
so that
∂ψ ∂ϕ
ϕ −ψ dS = 0
∂n ∂n
S
∇2 ϕ = g (x,y,ϕ)
Then the integral appearing in Equation 50 must
be evaluated over the entire domain, and the so-
lution in the interior is given by Equation 49. For
further applications, see [167] and [168].
10. Optimization
Figure 34. Trial function on boundary for boundary finite
element method We provide a survey of systematic methods for
a broad variety of optimization problems. The
survey begins with a general classification of
mathematical optimization problems involving
N
∂lnri ∂ϕj
πϕi ϕ j
dS− lnri ds
continuous and discrete (or integer) variables.
∂n ∂n
j=1 sj sj This is followed by a review of solution methods
of the major types of optimization problems for
The function ϕj is of course known along s1 ,
continuous and discrete variable optimization,
whereas the derivative ∂ϕj /∂n is known along
particularly nonlinear and mixed-integer nonlin-
s2 . This set of equations is then solved for ϕi and
ear programming. In addition, we discuss direct
∂ϕi /∂n along the boundary. This constitutes the
search methods that do not require derivative in-
boundary integral method applied to the Laplace
formation as well as global optimization meth-
equation.
ods. We also review extensions of these methods
If the problem is Poisson’s equation
for the optimization of systems described by dif-
∇2 ϕ = g (x,y) ferential and algebraic equations.
have multiple local optima, and an important to a mixed-integer nonlinear program (MINLP)
question is whether a global solution is required when any of the functions involved are nonlin-
for the NLP. Another important distinction is ear. If all functions are linear it corresponds to
whether the problem is assumed to be differ- a mixed-integer linear program (MILP). If there
entiable or not. are no 0–1 variables, then problem 51 reduces
Mixed-integer problems also include discrete to a nonlinear program 52 or linear program 65
variables. These can be written as mixed-integer depending on whether or not the functions are
nonlinear programs (MINLP) or as mixed- linear.
integer linear programs (MILP) if all variables
appear linearly in the constraint and objective
functions. For the latter an important case oc-
curs when all the variables are integer; this gives
rise to an integer programming (IP) problem. IPs
can be further classified into many special prob-
lems (e.g., assignment, traveling salesman, etc.),
which are not shown in Figure 35. Similarly, the
MINLP problem also gives rise to special prob-
lem classes, although here the main distinction
is whether its relaxation is convex or nonconvex.
The ingredients of formulating optimization
problems include a mathematical model of the
system, an objective function that quantifies a
criterion to be extremized, variables that can
serve as decisions, and, optionally, inequality
constraints on the system. When represented in
algebraic form, the general formulation of dis-
crete/continuous optimization problems can be Figure 35. Classes of optimization problems and algorithms
written as the following mixed-integer optimiza-
tion problem:
Min f (x,y)
We first start with continuous variable opti-
mization and consider in the next section the so-
s.t. h (x,y) =0 lution of NLPs with differentiable objective and
(51) constraint functions. If only local solutions are
g (x,y) ≤0
required for the NLP, then very efficient large-
x∈ℜn ,y∈{0,1}t scale methods can be considered. This is fol-
where f (x, y) is the objective function (e.g., lowed by methods that are not based on local
cost, energy consumption, etc.), h(x, y) = 0 are optimality criteria; we consider direct search op-
the equations that describe the performance of timization methods that do not require deriva-
the system (e.g., material balances, production tives, as well as deterministic global optimiza-
rates), the inequality constraints g(x, y) ≤ 0 can tion methods. Following this, we consider the
define process specifications or constraints for solution of mixed-integer problems and outline
feasible plans and schedules, and s.t. denotes the main characteristics of algorithms for their
subject to. Note that the operator Max f (x) is solution. Finally, we conclude with a discussion
equivalent to Min −f (x). We define the real n- of optimization modeling software and its im-
vector x to represent the continuous variables plementation in engineering models.
while the t-vector y represents the discrete vari-
ables, which, without loss of generality, are of-
ten restricted to take 0–1 values to define logi- 10.2. Gradient-Based Nonlinear
cal or discrete decisions, such as assignment of Programming
equipment and sequencing of tasks. (These vari-
ables can also be formulated to take on other in- For continuous variable optimization we con-
teger values as well.) Problem 51 corresponds sider problem 51 without discrete variables y.
Mathematics in Chemical Engineering 97
The general NLP problem 52 is presented be- direction. Invoking a theorem of the alterna-
low: tive (e.g., Farkas – Lemma) leads to the cele-
brated Karush – Kuhn – Tucker (KKT) condi-
Min f (x)
tions [169]. Instead of a formal development of
s.t. h (x) =0 (52) these conditions, we present here a more intu-
itive, kinematic illustration. Consider the con-
g (x) ≤0
tour plot of the objective function f (x) given in
and we assume that the functions f (x), h(x), and Figure 36 as a smooth valley in space of the vari-
g(x) have continuous first and second deriva- ables x 1 and x 2 . For the contour plot of this un-
tives. A key characteristic of problem 52 is constrained problem, Min f (x), consider a ball
whether the problem is convex or not, i.e., rolling in this valley to the lowest point of f (x),
whether it has a convex objective function and a denoted by x*. This point is at least a local min-
convex feasible region. A function φ(x) of x in imum and is defined by a point with zero gra-
some domain X is convex if and only if for all dient and at least nonnegative curvature in all
points x 1 , x 2 ∈ X: (nonzero) directions p. We use the first deriva-
tive (gradient) vector ∇f (x) and second deriva-
φ [αx1 + (1−α) x2 ] ≤αφ (x1 ) + (1−α) φ (x2 ) (53) tive (Hessian) matrix ∇xx f (x) to state the nec-
essary first- and second-order conditions for un-
holds for all α ∈ (0, 1). If φ(x) is differentiable, constrained optimality:
then an equivalent definition is:
∇x f (x∗ ) =0 pT ∇xx f (x∗ ) p≥0 for all p=0 (55)
T
φ (x1 ) +∇φ(x1 ) (x−x1 ) ≤φ (x) (54) These necessary conditions for local optimality
Strict convexity requires that the inequalities 53 can be strengthened to sufficient conditions by
and 54 be strict. Convex feasible regions require making the inequality in the relations 55 strict
g(x) to be a convex function and h(x) to be linear. (i.e., positive curvature in all directions). Equiv-
If 52 is a convex problem, than any local solution alently, the sufficient (necessary) curvature con-
is guaranteed to be a global solution to 52. More- ditions can be stated as: ∇xx f (x∗ ) has all pos-
over, if the objective function is strictly convex, itive (nonnegative) eigenvalues and is therefore
then this solution x* is unique. On the other defined as a positive (semi-)definite matrix.
hand, nonconvex problems may have multiple
local solutions, i.e., feasible solutions that min-
imize the objective function within some neigh-
borhood about the solution.
We first consider methods that find only lo-
cal solutions to nonconvex problems, as more
difficult (and expensive) search procedures are
required to find a global solution. Local methods
are currently very efficient and have been devel-
oped to deal with very large NLPs. Moreover,
by considering the structure of convex NLPs
(including LPs and QPs), even more powerful
methods can be applied. To study these methods,
we first consider conditions for local optimality.
h(x) = 0 as “rails”. Consider now a ball, con- is required. The KKT conditions are derived
strained on a rail and within fences, to roll to from gradient information, and the CQ can be
its lowest point. This stationary point occurs viewed as a condition on the relative influence of
when the normal forces exerted by the fences constraint curvature. In fact, linearly constrained
[−∇g (x∗ )] and rails [−∇h (x∗ )] on the ball are problems with nonempty feasible regions re-
balanced by the force of “gravity” [−∇f (x∗ )]. quire no constraint qualification. On the other
This condition can be stated by the following hand, as seen in Figure 38, the problem Min x 1 ,
KKT necessary conditions for constrained opti- s.t. x 2 ≥ 0, (x 1 )3 ≥ x 2 has a minimum point at the
mality: origin but does not satisfy the KKT conditions
because it does not satisfy a CQ.
pT ∇xx L(x∗ ,λ,ν)p≥0 for all and f (x*) = −1.8421. Checking the second-
order conditions 59 leads to:
p=0, ∇h(x∗ )T p=0,
∇xx L(x∗,λ∗,ν∗)=∇xx [f (x∗)+h(x∗)λ∗+g(x∗)ν∗] =
∇gA (x∗ )T p=0 (59)
2+1/(x1 )2
The corresponding sufficient conditions require 1−ν 2.5 0.068
=
that the inequality in 59 be strict. Note that for the
1−ν 3+1/(x2 )2 0.068 3.125
example in Figure 36, the allowable directions
p span the entire space for x while in Figure 37, [∇h(x∗)|∇gA (x∗)]
there are no allowable directions p.
Example: To illustrate the KKT conditions, 2 −2.8284
consider the following unconstrained NLP: p= p=0,p=0
−1 −1.4142
Min (x1 )2 −4x1 +3/2(x2 )2
Note that LICQ is satisfied. Moreover, because
−7x2 +x1 x2 +9−lnx1 −lnx2 (60)
[∇h (x∗ ) |∇gA (x∗ )] is square and nonsingular,
corresponding to the contour plot in Figure 36. there are no nonzero vectors p that satisfy the al-
The optimal solution can be found by solving lowable directions. Hence, the sufficient second-
for the first order conditions 54: order conditions (pT ∇xx L(x∗ ,λ∗ ,ν ∗ )>0, for all
allowable p) are vacuously satisfied for this
& '
2x1 −4+x2 −1/x1
∇f (x) = =0 problem.
3x2 −7+x1 −1/x2
& '
1.3475 Convex Cases of NLP. Linear programs and
⇒x∗ = (61)
2.0470 quadratic programs are special cases of prob-
and f (x*) = −2.8742. Checking the second- lem 52 that allow for more efficient solution,
order conditions leads to: based on application of KKT conditions 56–59.
&
2
' Because these are convex problems, any locally
2+1/ x∗1 1
∇xx f (x∗)= 2 ⇒ optimal solution is a global solution. In partic-
1 3+1/ x∗2
& ' ular, if the objective and constraint functions in
2.5507 1 problem 52 are linear then the following linear
∇xx f (x∗)= (positive definite) (62)
1 3.2387 program (LP):
Now consider the constrained NLP: Min cT x
Min 2 2
(x1 ) −4x1 +3/2(x1 ) −7x2 +x1 x2 s.t. Ax=b (65)
+9−lnx1 −lnx2 Cx≤d
(63)
s.t. 4−x1 x2 ≤0 can be solved in a finite number of steps, and
2x1 −x2 =0 the optimal solution lies at a vertex of the poly-
that corresponds to the plot in Figure 37. The hedron described by the linear constraints. This
optimal solution can be found by applying the is shown in Figure 39, and in so-called primal
first-order KKT conditions 56–58: degenerate cases, multiple vertices can be alter-
∇L(x,λ,ν)=∇f (x) +∇h (x) λ+∇g (x) ν=
nate optimal solutions with the same values of
the objective function. The standard method to
2x1 −4+x2 −1/x1
2
−x2
solve problem 65 is the simplex method, devel-
+ λ+ ν=0 oped in the late 1940s [170], although, starting
3x2 −7+x1 −1/x2 −1 −x1 from Karmarkar’s discovery in 1984, interior
g (x) =4−x1 x2 ≤0, h (x) =2x1 −x2 =0 point methods have become quite advanced and
competitive for large scale problems [171]. The
g (x) ν=(4−x1 x2 )ν, ν≥0
simplex method proceeds by moving succes-
⇓ sively from vertex to vertex with improved ob-
jective function values. Methods to solve prob-
lem 65 are well implemented and widely used,
1.4142
x∗= , λ∗=1.036, ν∗=1.068
2.8284
100 Mathematics in Chemical Engineering
especially in planning and logistical applica- If the matrix Q is positive semidefinite (posi-
tions. They also form the basis for MILP meth- tive definite), when projected into the null space
ods (see below). Currently, state-of-the-art LP of the active constraints, then problem 66 is
solvers can handle millions of variables and con- (strictly) convex and the QP is a global (and
straints and the application of further decom- unique) minimum. Otherwise, local solutions
position methods leads to the solution of prob- exist for problem 66, and more extensive global
lems that are two or three orders of magnitude optimization methods are needed to obtain the
larger than this [172, 173]. Also, the interior global solution. Like LPs, convex QPs can be
point method is described below from the per- solved in a finite number of steps. However, as
spective of more general NLPs. seen in Figure 40, these optimal solutions can
lie on a vertex, on a constraint boundary, or in
the interior. A number of active set strategies
have been created that solve the KKT conditions
of the QP and incorporate efficient updates of
active constraints. Popular QP methods include
null space algorithms, range space methods, and
Schur complement methods. As with LPs, QP
problems can also be solved with interior point
methods [171].
∇f (x) +∇h (x) λ+∇g (x) ν=0 (67a) step for 67a–67c applied at iteration k. Also, se-
lection of the active set is now handled at the QP
h (x) =0 (67b) level by satisfying the conditions 69d, 69e. To
evaluate and change candidate active sets, QP
g (x) +s=0 (67c) algorithms apply inexpensive matrix-updating
strategies to the KKT matrix associated with the
SV e=0 (67d)
QP 68. Details of this approach can be found in
[175, 176].
As alternatives that avoid the combinatorial
(s,ν) ≥0 (67e)
problem of selecting the active set, interior point
where e = [1, 1, . . ., 1]T , S = diag(s) and V = (or barrier) methods modify the NLP problem 52
diag(ν). SQP methods find solutions that sat- to form problem 70
isfy Equations 67a, 67b, 67c, 67d and 67e by Min f xk −µ i ln si
generating Newton-like search directions at it- s.t. k
h x =0 (70)
eration k. However, equations 67d and active g xk +s=0
bounds 67e are dependent and serve to make
the KKT system ill-conditioned near the solu- where the solution to this problem has s > 0 for
tion. SQP algorithms treat these conditions in the penalty parameter µ > 0, and decreasing µ to
two ways. In the active set strategy, discrete de- zero leads to solution of problem 52. The KKT
cisions are made regarding the active constraint conditions for this problem can be written as
set, i∈I= {i|gi (x∗ ) =0}, and Equation 67d is Equation 71
replaced by si = 0, i ∈ I, and vi = 0, i ∈ I. Deter-
∇f (x∗) +∇h (x∗) λ+∇g (x∗) ν=0
mining the active set is a combinatorial problem,
and a straightforward way to determine an esti- h (x∗) =0
mate of the active set [and also satisfy 67e] is to (71)
formulate, at a point x k , and solve the following g (x∗) +s=0
QP at iteration k: SV e=µe
T and at iteration k the Newton steps to solve 71
Min ∇f xk p+ 21 pT ∇xx L xk ,λk ,ν k p
s.t.
T
h xk +∇h xk p=0 (68) are given by:
k k T
g x +∇g x p+s=0, s≥0
∇xx L(xk ,λk ,νk ) ∇h (xk ) ∇g (xk )
The KKT conditions of 68 are given by:
Sk−1 Vk I
∇h(xk )T
∇f x +∇ L xk ,λk ,ν k p
k 2
∇g(xk )T I
∆x ∇x L(xk ,λk ,νk )
+∇h xk λ+∇g xk ν=0 (69a) ∆s ν −S −1 µe
k k
=− (72)
∆λ h (xk )
T
h xk +∇h xk p=0 (69b) ∆ν g (xk ) +sk
T A detailed description of this algorithm, called
g xk +∇g xk p+s=0 (69c)
IPOPT, can be found in [177].
Both active set and interior point methods
SV e = 0 (69d) have clear trade-offs. Interior point methods may
require more iterations to solve problem 70 for
(s,ν) ≥0 (69e) various values of µ, while active set methods
where the Hessian of require the solution of the more expensive QP
the Lagrange function subproblem 68. Thus, if there are few inequality
T T
∇xx L (x,λ,ν) =∇xx f (x) +h(x) λ+g(x) ν constraints or an active set is known (say from
is calculated directly or through a quasi- a good starting guess, or a known QP solution
Newtonian approximation (created by differ- from a previous iteration) then solving problem
ences of gradient vectors). It is easy to show 68 is not expensive and the active set method
that 69a–69c correspond to a Newton–Raphson is favored. On the other hand, for problems with
102 Mathematics in Chemical Engineering
The SOLVER code has been incorporated into not checked by methods that do not use second
Microsoft Excel. CONOPT [184] is an efficient derivatives). With the recent development of
and widely used code in several optimization automatic differentiation tools, many model-
modeling environments. ing and simulation platforms can provide ex-
MINOS [185] is a well-implemented package act first and second derivatives for optimiza-
that offers a variation on reduced gradient strate- tion. When second derivatives are available
gies. At iteration k, equation 75d is replaced by for the objective or constraint functions, they
its linearization (Eq. 77) can be used directly in LANCELOT, SQP and,
k k k
T T less efficiently, in reduced gradient methods.
c(zN ,zS ,zB )+∇B c z k (zB −zB
k
)+∇S c z k
Otherwise, for problems with few superbasic
(zS −zSk )=0 (77) variables, reduced gradient methods and re-
duced space variants of SQP can be applied.
and (75a–75c, 75e) are solved with Equation 77
Referring to problem 74 with n variables and
as a subproblem using concepts from the re-
m equalities, we can write the QP step from
duced gradient method. At the solution of this
problem 68 as Equation 78.
subproblem, the constraints 75d are relinearized
and the cycle repeats until the KKT conditions of p+1 2 pT ∇xx L z k ,γ k p
1
T
Min ∇f z k
75a, 75b, 75c, 75d and 75e are satisfied. The aug- T
mented Lagrangian function 73 is used to penal- s.t. c z k +∇c z k p=0 (78)
ize movement away from the feasible region. For a≤z k +p≤b
problems with few degrees of freedom, the re-
sulting approach leads to an extremely efficient Defining the search direction as p = Z k pZ +
method even for very large problems. MINOS Y k pY , where ∇c(x k )T Z k = 0 and [Y k |Z k ] is
has been interfaced to a number of modeling a nonsingular n × n matrix, allows us to form
systems and enjoys widespread use. It performs the following reduced QP subproblem (with
especially well on large problems with few non- n−m variables)
linear constraints. However, on highly nonlinear "T
ZkT ∇f z k +wk pZ + 21 pz T Bk pZ
!
Min
problems it is usually less reliable than other re- (79)
duced gradient methods. s.t. a≤z k +Zk pZ +Yk pY ≤b
Algorithmic Details for NLP Methods. All where pY = −[∇c(zk )T Y k ]−1 c(zk ). Good
of the above NLP methods incorporate con- choices of Y k and Z k , which allow sparsity
cepts from the Newton – Raphson Method for k T
to be exploited in ∇c(z ), are: Yk =[0|I]
equation solving. Essential features of these
−1
and ZkT = I|−∇N,S c z k ∇B c z k .
methods are a) providing accurate derivative in-
formation to solve for the KKT conditions, Here we define the
reduced Hessian
Bk =ZzT ∇zz L z k,γ k Zk and wk
b) stabilization strategies to promote conver- =
gence of the Newton-like method from poor ZkT ∇zz L z k , γ k Yk pY . In the absence of
starting points, and c) regularization of the Ja- second-derivative information, Bk can be
cobian matrix in Newton’s method (the so- approximated using positive definite quasi-
called KKT matrix) if it becomes singular or Newton approximations [175]. Also, for the
ill-conditioned. interior point method, a similar reduced space
decomposition can be applied to the Newton
a) Providing first and second derivatives: The step given in 72.
KKT conditions require first derivatives to de- Finally, for problems with least-squares func-
fine stationary points, so accurate first deriva- tions, as in data reconciliation, parameter es-
tives are essential to determine locally optimal timation, and model predictive control, one
solutions for differentiable NLPs. Moreover, can often assume that the values of the objec-
Newton–Raphson methods that are applied to tive function and its gradient at the solution
the KKT conditions, as well as the task of are vanishingly small. Under these conditions,
checking second-order KKT conditions, nec- one can show that the multipliers (λ, ν) also
essarily require information on second deriva- vanish and ∇xx L(x*, λ, ν) can be substituted
tives. (Note that second-order conditions are
104 Mathematics in Chemical Engineering
by ∇xx f (x*). This Gauss – Newton approxi- leads to better performance than those based
mation has been shown to be very efficient for on merit functions.
the solution of least-squares problems [175]. c) Regularization of the KKT matrix for the NLP
b) Line-search and trust-region methods are subproblem (e.g., in Equation 72) is essen-
used to promote convergence from poor start- tial for good performance of general purpose
ing points. These are commonly used with algorithms. For instance, to obtain a unique
the search directions calculated from NLP solution to Eqn. 72, active constraint gradi-
subproblems such as problem 68. In a trust- ents must be full rank, and the Hessian matrix,
region approach, the constraint, p ≤ ∆ is when projected into the null space of the ac-
added and the iteration step is taken if there tive constraint gradients, must be positive def-
is sufficient reduction of some merit func- inite. These properties may not hold far from
tion (e.g., the objective function weighted the solution, and corrections to the Hessian in
with some measure of the constraint viola- SQP may be necessary [176]. Regularization
tions). The size of the trust region ∆ is ad- methods ensure that subproblems like Eqns.
justed based on the agreement of the reduc- 68 and 72 remain well-conditioned; they in-
tion of the actual merit function compared clude addition of positive constants to the di-
to its predicted reduction from the subprob- agonal of the Hessian matrix to ensure its pos-
lem [183]. Such methods have strong global itive definiteness, judicious selection of active
convergence properties and are especially ap- constraint gradients to ensure that they are lin-
propriate for ill-conditioned NLPs. This ap- early independent, and scaling the subprob-
proach has been applied in the KNITRO code lem to reduce the propagation of numerical er-
[186]. Line-search methods can be more effi- rors. Often these strategies are heuristics built
cient on problems with reasonably good start- into particular NLP codes. While quite effec-
ing points and well-conditioned subproblems, tive, most of these heuristics do not provide
as in real-time optimization. Typically, once convergence guarantees for general NLPs.
a search direction is calculated from problem
68, or other related subproblem, a step size α Table 11 summarizes the characteristics of
∈ (0, 1] is chosen so that x k + αp leads to a a collection of widely used NLP codes. Much
sufficient decrease of a merit function. As a more information on widely available codes
recent alternative, a novel filter-stabilization can also be found on the NEOS server (www-
strategy (for both line-search and trust-region neos.mcs.anl.gov) and the NEOS Software
approaches) has been developed based on a Guide.
bicriterion minimization, with the objective
function and constraint infeasibility as com-
peting objectives [187]. This method often
netic modification, crossover or mutation, are and FOCUS developed at Boeing Corporation
used and the elements of the optimization vec- [205].
tor x are represented as binary strings. Crossover Direct search methods are easy to apply to
deals with random swapping of vector elements a wide variety of problem types and optimiza-
(among parents with highest objective function tion models. Moreover, because their termina-
values or other rankings of population) or any tion criteria are not based on gradient informa-
linear combinations of two parents. Mutation tion and stationary points, they are more likely
deals with the addition of a random variable to el- to favor the search for global rather than locally
ements of the vector. Genetic algorithms (GAs) optimal solutions. These methods can also be
have seen widespread use in process engineering adapted easily to include integer variables. How-
and a number of codes are available. A related ever, rigorous convergence properties to glob-
GA algorithm is described in [173]. ally optimal solutions have not yet been disco-
Derivative-Free Optimization (DFO). In the vered. Also, these methods are best suited for
past decade, the availability of parallel com- unconstrained problems or for problems with
puters and faster computing hardware and the simple bounds. Otherwise, they may have diffi-
need to incorporate complex simulation models culties with constraints, as the only options open
within optimization studies have led a number of for handling constraints are equality constraint
optimization researchers to reconsider classical elimination and addition of penalty functions for
direct search approaches. In particular, Dennis inequality constraints. Both approaches can be
and Torczon [202] developed a multidimen- unreliable and may lead to failure of the opti-
sional search algorithm that extends the simplex mization algorithm. Finally, the performance of
approach [196]. They note that the Nelder– direct search methods scales poorly (and often
Mead algorithm fails as the number of variables exponentially) with the number of decision vari-
increases, even for very simple problems. To ables. While performance can be improved with
overcome this, their multidimensional pattern- the use of parallel computing, these methods are
search approach combines reflection, expansion, rarely applied to problems with more than a few
and contraction steps that act as line search algo- dozen decision variables.
rithms for a number of linear independent search
directions. This approach is easily adapted to
parallel computation and the method can be 10.4. Global Optimization
tailored to the number of processors available.
Moreover, this approach converges to locally Deterministic optimization methods are avail-
optimal solutions for unconstrained problems able for nonconvex nonlinear programming
and exhibits an unexpected performance syn- problems of the form problem 52 that guaran-
ergy when multiple processors are used. The tee convergence to the global optimum. More
work of Dennis and Torczon [202] has spawned specifically, one can show under mild conditions
considerable research on analysis and code de- that they converge to an ε distance to the global
velopment for DFO methods. Moreover, Conn optimum on a finite number of steps. These
et al. [203] construct a multivariable DFO al- methods are generally more expensive than local
gorithm that uses a surrogate model for the NLP methods, and they require the exploitation
objective function within a trust-region method. of the structure of the nonlinear program.
Here points are sampled to obtain a well-defined Global optimization of nonconvex programs
quadratic interpolation model, and descent con- has received increased attention due to their
ditions from trust-region methods enforce con- practical importance. Most of the deterministic
vergence properties. A number of trust-region global optimization algorithms are based on spa-
methods that rely on this approach are reviewed tial branch-and-bound algorithm [206], which
in [203]. Moreover, a number of DFO codes divides the feasible region of continuous vari-
have been developed that lead to black-box op- ables and compares lower bound and upper
timization implementations for large, complex bound for fathoming each subregion. The one
simulation models. These include the DAKOTA that contains the optimal solution is found by
package at Sandia National Lab [204], eliminating subregions that are proved not to
http://endo.sandia.gov/DAKOTA/software.html contain the optimal solution.
Mathematics in Chemical Engineering 107
For nonconvex NLP problems, Quesada and containing the feasible region) into subregions
Grossmann [207] proposed a spatial branch- (see Fig. 42). Upper bounds on the objective
and-bound algorithm for concave separable, lin- function are computed over all subregions of
ear fractional, and bilinear programs using lin- the problem. In addition, lower bounds can be
ear and nonlinear underestimating functions derived from convex underestimators of the ob-
[208]. For nonconvex MINLP, Ryoo and Sahini- jective function and constraints for each subre-
dis [209] and later Tawarmalani and Sahini- gion. The algorithm then proceeds to eliminate
dis [210] developed BARON, which branches all subregions that have lower bounds that are
on the continuous and discrete variables with greater than the least upper bound. After this, the
bounds reduction method. Adjiman et al. [211, remaining regions are further partitioned to cre-
212] proposed the SMIN-αBB and GMIN-αBB ate new subregions and the cycle continues until
algorithms for twice-differentiable nonconvex the upper and lower bounds converge. Below
MINLPs. Using a valid convex underestima- we illustrate the specific steps of the algorithm
tion of general functions as well as for special for nonlinear programs that involve bilinear, lin-
functions, Adjiman et al. [213] developed the ear fractional, and concave separable terms [207,
αBB method, which branches on both the con- 214].
tinuous and discrete variables according to spe-
cific options. The branch-and-contract method Nonconvex NLP with Bilinear, Linear
[214] has bilinear, linear fractional, and concave Fractional, and Concave Separable Terms.
separable functions in the continuous variables Consider the following specific nonconvex NLP
and binary variables, uses bound contraction, problem,
and applies the outer-approximation (OA) algo-
xi xj + (i,j)∈LF0 bij xxi
rithm at each node of the tree. Smith and Pan- Min f (x) = (i,j)∈BL0 aij
x j
x∈S∩Ω0 ⊂Rn
problem 51 through the representation by the in- BL+ = {(i,j):(i,j)∈BL0 ∪BLk , aij >0
2
equalities fk (x) ≤0 and −fk (x) ≤0, provided or aijk >0, k∈K
hk (x) is separable.
BL− = {(i,j):(i,j)∈BL0 ∪BLk , aij <0
To obtain a lower bound LB (Ω) for the 2
or aijk <0, k∈K
global minimum of problem 80 over D∩Ω,
where Ω={x∈Rn :xL ≤x≤xU }⊆Ω0 , the fol- The inequalities 84 were first derived by Mc-
lowing problem is proposed: Cormick [208], and along with the inequalities
85 theoretically characterized by Al-Khayyal
fˆ (x,y,z) = (i,j)∈BL0 aij yij
Min and Falk [217, 218].
(x,y,z)
+ (i,j)∈LF0 bij zij + i∈C0 ĝi (xi ) +h (x)
c) z= {zij } is a vector of additional variables for
relaxing the linear fractional terms in problem
subject to 80; these variables are used in the following
inequalities:
fˆk (x,y,z) = (i,j)∈BLk aijk yij + (i,j)∈LFk bijk zij
zij ≥ xLi +xU
i
1
− 1
(i,j)∈LF +
xj xj xL
j
+ i∈Ck ĝi,k (xi ) +hk (x) ≤0 k∈K (86)
zij ≥ xUi +xL
i
1
− 1
(i,j)∈LF +
xj xj xU
(x,y,z) ∈T (Ω) ⊂Rn ×Rn1 ×Rn2 j
* 2
n1 n2 x + x L xU
x∈S∩Ω⊂Rn , y∈R+ , z∈R+ , 1 i i i
(81) zij ≥ * * (i,j)∈LF + (87)
xj xL + xU
i i
where the functions and sets are defined as fol-
lows:
1
zij ≤ xU L L L
j xi −xi xj +xi xj (i,j) ∈LF −
xL U
j xj
a) ĝi (xi ) and ĝi,k (xi ) are the convex envelopes 1
zij ≤ xL U U U
j xi −xi xj +xi xj (i,j) ∈LF −
for the univariate functions over the domain xL U
j xj
xi ∈[xL U
i ,xi ] [216]: (88)
Note that the underestimating problem 81 is This basic concept in spatial branch-and-
a linear program if LF + =∅. During the ex- bound for global optimization is as follows.
ecution of the spatial branch-and-bound algo- Bounds are related to the calculation of upper
rithm, problem (NBNC) is solved initially over and lower bounds. For the former, any feasi-
M (Ω0 ) (root node of the branch-and-bound ble point or, preferably, a locally optimal point
tree). If a better approximation is required, in the subregion, can be used. For the lower
M (Ω0 ) is refined by partitioning Ω0 into two bound, convex relaxations of the objective and
smaller hyperrectangles Ω01 and Ω02 , and two constraint function are derived such as in prob-
children nodes are created with relaxed feasible lem 81. The refining step deals with the construc-
regions given by M (Ω01 ) and M (Ω02 ). Prob- tion of partitions in the domain and further parti-
lem 81 might be regarded as a basic underesti- tioning them during the search process. Finally,
mating program for the general problem 80. In the selection step decides on the order of ex-
some cases, however, it is possible to develop ad- ploring the open subregions. Thus, the feasible
ditional convex estimators that might strengthen region and the objective function are replaced
the underestimating problem. See, for instance, by convex envelopes to form relaxed problems.
the projections proposed by Quesada and Gross- Solving these convex relaxed problems leads to
mann [207], the reformulation–linearization global solutions that are lower bounds to the
technique by Sherali and Alameddine [220], NLP in the particular subregion. Finally, we see
and the reformulation–convexification approach that gradient-based NLP solvers play an impor-
by Sherali and Tuncbilek [221]. tant role in global optimization algorithms, as
The Set of Branching Variables. A set of they often yield the lower and upper bounds for
branching variables, characterized by the index the subregions. The following spatial branch-
set BV (Ω) defined below, is determined by con- and-bound global optimization algorithm can
sidering the optimal solution (x̂,ŷ,ẑ)Ω of the un- therefore be given by the following steps:
derestimating problem: 0. Initialize algorithm: calculate upper bound by
BV (Ω) ={i,j: |ŷij −x̂i x̂j | =ζl or |ẑij −x̂i /x̂j | =ζl obtaining a local solution to problem 80. Cal-
culate a lower bound solving problem 81 over
or gi (x̂i ) −ĝi (x̂i ) =ζl or gi,k (x̂i ) −ĝi,k (x̂i ) =ζl ,
the entire (relaxed) feasible region Ω0 .
for i∈I, j∈J, k∈K, l∈L} (90)
For iteration k with a set of partitions Ωk,j
where, for a prespecified number ln , and bounds in each subregion OLBj and OUBj :
L={1,2,. . .,ln } and ζ1 is the magnitude of the
largest approximation error for a nonconvex 1) Bound: Define best upper bound: OUB =
term in problem 80 evaluated at (x̂,ŷ,ẑ)Ω : Minj OUBj and delete (fathom) all subre-
gions j with lower bounds OLBj ≥ OUB. If
ξ1 =Max [|ŷij −x̂i x̂j | , |ẑij −x̂i /x̂j | , gi (x̂i ) −ĝi (x̂i ) , OLBj ≥ OUB − ε, stop.
" 2) Refine: Divide the remaining active subre-
gi,k (x̂i ) −ĝi,k (x̂i ) gions into partitiions Ωk,j1 and Ωk,j2 . (Sev-
eral branching rules are available for this
i∈I,j∈J,k∈K
step.)
Similarly, we define ξl <ξl−1 with l∈L\ {1} as 3) Select: Solve the convex NLP 81 in the new
the l-th largest magnitude for an approximation partitions to obtain OLBj 1 and OLBj 2 . Delete
error; for instance, ξ2 <ξ1 is the second largest partition if no feasible solution.
magnitude for an approximation error. Note that 4) Update: Obtain upper bounds, OUBj 1 and
in some cases it might be convenient to intro- OUBj 2 to new partitions if present. Set k =
duce weights in the determination of BV (Ω) k + 1, update partition sets and go to step 1.
in order to scale differences in the approxima- Example: To illustrate the spatial branch-
tion errors or to induce preferential branching and-bound algorithm, consider the global solu-
schemes. This might be particularly useful in tion of:
applications where specific information can be
exploited by imposing an order of precedence Min f (x) =5/2 x4 −20 x3 +55 x2 −57 x
(91)
on the set of complicating variables. s.t. 0.5≤x≤2.5
110 Mathematics in Chemical Engineering
(MILP) are LP-based branch-and-bound meth- MILPs that can be solved with efficient special-
ods, which are enumeration methods that solve purpose methods are the knapsack problem, the
LP subproblems at each node of the search tree. set-covering and set-partitioning problems, and
This technique was initially conceived by Land the traveling salesman problem. See [237] for a
and Doig [229], Balas [230], and later formal- detailed treatment of these problems.
ized by Dakin, [231]. Cutting-plane techniques, The branch-and-bound algorithm for solv-
which were initially proposed by Gomory [232], ing MILP problems [231] is similar to the spa-
and consist of successively generating valid in- tial branch-and-bound method of the previous
equalities that are added to the relaxed LP, have section that explores the search space. As seen
received renewed interest through the works of in Figure 45, binary variables are successively
Crowder et al. [233], van Roy and Wolsey [234], fixed to define the search tree and a number of
and especially the lift-and-project method of bounding properties are exploited in order to
Balas et al. [228]. A recent review of branch- fathom nodes in to avoid exhaustive enumera-
and-cut methods can be found in [235]. Finally, tion of all the nodes in the tree.
Benders decomposition [236] is another tech-
nique for solving MILPs in which the problem is
successively decomposed into LP subproblems
for fixed 0–1 and a master problem for updating
the binary variables.
y variables take on 0–1 values, the MILP prob- commonly used: branch on the dichotomy 0–1
lem is solved, and no further search is required. at each node (i.e., like breadth-first), but expand
If at least one of the binary variables yields a as in depth-first. Additional description of these
fractional value, the solution of the LP relax- strategies can be found in [237].
ation yields a lower bound to problem 93. The Example: To illustrate the branch-and-bound
search then consists of branching on that node approach, we consider the MILP:
by fixing a particular binary variable to 0 or 1,
Min Z=x+y1 +2y2 +3y3
and the corresponding restricted LP relaxations
are solved that in turn yield a lower bound for s.t.−x+3y1 +y2 +2y3 ≤0
any of their descendant nodes. In particular, the (94)
−4y1 −8y2 −3y3 ≤−10
following properties are exploited in the branch-
and-bound search: x≥0,y1 ,y2 ,y3 ={0,1}
• Any node (initial, intermediate, leaf node) that The solution to problem 94 is given by x = 4, y1 =
leads to feasible LP solution corresponds to a 1, y2 = 1, y3 = 0, and Z = 7. Here we use a depth-
valid upper bound to the solution of the MILP first strategy and branch on the variables closest
problem 93. to zero or one. Figure 45 shows the progress of
• Any intermediate node with an infeasible LP the branch-and-bound algorithm as the binary
solution has infeasible leaf nodes and can be variables are selected and the bounds are up-
fathomed (i.e., all remaining children of this dated. The sequence numbers for each node in
node can be eliminated). Figure 45 show the order in which they are pro-
• If the LP solution at an intermediate node is cessed. The grayed partitions correspond to the
not less than an existing integer solution, then deleted nodes and at termination of the algo-
the node can be fathomed. rithm we see that Z = 7 and an integer solution
These properties lead to pruning of the nodes is obtained at an intermediate node where coin-
in the search tree. Branching then continues in cidentally y3 = 0.
the tree until the upper and lower bounds con- Mixed integer linear programming (MILP)
verge. methods and codes have been available and
The basic concepts outlined above lead to a applied to many practical problems for more
branch-and-bound algorithm with the following than twenty years (e.g., [237]. The LP-based
features. LP solutions at intermediate nodes are branch-and-bound method [231] has been im-
relatively easy to calculate since they can be ef- plemented in powerful codes such as OSL,
fectively updated with the dual simplex method. CPLEX, and XPRESS. Recent trends in MILP
The selection of binary variables for branching, include the development of branch-and-price
known as the branching rule, is based on a num- [238] and branch-and-cut methods such as the
ber of different possible criteria; for instance, lift-and-project method [228] in which cutting
choosing the fractional variable closest to 0.5, or planes are generated as part of the branch-and-
the one involving the largest of the smallest pseu- bound enumeration. See also [235] for a recent
docosts for each fractional variable. Branching review on MILP. A description of several MILP
strategies to navigate the tree take a number of solvers can also be found in the NEOS Software
forms. More common depth-first strategies ex- Guide.
pand the most recent node to a leaf node or infea- Mixed Integer Nonlinear Programming The
sible node and then backtrack to other branches most basic form of an MINLP problem when
in the tree. These strategies are simple to pro- represented in algebraic form is as follows:
gram and require little storage of past nodes. On min z=f (x,y)
the other hand, breadth-first strategies expand
all the nodes at each level of the tree, select the s.t. gj (x,y) ≤0 j∈J (P1) (95)
node with the lowest objective function, and then x∈X, y∈Y
proceed until the leaf nodes are reached. Here,
more storage is required, but generally fewer where f (·), g(·) are convex, differentiable func-
nodes are evaluated than in depth-first search. tions, J is the index set of inequalities, and
In practice, a combination of both strategies is x and y are the continuous and discrete
Mathematics in Chemical Engineering 113
yi ≤αki i∈IF
k
L
yi ≥βik i∈IF
k
U
The convexity of the nonlinear functions is BB method starts by solving first the continu-
exploited by replacing them with supporting hy- ous NLP relaxation. If all discrete variables take
perplanes, which are generally, but not necessar- integer values the search is stopped. Otherwise,
ily, derived at the solution of the NLP subprob- a tree search is performed in the space of the
lems. In particular, the new values yK (or (x K , integer variables yi , i ∈I. These are successively
yK ) are obtained from a cutting-plane MILP fixed at the corresponding nodes of the tree, giv-
problem that is based on the K points (x k , yk ), k ing rise to relaxed NLP subproblems of the form
= 1. . .K, generated at the K previous steps: (NLP1) which yield lower bounds for the sub-
problems in the descendant nodes. Fathoming of
Min ZLK =α
nodes occurs when the lower bound exceeds the
st α≥f (xk ,y k )
current upper bound, when the subproblem is in-
x−xk
feasible, or when all integer variables yi take on
T
+∇f (xk ,y k )
discrete values. The last-named condition yields
y−y k
k=1,. . .K an upper bound to the original problem.
gj (xk ,y k )
k
x−x
k k T k
+∇gj (x ,y ) ≤0 j∈J
y−y k
x∈X, y∈Y
(99)
Algorithms. The different methods can be The BB method is generally only attractive if
classified according to their use of the subprob- the NLP subproblems are relatively inexpensive
lems 96–98, and the specific specialization of to solve, or when only few of them need to be
the MILP problem 99, as seen in Figure 47. In solved. This could be either because of the low
the GBD and OA methods (case b), as well in the dimensionality of the discrete variables, or be-
LP/NLP-based branch-and-bound mehod (case cause the integrality gap of the continuous NLP
d), problem 98 is solved if infeasible subprob- relaxation of 95 is small.
lems are found. Each of the methods is explained
next in terms of the basic subproblems. Outer Approximation [245 – 247]. The
OA method arises when NLP subproblems 97
and MILP master problems 99 with J k = J are
Branch and Bound. While the earlier work
solved successively in a cycle of iterations to
in branch and bound (BB) was aimed at lin-
generate the points (x k , yk ). Since the master
ear problems [231] this method can also be ap-
problem 99 requires the solution of all feasi-
plied to nonlinear problems [239 – 243]. The
ble discrete variables yk , the following MILP
Mathematics in Chemical Engineering 115
relaxation is considered, assuming that the so- infeasible the choice of the previous 0–1 values
lution of K different NLP subproblems (K = generated at the K previous iterations [245]:
|KFS ∪ KIS|), where KFS is a set of solutions
yi − yi ≤ B k −1 k=1,. . .K (101)
from problem 97, and KIS set of solutions from i∈B k i∈N k
problem 98 is available: where B k = i|yik =1 , N k = i|yik =0 , k=1,
Min ZLK =α . . .K. This cut becomes very weak as the dimen-
sionality of the 0–1 variables increases. How-
α≥f (xk ,y k ) ever, it has the useful feature of ensuring that
st
new 0–1 values are generated at each major it-
x−xk
T
+∇f (xk ,y k )
eration. In this way the algorithm will not return
y−y k to a previous integer point when convergence is
k=1,. . .K
gj (xk ,y k )
achieved. Using the above integer cut the termi-
nation takes place as soon as ZLK ≥UBK .
x−xk
T
The OA algorithm trivially converges in one
k k
+∇gj (x ,y ) ≤0 j∈J
k
y−y iteration if f (x,y) and g(x,y) are linear. This prop-
x∈X, y∈Y erty simply follows from the fact that if f (x,y)
(100) and g(x,y) are linear in x and y the MILP master
problem 100 is identical to the original problem
Given the assumption on convexity of the func- 95. It is also important to note that the MILP
tions f (x,y) and g(x,y), it can be proved that the master problem need not be solved to optimal-
solution of problem 100 ZLk corresponds to a ity.
lower bound of the solution of problem 95. Note
that this property can be verified in Figure 46. Generalized Benders Decomposition
Also, since function linearizations are accumu- (GBD) [244]. The GBD method [250] is
lated as iterations proceed, the master problems similar to the outer-approximation method.
100 yield a nondecreasing sequence of lower The difference arises in the definition of
bounds ZL1 . . .≤ZLk . . .ZLk since linearizations are the MILP master problem 99. In the GBD
accumulated as iterations k proceed. method only active inequalities are considered
The OA algorithm as proposed by Duran and J k = {j| gj (xk , y k )=0} and the set x ∈ X is
Grossmann consists of performing a cycle of ma- disregarded. In particular, consider an outer-
jor iterations, k = 1,..K, in which problem 97 is approximation given at a given point (x k , yk )
solved for the corresponding yK , and the relaxed
x−xk
MILP master problem 100 is updated and solved T
α≥f xk ,y k +∇f xk ,y k
with the corresponding function linearizations
y−y k
at the point (x k ,yk ) for which the correspond-
x−xk
(102)
ing subproblem NLP2 is solved. If feasible, the g xk ,y k +∇g xk ,y k
T ≤0
solution to that problem is used to construct the y−y k
first MILP master problem; otherwise a feasi-
bility problem 98 is solved to generate the cor- where for a fixed yk the point x k corresponds
responding continuous point [247]. The initial to the optimal solution to problem 97. Making
MILP master problem 100 then generates a new use of the Karush – Kuhn – Tucker conditions
vector of discrete variables. The subproblems and eliminating the continuous variables x, the
97 yield an upper bound that is used inequalities in 102 can be reduced as follows
tok define [248]:
the best current solution, UBk =min ZU . The
k T
cycle of iterations is continued until this upper α≥f xk ,y k +∇y f xk ,y k y−y k +
bound and the lower bound of the relaxed mas- T T
ter problem ZLk are within a specified tolerance. µk g xk ,y k +∇y g xk ,y k y−y k (103)
One way to avoid solving the feasibility problem
98 in the OA algorithm when the discrete vari- which is the Lagrangian cut projected in the y-
ables in problem 95 are 0–1, is to introduce the space. This can be interpreted as a surrogate con-
following integer cut whose objective is to make straint of the equations in 102, because it is ob-
tained as a linear combination of these.
116 Mathematics in Chemical Engineering
For the case when there is no feasible solu- added per iteration is equal to the number of non-
tion to problem 97, then if the point x k is ob- linear constraints plus the nonlinear objective.
tained from the feasibility subproblem (NLPF), If problem 95 has zero integrality gap, the
the following feasibility cut projected in y can GBD algorithm converges in one iteration once
be obtained using a similar procedure. the optimal (x*, y*) is found [252]. This prop-
T T erty implies that the only case in which one can
λk g xk ,y k +∇y g xk ,y k y−y k ≤0 expect the GBD method to terminate in one iter-
(104) ation is that in which the initial discrete vector is
the optimum, and when the objective value of the
In this way, problem 99 reduces to a problem NLP relaxation of problem 95 is the same as the
projected in the y-space: objective of the optimal mixed-integer solution.
Min ZLK =α
T Extended Cutting Plane (ECP) [249]. The
st α ≥ f xk ,y k +∇y f xk ,y k y−y k +
(105) ECP method, which is an extension of Kelley’s
T T
µk g xk ,y k +∇y g xk ,y k y−y k cutting-plane algorithm for convex NLP [253],
k∈KF S does not rely on the use of NLP subproblems
T
T
and algorithms. It relies only on the iterative so-
λk g xk ,y k +∇y g xk ,y k y−y k ≤0
lution of problem 99 by successively adding a
k∈KIS linearization of the most violated constraint at
x∈X, α∈R1
the predicted point (x k , yk ):
# # %%
where KFS is the set of feasible subproblems J k = ĵ∈arg max gj xk ,y k
j∈J
97, and KIS the set of infeasible subproblems
whose solution is given by problem 98. Also Convergence is achieved when the maximum
|KFS ∪ KIS| = K. Since master problem 105 can constraint violation lies within the specified tol-
be derived from master problem 100, in the con- erance. The optimal objective value of problem
text of problem 95, GBD can be regarded as a 99 yields a nondecreasing sequence of lower
particular case of the outer-approximation al- bounds. It is of course also possible to either
gorithm. In fact one can prove that given the add to problem 99 linearizatons of all the vio-
same set of K subproblems, the lower bound lated constraints in the set J k , or linearizations
predicted by the relaxed master problem 100 is of all the nonlinear constraints j ∈ J. In the ECP
greater than or equal to that predicted by the re- method the objective must be defined as a linear
laxed master problem 105 [245]. This proof fol- function, which can easily be accomplished by
lows from the fact that the Lagrangian and fea- introducing a new variable to transfer nonlinear-
sibility cuts 103 and 104 are surrogates of the ities in the objective as an inequality.
outer-approximations 102. Given the fact that Note that since the discrete and continuous
the lower bounds of GBD are generally weaker, variables are converged simultaneously, the ECP
this method commonly requires a larger num- method may require a large number of itera-
ber of cycles or major iterations. As the num- tions. However, this method shares with the OA
ber of 0–1 variables increases this difference method Property 2 for the limiting case when all
becomes more pronounced. This is to be ex- the functions are linear.
pected since only one new cut is generated per
iteration. Therefore, user-supplied constraints LP/NLP-Based Branch and Bound [248].
must often be added to the master problem to This method is similar in spirit to a branch-and-
strengthen the bounds. Also, it is sometimes pos- cut method, and avoids the complete solution of
sible to generate multiple cuts from the solution the MILP master problem (M-OA) at each major
of an NLP subproblem in order to strengthen the iteration. The method starts by solving an initial
lower bound [251]. As for the OA algorithm, NLP subproblem, which is linearized as in (M-
the trade-off is that while it generally predicts OA). The basic idea consists then of performing
stronger lower bounds than GBD, the computa- an LP-based branch-and-bound method for (M-
tional cost for solving the master problem (M- OA) in which NLP subproblems 97 are solved at
OA) is greater, since the number of constraints those nodes in which feasible integer solutions
Mathematics in Chemical Engineering 117
are found. By updating the representation of the is obtained. This, however, comes at the price
master problem in the current open nodes of the of having to solve an MIQP instead of an MILP
tree with the addition of the corresponding lin- at each iteration.
earizations, the need to restart the tree search is
avoided.
This method can also be applied to the GBD
and ECP methods. The LP/NLP method com-
monly reduces quite significantly the number of
nodes to be enumerated. The trade-off, however,
is that the number of NLP subproblems may in-
crease. Computational experience has indicated
that often the number of NLP subproblems re-
mains unchanged. Therefore, this method is bet-
ter suited for problems in which the bottleneck
corresponds to the solution of the MILP master
problem. Leyffer [254] has reported substantial
savings with this method.
Example: Consider the following MINLP
problem, whose objective function and con-
straints contain nonlinear convex terms
Min Z=y1 +1.5y2 +0.5y3 +x21 +x22
Figure 48. Progress of iterations with OA and GBD meth-
s.t. (x1 −2)2 −x2 ≤0 ods, and number of subproblems for the BB, OA, GBD, and
ECP methods.
x1 −2y1 ≥0
The master problem 100 can involve a rather
x1 −x2 −4 (1−y2 ) ≥0
large number of constraints, due to the accumu-
x2 −y2 ≥0 (106) lation of linearizations. One option is to keep
only the last linearization point, but this can
x1 +x2 ≥3y3
lead to nonconvergence even in convex prob-
y1 +y2 +y3 ≥1 lems, since then the monotonic increase of the
lower bound is not guaranteed. As shown [248],
0≤x1 ≤4,0≤x2 ≤4
linear approximations to the nonlinear objective
y1 ,y2 ,y3 =0,1 and constraints can be aggregated with an MILP
master problem that is a hybrid of the GBD and
The optimal solution of this problem is given by OA methods.
y1 = 0, y2 = 1, y3 = 0, x 1 = 1, x 2 = 1, Z = 3.5. Fig- For the case when linear equalities of the form
ure 48 shows the progress of the iterations with h(x, y) = 0 are added to problem 95 there is no
the OA and GBD methods, while the table lists major difficulty, since these are invariant to the
the number of NLP and MILP subproblems that linearization points. If the equations are nonlin-
are solved with each of the methods. For the case ear, however, there are two difficulties. First, it
of the MILP problems the total number of LPs is not possible to enforce the linearized equali-
solved at the branch-and-bound nodes are also ties at K points. Second, the nonlinear equations
reported. may generally introduce nonconvexities, unless
they relax as convex inequalities [255]. Kocis
Extensions of MINLP Methods. Exten- and Grossmann [256] proposed an equality re-
sions of the methods described above include laxation strategy in which the nonlinear equali-
a quadratic approximation to (RM-OAF) [247] ties are replaced by the inequalities
using an approximation of the Hessian matrix.
T x−xk
The quadratic approximations can help to re- k
k k
duce the number of major iterations, since an T ∇h x ,y ≤0 (107)
y−y k
improved representation of the continuous space
118 Mathematics in Chemical Engineering
0≤x≤xU ,c≥0
Constraint Programming. Constraint pro-
gramming (CP) [268, 269] is a relatively new
where hjk (x) are assumed to be convex and modeling and solution paradigm that was orig-
bounded over x. The convex hull relaxation of inally developed to solve feasibility problems,
disjunction 112 [242] is given as follows: but it has been extended to solve optimiza-
jk ,
tion problems as well. Constraint programming
x= j∈Jk v c= j∈J λjk γjk is very expressive, as continuous, integer, and
0≤v jk ≤λjk xU
jk , j∈Jk
Boolean variables are permitted and, moreover,
variables can be indexed by other variables. Con-
j∈Jk λjk =1, 0≤λjk ≤1, j∈Jk (CH) (113)
straints can be expressed in algebraic form (e.g.,
λjk hjk v jk /λ jk ≤0, j∈Jk h(x) ≤ 0), as disjunctions (e.g., [A1 x ≤ b1 ] ∨ A2 x
≤ b2 ]), or as conditional logic statements (e.g., If
x,c,v jk ≥0, j∈J k
g(x) ≤ 0 then r(x) ≤ 0). In addition, the language
where vjk are disaggregated variables that are as- can support special implicit functions such as
signed to each term of the disjunction k ∈ K, and the all-different (x 1 , x 2 , . . .x n ) constraint for
Mathematics in Chemical Engineering 121
assigning different values to the integer vari- discretization that converts the original continu-
ables x 1 , x 2 , . . .x n . The language consists of ous time problem into a discrete problem. Early
C ++ procedures, although the recent trend has solution strategies, known as indirect methods,
been to provide higher level languages such as were focused on solving the classical variational
OPL. Other commercial CP software packages conditions for optimality. On the other hand,
include ILOG Solver [278], CHIP [279], and methods that discretize the original continuous
ECLiPSe [280]. time formulation can be divided into two cate-
gories, according to the level of discretization.
Here we distinguish between the methods that
10.6. Dynamic Optimization discretize only the control profiles (partial dis-
cretization) and those that discretize the state and
Interest in dynamic simulation and optimiza- control profiles (full discretization). Basically,
tion of chemical processes has increased sig- the partially discretized problem can be solved
nificantly during the last two decades. Chem- either by dynamic programming or by applying a
ical processes are modeled dynamically using nonlinear programming (NLP) strategy (direct-
differential-algebraic equations (DAEs), con- sequential). A basic characteristic of these meth-
sisting of differential equations that describe the ods is that a feasible solution of the DAE system,
dynamic behavior of the system, such as mass for given control values, is obtained by integra-
and energy balances, and algebraic equations tion at every iteration of the NLP solver. The
that ensure physical and thermodynamic rela- main advantage of these approaches is that, for
tions. Typical applications include control and the NLP solver, they generate smaller discrete
scheduling of batch processes; startup, upset, problems than full discretization methods.
shut-down, and transient analysis; safety stud- Methods that fully discretize the continuous
ies; and the evaluation of control schemes. We time problem also apply NLP strategies to solve
state a general differential-algebraic optimiza- the discrete system and are known as direct-
tion problem 115 as follows: simultaneous methods. These methods can use
different NLP and discretization techniques, but
Min Φ (z (tf ) ;y (tf ) ;u (tf ) ;tf ;p) the basic characteristic is that they solve the DAE
s.t. F (dz/dt; z (t) ; u (t) ; t; p) =0, z (0) =z0 system only once, at the optimum. In addition,
they have better stability properties than partial
Gs [z (ts ) ; y (ts ) ; u (ts ) ; ts ; p)] =0 discretization methods, especially in the pres-
z L ≤z (t) ≤xU ence of unstable dynamic modes. On the other
(115) hand, the discretized optimization problem is
y L ≤y (t) ≤y U larger and requires large-scale NLP solvers, such
uL ≤u (t) ≤y U as SOCS, CONOPT, or IPOPT.
With this classification we take into account
pL ≤p≤pU the degree of discretization used by the differ-
ttf ≤tf ≤tU
f
ent methods. Below we briefly present the de-
scription of the variational methods, followed
where Φ is a scalar objective function at final by methods that partially discretize the dynamic
time t f , and F are DAE constraints, G s additional optimization problem, and finally we consider
point conditions at times t s , z(t) differential state full discretization methods for problem 115.
profile vectors, y(t) algebraic state profile vec-
tors, u(t) control state profile vectors, and p is a Variational Methods. These methods are
time-independent parameter vector. based on the solution of the first-order necessary
We assume, without loss of generality, that conditions for optimality that are obtained from
the index of the DAE system is one, consistent Pontryagin’s maximum principle [281, 282]. If
initial conditions are available, and the objective we consider a version of problem 115 without
function is in the above Mayer form. Otherwise, bounds, the optimality conditions are formulated
it is easy to reformulate problems to this form. as a set of DAEs:
Problem 115 can be solved either by the vari- ∂F (z,y,u,p,t) ′ ∂H ∂F (z,y,u,p,t)
ational approach or by applying some level of λ= = λ (116a)
∂z ′ ∂z ∂z
122 Mathematics in Chemical Engineering
The main idea is to use an algorithm for con- but path constraints for the states may not be
strained dynamic simulation, so that any admis- satisfied between grid points. This problem can
sible combination of the control parameters pro- be avoided by applying penalty techniques to
duces an initial value problem that is feasible enforce feasibility, like the ones used in the se-
with respect to the path constraints. The algo- quential methods.
rithm proceeds by detecting activation and de- The resulting NLP is solved using SQP-type
activation of the constraints during the solution, methods, as described above. At each SQP iter-
and solving the resulting high-index DAE sys- ation, the DAEs are integrated in each stage and
tem and their related sensitivities. objective and constraint gradients with respect to
p, zi , and νi are obtained using sensitivity equa-
Full Discretization. Full discretization tions, as in problem 118. Compared to sequen-
methods explicitly discretize all the variables tial methods, the NLP contains many more vari-
of the DAE system and generate a large-scale ables, but efficient decompositions have been
nonlinear programming problem that is usually proposed [290] and many of these calculations
solved with a successive quadratic programming can be performed in parallel.
(SQP) algorithm. These methods follow a simul- In collocation methods, the continuous time
taneous approach (or infeasible path approach); problem is transformed into an NLP by ap-
that is, the DAE system is not solved at each iter- proximating the profiles as a family of poly-
ation; it is only solved at the optimum point. Be- nomials on finite elements. Various polynomial
cause of the size of the problem, special decom- representations are used in the literature, in-
position strategies are used to solve the NLP ef- cluding Lagrange interpolation polynomials for
ficiently. Despite this characteristic, the simul- the differential and algebraic profiles [291]. In
taneous approach has advantages for problems [191] a Hermite–Simpson collocation form is
with state variable (or path) constraints and for used, while Cuthrell and Biegler [292] and
systems where instabilities occur for a range of Tanartkit and Biegler [293] use a monomial
inputs. In addition, the simultaneous approach basis for the differential profiles. All of these re-
can avoid intermediate solutions that may not presentations stem from implicit Runge–Kutta
exist, are difficult to obtain, or require excessive formulae, and the monomial representation is
computational effort. There are mainly two dif- recommended because of smaller condition
ferent approaches to discretize the state variables numbers and smaller rounding errors. Control
explicitly, multiple shooting [289, 290] and col- and algebraic profiles, on the other hand, are ap-
location on finite elements [181, 191, 291]. proximated using Lagrange polynomials.
With multiple shooting, time is discretized Discretizations of problem 115 using colloca-
into P stages and control variables are tion formulations lead to the largest NLP prob-
parametrized using a finite set of control parame- lems, but these can be solved efficiently using
ters in each stage, as with partial discretization. large-scale NLP solvers such as IPOPT and by
The DAE system is solved on each stage, i = exploiting the structure of the collocation equa-
1,. . .P and the values of the state variables z(t i ) tions. Biegler et al. [181] provide a review of
are chosen as additional unknowns. In this way a dynamic optimization methods using simultane-
set of relaxed, decoupled initial value problems ous methods. These methods offer a number of
(IVP) is obtained: advantages for challenging dynamic optimiza-
tion problems, which include:
F (dz/dt; z (t) ; y (t) ; νi ; p) =0,
• Control variables can be discretized at the
t∈ [ti−1 , ti ] , z (ti−1 ) =zi (119) same level of accuracy as the differential and
algebraic state variables. The KKT conditions
zi+1 −z (ti ; zi ; νi ; p) =0, i=1, . . .P −1
of the discretized problem can be shown to be
Note that continuity among stages is treated consistent with the variational conditions of
through equality constraints, so that the final so- problem 115. Finite elements allow for dis-
lution satisfies the DAE system. With this ap- continuities in control profiles.
proach, inequality constraints for states and con- • Collocation formulations allow problems
trols can be imposed directly at the grid points, with unstable modes to be handled in an
124 Mathematics in Chemical Engineering
efficient and well-conditioned manner. The sified into two broad areas: optimization mod-
NLP formulation inherits stability properties eling platforms and simulation platforms with
of boundary value solvers. Moreover, an ele- optimization.
mentwise decomposition has been developed Optimization modeling platforms provide
that pins down unstable modes in problem general purpose interfaces for optimization al-
115. gorithms and remove the need for the user to
• Collocation formulations have been proposed interface to the solver directly. These platforms
with moving finite elements. This allows allow the general formulation for all problem
the placement of elements both for accurate classes discussed above with direct interfaces
breakpoint locations of control profiles as well to state of the art optimization codes. Three re-
as accurate DAE solutions. presentative platforms are GAMS (General Al-
gebraic Modeling Systems), AMPL (A Mathe-
Dynamic optimization by collocation meth-
matical Programming Language), and AIMMS
ods has been used for a wide variety of pro-
(Advanced Integrated Multidimensional Model-
cess applications including batch process opti-
ing Software). All three require problem-model
mization, batch distillation, crystallization, dy-
input via a declarative modeling language and
namic data reconciliation and parameter estima-
provide exact gradient and Hessian information
tion, nonlinear model predictive control, poly-
through automatic differentiation strategies. Al-
mer grade transitions and process changeovers,
though possible, these platforms were not de-
and reactor design and synthesis. A review of
signed to handle externally added procedural
this approach can be found in [294].
models. As a result, these platforms are best ap-
plied on optimization models that can be devel-
oped entirely within their modeling framework.
10.7. Development of Optimization Nevertheless, these platforms are widely used
Models for large-scale research and industrial applica-
tions. In addition, the MATLAB platform allows
The most important aspect of a successful op- the flexible formulation of optimization models
timization study is the formulation of the op- as well, although it currently has only limited ca-
timization model. These models must reflect pabilities for automatic differentiation and lim-
the real-world problem so that meaningful op- ited optimization solvers. More information on
timization results are obtained, and they also these and other modeling platforms can be found
must satisfy the properties of the problem class. on the NEOS server www-neos.mcs.anl.gov
For instance, NLPs addressed by gradient-based Simulation platforms with optimization are
methods require functions that are defined in the often dedicated, application-specific modeling
variable domain and have bounded and continu- tools to which optimization solvers have been
ous first and second derivatives. In mixed integer interfaced. These lead to very useful optimiza-
problems, proper formulations are also needed tion studies, but because they were not originally
to yield good lower bounds for efficient search. designed for optimization models, they need to
With increased understanding of optimization be used with some caution. In particular, most of
methods and the development of efficient and these platforms do not provide exact derivatives
reliable optimization codes, optimization prac- to the optimization solver; often they are approx-
titioners now focus on the formulation of opti- imated through finite difference. In addition, the
mization models that are realistic, well-posed, models themselves are constructed and calcu-
and inexpensive to solve. Finally, convergence lated through numerical procedures, instead of
properties of NLP, MILP, and MINLP solvers through an open declarative language. Examples
require accurate first (and often second) deriva- of these include widely used process simulators
tives from the optimization model. If these con- such as Aspen/Plus, PRO/II, and Hysys. More
tain numerical errors (say, through finite differ- recent platforms such as Aspen Custom Mod-
ence approximations) then performance of these eler and gPROMS include declarative models
solvers can deteriorate considerably. As a result and exact derivatives.
of these characteristics, modeling platforms are For optimization tools linked to procedural
essential for the formulation task. These are clas- models, reliable and efficient automatic differ-
Mathematics in Chemical Engineering 125
The value n − 1 is used in the denominator be- error must be present and one error cannot pre-
cause the deviations from the sample average dominate (unless it is normally distributed). The
must total zero: normal distribution function is calculated eas-
n ily; of more value are integrals of the function,
(yi −y) = 0 which are given in Table 12; the region of inter-
i=1 est is illustrated in Figure 50.
Thus, knowing n − 1 values of yi − ȳ and the
fact that there are n values automatically gives
the n-th value. Thus, only n − 1 degrees of free-
dom ν exist. This occurs because the unknown
mean E ( y) is replaced by the sample mean y
derived from the data.
If data are taken consecutively, running totals
can be kept to permit calculation of the mean and
variance without retaining all the data:
n n n
(yi −y)2 = y12 −2y yi +(y)2
i=1 i=1 i=1
n
y=
yi /n Figure 49. Frequency of occurrence of different scores
i=1
Thus,
n n
n, y12 , and yi
i=1 i=1
* Table gives the probability F that a random variable will fall in the shaded region of Figure 50. For a more complete table (in slightly
different form), see [23, Table 26.1]. This table is obtained in Microsoft Excel with the function NORMDIST(z,0,1,1)-0.5.
* Table gives t values such that a random variable will fall in the shaded region of Figure 52 with probability α. For a one-sided test the
confidence limits are obtained for α/2. For a more complet table (in slightly different form), see [23, Table 26.10]. This table is obtained in
Microsoft Excel with the function TINV(α,ν).
More generally, consider the random variables mean of some property could be measured be-
y1 , y2 , . . . with means E ( y1 ), E ( y2 ), . . . and fore and after the change; if these differ, does
variances σ 2 ( y1 ), σ 2 ( y2 ), . . . and correlation it mean the process modification caused it, or
coefficients ̺ij . The variable could the change have happened by chance? This
is a statistical decision. A hypothesis H 0 is de-
Y = α1 y1 +α2 y2 +. . . fined; if it is true, action A must be taken. The
has a mean reverse hypothesis is H 1 ; if this is true, action
B must be taken. A correct decision is made if
E (Y ) = α1 E (y1 ) +α2 E (y2 ) +. . . action A is taken when H 0 is true or action B
is taken when H 1 is true. Taking action B when
and variance [303, p. 87] H 0 is true is called a type I error, whereas taking
n action A when H 1 is true is called a type II error.
σ 2 (Y ) = α2i σ 2 (yi )
i=1 The following test of hypothesis or test of
n
n
significance must be defined to determine if the
+2 αi αj σ (yi ) σ (yj ) ̺ij
i=1j=i+1 hypothesis is true. The level of significance is
or the maximum probability that an error would be
accepted in the decision (i.e., rejecting the hy-
n
σ 2 (Y ) =
α2i σ 2 (yi ) pothesis when it is actually true). Common lev-
i=1 els of significance are 0.05 and 0.01, and the test
n n (120)
+2
αi αj Cov (yi ,yj )
of significance can be either one or two sided. If
i=1j=i+1 a sampled distribution is normal, then the prob-
ability that the z score
If the variables are uncorrelated and have the
y−y
same variance, then z=
sy
n
( )
σ 2 (Y ) = α2i σ2
i=1
y) over all y is 1. The probability of a deviation There is also an F-test to decide if two dis-
larger than χ2 is given in Table 14; the area in tributions have significantly different variances.
question, in Figure 54. For example, for 10 de- In this case, the ratio of variances is calculated:
grees of freedom and a 95 % confidence level,
the critical values of χ2 are 0.025 and 0.975. var (x)
F =
Then var (y)
√ √ where the variance of x is assumed to be larger.
s n s n
<σ< Then, a table of values is used to determine the
χ0.975 χ0.025
significance of the ratio. The table [23, Table
or
26.9] is derived from the formula [15, p. 169]
√ √
s n s n ν
2 ν1
<σ< Q (F |ν1 ,ν2 ) = Iν2 /(ν2 +ν1 F ) ,
20.5 3.25 2 2
with 95 % confidence. where the right-hand side is an incomplete beta
Tests are available to decide if two distribu- function. The F table is given by the Microsoft
tions that have the same variance have differ- Excel function FINV(fraction, νx , νy ), where
ent means [15, p. 465]. Let one distribution be fraction is the fractional value (≤ 1) representing
called x, with N 1 samples, and the other be called the upper percentage and νx and νy are the de-
y, with N 2 samples. First, compute the standard grees of freedom of the numerator and denomi-
error of the difference of the means: nator, respectively.
Example. For two sample variances with 8 de-
grees of freedom each, what limits will bracket
their ratio with a midarea probability of 90 %?
FINV(0.95,8,8) = 3.44. The 0.95 is used to get
both sides to toal 10 %. Then
P [1/3.44≤var (x) /var (y) ≤3.44] = 0.90.
* Table value is χ2α ; χ2 < χ2α with probability α. For a more complete table (in slightly different form), see [23, Table 26.8]. The
Microsoft Excel function CHIINV(1-α,ν) gives the table value.
11.3. Error Analysis in Experiments seemingly identical conditions and several dif-
ferent values are obtained, with means E (y1 ), E
Suppose a measurement of several quantities is (y2 ), . . . and variances σ12 , σ22 , . . . Next suppose
made and a formula or mathematical model is the errors are small and independent of one an-
used to deduce some property of interest. For other. Then a change in Y is related to changes
example, to measure the thermal conductivity in yi by
of a solid k, the heat flux q, the thickness of the ∂Y ∂Y
dY = dy1 + dy2 +. . .
sample d, and the temperature difference across ∂y1 ∂y2
the sample ∆T must be measured. Each mea- If the changes are indeed small, the partial
surement has some error. The heat flux q may be derivatives are constant among all the samples.
.
the rate of electrical heat input Q divided by the Then the expected value of the change is
area A, and both quantities are measured to some N
∂Y
tolerance. The thickness of the sample is mea- E (dY ) = E (dyi )
∂yi
sured with some accuracy, and the temperatures i=1
are probably measured with a thermocouple, to Naturally E (dyi ) = 0 by definition so that E (dY
some accuracy. These measurements are com- ) = 0, too. However, since the errors are indepen-
bined, however, to obtain the thermal conduc- dent of each other and the partial derivatives are
tivity, and the error in the thermal conductivity assumed constant because the errors are small,
must be determined. The formula is the variances are given by Equation 121 [296, p.
d .
550]
k= Q N 2
A∆T
∂Y
σ 2 (dY ) = σi2 (121)
If each measured quantity has some variance, i=1
∂yi
what is the variance in the thermal conductiv- Thus, the variance of the desired quantity Y can
ity? be found. This gives an independent estimate
Suppose a model for Y depends on various of the errors in measuring the quantity Y from
measurable quantities, y1 , y2 , . . . Suppose sev- the errors in measuring each variable it depends
eral measurements are made of y1 , y2 , . . . under upon.
Mathematics in Chemical Engineering 133
11.4. Factorial Design of Experiments pling analysis discussed in Section 11.2 can be
and Analysis of Variance used to deduce if the means of the two treatments
differ significantly. With more treatments, the
Statistically designed experiments consider, of analysis is more detailed. Suppose the experi-
course, the effect of primary variables, but they mental results are arranged as shown in Table 15,
also consider the effect of extraneous variables, i.e., several measurements for each treatment.
the interactions among variables, and a measure The objective is to see if the treatments differ sig-
of the random error. Primary variables are those nificantly from each other, that is, whether their
whose effect must be determined. These vari- means are different. The samples are assumed to
ables can be quantitative or qualitative. Quanti- have the same variance. The hypothesis is that
tative variables are ones that may be fit to a model the treatments are all the same, and the null hy-
to determine the model parameters. Curve fitting pothesis is that they are different. Deducing the
of this type is discused in Chapter 2. Qualita- statistical validity of the hypothesis is done by
tive variables are ones whose effect needs to be an analysis of variance.
known; no attempt is made to quantify that ef-
fect other than to assign possible errors or mag- Table 15. Estimating the effect of four treatments
nitudes. Qualitative variables can be further sub- Treatment 1 2 3 4
divided into type I variables, whose effect is de-
− − − −
termined directly, and type II variables, which − − − −
contribute to performance variability, and whose − − − −
effect is averaged out. For example, in studying − − −
− −
the effect of several catalysts on yield in a chem- −
ical reactor, each different type of catalyst would
be a type I variable, because its effect should be Treatment average, y t − − − −
known. However, each time the catalyst is pre- Grand average, y
−
pared, the results are slightly different, because
of random variations; thus, several batches may
exist of what purports to be the same catalyst. The data for k = 4 treatments are arranged in
The variability between batches is a type II vari- Table 15. Each treatment has nt experiments,
able. Because the ultimate use will require us- and the outcome of the i-th experiment with
ing different batches, the overall effect including treatment t is called yti . The treatment average
that variation should be known, because know- is
ing the results from one batch of one catalyst
nt
precisely might not be representative of the re-
yti
sults obtained from all batches of the same cat- yt =
i=1
Treatment 1 2 3 4 Block
average Table 17. Two-level factorial design with three variables
Block 1 − − − − −
Run Variable Variable Variable
Block 2 − − − − −
1 2 3
Block 3 − − − − −
Block 4 − − − − − 1 − − −
Block 5 − − − − − 2 + − −
3 − + −
4 + + −
Treatment − − − − grand
5 − − +
average average
6 + − +
7 − + +
8 + + +
Next consider the case in which randomized
blocking is used to eliminate the effect of some
variable whose effect is of no interest, such as To measure the effects of variables on a single
the batch-to-batch variation of the catalysts in outcome, a factorial design is appropriate. In a
the chemical reactor example. With k treatments two-level factorial design, each variable is con-
and n experiments in each treatment, the results sidered at two levels only, a high and low value,
from n k experiments can be arranged as shown often designated as a + and a −. The two-level
in Table 16; within each block, various treat- factorial design is useful for indicating trends
ments are applied in a random order. The block and showing interactions; it is also the basis for
average, the treatment average, and the grand a fractional factorial design. As an example, con-
average are computed as before. The following sider a 23 factorial design, with 3 variables and
quantities are also computed for the analysis of 2 levels for each. The experiments are indicated
variance table: in Table 17. The main effects are calculated by
determining the difference between results from
Mathematics in Chemical Engineering 135
all high values of a variable and all low values of 12. Multivariable Calculus Applied
a variable; the result is divided by the number of to Thermodynamics
experiments at each level. For example, for the
first variable, calculate Many of the functional relationships required
Effect of variable 1 = [(y2 +y4 +y6 +y8 )
in thermodynamics are direct applications of
the rules of multivariable calculus. In this short
− [(y1 +y3 +y5 +y7 )]] /4
chapter, those rules are reviewed in the context
Note that all observations are being used to sup- of the needs of thermodynamics. These ideas
ply information on each of the main effects and were expounded in one of the classic books on
each effect is determined with the precision of a chemical engineering thermodynamics [299].
fourfold replicated difference. The advantage of
a one-at-a-time experiment is the gain in preci-
sion if the variables are additive and the measure 12.1. State Functions
of nonadditivity if it occurs [303, p. 313].
Interaction effects between variables 1 and 2 State functions depend only on the state of the
are obtained by comparing the difference bet- system, not on its past history or how one got
ween the results obtained with the high and low there. If z is a function of two variables x and
value of 1 at the low value of 2 with the differ- y, then z (x, y) is a state function, because z is
ence between the results obtained with the high known once x and y are specified. The differen-
and low value 1 at the high value of 2. The 12- tial of z is
interaction is
dz = M dx+N dy
12 interaction = [(y4 −y3 +y8 −y7 )
The line integral
− [(y2 −y1 +y6 −y5 )]] /2
The key step is to determine the errors associ- (M dx+N dy)
ated with the effect of each variable and each C
interaction so that the significance can be de- is independent of the path in x – y space if and
termined. Thus, standard errors need to be as- only if
signed. This can be done by repeating the exper-
∂M ∂N
iments, but it can also be done by using higher = (122)
∂y ∂x
order interactions (such as 123 interactions in a
24 factorial design). These are assumed negligi- Because the total differential can be written as
ble in their effect on the mean but can be used
∂z ∂z
to estimate the standard error [303, pp. 319 – dz = dx+ dy (123)
∂x y ∂y x
328]. Then calculated effects that are large com-
pared to the standard error are considered impor- for path independence
tant, whereas those that are small compared to
∂ ∂z ∂ ∂z
the standard error are considered due to random =
∂y ∂x y ∂x ∂y x
variations and are unimportant.
In a fractional factorial design, only part of or
the possible experiments is performed. With k ∂2z ∂2z
variables, a factorial design requires 2k exper- ∂y∂x
=
∂x∂y
(124)
iments. When k is large, the number of exper-
iments can be large; for k = 5, 25 = 32. For k is needed.
this large, Box et al. [296, p. 235] do a frac- Various relationships can be derived from
tional factorial design. In the fractional factorial Equation 123. If z is constant,
design with k = 5, only 8 experiments are cho- & '
∂z ∂z
sen. Cropley [298] gives an example of how to 0= dx+ dy
∂x y ∂y x
combine heuristics and statistical arguments in z
application to kinetics mechanisms in chemical Rearrangement gives
engineering.
136 Mathematics in Chemical Engineering
(∂y/∂x)z
∂z ∂y ∂z which here is
=− =− (125)
∂x y ∂x z ∂y x (∂y/∂z)x
∂T ∂p
Alternatively, if Equation 123 is divided by dy =− (129)
∂V S ∂S V
while some other variable w is held constant,
∂z
∂z
∂x
∂z
This is one of the Maxwell relations and is
= + (126) merely an expression of Equation 124.
∂y w ∂x y ∂y w ∂y x
The differentials of the other energies are
Dividing both the numerator and the denomina-
tor of a partial derivative by dw while holding a dH = T dS+V dp (130)
variable y constant yields
∂z
(∂z/∂w)y
∂z
∂w
dA = −SdT −pdV (131)
= = (127)
∂x y (∂x/∂w)y ∂w y ∂x y
dG = −S dT +V dp (132)
In thermodynamics the state functions in-
clude the internal energy U, the enthalpy H, and From these differentials, other Maxwell rela-
the Helmholtz and Gibbs free energies A and G, tions can be derived in a similar fashion by ap-
respectively, which are defined as follows: plying Equation 124.
H = U +pV ∂T ∂V
= (133)
∂p S ∂S p
A = U −T S
∂S ∂p
= (134)
G = H−T S = U +pV −T S = A+pV ∂V T ∂T V
Cp
∂V
This relates all three partial derivatives of this
dS = dT − dp (136) type.
T ∂T p
Then the total differential for H is Type 2 (30 possibilities plus reciprocals).
∂α ∂G
∂V
dH = T dS+V dp = Cp dT −T dp+V dp General: , Specific:
∂T p ∂b c ∂T V
Rearranging this, when H (T, p), yields Using Equation 132 gives
& ' ∂G ∂p
= −S+V
∂V
dH = Cp dT + V −T dp (137) ∂T V ∂T V
∂T p
Using the other equations for U, H, A, or S gives
This equation can be used to evaluate enthalpy the other possibilities.
differences by using information on the equation
of state and the heat capacity: Type 3 (15 possibilities plus reciprocals).
T
2 ∂a ∂V
H (T2 ,p2 ) −H (T1 ,p1 ) = Cp (T ,p1 ) dT General: , Specific:
T1 ∂b α ∂T S
p2 (138)
First the derivative is expanded by using Equa-
∂V
+ V −T ∂T
|T2 ,p dp
p1 p tion 125, which is called expansion without in-
The same manipulations can be done for in- troducing a new variable:
(∂S/∂T )V
ternal energy: ∂V ∂S ∂V
=− =−
∂T S ∂T V ∂S T (∂S/∂V )T
∂S Cv
= (139)
∂T V T Then the numerator and denominator are evalu-
& ' ated as type 2 derivatives, or by using Equations
(∂V /∂T )p Cv (99) and (100):
dS = − dV + dT (140)
(∂V /∂p)T T
∂V Cv /T
=−
∂T −(∂V /∂T )p (∂p/∂V )T
& '
(∂V /∂T )p S
dU = Cv dT − p+T dV
(∂V /∂p)T ∂V
Cv ∂p T
= (142)
T ∂V
∂T p
12.3. Partial Derivatives of All These derivatives are important for reversible,
Thermodynamic Functions adiabatic processes (e.g., in an ideal turbine or
compressor) because the entropy is constant.
The various partial derivatives of the thermody- Similar derivatives can be obtained for isen-
namic functions can be classified into six groups. thalpic processes, such as a pressure reduction
In the general formulas below, the variables U, at a valve. In that case, the Joule – Thomson co-
H, A, G, or S are denoted by Greek letters, efficient is obtained for constant H:
whereas the variables V, T, or p are denoted by & '
∂T 1 ∂V
Latin letters. = −V +T
∂p H Cp ∂T p
138 Mathematics in Chemical Engineering
Type 4 (30 possibilities plus reciprocals). To evaluate the derivative, Equation 126 is used
∂α
∂G
to express dS in terms of p and T :
General: , Specific:
∂β c ∂A p ∂S ∂V ∂p Cp
=− +
∂T V ∂T p ∂T V T
Now, expand through the introduction of a new
variable using Equation 127: Substitution for (∂p/∂T )V and rearrangement
(∂G/∂T )p
give
∂G ∂G ∂T
= =
∂V
∂p
∂A p ∂T p ∂A p (∂A/∂T )p Cp −Cv = T
∂T p ∂T V
This operation has created two type 2 deriva-
∂V 2 ∂p
tives. Substitution yields = −T
∂T p ∂V T
∂G S
=
∂A p S+p(∂V /∂T )p Use of this equation permits the rearrangement
of Equation 142 into
Cp
(∂V /∂T )2p + (∂V /∂p)T
Type 5 (60 possibilities plus reciprocals). ∂V T
=
∂T S (∂V /∂T )p
∂α ∂G
General: , Specific:
∂b β ∂p A The ratio of heat capacities is
Starting from Equation 132 for dG gives Cp T (∂S/∂T )p
=
Cv T (∂S/∂T )V
∂G ∂T
= −S +V
∂p A ∂p A Expansion by using Equation 125 gives
The derivative is a type 3 derivative and can be Cp −(∂p/∂T )S (∂S/∂p)T
=
evaluated by using Equation 125. Cv −(∂V /∂T )S (∂S/∂V )T
42. J. T. Hsu, J. S. Dranoff: “Numerical Inversion 59. W. F. Ramirez: Computational Methods for
of Certain Laplace Transforms by the Direct Process Simulations, 2nd ed.,
Application of Fast Fourier Transform (FFT) Butterworth-Heinemann, Boston 1997.
Algorithm,” Comput. Chem. Eng. 11 (1987) 60. U. M. Ascher, L. R. Petzold: Computer
101 – 110. methods for ordinary differential equations
43. E. Kreyzig: Advanced Engineering and differential-algebraic equations, SIAM,
Mathematics, 9th ed., John Wiley & Sons, Philadelphia 1998.
New York 2006. 61. K. E. Brenan, S. L. Campbell, L. R. Petzold:
44. H. S. Carslaw, J. C. Jaeger: Conduction of Numerical Solution of Initial-Value Problems
Heat in Solids, 2nd ed., Clarendon Press, in Differential-Algebraic Equations, Elsevier,
Oxford – London 1959. Amsterdam 1989.
45. R. B. Bird, R. C. Armstrong, O. Hassager: 62. C. C. Pontelides, D. Gritsis, K. R. Morison, R.
Dynamics of Polymeric Liquids, 2nd ed., W. H. Sargent: “The Mathematical Modelling
Appendix A, Wiley-Interscience, New York of Transient Systems Using
1987. Differential-Algebraic Equations,” Comput.
46. R. S. Rivlin, J. Rat. Mech. Anal. 4 (1955) 681 Chem. Eng. 12 (1988) 449 – 454.
– 702. 63. G. A. Byrne, P. R. Ponzi:
47. N. R. Amundson: Mathematical Methods in “Differential-Algebraic Systems, Their
Chemical Engineering; Matrices and Their Applications and Solutions,” Comput. Chem.
Application, Prentice-Hall, Englewood Cliffs, Eng. 12 (1988) 377 – 382.
N.J. 1966. 64. L. F. Shampine, M. W. Reichelt: The
48. I. S. Sokolnikoff, E. S. Sokolnikoff: Higher MATLAB ODE Suite, SIAM J. Sci. Comp. 18
Mathematics for Engineers and Physicists, (1997) 122.
McGraw-Hill, New York 1941. 65. M. Kubicek, M. Marek: Computational
49. R. C. Wrede, M. R. Spiegel: Schaum’s outline Methods in Bifurcation Theory and Dissipative
of theory and problems of advanced calculus, Structures, Springer Verlag, Berlin –
2nd ed, McGraw Hill, New York 2006. Heidelberg – New York – Tokyo 1983.
50. P. M. Morse, H. Feshbach: Methods of 66. T. F. C. Chan, H. B. Keller: “Arc-Length
Theoretical Physics, McGraw-Hill, New York Continuation and Multi-Grid Techniques for
1953. Nonlinear Elliptic Eigenvalue Problems,”
51. B. A. Finlayson: The Method of Weighted SIAM J. Sci. Stat. Comput. 3 (1982) 173 – 194.
Residuals and Variational Principles, 67. M. F. Doherty, J. M. Ottino: “Chaos in
Academic Press, New York 1972. Deterministic Systems: Strange Attractors,
52. G. Forsythe, M. Malcolm, C. Moler: Computer Turbulence and Applications in Chemical
Methods for Mathematical Computation, Engineering,” Chem. Eng. Sci. 43 (1988) 139
Prentice-Hall, Englewood Cliffs, N.J. 1977. – 183.
53. J. D. Lambert: Computational Methods in 68. M. P. Allen, D. J. Tildesley: Computer
Ordinary Differential Equations, J. Wiley and Simulation of Liquids, Clarendon Press,
Sons, New York 1973. Oxford 1989.
54. N. R. Amundson: Mathematical Methods in
69. D. Frenkel, B. Smit: Understanding Molecular
Chemical Engineering, Prentice-Hall,
Simulation, Academic Press, San Diego 2002.
Englewood Cliffs, N.J. 1966.
70. J. M. Haile: Molecular Dynamics Simulation,
55. J. R. Rice: Numerical Methods, Software, and
John Wiley & Sons, New York 1992.
Analysis, McGraw-Hill, New York 1983.
71. A. R. Leach: Molecular Modelling: Principles
56. M. B. Bogacki, K. Alejski, J. Szymanowski:
and Applications, Prentice Hall, Englewood
“The Fast Method of the Solution of Reacting
Cliffs, NJ, 2001.
Distillation Problem,” Comput. Chem. Eng. 13
72. T. Schlick: Molecular Modeling and
(1989) 1081 – 1085.
Simulations, Springer, New York 2002.
57. C. W. Gear: Numerical Initial-Value Problems
73. R. B. Bird, W. E. Stewart, E. N. Lightfoot:
in Ordinary Differential Equations,
Transport Phenomena, 2nd ed., John Wiley &
Prentice-Hall, Englewood Cliffs, N.J. 1971.
Sons, New York 2002.
58. N. B. Ferguson, B. A. Finlayson: “Transient
74. R. B. Bird, R. C. Armstrong, O. Hassager:
Modeling of a Catalytic Converter to Reduce
Dynamics of Polymeric Liquids, 2nd ed.,
Nitric Oxide in Automobile Exhaust,” AIChE
Wiley-Interscience, New York 1987.
J 20 (1974) 539 – 550.
Mathematics in Chemical Engineering 141
Scientists, Chapman and Hall/CRC, Boca 120. M. D. Gunzburger: Finite Element Methods for
Raton, FL 2002. Viscous Incompressible Flows, Academic
104. D. Ramkrishna, N. R. Amundson: Linear Press, San Diego 1989.
Operator Methods in Chemical Engineering 121. H. Kardestuncer, D. H. Norrie: Finite Element
with Applications to Transport and Chemical Handbook, McGraw-Hill, New York 1987.
Reaction Systems, Prentice Hall, Englewood 122. J. N. Reddy, D. K. Gartling: The Finite
Cliffs, NJ, 1985. Element Method in Heat Transfer and Fluid
105. D. D. Joseph, M. Renardy, J. C. Saut: Dynamics, 2nd ed., CRC Press, Boca Raton,
“Hyperbolicity and Change of Type in the FL 2000.
Flow of Viscoelastic Fluids,” Arch. Rational 123. O. C. Zienkiewicz, R. L. Taylor, J. Z. Zhu: The
Mech. Anal. 87 (1985) 213 – 251. Finite Element Method: Its Basis &
106. H. K. Rhee, R. Aris, N. R. Amundson: Fundamentals, vol. 1, 6th ed., Elsevier
First-Order Partial Differential Equations, Butterworth-Heinemann, Burlington, MA
Prentice-Hall, Englewood Cliffs, N.J. 1986. 2005.
107. B. A. Finlayson: Numerical Methods for 124. O. C. Zienkiewicz, R. L. Taylor: The Finite
Problems with Moving Fronts, Ravenna Park Element Method, Solid and Structural
Publishing Inc., Seattle 1990. Mechanics, vol. 2, 5th ed.,
108. D. L. Book: Finite-Difference Techniques for Butterworth-Heinemann, Burlington, MA
Vectorized Fluid Dynamics Calculations, 2000.
Springer Verlag, Berlin – Heidelberg – New 125. O. C. Zienkiewicz, R. L. Taylor: The Finite
York – Tokyo 1981. Element Method, Fluid Dynamics, vol. 3, 5th
109. G. A. Sod: Numerical Methods in Fluid ed., Butterworth-Heinemann, Burlington, MA
Dynamics, Cambridge University Press, 2000.
Cambridge 1985. 126. Z. C. Li: Combined Methods for Elliptic
110. G. H. Xiu, J. L. Soares, P. Li, A. E. Rodriques: Equations with Singularities, Interfaces and
“Simulation of five-step one-bed Infinities, Kluwer Academic Publishers,
sorption-endanced reaction process,” AIChE J. Boston, MA, 1998.
48 (2002) 2817–2832. 127. G. J. Fix, S. Gulati, G. I. Wakoff: “On the use
111. A. Malek, S. Farooq: “Study of a six-bed of singular functions with finite element
pressure swing adsorption process,” AIChE J. approximations,” J. Comp. Phys. 13 (1973)
43 (1997) 2509–2523. 209–238.
112. R. J. LeVeque: Numerical Methods for 128. N. M. Wigley: “On a method to subtract off a
Conservation Laws, Birkhäuser, Basel 1992. singularity at a corner for the Dirichlet or
113. W. F. Ames: “Recent Developments in the Neumann problem,” Math. Comp. 23 (1968)
Nonlinear Equations of Transport Processes,” 395–401.
Ind. Eng. Chem. Fundam. 8 (1969) 522 – 536. 129. H. Y. Hu, Z. C. Li, A. H. D. Cheng: “Radial
114. W. F. Ames: Nonlinear Partial Differential Basis Collocation Methods for Elliptic
Equations in Engineering, Academic Press, Boundary Value Problems,” Comp. Math.
New York 1965. Applic. 50 (2005) 289–320.
115. W. E. Schiesser: The Numerical Method of 130. S. J. Osher, R. P. Fedkiw:, Level Set Methods
Lines, Academic Press, San Diego 1991. and Dynamic Implicit Surfaces, Springer, New
116. D. Gottlieb, S. A. Orszag: Numerical Analysis York 2002.
of Spectral Methods: Theory and Applications, 131. M. W. Chang, B. A. Finlayson: “On the Proper
SIAM, Philadelphia, PA 1977. Boundary Condition for the Thermal Entry
117. D. W. Peaceman: Fundamentals of Numerical Problem,” Int. J. Num. Methods Eng. 15
Reservoir Simulation, Elsevier, Amsterdam (1980) 935 – 942.
1977. 132. R. Peyret, T. D. Taylor: Computational
118. G. Juncu, R. Mihail: “Multigrid Solution of Methods for Fluid Flow, Springer Verlag,
the Diffusion-Convection-Reaction Equations Berlin – Heidelberg – New York – Tokyo
which Describe the Mass and/or Heat Transfer 1983.
from or to a Spherical Particle,” Comput. 133. P. M. Gresho, S. T. Chan, C. Upson, R. L. Lee:
Chem. Eng. 13 (1989) 259 – 270. “A Modified Finite Element Method for
119. G. R. Buchanan: Schaum’s Outline of Finite
Solving the Time-Dependent, Incompressible
Element Analysis,” McGraw-Hill, New York
Navier – Stokes Equations,” Int. J. Num.
1995.
Mathematics in Chemical Engineering 143
Method. Fluids (1984) “Part 1. Theory”: 557 – 150. L. M. Delves, J. Walsh (eds.): Numerical
589; “Part 2: Applications”: 619 – 640. Solution of Integral Equations, Clarendon
134. P. M. Gresho, R. L. Sani: Incompressible Flow Press, Oxford 1974.
and the Finite Element Method, 151. P. Linz: Analytical and Numerical Methods for
Advection-Diffusion, vol. 1, John Wiley & Volterra Equations, SIAM Publications,
Sons, New York 1998. Philadelphia 1985.
135. P. M. Gresho, R. L. Sani: Incompressible Flow 152. M. A. Golberg (ed.): Numerical Solution of
and the Finite Element Method , Isothermal Integral Equations, Plenum Press, New York
Laminar Flow, vol. 2, John Wiley & Sons, 1990.
New York 1998. 153. C. Corduneanu: Integral Equations and
136. J. A. Sethian, Level Set Methods, Cambridge Applications, Cambridge Univ. Press,
University Press, Cambridge 1996. Cambridge 1991.
137. C. C. Lin, H. Lee, T. Lee, L. J. Weber: “A level 154. R. Kress: Linear Integral Equations, 2nd ed.
set characteristic Galerkin finite element Springer, Heidelberg 1999.
method for free surface flows,” Int. J. Num. 155. P. K. Kythe P. Purl: Computational Methods
Methods Fluids 49 (2005) 521–547. for Linear Integral Equations, Birkhäuser,
138. S. Succi: The Lattice Boltzmann Equation for Basel 2002.
Fluid Dynamics and Beyond, Oxford 156. G. Nagel, G. Kluge: “Non-Isothermal
University Press, Oxford 2001. Multicomponent Adsorption Processes and
139. M. C. Sukop, D. T. Thorne, Jr.: Lattice Their Numerical Treatment by Means of
Boltzmann Modeling: An Introduction for Integro-Differential Equations,” Comput.
Geoscientists and Engineers, Springer, New Chem. Eng. 13 (1989) 1025 – 1030.
York 2006. 157. P. L. Mills, S. Lai, M. P. Duduković, P. A.
140. S. Chen, S. G. D. Doolen: “Lattice Boltzmann Ramachandran: “A Numerical Study of
method for fluid flows,” Annu. Rev. Fluid Approximation Methods for Solution of Linear
Mech. 30 (2001) 329–364. and Nonlinear Diffusion-Reaction Equations
141. H. T. Lau: Numerical Library in C for with Discontinuous Boundary Conditions,”
Scientists and Engineers, CRC Press, Boca Comput. Chem. Eng. 12 (1988) 37 – 53.
Raton, FL 1994. 158. D. Duffy: Green’s Functions with
142. Y. Y. Al-Jaymany, G. Brenner, P. O. Brum: Applications, Chapman and Hall/CRC, New
“Comparative study of lattice-Boltzmann and York 2001.
finite volume methods for the simulation of 159. I. Statgold: Green’s Functions and Boundary
laminar flow through a 4:1 contraction,” Int. J. Value Problems, 2nd ed., Interscience, New
Num. Methods Fluids 46 (2004) 903–920. York 1997.
143. R. L. Burden, R. L J. D. Faires, A. C. 160. B. Davies: Integral Transforms and Their
Reynolds: Numerical Analysis, 8th ed., Brooks Applications, 3rd ed., Springer, New York
Cole, Pacific Grove, CA 2005. 2002.
144. S. C. Chapra, R. P. Canal: Numerical Methods 161. J. M. Bownds, “Theory and performance of a
for Engineers,” 5th ed., McGraw-Hill, New subroutine for solving Volterra integral
York 2006. equations,” J. Comput. 28 317–332 (1982).
145. H. T. Lau: Numerical Library in Java for 162. J. G. Blom, H. Brunner, “Algorithm 689;
Scientists and Engineers, CRC Press, Boca discretized collocation and iterated collocation
Raton, FL 2004. for nonlinear Volterra integral equations of the
146. K. W. Morton, D. F. Mayers: Numerical second kind,” ACM Trans. Math. Software 17
Solution of Partial Differential Equations, (1991) 167–177.
Cambridge University Press, New York 1994. 163. N. B. Ferguson, B. A. Finlayson: “Error
147. A. Quarteroni, A. Valli: Numerical Bounds for Approximate Solutions to
Approximation of Partial Differential Nonlinear Ordinary Differential Equations,”
Equations, Springer, Heidelberg 1997. AIChE J. 18 (1972) 1053 – 1059.
148. F. Scheid: “Schaum’s Outline of Numerical 164. R. S. Dixit, L. L. Taularidis: “Integral Method
Analysis,” 2nd ed., McGraw-Hill, New York of Analysis of Fischer – Tropsch Synthesis
1989. Reactions in a Catalyst Pellet,” Chem. Eng.
149. C. T. H. Baker: The Numerical Treatment of
Sci. 37 (1982) 539 – 544.
Integral Equations, Clarendon Press, Oxford
1977.
144 Mathematics in Chemical Engineering
248. I. Quesada, I.E. Grossmann: “An LP/NLP 261. R. Pörn, T. Westerlund: “A Cutting Plane
Based Branch and Bound Algorithm for Method for Minimizing Pseudo-convex
Convex MINLP Optimization Problems,” Functions in the Mixed-integer Case,” Comp.
Comp. Chem. Eng. 16 (1992) 937–947. Chem. Eng. 24 (2000) 2655–2665.
249. T. Westerlund, F. Pettersson: “A Cutting Plane 262. J. Viswanathan, I. E. Grossmann: “A
Method for Solving Convex MINLP Combined Penalty Function and
Problems,” Comp. Chem. Eng. 19 (1995) Outer-Approximation Method for MINLP
S131–S136. Optimization,” Comp. Chem. Eng. 14 (1990)
250. O. E. Flippo, A. H. G. R. Kan: 769.
“Decomposition in General Mathematical 263. A. Brooke, D. Kendrick, A. Meeraus, R.
Programming,” Mathematical Programming Raman: GAMS – A User’s Guide,
60 (1993) 361–382. www.gams.com, 1998.
251. T. L. Magnanti, R. T. Wong: “Acclerated 264. N. V. A. Sahinidis, A. Baron: “A General
Benders Decomposition: Algorithm Purpose Global Optimization Software
Enhancement and Model Selection Criteria,” Package,” Journal of Global Optimization 8
Operations Research 29 (1981) 464–484. (1996) no.2, 201–205.
252. N. V. Sahinidis, I. E. Grossmann: 265. C. A. Schweiger, C. A. Floudas: “Process
“Convergence Properties of Generalized Synthesis, Design and Control: A Mixed
Benders Decomposition,” Comp. Chem. Eng. Integer Optimal Control Framework,”
15 (1991) 481. Proceedings of DYCOPS-5 on Dynamics and
253. J. E. Kelley Jr.: “The Cutting-Plane Method Control of Process Systems, Corfu, Greece
for Solving Convex Programs,” J. SIAM 8 1998, pp. 189–194.
(1960) 703–712. 266. R. Raman, I. E. Grossmann: “Modelling and
254. S. Leyffer: “Deterministic Methods for Computational Techniques for Logic Based
Mixed-Integer Nonlinear Programming,” Integer Programming,” Comp. Chem. Eng. 18
Ph.D. thesis, Department of Mathematics and (1994) no.7, 563.
Computer Science, University of Dundee, 267. J. N. Hooker, M. A. Osorio: “Mixed
Dundee 1993. Logical.Linear Programming,” Discrete
255. M. S. Bazaraa, H. D. Sherali, C. M. Shetty: Applied Mathematics 96-97 (1994) 395–442.
Nonlinear Programming, John Wiley & Sons, 268. P. V. Hentenryck: Constraint Satisfaction in
Inc., New York 1994. Logic Programming, MIT Press, Cambridge,
256. G. R. Kocis, I. E. Grossmann: “Relaxation MA, 1989.
Strategy for the Structural Optimization of 269. J. N. Hooker: Logic-Based Methods for
Process Flowsheets,” Ind. Eng. Chem. Res. 26 Optimization: Combining Optimization and
(1987) 1869. Constraint Satisfaction, John Wiley & Sons,
257. I. E. Grossmann: “Mixed-Integer Optimization New York 2000.
Techniques for Algorithmic Process 270. E. Balas: “Disjunctive Programming,” Annals
Synthesis”, Advances in Chemical of Discrete Mathematics 5 (1979) 3–51.
Engineering, vol. 23, Process Synthesis, 271. H. P. Williams: Mathematical Building in
Academic Press, London 1996, pp. 171–246. Mathematical Programming, John Wiley,
258. E. M. B. Smith, C. C. Pantelides: “A Symbolic Chichester 1985.
Reformulation/Spatial Branch and Bound 272. R. Raman, I. E. Grossmann: “Relation
Algorithm for the Global Optimization of between MILP Modelling and Logical
Nonconvex MINLPs,” Comp. Chem. Eng. 23 Inference for Chemical Process Synthesis,”
(1999) 457–478. Comp. Chem. Eng. 15 (1991) no. 2, 73.
259. P. Kesavan P. P. I. Barton: “Generalized 273. S. Lee, I. E. Grossmann: “New Algorithms for
Branch-and-cut Framework for Mixed-integer Nonlinear Generalized Disjunctive
Nonlinear Optimization Problems,” Comp. Programming,” Comp. Chem. Eng. 24 (2000)
Chem. Eng. 24 (2000) 1361–1366. no. 9-10, 2125–2141.
260. S. Lee I. E. Grossmann: “A Global 274. J. Hiriart-Urruty, C. Lemaréchal: Convex
Optimization Algorithm for Nonconvex Analysis and Minimization Algorithms,
Generalized Disjunctive Programming and Springer-Verlag, Berlin, New York 1993.
Applications to Process Systems,” Comp. 275. E. Balas: “Disjunctive Programming and a
Chem. Eng. 25 (2001) 1675–1697. Hierarchy of Relaxations for Discrete
148 Mathematics in Chemical Engineering
Optimization Problems,” SIAM J. Alg. Disc. 289. H. G. Bock, K. J. Plitt: A Multiple Shooting
Meth. 6 (1985) 466–486. Algorithm for Direct Solution of Optimal
276. I. E. Grossmann, S. Lee: “Generalized Control Problems, 9th IFAC World Congress,
Disjunctive Programming: Nonlinear Convex Budapest 1984.
Hull Relaxation and Algorithms,” 290. D. B. Leineweber, H. G. Bock, J. P. Schlöder,
Computational Optimization and Applications J. V. Gallitzendörfer, A. Schäfer, P. Jansohn:
26 (2003) 83–100. “A Boundary Value Problem Approach to the
277. S. Lee, I. E. Grossmann: “Logic-based Optimization of Chemical Processes Described
Modeling and Solution of Nonlinear by DAE Models,” Computers & Chemical
Discrete/Continuous Optimization Problems,” Engineering, April 1997. (IWR-Preprint
Annals of Operations Research 139 2005 97-14, Universität Heidelberg, March 1997.
267–288. 291. J. E. Cuthrell, L. T. Biegler: “On the
278. ILOG, Gentilly Cedex, France 1999, Optimization of Differential- algebraic Process
www.ilog.com, Systems,” AIChE Journal 33 (1987)
279. M. Dincbas, P. Van Hentenryck, H. Simonis, 1257–1270.
A. Aggoun, T. Graf, F. Berthier: The 292. A. Cervantes, L. T. Biegler: “Large-scale DAE
Constraint Logic Programming Language Optimization Using Simultaneous Nonlinear
CHIP, FGCS-88: Proceedings of International Programming Formulations. AIChE Journal
Conference on Fifth Generation Computer 44 (1998) 1038.
Systems, Tokyo, 693–702. 293. P. Tanartkit, L. T. Biegler: “Stable
280. M. Wallace, S. Novello, J. Schimpf: Decomposition for Dynamic Optimization,”
“ECLiPSe: a Platform for Constraint Logic Ind. Eng. Chem. Res. 34 (1995) 1253–1266.
Programming,” ICL Systems Journal 12 294. S. Kameswaran, L. T. Biegler: “Simultaneous
(1997) no.1, 159–200. Dynamic Optimization Strategies: Recent
281. V. Pontryagin, V. Boltyanskii, R. Gamkrelidge, Advances and Challenges,” Chemical Process
E. Mishchenko: The Mathematical Theory of Control – 7, to appear 2006.
Optimal Processes, Interscience Publishers 295. B. van Bloemen Waanders, R. Bartlett, K.
Inc., New York, NY, 1962. Long, P. Boggs, A. Salinger: “Large Scale
282. A. E. Bryson, Y. C. Ho: “Applied Optimal Non-Linear Programming for PDE
Control: Optimization, Estimation, and Constrained Optimization,” Sandia Technical
Control,” Ginn and Company, Waltham, MA, Report SAND2002-3198, October 2002.
1969. 296. G. P. Box, J. S. Hunter, W. G. Hunter:
283. V. Vassiliadis, PhD Thesis, Imperial College, Statistics for Experimenters: Design,
University of London 1993. Innovation, and Discovery, 2nd ed., John
284. W. F. Feehery, P. I. Barton: “Dynamic Wiley & Sons, New York 2005.
Optimization with State Variable Path 297. D. C. Baird: Experimentation: An Introduction
Constraints,” Comp. Chem. Eng., 22 (1998) to Measurement Theory and Experiment
1241–1256. Design, 3rd ed., Prentice Hall, Engelwood
285. L. Hasdorff:. Gradient Optimization and Cliffs, NJ, 1995.
Nonlinear Control, Wiley-Interscience, New 298. J. B. Cropley: “Heuristic Approach to
York, NY, 1976. Comples Kinetics,” ACS Symp. Ser. 65 (1978)
286. R. W. H. Sargent, G. R. Sullivan: 292 – 302.
299. S. Lipschutz, J. J. Schiller, Jr: Schaum’s
“Development of Feed Changeover Policies
Outline of Theory and Problems of
for Refinery Distillation Units,” Ind. Eng.
Introduction to Probability and Statistics,
Chem. Process Des. Dev. 18 (1979) 113–124.
McGraw-Hill, New York 1988.
287. V. Vassiliadis, R. W. H. Sargent, C. Pantelides:
300. D. S. Moore, G. P. McCabe: Introduction to
“Solution of a Class of Multistage Dynamic
the Practice of Statistics,” 4th ed., Freeman,
Optimization Problems,” I & EC Research 33
New York 2003.
(1994) 2123.
301. D. C. Montgomery, G. C. Runger: Applied
288. K. F. Bloss, L. T. Biegler, W. E. Schiesser:
Statistics and Probability for Engineers, 3 rd
“Dynamic Process Optimization through
ed., John Wiley & Sons, New York 2002.
Adjoint Formulations and Constraint 302. D. C. Montgomery, G. C. Runger, N. F.
Aggregation,”. Ind. Eng. Chem. Res. 38 (1999) Hubele: Engineering Statistics, 3rd ed., John
421–432. Wiley & Sons, New York 2004.
Mathematics in Chemical Engineering 149