Talanta 148 (2016) 641–648

Contents lists available at ScienceDirect

Talanta
journal homepage: www.elsevier.com/locate/talanta

Optimization using the gradient and simplex methods

Víctor Cerdà a,n, Juan Luis Cerdà b, Abubakr M. Idris c
a
Chemistry Department, University of the Balearic Islands, 07122 Palma de Mallorca, Spain
b
Department of Applied Mathematics and Analysis, University of Barcelona, Spain
c
Department of Chemistry, College of Science, King Khalid University, Abha 61321, Saudi Arabia

art ic l e i nf o a b s t r a c t

Article history: Traditionally optimization of analytical methods has been conducted using a univariate method, varying
Received 11 February 2015 each parameter one-by-one holding fixed the remaining. This means in many cases to reach only local
Received in revised form minima and not get the real optimum. Among the various options for multivariate optimization, this
21 May 2015
paper highlights the gradient method, which involves the ability to perform the partial derivatives of a
Accepted 25 May 2015
mathematical model, as well as the simplex method that does not require that condition.
Available online 12 June 2015
The advantages and disadvantages of those two multivariate optimization methods are discussed,
Keywords: indicating when they can be applied and the different forms that have been introduced. Different cases
Multivariate optimization are described on the applications of these methods in analytical chemistry.
Simplex
& 2015 Elsevier B.V. All rights reserved.
Maximum slope
Gradient

1. Introduction minimize for example interferences or an experimental error. In

Optimization is usually a mandatory step when working in
experimental sciences and engineering, covering from mathema-
tical functions and industrial processes to new analytical methods.
Despite its familiarity and ease of use, univariate optimization
process is time- and reagent-consuming, besides it is unable to
consider interaction effect between conditions. Hence, the max-
imum efficiency of analytical methods might not be obtained. This
because univariate is based on optimizing conditions one-by-one
by varying levels of one condition while levels of other conditions
are held constant. The more optimization effective approach is
multivariate, which optimizes all conditions simultaneously by
varying levels of all of them. Accordingly, multivariate optimiza-
tion could obtain the highest efficiency of analytical methods in
the shortest time period.
When optimizing a magnitude or a mathematical function, e.g.
U¼ f(x,y,….,z)¼ f(X), that depends on one or more variables or
factors, X ¼(x,y,…,z), the function is called the target function and
the values of the variables giving an optimum value (maximum or
minimum) for the target function have to be determined. Thus, a
maximum will be the aim when for example optimizing the effi-
ciency of a reaction, the maximum signal of an instrument and the
maximum difference between a signal and the instrumental noise.
On the contrary, a minimum will be desirable if the aim is to
n
Corresponding author. Fax: þ 34 971173426.
E-mail address: victor.cerda@uib.es (V. Cerdà).
other cases, targeted values are desirable to be obtained like re-
covery (100%) in sample treatment and resolution in separation
techniques with maintaining short analysis time.
We are going to focus on how to optimize a minimum. How-
ever, if a maximum of a function f(X) is the aim, we just have to
minimize the opposite function, i.e. U¼  f(X).
Although the main aim is to find absolute maxima and minima,
mathematical methods frequently provide relative or local max-
ima and minima depending on the starting point (Fig. 1). There-
fore, it is advisable to assure that the maximum or minimum
found are the absolute ones repeating the calculation starting from
different initial points.
There are different strategies to obtain the optimum values for
different optimization cases, which may be simultaneous (e.g.
Gradient, Simplex and Evolutionary Operation) [1–3] or sequential
(e.g. Box–Behnken, Central Composite, Doehlert and Factorial De-
sign) [3]. The calculation method has to be chosen according to
each system [1,4]. Below are described some sequential methods
that can efficiently meet most common optimization cases in ex-
perimental sciences. The objective function has to be classified in
those having derivatives and those not having. Thus, there are two
different options,
1. The gradient method is recommended for functions with sev-
eral variables and obtainable partial derivatives, using a varia-
tion of the Newton, Davidon, Fletcher and Powell methods
2. For functions with several variables and unobtainable partial
derivatives, the simplex method is then the best option.

http://dx.doi.org/10.1016/j.talanta.2015.05.061
0039-9140/& 2015 Elsevier B.V. All rights reserved.
642 V. Cerdà et al. / Talanta 148 (2016) 641–648
Relative
maxima
Absolute
maximum
Fig. 1. Level curves with some local maxima and an absolute maximum.
All cited these are numerical methods, which are based on
refining initial values. These initial values are estimated according
to the knowledge of the particular problem, e.g. when estimating
the pK of a functional group, or by values obtained with other
calculation methods of approximation, either graphical or analy-
tical ones.
If possible, it is advisable to take advantage of algorithms using
derivatives to achieve a better reliability and rapid convergence to
the optimum (gradient method), avoiding direct search despite
their simplicity (simplex method).
The aim of this review is to provide a general knowledge on the
gradient and simplex methods in the matter of optimization in the
experimental sciences, avoiding an excessive use of mathematical
language, and providing an intuitive justification of the proposed
methods. More specialized texts in the matter can be found in the
literature [5–7]. A good introduction to numerical methods of
analysis can be found in bibliography [8]. Below are described a
selection of useful methods to apply in the experimental sciences
field to achieve an optimum.
2. The gradient method
In the “gradient”, “maximum slope” or “steepest-ascent”
method, a level set, or a level curve in the case of having two
variables of U¼ f(X), is a set of points where the studied function
has always the same value. Furthermore, not only U does not vary
in this set, but also it is easily demonstrated that the variation of U
respect to the parameters is the maximum in the direction per-
pendicular to the level set. This direction is determined by the
gradient vector G(X) (Eq. (1)), whose components are first partial
derivatives of function “f”. It has to be taken into account that the
gradient vector size shouldn’t be constant, otherwise the max-
imum won’t be reached, it will be surpassed (Fig. 2a). The criterion
to be followed is to reduce the gradient vector size as the slopes do
Fig. 2. The gradient method with (a) constant vector size and (b) variable vector size.
Fig. 3. The gradient method for two variables.
so (Fig. 2b).
G(X ) = (fx (X ), …, fz (X )), (1)
In the case of having two variables affecting the target function,
if we follow the gradient direction, i.e. a way perpendicular to the
level curves (Fig. 3), we will be following the curve with maximum
slope. Thus, we will pass from point (Xm) to the next (Xm þ 1)
considering Eq. (2) for maximizing, and Eq. (3) for minimizing,
being the step “k” always k4 0. It will be clearer if the points Xm,
Xm þ 1, etc. are presented in Fig. 4.
X m + 1 = X m + k G(X m) (2)
X m + 1 = X m –k G(X m) (3)
2.1. Examples of application of the gradient method
The gradient method has been widely used in programs to
refine thermodynamic parameters, such as equilibrium constants,
reaction enthalpies and entropies, where the sum of the residuals
squares is minimized (Table 1).
All programs listed in Table 1 need initial parameters values. As
closer these initial values to the optimum, as faster will be the
convergence to it and the risk to diverge will also be lower.
However, the chemical criterion has to be always in mind, other-
wise absurd results will be obtained, e.g. to obtain three constants
of protonation for a diprotic acid because the residuals value is
lower.
With the last four programs shown in Table 1, two different
processes can be applied to reach the optimum, i.e. a process
 V. Cerdà et al. / Talanta 148 (2016) 641–648 643

Fig. 4. (a) Search method (univariate). (b) Refinement method (multivariate).

Table 1 optimum. Also, difficulties increase along with the number of

Examples of programs using the gradient method. variables (N) to optimize and if the target function cannot be
analytically differentiated. Notably, the optimization of non-dif-
Program Function Parameters to refine Reference
ferentiating functions is quite common in experimental sciences as
MINIQUAD 2 Equilibrium constants [9] in analytical chemistry. An algebraic expression for all variables in
(
U = ∑n pHcalculated − pHexp ) experimental sciences is not usually available.
MINIPOT 2 Equilibrium constants [10]
U = ∑n (Ecalculated − Eexp) When it is difficult or impossible to use the gradient method,
MINISPEF
U = ∑n (Acalculated − Aexp )
2 Equilibrium constants [11] there are great and widely known alternatives provided by direct
and molar sequential procedures. These are generally based on modifications
absorbtivities
MINIPOL 2 Equilibrium constants [12]
of the simplex method. The simplex method was first suggested by
( )
U = ∑n Icalculated − Iexp
and intensities Spendley, Hext and Himsworth [22]. The most common mod-
MINITERM 2 Equilibrium constants [13] ifications used are the Modified Simplex Method (MSM),proposed
U = ∑n (ΔTcalculated − ΔTexp)
and enthalpies by Nelder and Mead [23], and related variants such as the Super
Modified Simplex (SMS) proposed by Routh, Swartz and Denton
based on iterative search following an univariate procedure [24],and the Gaussian adjustment method proposed by Van der
(Fig. 4a), and the other based on multivariate refining exploiting Wiel [25].
the gradient method (Fig. 4b). Below are described in more detail the MSM method and its
With two variables affecting the target function, in the search most representative variants, selected by their usefulness in ex-
method first a wide screening for one variable is made to find a perimental applications.
local minimum for this first variable, maintaining the other vari-
able at a fixed value. Then for the second variable a screening is
made, maintaining the first variable fixed at the value found in the 3. The simplex method
first local minimum, and a second local minimum is found. In
Fig. 4a, it can be seen that an absolute minimum is not achieved The simplex method is based on a geometric figure defined by a
with only two screenings. Univariate screenings have to be re- number of points equal N þ 1 to the number of factors to optimize
peated to go closer to the optimum involving the handling of too (N).
many points. The search method can be helpful as a previous step The simplest scenario comprises two factors and the simplex
to the gradient method application in order to start in a point close would be a triangle. As a way of example, imagine that we aim to
to the optimum, avoiding divergent processes. However, the optimize the combustible flow rate (factor 1, F1) and the oxidizer
number of points is greatly reduced in the gradient method in flow rate (factor 2, F2) to determine an element X using atomic
comparison with the search method. absorption spectrometry.
As a way of example, in the supplementary material can be Fig. 5 shows the level curves with the same response, that is to
found tables of input data and output results of the program say those with absorbances with the same value at different flow
MINITERM when applying first the univariate search method, and rates combination. The aim is to obtain the combination where the
then the multivariate gradient method to refine two equilibrium absorbance is maximal.
constants using enthalpimetric experiments. The optimization starts at the points 1, 2 and 3, which form an
In other applications, the steepest-ascent method was exploi-
ted for optimizing the most effective analytical conditions after
screening using multivariate simultaneous optimization ap-
proaches as full and partial factorial design and Plackett–Burman
[14–20].

2.2. Gradient method difficulties

The gradient method or “steepest descent” and its variants are

considered the best option to evaluate an optimum and its para-
meters or coordinates when it is possible to obtain the derivative
of the target function and a good estimation of the initial values
[21]. However, it is well known that the effectiveness of these
methods decreases if the initial values are far away from the Fig. 5. Simplex advance.
644 V. Cerdà et al. / Talanta 148 (2016) 641–648
equilateral triangle and giving the point 3 as the worst response.
Thus, the response will probably increase in the opposite direction
to point 3. Therefore, the reflected triangle 1-2-4 is studied, being
4 the reflected point of point 3. A new experiment is carried out to
get the absorbance of point 4. If point 2 gives the worst answer of
the new simplex, the new triangle with the reflected point of 2 is
obtained. This procedure is successively repeated producing a
succession of simplexes that move rapidly over the response
surface.
To achieve a more efficient progress strategy, a series of rules
have been proposed as detailed below:
1. Rule 1: The new simplex is created discarding the point giving
the worst response and replacing it with its mirror image on a
line defined according to the other two points. If the new point
gives the following worst response, rule 1 can no longer be
applied since the reflected point will be the original one. The
application of rule 1 will result in an endless loop between the
two created simplexes. Thus, rule 2 has to be followed.
2. Rule 2: So if the new point gives the worst response, then the
second worst point has to be selected and its reflected point
will be obtained to create the new simplex. A change in the
progression direction to the optimum is obtained. This happens
usually when we are approaching the optimum.
3. Rule 3: Thus, if the new point is close to the optimum, all new
points will surpass the response surface apex and next sim-
plexes will turn around the optimum. This phenomenon is
shown in Fig. 6 where the turning starts from point 9.
Simplex method variants arise with the aim of solving this
difficulty and to speed up the search of the optimum.
3.1. Simplex method modifications
3.1.1. The modified simplex method
Having N parameters (being N 42), the MSM [23] starts the
evaluation of the target function “U” in the N þ1 vertexes, e.g. S(1),
S(2) …..S(N þ 1), of the initial simplex. These vertexes can be cal-
culated from the initial point P(1),…, P(N) and the size of the
simplex side “K” as in Eq. (4)
S(1) P(1) + RP(2) + T…P(N ) + TS(2) P(1) + TP(2) + R…P(N )
+ TS(N ) P(1) + TP(2) + T…P(N ) + RS(N + 1) P(1) P(2)…P(N )
: evaluation of N new vertexes is required before continuing
where,
(N + 1)2 + N − 1 (N + 1)1/2 − 1
R= K, T = K
2N 2N
S(1)
C B C R
S(3) D
S(2)
Fig. 6. Reflected, expanded, and contracted simplex.
Supposing the objective is to find a minimum, for example the
minimum of the sum of squared residuals, the target function “U”
will be calculated as in Eq. (6):
2
U (PN ) = ∑ (Mcalc − Mexp)
i (6)
This function is very used to refine experimental parameters
aiming to obtain a maximum adjustment between the calculated
values of a function Mcalc(PN)i and the experimental data Mexp se-
lecting the right N parameters P(1),…..,P(N).
In this example, the worst vertex will be that giving the highest
U value. Following the previously described strategy this vertex
has to be discarded to choose a new one according to a series of
simple rules. The method is iterated to progressively improve the
U value until the optimum region is located.
The simplex vertexes are numbered, being S(3) the worst, S
(2) the second worst and S(1) the best point. When defining B as
the barycenter of the opposite face of S(3), the worst vertex S(3) is
deleted and the new one “R” is obtained by reflection according to
Eq. (7).
R = B + (B − S(3)) (7)
The following exceptions should be taken into account:
a. If U(R)oU(S(1)) and U(R)4U(D), where D is the expanded, i.e.
at double distance of the reflected point, then:
D = B + 2(B − S(3)) (8)
b. If U (R) < U (S(3)) and U (R) > US((2)) (9)
c. If U (R) > U (S(3)) (10)
In case (a) the reflected simplex is expanded and the new
vertex is D (Fig. 6). On the other hand, in cases (b) and (c) an
external and internal contraction is produced and the new
vertex is C ¼B 70.5(B  S(3)) (Fig. 7).
The so called critical case is when the worst response is ob-
tained in the new point C. In order to continue it is re-
commended to follow one of the alternatives described below:
(4) 1. Massive contraction (Fig. 8), proposed by Nelder and Mead
[23]. S is contracted to S(1) by a factor of 0.5. Then, the
7
8
(5)
5
6
1
3 4 4
2
Fig. 7. Simplex progress with reflections, expansions and contractions.
 V. Cerdà et al. / Talanta 148 (2016) 641–648
Fig. 8. Simplex massive contraction.
Q n = B + 2n (D − B) (n = 1, 2)
Fig. 9. Simplex translation.
with the algorithm execution. There is the possibility of
premature convergence in case of error.
2. Total simplex translation (Fig. 9), proposed by Ernst [7]
3. The simplex is relocated being (1) the barycenter B’ of the
new simplex. Simplex in this case does not contract, but
the evaluation of N þ1 requires new vertices (Please check
translation).
4. The variant “next worst”, consists in permuting S(2) with S
(3) when modifying the simplex, to change the simplex
progress avoiding a premature contraction.
3.1.2. Supermodified simplex method
In SMS method (Fig. 10), instead of a sequence of reflections, ex-
pansions and contractions to get the new vertexes, the fitting of a
second order polynomial curve for the values of U(S(3)), U(B), U(R)
of the vertexes S(3), B and R is used.
The curve is extrapolated further than S(3) and R, a distance
proportional to the distance between S(3) and R. The new vertex
Fig. 10. Supermodified simplex method.
645
Fig. 11. Unidirectional progress.
will be the point with the corresponding minimum value of the
quadratic function. Vicinities of B are excluded to avoid the resize
reduction during the process.
3.2. Other modifications
In order to have a faster progress to the optimum and to pre-
vent simplex degeneration due to consecutives deformations
when applying MSM and SMS, i.e. when the simplex is not close to
the optimum region, we have proposed an expanded simplex
translation in the direction S(3) -D (unidirectional progress).
When the cases (a) of the MSM takes place, this unidirectional
progress is suggested if U(D) oU(S(1)) and U(Qn)oU(D), where
The last point Q ¼Qn is related to condition U(M) 4U(Q) being
M¼Qn þ i (Fig. 11). Unidirectional progress is recommended only if
n 42; otherwise experience indicates that the expansion of the
MSM (case (a)) is preferable. There are two variants of the uni-
directional progress, namely:
1. Simple unidirectional progress. The new expanded simplex in
(a) is transferred. Hence, the new vertexes are Q¼S(1)þ V, S
(2)þV, …S(N þ1) þV, where V¼Q  S(N þ1).
2. Unidirectional progress with interpolation. The translation
vector is V¼A  D, being A the point giving better response than
Q and P, where P is the point obtained by quadratic inter-
polation respect to values U(D),U(Q), U(M). That is to say P is the
point corresponding to the minimum of the second order
polynomial.
These methods can also be applied in the critical case, pre-
viously described, with an unidirectional progress of S in the di-
rection S(3)–S(1). It is enough to use S(1) instead of B to construct
Q¼Qn (Fig. 12).
Fig. 13 shows the simplex progress witha reflexion, unidirec-
tional translation and internal contraction.
3.3. Simplex method and simplex variants applications
As previously stated, the simplex is an optimization method
based on the logical search strategy of the minimum or maximum
of a function or an experiment with or without mathematical
model. In fact, the significant difference in comparison with other
methods of refining, such as the gradient method, is the no need to
differentiate a mathematical function.
As other optimization methods, it requires approximately
knowing the values of the factors to refine, that will be used as
initial values.
As previously stated, one of the greatest problems of refining
Fig. 12. Unidirectional progress of Sin direction S(3)–S(1).
646 V. Cerdà et al. / Talanta 148 (2016) 641–648
Fig. 13. Simplex progress with a reflexion, unidirectional translation and internal
contraction.
methods is the convergence to a local minimum and not to the
absolute one. Therefore, it is recommended to try different initial
points and values for the side size of the initial simplex. Experi-
ence will indicate the most adequate initial values for each parti-
cular problem.
3.3.1. Simplex applications
Due to our research needs, we have developed different pro-
grams to apply the simplex method to three different cases:
a. Simplex program: to find the minimum of a mathematical
function, e.g. U¼X2 þY2, where its value is 0 and located in
x¼ 0, and y¼ 0.
b. SIMFIA program: to find the minimum of an experimental
function knowing its mathematical model (least square ad-
justment) [26,27].
c. ExpSimplex program: to find the minimum in an experimental
system not knowing its mathematical model. For example, the
optimization of an atomic absorption instrument adjusting
combustible and oxidizer flow rates. The simplex method was
applied for optimizing the automation of the Winkler metho-
dology using sequential injection analysis (SIA) [28] and opti-
mizing the set-up of dual-laser crossed-beam thermal lens
spectrophotometer [29]. Recently, the Simplex method was
applied as well for optimizing four volumetric and two che-
mical variables controlling the quantification of dissolved
oxygen by SIA [28]. Vertex calculation and consequent appli-
cation including in-line preparation of one reagent were car-
ried out in real-time using the software AutoAnalysis.
Actually, examples a and b can be solved with higher efficiency
using algorithms including derivatives, which should be preferably
used to sequential methods.
3.3.2. Simplex method variants applications
The three programs cited previously have been created af-
fording different progress strategies of the simplex during opti-
mization, including translation, unidimensional progress, expan-
sion and interpolation between others.Up to eight variants can be
chosen, covering from the original of the method by Nelder and
Mead [23] to own modifications. The convergence velocity de-
pends on the kind of problem and the quality of the initial values.
Experience will determine the most convenient variant.
A more detailed description of applications and lists of these
three programs can be found in the supplementary material.
4. Scales and finalization criteria
There are common practical considerations for all optimization
methods which are not usually taken into account, despite being
important to assure effectiveness.
On the one hand, a correct election of the scale, i.e. origin and
unit of measurement for all the variables involved, is necessary to
achieve a method sensible to the variation of all factors and to
assure a size adequate to the chosen error bounds. In another
context, finalization criteria have to avoid unnecessary and ex-
cessive iterations and also premature finalization in false optima.
4.1. Scale selection for the variables and the function
A change of scale for the variables involves a correspondence
between the initial variables X¼ (x1,…,xN) of the target function f
(X) and the new variables Y ¼(y1,…,yN) obtained changing the
origin and the units of measurement as shown in Eq. (11):
Xj = αjY j + βj , (j = 1, …, N ) (11)
Below are listed some basic rules to select the right scale:
1. All involved variables should have a similar magnitude, and of the
order of a unit in the region of interest.
2. It is very useful to know a fitted and realistic field of variability of
each parameter, aj oxj obj, and do the change of scale according
to Eq. (12).
Xj = (bj − aj )yj + αj (12)
3. No change in the scale should be made if it is possible that the
limits aj and bj are wrong.
4. For the scale of the function, it is recommended to choose the
values of α and β between  1 and 1, or between 0 and 1, when the
parameters are in the region of interest.
5. The scale should be so that the optimum value of the function
would be 0.Thus, it is better to consider f(x,y) ¼ x2 þ y2 than f(x,
y)¼ x2 þ y2 þ 100000.
In the optimization with derivatives, the scale is so that the
variation of the function is very different respect to the variation of
the parameters. This issue leads to great differences in the values
of the partial derivatives when changing the variable of derivation.
Accordingly, scale changes are needed for the variables where αj
accomplish Eq. (12). Moreover, it has to be taken into account that
the gradient method depends strongly on the selected scale.
1 + f (X )
αj =
2f j (X ) (13)
where fj(X) is the partial derivative of “f” respect to Xj.
4.2. Finalization criteria
Although an optimization is solved when determining a point X
where G(X) ¼0 or, if knowing the minimum value, f(X) ¼ min, with
algorithms of finite precision to obtain the convergence to the
optimum (such as those applied in practice), only approximate
solutions will be obtained. That is to say, previous conditions
should be replaced by others of the kind G(X)¼ 0 or f(X)¼min.
Accordingly, by establishing the finalization criteria, the fol-
lowing objectives could be achieved:
 V. Cerdà et al. / Talanta 148 (2016) 641–648
1. To check the acceptance of an estimated optimum having the
margin of error or the working precision.
2. In case of an estimated optimum is not accepted, it is advisable
to adopt a criterion to stop the process, especially if the process
is slow or requires an elevated number of evaluations of the
function, since the method can lead to infinite iterations
without convergence to the optimum.
Universal finalization criteria cannot be established for all op-
timization methods or cases. However, the correct selection of the
scale should be the norm.
Generally, criteria are elaborated by the user fixing the value of
a positive magnitude ε , responsible of the significant figures of the
function and/or parameters values at the optimum.
To describe some criteria, suppose that the successive refine-
ments of the parameters are X1, X2 ….and the corresponding va-
lues of the function are y1, y2, …. In the case of optimization using
derivatives, the criterion of finalization will be to simultaneously
accomplish the three conditions described below:
(1) |yk  yk  1| o ε(1 þ|yk|)
(2) I|xjk  xjk  1| o(ε(1 þ |xjk|))½ (j¼1,…,N)
(3) |fj(Xk)| r ε⅓(1 þ|yk|)
where Xk ¼(xjk)¼ (x1,…,XN
k ).
Condition (2) can be ignored if condition (1) is accomplished
and a right selection of scales has been made.
In the simplex method it can be ended if f(S(3))  f(S(1)) o ε (1
þf(S(1)); or if knowing the minimum value OPT of the function f, if
f(S(1))–OPTo ε.
Given the possibility of an experimental error, it is re-
commended that an extra condition, described below, should be
applied in order to avoid a premature finalization.
( )
|x3j − x1j | < (ε 1 + |x1j | )½ (j = 1, … ... , N ),
where X3 ¼S(3) and X1 ¼ S(1).
Nelder and Mead [23] adopt the finalization criterion to ac-
complish the following condition.
⎛ N+1 ⎞1/2
1 ⎜ 2⎟
∑ [f (B ) − f (S (i ))] ⎟ <ε
N + 1 ⎜⎝ i = 1
⎠ and the translation when the unidirectional progress has been
where S(1),…,S(N þ 1)are the N þ1 vertexes of the last simplex and
B the barycenter of the opposite face to S(3).
When dealing with experimental functions and taking into
account possible measurement errors, it is advisable to apply the
rule “N þ1” since the value of the function is reevaluated in S(1) if
this vertex gives the best response after N þ1 consecutive
iterations.
5. Assay functions
Once elaborated an optimization computerized program, it has
to be evaluated and compared with others.
It is advisable to check its operation by stages. First with simple
problems and then with more complex ones, increasing the
number of variables and the precision required to finalize. Then if
the components work properly, the program has to be checked
with different functions [30].
Some of the most assayed functions to check the efficacy of
different alternatives are:
U = x12 + … … + x n2 (from N = 2 up to 8) (14)
647
2
U = [1.5 − x(1 − y)]2 + ⎡⎣2.25 − x(1 − y)2⎤⎦
2
+ ⎡⎣2.65 − x(1 − y)3⎤⎦ (15)
2
(
U = 100 y − x2 ) + (1 − x)2 (16)
The optimum will be located at (1,1,…, 1) and its value will be
zero. (  1, 2, 1,….,  1, 2, 1) can be taken as initial points [31]
U = (x − y + z )2 + (−x + y + z )2 + (x + y − z )2 (17)
Zangwill [32]
U = x2 + 2Y 2 + 3Z2 + 4W 2 + (x + y + z + w )4 (18)
U = (x + 10y)2 + 5(z − t )2 + (y − 2x)4 + 10(z − t )2 (19)
With the minimum at (0,0,0,0), (3,  1, 0, 1)is suggested as
initial point.
As in Rosenbrock [31] function, Powell function [33] can be
expanded to 4n variables repeating in groups of 4 variables the
above expression.
2
U=
n
∑i = 1 (n − ∑ n
j=1
( ) )
(cos xj + i 1 − cosxj − senxj )
(20)
Trigonometric function, with n variables for X¼ (x1, x2, …., xn).
(1/n, 1/n,…, 1/n) will be the initial point.
In order to study the different optimization methods we have
conducted a great number of assays with a random selection of the
initial point and of the size K of the first simplex.
We have mainly focused in comparing the Nelder and Mead
method [23] and the two variants of the unidirectional progress,
together with the different options in the critical cases, i.e. massive
contraction, translation and unidirectional progress. The number
of evaluations needed to reach the optimum has been determined.
For all functions, the unidirectional progress approximation has
proved to be more efficient than just the expansion of the reflected
simplex in MSM. This advantage increases when increasing N, with
the relation of the distance of the initial point and the optimum
and also with the size K of the first simplex.
Nonetheless, in the critical cases, the massive contraction ap-
proach is slightly more beneficial than the unidirectional progress
used instead of the expansion. This can be attributed to the fact
that this progress avoids premature contraction or distortion of
the simplex.
In the unidirectional progress there is no improvement when
using the quadratic interpolation. Thus, results are similar and in
general, the simple unidirectional progress is preferable.
Therefore it is advisable to use the MSM, with massive con-
traction in the critical cases, improved with a simple unidirectional
progress in the cases (a) when U(D)4 U(R) 4U(S(1))and U(Q2) 4U
(Q1)oU(D). This modification of the sequential Simplex method is
very simple and effective. As in the MSM, no especial mathema-
tical knowledge is required and the calculations involved are tri-
vial and easy to program.
5.1. Robustness of the methods
Finally, optimization shouldn’t be end when obtaining the op-
timum parameters. Results should be complemented, i.e. evalu-
ating each parameter behavior taking its optimum as the central
point of a new assay. Thereafter, the robustness of the method
should be evaluated. The method will be more robust when
showing higher tolerance to variations of the studied parameters
in the region of the found optimum.
648 V. Cerdà et al. / Talanta 148 (2016) 641–648
5.2. Flow analysis applications
In 2010, an overview on the applications of the SIMPLEX al-
gorithm for the optimization of analytical systems based on un-
segmented flow techniques was published by our group
[28,30,34]. The different response functions used for evaluation of
the optimization experimental data were compared and their ad-
vantages and shortcomings and the problematic of using SIMPLEX
with a special focus on real-time application was shortly dis-
cussed. The analytical applications and parameters optimized by
SIMPLEX method were summarized in tabular form.
6. Conclusions
1. In sequential optimization methods, the gradient method is
recommended for functions with several variables and ob-
tainable partial derivatives while the simplex method is re-
commended for functions with several variables and un-
obtainable partial derivatives.
2. It is advisable to start in as close to the optimum as possible.
Methods of approximate calculation can be used to fulfill this
purpose.
3. The gradient method is the best option when possible.
4. Optimization has to be repeated starting from different initial
points to assure that the optimum found is the absolute max-
imum or minimum, and not a local one.
5. Various programs have been developed to refine thermo-
dynamic parameters based on the gradient method.
Acknowledgments
The authors acknowledge financial support from Spanish
Ministry of Economy and Competitiveness (MINECO) through
Project CTQ2013-47461-R co-financed by FEDER funds.
Appendix A. Suplementary Information
Supplementary data associated with this article can be found in the
online version at http://dx.doi.org/10.1016/j.talanta.2015.05.061.
References
[1] P. Gans, Coord. Chem. Rev. 19 (1976) 99–124.
[2] B. Dejaegher, Y.V. Heyden, Sequential optimization methods, Ref. Modul.
Chem. Mol. Sci. Chem. Eng. Compr. Chemom. 1 (2009) 547–575.
[3] L.V. Candioti, M.M. De Zan, M.S. Cámara, H.C. Goicoechea, Experimental design
and multiple response optimization. Using the desirability function in analy-
tical methods development, Talanta 124 (2014) 123–138.
[4] L.A. Sarabia, M.C. Ortiz, Response surface methodology, Ref. Modul. Chem.
Mol. Sci. Chem. Eng. Compr. Chemom. 1 (2009) 345–390.
[5] J.E. Dennis Jr., R.B. Schnabel, Numerical Methods for Unconstrained Optimi-
zation aAnd Nonlinear Equations, Prentice-Hall, 1983.
[6] R. FIetcher, Practical Methods of Optimization, Vol I, Wiley, 1980.
[7] D.M. Himmelblau, Applied Nonlinear Programing, McGraw, New York, 1972.
[8] A.M. Cohén, Análisis Numérico, Reverte, Barcelona, 1977.
[9] P. Gans, A. Sabatini, A. Vacca, Talanta 43 (1996) 1739–1753.
[10] F. Gaizer, A. Puskás, Talanta 28 (1981) 565.
[11] F. Gaizer, A. Puskás, Talanta 28 (1981) 925.
[12] V. Cerdà, R. Jara, Thermochim. Acta 87 (1985) 13.
[13] R. Forteza, V. Cerdá, Talanta 32 (1985) 1159.
[14] H. Chen, J. Zhang, Y. Dang, G. Shu, J. Chem. Pharm. Res. 6 (2014) 612–616.
[15] L. Escuder-Gilabert, Y. Martín-Biosca, S. Sagrado, M.J. Medina-Hernández, J.
Chromatogr. A 1363 (2014) 331–337.
[16] G. Di Nicola, M. Pacetti, F. Polonara, G. Santori, R. Stryjek, J. Chromatogr. A 1190
(2008) 120–126.
[17] K. Selber, F. Nellen, B. Steffen, J. Thömmes, M.-R. Kula, J. Chromatogr. B 743
(2000) 21–30.
[18] A. Herrero, M.C. Ortiz, M.J. Arcos, J. López-Palacios, Analyst 119 (1994)
1585–1592.
[19] P. Petersson, N. Lundell, K.E. Markides, J. Chromatogr. A 623 (1992) 129–137.
[20] D. Wienke, C. Lucasius, G. Kateman, Anal. Chim. Acta 265 (1992) 211–225.
[21] J. Cerdá, V. Cerdá, Cuadernos de Ciencias y Técnicas Ambientales, in:
M. Blanco, V. Cerdà (Eds.), Serie Química Analítica, AEST, Barcelona, 1988.
[22] W. Spendley, G.R. Hext, F.R. Himsworth, Technometrics 4 (1962) 441.
[23] J.A. Nelder, R. Mead, Comput. J. 7 (1965) 308.
[24] M.W. Routh, P.A. Swartz, M.B. Denton, Anal. Chem. 49 (1977) 1422–1428.
[25] P.F.A. van der Wiel, Anal. Chim. Acta 122 (1980) 421–433.
[26] M. Blanco, V. Cerdà, J. Coello, J. Gené, H. Iturriaga, S. Maspoch, Anal. Lett. 25
(1992) 543–560.
[27] V. Cerdà, J.L. Cerdà, Optimization of analytical techniques using the gradient
and simplex methods. 19th International Conference on Flow Injection Ana-
lysis and Related Techniques. Fukuoka, Japan,.
[28] B. Horstkotte, A.T. Sánchez, C.M. Duarte, V. Cerdà, Anal. Chim. Acta 658 (2010)
147.
[29] A. Cladera, C. Tomás, J.M. Estela, V. Cerdà, G. Ramis, Anal. Chim. Acta 282
(1993) 613–623.
[30] M. Blanco, V. Cerdà (Eds.), Temas avanzados de Quimiometría, University of
the Balearic Islands, Palma de Mallorca. Spain, 2007, ISBN: 987-84-8384-006-
1.
[31] H.H. Rosenbrock, Comput. J. 3 (1960) 175.
[32] W.I. Zangwill, Manag. Sci. 13 (1967) 334.
[33] M.J.D. Powell, Non convex minimization calculations and the conjugate gra-
dient method, in: Lecture Notes in Mathematics, vol. 1066, Springer (1984), p.
122–141.
[34] Burkhart Horstkotte, Carlos M. Duarte, Víctor Cerdà, TrAC Trends Anal. Chem.
29 (2010) 1224–1235.