Optimizing using parabolic interpolation

Sama Motasem Armouti

0208546
Computer Engineering Department
The University of Jordan
sma0208546@ju.edu.jo

Abstract—In mathematics, optimization problems ask for 4𝑎 + 2𝑏 + 𝑐 = 8 ()

minimize or maximize a quantity. Having several methods to solve
them, in this report optimizing with parabolic interpolation will be 9𝑎 + 3𝑏 + 𝑐 = 14 ()
discussed.
Keywords—optimization, parabolic interpolation, extremum in matrix form:
values, approximation
1 1 1 𝑎 4
I. PARABOLA
[4 2 1] [𝑏 ] = [ 8 ] ()
Parabola, is a polynomial of degree two, U-shaped 9 3 1 𝑐 14
symmetrical curve. As we know only one straight line is valid
to connect two points, there is only one parabola to connect We’ll solve this system using inverses A-1 * B.
three points. One of these three point is an extremum point
(maximum or minimum). 𝑎 0.5 −1 0.5 4 1
[𝑏 ] = [−2.5 4 −1.5] [ 8 ] = [1] ()
II. PARABOLIC INTERPOLATION 𝑐 3 −3 1 14 2
A. What is parabolic interpolation
The curve is 𝑓(𝑥) = 𝑥 2 + 𝑥 + 2
It is a method, that finds the extremum value of a
continuous function by successively fitting parabolas to a The extreme point of this parabola is the minimum point can
function of n-variables at 1 + (𝑛(𝑛 + 3)⁄2) points, and at be easily computed by setting the derivative of the parabola to
each iteration replacing the "oldest" point with the extremum zero and solving for 𝑥,
of the fitted parabola.
𝑓́ (𝑥) = 2𝑎𝑥 + 𝑏 = 0 ()
B. Advantage of parabolic interpolation
Parabolic interpolation takes the advantage of the fact that Then,
a second-order polynomial over a small interval that bounds
(𝑥e) often provides a good approximation to the shape of f(x) 𝑥 = −𝑏 ⁄2𝑎 ()
near an optimum.
−1
C. Functionality of parabolic interpolation We obtain 𝑥𝑒 = 2
= −0.5
As mentioned above, three points are required to compute
A formula of xe as shown below, x4 is defined as the next
the coefficients of a parabola 𝑓(𝑥) = 𝑎𝑥 2 + 𝑏𝑥 + 𝑐. For approximation for the root.
example, the three coefficients of the parabola that passes
through the points (𝑥1, 𝑓 (𝑥1)), (𝑥2, 𝑓(𝑥2)) and (𝑥3, 𝑓(𝑥3)) are
unique and are the solution for the system of three linear 1 (𝑥2 − 𝑥3)2[𝑓(𝑥2) − 𝑓(𝑥1)] − (𝑥2 − 𝑥1)2 [𝑓(𝑥2) − 𝑓(𝑥3)]
𝑥4 = 𝑥2 +
equations. 2 (𝑥2 − 𝑥1 )[𝑓(𝑥2) − 𝑓(𝑥3)] − (𝑥2 − 𝑥3)[𝑓(𝑥2) − 𝑓(𝑥1)]

𝑓(𝑥1 ) = 𝑎𝑥1 2 + 𝑏𝑥1 + 𝑐 () ()

𝑓(𝑥2 ) = 𝑎𝑥2 2 + 𝑏𝑥2 + 𝑐 () The process will be like for x1, x2, and x3 find x4 and then based
on x2, x3, and x4 find x5, and keep doing the iteration over again
and again until you are close enough to the optimum.
𝑓(𝑥3 ) = 𝑎𝑥3 2 + 𝑏𝑥3 + 𝑐 ()
In this example, for the next iteration we will use the points
or, in matrix form. (2,8), (3,14) and the new point (-0.5,1.75) to solve the new
parabola as the pervious steps.
𝑥1 2 𝑥 1 𝑎 𝑓(𝑥1 ) In MATLAB, start solution with finding a, b, and c for
[𝑥2 2 𝑥 1] [𝑏 ] = [𝑓(𝑥2 )] () your first three points, after writing the matrix of your system
𝑥3 2 𝑥 1 𝑐 𝑓(𝑥3 ) find the inverse using inv function to find the inverse of
matrix, we obtain
Example: Determine the equation of the parabola that inv([0.5 -1 0.5; -2.5 4 -1.5; 3 -3 1])*[4;8;14]
passes throw the points (1,4), (2,8), and (3,14).
Solution: mathematically, write a system:
the solution is 𝑓(𝑥) = 𝑥 2 + 𝑥 + 2, and its associated plot is
𝑎+ 𝑏 + 𝑐=4 () shown below.
f=@(x) x.^2+x+2;
 x=-4:0.01:4;
x1=[-1 4 -3];
plot(x,f(x))
But why not use Matlab to (symbolically) solve the
general formulation,
𝑎𝑥1 2 + 𝑏𝑥1 + 𝑐 − 𝑓(𝑥1 ) = 0
𝑎𝑥2 2 + 𝑏𝑥2 + 𝑐 − 𝑓(𝑥2 ) = 0
𝑎𝑥3 2 + 𝑏𝑥3 + 𝑐− 𝑓(𝑥3 ) = 0
or the parabola coefficients and obtain in a script (interp2_mh)
syms x x1 x2 x3 f1 f2 f3 a b c
[a,b,c]=solve(a*x1^2+b*x1+c-f1,a*x2^2+b*x2+c-
f2,a*x3^2+b*x3+c-f3,a,b,c)
a=(f1*x2-f2*x1-f1*x3+f3*x1+f2*x3-f3*x2)/((x1-
x2)*(x1*x2-x1*x3-x2*x3+x3^2))
b=-(f1*x2^2-f2*x1^2-f1*x3^2+f3*x1^2+f2*x3^2-
f3*x2^2)/((x1-x2)*(x1*x2-x1*x3-x2*x3+x3^2))
c=-(-f3*x1^2*x2+ f2*x1^2*x3+ f3*x1*x2^2-f2*x1*x3^2-
f1*x2^2*x3+f1*x2*x3^2)/((x1-x2)*(x1*x2-x1*x3-
x2*x3+x3^2))
xe=(1/2)*((x2^2-x3^2*f1+x3^2-x1^2)*f2+(x1^2-
x2^2)*f3)/((x2-x3)*f1+(x3-x1)*f2+(x1-x2)*f3)
From which the extreme point 𝑥𝑒 can be solved for (using
Matlab’s solve function) by differentiating the resulting
quadratic function and setting it equal to zero:
pretty(solve(diff(a*x^2+b*x+c,x),x))
The above Matlab results are rewritten more clearly
below.
(𝑥2 −𝑥3 )𝑓(𝑥1 )+(𝑥3 −𝑥1 )𝑓(𝑥2 )+(𝑥1 −𝑥2 )𝑓(𝑥3 )
𝑎=
(𝑥1 −𝑥2 )(𝑥1 𝑥2 −𝑥1 𝑥3 −𝑥2 𝑥3 +𝑥3 2 )
(𝑥3 2 −𝑥2 2 )𝑓(𝑥1 )+(𝑥1 2 −𝑥3 2 )𝑓(𝑥2 )+(𝑥2 2 −𝑥1 2 )𝑓(𝑥3 )
𝑏= (𝑥1 −𝑥2 )(𝑥1 𝑥2 −𝑥1 𝑥3 −𝑥2 𝑥3 +𝑥3 2 )
(𝑥22𝑥3 −𝑥2𝑥32 )𝑓(𝑥1)+(𝑥1𝑥32 −𝑥12 𝑥3)𝑓(𝑥2)+(𝑥12 𝑥2−𝑥1𝑥22 )𝑓(𝑥3)
𝑐=
(𝑥1−𝑥2)(𝑥1𝑥2−𝑥1𝑥3−𝑥2𝑥3 +𝑥32)
1 (𝑥2 − 𝑥3)2[𝑓(𝑥2) − 𝑓(𝑥1)] − (𝑥2 − 𝑥1)2 [𝑓(𝑥2) − 𝑓(𝑥3)]
𝑥𝑒 = 𝑥2 +
2 (𝑥2 − 𝑥1 )[𝑓(𝑥2) − 𝑓(𝑥3)] − (𝑥2 − 𝑥3)[𝑓(𝑥2) − 𝑓(𝑥1)]
()
Given three distinct points as input, then the previous
Matlab script (interp2_mh) can be used to evaluate the
preceding formulas.
D. Parabolic interpolation-based optimization
The shape of a nonlinear function 𝑓(𝑥) in the vicinity of
any of its extreme points is similar to a parabola, this
optimization is based on searching for an extreme points is
similar to a parabola, Parabolic interpolation optimization is
based on searching for an extreme point, 𝑥e , by fitting a
second-degree polynomial (parabola) to the function 𝑓(𝑥) over
a small interval that bounds 𝑥𝑒 . By selecting three points close
to an extreme point (say, two that bound 𝑥𝑒 and a third internal
point), we may employ the method of parabolic interpolation
to approximate the shape of 𝑓(𝑥), and arrive at an estimate for
()
𝑥𝑒 . The process can be repeated in order to improve the
estimate. The following example illustrates the application of
() the parabolic optimization method.
() Example. Use parabolic interpolation to approximate the
minimum of the function 𝑓(𝑥) = 𝑥 2 ⁄10 − 2sin (𝑥) with
initial values x1=0, x2=1 and x3=4.
Solution. Substituting the initial 𝑥 values in the function
𝑓(𝑥) leads to (remember to set your calculator’s mode to
radians) the three interpolation points: (0, 0), (1, −1.5829) and
(4, 3.1136). Next, we use the interp2_mh script to compute the
parabola coefficients and the extreme point estimate:
x=[0 1 4]; f=[0 -1.5829 3.1136];
format short
interp2_mh
Before applying a second iteration, check where the
interpolation parabola 𝑦(𝑥) = 0.7871𝑥 2 − 2.37𝑥 and
compare it to the parabola 𝑓(𝑥) = 𝑥 2 ⁄10 − 2sin (𝑥) within
the interval [-0.5 4.5].
As expected, the two functions pass through the three
interpolation points. More importantly, note how close the
shape of the parabola is to that of 𝑓(𝑥) in the neighborhood of
the minimum point.
() The function value at the estimate of the minimum,𝑥̃𝑒 , is
𝑓(1.5055) = 1.50552 ⁄10 − 2 sin(1.5055) = −1.7691 .
We may now retain the two interpolation points closest
() to ̃𝑥𝑒 (in this case, the initial point (𝑥3, 𝑓(𝑥3 )) is discarded).
Therefore, for the next iteration, we use a new set of
interpolation points: (0, 0), (1, −1.5829) and (1.5055,
() −1.7691).
x=[0 1 1.5055]; f=[0,-1.5829 -1.7691];
interp2_mh
The new estimate of the minimum point, 𝑥̃𝑒 = 1.4810, is
closer to the true minimum 𝑥̃𝑒 = 1.4275 (obtained using any
of the optimization methods of Lecture 13). The function
value at the estimate of the new minimum, 𝑥̃𝑒 = 1.4810, is
𝑓(1.4810) = −1.7726. The third iteration will use the
interpolation points (1, −1.5829), (1.4810, −1.7726) and
(1.5055, −1.7691) to obtain the parabola.
x=[1 1.4810 1.5055]; f=[-1.5829 -1.7726 -1.7691];
interp2_mh
and leads to the estimate 𝑥̃𝑒 = 1.4260, which is quickly
converging to the true minimum at 1.4275. The following
(green) plot depicts the interpolating parabola,
𝑦(𝑥) = 1.0628𝑥 2 − 3.0312𝑥 + 0.3855 ()
REFERENCES
[1] lecture14.pdf
[2] https://people.sc.fsu.edu/~jburkardt/classes/math2070_2019/optimizat
ion_parabolic/optimization_parabolic.pdf
[3] https://en.wikipedia.org/wiki/Parabola
[4] https://en.wikipedia.org/wiki/Successive_parabolic_interpolation
[5] https://www.youtube.com/watch?v=noczK51tOgE
[6] https://www.youtube.com/watch?v=zz3SdE4PN_w
[7] https://www.youtube.com/watch?v=yvkS-griOkg