PROCESS OPTIMIZATION

TOPICS
Stationary Optimization
Different Approach
Numerical Approach
Linear and Non-Linear Programming
Trajectory Optimization
Single and multivariable techniques
Constrained optimization techniques
Direct events
Teaching
Module 1: Stationary optimization
Lecture 1: Different Approach
Lecture 2: Numerical Approach
Lecture 3: Linear Programming
Lecture 4: Non-Linear Programming
Module 2: Trajectory Optimization
Lecture 5: Single Search Techniques
Lecture 6: Multivariable Search Techniques
Lecture 7: Constrained Optimization Techniques
Lecture 8: Discreet Events.

1
MODULE 1: STATIONARY OPTIMIZATION
LECTURE 1: DIFFERENTIAL APPROACH
1.0 INTRODUCTION
Optimization can be viewed from two perspectives. It can mean the maximum of a
dependent variable – e.g. the maximization of the product obtained from a chemical process for a
given capital outlay and operating conditions. On the other hand it could involve the
determination of the minimum value of a variable given a specific target variable such as the
determination of a minimum investment in terms of equipment services, labour etc. for a
specified production rate.
1.1 TYPES OF OPTIMIZATION
There are two types, namely the static optimization and the dynamic optimization.
1.1.1 STATIC OPTIMIZATION
This as its objective the establishment of the most variable steady state operation
conditions of the process. The operation variable includes temperature flow rate, pressures, size
of equipment etc. The essence is to obtain the most favourable operating condition.
1.1.2 DYNAMIC OPTIMIZATION
Normally static optimization establishes the normal conditions and does not account for
fluctuation in the variables. Dynamic optimization establishes procedure for optimizing the
fluctuations about the normal conditions.

Addendum
i. Other types – operational research methods.
ii. Optimization involves finding the best way of designing or operating a system.
iii. Optimization is applied to
- allocation of resources
- investment decisions
- use of money, men, time
- making design decisions
- type, size, layout of plans
- making operating decisions among set of operation parameters.

2
1.2 DIFFERENTIAL APPROACH
This method involves determing the maxima (or minimum) of an equation containing one
dependent variable and a number of independent variables. The procedures are to differentiate
the dependent variable with respect to the independent variable and equating the derivatives to
zero to obtain the maximum or minimum. This method falls into the realm of the classical
methods.
For book purpose:- The procedure involves (i), (ii), (iii).
1.2.1 TWO VARIABLES
This is of the form:-
y = f (x)
Where y is the dependent variable and x is the independent variable. To optimize the function,
we need to find dy and d2y . dy is equated
dx dx2 dx
to zero to determine the point which the minimum or maximum occurs. The nature of this point
(is whether minimum or maximum ) or point of inflexion) is determined by substituting into d2y
the values of
dx2
x obtained when dy is equated to zero
Note: If value of y is position, as that value of x in minimum and y “ is regular, y is maximum.
Example 1.1
If y = 3x3 – 6x2 – 6
Determine the value of x that maximizes or minimizes y.
Solution
y = 3x3 – 6x2 –6
y1 = 9x2 – 12x
y11 = 18x – 12
At the minimum or maximum
y1= 9x2 – 12x = 0
i.e. 3x (3x-4) = 0
This implies x = 0 or x = 4
3

3
Nature of Point
For x = 0
y” = -12
 at x = 0 y is a maximum
for x = 4/3,
y” = 24 – 12 = 12
at x = 4/3 y is a minimum
1.2.2 THREE VARIABLE
These involve a function of the form = f (x,y)……………(2)
To obtain the minimum or maximum of the function in terms of the two independent variables,
we need to determine zx, zy, zxx, zyy znd zxy. Solution of the simultaneous equation involving
zx and zy gives the likely points at which the minimum or maximum value of the function z
exists. To determine the nature of the points, we apply the following rules:
If Zxx Zyy-Z2xy > 0 and Zxx < 0 – maximum
“ “ “ “ > 0 and Zxx >0 minimum
“ “ “ “ <0 saddle point.
Addendum
Saddle point
Example 1.2
If Z = x3 + 3xy2 – 3x2 – 3y2+4
Find the values of x and y at which z has minimum or maxima value and distinguishes them.
Solution
Z = x3+3xy2-3x2-3y2+4……………………………(1)
Zx = 3x2+3y2 – 6x ………………………………..(2)
Zy = 6xy – 6y …………………………………….(3)
Zxx = 6x – 6 ………………………………………(4)
Zyy = 6x – 6 ……………………………………….(5)
Solving (1) and (2) simultaneously gives
x = 0, y = 0
x = 1, = 1
x = 2, y = 0
4
 x = 1, y = -1
Addendum
6xy – 6y = 0 ………………….(1)
3x2 + 3y2-6x = 0 ……………..(2)
From (1) 6y (x-1) = 0 ………………..(3)
:- 6y = 0 or x – 1 = 0
 y = 0, x = 1 ………………(4)
From (4) when y = 0 (2) become
3x2 – 6x = 0
3x (x – 2) = 0  x = 0, x = 2
From (2) when x = 1 (2) becomes
3 + 3y2 – 6 = 0
i.e 3y2 – 3 = 0  3 (y2 – 1) = 0
i.e y2 = 1  y =  1
Hence complete solution is
y x
0, 0
2 0
1 1
1 -1
Nature of points
x = 0, y = 0 Zxx Zyy – Z2xy >0 Zxx <0 – Maximum
x = 1, y = 1 Zxx Zyy – Z2xy <0 saddle point
x = 2, y = 0 Zxx Zyy – Z2xy >0 Zxx >0 minimum
x = 1, y = -1 Zxx Zyy – Z2xy <0 saddle point
Addendum
Hence with regard to optimization, the first condition that must be examined is whether Zxx Zyy
– Z2xy – Z2xy >0.

5
1.2.3 MORE THAN 3 VARIABLES
As the number of variables becomes larger, it becomes very difficult to use the usual
classical method of differential calculus to optimize the function. Besides, the optimum values
are usually curtailed because in solving the equations, one or two variables ax usually specified
arbitrarily.
QUIZ 1
ASSIGNMENT 1
Solve Zx - and Zy = 0 to obtain the x and y
ASSIGNMENT 2
Determine the nature of the points from the values of x and y obtained from the
simultaneous solution of zx = 0 and zy = 0.
TUTORIAL QUESTIONS
1. Given z = 4x3 – 3x2 + 4xy – 5xy2+6
Find the values of x and y which z has a maximum or minimum value and determine the
nature of these points.
2. A chemical process makes use of certain variables. Formulate possibly objective
function (or mathematical model) for a condenser.
(Hint: Function – cooling; heat exchange, pressure, temperature composition, and
material properties – need for interaction of variable).
3. Given the following model which describes the hints flow rates into and out of a process
find the optimum values of stream A.
Assume steady state operation.

A D Bstream Cstream Dstream

a. 10.2 20.4 75.2
b. 11.3 18.4 77.9
B c. 10.8 19.3 75.8
d. 11.4 18.7 76.4

6
LECTURE 2: NUMERICAL APPROACH
Reference See also pg 14
(1) Any good book on optimization research
(2) Optimization of chemical processes
T.F. Edgar & D.M. Aimmeblan.
McGraw Hill International Edition, Chem. E.g. Series.

LECTURE 3.1: LINEAR PROGRAMMING

3.0 INTRODUCTION
For classes of problems in which this type of optimization technique is used, a linear
objective function is formed subject to restrictive conditions or constraints. The constraints may
be linear equality or inequality or combination of both. The constraints define a region called the
feasible area and the function is optimized by searching the extreme points of the feasible area.
All points within the feasible area are feasible points. The solution of a linear objective function
through the use of certain theorems and method, which can be programmed, constitutes linear
programming.
Addendum
Formally – linear programming is the problem of finding the optimum of a linear
function of a set of non- negative variable subject to a series of linear constraints e.g. maximize
(or minimize) y = cj xj J = 1
Subject to
 aij xj  bi I i = 1,2… (m)
xj  0 -
It is characterized by
a. The objective function which is a linear function of the independent variable x1,x2 = xn
b. The independent variables are constrained by a vet of linear inequalities.
Ref: (1) G.B. Dantx of – Linear Programming and Extension
Prunceton University Prem (1963)
(3) G. Handlet “Linear Programming
Addision – Wesley – Ready Class (196)

7
Many methods abound for solving linear programming problems because surprising large
number of practical problems fall into this type. The most widely used method is the SIMPLEX
METHOD. Other are Transportation Problem Algorithm, Stepping Stone Algorithm, Integer
Programming and Assignment Problems.
Ref: Transportation Algorithm Problem
G.B. Dantzif Linear Programming Extension.

Integer Programming
P.G. Moore & S.D. Hodges
Programming for optimal decisions
Penguin – 1974.
3.1 Statement of two dimension linear programming problem (2-DLP).

Maximize or minimize
Z = a1x1 + q2x2 …….+ cpxp ………………………..3.1
Subject to constraints
xi  0 i = 1, ………….P…………………………….3.2
and
b11x1 + b12x2 + bIP xP {  = } c1……….3.3
b21x1 + b22x2 + b2pxp { = } c 2……….3.4
bm1x1 + bm2x2 +…bmpcp { = }cm……….3.5
Where bij, ai, ci are known constants. The constriant equations can take only one sign  or = or
. The function z is called the OBJECTIVE FUNCTION.
Both the objective function and the constants are both linear in xi (i.e there is no xi with a power
more than one).
3.1.1 Graphical Solution to Z – D, Linear Programming Problem
Solution to a 2-D, LP problem can be obtained graphically provided the constraints can
be plotted on a graph. Since the problems involve two variables, this can be acomplished on an x
– y plot. However, where the number of variables is more than two, it becomes relatively
difficult to plot the function. The procedure will be shown by an example.

8
Example 3.1
Maximize
y = 2x1 + 4x2 ……………………..1
Subject to
3x1 + 4x2  1700 ………………(2)
0.2x1+0.5x2  160 ……………..(3)
and x1,x2,  0 …………………………(4)
SOLUTION
Equation (3) can be written as
2x1 +5x2  1600 ………………(5)
Step 1
Plot the constraints by writing equation (2) a no @ as equality equation
From (2)
If x1 = 0, 4x2 = 1700  x2 = 425
If x2 = 0, 3x1 = 1700  x1 = 5662/3
From equation (2), the following set of points are obtained (0,425),(5662/3,0).
Similarly from equation (5)
If x1 = 0 5x2 = 1600  x2 = 320
If x2 = 0 2x1 = 1600  x1 = 800
Hence another sets of points is (0,320), (800,0).
Hence the constraints can be plotted as straight lines as follow:

Fig.3.1: Graphical solution of example 3.1

9
The latched region bounded by the constrained lines and the abscissa (x) and ordinate lines
constitutes the feasible region. Any point within the feasible is a feasible solution of the
problem. To obtain the point that constitutes the optimal solution, first a family of parallel lines
(called contour lines) of the objectives function is constructed as follows: From the objective
function.
y = 2x1 + 4xn
4x2 = y – 2x1
x2 = y - 1 x1 = 0.25y – 0.5x1 ………………(6)
4 2
The contour lines are therefore lines will slope = -0.5 and intercept 0.25y. In real terms, they can
be obtained using two methods.
Method 1
This involve using one pair of x1, x2 value to draw the first line and then constructing other
parallel lines from it.
e.g. if x1 = 400, then x2 = 0.5x1 = 200.
A line is drawn through (400,0) (0, 200). And other lines are constructed parallel to this line.
Method 2
A series of pairs of x1 and x2 are used (using the above method) and lines are drawn through
them.
e.g x1 x2
400 200
600 300
700 350
800 400
The optimal solution will be obtained from the points where the contour lines intersect any of the
point that define the feasible region. In this case of this example, since we are looking for
maximum y, it correspons to point B. On the graph where x1 = 300, x2 = 200 giving
y = 2x1 + 4x2 = 2 (300) + 4 (200) = 600 +800 = 1400
From the graphical solution, it can be noted that the OPTIMAL SOLUTION OCCURS
AT ONE OF THE CORNERS OF A FEASIBLES REGION and precisely and the intersection of

10
any two constarinst. All the points corresponding to the intersection of any two constraints are
called BASIC SOLUTIONS.
The basic solution that lies inside the feasible region are called basic feasible solution and
the optimal solution is one of the basic feasible solutions. It should also be noted that the basic
feasible solutions are located at the corners of the feasible region. Hence to optimize an objective
function, the graphical solution has shown that all one needs to do is to evaluate the objective
function at all the basic feasible solutions.
3.1.2 VARIATIONS TO GRAPHICAL SOLUTIONS
The above example is a very simple one in that only one optimal solution exists. This is
often the case in real life optimization problem. However, they are problems which can pose
difficulties and therefore require variations from the above.
3.1.2.1 CONTOUR LINES PARALLEL TO THE BOUNDARY OF THE
FEASIBLE REGION.
When this occur, then more than one optimal solution exist.
Example 3.2
Maximize y = 2x1+4x2
Subject to x1+5x2  5
x1+2x2  4
x1,x2  0

It will be noticed from the graph that the z – line (one of the contour lines is parallel to one of the
constraints line (x1-12x2 = 4).

3.1.2.2 OPTIMAL SOLUTION NOT BOUNDED

When the optimal solution is not bounded, then z can be increased indefinitely.
Example 3.3
Maximixe y = x1 + x2
11
Subject to x1 – x2  1
x2  5
x1,x2  0
The constriant equation are
x1 – x2 = 1
x2 = 5
Objective x2 = y – x,
The graphical solution looks thus.
From the contopur lines, it could be seen that linear increase indefinitely in the bounded region.
3.1.2.3 SITUATION WHEN NO SOLUTION EXISTS
This situation is encountered when there is no feasible region.
Example 3.4
Maximize y = 2x1+4x2
Subject to x1+x2  4
2x1+x2  10
x1,x2  0
The graphical solution looks thus.

There is no feasible region because the boundary are diametrically opposite to one another.
Therefore there can be no solution.

12
Addendum
The constraints equation for example 3.4 are
x1 + x2 = 4
2x1+x2 = 10
Objective x2 = y - 1 x,
4 2
3.2 SIMPLEX METHOD
Graphical solutions possible when the objective function ivolves only two or less
independent variables. Beyond two variables, the graphical method becomes inconvenient or
utterly impossible. Other methods have to be used. Care of these methods is the simplex
method. However it must be noted that the cardinal principle demonstrated by the graphical
method still holds. That is, the optimum solution will always lie at a vertx of a feasible region
and not at an interior point within the region. This optimum solution also corresponds to one of
the basic feasible solutions.
3.2.1 ALGEBRAIC STEPS OF SIMPLEX METHOD
The procedure necessary to obtain the optimal solution of an objective function can best
be demonstrated using an example.
Example 3.5
Source – mathematical methods in chemical Engineering by V.G. Jenseen and G.V. Jeffreys.
Find the maximum value of the function.
P = 3x, + x2 +2x3……………………..(1)
Subject to
x1 – x2  1 …………………………...(2)
4x, + 2x3 – 3x3  10…………………..(3)
5x, + x2 + 3x3  14…………………….(4)
x1, x2, x3  0 …………………………...(5)
Step 1
Convert the inequality constraints to linear equation by introducing what are called „slack
variables“ on the less than sides of the inequalites when this is done,
2. becomes x1 – x2 + x4 = 1 …………………….(6)
3. becomes 4x1 + 2x2 – 3x3 + x5 =10 …………..(7)

13
 4. becomes 5x1+x2+3x3+x6 = 14……………..…(8)
The Linear program therefore becomes maximize.
P = 3x1+x2+x3 ……………………..(9)
Subject to
x1 – x2 + x4 = 1 …………………….(10)
4x1+2x2-3x3+x5 = 10……………….(11)
5x1+x2+3x3+x6 = 14…………………(12)
with x1,x2…………..x6 being non-negative.
Addendum
The problem has 3 constraint via 6 variable. To reduce it to a system of 3 equation
(corresponding to the number of constraints) in 3 variables, we set (6-3) variables equals to zero.
The variables put to zero are called the NON-BASIC variables while the othere are called the
BASIC variables. The basic variables form the BASIC of the solution. Generally for a problem
withj m constraints in P variables, there are pcm = p! basic solution.
M! (p-m)!
Step 2
Choose the most obvious basic feasible solutions. Generally this is given by x i………xn = 0,
hence when x1 = 0, x2 = 0 and x3 = 0, the P = 0, x4 = 1,
x5 = 10, x6 = 14.
The variables x1, x2, x3 are called the non- basic variables while x4, x5, x6 are called the basic
variables.
Addendum
Rearrange the problem into standard form as follows.
P = 3x1+x2+2x3
x1 – x2 ………………+x4 = 1 ………………..(10)
4x1+2x2 – 3x3 ……….+x5 = 10………………..(12)
5x1+x2+3x3……………+x6 = 14……………….(13)
Step 3
Choose the non-basic variable with the suggest positive coefficient (that is one while maximize
the objective function most when increased). Let the other have values equal to zero. For this
problem choose x1

14
Step 4
Determine the maximum value to which the non-basic variable (choosen in step3) must be
increased for this problem, since x4–x6 must be non-negative, x1 can only be increased until x4,
x5 and x6 are zero.
From (10) when x4 = 0 , x1 = 1
From (11) when x5 = 0 , x1 = 10 = 2.5
5
From(12) when x6 = 0, x1 = 14 = 2.8
5
So x1 cannot be increased beyond 1 without violating the non-negativity constraint in x4,x5, n = 8
make x1 = 1. This means equation (10) is choosen. Eluminate x1 from the other equation to give.
4x2 + 2x3 – 3x4 = p-3…………..(13)
x1 – x2 ………….+x4 = 1……………..(14)
6x2 – 3x3-4x4+x5 = 6……………..(15)
6x2+3x3-5x4…….+x6 = 9……………..(16)
Hence the new objectives functions
P = 3+4x2+2x3-3x4………………..(17)
Addendum
To eliminate x1 from (9) using (10)
x1-x2 +x4 = 1…………………..(1)
-3x1+3x2 -3x4 = -3………………….(2)
State (a) as 3x1+x2+2x3 = p ………………….(3)
(2)+(3) 4x2+2x3-3x4 =-3
Similarly to eliminate x1 from (10) using (11)
(10) x1-x2+x4 = 1…………………….(1)
x-4 -4x1+4x2-4x4 = -4…………..(2)
(11) 4x1+2x2-3x3+x5 = 10………………. (3)
(2) + (3) 6x2-3x3-4x4+x5 = 6
Step 5
Repeat steps 2 to 4 for the new set of equation 14 to 17
x2 the end of step 4,
P = 3+4x2+3x3-3x4

15
Which means, x2 has the largest positive coefficient.
From equation (14) when x1 = 0,x2 = 1
From equation (15), when x3 = 0, x2 = 1
From equation (16), when x3 = 0, x2 = 1.5
Therefore as arqued in step 4, equation (15) is chosen as the basic of eliminating x 2 from other
equation. Making x2 equal to unity equation (15) becomes
x2 - x3 – 2 x4+ 1 x5 = 1……………………..(18)
3 6
Using equation (18) to eliminate x2 from equations (13), (14) and (16) gives a new set of
equations.
4x3 - 1 x4- 2 x5 = p – 7 …………………...(19)
3 3
From equation (15), when x3 = 0, x2 = 1
From equation (16), when x3 = 0, x2 = 1.5
Therefore as argued in step 4, equation (15) is chosen as the basic of eliminating x2 from other
equations. Making x2 equal to unity equation (15) becomes.
x2 – 1 x3 – 2 x4 + 1 x5 = 1 ……………………..(18)
2 3 6
Using equation (18) to eliminate x2 from equations (13), (14) and (16) gives a new set of
equations.
4x3 – 1x4 – 2x5 = p – 7 …………………(10)
3 3
x1-1x3+1x4+1x5 = 2…………………… (20)
2 3 6
x2-1x3-2x4+1x5 = 1 …………………..(21)
2 3 6
6x3 – x4 – x5 +x6 = 3…………………..(22)

Therefore the new objective function is given by:

P = 7+4x3 – 1x4 –2x5………………..…(23)
2 3
Step 6
Repeat step 2 to 4 again for the new set of equation.
From equation (23), x3 has the biggest positive coefficient .
From equation (20), when x1 = 0, x3 = -4
From equation (21), when x2 = 0, x3 = -2

16
From equation (22), when x4 = 0, x3 = 
Therefore equation (22) is chosen to eliminate x3 from other equation. This equation is first
converted (normalized) so that x3 has a value of unit to give.
x3-1x4-1x5+1x6 = 1…………………..(24)
6 6 6 2
Equation (24) is then used to eliminate x3 from equation (19) (2) and (21). This procedure gives
rise to a net set of equations.
1x4 - 2 x6 = p – 9 ……………….(25)
3 3
x1 +1x4 + 1 x5 + 1x6 = 9 ………………………(26)
4 12 4
x2 -3 x4 + 1 x5 + 1 x6 = 5……………………….(27)
4 12 12
x3 -1x4 – 1 x5 + 1 x6 = 1 ……………………….(28)
6 6 6 2
hence the new objective function is
p = 9 +1 x4 – 2 x6 ………………………..…(29)
3 3
Step 7
Again, step 2 to 4 are repeated from equation (29), x4 has the biggest positive coefficient, hence
x4 has to be eliminated.
From equation (26), when x1 = 0, x4 = 9
From equation (27), when x2 = 0, x4 = -5
3
From equation (28) , when x3 = 0, x4 = -3
Therefore equation (26) is chosen as the basic equation for the elimination. Normalizing this
equation. Gives
4x1 +x4+1x5+1x6 = 9………………………….(30)
3 3
Equation (30) is used to eliminate x4 ffrom equation (25), (27) and (28). This procedure gives
rise to a new set of equations.
-4 x1 -1 x5 – 7 x6 = p – 12 ………………………..(31)
3 9 9

4x1 +x4 +1 x5 + 1x6 = 9 ………………….(32)

3 3

17
3x1 +x2 +1 x5 + 1 x6 = 8 …………………………..(33)
3 3

x3 -1 x5 + 2 x6 = 2…………………………..(34)
3 9
Therefore the new objective function is
P = 12 – 4 x1 – 1 x5 – 7 x6 ………………………….………(35)
3 9 9

Since all the coefficient in the objective function (raw) are negative, the optimum solution has
been obtained. This is given by
x4 = 9 ………..(36) from equation (32) if x1 = 0, x5 = 0 and x6 = 0
x2 = 8 f………..(37) from eqation (33) if x1 = 0, x5 = 0, x6 = 0
x3 = 2…………(38) from equation(34) if x5 = 0, x6 = 0
Hence p = 12
3.2.2 SOLVING BY SIMPLEX TABLEAU
The simplex m.tableau sets out a neatway of carrying out the iterative process enuciated
in 3.2.1.
3.2.2.1 PROCEDURE FOR SIMPLEX TABLEAU
Step 2
Construct the first feasible tableau from the standard form of the problem. The standard form is
given with the decision variables set equal to zero. With respect to the example, then
corresponds to when
x1 = 0, x2 = 0, x3 = 0, which implies y = 0
So the first tableau, designated tableau 1 looks thus.
Tableau 1
Basic x1 x2 x3 x4 x5 x6 Value
X4 1 -1 0 1 0 0 1pivotal row
X5 4 2 -3 0 1 0 10
X6 5 1 3 0 0 1 14
Yp 3 1 2 0 0 0 0
Pivotal
Column
18
Step 2
Select the variables that comes into the basis. To do this, the pivotal column must first of all be
identified. It is the column with the biggest positive value of the coefficients of the variables for
this example in the objective function row (p row for this exampled). The variables
corresponding to this column represents the variable that comes into the basis for elimination.
For this example.
The pivotal column is column with coefficient 3 and the variable is x. The next thing is
to determine the pivotal row. It is determined by diving the entry in a value column by the
corresponding value of the variable of the pivotal column for a given row. The row with the
lowest positive quatient (ratio) gives the pivotal row with respect to the example.
For row 1 quotient or ratio = 1/1 = 1
For row 1 quotient or ratio = 10/4 = 2.5
For row 1 quotient or ratio = 14/4 = 2.8
Since the quotient for row is the least positive the pivotal row is row 1.
Step 3
Convert the coefficient of the decision variable in the pivotl row to unity (if it is not already so)
by dividing all the coefficient in the row by the value of the coefficient in the decision variable.
For this example, the coefficient already unity.
Step 4
Construct 2nd tableau by eliminating the decision variable from the other rows. For this example,
the following tableau results.
Table 2
Basic x1 x2 x3 x4 x5 x6 Value
x1 1 -1 0 0 0 0 1
x5 0 6 -3 -4 1 0 6 pivotal row
x6 0 6 3 -5 0 1 9

P 0 4 2 -3 0 0 -3
Pivotal
Column

19
Addendum
To eliminate x1from the 2nd row, multiply elements of row 1 by –4 and add the elements to that
of row 2

-4 +4 -4 0 0 0 0-4
9 2 -3 1 0 0 10
0 6 -3 -4 1 0 6

Similarly to eliminate x1 from the zero row,multiply row 1 by –5 and the elemnt to that of rows 3
i.e -5 5 0 -5 0 0 -5
5 1 3 0 0 1 14
0 6 3 -5 0 1 9
and for row
Row x3 -3 +3 0 -3 0 0 -3
3 1 2 0 0 0 0
0 4 2 -3 0 0 -3
Step 5
Repeat step 2 to 4 until there is no longer any positive coeficient in the objective function (y row
for this example).
From Tableau 2, the pivotal column is one corresponding to where coefficient is 4, so that x2 is
the new basis. From he tableau .
For row 1, quatient = 1/1-1 = -1
For row 2, quatient = 6/6 =1
For row 3, quatient = 9/6 = 1.5
 Row 2 is the new pivotal row, which is use in eliminating the other rows when normalized (i.e
the coefficient of x2 in the pivotal row made equal to unity) and used to eliminated the other,
Tableau 3 result.

20
Tableau 3
Basic x1 x2 x3 x4 x5 x6 Value
x1 1 0 - 1/3 1/6 0 2
x2 0 1 - -2/3 1/6 0 1
x6 0 0 6 -1 -1 1 3 vipotal row
P 0 0 4 -1/3 -2/3 0 -7
Pivotal
Column
Step 6
Steps 2 to 4 is repeated to obtain tableau 4.
x3 comes into the basis because the pivotal column corresponds to where the coefficient of the
objective row is 4. From tableau 3,
For row 1. quotient = 2/- = -4
For row 2. quotient = 1/- = -2
For row 3. quotient = 3/6 = 
 Row row 3 is the new pivotal row. This row is normalized and used to eliminate other rows.
Tableau 4 results
Tableau 4
Basic x1 x2 X3 x4 x5 x6 Value
x1 1 0 0 ¼ 1/12 1/12 9/4 pivotal row
x2 0 1 0 -3/4 1/12 1/12 5/4
x6 0 0 1 -1/6 -1/6 1/6 
P 0 0 4 1/3 0 -2/3 -9
Pivotal
Column
Step 7
Again, step 2, to 4 is repeated to obtain Tableau 5.
x4 comes into the basis because the pivotal columns. Corresponds to column where the
coefficient of the objective function is 1/3.
From Tableau 4

21
 For row 1. Quotient is = 9/4/9/1 = 9
For row 2. quotient = 5/4/-3/4 = -5/3
For row 3. quotient = /-1/6 = -3
 Row 1 is chosen to the new pivotal row. It is first normalized and then used to eliminate other
rows. Tableau 5 results
Tableau 5
Basic x1 x2 x3 x4 x5 x6 Value
x4 4 0 0 1 1/3 1/3 9
x2 3 1 0 0 1/3 1/3 8
x3 0 0 1 0 -1/9 2/9 2
-1/9 -7/9
P -4/3 0 0 0 -1/9 -7/9 -12
At the end of tableau 5, no coefficient in the objective row has a positive value. Therefore the
optimum solution has been reached. This is given by:
x2 = 8
x3 = 2
x4 = 9
x1 = 0 and p = 12
The solution of p can also be verified from
P = 3x1 + x2 + 2x3 = 8+4 = 12
3.2.3 PROBLEM INVOLVING MINIMIZING
So far we have dealt with a problem in which the objective function has to be maximized.
There are thousands of real – life problem where the objective function must be minimized.
Such problem can be tacled in two ways.
3.2.3.1 METHOD 1
The problem is recast into the standard form (i.e involving maximizing). The usual
solution is then applied.

22
Example 3.6
Minimize:
y = -3x1 – 4x2
Subject to
4x1+3x2  14
3x1+5x2  19.
Solution
Recast the problem as follows:
Objective function: maximize y = -y = 3x1+4x2
Constrained Equations
4x1+3x2+x3 = 14
3x1+5x2+x4 = 19
The manual method is then applied . The solution is to be attempted as a tutorial or assignment.
Addendum
Maximize y = 3x1+4x2
Subject to 4x1+3x2+x3 = 14
3x1+5x2+x4 = 19
Table 1
Basic x1 x2 X3 x4 Value
x3 4 3 1 0 14
x4 3 5 0 1 19

y 3 4 0 0 0

3.2.3.2 METHOD 2
The problem is regarded as being in the standard form as it is i.e
Minimize y = -3x1-4x2
Subject to:
4x1+3x2+x3 = 14
3x1+5x2+x4 = 19
23
However, the following points must be observed.
1. The pivot column corresponds to that which has the biggest negative value in the
objective row.
2. The optimum solution is obtained when all the coefficient in the objective row are
positve.
3.2.4 PROBLEM INVOLVING MIXTURE OF  AND  CONSSTRAINTS
Slack variables are usually introduced on the less than side of the constraint equation. If
a constraint equation is of  type, the slack variable will be in the corresponding measure.
However if the constraint equation is of  type the slack variable will be negative in the
corresponding linear equation
This means that the slack variable will have a coefficient of –1 when the first tableau is
formed. To resolve the latter problem, an auxillary (or sometimes called artificial variable is
introduced into the constraint equation and a second linear programming that resulst in first of all
solved. The objective function of this second linear programming problem is called the
auxilliary (artificial) objective function. It is made up of the auxilliary variables which must be
zero if the original problem has a feasible solution.
Example 3.7
Maximize y = 5x1+10x2
Subject to:
x1 + x2  1
14 7

x1 + x2  1
7 17
x1+x2  8
2x1+x2  2
SOLUTION
Convert the problem into standard form by introducing slack and auxilliary variables.
Maximize y = 5x1+10x2………………..(1)
Subject to x1 + x2 + x3 = 1………………(2)
x1 + x2 +x4 = 1…………….(3)
24
 7 12
x1+x2+x5 = 8……………….(4)
2x1+x2-x6+x7 = 2…………..(5)
z = x2 ……………………..…(6)
(5)+(6) gives z-2 = - 2x1-x2+x6…….….(7)
In equation 5, x6 is slack variable while x7 is the auxilliary variable. The auxilliary objective
function is obtained from (5) +(6) i.e z-2 = - 2x1-x2+x6.
Tableau 1
Basis x1 x2 x3 X4 X5 x6 x7 Value
x3 1/14 1/7 1 0 0 0 0 1 (14)
x4 1/7 1/12 0 1 0 0 0 1(7)
x5 1 1 0 0 1 0 0 8 (8)
x7 2 1 0 0 0 -1 1 1 Pivotal row
-y 5 10 0 0 0 0 0 0
z -2 -1 0 0 0 1 0 -2
P.V
Tableau 2
Basis x1 x2 x3 X4 X5 x6 x7 Value
x3 0 3/28 1 0 0 1/28 -1/28 27/28
x4 0 1/84 0 1 0 1/14 -1/14 13/14
x5 0 1/2 0 0 1 1/2 -1/2 15/16
X1 1  0 0 0 -  1
y 0 15/2 0 0 0 5/12 -5/12 -5
z 0 0 0 0 0 0 1 0

Since there is no longer any negative coefficient in the auxilliary objective row (at the end of
tableau 2, the auxilliary objective function has been minimized. The z row and x7 column can
therefore be removed to give

25
 Tableau 3
Basis x1 x2 x3 X4 X5 x6 x7 Value
x3 0 3/28 1 0 0 1/28 13/14 -26/3
x4 0 1/84 0 1 0 1/14 6/7 -2
x5 0  0 0 1  7 14

X1 1  0 0 0 - 1 2 pivotal row
Y 0 15/2 0 0 0 5/2 -5 -5
Pivotal
Column

Tableau 4
Basis x1 x2 x3 x4 X5 x6 x7 Value
X3 -3/14 0 1 0 0 1/7 5/7 -5 pivotal
row
X4 -1/42 0 0 0 1 0 1/12 -10
X5 -1 0 0 0 0 1 1 6
X1 2 1 0 0 0 -1 2
Y -15 0 0 0 0 10 -20
P.C
Tableau 5
Basic X1 x2 X3 X4 x5 x6 Value
X6 -3/2 0 7 0 0 1 5
X4 17/168 0 -7/12 1 0 0 70/17
X5 ½ 0 -7 0 1 0 2 pivotal row
X2 ½ 1 7 0 0 0 14

Y 0 0 -7 0 0 0 -70
From tableau 5,
x2 = 7
26
 x1 = 0
 y = 10 x 7 = 70
If we allow the coefficient 0 of x1 in tableau 5 to initiate another cycle, we obtain tableau 6
below.
Tableau 6
Basic X1 x2 X3 X4 x5 x6 Value
X6 0 0 -14 0 3 1 8
X4 0 0 5/6 1 -17/84 0 3/14
X1 1 0 -14 0 2 0 2
X2 0 1 14 0 -1 0 6

Y 0 0 -70 0 0 0 -70

From Tableau 6,
x1 = 2, x2 = 6
Giving y = (5x2) + (10x6) = 10+60 = 70
PROBLEMS ASSOCIATED WITH TABLEAU AND IMPLICATIONS.
As was demonstrated using graphical solution, the following problems may be
encountered in the tableau.
1. As shown in example 3.7, the objectives row of tableau may contain only zero and
negative values( for maximizing problem). This mean that more than one optimal
solution is possible (as shown by tableau 5 and 6, though the same value of y is returned.
2. Degeneracy may occur. This is a situation where two rows of a pivotal column produce
the same ratio or value. It seldom occurs in real problem though.
3. Elements in the pivotal column are zero or negative. This is a case of unbounded
solution because the non-basis variable could be increased indefinitely without making
any of the current basic variable negative.
3.2.6 DUALITY IN LINEAR PROGRAMMING
This means that for each LP problem, called a prumal LP problem, there exist a related
problem called its Dual. Both the prumal and dual has the same optimal value (solution).

27
 To obtain the dual of a premal LP problem, it is essential that there is only one type of
constraints. That is all are either  or . The procedure required will be shown by an example.

Example 3.8
Y = a1x1+a2x2+…………………..+anxn (n variable)
Subject to
b11x1+b12x2+……………….b1nxn  61
bm1x1+ bm2x2+…………….+bmnxn  cm
The dual is obtained as follows:
1. Transform the n x m matrix of coefficient bij in the prumal to mxn matrix of coefficient
for the dual.
2. Swap a„s with c„s
3. Swap the type of constraints.
Hence the dual of the above example will be.
Munimize y1 = Gz, +………………………..CmZm
Subject to:
b11z1+b2,z2………………….+bm1,zm  a1
b12z,+b22z2……………………+bm2zm a2
b1nz,+b2nz2…………………..+bmnzm  an
n Constraints
Example 3.9
Obntain the dual of the problem
Maximize y = 35x1+60x2+30x3
Subject to 3x1+5x2+2x3  50
2x1+6x2+3x3  40
x1,x2,x3  0
Solution
1. Matrix of coefficient of constarints
352 = 2x3
263
is transformed to
28
 32
56 = 3x2
23
2. Swap 50 for 35
40 60
30
2. Swap  for 
The dual is therefore
Y1 = 50z1+40zz
Subject to: 321+2z2  35
5z1+6z2  60
2z1+3z2  30 z1,z2  0
Tutorial/Assignment
Show that both this primal and dual have the same value at the optimum solution
Tutorial
Minimize y = x1+x2+x3
Subject to: x1+x4-3x5 = 1
x2+2x4 = 2
x3+x4-2x5 = 3
Solution – (x1 = 0, x2 = 1, x3 = 2, x4 = 1, x5 = 0)

29
LECTURE 4: NON-LINEAR PROGRAMMING.
4.0 INTRODUCTION
When an object function (sometimes called cost function) is expressed mathematically in
terms of variables which are non-linear, then the methods used in optimizing such an objective
function is called non-linear programming.
4.1 DEFINITION OF THE PROBLEM
The unconstrained problem is often defined as:
Objective function
Optimize
Y = f (x ) ………………………………….(4.1)
Where f (x) is a non-linear function in x (a vector x1,x2…….xn) of an independent variables.
For constrained problem, in addition to equation 4.1, constraint equations of the form:
gi (x )  Fi ( x )  hi ( x ) ……………………..4.2
i = 1, 2………………………m
are introduced.
4.2 TYPES OF NON-LINEAR PROGRAMMING
There are two kinds. One involving single variable and the other involving multi-
variable. Both types may be unconstrained or constrained.
4.2.1 SINGLE VARIABLE NON-LINEAR PROGRAMMING
The form of the unconstrained single variable non-linear program is:
Optimize y = f ( x) ……………………………….4.3
where f (x) is a non-linear function of the single variable x.
For constrained program, the condition subject to
a  x  b …………………………………4.4
is introduced.
For unconstrained program, the search for solution is done over interval -  < x < , which is an
infinite interval.

30
 4.2.1.1 SOLUTION OF SINGLE VARIABLE NON- LINEAR PROGRAMMING
PROBLEM.

Two methods exist for solving single variable non-linear program. They are the direct
and indirect methods. The indirect method make use of calculus. The problem with this method
is that is sometimes difficult or impossible to find the first and second differentials, which are
necessary to determine the turning points and their nature.
The direct method which is usually called the search or sequential search techniques are
numerical and iterative in nature. The values of the objective function is determined within a
(starting finite) interval withing which the objective function is presumed to have a minimum or
maximum. This interval is continually narrowed until the optimum solution is confined to an
interval that satisfies certain reasonable conditions. The procedure implies that the objective
function is sequentually evaluated at points determined by certain criteria and that the unimodal
properly (i.e property which assumed that the objective function has only one optimum value
within the interval) is used to eliminate certain parts of the interval being considered to
determine the next interval in the iteration. Search methods themselves either use a linear or
vector search .
4.2.1.1.1 SOLUTION BY LINEAR SEARCH
For this type, the problem is often of the form
Y = f (x)……………………………. 4.5
Subject to  < x <  ……………………..4.6
The solution starts with the evaluation of the objective function at one end of the constraint
interval or range. The value of the independent variable (x) is then increased and the value of the
objective function re-evaluated. If the optimal solution is not found within the new interval, the
search is then extended beyound the interval (subject to lower and upper bounds of the interval)
until the optimal solution is obtained .
4.2.1.1.2 SOLUTION BY VECTOR SEARCH
This is a direct search in one dimension (direct) for instance to minimize.
Y = f (x) …………………………………4.7
S ubject to
31
 s < x < t …………………………..4.8
y is evaluated at s. A small increase x is then added to s and y is then re-evaluated. This
process is continued until value of y increases. At this point, a new constraint is generated by
using
s+ (j – z)x < x < s+jh ……………………..4.9
where j is the number of increament x from s.
The objective function y is re-evaluated within the new interval. If the minimum cannot be
found within the new interval, the evaluated is extended beyond the interval.
4.2.1.2 FREQUENTLY USED SEQUENTIAL SEARCH TECHNIQUES FOR SINGLE
VARIABLE NON-LINEAR PROGRAMMING
Many sequential search methods exist, but the ones most frequently used are Three-Point
Interval search, Golden Section or Goden Mean Search, Fibonacci Search, Davies-Swann-
Campey Methods.
4.2.1.2.1 THREE POINT INTERVAL SEARCH
This method uses equally spaced points within the interval containing the optimum
solution. To achieve this, the interval is divided into quarters. Ther objective function is
evaluated at each of the equally space points. The point yelding the best value of the objective
function is selected as the centre of the new interval which consists of the nearest point (among
the points considered so far, including the interval bounds) to the left and right of the determined
centre point. The procedure is repeated. This is continued until the prescribed tolerance is met.
Algorithm
Iteration 1:
Step1: Divide interval by 4 and obtain the 3 points within initial interval
Step2: Evaluate the objective function at the 3 newly generated points.
Step3: Identify the point that is the minimum (or maximum).
Iteration 2:
Step4: Choose next interval from point flanking the point corresponding to the minimum
(maximum) on the right and left.
Step5: Repeat Steps 2 and 3.
Calculate the the tolerance from the minimum points obtained in Iterations 1 and 2. If the
tolerance is satisfied stop. If it is not, return to Step 4.
32
Example 4.1
Find the minimum of
M = (x – 3)2 where the constraint is 0 < x < 10
Using the 3 point interval search and  = xi  xi 1 = 0.0005
SOLUTION
Iteration 1
The interval = 10 – 10 = 10
The quarter of interval = 10 = 2.5
4
The 3 interval points = 0+(2.5) = 2.5
0+2(2.5) = 5.0
0 + 3(2.5) = 7.5
hence: M is determined at x = 2.5,5.0,7.5)
M(2.5) = 0.25
M(5.0) = 4
M(7.5) = 20.25
M(2.5) is the least, hence x = 2.5 is chosen the centre for the new three point interval i.e the new
interval is (0 - 5.0) and the new interior points are, 1.25, 2.5, 3.75.
Interation 2
M(1.25) = 3.0625
M(2.5) = 0.25
M(3.75) = 0.5625
M(2.5) still remains the least value and x = 2.5 still remains at the centre point of three points of
interval (1.25 - 3.75) given by x = 1.875, 2.5, 3.125.
Iteration 3
M(1.875) = 1.2656
M(2.5) = 0.25
M(3.125) = 0.01563
M(3.125) is the least value, therefore x =3.125 is chosen as the centre of a three point interval
(2.5 – 3.75) in which the values of x = 2.8125, 3.125, 3.4375.

33
Iteration 4
M(2.8125) = 0.03516
M(3.125) = 0.01563
M(3.4375)= 0.1914
M(3.125) is the least value, therefore x =3.125 is chosen as the centre of a three point interval
(2.8125 – 3.4375) in which the values of x = 2.96875, 3.125, 3.28125.
Iteration 5
M(2.96875) =0.0009765
M(3.125) = 0.01563
M(3.28125) =0.07912
M(2.96875) is the least values and is therefore chosen as the centre of the new interval.
(2.8125 - 3.125) where the values of x = 2.890625, 2.96875, 3.046875
Iteration 6
M(2.890625) =0.0.011963
M(2.96875) =0.0009765
M(3.046875) =0.002197
M(2.96875) is the least values and is therefore chosen as the centre of the new interval.
(2.890625 - 3.046875) where the values of x = 2.929688, 2.96875, 3.007813
Iteration 7
M(2.929688) = 0.004944
M(2.96875) = 0.0009765
M(3. 007813) = 6.1x 10-5
M(3. 007813) is still the least, so the new interval is (2.96875 - 3.046875) and the values of x =
2.88281, 3.007813, 3.027344
Iteration 8
M(2.988281) = 0.000137
M(3.007813) = 6.1 x 10-5
M(3.027344) = 0.000748
M(3.0078125) is the least, so the new interval is (2.988281 - 3.027344) and the value of x =
2.998047, 3.007813, 3.017578

34
Iteration 9
M(2.998047) = 3.81 x 10-6
M(3.007813) = 6.1 x 10-5
M(3.017578) = 0.000309
M(2.998047) is still the least, so the new interval is (2.988281 - 3.007813) and the values of x
are x= 2.993164, 2.998047, 3.00293
Iteration 10
M(2.993164) = 4.67 x 10-5
M(2.998047) = 3.81 x 10-6
M(3.00293) = 8.58 x 10-6
M (2.998047) still remains the least, hence x = 2.998047 is chosen as the new centre of the
interval (2.993164 - 3.002923). The values of these points are : x = 2.995605, 2.998047,
3.000488
Iteration 11
M(2.995605) = 1.93 x 10-5
M(2.998047) 3.81 x 10-6
M(3.000488) = 2.38 x 10-7
M(3.000488) is the least, so x = 3.000488 is the new centre of interval (2.998047 - 3.002923).
The values of x are x = 2.99268, 3.000488, 3.001709.
Iteration 12
M(2.999268) 5.36x 10-7
M(3.000488) = 2.38 x 10-7
M(3.001709) = 2.92 x 10-6
M(3.000488) still remains the least, so x = 3.000488 is the new centre of interval (2.999268 -
3.001709). The values of x are x = 2.999878, 3.000488, 3.001099.
Iteration 13
M(2.999878)= 1.49x 10-8
M(3.000488) = 2.38 x 10-7
M(3.001099) = 1.21 x 10-6

35
M(2.999878) is the least, so x = 2.999878 is the new centre of interval (2.999268 - 3.000488).
The values of x are x = 2.999573, 2.999878, 3.000183.
Iteration 14
M(2.999573) = 1.82 x 10-7
M(2.999878)= 1.49x 10-8
M(3.000183) = 3.36 x 10-8
M(2.999878) is still the least, so x = 2.999878 is the new centre of interval (2.999573 -
3.000183). The values of x are x = 2.999725, 2.999878, 3.000031.
Iteration 15
M(2.999725) = 7.54 x 10-8
M(2.999878)= 1.49x 10-8
M(3.000031) = 9.36 x 10-10
M(3.000031) is the least. The difference between optimal value of x in iteration 14 and 15 is
0.000153, which is less than  = xi  xi 1 = 0.0005. Hence the optimal x= 3.000031 and M= 9.36 x
10-10.
The solution of Example 4.1 can be summarised in tabular form as shown below.
Solution of Example 4.1 in tabular form
Problem: Find the minimum of M = (x – 3)2
Where the constraint is 0 < x < 10
Using the 3 Point Interval method and  = xi  xi 1 = 0.005
Interval Points Value of function M at points
Iteration Lower Upper left middle right left middle right Tolerance
1 0 10 2.5 5 7.5 0.25 4 20.25
2 0 5 1.25 2.5 3.75 3.0625 0.25 0.5625
3 1.25 3.75 1.875 2.5 3.125 1.265625 0.25 0.015625 0.625
4 2.5 3.75 2.8125 3.125 3.4375 0.035156 0.015625 0.191406 0
5 2.8125 3.4375 2.96875 3.125 3.28125 0.000977 0.015625 0.079102 -0.15625
6 2.8125 3.125 2.890625 2.96875 3.046875 0.011963 0.000977 0.002197 0
7 2.890625 3.046875 2.929688 2.96875 3.007813 0.004944 0.000977 6.1E-05 0.039063
8 2.96875 3.046875 2.988281 3.007813 3.027344 0.000137 6.1E-05 0.000748 0
9 2.988281 3.027344 2.998047 3.007813 3.017578 3.81E-06 6.1E-05 0.000309 -0.00977
10 2.988281 3.007813 2.993164 2.998047 3.00293 4.67E-05 3.81E-06 8.58E-06 0
11 2.993164 3.00293 2.995605 2.998047 3.000488 1.93E-05 3.81E-06 2.38E-07 0.002441
12 2.998047 3.00293 2.999268 3.000488 3.001709 5.36E-07 2.38E-07 2.92E-06 0
13 2.999268 3.001709 2.999878 3.000488 3.001099 1.49E-08 2.38E-07 1.21E-06 -0.00061
36
 14 2.999268 3.000488 2.999573 2.999878 3.000183 1.82E-07 1.49E-08 3.36E-08 0
15 2.999573 3.000183 2.999725 2.999878 3.000031 7.54E-08 1.49E-08 9.36E-10 0.000153

4.2.1.2.2 FIBONNACI METHOD

This is also a linear search method and works for unimodal functions. The tolerance of
the problem and the interval length is used to determine a number of observations from a series
called Fibonnaci Series or Sequence.. The number of functional evaluations needed to solve the
problem is equal to number of observations less 1. Fibonnaci Serries is generated from a series
of numbers often referred to as “Magic numbers“. These numbers are :
1 1 2 3 5 8 13 21 34 55 89etc
With the exception of the first two numbers, each number in the series is obtained by adding the
two preceeding numbers. The Fibonnaci numbers, Fn,its reciprocal (1/ Fn) and the number of
observation associated with each number, In , are given in Table 4.2
Table 4.2: Fibonnaci Series

Fn 1 2 3 5 8 13 21 34 55 89 144 233 377 610

1/Fn 1 0.5 0.3333 0.2 0.125 0.0769 0.0476 0.0294 0.0182 0.0112 0.0069 0.0043 0.0027 0.0016

In 1 2 3 4 5 6 7 8 9 10 11 12 13 14

The Algorithm of Fibonacci method is as follows:

Step 1: Use the tolerance to determine a parameter called resolution.
Step 2: Use the resolution and the interval to calculate the interval reduction
Step 3: Compare the value of the interval reduction with 1/Fn in the table to determine the
corresponing number of observations and consequently the number of functional evaluations
needed to solve the problem.
Step 4: Start the search from the highest number of the observations.
a. Compute two points corresponding to each of the two fractions for the observation.
b. Compute the corresponding values of the functions at the two points by multiplying the
interval for the (current) range by the corresponding fractions.
c. Determine the optimal point.
d. Determine the range for the next observation and return to Step 4 a.
e. Continue until the number of functional evaluations is exhausted.
Example 4.2
37
Solve problem 4.1 using Fibonnaci method.

Solution.
Function to minimize: M = (x – 3)2
Range: (0-10); Interval = 10-0=10

Resolution = 0.0005 0.0005 20.0005  0.001units

Re solution  0.001  0.0001

 Interval reduction = Interval 10
Extension of Fibonnaci Series (Table 4.2), we have

Fn 987 1597 2584 4181 6765 10946

1/Fn 0.0010 0.0006 0.0004 0.00024 0.00015 0.00009

In 15 16 17 18 19 20

From the Table, 0.0001 lies between 0.00015 and 0.00009, hence 20 observations (and 19
functional evaluations) are needed. The optimization of the problem is therefore done as shown
in the table below.
Table 4.3. Solution of Problem 4.1 using Fibonacci Method.
Functional Number of Fractions Range Value of Points Value of Function
Evaluations Observations Lower Upper Lower Upper Lower Upper Lower Upper
1 20 0.3820 0.6180 0 10 3.8197 6.1803 0.671842761 10.11456156
2 19 0.3820 0.6180 0 6.1803 2.3607 3.8197 0.408730446 0.671842761
3 18 0.3820 0.6180 0 3.8197 1.4590 2.3607 2.374741255 0.408730446
4 17 0.3820 0.6180 1.4590 3.8197 2.3607 2.9180 0.408730446 0.006730414
5 16 0.3820 0.6180 2.3607 3.8197 2.9180 3.2624 0.006730414 0.068842714
6 15 0.3820 0.6180 2.3607 3.2624 2.7051 2.9180 0.086967336 0.006730414
7 14 0.3820 0.6180 2.7051 3.2624 2.9180 3.0495 0.006730414 0.002451815
8 13 0.3820 0.6180 2.7051 3.0495 2.8367 2.9180 0.026682354 0.006730414
9 12 0.3820 0.6180 2.8367 3.0495 2.9180 2.9682 0.006730414 0.001010759
10 11 0.3819 0.6181 2.9180 3.0495 2.9682 2.9993 0.001010759 5.34157E-07
11 10 0.3820 0.6180 2.9682 3.0495 2.9993 3.0185 5.34157E-07 0.000340559
12 9 0.3818 0.6182 2.9682 3.0185 2.9874 2.9993 0.000158945 5.34157E-07
13 8 0.3824 0.6176 2.9874 3.0185 2.9993 3.0066 5.34157E-07 4.32667E-05
14 7 0.3810 0.6190 2.9874 3.0066 2.9947 2.9993 2.80766E-05 5.34157E-07
15 6 0.3846 0.6154 2.9947 3.0066 2.9993 3.0020 5.34157E-07 4.03956E-06
16 5 0.3750 0.6250 2.9947 3.0020 2.9974 2.9993 6.54343E-06 5.34157E-07
17 4 0.4000 0.6000 2.9974 3.0020 2.9993 3.0002 5.34157E-07 3.33848E-08
18 3 0.3333 0.6667 2.9993 3.0020 3.0002 3.0011 3.33848E-08 1.20185E-06
38
 19 2 0.5000 0.5000 2.9993 3.0011 3.0002 3.0002 3.33848E-08 3.33848E-08

From this table the solution is: Optimal x= 3.0002 and M= 3.338 x 10-8.
4.2.1.2.3 GOLDEN SECTION SEARCH
This is an example of a sector search. Consider Figure 4.1 which contains a single
minimum point of an objective function within an interval AB.

A C D B
Fig 4.1: Graphical Illustration of Golden Section Search
To apply the golden section (golden mean) search, the function, y, is first estimated at
points C and D that satisfy the following equation.

BC 1  5
  1.6180..................................4.10
CA 2
AC 1  5
  1.6180...................................4.11
CD 2
If point C yield a lower value, then the minimum is contained within AC. However, if point D
yields a a lower value, then AC cannot contain the minimum. The value of y is then estimated at
E so that:

BC 1  5
  1.6180...................................4.12
CE 2
To simplify the procedure, the first two points C and D at which y is evaluated is obtained from
C = 0.3820 I + value of A ……………………4.13
D = 0.6180 I + value of A ……………………4.14
Where I is the length of interval AB. These point C and D are therefore located at 0.6180I units
from the end point of the interval I.

39
  
 0.6180I  1  5 
 2 
 

is called the golden mean).

The direction in which the search will move will depend upon whether the function to be
optimized is to have a minimum or maximum and therefore can be towards the left or right of C
and D. The next point to be determined will be obtained by adding or substracting 0.6180I from
C or D where I is the current interval. It is substracted if the search is towards the right of C or D
or added if it is to the left of C and D. A new interval is determined for the third point, and the
procedure is repeated until the tolerance is satisfied .
Algorithm for Goden Search Method
Iteration 1:
Steps:
a. Start with the new range and evaluate the interval of the rangewhich is = I
b. Determine the two points (1 and 2) according to:
a. Point 1=0.382I + lower point (bound) of the range.
b. Point 2=0.618I + lower point (bound) of the range.
c. Evaluate objective function at the two points
d. Reject one of the points according to whether the objective function is to be maximized or
minimized.
e. Determine a new range to :
i. If rejected point is Point 1, the new range is Point 1 to higher point (bound) of the
old rangel.
ii. If rejected point is Point 2, the new range is lower point (bound) of the old
range.to Point 2
f. Check tolerance. If it is met, stop. If it is not met, return to Step a for the next iteration.
Example 4.3
Solve problem 4.1 using Golden Section search.
Solution
The procedure is often aided by graphical representation.
Iteration 1
Step a: Range =(0-10). Therefore interval = 10-0=10
40
Step b: Point 1= 0.382x10+0 = 3.82 and Point 2 = 0.6180 x 10+ 0 = 6.18
Step c: M( 3.82) = 0.6724 and M(6.18) = 10.11
Step d: Since we are minimizing, we reject Point 2
Step e: Since we are rejecting Point 2, the new range is lower point (bound) of the old range.to
Point 2, i.e. (0-6.18).
Step f: There is no need to check for tolerance at the first iteration. So we start iteration 2.
Iteration 2
Step a: Range =(0-6.18). Therefore interval = 6.18-0=6.18
Step b: Point 1= 0.382x6.18+0 = 2.3608 and Point 2 = 0.6180 x 6.18+0 = 3.82
Step c: M( 2.3608) = 0.4086 and M(3.82) = 0.6724
Step d: Since we are minimizing, we reject Point 2
Step e: Since we are rejecting Point 2, the new range is lower point (bound) of the old range.to
Point 2, i.e. (0-3.82).
Step f: The tolerance is mod(2.3608-3.82)=1.4592. The tolerance is not yet met, so we start
iteration 3.
Iteration 3
Step a: Range =(0-3.82). Therefore interval = 3.82-0=3.82
Step b: Point 1= 0.382x3.82+0 =1.4592 and Point 2 = 0.6180 x 3.82+0 = 2.3608
Step c: M( 1.4592) = 2.3739 and M(2.3608) = 0.4086
Step d: Since we are minimizing, we reject Point 1
Step e: Since we are rejecting Point 1, the new range is Point 1 to higher point (bound) of the
old range, i.e., (1.4592-3.82)
Step f: The tolerance is mod(2.3608-2.3608)=0. The tolerance is not yet met, so we start
iteration 4.
Iteration 4
Step a: Range =(1.4592-3.82). Therefore interval = 3.82-1.4592=2.3608
Step b: Point 1= 0.382x2.3608+1.4592 =2.3608 and Point 2 = 0.6180 x 2.3608+1.4592 = 2.9182
Step c: M(2.3608) = 0.4086 and M(2.9182) = 0.006691
Step d: Since we are minimizing, we reject Point 1
Step e: Since we are rejecting Point 1, the new range is Point 1 to higher point (bound) of the
old range, i.e., (2.3608-3.82

41
Step f: The tolerance is mod(2.3608-2.9182)=05574. The tolerance is not yet met, so we start
iteration 5.
Iteration 5
Step a: Range =(2.3608-3.82). Therefore interval = 3.82-2.3608=1.4592
Step b: Point 1= 0.382x1.4592+2.3608 =2.9182 and Point 2 = 0.6180 x 1.4592+2.3608 = 3.2626
Step c: M(2.9182) = 0.006691 and M(3.2626) = 0.06896
Step d: Since we are minimizing, we reject Point 2.
Step e: Since we are rejecting Point 2, the new range is lower point (bound) of the old range.to
Point 2, i.e. (2.3608-3.2626)
Step f: The tolerance is mod(2.9182-2.9182)=0. The tolerance is not yet met, so we start
iteration 6.
Iteration 6
Step a: Range =(2.3608-3.2626). Therefore interval =3.2626-2.3608 =0.9018
Step b: Point 1= 0.382x0.9018+2.3608 =2.7053 and Point 2 = 0.6180 x 0.9018+2.3608 = 2.9182
Step c: M(2.7053) = 0.08685 and M(2.9182) = 0.006691
Step d: Since we are minimizing, we reject Point 1
Step e: Since we are rejecting Point 1, the new range is Point 1 to higher point (bound) of the
old range, i.e., (2.7053-3.2626)
Step f: The tolerance is mod(2.9182-2.9182)=0. The tolerance is not yet met, so we start
iteration 7.
THIS PROCESS IS CONTINUED UNTIL INTERATION 18.
Iteration 18
Step a: Range =(2.9995-3.0005). Therefore interval =3.0005-2.9995 =0.001
Step b: Point 1= 0.382x0.001+2.9995 =2.9999 and Point 2 = 0.6180 x 0.001+2.9995 = 3.00012
Step c: M(2.9999) =1x10-8 and M(3.00012) = 1.44 x10-8
Step d: Since we are minimizing, we reject Point 2
Step e: Since we are rejecting Point 2, the new range is lower point (bound) of the old range.to
Point 2, i.e. (2.9995-3.00012)
Step f: The tolerance is mod(2.9999-2.9995)=00004.
This tolerance is less than 0.0005, so the optimal solution has been reached. That is x = 2.9999
and M = 1.0 x 10-8

42
The following table is also used to summarise the Golden Search Method.
Range Value of Points Value of Function
Iteration Lower Upper Point1 Point2 Point1 Point2 Tolerance
1 0.0000 10.0000 3.8200 6.1800 0.6724 10.1124
2 0.0000 6.1800 2.3608 3.8200 0.40857664 0.6724 1.4592
3 0.0000 3.8200 1.4592 2.3608 2.373941378 0.40857664 0
4 1.4592 3.8200 2.3608 2.9182 0.40857664 0.00669124 0.5574
5 2.3608 3.8200 2.9182 3.2626 0.00669124 0.06895876 0
6 2.3608 3.2626 2.7053 2.9182 0.08684809 0.00669124 0
7 2.7053 3.2626 2.9182 3.0497 0.00669124 0.00247009 0.1315
8 2.9182 3.2626 3.0497 3.1310 0.00247009 0.017161 0
9 2.9182 3.1310 2.9995 3.0497 2.5E-07 0.00247009 0.0502
10 2.9182 3.0497 2.9684 2.9995 0.00099856 2.5E-07 0
11 2.9684 3.0497 2.9995 3.0186 2.5E-07 0.00034596 0
12 2.9684 3.0186 2.9876 2.9995 0.00015376 2.5E-07 0
13 2.9876 3.0186 2.9995 3.0068 2.5E-07 4.624E-05 0
14 2.9876 3.0068 2.9949 2.9995 2.601E-05 2.5E-07 0
15 2.9949 3.0068 2.9995 3.0023 2.5E-07 5.29E-06 0
16 2.9949 3.0023 2.9977 2.9995 5.29E-06 2.5E-07 0
17 2.9977 3.0023 2.9995 3.0005 2.5E-07 2.5E-07 0
18 2.9995 3.0005 2.9999 3.00012 1E-08 1.44E-08 0.0004

Note: Solution is done to 4 decimal places in the x values.

43
Addendum
Fundamental formula: xi+1 = xi + > Jxi
(see page 96) minimum z = -(2x1 –5)2 – (x2-3)2 with initial point
x1 = 2, x2 = 2 with E = 0.005.
Solution

Z x1  42 x1  5
Z x 2  2 x2  3
2
x 0   , Z  x0   2
2
 2    44  5 
x1  x 0  xx     x 
 2    22  3 
 2   4   2  4x 
    x    
 2   2   2  2x 
 Z  x   4  8 X  5  2 X 3 X  3
2

 8 X  1  2 X  1
2 2

  
 64 x 2  1  66 x  4 x 2  1  4 x 
 64 x  4 x  1  1  16 x 4 x
2 2

 68 x 2  2  20 x

Using calculus, the optimal values of x is

44
 20
x1opv   0.15
136
2  40.15  2.6 
 x1   
2  20.15  2.3 
and
Z x1   0.53  check ) Z  x1 
 2.6 
x0   
 2.3 
 2.6    45.2  5 
x1    x  
 2.3    22.3  3 
 2.6    0.8   2.6  0.8 x 
  x   
 2.3   1.4   2.3  1.4 x 
Z x1   5.2  1.6 x  5  2.3  1.4 x  3
2 2

45
 0.2  1.6 x   1.4 x  0.7 
2

  
  0.04  2.56 x 2  0.64 x  1.96 x 2  0.49  1.960 
 0.53  4.52 x  2.6 x
2

2.6
opt  0.29
0.04
 2.6  0.80.29    2.37 
 x1    
 2.3  1.410  29   2.71 
and
Z  x1 .............0.156 zx0  zx1  0.374
 2.37 
x0   
 2.71 
 2.37    44.74  5 
x1       
 2.71    22.71  3 
 2.37  1.04 
 
 2.71  0.58 
Z  x1   4.74  2.08  5  2.71  0.58  3
2 2

  2.08  0.26   0.58  0.29 

2 2

 4.3262  0.3362  0.0676  0.084  1.082  0.336

 4.6622  0.1516  1.418
1.418
xopt   0.15
9.324
 2.37  1.040.15   2.526 
x1     
 2.71  0.580.15   2.797 
Z  x1   0.044
zx0  z x1   0.112
 2.526 
x0   
 2.797 

46
  2.526    45.052  5   2.526  0.208 
x1     x    
 2.797    22.797  3   2.797 x0.406 
Z  x1   5.052  0.416  5  2.797  0.406  3
2 2

 0.052  0.416   0.406  0.203

2 2

 0.0027  0.0417  0.173 x 2  0.165 x 2  0.0432 x10.1648

 0.0439  0.338 x 2  0.208 x
0.208
x4 opt   0.308
0.676
2.526  0.2080.308  2.462 
 x1   
2.797  0.4060.308  2.922 
 0.0119
Z x1   zx0  zx1  0.035
 2.462 
x0   
 2.922 
 2.462    44.924  5   2.462  0.304 x 
x1          
 2.922    22.922  3   2.922  0.156 x 
z  x1   4.924  0.608 x  5  2.922  0.156 x  3
2 2

 0.608 x  0.076   0.156 x  0.078

2 2

 0.0058 
 0.37  0.024 x 2  0.0924  0.0243x 2   
  0.0061 
 0.394 x 2  0.1167 x  0.0119
0.1167
x5opt   0.149
0.788
2.462  6.3040.149   2.507 
x1   
2.922  0.1560.149   2.945 
and
Z  x1   0.0033 zx1  zx1  0.0058
 2.507 
x0   
 2.945 
 2.507    45.014  5 
x1     x 
 2.945    22.945  3 
2.507  0.056 x

2.945  0.11x
Z  x1   5.014  0.112  5 x   2.945  11x  3
2 2

 0.014  0.112 x   0.11x  0.055

2 2

 0.0125 x0.0121x 2  0.00314  0.0121x  0.000196  0.00303

 0.0246 x 2  0.0152 x  0.00323
0.0152
x6 opt   0.309
0.0492
2.507  0.0560.309   2.49 
x1   
2.945  0.110.309   2.979  47
z  x1   0.00088 z  x6   z  x6   0.0024
Since the difference is less than the tolerance the solution is given by x 1 = 2.49, x2 = 2.98 and z =
-0.00088.

4.2.1.2 FLECTCHER AND POWELL METHOD

This method also employs the gradients method provided the gradient of the function can
be found. The procedure is as follows (it is developed for objective function of two variables).
Algorithm
Step 1
Using the starting point, calculate the following:

x 
z0  z x0 wherex0   1 
 x2 0
A0   x0 
 10 
makeI   
 01 
makeA  A, I  I
Algorithm
1. Calculate zo = z(x0)
Where

x 
x0   1 
 xn 0
b. A0   x0 
Where  (x0) where  is the matrix of the partial derivative.
2. Calculate
P =  I0A where I0 = (unit matrix of the same order A
3. Calculate X1 = X0+p
4. Evaluate Z1 = Z(x1)
5. Determine the tolerance Z0 – Z1
6. If tolerance is satisfied write the solution.
7. If tolerance is not satisfy calculate

48
a. A1 x1 
b.Q  A0  A1
ppT
R
pT Q
I 0QQI0
c.S 
QT T0Q

d. I1 = I0+R + S
8. Make (a) I1 = I0
(b). X1 = X0
( c ). A1 = A0
(d). Z1 = Z0
Repeat steps 2 to 8.
Step 2
Calculate P = x IA …………………..(1)
x1 = (x0 +P) …………………..(2)
z1 = z (x1) ……………………(3)
Check whether Z2 – Z1  E
If yes, stop
If No continue
Calculate A1 = (X1)………………..(4)
Q = A0 – A1…………………...(5)
PTQ ….………………………(6)
R = PPT………………………..(7)
PTQ

QTIQ………………………….(8)
S = IQQTI……………………..(9)
QTIQ
I1 = I0+R +S
I = I1
X0 = X1

49
 A = A1
Algorithm for Steepest Descent Ascent
For z = f (xi)I = 1,A,n,
1. Obtain the partial derivaties of Z i.e dz I = 1……n)
2xi
2. Evaluate Z (X0) where X0

x 
  i 
 xn 0
3. Determine X1 = X0 * (X0) where  is value of matrix of partial derives. Hence (X0)
is the matrix of partial derivatives at the initial points.
4. Obtain Z (X1)
5. Calciulate value of  that optimize Z(X1) and evaluate X1.
6. Determine tolerance from Z(X0) – Z(X1)
7. If tolerance is satisfied write the solution. If it is not make X1 = X0 and repeat steps 3 to
7.

Example
50
 Solve the last problem (example) using Fletcher and Powell method.
Iteration

Step 1

Z 0  Z  X 0   4  5  2  3  1  1  2.
2 2

  44  5   4 
A0   X 0       
  22  3   2 
 10 
I  
 01 

Step 2

 10  4  4
P  IA         
 01  2  2
2  4   2  4 
X  X 0  P         
2  2   2  2 
Z1  Z  4  8  5  2  2  3
2 2

 8  1  2  1
2 2

  
  642  16  1  42  4  1 
 682  20  2
20
X opt   0.15
136
hence
 4   0.6 
P  0.15    
 2   0.3 
and
Z1  0.53.Since Z1  Z 2  2.53,
 2   0.6   2.6 
X1       
 2   0.3   2.3 

  45.2  5    0.8 
A1   X 1       
  22.3  3   1.4 
 4    0.8   4.8 
Q  A A     
 2   1.4   0.6 

51
We now go back to step 2.

 4. 8 
PTQ  0.60.3   3.06
 0. 6 
1  0.6 
0.60.3
1
R PP T  
3.06 3.06  0.3 
1  0.36 0.18 
  
3.06  0.18 0.09 
 0.1176 0.0588 
 
 0.0588 0.0294 
  10  4.8 
QT IQ   4.80.6   
  01  0.6 
  4. 8 
  4.80.6    23.4
  0. 6 

I  10   0.1176 0.0588   0.9846  0.1231 

New  I 0  R  S       
1  01   0.0588 0.0294   0.1231  0.0154 
 0.133  0.0643 
 
  0.643  1.014 
1
S IQQT I
23.4
 1  10  4.8   10 
   4.80.6  
23.4  0.6  0.6   01 
 1  4.8 
  4.806
23.4  0.6 
 1  23.04 2.88 
  
23.4  2.88 0.36 
  0.9846  0.1231 
 
  0.1231  0.0154 

 0.125 
 
 0.5003 

52
   0.048 
A1   X 1   45.012  5     23.003  3
  0.006 
  0.8    0.048    0.752 
Q  A0  Anew     
0  1.4    0.006   1.406 
  0.752 
PT Q   0.0939  0.703 
 1.406 
 1.059
1   0.0939 
 0.0939  0.7031
1
R PP T  
1.059 1.059  0.7031 
1  0.0088  0.066 
  
1.059   0.066  0.4943 
 0.0083  0.0623 
 
  0.0623  0.4668 
 0.133  0.0643    0.752 
0

Q IQ   0.752  1.406
T
  
  0.0643  1.014   1.406 
  0.752 
  0.1904  1.474 
 1.406 
 2.216
S  1IQQT I
1  0.133  0.0643   0.752 
    0.752  1.406 
2.216   0.0643  1.014  1.406 
 0.133  0.0643 
 
  0.0643  1.015 
 1   0.1904 
   0.1904  1.474
2.216  1.474 
 1  0.036  0.2806 
  
2.216   0.2806  2.1727 
  0.0163  0.1266 
 
 0.1266  0.9805 
 0.133  0.0643   0.0083  0.0623    0.0163  0.1266 
I1  I 0  R  S     
  0.0643  1.014    0.0623  0.4668   0.1266  0.9805 
 0.125 
 
 0.5003 

53
  0.125    0.0939    0.048 
I  I1   X0   A   
 0.5003   0.7031    0.006 
 0.125   0.048 
P  IA    
 0.5003   0.006 
  0.006 
  
  0.003 
 2.506    0.006 
X 1  X0  P      
 3.003    0.003 
 2.506  0.006 
 
 3.003  0.003 
Z 2  2 X 1   5.12  0.012  5  3.003  0.003  3
2 2

 0.012  0.012   0.003  0.0032 

2 2

 0.000144  0.0000092  0.000288  0.000018  0.000144  0.00009

 0.0001532  0.000306  0.000153
0.000306
op  1
0.000306
 2.5 
 X new   andZ 4  0
 3.0 

ADDENDUM
3 Point Interval Method.
Addendum Source Outline Series
1. Example – use the 3 point interval dearest to approximate the location of the global
minimum f (x) = x sin 4x on {0,3} to within =0.001
Solution
Sometimes c rough graph of the function can give us an idea about which interval to restrict our
search as to minimize the number of iteration.
A plot of this graph gives.
The stationary point alternate with the zero of f(x) (Rolles theore) which are zero of 4x. The
global minin is attaine while the initial
( 7ie 3) i.e 2.73  x6  3.
8

54
Because it is the region in which the negative values of sum 4x are multiplied by the largest point
value have the following iteration
Iteration 2
1st interation (2,75,3)
Point 2.8117, 2.8744, 2.9372.
f (2.8117) = -2.7234
f (2.8744) =
f (2.9372) =

55