Optimization

PhD Course Lectures

Optimization is the act of obtaining the best result under given circumstances.

Optimization can be defined as the process of finding the conditions that give the
maximum or minimum value of a function.

1
Example 1.1
Design a uniform column of tubular section, with hinge joints at both ends, to carry a
compressive load P = 2500 kgf for minimum cost. The column is made up of a
material that has a yield stress (σy) of 500 kgf/cm2, modulus of elasticity (𝐸) of 0.85
× 106 kgf/cm2, and weight density (𝜌) of (0.0025) kgf/cm3. The length of the column
is 250cm. The stress induced in the column should be less than the buckling
stress as well as the yield stress. The mean diameter of the column is restricted
to lie between 2 and 14 cm, and columns with thicknesses outside the range 0.2
to 0.8 cm are not available in the market. The cost of the column includes material
and construction costs and can be taken as (𝟓𝑾 + 𝟐𝒅), where 𝑊 is the weight in
kilograms force and 𝑑 is the mean diameter of the column in centimeters.

2
 Fig. (1) The system under study

The objective function is the cost function

𝑓(𝐗) = 5𝑊 + 2𝑑
𝑊 = 𝜌(𝜋𝑑) × 𝑡 × 𝑙
Then
𝑓(𝐗) = 5 × (𝜌𝜋𝑑 × 𝑡 × 𝑙) + 2𝑑
𝑓(𝐗) = 5𝜌𝜋𝑙 × (𝑑 × 𝑡) + 2𝑑

Since
5𝜌𝜋𝑙 = 5 × 0.0025 × 3.14 × 250 = 9.817
Then
𝑓(𝐗) = 9.817 × 𝑑 × 𝑡 + 2𝑑

Let 𝑥1 = 𝑑 and 𝑥2 = 𝑡, then the objective function to be minimized is given by

3
 𝑓(𝐗) = 9.817𝑥1 𝑥2 + 2𝑥1

The design variables are the mean diameter (𝑑) and tube thickness (𝑡):
𝑥1 𝑑
𝐗 = {𝑥 } = { }
2 𝑡

The behavior constraints can be expressed as:

Induced stress ≤ yield stress

𝜎𝑖 ≤ 𝜎𝑦
Or
𝑔1 (𝐗) = 𝜎𝑖 − 500 ≤ 0

The induced stress (𝜎𝑖 )is given by

𝑃 2500
𝜎𝑖 = =
𝜋 × 𝑑 × 𝑡 𝜋 × 𝑥1 × 𝑥2
2500
𝑔1 (𝑿) = − 500 ≤ 0
𝜋𝑥1 𝑥2

Induced stress ≤ buckling stress

𝜎𝑖 ≤ 𝜎𝑏
Or
𝑔2 (𝐗) = 𝜎𝑖 − 𝜎𝑏 ≤ 0

Buckling stress 𝜎𝑏 is given by

Euler buckling load

𝜎𝑏 =
cross − sectional area

𝜋 2 𝐸𝐼
𝜎𝑏 = 𝑙2
𝜋×𝑑×𝑡
Where
𝐼 = Second moment of area of the cross section of the column
𝜋 4
𝐼= (𝑑𝑜 − 𝑑𝑖4 )
64
𝜋
𝐼= (𝑑 2 + 𝑑𝑖2 )(𝑑𝑜 + 𝑑𝑖 )(𝑑𝑜 − 𝑑𝑖 )
64 𝑜
4
 𝜋
𝐼= [(𝑑 + 𝑡)2 + (𝑑 − 𝑡)2 ] × [(𝑑 + 𝑡) + (𝑑 − 𝑡)][(𝑑 + 𝑡) − (𝑑 − 𝑡)]
64
𝜋
𝐼 = (𝑑 × 𝑡)(𝑑2 + 𝑡 2 )
8
𝜋
𝐼 = 𝑥1 𝑥2 (𝑥12 + 𝑥22 )
8
Hence,
𝜋 2𝐸 𝐼
𝜎𝑏 = 2 ×
𝑙 𝜋×𝑑×𝑡
𝜋 2 2
𝜋 2 𝐸 8 𝑥1 𝑥2 (𝑥1 + 𝑥2 )
𝜎𝑏 = 2 ×
𝑙 𝜋𝑥1 𝑥2

𝜋 2 𝐸 (𝑥12 + 𝑥22 )
𝜎𝑏 = 2 ×
𝑙 8
2 (0.85 6 )(𝑥 2 2
𝜋 × 10 1 + 𝑥2 )
𝜎𝑏 =
8(250)2
2500 𝜋 2 (0.85 × 106 )(𝑥12 + 𝑥22 )
𝑔2 (𝐗) = − ≤0
𝜋𝑥1 𝑥2 8(250)2

The side constraints are given by

2 ≤ 𝑑 ≤ 14
0.2 ≤ 𝑡 ≤ 0.8
These constraints can be expressed in standard form as:
𝑔3 (𝐗) = −𝑥1 + 2.0 ≤ 0
𝑔4 (𝐗) = 𝑥1 − 14.0 ≤ 0
𝑔5 (𝐗) = −𝑥2 + 0.2 ≤ 0
𝑔6 (𝐗) = 𝑥2 − 0.8 ≤ 0

Since there are only two design variables, the problem can be solved graphically as
shown below.
First, the constraint surfaces are to be plotted in a two-dimensional design
space where the two axes represent the two design variables 𝑥1 and 𝑥2 . To plot the
first constraint surface, we have
2500
𝑔1 (𝐗) = − 500 ≤ 0
𝜋𝑥1 𝑥2
That is,
𝑥1 𝑥2 ≥ 1.593
5
Thus the curve 𝑥1 𝑥2 = 1.593 represents the constraint surface 𝑔1 (𝐗) = 0. This
curve can be plotted by finding several points on the curve. The points on the curve
can be found by giving a series of values to 𝑥1 and finding the corresponding values
of 𝑥2 that satisfy the relation 𝑥1 𝑥2 = 1.593:

𝑥1 2.0 4.0 6.0 8.0 10.0 12.0 14.0

𝑥2 0.7965 0.3983 0.2655 0.199 0.1593 0.1328 0.114

These points are plotted and a curve 𝑃1 𝑄1 passing through all these points is drawn as
shown in Fig. (2), and the infeasible region, represented by 𝑔1 (𝐗) > 0 or 𝑥1 𝑥2 <
1.593, is shown by hatched lines. Similarly, the second constraint 𝑔2 (𝐗) ≤ 0 can be
expressed as 𝑥1 𝑥2 (𝑥12 + 𝑥22 ) ≥ 47.3 and the points lying on the constraint surface
𝑔2 (𝐗) = 0 can be obtained as follows for 𝑥1 𝑥2 (𝑥12 + 𝑥22 ) = 47.3:

𝑥1 2.0 4.0 6.0 8.0 10.0 12.0 14.0

𝑥2 2.41 0.716 0.219 0.0926 0.0473 0.0274 0.0172

These points are plotted as curve 𝑃2 𝑄2 , the feasible region is identified, and the
infeasible region is shown by hatched lines as in Fig. 2. The plotting of side
constraints is very simple since they represent straight lines. After plotting all the six
constraints, the feasible region can be seen to be given by the bounded
area 𝐴𝐵𝐶𝐷𝐸𝐴.
Next, the contours of the objective function are to be plotted before finding the
optimum point. For this, we plot the curves given by
𝑓(𝐗) = 9.817𝑥1 𝑥2 + 2𝑥1 = 𝑐 = constant
Hence, 𝑓(𝐗) is constant for a series of values of 𝑐. By giving different values to 𝑐, the
contours of 𝑓 can be plotted with the help of the following points.

6
 Fig. (2)

For
9.82𝑥1 𝑥2 + 2𝑥1 = 50.0

𝑥2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

𝑥1 16.77 12.26 10.10 8.44 7.24 6.33 5.64 5.07
For
9.82𝑥1 𝑥2 + 2𝑥1 = 40.0

𝑥2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

𝑥1 13.4 10.10 8.08 6.75 5.79 5.06 4.51 4.05
7
For
9.82𝑥1 𝑥2 + 2𝑥1 = 31.5 (Passing through the corner point C):

𝑥2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

𝑥1 10.57 7.96 6.38 5.33 4.57 4.00 3.56 3.20
For
9.82𝑥1 𝑥2 + 2𝑥1 = 26.53 (Passing through the corner point B):

𝑥2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

𝑥1 8.88 6.69 5.36 4.48 3.84 3.36 2.99 2.69
For
9.82𝑥1 𝑥2 + 2𝑥1 = 20.0

𝑥2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

𝑥1 6.70 5.05 4.04 3.38 2.90 2.53 2.26 2.02

These contours are shown in Fig. 2 and it can be seen that the objective function
cannot be reduced below a value of 26.53 (corresponding to point 𝐵) without
violating some of the constraints. Thus the optimum solution is given by point 𝐵 with
𝑑 ∗ = 𝑥1∗ = 5.44 cm and 𝑡 ∗ = 𝑥2∗ = 0.293 cm with 𝑓𝑚𝑖𝑛 = 26.53.

8
 Chapter 5
Nonlinear Programming I:
One-Dimensional Minimization Methods

If the optimization problem

a) Involves the objective function and/or constraints that are not stated as explicit
function of the design variable.
b) Is too complicated to manipulate.
It cannot be solved using classical analytical methods. Numerical methods are used to
solve such problems. For example the following problem cannot be solved
analytically as that of Ex: 1, then suppose that its solution is as it shown in figure
below:

A
E Solution Space
ABCDE

C D

Two design variables (𝑥1 , 𝑥2 ), with five restrictions and an objective function (𝑓).
The solution region (ABCDE) is bounded by restrictions. Since, its solution cannot
be achieved, then the numerical solution has to be considered. The basic philosophy

9
of most of the numerical methods of optimization is to produce a sequence of
improved approximations to the optimum according to the following scheme:
1. Start with an initial trial point 𝐗1
2. Find a suitable direction 𝐒𝑖 (𝑖=1 to start with) that points in the general direction
of the optimum.
3. Find the appropriate step length 𝜆∗𝑖 for movement along the direction 𝐒𝑖 .
4. Obtain the new approximation 𝐗 𝑖+1 as
𝐗 𝑖+1 = 𝐗 𝑖 + 𝜆∗𝑖 𝐒𝑖
5. Test whether 𝐗 𝑖+1 is optimum, if 𝐗 𝑖+1 is optimum, stop the procedure.
Otherwise, set a new 𝑖 = 𝑖 + 1 and repeat step (2) onward.

One-dimensional minimization methods can be solved analytically using differential

calculus methods. But for those described above are solved numerically as shown in
the table (5-1) below:
Table (5-1)
Numerical Methods
Elimination Methods Interpolation Methods
1 Unrestricted Search Requiring no derivatives Requiring derivatives
a- Search with fixed step 1 Quadratic 1 Cubic interpolation
size. interpolation method
method
b- Search with 2 Direct root methods
accelerated step size
2 Exhaustive search a- Newton
3 Dichotomous search b- Quasi-Newton
4 Interval halving method c- Secant
5 Fibonacci method
6 Golden section method

UNIMODAL FUNCTION
A unimodal function is one that has only one peak (maximum) or valley (minimum)
in a given interval.

Fig. 1

Unimodal
Functions

10
A unimodal functions can be nondifferentiable or even a discontinuous function. The
assumption of unimodality is made in all the elimination techniques. If a function is
known to be multimodal (i.e., having several valleys or peaks), the range of the
function can be subdivided into several parts and the function treated as a unimodal
function in each part.

Elimination Methods

1. Unrestricted Search:
This approach adopted in case of a problem in which the range of the optimum
solution is not restricted. Two types of searches considered:

A: Search with fixed step size.

B: Search with accelerated step size.

2. Exhaustive Search: ‫البحث الشامل‬

This approach adopted in case of a problem in which the range of the optimum
solution is known. As shown in figure 5.2 that the starting point (𝑥𝑠 ) and the final
point (𝑥𝑓 ) are determined. This method is described as "Simultaneous" in that all
the experiments are conducted before any judgment is made regarding the location of
the optimum point.

Fig. (5.1) The Exhaustive search.

3. Dichotomous Search: )‫(تلفظ دايكوتمس‬ ‫البحث الثنائي التفرع‬

This method, Fibonacci and golden section methods are considered as "Sequential"
search methods in which the results of any experiment influences the location of the
subsequent experiment, as shown in figure (5-3).

11
 Fig. (5-3) Dichotomous search

Plot of Ex: 5.5 Page 258

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
-0.1

-0.2
f(x)

-0.3

-0.4

-0.5

-0.6
X

12
4. Interval Halving Method: )‫طريقة تخفيض الفترة الزمنية )تنصيف الفتره‬
In this method, exactly one-half of the current intervals of uncertainty are
deleted in every stage. Figure (5-4) shows the elimination criteria of this method.

Fig. (5-4) Interval halving method

13
5. Fibonacci Method: ‫أسلوب فيبوناتشي‬
In this method the final interval of uncertainty will be known and it can be
made as small as desired as well as the required resolution can be determined. The
table (5-2) shows this ability.
Table (5-2) Fibonacci Method

Ex. 5.7 Find the minimum of the function

0.75 −1
1
𝑓(𝑥) = 0.65 − − 0.65𝜆 tan … … . (1)
1 + 𝑥2 𝑥

𝑥 -1 0 0.3 0.5 1 2 3 4
𝑓(𝑥) 0.5145 -0.1 -0.287 -0.34 -0.2355 -0.103 -0.0524 -0.03106

14
 0.6
𝒇 𝒙
0.5
0.4
0.3
0.2
0.1
𝒙
0
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
-0.1
-0.2
-0.3
-0.4

The exact shape of the function

15
6. Golden Section Method:
In this method a large number of experiments are conducted, but the total
number of experiments can be decided during the computation.

Interpolation Methods

1. Quadratic Interpolation Method

To understand this method it is important to make a review about the quadratic

function.
The typical equation of the quadratic function is:
ℎ(𝜆) = 𝑎 + 𝑏𝜆 + 𝑐𝜆2

Let ℎ(𝜆) = 1 + 2𝜆 + 3𝜆2 , means (𝑎 = 1, 𝑏 = 2, 𝑎𝑛𝑑 𝑐 = 3)

Then

𝑑ℎ(𝜆)
= 𝑏 + 2𝑐𝜆
𝑑𝜆
Or

𝑑ℎ(𝜆)
= 2 + 6𝜆
𝑑𝜆
Minimum value of ℎ(𝜆) when

𝑑ℎ(𝜆)
=0
𝑑𝜆
Hence,
𝑏
𝜆𝑚𝑖𝑛 = −
2𝑐
Or,
2 1
𝜆𝑚𝑖𝑛 = − = − = −0.33333333333333333333333333333333
6 3
Also,

𝑑2ℎ
= 2𝑐 > 0 (𝑓𝑜𝑟 𝑚𝑖𝑛𝑖𝑚𝑢𝑚)𝑖𝑓 𝑐 > 0 = 2 × 3 = 6 > 0
𝑑𝜆2
Let us see at figure (1) the real plot of this function for
16
 𝑎 = 1, 𝑏 = 2, 𝑎𝑛𝑑 𝑐 = 3

𝜆 -2 -1 -0.333 0 1 2 3 4
ℎ(𝜆) 9 2 0 1 6 17 34 57

At 𝜆𝑚𝑖𝑛 ,
ℎ(𝜆) = 0.0

f(t)
𝒉(𝝀)
60

50

40

30

20

10

0 𝝀
-3 -2 -1 0 1 2 3 4 5

Fig. (5-5) the function ℎ(𝜆) = 1 + 2𝜆 + 3𝜆2

The quadratic method requiring nonderivatives

17
2. Cubic Interpolation Method

What is the Cubic Equation?

The Cubic Equation

ℎ(𝜆) = 𝑎 + 𝑏𝜆 + 𝑐𝜆2 + 𝜆3

Let

𝑎 = 1, 𝑏 = 2, 𝑐 = 3

𝜆 -5 -4 -3 -2 -1 0 1 2 3 4 5
ℎ(𝜆) -59 -23 -5 1 0 1 7 25 61 121 211

The function ℎ(𝜆) is shown in the figure below

𝒉f(t)
𝝀
80
75
70
65
60
55
50
45
40
35
30
25
20
15
10
5 𝝀
0
-6 -5 -4 -3 -2 -1 -5 0 1 2 3 4 5 6
-10
-15
-20
-25
-30
-35
-40
-45
-50
-55
-60

Fig. (5-6)

Since, this method requiring derivatives to be known then

18
7. Direct Root Methods:
This group of methods depending on a truth states that; "Since the necessary function
𝑓(𝜆) has a minimum (𝜆∗ ), three root-finding methods; the Newton, the quasi-
Newton, and the secant methods are discussed below:

a. Newton Method:
To understand this method it is preferable to make a review about "Taylor's Series".
This series depends on the truth that if one point on a function is known, and the
derivatives of this function are also known for that point, then the function can be
written as:

𝑓 ′ (𝑎) 𝑓 ′′ (𝑎) 𝑓 ′′′ (𝑎)

𝑓(𝑥 = 𝑎) = 𝑓(𝑎) + (𝑥 − 𝑎) + (𝑥 − 𝑎)2 + (𝑥 − 𝑎)3 + . . ..
1! 2! 3!
Or
𝑛=∞
𝑓 𝑛 (𝑎)
𝑓(𝑥 = 𝑎) = ∑ (𝑥 − 𝑎)𝑛
𝑛!
𝑛=0

19
Fig. (5-7) as the degree of the Taylor polynomial rises, it approaches the correct
function. This image shows sin(x) and its Taylor approximations, polynomials of
degree 1, 3, 5, 7, 9, 11 and 13.

Now, Newton method states that:

For any function 𝑓(𝜆), if a point (𝜆𝑖 ) on this function and its first and second
derivatives are known too, then the first three terms of Taylor's series can be written
to represent the function 𝑓(𝜆) approximately as:

1
𝑓(𝜆) = 𝑓(𝜆𝑖 ) + 𝑓 ′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 ) + 𝑓 ′′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 )2 … … … … … (1)
2

Then, the first derivative of this function 𝑓(𝜆) will be:

20
 𝑑𝑓(𝜆)
= 𝑓 ′ (𝜆) = 𝑓 ′ (𝜆𝑖 ) + 𝑓 ′′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 ) … … … … … (2)
𝑑𝜆

Since, the substitution of the value of (𝜆𝑖 ) in equation (2) will not make it equal to
zero, because (𝜆𝑖 ) is not the minimum of this function. The solution of this equation
(𝜆) will be considered as a new starting point (𝜆𝑖+1 ). Hence, equation (2) can be
rearranged as:
𝑓 ′ (𝜆𝑖 ) + 𝑓 ′′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 ) = 0
𝑓 ′ (𝜆𝑖 ) + (𝜆)𝑓 ′′ (𝜆𝑖 ) − (𝜆𝑖 )𝑓 ′′ (𝜆𝑖 ) = 0
(𝜆)𝑓 ′′ (𝜆𝑖 ) − (𝜆𝑖 )𝑓 ′′ (𝜆𝑖 ) = −𝑓 ′ (𝜆𝑖 )
Divide both sides by 𝑓 ′′ (𝜆𝑖 ) results in:

−𝑓 ′ (𝜆𝑖 )
(𝜆) − (𝜆𝑖 ) = ′′
𝑓 (𝜆𝑖 )
Or
𝑓 ′ (𝜆𝑖 )
(𝜆) = (𝜆𝑖 ) − ′′
𝑓 (𝜆𝑖 )
Then, this value of (𝜆) is considered as (𝜆𝑖+1 ) for the new step as:

𝑓 ′ (𝜆𝑖 )
(𝜆𝑖+1 ) = (𝜆𝑖 ) − ′′
𝑓 (𝜆𝑖 )

21
Newton or Newton-Raphson Method

Ex. 5.12 Find the minimum of the function

0.75 −1
1
𝑓(𝜆) = 0.65 − − 0.65𝜆 tan … … . (1)
1 + 𝜆2 𝜆
Use Newton-Raphson method with the starting point equal to (𝜆1 = 0.1).
Assume 𝜖 = 0.01.

𝜆 -1 0 0.3 0.5 1 2 3 4
𝑓(𝜆) 0.5145 -0.1 -0.287 -0.34 -0.2355 -0.103 -0.0524 -0.03106

0.6
𝒇 𝝀
0.5
0.4
0.3
0.2
0.1
𝝀
0
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
-0.1
-0.2
-0.3
-0.4

The exact shape of the function

22
 1.5𝜆 0.65𝜆 1
𝑓 ′ (𝜆) = + − 0.65 tan−1
( ) … … … (2)
(1 + 𝜆2 )2 1 + 𝜆2 𝜆

𝜆 0.2 0.4 0.6 0.8 1 1.5 2 3 4 5

′ (𝜆)
𝑓 -0.49 -0.104 -0.06 0.187 0.1895 0.1308 0.0786 0.03 0.0145 0.0078

0.3
𝒇′ (𝝀)
0.2

0.1
𝝀
0
0 1 2 3 4 5 6
-0.1

-0.2

-0.3

-0.4

-0.5

-0.6

The exact shape of the First derivative variation of the objective function

′′ (𝜆)
2.8 − 3.2𝜆2
𝑓 = … … … (3)
(1 + 𝜆2 )3

Now, we have only three known quantities about an objective function, these are:

i. The exact magnitude of the objective function at a single point (𝜆1 = 0.1) is
equal to 𝑓(𝜆1 ) = −0.188197.
ii. The first derivative of the objective function at (𝜆1 = 0.1), equal
to (−0.744832).
iii. The second derivative of the objective function at (𝜆1 = 0.1) , equal
to (2.68659 ) .
Solution:
The quadratic function extracted from Taylor's series due to the above given
information, the objective function will be due to equation (1):
1
𝑓(𝜆) = 𝑓(𝜆𝑖 ) + 𝑓 ′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 ) + 𝑓 ′′ (𝜆𝑖 )(𝜆 − 𝜆𝑖 )2 … … … … … (4)
2
1
𝑓(𝜆) = −0.188197 + (−0.744832)(𝜆 − 0.1) + × (2.68659 )(𝜆 − 0.1)2
2

23
 ‫)‪𝑓(𝜆) = 0.0206157 − 1.013491𝜆 + 1.343295𝜆2 … … … … … . . (5‬‬

‫𝜆‬ ‫‪0‬‬ ‫‪0.1‬‬ ‫‪0.2‬‬ ‫‪0.3‬‬ ‫‪0.4‬‬ ‫‪0.45‬‬ ‫‪0.5‬‬ ‫‪0.6‬‬ ‫‪0.7‬‬
‫‪𝑓(𝜆) Sim. 0.02 -0.067 -0.128 -0.163 -0.17 -0.163 -0.15 -0.104 -0.031‬‬
‫‪𝑓(𝜆)Real -0.1 -0.188 -0.25 -0.287 -0.306 -0.309 -0.31 -0.3‬‬ ‫‪-0.29‬‬

‫‪0.05‬‬

‫‪0‬‬
‫‪0‬‬ ‫‪0.2‬‬ ‫‪0.4‬‬ ‫‪0.6‬‬ ‫‪0.8‬‬
‫‪-0.05‬‬

‫‪-0.1‬‬
‫‪sim‬‬
‫‪-0.15‬‬
‫‪real‬‬
‫‪-0.2‬‬

‫‪-0.25‬‬

‫‪-0.3‬‬

‫‪-0.35‬‬

‫‪Taylor's series representation of the real and the simulated objective functions‬‬

‫)‪𝑓 ′ (𝜆) = −1.013491 + 2.68659𝜆 … … … … . (6‬‬

‫)‪𝑓 ′′ (𝜆) = 2.68659 … … … … . (7‬‬

‫‪Iteration 1‬‬
‫‪𝜆1 = 0.1, 𝑓(𝜆1 ) = −0.188197, 𝑓 ′ (𝜆1 ) = −0.744832, 𝑓 ′′ (𝜆1 ) = 2.68659‬‬

‫في هذه المرحله تتساوى قيم المشتقه األولى و المشتقة الثانية سواءا أستخدمنا (‪ )2, 3‬إليجادهما أم (‪)6, 7‬‬

‫) ‪𝑓 ′ (𝜆1‬‬
‫‪𝜆2 = 𝜆1 − ′′‬‬ ‫‪= 0.377241‬‬
‫) ‪𝑓 (𝜆1‬‬

‫𝜖 > |‪Convergence check: |𝑓 ′ (𝜆2 )| = |−0.13823‬‬

‫في التكرار األول يتم تحديد قيمة المشتقه األولى لـ( ‪ )𝜆2‬أي ) ‪ 𝑓 ′ (𝜆2‬بإستخدام المعادلة (‪ )2‬أعآله؟؟؟؟؟ فالسؤال‬
‫هنا هو لم كل هذه الفوضى إذا كانت المعادلة رقم (‪ )1‬موجودة ‪ ...‬فيمكننا إيجاد قيمتها الدنيا مباشرة وبدون‬
‫مشاكل؟؟؟ وإذا اعتمدنا المعادلة رقم (‪ )5‬فسنصل الى قيمتها الدنيا بدون مشاكل أيضا!!! ترى ما المقصود بهذه‬
‫الطريقة؟ ولم هذا الحل الغريب؟‬

‫‪Iteration 2‬‬

‫مالحظة‪ :‬تم إيجاد هذه القيم بإستخدام المعادالت (‪)1, 2, 3‬‬

‫‪24‬‬
 𝑓(𝜆2 ) = −0.303279, 𝑓 ′ (𝜆2 ) = −0.13823, 𝑓 ′′ (𝜆2 ) = 1.57296

𝑓 ′ (𝜆2 )
𝜆3 = 𝜆2 − ′′ = 0.465119
𝑓 (𝜆2 )

Convergence check: |𝑓 ′ (𝜆3 )| = |−0.0179078| > 𝜖

Iteration 3
𝑓(𝜆3 ) = −0.309881, 𝑓 ′ (𝜆3 ) = −0.0179078, 𝑓 ′′ (𝜆3 ) = 1.17126

𝑓 ′ (𝜆3 )
𝜆4 = 𝜆3 − ′′ = 0.480409
𝑓 (𝜆3 )

Convergence check: |𝑓 ′ (𝜆4 )| = |−0.0179078| < 𝜖

Since the process has converged, the optimum solution is taken as 𝜆∗ ≈ 0.480409

b. Quasi-Newton Method:

25