Optimization

optimization typically deals with finding the best result or optimum

solution of a problem. Engineers must continuously design devices and products
that perform tasks in an efficient fashion. In doing so, they are constrained by the
limitations of the physical world. Also, they must keep cost down. Thus, they are
always confronting optimization problems that balance performance and
limitation such as
1- Design aircraft for minimum weight and maximum strength.
2- Design civil engineering structures for minimum cost.
3- Design pump and heat transfer equipment for maximum efficiency.
4- Maximize power output of electrical networks and machinery while
minimizing heat generation.
5- Statistical analysis and model with minimum error.
6- Design waste water treatment systems to meet water quality standards
at least cost.

Optimization is like root finding method

𝒇(𝒙) = 𝟎 𝒙 =? 𝒓𝒐𝒐𝒕 𝒇𝒊𝒏𝒅𝒊𝒏𝒈 𝒎𝒆𝒕𝒉𝒐𝒅
𝒇̅(𝒙) = 𝟎 𝒙 =? 𝒐𝒑𝒕𝒊𝒎𝒊𝒛𝒂𝒕𝒊𝒐𝒏 𝒎𝒆𝒕𝒉𝒐𝒅

In optimization we must specify three keys components: -

1- The objective function.
2- The process model.
3- The constraints.

Page | 1
 Minimize or maximize f(x) = objective function

Subject to h(x) = 0
h(x) = b Equality constraints

h(x) ≥ 0
h(x) ≤ b Inequality constraints

Optimization method according to the variables:-

1- Single variable optimization.
2- Multi-variables unconstraint optimization.
3- Multi-variables constraints optimization.
4- Linear programing.
5- Non-linear programing.
6- Dynamic programing.

The necessary condition for optimization:-

The condition of optimum is
𝝏𝒇𝒊
=𝟎
𝝏𝒙𝒊
𝑤ℎ𝑒𝑟𝑒 𝑖 = 1,2,3, … , 𝑛

➢ For one variable :-

𝜕2𝑓 Minimum point
>0
𝜕𝑥 2
𝜕2𝑓 Maximum point
<0
𝜕𝑥 2
𝜕2𝑓 Inflection point
=0
𝜕𝑥 2

Page | 2
➢ For two variables :-
𝝏𝟐 𝒇 𝝏𝟐 𝒇 𝝏𝟐 𝒇
A= B= C =
𝝏𝒙𝟐𝟏 𝝏𝒙𝟏 𝒙𝟐 𝝏𝒙𝟐𝟐

C*A – B2 C,A Optimum Type

+ + Minimum point
+ - Maximum point
- Without signal Saddle point
zero Without signal Intermediate point

Single or Multi-variables Un constrained Optimization:-

Optimize f(x)
Differentiate the the function f(x) and set the derivative equal to zero
𝝏𝒇 𝝏𝒇 𝝏𝒇
=𝟎 =𝟎 =𝟎
𝝏𝒙𝟏 𝝏𝒙𝟐 𝝏𝒙𝒏

Example:-
𝑂𝑝𝑡𝑖𝑚𝑖𝑧𝑒 𝒇(𝒙, 𝒚) = 𝒙𝟐 + 𝒙𝒚 + 𝒚𝟐 + 𝒚

find the optimum point? And prove it?

Solution:-
𝝏𝒇
= 𝟐 ∗ 𝒙 + 𝒚 = 𝟎 ⟹ 𝒚 = −𝟐 ∗ 𝒙
𝝏𝒙
𝝏𝒇 𝟏
= 𝒙 + 𝟐 ∗ 𝒚 + 𝟏 = 𝟎 ⟹ 𝒙 + 𝟐 ∗ (−𝟐𝒙) + 𝟏 = 𝟎 ⟹ 𝒙 = −
𝝏𝒚 𝟑
𝟐
∴𝒚=
𝟑
To find the optimum type:-
𝝏𝟐 𝒇 𝝏𝟐 𝒇 𝝏𝟐 𝒇
A= =𝟐 B= =𝟏 C = =𝟐
𝝏𝒙𝟐 𝝏𝒙𝒚 𝝏𝒚𝟐

C*A – B2 = + C,A = + Minimum point

Page | 3
Equality Constraints Optimization:-
For n independent variables and m equality constraints

𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒆 𝒚(𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝟑 )
𝒔𝒖𝒃𝒋𝒆𝒄𝒕 𝒕𝒐 𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝟑 ) = 𝟎

There are three methods to locating the optimum points of the function y.

A- Direct Methods:-
Substitute the constraints equation directly into the objective function to
be optimize.

B- Constraints Variation:-
The optimum point is found from the following equation

𝝏𝒚 𝝏𝒇 𝝏𝒚 𝝏𝒇
∗ − ∗ =𝟎
𝝏𝒙𝟏 𝝏𝒙𝟐 𝝏𝒙𝟐 𝝏𝒙𝟏

Example:-
𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒆 𝒚(𝒙𝟏 , 𝒙𝟐 ) = 𝒙𝟏 ∗ 𝒙𝟐
𝒔𝒖𝒃𝒋𝒆𝒄𝒕 𝒕𝒐 𝒇 (𝒙𝟏 , 𝒙𝟐 ) = 𝒙𝟐𝟏 + 𝒙𝟐𝟐 − 𝟏 = 𝟎
1- Find the value of x using direct optimization?
2- Find the value of x using constraints variation?
3- Is the point minimum or maximum?

Solution:-

1- 𝑭𝒓𝒐𝒎 𝒇 (𝒙𝟏 , 𝒙𝟐 ) ⟹ 𝒙𝟏 = √𝟏 − 𝒙𝟐𝟐 … (𝟏) 𝒔𝒖𝒃. 𝒊𝒏 𝒐𝒃𝒆𝒄𝒕𝒊𝒗𝒆 𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏

∴ 𝒚 = (√𝟏 − 𝒙𝟐𝟐 ) ∗ 𝒙𝟐

𝝏𝒚 𝟏 𝟏 𝟏
𝟐 𝟐 𝟐 −𝟐
= (𝟏 − 𝒙𝟐 ) ∗ 𝟏 + 𝒙𝟐 ∗ ( ∗ (𝟏 − 𝒙𝟐 ) ∗ (−𝟐 ∗ 𝒙𝟐 )) = 𝟎
𝝏𝒙𝟐 𝟐

𝟏 𝟏
∴ 𝒙𝟐 = ±√ 𝒇𝒊𝒏𝒂𝒍𝒍𝒚 𝒔𝒖𝒃 𝒊𝒏 𝒆𝒒(𝟏)𝒕𝒐 𝒈𝒆𝒕 𝒙𝟏 = ±√
𝟐 𝟐

Page | 4
 2- Solve using constraints variation:-
𝜕𝑦 𝜕𝑦
= 𝑥2 = 𝑥1
𝜕𝑥1 𝜕𝑥2
𝜕𝑓 𝜕𝑓
= 2𝑥1 = 2𝑥2
𝜕𝑥1 𝜕𝑥2

𝝏𝒚 𝝏𝒇 𝝏𝒚 𝝏𝒇
∗ − ∗ =𝟎
𝝏𝒙𝟏 𝝏𝒙𝟐 𝝏𝒙𝟐 𝝏𝒙𝟏

𝑥2 ∗ 2 ∗ 𝑥2 − 𝑥1 ∗ 2 ∗ 𝑥1 = 0 ⟹ 𝒙𝟐𝟐 − 𝒙𝟐𝟏 = 𝟎 … (1)

Which solve simultaneously with constraint equation
𝒙𝟐𝟏 + 𝒙𝟐𝟐 − 𝟏 = 𝟎 … (𝟐)
The result is

𝟏 𝟏
∴ 𝒙𝟐 = ±√ 𝒇𝒊𝒏𝒂𝒍𝒍𝒚 𝒔𝒖𝒃 𝒊𝒏 𝒆𝒒(𝟏)𝒕𝒐 𝒈𝒆𝒕 𝒙𝟏 = ±√
𝟐 𝟐

Home work:-
1- Is the point minimum or maximum for the above example?

2- Optimize the following function

𝒚(𝒙) = 𝟐𝒙𝟐 + 𝟓𝒙 + 𝟒

Page | 5
Chemical engineering Department Assist Lecturer Ghadeer Jassim

C- Lagrange Multiplier Method (Multi-varaiables) :-

For example two variables and one constraint

𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒆 𝒚(𝒙𝟏 , 𝒙𝟐 )
𝑺𝒖𝒃𝒋𝒆𝒄𝒕 𝒕𝒐 𝒇 (𝒙𝟏 , 𝒙𝟐 ) = 𝟎

We define 𝑳(𝒙𝟏 , 𝒙𝟐 , 𝝀) = 𝒚(𝒙𝟏 , 𝒙𝟐 ) + 𝝀𝒊 𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 )

Where 𝜆𝑖 𝑙𝑎𝑔𝑟𝑎𝑛𝑔𝑒 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑟

With the necessary conditions

𝝏𝑳 𝝏𝑳 𝝏𝑳
=𝟎 =𝟎 =𝟎
𝝏𝒙𝟏 𝝏𝒙𝟐 𝝏𝝀𝒊

Example:-
𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒆 𝒚(𝒙𝟏 , 𝒙𝟐 ) = 𝒙𝟏 ∗ 𝒙𝟐
𝑺𝒖𝒃𝒋𝒆𝒄𝒕 𝒕𝒐 𝒇 (𝒙𝟏 , 𝒙𝟐 ) = 𝒙𝟐𝟏 + 𝒙𝟐𝟐 − 𝟏 = 𝟎

Find the value of x’s using Lagrange Multiplier Method?

Solution:-

𝑳(𝒙𝟏 , 𝒙𝟐 , 𝝀) = 𝒙𝟏 ∗ 𝒙𝟐 + 𝝀 (𝒙𝟐𝟏 + 𝒙𝟐𝟐 − 𝟏)

𝝏𝑳
= 𝟎 ⟹ 𝒙𝟐 + 𝟐 ∗ 𝝀 ∗ 𝒙𝟏 = 𝟎 −−−𝟏
𝝏𝒙𝟏
𝝏𝑳
= 𝟎 ⟹ 𝒙𝟏 + 𝟐 ∗ 𝝀 ∗ 𝒙𝟐 = 𝟎 −−−𝟐
𝝏𝒙𝟐
𝝏𝑳
= 𝟎 ⟹ 𝒙𝟐𝟏 + 𝒙𝟐𝟐 − 𝟏 = 𝟎 −−−𝟑
𝝏𝝀
Solving these equations simultaneously gives

𝟏 𝟏 𝟏
𝒙𝟏 = √ 𝒙𝟐 = √ 𝝀= − 𝑴𝒂𝒙𝒊𝒎𝒖𝒎 𝒑𝒐𝒊𝒏𝒕
𝟐 𝟐 𝟐

Page | 6
Chemical engineering Department Assist Lecturer Ghadeer Jassim

𝟏 𝟏 𝟏
𝒙𝟏 = −√ 𝒙𝟐 = −√ 𝝀= − 𝑴𝒂𝒙𝒊𝒎𝒖𝒎 𝒑𝒐𝒊𝒏𝒕
𝟐 𝟐 𝟐

𝟏 𝟏 𝟏
𝒙𝟏 = −√ 𝒙𝟐 = √ 𝝀= 𝑴𝒂𝒏𝒊𝒎𝒖𝒎 𝒑𝒐𝒊𝒏𝒕
𝟐 𝟐 𝟐

𝟏 𝟏 𝟏
𝒙𝟏 = √ 𝒙𝟐 = −√ 𝝀= 𝑴𝒂𝒏𝒊𝒎𝒖𝒎 𝒑𝒐𝒊𝒏𝒕
𝟐 𝟐 𝟐

D- Inequality constraint optimization (Kuhn Tucker Method):-

For n independent variables and m inequality constraints
𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒆 𝒚(𝒙𝟏 , 𝒙𝟐 )
𝑺𝒖𝒃𝒋𝒆𝒄𝒕 𝒕𝒐 𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 ) ≥ 𝟎
𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 ) ≤ 𝑪

We define 𝑳(𝒙𝟏 , 𝒙𝟐 , 𝝁) = 𝒚(𝒙𝟏 , 𝒙𝟐 ) − 𝝁𝒊 𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 )

Where 𝜇𝑖 𝑖𝑠 𝐾𝑢ℎ𝑛 𝑇𝑢𝑐𝑘𝑒𝑟 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑟, with the necessary conditions

𝝏𝑳
=𝟎
𝝏𝒙𝟏

𝝏𝑳
=𝟎
𝝏𝒙𝟐

𝝁𝒊 [𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 ) − 𝑪] = 𝟎

𝝁𝒊 ≥ 𝟎

𝒇𝒊 (𝒙𝟏 , 𝒙𝟐 ) ≤ 𝑪

Example:-

Maximize [−(𝑥1 − 4)2 − (𝑥2 − 4)2 ]

Subject to 𝑥1 + 𝑥2 ≤ 4

3𝑥1 + 𝑥2 ≤ 9
Page | 7
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Solution:-
𝑳 = −(𝒙𝟏 − 𝟒)𝟐 − (𝒙𝟐 − 𝟒)𝟐 − 𝝁𝟏 (𝒙𝟏 + 𝒙𝟐 − 𝟒) − 𝝁𝟐 (𝟑𝒙𝟏 + 𝒙𝟐 − 𝟗)

𝝏𝑳
= 𝟎 ⟹ −𝟐(𝒙𝟏 − 𝟒) ∗ 𝟏 − 𝝁𝟏 ∗ 𝟏 − 𝝁𝟐 ∗ 𝟑 = 𝟎
𝝏𝒙𝟏
−𝟐(𝒙𝟏 − 𝟒) − 𝝁𝟏 − 𝟑𝝁𝟐 = 𝟎 − − − 𝒆𝒒(𝟏)

𝝏𝑳
= 𝟎 ⟹ −𝟐(𝒙𝟐 − 𝟒) ∗ 𝟏 − 𝝁𝟏 ∗ 𝟏 − 𝝁𝟐 ∗ 𝟏 = 𝟎
𝝏𝒙𝟐
−𝟐(𝒙𝟐 − 𝟒) − 𝝁𝟏 − 𝝁𝟐 = 𝟎 − − − 𝒆𝒒(𝟐)

𝝁𝟏 (𝒙𝟏 + 𝒙𝟐 − 𝟒) = 𝟎 − − − 𝒆𝒒(𝟑)

𝝁𝟐 (𝟑𝒙𝟏 + 𝒙𝟐 − 𝟗) = 𝟎 − − − 𝒆𝒒(𝟒)

𝝁𝟏 ≥ 𝟎 𝝁𝟐 ≥ 𝟎

𝑥1 + 𝑥2 ≤ 4

3𝑥1 + 𝑥2 ≤ 9

Case 1:-
𝝁𝟏 = 𝟎 𝝁𝟐 = 𝟎

From eq(1) ⟹ −𝟐(𝒙𝟏 − 𝟒) = 𝟎 ⟹ 𝒙𝟏 = 𝟒

From eq(2) ⟹ −𝟐(𝒙𝟐 − 𝟒) = 𝟎 ⟹ 𝒙𝟐 = 𝟒

Checking:-

𝑥1 + 𝑥2 ≤ 4 ⟹ 4 + 4 ≤ 4 ×

3𝑥1 + 𝑥2 ≤ 9 ⟹ 3 ∗ 4 + 4 ≤ 9 ×

Page | 8
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Case 2:-
𝝁𝟏 > 𝟎 𝝁𝟐 = 𝟎

From eq(1) ⟹ −𝟐(𝒙𝟏 − 𝟒) − 𝝁𝟏 = 𝟎 ∴ 𝝁𝟏 = −𝟐(𝒙𝟏 − 𝟒) − −𝒆𝒒(𝟒)

From eq(2) ⟹ −𝟐(𝒙𝟐 − 𝟒) − 𝝁𝟏 = 𝟎 ∴ 𝝁𝟏 = −𝟐(𝒙𝟐 − 𝟒) − −𝒆𝒒(𝟓)

From eq(3) ⟹ 𝝁𝟏 (𝒙𝟏 + 𝒙𝟐 − 𝟒) = 𝟎 ∴ 𝒙𝟏 = 𝟒 − 𝒙𝟐 − −𝒆𝒒(𝟔)

Solve simultaneously to get 𝒙𝟏 = 𝟐 𝒙𝟐 = 𝟐 𝝁𝟏 = 𝟒

Checking:-

𝑥1 + 𝑥2 ≤ 4 ⟹ 2 + 2 ≤ 4 ⟹ 4 = 4 𝑡𝑟𝑢𝑒

3𝑥1 + 𝑥2 ≤ 9 ⟹ 3 ∗ 2 + 2 ≤ 9 ⟹ 8 < 9 𝑡𝑟𝑢𝑒

𝜇1 ≥ 0 𝑡𝑟𝑢𝑒

Case 3:-
𝝁𝟏 = 𝟎 𝝁𝟐 > 𝟎
−𝟐(𝒙𝟏 −𝟒)
From eq(1) ⟹ −𝟐(𝒙𝟏 − 𝟒) − 𝟑𝝁𝟐 = 𝟎 ∴ 𝝁𝟐 = − −𝒆𝒒(𝟒)
𝟑

From eq(2) ⟹ −𝟐(𝒙𝟐 − 𝟒) − 𝝁𝟐 = 𝟎 ∴ 𝝁𝟐 = −𝟐(𝒙𝟐 − 𝟒) − −𝒆𝒒(𝟓)

From eq(3) ⟹ 𝝁𝟐 (𝟑𝒙𝟏 + 𝒙𝟐 − 𝟗) = 𝟎 ∴ 𝒙𝟐 = 𝟗 − 𝟑𝒙𝟏 − −𝒆𝒒(𝟔)

Solve simultaneously to get 𝒙𝟏 = 𝟏. 𝟗 𝒙𝟐 = 𝟑. 𝟑 𝝁𝟐 = 𝟏. 𝟒

Checking:-

𝑥1 + 𝑥2 ≤ 4 ⟹ 1.9 + 3.3 ≤ 4 ×

3𝑥1 + 𝑥2 ≤ 9 ⟹ 3 ∗ 1.9 + 3.3 ≤ 9 𝑡𝑟𝑢𝑒

𝜇2 ≥ 0 𝑡𝑟𝑢𝑒

Page | 9
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Home work:-
Case 4:-
𝝁𝟏 > 𝟎 𝝁𝟐 > 𝟎

Page | 10
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 1
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 2
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 3
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 4
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 5
-1 0 1 0 0 1 0
 -3

0 -2 2 -1 0 1 4

8
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 1
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 2
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 3
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 4
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 5
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 6
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 7
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 8
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 24
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 24
Chemical engineering Department Assist Lecturer Ghadeer Jassim

Page | 24
One dimensional optimization method: -

Quadratic Interpolation (Parabolic Interpolation): -

We start with three points x0, x1, and x2 in increasing order that might be equally
spaced, but the extreme points must bracket the minimum.

𝒇(𝒙) = 𝒂 + 𝒃𝒙 + 𝒄𝒙𝟐

we know that a quadratic function can be passed exactly through the three points, and
that the function can be differentiated and the derivative set equal to 0 to yield the
minimum of the approximating function
𝒃
𝒙∗ = −
𝟐𝒄

Suppose that f(x) is evaluated at x0, x1, and x2, to yield f(x0) =f0, f(x1) = f1, and
f(x2) = f2. The coefficients b and c can be evaluated from the solution of the three
equations

𝒇(𝒙𝟎) = 𝒂 + 𝒃𝒙𝟎 + 𝒄𝒙𝟐𝟎

𝒇(𝒙𝟏) = 𝒂 + 𝒃𝒙𝟏 + 𝒄𝒙𝟐𝟏
𝒇(𝒙𝟐) = 𝒂 + 𝒃𝒙𝟐 + 𝒄𝒙𝟐𝟐

∗
𝒃 𝒇𝟎 ∗ (𝒙𝟐𝟏 − 𝒙𝟐𝟐 ) + 𝒇𝟏 ∗ (𝒙𝟐𝟐 − 𝒙𝟐𝟎 ) + 𝒇𝟐 ∗ (𝒙𝟐𝟎 − 𝒙𝟐𝟏 )
𝒙 = − ( )
= 𝟎. 𝟓 ∗
𝟐𝒄 𝒇𝟎 ∗ (𝒙𝟏 − 𝒙𝟐 ) + 𝒇𝟏 ∗ (𝒙𝟐 − 𝒙𝟎 ) + 𝒇𝟐 ∗ (𝒙𝟎 − 𝒙𝟏 )

Example: -
Use parabolic interpolation to approximate the maximum of the following function

𝒙𝟐
𝒇(𝒙) = 𝟐 ∗ 𝒔𝒊𝒏(𝒙) −
𝟏𝟎
Given x0 = 0, x1=1, x2=4?

Solution: -
Iteration1:-
x0 = 0, x1=1, x 2= 4
f0 = 0, f1=1.583, f2= -3.3114

𝟎 ∗ (𝟏 − 𝟒𝟐 ) + 𝟏. 𝟓𝟖𝟑(𝟒𝟐 − 𝟎𝟐 ) + (−𝟑. 𝟏𝟏𝟑) ∗ (𝟎𝟐 − 𝟏𝟐 )

𝒙𝟑 = (𝟎. 𝟓) ∗ = 𝟏. 𝟓𝟎𝟔
𝟎 ∗ (𝟏 − 𝟒) + 𝟏. 𝟓𝟖𝟑 ∗ (𝟒 − 𝟎) + (−𝟑. 𝟏𝟏𝟑) ∗ (𝟎 − 𝟏)
Iteration2:-
x0 = 1, x1=1.506, x 2= 4
f0 = 1.583, f1=1.769, f2= -3.3114

𝟏. 𝟓𝟖𝟑 ∗ (𝟏. 𝟓𝟎𝟔𝟐 − 𝟒𝟐 ) + 𝟏. 𝟕𝟔𝟗(𝟒𝟐 − 𝟏𝟐 ) + (−𝟑. 𝟏𝟏𝟑) ∗ (𝟏𝟐 − 𝟏. 𝟓𝟎𝟔𝟐 )

𝒙𝟑 = (𝟎. 𝟓) ∗
𝟏. 𝟓𝟖𝟑 ∗ (𝟏. 𝟓𝟎𝟔 − 𝟒) + 𝟏. 𝟕𝟔𝟗 ∗ (𝟒 − 𝟏) + (−𝟑. 𝟏𝟏𝟑) ∗ (𝟏 − 𝟏. 𝟓𝟎𝟔)

= 𝟏. 𝟒𝟗𝟒

Iteration3:-
x0 = 1.494, x1=1.506, x2= 4
f0 = 1.771, f1=1.769, f2= -3.3114

𝒙𝟑 = 𝟏. 𝟑𝟗𝟑

Iteration4:-
x0 = 1.393, x1=1.506, x2= 4
f0 = 1.774, f1=1.769, f2= -3.3114

𝒙𝟑 = 𝟏. 𝟒𝟐𝟒

Newton’s method:-
To see what Newton's method implies about f(x), suppose f(x) is approximated
by a Taylor series as follow
𝟏
𝒇(𝒙) = 𝒇(𝒙𝒊 ) + 𝒇̅(𝒙𝒊 )(𝒙 − 𝒙𝒊 ) + 𝒇̿(𝒙𝒊 )(𝒙 − 𝒙𝒊 )𝟐
𝟐

Find df(x)/dx = 0

𝟏
𝟎 = 𝟎 + 𝒇̅(𝒙𝒊 ) ∗ 𝟏 + ∗ 𝟐 ∗ 𝒇̿(𝒙𝒊 ) ∗ (𝒙 − 𝒙𝒊 )
𝟐

𝒇̅(𝒙𝒊 )
𝒙𝒊+𝟏 = 𝒙𝒊 −
𝒇̿(𝒙𝒊 )

Example:-
Find the maximum of the following equation
𝒇(𝒙) = −𝟎. 𝟐𝟓 𝒙𝟒 + 𝟏. 𝟏 𝒙𝟑 − 𝟏. 𝟕𝟓𝒙𝟐 + 𝟐𝒙
By using newton’s method, given x0 =2.5 with 𝜺 = 𝟏%?
Solution: -
𝒇̅(𝒙) = − 𝒙𝟑 + 𝟑. 𝟑 𝒙𝟐 − 𝟑. 𝟓𝒙 + 𝟐
𝒇̿(𝒙) = − 𝟑𝒙𝟐 + 𝟔. 𝟔𝒙 − 𝟑. 𝟓

Iteration1:-
𝒇̅(𝒙𝟎 ) (− (𝟐. 𝟓)𝟑 + 𝟑. 𝟑 ∗ 𝟐. 𝟓𝟐 − 𝟑. 𝟓 ∗ 𝟐. 𝟓 + 𝟐)
𝒙𝟏 = 𝒙𝟎 − = 𝟐. 𝟓 −
𝒇̿(𝒙𝟎 ) (− 𝟑 ∗ 𝟐. 𝟓𝟐 + 𝟔. 𝟔 ∗ 𝟐. 𝟓 − 𝟑. 𝟓)

= 𝟐. 𝟏𝟗𝟓𝟔𝟓, 𝜺 = 𝟏𝟑. 𝟗%

Iteration2: -
𝒇̅(𝒙𝟏 )
𝒙𝟐 = 𝒙𝟏 −
𝒇̿(𝒙𝟏 )
(− (𝟐. 𝟏𝟗𝟓𝟔𝟓)𝟑 + 𝟑. 𝟑 ∗ 𝟐. 𝟏𝟗𝟓𝟔𝟓𝟐 − 𝟑. 𝟓 ∗ 𝟐. 𝟏𝟗𝟓𝟔𝟓 + 𝟐)
= 𝟐. 𝟏𝟗𝟓𝟔𝟓 −
(− 𝟑 ∗ 𝟐. 𝟏𝟗𝟓𝟔𝟓𝟐 + 𝟔. 𝟔 ∗ 𝟐. 𝟏𝟗𝟓𝟔𝟓 − 𝟑. 𝟓)

= 𝟐. 𝟎𝟗𝟏𝟕, 𝜺 = 𝟓%

Iteration3:-
𝒙𝟑 = 𝟐. 𝟎𝟕𝟗𝟓𝟏, 𝜺 = 𝟎. 𝟔%