OPTIMIZATION OF CHEMICAL

PROCESSES (CHE1011)

Dr. Dharmendra Kumar Bal

Assistant Professor (Sr.)
School of Chemical Engineering

11
 Gradient Search Methods
• Gradient search methods require the gradient vector of
the objective function.
• Consider a function f (X) where X =[x1, x2,…xn]T. The
gradient vector of f (X) is given by the partial derivatives
with respect to each of the independent variables.

• The gradient possess a very useful property.

• In an n-dimensional space if we move from any point along the
gradient direction, the value of f(X) increases at the fastest rate.
• Therefore, the gradient direction is known as the direction of steepest
ascent. Unfortunately, this steepest ascent direction is not a global
property and it is a local one as the shape of the contours is not
uniform.
 Gradient Search Methods

Figure shows that the direction of search changes from point to point.
At point 1, steepest direction is 1–2. Similarly, at point 2 and 3, steepest
directions are 2–3 and 3–4 respectively.
 Gradient Search Methods
 Gradient, f(X) is the direction of steepest increase.
 Search direction for minimizing f(X) is based on f(X).
 That is: s is selected such that Tf(X) s < 0
 Such s is a descent direction: f(X) decreases along s.

Several gradient search methods depending on s:

• Steepest Descent Method
• Newton Method In all methods, new estimate
is: X k+1 = Xk + k sk
• Levenberg-Marquardt Method
• Quasi-Newton Methods where  k is the step length

along search direction sk.

4
 Gradient Search Methods
STEEPEST DESCENT METHOD

Search direction in kth iteration: sk = - f(Xk)

and new estimate: Xk+1 = Xk - k f(Xk)

How do you find step length, k?

 Simple method; Line search is required.

 f(X) decreases in each iteration (Descent Property)
 For general functions, rapid progress when away from the
minimum but slow near the minimum

5
 Gradient Search Methods

Steepest descent
method can find the
minimum of certain
quadratics in one
iteration

6
 Gradient Search Methods

Steepest descent
method takes more
iterations to find the
minimum