Direct vs.

Indirect Methods
Direct Methods
Optimal Control Discretize the control problem, then apply NLP techniques to the resulting
Lecture 28: Indirect Solution Methods finite-dimensional optimization problem
A priori knowledge of the solution structure not required
Benoı̂t Chachuat <benoit@mcmaster.ca> But, approximate solution only (due to control parameterization)

Indirect Methods (or PMP-based Methods or Variational Methods)

Seek a solution to the (closed system of) necessary conditions of optimality
Department of Chemical Engineering Discretization of the control profile not needed
But, need to guess the optimal solution structure too
Spring 2009
Recommendation: First approximate the solution with a direct approach,
then refine this solution with an indirect approach!

Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 1/7 Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 2/7

Regular, Terminal-Constrained Problems Indirect Methods

Z tf
General Idea: Split the NCOs into 2 subsets:
min ℓ(t, x(t), u(t)) dt + φ(tf , x(tf )) Those NCOs that are enforced at each iteration
(u,tf )∈C [t0 ,T ]nu ×IR t0
The remaining NCOs that are modified, at each iteration, via
s.t. ẋ(t) = f (t, x(t), u(t)); x(t0 ) = x0 successive linearization, until convergence
ψk (tf , x(tf )) = 0, k = 1, . . . , nψ ◮ E.g., quasi-linearization, control vector iteration, indirect shooting

Necessary Conditions for (u∗ , tf∗ , x∗ , λ∗ , ν ∗ ) to be Optimal Indirect Shooting Approach

∆ 1 Guess the values of λ∗ (t0 ), ν ∗ and tf∗
Euler-Lagrange Equations (H = ℓ + λT f ):
2 Iteratively update these 
estimates to meet the transversal conditions
ẋ =Hλ, λ̇ = − Hx , 0 =Hu , t0 ≤ t ≤ tf T

λ + φx + ν k ψ x
∆
Transversal Conditions: F(λk0 , ν k , tfk ) = ψ −→ 0
 
 
T k T
ℓ + λ f + φt + ν ψ t t=t k
h i
T
[x − x0 ]t0 = 0, λ − φx + ν ψ x =0 f
tf
h i subject to: ẋ(t) = f (t, x(t), u(t)); x(t0 ) = x0
H + φt + ν T ψ t = 0, if tf is free T
λ̇(t) = −ℓx (t, x(t), u(t)) − λ(t) f x (t, x(t), u(t)); λ(t0 ) = λk0
tf
[ψ]tf = 0, and ψ satisfy a regularity condition 0 = ℓu (t, x(t), u(t)) + λ(t)T f u (t, x(t), u(t))

Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 3/7 Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 4/7
Indirect Shooting Approach: Algorithm Indirect Shooting Approach: Application
Initialization: Class Exercise: Consider the optimal control problem:
Z 1
Choose initial estimates λ00 , ν 0 , tf0 , termination tolerance ǫ > 0; Set k ← 0 ∆ 1 2
minimize: J(u) = 2 u(t) dt
Main Step: 0
subject to: ẋ(t) = u(t)[1 − x(t)]; x(0) = −1; x(1) = 0
1 Calculate the defect F(λk0 , ν k , tfk ); If kF(λk0 , ν k , tfk )k < ǫ, Stop
2 Calculate the defect gradient ∇λ0 F(λk0 , ν k , tfk ), ∇ν F(λk0 , ν k , tfk ) and
1 Formulate the defect F(λk0 , ν k , tfk ) in the indirect shooting approach
∇tf F(λk0 , ν k , tfk )
3 Calculate the search direction via solution of the linear system,
 T T 
∇λ0 F(λk0 , ν k , tfk ) dkλ

T
= −F(λk0 , ν k , tfk )
  k 
 ∇ν F(λk0 , ν k , tfk ) dν


∇tf F(λk0 , ν k , tfk )
T dtkf

4 Update the estimates,

λk+1
0 ← λk0 + dkλ , ν k+1 ← ν k + dkν , tfk+1 ← tfk + dtkf
5 Increment k ← k + 1, and return to step 1
Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 5/7 Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 6/7

Indirect Shooting Approach: Application Indirect Shooting Approach: Remarks

Class Exercise: Consider the optimal control problem: The methods applies readily in the case of a fixed terminal time, or in
Z 1 the absence of terminal constraints
∆ 1 2 Keep only the relevant transversal conditions!
minimize: J(u) = 2 u(t) dt
◮

0
subject to: ẋ(t) = u(t)[1 − x(t)]; x(0) = −1; x(1) = 0 The defect gradient ∇λ0 F(λk0 , ν k , tfk ), ∇ν F(λk0 , ν k , tfk ) and
∇tf F(λk0 , ν k , tfk ) is needed to perform the Newton iteration
0 λ0 (1) − ν 0 = 0 0.8
x 0 (1) = 1 ◮ Use forward or reverse sensitivity analysis
0.6

λ1 (1) ν1
− = −1.62 × 10−1
x⋆ ◮ Or, apply a quasi-Newton method with Broyden update scheme to
λ⋆
-0.2
u(t), x(t), λ(t)

x 1 (1) = −2.13 × 10−1 0.4

0.2 u⋆ estimate the gradient

-0.4

Difficulty 1. Find good initial estimates (λ00 , ν 0 , tf0 ) of (λ∗0 , ν ∗ , tf∗ )

0
ν

-0.2
-0.6 λ2 (1) − ν 2 = −3.45 × 10−2
x 2 (1) = −1.76 × 10−2
λ3 (1) − ν 3 = −3.95 × 10−4 -0.4
◮ Very high sensitivity towards the transversal conditions
x 3 (1) = −1.51 × 10−4
-0.8 λ(1) − ν = 0 -0.6 ◮ Calculate good estimates via a direct discretization approach!
x(1) = 0 -0.8
iterates
-1
-0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0
-1
0 0.2 0.4 0.6 0.8 1
Difficulty 2. Hard to apply in the presence of singular arcs and/or
λ(0) t path constraints
◮ Optimal arc sequence not known a priori
◮ Optimality conditions at junction times give rise to multi-point
boundary value problems (MPBVP)
Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 6/7 Benoı̂t Chachuat (McMaster University) Indirect Methods Optimal Control 7/7