## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

**MATLAB — Indirect methods
**

Xuezhong Wang

∗

1 Introduction

The theory of optimal control has been well developed for over forty years.

With the advances of computer technique, optimal control is now widely

used in multi-disciplinary applications such as biological systems, communi-

cation networks and socio-economic systems etc. As a result, more and more

people will beneﬁt greatly by learning to solve the optimal control prob-

lems numerically. Realizing such growing needs, books on optimal control

put more weight on numerical methods. In retrospect, [1] was the ﬁrst and

the “classic” book for studying the theory as well as many interesting cases

(time-optimal, fuel-optimal and linear quadratic regulator(LQR) problems).

Necessary conditions for various systems were derived and explicit solutions

were given when possible. Later, [2] proved to be a concise yet excellent book

with more engineering examples. One of the distinguish features of this book

is that it introduced several iterative algorithms for solving problems numer-

ically. More recently, [3] uses MATLAB to solve problems which is easier

and more precise. However, the numerical methods covered in these books

are insuﬃcient for the wide range of problems emerging from various ﬁelds.

Especially, for those problems with free ﬁnal time and nonlinear dynamics.

This tutorial shows common routines in MATLAB to solve both ﬁxed and

free ﬁnal time problems. Speciﬁcally, the following subjects are discussed

with examples:

1. How to use Symbolic Math Toolbox to derive necessary conditions and

solve for explicit solutions?

2. How to solve a ﬁxed-ﬁnal-time optimal control problem with steepest

descent method?

∗

ISE. Dept., NCSU, Raleigh, NC 27695 (xwang10@ncsu.edu)

1

3. How to reformulate the original problem as a Boundary Value Problem

(BVP) and solve it with bvp4c?

4. How to get ‘good enough’ solutions with bvp4c and under what condi-

tions will bvp4c fail to ﬁnd a solution?

It should be noted that all the routines (except the steepest descent

method) discussed in this tutorial belong to the “indirect methods” category.

This means that constraints on the controls and states are not considered.

1

In other words, the control can be solved in terms of states and costates and

the problem is equivalently to a BVP. The reference of all the examples used

in this tutorial are stated such that the results can be compared and veriﬁed.

2 Optimal control problems with fixed-final-

time

In most books [1] [2], it is free-ﬁnal-time problem that being tackled ﬁrst to

derive the necessary conditions for optimal control. Fixed-ﬁnal-time prob-

lems were treated as an equivalent variation with one more state for time.

However, for numerical methods, ﬁxed-ﬁnal-time problems are the general

form and we solve the free-ﬁnal-time problem by converting it into a ﬁxed-

ﬁnal-time problem. The reason is simple: when dealing with optimal control

problems, it is inevitable to do numerical integration (either by indirect or

direct methods). Therefore, a time interval must be speciﬁed for these meth-

ods.

The ﬁrst problem is from [2], Example 5.1-1 from page 198 to 202. The

routine deployed in this example shows how to derive necessary conditions

and solve for the solutions with MATLAB Symbolic Math Toolbox.

Example 1 The system

˙ x

1

(t) = x

2

(t) (1)

˙ x

2

(t) = −x

2

(t) + u(t) (2)

(a) Consider the performance measure:

J(u) =

_

t

f

0

1

2

u

2

(t) dt

1

The last example implements a simple constraint on the control to show limitations

of bvp4c solver.

2

with boundary conditions:

x(0) = 0 x(2) =

_

5 2

¸

T

(b) Consider the performance measure:

J(u) =

1

2

(x

1

(2) − 5)

2

+

1

2

(x

2

(2) − 2)

2

+

_

2

0

1

2

u

2

(t) dt

with boundary conditions:

x(0) = 0 x(2) is free

(c) Consider the performance measure as in (a) with following boundary con-

ditions:

x(0) = 0 x

1

(2) + 5x

2

(2) = 15

The ﬁrst step is to form the Hamiltonian and apply necessary conditions for

optimality. With MATLAB, this can be done as follows:

% State equations

syms x1 x2 p1 p2 u;

Dx1 = x2;

Dx2 = -x2 + u;

% Cost function inside the integral

syms g;

g = 0.5*u^2;

% Hamiltonian

syms p1 p2 H;

H = g + p1*Dx1 + p2*Dx2;

% Costate equations

Dp1 = -diff(H,x1);

Dp2 = -diff(H,x2);

% solve for control u

du = diff(H,u);

sol_u = solve(du, ’u’);

The MATLAB commands we used here are diff and solve. diff diﬀeren-

tiates a symbolic expression and solve gives symbolic solution to algebraic

3

equations. For more details about Symbolic Math Toolbox, please refer to

[5]. Applying the necessary conditions for optimality we get two equations:

2

˙ p

∗

i

= −

∂H

∂x

∗

i

(3)

∂H

∂u

∗

= 0 (4)

The ﬁrst equation gives costate equations. From the second equation, we

solve for control u in terms of states and costates. The second step is to

substitute u from (4) back to the state and costate equations to get a set of

2n ﬁrst-order ordinary diﬀerential equations (ODE’s). A solution (with 2n

arbitrary coeﬃcients) can be obtained by using the dsolve command without

any boundary conditions. The symbolic solution looks diﬀerent from the one

in [2]. By simplifying the expression and rearrange the arbitrary coeﬃcients,

it is not diﬃcult to see that the two are the same.

% Substitute u to state equations

Dx2 = subs(Dx2, u, sol_u);

% convert symbolic objects to strings for using ’dsolve’

eq1 = strcat(’Dx1=’,char(Dx1));

eq2 = strcat(’Dx2=’,char(Dx2));

eq3 = strcat(’Dp1=’,char(Dp1));

eq4 = strcat(’Dp2=’,char(Dp2));

sol_h = dsolve(eq1,eq2,eq3,eq4);

As stated in [2], the diﬀerences of the three cases in this problem are

merely the boundary conditions. For (a), the arbitrary coeﬃcients can be

determined by supplying the 2n boundary conditions to dsovle:

% case a: (a) x1(0)=x2(0)=0; x1(2) = 5; x2(2) = 2;

conA1 = ’x1(0) = 0’;

conA2 = ’x2(0) = 0’;

conA3 = ’x1(2) = 5’;

conA4 = ’x2(2) = 2’;

sol_a = dsolve(eq1,eq2,eq3,eq4,conA1,conA2,conA3,conA4);

Again the solutions given by MATLAB and [2] look diﬀerent from each

other. Yet Figure 1 shows that the two are in fact equivalent. For all the

ﬁgures in this problem, * represent state trajectory from [2] while symbolic

solution from MATLAB is plotted with a continuous line.

2

We use * to indicate the optimal state trajectory or control.

4

Figure 1: Trajectories for Example 1 (a)

For case (b), we substitute t

0

= 0, t

f

= 2 in the the general solution

to get a set of 4 algebraic equations with 4 arbitrary coeﬃcients. Then

the four boundary conditions are supplied to determine these coeﬃcients.

sol_b is a structure consists of four coeﬃcients returned by solve. We get

the ﬁnal symbolic solution to the ODE’s by substituting these coeﬃcients

into the general solution. Figure 2 shows the results of both solutions from

MATLAB and [2].

It should be noted that we cannot get the solution by directly supplying

the boundary conditions with dsolve as we did in case (a). Because the

latter two boundary conditions: p

∗

1

(2) = x

∗

1

(t) −5, p

∗

2

(2) = x

∗

2

(t) −2 replaces

costates p

1

, p

2

with states x

1

, x

2

in the ODE’s which resulted in more ODE’s

than variables.

% case b: (a) x1(0)=x2(0)=0; p1(2) = x1(2) - 5; p2(2) = x2(2) -2;

eq1b = char(subs(sol_h.x1,’t’,0));

eq2b = char(subs(sol_h.x2,’t’,0));

eq3b = strcat(char(subs(sol_h.p1,’t’,2)),...

’=’,char(subs(sol_h.x1,’t’,2)),’-5’);

eq4b = strcat(char(subs(sol_h.p2,’t’,2)),...

’=’,char(subs(sol_h.x2,’t’,2)),’-2’);

sol_b = solve(eq1b,eq2b,eq3b,eq4b);

5

Figure 2: Trajectories for Example 1 (b)

% Substitute the coefficients

C1 = double(sol_b.C1);

C2 = double(sol_b.C2);

C3 = double(sol_b.C3);

C4 = double(sol_b.C4);

sol_b2 = struct(’x1’,{subs(sol_h.x1)},’x2’,{subs(sol_h.x2)}, ...

’p1’,{subs(sol_h.p1)},’p2’,{subs(sol_h.p2)});

Case (c) is almost the same as case (b) with slightly diﬀerent bound-

ary conditions. From [2], the boundary conditions are: x

∗

1

(2) + 5x

∗

2

(2) =

15, p

∗

2

(2) = 5p

∗

1

(2). Figure 3 shows the results of both solutions from MAT-

LAB and [2].

% case c: x1(0)=x2(0)=0;x1(2)+5*x2(2)=15;p2(2)= 5*p1(2);

eq1c = char(subs(sol_h.x1,’t’,0));

eq2c = char(subs(sol_h.x2,’t’,0));

eq3c = strcat(char(subs(sol_h.p2,’t’,2)),...

’-(’,char(subs(sol_h.p1,’t’,2)),’)*5’);

eq4c = strcat(char(subs(sol_h.x1,’t’,2)),...

’+(’,char(subs(sol_h.x2,’t’,2)),’)*5-15’);

sol_c = solve(eq1c,eq2c,eq3c,eq4c);

% Substitute the coefficients

C1 = double(sol_c.C1);

6

Figure 3: Trajectories for Example 1 (c)

C2 = double(sol_c.C2);

C3 = double(sol_c.C3);

C4 = double(sol_c.C4);

sol_c2 = struct(’x1’,{subs(sol_h.x1)},’x2’,{subs(sol_h.x2)}, ...

’p1’,{subs(sol_h.p1)},’p2’,{subs(sol_h.p2)});

If the state equations are complicated, it is usually impossible to derive

explicit solutions. We depend on numerical methods to ﬁnd the solutions. In

the next example, we will illustrate two numerical routines: steepest descent

method and convert to a BVP. The problem can be found in [2] page 338 to

339, Example 6.2-2.

Example 2 The state equations for a continuous stirred-tank chemical re-

actor are given as:

_

_

_

˙ x

1

(t) = −2[x

1

(t) + 0.25] + [x

2

(t) + 0.5]exp

_

25x

1

(t)

x

1

(t)+2

_

− [x

1

(t) + 0.25]u(t)

˙ x

2

(t) = 0.5 − x

2

(t) − [x

2

(t) + 0.5]exp

_

25x

1

(t)

x

1

(t)+2

_

(5)

The ﬂow of a coolant through a coil inserted in the reactor is to control

the ﬁrst-order, irreversible exothermic reaction taking place in the reactor.

x

1

(t) = T(t) is the deviation from the steady-state temperature and x

2

(t) =

7

C(t) is the deviation from the steady-state concentration. u(t) is the normal-

ized control variable, representing the eﬀect of coolant ﬂow on the chemical

reaction.

The performance measure to be minimized is:

J =

_

0.78

0

[x

2

1

(t) + x

2

2

(t) + Ru

2

(t)] dt,

with the boundary conditions

x(0) =

_

0.05 0

¸

T

, x(t

f

) is free

First, we will implement the steepest descent method based on the scheme

outlined in [2]. The algorithm consists of 4 steps:

1. Subdivide the interval [t

0

, t

f

] into N equal subintervals and assume

a piecewise-constant control u

(0)

(t) = u

(0)

(t

k

), t ∈ [t

k

, t

k+1

] k =

0, 1, · · · , N − 1

2. Applying the assumed control u

(i)

to integrate the state equations from

t

0

to t

f

with initial conditions x(t

0

) = x

0

and store the state trajectory

x

(i)

.

3. Applying u

(i)

and x

(i)

to integrate costate equations backward, i.e.,

from [t

f

, t

0

]. The “initial value” p

(i)

(t

f

) can be obtained by:

p

(i)

(t

f

) =

∂h

∂x

_

x

(i)

(t

f

)

_

.

Evaluate ∂H

(i)

(t)/∂u, t ∈ [t

0

, t

f

] and store this vector.

4. If

_

_

_

_

∂H

(i)

∂u

_

_

_

_

≤ γ (6)

_

_

_

_

∂H

(i)

∂u

_

_

_

_

2

≡

_

t

f

t

0

__

_

_

_

∂H

(i)

∂u

_

_

_

_

_

T

__

_

_

_

∂H

(i)

∂u

_

_

_

_

_

dt (7)

then stop the iterative procedure. Here γ is a preselected small positive

constant used as a tolerance.

If (6) is not satisﬁed, adjust the piecewise-constant control function by:

u

(i+1)

(t

k

) = u

(i)

(t

k

) − τ

∂H

(i)

∂u

(t

k

), k = 0, 1, · · · , N − 1

Replace u

(i)

by u

(i+1)

and return to step 2. Here, τ is the step size.

8

The main loop in MATLAB is as follows:

for i = 1:max_iteration

% 1) start with assumed control u and move forward

[Tx,X] = ode45(@(t,x) stateEq(t,x,u,Tu), [t0 tf], ...

initx, options);

% 2) Move backward to get the trajectory of costates

x1 = X(:,1); x2 = X(:,2);

[Tp,P] = ode45(@(t,p) costateEq(t,p,u,Tu,x1,x2,Tx), ...

[tf t0], initp, options);

p1 = P(:,1);

% Important: costate is stored in reverse order. The dimension of

% costates may also different from dimension of states

% Use interploate to make sure x and p is aligned along the time axis

p1 = interp1(Tp,p1,Tx);

% Calculate deltaH with x1(t), x2(t), p1(t), p2(t)

dH = pH(x1,p1,Tx,u,Tu);

H_Norm = dH’*dH;

% Calculate the cost function

J(i,1) = tf*(((x1’)*x1 + (x2’)*x2)/length(Tx) + ...

0.1*(u*u’)/length(Tu));

% if dH/du < epslon, exit

if H_Norm < eps

% Display final cost

J(i,1)

break;

else

% adjust control for next iteration

u_old = u;

u = AdjControl(dH,Tx,u_old,Tu,step);

end;

end

Because the step size of ode45 is not predetermined, interpolation is used to

make sure x(t), p(t) and u(t) are aligned along the time axis.

% State equations

function dx = stateEq(t,x,u,Tu)

dx = zeros(2,1);

u = interp1(Tu,u,t); % Interploate the control at time t

9

dx(1) = -2*(x(1) + 0.25) + (x(2) + 0.5)*exp(25*x(1)/(x(1)+2)) ...

- (x(1) + 0.25).*u;

dx(2) = 0.5 - x(2) -(x(2) + 0.5)*exp(25*x(1)/(x(1)+2));

% Costate equations

function dp = costateEq(t,p,u,Tu,x1,x2,xt)

dp = zeros(2,1);

x1 = interp1(xt,x1,t); % Interploate the state varialbes

x2 = interp1(xt,x2,t);

u = interp1(Tu,u,t); % Interploate the control

dp(1) = p(1).*(u + exp((25*x1)/(x1 + 2)).*((25*x1)/(x1 + 2)^2 - ...

25/(x1 + 2))*(x2 + 1/2) + 2) - ...

2*x1 - p(2).*exp((25*x1)/(x1 + 2))*((25*x1)/(x1 + 2)^2 - ...

25/(x1 + 2))*(x2 + 1/2);

dp(2) = p(2).*(exp((25*x1)/(x1 + 2)) + 1) - ...

p(1).*exp((25*x1)/(x1 + 2)) - 2*x2;

In step 4, we calculate ∂H/∂u|

t

= 2Ru(t) − p

1

(t)[x

1

(t) + 0.25] on each time

subinterval (function pH) and compare ∂H/∂u

2

with the preselected γ.

If ∂H/∂u

2

> γ, adjust u (function AdjControl).

% Partial derivative of H with respect to u

function dH = pH(x1,p1,tx,u,Tu)

% interploate the control

u = interp1(Tu,u,tx);

R = 0.1;

dH = 2*R*u - p1.*(x1 + 0.25);

% Adjust the control

function u_new = AdjControl(pH,tx,u,tu,step)

% interploate dH/du

pH = interp1(tx,pH,tu);

u_new = u - step*pH;

The step size τ is set as a constant in this example by some post hoc

investigation. A better strategy is to select τ with a line search method which

will maximize the reduction of performance measure with given ∂H

(i)

/∂u in

each iteration. Figure 4 shows the optimal state trajectory and control over

the time. The value of performance measure as a function of iteration number

is shown in Figure 5. In [2], two more numerical algorithms were introduced:

variation of extremals and quasilinearization. These two methods basically

reformulate and solve the original problem as a Boundary Value Problem

(BVP). In MATLAB, a BVP is typically solved with bvp4c. [4] is an excellent

reference on using bvp4c. For ﬁx-ﬁnal-time problems, u can always be solved

10

Figure 4: Example 2 Steepest descent method

Figure 5: Performance measure reduction

with respect to x and p by applying the necessary conditions (3) and (4).

And we will have 2n ODE’s and 2n boundary conditions.

11

Figure 6: Solution from bfv4c for Problem 2

% Initial guess for the solution

solinit = bvpinit(linspace(0,0.78,50), ...

[0 0 0.5 0.5]);

options = bvpset(’Stats’,’on’,’RelTol’,1e-1);

global R;

R = 0.1;

sol = bvp4c(@BVP_ode, @BVP_bc, solinit, options);

t = sol.x;

y = sol.y;

% Calculate u(t) from x1,x2,p1,p2

ut = (y(3,:).*(y(1,:) + 1/4))/(2*0.1);

n = length(t);

% Calculate the cost

J = 0.78*(y(1,:)*y(1,:)’ + y(2,:)*y(2,:)’ + ...

ut*ut’*0.1)/n;

The ODE’s and boundary conditions are two major considerations in

using bvp4c. A rule of thumb is that the number of ODE’s must equal to the

number of boundary conditions such that the problem is solvable. Once the

optimal control problem is converted to a BVP, it is very simple to solve.

%------------------------------------------------

% ODE’s for states and costates

12

function dydt = BVP_ode(t,y)

global R;

t1 = y(1)+.25;

t2 = y(2)+.5;

t3 = exp(25*y(1)/(y(2)+2));

t4 = 50/(y(1)+2)^2;

u = y(3)*t1/(2*R);

dydt = [-2*t1+t2*t3-t2*u

0.5-y(2)-t2*t3

-2*y(1)+2*y(3)-y(3)*t2*t4*t3+y(3)*u+y(4)*t2*t4*t3

-2*y(2)-y(3)*t3+y(4)*(1+t3)];

% -----------------------------------------------

% The boundary conditions:

% x1(0) = 0.05, x2(0) = 0, tf = 0.78, p1(tf) = 0, p2(tf) = 0;

function res = BVP_bc(ya,yb)

res = [ ya(1) - 0.05

ya(2) - 0

yb(3) - 0

yb(4) - 0 ];

In this example, bvp4c works perfectly. It is faster and gives better re-

sults, i.e. a smaller performance measure J comparing to the steepest descent

method (see Figure 6). In the following section, we will solely use bvp4c when

numerical solutions are needed.

3 Optimal control problems with free-final-

time

Now we are prepared to deal with free-ﬁnal-time problems. We will use both

Symbolic Math Toolbox and bvp4c in the next example, which can be ﬁnd

from [3] on page 77, Example 2.14.

Example 3 Given a double integral system as:

˙ x

1

(t) = x

2

(t) (8)

˙ x

2

(t) = u(t) (9)

The performance measure is:

J =

1

2

_

t

f

0

u

2

(t)dt

13

ﬁnd the optimal control given the boundary conditions as:

x(0) =

_

1 2

¸

T

, x

1

(t

f

) = 3, x

2

(t

f

) is free

To use the Symbolic Math Toolbox, the routine is very similar to Problem

1. We ﬁrst supply the ODE’s and boundary conditions on states and costates

to dsolve. The only diﬀerence is that the ﬁnal time t

f

itself is now a variable.

As a result, the solution is a function of t

f

. Next, we introduce four more

variables, namely x

1

(t

f

), x

2

(t

f

), p

1

(t

f

), p

2

(t

f

) into the solution obtained

above. With one additional boundary condition from

H (x

∗

(t

f

), u

∗

(t

f

), p

∗

(t

f

), t

f

) +

∂h

∂t

(x

∗

(t

f

), t

f

) = 0

For this problem, h ≡ 0 and we have p

1

(t

f

)x

2

(t

f

) − 0.5p

2

2

(t

f

) = 0. Now we

have 5 algebraic equations with 5 unknowns. And solve comes in handy

to solve this problem. Figure 7 shows the results from MATLAB and the

analytical solution in [3]. Although Symbolic Math Toolbox works ﬁne in this

example, it should be pointed out that in most problems, it is impossible to

get explicit solutions. For example, there is no explicit solutions for Example

1 even though the state equations are similar to those of Example 3.

sol = dsolve(’Dx1 = x2, Dx2 = -p2, Dp1 = 0, Dp2 = -p1’,...

’x1(0) = 1, x2(0) = 2, x1(tf) = 3, p2(tf) = 0’);

eq1 = subs(sol.x1) - ’x1tf’;

eq2 = subs(sol.x2) - ’x2tf’;

eq3 = subs(sol.p1) - ’p1tf’;

eq4 = subs(sol.p2) - ’p2tf’;

eq5 = sym(’p1tf*x2tf - 0.5*p2tf^2’);

%%

sol_2 = solve(eq1, eq2, eq3, eq4, eq5);

tf = sol_2.tf;

x1tf = sol_2.x1tf;

x2tf = sol_2.x2tf;

x1 = subs(sol.x1);

x2 = subs(sol.x2);

p1 = subs(sol.p1);

p2 = subs(sol.p2);

Because of the limitations of symbolic method, numerical methods are

more useful in dealing with more general problems. However, when we try

to use a numerical method such as bvp4c, we immediately encountered with

14

Figure 7: Example 3 Symbolic method

a problem: the time interval is not known. One common treatment [4] [7]

for such a situation is to change the independent variable t to τ = t/T, the

augmented state and costate equations will then become

˙

˜ x = Tf(x, q, τ).

3

Now the problem is posed on ﬁxed interval [ 0, 1 ]. This can be implemented

in bvp4c by treating T as an auxiliary variable. The following code snippet

shows the details.

solinit = bvpinit(linspace(0,1),[2;3;1;1;2]);

sol = bvp4c(@ode, @bc, solinit);

y = sol.y;

time = y(5)*sol.x;

ut = -y(4,:);

% -------------------------------------------------------------------------

% ODE’s of augmented states

function dydt = ode(t,y)

dydt = y(5)*[ y(2);-y(4);0;-y(3);0 ];

% -------------------------------------------------------------------------

% boundary conditions: x1(0)=1;x2(0)=2, x1(tf)=3, p2(tf)=0;

3

f denotes the ODE’s for state and costates.

15

% p1(tf)*x2(tf)-0.5*p2(2)^2

function res = bc(ya,yb)

res = [ ya(1) - 1; ya(2) - 2; yb(1) - 3; yb(4);

yb(3)*yb(2)-0.5*yb(4)^2];

Alternatively, we can accomplish this by treating T as a parameter for

bvp4c [4] The diﬀerence between the two lies in the parameter list of bvpinit,

and function deﬁnition of the ODE’s and boundary conditions. Figure 8

shows the result from numerical method which is the same as the analytical

solution.

solinit = bvpinit(linspace(0,1),[2;3;1;1],2);

sol = bvp4c(@ode, @bc, solinit);

y = sol.y;

time = sol.parameters*sol.x;

ut = -y(4,:);

% -----------------------------------------------------------

% ODE’s of augmented states

function dydt = ode(t,y,T)

dydt = T*[ y(2);-y(4);0;-y(3) ];

% -----------------------------------------------------------

% boundary conditions: x1(0)=1;x2(0)=2, x1(tf)=3, p2(tf)=0;

% p1(tf)*x2(tf)-0.5*p2(2)^2

function res = bc(ya,yb,T)

res = [ ya(1) - 1; ya(2) - 2; yb(1) - 3; yb(4);

yb(3)*yb(2)-0.5*yb(4)^2];

Now it is the time to talk about the limitations of bvp4c. Thought it

works well so far, we must bear in mind that the quality of the solution from

bvp4c is heavily dependent on the initial guess. A bad initial guess may

result in inaccurate solutions, or no solutions, or solutions which make no

sense. For example, you may try the initial guess p

1

(0) = 0 and compare the

results with the analytical solution to see the diﬀerence. By looking at the

ODE’s closely, we ﬁnd that ˙ p

1

(t) = 0. When supplied with an initial guess

of 0, bvp4c fails to ﬁnd the solution due to the state singularity.

The last problem is to further illustrate the capability, as well as the

limitation, of bvp4c in solving optimal control problems. This example can

be ﬁnd from [6] which is a time-optimal problem for the double integral

system with a simple control constraint.

16

Figure 8: Example 3 Numerical method

Example 4 Given a double integral system as:

˙ x

1

(t) = x

2

(t) (10)

˙ x

2

(t) = u(t) (11)

Minimize the ﬁnal time:

T =

_

t

f

0

dt

to drive the states to the origin:

x(0) =

_

2 2

¸

T

, x(t

f

) =

_

0 0

¸

T

, |u| ≤ 1

To formulate this problem as a BVP, control domain smoothing technique

must be applied ﬁrst [6]. The resulted ODE’s are listed below:

˙ x

1

= x

5

_

x

2

+

µx

3

_

µx

2

3

+ x

2

4

_

(12)

˙ x

2

= x

5

_

x

4

_

µx

2

3

+ x

2

4

_

(13)

˙ x

3

= 0 (14)

˙ x

4

= −x

5

x

3

(15)

˙ x

5

= 0 (16)

17

The boundary conditions are:

x

1

(0) = 2, x

2

(0) = 2, x

1

(1) = 0, x

2

(1) = 0, x

3

(1)

2

+ x

4

(1)

2

= 1.

Auxiliary variable x

5

= T, and u = x

4

/

_

µx

2

3

+ x

2

4

. It is well known that

the optimal control for this problem is the piecewise continuous “bang-bang”

control, which means that u will take either the maximum or the minimum

value within the admissible control set, i.e. +1 or −1 in this problem. Because

it is a linear second order system, there will be only one switch point and u

is not continuous at the switch point. The MATLAB code is as follows

global mu;

mu = 0.5;

solinit = bvpinit(linspace(0,1,10),[2;2;0;-1],4);

sol = bvp4c(@ode,@bc,solinit);

...

% the solution for one value of mu is used as guess for the next.

for i=2:3

if i==2

mu = 0.3;

else

mu = .1;

end;

% After creating function handles, the new value of mu

% will be used in nested functions.

sol = bvp4c(@ode,@bc,sol);

...

end

% -----------------------------------------------------------

function dydt = ode(t,y,T)

global mu;

term = sqrt(mu*y(3)^2+y(4)^2);

dydt = T*[ y(2) + mu*y(3)/term

y(4)/term

0

-y(3)];

% ------------------------------------------------------------

% boundary conditions, with 4 states and 1 parameters, 5 conditions are

% needed: x1(0) =2, x2(0) = 2; x1(1) = 0; x2(1) = 0; x3(1)^2+x4(1)^2 = 1;

function res = bc(ya,yb,T)

res = [ya(1)-2

ya(2)-2

18

Figure 9: Example 4 State trajectory

yb(1)

yb(2)

yb(3)^2+yb(4)^2-1];

As µ → 0, u appears to be more and more stiﬀ at the switch point,

approximating the “bang-bang” control. Three values were tested: µ =

0.5, µ = 0.3, µ = 0.1. We uses continuation to solve this problem, by which

means the solution for µ = 0.5 is used as guess for the BVP with µ = 0.3.

The trajectory and control corresponding to diﬀerent µ values are shown in

Figure 9 and Figure 10.

We can clearly see that u is approximating a “bang-bang” control at

switch point t = 4 and the state trajectory resembles the optimal trajectory

more and more closely as µ decrease. However, we cannot keep decreasing µ

to get xand u arbitrarily close to the optimal x

∗

and u

∗

. For example, if we

set µ = 10

−3

or smaller, bvp4c will fail due to the state singularity. In other

words, when the optimal control is piecewise continuous, bvp4c will fail at

discontinuous points.

19

Figure 10: Example 4 Control

4 Discussion

For many optimal control problems, bvp4c is the best option we have. Once

the problem is reformatted into a BVP, bvp4c can usually solve it eﬃciently.

However, we must pay attention to the limitations of bvp4c, more speciﬁcally

• Good initial guess is important to get an accurate solution, if a solution

exists.

• Discontinuous points (either in state/costate equations or control func-

tion) will cause bvp4c to fail.

For optimal control problems with no constraints on states or control, the

discontinuity is not that severe a problem. By applying the minimum prin-

ciple (4), we can convert the problem into a BVP and solve it with indirect

methods. Nevertheless, for problems with constraints, (4) does not neces-

sarily hold although the Hamiltonian still achieves the minimum within the

admissible control set. In this case, the optimal control problem cannot

be formulated as a BVP and indirect methods are not applicable. These

general optimal control problems are typically handled by direct methods,

most of which are closely related to nonlinear programming. Several pack-

ages for MATLAB are commercially available for such applications. Par-

ticularly, an open-source software GPOPS attracts more and more people

20

and improves with time. More information on GPOPS can be found in

http://www.gpops.org/.

References

[1] M. Athans and P. Falb (2007). Optimal control: an introduction to the

theory and its applications. Dover Publications, Inc.

[2] D. E. Kirk (2004). Optimal control theory: an introduction. Dover Pub-

lications, Inc.

[3] D. S. Naidu (2003). Optimal control systems. CRC Press LLC.

[4] L. F. Shampine., J. Kierzenka and M. W. Reichelt (2000). Solving

Boundary Value Problems for Ordinary Diﬀerential Equations in MAT-

LAB with bvp4c.

[5] The MATHWORKS, INC., Symbolic Math Toolbox User’s Guide.

[6] S. N. Avvakumov and Yu.N. Kiselev Boundary value problem for or-

dinary diﬀerential equations with applications to optimal control.

[7] Prof.Jonathan How MIT Open Courses Ware: 16.323 Principles of

optimal control, lecture 7.

21

3. How to reformulate the original problem as a Boundary Value Problem (BVP) and solve it with bvp4c? 4. How to get ‘good enough’ solutions with bvp4c and under what conditions will bvp4c fail to ﬁnd a solution? It should be noted that all the routines (except the steepest descent method) discussed in this tutorial belong to the “indirect methods” category. This means that constraints on the controls and states are not considered.1 In other words, the control can be solved in terms of states and costates and the problem is equivalently to a BVP. The reference of all the examples used in this tutorial are stated such that the results can be compared and veriﬁed.

2

Optimal control problems with fixed-finaltime

In most books [1] [2], it is free-ﬁnal-time problem that being tackled ﬁrst to derive the necessary conditions for optimal control. Fixed-ﬁnal-time problems were treated as an equivalent variation with one more state for time. However, for numerical methods, ﬁxed-ﬁnal-time problems are the general form and we solve the free-ﬁnal-time problem by converting it into a ﬁxedﬁnal-time problem. The reason is simple: when dealing with optimal control problems, it is inevitable to do numerical integration (either by indirect or direct methods). Therefore, a time interval must be speciﬁed for these methods. The ﬁrst problem is from [2], Example 5.1-1 from page 198 to 202. The routine deployed in this example shows how to derive necessary conditions and solve for the solutions with MATLAB Symbolic Math Toolbox. Example 1 The system x1 (t) = x2 (t) ˙ x2 (t) = −x2 (t) + u(t) ˙ (a) Consider the performance measure:

tf

(1) (2)

J(u) =

0

1 2 u (t) dt 2

1 The last example implements a simple constraint on the control to show limitations of bvp4c solver.

2

% Costate equations Dp1 = -diff(H. Dx1 = x2.x2).5*u^2. sol_u = solve(du.with boundary conditions: x(0) = 0 x(2) = 5 2 T (b) Consider the performance measure: 1 1 J(u) = (x1 (2) − 5)2 + (x2 (2) − 2)2 + 2 2 with boundary conditions: x(0) = 0 x(2) is free 2 0 1 2 u (t) dt 2 (c) Consider the performance measure as in (a) with following boundary conditions: x(0) = 0 x1 (2) + 5x2 (2) = 15 The ﬁrst step is to form the Hamiltonian and apply necessary conditions for optimality. Dx2 = -x2 + u.u). % Hamiltonian syms p1 p2 H.x1). % Cost function inside the integral syms g. this can be done as follows: % State equations syms x1 x2 p1 p2 u. g = 0. diff diﬀerentiates a symbolic expression and solve gives symbolic solution to algebraic 3 . Dp2 = -diff(H. With MATLAB. % solve for control u du = diff(H. ’u’). H = g + p1*Dx1 + p2*Dx2. The MATLAB commands we used here are diff and solve.

sol_u).eq3. 2 We use * to indicate the optimal state trajectory or control. sol_a = dsolve(eq1. sol_h = dsolve(eq1.eq2. x2(2) = 2. The symbolic solution looks diﬀerent from the one in [2].char(Dp1)). Yet Figure 1 shows that the two are in fact equivalent. For all the ﬁgures in this problem.char(Dx2)). it is not diﬃcult to see that the two are the same. the diﬀerences of the three cases in this problem are merely the boundary conditions. 4 . eq2 = strcat(’Dx2=’. conA1 = ’x1(0) = 0’. u. the arbitrary coeﬃcients can be determined by supplying the 2n boundary conditions to dsovle: % case a: (a) x1(0)=x2(0)=0.eq4.conA2. % convert symbolic objects to strings for using ’dsolve’ eq1 = strcat(’Dx1=’. % Substitute u to state equations Dx2 = subs(Dx2.conA3. The second step is to substitute u from (4) back to the state and costate equations to get a set of 2n ﬁrst-order ordinary diﬀerential equations (ODE’s). eq4 = strcat(’Dp2=’. x1(2) = 5. By simplifying the expression and rearrange the arbitrary coeﬃcients.char(Dp2)).eq3. conA3 = ’x1(2) = 5’. As stated in [2].conA4).char(Dx1)). please refer to [5]. For more details about Symbolic Math Toolbox. eq3 = strcat(’Dp1=’. From the second equation. we solve for control u in terms of states and costates.eq2.eq4).equations. For (a). A solution (with 2n arbitrary coeﬃcients) can be obtained by using the dsolve command without any boundary conditions. * represent state trajectory from [2] while symbolic solution from MATLAB is plotted with a continuous line. conA2 = ’x2(0) = 0’. Applying the necessary conditions for optimality we get two equations:2 p∗ = − ˙i ∂H = 0 ∂u∗ ∂H ∂x∗ i (3) (4) The ﬁrst equation gives costate equations.conA1. conA4 = ’x2(2) = 2’. Again the solutions given by MATLAB and [2] look diﬀerent from each other.

p1(2) = x1(2) .’t’. Figure 2 shows the results of both solutions from MATLAB and [2].0)). x2 in the ODE’s which resulted in more ODE’s than variables.x1.char(subs(sol_h.p2. p2(2) = x2(2) -2.eq4b). ’=’. 5 . p2 with states x1 . eq4b = strcat(char(subs(sol_h.. ’=’. % eq1b eq2b eq3b case b: (a) x1(0)=x2(0)=0.’-2’).. = char(subs(sol_h.2)). sol_b = solve(eq1b.’t’.eq3b.eq2b.p1.2)). tf = 2 in the the general solution to get a set of 4 algebraic equations with 4 arbitrary coeﬃcients.0)).’t’.x1.2)).’t’.’t’.. We get the ﬁnal symbolic solution to the ODE’s by substituting these coeﬃcients into the general solution. It should be noted that we cannot get the solution by directly supplying the boundary conditions with dsolve as we did in case (a).5.Figure 1: Trajectories for Example 1 (a) For case (b). p∗ (2) = x∗ (t) − 2 replaces 2 2 1 1 costates p1 .char(subs(sol_h... sol_b is a structure consists of four coeﬃcients returned by solve.x2. = char(subs(sol_h. = strcat(char(subs(sol_h.’-5’). we substitute t0 = 0.. Then the four boundary conditions are supplied to determine these coeﬃcients.x2.’t’. Because the latter two boundary conditions: p∗ (2) = x∗ (t) − 5.2)).

{subs(sol_h.C1).p2.x1.p1. C2 = double(sol_b.{subs(sol_h.C3).. strcat(char(subs(sol_h. eq4c = strcat(char(subs(sol_h. .char(subs(sol_h.Figure 2: Trajectories for Example 1 (b) % Substitute the coefficients C1 = double(sol_b.. p∗ (2) = 5p∗ (2).2)). the boundary conditions are: x∗ (2) + 5x∗ (2) = 1 2 15.’t’.{subs(sol_h..’p2’.{subs(sol_h. Figure 3 shows the results of both solutions from MAT2 1 LAB and [2].char(subs(sol_h.eq3c. char(subs(sol_h.. From [2].2)).. Case (c) is almost the same as case (b) with slightly diﬀerent boundary conditions.x2)}. sol_b2 = struct(’x1’.C2). C4 = double(sol_b.2)). ’-(’..C4). ’+(’. % case eq1c = eq2c = eq3c = 6 .’)*5’).x1(2)+5*x2(2)=15. C3 = double(sol_b.p2)}).x2.p2(2)= 5*p1(2).’t’. char(subs(sol_h.x1)}.C1).’t’.’t’.p1)}.0))..’x2’.2))..x2.x1.0)).’t’. sol_c = solve(eq1c.eq2c.’t’. ’p1’. % Substitute the coefficients C1 = double(sol_c. c: x1(0)=x2(0)=0.’)*5-15’).eq4c).

p2)}).5 − x2 (t) − [x2 (t) + 0.{subs(sol_h. Example 6.x2)}. C3 = double(sol_c.Figure 3: Trajectories for Example 1 (c) C2 = double(sol_c. x1 (t) = T (t) is the deviation from the steady-state temperature and x2 (t) = 7 .C4). irreversible exothermic reaction taking place in the reactor.5]exp 25x1 (t) − [x1 (t) + 0.25]u(t) ˙ x1 (t)+2 x2 (t) = 0.{subs(sol_h. In the next example.. If the state equations are complicated. The problem can be found in [2] page 338 to 339.’p2’.2-2. .p1)}.{subs(sol_h.5]exp ˙ 25x1 (t) x1 (t)+2 (5) The ﬂow of a coolant through a coil inserted in the reactor is to control the ﬁrst-order. sol_c2 = struct(’x1’.’x2’. Example 2 The state equations for a continuous stirred-tank chemical reactor are given as: x1 (t) = −2[x1 (t) + 0. we will illustrate two numerical routines: steepest descent method and convert to a BVP. ’p1’. it is usually impossible to derive explicit solutions.x1)}. We depend on numerical methods to ﬁnd the solutions.C3). C4 = double(sol_c..{subs(sol_h.C2).25] + [x2 (t) + 0.

8 . · · · . The algorithm consists of 4 steps: 1. 4. t ∈ [t0 . Applying u(i) and x(i) to integrate costate equations backward. representing the eﬀect of coolant ﬂow on the chemical reaction. Applying the assumed control u(i) to integrate the state equations from t0 to tf with initial conditions x(t0 ) = x0 and store the state trajectory x(i) . Subdivide the interval [t0 . u(t) is the normalized control variable. N − 1 Replace u(i) by u(i+1) and return to step 2. N − 1 2. If (6) is not satisﬁed.. The performance measure to be minimized is: 0. Here γ is a preselected small positive constant used as a tolerance. tf ] into N equal subintervals and assume a piecewise-constant control u(0) (t) = u(0) (tk ). from [tf . 1 2 with the boundary conditions x(0) = 0. adjust the piecewise-constant control function by: u(i+1) (tk ) = u(i) (tk ) − τ ∂H (i) (tk ). we will implement the steepest descent method based on the scheme outlined in [2]. τ is the step size. 3. ∂x Evaluate ∂H (i) (t)/∂u.e. · · · .C(t) is the deviation from the steady-state concentration. ∂u k = 0. 1. i. If ∂H (i) ∂u ∂H (i) ∂u 2 ≤ γ tf (6) ∂H (i) ∂u T ≡ t0 ∂H (i) ∂u dt (7) then stop the iterative procedure. tf ] and store this vector. t ∈ [tk . tk+1 ] k = 0. The “initial value” p(i) (tf ) can be obtained by: p(i) (tf ) = ∂h (i) x (tf ) . x(tf ) is free First. 1.05 0 T . Here. t0 ].78 J= 0 [x2 (t) + x2 (t) + Ru2 (t)] dt.

2).1*(u*u’)/length(Tu)).x.1). interpolation is used to make sure x(t).Tu).u.Tx. H_Norm = dH’*dH.u..X] = ode45(@(t.Tu). 0.u.Tx). u = AdjControl(dH.Tu) dx = zeros(2.x1. p(t) and u(t) are aligned along the time axis.u. exit if H_Norm < eps % Display final cost J(i. % Calculate deltaH with x1(t).Tx).x.The main loop in MATLAB is as follows: for i = 1:max_iteration % 1) start with assumed control u and move forward [Tx. options). .p. [t0 tf].x2.1).step)..Tx.Tu.u_old.1) break. % Important: costate is stored in reverse order. . p1 = P(:. % Calculate the cost function J(i. % State equations function dx = stateEq(t. end Because the step size of ode45 is not predetermined. x2 = X(:.x) stateEq(t..1). p2(t) dH = pH(x1. % if dH/du < epslon..Tu. p1(t)..u. [Tp.p) costateEq(t.p1. end. else % adjust control for next iteration u_old = u. % Interploate the control at time t 9 . u = interp1(Tu. initx.P] = ode45(@(t.p1.. initp. % 2) Move backward to get the trajectory of costates x1 = X(:.1) = tf*(((x1’)*x1 + (x2’)*x2)/length(Tx) + . options). The dimension of % costates may also different from dimension of states % Use interploate to make sure x and p is aligned along the time axis p1 = interp1(Tp. x2(t).t). [tf t0].

Tu. In MATLAB.t).*exp((25*x1)/(x1 + 2)) ..p(2).. 2*x1 .25) + (x(2) + 0.step) % interploate dH/du pH = interp1(tx. The value of performance measure as a function of iteration number is shown in Figure 5. For ﬁx-ﬁnal-time problems.. % Costate equations function dp = costateEq(t.step*pH..tu).u.u. p(1). dx(2) = 0. R = 0.*exp((25*x1)/(x1 + 2))*((25*x1)/(x1 + 2)^2 .25] on each time subinterval (function pH) and compare ∂H/∂u 2 with the preselected γ. u_new = u .u.x(2) -(x(2) + 0.*(u + exp((25*x1)/(x1 + 2)).2*x2. These two methods basically reformulate and solve the original problem as a Boundary Value Problem (BVP). a BVP is typically solved with bvp4c. [4] is an excellent reference on using bvp4c. we calculate ∂H/∂u|t = 2Ru(t) − p1 (t)[x1 (t) + 0.xt) dp = zeros(2.tx).. % Partial derivative of H with respect to u function dH = pH(x1. If ∂H/∂u 2 > γ. In [2]..Tu) % interploate the control u = interp1(Tu. dp(2) = p(2).u. % Interploate the state varialbes x2 = interp1(xt.p1..tu. % Interploate the control dp(1) = p(1).(x(1) + 0.tx..*(x1 + 0. 25/(x1 + 2))*(x2 + 1/2).pH..t). ..dx(1) = -2*(x(1) + 0. % Adjust the control function u_new = AdjControl(pH.. A better strategy is to select τ with a line search method which will maximize the reduction of performance measure with given ∂H (i) /∂u in each iteration..tx. two more numerical algorithms were introduced: variation of extremals and quasilinearization.u.25). 25/(x1 + 2))*(x2 + 1/2) + 2) . dH = 2*R*u . u = interp1(Tu..x1.5 .x1.p. x1 = interp1(xt.t).1). In step 4.1.*(exp((25*x1)/(x1 + 2)) + 1) . Figure 4 shows the optimal state trajectory and control over the time.p1.*u. The step size τ is set as a constant in this example by some post hoc investigation.5)*exp(25*x(1)/(x(1)+2)). adjust u (function AdjControl).x2.5)*exp(25*x(1)/(x(1)+2)) .25).*((25*x1)/(x1 + 2)^2 . u can always be solved 10 .x2..

And we will have 2n ODE’s and 2n boundary conditions. 11 .Figure 4: Example 2 Steepest descent method Figure 5: Performance measure reduction with respect to x and p by applying the necessary conditions (3) and (4).

.. The ODE’s and boundary conditions are two major considerations in using bvp4c.:)*y(2.*(y(1.Figure 6: Solution from bfv4c for Problem 2 % Initial guess for the solution solinit = bvpinit(linspace(0.x. % Calculate u(t) from x1. [0 0 0. options). Once the optimal control problem is converted to a BVP.5 0.:)’ + .. global R. t = sol. .’RelTol’. % Calculate the cost J = 0.x2. %-----------------------------------------------% ODE’s for states and costates 12 .1e-1).p2 ut = (y(3.:).y.78*(y(1.78.’on’. A rule of thumb is that the number of ODE’s must equal to the number of boundary conditions such that the problem is solvable. sol = bvp4c(@BVP_ode. @BVP_bc. ut*ut’*0.0.50).:)*y(1.:)’ + y(2.5]). y = sol. R = 0.1.:) + 1/4))/(2*0.1)/n.p1.. options = bvpset(’Stats’.1). solinit. n = length(t). it is very simple to solve.

0 yb(3) . i.14. In the following section. x2(0) = 0. which can be ﬁnd from [3] on page 77. u = y(3)*t1/(2*R). We will use both Symbolic Math Toolbox and bvp4c in the next example. In this example.e.0 ]. t3 = exp(25*y(1)/(y(2)+2)).0 yb(4) .05. It is faster and gives better results. t4 = 50/(y(1)+2)^2. % ----------------------------------------------% The boundary conditions: % x1(0) = 0. Example 3 Given a double integral system as: x1 (t) = x2 (t) ˙ x2 (t) = u(t) ˙ The performance measure is: J= 1 2 tf (8) (9) u2 (t)dt 0 13 . p1(tf) = 0. p2(tf) = 0. 3 Optimal control problems with free-finaltime Now we are prepared to deal with free-ﬁnal-time problems.78. t2 = y(2)+.05 ya(2) . Example 2. bvp4c works perfectly.yb) res = [ ya(1) .0.5-y(2)-t2*t3 -2*y(1)+2*y(3)-y(3)*t2*t4*t3+y(3)*u+y(4)*t2*t4*t3 -2*y(2)-y(3)*t3+y(4)*(1+t3)]. dydt = [-2*t1+t2*t3-t2*u 0. function res = BVP_bc(ya. tf = 0.function dydt = BVP_ode(t. a smaller performance measure J comparing to the steepest descent method (see Figure 6).y) global R. t1 = y(1)+. we will solely use bvp4c when numerical solutions are needed.25.5.

x1 (tf ) = 3. subs(sol. Dx2 = -p2. x1 x2 p1 p2 = = = = subs(sol. eq4. sol = dsolve(’Dx1 = x2. tf = sol_2. u∗ (tf ). x2 (tf ). namely x1 (tf ). h ≡ 0 and we have p1 (tf )x2 (tf ) − 0. eq4 = subs(sol.’x2tf’. x2(0) = 2. And solve comes in handy to solve this problem.x1). However. when we try to use a numerical method such as bvp4c. it should be pointed out that in most problems. As a result. Next. eq2. tf ) + ∂h ∗ (x (tf ).5*p2tf^2’). ’x1(0) = 1.p2)..5p2 (tf ) = 0. With one additional boundary condition from H (x∗ (tf ). The only diﬀerence is that the ﬁnal time tf itself is now a variable.x2).’p2tf’. Dp1 = 0. x2 (tf ) is free To use the Symbolic Math Toolbox. numerical methods are more useful in dealing with more general problems. eq5 = sym(’p1tf*x2tf .. For example. we immediately encountered with 14 .0.’p1tf’. p1 (tf ). the routine is very similar to Problem 1.x2) .x1) .tf. eq1 = subs(sol. x1(tf) = 3. We ﬁrst supply the ODE’s and boundary conditions on states and costates to dsolve. eq3 = subs(sol. p∗ (tf ). subs(sol. eq5). it is impossible to get explicit solutions. we introduce four more variables. the solution is a function of tf . Now we 2 have 5 algebraic equations with 5 unknowns.. Although Symbolic Math Toolbox works ﬁne in this example. Because of the limitations of symbolic method. x2tf = sol_2.ﬁnd the optimal control given the boundary conditions as: x(0) = 1 2 T . p2 (tf ) into the solution obtained above.’x1tf’. subs(sol. %% sol_2 = solve(eq1.p1). tf ) = 0 ∂t For this problem.x1tf. x1tf = sol_2.x2tf. eq2 = subs(sol.p2) .p1) . Figure 7 shows the results from MATLAB and the analytical solution in [3]. p2(tf) = 0’). Dp2 = -p1’. there is no explicit solutions for Example 1 even though the state equations are similar to those of Example 3. eq3.

x. 1 ].1.-y(4).-y(3). ut = -y(4.0. x1(tf)=3. This can be implemented in bvp4c by treating T as an auxiliary variable.3. q. @bc.3 Now the problem is posed on ﬁxed interval [ 0.1).[2. One common treatment [4] [7] for such a situation is to change the independent variable t to τ = t/T . The following code snippet shows the details. time = y(5)*sol. sol = bvp4c(@ode. % ------------------------------------------------------------------------% boundary conditions: x1(0)=1. solinit). 3 f denotes the ODE’s for state and costates. % ------------------------------------------------------------------------% ODE’s of augmented states function dydt = ode(t. y = sol. the ˙ ˜ augmented state and costate equations will then become x = T f (x. τ ).:). 15 .x2(0)=2.0 ].y. solinit = bvpinit(linspace(0.y) dydt = y(5)*[ y(2).1. p2(tf)=0.2]).Figure 7: Example 3 Symbolic method a problem: the time interval is not known.

yb(1) .-y(4).:). yb(4). yb(3)*yb(2)-0.% p1(tf)*x2(tf)-0. ya(2) .5*yb(4)^2]. p2(tf)=0. When supplied with an initial guess ˙ of 0. The last problem is to further illustrate the capability.5*yb(4)^2]. as well as the limitation. yb(4).T) dydt = T*[ y(2). % p1(tf)*x2(tf)-0. Now it is the time to talk about the limitations of bvp4c.3. yb(1) . ya(2) . we ﬁnd that p1 (t) = 0.2.-y(3) ]. y = sol.3. @bc.5*p2(2)^2 function res = bc(ya. Figure 8 shows the result from numerical method which is the same as the analytical solution. A bad initial guess may result in inaccurate solutions.1.T) res = [ ya(1) .1]. we must bear in mind that the quality of the solution from bvp4c is heavily dependent on the initial guess.yb. sol = bvp4c(@ode.0. and function deﬁnition of the ODE’s and boundary conditions.y.1. yb(3)*yb(2)-0. of bvp4c in solving optimal control problems. By looking at the ODE’s closely.3. Thought it works well so far.y.1). For example.parameters*sol. you may try the initial guess p1 (0) = 0 and compare the results with the analytical solution to see the diﬀerence. or no solutions. This example can be ﬁnd from [6] which is a time-optimal problem for the double integral system with a simple control constraint.x2(0)=2.2). we can accomplish this by treating T as a parameter for bvp4c [4] The diﬀerence between the two lies in the parameter list of bvpinit. ut = -y(4.x. x1(tf)=3.yb) res = [ ya(1) . solinit = bvpinit(linspace(0.2. % ----------------------------------------------------------% ODE’s of augmented states function dydt = ode(t. time = sol.5*p2(2)^2 function res = bc(ya.[2. bvp4c fails to ﬁnd the solution due to the state singularity. solinit). Alternatively. 16 . % ----------------------------------------------------------% boundary conditions: x1(0)=1.1. or solutions which make no sense.

control domain smoothing technique must be applied ﬁrst [6].Figure 8: Example 3 Numerical method Example 4 Given a double integral system as: x1 (t) = x2 (t) ˙ x2 (t) = u(t) ˙ Minimize the ﬁnal time: tf (10) (11) T = 0 dt to drive the states to the origin: x(0) = 2 2 T . |u| ≤ 1 To formulate this problem as a BVP. The resulted ODE’s are listed below: x1 = x5 ˙ x2 = x5 ˙ x3 = 0 ˙ x4 = −x5 x3 ˙ x5 = 0 ˙ 17 x2 + x4 µx2 + x2 3 4 µx3 µx2 + x2 3 4 (12) (13) (14) (15) (16) . x(tf ) = 0 0 T .

function res = bc(ya..@bc.The boundary conditions are: x1 (0) = 2.5. The MATLAB code is as follows global mu.sol). i. for i=2:3 if i==2 mu = 0. sol = bvp4c(@ode. x1 (1) = 0. mu = 0. % -----------------------------------------------------------% boundary conditions. dydt = T*[ y(2) + mu*y(3)/term y(4)/term 0 -y(3)].1. end % ----------------------------------------------------------function dydt = ode(t. the new value of mu % will be used in nested functions. which means that u will take either the maximum or the minimum value within the admissible control set. x3(1)^2+x4(1)^2 = 1.2.y. else mu = . It is well known that 4 3 the optimal control for this problem is the piecewise continuous “bang-bang” control.10).1.T) global mu.4). . 5 conditions are % needed: x1(0) =2.T) res = [ya(1)-2 ya(2)-2 18 .yb. x2(0) = 2. Because it is a linear second order system. x2 (0) = 2. there will be only one switch point and u is not continuous at the switch point. Auxiliary variable x5 = T .e.0..solinit)..@bc.. % the solution for one value of mu is used as guess for the next. x2 (1) = 0. . x3 (1)2 + x4 (1)2 = 1. and u = x4 / µx2 + x2 . x2(1) = 0. sol = bvp4c(@ode.-1]. with 4 states and 1 parameters. +1 or −1 in this problem.3. end. solinit = bvpinit(linspace(0.[2. % After creating function handles. term = sqrt(mu*y(3)^2+y(4)^2). x1(1) = 0.

if we set µ = 10−3 or smaller. The trajectory and control corresponding to diﬀerent µ values are shown in Figure 9 and Figure 10. when the optimal control is piecewise continuous.5 is used as guess for the BVP with µ = 0. u appears to be more and more stiﬀ at the switch point. bvp4c will fail at discontinuous points. by which means the solution for µ = 0. In other words.3.5.1. For example. As µ → 0. However. approximating the “bang-bang” control. 19 . µ = 0. we cannot keep decreasing µ to get x and u arbitrarily close to the optimal x∗ and u∗ .3. We uses continuation to solve this problem. Three values were tested: µ = 0. bvp4c will fail due to the state singularity. We can clearly see that u is approximating a “bang-bang” control at switch point t = 4 and the state trajectory resembles the optimal trajectory more and more closely as µ decrease.Figure 9: Example 4 State trajectory yb(1) yb(2) yb(3)^2+yb(4)^2-1]. µ = 0.

bvp4c is the best option we have. (4) does not necessarily hold although the Hamiltonian still achieves the minimum within the admissible control set. the discontinuity is not that severe a problem. Several packages for MATLAB are commercially available for such applications. • Discontinuous points (either in state/costate equations or control function) will cause bvp4c to fail. we must pay attention to the limitations of bvp4c. more speciﬁcally • Good initial guess is important to get an accurate solution. for problems with constraints. the optimal control problem cannot be formulated as a BVP and indirect methods are not applicable. an open-source software GPOPS attracts more and more people 20 . if a solution exists.Figure 10: Example 4 Control 4 Discussion For many optimal control problems. we can convert the problem into a BVP and solve it with indirect methods. For optimal control problems with no constraints on states or control. most of which are closely related to nonlinear programming. By applying the minimum principle (4). Nevertheless. Once the problem is reformatted into a BVP. bvp4c can usually solve it eﬃciently. These general optimal control problems are typically handled by direct methods. Particularly. In this case. However.

Shampine. Kierzenka and M. Avvakumov and Yu. Athans and P. S. lecture 7. J. References [1] M. Kiselev Boundary value problem for ordinary diﬀerential equations with applications to optimal control. Dover Publications.org/. More information on GPOPS can be found in http://www. 21 . Optimal control: an introduction to the theory and its applications. [5] The MATHWORKS. E. F. [7] Prof. INC. Inc. W. [2] D.323 Principles of optimal control. N. [4] L. Dover Publications.. Solving Boundary Value Problems for Ordinary Diﬀerential Equations in MATLAB with bvp4c.Jonathan How MIT Open Courses Ware: 16. CRC Press LLC. Optimal control systems. Reichelt (2000). Symbolic Math Toolbox User’s Guide. Inc. Naidu (2003).gpops. [6] S. Optimal control theory: an introduction. Falb (2007).and improves with time. [3] D.. Kirk (2004).N.

- [Mark S. Gockenbach] Partial Differential Equation
- applied nonlinear control
- x 04405132143
- M.E. _Power System Engineering_
- Optimal Control Theory and Static Optimization in Economics
- Kirk Optimal Control Theory
- hw01sol.pdf
- Optimal Control
- Final Thesis process modeling simulation and control of multi component batch distillation
- Computer Methods for Solving Dynamic Separation Problems
- FALLSEM2013-14 CP1806 30-Oct-2013 RM01 II OptimalControl Uploaded
- 431-hw6-key
- 0904.0833
- 2012. Q. Q. CHAI. Optimal Control of an Industrial Scale Evaporation Process. Sodium Aluminate Solution
- Optimal Control Systems (Electrical Engineering Handbook) - Malestrom
- le lesson 1 6 slide show
- UEM16 (1).doc
- Math lesson 7_2
- SCRP
- CHE356
- Computational Models for Reduction of Quarry Waste for Breakwater Construction Projects
- 11-7449 Brochure Pims Final
- An Algorithm for Solving Integer Linear Programming Problems
- Problemas_Taller_0.pdf
- Scheduling Thesis to Read
- How to Solve Quadratic Equation_ – Tips & Tricks
- IRJET-ON THE BINARY QUADRATIC DIOPHANTINE EQUATION
- Differential Equation
- (IET Control Robotics and Sensors Series 83) Robust and Adaptive Model Predictive Control of Non-linear Systems
- Nonlinear Equations

Read Free for 30 Days

Cancel anytime.

Close Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Loading