Controllability and state transfer 16 – 1

Lecture 16
Controllability and state transfer
EE263 Autumn 2003

• state transfer
• reachable set, controllability matrix
• minimum norm inputs
• infinite-horizon minimum norm transfer
• controllability Gramian
Controllability and state transfer 16 – 2

State transfer

consider ẋ = Ax + Bu (or x(t + 1) = Ax(t) + Bu(t))

over time interval [ti, tf ]

we say input u : [ti, tf ] → Rm steers or transfers state

from x(ti) to x(tf ) (over time interval [ti, tf ])
(subscripts stand for initial and final )

questions:
• where can x(ti) be transfered to at t = tf ?
• how quickly can x(ti) be transfered to some xtarget?
• how do we find a u that transfers x(ti) to x(tf )?
• how do we find a ‘small’ or ‘efficient’ u that transfers
x(ti) to x(tf )?
Controllability and state transfer 16 – 3

Reachability

consider state transfer from x(0) = 0 to x(t)

we say x(t) is reachable (in t seconds or epochs)

we define Rt ⊆ Rn as the set of points reachable in t

seconds or epochs

for CT system ẋ = Ax + Bu,

( Z )
t A(t−τ ) m
Rt = 0 e Bu(τ ) dτ | u : [0, t] → R
and for DT system x(t + 1) = Ax(t) + Bu(t),
 
 t−1 
At−1−τ Bu(τ ) | u(t) ∈ Rm
 
Rt = 
X

τ =0 

• Rt is a subspace of Rn
• Rt ⊆ Rs if t ≤ s
(i.e., can reach more points given more time)

we define the reachable set R as the set of points

reachable for some t: R = Rt
[

t≥0
Controllability and state transfer 16 – 4

Reachability for discrete-time LDS

DT system x(t + 1) = Ax(t) + Bu(t), x(t) ∈ Rn

 
 u(t − 1) 
x(t) = Ct




.. 


 
u(0)
 

· ¸
t−1
where Ct = B AB · · · A B

so reachable set at t is Rt = range(Ct)

by C-H theorem, we can express each Ak for k ≥ n as

linear combination of A0, . . . , An−1

hence for t ≥ n, range(Ct) = range(Cn)

thus we have



 range(Ct) t < n
Rt = 
range(C) t ≥ n
where C = Cn is called the controllability matrix

• any state that can be reached can be reached by

t=n
• the reachable set is R = range(C)
Controllability and state transfer 16 – 5

system is called reachable or controllable if all states are

reachable (i.e., R = Rn)

system is reachable if and only if Rank(C) = n

   
0 1   1 
example: x(t + 1) =  x(t) +   u(t)

 
1 0 1


 
1 1 
controllability matrix is C = 


1 1


hence system is not controllable; reachable set is

R = range(C) = { x | x1 = x2 }
Controllability and state transfer 16 – 6

Invariance of reachable set

fact: the reachable set R is invariant
• if the state starts in R, it will never drift out of R
(u = 0)
• not interesting when system is controllable (R = Rn)
proof: we need to show that
z ∈ range(C) =⇒ Az ∈ range(C)
n−1
suppose z ∈ range(C), i.e., z = AiBu(i)
X

i=0
then
n−1
Az = Ai+1Bu(i)
X

i=0
n−1
= AiBu(i − 1) + AnBu(n − 1)
X

i=1
n−1
= AiB(u(i − 1) − αiu(n − 1)) − α0u(n − 1)B
X

i=1
∈ range(C)
where
X (s) = sn + αn−1sn−1 + · · · + α0 = det(sI − A)
and we have used the C-H theorem:
An = −α0I − · · · − αn−1An−1
Controllability and state transfer 16 – 7

General state transfer

with tf > ti,

 
 u(tf − 1) 

x(tf ) = A tf −ti

x(ti) + Ctf −ti 

 .
.





u(ti)
 

hence can transfer x(ti) to x(tf ) = xdes

⇔ xdes − Atf −ti x(ti) ∈ Rtf −ti

• general state transfer reduces to reachability problem

• if system is controllable any state transfer can be
achieved in ≤ n steps
• important special case: driving state to zero
(sometimes called regulating or controlling state)
• if the state starts in R, it will never leave R, no
matter what u is (using invariance of R)
Controllability and state transfer 16 – 8

Least-norm input for reachability

assume system is reachable, Rank(Ct) = n

to steer x(0) = 0 to x(t) = xdes, inputs
u(0), . . . , u(t − 1) must satisfy
 
 u(t − 1) 
xdes = Ct




.. 


 
u(0)
 

among all u that steer x(0) = 0 to x(t) = xdes, the one

that minimizes
t−1
ku(τ )k2
X

τ =0
is given by
 
 uln(t − 1) 




.. 
 T T −1
 = Ct (Ct Ct ) xdes
 
uln(0)
 

uln is called least-norm or minimum energy input that

effects state transfer
can express as

t−1

−1
T T (t−1−τ )  s T T s
uln(τ ) = B (A ) A BB (A ) xdes,
X
 
s=0

for τ = 0, . . . , t − 1
Controllability and state transfer 16 – 9

t−1
Emin, the minimum value of ku(τ )k2 required to reach
X

τ =0
x(t) = xdes, is sometimes called minimum energy
required to reach x(t) = xdes

t−1
Emin = kuln(τ )k2
X

τ =0
µ ¶T
= T T −1
Ct (CtCt ) xdes CtT (CtCtT )−1xdes
= xTdes(CtCtT )−1xdes

t−1

−1
T  τ T T τ
= xdes  A BB (A )  xdes
X 

τ =0

• Emin(xdes, t) gives measure of how hard it is to reach

x(t) = xdes from x(0) = 0
(i.e., how large a u is required)
• Emin(xdes, t) gives practical measure of
controllability/reachability (as function of xdes, t)
• ellipsoid { z | Emin(z, t) ≤ 1 } shows points in state
space reachable at t with one unit of energy
(shows directions that can be reached with small
inputs, and directions that can be reached only with
large inputs)
Controllability and state transfer 16 – 10

Emin as function of t:

if t ≥ s then
t−1 τ T T τ s−1
A BB (A ) ≥ Aτ BB T (AT )τ
X X

τ =0 τ =0

hence

t−1

−1 
s−1

−1
τ T T τ τ T T τ
A BB (A ) ≤ A BB (A )
 X  X
   
τ =0 τ =0

so Emin(xdes, t) ≤ Emin(xdes, s)

i.e.: takes less energy to get somewhere more leisurely

 Controllability and state transfer 16 – 11

   
1.75 0.8   1 
example: x(t + 1) =   x(t) +   u(t)

 
−0.95 0 0
Emin(z, t) for z = [1 1]T :
10

6
Emin

2
PSfrag replacements
1

0
0 5 10 15 20 25 30 35
t

ellipsoids Emin ≤ 1 for t = 3 and t = 10:

10
Emin (x, 3) ≤ 1

0
x2

−5

−10
−10 −8 −6 −4 −2
x10 2 4 6 8 10

10
Emin (x, 10) ≤ 1

5
PSfrag replacements
0
x2

−5

−10
−10 −8 −6 −4 −2
x10 2 4 6 8 10
Controllability and state transfer 16 – 12

Minimum energy over infinite horizon

the matrix

t−1

−1
τ T T τ
P = lim A BB (A )
 X
 
t→∞ τ =0

always exists, and gives the minimum energy required to

reach a point xdes (with no limit on t):

 ¯ 
 t−1 ¯ 
ku(τ )k2 x(0) = 0, x(t) = xdes = xTdesP xdes
 ¯ 
min 
X ¯
¯
¯ 
τ =0 ¯ 

if A is stable, P > 0 (i.e., can’t get anywhere for free)

if A is not stable, then P can have nonzero nullspace

• P z = 0, z 6= 0 means can get to z using u’s with
energy as small as you like
(u just gives a little kick to the state; the instability
carries it out to z efficiently)
• basis of highly maneuverable, unstable aircraft
Controllability and state transfer 16 – 13

Discrete reachability Gramian

suppose x(t + 1) = Ax(t) + Bu(t) is controllable and

stable

t−1
then Aτ BB T (AT )τ converges as t → ∞ since Aτ
X

τ =0
decays geometrically

the matrix ∞
Wr = Aτ BB T (AT )τ
X

τ =0
is called the reachability (or controllability ) Gramian

Wr satisfies the matrix equation

Wr − AWr AT = BB T
which is called the controllability Lyapunov equation

Lyapunov equation is a linear equation in the variables

(Wr )ij and can be efficiently solved
(i.e., can compute infinite sum exactly)
Controllability and state transfer 16 – 14

Continuous-time reachability

consider now ẋ = Ax + Bu with x(t) ∈ Rn

reachable set at time t is

( Z )
t A(t−τ ) m
Rt = 0 e Bu(τ ) dτ | u : [0, t] → R

fact: for t > 0, Rt = R = range(C), where

· ¸
n−1
C = B AB · · · A B
is the controllability matrix of (A, B)

• same R as discrete-time system

• for continuous-time system, any reachable point can
be reached as fast as you like (with large enough u)
Controllability and state transfer 16 – 15

first let’s show for any u (and x(0) = 0) we have

x(t) ∈ range(C)

write eAt as power series:

t t2 2
eAt = I + A + A + ···
1! 2!

by C-H, express An, An+1, . . . in terms of A0, . . . , An−1

and collect powers of A:
eAt = α0(t)I + α1(t)A + · · · + αn−1(t)An−1

therefore
Z
t Aτ
x(t) = 0 e Bu(t − τ ) dτ

Z
t n−1 i
= α  Bu(t − τ ) dτ
X
0
 i (τ )A
i=0
n−1 i
Z
t
= A B 0 αi(τ )u(t − τ ) dτ
X

i=0
= Cz
Z
t
where zi = 0 αi (τ )u(t − τ ) dτ

hence, x(t) is always in range(C)

need to show converse: every point in range(C) can be

reached
Controllability and state transfer 16 – 16

Impulsive inputs

suppose x(0−) = 0 and we apply input u(t) = δ (k)(t)f ,

where δ (k) denotes kth derivative of δ and f ∈ Rm

then U (s) = sk f , so
X(s) = (sI − A)−1Bsk f
µ ¶
= s I + s A + · · · Bsk f
−1 −2

= ( s| k−1 + · · · + {zsAk−2 + Ak−1} +s−1Ak + · · · )Bf

impulsive terms

hence
k k+1 t k+2 t2
x(t) = impulsive terms +A Bf +A Bf +A Bf +· · ·
1! 2!

in particular, x(0+) = Ak Bf

thus, input u = δ (k)f transfers state from x(0−) = 0 to

x(0+) = Ak Bf
Controllability and state transfer 16 – 17

now consider input of form

u(t) = δ(t)f0 + · · · + δ (n−1)(t)fn−1
where fi ∈ Rm

by linearity we have
 
f0 


x(0+) = Bf0 + · · · + An−1Bfn−1 =C .. 





 
fn−1
 

hence we can reach any point in range(C)

(at least, using impulse inputs)

can also be shown that any point in range(C) can be

reached for any t > 0 using nonimpulsive inputs

fact: if x(0) ∈ R, then x(t) ∈ R for all t (no matter

what u is)

to show this, need to show eAtx(0) ∈ R if x(0) ∈ R . . .

Controllability and state transfer 16 – 18

example
• two unit masses with positions y1, y2, connected by
unit springs, dampers
• input is tension between masses
• state is x = [y T ẏ T ]T

u(t) u(t)

PSfrag replacements

system is
   
0 0 1 0 




0 
0 0 0 1  0
   
  
ẋ = x +  u
  
  
−1 1 −1 1  1

   
  
  
1 −1 1 −1 −1
   

• can we maneuver state anywhere, starting from

x(0) = 0?
• if not, where can we maneuver state?
Controllability and state transfer 16 – 19

controllability matrix is
 
0 1 −2 2  

0 −1 2 −2
 
· ¸  
2 3
C = B AB A B A B =
 
 
1 −2 2 0 
 



−1 2 −2 0
 

hence reachable set is

     








1  

0 


 
 

 
−1  0

     

    

 
R = span  ,
 
  
0  1
   
   

    


    

  

0 −1
    

 


we can reach states with y1 = −y2, ẏ1 = −ẏ2, i.e.,

precisely the differential motions

it’s obvious — internal force does not affect center of

mass position or total momentum!
Controllability and state transfer 16 – 20

Least-norm input for reachability

(also called minimum energy input)

assume that ẋ = Ax + Bu is reachable

we seek u that steers x(0) = 0 to x(t) = xdes and

minimizes Z
t 2
0 ku(τ )k dτ

let’s discretize system with interval h = t/N

(we’ll let N → ∞ later)

thus u is piecewise constant:

u(τ ) = ud(k) for kh ≤ τ < (k+1)h, k = 0, . . . , N −1
so
 
 ud(N − 1) 
·
x(t) = Bd AdBd · · · AN −1
Bd
¸

 .. 


d 



ud(0)
 

where
hA
Z
h τA
Ad = e , Bd = 0 e dτ B
Controllability and state transfer 16 – 21

least-norm ud that yields x(t) = xdes is

N −1
 
−1
T T (N −1−k)  i T T i
udln(k) = Bd (Ad ) AdBdBd (Ad )  xdes
X 

i=0

let’s express in terms of A:

T AT (t−τ )
BdT (ATd )(N −1−k) = Bd e
where τ = t(k + 1)/N

for N large, Bd ≈ (t/N )B, so this is approximately

T (t−τ )
(t/N )B T eA

similarly
NX
−1 NX
−1 Ati/N T ti/N
AidBdBdT (ATd )i = e BdBdT eA
i=0 i=0
t At̄ T AT t̄
Z
≈ (t/N ) 0 e BB e dt̄
for large N

hence least-norm discretized input is approximately

 
−1
T AT (t−τ )  t At̄ T AT t̄
Z
uln(τ ) = B e 0 e BB e dt̄  xdes, 0≤τ ≤t
for large N

hence, this is the least-norm continuous input

Controllability and state transfer 16 – 22

• can make t small, but get larger u

• cf. DT solution: sum becomes integral

min energy is
Z
t 2
0 ku ln (τ )k dτ = xTdesWr (t)−1xdes
where
t At̄ T AT t̄
Z
Wr (t) = 0 e BB e dt̄

can show
(A, B) controllable ⇔ Wr (t) > 0 for all t > 0
⇔ Wr (s) > 0 for some s > 0
in fact, range(Wr (t)) = R for any t > 0
Controllability and state transfer 16 – 23

minimum energy over infinite horizon: the matrix

P = lim Wr (t)−1
t→∞
always exists, and gives minimum energy required to
reach a point xdes (with no limit on t):

( Z ¯ )
t 2
= xTdesP xdes
¯
min 0 ku(τ )k dτ ¯
¯
¯ x(0) = 0, x(t) = xdes

if A is stable, P > 0 (i.e., can’t get anywhere for free)

if A is not stable, then P can have nonzero nullspace

• P z = 0, z 6= 0 means can get to z using u’s with
energy as small as you like
(u just gives a little kick to the state; the instability
carries it out to z efficiently)
Controllability and state transfer 16 – 24

Reachability Gramian

if ẋ = Ax + Bu is controllable and stable

then Wr (t) converges as t → ∞ to

∞ At̄ T AT t̄
Z
Wr = 0 e BB e dt̄,
the reachability (or controllability ) Gramian

the cts-time reachability Gramian Wr satisfies the matrix

equation
AWr + Wr AT + BB T = 0
which is called the controllability Lyapunov equation

to see this, note that

d At T T T
e BB T eA t = AeAtBB T eA t + eAtBB T eA tAT
dt

integrate from t = 0 to ∞ to get:

¯∞
¯
T
eAtBB T e A t ¯ ¯
¯ = AWr + Wr AT
¯
0
which gives the Lyapunov equation (a linear equation in
Wr which can be efficiently solved)
Controllability and state transfer 16 – 25

General state transfer

consider state transfer from x(ti) to x(tf ) = xdes, tf > ti

since
A(tf −ti )
Z
tf
x(tf ) = e x(ti) + ti eA(tf −τ )Bu(τ ) dτ

u steers x(ti) to x(tf ) = xdes ⇔

u (shifted by ti) steers x(0) = 0 to
x(tf − ti) = xdes − eA(tf −ti)x(ti)

• general state transfer reduces to reachability problem

• if system is controllable, any state transfer can be
effected
– in ‘zero’ time with impulsive inputs
– in any positive time with non-impulsive inputs
 Controllability and state transfer 16 – 26

example
replacements u1 u1 u2 u2

• unit masses, springs, dampers

• u1 is force between 1st & 2nd masses
• u2 is force between 2nd & 3rd masses
• y ∈ R3 is displacement of masses 1,2,3
 
y 
• x = 


ẏ


system is:
   
0 0 0 1 0 0 




0 0 
   
0 0 0 0 1 0 






0 0 

 
   
0 0 0 0 0 1 


 
 0 0   u1 


ẋ = 
 x + 


−2 1 0 −2 1 0  1 0 u
    



 





2
1 −2 1 1 −2 1 






 −1



1 



0 1 −2 0 1 −2
  
0 −1 

steer state from x(0) = e1 to x(tf ) = 0

i.e., control initial state e1 to zero at t = tf
Controllability and state transfer 16 – 27

Z
tf
Emin = 0 kuln(τ )k2 dτ vs. tf :
50

45

40

35

30

Emin
25

20

15

PSfrag replacements 10

0
0 2 4 6 8 10 12

tf
for tf = 3, u = uln is:
5
u1 (t)

−5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

5
t

PSfrag replacements
u2 (t)

−5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

t
and for tf = 4:
5
u1 (t)

−5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

5
t

PSfrag replacements
u2 (t)

−5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

t
Controllability and state transfer 16 – 28

output y1 for u = 0:
1

0.8

0.6

y1 (t)
0.4

0.2

0
PSfrag replacements
−0.2

−0.4
0 2 4 6 8 10 12 14

t
output y1 for u = uln with tf = 3:
1

0.8

0.6
y1 (t)

0.4

0.2

0
PSfrag replacements
−0.2

−0.4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

t
output y1 for u = uln with tf = 4:
1

0.8

0.6
y1 (t)

0.4

0.2

0
PSfrag replacements
−0.2

−0.4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5