EMT 4203: CONTROL ENGINEERING

II

Bsc. Mechatronic Engineering

DeKUT

Lecture Notes

By

Dr. Inno Oduor Odira

February 2024
Table of contents

II Modern Control 1

4 Optimal Control (DP, LQR and Kalman Filters) 57

4.1 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 LQR and Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.1 Dynamic Programming and Full-State Feedback . . . . . . . . . . . . . 65
4.2.2 Hamilton-Jacobi Bellman method . . . . . . . . . . . . . . . . . . . . . 67

References 77
 Part II

(Modern Control)
Chapter 3a

State-space analysis and State space

design methods

3.1 State-space analysis

3.1.1 Introduction
We defined the state of a system as any set of quantities which must be specified at a given
time in order to completely determine the behavior of the system. The quantities constitut-
ing the state are called the state variables, and the hypothetical space spanned by the state
variables is called the state-space.

Modern control theory is based on the description of system equations in terms of n

first-order differential equations, which may be combined into a first order vector-matrix
differential equations. The use of vector-matrix notation greatly simplifies the mathematical
representation of systems of equations.

Consider a system described by the following couple of coupled differential equations

y¨1 + k1 y˙1 + k2 y1 = u1 + k3 u2 (1.1)

ẏ2 + k4 y2 + k5 ẏ1 = k6 u1 (1.2)

where u1 and u2 is defined as the control inputs and y1 and y2 is defined as the measurements
or outputs

We now define the outputs and if necessary the derivatives of the outputs as states.
Hence, define the states

x1 = y1 , x2 = y˙1 , x3 = y2

1
 2
This gives the following set of 1st order differential equations for the states

x˙1 = x2 (1.3)
x˙2 = −k2 x − 1 − k1 x2 + u1 + k3 u2 (1.4)
x˙3 = −k5 x2 − k4 x3 + k6 u1 (1.5)

and the following measurements (outputs) variables

y 1 = x1 (1.6)
y 2 = x3 (1.7)

The model is put on matrix (State Space) form as follows

      
x˙1 0 1 0 x1 0 0 " #
       u1
 x˙2  =  −k2 −k1 0   x2  +  1 k3  (1.8)
      
u2
x˙3 0 −k5 k4 x3 k6 0
 
" # " # x1
y1 1 0 0   x2 

=   (1.9)
y2 0 0 1
x3

and finally in matrix form as follows

ẋ = Ax + Bu (1.10)
y = Cx + Du (1.11)

where u is the control vector, x is the state vector, y is the measurements vector and
x0 = x(t0 ) is the initial value of the state vector, which usually is assumed to be known.

Figure 1.1: Block diagram in state space

 3
3.1.2 Stability: System Characteristics
We defined the characteristics of a system by its characteristic equation, whose roots are
the poles of the system. We also saw how the locations of the poles indicate a system’s
performance - such as natural frequency, damping factor, system type - as well as whether
the system is stable.

The characteristic equation was defined to be the denominator polynomial of the sys-
tem’s transfer function (or transfer matrix) equated to zero. Hence, we should first obtain
an expression for the transfer matrix in terms of the state-space coefficient matrices, A, B,
C, D.

Recall that the transfer matrix is obtained by taking the Laplace transform of the gov-
erning differential equations, for zero initial conditions. Taking the Laplace transform of
both sides of the matrix state-equation, assuming zero initial conditions (i.e.x(0) = 0) yields
the following result:

sX(s) = AX(s) + BU (s) (1.12)

We can write the same as
(sI − A)X(s) = BU (s) (1.13)
or
X(s) = (sI − A)−1 BU (s) (1.14)
Similarly, taking the Laplace transform of the output equation
Y (s) = CX(s) + DU (s) (1.15)
Substituting equation 1.14 in equation 1.15 we get
Y (s) = C(sI − A)−1 BU (s) + DU (s) = [C(sI − A)−1 B + D]U (s) (1.16)
From Eq.1.16, it is clear that the transfer matrix, G(s), defined by Y (s) = G(s)U (s)
is given as:
G(s) = C(sI − A)−1 B + D (1.17)
Equation 1.17 tells us that the transfer matrix is a sum of the rational matrix (i.e. a
matrix whose elements are ratios of polynomials in s), C(sI − A)−1 B, and the matrix D.
Thus, D represents a direct connection between the input, U(s), and the output, Y(s), and
is called the direct transmission matrix. Systems having D = 0 are called strictly proper,
because the numerator polynomials of the elements of G(s) are smaller in degree than the
corresponding denominator polynomials . Hence, the characteristic polynomial of the system
must be related to the denominator polynomial resulting from the matrix, C(sI − A)−1 B
 4
Example: 1

Substituting Eqs. (3.107) and (3.108) into Eq. (3.106) we get

Note that the single-input, single-output system of Example above, the transfer function
has a denominator polynomial s2 − 2s + 5, which is also the characteristic polynomial of the
system . The denominator polynomial is equal to |(sI − A)| (Eq. (3.107)). Thus, the poles of
the transfer function are the roots of the characteristic equation, |(sI − A)| = 0. This is
also true for the multivariable system
 Example: 2

Obtain the transfer function. for lh.e system,

-[ 1 0 -
'.X;;;;0 I] [OJ
-1 1 X l u
L -2 -3 1
V = [1 0 I] X

[s
Sel1111tion

I O l
(s] - A) = 0 S, + 1 - L
-1 2 s 3

[2 2 -(s. + ])
(sI- -A )-I = s ? 5s 2 1 .s 2 +4s 4 s,+l l
s +4s+5

+ + 10s + 6 2
5+1 -2(s + L) (s+1)
Toe transfer fu.R.Clion �s give-n by,
- (s
s + 4s + 4
+l

[OJ
s + I )l l
2
T(s)
t!.(s) [J -2(s ie 1)
2

(s + o- 1 L
,

Where �(s} = sl + 5s2 + lOs + 6

l- s
T(s) =
A(s)
....!... [
[1 O l) s 2 5s :5 ]
(s IXs-J)

s(s -1)
s +Ss' + 10s 6
=-----
1
 5

Using linear algebra, the characteristic equation of a general, linear time-invariant system
is obtained from the following eigenvalue problem for the system:

Avk = λk vk (1.18)

where λk is the k th eigenvalue of the matrix A, and vk is the eigenvector associated with
the eigenvalue, λk The same equation can written as

(λI − A)v = 0 (1.19)

For the nontrivial solution of Eq. 1.19 (i.e.v 6= 0), the following must be true:

|(λI − A)| = 0 (1.20)

Equation 1.20 is another way of writing the characteristic equation, whose the roots are
the eigenvalues of A. Hence, the poles of the transfer matrix are the same as the eigenvalues of
the matrix A. Since A contains information about the characteristic equation of a system, it
influences all the properties s uch as s tability, performance and robustness of the s ystem. For
this reason, A is c alled the s ystem’s s tate-dynamics matrix.

Transition matrix:
The free (homogeneous) response of a system can be used to find the state transition
matrix. The homogenous state equation is given as

ẋ(t) = Ax(t) (1.21)

Taking laplace transform of both sides

sX(s) − X(0) = AX(s) (1.22)

Thus
(sI − A)X(s) = X(0) (1.23)
or
X(s) = (sI − A)−1 X(0) (1.24)
Taking Inverse Laplace transform

x(t) = `−1 [(sI − A)−1 ]X(0) (1.25)

but
I A A2
(sI − A)−1 = + 2 + 3 + ... (1.26)
s s s
 6
Hence
A2 t2 A3 t3
`−1 [(sI − A)−1 ] = I + At + + + . . . = eAt (1.27)
2!‘ 3!‘
Thus
x(t) = eAt x(0) (1.28)

eAt = T ransitionM atrix

The forced response (particular) response of the system can be found using convo-
lution,
For the state equation
ẋ = Ax + Bu (1.29)
Taking laplace transform yields

sX(s) − X(0) = AX(s) + BU (s) (1.30)

or
(sI − A)X(s) = X(0)+BU (s) (1.31)
or
X(s) = (sI − A)−1 X(0) + (sI − A)−1 BU (s) (1.32)
but
(sI − A)−1 = `[eAt ] (1.33)
Thus
X(s) = `[eAt ]X(0) + `[eAt ]BU (s) (1.34)
Using convolution for the forced component
Z t
A(t)
x(t) = e x(0) + eA(t−τ ) Bu(τ )dτ (1.35)
0

This assumed initial time as zero. If the initial time is given as t0 instead of 0 the solution
is modified to Z t
x(t) = eA(t−t0 ) x(t0 ) + eA(t−τ ) Bu(τ )dτ (1.36)
t0

The solution consists of two parts. The first part represents the autonomous response
(homogenous solution) driven only by initial values diferent from zero. The second term
represents the inhomogenous solution driven by the control variable, u(t).

The output equation can also be solved in time domain by substituting the state solution
into the output equation
 7

Example:
 8

Understanding the Stability Criteria through the State-Transition Matrix

The elements of eAt are linear combinations of eλk t , where λk for k = 1, 2, . . . n are the
distinct eigenvalues of the s ystem . Such elements can be expressed as e akt multiplied by
oscillatory terms, s in b k t and cos b k t where ak and b k are real and i maginary parts of the kth
eigenvalue, λk = ak + ibk .

If the real parts, ak , of all the eigenvalues are negative (−) , the initial responses of all
the state-variables will decay to zero as time t becomes large due to the presence of exp ak t
as the factor in all the elements of eAt. Hence, a system with all eigenvalues having negative
real parts is asymptotically stable. This is the first stability criterion(Asymptotic Stability).

By the s ame token, i f any eigenvalue λk , has a positive real part, ak then the correspond-
ing f actor e At will diverge to i nfinity as time, t, becomes l arge, s ignifying an unstable system .
This is the second stability criterion.

If a zero eigenvalue is repeated twice, it leads to the presence of terms such as ct, where c
is a constant, in the elements of eAt . More generally, if an eigenvalue, λk , which is repeated
twice has zero real part (i.e. λk = ibk , then eAt will have terms such as tsin(bk t) and tcos(bk t)
- and their combinations - in its elements. If an eigenvalue with zero real part is repeated
thrice, then eAt will have combinations of t2 sin(bk t), t2 cos(bk t), tsin(bk t), and / tcos(bk t) in
its elements. Similarly, for eigenvalues with zero real parts repeated larger number of times,
there will be higher powers of t present as coefficients of the oscillatory terms in the elements
of eAt . Hence, if any eigenvalue,λk having zero real part is repeated two or more times, the
presence of powers of t as coefficients of the oscillatory terms, sin(bk t) and cos(bk t), causes
 9
elements of eAt? to blow-up as time, t, increases, thereby indicating an unstable system.
This is the third stability criterion.

Note that individual initial responses to a specific initial condition may not be sufficient to
tell us whether a system is stable. Hence the need for BIBO stability

3.1.3 Controllability
The question of how we can (and if it is possible to) find a suitable control input u(t) that
will take the system from an initial state x(t0 ) to any desired final state x(t1 ) in a finite
(often very small) time, is answered by the theory of state controllability.
Definition 1.2 (State Controllability)
A system is said to be controllable if a control vector u(t) exists that will transfer the system
from any initial state x(t0 ) to some final state x(t) in a finite time interval.

A system described by a state space model ẋ = Ax + Bu with initial state x(t0 ) given is
controllable if for an arbitrarily finite time t1 > t0 exist a control function u(t) defined over
the time interval t0 ≤ t ≤ t1 , such that the final state, x(t1 ), can be arbitrarily specified.
There exist a few algebraic definitions which can be used for the analysis of state con-
trollability. Such a theorem is defined via the so called controllability matrix.
Applying the defination of complete state controllability we have
Z t1
x(t1 ) = 0 = eA(t1 )
x(0) + eA(t1 −τ ) Bu(τ )dτ (1.37)
0

or Z t1
x(0) = − e−Aτ Bu(τ )dτ (1.38)
0
From Sylvesters method to transition matrix solution
n−1
X
e−Aτ = αk (τ )Ak (1.39)
k=0

Substituting this in the above equation

n−1
X Z t1
k
x(0) = − A B αk (τ )u(τ )dτ (1.40)
k=0 0

Let Z t1
αk (τ )u(τ )dτ = βk (1.41)
0
Thus
n−1
X
x(0) = − Ak Bβk (1.42)
k=0
 10
or  
β0
 
h i  β1 
 
x(0) = − B AB . . . An−1 B  .  (1.43)
 .. 
 
βn−1
If the system is completely state controllable, the the above equation must be satisfied.
c [B AB A2B . . . An−1 B] are linearly independent or the
This requires that the vector
h i
2
rank of the n × n matrix B AB A B . . . A n−1 B be n or full rank
Theorem 1.11.1 (Controllability matrix)
A system described by a state space model ẋ = Ax + Bu is state controllable if the control-
lability matrix
Cn = [ B AB A2 B . . . An−1 B ] (1.44)
has full rank, i.e.,

rank(Cn) = n (1.45)
Note that for single input systems, i.e., r = 1, then Cn which implies that Cn should be
6 0 in order for the system to be state controllable.
invertible and that det(Cn) =

Controllability of a system can be easily determined if we can decouple the state-equations

of a system. Each decoupled scalar state-equation corresponds to a sub-system. If any of
the decoupled state-equations of the system is unaffected by the input vector, then it is
not possible to change the corresponding state variable using the input, and hence, the sub-
system is uncontrollable. If any sub-system is uncontrollable, i.e. if any of the state variables
is unaffected by the input vector, then it follows that the entire system is uncontrollable.
Controllability by Inspection
We can explore controllability from another viewpoint: that of the state equation itself.
When the system matrix is diagonal, as it is for the parallel form, it is apparent whether or
not the system is controllable. For example,
   
−a1 0 0 1
.    
x=
 0 −a2 0 x +  1 u
   (1.46)
0 0 −a3 1
or
x˙1 = −a1 x1 +u
x˙2 = −a2 x2 +u (1.47)
x˙3 = −a3 x3 +u
 11
Since each of Eqs. 1.46 or Eqs. 1.47is independent and decoupled from the rest, the control
u affects each of the state variables. This is controllability from another perspective.

Now let us look at the state equations for the system

   
−a4 0 0 0
   
x= 
 0 −a5 0  x +  1  u
  (1.48)
0 0 −a6 1

or
x˙4 = −a4 x4
x˙5 = −a5 x5 +u (1.49)
x˙6 = −a6 x6 +u
From the state equations in Eqs. 1.48 or Eqs. 1.49, we see that state variable x4 is not
controlled by the control u. Thus, the system is said to be uncontrollable. In summary, a
system with distinct eigenvalues and a diagonal system matrix is controllable if the input
coupling matrix B does not have any rows that are zero.

3.1.4 Observability
Definition 1.3 (State Observability)
A s ystem i s s aid to be observable i f at time t0, the s ystem s tate x(t0) can be exactly deter-
mined f rom observation of the output y(t) and inputs u(t), over a finite time i nterval.

A system described by a state space model ẋ = Ax + Bu and y = Cx with initial state

x(t0 ) is observable if from knowledge of known inputs, u(t), and outputs, y(t), over a time
it
interval t0 ≤ t ≤ t1 , is possible to compute the (initial) state vector, x(t0 ).

Theorem 1.12.1 (Observability matrix)

Define the observability matrix
 
C
 
 CA 
 
 
Oi = 
 CA
2 
 (1.50)
 .. 
 . 
 
CAi−1
The pair (C; A) is observable if and only if the observability matrix Oi for i = n has rank
n, i.e. rank(Cn) = n.
 11b

Simply stated, observability is the ability to deduce the state variables from a knowledge
of the input, u (t) , and the output, y(t) .
Observability by Inspection
We can also explore observability from the output equation of a diagonalized system. The
output equation for the diagonalized system of Figure 12.15(a) is
h i
y = Cx = 1 1 1 x (1.51)

On the other hand, the output equation for the unobservable system of s
h i
y = Cx = 0 1 1 x (1.52)

Notice that the first column of Eq. (1.52) is zero. For systems represented in parallel form
with distinct eigenvalues, if any column of the output coupling matrix is zero, the diagonal
system is not observable.
 1

State space System analysis tutorials

Stability Analysis
1. Transfer matrix (Poles)
2. State dynamics matrix A (Eigen values)
3. Transition matrix (transient and BIBO stability)

Example 3.1 Consider the two-dimensional state equation

        
ẋ1 (t) 0 1 x1 (t) 0 −1
= + u(t) x(0) = x0 =
ẋ2 (t) −2 −3 x2 (t) 1 1
 
 x1 (t)
y(t) = 1 1
x2 (t)

From      
s 0 0 1 s −1
sI − A = − =
0 s −2 −3 2 s+3

we find, by computing the matrix inverse and performing partial fraction

expansion on each element,
 
s+3 1
adj(sI − A) −2 s
(sI − A)−1 = = 2
|sI − A| s + 3s + 2
 
s+3 1
 (s + 1)(s + 2) (s + 1)(s + 2) 
=


−2 s
(s + 1)(s + 2) (s + 1)(s + 2)
 
2 −1 1 −1
s+1 + s+2 s+1 + s+2
= −2

2 −1 2 
+ +
s+1 s+2 s+1 s+2
It follows directly that

eAt = L−1 [(sI − A)−1 ]

  t ≥0
2e−t − e−2t e−t − e−2t
=
−2e−t + 2e−2t −e−t + 2e−2t
 2

For the specified initial state, the zero-input response component of the
state and output are
 −t 
−e
xzi (t) = e x0 =
At
yzi (t) = CeAt x0 = Cxzi (t) = 0 t ≥ 0
e−t

For a unit-step input signal, the Laplace domain representation of the

zero-state components of the state and output response are

Xzs (s) = (sI − A)−1 BU (s)

 
s+3 1
 
−2 s 0 1
= 2
s + 3s + 2 1 s
 
1
 (s + 1)(s + 2)  1
=

s
s
(s + 1)(s + 2)
 
1
 s(s + 1)(s + 2) 
=


1
(s + 1)(s + 2)
 
1/2 1 1/2
 s − +
s+1 s+2
=


1 1
−
s+1 s+2
Yzs (s) = CXzs (s) + DU (s)
 
1/2 1 1/2
  s − s+1 + s+2
= 1 1   + [0] 1
 1 1  s
−
s+1 s+2
1/2 1/2
= −
s s+2
from which
 
1/2 − e−t + 1/2e−2t 1
xzs (t) = yzs (t) = (1 − e−2t ) t ≥ 0
e−t − e−2t 2
 3

and complete state and output responses then are

 
1/2 − 2e−t + 1/2e−2t 1
x(t) = −t −2t y(t) = (1 − e−2t ) t ≥0
2e − e 2

Finally, the transfer function is given as

H (s) = C(sI − A)−1 B + D

 
s+3 1
 
 −2 s 0
= 1 1 2 +0
s + 3s + 2 1
s+1 s+1 1
= = =
s2 + 3s + 2 (s + 1)(s + 2) s+2

with associated impulse response

h(t) = e−2t t ≥0

 11

Controllability analysis

1. Through controllability matrix

2. Through inspection after decoupling state
equations

3.4 CONTROLLABILITY EXAMPLES

Example 3.4 Given the following single-input two–dimensional linear

state equation, we now assess its controllability.

ẋ1 (t) 1 5 x1 (t) −2
= + u(t)
ẋ2 (t) 8 4 x2 (t) 2
The controllability matrix P is found as follows:
−2 8 −2 8
B= AB = P =
2 −8 2 −8
Clearly, |P | = 0, so the state equation is not controllable. To see why this
is true, consider a different state definition
z1 (t) x1 (t)
=
z2 (t) x1 (t) + x2 (t)
The associated coordinate transformation is
x(t) = T z(t)
x1 (t) 1 0 z1 (t)
=
x2 (t) −1 1 z2 (t)
 12

Applying this coordinate transformation yields the transformed state

equation
ż1 (t) −4 5 z1 (t) −2
= + u(t)
ż2 (t) 0 9 z2 (t) 0

We see that ż2 (t) does not depend on the input u(t), so this state variable
is not controllable.

Example 3.5 Given the following three-dimensional single-input state

equation, that is,
      
ẋ1 (t) 0 1 0 x1 (t) 0
 ẋ2 (t)  = 0 0 1  x2 (t)  + 1 u(t)
ẋ3 (t) −6 −11 −6 x3 (t) −3

we construct the controllability matrix P using

 
0
B= 1
−3
    
0 1 0 0 1
AB = 0 0 1 1 = −3
−6 −11 −6 −3 7
    
0 1 0 1 −3
A2 B = A(AB) = 0 0 1 −3 = 7
−6 −11 −6 7 −15

This yields
 
P = B AB A2 B
 
0 1 −3
= 1 −3 7
−3 7 −15

To check controllability, we calculate

|P | = | B AB A2 B |
 
 0 1 −3 
 
=  1 −3 7
 
−3 7 −15
= [0 + (−21) + (−21)] − [(−27) + 0 + (−15)]
 13

= −42 − (−42)
=0

and thus rank P < 3. This indicates that the state equation is not control-
lable. The upper left 2 × 2 submatrix

0 1
1 −3

has nonzero determinant, indicating that rankP = 2.

Example 3.6 We investigate the controllability of the three–dimen-

sional state equation
      
ẋ1 (t) 0 1 0 x1 (t) 0
       
 ẋ2 (t)  = 0 0 1  x2 (t)  +  0  u(t)
ẋ3 (t) −a0 −a1 −a2 x3 (t) 1
 
x (t)
  1 
y(t) = b0 b1 b2  x2 (t) 
x3 (t)

which the reader will recall is a state-space realization of the transfer

function
b2 s 2 + b1 s + b0
H (s) = 3
s + a2 s 2 + a1 s + a0

The controllability matrix P is found as follows:

   
0 0
B =  0  AB =  1 
1 −a2
 
1
AB=
2  −a 2 
a2 − a1
2
 
P = B AB A2 B
 
0 0 1
= 0 1 −a2 
1 −a2 a22 − a1
 14
Example 3.7 Given the five-dimensional, two-input state equation
      
ẋ1 (t) 0 1 0 0 0 x1 (t) 0 0
 ẋ2 (t)   0 0 0 0    
0   x2 (t)   1 0
    u (t)
       1
 ẋ3 (t)  =  0 0 0 1 0   x3 (t)  +  0 0
       u2 (t)
 ẋ4 (t)   0 0 0 0 1   x4 (t)   0 0
ẋ5 (t) 0 0 0 0 0 x5 (t) 0 1

the controllability matrix is

 
P = b1 b2 | Ab1 Ab2 | A2 b1 A2 b2 | A3 b1 A3 b2 | A4 b1 A4 b2
     
0 0 1 0 0 0 0 0 0 0
   
1 0 0 0 0 0 0 0 0 0
     
     
= 0 0 0 0 0 1 0 0 0 0
     
0 0 0 1 0 0 0 0 0 0
   
0 1 0 0 0 0 0 0 0 0

This state equation is controllable because P has full-row rank, as can

be seen from the pattern of ones and zeros. Also, columns 1, 2, 3, 4,
and 6 form a linearly independent set. Since the remaining columns are
each zero vectors, this turns out to be the only way to select five linearly
independent columns from P .

Example 3.8 Now consider the following five-dimensional, two-input

state equation
      
ẋ1 (t) 0 1 0 0 0 x1 (t) 0 0
 ẋ2 (t)   1 0 −1 0 1    
     x2 (t)   1 −1  u (t)
       1
 3  = 0
ẋ (t) 0 0 1 0   x3 (t)  +  0 0
       u2 (t)
 ẋ4 (t)   0 0 0 0 1   x4 (t)   0 0
ẋ5 (t) 0 −1 0 −1 0 x5 (t) 1 1
This equation differs from the preceding example only in the second and
fifth rows of both A and B. These adjustments yield the more interesting
controllability matrix
 
P = b1 b2 | Ab1 Ab2 | A2 b1 A2 b2 | A3 b1 A3 b2 | A4 b1 A4 b2
     
0 0  1 −1  1 1  0 0  −2 −2
 1 −1  1 1  0 0  −2 −2  2 −2 
  
     
= 0 0 0 0 1 1  −1 1  −2 −2 
     
0 0  1 1  −1 1  −2 −2  1 −1 
1 1  −1 1  −2 −2  1 −1  4 4
which also has rank equal to 5. If we search from left to right for five
linearly independent columns, we find that the first five qualify. However,
there are many more ways to select five linearly independent columns
from this controllability matrix, as the reader may verify with the aid of
 15
Observability analysis

4.2 OBSERVABILITY EXAMPLES

Example 4.1 Consider the two–dimensional single-output state equation

      
ẋ1 (t) 1 5 x1 (t) −2
= + u(t)
ẋ2 (t) 8 4 x2 (t) 2
 
x (t)
y(t) = [ 2 2 ] 1 + [0]u(t)
x2 (t)

for which the associated (A, B) pair is the same as in Example 4.1. The
observability matrix Q is found as follows:

C = [2 2]
CA = [ 18 18 ]
so  
2 2
Q=
18 18
Clearly |Q| = 0 so the state equation is not observable. Because rank
Q < 2 but Q is not the 2 × 2 zero matrix, we have rankQ = 1 and
nullityQ = 1.
To see why this state equation is not observable, we again use the state
coordinate transformation given by:
      
z1 (t) x1 (t) 1 0 x1 (t)
= =
z2 (t) x1 (t) + x2 (t) 1 1 x2 (t)

which yields the transformed state equation

      
ż1 (t) −4 5 z1 (t) −2
= + u(t)
ż2 (t) 0 9 z2 (t) 0
 
z1 (t)
y(t) = [ 0 2 ] + [0]u(t)
z2 (t)
 16

Here both the state variable z2 (t) and the output y(t) are decoupled from
z1 (t). Thus, z1 (0) cannot be determined from measurements of the zero-
input response yzi (t) = 2e9t z2 (0). This is why the given state equation is
not observable.
Also, note that x0 = [1, −1]T satisfies Qx0 = [0, 0]T and we conclude
from the proof of Theorem 4.2 that x0 is a nonzero unobservable state.

Example 4.2 Given the following three-dimensional single-output

homogeneous state equation, that is,

    
ẋ1 (t) 0 0 −6 x1 (t)
 ẋ2 (t)  =  1 0 −11   x2 (t) 
ẋ3 (t) 0 1 −6 x3 (t)
 
x1 (t)
y(t) = [ 0 1 −3 ]  x2 (t) 
x3 (t)

we construct the observability matrix as follows:

C = [ 0 1 −3 ]
 
00 −6
CA = [ 0 1 −3 ]  10 −11  = [ 1 −3 7 ]
01 −6
 
0 0 −6
CA2 = (CA)A = [ 1 −3 7 ]  1 0 −11  = [ −3 7 −15 ]
0 1 −6

yielding    
C 0 0 −6
 
Q =  CA  =  1 −3 7
CA2 −3 7 −15

To check observability, we calculate

C
|Q| = CA
CA2
 17

0 1 −3
= 1 −3 7
−3 7 −15
= [0 + (−21) + (−21)] − [(−27) + 0 + (−15)]
= −42 − (−42)
=0

and thus rank Q < 3. This indicates that the state equation is not observ-
able, so there exist nonzero unobservable states for this state equation.
The upper left 2 × 2 submatrix
 
0 1
1 −3

has nonzero determinant, indicating that rankQ = 2 and nullityQ = 3 −

2 = 1 (by Sylvester’s law of nullity). Consequently, any nonzero solution
to the homogeneous equation

Qx0 = 0

will yield a nonzero unobservable state. Applying elementary row opera-

tions to the observability matrix Q yields the row-reduced echelon form
 
1 0 −2
 
QR =  0 1 −3 
0 0 0

from which an easily identified solution to

QR x0 = 0

is  
2
 
x0 =  3 
1

Moreover, any nonzero scalar multiple of this solution also yields a

nonzero unobservable state.

 18

Example 4.3 We investigate the observability of the three-dimensional

state equation
      
ẋ1 (t) 0 0 −a0 x1 (t) b0
 ẋ2 (t)  =  1 0 −a1   x2 (t)  +  b1  u(t)
ẋ2 (t) 0 1 −a2 x2 (t) b2
 
x1 (t)
y(t) = [ 0 0 1 ]  x2 (t) 
x2 (t)

which the reader will recall is a state-space realization of the transfer

function
b2 s 2 + b1 s + b0
H (s) = 3
s + a2 s 2 + a1 s + a0

The observability matrix Q is found as follows:

 
C
Q =  CA 
CA2
 
0 0 1
= 0 1 −a2 
1 −a2 a2 − a1
2

This observability matrix is identical to the controllability matrix P from

Example 3.3. The observability matrix Q is independent of the transfer
function-numerator coefficients b0 , b1 , and b2 . The determinant of the
observability matrix is |Q| = −1  = 0, so the state equation is observable.
Note that this outcome is independent of the characteristic polynomial
coefficients a0 , a1 , and a2 , so a state-space realization in this form is
always observable. This is also true for any system order n, as we will
demonstrate shortly.

Example 4.4 Consider the five-dimensional, two-output homogeneous

state equation
    
ẋ1 (t) 0 0 0 0 0 x1 (t)
 ẋ2 (t)   1 0 0 0 0   x2 (t) 
    
 ẋ3 (t)  =  0 0 0 0 0   x3 (t) 
 ẋ (t)   0 0 1 0 0   x4 (t) 
4
ẋ5 (t) 0 0 0 1 0 x5 (t)
 19

 
x1 (t)
     x (t) 
y1 (t) 0 1 0 0 0  2 
=  x (t) 
y2 (t) 0 0 0 0 1  3 
x4 (t)
x5 (t)

The observability matrix is constructed as follows

 
0 1 0 0 0
0
 0 0 0 1 

  1 
C  0 0 0 0 
 CA

 0 0 0 1 0 

   
  0 0 0 0 0 
Q= 2 = 
 CA  0 0 1 0 0 
   
 CA3   
0 0 0 0 0 
CA4  
0 0 0 0 0 
 
0 0 0 0 0 
0 0 0 0 0

Q has full-column rank 5 because of the pattern of ones and zeros. There-
fore, the state equation is observable. Furthermore, rows 1, 2, 3, 4, and 6
form a linearly independent set of 1 × 5 row vectors. Since the remaining
rows are each 1 × 5 zero vectors, there is only one way to select five
linearly independent rows from Q.

Example 4.5 Consider now the five-dimensional, two-output homoge-

neous state equation

    
ẋ1 (t) 0 1 0 0 0 x1 (t)
 ẋ2 (t)   1 0 0 0 −1   x2 (t) 
    
 ẋ3 (t)  =  0 −1 0 0 0   x3 (t) 
 ẋ (t)   0 0 1 0 −1   x4 (t) 
4
ẋ5 (t) 0 1 0 1 0 x5 (t)
 
x1 (t)
     x (t) 
y1 (t) 0 1 0 0 1  2 
=  x (t) 
y2 (t) 0 −1 0 0 1  3 
x4 (t)
x5 (t)
 20

differing from the preceding example only in the second and fifth columns
of both A and C. These modifications lead to the observability matrix
 
0 1 0 0 1
 0 −1
 0 0 1
   1 
C  1 0 1 −1 
 CA  
 −1 1 0 1 1
   
   1 0 1 −1 −2 
Q=  
 CA  =  1
2 
   0 1 1 −2 
 CA3   
 0 −2 −1 −2 1
4  
CA  0 −2 1 −2 −1 
 
 −2 2 −2 1 4
−2 −2 −2 −1 4
This observability matrix also has rank equal to 5, indicating that the state
equation is observable. In contrast to the preceding example, however,
there are many ways to select five linearly independent rows from this
observability matrix, as the reader may verify with the aid of MATLAB.
Lecture 4

State Feedback
Designing controllers in state space.
8.1 State Feedback
We assume that the process to be controlled is given in state space form, i.e.
ẋ = Ax + Bu
(8.1)
y = Cx
For simplicity we assume that the process lacks a direct term, i.e. that the ma-
trix D = 0. This is a realistic assumption since processes with a direct term are
uncommon.
The transfer function of the process is given by

Y (s) = C (s I − A)−1 BU (s)

where the denominator polynomial
det(s I − A)
is the characteristic polynomial of the process.
We further assume that we can measure all process states. This is obviously
an unrealistic assumption in most cases. In the next chapter we will, however, see
that this assumption can be relaxed and that the states can be computed from the
only signals we normally posses over, i.e. the control and output signals. The
controller structure is shown in Figure 8.1. The controller equation becomes

r P u y
kr Process

x
−K

Figure 8.1 State feedback.

The controller equation becomes

u = kr r − k1 x1 − k2 x2 − · · · − kn xn = kr r − K x (8.2)
where the vectors K and x are given by
 
 x1 

 

  

 x2 


K =  k1 k2 kn  
x= 

··· 
 .
. 


 
 . 
  
xn

68
 8.1 State Feedback

In state feedback the control signal is hence a linear combination of the state
variables and the setpoint.

The Closed-Loop System If we combine the control law (8.2) with the process
model (8.1) the state-space description of the closed-loop system is obtained.

ẋ = ( A − B K ) x + Bkr r
(8.3)
y = Cx

Here, the setpoint r is our new input. The corresponding transfer function is given
by
Y (s) = C (s I − ( A − B K ))−1 Bkr R(s)
where the characteristic polynomial has become

det(s I − ( A − B K ))

The state feedback has changed the matrix A of the open-loop system (8.1) into
A − B K, which is the corresponding matrix for the closed-loop system (8.3). Since K
can be chosen arbitrarily, we have a certain freedom in determining the eigenvalues
of this matrix.
The controller parameter kr is apparently unaffected by the pole placement of
the closed-loop system. To begin with we shall choose kr such that the static gain of
the closed loop system becomes unity in order to achieve y = r in stationarity.
However, we may introduce introduce integral action to achieve this goal.
We will now illustrate the synthesis method by an example.

Example 8.1—State feedback control of an electric motor

The transfer function of an electric motor is given by

100
G P (s) =
s(s + 10)

where the current is the input and the shaft angle is the output. The transfer
function can be divided into two parts according to Figure 8.2, where we have also
marked the angular speed. If we introduce the angle and the angular speed as states,
the process can be described in state-space form as

ẋ1 = −10x1 + 100u

ẋ2 = x1
y = x2

which can be written in the matrix form

   
 −10 0 
  100 
ẋ = 
 x+
 u

1 0 0
 
y = 0 1 x

u 100 x1 1 y = x2
s + 10 s

Figure 8.2 Block diagram of the motor in Example 8.1. The state x1 corresponds to the angular
speed, while the state x2 represents the angle.

69
Lecture 8. State Feedback

r P u 100 x1 1 y = x2
kr
s + 10 s

− k1

− k2

Figure 8.3 The state feedback motor control in Example 8.1.

Now we establish feedback connections from the angle and the angular speed
according to Figure 8.3. The control law

u = kr r − k1 x1 − k2 x2 = kr r − K x

yields the closed-loop system

   
 −10 − 100k1 −100k2   100 
ẋ = 
 x+
   kr r

1 0 0
 
y = 0 1 x

The characteristic polynomial becomes

s + 10 + 100k1 100k2
det(s I − ( A − B K )) =
−1 s

= s2 + (10 + 100k1 )s + 100k2

Since we can achieve an arbitrary second-order characteristic polynomial by choosing

the parameters k1 and k2 , the poles of the closed-loop system can be places freely.
Assume that the characteristic polynomial of the closed-loop system is given by

s2 + 2ζωs + ω 2

This yields the following controller parameters

2ζω − 10 ω2
k1 = k2 =
100 100

Now it remains to determine the parameter kr such that y = r in stationarity.

One could always determine kr by calculating the closed-loop transfer function and
thereafter assure that G(0) = 1, i.e. that the static gain becomes unity. It is, however,
often more efficient to investigate the stationary relation as ẋ = 0 directly in the
state-space description. For our example it holds that
   
 −10 − 100k1 −100k2   100 
ẋ = 0 = 
 x+
   kr r

1 0 0
 
y = 0 1 x

70
 8.2 Controllability

The measurement signal y for different ω The measurement signal y for different ζ
1.2 1.2

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

The control signal u for different ω The control signal u for different ζ
10 5

8 4

6 3

4 2

2 1

0 0

−2 −1
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Figure 8.4 State feedback control of the motor in Example 8.1. The figures to the left show
the controller corresponding to ζ = 0.7 and ω = 10, 20, 30, where the fastest control action
is achieved by the highest frequency. The figures to the right show the control performance for
ω = 20 and ζ = 0.5, 0.7, 0.9, where the most damped control action corresponds to the largest
relative damping.

The second state-space equation implies that x1 = 0 in stationarity. That this must
be the case is obvious, since x1 represents the angular speed. By inserting x1 = 0
into the first equation and exploiting that y = x2 we observe that y = r if

kr = k2

We have now completed our synthesis. Figure 8.4 shows step responses corre-
sponding to different choices of the design parameters ζ and ω . The figure shows
that the design parameter ω is an efficient way to specify the speed of a system and
that the relative damping ζ is a good measure of the damping of the system.

8.2 Controllability

In the previous example we could place the poles of the closed-loop system arbitrarily
by means of the state feedback. An interesting question is whether this is always
possible. We begin the investigation by an example.

Example 8.2—Controllability
Assume that the process we are about to control is described by the following
equations
   
 −1 0  1
ẋ = Ax + Bu = 
 x+
   u
0 −2 0

y = C x + Du

71
Lecture 8. State Feedback

Since the A matrix is diagonal, one can directly observe its eigenvalues and thereby
the poles of the process, which lie in −1 and −2. State feedback yields the matrix
 
 −1 − k1 − k2 
A − BK = 
 

0 −2

This, in terms, gives us the characteristic polynomial of the closed-loop system

det(s I − ( A − B K )) = (s + 1 + k1 )(s + 2)

Here we see that we cannot place the poles arbitrarily. By means of k1 we can
move the pole which initially lays in −1. However, there is no way for us to move the
pole in −2. We also see that the parameter k2 does not appear in the characteristic
polynomial. Hence we do not benefit from measuring x2 and involve the measurement
in the feedback.
The cause of our problem is clearly visible in the state-space description. Its
second equation is given by
ẋ2 = −2x2
and is unaffected by the control signal u. Consequently, this state is not controllable.

This reasoning leads us to the concept of controllability, which is defined in the

following way:
A state vector x0 is said to be controllable if there exists a control sig-
nal which brings x from the origin to x0 in a finite time. A system is
controllable if all its states are controllable.
If a system is controllable it is possible to place its poles arbitrarily using state
feedback. As seen from the definition, controllability is unrelated to the output y.
The definition concerns the state vector and the control signal. In the state-space
description (8.1) we thus see that the controllability is determined by the A and B
matrices.
Whether a system is controllable can be determined by studying the controlla-
bility matrix, which is defined as
 
Ws =  B AB A2 B ··· An−1 B 

where n is the degree of the system. One can show that a system is controllable if
and only if the controllability matrix Ws consists of n linearly independent columns.
In the non-controllable case the columns of Ws tell which, if any, of the states are
controllable. This is illustrated in the examples below.
We investigate the controllability of the two systems studied in examples previ-
ously in this lecture.

Example 8.3—Controllability
In Example 8.1 an electric motor described by the equations
   
 −10 0   100 
ẋ = 
 x+
  u

1 0 0
 
y = 0 1 x

was studied. The controllability matrix of this process is given by

   100 −1000 
 
Ws =  B AB  = 
 

0 100

72
 8.2 Controllability

The columns of this matrix are linearly independent. One way to determine whether
the columns of a quadratic matrix are linearly independent is to investigate if the
determinant is non-zero. Since the columns of Ws are linearly independent, the
process is controllable. We observed this already in Example 8.1, since the poles
could be arbitrarily placed.
Let us now investigate the controllability of the system in Example 8.2. There
the A and B matrices were given by
   
 −1 0  1
A=
 
 B=
  
0 −2 0

This yields the controllability matrix

   1 −1 
 
Ws =  B AB  = 
 

0 0

The columns of this matrix are linearly dependent and det Ws = 0. The system in
Example 8.2 is thus not controllable. The columns of Ws also show that x1 is a
controllable state, while x2 is not controllable.

A Controllability Example
Controllability is an abstract concept. Processes are often constructed so that con-
trollability is achieved for the states which one wishes to control. Despite this, the
concept of controllability is still important and it is good to develop an intuitive
understanding of it. We will return to this later in the course.
In order to increase the understanding of the controllability concept, we will now
investigate a few physical processes regarding their controllability. The processes to
be investigated are shown in Figure 8.5 and are made up by interconnected water
tanks and the corresponding flows.
By writing down the mass balance equations for the individual tanks, we obtain
a dynamical model. If the level in a tank is denoted x it holds that

ẋ = qin − qout

where qin is the inflow and qout the outflow from the tank. All tanks are equipped
with a hole in their bottom which makes the inflow approximately proportional to
the level in the corresponding tank. The inflow of a tank is made up by either of the
outflow from a higher tank or an auxiliary flow u, which is our control signal.

Process A First consider process A in Figure 8.5. In this case the controlled flow
enters the upper tank. The following mass balances are obtained

ẋ1 = u − ax1
ẋ2 = ax1 − ax2

where x1 and x2 denote the levels in the upper and lower tank, respectively. The
first equation tells us that the inflow of the upper tank is given by u whereas the
outflow is proportional to the level in the tank. From the second equation we obtain
that the inflow of the lower tank equals the outflow of the upper tank and that the
outflow of the lower tank is proportional to the level in the tank. The dynamics of
process A are thus described by
   
 −a 0  1
ẋ = Ax + Bu = 
 x+
   u
a −a 0

As a consequence, the controllability matrix is given by

   1 −a 
 
Ws =  B AB  = 
 

0 a

73
Lecture 8. State Feedback

1 1

u u

2 2 1 2

A B C

Figure 8.5 Several connections of water tanks leading to different cases regarding controlla-
bility.

The columns of this matrix are linearly independent, and the determinant is
det Ws = a ,= 0. Hence, the system is controllable. The columns of the controllability
matrix are shown graphically in Figur 8.6, where it’s obvious that they are linearly
independent. This means that we can control both levels to take on arbitrary values,
in accordance with the definition of controllability. Observe that the definition does
not require that we should be able to maintain arbitrary constant levels. From
studying the equations one realizes that this is only possible for levels which satisfy

u = ax1 = ax2

In stationarity we thus have the same level in both tanks. Finally we must consider
the validity of the equations. In reality it holds that the levels cannot take on
negative values, neither can the control signal be negative given that there is no
pump which could suck water from the tank. As a consequence we have x1 ≥ 0,
x2 ≥ 0 and u ≥ 0.

Process B Now consider process B, shown in Figure 8.5. In this case the controlled
flow enters the lower tank. This infers that we can no longer control the level in the
upper tank. Let us now confirm this intuitively drawn conclusion by applying the
above introduced analysis method.
The following balance equations constitute a model of process B

ẋ1 = − ax1
ẋ2 = u + ax1 − ax2

x2 x2 x2

x1 x1 x1

Process A Process B Process C

Figure 8.6 The columns of the controllability matrix in the three examples.

74
 8.2 Controllability

The process dynamics can therefore be written

   
 −a 0  0
ẋ = Ax + Bu = 
 x+
   u
a −a 1

The controllability matrix is thus

  0 0 


Ws =  B AB  = 
 

1 −a

The columns of the matrix are linearly dependent, the determinant is given by
det Ws = 0 and the system is hence not controllable. The columns of Ws further
show that the level in the lower tank is controllable, as opposed to the upper tank
level. This is also obvious from Figur 8.6.

Process C In process C, shown in Figure 8.5, the flow enters both tanks, such
that half of the flow enters tank 1 whereas the other half enters tank 2.
The following balance equations hold for tank C

ẋ1 = 0.5u − ax1

ẋ2 = 0.5u − ax2

The dynamics of process C can therefore be written

   
 −a 0   0.5 
ẋ = Ax + Bu = 
 x+
  u

0 −a 0.5

The controllability matrix becomes

   
 0.5 −0.5a 
Ws =  B AB  = 
 

0.5 −0.5a

The columns of the matrix are linearly dependent, the determinant is det Ws = 0 and
the system is hence not controllable. The construction is such that if the levels are
initially equal, they will remain equal. This can also be observed from the columns
of Ws , and the illustration in Figur 8.6.

75
The roots of the equation 1.57 are the open loop poles or eigenvalues. For the closed loop 14
system described above, the characteristic equation is

|(sI − A + BK)| = 0 (1.58)

The roots of equation 1.58 are the closed loop poles or eigenvalues

3.2.1 Regulator design by pole placement

The pole placement control problem is to determine a value of K that will produce a desired
set of closed-loop poles. With a regulator, r(t) = 0 and therefore equation 1.54 becomes

u = −Kx (1.59)
Thus the control u(t) will drive the system from a set of initial conditions x(0) to a set of
zero states at time tl , i.e. x(tl ) = 0.

There are several methods that can be used for pole placement.
(a) Direct comparison method: If the desired locations of the closed-loop poles (eigen-
values) are

s = µ1 s = µ2 . . . s = µn (1.60)
then from equation 1.58

14

(b) Using Arckerman’s method:

Example: 15
Given the system described by the state and output equation below, Evaluate the coefficints
of the state feedback gain matrix.

Solution:

Using direct comparison method

Using Arckerman’s method
16
9.2 Observability

So far we have assumed that all process states have been directly measurable.
Normally this is not the case. However, if one can estimate the state vector by
studying the control and measurement signals, one could close the loop over the
estimates rather than the real states.
In Example 8.1 we utilized state feedback from two states in order to control an
electric motor. The measurement signal was the angle of the motor shaft and the
other state was given by the angular speed of the shaft. If the angular speed cannot
be directly measured, it can be estimated by e.g. deriving the angle measurement.
Before describing how to estimate the state vector, we will ask ourselves the
principal question whether it is generally possible to estimate the state vector merely
by studying u and y. The answer is that it is possible if the system is observable.
Observability is defined in the following way:

77
Lecture 9. Kalman Filtering

A state vector x0 ,= 0 is not observable if the output is y( t) = 0 when

the initial state vector is x(0) = x0 and the input is given by u( t) = 0. A
system is observable if it lacks non-observable states.
As seen from the definition, observability has nothing to do with the control
signal u. Rather, it concerns the state vector and the measurement signal. In the
state-space description (9.1) we see that this implies that observability is determined
solely by A and C.
Whether a system is observable can be determined by studying the observability
matrix which is defined in as
 
 C 

 


 CA  

 


 2 

Wo = 
 C A 

 


 .
..




 


 

n−1
CA

Here n is the degree of the system. One can show that the system is observable if
and only if the observability matrix Wo has n linearly independent rows. If x0 is a
non-observable state, it fulfills the equation

Wo x0 = 0

As seen, observability and the observability matrix have strong resemblance

to controllability and the controllability matrix, which we studied in the previous
lecture. We will nu study an observability example.

Example 9.1—Observability
A process is described by the following equations:
 
 −1 0 
ẋ = 
 x

0 −1
 
y = 1 −1  x

We can e.g. imagine that the equations describe the process in Figure 9.2, where the
states are given by the tank levels. The measurement signal is the difference between
the tank levels. The system lacks input and we study only the state evolution after
various initial states.
We shall now investigate the observability of the process. The observability
matrix is    
 C     1 −1  
Wo =   = 
CA −1 1
The rows of this matrix are not linearly independent, the determinant is det Wo = 0
and the system is not observable. From the equation

Wo x0 = 0

x1 x2

1 − 2
+ P

Figure 9.2 A physical interpretation of the process dynamics in Example 9.1.

78
 9.2 Observability

x2

b d x1

Figure 9.3 Different initial states in Example 9.1.

a b
2 2

1.5 1.5

1 1

0.5 0.5

0 0
0 1 2 3 4 5 0 1 2 3 4 5

c d
2 2

1.5 1.5

1 1

0.5 0.5

0 0
0 1 2 3 4 5 0 1 2 3 4 5

Figure 9.4 The measurement y corresponding to the initial states in Figure 9.3.

we see that the non-observable states can be written

 
 a
x0 = 
 
a

If we return to the definition of observability we can justify that this is correct.

If the initial levels in the two tanks are equal they will empty in exactly the same
way and the measurement remains y = 0.
Figure 9.3 shows four initial states for the levels in the two tanks. The corre-
sponding outputs y are shown in Figure 9.4
The initial state a is a non-observable state. The measurement is consequently
y = 0. The initial state b yields a response in the measurement. However, this is
the same response obtained for the initial state c. All initial states which have the
same difference between the levels x1 and x2 will yield the same response in the
measurement signal y and it is hence not possible to tell them apart. The initial
state d yields a response which differs from the others since the difference between
x1 and x2 is not the same as for the other three cases.

79
 We shall examine a class of feedback controllers constructed in two stages:
1. building an observer : a dynamic system that is driven by the inputs and the outputs of the
plant, and produces an estimate of its state variables;
2. using the estimated state instead of the actual state in a state feedback scheme.
The resulting controller is termed an observer-based controller or a model-based controller. A
diagram of the structure of such a controller is given in Figure 1.4.

A full-order state observer estimates all of the system state variables. If, however, some of the
state variables are measured, it may only be necessary to estimate a few of them. This is
referred to as a reduced-order state observer.
An observer comprises a real-time simulation of the system or plant, driven by the same input as
the plant, and by a correction term derived from the difference between the actual output of the
plant and the predicted output derived from the observer.
The term 'model-based" for controllers based on an observer refers to the fact that the observer
uses a model of the plant as its core.
All observers use some form of mathematical model to produce an estimate x̂ of the actual state
vector x. Figure 1.4 shows a simple arrangement of a full-order state observer.

9.3 State Observers

If the system is observable it is possible to estimate the state vector by studying the
input u and the output y. We shall deduce the state observer in two steps. As a first step
we shall see what happens if we try to estimate x by simply simulating the process.

State Estimation through Simulation

The prerequisite of state estimation is that the process is known and given in state
space form and that the control signal u as well as the measurement signal y are
available. We can simulate the process in the following way:

x̂˙ = Ax̂ + Bu

Here x̂ is the estimate of the state vector

˙ˆ x. We drive the simulation with the same
control signal u as we apply to the real process. Will this work? Will the estimated
state vector x̂ converge to x?

Introduce the estimation error

x̃ = x − x̂

The derivative of the estimation error becomes

x̃˙ = ẋ − x̂˙ = Ax + Bu − A x̂ − Bu = A x̃

Whether the estimation error approaches zero depends on the matrix A. If A has all
its eigenvalues in the left half plane, i.e. if the process is asymptotically stable, the
estimation error will converge to zero. The convergence rate depends on where in
the left half plane the eigenvalues lie. Estimating the state vector by simulating the
process might thus work in some cases. For the estimation we use the control signal
u, while we ignore the information available through the measurement y. State
estimation by derivation of y has a corresponding shortcoming.
80
 45

Fig. 3.1 Figure 1.3: A simple full-order state observer

In Figure 8.8, since the observer dynamics will never exactly equal the system dynamics,
this open-loop arrangement means that x and x̂ will gradually diverge. If however, an output
vector ŷ is estimated and subtracted from the actual output vector y, the difference can be used,
in a closed-loop sense, to modify the dynamics of the observer so that the output error (y − ŷ)
is minimized. This arrangement, some- times called a Luenberger observer (1964), is shown in
Figure 8.9.

State observer
The state observer estimates the state vector in the following way:

x̂˙ = Ax̂ + Bu + Ke (y − ŷ)

ŷ = Cx̂ (9.3)

Compared to the simple simulation method, we have now introduced a correction term. Here
the difference between the real measurement signal y and the estimated measurement signal ŷ
affects the estimation.
46

Fig. 3.2 Figure 1.4: The Luenberger full-order state obsever

By merging the two equations in Equation (9.3) the state observer can be written in the form

ẋ = (A − KeC)x̂ + Bu + Ke y(9.4)

From this form it is obvious how the state observer is driven by the two signals u and y.
The estimation error x̃ decreases according to

ẋ = ẋ − ẋ = Ax + Bu − Ax̂ − Bu − KeC(x − x̂)

= (A − KeC)x̃
The properties of the state observer are no longer determined merely by the A matrix, but rather
by the A − KeC matrix, in which the vector Ke is a free parameter. We can consequently choose a
desired decrease rate of the estimation error x̃. The choice is a compromise between speed and
sensitivity towards disturbances and modelling errors. A method used to determine Ke is illustrated
by the following example.
 47

EXAMPLE 9.2-STATE ESTIMATION OF PENDULUM DYNAMICS

In this example we shall balance an inverted pendulum. We leave the control part to the next
lecture and put our focus on process state estimation. The pendulum to
be controlled is shown in Figure 9.5.
We assume that we can measure the angle ϕ which consequently becomes our measurement
signal, i.e. y = ϕ. The control objective is to balance the pendulum so that y = ϕ = 0. The setpoint
will thus be r = 0. The control signal u is proportional to the force affecting the cart. For simplicity
we assume that u = z̈.
There are two forces affecting the pendulum. The gravitational force strives to increase the
angle ϕ and thus causing the pendulum to fall down. The control signal can give the pendulum an
acceleration in the opposite direction and thus re-erect the pendulum.
A momentum equation with suitable choice of states, linearization and parameters yields the
following process model of the pendulum:

ϕ̈ = ϕ − u

A second-order system requires at least two states for its state-space representation. We choose
the angle ϕ and its derivative ϕ̇ to represent these states.

x1 = ϕ
x2 = ϕ̇
This yields the following state-space description of the process:
! !
0 1 0
ẋ = x+ u = Ax + Bu
1 0 −1
 
y= 1 0 x = Cx

A state observer estimating the state vector is given by Equation (9.4) and our task is to
determine the vector L so that the matrix A − LC obtains the desired eigenvalues. We have
! ! !
0 1 ke1   −ke1 1
A − KeC = − 1 0 =
1 0 ke2 1 − ke2 0
The eigenvalues are given by the characteristic polynomial

s + ke1 −1
det(sI − (A − KeC)) = = s2 + ke1 s − 1 + ke2
−1 + ke2 s
We can now place the two poles arbitrarily by choosing ke1 and ke2 adequately. E.g. assume
that we desire to place the poles in s = −4 ± 4i. This yields the desired characteristic polynomial

(s + 4 − 4i)(s + 4 + 4i) = s2 + 8s + 32
48

Fig. 3.3 Figure 9.5 The pendulum in Example 9.2.

By comparing the two polynomials, we arrive at the following state observer parameters:

ke1 = 8 ke2 = 33

Figure 9.6 shows the result of a simulation experiment with the state observer The initial state
is given by ϕ(0) = −0.6 and ϕ̇(0) = 0.4 Since the process is unstable, it has to be controlled. This
control is treated in the next lecture.
The solid curves in Figure 9.6 show the actual states. The angle is initially ϕ(0) = −0.6. After
an initial overshoot to ϕ ≈ 0.4 it stabilizes at ϕ = 0 after approximately 3 s.
The dashed curves show the estimates and are thus the ones of highest interest for us. They
both start out in zero. After approximately 1.5 s they have converged well to the actual states.
The example shows that the state observer can be used to estimate the state vector. One thus
realizes that it should be possible to use the estimated state for the

Fig. 3.4 Figure 9.6 Actual and estimated angle and angular speed of the pendulum in Example 9.2.

feedback, rather than the actual ones. It is, however, important that the Kalman filter is
sufficiently fast, since the transient behavior shown in Figure 9.6 will be repeated every time the
 49

process is disturbed, which would happen e.g. if someone pokes at the pendulum. The relationship
between the Kalman filter and the state feedback will be further investigated in the next lecture.
Finally it is worthwhile noting that the Kalman filter is not merely interesting in the context
of state feedback control. The method to estimate variables which are not directly measurable
by exploiting the process dynamics and the availability of certain measurement signals is used in
a large number of other applications. Examples include technical systems as well as biological,
medical and economical ones.

10.1 Output Feedback

We shall now merge the state feedback and the Kalman filter and close the loop from the estimated
states, rather than the actual ones. We call this output feedback in order to emphasize that we do
no longer close the loop from the states of the process. Rather, we use the setpoint r, measurement
y and control signal u. The controller structure is shown in Figure 10.1, where the controller is
confined within the dashed rectangle.
We choose to describe the process in state space and assume that it lacks a direct

Fig. 3.5 Figure 10.1 Output feedback. The controller is confined within the dashed rectangle. term.

ẋ = Ax + Bu
y = Cx
The controller now becomes a combination of the Kalman filter and the state feedback

x̂˙ = Ax̂ + Bu + Ke (y − ŷ)

ŷ = Cx̂
u = kr r − K x̂
We shall investigate the closed-loop system and begin by doing so in state space. A natural
way to choose the state vector would have been to append the Kalman filter estimates x̂ to the
process states x. For reasons, which will become evident later in the lecture, we will rather append
the process states x with the estimation error x̃ = x − x̂ in order to form the state vector.
50

!
x
xe =
x̃
The state-space equations can now be written

ẋ = Ax + Bu = Ax + Bkr r − BK x̂ = Ax + Bkr r − BK(x − x̃)

= (A − BK)x + BK x̃ + Bkr r
x̃˙ = ẋ − x̂˙ = Ax + Bu − Ax̂ − Bu − KeC(x − x̂) = (A − KeC)x̃
On block matrix form this becomes

! ! ! ! !
ẋ A − BK BK x Bkr x
= + r = Ae + Be r
x̃˙ 0 A − LC x̃ 0 x̃
! !
  x x
y= C 0 = Ce (10.1)
x̃ x̃

Thanks to the introduction of x and x̃ as system states, the Ae , Be and Ce matrices now contain
a number of zeros. We will benefit from this substantially now that we will investigate the
closed-loop system.
Due to the block triangularity of Ae , its characteristic polynomial is given by

det (sI − Ae ) = det(sI − (A − BK)) · det(sI − (A − KeC))

This is an appealing result. It shows that the characteristic polynomial is a product of the
characteristic polynomial from the nominal state feedback and that of the Kalman filter. Conse-
quently it is possible to separate the control problem into two parts, as we have already attempted
in the previous lectures. The state feedback can first be determined and its poles placed as if all
states were actually measurable. When later a Kalman filter is introduced in order to obtain state
estimates and realize output feedback, this will not affect the pole placement of the feedback.
Let us compute the transfer function from r to y. It is given by
!
  E
Ge (s) = Ce (sI − Ae )−1 Be = C 0
F
where
! !−1 !
E sI − (A − BK) −BK Bkr
=
F 0 sI − (A − KeC) 0
By multiplying both sides with (sI − Ae ) we get
 51

! ! !
Bkr sI − (A − BK) −BK E
=
0 0 sI − (A − KeC) F
!
(sI − (A − BK))E − BKF
=
(sI − (A − KeC))F
The latter part of the equation yields F = 0. Consequently, we arrive at

E = (sI − (A − BK))−1 Bkr

which gives us the transfer function

Ge (s) = C(sI − (A − BK))−1 Bkr

This is a remarkable result. The transfer function is identical to the one which we obtained
when we closed the loop from the real states. The dynamics of the Kalman filter are hidden in
the transfer function. The characteristic polynomial of the Ae matrix is of order 2n, whereas the
transfer function is merely of order n.
In the pendulum example from the previous lecture we saw how the estimated state converged
to the actual state after a few seconds long transient. Following this transient there is inherently no
difference between feedback from the estimated and actual states, as long as no disturbances forces
the state estimate away from the actual state. This explains why the dynamics of the Kalman filter
are not visible in the transfer function.
A transfer function of order n can always be described in state space using n states, which
are all both controllable and observable. One could also introduce more than n states, but the
additional states cannot be both observable and controllable. It is e.g. possible to introduce a state
which has no significant relation to the process. This state will obviously be neither controllable
nor observable.
Correspondingly a transfer function of degree n obtained from a state space model of degree
> n shows us that there are states which lack controllability or observability. In our case the
Kalman filter state is not controllable. This can be seen by constructing the controllability matrix
of the system in Equation (10.1).
!
  Bkr (A − BK)Bkr · · ·
Ws = Be Ae Be · · · An−1
e Be =
0 0 ···
Since the last n elements in the columns are zero, there are n linearly independent columns and
the non-controllable states correspond to those of the Kalman filter. The fact that the states of the
Kalman filter are not controllable can be seen directly from the state space Equation (10.1).

x̃˙ = (A − KeC)x̃

We can apparently not affect the estimates by means of the closed-loop system input, i.e. by
the setpoint r.
52

Summary
We have now introduced a method to determine a controller-state feedback combined with Kalman
filtering. The analysis has shown that the design process can be separated into two parts.
The first part was to estimate the state vector by means of the Kalman filter. The properties
of the Kalman filter are determined by the gain vector L. The choice of this vector, i.e. the pole
placement of the Kalman filter, is as always a balance between performance and robustness against
model errors, process variations and disturbances.
Let the system in Figure 1.4 be defined by

ẋ = Ax + Bu (8.125)
y = Cx (8.126)

Assume that the estimate x̂ of the state vector is

x̂˙ = Ax̂ + Bu + Ke (y − Cx̂) (8.127)

where Ke is the observer gain matrix.

If equation (8.127) is subtracted from (8.125), and ( x − x̂) is the error vector e, then

ė = (A − Ke C) e (8.128)

and, from equation (8.127), the equation for the full-order state observer is

x̂˙ = (A − Ke C) x̂ + Bu + Ke y (8.129)

Thus from equation (8.128) the dynamic behaviour of the error vector depends upon the
eigenvalues of (A − Kc C). As with any measurement system, these eigenvalues
should allow the observer transient response to be more rapid than the system itself (typically
a factor of 5), unless a filtering effect is required.
The problem of observer design is essentially the same as the regulator pole placement problem,
and similar techniques may be used.
(a) Direct comparison method: If the desired locations of the closed-loop poles (eigenvalues)
of the observer are

s = µ1 , s = µ2 , . . . , s = µn

then

|sI − A + Ke C| = (s − µ1 ) (s − µ2 ) . . . (s − µn )
= sn + αn−1 sn−1 + · · · + α1 s + α0 (8.130)
 53

Ackermann’s formula: As with regulator design, this is only applicable to systems where u(t)
and y(t) are scalar quantities. It may be used to calculate the observer gain matrix as follows
h iT
Ke = φ (A)N−1 0 0 ... 0 1

or alternatively
 −1  
C 0
   
 CA   0 
Ke = φ (A) 
 .. 


 .. 
 (8.134)
 .   . 
CAn−1 1
where φ (A) is defined in equation (8.104).

Example:
A system is described by
" # " #" # " #
ẋ1 0 1 x1 0
= + u
ẋ2 −2 −3 x2 1
" #
h i x
1
y= 1 0
x2
Design a full-order observer that has an undamped natural frequency of 10 rad/sand a damping
ratio of 0.5 .

Solution:
The obsevability matrix is
" #
  1 0
N = CT : AT CT = (8.135)
0 1
N is of rank 2 and therefore non-singular, hence the system is completely observable and the
calculation of an appropriate observer gain matrix Ke realizable.
Open-loop eigenvalues:

|sI − A| = s2 + 3s + 2 = s2 + a1 s + a0 (8.136)

Hence

a0 = 2, a1 = 3

And the open-loop eigenvalues are

54

s2 + 3s + 2 = 0
(s + 1)(s + 2) = 0
s = −1, s = −2 (8.137)

Desired closed-loop eigenvalues:

s2 + 2ζ ωn s + ωn2 = 0
s2 + 10s + 100 = s2 + α1 s + α0 = 0 (8.138)

Hence

α0 = 100, α1 = 10

and the desired closed-loop eigenvalues are the roots of equation (8.138)

µ1 = −5 + j8.66, µ2 = −5 − j8.66 (8.139)

Using Direct comparison method:

|I − A + Ke C| = s2 + α1 s + α0 (22)
" # " # " #
s 0 0 1 kel h i
− + 1 0 = s2 + 10s + 100
0 s −2 −3 ke2
" # " #
s −1 kel 0
+ = s2 + 10s + 100
2 s+3 ke2 0
s + kel −1
= s2 + 10s + 100
2 + ke2 s + 3
s2 + (3 + ke1 ) s + (3kel + 2 + ke2 ) = s2 + 10s + 100

From equation (8.140)

(3 + kel ) = 10, ke1 = 7 (8.141)

(3kel + 2 + ke2 ) = 100
ke2 = 100 − 2 − 21 = 77 (8.142)
 55

Using Arckermans formula:

" #−1 " #
C 0
Ke = φ (A)
CA 1
Using the definition of φ (A) in equation (8.104)

" #−1 " #

1 0 0
Kc = A2 + α1 A + α0 I (8.149)
0 1 1
"" # " # " ## " #" #
−2 −3 0 10 100 0 1 0 0
Ke = + +
6 7 −20 −30 0 100 0 1 1
" #" #" #
98 7 1 0 0
=
14 77 0 1 1
" #" # " #
98 7 0 7
= = (8.150)
14 77 1 77
Chapter 4

Optimal Control (DP, LQR and Kalman

Filters)

4.1 Dynamic Programming

• From IEEE History Center: Richard Bellman:

• "His invention of dynamic programming in 1953 was a major breakthrough in the theory of
multistage decision processes..."

• "A breakthrough which set the stage for the application of functional equation techniques in
a wide spectrum of fields..."

• “...extending far beyond the problem-areas which provided the initial mo-tivation for his
ideas."

Fig. 4.1 Aircraft Fuelling Problem

58 Optimal Control (DP, LQR and Kalman Filters)

- Example: travel from A to B with least cost (robot navigation or aircraft path)

- 20 possible options, trying all would be so tedious

- Strategy: start from B, and go backwards, invoking Proof of Optimum (PoO)

2. Discrete-Time Systems
The plant is described by| the general nonlinear discrete-time dynamical equation

xk+1 = f (k, xk , uk ) , k = i, . . . , N − 1

with initial condition xi given. The vector xk has n components and the vector uk has m components.
Suppose we associate with this plant the performance index

N−1
Ji (xi ) = φ (N, xN ) + ∑ L (k, xk , uk )
k=i

where [i, N] is the time interval of interest. We want to use Bellman’s principle of optimality to
find the sequence uk that minimizes the performance index.
Suppose we have computed the optimal cost

∗
Jk+1 (xk+1 )

for time k + 1 to the terminal time N for all possible states xk+1 , and that we have also found
the optimal control sequences from time k + 1 to N for all possible states xk+1 . The optimal cost
results when the optimal control sequences u∗ k+1 , u∗ k+2 , . . . , u∗ N−1 is applied to the plant with a
state of xk+1 . Note that the optimal control sequence depends on xk+1 . If we apply any arbitrary
control uk at time k and then use the known optimal control sequence from k + 1 on, the resulting
cost will be
∗
Jk (xk ) = L (k, xk , uk ) + Jk+1 (xk+1 )

where xk is the state at time k, and xk+1 is given by the state equation. According to Bellman, the
optimal cost from time k is equal to


Jk∗ (xk ) = min L (k, xk , uk ) + Jk+1
∗
(xk+1 )
uk

and the optimal control u∗k at time k is the uk that achieves this minimum.
3. Discrete LQR via Dynamic Programming
The plant is described by the linear discrete-time dynamical equation

xk+1 = Ak xk + Bk uk ,
4.1 Dynamic Programming 59

with initial condition xi given and final state xN free. We want to find the sequence uk on the
interval [i, N] that minimizes the performance index:

1 1 N−1

Ji = xNT SN xN + ∑ xkT Qk xk + uTk Rk uk , SN ≥ 0, Q ≥ 0, R > 0
2 2 k=i

Let k = N and write

1
JN = xNT SN xN = JN∗
2
Now let k = N − 1 and write

1 T 1 1
JN−1 = xN−1 QN−1 xN−1 + uTN−1 RN−1 uN−1 + xNT SN xN
2 2 2

According to Bellman’s principle of optimality,

Jk∗ (xk ) = min L (k, xk , uk ) + Jk+1
∗
(xk+1 )
uk

we find uN−1 by minimizing JN−1 , which can be rewritten as

1 T 1 1
JN−1 = xN−1 QN−1 xN−1 + uTN−1 RN−1 uN−1 + (AN−1 xN−1 + BN−1 uN−1 )T SN (AN−1 xN−1 + BN−1 uN−1 )
2 2 2

Since there is no input constraint, the minimum is found by setting

∂ JN−1
0= = RN−1 uN−1 + BTN−1 SN (AN−1 xN−1 + BN−1 uN−1 )
∂ uN−1

which gives

−1
uN−1 = − BTN−1 SN BN−1 + RN−1 BTN−1 SN AN−1 xN−1

Defining

−1
KN−1 = BTN−1 SN BN−1 + RN−1 BTN−1 SN AN−1

we can rewrite
u∗N−1 = −KN−1 xN−1

The optimal cost to go from k = N −1 is found substituting the optimal control in the expression
for JN−1 ,

∗ 1 T h T T
i
JN−1 = xN−1 (AN−1 − BN−1 KN−1 ) SN (AN−1 − BN−1 KN−1 ) + KN−1 RN−1 KN−1 + QN−1 xN−1
2

if we define

SN−1 = (AN−1 − BN−1 KN−1 )T SN (AN−1 − BN−1 KN−1 ) + KN−1

T
RN−1 KN−1 + QN−1

this can be written as

∗ 1 T
JN−1 = xN−1 SN−1 xN−1
2
60 Optimal Control (DP, LQR and Kalman Filters)

For k = N
1
JN = xNT SN xN
2
For k = N − 1

1 T 1 1
JN−1 = xN−1 QN−1 xN−1 + uTN−1 RN−1 uN−1 + xNT SN xN
2 2 2

For k = N − 2

1 T 1 1 T
JN−2 = xN−2 QN−2 xN−2 + uTN−2 RN−2 uN−2 + xN−1 SN−1 xN−1
2 2 2

The structure of the problem is the same. To obtain u∗N−2 We just need to replace N − 1 by
N − 2. If we continued to decrement k and apply the optimality principle, the result for each
k = N − 1, . . . , 1, 0 is

−1
uk = −Kk xk , Kk = BTk Sk+1 Bk + Rk BTk Sk+1 Ak
Sk = (Ak − Bk Kk )T Sk+1 (Ak − Bk Kk ) + KkT Rk Kk + Qk
1
Jk∗ = xkT Sk xk
2

Kalman Gain Sequence Joseph Stabilized Ricatti Difference Equation (RDE) Optimal Cost
 5
Algorithm:

Example:

Given a discrete time dynamic system

x(k + 1) = 4x(k) − 6u(k), x(0) = 8

and a performance index

∗
J0 = F (x(k), u(k)) + Jk+1 (x(k + 1))
1=N −1
1 X
= (x(2) − 20)2 + (2x2 (k) + 4u2 (k))
2 k=0

Using principle of optimality, find control sequence u(0) and u(1), Assume no constraints on
u(k)
 6

Solution:
7