Differential Equations, Discrete Systems and Control.

Economic Models

Aristide Halanay and Judita Samuel

Kluwer Academic Publishers, Dordrecht/Boston/London, 1997

Pag. 9-19, 31-34, 198-227

Chapter 1 1.4.
Pg. 9-19
Simplest Models of Price Evolution in a Market Economy

We consider a commodity having the price P (t ) at time t . We admit than the evolution of the
price is governed by the law of supply and demand, expressed in the form : the velocity of price

evolution is proportional to the difference between demand and supply. Denoting by D (t ) demand at

time t and by S (t) supply at time t , the law can be written as P (t )= λ [ D ( t )−S ( t ) ] .

Since the price increases when demand exceeds supply, it is natural to assume λ≻0 . At the same
time, demand, as well as supply depend on the price level and the simplest assumption is that of affine

dependence: D (t )=α+aP ( t ) , S ( t )= β+bP ( t ) .

Pg. 10

If we admit these assumptions, the following mathematical model of price evolution is obtained:

P' (t )= λ [ α +aP ( t )−β−bP ( t ) ] =λ ( α−β ) + λ ( a−b ) P ( t ) .

The price evolution is described by a linear affine differential equation.

To obtain explicit expressions, let us start from the last remark concerning the representation of the
solutions of affine equations and let us look for a simple particular solution.

The simplest such solution is a constant solution, expressing the equilibrium situation, when demand

^ α−β
P=
and supply are equal ( P'≡0 ) . Such a solution is given by b−a . It has an economic
significance only if
^
P≻0 (price have to be positive).

We remark that α is close to the level of demand when the price is small and β to the level of
supply when the price is small, so that the assumption α≻β seems to be natural (we could even
accept that β=0 ). The coefficients a and b show the way in which the price level influences
demand and supply;

When the price increases, the demand decreases, what means that it is natural to assume a≺0 ;
when the price increases, supply increases as well, so that we admit b≻0 . It follows that
b−a≻0 , hence P^ has an economic meaning.

^ p (t )
P (t )= P+ p is the general
Taking into account the general remark, we can write , where
solution of the linear equation

p' ( t )= λ ( a−b ) p ( t ) . We have p (t )=e λ ( a−b ) t c , hence

λ ( a−b ) t0
^ λ ( a−b ) t c
P (t )= P+e c=e [ P ( t ) − P^ ]
, .

^ λ ( a−b ) ( t−t 0) [ P ( t 0 ) − P
P+e ^]
Finally, .

lim e
λ ( a−b ) ( t −t 0)
=0 lim P ( t )= P^
Since a−b≺0 , it follows t →∞ , hence t →∞ , what expresses the
tendency of prices to move towards the equilibrium value.

Thus, the model adopted corresponds to a “stable” market.

The very simplified character of the model is manifest, among other facts, in that we considered an
isolated commodity, without taking into account the influence of all price in a market on the price of
given commodity.

The models of this type can be presedent in several variants. Thus, we can accept the idea that demad
and supply depend not only on the instantaneous level of prices, but also on their evolution tendency,
expressed by means of the derivative P' . In this way, we obtain a model of the form

P' (t )= λ [ α + a0 P ( t ) + a1 P' ( t )−β −b0 P ( t )−b1 P ' ( t ) ]

.

Which also leads to an affine equation:

[ 1+ λ ( b 1−a1 ) ] P ' ( t )=λ ( α −β )− λ ( b0 −a0 ) P ( t )

or

λ ( α −β ) λ ( b0 −a0 )
P' (t )= − P (t)
1+ λ ( b1 −a1 ) 1+ λ ( b1 −a1 )
.

We remark that the equilibrium value in thjis model is the same as in the previous one.
Pg. 12

1.5. Discrete –Time Models for Price Evolution

In this case, the general law of price evolution, on the basis of demand and supply, is written as

Pt−1−Pt = λ ( Dt −St )
.

If
Dt =α +aPt ,
S t =β +bPt , it follows

Pt +1 =Pt + λ ( α + aPt −β−bPt )=λ ( α−β ) + [ 1− λ ( b−a ) ] Pt

.

The equilibrium price corresponds, in this case as well, to the situation of equality between demand and
supply, hence

^ α−β
P=
^ +b P^
α+a P=β or a−b , the same as in continuous time case.

Denoting
pt =Pt − P^ , we obtain

^ p − P+
pt +1 =Pt +1− P= t
^ λ α + a( p + P
t [
^ ) −β−b ( p + P
1
^)= ]
pt + λ ( a−b ) p t =[ 1−λ ( b−a ) ] pt

We deduce
t −t 0 t −t0
pt =[ 1−λ ( b−a ) ] pt 0 . P=P+ [ 1− λ ( b−a ) ] (P t
^)
−P
hence 0

lim Pt = P^
We have t →∞ if −1≺1−λ ( b−a )≺1.

Accepting, as above , that b≻0 , a≺0 and λ≻0 , the second inequality is automatically
verified.

2
λ≺
The first inequality reduces to λ ( b−a )≺2 or b−a

We remark that a too strong influence of the difference between demand and supply on price increase,
as well as a too strong dependence of demand and of supply on preces can have destabilizing effects on
the price evolution.

The discrete-time variant allow for the description of several types of dependence of demand and
supply on the level of prices. Thus, we can admit that supply depends not on the instantaneous level of
prices, but on the previous one, while demand depends only on the present level. Under such
assumptions, even the equality condition of demand and supply will determine a dynamics of prices.

Indeed. If
D t =α +aPt and
S t =β +bPt−1 , then the equality
Dt =St will lead to
1
α+aPt =β +bPt−1 . Pt = ( β−α + bPt −1 )
,hence a

Essentially , we have an affine dependence of the same type as before. There exists a constant price

^ 1 ( β−α + b P^ )
P= ^ α−β
P=
satisfying the above relation, given by a , hence a−b , the same as for the
previous models.

Denoting again
pt =Pt − P^ , we have now

1 ^ bp
pt =
a[β−α+ b ( pt−1 + P^ ) − P=
a t−1]
t−t 0 t−t 0
b b
hence
pt = ()
a
pt 0
and
^
Pt = P+ ()
a ( P − P^ )
t0
.

lim Pt = P^
In this case, stability of equilibrium, expressed by the condition t →∞ , will be obtained if
b
| |≺1
a , hence if b≺|a|.
Pg. 14.

We stress again that the above discussion corresponds to the equality of supply and demand. If not, we

would obtain a connection between

Pt+1 ,
Pt and
Pt−1 . Such equations will be discussed later on.

1.6 Simplest Models for Economic Growth

Let Y (t) be the national product at time t allotted for consumption and investment, investment, in
its turn, has an autonomous component. A (t ) independent of the income (or national product ), and

another component. I (t ) , which is related to income by the relation It=vY ' ( t ) .

Thus, we have the fundamental relationship

I ( t )=C ( t ) +A ( t ) +I (t )
Accepting the simple relation C ( t )=cY ( t ) (meaning that a fixed ratio of income is allotted for
consumption), we obtain the affine differential equation

Y ( t )=cT ( t )+vY ' ( t )+ A ( t ) , i.e

u 1
Y ' ( t )= ( 1−v ) Y ( t )− A ( t )
v v .

We denote 1−c by s , where s has the meaning of “propensity to save”, the modfel takes the
from

s 1
Y ( t ) = Y ( t )− A ( t ) .
v v
This model is known as the Harrod-Domar model.

A
Y^ =
If A (t) is equal to a constant A , then we have an equilibrium level of the income s , and
the evolution is given by
s
A ( t−t 0 ) A
Y ( t )= +e v
s [
Y ( t 0 )−
s ]
Pg. 15

There exist several other models for the evolution of ioncome (national product ), leading to affine
differential equations. Thus we can define the “demand”

Z ( t )=C ( t )+ A (t )=cY ( t ) + A ( t )

and adopt the assumption that the velocity of evolution for the product Y ( t ) is proportional to the
deviation between its actual value and the “demand” (production is stimulated by demand).

Y ' ( t )= λ [ Z ( t )−Y ( t ) ] , λ≻0 .

If adopt again the assumption that C ( t )=cY ( t ) , then it follows

Y ' ( t )= λ [ cY ( t ) + A ( t )−Y ( t ) ] =−λ sY ( t ) + λA ( t ) .

This model is known as the Phillips model with multiplier. If A ( t ) =A is constant, we have again the
A
Y^ =
equilibrium s and a stable evolution towards the equilibrium, since λs≻0 .
1.7 Discrete-Time Models for Economic Growth

We write again the fundamental identy

Y t =C t +I t + At .

We admit now that the level of consumption represents a given ration of the pevious level of income,

hence
Ct =cY t−1 , and that investments at time t−1 induce the increase in income
1
Y t −Y t−1 = I t−1 .
v
Taking into accouting the relationship

Y t−1=cY t−1 +I t−1 + A t−1 , it follows Pag. 16.

I t−1 =sY t−1− A t−1 .

hence

1
Y t =Y t−1 + [ aY t−1− A t−1 ]
v
or

s 1
[ ]
Y t = 1+
v
Y t−1− A t−1
v

A
At= A Y^ =
If we assume that is constant, we have the equilibrium given by s , as in the previous
t−t 0
s
case, and the evolution is given by
Y t =Y^ + 1+
v [ ] (Y t0 −Y^ )
.

1.8 Production Functions

Production functions describe the connection between production factors (capital and labour) and the
level of production. If we denote capital by k and labour by L. and we consider a function
F : R+ ×R+ → R+ . F ( K ,L ) is interpreted as the corresponding level of production. If is natural
to admit the following properties:

F ( 0,0 ) =0 . F ( λK . λL ) =λF ( K . L ) .

The last property, representing homogeneity of the fisrt degree, expresses the fact that, if we multiply
capital and labour by the same factor, then the result is multiplied by the same factor.
For a given production function, we associate in a natural way the following average indicators.

Y F ( K . L)
y= =
L L , the average productivity of labour,

Y F ( K . L)
=
K K , the average efficiency of capital,

K
k=
L , the average capital-labour ratio.

We also associate the differential (marginal) indicators:

∂ F ( K . L)
η L ( K , L )=
∂K

We have

1 K
y= F ( K . L )
L
=F ( )
L
.1 =F ( k , 1 )=f ( k )
;

K K K K K K K
η L ( K . L )= ∂ Lf
∂L L [ ( )] ( ) ( )( ) ( ) ( ) ( )
=f
L
−Lf '
L L 2
=f
L
−
L
f ''
L
=f ( k )−kf ' ( k )
;

K K
η K ( K . L )= ∂ Lf
∂K L [ ( )] ( )
=f '
L
=f ' ( k )
.

We also consider the elasticities.

∂F K K K
E L ( K . L )=
∂L
( K . L) f
=
L ( )
− f'
L L
=1−
( )
kf ' ( k )
F (K . L) K f (k )
L
f
L ( ) .
 ∂F K
Ek ( K . L )=
∂K
( K . L)
=
f'
L ( )
=1−
kf ' ( k )
F ( K . L) K K f (k )
K
f
L
/
L ( )( ) ;

We define the rate of factor substitution as

∂F K K K
r ( K . L )=
∂L
( K . L) f
=
L
− f'
L L ( )
=ρ ( k )
( )
∂F K
∂K
( K . L) f'
L ( )
and the elasticity coefficient of substitution as

K
K
σ ( )=
( L)
=k
ρ'
p' (k)
L K K ρ (k )
ρ ( )/
L L

We can identify several types of production functions, starting from simple ssumptions concerting the
indicators defined above.

Thus, we can assume that the elasticities are constant. It can seen that for such functions we have

f ' (k )
k =1−α
f (k ) , where α is a constant, hence f is a solution of a linear differential equation

1−α
f ' ( k )= f (k )
k .

The representation formula of the solution, given in 1.2, leads to

k 1−α
∫t t
dt
f ( k )=e f ( 1) .

k 1−α
∫1 dt=( 1−α ) ln k=ln k 1−α ln k1 −∞
We have t , hence f ( k )=e f ( 1 )=h1−α f (1 )

and therefore

1−α
K K
F ( K . L )= Lf
L
=L
L( ) () F ( 1. 1 ) K 1−α Lα
.
We found, in this way, the Cobb-douglas type production functions, if 0≺α≺1 . If α=0 ,

Then the function is proportional to capital and independent of labor, and if α=1 , then F ( K .L )

Does not depend upon k and is proportional to L .

Let us look now for the production functions having constant elasticity of factor substitution.

If
σ ( KL )=α , then we deduce kρ ' ( k ) =αρ ( k ) , hence ρ is a solution of a linear differential
equation:

α
ρ ( k )= ρ ( 1 ) k . Further

f (k ) f (k )
ρ ( k )=−k + =k + ρ ( k )
f (k ) , f ' (k )

and f is a solution of the linear differential equation

Pg. 19

1 k dt
f ' ( k )= α
f ( k ) ∫1
t+ ρ ( 1 ) t α
k +ρ ( 1 ) k , hence f ( k )=f ( 1 ) e

dt dt
∫ t+βt α =∫ ( α −1 )
We can compute t 1+βt

α −2
We denote t α−1 =τ ; ( α−1 ) t dt=dτ

dt dτdτ 1 dτ βdτ
We denote t ( 1+ βt α−1
) ( α−1 ) t α −2
t (
=
1+βt α−1
) ( α−1 ) τt ( 1+ βτ )
=
( α −1
=
)
−
τ 1+βτ [ ]
Hence

1
dt t α−1
∫ t+βt α =ln 1+ βtα −1 ( ) α−1

1
kdt k α −1 ( 1+β )
∫1 t +βt α =ln 1+ βk α −1 ( ) α −1
What leads to

1
k α−1 ( 1+β ) 1+ β 1

(
f ( k )=f ( 1 )
1+βk α−1 ) α−1 =f ( 1 )
( β +k 1−α ) α−1

1
K 1+ β
() 1−α 1−α 1−α
F ( K .L )= Lf =f ( 1 ) L
L K
L
1−α
+β (( ) )
= A [ K +βL ]

.
1

Where A is a constant equal to f ( 1 ) ( 1+ β ) α−1 and β=ρ ( 1 ) .

There production function (Constant Elasticity of Substitution).

Pg. 31-34
Price Evolution in the Presence of Inventories

In the simple price evolution models discussed earlier, only two agents were present, one of them
representing demand (consumption) and the other – supply (production). In the models with inventories
a third agent occurs, which keep the commodity in stores (trade) and which essentially influences price
evolution.

If we denote by
Qt the level of inventories at the end time interval t , we have the natural

relationship t Q −Q =S −D
t−1 t t , expressing the fact that the increase of inventories during the time

interval t is due to the difference between supply and demand.

2.3.1 We adopt first as evolution law for prices the principle of proportionality between the variations of
and of inventories:

Pt −Pt−1=−λ ( Qt−1−Qt−2 ) λ≻0 .

,

Pg. 32

(when there is an excess of inventories, then prices tend to decrease).

If we accept again the relations

Dt =α +aPt ,
S t =β +bPt .

Then we have
 Qt−1−Qt−2 =S t−1−D t−1 =β +bPt−1 −α−aPt−1 =β−α + ( b−a ) Pt−1

and we obtain the model

Pt =[ 1−λ ( b−a ) ] Pt−1 +λ ( α− β ) .

This is a first order affine recurrence.

^ α−β
P=
^
The equilibrium price P corresponds to b−a , and price evolution is given by

^ [ 1−λ ( b−a ) ] t ( P − P^ )
Pt = P+ 0 .

We have b−a≻0 and Pt will converge to the equilibrium price if −1≺1−λ ( b−a )≺1 , hence
if λ ( b−a )≺2 .

If λ ( b−a )≻2 , then we have unbounded oscillation of price. This situation would occur for great
values of λ , expressing a strong and fast reaction of trade to inventory variation; such reaction
generates instability.

2.3.2. We consider now the case when there exists a “normal” level Q̄ of inventories and we admit
that price variation is proportional to the deviation of the inventories level from the normal one:

Pt −Pt−1=−λ ( Qt−1− Q̄ ) λ≻0 .

,

Writing the analogous relationship for time interval t−1 ;

Pg. 33

Pt−1−Pt−2 =−λ ( Qt−2 −Q̄ )

and subtracting, we get

Pt −2 Pt−1 +Pt−2=− λ ( Qt−1−Qt−2 )=−λ ( S t−1−D t−1 ) =−λ ( β−α ) −λ ( b−a ) P t−1
.

The model obtained in this way is described by a second order recurrence:

Pt −[ 2− λ ( b−a ) ] Pt−1 +Pt−2 =λ ( α −β ) .

 ^
P^ as above. If we denote pt =Pt − P
The equilibrium level of prices is given by the same value
then

we obtain

pt −[ 2− λ ( b−a ) ] pt−1 + p t−2 =0 .

2
The associated second degree equation (characteristic equation) is q −[ 2−λ ( b−a ) ] q+1=0 .

The discriminant of this equation is

2
[ 2−λ ( b−a ) ] −4=λ2 ( b−a )2−4 λ ( b−a )=λ ( b−a ) [ λ ( b−a )−4 ] .

If λ ( b−a )≻4 , then the roots are real and distinct, their product being 1 and their sum negative,
hence the roots are negative and one of them has modulus greater than 1 , what corresponds to an
unbounded oscillation of the price.

If λ ( b−a )≺4 , then the roots are not real, their product is equal to the square of the modulus,
hence the modulus is 1 and a bounded, oscillatory evolution around equilibrium price results.

We obtait second order recurrences for price evolution also in the case of the simple law of demand and
supply, studied in

Pg.34

1.5.

when the supply depends not on the current price, but on the previous one.

2.3.3

Let us discuss now the continuos time analogues of the models with inventories.

We consider the velocity of inventory evolution to be proportional to excess demand

Q' ( t )=−μ [ D ( t )−S ( t ) ] , μ≻0 .

In a first model, the velocity of price evolution will be assumed proportional to that of inventory
evolution:

P' (t )=−λQ' ( t )=λμ [ D ( t )−S ( t ) ] .

What reduce to the simple model based on supply and demand only, which was analyzed in 1.4.

For the model in which the “normal” level of inventories is present, we consider

P' (t )=−λ [ Q ( t )−Q̄ ] , wherefrom it follows P' (t )=−λQ' ( t )=λμ [ D ( t )−S ( t ) ] .

Taking into account the relations expressing the dependence of supply and of demand on prices, we
obtain the second order affine differential equation

P' (t ) +λμ ( b−a ) P ( t )= λμ ( α−β ) .

The equilibrium price P^ is the same as above; hence

p (t )=C 1 cos √ λ ( b−a ) t+C2 sin √ λ ( b−a ) t

Again, we have price oscillation around the equilibrium level.

CAPITOLUL III Pg. 198-227

Optimal Stabilization
7.1) Linear-Quadric Optimization on Infinite Horizon. Continuous Time

We have seen in the previous chapter that, given a linear control system, there is much freedom in the
choice of a feed –bach gain that stabilizes it.

This situation suggests to try to make an optimal choce. Such optimal choice assumes a specific
performance allowing to compare various controls.

To cost of such deviation is measured is measured in the simplest way by a quadric form with respect to
the state; in a similar way a simple measure of the control correction will be a quadric from with respect
to the control.

The simple reasoning allows to state the problem of optimal stabilizationas a linear-quadric optimization
problem on infinite horizon, which will be precisely stated below.

We consider the linear dynamisc

x '= Ax+Bu
and we look for a control u such that the evolution of the system be exponenţially stable and, moreover,
the quadric cost
∞
∫0 [ μ ( u,u ) +( Cx ,Cx ) ] dt
be minimized.

Theorem 7.1 We consider the system

¿ 1 ¿ ¿
A R+RA− RBB R+C C=0
μ (Algebric Matrix Riccati Ecuation).

We assume that this system admits a solution R≥0 such that the eigenvalues of the matrix

1 ¿
A− BB R
μ
have stricly negative real parts. Then the linear feed-back control

1 ¿
u=− B Rx
μ
is stabilizing and achieves the minimum cost.
¿
Proof We substitute into the expression of the cost the term C C , in accordance with the algebric
Riccati ecuation and get
 ∞ μ ( u ,u ) +1 B¿ Rx.B¿ Rx −
∞
[ 1 ¿ ¿
J (u )=∫0 μ (u ,u)+ ( RBB Rx,x )−( A Rx,x)−( RAx,x ) dt=∫0
μ ] [ ¿
μ
( ]
) ¿ ¿ ¿¿¿
¿
¿ lim x ( t )=0
We deduce that for any admisible u (for which the integral converges and t →∞ we have

1 ¿
ℑ ( u ) ≥( Rx0 , x 0 )
u=− B Rx .
and eequality is obtained if μ

Since, for such choice of u , the evolution is given by

1
(
x '= A− BB¿ R x
μ )
αt
and this evolution admite the evoluation
|x ( t )|≤ βe |x 0|. we deduce that u is an admissible
control and hence it achieves optimal stabilization; we have used here the assumption that the
1 ¿
A− BB R
eigenvalues of matrix μ have strictly negative real parts.

Remark 7.2 We did not use in the proof the fact that R≥0 în fact, this property follows from the
1 ¿
A− BB R .
condition imposed on the matrix μ Inded, the algebric Riccati ecuation can be rewritten
as

1 1 1
( )(
R A− BB¿ R + A ¿ − RBB¿ R+ RBB ¿ R+ C¿ C=0
μ μ μ )
Herefrom we deduce

7.1. Linear-Qudratic Optimization an Infinite Horizon

 (e A −1μ RBB ) t R
¿ ¿

1
1
( A − BB R ) ( A −μ RBB )t ¿ 1 ¿
( A − BB R )t
¿ 1 ¿ 1 ¿

( A− BB¿ R e μ
μ )
+e A − RBB ¿ Re μ
μ
+ ( )
1 1
( A − RBB ) t
¿

1 (
¿ ¿
A − BB R t ) =0
+e
μ
RBB R+C C e [ μ
¿ ¿
] μ

What can be wrtten as

dt
[
e Re +e
1
d ( A − μ RBB )t ( A − μ BB ' R ) t ( A − μ RBB )t 1
¿ ¿

μ
¿ ¿
]
RBB R+C C e
1
( A − μ BB R )t =0
¿ ¿

[ ]
1 ¿

We integrate from 0 to ∞ , what is allowed, due to the evaluation

(|e A− 1μ RBB )t|≤βe ¿

−αt

end deduce

(e A − 1μ RBB ) t
¿ ¿

1 (A − BB R)t 1 ¿

R=
∫0
∞

[ μ
RBB ¿ R+C ¿ C e μ ] dt .

Whrefrom it follows R≥0

We shall prove now a stability theorem.

Theorem 7.3 We consider the Liapunov equation

AP+PA ¿ +BB¿=−Q . Q≥0 .

We shall prove now a stability theorem.

We assume that there exists F such that the eingenvalues of A + BF have strictly negative real perts
and that the equation admits a solution P≥0 .

Pg. 202

Then the eigenvaluies of matrix A have strictly negative real parts.

¿
Proof. Let x be an arbitrary solution of the system x'= A x

We have
 d
( Px ( t ) , x ( t ) )=( PA¿ x ( t ) , x ( t ) ) + ( Px ( t ) , Ax ( t ) )=( ( AP+PA ¿ ) x ( t ) , x ( t ) ) ≤−( BB¿ x ( t ) , x ( t ) ) =−|B¿ x ( t )|2
dt
Integrating from 0 to t , we obtain

t
( Px ( t ) , x ( t ) )− ( Px ( 0 ) , x ( 0 ) ) +∫0 |B¿ x ( τ )|2 dτ≤0.
hence

t ∞
∫0 |B¿ x ( t )|2 dt≤( Px ( 0 ) , x ( 0 ) )≤~μ|x ( 0 )|2 and, for t →∞ we deduce ∫0 |B¿ x ( t )|2 dt≤~μ|x ( 0)|2 .
By assumption, there exists F such that the eigenvalues of A + BF have strictly negative real parts;
¿ ¿ ¿
These eigenvalues coincide with these of the matrix A +F +B .

We can write

x ' ( t )= A¿ x ( t )=( A ¿ +F ¿ B¿ ) x ( t )−F ¿ B ¿ x ( t )

hence
¿ ¿ ¿ t
x ( t )=e A +F B t x ( 0 )−∫0 e( A + F B ) (t −τ ) F B x ( τ ) dτ
¿ ¿ ¿
( ) ¿ ¿

( A ¿+F ¿ B¿ ) −αt −αt t −α ( t−τ ) ¿

|e |≤βe |x ( t )|≤ βe |x ( 0 )|+∫0 βe |F||B x ( τ )|dτ
, we deduce Pg.203
Since

Herefrom is follows

t 2
[ (
|x ( t )|2 ≤2 β 2 e−2 αt|x ( 0 )|2 + β^ 2 ∫0 e−α (t−τ )|B¿ x ( τ )|dτ )]
Using the Schwarz-Cauchy inequality, we deduce

t 1
[
|x ( t )|2 ≤2 β 2 e−2 αt|x ( 0 )|2 + β^ 2 ∫0 e−α ( t−τ ) dτ ∫0 e−α (t−τ )|B¿ x ( τ )|2 dτ = ]
2 −2 αt 2 B^ 2 t −α ( t−τ ) ¿
2
¿2β e |x ( 0 )| + ∫ e |B x ( τ )|2 dτ
α 0

Herefrom, integration from 0 to ∞ , we obtain

 2
2 β^ 2 ∞
∫0 |x ( t )|2 dt≤αβ
t
∞
|x ( 0 )|2 +
α
∫0 [∫ e
0
−α ( t−τ )
]
|B ¿ x ( τ )|2 dτ dt=

β2 2 2B ^2 ∞ ∞ ¿ 2
¿ |x ( 0 )| + 2 ∫0 (∫0 e−αt dt )|B x ( τ )| dτ=
α α
2
β 2 β^ 2 ∞ ¿
|x ( 0 )|2 + 2 ∫0 |B x ( τ )|2 dτ .
α α
∞

Using the estimte

∫0 |B¿ x ( τ )|2 dτ≤~μ|x ( 0)|2 , we deduce

β 2 2 β^ 2 ~
∞ 2
∫0 |x ( t )| dt≤ ( α α
+
2
) 2
μ | x ( 0 )| = μ^ |x ( 0 )| .

Since x ( t )=e A t x ( 0 ) , we can write

∞ ¿

∫0 ( e A t x ( 0 ) ,e A t x ( 0 )) dt≤^μ|x ( 0)|2 .
¿

Pg. 204

It is important to remark that this inequality holds for any x (0 ).

∞ ¿ ¿
Aτ
V ( x ) =∫0 (e A τ x ,e )dτ .
Denote
~
For an arbitrary solution x of the system, we denote V ( t )=V ( x ( t ) ) . We have

~ ∞ ¿ ∞
V ( t )=∫0 ( e A τ e A t x ( 0 ) ,e A ( t +τ ) .x ( 0 ) ) dτ =∫0 ( e A (t+τ ) x ( 0 ) ,e A (t+τ ) x ( 0 ) dτ )=
¿ ¿ ¿ ¿

∞ ¿ ∞
∫t ( e A s x ( 0 ) ,e A s x ( 0 ) ) ds=∫t |x ( s)|2 ds.
¿

~
It follows V ' ( t )=−|x ( t )|2 .
1~
2 −|x (t )|2 ≤− V ( t ) .
Since V ( x ( t ) )≤ μ^ |x ( t )| , we deduce μ^

Hence

~ 1~ ~ 1~
V ' ( t )≤− V ( t ) V ' ( t )+ V ( t )≤0 .
μ^ or μ^
Hence

1
1
t
μ^ ~ 1
1
t
μ^ ~
e V ' ( t )+ e V ( t ) ≤0 ,
μ^ dt
[
d μ^ t ~
e V ( t ) ≤0. ]
1 1 1
− t − t − t
~ ~ 2
V ( t )≤e μ^ V ( 0 )=e μ^ V ( x ( 0 ) )≤ μ^ e ^μ |x ( 0 )| .

On the other hand, it can be checked that

∞ ¿ δ
V ( x ) =∫0 |e A τ x| dτ≥∫0 |e x| dτ ≥v|x|
2 A¿ τ 2 2
;

The last inequality uses the fact that δ can be chosen small enough. It follows that
1
μ^ − t
|x ( t )| ≤ e μ^ |x ( 0 )|2 .
2
v^
Pg. 205
¿
This behaviour of the solutions implies that the eigenvalues of the matrix A have strictly negative
real parts; obviously, the same conclusion holds for the eigenvalues of matrix A.

We shall use the above theorem in order to define an iterative procedure for obtaining the solution of
the algebic Riccati equation.

Theorem 7.4

If there exist F and H such that the eigenvalues of A + BF and of A +HC have strictly
negative real parts, then the equation

¿ 1 ¿ ¿
A R+RA− RBB R+C C=0
μ
1 ¿
A− BB R
admits a solution R such that the eigenvalues of μ

had strictly negative real parts.

We denote
W 1 =−F and write the Liaponov equation

¿ ¿ ¿
( A−BW 1 ) R1 + R1 ( A−BW 1 ) +C C+μW 1 W =0
Since the eingevalues of
A−BW 1 have strictly negative real parts, using the reasoning of Remark 7.2

we deduce that
R1 ≥0 .

1 ¿
W 1= B R
We choose now μ , and chech the fact that the eigenvalues of
A−BW 2 have strictly
negative real parts.

1 ¿
A−BW 2 =A− BB R1 ,
We have μ hence

Pagina 206

1 ¿ 1 ¿ ¿ 2
( )( )
R1 A− BB R1 + A− BB R1 R1 =R1 A + A¿ R1 − R1 BB¿ R1 +
μ μ μ
¿ 2
+R1 ( A−BW 1 ) + ( A−BW 1 ) R1 +R1 BW 1 +W ¿1 B ¿ R1 − R 1 BB¿ R1 =
μ
2 1 1 1
μ μ ( μ μ )(
¿−C¿ C−W ¿1 W 1 +R1 BW 1 +W ¿1 B¿ R 1− R1 BB¿ R1 =−C ¿ C−μ W ¿1 − B 1 B W 1− B¿ B1 − R1 BB¿ R1 . )
We deduce

R1 ( A−BW 2 ) + ( A−BW 2 )¿ R1 + C¿ C+ μ ( W ¿1 −W ¿2 ) ( W 1 −W 2 ) + μW ¿2 W 2 =0 .

Since the eigenvalues of

A−BW 2 −B ( W 1−W 2 ) = A−BW 1

have strictly negative real parts, we can apply Theorem 7.3 to conclude that the eigenvalues of
A−BW 2 have strictly negative real parts. Wecan now continue the procedure, by choosing R2 as a
solution of the Liapunov equation

R2 ( A−BW 2 ) + ( A−BW 2 )¿ R2 + C¿ C + μW ¿2 W 2 =0 .

1 ¿
W 3= B B 2 A−BW 3
As above, we have
R ≥0 and taking
2 μ , we get having eigenvalue with
strictly negative real parts.

Taking into account the relation

R1 ( A−BW 2 ) + ( A−BW 2 )¿ R1 + C¿ C+ μW ¿2 + μ ( W ¿1−W ¿2 ) ( W 1−W 2 ) =0

It follows
 ¿ ¿ ¿
( R1−R2 )( A−BW 2) + ( A−BW 2 ) ( R1 −R2 ) + μ ( W 1 −W 2 ) ( W 1 −W 2 )=0 .

Since
A−BW 2 has eigenvalues with strictly negative real parts, we deduce R1 ≥R2 (using the

representation formula given in Remark 7.2).

The procedure continues and we obtain a sequence

Rk ≥0 and a sequence
Wk such that

1 ¿
W k+1= B R k 0≤R k+1≤R k.
μ and

This inequality means that, for any x , we have

0≤( Rk +1 x , x ) ≤( Rk x , x ) ;

Therefore, the sequence ( Rk x , x ) is decreasing and bounded from below, hence it has a limit.

Hereform it follows that, for any x and y , the sequence ( Rk x , y ) has a limit, indeed, we have

( R k ( x+ y ) , x+ y )=( R k x , x ) +2 ( R k x , x ) + ( R k y , y )
and we deduce

1
lim ( R k x , y ) = lim ( R ( x + y ) , x+ y ) −lim ( Rk x , x ) −lim ( R k y , y ) .
k →∞ [
2 k →∞ k k→ ∞ k →∞ ]
By a convenient choice of the vectors x and y , we obtain the convergence of the entries of the

matrices
Rk to the entries of a matrix R≥0.

1 ¿
Wk W= B R .
Herefrom we deduce that the sequence has also a limit μ Taking the in the equation

satisfied by
Rk , we get

Pg. 208

1 ¿ 1 ¿ ¿ 1
( )(
R A− BB R + A− BB R R+C¿ C + RBB¿ R=0 .
μ μ μ )
What shows that R is a solution of the Riccati eqation. We still have to verify that the eigenvalues of
1 ¿
A− BB R
μ have strictly negative real parts.
¿
We proceed as in the proof of Theorem 7.3 and we deduce that if x'=( A−BW ) x , then
∞
∫0 |Cx (t )| dt≤2 ~μ|x ( 0 )|2 , ∫∞ |W ¿ x ( t )|2 dt≤~μ|x ( 0 )|2
0

Further, we can write

x' ( t )=( A+HC ) x ( t )−BW ¿ x ( t )−HCx ( t )

and werepeat the reasoning of Theorem 7.3 to conclude that the eigenvalues of

¿ 1 ¿
A−BW = A− BB R
μ have strictly negative real parts.

This ends the proof of Theorem 7.4.

Theorem 7.1. and theorem 7.4 solve completely the linear-quadratic optimization problem of infinite
horizon.

Remark 7.5 The solution R obtained by the above iterative procedure is the maximal solution of the
Riccati equation.
~
Indeed, if R is a solution of the equation, then we can write

~ 1 ¿ 1 ¿ ¿~ 1 ~ ¿ 1 1
R A− BB R + A− BB R R+ R BB R+ RBB ¿ ~
( )( R− ~R BB¿ ~ )
R+C¿ C=0 .
μ μ μ μ μ
Pg. 209

~ 1 ¿ 1 ¿ ¿~ 1 ~ 1
R A− BB R + A− BB R R− ( R−R ) BB¿ (~
( )( )
R−R ) +C¿ C + RBB ¿ R=0
μ μ μ μ
Herefrom

1 ¿ 1 ¿ ¿ 1
( R− R ) A− BB R + A− BB R ( R−~
~
( R ) + (~
)( R−R ) BB¿ (~
R−R )=0. )
μ μ μ
 1 ¿
A− BB R ~
Since the eigenvalues of μ have strictly negative real parts, it follows that R− R≥0
~
and therefore R≥ R and R is maximal. From this maximality we deduce the uniqueness of the
stabilizing solution of the Algebric Matrix Riccati Equation.

Remark 7.6 The Liapunov equation, which have to be solved at each step, are linear equations, that can
be solved by usual procedures. Tothis end, there exist efficient computer cosed based on reducing
matrices to triangular from by orthogonal transformations.

7.2 Application to a Price Model

We can analyze the model of price evolution, considered in 1.4.i.e. p'=λ [ ( a−α ) p+b−β ] .

by assuming that the term b−β can be controlled by a decision-marker, by complementary demand
or complementary supply, respectively.

Pg. 210

Therefore, we consider the scalar control system, described by p'=λ [ ( a−α ) p−u ] .

For a given value u^ of the control, we have the equilibrium

u^
^p=−
a−α , a−α≺0.

By introducing new variables, representing deviations from equilibrium: x= p− p^ , v =u−u^ , we

obtain

x '=λ [ ( a−α ) x +v ] .

By associating to this control system a cost of the form

∞
J ( v )=∫0 ( q2 x 2 =μv 2 ) dt ,
We obtain an optimal stabilization problem.

The corresponding algebraic Riccati equation is

2
λ 2
r −2 λ ( a−α ) r −q 2=0.
μ
When q=0 , hence when there is no penalty for the deviation from equilibrium, then the optimal
control is obviously zero and corresponds to the root r=0.

If q≻0 , then the Riccati equation has two real roots,

r 1 ≻0 and r 2 ≺0

q2 μ
( sin ce r 1 r 2 =−
λ
≺0 . )
~v=− λ r x.
The optimal control coresponds to the positive root: μ 1

Where

r 1=
√ μ2 ( a−α )2 +q 2 μ−μ ( a−α )
λ
2
The optimal cost is r 1 x 0 .

Pg. 211

7.3 Optimal Stabilization in Discrete Time

We consider a linear problem with quadratic cost function, in discrete time, on infinite horizon.

The evolution of the process is described by

x t+1= Ax t +But

and the cost is

∞
J (u )=∑ [ ( Qxt , xt ) + ( Rut , ut ) ]
t=0 , with R≻0 , Q≥0 .

u x
Acontrol t is admissible if the corresponding solution t converges to zero and series is
convergent; among all these admisible controls we look for one for which the cost is minimized.

Proposition 7.7 Letus assume that the nonlinear system

R+ B¿ PB=V 2
¿
A PB=W V
¿
P= A ¿ PA+Q−W ¿ W
Admis a solution with P≥0 , such that the eigenvalues of the matrix A + BF , with
−1
F=−V W , are located inside the unit circle (with modulus strictly smaller than 1). Then the

control
ut =Fx t is admissible and provides the minimum cost.

Proof The fact that such a control is admissible follows directly: if

ut =Fx t , then the evolution is

ut =( A+BF ) x t . hence x t =( A +BF ) ' x 0 and if the eigenvalues of matrix A + BF

Pg 212

have absolute values smaller that I then there exist 0≺q≺1 . β≻1 such that
|x t|≤βq'|x 0|
∞ ∞
2t 2 μ|x 0|2
|ut|≤ β^ q t|x 0| and ∑ [ ( Qxt , x t )+( Ru t , ut ) ] ≤∑ μq |x 0| = 2
.
for all t≥0 . hence t=0 t=0 1−q

Let us consider now an arbitrary admissible aontrol and transform the expression of the cost, using the

solution ( P ,V ,W ) of the system. We have

∞ ∞
∑ [ (Qxt ,xt )+( Ru t ,ut )] =∑ [ (( P+W ¿W −A¿ PA ) xt ,xt )+( (V 2−B¿ PB )ut ,ut ) ]=
t=0 t=0
∞ ∞ ¿
∑ [ ( Pxt , xt )+( Wxt ,Wxt )−( PAxt , Axt )+( Vut ,Vut)−( PBut ,But )]=∑ [( Px t ,xt )+(Wxt ,Wxt )− ¿] ( P ( xt +1−But ) ,xt+1−But) + ¿ ¿ ¿¿
[ ]
t=0 t=0 ¿
¿
We deduce
 ∞
J (u )=∑ [ ( Wx t ,Wx t ) + ( Vu t , Vut ) + ( Pxt , xt ) −( Px t+1 , xt +1 )−2 ( PBut , But ) + ( Px t+1 Bu t ) + ( PBut , xt +1 ) ] =
t=0
∞
( Px 0 , x 0 ) + ∑ [ ( Wxt , Wxt ) +( Vut ,Vut ) −2 ( PBut , But ) +( P ( Ax t +But ) , But )+ ( PBut , Ax t +But ) ]=
t=0
∞
¿ ¿
( Px 0 , x 0 ) + ∑ [ ( Wxt , Wxt ) +( Vut ,Vut ) + ( B PAxt , ut ) +( A PBut , x t ) ]=
t=0
∞
¿ ( Px 0 , x 0 ) + ∑ [ ( Wx t , Wx t ) + ( Vut ,Vut ) + ( W ¿ Vut , x t ) + ( VWx t , ut ) ]=
t=0
∞
( Px 0 , x 0 ) + ∑ ( Wx t +Vut , Wxt +Vut ) .
t=0
J (u )≥( Px 0 , x 0 ) ,
It follows that for any dmissible control we have equality being achieved for

ut =−V −1 Wx t =Fxt

The question occurs now in a natural way, in what circumstances does a solution of the nonlinear
system exist and how can we compute it?

We shall show that, if there exists matrices 0 F ,H

0 such that 0 , 0 A +BF
have
A +H C
eigenvalues with modulus strictly smaller than 1, then there exists a solution (P,V,W) of the system

R+ B¿ PB=V 2
¿ ¿ ¿ ¿ ¿
A PB=W V P= A PA+C C−W W

Such than P≥0 , A + BF has eigenvaues strictly smaller than 1 in modulus where
F=−V −1 W .

Pg. 214

The solution is obtained as the limit of a sequence defined recurrently by the relations:

A k = A+BF k .
Pk +1= A ¿k Pk +1 Ak +C¿ C +F ¿k RF k . V 2k +1 =R +B ¿ P k +1 B .

W ¿k+1 V k+1=A ¿ Pk+1 B , F k+1 =−V −1

k+1 W k +1 ,

The eigenvalues of the matrix

A +BFk being strictly smaller than 1 in modulus.

We strat with
F0 such that
A 0 = A+BF 0 has eigenvalues strictly smaller than 1 in modules, then

we define
P1 , V 1 , W 1 , F1 and the process continues. We have to check that we can define
Pk+1≥0 if
 Ak has eigenvalues with modulus strictly smaller than 1 and that, if we defined
Pk+1 , hence also
F k+1 , then the eigenvalues of
A k+1 have modulus strictly smaller than 1.

Finally, we have to prove that the sequences

Pk , V k , W k are convergent and that ther limits
provide the solution of the system.

We bigin by proving some propositions interesting by themselves.

Proposition 7.8 Let A be a matrix such that its eigenvalues are located inside the unit circle. Them
¿
the equation P= A PA+Q has a solution for any Q and, if Q≥0 , then the solution P
satisfied P≥0 .

Pg. 215

Proff Since the eigenvalues of matrix A are strictly smaller than 1 in modulus, the serics
∞
∑ ( A ¿)k QA k
k=0 is convergent.

We show that this serics given the solution of the equation.

∞
k+1
P= ∑ ( A ) QA k
¿

Denoting k =0 , we have indeed

∞ ∞
¿ k+1
QA =∑ QA k =P−Q
k +1
¿
A PA=∑ ( A )
k=0 k=1

what shows that the serics provides a solution of the equations.

Summing up from k=0 to ∞ (the convergence being assured by the assumption concerning the
matrix A), it follows
∞ ∞ ∞
¿k k ¿ k+1 k+1 k
+ ∑ ( A ) QA k .
¿
∑ ( A ) PA = ∑ ( A ) PA
k=0 k=0 k=0

Hence
∞ ∞ ∞
¿ k ¿k k
P+ ∑ ( A ) PA = ∑ ( A ) PA + ∑ ( A ) QA k
k k
¿

k =1 k =1 k =0
 ∞
k
P= ∑ ( A ) QA k .
¿

what shows that k=0

The conclusion of the proposition follows automatically; moreover, it can be seen that the solution of
the equations is unique.

Pg. 216.

Propositions 7.9
¿ ¿
Let us consider the equations P= A PA+C C .

If the equations has a solution P≥0 and if there exists H 0 such that the eigenvalues of
A +H 0 C have modulus strictly smaller than 1, then the eigenvalues of A have modulus strictly
smaller than 1.

Proof Let
x0 be an arbitrary vector, we define the sequence

x t+1= Ax t , hance x t =A t x 0 .

V t =( Px t , x t )
We define ; we have

V t+1 =( Pxt +1 , x t +1 )=( PAx t , Axt ) =( A¿ PAx t , xt ) =( Px t , x t ) −( C ¿ Cxt , x t ) =V t −|Cxt|2 .

Summing up from 0 to T−1 , we deduce

T−1
V T −V 0=− ∑ |Cx t|2
t=0 ;
T−1
∑ |Cxt|2=V 0−V T ≤V 0 .
t=0

∞ ∞
2
∑|Cxt| ∑|Cxt|2≤μ|x 0|2 .
hence t=0 converges and, moreover, t=0

Further, we can write

x t+1= Ax t =( A+ H 0 C ) x t −H 0 Cx 0 .

We deduce
 x 1=( A +H 0 C ) x 0 −H 0 Cx 0 .

x 2=( A +H 0 C ) x1 −H 0 Cx 1=( A +H 0 C )2 x 0 −( A+H 0 C ) H 0 Cx 0 −H 0 Cx 1 .

Pg. 217

We chek by induction that

t−1
x t =( A+H 0 C ) ' x 0 −∑ ( A +H 0 C )t− j−1 H 0 Cx j .
j=0

The formula is valid for t=1 , t=2 ; then we have

t −1

t−1
[
x t +1=( A +H 0 C ) xt −H 0 Cxt =( A +H 0 C ) ( A+H 0 C ) t x 0 −∑ ( A +H 0 C )t− j−1 H 0 Cx j −H 0 Cxt =
j=0
t
]
( A+H 0 t ) t+1 x 0 − ∑ ( A+H 0 C ) t− j H 0 Cx j −H 0 Cxt =( A +H 0 C )t+1 −∑ ( A +H 0 C )t− j H 0 C j
j=0 j=0

What shows that, if the formula is valid for

xt , then it remains valid for
x t+1

Herefrom it follows that

t−1
|x t|≤βq '|x 0|+ β^ ∑ q t− j−1|Cx j|.
j=0

t−1 2 t−1 t−1

|x t| ≤2 β q |x 0| +2 β^ 2
2 2 2t 2

t−1
(∑
j=0
t− j−1
q
)
|Cx j| ≤2 β q |x 0| +2 β^ 2
2 2t 2
[ ∑q
j=1
t− j−1
∑ qt− j−1|Cx j|2 ≤
j=0
]
2 2t 2 β^ 2
2
¿ 2 β q |x 0| +
1−q
∑ qt− j−1|Cx j|2 .
j=0

Further, we obtain

∞
2 β^ 2 ∞ ∞ t−1 2 β 2|x 0|2 2 B^ 2 ∞

t=0
2 2

t=0
2
∑|xt| ≤2 β |x 0| ∑ q +1−q 2t
∑∑q
t=0 j=0
t− j−1 2
|Cx j| =
1−q 2
+
1−q ∑ ∑ (
j=0 t= j+1
)
q t− j−1 |Cx j|2 =

2 β2|x0|2 2 β^ 2 ∞

2
+ 2∑
|Cx j|2 ≤^μ|x0|2 .
1−q ( 1−q ) j=1
The convergence of the series implies that
 lim |x t|2=0 lim A t =0 .
t →∞ , t→

what is possible only if the eigenvalues of matrix A have modulus strictly smaller than 1.

We remark that the same proof works if

¿ ¿
P= A PA+C C +Q , with Q≥0 .

From Proposition 7.8 it follows that, given

Ak with eigenvalues strictly smaller than 1 in modulus and
Fk , we can find
Pk+1 in unique way and
Pk+1≥0.

Let us assume now that we defined

Pk+1 , V k+1 , W k+1 , F k+1 ; we shall prove that the eigenvalues

of matrix
A k+1 =A +BFk +1 are strictly inside the unit circle. To this end, we write the relation for
Pk+1 , using
A k+1 . We have
¿ ¿ ¿ ¿ ¿
Pk +1=( A +F k B ) P k +1 ( A +BF k ) +C C + F k RF k =
[ ¿ ¿ ¿ ¿ ¿ ¿
]
¿ A + F k +1 B + ( F k −F k +1 ) B Pk +1 [ A+ BF k +1 +B ( F k−F k +1 ) ] +C C + F k RF k =
¿ ¿

¿ ¿ ¿ ¿ ¿
¿ ( A + F k +1 B ) Pk +1 ( A +BF k +1 ) + ( A +F k +1 ) Pk +1 B ( F k −F k +1 ) +
¿ ¿ ¿ ¿ ¿ ¿ ¿
( F k −Fk +1 ) B P k +1 ( A +BF k +1 ) + ( F k−F k +1 ) B Pk +1 B ( F k−F k +1 ) + C C + Fk RF k =
A k +1 Pk +1 A k +1 +W k +1 V k +1 ( F k−F k +1 ) +F k +1 (V 2k +1 −R ) ( F k−F k +1 ) + ( F k−F k +1 ) V k W k +1 +
¿ ¿ ¿ ¿ ¿

+ ( F k −Fk +1 ) ( V 2k +1 −R ) F k +1 +C C + F k RFk + ( F k −Fk +1 ) ( V 2k +1 −R ) ( F k −F k +1 )=

¿ ¿ ¿ ¿ ¿ ¿

¿ A k +1 Pk +1 A k +1 +W k +1 V k +1 ( F k−F k +1 ) + F k +1 (V 2k +1 −R ) ( Fk −F k +1 ) + ( Fk −F k +1 ) V k W k +1 +
¿ ¿ ¿ ¿ ¿

+ ( F k −Fk +1 ) ( V 2k +1 −R ) F k +1 +C C + F k RFk + ( F k −FF k +1 ) ( V 2k +1 −R ) ( F k −F k+1 )=

¿ ¿ ¿ ¿ ¿ ¿

¿ A ¿k +1 Pk +1 A k +1 −F¿k +1 V 2k +1 ( F k −F k +1 ) + F ¿k +1 ( V 2k +1 −R ) F k −
−F k +1 ( V 2k +1−R ) F k+1 −( F k −F k +1 ) V 2k +1 Fk +1 + F k ( V 2k +1 −R ) F k +1 −F k +1 ( V 2k +1 −R ) F k +1 +
¿ ¿ ¿ ¿ ¿

+ F¿k ( V 2k +1 −R ) F k −F¿k +1 ( V 2k +1−R ) F k −F ¿k (V 2k +1 −R ) F k +1 +F ¿k +1 (V 2k +1 −R )+ C¿ C +F¿k RF k =

¿ A k +1 Pk +1 A k +1 −Fk +1 V 2k +1 F k + F k +1 V 2k +1 Fk +1 −F k V k +1 F k+1 + F 2k +1 V 2k +1 F k +1 +F k +1 V 2k +1 F k +1 +
¿ ¿ ¿ ¿ ¿ ¿

+ F k +1 RF k +1 +F k V 2k +1 Fk +1 −F k +1 RF k +C C+ F k RF k =A k +1 Pk +1 A k +1 + ( F k −F k +1 ) V 2k +1 ( F k −F k +1 ) +
¿ ¿ ¿ ¿ ¿ ¿ ¿ ¿

+ A k +1 Pk +1 A k +1 + ( Fk −F k +1 ) V 2k +1 ( F k −F k +1 ) +C C+ F k +1 RFk +1 .
¿ ¿ ¿ ¿ ¿

Thus we got
¿ ¿ ¿ 2 ¿ ¿
Pk +1= A k +1 Pk +1 A k +1 + ( F k −F k +1 ) V k +1 ( F k −F k +1 ) +C C +F k +1 RF k +1 .
Pg. 220

We also have
−1
A k +1 =A + BF k +1 = A+ BV k +1 V k +1 ( F k +1 −F k ) + BF k .

Hence
−1
A k +1 −BV k +1 V k +1 ( F k +1 −F k ) =A k .

Since the eigenvalues of matrix

Ak have modulus strictly smaller than 1, we deduce that the pair
A k+1 ,V k +1 ( F k +1−F k )
satisfied the condition staded in Proposition 7.9 hence the eigenvalues of

matrix
A k+1 have modules strictly smaller than 1.

In this way, we proved the possibility of perfrming the iterative construction for
Pk ,V k ,W k ,F k ,

starting from
F0 . .

We shall prove that sequence

Pk converges. Wehave:
¿ ¿ ¿
Pk+1= A k Pk +1 Ak +C C +F k RFk .

On the other hand,

¿ ¿ 2 ¿ ¿
Pk = Ak P k +1 A k + ( F k −1 F k ) V k ( F k−1 −F k ) +C C+F k RF k .

We deduce that

¿ ¿ 2
Pk −Pk+1 =A k ( P k−P k +1 ) + ( F k−1−F k ) V k ( F k−1−F k ) .

Since the eigenvalues of

Ak have modulus strictly smaller than 1. We deduce, on the basis of

Proposition 7.8 that

Pk −Pk +1 ≥0, hence the sequence
Pk is decreasing and bounded from below
( Pk ≥0 ) , wat ensures its convergence.

Recall that

V 2k =R+B¿ Pk B . W ¿k V k= A ¿ Pk B . F k=−V −1
k Wk .

Since
Pk ≥0 , we have V 2k ≥R≻0 .
We deduce that
Vk with
V k ≻0 exists and is invertible. It follows that W k =V −1 ¿
k B Pk A .

From here we see that

¿ ¿ −1 ¿
F k=−V −2
k B P k A=−( R+B Pk B ) B P k A .

lim Pk =P
From k→ it follows that
−1
lim ( R+ B¿ P k B )−1 =( R+B¿ PB )
k →∞

−1
(the sequence |( R+B¿ PB ) | is bounded by
−1
|R | , hence

−1
lim F k =−( R+ B¿ PB ) B ¿ PA=F .
k ←∞

A k = A+BF k lim A k= A+ BF
Since it results that k →∞ and it follows that

P=( A+BF )¿ P ( A+BF ) +C ¿ C+F ¿ RF=( A ¿ +F ¿ B¿ ) ( PA+PBF ) +C ¿ C +F ¿ RF=

¿ ¿ ¿ ¿ ¿ ¿ ¿
A PA+ A PBF+F B PA+F B PBF+CC +F RF=
A ¿ PA+ A ¿ PBF+F ¿ B¿ PA+C ¿ C+F ¿ ( B ¿ PB+R ) F .
Replacing the formula for F , we obtain:

−1 −1 −1
P= A ¿ PA+C¿ C−2 A ¿ PB ( R +B ¿ PB ) B¿ PA+ A ¿ PB ( R+B¿ PB ) ( R+B¿ PB ) ( R+B¿ PB ) B ¿ PA
.

We obtain in this way for P the equation:

−1
A ¿ PA−P− A¿ PB ( R+B¿ PB ) B ¿ PA +C¿ C=0
Which is termed as the algebraic matrix Riccati equation.

Let us denote V 2 =R+B ¿ PB and W=V −1 B ¿ PA .

We see now that the equation for P may be written as:

¿ ¿ ¿
A PA−P−W W −C C=0 .
Pg. 222

We finally obtained a solution ( V ,P ,W ) to the system

¿ ¿
¿
R+ B PB=V
2 ¿
A PB=W V
¿
P= A PA+Q−W W .

as required.

We have still to check that the eigenvalues of A + BF have modulus strictly smaller than 1. We have
¿ ¿ ¿
P=( A+BF ) P ( A+BF ) +C C+F RF .

The conclusion will follow as in the proof of Proposition 7.9.

t
We consider
x t+1=( A+BF ) x t , hence x t =( A + BF ) x 0 .

ut =( Px t , x t )
: it follows

1
¿ 2 2 2
ut+1 =( P ( A +BF ) x t , ( A +BF ) xt ) =( ( A+BF ) P ( A+BF ) x t , x t )=( Pxt , x t ) −|Cxt| −|R Fxt| .

1
ut+1 −ut =−|Cxt| −|R 2 Fxt|2 2
.

By summation from 0 to T−1 , we obtain

T −1 T −1 1
2 2 2 2
∑ |Cxt| + ∑ |R Fxt| =u0 −uT ≤u0 ≤μ|x|
t =0 t =0 .

Assuming the existence of a matrix

H0 such that the eigenvalues of
A +H 0 C were strictly inside
the unit circle, we can write
−1 1
2 2
x t +1=( A+ BF ) x t =( A + H 0 C ) x t − H 0 Cx t +BR R Fxt .

Pg. 223

t
|( A+H 0 C ) |≤βq '
We take into account that and we repeat the reasoning of the proff Proposition
7.9.

Let us summarize the conclusion we reached:

 x t+1= Ax t +But and the cost-function
∞
J (u )=∑ |Cx t|2 + ( Rut ,u t )
[ ]
t=0 , with R≻0 .

We assume the existence of matrices 0 , F H0 such that

A +BF0 ,
A +H 0 C have
eigenvalues with modulus strictly smaller 1.

Then the sequences defined by

Pk +1=( A +BFk )¿ Pk +1 ( A+BF k ) +C¿ C+ F¿k RF k

,

V 2k+1 =R +B ¿ P k+1 B .

W ¿k+1 V k+1=A ¿ Pk+1 B .

F k+1 =−V −1
k+1 W k +1 .

Starting with
F0 , are convergent and the control
ut =Fx t stabilizes the system (in the sense that

|x t|≤βq'|x 0| , 0≺q≺1 )

and achieves the minimum cost. The minimum value of the cost is

F=lim F k P=lim P k
( Px 0 , x 0 ) ; here k →∞ , k →∞ .

Finally, let us make the following important remark. If, during the iterative process, we stop at a step
k

And choose the feed-back control

ut =F k x t , then the evolution will by

x t+1= Ax t + But =( A +BF k ) x t = A k x t

Pagina 224

and, hence, it will satisfy

|x t|≤βq'|x 0| , 0≺q≺1 .

Moreover, the cost associated to this evolution is

 ∞ ∞ ∞
∑[ ¿
] [ ¿
] ¿ ¿
(C Cxt , x t )+( Rut , ut ) =∑ ( C Cx t , x t )+ ( RF k x t , F k xt ) =∑ ( ( C C+ Fk RF k ) x t , x t )=
t=0 t=0 t=0
∞ ∞
¿
∑ [ ( Pk+1 x t , x t )−( A k Pk +1 Ak x t , xt ) ]=∑ [ ( Pk+1 x t , x t )−( P k +1 xt +1 , x t+1) ]=( P k +1 x 0 , x 0 )
t=0 t=0

Since we have k+1 P ≤P k , it can be seen that, at each iteration, we get a stabilizing feed-bach which
improves the cost.

7.4. Optimal Stabilization in a Discrete-Time Model of Price Evolution

We consider the discrete-time model of price evolution described in 1.5. This time, we shall denote the

price at time t by
yt . We can transform this evolution in a controlled one assuming, for instance,
that α−β is no longer constant, but can be controlled by a decision-maker, by influencing
correspondingly

the supply or demand. Thus we have the evolution:

y t+1 =[ 1−λ ( b−a ) ] y t +v t (7.2)

where
vt is the control.

For a desired equilibrium value for the control; the equilibrium condition

^y = ^y [ 1−λ ( b−a ) ] + λ v^ implies v^ =( b−a ) ^y .

Denoting
x t = y t − ^y ,
ut =v t −^v , we obtain the evolution:

x t +1= [ 1−λ ( b−a ) ] xt + λu t (7.3)

We associate to this evolution the cost:

∞
J (u )=∑ ( c 2 x 2t +r 2 u 2t )
t=0 (7.4)

and state the problem of finding the control u that minimizes this cost. Here
xt represents the
u
deviation of the actual price from the desired one and t represents the effort of control; the
coefficients c and r represent weights associated to the above quantities, reflecting preferences of
the decision-maker.
Remark 7.11 We remark that the evolution given by relation (7.3) is of the same type as that given by
(7.2). Nevertheless, we preferred to work with thre variables x and u instead of y and v
respectively, in order to have the usual of a stabilization problem.

Pg. 226

Remark 7.12 The evolutions given by (7.2) and (7.3) are stable if λ ( b−a )≺2 , since them the

modulus of the coefficient 1−λ ( b−a ) is strictly smaller tham 1 (see 1.5).

In order to find the optimal control, we associate to our problem (which consistes of relations (7.3) and
(7.4) the nonlinear system:
2
r 2 + λ2 p=V 2 λ [ 1−λ ( b−a ) ] p=WV p=[ 1− λ ( b−a ) ] p+c 2 −W 2 .

We have

λ [ 1− λ ( b−a ) ] p
W=
V = √ r 2 + λ2 p ; √r 2 +λ2 p
Denoting λ ( b−a ) =d and substituting V and W into the third aquation, we get

2
2 2 λ 2 ( 1−d ) p2
p [ 1−( 1−d ) ]−c + 2 2
=0
r +λ p
Or

λ2 p2 −( d2 r 2−2 dr 2 + c 2 λ2 ) p−c 2 r 2 =0 .

Which in an equation of second degree with respect to p . This equation has real roots p1 , p 2 and

c2 r 2
p1 p 2 =− ≺0 p1 .
λ , therefore the equation has a unique positive root, say

~
u t =Fxt
We get then optimal feed-back control in the form where

W λ ( 1−d ) p 1
F=− =− 2 2
V r + λ p1 .

7.5 Notes and References

A source for linear-quadratic optimization problem can be considered to be the paper of R.E. Kalman
[15].
For further developments and the discussion of the discrete-time case, see the monograph of Popov
[25].

The iterative procedure to solve the algebraic Riccati equation has been proposed by Kleiman in [16].

The Kalman-Szego equations, which apper in the solution of the discrete-time optimal stabilization
problem, were obtained in connection with the existence of a Liapunov function in the absolute stability

problem. The connection with the linear-quadratic optimization problem was mentioned first by
V.M.

Popov in 1966, in his book on hyperstability [25].

The Riccati equation approach to solve the optimal stabilization problem may be found, for example, in
the book by Kwakernaak and Sivin [18].