Circuits Syst Signal Process (2017) 36:1406–1425

DOI 10.1007/s00034-016-0368-6

Recursive Least Squares Algorithm for Nonlinear

Dual-rate Systems Using Missing-Output Estimation
Model

Jing Chen1 · Yanjun Liu1 · Xuehai Wang1

Received: 16 February 2016 / Revised: 8 July 2016 / Accepted: 11 July 2016

Published online: 18 July 2016
© Springer Science+Business Media New York 2016

Abstract In this paper, a recursive least squares algorithm is proposed for a class
of nonlinear dual-rate systems. By using the missing-output estimation model, the
unavailable outputs can be estimated. Then, the unknown parameters can be estimated
from all the inputs and outputs. Compared with the polynomial transformation tech-
nique and the lifting technique, the unknown parameters can be estimated directly by
using the missing-output estimation model, without increasing the number of parame-
ters. The convergence analysis and the simulation results indicate that the proposed
method is effective.

Keywords Parameter estimation · Recursive least squares algorithm · Missing-output

estimation model · Martingale convergence theorem · Nonlinear system

This work was supported by the National Natural Science Foundation of China (Nos. 61403165,
61374126), the Natural Science Foundation of Jiangsu Province (No. BK20131109) and the Post Doctoral
Foundation of Jiangsu Province (No. 1501015A).

B Jing Chen
chenjing1981929@126.com
Yanjun Liu
yanjunliu_1983@126.com
Xuehai Wang
xuehaiwang735@163.com

1 Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), School
of Internet of Things Engineering, Jiangnan University, Wuxi 214122, People’s Republic of China
Circuits Syst Signal Process (2017) 36:1406–1425 1407

1 Introduction

Nonlinear systems have many applications in industrial processes, and the identifica-
tion of nonlinear systems has been an active research area for the past few decades
[14,15,17,22,36]. Compared with linear identification, nonlinear identification is
more difficult because of the rich nonlinear structures [25,45]. The polynomial non-
linearity which can be written as an analytic function of the input is one of the most
studied nonlinearities and has been received much attention in recent years. There
exist many identification methods for systems with polynomial nonlinearity, such as
the stochastic gradient (SG) algorithms [18,35], the recursive least squares (RLS)
algorithms [6,9] and the iterative algorithms [1,7,16]. However, most of the work
assumed that the nonlinear systems are single-rate systems. When the systems are
dual-rate systems, these methods cannot be applied directly.
Dual-rate systems often exist in process industries, for example, in digital sig-
nal processing, polymer reactors and process estimation [21,23,34,46]. The lifting
technique and the polynomial transformation technique are two methods which are
generally used for dual-rate systems identification [3,24,31,44]. The basic idea of the
lifting technique is to use a lifting operator to turn the dual-rate system into a lifted
system which contains all the fast inputs and the slow outputs [10]. Unfortunately,
the number of the unknown parameters will increase due to the transformation. The
polynomial transformation technique is used to find a polynomial to transform the
single-rate system into a dual-rate system, and then, the parameters can be estimated
from the dual-rate sampled data [8]. However, it has a restriction of the polynomial
and cannot directly estimate the parameters of the dual-rate systems.
The auxiliary model is often applied to overcome the shortcomings of the lifting
technique and the polynomial transformation technique [37,38]. Ding et al. [5] pro-
posed an auxiliary model-based RLS algorithm for dual-rate state space systems with
time delay, the inner variables can be predicted by the auxiliary model, and the para-
meters can be estimated by the RLS algorithm. Recently, a missing-output estimation
model-based SG algorithm for a class of dual-rate linear systems was developed [2].
The basic idea of the missing-output estimation model is to keep the parameter esti-
mate updating at the slow rate and then to replace the unavailable outputs in the two
slow samples with the outputs of the missing-output estimation model by using the
parameter estimate. Finally, the unavailable outputs and the unknown parameters are
estimated iteratively. This algorithm can estimate not only the unknown parameters
and the unavailable outputs directly but also can keep the number of the unknown
parameters unchanged. Although a simulation example was proposed to verify the
effective of the algorithm, the convergence properties were not involved.
Based on the work in [2,26–28], this paper presents a missing-output estimation
model-based RLS algorithm for a nonlinear dual-rate system and theoretically analyzes
the convergence properties of the proposed algorithm. The main contributions of this
paper are as follows.

1. To use the martingale theory to prove the convergence properties of the proposed
algorithm.
1408 Circuits Syst Signal Process (2017) 36:1406–1425

v(t)
1
A(z)

u(k) u(t) y(t) y(kq)

B(z)
Hh f (·) A(z)
+ Sqh

Fig. 1 The nonlinear dual-rate system

2. To use the missing-output estimation model-based RLS algorithm to estimate the

unavailable outputs and the unknown parameters iteratively.
3. To use the missing-output estimation model to keep the number of the unknown
parameters unchanged and to make the polynomial with no restriction.
Briefly, this paper is organized as follows. Section 2 introduces the nonlinear dual-
rate system and the identification algorithms. Section 3 proves the convergence of the
proposed algorithm. Section 4 provides three illustrative examples. Finally, concluding
remarks are given in Section 5.

2 The Nonlinear Dual-rate System and the Identification Algorithms

Let “A =: X ” or “X := A” stand for “A is defined as X ”, and the superscript T

denote the matrix transpose. X = det[X] denotes the matrix determinant, ||X||2 =
tr[XXT ]. λmax [X] and λmin [X] represent the maximum and minimum eigenvalues of
X. f (k) = O[g(k)] means f (k) = ηg(k), and η is a positive constant. f (k) = o[g(k)]
f (k)
means lim g(k) = 0.
k→∞
Consider the following nonlinear dual-rate system which is shown in Fig. 1,

A(z)y(t) = B(z) f (u(t)) + v(t), (1)

where y(t) is the system output, u(t) is the system input and v(t) is a stochastic
white noise with zero mean. A(z) and B(z) are polynomials in the unit backward shift
operator [z −1 y(t) = y(t − 1)] and

A(z) := 1 + a1 z −1 + a2 z −2 + · · · + an z −n ,
B(z) := b1 + b2 z −1 + · · · + bn z −n+1 .

The dual-rate sampled-data system in which all the input data {u(t), t = 0, 1, 2, . . .}
and only the scarce output data {y(tq), t = 0, 1, 2, . . . , } are available (q  2).
Figure 1 shows the dual-rate sampling case, and Hh is the zero-order hold with period
h which converts the discrete-time signal u(k) into a continuous-time signal u(t),
where Sqh is a sampler with period qh, sampling continuous-time signal y(t) to yield
a discrete-time signal y(kq).
Replacing t with kq, the dual-rate system can be rewritten as

A(z)y(kq) = B(z) f (u(kq)) + v(kq). (2)

Circuits Syst Signal Process (2017) 36:1406–1425 1409

The nonlinear input f (u(kq)) is a polynomial nonlinearity which can be written as

f (u(kq)) = r1 f 1 (u(kq)) + r2 f 2 (u(kq)) + · · · + rm f m (u(kq)),

where f i (u(kq)), i = 1, 2, . . . , m, are known basis and ri are the unknown parame-
ters to be estimated. It is impossible to distinguish ri from ri B(z), because any pair
(ri f (u(kq)), ri−1 B(z)) would produce identical input and output measurements. In
this paper, we assume b1 = 1.
Equation (2) can be written as
A(z)y(kq) = B(z)(r1 f 1 (u(kq)) + r2 f 2 (u(kq)) + · · · + rm f m (u(kq))) + v(kq).
(3)

Assuming that the nth-order polynomial A(z) has n real roots, z i , i = 1, 2, . . . , n,

that is A(z) can be written as

A(z) = (1 − z 1 z −1 )(1 − z 2 z −1 ) · · · (1 − z n z −1 ).

Let

n
(1 + z i z −1 + z i2 z −2 + · · · + z i
q−1 −q+1
φ(z) := z ).
i=1

Multiplying both sides of (3) by φ(z) gets

φ(z)A(z)y(kq) = φ(z)B(z)(r1 f 1 (u(kq)) + r2 f 2 (u(kq)) + · · · +

rm f m (u(kq))) + φ(z)v(kq). (4)

Rewritten (4) as an identification model

y(kq) = ψ T (kq)θ 1 + v(kq), (5)

where the parameter vector θ 1 and the information vector ψ(kq) are defined as

θ 1 := [α1 , α2 , . . . , αn , r1 , r2 , . . . , rm , β2 r1 , β2 r2 , . . . , β2 rm , . . . ,
βnq r1 , . . . , βnq rm , d1 , d2 , . . . , dnq−n ]T ∈ Rnq+mnq , (6)
ψ(kq) := [−y(kq − q), . . . , −y(kq − nq), f 1 (u(kq)), . . . ,
f m (u(kq)), f 1 (u(kq − 1)), . . . , f m (u(kq − 1)), . . . ,
f 1 (u(kq − nq + 1)), . . . , f m (u(kq − nq + 1)), v(kq − 1),
v(kq − 2), . . . , v(kq − nq + n)]T ∈ Rnq+mnq . (7)

Then, we can use the following polynomial transformation technique based RLS (PT-
RLS) algorithm to estimate the unknown parameter vector θ 1 ,

θ̂ 1 (kq) = θ̂ 1 (kq − q) + P̂ 1 (kq)ψ̂(kq)e1 (kq), (8)

1410 Circuits Syst Signal Process (2017) 36:1406–1425

ψ̂(kq) = [−y(kq − q), . . . , −y(kq − nq), f 1 (u(kq)), . . . , f m (u(kq)),

f 1 (u(kq − 1)), . . . , f m (u(kq − 1)), . . . , f 1 (u(kq − nq + 1)), . . . ,
f m (u(kq −nq +1)), v̂(kq −1), v̂(kq −2), . . . , v̂(kq −nq +n)]T , (9)
T
e1 (kq) = y(kq) − ψ̂ (kq)θ̂ 1 (kq − q), (10)
T
v̂(kq) = y(kq) − ψ̂ (kq)θ̂ 1 (kq), (11)
−1 −1 T
P̂ 1 (kq) = P̂ 1 (kq − q) + ψ̂(kq)ψ̂ (kq). (12)

Next, we use a missing-output estimation model-based method to estimate the

unknown parameters. Define the parameter vector θ and the information vector ϕ(kq)
as
θ := [a1 , a2 , . . . , an , r1 , r2 , . . . , rm , b2 r1 , b2 r2 , . . . ,
b2 rm , . . . , bn r1 , . . . , bn rm ]T ∈ Rn+nm , (13)
ϕ(kq) := [−y(kq − 1), . . . , −y(kq − n), f 1 (u(kq)), . . . , f m (u(kq)),
f 1 (u(kq − 1)), . . . , f m (u(kq − 1)), . . . , f 1 (u(kq − n + 1)), . . . ,
f m (u(kq − n + 1))]T ∈ Rn+nm . (14)

Then, we can get an identification model

y(kq) = ϕ T (kq)θ + v(kq). (15)

The problem is that in ϕ(kq), y(kq − 1), . . . , y(kq − q + 1), y(kq − q − 1), . . .
are unavailable. To solve this problem, we use their estimates to replace them. The
estimates of the missing outputs can be obtained by

ŷ(kq − i) = ϕ̂ T (kq − i)θ̂ (kq − q), i = 1, 2, . . . , q − 1, (16)

where
ϕ̂(kq − i) = [− ŷ(kq − i − 1), . . . , − ŷ(kq − q + 1), −y(kq − q), . . . ,
− ŷ(kq −i −n), f 1 (u(kq −i)), . . . , f m (u(kq −i)), f 1 (u(kq −i −1)),
. . . , f m (u(kq − i − 1)), . . . , f 1 (u(kq − i − n + 1)), . . . ,
f m (u(kq − i − n + 1))]T . (17)

Then, we can get the following missing-output estimation model-based recursive

least squares algorithm (MOE-RLS):

θ̂ (kq) = θ̂ (kq − q) + P̂(kq)ϕ̂(kq)e(kq), (18)

ϕ̂(kq) = [− ŷ(kq − 1), . . . , − ŷ(kq − n), f 1 (u(kq)), . . . ,
f m (u(kq)), f 1 (u(kq − 1)), . . . ,
f m (u(kq − 1)), . . . , f 1 (u(kq − n + 1)), . . . , f m (u(kq − n + 1))]T ,
(19)
Circuits Syst Signal Process (2017) 36:1406–1425 1411

ŷ(kq − i) = ϕ̂ T (kq − i)θ̂ (kq − q), i = 1, 2, . . . , q − 1, (20)

ϕ̂(kq − i) = [− ŷ(kq − i − 1), . . . , − ŷ(kq − q + 1), −y(kq − q), . . . ,
− ŷ(kq − i − n), f 1 (u(kq − i)), . . . , f m (u(kq − i)),
f 1 (u(kq − i − 1)), . . . , f m (u(kq − i − 1)), . . . ,
f 1 (u(kq − i − n + 1)), . . . , f m (u(kq − i − n + 1))]T , (21)
e(kq) = y(kq) − ϕ̂ (kq)θ̂ (kq − q),
T
(22)
−1 −1
P̂ (kq) = P̂ (kq − q) + ϕ̂(kq)ϕ̂ (kq).
T
(23)

The steps of computing the parameter estimation vector θ̂ (kq) by using the MOE-RLS
algorithm are listed in the following.
1. Let θ̂ (0) = 1/ p0 and P̂(0) = p0 I with 1 being a column vector whose entries
are all unity, I be an identity matrix of appropriate size and p0 = 106 .
2. Let k = 1, y(− j) = 0, u(− j) = 0, j = 0, 1, 2, . . . , n − 1, and give a small
positive number ε.
3. Let i = q −1, and collect the input-output data {u(k), u(k +1), . . . , u(kq), y(kq)}.
4. Form ϕ̂(kq − i) by (21).
5. Compute ŷ(kq − i) by (20), decrease i by 1, if i  1, go to step 4; otherwise, go
to next step.
−1
6. Compute e(kq) and P̂ (kq) by (22) and (23), respectively.
7. Update the parameter estimation vector θ̂(kq) by (18).
8. Compare θ̂ (kq) and θ̂ (kq − q): if θ̂ (kq) − θ̂ (kq − q)  ε, then terminate the
procedure and obtain the θ̂ (kq); otherwise, increase k by 1 and go to step 3.
Compared with the PT-RLS algorithm, the MOE-RLS algorithm has the following
advantages:
1. The number of the unknown parameters in the MOE-RLS algorithm (n + mn) is
less than that in the PT-RLS algorithm (nq + mnq).
2. The nth-order polynomial A(z) in the MOE-RLS algorithm does not require to
have n real roots.
3. The noise in the MOE-RLS algorithm is a white noise. However, the noise in the
PT-RLS algorithm is a colored noise.

3 Convergence of the MOE-RLS Algorithm

Define
−1
r̂ (kq) := tr[ P̂ (kq)], r (kq) := tr[ P −1 (kq)], θ̂(kq) − θ := θ̃ (kq),
θ̂ (kq − q) − θ := θ̃(kq − q), ε(kq) := y(kq) − ϕ̂ T (kq)θ̂ (kq).

From (18), we can get

θ̃ (kq) = θ̃ (kq − q) + P̂(kq)ϕ̂(kq)e(kq). (24)

1412 Circuits Syst Signal Process (2017) 36:1406–1425

According to (22), we have

 −1
ε(kq) = 1 + ϕ̂ T (kq) P̂(kq − q)ϕ̂(kq) e(kq). (25)

−1
It is easy to get ln[ P̂ (kq)] = O(ln r̂ (kq)) [4].
In order to prove the convergence properties of the MOE-RLS algorithm, the fol-
lowing lemmas are introduced.

Lemma 1 The following inequalities hold:


k
−1
1. ϕ̂ T ( jq) P̂( jq)ϕ̂( jq)  ln | P̂ (kq)| + n 0 ln p0 , a.s., n 0 = n + mn.
j=1
∞
ϕ̂ T ( jq) P̂( jq)ϕ̂( jq)
2. −1
< ∞, a.s., for any c > 1.
j=1 {ln | P̂ ( jq)|}c

The proof of Lemma 1 is given in [4] and hence omitted.

Lemma 2 For the following ARMA model

C(z)y(t) = D(z)v(t),
C(z) = 1 + c1 z −1 + · · · + cnc z −nc ,
D(z) = 1 + d1 z −1 + d2 z −2 + · · · + dnd z −nd ,

assume the noise sequence {v(t)} with zero mean and bounded time-varying variance,
and all the roots of the polynomial C(z) are inside the unit circle. Then, the output
sequence {y(t)} satisfies:

E(y(t)) = 0.

Proof The ARMA model can be rewritten as

y(t) = −c1 y(t − 1) − c2 y(t − 2) − · · · − cnc y(t − n nc ) + v(t) + d1 v(t − 1) + · · ·

+ dnd v(t − nd). (26)

All the roots of the polynomial C(z) are inside the unit circle, and thus, we can get
that the sequence {y(t)} is a stationary series and assume

E(y(t)) = μ,

where μ is a constant.

Taking the conditional expectation on both sides of (26) gets

Circuits Syst Signal Process (2017) 36:1406–1425 1413

E(y(t)) = −c1 E(y1 (t − 1)) − c2 E(y1 (t − 2))− · · · − cnc E(y1 (t − n nc )) + E(v(t))+

d1 E(v(t − 1)) + · · · + dnd E(v(t − nd)).

The noise sequence {v(t)} is with zero mean, and thus, we have

μ = −c1 μ − c2 μ − · · · − cnc μ + 0 + d1 0 + · · · + dnd 0,

(1 + c1 + c2 + · · · + cnc )μ = 0.

1 + c1 + c2 + · · · + cnc = 0, and thus, we can get E(y(t)) = μ = 0.

Lemma 3 For the nonlinear dual-rate system in (1) and the MOE-RLS algorithm
in (18)–(23), assume that the noise sequence {v(t)} with zero mean and bounded
time-varying variance satisfies:

(A1) E[v(k)|Fk−1 ] = 0, a.s.,

(A2) E[v 2 (k)|Fk−1 ] = σ 2 (k)  σ 2 < ∞, a.s.,
1 1
(A3) H (z) = − is strictly positive real,
A(z) 2
(A4) A(z) is stable and all the roots of A(z) are inside the unit circle.

Then, the following inequality holds


k
E[V (kq) + (2 − A(z))[ϕ̂ T ( jq)θ̃ ( jq)]2 |Fkq−1 ]  V (kq − q) +
j=1


k−1
(2 − A(z))[ϕ̂ T ( jq)θ̃ ( jq)]2 + 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)σ 2 , (27)
j=1

T −1
where V (kq) = θ̃ (kq) P̂ (kq)θ̃ (kq), {v(k), Fk } is a martingale sequence and {Fk }
is the σ -algebra sequence generated by {v(k)}.

−1
Proof Since P̂ (kq) is a positive definite matrix, V (kq)  0. From (24), we have

T −1
 
V (kq) = θ̃ (kq) P̂ (kq) θ̃ (kq − q) + P̂(kq)ϕ̂(kq)e(kq)
T −1 T
= θ̃ (kq − q) P̂ (kq)θ̃ (kq − q) + 2θ̃ (kq − q)ϕ̂(kq)e(kq)
+ ϕ̂ T (kq) P̂(kq)ϕ̂(kq)e2 (kq)
T −1 T
= θ̃ (kq − q) P̂ (kq − q)θ̃ (kq − q) + θ̃ (kq − q)ϕ̂(kq)ϕ̂ T (kq)θ̃ (kq − q)
T T
+ 2θ̃ (kq − q)ϕ̂(kq)(e(kq) − ε(kq)) + 2θ̃ (kq − q)ϕ̂(kq)ε(kq)
+ ϕ̂ T (kq) P̂(kq)ϕ̂(kq)e2 (kq)
 T 
= V (kq − q) + θ̃ (kq) − P̂(kq)ϕ̂(kq)e(kq) ϕ̂(kq)ϕ̂ T (kq) θ̃ (kq)
1414 Circuits Syst Signal Process (2017) 36:1406–1425

  T
− P̂(kq)ϕ̂(kq)e(kq) + 2 θ̃ (kq) − P̂(kq)ϕ̂(kq)e(kq) ϕ̂(kq)(e(kq)
 T
− ε(kq)) + 2 θ̃ (kq) − P̂(kq)ϕ̂(kq)e(kq) ϕ̂(kq)ε(kq)

+ ϕ̂ T (kq) P̂(kq)ϕ̂(kq)e2 (kq). (28)

Substituting (25) into (28) gets

 T 2 T
V (kq) = V (kq − q) + θ̃ (kq)ϕ̂(kq) + 2θ̃ (kq)ϕ̂(kq)ε(kq)
 
− ϕ̂ T (kq) P̂(kq)ϕ̂(kq) 1 − ϕ̂ T (kq) P̂(kq)ϕ̂(kq) e2 (kq). (29)

From the definition of P̂(kq − q), the third term in the right-hand side of Eq. (29)
satisfies

1 − ϕ̂ T (kq) P̂(kq)ϕ̂(kq) = [1 + ϕ̂ T (kq) P̂(kq − q)ϕ̂(kq)]−1  0.

Then, we have
 T 2 T
V (kq)  V (kq − q) + θ̃ (kq)ϕ̂(kq) + 2θ̃ (kq)ϕ̂(kq)ε(kq)
T
= V (kq − q) + 2θ̃ (kq)ϕ̂(kq)v(kq)
 
T 1 T
+ 2θ̃ (kq)ϕ̂(kq) θ̃ (kq)ϕ̂(kq) + ε(kq) − v(kq)
2
T
= V (kq − q) + 2θ̃ (kq − q)ϕ̂(kq)v(kq) + 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)
T
[(e(kq) − v(kq))v(kq) + v 2 (kq)] + 2θ̃ (kq)ϕ̂(kq)
 
1 T
θ̃ (kq)ϕ̂(kq) + ε(kq) − v(kq) .
2
T
Clearly, θ̃ (kq − q)ϕ̂(kq) and ϕ̂ T (kq) P̂(kq)ϕ̂(kq) are uncorrelated with v(kq).
Because e(kq) − v(kq) = ϕ T (kq)θ − ϕ̂ T (kq)θ̂ (kq − q), ϕ T (kq) and θ are the true
information vector and the true parameter vector which are uncorrelated with v(kq),
the estimated information vector ϕ̂ T (kq) at time kq and the estimated parameter vector
θ̂ (kq − q) at time kq − q are also uncorrelated with v(kq). Thus, e(kq) − v(kq) is
uncorrelated with v(kq). We can get
 
T 1 T
E[V (kq)|Fkq−1 ]  V (kq − q) + 2E θ̃ (kq)ϕ̂(kq) θ̃ (kq)ϕ̂(kq)
2

+ ε(kq) − v(kq)] |Fkq−1
+ 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)σ 2 . (30)

Due to

ε(kq)−v(kq) = y(kq)− ϕ̂ T (kq)θ̂ (kq)− y(kq)+ϕ T (kq)θ = ϕ T (kq)θ − ϕ̂ T (kq)θ̂ (kq),

Circuits Syst Signal Process (2017) 36:1406–1425 1415

we have

A(z)[ε(kq + i) − v(kq + i)]

= A(z)[ϕ T (kq + i)θ − ϕ̂ T (kq + i)θ̂ (kq)]
= A(z)[y(kq + i) − v(kq + i) − ŷ(kq + i)]
= A(z)y(kq + i) − A(z) ŷ(kq + i) − A(z)v(kq + i)
= − ŷ(kq + i)) + B(z) f (u(kq + i)) + v(kq + i)
−(A(z) − 1) ŷ(kq + i) − A(z)v(kq + i)
= −ϕ̂ T (kq + i)θ̂ (kq) + ϕ̂ T (kq + i)θ + (1 − A(z))v(kq + i)
= −ϕ̂ T (kq + i)θ̃ (kq) + (1 − A(z))v(kq + i), i = 1, 2, . . . , q − 1. (31)

It follows that


1 T 1
ε(kq) − v(kq) = − ϕ̂ (kq)θ̃ (kq) + − 1 v(kq). (32)
A(z) A(z)

Substituting (32) into (30) gets


2
E[V (kq)|Fkq−1 ]  V (kq − q) + 1 − [ϕ̂ T (kq)θ̃ (kq)]2
A(z)


1
+ 2E[ϕ̂ (kq)θ̃ (kq)
T
− 1 v(kq)] + 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)σ 2 .
A(z)

Define


1 1 − A(z)
w(kq) = − 1 v(kq) = v(kq). (33)
A(z) A(z)

System (33) is an ARMA model; according to Lemma 2 and (A4), we can get
E(w(kq)) = 0. Then, E[V (kq)|Fkq−1 ] can be simplified as

2
E[V (kq)|Fkq−1 ]  V (kq −q)+(1− )[ϕ̂ T (kq)θ̃ (kq)]2 +2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)σ 2 .
A(z)

According to (A3), we have

 
2
− 1 [ϕ̂ T ( jq)θ̃ ( jq)]2  0.
A(z)
j=1

Thus, we can get

 
2
E[V (kq) + − 1 [ϕ̂ T ( jq)θ̃ ( jq)]2 |Fkq−1 ]  V (kq − q)
A(z)
j=1
1416 Circuits Syst Signal Process (2017) 36:1406–1425

k−1

 
2
+ − 1 [ϕ̂ T ( jq)θ̃ ( jq)]2 + 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)σ 2 . (34)
A(z)
j=1

Theorem For the system (1), assume that (A1)–(A4) hold. Then for any c > 1, the
parameter estimation error of the nonlinear dual-rate additive system based on the
MOE-RLS algorithm in (18)–(23) satisfies


[ln r (kq)]c
||θ̂(kq) − θ ||2 = O , a.s.
λmin [ P −1 (kq)]

−1
Proof Dividing both sides of (34) with [ln | P̂ (kq)|]c gets

k
 
V (kq) + 2
A(z) − 1 [ϕ̂ T ( jq)θ̃ ( jq)]2
j=1
−1
[ln | P̂ (kq)|]c
 2
k−1 
V (kq − 1) + A(z) − 1 [ϕ̂ T ( jq)θ̃ ( jq)]2
j=1 2ϕ̂ T (kq) P̂(kq)ϕ̂(kq)
 −1
+ −1
σ 2.
[ln | P̂ (kq)|]c [ln | P̂ (kq)|]c

According to Lemma 1 and use the martingale convergence theorem in [11], we can
get

k
 
V (kq) + 2
A(z) − 1 [ϕ̂ T ( jq)θ̃ ( jq)]2
j=1
→ V0 ,
[ln | P −1 (kq)|]c

where V0 is a finite random variable. This means that

V (kq) = O[ln | P −1 (kq)|]c = O([ln r̂ (kq)]c ),

k

2
− 1 [ϕ̂ T ( jq)θ̃ ( jq)]2 = O[ln | P −1 (kq)|]c = O([ln r̂ (kq)]c ),
A(z)
j=1

and we have
T −1
T θ̃ (kq) P̂ (kq)θ̃ (kq) V (kq) O([ln r̂ (kq)]c )
θ̃ (kq)θ̃ (kq)  −1
= −1
= −1
.
λmin P̂ (kq) λmin P̂ (kq) λmin P̂ (kq)

Then, we can get

 
T [ln r̂ (kq)]c
θ̃ (kq)θ̃ (kq) = O −1
, a.s.,
λmin P̂ (kq)
Circuits Syst Signal Process (2017) 36:1406–1425 1417


k
(2 − A(z))[ϕ̂ T ( jq)θ̃ ( jq)]2 = O([ln r̂ (kq)]c ), a.s.
j=1

Define

ϕ̂(kq) − ϕ(kq) := [ ŷ(kq − 1) − y(kq − 1), ŷ(kq − 2) − y(kq − 2), . . . ,

ŷ(kq − q + 1) − y(kq − q + 1), 0, ŷ(kq − q − 1) − y(kq − q − 1), . . . ,
ŷ(kq − q − n) − y(kq − q − n), 0, 0, . . . , 0]T ∈ Rn+mn . (35)

According to (31) and (32), we have

A(z)[ ŷ(kq − i) − y(kq − i)] = −ϕ̂ T (kq − i)θ̃ (kq − q) + v(kq − i),

and thus, we can get

A2 (z)[ ŷ(kq − i) − y(kq − i)]2 = [ϕ̂ T (kq − i)θ̃ (kq − q)]2

+ v 2 (kq − i) − 2ϕ̂ T (kq − i)θ̃ (kq − q)v(kq − i). (36)

Taking expectation of both sides of the above equation and using (A1) and (A2), we
have

E{A2 (z)[ ŷ(kq − i) − y(kq − i)]2 } = E{[ϕ̂ T (kq − i)θ̃ (kq − q)]2 } + σ 2 .

From (A4), we have


k
[ ŷ( jq) − y( jq)]2 = O([ln r̂ (kq)]c ), a.s.,
j=1

and


k 
k 
n
[ϕ̂( jq) − ϕ( jq)]2 = [ ŷ( jq − i) − y( jq − i)]2
j=1 j=1 i=1
⎛ ⎞
k
= O ⎝ [ ŷ( jq) − y( jq)]2 ⎠ = O([ln r̂ (kq)]c ).
j=1

−1
Since r̂ (kq) = tr[ P̂ (kq)] and r (kq) = tr[ P −1 (kq)], it is easy to get


k
r̂ (kq)  2r (kq) + 2 [ϕ̂( jq) − ϕ( jq)]2 = 2r (kq) + O([ln r̂ (kq)]c ).
j=1
1418 Circuits Syst Signal Process (2017) 36:1406–1425

Thus, we can get

r̂ (kq) = O(r (kq)).

Also we can deduce

−1
λmin [ P̂ (kq)] = O(λmin [ P −1 (kq)]).

Finally, we can get


T [ln r (kq)]c
θ̃ (kq)θ̃ (kq) = O .
λmin P −1 (kq)

Corollary Assume that there exist positive constants d0 , d1 , d2 and k0 such that
for k  k0 , the following generalized persistent excitation condition (unbounded
condition number) holds:

1
k
d1 I  ϕ( jq)ϕ T ( jq)  d2 k d0 I.
k
j=1

Then for any c > 1, we have


[ln k]c
||θ̂(kq) − θ || = O 2
→ 0, a.s.
k

4 Examples

Example 1 Consider the following nonlinear dual-rate system with q = 2,

[1 + 0.25z −1 + 0.46z −2 ]y(t) = [1 + 0.9z −1 ] f (u(t)) + v(t),

f (u(t)) = u(t) − 1 + eu(t) = 2u(t) + 0.5u 2 (t) + 0.17u 3 (t),

the input {u(t)} is taken as a persistent excitation signal sequence with zero mean
and unit variance, and {v(t)} is taken as a white noise sequence with zero mean and
variance σ 2 = 0.102 . The parameter vector θ and the information vector ϕ(t) can be
expressed as

θ = [a1 , a2 , r1 , r2 , r3 , b2 r1 , b2 r2 , b2 r3 ]T = [0.25, 0.46, 2, 0.5, 0.17, 1.8, 0.45, 0.153]T ,

ϕ(t) = [−y(t − 1), −y(t − 2), u(t), u 2 (t), u 3 (t), u(t − 1), u 2 (t − 1), u 3 (t − 1)]T .

Applying the proposed MOE-RLS algorithm to estimate the parameters of the system,
the parameter estimates and their errors are shown in Table 1, and the parameter
estimation errors δ := θ̂ − θ /θ  versus t are shown in Fig. 2, and the estimated
outputs and the true outputs are shown in Fig. 3 (samples 1000–1020).
Circuits Syst Signal Process (2017) 36:1406–1425 1419

Table 1 The MOE-RLS estimates and errors

t 100 200 300 500 1000 True values

a1 0.25142 0.24212 0.25568 0.26320 0.27770 0.25000

a2 0.45217 0.46524 0.45290 0.46406 0.45689 0.46000
r1 1.89310 1.92808 1.95084 2.02515 1.98465 2.00000
r2 0.47167 0.47968 0.49211 0.45100 0.49946 0.50000
r3 0.16880 0.16134 0.21686 0.13400 0.17168 0.17000
b2 r 1 1.64485 1.68783 1.68651 1.68009 1.82623 1.80000
b2 r 2 0.44125 0.44253 0.46054 0.52292 0.45062 0.45000
b2 r 3 0.31181 0.32084 0.30667 0.27920 0.11883 0.15300
δ (%) 8.76930 7.61968 7.18225 7.07534 1.89247

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 100 200 300 400 500 600 700 800 900 1000

Fig. 2 The parameter estimation errors δ versus t

Then, the following conclusions can be obtained.

1. From Fig. 2, we can see that the parameter estimation errors be given by the MOE-
RLS algorithm become smaller and smaller and go to zero with t increasing.
2. From Fig. 3, we can conclude that the estimated outputs are close to the true
outputs.
3. From Table 1, we can get that the parameters of the dual-rate system can be
estimated directly and the number of the unknown parameters is unchanged.

Example 2 Consider a simple computer-controlled experimental dual-rate setup of a

water tank system in Fig. 4. In this system, u(t) is the valve opening, and y(t) is the
water level in the tank. Assume the system be modeled by the following first-order
nonlinear ARX model,
1420 Circuits Syst Signal Process (2017) 36:1406–1425

2
System output

-1

-2

-3

-4
1000 1002 1004 1006 1008 1010 1012 1014 1016 1018 1020

Dotted line: the true output; +: the estimated output

Fig. 3 Estimated model outputs and true outputs

Fig. 4 A water tank system u(t)

with one water tank

y(t)

Table 2 The MOE-RLS estimates and errors

t 100 200 300 500 1000 True values

a1 0.41831 0.39699 0.39754 0.39761 0.40155 0.40000

r1 0.78247 0.77428 0.77918 0.78779 0.79640 0.80000
r2 0.39336 0.38557 0.38995 0.39029 0.39623 0.40000
δ (%) 2.67465 3.02568 2.37307 1.61055 0.55533
Circuits Syst Signal Process (2017) 36:1406–1425 1421

Table 3 The PT-RLS estimates and errors

t 100 200 300 500 1000 True values

α1 0.20560 0.21519 0.20352 0.16566 0.16168 0.16000

r1 0.78488 0.77107 0.77720 0.78819 0.80027 0.80000
r2 0.37784 0.36383 0.38209 0.38483 0.40189 0.40000
β2 r1 −0.28705 −0.26197 −0.26669 −0.30353 −0.31443 −0.32000
β2 r2 −0.13673 −0.11918 −0.13435 −0.15317 −0.16746 −0.16000
d1 −0.41621 −0.32212 −0.34450 −0.37109 −0.37697 −0.40000
δ (%) 6.48822 12.09512 9.14568 3.73761 2.36635

(1 + 0.4z −1 )y(t) = f (u(t)) + v(t),

f (u(t)) = 0.8u(t) + 0.4u 2 (t), (37)

the input {u(t)} is taken as an uncorrelated persistent excitation signal sequence and
updated at every h = 30 s, the output {y(t)} is sampled every qh = 60 s, q = 2, and
{v(t)} is taken as a white noise sequence with zero mean and variance σ 2 = 0.202 .
The parameter vector θ and the information vector ϕ(t) can be expressed as

θ = [a1 , r1 , r2 ]T = [0.4, 0.8, 0.4]T ,

ϕ(t) = [−y(t − 1), u(t), u 2 (t)]T .

Applying the proposed MOE-RLS algorithm to estimate the parameters of the system,
the parameter estimates and their errors are shown in Table 2.
Next, we use the PT-RLS algorithm to estimate the parameters of the system.
Multiplying both sides of (37) by φ(z) = 1 − 0.4z −1 gets

(1 − 0.16z −2 )y(tq) = (1 − 0.4z −1 ) f (u(tq)) + (1 − 0.4z −1 )v(tq),

f (u(tq)) = 0.8u(tq) + 0.4u 2 (tq).

The parameter vector θ 1 and the information vector ψ(tq) can be expressed as

θ 1 = [α1 , r1 , r2 , β2 r1 , β2 r2 , d1 ]T
= [0.16, 0.8, 0.4, −0.32, −0.16, −0.4]T ,
ψ(tq) = [−y(tq − q), u(tq), u 2 (tq), u(tq − 1), u 2 (tq − 1), u(tq − 1),
u 2 (tq − 1), v(tq − 1)]T .

Clearly, the white noise becomes a colored noise, and the number of the unknown
parameters is 6 which is larger than the initial number 3. Then, the parameter estimates
and their errors are shown in Table 3.
From Tables 2, 3, we can see that the MOE-RLS algorithm has higher estimation
accuracy than the PT-RLS algorithm. This is because the MOE-RLS algorithm has
less unknown parameters and has a white noise.
1422 Circuits Syst Signal Process (2017) 36:1406–1425

Fig. 5 A water tank system q1

with two water tanks u(t)

q2

Tank 1

y(t)

Tank 2

Table 4 The MOE-RLS estimates and errors

t 100 200 300 500 1000 True Values

a1 0.50345 0.48701 0.49655 0.49839 0.49159 0.50000

a2 0.29481 0.29552 0.29299 0.29919 0.30089 0.30000
r1 0.89409 0.90217 0.89564 0.90291 0.89982 0.90000
r2 0.39097 0.38365 0.38940 0.39131 0.39810 0.40000
b2 r 1 0.43632 0.42387 0.44752 0.45161 0.45421 0.45000
b2 r 2 0.19989 0.20060 0.19789 0.19680 0.19049 0.20000
δ (%) 1.48572 2.71453 1.14405 0.80310 1.08668

Example 3 Consider a water tank system with two water tanks in Fig. 5, where u(t)
is the valve opening and y(t) is the liquid level. Assume the system be modeled by
the following second-order nonlinear ARX model,

(1 + 0.5z −1 + 0.3z −2 )y(t) = (1 + 0.5z −1 ) f (u(t)) + v(t),

f (u(t)) = 0.9u(t) + 0.4u 2 (t),

the input {u(t)} is taken as an uncorrelated excitation signal sequence and updated
at every h = 20 s, the output {y(t)} is sampled every qh = 60 s, q = 3, and {v(t)}
is taken as a white noise sequence with zero mean and variance σ 2 = 0.202 . The
parameter vector θ and the information vector ϕ(t) can be expressed as

θ = [a1 , a2 , r1 , r2 , b2 r1 , b2 r2 ]T = [0.5, 0.3, 0.9, 0.4, 0.45, 0.20]T ,

ϕ(t) = [−y(t − 1), −y(t − 2), u(t), u 2 (t), u(t − 1), u 2 (t − 1)]T .

Since A(z) = 1 + 0.5z −1 + 0.3z −2 does not have two real roots, the polynomial
transformation technique cannot be applied for this dual-rate system. We can use the
MOE-RLS algorithm for this dual-rate system.
Circuits Syst Signal Process (2017) 36:1406–1425 1423

0.6

0.5

0.4

0.3

0.2

0.1

0
0 100 200 300 400 500 600 700 800 900 1000

Fig. 6 The parameter estimation errors δ versus t

Applying the proposed MOE-RLS algorithm to estimate the parameters of the

system, the parameter estimates and their errors are shown in Table 4, the parameter
estimation errors δ := θ̂ − θ /θ  versus t are shown in Fig. 6.

5 Conclusions

This paper presents a missing-output estimation model-based recursive least algorithm

for a class of nonlinear dual-rate systems. Based on the missing-output estimation
model, the unavailable outputs and the parameters can be estimated iteratively. The
convergence properties are also proved. This method can estimate the parameters
and the unavailable outputs directly and can keep the number of unknown parameters
unchanged and thus can be widely used in dual-rate system identification. Furthermore,
the method in this paper can be extended to the multivariable systems [19,39] and can
combine the iteration methods [40–43] to study the identification problems of other
linear systems [12,13,20] and nonlinear systems [29,30] and can be applied to other
fields [32,33].

References
1. F.Y. Chen, F. Ding, J.H. Li, Maximum likelihood gradient-based iterative estimation algorithm for a
class of input nonlinear controlled autoregressive ARMA systems. Nonlinear Dyn. 79(2), 927–936
(2015)
2. J. Chen, Several gradient parameter estimation algorithms for dual-rate sampled systems. J. Franklin
Inst. 351(1), 543–554 (2014)
3. J. Chen, Y.X. Ni, Parameter identification methods for an additive nonlinear system. Circuits Syst.
Signal Process. 33(10), 3053–3064 (2014)
1424 Circuits Syst Signal Process (2017) 36:1406–1425

4. F. Ding, T. Chen, Combined parameter and output estimation of dual-rate systems using an auxiliary
model. Automatica 40(10), 1739–1748 (2004)
5. F. Ding, X.M. Liu, Y. Gu, An auxiliary model based least squares algorithm for a dual-rate state space
system with time-delay using the data filtering. J. Franklin Inst. 353(2), 398–408 (2016)
6. F. Ding, X.M. Liu, M.M. Liu, The recursive least squares identification algorithm for a class of Wiener
nonlinear systems. J. Franklin Inst. 353(7), 1518–1526 (2016)
7. F. Ding, X.M. Liu, X.Y. Ma, Kalman state filtering based least squares iterative parameter estimation
for observer canonical state space systems using decomposition. J. Comput. Appl. Math. 301, 135–143
(2016)
8. F. Ding, X.P. Liu, H.Z. Yang, Parameter identification and intersample output estimation for dual-rate
systems. IEEE Trans. Syst. Man Cybern. A Syst. Hum. 38(4), 966–975 (2008)
9. F. Ding, X.H. Wang, Q.J. Chen, Y.S. Xiao, Recursive least squares parameter estimation for a class
of output nonlinear systems based on the model decomposition. Circuits Syst. Signal Process. (2016).
doi:10.1007/s00034-015-0190-6
10. J. Ding, J.X. Lin, Modified subspace identification for periodically non-uniformly sampled systems
by using the lifting technique. Circuits Syst. Signal Process. 33(5), 1439–1449 (2014)
11. G.C. Goodwin, K.S. Sin, Adaptive Filtering, Prediction and Control (Prentice-Hall, Englewood Cliff,
1984)
12. H. Li, Y. Shi, W. Yan, On neighbor information utilization in distributed receding horizon control for
consensus-seeking. IEEE Trans. Cybern. (2016). doi:10.1109/TCYB.2015.2459719
13. H. Li, Y. Shi, W. Yan, Distributed receding horizon control of constrained nonlinear vehicle formations
with guaranteed γ -gain stability. Automatica 68, 148–154 (2016)
14. H. Li, P. Xie, W. Yan, Receding horizon formation tracking control of constrained Underactuated
autonomous underwater vehicles. IEEE Trans. Ind. Electron. (2016). doi:10.1109/TIE.2016.2589921
15. H. Li, W. Yan, Model predictive stabilization of constrained underactuated autonomous underwater
vehicles with guaranteed feasibility and stability. IEEE/ASME Trans. Mechatron. (2016). doi:10.1109/
TMECH.2016.2587288
16. Y. Liu, E.W. Bai, Iterative identification of Hammerstein systems. Automatica 43(2), 346–354 (2007)
17. Y.W. Mao, F. Ding, Adaptive filtering parameter estimation algorithms for Hammerstein nonlinear
systems. Signal Process. 128, 417–425 (2016)
18. Y.W. Mao, F. Ding, Multi-innovation stochastic gradient identification for Hammerstein controlled
autoregressive autoregressive systems based on the filtering technique. Nonlinear Dyn. 79(3), 1745–
1755 (2015)
19. G. Mercère, L. Bako, Parameterization and identification of multivariable state-space systems: a canon-
ical approach. Automatica 47(8), 1547–1555 (2011)
20. J. Pan, X.H. Yang, H.F. Cai, B.X. Mu, Image noise smoothing using a modified Kalman filter. Neuro-
computing 173, 1625–1629 (2016)
21. R. Piza, J. Salt, A. Sala, A. Cuenca, Hierarchical triple-maglev dual-rate control over a profibus-DP
network. IEEE Trans. Control Syst. Technol. 22(1), 1–12 (2014)
22. M. Schoukens, A. Marconato, R. Pintelon, G. Vandersteen, Y. Rolain, Parametric identification of
parallel Wiener–Hammerstein systems. Automatica 51, 111–122 (2015)
23. Y. Shi, H. Fang, M. Yan, Kalman filter based adaptive control for networked systems with unknown
parameters and randomly missing outputs. Int. J. Robust Nonlinear Control. 19(18), 1976–1992 (2009)
24. M. Srinivasarao, S.C. Patwardhan, R.D. Gudi, Nonlinear predictive control of irregularly sampled
multirate systems using blackbox observers. J. Process Control 17(1), 17–35 (2007)
25. J. Vörös, Modeling and identification of systems with backlash. Automatica 46(2), 369–374 (2010)
26. C. Wang, T. Tang, Several gradient-based iterative estimation algorithms for a class of nonlinear systems
using the filtering technique. Nonlinear Dyn. 77(3), 769–780 (2014)
27. C. Wang, T. Tang, Recursive least squares estimation algorithm applied to a class of linear-in-parameters
output error moving average systems. Appl. Math. Lett. 29, 36–41 (2014)
28. C. Wang, L. Zhu, Parameter identification of a class of nonlinear systems based on the multi-innovation
identification theory. J. Franklin Inst. 352(10), 4624–4637 (2015)
29. D.Q. Wang, Hierarchical parameter estimation for a class of MIMO Hammerstein systems based on
the reframed models. Appl. Math. Lett. 57, 13–19 (2016)
30. D.Q. Wang, F. Ding, Parameter estimation algorithms for multivariable Hammerstein CARMA sys-
tems. Inf. Sci. 355, 237–248 (2016)
Circuits Syst Signal Process (2017) 36:1406–1425 1425

31. D.Q. Wang, W. Zhang, Improved least squares identification algorithm for multivariable Hammerstein
systems. J. Franklin Inst. 352(11), 5292–5307 (2015)
32. T.Z. Wang, J. Qi, H. Xu, L. Liu, D.J. Gao, Fault diagnosis method based on FFT-RPCA-SVM for
cascaded-multilevel inverter. ISA Trans. 60, 156–163 (2016)
33. T.Z. Wang, H. Wu, M.Q. Ni, An adaptive confidence limit for periodic non-steady conditions fault
detection. Mech. Syst. Signal Process. 72–73, 328–345 (2016)
34. X.H. Wang, F. Ding, Convergence of the recursive identification algorithms for multivariate pseudo-
linear regressive systems. Int. J. Adapt. Control Signal Process. 30(6), 824–842 (2016)
35. X.H. Wang, F. Ding, Convergence of the auxiliary model based multi-innovation generalized extended
stochastic gradient algorithm for Box–Jenkins systems. Nonlinear Dyn. 82(1–2), 269–280 (2015)
36. X.H. Wang, F. Ding, Recursive parameter and state estimation for an input nonlinear state space system
using the hierarchical identification principle. Signal Process. 117, 208–218 (2015)
37. Y.J. Wang, F. Ding, Novel data filtering based parameter identification for multiple-input multiple-
output systems using the auxiliary model. Automatica 71, 308–313 (2016)
38. Y.J. Wang, F. Ding, The auxiliary model based hierarchical gradient algorithms and convergence
analysis using the filtering technique. Signal Process. 128, 212–221 (2016)
39. Y.J. Wang, F. Ding, The filtering based iterative identification for multivariable systems. IET Control
Theory Appl. 10(8), 894–902 (2016)
40. L. Xu, The damping iterative parameter identification method for dynamical systems based on the sine
signal measurement. Signal Process. 120, 660–667 (2016)
41. L. Xu, A proportional differential control method for a time-delay system using the Taylor expansion
approximation. Appl. Math. Comput. 236, 391–399 (2014)
42. L. Xu, Application of the Newton iteration algorithm to the parameter estimation for dynamical systems.
J. Comput. Appl. Math. 288, 33–43 (2015)
43. L. Xu, L. Chen, W.L. Xiong, Parameter estimation and controller design for dynamic systems from
the step responses based on the Newton iteration. Nonlinear Dyn. 79(3), 2155–2163 (2015)
44. B. Yu, Y. Shi, H. Huang, l2 − l∞ filtering for multirate systems using lifted models. Circuits Syst.
Signal Process. 27(5), 699–711 (2008)
45. H. Zayyani, Continuous mixed p-norm adaptive algorithm for system identification. IEEE Signal
Process. Lett. 21(9), 1108–1110 (2014)
46. H. Zhang, Y. Shi, J. Wang, On energy-to-peak filtering for nonuniformly sampled nonlinear systems:
a Markovian jump system approach. IEEE Trans. Fuzzy Syst. 22(1), 212–222 (2014)