IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 62, NO.

3, FEBRUARY 1, 2014 557

Optimal Memory for Discrete-Time FIR Filters in II. ESTIMATION OF OPTIMAL MEMORY
State-Space Assume that the -state process is described in discrete-time state-
Felipe Ramirez-Echeverria, Amadou Sarr, and space from zero to with the time-invariant state and obser-
Yuriy S. Shmaliy, Fellow, IEEE vation equations, respectively,

(1)
(2)
Abstract—In this correspondence, we propose an efficient estimator of
optimal memory (averaging interval) for discrete-time finite impulse re-
sponse (FIR) filters in state-space. Its crucial property is that only real mea- where , and
surements and the filter output are involved with no reference and noise . Here, and are zero mean,
statistics. Testing by the two-state polynomial model has shown a very good and , white sequences with the covariances
correspondence with predicted values. Even in the worst case of the har- and , respectively, and property
monic model, the estimator demonstrates practical applicability.
for all and .
Index Terms—FIR filtering, optimal memory, state space, discrete time. We can now apply a FIR estimator with memory and
assign the estimate obtained via measurements from zero to as
. The MSE in this estimate can be defined as
I. INTRODUCTION
Since Jazwinski has proposed a limited memory optimal filter [1], (3)
the latter was developed by various authors in different forms [2]–[5].
In contrast to the recursive Kalman filter having the infinite impulse where means averaging of and is the estimation
response (IIR), the discrete-time finite impulse response (FIR) filter error. The MSE will be truly minimal if we minimize (3) by and
utilizes data from the nearest past of points (memory). In order for let . In doing so, one can either minimize the trace
the mean square errors (MSEs) in such filters to be truly minimal, , if needs to be applied to all of the states at once, or the
should be optimal, . Otherwise, if , noise reduction is th component of corresponding to the desired th state,
inefficient and, when , the estimate bias dominates. Note respectively,
that such filters are not selective filters. Thus, there is no exact rela-
(4)
tion between the filter order and memory. The FIR filter memory opti-
mization in state space needs finding the derivative of the MSE matrix (5)
with respect to the convolution batch length that implies a mathemat-
ical problem. Therefore, the authors typically suppose that is given
and derive conditionally optimal filters. The value is then han- Both estimates, and , can serve as depending on ap-
dled heuristically or by employing real measurements for some refer- plications. For example, in the velocity measuring radars or frequency
ence model [5], [6]. In [7], has been determined for polynomial measuring systems [6], required only for the second state can be
models via the higher order states that has clear limitations. For sig- found using (5) with . In GPS-based ground applications, all of
nals with known bandwidth , the approach was proposed in [4]. Its the vehicular states commonly need to be estimated at once. Thus (4)
flaw is that is highly sensitive to . Still no estimator was ad- must be used.
dressed for the most demanded case: determination of for arbi- The problem with (4) or (5) arises instantly when the reference be-
trary models via real measurements and estimates with no reference havior is unknown as it usually is. If so, then let us substitute with
and cost equipment. and examine the MSV of the difference between the actual mea-
In this correspondence, we show that can efficiently be esti- surement and the relevant estimate . Substituting with (2)
mated via the mean square value (MSV) associated with real measure- gives us
ments and -variant estimates. In Section II, we discuss the estima-
tion of in a learning cycle. The -variant MSE and MSV are de- (6a)
rived in Section III employing the Kalman-like unbiased FIR (UFIR) (6b)
filter. The optimal memory estimation algorithm is discussed in Section
IV. Applications to the two-state polynomial and harmonic models are in which two last terms strongly depend on . Three limiting cases can
given in Section V and concluding remarks are drawn in Section VI. be considered:
• When , the FIR filter degree is equal
Manuscript received April 19, 2012; revised June 26, 2012 and January 23, to the number of measured points and the filter thus tracks each
2013; accepted January 23, 2013. Date of publication November 08, 2013; date of them. No noise reduction is provided and the error with
of current version January 13, 2014. The associate editor coordinating the re- its sign changed almost repeats measurement noise. Substituting
view of this manuscript and approving it for publication was Prof. Shuguang with makes (6b) near zero
(Robert) Cui.
F. Ramirez-Echeverria and Y. S. Shmaliy are with the Department of Elec-
tronics Engineering, Universidad de Guanajuato, Salamanca 36885, Mexico (7)
(e-mail: shmaliy@ugto.mx).
Amadou Sarr is with the Department of Mathematics and Statistics, Mc- • At the optimal point , two last terms in (6b)
Master University, Hamilton, ON L8S 4L8, Canada (e-mail: asarr@math.mc-
master.ca).
vanish by the orthogonality condition, the measurement noise
Color versions of one or more of the figures in this paper are available online dominates the estimation error, , and we get
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSP.2013.2290504 (8)

1053-587X © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
558 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 62, NO. 3, FEBRUARY 1, 2014

• With , the estimation error is mostly caused by large

bias. Accordingly, the first term in (6b) substantially dominates
and MSV asymptotically approaches MSE,

(9)

Transitions between these points can now be learned, to be neatly

seen further in Figs. 3(a) and 5(a). When , the estimate
bias is almost negligible. It rises slowly and approaches the estimate
standard deviation at when the filter slightly looses an ability to fit
the system owing to the process noise. Because in (6b) generates
and the latter monotonously decreases here owing to noise reduction,
the MSV is also monotonous. It rises from to Fig. 1. A generalized dependence on and of the -variant
and passes across with a minimum rate when is minimal. MSE of the th state FIR estimate.
Note that the rate of cannot be zero at due to the estimate bias
increasing. This analysis holds true irrespective of the model, because
FIR filters reduce the white noise variance as a reciprocal of and (16)
effect of bias is still small when .
Another rules dominate when , since the transition and ranges starting with . The auxiliary matrices are
strongly depends here on the estimate bias that, in turn, depends on
the model. Although is always concave on in stable filters, the , and .
estimate bias affecting its ascending right side can be either monoto-
nously increasing, as in polynomial models, or rising with oscillations, Before specifying optimally, by (12), the -variant MSE and
as in harmonic models. That means that with may not MSV need to be found.
obligatorily be monotonous and rapidly increasing. Accordingly,
may reach oscillating, even with a constant average rate, and A. -Variant MSE
have multiple minima. We thus come up with the following
For given by (12) and the estimation error , the
proposition that will further be justified in more detail employing the
MSE can be defined as
UFIR filter [8].
Corollary 1: Given (1) and (2) with zero mean and mutually uncor-
related white Gaussian and . Then the optimal memory for
FIR filtering of all of the measured states or the measured th state can
be estimated in a learning cycle by, respectively, (17)

(10) A generalized dependence of on is sketched in Fig. 1 for

different noise-to-noise ratios (NNRs) . A decrease in this
(11) function (dashed) is due to the ability of averaging to reduce the vari-
ance of white noise by the factor of . The minimum value of is typ-
ically limited with the number of states . When measurement noise
where is specified by (6a) and is the th component of is negligible, , denoising becomes unnecessary and estimation
. The minimization must be provided by increasing starting with is provided with . In an opposite case of a near determin-
until the first minimum. To avoid ambiguities while minimizing istic model, , the filter fits it over very long horizons requiring
the functions, the number of points in the average (6a) must be suffi- that makes denoising highly efficient.
ciently large. Also, and can be smoothed before applying Supposing that is equal to for any (Fig. 1), the minimum
(10) or (11). MSE can be found by letting in (17) and
Proof: A justification for this procedure is given above and will then solving the discrete-time algebraic Lyapunov equation [9]
be extended below.
(18)
III. -VARIANT MSE AND MSV
in which all matrices are real, is
It follows that, to specify by (4), (5), (10), or (11), the FIR filter
supposed to be stable, and
must be organized such that its memory covers all the measured points
is symmetric. For stable ,a
from zero to . In this regard, a convenient tool is proposed in [8] the
solution to (18) is known to be an infinite sum [9]:
Kalman-like UFIR estimator ignoring noise and initial conditions. The
relevant full-horizon time-invariant filtering algorithm (Table 1 with
in [8]) is as follows: (19)

(12)
which convergence depends on the model. Note that the bias correction
gain is associated in (12) with the noise power gain (NPG) which
where is the bias correction gain,
diminishes as a reciprocal of [4]. Even so, cannot be neglected
(13) in (19) with large , because of divergence.
Provided , by (17), can be estimated numerically using (4)
(14) or (5) by increasing from until or reaches a first
(15) minimum. Note that (19) agrees with the fact that when a model is
RAMIREZ-ECHEVERRIA et al.: OPTIMAL MEMORY FOR DISCRETE-TIME FIR FILTERS IN STATE-SPACE 559

deterministic ( and ) and the filter perfectly matches it,

then and .

B. -Variant MSV
Although can be estimated by minimizing (17), and are
commonly not fully known and the estimate of may thus not be
correct. If that is the case, then the MSV (6a) may serve better.
Employing (12), the MSV (6b) can be transformed by
, and

(20)

(21)

That brings (6b) to

(22)

where can be computed using (17). Again, we watch here for

the same picture. When , the bias correction is near unity,
, and (22) becomes near zero, as in (7). At ,
both and are small enough and (22) transforms to (8). Finally,
when is large and negligible. That leads to (9).
Let us now show that the slope of is always positive at and
prove that is concave and positive. The derivative
can be viewed as that gives us

Fig. 2. An algorithm for blind estimation of in a learning cycle for (10)

via measurements and estimates with no reference.

(23)
cause ambiguities in the determination of and long baseline mea-
At , the term in brackets of (23) vanishes by (18) and we have
surement would be required. The algorithm we propose (Fig. 2) implies
multiple short measurements and denoising by averaging. At the early
stage, such measurements , are conducted with length
(24) . Provided , the above-discussed full-horizon UFIR filter
(12)–(16) produces the estimates . Next, the MSVs
Now observe that and, by simple manipulations, (13) are calculated via as . The
can be transformed to -variant trace of the MSV is then computed via the average of partic-
ular MSVs as
(25)
(27)
representing the filter NPG see (18) in [10] . Because, the NPG di-
minishes in FIR structures once rises, both and are also
diminished by . That implies and we come up with a Because still may have some ripples around , extra de-
conclusion that (24) is always positive and so is the derivative of , noising can be organized with -point smoothing. In our algorithm,
we employ the -degree and -lag UFIR smoothing filter (42, [10])
(26)

(28)
On the other hand, since decreases with and reaches a min-
imum at , then increases with and, by (26), passes
across with a minimum and positive slope. Because the esti- in which and means a maximum integer smaller
mate bias forces MSV to converge to MSE, is concave as well than or equal to . The low-degree impulse responses
as MSE. Further minimization of by gives us that can , are specified in [10] by (52), (53), (59), and (71). Note that any
be said to be a crucial property of the MSV. However, since analytical smoother such as the Savitzky-Golay’s can be used here and and
solutions to (10) and (11) are not available in state space, efficient nu- must be set individually for each trace. Provided , the trace mini-
merical algorithms are required. mization can be obtained by (10) to produce finally .

IV. OPTIMAL MEMORY ESTIMATION ALGORITHM V. APPLICATIONS

It follows from (8) that, when , the vector Below, we test the algorithm (Fig. 2) by the polynomial and har-
always has larger noise than in . Ripples in may thus monic two-state models.
560 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 62, NO. 3, FEBRUARY 1, 2014

Fig. 4. Optimal found for the 2-state polynomial model: (a) by

and (b) by . Experiments depicted with (c) and (d) cor-
respond to sub Fig. (c) and sub Fig. (d), respectively. Sub Fig. (c) corresponds
to small system noise and sub Fig. (d) to large system noise.

Fig. 3. Effect of on the MSE and MSV in the 2-state polynomial model: (a)
errors as functions of and (b) derivative of MSV in the th
measurement, trace (bold) by (27), and trace (circles) smoothed by (28).
Experimental dependencies, by (3) and (6a), are dashed in (a).

A. One-Dimensional Tracking of a Two-State Model

In a simplest case of one-dimensional tracking, the two-state
object behavior with the first state measured can be described by (1)
and (2) if to set , and . We let

to have and with .

In the first experiment, we normalize distance, let s,
s, and and compute MSE by via (17) and MSV
using (22) with as shown in Fig. 3(a).
As can be seen, behaves exactly as was predicted in Section II.
To support this observation experimentally, was measured for
over points using (3) (circles) and
using (6a) (triangles). Then was calculated (boxes). Certainly,
the measurement fits very well the predicted values, although
measured is accompanied with ripples. Direct smoothing of in a
single cycle by (28) has appeared to be inefficient and multiple Fig. 5. Effect of on MSE and MSV in the 2-state harmonic model: (a) errors
measurements were conducted with . as functions of and (b) derivatives of MSE and MSV with estimated at
The multiply measured and a smoothed functions are shown first minima. Measurement is depicted with circles.
in Fig. 3(b). One infers that each , gives too rough
estimate of owing to ripples. In the average provided by (27)
with , noise is reduced essentially, but it still may cause ambi- same point as that predicted by in Fig. 3(a). At this
guities. Therefore, was smoothed by (28) with and point, computed via (19) also exactly corresponds
to be and the minimum of has finally appeared exactly at the to the relevant minimum in Fig. 3(a). Note that, in this example, (19)
RAMIREZ-ECHEVERRIA et al.: OPTIMAL MEMORY FOR DISCRETE-TIME FIR FILTERS IN STATE-SPACE 561

that has the same properties around as in the poly-

nomial model. Finally, the dependencies found in different
ways are shown in Fig. 6. They do not fit well each other, except for a
narrow region of . On the other hand, Fig. 5(a) re-
veals that MSE is not as sensitive to as in Fig. 3(a). The latter means
that rigorous determination of may be redundant for this model
and approximate values reasonably accurate. Such an approximation
(dotted in Fig. 6) was found as

where has happened to be the same, .

VI. CONCLUSION
The algorithm proposed in this correspondence is crucial for the es-
timation of optimal memory making the MSE in discrete-time state-
space FIR filters truly minimal. We have shown how to find optimal
memory via the -variant mean square value involving real measure-
Fig. 6. Optimal found for the 2-state harmonic model: (a) by , ments and estimates while ignoring any reference and noise statistics.
by (b) , and (c) experimental. An application to the polynomial two-state model has shown a very
good correspondence with the theory. Even in the worst case of the har-
monic model, the memory estimated experimentally has demonstrated
practical applicability.
converges slowly and the sum length should not be shorter than .
Next, we varied NNR and estimated by (4) and (10) as shown in
Fig. 4. Here, curves (a) and (b) trace very close to each other. Moreover, REFERENCES
the dependences suggest a simple empirical rule (dotted in [1] A. H. Jazwinski, Stochastic Processes and Filtering Theory. New
Fig. 4) York, NY, USA: Academic, 1970.
[2] A. M. Bruckstein and T. Kailath, “Recursive limited memory filtering
and scattering theory,” IEEE Trans. Inf. Theory, vol. IT-31, no. 3, pp.
440–443, May 1985.
[3] W. H. Kwon, P. S. Kim, and S. H. Han, “A receding horizon unbiased
FIR filter for discrete-time state space models,” Automatica, vol. 38,
where one can set . Experiment, however, reveals that fits no. 3, pp. 545–551, Mar. 2002.
actual only if is small. The effect is due to short time measure- [4] Y. S. Shmaliy, “Linear optimal FIR estimation of discrete time-in-
variant state-space models,” IEEE Trans. Signal Process., vol. 58, no.
ments and it almost disappears on a long baseline. 6, pp. 3086–3096, June 2010.
[5] P. S. Kim, “An alternative FIR filter for state estimaiton in discrete-
B. Two-State Harmonic Model time systems,” Digit. Signal Process., vol. 20, no. 5, pp. 935–943, May
2010.
The most inconvenient case for the estimation of is perturbed [6] Y. Kou, Y. Jiao, D. Xu, M. Zhang, Y. Liu, and X. Li, “Low-cost precise
by noise the harmonic model with its quasi periodic behavior. We sim- measurement of oscillator frequency instability based on GNSS carrier
observation,” Adv. Space Res., vol. 51, no. 6, pp. 969–977, 2012.
ulate it with [7] Y. S. Shmaliy, J. Muñoz-Diaz, and L. Arceo-Miquel, “Optimal hori-
zons for a one-parameter family of unbiased FIR filters,” Digit. Signal
, and uncorrelated zero mean white Gaussian and . Process., vol. 18, no. 5, pp. 739–750, Sep. 2008.
The noise was set with [8] Y. S. Shmaliy, “An iterative Kalman-like algorithm ignoring noise and
initial conditions,” IEEE Trans. Signal Process., vol. 59, no. 6, pp.
and and with . 2465–2473, Jun. 2011.
For , errors in the first state estimates are shown in Fig. 5(a) [9] T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation. Upper
Saddle River, NJ, USA: Prentice-Hall, 2000.
and, by , we have obtained as shown in Fig. 5(b). Then [10] Y. S. Shmaliy and L. J. Morales-Mendoza, “FIR smoothing of discrete-
smoothing with and minimization of by gave us time polynomial signals in state space,” IEEE Trans. Signal Process.,
closely related to in Fig. 5(a). Observing Fig. 5, one notices vol. 58, no. 5, pp. 2544–2555, May 2010.