Copyright e IFAC System Identification.

Copenhagen, Denmark, 1994

RECENT ADVANCES IN SIGNAL PROCESSING WITH

a-STABLE DISTRIBUTIONS

C.L. NIKIAS and M. SHAD

Signal and Image Processing Institute, Department of EE-Systems, University of

Southern California, Los Angeles, CA 90089-2564, USA

Abstract. Stable distributions share defining characteristics with the Gaussian

distribution, such as the stability property and central limit theorems, and in fact
include the Gaussian as a limiting case. They have been found to provide useful
statistical models in many disciplines. In this paper, we present some of the recent
advances in signal processing based on stable distributions.

Key Words. a--stable distributions, generalized central limit theorem, fractional

lower-order moments, covariations, minimum dispersion criterion, impulsive noise

1. SYMMETRIC a-STABLE the following power series [2):

DISTRIBUTIONS AND RANDOM
VARIABLES
Symmetric stable distributions belong to a more gen-
eral class of distributions called the dable distribu-
tions, which may be symmetric or asymmetric. For a
tutorial on the properties of general stable distribu-
tions, see [1]. In this paper, we will deal exclusively
with symmetric stable distributions.
A real random variable is symmetric a-stable
(Sa-S) if its characteristic function has the follow-
ing form
cp(t) = exp{iat - "Y I t IQ} (1)
where
-00 < a < 00, "Y > 0, 0 < a :5 2
a is called the characteridic erponent. It measures
the "thickness" of the tail of the distribution. Thus,
if a stable random variable is observed, the larger
the value of a, the less likely it is to observe val-
ues of the random variable which are far from its
central location. Note that the normal and Cauchy
distributions are both SaS with a =
2 and a =
1 re-
spectively. For convenience, all stable distributions
are assumed to be non-Gaussian in this paper. "Y is
a scale parameter, called the di"per"ion. It behaves
like the variance. a is a location parameter. It is
the mean when 1 < a :5 2 and the median when
o < a :5 1. In this paper we shall assume a O. = may justify the use of stable laws to describe non-
The probability density function of the standard-
=
ized Sa-S random variable (a = 0, "Y 1) is given by
·This work was supported by the Office of Naval Re-
search under contract N-OOOl4-91-J-1l24
!Q{X) =
1 (_I)k-1 k
-1-1
~x.
L
k'
oc
[(ak + 1)lxl- Qk sin[~)
2
k=1
O<a<1
l<a:52
(2)
The standardized SaS density function for some
values of a is shown in Figure· 1. Observe that non-
Gaussian SaS distributions have many features of
the Gaussian. They are smooth, unimodal, symmet-
ric with respect to the median and bell-shaped. But,
they have heavier (longer) tails than the Gaussian.
The smaller a is. the longer the tail.
The real r.v.'s XI, . ... X n are jointly SaS if and
only if all the linear combinations 0lX1 +o~X~+··+
anXn are SaS. SaS random processes are defined
similarly.
One of the remarkable properties of the stable dis-
tribution is the Generalized Central Limit Theorem.
This theorem states that if the sum of any LLd ran-
dom variables with or without variances converges
to a distribution by increasing the number of vari-
ables, the limit distribution must be one of the sta-
ble laws [3]. Thus, non-Gaussian stable distributions
arise as sums of random variables in the same way
as the Gaussian distribution. This suggests that we
Gaussian phenomena as we justify the use of Gaus-
sian laws.
Another useful property of the stable distribution
is the linear stability property: If Xl, )(~, ... , X n are
independent and follow a stable law with the same

65
a, then E7=1Xj follows the same law, except for a
possible location and scale change. This is particu-
larly relevant when stable distributions are used to
model impulsive noise.
2. FRACTIONAL LOWER-ORDER
MOMENTS
The main cause of the different behaviors of the
Gaussian and (non-Gaussian) stable distributions is
their tails. It can be shown (2) that for a non-
Gaussian a-stable random variable X with zero loca-
tion parameter and dispersion "'t, its density function
10.., satisfies
lim Ixlo+l/o..,(x) = C(a, "'t)
1%1.... 00
where
C(a,"'t)="'t~r(a)sin~
71' 2 [X, Y]o
is a positive constant depending on a and "'t. Hence,
the decay rate of the tails of a 50'S density is asymp-
totically on the order of X-O-I, much slower than the
exponential decay rate of the tails of the Gaussian
density.
An important consequence of (3) is the non-
uiJtence of the second-order moment of stable dis-
tributions, except for the limiting case a = 2 (2).
Specifically, let X be an a-stable random variable.
If 0 < a < 2, then
where C(p, a) depends only on a and p. Thus, the
distance between two 50'S random variables mea-
sures the pth order moment of the difference of these
two random variables. In the case of a = 2 this dis-
tance measures the variance of the difference, which
is consistent with the second-order moment theory.
An important observation is that all the lower order
moments of a 50'S random variable are equivalent
in the sense that the pth and qth moments differ by
a constant factor independent of the 50'S random
variable for all p, q < a.
Although covariances do not exist for 50'S ran-
dom variables due to the lack of finite variances, a
quantity called covariation plays an analogous role
(3) for 50'S random variables.
Let (X, Y) be jointly 50'S. The covariation
[X, Y)o of X with Y is defined as
(4) o E(Xy<P-I»
= IIYllo E(lYI P ) (8)
where x<o> = xlxl o - I . It can be proved that the
above quantity is independent of the choice for p as
long as p < a [2].
Some of the useful properties of covariations [4. 5)
are:
1. The covariation [X, Y]o is linear in X: if
XI, X 2 , Y are jointly 50'S then

EIXI P = 00, if p ~ a for any a and b.

and 2. [X, Y)o is not linear with respect to the second
EIXI P < 00, if 0 $ p < a. variable Y. But it does possess the following
pseudo-linearity property with respect to Y: if
If 0'= 2, then Y I , Y2 are independent and X. Y I , Y2 are jointly
50'S, then
EIXI P < 00, for all p ~ o.
a<o-I>[X, Ydo+
In other words, every stable distribution with char- b<o-I>[X, Y2 ]o
acteristic exponent 0 < a < 2 has finite absolute (10)
moments E(lxI P ) of order p for 0 < P < a. All ab-
3. If X, Y are independent and jointly 50'S, then
solute moments of order p ~ a are infinite. Thus
for 0 < a $ 1 50'S random variables have no first [X, Y)o =0
or higher order moments; for 1 < a < 2 50'S ran-
dom variables have the first moment and all the frac- while the converse is not true in general.
tional moments of order p where p < ai for a = 2, 4. For any jointly 50'S random variables X, Y:
all moments exist. Hence, stable signal processing is
mainly based on the moments ElxIP,O < p < a $ 2. E(XIY) = >'XY Y (11)
These moments are called fractional lower-order mo-
ments (FLOM's). where
Let X be a 50'S random variable with dispersion
"'t > O. Define the norm of X as

IIXllo = { ~* 1$0'$2
0<0'<1
(5) is the covariation coefficient of X with Y.

3. PARAMETER ESTIMATION OF AR
If X, Y are jointly 50'S, the distance between X and STABLE PROCESSES
Y is defined as As an application of the covariations, let us consider
the 50'S process generated by the following AR sys-
d(X, Y) = IIX - Yllo (6) tem
It can be shown [2) that for any 50'S random variable X(n) = alX(n - 1) + ... + apX(n - p) + U(n) (13)
X with zero location parameter, and any 0 < P < a:
where {U(n)} is a sequence ofi.i.d 50'S random vari-
(7) ables of characteristic exponent a and dispersion "'tu.

66
The poles are assumed to be inside the unit circle. for 0 < p < er. Since L(X(t), t E T) is a Banach
We are interested in the problem of identifying the space, Y always exists and is unique for 1 < er < 2.
AR coefficients 0 I, ... ,Op from the observation of It is obtained by a metric projection of Y onto the
output X(n). convex Banach space L(X(t), t ET). For 1 < er < 2,
Let A(l) be the covariation coefficient of X(n + I) Y is also uniquely determined by:
with X(n). Define
[X(t), Y - Y]o = 0 for all t E T (18)

This is analogous to the orthogonalitll principle used

extensively in linear estimations of second-order pro-
cesses. Unfortunately, this equation can not be
solved analytically.

and 4.2. Adaptive Wiener Filters for Stable Pro-

cesses
A(O) A( -1) A(1 - p) ] Adaptive solutions of the linear estimation problems
A(I) A(O) A(2 - p) for stable processes are much easier to implement be-
C= . (14) cause they do not require closed-form solutions. The
~(p -
[ dispersion of the estimation error is usually a convex
1) A(P - 2) A(O) function of the parameters. So numerical methods,
such as stochastic gradient methods, may be used to
Then the coefficients of the AR system can be found
find the parameters by minimizing the dispersion of
by solving the following system of linear equations:
the error function.
(15) Let us consider designing an FIR filter with
Ca=p
an input consisting of a stationary SerS process
{u(n), n = 0, 1, ...}. The problem is to choose
Equation (15) is a direct generalization of the
Yule-Walker equation for the case er = 2. The co- the tap weight wo, WI, ... , WM-I such that the out-
put of the filter is as close to the desired response
=
variation coefficients p(n), n 0, ±1,' ", ±(p - In
d(n) as possible. Here we assume d(n) and u(n)
can be estimated from observations, see [I, 2).
are jointly SerS. Specifically, we would like to find
4. LINEAR ESTIMATIONS OF STABLE wo, WI , ... , W M - I such that the dispersion of the er-
PROCESSES ror
M-I
4.1. The Minimum Dispersion Criterion e(n) = d(n) - L Wku(n - k) (19)
For the linear estimation problem of stable processes, k=O
the familiar minimum mean-squared error (MMSE) is minimized. The cost function is thus given by
criterion is no longer applicable, due to the lack of
M-I
finite variances. But the concept of MMSE criterion
can be easily generalized to stable processes. Specifi- J = Ild(n) - L Wku(n - k)lIo (20)
cally, the minimum di~per~ion (MD) criterion is used k=O
in discussing linear theory of stable processes. Under
Since the norm of a SerS random variable is propor-
the MD criterion, the best estimate of a SerS random
tional to the usual Lp norm of the random variable
variable in the linear space of observations is the one
for any 0 < p < er, an equivalent cost function is
that minimizes the dispersion of estimation error.
given by
Recall that the dispersion of a stable random vari-
able plays an analogous role of the variance. For ex- M-I
ample, the larger the dispersion, the more spread the
stable random variable around the median. Thus, by
J = E(ld(n) - L Wku(n - kW) (21)
k ..O
minimizing the dispersion we minimize the average
magnitude of estimation error. From (7), we see that where 0 < p < er. By using stochastic gradient
the MD criterion is also equivalent to minimizing the method, the coefficients can be adaptively found
FLOM's of estimation errors. This result is not sur- through LMS-like algorithms. In particular, we pro-
prising since the Lp norms for p < 2 are well known pose the following LMP (Least Mean P-norm) algo-
for being robust against outliers such as those that rithm.
may be described by stable laws. LMP Algorithm: Fix 0 < p < er
Let Y, {X(t), t E T} be jointly SerS. Under the 1. Filter output:
MD criterion, the linear estimation problem of sta-
ble processes can be formulated as follows: Find an y(n) = wT(n)u(n) (22)
element Y in the linear space L(X{t), t ET) of the
observations such that 2. Estimation error:

IIY - Yllo = ZEL(X(I),IET)

inf IIY - ZII (16) e(n) = d(n) - y(n) (23)

3. Tap weight adaptation:

or equivalently
w(n + 1) = w(n) + J..IU(n)le(n)IP-lsign(e(n))
ElY - YI P = inf
ZEL(X(I),IET)
ElY - ZIP (17) (24)

67
For p = 1 the above algorithm is the same as the
signed LMS algorithm.
Figure 2 shows the performances of the LMS and
LMP algorithms (p=1), applied to a first-order AR
stable process with Ot = 1.5. It is obvious that LMP
converges much faster than the LMS algorithm.
5. IDENTIFICATION OF LSI SYSTEMS
We consider a linear shift-invariant (LSI) system
with a SOtS stationary process X(n) as the input.
Assume the impulse response of the system is hen).
The output yen) is then given by the following con-
volution
00
yen) = 2::: h(k)X(n - k) (25)
k=O
Observe that the output yen) is also a SOtS process
and yen) and X(n) are jointly stationary.
In the following we show that if we know the co-
variation sequence Cu(n) and the cross covariation
CI/Z (n) sequence, we can identify the system (that
is, find h (n)). In fact by taking covariations of both
sides with X(n - m), one has the following relation
00
CI/Z(m) = 2::: h(k)Cu(m - k) (26)
k=1
We now take the Fourier transform of both sides to
get
=
CI/Z(w) H(w)Czz(w) (27)
where CI/z(w), H(w),Cu(w) are the Fourier trans-
forms of CI/z(m), hen), C u (n) respectively.
So in theory one may recover the impulse response
hen) and/or its Fourier transform H(w) from the co-
variation sequences through
(28)
Note, however, that Cu(w) is just the Fourier trans-
form of the covariation sequence of X (n) and has no
power spectrum meaning.
For a similar development for continuous-time sys-
tems, see [5].
6. SOtS MODEL FOR IMPULSIVE NOISE
Most of the natural and man-made impulsive in-
terferences may be considered as the results of a
large number of spatially and temporally distributed
sources which produce random noise pulses of short
duration. The noise received at any location is the
superposition of these pulses, and hence depends on
the spatial and temporal distributions of the indi-
vidual sources and the propagation of their pulses to
the receiver. On the basis of these observations, the
filtered-impulse mechanism [6, 2] assumes that the
received interference at any instant of observation
can be represented by a random variable
where Vj (t, x,!.) is the pulse waveform emitted from
a noise source iocated at x, and ! represents a set
of random parameters that describes the waveform
scale, duration and etc. Under the assumptions that
68
the sources emit their waveforms independently ac-
cording to the Poisson distribution in time and space
and that the medium has an inverse-power propaga-
tion law, the instantaneous noise amplitude has been
shown to obey a SOtS distribution [6, 2].
In the case of narrowband reception, the joint
distribution of the in-phase and quadrature compo-
nents of the received narrowband noise is i!otropic
ao!table, whose characteristic function is given by
where 0 < 0' ::; 2 and "f > 0 are again the charac-
teristic exponent and dispersion, respectively. Note
that the bivariate isotropic Gaussian (0' = 2) and
Cauchy (0' = 1) distributions are two special cases
of the isotropic stable distribution. Also note that
the two marginal distributions of the isotropic stable
distribution defined by (29) are So'S with the same
parameters (-y, 0').
From (29), one can show that the random phase is
uniformly di!tributedin [0,271"] and is independent of
the envelope. The density of the envelope, on other
hand, is given by
100
f(a) = a sexp(-'YsO)Jo(as)ds, a;::: O. (30)
Note that when 0' = 2, one obtains the familiar
Rayleigh distribution for the envelope.
Two examples are given for comparing the So'S
model with experimental data. (Note that the am-
plitude probability distribution (APD) is defined as
P(lXI > x).) Fig. 3 compares the So'S model with
experiment for typical ELF noise. The measured
points for moderate-level Malta ELF noise in the
bandwidth from 5 to 320 Hz are taken from [7]. Since
the ratio of bandwidth to center frequency is not
small at ELF, the APD of wideband SOtS noise is
used. The characteristic exponent 0' and the disper-
sion 'Y are selected to best fit the data. Fig. 4 shows
wideband impulsive noise of fluorescent lights in a
mine shop [8]. In both cases, the agreement between
theory and experiment is very good.
Another example of So'S noise is the impulsive
noise observed on telephone lines. These impulsive
interferences are caused by several sources includ-
ing lightning, switching transients, and accidental
hits during maintenance work. A detailed empirical
study shows that noise on several telephone lines can
be adequately modeled by So'S distributions with
characteristic exponents close to but definitely less
than 2 [9].
7. CONCLUSION
Stable distributions can become useful statistical
tools for researchers in signal processing because of
their attractive theoretical properties. Yet, they
have attracted little attention in the signal process-
ing literature [9, 5]. There are historically two rea-
sons for this. The first reason is the fact that sta-
ble distributions do not have explicit expressions for
their densities or distributions. Computations of
density and distributions functions must rely on infi-
nite series expansions or numerical integration of the
Fourier inversion formula based on the characteris-
tic function. However, with today's computational
 0.7
power, these numerical calculations are inexpensive
to carry out. The second reason is the lack of fi- 0.6
nite variance for non-Gaussian stable distributions.
Since the second-order moment or the variance is 0.5
often associated with the concept of power, it has
been argued that infinite variance is inappropriate 0.•

in almost any signal processing context. We believe

that this kind of reasoning is superficial and inap- 0.3

propriate. As it is pointed out in [10], with such

reasoning, the Gaussian distribution would not be a 0.2

physically appropriate assumption because it is un-

0.1
bounded. Gaussian models are adequate for mod-
eling many types of observed data. In some cases,
especially when outliers or heavy tails appear in the
observed data, an infinite-variance stable model may
0
~ ..
10)
have to be used [10].
There is still a lot research to be done to better
understand stable models and to develop algorithms 0.D35

for specific signal processing applications. For ex-

0.D3
ample, adaptive signal processing algorithms can be
developed and applied to equalization, noise can-
0.0"...5
cellation, underwater acoustic signal processing and
signal enhancement, under non-Gaussian stable en-
0.D2
vironments. These schemes can be expected to ex-
hibit much faster convergence and be more robust O.OIS
than other existing methods based on the Gaussian
assumption. One could also investigate signal detec- 0.01
tion, parameter estimation and classification prob-
lems with stable distributions and fractional lower 0.005
order moments, especially for signals and/or noise
with infinite or very large variances. Filtering of im- 0
~-lj J.5 4.5 S.S
pulsive noise and nonlinear stable signal analysis are
Ib)
yet two more areas worth investigating.
Figure 1. (a) Graphs of the standard SaS density func-
REFERENCES tion corresponding to the values a =
2.0,1.5,1.0 and
0.5; (b) tails of the densities in (a).
[1] M. Shao and C. L. Nikias, "Signal processing
with fractional lower order moments: Stable
First-order probability models of the enve-
processes and their applications," IEEE Proc.,
lope and phase," Report OT 76-86, Office of
July 1993.
Telecommunications, 1976.
[2] M. Shao, Signal Proceuing Baud on Stable Di$- [9] B. W. Stuck and B. Kleiner, "A statistical anal-
tribution$. PhD thesis, Department of Electrical ysis of telephone noise," Bell SY$t. Tech. J.,
Engineering, University of Southern California, vol. 53, no. 7, pp. 1263-1320, 1974.
August 1993.
[10] B. W. Stuck, "Minimum error dispersion liner
[3] L. Breiman, Probability. Reading, MA: filtering of scalar symmetric stable processes,"
Addison-Wesley, 1968. IEEE Tran$. Automat. Contr., vol. AC-23,
[4] G. Miller, "Properties of certain symmetric no. 3, pp. 507-509, 1978.
stable distribution," J. of Multivariate Anal.,
vol. 8, pp. 346-360, 1978.
[5] S. Cambanis and G. Miller, "Linear problems in
pth order and stable processes," SIAM J. Appl.
Math., vol. 41, pp. 43-69, Aug. 1981.
[6] M. Shao and C. L. Nikias, "On symmetric stable
models for impulsive noise," Technical Report
USC-SIPI-231, University of Southern Califor-
nia, February 1993. also, submitted to IEEE
Tron$. Commun.
[7] J. Evans and A. S. Griffiths, "Design of a San-
guine noise processor based upon world-wide ex-
tremely low frequency (ELF) recordings," IEEE
Tron$. Commun., vol. 22, pp. 528-539, 1974.
[8] D. Middleton, "Statistical-physical models of
man-made and natural radio noise, Part 11:

69
 0.9
.....~-~-----------
Q.I ./
.,.".,
0.7 ....'

~ 0.6 //
.j 0.'
...
~ 0.4

0.)

0.2

01/
oL._ _ __ ---'

~
~
o ~ ~ ~ ~ ~ ~

Figure 2. Transient behavior of tap weights In LMS

and LMP with 0 1.5 =

._•. CALCIlUlEl AID: AlJ'IIA,I.'Z. GAMIo4kQI4

o WEASlIlED MOOEJ:ATE-L£VfL MALTA NOISE

i 0

1 ·10

.2Il

.)0

-4OQ~0""'Q~''''''''I---:'O--:20'-:)o'::'-.4O'''')o~6O~70'-IO-=--~''--9S~-:::'''''--9O~--J
'erClMl oIT~ 0nAiM..... ~

Figure 3. Comparison of a measured APD of ELF at-

mospheric noise with the SoS model

·20
._.. CALCIlUTED APO; AUtlA , I...... GA/oO(A , 0.00lllIlIl6
o NEA.SUaED NasE [N WINE SHOP

.)0

'lix'iO, 0.010." , 10 20)0 40)0 60 70 10 .. 9S III 90

'~olT. . ~ • • ~

Figure 4. Comparison of a measured envelope APD of

fluorescent lights in a mine shop office with the SoS
model

70