ADAPTED FILTER BANKS IN MACHINE LEARNING: APPLICATIONS IN BIOMEDICAL

SIGNAL PROCESSING

D. J. S t r a ~ s s ' ~W~ Delb3,

- ~ , J. .lung3, and I? K. Plinkert3

'Key Numerics, Saarbriicken, Germany

'Institute for New Materials, Saarbriicken, Germany
3Saarland University Hospital, Homburg/Saar, Germany

ABSTRACT on the data have been suggested for feature extraction, e.g.,
see [6, 71. Data dependent schemes mainly rely on an ad-
The theory of signal-adapted filter banks has been devel-
justment of the decomposition tree, see [8] for a comparison
oped in signal compression in recent years and only rarely.
of several schemes. The local discriminant basis algorithm
be applied to other applications fields such as machinelean-
(LDB) [9] is a well accepted scheme which relies on the best
ing. In this paper, we propose lattice structure based signal-
basis paradigm. We extend this scheme to the construction
adapted filter banks and time-scale atoms, respectively, for
of more powerful morphological LDBs (MLDBs) using the
the construction of morphological local discriminant bases
lattice srrucrure.
and hybrid wavelet-support vector classifiers. The first men-
We also present hybrid wavelet-support vector classi-
tioned method is a more powerful construction of the re-
fiers where we apply an adaptation of wavelet~decomposi-
cently introduced local discriminant bases algorithm which
tions which is tailored for the recently introduced support
employs, additionally to the conventional wavelet-packet
vector machine class$ers with radial basis functions as ker-
tree adjustment, an adaptation of the analyzing time-scale
nels. This adaptation strategy is well motivated from the
atoms. The latter mentioned method utilizes adapted wavelet
paradigm of large margin classifiers [IO] and allows for an
decompositions which are tailored for support vector clas-
optimization of the representation of the data before solving
sifiers with radial basis functions as kernels. For both meth-
the quadratic programming problem of support vector ma-
ods, we present applications in biomedical signal process-
chines by including a priori knowledge of the pattern recog-
ing.
nition task.

1. INTRODUCTION
A general issue when applying wavelet like frequency de-
compositions is the design of appropriate hasis functions
and filter banks, respectively. The design of signal-adapted
filter banks by means of energy compaction is an active and
ongoing field of research [ 1,2,3,4]. Often these approaches
are closely related to the design of optimal wavelets for sig-
nal compression.
In this paper, we employ some of the ideas developed
in this area for machine leaming where we want to model
some dependency between an input and an output space
given a finite number of associations. The major difference
between our approach and the design of compaction filter
is that we apply other adaptation criteria than energy com-
paction which is not necessarily the best criterion for signal
discrimination [5].
In particular, we present a wavelet packet feature extrac-
tion scheme. Wavelet based methods with no special focus
This work has been supported by the MED-EL Deulschland GmbH,
Stamberg, Germany.
2. ADAPTED WAVELET PACKET
DECOMPOSITIONS
Let Ho(z) := C k E Z h ~ [ k ] tbe- kthe z-transform of the
analysis lowpass filter and H l ( z ) := & Z h l [ k ] ~ - k the
t-transform of the analysis highpass filter of a two-channel
paraunitary filter bank with real-valued filter coefficients
and at least one vanishing moment of the high pass filter,
i.e., we have that H1(1) = 0. The polyphase matrix Hpl(z)
of the analysis bank of such filter banks can be decomposed
as follows
and finally the space P K := {fi = (290,. . . ,29~-1) : 29k E
[0, T)} can serve to parameterize all two-channel parauni-
tary filter banks with at least one vanishing moment of the

0-7803-7663-3/03/$17.00 02003 IEEE VI - 425 ICASSP 2003

highpass filter, see [ I I], [12, Theorem4.71, and [13] forde-
tailed discussions. We will use (8) to denote the parameter-
ization if necessary. We use K = 2 in ( I ) in the application
section.
For the implementation of wavelet packet decomposi-
tions, twMhannel paraunitary filter banks are arranged in
a binary tree of decomposition depth J , see [14, 151 for an
introduction of wavelet packets. For the decomposition of
signals x E e2, we define the projections onto the orthog-
onal wavelet packet spaces n;,k = Oj+~,zk O ; + I , ~ ~ + I
(OO,O:= ea, ( j , k) E Z, where Z specifies the binary de-
composition tree) by Pzk : t2 -+Oj,k, with the following
expansions
~j,kx= ~j,k[r~IqTk,
mtz
Yj,k[4 = (X,$k)P,
where qTk are the wavelet packet basis functions. In prac-
tice our signals are of finite length d. Here we will exclu-
sively deal with signals which length d is a power of 2.
For this, we define the maximal decomposition depth by
J := log, d and the sets of indices := {O, 1,.. . ,Z J - 3 -
l}. Then we have the finite wavelet packet bases Bj,k :=
{qTk : m E T j } .
3. MORPHOLOGICAL LOCAL DISCRIMINANT
BASES
Suppose now that we are given L classes A! ( 1 = 1,. . . ,L )
of Mi training patterns, respectively, i.e., At := {xi,! E
lRd : i = 1,.. . , M ! } . We denote the expansion coeffi-
cients (2) ofthesepattems by yi,i,j,k = ( g i ~ , j , k [ m ]=) ~ ~ ~
( ( x < ~ , q ; ~ ). )F ~
o r~
(j~
, k ) E Z a n d m E 7,,thetime-
scale energy map Tl of class 1 is defined by
i=1
Now the LDB algorithm can be summarized as follows [9]:
LDB Algorithm
Choose a dictionary and an additive discriminant mea-
sure V which may be, e.g.,the e2 distance or the rel-
ative entropy, see [9].
1. For every class 1 = 1,. . . , L construct the time-scale
energy map T 1 .
2. Fork = 0 , . . . ,2-' - 1 set A J , :=
~ BJ,~
and
A J ,:=
~ 2) (TI(J, k , m ) , . . . ,T L ( Jk, m ) ) .
Vl€TJ
3. F o r j = J - 1 , . . . , O a n d k = O , . _ . 2, ' - l d e t e r -
mine the "best subset" A ; , k by the following rule:
Set A;,k := CmET (TI(j,k m), . . . , T L ( k, ~ 4).,
If A;,k 2 A;+i,zk + A;+i,zk+i
then Aj,k := B j , k
e k e Aj,k := Aj+i,znUAj+i,zn+i andAj,k := Aj+i,zk+
Aj+i,zk+i.
Output: Ao,o
The basis Ao,o is called the local discriminant basis
(LDB). The discriminatory power of this basis is measured
by A o , ~ . By the considerations in the previous section,
each angle vector I9 E P K defines a different dictionary
D ( 8 ) . The discriminatory power of the dictionary D(I9)
can be measured by Ao,o(8)computed in the LDB algo-
rithm. Thus the MLDB is given by
(2)
B = arg max A~,~(B). (4)
UEPK
This problem can either be solved by a hypercube evalu-
ation, parameter space reduction, or genetic algorithms, see
[13, 161 fordetails.
4. HYBRID WAVELETSUPPORT VECTOR
CLASSIFIERS
We only present our adaptation strategy here and refer to
[IO] for of introduction of SVMs.
Let K : X x X -+ R (Xis a compact subset of Rd) be
a positive definite symmetric function in L 2 ( X x X).For a
given K , there exists a reproducing kernel Hilbert space
3 1 =~span {IC(?., .) : j , E X}
of real valued functions on X with inner product determined
, K(%,x)whichhastherepro-
by ( K ( f , x ) , K ( x , x ) ) . ~=
ducing kernel K , i.e., (f(.),K(f;)).~, = f(i) (f E
X K ) . By Mercer's Theorem, the reproducing kernel K can
be expanded in a uniformly convergent series on X x X
(3) m
K(X>Y)= C%3(Oj(X)3Dj(Y)> (5)
j=1
where qj 2 0. We restrict OUT interest to functions K that
arise from a radial basis function (RBF) such that
K(x,Y)= k(llx - YIIz)
, where I I . 112 denotes the Euclidean norm on I t d . We intro-
duce a so+alledfeature map Q : X +t2 by
= (fiP;(.));GN'
Let e* denote the Hilbert space of real valued quadratic
summable sequences. By (3,we have that Q(x)(x E X)
VI - 426
is an element in e2 with ll@(x)11$ = C,”=lqjip:(x) = 5. APPLICATIONS
K ( x , x) = k ( 0 ) .We define thefeature space FK c e* by
the e2%losure of all finite linear combinations of elements 5.1. MLDB Feature Extraction
Wx) (x E 4 An auditory evoked response is an response within the audi-
F K = span{@(x): x E X } . tory system that is produced or evoked by sounds (an audi-
tory or acoustic stimuli). Of a particular interest is the audi-
Then FK is a Hilbert space with I I 1 1 = ~11 . 1~ tory brainstern response (ABR), that is, the auditory evoked
response that stems from the brainstem. The detection of
4.1. Adaptation in FK discriminating features between the binaurally (stimulus to
both ears) evoked ABRs and sum of the monaurally (stim-
For a fixed waveform, x we define the function ulus to one ear) evoked ABRs is of diagnostic interest [19].
l 9 P
For a total of ten patients, we have shown the the discrim-
= (llYBllL,... >llYNll@) (6) inant power of MLDBs and LDB, measured by for
the individual patient in Fig. 1. The LDB was constructed
where yn (n = 1 , . . . , N ) are a number of N pre-selected
using the Daubechies wavelet with 6 filter coefficients. The
subbands of a wavelet packet decomposition, e.g., the sub-
better performance of the MLDB is clearly noticeable, see
bands of an octave banddecomposition. We normalize< ,(29)
[I61 for a connection with a classifier.
by its concentration in e’. This feature vector carries the
multilevel concentration of the waveform x which is ro-
bust against local instabilities in time [13]. To gain sbift-
invariance of this feature vector, frame decompositions us-
ing nonsubsampled filter banks [I41 can be applied instead
of orthogonal decompositions which are shift-variant [17].
SVMs rely on the optimal hyperplane classificafion [lo]
in the feature space FK. Thus we have to ensure that our ex-
tracted features are mapped to far apart points in F K before
Fig. 1. MLDB and LDB performance
solving the quadratic programming problem of the SVM.
For a given training set

A ( 8 ) := { ( S i ( 2 9 ) , Y i ) E x c Rd x {-1,1} : i = 1 , . . . , M } ,
we denote sets of indices i E { 1, ,M } with yi = 1 and
yi = -1 by M+ and M - , re2pectively. Consequently, for a
large margin, we try to find I9 such that

8 = argm={
BC’F min
iEM+,jEM- /I*(<~(@))- w ~ ( I ~ ) ) I I ; ~ } .
(7)
. ~
..........~
..
It follows that
&OF “.,!RI RRF -ling
Il@(€i(8))
- @(€j(19))ll;x =
Fig. 2. SVMs and hybrid-wavelet SVMs
W O ) - 2k ( I I E i ( f v -€j(19)llz)
We suppose that k(.) is monotonely decreasing in 1 ‘1. Then
(7) can be rewritten as 5.2. Hybrid Classification
The rate independent discrimination of endocardial electro-
a = a r g mBE?
=( i € Mmin
+,j€M- /IE~(I~)-FJI~)II~}. grams (EEs), i.e., bioelectric signals which stem from the
inner heart, representing a physiological and a pathologi-
Now we have an optimization problem in the original space cal rhythm is of diagnostic interest [ZO]. For an applica-
X instead of the feature FK which is not explicitly given. tion of the presented hybrid wavelet-support vector clas-
An approach to gain an approximated solution to problems sifier (Gaussian RBF, and the suhbands of an octave hand
of this type is given [IS]. decomposition in (6)) to the EEs from ten patients with a

VI - 421
physiological and a pathological rhythm, the number of sup-
port vectors found as well as the error rate is shnwn in Fig.
2 using a training set of 8 beats, and a test set of 16 beats
from the physiological and pathological rhythm for the indi-
vidual patient. The results for SVM applied to time domain
waveforms are also shown. A much better performance of
the hybrid-wavelet support vector classifier in terms of the
number of support vectors found and the error rate is clearly
noticeable (note that we expect a better generalization if we
have less support vectors [lo]). See [IS] for an application
using shift-invariant frames.
6. CONCLUSIONS
We have presented two approaches in computational intel-
ligence using signal-adapted filter banks. In particular, we
constructed morphological local discriminant bases where
we adapted the morphology of the analyzing atoms addi-
tionally to the tree adjustment. We also presented hybrid
wavelet-support vector classifiers where the theoretically
well founded support vector machines allowed the optimiza-
tion of the representation of the data for the subsequent clas-
sification. The efficiency of each approach was verified in
two applications. We conclude that signal-adapted filter
banks are a sensible approach in machine learning.
7. REFERENCES
P. H. Delsarte, B. Macq, and D. T. M. Slock, “Signal
adapted multiresolution transforms for image coding,”
IEEE Trans. on Information Theory, vol. 38, pp. 897-
903,1992.
M. K. Tsatsanis and G. B. Giannakis, “Principal com-
ponent filter banks for optimal multiresolution analy-
sis,” IEEE Trans. on Signal Processing, vol. 43, pp.
1766-1777,1995.
P. P. Vaidyanathan, “Theory of optimum orthonormal
filter banks,” IEEE Trans. on Signal Processing, vol.
46, pp. 1528-1543,1998.
P. P. Vaidyanathan and S. Akkarakaran, “A review of
the theory and applications of principal component fil-
ter banks,” J. ofAppliedandComputationa1Harmonic
Analysis, to appear.
B. D. Ripley, Pattern Recognition and Neural Net-
works, Cambridge University Press, Cambridge;
1996.
S. Pittner and S. V. Kamarthi, “Feature extrac-
tion from wavelet coefficients for pattern recognition
tasks,” IEEE Trans. on Pattern Recognition and Ma-
chine Intelligence. vol. 21, pp. 83-88, 1999.
VI - 428
[7] L. J. Trejo and M. J. Shensa, “Feature extraction of
event-related potentials using wavelets: An applica-
tion to human performance monitoring,” Brain and
Lunguage, vol. 66, pp. 89-107.1999.
[8] G. Rutledge and G. McLean, “Comparison of several
wavelet packet feature extraction algorithms,” Submit-
ted to IEEE Trans. on Pattern Recognition and Ma-
chine Intelligence, 2000.
191 N. Saito and R. R. Coifman, “Local discriminant
bases,” in Wavelet Applications in Signal and Image
Processing 11, Proc. SPIE 2303, A. F. Laine and M. A.
Unser, Eds., July 1994, pp. 2-14.
[lo] V. Vapnik, The Nature ofStatistical Learning Theory,
Springer, NY, 1995.
[ I l l P. P. Vaidyanathan, Multirate Systems and Filter
Banks, Prentice Hall, Englewood Cliffs, NJ, 1993.
[ 121 G. Strang and T. Nguyen, Wavelets and Filter Banks,
Wellesley-Cambridge Press, Wellesley, MA, 1996.
[I31 D. J. Strauss, AdaptedFilterBanksandSupporr Vector
Architectures: Hybrid Approaches to Machine Leam-
ing, Logos-Verlag, Berlin, Germany, 2002.
[I41 M. Vetterli and J. KovaEeviC, Wavelets and Subband
Coding, Prentice-Hall, Englewood Cliffs, NJ, 1995.
1151 M. V. Wickerhauser, Adapted Wavelet Analysisform
Theory to Software, A. K. Peters, Ltd., Wellesley, MA,
1994.
1161 D. J. Strauss, G. Steidl, and W. Delb, “Feature ex-
traction by shape-adapted local discriminant bases,”
Signal Processing, 2002, In press.
[I71 E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and
D. J. Hegger, “Shiftable multiscale transforms,” IEEE
Trans. on Information Theory, vol. 38, pp. 587-608,
1992.
[I81 D. J. Strauss and G. Steidl, “Hybrid wavelet-support
vector classification of waveforms,” J. of Computa-
tional and Applied Mathematics, 2002, In press.
[I91 W. Delb, D. J. Strauss, G. Hohenberg, and K. P. Plink-
ert, “The binaural interaction component in children
with central auditory processing disorders,” Intema-
rional Journal ofdudiology, 2002, In Press.
[20] M. M. Shieh, L. Clem, L. Malden, Pixley A., and
E. Fain, “Improved supraventricular and ventricu-
lar tachycardia discrimination using electrogram mor-
phology in an implantable cardioverter defibrillator,”
G. Ital. Cardiol., vol. 28 (supplement I),
pp. 245-251,
1998.