FIR Digital Filter Design
Julius O. Smith III (jos@ccrma.stanford.edu)
Center for Computer Research in Music and Acoustics (CCRMA)
Department of Music, Stanford University
Stanford, California 94305
March 28, 2005
Outline
• Ideal Lowpass Filter
• Optimal Least Squares in Time Domain
• Window Method
• Optimal Methods
• Case Study: FIR HilbertTransform Design
1
FIR Digital Filter Design
In a previous lecture, we looked at many windows, and
examined their properties. Today, we are going to see
how these windows can be used to design Finite Impulse
Response (FIR) digital ﬁlters.
FFT processors implement long FIR ﬁlters more
eﬃciently than any other method (using OverlapAdd).
We need ﬂexible ways to design all kinds of FIR ﬁlters for
use in FFT processors.
2
Ideal Lowpass Filter
Amplitude Response:
Frequency 0 f
s
/2
0
1
G
a
i
n
f
c
• Gain = 1 for f < f
c
• Gain = 0 for f > f
c
3
Impulse Response of Ideal LPF
h
ideal
(n)
∆
= DTFT
−1
n
H
ideal
∆
=
1
2π
ω
c
−ω
c
e
jωn
dω
=
sin(ω
c
n)
πn
= 2f
c
sinc(2f
c
n), n ∈ Z
Problems:
• Inﬁnitely long
• Noncausal
• Cannot be shifted to make it causal
• Cannot be implemented in practice
Conclusion:
• We must accept some compromise(s) in the design of
any practical lowpass ﬁlter.
• Managing such tradeoﬀs is the topic of digital ﬁlter
design.
4
Optimal (but Poor) LeastSquares
Impulse Response Design
Let
• h(n) = ideal ﬁlter impulse response.
•
ˆ
h(n) = length L causal FIR ﬁlter (to be designed).
Sum of squared errors:
J
2
(
ˆ
h)
∆
=
∞
¸
n=−∞
h(n) −
ˆ
h(n)
2
=
L−1
¸
n=0
h(n) −
ˆ
h(n)
2
+ c
2
where c
2
∆
=
¸
0
n=−∞
[h(n)[
2
+
¸
∞
n=L
[h(n)[
2
does not
depend on
ˆ
h. Note that J
2
(
ˆ
h) ≥ c
2
.
Result: The error is minimized (in the leastsquares
sense) by simply matching the ﬁrst L terms in the desired
impulse response.
Optimal leastsquares FIR ﬁlter:
ˆ
h(n)
∆
=
h(n), 0 ≤ n ≤ L −1
0, otherwise
Also optimal under any L
p
norm with any error weighting:
J
p
(
ˆ
h) =
L−1
¸
n=0
w(n)
h(n) −
ˆ
h(n)
p
+ c
p
≥ c
p
5
Length L = 30 LeastSquares LPF Design
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
L
in
e
a
r
)
Length 30 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
d
B
)
• Desired cutoﬀ frequency is f
c
= 1/4
• Stopband attenuation only around 10 dB
• Passband gain has a “resonance” near cutoﬀ
• Filter is quite poor for audio use
•
ˆ
H = H ∗ asinc
L
6
Length L = 71 LeastSquares LPF Design
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
L
in
e
a
r
)
Length 71 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−70
−60
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
d
B
)
• Stopband attenuation still only around 10 dB
• Cornerfrequency resonance is higher, though narrower
• “Gibb’s phenomenon” in the frequency domain
What we’re doing wrong:
• Desired ﬁlter speciﬁcation asks for too much
(e.g, an inﬁnite rolloﬀ rate).
• Timedomainleastsquares is a poor choice of error
criterion for audio work.
7
Length L = 71 Optimal Chebyshev LPF Design
remez(70,[0 0.5 0.6 1],[1 1 0 0]);
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
L
in
e
a
r
)
Length 71 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−70
−60
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e
(
d
B
)
• Stopband attenuation better than 60 dB
• No cornerfrequency resonance
• Transition region from passband to stopband is
critical
• Error is “equiripple” in both stopband (visible) and
passband (not visible)
8
• Impulse response is slightly “impulsive” at the
endpoints (not shown).
9
Lowpass Filter Speciﬁcations
δs
δp
TW
Ideal lowpass
filter response
1
0
1 δ + p
1 δ − p
0 ωp ωs Frequency
Amplitude
Lowpass Filter Specifications
Passband Stopband
Transition
band
• δ
s
: stopband ripple (≤ 0.001 = −60 dB typical)
• δ
p
: passband ripple (≤ 0.1 dB typical)
• ω
s
: stopband edge frequency
• ω
p
: passband edge frequency
• TW: transition width = ω
s
−ω
p
• SBA: stopband attenuation = −20 log(δ
s
)
10
Example Ripple Calculations in Matlab
Let’s ﬁrst consider the passband ripple spec, ±0.1 dB.
Converting that to linear ripple amplitude gives, in
Matlab,
format long;
dp=10^(0.1/20)1
dp =
0.01157945425990
Let’s check it:
>> 20*log10(1+dp)
ans =
0.10000000000000
>> 20*log10(1dp)
ans =
0.10116471483635
Ok, close enough. Now let’s set the stopband ripple to
1/10 times the passband ripple and see where we are:
>> ds=dp/10;
>> 20*log10(ds)
ans =
58.72623816882052
11
So, that’s about 60 dB stopband rejection, which is not
too bad. Now let’s set the stopband ripple to 1/100
times the passband ripple:
>> ds=dp/100;
>> 20*log10(ds)
ans =
78.72623816882052
So now we’re close to 80dB SBA, which is getting into
the ”high ﬁdelity” zone.
Ideal Lowpass Filter
The ideal lowpass ﬁlter is deﬁned by the following
speciﬁcations:
• ω
s
= ω
p
= ω
c
∆
= 2πf
c
⇒TW = 0
• δ
p
= δ
s
= 0 ⇒SBA = ∞
12
The Window Method for FIR Filter
Design
• In practical FFT processors, we are limited to a ﬁnite
duration impulse response.
• An “obvious” method for FIR ﬁlter design is to simply
window the ideal impulse response, just like we
window ideal sinusoids in spectrum analysis:
ˆ
h
w
(n)
∆
= w(n) h
ideal
(n)
where w is a zero phase window of length M (odd).
• We saw that the rectangular window is optimal in the
timedomain leastsquares sense, but a poor choice
for typical audio ﬁlter design. What about other
windows?
• Windowing is multiplication in the time domain ↔
convolution in the frequency domain.
• Thus, in the window method, the ideal frequency
response is convolved with the window transform:
ˆ
H
w
(ω) = (W ∗ H
ideal
)(ω)
The convolution ‘smears’ the response, introducing
deviations in both the passband and stopband.
13
Example
A typical windowed ideal lowpass ﬁlter response is
depicted in the following diagram:
60 40 20 0 20 40 60
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
Length 101 Lowpass FIR Filter  Hamming Window
time (samples)
A
m
p
lit
u
d
e
14
• At this point in the design, we have a noncausal
(zero phase) ﬁlter.
• In order to implement this in a real time situation, we
need to shift the ﬁlter by an amount equal to half its
length.
ˆ
h
causal
(n) =
ˆ
h
n −
M −1
2
• This shift in the time domain results in a linear phase
term in the frequency domain:
ˆ
H
causal
(ω) = e
−jω
M−1
2 ˆ
H(ω)
15
Examples
• length 25 FIR ﬁlter
• ω
c
= π/4
• h
ideal
(n) =
sin(ω
c
n)
πn
n ∈ Z
• We will examine several windows:
0 10 20
0.1
0
0.1
0.2
0.3
Rectangular
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
0.1
0
0.1
0.2
0.3
Hanning
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
0.1
0
0.1
0.2
0.3
Hamming
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
0.1
0
0.1
0.2
0.3
Kaiser
time (samples)
A
m
p
l
i
t
u
d
e
16
Amplitude Response of Examples
2 0 2
60
40
20
0
Rectangular
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

d
B
2 0 2
60
40
20
0
Hanning
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

d
B
2 0 2
60
40
20
0
Hamming
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

d
B
2 0 2
60
40
20
0
Kaiser
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

d
B
17
Eﬀect of Changing Filter Length, M = 11, 15, 41
−3 −2 −1 0 1 2 3
−70
−60
−50
−40
−30
−20
−10
0
Hamming Window
Frequency (radians/sample)
A
m
p
lit
u
d
e
−
d
B
18
Other Types of Filters
Ideal highpass, bandpass, and bandstop ﬁlters can all be
considered as modiﬁcations of an ideal lowpass ﬁlter.
A highpass ﬁlter can be constructed by shifting a lowpass
ﬁlter by an amount equal to half the sampling rate.
i
i
PSfrag replacements
ω
ω
π
π
0
0
[H[
[H[
2π
2π
−π
−π
−2π
−2π
This is equivalent to a modulation by e
−jπn
in the time
domain. Hence,
h
hp
(n) = h
lp
(n)e
−jπn
= h
lp
(n)(−1)
n
19
Summarizing:
• To design a highpass ﬁlter, ﬁrst design a ﬁnite
length, causal lowpass ﬁlter
• Use a lowpass cutoﬀ frequency equal to
π−(highpass cutoﬀ)
• Design the lowpass ﬁlter as in the previous section
• Modulate the impulse reponse by (−1)
n
to exchange
DC and f
s
/2
Alternatively, if the passband of the prototype lowpass
ﬁlter
ˆ
H(ω) is very ﬂat about unity gain, a highpass ﬁlter
can be designed by simply subtracting from 1:
H
hp
(ω)
∆
= 1 −H
lp
(ω)
↔ δ(n) −h
lp
(n)
A bandpass can be derived by designing the appropriate
lowpass ﬁlter (with cutoﬀ frequency equal to half the
width of the bandpass), and then modulating it to the
desired center frequency ω
c
. In the frequency domain,
this means shifting two copies of the lowpass prototype,
one to the left by −ω
c
, and the other to the right by ω
c
.
The two shifted copies are added together to give a
bandpass ﬁlter having a real impulse response.
20
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
ω
0
0
[H[
[H[
π
π
ω
c
ω
0
ω
u
ω
l
−π
−π
−ω
c
−ω
0
To design a bandpass ﬁlter:
• Design a lowpass prototype ﬁlter of the desired width
(cutoﬀ ω
c
= ω
u
−ω
0
)
• Shift to the left and right by ω
0
and sum:
h
bp
(n) = 2 cos(nω
0
)h
lp
(n)
A bandstop ﬁlter can be constructed by ﬁrst designing a
bandpass with similar speciﬁcations.
In the frequency domain, we desire a ﬁlter with a transfer
function equal to H
bs
= 1 −H
bp
. Hence, in the time
domain we have:
h
bs
(n) =
1 −h
bp
(0) n = 0
−h
bp
(n) n = ±1, ±2, . . . , ±
M−1
2
21
Summary of the Window Method
Basic Idea:
• Start with Ideal LowpassFilter Impulse Response
• Sinc function sin(πt/T)/(πt/T)
• Inﬁnite duration
• Noncausal
• Window the inﬁnite duration sinc to get a ﬁnite
duration ﬁlter
• Window length determines transition width
• Window type determines the sidelobe level (SLL)
• Rectangular window yields the optimal ﬁlter in the
unweighted least squares sense
• Shift the noncausal (zero phase) ﬁlter to get a causal
(linear phase) ﬁlter
• The linear phase slope equals half the ﬁlter length
• Other types of ﬁlters (highpass, bandpass, bandstop)
are designed by ﬁrst designing a low pass, and
applying transformations
22
Important Points (Window Method)
• Easy to design (see fir1 and fir2 in Octave and
Matlab SPTB)
• Can design extremely long FIR ﬁlters without
numerical problems
• Not usually optimum in any sense
General Remarks (WindowMethod)
• Transition width TW determined by window length M
• As window gets longer, its main lobe narrows
• Narrower main lobe ⇒ narrower transition band
• Tighter specs demand longer ﬁlters
• Length M ∝ 1/TW in general
• Passband and stopband ripple are determined by
window side lobes
• We do not have individual control over passband
and stopband ripple
(This is a limitation of the window method)
• Ripple determined by window used
• Ripple normally largest around transition band
23
Optimal FIR Digital Filter Design
• Optimal Chebyshev FIR Design
• L
p
Norm Minimization
• Least squares problems
• MinMax problems
• Least Squares Optimization
• PseudoInverse
• FIR Filter Design
• Linear Phase
• Complex Filter Design
• Other Applications
Books including digital ﬁlter design:
• Boyd and Vandenberghe
• Parks and Burrus
• Rabiner and Gold
• Oppenheim and Schafer
• Steiglitz
24
• Strum and Kirk
See Music 421 References
1
1
http://ccrma.stanford.edu/˜jos/refs421/
25
Optimal Chebyshev Filters
• Minimize the maximum of the error
• Stopband and passband errors are equiripple
• Remez Multiple Exchange (ParksMcClellan
algorithm)
• Available in Octave and the Matlab SP Tool Box:
remez(), cremez()
Problems with Remez Exchange
• Convergence unlikely for FIR ﬁlters longer than a few
hundred taps or so
• Fundamentally real polynomial approximation on the
unit circle
• Applies only to linear phase ﬁlters
• Cannot design individual (nonminimum phase)
“subﬁlters” in polyphase ﬁlter bank form
• Really need optimal weighted complex approximation
e.g., minimum phase FIR ﬁlter design
26
Example: Chebyshev window and its transform:
100 50 50 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time (samples)
A
m
p
l
i
t
u
d
e
Chebyshev Window: M = 101, Ripple = 40 dB
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
27
3 2 1 1 2 3
70
60
50
40
30
20
10
freq
d
B
M
a
g
n
i
t
u
d
e
DFT of Chebyshev Window: M = 11, Ripple = 40 dB
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
28
Lp norms
The L
p
norm of an Ndimensional vector x is deﬁned as
 x 
p
∆
=
N−1
¸
i=0
[x
i
[
p
1
p
Special cases
• L
1
norm
 x 
1
∆
=
N−1
¸
i=0
[x
i
[
• Sum of the absolute values of the elements
• “City block” distance
• L
2
norm
 x 
2
∆
=
N−1
¸
i=0
[x
i
[
2
• “Euclidean” distance
• Minimized by “Least Squares” techniques
29
• L
∞
norm
 x 
∞
∆
= lim
p→∞
N−1
¸
i=0
[x
i
[
p
1
p
In the limit as p →∞, the L
p
norm of x is
dominated by the maximum element of x.
30
Filter Design using Lp Norms
In terms of L
p
norm minimization, the FIR ﬁlter design
problem becomes:
min
h
W(ω
k
) [H(ω
k
) −D(ω
k
)]
p
Where
• ω
k
= suitable discrete set of frequencies
• H(ω
k
) = frequency response of our FIR ﬁlter
• D(ω
k
) = desired (complex) frequency response
• W(ω
k
) = (optional) weighting function
Lets look at some speciﬁc cases:
31
LeastSquares LinearPhase FIR Filter Design
Let the FIR ﬁlter length be L + 1 samples, with L even,
and suppose we’ll initially design it to be centered about
the time origin (“zerophase”). Then the frequency
response is given on our frequency grid ω
k
by
H(ω
k
) =
L/2
¸
n=−L/2
h
n
e
−jω
k
n
Enforcing even symmetry in the impulse response, i.e.,
h
n
= h
−n
, gives a zero phase FIR ﬁlter which we can
later rightshift L/2 samples to make a causal, linear
phase ﬁlter. In this case, the frequency response reduces
to a sum of cosines:
H(ω
k
) = h
0
+ 2
L/2
¸
n=1
h
n
cos(ω
k
n), k = 0, 1, 2, . . . , N −1
or in matrix form:
H(ω
0
)
H(ω
1
)
.
.
.
H(ω
N−1
)
=
1 2 cos(ω
0
) . . . 2 cos[ω
0
(L −1)]
1 2 cos(ω
1
) . . . 2 cos[ω
1
(L −1)]
.
.
.
1 2 cos(ω
N−1
) . . . 2 cos[ω
N−1
(L −1)]
A
h
0
h
1
.
.
.
h
L/2
x
(Note that Remez exchange algorithms are also based on
this formulation internally.)
32
Matrix Formulation: Optimal L
2
Design, Cont’d
In matrix notation, our ﬁlter design problem can be stated
min
x
Ax −b
2
2
where, for zerophase ﬁlters,
A
∆
=
1 2 cos(ω
0
) . . . 2 cos [ω
0
(L −1)]
1 2 cos(ω
1
) . . . 2 cos [ω
1
(L −1)]
.
.
.
1 2 cos(ω
N−1
) . . . 2 cos [ω
N−1
(L −1)]
¸
¸
¸
¸
x
∆
= h
and b = [D(ω
k
)] is the desired frequency response at the
speciﬁed frequencies.
The function firls in the Matlab Signal Processing Tool
Box implements frequencydomain weightedleastsquares
FIR ﬁlter design (not available in Octave as of 2/2003).
33
Least Squares Optimization
ˆ x
∆
= arg min
x
 Ax −b 
2
= arg min
x
 Ax −b 
2
2
Hence we can minimize
 Ax −b 
2
2
= (Ax −b)
T
(Ax −b)
Expanding this, we have:
(Ax −b)
T
(Ax −b) = (b
T
−x
T
A
T
)(Ax −b)
This is quadratic in x, hence it has a global minimum
which we can ﬁnd by taking the derivative, setting it to
zero, and solving for x. Doing this yields:
A
T
Ax −A
T
b = 0
These are the famous normal equations whose solution is
given by:
ˆ x =
(A
T
A)
−1
A
T
b
The matrix
A
†
∆
= (A
T
A)
−1
A
T
is known as the pseudoinverse of the matrix A.
34
Geometrical Interpretation of Least Squares
Typically, the number of frequency constraints is much
greater than the number of design variables (ﬁlter taps).
In these cases, we have an overdetermined system of
equations (more equations than unknowns). Therefore,
we cannot generally satisfy all the equations, and we are
left with minimizing some error criterion to ﬁnd the
“optimal compromise” solution.
In the case of leastsquares approximation, we are
minimizing the Euclidean distance, which suggests the
following geometrical interpretation:
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Aˆ x
columnspace of A
Aˆ x = b + e
Minimize
x
e
2
= b −Ax
2
35
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Aˆ x
columnspace of A
This diagram suggests that the error vector b −Ax is
orthogonal to the column space of the matrix A, hence it
must be orthogonal to each column in A.
A
T
(b −Aˆ x) = 0 ⇒A
T
Aˆ x = A
T
b
This is how the orthogonality principle can be used to
derive the fact that the best least squares solution is
given by
ˆ x = (A
T
A)
−1
A
T
b = A
†
b
Note that the pseudoinverse A
†
projects the vector b
onto the column space of A.
In Matlab:
• x = A ` b
• x = pinv(A) * b
36
Chebyshev Optimal Linear Phase ﬁlter Design
Consider the L
∞
norm minimization problem:
min
x
 Ax −b 
∞
We said earlier that the ∞ norm of a vector is the
maximum element of that vector, hence minimizing the
∞ norm is minimizing the maximum element of a vector:
min
x
max
k
[a
T
k
x −b
k
[
where a
T
k
denotes the kth row of the matrix A.
We can then write this as:
min t
s.t. [a
T
k
x −b
k
[ < t
If we introduce a new variable ˜ x
∆
=
¸
x
t
, then
t = f
T
˜ x = [ 0 . . . 0 1 ] ˜ x, and our optimization problem
becomes:
min
˜ x
f
T
˜ x
s.t.
[ a
T
k
0 ] ˜ x −b
k
< ˜ xf
T
˜ x
37
Hence we are minimizing a linear objective, subject to a
set of linear constraints. This is known as a linear
programming problem (linprog in Matlab).
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Linear
Linear
Objective
Constraints
38
More General Real FIR Filters
So far we have looked at the design of linear phase FIR
ﬁlters. In this case, A, x and b are all real.
Sometimes we may want to design ﬁlters and specify both
the magnitude and the phase of the frequency response.
Examples:
• Minimum phase ﬁlters
• Inverse ﬁlters
• Fractional delay ﬁlters
• Interpolation polyphase subﬁlters
39
Complex FIR Filter Design
In linearphase ﬁlter design, we assumed symmetry of our
ﬁlter coeﬃcients [h(n) = h(−n)] ⇒
• The ﬁlter frequency response became a sum of
cosines (“zero phase”)
• The matrix A was real
• The desired magnitude response b was real
• The ﬁnal zerophase ﬁlter ˆ x could be rightshifted
L/2 samples to get a corresponding causal
linearphase FIR ﬁlter
Now we would like to specify a complex frequency
response. This means that:
• b is complex
• A is complex
• We still want x (our ﬁlter coeﬃcients) to be real
If we try to use ’`’ or pinv in Matlab, we will generally
get a complex result for ˆ x
40
Summarizing our problem:
min
x
 Ax −b 
2
where, A ∈ C
NxM
, b ∈ C
Nx1
, and x ∈ R
Mx1
Hence we have,
min
x
[{(A) + j1(A)] x −[{(b) + j1(b)]
2
2
which can be written as:
min
x
 {(A)x −{(b) + j [1(A)x + 1(b)] 
2
2
or
min
x
¸
{(A)
1(A)
x −
¸
{(b)
1(b)
2
2
which is written in terms of only real variables.
Hence, we can use the standard least squares solvers in
Matlab and end up with a real solution.
Related paper
“Design of Fractional Delay Filters Using Convex
Optimization” (Mohonk97, Music 421 handout):
http://ccrma.stanford.edu/~jos/eps/mohonk97.ps.gz
41
Optimal FIR Filters — Arbitrary Magnitude and
Phase Speciﬁciations
cremez (Matlab Signal Processing Toolbox) performs
complex L
∞
FIR ﬁlter design:
• Documented online at The Mathworks
(search for cremez)
• Convergence theoretically guaranteed for arbitrary
magnitude and phase speciﬁcations versus frequency.
• Reduces to ParksMcClellan algorithm (Remez second
algorithm) as a special case.
• Written by Karam and McClellan. See “Design of
Optimal Digital FIR Filters with Arbitrary Magnitude
and Phase Responses,” by Lina J. Karam and James
H. McClellan, ISCAS96. The paper may be
downloaded at
http://www.eas.asu.edu/~karam/papers/iscas96 km.html
42
SecondOrder Cone Problems (SOCP)
• A linear function is minimized over the intersection of
an aﬃne set and the product of secondorder
(quadratic) cones
• Nonlinear, convex problem including linear and
(convex) quadratic programs as special cases
• Solved by eﬃcient primaldual interiorpoint methods
• Number of iterations required to solve a problem
grows at most as the square root of the problem size
• Typical number of iterations ranges between 5 and
50, almost independent of the problem size
See “Applications of secondorder cone programming,” by
M. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret,
Linear Algebra and its Applications, 284:193228,
November 1998, Special Issue on Linear Algebra in
Control, Signals and Image Processing. Available online:
http://www.stanford.edu/~boyd/socp.html
For more on this topic, see also Prof. Boyd’s course on
convex optimization: EE 364
2
and text book (Boyd and
Vandenberghe, Cambridge U. Press, 2004).
2
http://www.stanford.edu/class/ee364/
43
Hilbert Transform Filter Design Example:
A Case Study in FIR Filter Design
Design Example:
• Ideal SingleSideband Filter (Hilbert transformer)
• Window Method using Kaiser Window
• Optimal Chebyshev using Remez Exchange
44
Problem Statement
• Design a “negativefrequency ﬁlter” which converts a
real signal into its complex “analytic signal”
counterpart by ﬁltering out negative frequencies.
• Equivalently, design a “single sideband” (SSB) ﬁlter
which outputs a singlesideband signal in response to
a complexconjugate pair of sidebands (a Hermitian
spectrum).
• We could also call it a “positivefrequencypass” ﬁlter.
• The imaginary part of such a ﬁlter is called a
Hilbert transform ﬁlter.
• The ideal Hilbert transform is an allpass ﬁlter that
delays positivefrequency components by 90 degrees,
and advances negativefrequency components by 90
degrees.
45
Hilbert Transform
y(t)
∆
= (h
i
∗ x)(t) (Hilbert transform of x)
h
i
(t)
∆
=
1
πt
(Hilbert transform “kernel”)
H
i
(ω)
∆
=
−j, ω > 0
j, ω < 0
1, ω = 0
(Hilbert frequency response)
x
a
(t)
∆
= x(t) + jy(t) (Analytic signal from x)
=
1
π
∞
0
X(ω)e
jωt
dω (Note: Lower limit usually −∞)
Proof:
X
a
(ω) = X(ω) + jY (ω) (By linearity of Fourier transform)
= X
+
+ X
−
(Apply frequency response)
+j[−jX
+
+ jX
−
]
= 2X
+
(ω)
46
Kaiser Window
Kaiser window in causal, linearphase form:
w = kaiser(M,beta); % M=length=257, beta=8
Zeropad by a factor of 8 and convert to zerophase form
(N = FFT size = 2048):
wzp = [w((M+1)/2:M),zeros(1,NM),w(1:(M1)/2)];
Kaiser window transform:
W = fft(wzp);
47
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−160
−140
−120
−100
−80
−60
−40
−20
Kaiser Window Transform, FFT size = 2048, Window length = 257
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
48
Kaiser window transform, closeup on main lobe
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−120
−100
−80
−60
−40
−20
Close Up on Kaiser LF Response, FFT size = 2048, Window length = 257
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
49
Oversimpliﬁed Window Method
Sample the ideal Hilberttransform kernel
h(t) = 1/πt
to get
ˆ
h
i
(n)
∆
=
1
πnT
(Sampled Hilbert transform kernel)
ˆ
h
w
(n)
∆
= w(n)
ˆ
h
i
(n) (Windowed ideal impulse response)
ˆ
H
w
(ω) = (W ∗
ˆ
H)(n) (Smoothed ideal frequency response)
Design Parameters:
fs = 22050; % sampling rate (Hz)
T = 1/fs; % sampling period (sec)
M = 257; % FIR filter length = window length
N = 8*(M1); % for interpolated spectral displays
beta = 8; % beta for Kaiser window
Filter Design:
n = [N/2+0.5:N/20.5]; % Time axis (avoid t=0)
hi = T ./ (pi*n*T); % Sampled Hilbert kernel
hr = sin(pi*n) ./ (pi*n); % 1/2 sample delay filter
h = (hr + j*hi)/2; % Sampled ideal final filter
plot(f,fftshift(max(100,20*log10(abs(fft(h)))))); grid;
...
50
FFT of Truncated(2048) Ideal Impulse Response
0.6 0.4 0.2 0.2 0.4 0.6
100
80
60
40
20
20
FFT of Truncated Analytically Specified "Ideal" Impulse Response
Normalized Frequency (cycles/sample)
G
a
i
n
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω 0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
h = [h(N/2+1:N),h(1:N/2)]; % zerophase form (almost)
hw = wzp .* h; % Apply window to ideal impulse response
Hw = fft(hw); % Frequency response we really get
% Compute total stopband attenuation:
ierr = norm(Hw(N/2+2:N))/norm(Hw)
= 0.0378 = 3.8 percent
For spectral displays, rotate buﬀer so negative frequencies
are on the left and DC is at k = N/2:
51
Hwp = [Hw(N/2+2:N), Hw(1:N/2+1)]; % Neg. freqs on left
Hwpn = abs(Hwp); Hwpn = Hwpn/max(Hwpn);
plot(f,20*log10(Hwpn)); grid;
...
FIR Frequency Response by the Window Method
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−140
−120
−100
−80
−60
−40
−20
Length 257 Analytic−Derived FIR Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
52
Zoom in on lowfrequency transition band
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−140
−120
−100
−80
−60
−40
−20
20
Analytic−Derived Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
• Poorly positioned transition region from positive to
negative frequencies.
• Need to ﬁrst bandpassﬁlter the ideal impulse
response.
• Highfrequency transition identical, by symmetry
53
Ideal Bandlimited Impulse Response
Since we are working with sampled data, the ideal
singlesideband ﬁlter impulse response is
h(n) = IDTFT
n
(u)
∆
=
1
2π
π
−π
u(ω)e
jωn
dω
where u(ω) is the unit step function in the frequency
domain:
u(ω)
∆
=
1, ω ≥ 0
0, ω < 0
We can evaluate the IDTFT directly as follows:
h(n) =
1
2π
π
0
e
jωn
dω =
1
2πjn
e
jωn
π
0
=
e
jπn
−1
2πjn
=
(−1)
n
−1
2πjn
=
0, n even, n = 0
j
1
πn
, n odd
For n = 0, going back to the original integral gives
h(0) =
1
2π
π
0
e
jω·0
dω =
1
2
.
Thus, the real part of h(n) is δ(n)/2, and the imaginary
part is 1/(πn) for odd n and zero otherwise (as a result
of bandlimiting). There is no aliasing. However, we also
54
need a transition band. We can work this out analytically,
or we can simply “draw” it in the frequency domain, take
a (large) inverse DFT, and window that. Since the latter
approach is more general, we’ll do that.
Window Method applied to Bandlimited Ideal
Impulse Response
Further Design Parameters:
f1 = 530; % lower passband limit (Hz)
N = 2^(nextpow2(8*fs/f1)) % FFT size from xition width
if N<8*M, N = 8*(M1); end; % spectral interpolation
Bandlimited “Ideal” Hilbert transformer frequency
response (FR), including transition bands ∆ matched to
FFT size N such that N f
s
/∆:
H = [ ([0:k12]/(k11)).^8,ones(1,k2k1+1), ...
([k12:1:0]/(k11)).^8,zeros(1,N/21)];
Hp = [H(N/2+2:N), H(1:N/2+1)];
f = [0.5 + 1/N : 1/N : 0.5]; normalized freq axis
plot(f,Hp); grid; ...
55
Desired Frequency Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−150
−100
−50
Desired Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
56
Desired Impulse Response
h = ifft(H); % zerophase form
% measure of roundoff error:
ierr = norm(imag(h(1:2:N)))/norm(h) % zero?
ierr =
4.1958e15
% rough estimate of timealiasing error:
aerr = norm(h(N/2N/32:N/2+N/32))/norm(h)
aerr =
4.8300e04
% for plots, prefer negative times on the left:
hp = [h(N/2+2:N), h(1:N/2+1)];
57
Real Part of Desired Impulse Response
−150 −100 −50 50 100 150
−0.1
0.1
0.2
0.3
0.4
0.5
0.6
Ideal SSB Filter Impulse Response, Real Part, FFT size = 2048
Time (samples)
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
58
Imaginary Part of Desired Impulse Response
−150 −100 −50 50 100 150
−0.4
−0.3
−0.2
−0.1
0.1
0.2
0.3
0.4
Ideal SSB Filter Impulse Response, Imaginary Part, FFT size = 2048
Time (samples)
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Apply window to ideal impulse response:
hw = wzp .* h; % Final FIR coefficients
Hw = fft(hw); % Frequency response we’ll see
Total stopband energy:
ierr = norm(Hw(N/2+2:N))/norm(Hw)
disp(sprintf([’Stopband energy is %g percent of’,...
total spectral energy’], 100*ierr));
59
Final FIR Filter Frequency Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−180
−160
−140
−120
−100
−80
−60
−40
−20
Length 257 FIR filter Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
60
Zoomin on Transition Region of Final Frequency
Response
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−140
−120
−100
−80
−60
−40
−20
20
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
• Transition band better positioned
• ≈ 100 dB stobband attenuation
61
Remez Multiple Exchange Method
• Remez exchange algorithm for optimal Chebyshev
(equiripple) FIR ﬁlter design.
• Original reference: J. H. McClellan, T. W. Parks, and
L. R. Rabiner, “A Computer Program for Designing
Optimum FIR Linear Phase Digital Filters,” IEEE
Trans. Audio Electro., vol AU21, Dec. 1973,
pp. 506526.
• Text reference: L. R. Rabiner and B. Gold, Theory
and Application of Digital Signal Processing,
PrenticeHall, 1975.
• Pertinent recent reference: A. Reilly, G. Frazer, and
B. Boashash, “Analytic Signal Generation—Tips and
Traps,” IEEE Tr. Signal Processing, vol. 42, no. 11,
Nov. 1994.
Design an optimal lowpass ﬁlter of the desired width
(takes quite a while):
hrm = remez(M1, [0,(f2fs/4)/fn,0.5,1], ...
[1,1,0,0], [1,10]);
The weighting [1,P] says make the passband ripple P
times that of stopband. For steadystate audio spectra,
62
passband ripple can be 0.1 dB or more. However,
consider an FM signal in which a sinusoid is sweeping
back and forth in the passband. In that case, the
passband ripple generates AM sidebands, so a spec more
like that in the stopband may be called for. Here, we
allow the passband ripple to be 10 times the stopband
ripple, as a compromise.
Modulate it up to where it’s a singlesideband ﬁlter. I.e.,
rightshift by π/2 in the frequency domain. I.e., rotate
the frequency response counterclockwise along the unit
circle by 90 degrees:
hr = hrm .* j .^ [0:M1];
63
Optimal Chebyshev FIR(257) Frequency
Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−200
−150
−100
−50
50
Length 257 Optimal Chebyshev FIR Single−Sideband−Filter Freq. Response
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
64
ZoomIn on Transition Band
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−200
−150
−100
−50
50
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
65
Passband Ripple
0.18 0.2 0.22 0.24 0.26 0.28 0.3 0.32
−4
−3
−2
−1
1
2
3
4
x 10
−4
Passband Ripple
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
66
Initial Impulse Response
5 10 15 20 25 30 35 40 45 50
−8
−6
−4
−2
2
4
6
8
x 10
−4
First 50 impulse−response coefficients
Index
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
67
Summary of TradeOﬀs
Windowmethod ﬁlter:
• Stopband ”droops” which ”wastes” ﬁlter taps if the
worstcase stopband attenuation is the only stopband
spec. However, such ”rolloﬀ” is natural in the
frequency domain and corresponds to a ”smoother
pulse” in the time domain.
• Passband is degraded by early rolloﬀ. Edge is oﬀ.
• Filter length can be thousands of taps as needed for
full audio band.
Optimal RemezExchange ﬁlter:
• Stopband is ideal, equiripple.
• Transition region close to half that of the window
method.
• Passband is ideal, equiripple.
• Timedomain impulses from passband ripple are small.
• Design time orders of magnitude larger than that for
window method.
68
• Fails to converge for ﬁlters much longer than 256
taps. (Need to increase working precision to get
longer ﬁlters.)
69
Matlab hilbert() function
The Matlab Hilbert function returns an analytic signal
given its real part.
Nh = M2; % This looks best
delta = [1,zeros(1,Nh)]; % zerophase impulse
hh = hilbert(delta); % zeros negativefreq FFT bins
Hh = fft(hh); % FIR frequency response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−350
−300
−250
−200
−150
−100
−50
Frequency Response of Length 256 hilbert() using Matlab
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Looks pretty good, but let’s zeropad the impulse
response to interpolate the frequency response:
70
hhzp = [hh(Lh/2+1:Lh), zeros(1,NLh), hh(1:Lh/2)];
Hhzp = fft(hhzp); % Frequency response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−350
−300
−250
−200
−150
−100
−50
Interpolated Frequency Response of Length 256 hilbert() using Matlab
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
71
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−350
−300
−250
−200
−150
−100
−50
50
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in
(
d
B
)
PSfrag replacements
ω
π
0
[H[
2π
−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Problems:
• Transition bandwidth only 2 bins wide.
• Massive time aliasing.
• Only justiﬁable for periodic signals with period exactly
equal to ﬁlter length Nh. In this case only, time
aliasing is equivalent to overlapadd from adjacent
periods.
72
More generally, any real signal given to hilbert() must be
interpreted as precisely one period of a periodic signal.
73