MUS421/EE367B Lecture 6

FIR Digital Filter Design
Julius O. Smith III (jos@ccrma.stanford.edu)
Center for Computer Research in Music and Acoustics (CCRMA)
Department of Music, Stanford University
Stanford, California 94305
March 28, 2005
Outline
• Ideal Lowpass Filter
• Optimal Least Squares in Time Domain
• Window Method
• Optimal Methods
• Case Study: FIR Hilbert-Transform Design
1
FIR Digital Filter Design
In a previous lecture, we looked at many windows, and
examined their properties. Today, we are going to see
how these windows can be used to design Finite Impulse
Response (FIR) digital filters.
FFT processors implement long FIR filters more
efficiently than any other method (using Overlap-Add).
We need flexible ways to design all kinds of FIR filters for
use in FFT processors.
2
Ideal Lowpass Filter
Amplitude Response:
Frequency 0 f
s
/2
0
1
G
a
i
n
f
c
• Gain = 1 for f < f
c
• Gain = 0 for f > f
c
3
Impulse Response of Ideal LPF
h
ideal
(n)

= DTFT
−1
n

H
ideal


=
1

ω
c
−ω
c
e
jωn

=
sin(ω
c
n)
πn
= 2f
c
sinc(2f
c
n), n ∈ Z
Problems:
• Infinitely long
• Non-causal
• Cannot be shifted to make it causal
• Cannot be implemented in practice
Conclusion:
• We must accept some compromise(s) in the design of
any practical lowpass filter.
• Managing such trade-offs is the topic of digital filter
design.
4
Optimal (but Poor) Least-Squares
Impulse Response Design
Let
• h(n) = ideal filter impulse response.

ˆ
h(n) = length L causal FIR filter (to be designed).
Sum of squared errors:
J
2
(
ˆ
h)

=

¸
n=−∞

h(n) −
ˆ
h(n)

2
=
L−1
¸
n=0

h(n) −
ˆ
h(n)

2
+ c
2
where c
2

=
¸
0
n=−∞
[h(n)[
2
+
¸

n=L
[h(n)[
2
does not
depend on
ˆ
h. Note that J
2
(
ˆ
h) ≥ c
2
.
Result: The error is minimized (in the least-squares
sense) by simply matching the first L terms in the desired
impulse response.
Optimal least-squares FIR filter:
ˆ
h(n)

=

h(n), 0 ≤ n ≤ L −1
0, otherwise
Also optimal under any L
p
norm with any error weighting:
J
p
(
ˆ
h) =
L−1
¸
n=0
w(n)

h(n) −
ˆ
h(n)

p
+ c
p
≥ c
p
5
Length L = 30 Least-Squares LPF Design
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
L
in
e
a
r
)
Length 30 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
d
B
)
• Desired cut-off frequency is f
c
= 1/4
• Stopband attenuation only around 10 dB
• Passband gain has a “resonance” near cut-off
• Filter is quite poor for audio use

ˆ
H = H ∗ asinc
L
6
Length L = 71 Least-Squares LPF Design
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
L
in
e
a
r
)
Length 71 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−70
−60
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
d
B
)
• Stopband attenuation still only around 10 dB
• Corner-frequency resonance is higher, though narrower
• “Gibb’s phenomenon” in the frequency domain
What we’re doing wrong:
• Desired filter specification asks for too much
(e.g, an infinite roll-off rate).
• Time-domain-least-squares is a poor choice of error
criterion for audio work.
7
Length L = 71 Optimal Chebyshev LPF Design
remez(70,[0 0.5 0.6 1],[1 1 0 0]);
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
0.5
1
1.5
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
L
in
e
a
r
)
Length 71 FIR Amplitude Response
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
−70
−60
−50
−40
−30
−20
−10
0
Normalized Frequency (cycles/sample)
M
a
g
n
it
u
d
e

(
d
B
)
• Stopband attenuation better than 60 dB
• No corner-frequency resonance
• Transition region from passband to stopband is
critical
• Error is “equiripple” in both stopband (visible) and
passband (not visible)
8
• Impulse response is slightly “impulsive” at the
endpoints (not shown).
9
Lowpass Filter Specifications
δs
δp
TW
Ideal lowpass
filter response
1
0
1 δ + p
1 δ − p
0 ωp ωs Frequency
Amplitude
Lowpass Filter Specifications
Pass-band Stop-band
Transition
band
• δ
s
: stopband ripple (≤ 0.001 = −60 dB typical)
• δ
p
: passband ripple (≤ 0.1 dB typical)
• ω
s
: stopband edge frequency
• ω
p
: passband edge frequency
• TW: transition width = ω
s
−ω
p
• SBA: stop-band attenuation = −20 log(δ
s
)
10
Example Ripple Calculations in Matlab
Let’s first consider the passband ripple spec, ±0.1 dB.
Converting that to linear ripple amplitude gives, in
Matlab,
format long;
dp=10^(0.1/20)-1
dp =
0.01157945425990
Let’s check it:
>> 20*log10(1+dp)
ans =
0.10000000000000
>> 20*log10(1-dp)
ans =
-0.10116471483635
Ok, close enough. Now let’s set the stopband ripple to
1/10 times the passband ripple and see where we are:
>> ds=dp/10;
>> 20*log10(ds)
ans =
-58.72623816882052
11
So, that’s about 60 dB stop-band rejection, which is not
too bad. Now let’s set the stopband ripple to 1/100
times the passband ripple:
>> ds=dp/100;
>> 20*log10(ds)
ans =
-78.72623816882052
So now we’re close to 80dB SBA, which is getting into
the ”high fidelity” zone.
Ideal Lowpass Filter
The ideal lowpass filter is defined by the following
specifications:
• ω
s
= ω
p
= ω
c

= 2πf
c
⇒TW = 0
• δ
p
= δ
s
= 0 ⇒SBA = ∞
12
The Window Method for FIR Filter
Design
• In practical FFT processors, we are limited to a finite
duration impulse response.
• An “obvious” method for FIR filter design is to simply
window the ideal impulse response, just like we
window ideal sinusoids in spectrum analysis:
ˆ
h
w
(n)

= w(n) h
ideal
(n)
where w is a zero phase window of length M (odd).
• We saw that the rectangular window is optimal in the
time-domain least-squares sense, but a poor choice
for typical audio filter design. What about other
windows?
• Windowing is multiplication in the time domain ↔
convolution in the frequency domain.
• Thus, in the window method, the ideal frequency
response is convolved with the window transform:
ˆ
H
w
(ω) = (W ∗ H
ideal
)(ω)
The convolution ‘smears’ the response, introducing
deviations in both the pass-band and stop-band.
13
Example
A typical windowed ideal lowpass filter response is
depicted in the following diagram:
-60 -40 -20 0 20 40 60
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Length 101 Lowpass FIR Filter - Hamming Window
time (samples)
A
m
p
lit
u
d
e
14
• At this point in the design, we have a non-causal
(zero phase) filter.
• In order to implement this in a real time situation, we
need to shift the filter by an amount equal to half its
length.
ˆ
h
causal
(n) =
ˆ
h

n −
M −1
2

• This shift in the time domain results in a linear phase
term in the frequency domain:
ˆ
H
causal
(ω) = e
−jω
M−1
2 ˆ
H(ω)
15
Examples
• length 25 FIR filter
• ω
c
= π/4
• h
ideal
(n) =
sin(ω
c
n)
πn
n ∈ Z
• We will examine several windows:
0 10 20
-0.1
0
0.1
0.2
0.3
Rectangular
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
-0.1
0
0.1
0.2
0.3
Hanning
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
-0.1
0
0.1
0.2
0.3
Hamming
time (samples)
A
m
p
l
i
t
u
d
e
0 10 20
-0.1
0
0.1
0.2
0.3
Kaiser
time (samples)
A
m
p
l
i
t
u
d
e
16
Amplitude Response of Examples
-2 0 2
-60
-40
-20
0
Rectangular
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

-

d
B
-2 0 2
-60
-40
-20
0
Hanning
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

-

d
B
-2 0 2
-60
-40
-20
0
Hamming
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

-

d
B
-2 0 2
-60
-40
-20
0
Kaiser
Frequency (radians/sample)
A
m
p
l
i
t
u
d
e

-

d
B
17
Effect of Changing Filter Length, M = 11, 15, 41
−3 −2 −1 0 1 2 3
−70
−60
−50
−40
−30
−20
−10
0
Hamming Window
Frequency (radians/sample)

A
m
p
lit
u
d
e



d
B
18
Other Types of Filters
Ideal highpass, bandpass, and bandstop filters can all be
considered as modifications of an ideal lowpass filter.
A highpass filter can be constructed by shifting a lowpass
filter by an amount equal to half the sampling rate.
i
i
PSfrag replacements
ω
ω
π
π
0
0
[H[
[H[


−π
−π
−2π
−2π
This is equivalent to a modulation by e
−jπn
in the time
domain. Hence,
h
hp
(n) = h
lp
(n)e
−jπn
= h
lp
(n)(−1)
n
19
Summarizing:
• To design a high-pass filter, first design a finite
length, causal lowpass filter
• Use a lowpass cutoff frequency equal to
π−(highpass cutoff)
• Design the lowpass filter as in the previous section
• Modulate the impulse reponse by (−1)
n
to exchange
DC and f
s
/2
Alternatively, if the passband of the prototype lowpass
filter
ˆ
H(ω) is very flat about unity gain, a highpass filter
can be designed by simply subtracting from 1:
H
hp
(ω)

= 1 −H
lp
(ω)
↔ δ(n) −h
lp
(n)
A bandpass can be derived by designing the appropriate
lowpass filter (with cutoff frequency equal to half the
width of the bandpass), and then modulating it to the
desired center frequency ω
c
. In the frequency domain,
this means shifting two copies of the lowpass prototype,
one to the left by −ω
c
, and the other to the right by ω
c
.
The two shifted copies are added together to give a
bandpass filter having a real impulse response.
20
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
ω
0
0
[H[
[H[
π
π
ω
c
ω
0
ω
u
ω
l
−π
−π
−ω
c
−ω
0
To design a bandpass filter:
• Design a lowpass prototype filter of the desired width
(cutoff ω
c
= ω
u
−ω
0
)
• Shift to the left and right by ω
0
and sum:
h
bp
(n) = 2 cos(nω
0
)h
lp
(n)
A bandstop filter can be constructed by first designing a
bandpass with similar specifications.
In the frequency domain, we desire a filter with a transfer
function equal to H
bs
= 1 −H
bp
. Hence, in the time
domain we have:
h
bs
(n) =

1 −h
bp
(0) n = 0
−h
bp
(n) n = ±1, ±2, . . . , ±
M−1
2
21
Summary of the Window Method
Basic Idea:
• Start with Ideal Lowpass-Filter Impulse Response
• Sinc function sin(πt/T)/(πt/T)
• Infinite duration
• Non-causal
• Window the infinite duration sinc to get a finite
duration filter
• Window length determines transition width
• Window type determines the sidelobe level (SLL)
• Rectangular window yields the optimal filter in the
unweighted least squares sense
• Shift the noncausal (zero phase) filter to get a causal
(linear phase) filter
• The linear phase slope equals half the filter length
• Other types of filters (highpass, bandpass, bandstop)
are designed by first designing a low pass, and
applying transformations
22
Important Points (Window Method)
• Easy to design (see fir1 and fir2 in Octave and
Matlab SPTB)
• Can design extremely long FIR filters without
numerical problems
• Not usually optimum in any sense
General Remarks (Window-Method)
• Transition width TW determined by window length M
• As window gets longer, its main lobe narrows
• Narrower main lobe ⇒ narrower transition band
• Tighter specs demand longer filters
• Length M ∝ 1/TW in general
• Pass-band and stop-band ripple are determined by
window side lobes
• We do not have individual control over passband
and stopband ripple
(This is a limitation of the window method)
• Ripple determined by window used
• Ripple normally largest around transition band
23
Optimal FIR Digital Filter Design
• Optimal Chebyshev FIR Design
• L
p
Norm Minimization
• Least squares problems
• Min-Max problems
• Least Squares Optimization
• PseudoInverse
• FIR Filter Design
• Linear Phase
• Complex Filter Design
• Other Applications
Books including digital filter design:
• Boyd and Vandenberghe
• Parks and Burrus
• Rabiner and Gold
• Oppenheim and Schafer
• Steiglitz
24
• Strum and Kirk
See Music 421 References
1
1
http://ccrma.stanford.edu/˜jos/refs421/
25
Optimal Chebyshev Filters
• Minimize the maximum of the error
• Stopband and passband errors are equiripple
• Remez Multiple Exchange (Parks-McClellan
algorithm)
• Available in Octave and the Matlab SP Tool Box:
remez(), cremez()
Problems with Remez Exchange
• Convergence unlikely for FIR filters longer than a few
hundred taps or so
• Fundamentally real polynomial approximation on the
unit circle
• Applies only to linear phase filters
• Cannot design individual (non-minimum phase)
“sub-filters” in polyphase filter bank form
• Really need optimal weighted complex approximation
e.g., minimum phase FIR filter design
26
Example: Chebyshev window and its transform:
-100 -50 50 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time (samples)
A
m
p
l
i
t
u
d
e
Chebyshev Window: M = 101, Ripple = -40 dB
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
27
-3 -2 -1 1 2 3
-70
-60
-50
-40
-30
-20
-10
freq
d
B

M
a
g
n
i
t
u
d
e
DFT of Chebyshev Window: M = 11, Ripple = -40 dB
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
28
Lp norms
The L
p
norm of an N-dimensional vector x is defined as
| x |
p

=

N−1
¸
i=0
[x
i
[
p

1
p
Special cases
• L
1
norm
| x |
1

=
N−1
¸
i=0
[x
i
[
• Sum of the absolute values of the elements
• “City block” distance
• L
2
norm
| x |
2

=

N−1
¸
i=0
[x
i
[
2
• “Euclidean” distance
• Minimized by “Least Squares” techniques
29
• L

-norm
| x |


= lim
p→∞

N−1
¸
i=0
[x
i
[
p

1
p
In the limit as p →∞, the L
p
norm of x is
dominated by the maximum element of x.
30
Filter Design using Lp Norms
In terms of L
p
norm minimization, the FIR filter design
problem becomes:
min
h
|W(ω
k
) [H(ω
k
) −D(ω
k
)]|
p
Where
• ω
k
= suitable discrete set of frequencies
• H(ω
k
) = frequency response of our FIR filter
• D(ω
k
) = desired (complex) frequency response
• W(ω
k
) = (optional) weighting function
Lets look at some specific cases:
31
Least-Squares Linear-Phase FIR Filter Design
Let the FIR filter length be L + 1 samples, with L even,
and suppose we’ll initially design it to be centered about
the time origin (“zero-phase”). Then the frequency
response is given on our frequency grid ω
k
by
H(ω
k
) =
L/2
¸
n=−L/2
h
n
e
−jω
k
n
Enforcing even symmetry in the impulse response, i.e.,
h
n
= h
−n
, gives a zero phase FIR filter which we can
later right-shift L/2 samples to make a causal, linear
phase filter. In this case, the frequency response reduces
to a sum of cosines:
H(ω
k
) = h
0
+ 2
L/2
¸
n=1
h
n
cos(ω
k
n), k = 0, 1, 2, . . . , N −1
or in matrix form:





H(ω
0
)
H(ω
1
)
.
.
.
H(ω
N−1
)





=





1 2 cos(ω
0
) . . . 2 cos[ω
0
(L −1)]
1 2 cos(ω
1
) . . . 2 cos[ω
1
(L −1)]
.
.
.
1 2 cos(ω
N−1
) . . . 2 cos[ω
N−1
(L −1)]






A





h
0
h
1
.
.
.
h
L/2






x
(Note that Remez exchange algorithms are also based on
this formulation internally.)
32
Matrix Formulation: Optimal L
2
Design, Cont’d
In matrix notation, our filter design problem can be stated
min
x
|Ax −b|
2
2
where, for zero-phase filters,
A

=

1 2 cos(ω
0
) . . . 2 cos [ω
0
(L −1)]
1 2 cos(ω
1
) . . . 2 cos [ω
1
(L −1)]
.
.
.
1 2 cos(ω
N−1
) . . . 2 cos [ω
N−1
(L −1)]
¸
¸
¸
¸
x

= h
and b = [D(ω
k
)] is the desired frequency response at the
specified frequencies.
The function firls in the Matlab Signal Processing Tool
Box implements frequency-domain weighted-least-squares
FIR filter design (not available in Octave as of 2/2003).
33
Least Squares Optimization
ˆ x

= arg min
x
| Ax −b |
2
= arg min
x
| Ax −b |
2
2
Hence we can minimize
| Ax −b |
2
2
= (Ax −b)
T
(Ax −b)
Expanding this, we have:
(Ax −b)
T
(Ax −b) = (b
T
−x
T
A
T
)(Ax −b)
This is quadratic in x, hence it has a global minimum
which we can find by taking the derivative, setting it to
zero, and solving for x. Doing this yields:
A
T
Ax −A
T
b = 0
These are the famous normal equations whose solution is
given by:
ˆ x =

(A
T
A)
−1
A
T

b
The matrix
A


= (A
T
A)
−1
A
T
is known as the pseudo-inverse of the matrix A.
34
Geometrical Interpretation of Least Squares
Typically, the number of frequency constraints is much
greater than the number of design variables (filter taps).
In these cases, we have an overdetermined system of
equations (more equations than unknowns). Therefore,
we cannot generally satisfy all the equations, and we are
left with minimizing some error criterion to find the
“optimal compromise” solution.
In the case of least-squares approximation, we are
minimizing the Euclidean distance, which suggests the
following geometrical interpretation:
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Aˆ x
column-space of A
Aˆ x = b + e
Minimize
x
|e|
2
= |b −Ax|
2
35
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Aˆ x
column-space of A
This diagram suggests that the error vector b −Ax is
orthogonal to the column space of the matrix A, hence it
must be orthogonal to each column in A.
A
T
(b −Aˆ x) = 0 ⇒A
T
Aˆ x = A
T
b
This is how the orthogonality principle can be used to
derive the fact that the best least squares solution is
given by
ˆ x = (A
T
A)
−1
A
T
b = A

b
Note that the pseudo-inverse A

projects the vector b
onto the column space of A.
In Matlab:
• x = A ` b
• x = pinv(A) * b
36
Chebyshev Optimal Linear Phase filter Design
Consider the L

-norm minimization problem:
min
x
| Ax −b |

We said earlier that the ∞ norm of a vector is the
maximum element of that vector, hence minimizing the
∞ norm is minimizing the maximum element of a vector:
min
x
max
k
[a
T
k
x −b
k
[
where a
T
k
denotes the kth row of the matrix A.
We can then write this as:
min t
s.t. [a
T
k
x −b
k
[ < t
If we introduce a new variable ˜ x

=
¸
x
t

, then
t = f
T
˜ x = [ 0 . . . 0 1 ] ˜ x, and our optimization problem
becomes:
min
˜ x
f
T
˜ x
s.t.

[ a
T
k
0 ] ˜ x −b
k

< ˜ xf
T
˜ x
37
Hence we are minimizing a linear objective, subject to a
set of linear constraints. This is known as a linear
programming problem (linprog in Matlab).
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Linear
Linear
Objective
Constraints
38
More General Real FIR Filters
So far we have looked at the design of linear phase FIR
filters. In this case, A, x and b are all real.
Sometimes we may want to design filters and specify both
the magnitude and the phase of the frequency response.
Examples:
• Minimum phase filters
• Inverse filters
• Fractional delay filters
• Interpolation polyphase subfilters
39
Complex FIR Filter Design
In linear-phase filter design, we assumed symmetry of our
filter coefficients [h(n) = h(−n)] ⇒
• The filter frequency response became a sum of
cosines (“zero phase”)
• The matrix A was real
• The desired magnitude response b was real
• The final zero-phase filter ˆ x could be right-shifted
L/2 samples to get a corresponding causal
linear-phase FIR filter
Now we would like to specify a complex frequency
response. This means that:
• b is complex
• A is complex
• We still want x (our filter coefficients) to be real
If we try to use ’`’ or pinv in Matlab, we will generally
get a complex result for ˆ x
40
Summarizing our problem:
min
x
| Ax −b |
2
where, A ∈ C
NxM
, b ∈ C
Nx1
, and x ∈ R
Mx1
Hence we have,
min
x
|[{(A) + j1(A)] x −[{(b) + j1(b)]|
2
2
which can be written as:
min
x
| {(A)x −{(b) + j [1(A)x + 1(b)] |
2
2
or
min
x

¸
{(A)
1(A)

x −
¸
{(b)
1(b)

2
2
which is written in terms of only real variables.
Hence, we can use the standard least squares solvers in
Matlab and end up with a real solution.
Related paper
“Design of Fractional Delay Filters Using Convex
Optimization” (Mohonk-97, Music 421 handout):
http://ccrma.stanford.edu/~jos/eps/mohonk97.ps.gz
41
Optimal FIR Filters — Arbitrary Magnitude and
Phase Specificiations
cremez (Matlab Signal Processing Toolbox) performs
complex L

FIR filter design:
• Documented online at The Mathworks
(search for cremez)
• Convergence theoretically guaranteed for arbitrary
magnitude and phase specifications versus frequency.
• Reduces to Parks-McClellan algorithm (Remez second
algorithm) as a special case.
• Written by Karam and McClellan. See “Design of
Optimal Digital FIR Filters with Arbitrary Magnitude
and Phase Responses,” by Lina J. Karam and James
H. McClellan, ISCAS-96. The paper may be
downloaded at
http://www.eas.asu.edu/~karam/papers/iscas96 km.html
42
Second-Order Cone Problems (SOCP)
• A linear function is minimized over the intersection of
an affine set and the product of second-order
(quadratic) cones
• Nonlinear, convex problem including linear and
(convex) quadratic programs as special cases
• Solved by efficient primal-dual interior-point methods
• Number of iterations required to solve a problem
grows at most as the square root of the problem size
• Typical number of iterations ranges between 5 and
50, almost independent of the problem size
See “Applications of second-order cone programming,” by
M. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret,
Linear Algebra and its Applications, 284:193-228,
November 1998, Special Issue on Linear Algebra in
Control, Signals and Image Processing. Available online:
http://www.stanford.edu/~boyd/socp.html
For more on this topic, see also Prof. Boyd’s course on
convex optimization: EE 364
2
and text book (Boyd and
Vandenberghe, Cambridge U. Press, 2004).
2
http://www.stanford.edu/class/ee364/
43
Hilbert Transform Filter Design Example:
A Case Study in FIR Filter Design
Design Example:
• Ideal Single-Sideband Filter (Hilbert transformer)
• Window Method using Kaiser Window
• Optimal Chebyshev using Remez Exchange
44
Problem Statement
• Design a “negative-frequency filter” which converts a
real signal into its complex “analytic signal”
counterpart by filtering out negative frequencies.
• Equivalently, design a “single sideband” (SSB) filter
which outputs a single-sideband signal in response to
a complex-conjugate pair of sidebands (a Hermitian
spectrum).
• We could also call it a “positive-frequency-pass” filter.
• The imaginary part of such a filter is called a
Hilbert transform filter.
• The ideal Hilbert transform is an allpass filter that
delays positive-frequency components by 90 degrees,
and advances negative-frequency components by 90
degrees.
45
Hilbert Transform
y(t)

= (h
i
∗ x)(t) (Hilbert transform of x)
h
i
(t)

=
1
πt
(Hilbert transform “kernel”)
H
i
(ω)

=

−j, ω > 0
j, ω < 0
1, ω = 0
(Hilbert frequency response)
x
a
(t)

= x(t) + jy(t) (Analytic signal from x)
=
1
π


0
X(ω)e
jωt
dω (Note: Lower limit usually −∞)
Proof:
X
a
(ω) = X(ω) + jY (ω) (By linearity of Fourier transform)
= X
+
+ X

(Apply frequency response)
+j[−jX
+
+ jX

]
= 2X
+
(ω)
46
Kaiser Window
Kaiser window in causal, linear-phase form:
w = kaiser(M,beta); % M=length=257, beta=8
Zero-pad by a factor of 8 and convert to zero-phase form
(N = FFT size = 2048):
wzp = [w((M+1)/2:M),zeros(1,N-M),w(1:(M-1)/2)];
Kaiser window transform:
W = fft(wzp);
47
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−160
−140
−120
−100
−80
−60
−40
−20
Kaiser Window Transform, FFT size = 2048, Window length = 257
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
48
Kaiser window transform, close-up on main lobe
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−120
−100
−80
−60
−40
−20
Close Up on Kaiser LF Response, FFT size = 2048, Window length = 257
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
49
Oversimplified Window Method
Sample the ideal Hilbert-transform kernel
h(t) = 1/πt
to get
ˆ
h
i
(n)

=
1
πnT
(Sampled Hilbert transform kernel)
ˆ
h
w
(n)

= w(n)
ˆ
h
i
(n) (Windowed ideal impulse response)
ˆ
H
w
(ω) = (W ∗
ˆ
H)(n) (Smoothed ideal frequency response)
Design Parameters:
fs = 22050; % sampling rate (Hz)
T = 1/fs; % sampling period (sec)
M = 257; % FIR filter length = window length
N = 8*(M-1); % for interpolated spectral displays
beta = 8; % beta for Kaiser window
Filter Design:
n = [-N/2+0.5:N/2-0.5]; % Time axis (avoid t=0)
hi = T ./ (pi*n*T); % Sampled Hilbert kernel
hr = sin(pi*n) ./ (pi*n); % 1/2 sample delay filter
h = (hr + j*hi)/2; % Sampled ideal final filter
plot(f,fftshift(max(-100,20*log10(abs(fft(h)))))); grid;
...
50
FFT of Truncated(2048) Ideal Impulse Response
-0.6 -0.4 -0.2 0.2 0.4 0.6
-100
-80
-60
-40
-20
20
FFT of Truncated Analytically Specified "Ideal" Impulse Response
Normalized Frequency (cycles/sample)
G
a
i
n

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω 0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
h = [h(N/2+1:N),h(1:N/2)]; % zero-phase form (almost)
hw = wzp .* h; % Apply window to ideal impulse response
Hw = fft(hw); % Frequency response we really get
% Compute total stopband attenuation:
ierr = norm(Hw(N/2+2:N))/norm(Hw)
= 0.0378 = 3.8 percent
For spectral displays, rotate buffer so negative frequencies
are on the left and DC is at k = N/2:
51
Hwp = [Hw(N/2+2:N), Hw(1:N/2+1)]; % Neg. freqs on left
Hwpn = abs(Hwp); Hwpn = Hwpn/max(Hwpn);
plot(f,20*log10(Hwpn)); grid;
...
FIR Frequency Response by the Window Method
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−140
−120
−100
−80
−60
−40
−20
Length 257 Analytic−Derived FIR Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
52
Zoom in on low-frequency transition band
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−140
−120
−100
−80
−60
−40
−20
20
Analytic−Derived Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
• Poorly positioned transition region from positive to
negative frequencies.
• Need to first bandpass-filter the ideal impulse
response.
• High-frequency transition identical, by symmetry
53
Ideal Bandlimited Impulse Response
Since we are working with sampled data, the ideal
single-side-band filter impulse response is
h(n) = IDTFT
n
(u)

=
1

π
−π
u(ω)e
jωn

where u(ω) is the unit step function in the frequency
domain:
u(ω)

=

1, ω ≥ 0
0, ω < 0
We can evaluate the IDTFT directly as follows:
h(n) =
1

π
0
e
jωn
dω =
1
2πjn
e
jωn

π
0
=
e
jπn
−1
2πjn
=
(−1)
n
−1
2πjn
=

0, n even, n = 0
j
1
πn
, n odd
For n = 0, going back to the original integral gives
h(0) =
1

π
0
e
jω·0
dω =
1
2
.
Thus, the real part of h(n) is δ(n)/2, and the imaginary
part is 1/(πn) for odd n and zero otherwise (as a result
of bandlimiting). There is no aliasing. However, we also
54
need a transition band. We can work this out analytically,
or we can simply “draw” it in the frequency domain, take
a (large) inverse DFT, and window that. Since the latter
approach is more general, we’ll do that.
Window Method applied to Bandlimited Ideal
Impulse Response
Further Design Parameters:
f1 = 530; % lower passband limit (Hz)
N = 2^(nextpow2(8*fs/f1)) % FFT size from xition width
if N<8*M, N = 8*(M-1); end; % spectral interpolation
Bandlimited “Ideal” Hilbert transformer frequency
response (FR), including transition bands ∆ matched to
FFT size N such that N f
s
/∆:
H = [ ([0:k1-2]/(k1-1)).^8,ones(1,k2-k1+1), ...
([k1-2:-1:0]/(k1-1)).^8,zeros(1,N/2-1)];
Hp = [H(N/2+2:N), H(1:N/2+1)];
f = [-0.5 + 1/N : 1/N : 0.5]; normalized freq axis
plot(f,Hp); grid; ...
55
Desired Frequency Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−150
−100
−50
Desired Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
56
Desired Impulse Response
h = ifft(H); % zero-phase form
% measure of round-off error:
ierr = norm(imag(h(1:2:N)))/norm(h) % zero?
ierr =
4.1958e-15
% rough estimate of time-aliasing error:
aerr = norm(h(N/2-N/32:N/2+N/32))/norm(h)
aerr =
4.8300e-04
% for plots, prefer negative times on the left:
hp = [h(N/2+2:N), h(1:N/2+1)];
57
Real Part of Desired Impulse Response
−150 −100 −50 50 100 150
−0.1
0.1
0.2
0.3
0.4
0.5
0.6
Ideal SSB Filter Impulse Response, Real Part, FFT size = 2048
Time (samples)
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
58
Imaginary Part of Desired Impulse Response
−150 −100 −50 50 100 150
−0.4
−0.3
−0.2
−0.1
0.1
0.2
0.3
0.4
Ideal SSB Filter Impulse Response, Imaginary Part, FFT size = 2048
Time (samples)
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Apply window to ideal impulse response:
hw = wzp .* h; % Final FIR coefficients
Hw = fft(hw); % Frequency response we’ll see
Total stopband energy:
ierr = norm(Hw(N/2+2:N))/norm(Hw)
disp(sprintf([’Stop-band energy is %g percent of’,...
total spectral energy’], 100*ierr));
59
Final FIR Filter Frequency Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−180
−160
−140
−120
−100
−80
−60
−40
−20
Length 257 FIR filter Frequency Response, FFT size = 2048
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
60
Zoom-in on Transition Region of Final Frequency
Response
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−140
−120
−100
−80
−60
−40
−20
20
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
• Transition band better positioned
• ≈ 100 dB stob-band attenuation
61
Remez Multiple Exchange Method
• Remez exchange algorithm for optimal Chebyshev
(equiripple) FIR filter design.
• Original reference: J. H. McClellan, T. W. Parks, and
L. R. Rabiner, “A Computer Program for Designing
Optimum FIR Linear Phase Digital Filters,” IEEE
Trans. Audio Electro., vol AU-21, Dec. 1973,
pp. 506-526.
• Text reference: L. R. Rabiner and B. Gold, Theory
and Application of Digital Signal Processing,
Prentice-Hall, 1975.
• Pertinent recent reference: A. Reilly, G. Frazer, and
B. Boashash, “Analytic Signal Generation—Tips and
Traps,” IEEE Tr. Signal Processing, vol. 42, no. 11,
Nov. 1994.
Design an optimal lowpass filter of the desired width
(takes quite a while):
hrm = remez(M-1, [0,(f2-fs/4)/fn,0.5,1], ...
[1,1,0,0], [1,10]);
The weighting [1,P] says make the passband ripple P
times that of stopband. For steady-state audio spectra,
62
passband ripple can be 0.1 dB or more. However,
consider an FM signal in which a sinusoid is sweeping
back and forth in the passband. In that case, the
passband ripple generates AM sidebands, so a spec more
like that in the stopband may be called for. Here, we
allow the passband ripple to be 10 times the stopband
ripple, as a compromise.
Modulate it up to where it’s a single-sideband filter. I.e.,
right-shift by π/2 in the frequency domain. I.e., rotate
the frequency response counterclockwise along the unit
circle by 90 degrees:
hr = hrm .* j .^ [0:M-1];
63
Optimal Chebyshev FIR(257) Frequency
Response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−200
−150
−100
−50
50
Length 257 Optimal Chebyshev FIR Single−Sideband−Filter Freq. Response
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
64
Zoom-In on Transition Band
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−200
−150
−100
−50
50
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
65
Passband Ripple
0.18 0.2 0.22 0.24 0.26 0.28 0.3 0.32
−4
−3
−2
−1
1
2
3
4
x 10
−4
Passband Ripple
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
66
Initial Impulse Response
5 10 15 20 25 30 35 40 45 50
−8
−6
−4
−2
2
4
6
8
x 10
−4
First 50 impulse−response coefficients
Index
A
m
p
lit
u
d
e
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
67
Summary of Trade-Offs
Window-method filter:
• Stopband ”droops” which ”wastes” filter taps if the
worst-case stopband attenuation is the only stopband
spec. However, such ”roll-off” is natural in the
frequency domain and corresponds to a ”smoother
pulse” in the time domain.
• Passband is degraded by early roll-off. Edge is off.
• Filter length can be thousands of taps as needed for
full audio band.
Optimal Remez-Exchange filter:
• Stopband is ideal, equiripple.
• Transition region close to half that of the window
method.
• Pass-band is ideal, equiripple.
• Time-domain impulses from passband ripple are small.
• Design time orders of magnitude larger than that for
window method.
68
• Fails to converge for filters much longer than 256
taps. (Need to increase working precision to get
longer filters.)
69
Matlab hilbert() function
The Matlab Hilbert function returns an analytic signal
given its real part.
Nh = M-2; % This looks best
delta = [1,zeros(1,Nh)]; % zero-phase impulse
hh = hilbert(delta); % zeros negative-freq FFT bins
Hh = fft(hh); % FIR frequency response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−350
−300
−250
−200
−150
−100
−50
Frequency Response of Length 256 hilbert() using Matlab
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Looks pretty good, but let’s zero-pad the impulse
response to interpolate the frequency response:
70
hhzp = [hh(Lh/2+1:Lh), zeros(1,N-Lh), hh(1:Lh/2)];
Hhzp = fft(hhzp); % Frequency response
−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5
−350
−300
−250
−200
−150
−100
−50
Interpolated Frequency Response of Length 256 hilbert() using Matlab
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
71
−0.05 −0.04 −0.03 −0.02 −0.01 0.01 0.02 0.03 0.04 0.05
−350
−300
−250
−200
−150
−100
−50
50
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
Normalized Frequency (cycles/sample)
G
a
in

(
d
B
)
PSfrag replacements
ω
π
0
[H[

−π
−2π
ω
0
0
[H[
π
ω
c
ω
0
ω
u
ω
l
−π
−ω
c
−ω
0
Problems:
• Transition bandwidth only 2 bins wide.
• Massive time aliasing.
• Only justifiable for periodic signals with period exactly
equal to filter length Nh. In this case only, time
aliasing is equivalent to overlap-add from adjacent
periods.
72
More generally, any real signal given to hilbert() must be
interpreted as precisely one period of a periodic signal.
73