Statistical Digtal Signal Processing (EE4C03)

Non-Parametric Spectrum Estimation (chapter 8.2)

1 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Given a signal . We would like to show its “spectrum”.

If is very long (e.g. an hour of music...) it does not make sense to compute
the
Fourier transform—the signal is nonstationary.

Typically we break up the signal into shorter segments and compute the Fourier
transform of each segment (time-frequency analysis).


 otherwise

How does this relate to the Fourier transform of the original signal?

2 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Describe the domain restriction by a multiplication with a rectangular window:



 otherwise

Multiplication in time domain is convolution in frequency domain:

where

This is the Dirichlet kernel (generalized sinc function)

3 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Rectangular window

N=19
0.8

0.6

0.4

W(ω
)
0.2

−0.2

−0.4
−2pi −pi 0 pi 2pi
ω

The first zero crossing of is at . The highest sidelobe occurs for

and is at dB of the main lobe.

The window gives distortions: smearing (resulting in loss of resolution) and side-
lobe interference (resulting in confusion)
4 8a non-parametric spectrum est. October 1, 2015
Spectral analysis

Example:

10 cos(ω0 n)
N=19 w[n]

5
R e ( X (ω))
^

−
5−p −pi/ 0 pi/2 pi
i 2 ω

5 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Spectrum via DFT

In practice, we use the DFT (or FFT). This corresponds to sampling .

, with

Because is periodic (period ) we can also take ( )

corresponding with

6 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Example

Consider the following signal . What is its frequency?

cos(ω0 n)
0
x[n]

−1
0 5 10 15 20
n
10 N=19
L=19
|X(ω)|

0
−pi −pi/2 0 pi/2 pi
ω

7 8a non-parametric spectrum est. October 1, 2015

Spectral analysis
To get a smoother spectrum, we can use zero padding, i.e., with
extend
additional zeros to length . (The amount of information will not change)

0.5 cos(ω 0 n)

x [n] 0
^

−0.5

−1
0 5 10 15 20 25 30 35 40 45 50
n
10 N=19
L=50
|X(ω)|

0
−pi −pi/2 0 pi/2 pi
ω

Spectral “leak” (distortion) by the side lobes of the window

8 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Sum of two frequencies

2

1 cos(ω 1 n) + cos(ω 2 n)

x [n]
0

^
−1

−2
0 10 20 30 40 50 60
n
20 N=20 omega1=0.2 ⋅ 2π
L=500 omega2=0.22 ⋅ 2π
15 1/N=0.05
|X(ω)|

10

0
−pi −pi/2 0 pi/2 pi
ω

The first zero crossing of is at . This determines the resolution.

Frequencies with differences smaller than (or ) cannot be
resolved.

9 8a non-parametric spectrum est. October 1, 2015

Spectral analysis
For better resolution: enlarge (number of informative samples):

2
cos(ω1 n) + cos(ω 2 n)
1

x^[n]
0

−1

−2
0 10 20 30 40 50 60
n
20 N=40 Rectangular windowed
omega1=0.2 ⋅ 2π
L=500 omega2=0.22 ⋅ 2π
15 1/N=0.025
|X (ω)|

10
^

0
−pi −pi/2 0 pi/2 pi
ω

The resolution is further limited by the height of the side lobes (confusion).

10 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Side lobe reduction

Other windows can be designed to obtain lower side lobes, at the expense of a
wider main lobe. E.g., Bartlett, Hann, Hamming, Blackman, Kaiser, etc.

Bartlett window

0.5

0
0 2 4 6 8 10 12 14 16 18
n
1 N=19

Bartlett window
|W(ω)|

0.5 (rectangular window)

0
−pi −pi/2 0 pi/2 pi
ω

11 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Bartlett window
2

1 cos(ω1 n) + cos(ω 2 n)

x^[n]
0

−1

−2
0 20 40 60 80 100 120
n
8 N=40 Bartlett windowed
omega1=0.2 ⋅ 2π
L=500 omega2=0.22 ⋅ 2π
6 1/N=0.025
|X (ω)|

4
^

0
−pi −pi/2 0 pi/2 pi
ω

Lower side lobes at the expense of some resolution

12 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Hamming window

Hamming window

0.5

0
0 2 4 6 8 10 12 14 16 18
n

1 N=19

Hamming window
|W(ω)|

0.5 (rectangular window)

0
−pi −pi/2 0 pi/2 pi
ω

13 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Hamming window

1 cos(ω1 n) + cos(ω 2 n)
x^[n]
0

−1

−2
0 20 40 60 80 100 120
n
8 N=40 Hamming windowed
omega1=0.2 ⋅ 2π
L=500 omega2=0.22 ⋅ 2π
6 1/N=0.025
|X (ω)|

4
^

0
−pi −pi/2 0 pi/2 pi
ω

14 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Wide Sense Stationary random signal

Example: communication signal (binary phase shift keying, with random bits )
bpsk signaal
2

x[n] −1

−2
0 10 20 30 40 50 60 70 80 90 100
n
40
N=100
30
|X(ω)|

20

10

0
−pi −pi/2 0 pi/2 pi
ω

Applying the DFT directly on the signal gives a chaotic picture. Do the
peaks correspond to certain frequencies?

15 8a non-parametric spectrum est. October 1, 2015

Spectral analysis
bpsk signaal
2

x[n]
−1

−2
0 100 200 300 400 500 600 700 800 900 1000
n
200
N=1000
150
|X(ω)|

100

50

0
−pi −pi/2 0 pi/2 pi
ω

A larger gives points in the frequency domain, but the spectrum does not
converge. We need to average.

16 8a non-parametric spectrum est. October 1, 2015

 Spectral analysis
2 2 2 2 2

1 1 1 1 1

0 0 0 0 0

−1 −1 −1 −1 −1

−2 −2 −2 −2 −2
0 50 100 100 150 200 200 250 300 300 350 400 400 450 500
n n n n n
20 40 30 30 40

15 30 30
20 20
10 20 20
10 10
5 10 10

0 0 0 0 0
−pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi
ω ω ω ω ω

Separate the signal into segments, each of length (here ). Do a

windowed DFT on each segment. Compute the amplitude-square, and average.
This gives the averaged periodogram (a.k.a. Welch method).

17 8a non-parametric spectrum est. October 1, 2015

Spectral analysis

Averaged periodogram

1
N=10000 bpsk spectrum
L=100 single pulse (dirichlet)
0.8 Ts=6

0.6 Hamming window

0.4

0.2

0
−pi −pi/2 0 pi/2 pi

The spectrum is seen to converge to that of the time-domain pulse (a

rectangular
pulse Dirichlet-kernel)

18 8a non-parametric spectrum est. October 1, 2015

Random process
After this introduction, we now follow the theory in Hayes, Ch. 8.2.

Given a random sequence, how can we meaningfully estimate its spectrum?

Definition

For an autocorrelation ergodic process:

Hence: given for all , to compute the power spectrum,

determine

compute its Fourier transform

But what if we have data only over a finite interval ( )?

19 8a non-parametric spectrum est. October 1, 2015

Periodogram
For finite data ( ),

Use a finite sum,

Exclude data outside the interval (equivalent to setting it to zero):

(as we see later, this is a biased estimate)

Use also conjugate symmetry: , and set for .

Then the periodogram is defined as

20 8a non-parametric spectrum est. October 1, 2015

Periodogram
Expressed in terms of the original data: define


 otherwise

so that , where is a rectangular window. Then

(The notation is theoretically incorrect and sometimes

confusing. . . )
Taking the Fourier transform gives

where

Thus, the periodogram can be computed from the DFT of .

21 8a non-parametric spectrum est. October 1, 2015

Periodogram

Performance of the periodogram

Ideally, a (spectrum) estimate is consistent:

asymptotically unbiased:

variance goes to 0 as : Var

Bias of the periodogram

and for .

Thus where is a Bartlett window:





Therefore, is a biased estimate.

22 8a non-parametric spectrum est. October 1, 2015

Periodogram
It follows for the periodogram that

so that

where is the Fourier transform of the Bartlett window,

(Note: it is times the square of the Dirichet window we saw earlier;

)

The periodogram is biased. But as , converges to an impulse,

and the periodogram is asympotically unbiased:

23 8a non-parametric spectrum est. October 1, 2015

Periodogram

Effect of the convolution by

Spectral smoothing (smearing): the response to a signal consisting of a single

frequency is not an impulse, , with a bandwidth of .
but approx.

Confusion: due to sidelobes of at , one might think there

are additional frequencies.

This limits the resolution: ability to resolve closely spaced sinusoids. The
resolution
is defined by the half-power width of the main lobe,

Res

24 8a non-parametric spectrum est. October 1, 2015

Periodogram

Variance of the periodogram

The computation of the variance is involved. One finds

Var

(exact for white Gaussian noise)

This is not a function of . the variance does not go to 0 as and the

periodogram is not a consistent estimate of the power spectrum.

bpsk signaal
2

0
x[n]

−1

−2
0 100 200 300 400 500 600 700 800 900 1000
n
200
N=1000
150

100
|X(ω)|

50

0
−pi −pi/2 0 pi/2 pi
ω

25 8a non-parametric spectrum est. October 1, 2015

Periodogram

Alternative derivation of the same

Describe the domain restriction by a multiplication with a rectangular window:



 otherwise

Multiplication in time domain is convolution in frequency domain:

where

This is the Dirichlet kernel (generalized sinc function)

26 8a non-parametric spectrum est. October 1, 2015

Modified Periodogram
Recall

for which (derivation see Hayes p.408)

(note that this is consistent to the earlier )

result
Instead of a rectangular window, try other windows , e.g. a Bartlett window,

where is a suitable normalization constant (energy of the window),

This is called the modified periodogram.

27 8a non-parametric spectrum est. October 1, 2015

Modified Periodogram

Bias

A derivation similar as before gives

where is the Fourier transform of .

Since (by Parseval)

it follows that

so that will converge to a unit impulse.

For , the modified periodogram is asympotically unbiased.

28 8a non-parametric spectrum est. October 1, 2015

Modified Periodogram

Variance

The window does not change the situation,

Var

Thus, the modified periodogram is not a consistent estimate of the power

spectrum.
Effect of the window

The window gives a way to trade-off the spectral resolution (width of main lobe)
with
the confusion (sidelobe amplitude).

Sidelobe (dB) 3dB BW

Rectangular -13
Bartlett -27
Hann -32
Hamming -43
Blackman -58

29 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging

Our aim is to find a consistent estimate for . Recall

It is sufficient to find a consistent estimate for the mean, . This can

be done by averaging several periodograms (based on independent realizations
of the same random process).

Let , be uncorrelated realizations of over the interval

. We have . Compute the periodogram of each realization:

Then compute the average:

30 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging
The resulting averaged periodogram estimate is

Bias

with based on a rectangular window with samples.

As before, the estimate is asympotically unbiased (as ).

Variance

Var Var

This goes to as .

Thus, is a consistent estimate if both and go to infinity.

31 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging

Bartlett’s method

For a given , , split it nonoverlapping segments

sequence into
of length where ,

then compute the averaged periodogram,

2 2 2 2 2

1 1 1 1 1

0 0 0 0 0

−1 −1 −1 −1 −1

−2 −2 −2 −2 −2
0 50 100 100 150 200 200 250 300 300 350 400 400 450 500
n n n n n
20 40 30 30 40

15 30 30
20 20
10 20 20
10 10
5 10 10

0 0 0 0 0
−pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi −pi −pi/2 0 pi/2 pi
ω ω ω ω ω

32 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging: Bartlett’s method

Bias

Precisely as before,

where based on samples. The estimate is asympotically unbiased.

The resolution is now determined by :

Res

which is times worse than for the periodogram.

Variance

Var Var

Thus, is a consistent estimate if both and go to infinity.

33 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging

Welch method: averaged modified periodograms

Two modifications:

Use overlapping segments (each of length , but spaced by ),

The total number of data points that is used is

Often, 50% overlap is used: .

Use data windows instead of simple rectangular windowing

The resulting Welch’s estimate is given by

34 8a non-parametric spectrum est. October 1, 2015

Periodogram averaging: Welch Method

Bias

where is the Fourier transform of the window (length )

Variance

One shows that, for 50% overlap (some correlation among the sequences )

Var

This is a factor worse than for Bartlett’s estimate, for the same . But, twice
as many segments can be averaged ( doubles), so in fact the variance is a
factor
better:
Var

35 8a non-parametric spectrum est. October 1, 2015