You are on page 1of 643

Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

PDS

2018-11-19

Slides mainly compiled from [1] and [2]

1 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

1 Introduction to digital signal processing


Signal
Discrete-time signals
Complex Exponentials
2 Vectors spaces and DSP
Signal Processing and vector spaces
Vector Spaces
Signal Spaces
Bases
3 Basic of Fourier Analysis
Introduction to Fourier Analysis
The Discrete Fourier Transform
Examples of DFT calculation
Interpreting a DFT plot
DFT analysis
DFT synthesis
2 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering
THe STFT
The Spectrogram
Time-frequency tiling

4 Advanced Fourier Analysis


DFS
DTFT
Existence and properties of the DFTF
The DTFT as a change of basis

5 Introduction to Filtering
Linear Filters
Filtering: Intuition
Filter Stability
Frequency Response
Ideal Filters

3 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Introduction to digital signal processing

4 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Signal

Signal: Description of the evolution of a physical phenomenon.


weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market

5 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Signal

Signal: Description of the evolution of a physical phenomenon.


weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market

6 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Signal

Signal: Description of the evolution of a physical phenomenon.


weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market

7 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Signal

Signal: Description of the evolution of a physical phenomenon.


weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market

8 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Signal

Signal: Description of the evolution of a physical phenomenon.


weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market

9 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Processing

Analysis
understanding the information carried by the signal
extracting features from the signal
Synthesis
creating a signal to contain the given information (e.g. to transmit information)

10 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog Signals (physics)

11 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog Signals (electronics)

12 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog Signals

f :R→R

13 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog Signals (physics)

only 1 degree of freedom: vy

14 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog Signals (electronics)

only 2 degrees of freedom: R, C

15 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What about “interesting” signals?

f (t) =?

16 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

To each analog signal, a different device

17 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

From analog ...

18 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

... to digital

... 12 −45 33 41 −89 60 70 −99 ...

19 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Digital Paradigm

Key ingredients
discrete time
discrete amplitude

20 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Digital Paradigm

21 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discretizing time

What is time?

22 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Two competing models for reality

23 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Two competing models for reality

24 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete time is very practical! Computing the average

25 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete time is very practical! Computing the average

26 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete time is very practical! Computing the average

27 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete time is very practical! Computing the average

28 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The discrete-time model

29 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The discrete-time model

x[n] = . . . , 1.2390, −0.7372, 0.8978, 0.1498, −1.1501, −0.2642

30 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How can we go from continuous to discrete time?

x[n] =?

31 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time: the founding fathers

32 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Sampling Theorem (1920)

Under the appropriate “slowness” conditions for x(t) we have:

∞  
X t − nTs
x (t) = x[n] sinc
n=−∞
Ts

33 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Sinc: the building block

34 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

35 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

36 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

37 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

38 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

39 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

from continuous to discrete time (and back)

40 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

When can we do all this? Joseph Fourier will tell us

41 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

Key ingredients
discrete time
discrete amplitude (Quantization)

42 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

43 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

44 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

45 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

46 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signals

Quantization: Each sample can take values from a predetermined set of possible
levels.
The set of levels is countable.
47 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital amplitude

Why it is important
storage
General purpose devices (e.g. computer memory)
processing
Does not require specific devices (e.g. hardware audio equalizers, analog
thermostats)
Processing is performed in software
transmission

48 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analog storage

49 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital storage

{0, 1}

50 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Processing

51 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Data transmission

52 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What happens to analog signals

53 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What happens to analog signals

54 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What happens to analog signals

55 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What happens to analog signals

56 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

We can amplify to compensate attenuation

57 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

58 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

59 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

60 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

61 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

62 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmitting a signal overseas

63 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmitting a signal overseas

For a long, long channel we need repeaters

64 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

65 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

66 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

67 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

68 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

69 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

70 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of analog signals

71 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

In digital signals we can threshold

72 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of quantized signals

73 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of quantized signals

74 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of quantized signals

75 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Transmission of quantized signals

76 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital data throughputs

Transatlantic cable:
1866: 8 words per minute (≈ 5 bps)
1956: AT&T, coax, 48 voice channels (≈ 3 Mbps)
2005: Alcatel Tera10, fiber, 8.4 Tbps (8.4 × 1012 bps)
2012: fiber, 60 Tbps
Voiceband modems
1950s: Bell 202, 1200 bps
1990s: V90, 56 Kbps
2008 ADSL2+, 24Mpbs

77 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital Signal Processing: key ideas

discretization of time
samples replace idealized models
simple math replaces calculus
discretization of amplitude:
general-purpose storage
general purpose processing (CPU)
noise can be controlled

78 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-time signals

Discrete-time signal: a sequence of complex numbers


one dimension (for now)
notation: x[n]
two-sided sequences: x : Z → C
n is a-dimensional “time”

79 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The delta signal

x[n] = δ[n]

80 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How do you synchronize audio and video ...

81 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How do you synchronize audio and video ...

82 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The unit step

x[n] = u[n]

83 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A switch

84 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The exponential decay

x[n] = |a|n u[n], |a| < 1

85 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How fast does your coffee get cold ...

86 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Also, how fast a capacitor discharges ...

87 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The sinusoid

x[n] = sen(ω0 n + θ)

88 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Oscillations are everywhere

89 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Four signal classes

finite-length
infinite-length
periodic
finite-support

90 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length signals

notation: x[n], n = 0, 1, . . . , N − 1
vector notation: x = [ x0 x1 . . . xN−1 ]T
practical entities, good for numerical packages (e.g.matlab, numpy)

91 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length signals

notation: x[n], n = 0, 1, . . . , N − 1
vector notation: x = [ x0 x1 . . . xN−1 ]T
practical entities, good for numerical packages (e.g.matlab, numpy)

92 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length signals

notation: x[n], n = 0, 1, . . . , N − 1
vector notation: x = [ x0 x1 . . . xN−1 ]T
practical entities, good for numerical packages (e.g.matlab, numpy)

93 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Infinite-length signals

sequences notation: x[n], n ∈ Z


abstraction, good for theorems
They do not depend on the length of the signal N

94 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Infinite-length signals

sequences notation: x[n], n ∈ Z


abstraction, good for theorems
They do not depend on the length of the signal N

95 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Periodic signals

N-periodic sequence: x̃[n] = x̃[n + kN], n, k, N ∈ Z


same information as finite-length of length N
“natural” bridge between finite and infinite lengths

96 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Periodic signals

N-periodic sequence: x̃[n] = x̃[n + kN], n, k, N ∈ Z


same information as finite-length of length N
“natural” bridge between finite and infinite lengths

97 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Periodic signals

N-periodic sequence: x̃[n] = x̃[n + kN], n, k, N ∈ Z


same information as finite-length of length N
“natural” bridge between finite and infinite lengths

98 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-support signals

Finite-support sequence:

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

same information as finite-length of length N


anther bridge between finite and infinite lengths

99 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-support signals

Finite-support sequence:

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

same information as finite-length of length N


anther bridge between finite and infinite lengths

100 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-support signals

Finite-support sequence:

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

same information as finite-length of length N


anther bridge between finite and infinite lengths

101 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Elementary operators

scaling:
y [n] = αx[n]

sum:
y [n] = x[n] + z[n]

product:
y [n] = x[n] · z[n]

shift by k (delay):
y [n] = x[n − k]

102 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Elementary operators

scaling:
y [n] = αx[n]

sum:
y [n] = x[n] + z[n]

product:
y [n] = x[n] · z[n]

shift by k (delay):
y [n] = x[n − k]

103 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Elementary operators

scaling:
y [n] = αx[n]

sum:
y [n] = x[n] + z[n]

product:
y [n] = x[n] · z[n]

shift by k (delay):
y [n] = x[n − k]

104 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Elementary operators

scaling:
y [n] = αx[n]

sum:
y [n] = x[n] + z[n]

product:
y [n] = x[n] · z[n]

shift by k (delay):
y [n] = x[n − k]

105 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

106 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

107 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

108 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

109 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

110 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: finite support

111 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: periodic extension

112 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: periodic extension

113 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: periodic extension

114 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: periodic extension

115 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Shift of a finite-length: periodic extension

116 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Energy and power


X
Ex = |x [n]|2
n=−∞

N
1 X
Px = lim |x [n]|2
N→∞ 2N + 1
n=−N

117 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Energy and power: periodic signals

Ex̃ = ∞

N−1
1 X
Px̃ = |x̃ [n]|2
N
n=0

A signal can be an energy signal, a power signal, or neither type:


power signals has finite Px and infinite Ex (e.g. periodic signals)
Energy signals has finite Ex and Px = 0

118 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital vs physical frequency

Discrete time
n: no physical dimension (just a counter)
periodicity: how many samples before pattern repeats
Physical world
periodicity: how many seconds before pattern repeats
frequency measured in Hz

119 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How your PC plays sounds

120 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital vs physical frequency

set Ts , time in seconds between samples


periodicity of M samples → periodicity of MTs seconds
real world frequency:
1
f = in Hz
MTs

121 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital vs physical frequency

usually we choose Fs the number of samples per second


Ts = 1/Fs
E.g. for a typical value Fs = 48000, Ts ≈ 20.8µs. if M = 110, then
1
f = MTs ≈ 440 Hz

122 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Digital vs physical frequency

Suppose the system clock of your computer has an associated frequency of 44 kHz.
What is the period of a discrete-time sinusoid stored on your computer that
corresponds to 880 Hz?
Answer: M = 50

123 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Building blocks: Adder

124 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Building blocks: Multiplier

125 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Building blocks: Delay

126 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The 2-point moving average

moving average: take a “local” average

x [n] + x [n − 1]
y [n] =
2

127 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The 2-point Moving Average DSP Blocks

128 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Moving average example

x[n] = δ[n]

129 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What if we reverse the loop?

130 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What if we reverse the loop? Recursion

y [n] = x[n] + αy [n − 1]

131 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How we solve the chicken-and-egg problem

Zero initial Conditions


set a start time (usually n0 = 0)
assume input and output are zero for all time before n0

132 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Recursion: the compound interest problem

Consider the following compound interest problem:


constant interest rate of 5% per year
deposits/withdrawals during year n: x[n]
balance at year n:
y [n] = 1.05y [n − 1] + x [n]

133 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Accumulation of interest: first-order recursion

y [n] = 1.05y [n − 1] + x [n]

134 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: the one-time investment

x [n] = 100δ [n]


y [n] = 1.05y [n − 1] + x [n]

135 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

An interesting generalization

y [n] = αy [n − M] + x[n]

136 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Creating loops

137 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

M = 3, α = 1, x[n] = δ[n] + 2δ[n − 1] + 3δ[n − 2]

138 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

We can make music with that!

build a recursion loop with a delay of M


choose a signal x̄ [n] that is nonzero only for 0 ≤ n < M
choose a decay factor
input x̄ [n] to the system
play the output

139 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Playing a sine wave

M = 100, α = 1, x̄ [n] = sen(2πn/100) for 0 ≤ n < 100 and zero elsewhere.

140 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Introducing some realism

M controls frequency (pitch)


α controls envelope (decay)
x̄[n] controls color (timbre)

141 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A proto-violin

M = 100, α = 0.95, x̄ [n] : zero-mean sawtooth wave for 0 ≤ n < 100 and zero
elsewhere.

142 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Karplus-Strong Algorithm

M = 100, α = 0.9, x̄ [n]: 100 random values between 0 and 99, zero elsewhere.

143 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Complex Exponentials

e jx
Oscillations are everywhere
sustainable dynamic systems exhibit oscillatory behavior
intuitively: things that don’t move in circles can’t last:
bombs
rockets
human beings ...

144 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Continuous-time oscillations

145 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Continuous-time oscillations

146 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-time oscillations

a frequency ω (units: radians)


an initial phase φ (units: radians)
an amplitude A

x[n] = Ae j(ωn+φ) = A[cos(ωn + φ) + j sen(ωn + φ)]

147 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Why complex exponentials?

we can use complex numbers in digital systems, so why not?


it makes sense: every sinusoid can always be written as a sum of sine and cosine
math is simpler: trigonometric becomes algebra

148 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Why complex exponentials?

we can use complex numbers in digital systems, so why not?


it makes sense: every sinusoid can always be written as a sum of sine and cosine
math is simpler: trigonometric becomes algebra

149 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Why complex exponentials?

we can use complex numbers in digital systems, so why not?


it makes sense: every sinusoid can always be written as a sum of sine and cosine
math is simpler: trigonometric becomes algebra

150 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential

e jα = cos(α) + j sen(α)

151 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential

rotation: z0 = ze jα

152 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

153 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

154 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

155 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

156 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

157 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

158 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

159 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

160 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

161 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

162 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

163 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

164 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The complex exponential generating machine

x[n] = e jωn ; x[n + 1] = e jω x[n]

165 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Initial phase

x[n] = e jωn+φ ; x[n + 1] = e jω x[n], x[0] = e jφ

166 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Initial phase

x[n] = e jωn+φ ; x[n + 1] = e jω x[n], x[0] = e jφ

167 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

x[n] = e jωn ; x[n + 1] = e jω x[n]

168 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

x[n] = e jωn ; x[n + 1] = e jω x[n]

169 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

x[n] = e jωn ; x[n + 1] = e jω x[n]

170 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

x[n] = e jωn ; x[n + 1] = e jω x[n]

171 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

x[n] = e jωn ; x[n + 1] = e jω x[n]

172 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Careful: not every sinusoid is periodic in discrete time

M
e jωn periodic in n ⇔ ω = 2π, for M, N ∈ N
N

173 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2π-periodicity

e jα = e j(α+2kπ) ∀k ∈ Z

174 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2π-periodicity: one point, many names (ALIASING)

175 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2π-periodicity: one point, many names (ALIASING)

176 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2π-periodicity: one point, many names (ALIASING)

177 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2π-periodicity: one point, many names (ALIASING)

178 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

Wagon-wheel effect:
https://www.youtube.com/watch?v=jHS9JGkEOmA

179 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remember the complex exponential generating machine

180 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

0 ≤ ω < 2π

181 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/12

182 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/6

183 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/5

184 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/4

185 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/2 = π

186 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/2 = π

187 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

ω = 2π/2 = π

188 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

How “fast” can we go?

π < ω < 2π

189 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Let’s go really too fast

ω = 2π − α, αsmall

190 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Let’s go really too fast

ω = 2π − α, αsmall

191 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector Spaces and DSP

192 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Prerequisite Warning!

  
a0,0 a0,1 ··· a0,N−1 x0
 a1,0 a1,1 ··· a1,N−1  x1 
 = Ax
  
 .. .. .. ..  ..
 . . . .  . 
aM−1,0 aM−1,1 · · · aM−1,N−1 xN−1

193 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Prerequisite Warning!

 T
x+y = x0 + y0 x1 + y1

194 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A generic discrete-time signal

x[n] = 1.1, 2.2., −1.0, 3.3, · · ·

finite length?
infinite length?
periodic?
finite support?
We need a common framework: vector space

195 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Why using vector space in DSP?

Easier math and unified framework for signal processing:


same framework for different classes of signals
same framework for continuous-time signals
easy explanation of the Fourier Transform
easy explanation of sampling and interpolation
useful in approximation and compression
fundamental in communication system design

196 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Why using vector space in DSP?

vector spaces are very general objects


vector spaces are defined by their properties
once you know the properties are satisfied, you can use all the tools for the space

197 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analogy #1: Object Oriented Programming

198 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analogy #1: Object Oriented Programming

199 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analogy #2: LEGO

basic building block:

200 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analogy #2: LEGO

scaling (4x2):

201 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Analogy #2: LEGO

adding:

202 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector Spaces

Some spaces should be very familiar:


R2 , R3 : Euclidean space, geometry
RN , CN : linear algebra

Others perhaps not so much . . .


`2 (Z): space of square-summable infinite sequences
L2 ([a, b]): space of square-integrable functions over an interval
yes, vectors can be functions!

203 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector Spaces

Some spaces should be very familiar:


R2 , R3 : Euclidean space, geometry
RN , CN : linear algebra

Others perhaps not so much . . .


`2 (Z): space of square-summable infinite sequences
L2 ([a, b]): space of square-integrable functions over an interval
yes, vectors can be functions!

204 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector Spaces

Some spaces should be very familiar:


R2 , R3 : Euclidean space, geometry
RN , CN : linear algebra

Others perhaps not so much . . .


`2 (Z): space of square-summable infinite sequences
L2 ([a, b]): space of square-integrable functions over an interval
yes, vectors can be functions!

205 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Some spaces can be represented graphically

T
R2 :

x= x0 x1

206 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Some spaces can be drawn

T
R3 :

x= x0 x1 x2

207 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Some spaces can be drawn

L2 [−1, 1] : x = x(t), t ∈ [−1, 1]

208 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Others can’t

RN , for N > 3
CN , for N > 1

209 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector spaces: operational definition

Ingredients:
the set of vector V
a set of scalars (say C)

We need at least to be able to:


resize vectors, i.e. multiply a vector by a scalar
combine vectors together, i.e. sum them

210 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector spaces: operational definition

Ingredients:
the set of vector V
a set of scalars (say C)

We need at least to be able to:


resize vectors, i.e. multiply a vector by a scalar
combine vectors together, i.e. sum them

211 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Formal properties of a vector space

For x, y, z ∈ V and α, β ∈ C
x+y =y+x
(x + y) + z = x + (y + z)
α(x + y) = αx + αy
(α + β)x = αx + βx
α(βx) = (αβ)x
∃0 ∈ V | x+0=0+x=x
∀x ∈ V ∃(−x) | x + (−x) = 0

212 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Example: scalar multiplication in R

 T
αx = αx0 αx1

213 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Example: addition in R

x + y = [ x0 + y0 x1 + y1 ]T

214 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: scalar multiplication in L2 [−1, 1]

αx = αx(t)

215 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: Addition in L2 [−1, 1]

x + y = x(t) + y (t)

216 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: Addition in L2 [−1, 1]

x + y = x(t) + y (t)

217 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector spaces: we need something more

the set of vectors V


a set of scalars (say C)
scalar multiplication
addition
We need something to measure and compare:
inner product (aka dot product)

218 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Vector spaces: we need something more

the set of vectors V


a set of scalars (say C)
scalar multiplication
addition
We need something to measure and compare:
inner product (aka dot product)

219 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product

h·, ·i : V × V → C

measure of similarity between vectors


inner product is zero? vectors are orthogonal (maximally different)

220 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Formal properties of the inner product

For x, y, z ∈ V and α, β ∈ C

221 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Inner product in R : the norm

hx, xi = x02 + x12 = kxk2

222 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Inner product in R

hx, yi = x0 y0 + x1 y1 = kxk kyk cos(α)

223 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Inner product in R : orthogonality

hx, yi = x0 y0 + x1 y1 = kxk kyk cos(α)

224 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1]

ˆ1
hx, yi = x (t) y (t) dt
−1

225 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1] : the norm

ˆ1
2
hx, xi = kxk = sen2 (πt) dt = 1
−1

226 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1] : the norm

ˆ1
2
kyk = t 2 dt = 2/3
−1

227 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1]

ˆ1 p
hx, yi = 3/2t sen (πt) dt ≈ 0.78
−1

228 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1]

x, y from orthogonal subspaces: hx, yi = 0

229 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1]

sinusoids with frequencies integer multiples of a fundamental

230 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product in L2 [−1, 1]

sinusoids with frequencies integer multiples of a fundamental

231 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Norm vs Distance

p
inner product defines a norm: kxk = hx, xi
norm defines a distance: d (x, y) = kx − yk

232 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Norm vs Distance

p
inner product defines a norm: kxk = hx, xi
norm defines a distance: d (x, y) = kx − yk

233 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Norm and distance in R

q
kx − yk = (x0 − y0 )2 + (x1 − y1 )2

234 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Distance in L2 [−1, 1] : the Mean Square Error

ˆ1
2
kx − yk = |x (t) − y (t)|2 dt
−1

235 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Distance in L2 [−1, 1] : the Mean Square Error

ˆ1
2
kx − yk = |x (t) − y (t)|2 dt = 2
−1

236 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-Length Signals

finite-length and periodic signals live in CN


 T
vector notation: x = x0 x1 . . . xN−1
all operations well defined and intuitive
space of N-periodic signals sometimes indicated by C̃N

237 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inner product for signals

N−1
X
hx, yi = x ∗ [n] y [n]
n=0

well defined for all finite-length vectors (i.e. vectors in CN )

238 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What about Infinite-length signals?


X
hx, yi = x ∗ [n] y [n]
n=−∞

careful: sum may explode! (e.g. u[n] )

239 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What about Infinite-length signals?


X
hx, yi = x ∗ [n] y [n]
n=−∞

|x [n]|2 < ∞
P
We require sequences to be square-summable:

Space of square-summable sequences: `2 (Z)

240 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Infinite-Length Signals

“well-behaved” infinite-length signals live in `2 (Z)

 T
vector notation: x = . . . x−2 x−1 x0 x1 x2 . . .
many interesting signals not in `2 (Z) unfortunately (x[n] = 1, x[n] = cos(ωn), etc)

241 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Bases

linear combination is the basic operation in vector spaces:

g = αx + βy

can we find a set of vectors {w(k) } so that we can write any vector as a linear
combination of {w(k) }?

242 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Bases

linear combination is the basic operation in vector spaces:

g = αx + βy

can we find a set of vectors {w(k) } so that we can write any vector as a linear
combination of {w(k) }?

243 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
The canonical R basis

   
1 0
e(0) = e(1) =
0 1

     
x0 1 0
= x0 + x1
x1 0 1

244 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
The canonical R basis

   
1 0
e(0) = e(1) =
0 1

     
x0 1 0
= x0 + x1
x1 0 1

245 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
The canonical R basis
 
2
x= = 2e(0) + e(1)
1

246 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
Another R basis
   
1 1
v(0) = v(1) =
0 1
 
2
x= = v(0) + v(1)
1

247 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

2
But this is not a basis for R
   
1 −1
g(0) = g(1) =
0 0
 
2
x 6= = α1 g(0) + α2 g(1)
1

248 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What about infinite-dimensional spaces?


X
x= αk w(k)
k=0

249 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A basis for `2 (Z)

..
 
 . 

 0 


 0 

1
e(k)
 
=  k-th position, k ∈ Z

 0 


 0 


 0 

..
.

250 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What about function vector spaces?

X
f (t) = αk h(k) (t)
k

251 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A basis for the functions over an interval?

the Fourier basis for [−1, 1] :

1
√ , cos (πt) , sen (πt) , cos (2πt) , sen (2πt) , cos (3πt) , sen (3πt) , . . .
2

252 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the Fourier Basis

N
X sen (2k + 1) πt
2k + 1
k=0

253 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the Fourier Basis

N
X sen (2k + 1) πt
2k + 1
k=0

254 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the Fourier Basis

N
X sen (2k + 1) πt
2k + 1
k=0

255 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the Fourier Basis

N
X sen (2k + 1) πt
2k + 1
k=0

256 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Bases: formal definition

Given:
a vector space H
a set of K vectors from H: W = {wk }k=0,1,...,K −1

W is a basis for H if:


1 we can write for all x ∈ H:
K
X −1
x= αk w(k) , αk ∈ C
k=0

2 the coefficients αk are unique

257 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Bases: formal definition

Unique representation implies linear independence:

K
X −1
αk w(k) = 0 ⇔ αk = 0, k = 0, 1, . . . , K − 1
k=0

258 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Special bases

Orthogonal basis:

D E
w(k) , w(n) = 0 for k 6= n

Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]

We can alway orthonormalize a basis via a Gram-Schmidt algorithm

259 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Special bases

Orthogonal basis:

D E
w(k) , w(n) = 0 for k 6= n

Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]

We can alway orthonormalize a basis via a Gram-Schmidt algorithm

260 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Special bases

Orthogonal basis:

D E
w(k) , w(n) = 0 for k 6= n

Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]

We can alway orthonormalize a basis via a Gram-Schmidt algorithm

261 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basis expansion

K
X −1
x= αk w(k) , αk ∈ C
k=0

How do we find the α’s?

Orthonormal bases are the best:

D E
αk = w(k) , x

262 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basis expansion

K
X −1
x= αk w(k) , αk ∈ C
k=0

How do we find the α’s?

Orthonormal bases are the best:

D E
αk = w(k) , x

263 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis

K
X −1 K
X −1
x= αk w(k) = βk v(k)
k=0 k=0
(k)
if {v} is orthonormal:
D E
βh = v(h) , x
K −1
* +
X
βh = v(h) , αk w(k)
k=0
K
X −1 D E
βh = αk v(h) , w(k)
k=0

264 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis

K
X −1 D E
βh = αk v(h) , w(k)
k=0
K
X −1
βh = αk chk
k=0
  
c00 c01 ... c0(K −1) α0
 c10 c11 ··· c1(K −1)  α1 
βh = 
  
.. .. .. ..  .. 
 . . . .  . 
c(K −1)0 c(K −1)1 · · · c(K −1)(K −1) αK −1

265 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

canonical basis E = {e(0) , e(1) }


x = α0 e(0) + α1 e(1)

266 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

canonical basis E = {e(0) , e(1) }


x = α0 e(0) + α1 e(1)

new basis V = {v(0) , v(1) } with


T
v(0) =

cos (θ) sen (θ)
T
v(1) =

− sen (θ) cos (θ)

v = β0 v(0) + β1 v(1)

267 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

new basis is orthonormal


in compact form:
    
β0 cos (θ) sen (θ) α0
= = Rα
β1 − sen (θ) cos (θ) α1

R: rotation matrix
key fact: RT R = I

268 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

new basis is orthonormal


in compact form:
    
β0 cos (θ) sen (θ) α0
= = Rα
β1 − sen (θ) cos (θ) α1

R: rotation matrix
key fact: RT R = I

269 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

new basis is orthonormal


in compact form:
    
β0 cos (θ) sen (θ) α0
= = Rα
β1 − sen (θ) cos (θ) α1

R: rotation matrix
key fact: RT R = I

270 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis: example

new basis is orthonormal


in compact form:
    
β0 cos (θ) sen (θ) α0
= = Rα
β1 − sen (θ) cos (θ) α1

R: rotation matrix
key fact: RT R = I

271 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basic of Fourier Analysis

272 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Oscillations are everywhere


sustainable dynamic systems exhibit oscillatory behavior
intuitively: things that don’t move in circles can’t last:
bombs
rockets
human beings ...

273 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

You too can detect sinusoids!

the human body has two receptors for sinusoidal signals:

air pressure sinusoids electromagnetic sinusoids


frequencies from 20Hz to 20kHz frequencies from 430THz to 790THz
274 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The intuition

humans analyze complex signals (audio, images) in terms of their sinusoidal


components
we can build instruments that “resonate” at one or multiple frequencies (tuning
fork vs piano)
the “frequency domain” seems to be as important as the time domain

275 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Fundamental question

can we decompose any signal into sinusoidal elements?

yes, and Fourier showed us how to do it exactly !

analysis synthesis

from time domain to frequency domain from frequency domain to time domain
find the contribution of different create signals with known frequency
frequencies content
discover “hidden” signal properties fit signals to specific frequency regions

276 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Fundamental question

can we decompose any signal into sinusoidal elements?

yes, and Fourier showed us how to do it exactly !

analysis synthesis

from time domain to frequency domain from frequency domain to time domain
find the contribution of different create signals with known frequency
frequencies content
discover “hidden” signal properties fit signals to specific frequency regions

277 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Fundamental question

can we decompose any signal into sinusoidal elements?

yes, and Fourier showed us how to do it exactly !

analysis synthesis

from time domain to frequency domain from frequency domain to time domain
find the contribution of different create signals with known frequency
frequencies content
discover “hidden” signal properties fit signals to specific frequency regions

278 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Fundamental question

can we decompose any signal into sinusoidal elements?

yes, and Fourier showed us how to do it exactly !

analysis synthesis

from time domain to frequency domain from frequency domain to time domain
find the contribution of different create signals with known frequency
frequencies content
discover “hidden” signal properties fit signals to specific frequency regions

279 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The mathematical setup

let’s start with finite-length signals (i.e. vectors in CN )


Fourier analysis is a simple change of basis
a change of basis is a change of perspective
a change of perspective can reveal things (if the basis is good)

280 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal

281 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal in the Fourier basis

282 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
The Fourier Basis for C

Claim: the set of N signals in CN



wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1

is an orthogonal basis in CN .

283 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
The Fourier Basis for C

In vector notation:

{w(k) }k=0,1,...,N−1

with


wn (k) = e j N
nk

is an orthogonal basis in CN

284 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Recall the complex exponential generating machine...

285 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Recall the complex exponential generating machine...

286 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(0) 64
Basis vector w ∈C

287 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(1) 64
Basis vector w ∈C

288 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(2) 64
Basis vector w ∈C

289 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(3) 64
Basis vector w ∈C

290 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(4) 64
Basis vector w ∈C

291 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(5) 64
Basis vector w ∈C

292 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(16) 64
Basis vector w ∈C

293 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(20) 64
Basis vector w ∈C

294 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(30) 64
Basis vector w ∈C

295 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(31) 64
Basis vector w ∈C

296 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(32) 64
Basis vector w ∈C

297 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(33) 64
Basis vector w ∈C

298 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(34) 64
Basis vector w ∈C

299 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(60) 64
Basis vector w ∈C

300 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(62) 64
Basis vector w ∈C

301 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

(63) 64
Basis vector w ∈C

302 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Proof of orthogonality

D E N−1
X  2π ∗ 2π
w(k) , w(h) = e j N nk e j N nh
n=0
D E N−1
X 2π
w(k) , w(h) = ej N
(h−k)n

n=0

D E N n=k
w(k) , w(h) =
0 otherwise

303 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remarks

N orthogonal vectors → basis for CN



vectors are not orthonormal. Normalization factor would be 1/ N

304 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remarks

N orthogonal vectors → basis for CN



vectors are not orthonormal. Normalization factor would be 1/ N

305 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Discrete Fourier Transform


DFT

306 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
The Fourier Basis for C


in “signal” notation: wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1

in vector notation: {w(k) }k=0,1,...,N−1 with wn (k) = e j N
nk

307 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
The Fourier Basis for C


in “signal” notation: wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1

in vector notation: {w(k) }k=0,1,...,N−1 with wn (k) = e j N
nk

308 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
The Fourier Basis for C

N orthogonal vectors → basis for CN



vectors are not orthonormal. Normalization factor would be 1/ N
will keep normalization factor explicit in DFT formulas

309 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basis expansion

Analysis formula:
D E
Xk = w(k) , x

Synthesis formula:
N−1
1 X
x= Xk w(k)
N
k=0

310 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis in matrix form


Define WN = e −j N
(or simply W when N is evident from the context)

Change of basis matrix W with W[n, m] = WNnm :

1 1 1 1 ... 1
 
 1 W1 W2 W3 . . . W N−1 
W2 W4 W6 . . . W 2(N−1)
 
W=
 1 

 1 ... 
2
1 W N−1 W 2(N−1) W 3(N−1) . . . W (N−1)

311 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis in matrix form


Define WN = e −j N
(or simply W when N is evident from the context)

Change of basis matrix W with W[n, m] = WNnm :

1 1 1 1 ... 1
 
 1 W1 W2 W3 . . . W N−1 
W2 W4 W6 . . . W 2(N−1)
 
W=
 1 

 1 ... 
2
1 W N−1 W 2(N−1) W 3(N−1) . . . W (N−1)

312 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Change of basis in matrix form

Analysis formula:

X = Wx

Synthesis formula:
1 H
x= W X
N

313 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basis expansion (signal notation)

Analysis formula:
N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0

N-point signal in the frequency domain

Synthesis formula:

N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0

N-point signal in the “time” domain

314 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Basis expansion (signal notation)

Analysis formula:
N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0

N-point signal in the frequency domain

Synthesis formula:

N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0

N-point signal in the “time” domain

315 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT is obviously linear

DFT {αx[n] + βy [n]} = α DFT{x[n]} + β DFT{y [n]}

316 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
DFT of x[n] = δ[n], x[n] ∈ C

N−1
X 2π
X [k] = δ[n]e −j N
nk
=1
n=0

317 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
DFT of x[n] = 1, x[n] ∈ C

N−1
X 2π
X [k] = e −j N
nk
= Nδ[k]
n=0

318 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

 

x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2

319 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

 

x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2

320 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

 

x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2

321 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

 

x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2

322 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

X [k] = hwk [n], x[n]i


 
3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
X [k] = hwk [n], w4 [n]i + hwk [n], w60 [n]i
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise

323 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

X [k] = hwk [n], x[n]i


 
3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
X [k] = hwk [n], w4 [n]i + hwk [n], w60 [n]i
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise

324 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

X [k] = hwk [n], x[n]i


 
3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
X [k] = hwk [n], w4 [n]i + hwk [n], w60 [n]i
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise

325 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

X [k] = hwk [n], x[n]i


 
3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
X [k] = hwk [n], w4 [n]i + hwk [n], w60 [n]i
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise

326 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

327 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

 
2π π
x[n] = 3 cos n+
16 3
 
2π π
x[n] = 3 cos 4n +
64 3
3 h j 2π 4n j π 2π π
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3  jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2

328 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

 
2π π
x[n] = 3 cos n+
16 3
 
2π π
x[n] = 3 cos 4n +
64 3
3 h j 2π 4n j π 2π π
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3  jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2

329 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

 
2π π
x[n] = 3 cos n+
16 3
 
2π π
x[n] = 3 cos 4n +
64 3
3 h j 2π 4n j π 2π π
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3  jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2

330 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

 
2π π
x[n] = 3 cos n+
16 3
 
2π π
x[n] = 3 cos 4n +
64 3
3 h j 2π 4n j π 2π π
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3  jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2

331 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

X [k] = hwk [n], x[n]i


π
 96e j 3

for k = 4
π
X [k] = −j
96e 3 for k = 60
0 otherwise

332 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

333 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

334 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/10n), x[n] ∈ C

2π 2π 2π
6< < 7
64 10 64

335 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of x[n] = 3 cos(2π/10n), x[n] ∈ C

336 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
DFT of length-M step in C

M−1
X
x[n] = δ[n − h], n = 0, 1, . . . , N − 1
h=0

337 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
DFT of length-M step in C

N−1 M−1
−j 2π 2π
X X
X [k] = x[n]e N
nk
= e −j N
nk

n=0 n=0
sen Nπ Mk −j π (M−1)k

X [k] =  e N
sen Nπ k

338 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

N
DFT of length-M step in C

sen Nπ Mk −j π (M−1)k

X [k] =  e N
sen Nπ k

X [0] = M, from the definition of the sum


X [k] = 0 if Mk/N integer (0 ≤ k < N)
∠X [k] linear in k (except at sign changes for the real part)

339 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

64
DFT of length-4 step in C

340 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wrapping the phase

Often the phase is displayed “wrapped” over the [−π, π] interval.


most numerical packages return wrapped phase
phase can be unwrapped by adding multiples of 2π

341 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The DFT in Practice

342 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

343 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

344 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

345 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

346 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

x[n] = 1 (slowest signal)

only lowest frequency

347 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

x[n] = cos(πn) = (−1)n (fastest signal)

only highest frequency

348 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Energy distribution

Recall Parseval: kxk2 = |αk |2


P

N−1 N−1
X 1 X
|x[n]|2 = |X [k]|2
N
n=0 k=0

square magnitude of k-th DFT coefficient proportional to signal’s energy at frequency


ω = (2π/N)k

349 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

x[n] = 3 cos(2π/16n)

energy concentrated on single frequency


(counterclockwise and clockwise combine to give real signal)

350 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Interpreting a DFT plot

x[n] = u[n] − u[n − 4] (step)

energy mostly in low frequencies

351 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of real signals

For real signals the DFT is “symmetric” in magnitude

|X [k]| = |X [N − k]| for k = 1, 2, . . . , bN/2c

352 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of real signals

For real signals, magnitude plots need only bN/2c + 1 points

353 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal revisited

354 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal revisited

355 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal revisited

x[n] = cos (ωn + φ) + η[n]

with
φ=0

ω = 1024 64

356 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal revisited

x[n] = cos (ωn + φ) + η[n]

with
φ=0

ω = 1024 64

357 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Mystery signal revisited


Cosine and noise components

358 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

sunspot number: s = 10 × # of clusters + # of spots


data set from 1749 to 2003, 2904 months

359 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

sunspot number: s = 10 × # of clusters + # of spots


data set from 1749 to 2003, 2904 months

360 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

361 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

362 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

363 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Solar spots

DFT main peak for k = 22


22 cycles over 2904 months
2904
period: 22 ≈ 11 years

364 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature (2920 days)

365 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature: DFT

first few hundred DFT coefficients


(in magnitude and normalized by the length of the temperature vector)

366 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature: DFT

first few hundred DFT coefficients


(in magnitude and normalized by the length of the temperature vector)

367 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

average value (0-th DFT coefficient): 12.3◦ C


DFT main peak for k = 8, value 6.4◦ C
8 cycles over 2920 days
2920
Period: 8 = 365 days
temperature excursion: 12.3◦ C ± 12.8◦ C

368 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

if we know the “clock” of the system Ts


fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz

369 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

if we know the “clock” of the system Ts


fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz

370 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

if we know the “clock” of the system Ts


fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz

371 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

if we know the “clock” of the system Ts


fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz

372 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Daily temperature

if we know the “clock” of the system Ts


fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz

373 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: train whistle

374 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: train whistle

32768 samples (the “clock” of the system Fs = 8000 Hz).


So, Ts = 1/8000 seconds

375 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: train whistle

32768 samples (the “clock” of the system Fs = 8000 Hz).


So, Ts = 1/8000 seconds

376 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: train whistle

32768 samples (the “clock” of the system Fs = 8000 Hz).


So, Ts = 1/8000 seconds

377 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: train whistle

If we look up the frequencies:

378 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Synthesis: the sinusoidal generator


wk [n] = e j ( N kn+φk )

379 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Synthesis: the sinusoidal generator


wk [n] = e j ( N kn+φk )

380 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Synthesis: the sinusoidal generator

381 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT synthesis formula

382 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Initializing the machine

|X [k]|
Ak =
N

φk = ∠X [k]

383 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

x = [ 1 2 3 4 3 2 1 ]T

384 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

385 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT synthesis formula

386 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

387 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

388 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

389 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

390 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

391 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

392 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example

393 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Running the machine too long ...

x[n + N] = x[n]

output signal is N-periodic!

394 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inherent periodicities in the DFT

the analysis formula:


N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0

produces an N-point signal in the frequency domain

the synthesis formula:

N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0

produces an N-point signal in the “time” domain


395 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inherent periodicities in the DFT

the analysis formula:


N−1
X 2π
X [k] = x[n]e −j N
nk
, k∈Z
n=0

produces an N-periodic signal in the frequency domain

the synthesis formula:

N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0

produces an N-point signal in the “time” domain


396 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inherent periodicities in the DFT

the analysis formula:


N−1
X 2π
X [k] = x[n]e −j N
nk
, k∈Z
n=0

produces an N-periodic signal in the frequency domain

the synthesis formula:

N−1
1 X 2π
x[n] = X [k]e j N nk , n∈Z
N
k=0

produces an N-periodic signal in the “time” domain


397 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Short-Time Fourier Transform


STFT

398 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Dual-Tone Multi Frequency Dialing (DTMF)

399 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF Signaling

400 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

1-5-9 in time

401 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

1-5-9 in time (in detail)

402 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

1-5-9 in frequency (magnitude)

403 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The fundamental tradeoff

time representation obfuscates frequency


frequency representation obfuscates time

404 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform

Idea:
take small signal pieces of length L
look at the DFT of each piece
L−1
X 2π
X [m; k] = x[m + n]e −j L
nk

n=0

405 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform

Idea:
take small signal pieces of length L
look at the DFT of each piece
L−1
X 2π
X [m; k] = x[m + n]e −j L
nk

n=0

406 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform (L=256)

407 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform (L=256)

408 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform (L=256)

409 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Short-Time Fourier Transform (L=256)

410 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

color-code the magnitude: dark is small, white is large


use 10 log (|X [m; k]|) to see better (power in dBs)
plot spectral slices one after another

411 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

color-code the magnitude: dark is small, white is large


use 10 log (|X [m; k]|) to see better (power in dBs)
plot spectral slices one after another

412 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

color-code the magnitude: dark is small, white is large


use 10 log (|X [m; k]|) to see better (power in dBs)
plot spectral slices one after another

413 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF spectrogram

414 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Labeling the Spectrogram

If we know the “system clock” Fs = 1/Ts we can label the axis


highest positive frequency: Fs /2 Hz
frequency resolution: Fs /L Hz
width of time slices: LTs seconds

415 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Labeling the Spectrogram

If we know the “system clock” Fs = 1/Ts we can label the axis


highest positive frequency: Fs /2 Hz
frequency resolution: Fs /L Hz
width of time slices: LTs seconds

416 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Labeling the Spectrogram

If we know the “system clock” Fs = 1/Ts we can label the axis


highest positive frequency: Fs /2 Hz
frequency resolution: Fs /L Hz
width of time slices: LTs seconds

417 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF spectrogram (Fs = 8000)

418 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

Questions:
width of the analysis window?
position of the windows (overlapping?)
shape of the window ?(weighing the samples)

419 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

Questions:
width of the analysis window?
position of the windows (overlapping?)
shape of the window ?(weighing the samples)

420 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Spectrogram

Questions:
width of the analysis window?
position of the windows (overlapping?)
shape of the window ?(weighing the samples)

421 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wideband vs Narrowband

Long window: narrowband spectrogram


long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time

Short window: wideband spectrogram


short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution

422 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wideband vs Narrowband

Long window: narrowband spectrogram


long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time

Short window: wideband spectrogram


short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution

423 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wideband vs Narrowband

Long window: narrowband spectrogram


long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time

Short window: wideband spectrogram


short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution

424 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wideband vs Narrowband

Long window: narrowband spectrogram


long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time

Short window: wideband spectrogram


short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution

425 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Wideband vs Narrowband

Long window: narrowband spectrogram


long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time

Short window: wideband spectrogram


short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution

426 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF spectrogram (wideband)

N = 16800, L = 32

427 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF spectrogram

N = 16800, L = 256

428 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTMF spectrogram (narrowband)

N = 16800, L = 1024

429 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Speech analysis

430 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Speech analysis

8ms analysis window (125Hz frequency bins), 4ms shifts

431 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Speech analysis

32ms analysis window (31Hz frequency bins), 4ms shifts

432 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Time-Frequency tiling

L = 20

433 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Time-Frequency tiling

L = 10

434 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Time-Frequency tiling

L=4

435 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Food for thought

time “resolution” ∆t = L
frequency “resolution” ∆f = 2π/L
∆t∆f = 2π

uncertainty principle!

436 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Food for thought

time “resolution” ∆t = L
frequency “resolution” ∆f = 2π/L
∆t∆f = 2π

uncertainty principle!

437 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Food for thought

time “resolution” ∆t = L
frequency “resolution” ∆f = 2π/L
∆t∆f = 2π

uncertainty principle!

438 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Even more food for thought

more sophisticated tilings of the time-frequency planes


can be obtained with the “wavelet” transform

439 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Advanced Fourier Analysis

440 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete Fourier Series

441 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete Fourier Series (DFS)

DFS = DFT with periodicity explicit

the DFS maps an N-periodic signal onto an N-periodic sequence of Fourier


coefficients
the inverse DFS maps an N-periodic sequence of Fourier coefficients a set onto an
N-periodic signal.
the DFS of an N-periodic signal is mathematically equivalent to the DFT of one
period.

442 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

The DFS helps us understand how to define time shifts for finite-length signals.

For an N-periodic sequence x̃[n]:


x̃[n − M] is well-defined for all M ∈ N

DFS{x̃[n − M]} = e −j N
Mk
X̃ [k] (easy derivation)
IDFS{X̃[k]} = x̃[n-M]

443 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

The DFS helps us understand how to define time shifts for finite-length signals.

For an N-periodic sequence x̃[n]:


x̃[n − M] is well-defined for all M ∈ N

DFS{x̃[n − M]} = e −j N
Mk
X̃ [k] (easy derivation)
IDFS{X̃[k]} = x̃[n-M]

444 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

The DFS helps us understand how to define time shifts for finite-length signals.

For an N-periodic sequence x̃[n]:


x̃[n − M] is well-defined for all M ∈ N

DFS{x̃[n − M]} = e −j N
Mk
X̃ [k] (easy derivation)
IDFS{X̃[k]} = x̃[n-M]

445 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

The DFS helps us understand how to define time shifts for finite-length signals.

For an N-periodic sequence x̃[n]:


x̃[n − M] is well-defined for all M ∈ N

DFS{x̃[n − M]} = e −j N
Mk
X̃ [k] (easy derivation)
IDFS{X̃[k]} = x̃[n-M]

446 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

For an N-point signal x[n]:


x[n − M] is not well-defined
build x̃[n] = x[n mod N] ⇒ X̃ [k] = X [k]
2π 2π
IDFT{e −j N
Mk
X [k]} = IDFS{e −j N
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
shifts for finite-length signals are “naturally” circular

447 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

For an N-point signal x[n]:


x[n − M] is not well-defined
build x̃[n] = x[n mod N] ⇒ X̃ [k] = X [k]
2π 2π
IDFT{e −j N
Mk
X [k]} = IDFS{e −j N
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
shifts for finite-length signals are “naturally” circular

448 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

For an N-point signal x[n]:


x[n − M] is not well-defined
build x̃[n] = x[n mod N] ⇒ X̃ [k] = X [k]
2π 2π
IDFT{e −j N
Mk
X [k]} = IDFS{e −j N
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
shifts for finite-length signals are “naturally” circular

449 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Finite-length time shifts revisited

For an N-point signal x[n]:


x[n − M] is not well-defined
build x̃[n] = x[n mod N] ⇒ X̃ [k] = X [k]
2π 2π
IDFT{e −j N
Mk
X [k]} = IDFS{e −j N
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
shifts for finite-length signals are “naturally” circular

450 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Periodic sequences: a bridge to infinite-length signals

N-periodic sequence: N degrees of freedom


DFS: only N Fourier coefficients capture all the information

451 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited

y [n] = x[n] + αy [n − 1]

452 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited

choose a signal x̄[n] that is nonzero only for 0 ≤ n < M


α = 1 (for now)

y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . .

453 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited

choose a signal x̄[n] that is nonzero only for 0 ≤ n < M


α = 1 (for now)

y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . .

454 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: 32-tap sawtooth wave

455 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Example: DFT of 32-tap sawtooth wave

456 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What if we take the DFT of two periods?

457 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

What if we take the DFT of two periods?

458 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of two periods: intuition

459 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of two periods: intuition

460 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of two periods: intuition

461 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of two periods: intuition

462 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of two periods: intuition

463 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of L periods


LX̄ [k/L] if k = 0, L, 2L, 3L, . . .
XL [k] =
0 otherwise

464 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT and DFS

again, all the spectral information for a periodic signal is contained in the DFT
coefficients of a single period
to stress the periodicity of the underlying signal, we use the term DFS

465 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-Time Fourier Transform

466 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The situation so far

Fourier representation for signal classes:


N-point finite-length: DFT
N-point periodic: DFS
infinite length: ?

467 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The situation so far

Fourier representation for signal classes:


N-point finite-length: DFT
N-point periodic: DFS
infinite length: ?

468 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The situation so far

Fourier representation for signal classes:


N-point finite-length: DFT
N-point periodic: DFS
infinite length: ?

469 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited, part 2

consider now α < 1


generated signal is infinite-length but not periodic:

y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], αx̄[0], αx̄[1], . . . , αx̄[M − 1], α2 x̄[0], α2 x̄[1], . . .

what is a good spectral representation?

470 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited, part 2

consider now α < 1


generated signal is infinite-length but not periodic:

y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], αx̄[0], αx̄[1], . . . , αx̄[M − 1], α2 x̄[0], α2 x̄[1], . . .

what is a good spectral representation?

471 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Karplus-Strong revisited, part 2

consider now α < 1


generated signal is infinite-length but not periodic:

y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], αx̄[0], αx̄[1], . . . , αx̄[M − 1], α2 x̄[0], α2 x̄[1], . . .

what is a good spectral representation?

472 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of increasingly long signals

Start with the DFT. What happens when N → ∞


(2π/N)k becomes denser in [0, 2π] . . .
In the limit (2π/N)k → ω:
X
x [n] e −jωn ω∈R
n

473 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of increasingly long signals

Start with the DFT. What happens when N → ∞


(2π/N)k becomes denser in [0, 2π] . . .
In the limit (2π/N)k → ω:
X
x [n] e −jωn ω∈R
n

474 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DFT of increasingly long signals

Start with the DFT. What happens when N → ∞


(2π/N)k becomes denser in [0, 2π] . . .
In the limit (2π/N)k → ω:
X
x [n] e −jωn ω∈R
n

475 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-Time Fourier Transform (DFTF)

Formal definition:
x[n] ∈ `2 (Z)
define the function of ω ∈ R

X
F (ω) = x[n]e −jωn
−∞

inversion (when F (ω) exists ):


ˆπ
1
x[n] = F (ω) e jωn dω n∈Z

−π

476 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-Time Fourier Transform (DFTF)

Formal definition:
x[n] ∈ `2 (Z)
define the function of ω ∈ R

X
F (ω) = x[n]e −jωn
−∞

inversion (when F (ω) exists ):


ˆπ
1
x[n] = F (ω) e jωn dω n∈Z

−π

477 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Discrete-Time Fourier Transform (DFTF)

Formal definition:
x[n] ∈ `2 (Z)
define the function of ω ∈ R

X
F (ω) = x[n]e −jωn
−∞

inversion (when F (ω) exists ):


ˆπ
1
x[n] = F (ω) e jωn dω n∈Z

−π

478 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT periodicity and notation

F (ω) is 2π-periodic
to stress periodicity (and for other reasons) we will write

 X
X e jω = x[n]e −jωn
−∞


by convention,X e is represented over [−π, π]

479 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT periodicity and notation

F (ω) is 2π-periodic
to stress periodicity (and for other reasons) we will write

 X
X e jω = x[n]e −jωn
−∞


by convention,X e is represented over [−π, π]

480 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT periodicity and notation

F (ω) is 2π-periodic
to stress periodicity (and for other reasons) we will write

 X
X e jω = x[n]e −jωn
−∞


by convention,X e is represented over [−π, π]

481 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

n
x[n] = α u[n] |α| < 1

482 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

n
x[n] = α u[n] |α| < 1


 X
X e jω = x[n]e −jωn
−∞

X
X e jω = αn e −jωn


n=0

 1
X e =
1 − αe −jω

483 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

n
x[n] = α u[n] |α| < 1

X e jω =
 1
1 + α2 − 2α cos (ω)

484 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Plotting a DTFT

485 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Plotting a DTFT

486 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Plotting a DTFT

487 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Plotting a DTFT

488 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

n
x[n] = α u[n] |α| < 1

489 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remember the periodicity

490 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remember the periodicity

491 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remember the periodicity

492 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Remember the periodicity

493 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Existence and properties of the DFTF

494 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Existence easy for absolutely summable sequences




 X −jωn

X e = x[n]e



n=−∞

X
X e jω 6 x[n]e −jωn


n=−∞
X∞
X e jω =

|x[n]|
n=−∞
X e jω < ∞


495 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Inversion easy for absolutely summable sequences

ˆπ ˆπ ∞
!
1 1 X

e jωn dω = x [k] e −jωk e jωn dω

X e
2π 2π
−π −π k=−∞
ˆπ ∞ ˆπ
1 jω
 jωn
X e jω(n−k)
X e e dω = x [k] dω
2π 2π
−π k=−∞ −π
ˆπ
1
X e jω e jωn dω = x[n]


−π

496 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

A formal change of basis

formally DTFT is an inner product in C∞ :



X
x[n]e −jωn = e jωn , x[n]


−∞

“basis” is an infinite, uncountable basis: {e jωn }ω∈R


something “breaks down”: we start with sequences but the transform is a function
we used absolutely summable sequences but DTFT exists for all square-summable
sequences (proof is rather technical)

497 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Review: DFT

498 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Review: DFS

499 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Review: DTFT

500 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT properties

501 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT properties

502 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Some particular cases:

503 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The DTFT as a change of basis

504 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT as basis expansion

Some things are OK:


DFT{δ[n]} = 1
DTFT{δ[n]} = e jωn , δ[n] = 1

505 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT as basis expansion

Some things are OK:


DFT{δ[n]} = 1
DTFT{δ[n]} = e jωn , δ[n] = 1

506 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT as basis expansion

Some things aren’t


DFT{1} = Nδ[k]

e −jωn =?
P
DTFT{1} =
−∞
problem: too many interesting sequences are not square summable

507 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT as basis expansion

Some things aren’t


DFT{1} = Nδ[k]

e −jωn =?
P
DTFT{1} =
−∞
problem: too many interesting sequences are not square summable

508 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

DTFT as basis expansion

Some things aren’t


DFT{1} = Nδ[k]

e −jωn =?
P
DTFT{1} =
−∞
problem: too many interesting sequences are not square summable

509 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Dirac delta functional

Defined by the “sifting” property:


ˆ∞
δ (t − s) f (t) dt = f (s)
−∞

for all functions of s, t ∈ R

510 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The “pulse train”


X
δ̃ (ω) = 2π δ (ω − 2πk)
k=−∞

just a technicality to use the Dirac delta in the space of 2π-periodic functions

511 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Graphical representation

512 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Now let the show begin!

ˆπ
1
IDTFT{δ̃ (ω)} = δ̃ (ω) e jωn dω

−π
ˆπ
IDTFT{δ̃ (ω)} = δ (ω) e jωn dω
−π
jωn

IDTFT{δ̃ (ω)} = e ω=0
IDTFT{δ̃ (ω)} = 1

This is similar to the fact that the IDFT{Nδ[n]} = 1

513 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Now let the show begin!

ˆπ
1
IDTFT{δ̃ (ω)} = δ̃ (ω) e jωn dω

−π
ˆπ
IDTFT{δ̃ (ω)} = δ (ω) e jωn dω
−π
jωn

IDTFT{δ̃ (ω)} = e ω=0
IDTFT{δ̃ (ω)} = 1

This is similar to the fact that the IDFT{Nδ[n]} = 1

514 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

In other words

DTFT{1} = δ̃ (ω)

515 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the same technique

IDTFT{δ̃ (ω − ω0 )} = e jωo n
So:
DTFT{1} = δ̃ (ω)
DTFT{e jωo n } = δ̃ (ω − ω0 )
h i
DTFT{cos (ω0 n)} = δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
h i
DTFT{ sen (ω0 n)} = −j δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2

516 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the same technique

IDTFT{δ̃ (ω − ω0 )} = e jωo n
So:
DTFT{1} = δ̃ (ω)
DTFT{e jωo n } = δ̃ (ω − ω0 )
h i
DTFT{cos (ω0 n)} = δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
h i
DTFT{ sen (ω0 n)} = −j δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2

517 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the same technique

IDTFT{δ̃ (ω − ω0 )} = e jωo n
So:
DTFT{1} = δ̃ (ω)
DTFT{e jωo n } = δ̃ (ω − ω0 )
h i
DTFT{cos (ω0 n)} = δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
h i
DTFT{ sen (ω0 n)} = −j δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2

518 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Using the same technique

IDTFT{δ̃ (ω − ω0 )} = e jωo n
So:
DTFT{1} = δ̃ (ω)
DTFT{e jωo n } = δ̃ (ω − ω0 )
h i
DTFT{cos (ω0 n)} = δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
h i
DTFT{ sen (ω0 n)} = −j δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2

519 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Introduction to Filtering

520 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear Filters

521 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


A generic signal processing device

y [n] = H {x [n]}

522 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Linearity

H {αx1 [n] + βx2 [n]} = αH {x1 [n]} + βH {x2 [n]}

523 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Linearity

524 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


(Non) Linearity

525 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Time Invariance

y [n] = H {x [n]} ⇔ H {x [n − n0 ]} = y [n − n0 ]

526 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Time Invariance

527 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Time Variance

528 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


Linear, Time-Invariant Systems

529 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear time-invariant filters


LTI and Causality

y [n] = H (x[n], x[n − 1], x[n − 2], . . . , y [n − 1], y [n − 2], . . .)

with H() a linear function of its arguments

530 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Impulse response

y [n] = H {x [n]}
h[n] = H {δ [n]}

Fundamental result: impulse response fully characterizes the LTI system!

531 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Example

532 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Example

x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]


we know the impulse response h[n] = H {δ [n]}
compute y [n] = H {x [n]} exploiting linearity and time-invariance.

533 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Example

x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]


we know the impulse response h[n] = H {δ [n]}
compute y [n] = H {x [n]} exploiting linearity and time-invariance.

534 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Example

x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]


we know the impulse response h[n] = H {δ [n]}
compute y [n] = H {x [n]} exploiting linearity and time-invariance.

535 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Convolution
Example

536 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Linear Filters

537 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Typical filtering scenario: denoising

538 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

idea: replace each sample by the local average


for instance: y [n] = (x[n] + x[n − 1])/2
more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0

539 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

idea: replace each sample by the local average


for instance: y [n] = (x[n] + x[n − 1])/2
more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0

540 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

idea: replace each sample by the local average


for instance: y [n] = (x[n] + x[n − 1])/2
more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0

541 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

542 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

543 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

544 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Denoising by Moving Average

545 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


MA: impulse response

M−1
1 X
h[n] = δ[n − k]
M
k=0
= 1/M for 0 ≤ n < M

546 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


MA: impulse response

547 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


MA: analysis

smoothing effect proportional to M


number of operations and storage also proportional to M

548 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The moving average filter


Generalization: The Moving Average (MA) process

For a MA filter
1 1 1
y [n] = x[n] + x[n − 1] + · · · + x[n − (M − 1)]
M M M
and
1 1 −1 1 −(M−1)
M + Mz + ... M z
H(z) =
1
In general, for a MA process we have

b0 + b1 z −1 + . . . bQ z −Q
H(z) =
1

549 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator

try the filter


y [n] = λy [n − 1] + (1 − λ)x[n]

filter is now recursive, since it uses its previous output value


it can be shown that for λ ≈ 1, the Leaky Integrator is similar to the moving
average with a large M.

550 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator

try the filter


y [n] = λy [n − 1] + (1 − λ)x[n]

filter is now recursive, since it uses its previous output value


it can be shown that for λ ≈ 1, the Leaky Integrator is similar to the moving
average with a large M.

551 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Denoising recursively

552 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Denoising recursively

553 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Denoising recursively

554 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


What about the impulse response?

555 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Impulse response?

h[n] = (1 − λ)λn u[n]

556 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Why the name?

Discrete-time integrator is a boundless accumulator:


n
X
y [n] = x[k]
k=−∞

We can rewrite the integrator as

y [n] = y [n − 1] + x[n]

557 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Why the name?

558 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The Leaky Integrator


Generalization: The AutoRegressive (AR) process

In general, for a AR process we have

b0
H(z) =
1 + a1 z −1 + . . . aP z −P

559 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter Stability

560 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


Filter types according to impulse response

Finite Impulse Response (FIR)


Infinite Impulse Response (IIR)
causal
noncausal

561 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


Filter types according to impulse response

Finite Impulse Response (FIR)


Infinite Impulse Response (IIR)
causal
noncausal

562 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


Filter types according to impulse response

Finite Impulse Response (FIR)


Infinite Impulse Response (IIR)
causal
noncausal

563 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


FIR

impulse response has finite support


only a finite number of samples are involved in the computation of each output
sample

564 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


FIR (example)

565 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


IIR

impulse response has infinite support


a potentially infinite number of samples are involved in the computation of each
output sample
surprisingly, in many cases the computation can still be performed in a finite
amount of steps

566 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


IIR (example)

h[n] = (1 − λ)λn u[n]

Leaky Integrator

567 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


Causal vs Noncausal

causal
impulse response is zero for n < 0
only past samples (with respect to the present) are involved in the computation of
each output sample
causal filters can work ”on line” since they only need the past
noncausal
impulse response is nonzero for some (or all) n<0
can still be implemented in a offline fashion (when all input data is available on
storage, e.g., in Image Processing)

568 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


Causal example

Moving Average filter

569 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter classification in the time domain


NonCausal example

Zero-centered Moving Average filter

570 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
Stability

key concept: avoid “explosions” if the input is nice


a nice signal is a bounded signal: |x[n]| < M for all n
Bounded-Input Bounded-Output (BIBO) stability: if the input is nice the output
should be nice

571 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
Stability

key concept: avoid “explosions” if the input is nice


a nice signal is a bounded signal: |x[n]| < M for all n
Bounded-Input Bounded-Output (BIBO) stability: if the input is nice the output
should be nice

572 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
Stability

key concept: avoid “explosions” if the input is nice


a nice signal is a bounded signal: |x[n]| < M for all n
Bounded-Input Bounded-Output (BIBO) stability: if the input is nice the output
should be nice

573 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
Fundamental Stability Theorem

A filter is BIBO stable if and only if its impulse response is absolutely summable

X
|h[n]| = L < ∞ ⇔ |y [n]| < ML and |x[n]| < M
n

574 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
The good news

FIR filters are always stable

575 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Filter stability
Checking the stability of IIRs

Let’s check the Leaky Integrator:

576 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Frequency Response

577 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


A remarkable result

578 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


A remarkable result

579 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


A remarkable result

complex exponentials are eigensequences of LTI systems, i.e., linear filters cannot
change the frequency of sinusoids
DTFT of impulse response determines the frequency characteristics of a filter

580 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


A remarkable result

complex exponentials are eigensequences of LTI systems, i.e., linear filters cannot
change the frequency of sinusoids
DTFT of impulse response determines the frequency characteristics of a filter

581 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


Magnitude and phase

582 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem

In general:

DTFT{x[n] ∗ h[n]} =?

Intuition: the DTFT reconstruction formula tell us how to build x[n] from a set of
complex exponential “basis” functions. By linearity . . .

583 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem

In general:

DTFT{x[n] ∗ h[n]} =?

Intuition: the DTFT reconstruction formula tell us how to build x[n] from a set of
complex exponential “basis” functions. By linearity . . .

584 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem

585 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


Frequency response

H(e jω ) = DTFT{h[n]}
Two effects:
magnitude: amplification (|H(e jω )| > 1) or attenuation (|H(e jω )| < 1) of input
frequencies
phase: overall delay and shape changes

586 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

The convolution theorem


Frequency response

H(e jω ) = DTFT{h[n]}
Two effects:
magnitude: amplification (|H(e jω )| > 1) or attenuation (|H(e jω )| < 1) of input
frequencies
phase: overall delay and shape changes

587 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Moving Average revisited

588 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Moving Average, magnitude response

589 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Moving Average, magnitude response

590 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Moving Average, magnitude response

591 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

592 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

593 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

594 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

595 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

596 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

597 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Denoising revisited

598 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


By the way, remember the time-domain analysis

599 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


By the way, remember the time-domain analysis

600 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


What about the phase?

Assume |H(e jω )| = 1
zero phase: ∠H(e jω ) = 0
linear phase: ∠H(e jω ) = dω, where d ∈ R
nonlinear phase

601 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


What about the phase?

Assume |H(e jω )| = 1
zero phase: ∠H(e jω ) = 0
linear phase: ∠H(e jω ) = dω, where d ∈ R
nonlinear phase

602 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


What about the phase?

Assume |H(e jω )| = 1
zero phase: ∠H(e jω ) = 0
linear phase: ∠H(e jω ) = dω, where d ∈ R
nonlinear phase

603 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Phase and signal shape: zero phase

604 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Phase and signal shape: linear phase

605 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Phase and signal shape: nonlinear phase

In all three cases (zero, linear and nonlinear), the spectrum magnitude
remains the same. 606 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Linear phase

y [n] = x[n − d]
Y (e jω ) = e −jωd X (e jω )
H(e jω ) = e −jωd
linear phase term

607 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Linear phase

y [n] = x[n − d]
Y (e jω ) = e −jωd X (e jω )
H(e jω ) = e −jωd
linear phase term

608 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Linear phase

y [n] = x[n − d]
Y (e jω ) = e −jωd X (e jω )
H(e jω ) = e −jωd
linear phase term

609 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Linear phase

y [n] = x[n − d]
Y (e jω ) = e −jωd X (e jω )
H(e jω ) = e −jωd
linear phase term

610 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Linear phase

611 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Moving Average is Linear phase

1 sen ω2 M −j M−1 ω



H e =  e 2
M sen ω2

612 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator revisited

h[n] = (1 − λ)λn u[n]

613 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator revisited

1−λ
H e jω =

1 − λe jω
Finding magnitude and phase require a little algebra ...

614 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator revisited

615 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator, magnitude response

616 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator, magnitude response

617 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator, magnitude response

618 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator, phase response

619 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Leaky integrator, phase response

620 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Phase is sufficiently linear where it matters

621 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Examples of frequency response


Phase is sufficiently linear where it matters

622 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters

623 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to magnitude response

Lowpass
Highpass
Bandpass
Allpass

Moving Average and Leaky Integrator are lowpass filters

624 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to magnitude response

Lowpass
Highpass
Bandpass
Allpass

Moving Average and Leaky Integrator are lowpass filters

625 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to magnitude response

Lowpass
Highpass
Bandpass
Allpass

Moving Average and Leaky Integrator are lowpass filters

626 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to magnitude response

Lowpass
Highpass
Bandpass
Allpass

Moving Average and Leaky Integrator are lowpass filters

627 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to magnitude response

Lowpass
Highpass
Bandpass
Allpass

Moving Average and Leaky Integrator are lowpass filters

628 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to phase response

Linear phase
Nonlinear phase

629 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Filter types according to phase response

Linear phase
Nonlinear phase

630 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
What is the best lowpass we can think of?

631 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
What is the best lowpass we can think of?

632 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Ideal lowpass filter



 1 for |ω 6 ωc |
H e = (2πperiodicity implicit)
0 otherwise

perfectly flat passband


infinite attenuation in stopband
zero-phase (no delay)

633 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Ideal lowpass filter:impulse response

634 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Ideal lowpass filter:impulse response

635 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
The bad news

impulse response is infinite support, two-sided


cannot compute the output in a finite amount of time
that’s why it’s called “ideal”
impulse response decays slowly in time
we need a lot of samples for a good approximation

636 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
The bad news

impulse response is infinite support, two-sided


cannot compute the output in a finite amount of time
that’s why it’s called “ideal”
impulse response decays slowly in time
we need a lot of samples for a good approximation

637 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Nevertheless ...

The sinc-rect pair:

638 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Little-known fact

the sinc is not absolutely summable


the ideal lowpass is not BIBO stable

639 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Nevertheless ...

The sinc-rect pair:

640 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Idel higpass filter

641 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Ideal Filters
Idel bandpass filter

642 / 643
Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

References I

[1] Paolo Prandoni and Martin Vetterli.


Digital Signal Processing.
https://www.coursera.org/learn/dsp/, 2018.

[2] Martin Vetterli and Paolo Prandoni.


Signal Processing for Communications.
EPFL press, 2008.

643 / 643