74 CHAPTER 1.

ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

 1.4 Fast Fourier Transform (FFT) Algorithm

Fast Fourier Transform, or FFT, is any algorithm for computing the N -point DFT with
a computational complexity of O(N log N ). It is not a new transform, but simply an
efficient method of calculating the DFT of x(n).
If we assume that N is even, we can write the N -point DFT of x(n) as
X
X (N ) (k) = x(n)ej N n
2k

n is even: n=2m,m=0,1, , N
2
1
X
x(n)ej n
2k
+ N

n is odd: n=2l+1,l=0,1, , N
2
1
N N
1 1
X
2 X
2
j 2k 2m
x(2l + 1)ej (2l+1)
2k
= x(2m)e N + N (1.31)
m=0 l=0

We make the following substitutions:

N
x0 (m) = x(2m), where m = 0, , 1,
2
N
x1 (l) = x(2l + 1), where l = 0, , 1.
2

Rewriting Eq. (1.31), we get

N N
1 1
X
2
j 2k
N m
X
2
j 2k
N l
(N ) j 2k
X (k) = x0 (m)e 2 +e N x1 (l)e 2

m=0 l=0
(N ) (N )
= X0 2 (k) + ej
2k
N X1 2 (k), (1.32)
(N ) (N )
where X0 2 (k) is the N2 -point DFT of the even-numbered samples of x(n) and X1 2 (k)
is the N2 -point DFT of the odd-numbered samples of x(n). Note that both of them are
N
2 -periodic discrete-time functions.
We have the following algorithm to compute X (N ) (k) for k = 0, , (N 1) :
(N )
1. Compute X0 2 (k) for k = 0, , N2 1.
(N )
2. Compute X1 2 (k) for k = 0, , N2 1.

3. Perform the computation (1.32) with N complex multiplications and N complex

additions.
Actually, it is possible to use fewer than N complex multiplications. Let

WN = ej N .
2
Sec. 1.4. Fast Fourier Transform (FFT) Algorithm 75
(N )
X0 2 (0) (N ) N
0 X ( 2 ) (0)
x(0) + X (N ) (0) = X0 2 (0) + WN 1

x(2) N (N )
X0 2 (1) A
+
2 -pt. A
+ X (N ) (1)
.. .. A A

+ .
..
. . AA


DFT (N ) AA


x(N 2) X0 2 ( N
2
1) A
A
+ X (N ) N 1
2

A
A
AA
A
+
A

A A



(N )
X1 2 (0)
A
A
A + X (N ) N = X ( N2 ) (0) W 0 X ( N2 ) (0)
x(1)

A
A A 2 0 N 1
(N WN 0 A
A
x(3) N 2
)
X1 (1)

A + X (N ) N
+1
2 -pt. 1


A A 2
.. .. WN ..
. .
A .
DFT
A
(N
1)
A +
)
x(N 1) X1 2 ( N
2

A X (N ) (N 1)

N 1

W 2
N

Figure 1.36. The FFT algorithm.

Then
k+ N
= ej ( +
2k
WN 2 N )

= ej
2k
N

= WNk

Therefore,
(N ) N
X (N ) (k) = X0 2 (k) + WNk X1 (k) for k = 0, , 1,
  2
(N ) N (N ) N
X k+ = X0 2 (k) WNk X1 (k) for k = 0, , 1,
2 2
as illustrated in Fig. 1.36. This shows that we do not need to actually perform N
complex multiplications, but only N2 . 8
Fig. 1.37 illustrates the recursive implementation of the FFT supposing that N =
2M . There is a total of M = log2 N stages of computation, each requiring 32 N complex
operations. Hence, the total computational complexity is O(N log N ). We see that the
process ends at a 1-point DFT. A 1-point DFT is the sample of the original signal:
0
X
x(n)ej ( )n = x(0).
20
X(0) = 1

n=0

The following remarks apply to the FFT:

8
Actually, slightly fewer if we do not count multiplications by 1 and j.
76 CHAPTER 1. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

1-pt. DFT 2M 2 -pt. DFT 2M 1 -pt. DFT 2M -pt. DFT

N
X004 (k)
```
N
2

X0 (k)

N
X014 (k) @
t @
3 N
t t t t 2 2 operations @ X (N ) (k)
t

N
X104 (k)
```
N
2

X1 (k)

N
X114 (k) 3
2N operations
3 N
2 2 operations

3 3 3
2N 2N 2N

Total number of complex operations

x per DFT operation.

Total number of complex operations

x
per level of recursion.

Figure 1.37. The recursive implementation of the FFT supposing that N = 2M . There is a to-
tal of M = log2 N stages of computation, each requiring 32 N complex operations. Hence, the total
computational complexity is O(N log N ).

1. For large N , the FFT is much faster than the direct application of the definition
of DFT, which is of complexity O(N 2 ).

2. The particular implementation of the FFT described above is called decimation-

in-time radix-2 FFT.

3. The number of operations required by an FFT algorithm can be approximated

as CN log N , where C is a constant. There are many variations of FFT aimed at
reducing this constante.g., if N = 3M , it may be better to use a radix-3 FFT.

4. Note that
  ( N 1 )
1 1 X
x (n)ej ( N )n
2k
DFT[x (n)] =
N N
n=0
N
X 1
1
x(n)ej ( )n
2k
= N
N
n=0

which is the IDFT of x(n). Thus, the FFT can also be used to compute the IDFT.
Sec. 1.4. Fast Fourier Transform (FFT) Algorithm 77
0 1
x(0)
x(0) B x(2) C
2-point DFT of 4-point DFT of B C
@ x(4) A
x(4)
x(6)

+ + +
x(0) Z +
X (8) (0)
Z S + L +
Z + S + L 
+
x(4)  Z S  L  X (8) (1)
S S + L L  +
S S L L  
+ S S + L L  +
x(2) Z +
 S S L L   X (8) (2)
Z  S L L L   +
Z +  S + L L L  + 
x(6)  Z  S L L L   X (8) (3)
W41 L L L L  +
L L L L  
2-point DFT of
x(2) L L L L 
x(6) L L  L L 

2-point DFT of
x(1) LL L L L
x(5)  LL L
 L L L L
  L L L L +
  LLL L L
+ +
x(1) X (8) (4)
Z + S +   L L
Z S +   L L L+
Z +
x(5)  Z S     L L L X (8) (5)
S S + W81   L L
S S   L L
+ S S +   L L
x(3)  S S   L L+ X (8) (6)
Z +  S W82  L
Z  S+  L+
Z +
x(7)  Z  S  L X (8) (7)
W41 W83
0 1
x(0)
x(3) B x(2) C
2-point DFT of
x(7) 4-point DFT of B C
@ x(4) A
x(6)

Figure 1.38. The 8-point FFT.

Example 1.26. The 8-point FFT is depicted in Fig. 1.38. The values of the twiddle
factors are:

W2 = ej
2
2 = 1,
j 2
W4 = e 4 = j,
j 2
W8 = e 8 .
78 CHAPTER 1. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

X = A(N ) x

N 1 N N N 1

0
1 WN 10000 0
1. 0 WN1
.. 0 N
00100 0
.. . A( ) 2 0 0. 0 0 0 1 ..0
N 1
.. .
0 1 0 WN2 00000 10
= 0
x
1 WN 01000 0
1. 0 WN1
.. .. 0 N
00010
..
0
..
. 0 A( 2 ) . .
N 1
0 1 0 WN2 00000 1

1 0
WN x(0)
1
1. 0 WN 0
.. N x(2)
.. . A( ) 2 0 ..
N 1
.
0 1 0 WN2
= 0
x(N 2)
1 WN
WN1 x(1)
1. 0 .. 0
.. N x(3)
. 0 A( 2 ) .
.
N 1 .
0 1 0 WN2
x(N 1)

0
1 WN (N )
1. 0 WN1
0 X0 2 (0)
.. .
.. .
. .
N 1 (N )
0 1 0 WN2 X0 2 ( N 1)
= 0 (2)N
2

1 WN
1 X1 (0)
1. 0 WN
.. .. 0 .
..
.
N 1 (N )
0 1 0 WN2 X1 2 ( N
2
1)

Figure 1.39. The FFT reduces the number of operations required to calculate the DFT by reducing
N
A(N ) to two A( 2 ) that is only half the size of A(N ) . This operation is repeated with every recursion
until we reach the 1-point DFT.
Sec. 1.4. Fast Fourier Transform (FFT) Algorithm 79

Recall that the DFT is a matrix multiplication (Fig. 1.35). One stage of the FFT
essentially reduces the multiplication by an N N matrix to two multiplications by
N N
2 2 matrices. This reduces the number of operations required to calculate the DFT
by almost a factor of two (Fig. 1.39).
Another interpretation of FFT involves analyzing the matrix
!
j 2k
1 1 e 2L
Ak,L = j 2k
,
2 1 e 2L

where k and L are nonnegative integers such that k < 2L . Note that

hAk,L x, Ak,L yi = (Ak,L y)H (Ak,L x)

= yH AH
k,L Ak,L x
! !
j 2k
1 1 1 1 1 e N
= yH x
ej N ej N
2k 2k
1 ej N
2k
2 2
 
H1 2 0
= y x = yH x = hx, yi,
2 0 2
i.e., multiplication by Ak,L preserves distances and angles roughly speaking, it is a
rotation or reflection. Continuing the matrix decomposition of Fig. 1.39 further until
we get the full FFT, it can be shown that FFT consists of N2 log N multiplications by

2 2 matrices of the form 2Ak,L , each operating on a pair of coordinates.9 Therefore,
FFT breaks down the multiplication by the DFT matrix A into elementary planar
transformations.

 1.4.1 Fast Computation of Convolution

Consider a linear system described by

y = Sx, (1.33)

where x is the N 1 input vector, representing an N -periodic input signal; S is an N N

matrix; and y is the N 1 output vector, representing an N -periodic output signal.
What conditions must the matrix S satisfy in order for the system to be time-invariant,
i.e., invariant to circular shifts of the input vector?
Note that a circular shift by one sample is

x(0) x(1) = x(N 1)
x(1) x(0)

x(2) x(1)
.
.. ..
. .
x(N 1) x(N 2)
9
The same conclusion can be reached by examining an FFT diagram such as Fig. 1.38.
80 CHAPTER 1. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

Let the first column of S be

h(0)
h(1)

h(2)
h= .
..
.
h(N 1)
Note that when
1
0

0
x= , then y = h,
..
.
0
and when
0
1

0
x= ,
..
.
0
then y is the second column of S, which therefore, in order for S to be invariant to
circular shifts, must be equal to:

h(N 1)
h(0)

h(1)
.
..
.
h(N 2)
Similarly, when

0
0

1

x= 0 , then y is the third column of S, etc.

..
.
0,
Thus, the matrix S must have the following structure:

h(0) h(N 1) h(N 2) h(1)
h(1) h(0) h(N 1) h(2)

h(2) h(1) h(0) h(3)
S= .
.. .. .. .. ..
. . . . .
h(N 1) h(N 2) h(N 3) h(0)
Sec. 1.4. Fast Fourier Transform (FFT) Algorithm 81

This is called a circulant matrix. We can then write Eq. (1.33) as

N
X 1
y(n) = x(m)h(n m)
m=0
N
X 1
= x(m)h((n m) mod N ) (1.34)
m=0
= x ~ h(n) = x
N h (1.35)

Eq. (1.35) is called a circular convolution or a periodic convolution. Note that formula
(1.34) works even when x or h are non-periodic. Observe the following:

For y(0), the sum of the indices of x and h is always 0 mod N for every term.

y(0) = x(0)h(0) + x(1)h(N 1) + x(2)h(N 2) + + x(N 1)h(1)

For y(1), the sum of the indices of x and h is always 1 mod N for every term.

y(1) = x(0)h(1) + x(1)h(0) + x(2)h(N 1) + + x(N 1)h(2)

This is true for all y(k), k = 0, 1, , N 1.

What are the eigenvectors of S? Let us try
1

N
1 j 2k
e 1
N
N
1 j N 2
2k
gk =
Ne
, where k = 0, 1, , N 1.

..
.
1 j 2k (N 1)
Ne
N

We have:

y(n) = h(n) ~ gk
N
X 1
= h(m)gk (n m)
m=0
N
X 1
1 j 2k (nm)
= e Nh(m)
N
m=0
(N 1 )
X 1 j 2k n
j 2k m
= h(m)e N e N
N
m=0
1 j 2k n
= H(k) e N
| {z } N
DFT of h
82 CHAPTER 1. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

Hence we have that

Sgk = H(k)gk
where gk is the k-th eigenvector and H(k) gives the corresponding eigenvalue. There-
fore,

H(0)
0


H(1)

S g0 g1 gN 1 = g0 g1 gN 1 .. .
| {z } .
The IDFT matrix B
0
H(N 1)

Then S can be written as:

H(0)
0
H(1)
S =B .. A,
.
0
H(N 1)

where the DFT matrix A is:

g0H
g1H

A = N BH = .. .
.
H
gN 1

Complex exponentials are the eigenvectors of circulant matrices. They diagonalize

circulant matrices. Thus, for any x CN ,

H(0)
0
H(1)
Sx = B .. Ax.
.
0
H(N 1)

Let us compare two algorithms for computing the circular convolution of x and h.

Algorithm 1 Directly

perform the multiplication Sx. This has computational com-
2
plexity O N .

Algorithm 2 1. Represent x in the eigenbasis of S, i.e., the Fourier basis,

X = Ax.

This step can be done with FFT whose complexity is O(N log N ).
Sec. 1.4. Fast Fourier Transform (FFT) Algorithm 83

Step 1 Step 2 Step 3

N-point DFT
x(n) X(k)
N-point IDFT
Y (k) = X(k)H(k) Y (k) y(n) = x ~ h(n)
N-point DFT
h(n) H(k)

Figure 1.40. An illustration of the FFT implementation of the circular convolution.

2. Compute the representation of y in the eigenbasis of S:

H(0)
0
H(1)
Y= .. X.
.
0
H(N 1)
This computation has complexity O(N ).
3. Reconstruct y from its Fourier coefficients:
y = BY.
This has complexity O(N log N ), if done using the FFT.
This algorithm is summarized in Fig. 1.40. Its total complexity is O(N log N ).
(Note that the second algorithm does not necessarily perform better for any matrix.)
Example 1.27. This example explores the relationship between the convolution and the
circular convolution. Let x and h be N -periodic signals, and let

x(n), 0 n N 1
xz =
0, otherwise

h(n), 0 n N 1
hz =
0, otherwise
If we let
yz (n) = xz hz (n)
y(n) = x ~ h(n)
then y(n) can be expressed as

yz (n) + yz (N + n), n = 0, 1, , N 2
y(n) =
y(N 1), n=N 1
84 CHAPTER 1. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS

Note that the overlap of yz (n) and yz (N + n) causes temporal aliasing in the resulting
y(n). This is the main difference between convolution and circular convolution.
x(n) h(n)
3
2 2
s 1 s
s s s 1 s
s s s 0 s
s s
1 n 1 n
0 1 2 3 4 0 1 2 3 4

(a) Convolution
yz (n) 8
s
yz (n) = xz hz (n)
5
s 3
2
s
s
s s s
1 n
0 1 2 3 4 5

(b) Circular convolution

y(n) 8 8
s s
y(n) = x ~ h(n)
5 5 5 5
s s s s
y(0) = yz (0) + yz (3) s s s s s s
y(1) = yz (1) + yz (4)

1 n
y(2) = yz (2) + yz (5) 0 1 2 3 4 5

Figure 1.41. A comparison between circular convolution and convolution.

Fig. 1.41 illustrates the effect of temporal aliasing. To remove or minimize the effect
of temporal aliasing, we could zero-pad x and h so that the temporal replicas are spread
further apart, and thus, overlapping would not occur.