You are on page 1of 18

Fast Fourier

Transform


Irina Bobkova
Overview
I. Polynomials
II. The DFT and FFT
III. Efficient implementations
IV. Some problems
A polynomial in the variable x over an algebraic field F is
representation of a function A(x) as a formal sum



•Coefficient representation


•Point-value representation


1
0
( )
n
j
j
j
A x a x
÷
=
=
¿
0 1 1
( , ,... )
n
a a a a
÷
=
0 0 1 1 1 1
{( , ), ( , ),..., ( , )}
n n
x y x y x y
÷ ÷
Coefficient
representation
Point-value representation
Adding
Multiplication
Representation of polynomials
( ) n O
2
( ) n O
( ) n O
( ) n O
Interpolation
Interpolation-the inverse of evaluation –determining the
coefficient form from a point-value representation
Lagrange’s formula


The coefficients can be computed in time
Exercise. Prove it.
Thus, n-point evaluation and interpolation are well-defined inverse
operations between two representations. The algorithms
described above for these problems take time .
( )
1
( )
0
( )
j
j k
k j
j k
x x
n
k
x x
k
A x y
=
=
÷
÷
÷
=
[
=
[
¿
2
( ) n O
2
( ) n O
Fast multiplication
Question.Can we use the linear-time multiplication method for
polynomials in point-value form to expedite polynomial
multiplication in coefficient form?
Answer.Yes, but we are to be able to convert quickly from one form to
another.


0 1 n 1
0 1 n 1
a , a ,...., a
b , b ,...., b
÷
÷
0 1 2n 2
c , c ,..., c
÷
0 0
2n 2n
1 1
2n 2n
2n 1 2n 1
2n 2n
A( ), B( )
A( ), B( )
A( ), B( )
÷ ÷
e e
e e
e e
0
2n
1
2n
2n 1
2n
C( )
C( )
C( )
÷
e
e
e
Ordinary multiplication
Time Θ(n²)
Pointwise multiplication
Time Θ(n)
Evaluation
Time Θ(n lg n)
Interpolation
Time Θ(n lg n)
Complex roots of unity

There are exactly n complex roots of unity.They form a cyclic
multiplication group:


The value is called the primitive root of unity; all of the
other complex roots are powers of it.
2
1
=
i
n
e
t
e
n
Z 1 0 ÷ =
2
=
ik
n
k
e
t
e
Discrete Fourier Transform
Let F(x) be the polynomial with degree-
bound n, which is a power of 2.e is a primitive n-th root of unity.
Let .Then





The vector is called the Discrete Fourier Transform
of vector a. The matrix is denoted by .

2
0 0
2 1
1 1
2 4 2( 1)
2 2
1 2( 1) ( 1)
1 1
1 1 1 1
1
1
*
1
n
n
n n n
n n
y a
y a
y a
y a
e e e
e e e
e e e
÷
÷
÷ ÷ ÷
÷ ÷
| | | | | |
| | |
| | |
| | |
=
| | |
| | |
| | |
\ . \ . \ .
1 2
1 2 0
( ) ...
n n
n n
F x a x a x a
÷ ÷
÷ ÷
= + + +
( )
k
k
y F e =
0 1 1
( , ,... )
n
y y y y
÷
=
( )
n
F e
How to find F
n
-1
?

Proposition. Let e be a primitive l-th root of unity over a field L.Then


Proof. The l =1 case is immediate since e=1.
Since e is a primitive l-th root, each e
k
,k=0 is a distinct l-th root of unity.




Comparing the coefficients of Z
l-1
on the left and right hand sides of this
equation proves the proposition.





1
0
0
÷
=
¦ ¹
e =
´ `
¹ )
¿
l
k
k
if l > 1
1 otherwise
0 2 1
1 1
1
0 0
1 ( )( )( )...( )
( ) ... ( 1)
÷
÷ ÷
÷
= =
÷ = ÷e ÷e ÷e ÷e =
= ÷ e + + ÷ e
¿ [
l l
l l l l
l l
l k l l k
l l
k k
Z Z Z Z Z
Z Z
Inverse matrix to F
n
Proposition. Let ω be an n-th root of unity.Then,


Proof.



The i=j case is obvious.If i=j then will be a primitive root of unity of
order l, where l|n.Applying the previous proposition completes the proof.

So,



1
n n n
F ( ) F ( ) nE
÷
e · e =
th 1
n n
n-1 n-1
ik ik k(i j)
k=0 k=0
The ij element of F ( )F ( ) is
0, if i j
n, otherwise
÷
÷ ÷
e e
=
¦ ¹
e e = e =
´ `
¹ )
¿ ¿
i j ÷
e
Evaluating y= F
n
(e)

a
Interpolation a= y
1 1
n n
1
F ( ) F ( )
n
÷ ÷
e = e
1
n
1
F ( )
n
÷
e
Fast Fourier Transform

[0] 2 n / 2 1
0 2 4 n 2
[1] 2 n / 2 1
1 3 5 n 1
[0] 2 [1] 2
A (x) a a x a x ... a x
A (x) a a x a x ... a x
A(x) A (x ) xA (x )
÷
÷
÷
÷
= + + + +
= + + + +
= +
So, the problem of evaluating A(x) reduces to:
1. Evaluating the degree-bound n/2 polynomials



2. Combining the results
[0] [1]
A (x) and A (x)
Recursive FFT
2 i
n
n
[0]
0 2 n-2
[1]
1 3 n-1
[0] [0]
[1] [1]
1 n length[a]
2 if n=1
3 then return a
4 e
5 1
6 a (a ,a ,...,a )
7 a (a ,a ,...,a )
8 y Recursive-FFT(a )
9 y Recursive-FFT(a )
10 for k 0 to n/2-1
11
t
÷
e ÷

÷
÷
÷
÷
÷
[0] [1]
k k k
[0] [1]
k+(n/2) k k
n
do y y + y
12 y y - y
13
14 return y
÷ e
÷ e
e÷ ee
Time of the Recursive-FFT
To determine the running time of procedure Recursive-FFT, we note, that
exclusive of the recursive calls, each invocation takes time Θ(n), where n is
the length of the input vector.The recurrence for the running time is therefore

T(n) = 2T(n/2) + Θ(n) = Θ(n log n)
More effective implementations
The for loop involves computing the value twice.We can change the
loop(the butterfly operation):














[1]
e
k
n k
y
÷
÷ e
÷
÷
e÷ ee
[1]
k
[0]
k k
[0]
k+(n/2) k
n
for k 0 to n/2-1
do t y
y y +t
y y -t


[0]
k
y
[0]
k
y
[0] [1]
÷e
k
k n k
y y
[0] [1]
+ e
k
k n k
y y
e
k
n
.
+
÷
Iterative FFT
1) We take the elements in pairs, compute the DFT of each pair, using one
butterfly operation, and replace the pair with its DFT
2) We take these n/2 DFT’s in pairs and compute the DFT of the four vector
elements
.
.
We take 2 (n/2)-element DFT’s and combine them using n/2 butterfly
operations into the final n-element DFT

0 2 4 6
( , , , ) a a a a
0 4
( , ) a a
6
( ) a
2 6
( , ) a a
2
( ) a
4
( ) a
0
( ) a
2
log ) n
1 3 5 7
( , , , ) a a a a
1 5
( , ) a a
7
( ) a
3 7
( , ) a a
3
( ) a
5
( ) a
1
( ) a
0 1 2 3 4 5 6 7
( , , , , , , , ) a a a a a a a a
Iterative-FFT.Code.
0,4,2,6,1,5,3,7000,100,010,110,001,101,011,111000,001,010,011,100,101,110,111

BIT-REVERSE-COPY(a,A)
nlength [a]
for k0 to n-1
do A[rev(k)]a
k

ITERATIVE-FFT
1. BIT-REVERSE-COPY(a,A)
2. nlength [a]
3. for s1 to log n
4. do m2
s
5.

e
m
e
2ti/m

6. for j0 to n-1 by m 1
7. for j0 to m/2-1
8. do for kj to n-1 by m
9. do t eA[k+m/2]
10. uA[k]
11. A[k]u+t
12. A[k+m/2]u-t
13. e e e
m

14. return A
A parallel FFT circuit
a
0
y
0
a
1
y
1
a
2
y
2
a
3
y
3
a
4
y
4
a
5
y
5
a
6
y
6
a
7
y
7

0
2
e
0
2
e
0
2
e
0
2
e
0
4
e
1
4
e
0
4
e
1
4
e
3
8
e
2
8
e
1
8
e
0
8
e
Problem: evaluating all derivatives of a polynomial at a point
a. Given coefficients b
0
,b
1
,…,

b
n-1
such that


Show how to compute A
(t)
(x
0
), for t=0,1,2,…,n-1, in O(n) time.
b. Explain how to find b
0
,b
1
,…,

b
n-1
in O(n lg n) time, given A( ) for
k=0,1,2,…,n-1.


0
k
n
x + e
1
0
0
( ) ( )
n
j
j
j
A x b x x
÷
=
= ÷
¿
Problem: Toeplitz matrices
A Toeplitz matrix is an n × n matrix , such that
for i=2,3,…,n and j=2,3,…,n.
a. Is the sum of two Toeplitz matrices necessarily Toeplitz? What about the
product?
b. Describe how to represent a Toeplitz matrix so that two n × n Toeplitz
matrices can be added in O(n) time.
c. Give an O(n lg n)-time algorithm for multiplying an n × n Toeplitz matrix by
a vector of length n. Use your representation from part (b).
d. Give an efficient algorithm for multiplying two n × n Toeplitz matrices.
Analyze its running time.
( )
ij
A a =
1, 1 ij i j
a a
÷ ÷
=