Fourier Transform Model of Image Formation

Discrete Fast Fourier Transform Fourier analysis has long been used in optics as an effective means of predicting what occurs when light passes through various apertures, slits, and curved mirrors. Now, it has gone further to be the foundational concept behind advanced techniques in image processing. Fourier transformation creates a representation of an image in the frequency domain, from dimension X to dimension 1/X [1]. The FT is one technique used to make more compact representation of images (e.g. JPEG compression) by rounding its components to lower arithmetic precisions and removing weak signals. The inverse of this FT provides an approximated reconstruction of the original image [2]. The Fourier transform of a 2-dimensional image f(x,y) is given by [3]:

which in discrete can be expressed as:

The Scilab tool contains a built-in function for calculating Fourier Transforms for 1- and 2dimensional arrays, fft() and fft2(). For images, it is best to use the FFT function with images of dimension in power of 2 (e.g. 128128). Calling this function returns the result with the quadrants of the image interchanged. The fftshift() function rearranges these blocks to their original positions (see Figure 1).

Figure 1. A. original quadrants. B. orientation of quadrants after FFT.

Noting that the FFT is a set of complex numbers, we view the intensities by taking absolute values using the abs() function. Lens as Fourier Transformer

We can see this basic concept by simulating an image created by light entering a lens. For instance, we create a binary image of a circle centered, colored white, and embedded on a black background.
01 I = imread('circle.bmp'); 02 Igray = im2gray(I);

Alternatively, we can create the shape within Scilab and also create a gray representation of it (Figure 2A).
01 x = [-64:1:64]; 02 [X,Y] = meshgrid(x); 03 r = sqrt(X.^2 + Y.^2); 04 circle = zeros(size(X, 1), size(X, 2)); 05 circle(find (r <= 32)) = 1; 06 imshow(circle, []); 07 Igray = im2gray(circle);

We implement FFT on the image and then display the intensity of the result.
08 FIgray = fft2(Igray); // FIgray is complex 09 imshow(abs(FIgray),[]);

The brackets passed to the imshow() function adjusts the image such that the maximum value is set to white (or 1) and the minimum is set to black (or 0). Figure 2B shows the result of our transformation, however with quadrants still interchanged.

Figure 2. A. a binary image of a circle; B. the FFT of the circle; C. its shifted FFT; D. its inverse FFT

We apply fftshift() to have the quadrants back to their proper placements. This result is displayed in Figure 2C.
10 imshow(fftshift(abs(FIgray))), []);

Now, if we want to do an inverse of the FFT we have performed, we do this by applying another FFT on our previous result.

11 imshow(abs(fft2(fft2(Igray))));

This approximately reconstructs our original circle (Figure 2D). However, though the resulting image appears similar to our input, it is an inverted version of the original. We can verify this if we perform the same methods on a text character, as in Figure 3.

Figure 3. A. binary image containing character A; B. the FFT of the image; C. its shifted FFT; D. its inverse FFT

Convolution is a function that solves for the amount of overlap of one function g as it is shifted over the function f. For 2-dimensional functions, convolution is expressed by [4]:

This equation basically describes the relationship between three signals: the input, the impulse response and the output. Simulation of an imaging device Photographs are usually defined by the transfer function (transform of the impulse response) of the camera, which is limited by finite radius of the lens. If minimal light rays are able to enter through the camera due to the lens, the image is reconstructed with lesser resolution. To simulate this, we create an image composed of the text VIP covering about 50% of its total area, and a circle to serve as a lens aperture. We take the FFT of the VIP image and multiply it with the fftshift of the circular aperture. We do not anymore perform FFT on the circle since it already is part of the Fourier plane. Our Scilab code will read as follows:
01 r = imread('circle.bmp'); 02 a = imread('vip.bmp'); 03 04 rgray = im2gray(r); 05 agray = im2gray(a);

06 07 Fr = fftshift(rgray); 08 Fa = fft2(agray); 09 10 FRA = Fr.*(Fa); 11 IRA = fft2(FRA); // inverse FFT 12 FImage = abs(IRA); 13 14 imshow(FImage, [ ]);

We can investigate on the properties (and integrity) of this method by observing the resulting image if allowed to pass through various aperture sizes.

Figure 4. The convolution formed from the text VIP through various aperture sizes. Top row: circular apertures of various diameters (32, 16, 8, 4 pixels). Bottom row: the resulting images after convolution with the different aperture sizes. NOTE: the VIP text was originally written upright.

The results in Figure 4 show that the size of the aperture importantly matters to produce a wellresolved image of the object and to create a better representation of it by allowing more light to enter through the lense. Correlation is the function for measuring the degree of similarity between two functions [1]. For 2-dimensional objects, it is expressed as:

High resulting values mean high correlation, which occur when functions (or images) are greatly identical. This is why correlation is widely used in image processing for pattern recognition. Template matching / Pattern recognition We test the method of correlation by creating an image containing a short text and correlate it with another image containing a letter frequently occurring in it. We perform correlation by multiplying the FT of the single-letter image and the conjugated FT of the text image. We then obtain the result by solving for the inverse FT of their product. The Scilab implementation reads:
01 r = imread('a.bmp'); 02 a = imread('spain.bmp'); 03 04 rgray = im2gray(r); 05 agray = im2gray(a); 06 07 Fr = fft2(rgray); 08 Fa = conj(fft2(agray)); 09 10 FRA = Fr.*(Fa); 11 IRA = fft2(FRA); // inverse FFT 12 FImage = abs(fftshift(IRA)); 13 imshow(FImage, [ ]);

We obtain a resulting image representing areas on which the images are of high correlation (Figure 5C). However, the values are broadly scattered due to the large difference in the character size. We repeat the procedure but this time correlating with a character size more identical to the ones in the text.

Figure 5. A. a sample short text; B. our pattern, which is a particular letter repeating in the text; C. correlation of the images; D. pattern, now a single-letter image more identical to the ones in the short text; E. correlation of the images with more defined peaks.

Notice that the more identical the images, the higher the corresponding values are in the resultant matrix (Figure 5E). We find the highest peaks (or the brightest pixels) on the specific words where the letter A is present (i.e. rain, spain, mainly, and plain). Edge detection using the convolution integral

Edge detection in images can be a way of template matching. For instance, we create a pattern within a 33 matrix such that the total of the values equal to zero. We then convolve this with a sample image for edge detection via the imcorrcoef() function.
01 pattern = [-1 -1 -1; 2 2 2; -1 -1 -1]; 02 03 a = imread('vip.bmp'); 04 agray = im2gray(a); 05 06 img = imcorrcoef(agray, pattern); 07 imshow(img); // horizontal pattern

We also test other templates such as vertical, diagonal and spot.

01 pattern = [-1 2 -1; -1 2 -1; -1 2 -1]; 02 //pattern = [-1 -1 2; -1 2 -1; 2 -1 -1]; 03 // vertical // diagonal right //

//pattern = [-1 -1 -1; -1 8 -1; -1 -1 -1]; spot pattern

Figure 6. Template matching as method for edge detection. Top row: 33 patterns (horizontal, vertical, diagonal and spot); Bottom row: the edges detected via correlation from an image with text VIP.

From our results in Figure 6, we notice that the edges more pronounced in the resulting image are those similar to the template with which the text is correlated with. The letters I and P are the ones least detected with edges when related with the horizontal and diagonal templates. The best detection, however, was obtained by the spot template which, in a sense, does not outline a specific pattern and thus was able to trace approximately all the boundaries of the text image. Special thanks to Gladys R. for sharing her thoughts and immediate realizations, and Dr. Soriano for the supplementary insights. I would rate myself 10 in this activity for comprehending the ideas and producing the desired outputs.

 References: [1] Soriano, 2010. Fourier Transform Model of Image Formation. Applied Physics 186. [2] Wikipedia, 2010. Fourier Analysis. [3] Wang, R., 2007. Two-Dimensional Fourier Transform. [4] Weisstein, E., 2010. Convolution. From MathWorldA Wolfram Web Resource. [5] Peters, R. II, 2008. Image processing: Spatial convolution. Vanderbilt University School of Engineering.

DFT
(Discrete Fourier Transform)

FFT
(Fast Fourier Transform)

Introduction This document describes the Discrete Fourier Transform (DFT), that is, a Fourier Transform as applied to a discrete complex valued series. The mathematics will be given and source code (written in the C programming language) is provided in the appendices.
Theory

Continuous For a continuous function of one variable f(t), the Fourier Transform F(f) will be defined as:

and the inverse transform as

where j is the square root of -1 and e denotes the natural exponent

Discrete Consider a complex series x(k) with N samples of the form

where x is a complex number

Further, assume that that the series outside the range 0, N-1 is extended N-periodic, that is, xk = xk+N for all k. The FT of this series will be denoted X(k), it will also have N samples. The forward transform will be defined as

The inverse transform will be defined as

Of course although the functions here are described as complex series, real valued series can be represented by setting the imaginary part to 0. In general, the transform into the frequency domain will be a complex valued function, that is, with magnitude and phase.

The following diagrams show the relationship between the series index and the frequency domain sample index. Note the functions here are only diagramatic, in general they are both complex valued series.

For example if the series represents a time sequence of length T then the following illustrates the values in the frequency domain.

Notes

The first sample X(0) of the transformed series is the DC component, more commonly known as the average of the input series. The DFT of a real series, ie: imaginary part of x(k) = 0, results in a symmetric series about the Nyquist frequency. The negative frequency samples are also the inverse of the positive frequency samples. The highest positive (or negative) frequency sample is called the Nyquist frequency. This is the highest frequency component that should exist in the input series for the DFT to yield "uncorrupted" results. More specifically if there are no frequencies above Nyquist the original signal can be exactly reconstructed from the samples. The relationship between the harmonics returns by the DFT and the periodic component in the time domain is illustrated below.

DFT and FFT algorithm.

While the DFT transform above can be applied to any complex valued series, in practice for large series it can take considerable time to compute, the time taken being proportional to the square of the number on points in the series. A much faster algorithm has been developed by Cooley and Tukey around 1965 called the FFT (Fast Fourier Transform). The only requirement of the the most popular implementation of this algorithm (Radix-2 Cooley-Tukey) is that the number of points in the series be a power of 2. The computing time for the radix-2 FFT is proportional to

So for example a transform on 1024 points using the DFT takes about 100 times longer than using the FFT, a significant speed increase. Note that in reality comparing speeds of various FFT routines is problematic, many of the reported timings have more to do with specific coding methods and their relationship to the hardware and operating system.
Sample transform pairs and relationships

The Fourier transform is linear, that is a f(t) + b g(t) ---> a F(f) + b G(f)

a xk + b yk ---> a Xk + b Yk

Scaling relationship f(t / a) ---> a F(a f) f(a t) ---> F(f / a) / a

Shifting f(t + a) ---> F(f) e-j 2 pi a f

Modulation f(t) ej 2 pi a t ---> F(t - a)

Duality Xk ---> (1/N) xN-k

Applying the DFT twice results in a scaled, time reversed version of the original series.

The transform of a constant function is a DC value only.

The transform of a delta function is a constant

The transform of an infinite train of delta functions spaced by T is an infinite train of delta functions spaced by 1/T.

The transform of a cos function is a positive delta at the appropriate positive and negative frequency.

The transform of a sin function is a negative complex delta function at the appropriate positive frequency and a negative complex delta at the appropriate negative frequency.

The transform of a square pulse is a sinc function

More precisely, if f(t) = 1 for |t| < 0.5, and f(t) = 0 otherwise then F(f) = sin(pi f) / (pi f)

Convolution f(t) x g(t) ---> F(f) G(f)

F(f) x G(f) ---> f(t) g(t)

xk x yk ---> N Xk Yk xk yk ---> (1/N) Xk x Yk Multiplication in one domain is equivalent to convolution in the other domain and visa versa. For example the transform of a truncated sin function are two delta functions convolved with a sinc function, a truncated sin function is a sin function multiplied by a square pulse.

The transform of a triangular pulse is a sinc2 function. This can be derived from first principles but is more easily derived by describing the triangular pulse as the convolution of two square pulses and using the convolution-multiplication relationship of the Fourier Transform.

Sampling theorem

The sampling theorem (often called "Shannons Sampling Theorem") states that a continuous signal must be discretely sampled at least twice the frequency of the highest frequency in the signal. More precisely, a continuous function f(t) is completely defined by samples every 1/fs (fs is the sample frequency) if the frequency spectrum F(f) is zero for f > fs/2. fs/2 is called the Nyquist frequency and places the limit on the minimum sampling frequency when digitising a continuous sugnal. If x(k) are the samples of f(t) every 1/fs then f(t) can be exactly reconstructed from these samples, if the sampling theorem has been satisfied, by

where

Normally the signal to be digitised would be appropriately filtered before sampling to remove higher frequency components. If the sampling frequency is not high enough the high frequency components will wrap around and appear in other locations in the discrete spectrum, thus corrupting it. The key features and consequences of sampling a continuous signal can be shown graphically as follows.
Consider a continuous signal in the time and frequency domain.

Sample this signal with a sampling frequency fs, time between samples is 1/fs. This is equivalent to convolving in the frequency domain by delta function train with a spacing of fs.

If the sampling frequency is too low the frequency spectrum overlaps, and become corrupted.

Another way to look at this is to consider a sine function sampled twice per period (Nyquist rate). There are other sinusoid functions of higher frequencies that would give exactly the same samples and thus can't be distinguished from the frequency of the original sinusoid.
Appendix A. DFT (Discrete Fourier Transform)
/* Direct fourier transform */ int DFT(int dir,int m,double *x1,double *y1) { long i,k; double arg; double cosarg,sinarg; double *x2=NULL,*y2=NULL; x2 = malloc(m*sizeof(double)); y2 = malloc(m*sizeof(double)); if (x2 == NULL || y2 == NULL)

return(FALSE); for (i=0;i<m;i++) { x2[i] = 0; y2[i] = 0; arg = - dir * 2.0 * 3.141592654 * (double)i / (double)m; for (k=0;k<m;k++) { cosarg = cos(k * arg); sinarg = sin(k * arg); x2[i] += (x1[k] * cosarg - y1[k] * sinarg); y2[i] += (x1[k] * sinarg + y1[k] * cosarg); } } /* Copy the data back */ if (dir == 1) { for (i=0;i<m;i++) { x1[i] = x2[i] / (double)m; y1[i] = y2[i] / (double)m; } } else { for (i=0;i<m;i++) { x1[i] = x2[i]; y1[i] = y2[i]; } } free(x2); free(y2); return(TRUE); }

Appendix B. FFT (Fast Fourier Transform)

/* This computes an in-place complex-to-complex FFT x and y are the real and imaginary arrays of 2^m points. dir = 1 gives forward transform dir = -1 gives reverse transform */ short FFT(short int dir,long m,double *x,double *y) { long n,i,i1,j,k,i2,l,l1,l2; double c1,c2,tx,ty,t1,t2,u1,u2,z; /* Calculate the number of points */ n = 1; for (i=0;i<m;i++) n *= 2; /* Do the bit reversal */ i2 = n >> 1; j = 0; for (i=0;i<n-1;i++) { if (i < j) { tx = x[i]; ty = y[i];

x[i] y[i] x[j] y[j]

= = = =

x[j]; y[j]; tx; ty;

} k = i2; while (k <= j) { j -= k; k >>= 1; } j += k; } /* Compute the FFT */ c1 = -1.0; c2 = 0.0; l2 = 1; for (l=0;l<m;l++) { l1 = l2; l2 <<= 1; u1 = 1.0; u2 = 0.0; for (j=0;j<l1;j++) { for (i=j;i<n;i+=l2) { i1 = i + l1; t1 = u1 * x[i1] - u2 * y[i1]; t2 = u1 * y[i1] + u2 * x[i1]; x[i1] = x[i] - t1; y[i1] = y[i] - t2; x[i] += t1; y[i] += t2; } z = u1 * c1 - u2 * c2; u2 = u1 * c2 + u2 * c1; u1 = z; } c2 = sqrt((1.0 - c1) / 2.0); if (dir == 1) c2 = -c2; c1 = sqrt((1.0 + c1) / 2.0); } /* Scaling for forward transform */ if (dir == 1) { for (i=0;i<n;i++) { x[i] /= n; y[i] /= n; } } return(TRUE); }

Modification by Peter Cusack that uses the MS complex type fft_ms.c.

References

Fast Walker, CRC Press. 1996 Fast Fourier Elliot, D.F. Academic Press, New York, 1982 Fast Fourier Nussbaumer, Springer, New York, 1982

Fourier

Transforms J.S.

Transforms: and Rao,

Algorithms K.R.

Transforms

and

Convolution

Algorithms H.J.

Digital Signal Oppenheimer, A.V. and Prentice-Hall, Englewood Cliffs, NJ, 1975

Shaffer,

Processing R.W.

2 Dimensional FFT
See also Discrete Fourier Transform

Written July 1998

by

Paul

Bourke

The following briefly describes how to perform 2 dimensional fourier transforms. Source code is given at the end and an example is presented where a simple low pass filtering is applied to an image. Filtering in the spatial frequency domain is advantageous for the same reasons as filtering in the frequency domain is used in time series analysis, the filtering is easier to apply and can be significantly faster.
It is assumed the reader is familiar with 1 dimensional fourier transforms as well as the key time/frequency transform pairs.

In the most general situation a 2 dimensional transform takes a complex array. The most common application is for image processing where each value in the array represents to a pixel, therefore the real value is the pixel value and the imaginary value is 0. 2 dimensional Fourier transforms simply involve a number of 1 dimensional fourier transforms. More precisely, a 2 dimensional transform is achieved by first transforming each row, replacing each row with its transform and then transforming each column, replacing each column with its transform. Thus a 2D transform of a 1K by 1K image requires 2K 1D transforms. This follows directly from the definition of the fourier transform of a continuous variable or the discrete fourier transform of a discrete system. The transform pairs that are commonly derived in 1 dimension can also be derived for the 2 dimensional situation. The 2 dimensional pairs can often be derived simply by considering the procedure of applying transforms to the rows and then the columns of the 2 dimensional array.

Delta function transforms to a 2D DC plane

Line of delta functions transforms to a line of delta functions

<--FFT-->

Square pulse

2D sinc function

Note The above example has had the quadrants reorganised so as to place DC (freq = 0) in the center of the image. The default organisation of the quadrants from most FFT routines is as below

Example The following example uses the image shown on the right.

In order to perform FFT (Fast Fourier Transform) instead of the much slower DFT (Discrete Fourier Transfer) the image must be transformed so that the width and height are an integer power of 2. This can be achieved in one of two ways, scale the image up to the nearest integer power of 2 or zero pad to the nearest integer power of 2. The second option was chosen here to facilitate comparisons with the original. The resulting image is 256 x 256 pixels.

The magnitude of the 2 dimension FFT (spatial frequency domain) is

Image processing can now be performed (for example filtering) and the image converted back to the spatial domain. For example low pass filtering involves reducing the high frequency components (those radially distant from the center of the above image). Two examples using different cut-off frequencies are illustrated below.

Low pass filter with a low corner frequency Low pass filter with a higher corner frequency

Source Code
/*------------------------------------------------------------------------Perform a 2D FFT inplace given a complex 2D array The direction dir, 1 for forward, -1 for reverse The size of the array (nx,ny) Return false if there are memory problems or the dimensions are not powers of 2 */ int FFT2D(COMPLEX **c,int nx,int ny,int dir) { int i,j; int m,twopm; double *real,*imag; /* Transform the rows */ real = (double *)malloc(nx * sizeof(double)); imag = (double *)malloc(nx * sizeof(double)); if (real == NULL || imag == NULL) return(FALSE); if (!Powerof2(nx,&m,&twopm) || twopm != nx) return(FALSE); for (j=0;j<ny;j++) { for (i=0;i<nx;i++) { real[i] = c[i][j].real; imag[i] = c[i][j].imag; } FFT(dir,m,real,imag); for (i=0;i<nx;i++) { c[i][j].real = real[i]; c[i][j].imag = imag[i]; } } free(real); free(imag); /* Transform the columns */ real = (double *)malloc(ny * sizeof(double)); imag = (double *)malloc(ny * sizeof(double)); if (real == NULL || imag == NULL) return(FALSE); if (!Powerof2(ny,&m,&twopm) || twopm != ny) return(FALSE);

for (i=0;i<nx;i++) { for (j=0;j<ny;j++) { real[j] = c[i][j].real; imag[j] = c[i][j].imag; } FFT(dir,m,real,imag); for (j=0;j<ny;j++) { c[i][j].real = real[j]; c[i][j].imag = imag[j]; } } free(real); free(imag); return(TRUE); } /*------------------------------------------------------------------------This computes an in-place complex-to-complex FFT x and y are the real and imaginary arrays of 2^m points. dir = 1 gives forward transform dir = -1 gives reverse transform Formula: forward N-1 --1 \ X(n) = --> N / --k=0

- j k 2 pi n / N x(k) e = forward transform n=0..N-1

Formula: reverse N-1 --\ j k 2 pi n / N X(n) = > x(k) e / --k=0 */ int FFT(int dir,int m,double *x,double *y) { long nn,i,i1,j,k,i2,l,l1,l2; double c1,c2,tx,ty,t1,t2,u1,u2,z; /* Calculate the number of points */ nn = 1; for (i=0;i<m;i++) nn *= 2; /* Do the bit reversal */ i2 = nn >> 1; j = 0; for (i=0;i<nn-1;i++) { if (i < j) { tx = x[i];

= forward transform n=0..N-1

ty = x[i] y[i] x[j] y[j]

y[i]; = x[j]; = y[j]; = tx; = ty;

} k = i2; while (k <= j) { j -= k; k >>= 1; } j += k; } /* Compute the FFT */ c1 = -1.0; c2 = 0.0; l2 = 1; for (l=0;l<m;l++) { l1 = l2; l2 <<= 1; u1 = 1.0; u2 = 0.0; for (j=0;j<l1;j++) { for (i=j;i<nn;i+=l2) { i1 = i + l1; t1 = u1 * x[i1] - u2 * y[i1]; t2 = u1 * y[i1] + u2 * x[i1]; x[i1] = x[i] - t1; y[i1] = y[i] - t2; x[i] += t1; y[i] += t2; } z = u1 * c1 - u2 * c2; u2 = u1 * c2 + u2 * c1; u1 = z; } c2 = sqrt((1.0 - c1) / 2.0); if (dir == 1) c2 = -c2; c1 = sqrt((1.0 + c1) / 2.0); } /* Scaling for forward transform */ if (dir == 1) { for (i=0;i<nn;i++) { x[i] /= (double)nn; y[i] /= (double)nn; } } return(TRUE); }