Reconstruction of Low Light Images using the Vector Wiener Filter

Stephen D. Ford y, Byron M. Welsh y, Michael C. Roggemann z, and David J. Lee z

yDepartment of Electrical and Computer Engineering zDepartment of Engineering Physics

Air Force Institute of Technology 2950 P Street, Bldg 640 Wright-Patterson AFB, OH 45433-7765

This preprint submitted to JOSA A January 1997. Revised after reviewer comments March 1997. Approved for public release; distribution unlimited.

ABSTRACT
Photon noise imposes severe limitations on the resolution of images detected under low light conditions. These resolution limitations are typically more severe than those imposed by di raction. Nonlinear iterative reconstruction techniques are available which can successfully process these images. Typical iterative methods are computationally expensive, incorporate ad hoc termination criteria, and may su er convergence problems. The scalar Wiener lter is linear, computationally inexpensive, and non-iterative. However, the scalar Wiener lter cannot improve Fourier domain signal-to-noise ratio (SNR). We extend the vector Wiener lter to properly account for photon noise to provide a reconstruction technique that better complements nonlinear iterative methods. Our analysis con rms previous research which showed that photon noise is correlated with respect to spatial frequency (C. L. Matson, \Fourier Spectrum extrapolation and enhancement using support constraints," IEEE Trans. Sig. Processing, vol. 42, pp. 156-163, 1994). The amount of correlation is dependent on the product of the mean optical transfer function (OTF) and mean object spectrum at a di erence frequency. We conduct computer simulations for a class of random Gaussian objects degraded by a known OTF. These results show the extended vector Wiener lter provides dramatic improvement in normalized mean squared error performance when compared to the scalar Wiener lter. The vector Wiener lter can also provide a superresolution capability due to knowledge of the object spatial frequency statistics for frequencies beyond the OTF cuto .

Keywords: image reconstruction, vector Wiener lter, photon-limited detection, mean squared error. 1 Introduction
Image reconstruction refers to post-detection processing techniques that attempt to recover an accurate object estimate based on a priori knowledge of the imaging scenario. Reconstruction of photon-limited imagery has been widely studied with regard to astronomical imaging. In general, two classes of post-detection processing have found application to this problem: correlation-based techniques and blind deconvolution. These methods are implemented to increase Fourier domain signal-to-noise ratio (SNR) at spatial frequencies out to the di raction-limit of the optical system 1]. Correlation-based techniques include Labeyrie's method 2], the Knox-Thompson method 3], and bispectrum processing 4]. In each case, di raction-limited information is encoded as a nonlinear function of the true object intensity. Thus, a computationally intensive phase retrieval problem must be solved to get an estimate of

the object. Also, an estimate of the atmospheric-optical system point spread function (PSF) is usually required to deconvolve the atmospheric turbulence. In contrast, blind deconvolution has been shown to successfully reconstruct photon-limited images without explicit prior knowledge of the PSF 5, 6]. However, these methods typically require the solution of another nonlinear iterative problem. In addition, many blind deconvolution techniques experience convergence problems due to numerical inaccuracies, sensitivity to local minima, and the choice of an initial estimate 7]. We propose a new vector Wiener lter 8] which properly accounts for photon noise in low light images. This new lter can provide a useful, non-iterative complement to the techniques discussed above. The well-known scalar Wiener lter was rst derived for two dimensional images by Helstrom 9] and Slepian 10]. The lter minimizes mean squared error between the true object irradiance and object irradiance estimate. The estimate is constrained to be linear to the detected data which produces a tractable solution. This approach has been widely studied 11, 12, 13, 14, 15, 16] and used in many imaging applications to include enhancement of scanning electron micrographs 17], multichannel processing 18], image recognition 19], and deconvolution of adaptive optics compensated images 20]. The scalar Wiener lter, while useful in many ways, does not improve Fourier domain SNR. Thus, it is not a suitable alternative to nonlinear iterative processing. In addition, the scalar Wiener lter is based on a wide sense stationary (WSS) assumption about the object random process. Thus, the lter is sub-optimal with respect to mean squared error for object ensembles that are not WSS. Since many object ensembles are not WSS in the mean and may not be WSS with regard to covariance about the non-WSS mean 21], the scalar Wiener lter may discard valuable a priori information about a given object. Pratt proposed a generalized Wiener lter that is optimal with respect to mean squared error for non-WSS object ensembles in signal independent noise. He showed that mean squared error is minimized in the transform domain regardless of the type unitary transform applied 8]. The generalized Wiener lter is a vector lter. Vector lters combine the components of the transformed data in an optimal way to yield a nal estimate of each component 8]. In contrast, a scalar lter weights each component of the data independently. In the Fourier domain, diagonal spatial frequency correlation matrices reduce the vector Wiener lter to the scalar Wiener lter. A WSS random process has uncorrelated spatial frequency components 22] and, therefore, a diagonal spatial frequency correlation matrix. Thus, vector and scalar ltering are equivalent for WSS object ensembles 8]. Pratt's vector Wiener lter theory has been extended to images degraded by both blur and signal independent noise 23, 24]. Other extensions to the theory are associated with fast Wiener lter processing algorithms using the Karhunen-Loeve 25], Sine 26], and lower triangular 27] transforms. In addition, vector Wiener lter theory has been applied to image bandwidth compression 28]. In all cases, the generalized vector Wiener lter was applied to images degraded by a known OTF and signal independent noise. We extend the vector Wiener lter in the Fourier domain to properly account for a random optical transfer function (OTF) and measurement noise. We will refer to measurement noise as the combined contribution of signal dependent photon noise and signal independent detector noise. The new lter incorporates the mean and correlation of the OTF, the spatial frequency correlation of the object, and the spatial frequency correlation of the measurement noise component. Our analysis con rms previous research 29, 30] which showed that photon noise is correlated with respect to spatial frequency. The amount of correlation is dependent on the product of the mean OTF and mean object spectrum at a di erence frequency. Computer simulation results are provided for a class of random binary Gaussian function objects degraded by a known OTF and measurement noise. These results show that the vector Wiener lter provides superior reconstructions when compared to the scalar Wiener lter over a wide range of imaging conditions. In addition, the vector Wiener lter can provide a superresolution capability due to knowledge of the object spatial frequency statistics for frequencies beyond the OTF cuto . The previous discussion assumed the object irradiance distribution was a random process with known spatial frequency correlation. The concept of a random object in image reconstruction is not new 8, 9, 10]. In this paper, we assume a priori knowledge of the class of objects to be imaged. For example, one common class of astronomical objects is the binary star pair. A priori knowledge could include the number of components (two), approximate ratio between primary and secondary component irradiance, and approximate object support. This information could be used to develop a frequency domain statistical description as required by the vector lter. In this example, we do not have exact knowledge of the true object irradiance distribution since no knowledge is available about the

component separation or orientation. In general, the statistical object model should be viewed as a constraint on the lter output. Clearly, the object statistics can never be known perfectly in a real application. Thus, the accuracy of the object model will have an impact on the performance of a vector lter. The same observation can be made with regard to a priori constraints associated with other reconstruction methods. The remainder of the paper is organized as follows. Theory is presented in Section 2 to include relevant image models, current lter methods, and the extended vector Wiener lter derivation. Section 3 discusses a simulation and performance metrics used to compare the extended vector Wiener lter with the scalar Wiener lter. Results are presented in Section 4, while Section 5 provides a brief conclusion.

2 Theory
In this section, we review the pertinent image models, linear algebra notation, and current lter methods. We conclude with a development which extends the vector Wiener lter to properly account for measurement noise.

2.1 Image Models

We rst introduce the conventional linear, shift-invariant (LSI) model for incoherent imaging as a precursor to our discussion of current lter methods. The model describes a noisy detected image d(x) which is related to the object irradiance o(x) via 31] d(x) = h(x) o(x) + n(x); (1) where h(x) denotes the PSF, n(x) is an additive noise process, is the convolution operator, and x denotes a point in the spatial domain. In all scalar analysis, one dimensional notation has been adopted for simplicity. Taking the Fourier transform of both sides yields the frequency domain model D(u) = H(u)O(u) + N(u); (2) where a capital letter denotes the Fourier spectrum of the associated lower case quantity and u is a point in the Fourier domain. H(u) is the OTF. As noted in Section 1, the OTF and object spectrum are random quantities. More compact linear algebra versions of Equations (1) and (2) are helpful when considering the higher dimensionality of the vector Wiener lter. Consider an N by N detector array and note that Equation (1) can be written as 32] d = H o + n; (3) where d, o, and n are N 2 by 1 lexicographic ordered column vectors representing detected image, object, and noise distributions, respectively. A boldface character denotes a column vector in all cases. Lexicographic ordering is de ned as the concatenation of a matrix into a column vector 21]. Since Equation (3) represents an LSI system, H is an N 2 by N 2 block circulant matrix representing a single realization of the random PSF 32]. Equation (2) can also written using this linear algebra notation such that D = H O + N; (4) where is an entrywise or Hadamard product 33]. H is an N 2 by 1 column vector containing the OTF. The image model given above considers randomness introduced by the OTF, object, and additive noise. In reality, only a nite number of photons are available for image measurement. Thus, the quantized, random nature of light detection must be considered to better model true lter performance. As noted earlier, we will consider two categories of noise: photon noise and detector noise. Photon noise is due to the random arrival times and locations of photoevents in all photoelectric detectors and is a form of signal dependent noise. Signal dependent refers to the situation in which the strength of the noise depends on the number and distribution of photoevents 1]. Thus,

photon noise cannot be adequately modeled as a signal independent noise process. Photon noise typically imposes more severe limitations than di raction, especially at low light levels. Ideal light detectors directly measure the number of photoevents K occurring in an image and the exact location xk of each photoevent. These detectors are known as photon-limited 1]. Detector noise can be adequately modeled as statistically independent from the signal. Thus, it is clearly di erent than the signal dependent photon noise. The detector noise is represented by the random variable ndet(x) which is assumed to have the following properties: 1. E ndet(x)] = 0 where E ] denotes the expectation operator; ndet is zero mean. E n (x)n (x0 )] = 2 x = x0 2. E ndet(x)ndet(x0 )] = 0 det x 6= x0 ; the detector noise is spatially uncorrelated with standard deviation det. det det 3. ndet is statistically independent of the random variables describing the signal dependent noise. To derive the extended vector Wiener lter, we use the semi-classical model for photoelectric light detection for the photon noise component and an additive term for the detector noise component. The semi-classical model uses Dirac delta functions to represent the arrival locations of the photons such that the detected image d(x) can be written as 1] K X (x ? xn ) + ndet(x); (5) d(x) = where K is the number of photoevents making up the image, xn is the location of the nth photoevent in the image plane, and denotes the Dirac delta function. Taking the Fourier transform yields the frequency domain version of Equation (5) 1] K X (6) D(u) = exp f?j2 uxn g + Ndet (u): Equation (6) is a key expression used in our analysis to account for all noise sources.
n=1 n=1

2.2 Current Filter Methods

Early reconstruction techniques relied on exact knowledge of the OTF or an accurate method to estimate the OTF. One result was the conceptually simple inverse lter 32] D(u) ^ (7) O(u) = H(u) ; ^ where O(u) is the object spectrum estimate. The inverse lter produces acceptable reconstructions under low noise conditions as typically encountered at low spatial frequencies. However, noise at high spatial frequencies is ampli ed. In contrast, Helstrom proposed applying a scalar Wiener lter 9] ^ O(u) = H (u)D(u)(u) ; (8) jH(u)j2 + Gn(u) Go where Gn (u) is the power spectral density of the noise, Go (u) is the power spectral density of the object, and denotes a complex conjugate. The ratio of noise and object power spectral densities alleviates the noise ampli cation problem associated with the inverse lter. The object ensemble was assumed to be WSS to derive the computationally convenient scalar form given in Equation (8). For non-WSS object ensembles, the generalized vector Wiener lter o ers a performance advantage with respect to mean squared error 8]. Rosenfeld and Kak give a generalized transform domain vector lter expression which can be written in the Fourier domain as 24] ^ O = ?OO Hd fHd ?OO Hd + ?NN g?1 D (9)

where ?OO = E OOH ] is the spatial frequency correlation matrix of the object, ?NN = E NNH ] is the spatial frequency correlation matrix of the signal independent noise, and Hd is a diagonal matrix of known OTF elements. The vector Wiener lter has not been investigated on images degraded by a random OTF and photon noise. We conclude this section by deriving a new vector Wiener lter which explicitly considers a statistical description of both the OTF and measurement noise.

2.3 Extended Vector Wiener Filter

We reviewed the vector Wiener lter 8, 24] above and noted the potential performance advantage for non-WSS object ensembles. In this section, we will extend the vector Wiener lter to properly account for a random OTF and measurement noise. We wish to nd an N 2 by N 2 transformation matrix M such that ^ O = M D; (10) where the mean squared error 2 = E (O ? O)H (O ? O)]; ^ ^ (11) is minimized. The overline is shorthand notation for the expectation operator and H denotes a matrix Hermitian transpose. Using the matrix trace operator 33], Equation (11) can be rewritten as 2 = E Trf(O ? O)(O ? O)H g]; ^ ^ (12) where Tr denotes the trace of the designated matrix. Substituting Equation (10) into Equation (12), bringing the expectation operator inside the trace operator, and expanding the expression yields 2 = Tr ? (13) OO ? M?H ? ?OD M H + M?DD M H ; OD where ?OO = E OOH ] is the object spatial frequency correlation matrix, ?OD = E ODH ] is the object-detected image spatial frequency correlation matrix, and ?DD = E DDH ] is the detected image spatial frequency correlation matrix. To nd the transformation M that minimizes 2, take the derivative of Equation (13) with respect to M, set equal to the zero matrix, and solve for M. The resultant derivative is 33] @ 2 = ?2? + 2? M H = 0: OD DD @M Thus, the minimum mean squared error spectral estimate is ^ O = ?OD (?DD )?1 D; where the transformation M = ?OD (?DD )?1 satis es a Wiener-Hopf equation 34]. (14) (15)

To derive the extended vector lter expression, we must nd ?OD and ?DD using the image model given at Equation (6). First, consider the correlation between the uth spatial frequency of the object and vth spatial frequency of the detected image denoted ?OD (u; v) = E O(u)D (v)]: (16) Using standard correlation calculation methods 1, 22, 35], we note that (See Appendix A) (17) ?OD = (K)2 H (v)?On On (u; v); where K is the average number of photoevents in the image, H is the mean OTF, and On is the normalized object spectrum de ned as (18) On(u) = O(u) = O(u) : O(0) K

?On On (u; v) is the correlation between the uth and vth spatial frequencies in the normalized object spectrum. Equation (17) can be expressed using the linear algebra notation introduced earlier in this section. Note that the functional dependence of the OTF on the spatial frequency v is equivalent to multiplying the vth column of ?On On by H(v). This process is equivalent to multiplying by the diagonal matrix Hd such that the nal result for the object-detected image spatial frequency correlation matrix is (19) ?OD = (K)2 ?On On Hd : To complete the derivation, we need to nd the correlation between arbitrary uth and vth spatial frequencies of the detected image denoted ?DD (u; v) = E D(u)D (v)]: (20) Matson and Roggemann have calculated the detected image covariances associated with the real and imaginary components as well as the crosscovariance between real and imaginary components 30]. Our analysis does not explicitly consider the correlation of the real and imaginary components but treats the detected image spectral elements as single complex numbers 1]. The resultant expression is attained using the same methods as above 1, 22, 35] such that (See Appendix B) 2 (21) ?DD (u; v) = (K)2 ?On On (u; v)?HH (u; v) + K H(u ? v)On (u ? v) + P det (u ? v); where ?HH (u; v) is the correlation between the uth and vth spatial frequencies of the OTF, On denotes the normalized mean object spectrum, and P = N 2 is the number of pixels in the detector. Equation (21) can be written conveniently using linear algebra as before which yields the compact expression ?DD = (K)2 ?On On ?HH + ?NN ; (22) 2 where ?NN (u; v) = K H(u ? v)O n (u ? v)+P det (u ? v) is the correlation between uth and vth measurement noise spatial frequencies. Returning to Equation (15), inserting the results given in Equations (19) and (22), and dividing through by the scalar (K)2 gives the nal expression for the extended vector Wiener lter ^ O = ?On On Hd ?On On ?HH + K ?2 ?NN
n o?1

D:

(23)

The above expression is the main result of our analysis and the subject of discussion and experiment in Sections 3 and 4, respectively. To our knowledge, Equation (23) is new and has yet to be applied to images degraded by measurement noise. We nd it convenient to express the measurement noise spatial frequency correlation in terms of SNR quantities. Multiplying ?NN by the scalar K ?2 as required by Equation (23) yields
2 K ?2 ?NN (u; v) = K ?1 H(u ? v)On (u ? v) + (P det=(K)2 ) (u ? v): The single frame image spectrum SNR is de ned as 1] SNR1 (u) = pE jD(u)j] ; varfD(u)g and has been calculated previously for an image degraded by measurement noise yielding 1]

(24) (25)

SNR1(u) = q

jH(u)jjOn(u)j jOn(u)j2varfH(u)g + K ?1 + P

det=(K)
2

(26)

where var denotes the variance of the bracketed expression. The last two quantities in the denominator of Equation (26) are related to the photon noise and the signal independent detector noise, respectively. Consider the following SNR expressions 36] p (27) SNRk = K;

and

SNRr = p K 2 ; (28) P det where SNRk is associated with the photon noise and SNRr is associated with the detector noise. Equations (27) and (28) have been used previously to rewrite the single frame image spectrum SNR 36]. In the same way, we are motivated to write the measurement noise spatial frequency correlation in Equation (23) as K ?2?NN (u; v) = SNR?2 H(u ? v)O n (u ? v) + SNR?2 (u ? v): r k (29)

These SNR quantities control the amount of regularization provided to each spatial frequency correlation component by the vector lter. When the measurement noise is low (i.e. high light level and minimal detector noise), negligible regularization occurs and the lter is faithful to the data. When the measurement noise is high (i.e. low light level or signi cant detector noise), regularization is applied based on the spatial frequency correlation of the photon noise and the strength of the uncorrelated detector noise. Under extremely high noise conditions, the extended vector Wiener lter will \smooth" the data but always consistent with the known object spatial frequency statistics.

3 Simulation
In this section, we provide the details of a simulation used to compare extended vector Wiener lter performance against the scalar Wiener lter. Here, we use the term simulation to refer to Monte Carlo experiments involving repeated application of the lters to random draws of object and measurement noise. Filter performance is evaluated over a large data ensemble using mean squared error and other metrics. The discussion in this section includes details associated with random object, OTF, and measurement noise. We conclude with a brief discussion of the performance metrics used to present results in Section 4.

3.1 Random Object

To compare lter performance on images degraded by measurement noise, a class of simple non-WSS random objects is needed whose spatial frequency statistics can be derived analytically. The data generated for this study is based on a pair of Gaussian functions whose locations are random. Figure 1 shows a sample object realization which can be expressed mathematically as 31] o(x) = hp exp ?
(

x ? xp wp

+ hs exp ?

x ? xs ws

(30)

where xp is the location of the primary function, xs is the location of the secondary function, hp is the peak irradiance of the primary, hs is the peak irradiance of the secondary, wp is the primary width parameter, and ws is the secondary width parameter. xp and xs are independent, uniformly distributed random vectors restricted to a W by W pixel region in the center of the image plane. For example, W = 1 requires both functions to be located at the center of the image plane and the object realization is deterministic. Since the function locations have a uniform distribution, the parameter W can also be viewed as a spatial domain support constraint. As shown in Figure 1, the normalized peak irradiance of the primary is hp = 1 and the secondary is hs = 0:5. Both primary and secondary functions have width parameters wp = ws = 0:5 pixels to closely simulate unresolved point sources. Based on the discussion above, the object spatial frequency statistics are straightforward to derive. The mean object spectrum is O(u) = sinc(Wu) hp wp expf? (wp u)2g + hsws expf? (ws u)2g ; (31)

Normalized Irradiance

1 0.8 0.6 0.4 0.2 0 20 15 10 15 10 5 0 0 20

Pi

xe

ls

Pix

els

Figure 1: Binary Gaussian random object realization of the type used in the lter simulation. where the sinc function 31] is the Fourier transform of the uniform probability density function. The correlation of uth and vth object spatial frequencies can be expressed as
2 2 2 2 ?OO (u; v) = sinc(W(u ? v)) h2 wp expf? wp (u2 + v2 )g + h2ws expf? ws (u2 + v2 )g p s 2 2 2 2 +sinc(Wu)sinc (Wv)hp hs wp ws expf? (wp u2 + ws v2 )g + expf? (ws u2 + wp v2 )g :

(32)

Equations (31) and (32) provide the object spatial frequency statistics for all experiments. Each experiment trial begins by generating a new random object realization with these statistics.

3.2 OTF
The derivation of the extended vector Wiener lter found at Equation (23) assumed a random OTF and incorporated a statistical description of this quantity. However, much valuable information can be gained from scenarios in which the OTF is deterministic and known. All results presented in Section 4 incorporate a known OTF. In all cases, a square pupil function is used to compute the OTF. The PSF associated with the square pupil is a sinc2 (x) pattern with the width of the rst zero crossing given by 31] l = 2Dz ; (33) where l is the width of the rst zero crossing, is the imaging wavelength, z is the observation distance, and D is the square pupil dimension. To study the e ect of the OTF on lter performance, we change the OTF cuto frequency by adjusting D. Since it is more convenient to express these quantities in a normalized pixel space, Equation (33) becomes lp = 2N ; (34) Dp where lp is the width of the rst zero crossing in pixels, N is the length of one side of the detector in pixels, and Dp is the square pupil dimension in pixels. In Section 4, Dp will be used to identify the pupil size and corresponding PSF-OTF for each case.

Table 1: Measurement Noise Cases K (photoevents) SNRk SNRr 10000 100 278 1000 32 28 750 27 21 500 22 14 250 16 7 100 10 3

3.3 Measurement Noise

Equation (29) suggests a measurement noise spatial frequency correlation matrix which is the sum of two components. The photon noise component is the product of the mean OTF and the mean object spectrum at a di erence frequency with a scaling factor SNR?2 . The component associated with the uncorrelated detector noise increases the diagonal k elements by a scaling factor SNR?2. Thus, the measurement noise is correlated with respect to spatial frequency. r Since the scalar Wiener lter does not exploit this information, we expect the extended vector Wiener lter to have an advantage in noise suppression. Each experiment trial generates a detected image which is corrupted by both photon noise and detector noise. Both noise sources can be varied independently using the K and det parameters which correspond directly with light level and detector noise, respectively. An individual trial begins by generating a normalized object realization and scaling to the selected light level as given by K. A noiseless image is then created by multiplying the object spectrum and OTF. A Poisson random number generator is used to corrupt the noiseless image by using its intensity values as mean parameters in the Poisson distribution. Finally, zero mean Gaussian random numbers with standard deviation det are added to model signal independent detector noise. This process is repeated a large number L times to constitute a complete experiment. Table 1 provides corresponding K, SNRk , and SNRr values for cases of interest in the next section. The detector p size was N = 16 by N = 16 pixels with detector noise xed at det = 5 electrons per pixel in all cases. K = 10000 photoevents represents high light level and low noise. Note that SNRk and SNRr provide an indication of the relative contribution of the two noise e ects in each case. For instance, when K = 10000 photoevents, SNRk is much lower than SNRr which implies detector noise will not have a large impact on performance. When K = 100 photoevents, SNRr is lower, indicating detector noise will play a more signi cant role.

3.4 Performance Metrics

To properly compare lter performance, we must have metrics that provide a realistic performance measure. We use mean squared error, a correlation coe cient, and mean squared phase error. Each of these performance metrics provides a di erent method of comparison between the true and estimated object spectra.

3.4.1 Mean Squared Error

First, return to the mean squared error expression at Equation (13) and substitute the M, ?OD , and ?DD matrices associated with the extended vector Wiener lter. After simpli cation and multiplication by K ?2 , the normalized

mean squared error per frequency component for the vector lter is given as 2 = 1 Tr f? (35) On On ? M Hd ?On On g : P We can also derive an equivalent expression for the scalar Wiener lter. Substituting the diagonal transformation matrix Md associated with Equation (8) whose non-zero elements are de ned as H (u) Md (u) = ; (36) 2 1 jH(u)j2 + K + PKdet =Go (u) 2 ( ) into Equation (13) yields o n ?2 2 = 1 Tr ? (37) On On ? Md Hd ?On On ? ?On On Hd Md + Md ?On On ?HH + K ?NN Md : P Equation (37) represents the suboptimal mean squared error associated with the scalar Wiener lter applied to a non-WSS object ensemble. In our Monte Carlo simulation experiment, the squared error per realization is computed and then accumulated over a large ensemble of images created under identical statistical conditions. The normalized squared error as calculated in an individual experiment trial is 1 2 ^ H ^ (38) i = P(K)2 (Oi ? Oi ) (Oi ? Oi): The pertinent sample statistics are the mean of Equation (38) and standard deviation of the mean de ned as
2

v u u t

L 1 X( 2 )2 ? ( 2 )2 ; L(L ? 1) l=1 l

(39)

where L is the number of images in the ensemble. The theoretical mean squared error metrics represented by Equations (35) and (37) and the experimental mean of Equation (38) should be identical within 2 if the experimental i results are accurate. Figure 2 provides a con rmation of this fact for several K values where the object randomness parameter is W = 8 pixels and the pupil size is Dp = 6 pixels. Theoretical and experimental mean squared error values are within 2 in all cases. Similar normalized mean squared error plots are provided in Section 4 using only i the theoretical data from Equations (35) and (37).

3.4.2 Magnitude and Phase Error

In some cases, mean squared error may be a misleading quality indicator with respect to the human observer 37]. Clearly, we can visually compare the accuracy of lter estimates. However, visual comparison is only valid for individual realizations. We need additional metrics which re ect performance over a complete ensemble. In the Fourier domain, magnitude and phase play very di erent roles. In fact, phase plays the more important role in many situations 38]. The scalar Wiener lter cannot compensate for phase distortions due to noise which leads to less deblurring as noise increases 32]. In contrast, the vector Wiener lter incorporates some a priori knowledge of the true object phase which can help compensate for noise without excessive smoothing. Thus, a separate comparison of magnitude and phase error performance is needed to better understand the value of additional a priori object information. Also, the metrics given below will be important in demonstrating lter performance beyond the OTF cuto frequency. A correlation coe cient between true and estimated Fourier spectra has been used previously to compare esti^ mation performance 39]. We will use the following correlation coe cient between O(u) and O(u) given as ^ hO(u)O (u)i ; (40) ^ OO (u) = q ^ hjO(u)j2ihjO(u)j2i

0.5 Theoretical Scalar Filter 0.45 Experimental Scalar Filter Theoretical Vector Filter Experimental Vector Filter 0.4

0.35

0.3

0.25

0.2 0

200

400

600 800 K (Photoevents)

1000

1200

Figure 2: Normalized mean squared error 2 versus K for the scalar and vector lters. Object randomness parameter W = 8 pixels and pupil size Dp = 6 pixels. Theoretical and experimental mean squared error is identical within a standard deviation of the experimental mean. where h i denotes an ensemble average. j OO j takes on values between 0 and 1. A value close to one implies the ^ lter is doing a good job of reconstructing that frequency component. Now consider the phase of the reconstructed object spectrum. Mean squared phase error is de ned as (u) = h( o (u) ? o (u))2i; ^ where o is the phase of the true object spectrum and o is the phase of the estimated object spectrum. ^
2

(41)

Both OO and 2 are two dimensional functions. Direct comparison of two dimensional functions is di cult. ^ However, Fourier spectra exhibit a high degree of radial symmetry. This symmetry allows for radial averaging of OO ^ 2 . Radial averaging is the process of averaging a two dimensional function along concentric circles of symmetry and to produce a one dimensional plot. The data in Section 4 associated with these two metrics will be presented in one dimensional form using radial averaging.

4 Results
In this section, we present data which illustrates the superior performance of the extended vector Wiener lter when compared to the scalar Wiener lter. The detector size is N = 16 by N = 16 pixels for all data cases. j OO j and 2 ^ plots were generated using L = 100000 images. As we noted in Section 3, W represents the square dimension in the center of the image plane in which the random binary Gaussian object components are allowed to exist. Thus, W represents the amount of randomness associated with our object ensemble. Figure 3 compares scalar and vector lter mean squared error performance as a function of W when the pupil size is Dp = 6 pixels and K = 10000 photoevents (SNRk = 100, SNRr = 278). First, note the extended vector lter provides lower mean squared error over a large range of W values. This improved performance is due in large part to the a priori object spatial frequency correlation information used by the vector

0.3

0.25 Scalar Filter Vector Filter 0.2

0.15

0.1

0.05

0 2

8 10 W (Pixels)

12

14

16

Figure 3: Normalized mean squared error 2 versus object randomness parameter W for the scalar and vector lters. Pupil size Dp = 6 pixels and K = 10000 photoevents (SNRk = 100, SNRr = 278). The vector lter provides dramatic performance improvements, especially as W is decreased. lter. The scalar lter has access only to the object power spectrum which is equivalent to the diagonal values of the ?OO matrix. Since the object ensembles are non-WSS, ?OO has non-zero o -diagonal elements. The scalar lter ignores this valuable information and, therefore, is no longer optimal with respect to mean squared error as evidenced by Figure 3. We should also note that the mean squared error associated with the vector lter increases as W increases. Matson noted similar performance for iterative algorithms 29]. His analysis showed that applying support constraints can provide both a superresolution e ect and variance reduction in the noisy Fourier data 29]. However, the superresolution e ect is minimal for well-resolved objects 29]. For our data, the resolution of the object is dependent on the random relative positions of the Gaussian object components. Thus, our results over an ensemble of data are due to both superresolution and variance reduction. Support constraints provide variance reduction by maintaining spatial frequency correlations in the data which provide weighted interpixel averaging 29]. The weights are associated with the Fourier transform of the support function. Thus, as support increases, the Fourier transform of the support function narrows providing less averaging and degraded performance. The vector Wiener lter also provides interpixel averaging based on enforcing degraded data spatial frequency correlations. As W increases, support increases and the Fourier transform of the rect function associated with the support area narrows. The o -diagonal elements of ?OO are reduced which results in a lter transformation matrix M with less o -diagonal structure. Less o -diagonal structure in M is analogous to less interpixel averaging as each spectral component is estimated. In the limit, when no support constraint is used, M is a diagonal matrix and no interpixel averaging occurs. Hence, the mean squared error performance of the vector and scalar Wiener lters is the same for W = 16 pixels in Figure 3. Figure 4 shows a single detected image realization with the associated lter outputs for W = 8 pixels, pupil size Dp = 6 pixels, and K = 10000 photoevents (SNRk = 100, SNRr = 278). Each mesh plot is normalized to a peak value of unity for visual comparison. Clearly, the vector lter produces a sharper output which more closely resembles the true object realization as suggested by the mean squared error data. The mean object at (b) and the PSF at (c) are provided for the convenience of the reader and later reference. As noted in the discussion associated with Figure 3, the vector lter can provide superresolution. Superresolution is de ned as the extension of the detected image to regions where no data was measured 30]. Figures 5 and 6 illustrate

Random Object
1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 5 15 20 1 0.8 0.6 0.4 0.2 0 20 15 10

Mean Object

20 15 10 5 0 0

(a)
1 0.8 0.6 0.4 0.2 0 20

(b)

PSF
1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 15 20

Detected Image K = 10000 Photoevents

15 10 10 5 0 0 5 15

20

(c)
1 0.8 0.6 0.4 0.2 0 20

(d)

Scalar Filter Estimate

1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 15 20

Vector Filter Estimate

15 10 10 5 0 0 5 15

20

(e) (f) Figure 4: Normalized mesh plots showing the improved performance characteristics of the vector lter on photonlimited binary Gaussian function objects. (a) True object realization, object randomness parameter W = 8 pixels, (b) mean object, (c) PSF, pupil size Dp = 6 pixels, (d) detected image K = 10000 photoevents (SNRk = 100, SNRr = 278), (e) scalar lter estimate, (f) vector lter estimate. The vector lter estimate is sharper and better resolved than the scalar lter estimate.

1 0.9 0.8 0.7 0.6 Scalar Filter Vector Filter

|O |

0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 Normalized Spatial Frequency 1.2 1.4

Figure 5: Radially averaged j OO j as a function of normalized spatial frequency for the scalar and vector lters. ^ The spatial frequency at unity corresponds to the OTF cuto frequency for a Dp = 6 pixel pupil function. Object randomness parameter W = 8 pixels and K = 1000 photoevents (SNRk = 32, SNRr = 28). The vector lter provides superresolution. this idea by comparing vector lter j OO j and 2 with those associated with the scalar lter. In both plots, the pupil ^ size is Dp = 6 pixels, the object randomness parameter is W = 8 pixels and K = 1000 photoevents (SNRk = 32, SNRr = 28). Note that the light level has been reduced in this case. In addition, the spatial frequency unity represents the OTF cuto frequency. The scalar lter cannot provide superresolution since it sets spatial frequencies beyond the OTF cuto to zero 32]. Thus, j OO j for the scalar lter drops sharply beyond the cuto frequency. ^ However, the vector lter is able to maintain a high correlation coe cient at these same frequencies. Figure 6 supports the idea of a superresolution e ect since 2 associated with the vector lter is much lower beyond unity spatial frequency. As noted above, the a priori object knowledge incorporated in the vector Wiener lter provides superresolution since the object randomness parameter W can be viewed as a spatial domain support constraint. This support constraint is incorporated directly in ?OO and O. Support constraints have along research history as a means to superresolve data in the frequency domain 29, 30]. If we reconsider the form of the vector Wiener lter at Equation (23) and the scalar Wiener lter at Equation (8), it is obvious that neither lter incorporates detected image information beyond the OTF cuto frequency. At those high spatial frequencies, the vector lter relies exclusively on object statistical information. In contrast, the scalar lter is not capable of incorporating this knowledge. Instead, the frequency components beyond the OTF cuto are set to zero. Thus, the vector lter should continue to perform better than the scalar lter as the OTF cuto frequency is adjusted lower. Figure 7 shows lter mean squared error performance as a function of the pupil size Dp when W = 8 pixels and K = 1000 photoevents (SNRk = 32, SNRr = 28). Figure 7 shows that the vector lter provides lower mean squared error than the scalar lter in all cases. As in Figure 3, this mean squared error performance is the combined result of both a superresolution e ect and enforcing detected data spatial frequency correlations. In Section 2, we derived the extended vector Wiener lter based on the image model given at Equation (6) with the idea of properly modeling all noise e ects. Thus, we are clearly interested in the performance of this new vector lter as measurement noise becomes more dominant. Figure 2 of Section 3 provides mean squared error performance as a function of K for W = 8 pixels and pupil size Dp = 6 pixels. Clearly, the performance of both lters improves

3 Scalar Filter Vector Filter

2.5
2

Radians

< ( - ) > O
2

1.5

0.5

0 0

0.2

0.4 0.6 0.8 1 Normalized Spatial Frequency

1.2

1.4

Figure 6: Radially averaged 2 as a function of normalized spatial frequency for the scalar and vector lters. The spatial frequency at unity corresponds to the OTF cuto frequency for a Dp = 6 pixel pupil function. Object randomness parameter W = 8 pixels and K = 1000 photoevents (SNRk = 32, SNRr = 28). The vector lter provides superresolution.

0.5 0.45 0.4 0.35 Scalar Filter Vector Filter

0.3 0.25 0.2 0.15 0.1 0.05 3

6 Dp (Pixels)

Figure 7: Normalized mean squared error 2 versus pupil size Dp for the scalar and vector lters. Object randomness parameter W = 8 pixels and K = 1000 photoevents (SNRk = 32, SNRr = 28). The vector lter provides lower mean squared error than the scalar lter due to superresolution and degraded data correlation e ects.

1 0.9 0.8 0.7 0.6 K = 1000 K = 750 K = 500 K = 250

s v v s v s v s

|O |

0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 Normalized Spatial Frequency 1.2 1.4

Figure 8: Radially averaged j OO j as a function of normalized spatial frequency for the scalar and vector lters. The ^ spatial frequency at unity corresponds to the OTF cuto frequency for a Dp = 6 pixel pupil function and object randomness parameter W = 8 pixels. The v and s designators di erentiate between vector and scalar lter traces. The vector lter continues to provide superior reconstruction for frequencies beyond the OTF cuto even as light level is reduced. as light level increases. However, we must also note that the vector lter reduces mean squared error by a wider margin as light level increases. At K = 100 photoevents the vector Wiener lter provides an additional 14 % decrease in mean squared error below the baseline established by the scalar lter. At K = 1000 photoevents the decrease in mean squared error is 36 %. This trend is expected since the o -diagonal elements of the measurement noise spatial frequency correlation matrix ?NN are due to the photon noise. As light level is reduced, photon noise is less dominant and ?NN becomes more diagonal which minimizes the importance of the o -diagonal elements. As noted before, it is access to the o -diagonal components of the correlation matrices which provides the di erence between vector and scalar lters. Figures 8 and 9 also illustrate the superior noise suppression of the extended vector Wiener lter where the object randomness parameter W and pupil size Dp are unchanged from Figure 2. The j OO j and 2 ^ metrics are superior across all noise cases with a superresolution e ect still visible beyond the OTF cuto frequency. Figure 10 provides detected image and lter outputs when K = 500 photoevents (SNRk = 22,SNRr = 14). The scalar lter simply smooths the noisy data and, therefore, is not able to resolve the two object components in (e). In contrast, the extended vector Wiener lter resolves the components in (f). Note how the object spatial frequency statistics enforce the support constraint imposed by the parameter W.

5 Conclusion
We extended the vector Wiener lter to properly account for both photon noise and detector noise. Our analysis con rmed previous research which showed that photon noise is correlated with respect to spatial frequency 29, 30]. The amount of correlation is dependent on the product of the mean OTF and mean object spectrum at a di erence frequency. Simulation results show that the new lter provides superior reconstructions when compared to the scalar Wiener lter on non-WSS object ensembles. Comparisons were conducted while varying the object randomness parameter W, the support of the OTF, and the measurement noise level. In addition, we illustrated

3.5 K = 1000 K = 750

2

s s s s

Radians

K = 500 2.5 K = 250

< ( - ) > O

2
v v v

1.5

0.5

0 0

0.2

0.4 0.6 0.8 1 Normalized Spatial Frequency

1.2

1.4

Figure 9: Radially averaged 2 as a function of normalized spatial frequency for the scalar and vector lters. The spatial frequency at unity corresponds to the OTF cuto frequency for a Dp = 6 pixel pupil function and object randomness parameter W = 8 pixels. The v and s designators di erentiate between vector and scalar lter traces. The vector lter continues to provide superior reconstruction for frequencies beyond the OTF cuto even as light level is reduced. the superresolution capability of the vector lter by examining lter performance beyond the OTF cuto frequency. Finally, individual lter output realizations were presented to graphically demonstrate extended vector Wiener lter capabilities. The analysis in this paper assumed the object irradiance distribution was a random process with known rst and second order spatial frequency statistics. In all cases, we assumed a priori knowledge of the class of objects, not knowledge of a given object realization. Clearly, the vector Wiener lter is not applicable in situations where no information is available about the type of objects to be imaged. However, this technique can provide an alternative to nonlinear iterative methods when a useful statistical description of the object is available. The vector lter also involves the manipulation of large matrices to generate the lter transformation matrix M. However, this calculation need only be accomplished once for a given set of imaging parameters. In contrast, most iterative methods involve signi cant computational burden at each application.

Random Object
1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 5 15 20 1 0.8 0.6 0.4 0.2 0 20 15 10

Mean Object

20 15 10 5 0 0

(a)
1 0.8 0.6 0.4 0.2 0 20

(b)

PSF
1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 15 20

Detected Image K = 500 Photoevents

15 10 10 5 0 0 5 15

20

(c)
1 0.8 0.6 0.4 0.2 0 20

(d)

Scalar Filter Estimate

1 0.8 0.6 0.4 0.2 0 20 15 10 10 5 0 0 5 15 20

Vector Filter Estimate

15 10 10 5 0 0 5 15

20

(e)

(f)

Figure 10: Normalized mesh plots showing the e ect of measurement noise on lter performance. (a) True object realization, object randomness parameter W = 8 pixels, (b) mean object, (c) PSF, pupil size Dp = 6 pixels, (d) detected image, K = 500 photoevents (SNRk = 22, SNRr = 14), (e) scalar lter estimate, (f) vector lter estimate. The vector lter separates the primary and secondary component where the scalar lter simply smooths the detected image.

6 REFERENCES
1] M. C. Roggemann and B. M. Welsh, Imaging Through Turbulence. CRC Press: Boca Raton, FL, 1996. 2] A. Labeyrie, \Attainment of di raction limited resolution in large telescopes by fourier analyzing speckle patterns in star images," Astron. Astrophys., vol. 6, pp. 85{87, 1970. 3] K. T. Knox and B. J. Thompson, \Recovery of images from atmospherically degraded short exposure images," Astrophys. J., vol. 193, pp. 45{48, 1974. 4] A. Lohman, W. G. Weigelt, and B. Wirnitzer, \Speckle masking in astronomy: triple correlation theory and applications," Appl. Opt., vol. 22, pp. 4028{4037, 1983. 5] T. J. Schulz, \Multiframe blind deconvolution of astronomical images," J. Opt. Soc. Am. A, vol. 10, pp. 1064{ 1073, 1993. 6] T. J. Holmes, \Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach," J. Opt. Soc. Am. A, vol. 9, pp. 1052{1061, 1992. 7] D. Kundur and D. Hatzinakos, \Blind image deconvolution," IEEE Signal Processing Magazine, vol. 13, pp. 43{ 64, 1996. 8] W. K. Pratt, \Generalized wiener lter computation techniques," IEEE Trans. Computers, vol. C-21, pp. 636{ 641, 1972. 9] C. W. Helstrom, \Image restoration by the method of least squares," J. Opt. Soc. Am., vol. 57, pp. 297{303, 1967. 10] D. Slepian, \Linear least-squares ltering of distorted images," J. Opt. Soc. Am., vol. 57, no. 7, pp. 918{922, 1967. 11] J. L. Horner, \Optical spatial ltering with the least mean-square-error lter," J. Opt. Soc. Am., vol. 59, pp. 553{558, 1969. 12] K. Heide, \Least squares image restoration," Opt. Comm., vol. 31, pp. 279{284, 1979. 13] S. A. Kassam, T. L. Lim, and L. J. Cimini, \Two-dimensional lters for signal processing under modeling uncertainties," IEEE Trans. Geosci. Rem. Sensing, vol. GE-18, pp. 331{336, 1980. 14] S. Jiang and A. A. Sawchuk, \Noise updating repeated wiener lter and other adaptive noise smoothing lters using local image statistics," Appl. Opt., vol. 25, pp. 2326{2337, 1986. 15] L. Guan and R. K. Ward, \Restoration of randomly blurred images by the wiener lter," IEEE Trans. Acoust. Speech, Signal Processing, vol. 37, pp. 589{592, 1989. 16] L. Guan and R. K. Ward, \Restoration of stochastically blurred images by the geometrical mean lter," Opt. Eng., vol. 29, pp. 289{295, 1990. 17] B. L. Lewis and D. J. Sakrison, \Computer enhancement of scanning electron micrographs," IEEE Trans. Circuits Syst., vol. CAS-22, pp. 267{278, 1975. 18] N. P. Galatsanos and R. T. Chin, \Digital restoration of multichannel images," IEEE Trans. Acoust. Speech, Signal Processing, vol. 37, pp. 415{421, 1989. 19] E. Marom and H. Inbar, \New interpetations of wiener lters for image recognition," J. Opt. Soc. Am. A, vol. 13, pp. 1325{1330, 1996. 20] M. C. Roggemann, D. W. Tyler, and M. F. Bilmont, \Linear reconstruction of compensated images: theory and experimental results," Appl. Opt., vol. 31, pp. 7429{7441, 1992.

21] B. R. Hunt and T. M. Cannon, \Nonstationary assumptions for gaussian models of images," IEEE Trans. Sys. Man Cyber., vol. SMC-6, pp. 876{882, 1976. 22] A. Papoulis, Probability, Random Variables, and Stochastic Processes. McGraw-Hill: New York, 3rd ed., 1991. 23] W. K. Pratt and F. Davarian, \Fast computational techniques for pseudoinverse and wiener image restoration," IEEE Trans. Comput., vol. C-26, pp. 571{580, 1977. 24] A. Rosenfeld and A. C. Kak, Digital Picture Processing, vol. 1. Academic Press: New York, 2nd ed., 1982. 25] A. K. Jain, \An operator factorization method for restoration of blurred images," IEEE Trans. Comput., vol. C26, pp. 1061{1071, 1977. 26] A. K. Jain, \A fast karhunen-loeve transform for digital restoration of images degraded by white and colored noise," IEEE Trans. Comput., vol. C-26, pp. 560{571, 1977. 27] A. Habibi, \Fast suboptimal wiener ltering of markov sequences," IEEE Trans. Comput., vol. C-26, pp. 443{ 447, 1977. 28] H. B. Kekre and J. K. Solanki, \Image model and spectral extrapolation in transform image coding," Int. J. Electronics, vol. 45, pp. 465{474, 1978. 29] C. L. Matson, \Fourier spectrum extrapolation and enhancement using support constraints," IEEE Trans. Sig. Processing, vol. 42, pp. 156{163, 1994. 30] C. L. Matson and M. C. Roggemann, \Noise reduction in adaptive-optics imagery with the use of support constraints," Appl. Opt., vol. 34, pp. 767{780, 1995. 31] J. W. Goodman, Fourier Optics. McGraw-Hill: New York, 1968. 32] A. K. Jain, Fundamentals of Digital Image Processing. Prentice-Hall: Englewood Cli s, New Jersey, 1989. 33] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis. Cambridge University Press: New York, 1991. 34] L. L. Scharf, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis. Addison-Wesley: Reading, Massachusetts, 1991. 35] J. W. Goodman, Statistical Optics. John Wiley and Sons: New York, 1985. 36] B. M. Welsh, \Speckle imaging signal-to-noise ratio performance as a function of frame integration time," J. Opt. Soc. Am A., vol. 12, pp. 1364{1374, 1995. 37] M. M. Sondhi, \Image restoration: The removal of spatially invariant degradations," Proc. IEEE, vol. 60, pp. 842{853, 1972. 38] A. V. Oppenheim and J. S. Lim, \The importance of phase in signals," Proc. IEEE, vol. 69, pp. 529{541, 1981. 39] B. M. Welsh and M. C. Roggemann, \Signal-to-noise comparison of deconvolution from wave-front sensing with traditional linear and speckle image reconstruction," Appl. Opt., vol. 34, pp. 2111{2119, 1995.

A Object-Detected Image Spatial Frequency Correlation Matrix

In this appendix, Equation (17) will be derived from Equation (16). Substituting Equation (6) into Equation (16) and writing as the sum of two terms yields
"

?OD (u; v) = E O(u)

K X n=1

exp fj2 vxn g + E O(u)Ndet (v)]

= ?(1) (u; v) + ?(2) (u; v): (42) OD OD The second term in Equation (42) is zero since Ndet is both independent of O and zero mean. The rst term can be evaluated using nested conditional expectations following the technique presented in references 1, 35]. The random quantities are the object spectrum, OTF, total number of photoevents K, and photon arrival location xn. Bayes rule 22] can be used to rewrite the joint probability density function in the rst term using conditional probability density functions. These conditional probability density functions translate to conditional expectations such that
" " " "

?OD (u; v) = ?OD (u; v) = EO EHjO EK jH;O ExjK;H;O O(u)

(1)

K X n=1

exp fj2 vxn g

####

(43)

where the notation EAjB denotes the expected value of the random variable A given B. The innermost conditional expectation of Equation (43) has been evaluated previously 35] which allows ExjK;H;O to take the form
"

ExjK;H;O O(u)

K X n=1

exp fj2 vxn g = K H (v)O(u)On (v);

(44)

where the normalized object spectrum On(u) can be written as On(u) = O(u) = O(u) : (45) O(0) K Evaluation of the remaining nested expectations is trivial since K, H, and O are all mutually independent. This mutual independence converts the nested quantities into an uncoupled product of expectations which yields (46) ?OD (u; v) = K H (v)E O(u)On(v)] : Clearly, the expectation above is the object spatial frequency correlation except for a normalization factor associated with O(u). Thus, Equation (45) can be used to write Equation (46) as K H (v)E O(u)On(v)] = (K)2 H (v)E On(u)On (v)] : (47) Finally, Equation (42) is (48) ?OD (u; v) = (K)2 H (v)?On On (u; v); where ?On On (u; v) is the correlation between the uth and vth spatial frequencies of the normalized object spectrum.

B Detected Image Spatial Frequency Correlation Matrix

In this appendix, Equation (21) will be derived from Equation (20). Substituting the image model at Equation (6) into Equation (20) and expanding the expression within the expectation yields
"

?DD (u; v) = E

K K XX n=1 m=1

exp f?j2 (uxn ? vxm )g + E Ndet (u)Ndet (v)] (49)

= ?(1) (u; v) + ?(2) (u; v); DD DD

where the cross terms are zero since Ndet is zero mean and independent of K, H, and O. Consider the rst term and apply the nested conditional expectation procedure from Appendix A which yields 1, 35]
" " " "

?DD (u; v) = EO EHjO EK jH;O ExjK;H;O

(1)

K K XX

The double summation in ExjK;H;O has two types of terms: 1. K terms in which n = m. 2. K 2 ? K terms in which n 6= m.

n=1 m=1

exp f?j2 (uxn ? vxm )g

####

(50)

Thus, we can rewrite ExjK;H;O in Equation (50) in a di erent form

"

ExjK;H;O

exp f?j2 (uxn ? vxm )g = n=1 m=1 " # K X ExjK;H;O exp f?j2 ((u ? v)xn )g n=1 n=m # " K K XX + ExjK;H;O exp f?j2 (uxn ? vxm )g : n=1 m=1 n6=m

K K XX

(51)

Matson and Roggemann have previously calculated the two terms in Equation (51) associated with the real and imaginary components as well as the crosscovariance between real and imaginary components 30]. Our analysis does not explicitly consider the correlation of the real and imaginary components but treats the detected image spectral elements as single complex numbers 1]. The resultant expression is attained using standard correlation calculation methods 1, 22, 35] yielding
"

ExjK;H;O

K K XX n=1 m=1

exp f?j2 (uxn ? vxm )g

= (K 2 ? K)On (u)On(v)H(u)H (v)

+ K H(u ? v)On (u ? v): (52) With Equation (52) in hand, EK jH;O , EHjO , and EO can be evaluated exactly as in Appendix A by noting the mutual independence of K, H, and O which leads to ?(1) (u; v) = (K 2 ? K)?On On (u; v)?HH (u; v) + K H(u ? v)On (u ? v): (53) DD The random variable K is conditionally Poisson distributed given H and O 1]. Therefore, the second moment of K can be written as ? (54) E K 2] = K 2 = K + K 2 : Using Equation (54), we can rewrite Equation (53) in its nal form ?(1) (u; v) = (K)2?On On (u; v)?HH (u; v) + K H(u ? v)O n(u ? v): (55) DD The second term of Equation (49) has been evaluated previously 1] for the assumptions given in Section 2 such that
2 ?(2) (u; v) = E Ndet (u)Ndet (v)] = P det (u ? v); (56) DD where P is the number of pixels in the detector array. Combining the results from Equations (55) and (56) gives the nal expression for ?DD (u; v) 2 (57) ?DD (u; v) = (K)2 ?On On (u; v)?HH (u; v) + K H(u ? v)On (u ? v) + P det (u ? v):