Image Denoising Figure 2.7: Estimation of pixel shiftmaps: (a) reference image, (b) source image, (c) 2, and (d) y,. 2.3 Image Denoising Image denoising is one of the fundamental challenges in the fcld of image processing, where the underlying goal is to recover the original image by removing noise from a noise-contaminated version of the image, Of course, “noise” does not exist on its own, but is created by observers. However, it is generally thought that the features that exist on a very small scale in the image are noise, and that suppressing these while maintaining larger features would provide a clearer output. A digital image is generally represented as a matrix of grey level or colour values. 352.3. Image Denoising Each one of the pixel values of the images is the result of a light intensity measure- ment [79]. Noise may be contaminated in the images during this measurement due to sensor imperfections, environmental conditions, or communication errors, which are often not possible to avoid in practical situations [80,81]. Removing such noise is of great benefit in many applications such as image registration, image restora- tion, visual tracking, image segmentation, and image classification. While image denoising is a well-studied problem, yet it is still an open challenge and several researchers are still continuing to improve the state-of the-art techniques [82]. Depending on the type of disturbances, the noise can affect the image to different extents. Therefore, noise in digital images can be classified into various types, such as salt and pepper noise, Ganssian noise, speckle noise, Poisson noise, and so on. Salt and pepper noise is a common form of noise, which is caused by errors in the data transmission, In this noise, the corrupted pixels can take only the maximum and minimum values in the dynamic range. A median filter is effective for this type of noise. The presence of Gaussian noise is also very common, which is due to electronic components in the imaging system. This noise is additive in nature, that is, it is independent of the pixel intensities of the image. This noise, as the name implies, follows Ganssian distribution and can be suppressed using a spatial filter 2.3.1 Noise Variance Estimation Image noise estimation is of crucial importance for many machine vision algorithms for proces ng and analysing images. It allows the algorithm parameter to adjust to account for the variations in noise level instead of relying on a fixed threshold Examples of the algorithms requiring estimation of noise level include motion esti- mation [83), denoising (84-86), super-resolution [87], shape-from-shading [88], and feature extraction [89]. An image that is corrupted by the additive, zero mean Gaussian noise, can be 362.3. Image Denoising modelled as 12,9) = Inlay) + Iv(z,9), (2.21) where x and y are the vertical and horizontal coordinates of a pixel, J is the noisy image, Ip is the original (true) image, and Jy is the additive Gaussian noise. Noise variance can be estimated from a single image or a sequence of images. Estimation from an image sequence is an over-constrained problem, while the same from a single image is an under-constrained problem, as the algorithm requires further assumptions [90]. A noise estimation algorithm using a camera model from a sequence of images was proposed in [91]. A simple piecewise smooth image prior model was employed in [92] to estimate the noise level from a single image without, any knowledge of the image content. Three different methods to estimate noise level, based on training samples and the Laplacian of images, were proposed in {93]. In {94}, a widely used noise estimation method was proposed based on the mean absolute deviation. A detailed analysis of six noise parameter estimators was presented in [95]. In this part, a fast algorithm [96] to estimate the variance (o2) of noise from a single image is described in brief. The algorithm uses a zero mean operator, which is almost insensitive to the structure of an image. At first, two 3x3 masks, L; and Zs, are considered, each of which approximates the Laplacian of an image. The elements of the masks are [96] o 10 101 1 hh=-hh = and p= ]0 — 1-41 z|9 -4 0 01 0 101 ‘The operator for noise estimation, .V, is then determined using L1 and Ly by 37Image Denoising the following formula [96} 1-2 1 No=2A(la-L)=|-2 4 -2 1-2 1 As can be calculated using Eq. (2.22), the mean value of N, is zero and its variance is 3602, assuming that noise at each pixel has a standard deviation of a. ‘The variance is estimated using absolute deviation as it reduces the compu- tational complexity of the algorithm [96]. ‘The absolute deviation of a Gaussian distribution with zero mean and variance 0? is [96] [ (2.23) (2.24) (2.25) where I(x,y) * N, denotes the value of applying the mask N, at position (x,y) in the image J, and M x N is the size of the image. 2.3.2 Denoising Algorithm The goal of image denoising algorithms is to recover the original image by suppress ing the noise within the observed image. Even though several methods for image denoising exist in the literature, most of them are based on a common theme: av- eraging the pixel values [86]. The methods differ from each other by the way of 382.3. Image Denoising averaging, which may be performed locally: the neighbourhood filtering [97-99], ‘the anisotropic filtering (84, 100] and the Gaussian smoothing model [101], or non- locally: the non-local means filter [86], in the frequency domain: the empirical ‘Wiener filters [97] and wavelet thresholding methods (94, 102], or by the caleulus of variations: the total variation minimization [103]. In this section, two popular filters are described in brief. 2.3.2.1 Median filter ‘The median filter is a nonlinear digital filtering technique used Lo remove noise from, images. This filter is widely used as it is effective at removing noise while preserving useful details in the images (104-107). Specifically, median filter does a better job at removing salt and pepper noise, and those of similar nature. ‘The median filter runs through the image on a pixel-by-pixel basis and looks at cach pixel’s neighbours to decide whether or not it is representative of its surround- ings. With a user-specified neighbourhood, cach pixel value of the original image is, replaced with the median value of the neighbourhood pixels. The pattern of neigh- bours is called the “window” and it is generally of a square size. The window slides over the entire image to cover each individual pixel. The median is calculated, as ustal, by first sorting all the pixel values of a certain window into a numerical order, and then determining the pixel value in the middle position. If the window is of an even size, the average of the two middle pixel values is used. ‘The process of median filtering is illustrated in Figure 2.8, where the solid lines constitute a 3x3 window for the central pixel value of 160. As can be seen, the cen- tral pixel value is rather unrepresentative of the surrounding pixels, thns, would be replaced with the median value of 134, In order to handle pixel values at boundaries, several approaches exist, which have different properties that might be preferred in particular circumstances, Some examples of those approaches are: avoiding process- ing border values, extending border values outside with values at the boundaries, mirroring boundary neighbourhoods, and extending (padding) border values outside 392.3. Image Denoising Neighbourhood values: 125, 129, 130, 133, 134, 135, 136, 137, 160 ‘Median value: 134 Figure 2.8: Calculating the median value of a pixel neighbourhood. with zeros, Figure 2.9 shows the performance of a median filter on a noisy Lena image. The image was distorted with a certain amount of salt and pepper noise, as shown in Figure 2.9(a). Almost 2.5% of the total pixels in the Lena image are randomly contaminated by the salt noise by setting them to the maximum grey level (255 in the case of S-bit greyscale) and another 2.5% by the pepper noise by setting them to the minimum grey level (0 in the case of 8-bit greyscale). Figure 2.9(b) shows the same image after processing with a 3x3 median filter; note that the edges in the image are preserved well. A larger window would provide more smoothing at the cost of higher computational time. ‘The main advantage of a median filter is that it does not create new unrealistic pixel values when it straddles an edge, since the median value is also a value of a pixel in the image. For this reason, this filter is much better at preserving sharp edges than a typical mean filter that replaces each pixel value with the average value of all pixels within the specified image window. The median filter has disadvantages too, It has higher computational burden and is more complex to implement than a mean filter 40Image Denoising ©) Figure 2.9; Use of a median filter to improve an image distorted by salt and pepper noise: (a) noisy image and (b) denoised image. 2.3.2.2 Non-local means filter Unlike a mean or median filter, a non-local means filter takes the mean of the values, of all pixels in the image, weighted by factors that depend on the similarities of these pixels with the target pixel. Assuming @ as the area of an image, and ¢ and j as two pixels within the image, the non-local means algorithm can be defined by the simple formula [86] NENG = FLOW), 6) where NVL[I)(i) is the estimated value for the ith pixel, I(j) is the observed value for the jth pixel, Wi, 9) is the weighting function, and C(}) is a normalizing constant, expressed as [86] CW = we). (2.27) iat ‘The weighting function W(i, 3) can be of many forms. A Gaussian weighting function is given by [86] os Wii) = exp ( aa rat) . (2.28) te 42.4. Image Deblurring where A acts as a degree of filtering and sets the decay rate of the exponential function, and B(i) and B(j) are the local means of the pixel values surrounding i and j, respectively. (i) can be calculated as [86] 1 Bi) Ror DY 10, (2.29) Ker where R(i) © © and denotes a square neighbourhood surrounding i, and |R(i)| is its size. B(j) can also be calculated in the same way. A search window of 5x5 pixels and similarity square neighbourhood of 3x3 pixels are chosen. In order to show how well a non-local means filter works, Figure 2.10(a) shows the Lena image distorted by the Gaussian noise, providing a signal-to-noise ratio (SNR) of 9.134B, and Figure 2.10(b) shows its filtered version. As seen in Fig- ure 2.10(b), the image is almost noise-free, though with a slight loss of details. (b) Figure 2.10: Use of a non-local means filter to improve an image distorted by Gaussian noise: (a) noisy image and (b) denoised image. 2.4 Image Deblurring While using a camera or a ground-based telescope, it is always expected that the recorded image will be a faithful representation of the observed scene, but unfor- 42