A Survey on Image Enhancement Techniques: Classical Spatial Filter, Neural Network, Cellular Neural Network, and Fuzzy Filter

Dr. D. H. Rao, Senior Member, IEEE Principal, KLS Gogte Institute of Technology, Udyambag, Belgaum, Karnataka (INDIA) and Patavardhan Prashant Panduranga, Student Member, IEEE Research Scholar, Research Centre, Department of Electronics and Communication Engineering, KLS Gogte Institute of Technology, Udyambag, Belgaum, Karnataka (INDIA) E-mail: prashantgemini(,vahoo.coin / pat itedu, d.h.rao itedu

image has to undergo a process called image enhancement which consists of a collection of techniques that seek to improve the visual appearance of an image. In this paper, a classical spatial filter, neural network (NN), cellular neural network (CNN) and fuzzy filters are presented for the noise reduction of images that are corrupted with additive noise. A three layer neural network is trained for few test images and is used to filter the corrupted colour images. A single layer CNN is developed to reduce the noise in the colour image and compared with that of the classical spatial filter. A new fuzzy filter technique is studied with respect to noisy gray scale images. All the techniques produce convincing results when applied to additive (Gaussian) noisy images. Experimental results are obtained based on the mathematical models of expert systems and compared by numerical measures and visual inspection. It is envisaged to train CNN using gradient descent back-propagation algorithm for better results and extend fuzzy filter technique to reduce noise in colour images.

Abstract- Present day applications require various kinds of noise is completely characterized by its variance, and it is images and pictures as sources of information for interpretation either superimposed or added to the image. and analysis. Whenever an image is converted from one form to Hence, it is a great challenge to develop algorithms another, such as, digitizing, scanning, transmitting, storing, etc., that can remove noise from an image without disturbing its some form of degradation occurs at the output. Hence, the output content. The neighborhood averaging and smoothing by

techniques for noise removal. The classical spatial filtering

ima eavraging

re

The classical

iag procesing

refers to the aggregate of pixels composing an image and operating directly on these pixels. The NN techniques have shown to be a powerful in formation processing tools, especially for those tasks that are difficult to be solved, such as: pattern recognition, image analysis, etc. [4]. These can be perfectly adapted for image
enhancement.

Keywords: Classical Spatial Filter, Neural Network, Gradient Descent Back-propagation Algorithm, Cellular Neural network, Fuzzy Filter
I. INTRODUCTION Images can be contaminated with different types of noise, for different reasons. For example, noise can occur because of the circumstances of recording (electronic noise in a cameras, dust in front of the lens), because of the circumstances of transmission (damaged data) or because of storage, copying, scanning, etc. Impulse noise (e.g. salt and pepper noise) and additive noise (e.g. Gaussian noise) are the most commonly found. Impulse noise is characterized by the fact that the pixels in an image either remain unchanged or get one of the two specific values 0 (black) and 1 (white); an important parameter is the noise density which expresses the fraction of the image pixel that are contaminated. Gaussian

The CNNs proposed by Chua et.al [5] [6] are 6 are aea[5 hea CNNs poste by systems of a large array of nonlinear dynamic with each unit connected only to its processing units (cells) neighboring units in a neighborhood. The 2-dimensional discrete time (DT) CNNs have been shown to be very efficient in image processing and pattern recognition applications [6], such as, shadow detection, edge detection, feature extraction, hole filling, etc. A very important added value of fuzzy set theory is its ability to model and to reason with uncertainty. When processing an image for noise reduction uncertainty occurs. Fuzzy set theory and fuzzy logic allow to model and to work with this uncertainty, and to improve the quality of noise reduction. This paper is divided into seven sections. In section II, the paper described the basis for classical spatial filters. Section III describes hoe noise reduction can be achieved by training conventional NN. The DTCNN architecture for colour image enhancement is described in section IV. In section V a new fuzzy filter method is explained. Section VI is devoted to compare the performance of different image enhancement techniques and discussed with numerical measures.

consisihng

1-4244-0726-5/06/$20.OO '2006 IEEE

2821

they treat all the pixels in the same way. Neural network is a group of elementary processes with strong interconnections. As the objective is to achieve noise reduction rather than blurring.II. Secondly. the average. A spatial filtered output image g(x. such as additive Gaussian distributed noise and they tend to provide too much soothing for impulse like noise [2]. are normalized to unit weights so that the noise reduction process does not introduce an amplitude bias in the processed image. y)h(m + x + C. instead of by the weights in the network are adapted in every iteration during 2822 . y) can be formed by discrete convolution of an input image f(x. In this architecture. y). because not all the pixels will be contaminated with noise in the same way. Lowpass spatial filtering of the smoothing method Figure 2. Weights Weights OutputLayer Median Filter Inputs The linear processing techniques perform reasonably well on images with continuous noise. taking into account characteristics of the neighborhood ofthe pixel. First. called noise reduction masks [1]. Averaging or Lowpass Spatial Filtering This is a type of linear spatial filter. one should try to find an adaptive way to replace a pixel value. For noise reduction. an alternative Each neuron in a layer is connected to the corresponding median filter is developed by Tukey [1] for noise suppression. since all the gray levels in a neighborhood of that pixel. Several common pixel impulse response arrays of lowpass form are listed as. whereby the input and output arrays are centered with respect to one another. In order to perform median filtering in a neighborhood of a pixel. n+ y + C) (1) where C =(L +1)/2. y) with LxL impulse response array h(x. This is not desirable. 1. g(x. NN can be perfectly adapted for image enhancement. The That is the gray level of each pixel is replaced by the median of algorithm involves a high computational complexity. y) according to the relation. Based on the learning algorithm of error back-propagation. NEURAL NETWORK The classical noise reduction spatial filters have two main disadvantages. hl= -{I I 1 (2) h3=-_ 12 10 h=11 2 1(3 (1) (4) Figure 1. determine the median. first sort the values of the pixel and its neighbors. III. The use of NN and fuzzy technique offers a solution. with the outer boundary of g(x.1)/2 pixels set to zero. h should be of lowpass form. CLASSICAL SPATIAL FILTERS The principle objective of enhancement techniques is to process an image so that the result is more suitable than the original image for specific applications. Spatial filters are broadly classified as linear and nonlinear filters and both operate on neighborhood. the number of neurons in each layer is equal to number of pixels with 3 x3 neighborhood as shown in Fig. Spatial domain methods are procedures that operate directly on these pixels. Feed forward NN structure blurs edges and other sharp details. The principal approach to defining a neighborhood about (x. neuron in the previous layer and its neighbors via weights. y) = E f(x. with all positive elements. Neural network architecture Input Layer Hidden Layer h3=- 1I 16 2 4 2 1 2 1 1 These arrays. y) (mask) is to use a square or rectangular subimage area centered at (x. and assign this value to the pixel. Nonlinear techniques often provide a better trade-off between noise smoothing and retention of fine image detail. A self organizing three layered feed forward NN is employed for image enhancement. The equation utilizes the centered convolution. y) of width (L .

a constant external input u. adjacent cells interact directly with each other. Block diagram of one cell of CNN (8) u(t) ...) are application dependent.. The first-order non-linear differential equation defining the dynamics of a cellular neural network can be written as follows: a x'J (t) 1 . j. Cells not directly connected may affect each other indirectly. In general. A constant bias I and the cloning templates lI determine the transient behavior of the cellular non-linear . in which the change in weight is directly proportional to the error signal and the input..self-organization.. The output equation. A(i j. Cellular neural networks were introduced in 1988 [5].. where A and B are (2r+1) x (2r+1) real matrices and I is a scalar number in 2-D CNN's. 1) ==A(k.e. and sat(. then the CNN is called symmetrical or reciprocal. k.. Any cell is connected only to its neighbor cells. Each cell has a state x. we will refer to equation (11) as a zero-input non-symmetric cellular neural network where the n neurons are arranged in a MxN array with n =(MxN) and the interconnection structure is restricted to local neighborhoods of radius r.<i 2823 i . In the original Chua and Yang model.....l<1and lx(t) 1 .. kl sat( Xkl(t) ) + Ii[ uF 1F. y11(t) = f(x11(t)). because of propagation effects of the dynamics of the network.) are known as cloning |.l) ukl.. (t) (9) Yqj (t) = sat( x11(t)) where 1<1<Mi 1. In many applications.b)ykl(t) +± B(ij. When used as an array-processing device..+ / C at R CijeNrij CkleNrlj (6) (7) 1l) Yy. For training the NN gradient descent based supervised learning is used. These adjacent cells can interact directly with each other. 1.k. < Similarly. if IJ(. and I is an independent bias constant. a Xi (t) (1 1) . I. The templates A(. B.. I. where f can be any convenient non-linear function [3]. Cells not in the immediate neighborhood have indirect effect because of the propagation effects of the dynamics in the network.<jN. i.. The template A(. It is the basic unit of a CNN. they can be. j.. the cloning templates do not have to be / space invariant. these Under circumstances. network. and its r-neighborhood NVj> is defined by N>ij = {Ckl max {lk-il.IljI} < r.. Input. (t) I y. ykl(t) + E B(ij. There are two main cases of CNN: continuous-time (CT-CNN) and discrete-time (DT-CNN) cellular neural networks [8]. each cell is a one-dimensional dynamic system.) . j). and xij andYij are the states and the outputs of the network. A= /x(O i) significant feature of CNN is that it has two independent input * capabilities: the generic input and the initial state of the cells.. 0 ukl + Ii C eNrij CkleNrij Y (k) =f ( X (k 1) x( f (x) = sgn(x).. templates.. aij 1/RC > 0. and an output y. the CNN performs a mapping IV.k. the cells. Consider zero inputs (uij= 0 for all i and j) and a constant bias vector I = [1.. The template \| Output B(. 1) and B(i.. The functionality of the CNN array can be controlled by the cloning template A..aij xi.. k.. . C and R conform the integration time constant of the system... Normally they are bounded by. The equations for each cell of a DT-CNN are (10) xij(k) = E A(ij. (t) + E Tj. I). 1< k < M. A special class of two-dimensional cellular neural network is described by ordinary differential equations of the form [6]. The cell located in the position (i j) of a two-dimensional MxN array is denoted by Cij. CELLULAR NEURAL NETWORK CNNs are nonlinear dynamic systems [5] [6]. cells and is referred to as the feedback operator. which are well suited for image processing and pattern recognition.k.) represents the activation function.xx1 (t) where xij is the state of cell C11. xij (0) U ( where F is a function of the cloning template (A.) in turn affects the input control and is referred to as the lll control operator. respectively...k. k.... i.) acts on the output of neighboring . The most general definition of such networks is that they are arrays of identical dynamical systems.J. 1) are space invariant. 1 then yij(t) ... 1< 1 < N } (5) where the size of the neighborhood r is a positive integer number.. that are only locally connected [6].) and B(. but it is not a necessity.. (t) =1/2 (|xij (t) ± 1 . Figure 3.xij (t) +± A(ij. If A(i. B. The equivalent block diagram of a continuous time cell is shown in Fig... at The matrices A(. 3. They consist of a large array of identical processing units with each cell connected only to its nearest neighbors in a neighborhood. 12 MN]T.) and B(.

) l set wrt (. y) and CNW (x. l (x+.1) as shown in Fig.O)(1l1)} 1 -1) 0(-10M.NE direction. This section focuses on fuzzy techniques for image filtering.(O) (1)} {(.1) . The derivative value CNW (x. 2824 . if two out of three derivative values are small. Estimating derivatives and filtering can be seen as a chicken-and-egg problem. (b) Pixel values used to compute the "fuzzy derivative" of the central pixel (x. 5 (a). A simple derivative at the central pixel position (x. NE. This observation will be taken into account when the fuzzy rules formulated to calculate the fuzzy derivative values.. y . y).. .f =. y) in the SW .) {(Q-1).080i-1)} | {(0 1}.lW (a) NW N NEW (b) N )E L.(OtO)>(U1)} Table 1.direction direction NW W S-. The result of these rules (16 in total) is defuzzified and a "correction term" is obtained for the processed pixel value. W.) (. second. for each pixel we derive a value that expresses the degree in which the derivative in a certain direction is small. the principle of the fuzzy derivative is based on the following observation. {(-1)(O1. y) = I (x . The basic cell architecture of CNN for image enhancement_l The Fig. The single layer implementation with template and bias adjustment provides a promising result and reduces the computational complexity. SE.rY. y) as displayed in Fig. For example. y + l) |1. N}) is defined as the difference between the pixel at (x. .1.by -1) position 1. SW.J .({L-1)} ( I (. Nevertheless.10) (Q. the filter estimates a "fuzzy derivative" in order to be less sensitive to local variations due to image structures such as edges.(-f-l(O. The idea is to cancel out the effect of one derivative value that turns out to be high due to noise. the membership functions are adapted accordingly to the noise level to perform "fuzzy smoothing. 4 shows the basic cell architecture of DTCNN for image enhancement operation. CNW (x .I (x. In this approach. in the NW direction the values of CNW (x. For example.(0QO)I. The further construction of the filter is then based on the observation that a small fuzzy derivative most likely is caused by noise. l. Next. Table 1 shows pixels involved in calculating the fuzzy derivatives in each direction. Most fuzzy techniques in image noise reduction mainly deal with fat-tailed noise like impulse noise. . e.I (x. (a) Neighborhood of a central pixel (x. 5 (b) can be calculated.l. most fuzzy techniques are not specifically designed for Gaussian noise or do not produce convincing results when applied to handle this type of noise. This derivative value is denoted by CD (x. for each direction we will apply two fuzzy rules that take this observation into account (and thus distinguish between local variations due to noise and due to image structure). while to find these edges we need filtering.g. Pixels involved calculating the fuzzy derivatives in each direction Consider the 3x3 neighborhood of a pixel (x. y + 1) and CNW (x + 1. y) = I (x. for filtering system wants a good vuS +lI} rindication of the edges. Already several fuzzy filters for noise reduction have been developed. first look for the edges and try to provide a robust estimate by applying fuzzy rules. y).1. These fuzzy filters are able to outperform rank-order filter schemes (such as the median filter). Fuzzy Derivative Estimation Figure 4. (-1. the well known FIRE-filter.t (o. it is safe to assume that no edge is present in the considered direction." The main concern of the proposed filter is to distinguish between local variations due to noise and due to image structure [9]. S. Two important features are presented: first. The application of fuzzy techniques in image processing is a promising research field [9]. y). V. y . In order to accomplish this. Consider an edge passing through the neighborhood of a pixel (x. the weighted fuzzy mean filter. y -1) . y) will be large. y). Figure 5. y) and its neighbor in the direction D. FUZZY FILTER VUSA-!) r. but also derivative values of neighboring pixels perpendicular to the edge's direction can expected to be large.)} l.O)( |? N NE $ ( y + 1) + E | 1 (x+± 1y-1) (0. CN (x. Consequently. E.) S E (-x. and the iterative fuzzy control based filter.O). y) for the NW. Such a value is derived for each direction corresponding to the neighboring pixels of the processed pixel by a fuzzy rule. and that determine the contribution of the neighboring pixel values. Therefore. while a large fuzzy derivative most likely is caused by an edge in the image. y) in the direction D (D C dir = {NW.

Filtered image of Lowpass Spatial Filter for Fig. 13 top left shows input layer output. The fuzzy filter is applied to grayscale test images after adding Gaussian noise and the simulated results are shown in Fig. 15 shows the output of the DTCNN for the % A= 0 -4 0 B= I with I =-1 Figure 8. 7. 8. Original Image Figure 12. The final step in the computation For the classical spatial filter h3 mask for lowpass and of the fuzzy filter is the defuzzification. Figure 10. Noisy Image with salt and pepper noise second part (filtering) have to be distinguished between positive and negative values. 9 shows the noisy images for the original test image Fig. 14 is an output of trained NN for the input noisy templates image Fig. which can be added to the pixel value of masks and techniques and are shown in Fig. for the Figure 9. 7 Figure 11. VI. 16. Filtered image of Median Filter for Fig. 9 Figure 7. Fig. The correction term median filter provides a better results as compared to other delta obtained. a pair of fuzzy rules for each direction is used. RESULTS The efficacy of various image enhancement techniques is demonstrated through simulation results. Filtered image using cascaded structure of Lowpass and Median filters for Fig. Noisy Image with zero mean 0. 0. Fig. 10.01-variance Gaussian noise The three layer NN is trained for noise reduction with the learning rate of 0. and Fig. Fig. top right shows hidden layer output and bottom left shows output layer output which is similar to the original test image. The image quality is measured based on the 2825 . 7 Figure 6. 6. 8.Fuzzy Smoothing To compute the correction term delta for the processed pixel value. The idea behind the rules is the following: if no edge is assumed to be present in a certain direction.1 variance Gaussian noise energy content in the image as shown in Table 2. The shape of the membership function small is adapted each iteration according to an estimate of the (remaining) amount of noise. The first part (edge assumption) can be realized by using the fuzzy derivative value. y). the (crisp) derivative value in that direction can and will be used to compute the correction term.3 and Fig. The method assumes that a percentage p of the image can be considered as homogeneous and as such can be used to estimate the noise density. Adaptive Threshold Selection Instead of making use of larger windows to obtain better results for heavier noise. The objective measure of the image is also done through Peak Signal to Noise Ratio. choose to apply the filter iteratively. 11 and 12 location (x. Noisy Image with zero mean. Fig.

2494* 106 1.6143* 0. 64-74. B. [4] Satyanarayana S. Tests are carried out for different noise levels and the results are demonstrated through simulation. Introduction to Artificial Neural Network Systems. and Gyvez J. Digital Image Processing. 7 computational requirements as the weights in each layer are to be adjusted.0864 * 10 Figure 13. 2664. Filtered image after training NN for Fig. 2001. 1996. IEEE Trans. 2003.. Asia.. Etienne. pp. pp. P. IEEE Trans. I. Filtered image of Neural Network after 140 iterations for Fig.Technique Original Lowpass Spatial Filter Median Filter Salt and Pepper Neural Network Gaussian Noise Red 1. Pearson Education. vol. "Cellular Neural Networks: pp.6264 * 10 Green 0.. vol. 1988. Filtered image for DTCNN architecture [1] Pratt W. K. IEEE Trans. NewYorkWiley. REFERENCES Figure 14. "Colour Image Processing in a Neural Network Environment".1991 * 104' 0. Fuzzy Systems. 1273-1290. Digital Image Processing. [9] Dimitri. 2003.3d ed. vol.. 1996 [5] Chua L. and Yang L.6044* 10 0.6336* 10 0. pp. C and Woods R.. 2000. 1086-1096.8180 * 10 1. Wilfried and Ignace. pp. 8 Figure 15. and Hosur P.01 16* 106 Table 2. [2] Gonzalez R. Jaico Publications. It can be inferred that the image enhancement techniques provides promising results for image analysis and pattern recognition applications. 1996. [8] Rao D. Neural Networks. vol. pp. This technique involves very high (a) (b) Reduction by Fuzzy Image Filtering".7. "Fuzzy Cellular Neural Network for Image Enhancement".8067 * : 0. Dietrich. 11. Energy content of the images for various image enhancement techniques. "Noise Figure 16.0964 * 10 1. H. 1988. vol. M. 35.6258* 10 0.0993 * 10 1.. The NN is trained after 140 iterations where the error is very small in each iteration. CONCLUSION The classical spatial filters provide the promising results and computational complexity involved is negligible. SPIE Proceedings.0791 * 10 1. 1257-1272.. and Dalal S. The fuzzy filter distinguishes between local variations due to noise and due to image structures. "Cellular Neural Networks: Theory". Newdelhi. using a fuzzy derivative estimation. 0. [6] Chua L. 1086-1096. 429-436. 2826 .6528 * 10 Energy Blue 0.8182 * 10 1.. 7.." Video Colour Enhancement Using Neural Networks". Whereas a self-organizing multiplayer NN for image enhancement with back-propagation of error based on gradient descent has been compared. Neural Networks. IEEE Trans. [7] Lee C. [3] Zurada J. (a) Cameraman test image corrupted with Gaussian noise (b) Fuzzy filtered cameraman image VII.7965 * 10 0.9093* 104 0. The benefit of CNNs is used for single layer and the computational speed is very high as compared to NN. and Yang L. Circuits and Systems. 2. 0.