398 views

Uploaded by Alessio

save

You are on page 1of 10

North-Holland

**Nonlinear total variation based noise removal algorithms*
**

L e o n i d I. R u d i n 1, S t a n l e y O s h e r a n d E m a d F a t e m i 2

Cognitech Inc., 2800, 28th Street, Suite 101, Santa Monica, CA 90405, USA

**A constrained optimization type of numerical algorithm for removing noise from images is presented. The total
**

variation of the image is minimized subject to constraints involving the statistics of the noise. The constraints are imposed

using Lagrange multipliers. The solution is obtained using the gradient-projection method. This amounts to solving a time

dependent partial differential equation on a manifold determined by the constraints. As t---~0othe solution converges to a

steady state which is the denoised image. The numerical algorithm is simple and relatively fast. The results appear to be

state-of-the-art for very noisy images. The method is noninvasive, yielding sharp edges in the image. The technique could

be interpreted as a first step of moving each level set of the image normal to itself with velocity equal to the curvature of

the level set divided by the magnitude of the gradient of the image, and a second step which projects the image back onto

the constraint set.

**1. I n t r o d u c t i o n ensemble of all possible pictures. This procedure
**

is L 2 n o r m dependent. H o w e v e r it has been

The presence of noise in images is unavoid- conjectured in ref. [6] that the proper norm for

able. It m a y be introduced by the image forma- images is the total variation ( T V ) n o r m and not

tion process, image recording, image transmis- the L 2 norm. T V norms are essentially L 1 norms

sion, etc. These r a n d o m distortions m a k e it dif- of derivatives, hence L1 estimation procedures

ficult to p e r f o r m any required picture processing. are m o r e appropriate for the subject of image

F o r example, the feature oriented enhancement estimation (restoration). The space of functions

introduced in refs. [6,7] is very effective in re- of b o u n d e d total variation plays an important

storing blurry images, but it can be " f r o z e n " by role when accurate estimation of discontinuities

an oscillatory noise component. E v e n a small in solutions is required [6,7].

a m o u n t of noise is harmful when high accuracy is Historically, the L~ estimation methods go

required, e.g. as in subcell (subpixel) image back to Galileo (1632) and Laplace (1793). In

analysis. comparison to the least square methods where

In practice, to estimate a true signal in noise, closed f o r m linear solutions are well understood

the most frequently used methods are based on and easily computed, the L 1 estimation is non-

the least squares criteria. The rationale comes linear and computationally complex. Recently

f r o m the statistical argument that the least the subject of L 1 estimation of statistical data has

squares estimation is the best over an entire received renewed attention by the statistical

* Research supported by DARPA SBIR Contract c o m m u n i t y , see e.g. ref. [13].

#DAAH01-89-C0768 and by AFOSR Contract #F49620-90- Drawing on our previous experience with

C-0011. shock related image e n h a n c e m e n t [6,7], we pro-

1 E-mail: cogni!leonid@aerospace.aero.org.

2 Current address: Institute for Mathematics and its Appli- pose to denoise images by minimizing the total

cations, University of Minnesota, Minneapolis, MN 55455, variation n o r m of the estimated solution. We

USA. derive a constrained minimization algorithm as a

**0167-2789/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved
**

260 L.I. Rudin et al. / Noise removal algorithms

time dependent nonlinear PDE, where the con- better than the MEM) with the same

straints are determined by the noise statistics. constraints)- see e.g. ref. [5].

Traditional methods attempt to reduce/ The L 1 norm is usually avoided since the

remove the noise component prior to further variation of expressions like Salu[ dx produces

image processing operations. This is the ap- singular distributions as coefficients (e.g. 6 func-

proach taken in this paper. However, the same tions) which cannot be handled in a purely alge-

TV/L1 philosophy can be used to design hybrid braic framework. However, if L 2 and L 1 approxi-

algorithms combining denoising with other noise mations are put side by side on a computer

sensitive image processing tasks. screen, it is clear that the L 1 approximation looks

better than the "same" L 2 approximation. The

" s a m e " means subject to the same constraints.

2. Nonlinear partial differential equations based This may be at least partly psychological; how-

denoising algorithms. ever, it is well known in shock calculations that

the L 1 n o r m of the gradient is the appropriate

Let the observed intensity function u0(x, y) space. This is basically the space of functions of

denote the pixel values of a noisy image for x, bounded total variation: BV. For free, we get the

y ~ O. Let u(x, y) denote the desired clean removal of spurious oscillations, while sharp sig-

image, so nals are preserved in this space.

In ref. [6] the first author has introduced a

Uo(X, y) = u(x, y) + n(x, y), (2.1) novel image enhancement technique, called

Shock Filter. It had analogy with shock wave

when n is the additive noise. calculations in computational fluid mechanics.

We, of course, wish to reconstruct u from u 0. The formation of discontinuities without oscilla-

Most conventional variational methods involve a tions and relevance of the TV norm was ex-

least squares L 2 fit because this leads to linear plored here.

equations. The first attempt along these lines was In a paper written by the first two authors [7],

made by Phillips [1] and later refined by Twomey the concept of total variation preserving en-

[2,3] in the one-dimensional case. In our two- hancement was further developed. Finite differ-

dimensional continuous framework their con- ence schemes were developed there which were

strained minimization problem is used to enhance mildly blurred images signifi-

cantly while preserving the variation of the origi-

minimize f (Uxx + Uyy) 2 (2.2a) nal image.

Additionally, in [8], Alvarez, Lions and Morel

devised an interesting stable image restoration

subject to constraints involving the mean algorithm based on mean curvature motion, see

also ref. [9]. The mean curvature is just the

f u= f Uo (2.2b) Euler-Lagrange derivative of the variation.

We therefore state that the space of BV func-

and standard deviation tions is the proper class for many basic image

processing tasks.

Thus, our constrained minimization problem

f(u - u0) 2 = tr z . (2.2c)

is:

**The resulting linear system is now easy to solve minimize ~ x 2 + Uy2dx dy (2.3a)
**

using modern numerical linear algebra. How- a

ever, the results are again disappointing (but subject to constraints involving u 0.

L.I. Rudin et al. / Noise removal algorithms 261

**In our work so far we have taken the same two As t increases, we approach a denoised version
**

constraints as above: of our image.

We must compute A(t). We merely multiply

f u dx dy = f u o dx d y . (2.3b) (2.5a) by (u - u0) and integrate by parts over 12.

n 12

If steady state has been reached, the left side of

(2.5a) vanishes. We then have

This constraint signifies the fact that the white

noise n(x, y) in (2.1) is of zero mean and

A- 2o.2 + Uy

f

0

l ( u _ u0) 2 d x d y = Or2 where o" > 0 is given

( (u°)xux

2 2 q'-

(u°)ruY

2

~] dx d y .

2,/3

(2.6)

(2.3c)

**The second constraint uses a priori information This gives us a dynamic value A(t), which
**

that the standard deviation of the noise n(x, y) is appears to converge as t---~oo. The theoretical

or. justification for this approach comes from the

Thus we have one linear and one nonlinear fact that it is merely the gradient-projection

constraint. The method is totally general as re- method of Rosen [14].

gards number and shape of constraints. We again remark that (2.5a) with A = 0 and

We arrive at the Euler-Lagrange equations right part multiplied by [Vu I was used in ref. [8]

as a model for smoothing and edge detection.

Following ref. [9] we note that this equation

moves each level curve of u normal to itself with

normal velocity equal to the curvature of the

- A1 - A2(u - Uo) in 12, with (2.4a)

level surface divided by the magnitude of the

Ou gradient of u. Our additional constraints are

On 0 on the boundary of 12 = 012. (2.4b) needed to prevent distortion and to obtain a

nontrivial steady state.

The solution procedure uses a parabolic equa- We remark that Geman and Reynolds, in a

tion with time as an evolution parameter, or very interesting paper [10], proposed minimizing

equivalently, the gradient descent method. This various nonlinear functionals of the form

means that we solve

uy f q ~ ( ~ x2 + u 2) dx dy

12

**-A(u-u0) , fort>O,x, yE/2, (2.5a) with constraints. Their optimization is based on
**

simulated annealing, which is a computationally

u(x, y,O) given, (2.5b) slow procedure used to find the global minimum.

We, by contrast, seek a fast PDE solver that

On computes a " g o o d " local minimum of the TV

On 0 on a12. (2.5c) functional. There is reason to believe that the

local extrema approach is more relevant to this

Note, that we have dropped the first constraint image processing task.

(2.3b) because it is automatically enforced by Finally, we note that we originally introduced

our evolution procedure (2.5a-c) if the mean of this method in two confidential contract reports

u(x, y, 0) is the same as that of u0(x, y). [11,12].

262 L.I. Rudin et al. / Noise removal algorithms

**T h e n u m e r i c a l m e t h o d in two spatial dimen- Here
**

sions is as follows. We let:

A X u i j = "T-(UiZ. 1, j -- Uij ) (2.9a)

x~=ih, y/=jh, i,j=O, 1,...,N, with

and similarly for AYuq.

Nh = 1, (2.7a)

re(a, b) = m i n m o d ( a , b)

t/1=nAt, n=0,1,..., (2.7b)

**/1 _ ( sgn a + sgn b)min(lal, Ibl) (2.9b)
**

uij = u(xi, y~, t , ) , (2.7c)

**and A/1 is defined discreetly via
**

u oo = Uo(ih, jh) + crq~(ih, j h ) . (2.7d)

= - - - +

T h e modified initial data are chosen so that 20.2 .. (A+uq)

the constraints are both satisfied initially, i.e. q~

has m e a n zero and L 2 n o r m one. (A+uq)(A+u,j)

x 0 x /1

T h e numerical approximation to (2.5), (2.6) is x n 2 n 2

V(A + uij) ~- (AY+ u i j )

n+l n

UO = Uij y 0 y n ~]

_ (A+uq)(A+kuq) )J (2.9c)

V(a+u.) + (a÷u,;)

x n 2 y n 2 "

+--£ ,A +uq)

. , 2 + (m(A+uq,

. Ay-un))2)l/2ij/::

A step size restriction is i m p o s e d for stability:

A+ uq

+ A y_ y ..... u n Axun))2) 1/2 At

(A+uq + ~,m~a+ q, __ 0,, , h-~ ~< c . (2.9d)

- At A"(u~ - Uo(ih, j h ) ) , (2.8a)

for i, j = l , . . . , N , 3. Results

**with b o u n d a r y conditions We have run o u r two-dimensional denoising
**

n n n n /1 n n

UOj ~ Ulj , UNj ~ i,I N _ I , j ) Uio ~ UiN ~- Ui,N-1 " a l g o r i t h m on graphs and real images.

(2.8b) T h e graphs and images displayed take on in-

(a) Signal

!

**Fig. 1. (a) "Bars". (b) Plot of (a). (e) Plot of noisy "bars", SNR = 1.0. (d) Noisy "bars", SNR = 1.0. (e) Plot of the
**

reconstruction from (d). (e) TV reconstruction from (d). (g) Plot of the reconstruction error.

264 L.I. Rudin et al. / Noise removal algorithms

Noisy Signal

Recovered Signal

(c

Error

(e)

**Fig. 2. (a) Plot of fig. la plus noise, S N R = 0.5. (b) Noisy fig. la, SNR = 0.5. (c) Plot of the reconstruction from (b). TV
**

reconstruction from (b). (e) Plot of the reconstruction error.

L.I. Rudin et al. I Noise removal algorithms 265

**teger values from 0 to 255. When Gaussian white SNR = Z a ( u u - t~)2
**

noise is added the resulting values generally lie Zn(nu) 2 , (3.1)

outside this range. For display purposes only we

threshold; however, the processing takes place

on a function whose values generally lie arbit- where ti is the mean of the signal u u and n o is the

rarily far outside the original range. noise.

Signal to noise ratio (SNR) is defined by: Figs. 1 and 2 concern three parallel steps taken

0 I

--III

2~

m

'

3,,,,-

m

III fll :.t

Ill

tlI =

-- :~

'~

I II ~ .~:;

4 - - - I!! °:!:ill Itl E :

II1~~ ~

s = iii .__0

III " " I

**Fig. 3. (a) "Resolution Chart". (b) Noisy "Resolution Chart", SNR = 1.0. (c) Wiener filter reconstruction from (b). (d) TV
**

reconstruction from (b).

266 L.I. Rudin et al. / Noise removal algorithms

from fig. 3a. This is 38 by 38 pixels wide 256 gray sharp signal and fig. If shows the recovered

levels black and white original image. Fig. la signal. Finally, fig. lg shows the error which is

shows the original signal. Fig. lb shows its inten- fairly "hollow". It is zero both within the origi-

sity plot. Fig. lc shows the intensity of the noisy nal steps and also beyond a few pixels outside of

signal with additive Gaussian white noise, signal them. Fig. 2a shows the intensity plot of a noisy

to noise ratio SNR 1. Fig. ld shows the noisy signal when SNR = 1, twice as much Gaussian

signal. Fig. le shows a graph of the recovered white noise as signal. Fig. 2b shows the noisy

**Fig. 4. (a) "Airplane". (b) Noisy "Airplane", SNR = 1.0. (e) Wiener filter reconstruction from (b). (d) TV reconstruction from
**

(b).

L . L Rudin et al. / Noise removal algorithms 267

image. Fig. 2c shows the intensity plot of the dard black and white images taken from the

recovered signal and fig. 2d shows the recovered USC IPI image data base. Fig. 3a shows the

image. Finally, fig. 2e shows the almost "hollow" original 256 x 256 pixels resolution chart. Fig. 3b

error. shows the result of adding Gaussian white noise,

It appears that our denoising procedure beats SNR 1. Fig. 3c shows the result of our denoising

the capability of the human eye - see figs. lb, 2b algorithm. Finally fig. 3d shows the result of

and 2c. using a Weiner filter where the power spectrum

The remaining figures are 256 gray level stan- was estimated from fig. 3b. Notice fig. 3d has a

**Fig. 5. (a) "Tank". (b) Noisy "Tank", SNR = 1.0. (c) Wiener filter reconstruction from (b). (d) TV reconstruction from (b),
**

268 L.I. Rudin et al. / Noise removal algorithms

**lot of background noise which makes it prob- [3] S. Twomey, On the numerical solution of Fredholm
**

lematic for automatic processing. Fig. 4a shows a integral equations of the first kind by the inversion of

the linear system produced by quadrature, J. ACM 10

256 × 256 airplane in the desert (clean image). (1963) 97.

Fig. 4b shows the result of adding Gaussian [4] S. Twomey, The application of numerical filtering to the

white noise - SNR 1. Fig. 4c shows the result of solution of integral equations encountered in indirect

sensing measurements, J. Franklin Inst. 297 (1965) 95.

a denoising via our algorithm. Fig. 4d shows the

[5] B.R. Hunt, The application of constrained least squares

result of a Weiner filter denoising with the true estimation to image restoration by digital computer,

spectrum estimated from the noisy image, via a IEEE Trans. Comput. 22 (1973) 805.

moving average. Fig. 5a shows the original [6] L. Rudin, Images, numerical analysis of singularities

and shock filters, Caltech, C.S. Dept. Report #TR:

512 × 512 picture of a tank. Fig. 5b shows the 5250:87 (1987).

result of adding Gaussian white noise SNR 4. [7] S. Osher and L.I. Rudin, Feature oriented image en-

Fig. 5c shows a Weiner filter denoising with hancement using shock filters, SIAM J. Num. Anal. 27

(1990) 919.

spectrum estimates from fig. 5b. Fig. 5d shows [8] L. Alvarez, P.L. Lions and J.M. Morel, Image selective

our algorithm applied to the same window. smoothing and edge detection by nonlinear diffusion,

Notice that the discontinuities are much clearer SIAM J. Num. Anal. 29 (1992) 845.

[9] S. Osher and J. Sethian, Fronts propagating with

in the last case. Also Wiener restoration has

curvature dependent speed: Algorithms based on a

oscillatory artifacts. Hamilton-Jacobi formulation, J. Comput. Phys. 79

Our recent experiments indicate that the use (1985) 12.

of more constraints (information about the noise [10] D. Geman and G. Reynolds, Constrained restoration

and the recovery of discontinuities, preprint (1990).

and the image) in this method will yield more [11] E. Fatemi, S. Osher and L.I. Rudin, Removing noise

details of the solution in our denoising pro- without excessive blurring, Cognitech Report #5, (12/

cedure. 89), delivered to DARPA US Army Missile Command

under contract #DAAH01-89-C-0768.

[12] L.I. Rudin and S. Osher, Reconstruction and enhance-

ment of signals using non-linear non-oscillatory varia-

tional methods, Cognitech Report #7 (3/90), delivered

References to DARPA US Army Missile Command under contract

#DAAH01-89-C-0768.

[13] Y. Dodge, Statistical data analysis based on the Lj norm

[1] B.R. Frieden, Restoring with maximum likelihood and and related methods (North-Holland, Amsterdam,

maximum entropy, J. Opt. Soc. Am. 62 (1972) 511. 1987).

[2] D.L. Phillips, A technique for the numerical solution of [14] J.G. Rosen, The gradient projection method for non-

certain integral equations of the first kind, J. ACM 9 linear programming, Part II, nonlinear constraints, J.

(1962) 84. Soc. Indust. Appl. Math. 9 (1961) 514.

- Analysis of Process Parameters for Optimization of Plastic Extrusion in Pipe ManufacturingUploaded byAnonymous 7VPPkWS8O
- 1-s2.0-S1877050915010066-mainUploaded bysainiashish57
- cofvUploaded byFuat Yıldız
- guided_filter_eccv10.pdfUploaded byasdfgh
- Evaluation of Neutrosophic Set Approach Filtering Technique for Image DenoisingUploaded byIJMAJournal
- A New Approach to Remove Gaussian Noise in Digital Images by Immerkaer Fast MethodUploaded byGeerthikDotS
- A Method for Blind Deblurring of Natural ImagesUploaded byIJSTE
- Noise Variance Estimation for Spectrum Sensing in Cognitive Radio Networks 2014 AASRI ProcediaUploaded byAnonymous VIHgLt5
- VarCalcUploaded byVigneshwar Narayanan
- Calculus of Variation NotesUploaded byJoel Macaluso
- syllabus2009.pdfUploaded byHariharan
- class2Uploaded byKhanh Nam
- Planet Ethernet over VDSL2 Bridge.pdfUploaded byDynDNS
- 14659.pdfUploaded byJorge Luis Garcia Zuñiga
- Ordinary Differential EquationUploaded byNitish Boodhram
- Surface Evolver ManualUploaded bySunny Sun
- non linear programmingUploaded byapi-3774614
- Compressed Sensing Data 3Uploaded byLe Ly Tuong
- An Application of the ADM to Seven-Order DKAYAUploaded by3873263
- shannon capacity basicsUploaded byFahadKhan
- L8 Feedback Amp CBY2016Apr5 (5)Uploaded byLuqmanMaraaba
- HandbookUploaded byganeshlaveti
- AGPS and XTRA GPS _ Code Aurora ForumUploaded byVikthor Lopez

- 09.04.Detection of Missing Data in Image SequencesUploaded byAlessio
- 09.05.Interpolation of Missing Data in Image SequencesUploaded byAlessio
- 06.12.Image Stabilization by Features TrackingUploaded byAlessio
- 09.13.A new image processing primitive_reconstructing images from modiﬁed ﬂow ﬁeldsUploaded byAlessio
- 06.10.Digital Image Stabilizing Algorithms Based on Bit-plane MatchingUploaded byAlessio
- 08.14.Jetstream Probabilistic Contour Extraction With ParticlesUploaded byAlessio
- 09.24.Filling-in by joint interpolation of vector ﬁelds and grey levelsUploaded byAlessio
- 01.20.Texture Synthesis by Non-parametric SamplingUploaded byAlessio
- 09.12.Level-Lines Based Dis OcclusionUploaded byAlessio
- Image Motion DeblurringUploaded byAlessio
- 09.09.Pyramid Based Texture Analysis_synthesisUploaded byAlessio

- BTech Civil III IV Semester 0Uploaded byStuart ashraf
- FMVUploaded byIoana Acsente
- Elementos Finitos Aplicados a IngenieriaUploaded byGuillermo Zubieta reyes
- BTech Production curriculum and syllabus 22-07-14(Frm 2013-14) (1).pdfUploaded bySabhari Natarajan
- Banco de Preguntas Prueba 1Uploaded byAndrés Guerra
- EXPRESIONES ALGEBRAICASUploaded byDanie Romani
- HojaTrab10_EDOSuperior(x1)Uploaded byjhonatan
- Analisis Numerico Basico Funciones en MatlabUploaded byjbzamora
- Mathematik- Lineare Algebra- Lineare Gleichungssysteme- MatrizenrechnungUploaded by유문희
- CSE-2013 BTech Syllabus NewUploaded bymaruthi631
- B.E.AERO-SYLLABUS-III semesterUploaded byranand sankar
- Divided Difference ExamplesUploaded bySYifa ShuhaIli
- programacion-no-lineal.docxUploaded byAdalberto Hernandez
- practica uno métodos numéricos esime cuUploaded byLeon Mgmt
- Equae Sal g Bri Case Trance DentesUploaded byHebson Bezerra
- Lineare Optimierung im Mathematikunterricht_by Hamacher & MüllerUploaded byarchonV
- EDP pdfUploaded byAndré Cuadros Mamani
- 04 y 05 2016-04-20 y 27 Metodo Simplex - Solución AlgebraicaUploaded byCarlos Miguel Polo Alza
- UT Dallas Syllabus for phys3330.501.09f taught by Roy Chaney (chaneyr)Uploaded byUT Dallas Provost's Technology Group
- calificacion 33Uploaded bynelson
- Determination of Springback in Sheet Metal FormingUploaded byThiru Moorthy
- PRACTICA1_2016Uploaded byMarco Antonio Flores
- Examples for Chemical Engineering Thermodynamics Using MatlabUploaded bymbolantenaina
- Factorización LUUploaded byAguilera Je
- 809-Lista de Exercícios - Sistemas Lineares.originalUploaded byDavid Almeida Matos
- Cálculo Numérico - RUGGIEROUploaded byJ Sophia Cris
- Chapter2 Rectangular systems and Echclon Forms.pdfUploaded byAngel Leandro
- Aerofólio - Análise Do Escoamento Elementos FinitosUploaded byluizotaviocfg
- Eigenvalues Exercises and SolutionsUploaded bykaliman2010
- Regla Simpson 1/3 C++Uploaded bylachv346@hotmail.com