Characterization of Signals from Multiscale Edges ‘Stephane Mallat and Abstract—A multiscale Canny edge detection is equivalent to finding the local maxima of a wavelet transform. We study the properties of muliscale edges through the wavelet theory. For pattern recognition, ome often needs to discriminate different {yes of edges. We show that the evolution of wavelet local ‘maxima across scales characterize the local shape of iregslat sractare. Numerical descriptor of edge types are derived. The ‘We describe an algorithm tat reconstructs clove approx of LD and 2-D signals from thelr muldseale edges For images, the reconstruction errors are below our visial sensitivity. Ar a application, we implement a compact image coding algorithm that Selects important edges and compresses the image data by factors over Index Terms— Edge detection, feature extraction, level cross: Ings, multiscale wavelets 1. INtRopuerion JOINTS OF SHARP variations are often among the most important features for analyzing the properties of transient ‘signals or images. n images, they ae generally located at the boundaries of important image structures. In order to detect the contours of small structures as well as the boundaries of larger objects, several researchers in computer vision have introduced the concept of mulisale edge detection [18], (23], [25}, The scale defines the size of the neighborhood where the signal changes are computed. The wavelet tansform is closely related to multiscale edge detection and can provide a deeper understanding of these algorithms. We concentrate on the Canny edge detector [2], which is equivalent to finding the local maxima of @ wavelet transform modulus. ‘There are many different types of sharp variation points in images. Edges created by occlusions, shadows, highlighs, roofs, textures, ec. have very different focal intensity profiles, ‘To label more precisely an edge that has been detected, i is necessary to analyze its local properties. In mathematics, singularities ate generally characterized by their Lipschite exponents. The wavelet theory proves that these Lipschitz ‘exponents can be computed from the evolution across scales of ‘the wavelet transform modulus maxima. We derive a numerical procedure to measure these exponents. If an edge is smooth, ‘we can also estimate how smoath its fom the decay of the Mana rective Jamary 101960 evel assey 23,1992 Tis sno was tpg bythe NSE gum TRL-SOUSS, AFOSR pant AFOSR ‘hot, snd ONK rat NODIS 11967. Reomnnde for steps by Asai Er R Woohan atts wih the Coun Testis, New York Unies, New Yor, Ny 5: Zhong was with th Curae ini, Now York Universi, New Yk NY 12: He's tow wih be Deprient af Mathematics of Calin. Lis Angee. Ca TERE Log Nar 930010 Sifen Zhong wavelet transform ms ross scales, Lipschitz exponents and smoothing factors are numerical descriptors that allow ws to discriminate the intensity profiles of diferent types of edges. An important open problem in computer vision i to under. stand how much iaformation is caried by multiscale edges and how stable a multiscale edge representation is. This issue is important in pattern recognition, where one needs to know ‘whether Some interesting information is lost when represent 4 pattem with edges. We study the reconstruction of I-D and 2D signals from multiscale edges detected by the wavelet transform modulus maxima, It has been conjectured [16], {18} that multiscale edges characterize uniquely 1-D and 2-D signals, but recently, Meyer [21] has found counterexamples to these conjectures. In spite ofthese counterexamples, we show that one can reconstruct a close approximation ofthe original signal from multiscale edges. The reconstruction algorithm is based on alternate projections. We prove its convergence and derive a lower bound forthe convergence rate. Numerical ‘esulls are given both for I-D and 2-D signals, The differences between the original and reconstructed images are not visible na high-quality video monitor. ‘The ability 10 reconstuet images from multiscale edges hhas many applications in signal processing. It allows us to process the image information with edge-based algorithms. ‘We describe a compact image coding algorithm that keeps conly the “important” edges. The image that is recovered from these main features has Tost some small details but i visually fof good quality. Examples with compression ratio over 30 are shown, Another application to the removal of noises from signals is desribed in [17} The article is organized as follows. Section Il relates mul tiscale edge detection to the wavelet transform. It shows that a Canny edge detector is equivalent to finding the local maxima of a wavelet transform modulus. Until Section VI, wwe concentrate on I-D signals. Section TIA reviews the wavelet transform properties that are important for under- standing. multiscale edges. The wavelet tansform is. fst defined over functions of continuous variables, and Section LB explains how to discretize this model, The numerical implementation of fast wavelet transform algorithms is given in Appendix B. Section IV explains how to characterize diferent types of sharp signal variations from the evolution across scales of the wavelet transform maxima. Section V studies the reconstruction of signals from muliscale edges. We review some previous results and explain how to formalize the recoastruction problem within the wavelet framework ‘The reconstruction algorithm is described in Section V-1 and numerical results are presented in Section V-C. A 2-D extension of the wavelet transform is given in Section VIA,and its discrete version is explained in Section VIEB. Fast 2- D wavelet algorithms are given in Appendix D. Section VIL Aiferentiates the edges ofan image from the evolution across sales of the wavelet modulus maxima. The reconstrction of images from multiscale edges is explained in Section VIIA, and numerical examples are shown in Section VIIL-B. Section 1X describes an application to compact image coding. "Notation: L3(R) denotes the Hiller space of measurable, sauare-integrable 1-D functions f(x). For f © °(R) and 9 € L(R), the inner product of f(2) with g(x) is writen: (ote) see =f oteystarte ‘The norm of f(z) € L7(R) is given by [2 weortas wr We denote the convolution of two functions f(x) € 1°(R) and g(z) € 13(R) by peatey= [> soagle—nte ‘The Fourier transform of f(r) € L#(R) is writen f(w) and is defined by ftay= [> slayer (R) is the Hilbert space of measurable, square-integrable 2D functions f(z. )- The norm of f(y) € L*(R) is given iy Lo [E veenytaeas. ‘The Fourier tansform of fry) € LARP) is writen flseiwy) apd is defined by ia i ‘Feyenoord, 1, MULTISCALE EDGE DeTECTION Most multiscale edge detectors smooth the signal at various scales and detect sharp variation points from their first ot second-order derivative, The extrema of the frst derivative correspond to the zero crossings ofthe second derivative and to the inflection points of the smoothed signal. This section ‘explains how these multiscale edge detection algorithms are felated to the wavelet transform, We call a smoothing function any function fr) whose integral is equal to 1 and that converges to O at infinity, For ‘example, one can choose (2) equal toa Gaussian. We suppose that (2) is twice differentiable and define, respectively, #"(x) and W"(2) as the first- and seeond-order derivative of 1) wr Sees) o By definition, the functions »"(2) and y¥(r) can be consid- ered to be wavelets because their integral is equal 10 0 [oO etoxe ome f°" vous =o In his paper we dente et ate) = 12) nich te itaon by sang factor any fenton [wavelet rancor f computed y convolving the Sg Sib dated wavlt The wavelet tansorm of f(s) a he scale » and postion x, computed with respect to the wavelet "C2, is defined by Wy fla) = f+ 32). @ “The wavelet transform of J(e) with respect to yx) is Wesle) =F evtla) @ We derive that Wry) =f « West) 6) The wavelet transforms W:f(2) and W2/(x) are, respec: tively the first and second derivative of the signal smoothed atthe scale s. The local extrema of 1V2 f(z) thus correspond to the zero crossings of 173 f(z) and tothe inflection points of if 8s). In the particular case where (2) is a Gavssian, the ‘zoro-crossng detection is equivalent to a Mare-ildeth (19) fedge detection, whereas the extrema detection corresponds 10 a Canny [2] edge detection. When the scale is large, the convolution with @(r) removes small signal fluctuations; we therefore only detect the sharp variations of large structures. Detecting zero crossings or local extrema are similar pro: cedure, but the focal exirema approach has some imporiant advantages. An inflection point of f + @,(z) can either be ‘2 maximum of a minimum of the absolute value of is fist Gerivative, ‘The maxima of the absolute value of the first Gerivaive are sharp variation points of f = A(x), whereas the minima correspond to slow variations. With second derivative operator, its dificult to distinguish these two types fof veto crossings. On the contrary, with a first-order derivative, ‘we easily selet the sharp variation poinis by detecting only the local maxima of |W! (In addition, zero crossings give position information but do not differentiate small amplitude Fluctuations from important discontinuities. When detecting focal maxima, we can also record the values of W f(x) at the maxima locations, which measure the derivative at the inflection points. Section IV explains how to characterize diferent types of sharp variation points from the evolution cross scales of W2 f(r) atthe modulus maxima locations. "The Canny edge detector is easily extended in two dimen- sions, We denote by Heh riley)the dilation by s of any 2-D function €(z,y)- We use the term 2D smoothing function to describe any function 7, u) whose integral over x and y is equal to 1 and converges to 0 at infinity. The image f(x,y) is smoothed at diferent scales ¢ by a convolution with ,( 1). We then compute the gradient vector V(f+8,)( x,y). The direction of the gradient vector at & Point (ro; yo) indicates the direction in the image plane (7) 1 which the directional derivative of f(y) has the largest absolute value. Edges are defined as points (zo,yo) where the ‘modulus of the gradient vector is maximum in the direction towards which the gradient vector points in the image plan. ge points are inflection points ofthe surface f f(y). Let us relate this edge detection to a 2-D wavelet transform. We define wo wavelet functions Y*(z, y) and Y(zy) such that (2.9) = MED na y2(0,y) = MED.) Let ew) = 2v8(E, 2) and YBlew) = bvACE 2). Let {fle,y) © LAURE), The wavelet ransform of f(z) a the seals has to components defined by WES (eu) = Fovis(z.u) and W3 flesu) = frvstesy). Similarly to (4), one can easily prove that Wiley) HU OMe) _ (aes) (Hert oh) 80 00 2 Hence, edge points can be located from the two components W3JCe,y) and W2 f(x,y) of the wavelet transform. TIL, DYADIC WAVELETTRANSFORM IN ONE DIMENSION A. General Properties For most purposes, the wavelet model is not required to keep a continuous scale parameter s. To allow fast numerical implementations, we impose that the scale varies only slong the dyadic sequence (2). We review the main properties of a dyadic wavelet transform and explain under what con- dition itis complete and stable. For thorough presentations fof the wavelet transform, refer to the mathematical books of Meyer [20] and Daubechies [5] or t signal processing oriented reviews (15], [22]. The wavelet model has first been formalized by Grossmann and Morlet [10]. A wavelet is 2 function y(r) whose average is zero. We denote by ¥2i(z) the dilation of y(z) by a factor 27 vole) = 3505) ‘The wavelet transform of (2) at the scale 2” and at the position x is defined by the convolution product Way fle) = f+ Vaile) o We refer to the dyadic wavelet transform as the sequence of functions WI = We f),ec (0) and W is the dyadic wavelet transform opertor. Let us study the completeness and stability of a dyadic ‘wavelet transform. The Fourier transform of W f() is Wa fle) = Hlwyi@e). any ‘By imposing that thee exists two strictly positive constants Ay and By such that WweERAS 5 Wie)? < By «ay vegas that he whole frequency axis is coveted by dilations of sa) by 2) jeg 0 that fla) and thas, f() can bere covered from its adie wavelet ansform, The reconsevting wavelet (7) is any function whose Fourier transform satisfies ¥ deen «sy property (12) s valid, there exists an infinite numberof func- tions (2) that satisfy (13). The function f(2) is recovered from its dyadic wavelet transform with the summation Stz)= Yo Warf «xara. aay ‘This equation is proved by computing its Fourier transform and inseing (11) and (13). With the Paseval theorem, we ‘erve from (11) and (12) a norm equivalence elation ALIS SSO We f ai < Ballsle as) ‘This proves thatthe dyadic wavelet transform is aot only complete but stable as wel. If 8 is closer to 1, it will be more stable. ‘A dyadic wavelet transform is more than complet; it is redundant. Any sequence (95(1)) cz, with 9s(«) € L2(R), is not necessarily the dyadic wavelet transform of some function in L4(2), We denote by W~" the operator defined by WaileV)jex= Do ye xail2). a6) “The reconstruction formula (14) shows that (5 (2)jez) i the dyadic wavelet transform of some function in L*(R), if and only if WWal2),ez) Hier a we replace the operators W and W by their expression sven in (9) and (16), we obtain Vie Z YY a Kisl2) = 9j(2)s with Kis(2) = x0 «Vala as) ay) ‘These equations are known as reproducing kernel equations “The energy of the kernel K(x) measures the redundancy of the wavelet transform atthe scales 2/ and 2!Fig. 1. () This wavelet 2 ganda apline of comput Suppor ths ontinwaly aierentiahe. ie dened Appx andthe deratve ‘be eae spine neon Hs) shown Fig. 1(a) is a quadratie spline wavelet of compact suppor, “whieh is Further defined in Appendix A. Its the derivative of the smoothing function #(z) shown in Fig. (6). Fig. 2(a) is the plot of a disret signal of 256 samples. Fig. 2(b) shows its dlisrete dyadic wavelet transform computed on nine seals. At cach scale 2’, we compute a uniform sampling of the wavelet transform that we denote Wf. The next section explains how to discretize the continuous wavelet model and solve border problems. Fast algorithms to compute the wavelet and the inverse wavelet transform are described in Appendix B. ‘The reader not interested in numerical issues might want (0 jp Section II-B. Since our wavelet isthe derivative of & tmoothing function, (4) proves that W'S fis proportional 10 the derivative ofthe original signal smoothed atthe scale 2! Fig, 2) gives the locations and values of the local maxima of the dyadic wavelet wansform modulus, as in a Canny fedge detection, At each scale 2/, each modulus maximum is represented by 2 Dirac that has the same location and whose amplitude is equal to the value of Wp, f(x). The modulus maxims detection isan adaptive sampling that finds the signal sharp variation points. B. Diserete Wavelet Transform a numerical applications, the input signal is measured at finite resolution; therefore, we cannot compate the wavelet transform at an arbitrary ine scale. Let us normalize the finest seal to 1 In order to model this scale limitation, we introduce {real function d(2) whose Fourier transform isan aggregation fof yi(2%a) and §(2%) at scales 2? larger than 1 Hw)P = S7 (up x2e). a0 ‘We suppose here that the reconstructing wavelet (2) is such that G2) (2) isa postive, real, even funetion. One ean prove that property (13) implies thatthe integral of g(x) is equal to 1 and, hence, that it i a smoothing function. Let Sy be the smoothing operator defined by Safle) = fe dn(2) wit do(2)= EAS) QD Ifthe sale 2 is age, the more dts of f() ane removed by Spy. For any seale 2! > 1, (20) yields ° ig 2 (a Sigal of 256 simple: cre dade wave aso of Signa) compucdon sine nloe AT exes, lo the ial 1 J Wich Sho fav 386 tmp f0) moduler maxima ofthe dae wate orto showin (@) Each Da sents the poston ae ample of ‘oso mn [day aay] = 52 6p), cc (ne can derive from this equation thatthe hake frequencies OF Sifleh which have disappeared in Sy f(z), can be re ered from the dyadic wavelet tansform (a /@V csc between the scales 24 and 27, ‘We supose tha the orginal signal discrete sequenee D = (da)res of finite ney. there ex 0 constants ©, > 0 and Cy > 0 such that Ge) sates WweR OS F iwermmesce @then one can prove prove [18] that there exists @ function Sle) € L(A) (aot unique) such that Yue Z, Sif(n) ‘The input signal can thus be rewrten D = (Sif(W))yeg Fora paiculr class of wavelets defined in Appendix Athe discrete signal D = (S1f(n)yez allows us to compute a Uniform sampling of the wavelet ‘ransform of (2) a1 any Seale larger than 1. Let us denote WES le ew Wa f(r + nex and $3 f = (Su fln+ neg 8 where w is a sampling shif that depends only on w(x). For ‘any coarse scale 2, the sequence of discrete signals. {Sh5.E sss} is called the discrete dyadic wavelet transform of D = (5:F17))yeg- The coatse signal $4, provides the signal com- Ponenis below the scale 2/. A fast discrete wavelet transform algorithm and its inverse ate described in Appendix B. In practice, the original discrete signal D has a finite umber N of nonzero values: D = (dy),cycy To sObve the order problems, we use the same priadizion technique 2 in a cosine transform. We suppose that our signal has a period of 2V samples, and we extend it with a symmetry for N O such that for all's eja,0, the wavelet transform sates [Was slo)] < KY 8) ‘The proof ofthis theorem can be found in (20), From (28), we derive that Nog [Wa f(z)] < loga( KC) + 03. @) If the uniform Lipschitz regularity is positive, (28) implies that the amplitude of the wavelet transform modulus maxima should decrease when the scale decreases. On the contrary, the singularity atthe abscissa 3 of Fig. 3(0) produces wavelettransform maxima that increase when the scale decreases. Such singularities can be described with negative Lipschitz exponents, which means that they are more singular than iscontinvities. The signal is then viewed as a tempered ‘bation. At the abscissa 3 of Fig. 3(b), this distribution ‘equal to a Dirac. The reader might want to consult Folland 7] for a quick presentation of the mathematical theory of distributions. The wavelet transform of tempered distributions is well defined if the wavelet (2) is smooth enough (17]. For example, if x) is continuous, the wavelet transform of a Dirac 6(2) is given by Wy Slz) = 5+ vas(2) = dale) G0) ‘To extend Lipschitz exponents to distributions, we say tha 4 distribution has @ uniform Lipschitz regularity equal 10 a ‘on ja [if and only if its primitive has a uniform Lipschitz regularity equal to a + Lon Ja, 6, For example, the primitive of a Dirge centered at zo is a function that is bounded and has a discontinuity at xy (step edge). The uniform Lipschitz regularity of the primitive of this Dirac is thus equal to 0 in the neighborhood of ro. Hence, a Dirac centered at zo has @ ‘uniformly Lipschitz reglarity equal to —1 inthe neighborhood ‘of 2p. One can prove that Theorem I is also valid for negative Lipschitz exponents. Let a9 < 1 be areal number that may be negative. A tempered distribution f(r) hasa uniform Lipschitz regularity equal to ag over Ja,6[ if and only if for any «<9 there exists AC such that [Was fells K@ ey Since the Lipschitz regularity of a Dirac is —1, this result implies thatthe maxima values of |W, 4()] increase propor: tionally to the scale 2/. This can indeed be verfed in Fig 300), In practice, we can only process discrete signals that ap- proximate the original function at a finite resolution, which ‘we normalize to 1. Strictly speaking, itis not meaningful to speak about singularities, discontinuities, or Dirac. n fact, we cannot compute the wavelet transform a scales finer than 1 and thus cannot verify (31) at scales smaller than 1. Even though wwe ae limited by the resolution of measurements, we can still use the mathematical ools that differentiate singularities. ‘Suppose thatthe approximation of f(r) atthe resoltion 1 is sgiven by a set of samples (fn)yez+ with fy = 0 for n < no and fj, = 1 for n > np, lke at abscissa 2 of Fig. 32). [At resolution 1, f(x) behaves a8 iF 1 has @ discontinuity at 1 = no, although f(z) might be continuous at no with a continuous sharp transition at that point, which is not visi at this resolution. The characterization of singularities from the decay of the wavelet transform gives a precise meaning 10 this “discontinuity atthe resolution 1.” We measure the decay of the wavelet transform up to the finer scale available, and the Lipschitz regulary is computed by finding the coeficient ‘4g such that K(2)° approximates at best the decay of [Wa f(e)] over @ given range of scales larger than 1. In Fig. 3) in the neighborhood of = 2, the maxima values of IVs f(z) remain constant over large range of scales Equation (31) implies that the Lipschitz regularity ys equal 4 0 at that point, which means that this singularity is & discontinuity. In the edge detection procedure: described in Section Il, we only Keep the local maxima of the wavelet transform moduls. It has been proved [17] tht if a signal is singular at a point zo, thee exists a Sequence of wavelet ttansform modulus maxima that converge to D. Hence, a isthe upper ‘bound of the set of a such that there exists K that satisfy Wha) < Ks"for any scale ¢ > and any inthe corresponding neighbor= hood of 9. By inserting this inequaliy in (35), we obtain We flay) SRP! 559%, with sy = VP 6 ‘This equation is satisfied at all points» if and only i itis Satisied a the locations of al the local maxima of |W, f)| I the signal is multiplied by a constant, then Kis a ‘multiplied by A, but and ng are not affected. On the contrary, if the signal is smoothed by a Gaussian of variance 03 (and imegnal 1), then K and cy ate not affected, but 7? becomes ‘This shows clearly that the parameters «, 0. and K describe differen properties of the sharp variation that ‘occurs at. Fig. 3 gives the examples of a step edge and a Dirac smoothed by Gaussians of different variances, The decay of the maxims are cleatly affected by the diferent Lipschite exponents as well asthe variance of the Gaussian smoothing. Let us explain how to compute numerically the Lipschitz regularity ao and the smoothing scale = from the evolution ff the wavelet transform modulus maxims across scles. I we deteet the modulus maxima at all scales s, instead of jst atic seales 2, their positon would define 2 smooth curve in the sealespace plane (s.r, These curves have been called “finger prints” by Witkin [25]. We say that « modulus maxima atthe seale 2 propagates to 4 maxims at the coarser scale 2/41 iF and only if both maxima belong the same maxima ceurve in the scale-space plane (s.r). In Fig. 3, there is one sequence of maxima that belongs to the same maxima curve and converges to the position ofthe giscontinuit at = 2. For the Dirac at abscissa 3, there are two such sequences, Each one gives information, respectively, on the lel and the right part of| the Dirae singularity. In order to find which maxima propagate to the next scale, one should compute the wavelet transform fon a dense sequence of scales. However, with a simple ad- hoc algorithm, one can sill estimate which maxima propagate te the next scale by looking atthe value and postion with respect to other maxima at the next scale, The propagation algorithm supposes that a modulus maximum propagates fom a scale 2! 10 a coarser seale 2! +1 i it has a large amplitude And if its position is close 19 a maximum atthe scale 2+ 1 ‘that has the same sign. This algorih is nat exact but saves computations since we do not need to compute the wavelet ‘ransform at any other scale. The Lipschitz regularity 2s well as the smoothing variance ¢? of a sharp vatation point are then computed fom the evolution of the modulus. maxima that propagate across scales. Let us suppose that we have a ‘sequence of modulus maxima that propagate from the scale 2! up to the scale 2" and converge to the abscissa rp. Let fe the value of the wavelet transform at the maximum location atthe scale 2, and lt ws also suppose tht in 8 given neighborhood of 9, the wavelet transform modulus is smaller than a. This means thatthe signal change at isthe sharpest variation in this neighborhood. We compute the three values KK, 0, and ang so that the inequality of (37) is as close as possible (© an equality for euch maximam n,. These values are obtained by minimizing E (vata ~logs(k) ~ J 8" Noga? 428 ) 8) This is done with steepest gradient descent algorithm. The value AC gives the amplitude of the sharp variation, When computing the values of ¢ and «from the evolution of the maxima across scales in Fig. 3, we hive # numerical error fof less than 105%, which is mainly due to the fact that the wavelet we use is not the derivative of a Gaussian but is only an approximation. In this cae, (1) is the cubic spline shown in Fig. 1(b). When the variance 0? inezeases, the measurement of ag hecomes more unstable because the smoothing removes the fine scale components that characterize reliably «a. For singularities of fractal textures such as inthe eight part of Fi. 2(a), this analysis is not valid because singularities are not Isolated, and none of the singularities dominate the others in ‘given neighborhood. The behavior of the wavelet transform ‘modulus maxima of nonisolated singularities i studied in more detail in {17} V. SIONAL RECONSTRUCTION FROM MULTISCALE EDGES Section IV shows that one can get a precise description of the signal sharp variation points from the evolution of the wavelet transform modulus maxima across scales. An important question is to understand whether the whole signal information is embedded into these modulus maxima, Is it possible o have a stale signal reconstruction only rom the ‘modulus maxima information at the dyadic scales 2? The next section reviews briefly some results on the reconstruction of signals trom zero erossings and multiscale edges. Section V-B Ueserbes an algorithm that reconstructs a close approximation fof the original signal from the wavelet transform modulus ‘maxima. Numerical results are presented in Section V-C. A. Previous Results The reconsetion of signals fom multiscale edges has manly been sued inthe zero-rosing framework, We saw in Section If that if the wavelet is given by "(x)= “#17, tulscale edges are detected from the zero crossings of the Wavelet ransform V(r). The most base result conceraing, the reconstruction of signals from the zero crossing is she Logan theorem (14) However, ais explained in (18) the hypotheses ofthe Logan theorem are no appropriate to sly the reonstucion of signals from multiscale edges. Te Logan theorem has been generalized by several authors (3) [24 [28h Refer toa review by Hummel and Monit (11] for more deals The smoothing function 02) i Gaussian, the propetes ofthe wavelet transform zero tosings are move easly = erstood because Hx) canbe interpreted asthe solution of heat difusion press at time ¢ = +! (12), With this ‘approach, Hummel and Moniot (11, as well as Yuille and Poguio [27], have proved some completeness propertics under restive condition, ike supposing tht f(z) i polynomil